Downloading RTMP Flash media with rtmpdump

EDIT: These instructions are for verson 1.9 of rtmpdump! Newer versions make this process easier but also make some of this how-to obsolete. But I've modified the process below so that you can still use them for the legacy 1.9 version.

Today I attended a lecture/demo session on "hacking" various media sites like YouTube, Hulu, last.fm. Essentially it was a practical demonstration of the cat and mouse game hackers play with content pedlars on the internet. It was held at Cyberpipe, a nifty club in Ljubljana, Slovenia where I study.

One of the tools mentioned was rtmpdump which is apparently made by the same guys as the legendary mplayer, mencoder, and ffmpeg.

I decided to compile it myself from the most recent development version. If you're lazy and brave enough to trust compiled binaries and packages from third parties, go to LinuxCentre and install flvstreamer for your operating system. These programs were once one and so far they use the same commands. Read: they work the same.

These instrucitons work on Ubuntu 9.10 (karmic) but please be aware things change rapidly so the following might not work forever.

First of all install the tools needed for accessing the code and compilation.

sudo apt-get install build-essential gcc make subversion libssl0.9.8 libssl-dev libssl0.9.8

Now check out the latest code from the Subversion repository:

svn co svn://svn.mplayerhq.hu/rtmpdump/branches/1.x rtmpdump-1.9

If you prefer a stable version, run this instead:

wget http://rtmpdump.mplayerhq.hu/rtmpdump-1.9.tgz
tar xfz rtmpdump-1.9.tgz

I'm sure the impatient are looking forward to completing this ASAP so here's the final step in setting up rtmpdump on your computer:

cd rtmpdump-1.9
make linux

The program now lies in the same folder and you can only run it by telling your shell (like bash) exactly where it is. If you're in the same folder, running it is as simple as ./rtmpdump but ./ is only short for the current folder and it gets expanded to (in my case) /home/jasa/rtmpdump/rtmpdump.

Great! Now we need a media site which uses a Adobe's RTMP server. I think I won't get in trouble if I use VideoLectures.net as an example. :)

Lets say I'm interested in probability and statistics. And I want to watch these lectures offline, on a train or wherever.

Unfortunately this isn't a straightforward process and quite a bit of work is required. It is derived from a mailing list thread.

I need to see the source code of the web page above, not the rendered output of the web browser. In Firefox press Ctrl+U. Now I need some hard data to pass to rtmpdump. Search for a javascript section which launches the flash video player. For the statistics video this is the interesting section:

var flashvars = {
    
      streamer: "rtmp://oxy.videolectures.net/video",
      file: "2007/pascal/bootcamp07_vilanova/keller_mikaela/bootcamp07_keller_bss_01.flv",
    
    height: '288',
    autostart: "true",
    bufferlength: '5',
  
  
    image: "http://media.videolectures.net/play.png",
  
    id: "FlvPlayer" // last line, no colon ',' !
  };
  swfobject.embedSWF("http://media.videolectures.net/jw-player/player.swf", "video_embed", "384", "307", "9.0.0", "http://media.videolectures.net/swfobject/expressInstall.swf", flashvars, params, attributes);

Generally searching (Ctrl+F) for flashvars should get you near the required data. The line swfobject.embedSWF(...) causes the video player to load inside your browser and display the correct media. This media rests on a different server than the webpage and I will use rtmpdump to connect to that server instead of the flash player. Now Adobe has put some roadblocks into the process but lucky for me I have a flying car. :)

I'll just give you the final command now and explain later!

./rtmpdump -r rtmp://oxy.videolectures.net/video/ -y 2007/pascal/bootcamp07_vilanova/keller_mikaela/bootcamp07_keller_bss_01 -a video -s http://media.videolectures.net/jw-player/player.swf -w ffa4f0c469cfbe1f449ec42462e8c3ba16600f5a4b311980bb626893ca81f388 -x 53910 -o test.flv

The -r switch requires an argument which is the URL of the media server and is found in the variable streamer in the JavaScript source code above.
The -y switch needs the playpath and that is found in the JavaScript variable file, (minus the extension .flv or .mp4).
The -a switch is the name of the used player and usually automatically inferred from the URL. Defining it manually works by copying the part after the server name in the streamer.
The -s switch defines the flash video player which normally connects to the media server. In the example it is the first argument of the function embedSWF.

Here's where things get even more complicated. The media server wants some extra data about this player, specifically its sha256 hash-sum and size in bytes. So lets get them:

wget http://media.videolectures.net/jw-player/player.swf
sha256sum player.swf
ls -l player.swf

Supply the sha256sum to switch -w and the file size to the -x switch.

Anything else? Yeah, I need to specify where to save the video with -o.

Run the longest command ever and get yourself a beer (you've earned it)! :)

I admit that doing this for every video gets time consuming but unfortunately it is a procedure specific to every site (look at the mailing list thread link again to see different javascript). But for the same site there's very little extra work.

Hope you lasted this long, be seeing ya!