Image via WikipediaEDIT 2012-05-15: Updated guide so it uses the "new" git code repository! Thanks BecomingJulie!
Time for an updated guide - I told you previously that things change quickly! :)The new major release 2.x of rtmpdump brings new tools which make things much easier. How much? Well downloading shows of The Colbert Report, The Daily Show and episodes from South Park Studios is now almost a breeze!
What is this gamechanger you say? It's the new rtmpsrv program which comes with rtmpdump 2. It does the guessing game for us and gives a copy-pastable command. In short you have to position it between your browser and the internet and allow it to perform essentially a man-in-the-middle attack on the handshake of the proprietary protocol.
Compilation on GNU/Linux (for other system please read the README file):
sudo apt-get install build-essential git zlib1g-dev libssl-dev
git clone git://git.ffmpeg.org/rtmpdump cd rtmpdump make SHARED= SYS=posixThe program is now compiled for use on your system! Now the harder part - you need root access to configure the iptables firewall which should be a part of any modern Linux distro. This is the part which will reroute the RTMP communication of the flash plugin in your browser through a local port:
sudo iptables -t nat -A OUTPUT -p tcp --dport 1935 -j REDIRECTExcellent, now start the program rtmpsrv which will listen on the local port:
./rtmpsrvBrowse to a media site and wait for rtmpsrv to write out a command in the console. If you see multiple chapters on the media site, try skipping to another one. I'll describe why in a minute.
OK! You should see at least one long string starting with rtmpdump. rtmpsrv did its job so close the browser tab and stop rtmpsrv with a Control+C signal. Ending rtmpsrv seems to be a problem. The cleaning up process seems to go nowhere at times but sending several Ctrl+C events usually produces a segfault. :/
Also delete the firewall redirect so that you can access the media server directly:
sudo iptables -t nat -D OUTPUT -p tcp --dport 1935 -j REDIRECTIf you're in luck taht is it and you only need to run the output of rtmpsrv, like this:
./rtmpdump -r "rtmpe://cp9950.edgefcs.net:1935/ondemand?ovpfv=1.1" -a "ondemand?ovpfv=1.1" -f "LNX 10,0,42,34" -W "http://media.mtvnservices.com/player/release/?v=4.2.2-8" -t "rtmpe://cp9950.edgefcs.net:1935/ondemand?ovpfv=1.1" -p "http://www.colbertnation.com/full-episodes/thu-january-14-2010-kathleen-sebelius" -C Z: -y "mp4:mtvnorigin/gsp.comedystor/com/colbert/season_06/episode_008/cr_06008_act1_768x432_1720.mp4" -o output.flvThe problem with Comedy Central shows are commercial breaks which cut the show into different separate chapters called acts (see the end of command above). This gives us a bit more work but it's so worth it avoiding the horrible full screen flash experience! Smooth full-screen playback has arrived, Randall! :)
This particular episode of The Colbert Report has 4 acts - all you need to do is change the actX in the command and the output filename must be different for each act too. Ending up with 4 differnet videos is kind of lame but manageable. If you know of a FLOSS stitching solution, please comment! (Avidemux doesn't wanna, PiTiVi wants to transcode, haven't tried LiVES yet).
SouthParStudios.com has a different gotcha - the first clip is the intro which is the same for all shows. Only the following acts are show specific. I managed to get a second command from rtmpsrv by seeking in the browser as mentioned above. Behold:
./rtmpsrv RTMP Server v2.1c (c) 2010 Andrej Stepanchuk, Howard Chu; license: GPL Streaming on rtmp://0.0.0.0:1935 WARNING: Trying different position for client digest! rtmpdump -r "rtmpe://cp40493.edgefcs.net:1935/ondemand?ovpfv=1.1" -a "ondemand?ovpfv=1.1" -f "LNX 10,0,42,34" -W "http://media.mtvnservices.com/player/release/?v=4.2.1" -t "rtmpe://cp40493.edgefcs.net:1935/ondemand?ovpfv=1.1" -p "http://www.southparkstudios.com/episodes/251890" -C Z: -y "mp4:mtvnorigin/gsp.comedystor/com/sp/extras/intros/PromoIntros/HDintroPromoXX_768x432_750.mp4" -o output.flv rtmpdump -r "rtmpe://cp40493.edgefcs.net:1935/ondemand?ovpfv=1.1" -a "ondemand?ovpfv=1.1" -f "LNX 10,0,42,34" -W "http://media.mtvnservices.com/player/release/?v=4.2.1" -t "rtmpe://cp40493.edgefcs.net:1935/ondemand?ovpfv=1.1" -p "http://www.southparkstudios.com/episodes/251890" -y "mp4:mtvnorigin/gsp.comedystor/com/sp/season_13/1313/acts/sp_1313_act1_768x432_750.mp4" -o output.flv ^CCaught signal: 2, cleaning up, just a second...And we have lift-off! See the act1 above? Incrementing this value will give you all the acts.
I am the first to admit this should be a total overkill for a good viewing experience but the current situation isn't really bearable. While fast modern multi-core desktop processors are capable of much smoother playback my aging Pentium M and a masses of netbooks struggle with this task. And of course (legally) downloading the shows and watching them offline is now possible! I think modern smartphones and media devices should have no problem with playback of these local files but as always YMMV.
Creators of rtmpdump, on behalf of the Linux masses suffering from stuttering full-screen playback with flash, I salute you!
Before anyone asks: the source code of rtmpdump comes with a README file. In there are more instructions, e.g. using the pf firewall from OpenBSD.
ReplyDeleteHow to do this on Windows? There's a compiled version right on the rtmpdump homepage but I don't know how to easily redirect traffic.
hello !
ReplyDeletesuch a pretty tuto ! thanks a lot :)
But I have a question : i'm with mac os x snow leopard ... so I don't have iptables on my system ... what can I do ?
thanks !
carmelo
@Carmelo
ReplyDeleteMacOS X fortunately uses the ipfw firewall that is common in *BSD operating systems.
Fortunately the README of rtmpdump conatins instructions for ipfw as well:
ipfw add 40 fwd 127.0.0.1 1935 tcp from any to any 1935
ipfw add 40 fwd 127.0.0.1 1935 tcp from any to any 1935 not uid proxy
You'll have to find a way to delete these rules after using rtmpsrv yourself!
For more see the ipfw manpage and a graphical interface called NoobProof.
hey !
ReplyDeletethanks for your answer !
so .. with this command, there is a problem :
MacPro:~ Carmelo$ ipfw add 40 fwd 127.0.0.1 1935 tcp from any to any 1935
ipfw: missing ``from''
MacPro:~ Carmelo$
the ipfw tools doesn't like 'any' parameters ... I will look at man page and NoobProof
thanks again :)
@Carmelo
ReplyDeleteTry this one:
# ipfw add 41 fwd 127.0.0.1,1935 tcp from me to any dst-port 1935
Deleting the rule might work with:
# ipfw delete 41
To make sure the rule is applied or deleted, see if it is in the ruleset with:
# ipfw show
thanks !
ReplyDeletebut i tried your command.
i can see with ipfw show the rule #41 ...
but still nothing with rtmpsuck ...
(i have tried under ubuntu with same video site : no problem, rtmpsuck shows good informations)
rtmpsrv 2.1d automatically downloads videos using rtmpdump now.
ReplyDeleteno more copy/pasting.
Hello everyone!
ReplyDeleteI don't know if rtmpdump can be installed into a IP Cameras? The OS of IP Camera is embedded Linux. Because we try to publish the live streaming of IP Camera via RTMP protocol. Could you kindly give me more information about this?
Thank you very much!
Hello, Anonymous poster!
ReplyDeleteThere certainly are server implementations. take a look on lkcl.net (search for crtmpserver on that page).
Thanks to the help here and a couple of other places, I was finally able to get this to work.
ReplyDeleteI kept getting segmentation faults when running rtmpdump and it turned out that I have librtmp installed on my machine (Fedora Core 14).
In order to get around that - as stated at the very end here - you must use the LD_PRELOAD variable to specify the librtmp that comes with (or is compiled with) the rtmpdump package/source.
My basic routine for doing this now consists of this:
sudo iptables -t nat -A OUTPUT -p tcp --dport 1935 -j REDIRECT
rtmpsrv
sudo iptables -t nat -D OUTPUT -p tcp --dport 1935 -j REDIRECT
export LD_PRELOAD=/home/lavid/lib/librtmp.so.0
rtmpdump -r [...]
Hope this helps someone...
Awesome tutorial! Haha, now to go back through the man pages with this in hand and get closer to actually learning how to fish.
ReplyDeleteThank you very much for this. You are a star!
ReplyDeleteMissed the first episode of My Transsexual Summer and tried for ages to get the stream to download. Obviously needed rtmpdump, and it needed to be 2.4 out of git to work with "version 9". Grr. Oh well. I'd already built the one from the tarball, so I knew I had the dependencies. Segmentation fault! Turned out to be cos some tart forgot to replace /usr/local/lib/librtmp.so.0 with the new one. Used tcpflow to get the xml file with the StreamUri in it, and it almost worked. But only almost.
Even tried running a shareware program in Wine. Didn't work, quelle surprise, but the fact that others had had success under real Windows suggested it was at least possible to pull the streams as I wanted. What I couldn't work out was how to get rtmpsrv to listen to the address it was supposed to. Fortunately, before I went as far as installing and configuring BIND9 just for the purpose of a bit of DNS redirection, I guessed there must be a trick with iptables.
Now, I probably would have figured it out for myself sooner or later, me being such a kewl h4xx0r chick and all that :) (Well, *eventually* the chick bit, but that's another story.) But thanks anyway for taking the time to write this up.
BTW, since I believe in giving back, here's a tip from me. If you ever want to do a bunch of commands in a row that all need sudo, try
$ sudo /bin/bash
The prompt will change to a comment mark, and you're now effectively root (just like on any old-fashioned Unix). Press Ctrl+D to get back to normal.
Hi everyone,
ReplyDeleteI have a problem with Rtmpdump . I hope someone can help me...
I did everthing up here but error has occured. I want to download stream from this adress "http://www.ttnetmuzik.com.tr/radio". I wrote this command on console window.
ercan@ubuntu:~$ sudo iptables -t nat -A OUTPUT -p tcp --dport 1935 -j REDIRECT
ercan@ubuntu:~$ rtmpsrv
RTMP Server v2.3
(c) 2010 Andrej Stepanchuk, Howard Chu; license: GPL
Streaming on rtmp://0.0.0.0:1935
rtmpdump -r "rtmpe://streaming.ttnetmuzik.com.tr/ttnet?QzlHbWxPZFJWNndCUnBKSDZNM1VmNEpZYUVvN0pXQTM=" -a "ttnet?QzlHbWxPZFJWNndCUnBKSDZNM1VmNEpZYUVvN0pXQTM=" -f "LNX 11,2,202,228" -W "http://www.ttnetmuzik.com.tr/TTNETMuzik/miniplayer/swf/flowplayer.rtmp-3.1.3.swf?v1" -p "http://www.ttnetmuzik.com.tr" -y "mp4:20835b420fda63a6bd7c9c944939df31ffc446a059001fc7.m4a" -o mp4_20835b420fda63a6bd7c9c944939df31ffc446a059001fc7.flv
Closing connection... done!
RTMPDump v2.3
(c) 2010 Andrej Stepanchuk, Howard Chu, The Flvstreamer Team; license: GPL
Connecting ...
WARNING: HandShake: Type mismatch: client sent 6, server answered 8
INFO: Connected...
Duplicate request, skipping.
Closing connection... ERROR: RTMP_ReadPacket, failed to read RTMP packet header
done!
Can anyone help me this error?
Hi, ercan!
ReplyDeleteThe website will only stream to Turkish IPs: "We are sorry to say that due to licensing constraints, we can not allow access to visitors located outside of Turkey." but from your error it might be that the server at ttnetmuzik.com.tr only allows clients that know how to talk RTMP version 8. Your program only seems to offer version 6 but the server does not downgrade the protocol version. This seems to be a misconfiguration on the server.
On the homepage of rtmpdump the authors say that it should support up to protocol version 9. I suggest you (re)do the steps in the blog post above so you have the absolute newest program. Then try dumping the stream again.
This comment has been removed by the author.
ReplyDeleteI'm very glad to interest my problem. Thank you very much for helping me.
ReplyDeleteI installed newest version of rtmpdump. I's new version is V2.4. It's suported protocol 9. But this time another error ocured. Error is shown like this:
ercan@ubuntu:~/rtmpdump$ rtmpsrv
RTMP Server v2.3
(c) 2010 Andrej Stepanchuk, Howard Chu; license: GPL
Streaming on rtmp://0.0.0.0:1935
rtmpdump -r "rtmpe://streaming.ttnetmuzik.com.tr/ttnet?VDg4TGJEV3VxQ0grc2RLdUE1QjlqOEs1UEI5OXVVOTk=" -a "ttnet?VDg4TGJEV3VxQ0grc2RLdUE1QjlqOEs1UEI5OXVVOTk=" -f "LNX 11,2,202,233" -W "http://www.ttnetmuzik.com.tr/TTNETMuzik/miniplayer/swf/flowplayer.rtmp-3.1.3.swf?v1" -p "http://www.ttnetmuzik.com.tr" -y "mp4:c03ac34907a9946890aa7c16b39653494fe44fd9d26a9419.m4a" -o mp4_c03ac34907a9946890aa7c16b39653494fe44fd9d26a9419.flv
Closing connection... done!
RTMPDump v2.3
(c) 2010 Andrej Stepanchuk, Howard Chu, The Flvstreamer Team; license: GPL
Failed to open file! mp4_c03ac34907a9946890aa7c16b39653494fe44fd9d26a9419.flv
If i write sudo begining of the code line (ercan@ubuntu:~/rtmpdump$ sudo rtmpsrv
) program give same error which i tried earlier.
What should i do to overcome this error ?
sudo apt-get install git
ReplyDeletegit clone git://git.ffmpeg.org/rtmpdump
sudo apt-get install build-essential subversion zlib1g-dev libssl-dev
cd /home/ercan/rtmpdump rtmpdump
make SHARED= SYS=posix
I wrote this code respectively. But after that i run the "rtmpsrv" code. The error is given like this.
"rtmpsrv" is not installed. You can install by typing the following lines
"sudo apt-get install rtmpdump"
When i wrote this line, v2.3 is installed. Then error is the same as previously error. I couldn't solve this problem.
ercan, did you forget the ./ before the rtmpsrv command? Try running $rm -rf ~/rtmpdump and then the commands from the above post again.
DeleteAlso thanks for your comments, they belatedly made me realize my guide was a bit obsolete. I should have read BecomingJulie's comment more thoroughly.
I updated the post above with the new repository location.
Thank you very much for this great tutorial!
ReplyDeleteIt helped me a lot.
:-)
There's a further refinement you can do that saves you the bother of cutting and pasting.
ReplyDeleteThe latest rtmpsrv actually tries to do the rtmpdump command itself, but it fails because the command it was trying to send to the server gets redirected to itself ad nauseam. So try this:
First make a new folder to save your downloaded files in, and navigate to there:
$ mkdir downloaded_video
$ cd downloaded_video
Now start iptables -- note the slight modification:
$ sudo iptables -t nat -A OUTPUT -p tcp --dport 1935 -m owner --uid-owner jules -j REDIRECT
(you will need to replace "jules" above with your own login name.) This will only redirect packets if the owner of the connection is "jules".
$ sudo /path/to/rtmpsrv
(you may not need to give the full path, if rtmpsrv is copied somewhere in your PATH.)
Now, since we used sudo, rtmpsrv is running as "root"; so the connection set up by rtmpsrv will belong to "root", not "jules". Therefore, the packets coming from our person in the middle will *not* match the rule, and so will not be redirected, but sent straight to the real server.
When you're done, press ctrl+C to close down rtmpdump -- but you are not quite ready to close the terminal yet. The files were downloaded by, and still belong to, "root". So now you need to use the "chown" command to CHange the OWNer of the files to yourself.
Type
$ sudo chown jules:jules *
(replace "jules" with your own login name, of course)
Now the files you saved belong properly to you. Before you close down the terminal, run this one final command:
$ sudo iptables -t nat -D OUTPUT -p tcp --dport 1935 -m owner --uid-owner jules -j REDIRECT
This will remove the iptables redirection we applied earlier.
Nice tutorial...
ReplyDeleteAs for a "FLOSS stitching solution" this works for me most of the time. .
Run this on each of the parts:
ffmpeg -i INCOMING.(FLV/MP4) -vcodec copy -acodec copy VIDEO_(01,02).mkv
then:
mkvmerge -o FULL_VIDEO.mkv VIDEO_01.mkv +VIDEO_02.mkv +VIDEO_03.mkv
Results can vary depending on your source files!
Thanks the tutorial. . .
Hi all,
ReplyDeleteFirst of all Thanks for the great tutorial.
Now for the shitty part... I'm running this on osx using ipfw and when I set the rule for port 1935, the flash player will load the stream from https instead, like it's understanding that I'm trying route it to localhost. When I add a second rule, this time IP specific I do see traffic going through it, but the player does nothing on the browser and I see no output on rtmpsrv :(
Do you think it might have anything to do with me being on osx? Has anyone had any luck getting Colbert report lately?
Thanks,
Pedro
Hello Pedro!
DeleteYes, apparently MTV changed something on their Adobe media servers and rtmpdump developers have not yet come up with a solution. I tried some older versions of rtmpdump and they all don't work for TDS & TCR.
I can't really comment on the ipfw firewall issue though, I don't have access to any Macs. To see if the firewall really has anything to do with your issues it would be interesting to try dumping the video on Linux. Can you share the website address which is giving you grief?