Learning rtmpdump Through Examples

Learning rtmpdump Through Examples

by pstranger

Today, technology goes forward and we get some new possibilities. TV broadcasting doesn't lose time and goes forward too. Today, you can watch online TV and watch TV programs on demand. But what if you want to record them? Don't worry – some programs and live broadcasting (not all yet) can be recorded.

If you are not so young, then you can remember the VCR (video cassette recorder) era. You could turn on your video recorder, push the "Record" button, stop it after some time and after rewinding the tape you could play a recorded program on your TV. So what does the computer era offer?

Some theory

Today, TV broadcasting via internet goes in many popular stream formats and protocols.

You have certainly heard about such streaming formats:

• flash video streams (Adobe Flash Player): FLV, MP4, F4V
• Windows media streams (Windows Media Player): WMV, ASF, ASX and others.

There are also various streaming protocols. Examples of streaming protocols:

• flash video: HTTP, RTMP, RTMPE
• Windows media video: HTTP, RTSP, MMS and others.

Let's take a look at services which are broadcasting in the RTMP protocol. What is it? Wikipedia says: "Real Time Messaging Protocol (RTMP) was initially a proprietary protocol developed by Macromedia for streaming audio, video and data over the Internet, between a Flash player and a server. Macromedia is now owned by Adobe, which has released the specification of the protocol for public use."

So if your favorite online service (TV or video archive) uses the RTMP protocol you have a good chance of being able to save its video stream as a file on your local drive. There is an excellent utility for recording streams broadcasting TV and video on demand. This utility is called "rtmpdump." What is it? "rtmpdump" is a console toolkit for RTMP streams. Does the word "console" mean advantage or disadvantage in this case? I think it is big advantage for such a tool. Its operating principle is simple: you input the name of a command with some options, which include the address of the video server, and get a desirable file containing the recorded TV program. You will see no graphical input on the screen, nor other useless information. It just connects to the server, consuming only the network traffic containing the video, and records it to your hard disk. If you know about the powerful "wget" utility, then you can imagine what it is like.

Prepare

It's time to start. To have "rtmpdump" in your operational system, you should install the package having the same name from the repository. This package goes with another utility, which is called "rtmpsuck." This tool can also record video stream, but we will use it as auxiliary tool to find options for "rtmpdump."

In order to use "rtmpsuck," we need some preparation. "rtmpsuck" is a local RTMP proxy server which passes traffic into the internet through itself, detecting the flowing RTMP protocol and it's parameters. "rtmpsuck" goes with no parameters. So if you run "rtmpsuck" you can only see:

RTMP proxy server … … Streaming on rtmp://0.0.0.0:1935

What that says is that the server is running and listening on port 1935. It's necessary to notice that port 1935 is a default port of RTMP servers. But in this case, "rtmpsuck" is useless. Why? Because to make it work, you need to redirect RTMP traffic to a proxy server, which in turn will help us to know the connection parameters to the media server. To do this, you need to run the following command as a "root":

iptables -t nat -A OUTPUT -p tcp --dport 1935 -m owner \! --uid-owner root -j REDIRECT

The meaning of this command is to add a new rule in the table controlling your network traffic in order to redirect outbound TCP traffic flowing to the default RTMP external server port (1935) to a local RTMP proxy server, except traffic which belongs to root. So, if you now open a page in you browser to watch a media stream, your traffic will flow via the proxy server, and "rtmpsuck" will provide you information which can be used in "rtmpdump." After getting all the information, you can close "rtmpsuck" (Ctrl+C) and must remove the "iptabels" rule which you added before:

iptables -t nat -D OUTPUT -p tcp --dport 1935 -m owner \! --uid-owner root -j REDIRECT

1. Example: Online IP TV broadcasting ("Russia Today")

   Step 1 Run as a root in terminal:

   iptables -t nat -A OUTPUT -p tcp --dport 1935 -m owner \! --uid-owner root -j REDIRECT
   

   Step 2 Run as a root in terminal:

   rtmpsuck
   

   Step 3 As an ordinary user, open in Firefox this address: http://rt.com/on-air/rt-america-air/

   Step 4 Switch to root's terminal. You should see something like the following:

   app: live flashVer: LNX 10,1,82,76
   swfUrl: http://rt.com/s/swf/player5.4.viral.swf
   tcUrl: rtmp://rt.fms.visionip.tv/live
   pageUrl: http://rt.com/on-air/rt-america-air/
   Playpath: RT_US_3
   

   Break the command after a while (Ctrl+C).

   Depending on your connection quality, you can get different meanings of the option "Playpath".

   You can change the options of video quality by using the following entries after the command Playpath:

   RT_US_3 – high
   RT_US_2 – medium
   RT_US_1 – low
   

   Step 5 Run the command in root terminal:

   iptables -t nat -D OUTPUT -p tcp --dport 1935 -m owner \! --uid-owner root -j REDIRECT
   

   Step 6 Run the command in terminal as an ordinary user:

   rtmpdump -r rtmp://rt.fms.visionip.tv/live \
   -a live \
   -y RT_US_3 \ -W http://rt.com/s/swf/player5.4.viral.swf \
   -p http://rt.com/on-air/rt-america-air/ \
   -f "LNX 10,1,82,76" \
   -o ~/rt.flv \
   -V \
   -B 600
   

   After 10 minutes, you will get the video file "rt.flv" in your home directory which you can play in your favorite media player.

   You can know what these parameters mean if you run the manual of command "rtmpdump":

   man rtmpdump
   

   Option -V is just verbose, turn it off if your test recording is passed.

   Exercise 1: If you live in Europe or Asia (or if it is just convenient to you), you can try to get the parameters for main broadcasting. Firefox address is http://rt.com/on-air/

2. Example: Online IP TV broadcasting ("Bloomberg")

   Repeat steps 1 through 5 for address http://www.bloomberg.com/tv/

   Step 6 Run the following command in terminal as an ordinary user:

   rtmpdump -r rtmpt://cp87869.live.edgefcs.net/live \
   -a live \
   -y us_300@21006 \
   -W http://player.ooyala.com/static/cacheable/29d28e187072cf05af879e362d57b42d/player_v2.swf \
   -p http://www.bloomberg.com/tv/ \
   -f "LNX 10,1,82,76" \
   -o ~/bloomberg.flv \
   -V \
   -B 600 \
   --live
   

   Here we added the option "-- live" to make utility work right.

3. Example: TV streaming LIVE ("Euronews")

   The address of streaming broadcasting is: http://www.euronews.net/news/streaming-live/

   "rtmpsuck" gives us the next:

   app: rtpeuronewslive
   flashVer: LNX 10,1,82,76
   swfUrl: http://www.euronews.net/media/player_live_1_7.swf
   tcUrl: rtmp://178.32.100.43:1935/rtpeuronewslive
   pageUrl: http://www.euronews.net/news/streaming-live/
   Playpath: eng_euronews-flash-750.sdp
   

   so our command will be:

   rtmpdump -r rtmp://178.32.100.43:1935/rtpeuronewslive \
   -a rtpeuronewslive \
   -y eng_euronews-flash-750.sdp \
   -W http://www.euronews.net/media/player_live_1_7.swf \
   -p http://www.euronews.net/news/streaming-live \
   -f "LNX 10,1,82,76" \
   -o ~/euronews.flv \
   -B 600
   

4. Example: Video on demand ("Deutsche Welle")

   There are many good programs you can find on this page: http://www.dw-world.de/dw/0,,4756,00.html

   But if you try above technique, disappointment will wait for you. The reason for this is the video server is broadcasting on a different port. If you look into the source code of "rtmpsuck" you can see that default port of RTMP server (1935) is hard-coded, and that is why "rtmpsuck" is not a helper to us in this case.

   The tool that will help us is called "tcpdump." It is a console network sniffer. The nice feature of this tool is that it can show only printable symbols contained in network packets, which is handy for capturing web pages and this kind of stuff. If this tool is not yet installed on your system, you should do it now. The next utility is "grep" which helps to print lines matching a pattern which can be set.

   Let's get one of the excellent programs about lifestyle Europe, which is called "Euromaxx." We assume that RTMP proxy server is not running and redirection traffic is turned off.

   Step 1 Run as a "root" in terminal next set of commands:

   tcpdump -ieth0 -nn -A |grep -e"rtmp" -e"connect" -e"play">/euromaxx.txt
   

   It means that strings which included patterns (rtmp, connect or play) containing in network traffic and flowing trough network interface (eth0 in my case) will be recorded in file "euromaxx.txt," which will be located in root directory ("/").

   Step 2 As an ordinary user, open in Firefox page: http://mediacenter.dw-world.de/english/video/#!/93184/euromaxx/Program=7555 and start to play the video. Wait a moment for the video to start to play, click the "pause" button on the built-in player, and click the "Update page" button on Firefox. Repeat this action 4 times.

   Step 3 Go to root terminal and break running commands (Ctrl+C).

   Step 4 Open the file "euromaxx.txt" in text editor and find piece of text:

   connect.?..........app..
   a4337/dwwod1/..flashVer...LNX 10,1,82,76..swfUrl..;http://mediacenter.dw-world.de/player/flash/media.player.swf..tcUrl..*rtmpt://tvone.fcod.llnwd.net/a4337/dwwod1/..fpad....capabilities.@m........audioCodecs.@.........videoCodecs..@o....... videoFunction.?.........pageUrl..Khttp://mediacenter.dw-world.de/english/video/#!/93184/euromaxx/Program=7555..object.Encoding.@......... ................closeStream.....................H.....P.....play............;mp4:dwtv_video/flv/eme/emagen100311-euromaxx01ep_sd_avc.mp4
   

   Now you can easily determinate options for "rtmpdump".

   Step 5 Run command in terminal as ordinary user:

   rtmpdump -r rtmpt://tvone.fcod.llnwd.net/a4337/dwwod1/ \
   -a a4337/dwwod1/ \
   -y mp4:dwtv_video/flv/eme/emagen100311-euromaxx01ep_sd_avc.mp4 \
   -f "LNX 10,1,82,76" \
   -W http://mediacenter.dw-world.de/player/flash/media.player.swf \
   -p http://mediacenter.dw-world.de/english/video/#!/93184/euromaxx/Program=7555 \
   -o ~/euromaxx.flv
   

   After a while you will find the recorded program in your home directory.

   The pitfall of this technique is that you can meet dots in an inappropriate place in the string. The reason for this is the fact that the text string, which should be continuous, is situated in two network packets and "tcpdump" changes unprintable symbols to dots. So, if something goes wrong, then place the key -V into you "rtmpdump" command and look at the debug messages. If "rtmpdump" can't even connect to the server, then the reason in this case is rather a wrong URL.

Finishing stroke

There is a natural desire to record some program on a schedule (placing recording script for example in "cron"). If you will use the same file name of a recorded video program, then you will rewrite the previous one. In order to avoid this, it is very convenient to use the file name which contains the name of the broadcasting server, including the date and time of the recording. The next script shows the usage of this trick for our first TV broadcasting example:

#!/bin/bash
BASENAME="rt"
FILENAME="$BASENAME"_`date +%Y-%m-%d_%H%M%S`.flv
rtmpdump -r rtmp://rt.fms.visionip.tv/live \
-y RT_US_3 \
-o ~/$FILENAME \
-B 1200

Exercise 2: Write a script for the scheduler in order to record a program which was yesterday in the last video on demand example using this hint:

YESTERDAY=$(date -d yesterday +"%d%m%y")

The answers will appear elsewhere in this issue of magazine. Also, you can find more information about streaming media recording on this site http://all-streaming-media.com/