Constant Curiosity

Rooting Android

2010-09-23T18:32:00.001+02:00

Image via WikipediaRooting enables an Android phone owner to do interesting things with his hardware. Let me write a few: performing whole-system backups, upgrading the operating system found on the device, removing clutter from the default install, even underclocking and overclocking become possible. These procedures are especially attractive when the phone manufacturer decides to no longer support the device with any software updates.
Unfortunately, doing such a procedure usually voids any warranty you may have on your phone, so all responsibility rests on your shoulders.

OK, why the name rooting? On UNIX-like operating systems like *BSDs and Linuxes such as Android, root denotes the do-all super-user on the machine. That means it can modify almost any file, anywhere. Getting this privilege level is therefore essential for manipulating protected files! A modern operating system has methods for entering such a privileged state but they are missing on Android. To make matters worse, the OS by design limits non-privileged code from attemting any other privilege escalation methods. All is not lost, however, since operating systems are usually so big, that there's bound to be some weak code, which can be exploited to cut the flow of normal execution and execute the exploit's code instead.

Here enters an exploit. It exploits some specific functionality of the OS' kernel which has been vulnerably implemented. Such code is found all the time by security researchers, hobby hackers and also malevolent crackers, but more importantly more or less quickly patched by software authors. Because security software updates are relatively unneeded on Android due to the isolationist application model, updates for phones are rare or even non-existent and usually focus on improved functionality.

That should be enough backround information, onto a specific procedure! My HTC Tattoo uses Linux version 2.6.29, which is rather old and vulnerable. An exploit exists for it! Good, now a would be hacker also needs a shell to run this program in. The application model on Android doesn't run software stored as native processor code but the software developer kit for Android does. Once you have it extracted, open a terminal, navigate to that folder and connect your phone to the PC in debugging mode. To see if the phone is accessible run:

./adb devices

This should give you a serial number and a "running" status. Cool, now you can upload the compiled exploit file (find it on xda-developers) to your phone:

./adb shell "mkdir /data/local/bin"
./adb push path/on/PC/to/m7 /data/local/bin/m7
./adb shell "chmod 755 /data/local/bin/m7"

Now you can enter a full shell running on your phone and actually execute the exploit:

./adb shell
cd /data/local/bin
while ./m7 ; do : ; done

After a while you should be greeted by a # sign, the tell-tale mark of full privilege! But nothing works. That is because the way you got to the root shell is highly non-standard and the interpreter knows not where to look for commands you type in. To remedy this, type:

# export LD_LIBRARY_PATH=/system/lib
# export PATH=/system/bin
# id

The last command should confirm you are root with user id zero. Congrats! But know that what you have just accomplished is just a temporary root. It will only work for this shell session.

To have a permanent privilege escalation mechanism similar to more feature-full GNU/Linux distributions and that works whenever, you must install a program within your root shell called su, short for super user. You can find one here, but please install the Superuser app as well. Upload the su program the same way you uploaded the exploit file, with adb push. Once in the root shell, you must remount the system partition in the phone as writeable, because it is mounted as read-only by default:

# mount -o rw,remount /dev/block/mtdblock3 /system
# cat /data/local/bin/su > /system/bin/su
# chmod 6755 /system/bin/su

Now reboot your phone.

# reboot

Once the phone completes the start up process it should be exactly the same as before, but with one added mechanism: the su program. The problem now is that any program can launch su and become the superuser so your first step to control its use should be to install an app called Superuser (it's also on the Android Market). For usage instructions see link. It's pretty simple though - if any non-approved app wants to run su, Superuser will be invoked to ask you if you allow or deny the requesting app such privileges.

Congratulations, you are now a mobile super user! :)

On Android communities

2010-09-18T18:12:00.001+02:00

Image by ToastyKen via FlickrWhile not writing for the blog many things changed in all of our lives but to be a bit more specific, I bought an Android mobile phone. It is a budget second generation phone made by HTC and called Tattoo.

Android is the operating system that runs on the phone. It manages the hardware and offers a unified platform for apps to run on it. One of the main advantages of Android is its freedom. Because it is based on Linux, its licence forces the source code to be available for anyone to hack. However, the devices themselves aren't nearly as open. That's why hackers across the world love to break into phones and change their inner workings. Such actions are usually deemed entirely legal and can offer quite a bit of insight into how computers are made to work and the quality of work done by industry engineers.

Why would someone root their phone? To do advanced things to it that are otherwise impossible! For instance my Tattoo had faulty buttons under the screen. Before sending it in for repairs under warranty I copied the contents of the on-board memory chips onto my memory card and did a factory reset. That way no prying eyes could access my data, my email account, etc. That turned out to be a good move because the phone had to have its motherboard replaced. On this motherboard the memory chips were soldered and I would have lost months of customizations had I not made backups.

Many discussions on mobile phone hacking and development take place on the forum xda-developers.com. Of course there are many other, more specific sites, but usually everything worth noting can be found on these forums. A problem I had at the beginning while reading various threads there was that, sure, there are many technical and newbie guides but very little is said what the accomplished result is useful for. Perhaps that is also a consequence of the bulletin board nature of the site. I find that wiki sites are much better for new people because all the information is centralised on one page, condensed and more formatted for legibility than countless bits of information spread around forum posts.

By the way, if you are considering buying a smartphone, please, PLEASE, do yourself a favour and don't get a budget phone. I find myself wanting the characteristics of a better phone all the time. The hardware can be very limiting to the usefulness of various third party applications and even to their availability. Fortunately some of the limitations are software based and therefore the attractiveness and promise of community based upgrades and hacks becomes quickly evident.

I imagined this blog as being more technically orientated so as not to deviate from that premise I will soon post some guides how (and why) I have fun with my mobile phone.

Bye!

Lets get Flash squared away

2010-09-18T12:56:00.002+02:00

Hello after a long time. I have some free time again and my almost forgotten blog crept into my mind. Well the big news of the day, actually yesterday, is that a new version of the Flash player has been released, not only for the usual suspect like Windows and OS X but also for 64-bit Linux!

The big Flash 10.1 release a few months ago obviated users of 64-bit GNU/Linux distributions and their users have been left out in the cold regarding support for a native patched and secure Flash player. The last version was just suddenly made unavailable due to security holes and no update was released.

Fortunately a new version has been released, interestingly according to one not only due to outside pressure.

Here are a few tips regarding its installation on Ubuntu amd64:

If you have any flash packages installed, it is prudent to remove them first:

sudo apt-get remove -y --purge flashplugin-nonfree flashplugin-installer gnash

If you had followed my previous guide, you need to clean up after it:

sudo rm -f /usr/lib/mozilla/plugins/*flash*
sudo rm -f ~/.mozilla/plugins/*flash*
sudo rm -f /usr/lib/firefox/plugins/*flash*
sudo rm -f /usr/lib/firefox-addons/plugins/*flash*

Ubuntu users are welcome to install the plugin from a Personal Project Archive (Wikipedia article). This is a volunteer-based solution which bridges the gap between upstream release and Ubuntu official packaging. It should be as easy as running:
```
sudo add-apt-repository ppa:sevenmachines/flash
sudo apt-get update
sudo apt-get install flashplugin64-nonfree
```
If you wish to remove the PPA and the installed software it provides, you have two options. The manual way is to remove the PPA from the Software repositories list and reverting or uninstalling software packages the PPA provides or you could settle for an automatic process. You need to install another PPA first, though. Removal of the unwanted PPA is then as easy as this:
```
sudo ppa-purge ppa:sevenmachines/flash
```

Users of other GNU/Linux distributions can use the procesure below (check the plugin paths, though!)

wget -qO- http://download.macromedia.com/pub/labs/flashplayer10/flashplayer_square_p1_64bit_linux_091510.tar.gz | tar xz > libflashplayer.so
sudo mv libflashplayer.so /usr/lib/mozilla/plugins/

System monitoring beyond GNOME's System Monitor

2010-03-16T10:41:00.000+01:00

I've been having problems with my quad-core computer lately. I noticed that 2 out of 4 logical cores were misbehaving, i.e. they were not scaling their frequency up to meet the load I was putting on them and there were also fishy temperature readings.

You may ask yourself how to monitor things like load, frequency and temperature on a Linux box. The answer isn't all that complicated. If you want a graphical program, there are many: sensors-applet for GNOME's bars, conky and gkrellm for your desktop and probably many others I don't know about. All of them need some level of setting up, please look at the relevant docs on te tools' websites.

What about console tools? The main tools here are lm-sensors, hddtemp and acpi. For experienced users console use is usually simpler, faster and more importantly, more precise. Setting up lm-sensors is simple. Running the following and pressing enter a bunch of times will tell you which drivers (modules, actually) you need to load so that sensors can reliably be read.

$ sudo sensors-detect
...
To load everything that is needed, add this to /etc/modules:

#----cut here----
# Chip drivers
it87
coretemp
#----cut here----

Do you want to add these lines automatically? (yes/NO)

My computer has an Intel Core 2 Quad with embedded on-die sensors (module coretemp) and a motherboard based on the Intel P45 chipset (module it87).

Adding the modules to the mentioned file will cause them to be loaded every system start but we want to read stuf ASAP!

$ sudo modprobe coretemp
watch sensors

The watch command is useful here as it executes whatever command every given time period. The default is 2 seconds, enough for my purposes.

Every 2.0s: sensors                                     Mon Mar 15 17:43:13 2010

ERROR: Can't get value of subfeature temp1_input: Can't read
coretemp-isa-0000
Adapter: ISA adapter
Core 0:       +0.0C  (high = +82.0C, crit = +100.0C)  ALARM

coretemp-isa-0001
Adapter: ISA adapter
Core 1:      +37.0C  (high = +82.0C, crit = +100.0C)

coretemp-isa-0002
Adapter: ISA adapter
Core 2:      +58.0C  (high = +82.0C, crit = +100.0C)

coretemp-isa-0003
Adapter: ISA adapter
Core 3:      +36.0C  (high = +82.0C, crit = +100.0C)

Definitely something amiss with the CPU or the LGA775 socket with its bend-prone pins. Please note that an Intel stock cooler is installed for service-personnel-excuse-finding-avoidance purposes.

One more thing to monitor is the CPU frequency. Modern CPUs need to be green so features originating in laptops came to the desktop, specifically frequency scaling based on CPU load. A good way of watching this is

$ watch grep MHz /proc/cpuinfo

. There are tons of information about the CPU cores in that file and getting just what we need out of it is just one scalpel^W grep away!

Now we need to generate load on the processor cores. There benchmarking tools on Linux aren't as easy to use as on Windows but many of them are quite interesting. I like the Phoronix Test Suite, used by many websites to test in Linux and encompasses so many tests that you really need to pick and choose. A good way of seeing what results should be is comparing your own to other systems on Phoronix Global. One of the better tests in this suite is sunflow. It uses Java and is parallelized, which means that it uses all CPU cores it can find to solve a problem.

$ phoronix-test-suite benchmark sunflow

During this test the things to watch are the frequency and perhaps the temperatures. I noticed that my frequencies didn't scale:

Every 2.0s: grep -i MHz /proc/cpuinfo                   Mon Mar 15 18:03:52 2010

cpu MHz         : 1600.000
cpu MHz         : 2400.000
cpu MHz         : 1600.000
cpu MHz         : 2400.000

Definitely something wrong. To further analyse these peculiarities an unparallelized test with short runs is needed. One of them is java-scimark2, a collection of mathematical algorithms. Here are the results on my faulty system:

$ phoronix-test-suite run java-scimark2

========================================
Test Configuration: Java SciMark
========================================


Computational Test:

1: Composite
2: Fast Fourier Transform
3: Jacobi Successive Over-Relaxation
4: Monte Carlo
5: Sparse Matrix Multiply
6: Dense LU Matrix Factorization
7: Test All Options

Enter Your Choice: 2

Would you like to save these test results (Y/n)? n

========================================
Estimated Run-Time: 5 Minutes
========================================



Java SciMark:
      java-scimark2 [Computational Test: Fast Fourier Transform]
      Estimated Test Run-Time: 5 Minutes
      Expected Trial Run Count: 4
            Started Run 1 @ 18:07:15
            Started Run 2 @ 18:07:49
            Started Run 3 @ 18:08:26
            Started Run 4 @ 18:09:00
            Started Run 5 @ 18:09:35
            Started Run 6 @ 18:10:07
            Started Run 7 @ 18:10:46
            Started Run 8 @ 18:11:18

      Test Results:
            481.9675767405814
            465.3790214886224
            167.040859476713
            484.107875662264
            481.9675767405814
            482.10078089758196
            142.08542169663755
            491.33539814892885

      Average: 399.49 Mflops

I would expect all runs to operate at about 480 mega FLOPS, however, there are two results that pull the average significantly down. Other tests for the java-scimark2 collection confirm these results so the CPU is definitely not operating as intended.

I've also tried swapping the power supply and memory modules to no avail and fortunately both the CPU and motherboard are still covered by warranty.

And once these tools become ubiquitous, there's the blog NIXCraft. It will just knock your socks off with the quantity of quality content. It really is ...simply the best! :)

Tweaking Ubuntu's GDM and nautilus

2010-02-23T15:19:00.001+01:00

Image via Wikipedia

There are tons of little tweaks I constantly make in my system to have an easier time spending a lot of time in it.

With this post I'd like to show you two of them.

The first tweak is actually a fix to the new login manager GDM. Ubuntu got a new login manager in release 9.10 to essentially save a precious few seconds during start-up. Sadly the rewrite is much less configurable and some argue is a trend the GNOME project has been showing. I agree to an extent that some features are being rushed and not enough care is being made that functionality is preserved. For instance the configuration dialogue of the new GDM is very Spartan and doesn't even have many of the options that it should have. I find that NetworkManager actually lacks a useful command line interface because the D-Bus way is just way to cumbersome.
Anyway, I found the drum sound that played on every start-up very annoying. If you haven't guessed from my rant, there is no obvious way to disable it. Fortunately other people were as well(a moment of Schadenfreude indeed) so a quick tweak/hack arose and this is it:

$ cd /usr/share/sounds/ubuntu/stereo
$ sudo mv system-ready.ogg system-ready.ogg.DISABLED-FOR-BEING-ANNOYING

Some argue that such tweaks are actually hacks because they interfere with the system in unexpected ways (like my flash on amd64 guide). Unfortunately in the absence of supported solutions such quick fixes are necessary. It always helps to make a mental note or better yet to keep a log of such modifications in case things go awry on some upgrade.

The second tweak I wanted to mention is in fact something that is supported in the official Ubuntu software channels and those are pug-ins for the nautilus file manager. On its own it lacks some features a power user really needs, the most glaring of which is an Open terminal here command. I actually tried implementing this myself because nautilus has a simple extension mechanism using scripts but I found a better solution by just installing one package:

$ sudo aptitude install nautilus-open-terminal

Here are the other interesting extension packages you might want to use:

nautilus-filename-repairer - Nautilus extension for filename encoding repair
nautilus-gksu - privilege granting extension for nautilus using gksu
nautilus-image-converter - nautilus extension to mass resize or rotate images
nautilus-open-terminal - nautilus plugin for opening terminals in arbitrary local paths
nautilus-script-audio-convert - A nautilus audio converter script
nautilus-script-manager - A simple management tool for nautilus scripts
nautilus-wallpaper - Nautilus extension. Add a "set as wallpaper" entry in context menu

Finding old Ubuntu ISOs

2010-02-22T23:34:00.002+01:00

Just a quick note so I have a post this month. You know, I'm super busy, blah blah work, etc. etc.

For a course I'm doing this year I needed several older Ubuntu releases to test Linux's Kernel Virtual Machine. It was included into Linux for the 2.6.20 release just over three years ago. It just so happens that the first Ubuntu revision that had this kernel was Feisty Fawn or the more commonly used label 7.04. Because of Ubuntu's support policy (18 months per revision, except LTS which has 36 months) revisions 7.04 and 7.10 are no longer supported. That also means the isos (CD images) are hard to come by because most mirror servers delete unneeded files. Fortunately I quickly found a nearby mirror on the mirror server listing that still had some older versions so I am a bit happier!

Running unsupported versions is of course discouraged because critical imperfections are no longer being fixed for the abandoned distribution but for educational purposes there is no objection.

Another good tip I found is that although these older Ubuntu revisions are no longer supported with improvements to software packages, at least improvements since the release of the CD image are still available through the in-built package management system. There are a few tweaks needed for adjusting software package sources, namely renaming all software package servers from archive.ubuntu.com to old-releases.ubuntu.com.

A quick history of Ubuntu releases:

Ubuntu release	codename	date	kernel version
Ubuntu 4.10	Warty Warthog	2004-10-20	Linux 2.6.8
Ubuntu 5.04	Hoary Hedgehog	2005-04-08	Linux 2.6.10
Ubuntu 5.10	Breezy Badger	2005-10-13	Linux 2.6.12
Ubuntu 6.06 LTS	Dapper Drake	2006-06-01	Linux 2.6.15
Ubuntu 6.10	Edgy Eft	2006-10-26	Linux 2.6.17
Ubuntu 7.04	Feisty Fawn	2007-04-19	Linux 2.6.20
Ubuntu 7.10	Gutsy Gibbon	2007-10-18	Linux 2.6.22
Ubuntu 8.04 LTS	Hardy Heron	2008-04-24	Linux 2.6.24
Ubuntu 8.10	Intrepid Ibex	2008-10-30	Linux 2.6.27
Ubuntu 9.04	Jaunty Jacklope	2009-04-23	Linux 2.6.28
Ubuntu 9.10	Karmic Koala	2009-10-29	Linux 2.6.31

After I complete my report I might have to make a little series on KVM! :)

Revisiting rtmpdump

2010-01-26T16:36:00.008+01:00

Image via WikipediaTime for an updated guide - I told you previously that things change quickly! :)
The new major release 2.x of rtmpdump brings new tools which make things much easier. How much? Well downloading shows of The Colbert Report, The Daily Show and episodes from South Park Studios is now almost a breeze!
What is this gamechanger you say? It's the new rtmpsrv program which comes with rtmpdump 2. It does the guessing game for us and gives a copy-pastable command. In short you have to position it between your browser and the internet and allow it to perform essentially a man-in-the-middle attack on the handshake of the proprietary protocol.
Compilation on GNU/Linux (for other system please read the README file):

sudo apt-get install build-essential subversion zlib1g-dev libssl-dev

svn checkout svn://svn.mplayerhq.hu/rtmpdump/trunk rtmpdump
cd rtmpdump
make SHARED= SYS=posix

The program is now compiled for use on your system! Now the harder part - you need root access to configure the iptables firewall which should be a part of any modern Linux distro. This is the part which will reroute the RTMP communication of the flash plugin in your browser through a local port:

sudo iptables -t nat -A OUTPUT -p tcp --dport 1935 -j REDIRECT

Excellent, now start the program rtmpsrv which will listen on the local port:

./rtmpsrv

Browse to a media site and wait for rtmpsrv to write out a command in the console. If you see multiple chapters on the media site, try skipping to another one. I'll describe why in a minute.
OK! You should see at least one long string starting with rtmpdump. rtmpsrv did its job so close the browser tab and stop rtmpsrv with a Control+C signal. Ending rtmpsrv seems to be a problem. The cleaning up process seems to go nowhere at times but sending several Ctrl+C events usually produces a segfault. :/
Also delete the firewall redirect so that you can access the media server directly:

sudo iptables -t nat -D OUTPUT -p tcp --dport 1935 -j REDIRECT

If you're in luck taht is it and you only need to run the output of rtmpsrv, like this:

./rtmpdump -r "rtmpe://cp9950.edgefcs.net:1935/ondemand?ovpfv=1.1" -a "ondemand?ovpfv=1.1" -f "LNX 10,0,42,34" -W "http://media.mtvnservices.com/player/release/?v=4.2.2-8" -t "rtmpe://cp9950.edgefcs.net:1935/ondemand?ovpfv=1.1" -p "http://www.colbertnation.com/full-episodes/thu-january-14-2010-kathleen-sebelius" -C Z: -y "mp4:mtvnorigin/gsp.comedystor/com/colbert/season_06/episode_008/cr_06008_act1_768x432_1720.mp4" -o output.flv

The problem with Comedy Central shows are commercial breaks which cut the show into different separate chapters called acts (see the end of command above). This gives us a bit more work but it's so worth it avoiding the horrible full screen flash experience! Smooth full-screen playback has arrived, Randall! :)
This particular episode of The Colbert Report has 4 acts - all you need to do is change the actX in the command and the output filename must be different for each act too. Ending up with 4 differnet videos is kind of lame but manageable. If you know of a FLOSS stitching solution, please comment! (Avidemux doesn't wanna, PiTiVi wants to transcode, haven't tried LiVES yet).
SouthParStudios.com has a different gotcha - the first clip is the intro which is the same for all shows. Only the following acts are show specific. I managed to get a second command from rtmpsrv by seeking in the browser as mentioned above. Behold:

./rtmpsrv
RTMP Server v2.1c
(c) 2010 Andrej Stepanchuk, Howard Chu; license: GPL

Streaming on rtmp://0.0.0.0:1935
WARNING: Trying different position for client digest!


rtmpdump -r "rtmpe://cp40493.edgefcs.net:1935/ondemand?ovpfv=1.1" -a "ondemand?ovpfv=1.1" -f "LNX 10,0,42,34" -W "http://media.mtvnservices.com/player/release/?v=4.2.1" -t "rtmpe://cp40493.edgefcs.net:1935/ondemand?ovpfv=1.1" -p "http://www.southparkstudios.com/episodes/251890" -C Z: -y "mp4:mtvnorigin/gsp.comedystor/com/sp/extras/intros/PromoIntros/HDintroPromoXX_768x432_750.mp4" -o output.flv


rtmpdump -r "rtmpe://cp40493.edgefcs.net:1935/ondemand?ovpfv=1.1" -a "ondemand?ovpfv=1.1" -f "LNX 10,0,42,34" -W "http://media.mtvnservices.com/player/release/?v=4.2.1" -t "rtmpe://cp40493.edgefcs.net:1935/ondemand?ovpfv=1.1" -p "http://www.southparkstudios.com/episodes/251890" -y "mp4:mtvnorigin/gsp.comedystor/com/sp/season_13/1313/acts/sp_1313_act1_768x432_750.mp4" -o output.flv

^CCaught signal: 2, cleaning up, just a second...

And we have lift-off! See the act1 above? Incrementing this value will give you all the acts.
I am the first to admit this should be a total overkill for a good viewing experience but the current situation isn't really bearable. While fast modern multi-core desktop processors are capable of much smoother playback my aging Pentium M and a masses of netbooks struggle with this task. And of course (legally) downloading the shows and watching them offline is now possible! I think modern smartphones and media devices should have no problem with playback of these local files but as always YMMV.
Creators of rtmpdump, on behalf of the Linux masses suffering from stuttering full-screen playback with flash, I salute you!

Downloading RTMP Flash media with rtmpdump

2009-11-24T23:22:00.006+01:00

EDIT: These instructions are for verson 1.9 of rtmpdump! Newer versions make this process easier but also make some of this how-to obsolete. But I've modified the process below so that you can still use them for the legacy 1.9 version.

Today I attended a lecture/demo session on "hacking" various media sites like YouTube, Hulu, last.fm. Essentially it was a practical demonstration of the cat and mouse game hackers play with content pedlars on the internet. It was held at Cyberpipe, a nifty club in Ljubljana, Slovenia where I study.

One of the tools mentioned was rtmpdump which is apparently made by the same guys as the legendary mplayer, mencoder, and ffmpeg.

I decided to compile it myself from the most recent development version. If you're lazy and brave enough to trust compiled binaries and packages from third parties, go to LinuxCentre and install flvstreamer for your operating system. These programs were once one and so far they use the same commands. Read: they work the same.

These instrucitons work on Ubuntu 9.10 (karmic) but please be aware things change rapidly so the following might not work forever.

First of all install the tools needed for accessing the code and compilation.

sudo apt-get install build-essential gcc make subversion libssl0.9.8 libssl-dev libssl0.9.8

Now check out the latest code from the Subversion repository:

svn co svn://svn.mplayerhq.hu/rtmpdump/branches/1.x rtmpdump-1.9

If you prefer a stable version, run this instead:

wget http://rtmpdump.mplayerhq.hu/rtmpdump-1.9.tgz
tar xfz rtmpdump-1.9.tgz

I'm sure the impatient are looking forward to completing this ASAP so here's the final step in setting up rtmpdump on your computer:

cd rtmpdump-1.9
make linux

The program now lies in the same folder and you can only run it by telling your shell (like bash) exactly where it is. If you're in the same folder, running it is as simple as ./rtmpdump but ./ is only short for the current folder and it gets expanded to (in my case) /home/jasa/rtmpdump/rtmpdump.

Great! Now we need a media site which uses a Adobe's RTMP server. I think I won't get in trouble if I use VideoLectures.net as an example. :)

Lets say I'm interested in probability and statistics. And I want to watch these lectures offline, on a train or wherever.

Unfortunately this isn't a straightforward process and quite a bit of work is required. It is derived from a mailing list thread.

I need to see the source code of the web page above, not the rendered output of the web browser. In Firefox press Ctrl+U. Now I need some hard data to pass to rtmpdump. Search for a javascript section which launches the flash video player. For the statistics video this is the interesting section:

var flashvars = {
    
      streamer: "rtmp://oxy.videolectures.net/video",
      file: "2007/pascal/bootcamp07_vilanova/keller_mikaela/bootcamp07_keller_bss_01.flv",
    
    height: '288',
    autostart: "true",
    bufferlength: '5',
  
  
    image: "http://media.videolectures.net/play.png",
  
    id: "FlvPlayer" // last line, no colon ',' !
  };
  swfobject.embedSWF("http://media.videolectures.net/jw-player/player.swf", "video_embed", "384", "307", "9.0.0", "http://media.videolectures.net/swfobject/expressInstall.swf", flashvars, params, attributes);

Generally searching (Ctrl+F) for flashvars should get you near the required data. The line swfobject.embedSWF(...) causes the video player to load inside your browser and display the correct media. This media rests on a different server than the webpage and I will use rtmpdump to connect to that server instead of the flash player. Now Adobe has put some roadblocks into the process but lucky for me I have a flying car. :)

I'll just give you the final command now and explain later!

./rtmpdump -r rtmp://oxy.videolectures.net/video/ -y 2007/pascal/bootcamp07_vilanova/keller_mikaela/bootcamp07_keller_bss_01 -a video -s http://media.videolectures.net/jw-player/player.swf -w ffa4f0c469cfbe1f449ec42462e8c3ba16600f5a4b311980bb626893ca81f388 -x 53910 -o test.flv

The -r switch requires an argument which is the URL of the media server and is found in the variable streamer in the JavaScript source code above.
The -y switch needs the playpath and that is found in the JavaScript variable file, (minus the extension .flv or .mp4).
The -a switch is the name of the used player and usually automatically inferred from the URL. Defining it manually works by copying the part after the server name in the streamer.
The -s switch defines the flash video player which normally connects to the media server. In the example it is the first argument of the function embedSWF.

Here's where things get even more complicated. The media server wants some extra data about this player, specifically its sha256 hash-sum and size in bytes. So lets get them:

wget http://media.videolectures.net/jw-player/player.swf
sha256sum player.swf
ls -l player.swf

Supply the sha256sum to switch -w and the file size to the -x switch.

Anything else? Yeah, I need to specify where to save the video with -o.

Run the longest command ever and get yourself a beer (you've earned it)! :)

I admit that doing this for every video gets time consuming but unfortunately it is a procedure specific to every site (look at the mailing list thread link again to see different javascript). But for the same site there's very little extra work.

Hope you lasted this long, be seeing ya!

Automatic Adobe Flash on amd64

2009-10-26T19:39:00.005+01:00

Image via WikipediaUPDATE: Adobe has closed down their webpage which hosts the 64-bit version of Flash for Linux so the method described below no longer works! :(

Yes, apparently I have had a one month lurch. Things have been hectic as the new school year started and I couldn't really believe that almost a month has passed since my last effort!

The state of Adobe's Flash Player on Linux is a sad story, even more so for the next architecture PCs are migrating to (AMD64). The simple truth is that the modern web is unusable sans Flash for the common web surfer.
While I and others would like to see it displaced by open standards, such as the forthcoming HTML5 with the video and canvas elements, that is still far on the horizon and Adobe isn't exactly idling.

Okay, we Linux users are a pretty small part of the global pie, but we're arguably more technically savvy than the average computer drone. The 32-bit (x86) version of Flash player can easily be installed from Adobe's site but the amd64 flavour is still an exercise in bash. The software is still deemed to be of alpha quality so perhaps that is the reason why a more friendly install procedure isn't available. Luckily, Linux geeks like Romeo-Adrian Cioaba are friendly. How quickly a need is scratched now that Ubuntu has gained so much traction!

By modifying his script I have managed to make a version that will automatically download and install the latest version available from Adobe Labs. Be warned: I make no warranties that this procedure will work indefinitely or that it may not harm your computer. But the code is available for you to analyse so you may convince yourself of its correctness. Best way to learn is probably to execute each line by hand to see what happens. The manuals for the commands are your friends. :) Also, know that whenever you wish to install the version that comes with ubuntu, you need to run just the commands in the first 17 lines. Remember that, write it down somewhere, like in ~/FLASH_UPGRADE.txt

#!/bin/bash
# Script  created by
# Romeo-Adrian Cioaba romeo.cioaba@spotonearth.com
# Jaša Bartelj jasa.bartelj@gmail.com

echo "Stopping any Firefox that might be running."
sudo killall -9 firefox

echo "Removing any other flash plugin previously installed."
sudo apt-get remove -y --purge flashplugin-nonfree gnash gnash-common mozilla-pl
ugin-gnash swfdec-mozilla libflashsupport nspluginwrapper
sudo rm -f /usr/lib/mozilla/plugins/*flash*
sudo rm -f ~/.mozilla/plugins/*flash*
sudo rm -f /usr/lib/firefox/plugins/*flash*
sudo rm -f /usr/lib/firefox-addons/plugins/*flash*
sudo rm -rfd /usr/lib/nspluginwrapper

echo "Installing Flash Player 10."
cd /tmp
# EDIT: when releasing flash 10.1 adobe changed the site's address.
# I want permalinks! :(  --Jasa B.
#wget http://labs.adobe.com/downloads/flashplayer10.html
wget -q http://labs.adobe.com/downloads/flashplayer10_64bit.html
wget `cat flashplayer10_64bit.html | egrep -o "http:.*"|cut -d\" -f1|grep linux-x86_64.so.tar.gz`
ARCHIVE=`ls libflashplayer-*.linux-x86_64.so.tar.gz`
echo "Version is $ARCHIVE."
tar zxvf $ARCHIVE
sudo cp libflashplayer.so /usr/lib/mozilla/plugins/ 

echo "Linking the libraries so Firefox and apps built on XULRunner can find it."
sudo ln -sf /usr/lib/mozilla/plugins/libflashplayer.so /usr/lib/firefox-addons/plugins/
sudo ln -sf /usr/lib/mozilla/plugins/libflashplayer.so  /usr/lib/xulrunner-addons/plugins/

# now doing some cleaning up:
sudo rm -rf flashplayer10.html
sudo rm -rf libflashplayer.so
sudo rm -rf $ARCHIVE

The most complicated part is really just snipping the HTML for the URL of the archive. Please, enjoy and comment.

The mighty PDF

2009-09-25T22:38:00.005+02:00

Image via Wikipedia

PDF documents are a perfect export target in many applications because they're portable and look the same everywhere. So perfect in fact that there are virtual printers (like PDFCreator for Windows and integrated ones in Linux/*BSDs) which transform any printable file to a PDF.

But they are difficult when you want to change them or combine them.

I've had a folder sitting on my otherwise unburdened desktop for a year. It contained a bunch of scanned images which I wanted to aggregate into a nice and shiny single PDF file. Yeah. My google-fu had some slow reflexes this time and the first tries with FPDF, a PHP module, were unsuccessful. The images were in the correct format their manual demanded but I could only helplessly bang my head away as the scripts failed with errors of unsupported formats.

Sometimes my determination falters at moments like this and I put off trying to resolve the problem. But I had to push!

My google-fu got some well-deserved rest and found what I was looking for - a solution using GhostScript. By the way - this solution works for me on Ubuntu Linux but elsewhere YMMV.

Here's a script I ended up using (warning: It overwrites the original images.):

#!/bin/bash
for filename in *
do
    file -bi "$filename" | grep -q "image"
    if [ $? -eq 0 ]
    then
        echo "  Converting $filename"
        #Reduce image size by half 
        convert $filename -resize 50% $filename
        #Convert image to single-page PDF 
        convert $filename ${filename%.*}.pdf
    fi
done
gs -dNOPAUSE -sDEVICE=pdfwrite -sOUTPUTFILE=combined.pdf -dBATCH *.pdf

The bash script relies on ImageMagick (the convert command) and GhostScript (the gs command). Best thing is to copy it to a file in the folder with the images destined to comprise the PDF, marking the file as executable and running it! The final PDF will contain all the single-paged PDFs sorted in lexical order.

For the intricacies of GhostScript seen above I will let the devil speak for itself:

$ gs -h
GPL Ghostscript 8.64 (2009-02-03)
Copyright (C) 2009 Artifex Software, Inc.  All rights reserved.
Usage: gs [switches] [file1.ps file2.ps ...]
Most frequently used switches: (you can use # in place of =)
 -dNOPAUSE           no pause after page   | -q       `quiet', fewer messages
 -g<width>x<height>  page size in pixels   | -r<res>  pixels/inch resolution
 -sDEVICE=<devname>  select device         | -dBATCH  exit after last file
 -sOutputFile=<file> select output file: - for stdout, |command for pipe,
                                         embed %d or %ld for page #
Input formats: PostScript PostScriptLevel1 PostScriptLevel2 PostScriptLevel3 PDF

Vanity

2009-09-25T20:52:00.005+02:00

Seeing my first post published was accompanied by a healthy dose of feeling accomplishment. But soon I found that the script code left something to be desired.

Prettiness!

After scouring the now humongous blogosphere I found people to relate to. But that's usually easy on the Internet. So after applying google-fu to the problem I found SyntaxHighlighter! Using it on Blogger requires a few simple steps which others have explained but I always try to do it my way so I learn something and memorize the process.

The gist of it is adding more javascript to your blog's HTML template. I appended my <head> section with this:

<link href='http://alexgorbatchev.com/pub/sh/current/styles/shCore.css' rel='stylesheet' type='text/css'/> 
<link href='http://alexgorbatchev.com/pub/sh/current/styles/shThemeDefault.css' rel='stylesheet' type='text/css'/> 
<script language='javascript' src='http://alexgorbatchev.com/pub/sh/current/scripts/shCore.js'></script> 
<script language='javascript' src='http://alexgorbatchev.com/pub/sh/current/scripts/shBrushBash.js'></script> 
<script language='javascript' src='http://alexgorbatchev.com/pub/sh/current/scripts/shBrushJava.js'></script> 
<script language='javascript' src='http://alexgorbatchev.com/pub/sh/current/scripts/shBrushPlain.js'></script> 
<script language='javascript' src='http://alexgorbatchev.com/pub/sh/current/scripts/shBrushXml.js'></script> 
<script type='text/javascript'> 
 SyntaxHighlighter.config.bloggerMode = true;
 SyntaxHighlighter.all();
</script>

The javascripts parse any <pre> blocks in your blog post and display the code in a nice way. The default style clashes a bit with my blog template but I might fix that. What good is dabbling in vanity if it doesn't achieve its purpose? :)

So let me try Hello World again:

#!/bin/bash
echo 'Hello, world!'
if [ $? -eq 0 ]
then
    ./comment > /dev/null
fi

First post

2009-09-23T23:14:00.001+02:00

Image by oskay via Flickr

Time for the first entry.

I could never come up with a good plan on what to write about or perhaps I just didn't want to write strictly about one subject. Oh how the times have changed.

Here I am, typing away precious moments trying to convey a message of what this blog will be about in an ambiguous natural language.

Let me give you a hint:

#!/bin/bash

echo 'Hello, world!'

if [ $? -eq 0 ]

then 

    ./comment

fi

Hopefully that didn't drive you away with WARP 9.9! :) Why don't you subscribe to the feed and be notified of changes automatically.

Constant Curiosity

Rooting Android

Related articles by Zemanta

On Android communities

Related articles by Zemanta

Lets get Flash squared away

Related articles by Zemanta

System monitoring beyond GNOME's System Monitor

Tweaking Ubuntu's GDM and nautilus

Finding old Ubuntu ISOs

Revisiting rtmpdump

Downloading RTMP Flash media with rtmpdump

Automatic Adobe Flash on amd64

The mighty PDF

Vanity

First post