Getting file position using /proc

As I wrote in my post of 2009, "Often there is a need to check what's some process doing right now". That time I discovered this simple way of checking which file a process(e.g. grep) was using.

Another trick of this kind is to get a file position. This is a very common scenario. For example, you want to know the progress of unpacking of a really huge .gz file. You cannot just estimate this by analyzing the size of the data already unpacked.

Getting file position is as easy as getting a name of an open file. Thanks to Miklos Szeredi starting from Linux-2.6.22 this info can be obtained from /proc/PID/fdinfo/FD:

$ python
>>> f=open('/home/turist/work/big.txt')
>>> f.seek(1234)

(in another terminal, 3580 is python's PID)
$ ls -al /proc/3580/fd/
total 0
dr-x------ 2 turist turist  0 2011-09-21 14:40 .
dr-xr-xr-x 7 turist turist  0 2011-09-21 14:40 ..
lrwx------ 1 turist turist 64 2011-09-21 14:40 0 -> /dev/pts/1
lrwx------ 1 turist turist 64 2011-09-21 14:40 1 -> /dev/pts/1
lrwx------ 1 turist turist 64 2011-09-21 14:40 2 -> /dev/pts/1
lr-x------ 1 turist turist 64 2011-09-21 14:40 3 -> /home/yevgen/work/big.txt

(so 3 is our file descriptor, we will use it now)

$ cat /proc/3580/fdinfo/3
pos:	1234
flags:	0100000

Easy!

PR 1.2 - major disappointment of the year

Just updated my two N900's to PR 1.2. Well...

Pure disappointment. None of the bugs I cared about was fixed even though it's set as fixed on bugzilla:

Bug 7211(Unable to play media. Media format not supported" for recently recorded video) - not fixed.
Bug 6823(media player won't play any video files now (.avi) divx / xvid) - not fixed.

It fails even to play its own built-in "Nokia N900.avi". Since December there's been no way to play video files on this so called "Mobile Computer"!

They added Skype video calls, but it's like slideshow.

Calling with Skype? "General error" after 1 minute and no even a sign of your call on the other side. After call pulseaudio uses 20% of CPU. I found that because both of devices were too hot for no reason.

Portrait browsing? Really?! Check the screenshot. Where is the address bar? Where is the window switching button? Exit fullscreen mode button? Is this portrait browsing or portrait viewing only?

I'm not sure whether I should wait for MeeGo or just give up and buy Android next time.

Maemo DDP: a fly in the ointment

Last week I got my DDP's N900 and since I still have my Summit's device I haven't played with it much. I wish Marcin Juszkiewicz wrote his DDP device checklist earlier. I got this nasty bug 6823 "media player won't play any video files now (.avi) divx / xvid". I tried all files, even the default "Nokia N900" and "9". Since Mplayer plays all video files, this is a software fault, not hardware issue. But still it's frustrating and I'm waiting for any bug activity.

So my point is, all people who got their devices, please check for video playback in media player. And vote for the bug if necessary.

Making your hard drive faster

Some manufacturers worry too much about their customers' comfort. For example, Apple disabled multitasking on their iPhones because too many running processes could slow down the system's responsiveness and everyone would say "iPhone is slow" rather than "you have too much stuff running, close something". In this case IMHO Apple worries more about themselves, but whatever.

In my case it's Dell/WD who worried too much about my comfort. My workhorse is Dell Precision T5400 with 2 quad-core Xeons. While running anything on it in 8 processes is fun, disk is always a bottleneck.

Let me introduce AAM(wikipedia: Automatic acoustic management). This option is used for decreasing the speed of disk rotation and head positioning. And for some people it's really necessary. But at work I have other people who, combined, produce much more noise. Thus, I don't need an additional care.

For my Linux PC 'hdparm' is a trusted tool:

# hdparm
...

-M  get/set acoustic management (0-254, 128: quiet, 254: fast)
...

Checking current setting:

# hdparm -M /dev/sda

/dev/sda:
acoustic      = 128 (128=quiet ... 254=fast)

And changed to max performance:

# hdparm -M 254 /dev/sda

Useful tool for testing the average random access time - seeker.c:

Benchmark results:

AAM=128:
./seeker /dev/sda
Results: 58 seeks/second, 17.18 ms random access time


AAM=254:
./seeker /dev/sda
Results: 76 seeks/second, 13.05 ms random access time

I think I already feel this 30% improvement. Or maybe it's the placebo effect :)

First steps with Maemo 5 SDK - Stardict

It looks like there is a lot of hype about new N900. I don't think I'll be able to afford one for myself but there are still people who will need a dictionary program while they are offline.

4 hours spent to get the first working binary. And there is a huge amount of work to do:

• read Maemo 5 HIG doc
• change the layout of potentially everything as all the buttons are HUGE now and previous useful area got shrinked.
• work around the bug "Xephyr crashes on clicking any editable area". Hate to have compiled/make_installed thing in my system.
  

And, of course, a screenshot:

Look inside the process using /proc

That's what might be useful even for web-developers, not just sysadmins and geeky geeks.

Often there is a need to check what's some process doing right now. `top` is of course showing us 100% of CPU consumption but you need to have some sort of progress to decide if you can wait or you'll just kill the task and do it in a different way.

Let's take `grep` as an example of some task that may take hours on a big set of files:

user@host [/tmp/files] $ grep -e "$PATTERN" *.dat

So to use magic you don't need patch grep or to have knowledge of how to debug Linux kernel. I use `/proc` for this task:

1. Find PID of the grep process, like this "ps axuww | grep grep" :)
   
2. Exec "ls -al /proc/$PID/fd/"
3. You'll get directory list as an output. One of these file descriptors the a process is grep's current file it's processing right now.
   

lrwx------ 1 user 64 2009-07-05 04:12 0 -> /dev/pts/2
lrwx------ 1 user 64 2009-07-05 04:12 1 -> /dev/pts/2
lrwx------ 1 user 64 2009-07-05 04:12 2 -> /dev/pts/2
lr-x------ 1 user 64 2009-07-05 04:12 3 -> /tmp/files/2009-07-01_4534545435.dat

Voila!

For progress calculation you  can use "ls *.dat > list.txt" and then find which line you on right now.

Python surprises - scope of variables

Yesterday while debugging one place in some function I found one "surprise" in Python - language that I thought was designed to avoid surprises as much as possible(Principle of least astonishment).

if 1 == 1:
    print dir()
    found = True
    print dir()

print dir()
print "found:", found

Output:

['__builtins__', '__doc__', '__file__', '__name__', '__package__']
['__builtins__', '__doc__', '__file__', '__name__', '__package__', 'found']
['__builtins__', '__doc__', '__file__', '__name__', '__package__', 'found']
found: True

As you would have guessed coming from C++, here we had variable used out of its scope. But in Python variables live in special dictionaries - one is global where global variables and other stuff are stored and another one is local, for keeping local symbols inside the function. So all variables you create inside "for" loop or "if" scope will be stored in that local dictionary and will only be removed from there when Python's garbage collector decides.

That surprise cost me 30 minutes of wondering why some polygons on a map suddenly went nuts and lost their shapes :)