Recent Entries

Suppose you had a broadband account with a static IP address (or small network) and an NNTP service, and that you wanted to read news from places other than the broadband network. Suppose further that your newsreader of choice is Gnus.

The intersection of all these assumptions is that you'd probably be me. In which case you already have incoming ssh access to the machine(s) on the broadband and all you need do is add the emacs magic to connect back from wherever in The Cloud(sic) you are through your home net to your news service

(defun nntp-open-via-ssh-stream (buffer)
  "Open a nntp connection by ssh to coruskate then then nc to zen"
  (let ((command '("ssh" "me.homeip.example.com" "nc" "news.myisp.example.com" "119"))
	proc)
    (setq proc (apply 'start-process "nntpd" buffer command))
    (save-excursion
      (set-buffer buffer)
      (nntp-wait-for-string "^\r*20[01]")
      (beginning-of-line)
      (delete-region (point-min) (point))
      proc)))


(setq gnus-select-method '(nntp "news"
				(nntp-open-connection-function 
				 nntp-open-via-ssh-stream)))

Zen Internet come highly recommended as usual

The trouble with doing long-term reviews of mobile phones is that by the time you've had them anything approaching a long time, they've been superseded already and are probably about to be discontinued.

But anyway. I've owned this thing since shortly after they came out and having recently finally peeled off the increasingly tatty piece of cellophane it shipped with to protect the screen, I reckon it's time for an update.

The hardware

I won't go into detail about the hardware, as the software aspects are going to be more interesting long-term. The summary is "good but not excellent": it has not bent, broken, warped, wefted, spindled, folded or self-mutilated, but on the other hand its styling is still closer to a brick than to an Iphone. Apart from the battery, which deserves its own section, the single stupidest hardware feature is the plastic plug which is supposed to cover the mini-usb charging/headphone socket. Second stupidest is probably the socket itself: although mini-usb is the only sensible design for power and data connections, the adaptor necessary to plug normal audio headphones in is wickedly fragile and can be had only from chinese ebay sellers

I said the battery deserves its own section. Here it comes:

Battery life is crap. Recharge every night, because it will not last a second day

Message ends. I know it's an incredibly versatile device that's more like an ultraportable computer than a mobile phone as we know them, and also that by disabling all the interesting features I could probably get it to last two days, maybe, at a push, but that would kind of negate the point of having paid for the extra features in the first place, now wouldn't it?

The software

It's now six months after launch and I have to concede that despite having upgraded to the "Cupcake" OS version 1.5, the software still feels unfinished in some areas. The Google contact/calendar integration is a win, but it still lacks certain things that an ordinary phone can do without problems, like selecting a phonebook entry and beaming it or texting it to a friend. And some of the things it does do give the strong impression of never having been tested on real people. The Mail app at launch was famously bad enough that a third-party group of hackers forked it just to get it to a usable state (their fork, K-9, has since come on in leaps and bounds and I strongly recommend it for non-Gmail IMAP users). My particular ire is reserved for the calendar, which defaults to presenting a view of the month with coloured blobs to indicate when things are happening but no clue as to what those things are. And the camera - I don't know whether to blame hardware or software or both, but I could probably paint the scene in less time than it takes to focus, and with a better result in any setting less well-lit than an operating theatre.

This sounds pretty negative and it's mostly not. It's just that when you do get smacked by the 10% stupidity sections after the 90% (in most cases) that is there has lulled you into quiet babylike stuporific contentment, it smarts twice as much. My recommended apps, for the record:

• K-9 (unless you use Gmail)
• ConnectBot: an ssh client
• Twidroid: still the best twitter client I've seen
• Greed: RSS reader syncs with Google Reader, so you can discover feeds and manage subscriptions on your desktop instead of having to type the URLs in on the device keyboard
• Ringdroid: MP3 editor and ring-tone creator

Development

The phone this replaces is a Sony-Ericcson M600. The PDA it replaces is a Palm TX. The first thing to say about it, then, is that for openness it's light-years ahead of either. Reasonably well documented, easy to get into (no need to join developer programmes or send your apps away to be certified) and there's a linux-hosted toolchain. I haven't explored it extensively but I did without too much trouble write half an app that logged gpx tracks (but then My Tracks came along and obsoleted my effort), and poked around inside K-9 a bit. It's Java (practically speaking: legally it's merely Java-like) which language I grant you nobody ever expanded their minds by learning, but on the other hand has never caused anyone's brain to leak out through their ears in quite the same way as C++ or PL/SQL can.

That sounds disturbingly - albeit quite faintly - like praise, so I have to add that the second thing to say is that they've tragically missed the point: it's still an ocean removed from hackable in any useful sense. All the support is there to write your own apps is no use to you if you merely want to modify someone else's app, and even if you are starting with one of the open-source apps you still have the burden of maintaining your own private fork just to support your private hacks.

It's not all bad: the use of Intents should make it possible to "deep-link" into other people's apps (see Oilcan for example), but it's still more work than, say, Emacs hacking or Amiga ReXX ever was.

Hacking

Underneath it all, as I'm sure everyone who cares already knows, is an ARM chip running a slightly hacked Linux 2.6. If you install the Terminal Emulator app you can get a shell prompt: if you follow instructions on the xda-developers forum you can get root access too. I did: while I'm not sure it's essential to normal use, it does open up a couple of bits that you can't do on the regular phone such as wifi tethering and a task manager to kill apps you're not using when the thing starts feeling sluggish. (In theory this is unnecessary, in practice I'm not convinced either way but at the very least it's a great placebo effect)

But the bottom line is that it's fun to play with, which is probably the major thing, and seems to be quite low risk: at least, going by the comparative rarity of "I bricked my phone" posts on the xda-dev forum, most seem to have managed it successfully. You can even install an entire Debian system that lives alongside Android, though I still can't think of a better reason to do so than bragging rights.

Until very recently I ran this blog site, and a couple of others (test sites for work, that kind of thing) in a Xen virtual host on the pc sitting on my desk. While it worked ok for that purpose, it really didn't coexist nicely with much else: the Xen kernels available in debian have a showstopper bug for running X11, are missing modules for e.g. cpu temperature monitoring, etc. And the process to build ones own is not usefully documented. (I apt-get sourced my kernel image, and it downloaded a completely different kernel version). I don't think the stuff is unusually buggy, it probably just doesn't have enough eyeballs on it to find all the weird interactions with other components or to patch them when they do turn up.

So when my four year old Athlon motherboard died recently, the opportunity to rejoin the mainstream and start using kvm was the silver lining in the "ew, hardware" cloud.

A hard ware is gonna fall

I decided to go Intel this time despite my AMD-supporting underdog instincts, chiefly because I figured that (a) every Intel cpu on the planet should have VT support by now, (b) onboard graphics are well-supported in xorg, (c) everything I read about the E5300 CPU said it had very good price/performance ratio and overclocking it someday sounded like a fun project. So basically I bought that and the first G45 board I found that had onboard graphics (this is the distinction between G45 and P45 which doesn't), three SATA ports and was made by someone I'd heard of. It's the Asus P5Q-EM, and it apparently also is a good one for overclocking.

After putting it all together, I found that it didn't work. Important lesson here: for 'market segmentation' reasons - and I was certainly pretty cut up about it, though that may be not the kind of segmentation they mean - Intel only implement the necessary VT bits on CPUs that cost north of £100, not on the cheap and lovely E5x00 series even though they have the (arguably equally "enterprise niche") 64 bit support. So, gah. So I ended up with an E8200 as well, and a spare cpu+fan that is probably going on Ebay soon.

This walk is made for booting

KVM now runs, which is a start. Note that you need the kvm module and the kvm-intel module, or it will complain that you have no CPU support. You may also have to fiddle with your bios settings, although I didn't.

To convert a Xen disk image into an image that kvm understands is still a bit fiddly: in brief, xen images are filesystem (= partition) images, whereas kvm wants entire disks and they need to be bootable (i.e. contain grub and a kernel). Here's how I did it, prefaced by the obligatory "it worked for me, but it's not my fault if it doesn't for you. Back up first" caveat

Obligatory caveat: it worked for me, but it's not my fault if it doesn't for you. Back up first.

1) Identify your disk image. On a Debian box it will be in some place like /usr/local/lib/xen/domains/vitrual.4a.telent.net/disk.img

 # cd /usr/local/lib/xen/domains/no.stargreen.com
 # file disk.img 
 disk.img: Linux rev 1.0 ext3 filesystem data, UUID=e6143cef-c3ed-4c21-94d9-32d4ba186910 (large files)
 # ls l disk.img 
 -rw-r-r-- 1 root staff 4294967296 2009-06-03 14:33 disk.img

 # dd if=/dev/zero of=mbr bs=8225280 count=1
 # cat mbr disk.img mbr | cp --sparse=always /dev/stdin qemudisk.raw

And the additional space on the end? In my first few attempts I ran into trouble with e2fsck complaining that the filesystem was bigger than the disk image. This is, of course, impossible, and I imagine was due to some kind of rounding error, but padding it out this way is easier than investigating.

The sparse flag to cp might save us a bit of disk space if we have any great big expanses of zero in the image. Mine didn't, as it turned out, but YMMV.

Next up, partitioning. sfdisk accepts input on stdin and uses it to destroy your data, which makes it in some ways the perfect Unix-philosophy tool. GNU Parted will never give you this kind of buzz, so be very careful here.

 
 # echo '1' | sfdisk qemudisk.raw  # DO NOT MISTYPE THIS
 # sfdisk -l qemudisk.raw 
 Disk qemudisk.raw: cannot get geometry

 Disk qemudisk.raw: 524 cylinders, 255 heads, 63 sectors/track
 Units = cylinders of 8225280 bytes, blocks of 1024 bytes, counting from 0

   Device Boot Start     End   #cyls    #blocks   Id  System
 qemudisk.raw1          1     523     523    4200997+  83  Linux
 qemudisk.raw2          0       -       0          0    0  Empty
 qemudisk.raw3          0       -       0          0    0  Empty
 qemudisk.raw4          0       -       0          0    0  Empty

At this point if you have a rescue cd image or something you could start kvm and have a poke around.

 # kvm -hda /usr/local/lib/xen/domains/no.stargreen.com/qemudisk.raw
   -cdrom trinity-rescue-kit.3.1-build-210.iso -boot d

 # mount qemudisk.raw /mnt -t ext2 -o loop,offset=8225280
 # chroot /mnt apt-get update
 # chroot /mnt apt-get linux-image-2.6.26-2-amd64 grub  
 # mkdir -p /mnt/boot/grub
 # cp /mnt/usr/lib/grub/x86_64-pc/* /mnt/boot/grub/
 # cat | grub --device-map=/dev/null
 device (hd0) qemudisk.raw 
 root (hd0,0)
 setup (hd0)
 # cat > /mnt/boot/grub/menu.lst
 title           Debian GNU/Linux, kernel 2.6.26-2-amd64
 root            (hd0,0)
 kernel          /boot/vmlinuz-2.6.26-2-amd64 root=/dev/hda1 
 initrd          /boot/initrd.img-2.6.26-2-amd64

That's basically it as far as getting a bootable image is concerned. You can start kvm with this image: I would probably suggest running update-grub inside it to replace this minimal config with all the Debian grub goodness

Take me to the bridge: networking

The kvm networking default is 'user' networking, which is in summary dead convenient if all you want to do in the virtual host is run clients, but a little bit slow and a little bit weird. If you want to run servers on your kvm instance, you will probably instead want to bridge its network with your real host so the outside world can see it.

I previously had a statically addressed eth0, now I have a statically addressed br0 which bridges that and a couple of tap devices (you will need one per kvm virtual host). The necessary debs are bridge-utils and vde2, and your /etc/network/interfaces stanza, on the real host, is something like this

 # The primary network interface
iface br0 inet static
    address 192.168.1.2
    netmask 255.255.255.0
    gateway 192.168.1.1
    pre-up /usr/bin/vde_tunctl -u www-data -t tap0
    pre-up /usr/bin/vde_tunctl -u www-data -t tap1
    pre-up ifconfig tap0 up
    pre-up ifconfig tap1 up
    bridge_ports all tap0 tap1
    post-down ifconfig tap0 down
    post-down ifconfig tap1 down
    post-down tunctl -d tap0 tap1

and then I start kvm with

 # kvm -hda qemudisk.raw -net nic -net tap,ifname=tap0,script=no

And there it is. Write yourself some fancy startup scripts: I invoke kvm with -daemonize and -vnc options so that it doesn't need to open a window, but there are plenty of different ways to do that. You will probably want to play with other options too: for example to change the kvm instances ram allocations or to give them a second disk image suitable for swapping on. If you have lots of kvm instances all running at once you should maybe also look into VDE networking rather than the tun/tap devices, but I haven't spent the time to work that out yet.

If you go googling to solve your own problems: you probably know this already ,but it bears repeating: KVM is in some sense a fork/branch/derivative of QEMU, and the two systems have largely or maybe even entirely common code in this area. So a lot of what you see about qemu booting is directly applicable to kvm too.

If you found this useful, feel free to get in touch. If I've left any step out, tell me that too and I'll edit it.

After a three-year hiatus I've decided to separate my tech and skating blogs again, in the hope that more people will enjoy reading one or the other than the vanishingly small intersection of people interested in both.

MORE TOYS.

I bought a Palm TX the other day. This may actually be more connected with skating than with Lisp: basically, I was looking for something that I could use to quell uprisings on web servers and suchlike even when I was away at a race or in an airport or something, and didn't want to lug my laptop around with me. It's quite a small laptop, but it's still space that I could be using for another change of clothes.

So far I mostly like it, but I'm still learning my way around. Builtin Wi-Fi worked out of the box on my home network: builtin bluetooth was eventually persuaded to talk to my work computer using a disgustingly Heath Robinson combination of dund and pppd, and to my mobile phone (though I had to mess around for a while there as there's not a configuration for my phone and mobile network). There's a workable ssh client (pssh) which in combination with the Universal Wireless Keyboard ought to fulfill the stated purpose of emergency access to servers - at least, will when I can persuade pssh to recognise the Control key on the keyboard, which isn't yet happening. It does at least now recognise the majority of the other keys, whch was not the case until I found the updated v1.10 keyboard driver

Other things I would like to be able to do with it now I've got it: (1) organise my race calendar and remind me to do things like book flights; (2) email, sensibly (VersaMail seems to be a bit crap); (3) control the xine on my desktop, preferably over the xine TCP interface. (Yes, I know about OmniRemote, but my desktop doesn't have IR, so that's a bit of a non-starter). The screenshot on the right is the result of my first tentative step towards learning how to program the thing, and comes to you from my mobile phone via bluetooth to the palm, where I attached it to a VersaMail message, then sent it to my work address which cause it to go by smtp-over-(ppp-over-bluetooth to my desktop) to gmail, smtp back to the office mailserver, pop3 to my desktop again, then scp to the machine that hosts this blog. Marvellous.

I was going to tell you I found a German dictionary for it too, but I suppose my use for that is more about the skating than the Lisp as well. Oh well. More about LispMe to follow as I get to grips with it ...