Month of July 2004

To pacify the end-of-month bug, here is another placeholder. Without going into any particularly interesting details, this one says that

• There's enough of a mess in the SBCL AMD64 backend regarding floating point (mostly to do with object sizes and alignment) that it'd be quicker and more reliable to actually go over it carefully and check it all at once instead of fixing bugs as they happen. That being so, this would be a great opportunity to rewrite it to use XMM registers and the "128 bit media instructions" instead of the x87. AMD claims that they run faster,there are 16 random-access registers instead of the slightly weird x87 stack thing, and it has perfectly normal IEEE single and double precision (none of this 80 bit nonsense) so we might even get less freaky results too.

• I've decided to enter the London Inline Marathon in September, so am now skating to work three days a week (or so; depends on weather etc). There was a tube (metro train system) strike here in London yesterday; so I ended up skating to work, across town to the Londonskate (a and less confusingly k.a the Wednesday Night Skate), did the skate itself, and then, finding that the tube was still closed after the pub shut, skated back across town to Liverpool St station from where I got a surface rail train to Stratford (the one in East London, not the one where Shakespeare lived) and skated home from there. Probably around 20-22 miles in all, I guess, and must admit that today I am just a little tired.

A review of my new cheap hardware MP3 player will follow (summary for the impatient: it was cheap). Right now I'm merely noting that it provided an impetus to clear some disk space on my laptop.

:; pwd
/home/dan/.emacs.d/auto-save-list
:; du .
4238340 .

Does that seem like a lot?

Announcing BORDEAUX-MP, a proposed standard interface for MP in Lisp. (No, I haven't ever tried this before). Discussion on the new threads-standard-discuss list.

Which list is one of a matching set of three; the other two being for discussion of the MOP and for extensible (eg Grey, simple-) streams. There's not been a lot of blogging done from LSM that I can see (at least, not in our own private little Lisp world) and I'm not going to start now (I will later), but Christophe Rhodes is compiling "proceedings" of yesterday's lightning talks, so please look out for those.

Random bits of post-LSM notes, many inconsequential.

• The weather was alternately bright warm sunshine and torrential rain. The alternation was on a period of two hours or so: at least, it was quite rare that the ground actually dried out

• That notwithstanding, I think I've got at least the basics of T-stopping by dragging the right foot, and of 180 degree anticlockwise spins. (Previously I've only been a ble to do left foot and/or clockwise, so it's nice to be moving back towards symmetry)

• There is a brand-new tram system in Bordeaux which runs out as far as the University. On a the good day, at least. On all the other days, getting into town involved a bus replacement service for at least part of the journey. Still, apparently it was only launched a week ago, so this is probably all just teething trouble.

• The food was pretty excellent

• Rough order of events :
  Tuesday: formal presentations
  Wednesday: de facto standards process discussion, and debate about how to produce a CL Emacs
  Thursday: lightning presentations
  Friday: impromptu GPG keysigning session, creation and CVS upload of Portable Hemlock to common-lisp.net

I have some thoughts about Protocol-Oriented Programming (one of the topics touched on in the formal presentation on Tuesday) and Smalltalk (the other people in our track on Tuesday), which will wait for another diary entry. There's also much more to say about the protostandards stuff, which is waiting on the others involved to reply to their email, and about lightning presentations, but Christophe Rhodes is collating that not me.

Oh, yeah. And I have an (entirely untested) first cut at a BORDEAUX-MP implementation for SBCL, which will shortly be committed to CVS

I'm finding more SBCL threading bugs: so far, the count is three bugs and two fixes.

Edi Weitz pointed out recently that his cl-ppcre libarry comes with a test suite whose threading tests fail when run under SBCL, so I've been working on that. So far

1. Bad header file ordering in the runtime was causing the non-threaded versions of spinlock get/release inline functions to be compiled in some places even on a threaded build. (Fixed)

2. SIGCHLD will call destroy_thread, which is fair enough, but not if you're holding all_threads_lock already because it wants to acquire it. (Fixed)

3. After beating on the threading for long enough it's still somehow possible to get into a state where GC wants to stop a thread so that it can collect the heap, but the thread has disappeared. Not fixed

Thread bug 3 identified. Background: gc_stop_the_world is called from within a WITHOUT-INTERRUPTS form. WITHOUT-INTERRUPTS doesn't actually mask signals at the kernel level, but does set a flag that the C-level signal handlers check when deciding whether to defer a signal. SIG_THREAD_EXIT (the RT signal that the parent thread is sent when a child thread dies, a la SIGCHLD for traditional processes) is on the deferrable list (because it calls Lisp code), so if a thread dies between the start of WITHOUT-INTERRUPTS and the world getting stopped, there's nothing available to wait() for it. Thus it becomes a zombie and will never react to requests to suspend itself for GC.

Alternate strategy: make thread_exit_handler just wait() and then set thread->state=STATE_DEAD, so the GC can see the thread's no more, then do the Lisp-level stuff and destroy_thread at some later time. Now we can remove SIG_THREAD_EXIT from the deferrable list. Pretty arbitrarily, we've decided that at "after a GC" would be a reasonable later time; so now if you disable GC you have also disabled dead thread collection.

I've had this laptop (Toshiba Portégé 3440CT) for something over three years now, and I'm still pretty happy with it, mostly. Yes, it's old and underpowered (500MHz mobile PIII), but it weighs practically nothing, it has a trackpoint thing, three mouse buttons (actually, two buttons and two scrollwheel simulating buttons but I can remap one of those as button 3 and it mostly works apart from slightly evil autorepeating behaviour), and it's old enough to use APM not ACPI. Which during most of its early life was a distinct advantage on a machine that runs Linux.

It does have a few oddities, though; readers may recall its habit of saving power by randomly frobbing CPU speed and not telling the OS. This is not in itself a problem now I've weaned Linux away from using the TSC as a time source, and probably also saving some power, but the manner of its doing was starting to annoy me, in that it will decide apparently at random that it should only speed up as far as 266MHz. Usually it can be persuaded to speed up again by removing and reapplying mains power, but after a while (and lately it seems to be quite a short while) it's forgotten again. This is tedious when recompiling SBCL.

Today I found that (a) the version of toshset that I already have installed has an option for cpu speed, (b) it was set to 'low', (c) setting it to 'high' does actually make a difference, and to cap it all (d) the cpu speed still varies continuously according to system load, so as I mostly use well-behaved apps that only eat cpu when they're actually doing things (exception made for the ISC dhcp client which from time to time likes to spin on CPU), I still get the battery savings. Happy Bunnies R Us.

Bcc: dan
To: sbcl-devel <at> lists.sourceforge.net
Subject: Re: [Sbcl-devel] Status of native thread support?

Edi Weitz writes:

> FWIW, I just tried the threaded CL-PPCRE tests again (I had an email
> exchange with Cristophe about this some months ago) and they still
> fail with SBCL 0.8.12 on Debian unstable. (Kernel is 2.6.6.) They run
> fine with Lispworks' green threads and they used to run fine with
> Scieneer's native threads when I had access to their 1.1 trial
> version.

The CL-PPCRE thread tests seem to work quite nicely as a test of
thread creation/exiting, which had a number of exciting race
conditions.  You may (and everyone else using or wanting to use
threaded SBCL may also) want to check out 0.8.12.42 which fixes at
least three of these and seems to run the threaded tests in cl-ppcre
0.7.9 quite happily.

This also fixes the thundering herd at thread gc rather more cleanly
than inserting sched_yields around the place: thanks to Juho Snellman
and Christophe Rhodes on #lisp for pointing this stuff out.

So, unless Bill advises to the contrary, SBCL will shortly be going
into code-mostly-freeze for the 0.8.13 release.  Therefore it would be
a really good thing if people would test their threaded stuff for
regressions /now/ instead of leaving it a week: then any bugs can be
fixed in time for this month instead of next.

(Incidentally, your threaded-scan function seems to be doing incf of
a shared variable COUNTER without the aid of a lock; this is probably
not guaranteed to work everywhere.  I think it happens to work in SBCL
(if so, mostly because it happens to work on x86), but you probably
want to have some kind of atomic-incf there, or protect it with a
mutex)


-dan

 
"please make sure that the person is your friend before you confirm"

I'm playing with last.fm. A few months ago I got annoyed at some of the wailing and teeth-gnashing that was happening on uk.music.guitar and posted an article saying approximately "if the `channel' sucks so much anyway, stop complaining that you can't break into it and start routing around it". Then I completely omitted to do any of the things I'd suggested that people start doing. But it seems like last.fm is doing at least some of them - at least the technical bits - even if so far it's mostly just playing stuff at me that I've already heard of. First impressions: really rather cool, but minor downsides: does come unsynced every so often (the stream is not playing what the popup window says it's playing) and involves way more screen space than I think it needs to use.

I registered as 'telent', but I don't think it knows enough about me yet for it to be worth anyone's time going to find me there right now.

Earlier this week I hacked up some inline allocation routines for SBCL. Each thread already has its own region, so Lisp code should be able to allocate within that region just by incrementing the (thread-local) free pointer, rather than calling into C for every 8 bytes. And now it can.

(For a long time pre-threading it used to be able to do this, but it was using a global variable to store the free pointer which was obviously not going to fly in an MP environment, so I ripped out that code when initially doing threads)

After a minor misapprehension regarding whether it would make any useful difference (summary: 5% to 15% speedup on some consy benchmarks), I've merged this change (or reinstation, if you prefer) into 0.8.13.11