If you're reading this at about the time I wrote it, some of the

Thu, 31 Oct 2002 17:44:26 +0000

If you’re reading this at about the time I wrote it, some of the preceding (lexically following) diary entries are really rather short. Apologies for that. I might go back and fill them in later, but in the meantime: the URL was written down during Duane Rettig’s simple-streams tutorial, and should point to Paul Foley’s simple-streams implementation for CMUCL. The aardvark was an example in Nick Levine’s CommonSQL tutorial.

The conference (I’m at the ILC, if I’d forgotten to mention it before) is now about 80% done. Noteworthy points

If you're reading this at about the time I wrote it, some of the

McCLIM installation notes

Mon, 04 Nov 2002 19:51:53 +0000

McCLIM installation notes

McCLIM installation notes

Fwiw, Mr Shaver (if

Thu, 07 Nov 2002 01:15:32 +0000

Fwiw, Mr Shaver (if you do in fact read this diary as well as linking to it), I have it on reasonably good authority (one of my friends works there) that the OED typically requires five independent citations for a new word before they will consider it for inclusion. So, you can submit as many words as you like over the net, but they still exercise a more marked degree of editorial control over the result than, say, the Jargon File

For which relief, much thanks. And on a related note (related to the OED, not to the Jargon File, exactly how cruel do you think I am?), science fiction readers should look at http://www.jessesword.com/SF/sf_citations.shtml and see whether they can supply any of the missing bits. No, writing and submitting your own web page does not count.

I had opportunity this evening to attempt to corrupt the minds of youth by presenting Lisp to the OU Computer Society. Score: 0 people walked out, 2 people closed eyes and gave every impression of having driften off to sleep, at least 2 people asked intelligent questions at the end (and a few others during the talk itself). Magicpoint users (come on, it’s only an apt-get, the first one’s free) can experience the full beauty of my presentation by downloading the slides’ source from http://ww.telent.net/compsoc-slides.mgp – the rest of you can just read the source and imagine how the slides must have looked. Which is to say, basically the same as the ASCII, but in white Helvetica on a blue background, with a yellow stripey thing in the corner. Just like the ILC slides, yes.

Fwiw, Mr Shaver (if

While I was dreaming (about jumping the ticket barrier at Black Horse

Tue, 12 Nov 2002 14:15:08 +0000

While I was dreaming (about jumping the ticket barrier at Black Horse Rd tube station) this morning, I realised that I had a plane to catch today and had no idea what time it was and whether I’d be late. So, I pinched myself, and woke up. In someone else’s bed. The bed was fixed to a tall wall, about twenty feet up, and obviously wasn’t mine because (a) mine isn’t, and (b) the owner had left a digital watch in it – I don’t own a digital watch. So, I pinched myself and woke up. This time I woke up in a bed in a large open space with a sky the colour of the sky in whichever Pink Floyd album cover involves a desert full of beds, situated next to a glass computer laboratory partially submerged in a water feature comprising a pool with a fountain and a wide stream running down from it. A brief moment more concentration, and I was in a normal bed in a fairly normal looking place, which only really gave itself away as not my own bedroom when I remembered that I’d left California over a week ago and didn’t need to catch a plane after all.

So when I woke up for real this morning (in my own bed, yay) my first thought was “where am I?” and my second was “that was a lot of stack unwinding”

I mention this only because the diary entry I had previously been planning to write started out something like “When I was five years old, I had a vivid imagination that seems to have dissipated somewhat in the two and a half decades since”. Suddenly this seems to be not so unarguably true.

While I was dreaming (about jumping the ticket barrier at Black Horse

Once upon a time (insert wavy lines here) in the dim and

Tue, 12 Nov 2002 14:34:12 +0000

Once upon a time (insert wavy lines here) in the dim and distant past of, oh 1997 or so,

Once upon a time (insert wavy lines here) in the dim and

I was going to post an entry about end-user scripting of the unix

Thu, 14 Nov 2002 01:52:42 +0000

I was going to post an entry about end-user scripting of the unix desktop environment, and how nice it would be to have some way of remote-controlling the stuff (web browsers, time trackers, cd players, screensavers) that presently only really has graphical interfaces. Then I decided that more people would see it if I posted it on Advogato instead.

Then I realised that it was basically on-topic for the existing “Cranky User” thread on Advogato, so I ended up posting it as a comment instead.

I was going to post an entry about end-user scripting of the unix

Courtesy of lemonodor: Tim Converse

Thu, 14 Nov 2002 02:52:03 +0000

Courtesy of lemonodor: Tim Converse on Lisp advocates.

I don’t really believe that Lisp advocates aren’t significantly worse than any other subculture that considers “you just don’t get it” as a winning move in a debate (ok ok, I’m also a cyclist, a free software hacker, a linux user and a former amiga user, so hardly in a position to criticise), but the article is a good read nonetheless.

I will readily concede that the `solipsist’ point is well-made, and in my defence offer that cirCLe was originally titled Play Nice With Unix. There is no need to be “to be any more isolationist than, say, Emacs is” after all.

Courtesy of lemonodor: Tim Converse

And another thing

Thu, 14 Nov 2002 03:47:26 +0000

And another thing …

Sometimes the people asking for better OS integration are just so fucking snide – perhaps they don’t mean to be, perhaps they’re just frustrated and don’t realise that some of the rest of us also recognise this is a suboptimal situation, but the effect is the same no matter what the excuse – that it’s hard not to get really quite pissy. Of course, that rarely leads to much sweetness and light promulgated in either direction, but in the short term it’s quite therapeutic.

And another thing

bash-2.03$ strings /opt/bo/webi26/Server/WebIntelligence2.5/http-server/bin/aphwi.so | grep empty

Fri, 15 Nov 2002 17:20:33 +0000

bash-2.03$ strings /opt/bo/webi26/Server/WebIntelligence2.5/http-server/bin/aphwi.so | grep empty
<HTML><BODY bgcolor=white><CENTER><B><BLINK>This page is empty (WI0100)</BLINK></B></CENTER></BODY></HTML>
<HTML><BODY bgcolor=white><CENTER><B><BLINK>This page is empty (WI0100)</BLINK></B></CENTER></BODY></HTML>

bash-2.03$ strings /opt/bo/webi26/Server/WebIntelligence2.5/http-server/bin/aphwi.so | grep empty

Today I finally got so annoyed with the bizarre behaviour of Linux

Fri, 22 Nov 2002 15:51:35 +0000

Today I finally got so annoyed with the bizarre behaviour of Linux printing (especial culprits: error reporting in every print spooler I have tried, transparency of operation in printer filters when attempting to debug them ditto, paucity of useful and well-arranged documentation for ghostscript drivers which answers questions like “if this works with the stc driver, why does the exact same command with the uniprint driver give an incomprehensible error message”) that I bought what professes to be a real PostScript&tm; (well, real HP PostScript&tm; emulation) printer. Random Googling suggests that the scanner part of it can be persuaded to work with Linux too, so perhaps in future I’ll be able to sign and send faxes without having to walk up the road to the video shop. Should arrive Monday, with luck, so we’ll see what it does.

Today was also the massive new-version-of-CLiki rollout, where at last I got around to relocating it in its own domain. By bizarre coincidence, exactly 12 months and one day after I registered the domain, too. Anyway, the well-known URLs for CLiki and sbcl-internals both redirect to the new URLs, so apart from doubling the number of tcp connections your browser makes, you probably shouldn’t notice much. As part of this process I also (at last) moved Entomotomy from the imac on my cable modem to the colo box. New cclan packages for db-sockets, araneida, cliki, and entomotomy to follow in the next week or so.

URLs for your reference, Mr Google:

Eric Marsden has been doing some cool stuff lately with Araneida as an annotation server. Gilbert Baumann has been doing some other cool stuff lately to produce an annotatable CLIM specification (as far as I know, without Araneida). I’m not sure I’m supposed to link to either effort yet, so I won’t, but we hope the two will converge somehow.

Last night was the followup “hands-on Lisp” session to the talk I gave two weeks ago for the OU Computer Society. Kudos to the Debian packaging guys (who more usually get complaints than accolades from me on IRC) - it was a lot easier to tell the compsoc sysadmins “apt-get install cmucl cmucl-normal ilisp hyperspec” than it would have been to direct them through installing cmucl 18d from tarchives, defsystem frpom sourceforge, dump a core, install ilisp (which version?) … The actual session was pretty unstructured (I had been advised previously that they tended to be fairly unstructured so hadn’t prepared anything specific), but people found entertainment in mucking around with loops and suchlike, and one guy produced a Mandelbrot generator using CLX which I thought was pretty cool.

Today I finally got so annoyed with the bizarre behaviour of Linux

Somewhere between 2.4.20-pre5-ac3 and 2.4.20-rc2-ac3, the paging

Mon, 25 Nov 2002 18:37:49 +0000

Somewhere between 2.4.20-pre5-ac3 and 2.4.20-rc2-ac3, the paging behaviour of Linux changed drastically, to the point where my 128Mb laptop is no longer comfortable for many of the tasks I use it for. Alternatively, something else I changed (I’m running Debian unstable and staying reasonably up to date) is being evil, but I’m reasonably sure I rebooted more than a few times in the days immediately before the new kernel (ugly ext3 errors, hence the reason for trying a new kernel) and didn’t notice anything)

top(1) is reporting less swap used than normal, for what that’s worth

New printer arrived. I’d been quite irrationally excited about its imminence over the weekend – which is silly, it’s not exactly the most interesting possible purchase ever, even in the field of computer hardware. Plugged it in, turned it on, sent it postscript. It works.

Also over weekend, SBCL threads hacking. This is, uh, the third or possibly fourth attempt, and mostly so far consists of some messing about with segment selectors plus a forward port of the second (possibly third) attempt, so in spirit could be seen as a continuation of that one.

Somewhere between 2.4.20-pre5-ac3 and 2.4.20-rc2-ac3, the paging

On a train, on the way back from Alan &amp Telsa's, after my

Thu, 28 Nov 2002 12:02:37 +0000

On a train, on the way back from Alan &amp Telsa’s, after my spur-of-the-moment decision to go to the SWLUG meeting last night. A more informal gathering than OxLUG, held in a café then moved to the pub next door when it became obvious that there were more people present than would fit. Fun. Back to railway station for what is apparently fairly normal confusion over the timetable: “where’s this train going?” “the board says `terminates’” “ok, where’s Terminates, then?” — got on a train that showed some sign of wanting to head in the right direction, and after it pulled out, the train manager walked through with a notepad polling the passengers for where they wanted to go. OK, possibly just for the entirely prosaic reason that some of the stops en route are basically request stops, but I think that explanation is less entertaining.

Today, tour of Swansea (“here’s the castle, here’s the sheep shop, here’s the market, here’s the marina, this is the beach”) and introduction to Welshcakes. These can be approximately described as half-height dense scones, dusted with sugar

A few words about the progress of threading for SBCL, after my vague hints on Monday:

So far …

  1. CL has dynamically scoped variables. Not, thank goodness, by default, but available. The semantics we’ve adopted are that (1) a symbol’s global value can be set and is visible in all threads, and (2) it can also be dynamically bound: the dynamic binding is visible in only the thread that it was bound in. Other threads running concurrently still see and may also change the global value, but the thread that’s bound the variable won’t see those changes.
  2. This requires thread-local storage of some kind for the symbol values. On x86, we point to this with a segment register (%gs) as we’re already using all the real registers. Each thread has a vector of values, and each symbol gains a tls-index slot which has its index symbol-value.
  3. Spent much of the weekend fiddling with modify_ldt, and teaching the sbcl assembler how to assemble instructions with segment override prefixes (in fact, I still need to go back to it and teach it how to disassemble them again)
  4. On Monday, I looked through the diffs from the previous round of hacking and forward-ported them – or at least, the bits that still made sense – into the current version. This was for formerly-global variables which are actually supposed to be per-thread (stack pointers and the like)
  5. On Tuesday, I had other unrelated work to do for, which I actually get paid (note to interested readers: if you would like to see a threaded SBCL sooner rather than later, I am available to implement this and other SBCL/CMUCL enhancements on a contract basis. Email me for details). Spent a couple of hours on various kinds of public transport thinking about symbol binding and unbinding
  6. Wednesday (yesterday) and today I wrote bind, unbind, set and symbol-value vops, and equivalent functionality in C (following a variation of the Extreme Programming methodologists’ Once And Only Once principle known as “Once And Only Once More”) for the bits of C code that need to do these things in situations where not enough of Lisp is running to use the normal vops. Then I debugged it. This is mostly a question of rebuilding, watching it segmentation fault, attaching gdb, disassembling, cursing, set disassemble intel, disassembling again, and scratching head. With intermissions for previously noted LUG meeting in Cardiff, and tour of Swansea.
    <dan_b> straw poll: how many special variables do we think that sbcl binds during cold init?
    <dan_b> I think 4096 sounds like a lot
    <Krystof> I’d have been surprised to find that we had 4096 distinct special variables
    <Krystof> * (length (apropos-list “*”))
    <Krystof> 1055
    <dan_b> hmm
    […]
    <dan_b> aha.  got it, i think
    <dan_b>       (storew tls-index symbol symbol-tls-index-slot other-pointer-lowtag)
    <dan_b> having assigned a new tls index for the symbol, it would help if we actually stored it in the symbol strcuture for next time
    

    Right now work is paused, because recompiling takes an hour and eats battery life, and I’m on a train. So, time out to write diary and plan the next bit.

    (The next bit will actually be integration with the allocator and garbage collector, given that right now it would be completely unsafe for two threads to cons at once)

    Ext3 errors continue (though no panics today yet, at least) even with the new kernel, so that’s not the problem. Would blame hardware except that there’s none of the usual scary messages from the ide driver. Maybe a filesystem integrity problem caused (I’m guessing here) by apm forced shutdown when battery low, at a time that the disk was being written to, and not fully fixed by subsequent fscks. Tempted to try mkfs and reinstall (it’s the root disk, not /home or anything important), and see if that helps. I probably have a lot of old config files, orphaned library packages and other similar stuff anyway, and it would be nice to get rid of them.

    On a train, on the way back from Alan &amp Telsa's, after my

Weekend off, pretty much

Mon, 02 Dec 2002 16:30:43 +0000

Weekend off, pretty much. Weekend spent preparing for, co-ordinating and recovering from my friend Simon’s stag night.

write diary and plan the next bit, he said on Thursday. Said plans have not yet reached the enemy, because still wrestling with bugs in the current bit. Most of them fairly silly bugs once found : what’s wrong with this code?



(define-vop (bind)
  (:args (val :scs (any-reg descriptor-reg))
	 (symbol :scs (descriptor-reg)))
  (:temporary (:sc unsigned-reg) tls-index temp bsp)
  (:generator 5
    (let ((tls-index-valid (gen-label)))
      (load-tl-symbol-value bsp binding-stack-pointer)
      (loadw tls-index symbol symbol-tls-index-slot other-pointer-lowtag)
      (inst add bsp (* binding-size n-word-bytes))
      (store-tl-symbol-value bsp binding-stack-pointer temp)
      (inst jmp :ne tls-index-valid)
      ;; allocate a new tls-index
[…]
That’s right. There’s no test before we do the conditional jump. Let’s try
(define-vop (bind)
  (:args (val :scs (any-reg descriptor-reg))
	 (symbol :scs (descriptor-reg)))
  (:temporary (:sc unsigned-reg) tls-index temp bsp)
  (:generator 5
    (let ((tls-index-valid (gen-label)))
      (load-tl-symbol-value bsp binding-stack-pointer)
      (loadw tls-index symbol symbol-tls-index-slot other-pointer-lowtag)
      (inst add bsp (* binding-size n-word-bytes))
      (store-tl-symbol-value bsp binding-stack-pointer temp)
      (or tls-index tls-index)
      (inst jmp :ne tls-index-valid)
      ;; allocate a new tls-index
Hmm. Why doesn’t that work either? This is (also) the kind of bug that’s blindingly obvious once you realise it, of course: (or tls tls) is not a form that will assemble into the instruction that does set the condition register. It’s a Lisp expression that evaluates tls then evaluates tls if tls was NIL. But then, as you see, doesn’t actually use the result in any case. Eventually I noticed the missing bit:
      (inst or tls-index tls-index)
and things are substantially rosier. Now we make it all the way through cold initialization to the toplevel, but break with what looks like random memory corruption shortly afterwards.

Weekend off, pretty much

There is one and only one aspect in which doing low-level stuff on an

Tue, 03 Dec 2002 23:01:41 +0000

There is one and only one aspect in which doing low-level stuff on an x86 is appealing: hardware watchpoints.

After spending a lot of time today tracing through the disassembly for the note-undefined-reference function to find out how it was getting (0 . 0) into undefined-warnings - supposedly a list – I put a watchpoint on that memory location and reran. The value subsequently changed in get-output-stream-string, which is a perfectly normal library function that has nothing to do with the compiler and certainly no references to undefined-warnings. After seeing that it was the work of mere minutes (although, I concede, entirely too many of them) to realise that things would probably work a Whole Lot Better if GC were allowed to see the thread-local value vectors so it could update pointers when the objects move. Sigh.

Now we’re back to a system that actually gets all the way through cold-init and PCL compilation to produce a usable Lisp. Admittedly, still not one that lets the user actually create threads (not much point adding thread creation primitives until consing is thread-safe, after all), but it’s a start.

There is one and only one aspect in which doing low-level stuff on an