Month of January 2004

That's the last time I turn computers off when I go away. The PSU in the Alpha appears to be dead, and the PSU fan in the x86 desktop was sticking as it went round. It seems to be better now it's warmed up a little, though. For the moment I've pulled the disk out of the Alpha to regain access to some CVS repositories,

OK, see how you get on with http://ww.telent.net/diary/diary.rss

There are people (well, there's one particular person using an RSS aggregator called rawdog) grabbing the rss feed for my rambles every twenty minutes. Right through the night, too. Which seems a trifle excessive to me, but hey, it's getting a 304 every time, so no big deal.

Means I probably should tweak the Analog configuration, though, if I want its lovely statistics to be at all meaningful.

I see that LWN has got its predictions for what happens to Linux in 2004 up. Inspired by that, I'm going to make some predictions of my own about Lisp in 2004

• 2004 will not be the "Year of Lisp". When was the Year of Linux? The Year of the Internet? The Year of the LAN (remember that one?) Exactly. 2004 will be the year that some number of people suddenly realise they can use Lisp to solve their problems, and it will be a bigger number than had the same realisation last year, and 2005 will have more still. But that doesn't make either (any) of them the Year of Lisp.

• SBCL will get callback support, and threads on some non-x86 platform (most likely PPC). It's kind of cheating for me to make SBCL predictions, though, so expect more when I've talked to Christophe and we've decided what we want to do.

• Sometime after it gets its CVS repository back on the net, CMUCL will release 19a, which will have approximately what the current CMUCL CVS versions have in them. The CMUCL project suffers hosting problems on an approximately two-yearly basis, so will not seriously drop off the net again until 2005.

• Gary Byers will tire of the low takeup of OpenMCL 0.14 pre-releases and will label it an official release so that users actually get it and start beating on the native threads support seriously. No major problems will result.

• SLIME will replace ILISP as the "default" emacs interface for free Lisps. It will gain thread support.

• MCCLIM will progress from its current status as the project that everyone admires and nobody actually uses, to a new position as a project that everyone installs to play with the Listener, but doesn't then use for anything serious. This will not change until a real application (probably Maxima) is released that depends on it. Lots of talking and approximately no hacking will be done on a CLIM-based editor/IDE, and a CLIM Gtk backend.

• A "major" Linux distribution (i.e. Red Hat or SUSE) will follow the lead set by Debian, Gentoo and Slackware, and put a decent general-purpose CL system (CMUCL or SBCL) into their distribution

• They'll screw it up by having someone do it who doesn't actually use Lisp, so it'll be (a) an old version, (b) a brand-new release which is missing the rushed post-release patches that actually make it work, (c) compiled with really dumb compilation options (SBCL without threads, or CMUCL with incorrect source paths, or similar), or (d) missing bits (like the FreeBSD SBCL port has no contrib). Maintainers of whichever CL it is will bemoan the fact that the distro fouled it up so badly. The distro hacker will claim that he asked for advice on IRC/usenet but everyone was too busy arguing about politics and/or Norway to give him an answer. People will look back through Dejanews or the IRC logs and find his question, which was written at the end of a 16 hour day wrestling with some out-of-date tarball from Sunsite and started with "Why is this stuff all so totally undocumented, broken, and completely fucking incomprehensible. Lisp sucks" and escalated from there. Everyone will take sides. A bug will be reported in the vendor's bug reporting system, which will be closed with the tag 'WONTFIX' or local equivalent.

  (Note: it doesn't have to go that way. Red Hat and SUSE hackers are warmly invited to introduce themselves as such on the relevant development lists and ask for packaging advice. We're friendly enough if we don't think we're being trolled, and it's in all our interests to get decent quality packages into the major distributions)

• comp.lang.lisp will become still more offtopic and still less useful. At least one defender of proprietary Lisp development and one prominent free Lisp hacker will blow up loudly and publicly and swear that he is leaving {cll, usenet, Lisp, software} forever. Nobody will believe him, though many will claim to.

  (Hmm. How public have I been about my departure from cll? I expect it's temporary until I get broadband sorted out again, but after an hour playing with Dejanews yesterday I have to admit I'm not missing it much. But it's still 6 hours before 2004 as I write this, so you should probably look for another free Lisp hacker anyway)

• Peter Siebel's book will be released, making many people happy and me jealous.

• Sourceforge will suffer some major and long-term screwage, affecting many free Lisp projects. By the time it returns to normal service

  • SBCL and ECL will have moved elsewhere

  • CCLAN as a brand name will have been largely forgotten, but asdf and asdf-install will be moved to common-lisp.net.

  • ILISP probably won't have noticed.

• I haven't heard anything about ILC 2004 yet; I don't know what the plans are. I do know that Ray de Lacaze did an absolute ton of work last year and I was sorry to miss it. Whether ILC happens or not this year, I predict more Lisp presence at other conferences, including LSM/RMLL, UKUUG, LinuxTAG etc.

• Probable vapourware in 2004: CL-Emacs (unless Gilbert resurfaces or Luke Gorrie gets into it), productized cirCLe, an SMTP server with Bayesian spam filtering, another LispOS and another attempt at a c.l.l FAQ

I don't know if this is something that Livejournal users can tell already for themselves, but just in case it's useful: I see http://www.livejournal.com/users/dan_b_feed/ in my referrer log. If this helps you with "friends lists" or however that stuff works, please feel free.

Remember what I said before Christmas about GC frobbing? With the new simpler region allocation policy described there, but no changes to static space usage :

 (time (dotimes (i 400) (gc)))

Evaluation took:
                 30.413 seconds of real time
                 30.354383 seconds of user run time
                 0.058991 seconds of system run time
                 0 page faults and
                 4939776 bytes consed.
NIL

and in fact we can easily double that again by disabling the check for read-only objects and dumping everything in static space, so it looks like the write barrier stuff does make a difference. So why is CMUCL doing in 13 seconds what takes us 30 (the easy answer is "it has a better object/region allocation policy", obviously, but better why?) and how much faster would it be if it were also to remove purify-into-static-space?

In happier news, while thinking about tidying up today I found a Brian Aldiss novel I apparently haven't read yet.

I lied. Well, I was mistaken: in the course of tidying up purify so that I might understand it, I managed to break it. !(a && !b) is not, De Morgan will happily tell you, the same thing as (!a && b). Doh. I was copying a whole bunch of stuff (constant boxed vectors, or possibly unboxed inconstant vectors) to static space that should have been happy in read-only space. So static space was up to 9.5Mb from its normal 5, hence the extra time spent checking it. If it takes an approximate extra 10s to GC when there's an extra 5Mb of static space to check, this also goes some way to explain why CMUCL is faster: in 18e it only has 2.5Mb of static space, so that's 5s off the GC time, leading us to predict that it should take about 15s. It's actually a bit under (13s) but that's only a couple of seconds still to claw back, then.

All times are to execute (time (dotimes (i 400) (gc)))

For a change I decided to google "dan_b" instead of some other variation on my name, and stumbled across (among other things) an irc log from 1998

<lilo> it seems to me we do this for the fun of it, and for things we can use
<esr> lilo, I don't play politics and I don't spend any time trying to tear down or disband anybody else.  I wish 
everybody would realises that that shit is a waste of time.
<lilo> some of us definitely do :)

I remember watching this at the time. I don't remember if it was before or after VA, but I'm guessing it was before.

More RSS fun. Apparently weblog rss feeds should have items in reverse chronological order, just like the weblogs they syndicate. So, I switched mine around. Also added the first sentence of each item as the item title: this will fall down in cases where my rather poor heuristics for detecting sentences, or in entries where my inverted pyramid has overbalanced. Apologies to anyone using an RSS client which will therefore decide that these are all brand new entries and make them read all my old entries again.

If anyone would like to recommend a GUI RSS aggregator for Linux that's not (a) Straw, or (b) Emacs-based, please feel free. There's nothing particularly wrong with Straw, but I worry whether the debugging messages it spits out from time to time are caused by problems with it or problems with my rss.

I have just come back from my first rollerblading in three years or so. It was more fun than I remembered, and obviously not as difficult: I only fell over once, which evasive manoeuvre was more or less planned, and involved comparatively^* little loss of skin. Oxford may not have a worthwhile Hyde Park equivalent, but it still has more smooth pavement than I'd realised.

OK, you'd rather have heard about some kind of low-level SBCL hacking or something. Bad luck.

^*compared to, say, last Sunday morning, when I fell over - sober, which is more than I am now - while showering, and cut my head on the corner of the towel rail.

Hello world

... hello trees, hello flowers, hello sky ...

The novelty of being back on the internet has still not entirely subsided.

I am working on a 64 bit SBCL for the AMD64 (formerly x86-64, née Hammer) architecture. So far it makes it successfully through make-host-1 (the cross-compiler) and make-target-1 (the runtime), and all the offsets and suchlike that genesis spits out look correct - all the ones I've checked, at least. Before anyone gets too excited, though, it is without question utterly broken in several important ways: not least that I haven't taught it how to generate code for the 64 bit registers, or taught gencgc about all the interesting new types and widths

I can't help feeling it would be nice if something could be automated to dig the information that gencgc needs from Lisp.

TODO list as soon as I take a break from this:

• cliki index bug

• stargreen behaviour-under-load frobbing

• get version-control archive thingies for araneida/cliki "out there" so that next time my connectivity goes away it doesn't stop other people from hacking on them. This may involve learning arch, if its replicated repository stuff means that I get to hack on them too in such circumstances.

Some progress on AMD64: the cross-compiler has managed to cross-compile something

; compiling file "/opt/home/dan/src/sourceforge/sbcl/src/code/show.lisp" (written 02 OCT 2002 06:09:17 AM):
; compiling top level form: 
; compiling function SB!INT::CANNOT-/SHOW: 
; recognizing DEFUN CANNOT-/SHOW
; compiling top level form: 
; compiling top level form: 
; compiling DEFMACRO /SHOW: 
; compiling top level form: 

The code that it's generating is, I have no doubt, seriously bogus, but hey! It's code! At that point it fails with

debugger invoked on a SIMPLE-ERROR in thread 27960:
    failed AVER: "(NOT (EQ SIZE QWORD))"

which comes from the MOV case for big immediate constants which checks they're not 64 bit immediate constants - seems that at leastg one of them is. On the basis of the bits of amd64 manual I've read so far it seems that I can only load a 64 bit constant into rax and not into any other register, which is a shame, because the backtrace says it wanted to use rbx. Perhaps it's bedtime.

<rtoy3> It compiles!  Ship it!

I'm shortly going to be a normal person again. On the sofa next to me is a written offer letter from clara.net in which they describe a money-making scheme requiring only 40 hours of my time each week in exchange for which they will transfer funds directly into my bank account once a month. Which seems like a pretty neat idea compared to the more usual "do some work, invoice for it, wait a month, phone customer to enquire whereabouts of cheque, wait two weeks, pay cheque into bank" arrangement, and I wonder if it will catch on more widely.

Answers to FAQ: (1) in London. Yes, I probably will move back there. (2) A certain amount of trepidation, but flexitime makes it not so bad, and I can use chemical crutches in the form of unlimited free coffee to cushion the blow. (3) AFAICS the contract doesn't allow it, although it's hard to tell how some of the clauses group: it's written in legal English and there are no commas anywhere. Perhaps it only forbids moonlighting when the business is "competing or tending to compete with the Business of the Company"

AMD64 SBCL now builds 50-odd files before running into an array type problem. Said files are doubtless still riddled with garbage; the easiest way to see the disassembled output will be to get all the way through make-host-2 (building the target compiler) then starting up the the new runtime with the cold core thus generated, so I'm not worrying about that just now. There is a disassembler in SBCL, but it's no more likely to be correct than the assembler is, so let's hope gdb on linux x86-64 is up to the standard of gdb on linux x86 as opposed to the less well-supported platforms like, ooh, almost anything else it claims to run on.

Anyone reading this whose workday is in large part concerned with gdb on Linux x86 may be forgiven for wondering if I've been drinking. As it happens, I've been drinking "Bat-currant Ribena" (the carton has a disclaimer on it saying "does not contain real bat", which is reasonable as they're a protected species), but that's not relevant: let me assure you that the contrast with, say, linux alpha (tru64 alpha is even worse) is readily apparent.

I watched "A Life Less Ordinary" on tv last night. If I had the year 1997 all over again, I'd have paid more attention to the music.