Cool URL of the Day: In Pursuit of Simplicity: the manuscripts of Edsger W. Dijkstra
They're scanned PDFs, of a somewhat ropey quality: readable at 1.4 magnification, though
Today is random asides day
Well, slightly under the weather at least. This weekend I had a wedding (not my own, no), a barbecue, a party and a nose full of microwaved cotton wool. Last Friday I had an all-afternoon wish to lie down and sleep unobtrusively in the middle of the carpet, which I now realise was probably early cold symptoms, plus a bunch of scary disk error messages from my colo box (Rackspace fitted a new disk, upgrading the OS version while they were at it, then I just had to reinstall all the locally-changed stuff), plus an evening in the pub. This is all in the nature of an excuse for not having done much hacking lately.
What really wasn't helping too much is that I'd originally set out at the start of the week to add some form of access control to the CLiki application (the CLiki site will stay as it is, but there are lots of other applications I'm imagining for which it would be better to know who the users are and whether they're Allowed), realised that the handler system for Araneida is suckier than it really wants to be for these kinds of application, and ended up redesigning large parts of it. And now I've created a new class called handler, I find that I want to introduce lots of new GFs whose names clash with existing Araneida functions. So perhaps this is time for a name change. Nobody appears to have produced http serving software called Boris yet, which is a pretty compelling reason in itself.
Anyway, we've got to a state where it basically runs again, so maybe we can get this wrapped up in time for Thursday
We have working CLiki again! Well, at least, all the bits I've checked so far are working again: it's not exactly been tested exhaustively yet.
I got sidetracked halfway through the afternoon staring at my four or five new GFs and thinking "hmm, why don't I use define-method-combination" until I realised that "because it provides no advantage that I can reasonably easily envisage and I do want to actually get this finished one day" was in fact the Correct Answer.
(For the avoidance of doubt, GF in this context stands for Generic Function. If I had four or five new girlfriends I'd ... well, for a start I'd remember whether I had four or five)
For some reason I've started reading and contributing to Ward's Wiki a lot more lately. When I was first doing Wiki I was mostly using Netscape 4, which gives you some idea of how long ago it was, because I can't actually remember when was the last time I used Netscape 4. Mozilla rocks.
But, anyway, well. Except for this bit. "This bit" is that wiki
has Text
Formatting Rules which demand a variety of keyboard gymnastics,
not least among them that various formatting (UL lists, BLOCKQUOTE,
PRE etc) demands the insertion of a literal TAB (ASCII 007) character.
Pressing TAB in a Mozilla text field, though, makes it go to the next
text field instead of inserting anything, ^V doesn't seem to do the
conventional readline "insert next character as literal" thing, and
the conventional Emacs "insert next key as literal character" thing -
don't press that. Without prompting
to do anything with the four paragraphs you just typed, either ...
So, I cut and paste from my scratch buffer, which is silly but just about livable. But, this is wrong. This is Free Software. Even if I will probably never ever myself actually write code for a project in which the majority of a 45 minute "how to hack on $project" talk is given over to a description of getting reference counting right, I can at least file a bug report. Or at least, read the bug report that someone must already have opened on this subject and see if there's some way to do it already that I'm missing. Well, they did, and there isn't. Bug 29086, has been open for more than two years, and attracted so much argument that it's over 100k long as viewed by lynx -dump
Lessons we can learn:
(Yes, I know there's a workaround in Wiki, but it's heuristic and doesn't seem to do too great a job)
Of course, the Wiki markup rules started life as simple things, and it wasn't until later that people realised they could do things like typing indented-but-not-preformatted text by typing TAB : SPACE TAB at the start of the line (and possibly also typing the entire paragraph on one line; I forget whether it makes a difference in that specific instance. it does for italics, though)
This reminds me of some other software I've been using recently that starts out with a simple set of rules which combine in dizzyingly unpredictable ways to do things you would never have dreamed sensible. I'm talking about spamassassin here, of course.
I've never been entirely clear, it must be admitted, on how best to use all the weird gnus options to keep copies of outgoing mail, and eventually came to the conclusion that the simplest answer would be to Bcc it all to myself. This has the neat advantage that I can then use gnus splitting to filter incoming mail to some regular correspondent into the same folder as mail from said correspondent, and then I have both sides of the conversation in the same place. It has the (fairly innocuous) side-effect, though, that my outgoing mail loops through spamassassin before coming back to me, so I get to see how spammy my mail looks. Well, look at this
X-Spam-Status: No, hits=-98.4 required=5.0
tests=SIGNATURE_DELIM,FOR_FREE,USER_IN_WHITELIST
version=2.31
body Standard signature delimiter present SIGNATURE_DELIM 0.488
A positive score for a signature delimiter? "- - SPACE \n", so rarely correctly implemented that it became favoured all over Usenet as the choice "get a real news reader" stick with which to beat Outlook users (and another great example of why whitespace sensitivity is a dumb idea, fwiw) is now suddenly an indication that the mail was sent from some bulk mailing program that injects fake unparseable received headers, has the wrong system date, and probably still thinks X-UIDL is a header that should be provided by the sending host? I find this not entirely plausible.
From conversations on IRC, I understand that the actual scores for each rule are computed by feeding a large pile of known-spam and another large pile of known-nonspam into a genetic algorithm and letting it work them out. This sounds like really great news for the stability of weights on each rule ...
I wish I could find my neural networks book. It was possibly the most boring book ever written about what should be an interesting subject, but I desperately want to be able to liken spamassassin to a neural network with only one neuron after the kind of particularly vicious overtraining that should have the owner in court on animal cruelty grounds, and I really could do with a reliable source to tell me whether it's a remotely fair comparison.
<dan-b> roll on, version control systems that allow directory renaming <dan-b> this whole thing with having to pick a name before starting work really sucks
entomotomy, Zool. the science of the dissection of insects to ascertain their structure, insect anatomy.
I'm sitting on a train, going to London. I don't actually want to go to London. I want to go to Didcot - not per se, I add hastily, but as the first target in a plan that subsequently involves boarding another train that will take me to Bristol. I joined this train under the impression that it also wanted to go to Didcot. Either I got the train destination wrong, or some other body (Railtrack, Network Rail, Oxford station managers, Thames Trains, the driver - it's hard to tell who's responsbile these days) did - but the Railtrack web and wap sites both agree with me, and the information screens at the station likewise. Bum.
Train number two, then. The two ticket inspectors who've looked at my ticket so far have not commented on the bit that says "Route: NOT READING". Perhaps that only restricts people who get off at Reading: I've now been through said station twice, in opposite directions, but have remained on the train. Or perhaps they could sense I was looking to start an argument and wisely didn't comment.
Well, at least I'm here, and in a talk ...
Wireless networking aside: the university has wireless access, but ...
First you get a username and password from the registration desk. Then you wrestle with it.
Bum. authentication isn't happening
aha, that's better
A half hour wasted by spamassassin. I reattached to the wireless lan when I got here this morning, ran fetchmail, and watched my computer eat all its swap and drive the load average to something above 40 trying to start 70 copies of spamassassin. It's really time I sorted some kind of daemon-based approach; in the meantime I just disabled the thing. Perhaps I should replace it with some elisp/gnus stuff that I can actually trust.
A PHP talk: "Using PHP for large web sites". Can be summarised as "PHP is too evil to use for large web sites" (slightly longer summary: "PHP is far too evil to use for large web sites), but was more entertaining in the long form. It was gratifying to have my uninformed three-hours-reading-the-docco opinions on PHP confirmed by someone who has actually used it in anger.
A Bugzilla talk: how to write a scalable featured database-backed bug tracking and management system.
Not that I think any of these are going to be requirements for Entomotomy in the near future, but we can hope.
They also have serious problems with low-quality bug reports, and a lot of their development (the UNCONFIRMED status, voting on bugs, etc) has been driven by the need to manage this. Hope this won't be our problem either in the near term.
Out-of-context quote of the day: "Bugzilla is a very forked project"
Bug 12411: Mozilla does not have a kitchen sink (look at the attachments)
An RT talk: RT is an insanely customizable ticket tracking system. But then, it's written in Perl, which (when done right) is almost sufficient to get you both of "customizable" and "insane" For Free.
Zope 3. You thought Zope was a web application server, but actually it's an "Operating System for the Network". I found the combination of buzzwords, warmth in the lecture theatre, and XML-like code in the slides to be unfortunately soporific, but as my network doesn't actually need an operating system I thought I'd not be missing anything earth-shattering if I slipped out early.
glibc 2.3: the existing C library locale model is inadequate for threaded applications. I can believe this but I'm not sure I see why the POSIX locale model is worth caring about anyway. The current thread-local storage API is ugly, so 2.3 introduces a new thread keyword to C, which has the same approximate syntax as register or static. And people are using internal symbols (setfpucw, anyone?) and geting broken by it on glibc upgrades, so from now on all the internal symbols are marked with the version GLIBCPRIVATE.
Two years after I first started hacking on CMUCLish stuff, I still don't understand the CMUCL compilation procedure.
(Actually I don't understand a lot of the SBCL compilation procedure either, but that's ok, because I don't need to - it Just Works (tm))
Eric Marsden is about to tell us all how it works. Once he manages to get the projector working with his Powerbook, anyway.
I've been lax updating this thing given the amount of stuff I potentially could have been saying in it lately. So, where are we?
We are at the Libre Software Meeting 2002, in the Very high-level languages for writing applications topic. The rationale here, according to the programme - look, if you would only go and read the programme, it would save me from having to paraphrase it like this - is that the kernel is done, so now we need to write applications, and we need these very high-level languages if we're going to finish the job in timescales that the human brain can relate to. Just ask any Mozilla developer.
In fact (as you can reasonably infer from the schedule) it's all a thinly-veiled plot to get many free Common Lisp developers together in the same place. Works For Me. So we've spent the last few days discussing things like garbage collection, coping with special (a.k.a dynamically-bound) variables during multithreading, packaging, bug tracking, etc etc). And, as is often the case where two or more lisp implementors are gathered in the same room, the very high-level languages we've actually ended up using are C and x86 assembler. Yeah. Um. Christophe is sitting next to me working on floating point trap handling (see also the previous attempts to tackle this), Eric Marsden was - until an hour ago when he went to catch his train - poring over trace files to figure out why his CMUCL "small" images no longer worked, and I am watching SBCL build with a new design of non-invasive stack overflow handling. Currently it can non-invasively detect stack overflow, but it can't actually recover from same, which makes it a bit useless. Cargo cult programming is not a sane or sensible way to write x86 assembler glue.
And we've had some talks too, of which more some other time.
More on control stack exhaustion detection, then, seeing as it pretty much works.
The goal is that when a (probably newbie) user evaluates an infinitely recursive function, we handle it safely instead of just running out of space for our stack frames and crashing. In current SBCL releases we do this by insterting a call to %detect-stack-exhaustion at the start of each lambda in safe code, but that tends to increase the code size (internally, lambdas are used all over the place), and it's not very fast, and it doesn't work at all (it's turned off) when optimization qualities are set to prefer speed to safety.
So, why don't we do it by protecting a page near the end of the control stack and inserting another clause into the SIGSEGV handler that notices this, presents the user with an error message and lets them abort the computation. Sounds simple.
Oh Intel, we love you.
So why am I telling you all this? I don't know, but I felt I had to tell someone.
Control stack exhaustion is still in a state of "pretty much" (i.e. "not") working. This past weekend was more or less the deadline for commits for the next SBCL release (0.7.6) so I thought I had better try it on something other than an x86 Linux box. First step, the Alpha. That worked OK, so, FreeBSDeers, prepare for battle.
(Why FreeBSD? There's one in the sourceforge build farm. The sourceforge build farm is sucky in entirely numerable ways, but at least it's there).
First hurdle: it uses SIGBUS instead of SIGSEGV for writes on unmapped pages. That's easy enough. Second problem, some screwup with overlapping mmaps that really ought not to (defining a signal stack in the middle of the dynamic space is bad, ok?). Third problem: the second time you hit the guard page, you get a SIGBUS followed immediately (before the sigbus handler runs, even) by a SIGILL. I don't know why.
Funniest URL of the day: this bit from the FreeBSD Developer's Handbook.
While a typical Windows application is attempting to do everything imaginable (and is, therefore, riddled with bugs), a typical Unix program does only one thing, and it does it well.
Sure. So I imagined cat -s and ls -C, or are they not typical Unix programs?
In the Olden Days when I was writing FTX13, I used periodically to report on new versions of LISA, a production rule system for Common Lisp. As with so much of the other stuff that went into FTX13 I'd never actually downloaded it to try for myself, so these reports tended just to recycle whatever the project release notes said. The accrued boredom and low-level guilt from doing this over a six month period was one of the factors in giving up FTX13.
So, there's a new version of LISA out today (1.3), and I still haven't looked at it, and i still feel vaguely uneasy about this. Anyway, you can probably determine whether it's useful for you: see e.g. mab-clos.lisp
Ha!
The FreeBSD problem was that it really didn't like us fixing up the stack pointer (to swap back to the normal control stack) by hand in assembly: when it's about to call the signal handler on subsequent occasions, it seems to remember that it's still on the alternate stack - note if you will that we aren't actually ever returning from the signal handler, we're just calling into lisp and letting lisp unwind past it. I assume it's storing the running-on-external-stack-p property somewhere instead of just comparing the stack pointer with the bounds of the alternate stack.
Either we fix this somehow, probably involving the perusal of FreeBSD kernel sources, or we think of a new approach. The latter sounds tempting ...
Here's how to do it, then.
function=
&(((struct simple_fun *)
native_pointer(SymbolFunction(CONTROL_STACK_EXHAUSTED_ERROR)))
->code);
*os_context_pc_addr(context)= function;
Patch going to sbcl-devel mailing list and/or being committed (I'll decide which after I've read it again and formed an opinion on the essential/accidental grottiness ratio)
ICanCAD is a GPLed CAD editor for analog(ue) and mixed-mode signals. It's built using Allegro CL and their CLIM, which unfortunately makes it difficult for anyone else to hack on it unless they have the full Allegro system - CLIM doesn't come with the eval version. But you can download the ICanCAD Linux/x86 binary and play with it for yourself, which makes it kind of rare as CLIM apps go. Requires Motif 2, which is in Debian - albeit in non-free. Here's a completely unattractive screenshot of ten minutes wild flailing around.
What was it I said the other day? Oh, yeah, ``CLIM Rocks''. Well, um. it could rock even harder if they didn't all want to use that ugly Courier font all the time. Does Not Make For Pretty Pictures. Contrast with, say, Closure (that's from McCLIM, probably quite a recent snapshot of) and you'll see what I mean. Come on guys, give us stuff we can make screenshots from.
SBCL 0.7.6 is out, doesn't include my stack checking fixes. SBCL 0.7.6.2 is in CVS, and does. And I even built it on PPC to check they work there too.
I've just spent the last hour reading the SBCL generational garbage collector in Borders' cafe, while thinking "I'm getting remarkably good battery life here". Too late I realise that I'm looking at the wrong percentage figure in the emacs mode line - that 91% is a measure of how much I've read, not how many of the electicity beans I've retained. That would be 42%. Doh
Aforementioned cafe hacking was in Borders on Oxford St (in London), after a shorter-than-expected meeting with the accountant. After finishing my battery and my orange juice (and confirming that both of my wireless cards work, insofar as the ability to find the access point for some random local wireless network denotes workingness) I went off in search of electricity.
Twenty minutes later I was accosted by an employee who told me that I couldn't plug that in there. She was perfectly polite about it, and maybe it is official policy there, but it seems kind of an odd policy when so many other places (airports, train stations, even the train I took to London) either tolerate or encourage it. Oh well.
So I came home instead.
While we are on this subject, this kernel change on the sourceforge machine meant that I had to transfer my ppc testing laboratory to Dan's iMac. And I found another nasty surprise waiting for me there. On the SF RS/6000 PPC, the floating point modifications worked as expected, giving the right kind of exceptions in the right circumstances. On Dan's iMac: * (/ 1.0 0.0) 1.0 ; should signal DIVIDE-BY-ZERO
(Christophe's email to sbcl-devel). So, somehow, I got nominated to look at it.
I haven't actually fixed it yet. The immediate problem is that it's not sufficient to twiddle the FPSCR to enable floating point traps: you also have to set two bits in MSR (which, incidentally, you can't directly, as it's a privileged register). That's not actually a major problem, because there's a neato glibc function called feenableexcept() that does this (strace suggests that it works by installing a signal handler for SIGUSR1 that frobs the on-stack MSR, then doing kill(getpid(), SIGUSR1)). What is a major problem is getting it to stay set, because it gets reset in signal handlers, and restored when the handler returns - which punts us back into last week's problem, that half of the time, we don't return from said handlers in any conventional kind of way. Um. More details here
