Fri, 30 Jan 2004 00:58:48 +0000
Some progress on AMD64: the cross-compiler has managed to
cross-compile something
; compiling file "/opt/home/dan/src/sourceforge/sbcl/src/code/show.lisp" (written 02 OCT 2002 06:09:17 AM):
; compiling top level form:
; compiling function SB!INT::CANNOT-/SHOW:
; recognizing DEFUN CANNOT-/SHOW
; compiling top level form:
; compiling top level form:
; compiling DEFMACRO /SHOW:
; compiling top level form:
The code that it’s generating is, I have no doubt, seriously bogus,
but hey! It’s code! At that point it fails with
debugger invoked on a SIMPLE-ERROR in thread 27960:
failed AVER: "(NOT (EQ SIZE QWORD))"
which comes from the MOV case for big immediate constants which checks
they’re not 64 bit immediate constants – seems that at leastg one of
them is. On the basis of the bits of amd64 manual I’ve read so far it
seems that I can only load a 64 bit constant into rax and not into any
other register, which is a shame, because the backtrace says it wanted
to use rbx. Perhaps it’s bedtime.
<rtoy3> It compiles! Ship it!
Some progress on AMD64: the cross-compiler has managed to
Fri, 30 Jan 2004 10:13:57 +0000
I’m shortly going to be a normal person again. On the sofa next to me
is a written offer letter from clara.net in which they describe a
money-making scheme requiring only 40 hours of my time each week in
exchange for which they will transfer funds directly into my bank
account once a month. Which seems like a pretty neat idea compared to
the more usual “do some work, invoice for it, wait a month, phone
customer to enquire whereabouts of cheque, wait two weeks, pay cheque
into bank” arrangement, and I wonder if it will catch on more widely.
Answers to FAQ: (1) in London. Yes, I probably will move back
there. (2) A certain amount of trepidation, but flexitime makes it
not so bad, and I can use chemical crutches in the form of unlimited
free coffee to cushion the blow. (3) AFAICS the contract doesn’t
allow it, although it’s hard to tell how some of the clauses group:
it’s written in legal English and there are no commas anywhere.
Perhaps it only forbids moonlighting when the business is “competing
or tending to compete with the Business of the Company”
I'm shortly going to be a normal person again
Mon, 02 Feb 2004 13:21:02 +0000
AMD64 SBCL now builds 50-odd files before running into an array type
problem. Said files are doubtless still riddled with garbage; the
easiest way to see the disassembled output will be to get all the way
through make-host-2 (building the target compiler) then starting up
the the new runtime with the cold core thus generated, so I’m not
worrying about that just now. There is a disassembler in SBCL, but
it’s no more likely to be correct than the assembler is, so let’s hope
gdb on linux x86-64 is up to the standard of gdb on linux x86 as
opposed to the less well-supported platforms like, ooh, almost
anything else it claims to run on.
Anyone reading this whose workday is in large part concerned with
gdb on Linux x86 may be forgiven for wondering if I’ve been drinking.
As it happens, I’ve been drinking “Bat-currant Ribena” (the carton
has a disclaimer on it saying “does not contain real bat”, which is
reasonable as they’re a protected species), but that’s not relevant:
let me assure you that the contrast with, say, linux alpha (tru64
alpha is even worse) is readily apparent.
I watched “A Life Less Ordinary” on tv last night. If I had the
year 1997 all over again, I’d have paid more attention to the music.
AMD64 SBCL now builds 50-odd files before running into an array type
Tue, 03 Feb 2004 03:00:23 +0000
Another number you weren’t interested in: up from 50 to
dan@pup:~> find ~/src/sourceforge/sbcl/obj/from-xc/ -type f |wc -l
196
object files built by the cross-compiler.
A convenient x86 tree of approximately the same vintage has 284
object files in it, so by some arbitrary metric we’re about two thirds
of the way through. On the way we have however left undone more than
a few of the things which we ought to have done, so it’s quite
reaonable to suppose that there is no health in this.
Another number you weren't interested in: up from 50 to
Wed, 04 Feb 2004 02:28:30 +0000
2^8 files compiled
dan@pup:~> find ~/src/sourceforge/sbcl/obj/from-xc/ -type f |wc -l
256
Currently wondering whether we can rewrite the ALLOCATION function and
friends so that OAOOM is less of a concern. At present it involves lots of
cut & pasted x86 assembler functions with names matching
alloc_{8_,16_,}to_e{ax,bx,cd,dx,si,di}), and if I
can’t then I need to copy and paste the code another eight
times to cater for r8-r15.
2^8 files compiled
Wed, 04 Feb 2004 06:25:28 +0000
One more meaningless milestone to share:
//doing warm init
This is SBCL 0.8.7.25, an implementation of ANSI Common Lisp.
More information about SBCL is available at <http://www.sbcl.org/>.
SBCL is free software, provided as is, with absolutely no warranty.
It is mostly in the public domain; some portions are provided under
BSD-style licenses. See the CREDITS and COPYING files in the
distribution for more information.
fatal error encountered in SBCL pid 19288:
unknown core file entry: 0
There's no LDB in this build; exiting.
dan@pup:~/src/sourceforge/sbcl> ls -l output/cold-sbcl.core
-rw-r--r-- 1 dan users 62128128 2004-02-03 21:44 output/cold-sbcl.core
I guess I forgot some stuff for the target-features list. The next
step, after I’ve been to bed for a few hours, is to rewrite the
assembler glue that I copied from x86 – and probably quite a lot of
the c-call lisp glue which I obtained in similar fashion – for x86-64
calling conventions. This thing has registers, and it uses quite a
lot of them for passing arguments and return values to/from functions.
One more meaningless milestone to share:
Sat, 07 Feb 2004 04:24:38 +0000
Welcome to sigtrap_handler – please drive carefully. When I left you
last time we’d got as far as genesis – the process that generates the
initial core file. Since then
- rewrote bits of call_into_lisp (assembler thing that
calls the first lisp function) to expect its arguments passed in
registers and returned in other registers a la x86-64 calling
convention. Then it jumped into lisp and executed 4 instructions or
so
- before hitting a RIP-relative move instruction which I don’t
remember writing. Ah, so there are bits of the addressing mode that
have been stolen for this and I need a SIB byte if I want the previous
behahiour
- and, because I am thick, I was encoding the register number mod 8
instead of mod 16, so using rax when I wanted rbp (uh, really. The
sbcl x86-64 assembler inherits the cmucl x86 assembler’s decision to
represent registers as 0,2,4,6… instead of 0,1,2,3… apparently
because it makes the al+ah=ax overlap easier to deal with).
- observed that there were sequences of 0 bytes in random places.
Eventually realised I was writing 64 bit immediate constants out for
opcodes that only want 32 bit immediates (which is almost all the
instructions that take immediates at all, in fact). So, restored
the fixup vector to 32 bits in width
- which turned up a bunch of other places that I’d unthinkingly
widened to 64 bits but which really shouldn’t be: many VOPs with the /C
(constant) modifier are using unsigned-num as an argument
type, which causes the backend to attempt to use them for anything up
to 64 bits even though the instructions they use can only cope with 32.
So, fixed lots of those
- fixed up some of the iffier code for accessing ucontexts in the
runtime
This is SBCL 0.8.7.25, an implementation of ANSI Common Lisp.
More information about SBCL is available at <http://www.sbcl.org/>.
SBCL is free software, provided as is, with absolutely no warranty.
It is mostly in the public domain; some portions are provided under
BSD-style licenses. See the CREDITS and COPYING files in the
distribution for more information.
internal error #31
SC: 14, Offset: 22 0x7fbfffef60: even fixnum: 137170516952
fatal error encountered in SBCL pid 8253:
internal error too early in init, can't recover
Internal error 31 is apparently “UNKNOWN_KEY_ARG_ERROR”, which I must
admit is not what I was expecting. Usually it’s “object is not type foo”.
Welcome to sigtrap_handler - please drive carefully
Sun, 08 Feb 2004 15:12:17 +0000
Something odd is happening somewhere in the assembler. Here’s a
bit of the SBCL source code
(defun allocation (alloc-tn size &optional ignored)
(declare (ignore ignored))
(flet ((load-size (dst-tn size)
(unless (and (tn-p size) (location= alloc-tn size))
(inst mov dst-tn size))))
(let ((caller-saves (list rax-tn rcx-tn ;rdx-tn rsi-tn
;rdi-tn r8-tn r9-tn r10-tn r11-tn
)))
(dolist (r caller-saves)
(inst push r))
(move rdi-tn size)
(inst call (make-fixup (extern-alien-name "alloc") :foreign))
(move alloc-tn rax-tn)
(dolist (r (reverse caller-saves))
(if (location= r alloc-tn)
(inst add rsp-tn 8)
(inst pop r)))))
(values))
As the name suggests, this function outputs some code for allocating
memory that Lisp will use: it frobs some registers around, then calls
the C alloc() function, arranging that its return value ends
up in alloc-tn, whichever register that is. At least, that’s what
it’s supposed to do – apart from anything else, you’ll notice that
it’s not actually saving most of the registers it’s supposed to save.
But it’s not actually relevant what the output code does right now; what’s
important is we don’t seem to be getting the right stuff assembled.
Witness:
0×000000000b22c57f: push %rax
0×000000000b22c580: push %rcx
0×000000000b22c581: mov $0×10,%rdi
0×000000000b22c58b: callq 0×4162d0 <alloc>
0×000000000b22c590: rexXYZ mov %r8d,%r11d
0×000000000b22c593: pop %rcx
0×000000000b22c594: pop %rax
or if you’d like to see that mov instruction in more detail
(gdb) x/8bx 0×000000000b22c590
0xb22c590: 0×47 0×8b 0xd8 0×59 0×58 0xc7 0×03 0×12
That ‘rexXYZ’ prefix is presumably gdb’s disassembler getting more
than slightly confused by our 0×47 rex byte. I’m reasonably certain
that %r8d should be %rax, likewise supposing that the operand size
(rex bit 3, counting from lsb=0) should be 64 not 32, and I’ll give
odds of about 1 in 2 that that’s r11 not rdx.
Strangre thing is, even after placing assertions where we output
the rex prefix that “you’re not outputting #x47, are you?” it still
doesn’t stop. And there’s still #x47 bytes in the core file. Very
odd.
Something odd is happening somewhere in the assembler
Mon, 16 Feb 2004 09:50:25 +0000
It was a fixup problem (again). Having successfully proved that there
was nowhere I was writing #x47, but that said octet nevertheless
appeared in the output, there were only as many suspects as there were
places that rewrite bits of the generated code before it’s dumped.
I found that out about a week ago, and haven’t looked at AMD64
since, owing to being absurdly busy – mostly busy sitting on public
transport journeying to and from London, admittedly, but with a laptop
that (a) runs for about 40 minutes on a full battery, (b) isn’t amd64
anyway, that’s basically just downtime.
Job continues fine. Weekend consumed mostly by araneida/postgresql
performance experimentation (I’d like to say “tuning”, but that
implies a goal and a sense of direction largely absent from my
meanderings)
It was a fixup problem (again)
Fri, 20 Feb 2004 00:00:32 +0000
More postgres, mostly. And more rollerblading: still looking for the
optimal route between Baker St and Old St. Distance is not the overriding concern: more important
is good surface and the minimum number of (a) junctions at which I
have to give way, (b) ambulant obstacles.
Is this an appropriate place to mention that I’m due to move to
London in about a month from now, and therefore (should be) looking
for somewhere to live?
More postgres, mostly
Thu, 26 Feb 2004 23:35:07 +0000
The latest iteration of my standard c.l.l rant:
> It's a business advantage when other people don't use "better"
> programming languages.
Meanwhile it’s a quality-of-life disadvantage for every software
system you use but are not in competition with the suppliers of. I
regularly use or am affected by the workings of many many more
computer systems than I have any hope of personally reimplementing in
Lisp, and every time one of them fucks up it affects me.
When I can find Lisp-using alternatives for my desktop software, all
the web sites I visit regularly, my bank, my government, the utilities
that provide my gas, electricity, and telephone service, the shops I
buy groceries from, the airlines and travel agents who organise my
vacations and business travel, and all the thousands of other computer
users who impact on my daily life, then I’ll buy this “popularity
doesn’t matter” argument. Until then, I have to (and for that matter,
choose to) exist in a social environment, and I suspect that the same
is true even for entrepreneurs.
The latest iteration of my standard c.l.l rant:
Sat, 28 Feb 2004 00:09:05 +0000
This entry is more in the nature of “hello, I’m still alive” than
“hey, see the cool stuff I’ve hacked on lately”. I suspect that
you’re not so interested in the news that I think I’ve optimised my
route for skating from Baker St to the office (one way systems are
involved; the return journey still needs some work).
In no particular order
- sbcl amd64
- sbcl threading special variable binding changes: in the interests
of predictability, make new threads start witha default set of special
variable values/bindingsd instead of whatever the parent had.
- a clim-based postgresql client to kick psql’s ass ($dayjob uses a
lot of postgres, so this is my excuse to install lisp on my work
machine). It’s going to interrogate posgres metadata so the user can
navigate through the database by clicking on field names, stuff like
that. Colourised/marked up illustrations of the query plan, perhaps
some kind of interface to help with query generation, etc etc. Lots
of ideas I haven’t quite locked down that need prototyping before I
can tell if they’re good ones or not.
- longterm handwavey project: a lisp-based “thin platform” based on
SBCL/Linux that’s easily scriptable.
Installing SBCL on my work machine provided an illustration of just
how far we’ve come lately. Untar and install sbcl then
* (require 'asdf-install)
* (asdf-install:install 'clx)
;; fiddle with cliki for 2 minutes to add a :package link for pg
* (asdf-install:install 'pg)
and I have graphics, sockets, and database connectivity. McCLIM took
a little longer, but it was still pretty straightforward. All told,
it took me less time to get lisp-based web serving environment up than
it did to install coldfusion, and that’s a real commercial product.
(Actually, it took less time to install the lisp-based web serving
environment than it did just to unpack coldfusion, and it eats
less ram, too)
This entry is more in the nature of "hello, I'm still alive" than
Sun, 29 Feb 2004 20:04:25 +0000
Is this cool, or what?
No, it’s not really an Apple II port of CMUCL, it’s the
xscreensaver apple2 hack. Still cute, though.
Is this cool, or what
Wed, 03 Mar 2004 16:31:23 +0000
I’m thinking hard about the cirCLe project at the moment: not the
attempt last year to produce and charge for a CD full of Lisp
software, but the overall objective of a free lisp-based system. This
particular chunk of text started life as “Why I must write cirCLe” in
the project goals document, but clearly it wants to be longer than the
single paragraph demanded there. It also wants to be coherent, and
isn’t entirely yet, so consider this an early draft
The GNU operating system is nearly done. The OS kernels, libraries,
basic programming tools and Unix-compatible utilities in systems based
on Linux and other free Unixlike kernels are good enough that they’ve
displaced proprietary Unixes and other operating systems for many
purposes. What next? One obvious answer is “applications”
For many hackers, working on free operating system components has been
an opportunity to write code that they themselves want to use
(“scratching an itch”) and the resulting code has been reliable and
high-quality. But now we’re writing for “nontechnical” people, and
we’re to believe that
- the typical user can’t even understand our user interface, never
mind contribute code to our projects
- software should be designed to preserve this state of affairs
This is only partly true. The user should not need to know the
innards of a piece of software to use it effectively, but there’s a
difference between making it unnecessary to learn more and making it
difficult to learn more.
I’m not suggesting that anybody intends to make it difficult, but when
the typical user is not expected even to try to investigate their
software, then extensibility, remote control and programmable
interfaces are – quite reasonably – going to be a low priority.
Better just to work on the features that the user wants.
So what’s likely to happen? Users have no way to contribute back to
the software they’re using. Everyone knows the first rule of
technical support: “``the person on the other end of the phone is an
idiot’’. Curiously, this applies no matter whether you’re asking the
questions or answering them”. Without the basis for mutual respect
between the people writing software and the people using it, do we
really have a community? More pragmatically, if the only people with
the expertise to make changes to our free software system are the
people who wrote it, are we really so much better off than we would
have been with a proprietary system?
I contend that the answer is “no”; that this disconnect between
developers and users is harmful, and that we need a system where the
users and the developers are on equal terms and have the same
opportunities and the same tools. So we must use a language
expressive enough that we can write the whole system in it; where
the extension language and the implementation language are the same.
I'm thinking hard about the cirCLe project at the moment: not the
