diary at Telent Netowrks

AMD64 SBCL now builds 50-odd files before running into an array type#

Mon, 02 Feb 2004 13:21:02 +0000

AMD64 SBCL now builds 50-odd files before running into an array type problem. Said files are doubtless still riddled with garbage; the easiest way to see the disassembled output will be to get all the way through make-host-2 (building the target compiler) then starting up the the new runtime with the cold core thus generated, so I'm not worrying about that just now. There is a disassembler in SBCL, but it's no more likely to be correct than the assembler is, so let's hope gdb on linux x86-64 is up to the standard of gdb on linux x86 as opposed to the less well-supported platforms like, ooh, almost anything else it claims to run on.

Anyone reading this whose workday is in large part concerned with gdb on Linux x86 may be forgiven for wondering if I've been drinking. As it happens, I've been drinking "Bat-currant Ribena" (the carton has a disclaimer on it saying "does not contain real bat", which is reasonable as they're a protected species), but that's not relevant: let me assure you that the contrast with, say, linux alpha (tru64 alpha is even worse) is readily apparent.

I watched "A Life Less Ordinary" on tv last night. If I had the year 1997 all over again, I'd have paid more attention to the music.

Another number you weren't interested in: up from 50 to#

Tue, 03 Feb 2004 03:00:23 +0000

Another number you weren't interested in: up from 50 to 

dan@pup:~> find ~/src/sourceforge/sbcl/obj/from-xc/ -type f |wc -l
    196

object files built by the cross-compiler.

A convenient x86 tree of approximately the same vintage has 284 object files in it, so by some arbitrary metric we're about two thirds of the way through. On the way we have however left undone more than a few of the things which we ought to have done, so it's quite reaonable to suppose that there is no health in this.

2^8 files compiled#

Wed, 04 Feb 2004 02:28:30 +0000

2^8 files compiled

dan@pup:~> find ~/src/sourceforge/sbcl/obj/from-xc/ -type f |wc -l
    256

Currently wondering whether we can rewrite the ALLOCATION function and friends so that OAOOM is less of a concern. At present it involves lots of cut & pasted x86 assembler functions with names matching alloc{8,16,}toe{ax,bx,cd,dx,si,di}), and if I can't then I need to copy and paste the code another eight times to cater for r8-r15.

One more meaningless milestone to share:#

Wed, 04 Feb 2004 06:25:28 +0000

One more meaningless milestone to share:

//doing warm init
This is SBCL 0.8.7.25, an implementation of ANSI Common Lisp.
More information about SBCL is available at <http://www.sbcl.org/>.
SBCL is free software, provided as is, with absolutely no warranty.
It is mostly in the public domain; some portions are provided under
BSD-style licenses.  See the CREDITS and COPYING files in the
distribution for more information.
fatal error encountered in SBCL pid 19288:
unknown core file entry: 0
There's no LDB in this build; exiting.
dan@pup:~/src/sourceforge/sbcl> ls -l output/cold-sbcl.core 
-rw-r--r--    1 dan      users    62128128 2004-02-03 21:44 output/cold-sbcl.core

I guess I forgot some stuff for the target-features list. The next step, after I've been to bed for a few hours, is to rewrite the assembler glue that I copied from x86 - and probably quite a lot of the c-call lisp glue which I obtained in similar fashion - for x86-64 calling conventions. This thing has registers, and it uses quite a lot of them for passing arguments and return values to/from functions.

Welcome to sigtrap_handler - please drive carefully#

Sat, 07 Feb 2004 04:24:38 +0000

Welcome to sigtrap_handler - please drive carefully. When I left you last time we'd got as far as genesis - the process that generates the initial core file. Since then

  1. rewrote bits of callintolisp (assembler thing that calls the first lisp function) to expect its arguments passed in registers and returned in other registers a la x86-64 calling convention. Then it jumped into lisp and executed 4 instructions or so

  2. before hitting a RIP-relative move instruction which I don't remember writing. Ah, so there are bits of the addressing mode that have been stolen for this and I need a SIB byte if I want the previous behahiour

  3. and, because I am thick, I was encoding the register number mod 8 instead of mod 16, so using rax when I wanted rbp (uh, really. The sbcl x86-64 assembler inherits the cmucl x86 assembler's decision to represent registers as 0,2,4,6... instead of 0,1,2,3... apparently because it makes the al+ah=ax overlap easier to deal with).

  4. observed that there were sequences of 0 bytes in random places. Eventually realised I was writing 64 bit immediate constants out for opcodes that only want 32 bit immediates (which is almost all the instructions that take immediates at all, in fact). So, restored the fixup vector to 32 bits in width

  5. which turned up a bunch of other places that I'd unthinkingly widened to 64 bits but which really shouldn't be: many VOPs with the /C (constant) modifier are using unsigned-num as an argument type, which causes the backend to attempt to use them for anything up to 64 bits even though the instructions they use can only cope with 32. So, fixed lots of those

  6. fixed up some of the iffier code for accessing ucontexts in the runtime 
This is SBCL 0.8.7.25, an implementation of ANSI Common Lisp.
More information about SBCL is available at <http://www.sbcl.org/>.
SBCL is free software, provided as is, with absolutely no warranty.
It is mostly in the public domain; some portions are provided under
BSD-style licenses.  See the CREDITS and COPYING files in the
distribution for more information.
internal error #31
    SC: 14, Offset: 22            0x7fbfffef60: even fixnum: 137170516952
fatal error encountered in SBCL pid 8253:
internal error too early in init, can't recover

Internal error 31 is apparently "UNKNOWNKEYARG_ERROR", which I must admit is not what I was expecting. Usually it's "object is not type foo".

Something odd is happening somewhere in the assembler#

Sun, 08 Feb 2004 15:12:17 +0000

Something odd is happening somewhere in the assembler. Here's a bit of the SBCL source code

(defun allocation (alloc-tn size &optional ignored)
  (declare (ignore ignored))
  (flet ((load-size (dst-tn size)
           (unless (and (tn-p size) (location= alloc-tn size))
             (inst mov dst-tn size))))
    (let ((caller-saves (list rax-tn rcx-tn ;rdx-tn rsi-tn
                              ;rdi-tn r8-tn r9-tn r10-tn r11-tn
                              )))
      (dolist (r caller-saves)
        (inst push r))
      (move rdi-tn size)
      (inst call (make-fixup (extern-alien-name "alloc") :foreign))
      (move alloc-tn rax-tn)
      (dolist (r (reverse caller-saves))
        (if (location= r alloc-tn)
            (inst add rsp-tn 8)
            (inst pop r)))))
  (values))

As the name suggests, this function outputs some code for allocating memory that Lisp will use: it frobs some registers around, then calls the C alloc() function, arranging that its return value ends up in alloc-tn, whichever register that is. At least, that's what it's supposed to do - apart from anything else, you'll notice that it's not actually saving most of the registers it's supposed to save. But it's not actually relevant what the output code does right now; what's important is we don't seem to be getting the right stuff assembled. Witness: 

0x000000000b22c57f:     push   %rax
0x000000000b22c580:     push   %rcx
0x000000000b22c581:     mov    $0x10,%rdi
0x000000000b22c58b:     callq  0x4162d0 <alloc>
0x000000000b22c590:     rexXYZ mov    %r8d,%r11d
0x000000000b22c593:     pop    %rcx
0x000000000b22c594:     pop    %rax
or if you'd like to see that mov instruction in more detail

(gdb) x/8bx 0x000000000b22c590
0xb22c590:      0x47    0x8b    0xd8    0x59    0x58    0xc7    0x03    0x12

That 'rexXYZ' prefix is presumably gdb's disassembler getting more than slightly confused by our 0x47 rex byte. I'm reasonably certain that %r8d should be %rax, likewise supposing that the operand size (rex bit 3, counting from lsb=0) should be 64 not 32, and I'll give odds of about 1 in 2 that that's r11 not rdx.

Strangre thing is, even after placing assertions where we output the rex prefix that "you're not outputting #x47, are you?" it still doesn't stop. And there's still #x47 bytes in the core file. Very odd.

It was a fixup problem (again)#

Mon, 16 Feb 2004 09:50:25 +0000

It was a fixup problem (again). Having successfully proved that there was nowhere I was writing #x47, but that said octet nevertheless appeared in the output, there were only as many suspects as there were places that rewrite bits of the generated code before it's dumped.

I found that out about a week ago, and haven't looked at AMD64 since, owing to being absurdly busy - mostly busy sitting on public transport journeying to and from London, admittedly, but with a laptop that (a) runs for about 40 minutes on a full battery, (b) isn't amd64 anyway, that's basically just downtime.

Job continues fine. Weekend consumed mostly by araneida/postgresql performance experimentation (I'd like to say "tuning", but that implies a goal and a sense of direction largely absent from my meanderings)

More postgres, mostly#

Fri, 20 Feb 2004 00:00:32 +0000

More postgres, mostly. And more rollerblading: still looking for the optimal route between Baker St and Old St. Distance is not the overriding concern: more important is good surface and the minimum number of (a) junctions at which I have to give way, (b) ambulant obstacles.

Is this an appropriate place to mention that I'm due to move to London in about a month from now, and therefore (should be) looking for somewhere to live?

The latest iteration of my standard c.l.l rant:#

Thu, 26 Feb 2004 23:35:07 +0000

The latest iteration of my standard c.l.l rant:

> It's a business advantage when other people don't use "better" 
> programming languages.

Meanwhile it's a quality-of-life disadvantage for every software system you use but are not in competition with the suppliers of. I regularly use or am affected by the workings of many many more computer systems than I have any hope of personally reimplementing in Lisp, and every time one of them fucks up it affects me.

When I can find Lisp-using alternatives for my desktop software, all the web sites I visit regularly, my bank, my government, the utilities that provide my gas, electricity, and telephone service, the shops I buy groceries from, the airlines and travel agents who organise my vacations and business travel, and all the thousands of other computer users who impact on my daily life, then I'll buy this "popularity doesn't matter" argument. Until then, I have to (and for that matter, choose to) exist in a social environment, and I suspect that the same is true even for entrepreneurs.

This entry is more in the nature of "hello, I'm still alive" than#

Sat, 28 Feb 2004 00:09:05 +0000

This entry is more in the nature of "hello, I'm still alive" than "hey, see the cool stuff I've hacked on lately". I suspect that you're not so interested in the news that I think I've optimised my route for skating from Baker St to the office (one way systems are involved; the return journey still needs some work).

In no particular order

Installing SBCL on my work machine provided an illustration of just how far we've come lately. Untar and install sbcl then

and I have graphics, sockets, and database connectivity. McCLIM took a little longer, but it was still pretty straightforward. All told, it took me less time to get lisp-based web serving environment up than it did to install coldfusion, and that's a real commercial product. (Actually, it took less time to install the lisp-based web serving environment than it did just to unpack coldfusion, and it eats less ram, too)

Is this cool, or what#

Sun, 29 Feb 2004 20:04:25 +0000

Is this cool, or what?

No, it's not really an Apple II port of CMUCL, it's the xscreensaver apple2 hack. Still cute, though.

⟪Jan 2004 Mar 2004⟫