Another huge long gap between updates. I'd say that you might as well
get used to it, but you probably are already. This is a short note to say that I have developed a Capistrano extension which runs Puppet as part of an app deploy, and you can read about it on the Simply Business tech blog
In other news, having confirmed that the X11 touchpad
support is broken by design - the
choice of whether to emulate mouse events (left click, middle click,
drag etc) is set at a global level in the driver and cannot be overridden
per client, so legacy mouse-only clients cannot meaningfully coexist
with clients that usefully could handle touch events directly - I am
working on a client implementation of the Wayland protocol in Clojure.
I hope this will eventually will turn into a habitable text editor.
Ignoring, because I can, this whole Ubuntu Mir thing completely, I
have begun to learn stuff about the Wayland protocol (and about
Clojure, with which I am still at the
constantly-having-to-google-stuff stage). Some random notes on what I
have learnt so far
First off: Java has no builtin support for talking to unix-domain
(AF_FILE) sockets. And nobody seems to make a Maven-packaged library
that does it either. This is a shame because Leiningen makes Maven
manageable, but anything unmavened involved tedious mucking around.
Eventually I did
which seems to be holding up. Then I opened a socket and tried reading from it
in the hope of getting some lovely protocoly stuff
(defn open-socket [name]
(let [s (cx.ath.matthew.unix.UnixSocket. name)
in (. s getInputStream)
out (. s getOutputStream)
]
{:socket s :input in :output out}))
and watched it hang. After some time looking at documentation and
mucking about with strace to see if it was trying to read the full
buffer instead of doing the short read I wanted it to, I eventually
thought to use socat instead of clojure. It turns out that the
client is expected to make the first request before the server sends
anything, and with the use of strace weston-info I was able to find
out what.
Time to start writing some code to parse
wayland.xml
so we can actually find out what this means. The usual way to do XML parsing in clojure seems to be using
zippers and the easy examples seem to be somewhat lacking or slightly
ungooglable. You need a project dependency on org.clojure/data.zip
and a bunch of package requires, then you call clojure.zip/xml-zip on
the return value of clojure.xml/parse and that gets you a zipper
where I have conveniently left out the definition of parse-protocol
and everything it calls because it's longwinded and tedious (but the
code will be on github as soon as I'm not ashamed of it) but it might
hypothetically do things like
(x/xml-> my-zipper :protocol :interface :request)
to descend the tree through <protocol> <interface> <request>
and return all the elements. Use the similarly named x/xml1-> to
get the first matching element only. The return values from these things
are themselves zippers and you can call up, down etc - or xml->
again - to traverse the tree further, then eventually call node when
you want to get the element itself. So e.g.
(defn parse-interface [i n]
;; n is a badly named zippered xml object thingy
(let [el (z/node n)
requests (map-indexed parse-message (x/xml-> n :request))
events (map-indexed parse-message (x/xml-> n :event))
enums (map parse-enum (x/xml-> n :enum))]
{:index i
:name (:name (:attrs el))
:requests requests
:events events
:enums enums
}))
So let's handwave over the details and take it on trust that I have
parsed the whole file. There are two other things I discovered -
mostly thanks to the #wayland IRC channel participants - about the
wayland wire protocol that the docs don't mention:
where it says "The first word is the sender's object id (32-bit)",
when it's describing a message sent from client to compositor, what it
means is "The first word is the target object's id (32-bit)".
object id 1 is special: it refers to the core global singleton
object, which implements the interface wl_display
Given which, we can attempt to parse that first client->compositor message
Next up, some code to create messages. And something, which may
involve an atom, to map object ids to their corresponding interfaces
as we learn them. After that, we find out what the Wayland FAQreally means by "shareable buffer"
Addendum, penned the following morning
Because this is supposed to be a log of what I am learning, and I had
never previously learned about the -> operator (seen it, never used
it), for completeness' sake:
(-> x fn1 fn2 fn3)
is the same thing as
(fn3 (fn2 (fn1 x)))
Googling for clojure -> is not a terribly fulfilling endeavour, so
it's helpful to know that it's also referred to as the 'arrow'
operator, the 'threading' operator, and apparently also the 'thrush'
operator. This
blog article
from Fogus explains further.
Yesterday's lunchtime hacking was all about splitting the project into
multiple files and getting it into git and onto Github - note that the mere fact of
it being publically browsable does not imply that it will run,
build, walk, make tea, perform any other useful function, or even
forbear from exploding inside your computer and rendering the SSD to
molten slag. Nor that I'm not still ashamed of it. It just keeps me
slightly more honest.
Today I implemented enough of pack-message to be able to recreate
the initial client->compositor message that we observed weston-info
send last week. Still taking extraordinary liberties with signed vs
unsigned longs, and plase note that all this code will work only on
little-endian machines (there are any big-endian machines left?).
Lessons, puzzles
Leiningen does not need you to list the source files in your
repository individually: it finds them magically. I believed
otherwise for a while, but it turned out (slightly embarrassingly)
that I had a parenthesis in the wrong place. My working hypothesis is
that it assumes there is one namespace for each file, and that any
reference to a namespace it doesn't know about it can be satisfied by
loading a file with that name.
If I type (in-ns 'psadan.core) at the repl and that ns does not
include a (:refer-clojure) form, I can't use the symbols in
clojure.core at the repl. I have not observed a similar issue wrt
uses of clojure.core/foo in core.clj itself, just at the repl.
atoms! An atom is dead simple, really - conceptually at least, if not
also under the hood: it's a wrapper for an object that lets you look
inside with deref and lets you change what's inside with swap!.
For each connection we use an atom holding a mapping from object ids
to the corresponding objects, which starts out holding the singleton
object for wl_display and then needs to be updated each time we
generate an object locally and each time we learn of a new object from
the peer.
(defn open-connection [name]
(let [s (cx.ath.matthew.unix.UnixSocket. name)
in (. s getInputStream)
out (. s getOutputStream)
wl-display (global-object-factory)
]
{:socket s
:input in
:output out
:display wl-display
:objects (atom (assoc {} 1 wl-display))
}))
(defn remember-object [conn id object]
;; (swap r fn args...) gets the current value of the atom inside r,
;; which for the sake of argument we shall call oldval, then sets the atom
;; to the result of calling (fn oldval args...)
(swap! (:objects conn) assoc id object)
object)
(defn get-object [conn id]
;; @foo is another way to write (deref foo)
(let [o (get @(:objects conn) id)]
o))
I have probably not chosen the fastest possible way of building up the
messages I plan to send, in terms of fiddling around sticking vectors
of bytes together. Will worry about that later if it turns out to be
a bottleneck (but suggestions are welcome).
There was not a lot of Wayland learning this time. In the next round
we shall be sending it the messages we have so lovingly composed from
whole cloth and see if it reacts the same way as it did when the same
bytes were sent from weston-info
In the next round we shall be sending it the messages we have so lovingly composed from whole cloth and see if it reacts the same way as it did when the same bytes were sent from weston-info
And the answer is ... yes, pretty much. We had to fix up our string
parsing to make sense of the replies, but watch:
My interpretation of what's happening here is that we're sending to
the server a 'tell object 2 about all your global objects' message,
followed by a 'tell object 3 done when you're finished doing stuff'
message, and as you can see from the output, the reply is a bunch of
ids for global objects sent to object 2, a done event sent to object
3, and then a delete_id event for object 3 sent to object 1. I'm
actually not sure why that last one triggers, as I don't think I asked
it to. Perhaps it's just tidying up for me.
I'm also handwaving - if not actually handdrowning - a litle bit,
because really ... are these :global messages sent to object 2 or
from object 2? For the moment, I am using the two directions
interchangeably, which is probably not a recipe for an easier future
life, but in the meantime I can continue to tread water.
It's instructive, or at least reassuring, to compare this stuff with
what weston-info says:
Top Wayland tip for today: it appears to be the case that you can make
the C library clients log protocol exchanges to stderr by putting
WAYLAND_DEBUG=client in the environment. When doing that it's clear
to see that weston-info is making a couple of additional requests
that we're not. We could add them, but I think the more pressing
concern is to make it do something with the events we're getting
already - if it's sending us details of global objects that we might
need to know about, we should at a minimum be storing those details
somewhere instead of throwing them away ...
The latest round of psadan hacking was motivated by two goals
that it would be good to actually remember the globals we're
receiving when we send get_registry to the magic registry object
that if we're going to ask for a callback when all the globals are
notified, we should wait for it before continuing.
We went down a couple or three dead ends on our way to this goal, but
eventually we settled on creating an agent responsible for listening
to the connection (I called it channel, in the absence of any better
ideas) and dispatching (using a handle-message multimethod) to some
code appropriate for each kind of message we're receiving.
Learnings
multimethods the clojure way are pretty versatile: you can dispatch
on pretty much any property - or derived property - of the function
argument, not just on type. In our case that's the interface name
and the message (event) name:
we handle the "tell me when you're done" requirement with a
promise. The get-globals code adds an unfulfilled
promise to the callback object it creates, then once it has sent out
its messages it derefs the promise , causing it to wait until
something delivers the promise. That something is the
handle-message implementation for wl_callback, for which, see above.
we were trying to map handle-message onto each of the messages
parsed out of the buffer, but not doing anything with the result.
Given that map is lazy, this meant our handle-message code was
for the most part not being called. Surrounding the map form with
a dorun fixed this.
you send work to an agent with (send fn ...) or (send-off fn ...)
which invoke the fn with the current state of the agent, not with
the agent itself. Which is fine but offers no facility for the
agent to send work to itself - happily, the global/magic/special
variable *agent*, which evaluates to the currently running agent
if any is, works nicely for this purpose
the 'name' field in a global is (confusingly) a number, and (more
confusingly still) not an object id. Object number 3 in our client is a
wl_callback object, whereas the global named 3 is .. well, let's
check ...
Next up? At some point we need to decide whether sending messages
should be done by the channel or whether it's OK to carry on doing
that directly in whatever thread we happen to be in. But what would
be much more fun is to see if we can actually render a window...
Why are our bind messages to wl_registry erroring out?
15:19 < daniels> yeah, wl_registry_bind is a special case - i think it's the
one non-autogenerated marshaller we have
15:32 < daniels> actually no, sorry, i'm lying through my teeth
15:32 < daniels> it's no longer autogenerated
15:32 < daniels> but the parser expands new_id to interface + version + id
15:33 < daniels> it used to be hand-written, but is now autogenerated
15:33 < daniels> http://cgit.freedesktop.org/wayland/wayland/tree/src/scanner.c#n614
16:28 < jekstrand> dan_b: It is a somewhat-special case. Basically, every time
there's a new_id that does not have any type information
specified, two aditional pieces of information (interface
name and version) get added.
16:29 < jekstrand> dan_b: That really should be documented, but it's not. I
had to grep through the scanner.c file to find it all.
Armed with this additional info the fix was fairly straightforward: I
augmented the XML parsing/protocol generation to stick an additional
pair of arguments into each message that contains a new_id and no interface.
Thanks to both daniels and jekstrand on irc for clearing it up.