Latest in my ever-expanding series of diversions from NixWRT is this
foolish attempt to write a Wayland compositor using a language I don't
actually understand. I'd add:
not for the first time , except that now I check the link I see that was going to be a text editor not a compositor.
Anyway. This time it's a Wayland compositor, using the wlroots
library (described by its author as "about 50,000 lines of code you
were going to write anyway") and written in Lua. Mostly because when I
get a bit further along with it I'm going to integrate
Fennel into it, and then with a bit of luck
will have a respectable replacement for the once-brilliant but now
rather long in the tooth
Sawfish . I say "when" but
we all know I really mean "if".
At the time of writing it works to the extent that I have a Lua script
that can set up the display appropriately then allow me to display a
Konsole window into which I can type, and it renders a pointer which
moves around in expected and unsurprising ways when I stroke my
touchpad. It has no knowledge, though, of window decorations or of
stacking order or even of focus or the channeling of pointer
button/movement events onto a client.
(Decorations? Yes, I'm with the KDE guys on this one. Requiring
clients to render their own window borders and resize themselves on
the screen is like a bank requiring customers to track their own
account balances and know which market funds their deposits were
invested in: how can you ever be sure the app you thought was
"Password Wallet Master Key Entry" was actually that and not, I dunno,
"I'd like to add you to my professional Botnet" if you don't have some
kind of trustable fence around the rectangle it's permitted to draw
into. Honour system?)
Some brief notes on how I'm doing it, for the benefit of future-me and
potential benefit of present-other. I'm using LuaJIT: not for speed
but because the FFI can parse (most) C header files and there's a
lot of FFI in this project. So far I have made a fairly literal
transcription of the code in Drew DeVault's blog
posts
parts I-III, and then a rather sketchier interpretation of the code in Input handling in wlroots
Some stuff has changed since the blog posts were written. Here are
the differences and/or relevant gotchas that I noted or can remember
wlr_renderer_begin now takes int width, int height as second and third params instead of needing a wlr_output
wlr_compositor.surfaces is probably the old name for what is now wlr_compositor.surface_resources
wlr_render_with_matrix has become wlr_render_texture_with_matrix
without an xkb keymap set (wlr_keyboard_set_keymap) you won't get
any key events. You might think there's no need for keymaps if all
you want is to look at the raw codes, but nope, it doesn't work.
wlr_seat_set_capabilities is not mentioned in the blog, but you
need it set right according to the inputs available at your "seat"
to render the mouse pointer when you're running wayland-under-X (for
development, before the server is ready for dogfooding) you need to
call wlr_output_render_software_cursors(output, nil) while
rendering output. So you don't leave a ghost trail of former
pointer positions whenever the mouse is moved, you probably also
want to call wlr_renderer_clear as the first thing after
wlr_renderer_begin
there's a bit more to wlr_xcursor than it explains, and TBH I'm just going to point you to my diff instead of trying to explain it here. Because while as a general rule I would never claim that "life's too short" for doing fun things, I must concede that the remaining available time for sleeping before my alarm goes off tomorrow morning is rapidly getting that way.
There are a few reasons I've not been blogging much lately, but chief
among them is that a month or so ago my laptop was crushed by a taxi,
and setting up my blogging environment anywhere else was deemed Too
Much Of A Faff.
Happily my contents insurance company eventually agreed to pay me
money to replace it, and now I have replaced it, and various kinds of
pre-crush activity including blogging and nixwrt hacking may now
resume. This entry is mostly just a placeholder and "welcome back"
message, but it does contain a link to my presentation at Nixcon 2018
An unusual - and potentially ill-advised - method and apparatus for
the querying of an external database from a
re-frame
Single Page Application.
Bit of background
Re-frame, is described by its author as "a pattern for writing
SPAs in ClojureScript, using Reagent" and "... impressively
buzzword-compliant". Both these things are true.
The documentation talks a lot about "dominoes": there is a six step
loop that goes around and around.
events (these are re-frame events, not DOM events) are dispatched.
This might be in response to timers or to user commands - usually
you'd put DOM event handlers (on-foo and friends) on document
nodes that call re-frame/dispatch. (If you're wondering how those
elements came to exist, they were rendered in the previous iteration
of the loop.)
event handlers are pure functions which are called by the framework
when the events they're registered to are dispatched. They receive
the value of the application state (app-db), and return an updated
value. (This is an over-simplification: in general, event
handlers are pure functions which return a description of some
effect which something else needs to make happen, but updating the
app-db is almost always a required effect so in practice we tend
to focus on it.)
effect handlers are the magic that do side effects based on the
values returned by the event handlers in the previous stage.
There's one builtin to update the app-state, and you can plug others
in to e.g. do XHR or other stuff.
"query" functions - also known as subscription handlers - are
registered, which do computations (or just lookups) in the
application state.
"view" functions call query functions as required and generate
hiccup (HTMLish clojurescript data structures). These are made into
React components for you, and you attach your top-level component to
some element (typically a DIV) in an otherwise static HTML document.
Reagent/React does its thing: every 16ms, it renders your components
using the minimal amount of DOM changes to make its model of what is
match its model of what should be.
Then the user clicks on things, and DOM event handlers are called, and
the handlers call dispatch, and the loop goes around again.
Signal graph
When you read the example application, the query functions you see are
so simple that you might wonder why we even bother with them -
;; 4 lines of code to register a query called :time that returns
;; an app-db key of the same name, what is the point of this?
(rf/reg-sub
:time
(fn [db _] ;; db is current app state. 2nd unused param is query vector
(:time db))) ;; return a query computation over the application state
but where it gets interesting is that
queries can take parameters
queries can depend not just on app-db but on the result of other queries, and
queries are evaluated only when called for, and only when their inputs change
so we actually have a query graph with the app-db at one end of it and
the view functions at the other end, and only the relevant bits of it
are computed when needed. Hold onto that thought
Prior art
The re-frame documentation has an incredibly useful page on Subscribing to external data which describes two sensible ways to do it and warns against a third:
With subscriptions
If we start from the premise that components must always get their
data from a subscription handler, the obvious route is to create a
subscription handler which kicks off an async request to the external
data source, then returns a reaction which wraps some path within
the app-db where the results will be found when ready. The async
request, when finished, runs a callback function which stores the
results in the said path.
Bit of a learning curve here if you've only previously looked at the
example app, because you won't have learned what a reaction is. A
reaction is a "a macro which wraps some computation (a block of code)
and returns a ratom holding the result of that computation" and a
ratom (short for react atom) is like a regular clojure atom but with
extra gubbins so you can subscribe to it and be told every time it
changes.
With events
With a different premise, we get a different answer. Your requirement
to look something up externally is most probably driven by some user
command which is mapped onto a reframe event. In this approach, the
event handler is responsible for getting the data it needs to handle
the event, and pushing it into the app-db. Probably this means you
have one event to initiate the transfer which dispatches another event
when the server eventually responds.
Using React lifecycle methods (don't do it)
Views should be dumb. Don't do this.
A third way (unless it is the fourth way)
What if we said: the external data lookup is notionally a "pure
function" (let us suppose that the server gives the same response
every time it is provided with the same inputs, and for the moment let
us handwave over error conditions), so should sit in the middle of the
signal graph somewhere and provide its output as a subscribable value
instead of scribbling into app-db.
The reason I started doing this was that I was trying to write an XHR
subscription handler per the first approach above which subscribed to
app-db so that it could find the query terms, and because it was also
writing to app-db, it was triggering itself. Probably I was doing
something wrong (it occurs to me now that it should probably have
subscribed to a query of only the search term and not to the entirety
of app-db) but that thought did not occur to me at the time, so this
pushed me into looking for another way.
So let me start by showing some example code, and then I can try explaining it:
What are we doing here? The value we want to send on through the
signal graph is not returned by anything we call, it's provided as an
input to our ajax callback function. This means we can't use the
reaction macro, so we have to create a reaction by hand and take
care ourselves of updating the value it wraps. To do this we use
reagent.ratom/run!, which (to my limited understanding) is kinda
sorta half of the reaction macro - like reaction it runs a loop
every time the subscriptions change but unlike reaction we have to
call reset! on our ratom every time we want to provide a new value
downstream.
On the whole I think I like this pattern, at least for services which
(we can reasonably pretend) are functional - i.e. they give the same
answer every time when called with the same inputs. The external
service doesn't put stuff in app-db, which I guess might be an issue
if it needs to be merged with other data from other sources or if
multiple downstream subscriptions want to use it (but in that case why
can't they subscribe to it?) but I haven't/can't see how that
hypothetical would become real.
I hadn't realised, until I checked the site, that I'd completely
forgotten to blog last week.
Nor had I been entirely aware, until I checked on "Mastodon":
https://mastodon.social/web/accounts/28445 , that it's been an entire
month since I put NixWRT down to make Epsilon happen. Some weekend
hack that turned out to be. Bother.
On the bright side, I haven't needed to launch K-9 for a bit over a
week and I am learning reagent/re-frame/FRP. On the less bright side,
I don't have a lot to blog about right now.
Slightly over a year ago I wrote about how to build a Clojure project with Nix and this week I had occasion to refer back to it because I want to have a production-style dogfood build of Epsilon
(Note that the clojure command will fail silently if it cannot find
a java runtime on the path, or if the current directory contains a
malformed deps.edn file. Each of these problems took me a while to
find - I tell you this so that you may learn from my mistakes)
The second half of the puzzle - build time - is not much changed from
how we did it last time around, though I did take the opportunity to
make it use the generic builder instead of making its own
builder.sh.
Probably I should point out that this downloads all the dependencies
into /nix/store as JARs and builds a classpath so the JRE can find
them - it doesn't make an uberjar. This fits my current use case,
because I want to run it on a NixOS box and separate jars provides at
least the possibility that some of them might be shared by more than
one app. I will obviously have to come back to this if/when I need to
build an uberjar for distribution.
Also, if you are doing advanced things with CLJS external libraries
(like, Node dependencies and stuff) then this will not help and you
are on your own. There is a name for the (CL)JS dependency ecosystem,
it's a compound word in which the first part is the collective noun
for a collection of stars, and the second part rhymes with "duck"
