However, the details are a bit fiddly. Here’s a cheat-sheet I used recently for a simple TCP
service written using Erlang.
My program was a single module, running outside of any OTP application context. The
instructions here need minor emendation to either explicitly list modules to purge and reload
or to discover all modules within a single application; see the places in server-reload
below mentioning the atom my_server.
I did not use the -on_load() directive, because I wanted to be able to use multiple nodes
rather than controlling reloads from a single node’s shell repl, and I couldn’t figure out how
to make the two play nicely together.
The Erlang
I exported a code_change/0 from my module, to be called after loading a new version of the
module into a node. It sends a message code_change to each “global” actor in my program (in
this case, there was only one).
That actor distributes the notification on to any inferior actors it is managing, and then does
an “MFA” self-call to upgrade its own codebase.
Similarly, all other notified actors perform “MFA” self-calls.
Actors need to take care to manage upgrades of their state at the same time as they do the
“MFA” self-calls.
Starting the program
I wanted it to be run by daemontools, so created the
following shell script called run, which daemontools will pick up to start a service:
Pick a fresh random cookie for the -setcookie argument. I used uuid(1).
Then, I created this script, server-reload:
Running server-reload causes the source code to be compiled and hot-loaded into the running
server.
Grace notes
Then, I used a git post-receive hook to automatically recompile and reload the code on push to live:
That’s it
That’s all. The end result worked well: I used it to run a hotfix to my TCP service with many
tens of live, active connections, and not one of them noticed a thing.
I had a small insight yesterday while building a component for a small web
app: the user interface for editing an incomplete value of sum type
A+B needs to remember a product of input 2×A×B from the user:
A + B ⟿ 2 × A × B
This allows the user to ergonomically change their mind about whether
they’re building an A or a B without losing partially constructed
values.
More precisely, the UI for a value of type A+B needs in general to be
able to remember and manipulate 2×(A+1)×(B+1):
A + B ⟿ 2 × (A+1) × (B+1)
The extra 1s allow for nulls, for temporarily missing but required values.
You could similarly generalise to allow for temporarily invalid or
unparseable values.
Example
Consider UI for creating a new project in an IDE, with two available
options: create a new local project, by simply creating a new directory,
or clone an existing git repository.
Abstractly, this is roughly Str + Str×Str×Str.
The user interface for this will look something like
Here we see that while a value of type NewProject is being built, we need
to remember four strings (abstractly, Str×Str×Str×Str),
plus a boolean indicating whether we ultimately want a “local” or “clone”
project type (abstractly, 2).
All told, that’s
Str + Str×Str×Str ⟿ 2 × Str×Str×Str×Str
which exactly fits the pattern of
A + B ⟿ 2 × A × B
Generalization to bigger sums
The translation can be applied recursively, but it (harmlessly) remembers
slightly too much transient UI state,
A+(B+C) ⟿ 2 × A × (2 × B × C)
so perhaps it’s better to think about it applying directly to n-ary sums:
I noticed a bug in Guile
3.0.9’s aarch64 atomics
handling, and found a couple of apparent solutions (1, 2), but one of them
is weird enough for me to write this post.
(ETA: Nonstory. The problem was that the mov instruction isn’t idempotent! Hat tip to
Andy Wingo for figuring out what the issue was. I’ve updated the rest of the article, and I’ll
leave it here for posterity.)
Long story short, the problem was with the equivalent of C’s
atomic_exchange. Here’s the code
that Guile’s JIT was generating:
This code appears to occasionally lose writes (!).ETA: This code definitely loses
writes when interference means it has to go around the loop.
The first patch I wrote boringly replaced the lot with a single
swpal x0, x0, [x1]
which is fine, if you have an ARM v8.1 device to hand, but not if you don’t have a machine with
Large System
Extensions. So I
tried, on a hunch, the second patch, which just changed the target of the cbnz,
producing code like this:
… and the issue disappeared! What! This shouldn’t have made a difference! Should it?ETA: And fair enough, too! If the branch targets the mov instruction, the value of x0
that ldaxr set is used, meaning that the whole operation simply becomes a no-op assignment.
Are aarch64 atomics really this sensitive? Is there only One True Instruction Sequence that
should be used to implement atomic_exchange? Why does making this seemingly-insignificant
change produce such a noticeable effect?ETA: Nothing to see here :-)
As is
well-known,
JavaScript’s Promise is not a monad. It will happily treat Promise<Promise<T>> as if it was
Promise<T>:
This can bite you in unexpected ways. Imagine you have a CSP-like Channel<T> class for
sending Ts back and forth. Channel<T> might have a method like this:
There’s an obvious problem here: what if undefined ∈ T? So you make sure to note, in the
comment attached to Channel<T>, that T is not allowed to include undefined.
But the less obvious problem is that T is not allowed to contain Promise<undefined> either,
even though in other contexts a promise of undefined cannot be confused with undefined:
To see why this is a problem, instantiate T with Promise<undefined>, and look at the type
of pop():
Because JavaScript collapses promises-of-promises to just promises, this is equivalent to just
and you’ve lost the ability to tell whether pop() yielded a T or an undefined.
TypeScript does not warn you about this, incidentally. (Ask me how I know.)
Workaround
Instead of accepting this loss of structure and adding another caveat to Channel<T> to work
around JavaScript’s broken design—“T must not include either undefined or
Promise<undefined> or Promise<Promise<undefined>> etc.”—I decided to change the signature
of pop():
Now both Channel<undefined> and Channel<Promise<undefined>> are sensible and work as
expected. No more exceptions regarding what Ts a Channel may carry.
When T is Promise<undefined>, in particular, we see that the type of pop() is
Promise<{ item: Promise<undefined> } | undefined>
Because the Promises aren’t immediately nested, JavaScript won’t erase our structure.
(Ironically, we’ve introduced a monad (Maybe<T>) to fix the bad behaviour of something that
should have been a monad…)
Python 3.13 will, for inscrutable reasons, remove the crypt module from
the standard library. The excuses given in PEP
0594 boil down to “here are some
good reasons why new code shouldn’t use this module.” What about existing
code? Ah well.
So anyway, for those of us who need some way of generating $6$ SHAcrypt
SHA-512 shadow-password-database entries from Python, stick the following
module into your codebase (you can also download it, shacrypt512.py) and replace code like
direnv works fine on Windows 11, but if an .envrc tries to set the PATH, the result will
be a path in Windows format, not Unix format.1
Instead of adding eval $(direnv hook bash) to your .bashrc, try the following snippet:
This modifies the output of direnv hook bash slightly, adding code to fix path-like variables
after direnv sets the environment up.2
The variable names to unmangle are drawn from a new variable, _unmangle_direnv_names,
initially set to PATH, which should contain a space-separated list of variable names.
If, in a particular .envrc, you need path-unmangling for an additional variable, you can add
that variable’s name to _unmangle_direnv_names. For example,
will unmangle both PATH and XPATH.
See direnv issues
253 (“PATH gets mangled when using direnv
from git-bash on Windows”) and 796
(“Incorrect path format is exported on Windows 10 with mintty / git bash, breaking the PATH
and command resolution”). ↩
While experimenting, I discovered direnv export json! Very nice. It’s great
to see more and more tools using structured data for their inputs and outputs. ↩
Trusting Signal and/or WhatsApp and/or Android (google) and/or iOS (apple) with your precious
photos, videos, and chats is a huge mistake.
All the photos, videos and chat history on my mother’s phone are now completely inaccessible to
us, despite having recent backups and all the necessary keys and passphrases.
Signal
Can I transfer Signal backups directly from android to iphone? No.
Can I make a backup file and transfer that wirelessly without uploading it to the cloud? No.
Can I transfer it using a USB cable? Also no.
If I transfer it using the cloud, can I then restore from it on the iphone? … No.
It is not possible to transfer Signal chat history and media from an android phone to an iphone.
WhatsApp
What about Whatsapp?
Can I transfer Whatsapp backups directly from android to iphone? No.
Can I make a backup file and transfer that? No.
Can I back up to google drive and use that? No.
Even if I could get the backup file, would I be able to import it on the iphone? Also no.
It is not possible to transfer WhatsApp chat history and media from an android phone to an iphone.
The entire ecosystem is sick
Signal is to blame: they do not make it possible to import backup data on iphone. They do not
offer tools for working with backup data.
WhatsApp is to blame: they do not make it possible to retrieve or work with android backup
data. They do not make it possible to migrate data from android to ios without fully wiping and
resetting the iphone (and it didn’t work even when we did fully wipe it).
Apple is to blame: there is no way to transfer files from android without going through the
cloud. Even using a usb cable doesn’t work.
Google is to blame: there is no way to transfer files to iphone without going through the
cloud. There is no way to access a whatsapp backup blob in google drive containing my own data.
We all are to blame: we have accepted and continue to make excuses for an industry that acts in
such a user-hostile way.
I’ve just updated the Preserves spec to version 0.990.0. I feel like
a 1.0-rc is approaching!
The main change since spec version 0.7.1 has been to simplify the binary syntax for Preserves
Values:
Both “short” and “medium” SignedInteger representations (starting with tags 0x9x/0xAx)
were removed. They weren’t pulling their weight. Every SignedInteger now has tag 0xB0.
Float and Double are now encoded with tag 0x87 and a length, rather than with fixed
tags 0x82 and 0x83, opening the door to other IEEE754 formats in future.
Here’s the 0.990.0 syntax in “reference card” format,
where we write «V» for the binary encoding of some value V:
«#f» = [0x80]
«#t» = [0x81]
«@W V» = [0x85] ++ «W» ++ «V»
«#!V» = [0x86] ++ «V»
«V» if V ∈ Float = [0x87, 0x04] ++ binary32(V)
«V» if V ∈ Double = [0x87, 0x08] ++ binary64(V)
«V» if V ∈ SignedInteger = [0xB0] ++ varint(|intbytes(V)|) ++ intbytes(V)
«V» if V ∈ String = [0xB1] ++ varint(|utf8(V)|) ++ utf8(V)
«V» if V ∈ ByteString = [0xB2] ++ varint(|V|) ++ V
«V» if V ∈ Symbol = [0xB3] ++ varint(|utf8(V)|) ++ utf8(V)
«<L F_1...F_m>» = [0xB4] ++ «L» ++ «F_1» ++...++ «F_m» ++ [0x84]
«[X_1...X_m]» = [0xB5] ++ «X_1» ++...++ «X_m» ++ [0x84]
«#{E_1...E_m}» = [0xB6] ++ «E_1» ++...++ «E_m» ++ [0x84]
«{K_1:V_1...K_m:V_m}» = [0xB7] ++ «K_1» ++ «V_1» ++...++ «K_m» ++ «V_m» ++ [0x84]
varint(v) = [v] if v < 128
[(v & 0x7F) + 128] ++ varint(v >> 7) if v ≥ 128
I’ve been working with Markdown tables a lot recently. The lovely
markdown-mode for Emacs makes this easy and
pleasant. However, I find myself treating the tables a little like CSV, and started wanting to
run joins on tables often enough that I hacked together my first little Emacs package,
markdown-join.
Here’s a demo:
The code is a single file,
markdown-join.el.
I’ve submitted it to MELPA.