Working Notes: a commonplace notebook for recording & exploring ideas.
Home. Site Map. Subscribe. More at expLog.
— Kunal
A long but pleasant week, filled with experiments.
After several years of wanting to, I managed to register for The Great Saunter way back in February -- they give you a route to walk roughly 32 miles and circumnavigate Manhattan. Unfortunately, I didn't make the time to actually train for it, and decided to attempt just about half this year to get started -- which I succeeded in, 16 miles / 25 km.
I have unhappy feet and a couple of muscle aches, but I'm sure that'll pass. The walk itself, and the surrounding organization was amazing and encouraging. The route itself is beautiful, particularly around the upper west side leading to the cloisters: I hadn't realized we had such beatiful parks in New York. Finally checking out the Little Red Lighthouse was a treat in itself.
I remembered writing about Marimo in these letters at some point, and then found it really hard to actually find what I'd written from the actual website. I'm tentatively planning to introduce some form of automated indexing and backlinks: there're already cross links, but I can probably do a better job extracting and normalizing headings and tags and linking to them with the system I already have.
My ideal workflow is to write these letters as I always do, but then be able to have topics across weeks automatically collated into a single page I can also look at -- without having to fiddle around with different forms of tagging manually.
This week's books included Service Model by Adrian Tchaikovsky. It's a bit blunt about the message being imparted, but also lands really close to home as I can see the robots interact in the same way I see LLMs interact; and at the same time it makes for a great story I enjoyed reading.
Alternatively, what if, even as you replace everyone with robots that are cheaper and quicker and less likely to join a union or complain about working conditions, you also continue to insist that individual value is tied to production, and everyone who’s idle is a parasite scrounging off the state?
All of which were arguably soluble problems if your philosophy was to treat people like people, and not like robots. Or else you can just gather up your robots, pull up the ladder, and lock the gates of your compound.
It is possible that the world was once both kind and ordered. It is possible that it may be so again. Perhaps you will make it so.
Taking another stab at implementing transformers from scratch because I still don't find them intuitive enough to reason about comfortably -- I need to sit down with a pen and paper and several references and then ask Claude anyways. This time I'm doing it with Hy again (a throwback to 2023 when I worked through a GPT implementation in hy) -- just this time I'm tackling it in Jax and doing it on my terms instead.
While randomly browsing around I ran into Speech and Language Processing which feels like an extremely approachable book on language models in general.
Asked claude to look over my emacs configuration and suggest improvements: it recommended moving on from the ivy/consult based completion stack I'd been using and to try out vertico/corfu/marginalia; and then I ran into usage limits almost immediately with very little work done. Codex got me over the finish line and I'm experimenting with these.
I suspect I'll want to make my emacs workflows much more reproducible and less fragile as I go -- I've even started pottering around with Zed recently -- or build my own editor using agents at some point and make it trivially distributable.