Working Notes: a commonplace notebook for recording & exploring ideas.
Home. Site Map. Subscribe. More at expLog.
— Kunal
As much as I enjoy April in New York as the weather changes and opens up, allergies tend to ruin my days and nights; hopefully the pollen settles down soon or my allergy medicines start working.
I've been using the Go60 as my only keyboard for the full week now, and taking the time to slowly configure and customize it suit me as I get used to the slighty reduced set of keys. The smaller size of the keyboard definitely works well for me, and I appreciate the fast and soft keys, keyboard customization with has also worked out -- it did take quite a bit of effort to decide a layout that worked best for me -- particularly because I'm fairly used to having all 4 arrow keys available for window navigation, etc. My current compromise is to have left, right under my left hand -- right below j, k in the dvorak layout, which works out really well. And then with the layer key I can switch left/right to up/down, and that's been fairly reasonable.
There's also been a lot of typ.ing practice to build familiarity faster: I can hit 100-110wpm most days, and 120 on very very good days; but still a little bit slower than the Glove80. Pure comfort wise I think the glove80 is still the best, and I'll keep using it as my primary keyboard at office, while the Go60 will travel with me and otherwise stay at the home office for presumably less hours of typing.
My current Go60 layout is available here (only meant for dvorak/emacs with vim binding people). It says qwerty because I want to be able to keep my laptop keyboard set to dvorak, so the keys are adjusted to work perfectly for users who also have their laptop itself set to Dvorak.
After seeing a couple of people recommend it I thought I'd take a stab at using Jujutsu version control and read through Steve Klabnik's book on using jj as I experimented with a new project. I definitely like the mercurial-like feel of the system and generally the idea of "opening" a commit for use (I was reminded of my own lab notebook project that has a similar behavior).
At the same time, I'm not sure I'll start using it day to day just yet -- only for some small, personal projects till I'm willing to deal with complex shenanigans like performing branch surgery -- with it.
A lot of my professional work this week was spent dealing with systems overloading -- particularly around service restarts -- and then being DDOS'd by clients as they tried to come back up. I also experienced the joy of seeing clients assuming every single request was failing -- and retrying dozens of times -- while the requests were actually succeeding but not getting acknoweledged correctly; this meant each client would keep hammering the server which was already overloaded and never actually managed to recover because traffic became multiplied by 10 times.
Having meaningful backoffs, failing fast, and not hammering the single points of failure hoping things will work out seems like the order of the week; which is where I expect to spend most of my time tuning and stress testing. What always works for me best while working with systems like this is to make sure I have a canary that I completely control and can explain the behavior of -- it may be doing trivial, nonsensical work -- but it should be deterministic and easy to explain and evaluate.
Then I can confirm if the system is behaving appropriately in reaction to the canary or not; and wherever things don't add up I immediately have signal that there's something interesting to dig into and understand or fix.
In this particular case I've been exploring the observability stack: and I set up a canary that publishes the timestamp every 15 seconds -- and then I can check how much lag / drops there are in the system. Funnily enough I ended up with much more data than I expected because of the retries I just mentioned, which was definitely not what I'd been expecting when I put this into place.
I'm still fairly naive about how to make inference go brr -- and I'm finally picking up a side project to go run inference for Gemma 4 -- using different frameworks and mechanisms as a learning exercise. This is a brand new model so I have to use versions from master for a lot of things.
Once I have my handle on inference I suspect I'll be able to build much more interesting tools than the ones I've built so far.
As I learn more about Gemma 4 the more excited I get about the different techniques applied in it; which means I'll be able to learn more about inference.
While I was planning to switch jobs I'd also been considering simply taking a sabbatical -- I'm not sure I'd do well in a completely unstructured setting -- but I enjoyed hearing that Rich Hickey took a 2 year sabbatical to build out Clojure. He said something that stood out -- he didn't have explicit goals for the sabbatical which is what made a lot of this possible; I think I'd have ended up making fairly harsh and aggressive goals for myself if I'd taken a sabbatical in between jobs, so this was good to digest. I'll probably take my own break after this job, and tackle all the older pending programming and learning projects that I just haven't made time for -- particularly (sic).
The rest of the documentary is also fascinating for all the decisions taken along the lifetime of clojure -- along with the financial consequences for Rich and his wife; a reminder to go re-watch Simple Made Easy. I should also say that my ideal retirement looks a lot like the one described by Rich. "Intellectual freedom, all the way to the finish line."
Watching Alex Miller and the references to Strange Loop also made me extremely nostalgic for the conference: and garteful that I made it to the last two instances.