blog posts on joe blubaugh

Zen: A Browser You Can Love

Sat, 07 Feb 2026 01:17:18 +0000

Everyone is putting AI in their browsers now. Good (maybe even great!) browsers have been discontinued so users can focus on AI.

I’m not opposed to using AI agents for all sorts of things, but I don’t feel comfortable giving it nigh-unfettered access to the web. I’m not ready to let LLMs take on life-altering responsibilities like booking flights, looking at my banking sites, or reading my emails. I don’t trust their data retention policies with potentially sensitive context like calendars or emails. Nobody’s

So I don’t want it in my web browser, which is where most of this work gets done in 2026. I’ve been using Firefox at home and Arc at work, and they’ve got different problems. Arc is over. “The Browser Company” has gone all in on an AI Browser that’s not dissimilar to OpenAI’s browser. Firefox added AI features in November and is only now adding controls that can give you some control over data sharing.

Fortunately, someone recommended Zen to me. Arc’s approach to tabs always clicked with me: every view of a profile, even in a different window, has the same tab set. You can use multiple profiles (“spaces”) to manage different tab groups, and different sign-in contexts. Zen’s latest release uses the same behavior, and it’s delightful. It has a simple but very useful split-view feature. I use it for things like reading the meeting notes and having the video call in the same window, or comparing two related documents without having to tab back and forth. Where it has any AI features at all, they are limited and require advanced settings to turn on. You’re not going to get them intruding on you, at least for now.

It turns out I’m not the only one who’s had this experience.

Multi-agent coding and the resurgence of the terminal

Thu, 05 Feb 2026 15:23:52 +0000

All of the exciting AI coding is happening in terminals. Codex is good, the various VSCode forks are good, but if you’re doing multi-agent orchestration you’re generally using terminal UIs, roped together with Gastown, agent-deck, OpenClaw, lemonaid, or something else.

It’s gratifying, as someone who never left the terminal for a “grown up” IDE, to see how small programs piped together is the paradigm for innovation here. The stuff that makes it possible? Simple, largely open-source tech:

tmux: lemonaid, agent-deck, and Gastown all use tmux panes and windows to let you jump to the right place and give the right info to the right agent at the right time.
git: Lots of work-tracking and communication systems (like beads) use git. Git worktrees are an essential component for multiple agents to work on the same codebase at the same time.
sqlite: agent-deck uses this for communication, and so many other tools seem to as well.

Introducing nvim-beads: Manage beads in neovim

Wed, 14 Jan 2026 06:29:35 +0000

A few days ago my co-worker Bob turned me on to Beads. Beads is a step up from things like PLAN.md when managing work for coding agents, because it provides more structure, is easy for agents to parse, and provides things like progress tracking for multi-agent workflows.

I was intrigued by the setup because it seemed like a way to improve my agent’s ability to work autonomously, but I needed a way to try it out without stepping on top of some complex multi-person projects at work. I decided to use bd and a coding agent to implement a neovim plugin for managing beads.

The result is nvim-beads. Implementing this took about 106 commits over 3 hours of pairing between me and the Claude CLI. It’s written in technologies I know about but am not good at: Lua and the Neovim API. There’s no way I would have built something with this level of complexity in this environment in less than a week of good, solid work. It has tests - which I wouldn’t know how to write.

I don’t think it would have gotten built without Beads, either. The workflow of building a PLAN.md and then having an agent execute it works OK, but if the plan is large, you’re wasting a lot of context using it over and over again. Beads’ structure wastes less context on large plans, and make it easy to either blow away an agent and start again, or makes conversation compaction less of an issue. It also makes parallel agent workflows on multiple git worktrees easier to manage, because the agents can share beads state on one beads-specific branch while doing their work implementing those beads on feature branches.

I think the tech is worth trying, and this post from the author Steve Yegge is a good place to start. After that, read the best practices post.

Podcasting is dominated by warm applesauce

Tue, 13 Jan 2026 11:37:44 +0000

The end of podcasting as a viable cottage industry really does seem to be here now, and the end of podcasting as a culture-moving media format has arrived. Amy Poehler won the first podcasting Golden Globe for a “hang out and promote my new project” podcast. Most celebrity podcasts are this way. Even the ones that ostensibly have a theme: Smartless is barely anything anymore. They’re warm applesauce, and they dominate the format to the point that ad revenue is draining away from a lot of better shows.

“Actors give podcast award to actor with a podcast” isn’t all that shocking, but still, look at the nominees for “best podcast”:

Good Hang with Amy Poehler
Armchair Expert With Dax Shepard
Call Her Daddy
The Mel Robbins Podcast
SmartLess
NPR’s Up First

I guess we can at least be grateful they didn’t nominate JRE. For every single niche these shows occupy - maybe with the exception of Up First - you can easily find a better version that is less popular. I guess the fact that “massive media breeds mediocre media” is maybe not all that surprising, either.

All the data had nowhere to go

Tue, 06 Jan 2026 12:24:03 +0000

Pablo Torre is on episode 506 of TrueAnon¹, which is mostly about the NBA-adjacent poker fixing scandals, but touches on sports betting. The fact that data collection is not a neutral activity comes up repeatedly.

Player performance is monitored and measured for non-nefarious reasons: players and teams both benefit from better understanding of strengths and weaknesses in different ways. Pablo makes the case that the mere existence of this data - the predominance of statistical analysis in front offices and the advanced stats published by the leagues - changed the culture of American sports in negative ways. Rationalization of athletic performance led fans to believe the could get some edge (alpha in the financier’s language), and sports gambling was the natural outlet for this.

Black markets were bad enough, but after sports gambling was effectively legalized in the entire USA, goody-two-shoes started to get in on the act, too. If you’re desperate not to fall behind, windfalls are really attractive. If you’ve known any really committed gamblers, you know this is really a path of destruction. It’s leading to behaviors in basketball that we’d eradicated in the US (aside from boxing): players intimidated by organized crime, refs in league with bookies.²

The NBA is particularly affected, though the MLB is, too. Pablo says the leagues are “fracking themselves” extracting money from this gambling culture while destroying the joy in sports that brings the audience in the first place.

This discussion is a great companion to Maciej Cegłowski’s metaphor of data as “toxic waste”. A company retaining data after its useful life presents incredible privacy risks for individuals. I do not think the GDPR has made that much difference here - legitimate business purposes can still be defined with an infinite lifespan. Maciej focuses on these risks to individuals and companies, but I think this near-infinite data collection has had corrosive effects on our culture, too.

Because we routinely hand over data that we must assume will be leaked, we weaken our own expectations for privacy, which makes us more likely to hand over this information over and over again. Because we have this data, we are tempted - nearly compelled - to think of our lives in terms of this data. Systematizing and rationalizing ourselves and our cultural institutions are not leading to human flourishing. Rationalizing commerce has pulled people out of poverty, but we need some way to contain these tendencies.

Be forwarned that TrueAnon is quite profane and sarcastic by design. ↩︎
Maybe there’s a comparison to be made with the resurgence of measles, too. ↩︎

Finding "Just Fine"

Thu, 04 Dec 2025 06:01:17 +0000

I’ve had a record player for at least 20 years. For the last 10 or so it was an Audio Technica LP120, which is on the upper end of ’non-audiophile’ turntables. Swappable cartridges, tons of tracking force adjustments, speed adjustments and strobes to check for accuracy. Direct drive. I ended upgrading the cartridge at some point. I loved that turntable, and it was darn reliable.

I did manage to fry it on 220v here in Singapore, despite the fact that it advertised compatibility with 220v. After it broke, it didn’t take me long to find a successor: an Audio Technica LP70X, which is in many ways the opposite: Fixed cartridge, no tracking force adjustment or speed adjustment. Belt driven.

So why the “downgrade”? And given that I’ve been buying and playing LPs for so long, why not upgrade to something really great?

Because the price+time vs results curve on so many things in our life looks like this:

and the LP70 is just past the sigmoid component, while the LP120 is higher up: almost twice the price, but diminishing returns.

You can apply this to tons of things. There’s a relationship like this when it comes to cars, musical instruments, cameras, youth sports, vacations, kitchen equipment. For myself, it was even true with schooling - thankfully I was blessed with choice. Going to Purdue over something like Indiana State made a much bigger difference in my life than going to Yale or MIT over Purdue would have.

For some people, those top end results are worth the time and whatever money it takes. For most of us, finding that point just past the sigmoid is a real sweet spot for quality of life. Except the one or two things you’re truly passionate about.

LLMs are great at box diagrams

Tue, 02 Dec 2025 12:26:16 +0000

I love diagram languages like mermaid.js and PlantUML because the text format that backs the diagram can be tweaked, takes up very little space in storage, and can be re-rendered at various resolutions any time you need. They’re great! I hate writing them. Mermaid has a different syntax for every diagram type, and they’re hard to remember. PlantUML is more consistent, though not totally, and in my experience can be quite verbose.

I use drawing tools to make diagrams for presentations. I’ve gotten pretty good at Excalidraw. I like that I have total control over the layout, which is difficult for me to get with diagram languages. They stink for documentation. The images look good, but need to be rendered at multiple resolutions. You need to save the file somewhere to make edits. Nodes get disconnected accidentally all the time.

What I want is to draw the diagram by hand - the fastest way. Then I want it digitized into a format I can tweak. We’ve been trying to do that for ages with digital whiteboards, etc. It’s “eh”.

I’ve been incorporating LLMs more and more into my software development process in the last month or so. The Claude CLI and Cursor have both been working well for me. I’ve done a lot of the “normal” software dev stuff: have an agent write up and revise benchmark tests, write a CRUD API for a new type I’m introducing.

So I decided to see how good it would be at making Mermaid diagrams for me. I drew this diagram on paper, snapped a photo and dropped it onto my laptop.

Here’s the prompt I gave the Claude CLI:

Turn the IMG_ file in this folder into a mermaid diagram in the .mmd format. It should be either a flowchart or a box diagram

Claude can’t read HEIC, so it even converted the image to a JPEG first with sips.

After that I used mermaid-cli to render a PNG and opened it in Chrome:

$ npx @mermaid-js/mermaid-cli -i file.mmd && open file.mmd.svg

Here’s the first output:

Pretty good!

I’ve used this at work too, for box diagrams with around 20 elements, and it’s done pretty well. I get the diagram I want after two or three rounds of tweaking. The feedback loop is really fast and less error prone than the mermaid live editor. I think I cut a diagram drawing session about in half, because my hand-drawn diagram turns into text so quickly.

Ten years of joeblu.com

Tue, 28 Oct 2025 03:37:40 +0000

I’ve had this domain, this site for 10 years. Running on Hugo the whole time. There are only a few things here, but it’s no big deal. There are very few things I have on the internet that have been live that whole time and are still a going concern:

this site
gmail
Facebook
Instagram

Only one of those gives me real control of what I put out here, with no rules but my own and the law. It’s a blessing to still be able to do that in 2025 - give anyone the power to publish anything.

What was WTF with Marc Maron?

Tue, 28 Oct 2025 00:00:00 +0000

I first met Marc Maron in 2010 at a comedy festival in Southern California. WTF was less than a year old, and most of the guests thus far were “alternative comedians”. All podcasts back then were either tech-focused or comedy-focused, particularly LA alternative comedy. I found the tech ones interminably boring, but alternative comedy was really important to me and I was hooked on WTF.

I was in the back of a van with Maron and Maria Bamford (in hindsight, insane situation) as we drove around Lake Arrowhead for half an hour. Marc bored into me for at least half the time, asking me about what made me move to California, my ethnic background and family history in the US, opinions on late night shows. It was more intense than any first date conversation, and I was flattered by the attention. Still, I doubt it’s an encounter he’d remember. He’d been doing his radio show for years, meeting hundreds of new people a year between that and standup sets. I think he’s just that way - curious about every person and what drives them.

15 years later, WTF with Marc Maron has ended, and Marc’s released a new special, Panicked. He never stopped releasing specials or doing sets, which is amazing when you consider the cadence of WTF. It seems like every comedian has a podcast now, but so many of them are just two hours of bullshitting on YouTube or reminiscing for a Millennial audience with very little preparation. Apologies to Bill Burr and Amy Poehler, but so many of them are nothing-burger ways to sell mattresses and athleisure. WTF interviews always had an impressive amount of prep - reading books, watching film, and actually thinking about the person you’re talking to.

It was more compelling (and obviously less intellectual) than Fresh Air, and most other interview shows. Not nearly as high-minded, while still being substantial. It’s telling that somebody as savvy as Barack Obama went on WTF five years before appearing on Fresh Air, and even came back two weeks ago for the final episode of the show. (edit: Obama actually went on Fresh Air in 2004 when he was an Illinois State Senator. He wouldn’t return until 2020. I don’t think this undermines the point about choosing WTF as an interview venue instead of traditional media)

That Obama episode in 2015 changed everything about the show, and possibly about podcasting in general. It’s so important to the show that it gets its own entry in the “About” section of the WTF site. Serial had come out a year earlier, so your Serious Aunt was finally thinking of it as a legitimate medium. Getting the sitting President was a media appearance previously reserved for 60 Minutes or the New York Times. It was the beginning of “everyone has a podcast” and also the beginning of big money investment in the medium. We got a lot out of it, and surprisingly little. Much like Spotify, once the money showed up, the power laws of internet distribution also took over. It’s hard to build a small audience for your podcast, and maybe harder than ever. I’m sad that what will fill the space left behind by WTF is likely to be either fluff or slop.

Short aside: Did you know the New England Patriots arguably created the first podcast?

I wonder a lot about the connection between WTF and The Joe Rogan Experience, and I’d guess Maron does too. They’re practically the same age, they used to compete for the same acting jobs and the same spots on stage. They both have an amazing capacity to be spiteful, and to hold a grudge. They started their podcasts the same year! One of them has grown as a person, one of them has become maybe the most important person in American media in a generation. This New York Times piece reflects on the connection a bit, but there’s probably a media studies thesis to be written there.

Appendix

Maron has been doing a few interviews, I enjoyed this one with Jesse Thorn, where Brendan McDonald calls out Marc’s intensity on initial meeting, too.

13 January, 2026: Well, that answers my question about what Maron thinks about Rogan and the show.

hugo-editor: A Claude experiment

Tue, 26 Aug 2025 02:52:59 +0000

I write and publish this blog with Hugo, a static site builder that uses YAML for its posts. It’s also what we use to build the Grafana docs site, so I use it often. For the past few years I’ve been using vim to write posts. The final site content is copied onto a server after it’s built.

I have an SSH client on my phone, so I can theoretically write posts from anywhere. However, using a text editor over SSH on a phone leaves a lot to be desired. There are also iOS git clients like Working Copy, but they can’t run a script to verify my builds. Before publishing, I need a separate step, and it’s not part of my writing workflow.

So I decided to play with Claude to build a basic blog editor. After about an hour I had hugo-editor, which just hosts a basic web app that interacts with the file system on a server directly. It has a simple plain-text, styled <textinput> and periodically saves the contents to the file system. It also keeps a hugo development server running at all times so I can preview the entries. There’s a configurable “publish” command, which I have set up to run a hugo build and then rsync the contents to my web server.

I skipped over some more complicated things like user authentication and management because I host the editor itself on my private Tailscale network, which makes it easy to bookmark the URL on trusted devices only. If someone wants to add user handling, contributions are welcome, but right now I’m very happy with the fact that there’s no database here, just a file system. If I wanted to add password protection, I would probably just use something like Caddy basic auth, which is what I use to protect the apps that I expose for myself over the public internet.

When I use Claude at work, I give it very limited freedom, and spend a lot of time in a tight loop making changes. For this, I decided to let it off the leash a bit. If you look at the code, you’ll find comments that go nowhere. For example, the Hugo development server is started, and there’s a comment // Set up cleanup on program exit, followed by code that does nothing to set up cleanup.

Claude decided to put everything in the main file, and use Go HTML templates for HTML rendering, which are parsed at startup time. This means the app can’t be distributed as a single binary. When I tried to convince it to //go:embed the templates, it struggled mightily. It also means the code is fairly ugly. It made config flags and then never used them to actually configure the application.

I would continue to be very wary of any products that proudly call themselves “vibe-coded” because of how many simple errors are in this very basic app, but I did get something genuinely useful in less than an hour.

Why would you use .gitkeep?

Mon, 30 Jun 2025 00:00:00 +0000

I’ve been using Git for 9 years now, and today was the first time I knowingly encountered a file named .gitkeep in a diff. “What’s the point of this empty file?”, I thought. Quickly realizing that the idea is to track an empty directory, I started to wonder “Why would you want to make sure a directory exists when there’s nothing to put there?” and I started seeing blog posts about preparing for future changes that will add files to the directory or making sure your team has a consistent directory structure.

I feel like this is putting the cart before the horse. If there’s no information, there’s no reason for the directory. If your tools rely on a structure full of empty directories, those tools should be able to create the directories. If for some reason, they can’t, a set of mkdir -p commands prepended to the call is all it takes.

An example given in that second blog post:

If you have a deployment directory that contains configuration files or scripts, adding a .keep file to it ensures that it’s included in the deployment process.

No! Your build process should create the deployment directory, and that directory should be in .gitignore! You shouldn’t be saving an empty directory into which you might then commit deployment-generated files in error.

I’m honestly struggling to think of a use-case for these .keep files that isn’t better solved by $tools that generate directory structure according to a template (project setup). If you have one you think is really valuable, I’d love an email to help me understand.

Git made the right design decision to ignore empty directories because they’re do-nothing data.

Think big, build small

Thu, 26 Jun 2025 00:00:00 +0000

I’ve been at a few software organizations navigating the transition from small company to established businesses with big customers. At that point, duplication starts to become a cost, and consistency and stability in the product becomes more important than earlier in the business cycle. This is when everything starts getting “platformized” and standardized: application platforms, app development frameworks, managed job queues, data pipelines. These all have the chance to be transformative for your company from a dev experience and cost control perspective. They also routinely get mired in endless migration timelines, and never being “ready enough” for big systems to transition to the platforms.

A phrase I’ve been using as a mantra when approaching platform projects and questions is “think big, build small”. Thinking big means finding leverage outside of your team, finding the opportunities that will make a real difference for your coworkers and hte business. Then, keeping a “build small” focus encourages incremental delivery, adding capabilities one-at-a-time, in order of usefulness.

Thinking big

If you’re building for your team, ask the question “could we build for the department”? If you’re building for the department, ask “could we build for the company?” Play out those scenarios to identify opportunities, even if you choose not to tackle them.
You don’t need a “platform team” to think big like this. If you find an opportunity you may end up with one, but finding these opportunities is a mark of solid and ambitious engineers, and a skill you should cultivate.
Treat any platform as an internal product. Mandated adoption - at least early on - is not likely to succeed. If people don’t want it, you’re not helping the company. Finding out why they don’t want it and address that. Or abandon the platform! Honorable retreat is a reasonable response to failure.

Build small:

Start with two customers, preferably within your target customer profile but fairly different from one another. One customer is too specific, and you won’t build generally enough. Three or four customers are going to have sufficiently different needs that you’re going to spend too much time building before you’re getting real usage. Teams building new systems are great candidates early on, because you can nail some features before you invest in migration tooling.
Continue to prioritize active customers over theoretic ones. Don’t build for someone you hope will use the tool in 2 years.
Assess often. A two-year roadmap may or may not be reasonable, but if you’re unable to make progress in 6-month intervals you’re biting off chunks that are too large. If active users and early customers are waiting two quarters after the start of work, it’s time to take a hard look at the project.
Accept that “the big complex team” may never adopt a given platform. As long as the costs of that are internalized to the team rather than externalized to the company, that’s actually fine. It’s an Engineering trade-off, and custom deployments are often necessary somewhere.

Our Furtive Encrypted Future

Wed, 04 Jun 2025 00:00:00 +0000

By 2030, real encryption may be illegal in the European Union. Heck, Spain wants to ban end-to-end encryption entirely It’s already under attack in the UK. Be sure that the USA and China will be nipping at their heels.

Because mathematics are mathematics, there’s not actual safe way to provide lawful government access to encrypted communications. If the government has a key, at some point the key will leak. The shared keys for DVD CSS and AACS leaked. Any shared keys for decrypting content eventually leak, and there’s no sound way to update content to disable those private keys.

So what’s are some likely outcomes?

Most messaging systems will eventually get hacked. Even if they aren’t most users will not know they’re being surveilled, as surveillance warrants will be secret. Assume the government is reading all your sexts.
Many users move to Signal, which will be technically illegal but somehow side-loadable. Moxie Marlinspike ends up living in Brunei.
Anybody with Signal installed on their phone gets special attention from the cops if they’re detained for something else.

If companies are required to do this, what are some strategies that can reduce the blast radius?

Companies retain a private key on a user-by-user basis (or even a conversation-by-conversation basis), meaning that a leaked key affects a small number of users. Von der Leyen’s keys are still worth millions, and leaking many keys is as likely as leaking a single one, in the event of a cyberattack.
Users can move communications move to more obscure platforms, things like fake websites with strange payloads
Users can stop using the internet, and much illegal activity that can be monitored at a fuzzy, obscure level now, can simply not be monitored without massive boots-on-the-ground surveillance. Welcome to the old world of spycraft.

American schools should all have uniforms

Sun, 01 Jun 2025 00:00:00 +0000

I love fashion as a means of self-expression. Getting creative with pieces found in thrift stores and flea markets was a big part of my 20s. I’ve been thinking a lot recently about fashion in a school context and its effects on children.

My daughters are in their early years of school. Even in preschool here in Singapore, kids wear a school uniform. It’s usually simple - a polo shirt and shorts, with embroidered school logos. Everybody buys them from the same uniform supplier, and you buy a PE uniform too. State schools, international schools, religious schools. They all have uniforms. My daughters have very different tastes in their own wardrobes, but at school they’re essentially equal.

I grew up wearing school uniforms, because I went to Catholic schools. For one year after we moved, in fourth grade, I attended a public school. I needed to build a “school wardrobe” quickly, going from wearing street clothes 2 days a week to 7. Having sidestepped American class-conscious fashion for my first 10 years, I was suddenly in a world where my shoes were the wrong brand, and my odd taste in t-shirts was ridiculed. Regularly. How much of the bullying is attributable to fourth graders being fourth graders - casually cruel and newly obsessed with internal hierarchy - and what of the bullying about my clothes was actually down to the clothes is hard to untangle. But it’s hard for me to see fashion in schools as anything but a distraction.

Most prestigious private schools have uniforms. Many public charter schools, especially those branded as “academies” in lower-income neighborhoods, wear uniforms. Why? They help build a school identity and culture. They raise the poor and lower the rich a tiny bit. They save families money - uniforms are often inexpensive compared to street clothes.

So what does “free dress” buy us in a school environment? A means of self-expression surely. Another opportunity for money to create a hierarchy.

Loog Piano: My review

Mon, 16 Dec 2024 00:00:00 +0000

I bought two of the original Loog guitars, little 3-string acoustic guitars intended for new learners and especially children. There’s a lot to like about them: they’re inexpensive, cute, and have surprisingly good sound and volume. The larger one gives a tenor guitar sort of sound that’s pleasant, and the smaller one is ukulele-ish. The kids are getting old enough to be into them, and I love having them around.

So when Loog announced a piano, I was excited. The design is cute and very “Loog”, and the small 3-octave keyboard seemed like a perfect fit for the small rooms in our condo. Simple on-off with one good sound seemed really nice to me. I was used to the 70 useless presets of the old Yamahas of my youth.

There was one problem with the original Loog guitar: they ran pretty late getting delivered. This isn’t uncommon on Kickstarter, of course. Especially for a first-time manufacturer, and especially with woodworking, there’s a lot to learn. Just ask the good folks at keyboard.io, who are some of the best at providing transparent updates about contract manufacturing. It was Loog’s first kickstarter project, so I wasn’t surprised.

All these years later, I figured Loog would have their ducks in a row. Boy was I wrong. The piano had numerous manufacturing and quality problems. The product features were repeatedly revised over the course of the campaign. My model didn’t even turn on when it first arrived, suggesting quite poor quality control. In the end, delivery was over 6 months past projections, and it was the sort of creeping “maybe next month!” lateness that any manager is familiar with in a poorly executed project.

First, some thoughts the piano itself:

Quality

The Loog claims on its box to be “a portable, digital piano for children and grown-ups, with pro sound and design.” Let’s investigate.

I’ll get the worst out of the way first: I was delivered a defective product that won’t turn on, and I have no idea when I’ll be able to get a working one. More on the service quality below, but I proceeded to check out other issues with the keyboard while hoping for a replacement.

Here’s the user guide. One thing I want you to notice is the pasted-on warning about what kind of charger to attach. The Loog doesn’t even come with the cheapest of wall warts that meet it’s requirements, and evidently the device is “engineered” in such a way that it can’t handle being connected to a fast charger setup. They evidently noticed this so late in the process that they needed to attach an after-thought of a warning to these cards rather than re-print something professional.

The loog has a 1/8" stereo connector for headphones, a USB-C port that serves double-duty for charging and for use as a MIDI controller. It has two more 1/8" connectors for the octave shifter and sustain pedal.

The fabric cover of the piano is nice enough. It’s a nylon mesh in the characteristic Loog red, though some Kickstarter backers complain that theirs came out pink. I wonder about how it would age with the hurting a kid may put on it.

The wood on the keyboard is quite nice. There’s clear coat maple for the sides and the octave shifter that’s beautiful, and the dark-stained volume knob is nice and chunky with really smooth rotation. It feels great.

Sound

// TODO if and when I get a working keyboard. However, there are quite a few comments on the Kickstarter page
// complaining that the sound quality is nowhere near the "pro" claimed on the packaging and marketing.

Keys

The plastic of the keys is pretty atrocious, though. It’s really slick and cheap feeling. Compared to an inexpensive Akai 25-key MIDI controller, it’s pretty disappointing. Let’s also be clear, it’s not “for kids and grownups”. Unless you have pretty small hands, it’s definitely “for kids”. Smaller adults who struggle to do stretches on a full-size keyboard may benefit from that, though. I found it almost unplayably cramped.

The action feels cheap. I can’t tell how good the key sensitivity is, and whether the volume response is smooth with varying force, but if I get a working one, I’ll update this.

Accessories

I got two significant accessories with the Loog: legs and an octave shifter to extend the three-octave range. The legs appear clever at first. They’re powder coated and come in to sections you can screw together. That means that you get 3 heights with two components. There are two big flaws. The longest version of the legs are still the size of a child-height desk, so I think by the time a kid is seven it’s already too short for them. Second, to install the legs you have to remove the black rubber pads on the bottom of the keyboard. This is an incredible design oversight - having to permanently alter the product to install some temporary legs. You’d be better off buying a keyboard stand from a music shop if you want to actually move the piano from place to place.

The octave shifter has a great feel - a nice click when you depress it makes it obvious what you’re doing. It’s got a really attractive design and was beautifully executed. Unfortunately there’s no indication on the keyboard what octave range it’s currently in. I think the octave shifter would make a really great control for a personal electronic project using an ESP32 or similar.

Production

So, there are quite a few disappointments from a product and quality perspective, and the production process itself did not inspire confidence, with late updates. Again, compare to someone like keyboard.io, who manage to be both timely and transparent with their updates. Even when the content of the updates is disappointing, the reasoning - and even the thought behind the reasoning - is clear and justifiable. They rarely make the same mistake twice.

Manufacturing delays and production startup issues are somewhat par for the course for new Kickstarter projects, but Loog has been producing wooden products in China for over five years. That they would still have this much trouble managing a manufacturing situation is concerning to me - they’re either still naive and incompetent or simply dishonest.

By the time Loog finally delivered a piano to me, they were already selling a production model co-branded with Duolingo, and the discount on the black friday piano brought it to nearly the same price as the one-year-long pre-order. So Loog got either interest or cheap capital for a year from my purchase and didn’t even actually give me a significant discount.

Support

When I first tried to contact Loog about the issues I encountered with the first piano I received, I got a promise to “follow up over email.” It took 12 days before I got a message from support, wherein they asked me to disassemble they keyboard and check the connections by hand. There’s a lithium ion battery inside, and they provided no documentation about what parts of the circuit board are “hot” or which connections that I should be checking, or the technique for checking them. They’re lucky I have an EE degree and an ohmmeter.

Disassembly required removing 12 (!) screws, and the top of the volume control, because the volume pot is soldered directly to the circuit board. I don’t think it’s a good sign for product longevity to have an external control that’s going to be manipulated a lot soldered directly to the board without much other support structure. The pot isn’t even secured to the top with a nut, so you can put a lot of force on the board just by pulling the pot side-to-side. I couldn’t find any obvious problems with the wiring, so I’m currently in a holding pattern with their support. I’ll update this post with new notes as that progresses.

Nevermind

What was originally intended as a present to learn and play with over the summer holidays has now slipped to post-Christmas and I have no idea if I actually will receive a working product. There are other similar complaints on the keyboard page. If I get this resolved, I’ll let you know how they addressed my concerns, but as of the date of publication I’m out $300 USD with no remedy in sight.

In the end, I bought my kids (and let’s be honest, myself) a Casio. The CT-S1 is a competently executed version of the Loog: simple buttons, with a minimalist style in an attractive cover. It can run off of a battery and has 15 sounds. It also functions as a basic MIDI controller like the Loog. 61 keys is more than I really wanted, but it’s still compact enough. It’s cheaper than the Loog, sounds great, and best of all, actually works. I managed to source brand new one for under $90 USD on Lazada, about 1/3 the price of a Loog.

What's a "thunk"?

Wed, 04 Dec 2024 00:00:00 +0000

I remember when everyone started talking about using “thunks” in Redux to execute code without blocking the UI. And I remember getting tripped up on the word “thunk”. What’s a “thunk”? Why are we calling it that?

The term “thunk” goes all the way back to ALGOL 60! It’s a play on “think”, because the ALGOL compiler needed to “think” about what sort of subroutines to generate. ALGOL allows passing expressions as arguments to subroutines, not just constants. One way of allowing this is to substitute the expression for another subroutine that evaluates the expression when the original subroutine call happens. This new subroutine is the “thunk” - the result of the “thinking”.

In redux-thunk, thunks are passed as an argument to dispatch, which delays their execution until after the React render cycle. This avoids blocking the UI thread when state updates need to happen. If actions need any async logic like making API calls, thunks are the way to handle it.

I think Redux made a mistake using the word “thunk” here, when most programming languages have a concept of an inline function definitions and delayed execution. This sort of behavior is everywhere. In Go’s sort.Slice, you pass a thunk that gets called over and over:

ints := []int{3, 1, 4, 1, 5, 9, 2, 6, 5, 3, 5}
sort.Slice(ints, func(i, j int) bool {
    return ints[i] < ints[j]
})

Python uses thunks all over, as “lambdas”:

ints = [3, 1, 4, 1, 5, 9, 2, 6, 5, 3, 5]
ints.sort(key=lambda x: -x) # sort in reverse

“Lambda” isn’t much better as a term for inline function definition, but at least most programmers are exposed to it. The term “thunk”, though, makes you think that you have to learn some deep new concept.

jq -n \-\-argjson

Mon, 02 Dec 2024 00:00:00 +0000

jq, a tool for manipulating JSON data, is one of the ~~easiest~~ most powerful ways to work with JSON text values in a terminal. I use it frequently. Mostly, I use it to extract values from HTTP responses:

$ http get https://api.github.com/repos/grafana/grafana | jq '{ name: .name, stars: .stargazers_count }'
{
  "name": "grafana",
  "stars": 65313
}

The other day I wanted to use jq to combine the values from two different API calls, so I pulled up the man page and found the --argjson option. Easy, I thought: just write a zsh function:

together() {
  orgjson=$(http get https://api.github.com/orgs/$1)
  repojson=$(http get https://api.github.com/repos/$1/$2)
  jq --argjson org $orgjson --argjson repo $repojson '{ org: $org.name, repo: $repo.name }'
}

This blocks forever. It blocks forever because jq is waiting on data to come in via stdin for it to process, even when it also has arguments to process. You can observe this by writing to the stdin file descriptor you find in /prod/<pid>/fd/, or closing the descriptor, which will terminate jq.

To run jq when no input is expected, use --null-input or -n. As the manual says:

--null-input / -n:

Don´t read any input at all. Instead, the filter is run once using null as the input. This is
useful when using jq as a simple calculator or to construct JSON data from scratch.

Add that to the function:

together() {
  orgjson=$(http get https://api.github.com/orgs/$1)
  repojson=$(http get https://api.github.com/repos/$1/$2)
  jq -n --argjson org $orgjson --argjson repo $repojson '{ org: $org.name, repo: $repo.name }'
}

And we get:

$ together grafana grafana
{
  "org": "Grafana Labs",
  "repo": "grafana"
}

Don't use mocks

Tue, 20 Jun 2023 00:00:00 +0000

Writing good unit tests is made much easier by dependency injection. This lets you separate your code’s behavior from that of your dependencies:

type MyType struct {
  dep1 DependencyOne
  dep2 DependencyTwo
}

func GoodConstructor(dep1 DependencyOne, dep2 DependencyTwo) *MyType {
  return &MyType{
    dep1: dep1,
    dep2: dep2,
  }
}

func BadConstructor() *MyType {
  return &MyType{
    dep1: NewDepOne(),
    dep2: NewDepTwo(),
  }
}

The example’s a little bit contrived, because you could just directly write dep1 to the struct field, but for types with complex initialization, the testability of a type created with GoodConstructor is much higher.

So, what should you pass into the constructor when testing? Many people pass in mock objects. These are objects that provide the same API as your concrete dependencies and provide tools for constructing responses to API calls and assert that certain calls were made with certain arguments. A popular example in Go is testify/mock. In Java, you’ll often encounter Mockito. Mocks look convenient: have the framework generate a mock for you, and then script the interactions. I’m here to argue that the cost of the convenience is too high.

Mocks create brittle tests

Most mocking frameworks create an assertion on each mock API call and then verify these assertions after a test. You may write a test like:

func TestAdd(t *testing.T) {
  dep1 := NewDepOneMock()
  dep2 := NewDepTwoMock()
  dut := GoodConstructor(dep1, dep2)
  
  dep1.On(t, "AddOne").Return(1)
  dep1.On(t, "AddOne").Return(2)
  
  dut.Add(2)
  assert.Equal(t, dut.Count(), 2)
}

What happens in dep1 gets a new function Add and you change your code to use it? You break your test, even though no behavior in your code has changed. We shouldn’t be rewriting tests that have the exact same inputs and expected outputs.

Mocks encourage tests that are too fine-grained

Because your mocked calls are dependent on the current state of the Device Under Test, you’re also tempted to assert on the internal state of your dut. These kinds of fine-grained tests are also fragile. You can’t modify the code without modifying the test, even if the inputs and outputs are unchanged. That’s unnecessary extra work.

Alternative: Use a Fake

Migrating to a “Fake” is probably the simplest way to move a unit tests off of a brittle mock. A Fake is an implementation of the dependency that isn’t “real.” It’s less complex to set up, perhaps, or it doesn’t require a network connection. An example might be an on-disk implementation of the Amazon S3 API. To the caller this is indistinguishable, but it’s much easier to use in tests.

Alternative: Don’t unit test calls to complex dependencies. Write integration and behavioral tests for multiple components

One reason we don’t use complex dependencies (even from other packages in the same codebase) is to avoid testing the code of the dependencies. That shouldn’t be a unit test’s responsibility.

But an integration test does care about the code in other packages. That’s the right place to test complex interactions with dependencies.

What about Stubs?

In most definitions, a Stub is like a mock that doesn’t do assertions. This can be less fragile than a mock - there aren’t assertions that break if the number of calls change - but it is still a programmable double of a dependency. It has many of the same problems with brittleness. Internal logic tests can break tests in the absence of observable external changes.

Supreme Court DoS attack

Wed, 22 Feb 2023 00:00:00 +0000

In the latest edition of Casey Newton’s newsletter, Platformer, he makes the case for why the plaintiffs in Gonzalez vs Google botched their oral arguments. The mechanism is fascinating:

[H]ow many … lawyers were conflicted out of representing Gonzalez. Platforms pay so many high-powered lawyers at so many DC firms that plaintiffs had to recruit an 80-year-old man who once argued a case before an appointee of Franklin Roosevelt.

They DoSed the Supreme Court! I’m on Google’s side in this particular case, so, “good”. But this isn’t a tactic that should work.

PS: Platformer is good, consider subscribing!

Looking in Go's Mirror: How and When to use `reflect`

Mon, 16 Jan 2023 00:00:00 +0000

This article was originally published on gopheradvent.com

Go’s static typing is a headline feature of the language. It prevents whole classes of bugs, makes code easier to navigate and refactor, and makes it easier for linters to analyze. But there are times when it’s very constricting. What if we’re reading JSON files from disk with unknown structure? We can’t define a type ahead of time that will cover all cases. The reflect package gives us the power to handle this situation, and to do much more. We can write functions that handle arbitrary types. reflect is also the technology behind Go’s “magical” struct tags, which are often used in serialization. reflect can be intimidating to new Go programmers because it’s very generic and you lose access to many niceties in the language. But it doesn’t have to be. Let’s build some programs that use reflect as a way to demystify the package and illustrate the power and pitfalls that come with using it.

`reflect` package basics

Value, Type, and Kind

The fundamental concepts of reflection are the Value and Type structs, and the key starting points are the ValueOf and TypeOf functions. You can use the TypeOf function to get the Type of any variable in Go; ValueOf provides access to the underlying value, along with the type information.

var x float64 = 8.4
var y struct{ Field float64}{ Field: 1.2 }
var z any = x
fmt.Println(reflect.TypeOf(z))
z = y
fmt.Println(reflect.TypeOf(z))
val := reflect.ValueOf(z)
fmt.Println(val.Type())
fmt.Println(val)

// Prints:
//  float64
//  struct { Field float64 }
//  {1.2}
//  struct { Field float64 }
//  {1.2}

Try it yourself

Every Type and Value also has a Kind that provides the building block for handling arbitrary data structures with reflect. The Kind() function returns one of Go’s primitive types like Float64, Map, Array, and Struct.

Settability

reflect also lets us modify the Value that we have as long as the value is “settable”. It’s convenient to think of “settability” as similar to “is a pointer”. reflect needs a pointer at the value it’s referring to if it wants to modify it. Here’s an example:

var x float64 = 8.4
fmt.Println("x:", x)

v := reflect.ValueOf(x)
fmt.Println("v is settable:", v.CanSet())

v = reflect.ValueOf(&x)
fmt.Println("pointer is settable:", v.CanSet())

v = v.Elem()
fmt.Println("pointer's value is settable:", v.CanSet())

v.SetFloat(6.1)
fmt.Println("x:", x)

Try it on goplay.tools

OK, let’s look at some programs using reflect! These programs will be missing some checks and cases in the interest of brevity. Don’t base your production code on this!

Deserializing structured data with `reflect`

Suppose you have a file with JSON in it, but you don’t know its structure:

{ 
  "name": "Joe Blubaugh", 
  "location": "Singapore", 
  "job": {
    "company": "Grafana",
	  "title": "Software Engineer"
  } 
}

The json.Unmarshal function uses reflection to deserialize data when it’s passed an any value as the destination. We can also inspect the result with reflect. The full example is available on goplay.tools, and the key element for our analysis is a function that recursively processes a reflect.Value according to the Kind:

func visit(val reflect.Value, indent int) string {
	var s string
	switch val.Kind() {
	case reflect.Interface:
		s += visit(val.Elem(), indent)
	case reflect.Map:
		s += indentIt("Map\n", indent)
		iter := val.MapRange()
		for iter.Next() {
			s += indentIt("Key: "+iter.Key().String(), indent+1)
			s += "\n"
			s += indentIt("Value: "+visit(iter.Value(), indent+1), indent+1)
			s += "\n\n"
		}
	case reflect.String:
		s += val.String()
	}
	return s
}

The switch on the kind of that Value tells us how to handle it. If it’s a map, we iterate over the keys and visit each value, and if it’s an interface we dereference it to get the underlying value before handling it. Most kinds are left out here for brevity, since the data supplied only has JSON objects and string.

How does Unmarshal create the right types when building up the structure here? As it scans the JSON, it chooses appropriate types based on the structure of the data, as documented here. It uses the MakeMap, MakeSlice, and New functions to create each Value and assign the parsed data to them. Using those functions, we can build any data structure we want at runtime without a prior definition. We can even build new structs without a predeclared type definition by using the StructOf function.

If you’re clever with reflection, that means that you can parse a schema at runtime and then validate whether any data matches that schema. The JSON below is an example of our schema. It has a value for every required field, with the correct type set.

{ 
  "name": "foo",
  "address": [123, "Street Name", "City", 11111],
  "age_days": 1,
  "height_cm":	1.0
}

and here is a structure that doesn’t match our schema because of a missing field.

{ 
  "name": "Joe Blubaugh", 
  "address": [101, "Foo Street", "Bazville", 10101],
  "height_cm": 180.1
}

This program can parse the schema and validate other input data by checking that the Type for each field matches and that there are no missing fields in the input data. Notice how similar the validate function is to the visit function that we used to analyze the structure of arbitrary JSON.

// This program loads a schema defined in JSON and validates other JSON 
// against the schema. Every field in the schema is required in the data, 
// and the order and type of values in any JSON arrays must also match.
package main

import (
	"encoding/json"
	"fmt"
	"reflect"
)

func main() {
	var schema []byte = []byte(`
{ 
  "name": "foo",
  "address": [123, "Street Name", "City", 11111],
  "age_days": 1,
  "height_cm":	1.0
}`)

	// This data does not match the schema because of a missing "age_days" field.
	var invalidData []byte = []byte(`
{ 
  "name": "Joe Blubaugh", 
  "address": [101, "Foo Street", "Bazville", 10101],
  "height_cm": 180.1
}`)

	// This data is valid
	var validData []byte = []byte(`
{ 
  "name": "Baby Blubaugh", 
  "address": [101, "Foo Street", "Bazville", 10101],
  "height_cm": 39,
  "age_days": 91
}`)

	var s, d any
	err := json.Unmarshal(schema, &s)
	if err != nil {
		panic(err)
	}
	err = json.Unmarshal(invalidData, &d)
	if err != nil {
		panic(err)
	}
	fmt.Println("Invalid data passed validation?", validate(s, d))

	err = json.Unmarshal(validData, &d)
	if err != nil {
		panic(err)
	}
	fmt.Println("Valid data passed validation?", validate(s, d))
}

func validate(schema, data any) bool {
	s := reflect.ValueOf(schema)
	d := reflect.ValueOf(data)

	sKind := s.Kind()
	dKind := d.Kind()
	if sKind != dKind {
		return false
	}

	switch s.Kind() {
	case reflect.Pointer:
		return validate(s.Elem(), d.Elem())
	case reflect.Map:
		// Do both maps have the same length? If not, the data can't be valid.
		if len(s.MapKeys()) != len(d.MapKeys()) {
			return false
		}

		iter := s.MapRange()
		for iter.Next() {
			key := iter.Key()
			sValue := iter.Value()
			dValue := d.MapIndex(key)
			if dValue.IsZero() {
				// The key isn't present in the data:
				return false
			}

			if !validate(sValue, dValue) {
				return false
			}
		}
	case reflect.Slice:
		// Our schema is restrictive: the slice *must* be present in both structures, 
		// they must have the same length, and the same order of data types.
		if d.Len() != s.Len() {
			return false
		}

		for i := 0; i < s.Len(); i++ {
			sVal := s.Index(i)
			dVal := d.Index(i)
			if !validate(sVal, dVal) {
				return false
			}
		}

		// Skip reflect.Struct because the code to iterate over fields is complex, 
		// and json Unmarshalling into `any` never produces a struct.
		// If types match for primitive values like ints, then this is valid.
	}
	return true
}

Play around yourself, modifying the schema and the data to understand better how that works.

Using struct tags to access metadata

In Go we often use struct tags on data transfer objects: HTTP body data, structs that are mapped to database tables, etc. reflect gives you access to those tags. Suppose we want to redact some fields (like a credit card number) from a struct before sending it to other functions. We could write this program using tags:

package main

import (
	"errors"
	"fmt"
	"reflect"
)

func main() {
	type PaymentForm struct {
		ValueCents       int
		SKU              string
		CreditCardNumber string `redact:"true"`
	}

	f := PaymentForm{
		ValueCents:       10_00,
		SKU:              "abe15f59",
		CreditCardNumber: "5555555555555555",
	}
	fmt.Println("Unredacted:", f)

	err := redact(&f)
	if err != nil {
		panic(err)
	}
	fmt.Println("Redacted:", f)
}

func redact(val any) error {
	v := reflect.ValueOf(val)
	if v.Kind() != reflect.Pointer {
		return errors.New("Must pass a struct pointer to be set.")
	}

	v = v.Elem()
	if v.Kind() != reflect.Struct {
		return errors.New("Must pass a struct pointer to be set.")
	}

	t := v.Type()
	for i := 0; i < v.NumField(); i++ {
		value := v.Field(i)
		field := t.Field(i)
		shouldRedact := field.Tag.Get("redact") == "true"
		if shouldRedact {
			// This is simplified to assume that all redacted fields are strings.
		 	// That's just for brevity in this example.
			value.SetString("XXXXX")
		}
	}
	return nil
}

The full program is also here on the playground

Let’s not overdo it

If you get really creative with reflect you can do all sorts of things. It becomes possible to write partially applied functions and implement other functional programming primitives. “Clever” programming is an awkward fit for Go, though. When you use reflect, your code can quickly become complex and recursive, which can make it harder to maintain and harder to explain to your teammates.

Reflection also impacts your program’s performance. Accessing type information for a value is slow compared to code generated by the compiler. Even type assertions at compile time are significantly faster than similar code using reflect, so it’s best to use the package only when you can’t use the language’s ordinary type handling features like interfaces and type assertions.

All that being said, reflection provides an essential tool that we should all be familiar with: handling data when we can’t predict its structure at build time. That makes it really essential when handling byte streams from reading data off of disk or as the result of network calls.

Additional reading

Rob Pike has an excellent blog post called The Laws of Reflection that expands on the basics above, providing more detail about the underlying implementation of interface types the reflect package uses to do its thing.

What is software for?

Fri, 13 Jan 2023 00:00:00 +0000

In the last blog, I wrote a few things that software is for:

putting people on the moon

safely deploying airbags

making my bank account add up

making sure Mom gets my texts

video editing and animation

drawing in 10 million colors

ebikes

But it’s also used for:

surveilling protesters
bombing weddings
tracking Uyghurs
helping Nazis track Jewish people

We often fail to reckon with this as an industry. (And yes, IBM punch cards are very much part of our history). Actively giving the Nazis an extra data column for “Jewish? (y/n):” to win business is just unbelievably craven.

Much of this software is developed by nation states, or their wholly-dependent defense contractor subsidiaries, but some of it’s not. Consider NSO Group, who has sold their hacks to anyone with an open wallet. How many people are dead because of them? How many people are arrested because of awful ShotSpotter systems, “predictive” crime AI, and faulty face recognition?

And even if software is state-sponsored, some professional designed and wrote that software under whatever ethical umbrella they constructed for themselves.

Consider very carefully how much data you need to have; be even more careful about what you build and for who.

Code is not Prose

Thu, 12 Jan 2023 00:00:00 +0000

Once a month or so, this idea comes rambling out of the programming community¹:

Software is prose. It is written to communicate ideas to others, it has the interesting side effect that it can be transformed into something a computer can execute.
- Chet Hendrickson

This sounds nice on paper. It makes for a good conference talk about how to structure software, how to design programming languages, and how to collaborate. I agree with what I think is the intent: write software so that other people can understand it and change it as they need to.

But it is dead wrong.

Software is for:

putting people on the moon
safely deploying airbags
making my bank account add up
making sure Mom gets my texts
video editing and animation
drawing in 10 million colors
ebikes

Software is not primarily for communicating with people. It is for creating tools to accomplish things. That can be tools for people to use, tools for other software to use, or tools that just produce things for the hell of it. That’s not a side effect, it’s the whole deal.

Does that mean code can’t be beautiful? Can’t communicate more than the program that it generates? No, of course not. Consider the famous brainfuck hello world:

>++++++++[<+++++++++>-]<.>++++[<+++++++>-]<+.+++++++..+++.>>++++++[<+++++++>-]<+
+.------------.>++++++[<+++++++++>-]<+.<.+++.------.--------.>>>++++[<++++++++>-
]<+.

That’s art. It communicates the abstract and arbitrary nature of communication via the code. It’s interesting to note that the code itself is the tool, but it’s not prose. It’s a puzzle.

You read software more than you write it, so make it readable. But don’t get caught up in the idea that you’re writing or editing a novel.

I know that Chet has done more for software development than I’m ever likely to, but sometimes Agile drives you into absurdity. ↩︎

Nasty refactorings with go.mod replace

Wed, 11 Jan 2023 00:00:00 +0000

At work I’ve been building a new program on top of an SDK that’s under very active development. After about 6 weeks without updating the version, the SDK had deleted some code I was using and had a ton of breaking changes. If I’d simply updated the library, my entire program would fail to build, and it also wouldn’t be clear how to get it back to a good state. Following in the spirit of Branch by Abstraction, I’d rather introduce the new code side-by-side with the old and incrementally migrate without breaking the program.

I work in Go, where our import statements look like this:

import (
  github.com/grafana/grafana
  github.com/joeblubaugh/lib
)

The import statements use package names, and if the packages are URLs, then go build will try and download modules by doing a git clone based on the package’s URL. Go’s dependency management uses require statements to describe the dependency tree:

require github.com/joeblubaugh/lib v0.1.0

So, let’s say I need to update from v0.1.0 to v0.5.0, but I need to keep the v0.1.0 code around. I can’t have duplicate module names in go.mod, but I can play a sneaky trick with replace directives:

require (
  github.com/joeblubaugh/lib v0.1.0
  joe/upgraded               v0.0.1
)

replace joe/upgraded v0.0.1 => github.com/joeblubaugh/lib v0.5.0

By introducing a fake module name and a replace directive, I can import code from two different versions into the same program and incrementally move over to the new package:

import (
  github.com/joeblubaugh/lib
  joe/upgraded
)

Caroline Ellison is not a sympathetic figure

Wed, 04 Jan 2023 00:00:00 +0000

I recently read this Washington Post Article (archive.org) titled Caroline Ellison wanted to make a difference. Now she’s facing prison. After the first read-through I was incredulous at the sympathetic presentation in the biography. It reads like a resume, like a “it all spun out of control!” story about someone who got in over her head. I tweeted:

Lol how do you get such a sympathetic bio after stealing billions of dollars @GerritD

I thought I’d elaborate on my reaction to the article, and what I think is wrong with the approach. First, it puts a nice gloss on Ellison’s CV and her stated motivations. Regarding herself and Sam Bankman-Fried:

Both were children of accomplished academics, studied math at prestigious universities and touted the importance of giving money away to make the world a better place.

This is the first thing we learn about Caroline Ellison, in the second paragraph of the article. That’s before we even learn that she’s pled guilty to lying to investors. Combine that with the headline. If you didn’t know who she was, the headline and first two paragraphs read like a story about a political activist tripped up by a justice system that opposes what she’s trying to do.

Are the paragraphs factually accurate? Yes. She “touted” effective altruism. I think the paragraph also implies that she believed in effective altruism; I don’t think there’s evidence for that, and I don’t think the article interrogates her touting of it, contrasted with her actions.

After a few paragraphs about her and Gary Wang’s guilty pleas, we get into the part that really got me going: The math whiz. We get six paragraphs about this bright rising star who learned to read early, got into top universities, did internships at Jane Street. What relevance does this have to why she’s facing prison? What does it tell us about wanting to make a difference? If you want to write about a crypto criminal, you should do it the way NPR did Dread Pirate Roberts. You write about the crimes first. You give us the person’s history in the middle end, and you don’t make it sound like a yearbook bio.

More paragraphs. About how exciting it was to run Alameda Research. About how rich she got. About how she promoted effective altruism. (At no point does the article present evidence of charitable giving or its lack.)

Over 3/4 of the way through the story, we finally get a mention that “FTX was allegedly breaking the law.” The only crime mentioned is that Ellison, Wang and Bankman-Fried lied to investors about “Alameda’s practices.” There’s ample evidence that the crimes are substantially more serious - not that customer funds were used to pay back loans, but that customer funds were likely embezzled and used to pay for that mansion in the Bahamas and the rest of the executives’ luxurious lifestyle.

So, there are two or four paragraphs that directly address the crimes she’s pleaded guilty to, and over 20 that discuss her early life, rise to wealth, etc in fairly glowing terms. If I get nicked for up to 110 years of crimes, I hope I’ll get such gentle treatment.

Unmarshaling JSON in Go: The weird parts

Thu, 15 Dec 2022 00:00:00 +0000

Quick, what does this program print?

package main

import (
	"encoding/json"
	"fmt"
)

func main() {
	content := []byte(`{
	"fieldone":"zero",
	"field_one":"one",
	"fiELD_one":"two",
	"field_One":"three",
	"field2":123
}`)
	type d0 struct {
		Fieldone  string
		Field1    string
		Field_One string
	}
	type d1 struct {
		Fieldone  string `json:"field_one"`
		Field1    string `json:"fiELD_one"`
		Field_One string `json:"field_One"`
	}

	type d2 struct {
		Fieldone  string `json:"field_one"`
		Field1    string `json:"fiELD_one"`
		Field_One string `json:"field_one"`
	}

	type d3 struct {
		Fieldone  string `json:"field_one"`
		Field1    string `json:"fiELD_one"`
		Field_One string
	}

	dZero := &d0{}
	dOne := &d1{}
	dTwo := &d2{}
	dThree := &d3{}
	_ = json.Unmarshal(content, dZero)
	fmt.Println(dZero)
	_ = json.Unmarshal(content, dOne)
	fmt.Println(dOne)
	_ = json.Unmarshal(content, dTwo)
	fmt.Println(dTwo)
	_ = json.Unmarshal(content, dThree)
	fmt.Println(dThree)
}

> go run main.go
&{zero  three}
&{one two three}
&{ three }
&{three two }

What the heck!? What’s going on with &{zero three}? Go uses a variety of hints to try and map JSON keys to struct names, and the rules, while deterministic, aren’t always intuitive and can lead to some strange results.

Let’s work out the rules from most specific to most general.

Try to map the JSON key directly to a struct tag, with case sensitivity. Call this “strict” matching.
Try to map the JSON key directly to a struct tag, without case sensitivity. Call this “loose” matching.
Try to map the JSON key to a field name, case sensitively.
Try to map the JSON key to a field name, case insensitively.

If you look back through the examples, you can see how this works, with one wrinkle: duplicated struct tags. That’s part of what causes &{ three }: Fieldone and Field_One both have the same struct tags, so neither of them wins and the data is discarded. Hm. But why is the value three and not two? Surely "fiELD_one" is a better match than "field_One"?

The other tricky part is key ordering, and case sensitive matching. The JSON is processed a key at a time, mapped into struct fields. Whether a struct field was previously filled or not, doesn’t matter. So let’s look at the d2 struct:

type d2 struct {
	Fieldone  string `json:"field_one"`
	Field1    string `json:"fiELD_one"`
	Field_One string `json:"field_one"`
}

We already know the first and last fields won’t get any data because they have the same struct tag. The data is deserialized in this order:

"field_one":"one",
"fiELD_one":"two",
"field_One":"three",

ALL of these match with "fiELD_one", and for each key that’s the field that matches best. If you shuffle the order of these keys, you’ll get a different outcome.

"field_One":"three",
"fiELD_one":"two",
"field_one":"one",

gives us &{ one }

And this is just with strings! What if you create a struct with two fields of different types that could be mapped to the data?

type d struct {
	S string  `json:"field_one"`
	F float64 `json:"field_One"`
}

try unmarshalling this:

{
  "field_ONE": 123.0
}

and you’ll get an error:

json: cannot unmarshal number into Go struct field d.field_one of type string

The code isn’t kind enough to check for other fields that might have a field name that matches “loosely” AND has the right type, it just takes the first “loose” match it can find.

Well, fine, but how often do problems like this actually happen? All you have to do is make sure that all the fields in your struct have different struct tags, right?

Right?

Yes, that’s true. But it’s not always obvious if you have unique struct tags, or struct tags that are unique in the “loose” sense. Consider struct embedding:

type A struct {
	Field float64 `json:"field"`
}

type B struct {
	Field int `json:"field"`
}

type C struct {
	A
	B
	
	Field string `json:"field"`
}

That’s a perfectly legal struct - each of the Fields is namespaced out of each other’s way. But that’s not the way the JSON decoder sees it. Try and deserialize into this and you’ll get an empty struct every time.

So, some lessons:

Normalize your JSON input, if that works with your API semantics. All lower case is just fine, keeping _ and -. This can help with unexpected “loose” matching.
Always set JSON struct tags for ser/des types. This also helps control “loose” matching and makes sure the data ends up in the field you expect it to go to.
Avoid struct embedding for ser/des types. You never know when an embedded struct could change under you. It’s “spooky action at a distance” and you should try to avoid it.

Evaluating New Tools

Tue, 27 Sep 2022 00:00:00 +0000

I was reading about Phoenix today, looking at guides and documentation. It’s great to see in a guide when it’s easy to quickly set up a project, that makes it fun to get started and explore. The most exciting thing about Phoenix is that “reactivity”, or live updates, are a core part of the system, not an add-on. Nowadays I find any software without reactive updates frustrating and annoying to use.

But even a feature as great as reactive updates is not enough to decide whether to use a tool for building web software. Now that I’m 12 years into a software engineering career, I have some nearly-subconscious evaluations I make when looking into new tools, critical things that can kill a project for me. The most prominent are:

How Painful Will Working With a Database Be? Working with databases (or almost any data storage) is painful. Reliable data storage is hard, but there’s a sweet spot between “looks easy” and “handle every little part of it.” If the by-default approach is to use a wrapper that says “just give us your types and we’ll store them!” I’m instantly suspicious. I know that in a few months I’m going to be digging around in debug and SQL server logs trying to understand why something is slow or broken instead of being able to inspect the code I wrote and understand what’s actually happening in the database as a first step.

How Painful are Authentication and Authorization Tools? Contra database tooling, auth systems should be extremely standardized in a tech stack. People mess up authorization all the time, and it’s really hard to fix after the fact. If you screw up badly enough, you either have to keep horrible practices around or break all your clients and force them to upgrade. Don’t write your own cryptography, don’t write your own auth.

How About Passing and Handling Encrypted Data? Like I said, don’t write your own cryptography. It should be straightforward to move data back and forth between encrypted and decrypted domains. Key retrieval and storage should be a first-class part of data access layer, and the scope of access that keys have should be easy to determine.

How are Errors Handled? Can you easily ignore errors (boo! No!) Are there multiple error types that have to be handled differently? (Exceptions vs Error? boo! No!) Is it easy, at runtime, to discover the underlying type and cause of the error? Are there standards about what will be thrown where and when?

How Much “Magic” is There? How Easy is it to Get Around? Too much magic means difficult debugging. My friend Arya always uses the physics term “spooky action at a distance” to explain the issue. In my experience, more magic makes code “rot” faster. If you or your company lose experience with a codebase, magical ones take longer to brush back up on.

Podcasts At the End of the Indie Web

Sun, 25 Sep 2022 00:00:00 +0000

There’s been a lot of restrospecting lately, lamenting the loss of the “indie web” and its subsumption by content platforms like Facebook, Instagram, Reddit and Twitter. (I’ve always wondered where Tumblr fit in - more indie than any of these, but still - owned by Yahoo!). A few casualties that fell by the wayside: blogs, web comics, and independent, topic specific forums.

All of these media still exist, much diminished and publishing social posts to route you to their sites, but they are still self-hosted, free of editorial control and in their author’s hands. To some degree, these forms will probably stay on the web until millenials die out.

I want to highlight the real last bastion of the indie web, beeping on life support now: podcasts. Why does it feel like podcasts are drying up when there are more listeners than ever before? Well, why can’t indie musicians make a living anymore?

Podcasts filled an interesting niche in the indie web ecosystem: harder to make and host than a blog, too small for the big media companies to notice or care about. It was powered for over a decade by an underground comedy scene (okay one more, a tech-enthusiast audience, public radio archives, and the benign neglect of Apple. I think there are so many interesting parallels and departures on the road of blogs and podcasts in their respective heydays.

RSS: Thank the world for Aaron Swartz and Dave Winer. This foundational format laid the groundwork for blogs and podcasts, making it possible to pull updates from across the web and consume the whole thing in one place. It’s the underlying tech that makes general-purpose podcast players like Overcast and aggregators like Reeder possible. It gave you web users a feed before Twitter.

Big Tech Platforms: Over time, Google Reader became the standard way to curate your RSS feed. It worked great, a thousand blogs bloomed. And then Google decided that the rich stream of data it had flowing through its backends about millions of people’s reading habits was worthless. The app was shutdown and the blog ecosystem withered as a result. Twitter can take some of the blame, too - microblogging was embraced by the tech blogging set, with a power-law curve of rewards. The rich got richer and the long tail shrunk.

Compare with podcasts. Apple added podcasts support to iTunes in 2005 and then promptly forgot about them. There were awesome people like Scott Simpson working at Apple on podcasts, but it was a small part of the iTunes app that was left to flourish and diversify rather than be cultivated and harvested. Apple didn’t release a podcast-specific app for seven more years, and Apple was smart enough to leave the ecosystem alone.

Big Media Arrives: Blogs died (sort of), but podcasts consolidated. A few years back, big media companies took notice, and there was blood in the water. Gimlet Media at Spotify. 99% Invisible at Stitcher. The Murder Podcast Industrial Complex. Every National News Organization. Let’s not even talk about Rogan. Popular shows have been snatched up, smaller shows starved of oxygen.

Podcasts already fit a media model that the industry understood, and they had loyal audiences who could be analogized to radio or TV audiences. The podcast that started the gold rush came out of professional radio. Podcasting’s biggest star is a former TV actor and sports host. The entire business was legible to broadcast media, and they swooped in with exactly the same tactics as streaming apps: buy all the shows, lock them behind exclusive subscription models, cross your fingers that your exclusivity is better than theirs. How long until Disney starts a podcast network?

I’m glad to see some real indie networks survive, and even thrive, in 2022, but with more listeners than ever, the ecosystem feels more consolidated than ever, and the playing field ever more tilted. I hope we’ll see a resurgence of indie media at some point, but video hosting is damned expensive, and it’s harder than every to pursue a passion project that brings in little money.

GitHub Squash Merges are a Menace

Fri, 16 Sep 2022 00:00:00 +0000

I love squash merging. I think it’s the simplest way to maintain a legible commit history on main, a shared dev branch, etc. It’s easy for most people to follow, and it doesn’t require you to be too Big Brained about git. GitHub even provides a convenient interface for doing this, right in the pull request UI!

But GitHub’s squash merge workflow undermines the biggest benefits of squash merges: clear, simple, atomic commit messages that explain what each commit does.

How many times have you seen a commit like this:

commit ed0c641ce592e9cdaa5008264569df068e4e2ab4
Author: Joe Blubaugh <joe.blubaugh@gmail.com>
Date:   Wed Aug 31 13:46:06 2022 +0800

Fix a bug where we never commit a database transaction

* Fix a bug where we never commit a database transaction

* lint

* pull request feedback

* lint AGAIN

If your team squash merges in GitHub, this is the easiest commit message, the one that GitHub automatically fills in for you when generating the squashed commit. What we want is:

commit ed0c641ce592e9cdaa5008264569df068e4e2ab4
Author: Joe Blubaugh <joe.blubaugh@gmail.com>
Date:   Wed Aug 31 13:46:06 2022 +0800

Fix a bug where we never commit a database transaction

In db/users.go, the transaction that updates users' subscription level in bulk
is never committed. Users don't get access to the proper subscription content
as a result.

This change fixes the bug by using a `defer` to guarantee that the transaction
is committed if there is no error or rollback.

You can promote this practice at your company person-by-person, one slog at a time, but I’ve not yet found a way to get GitHub to stop filling in terrible commit messages by default.

How We Do Our Best Work

Wed, 31 Aug 2022 00:00:00 +0000

Dan Luu tweeted about some great work interns that he has mentored accomplished at Twitter:

One intern did https://t.co/nsFW20j9Hm and another did interesting data analysis then built a working prototype for across the fleet profiling that others were able to use to find real inefficiencies.

Those are things that could go into a staff promo packet as a major project.
— Dan Luu (@danluu) August 31, 2022

It made me think of internships I had, and when I produced my best work. There are two outcomes I’m particularly proud of:

I was an intern at the National Ignition Facility in 2008, and I proposed and wrote image processing code to detect and monitor flaws in laser lenses as they grow. That code was still in the system 5 years later, still doing its work.
I built and programmed a battery qualification rig for rechargable “wet” batteries that GE Healthcare used in its mobile X-Rays. I found and bought equipment, built the rig in a lab with help from techs, programmed it, designed the test protocol and kicked off the first qualification run.

On the other hand, I had a few experiences that were interesting enough, but didn’t stand out to me as particularly impressive results:

I did ESD testing for patient monitoring equipment. This was fun, in that I got to shock things with static guns and visit an anechoic chamber, but it was largely “assigned work” without a lot of autonomy on my part.
I worked for GE Healthcare writing RF switching designs in VHDL. These were for small, fairly simple components and were almost a direct analog of microprocessor design work I did at school.

There are qualities of the work I’m most proud of that are missing from the two “eh"s:

Autonomy: Starting with a clear but high-level goal like “how do we make sure these batteries work?” and being responsible for everything that addresses the goal is a powerful motivator. You know that you’re being trusted to do something real, and do something right. If you’re just doing work that’s already speced and given to you, the motivation is considerably lower.
Focus: This is something that interns get more than anyone else. Everyone knows that you’re a goner in 3 months. You don’t do on-call rotations. You don’t do roadmapping. You find your project and you work your project. That’s a productivity superpower. Context switches are expensive, shipping quickly is essential, and focus fixes that.
Mentorship: The other side of the autonomy coin - you will get stuck on a high level goal, at any phase of your career. You need people you can safely ask for advice, and a surprising percentage of people don’t get this. Their internship hosts may be too busy or uninterested, their manager may be managing too many people, “competing” teams at dysfunctional companies may be actively undermining it.

These conditions won’t make up for a lack of skills or ambition, but they are critical enablers for skilled, ambitious people. When I think of professional frustrations of mine (dropping out of grad school, botched design proposals, having to fire a direct report), one of these was missing, or worse, being actively undermined.

PS: I’ll write that second blog post about OKRs, I swear.

Thoughts on OKRs

Wed, 18 May 2022 00:00:00 +0000

OKRs are one of those business ideas that are just simple enough to be dangerous. You think you understand it in a day, and you can see where your company is falling short: lacking focus and underdelivering. You see how clear, measurable goals could improve the situation.

Here’s the one sentence version: You set objectives, and for each objective come up with several key results that you can measure to see if you met the objectives. There’s a whole book for implementing this process in large organizations, and a mystique because of its association with Google’s early and frightening effectiveness.¹

I’ve worked at four companies that used some variation of OKRs (including Google!). All of them set a hierarchical pyramid of OKRs that cascaded down, rather than a set of OKRs that worked bottom-up, even if the company wasn’t in a crisis and would have benefited from promoting innovation.

I want to compare OKRs to Performance Reviews and Roadmapping. They’re all worthwhile ideas that can bring discipline and structure to the chaotic world of business, all dreaded by their participants for some reason. I think they have similar failure modes, and most companies get into a state of learned helplessness about them for similar reasons. For OKRs, here are the failure modes I saw repeatedly, and took part in. Each one of them leads to the next, and I bet you can think of the way these happen in performance reviews and product roadmaps, too.

It doesn’t take up enough of leadership’s time.

If you’re a manager, you’re screaming right now. I know, I know, they feel like they take forever. But if the process is going to be effective, then the connection between individuals’, teams’, groups’, divisions’ and the company’s OKRs need to be tight and logical. That means a high-bandwidth, highly synchronized exercise with bidirectional feedback. That’s expensive and time consuming and it’s the most important job corporate leadership can have. Set the goals for the people who do the work. Focus less on execution as a leader and more on goal setting and measurement.²

The process always starts too late and runs too long.

Ironically, the process starts late and takes a long time because it isn’t taking up enough of leadership’s time in the first place. So the business sets up some cascading set of “due dates”: First, company OKRs, then a week later a department and on and on to an individual. Say the due dates are at least a week apart. You’re looking at a six-week-long process, and every step of the process is going to be a day or two late, so you’ll fudge until it’s an eight-week process. This is where OKRs end up in the same place as performance reviews. Projects are running late, so planning is running late because it’s getting in the way of projects. You have to pull out of the death spiral.

OKRs aren’t actually committed to.

If it takes 3 weeks into the quarter to define your OKRs for the quarter, well, congratulations, you spent those 3 weeks choosing what you’ll actually do this quarter. You’ll make the case that the things you’re working on are really important and already in progress, even if they’re not lined up exactly with the department OKRs. And they may be really important, and the OKRs may be wrong because we didn’t spend enough time working on them and took too long to deliver them, and we didn’t leave room for bottom-up objective setting. So we do the important things we’re already doing, and we’ll fudge the KRs a bit at the end of the quarter.

There are rarely consequences for failure, or much learning from it.

If the Objectives aren’t committed to, and the KRs don’t matter, then why fire someone for a huge miss, or change the whole way your department works? Just write new ones, carry late projects over, and here we go again.

Welp.

How can you improve OKR definition and actually realize their benefits? Stick around for the next post.

Given Google’s business misses in everything except ads, I wonder if this isn’t due for another look. ↩︎
This happens to jibe nicely with the lessons of High Output Management, my favorite book on business execution. ↩︎

You Should Squash Merge to main

Thu, 10 Feb 2022 00:00:00 +0000

Every so often I’ll see a meme on Twitter like:

and it makes me so mad. For good reason! OSS project repositories that support merge commits to main are usually littered with useless comments like: “Merge change from $USER, $PR”. It makes the commit history on main utterly useless and you get to check the various feature branches in a never ending snake of commits to find out what the hell changed.

You should squash merge.

Here’s why you should squash-merge:

Every single commit on main should be releasable. A chain of commits that are merged in one go are not necessarily releasable.
The commit messages in feature branches are usually uninteresting, and that’s a good thing. It is absolutely OK for your commit log to look like:
- $TICKET: Prepare for new service by creating configuration files and code skeleton.
- Fix missing environment variable
- checkpoint for EOD
- finished!
as long as your merged commit on main is:
- $TICKET: Prepare for new service by creating configuration files and code skeleton.
I believe that advocates for keeping the commit history of a branch during merges think that you should re-write your feature branch commits to be more meaningful before actually merging. What a waste of time!
A messy feature branch history is good, actually. Messy branch history allows you to push to remote at any time without really thinking about it. Replication helps when you leave your laptop on the train, or to quickly share the state of your work with a coworker. After it’s on the remote you should try not to force push it, especially if you’ve shared it.
You don’t lose anything interesting in squash merge. It doesn’t matter what the commit was where you forked from main. It only matters what the difference between main and your branch is when you merge. You can most cleanly see that change by comparing sequential commits to main that have nice clear messages.
Merge commit messages are useless out-of-the-box. Why write the same commit message twice - you have it somewhere in your feature branch and then again (hopefully) in your merge commit.
You only need to view the commit log of main to understand what changed when. This is what you care about when you release software.

My actual favorite solution to this problem is to use the Gerrit workflow that guarantees any merged piece of code is a single “change set” that’s merged atomically. Squash Merge is the closest thing Github provides in its UI, so do that.

Why I (Still) Use Vim

Wed, 15 Dec 2021 00:00:00 +0000

Every so often I wonder if I’m making trouble for myself by doing all of my typing in vim. Writing for this blog, writing engineering plans and design documents, and writing code. There are tools that are made especially for doing these jobs, and vim certainly doesn’t have any facility for making diagrams.

All vim can really do is edit a bunch of bytes, trying to represent them as formatted text. It doesn’t even really understand code, which is what most people use it for.

But still, after I take a little trip over to VSCode or Jetbrains or Nova or Scrivener, I end up coming back to vim, finding a vim plugin that scratches my itch, and getting right back to it. Here’s what keeps me here, using this old, weird, but great tool.

Speed

Opening a file in vim is fast. Jumping from place-to-place in vim is fast. Typing in vim is fast. Fast is a feature

Splits

Every good idea has ways to split windows, but in vim it’s stupid fast, and stupid easy to manipulate. Here’s magic spell for splitting the window vertically, then the left one horizontally, then navigating to the right-hand one.

:vs|sp
^+W k

You end up with:

----------------
| foo   | baz_ |
|       |      |
| ----- |      |
| bar   |      |
|       |      |
|       |      |
----------------

Ubiquity

Have you ever been on a Linux box without at least vi? Even Arch and Alpine ship with vi installed.

CoC

With the advent of the Language Server Protocol, many language-specific features like navigation, highlighting, and completion have been decoupled from editors. Commander-of-Completion is a great vim plugin that adds LSP support to vim and makes it less arduous to add various new languages to your toolkit.

Modal interfaces like vim’s are generally regarded poorly in the UX community. The other pre-modern text editor, emacs, avoids modes but relies heavily on modifiers to enable non-typing modes.

Keyboard-Oriented

I never touch my mouse using vim. In many cases, you simply can’t use the mouse to get anything done. This takes some getting used to, but it’s a blessing for your wrists and shoulders.

My Country, Tis of Thee

Tue, 21 Jul 2020 00:00:00 +0000

I’ve spent a lot of time the last 3 years watching Umberto Eco’s sign’s of Ur-Facism manifest, and manifest, and manifest. I’ve seen a lot of things that I was raised to believe ~~wouldn’t~~ couldn’t happen here happen in sequence, with frightening speed. Out-of-control (willfully so) police beating and gassing people in the street, night after night after night. Destruction of public schools, along with requiring states fund religious schools. A Justice Department hell-bent on revenge for the President. Open xenophobia, brazen racism by members of congress. I thought we left that behind when Strom Thurmond finally died.

I worry about my wife, who holds a green card, and is thereby subject to the whims of immigration officers and state police who have an almost infinite number of crimes at their disposal. I worry for my co-worker Dan, a DACA kid who’s never done a thing wrong and still lives on unstable ground. I worry for everyone I know who opposes Donald Trump as his DoJ bends every law to its cruelest angle.

Looking back, you can see the seeds of these problems planted. Occupy Wall Street & Occupy Oakland happened during Barack Obama’s presidency, and protestors (including yours truly) were gassed and beaten by cops on more than one occasion. Expansive theories of executive power go back at least to John Yoo and the war on terror. John Yoo’s now sitting pretty at Berkeley and I wonder how he doesn’t get his car egged every day. Public Schools have been under assault for decades as standardized testing extends a widening grip on the entire system.

All of these seeds have been fertilized and nurtured by the fascists who prop up the President, and who will doubtless get sinecures at think tanks when they should be expelled from polite society and wind up unemployable. Even if we beat back the kudzu in November, I worry that we’ve passed a point of no return. Re-building after this can’t be about going back to before a mistake - it has to be going forward and changing our society.

Font Measurements

Mon, 06 Jul 2020 00:00:00 +0000

TLDR;

I learned a bunch about rendering fonts and I thought it would be interesting to read about them from a programmer’s perspective. I gained a ton of empathy for type design and type-setting developers, as they work with a dizzying variety of screen resolutions, font styles, and a wide variety of device speeds to produce type that looks as good as it possibly can under harsh conditions.

Background

I’ve been working on a project involving a two-color e-ink screen. I’m drawing on the screen and that involves drawing text. The project is written in Go. There are common font-rendering librares like Cairo, which are written in C. I find compiling against C libraries like using cgo difficult and complex, so I was really interested in a Go-only solution.

Fortunately, Go supplies a decent image package right out of the box. Combined with the “semi official” font package, it seemed I could do everything I needed.

Unfortunately, I’ve selected some fonts that the most popular font-parsing package can’t handle. So I’m using a different package that can parse many more fonts, but until yesterday didn’t support any of the rendering functions required for drawing on images. In writing and submitting that code, I ended up learning a lot about how fonts are represented in files and what happens when they’re rendered.

I’m going to focus on laying out text left-to-right below. However, there are many languages that use right-to-left horizontal layouts, vertical layouts that go either left-to-right or right-to-left, and languages with alternating line directions. Many of the measurement concepts below apply but may need to be rotated or flipped from a horizontal left-to-right perspective.

Font-y terms

A font is a collection of glyphs - letters, numbers, symbols, ideograms, emojis (😭) that the font is able to draw.

A typeface is a collection of fonts that share a style. This can include italic, bold, or even serif & sans-serif variants.

A serif is that little slab at the bottom or top of a glyph. Here’s an example.

Fonts then, may be serif or sans-serif fonts, though there are other styles like Blackletter. This primarily applies to the Roman alphabet, and Roman-influenced alphabets. Chinese fonts have their own styles that are similar - Songti and Heiti

Font metrics

The measurements described here are common to both manual typesetting and computer typesetting. Manual typesetting is very rare these days, but the terms originate from that practice.

The capital Q and lower-case e below both show their measurements.

The baseline is the line that all text on the line is laid-out relative to. For each rendered glyph there’s an “origin” point on the baseline.

The advance width is the space from a glyphs origin until the origin of the next glyph.

The ascent is the height of a glyph above the basline and the descent is the depth below the baseline. Many fonts also provide ascent and descent measurements that are the maximum of these values for all glyphs. This can help with line spacing.

The bearing of a glyph is the space between the glyph and its origin, or the origin of the next glyph.

Fonts are sized in units called points. There are 72 points per inch.

Special topic: Kerning

Kerning is the adjustment of glyph spacing based on glyph pairs. For example, WA is often kerned tighter than the default spacing to reduce the large diagonal line that you’d otherwise have between the two.

Special topic: ligatures

In some fonts, multiple letters may be represented by a single glyph. In some fonts, fl or ff may be joined together so that the characters overlap.

Font file formats

A very common text-rendering system on modern computers is Freetype. Windows is not included in that family. The common windows font-rendering system is ClearType.

Coordinate system

Font sizes themselves are expressed in “points”, and there are 72 of them in an inch. So you should expect a 72 pt font to take up about an inch from line-to-line.

However, computers don’t represent fonts that way. Each glyph in a font is represented by a set of “drawing instructions.” The instructions may specify “draw a line from A to B, then draw a cubic bezier from B to C, then draw another line from D to E.” These drawing instructions are represented on a single coordinate system, which is then scaled by the point size to create the final set of coordinates.

Font file coordinate systems use a special representation called “fixed point.” Because fonts need to represent sub-pixels, we can’t use whole integers. Why not use floating point numbers? At different scales, floating point numbers have different levels of precision. So font files use fixed-point representations for their various numbers. You can think of these as floats with a fixed precision. Several fixed-point formats are used in font files for kerning, glyphs etc. The glyph coordinates are written using “26.6” The integer part has twenty-six bits, and the fractional part has 6 bits, to fill the space of a 32-bit integer. This means that the resolution of the fractional part is 1/64. Here are some binary representations of numbers in fixed point:

Number	Fixed-point
1	1.000000
1.5	1.100000
2.25	10.010000
5.75	101.110000
1 63/64	1.111111

So, to render a font on-screen, we need to put all of these pieces together:

Take the string you need to render and get the coordinates for each glyph.
Substitute ligatures where necessary, and adjust the spacing based on the kerning between glyphts.
Create a pixel coordinate system based on the dots-per-inch (DPI) of the screen you’re rendering to. iPhone 11s have a DPI of 463 pixels per inch.
Scale all the font coordinates (at 72 points per inch) by your screen grid (463 pixels per inch), using fixed-point math.

You may do all the above glyph-by-glyph, though you may have to “go back” to deal with ligatures.

Rasterization

Once you have the set of all font coordinates, you need to actually draw those dots on-screen. Some coordinates will be for fractional pixels, but we have an integer number of pixels. This is where “rasterization” comes in. Rasterization is done for just about anything displayed on a computer screen, from video games to text, to photos. In font rendering it involves shifting items to fill pixels, and adjusting colors so some pixels on the edge of letters are a lighter gray so the pixels appear partially filled.

Here’s an excellent site that explains a rasterizer for TrueType in greater detail.

Type rendering is a topic filled with nuance and artistry - the above only provides a simple summary. Some of the computer industry’s most talented graphics engineers have worked in this field for decades. Computer displays are now the place where westerners spend the majority of their reading time, so clear and attractive type on screens remains incredibly important.

Influence

Sat, 30 May 2020 00:00:00 +0000

I got an email from an old co-worker the other day:

I was thinking about you and how you are a person I see as not being particularly embroiled in office politics, but also as having a lot of influence and knowing how and where to leverage it. I was curious if you had any words of advice for how you got to that place.

Of course I was happy to hear that I come off this way, and as I thought about how to reply, I found myself writing an essay.

So, about influence, and politics, and etc.

When most people say “office politics”, they seem to mean the “dark side” of relationships at a company. Who gets the boss’s ear? Who “wins” when there are decisions to be made? Who has their ear to the ground?

When people talk about influence and leveraging it, they’re talking about the “light side” of the same thing. Who shapes decisions? Who “makes things happen”? Who shifts behavior, changes the tone of the company?

Office politics and influence function in really similar ways: building relationships with your co-workers, building trust with them, and keeping lines of communication with them open. Most decisions are made before the decision gets made, so being around for those early conversations is where you can exercise leverage.

Some ways that I build relationships with co-workers:

private messages when they do something great.
always show up for those coffee chats and lunches. This is how you get to know people way outside your group. I have one lined up with our director of finance and a developer who lives in Europe. It's rare to get to talk with them both. If there isn't something like this at a job, it's usually pretty easy to kind-of start one.
not-too-frequent 1:1s with people in other teams. I have about 5 of these going, and I do them every 3 weeks. I try to vary the rotation enough from time-to-time. They're hard to drop, though! Be judicious.
ask people for help! most people love to feel helpful.

All of these have the advantage of being genuinely nice things to do! They make me feel good, they make my co-worker feel good. The hardest thing about them is making them happen when I have a lot of other stuff to do, or I’m feeling stressed out or distracted. But it really is a part of your job, especially as you grow into a leader in your team, organization, company.

I also work pretty hard to build trust. The best way I know to do this is to do your job as best as you can, and to be transparent with people. When you’re working on something hard, getting people’s opinion when the idea isn’t fully baked does a lot - you can get better ideas, and you show that you’re trusting a co-worker with something delicate, that you need and value their contribution. When things aren’t going smoothly, being straight with your boss

Another way that I build trust and credibility is calling attention to issues that affect “everybody” - but always offering to help make the improvement happen, or even do initial legwork to offer an idea for a solution. This makes it clear that you’re doing more than complaining; that you genuinely want things to improve and you want to be part of that.

Finally, asking for hard feedback, and giving hard feedback are important skills for building trust. The asking is much harder to do, and. I don’t agree with everything in the book - even maybe less than half of it - but Radical Candor helped reshape my ideas about trust. I think the book goes too far, but being able to give criticism honestly and with respect is crucia. Even more important is being ready to receive it without rejecting it or saying “yea, but”. Living with it. Thinking about it. Then accepting the parts that are true and rejecting the parts that are undermining you. If you get sexist feedback, throw it in the garbage and note how much you can trust that person.

So, how is that different from office politics? It’s not just tone, but tone is a big part of it. It’s about genuinely wanting good for other people and for yourself. It’s about character, and about honesty. It’s about genuinely wanting good things for yourself, your co-workers and your company, and striving to make that happen.

Influence is definitely related to competence, but also to prioritization. You’ll be more influential if you’re addressing real needs that people care about - especially your bosses. If they’re good bosses, this is both easy and not morally perilous.

Unfortunately, none of this holds up when you’re surrounded by people of ill will. In that sort of situation, find the other people of good character in your workplace and keep your head down. If you can get out, get out. Don’t spend your energy trying to change people who show you they can’t be trusted.

An aside about conflict:

Every so often, influence doesn’t work this way and you have to stick your neck out. Again, the tone is really important here. In our two big meetings after the layoffs, I directly asked our CEO and my VP some uncomfortable questions about how decisions about who was being laid off and how the layoffs were communicated. I followed up with more direct feedback in private conversations. I think this made a difference in how decisions will happen in the future and in gave my co-workers a better picture of what happened. The way I made those conversations work was by communicating respect for my bosses as people as early as I could in the conversation, and assuming good will by them.

About assuming good will - it’s complicated, but important. The key point is that by assuming good will, you’re able to get more honest and less-defensive communication with someone, even if they’re acting with ill will. That doesn’t mean avoid confrontation, but it means approaching confrontation as though it’s based on either conflicting priorities or mistakes rather than malice. This is a key conflict skill that I wish I’d developed so much sooner! Life would have been better. After the conversation you draw your own conclusions about whether you’ve been dealt with honestly and you can choose your next steps with good information. When good will is real, things will often change for the better as a result of the conflict.

This all sounds very high minded. In writing it, I’ve noticed many ways that I fall short of these ideals. But it feels right as a roadmap.

Lucky

Fri, 22 May 2020 00:00:00 +0000

We went through layoffs at work recently. Like a lot of companies, the belt is tightening, and the bathtub drain is getting blugged. I’m lucky to have not been laid off. Lucky again - this is the third “surprise” layoff where I’ve kept my job. I’m batting 1.000 but I know my number has to come up sometime.

I graduated from college in 2009, 9 months into the Great Financial Crisis. I was lucky to get into a good graduate school on a stipend that paid my rent, and even luckier to be leaving undergraduate without student debt. I was lucky that when I needed to quit graduate school I could find a government job.

I was lucky to go to a particular concert, meet a particular person, and lucky that we fell in love. Lucky that the US Government didn’t fuck around with her life and we were able to stay here in California.

I’m lucky to be living through this epidemic in good health, with plenty of food and a functioning society.

Count your blessings.

Fast is a Feature

Tue, 19 May 2020 00:00:00 +0000

We all use dozens of pieces of software a day - email clients, web browser, email clients inside web browsers. Cameras and chat, digital art tools and spreadsheets. We switch between them dozens of times, and when we really get into a task, when can sometimes achieve a state of flow, a state of satisfying and effective effort that lets us accomplish more than we thought and enjoy it all the way.

But then you add another layer to your Photoshop project, or attach a picture to an email. The beachball starts spinning, the text box gets laggy, and you start thinking about opening that Twitter tab again.

This is one of the reasons I still use vim after all these years. Even in large software projects with plugins for syntax highlighting, autocomplete and build-on-save, the navigation and input remain quick. But not always - vim on my home computer feels faster and more responsive than my work computer, because I’m asking it to do less. When compared to a product like IntelliJ/GoLand, it’s like a car against a horse. Very few editors are both as fast and as capable, and I can thereby keep going as long as the work requires.

Every time your software lags, it’s a chance for your user to lose their state. Every time you make them move the mouse to open a menu, they have to go away from their task. Put your menus in context with the work, not in a toolbar. Make your software as quick and as smooth as possible. This is what makes people love software, not flashy features.

Beef Barley Mushroom Soup

Mon, 15 Jan 2018 14:57:58 -0800

This is adapted from Tom Lacalamita’s Pressure Cookers for Dummies, but you should use the Serious Eats method of browning stew beef in one large piece, then cubing it before cooking, for better results.

Ingredients

1/2 lb stew meat
3 carrots, thinly sliced
3 stalks of celery, sliced
1/2 lb of mixed mushrooms, trimmed and quartered
1 cup barley
2 or 3 bay leaves
8 cups of liquid, stock preferred
Chopped parsley

Steps

Cook onion in oil until fragrant
Add the meat and cook until browned. Remove and chop into small cubes.
Add the carrots, celery, barley, mushrooms, meat, stock, and bay leaf to the pressure cooker.
Bring o high heat and cook for 20 minutes. Release pressure with the quick release method.
Add parsley, and season the soup with salt and pepper before serving.

Ricotta Chicken

Mon, 15 Jan 2018 14:57:33 -0800

The secret here is that you can replace the Butter in Butter Chicken with just about any high-fat dairy product. Ricotta, sour cream, greek yogurt will all work. They change the flavor of the dish, but they’re all good variations.

This is adapted from a recipe in Tom Lacalamita’s Pressure Cookers for Dummies

Ingredients

1.5 lbs chicken, cubed
1 onion, minced
1 Tbsp minced ginger
2 tsp garam masala
1 small chili, minced
6 oz tomato paste
2 cups chicken broth
2 Tbsp minced cilantro or parsley
1/2 - 2/3 cup high-fat dairy. Butter chicken is 1/2 cup cream, 3 tbsp butter. Limes for serving.

Steps

Salt and pepper the chicken.
Cook onion, ginger, chili and garam over high heat in oil, until browned.
Add the chicken and brown in spices. Add the broth and tomato paste, stir to deglaze the pan.
Bring the pressure cooker to high heat and cook for 8 minutes. Release pressure with a quick-release method.
Over low hit, stir in the dairy. Serve over white rice, and squeeze fresh lime juice just before serving.

Pressure Cooker Red Beans and Rice

Mon, 15 Jan 2018 14:38:51 -0800

Red beans and rice is simple, fast and good. Use a good hot sauce, use more seasoning than you think you’ll need. This recipe scales well - 2 or 3 cups of beans will fill most normal pressure cookers.

Ingredients

2 cups of red beans
1 large onion
1 head garlic
1 cup chopped celery
1 cup chopped carrots
1 14.5 oz can of diced tomatoes
2 tsp hot sauce
2 or 3 bay leaves
1 smoked ham hock or 3 strips of bacon or 1 can of (gasp!) spam, etc
4 cups of water
1/2 lb of andouille or other smoked sausage
salt to taste (around 1 or 2 Tbsp)

Steps

Rinse the beans, and let them soak while you do the other prep.
Chop the onion and garlic. Saute in olive oil until fragrant. Add the carrots, tomatoes, hot sauce, and bay leaf. Cook for 2 or 3 minutes. Add the ham hock, beans, water.
Bring pressure cooker to high pressure, cook for 20 minutes. Release pressure via “natural release”. If beans are still hard, cook another 5 minutes.
Add sausage, salt and pepper. Cook with cover off until desired thickness is reached, about 5 to 10 minutes.
Serve over white rice.

Michelada

Sun, 10 Dec 2017 18:52:49 -0800

Originally from Ever in Transit, this is the best tasting version of the drink to me. I like to use the tomato juice from a can of peeled plum tomatoes. I use the whole tomatoes to make sauce, and keep the juice to make two micheladas with.

Ingredients

Lager beer
Tomato Juice
3-4 splashes Tapatio
splash of worcestershire sauce
splash of soy sauce
juice of one lime or lemon

Roberta's Pizza Dough

Sun, 10 Dec 2017 18:52:38 -0800

Originally from the NYT, makes enough for two pizzas.

Ingredients

306 grams of flour (50/50 00 and bread, or 100% AP)
2 grams of active dry yeast
4 grams olive oil
8 grams of fine sea salt
200 g of tap water

Steps

Combine flour and salt in mixing bowl
Combine water, oil, and yeast in mixing cup. Add to dry ingredients and knead until incorporated. Let rest 15 minutes.
Knead dough about 3 minutes. Works well with a dough hook.
Divide dough in half and shape into two balls. Cover with a damp kitchen towel.
Let rise. 3 to 4 hours at room temperature, or 8 to 24 hours in the fridge.
Shape, top, and cook.

Persimmon Roses

Mon, 04 Dec 2017 19:58:51 -0800

My wife found this recipe on tastemade.com. These are fatty, sweet, and delicious. Great brunch food. Makes 4 roses.

Ingredients

2 persimmons
1 sheet of puff pastry
1/4 of a wedge of brie
4 slices of prosciutto

Equipment

Muffin tin

Steps

Cut the persimmons in half from the top, then cut the tops off of each half. Slice the halves from the bottom to the top, 1/8" or less thick.
Split each slice of prosciutto long-ways.
Cut the brie into thin chunks.
Lay out the puff pastry and roll it gently to increase the size. Cut the sheet into four pieces.
For each piece of puff pastry, lay the prosciutto, then the brie, then the persimmons down a long edge, overlapping about half of the piece.
Fold up the uncovered part of the pastry to form a sealed bottom crust of the rose.
Roll up the pastry to create the “rose”, then place in the pan.
Bake for 15 minutes at 400 degrees F, then 30 minutes at 350 degrees F.

Garage Shelves

Sun, 15 Oct 2017 22:18:16 -0800

I started taking paternity leave on my first daughter’s due date. (Un?)fortunately, she was born late, and I found myself with both a lot of free time, and a lot of pent-up anticipatory energy.

My workshop was a bit of a mess, since I’d acquired more tools - including a heavy miter saw and grinder. My mother-in-law was staying with us and helping with chores, so I decided to focus on cleaning up the shop. I built a sturdy set of shelves from scrap lumber and cheap sheathing plywood to hold my bench tools when they’re not in use.

Approach

The legs were made from two 2x4’s screwed together. The cross bars for each shelf were also made of spare 2x4s. Each shelf is made of a single thickness of 15/32" sheathing plywood that I picked up cheap from a big box store.

Similarly to my workbench, the inner shelf is cut with gaps for the legs, then slid into the frame at an angle before being set down. Instead of fixing the shelf tops like a table top, I chose to nail them directly to the frame.

Results

Ugly, and very functional. I made the upper shelves around my shoulder height, and the lower shelves around waist height, which means each shelf and the floor have plenty of clearance for large storage boxes, tall tools, etc.

This was way cheaper than the much less-good plastic shelves I bought from a big box store. Make your own garage shelves!

Guitar Shelf

Sun, 13 Aug 2017 22:39:34 -0800

I have a lot of guitars and ukuleles, and I was also feeling like I hadn’t played them enough. When we moved at one point, I sold all my guitars but one, but that honeslty didn’t last too long. Now that I own a home, it was high time to build myself a real place to keep them. Keeping instruments close at hand reduces my barriers to practice and play.

Approach

I bought some shelf brackets and used some spare 1/2" baltic birch plywood. I measured each of my instruments’ necks at the nut and added additional space for the thickness of some felt tape that I’d be lining the slots with. I left space for two guitars - a steel string and a classical guitar. I also cut space for three smaller instruments - two ukuleles and one Loog Pro three string guitar. Finished with wipe-on polyurethane, which is pretty much how I finish any piece of furniture.

Results

A good-looking shelf that I’m happy to have in my living room. The studs in our walls are 18" apart, which makes shelves a little awkward. I didn’t feel OK going 36" between brackets with such a thin piece of plywood, so the shelf looks kind of over-built.

I’m playing guitar more often :-)

Workbench

Tue, 03 Nov 2015 22:13:59 -0800

Every workshop needs a good work bench. A good bench is stable and flat, letting you estimate the flatness of your workpiece, and with plenty of space to hold the work and clamp it down.

You can buy nice workbenches for many hundreds of dollars, and then you can buy nice clamps for many more. I wanted to use this project as a learning experience, though. So I spent my time researching table plans, trying to pick one that would give me a good surface without being a big expense.

Since I knew my next project would be a dining table , I needed to make a bench big enough to build a table top on.

Plan

I ended up choosing this plan from Fine Woodworking because it’s easy, includes guidelines for a vice, and holes for bench dogs and holddowns, which I’ve found to be very useful for hand work and routing.

Results

This plan was truly pretty easy to build, but I think it has a few flaws. The threaded rod that holds the legs in tension can pop out of tracks in the stretchers. I found a rubber mallet essential to getting this done. Fastening the table top with clips and slots feels like overkill to me. I had a lot of trouble with that, and MDF doesn’t really shrink and expand much. I think you could get away with simple L-brackets, under-tightened.

The bench is incredibly sturdy, and can be easily tightened up with the threaded rod. The top is flat and solid - one panel of MDF is also cheap enough that if you ever totally screw up the top with a tool or some weird chemical, it’s not going to break your wallet to just get more and replace it.

Cardsharp

Thu, 03 Sep 2015 14:57:04 -0800

I’m a big fan of Zach Gage’s games. He recently released a game called “Sage Solitaire” in which solitaire is played with poker hands. You can make a Full House, Straight, Straight Flush, etc, to clear cards from the board. To choose what hands to make, it’s pretty important to know what cards remain in the deck, much as it can be in poker.

I wrote Cardsharp to help me keep track of remaining cards in a Sage Solitaire deck. It’s helped improve my score quite a bit, and gave me some recent Python experience before I started looking again for backend engineering jobs.

Cardsharp is build on npyscreen, an ncurses library for creating interactive Python terminal applications. It’s got some great basic layouts like “Simple Grid” and a easy-to-understand interaction model, with Action and Data Controllers that interact with shell input.

Mon, 01 Jan 0001 00:00:00 +0000

Pankaj Chat

Networking guy at VMWare. Looking into Obs space

Notes:

Honeycomb vs Lightstep
Observe vs Lightstep
“not the first tool”
“how come you guys didn’t start putting things together”
“when people are debugging things, they’re looking at everything”

Application-Side observability.

blog posts on joe blubaugh

Zen: A Browser You Can Love

Multi-agent coding and the resurgence of the terminal

Introducing nvim-beads: Manage beads in neovim

Podcasting is dominated by warm applesauce

All the data had nowhere to go

Finding "Just Fine"

LLMs are great at box diagrams

Ten years of joeblu.com

What was *WTF with Marc Maron*?

Appendix

hugo-editor: A Claude experiment

Why would you use .gitkeep?

Think big, build small

Thinking big

Build small:

Our Furtive Encrypted Future

American schools should all have uniforms

Loog Piano: My review

Quality

Sound

Keys

Accessories

Production

Support

Nevermind

What's a "thunk"?

jq -n \-\-argjson

Don't use mocks

Mocks create brittle tests

Mocks encourage tests that are too fine-grained

Alternative: Use a Fake

Alternative: Don’t unit test calls to complex dependencies. Write integration and behavioral tests for multiple components

What about Stubs?

Supreme Court DoS attack

Looking in Go's Mirror: How and When to use `reflect`

reflect package basics

Value, Type, and Kind

Settability

Deserializing structured data with reflect

Using struct tags to access metadata

Let’s not overdo it

Additional reading

What is software for?

Code is not Prose

Nasty refactorings with go.mod replace

Caroline Ellison is not a sympathetic figure

Unmarshaling JSON in Go: The weird parts

Evaluating New Tools

Podcasts At the End of the Indie Web

GitHub Squash Merges are a Menace

How We Do Our Best Work

Thoughts on OKRs

It doesn’t take up enough of leadership’s time.

The process always starts too late and runs too long.

OKRs aren’t actually committed to.

There are rarely consequences for failure, or much learning from it.

Welp.

You Should Squash Merge to main

Why I (Still) Use Vim

Speed

Splits

Ubiquity

CoC

Multi-modal

Keyboard-Oriented

My Country, Tis of Thee

Font Measurements

TLDR;

Background

Font-y terms

Font metrics

Special topic: Kerning

Special topic: ligatures

Font file formats

Coordinate system

Rasterization

Influence

So, about influence, and politics, and etc.

An aside about conflict:

What was WTF with Marc Maron?

`reflect` package basics

Deserializing structured data with `reflect`