10 interesting stories served every morning and every evening.

Interview With Mitchell Hashimoto

alexalejandre.com

Mitchell Hashimoto was be­hind Vagrant, Packer, Consul, Terraform, Vault, Nomad, Waypoint and now builds Ghostty and Vouch.

In this in­ter­view, we talk about ter­mi­nals, Zig and open source.

You’ve been in­ter­viewed a lot. Why do peo­ple like to in­ter­view you?

In in­ter­views, every­one comes from a dif­fer­ent an­gle. Many peo­ple want to know how the soft­ware en­gi­neer­ing to busi­ness founder mind­set tran­si­tion went. Then oth­ers are in­ter­ested in prod­uct stuff, the work I did at Hashicorp or Ghostty now. What’s dif­fer­ent here is there’s no known agenda com­ing into it; nei­ther of us have any­thing to sell.

What do you find so fun about ter­mi­nals? Like, why Ghostty?

I spent ~15 years build­ing CLI ap­pli­ca­tions (not TUIs like we see nowa­days). Through that process, I ac­ci­den­tally learned how to color things, move cur­sors etc. Leaving Hashicorp, I wanted to sharpen my tech­ni­cal skills (where they’d grown dull from ne­glect) and specif­i­cally work on: Pre-AI GPU pro­gram­ming, desk­top/​sin­gle node sys­tems pro­gram­ming (spending so much time on the dis­trib­uted side where you did­n’t worry about cache lo­cal­ity or vec­tor op­er­a­tions, since net­work costs dom­i­nated). I also re­ally wanted to play with Zig. I wanted to sat­isfy those 3 things.

After 15 years build­ing CLIs, I did­n’t un­der­stand how a ter­mi­nal em­u­la­tor worked. I knew the com­po­nents of a ter­mi­nal but re­ally wanted to un­der­stand how it worked, which would also let me work on the GPU, desk­top and in Zig. My goal was to run vim and the com­piler in it, have it build it­self, then throw it away. But as I learned more about the ter­mi­nal ecosys­tem, I un­der­stood noth­ing fit the niche I wanted: fast, fea­ture-rich and na­tively cross-plat­form. I shared it with a few friends in Discord, who asked if they could share it with oth­ers be­cause they were ac­tu­ally us­ing it every day. The Ghostty Discord was just my friends’ group chat which got re­pur­posed. I did­n’t want to pub­lish be­cause my pub­lic per­sona would gen­er­ate too much un­due at­ten­tion, so I ran a pri­vate beta for a long time.

How can we push ter­mi­nals harder?

I don’t sup­port push­ing ter­mi­nals to the ex­treme. Sure, they’re an ap­pli­ca­tion plat­form ca­pa­ble of the same things other ap­pli­ca­tion plat­forms on top of the OS are like the browser, old Java app run­time en­vi­ron­ments. You could build all func­tion­al­ity into it: video and mi­cro­phone ac­cess, re­spon­sive lay­outs… You could.

But the browser is good at some­thing, the desk­top is good at some­thing else and text-based (monospaced-grid) ap­pli­ca­tions are also good at some­thing unique. These text-based ap­pli­ca­tions should be quick to im­ple­ment, easy to in­ter­act with, clear in their se­cu­rity model. There’s a lot of op­por­tu­nity in the ecosys­tem here and I’d love to build more pro­to­cols to en­able that.

Terminal-based ap­pli­ca­tions lend them­selves to com­po­si­tion bet­ter than other par­a­digms. TUIs less so, but most CLI tools have mech­a­nisms (beyond stdin and std­out) to use them like a func­tion (the UNIX do one thing phi­los­o­phy is the ex­treme). Neovim and AI tool­ing of­fer ever more cmd­line flags. A world of bet­ter ter­mi­nal ap­pli­ca­tions, is a world of bet­ter au­toma­tion, script­abil­ity.

I want to make the ter­mi­nal a spe­cial place for ap­pli­ca­tions. The PTYs in-band sig­nalling (an un­struc­tured byte stream with es­cape se­quences) is a big prob­lem. The Nushell ecosys­tem tries to fix it with an­other layer, but we need a fun­da­men­tal im­prove­ment. Many peo­ple dis­like the Microsoft ecosys­tem, but PowerShell gets a lot right with struc­tured data.

What do you think about non-legacy ter­mi­nal APIs?

My guid­ing star is how we now have mul­ti­ple ma­jor, huge ap­pli­ca­tion plat­forms: the browser, emacs, the whole Apple ecosys­tem, Microsoft ecosys­tem, Android, video game con­sole plat­forms. These ecosys­tems have strengths and weak­nesses, but how do their frame­works work? On the web, it’s the DOM and JS APIs. On Apple, it’s AppKit, Cocoa and SwiftUI. On Windows, it’s Win32, WinUI etc. On Linux, it’s GTK and Qt etc. When some­one says we need a bet­ter way of ac­cess­ing clip­board data (historical pro­to­cols are text only, what about im­ages, mul­ti­ple MIME types etc. which desk­tops have han­dled for decades), I would grab the docs for clip­board man­agers on every plat­form to see what we’ve landed on. There’s no rea­son for us to build some­thing based on our own un­der­stand­ing with­out re­search­ing decades of prior art. That’s the ap­proach I’m try­ing to take here. I’ve not in­tro­duced any cus­tom pro­to­cols yet.

Two pro­to­cols scream at me. Currently, ter­mi­nals have a main screen and an alt (sometimes called pri­mary and sec­ondary) with dif­fer­ent prop­er­ties. Main screen is like your shell with scroll­back etc. and the al­ter­nate screen is like Neovim, most TUIs etc. There are only 2, you ei­ther turn a mode on or off putting you into pri­mary or sec­ondary (taking up the whole screen, los­ing scroll­back etc.)

I’d like to in­tro­duce an n-screen API to cre­ate and pop­u­late an un­lim­ited num­ber of screens in the back­ground, let you over­lay screens with sep­a­rate grid sizes etc. The ter­mi­nal em­u­la­tor could han­dle line wrap­ping, se­lec­tion, rout­ing mouse events etc. You could spec­ify a screen as a stand­alone win­dow which the ter­mi­nal em­u­la­tor ren­ders out­side of the grid - imag­ine your Neovim tabs be­ing na­tive win­dow tabs opened at the same time! This foun­da­tional layer would solve a lot of things.

I also have a spec’d out but­ton pro­to­col. Currently, there are mouse pro­to­cols to get no­ti­fied when some­one clicks a grid cell. But you only re­ceive events for what’s cur­rently on the screen, not his­tory, when things scroll back… We cur­rently sup­port hy­per­links (OSC 8) and I’d like some­thing sim­i­lar to OSC 8 where click­ing sends a mes­sage (which you spec­ify) to the pro­gram. You could cre­ate a but­ton with an open_pro­file ID which will still reg­is­ter when the user scrolls back in his­tory. This af­fects main screen ap­pli­ca­tions (the only ones with scroll back) like Claude Code. I have no in­ter­est dis­cussing AI here, it’s just a re­ally pop­u­lar main screen ap­pli­ca­tion. The mo­ment things go into his­tory, you lose the abil­ity to open files, in-app links etc.

To what ex­tent is that just re­do­ing the en­tire user space? There’s a lot of room for scope creep there.

I ex­per­i­mented with re­plac­ing the en­tire pty pro­to­col with Wayland. If you squint, a ter­mi­nal’s just a win­dow­ing server, man­ag­ing win­dows and wid­gets on win­dows. I stud­ied Wayland to make Ghostty run bet­ter on it and thought it’s a pretty good pro­to­col for what it tries to solve (local desk­tops, ren­der­ing win­dows). But I threw that out.

A prob­lem with ter­mi­nals is that there’s no stan­dards body any more. There are old specs, de jure stan­dard­ized, but the past 20 years have seen stan­dard­iza­tion based on what the most pop­u­lar ter­mi­nals do. We have a hodge-podge of fea­tures, but no en­tity push­ing a taste­ful vi­sion.

I don’t know what the right path for­ward is. You could make an al­ter­nate home for text-based ap­pli­ca­tions, not called a ter­mi­nal any­more, build some­thing new (with a ter­mi­nal trans­la­tion layer on top to bring legacy ap­pli­ca­tions on) which is try­ing to do some­thing dif­fer­ent.

How do you bal­ance these ideas with users’ day to day de­mands?

I’m very pub­lic about open-source main­tain­ers hav­ing 0 oblig­a­tion to users. The first line in OS li­censes is as is, no war­ranty”. That’s the agree­ment, you get free soft­ware and can’t make de­mands on it. But I like striv­ing to build good soft­ware (some may dis­agree and say my soft­ware’s shitty), so I do feel an oblig­a­tion to fix prob­lems, to make the soft­ware bet­ter.

Some days I wake up ex­pect­ing to go through is­sues just fix­ing other peo­ple’s prob­lems. But some­times I wake up and fo­cus on what I want, not read­ing a sin­gle is­sue, dis­cus­sion nor PR. Sometimes, you need to push the big­ger vi­sion; some­times, you need to ad­dress the re­al­ity on the ground.

You can build a per­fect city in the sky, then come back and find ter­ror and suf­fer­ing in the real world. So you need to clean that up some­times.

If all I did was pick through user is­sues every sin­gle day, you’d get sta­ble, stag­nant soft­ware. If you ac­cept all PRs, you will change but with­out vi­sion. I don’t mean to in­sult con­trib­u­tors, but only one per­son every few years fun­da­men­tally gets it. Most con­tri­bu­tions just scratch some­one’s spe­cific itch, ac­cept­ing them all leads to a moun­tain of code. Really un­der­stand­ing, you can some­times dis­cover grace­ful sys­tems which solve every­thing suc­cinctly.

I once did fea­ture de­sign video where I closed 3 – 4 sep­a­rate fea­ture re­quests solv­ing peo­ple’s in­di­vid­ual prob­lems, be­cause a sin­gle fea­ture (different from all of them) could solve them all at the same time. Very few peo­ple can do this, not be­cause it’s dif­fi­cult but be­cause it re­quires a level of care few peo­ple give to other pro­jects.

I’ve been think­ing about this off and on for like a year (along with hun­dreds of other things, noth­ing ded­i­cated, I’m al­ways just think­ing about this stuff when not at a com­puter). When I fi­nally sat down and thought I’m go­ing to solve this,” maybe… 1 hour? https://​x.com/​mitchellh/​sta­tus/​2003957851514126510

I’ve been think­ing about this off and on for like a year (along with hun­dreds of other things, noth­ing ded­i­cated, I’m al­ways just think­ing about this stuff when not at a com­puter). When I fi­nally sat down and thought I’m go­ing to solve this,” maybe… 1 hour? https://​x.com/​mitchellh/​sta­tus/​2003957851514126510

Fewer fea­tures com­pos­ing bet­ter at­tracts peo­ple to ex­otic pro­gram­ming par­a­digms. It de­mands a lot of per­spec­tive.

I got on an in­ter­net spat with some­one re­cently about my phi­los­o­phy. One of the high­est re­quested Ghostty fea­tures is search, which is done and shipped. But some­one com­plained search bloated and broke Ghostty’s min­i­mal­ism. I ad­ver­tise Ghostty as fea­ture rich! But I do dis­tin­guish that from bloat. I don’t think you should have to pay for the things you don’t use (besides disk space or res­i­dent code mem­ory). I ex­plained the way I ar­chi­tected search means it will take up disk space and be loaded into RAM but noth­ing will ex­e­cute; this is a free fea­ture if you don’t use it. I want Ghostty to be a rice­able, cus­tomiz­able ter­mi­nal fit­ting peo­ples’ needs, but also work­ing out of the box and hid­ing them un­til you need or search for them, with­out cost­ing any­thing.

But if you re­ally want this, just fork and main­tain it your­self. That’s not ask­ing any more of you than you’re ask­ing of me. If you want me to main­tain a flag to re­move it, I can ask you to main­tain a fork re­mov­ing it. Telling peo­ple to fork it” of­ten up­sets them.

Very few peo­ple main­tain their own patches etc. and de­mand en­tire pro­jects to move to com­fort them. It’s a very dis­em­pow­er­ing mind­set to beg oth­ers to do what they could very eas­ily do them­selves, for­go­ing their own agency.

I’ve al­ways be­lieved there should be way more forks, both per­sonal and main­tained ones.

I do blame my­self and ven­ture-backed open­source in gen­eral here. There’s a whole gen­er­a­tion which ex­pects highly pol­ished, funded, opin­ion­ated pro­jects with web­sites and paid sup­port staff (in Discord, Slack etc.) be­liev­ing an open sourced pro­ject is a prod­uct - and it was a prod­uct in these ven­ture backed cases. But that’s such a minute part of the ecosys­tem.

Open source in­cludes shar­ing, but it’s about free­doms and rights. That’s the core part of open source, defin­ing OSI-approved open source li­censes. Use the soft­ware as you want, mod­ify and fork it. None of those rights are about sta­bil­ity or oblig­a­tion to main­tain. People blame main­tain­ers for ship­ping se­cu­rity vul­ner­a­bil­i­ties, but why did­n’t you re­view that com­mit? You’re just as ob­lig­ated to re­view the com­mit as the main­tainer. But peo­ple hold main­tain­ers to this higher re­spon­si­bil­ity, when they could fork and be­come a main­tainer just like that.

If you want bet­ter guar­an­tees, if you want the en­ti­tle­ment to blame some­one, pay for soft­ware. When you have a ven­dor-cus­tomer re­la­tion­ship, you are now en­ti­tled to things. But there’s no en­ti­tle­ment in open source. Use it how you love it, that’s the path to get­ting what you want. If more peo­ple forked, they’d have more em­pa­thy for builders too.

Sometimes peo­ple as­sume some pro­ject of mine is a com­pany or send a PR fix­ing a bug I don’t per­son­ally hit, and I don’t merge it be­cause merg­ing means com­mit­ting to main­tain it for­ever. In a per­sonal pro­ject, I won’t merge be­cause I don’t hit that bug.

Consuming some­one else’s not-prod­uct, what’re your thoughts on Zig’s de­vel­op­ment? Clearly you’re fine with­out the 1.0.

I knew what I signed up for. I got into Zig by do­ing com­piler patches and got to know the com­mu­nity well. I grasp the cul­ture, phi­los­o­phy etc. well, so I’m not up­set by any of this.

I ex­pect the I/O change to be one of the hard­est things we’ve done, but we haven’t started yet.

Zig’s get­ting more pop­u­lar and Andrew, the BDFL, is­n’t back­ing down from changes he feels are nec­es­sary, which I like as a down­stream con­sumer. 0.15 was pretty sig­nif­i­cant, chang­ing the writer in­ter­face and thus any­thing print­ing any­thing. But the API is truly so much bet­ter.

Zig is just get­ting bet­ter and bet­ter. They fo­cus a lot on com­pi­la­tion tool­ing and even re­moved lan­guage fea­tures to im­prove com­pi­la­tion speed, mind blow­ing. You can build lib-ghostty (the en­tire ter­mi­nal) in­stantly, and Andrew still thinks these mil­lisec­onds are too slow.

I do think Zig will even­tu­ally reach a 1.0, though I think it’s still years away. It mat­ters far less when AIs in­volved. I hope peo­ple know, read­ing this, that I’m no AI hype-mas­ter.

At a very ba­sic level, we know these neural nets are re­ally good at pat­tern match­ing and pat­tern fill­ing. For these kind of lan­guage changes, I showed how to do it in a va­ri­ety of con­texts then asked it to draw the rest of the owl. And though, the diffs were huge, 90% was done au­to­mat­i­cally while I was in the kitchen. This hints to a fu­ture where back­wards com­pat­i­bil­ity means a lot less if you ex­plain how to go from state A to B.

This is a bit ironic given Zig’s strict anti-AI pol­icy, but AI dulls the pain changes in­flict on down­stream users.

How do you ap­proach li­brary and API de­sign? Ghostty uses lib-ghostty. A friend was rav­ing about how nice your li­braries are to use. Do you have con­crete meth­ods to im­prove them be­sides just car­ing?

The most con­crete way is to use a lot of li­braries in a lot of com­mu­ni­ties. Just like learn­ing a pro­gram­ming lan­guage you won’t ac­tu­ally use (professionally), us­ing li­braries across ecosys­tems ex­pands your per­spec­tive and ben­e­fits you. In uni­ver­sity, I spent a lot of time dab­bling with es­o­teric lan­guages. I made many toy prod­ucts in Prolog, Haskell, Clojure and even Java (I was never a pro­fes­sional Java pro­gram­mer so it was new for me. I wrote a full web side pro­ject in it. I did­n’t like it but learned a lot about the build sys­tem, er­gonom­ics, li­braries, web frame­works, web servers, app servers and all that stuff.) Every ecosys­tem has a dif­fer­ent cul­ture. That cul­ture is hu­man for sure, but it bleeds into how they sep­a­rate con­cerns at a li­brary and frame­work level, how they make those APIs look. For the longest time, Java used the builder pat­tern all over the place, which I did­n’t see in any other lan­guage but I tried it in Ruby and it felt pretty good. That’s an ex­am­ple of port­ing con­cepts. This is how I ap­proach li­brary de­sign: Try to use the con­cepts I found the most en­joy­able and hope oth­ers with sim­i­lar taste also en­joy it.

I firmly be­lieve that nouns” mat­ter and the prob­lem with Docker to me is there a ton of stuff that is fo­cused on de­ploy­ment/​run­time as­pects and it med­dles with the hu­man flow. The fact that Vagrant was fo­cused di­rectly, ex­clu­sively on de­vel­op­ment was by de­sign: the con­fig­u­ra­tion, the CLI, etc. re­volved around de­vel­op­ment-fo­cused nouns, and I think that was a good thing. - https://​lob­ste.rs/​c/​ddl137

I firmly be­lieve that nouns” mat­ter and the prob­lem with Docker to me is there a ton of stuff that is fo­cused on de­ploy­ment/​run­time as­pects and it med­dles with the hu­man flow. The fact that Vagrant was fo­cused di­rectly, ex­clu­sively on de­vel­op­ment was by de­sign: the con­fig­u­ra­tion, the CLI, etc. re­volved around de­vel­op­ment-fo­cused nouns, and I think that was a good thing. - https://​lob­ste.rs/​c/​ddl137

Do you your­self make ter­mi­nal ap­pli­ca­tions dog­food­ing and test­ing these APIs and thoughts?

Not enough, hon­estly. I’ve been a tool maker my en­tire ca­reer and firmly be­lieve in the tool mak­er’s dilemma where you des­per­ately need some­thing and un­der­stand the prob­lem space well, then build an ideal tool, but oth­ers like it and you be­come an un­grounded tool maker in­stead of the tool user. I’ve had this bite me many times. From the ter­mi­nal per­spec­tive, I live in the ter­mi­nal but from a TUI de­vel­op­ment per­spec­tive, I’m not do­ing enough, but a few of our main­tain­ers are pro­lific TUI cre­ators like rock­o­r­ager who main­tains mul­ti­ple email and IRC clients and au­thored a few spec­i­fi­ca­tions. I’ve been lean­ing on him for this.

Are you happy with to­day’s tech stacks? In the past, You men­tioned nice-pol­ished OS as prod­ucts, big tech used to dis­tin­guish it­self by tech­nol­ogy too. But it seems like they’ve given up en­tirely, not dog­food­ing nor car­ing. Their open-source feels stag­nated (with ex­cep­tion of the oc­ca­sional ex­cep­tion like Jujutsu).

I’m… … … …okay with to­day’s tech stacks. They’re okay. There’s ob­vi­ously so much I wish were dif­fer­ent but I can’t get too bogged down stress­ing about things I can’t change, about bat­tles that aren’t mine to fight. For ex­am­ple, there’s a lot of good in the fron­tend, TypeScript, React type com­mu­ni­ties. But there’s also way too much churn, too much com­plex­ity, the lay­ers of ab­strac­tion aren’t clear at all. I spent time study­ing them but just don’t agree at all with how the com­mu­nity cul­tur­ally ad­dresses them. But I’m not go­ing to fight it, re­place it. So I go with the flow, stick with the main­stream to as­sist with com­mu­nity, hir­ing etc.

I’m fine with to­day’s tech stacks. I think some of the more re­cent de­vel­op­ments are overly com­pli­cated across the board. HTTP/1 vs. HTTP/2 and HTTP/3, there’s a non-lin­ear change in com­plex­ity, jus­ti­fied in var­i­ous ways but still dif­fi­cult to swal­low. You see that in fron­tend tech, ter­mi­nal tech etc. I won­der if we’re mov­ing so fast that we’re build­ing com­pli­cated stuff which could have been sim­pler. I did­n’t have time to write a short let­ter, so I wrote a long one in­stead.” This seems to be play­ing out in­dus­try-wide, ac­cel­er­ated by AI and such tool­ing.

Talking about Zig, you said you un­der­stood its val­ues etc. At Hashicorp, you wrote a prin­ci­ples doc­u­ment. How do you de­cide, cre­ate, get buy-in and use prin­ci­ples day to day? Why is Ghostty e.g. fea­ture-rich? How do prin­ci­ples con­cretely im­pact de­vel­op­ment?

Even when my co­founder and I wrote the Hashicorp prin­ci­ples or when I de­cided how Ghostty would de­velop, it was all just per­sonal, a re­flec­tion of me, so very easy for me to live those prin­ci­ples day to day. I just have to act my­self. People run into prob­lems when they make prin­ci­ples un­like them­selves. It’s that New Year’s res­o­lu­tion prob­lem where you make these grandiose in­ten­tions to­wards dra­matic changes in your life but it’s very hard, be­cause it’s hard to ac­tu­ally make those dra­matic changes.

For Ghostty, I cared about some less hu­man, more tech­ni­cal, fea­ture-rich choices like hav­ing a cross plat­form core and very much not cross-plat­form but un­apolo­get­i­cally na­tive GUI. People who val­ued them would come on board and col­lab­o­rate; those who dis­agreed just would­n’t. And that’s awe­some. I’m a re­ally big fan of open-source pro­jects and the in­ter­net be­ing a col­lec­tion of tribes.

I’m most an­noyed by pro­gram­ming lan­guages here, where so many lan­guages are be­com­ing like least com­mon de­nom­i­na­tor things with peo­ple crit­i­ciz­ing so and so for be­cause it lacks this fea­ture every other lan­guage has and is there­fore use­less. Some hy­per­bolic state­ment, you know. I re­ally like the fact that cer­tain lan­guages lack cer­tain fea­tures other lan­guages en­joy, be­cause these con­straints breed cre­ativ­ity and cul­ture. I want dif­fer­ent places to feel dif­fer­ent. I don’t need every place to feel wel­com­ing to every per­son.

People are go­ing to get mad at me for this, but you can keep it in. For ex­am­ple, for me, I don’t like the Rust cul­ture. There’s no bet­ter way to put it. Every time I’ve in­ter­acted with them or hear how they talk about Rust, I just don’t like it. That does­n’t mean they’re bad peo­ple; I think they are re­ally good peo­ple. The phi­los­o­phy be­hind the lan­guage and the lan­guage it­self is re­ally good. I just don’t want to use it and there’s no prob­lem with that. Just be­cause I don’t want to be around a com­mu­nity does­n’t mean it’s bad. I also don’t like soc­cer.

But peo­ple on the in­ter­net get stuck into such bi­nary views about good and bad, which bleeds into how tech­nolo­gies be­come this con­formist pool of bleh.

Back to Zig, Zig has a re­ally po­lar­iz­ing spe­cific stance on what it does from tech­nol­ogy, to com­mu­nity man­age­ment and fund­ing to PR, blog posts and how they talk. I don’t agree with all of it but I so re­spect that they are un­apolo­get­i­cally weird. So I con­tinue to sup­port them fi­nan­cially and use their tech­nol­ogy be­cause I sup­port peo­ple try­ing to be their own per­son.

Large com­pa­nies have gap­ing qual­ity as­sur­ance is­sues, while smaller pro­jects like Ghostty or Zig let you ap­ply taste to re­place 4 PRs with a sin­gle, holis­tic so­lu­tion (human scale). How do you rec­on­cile ship­ping qual­ity with ship­ping fast, yea AI-generated code etc.?

To ship the right thing, there has to be a big­ger un­der­stand­ing of the prod­uct you’re work­ing on. This role’s solved by dif­fer­ent peo­ple in dif­fer­ent com­pa­nies but gen­er­ally cor­po­rate America does­n’t do a great job of this at scale. You can’t just lis­ten to a spe­cific prob­lem a set of cus­tomers or users have and solve that spe­cific prob­lem. You need to un­der­stand how they got to that prob­lem in the first place, out­side of your soft­ware. What mo­ti­vated that prob­lem? Whether they should have reached this in the first place or whether some­thing up­stream would have re­solved 3 other prob­lems? You need a big­ger, holis­tic un­der­stand­ing. IDK how to solve that at the cor­po­rate scale. But I han­dle it by be­ing a big user of my own soft­ware.

I know not every­one can do this, but I’ve only worked in jobs whose prod­uct I’m a user of, so I can be a good hu­man judge of whether my work is good. If you’re too far re­moved from the cus­tomer, you shift to I com­pleted the spec” or I checked the box” lack­ing a deep enough un­der­stand­ing to say whether it’s good or bad.

I know this com­mu­nity has a pretty po­lar­iz­ing split on the AI side, but I’ve been a big pro­po­nent of ra­tio­nal AI us­age. I fully slop­pi­fied demos etc. be­cause I won’t ship it. The code is com­plete trash, but I can play and check whether some­thing’s a use­ful di­rec­tion. If it’s good, I can restart with the care it de­serves. Right now, I have a 6 week old baby and am only on the com­puter about 3 hours a day, so any time sav­ings re­ally helps. Now, I can do so much more be­cause I can ship ideas to my­self with­out be­ing on the com­puter.

You just have to ship qual­ity, read and un­der­stand the code you ship and have em­pa­thy with the users who will use this. You want to cre­ate some­thing they’re go­ing to have joy us­ing. That’s all that mat­ters.

How would you sug­gest some­one learn C to­day? Would it make sense to start di­rectly with Zig?

It’s more im­por­tant to learn how com­put­ers work and make the lan­guage just a means to un­der­stand­ing how they work. My heav­i­est us­age of C was in col­lege, re­volv­ing around file sys­tems and op­er­at­ing sys­tems in 3 classes. C was just the mech­a­nism by which we in­ter­faced closely with the lower level sys­tems in­volved. My sug­ges­tion, even in this age of higher level ab­strac­tions and web de­vel­op­ment, it’s still im­por­tant to un­der­stand the ba­sics of CPU sched­ul­ing, mem­ory, cache hi­er­ar­chies, file sys­tems, disc and file ac­cess. When you work di­rectly above the syscall layer, whether in C, Zig or Rust, it re­ally helps you un­der­stand what’s hap­pen­ing. If you go too high level, a Python, JavaScript or a Ruby’s file open API re­ally ab­stracts quite a lot from you.

Another way I learned a lot was read­ing how the higher level lan­guages are im­ple­mented. Don’t take a stan­dard li­brary func­tion for granted, some hu­man wrote that and you could too. How does it work? Read the stdlib and dig into how things work. Languages are easy; lan­guages don’t mat­ter. The un­der­ly­ing un­der­stand­ing is what mat­ters.

QuadRF can spot drones and see WiFi through my wall

www.jeffgeerling.com

The QuadRF (pictured above) a phased-ar­ray ra­dio built around a Raspberry Pi 5 and an FPGA board with pi­cosec­ond-level tim­ing. It does ad­vanced sig­nal pro­cess­ing and beam­form­ing.

It can see WiFi through walls and track drones in flight.

If the open source com­mu­nity can come up with some­thing like this, just imag­ine what gov­ern­ments are ca­pa­ble of.

When you plug a com­puter into a net­work, tools like Wireshark can show all the hid­den traf­fic you might not even know is there. WiFi pack­ets are the same, but those travel through the air, al­low­ing snoop­ing with­out phys­i­cal ac­cess.

The QuadRF has built-in soft­ware that can stream and de­code RF, and you can pipe it out to a more pow­er­ful com­puter for things like WiFi traf­fic analy­sis.

I men­tion this not to scare you—gov­ern­ments have had tools like these for years. It’s just bet­ter to know what’s pos­si­ble and ex­pose bad se­cu­rity prac­tices than to ban use­ful tools like these. So if you’re in the CIA, don’t get any ideas.

To the Moon

After spot­ting QuadRF on Hackaday, I reached out to Martin McCormick, who’s been work­ing on QuadRF as part of a big­ger pro­ject: a Moon-scale an­tenna ar­ray, ca­pa­ble of EME (Earth-Moon-Earth) ra­dio ex­per­i­ments and ra­dio as­tron­omy.

I think Martin took in­spi­ra­tion from Dishy, SpaceX’s orig­i­nal Starlink ter­mi­nal. (Makes sense, since Martin worked at SpaceX on the team that built Dishy!)

Instead of lock­ing this phased ar­ray an­tenna sys­tem into a pro­pri­etary satel­lite sys­tem, li­censed op­er­a­tors will ide­ally be able to chain mul­ti­ple QuadRF mod­ules to­gether for in­ter­est­ing ra­dio ex­per­i­ments, with up to 1.15 MW EIRP—basically, a mas­sive amount of di­rec­tional an­tenna gain, for high power RF fun.

But QuadRF is scaled down to hand­held-size, and while it is­n’t pow­er­ful enough to send a sig­nal to the moon, it’s still quite use­ful in lo­cal SDR ap­pli­ca­tions and vi­su­al­iz­ing the RF en­vi­ron­ment—at least in its fre­quency range of 4.9 – 6 GHz.

Testing QuadRF

But I specif­i­cally asked Martin if he’d be will­ing to send over a pro­to­type QuadRF for my Dad (a re­tired broad­cast ra­dio en­gi­neer) and I to test.

I had al­ready placed a pre-or­der on Crowd Supply (where a ba­sic kit is $499), but I wanted to see if QuadRF was re­ally as use­ful or in­tu­itive as it seemed from the videos ScaleRF posted.

Spoilers: it’s still a lit­tle rough in the UI de­part­ment, but I was blown away by how well it works. Especially con­sid­er­ing every­thing’s run­ning on a Raspberry Pi 5.

When you turn it on, the Pi boots up and cre­ates a WiFi hotspot. You con­nect to that, and visit http://​quadrf/. That page runs a VNC ses­sion in your browser, where you can launch apps from GNU Radio to SDR soft­ware, and even their cus­tom AR (Augmented Reality) RF vi­su­al­izer.

The AR vi­su­al­izer is the most in­ter­est­ing in­cluded soft­ware, de­spite be­ing less use­ful for real-world SDR ap­pli­ca­tions.

The UI is a lit­tle rough, but you can ad­just the align­ment be­tween your cam­era and the phased ar­ray, and the gain of the re­ceiver.

Then it will vi­su­al­ize fre­quen­cies from 4.9 – 6 GHz as col­or­ful blobs’. The scale is not shown on the dis­play in this early ver­sion, but from my test­ing around the stu­dio, my 5 GHz WiFi net­work (which was run­ning on Channel 100, or around 5.5 GHz) showed up light blue. Neighboring WiFi net­works were show­ing up red or green.

If you or­der the Mobile Expansion Pack, it in­cor­po­rates a bat­tery power pack, and a hand­held phone mount, so you can walk around an­a­lyz­ing part of the C-band in real-time.

My Dad and I flew his DJI Mini Pro 4 be­hind the stu­dio, and the QuadRF had no trou­ble pick­ing it out of the sky. As it flew away, I had to in­crease the gain to keep see­ing it; it would be nice to have AGC or an eas­ier gain con­trol as the UI was a lit­tle clunky when car­ry­ing around the con­trap­tion.

It sounds like the crowd­fund­ing cam­paign is al­ready be­yond ex­pec­ta­tions, and they’ll be switch­ing the en­clo­sure to an in­jec­tion mold (the ver­sion I have is 3D printed).

Raspberry Pi 5 MIPI for high-band­width RF

One as­pect that in­trigued me was the use of the Raspberry Pi’s MIPI lanes for low la­tency SDR stream­ing I/Q (In-phase/Quadrature) at data rates over 5 Gbps. From the QuadRF Documentation:

The novel ap­proach of stream­ing I/Q over the Pi’s cam­era and dis­play FFC MIPI con­nec­tors has many ben­e­fits. MIPI can han­dle >5 Gbps, low-la­tency, full-du­plex data trans­fer through the Pi’s RP1 chip. It is sim­pler and more re­li­able than USB, adds al­most zero hard­ware cost to the RF board, and can sus­tain hun­dreds of MSPS of I/Q with no hic­cups or sam­ple loss. Considering cam­eras and dis­plays are the ul­ti­mate form of high-band­width sig­nal stream­ing, it makes sense their stan­dard dig­i­tal in­ter­face is a great match for SDR! We think the in­dus­try should adopt it more widely!

The novel ap­proach of stream­ing I/Q over the Pi’s cam­era and dis­play FFC MIPI con­nec­tors has many ben­e­fits. MIPI can han­dle >5 Gbps, low-la­tency, full-du­plex data trans­fer through the Pi’s RP1 chip. It is sim­pler and more re­li­able than USB, adds al­most zero hard­ware cost to the RF board, and can sus­tain hun­dreds of MSPS of I/Q with no hic­cups or sam­ple loss. Considering cam­eras and dis­plays are the ul­ti­mate form of high-band­width sig­nal stream­ing, it makes sense their stan­dard dig­i­tal in­ter­face is a great match for SDR! We think the in­dus­try should adopt it more widely!

It sounds like they had to re­verse-en­gi­neer the MIPI pro­to­col used on the Pi 5 to do this (since it goes through the RP1 chip), and the way it’s ar­chi­tected, you can daisy-chain mul­ti­ple QuadRF mod­ules to­gether, let­ting each mod­ule cal­cu­late its own phase shift.

I’m not sure how that will work in prac­tice, but it sounds pretty neat. PCIe could prob­a­bly work in a pinch, too, but this im­ple­men­ta­tion frees up the PCIe con­nec­tor in case you want high speed stor­age or even higher speed net­work­ing than the Pi of­fers.

Conclusion

As with all pre-pro­duc­tion gear I test, take every­thing I’ve shown with a grain of salt. And with any crowd­fund­ing cam­paign, if you back it, don’t ex­pect the QuadRF to show up on your doorstep overnight.

I was ini­tially skep­ti­cal about how use­ful and fun this lit­tle hand­held phased ar­ray could be, but af­ter us­ing it for a week, I can’t wait un­til the one I pre-or­dered ships!

Zuck saves Meta bucks by reusing memory from old servers with a custom CXL ASIC

www.theregister.com

REG AD

SYSTEMS

In pro­duc­tion on mil­lions of boxes and the pay­off is a 25% re­duc­tion in ma­chines needed for some in­fer­ence work­loads

Meta is re­cov­er­ing DDR4 mem­ory from old servers, in­stalling it in new ma­chines, and us­ing a cus­tom Compute Express Link (CXL) ASIC to share the mem­ory across ap­pli­ca­tions — with­out en­coun­ter­ing la­tency prob­lems.

The so­cial net­work­ing gi­ant calls its tech Vistara” and will pre­sent it at ISCA 2026 on Monday, but The Register found the com­pa­ny’s pa­per ahead of the talk. Our sis­ter site, Blocks and Files, also hap­pens to have re­ported on this on Friday.

The doc­u­ment opens with the ad­mis­sion that Meta can’t in­crease the amount of mem­ory in around 40 per­cent of its vast server fleet, mean­ing mil­lions of servers can’t han­dle some of its work­loads. That’s un­for­tu­nate be­cause the ex­pected ser­vice life of its servers is three to five years, but mem­ory is use­ful for seven to ten years.

REG AD

Meta’s re­sponse is to rip DDR4 DIMMs from old servers, put them into new ma­chines that rely on DDR5, and turn it all into a pool of ca­pac­ity — which in the­ory makes it pos­si­ble to com­pose vir­tual servers that share re­sources across mul­ti­ple phys­i­cal hosts.

REG AD

The pa­per points out that CXL is hard to put into pro­duc­tion be­cause shar­ing mem­ory across hosts can mean low band­width, high la­tency, and ex­tra com­put­ing over­heads to man­age ad­di­tional mem­ory lay­ers. Those prob­lems can arise in sys­tems that com­bine dif­fer­ent mem­ory tech­nolo­gies. Meta wanted to blend mem­ory types in a sin­gle ma­chine but found off-the-shelf CXL kit can’t do the job.

Most CXL so­lu­tions bun­dle DRAM with the con­troller — pre­vent­ing DIMM reuse — and of­ten omit DDR4 sup­port, which is a re­quire­ment for re­pur­pos­ing older mem­ory,” the pa­per states. Additionally, their high power con­sump­tion and high cost fur­ther limit their ap­peal.”

To make CXL sing, Meta cre­ated a cus­tom ASIC called Vistara.”

At its core, the Vistara ASIC is de­signed to bridge DDR4 mem­ory to host proces­sors via a CXL 2.0/1.1-compliant PCIe Gen5 x16 in­ter­face,” the pa­per ex­plains. Each Vistara ASIC in­te­grates two in­de­pen­dent 72-bit DDR4 mem­ory chan­nels, sup­port­ing speeds up to 3,200 MT/s and up to 256 GB per chip with 64 GB DIMMs.”

A pair of cus­tom RISC-V proces­sors drive the ASICs.

Vistara hard­ware lives in de­vices Meta calls a MemServer” pow­ered by an AMD Turin proces­sor pack­ing 158 cores and run­ning 316 threads. Each MemServer com­bines 768 GB of DDR5 mem­ory along­side 256 GB of DDR4 con­nected through Vistara ASICs.

The Vistara CXL cards are in­stalled in ded­i­cated rear-ac­ces­si­ble slots within each MemServer chas­sis,” the pa­per re­veals. To man­age the in­creased ther­mal load from high-den­sity mem­ory and CXL de­vices, the chas­sis em­ploys di­rected air­flow with high-ca­pac­ity fans that chan­nel cool air di­rectly across the Vistara mod­ules, for sta­ble op­er­a­tion un­der heavy work­loads.”

The soft­ware side of Vistara sees the DDR4 pre­sented to the OS as a dis­tinct, CPU-less NUMA node, sep­a­rate from the lo­cal DRAM nodes di­rectly at­tached to the proces­sor.” Meta’s plat­forms first use all avail­able lo­cal DDR4, then em­ploy the CXL-enabled mem­ory when needed.

REG AD

Zuck’s house of hy­per­scale hyp­no­tism makes this hap­pen with cus­tom tweaks to the Linux CXL dri­ver. All Linux ker­nel CXL dri­ver code in use for Vistara is ei­ther pre­sent in the up­stream ker­nel, or is on its way to be­ing in­cluded in the up­stream ker­nel,” the pa­per states.

The pa­per says Meta has put this CXL stuff to work in hy­per­scale in­fra­struc­ture with mil­lions of servers, across a va­ri­ety of pro­duc­tion work­loads, in­clud­ing dis­ag­gre­gated ML in­fer­ence (embedding ta­bles in rec­om­men­da­tion sys­tems), big data pro­cess­ing, data­bases, dis­trib­uted caches, and CI/CD build sys­tems.”

Some work­loads, in­clud­ing big data tools such as Spark and Hive, use ter­abyte and petabyte-scale datasets, and need hun­dreds of gi­ga­bytes of mem­ory per job. The pa­per says that if those work­loads ex­pe­ri­ence out-of-mem­ory events, it can disrupt crit­i­cal busi­ness an­a­lyt­ics and ML pipelines.”

The ex­panded mem­ory head­room pro­vided by CXL en­hances sys­tem re­li­a­bil­ity,” the pa­per ex­plains. By mit­i­gat­ing the risk of out-of-mem­ory (OOM) events, CXL re­duces the fre­quency of job fail­ures and the as­so­ci­ated over­head of job restarts and re­source frag­men­ta­tion by 33 per­cent.”

Meta says the sys­tem also cuts in­fra­struc­ture costs. These de­ploy­ments have demon­strated large ben­e­fits, such as re­duc­ing the server count by up to 25 per­cent for dis­ag­gre­gated in­fer­ence,” the pa­per states. And of course Meta is avoid­ing the sky-high mem­ory prices caused by the RAMpocalypse. ®

Write code like a human will maintain it

unstack.io

One of the best things about LLMs is that they’ll write code for you, all day long. Who cares about DRY? You don’t have to be the one up­dat­ing the same long con­di­tional in four dif­fer­ent files - the AI will just do it for you! Right?

I’ve no­ticed my­self let­ting it slide re­cently on a pro­ject I’d been build­ing with AI. I needed the same ac­cess check in a hand­ful of places: a route han­dler, a back­ground job, an API end­point, a web­hook, etc. Each time, I’d de­scribe what I needed, the model would gen­er­ate some­thing that worked, and I’d merge it.

Each ver­sion looked roughly like this:

if (user.isActive && user.hasPer­mis­sion(‘read’) && !user.isSuspended && ac­count.sta­tus === open’) { // do a thing }

Essentially the same con­di­tion­als every time. Four con­di­tions, maybe slightly dif­fer­ent vari­able names, copy-pasted logic with a word or two changed. There’s a much cleaner way to do this - a shared helper, for ex­am­ple, like some­thing I’d ex­tract if I were writ­ing this my­self. But I did­n’t. The code worked! The tests passed and I was­n’t the one who’d have to touch it again.

That’s the lazi­ness here: if it does­n’t fol­low best prac­tices, or I know a piece of code will be a pain to main­tain, what dif­fer­ence does it make? When I need to change some­thing later, the LLM deals with it, not me.

Except the LLM does­n’t write in a vac­uum. It reads your code­base. The files you have open, the pat­terns that are al­ready there, and the re­cent changes you’ve made. Every short­cut you merge into your code­base is a sig­nal about how things are done here. The next time you ask the LLM for an­other end­point with the same ac­cess rules, the model won’t start from first prin­ci­ples. It’ll start from the other four copies al­ready sit­ting in your repo.

So you ask for a fifth end­point, and you get a fifth con­di­tional, with the same copied code. You ask for a refac­tor, and the model pre­serves all five, be­cause that’s what your code looks like. The bad pat­tern is­n’t a one-off any­more, it’s con­sid­ered to be your style.

If you let things go on like this, can you re­ally trust that the LLM will catch every in­stance if you try to fix it later?

Sure, a few of these aren’t cat­a­strophic. That’s how it al­ways starts, but code smells do stack up. Each du­pli­cated con­di­tional, each god” func­tion, each I’ll clean this up later” merge adds an­other layer of sig­nal­ing to the next prompt. Eventually you can’t eas­ily prompt your way out of it. At least not with­out get­ting your hands dirty and rolling up your sleeves.

The most frus­trat­ing part: I thought I was out­sourc­ing main­te­nance to the LLM, but the slip­pery slope I found my­self on was ac­tu­ally train­ing it to have ever-wors­en­ing habits.

Write code like a hu­man will main­tain it. LLMs are sponges that soak up every­thing you do and re­peat it back to you. So make sure it’s good.

Why American ambulance rides are so expensive

davidoks.blog

In July 2023, a 25-year-old man named Jagdish Whitten was out for a run in San Francisco. As he crossed a busy street, a car hit him; he did, in his words, a lit­tle flip” over the ve­hi­cle, landed in the road, and dragged him­self to the curb. Those who had seen the ac­ci­dent called an am­bu­lance for him. But Whitten waved them off and called a friend, who drove him to a nearby hos­pi­tal in­stead: I knew that am­bu­lances were ex­pen­sive,” he said, and I did­n’t think I was go­ing to die.”

Whitten was right on both counts. At the hos­pi­tal, doc­tors found that he had a mild con­cus­sion, a bro­ken toe, and a few bruises—noth­ing too se­ri­ous. But be­cause he’d suf­fered a trau­matic in­jury, they were ob­lig­ated to send him to San Francisco General, the city’s only des­ig­nated trauma cen­ter. This time he did­n’t have a choice. He was loaded into an am­bu­lance for a six-mile trans­fer, eval­u­ated with­out ad­di­tional treat­ment, and sent home the same night.

Over the weeks that fol­lowed, Whitten got bills from both hos­pi­tals. Everything was roughly what he ex­pected, and all of it would be cov­ered by his in­sur­ance plan. But a few months later, Whitten got an­other bill—this time from American Medical Response, the am­bu­lance provider that had trans­ferred him be­tween hos­pi­tals. The am­bu­lance ride, he learned, would cost him $12,873: $737 for the miles trav­eled, $314 for mon­i­tor­ing his heart on the trip, $151 for in­fec­tion con­trol, and $11,670 as a base rate.”

He sent the bill to his in­sur­ance provider, which at first de­nied the claim, say­ing that AMR was out of net­work and that the ride had­n’t been pre-au­tho­rized. (Whitten, of course, had­n’t cho­sen the am­bu­lance, or any­thing else about the trip.) On ap­peal, his in­sur­ance agreed to cover $9,967 of the charge—bet­ter than noth­ing, but it still left him on the hook for about $3,000. After sev­eral failed at­tempts to con­test the bill with AMR, and not want­ing it sent to col­lec­tions to hurt his credit score, he paid the re­main­ing $2,900 or so. The brief am­bu­lance ride from one hos­pi­tal to an­other had cost him far more than any other part of the ex­pe­ri­ence.

What Whitten had re­ceived was a surprise bill”—a charge that lands on a pa­tient when they’re treated, with­out knowl­edge or con­sent, by a provider out­side their in­sur­er’s net­work. The in­surer pays what they con­sider rea­son­able; the provider bills the pa­tient for the dif­fer­ence; and the pa­tient, de­spite hav­ing in­sur­ance that’s meant to pay for treat­ment, is left hold­ing the bal­ance. This is a ter­ri­ble sit­u­a­tion to be in.

It’s also the de­fault way that am­bu­lance billing in the United States works. Each year, roughly three mil­lion pri­vately in­sured Americans take an emer­gency am­bu­lance ride; about half of them get an out-of-net­work bill for it, a rate un­matched any­where else in med­i­cine. And the unin­sured have it worse still: with no in­surer to ab­sorb any of the charge, they face the full, undis­counted bill on their own.

This has proven quite dif­fi­cult to fix. When Congress banned sur­prise billing across vir­tu­ally the en­tire health­care sys­tem in 2020, ground am­bu­lances were the great ex­cep­tion. If you’re pri­vately in­sured and need an emer­gency am­bu­lance, you’re en­ter­ing a lot­tery whose ticket price you’ll learn weeks or months later.

And that’s why so many Americans, like Whitten, avoid am­bu­lances when­ever they can. One poll from 2024 found that 23 per­cent of Americans have for­gone an am­bu­lance ride be­cause of con­cerns about cost.

So why are American am­bu­lances so ex­pen­sive?

The stan­dard an­swer is greed: ra­pa­cious am­bu­lance op­er­a­tors, owned by vil­lain­ous pri­vate eq­uity firms, ex­ploit pa­tients at their most help­less. But I don’t think that’s ac­tu­ally what’s go­ing on. Ambulance providers are chron­i­cally un­prof­itable busi­nesses; mar­gins are thin, crews are un­der­paid, and op­er­a­tors exit the in­dus­try every year. Whatever is be­ing ex­tracted from pa­tients like Whitten, it is­n’t padding any­one’s pock­ets.

The real prob­lem is much more spe­cific and much more in­ter­est­ing. American am­bu­lance bills are enor­mous and un­pre­dictable be­cause of how American law forces am­bu­lance ser­vices to work. In 1965, al­most as an af­ter­thought, Medicare de­cided to pay for am­bu­lances the way it paid for every­thing else: as a per-ride fee, af­ter the fact, as though a trip to the hos­pi­tal were just an­other med­ical pro­ce­dure. Commercial in­sur­ers, who build their pay­ment sys­tems on top of Medicare’s fee sched­ules, fol­lowed suit.

In the decades since then, the cost struc­ture of am­bu­lance ser­vices has changed enor­mously, and very lit­tle of their cost now comes from the ride it­self: nearly all of it goes to stand­ing ready—the sta­tions, the ve­hi­cles, the crews wait­ing around the clock for a call that may never come. But the way we pay for am­bu­lance rides has stayed the same. And nearly every­thing strange and cruel about am­bu­lance billing fol­lows from that dis­junc­tion.

To un­der­stand what went wrong, we need to learn a lit­tle bit about the eco­nomic logic of am­bu­lances, and why am­bu­lance ser­vices are more like op­tion sell­ers than taxi com­pa­nies. It’s more in­ter­est­ing than it sounds.

If you want to think in crude fi­nan­cial terms—not a ter­ri­ble way to go through life, all things con­sid­ered—you can think of an am­bu­lance ser­vice as an op­tion on res­cue.

What does that mean? An option,” in fi­nance, is sim­ply a con­tract that gives its holder the right, but not the oblig­a­tion, to make some trans­ac­tion in the fu­ture—to buy a share at a fixed price at some point in the next three months, for ex­am­ple. And when I write an op­tion—sell it, in the jar­gon—I take on the mir­ror-im­age oblig­a­tion: if the holder ever chooses to ex­er­cise it, I have to de­liver, whether it suits me or not. Options, in this broad sense, are all around us. Insurance, for ex­am­ple, is one type of op­tion: when I buy a fire pol­icy, I pay a small pre­mium every year for the right to a large pay­out if my house ever burns down, and the in­surer takes on the oblig­a­tion to pay it whether or not the tim­ing suits them. So are fire de­part­ments: they keep en­gines and fire­fight­ers ready at all times so that any mo­ment peo­ple can call them for help—a readi­ness that res­i­dents pay for with their taxes.

And am­bu­lance ser­vices work a lot like op­tion sell­ers as well.

Let’s say I live in San Francisco. By dint of liv­ing there, I hold a per­pet­ual right to sum­mon an am­bu­lance from the San Francisco Fire Department: when­ever I call, they’re ob­lig­ated to come and res­cue me. I haven’t en­tered into an ex­plicit con­tract with them, but the arrange­ment func­tions the same way re­gard­less. The SFFD, in other words, is sell­ing me—and every other res­i­dent of San Francisco—an op­tion on be­ing res­cued in an emer­gency. If I ever need an am­bu­lance, I can ex­er­cise that op­tion. But even if I never do, sim­ply hav­ing it is valu­able to me: know­ing that help would come if I needed it al­lows me to take risks I would­n’t take oth­er­wise. In that sense, I’m con­sum­ing the op­tion of res­cue every day of my life.

Of course, of­fer­ing me that op­tion is­n’t free for the SFFD. Selling op­tions al­ways im­poses costs on the seller: the SFFD has to spend money, around the clock, to en­sure that if I ever col­lapse on the side­walk and some­one calls 911, it’s pre­pared to ar­rive rapidly and re­sus­ci­tate me.

And for the whole sys­tem to work, they need to be com­pen­sated for that cost. The prod­uct is the readi­ness, not the ride; and the readi­ness has to be funded whether or not the ride ever hap­pens.

An am­bu­lance ser­vice, then, should work the same way. The SFFD should col­lect a small pre­mium from every house­hold that en­joys the guar­an­tee of res­cue, and use the pro­ceeds to pay for every­thing it needs to be ready to pick peo­ple up. And be­cause every­one would pay, no one would pay very much.

But—for strange and par­tic­u­lar rea­sons of American his­tory and law—that’s not how the American am­bu­lance sys­tem works.

As long as there have been hos­pi­tals, there’s been an oc­ca­sional need to get peo­ple to them rapidly. For most of his­tory, there was no spe­cial­ized in­fra­struc­ture for this. The badly sick and in­jured might travel by cart or horse to the near­est doc­tor, and a great many of them sim­ply died on the way; or they might just give up and die in place.

This be­gan to change in the nine­teenth cen­tury, with in­no­va­tions in bat­tle­field med­i­cine that even­tu­ally mi­grated into civil­ian life. And by the early twen­ti­eth cen­tury, with the ar­rival of the au­to­mo­bile, the am­bu­lance as we know it to­day be­gan to take shape: a mo­tor ve­hi­cle ded­i­cated to bring­ing peo­ple to the hos­pi­tal as quickly as pos­si­ble.

But the am­bu­lance could­n’t just be any mo­tor ve­hi­cle. Patients gen­er­ally needed to be trans­ported ly­ing flat, since sit­ting a badly in­jured per­son up­right wors­ens shock and blood loss: so the car needed a long, low, flat cargo bed. The only car that had one, in most places, was the fu­neral hearse: the car that car­ried coffins to the ceme­tery.

And so for most of the twen­ti­eth cen­tury, am­bu­lance ser­vices were pro­vided by fu­neral homes. They would use their hearses as combination cars,” able to trans­port both liv­ing pa­tients and dead bod­ies. They were open at every hour of the day any­way—since peo­ple can die at any time—and you could call at any hour of the night and they’d send some­one out with the hearse to pick you up. It was com­mon, in fact, for the same car to make both kinds of trip in a sin­gle day, per­haps even for the same per­son.

As you can prob­a­bly guess, the am­bu­lance ser­vices these fu­neral homes pro­vided weren’t very good. Their equip­ment was crude—a stretcher, a blan­ket, per­haps an oxy­gen bot­tle—and the at­ten­dant sent out on a call was sim­ply whichever em­ployee was free, with no pre­tense of med­ical train­ing. Death rates, un­sur­pris­ingly, were ex­tremely high.

But death was­n’t a huge prob­lem for the fu­neral homes, since they pro­vided am­bu­lance runs as a loss leader for the high-mar­gin busi­ness of fu­neral ser­vices. The real prize was the re­la­tion­ship: the fam­ily that called you for the ride to the hos­pi­tal was likely to also call you for the fu­neral. The runs weren’t very ex­pen­sive to pro­vide, ba­si­cally amount­ing to an em­ployee dri­ving the hearse some­where nearby. So the fu­neral homes might charge the pa­tient a nom­i­nal fee and not try very hard to col­lect it, or they might not charge any­thing at all.

In other words: the cost of writ­ing the op­tion was low, be­cause am­bu­lance ser­vices were sim­ple and cheap.

And in the 1960s, the logic of that pe­cu­liar arrange­ment was writ­ten into fed­eral law. In 1965, the United States gov­ern­ment cre­ated two mas­sive pro­grams of so­cial in­sur­ance: Medicare, which pro­vided pub­lic health in­sur­ance for Americans over 65, and Medicaid, which did the same for the poor. Both pro­grams worked by enu­mer­at­ing the med­ical ser­vices they would pay for, and re­im­burs­ing each per unit of ser­vice ren­dered: a hos­pi­tal that per­formed an ap­pen­dec­tomy, say, billed the gov­ern­ment for one ap­pen­dec­tomy. And among those ser­vices, added al­most as an af­ter­thought, was am­bu­lance trans­porta­tion. Trips would be cov­ered by both pro­grams, with a fee paid to the op­er­a­tor for each trip, af­ter the fact.

At the time it was a per­fectly rea­son­able choice: it made sense, given how cheap am­bu­lance rides were at the time, to treat them as just an­other pro­ce­dure to be billed. But that in­nocu­ous clas­si­fi­ca­tion turned out to warp the en­tire sys­tem of emer­gency care in the United States.

In the 1960s, at the very mo­ment that American law cod­i­fied how emer­gency care would be paid for, the na­ture of emer­gency care be­gan to change.

The trans­for­ma­tion had started around 1960, when CPR—cardiopulmonary re­sus­ci­ta­tion—was de­vel­oped at Johns Hopkins: rhyth­mic com­pres­sion of the chest, it was found, could keep blood cir­cu­lat­ing through a stopped heart. In 1965 came the portable de­fib­ril­la­tor, which dra­mat­i­cally im­proved chances of sur­viv­ing a heart at­tack out­side the hos­pi­tal; and around the same time came ra­dio teleme­try, which let a crew in the field trans­mit a pa­tien­t’s vi­tal signs to a physi­cian miles away. With all these in­no­va­tions, the chance of sur­viv­ing a car­diac ar­rest or a bad ac­ci­dent in­creased dra­mat­i­cally.

Within the space of a few years, then, it be­came pos­si­ble to reimag­ine emer­gency care. The fu­neral homes’ com­bi­na­tion car sim­ply fer­ried peo­ple to the hos­pi­tal; but with these new tech­niques, you could imag­ine some­thing like real med­i­cine be­ing prac­ticed on the way there.

Against that back­drop, the sys­tem of fu­neral home am­bu­lances be­gan to look badly in­ad­e­quate. In 1966, the National Academy of Sciences pub­lished a re­port called Accidental Death and Disability: The Neglected Disease of Modern Society, which found that American am­bu­lance sys­tems were badly un­trained and un­equipped for emer­gency care. A sol­dier gravely wounded in Vietnam, the re­port found, had a bet­ter chance of sur­vival than a mo­torist gravely in­jured on an av­er­age city street.

With a surge of pub­lic in­ter­est and tech­no­log­i­cal pos­si­bil­ity, from the mid-1960s on­ward there was a rev­o­lu­tion in American emer­gency med­i­cine. Soon there was talk of EMS, emergency med­ical ser­vices,” and of paramedics,” peo­ple who weren’t quite doc­tors but could nonethe­less prac­tice med­i­cine on the way to the hos­pi­tal. (One im­por­tant part of this cul­tural shift was the med­ical drama Emergency!, which ran for six sea­sons on NBC and proved im­por­tant in pop­u­lar­iz­ing EMS.) The Emergency Medical Services Systems Act of 1973 fi­nanced some three hun­dred re­gional EMS sys­tems, with trained para­medics and stan­dards for equip­ment and dis­patch; and count­less more EMS sys­tems sprouted up in the years that fol­lowed, with or with­out fed­eral sup­port. In the early 1960s, there had been es­sen­tially no cer­ti­fied emer­gency med­ical tech­ni­cians in the United States; by the early 1980s, there were hun­dreds of thou­sands.

The re­sult was a trans­for­ma­tion of the American emer­gency care sys­tem.

The fu­neral homes, fac­ing new stan­dards and no way to profit from meet­ing them, fled the in­dus­try; and for a time the gap was filled by vol­un­teer squads, small pri­vate op­er­a­tors, and pub­lic pro­grams like Seattle’s much-ad­mired Medic One. In the 1980s, fire de­part­ments be­gan to en­ter the busi­ness. They had fewer and fewer fires to fight, thanks to bet­ter build­ing codes and the dif­fu­sion of smoke de­tec­tors; and they needed to jus­tify their gen­er­ous mu­nic­i­pal bud­gets. Soon, many fire de­part­ments were EMS agen­cies in every­thing but name: by 2020, 64 per­cent of American fire de­part­ment runs were EMS-related, and only 4 per­cent had any­thing to do with fires.

From the 1960s on­ward, in short, a pro­fes­sional and cap­i­tal-in­ten­sive in­fra­struc­ture emerged around emer­gency care. And as the char­ac­ter of the am­bu­lance in­dus­try was trans­formed, its eco­nom­ics were trans­formed as well.

In the days of fu­neral home am­bu­lance ser­vices, EMS was a busi­ness with low fixed costs: in­deed that’s the whole rea­son why fu­neral homes were will­ing to pro­vide it as a loss leader for fu­ner­als. The fu­neral home was al­ready pay­ing for the staff and the hearse; it did­n’t cost much to send them out to pick peo­ple up and bring them to the hos­pi­tal.

But now EMS was an in­dus­try with high fixed costs. There were trained pro­fes­sion­als, who had to be paid; there was an am­bu­lance, filled with ex­pen­sive equip­ment; and there was the sta­tion. The cost of dis­patch­ing an am­bu­lance on a run was triv­ial. But the cost of keep­ing that am­bu­lance ready to be dis­patched was huge.

And those high fixed costs were made more se­vere by the na­ture of the prod­uct. In a nor­mal in­dus­try with high fixed costs, like an air­line or a taxi fleet, you com­pen­sate by max­i­miz­ing uti­liza­tion: you keep the ex­pen­sive as­sets work­ing as many hours as pos­si­ble, so that the fixed costs are spread over as much rev­enue as pos­si­ble. But be­cause am­bu­lance ser­vices are sell­ing an op­tion they can never refuse to honor, they’re re­quired to hold idle ca­pac­ity against their calls. A well-run emer­gency ser­vice will tar­get uti­liza­tion of about 30 to 50 per­cent; any­thing higher risks drop­ping a call, par­tic­u­larly if there’s a surge in de­mand.

From 1970 on­ward, then, the fixed cost of run­ning an am­bu­lance ser­vice soared. But the way peo­ple paid for am­bu­lances did­n’t change at all. Medicare had es­tab­lished the per-ride tem­plate in 1965, and com­mer­cial in­sur­ers uni­ver­sally fol­lowed it. This meant that the pay­ment struc­ture and the cost struc­ture were in­creas­ingly mis­matched: and so am­bu­lance ser­vices had to pay for their round-the-clock readi­ness by billing for in­di­vid­ual rides.

From the per­spec­tive of am­bu­lance ser­vices as op­tion sell­ers, this makes no sense. It’s as if a fire in­surer gave its poli­cies away for free, and then billed the whole cost of the fire­house to whichever un­lucky cus­tomer’s house hap­pened to burn down. It’s the ex­act op­po­site of how in­sur­ance is sup­posed to work: but it’s what the pay­ment struc­ture of Medicare made nec­es­sary. The en­tire cost of writ­ing the op­tion had to be re­cov­ered from the un­lucky mi­nor­ity of peo­ple who ac­tu­ally ex­er­cised it.

Or, rather, some of the peo­ple who ex­er­cised it: by de­sign, the most fre­quent users, in fact, were ex­empt. The heav­i­est users of am­bu­lances are the el­derly, who are cov­ered by Medicare; and since Medicare is the largest pur­chaser of med­ical ser­vices in the coun­try and is backed by the force and au­thor­ity of the U.S. gov­ern­ment, it en­joys the rare priv­i­lege of set­ting its own prices. And since 2002, Medicare has im­posed a na­tional fee sched­ule on am­bu­lance ser­vices—a fixed max­i­mum pay­ment for each cat­e­gory of ride.

And no­tably, the fees that Medicare sets run far be­low cost. The av­er­age am­bu­lance trans­port costs $2,673 to pro­vide; Medicare pays only about $329 of that. A typ­i­cal am­bu­lance ride for a Medicare pa­tient, in other words, loses the am­bu­lance ser­vice thou­sands of dol­lars. With Medicaid it’s even worse: state pro­grams typ­i­cally pay a frac­tion of what Medicare does, and the ser­vices lose even more per ride. Normally, when an in­surer pays less than the provider charges, the provider bills the pa­tient for the bal­ance, a prac­tice known as balance billing”; but with Medicare and Medicaid, bal­ance billing is il­le­gal.

On most of their book, then—rides for Medicare and Medicaid ben­e­fi­cia­ries—EMS providers lose money on every run. And that’s be­fore ac­count­ing for the fixed costs that make up the large ma­jor­ity of their ex­penses.

That leaves two groups from whom am­bu­lance ser­vices can try to re­cover the cost of their op­er­a­tions: the unin­sured and the pri­vately in­sured.

Let’s start with the unin­sured. On pa­per, they should be the most lu­cra­tive pa­tients an am­bu­lance ser­vice could ask for: they re­ceive the full, undis­counted charge, with no in­surer to ne­go­ti­ate it down. But unin­sured pa­tients are dis­pro­por­tion­ately un­likely to pay, since they’re dis­pro­por­tion­ately poor; and EMS providers typ­i­cally end up sell­ing most of that debt to col­lec­tions firms for pen­nies on the dol­lar. So in prac­tice, the unin­sured look less like a profit cen­ter and more like the Medicare and Medicaid pa­tients: an­other class of pa­tients that pays be­low cost.

And that leaves the one group who ac­tu­ally can be made to pay: the pri­vately in­sured. All the costs of the EMS sys­tem—the fixed costs of round-the-clock readi­ness, plus the losses on every Medicare, Medicaid, and unin­sured ride—must be ex­tracted from them for the ser­vice to come close to break­ing even.

But here there’s an­other prob­lem.

In or­di­nary med­ical care, when an in­surer wants to bring a hos­pi­tal into its net­work, it makes a deal: ac­cept our dis­counted rates, and we will steer our mem­bers to you. But even though Medicare treats them like one, am­bu­lance rides aren’t like other med­ical pro­ce­dures. The in­surer does­n’t steer any­one to the am­bu­lance: the am­bu­lance goes wher­ever the emer­gency calls come from. So the in­surer has noth­ing to of­fer the EMS provider: a net­work con­tract is sim­ply a pay cut in ex­change for noth­ing. The ra­tio­nal re­sponse, then, is sim­ply to de­cline, and charge in­sur­ance com­pa­nies the full amount. And so the vast ma­jor­ity of am­bu­lances in the U.S. don’t take in­sur­ance, and about 80 per­cent of ground am­bu­lance rides in the U.S. are billed to in­sur­ance com­pa­nies as out-of-net­work charges.

The re­sult is what you’d ex­pect. Ambulances set their own charges, to de­fray the full cost of their readi­ness; in­sur­ers pay what­ever frac­tion of them they con­sider rea­son­able; and the pa­tient gets a sur­prise bill for the dif­fer­ence—which is ex­actly how Jagdish Whitten had to pay $2,900 for a six-mile am­bu­lance ride.

So the American am­bu­lance in­dus­try is the vic­tim of a ter­ri­bly de­signed pay­ment struc­ture, en­shrined by American law. Ideally, it would op­er­ate like an in­sur­ance com­pany, charg­ing pre­mi­ums in ex­change for ser­vice; but in­stead it’s forced to pay for it­self through huge bills to the small mi­nor­ity of peo­ple who will pay, ei­ther through out-of-net­work bills to in­sur­ance providers or through sur­prise bills to the pa­tients them­selves. And be­cause of this up­side-down arrange­ment, emer­gency med­ical ser­vices are chron­i­cally in­sol­vent. Public ser­vices, whether run by fire de­part­ments or mu­nic­i­pal agen­cies, don’t come close to cov­er­ing their costs from billing; and the pri­vate in­dus­try, though it charges far more, is a peren­nial money-loser. So many op­er­a­tors exit the in­dus­try each year that ambulance deserts”—ar­eas more than 25 min­utes from the near­est am­bu­lance sta­tion—have spread across rural America. American am­bu­lance firms have man­aged the un­usual feat of charg­ing ex­tra­or­di­nary prices while also go­ing broke.

The clas­sic rem­edy for a busi­ness with high fixed costs and low mar­ginal costs is scale—spread­ing fixed costs out over more vol­ume. And in­deed, high-vol­ume am­bu­lance providers do con­sid­er­ably bet­ter than low-vol­ume ones: in a 2012 study, the Government Accountability Office found that costs per trans­port ranged from $224 at the high-vol­ume end to $2,204 at the low-vol­ume end, a ten­fold dif­fer­ence.

But even scale has­n’t been enough to save the in­dus­try. AMR, the same com­pany that trans­ferred Jagdish Whitten be­tween hos­pi­tals, is the coun­try’s largest pri­vate op­er­a­tor of ground am­bu­lances; and de­spite the enor­mous scale it has as­sem­bled over decades of ac­qui­si­tions and rollups, it still runs on thin mar­gins and a heavy debt load. Rural/Metro, for a time its largest com­peti­tor, filed for bank­ruptcy in 2013.

The up­shot is a sys­tem that leaves every­one un­happy. Insurers re­sent pay­ing charges they never ne­go­ti­ated; the am­bu­lance ser­vices them­selves, de­spite charg­ing more than any­where else on earth, can barely stay sol­vent; and pa­tients ei­ther fear the costs and wave off the am­bu­lance or are left with large sur­prise bills for ser­vices they never re­quested.

Most of the po­lit­i­cal at­ten­tion, of course, has fo­cused on the prob­lem of sur­prise billing. But no one has at­tempted to solve the ac­tual un­der­ly­ing prob­lem: at­tempts to re­strict sur­prise bills have sim­ply led the costs to mi­grate some­where else.

In 2015, for ex­am­ple, the state of New York banned sur­prise am­bu­lance bills by re­quir­ing in­sur­ers to pay out-of-net­work providers close to their billed charges—which, since providers set their own charges, meant pay­ing what­ever the am­bu­lance providers asked. This was done in the name of enforcing fair­ness in health­care costs.” But it did­n’t do much of any­thing for con­sumers. Insurers paid what the providers re­quired, and passed the costs along to con­sumers as higher pre­mi­ums: am­bu­lance ride prices in­creased by 13 per­cent.

Actually cap­ping am­bu­lance charges, mean­while, would sim­ply kill the am­bu­lance in­dus­try en­tirely. Since charges to the pri­vately in­sured are the only rev­enue that cov­ers the fixed cost of readi­ness, cap­ping them would leave that cost with no payer at all. In 2020, Congress passed the No Surprises Act, which banned sur­prise bal­ance bills for most parts of emer­gency care. But by ne­ces­sity, it ex­empted ground am­bu­lances from the law: ac­tu­ally re­strict­ing the prac­tice would have ren­dered much of the EMS in­dus­try in­sol­vent.

And so thanks to a small draft­ing de­ci­sion in 1965—a Medicare pro­vi­sion to re­im­burse rides like med­ical pro­ce­dures—the United States has ended up with an emer­gency care sys­tem that is ex­pen­sive, in­sol­vent, and dis­trusted all at once.

The rea­son for that re­ally is­n’t very com­pli­cated. Force am­bu­lance ser­vices to charge only when a ride oc­curs, and then force them to pro­vide be­low-cost rides to the large ma­jor­ity of their pa­tients, and those few pa­tients whom they can charge will get charged enor­mously.

The most ef­fi­cient way to fund am­bu­lance ser­vices would sim­ply be to pay for the op­tion the way that op­tions are nor­mally paid for: with a pre­mium, col­lected from every­one the ser­vice stands ready to res­cue. That’s how it’s done in the rest of the rich world. Some places, like the United Kingdom or Japan, sim­ply fund am­bu­lance ser­vices di­rectly out of taxes; oth­ers, like the Australian state of Victoria, sell mem­ber­ships in Ambulance Victoria,” with un­lim­ited ex­er­cise at the cost of about $70 a year per fam­ily.

There are parts of the U.S. that do this al­ready. Ambulance rides are al­ready sub­si­dized by tax­pay­ers in most places, thanks to pub­lic fund­ing for fire de­part­ments; and a grow­ing num­ber of places have taken this fur­ther. Some rural coun­ties sim­ply fund am­bu­lance ser­vices en­tirely out of prop­erty taxes. In Tulsa and Oklahoma City, mean­while, the gov­ern­ment buys staffed unit-hours from an am­bu­lance op­er­a­tor, while house­holds can pre­pay a few dol­lars a month on their util­ity bill and owe noth­ing if the am­bu­lance ever comes.

Over time, more places will be forced to make sim­i­lar con­ces­sions to re­al­ity. The eco­nom­ics of readi­ness don’t care how the bills are writ­ten; some­one, in the end, has to pay for the wait­ing. Until Americans agree to pay for the op­tion, they’ll be forced to pay for the ex­er­cise.

No posts

Good Tools Are Invisible

www.gingerbill.org

2026 – 07-10

TL;DR: A good tool is and ought to be in­vis­i­ble—striv­ing to make such tools is the goal of a tool­maker.

One habit I see a lot, and have to push back on, is tak­ing a tool’s short­com­ings and re­selling them as a puzzle game” which is fun” to solve.

I don’t want my tools to be fun”. I want my tools to be in­vis­i­ble.

Text Editor Wars

Let’s take vim as an ex­am­ple

This is just an ex­am­ple, and ap­plies to other ed­i­tors too.. I con­stantly see some peo­ple praise it not for what ac­tu­ally makes it good, but by tak­ing the things it’s bad at and turn­ing them into a puz­zle to have fun” solv­ing.

I’ve had peo­ple tell me how fun” it was to build a macro to han­dle some one-off text-refac­tor­ing prob­lem. But when I looked at what they were do­ing and how long it took, my hon­est re­ac­tion was: I could have done that in Sublime in a minute with mul­ti­ple cur­sors, or just writ­ten a quick script.

To be clear, I’m not say­ing text ed­i­tors don’t mat­ter to your work­flow. I’m ques­tion­ing the near-re­li­gious de­vo­tion peo­ple have to a tool be­cause it gives them a hacker vibe”—which is ba­si­cally the whole ap­peal for new­com­ers to vim or emacs.

That’s what I mean by invisible tools”. When you’re pro­fi­cient with your ed­i­tor of choice—what­ever it is—it dis­ap­pears into the back­ground. But the mo­ment it can­not han­dle some­thing eas­ily, it stops be­ing in­vis­i­ble. What baf­fles me is that so many peo­ple treat that fric­tion—the ef­fort of work­ing around a tool’s lim­i­ta­tions—as the fun” part, and then ad­ver­tise it as ev­i­dence that the tool is great.

I know plenty of things wrong with my own ed­i­tor of choice: Sublime. I don’t dress those flaws up as fun lit­tle puz­zles to solve. I just get an­noyed that it lacks the tools I ac­tu­ally need, forc­ing me to write a plu­gin or reach for a sep­a­rate pro­gram to write to trans­form text the way I want.

I’ve been us­ing Sublime for 15 years now. It’s my ed­i­tor of choice for a few rea­sons: its short­cuts are a su­per­set of the graph­i­cal OS en­vi­ron­ment (which min­i­mizes the men­tal con­text-switch when mov­ing be­tween ap­pli­ca­tions), mul­ti­ple cur­sors re­ally are bet­ter than macros 99.999% of the time

I think I’ve only needed” a macro in Sublime twice in the past decade, and in both cases, set­ting up the macro took longer than if I just wrote a script to do the same thing. (since they give di­rect vi­sual feed­back), and it leaves me with the fewest puzzles” to solve in my text-edit­ing work­flow. I’ve found some­thing like vim to be bet­ter at ba­sic edit­ing but worse at bulk op­er­a­tions—and I don’t mean grep-like op­er­a­tions—which is why I’ve stuck with Sublime for so long. I never found vim mo­tions to be that much more pro­duc­tive than my Sublime work­flow ei­ther, and that was­n’t just down to lack of try­ing or fa­mil­iar­ity

To be hon­est, I have for­got­ten most of my vim mo­tions” knowl­edge over the years, be­cause I don’t reg­u­larly ex­er­cise it, nor do I need to.. And since I vir­tu­ally never write code in a ter­mi­nal, my need for a ter­mi­nal-ori­ented ed­i­tor is ef­fec­tively nonex­is­tent.

If peo­ple find vim, emacs, or what­ever gen­uinely good and pro­duc­tive, I’m not go­ing to crit­i­cize them for us­ing it. People are most com­fort­able with what they know. But for the peo­ple I am dis­cussing, that same fa­mil­iar­ity blinds them to their tools’ flaws, and leads them to cel­e­brate those flaws, flaunt­ing them as games.

Tools as an Identity

Part of why these de­bates turn re­li­gious is that a tool choice be­comes a flag you plant—it says some­thing about who you are. The hacker vibe” is­n’t a mere aes­thetic; it’s tribal sig­nal­ing, and that’s the real trap. Once your iden­tity is in­vested in a tool, ad­mit­ting its flaws starts to feel like ad­mit­ting some­thing about your­self. So peo­ple don’t just tol­er­ate the flaws—they de­fend them, and even­tu­ally flaunt them. You can­not have an hon­est con­ver­sa­tion about a tool with some­one who’s de­cided the tool is part of their per­son­al­ity.

Feeling Productive ver­sus be­ing Productive

The text-ed­i­tor-macro anec­dote I men­tioned is re­ally about a gap be­tween feel­ing pro­duc­tive ver­sus be­ing pro­duc­tive. There’s a sen­sa­tion of clev­er­ness that comes from solv­ing a fid­dly prob­lem, and it’s easy to mis­take that feel­ing for ac­tual out­put. A tool that makes hard things feel heroic and clever feel like an achieve­ment can reg­is­ter as powerful” while qui­etly be­ing slow. The hon­est test is­n’t how en­gaged or clever you felt, it’s wall-clock time and how many mis­takes you made get­ting there. A lot of the tools peo­ple evan­ge­lize would lose that test.

If pro­duc­tiv­ity is ac­tu­ally the goal, ac­tu­ally ques­tion your own views on this, and try to see what makes you more pro­duc­tive. You will be sur­prised when you do.

Terminal UIs vs GUIs

Another ex­am­ple in this same vein is when peo­ple ad­vo­cate for ter­mi­nal apps over GUIs. If you’re stuck in a ter­mi­nal all day, then I com­pletely get the ob­vi­ous ad­van­tage, but most pro­gram­mers aren’t stuck in a ter­mi­nal all day.

From those peo­ple who gen­er­ally ad­vo­cate for a TUI over a GUI, one of the crit­i­cisms of GUI apps tends to be: I can­not nav­i­gate them with the key­board alone”.

Okay? That does­n’t make GUI apps in­her­ently bad. It just means the GUIs peo­ple build aren’t good enough to be key­board-nav­i­ga­ble. There’s noth­ing in­her­ently im­pos­si­ble about mak­ing a GUI nav­i­ga­ble with a key­board, rather it’s just that most tool­mak­ers never bother to im­ple­ment, and usu­ally be­cause they don’t re­al­ize how much more pro­duc­tive key­board nav­i­ga­tion is than reach­ing for the mouse a lot of the time. If the ar­gu­ment was that a spe­cific TUI app is bet­ter than the other al­ter­na­tives which are GUI based, then that is a fair ar­gu­ment, but ar­gu­ing that TUIs are in­her­ently bet­ter than GUIs is very mis­in­formed.

And this is the com­mon mis­take: peo­ple look at the cur­rent state of a cat­e­gory of tools and as­sume its cur­rent lim­i­ta­tions are in­her­ent/​es­sen­tial, when re­ally no one has put in the work to make those tools bet­ter.

Linux’s (Lack of) Popularity as a Desktop

The year of the Linux

I know I am go­ing to get peo­ple say­ing Linux is the Kernel, the OS is the [insert dis­tro name]”. I’m sorry but that’s not how most peo­ple talk about Linux, and I don’t re­ally care too much for your pen­dantry which aids noth­ing. Especially since to cri­tique it, you clearly had to un­der­stand what was be­ing said about it. desk­top still is­n’t upon us (in 2026), and part of the rea­son why it has taken so long to get to that point is fun­da­men­tal: a lot of the peo­ple who use Linux love fid­dling with con­fig­u­ra­tion files to re­shape their sys­tem—it’s their idea of fun”, their puz­zle game.

I went through that phase my­self. But af­ter a while, I just wanted things to work. Spending hours (if not days) con­fig­ur­ing every­thing is­n’t some­thing I want to do any more. I want the de­faults to be good and just work, and when I do need to tweak some­thing mi­nor, it should take sec­onds.

Maximal con­fig­ura­bil­ity should­n’t be a tool’s goal, it should be an op­tion for when it’s ac­tu­ally nec­es­sary. Designing an er­gonomic tool is fun­da­men­tally about hav­ing good de­faults, while still al­low­ing es­cape hatches where they’re pos­si­ble/​needed.

The ap­peal of ac­ci­den­tal

Characteristics that an ob­ject has con­tin­gently and can change with­out al­ter­ing the ob­jec­t’s es­sen­tial iden­tity. com­plex­ity is some­thing a lot of pro­gram­mers/​techy-folk love, giv­ing them a weird sense of se­cu­rity.

Having good de­faults is fun­da­men­tally a tool­mak­er’s re­spon­si­bil­ity. We as tool­mak­ers have a ten­dency to put the bur­den on the user: to con­fig­ure it, to tweak it, to learn it. A lot of that bur­den is re­ally a de­signer de­clin­ing to make a de­ci­sion. Highly con­fig­urable” is of­ten just an ex­cuse for ship­ping no opin­ion at all and call­ing the re­sult­ing work your prob­lem. Good de­faults are a form of re­spect for the user’s time: the tool­maker does the think­ing once so a thou­sand users don’t each have to. And part of de­sign­ing a tool is to al­low for some es­cape hatches tool; those es­cape hatches are there for the gen­uine mi­nor­ity who need some­thing un­usual; they should not be a sub­sti­tute for get­ting the com­mon case right.

Steep Learning Curve as a Feature”?

Another de­fence I’ve seen is that the dif­fi­culty is the whole point, it fil­ters out the un­com­mit­ted, and once you’re over the hump you’re re­warded for life. But a learn­ing curve is a cost, not a virtue. It could hy­po­thet­i­cally be ab­solutely a cost worth pay­ing, but the pay­off has to be gen­uine pro­duc­tiv­ity, not the sat­is­fac­tion of hav­ing paid it. Too of­ten the rea­son­ing is just sunk-cost dressed up as merit: I spent months learn­ing this, so it must be worth it, and you should copy in my foot­steps too”. That’s the puz­zle game again, only now the puz­zle is the tool it­self.

Conclusion

None of this is an ar­gu­ment against any par­tic­u­lar tool. It’s an ar­gu­ment against a way of think­ing. Use vim, use emacs, use Sublime, but use what­ever dis­ap­pears into the back­ground and lets you get on with the work. That is the whole test, and it’s a per­sonal one. What I’m push­ing back on is­n’t the choice, it’s the sto­ry­telling that grows up around the choice: the re­fram­ing of lim­i­ta­tions as fea­tures, the ef­fort of work­ing around a flaw sold as the re­ward, the tool qui­etly pro­moted from the thing-you-use to the part-of-who-you-are.

The clear­est sign a tool is serv­ing you is that you stop notic­ing it—it be­comes in­vis­i­ble. You don’t cel­e­brate its flaws be­cause you’re not turn­ing them into a hobby, rather you just get mildly an­noyed and route around them. You don’t de­fend it be­cause noth­ing about your iden­tity is rid­ing on it. And you don’t mis­take the feel­ing of clev­er­ness for the fact of pro­duc­tiv­ity, be­cause you’ve both­ered to check the dif­fer­ence.

So by all means, en­joy your tools, for the joy of pro­gram­ming it­self. Just be hon­est about which parts are gen­uinely good and which parts you’ve talked your­self into lov­ing. The best tool is­n’t the one with the best story. It’s the one you for­get you’re us­ing.

A good tool is and ought to be in­vis­i­ble—striv­ing to make such tools is the goal of a tool­maker.

Collections: The Late Bronze Age Collapse, A Very Brief Introduction

acoup.blog

This week, by or­der of the ACOUP Senate, we’re talk­ing about the Late Bronze Age Collapse (commonly ab­bre­vi­ated LBAC), the shock­ing col­lapse of the Late Bronze Age state sys­tem across the Eastern Mediterranean and Middle East dur­ing the 12th cen­tury (that is, the 1100s) BC. In the broader Mediterranean world, the Late Bronze Age Collapse is the event that prob­a­bly comes clos­est to a true end of civ­i­liza­tion’ event — mean­ing­fully more se­vere than the col­lapse of the Roman Empire in the West (although as we’ll see LBAC is also not as total’ of a col­lapse as was some­times sup­posed).

This is go­ing to be, by our stan­dards here, some­thing of a brief overview, roughly the equiv­a­lent to the lec­ture I give to my stu­dents when we cover this pe­riod (with a bit more de­tail, be­cause text is more com­pressed). A full deep di­ve’ of all of the de­bates and open ques­tions of this pe­riod would no doubt run quite a few posts and more im­por­tantly re­ally ought to be writ­ten by spe­cial­ists in the bronze age. This is also a very ar­chae­o­log­i­cally dri­ven topic, which makes it more sen­si­tive than most to new ev­i­dence — ar­chae­o­log­i­cal site work, but also epi­graphic ev­i­dence (mostly on clay tablets) — that can change our un­der­stand­ing of events. As we’ll see, our un­der­stand­ing has changed a fair bit.

So what we’ll do is run through what we know about what hap­pened in the col­lapse (which is the most vis­i­ble part of it) and then we’ll loop back to the ques­tion of causes (which re­main sub­stan­tially un­cer­tain) and then fi­nally look at the long-term im­pacts of the col­lapse, which are con­sid­er­able.

But first, as al­ways, if you like what you are read­ing here, please share it; if you re­ally like it, you can sup­port me on Patreon; mem­bers at the Patres et Matres Conscripti level get to vote on the top­ics for post-se­ries like this one! If you want up­dates when­ever a new post ap­pears or want to hear my more bite-sized mus­ings on his­tory, se­cu­rity af­fairs and cur­rent events, you can fol­low me on Bluesky (@bretdevereaux.bsky.social). I am also ac­tive on Threads (bretdevereaux) and main­tain a de min­imis pres­ence on Twitter (@bretdevereaux).

The (Partial?) Collapse

We need to be clear, to be­gin with, that while we have scat­tered frag­ments of epi­graphic ev­i­dence (that is, in­scrip­tions), al­most all of our ev­i­dence for the Late Bronze Age Collapse is ar­chae­o­log­i­cal. Without ar­chae­ol­ogy, we would re­main largely in the dark about this event. But ar­chae­o­log­i­cal ev­i­dence also brings with it chal­lenges: it can tell you what is hap­pen­ing (sometimes) but of­ten not why and dat­ing with pre­ci­sion can be chal­leng­ing. Most of what we’re track­ing in un­der­stand­ing LBAC is site de­struc­tion, iden­ti­fied by the de­mo­li­tion of key build­ings or destruction lay­ers’ (often a thin layer of ash or rub­ble in­di­cat­ing the site was burned or de­mol­ished), but dat­ing these pre­cisely can be dif­fi­cult and there are al­ways chal­lenges of in­ter­pre­ta­tion.

With that said, the Late Bronze Age Collapse is a se­quence of site de­struc­tions vis­i­ble ar­chae­o­log­i­cally from c. 1220 BC to c. 1170 BC, which are as­so­ci­ated with the col­lapse or se­vere de­cline of the ma­jor states of the re­gion (the Eastern Mediterranean and Middle East). We gen­er­ally con­cep­tu­al­ize these de­stric­tions as a wave’ mov­ing in se­quence be­gin­ning in the Aegean, mov­ing over Anatolia, sweep­ing down the Levant and ar­riv­ing in Egypt but in many cases my sense is the chronol­ogy is more com­plex than that. Many sites in the path of this wave’ were not de­stroyed, with some de­clin­ing slowly and oth­ers de­clin­ing not much at all; other sites (I have in mind Tiryns) see the de­struc­tion of their po­lit­i­cal cen­ter but the de­cline of the ur­ban set­tle­ment around it hap­pens slowly or later.

First, we ought to set the stage of the Late Bronze Age. What re­ally marks out the Late Bronze Age (c. 1500 BC to c. 1200 BC) from ear­lier pe­ri­ods is that the emerg­ing state sys­tems in Mesopotamia, Syria, Anatolia and Egypt had ex­panded to the point of com­ing quite fully into con­tact with each other, with a sig­nif­i­cant de­gree of diplo­matic, eco­nomic and cul­tural in­ter­con­nect­ed­ness, to the point that we some­times re­fer to the Late Bronze Age Concert of Powers’ (evoking 19th cen­tury European bal­ance of power pol­i­tics) when talk­ing in­for­mally about them.

Now I should cau­tion, we of­ten pro­vide these nice neat maps of the Late Bronze Age pow­ers (and they’re use­ful to a de­gree) but the bor­ders of these states were quite fuzzy — their outer possessions’ were of­ten trib­u­taries un­der the rule of lo­cal kings which might be weakly at­tached to the im­pe­r­ial cen­ter. Nevertheless, go­ing from East to West: south­ern Mesopotamia was dom­i­nated by the Middle Babylonian’ Empire, ruled by the Kassite dy­nasty (the Kassites be­ing an eth­nic group who had taken power around 1530 BC) while north­ern Mesopotamia was dom­i­nated by the Middle Assyrian Empire (from about c. 1350 BC). Anatolia and the Northern Levant was con­trolled by the multi-eth­nic Hittite Empire, which seems to have sparred reg­u­larly with the New Kingdom of Egypt which con­trolled Egypt and the south­ern Levant. Basically all of these pow­ers had less set­tled, of­ten pas­toral peo­ples in their hin­ter­lands which pre­sented on-go­ing se­cu­rity chal­lenges for them.

These larger im­pe­r­ial states were more eco­nom­i­cally com­plex as well. In par­tic­u­lar, their large armies re­quired sig­nif­i­cant amount of bronze which — be­cause its core in­gre­di­ents of tin and cop­per ef­fec­tively never oc­cur in the same place — de­manded sub­stan­tial long-dis­tance trade, though trade was hardly only in cop­per and tin, but also in­cluded other high value goods and even (where fea­si­ble) bulk sta­ples. So while these pow­ers clashed reg­u­larly, at the elite level (if not at the level of the sub­sis­tence econ­omy) they were also re­liant on each other to some de­gree.

Finally, at the edge of this state sys­tem is the Mediterranean and es­pe­cially the Aegean. In the Aegean — in Greece and Crete es­pe­cially — we see ef­fec­tively minia­ture ver­sions of these state struc­tures, com­plete with (by Near Eastern Standards) itty-bitty palaces (the Minoan ur­ban cen­ters on Crete had come un­der Mycenean (=Greek) rule in c. 1450, the palaces there largely aban­doned). Cyprus shifted be­tween be­ing nom­i­nally sub­or­di­nate to ei­ther the Hitties of the Egyptians but seems to have mostly run its own af­fairs and was in­te­grated through trade into the state sys­tem.

As noted above, LBAC starts per­haps as early as 1220 or so, and what we see in very rough se­quence is as fol­lows.

As far as I know, we still gen­er­ally think the ear­li­est rum­blings are in­sta­bil­ity in the Mycenean Greek palace states. Things had been un­sta­ble in this area for a few decades and we have some scat­tered de­struc­tions (Thebes) and in­ten­si­fied for­ti­fi­ca­tions around 1250, sug­gest­ing things were not go­ing great in Greece. Then from c. 1200 to c. 1180 we see the de­struc­tion or col­lapse of ba­si­cally all of the palace cen­ters in Greece. In some cases the ur­ban core con­tin­ues for a while, in other cases it does­n’t — in a num­ber of cases, once the site is aban­doned, it is not rein­hab­ited (e.g. Mycenae it­self, the largest of the palace cen­ters).

As we’ll see be­low, the im­pact in Greece is greater than ba­si­cally any­where else be­cause the col­lapse of the LBAC is more se­vere in Greece than ba­si­cally any­where else.

Meanwhite, the Hittite Empire was it­self not in good shape when this started. As far as we know, the Hittites were very much on the back foot’ in the late 1200s, pres­sured by the Assyrians and Egypt and so po­ten­tially al­ready short on re­sources when their neigh­bors to the West be­gan im­plod­ing. As far as I know, pre­cise dates are hard to nail down for this, but the Hittite Empire in the early 1100s comes apart un­der pres­sure and by 1170 or so it is gone. That col­lapse of im­pe­r­ial power is matched by a sig­nif­i­cant num­ber of site de­struc­tions across Anatolia, in­clud­ing the Hittite cap­i­tal at Hattusas and the large set­tle­ment at mod­ern Hisarlik, now fairly se­curely iden­ti­fied as an­cient Troy. Some (like Troy) were re­built, oth­ers (like Hattusas) were not, but cen­tral­ized Hittite power was gone and there’s a marked re­duc­tion in ur­ban­iza­tion and prob­a­bly pop­u­la­tion.

Moving into the Northern Levant, Syria and Northern Mesopotamia, we see Assyrian power — which had been ad­vanc­ing be­fore, you’ll re­call — con­tract sharply along­side more site de­struc­tions, though again chronol­ogy is tricky. One of the key sites here is Ugarit, a ma­jor Bronze Age Levantine coastal city which was de­stroyed c. 1190 — be­fore the last of the Mycenean palaces (but af­ter the first of them). The city’s de­struc­tion in fire pre­served clay tablets with diplo­matic mes­sages from the lo­cal king of Ugarit (a Hittite vas­sal) fran­ti­cally writ­ing to his Hittite su­pe­ri­ors for re­in­force­ments in the face of sig­nif­i­cant (but frus­trat­ingly un­named) threats prior to the de­struc­tion of the city.

That said, de­struc­tion in the Fertile Crescent is very un­even. The Middle Assyrian Empire con­tracts, but does not col­lapse, while the Kassite Dynasty in Babylon clearly suf­fers some de­cline, but largely sta­bi­lizes by the 1160s be­fore be­ing run over by the Elamites in the 1150s. Site de­stric­tions in the Levant are un­even and some key Bronze Age cen­ters like Sidon and Byblos were not de­stroyed and re­mained ma­jor cen­ters into the Iron Age.1 My un­der­stand­ing is that while there was sig­nif­i­cant de­cline in the south­ern Levant, it is hard to pin any spe­cific large-scale site de­struc­tion to the 1220 – 1170 pe­riod.

Finally we reach Egypt in a pe­riod we re­fer to as the New Kingdom’ (1570 – 1069); we can trace pol­i­tics more clearly here due to sur­viv­ing Egyptian in­scrip­tions. Egypt was also in a weak­ened po­si­tion go­ing into this cri­sis, fac­ing pres­sure from Libyan raiders com­ing over­land from the West and also some in­ter­nal in­sta­bil­ity. In c. 1188, civil war broke out as the last queen of the reign­ing 19th dy­nasty was un­able to re­tain con­trol, lead­ing to re­volt and the seizure of power by Setnakhte and the 20th dy­nasty; his son Ramesses III took power in c. 1185. Things did­n’t get eas­ier from there as we hear re­ports of re­newed Libyan in­cur­sions in c. 1180 (coming from the west) fol­lowed al­most im­me­di­ately by an in­va­sion by the sea peo­ples’ (see be­low) who were ev­i­dently fended off in at least two ma­jor bat­tles, the Battle of the Delta (c. 1179ish?) and the Battle of Djahy (c. 1178ish?).

Egypt holds to­gether, but there’s a fair bit of ev­i­dence eco­nomic strain (likely cli­mate based, see be­low) and the abil­ity of Egypt to pro­ject power out­side of Egypt seems largely spent by the end of the reign of Ramesses III; his suc­ces­sors do not ap­pear to have been able to right the ship and Egyptian power con­tin­ued to frag­ment and de­cline, with the dy­nasty stum­bling on un­til it col­lapsed in 1077 lead­ing to the Third Intermediate Period (‘Intermediate Periods’ are the term for pe­ri­ods of frag­men­ta­tion within Egypt).

I should note in this overview that our un­der­stand­ing of this se­quence of col­lapses and de­clines has changed sig­nif­i­cantly. The idea of the Late Bronze Age Collapse has been around since the early 1800s when his­to­ri­ans first no­ticed that the end of the Greek Age of Heroes’ (linked by them to the Fall of Troy, which the (Classical) Greeks be­lieved hap­pened in 1184) seemed to map neatly on to the fail­ure of the Egyptian 19th Dynasty. As ar­chae­ol­o­gists in the later 1800s and early 1900s started ac­tu­ally ex­ca­vat­ing the Greek Age of Heroes’ (thus dis­cov­er­ing the (Mycenaean) Greek Late Bronze Age, which we term the Late Helladic’ pe­riod (c. 1700-c. 1040 BC)) and then find­ing site de­struc­tions date­able within a band of per­haps 1250 to 1150 BC in Greece, Anatolia, Syria and the Levant the idea of a gen­eral col­lapse around the leg­endary date for the Fall of Troy picked up a lot of steam.

My sense of the schol­ar­ship is that this civilizational col­lapse’ nar­ra­tive has been drawn back a bit as it be­comes clear that some sites were not de­stroyed and also that some site de­struc­tions or aban­don­ments hap­pened sig­nif­i­cantly later or ear­lier than the rel­a­tively tight 1220 – 1170 BC time frame that emerged for the core of the col­lapse. No one (that I know of) is ar­gu­ing there was no LBAC — there was clearly an LBAC — but the scale of the col­lapse re­mains some­thing of a mov­ing tar­get as we ex­ca­vate more sites, adding them to lists of sites that were de­stroyed, de­clined or (sometimes seem­ingly ran­domly) were spared.

And the list of sites that were not de­stroyed is sig­nif­i­cant. Of note, Athens very clearly has a Mycenaean citadel on the Acropolis (which can’t be ex­ca­vated be­cause the Acropolis is in the way, but it is very ob­vi­ously there) but there’s no break in set­tle­ment in Athens. Already men­tioned, Byblos and Sidon re­mained very promi­nent cen­ters be­fore and af­ter, while Jerusalem and Tyre, both ap­par­ently mi­nor set­tle­ments be­fore LBAC (and not de­stroyed) will be­come in­creas­ingly promi­nent in the Iron Age Levant. Likewise the great cities of Egypt and Mesopotamia re­main, few to no site de­struc­tions in ei­ther re­gions. At the same time, many set­tle­ments that es­cape de­struc­tion do not es­cape de­cline: in many cases these cities con­tinue to shrink (and some places that es­cape de­struc­tion, like Tiryns, shrink slowly rather than van­ish­ing all at once) or grow vis­i­bly poorer in a longer process. So the mo­ment of de­struc­tion comes with a long tail’ of de­cline stretch­ing out decades.

So to sum­ma­rize, the Late Bronze Age Collapse is a se­ries of site de­struc­tions, aban­don­ments and de­clines run­ning from roughly 1220 to roughly 1170 (though de­cline con­tin­ues af­ter this point) dis­trib­uted quite un­evenly through the in­ter­con­nected Late Bronze Age Mesopotamian-and-Eastern-Mediterranean world. Greece and Anatolia are se­verely im­pacted, the Levant some­what less but still fairly strongly, while the states of Egypt and Mesopotamia do not col­lapse but en­ter long pe­ri­ods of de­cline.

What that de­scrip­tion leaves out, of course, are causes and ef­fects.

Bad Theories

While the what’ of LBAC can be pinned down fairly con­clu­sively with ar­chae­ol­ogy, the why’ is tougher — a lot of po­ten­tial causes (wars, armies, civil un­rest) don’t nec­es­sar­ily leave a lot of clues in our source ma­te­r­ial.

There are a few the­o­ries we can largely dis­count at the out­set though. The older of these were the­o­ries that as­sumed that the cause of at least some of the Late Bronze Age Collapse were large-scale mi­gra­tions of peo­ple into (rather than within) the set­tled, ur­ban zone we’ve been talk­ing about, in par­tic­u­lar the idea of a Dorian Invasion’ of Greece as the spark of the col­lapse. Proposed in the 1800s, the idea here was that the Dorians’ — the an­ces­tors of the Greeks — would have mi­grated into Greece, de­stroy­ing the Mycenaean cities and palaces and dis­plac­ing or dom­i­nat­ing the pre­vi­ous (non-Greek) in­hab­i­tants. This no­tion was based on mixed and com­pet­ing ideas within (Classical) Greek lit­er­a­ture: Greek au­thors both ex­pressed the idea of the Greeks be­ing au­tochtho­nous (indigenous to their ter­ri­tory, lit­er­ally [arising] on their own from the earth’) and also be­ing in­vaders, ar­riv­ing at some point forty to eighty years af­ter the Trojan War (e.g. Thuc. 1.12; Hdt. 1.56 – 58). That idea got picked up by 19th cen­tury European schol­ars who, to be frank, of­ten thought un­crit­i­cally in terms of pop­u­la­tion mi­gra­tion and re­place­ment, through an of­ten ex­plic­itly racist lens of superior stock’ dri­ving out inferior stock.’ And so they imag­ined a Dorian in­va­sion’ of the (racially) superior’ Greek-speaking Dorians2 dri­ving out the pre-Greek Mycenaean pop­u­la­tion, par­tic­u­larly in the Peloponnese.

As an aside, it is not un­com­mon for a sin­gle so­ci­ety to uti­lize both leg­endary myths of au­tochthony and ar­rival-by-con­quest, choos­ing whichever is more use­ful in the mo­ment, even though they are ob­vi­ously, from a log­i­cal stand­point, mu­tu­ally in­com­pat­i­ble.

Archaeology has fun­da­men­tally un­der­mined this the­ory — nuked it from or­bit, re­ally — in two key ways. First, we have Mycenaean writ­ing, which was dis­cov­ered in a strange script called Linear B (Minoan writ­ing is Linear A). Originally un­read­able to us, in 1952 Michael Ventris suc­cess­fully demon­strated that Linear B was, in fact, Greek (rendered in a dif­fer­ent, older script) and so the Mycenaeans were Greeks. Meanwhile a wide range of ar­chae­ol­o­gists and ma­te­r­ial cul­ture schol­ars, as more late Helladic and early Archaic pot­tery and art­work emerged, were able to demon­strate there sim­ply was no dis­con­ti­nu­ity in ma­te­r­ial cul­ture. The Greeks could not be ar­riv­ing at the end of the Bronze Age be­cause they were al­ready there and had been for cen­turies at least. Migrations within the Eastern Mediterranean might still play a role, but the idea that the col­lapse was caused by the ar­rival of the Greeks has been de­ci­sively aban­doned. There was no Dorian Invasion.

The other cause we can prob­a­bly dis­miss is a sin­gle, sud­den nat­ural calamity. There are two can­di­dates here to note. The first is sim­ply peo­ple con­fus­ing the ma­jor erup­tion of Thera (c. 1600) which is some­times as­so­ci­ated with the de­cline of the Minoan Palaces (though the chronol­ogy does­n’t re­ally work well there ei­ther) with LBAC. The sec­ond is ef­fort to con­nect the erup­tion of Hekla in Iceland with LBAC. The prob­lem again is that the chronol­ogy does not ap­pear to work out — es­ti­mates for the dat­ing of the Hekla erup­tion range from 1159 to 929 with the con­sen­sus be­ing, as I un­der­stand it, closer to 1000 BC. For our part, the range does­n’t mat­ter much — even that ear­li­est 1159 date would mean that Hekla’s mas­sive erup­tion could hardly ex­plain the col­lapse of Mycenean palaces hap­pen­ing at least forty years ear­lier. Climate played a role in LBAC, but it is not clear that vol­canic cli­mate in­flu­ence did and it is very clear that Hekla did not (though per­haps it con­tributed to make a bad de­cline worse.

So no Dorian Invasions’ and no vol­ca­noes, so what did cause it?

Causes of LBAC

We have no firm an­swers, but a num­ber of plau­si­ble the­o­ries and at this point my sense is that just about every­one work­ing on this pe­riod adopts some vari­a­tion of all of the above’ from this list.

We can start with cli­mate. For rea­sons there’s been quite a lot of re­search into his­tor­i­cal cli­mate con­di­tions and we can ac­tu­ally get a sense of those con­di­tions to a de­gree ar­chae­ol­ogy from things like tree rings (where very nar­row rings can in­di­cate dry years or oth­er­wise un­fa­vor­able con­di­tions). I don’t work on his­tor­i­cal cli­mate, but my un­der­stand­ing is there is quite a lot of com­pelling ev­i­dence that pe­riod of LBAC, es­pe­cially the 1190s, was un­usu­ally dry in the Eastern Mediterranean, which would have caused re­duced agri­cul­tural out­put (crop fail­ures). Interestingly, this would be most im­me­di­ately im­pact­ful in ar­eas en­gaged pri­mar­ily in rain­fall agri­cul­ture (Greece, Anatolia, the Levant) and less im­pact­ful in ar­eas en­gaged more in ir­ri­ga­tion agri­cul­ture (Egypt, Mesopotamia).3 And, oh look, the ar­eas where LBAC was more se­vere are in the rain­fall zone and the ar­eas where it was less se­vere are in the ir­ri­ga­tion zone.

Crop fail­ures may have been par­tic­u­larly po­lit­i­cally volatile be­cause of the struc­ture and val­ues of the kind of Near Eastern states (to in­clude Anatolia and Greece here) that we’re deal­ing with. We haven’t dis­cussed early bronze age states very much but the ev­i­dence we have sug­gests that these were sig­nif­i­cantly cen­tral­ized states, with a lot — not all, but a lot — of the re­sources mov­ing through ei­ther state (read: royal) struc­tures or through tem­ple in­sti­tu­tions which might as well have been state struc­tures. Which is to say these are so­ci­eties where the king and the tem­ples (which re­port to the king) own most of the land and so har­ness most of the agri­cul­tural sur­plus through rents and then em­ploy the li­on’s share of non-agri­cul­tural la­bor, re­dis­trib­ut­ing their pro­duc­tion. Again, I don’t want to over­state this — there is a private sec­tor’ in these economies — but it seems (our ev­i­dence is lim­ited!) to be com­par­a­tively small.

Meanwhile, the clearly at­tested re­li­gious role of the king in a lot of these so­ci­eties in­cludes a re­spon­si­bil­ity — of­ten the para­mount re­spon­si­bil­ity — to main­tain the good re­la­tions of the com­mu­nity with the gods (who pro­vide the rain and make the plants grow).

Repeated crop fail­ures are thus go­ing to be seen as a sign that the King is falling down on the job. Worse yet, they’ll have come at the same time as the King found him­self strained to main­tain his bu­reau­crats and sol­diers, be­cause the en­tire top-heavy royal ad­min­is­tra­tion this sys­tem re­lies on is fed off of the sur­plus it ex­tracts.

It is not hard to see how this is a recipe for po­lit­i­cal in­sta­bil­ity if large states do not have the re­sources to fall back on to re­spond to the cri­sis.

To which some schol­ars have noted that the pe­riod di­rectly lead­ing up to LBAC seems to have been a pe­riod of in­ten­si­fy­ing war­fare: we hear of larger armies op­er­at­ing in the wars in Mesopotamia, Egypt and the Levant and we see mas­sively greater in­vest­ment in for­ti­fi­ca­tion in the Aegean all sug­gest­ing that the states are pour­ing re­sources into war­fare. That may have left these states with fewer re­sources (idle la­bor, stored grain, money-covertable valu­ables or sim­ply re­serves of pub­lic good­will since long years of high taxes in long wars tends to tire peo­ple out) with which to con­front a sud­den wave of com­bined po­lit­i­cal un­rest and food short­age.

What is clear is that once the col­lapse started, it was con­ta­gious, likely for two rea­sons: first that col­laps­ing ar­eas pro­duced in­vad­ing forces and refugee flows that desta­bi­lized their neigh­bors and sec­ond be­cause as you will re­call above, these states are in­ter­linked and their rulers rely on trade to fur­nish the key mil­i­tary re­source (bronze) as well as to ac­quire key pres­tige goods nec­es­sary to main­tain the loy­alty of the aris­toc­racy.

The clear­est ev­i­dence of this are the re­ports in Egyptian in­scrip­tions of peo­ples grouped un­der the mod­ern head­ing of Sea Peoples’ be­cause they are of­ten de­scribed as be­ing of the sea’ in one way or an­other. The ev­i­dence here is tricky: what we have are a set of in­scrip­tions, span­ning from 1210 through to the mid-1100s de­scrib­ing fight­ing against — and, this be­ing Egyptian royal writ­ing, in­vari­ably the vic­tory of a Pharaoh over — a range of in­vad­ing peo­ples. What is tricky is these re­ports cover mul­ti­ple pe­ri­ods of fight­ing and they’re us­ing Egyptian names for these peo­ple mean­ing we’re not al­ways en­tirely con­fi­dent that we can tell who ex­actly the Egyptians meant to iden­tify.

Generally, how­ever, what we seem to be see­ing is in­creased pres­sure on Egypt from c. 1205 to c. 1170 from multi-eth­nic coali­tions of peo­ples drawn from the Aegean, Anatolia and the Levant. In par­tic­u­lar, in­scrip­tions from the reign of Merneptah (r. 1213 – 1203) re­port at­tacks by the Ekwesh (possibly an Egyptian ren­der­ing of Achaioi, Achaean,’ mean­ing Greek) along with the Lukka (an Anatolian peo­ple), the Sherden (probably a Levantine peo­ple, per­haps the Philistines) and oth­ers even harder to pin down like the Shekelesh (more Anatolians? Sicels? other peo­ple on boats?). Later in­scrip­tions from the reign of Ramesses III (r. 1185 – 1154) re­port rel­a­tively early in his reign vic­to­ries against coali­tions that in­clude the Denyen (possibly an Egyptian ren­der­ing of Danaioi,’ mean­ing Greek), the Sherden (again), the Shekelesh (again), the Peleset (Levantine peo­ple, prob­a­bly Philistines) and oth­ers.

The way this ev­i­dence is gen­er­ally read — and this seems the most plau­si­ble ex­pla­na­tion — is that the dis­rup­tions in the Aegean, Anatolia and Levant may have them­selves pro­duced armed mass-mi­gra­tions, mov­ing by sea (these were all sea-far­ing peo­ples), per­haps look­ing for safe har­bor. Or per­haps quite lit­eral bands of raiders — the col­lapse of state struc­tures in Greece and Anatolia might well have left a lot of full-time vi­o­lence-do­ers with­out steady em­ploy­ment and go­ing raid­ing may have been a nat­ural re­course for some. There is some sense in Hittite doc­u­ments, for in­stance that the Ahhiyawa’ (Hittite ren­der­ing for Achaioi, mean­ing Greek) might have been an hos­tile neigh­bors to the Hittites and given how heav­ily mil­i­ta­rized elite Mycenaean cul­ture seems to have been, it would­n’t be shock­ing if they reg­u­larly went on seaborne raids (though, again, the ev­i­dence here is very thin).

Meanwhile, while trade does not com­pletely stop, it cer­tainly seems to be re­duced by the col­lapse of these states, pos­si­bly in­ter­rupt­ing the sup­ply of key goods — the most ob­vi­ous be­ing bronze — and any state rev­enues de­rived from tax­ing trade (which they did).

Consequently the consensus’ vi­sion — which re­mains to a de­gree con­jec­tural, al­though it is the best fit’ for the ev­i­dence — runs roughly like this:

Intensifying war­fare in the E. Mediterranean and Mesopotamia may have re­duced the re­sources avail­able for ma­jor states to con­front a cri­sis and per­haps were al­ready as­so­ci­ated with some kind of un­rest.

A shift to a drier cli­mate causes har­vest fail­ures which be­gin to push the tee­ter­ing states over the edge into col­lapse.

In Greece, the palace states be­gin to col­lapse one by one — prob­a­bly from in­ter­nal strains (e.g. an op­pressed peas­antry) rather than ex­ter­nal in­va­sion.

Because the palace econ­o­my’ was so cen­tral (and em­ployed a lot of peo­ple, in­clud­ing a lot of war­riors), col­lapse within Greece may have been con­ta­gious as raids and refugees spawned by col­laps­ing palace sys­tems fa­tally strained oth­ers.

Because the palace econ­o­my’ was so cen­tral (and em­ployed a lot of peo­ple, in­clud­ing a lot of war­riors), col­lapse within Greece may have been con­ta­gious as raids and refugees spawned by col­laps­ing palace sys­tems fa­tally strained oth­ers.

Those col­lapses in turn be­gin to dis­rupt trade but also pro­duce out­ward move­ments of refugees and/​or raiders, which may in part be what is be­ing remembered’ in Homer’s ac­count of the Trojan War or the broader Greek mytho­log­i­cal as­sump­tion that the Trojan War marks the end of the Age of Heroes’ (which is how the Classical Greeks un­der­stood this pe­riod).

That same strain hits the al­ready ail­ing Hittite Empire, strained by wars and de­feats in the Levant against the Egyptians and Assyrians. Battered by har­vest fail­ures and in­creas­ing raids (such as those Ugarit is cry­ing for help from), Hittite power col­lapses.

The states of the Northern Levant, un­der pres­sure al­ready now lose their pro­tec­tor, while the other ma­jor states of the re­gion (Egypt, Assyria, Kassite Babylon) lose a key trade part­ner and at least some ac­cess to tin in par­tic­u­lar (required for bronze).

The re­sult­ing eco­nomic con­trac­tion pro­duces in­ter­nal in­sta­bil­ity (Nineteenth dy­nasty re­placed by Twentieth in Egypt) and com­bined with fur­ther raid­ing/​refugee pres­sures, all of these im­pe­r­ial pow­ers con­tract into their home­lands, no longer able to pro­ject power far afield.

In Babylon, the Kassites more or less sta­bi­lize by the 1160s, but in a weak­ened state, are over­run by the Elamites — a per­pet­ual lo­cal threat — in the 1150s. In Egypt there’s a mo­ment of re­cov­ery and sta­bil­ity un­der Ramesses III of the new Twentieth Dynasty, but fur­ther suc­ces­sion dis­putes — per­haps in part mo­ti­vated by bad eco­nomic con­di­tions — lead to power frag­ment­ing un­til cen­tral rule col­lapses in the early 1070s. Assyrian power con­tracts back to the Assyrian home­land in Northern Mesopotamia, but the state sur­vives, to reemerge as a stag­ger­ingly ma­jor power in the early Iron Age.

You will of course note that we can ob­serve all of these stages only very im­per­fectly: we’re work­ing with frag­men­tary let­ters, in­scrip­tions that are of­ten un­re­li­able and of­ten very good ar­chae­ol­ogy that can tell us what hap­pened (‘this palace was burned and all of the fin­ery was dumped in a well’) but not why.

The Effects of the Collapse

Just as the col­lapse it­self was un­even — some states and set­tle­ments de­stroyed, oth­ers largely spared — so too its ef­fects were un­even, so we might do a brief run­down by re­gion.

But first I want to note the ef­fect the col­lapse has on our ev­i­dence. In many places, I com­pare it to a light­ning bolt at night that takes out the power. Immediately be­fore the col­lapse, it was dim, but there was some light: though deep in the past, we have large states that are cre­at­ing records and in­scrib­ing things on stone some small por­tion of which sur­vive; we can’t see any­where near as well as we can dur­ing the last mil­len­nium BC, but we can see some things. Then the col­lapse hits like that bolt of light­ning and we sud­denly get a lot of ev­i­dence at once. Destruction lay­ers are of­ten ar­chae­o­log­i­cally rich (things get de­posited that would­n’t nor­mally) and when, for in­stance, some­one burns an archive full of clay tablets, that fires the clay tablets in ce­ramic, which can sur­vive. Meanwhile it is eas­ier to ex­ca­vate sites that were aban­doned and not re-in­hab­ited: they prob­a­bly don’t have ma­jor mod­ern cities on them and you don’t have to ex­ca­vate care­fully through cen­turies of dense, con­tin­u­ous habi­ta­tion to get down to the bronze age level.

But then in many ar­eas — es­pe­cially Greece — we are plunged into a lot of dark­ness. The states that were pro­duc­ing writ­ten records are ei­ther much smaller or gone en­tirely. Reduced at the same time is trade in goods that we can use to see long-dis­tance cul­tural con­nec­tions. And in many cases poorer so­ci­eties build in wood and mud­brick rather than stone; the lat­ter sur­vives far bet­ter than the for­mer to be ob­served ar­chae­o­log­i­cally.

The Aegean and main­land Greece — that is, the Mycenaean Greeks — were ev­i­dently hit hard­est by the col­lapse. Much like Britain when the Roman Empire col­lapsed in the West, be­ing on the very edge of the state sys­tem as it came apart left them ev­i­dently far more iso­lated with a much more se­vere de­cline. Large-scale stone build­ing ef­fec­tively van­ishes in Greece and won’t reap­pear un­til the Archaic pe­riod (750 – 480), which in turn makes it much harder to ob­serve things like set­tle­ment pat­terns dur­ing the in­ter­ven­ing pe­riod, some­times termed the Greek Dark Age (1100 – 750; many ar­chae­ol­o­gists of the pe­riod dis­like this term for ob­vi­ous rea­sons). But from what we can see, Greece seems to largely deur­ban­ize in this pe­riod, al­though at least one Mycenaean cen­ter sur­vives — Athens. That may in turn ex­plain to some de­gree why Athens is such a big po­lis in terms of its ter­ri­tory by the time we can see it clearly in the Archaic.

Perhaps most shock­ingly, main­land Greece loses writ­ing. The Mycenaean palaces had de­vel­oped a syl­labic script, which we call Linear B, to rep­re­sent their spo­ken Greek. This form of writ­ing is en­tirely lost. In the 8th cen­tury, the Greeks will adopt an en­tirely new script — bor­row­ing the one the Phoenicians are us­ing — to rep­re­sent their lan­guage and we (and they) will be un­able to read Linear B un­til 1953.

The to­tal­ity of the col­lapse of cen­tral state in­sti­tu­tions in Mycenaean Greece may in part ex­plain the emer­gence of a po­lit­i­cal in­sti­tu­tion as strange as the po­lis. It is clear that through the Greek Dark Ages’ and the sub­se­quent Archaic pe­riod, though Greek com­mu­ni­ties have kings’ — though called basileis (a word that in the Mycenaean Linear B tablets would mean village chief,’ a sub­or­di­nate to the ac­tual king in the palace, the wanax, a term Homer uses for Agamemnon and Priam only) — they lack the cen­tral­ized eco­nomic en­gine of the palace econ­omy and in­stead have much weaker cen­tral gov­ern­ing sys­tems. It is some­thing not quite but per­haps close to a clean slate’ from which to de­velop new sys­tems of gov­er­nance that will look very dif­fer­ent from what so­ci­eties to their East had de­vel­oped.

No other part of the Eastern Mediterranean suf­fers a civ­i­liza­tional set­back quite as in­tense as in Greece, but per­haps the most sig­nif­i­cant ef­fect is a pe­riod of pro­longed po­lit­i­cal frag­men­ta­tion in Anatolia and the Levant. These re­gions had been, over the Late Bronze Age, largely un­der the con­trol of ma­jor im­pe­r­ial pow­ers (Egypt, Assyria, the Hittites), but with those pow­ers re­moved they have a chance to de­velop some­what in­de­pen­dently. That pe­riod of rel­a­tive in­de­pen­dence is go­ing to slam shut when the Neo-Assyrian Empire — it­self a con­tin­u­a­tion of the Middle Assyrian Empire, re­cov­ered from LBAC — re­asserts it­self in the ninth cen­tury, dom­i­nat­ing the Levant and even Egypt.

But in the in­ter­ven­ing time a num­ber of dif­fer­ent smaller so­ci­eties have a chance to make their own way in the Levant, two of which are go­ing to leave a very large mark. In the north­ern Levant, this pe­riod of frag­men­ta­tion cre­ates space for the rise of the ma­jor Phoenician cen­ters — Byblos, Sidon and Tyre (of which the lat­ter will even­tu­ally be­come the most im­por­tant). As we’ve dis­cussed, those are go­ing to be the start­ing point for a wave of Phoenician col­o­niza­tion in the Mediterranean, as Phoenician traders steadily knit Mediterranean trade net­works (back) to­gether. They are also, as noted above, us­ing their own pho­netic script, the Phoenician al­pha­bet, which is in turn go­ing to form the ba­sic of many other re­gional scripts. Perhaps most rel­e­vant for us, the Greeks will adopt and mod­i­fy­ing the Phoenician al­pha­bet to rep­re­sent their own lan­guage and then peo­ples of pre-Ro­man Italy will adopt and mod­ify that to make the Old Italic al­pha­bet which in turn be­comes the Latin al­pha­bet which is the al­pha­bet in which I am typ­ing right now.

Meanwhile in the south­ern Levant this pe­riod of frag­men­ta­tion cre­ates the space for the emer­gence of two small king­doms whose peo­ple are de­vel­op­ing a very his­tor­i­cally im­por­tant re­li­gion cen­tered on the wor­ship of their God Yahweh. These are, of course, the king­doms of Israel and Judah. We are un­usu­ally well in­formed about the his­tory of these king­doms be­cause their his­tory was pre­served as part of Jewish scrip­ture, al­though ver­i­fy­ing el­e­ments of that scrip­ture as his­tor­i­cal fact is quite hard — schol­ars re­main di­vided, for in­stance, about the ex­is­tence of an ac­tual united monar­chy’ (in scrip­ture un­der Saul, David and Solomon) which would have ex­isted c. 1000 BC (by con­trast the later split king­doms are at­tested in Assyrian records). The de­vel­op­ment of these two king­doms — and thus the de­vel­op­ment of all of the Abrahamic faiths — is greatly in­flu­enced by this pe­riod of frag­men­ta­tion. Readers who know their Kings and Chronicles may have al­ready pieced to­gether that it is that re-ex­pan­sion of Assyrian power which will lead to the de­struc­tion of the north­ern king­dom of Israel in the 720s, while the south­ern king­dom of Judah per­sists as a quasi-de­pen­dency of Assyria be­fore be­ing dis­mem­bered and de­stroyed fi­nally by the Neo-Babylonian Empire (which re­places the Neo-Assyrian Empire, how­ever briefly) in 597 BC.

Of course the dif­fi­cult thing in all of this is that it is this ini­tial pe­riod, where a lot is clearly form­ing and brew­ing in the Eastern Mediterranean that our ev­i­dence is sig­nif­i­cantly weaker than we’d like (again, es­pe­cially in Greece, but note how much un­cer­tainty we have even in the Levant). The first few cen­turies of the Iron Age, im­me­di­ately fol­low­ing the Late Bronze Age Collapse are clearly a very im­por­tant for­ma­tive pe­riod which are go­ing to set some of the key pat­terns for events to play out in the rest of an­tiq­uity as the cur­tain goes up’ as it were and we start be­ing able to see those events clearly.

All that said, I have to stress this is re­ally a very ba­sic overview. I am doubt­less miss­ing out on some of the lat­est work in this field (because I am a late/​post Iron Age scholar) and in any case a lot of this can­not help but be a fairly ba­sic sum­mary. Perhaps one of these days I can get a Late Bronze Age or early Near Eastern Iron Age spe­cial­ist to guest-write some­thing more de­tailed on spe­cific facets of the col­lapse and its im­pact.

Sidon and Byblos, along­side Tyre (settled in the Bronze Age, but only promi­nent in the Iron Age) would be the most pow­er­ful and promi­nent Phoenician cities in the early Iron Age.

That is, speak­ers of the Doric di­alect of Greek

Reduced rain­fall in the Armenian high­lands could, of course, neg­a­tively ef­fect the Tigris and Euphrates, but that’s a less wa­ter’ prob­lem as op­posed to a no wa­ter’ prob­lem.

New York City becomes first in the US to ban deceptive subscription practices

www.theguardian.com

New York City has adopted a new rule that bans com­pa­nies from us­ing de­cep­tive sub­scrip­tions to trap cus­tomers into pay­ing for gym mem­ber­ships, stream­ing ser­vices and other re­cur­ring charges, the city’s con­sumer pro­tec­tion of­fice said.

The new rule, which will start on 1 October, promises hefty fines and ag­gres­sive en­force­ment for vi­o­la­tors. Companies that do not pro­vide a sim­ple way to can­cel could pay $525 per user sub­scrip­tion, back fees and ad­di­tional fines.

The city is also tar­get­ing so-called junk fees” that raise the fi­nal price of every­thing from apart­ments to sport­ing events, with a pro­posed rule that re­quires sell­ers to advertise the to­tal price for any good or ser­vice, in­clud­ing all manda­tory ad­di­tional charges and fees, up front”, ac­cord­ing to a re­lease shared with the Guardian.

New York would be the first US city to im­ple­ment such a ban.

People should­n’t have to wait on hold for half an hour or send a cer­ti­fied let­ter or show up to a store in per­son in or­der to can­cel” a sub­scrip­tion, said Samuel AA Levine, the city’s com­mis­sioner of con­sumer and worker pro­tec­tion, in an in­ter­view.

The new mea­sures were an­nounced in a press con­fer­ence on Friday.

The pro­posed fee rule could have an es­pe­cially wide ef­fect, send­ing rip­ples through New York’s ex­pen­sive hous­ing mar­ket, where about 70% of res­i­dents rent.

Apartment renters in the US face a ris­ing tide of add-on fees such as boiler man­age­ment” and lifestyle” charges from man­age­ment com­pa­nies, which make true rental costs hun­dreds of dol­lars higher than the price stated on real-es­tate com­pany web­sites.

If the pro­posed renters rule passes af­ter pub­lic com­ment and hear­ing, any manda­tory fees, in­clud­ing an­nual ones, would need to be in­cluded in the stated monthly rental price, Levine said.

The cur­rent sit­u­a­tion cre­ates a sce­nario where rather than com­pet­ing on price, com­pa­nies are com­pet­ing on their abil­ity to hide the true price. That’s the worst kind of in­cen­tive” — and one that deeply dis­torts the mar­ket, Levine said.

The moves are part of an ag­gres­sive push by Zohran Mamdani and Levine, a for­mer head of con­sumer pro­tec­tion in the Federal Trade Commission (FTC), to rein in what they see as preda­tory cor­po­rate mal­prac­tice na­tion­wide.

In the dawn of the [Ronald] Reagan era, the FTC and oth­ers in Washington said ex­pressly that … mar­kets could cor­rect them­selves, reg­u­late them­selves, they were go­ing to stop writ­ing rules,” and al­low com­pa­nies to po­lice their own be­hav­ior, Levine said. What it has got­ten us is 40 years of de­cep­tive pric­ing,” he said.

Bans on junk fees and sub­scrip­tion traps are gen­er­ally pop­u­lar with con­sumers, but have been fought ag­gres­sively by in­dus­try groups. When the Biden ad­min­is­tra­tion in­tro­duced a junk fee rule in 2024, the US Chamber of Commerce ar­gued it was an at­tempt to mi­cro­man­age busi­ness­es’ pric­ing struc­tures”, and apart­ment fees were cut from that fed­eral rule af­ter lob­by­ing by the real-es­tate in­dus­try.

A na­tional click-to-can­cel rule in­tro­duced by the Biden ad­min­is­tra­tion was struck down by a fed­eral judge in 2025, days be­fore it was set to go into ef­fect, over a pro­ce­dural rule. Donald Trump’s FTC plans to pass a sim­i­lar rule in com­ing months.

Companies make bil­lions a year in au­to­matic sub­scrip­tion re­newals that con­sumers do not want or do not know they have. The sub­scrip­tion rule could save New Yorkers alone as much as $162.5m per year, the Roosevelt Institute think­tank es­ti­mates.

While the sub­scrip­tion rule would only ap­ply to New York City res­i­dents, the pro­posed junk fee rule af­fects com­pa­nies such as ho­tels and rental car agen­cies that cater to vis­i­tors. If you are stay­ing in a ho­tel in the city that hits you with undis­closed fees upon check-in, you should com­plain to us”, Levine said.

The new rule is the Mamdani ad­min­is­tra­tion’s lat­est at­tempt to ad­dress the af­ford­abil­ity cri­sis af­ter heav­ily cam­paign­ing on mak­ing the city cheaper for res­i­dents. Members of Mamdani’s de­mo­c­ra­tic so­cial­ist group that were en­dorsed by the mayor won a flurry of pri­mary elec­tions in re­cent weeks, as some vot­ers em­brace left­wing pop­ulism that promises to em­power work­ing-class Americans, sim­i­lar to pledges by Trump in the past three pres­i­den­tial elec­tions.

The New York city coun­cil has also pro­posed a rule ban­ning surveillance pric­ing”, in which com­pa­nies charge con­sumers dif­fer­ent prices for the same good or ser­vice, based on al­go­rith­mic in­for­ma­tion from their spend­ing and other per­sonal habits.

Maryland banned the prac­tice in April. Colorado’s gov­er­nor ve­toed a ban last month.

The city will take pub­lic com­ments on the junk fee rule and then hold a hear­ing, Levine said. I cer­tainly hope that we can get this rule done by the end of the year.”

Neural-Guided Evolutionary Video Synthesis

nevo-project.epfl.ch

Apple Silicon Exec Explains Mac Mini AI Demand and On-Device Future

www.macrumors.com

Apple’s Mac mini and Mac Studio have be­come the ma­chines of choice for run­ning AI agents, ac­cord­ing to Doug Brooks, Apple’s se­nior prod­uct man­ager of Apple sil­i­con.

Brooks made the claim while dis­cussing Apple’s chip strat­egy in a newly pub­lished in­ter­view with The Deep View con­ducted just prior to WWDC 2026 in June.

Brooks says that the com­pany has seen incredible de­mand” for the two desk­top Macs. When it comes to agen­tic work­loads, people of­ten want a sys­tem that’s un­der their con­trol, iso­lated from their pri­mary ma­chine, and ca­pa­ble of run­ning 24 hours a day, seven days a week,” said Brooks.

A Mac mini is an amaz­ing sys­tem for that,” he added.

Many AI tools are also Mac-first or Mac-only, which Brooks says has helped ce­ment the Mac’s stand­ing among de­vel­op­ers, in­clud­ing those at fron­tier AI labs where Macs are said to be a com­mon sight.

The Apple ex­ec­u­tive also con­ceives of agen­tic AI as a whole-chip prob­lem rather than a GPU one. It’s not just about the GPU crunch­ing on an LLM any­more,” he said. It’s about the whole chip con­tribut­ing to dif­fer­ent parts of the task, tool-call­ing, and the things that are hap­pen­ing around those work­flows. It re­ally plays to the strengths of Apple sil­i­con.”

Brooks links Apple’s po­si­tion of strength in mod­ern AI back to chip de­ci­sions made long be­fore LLMs like ChatGPT ar­rived. He points to the Neural Engine, which is built for power-ef­fi­cient ma­trix math, along with lesser-known neural ac­cel­er­a­tors in­side the CPU that han­dle time-sen­si­tive tasks like speech.

Apple more re­cently added neural ac­cel­er­a­tors to the GPU, which has ex­tended AI per­for­mance across the board from iPhone-class parts up to the Mac’s largest sil­i­con. Brooks ties that progress to Apple’s de­sign method, where a chip is built for a spe­cific ma­chine, and the hard­ware and soft­ware are de­vel­oped in tan­dem.

He also de­scribed a shift to­ward run­ning AI lo­cally rather than in the cloud — a move mo­ti­vated by pri­vacy, se­cu­rity, and the ris­ing cost of in­fer­ence as agents con­sume more to­kens. However, Brooks en­vi­sions a hy­brid fu­ture in which agents de­cide what runs on-de­vice and what gets sent to the cloud.

He also sin­gled out what he calls transparent AI on iPhone and iPad, re­fer­ring to fea­tures scat­tered through­out the op­er­at­ing sys­tem and third-party apps that work qui­etly with­out an­nounc­ing them­selves as AI.

Some of the ex­am­ples he cited in­clude Draw Things, an im­age gen­er­a­tor that runs across iPhone, iPad, and Mac, and SwingVision, which an­a­lyzes ten­nis and pick­le­ball game­play in real time us­ing the iPhone’s cam­eras.

The speed of AI de­vel­op­ment right now is just crazy,” Brooks said. I can’t imag­ine where we’re go­ing to be a year from now, three months from now, or even a month from now,” he added.

You can read the full in­ter­view over on The Deep View web­site.

Popular Stories

Apple Just Increased Prices on MacBooks, iPads, and More

Apple to­day dra­mat­i­cally in­creased de­vice prices across mul­ti­ple prod­uct lines. Subscribe to the MacRumors YouTube chan­nel for more videos. After tem­porar­ily tak­ing it down ear­lier to­day, Apple’s on­line store is back up with a se­ries of prod­uct price in­creases. The changes are as fol­lows:

HomePod mini: $129, up from $99 (+$30) HomePod: $349, up from $299 (+$50) Apple TV: $199, up from…

Apple Hikes M4 Pro Mac Mini Starting Price Amid Rising Memory Costs

Thursday June 25, 2026 6:10 am PDT by Tim Hardwick

Apple to­day in­creased the start­ing price of the Mac mini with M4 Pro chip by $200, tak­ing the higher-tier model up to $1,599 on its on­line store. When the M4 Pro model launched in October 2024, the start­ing price was $1,399, but Apple has been hit by the rapid ex­pan­sion of AI data cen­tres, which has dri­ven up the de­mand for mem­ory and stor­age chips across the tech in­dus­try. Apple had…

Apple Silicon is Taking an Unexpected Turn

Ever since the Mac switched from Intel proces­sors to Apple sil­i­con start­ing in 2020, each gen­er­a­tion of M-series chips has in­cluded higher-end Pro and Max vari­ants. If a re­cent re­port proves to be ac­cu­rate, though, that streak will be com­ing to an end. According to Bloomberg’s Mark Gurman, Apple will be re­leas­ing a reg­u­lar M6 chip, but it has no plans to of­fer higher-end M6 Pro and M6 Max…

To add this web app to your iOS home screen tap the share button and select "Add to the Home Screen".

10HN is also available as an iOS App

If you visit 10HN only rarely, check out the the best articles from the past week.

Visit pancik.com for more.