To use the Mastodon web ap­pli­ca­tion, please en­able JavaScript. Alternatively, try one of the na­tive apps for Mastodon for your plat­form.

One of the cur­rent trends in mod­ern soft­ware is for de­vel­op­ers to slap an API call to OpenAI or Anthropic for fea­tures within their app. Reasonable peo­ple can quib­ble with whether those fea­tures are ac­tu­ally bring­ing value to users, but what I want to dis­cuss is the fun­da­men­tal con­cept of tak­ing on a de­pen­dency to a cloud hosted AI model for ap­pli­ca­tions.

This lazi­ness is cre­at­ing a gen­er­a­tion of soft­ware that is frag­ile, in­vades your pri­vacy, and fun­da­men­tally bro­ken. We are build­ing ap­pli­ca­tions that stop work­ing the mo­ment the server crashes or a credit card ex­pires.

We need to re­turn to a habit of build­ing soft­ware where our lo­cal de­vices do the work. The sil­i­con in our pocket is mind bog­glingly faster than what was avail­able a decade ago. It has a ded­i­cated Neural Engine sit­ting there, mostly idle, while we wait for a JSON re­sponse from a server farm in Virginia. That’s ridicu­lous.

Even if your in­ten­tions are pure, the mo­ment you stream user con­tent to a third party AI provider, you’ve changed the na­ture of your prod­uct. You now have data re­ten­tion ques­tions and all the bag­gage that comes with that (consent, au­dit, breach, gov­ern­ment re­quest, train­ing, etc.)

On top of that you also sub­stan­tially com­pli­cated your stack be­cause your fea­ture now de­pends on net­work con­di­tions, ex­ter­nal ven­dor up­time, rate lim­its, ac­count billing, and your own back­end health.

Congratulations! You took a UX fea­ture and turned it into a dis­trib­uted sys­tem that costs you money.

If the fea­ture can be done lo­cally, opt­ing into this mess is self in­flicted dam­age.

“AI every­where” is not the goal. Useful soft­ware is the goal.

Concrete Example: Brutalist Report’s On-Device Summaries

Years ago I launched a fun side pro­ject named The Brutalist Report , a news ag­gre­ga­tor ser­vice in­spired by the 1990s style web.

Recently, I de­cided to build a na­tive iOS client for it with the de­sign goal of en­sur­ing it would re­main a high-den­sity news read­ing ex­pe­ri­ence. Headlines in a stark list, a reader mode that strips the can­cer that has over­taken the web, and (optionally) an “intelligence” view that gen­er­ates a sum­mary of the ar­ti­cle.

Here’s the key point though: the sum­mary is gen­er­ated on-de­vice us­ing Apple’s lo­cal model APIs. No server de­tours. No prompt or user logs. No ven­dor ac­count. No “we store your con­tent for 30 days” foot­notes needed.

It has be­come so nor­mal for folks that any AI use is hap­pen­ing server-side. We have a lot of work to do to turn this around as an in­dus­try.

It’s not lost on me that some­times the use-cases you have will de­mand the in­tel­li­gence that only a cloud hosted model can pro­vide, but that’s not the case with every use-case you’re try­ing to solve. We need to be thought­ful here.

Available Tooling

I can only speak on the tool­ing avail­able within the Apple ecosys­tem since that’s what I fo­cused ini­tial de­vel­op­ment ef­forts on. In the last year, Apple has in­vested heav­ily here to al­low de­vel­op­ers to make use of a built-in lo­cal AI model eas­ily.

The core flow looks roughly like this:

im­port FoundationModels

let model = SystemLanguageModel.default guard model.avail­abil­ity == .available else { re­turn }

let ses­sion = LanguageModelSession { “”″ Provide a bru­tal­ist, in­for­ma­tion-dense sum­mary in Markdown for­mat. - Use **bold** for key con­cepts. - Use bul­let points for facts. - No fluff. Just facts. “”″ }

let re­sponse = try await ses­sion.re­spond(op­tions: .init(maximumResponseTokens: 1_000)) { ar­ti­cle­Text }

let mark­down = re­sponse.con­tent

And for longer con­tent, we can chunk the plain text (around 10k char­ac­ters per chunk), pro­duce con­cise “facts only” notes per chunk, then runs a sec­ond pass to com­bine them into a fi­nal sum­mary.

This is the kind of work lo­cal mod­els are per­fect for. The in­put data is al­ready on the de­vice (because the user is read­ing it). The out­put is light­weight. It’s fast and pri­vate. It’s okay if it’s not a su­per­hu­man PhD level in­tel­li­gence be­cause it’s sum­ma­riz­ing the page you just loaded, not in­vent­ing world knowl­edge.

Local AI shines when the mod­el’s job is trans­form­ing user-owned data, not act­ing as a search en­gine for the uni­verse.

There are plenty of AI fea­tures that peo­ple want but don’t trust. Summarizing emails, ex­tract ac­tion items from notes, cat­e­go­rize this doc­u­ment, etc.

The usual cloud ap­proach turns every one of those into a trust ex­er­cise. “Please send your data to our servers. We promise to be cool about it.”

Local AI changes that. Your de­vice al­ready has the data. We’ll do the work right here.

You don’t build trust with your users by writ­ing a 2,000 word pri­vacy pol­icy. You build trust by not need­ing one to be­gin with.

The tool­ing avail­able on the plat­form goes even fur­ther.

One of the best moves Apple has made re­cently is push­ing “AI out­put” away from un­struc­tured blobs of text and to­ward typed data.

Instead of “ask the model for JSON and pray”, the newer and bet­ter pat­tern is to de­fine a Swift struct that rep­re­sents the thing you want. Give the model guid­ance for each field in nat­ural lan­guage. Ask the model to gen­er­ate an in­stance of that type.

That’s it.

Conceptually, it looks like this:

im­port FoundationModels

@Generable struct ArticleIntel { @Guide(description: “One sen­tence. No hype.“) var tldr: String @Guide(description: “3 – 7 bul­lets. Facts only.“) var bul­lets: [String] @Guide(description: “Comma-separated key­words.“) var key­words: [String] }

let ses­sion = LanguageModelSession() let re­sponse = try await ses­sion.re­spond( to: “Extract struc­tured notes from the ar­ti­cle.”, gen­er­at­ing: ArticleIntel.self ) { ar­ti­cle­Text }

let in­tel = re­sponse.con­tent

Now your UI does­n’t have to scrape bul­let points out of Markdown or hope the model re­mem­bered your JSON schema. You get a real type with real fields, and you can ren­der it con­sis­tently. It pro­duces struc­tured out­put your app can ac­tu­ally use. And it’s all run­ning lo­cally!

This is­n’t just nicer er­gonom­ics. It’s an en­gi­neer­ing im­prove­ment.

And if you’re build­ing a lo­cal first app, this is the dif­fer­ence be­tween “AI as nov­elty” and “AI as a trust­wor­thy sub­sys­tem”.

“But Local Models Aren’t As Smart”

Correct.

But also so what?

Most app fea­tures don’t need a model that can write Shakespeare, ex­plain quan­tum me­chan­ics, and pass the bar exam. They need a model that can do one of these re­li­ably: sum­ma­rize, clas­sify, ex­tract, rewrite, or nor­mal­ize.

And for those tasks, lo­cal mod­els can be truly ex­cel­lent.

If you try to use a lo­cal model as a re­place­ment for the en­tire in­ter­net, you will be dis­ap­pointed. If you use it as a “data trans­former” sit­ting in­side your app, you’ll won­der why you ever sent this stuff to a server.

Use cloud mod­els only when they’re gen­uinely nec­es­sary. Keep the user’s data where it be­longs. And when you do use AI, don’t just glue it as a chat box. Use it as a real sub­sys­tem with typed out­puts and pre­dictable be­hav­ior.

Stop ship­ping dis­trib­uted sys­tems when you meant to ship a fea­ture.

Last year I said I’d prob­a­bly never rec­om­mend an­other Bambu Lab printer again.

I still use my P1S, but af­ter Bambu Lab started push­ing their al­ways-con­nected cloud so­lu­tion as the new de­fault:

I blocked the printer from the Internet via my OPNsense Firewall

I stopped up­dat­ing the firmware

I locked the printer into Developer mode

I deleted Bambu Studio and started us­ing OrcaSlicer

I had to do that to keep it un­der my con­trol, in­stead of Bambu’s.

But I’m weird—I ac­knowl­edge that. I’m one of those crazy ones who likes to own some­thing they pur­chased, and not have the com­pany watch every­thing I do with hard­ware I paid for.

Bambu Lab could’ve left the sta­tus quo at that, and I would­n’t be writ­ing this blog post.

But they did­n’t.

What hap­pened this time?

For con­text: OrcaSlicer is a fork of the open source pro­ject Bambu Studio, which is a fork of Prusa Slicer, which is a fork of slic3r. (They are all li­censed un­der the AGPLv3 open source li­cense).

OrcaSlicer al­ready has to dance around Bambu’s weird de­fault setup where every file you print goes through Bambu’s servers, mean­ing they can see every­thing you ever print on your printer.

That is, un­less you’re like me and you run it in Developer mode, and com­pletely block it from the Internet on old firmware.

Some peo­ple are okay with us­ing OrcaSlicer and print­ing through Bambu’s cloud. It’s con­ve­nient if you’re on the road and want to start a print on your printer at home, with­out man­ag­ing your own VPN.

I run my own WireGuard VPN, so I don’t need that, but I un­der­stand not every­one has the re­sources to man­age their own re­mote ac­cess.

Bambu saw a fork of OrcaSlicer that al­lowed you to use all your print­er’s fea­tures with­out hav­ing to route prints through Bambu’s cloud called OrcaSlicer-bambulab and was like, “You know what? No. For the 0.1% of power users who want to run OrcaSlicer with­out the cloud de­liv­ery mech­a­nism like we have in our AGPL-licensed Linux Bambu Studio code… no. You have to use our app, and only our app.”

So they threat­ened that OrcaSlicer fork’s de­vel­oper with le­gal ac­tion for things that de­vel­oper did­n’t do. For ex­am­ple, they in­di­cated the fork used an im­per­son­ation at­tack, de­spite the fork us­ing Bambu Studio’s up­stream code ver­ba­tim.

These are very se­ri­ous pub­lic ac­cu­sa­tions.Bambu Lab did not write to me with these spe­cific pub­lic claims first. They also re­fused my re­quest to pub­lish the full cor­re­spon­dence. Instead, they pub­lished a one-sided pub­lic state­ment where I can­not re­ply di­rectly.In prac­tice, this pre­sents me to the pub­lic as some­one by­pass­ing se­cu­rity, im­per­son­at­ing their client, and cre­at­ing a risk to their in­fra­struc­ture. I re­ject that char­ac­ter­i­za­tion.— OrcaSlicer-bambulabs de­vel­op­er’s re­sponse

These are very se­ri­ous pub­lic ac­cu­sa­tions.

Bambu Lab did not write to me with these spe­cific pub­lic claims first. They also re­fused my re­quest to pub­lish the full cor­re­spon­dence. Instead, they pub­lished a one-sided pub­lic state­ment where I can­not re­ply di­rectly.

In prac­tice, this pre­sents me to the pub­lic as some­one by­pass­ing se­cu­rity, im­per­son­at­ing their client, and cre­at­ing a risk to their in­fra­struc­ture. I re­ject that char­ac­ter­i­za­tion.

— OrcaSlicer-bambulabs de­vel­op­er’s re­sponse

Bambu is abus­ing the open source so­cial con­tract, and us­ing their le­gal might, to sup­press a tiny num­ber of their users1, for who knows what rea­son.

It seems dumb to me, be­cause it would’ve been eas­ier (and more prof­itable) to do noth­ing at all2. Instead, they wrote a blog post blam­ing an in­di­vid­ual open source de­vel­oper for their own in­fra­struc­ture and se­cu­rity prob­lems.

This is where the ac­tual is­sue arises: the mod­i­fi­ca­tion in ques­tion worked by in­ject­ing fal­si­fied iden­tity meta­data into net­work com­mu­ni­ca­tion.In sim­ple terms: it pre­tended to be the of­fi­cial Bambu Studio client when com­mu­ni­cat­ing with our servers.— Bambu Lab blog post

This is where the ac­tual is­sue arises: the mod­i­fi­ca­tion in ques­tion worked by in­ject­ing fal­si­fied iden­tity meta­data into net­work com­mu­ni­ca­tion.

In sim­ple terms: it pre­tended to be the of­fi­cial Bambu Studio client when com­mu­ni­cat­ing with our servers.

— Bambu Lab blog post

I don’t think they un­der­stand open source cul­ture. Security ei­ther, if a pub­lic user agent string is their only pro­tec­tion against DDoS at­tacks…

Instead of find­ing so­lu­tions to ecosys­tem prob­lems and build­ing a more se­cure plat­form, Bambu is putting de­voted power users like the fork’s de­vel­oper on blast3.

When ten­sions flared last year, they wrote a sim­i­lar blog post blam­ing com­mu­nity back­lash on ‘unfortunate mis­in­for­ma­tion’. I imag­ine they meant spec­u­la­tion from com­mu­nity mem­bers (like my­self) frus­trated the whole soft­ware ecosys­tem and own­er­ship model was turned up­side down post-pur­chase.

This year they’re blam­ing one de­vel­oper of a tiny slicer fork for the po­ten­tial im­pact he could have on their en­tire cloud in­fra­struc­ture.

It cre­ates struc­tural vul­ner­a­bil­ity. If this method were widely adopted or in­cor­rectly con­fig­ured, thou­sands of clients could si­mul­ta­ne­ously hit our servers while im­per­son­at­ing the of­fi­cial client. Our sys­tems would have no way to dis­tin­guish traf­fic, be­cause the re­quests would look iden­ti­cal.— Bambu Lab blog post

It cre­ates struc­tural vul­ner­a­bil­ity. If this method were widely adopted or in­cor­rectly con­fig­ured, thou­sands of clients could si­mul­ta­ne­ously hit our servers while im­per­son­at­ing the of­fi­cial client. Our sys­tems would have no way to dis­tin­guish traf­fic, be­cause the re­quests would look iden­ti­cal.

— Bambu Lab blog post

I love how they frame this as a de­vel­oper try­ing to im­per­son­ate their app, when he’s lit­er­ally us­ing the same AGPL-licensed code their Linux app uses.

I find it dou­bly ironic since their own fork caused Bambu users’ teleme­try to hit Prusa’s servers back in 2022, and (to my knowl­edge) Prusa did­n’t snap back with a C&D.

They spent the rest of their blog post talk­ing about vul­ner­a­bil­i­ties, bugs, and in­sta­bil­i­ties—as if that has any­thing to do with a de­vel­oper us­ing up­stream code ver­ba­tim in his fork.

Maybe they could take a new ap­proach and just not lock down their whole ecosys­tem in the first place.

But who am I kid­ding? Nothing I say, and no amount of com­plain­ing in the com­ments be­low, seems to help Bambu see the fault in their ways.

Spending a lit­tle more for a printer from an­other com­pany just might do it, though.

Louis Rossmann posted a video say­ing he’d pledge $10,000 to help the open source dev fight Bambu’s le­gal threats. And I’d hap­pily chip in too, but that’s only use­ful if the dev wants to put him­self back in Bambu’s crosshairs.

The bet­ter play might just be to skip Bambu al­to­gether.

The OrcaSlicer fork in ques­tion did­n’t seem to have much up­take out­side of a very small sub­set of users prior to Bambu Lab’s cease and de­sist or­der. ↩︎

The OrcaSlicer fork in ques­tion did­n’t seem to have much up­take out­side of a very small sub­set of users prior to Bambu Lab’s cease and de­sist or­der. ↩︎

Maybe ask for the fork to not in­clude “bambulabs” in the name, since that could be a rea­son­able trade­mark-re­lated de­mand. ↩︎

Maybe ask for the fork to not in­clude “bambulabs” in the name, since that could be a rea­son­able trade­mark-re­lated de­mand. ↩︎

The fork’s de­vel­oper men­tioned “I pre­vi­ously helped Bambu Studio users with Linux and Wayland is­sues, in­clud­ing on Bambu Lab’s own GitHub. That makes it es­pe­cially ab­surd to me that I am now be­ing pub­licly pre­sented as some­one dan­ger­ous to their in­fra­struc­ture.” ↩︎

The fork’s de­vel­oper men­tioned “I pre­vi­ously helped Bambu Studio users with Linux and Wayland is­sues, in­clud­ing on Bambu Lab’s own GitHub. That makes it es­pe­cially ab­surd to me that I am now be­ing pub­licly pre­sented as some­one dan­ger­ous to their in­fra­struc­ture.” ↩︎

by Tanner Linsley on May 11, 2026.

Last up­dated: 2026 – 05-11

On 2026 – 05-11 be­tween 19:20 and 19:26 UTC, an at­tacker pub­lished 84 ma­li­cious ver­sions across 42 @tanstack/* npm pack­ages by com­bin­ing: the pul­l_re­quest_­tar­get “Pwn Request” pat­tern, GitHub Actions cache poi­son­ing across the fork↔base trust bound­ary, and run­time mem­ory ex­trac­tion of an OIDC to­ken from the GitHub Actions run­ner process. No npm to­kens were stolen and the npm pub­lish work­flow it­self was not com­pro­mised.

The ma­li­cious ver­sions were de­tected pub­licly within 20 min­utes by an ex­ter­nal re­searcher ashishkurmi work­ing for stepse­cu­rity. All af­fected ver­sions have been dep­re­cated; npm se­cu­rity has been en­gaged to pull tar­balls from the reg­istry. We have no ev­i­dence of npm cre­den­tials be­ing stolen, but we strongly rec­om­mend that any­one who in­stalled an af­fected ver­sion on 2026 – 05-11 ro­tate AWS, GCP, Kubernetes, Vault, GitHub, npm, and SSH cre­den­tials reach­able from the in­stall host.

Tracking is­sue: TanStack/router#7383 GitHub Security Advisory: GHSA-g7cv-rxg3-hmpx

Packages af­fected

42 pack­ages, 84 ver­sions (two per pack­age, pub­lished roughly 6 min­utes apart). See the track­ing is­sue for the full table. Confirmed-clean fam­i­lies: @tanstack/query*, @tanstack/table*, @tanstack/form*, @tanstack/virtual*, @tanstack/store, @tanstack/start (the meta-pack­age, not @tanstack/start-*).

What the mal­ware does

When a de­vel­oper or CI en­vi­ron­ment runs npm in­stall, pnpm in­stall, or yarn in­stall against any af­fected ver­sion, npm re­solves the ma­li­cious op­tion­alDe­pen­den­cies en­try, fetches the or­phan pay­load com­mit from the fork net­work, runs its pre­pare life­cy­cle script, and ex­e­cutes a ~2.3 MB ob­fus­cated router_init.js smug­gled into the af­fected tar­ball. The script:

Harvests cre­den­tials from com­mon lo­ca­tions: AWS IMDS / Secrets Manager, GCP meta­data, Kubernetes ser­vice-ac­count to­kens, Vault to­kens, ~/.npmrc, GitHub to­kens (env, gh CLI, .git-credentials), SSH pri­vate keys

Exfiltrates over the Session/Oxen mes­sen­ger file-up­load net­work (filev2.getsession.org, seed{1,2,3}.get­ses­sion.org) — end-to-end en­crypted with no at­tacker-con­trolled C2, so block­ing by IP/domain is the only net­work mit­i­ga­tion

Self-propagates: enu­mer­ates other pack­ages the vic­tim main­tains via reg­istry.npmjs.org/-/​v1/​search?text=main­tainer:<user> and re­pub­lishes them with the same in­jec­tion

Because the pay­load runs as part of npm in­stal­l’s life­cy­cle, any­one who in­stalled an af­fected ver­sion on 2026 – 05-11 must treat the in­stall host as po­ten­tially com­pro­mised.

All times UTC. Local time­stamps from GitHub API and npm reg­istry.

Pre-attack (cache poi­son­ing phase)

Detonation (publish phase)

Workflow run 25613093674 starts (19:15:44), and fails.

Detection and re­sponse

Formal mal­ware re­ports are sub­mit­ted via npm

Tanner be­gins npm dep­re­ca­tion process for all 84 af­fected pack­ages.

Public Twitter/X/LinkedIn/Bluesky dis­clo­sure from @tan_stack and main­tain­ers

All cache en­tries for all TanStack/* GitHub repos­i­to­ries purged via API.

Hardening PR merged: bun­dle-size.yml re­struc­tured, repos­i­to­ry_owner guards added, third-party ac­tion refs pinned to SHAs.

Official GitHub Security Advisory is pub­lished, CVE re­quested

Three vul­ner­a­bil­i­ties chained to­gether. Each is nec­es­sary for the at­tack; none alone is suf­fi­cient.

1. pul­l_re­quest_­tar­get “Pwn Request” pat­tern in bun­dle-size.yml

bun­dle-size.yml ran pul­l_re­quest_­tar­get for fork PRs and, in­side that trig­ger con­text, checked out the fork’s PR-merge ref and ran a build:

yaml

on: pul­l_re­quest_­tar­get: paths: [‘packages/**’, ‘benchmarks/**’]

jobs: bench­mark-pr: steps: - uses: ac­tions/​check­out@v6.0.2 with: ref: refs/​pull/${{ github.event.pul­l_re­quest.num­ber }}/merge # fork’s merged code

- uses: TanStack/config/.github/setup@main # tran­si­tively calls ac­tions/​cache@v5

- run: pnpm nx run @benchmarks/bundle-size:build # ex­e­cutes fork-con­trolled code

on: pul­l_re­quest_­tar­get: paths: [‘packages/**’, ‘benchmarks/**’]

jobs: bench­mark-pr: steps: - uses: ac­tions/​check­out@v6.0.2 with: ref: refs/​pull/${{ github.event.pul­l_re­quest.num­ber }}/merge # fork’s merged code

- uses: TanStack/config/.github/setup@main # tran­si­tively calls ac­tions/​cache@v5

- run: pnpm nx run @benchmarks/bundle-size:build # ex­e­cutes fork-con­trolled code

The au­thor of the work­flow at­tempted a trust split (the com­ment-pr job is sep­a­rate from bench­mark-pr, with a com­ment in the YAML not­ing the in­tent to keep bench­mark-pr “untrusted with read-only per­mis­sions”). The split is cor­rect in spirit but missed two facts:

ac­tions/​cache@v5′s post-job save is not gated by per­mis­sions:. Cache writes use a run­ner-in­ter­nal to­ken, not the work­flow GITHUB_TOKEN. Setting per­mis­sions: con­tents: read does not block cache mu­ta­tion.

Cache scope is per-repo, shared across pul­l_re­quest_­tar­get runs (which use the base re­po’s cache scope) and pushes to main. A PR run­ning in the base re­po’s cache scope can poi­son en­tries that pro­duc­tion work­flows on main will later re­store.

2. GitHub Actions cache poi­son­ing across trust bound­aries

The ma­li­cious vite_setup.mjs was specif­i­cally de­signed to write data into the pnpm-store di­rec­tory un­der a key the le­git re­lease.yml work­flow would com­pute and look up: Linux-pnpm-store-${hashFiles(‘**/pnpm-lock.yaml’)}. When the bench­mark-pr job ended, ac­tions/​cache@v5′s post-step saved the (now-poisoned) pnpm store to that ex­act key. When re­lease.yml next ran on a push to main, its Setup Tools step re­stored the poi­soned en­try — en­tirely as de­signed.

This is the class of at­tack doc­u­mented by Adnan Khan in 2024. It’s not a TanStack-specific bug; it’s a known GitHub Actions de­sign is­sue that re­quires con­scious mit­i­ga­tion.

re­lease.yml de­clares id-to­ken: write (legitimately needed for npm OIDC trusted pub­lish­ing). When the poi­soned pnpm store is re­stored on the run­ner, at­tacker-con­trolled bi­na­ries are now on disk and get in­voked dur­ing the build step. Those bi­na­ries:

Locate the GitHub Actions Runner.Worker process via /proc/*/cmdline

Read /proc/<pid>/maps and /proc/<pid>/mem to dump the work­er’s mem­ory

Extract the OIDC to­ken (which the run­ner mints lazily, in mem­ory, when id-to­ken: write is set)

Use the to­ken to au­then­ti­cate POST re­quests di­rectly to reg­istry.npmjs.org — by­pass­ing the work­flow’s Publish Packages step en­tirely

This is the same mem­ory-ex­trac­tion tech­nique (and ver­ba­tim Python script, with at­tri­bu­tion com­ment) used in the tj-ac­tions/​changed-files com­pro­mise of March 2025. The at­tacker did not in­vent novel trade­craft; they re­com­bined pub­lished re­search.

Why none alone is enough

pul­l_re­quest_­tar­get alone is fine for trusted op­er­a­tions (labeling, com­ments)

Cache poi­son­ing alone (e.g., from in­side an al­ready-com­pro­mised dep) re­quires a sep­a­rate pub­lish ve­hi­cle

OIDC to­ken ex­trac­tion alone re­quires ex­ist­ing code ex­e­cu­tion on the run­ner

The chain only works be­cause each vul­ner­a­bil­ity bridges the trust bound­ary the oth­ers as­sumed: PR fork code cross­ing into base-repo cache, base-repo cache cross­ing into re­lease-work­flow run­time, and re­lease-work­flow run­time cross­ing into npm reg­istry write ac­cess.

How we found out

Detection was ex­ter­nal. External re­searcher ashishkurmi work­ing for StepSecurity opened is­sue #7383 ~20 min­utes af­ter the pub­lish, with full tech­ni­cal analy­sis. Tanner re­ceived a phone call from Socket.dev just mo­ments af­ter start­ing the war room con­firm­ing the sit­u­a­tion.

IOC fin­ger­prints (for down­stream main­tain­ers and se­cu­rity tools)

In any @tanstack/* pack­age’s man­i­fest:

json

“optionalDependencies”: { “@tanstack/setup”: “github:tanstack/router#79ac49eedf774dd4b0cfa308722bc463cfe5885c” }

“optionalDependencies”: { “@tanstack/setup”: “github:tanstack/router#79ac49eedf774dd4b0cfa308722bc463cfe5885c” }

File: router_init.js (~2.3 MB, pack­age root, not in “files”)

Cache key: Linux-pnpm-store-6f9233a50def742c09fde54f56553d6b449a535adf87d4083690539f49ae4da11

2nd-stage pay­load URLs: https://​lit­ter.cat­box.moe/​h8nc9u.js, https://​lit­ter.cat­box.moe/​7r­rc6l.mjs

Exfiltration net­work: filev2.get­ses­sion.org, seed{1,2,3}.get­ses­sion.org

Forged com­mit iden­tity: claude <claude@users.noreply.github.com> (note: not the real Anthropic Claude — fab­ri­cated GitHub no-re­ply email)

Real at­tacker ac­counts: zblgg (id 127806521), voicpro­ducoes (id 269549300)

Attacker fork: github.com/​zblgg/​con­fig­u­ra­tion (fork of TanStack/router re­named to evade fork searches)

Orphan pay­load com­mit (in fork net­work): 79ac49eedf774dd4b0cfa308722bc463cfe5885c

Workflow runs that per­formed the ma­li­cious pub­lishes:

github.com/​TanStack/​router/​ac­tions/​runs/​25613093674 (attempt 4) github.com/​TanStack/​router/​ac­tions/​runs/​25691781302

github.com/​TanStack/​router/​ac­tions/​runs/​25613093674 (attempt 4)

github.com/​TanStack/​router/​ac­tions/​runs/​25691781302

What went well

External re­searchers no­ticed and re­ported with full tech­ni­cal de­tail within ~20 min of the in­ci­dent

Maintainer team co­or­di­nated im­me­di­ately and ef­fec­tively across many time­zones

The de­tec­tion com­mu­nity al­ready had a clear pub­lic IOC pat­tern within hours

What could have been bet­ter

No in­ter­nal alert­ing. We learned about the com­pro­mise from a third party. We need mon­i­tor­ing on our own pub­lishes. We’ll be work­ing closely with se­cu­rity re­searcher firms in the ecosys­tem that have the abil­ity to de­tect these is­sues very quickly, po­ten­tially even in-house, and mak­ing the feed­back loop even tighter.

pul­l_re­quest_­tar­get work­flows had not been au­dited de­spite be­ing a long-known dan­ger­ous pat­tern

Floating refs (@v6.0.2, @main) on third-party ac­tions cre­ate stand­ing sup­ply-chain risk in­de­pen­dent of this in­ci­dent

Unpublish was un­avail­able for nearly all af­fected pack­ages be­cause of npm’s “no un­pub­lish if de­pen­dents ex­ist” pol­icy. We have to rely on npm se­cu­rity to pull tar­balls server-side, which adds hours of de­lay dur­ing which ma­li­cious tar­balls re­main in­stal­lable

The 7-maintainer list on the npm scope means seven sep­a­rate cre­den­tial-theft tar­gets for the same blast ra­dius

OIDC trusted-pub­lisher bind­ing has no per-pub­lish re­view. Once con­fig­ured, any code path in the work­flow can mint a pub­lish-ca­pa­ble to­ken. We need ei­ther (a) move to short-lived clas­sic to­kens with man­ual re­view, or (b) add prove­nance-source-ver­i­fi­ca­tion to de­tect pub­lishes from un­ex­pected work­flow steps

What we got lucky on

The at­tacker chose a pay­load that broke tests, which made the pub­lish step (which would have pro­duced cleaner-look­ing tar­balls) skip — mean­ing the at­tack was loud enough to de­tect quickly. A more care­ful at­tacker who did­n’t break tests could have pub­lished silently for hours longer

The at­tacker reused pub­lic trade­craft (verbatim mem­ory-dump script with at­tri­bu­tion com­ment) in­stead of writ­ing novel code — mak­ing the IOC-matching faster

These need an­swers be­fore we close the post­mortem.

Did bun­dle-size.ym­l’s Setup Tools step ac­tu­ally call ac­tions/​cache@v5? Verify by read­ing the post-job logs from one of the pul­l_re­quest_­tar­get runs against PR #7378 (e.g., run id 25666610798). Tanner has ac­cess; needs to be done man­u­ally

What was in the ini­tial PR head com­mit (before the force-pushes wiped it)? GitHub’s re­flog may have it. Check via gh api or the GitHub sup­port team

How did the ma­li­cious com­mit get into the fork’s git ob­ject store specif­i­cally — was it pushed di­rectly via git, or was it cre­ated via the GitHub web UI (which would leave au­dit-log en­tries)?

Was voicpro­ducoes a real ac­count or a sock pup­pet? Cross-reference its ac­tiv­ity his­tory

Did the npm cache also get poi­soned (the 6 du­pli­cate linux-npm-store-* en­tries)? Were any ac­tu­ally used?

Can we iden­tify any other fork in the TanStack/router fork net­work that con­tains the or­phan pay­load com­mit? (If yes, the cleanup is harder — every fork host­ing it keeps it ac­ces­si­ble via github:tanstack/​router#79ac49ee…)

Are any other TanStack re­pos (router, query, table, form, vir­tual, etc.) us­ing the same bun­dle-size.yml-style pat­tern? Audit needed

How many users ac­tu­ally down­loaded the af­fected ver­sions dur­ing the pub­lish win­dow? Get from npm sup­port

Did any of the seven listed main­tain­ers’ ma­chines get com­pro­mised sep­a­rately? (None of the ma­li­cious pub­lishes used a main­tain­er’s npm to­ken, but main­tainer ma­chines could have been the sec­ondary tar­get via the self-prop­a­ga­tion logic)

Tracking is­sue: TanStack/router#7383

GitHub Security Advisory: GHSA-g7cv-rxg3-hmpx

Related re­search:

Adnan Khan, “The Monsters in Your Build Cache: Github Actions Cache Poisoning” (May 2024) — ad­nan­thekhan.com GitHub Security Lab, “Keeping your GitHub Actions and work­flows se­cure: Preventing pwn re­quests” — se­cu­rity­lab.github.com StepSecurity, “Harden-Runner de­tec­tion: tj-ac­tions/​changed-files ac­tion is com­pro­mised” (March 2025) — stepse­cu­rity.io

Skip to con­tent

Secure your code as you build

We read every piece of feed­back, and take your in­put very se­ri­ously.

Include my email ad­dress so I can be con­tacted

Use saved searches to fil­ter your re­sults more quickly

To see all avail­able qual­i­fiers, see our doc­u­men­ta­tion.

Sign up

You signed in with an­other tab or win­dow. Reload to re­fresh your ses­sion.

You signed out in an­other tab or win­dow. Reload to re­fresh your ses­sion.

You switched ac­counts on an­other tab or win­dow. Reload to re­fresh your ses­sion.

There was an er­ror while load­ing. Please re­load this page.

Notifications

You must be signed in to change no­ti­fi­ca­tion set­tings

You can’t per­form that ac­tion at this time.

May 13th, 2026 | 14 minute read

Modern cars are com­put­ers on wheels - they have more sen­sors than you can count and are con­stantly phon­ing home with teleme­try data like your lo­ca­tion, speed, fuel lev­els, sud­den ac­cel­er­a­tions/​de­cel­er­a­tions, video footage, dri­ver at­ten­tion data from eye mon­i­tor­ing sys­tems, and hun­dreds of other data points. Cars have in­ward- and out­ward-fac­ing cam­eras. They have mi­cro­phones. They have al­ways-on modems. It’s all en­abled by de­fault with dif­fi­cult or mean­ing­less opt-outs, and your data is mon­e­tized through bro­kers like LexisNexis or Verisk. This all brings a host of se­cu­rity and pri­vacy is­sues - here are some over the years:

In 2025 Subaru had vul­ner­a­bil­i­ties al­low­ing any­one to re­motely un­lock cus­tomers’ cars, as well as ac­cess the real-time GPS lo­ca­tion and lo­ca­tion his­tory of the car of the car

Car man­u­fac­tur­ers share your dri­ving data with in­sur­ance com­pa­nies, which then in­crease your pre­mi­ums

In 2023 Tesla em­ploy­ees in­ter­nally shared cam­era footage of naked cus­tomers and other sen­si­tive im­ages

In 2015 Charlie Miller and Chris Valasek fa­mously took over a Jeep Cherokee with full con­trol of the ig­ni­tion, brakes, locks, steer­ing, etc.

Mozilla de­tailed how 25 car man­u­fac­tur­ers scored abysmally on pri­vacy and how they col­lect data in­clud­ing “sexual ac­tiv­ity, im­mi­gra­tion sta­tus, race, fa­cial ex­pres­sions, weight and ge­netic in­for­ma­tion.” They sell this data to third par­ties and use it to build pro­files about you cov­er­ing “intelligence, abil­i­ties, char­ac­ter­is­tics, pref­er­ences, and more.”

Tesla had a vul­ner­a­bil­ity in 2017 that al­lowed any­one to re­motely see your car’s lo­ca­tion, man­age other fea­tures, and even sum­mon the car to them­selves

The Car That Watches You Back de­tails how cars are now serv­ing you ads, as well as col­lect­ing vast amounts of data about you. The Hacker News dis­cus­sion about this ar­ti­cle is what prompted this blog post

Now that we’re suf­fi­ciently mo­ti­vated, what can we do about it? In this blog post, rather than re­ly­ing on com­pa­nies’ promises or mean­ing­less opt-outs, we’re go­ing to stop the data at the source by phys­i­cally re­mov­ing the mo­dem (the DCM, or Data Communication Module) as well as the built-in GPS on my 2024 RAV4 Hybrid, so the car will no longer have the ca­pa­bil­ity to send any teleme­try data back home. Let’s dive in:

Will the car still be func­tional?

Yes. Depending on how dif­fer­ent car man­u­fac­tur­ers have wired their cars, how their soft­ware and firmware were writ­ten, etc., vary­ing lev­els of func­tion­al­ity might be af­fected by re­mov­ing the mo­dem and GPS. For this car:

Everything that re­lies on a data con­nec­tion will no longer work. This in­cludes things like over-the-air up­dates as well as Toyota cloud-based ser­vices and SOS func­tion­al­i­tyThis is a safety trade­off - you’re dis­abling au­to­matic crash no­ti­fi­ca­tion and emer­gency call­ing

This is a safety trade­off - you’re dis­abling au­to­matic crash no­ti­fi­ca­tion and emer­gency call­ing

The car’s mi­cro­phone is wired through the DCM, and in the ab­sence of any other changes re­mov­ing the DCM means the in-car mi­cro­phone won’t work, which is in­con­ve­nient if you plan on tak­ing calls in the car. However we’ll in­stall a DCM Bypass Kit (discussed more be­low) to re­store all func­tion­al­ity and have a work­ing mi­cro­phone

CarPlay has a quirk: the phone uses its own GPS but also ac­cepts a lo­ca­tion sig­nal from the car’s GPS unit. After re­mov­ing the DCM, the car would get con­fused about its lo­ca­tion and some­times jump my po­si­tion to the mid­dle of Nevada (I live in San Francisco), mak­ing nav­i­ga­tion an­noy­ing. To work around this we’ll fully dis­con­nect the car’s GPS, so it can’t send a bad lo­ca­tion to the phone­From the ti­tle of the blog post you might have won­dered why bother re­mov­ing the GPS af­ter we’ve re­moved the mo­dem - who cares if the car has built-in lo­ca­tion when it can’t phone home with that data? This is whyThis is a well-doc­u­mented bug with dis­cus­sions on Apple Support threads as well as car-spe­cific fo­rums like rav4­world. This bug af­fects more than just Toyotas, it’s a generic Apple bug even for peo­ple who haven’t re­moved their mo­dem (but anec­do­tally re­mov­ing my mo­dem made the prob­lem worse)

From the ti­tle of the blog post you might have won­dered why bother re­mov­ing the GPS af­ter we’ve re­moved the mo­dem - who cares if the car has built-in lo­ca­tion when it can’t phone home with that data? This is why

This is a well-doc­u­mented bug with dis­cus­sions on Apple Support threads as well as car-spe­cific fo­rums like rav4­world. This bug af­fects more than just Toyotas, it’s a generic Apple bug even for peo­ple who haven’t re­moved their mo­dem (but anec­do­tally re­mov­ing my mo­dem made the prob­lem worse)

Removing the DCM and GPS may void parts of your war­ranty - just some­thing to be aware of. Thanks to the Magnuson–Moss Warranty Act, it can­not void the whole car war­ranty. It can void cov­er­age re­lated to the work you did (cloud ser­vices, telem­at­ics, etc.) but un­re­lated fail­ures like en­gine prob­lems must still be cov­ered

So thank­fully every­thing in the car re­mains 100% func­tional ex­cept the cloud-based ser­vices men­tioned above, which I did­n’t want any­way. There is also one crit­i­cal caveat about Bluetooth:

No more Bluetooth

Important: Even af­ter the mo­dem is re­moved, if you con­nect your phone to the car via Bluetooth then the car will use your phone as an in­ter­net con­nec­tion and send all the same teleme­try data back to Toyota. However, if you use a wired USB con­nec­tion then it does not do that (see the dis­cus­sion here and else­where), so I ex­clu­sively use CarPlay via USB. I wish I had a way to com­pletely dis­able the car’s Bluetooth func­tion­al­ity, but it’s deeply in­te­grated into the head unit.

If you need USB ca­bles for CarPlay I like these USB-A to Lightning and USB-A to USB-C ca­bles from Anker.

Or, if you pre­fer the con­ve­nience of Bluetooth, you can use a Bluetooth -> wired USB adapter like this one. The adapter re­ceives Bluetooth from your phone and pre­sents it­self to the car as a USB de­vice, so the car treats it like a wired con­nec­tion and won’t tether through your phone.

Now, onto the nec­es­sary tools and parts:

Tools/parts needed

For this pro­ject you’ll need:

A trim re­moval kit (I used this one)

A ratchet, ex­ten­sion, 10mm socket, and 8mm sock­etI’ve been ex­tremely happy with this set. However if you’re not plan­ning on do­ing more handyper­son type work then just bor­row these 4 parts from a neigh­bor in­stead of spend­ing the money on a whole set

I’ve been ex­tremely happy with this set. However if you’re not plan­ning on do­ing more handyper­son type work then just bor­row these 4 parts from a neigh­bor in­stead of spend­ing the money on a whole set

(Optional) A pre­ci­sion flat­head screw­driver (like this one). This can help with dis­con­nect­ing wire plugs

This Telematics DCM Bypass Kit, for fix­ing the in-car mi­cro­phone$90 is a bit steep for a part that prob­a­bly costs less than $1 to pro­duce, but the mak­ers of the kit did the work of read­ing the (paywalled) Toyota di­ag­nos­tics to pro­duce a work­ing prod­uct. If you’d like to build your own ver­sion you’ll need to sub­scribe to Toyota TIS to ac­cess the car wiring schemat­ics. It’s un­for­tu­nate that these schemat­ics and other re­pair man­u­als aren’t pub­lic

$90 is a bit steep for a part that prob­a­bly costs less than $1 to pro­duce, but the mak­ers of the kit did the work of read­ing the (paywalled) Toyota di­ag­nos­tics to pro­duce a work­ing prod­uct. If you’d like to build your own ver­sion you’ll need to sub­scribe to Toyota TIS to ac­cess the car wiring schemat­ics. It’s un­for­tu­nate that these schemat­ics and other re­pair man­u­als aren’t pub­lic

Overall this was a medium-dif­fi­culty pro­ject that took me a few hours to com­plete. Now, let’s get to work:

Removing the car mo­dem

1) Push down on the leather of your shifter and re­move the pin (don’t lose it!):

2) Remove the shifter top:

3) Use the trim tool to pop out the base of the shifter. Just lean it to the side, no need to dis­con­nect any­thing:

4) Use your hands to pop out the next panel and lean it to the side:

5) Remove these three 10mm bolts:

6) Pull on this light gray trim piece un­til it dis­con­nects slightly:

7) Pull the ra­dio out, dis­con­nect the plug, and put the ra­dio aside. The ra­dio is held on by clips only and can even be pulled out with your hands, but it re­quires a lit­tle force and the trim re­moval tool may be help­ful. When dis­con­nect­ing the plug it may help to use the pre­ci­sion screw­driver to push down on the tab to un­lock it, but you can also do it with your hands:

8) Pull the next panel (the seat warm­ing con­trols) out with your hands. It’s only held on by clips but may re­quire a bit of force to re­move:

9) Take a photo of all the wiring con­nec­tions on the seat warm­ing con­trols so you can as­sem­ble it cor­rectly later, un­plug all the wires, and set the con­trols aside:

10) You now have ac­cess to the DCM:

11) Removing the DCM re­quires a lot of ma­neu­ver­ing, tight spaces, and pa­tience, but you can do it. There are two 8mm bolts on the right and one 8mm bolt on the left that need to be re­moved. Getting ac­cess to them may re­quire re­mov­ing some of the other har­nesses or com­po­nents that are in the way - just go slow and steady, take your time, and take pho­tos of things be­fore you move them. After those 3 bolts are re­moved you have a lit­tle more play to pull the unit out, and af­ter dis­con­nect­ing the wires in the back you can com­pletely re­move the DCM. Here’s mine out of the car, part num­ber 86741 – 06130:

12) Now that the mo­dem is re­moved we need to in­stall the DCM Bypass Kit so the in-car mi­cro­phone con­tin­ues to work. It’s ex­tremely straight­for­ward, just plug it into the wiring har­ness that you re­moved from the DCM. The plugs will only fit on the cor­rect wires, there’s no way to get it wrong:

13) Reassemble every­thing by go­ing in re­verse or­der. Make sure all clips, bolts, etc. are back in their orig­i­nal po­si­tion and every­thing is seated cor­rectly. This part should go much faster than dis­as­sem­bly.

Now you’re done with the hard part. Next we dis­con­nect the GPS from the head unit, which is sig­nif­i­cantly eas­ier:

Removing the GPS an­tenna

1) Use the trim tool to re­move the back panel be­hind the in­fo­tain­ment screen:

2) Unscrew these four 10mm bolts:

3) Pop the head unit out (it’s only held on by 2 clips at this point). The part num­ber will vary but for my car it was 86140 – 0R710.

4) The GPS an­tenna is one of the sin­gle-wire ca­bles (not the multi-wire plugs). I had 3 sin­gle-wire ca­bles in my unit and the GPS wire was the black wire shown in the pic­ture. I was able to de­ter­mine this by process of elim­i­na­tion - un­plug­ging one of the wires dis­con­nected my car’s re­verse cam­era, un­plug­ging an­other one dis­con­nected CarPlay com­pletely, and the last one was the GPS - worked like a charm. Again, with a Toyota TIS sub­scrip­tion you can get ac­cess to the head unit wiring di­a­gram and not have to make guesses about which wire is which, but process of elim­i­na­tion worked fine for me:

5) Reassemble every­thing by go­ing in re­verse or­der. Again, make sure that all the clips seat prop­erly.

Confirming it worked

After you have every­thing re­assem­bled, turn the car on.

1) If you un­plugged the mo­dem suc­cess­fully then:

The in­fo­tain­ment screen will have an icon in the up­per right cor­ner in­di­cat­ing no con­nec­tion

The SOS light in the over­head con­sole will be off:

2) If the DCM Bypass Kit was in­stalled suc­cess­fully then:

Make a phone call through CarPlay. The re­cip­i­ent should be able to hear you / the mi­cro­phone should be work­ing

Congratulations - your car no longer has the ca­pa­bil­ity to trans­mit teleme­try data. Of course it may still be cap­tured to lo­cal stor­age and can be phys­i­cally col­lected later, but for me that was fine.

Conclusion

Overall I’m very happy with this pro­ject. Unfortunately I think it’s only a mat­ter of time be­fore the mo­dem and GPS be­come more deeply in­te­grated into the car (making this blog post in­fea­si­ble), or cars have more dras­tic fail­ure modes when the mo­dem/​GPS is re­moved, or anti-right-to-re­pair laws get passed to fur­ther clamp down on this be­hav­ior. For now the win stands - no teleme­try leaves the car. Strong Federal pri­vacy laws would make posts like this un­nec­es­sary, that’s the world I’d rather live in.

On dig­i­tal sov­er­eignty, and why European cloud is bet­ter than you think

April 29, 2026 10 min. Digital SovereigntyDigital InfrastructureDigital AutonomyEuropean CloudEurope

There’s a ver­sion of this post that starts with a spread­sheet and ends with a quiet sense of sat­is­fac­tion. That’s mostly how it went. But un­der­neath the prac­ti­cal ex­er­cise of swap­ping one SaaS tool for an­other was some­thing that felt more ur­gent, a grow­ing dis­com­fort with how much of my dig­i­tal in­fra­struc­ture sat on servers I did­n’t con­trol, in a ju­ris­dic­tion in­creas­ingly prone to un­pre­dictabil­ity, op­er­ated by com­pa­nies whose in­cen­tives don’t al­ways align with mine.

Digital sov­er­eignty sounds like a buzz­word un­til you think care­fully about what it means. It means know­ing where your data lives. It means not be­ing one pol­icy change, one ac­qui­si­tion, or one ex­ec­u­tive’s bad mood away from los­ing ac­cess to tools your busi­ness de­pends on. It means choos­ing in­fra­struc­ture based on val­ues, not just con­ve­nience.

So I started mi­grat­ing.

Analytics

Google Analytics was the ob­vi­ous first tar­get. It’s the canon­i­cal ex­am­ple of a ser­vice that’s free be­cause you are the prod­uct, your vis­i­tors’ be­hav­ior fun­neled back into Google’s ad­ver­tis­ing ma­chin­ery.

Self-hosting Matomo solved this cleanly. The data stays on my own server, and I’m fully GDPR-compliant with­out the cookie con­sent the­ater that Google Analytics typ­i­cally re­quires. The re­port­ing is com­pre­hen­sive, the in­ter­face is fa­mil­iar enough, and I own every­thing.

The main down­side is main­te­nance over­head. You’re now re­spon­si­ble for up­dates, back­ups, and keep­ing the server healthy. For most se­tups this is low-fric­tion, but it’s not zero fric­tion.

Email

Proton Mail is based in Switzerland, not EU ter­ri­tory, but Swiss pri­vacy law is closely aligned with GDPR and ar­guably stronger in some re­spects. Proton builds its busi­ness model around pri­vacy rather than ad­ver­tis­ing, and end-to-end en­cryp­tion is baked in at the pro­to­col level rather than bolted on. The email client is solid, the cal­en­dar works well, and for any­one mov­ing away from US-based ser­vices, it sits com­fort­ably in the same spirit as the rest of this stack.

One ad­just­ment is get­ting used to Proton’s fil­ter sys­tem, which is a bit more lim­ited than Gmail’s. Gmail lets you write fil­ters against vir­tu­ally any­thing, in­clud­ing the full body of the mes­sage. Proton does­n’t sup­port fil­ter­ing on email con­tent at all. So if you’ve built a work­flow around catch­ing spe­cific phrases or key­words in mes­sage bod­ies, you’ll have to re­think it. For most peo­ple this won’t be a deal­breaker, but it’s worth know­ing be­fore you mi­grate.

There’s also a prac­ti­cal lim­i­ta­tion worth flag­ging: Proton caps cus­tom do­mains at three, even on the Duo plan. If you run sev­eral do­mains, like sep­a­rate ad­dresses for dif­fer­ent pro­jects or busi­nesses, you’ll hit that ceil­ing quickly and need to re­think how you route and send mail. I ended up con­sol­i­dat­ing, which was prob­a­bly over­due any­way, but it was­n’t a choice I made en­tirely freely.

Proton is­n’t free and charges a sub­stan­tial fee com­pared to other op­tions. You’ll get ac­cess to a whole suite of Proton apps though.

Password Management

Once I was in the Proton ecosys­tem, mov­ing pass­word man­age­ment there as well made sense. Proton Pass is end-to-end en­crypted, open source, and ben­e­fits from the same Swiss ju­ris­dic­tion as the rest of Proton’s stack.

1Password is a gen­uinely great prod­uct and this was a lat­eral move more than an up­grade. The in­ter­face is sim­ple, the browser ex­ten­sion works re­li­ably, and hav­ing pass­words, email, and cal­en­dar un­der one en­crypted roof has a cer­tain sat­is­fy­ing co­her­ence to it.

Compute

DigitalOcean has earned its rep­u­ta­tion by do­ing one thing ex­cep­tion­ally well: get­ting out of your way. The UI is clean, the men­tal model is sim­ple, and spin­ning up in­fra­struc­ture never feels like a chore. It’s the plat­form that proved de­vel­oper ex­pe­ri­ence could be a com­pet­i­tive moat.

Scaleway was a pleas­ant sur­prise. I ex­pected a ca­pa­ble-but-rough European al­ter­na­tive, but what I found was a plat­form that’s gen­uinely well thought out. Servers spun up quickly in­side a pri­vate net­work of my own con­fig­u­ra­tion, the con­trol panel is clean, and the op­tions avail­able matched every­thing I ac­tu­ally needed. Scaleway dis­plays pro­jected CO₂ emis­sions along­side server lo­ca­tion choices, a nice touch.

Object Storage

Scaleway’s ob­ject stor­age is S3-compatible, which makes mi­gra­tion me­chan­i­cal rather than painful, up­date your end­point and cre­den­tials and ex­ist­ing code works un­changed.

I used a tool called rclone to sync my old AWS S3 stor­age buck­ets to the new Scaleway S3 buck­ets. This took a lit­tle more than a week of con­stant sync­ing, as these buck­ets were quite large.

Offsite Backups

OVH is the largest European cloud provider and brings the re­li­a­bil­ity and pric­ing you’d ex­pect at that scale. Their ob­ject stor­age works well as a backup des­ti­na­tion and ends up cheaper than Backblaze B2 once you con­fig­ure life­cy­cle rules to move older back­ups to the cold stor­age class.

Getting there, how­ever, re­quires some pa­tience. The OVHcloud con­trol panel is a labyrinth: the life­cy­cle rule con­fig­u­ra­tion is buried some­where in the doc­u­men­ta­tion, and it in­volves some work in the ter­mi­nal. Once it’s set up, it works re­li­ably and the cost dif­fer­ence is mean­ing­ful.

Transactional Emails

Lettermint is a European trans­ac­tional email ser­vice that does the job with­out the bloat. Deliverability is solid, the API is clean, and it has straight­for­ward pric­ing.

Compared to SendGrid, the an­a­lyt­ics are leaner and the ecosys­tem in­te­gra­tions are fewer. SendGrid has years of tool­ing, doc­u­men­ta­tion, and com­mu­nity an­swers be­hind it. Lettermint is newer and smaller. For most trans­ac­tional send­ing use cases (password re­sets, no­ti­fi­ca­tions, re­ceipts) that does­n’t mat­ter much. But if you’re do­ing com­plex multi-stream email in­fra­struc­ture, you’ll want to au­dit the fea­ture set care­fully first.

Error Tracking

Bugsink is a self-hosted er­ror track­ing tool that ac­cepts Sentry’s SDK, which means the mi­gra­tion path is al­most fric­tion­less, change one line of con­fig­u­ra­tion and you’re done.

To be hon­est: Bugsink is bare-bones. There’s no per­for­mance mon­i­tor­ing, no ses­sion re­plays, no ad­vanced alert­ing. It’s not a Sentry re­place­ment for teams that use Sentry prop­erly. For me, it’s a sim­ple re­mote er­ror log, when some­thing breaks in pro­duc­tion I get a stack trace and that’s enough. Sentry’s cloud prod­uct is gen­uinely ex­cel­lent if you need the full fea­ture set, and for larger en­gi­neer­ing teams the breadth al­most cer­tainly jus­ti­fies the cost. But if your use case is “tell me when some­thing broke and show me the stack trace”, self-hosted Bugsink does ex­actly that with no data leav­ing your in­fra­struc­ture.

AI API in­te­gra­tion

For my AI API in­te­gra­tions, I switched from OpenAI to Mistral. It worked out per­fectly as I was mostly us­ing sim­pler mod­els any­way.

Mistral is head­quar­tered in Paris and has pub­lished com­pelling open-weight mod­els along­side its API of­fer­ing. The API is clean, the mod­els are fast and ca­pa­ble, and there’s some­thing co­her­ent about a European AI provider that leans into open­ness rather than away from it. For my in­fer­ence work­loads, the switch was lat­eral in qual­ity and mean­ing­fully bet­ter in terms of where the money goes.

CDN

Exception № 1

Not every­thing moved. Cloudflare is a US com­pany, I still use it, and I’m at peace with that.

Here’s the rea­son­ing: Cloudflare sits in front of my pub­lic-fac­ing web­sites. Its job is to cache, pro­tect against DDoS at­tacks, and make con­tent load fast for vis­i­tors around the world. The data flow­ing through it is al­ready pub­lic by de­f­i­n­i­tion. I’m not rout­ing pri­vate com­mu­ni­ca­tions or sen­si­tive ap­pli­ca­tion data through Cloudflare; I’m us­ing it to serve pages that any­one on the in­ter­net can read. The sov­er­eignty cal­cu­lus is dif­fer­ent when the thing you’re pro­tect­ing is al­ready pub­lic.

I did try Bunny CDN, which is European-based and has a great rep­u­ta­tion. For straight­for­ward CDN use it’s ex­cel­lent. But Cloudflare’s fea­ture set (security rules, Workers plat­form, breadth of con­fig­u­ra­tion op­tions) was­n’t matched closely enough to jus­tify the switch for my spe­cific needs. Sometimes the prag­matic an­swer wins.

Payments

Exception № 2

Stripe is one of the few ser­vices I haven’t moved yet, even though pay­ment in­fra­struc­ture is ex­actly the kind of thing I care about hav­ing in a ju­ris­dic­tion I trust. Mollie is a Dutch pay­ment proces­sor with full EU in­cor­po­ra­tion, strong GDPR com­pli­ance by de­sign, and a prod­uct that has ma­tured con­sid­er­ably in re­cent years. The API has con­verged to­ward par­ity for most com­mon pay­ment flows, and for a European busi­ness the re­gional pay­ment method cov­er­age (iDEAL, Bancontact, SEPA) is ar­guably bet­ter.

The mi­gra­tion is on the list. It’s just not a triv­ial one. Payment in­te­gra­tions touch billing logic, web­hooks, tax in­voic­ing and cus­tomer-fac­ing flows in ways that re­quire care­ful test­ing and a good mo­ment to cut over. It’s also more ex­pen­sive than Stripe for my use­case.

AI Code as­sis­tance

Exception № 3

This one felt over­due. OpenAI works fine, but the com­pa­ny’s tra­jec­tory does­n’t align with my own views any­more. After a pe­riod of de­lib­er­ate drift, I felt the need to switch. Ideally I wanted to use Mistral Vibe here, but it just did­n’t make the cut as it could­n’t com­pete with Claude.

Claude Code is now my day-to-day AI as­sis­tant for cod­ing. The rea­son­ing qual­ity is strong, the con­text han­dling is gen­uinely im­pres­sive, and Anthropic’s ap­proach to safety and trans­parency feels more struc­turally grounded.

Anthropic is a US com­pany, so this does­n’t sat­isfy the ju­ris­dic­tional cri­te­rion I ap­plied else­where. But it sat­is­fies some­thing else, the sense that the or­ga­ni­za­tion build­ing the thing has given se­ri­ous thought to what it’s build­ing and why.

It’s also worth not­ing that lo­cal mod­els are be­com­ing in­creas­ingly vi­able. Qwen, Alibaba’s open-weight model fam­ily, is a strong ex­am­ple: ca­pa­ble enough for many real work­loads, run­ning en­tirely on your own hard­ware, with no data leav­ing your ma­chine. The gap be­tween fron­tier API mod­els and what you can run lo­cally is nar­row­ing faster than most peo­ple re­al­ize.

Not every­thing is ideal. Most data cen­ters still sit out­side Europe, and “open” means dif­fer­ent things to dif­fer­ent or­ga­ni­za­tions. But the di­rec­tion is right. A world where ca­pa­ble AI runs on your own hard­ware, with pub­lished weights and trans­par­ent train­ing, is a much bet­ter world for dig­i­tal au­ton­omy than one where all in­fer­ence routes through a hand­ful of closed API providers. We’re not there yet, but the tra­jec­tory is en­cour­ag­ing.

Git Version Control

Exception № 4

GitLab also re­mains for now. GitLab is head­quar­tered in the US but of­fers self-hosted op­tions, and the com­pany has long had a strong com­mit­ment to trans­parency and open source. A self-hosted in­stance is on the roadmap, but mov­ing source con­trol is a more sig­nif­i­cant un­der­tak­ing than most of these mi­gra­tions.

GitHub stays in the pic­ture for one spe­cific pur­pose: pub­lic-fac­ing NPM pack­ages and is­sue track­ing for open source soft­ware. When you pub­lish a pack­age or main­tain pub­lic tool­ing, GitHub is where de­vel­op­ers ex­pect to find it. The net­work ef­fects are real, it’s where the forks, stars, and is­sue re­ports come from. For the pub­lic-fac­ing sur­face of open source work, there’s no mean­ing­ful sov­er­eignty con­cern and a lot of prac­ti­cal up­side.

Was it worth it?

The prac­ti­cal fric­tion was real but man­age­able. Most mi­gra­tions were an af­ter­noon of work: up­date a cre­den­tial here, point a DNS record there, ex­port and im­port some data. A few took longer. None were cat­a­strophic. All in all it took longer than ex­pected, but most time was spent in re­search­ing and plan­ning when to do what. Two months in, every­thing is run­ning with­out in­ci­dent. No fires, no re­grets.

Digital sov­er­eignty is­n’t about para­noia. It’s about be­ing con­scious about your in­fra­struc­ture, where you de­cide who holds your data, who can reach it, and what hap­pens when pol­i­tics shift. The tools are there. The ecosys­tem is mostly ma­ture. The only thing that was stop­ping me was in­er­tia. It’s en­tirely pos­si­ble to run a re­li­able, ca­pa­ble, pro­fes­sional dig­i­tal stack mostly from European in­fra­struc­ture. This mi­gra­tion was proof of that.

Published May 10, 2026, 12:30 PM EDT

His love of PCs and their com­po­nents was born out of try­ing to squeeze every ounce of per­for­mance out of the fam­ily com­puter. Tinkering with his own build at age 10 turned into build­ing PCs for friends and fam­ily, fos­ter­ing a pas­sion that would ul­ti­mately take shape as a ca­reer path.

Besides be­ing the first call for tech sup­port for those close to him, Ty is a com­puter sci­ence stu­dent, with his fo­cus be­ing cloud com­put­ing and net­work­ing. He also com­peted in semi-pro Counter-Strike for 8 years, mak­ing him in­ti­mately fa­mil­iar with every­thing to do with pe­riph­er­als.

Sign in to your XDA ac­count

In March 2026, Linux crossed five per­cent of Steam’s user base for the first time, an all-time high for an op­er­at­ing sys­tem that spent two decades as a nov­elty when it came to any kind of gam­ing. Microsoft’s end-of-sup­port dead­line for Windows 10 last October pushed many users to look at al­ter­na­tives, and the Steam Deck has qui­etly turned mil­lions of peo­ple into Linux gamers with­out them re­ally think­ing about it, lead­ing to more wide­spread adop­tion on desk­top ma­chines.

Most of that progress used to hap­pen in­side a piece of soft­ware called Wine, the trans­la­tion layer that con­vinces Windows games they’re run­ning on Windows. Valve’s tuned ver­sion of Wine, called Proton, is what makes Steam Play and the Steam Deck work. For years, every mean­ing­ful im­prove­ment to Linux gam­ing came from changes to Wine and Proton them­selves. That’s still true, but in­creas­ingly the most im­por­tant changes are hap­pen­ing one layer deeper, in­side the Linux ker­nel. The lat­est ex­am­ple of that is some­thing called NTSYNC, a ker­nel-level dri­ver that has of­fered great per­for­mance gains over pre­vi­ous ver­sions of Wine, and is loaded by de­fault on every Steam Deck that’s up-to-date.

Related

What NTSYNC ac­tu­ally is

An ad­di­tional piece of the per­for­mance puz­zle

NTSYNC is a small piece of dri­ver added di­rectly to the Linux ker­nel that gives it a na­tive im­ple­men­ta­tion of a set of Windows-specific tools that games de­pend on to co­or­di­nate them­selves.

Modern games jug­gle dozens of things at once. While you’re play­ing, your CPU man­ages the ren­der­ing pipeline, load­ing as­sets, run­ning physics, pro­cess­ing au­dio, han­dling AI NPC rou­tines, and track­ing in­puts, all in par­al­lel across mul­ti­ple cores. All those jobs con­stantly have to co­or­di­nate so they don’t trip over each other.

Quiz

8 Questions · Test Your Knowledge

The his­tory of LinuxTrivia chal­lenge

From a Finnish stu­den­t’s side pro­ject to pow­er­ing the world — how well do you know the story of Linux?

OriginsKernelDistrosPioneersMilestones

Begin

01 / 8

Origins

In what year did Linus Torvalds first an­nounce the Linux ker­nel to the world?

A1989B1991C1993D1995

Correct! Linus Torvalds posted his now-fa­mous mes­sage to the comp.os.minix news­group on August 25, 1991, de­scrib­ing Linux as ‘just a hob­by’ pro­ject. Few could have pre­dicted it would one day run the ma­jor­ity of the world’s servers and smart­phones.

Not quite — Torvalds made his an­nounce­ment in 1991. He was a 21-year-old com­puter sci­ence stu­dent at the University of Helsinki at the time, and his mod­est post de­scribed the pro­ject as some­thing that ‘won’t be big and pro­fes­sion­al’ like GNU.

Continue

02 / 8

Pioneers

Which uni­ver­sity was Linus Torvalds at­tend­ing when he cre­ated the first ver­sion of the Linux ker­nel?

AStockholm UniversityBAalto UniversityCUniversity of HelsinkiDMIT

Correct! Torvalds was study­ing at the University of Helsinki in Finland when he be­gan work­ing on Linux as a per­sonal pro­ject, ini­tially in­spired by MINIX, a small Unix-like sys­tem used for ed­u­ca­tional pur­poses.

Not quite — Torvalds was a stu­dent at the University of Helsinki in Finland. He started Linux partly out of frus­tra­tion with the lim­i­ta­tions of MINIX, which his pro­fes­sor Andrew Tanenbaum had de­signed de­lib­er­ately to be sim­ple for teach­ing.

Continue

03 / 8

Kernel

What op­er­at­ing sys­tem pri­mar­ily in­spired Linus Torvalds to cre­ate the Linux ker­nel?

AMS-DOSBMINIXCBSD UnixDSolaris

Correct! MINIX, cre­ated by pro­fes­sor Andrew Tanenbaum, was the di­rect in­spi­ra­tion for Linux. Torvalds used MINIX on his new Intel 386 PC but found it too re­stricted for his needs, which pushed him to write his own ker­nel.

Not quite — the an­swer is MINIX. Torvalds was us­ing MINIX when he started Linux, and even held a fa­mous on­line de­bate with its cre­ator Andrew Tanenbaum about ker­nel de­sign phi­los­o­phy, specif­i­cally mono­lithic ver­sus mi­cro­ker­nel ar­chi­tec­tures.

Continue

04 / 8

Milestones

What was the ver­sion num­ber of the first pub­licly re­leased Linux ker­nel in 1991?

A0.01B0.1C1.0D0.99

Correct! Linux ver­sion 0.01 was the first ker­nel Torvalds re­leased pub­licly in September 1991. It was a rough, early build that could only run on Intel 386 hard­ware and had very lim­ited func­tion­al­ity, but it marked the true be­gin­ning of the Linux pro­ject.

Not quite — the first pub­lic re­lease was ver­sion 0.01 in September 1991. The ker­nel did­n’t reach ver­sion 1.0 un­til March 1994, by which point it had grown sig­nif­i­cantly in ca­pa­bil­ity and had at­tracted con­tri­bu­tions from de­vel­op­ers around the world.

Continue

05 / 8

Distros

Which Linux dis­tri­b­u­tion, first re­leased in 1993, is one of the old­est still ac­tively main­tained to­day?

AUbuntuBFedoraCSlackwareDDebian

Correct! Slackware, cre­ated by Patrick Volkerding, was first re­leased in July 1993, mak­ing it one of the old­est sur­viv­ing Linux dis­tri­b­u­tions. It is known for its sim­plic­ity and Unix-like phi­los­o­phy, and it con­tin­ues to be main­tained to this day.

Not quite — the an­swer is Slackware, re­leased in 1993 by Patrick Volkerding. While Debian was also founded in 1993, Slackware nar­rowly edges it out as the older re­lease. Ubuntu did­n’t ar­rive un­til 2004, and Fedora launched in 2003.

Continue

06 / 8

Origins

The GNU Project, which pro­vided many tools that paired with the Linux ker­nel, was founded by which de­vel­oper?

AEric RaymondBRichard StallmanCBruce PerensDIan Murdock

Correct! Richard Stallman founded the GNU Project in 1983 with the goal of cre­at­ing a com­pletely free Unix-like op­er­at­ing sys­tem. When the Linux ker­nel ap­peared in 1991, it filled the miss­ing piece GNU needed, and the com­bi­na­tion be­came what many call GNU/Linux.

Not quite — it was Richard Stallman who founded the GNU Project in 1983. Stallman is also known for cre­at­ing the GPL (GNU General Public License) and found­ing the Free Software Foundation, two pil­lars that shaped the le­gal and philo­soph­i­cal foun­da­tion of free soft­ware.

Continue

07 / 8

Milestones

Which com­pany re­leased a land­mark com­mer­cial Linux dis­tri­b­u­tion in 1994, help­ing bring Linux into the en­ter­prise world?

ACanonicalBSUSECRed HatDMandriva

Correct! Red Hat re­leased its first Linux dis­tri­b­u­tion in 1994 and be­came one of the most in­flu­en­tial com­mer­cial Linux com­pa­nies in his­tory. It pi­o­neered the en­ter­prise Linux mar­ket and was even­tu­ally ac­quired by IBM in 2019 for ap­prox­i­mately $34 bil­lion.

Not quite — Red Hat is the an­swer. Founded by Marc Ewing and Bob Young, Red Hat helped prove that com­pa­nies could build sus­tain­able busi­nesses around open-source soft­ware. SUSE Linux also launched in 1994, mak­ing it a close ri­val, but Red Hat be­came the more glob­ally dom­i­nant en­ter­prise force.

Continue

08 / 8

Distros

Ubuntu Linux, one of the most pop­u­lar desk­top dis­tri­b­u­tions, is based on which other Linux dis­tri­b­u­tion?

AArch LinuxBFedoraCDebianDGentoo

Correct! Ubuntu is based on Debian and was first re­leased in October 2004 by Mark Shuttleworth’s com­pany Canonical. It was de­signed to make Linux more ac­ces­si­ble to every­day users, and its six-month re­lease cy­cle and long-term sup­port ver­sions made it a fa­vorite for both desk­tops and servers.

Not quite — Ubuntu is built on top of Debian. Debian it­self was founded in 1993 by Ian Murdock and is known for its strict com­mit­ment to free soft­ware and sta­bil­ity. Ubuntu in­her­its Debian’s pack­age man­age­ment sys­tem (APT and .deb pack­ages) but adds its own user-friendly layer on top.

See My Score

Challenge Complete

Your Score

/ 8

Thanks for play­ing!

Try Again

Windows han­dles this co­or­di­na­tion by us­ing a spe­cific set of mech­a­nisms, and be­fore NTSYNC, Wine had to mimic these mech­a­nisms us­ing things like esync and fsync, which both worked, but did­n’t al­ways match Windows ex­actly. NTSYNC builds these mech­a­nisms straight into the Linux ker­nel for the first time, and it means Wine does­n’t have to em­u­late any­thing any­more. The de­vel­oper-fac­ing API calls don’t ac­tu­ally change, Linux just knows how to an­swer them na­tively.

Related

NTSYNC is part of a grow­ing pat­tern

Not the first time Linux has in­her­ited fea­tures be­cause of Windows

NTSYNC is­n’t the first time Linux has gained a new fea­ture specif­i­cally be­cause Windows games needed it. A few years back, Linux added a way for soft­ware to wait on sev­eral events at once, which is some­thing Windows had built in for decades, but Linux did­n’t. Wine had been work­ing around the gap with awk­ward tricks un­til the ker­nel fi­nally got na­tive sup­port.

This work is dri­ven by Valve, by CodeWeavers (the com­pany that em­ploys many of the core Wine de­vel­op­ers, in­clud­ing NTSYNC’s au­thor Elizabeth Figura), and by a steady stream of con­trib­u­tors who want Linux to be a real gam­ing plat­form with­out de­pend­ing on out-of-ecosys­tem patches for­ever.

Related

These aren’t mag­i­cal per­for­mance gains

fsync was al­ready pretty good

The head­line per­for­mance gains look great, but they need some con­text. The eye-catch­ing 40 to 200 per­cent FPS gains cited in NTSYNC’s orig­i­nal bench­marks were mea­sured against un­mod­i­fied up­stream Wine, which al­most no­body uses to play games on Linux any­more. Most Linux gamers, in­clud­ing every Steam Deck owner, use Proton, which al­ready has fsync. Compared to fsync, NTSYNC’s per­for­mance gains are far more mod­est. The games that ben­e­fit most from the change to NTSYNC are games that were re­ally strug­gling be­fore. Anything that was run­ning at de­cent fram­er­ates be­fore­hand is still go­ing to run fine.

Related

These 7 Linux myths you still be­lieve sim­ply aren’t true

Linux is a com­pletely dif­fer­ent beast than it was a decade ago.

Valve adopted it any­way

It’s a great sign

Pierre-Loup Griffais, an en­gi­neer at Valve, has gone on the record to say that fsync was al­ready fast enough, and de­spite that, Valve still shipped NTSYNC in sta­ble SteamOS in March any­way, which speaks to the fact that fsync is still a workaround at its core, and can be the cause of is­sues out­side of poor raw FPS.

These old workarounds got sub­tle edge cases wrong in ways that pro­duced oc­ca­sional hitches, dead­locks, or weird be­hav­ior in spe­cific games, which are bugs that don’t show up on bench­mark charts but can ab­solutely ruin in­di­vid­ual ex­pe­ri­ences. NTSYNC fixes those at the source by match­ing Windows be­hav­ior ex­actly, and that means as soon as your fa­vorite dis­tro moves to the new ker­nel ver­sion, whether it be Bazzite, CachyOS, Fedora, or a fla­vor of Ubuntu, they all get this much-needed fix.

Related

4 rea­sons Valve’s full SteamOS re­lease will change PC gam­ing again

Valve’s full SteamOS re­lease will change PC gam­ing again, and here are some of the most im­por­tant ways.

Gaming on Linux con­tin­ues to im­prove by the month

Linux has grown so much in the gam­ing de­part­ment. Where there once was noth­ing but clever Wine patches and com­mu­nity workarounds now lies sup­port from gam­ing be­he­moths like Valve, dri­ving changes to the Linux ker­nel it­self. NTSYNC won’t be the last time a piece of Windows gets re­built in­side Linux be­cause gamers needed it, and with more than five per­cent of Steam’s user base now run­ning Linux, the in­cen­tive to keep do­ing it has never been stronger.

09 May, 2026

This dev-log is get­ting a lot of at­ten­tion on HN (scary!): HN Thread.

To those who are com­ing here from HN: This started as an in­ves­ti­ga­tion or rather a ques­tion: “How far I can get with build­ing a piece of soft­ware by keep­ing my­self com­pletely out of the loop”. The tl;dr of this dev log is that I still need to be in the loop to make any­thing mean­ing­ful. Take aways:

like “em-dash” is to ai writ­ing, “god-object” is to ai cod­ing

vibe-cod­ing makes every­thing feel cheap and you may end up los­ing fo­cus and build­ing bloat

let a hu­man (you) write the ar­chi­tec­ture and don’t just keep ask­ing it for fea­tures

And some AGENTS.md/CLAUDE.md di­rec­tives that I felt help me stay out of the loop a lit­tle more

Humans in­ter­ven­tion is still needed as of 10/05/2026. You can to­tally go back now! [3]

I

Here is k10s: https://​github.com/​shvb­sle/​k10s/​tree/​archive/​go-v0.4.0

234 com­mits. ~30 week­ends. Built en­tirely on vibe-coded ses­sions with Claude, when­ever my to­kens lasted long enough to ship some­thing.

I’m archiv­ing my TUI tool and rewrit­ing it from scratch.

k10s started as a GPU-aware Kubernetes dash­board (and my first foray into build­ing some­thing se­ri­ous with AI). Think k9s but built for the peo­ple run­ning NVIDIA clus­ters, peo­ple who ac­tu­ally care about GPU uti­liza­tion, DCGM met­rics, and which nodes are sit­ting idle burn­ing $32/hr. I built it in Go with Bubble Tea [1] and it worked.

For a while… :(

I learned over these 7 months is worth more than the 1690 lines of model.go I’m throw­ing away. And I think any­one do­ing se­ri­ous vibe-cod­ing can ben­e­fit from this, be­cause this part does­n’t sur­face much (I feel it gets buried un­der the demo reels and the ve­loc­ity wins).

tl;dr: AI writes fea­tures, not ar­chi­tec­ture. The longer you let it drive with­out con­straints, the worse the wreck­age gets. The ve­loc­ity makes you think you’re win­ning right up un­til the mo­ment every­thing col­lapses si­mul­ta­ne­ously.

II

vibe cod­ing high

I started k10s in late September 2025. The first few weeks were magic. I’d prompt Claude with “add a pods view with live up­dates” and boom, it worked. Resource list views, name­space fil­ter­ing, log stream­ing, de­scribe pan­els, key­board nav­i­ga­tion. Each fea­ture landed clean be­cause the pro­ject was small enough that the AI could hold the whole thing in con­text.

The ba­sic k9s clone took maybe 3 week­ends. Resource views for pods, nodes, de­ploy­ments, ser­vices. A com­mand palette. Watch-based live up­dates. Vim key­bind­ings. All work­ing, all vibe-coded in sin­gle ses­sions. I was build­ing at maybe 10x my nor­mal speed and it felt in­cred­i­ble.

Then I wanted the main sell­ing point.

The whole rea­son k10s ex­ists is the GPU fleet view. A ded­i­cated screen that shows you every node’s GPU al­lo­ca­tion, uti­liza­tion from DCGM, tem­per­a­ture, power draw, mem­ory. Not buried in kubectl de­scribe node out­put, but right there in a pur­pose-built table with color-coded sta­tus. Idle nodes in yel­low. Busy in green. Saturated in red.

And Claude one-shot it. I prompted for the fleet view, it gen­er­ated the FleetView struct, the tab fil­ter­ing (GPU/CPU/All), the cus­tom ren­der­ing with al­lo­ca­tion bars. It looked beau­ti­ful. I was rid­ing the high.

Then I typed :rs pods to switch back to the pods view.

Nothing ren­dered. The table was empty. Live up­dates had stopped. I switched to nodes, it showed stale data from the fleet view’s fil­ter. I went back to fleet, the tab counts were wrong.

The god ob­ject had con­sumed it­self.

This is the ti­tle of the blog post. This is where I in­ter­vened for the first time. For 7 months I’d been prompt­ing and ship­ping with­out ever sit­ting down and ac­tu­ally read­ing the code Claude wrote. I’d look at the diff, ver­ify it com­piled, test the happy path, move on. But now some­thing was fun­da­men­tally bro­ken and I could­n’t just prompt my way out of it.

So I sat down and read model.go. All 1690 lines. I was hor­ri­fied.

Here’s what it looked like. One struct to rule them all:

type Model struct { // 3rd party UI com­po­nents table table.Model pag­i­na­tor pag­i­na­tor.Model com­mand­In­put tex­tin­put.Model help help.Model

// clus­ter info and state k8sClient *k8s.Client cur­rent­GVR schema.GroupVer­sion­Re­source re­source­Watcher watch.In­ter­face re­sources []k8s.OrderedResourceFields listOp­tions metav1.ListOp­tions clus­ter­Info *k8s.ClusterInfo log­Lines []k8s.LogLine de­scribeCon­tent string cur­rent­Name­space string nav­i­ga­tion­His­tory *NavigationHistory logView *LogViewState de­scribeView *DescribeViewState view­Mode ViewMode viewWidth int viewHeight int err er­ror plug­in­Reg­istry *plugins.Registry help­Modal *HelpModal de­scribeView­port *DescribeViewport logView­port *LogViewport logStream­Can­cel func() log­Li­nesChan <-chan k8s.Log­Line hor­i­zon­talOff­set int mouse *MouseHandler fleetView *FleetView cre­ation­Times []time.Time all­Re­sources []k8s.OrderedResourceFields // fleet’s un­fil­tered set all­Cre­ation­Times []time.Time // fleet’s time­stamps ra­wOb­jects []unstructured.Unstructured ageColumnIn­dex int // … }

UI wid­gets. K8s client. Per-view state for logs, de­scribe, fleet. Navigation his­tory. Caching. Mouse han­dling. All in one struct. And the Update() method was a 500-line func­tion dis­patch­ing on msg.(type) with 110 switch/​case branches.

This is the mo­ment I stopped vibe-cod­ing and started think­ing.

III

five tenets from the wreck­age

Here’s what I ex­tracted from 7 months of watch­ing AI gen­er­ate a code­base that slowly ate it­self. Each of these is some­thing I did wrong, why it hap­pens with AI-assisted cod­ing, and what you should ac­tu­ally put in your CLAUDE.md or agents.md to pre­vent it.

Tenet 1: AI builds fea­tures, not ar­chi­tec­ture.

Every time I prompted Claude for a fea­ture, it de­liv­ered. Perfectly. The fleet view worked on the first try. Log stream­ing worked. Mouse sup­port worked. The prob­lem is that each fea­ture was im­ple­mented in the con­text of “make this work right now” with­out any aware­ness of the 49 other fea­tures shar­ing the same state.

Here’s what the re­sources­Load­edMsg han­dler looks like. This is the code that runs every time you switch views:

case re­sources­Load­edMsg: m.log­Lines = nil // Clear log lines when load­ing re­sources m.hor­i­zon­talOff­set = 0 // Reset hor­i­zon­tal scroll on re­source change

if m.cur­rent­GVR != msg.gvr && m.re­source­Watcher != nil { m.re­source­Watcher.Stop() m.re­source­Watcher = nil } m.cur­rent­GVR = msg.gvr m.cur­rent­Name­space = msg.name­space m.listOp­tions = msg.listOp­tions m.ra­wOb­jects = msg.ra­wOb­jects

// For nodes: store the full un­fil­tered set, clas­sify, then fil­ter if msg.gvr.Re­source == k8s.Re­sourceNodes && m.fleetView != nil { m.all­Re­sources = msg.re­sources m.all­Cre­ation­Times = msg.cre­ation­Times if len(msg.ra­wOb­jects) > 0 { m.fleetView.Clas­si­fyAnd­Count(m.ra­wOb­jectP­trs()) } m.ap­plyFleet­Fil­ter() } else { m.re­sources = msg.re­sources m.cre­ation­Times = msg.cre­ation­Times m.all­Re­sources = nil m.all­Cre­ation­Times = nil }

See the if msg.gvr.Re­source == k8s.Re­sourceNodes && m.fleetView != nil con­di­tional? That’s the fleet view be­ing spe­cial-cased in­side the generic re­source load­ing path. Every new view that needed cus­tom be­hav­ior got an­other branch here. And every branch needed to man­u­ally clear the right com­bi­na­tion of fields or the pre­vi­ous view’s data would bleed through.

How many = nil cleanup lines ex­ist in this file? I counted:

m.log­Lines = nil // Clear log lines when load­ing re­sources m.all­Re­sources = nil // Clear fleet data when not on nodes m.re­sources = nil // Clear re­sources when load­ing logs m.re­sources = nil // Clear re­sources when load­ing de­scribe view m.log­Lines = nil // Clear log lines when load­ing de­scribe view m.re­sources = nil // Clear re­sources when load­ing yaml view m.log­Lines = nil // Clear log lines when load­ing yaml view m.log­Lines = nil // … two more in other han­dlers m.log­Lines = nil

Nine man­ual nil as­sign­ments scat­tered across a 1690-line file. Miss one and you get ghost data from the pre­vi­ous view. This is what hap­pens when there’s no view iso­la­tion. AI can’t see this pat­tern de­cay­ing over time be­cause each prompt only touches one code path.

What to do in­stead: Write the ar­chi­tec­ture your­self be­fore any code. Not a vague de­sign doc. A con­crete set of in­ter­faces, mes­sage types, and own­er­ship rules. Then put those rules in your CLAUDE.md so the AI sees them on every prompt:

# Architecture Invariants (CLAUDE.md)

- Each view im­ple­ments the View trait. Views do NOT ac­cess other views’ state. - All async data ar­rives via AppMsg vari­ants. No di­rect field mu­ta­tion from back­ground tasks. - Adding a new view MUST NOT re­quire mod­i­fy­ing ex­ist­ing views. - The App struct is a thin router. It owns nav­i­ga­tion and mes­sage dis­patch. Nothing else.

The AI will fol­low these if you write them down. It just won’t in­vent them for you.

Tenet 2: The god ob­ject is the de­fault AI ar­ti­fact.

AI grav­i­tates to­ward sin­gle-struct-holds-every­thing be­cause it sat­is­fies the im­me­di­ate prompt with min­i­mal cer­e­mony. But it gets worse. Because there’s no view iso­la­tion, key han­dling be­comes a night­mare. Here’s the ac­tual key dis­patch for the s key:

case m.con­fig.Key­Bind.For(con­fig.Ac­tion­Tog­gleAu­to­Scroll, key): if m.cur­rent­GVR.Re­source == k8s.Re­sourceL­ogs { m.logView.Au­to­scroll = !m.logView.Autoscroll if m.logView.Au­to­scroll { m.table.Go­to­Bot­tom() } re­turn m, nil } // Shell exec for pods and con­tain­ers views if m.cur­rent­GVR.Re­source == k8s.Re­sour­ce­Pods { // … 20 lines to look up se­lected pod, get name, name­space … re­turn m, m.com­mand­With­Pre­flights( m.ex­ecIn­to­Pod(se­lect­ed­Name, se­lect­ed­Name­space), m.re­quire­Con­nec­tion, ) } if m.cur­rent­GVR.Re­source == k8s.Re­source­Con­tain­ers { // … con­tainer exec logic … re­turn m, m.com­mand­With­Pre­flights(m.ex­ecIn­to­Con­tainer(), m.re­quire­Con­nec­tion) } re­turn m, nil

One key­bind­ing. Three com­pletely dif­fer­ent be­hav­iors de­pend­ing on which view you’re in. The s key means “autoscroll” in logs, “shell” in pods, and “shell into con­tainer” in con­tain­ers. This is all in one flat switch be­cause there are no per-view key maps. The AI gen­er­ated this be­cause I said “add shell sup­port for pods” and it found the near­est key han­dler and jammed it in.

And look at how Enter works. This is the drill-down han­dler:

case m.con­fig.Key­Bind.For(con­fig.Ac­tion­Sub­mit, key): // Special han­dling for con­texts view if m.cur­rent­GVR.Re­source == “contexts” { // … 12 lines … re­turn m, m.ex­e­cu­teC­tx­Com­mand([]string{con­textName}) } // Special han­dling for name­spaces view if m.cur­rent­GVR.Re­source == “namespaces” { // … 12 lines … re­turn m, m.ex­e­cuteN­sCom­mand([]string{name­space­Name}) } if m.cur­rent­GVR.Re­source == k8s.Re­sourceL­ogs { re­turn m, nil } // … 25 more lines of generic drill-down …

Every view is a con­di­tional in a flat dis­patch. There are 20+ oc­cur­rences of m.cur­rent­GVR.Re­source == used as a type dis­crim­i­na­tor in this sin­gle file. Not types. String com­par­isons. Every new view means touch­ing every han­dler.

What to do in­stead: Put this in your CLAUDE.md:

# State Ownership Rules

- NEVER add fields to the App/Model struct for view-spe­cific state. - Each view is a sep­a­rate struct im­ple­ment­ing the View trait/​in­ter­face. - Each view de­clares its own key bind­ings. The app dis­patches keys to the ac­tive view. - If you need to add a key­bind­ing, add it to the rel­e­vant view’s keymap, not a global one. - Adding a view means adding a file. If your change re­quires mod­i­fy­ing ex­ist­ing views, stop and ask.

The AI will al­ways take the short­est path (“add an­other if-branch”). Your job is to make the short­est path also the cor­rect path by putting guardrails in the file it reads on every in­vo­ca­tion.

Tenet 3: Velocity il­lu­sion widens your scope.

This one’s psy­cho­log­i­cal, not tech­ni­cal, and I think it’s the most dan­ger­ous.

When I started k10s, I wanted a GPU-focused tool. For peo­ple run­ning train­ing clus­ters. A niche au­di­ence that I’m part of. But vibe-cod­ing made every­thing feel cheap. “Oh I can add pods view in one ses­sion? Let me add de­ploy­ments too. And ser­vices. And a full com­mand palette. And mouse sup­port. And con­texts. And name­spaces.”

Suddenly I was build­ing k9s. A gen­eral-pur­pose Kubernetes TUI. For every­one. Because the AI made it feel like each fea­ture was free.

It was­n’t free. Each fea­ture was an­other branch in the god ob­ject. Here’s the key­bind­ing struct:

type keyMap struct { Up, Down, Left, Right key.Bind­ing GotoTop, GotoBottom key.Bind­ing AllNS, DefaultNS key.Bind­ing Enter, Back key.Bind­ing Command, Quit key.Bind­ing Fullscreen key.Bind­ing // log view Autoscroll key.Bind­ing // log view (also shell in pods!) ToggleTime key.Bind­ing // log view WrapText key.Bind­ing // log + de­scribe view CopyLogs key.Bind­ing // log view ToggleLineNums key.Bind­ing // de­scribe view Describe key.Bind­ing // re­source views YamlView key.Bind­ing // re­source views Edit key.Bind­ing // re­source views Shell key.Bind­ing // pods (CONFLICTS with Autoscroll!) FilterLogs key.Bind­ing // log view FleetTabNext key.Bind­ing // fleet view only FleetTabPrev key.Bind­ing // fleet view only }

One flat keymap for all views. Comments in parens show which view each bind­ing ap­plies to. Autoscroll and Shell are both s. This “works” be­cause the dis­patch checks m.cur­rent­GVR.Re­source be­fore act­ing. But it means you can’t rea­son about key­bind­ings lo­cally. You have to trace through the en­tire 500-line Update func­tion to know what a key does.

The com­plex­ity was ac­cu­mu­lat­ing in­vis­i­bly while the ve­loc­ity met­ric said “you’re ship­ping!”

What to do in­stead: Write a vi­sion doc that ex­plic­itly says who you’re NOT build­ing for, and put the scope bound­ary in your CLAUDE.md:

# Scope (do NOT ex­pand be­yond this)

k10s is for GPU clus­ter op­er­a­tors. Not all Kubernetes users. Supported views: fleet, node-de­tail, gpu-de­tail, work­load. That’s it. Do NOT add generic re­source views (pods, de­ploy­ments, ser­vices). Do NOT add fea­tures that du­pli­cate k9s func­tion­al­ity. If a fea­ture re­quest does­n’t serve some­one run­ning GPU train­ing jobs, re­ject it.

Vibe-coding makes you feel like you have in­fi­nite im­ple­men­ta­tion bud­get. You don’t. You have in­fi­nite LINE bud­get (the AI will gen­er­ate as much code as you want). But you have the same fi­nite com­plex­ity bud­get as al­ways. The ar­chi­tec­ture can only sup­port so many fea­tures be­fore it buck­les, re­gard­less of how fast you wrote them. The CLAUDE.md scope sec­tion is you say­ing no in ad­vance, be­fore the ve­loc­ity high con­vinces you to say yes.

Tenet 4: Positional data is a time bomb.

Every re­source in k10s was fetched from the Kubernetes API and im­me­di­ately flat­tened:

type OrderedResourceFields []string

Column iden­tity was purely po­si­tional. Here’s the sort func­tion for the fleet view. Look at the in­dex ac­cess:

func sort­Fil­tere­dResources(rows []k8s.OrderedResourceFields, times []time.Time, tab FleetTab) { sort.SliceS­table(in­dices, func(a, b int) bool { ra := rows[in­dices[a]] rb := rows[in­dices[b]]

switch tab { case FleetTabGPU: // Sort by Alloc col­umn (index 3) as­cend­ing al­locA, al­locB := “”, “” if len(ra) > 3 { al­locA = ra[3] } if len(rb) > 3 { al­locB = rb[3] } re­turn al­locA < al­locB

case FleetTabCPU: // Sort by Name col­umn (index 0) as­cend­ing nameA, nameB := “”, “” if len(ra) > 0 { nameA = ra[0] } if len(rb) > 0 { nameB = rb[0] } re­turn nameA < nameB

case FleetTabAll: // GPU nodes first, then CPU nodes. // Within GPU: sort by Alloc (index 3). // Within CPU: sort by Name (index 0). com­puteA, com­puteB := “”, “” if len(ra) > 2 { com­puteA = ra[2] } if len(rb) > 2 { com­puteB = rb[2] } aIs­GPU := strings.HasPre­fix(com­puteA, “gpu”) bIs­GPU := strings.HasPre­fix(com­puteB, “gpu”) // … } }) }

ra[3] is Alloc. ra[2] is Compute. ra[0] is Name. These are magic num­bers. The only thing con­nect­ing in­dex 3 to “Alloc” is a com­ment and the col­umn or­der de­fined in re­source.views.json:

{ “nodes”: { “fields”: [ { “name”: “Name”, “weight”: 0.28 }, { “name”: “Instance”, “weight”: 0.15 }, { “name”: “Compute”, “weight”: 0.12 }, { “name”: “Alloc”, “weight”: 0.12 }, … ] } }

Add a col­umn be­tween Instance and Compute? Every sort, every con­di­tional ren­der, every place that says ra[2] or ra[3] is now silently wrong. The com­piler can’t help you be­cause it’s all []string. And the JSON con­fig can’t ex­press sort be­hav­ior, con­di­tional ren­der­ing, or cus­tom drill tar­gets, so those live in Go code that hard­codes the po­si­tional as­sump­tions from the JSON.

AI gen­er­ates this pat­tern be­cause it’s the short­est path from “fetch data” to “render table.” A []string sat­is­fies any table wid­get im­me­di­ately. Typed structs re­quire more cer­e­mony up­front. So the AI picks the fast path, and six months later you’re de­bug­ging why sort puts “Name” val­ues in the “Alloc” col­umn.

What to do in­stead: Put this di­rec­tive in your CLAUDE.md:

# Data Representation

- NEVER flat­ten struc­tured data into []string, Vec<String>, or po­si­tional ar­rays. - All data flows as typed structs (FleetNode, PodInfo, etc.) un­til the ren­der() call. - Column iden­tity comes from struct field names, not ar­ray in­dices. - Sort func­tions op­er­ate on typed fields, never on po­si­tional ac­cess like row[3]. - The ONLY place strings are cre­ated for dis­play is in­side ren­der()/​view() func­tions.

Then your typed struct makes im­pos­si­ble states im­pos­si­ble [2]:

struct FleetNode { name: String, in­stance_­type: String, com­pute_­class: ComputeClass, al­loc: GpuAlloc, }

You can’t sort by the wrong col­umn when columns are named fields. You can’t ac­ci­den­tally com­pare Alloc strings as names. The com­piler en­forces this for you. AI will al­ways pick Vec<String> be­cause it sat­is­fies the prompt faster. Your CLAUDE.md makes the typed path the path of least re­sis­tance.

Tenet 5: AI does­n’t own state tran­si­tions.

The Bubble Tea ar­chi­tec­ture has a beau­ti­ful idea: Update() is the only place state mu­tates, dri­ven by mes­sages. But k10s vi­o­lated this. The up­dateTableMsg han­dler spawned a clo­sure that mu­tated Model fields from in­side a gor­ou­tine:

case up­dateTableMsg: re­turn m, func() tea.Msg { // block on some­one send­ing the up­date mes­sage. <-m.updateTableChan // Preserve cur­sor po­si­tion across col­umn/​row up­dates so that // back­ground re­freshes don’t re­set the user’s se­lec­tion. saved­Cur­sor := max(m.table.Cur­sor(), 0) // run the nec­es­sary table view up­date calls. m.up­date­Columns(m.viewWidth) m.up­dateTable­Data() // Restore cur­sor, clamped to valid range. row­Count := len(m.table.Rows()) if row­Count > 0 { if saved­Cur­sor >= row­Count { saved­Cur­sor = row­Count - 1 } m.table.SetCur­sor(saved­Cur­sor) } re­turn up­dateTableMsg{} }

This re­turned func­tion (a tea.Cmd) is ex­e­cuted by Bubble Tea in a sep­a­rate gor­ou­tine. It calls m.up­date­Columns(m.viewWidth) and m.up­dateTable­Data() which read and write m.re­sources, m.table, m.viewWidth. Meanwhile, View() is called on the main gor­ou­tine read­ing the same fields. There’s no lock. No mu­tex. The chan­nel <-m.updateTableChan blocks the gor­ou­tine un­til some­one sends an up­date sig­nal, but noth­ing pre­vents View() from read­ing half-writ­ten state.

This is a text­book data race. It worked 99% of the time. Corrupted the dis­play 1% of the time in ways that made me think I was go­ing in­sane.

AI gen­er­ates this be­cause “just mu­tate it in the clo­sure” is the short­est path to work­ing code. Proper mes­sage pass­ing (send a mes­sage back to Update(), let Update() ap­ply the mu­ta­tion atom­i­cally on the main loop) re­quires more types, more plumb­ing. The AI is op­ti­miz­ing for the prompt, not for cor­rect­ness un­der con­cur­rency.

What to do in­stead: All mu­ta­tions to ren­der-vis­i­ble state hap­pen on the main loop. Period. Background work­ers pro­duce data. They send it as a mes­sage. The main loop re­ceives the mes­sage and ap­plies it. This is the one rule you can­not break in con­cur­rent UI code.

// Background task: tx.send(AppMsg::Fleet­Data(nodes)).await;

// Main loop: match msg { AppMsg::FleetData(nodes) => { self.fleet_view.up­date_n­odes(nodes); } }

Be in the know.

Sorry, some­thing went wrong. Please en­ter your name and email again.

I con­firm I am 18 years of age or older. I ac­cept and ac­knowl­edge that my in­for­ma­tion will be used in ac­cor­dance with

Be in the know.

Sorry, some­thing went wrong. Please en­ter your name and email again.

I con­firm I am 18 years of age or older. I ac­cept and ac­knowl­edge that my in­for­ma­tion will be used in ac­cor­dance with

Intelligence is the new spec.

Link to Youtube Video (visible only when JS is dis­abled)

The best of Gemini meets our most ad­vanced lap­tops.

Select any­thing to ask, com­pare, or cre­ate with Gemini, in­stantly.1

Open your phone apps on your lap­top, no in­stalls needed.2

Access files from your phone as if they live on your lap­top.2

Be in the know.

Sorry, some­thing went wrong. Please en­ter your name and email again.

I con­firm I am 18 years of age or older. I ac­cept and ac­knowl­edge that my in­for­ma­tion will be used in ac­cor­dance with

Check re­sponses. Internet con­nec­tion re­quired. 18+. Results may vary based on vi­sual matches and are for il­lus­tra­tive pur­poses only. Sequences short­ened.

Setup re­quired. Phone with Android 17 or above re­quired.