Elon Musk’s claim that he was mis­treated by his OpenAI co-founders failed af­ter nine California ju­rors re­turned a unan­i­mous ver­dict that his law­suits had been filed too late.

Musk ac­cused Sam Altman, Greg Brockman, OpenAI, and Microsoft of “stealing a char­ity” by cre­at­ing a for-profit af­fil­i­ate of the fron­tier AI lab. Jurors, how­ever, found that any harms that Musk may have suf­fered came be­fore the dead­line for fil­ing his claims un­der the law.

While the trial delved deeply into the melo­dra­matic his­tory of OpenAI and fea­tured tes­ti­mony from lead­ing fig­ures in Silicon Valley, it ul­ti­mately turned on fairly nar­row ques­tions of the law. The trial fo­cused on whether and when Altman and the other de­fen­dants had made and bro­ken promises to Musk, but his case failed to con­vince ju­rors that he had a valid claim.

In par­tic­u­lar, OpenAI had ad­vanced a statute of lim­i­ta­tions de­fense, which sought to prove that any harms Musk sought to lit­i­gate had taken place be­fore 2021. (The spe­cific date var­ied by the charge: be­fore August 5, 2021, for the first count; August 5, 2022, for the sec­ond count; and November 14, 2021, for the third count.) Ultimately, the jury found that ar­gu­ment per­sua­sive, which made for a short de­lib­er­a­tion pe­riod.

“There was a sub­stan­tial amount of ev­i­dence to sup­port the ju­ry’s find­ing, which is why I was pre­pared to dis­miss on the spot,” Judge Yvonne Gonzalez Rogers said af­ter the ver­dict was de­liv­ered.

The end of the case means that one ma­jor threat to OpenAI — a pos­si­ble re­struc­tur­ing — is now off the table ahead of its re­ported IPO.

“It did not take [the jury] two hours to con­clude … that Mr. Musk’s law­suit is noth­ing more than an af­ter-the-fact con­trivance that bears no re­la­tion­ship to re­al­ity,” OpenAI’s lead at­tor­ney, Bill Savitt, said af­ter the ver­dict. “They kicked it ex­actly where it be­longs — just to the side. This law­suit is a hyp­o­crit­i­cal at­tempt to sab­o­tage a com­peti­tor.”

Microsoft, which Musk sued for aid­ing and abet­ting OpenAI’s al­leged breach of char­i­ta­ble trust, wel­comed the ver­dict. A spokesper­son for the com­pany said it “remained com­mit­ted to our work with OpenAI to ad­vance and scale AI for peo­ple and or­ga­ni­za­tions around the world.”

The ver­dict came in the mid­dle of a hear­ing to de­ter­mine the po­ten­tial dam­ages to Musk if the ver­dict had gone the other way. While that dis­cus­sion is moot for now, the judge ap­peared un­con­vinced by the anal­ogy Musk’s lawyers drew be­tween his char­i­ta­ble con­tri­bu­tions and in­vest­ments in a for-profit startup.

“Your analy­sis seems to be de­void of con­nec­tion to the un­der­ly­ing facts,” she told Dr. C. Paul Wazzan, the ex­pert who came up with Musk’s es­ti­mate of OpenAI and Microsoft’s wrong­ful gains at his ex­pense — some $78.8 bil­lion to $135 bil­lion.

In a tweet af­ter the rul­ing, Musk ap­peared to take the pro­ce­dural grounds of the dis­missal as a moral vic­tory. “There is no ques­tion to any­one fol­low­ing the case in de­tail that Altman & Brockman did in fact en­rich them­selves by steal­ing a char­ity. The only ques­tion is WHEN they did it!” Musk wrote. “I will be fil­ing an ap­peal with the Ninth Circuit, be­cause cre­at­ing a prece­dent to loot char­i­ties is in­cred­i­bly de­struc­tive to char­i­ta­ble giv­ing in America.”

Reached for com­ment by TechCrunch, Musk’s lead coun­sel, Marc Toberoff, said, “One word: Appeal.”

When you pur­chase through links in our ar­ti­cles, we may earn a small com­mis­sion. This does­n’t af­fect our ed­i­to­r­ial in­de­pen­dence.

Tim Fernholz is a jour­nal­ist who writes about tech­nol­ogy, fi­nance and pub­lic pol­icy. He has closely cov­ered the rise of the pri­vate space in­dus­try and is the au­thor of Rocket Billionaires: Elon Musk, Jeff Bezos and the New Space Race. Formerly, he was a se­nior re­porter at Quartz, the global busi­ness news site, for more than a decade, and be­gan his ca­reer as a po­lit­i­cal re­porter in Washington, D.C.

You can con­tact or ver­ify out­reach from Tim by email­ing tim.fern­holz@techcrunch.com or via an en­crypted mes­sage to tim_fern­holz.21 on Signal.

View Bio

May 13th, 2026 | 14 minute read

Modern cars are com­put­ers on wheels - they have more sen­sors than you can count and are con­stantly phon­ing home with teleme­try data like your lo­ca­tion, speed, fuel lev­els, sud­den ac­cel­er­a­tions/​de­cel­er­a­tions, video footage, dri­ver at­ten­tion data from eye mon­i­tor­ing sys­tems, and hun­dreds of other data points. Cars have in­ward- and out­ward-fac­ing cam­eras. They have mi­cro­phones. They have al­ways-on modems. It’s all en­abled by de­fault with dif­fi­cult or mean­ing­less opt-outs, and your data is mon­e­tized through bro­kers like LexisNexis or Verisk. This all brings a host of se­cu­rity and pri­vacy is­sues - here are some over the years:

In 2025 Subaru had vul­ner­a­bil­i­ties al­low­ing any­one to re­motely un­lock cus­tomers’ cars, as well as ac­cess the real-time GPS lo­ca­tion and lo­ca­tion his­tory of the car of the car

Car man­u­fac­tur­ers share your dri­ving data with in­sur­ance com­pa­nies, which then in­crease your pre­mi­ums

In 2023 Tesla em­ploy­ees in­ter­nally shared cam­era footage of naked cus­tomers and other sen­si­tive im­ages

In 2015 Charlie Miller and Chris Valasek fa­mously took over a Jeep Cherokee with full con­trol of the ig­ni­tion, brakes, locks, steer­ing, etc.

Mozilla de­tailed how 25 car man­u­fac­tur­ers scored abysmally on pri­vacy and how they col­lect data in­clud­ing “sexual ac­tiv­ity, im­mi­gra­tion sta­tus, race, fa­cial ex­pres­sions, weight and ge­netic in­for­ma­tion.” They sell this data to third par­ties and use it to build pro­files about you cov­er­ing “intelligence, abil­i­ties, char­ac­ter­is­tics, pref­er­ences, and more.”

Tesla had a vul­ner­a­bil­ity in 2017 that al­lowed any­one to re­motely see your car’s lo­ca­tion, man­age other fea­tures, and even sum­mon the car to them­selves

The Car That Watches You Back de­tails how cars are now serv­ing you ads, as well as col­lect­ing vast amounts of data about you. The Hacker News dis­cus­sion about this ar­ti­cle is what prompted this blog post

Now that we’re suf­fi­ciently mo­ti­vated, what can we do about it? In this blog post, rather than re­ly­ing on com­pa­nies’ promises or mean­ing­less opt-outs, we’re go­ing to stop the data at the source by phys­i­cally re­mov­ing the mo­dem (the DCM, or Data Communication Module) as well as the built-in GPS on my 2024 RAV4 Hybrid, so the car will no longer have the ca­pa­bil­ity to send any teleme­try data back home. Let’s dive in:

Will the car still be func­tional?

Yes. Depending on how dif­fer­ent car man­u­fac­tur­ers have wired their cars, how their soft­ware and firmware were writ­ten, etc., vary­ing lev­els of func­tion­al­ity might be af­fected by re­mov­ing the mo­dem and GPS. For this car:

Everything that re­lies on a data con­nec­tion will no longer work. This in­cludes things like over-the-air up­dates as well as Toyota cloud-based ser­vices and SOS func­tion­al­i­tyThis is a safety trade­off - you’re dis­abling au­to­matic crash no­ti­fi­ca­tion and emer­gency call­ing

This is a safety trade­off - you’re dis­abling au­to­matic crash no­ti­fi­ca­tion and emer­gency call­ing

The car’s mi­cro­phone is wired through the DCM, and in the ab­sence of any other changes re­mov­ing the DCM means the in-car mi­cro­phone won’t work, which is in­con­ve­nient if you plan on tak­ing calls in the car. However we’ll in­stall a DCM Bypass Kit (discussed more be­low) to re­store all func­tion­al­ity and have a work­ing mi­cro­phone

CarPlay has a quirk: the phone uses its own GPS but also ac­cepts a lo­ca­tion sig­nal from the car’s GPS unit. After re­mov­ing the DCM, the car would get con­fused about its lo­ca­tion and some­times jump my po­si­tion to the mid­dle of Nevada (I live in San Francisco), mak­ing nav­i­ga­tion an­noy­ing. To work around this we’ll fully dis­con­nect the car’s GPS, so it can’t send a bad lo­ca­tion to the phone­From the ti­tle of the blog post you might have won­dered why bother re­mov­ing the GPS af­ter we’ve re­moved the mo­dem - who cares if the car has built-in lo­ca­tion when it can’t phone home with that data? This is whyThis is a well-doc­u­mented bug with dis­cus­sions on Apple Support threads as well as car-spe­cific fo­rums like rav4­world. This bug af­fects more than just Toyotas, it’s a generic Apple bug even for peo­ple who haven’t re­moved their mo­dem (but anec­do­tally re­mov­ing my mo­dem made the prob­lem worse)

From the ti­tle of the blog post you might have won­dered why bother re­mov­ing the GPS af­ter we’ve re­moved the mo­dem - who cares if the car has built-in lo­ca­tion when it can’t phone home with that data? This is why

This is a well-doc­u­mented bug with dis­cus­sions on Apple Support threads as well as car-spe­cific fo­rums like rav4­world. This bug af­fects more than just Toyotas, it’s a generic Apple bug even for peo­ple who haven’t re­moved their mo­dem (but anec­do­tally re­mov­ing my mo­dem made the prob­lem worse)

Removing the DCM and GPS may void parts of your war­ranty - just some­thing to be aware of. Thanks to the Magnuson–Moss Warranty Act, it can­not void the whole car war­ranty. It can void cov­er­age re­lated to the work you did (cloud ser­vices, telem­at­ics, etc.) but un­re­lated fail­ures like en­gine prob­lems must still be cov­ered

So thank­fully every­thing in the car re­mains 100% func­tional ex­cept the cloud-based ser­vices men­tioned above, which I did­n’t want any­way. There is also one crit­i­cal caveat about Bluetooth:

No more Bluetooth

Important: Even af­ter the mo­dem is re­moved, if you con­nect your phone to the car via Bluetooth then the car will use your phone as an in­ter­net con­nec­tion and send all the same teleme­try data back to Toyota. However, if you use a wired USB con­nec­tion then it does not do that (see the dis­cus­sion here and else­where), so I ex­clu­sively use CarPlay via USB. I wish I had a way to com­pletely dis­able the car’s Bluetooth func­tion­al­ity, but it’s deeply in­te­grated into the head unit.

If you need USB ca­bles for CarPlay I like these USB-A to Lightning and USB-A to USB-C ca­bles from Anker.

Or, if you pre­fer the con­ve­nience of Bluetooth, you can use a Bluetooth -> wired USB adapter like this one. The adapter re­ceives Bluetooth from your phone and pre­sents it­self to the car as a USB de­vice, so the car treats it like a wired con­nec­tion and won’t tether through your phone.

Now, onto the nec­es­sary tools and parts:

Tools/parts needed

For this pro­ject you’ll need:

A trim re­moval kit (I used this one)

A ratchet, ex­ten­sion, 10mm socket, and 8mm sock­etI’ve been ex­tremely happy with this set. However if you’re not plan­ning on do­ing more handyper­son type work then just bor­row these 4 parts from a neigh­bor in­stead of spend­ing the money on a whole set

I’ve been ex­tremely happy with this set. However if you’re not plan­ning on do­ing more handyper­son type work then just bor­row these 4 parts from a neigh­bor in­stead of spend­ing the money on a whole set

(Optional) A pre­ci­sion flat­head screw­driver (like this one). This can help with dis­con­nect­ing wire plugs

This Telematics DCM Bypass Kit, for fix­ing the in-car mi­cro­phone$90 is a bit steep for a part that prob­a­bly costs less than $1 to pro­duce, but the mak­ers of the kit did the work of read­ing the (paywalled) Toyota di­ag­nos­tics to pro­duce a work­ing prod­uct. If you’d like to build your own ver­sion you’ll need to sub­scribe to Toyota TIS to ac­cess the car wiring schemat­ics. It’s un­for­tu­nate that these schemat­ics and other re­pair man­u­als aren’t pub­lic

$90 is a bit steep for a part that prob­a­bly costs less than $1 to pro­duce, but the mak­ers of the kit did the work of read­ing the (paywalled) Toyota di­ag­nos­tics to pro­duce a work­ing prod­uct. If you’d like to build your own ver­sion you’ll need to sub­scribe to Toyota TIS to ac­cess the car wiring schemat­ics. It’s un­for­tu­nate that these schemat­ics and other re­pair man­u­als aren’t pub­lic

Overall this was a medium-dif­fi­culty pro­ject that took me a few hours to com­plete. Now, let’s get to work:

Removing the car mo­dem

1) Push down on the leather of your shifter and re­move the pin (don’t lose it!):

2) Remove the shifter top:

3) Use the trim tool to pop out the base of the shifter. Just lean it to the side, no need to dis­con­nect any­thing:

4) Use your hands to pop out the next panel and lean it to the side:

5) Remove these three 10mm bolts:

6) Pull on this light gray trim piece un­til it dis­con­nects slightly:

7) Pull the ra­dio out, dis­con­nect the plug, and put the ra­dio aside. The ra­dio is held on by clips only and can even be pulled out with your hands, but it re­quires a lit­tle force and the trim re­moval tool may be help­ful. When dis­con­nect­ing the plug it may help to use the pre­ci­sion screw­driver to push down on the tab to un­lock it, but you can also do it with your hands:

8) Pull the next panel (the seat warm­ing con­trols) out with your hands. It’s only held on by clips but may re­quire a bit of force to re­move:

9) Take a photo of all the wiring con­nec­tions on the seat warm­ing con­trols so you can as­sem­ble it cor­rectly later, un­plug all the wires, and set the con­trols aside:

10) You now have ac­cess to the DCM:

11) Removing the DCM re­quires a lot of ma­neu­ver­ing, tight spaces, and pa­tience, but you can do it. There are two 8mm bolts on the right and one 8mm bolt on the left that need to be re­moved. Getting ac­cess to them may re­quire re­mov­ing some of the other har­nesses or com­po­nents that are in the way - just go slow and steady, take your time, and take pho­tos of things be­fore you move them. After those 3 bolts are re­moved you have a lit­tle more play to pull the unit out, and af­ter dis­con­nect­ing the wires in the back you can com­pletely re­move the DCM. Here’s mine out of the car, part num­ber 86741 – 06130:

12) Now that the mo­dem is re­moved we need to in­stall the DCM Bypass Kit so the in-car mi­cro­phone con­tin­ues to work. It’s ex­tremely straight­for­ward, just plug it into the wiring har­ness that you re­moved from the DCM. The plugs will only fit on the cor­rect wires, there’s no way to get it wrong:

13) Reassemble every­thing by go­ing in re­verse or­der. Make sure all clips, bolts, etc. are back in their orig­i­nal po­si­tion and every­thing is seated cor­rectly. This part should go much faster than dis­as­sem­bly.

Now you’re done with the hard part. Next we dis­con­nect the GPS from the head unit, which is sig­nif­i­cantly eas­ier:

Removing the GPS an­tenna

1) Use the trim tool to re­move the back panel be­hind the in­fo­tain­ment screen:

2) Unscrew these four 10mm bolts:

3) Pop the head unit out (it’s only held on by 2 clips at this point). The part num­ber will vary but for my car it was 86140 – 0R710.

4) The GPS an­tenna is one of the sin­gle-wire ca­bles (not the multi-wire plugs). I had 3 sin­gle-wire ca­bles in my unit and the GPS wire was the black wire shown in the pic­ture. I was able to de­ter­mine this by process of elim­i­na­tion - un­plug­ging one of the wires dis­con­nected my car’s re­verse cam­era, un­plug­ging an­other one dis­con­nected CarPlay com­pletely, and the last one was the GPS - worked like a charm. Again, with a Toyota TIS sub­scrip­tion you can get ac­cess to the head unit wiring di­a­gram and not have to make guesses about which wire is which, but process of elim­i­na­tion worked fine for me:

5) Reassemble every­thing by go­ing in re­verse or­der. Again, make sure that all the clips seat prop­erly.

Confirming it worked

After you have every­thing re­assem­bled, turn the car on.

1) If you un­plugged the mo­dem suc­cess­fully then:

The in­fo­tain­ment screen will have an icon in the up­per right cor­ner in­di­cat­ing no con­nec­tion

The SOS light in the over­head con­sole will be off:

2) If the DCM Bypass Kit was in­stalled suc­cess­fully then:

Make a phone call through CarPlay. The re­cip­i­ent should be able to hear you / the mi­cro­phone should be work­ing

Congratulations - your car no longer has the ca­pa­bil­ity to trans­mit teleme­try data. Of course it may still be cap­tured to lo­cal stor­age and can be phys­i­cally col­lected later, but for me that was fine.

Conclusion

Overall I’m very happy with this pro­ject. Unfortunately I think it’s only a mat­ter of time be­fore the mo­dem and GPS be­come more deeply in­te­grated into the car (making this blog post in­fea­si­ble), or cars have more dras­tic fail­ure modes when the mo­dem/​GPS is re­moved, or anti-right-to-re­pair laws get passed to fur­ther clamp down on this be­hav­ior. For now the win stands - no teleme­try leaves the car. Strong Federal pri­vacy laws would make posts like this un­nec­es­sary, that’s the world I’d rather live in.

On dig­i­tal sov­er­eignty, and why European cloud is bet­ter than you think

April 29, 2026 10 min. Digital SovereigntyDigital InfrastructureDigital AutonomyEuropean CloudEurope

There’s a ver­sion of this post that starts with a spread­sheet and ends with a quiet sense of sat­is­fac­tion. That’s mostly how it went. But un­der­neath the prac­ti­cal ex­er­cise of swap­ping one SaaS tool for an­other was some­thing that felt more ur­gent, a grow­ing dis­com­fort with how much of my dig­i­tal in­fra­struc­ture sat on servers I did­n’t con­trol, in a ju­ris­dic­tion in­creas­ingly prone to un­pre­dictabil­ity, op­er­ated by com­pa­nies whose in­cen­tives don’t al­ways align with mine.

Digital sov­er­eignty sounds like a buzz­word un­til you think care­fully about what it means. It means know­ing where your data lives. It means not be­ing one pol­icy change, one ac­qui­si­tion, or one ex­ec­u­tive’s bad mood away from los­ing ac­cess to tools your busi­ness de­pends on. It means choos­ing in­fra­struc­ture based on val­ues, not just con­ve­nience.

So I started mi­grat­ing.

Analytics

Google Analytics was the ob­vi­ous first tar­get. It’s the canon­i­cal ex­am­ple of a ser­vice that’s free be­cause you are the prod­uct, your vis­i­tors’ be­hav­ior fun­neled back into Google’s ad­ver­tis­ing ma­chin­ery.

Self-hosting Matomo solved this cleanly. The data stays on my own server, and I’m fully GDPR-compliant with­out the cookie con­sent the­ater that Google Analytics typ­i­cally re­quires. The re­port­ing is com­pre­hen­sive, the in­ter­face is fa­mil­iar enough, and I own every­thing.

The main down­side is main­te­nance over­head. You’re now re­spon­si­ble for up­dates, back­ups, and keep­ing the server healthy. For most se­tups this is low-fric­tion, but it’s not zero fric­tion.

Email

Proton Mail is based in Switzerland, not EU ter­ri­tory, but Swiss pri­vacy law is closely aligned with GDPR and ar­guably stronger in some re­spects. Proton builds its busi­ness model around pri­vacy rather than ad­ver­tis­ing, and end-to-end en­cryp­tion is baked in at the pro­to­col level rather than bolted on. The email client is solid, the cal­en­dar works well, and for any­one mov­ing away from US-based ser­vices, it sits com­fort­ably in the same spirit as the rest of this stack.

One ad­just­ment is get­ting used to Proton’s fil­ter sys­tem, which is a bit more lim­ited than Gmail’s. Gmail lets you write fil­ters against vir­tu­ally any­thing, in­clud­ing the full body of the mes­sage. Proton does­n’t sup­port fil­ter­ing on email con­tent at all. So if you’ve built a work­flow around catch­ing spe­cific phrases or key­words in mes­sage bod­ies, you’ll have to re­think it. For most peo­ple this won’t be a deal­breaker, but it’s worth know­ing be­fore you mi­grate.

There’s also a prac­ti­cal lim­i­ta­tion worth flag­ging: Proton caps cus­tom do­mains at three, even on the Duo plan. If you run sev­eral do­mains, like sep­a­rate ad­dresses for dif­fer­ent pro­jects or busi­nesses, you’ll hit that ceil­ing quickly and need to re­think how you route and send mail. I ended up con­sol­i­dat­ing, which was prob­a­bly over­due any­way, but it was­n’t a choice I made en­tirely freely.

Proton is­n’t free and charges a sub­stan­tial fee com­pared to other op­tions. You’ll get ac­cess to a whole suite of Proton apps though.

Password Management

Once I was in the Proton ecosys­tem, mov­ing pass­word man­age­ment there as well made sense. Proton Pass is end-to-end en­crypted, open source, and ben­e­fits from the same Swiss ju­ris­dic­tion as the rest of Proton’s stack.

1Password is a gen­uinely great prod­uct and this was a lat­eral move more than an up­grade. The in­ter­face is sim­ple, the browser ex­ten­sion works re­li­ably, and hav­ing pass­words, email, and cal­en­dar un­der one en­crypted roof has a cer­tain sat­is­fy­ing co­her­ence to it.

Compute

DigitalOcean has earned its rep­u­ta­tion by do­ing one thing ex­cep­tion­ally well: get­ting out of your way. The UI is clean, the men­tal model is sim­ple, and spin­ning up in­fra­struc­ture never feels like a chore. It’s the plat­form that proved de­vel­oper ex­pe­ri­ence could be a com­pet­i­tive moat.

Scaleway was a pleas­ant sur­prise. I ex­pected a ca­pa­ble-but-rough European al­ter­na­tive, but what I found was a plat­form that’s gen­uinely well thought out. Servers spun up quickly in­side a pri­vate net­work of my own con­fig­u­ra­tion, the con­trol panel is clean, and the op­tions avail­able matched every­thing I ac­tu­ally needed. Scaleway dis­plays pro­jected CO₂ emis­sions along­side server lo­ca­tion choices, a nice touch.

Object Storage

Scaleway’s ob­ject stor­age is S3-compatible, which makes mi­gra­tion me­chan­i­cal rather than painful, up­date your end­point and cre­den­tials and ex­ist­ing code works un­changed.

I used a tool called rclone to sync my old AWS S3 stor­age buck­ets to the new Scaleway S3 buck­ets. This took a lit­tle more than a week of con­stant sync­ing, as these buck­ets were quite large.

Offsite Backups

OVH is the largest European cloud provider and brings the re­li­a­bil­ity and pric­ing you’d ex­pect at that scale. Their ob­ject stor­age works well as a backup des­ti­na­tion and ends up cheaper than Backblaze B2 once you con­fig­ure life­cy­cle rules to move older back­ups to the cold stor­age class.

Getting there, how­ever, re­quires some pa­tience. The OVHcloud con­trol panel is a labyrinth: the life­cy­cle rule con­fig­u­ra­tion is buried some­where in the doc­u­men­ta­tion, and it in­volves some work in the ter­mi­nal. Once it’s set up, it works re­li­ably and the cost dif­fer­ence is mean­ing­ful.

Transactional Emails

Lettermint is a European trans­ac­tional email ser­vice that does the job with­out the bloat. Deliverability is solid, the API is clean, and it has straight­for­ward pric­ing.

Compared to SendGrid, the an­a­lyt­ics are leaner and the ecosys­tem in­te­gra­tions are fewer. SendGrid has years of tool­ing, doc­u­men­ta­tion, and com­mu­nity an­swers be­hind it. Lettermint is newer and smaller. For most trans­ac­tional send­ing use cases (password re­sets, no­ti­fi­ca­tions, re­ceipts) that does­n’t mat­ter much. But if you’re do­ing com­plex multi-stream email in­fra­struc­ture, you’ll want to au­dit the fea­ture set care­fully first.

Error Tracking

Bugsink is a self-hosted er­ror track­ing tool that ac­cepts Sentry’s SDK, which means the mi­gra­tion path is al­most fric­tion­less, change one line of con­fig­u­ra­tion and you’re done.

To be hon­est: Bugsink is bare-bones. There’s no per­for­mance mon­i­tor­ing, no ses­sion re­plays, no ad­vanced alert­ing. It’s not a Sentry re­place­ment for teams that use Sentry prop­erly. For me, it’s a sim­ple re­mote er­ror log, when some­thing breaks in pro­duc­tion I get a stack trace and that’s enough. Sentry’s cloud prod­uct is gen­uinely ex­cel­lent if you need the full fea­ture set, and for larger en­gi­neer­ing teams the breadth al­most cer­tainly jus­ti­fies the cost. But if your use case is “tell me when some­thing broke and show me the stack trace”, self-hosted Bugsink does ex­actly that with no data leav­ing your in­fra­struc­ture.

AI API in­te­gra­tion

For my AI API in­te­gra­tions, I switched from OpenAI to Mistral. It worked out per­fectly as I was mostly us­ing sim­pler mod­els any­way.

Mistral is head­quar­tered in Paris and has pub­lished com­pelling open-weight mod­els along­side its API of­fer­ing. The API is clean, the mod­els are fast and ca­pa­ble, and there’s some­thing co­her­ent about a European AI provider that leans into open­ness rather than away from it. For my in­fer­ence work­loads, the switch was lat­eral in qual­ity and mean­ing­fully bet­ter in terms of where the money goes.

CDN

Exception № 1

Not every­thing moved. Cloudflare is a US com­pany, I still use it, and I’m at peace with that.

Here’s the rea­son­ing: Cloudflare sits in front of my pub­lic-fac­ing web­sites. Its job is to cache, pro­tect against DDoS at­tacks, and make con­tent load fast for vis­i­tors around the world. The data flow­ing through it is al­ready pub­lic by de­f­i­n­i­tion. I’m not rout­ing pri­vate com­mu­ni­ca­tions or sen­si­tive ap­pli­ca­tion data through Cloudflare; I’m us­ing it to serve pages that any­one on the in­ter­net can read. The sov­er­eignty cal­cu­lus is dif­fer­ent when the thing you’re pro­tect­ing is al­ready pub­lic.

I did try Bunny CDN, which is European-based and has a great rep­u­ta­tion. For straight­for­ward CDN use it’s ex­cel­lent. But Cloudflare’s fea­ture set (security rules, Workers plat­form, breadth of con­fig­u­ra­tion op­tions) was­n’t matched closely enough to jus­tify the switch for my spe­cific needs. Sometimes the prag­matic an­swer wins.

Payments

Exception № 2

Stripe is one of the few ser­vices I haven’t moved yet, even though pay­ment in­fra­struc­ture is ex­actly the kind of thing I care about hav­ing in a ju­ris­dic­tion I trust. Mollie is a Dutch pay­ment proces­sor with full EU in­cor­po­ra­tion, strong GDPR com­pli­ance by de­sign, and a prod­uct that has ma­tured con­sid­er­ably in re­cent years. The API has con­verged to­ward par­ity for most com­mon pay­ment flows, and for a European busi­ness the re­gional pay­ment method cov­er­age (iDEAL, Bancontact, SEPA) is ar­guably bet­ter.

The mi­gra­tion is on the list. It’s just not a triv­ial one. Payment in­te­gra­tions touch billing logic, web­hooks, tax in­voic­ing and cus­tomer-fac­ing flows in ways that re­quire care­ful test­ing and a good mo­ment to cut over. It’s also more ex­pen­sive than Stripe for my use­case.

AI Code as­sis­tance

Exception № 3

This one felt over­due. OpenAI works fine, but the com­pa­ny’s tra­jec­tory does­n’t align with my own views any­more. After a pe­riod of de­lib­er­ate drift, I felt the need to switch. Ideally I wanted to use Mistral Vibe here, but it just did­n’t make the cut as it could­n’t com­pete with Claude.

Claude Code is now my day-to-day AI as­sis­tant for cod­ing. The rea­son­ing qual­ity is strong, the con­text han­dling is gen­uinely im­pres­sive, and Anthropic’s ap­proach to safety and trans­parency feels more struc­turally grounded.

Anthropic is a US com­pany, so this does­n’t sat­isfy the ju­ris­dic­tional cri­te­rion I ap­plied else­where. But it sat­is­fies some­thing else, the sense that the or­ga­ni­za­tion build­ing the thing has given se­ri­ous thought to what it’s build­ing and why.

It’s also worth not­ing that lo­cal mod­els are be­com­ing in­creas­ingly vi­able. Qwen, Alibaba’s open-weight model fam­ily, is a strong ex­am­ple: ca­pa­ble enough for many real work­loads, run­ning en­tirely on your own hard­ware, with no data leav­ing your ma­chine. The gap be­tween fron­tier API mod­els and what you can run lo­cally is nar­row­ing faster than most peo­ple re­al­ize.

Not every­thing is ideal. Most data cen­ters still sit out­side Europe, and “open” means dif­fer­ent things to dif­fer­ent or­ga­ni­za­tions. But the di­rec­tion is right. A world where ca­pa­ble AI runs on your own hard­ware, with pub­lished weights and trans­par­ent train­ing, is a much bet­ter world for dig­i­tal au­ton­omy than one where all in­fer­ence routes through a hand­ful of closed API providers. We’re not there yet, but the tra­jec­tory is en­cour­ag­ing.

Git Version Control

Exception № 4

GitLab also re­mains for now. GitLab is head­quar­tered in the US but of­fers self-hosted op­tions, and the com­pany has long had a strong com­mit­ment to trans­parency and open source. A self-hosted in­stance is on the roadmap, but mov­ing source con­trol is a more sig­nif­i­cant un­der­tak­ing than most of these mi­gra­tions.

GitHub stays in the pic­ture for one spe­cific pur­pose: pub­lic-fac­ing NPM pack­ages and is­sue track­ing for open source soft­ware. When you pub­lish a pack­age or main­tain pub­lic tool­ing, GitHub is where de­vel­op­ers ex­pect to find it. The net­work ef­fects are real, it’s where the forks, stars, and is­sue re­ports come from. For the pub­lic-fac­ing sur­face of open source work, there’s no mean­ing­ful sov­er­eignty con­cern and a lot of prac­ti­cal up­side.

Was it worth it?

The prac­ti­cal fric­tion was real but man­age­able. Most mi­gra­tions were an af­ter­noon of work: up­date a cre­den­tial here, point a DNS record there, ex­port and im­port some data. A few took longer. None were cat­a­strophic. All in all it took longer than ex­pected, but most time was spent in re­search­ing and plan­ning when to do what. Two months in, every­thing is run­ning with­out in­ci­dent. No fires, no re­grets.

Digital sov­er­eignty is­n’t about para­noia. It’s about be­ing con­scious about your in­fra­struc­ture, where you de­cide who holds your data, who can reach it, and what hap­pens when pol­i­tics shift. The tools are there. The ecosys­tem is mostly ma­ture. The only thing that was stop­ping me was in­er­tia. It’s en­tirely pos­si­ble to run a re­li­able, ca­pa­ble, pro­fes­sional dig­i­tal stack mostly from European in­fra­struc­ture. This mi­gra­tion was proof of that.

Published May 10, 2026, 12:30 PM EDT

His love of PCs and their com­po­nents was born out of try­ing to squeeze every ounce of per­for­mance out of the fam­ily com­puter. Tinkering with his own build at age 10 turned into build­ing PCs for friends and fam­ily, fos­ter­ing a pas­sion that would ul­ti­mately take shape as a ca­reer path.

Besides be­ing the first call for tech sup­port for those close to him, Ty is a com­puter sci­ence stu­dent, with his fo­cus be­ing cloud com­put­ing and net­work­ing. He also com­peted in semi-pro Counter-Strike for 8 years, mak­ing him in­ti­mately fa­mil­iar with every­thing to do with pe­riph­er­als.

Sign in to your XDA ac­count

In March 2026, Linux crossed five per­cent of Steam’s user base for the first time, an all-time high for an op­er­at­ing sys­tem that spent two decades as a nov­elty when it came to any kind of gam­ing. Microsoft’s end-of-sup­port dead­line for Windows 10 last October pushed many users to look at al­ter­na­tives, and the Steam Deck has qui­etly turned mil­lions of peo­ple into Linux gamers with­out them re­ally think­ing about it, lead­ing to more wide­spread adop­tion on desk­top ma­chines.

Most of that progress used to hap­pen in­side a piece of soft­ware called Wine, the trans­la­tion layer that con­vinces Windows games they’re run­ning on Windows. Valve’s tuned ver­sion of Wine, called Proton, is what makes Steam Play and the Steam Deck work. For years, every mean­ing­ful im­prove­ment to Linux gam­ing came from changes to Wine and Proton them­selves. That’s still true, but in­creas­ingly the most im­por­tant changes are hap­pen­ing one layer deeper, in­side the Linux ker­nel. The lat­est ex­am­ple of that is some­thing called NTSYNC, a ker­nel-level dri­ver that has of­fered great per­for­mance gains over pre­vi­ous ver­sions of Wine, and is loaded by de­fault on every Steam Deck that’s up-to-date.

Related

What NTSYNC ac­tu­ally is

An ad­di­tional piece of the per­for­mance puz­zle

NTSYNC is a small piece of dri­ver added di­rectly to the Linux ker­nel that gives it a na­tive im­ple­men­ta­tion of a set of Windows-specific tools that games de­pend on to co­or­di­nate them­selves.

Modern games jug­gle dozens of things at once. While you’re play­ing, your CPU man­ages the ren­der­ing pipeline, load­ing as­sets, run­ning physics, pro­cess­ing au­dio, han­dling AI NPC rou­tines, and track­ing in­puts, all in par­al­lel across mul­ti­ple cores. All those jobs con­stantly have to co­or­di­nate so they don’t trip over each other.

Quiz

8 Questions · Test Your Knowledge

The his­tory of LinuxTrivia chal­lenge

From a Finnish stu­den­t’s side pro­ject to pow­er­ing the world — how well do you know the story of Linux?

OriginsKernelDistrosPioneersMilestones

Begin

01 / 8

Origins

In what year did Linus Torvalds first an­nounce the Linux ker­nel to the world?

A1989B1991C1993D1995

Correct! Linus Torvalds posted his now-fa­mous mes­sage to the comp.os.minix news­group on August 25, 1991, de­scrib­ing Linux as ‘just a hob­by’ pro­ject. Few could have pre­dicted it would one day run the ma­jor­ity of the world’s servers and smart­phones.

Not quite — Torvalds made his an­nounce­ment in 1991. He was a 21-year-old com­puter sci­ence stu­dent at the University of Helsinki at the time, and his mod­est post de­scribed the pro­ject as some­thing that ‘won’t be big and pro­fes­sion­al’ like GNU.

Continue

02 / 8

Pioneers

Which uni­ver­sity was Linus Torvalds at­tend­ing when he cre­ated the first ver­sion of the Linux ker­nel?

AStockholm UniversityBAalto UniversityCUniversity of HelsinkiDMIT

Correct! Torvalds was study­ing at the University of Helsinki in Finland when he be­gan work­ing on Linux as a per­sonal pro­ject, ini­tially in­spired by MINIX, a small Unix-like sys­tem used for ed­u­ca­tional pur­poses.

Not quite — Torvalds was a stu­dent at the University of Helsinki in Finland. He started Linux partly out of frus­tra­tion with the lim­i­ta­tions of MINIX, which his pro­fes­sor Andrew Tanenbaum had de­signed de­lib­er­ately to be sim­ple for teach­ing.

Continue

03 / 8

Kernel

What op­er­at­ing sys­tem pri­mar­ily in­spired Linus Torvalds to cre­ate the Linux ker­nel?

AMS-DOSBMINIXCBSD UnixDSolaris

Correct! MINIX, cre­ated by pro­fes­sor Andrew Tanenbaum, was the di­rect in­spi­ra­tion for Linux. Torvalds used MINIX on his new Intel 386 PC but found it too re­stricted for his needs, which pushed him to write his own ker­nel.

Not quite — the an­swer is MINIX. Torvalds was us­ing MINIX when he started Linux, and even held a fa­mous on­line de­bate with its cre­ator Andrew Tanenbaum about ker­nel de­sign phi­los­o­phy, specif­i­cally mono­lithic ver­sus mi­cro­ker­nel ar­chi­tec­tures.

Continue

04 / 8

Milestones

What was the ver­sion num­ber of the first pub­licly re­leased Linux ker­nel in 1991?

A0.01B0.1C1.0D0.99

Correct! Linux ver­sion 0.01 was the first ker­nel Torvalds re­leased pub­licly in September 1991. It was a rough, early build that could only run on Intel 386 hard­ware and had very lim­ited func­tion­al­ity, but it marked the true be­gin­ning of the Linux pro­ject.

Not quite — the first pub­lic re­lease was ver­sion 0.01 in September 1991. The ker­nel did­n’t reach ver­sion 1.0 un­til March 1994, by which point it had grown sig­nif­i­cantly in ca­pa­bil­ity and had at­tracted con­tri­bu­tions from de­vel­op­ers around the world.

Continue

05 / 8

Distros

Which Linux dis­tri­b­u­tion, first re­leased in 1993, is one of the old­est still ac­tively main­tained to­day?

AUbuntuBFedoraCSlackwareDDebian

Correct! Slackware, cre­ated by Patrick Volkerding, was first re­leased in July 1993, mak­ing it one of the old­est sur­viv­ing Linux dis­tri­b­u­tions. It is known for its sim­plic­ity and Unix-like phi­los­o­phy, and it con­tin­ues to be main­tained to this day.

Not quite — the an­swer is Slackware, re­leased in 1993 by Patrick Volkerding. While Debian was also founded in 1993, Slackware nar­rowly edges it out as the older re­lease. Ubuntu did­n’t ar­rive un­til 2004, and Fedora launched in 2003.

Continue

06 / 8

Origins

The GNU Project, which pro­vided many tools that paired with the Linux ker­nel, was founded by which de­vel­oper?

AEric RaymondBRichard StallmanCBruce PerensDIan Murdock

Correct! Richard Stallman founded the GNU Project in 1983 with the goal of cre­at­ing a com­pletely free Unix-like op­er­at­ing sys­tem. When the Linux ker­nel ap­peared in 1991, it filled the miss­ing piece GNU needed, and the com­bi­na­tion be­came what many call GNU/Linux.

Not quite — it was Richard Stallman who founded the GNU Project in 1983. Stallman is also known for cre­at­ing the GPL (GNU General Public License) and found­ing the Free Software Foundation, two pil­lars that shaped the le­gal and philo­soph­i­cal foun­da­tion of free soft­ware.

Continue

07 / 8

Milestones

Which com­pany re­leased a land­mark com­mer­cial Linux dis­tri­b­u­tion in 1994, help­ing bring Linux into the en­ter­prise world?

ACanonicalBSUSECRed HatDMandriva

Correct! Red Hat re­leased its first Linux dis­tri­b­u­tion in 1994 and be­came one of the most in­flu­en­tial com­mer­cial Linux com­pa­nies in his­tory. It pi­o­neered the en­ter­prise Linux mar­ket and was even­tu­ally ac­quired by IBM in 2019 for ap­prox­i­mately $34 bil­lion.

Not quite — Red Hat is the an­swer. Founded by Marc Ewing and Bob Young, Red Hat helped prove that com­pa­nies could build sus­tain­able busi­nesses around open-source soft­ware. SUSE Linux also launched in 1994, mak­ing it a close ri­val, but Red Hat be­came the more glob­ally dom­i­nant en­ter­prise force.

Continue

08 / 8

Distros

Ubuntu Linux, one of the most pop­u­lar desk­top dis­tri­b­u­tions, is based on which other Linux dis­tri­b­u­tion?

AArch LinuxBFedoraCDebianDGentoo

Correct! Ubuntu is based on Debian and was first re­leased in October 2004 by Mark Shuttleworth’s com­pany Canonical. It was de­signed to make Linux more ac­ces­si­ble to every­day users, and its six-month re­lease cy­cle and long-term sup­port ver­sions made it a fa­vorite for both desk­tops and servers.

Not quite — Ubuntu is built on top of Debian. Debian it­self was founded in 1993 by Ian Murdock and is known for its strict com­mit­ment to free soft­ware and sta­bil­ity. Ubuntu in­her­its Debian’s pack­age man­age­ment sys­tem (APT and .deb pack­ages) but adds its own user-friendly layer on top.

See My Score

Challenge Complete

Your Score

/ 8

Thanks for play­ing!

Try Again

Windows han­dles this co­or­di­na­tion by us­ing a spe­cific set of mech­a­nisms, and be­fore NTSYNC, Wine had to mimic these mech­a­nisms us­ing things like esync and fsync, which both worked, but did­n’t al­ways match Windows ex­actly. NTSYNC builds these mech­a­nisms straight into the Linux ker­nel for the first time, and it means Wine does­n’t have to em­u­late any­thing any­more. The de­vel­oper-fac­ing API calls don’t ac­tu­ally change, Linux just knows how to an­swer them na­tively.

Related

NTSYNC is part of a grow­ing pat­tern

Not the first time Linux has in­her­ited fea­tures be­cause of Windows

NTSYNC is­n’t the first time Linux has gained a new fea­ture specif­i­cally be­cause Windows games needed it. A few years back, Linux added a way for soft­ware to wait on sev­eral events at once, which is some­thing Windows had built in for decades, but Linux did­n’t. Wine had been work­ing around the gap with awk­ward tricks un­til the ker­nel fi­nally got na­tive sup­port.

This work is dri­ven by Valve, by CodeWeavers (the com­pany that em­ploys many of the core Wine de­vel­op­ers, in­clud­ing NTSYNC’s au­thor Elizabeth Figura), and by a steady stream of con­trib­u­tors who want Linux to be a real gam­ing plat­form with­out de­pend­ing on out-of-ecosys­tem patches for­ever.

Related

These aren’t mag­i­cal per­for­mance gains

fsync was al­ready pretty good

The head­line per­for­mance gains look great, but they need some con­text. The eye-catch­ing 40 to 200 per­cent FPS gains cited in NTSYNC’s orig­i­nal bench­marks were mea­sured against un­mod­i­fied up­stream Wine, which al­most no­body uses to play games on Linux any­more. Most Linux gamers, in­clud­ing every Steam Deck owner, use Proton, which al­ready has fsync. Compared to fsync, NTSYNC’s per­for­mance gains are far more mod­est. The games that ben­e­fit most from the change to NTSYNC are games that were re­ally strug­gling be­fore. Anything that was run­ning at de­cent fram­er­ates be­fore­hand is still go­ing to run fine.

Related

These 7 Linux myths you still be­lieve sim­ply aren’t true

Linux is a com­pletely dif­fer­ent beast than it was a decade ago.

Valve adopted it any­way

It’s a great sign

Pierre-Loup Griffais, an en­gi­neer at Valve, has gone on the record to say that fsync was al­ready fast enough, and de­spite that, Valve still shipped NTSYNC in sta­ble SteamOS in March any­way, which speaks to the fact that fsync is still a workaround at its core, and can be the cause of is­sues out­side of poor raw FPS.

These old workarounds got sub­tle edge cases wrong in ways that pro­duced oc­ca­sional hitches, dead­locks, or weird be­hav­ior in spe­cific games, which are bugs that don’t show up on bench­mark charts but can ab­solutely ruin in­di­vid­ual ex­pe­ri­ences. NTSYNC fixes those at the source by match­ing Windows be­hav­ior ex­actly, and that means as soon as your fa­vorite dis­tro moves to the new ker­nel ver­sion, whether it be Bazzite, CachyOS, Fedora, or a fla­vor of Ubuntu, they all get this much-needed fix.

Related

4 rea­sons Valve’s full SteamOS re­lease will change PC gam­ing again

Valve’s full SteamOS re­lease will change PC gam­ing again, and here are some of the most im­por­tant ways.

Gaming on Linux con­tin­ues to im­prove by the month

Linux has grown so much in the gam­ing de­part­ment. Where there once was noth­ing but clever Wine patches and com­mu­nity workarounds now lies sup­port from gam­ing be­he­moths like Valve, dri­ving changes to the Linux ker­nel it­self. NTSYNC won’t be the last time a piece of Windows gets re­built in­side Linux be­cause gamers needed it, and with more than five per­cent of Steam’s user base now run­ning Linux, the in­cen­tive to keep do­ing it has never been stronger.

Be in the know.

Sorry, some­thing went wrong. Please en­ter your name and email again.

I con­firm I am 18 years of age or older. I ac­cept and ac­knowl­edge that my in­for­ma­tion will be used in ac­cor­dance with

Be in the know.

Sorry, some­thing went wrong. Please en­ter your name and email again.

I con­firm I am 18 years of age or older. I ac­cept and ac­knowl­edge that my in­for­ma­tion will be used in ac­cor­dance with

Intelligence is the new spec.

Link to Youtube Video (visible only when JS is dis­abled)

The best of Gemini meets our most ad­vanced lap­tops.

Select any­thing to ask, com­pare, or cre­ate with Gemini, in­stantly.1

Open your phone apps on your lap­top, no in­stalls needed.2

Access files from your phone as if they live on your lap­top.2

Be in the know.

Sorry, some­thing went wrong. Please en­ter your name and email again.

I con­firm I am 18 years of age or older. I ac­cept and ac­knowl­edge that my in­for­ma­tion will be used in ac­cor­dance with

Check re­sponses. Internet con­nec­tion re­quired. 18+. Results may vary based on vi­sual matches and are for il­lus­tra­tive pur­poses only. Sequences short­ened.

Setup re­quired. Phone with Android 17 or above re­quired.

In the con­text of con­cerns around young peo­ple’s in­ter­ac­tions with dig­i­tal tech­nolo­gies, the UK’s Department for Science, Innovation and Technology is con­sult­ing on ad­di­tional mea­sures to pre­pare young peo­ple for grow­ing up in a dig­i­tal world. Before the back­drop of users cir­cum­vent­ing age as­sur­ance sys­tems man­dated un­der the UK’s Online Safety Act, the con­sul­ta­tion con­sid­ers age-gat­ing vir­tual pri­vate net­works (VPNs).

Mozilla’s mis­sion is grounded in the be­lief that the in­ter­net must re­main open and ac­ces­si­ble to all, and that pri­vacy and se­cu­rity on­line are fun­da­men­tal hu­man rights. We rec­og­nize that the pro­tec­tion of young peo­ple on­line is one of the most press­ing and chal­leng­ing ques­tions of our time, and we are com­mit­ted to sup­port­ing pol­icy pro­pos­als that ad­dress the root causes of on­line harms. We are con­cerned, how­ever, that blunt in­ter­ven­tions like manda­tory age as­sur­ance and re­strict­ing ac­cess to tools like VPNs are not ef­fec­tive in im­prov­ing the pro­tec­tion af­forded to young peo­ple on­line, while un­der­min­ing the fun­da­men­tal rights of all users.

VPNs serve as crit­i­cal pri­vacy and se­cu­rity tools for users across all ages. By hid­ing users’ IP ad­dresses, VPNs help pro­tect users’ lo­ca­tion, re­duce track­ing and avoid IP-based pro­fil­ing. People use VPNs for lots of dif­fer­ent rea­sons: to con­nect to their school’s or em­ploy­er’s net­work re­motely, to avoid cen­sor­ship or to sim­ply pro­tect their pri­vacy and se­cu­rity on­line. While be­ing able to ac­cess VPNs is es­pe­cially im­por­tant for vul­ner­a­ble groups like ac­tivists, dis­si­dents or jour­nal­ists, VPNs im­prove every­one’s base­line pro­tec­tion on­line.

Young peo­ple are par­tic­u­larly vul­ner­a­ble to on­line track­ing, tar­geted ad­ver­tis­ing, and the risks that flow from per­sonal data be­ing col­lected and processed for com­mer­cial pur­poses with­out ad­e­quate con­sent or trans­parency. In a world in which young peo­ple are in­ter­act­ing with dig­i­tal tech­nolo­gies as part of their re­al­i­ties from young ages on­ward, re­strict­ing young peo­ple’s ac­cess to pri­vacy-pro­tect­ing tech­nolo­gies is in ten­sion with the goal of equip­ping them to nav­i­gate the in­ter­net safely and com­pe­tently. In or­der to be able to de­velop agency and re­spon­si­ble habits in en­gag­ing with dig­i­tal tech­nolo­gies, it is cru­cial for young peo­ple to be in­tro­duced to best prac­tices and key safety and pri­vacy tools as they en­gage with the on­line world.

Rather than age-gat­ing tech­nolo­gies like VPNs, we be­lieve that reg­u­la­tors should ad­dress the root causes of on­line harm by hold­ing plat­forms to ac­count, en­cour­ag­ing the re­spon­si­ble use of parental con­trols and in­vest­ing in dig­i­tal skills and a whole of so­ci­ety ap­proach to dig­i­tal well­be­ing.

Read our full sub­mis­sion to the Department for Science, Innovation and Technology.

Back in March, I wrote about Bitwarden dou­bling their Premium price — and specif­i­cally how they did it. Buried in a fea­ture an­nounce­ment. Priced in fake monthly in­cre­ments for a prod­uct that has never once of­fered monthly billing. Communicated to ex­ist­ing cus­tomers fif­teen days be­fore their re­newal, not be­fore.

Bitwarden re­sponded on Mastodon. They con­firmed every­thing in my post while ap­par­ently think­ing they were de­fend­ing them­selves. I noted at the time that the re­sponse was its own data point.

Well. There’s more data now.

The Changing of the Guard

In February, as Fast Company re­ported, long­time CEO Michael Crandell qui­etly tran­si­tioned to an ad­vi­sory role. No an­nounce­ment from the com­pany. You’d only know it hap­pened if you went look­ing on LinkedIn. Crandell had been with Bitwarden since 2019 — back when they were still the scrappy un­der­dog that every­one flocked to when LastPass started pulling the rug.

His re­place­ment is Michael Sullivan, for­mer CEO of Acquia and Insightsoftware. Sullivan’s LinkedIn page leads with his ex­pe­ri­ence in “all facets of merg­ers and ac­qui­si­tions, in­clud­ing di­rect ex­pe­ri­ence with lead­ing PE firms.”

In plain English: M&A is the busi­ness of buy­ing and sell­ing com­pa­nies. Private eq­uity firms buy busi­nesses, cut costs, grow rev­enue, and sell them at a profit. They’re not there to run a soft­ware com­pany long-term — they’re man­ag­ing an in­vest­ment to­ward an exit. The peo­ple hired to run those com­pa­nies are hired specif­i­cally be­cause they know how that process works.

That’s the new CEO of your pass­word man­ager. That’s what he leads with.

CFO Stephen Morrison also de­parted in April, re­placed by for­mer InVision CEO Michael Shenkman. Kyle Spearrin — who started build­ing Bitwarden as a hobby pro­ject in 2015 be­cause he was wor­ried about what would hap­pen to LastPass un­der new own­er­ship — re­mains as CTO.

The irony is al­most too much to type.

The Website Is Remodeling Too

The phrase “Always free” dis­ap­peared from the per­sonal pass­word man­ager page in mid-April. It used to sit promi­nently un­der the plan se­lec­tor. The free plan still ex­ists — for now — but the com­mit­ment lan­guage is gone.

And then there’s the val­ues rewrite.

Bitwarden used to de­fine its cul­ture with the acronym GRIT: Gratitude, Responsibility, Inclusion, and Trans­parency. After May 4th, that changed. GRIT now stands for Gratitude, Responsibility, Innovation, and Trust.

Inclusion and Transparency are out. Innovation and Trust are in.

Did They Announce Any of This?

I looked hard.

Their blog has noth­ing about the new CEO. No press re­lease about the val­ues change. No ded­i­cated post about “Always free” be­ing re­tired as a promise. The press room is silent on all of it.

There is one thing. A 2022 blog post by Crandell — “Defining and sus­tain­ing value for Bitwarden users” — was qui­etly edited. The GRIT list in the body now shows the new val­ues: Innovation and Trust. But the ex­plana­tory para­graph at the bot­tom of the same post still says the old ones: Inclusion and Transparency. Crandell’s name is still on it. The post now con­tra­dicts it­self, and no­body wrote a new one.

That’s their an­nounce­ment. A half-scrubbed edit of a four-year-old post they did­n’t even fin­ish up­dat­ing. Same play­book as the price hike — bury it in ex­ist­ing con­tent, don’t draw at­ten­tion, hope no­body reads closely enough to no­tice.

Somebody al­ways does.

And since we’re here — in a 2024 in­ter­view, Crandell told Fast Company the free tier was “a firm com­mit­ment from the com­pany. Fully fea­tured, free for­ever.”

He’s in an ad­vi­sory role now. “Always free” is­n’t on the page.

I’ve Already Moved On

My Vaultwarden in­stance has been run­ning since January. The Bitwarden cloud ac­count is closed — I shut it down around the time that last post went live. I’m not watch­ing this be­cause I’m wor­ried about my own pass­words. I’m watch­ing it be­cause this is what I doc­u­ment.

The pat­tern is al­ways the same: build trust, es­tab­lish de­pen­dency, then qui­etly rene­go­ti­ate the terms. And it never comes in a sin­gle dra­matic an­nounce­ment. It comes in lay­ers. A fea­ture post with a price change in­side it. A LinkedIn up­date no­body made a press re­lease about. A val­ues page that says some­thing slightly dif­fer­ent than it did last week.

If you’re still on Bitwarden cloud and this is giv­ing you pause — it should. I wrote about the GitHub ver­sion of this story in March — trusted open source plat­form, promises of in­de­pen­dence, years of quiet ero­sion, then Phase 3. The par­al­lel is close enough to make you ner­vous. And if you want to ac­tu­ally own your vault rather than wait and see: here’s how I did it.

My read on where this is go­ing: Sullivan’s en­tire ca­reer is tak­ing com­pa­nies to an exit. Maximize rev­enue, clean up the bal­ance sheet, make the num­bers at­trac­tive, find a buyer — a big tech com­pany, a ri­val like 1Password, some­one who wants the user base or the en­ter­prise con­tracts. That’s what you hire this pro­file of CEO to do. And if that hap­pens, the hard forks won’t be a ques­tion. The price hike got grum­bling. Watching your pass­word man­ager get swal­lowed by a com­pany you switched away from­would kick them off prop­erly.

A Note for Vaultwarden Users

Whether self-host­ing stays vi­able long-term is the real ques­tion worth sit­ting with.

Right now it works be­cause Bitwarden’s clients are open source and the server API is pub­lic. Vaultwarden im­ple­ments that API, and the of­fi­cial apps can’t tell the dif­fer­ence. That de­pends on Bitwarden con­tin­u­ing to pub­lish open source clients and not re­strict­ing which servers they’ll talk to — nei­ther of which is guar­an­teed un­der new man­age­ment.

The brake on the worst case: self-host­ing is a listed Enterprise fea­ture that gen­er­ates real rev­enue. Killing it up­sets pay­ing busi­ness cus­tomers. That mat­ters.

The catch: what Bitwarden sells to en­ter­prises is their own of­fi­cial server stack, not Vaultwarden. Vaultwarden ex­ists in a space they’ve tol­er­ated but never en­dorsed. If the cal­cu­lus shifts, the tol­er­ance ends with­out any an­nounce­ment. Just let the API drift un­til com­pat­i­bil­ity breaks on its own.

I don’t think that’s im­mi­nent. But I also thought the free tier com­mit­ment was iron­clad, and “Always free” is­n’t on the page any­more.

The real safety net is that Bitwarden’s clients are Apache 2.0 li­censed. A fork would need a re­brand to stay clear of the trade­mark — dif­fer­ent name, tweaked UI, same en­gine — but that’s a speed bump, not a wall. The web vault works through any browser re­gard­less of what hap­pens to the apps, so worst case you’d lose aut­ofill tem­porar­ily while a fork caught up. Inconvenient, not cat­a­strophic. Vaultwarden it­self is al­ready proof the model works.

Watch the clients. If they go closed, the com­mu­nity will no­tice fast, and the fork will fol­low.

Private, quiet space for think­ing. A sim­ple app for your .md files.

You can store your whole life:

📌 Notes

📝 Documents, Projects

💚 Journal, Habits

✅ Checklists, Tasks

All in plain .md files, lo­cal-first. LLM-friendly. Private - no data is sent to server.

You should own your files, and the soft­ware that opens them. You grow your knowl­edge with your brain. You grow soft­ware around it with an LLM. Both can last through the ages.

You should own your files, and the soft­ware that opens them. You grow your knowl­edge with your brain. You grow soft­ware around it with an LLM. Both can last through the ages.

Try it out: app.files.md (Beta). Main site: files.md.

I have been build­ing this pro­ject for 5 years. Consider spon­sor­ing it on GitHub 💚.

Dump your thoughts · How to think deeply · Second Brain?

Another note tak­ing app?

Maybe. But this time:

Only nec­es­sary fea­tures, re­stric­tions fos­ter cre­ativ­ity

No need to in­stall any­thing, all you need is a browser

Works of­fline

Local-first, files don’t leave your de­vice

Free and open source

Extremely sim­ple code. One per­son or an LLM can fit the whole pro­ject in head

The code­base is ready for your LLM to ex­tend to your needs

Portable, no build sys­tems, just open web/​in­dex.html

Optional out of the box syn­chro­niza­tion

The server is just one bi­nary (or use iCloud/​Drop­box/​Google Drive for sync)

Telegram chat­bot for on-the-go ac­cess to your files

How to use

Open app.files.md in Chrome browser

Click “Install files.md” on the right side of the ad­dress bar:

Open a lo­cal folder to per­sist changes

Occasionally hit force-re­fresh (Cmd+Shift+R) to get new up­dates.

How to sync

Dump your thoughts

You can use chat to quickly dump your thoughts.

Open the chat and send a mes­sage:

Choose where to save (can do later):

With this flow you can quickly save notes, tasks, jour­nal records and check­lists.

Save things in the chat­bot

Open the chat, write some­thing and press Enter:

That’s it.

Telegram Bot

Other mes­sen­gers will fol­low.

How to think deeply

Connect ideas. Let them com­pound. Think through.

I used app.files.md to grow my knowl­edge about brain and soft­ware de­vel­op­ment

I added new notes to ei­ther brain or dev fold­ers. One idea per note

I made con­nec­tions be­tween the rel­e­vant notes in the web app (type [)

Everything is con­nected, just as in our brain

I spent time trav­el­ling through the notes and think­ing it through

At one point, some brain and dev notes ap­peared very re­lated

This con­nec­tion be­tween two dif­fer­ent do­mains pro­duced an in­sight

I wrote an ar­ti­cle based on that in­sight: Cognitive Load in Software Development

All this ac­tiv­ity helped me to:

Think deeply (which is very im­por­tant in the AI-age)

Think sys­tem­at­i­cally and see the big­ger pic­ture

Write in­sight­ful texts

To achieve all that, you’ll have to use your brain, not ad­vanced tem­plates or AI work­flows.

Start with no struc­ture at all, 0 fold­ers

One idea per note

Every note should be un­der­stood with­out con­text

Apply new knowl­edge im­me­di­ately, don’t save it for fu­ture self

Link re­lated notes

Revisit your notes and think through

My friends and I have been us­ing this sim­ple setup for five years, and it works well.

Second Brain?

I’ll quote I Deleted My Second Brain:

Obsidian is a bril­liant piece of soft­ware. I love it, dearly. But like any­thing, with­out re­straint, it can also be a trap. Markdown files in nested fold­ers. Plugins that track your pro­duc­tiv­ity. Graph views that sug­gest om­ni­science. There’s an il­lu­sion of mas­tery in watch­ing your notes web into con­stel­la­tions. But con­stel­la­tions are pro­jec­tions. They tell sto­ries. They do not guar­an­tee un­der­stand­ing. When I first started us­ing PKM tools, I be­lieved I was solv­ing a prob­lem of for­get­ting. Later, I be­lieved I was solv­ing a prob­lem of in­te­gra­tion. Eventually, I re­al­ized I had cre­ated a new prob­lem: de­fer­ral. The more my sys­tem grew, the more I de­ferred the work of thought to some fu­ture self who would sort, tag, dis­till, and ex­tract the gold. That self never ar­rived.

Obsidian is a bril­liant piece of soft­ware. I love it, dearly. But like any­thing, with­out re­straint, it can also be a trap. Markdown files in nested fold­ers. Plugins that track your pro­duc­tiv­ity. Graph views that sug­gest om­ni­science. There’s an il­lu­sion of mas­tery in watch­ing your notes web into con­stel­la­tions. But con­stel­la­tions are pro­jec­tions. They tell sto­ries. They do not guar­an­tee un­der­stand­ing.

When I first started us­ing PKM tools, I be­lieved I was solv­ing a prob­lem of for­get­ting. Later, I be­lieved I was solv­ing a prob­lem of in­te­gra­tion.

Eventually, I re­al­ized I had cre­ated a new prob­lem: de­fer­ral. The more my sys­tem grew, the more I de­ferred the work of thought to some fu­ture self who would sort, tag, dis­till, and ex­tract the gold.

That self never ar­rived.

The Second Brain is thrilling. Advanced guru tem­plates, plu­g­ins and AI work­flows… One wants to scrape the wis­dom of the whole in­ter­net. There’s some beauty in this neat sys­tem. Every new note brings dopamine. Second Brain gets bet­ter and bet­ter.

However, the first brain never ac­tu­ally gets smarter. And that’s an is­sue - in the AI age, your first brain is as valu­able as ever.

Use your brain to think through the notes. The tool is not im­por­tant, your think­ing is.

Before adding a new note, try to an­swer these ques­tions:

How this new knowl­edge can sharpen my judg­ment or ex­pand my tax­on­omy?

How can I see the world dif­fer­ently, given this new knowl­edge?

Notes can pre­vent ex­pe­ri­ence

Reading and tak­ing notes can eas­ily fool us into be­liev­ing that we un­der­stand a text

We think we un­der­stand, but in re­al­ity we just know

At some point our “knowing” is so good, that we start feel­ing that we ac­tu­ally do it (or at least tried)

The worst thing is that we don’t let new ex­pe­ri­ences emerge be­cause we al­ready have knowl­edge. It’s a knowl­edge bar­rier. Life gives us op­por­tu­ni­ties to live through new ex­pe­ri­ences, but we refuse, be­cause “we al­ready know”.

Self-help through read­ing and tak­ing notes? 🧘‍

Harm caused at the emo­tional level must be healed at the emo­tional level.

Not through in­tel­lec­tual work and tak­ing notes. Reading with­out ac­tion is en­ter­tain­ment. A form of pro­cras­ti­na­tion. No amount of self-help books can heal emo­tional wounds. What can help is psy­chother­apy, re­script­ing and chair work. Meditation. Healing hap­pens by feel­ing.

When to take notes

If your goal is to:

Develop a deeper, more struc­tured un­der­stand­ing of some­thing

Do re­search

Write an ar­ti­cle or a book

Then tak­ing notes is per­fectly fine.

Files struc­ture

You don’t have to think about the struc­ture, it is pre­de­fined. Although, you’re free to use what­ever struc­ture you want.

Chat: Chat.md

Notes: brain/​Note.md, <category>/*.md

Checklists: Read.md, Watch.md, Shop.md, MyChecklist_.md

Journal: jour­nal/​2024.08 August.md

Tasks: Later.md

Habits: habits/​Ate con­sciously.md, habits/*.​md

Images: me­dia/* (png, jpg, webp, gif)

Archive: archive/*.​md

What if you could strap a full desk­top GPU to your MacBook Air? Turns out, you can.

Just a quick FTC re­quired note: When you buy through my links, I may earn a com­mis­sion.

Never tell me the odds

As much as I hate to ad­mit it, step one in most of my pro­jects now is to ask AI about it. Maybe it’ll tell me some­thing I don’t know.

Fortunately, bor­der­line-im­prac­ti­cal is kind of my thing.

What’s a Thunderbolt eGPU?

Ok, so the plan is to plug a big PC gam­ing GPU, an NVIDIA RTX 5090, into my M4 MacBook Air. To do that, we plug it into a Thunderbolt dock which adapts PCIe to Thunderbolt, and we plug that into a USB-C port.

Thunderbolt tun­nels PCIe over a USB-C ca­ble, so from the com­put­er’s per­spec­tive a Thunderbolt de­vice re­ally is a PCIe de­vice, not a USB one. You get 4 PCIe lanes at up to 40Gbps on Thunderbolt 4, with a small per­for­mance penalty for the tun­nel­ing. USB4 in­cludes the same PCIe tun­nel­ing as an op­tional fea­ture, so some non-Thun­der­bolt USB4 ports can do this too. You can use this to plug a GPU into a lap­top with a com­pat­i­ble port.

Thunderbolt from the lap­top plugs into the GPU dock. The GPU plugs into the mon­i­tor via DisplayPort. Shortly af­ter this was taken, I broke this dock.

From the com­put­er’s per­spec­tive, the de­vice looks more or less like a slightly slower PCIe de­vice, so you can usu­ally use the same dri­vers you’d nor­mally use for those de­vices. eG­PUs work pretty much out of the box on Linux and Windows. It’s even pos­si­ble to use one on a Raspberry Pi (albeit with Oculink, not Thunderbolt).

The first hur­dle is that ma­cOS does not ship with dri­vers for NVIDIA or AMD GPUs on Apple Silicon.

What about tiny­grad?

tiny­grad re­cently re­leased their own ma­cOS eGPU dri­vers. It’s a whole new AI stack with its own open source dri­ver pipeline for NVIDIA and AMD hard­ware.

Sadly, if your main ob­jec­tive is to run AI in­fer­ence or play games, tiny­grad prob­a­bly is­n’t the so­lu­tion you’re look­ing for. This video by YouTuber Alex Ziskind shows that us­ing an eGPU via tiny­grad for in­fer­ence is about 10 times slower than run­ning na­tive Metal in­fer­ence di­rectly on an M4 Pro with­out an eGPU. You can only use the tiny­grad eGPU dri­ver with the tiny­grad stack, not for any­thing else. It also has very lim­ited sup­port for dif­fer­ent AI mod­els.

Getting NVIDIA PTX code run­ning on the GPU is one thing. Writing a full gen­eral-pur­pose dis­play dri­ver that works with ar­bi­trary soft­ware is a sig­nif­i­cantly harder prob­lem. So for now, what can you ac­tu­ally do with an eGPU and a Mac?

The ex­ist­ing Linux dri­ver

Linux can run on Apple Silicon Macs now. Regrettably, at this time, the Linux ker­nel does not sup­port Thunderbolt on Apple Silicon (only in­ter­nal de­vices and USB3). But…

You can run Linux in a 64-bit ARM VM on a ma­cOS host. ma­cOS sup­ports Thunderbolt de­vices. Linux sup­ports NVIDIA GPUs. Let’s put the pieces to­gether and pass through the GPU into the Linux VM.

At a high level, we’re just go­ing to put the GPU in the Linux VM. The VM is the same ar­chi­tec­ture as the Mac host (arm64), so per­for­mance should be com­pa­ra­ble. Of course, the devil is in the de­tails.

There is no dri­ver for NVIDIA cards on ARM64 Windows. That’s why we use Linux.

There is no dri­ver for NVIDIA cards on ARM64 Windows. That’s why we use Linux.

For a quick video demo of the re­sult, take a look:

In the rest of the post, I’ll go through the long and wind­ing road of get­ting this to ac­tu­ally work. If you just want to see screen­shots and bench­marks, you can prob­a­bly skip to the bench­mark sec­tion.

Engineering PCI Passthrough on ma­cOS

PCI de­vice ba­sics

Let’s look at two things we need work­ing for the VM to talk to the PCI de­vice:

PCI BAR (Base Address Registers) - Each PCI de­vice com­mu­ni­cates through chunks of mem­ory that the com­puter can read and write to. There’s ba­si­cally a re­served re­gion of mem­ory on your com­puter for each de­vice. Those mem­ory re­gions have to be mir­rored into the VM for PCI passthrough to work.

PCI BAR (Base Address Registers) - Each PCI de­vice com­mu­ni­cates through chunks of mem­ory that the com­puter can read and write to. There’s ba­si­cally a re­served re­gion of mem­ory on your com­puter for each de­vice. Those mem­ory re­gions have to be mir­rored into the VM for PCI passthrough to work.

DMA (Direct Memory Access) - This is how the de­vice can read and write in­for­ma­tion di­rectly in/​out of your com­put­er’s mem­ory. Instead of hav­ing the CPU burn cy­cles copy­ing data from the de­vice, the de­vice can copy the mem­ory au­to­mat­i­cally. For a GPU, it might be used to copy tex­tures di­rectly from the com­put­er’s mem­ory into its own video mem­ory.

DMA (Direct Memory Access) - This is how the de­vice can read and write in­for­ma­tion di­rectly in/​out of your com­put­er’s mem­ory. Instead of hav­ing the CPU burn cy­cles copy­ing data from the de­vice, the de­vice can copy the mem­ory au­to­mat­i­cally. For a GPU, it might be used to copy tex­tures di­rectly from the com­put­er’s mem­ory into its own video mem­ory.

Mapping PCI BARs

When QEMU starts a VM, it sets up the guest’s mem­ory lay­out. For nor­mal RAM, this boils down to a call to hvf_set_­phys_mem() in QEMU, which uses the Hypervisor.framework method:

hv_vm_map(mem, guest_­phys­i­cal_ad­dress, size, HV_MEMORY_READ | HV_MEMORY_WRITE | HV_MEMORY_EXEC);

Next, we con­nect to the host PCIDriverKit dri­ver and ask to map the mem­ory from the PCI de­vice into our process. (I’m leav­ing the dri­ver-side code out for now, but it’s very sim­i­lar boil­er­plate.)

// map BAR0 into the cur­rent process and set `addr` to the lo­ca­tion // where it was mapped mach_vm_ad­dress_t addr = 0; mach_vm_­size_t size = 0; IOConnectMapMemory64(driverConnection, 0, mach_­task_­self(), &addr, &size, kIOMa­pA­ny­where);

Ok, so then we have addr, which now points to the BAR0 mem­ory that we can ac­cess di­rectly in our process. At this point you can just read and write stuff to it, like any other piece of mem­ory.

volatile uin­t32_t *bar0 = (volatile uin­t32_t *)addr; printf(“BAR0[0] = %x\n”, bar0[0]); // this would out­put: BAR0[0] = 0x1b2000a1 // which is a de­vice-spe­cific con­stant that de­scribes my RTX 5090 // // BAR0[0] is the BOOT_0 reg­is­ter. The fields break down as: // arch = 0x1b → GB200 GPU fam­ily // impl = 0x2 → GB202 die (RTX 5090) // ma­jor_rev = 0xa → step­ping A // mi­nor_rev = 0x1 → re­vi­sion 1 (together: step­ping A1)

Now we just make sure QEMU calls hvf_set_­phys_mem() for our de­vice mem­ory, and we can map that into the guest. When guest code touches that map­ping, it talks di­rectly to the GPU with min­i­mal host over­head. This is the best case for per­for­mance. At least, in the­ory.

In prac­tice, as soon as the VM touched the PCI BAR mem­ory, the host ker­nel crashed.

If you’ve never ex­pe­ri­enced this be­fore, it’s dis­ori­ent­ing. Your en­tire com­puter will hang, and be­cause the track­pad feed­back is con­trolled by soft­ware, sud­denly the track­pad will no longer click. The dogs and cats in your neigh­bor­hood start howl­ing. Pictures fall off the walls of your house. Eventually your com­puter will re­boot, and you will be pre­sented with this di­a­log.

Ok, so we can’t map de­vice mem­ory di­rectly, but we have other tricks up our sleeve. We can trap every ac­cess to the mem­ory, exit the guest back into QEMU, and have QEMU for­ward each read or write to the de­vice. That keeps be­hav­ior cor­rect, but it’s bru­tally slow. In many work­loads the pain is else­where. Most of the per­for­mance-sen­si­tive work is DMA, but some paths still care how fast you can push com­mands through the BAR.

I started prepar­ing a bug re­port for Apple and wrote a small re­pro­duc­tion (well, AI-assisted) to demon­strate the is­sue:

In ~100 lines of C, you can spin up a VM, map the de­vice BAR into the guest, and run code that touches it. I’m still not sure whether that was more frus­trat­ing or en­cour­ag­ing, but that ver­sion ran with­out crash­ing, while QEMU was still pan­ick­ing the host. I was stumped for a while. Was it the guest page ta­bles? Was the BAR col­lid­ing with guest RAM in some sub­tle way? Why were the dogs and cats still howl­ing?

Eventually, in my des­per­a­tion, I asked an AI cod­ing as­sis­tant to com­pare my sam­ple and QEMU. It im­me­di­ately flagged that my map­ping used HV_MEMORY_READ | HV_MEMORY_WRITE while QEMU used HV_MEMORY_READ | HV_MEMORY_WRITE | HV_MEMORY_EXEC. Alas, bested again by AI. Not even silly blog pro­jects are safe any­more (mostly kid­ding).

The workaround in QEMU was a small change:

It works, but it’s not per­fect. ARM has sev­eral fla­vors of de­vice mem­ory (the Device-nGnRnE/nGnRE/nGRE/GRE fam­ily), with dif­fer­ent rules for whether writes can be gath­ered, re­ordered, or ac­knowl­edged early. It’s roughly anal­o­gous to x86 write-com­bin­ing on the most per­mis­sive end.

On real hard­ware, the prefetch­able BARs on my GPU are sup­posed to al­low gath­er­ing, which makes them sev­eral times faster for bulk writes than BAR0. But hv_vm_map() has no flags to con­fig­ure this, so every de­vice map­ping ends up as the strictest nGn­RnE. There’s noth­ing we can do about it, and it’s still ~30x faster than trap­ping every ac­cess, but it makes writ­ing the BAR ~10x slower than it would be nor­mally.

DMA

This was by far the sketchi­est part of the pro­ject. To start, let’s go over how this works on a PC run­ning Linux with VM PCI-passthrough, and then we’ll com­pare to our chal­lenge on ma­cOS.

When there’s just a com­puter talk­ing to a de­vice (no VM in­volved), they can talk to­gether di­rectly. The PC will tell the de­vice “hey I got that DMA buffer ready at this mem­ory ad­dress” and the de­vice can ac­cess that mem­ory di­rectly (AKA DMA). Easy.

When a VM is in­volved, it’s more com­pli­cated. Guest phys­i­cal ad­dresses don’t cor­re­spond to host phys­i­cal ad­dresses. The VM’s RAM is just some chunk of host mem­ory al­lo­cated wher­ever it was avail­able. So if the guest tells the de­vice “DMA into 0x00000000,” the de­vice will hap­pily scrib­ble over what­ever ac­tu­ally lives there on the host. The sim­plest fix is two things:

Pin all guest mem­ory so it can’t be paged out while the de­vice might touch it.

Put a hard­ware unit called the IOMMU be­tween the de­vice and host mem­ory. The hy­per­vi­sor pro­grams it with the guest → host trans­la­tions, and every DMA re­quest from the de­vice gets remapped on the fly.

DMA Request:Read/Write0x00000000

IOMMU

Translation Table

0x00000000 – 0x80000000

0x20000000 – 0xA0000000

Translated to:0x20000000

Host Physical Memory

0x20000000 – 0xA0000000

This is a blunt so­lu­tion. The guest does­n’t have to do any­thing spe­cial, but the host has to keep all guest RAM pinned. There are more ad­vanced ap­proaches (like a vir­tual IOMMU), but they’re out­side the scope of this post.

DMA on Apple Silicon

On Apple Silicon, there’s a hard­ware unit called DART that’s more or less equiv­a­lent to an IOMMU. It’s not spe­cific to VMs; it also acts as a se­cu­rity bound­ary, pre­vent­ing de­vices from ac­cess­ing ar­bi­trary host mem­ory. Ideally we’d just use DART the same way Linux uses the IOMMU in the sim­ple case above.

Unfortunately, DART (at least via PCIDriverKit for Thunderbolt de­vices) has some hard con­straints:

~1.5GB map­ping limit. A VM with 1.5GB of RAM can tech­ni­cally boot, but CUDA runs out of mem­ory and any mod­ern game needs 8 – 16GB.

~64k map­ping cap. With many small DMA buffers the map­ping table fills up.

No ad­dress or align­ment con­trol. PCIDriverKit as­signs mapped ad­dresses for you. You can’t pick them, or spec­ify align­ment con­straints. This rules out a vir­tual IOMMU, which re­quires the guest to choose its own DMA ad­dresses.

The 1.5GB ceil­ing was the biggest ini­tial blocker. I tried a few workarounds: pre-map­ping ranges where I guessed DMAs might land (obviously did­n’t work), and us­ing a re­stricted-dma-pool de­vice tree at­tribute to force all DMA through a pre-al­lo­cated re­gion. The re­stricted pool ap­proach ac­tu­ally works for sim­pler de­vices, but GPU dri­vers are too weird to fit into that model. (If you’re cu­ri­ous about the specifics, there’s a qemu-de­vel thread where I dis­cuss it.)

ap­ple-dma-pci

I ended up de­sign­ing a new vir­tual PCI de­vice in QEMU called ap­ple-dma-pci. It gets in­serted into the VM along­side the passed-through GPU, and a com­pan­ion ker­nel dri­ver in the guest in­ter­cepts the NVIDIA dri­ver’s DMA map­ping calls. The so­lu­tion is, frankly, a very up­set­ting hack, but it works.

Because map­pings are cre­ated on de­mand per DMA re­quest and torn down when the buffer is freed, we re­duce the amount of mapped mem­ory we need at any given time. Only the work­ing set of live DMA buffers at any given mo­ment has to fit in our 1.5GB limit, as op­posed to the en­tirety of guest mem­ory.

The guest dri­ver is loaded early (via an /etc/modules-load.d/ con­fig), so it can find the GPU at probe time and swap in cus­tom DMA ops be­fore the NVIDIA dri­ver touches it:

sta­tic struct dma_map_ops ap­ple_d­ma_ops = { .map_page = ap­ple_d­ma_map_­page, .unmap_page = ap­ple_d­ma_un­map_­page, .map_sg = ap­ple_d­ma_map_sg, .unmap_sg = ap­ple_d­ma_un­map_sg, .alloc = ap­ple_d­ma_al­loc, .free = ap­ple_d­ma_free, };

sta­tic int ap­ple_d­ma_p­ci_probe(struct pci_dev *pdev, const struct pci_de­vice_id *id) { struct pci_dev *gpu = pci_get_de­vice(PCI_VEN­DOR_N­VIDIA, PCI_ANY_ID, NULL); if (!gpu) re­turn -ENODEV;

set_d­ma_ops(&gpu->dev, &apple_dma_ops); pci_de­v_put(gpu); re­turn 0; }

Each of the cus­tom ops is a thin wrap­per. It mar­shals its ar­gu­ments into a small re­quest, writes it into mem­ory for the ap­ple-dma-pci vir­tual BAR, kicks a door­bell reg­is­ter, and waits for a re­ply. On the host side, QEMU picks up the re­quest, hands it off to the PCIDriverKit dri­ver, which per­forms the ac­tual DART map­ping, and the re­sult­ing DMA ad­dress gets writ­ten back to guest mem­ory. The NVIDIA dri­ver should­n’t know the dif­fer­ence.

Linux VM (Guest)

NVIDIA Driver

dma_map_­page()

ap­ple_d­ma_ops han­dler

vir­tual PCI BAR write

ap­ple-dma-pci vir­tual de­vice

VM exit

ma­cOS Host

QEMU

IOConnectCallMethod()

PCIDriverKit dri­ver

IODMACommand

DART hard­ware

mapped ad­dress re­turned back up the stack

NVIDIA align­ment quirk

It did­n’t im­me­di­ately work well, though. While the dri­ver ini­tially loaded and ini­tial­ized the card, I was greeted with this fun ker­nel log mes­sage as soon as I at­tempted to run a CUDA work­load:

[ 456.194883] NVRM: nvAsser­tOk­Failed­NoLog: Assertion failed: The off­set passed is not valid [NV_ERR_INVALID_OFFSET] (0x00000037) re­turned from pRmApi->Al­loc(pRmApi, de­vice->ses­sion->han­dle, is­Sys­tem­Mem­ory ? de­vice->han­dle : de­vice->sub­han­dle, &physHandle, is­Sys­tem­Mem­ory ? NV01_MEMORY_SYSTEM : NV01_MEMORY_LOCAL_USER, &memAllocParams, sizeof(memAl­loc­Params)) @ nv_g­pu_ops.c:4972 [ 456.371282] NVRM: GPU0 nvAssert­Failed­NoLog: Assertion failed: 0 == (physAddr & (RM_PAGE_SIZE_HUGE - 1)) @ mem_m­gr_g­m107.c:1312 [ 456.372020] NVRM: nvAsser­tOk­Failed­NoLog: Assertion failed: The off­set passed is not valid [NV_ERR_INVALID_OFFSET] (0x00000037) re­turned from pRmApi->Al­loc(pRmApi, de­vice->ses­sion->han­dle, is­Sys­tem­Mem­ory ? de­vice->han­dle : de­vice->sub­han­dle, &physHandle, is­Sys­tem­Mem­ory ? NV01_MEMORY_SYSTEM : NV01_MEMORY_LOCAL_USER, &memAllocParams, sizeof(memAl­loc­Params)) @ nv_g­pu_ops.c:4972

If you re­call the ear­lier DMA sec­tion, we noted that we can’t con­trol the align­ment of DMA-mapped buffers. Bummer. At this point, I dug into the dri­ver to try to see if there was some­thing sim­ple we could patch.

Here’s the rel­e­vant seg­ment:

if (type == UVM_RM_MEM_TYPE_SYS) { if (size >= UVM_PAGE_SIZE_2M) al­loc_info.pa­ge­Size = UVM_PAGE_SIZE_2M; else if (size >= UVM_PAGE_SIZE_64K) al­loc_info.pa­ge­Size = UVM_PAGE_SIZE_64K;

sta­tus = uvm_r­m_locked_­call(nvU­vmInter­face­Mem­o­ryAl­loc­Sys(gpu->rm_ad­dress_­space, size, &gpu_va, &alloc_info));

// TODO: Bug 5042223 if (status == NV_ERR_NO_MEMORY && size >= UVM_PAGE_SIZE_64K) { UVM_ERR_PRINT(“nvUvmInterfaceMemoryAllocSys al­loc failed with big page size, retry with de­fault page size\n”); al­loc_info.pa­ge­Size = UVM_PAGE_SIZE_DEFAULT; sta­tus = uvm_r­m_locked_­call(nvU­vmInter­face­Mem­o­ryAl­loc­Sys(gpu->rm_ad­dress_­space, size, &gpu_va, &alloc_info)); } }

By adding more de­bug log­ging in the mod­ule, I could see it was a 16MB al­lo­ca­tion of type UVM_RM_MEM_TYPE_SYS. So, it uses the largest (2MB) page size. Ironically, there is al­ready a workaround here when the al­lo­ca­tion fails. It’ll just try again with a smaller page size. It just does­n’t take into ac­count the dif­fer­ent er­ror code for align­ment (NV_ERR_INVALID_OFFSET).

Blog post with de­tails com­ing soon.

It passes Bun’s pre-ex­ist­ing test suite on all plat­forms (and fixes sev­eral mem­ory leaks and flaky tests), the bi­nary size shrinks by 3 MB - 8 MB, the bench­marks are be­tween neu­tral and faster - and most im­por­tantly, we now have com­piler-as­sisted tools for catch­ing & pre­vent­ing mem­ory bugs, which have costed the team an enor­mous amount of de­vel­op­ment & de­bug­ging time over the years.

The code­base is oth­er­wise largely the same. The same ar­chi­tec­ture, the same data struc­tures. Bun still uses few 3rd party li­braries. No async rust.

To try this, run:

bun up­grade –canary

Please do file is­sues if you run into any. If this thread gets crazy I will lock it.

Note:

Still some op­ti­miza­tion work to do be­fore this lands in non-ca­nary ver­sion.

Still some cleanup work to do (which will come in a se­ries of fol­low-up PRs)