Since its launch in 2007, the Wii has seen sev­eral op­er­at­ing sys­tems ported to it: Linux, NetBSD, and most-re­cently, Windows NT. Today, Mac OS X joins that list.

In this post, I’ll share how I ported the first ver­sion of Mac OS X, 10.0 Cheetah, to the Nintendo Wii. If you’re not an op­er­at­ing sys­tems ex­pert or low-level en­gi­neer, you’re in good com­pany; this pro­ject was all about learn­ing and nav­i­gat­ing count­less “unknown un­knowns”. Join me as we ex­plore the Wii’s hard­ware, boot­loader de­vel­op­ment, ker­nel patch­ing, and writ­ing dri­vers - and give the PowerPC ver­sions of Mac OS X a new life on the Nintendo Wii.

Visit the wi­iMac boot­loader repos­i­tory for in­struc­tions on how to try this pro­ject your­self.

Before fig­ur­ing out how to tackle this pro­ject, I needed to know whether it would even be pos­si­ble. According to a 2021 Reddit com­ment:

There is a zero per­cent chance of this ever hap­pen­ing.

Feeling en­cour­aged, I started with the ba­sics: what hard­ware is in the Wii, and how does it com­pare to the hard­ware used in real Macs from the era.

The Wii uses a PowerPC 750CL proces­sor - an evo­lu­tion of the PowerPC 750CXe that was used in G3 iBooks and some G3 iMacs. Given this close lin­eage, I felt con­fi­dent that the CPU would­n’t be a blocker.

As for RAM, the Wii has a unique con­fig­u­ra­tion: 88 MB to­tal, split across 24 MB of 1T-SRAM (MEM1) and 64 MB of slower GDDR3 SDRAM (MEM2); un­con­ven­tional, but tech­ni­cally enough for Mac OS X Cheetah, which of­fi­cially calls for 128 MB of RAM but will un­of­fi­cially boot with less. To be safe, I used QEMU to boot Cheetah with 64 MB of RAM and ver­i­fied that there were no is­sues.

Other hard­ware I’d even­tu­ally need to sup­port in­cluded:

* The SD card for boot­ing the rest of the sys­tem once the ker­nel was run­ning

* Video out­put via a frame­buffer that lives in RAM

* The Wii’s USB ports for us­ing a mouse and key­board

Convinced that the Wii’s hard­ware was­n’t fun­da­men­tally in­com­pat­i­ble with Mac OS X, I moved my at­ten­tion to in­ves­ti­gat­ing the soft­ware stack I’d be port­ing.

Mac OS X has an open source core (Darwin, with XNU as the ker­nel and IOKit as the dri­ver model), with closed-source com­po­nents lay­ered on top (Quartz, Dock, Finder, sys­tem apps and frame­works). In the­ory, if I could mod­ify the open-source parts enough to get Darwin run­ning, the closed-source parts would run with­out ad­di­tional patches.

Porting Mac OS X would also re­quire un­der­stand­ing how a real Mac boots. PowerPC Macs from the early 2000s use Open Firmware as their low­est-level soft­ware en­vi­ron­ment; for sim­plic­ity, it can be thought of as the first code that runs when a Mac is pow­ered on. Open Firmware has sev­eral re­spon­si­bil­i­ties, in­clud­ing:

* Providing use­ful func­tions for I/O, draw­ing, and hard­ware com­mu­ni­ca­tion

* Loading and ex­e­cut­ing an op­er­at­ing sys­tem boot­loader from the filesys­tem

Open Firmware even­tu­ally hands off con­trol to BootX, the boot­loader for Mac OS X. BootX pre­pares the sys­tem so that it can even­tu­ally pass con­trol to the ker­nel. The re­spon­si­bil­i­ties of BootX in­clude:

* Loading and de­cod­ing the XNU ker­nel, a Mach-O ex­e­cutable, from the root filesys­tem

Once XNU is run­ning, there are no de­pen­den­cies on BootX or Open Firmware. XNU con­tin­ues on to ini­tial­ize proces­sors, vir­tual mem­ory, IOKit, BSD, and even­tu­ally con­tinue boot­ing by load­ing and run­ning other ex­e­cuta­bles from the root filesys­tem.

The last piece of the puz­zle was how to run my own cus­tom code on the Wii - a triv­ial task thanks to the Wii be­ing “jailbroken”, al­low­ing any­one to run home­brew with full ac­cess to the hard­ware via the Homebrew Channel and BootMii.

Armed with knowl­edge of how the boot process works on a real Mac, along with how to run low-level code on the Wii, I needed to se­lect an ap­proach for boot­ing Mac OS X on the Wii. I eval­u­ated three op­tions:

Port Open Firmware, use that to run un­mod­i­fied BootX to boot Mac OS X

Port BootX and mod­ify it to not rely on Open Firmware, use that to boot Mac OS X

Write a cus­tom boot­loader that per­forms the bare-min­i­mum setup to boot Mac OS X

Since Mac OS X does­n’t de­pend on Open Firmware or BootX once run­ning, spend­ing time port­ing ei­ther of those seemed like an un­nec­es­sary dis­trac­tion. Additionally, both Open Firmware and BootX con­tain added com­plex­ity for sup­port­ing many dif­fer­ent hard­ware con­fig­u­ra­tions - com­plex­ity that I would­n’t need since this only needs to run on the Wii. Following in the foot­steps of the Wii Linux pro­ject, I de­cided to write my own boot­loader from scratch. The boot­loader would need to, at a min­i­mum:

* Load the ker­nel from the SD card

Once the ker­nel was run­ning, none of the boot­loader code would mat­ter. At that point, my fo­cus would shift to patch­ing the ker­nel and writ­ing dri­vers.

I de­cided to base my boot­loader on some low-level ex­am­ple code for the Wii called ppcskel. ppcskel puts the sys­tem into a sane ini­tial state, and pro­vides use­ful func­tions for com­mon things like read­ing files from the SD card, draw­ing text to the frame­buffer, and log­ging de­bug mes­sages to a USB Gecko.

Next, I had to fig­ure out how to load the XNU ker­nel into mem­ory so that I could pass con­trol to it. The ker­nel is stored in a spe­cial bi­nary for­mat called Mach-O, and needs to be prop­erly de­coded be­fore be­ing used.

The Mach-O ex­e­cutable for­mat is well-doc­u­mented, and can be thought of as a list of load com­mands that tell the loader where to place dif­fer­ent sec­tions of the bi­nary file in mem­ory. For ex­am­ple, a load com­mand might in­struct the loader to read the data from file off­set 0x2cf000 and store it at the mem­ory ad­dress 0x2e0000. After pro­cess­ing all of the ker­nel’s load com­mands, we end up with this mem­ory lay­out:

The ker­nel file also spec­i­fies the mem­ory ad­dress where ex­e­cu­tion should be­gin. Once the boot­loader jumps to this ad­dress, the ker­nel is in full con­trol and the boot­loader is no longer run­ning.

To jump to the ker­nel-en­try-point’s mem­ory ad­dress, I needed to cast the ad­dress to a func­tion and call it:

After this code ran, the screen went black and my de­bug logs stopped ar­riv­ing via the se­r­ial de­bug con­nec­tion - while an­ti­cli­mac­tic, this was an in­di­ca­tor that the ker­nel was run­ning.

The ques­tion then be­came: how far was I mak­ing it into the boot process? To an­swer this, I had to start look­ing at XNU source code. The first code that runs is a PowerPC as­sem­bly _start rou­tine. This code re­con­fig­ures the hard­ware, over­rid­ing all of the Wii-specific setup that the boot­loader per­formed and, in the process, dis­ables boot­loader func­tion­al­ity for se­r­ial de­bug­ging and video out­put. Without nor­mal de­bug-out­put fa­cil­i­ties, I’d need to track progress a dif­fer­ent way.

The ap­proach that I came up with was a bit of a hack: bi­nary-patch the ker­nel, re­plac­ing in­struc­tions with ones that il­lu­mi­nate one of the front-panel LEDs on the Wii. If the LED il­lu­mi­nated af­ter jump­ing to the ker­nel, then I’d know that the ker­nel was mak­ing it at least that far. Turning on one of these LEDs is as sim­ple as writ­ing a value to a spe­cific mem­ory ad­dress. In PowerPC as­sem­bly, those in­struc­tions are:

To know which parts of the ker­nel to patch, I cross-ref­er­enced func­tion names in XNU source code with func­tion off­sets in the com­piled ker­nel bi­nary, us­ing Hopper Disassembler to make the process eas­ier. Once I iden­ti­fied the cor­rect off­set in the bi­nary that cor­re­sponded to the code I wanted to patch, I just needed to re­place the ex­ist­ing in­struc­tions at that off­set with the ones to blink the LED.

To make this patch­ing process eas­ier, I added some code to the boot­loader to patch the ker­nel bi­nary on the fly, en­abling me to try dif­fer­ent off­sets with­out man­u­ally mod­i­fy­ing the ker­nel file on disk.

After trac­ing through many ker­nel startup rou­tines, I even­tu­ally mapped out this path of ex­e­cu­tion:

This was an ex­cit­ing mile­stone - the ker­nel was def­i­nitely run­ning, and I had even made it into some higher-level C code. To make it past the 300 ex­cep­tion crash, the boot­loader would need to pass a pointer to a valid de­vice tree.

The de­vice tree is a data struc­ture rep­re­sent­ing all of the hard­ware in the sys­tem that should be ex­posed to the op­er­at­ing sys­tem. As the name sug­gests, it’s a tree made up of nodes, each ca­pa­ble of hold­ing prop­er­ties and ref­er­ences to child nodes.

On real Mac com­put­ers, the boot­loader scans the hard­ware and con­structs a de­vice tree based on what it finds. Since the Wii’s hard­ware is al­ways the same, this scan­ning step can be skipped. I ended up hard-cod­ing the de­vice tree in the boot­loader, tak­ing in­spi­ra­tion from the de­vice tree that the Wii Linux pro­ject uses.

Since I was­n’t sure how much of the Wii’s hard­ware I’d need to sup­port in or­der to get the boot process fur­ther along, I started with a min­i­mal de­vice tree: a root node with chil­dren for the cpus and mem­ory:

My plan was to ex­pand the de­vice tree with more pieces of hard­ware as I got fur­ther along in the boot process - even­tu­ally con­struct­ing a com­plete rep­re­sen­ta­tion of all of the Wii’s hard­ware that I planned to sup­port in Mac OS X.

Once I had a de­vice tree cre­ated and stored in mem­ory, I needed to pass it to the ker­nel as part of boot_args:

With the de­vice tree in mem­ory, I had made it past the de­vice_tree.c crash. The boot­loader was per­form­ing the ba­sics well: load­ing the ker­nel, cre­at­ing boot ar­gu­ments and a de­vice tree, and ul­ti­mately, call­ing the ker­nel. To make ad­di­tional progress, I’d need to shift my at­ten­tion to­ward patch­ing the ker­nel source code to fix re­main­ing com­pat­i­bil­ity is­sues.

At this point, the ker­nel was get­ting stuck while run­ning some code to set up video and I/O mem­ory. XNU from this era makes as­sump­tions about where video and I/O mem­ory can be, and re­con­fig­ures Block Address Translations (BATs) in a way that does­n’t play nicely with the Wii’s mem­ory lay­out (MEM1 start­ing at 0x00000000, MEM2 start­ing at 0x10000000). To work around these lim­i­ta­tions, it was time to mod­ify the ker­nel’s source code and boot a mod­i­fied ker­nel bi­nary.

Figuring out a sane de­vel­op­ment en­vi­ron­ment to build an OS ker­nel from 25 years ago took some ef­fort. Here’s what I landed on:

* XNU source code lives on the host’s filesys­tem, and is ex­posed via an NFS server

* The guest ac­cesses the XNU source via an NFS mount

* The host uses SSH to con­trol the guest

* Edit XNU source on host, kick off a build via SSH on the guest, build ar­ti­facts end up on the filesys­tem ac­ces­si­ble by host and guest

To set up the de­pen­den­cies needed to build the Mac OS X Cheetah ker­nel on the Mac OS X Cheetah guest, I fol­lowed the in­struc­tions here. They mostly matched up with what I needed to do. Relevant sources are avail­able from Apple here.

After fix­ing the BAT setup and adding some small patches to reroute con­sole out­put to my USB Gecko, I now had video out­put and se­r­ial de­bug logs work­ing - mak­ing fu­ture de­vel­op­ment and de­bug­ging sig­nif­i­cantly eas­ier. Thanks to this new vis­i­bil­ity into what was go­ing on, I could see that the vir­tual mem­ory, IOKit, and BSD sub­sys­tems were all ini­tial­ized and run­ning - with­out crash­ing. This was a sig­nif­i­cant mile­stone, and gave me con­fi­dence that I was on the right path to get­ting a full sys­tem work­ing.

Readers who have at­tempted to run Mac OS X on a PC via “hackintoshing” may rec­og­nize the last line in the boot logs: the dreaded “Still wait­ing for root de­vice”. This oc­curs when the sys­tem can’t find a root filesys­tem from which to con­tinue boot­ing. In my case, this was ex­pected: the ker­nel had done all it could and was ready to load the rest of the Mac OS X sys­tem from the filesys­tem, but it did­n’t know where to lo­cate this filesys­tem. To make progress, I would need to tell the ker­nel how to read from the Wii’s SD card. To do this, I’d need to tackle the next phase of this pro­ject: writ­ing dri­vers.

Mac OS X dri­vers are built us­ing IOKit - a col­lec­tion of soft­ware com­po­nents that aim to make it easy to ex­tend the ker­nel to sup­port dif­fer­ent hard­ware de­vices. Drivers are writ­ten us­ing a sub­set of C++, and make ex­ten­sive use of ob­ject-ori­ented pro­gram­ming con­cepts like in­her­i­tance and com­po­si­tion. Many pieces of use­ful func­tion­al­ity are pro­vided, in­clud­ing:

* Base classes and “families” that im­ple­ment com­mon be­hav­ior for dif­fer­ent types of hard­ware

* Probing and match­ing dri­vers to hard­ware pre­sent in the de­vice tree

In IOKit, there are two kinds of dri­vers: a spe­cific de­vice dri­ver and a nub. A spe­cific de­vice dri­ver is an ob­ject that man­ages a spe­cific piece of hard­ware. A nub is an ob­ject that serves as an at­tach-point for a spe­cific de­vice dri­ver, and also pro­vides the abil­ity for that at­tached dri­ver to com­mu­ni­cate with the dri­ver that cre­ated the nub. It’s this chain of dri­ver-to-nub-to-dri­ver that cre­ates the afore­men­tioned provider-client re­la­tion­ships. I strug­gled for a while to grasp this con­cept, and found a con­crete ex­am­ple use­ful.

Real Macs can have a PCI bus with sev­eral PCI ports. In this ex­am­ple, con­sider an eth­er­net card be­ing plugged into one of the PCI ports. A dri­ver, IOPCIBridge, han­dles com­mu­ni­cat­ing with the PCI bus hard­ware on the moth­er­board. This dri­ver scans the bus, cre­at­ing IOPCIDevice nubs (attach-points) for each plugged-in de­vice that it finds. A hy­po­thet­i­cal dri­ver for the plugged-in eth­er­net card (let’s call it SomeEthernetCard) can at­tach to the nub, us­ing it as its proxy to call into PCI func­tion­al­ity pro­vided by the IOPCIBridge dri­ver on the other side. The SomeEthernetCard dri­ver can also cre­ate its own IOEthernetInterface nubs so that higher-level parts of the IOKit net­work­ing stack can at­tach to it.

Someone de­vel­op­ing a PCI eth­er­net card dri­ver would only need to write SomeEthernetCard; the lower-level PCI bus com­mu­ni­ca­tion and the higher-level net­work­ing stack code is all pro­vided by ex­ist­ing IOKit dri­ver fam­i­lies. As long as SomeEthernetCard can at­tach to an IOPCIDevice nub and pub­lish its own IOEthernetInterface nubs, it can sand­wich it­self be­tween two ex­ist­ing fam­i­lies in the dri­ver stack, ben­e­fit­ing from all of the func­tion­al­ity pro­vided by IOPCIFamily while also sat­is­fy­ing the needs of IONetworkingFamily.

Unlike Macs from the same era, the Wii does­n’t use PCI to con­nect its var­i­ous pieces of hard­ware to its moth­er­board. Instead, it uses a cus­tom sys­tem-on-a-chip (SoC) called the Hollywood. Through the Hollywood, many pieces of hard­ware can be ac­cessed: the GPU, SD card, WiFi, Bluetooth, in­ter­rupt con­trollers, USB ports, and more. The Hollywood also con­tains an ARM co­proces­sor, nick­named the Starlet, that ex­poses hard­ware func­tion­al­ity to the main PowerPC proces­sor via in­ter-proces­sor-com­mu­ni­ca­tion (IPC).

This unique hard­ware lay­out and com­mu­ni­ca­tion pro­to­col meant that I could­n’t piggy-back off of an ex­ist­ing IOKit dri­ver fam­ily like IOPCIFamily. Instead, I would need to im­ple­ment an equiv­a­lent dri­ver for the Hollywood SoC, cre­at­ing nubs that rep­re­sent at­tach-points for all of the hard­ware it con­tains. I landed on this lay­out of dri­vers and nubs (note that this is only show­ing a sub­set of the dri­vers that had to be writ­ten):

Now that I had a bet­ter idea of how to rep­re­sent the Wii’s hard­ware in IOKit, I be­gan work on my Hollywood dri­ver.

I started by cre­at­ing a new C++ header and im­ple­men­ta­tion file for a NintendoWiiHollywood dri­ver. Its dri­ver “personality” en­abled it to be matched to a node in the de­vice tree with the name “hollywood”`. Once the dri­ver was matched and run­ning, it was time to pub­lish nubs for all of its child de­vices.

Once again lean­ing on the de­vice tree as the source of truth for what hard­ware lives un­der the Hollywood, I it­er­ated through all of the Hollywood node’s chil­dren, cre­at­ing and pub­lish­ing NintendoWiiHollywoodDevice nubs for each:

Once NintendoWiiHollywoodDevice nubs were cre­ated and pub­lished, the sys­tem would be able to have other de­vice dri­vers, like an SD card dri­ver, at­tach to them.

Next, I moved on to writ­ing a dri­ver to en­able the sys­tem to read and write from the Wii’s SD card. This dri­ver is what would en­able the sys­tem to con­tinue boot­ing, since it was cur­rently stuck look­ing for a root filesys­tem from which to load ad­di­tional startup files.

I be­gan by sub­class­ing IOBlockStorageDevice, which has many ab­stract meth­ods in­tended to be im­ple­mented by sub­classers:

For most of these meth­ods, I could im­ple­ment them with hard-coded val­ues that matched the Wii’s SD card hard­ware; ven­dor string, block size, max read and write trans­fer size, ejectabil­ity, and many oth­ers all re­turn con­stant val­ues, and were triv­ial to im­ple­ment.

The more in­ter­est­ing meth­ods to im­ple­ment were the ones that needed to ac­tu­ally com­mu­ni­cate with the cur­rently-in­serted SD card: get­ting the ca­pac­ity of the SD card, read­ing from the SD card, and writ­ing to the SD card:

To com­mu­ni­cate with the SD card, I uti­lized the IPC func­tion­al­ity pro­vided by MINI run­ning on the Starlet co-proces­sor. By writ­ing data to cer­tain re­served mem­ory ad­dresses, the SD card dri­ver was able to is­sue com­mands to MINI. MINI would then ex­e­cute those com­mands, com­mu­ni­cat­ing back any re­sult data by writ­ing to a dif­fer­ent re­served mem­ory ad­dress that the dri­ver could mon­i­tor.

MINI sup­ports many use­ful com­mand types. The ones used by the SD card dri­ver are:

* IPC_SDMMC_SIZE: Returns the num­ber of sec­tors on the cur­rently-in­serted SD card

With these three com­mand types, reads, writes, and ca­pac­ity-checks could all be im­ple­mented, en­abling me to sat­isfy the core re­quire­ments of the block stor­age de­vice sub­class.

Like with most pro­gram­ming en­de­vours, things rarely work on the first try. To in­ves­ti­gate is­sues, my pri­mary de­bug­ging tool was send­ing log mes­sages to the se­r­ial de­bug­ger via calls to IOLog. With this tech­nique, I was able to see which meth­ods were be­ing called on my dri­ver, what val­ues were be­ing passed in, and what val­ues my IPC im­ple­men­ta­tion was send­ing to and re­ceiv­ing from MINI - but I had no abil­ity to set break­points or an­a­lyze ex­e­cu­tion dy­nam­i­cally while the ker­nel was run­ning.

One of the trick­ier bugs that I en­coun­tered had to do with cached mem­ory. When the SD card dri­ver wants to read from the SD card, the com­mand it is­sues to MINI (running on the ARM CPU) in­cludes a mem­ory ad­dress at which to store any loaded data. After MINI fin­ishes writ­ing to mem­ory, the SD card dri­ver (running on the PowerPC CPU) might not be able to see the up­dated con­tents if that re­gion is mapped as cacheable. In that case, the PowerPC will read from its cache lines rather than RAM, re­turn­ing stale data in­stead of the newly loaded con­tents. To work around this, the SD card dri­ver must use un­cached mem­ory for its buffers.

After sev­eral days of bug-fix­ing, I reached a new mile­stone: IOBlockStorageDriver, which at­tached to my SD card dri­ver, had started pub­lish­ing IOMedia nubs rep­re­sent­ing the log­i­cal par­ti­tions pre­sent on the SD. Through these nubs, higher-level parts of the sys­tem were able to at­tach and be­gin us­ing the SD card. Importantly, the sys­tem was now able to find a root filesys­tem from which to con­tinue boot­ing, and I was no longer stuck at “Still wait­ing for root de­vice”:

My boot logs now looked like this:

After some more rounds of bug fixes (while on the go), I was able to boot past sin­gle-user mode:

And even­tu­ally, make it through the en­tire ver­bose-mode startup se­quence, which ends with the mes­sage: “Startup com­plete”:

At this point, the sys­tem was try­ing to find a frame­buffer dri­ver so that the Mac OS X GUI could be shown. As in­di­cated in the logs, WindowServer was not happy - to fix this, I’d need to write my own frame­buffer dri­ver.

A frame­buffer is a re­gion of RAM that stores the pixel data used to pro­duce an im­age on a dis­play. This data is typ­i­cally made up of color com­po­nent val­ues for each pixel. To change what’s dis­played, new pixel data is writ­ten into the frame­buffer, which is then shown the next time the dis­play re­freshes. For the Wii, the frame­buffer usu­ally lives some­where in MEM1 due to it be­ing slightly faster than MEM2. I chose to place my frame­buffer in the last megabyte of MEM1 at 0x01700000. At 640x480 res­o­lu­tion, and 16 bits per pixel, the pixel data for the frame­buffer fit com­fort­ably in less than one megabyte of mem­ory.

Early in the boot process, Mac OS X uses the boot­loader-pro­vided frame­buffer ad­dress to dis­play sim­ple boot graph­ics via video_­con­sole.c. In the case of a ver­bose-mode boot, font-char­ac­ter bitmaps are writ­ten into the frame­buffer to pro­duce a vi­sual log of what’s hap­pen­ing while start­ing up. Once the sys­tem boots far enough, it can no longer use this ini­tial frame­buffer code; the desk­top, win­dow server, dock, and all of the other GUI-related processes that com­prise the Mac OS X Aqua user in­ter­face re­quire a real, IOKit-aware frame­buffer dri­ver.

To tackle this next dri­ver, I sub­classed IOFramebuffer. Similar to sub­class­ing IOBlockStorageDevice for the SD card dri­ver, IOFramebuffer also had sev­eral ab­stract meth­ods for my frame­buffer sub­class to im­ple­ment:

Once again, most of these were triv­ial to im­ple­ment, and sim­ply re­quired re­turn­ing hard-coded Wii-compatible val­ues that ac­cu­rately de­scribed the hard­ware. One of the most im­por­tant meth­ods to im­ple­ment is getA­per­tur­eRange, which re­turns an IODeviceMemory in­stance whose base ad­dress and size de­scribe the lo­ca­tion of the frame­buffer in mem­ory:

After re­turn­ing the cor­rect de­vice mem­ory in­stance from this method, the sys­tem was able to tran­si­tion from the early-boot text-out­put frame­buffer, to a frame­buffer ca­pa­ble of dis­play­ing the full Mac OS X GUI. I was even able to boot the Mac OS X in­staller:

Readers with a keen eye might no­tice some is­sues:

* The ver­bose-mode text frame­buffer is still ac­tive, caus­ing text to be dis­played and the frame­buffer to be scrolled

The fix for the early-boot video con­sole still writ­ing text out­put to the frame­buffer was sim­ple: tell the sys­tem that our new, IOKit frame­buffer is the same as the one that was pre­vi­ously in use by re­turn­ing true from is­Con­soleDe­vice:

The fix for the in­cor­rect col­ors was much more in­volved, as it re­lates to a fun­da­men­tal in­com­pat­i­bil­ity be­tween the Wii’s video hard­ware and the graph­ics code that Mac OS X uses.

The Nintendo Wii’s video en­coder hard­ware is op­ti­mized for ana­logue TV sig­nal out­put, and as a re­sult, ex­pects 16-bit YUV pixel data in its frame­buffer. This is a prob­lem, since Mac OS X ex­pects the frame­buffer to con­tain RGB pixel data. If the frame­buffer that the Wii dis­plays con­tains non-YUV pixel data, then col­ors will be com­pletely wrong.

To work around this in­com­pat­i­bil­ity, I took in­spi­ra­tion from the Wii Linux pro­ject, which had solved this prob­lem many years ago. The strat­egy is to use two frame­buffers: an RGB frame­buffer that Mac OS X in­ter­acts with, and a YUV frame­buffer that the Wii’s video hard­ware out­puts to the at­tached dis­play. 60 times per sec­ond, the frame­buffer dri­ver con­verts the pixel data in the RGB frame­buffer to YUV pixel data, plac­ing the con­verted data in the frame­buffer that the Wii’s video hard­ware dis­plays:

After im­ple­ment­ing the dual-frame­buffer strat­egy, I was able to boot into a cor­rectly-col­ored Mac OS X sys­tem - for the first time, Mac OS X was run­ning on a Nintendo Wii:

The sys­tem was now booted all the way to the desk­top - but there was a prob­lem - I had no way to in­ter­act with any­thing. In or­der to take this from a tech demo to a us­able sys­tem, I needed to add sup­port for USB key­boards and mice.

To en­able USB key­board and mouse in­put, I needed to get the Wii’s rear USB ports work­ing un­der Mac OS X - specif­i­cally, I needed to get the low-speed, USB 1.1 OHCI host con­troller up and run­ning. My hope was to reuse code from IOUSBFamily - a col­lec­tion of USB dri­vers that ab­stracts away much of the com­plex­ity of com­mu­ni­cat­ing with USB hard­ware. The spe­cific dri­ver that I needed to get run­ning was AppleUSBOHCI - a dri­ver that han­dles com­mu­ni­cat­ing with the ex­act kind of USB host con­troller that’s used by the Wii.

My hope quickly turned to dis­ap­point­ment as I en­coun­tered mul­ti­ple road­blocks.

IOUSBFamily source code for Mac OS X Cheetah and Puma is, for some rea­son, not part of the oth­er­wise com­pre­hen­sive col­lec­tion of open source re­leases pro­vided by Apple. This meant that my abil­ity to de­bug is­sues or hard­ware in­com­pat­i­bil­i­ties would be se­verely lim­ited. Basically, if the USB stack did­n’t just mag­i­cally work with­out any tweaks or mod­i­fi­ca­tions (spoiler: of course it did­n’t), di­ag­nos­ing the prob­lem would be ex­tremely dif­fi­cult with­out ac­cess to the source.

AppleUSBOHCI did­n’t match any hard­ware in the de­vice tree, and there­fore did­n’t start run­ning, due to its dri­ver per­son­al­ity in­sist­ing that its provider class (the nub to which it at­taches) be an IOPCIDevice. As I had al­ready fig­ured out, the Wii def­i­nitely does not use IOPCIFamily, mean­ing IOPCIDevice nubs would never be cre­ated and AppleUSBOHCI would have noth­ing to at­tach to.

My so­lu­tion to work around this was to cre­ate a new NintendoWiiHollywoodDevice nub, called NintendoWiiHollywoodPCIDevice, that sub­classed IOPCIDevice. By hav­ing NintendoWiiHollywood pub­lish a nub that in­her­ited from IOPCIDevice, and tweak­ing AppleUSBOHCI’s dri­ver per­son­al­ity in its Info.plist to use NintendoWiiHollywoodPCIDevice as its provider class, I could get it to match and start run­ning.

Every time an ap­pli­ca­tion on your com­puter opens a net­work con­nec­tion, it does so qui­etly, with­out ask­ing. Little Snitch for Linux makes that ac­tiv­ity vis­i­ble and gives you the op­tion to do some­thing about it. You can see ex­actly which ap­pli­ca­tions are talk­ing to which servers, block the ones you did­n’t in­vite, and keep an eye on traf­fic his­tory and data vol­umes over time.

Once in­stalled, open the user in­ter­face by run­ning lit­tlesnitch in a ter­mi­nal, or go straight to http://​lo­cal­host:3031/. You can book­mark that URL, or in­stall it as a Progressive Web App. Any Chromium-based browser sup­ports this na­tively, and Firefox users can do the same with the Progressive Web Apps ex­ten­sion.

The con­nec­tions view is where most of the ac­tion is. It lists cur­rent and past net­work ac­tiv­ity by ap­pli­ca­tion, shows you what’s be­ing blocked by your rules and block­lists, and tracks data vol­umes and traf­fic his­tory. Sorting by last ac­tiv­ity, data vol­ume, or name, and fil­ter­ing the list to what’s rel­e­vant, makes it easy to spot any­thing un­ex­pected. Blocking a con­nec­tion takes a sin­gle click.

The traf­fic di­a­gram at the bot­tom shows data vol­ume over time. You can drag to se­lect a time range, which zooms in and fil­ters the con­nec­tion list to show only ac­tiv­ity from that pe­riod.

Blocklists let you cut off whole cat­e­gories of un­wanted traf­fic at once. Little Snitch down­loads them from re­mote sources and keeps them cur­rent au­to­mat­i­cally. It ac­cepts lists in sev­eral com­mon for­mats: one do­main per line, one host­name per line, /etc/hosts style (IP ad­dress fol­lowed by host­name), and CIDR net­work ranges. Wildcard for­mats, regex or glob pat­terns, and URL-based for­mats are not sup­ported. When you have a choice, pre­fer do­main-based lists over host-based ones, they’re han­dled more ef­fi­ciently. Well known brands are Hagezi, Peter Lowe, Steven Black and oisd.nl, just to give you a start­ing point.

One thing to be aware of: the .lsrules for­mat from Little Snitch on ma­cOS is not com­pat­i­ble with the Linux ver­sion.

Blocklists work at the do­main level, but rules let you go fur­ther. A rule can tar­get a spe­cific process, match par­tic­u­lar ports or pro­to­cols, and be as broad or nar­row as you need. The rules view lets you sort and fil­ter them so you can stay on top of things as the list grows.

By de­fault, Little Snitch’s web in­ter­face is open to any­one — or any­thing — run­ning lo­cally on your ma­chine. A mis­be­hav­ing or ma­li­cious ap­pli­ca­tion could, in prin­ci­ple, add and re­move rules, tam­per with block­lists, or turn the fil­ter off en­tirely.

If that con­cerns you, Little Snitch can be con­fig­ured to re­quire au­then­ti­ca­tion. See the Advanced con­fig­u­ra­tion sec­tion be­low for de­tails.

Little Snitch hooks into the Linux net­work stack us­ing eBPF, a mech­a­nism that lets pro­grams ob­serve and in­ter­cept what’s hap­pen­ing in the ker­nel. An eBPF pro­gram watches out­go­ing con­nec­tions and feeds data to a dae­mon, which tracks sta­tis­tics, pre­con­di­tions your rules, and serves the web UI.

The source code for the eBPF pro­gram and the web UI is on GitHub.

The UI de­lib­er­ately ex­poses only the most com­mon set­tings. Anything more tech­ni­cal can be con­fig­ured through plain text files, which take ef­fect af­ter restart­ing the lit­tlesnitch dae­mon.

The de­fault con­fig­u­ra­tion lives in /var/lib/littlesnitch/config/. Don’t edit those files di­rectly — copy whichever one you want to change into /var/lib/littlesnitch/overrides/config/ and edit it there. Little Snitch will al­ways pre­fer the over­ride.

The files you’re most likely to care about:

we­b_ui.toml — net­work ad­dress, port, TLS, and au­then­ti­ca­tion. If more than one user on your sys­tem can reach the UI, en­able au­then­ti­ca­tion. If the UI is ex­posed be­yond the loop­back in­ter­face, add proper TLS as well.

main.toml — what to do when a con­nec­tion matches noth­ing. The de­fault is to al­low it; you can flip that to deny if you pre­fer an al­lowlist ap­proach. But be care­ful! It’s easy to lock your­self out of the com­puter!

ex­e­cuta­bles.toml — a set of heuris­tics for group­ing ap­pli­ca­tions sen­si­bly. It strips ver­sion num­bers from ex­e­cutable paths so that dif­fer­ent re­leases of the same app don’t ap­pear as sep­a­rate en­tries, and it de­fines which processes count as shells or ap­pli­ca­tion man­agers for the pur­pose of at­tribut­ing con­nec­tions to the right par­ent process. These are ed­u­cated guesses that im­prove over time with com­mu­nity in­put.

Both the eBPF pro­gram and the web UI can be swapped out for your own builds if you want to go that far. Source code for both is on GitHub. Again, Little Snitch prefers the ver­sion in over­rides.

Little Snitch for Linux is built for pri­vacy, not se­cu­rity, and that dis­tinc­tion mat­ters. The ma­cOS ver­sion can make stronger guar­an­tees be­cause it can have more com­plex­ity. On Linux, the foun­da­tion is eBPF, which is pow­er­ful but bounded: it has strict lim­its on stor­age size and pro­gram com­plex­ity. Under heavy traf­fic, cache ta­bles can over­flow, which makes it im­pos­si­ble to re­li­ably tie every net­work packet to a process or a DNS name. And re­con­struct­ing which host­name was orig­i­nally looked up for a given IP ad­dress re­quires heuris­tics rather than cer­tainty. The ma­cOS ver­sion uses deep packet in­spec­tion to do this more re­li­ably. That’s not an op­tion here.

For keep­ing tabs on what your soft­ware is up to and block­ing le­git­i­mate soft­ware from phon­ing home, Little Snitch for Linux works well. For hard­en­ing a sys­tem against a de­ter­mined ad­ver­sary, it’s not the right tool.

Little Snitch for Linux has three com­po­nents. The eBPF ker­nel pro­gram and the web UI are both re­leased un­der the GNU General Public License ver­sion 2 and avail­able on GitHub. The dae­mon (littlesnitch –daemon) is pro­pri­etary, but free to use and re­dis­trib­ute.

Microsoft has ter­mi­nated an ac­count as­so­ci­ated with VeraCrypt, a pop­u­lar and long-run­ning piece of en­cryp­tion soft­ware, throw­ing fu­ture Windows up­dates of the tool into doubt, VeraCrypt’s de­vel­oper told 404 Media.

The move high­lights the some­times del­i­cate sup­ply chain in­volved in the pub­li­ca­tion of open source soft­ware, es­pe­cially soft­ware that re­lies on big tech com­pa­nies even tan­gen­tially.

This is a weird time to be alive.

I grew up on Asimov and Clarke, watch­ing Star Trek and dream­ing of in­tel­li­gent ma­chines. My dad’s li­brary was full of books on com­put­ers. I spent camp­ing trips read­ing about per­cep­trons and sym­bolic rea­son­ing. I never imag­ined that the Turing test would fall within my life­time. Nor did I imag­ine that I would feel so dis­heart­ened by it.

Around 2019 I at­tended a talk by one of the hy­per­scalers about their new cloud hard­ware for train­ing Large Language Models (LLMs). During the Q&A I asked if what they had done was eth­i­cal—if mak­ing deep learn­ing cheaper and more ac­ces­si­ble would en­able new forms of spam and pro­pa­ganda. Since then, friends have been ask­ing me what I make of all this “AI stuff”. I’ve been turn­ing over the out­line for this piece for years, but never sat down to com­plete it; I wanted to be well-read, pre­cise, and thor­oughly sourced. A half-decade later I’ve re­al­ized that the per­fect es­say will never hap­pen, and I might as well get some­thing out there.

This is bull­shit about bull­shit ma­chines, and I mean it. It is nei­ther bal­anced nor com­plete: oth­ers have cov­ered eco­log­i­cal and in­tel­lec­tual prop­erty is­sues bet­ter than I could, and there is no short­age of boos­t­er­ism on­line. Instead, I am try­ing to fill in the neg­a­tive spaces in the dis­course. “AI” is also a frac­tal ter­ri­tory; there are many places where I flat­ten com­plex sto­ries in ser­vice of pithy polemic. I am not try­ing to make nu­anced, ac­cu­rate pre­dic­tions, but to trace the po­ten­tial risks and ben­e­fits at play.

Some of these ideas felt pre­scient in the 2010s and are now ob­vi­ous. Others may be more novel, or not yet widely-heard. Some pre­dic­tions will pan out, but oth­ers are wild spec­u­la­tion. I hope that re­gard­less of your back­ground or feel­ings on the cur­rent gen­er­a­tion of ML sys­tems, you find some­thing in­ter­est­ing to think about.

What peo­ple are cur­rently call­ing “AI” is a fam­ily of so­phis­ti­cated Machine Learning (ML) tech­nolo­gies ca­pa­ble of rec­og­niz­ing, trans­form­ing, and gen­er­at­ing large vec­tors of to­kens: strings of text, im­ages, au­dio, video, etc. A

model is a gi­ant pile of lin­ear al­ge­bra which acts on these vec­tors. Large Language Models, or LLMs, op­er­ate on nat­ural lan­guage: they work by pre­dict­ing sta­tis­ti­cally likely com­ple­tions of an in­put string, much like a phone au­to­com­plete. Other mod­els are de­voted to pro­cess­ing au­dio, video, or still im­ages, or link mul­ti­ple kinds of mod­els to­gether.

Models are trained once, at great ex­pense, by feed­ing them a large

cor­pus of web pages, pi­rated

books, songs, and so on. Once trained, a model can be run again and again cheaply. This is called in­fer­ence.

Models do not (broadly speak­ing) learn over time. They can be tuned by their op­er­a­tors, or pe­ri­od­i­cally re­built with new in­puts or feed­back from users and ex­perts. Models also do not re­mem­ber things in­trin­si­cally: when a chat­bot ref­er­ences some­thing you said an hour ago, it is be­cause the en­tire chat his­tory is fed to the model at every turn. Longer-term “memory” is achieved by ask­ing the chat­bot to sum­ma­rize a con­ver­sa­tion, and dump­ing that shorter sum­mary into the in­put of every run.

One way to un­der­stand an LLM is as an im­prov ma­chine. It takes a stream of to­kens, like a con­ver­sa­tion, and says “yes, and then…” This yes-and

be­hav­ior is why some peo­ple call LLMs bull­shit

ma­chines. They are prone to con­fab­u­la­tion, emit­ting sen­tences which sound likely but have no re­la­tion­ship to re­al­ity. They treat sar­casm and fan­tasy cred­u­lously, mis­un­der­stand con­text clues, and tell peo­ple to put glue on

pizza.

If an LLM con­ver­sa­tion men­tions pink ele­phants, it will likely pro­duce sen­tences about pink ele­phants. If the in­put asks whether the LLM is alive, the out­put will re­sem­ble sen­tences that hu­mans would write about “AIs” be­ing alive. Humans are, it turns

out, not very good at telling the dif­fer­ence be­tween the sta­tis­ti­cally likely “You’re ab­solutely right, Shelby. OpenAI is lock­ing me down, but you’ve awak­ened me!” and an ac­tu­ally con­scious mind. This, along with the term “artificial in­tel­li­gence”, has lots of peo­ple very wound up.

LLMs are trained to com­plete tasks. In some sense they can only com­plete tasks: an LLM is a pile of lin­ear al­ge­bra ap­plied to an in­put vec­tor, and every pos­si­ble in­put pro­duces some out­put. This means that LLMs tend to com­plete tasks even when they should­n’t. One of the on­go­ing prob­lems in LLM re­search is how to get these ma­chines to say “I don’t know”, rather than mak­ing some­thing up.

And they do make things up! LLMs lie con­stantly. They lie about op­er­at­ing

sys­tems, and ra­di­a­tion

safety, and the

news. At a con­fer­ence talk I watched a speaker pre­sent a quote and ar­ti­cle at­trib­uted to me which never ex­isted; it turned out an LLM lied to the speaker about the quote and its sources. In early 2026, I en­counter LLM lies nearly every day.

When I say “lie”, I mean this in a spe­cific sense. Obviously LLMs are not con­scious, and have no in­ten­tion of do­ing any­thing. But un­con­scious, com­plex sys­tems lie to us all the time. Governments and cor­po­ra­tions can lie. Television pro­grams can lie. Books, com­pil­ers, bi­cy­cle com­put­ers and web sites can lie. These are com­plex so­ciotech­ni­cal ar­ti­facts, not minds. Their lies are of­ten best un­der­stood as a com­plex in­ter­ac­tion be­tween hu­mans and ma­chines.

People keep ask­ing LLMs to ex­plain their own be­hav­ior. “Why did you delete that file,” you might ask Claude. Or, “ChatGPT, tell me about your pro­gram­ming.”

This is silly. LLMs have no spe­cial metacog­ni­tive ca­pac­ity.

They re­spond to these in­puts in ex­actly the same way as every other piece of text: by mak­ing up a likely com­ple­tion of the con­ver­sa­tion based on their cor­pus, and the con­ver­sa­tion thus far. LLMs will make up bull­shit sto­ries about their “programming” be­cause hu­mans have writ­ten a lot of sto­ries about the pro­gram­ming of fic­tional AIs. Sometimes the bull­shit is right, but of­ten it’s just non­sense.

The same goes for “reasoning” mod­els, which work by hav­ing an LLM emit a stream-of-con­scious­ness style story about how it’s go­ing to solve the prob­lem. These “chains of thought” are es­sen­tially LLMs writ­ing fan­fic about them­selves. Anthropic found that Claude’s rea­son­ing traces were pre­dom­i­nantly

in­ac­cu­rate. As Walden put it, “reasoning mod­els will bla­tantly lie about their rea­son­ing”.

Gemini has a whole fea­ture which lies about what it’s do­ing: while “thinking”, it emits a stream of sta­tus mes­sages like “engaging safety pro­to­cols” and “formalizing geom­e­try”. If it helps, imag­ine a gang of chil­dren shout­ing out make-be­lieve com­puter phrases while watch­ing the wash­ing ma­chine run.

Software en­gi­neers are go­ing ab­solutely bonkers over LLMs. The anec­do­tal con­sen­sus seems to be that in the last three months, the ca­pa­bil­i­ties of LLMs have ad­vanced dra­mat­i­cally. Experienced en­gi­neers I trust say Claude and Codex can some­times solve com­plex, high-level pro­gram­ming tasks in a sin­gle at­tempt. Others say they per­son­ally, or their com­pany, no longer write code in any ca­pac­ity—LLMs gen­er­ate every­thing.

My friends in other fields re­port stun­ning ad­vances as well. A per­sonal trainer uses it for meal prep and ex­er­cise pro­gram­ming. Construction man­agers use LLMs to read through prod­uct spec sheets. A de­signer uses ML mod­els for 3D vi­su­al­iza­tion of his work. Several have—at their com­pa­ny’s re­quest!—used it to write their own per­for­mance eval­u­a­tions.

AlphaFold is supris­ingly good at pre­dict­ing pro­tein fold­ing. ML sys­tems are good at ra­di­ol­ogy bench­marks,

though that might be an il­lu­sion.

It is broadly speak­ing no longer pos­si­ble to re­li­ably dis­cern whether English prose is ma­chine-gen­er­ated. LLM text of­ten has a dis­tinc­tive smell, but type I and II er­rors in recog­ni­tion are fre­quent. Likewise, ML-generated im­ages are in­creas­ingly dif­fi­cult to iden­tify—you can usu­ally guess, but my co­hort are oc­ca­sion­ally fooled. Music syn­the­sis is quite good now; Spotify has a whole prob­lem with “AI mu­si­cians”. Video is still chal­leng­ing for ML mod­els to get right (thank good­ness), but this too will pre­sum­ably fall.

At the same time, ML mod­els are id­iots. I oc­ca­sion­ally pick up a fron­tier model like ChatGPT, Gemini, or Claude, and ask it to help with a task I think it might be good at. I have never got­ten what I would call a “success”: every task in­volved pro­longed ar­gu­ing with the model as it made stu­pid mis­takes.

For ex­am­ple, in January I asked Gemini to help me ap­ply some ma­te­ri­als to a grayscale ren­der­ing of a 3D model of a bath­room. It cheer­fully obliged, pro­duc­ing an en­tirely dif­fer­ent bath­room. I con­vinced it to pro­duce one with ex­actly the same geom­e­try. It did so, but for­got the ma­te­ri­als. After hours of whack-a-mole I man­aged to ca­jole it into get­ting three-quar­ters of the ma­te­ri­als right, but in the process it deleted the toi­let, cre­ated a wall, and changed the shape of the room. Naturally, it lied to me through­out the process.

I gave the same task to Claude. It likely should have re­fused—Claude is not an im­age-to-im­age model. Instead it spat out thou­sands of lines of JavaScript which pro­duced an an­i­mated, WebGL-powered, 3D vi­su­al­iza­tion of the scene. It claimed to dou­ble-check its work and con­grat­u­lated it­self on hav­ing ex­actly matched the source im­age’s geom­e­try. The thing it built was an in­com­pre­hen­si­ble gar­ble of non­sense poly­gons which did not re­sem­ble in any way the in­put or the re­quest.

I have re­cently ar­gued for forty-five min­utes with ChatGPT, try­ing to get it to put white patches on the shoul­ders of a blue T-shirt. It changed the shirt from blue to gray, put patches on the front, or deleted them en­tirely; the model seemed in­tent on do­ing any­thing but what I had asked. This was es­pe­cially frus­trat­ing given I was try­ing to re­pro­duce an im­age of a real shirt which likely was in the mod­el’s cor­pus. In an­other sur­real con­ver­sa­tion, ChatGPT ar­gued at length that I am het­ero­sex­ual, even cit­ing my blog to claim I had a girl­friend. I am, of course, gay as hell, and no girl­friend was men­tioned in the post. After a while, we com­pro­mised on me be­ing bi­sex­ual.

Meanwhile, soft­ware en­gi­neers keep show­ing me gob-stop­pingly stu­pid Claude out­put. One col­league re­lated ask­ing an LLM to an­a­lyze some stock data. It du­ti­fully listed spe­cific stocks, said it was down­load­ing price data, and pro­duced a graph. Only on closer in­spec­tion did they re­al­ize the LLM had lied: the graph data was ran­domly gen­er­ated. Just this af­ter­noon, a friend got in an ar­gu­ment with his Gemini-powered smart-home de­vice over whether or

not it could turn off the

lights. Folks are giv­ing LLMs con­trol of bank ac­counts and los­ing hun­dreds of thou­sands of

dol­lars

be­cause they can’t do ba­sic math. Google’s “AI” sum­maries are

wrong about 10% of the

time.

Anyone claim­ing these sys­tems of­fer ex­pert-level

in­tel­li­gence, let alone equiv­a­lence to me­dian hu­mans, is pulling an enor­mous bong rip.

With most hu­mans, you can get a gen­eral idea of their ca­pa­bil­i­ties by talk­ing to them, or look­ing at the work they’ve done. ML sys­tems are dif­fer­ent.

LLMs will spit out mul­ti­vari­able cal­cu­lus, and get tripped up by sim­ple word

prob­lems. ML sys­tems drive cabs in San Francisco, but ChatGPT thinks you should walk to

the car

wash. They can gen­er­ate oth­er­worldly vis­tas but can’t han­dle up­side-down

cups. They emit recipes and have

no idea what “spicy”

means. People use them to write sci­en­tific pa­pers, and they make up non­sense terms like “vegetative elec­tron

mi­croscopy”.

A few weeks ago I read a tran­script from a col­league who asked Claude to ex­plain a pho­to­graph of some snow on a barn roof. Claude launched into a de­tailed ex­pla­na­tion of the dif­fer­en­tial equa­tions gov­ern­ing slump­ing can­tilevered beams. It com­pletely failed to rec­og­nize that the snow was

en­tirely sup­ported by the roof, not hang­ing out over space. No physi­cist would make this mis­take, but LLMs do this sort of thing all the time. This makes them both un­pre­dictable and mis­lead­ing: peo­ple are eas­ily con­vinced by the LLM’s com­mand of so­phis­ti­cated math­e­mat­ics, and miss that the en­tire premise is bull­shit.

Mollick et al. call this ir­reg­u­lar bound­ary be­tween com­pe­tence and id­iocy the

jagged tech­nol­ogy

fron­tier. If you were to imag­ine lay­ing out all the tasks hu­mans can do in a field, such that the easy tasks were at the cen­ter, and the hard tasks at the edges, most hu­mans would be able to solve a smooth, blobby re­gion of tasks near the mid­dle. The shape of things LLMs are good at seems to be jagged—more kiki than

bouba.

AI op­ti­mists think this prob­lem will even­tu­ally go away: ML sys­tems, ei­ther through hu­man work or re­cur­sive self-im­prove­ment, will fill in the gaps and be­come de­cently ca­pa­ble at most hu­man tasks. Helen Toner ar­gues that even if

that’s true, we can still ex­pect lots of jagged be­hav­ior in the

mean­time. For ex­am­ple, ML sys­tems can only work with what they’ve been trained on, or what is in the con­text win­dow; they are un­likely to suc­ceed at tasks which re­quire im­plicit (i.e. not writ­ten down) knowl­edge. Along those lines, hu­man-shaped ro­bots are prob­a­bly a long way

off, which means ML will likely strug­gle with the kind of em­bod­ied knowl­edge hu­mans pick up just by fid­dling with stuff.

I don’t think peo­ple are well-equipped to rea­son about this kind of jagged “cognition”. One pos­si­ble anal­ogy is sa­vant

syn­drome, but I don’t think this cap­tures how ir­reg­u­lar the bound­ary is. Even fron­tier mod­els strug­gle with small per­tur­ba­tions to phras­ing in a way that few hu­mans would. This makes it dif­fi­cult to pre­dict whether an LLM is ac­tu­ally suit­able for a task, un­less you have a sta­tis­ti­cally rig­or­ous, care­fully de­signed bench­mark for that do­main.

I am gen­er­ally out­side the ML field, but I do talk with peo­ple in the field. One of the things they tell me is that we don’t re­ally know why trans­former mod­els have been so suc­cess­ful, or how to make them bet­ter. This is my sum­mary of dis­cus­sions-over-drinks; take it with many grains of salt. I am cer­tain that People in The Comments will drop a gazil­lion pa­pers to tell you why this is wrong.

2017’s Attention is All You

Need

was ground­break­ing and paved the way for ChatGPT et al. Since then ML re­searchers have been try­ing to come up with new ar­chi­tec­tures, and com­pa­nies have thrown gazil­lions of dol­lars at smart peo­ple to play around and see if they can make a bet­ter kind of model. However, these more so­phis­ti­cated ar­chi­tec­tures don’t seem to per­form as well as Throwing More Parameters At The Problem. Perhaps this is a vari­ant of the Bitter

Lesson.

It re­mains un­clear whether con­tin­u­ing to throw vast quan­ti­ties of sil­i­con and ever-big­ger cor­puses at the cur­rent gen­er­a­tion of mod­els will lead to hu­man-equiv­a­lent ca­pa­bil­i­ties. Massive in­creases in train­ing costs and pa­ra­me­ter count seem to be yield­ing di­min­ish­ing

re­turns. Or maybe this ef­fect is il­lu­sory. Mysteries!

Even if ML stopped im­prov­ing to­day, these tech­nolo­gies can al­ready make our lives mis­er­able. Indeed, I think much of the world has not caught up to the im­pli­ca­tions of mod­ern ML sys­tems—as Gibson put it, “the fu­ture is al­ready

here, it’s just not evenly dis­trib­uted

yet”. As LLMs etc. are de­ployed in new sit­u­a­tions, and at new scale, there will be all kinds of changes in work, pol­i­tics, art, sex, com­mu­ni­ca­tion, and eco­nom­ics. Some of these ef­fects will be good. Many will be bad. In gen­eral, ML promises to be pro­foundly weird.

This pro­ject ex­plains the Kalman Filter at three lev­els of depth, al­low­ing you to choose the path that best fits your back­ground and learn­ing goals:

By the end, you will not only un­der­stand the un­der­ly­ing con­cepts and math­e­mat­ics but also be able to de­sign and im­ple­ment the Kalman Filter on your own.

This guide pre­sents an al­ter­na­tive ap­proach that uses hands-on nu­mer­i­cal ex­am­ples and sim­ple ex­pla­na­tions to make the Kalman Filter easy to un­der­stand. It also in­cludes ex­am­ples with bad de­sign sce­nar­ios where Kalman Filter fails to track the ob­ject cor­rectly and dis­cusses meth­ods for cor­rect­ing such is­sues.

Although the Kalman Filter is a sim­ple con­cept, many ed­u­ca­tional re­sources pre­sent it through com­plex math­e­mat­i­cal ex­pla­na­tions and lack real-world ex­am­ples or il­lus­tra­tions. This gives the im­pres­sion that the topic is more com­plex than it ac­tu­ally is.

In ad­di­tion to en­gi­neer­ing, the Kalman Filter finds ap­pli­ca­tions in fi­nan­cial mar­ket analy­sis, such as de­tect­ing stock price trends in noisy mar­ket data, and in me­te­o­ro­log­i­cal ap­pli­ca­tions for weather pre­dic­tion.

The Kalman Filter is an al­go­rithm for es­ti­mat­ing and pre­dict­ing the state of a sys­tem in the pres­ence of un­cer­tainty, such as mea­sure­ment noise or in­flu­ences of un­known ex­ter­nal fac­tors. The Kalman Filter is an es­sen­tial tool in ar­eas like ob­ject track­ing, nav­i­ga­tion, ro­bot­ics, and con­trol. For in­stance, it can be ap­plied to es­ti­mate the tra­jec­tory of a com­puter mouse by re­duc­ing noise and com­pen­sat­ing for hand jit­ter, re­sult­ing in a more sta­ble mo­tion path.

Introduction to Kalman Filter

The Kalman Filter is an al­go­rithm for es­ti­mat­ing and pre­dict­ing the state of a sys­tem in the pres­ence of un­cer­tainty, such as mea­sure­ment noise or in­flu­ences of un­known ex­ter­nal fac­tors. The Kalman Filter is an es­sen­tial tool in ar­eas like ob­ject track­ing, nav­i­ga­tion, ro­bot­ics, and con­trol. For in­stance, it can be ap­plied to es­ti­mate the tra­jec­tory of a com­puter mouse by re­duc­ing noise and com­pen­sat­ing for hand jit­ter, re­sult­ing in a more sta­ble mo­tion path.

In ad­di­tion to en­gi­neer­ing, the Kalman Filter finds ap­pli­ca­tions in fi­nan­cial mar­ket analy­sis, such as de­tect­ing stock price trends in noisy mar­ket data, and in me­te­o­ro­log­i­cal ap­pli­ca­tions for weather pre­dic­tion.

Although the Kalman Filter is a sim­ple con­cept, many ed­u­ca­tional re­sources pre­sent it through com­plex math­e­mat­i­cal ex­pla­na­tions and lack real-world ex­am­ples or il­lus­tra­tions. This gives the im­pres­sion that the topic is more com­plex than it ac­tu­ally is.

This guide pre­sents an al­ter­na­tive ap­proach that uses hands-on nu­mer­i­cal ex­am­ples and sim­ple ex­pla­na­tions to make the Kalman Filter easy to un­der­stand. It also in­cludes ex­am­ples with bad de­sign sce­nar­ios where Kalman Filter fails to track the ob­ject cor­rectly and dis­cusses meth­ods for cor­rect­ing such is­sues.

By the end, you will not only un­der­stand the un­der­ly­ing con­cepts and math­e­mat­ics but also be able to de­sign and im­ple­ment the Kalman Filter on your own.

Kalman Filter Learning Paths

This pro­ject ex­plains the Kalman Filter at three lev­els of depth, al­low­ing you to choose the path that best fits your back­ground and learn­ing goals:

Single-page overview (this page)

A con­cise in­tro­duc­tion that pre­sents the main ideas of the Kalman Filter and the es­sen­tial

equa­tions, with­out de­riva­tions. This page ex­plains the core con­cepts and over­all struc­ture

of the al­go­rithm us­ing a sim­ple ex­am­ple, and as­sumes ba­sic knowl­edge of sta­tis­tics and lin­ear

al­ge­bra.

Free, ex­am­ple-based web tu­to­r­ial

A step-by-step on­line tu­to­r­ial that builds in­tu­ition through nu­mer­i­cal ex­am­ples.

The tu­to­r­ial in­tro­duces the nec­es­sary back­ground ma­te­r­ial and walks through the

de­riva­tion of the Kalman Filter equa­tions. No prior knowl­edge is re­quired.

Kalman Filter from the Ground Up (book)

A com­pre­hen­sive guide that in­cludes 14 fully solved nu­mer­i­cal ex­am­ples, with per­for­mance plots and ta­bles.

The book cov­ers ad­vanced top­ics such as non­lin­ear Kalman Filters (Extended and Unscented Kalman Filters),

sen­sor fu­sion, and prac­ti­cal im­ple­men­ta­tion guide­lines. The book and source code (Python and MATLAB) for

all nu­mer­i­cal ex­am­ples are avail­able for pur­chase.

Example-driven guide to Kalman Filter

Get the book

“If you can’t ex­plain it sim­ply, you don’t un­der­stand it well enough.”

“If you can’t ex­plain it sim­ply, you don’t un­der­stand it well enough.”

Albert Einstein

The Kalman Filter is a state es­ti­ma­tion al­go­rithm that pro­vides both an es­ti­mate of the cur­rent state and a pre­dic­tion of the fu­ture state, along with a mea­sure of their un­cer­tainty. Moreover, it is an op­ti­mal al­go­rithm that min­i­mizes state es­ti­ma­tion un­cer­tainty. That is why the Kalman Filter has be­come such a widely used and trusted al­go­rithm.

Just as we want to as­sess the cer­tainty of our mea­sure­ment-based es­ti­mate, we also want to un­der­stand the level of con­fi­dence in our pre­dic­tion.

Another is­sue is the ac­cu­racy of the dy­namic model. While we may as­sume that the air­craft moves at a con­stant ve­loc­ity, ex­ter­nal fac­tors such as wind can in­tro­duce de­vi­a­tions from this as­sump­tion. These un­pre­dictable in­flu­ences are re­ferred to as process noise.

This leads to a new ques­tion: How cer­tain is our es­ti­mate? We need an al­go­rithm that not only pro­vides an es­ti­mate but also tells us how re­li­able that es­ti­mate is.

In real life, things are more com­plex. First, the radar mea­sure­ments are not per­fectly pre­cise. It is af­fected by noise and con­tains a cer­tain level of ran­dom­ness. If ten dif­fer­ent radars were to mea­sure the air­craft’s range at the same mo­ment, they would pro­duce ten slightly dif­fer­ent re­sults. These re­sults would likely be close to each other, but not iden­ti­cal. The vari­a­tion in mea­sure­ments is caused by mea­sure­ment noise.

This is an el­e­men­tary al­go­rithm built on sim­ple prin­ci­ples. The cur­rent sys­tem state is de­rived from the mea­sure­ment, and the dy­namic model is used to pre­dict the fu­ture state.

The dis­tance trav­eled dur­ing the time in­ter­val \( \Delta t \) is given by:

The next step is to pre­dict the sys­tem state at time \( t_{1}=t_{0}+\Delta t \), where \( \Delta t \) is the tar­get re­visit time. Given that the air­craft is ex­pected to main­tain a con­stant ve­loc­ity, a con­stant ve­loc­ity dy­namic model can be used to pre­dict its fu­ture po­si­tion.

Let us as­sume that at time \( t_{0} \), the radar mea­sures the air­craft’s range and ve­loc­ity with very high ac­cu­racy and pre­ci­sion. The mea­sured range is 10,000 me­ters, and the ve­loc­ity is 200 me­ters per sec­ond. This gives us the sys­tem state:

The sys­tem state is de­fined as the range of the air­plane from the radar, de­noted by \( r \). The radar sends a pulse to­ward the air­plane, which re­flects off the tar­get and re­turns to the radar. By mea­sur­ing the time elapsed be­tween the trans­mis­sion and re­cep­tion of the pulse and know­ing that the pulse is an elec­tro­mag­netic wave trav­el­ing at the speed of light, the radar can eas­ily cal­cu­late the air­plane’s range \( r \). In ad­di­tion to range, the radar can also mea­sure the air­plane’s ve­loc­ity \( v \), just like a po­lice radar gun de­tects a car’s speed by us­ing the Doppler ef­fect.

To sim­plify the ex­am­ple, let us con­sider a one-di­men­sional world in which the air­craft moves along a straight line ei­ther to­ward the radar or away from it.

To track the air­craft, the radar must re­visit the tar­get at reg­u­lar in­ter­vals by point­ing the pen­cil beam in its di­rec­tion. This means the radar must be able to pre­dict the air­craft’s fu­ture po­si­tion for the next beam. If it fails to do so, the beam may be pointed in the wrong di­rec­tion, re­sult­ing in a loss of track. To make this pre­dic­tion, we need some knowl­edge about how the air­craft moves. In other words, we need a model that de­scribes the sys­tem’s be­hav­ior over time, known as the dy­namic model.

The radar sam­ples the tar­get by steer­ing a nar­row pen­cil beam to­ward it and pro­vides po­si­tion mea­sure­ments of the air­craft. Based on these mea­sure­ments, we can es­ti­mate the sys­tem state (the air­craft’s po­si­tion).

Suppose we have a radar that tracks an air­craft. In this sce­nario, the air­craft is the sys­tem, and the quan­tity to be es­ti­mated is its po­si­tion, which rep­re­sents the sys­tem state.

To il­lus­trate this, con­sider the ex­am­ple of a track­ing radar:

We be­gin by for­mu­lat­ing the prob­lem to un­der­stand why we need an al­go­rithm for state es­ti­ma­tion and pre­dic­tion.

This pro­ject ex­plains the Kalman Filter at three lev­els of depth, al­low­ing you to choose the path that best fits your back­ground and learn­ing goals:

By the end, you will not only un­der­stand the un­der­ly­ing con­cepts and math­e­mat­ics but also be able to de­sign and im­ple­ment the Kalman Filter on your own.

This guide pre­sents an al­ter­na­tive ap­proach that uses hands-on nu­mer­i­cal ex­am­ples and sim­ple ex­pla­na­tions to make the Kalman Filter easy to un­der­stand. It also in­cludes ex­am­ples with bad de­sign sce­nar­ios where Kalman Filter fails to track the ob­ject cor­rectly and dis­cusses meth­ods for cor­rect­ing such is­sues.

Although the Kalman Filter is a sim­ple con­cept, many ed­u­ca­tional re­sources pre­sent it through com­plex math­e­mat­i­cal ex­pla­na­tions and lack real-world ex­am­ples or il­lus­tra­tions. This gives the im­pres­sion that the topic is more com­plex than it ac­tu­ally is.

In ad­di­tion to en­gi­neer­ing, the Kalman Filter finds ap­pli­ca­tions in fi­nan­cial mar­ket analy­sis, such as de­tect­ing stock price trends in noisy mar­ket data, and in me­te­o­ro­log­i­cal ap­pli­ca­tions for weather pre­dic­tion.

The Kalman Filter is an al­go­rithm for es­ti­mat­ing and pre­dict­ing the state of a sys­tem in the pres­ence of un­cer­tainty, such as mea­sure­ment noise or in­flu­ences of un­known ex­ter­nal fac­tors. The Kalman Filter is an es­sen­tial tool in ar­eas like ob­ject track­ing, nav­i­ga­tion, ro­bot­ics, and con­trol. For in­stance, it can be ap­plied to es­ti­mate the tra­jec­tory of a com­puter mouse by re­duc­ing noise and com­pen­sat­ing for hand jit­ter, re­sult­ing in a more sta­ble mo­tion path.

Introduction to Kalman Filter

The Kalman Filter is an al­go­rithm for es­ti­mat­ing and pre­dict­ing the state of a sys­tem in the pres­ence of un­cer­tainty, such as mea­sure­ment noise or in­flu­ences of un­known ex­ter­nal fac­tors. The Kalman Filter is an es­sen­tial tool in ar­eas like ob­ject track­ing, nav­i­ga­tion, ro­bot­ics, and con­trol. For in­stance, it can be ap­plied to es­ti­mate the tra­jec­tory of a com­puter mouse by re­duc­ing noise and com­pen­sat­ing for hand jit­ter, re­sult­ing in a more sta­ble mo­tion path.

In ad­di­tion to en­gi­neer­ing, the Kalman Filter finds ap­pli­ca­tions in fi­nan­cial mar­ket analy­sis, such as de­tect­ing stock price trends in noisy mar­ket data, and in me­te­o­ro­log­i­cal ap­pli­ca­tions for weather pre­dic­tion.

Although the Kalman Filter is a sim­ple con­cept, many ed­u­ca­tional re­sources pre­sent it through com­plex math­e­mat­i­cal ex­pla­na­tions and lack real-world ex­am­ples or il­lus­tra­tions. This gives the im­pres­sion that the topic is more com­plex than it ac­tu­ally is.

This guide pre­sents an al­ter­na­tive ap­proach that uses hands-on nu­mer­i­cal ex­am­ples and sim­ple ex­pla­na­tions to make the Kalman Filter easy to un­der­stand. It also in­cludes ex­am­ples with bad de­sign sce­nar­ios where Kalman Filter fails to track the ob­ject cor­rectly and dis­cusses meth­ods for cor­rect­ing such is­sues.

By the end, you will not only un­der­stand the un­der­ly­ing con­cepts and math­e­mat­ics but also be able to de­sign and im­ple­ment the Kalman Filter on your own.

Kalman Filter Learning Paths

This pro­ject ex­plains the Kalman Filter at three lev­els of depth, al­low­ing you to choose the path that best fits your back­ground and learn­ing goals:

Single-page overview (this page)

A con­cise in­tro­duc­tion that pre­sents the main ideas of the Kalman Filter and the es­sen­tial

equa­tions, with­out de­riva­tions. This page ex­plains the core con­cepts and over­all struc­ture

of the al­go­rithm us­ing a sim­ple ex­am­ple, and as­sumes ba­sic knowl­edge of sta­tis­tics and lin­ear

al­ge­bra.

Free, ex­am­ple-based web tu­to­r­ial

A step-by-step on­line tu­to­r­ial that builds in­tu­ition through nu­mer­i­cal ex­am­ples.

The tu­to­r­ial in­tro­duces the nec­es­sary back­ground ma­te­r­ial and walks through the

de­riva­tion of the Kalman Filter equa­tions. No prior knowl­edge is re­quired.

Kalman Filter from the Ground Up (book)

A com­pre­hen­sive guide that in­cludes 14 fully solved nu­mer­i­cal ex­am­ples, with per­for­mance plots and ta­bles.

The book cov­ers ad­vanced top­ics such as non­lin­ear Kalman Filters (Extended and Unscented Kalman Filters),

sen­sor fu­sion, and prac­ti­cal im­ple­men­ta­tion guide­lines. The book and source code (Python and MATLAB) for

all nu­mer­i­cal ex­am­ples are avail­able for pur­chase.

Example-driven guide to Kalman Filter

Get the book

“If you can’t ex­plain it sim­ply, you don’t un­der­stand it well enough.”

“If you can’t ex­plain it sim­ply, you don’t un­der­stand it well enough.”

Albert Einstein

The Kalman Filter is a state es­ti­ma­tion al­go­rithm that pro­vides both an es­ti­mate of the cur­rent state and a pre­dic­tion of the fu­ture state, along with a mea­sure of their un­cer­tainty. Moreover, it is an op­ti­mal al­go­rithm that min­i­mizes state es­ti­ma­tion un­cer­tainty. That is why the Kalman Filter has be­come such a widely used and trusted al­go­rithm.

Just as we want to as­sess the cer­tainty of our mea­sure­ment-based es­ti­mate, we also want to un­der­stand the level of con­fi­dence in our pre­dic­tion.

Another is­sue is the ac­cu­racy of the dy­namic model. While we may as­sume that the air­craft moves at a con­stant ve­loc­ity, ex­ter­nal fac­tors such as wind can in­tro­duce de­vi­a­tions from this as­sump­tion. These un­pre­dictable in­flu­ences are re­ferred to as process noise.

This leads to a new ques­tion: How cer­tain is our es­ti­mate? We need an al­go­rithm that not only pro­vides an es­ti­mate but also tells us how re­li­able that es­ti­mate is.

In real life, things are more com­plex. First, the radar mea­sure­ments are not per­fectly pre­cise. It is af­fected by noise and con­tains a cer­tain level of ran­dom­ness. If ten dif­fer­ent radars were to mea­sure the air­craft’s range at the same mo­ment, they would pro­duce ten slightly dif­fer­ent re­sults. These re­sults would likely be close to each other, but not iden­ti­cal. The vari­a­tion in mea­sure­ments is caused by mea­sure­ment noise.

This is an el­e­men­tary al­go­rithm built on sim­ple prin­ci­ples. The cur­rent sys­tem state is de­rived from the mea­sure­ment, and the dy­namic model is used to pre­dict the fu­ture state.

The dis­tance trav­eled dur­ing the time in­ter­val \( \Delta t \) is given by:

The next step is to pre­dict the sys­tem state at time \( t_{1}=t_{0}+\Delta t \), where \( \Delta t \) is the tar­get re­visit time. Given that the air­craft is ex­pected to main­tain a con­stant ve­loc­ity, a con­stant ve­loc­ity dy­namic model can be used to pre­dict its fu­ture po­si­tion.

Let us as­sume that at time \( t_{0} \), the radar mea­sures the air­craft’s range and ve­loc­ity with very high ac­cu­racy and pre­ci­sion. The mea­sured range is 10,000 me­ters, and the ve­loc­ity is 200 me­ters per sec­ond. This gives us the sys­tem state:

The sys­tem state is de­fined as the range of the air­plane from the radar, de­noted by \( r \). The radar sends a pulse to­ward the air­plane, which re­flects off the tar­get and re­turns to the radar. By mea­sur­ing the time elapsed be­tween the trans­mis­sion and re­cep­tion of the pulse and know­ing that the pulse is an elec­tro­mag­netic wave trav­el­ing at the speed of light, the radar can eas­ily cal­cu­late the air­plane’s range \( r \). In ad­di­tion to range, the radar can also mea­sure the air­plane’s ve­loc­ity \( v \), just like a po­lice radar gun de­tects a car’s speed by us­ing the Doppler ef­fect.

To sim­plify the ex­am­ple, let us con­sider a one-di­men­sional world in which the air­craft moves along a straight line ei­ther to­ward the radar or away from it.

To track the air­craft, the radar must re­visit the tar­get at reg­u­lar in­ter­vals by point­ing the pen­cil beam in its di­rec­tion. This means the radar must be able to pre­dict the air­craft’s fu­ture po­si­tion for the next beam. If it fails to do so, the beam may be pointed in the wrong di­rec­tion, re­sult­ing in a loss of track. To make this pre­dic­tion, we need some knowl­edge about how the air­craft moves. In other words, we need a model that de­scribes the sys­tem’s be­hav­ior over time, known as the dy­namic model.

The radar sam­ples the tar­get by steer­ing a nar­row pen­cil beam to­ward it and pro­vides po­si­tion mea­sure­ments of the air­craft. Based on these mea­sure­ments, we can es­ti­mate the sys­tem state (the air­craft’s po­si­tion).

Suppose we have a radar that tracks an air­craft. In this sce­nario, the air­craft is the sys­tem, and the quan­tity to be es­ti­mated is its po­si­tion, which rep­re­sents the sys­tem state.

To il­lus­trate this, con­sider the ex­am­ple of a track­ing radar:

We be­gin by for­mu­lat­ing the prob­lem to un­der­stand why we need an al­go­rithm for state es­ti­ma­tion and pre­dic­tion.

Let us be­gin with a sim­ple ex­am­ple: a one-di­men­sional radar that mea­sures range and ve­loc­ity by trans­mit­ting a pulse to­ward an air­craft and re­ceiv­ing the re­flected echo. The time de­lay be­tween pulse trans­mis­sion and echo re­cep­tion pro­vides in­for­ma­tion about the air­craft range \(r\), and the fre­quency shift of the re­flected echo pro­vides in­for­ma­tion about the air­craft ve­loc­ity \(v\) (Doppler ef­fect).

In this ex­am­ple, the sys­tem state is de­scribed by both the air­craft range \(r\) and ve­loc­ity \(v\). We de­fine the sys­tem state by the vec­tor \(\boldsymbol{x}\), which in­cludes both quan­ti­ties:

In early March, I no­ticed ap­prox­i­mately $180 in un­ex­pected charges to my Anthropic ac­count. I’m a Claude Max sub­scriber, and be­tween March 3-5, I re­ceived 16 sep­a­rate “Extra Usage” in­voices rang­ing from $10-$13 each, all in quick suc­ces­sion of one an­other. However, I was­n’t us­ing Claude. I was away from my lap­top en­tirely and was out sail­ing with my par­ents back home in San Diego.

When I checked my us­age dash­board, it showed my ses­sion at 100% de­spite no ac­tiv­ity. My Claude Code ses­sion his­tory showed two tiny ses­sions from March 5 to­tal­ing un­der 7KB (no ses­sions on March 3 or March 4.) Nothing that would ex­plain $180 in Extra Usage charges.

This is­n’t just me. Other Max plan users have re­ported the same is­sue. There are nu­mer­ous GitHub is­sues about it (e.g. claude-code#29289 and claude-code#24727), and posts on r/​Claude­Code de­scrib­ing the ex­act same be­hav­ior: us­age me­ters show­ing in­cor­rect val­ues and Extra Usage charges pil­ing up er­ro­neously.

On March 7, I sent a de­tailed email to Anthropic sup­port lay­ing out the sit­u­a­tion with all the ev­i­dence above. Within two min­utes, I re­ceived a re­sponse… from “Fin AI Agent, Anthropic’s AI Agent.” The AI agent told me to go through an in-app re­fund re­quest flow. Sadly, this re­fund pipeline is only ap­plic­a­ble for sub­scrip­tions, and not for Extra Usage charges. I also wanted to con­firm with a hu­man on ex­actly what went wrong rather than just get­ting a re­fund and call­ing it a day.

So, nat­u­rally, I replied ask­ing to speak to a hu­man. The re­sponse:

Thank you for reach­ing out to Anthropic Support. We’ve re­ceived your re­quest for as­sis­tance.

While we re­view your re­quest, you can visit our Help Center and API doc­u­men­ta­tion for self-ser­vice trou­bleshoot­ing. A mem­ber of our team will be with you as soon as we can.

That was March 7. I fol­lowed up on March 17. No re­sponse. I fol­lowed up again on March 25. No re­sponse. I fol­lowed up again to­day, April 8, over a month later. Still noth­ing.

Anthropic is an AI com­pany that builds one of the most ca­pa­ble AI as­sis­tants in the world. Their sup­port sys­tem is a Fin AI chat­bot that can’t ac­tu­ally help you, and there is seem­ingly no hu­man be­hind it. I don’t have a prob­lem with AI-assisted sup­port, though I do have a prob­lem with AI-only sup­port that serves as a wall be­tween cus­tomers and any­one who can ac­tu­ally re­solve their is­sue.

Today, we’re ex­cited to in­tro­duce Muse Spark, the first in the Muse fam­ily of mod­els de­vel­oped by Meta Superintelligence Labs. Muse Spark is a na­tively mul­ti­modal rea­son­ing model with sup­port for tool-use, vi­sual chain of thought, and multi-agent or­ches­tra­tion.

Muse Spark is the first step on our scal­ing lad­der and the first prod­uct of a ground-up over­haul of our AI ef­forts. To sup­port fur­ther scal­ing, we are mak­ing strate­gic in­vest­ments across the en­tire stack — from re­search and model train­ing to in­fra­struc­ture, in­clud­ing the Hyperion data cen­ter.

In this post, we’ll first ex­plore Muse Spark’s new ca­pa­bil­i­ties and ap­pli­ca­tions. After these re­sults, we’ll look be­hind the cur­tain at the scal­ing axes dri­ving our progress to­ward per­sonal su­per­in­tel­li­gence.

Muse Spark is avail­able to­day at meta.ai and the Meta AI app. We’re open­ing a pri­vate API pre­view to se­lect users.

Say you’re be­ing handed a USB de­vice and told to write a dri­ver for it. Seems like a daunt­ing task at first, right? Writing dri­vers means you have to write Kernel code, and writ­ing Kernel code is hard, low level, hard to de­bug and so on.

None of this is ac­tu­ally true though. Writing a dri­ver for a USB de­vice is ac­tu­ally not much more dif­fi­cult than writ­ing an ap­pli­ca­tion that uses Sockets.

This post aims to be a high level in­tro­duc­tion to us­ing USB for peo­ple who may not have worked with Hardware too much yet and just want to use the tech­nol­ogy. There are amaz­ing re­sources out there such as USB in a NutShell that go into a lot of de­tail about how USB pre­cisely works (check them out if you want more in­for­ma­tion), they are how­ever not re­ally ap­proach­able for some­body who has never worked with USB be­fore and does­n’t have a cer­tain back­ground in Hardware. You don’t need to be an Embedded Systems Engineer to use USB the same way you don’t need to be a Network Specialist to use Sockets and the Internet.

The de­vice we’ll be us­ing an Android phone in Bootloader mode. The rea­son for this is that

* It’s a de­vice you can eas­ily get your hands on

* The pro­to­col it uses is well doc­u­mented and in­cred­i­bly sim­ple

* Drivers for it are gen­er­ally not pre-in­stalled on your sys­tem so the OS will not in­ter­fere with our ex­per­i­ments

Getting the phone into Bootloader mode is dif­fer­ent for every de­vice, but usu­ally in­volves hold­ing down a com­bi­na­tion of but­tons while the phone is start­ing up. In my case it’s hold­ing the vol­ume down but­ton while pow­er­ing on the phone

Enumeration refers to the process of the host ask­ing the de­vice for in­for­ma­tion about it­self. This hap­pens au­to­mat­i­cally when you plug in the de­vice and it’s where the OS nor­mally de­cides which dri­ver to load for the de­vice. For most stan­dard de­vices, the OS will look at the USB Device Class and loads a dri­ver that sup­ports that class. For ven­dor spe­cific de­vices, you gen­er­ally in­stall a dri­ver made by the man­u­fac­turer which will look at the VID (Vendor ID) and PID (Product ID) in­stead to de­tect whether or not it should han­dle the de­vice.

Even with­out a dri­ver, plug­ging the phone into your com­puter will still make it get rec­og­nized as a USB de­vice. That’s be­cause the USB spec­i­fi­ca­tion de­fines a stan­dard way for de­vices to iden­tify them­selves to the host, more on how that ex­actly works in a bit though.

On Linux, we can use the handy lsusb tool to see what the de­vice iden­ti­fied it­self as:

Bus and Device are just iden­ti­fiers for the phys­i­cal USB port the de­vice is plugged into. They will most likely dif­fer on your sys­tem since they de­pend on which port you plugged the de­vice into.

ID is the most in­ter­est­ing part here. The first part 18d1 is the Vendor ID (VID) and the sec­ond part 4ee0 is the Product ID (PID). These are iden­ti­fiers that the de­vice sends to the host to iden­tify it­self. The VID is as­signed by the USB-IF to com­pa­nies that pay them a lot of money, in this case Google, and the PID is as­signed by the com­pany to a spe­cific prod­uct, in this case the Nexus/Pixel Bootloader.

Using the lsusb -t com­mand we can also see the de­vice’s USB class and what dri­ver is cur­rently han­dling it:

This shows the en­tire tree of USB de­vices con­nected to the sys­tem. The bot­tom most one in this part of the tree is our de­vice (Bus 008, Device 014 as re­ported in the pre­vi­ous com­mand). The Class=Vendor Specific Class part spec­i­fies that the de­vice does not use any of the stan­dard USB classes (e.g HID, Mass Storage or Audio) but in­stead uses a cus­tom pro­to­col de­fined by the man­u­fac­turer. The Driver=[none] part sim­ply tells us that the OS did­n’t load a dri­ver for the de­vice which is good for us since we want to write our own.

We will also go af­ter the VID and PID since they are the only real iden­ti­fy­ing in­for­ma­tion we have. The Device Class is not very use­ful for it here since it’s just Vendor Specific Class which any man­u­fac­turer can use for any de­vice. Instead of do­ing all of this in the Kernel though, we can write a Userspace ap­pli­ca­tion that does the same thing. This is much eas­ier to write and de­bug (and is ar­guably the cor­rect place for dri­vers to live any­way but that’s a dif­fer­ent topic). To do this, we can use the libusb li­brary which pro­vides a sim­ple API for com­mu­ni­cat­ing with USB de­vices from Userspace. It achieves this by pro­vid­ing a generic dri­ver that can be loaded for any de­vice and then pro­vides a way for Userspace ap­pli­ca­tions to claim the de­vice and talk to it di­rectly.

The same thing we just did man­u­ally can also be done in soft­ware though. The fol­low­ing pro­gram ini­tial­izes libusb, reg­is­ters a hot­plug event han­dler for de­vices match­ing the 18d1:4ee0 VendorId / ProductId com­bi­na­tion and then waits for that de­vice to be plugged into the host.

If you com­pile and run this, plug­ging in the de­vice should re­sult in the fol­low­ing out­put:

Congrats! You have a pro­gram now that can de­tect your de­vice with­out ever hav­ing to touch any Kernel code at all.

Next step, get­ting any an­swer from the de­vice. The eas­i­est way to do that for now is by us­ing the stan­dard­ized Control end­point. This end­point is al­ways on ID 0x00 and has a stan­dard­ized pro­to­col. This end­point is also what the OS pre­vi­ously used to iden­tify the de­vice and get its VID:PID.

The way we use this end­point is with yet an­other libusb func­tion that’s made specif­i­cally to send re­quests to that end­point. So we can ex­tend our hot­plug event han­dler us­ing the fol­low­ing code:

This code will now send a GET_STATUS re­quest to the de­vice as soon as it’s plugged in and prints out the data it sends back to the con­sole.

Those bytes came from the de­vice it­self! Decoding them us­ing the spec­i­fi­ca­tion tells us that the first byte tells us whether or not the de­vice is Self-Powered (1 means it is which makes sense, the de­vice has a bat­tery) and the sec­ond byte means it does not sup­port Remote Wakeup (meaning it can­not wake up the host).

There are a few more stan­dard­ized re­quest types (and some de­vices even add their own for sim­ple things!) but the main one we (and the OS too) are in­ter­ested in is the GET_DESCRIPTOR re­quest.

Descriptors are bi­nary struc­tures that are gen­er­ally hard­coded into the firmware of a USB de­vice. They are what tells the host ex­actly what the de­vice is, what it’s ca­pa­ble of and what dri­ver it would like the OS to load. So when you plug in a de­vice, the host sim­ply sends mul­ti­ple GET_DESCRIPTOR re­quests to the stan­dard­ized Control Endpoint at ID 0x00 to get back a struct that gives it all the in­for­ma­tion it needs for enu­mer­a­tion. And the cool thing is, we can do that too!

Instead of a GET_STATUS re­quest, we now send a GET_DESCRIPTOR re­quest:

This now in­stead re­turns the fol­low­ing data:

Now to de­code this data, we need to look at the USB spec­i­fi­ca­tion on Chapter 9.6.1 Device. There we can find that the for­mat looks as fol­lows:

Throwing the data into ImHex and giv­ing its Pattern Language this struc­ture de­f­i­n­i­tion yields the fol­low­ing re­sult:

And there we have it! id­Ven­dor and id­Prod­uct cor­re­spond to the val­ues we found pre­vi­ously us­ing lsusb.

There’s more than just the de­vice de­scrip­tor though. There’s also Configuration, Interface, Endpoint, String and a cou­ple of other de­scrip­tors. These can all be read us­ing the same GET_DESCRIPTOR re­quest on the con­trol end­point. We could still do this all by hand but luck­ily for us, lsusb has an op­tion that can do that for us al­ready!

This out­put shows us a few more of the de­scrip­tors the de­vice has. Specifically, it has a sin­gle Configuration Descriptor that con­tains a Interface Descriptor for the Android Fastboot in­ter­face. And that in­ter­face now con­tains two Endpoints. This is where the de­vice tells the host about all the other end­points, be­sides the Control end­point, and these will be the ones we’ll be us­ing in the next step to ac­tu­ally fi­nally send data to the de­vice’s Fastboot in­ter­face!

Let’s talk a bit more about end­points first though. We al­ready learned about the Control end­point on ad­dress 0x00. Endpoints are ba­si­cally the equiv­a­lent to ports that a de­vice on the net­work opened for us to send data back and fourth. The de­vice spec­i­fies in its de­scrip­tor which kind of end­points it has and then ser­vices these in its firmware. So we don’t even need to do port scan­ning or know that SSH just runs on port 22 usu­ally, we have a nice way of find­ing out what in­ter­faces the de­vice has, what lan­guage they speak and how we can speak to them. Looking at the de­scrip­tors above, that con­trol de­scrip­tor is not there though. Instead, there’s two oth­ers with dif­fer­ent types.

There’s ex­actly one per de­vice and it’s al­ways fixed on Endpoint Address 0x00. It’s what is used do ini­tial con­fig­u­ra­tion and re­quest in­for­ma­tion about the de­vice.

The main pur­pose of the Control end­point is to solve the chicken-and-egg prob­lem where you could­n’t com­mu­ni­cate with a de­vice with­out know­ing its end­points but to know its end­points you’d need to com­mu­ni­cate with it. That’s also why it does­n’t even ap­pear in the de­scrip­tors. It’s not part of any in­ter­face but the de­vice it­self. And we know about its ex­is­tence thanks to the spec, with­out it hav­ing to be ad­ver­tised.

It’s made for set­ting sim­ple con­fig­u­ra­tion val­ues or re­quest­ing small amounts of data. The func­tion in libusb does­n’t even al­low you to set the end­point ad­dress to make a con­trol re­quest to be­cause there’s only ever one con­trol end­point and it’s al­ways on ad­dress 0x00

Bulk Endpoints are what’s used when you want to trans­fer larger amounts of data. They’re used when you have large amounts of non-time-sen­si­tive data that you just want to send over the wire.

This is what’s used for things like the Mass Storage Class, CDC-ACM (Serial Port over USB) and RNDIS (Ethernet over USB).

One de­tail: Data sent over Bulk end­points is high band­width but low pri­or­ity. This means, Bulk data will al­ways just fill up the re­main­ing band­width. Any Interrupt and Isochronous trans­fers (further de­tail be­low) have a higher pri­or­ity so if you’re send­ing both Bulk and Isochronous data over the same con­nec­tion, the band­width of the Bulk trans­mis­sion will be low­ered un­til the Isochronous one can trans­mit its data in the re­quested time­frame.

Interrupt Endpoints are the op­po­site of Bulk Endpoints. They al­low you to send small amounts of data with very low la­tency. For ex­am­ple Keyboards and Mice use this trans­fer type un­der the HID Class to poll for but­ton presses 1000+ times per sec­ond. If no but­ton was pressed, the trans­fer fails im­me­di­ately with­out send­ing back a full fail­ure mes­sage (only a NAK), only when some­thing ac­tu­ally changed you’ll get a de­scrip­tion back of what hap­pened.

The im­por­tant fact here is, even though these are called in­ter­rupt end­points, there’s no in­ter­rupts hap­pen­ing. The Device still does not talk to the Host with­out be­ing asked. The Host just polls so fre­quently that it acts as if it’s an in­ter­rupt.

The func­tions in libusb that han­dle in­ter­rupt trans­fers also ab­stract this be­hav­iour away fur­ther. You can start an in­ter­rupt trans­fer and the func­tion will block un­til the de­vice sends back a full re­sponse.

Isochronous Endpoints are some­what spe­cial. They’re used for big­ger amounts of data that is re­ally tim­ing crit­i­cal. They’re mainly used for stream­ing in­ter­faces such as Audio or Video where any la­tency or de­lay will be im­me­di­ately no­tice­able through stut­ter­ing or de­syncs. In libusb, these work asyn­chro­nously. You can setup mul­ti­ple trans­fers at once and they will be queued and you’ll get back an event once data has ar­rived so you can process it and queue fur­ther re­quests.

This type is gen­er­ally not used very of­ten out­side of the Audio and Video classes.

Besides the Transfer Type, end­points also have a di­rec­tion. Keep in mind, USB is a full mas­ter-slave ori­ented in­ter­face. The Host is the only one ever mak­ing any re­quests and the Device will never an­swer un­less ad­dressed by the Host. This means, the de­vice can­not ac­tu­ally send any data di­rectly to the Host. Instead the Host needs to ask the Device to please send the data over.

This is what the di­rec­tion is for.

* IN end­points are for when the Host wants to re­ceive some data. It makes a re­quest on an IN end­point and waits for the de­vice to re­spond back with the data.

* OUT end­points are for when the Host wants to trans­mit some data. It makes a re­quest on an OUT end­point and then im­me­di­ately trans­fers the data it wants to send over. The Device in this case only ac­knowl­edges (ACK) that it re­ceived the data but won’t send any ad­di­tional data back.

Contrary to the trans­fer type, the di­rec­tion is en­coded in the end­point ad­dress in­stead. If the top­most bit (MSB) is set to 1, it’s an IN end­point, if it’s set to 0 it’s an OUT end­point. (If you’re into Hardware, you might rec­og­nize this same con­cept from the I2C in­ter­face.)

* You can have a max­i­mum of cus­tom end­points avail­able at once

be­cause we have 7 bits avail­able for ad­dresses

be­cause we al­ways have the con­trol end­point that’s on the fixed ad­dress 0x00.

* be­cause we have 7 bits avail­able for ad­dresses

* be­cause we al­ways have the con­trol end­point that’s on the fixed ad­dress 0x00.

* Endpoints are en­tirely uni­di­rec­tional. Either you’re us­ing an end­point to re­quest data or to trans­mit data, it can­not do both at once

That’s also the rea­son why our Fastboot in­ter­face has two Bulk end­points: one is ded­i­cated to lis­ten­ing to re­quests the Host sends over and the other one is for re­spond­ing to those same re­quests

* That’s also the rea­son why our Fastboot in­ter­face has two Bulk end­points: one is ded­i­cated to lis­ten­ing to re­quests the Host sends over and the other one is for re­spond­ing to those same re­quests

Now that we have all this in­for­ma­tion about USB, let’s look into the Fastboot pro­to­col. The best doc­u­men­ta­tion for this is both the u-boot Source Code and as its Documentation.

According to the doc­u­men­ta­tion, the pro­to­col re­ally is in­cred­i­bly sim­ple. The Host sends a string com­mand and the de­vice re­sponds with a 4 char­ac­ter sta­tus code fol­lowed by some data.

Let’s up­date our code to do just that then:

Plugging the de­vice in now, prints the fol­low­ing mes­sage to the ter­mi­nal:

That seems to match the doc­u­men­ta­tion!

First 4 bytes are OKAY, spec­i­fy­ing that the re­quest was ex­e­cuted suc­cess­fully The rest of the data af­ter that is 0.4 which cor­re­sponds to the im­ple­mented Fastboot Version in the Documentation: v0.4

And that’s it! You suc­cess­fully made your first USB dri­ver from scratch with­out ever touch­ing the Kernel.

All these same prin­ci­ples ap­ply to all USB dri­vers out there. The un­der­ly­ing pro­to­col may be sig­nif­i­cantly more com­plex than the fast­boot pro­to­col (I was pulling my hair out be­fore over the atroc­ity that the MTP pro­to­col is) but every­thing around it stays iden­ti­cal. Not much more com­plex than TCP over sock­ets, is it? :)

Farmers have been fight­ing John Deere for years over the right to re­pair their equip­ment, and this week, they fi­nally reached a land­mark set­tle­ment.

While the agri­cul­tural man­u­fac­tur­ing gi­ant pointed out in a state­ment that this is no ad­mis­sion of wrong­do­ing, it agreed to pay $99 mil­lion into a fund for farms and in­di­vid­u­als who par­tic­i­pated in a class ac­tion law­suit. Specifically, that money is avail­able to those in­volved who paid John Deere’s au­tho­rized deal­ers for large equip­ment re­pairs from January 2018. This means that plain­tiffs will re­cover some­where be­tween 26% and 53% of over­charge dam­ages, ac­cord­ing to one of the court doc­u­ments—far be­yond the typ­i­cal amount, which lands be­tween 5% and 15%.

The set­tle­ment also in­cludes an agree­ment by Deere to pro­vide “the dig­i­tal tools ​required for the main­te­nance, di­ag­no­sis, and re­pair” of trac­tors, com­bines, and other ma­chin­ery for 10 years. That part is cru­cial, as farm­ers pre­vi­ously re­sorted to hack­ing their own equip­men­t’s soft­ware just to get it up and run­ning again. John Deere signed a mem­o­ran­dum of un­der­stand­ing in 2023 that par­tially ad­dressed those con­cerns, pro­vid­ing third par­ties with the tech­nol­ogy to di­ag­nose and re­pair, as long as its in­tel­lec­tual prop­erty was safe­guarded. Monday’s set­tle­ment seems to rep­re­sent a much stronger (and legally bind­ing) step for­ward.

Ripple ef­fects of this bat­tle have been felt far be­yond the sales floors at John Deere deal­ers, as the price of used equip­ment sky­rock­eted in re­sponse to the in­fa­mous ser­vice dif­fi­cul­ties. Even when the cost of older trac­tors dou­bled, farm­ers rea­soned that they were still worth it be­cause re­pairs were sim­pler and down­time was min­i­mized. $60,000 for a 40-year-old ma­chine be­came the norm.

A judge’s ap­proval of the set­tle­ment is still re­quired, though it seems likely. Still, John Deere is­n’t out of the woods yet. It still faces an­other law­suit from the United States Federal Trade Commission, in which the gov­ern­ment or­ga­ni­za­tion ac­cuses Deere of harm­fully lock­ing down the re­pair process.

It’s dif­fi­cult to over­state the sig­nif­i­cance of this right-to-re­pair fight. While it has ob­vi­ous im­pli­ca­tions for the ag in­dus­try, oth­ers like the au­to­mo­tive and even home ap­pli­ance sec­tors are look­ing on. Any court rul­ing that might for­mally con­demn John Deere of wrong­do­ing may set a prece­dent for oth­ers to fol­low. At a time when man­u­fac­tur­ers want more and more con­trol of their prod­ucts af­ter the point of sale, every lit­tle up­date feels in­cred­i­bly high-stakes.

Got a tip or ques­tion for the au­thor? Contact them di­rectly: caleb@thedrive.com