Pigeons & Planes is all about mu­sic dis­cov­ery, sup­port­ing new artists, and de­liv­er­ing the best mu­sic cu­ra­tion on­line and IRL. We’re al­ways lis­ten­ing. Pigeons & Planes is all about mu­sic dis­cov­ery, sup­port­ing new artists, and de­liv­er­ing the best mu­sic cu­ra­tion on­line and IRL. We’re al­ways lis­ten­ing. Pigeons & Planes is all about mu­sic dis­cov­ery, sup­port­ing new artists, and de­liv­er­ing the best mu­sic cu­ra­tion on­line and IRL. We’re al­ways lis­ten­ing. Pigeons & Planes is all about mu­sic dis­cov­ery, sup­port­ing new artists, and de­liv­er­ing the best mu­sic cu­ra­tion on­line and IRL. We’re al­ways lis­ten­ing. Pigeons & Planes is all about mu­sic dis­cov­ery, sup­port­ing new artists, and de­liv­er­ing the best mu­sic cu­ra­tion on­line and IRL. We’re al­ways lis­ten­ing. Pigeons & Planes is all about mu­sic dis­cov­ery, sup­port­ing new artists, and de­liv­er­ing the best mu­sic cu­ra­tion on­line and IRL. We’re al­ways lis­ten­ing. Pigeons & Planes is all about mu­sic dis­cov­ery, sup­port­ing new artists, and de­liv­er­ing the best mu­sic cu­ra­tion on­line and IRL. We’re al­ways lis­ten­ing. Pigeons & Planes is all about mu­sic dis­cov­ery, sup­port­ing new artists, and de­liv­er­ing the best mu­sic cu­ra­tion on­line and IRL. We’re al­ways lis­ten­ing.

Grok Build is SpaceXAI’s ter­mi­nal-based AI cod­ing agent. It runs as a full-screen TUI that un­der­stands your code­base, ed­its files, ex­e­cutes shell com­mands, searches the web, and man­ages long-run­ning tasks — in­ter­ac­tively, headlessly for script­ing/​CI, or em­bed­ded in ed­i­tors via the Agent Client Protocol (ACP).

Installing the re­leased bi­nary · Building from source · Documentation · Repository lay­out · Development · Contributing · License

Learn more about Grok Build at x.ai/​cli

This repos­i­tory con­tains the Rust source for the grok CLI/TUI and its agent run­time. It is synced pe­ri­od­i­cally from the SpaceXAI monorepo.

Installing the re­leased bi­nary

Prebuilt bi­na­ries are pub­lished for ma­cOS, Linux, and Windows:

curl -fsSL https://​x.ai/​cli/​in­stall.sh | bash # ma­cOS / Linux / Git Bash irm https://​x.ai/​cli/​in­stall.ps1 | iex # Windows PowerShell grok –version

See the changelog for the lat­est fixes, fea­tures, and im­prove­ments in each re­lease.

Building from source

Requirements:

Rust — the tool­chain is pinned by rust-tool­chain.toml; rustup in­stalls it au­to­mat­i­cally on first build.

pro­toc — proto code­gen re­solves bin/​pro­toc (a dot­slash launcher) or falls back to a pro­toc on PATH / $PROTOC.

ma­cOS and Linux are sup­ported build hosts; Windows builds are best-ef­fort and not cur­rently tested from this tree.

cargo run -p xai-grok-pager-bin # build + launch the TUI cargo build -p xai-grok-pager-bin –release # re­lease bi­nary: tar­get/​re­lease/​xai-grok-pager cargo check -p xai-grok-pager-bin # fast val­i­da­tion

The bi­nary ar­ti­fact is named xai-grok-pager; of­fi­cial in­stalls ship it as grok. On first launch it opens your browser to au­then­ti­cate — see the au­then­ti­ca­tion guide.

Documentation

Full on­line doc­u­men­ta­tion is avail­able at docs.x.ai/​build/​overview.

The user guide ships with the pager crate: crates/​code­gen/​xai-grok-pager/​docs/​user-guide/ — get­ting started, key­board short­cuts, slash com­mands, con­fig­u­ra­tion, them­ing, MCP servers, skills, plu­g­ins, hooks, head­less mode, sand­box­ing, and more.

Repository lay­out

Important

The root Cargo.toml (workspace mem­bers, de­pen­dency ver­sions, lints, pro­files) is gen­er­ated — treat it as read-only. Prefer edit­ing per-crate Cargo.toml files.

Development

cargo check -p <crate> # al­ways tar­get spe­cific crates; full-work­space builds are slow cargo test -p xai-grok-con­fig # per-crate tests cargo clippy -p <crate> # lint con­fig: clippy.toml at the repo root cargo fmt –all # rustfmt.toml at the repo root

Contributing

License

First-party code in this repos­i­tory is li­censed un­der the Apache License, Version 2.0 — see LICENSE.

Third-party and ven­dored code re­mains un­der its orig­i­nal li­censes. See:

THIRD-PARTY-NOTICES — crates.io / git de­pen­den­cies, bun­dled UI themes, and in-tree source ports (including ope­nai/​codex and sst/​open­code tool im­ple­men­ta­tions)

crates/​code­gen/​xai-grok-tools/​THIRD_­PAR­TY_NO­TICES.md — crate-lo­cal no­tice for the codex and open­code ports (license texts + Apache §4(b) change no­tice)

third_­party/​NO­TICE — ven­dored Mermaid-stack in­dex

Ask any­thing, or task an agent…

Explore in­spi­ra­tion

from the poof-it’s-gone dept

In all of our dis­cus­sions about how the dig­i­tal rev­o­lu­tion has cre­ated a sys­tem in which peo­ple don’t ac­tu­ally own the things they think they’re buy­ing, I get par­tic­u­larly frus­trated by the lack of change in it all. We’ve spilled much ink com­plain­ing that this clearly anti-con­sumer prac­tice needs to be done away with, where an un­sus­pect­ing pub­lic thinks they’re buy­ing “a thing” only to learn months or years later that “the thing” they bought was ac­tu­ally a li­cense to use/​view/​lis­ten to an­other “thing”, and that li­cense ex­ists at the plea­sure of the com­pany that col­lected the money for it. And if you want to see the lack of change or ac­tion re­ally honed in upon, let’s take a look at Sony’s PlayStation Store.

In 2022, due to “evolving li­cens­ing agree­ments” with dis­trib­u­tor StudioCanal, German and Austrian users had hun­dreds of movies dis­ap­pear from their PS ac­counts, long af­ter buy­ing them through Sony. Then in 2023, it hap­pened again in America, specif­i­cally when Sony ended its li­cens­ing agree­ment with Discovery af­ter the Warner Bros. merger, which, of course, has since been bought by Paramount Skydance. That re­sulted in cus­tomers hav­ing hun­dreds and hun­dreds of episodes of TV shows deleted from their ac­counts. Nowhere in any of this were there re­funds, of course. No rec­om­pense at all, ac­tu­ally. Just a thing you thought you’d bought taken away from you by the very peo­ple you thought you bought it from.

And now it’s hap­pen­ing again. Due to an­other li­cens­ing agree­ment fall­out with StudioCanal, hun­dreds of movies and TV shows are be­ing ripped from the ac­counts of PS Store cus­tomers, and there ap­pears to be fuck all that they can do about it.

This news was brought to peo­ple’s at­ten­tion by X user so­matyk, who posted the no­ti­fi­ca­tion they had re­ceived from PlayStation this week. Along with the un­apolo­getic news that the pur­chased movies would be deleted from their ac­count on September 1, the mes­sage con­cluded with, “Click here for a full list of af­fected ti­tles that will no longer be sup­ported. Thank you.” The same warn­ing is now re­pro­duced in full on the PlayStation web­site, along with the list of 551 films and TV se­ries that are be­ing pulled from peo­ple’s li­braries.

This news was brought to peo­ple’s at­ten­tion by X user so­matyk, who posted the no­ti­fi­ca­tion they had re­ceived from PlayStation this week. Along with the un­apolo­getic news that the pur­chased movies would be deleted from their ac­count on September 1, the mes­sage con­cluded with, “Click here for a full list of af­fected ti­tles that will no longer be sup­ported. Thank you.” The same warn­ing is now re­pro­duced in full on the PlayStation web­site, along with the list of 551 films and TV se­ries that are be­ing pulled from peo­ple’s li­braries.

As Kotaku notes later in their post, part of what is strik­ing in all of this is the sheer mun­dan­ity of the an­nounce­ment. Because there have been no con­se­quences, or any ac­tion at all from the pub­lic or gov­ern­ment, Sony treats this all as if it’s per­fectly nor­mal and no big deal. You can tell me all you want about how the Ts and Cs in these pur­chases do in fact note that the na­ture of the pur­chase is a tem­po­rary li­cens­ing of the con­tent for an un­de­ter­mined time pe­riod… but I can promise you that the pub­lic in gen­eral does­n’t un­der­stand that. They think they’re buy­ing a thing, not a li­cense.

And that’s be­cause of the pur­pose­ful ob­fus­ca­tion of that fact. Sony damned well knows that the vast ma­jor­ity of peo­ple don’t read those Ts and Cs. It knows that the pub­lic largely does­n’t un­der­stand how these back­end li­cens­ing agree­ments with dis­trib­u­tors work, or that they even ex­ist. And Sony is­n’t ex­actly putting out a big blink­ing sign on its store pages in­form­ing the pub­lic of all of this. Instead, the com­pany is only too happy to col­lect money from a pub­lic that is be­ing pur­pose­fully kept ig­no­rant of what they’re buy­ing.

Of course, when you scroll past the end­less EULAs when you first use your PlayStation, and click “Agree” the first time you load the store, you’re un­wit­tingly agree­ing that noth­ing you buy is re­ally truly bought, and that it can be taken away from you at any point, and there’s noth­ing you can do. The same is true of your games.

Of course, when you scroll past the end­less EULAs when you first use your PlayStation, and click “Agree” the first time you load the store, you’re un­wit­tingly agree­ing that noth­ing you buy is re­ally truly bought, and that it can be taken away from you at any point, and there’s noth­ing you can do. The same is true of your games.

This, too, will prob­a­bly pass with­out any real ac­tion. The gov­ern­ment has done its best to gut our con­sumer pro­tec­tion agen­cies, so they won’t be any help. Angry cus­tomers won’t co­a­lesce into ac­tivism or ac­tion, most likely. And I’ll prob­a­bly be writ­ing an­other one of these posts in a cou­ple of years when it all hap­pens again.

But it should­n’t be that way. There are com­mon sense things that can be done to bet­ter in­form the pub­lic. Rules for how the store should in­form peo­ple with each and every pur­chase. Someone just needs to de­mand it be done.

Filed Under: eula, own­er­ship, playsta­tion, playsta­tion store, video games

Companies: sony, stu­dio­canal

Date: 2026 – 07-15 Git: https://​git­lab.com/​mort96/​blog/​blob/​pub­lished/​con­tent/​00000-home/​00017-sqlite-edi­tions.md

SQLite is an amaz­ing data­base en­gine. I use it as a data­base for plenty of em­bed­ded pro­jects, and I don’t think it’s an ex­ag­ger­a­tion to call it the in­dus­try stan­dard for lo­cal data stor­age. Some server soft­ware even uses it; for ex­am­ple, lob­ste.rs is now run­ning on SQLite.

Unlike tra­di­tional RDBMSes (Relational DataBase Management Systems), SQLite is not a sep­a­rate process; it’s an RDBMS as a li­brary, mean­ing your soft­ware re­mains self con­tained. Unlike tra­di­tional file for­mats, you don’t need to write cus­tom se­ri­al­iz­ers and parsers. In some ways, it’s the best of both worlds.

There’s just one huge prob­lem though. Its de­faults are all wrong.

Bad de­fault #1: Foreign key con­straints are ig­nored by de­fault

You read that right. Foreign key con­straints are ar­guably the pri­mary tool we have to en­sure that a data­base re­mains con­sis­tent and don’t have dan­gling ref­er­ences.

As a quick primer, this is how an SQL for­eign key con­straint looks:

CREATE TABLE users ( id INTEGER PRIMARY KEY, dis­play_­name TEXT );

CREATE TABLE posts ( id INTEGER PRIMARY KEY, user_id INTEGER NOT NULL, con­tent TEXT NOT NULL, FOREIGN KEY(user_id) REFERENCES users(id) );

The typ­i­cal be­hav­ior for all other RDBMSes would be that the user_id col­umn of a post must al­ways ref­er­ence the ID of a valid user. You can’t cre­ate a new post with­out pro­vid­ing a valid user ID, you can’t delete a user with­out also delet­ing its posts, lest you get a for­eign key con­straint vi­o­la­tion er­ror.

The only RDBMS I’m aware of which does­n’t en­force this by de­fault is SQLite.

This is made even worse by SQLite’s ten­dency to re-use ROWID. You see, in this ex­am­ple, those INTEGER PRIMARY KEY rows be­come aliases for the table’s ROWID, which is a unique in­te­ger ID as­signed to every row of a table in SQLite. The al­go­rithm for as­sign­ing ROWID is a bit com­pli­cated (more de­tails in the SQLite doc­u­men­ta­tion), but it re­sults in ID re-use in some cases. This means that a dan­gling ref­er­ence eas­ily re­sults in a ref­er­ence to the wrong row, which is even worse than a dan­gling ref­er­ence be­cause every­thing will seem fine. You don’t even get an er­ror dur­ing lookup.

Just look at this hy­po­thet­i­cal se­quence of op­er­a­tions in our toy data­base schema:

– Bob cre­ates a user ac­count INSERT INTO users (display_name) VALUES (‘Bob’); SELECT * FROM users; — id | dis­play_­name — 1 | Bob

– Bob posts an in­tro­duc­tion post INSERT INTO posts (user_id, con­tent) VALUES (1, ‘Hello, I am Bob’); SELECT u.dis­play_­name, p.con­tent FROM users as u, posts as p WHERE u.id = p.user_id; — dis­play_­name | con­tent — Bob | Hello, I am Bob

– Bob deletes his ac­count, — but we for­got to delete the posts. — SQLite does­n’t pro­duce an er­ror be­cause it ig­nores our for­eign key. DELETE FROM users WHERE id = 1;

– Alice cre­ates an ac­count. — Alice gets the same ID that Bob had due to the ROWID al­go­rithm. INSERT INTO users (display_name) VALUES (‘Alice’); SELECT * FROM users; — id | dis­play_­name — 1 | Alice

– Alice has now in­her­ited Bob’s old post! SELECT u.dis­play_­name, p.con­tent FROM users as u, posts as p WHERE u.id = p.user_id; — dis­play_­name | con­tent — Alice | Hello, I am Bob

The fix is to en­able for­eign_keys with a pragma:

PRAGMA for­eign_keys = ON;

If we had done this in the be­gin­ning, the buggy DELETE would have pro­duced an er­ror:

DELETE FROM users WHERE id = 1; — Runtime er­ror: FOREIGN KEY con­straint failed (19)

Bad de­fault #2: Columns can store the wrong data type

SQLite has a sim­ple type sys­tem: a value can be NULL, an INTEGER, a REAL (aka a dou­ble pre­ci­sion float), TEXT, or a BLOB (aka bi­nary data). Consequently, a col­umn can be de­fined to hold val­ues of any of those types.

However, a col­umn de­fined as an INTEGER col­umn is­n’t re­stricted to only in­te­gers; in­stead, SQLite con­sid­ers it to “use INTEGER affin­ity”. What this means is es­sen­tially:

If you try to in­sert a TEXT value, and it is a valid string rep­re­sen­ta­tion of an in­te­ger, it is con­verted to an in­te­ger and stored as such.

If you try to in­sert a TEXT value, and it is a valid string rep­re­sen­ta­tion of a real num­ber, it is con­verted to a real (aka dou­ble pre­ci­sion float) and stored as such.

Otherwise, the value is stored as-is.

Other affini­ties have dif­fer­ent but sim­pler rules:

Columns with BLOB affin­ity store val­ues as-is.

Columns with TEXT affin­ity store BLOB, TEXT and NULL val­ues as-is, but con­vert nu­meric val­ues to TEXT.

Columns with REAL affin­ity work like columns with INTEGER affin­ity ex­cept that in­te­ger val­ues are con­verted to REAL.

Here’s how this looks in prac­tice:

CREATE TABLE mu­sic ( id INTEGER PRIMARY KEY, name TEXT, du­ra­tion_sec INTEGER );

INSERT INTO mu­sic (name, du­ra­tion_sec) VALUES (‘Lost In Hollywood’, 321); INSERT INTO mu­sic (name, du­ra­tion_sec) VALUES (‘Comfortably Numb’, 382); INSERT INTO mu­sic (name, du­ra­tion_sec) VALUES (‘The Way of All Flesh’, ‘Way too long, I mean come on’); SELECT * FROM mu­sic; — id | name | du­ra­tion_sec — 1 | Lost In Hollywood | 321 — 2 | Comfortably Numb | 382 — 3 | The Way of All Flesh | Way too long, I mean come on

I don’t think I need to ex­plain why it’s a bad idea for a data­base to be so care­less about data val­i­da­tion. It would be one thing if SQLite was an ex­plic­itly dy­nam­i­cally typed doc­u­ment data­base, but it’s not. SQLite asks me through its syn­tax rules, “What type do you want to go into this col­umn”.

I once had to clean up a pro­ject where some code had ac­ci­den­tally been writ­ing the strings ‘1’ and ‘0’ to a col­umn which was in­tended to store booleans (1 and 0). That was not a fun de­bug­ging story.

Luckily, SQLite has the con­cept of strict ta­bles, which makes SQLite pro­duce a type er­ror when the wrong type is in­serted into a col­umn:

CREATE TABLE mu­sic ( id INTEGER PRIMARY KEY, name TEXT, du­ra­tion_sec INTEGER ) strict;

INSERT INTO mu­sic (name, du­ra­tion_sec) VALUES (‘The Way of All Flesh’, ‘Way too long, I mean come on’); — Runtime er­ror: can­not store TEXT value in INTEGER col­umn mu­sic.du­ra­tion_sec (19)

Unfortunately, there is no pragma to glob­ally make all ta­bles strict. So you have to re­mem­ber to add the strict tag to every table man­u­ally.

There’s a cou­ple of ar­gu­ments against strict ta­bles which I want to cover here.

The au­thors of SQLite have writ­ten about their pref­er­ence for “flexible typ­ing”. Personally, I find this a re­ally strange piece of writ­ing. It does­n’t pro­vide any ex­am­ples for why it could ever be use­ful to in­sert a BLOB into an INTEGER col­umn. All it does is il­lus­trate why it’s some­times use­ful to have a col­umn which can store val­ues of any type. Strict ta­bles have a so­lu­tion for that; it’s called the ANY data type. You can still cre­ate columns which ac­cept any value, you just have to be ex­plicit about it.

A much bet­ter ar­gu­ment is pro­vided by user ‘zie’ on lob­ste.rs. You see, strict ta­bles in SQLite don’t just en­force types. They also change the rules for how type spec­i­fiers are parsed.

Non-strict SQLite ta­bles use the fol­low­ing rules to de­ter­mine the type of a col­umn (from SQLite’s doc­u­men­ta­tion):

If the de­clared type con­tains the string “INT” then it is as­signed INTEGER affin­ity. If the de­clared type of the col­umn con­tains any of the strings “CHAR”, “CLOB”, or “TEXT” then that col­umn has TEXT affin­ity. Notice that the type VARCHAR con­tains the string “CHAR” and is thus as­signed TEXT affin­ity. If the de­clared type for a col­umn con­tains the string “BLOB” or if no type is spec­i­fied then the col­umn has affin­ity BLOB. If the de­clared type for a col­umn con­tains any of the strings “REAL”, “FLOA”, or “DOUB” then the col­umn has REAL affin­ity. Otherwise, the affin­ity is NUMERIC.

If the de­clared type con­tains the string “INT” then it is as­signed INTEGER affin­ity.

If the de­clared type of the col­umn con­tains any of the strings “CHAR”, “CLOB”, or “TEXT” then that col­umn has TEXT affin­ity. Notice that the type VARCHAR con­tains the string “CHAR” and is thus as­signed TEXT affin­ity.

If the de­clared type for a col­umn con­tains the string “BLOB” or if no type is spec­i­fied then the col­umn has affin­ity BLOB.

If the de­clared type for a col­umn con­tains any of the strings “REAL”, “FLOA”, or “DOUB” then the col­umn has REAL affin­ity.

Otherwise, the affin­ity is NUMERIC.

A con­se­quence of this rule, com­bined with SQLite’s loose typ­ing, is that you can give your columns type names such as DATETIME or KEY_VALUE_SET or COLOR, and have a data­base con­nec­tor/​wrap­per which au­to­mat­i­cally knows to se­ri­al­ize and de­se­ri­al­ize columns with cus­tom types. And if noth­ing else, those cus­tom type names serve as use­ful doc­u­men­ta­tion.

I have to ac­knowl­edge that just chang­ing the de­fault from non-strict ta­bles to strict ta­bles, with no fur­ther changes, would give up on this some­what nifty quirk. However, I think we would be much bet­ter served by cus­tom type aliases.

If we could write some­thing like:

CREATE TYPE KEY_VALUE_SET = TEXT;

and then use KEY_VALUE_SET as a type name in a strict table, I think every­one would be happy. I would prob­a­bly start us­ing such a fea­ture lib­er­ally to doc­u­ment the ex­pected pat­tern of data in my columns. In a real world schema, you in­evitably end up with TEXT columns which have to be parsed by ap­pli­ca­tion code.

As an aside to this aside, it would be neat if we could as­so­ci­ate CHECK con­straints with a cus­tom type.

Update: ‘masklinn’ on lob­ste.rs points out that the SQL 99 stan­dard al­ready spec­i­fies type aliases, called CREATE DOMAIN. This al­ready sup­ports con­straints as well. So re­ally, SQLite just needs to add sup­port for the stan­dard CREATE DOMAIN state­ment.

Update: ‘masklinn’ on lob­ste.rs points out that the SQL 99 stan­dard al­ready spec­i­fies type aliases, called CREATE DOMAIN. This al­ready sup­ports con­straints as well. So re­ally, SQLite just needs to add sup­port for the stan­dard CREATE DOMAIN state­ment.

Bad de­fault #3: SQLITE_BUSY er­rors with con­cur­rent writ­ers

SQLite al­lows mul­ti­ple con­cur­rent read­ers, but only one writer at a time. By de­fault, if you have two processes try­ing to ac­quire a write lock at the same time, one of them will im­me­di­ately re­ceive an SQLITE_BUSY er­ror.

This is not the be­hav­ior I ex­pect. I ex­pect SQLite to wait for the lock to get un­locked, up to some time­out. It’s do­ing disk IO af­ter all, so I al­ready struc­ture my code with the as­sump­tion that a write could po­ten­tially be slow.

The de­fault be­hav­ior has lead me to writ­ing real-world bugs, where sys­tems would some­times just crash. I’ve man­u­ally writ­ten retry loops to fix it.

The fix is to set busy_­time­out with a pragma:

PRAGMA busy_­time­out = 5000;

This makes SQLite try to ac­quire the lock for up to 5 sec­onds be­fore er­ror­ing with a SQLITE_BUSY er­ror.

I did­n’t learn about this set­ting un­til re­cently. It seems like such an ob­vi­ous de­fault that I’m as­ton­ished that it’s not.

Update: I should add a note here about why sup­port for con­cur­rent writ­ers is de­sir­able. During nor­mal op­er­a­tion, you’re usu­ally best served by struc­tur­ing your soft­ware such that all writes are done by a sin­gle process, ide­ally a sin­gle thread. Concurrent writ­ers will never be fast. But there are non-typ­i­cal sit­u­a­tions. Maybe you need to man­u­ally clean up a data­base in­ter­ac­tively us­ing the sqlite3 tool in­ter­ac­tively on the com­mand line. Maybe you have scripts for un­com­mon ad­min­is­tra­tive tasks which you haven’t had the need to write a front-end for. These are per­fectly le­git­i­mate and, I be­lieve, fairly com­mon use cases. I think it’s bad that with SQLite’s de­faults, this kind use has a chance to just crash the soft­ware by mak­ing it throw an un­ex­pected SQLITE_BUSY er­ror.

Update: I should add a note here about why sup­port for con­cur­rent writ­ers is de­sir­able.

During nor­mal op­er­a­tion, you’re usu­ally best served by struc­tur­ing your soft­ware such that all writes are done by a sin­gle process, ide­ally a sin­gle thread. Concurrent writ­ers will never be fast. But there are non-typ­i­cal sit­u­a­tions. Maybe you need to man­u­ally clean up a data­base in­ter­ac­tively us­ing the sqlite3 tool in­ter­ac­tively on the com­mand line. Maybe you have scripts for un­com­mon ad­min­is­tra­tive tasks which you haven’t had the need to write a front-end for. These are per­fectly le­git­i­mate and, I be­lieve, fairly com­mon use cases. I think it’s bad that with SQLite’s de­faults, this kind use has a chance to just crash the soft­ware by mak­ing it throw an un­ex­pected SQLITE_BUSY er­ror.

Bad de­fault 4: Performance

There’s a lot to say about per­for­mance tun­ing in SQLite. When cor­rectly con­fig­ured, it can be a truly fast RDBMS, with the abil­ity to fill roles we typ­i­cally re­serve for the big servers like PostgreSQL or MySQL.

But by de­fault, its per­for­mance is­n’t great. Smarter peo­ple than me have writ­ten much more on this, and I rec­om­mend Sylvain Kerkour’s Optimizing SQLite for servers if you’re in­ter­ested in this topic.

But the most sig­nif­i­cant bad de­fault is that SQLite’s Write-Ahead Log (WAL) is dis­abled by de­fault. It can be en­abled with:

PRAGMA jour­nal_­mode = WAL;

The WAL pro­vides a dra­matic write speed-up in most cir­cum­stances. Additionally, it lets us dras­ti­cally re­duce the amount of disk syncs with­out risk­ing data cor­rup­tion by chang­ing an­other set­ting:

PRAGMA syn­chro­nous = NORMAL;

See the SQLite doc­u­men­ta­tion on what ex­actly syn­chro­nous does.

The so­lu­tion: edi­tions?

The oft-cited rea­son for why these de­faults re­main, well, de­fault, is of course back­wards com­pat­i­bil­ity. Changing de­faults now would likely break lots of old soft­ware and make peo­ple afraid to up­grade SQLite in the fu­ture in case it breaks every­thing again, just like how I’m afraid to up­grade Python be­cause every “upgrade” breaks a bunch of soft­ware I use. It’s a laudi­ble and rare goal to try to not break your de­pen­dents.

However, I think the so­lu­tion is sim­ple: add one “super pragma” which changes all the bad de­faults. I pro­pose that the fol­low­ing:

PRAGMA edi­tion = 2026;

should be an alias for at least the fol­low­ing set of prag­mas:

PRAGMA for­eign_keys = ON; PRAGMA busy_­time­out = 5000; PRAGMA jour­nal_­mode = WAL; PRAGMA syn­chro­nous = NORMAL;

And also make strict mode the de­fault for ta­bles.

This should be a nice mid­dle ground which avoids break­ing back­wards com­pat­i­bil­ity, but lets the data­base en­gine move for­wards and not be bogged down by its own his­tory.

The edi­tion idea is stolen straight from Rust edi­tions. The ad­van­tage of a year-based edi­tion rather than some­thing like JavaScript’s “use strict”; is that as the years go by, the sen­si­ble de­faults may change. Maybe some­thing like Hctree’s WAL2 makes its way into the main branch, say, in the year 2034, so maybe PRAGMA edi­tion = 2034 will some day set PRAGMA jour­nal_­mode = WAL2.

Anyway, that’s all. I think SQLite should have an edi­tion sys­tem with up­dated sets of de­faults. Are there any things I’ve missed which makes this a bad idea? Or more prag­mas which should be added to my imag­i­nary “2026 edi­tion”?

Counting each YC chap­ter by batch year shows when these founders came through. A founder with two YC star­tups ap­pears in both years.

I like my new-to-me MacBook be­cause it en­ables me to cre­ate more stuff com­pared to my now al­most 8 year old Thinkpad I had be­fore, mainly due to bat­tery and screen rea­sons. There is one thing about it that I can­not wrap my head around from a de­sign per­spec­tive and it’s the sharp edges es­pe­cially around the wrist area. On a flat sur­face it’s no prob­lem at all, but low and be­hold a lap­top will be fre­quently used on a lap, mean­ing the wrists will touch the sharp edge at an an­gle which is very un­com­fort­able.

I’ve found some posts on­line about peo­ple that have filed the edges off of their Macs be­fore, like this nice one by Kent Walters. In my case how­ever, al­most no one freaks out about this be­cause no one ac­tu­ally no­tices it. It went well, so I thought why not doc­u­ment the process.

Note I am ab­solutely not a good crafts­man, so do not use this as a guide!

I thought a long time about how to do it so as not to have it re­sult in a wavy bezel. Although I re­ally wanted to use this pro­ject as an ex­cuse to buy a ran­dom or­bital sander, I very quickly piv­oted from this idea be­cause I would prob­a­bly cause more dam­age with it than I would with a hand file. Another idea was to 3D print some sort of cham­fer to use as a guide, just to have the ini­tial part cleanly taken off at an an­gle. I played around with this idea for a while but con­cluded that I could prob­a­bly not get to the re­quired level of pre­ci­sion I wanted, es­pe­cially since it would have to hold the file/​sand­pa­per. I ended up just us­ing gen­eral pur­pose metal file I had at hand and pro­gres­sive sand pa­per (in block form, strongly rec­om­mended).

Some tape helped me to mark the ar­eas from be­low and above the bezel that I wanted to have file off. This worked re­ally well. Of course I taped off the track­pad, the key­board and head­phone plug etc. to avoid any residue from get­ting in­side. Additionally, I used very lit­tle soapy wa­ter that I ap­plied to the sand­pa­per through­out to keep the dust con­tained some­what. The ini­tial fil­ing with the gen­eral metal file was a bit scary, but it did not take off too much and the tape helped me to achieve an even level. The sand­pa­per blocks also helped with that, and I went up to 1200 coarse­ness.

The tricky part for me were these pointy dots on the lit­tle gap in the mid­dle. I did not want to go too heavy here with the fil­ing so I used some model mak­ing files very, very lightly and then just used the 1200 sand­pa­per un­til I was sat­is­fied with the re­sult. In the end, I was care­ful to re­move any dust that had ac­cu­mu­late, for which this lit­tle air blower sque­egy thing came in handy.

This is a “blue” M4 MacBook Air, so I’m cu­ri­ous how it will look over time due to the an­odized alu­minum. Another note would be to test it out a bit and see how it feels. I have quite large arms so when typ­ing I also touch the cor­ners, so I had to go back and file them too, but that might not be nec­es­sary de­pend­ing on how you do it.

What I liked about the Thinkpad was that I could just chuck it any­where with­out wor­ry­ing about it. I made it a point when I bought this Mac to treat it just the same way. Ultimately, it is a tool and should be used ap­pro­pri­ately. Even if it is nice and shiny, if mod­i­fy­ing it would make it serve it’s pri­mary pur­pose as a tool bet­ter, it’s worth a con­sid­er­a­tion. So I like this re­cent wave of en­cour­age­ment around sim­i­lar mod­i­fi­ca­tions, and it is also far eas­ier and ap­proach­able than it seems ini­tially.

SpaceX stock dropped be­low its ini­tial pub­lic of­fer­ing price for the first time on Wednesday, sig­nal­ing dwin­dling hype around the Elon Musk com­pany.

Shares dipped be­low their IPO price of $135 on Wednesday morn­ing for the first time since list­ing, a hum­bling loss for the stock, which had sky­rock­eted more than 50% in its first days of trad­ing last month.

The shares re­gained some ground later in the day, clos­ing at $135.27.

The ini­tial of­fer­ing gave the com­pany a mar­ket cap of $2.2 tril­lion, mak­ing it one of the world’s most valu­able pub­lic com­pa­nies. For a short pe­riod, the IPO also made Musk the world’s first tril­lion­aire, though his net worth now is about $800 bil­lion.

On July 7, the com­pany was added to the Nasdaq-100 af­ter a rule change al­lowed com­pa­nies to join 15 days af­ter their IPOs.

SpaceX raised a to­tal of $86 bil­lion af­ter un­der­writ­ers ex­er­cised their right to sell ad­di­tional shares, on top of the $75 bil­lion ini­tially raised. It was the largest IPO in his­tory.

SpaceX, based near Austin, Texas, is the lead­ing launch ser­vices com­pany in the world, with its Falcon 9 rocket ac­count­ing for the vast ma­jor­ity of satel­lites launched last year.

It is also the lead­ing satel­lite-based broad­band provider with its Starlink ser­vice. The ex­tra­or­di­nary in­ter­est in the IPO was dri­ven by Musk’s plans to make the com­pany an AI leader — in­clud­ing plans to launch or­bit­ing satel­lite data cen­ters pow­ered by the sun that crunch AI data.

The com­pa­ny’s head­quar­ters moved from Hawthorne to Texas in 2024, but it re­tains large op­er­a­tions in the South Bay city and blasts off reg­u­larly from Vandenberg Space Force Base in Santa Barbara County.

Since the IPO, SpaceX has used its new­found wealth to ex­pand in the AI space.

It an­nounced last month that it was ac­quir­ing the AI cod­ing startup Cursor for $60 bil­lion, with the deal ex­pected to close in the third quar­ter. The San Francisco com­pany, founded in 2022, en­ables en­gi­neers to in­struct soft­ware in English to run cod­ing tasks au­tonomously.

Musk also merged his xAI ar­ti­fi­cial in­tel­li­gence com­pany into SpaceX ear­lier this year. The com­bined en­tity re­cently an­nounced it was leas­ing com­put­ing power to ri­vals Anthropic and Google at two ter­res­trial data cen­ters it has con­structed.

Since the IPO, in­vestors have ex­pressed con­cerns about the com­pa­ny’s spend­ing plans and debt load.

Even with the volatil­ity of the last month, there’s still more un­cer­tainty to come.

The stock could fall fur­ther as locked-up shares held by cur­rent and for­mer em­ploy­ees are re­leased.

At least 20% of the shares will be re­leased af­ter sec­ond-quar­ter re­sults are dis­closed some­time in the com­ing months, with all the lock­ups ex­pir­ing in December.

But Space X is­n’t the only mega­cap stock to ex­pe­ri­ence ups and downs early on.

Shares of Meta, then named Facebook, fell sig­nif­i­cantly be­low the IPO price of $38 be­fore re­cov­er­ing. After its May 2012 launch, shares plum­meted by nearly 50% and hit a record low of $19.69 in August 2012.

The com­pany took more than 14 months to re­bound, fi­nally sur­pass­ing its $38 IPO price in July 2013.

More to Read

For the past year and a half, the team build­ing Roc’s com­piler has been rewrit­ing our 300,000 lines of Rust code into Zig, for rea­sons I’ll re­cap be­low. We re­cently passed an ex­cit­ing mile­stone: fea­ture par­ity with the orig­i­nal com­piler!

Since the Bun pro­ject re­cently shared an ex­pe­ri­ence re­port of their rewrite in the other di­rec­tion (from Zig to Rust, al­though that’s only the tip of the ice­berg of dif­fer­ences be­tween our rewrites), this seems like a nice time to re­flect on how our move from Rust to Zig is go­ing.

Passing Feature Parity

Hitting this mile­stone made it pos­si­ble to up­date Brendan Hansknecht’s charm­ing 2024 WASM-4 game, Rocci Bird (with art by Luke DeVault) to use the new com­piler. It’s a nice ex­am­ple be­cause the whole game is un­der a thou­sand lines of Roc code, and you can play it on itch.io or right here via WebAssembly:

Click or tap the game, then press Space (or tap) to flap. On mo­bile you don’t have a right ar­row key, so re­fresh the page to restart the game.

Rocci Bird’s up­dated source code is a bit more con­cise than the orig­i­nal, and roc build –opt=size now out­puts a 31KB wasm bi­nary. (The orig­i­nal com­piler pro­duced a bi­nary more than dou­ble that size.) Rocci Bird is by no means a large code base, but get­ting it to run at all re­quired land­ing a lot of fea­tures in the new com­piler. Seeing those chunky pur­ple pix­els brought a smile to my face when we fi­nally got there!

To be clear, this is a mile­stone but not a for­mal re­lease. (We aim to land ver­sion 0.1.0 later this year.) That said, it’s a won­der­ful mile­stone to have reached, and I’m ex­tremely grate­ful to all the peo­ple who came to­gether to make this hap­pen! I want to thank some in par­tic­u­lar who have been es­pe­cially help­ful in get­ting the lan­guage and com­piler to this point:

Anthony Bullard and Sam Mohr for col­lab­o­rat­ing on the new parser

Jared Ramirez for the new type-checker (among many other things!)

Ayaz Hafiz for the new lambda set res­o­lu­tion sys­tem, plus tons of the orig­i­nal com­piler

Aurélien Geron for hand-up­dat­ing 108 (!) be­gin­ner ex­er­cises in the Roc Exercism course he orig­i­nally cre­ated

Stephan for get­ting the com­pil­er’s new “echo” plat­form run­ning in the browser, so that any­one can now write and run ba­sic Roc pro­grams from the roc-lang.org home­page via a 2.5MB WebAssembly bi­nary!

Niclas Åhdén, Roc’s most pro­lific pro­duc­tion user, for pa­tiently fil­ing help­ful bug re­ports and giv­ing ac­tion­able feed­back about the up­grade process

JRI98 for me­thod­i­cally re­pro­duc­ing and in­ves­ti­gat­ing fuzzer er­rors and other bugs, clos­ing out is­sues that no longer re­pro­duced, and more

Jasper Woudenberg for it­er­at­ing on API de­signs for user­space pack­ages us­ing the new com­piler

Folkert de Vries, Brendan Hansknecht, Brian Carroll, Josh Warner, Agus Zubiaga, and Jelle Teeuwissen for build­ing the foun­da­tion of the orig­i­nal com­piler, with­out which the new com­piler never would have ex­isted

I’ve saved the undis­puted biggest con­trib­u­tors to the new com­piler for last: Anton-4 and Luke Boswell for so many things I can’t even keep track of them all—com­piler work, builtins, plat­forms, pack­ages, ex­am­ples, fix­ing bugs, help­ing be­gin­ners on Roc Zulip…enumerating it all could take up a whole sec­ond post! It’s been in­cred­i­ble see­ing how much you’ve built.

Thank you all so much! I feel hon­ored that you’ve put so much of your valu­able time into this pro­ject. Also thanks to our past and pre­sent spon­sors—rwx, Lambda Class, ohne-mak­ler, mar­t­ian, tweede golf, Vendr, NoRedInk, and many gen­er­ous in­di­vid­ual spon­sors—who have helped get us to this point by sup­port­ing our con­trib­u­tors.

Speaking of time: our 487-day rewrite took 476 days longer than Bun’s 11-day rewrite from their ~500K lines of Zig into Rust. There are many rea­sons for this dif­fer­ence which have noth­ing to do with Rust or Zig, in­clud­ing the fact that theirs was a di­rect port whereas we’d de­cided to rewrite be­cause of how much we were go­ing to change. The tech­niques they used would­n’t have worked in our case.

The laun­dry list of changes we made also means com­par­ing our orig­i­nal Rust code base and new Zig code base won’t be ap­ples-to-ap­ples. Still, we’ve reached a nice point to re­flect on how the rewrite has gone, both in terms of what new fea­tures it has un­locked for Roc pro­gram­mers, as well as how our ex­pe­ri­ences with Rust and Zig have com­pared.

Let’s get into it!

Hot Code Loading + Cross-Compiled Binaries

Roc’s new com­piler au­to­mat­i­cally does hot code load­ing dur­ing de­vel­op­ment. For ex­am­ple, I can run roc server.roc to start a Web server, then change some of its code while it’s run­ning. The next time that server han­dles a re­quest, it’ll au­to­mat­i­cally be han­dled us­ing the new code. Here it is in ac­tion, both in a server and in a sim­ple 2D game:

Hot load­ing is stan­dard be­hav­ior for in­ter­preted lan­guages like Python, but not so much for high-per­for­mance com­piled lan­guages like Roc. When I’m ready to de­ploy, roc build server.roc gets me an LLVM-optimized, self-con­tained bi­nary that I can drop onto a ma­chine and run.

Roc also cross-com­piles; build­ing a sta­tic bi­nary that runs on Alpine Linux is as sim­ple as roc build –target=x64musl, and that com­mand will pro­duce the same out­put bytes (for the same in­put source code bytes) when run on a Mac or any other sys­tem—which not all com­pil­ers guar­an­tee.

Pattern Matching with String Interpolation

The HTTP re­quest-han­dling logic from that video looks like this:

match (verb, path) { (“GET”, “/users/${id}/${page}“) => match page { “” | “profile” => ok(id) “settings” => ok(with­_de­fault(user_a­gent, id)) “posts/${post_id}” => ok(“Post ID: ${post_id}“) _ => not_­found }

(“GET”, “/users/${id}“) => ok(id)

(“POST”, “/posts/new”) => cre­ated(with­_de­fault(…))

_ => not_­found }

This uses sev­eral fea­tures we in­tro­duced in the new com­piler. For ex­am­ple, that “/users/${id}” syn­tax is not im­ple­mented with pars­ing tem­plate strings at run­time, but rather with a new lan­guage fea­ture: string in­ter­po­la­tion in­side pat­tern match­ing.

Not only is this type-safe at com­pile time, this en­tire code snip­pet per­forms zero heap al­lo­ca­tions. I’d ex­pect the typ­i­cal lan­guage that ships with hot code load­ing to av­er­age closer to 1 al­lo­ca­tion per line of code here…but Roc is aim­ing high on er­gonom­ics, type safety, and per­for­mance!

You can play around with this syn­tax on the new roc-lang.org home­page - if you scroll down a bit, there’s an WebAssembly build of the com­piler right there on the page that you can use to try out the lan­guage.

By the way, if you’re in­ter­ested in a post on the tech­ni­cal de­tails of how we used the new com­pil­er’s com­pile-time ex­e­cu­tion of pure func­tions to get HTTP re­quest rout­ing down to zero al­lo­ca­tions, let me know on Roc Zulip.

By the way, if you’re in­ter­ested in a post on the tech­ni­cal de­tails of how we used the new com­pil­er’s com­pile-time ex­e­cu­tion of pure func­tions to get HTTP re­quest rout­ing down to zero al­lo­ca­tions, let me know on Roc Zulip.

Why a Scratch-Rewrite?

Unlike Rust, C, and Zig, Roc is not a sys­tems lan­guage; it has au­to­matic mem­ory man­age­ment (using ref­er­ence count­ing, both to avoid trac­ing col­lec­tor pauses and also for Perceus op­ti­miza­tions and op­por­tunis­tic mu­ta­tion like Koka’s). Roc would have way more heap al­lo­ca­tions if it needed one heap al­lo­ca­tion per clo­sure cap­ture (like most non-sys­tems lan­guages do), but our clo­sure cap­tures don’t heap-al­lo­cate be­cause Roc is the first non-aca­d­e­mic lan­guage to im­ple­ment poly­mor­phic de­func­tion­al­iza­tion through lambda set spe­cial­iza­tion.

This might sound like a niche op­ti­miza­tion, but in a func­tional lan­guage like Roc, de­func­tion­al­iza­tion turns out to be sim­i­lar to in­lin­ing in that it un­locks a trea­sure trove of fol­low-up op­ti­miza­tions. Although this sys­tem proved in­cred­i­bly ben­e­fi­cial to Roc’s run­time per­for­mance, it also proved in­cred­i­bly dif­fi­cult for us to im­ple­ment cor­rectly. We strug­gled with nasty bugs in the orig­i­nal im­ple­men­ta­tion, and only af­ter Ayaz Hafiz pro­to­typed a new ar­chi­tec­ture in OCaml were we able to fi­nally get it right in the new com­piler.

Ayaz’s pro­to­type showed that the root of our prob­lems was ar­chi­tec­tural across sev­eral com­piler phases, and fix­ing it would re­quire rewrit­ing most of the com­piler. This was one rea­son we de­cided to rewrite in the first place—that, and sev­eral con­trib­u­tors in­de­pen­dently men­tion­ing they planned to rewrite var­i­ous parts of the com­piler for other rea­sons. We re­al­ized we were about to rewrite al­most all of the com­piler any­way, so it made sense to con­sider a full rewrite as an al­ter­na­tive to the Ship of Theseus ap­proach.

Compilers are un­usual in that scratch-rewrites are the norm among suc­cess­ful pro­jects. It’s of­ten the only way to self-host, al­though not all com­pil­ers rewrite into their own lan­guage; see for ex­am­ple TypeScript’s rewrite to Go. My po­si­tion has al­ways been that Roc’s com­piler should not self-host, so the idea that some­day the ben­e­fits of a rewrite might seem to out­weigh their no­to­ri­ous costs had frankly never oc­curred to me.

The more we talked about it, the more sense it made to do what ba­si­cally every main­stream com­piler to­day has done at some point: rewrite from scratch.

Why Zig?

Once we’d de­cided to scratch-rewrite, the next ques­tion was whether to choose Rust again. Based on our ex­pe­ri­ences with both Rust and Zig (we were al­ready us­ing Zig for a bunch of prim­i­tives in our stan­dard li­brary), we de­cided to build the en­tire com­piler in Zig this time.

I en­joy Rust, I’ve taught a course on it, and I hap­pily use it daily for my work at Zed. Despite what Internet com­ments might have us be­lieve, it’s ex­tremely nor­mal for one lan­guage to be the best fit for one pro­ject, while a dif­fer­ent lan­guage turns out to be the best fit for a dif­fer­ent pro­ject. One size does not ac­tu­ally fit all!

I’ve talked in depth about our rea­sons for go­ing with Zig else­where—in writ­ing, on pod­casts, and so on—and we only se­ri­ously con­sid­ered Rust and Zig, be­cause those were the only sys­tems lan­guages our team knew well enough. The biggest con­sid­er­a­tions on our minds when de­cid­ing be­tween Rust and Zig were:

Build times. Our cargo build times were a ma­jor pain point, even for in­cre­men­tal builds, and get­ting worse as our code base grew. We ex­pected build times in a Zig rewrite to be much faster.

Memory con­trol. We use a va­ri­ety of dif­fer­ent mem­ory al­lo­ca­tors through­out com­pi­la­tion, es­pe­cially are­nas, and struct-of-ar­rays lay­outs all over the place. Rust’s ecosys­tem con­sis­tently as­sumes one global al­lo­ca­tor, in­clud­ing soa_rs. Zig’s whole ecosys­tem as­sumes gran­u­lar al­lo­ca­tors, and struct-of-ar­rays sup­port is stan­dard.

Ecosystem rel­e­vance. Rust’s ecosys­tem is much big­ger than Zig’s over­all…but al­most no pack­ages in ei­ther ecosys­tem are rel­e­vant to our par­tic­u­lar needs. For the niche things we wanted to get off the shelf—such as a faster way to emit LLVM bit­code than wrap­ping LLVM’s C++ li­brary—more of that code ex­isted in Zig than in Rust.

Memory-unsafety as­sis­tance. Rust is de­signed to iso­late mem­ory-un­safe code in­side rare un­safe blocks, and use things like miri or Valgrind to vet those. Memory-unsafe code was­n’t rare for us, though (more on this later) and we ended up with about 1,200 uses of un­safe (out of our 300K lines of Rust code; com­pare to about 40,000 uses of un­safe in rust’s 3.5M lines, and re­mem­ber that for com­pil­ers which emit ma­chine code, like roc and rustc, do­ing mem­ory-un­safe things is a big part of the job). Zig has more fea­tures than Rust for mak­ing mem­ory-un­safe code work cor­rectly, and that was the area where we wanted the most help.

After a year and a half of rewrit­ing, how did our ex­pec­ta­tions of Zig’s ben­e­fits line up with the re­al­ity of what we got? And which parts of Rust did we end up miss­ing once we no longer had ac­cess to them?

Life Without Borrow-Checking

Let’s start with mem­ory safety. There’s a fa­mous 2019 Microsoft pre­sen­ta­tion that says, on slide 10:

~70% of the vul­ner­a­bil­i­ties ad­dressed through a se­cu­rity up­date each year con­tinue to be mem­ory safety is­sues.

~70% of the vul­ner­a­bil­i­ties ad­dressed through a se­cu­rity up­date each year con­tinue to be mem­ory safety is­sues.

The pre­sen­ta­tion’s next slide has a break­down by type of mem­ory safety is­sue, which paints the fol­low­ing pic­ture when it comes to Rust and Zig specif­i­cally:

83.6% of vul­ner­a­bil­i­ties ad­dressed through a se­cu­rity up­date in 2018 would have been com­pletely un­af­fected by the choice of Rust or Zig, be­cause both lan­guages han­dle all of these sce­nar­ios (out-of-bounds reads/​write, un­safe casts, unini­tial­ized reads, stack over­flows, and non-mem­ory-safety is­sues) in the same way.

16.4% of the vul­ner­a­bil­i­ties were specif­i­cally use-af­ter-free er­rors. These could have been caught by Zig’s ReleaseSafe run­time mem­ory-safety checks, or Rust’s bor­row checker, or the checks Fil-C uses…mod­ern lan­guages have a va­ri­ety of ways to help catch UAFs, al­though these CVEs from 2018 would have al­most cer­tainly been from C or C++ code in­stead.

ReleaseSafe catches use-af­ter-free er­rors through run­time checks which panic if the pro­gram tries to use freed mem­ory. Compared to Rust’s safe sub­set, Zig’s checks are less com­pre­hen­sive, have a run­time cost, and can panic. That said, Zig with ReleaseSafe has worked great in prac­tice for the TigerBeetle data­base, which re­cently un­der­went a leg­en­dar­ily metic­u­lous Jepsen re­port that found only two safety bugs, nei­ther re­lated to mem­ory safety.

ReleaseFast skips these checks in pro­duc­tion builds to avoid their over­head, but keeps them in de­bug builds and tests to catch mem­ory-safety is­sues dur­ing de­vel­op­ment. If your tests cov­ered every pos­si­ble real-world code path, ReleaseFast would give you the same safety as ReleaseSafe, but that level of test cov­er­age is rarely prac­ti­cal; the real ques­tion is what slips through the cov­er­age cracks in prac­tice. Bun talked about their strug­gles with use-af­ter-frees, but other widely-used pro­jects build­ing with ReleaseFast have had no CVEs caused by mem­ory un­safety in their Zig code. Ghostty is one, and Zig’s com­piler it­self is an­other.

If you want to learn more about these pro­jects, I’ve recorded in-depth con­ver­sa­tions with their cre­ators: Joran Greef on TigerBeetle, Mitchell Hashimoto on Ghostty, and Andrew Kelley on Zig.

If you want to learn more about these pro­jects, I’ve recorded in-depth con­ver­sa­tions with their cre­ators: Joran Greef on TigerBeetle, Mitchell Hashimoto on Ghostty, and Andrew Kelley on Zig.

Rust code has a dif­fer­ent source of mem­ory-safety gaps: the un­safe sec­tions that nearly every Rust pro­gram has some­where in its de­pen­den­cies. Unsafe Rust has all the mem­ory un­safety risk of ReleaseFast Zig code, but none of the run­time checks to catch is­sues dur­ing de­vel­op­ment. The Rust ecosyt­sem has miri to find bugs in non-FFI un­safe code, and Valgrind can help too, but few Rust pro­jects use ei­ther. That said, the cul­tural norm of us­ing un­safe rarely, and au­dit­ing it ex­tra care­fully, has worked out well enough to earn Rust a strong rep­u­ta­tion for mem­ory safety in prac­tice.

Of course, Rust mem­ory un­safety er­rors can and do still slip through the cracks. Deno, a Bun com­peti­tor which is writ­ten in Rust, has had mem­ory-un­safety CVEs in­clud­ing an out-of-bounds read as well as a use-af­ter-free, both in­volv­ing the use of Unsafe Rust. Rocket, a Rust Web Framework, has had a use-af­ter-free CVE, and Actix has had a va­ri­ety of mem­ory-un­safety CVEs from a pe­riod when its use of un­safe was ab­nor­mally high.

When we were de­cid­ing be­tween Rust and Zig for the new com­piler, we were aware of all of this. We knew Rust had a well-de­served rep­u­ta­tion for mem­ory safety, but that mem­ory un­safety could still hap­pen, and we’d ex­pe­ri­enced all of that first­hand with the orig­i­nal com­piler. We also knew we’d be us­ing un­safe way more than typ­i­cal Rust pro­jects, and even though we were al­ready us­ing Valgrind, get­ting help with in­nately mem­ory-un­safe code from Zig’s ad­di­tional checks sounded ap­peal­ing. We wanted the hard stuff to get eas­ier, and we weren’t wor­ried about use-af­ter-free is­sues in a com­piler where al­lo­ca­tions would be over­whelm­ingly done in are­nas with straight­for­ward life­times.

We knew high-pro­file Zig pro­jects had achieved great per­for­mance and mem­ory safety in prac­tice, and we de­cided to aim for be­com­ing an­other of those suc­cess sto­ries.

Memory Safety Post-Rewrite

It’s easy to the­o­rize about how things will go with a par­tic­u­lar tech­nol­ogy choice, but where the rub­ber meets the road is what end users en­counter in real-world us­age. So how has Zig with ReleaseFast worked out for us in prac­tice? How many mem­ory cor­rup­tion in­ci­dents—from use-af­ter-frees or any other cause—have we seen since rewrit­ing our com­piler from Rust to Zig?

Here’s a break­down of bug re­ports in Roc’s is­sue tracker, as clas­si­fied by Claude Opus 4.8:

You might be won­der­ing how the Rust-based com­piler had any mem­ory cor­rup­tion bugs at all, let alone more than dou­ble the to­tal count of the Zig-based one. Is it be­cause of that pesky Unsafe Rust again?

Actually, no. None of those 21 mem­ory cor­rup­tion bugs oc­curred in the com­pil­er’s logic it­self, which is a tes­ta­ment to Rust’s bor­row-checker work­ing as in­tended. The rea­son we had mem­ory cor­rup­tion bugs in our Rust-based com­piler is that it’s a com­piler.

Compilers emit ma­chine in­struc­tions. When a ma­chine ex­e­cutes those in­struc­tions, they can cause mem­ory cor­rup­tion, re­sult­ing in mem­ory cor­rup­tion bug re­ports from the peo­ple who ex­pe­ri­enced them. Regardless of which process had the bug—the com­piler or com­piled pro­gram—in both cases the proces­sor only did the bad thing be­cause the com­piler told it to. And in both cases the fix is the same: the com­pil­er’s code must change, since that code was what caused the mem­ory cor­rup­tion.

Just like every com­piler, Roc’s has had bugs, and some of those have been mis­com­pi­la­tions that led to mem­ory cor­rup­tion. That said, while 8 of the 10 mem­ory cor­rup­tion bugs in the Zig-based com­piler were also mis­com­pi­la­tions, the re­main­ing 2 were in the com­piler it­self. Both were use-af­ter-free bugs in er­ror re­port­ing, with the same symp­tom: file­names in er­ror mes­sages (one in roc check and the other in roc bun­dle) ren­dered as use­less ques­tion-mark-in-di­a­mond char­ac­ters. Rust’s bor­row checker would have caught both.

Now let’s sup­pose we had in­stead cho­sen Rust for our rewrite, or Zig with ReleaseSafe. What would have been the im­pact in prac­tice, hold­ing all else equal?

After 18 months of de­vel­op­ment, hun­dreds of to­tal bug re­ports, and hun­dreds of thou­sands of lines of code, my main take­away from ret­ro­spect­ing on this table is that pick­ing a dif­fer­ent row would have made no ap­pre­cia­ble dif­fer­ence to the pro­ject. So far our choice has got­ten us the out­come we’d hoped for.

As I noted ear­lier, every pro­ject has dif­fer­ent needs. When Bun rewrote in the op­po­site di­rec­tion—from Zig to Rust—their ac­com­pa­ny­ing post noted:

For Bun, cor­rectly han­dling the life­times of garbage-col­lected val­ues [from JavaScript] and man­u­ally-man­aged val­ues has been a ma­jor source of sta­bil­ity is­sues - most of­ten small mem­ory leaks and oc­ca­sion­ally, crashes. Every mem­ory al­lo­ca­tion has to be metic­u­lously re­viewed. Where do these bytes get freed? How do we en­sure it only gets freed once? Did we check for JavaScript ex­cep­tions prop­erly? Is this garbage-col­lected pointer vis­i­ble to the con­ser­v­a­tive stack scan­ner? Is this garbage col­lected mem­ory or man­u­ally man­aged mem­ory?

For Bun, cor­rectly han­dling the life­times of garbage-col­lected val­ues [from JavaScript] and man­u­ally-man­aged val­ues has been a ma­jor source of sta­bil­ity is­sues - most of­ten small mem­ory leaks and oc­ca­sion­ally, crashes. Every mem­ory al­lo­ca­tion has to be metic­u­lously re­viewed. Where do these bytes get freed? How do we en­sure it only gets freed once? Did we check for JavaScript ex­cep­tions prop­erly? Is this garbage-col­lected pointer vis­i­ble to the con­ser­v­a­tive stack scan­ner? Is this garbage col­lected mem­ory or man­u­ally man­aged mem­ory?

Roc’s com­piler does­n’t have these par­tic­u­lar chal­lenges be­cause it does­n’t in­ter­face with JavaScript or any other trac­ing garbage col­lec­tor. For Bun, “use-after-free, dou­ble-free, and ‘forgot to free’” er­rors have been “a large per­cent­age of bugs,” whereas er­rors like these have been a small per­cent­age of Roc’s bugs. And of course Roc’s com­piler faces other chal­lenges that Bun does­n’t. Different pro­jects have dif­fer­ent needs!

In our case, I’m not sure how I could look back at what’s ac­tu­ally hap­pened and con­clude that what we needed was a big­ger in­vest­ment in tool­ing to pre­vent mem­ory safety bugs in the com­piler it­self. There’s a much stronger case that we would ben­e­fit from bet­ter tool­ing to catch mem­ory safety bugs in our com­piled out­put, which has al­ways been out of scope for the bor­row checker.

Build Times

We wanted faster builds from Zig. Did we get them?

Well, the good news is that zig build –watch -fincremental can re­build a change to our cur­rent ~450K lines of Zig code in about 35 mil­lisec­onds. That’s even faster than what we were hop­ing for when we con­sid­ered Zig’s build speed a sell­ing point for the rewrite!

The bad news is that Zig’s cur­rent sta­ble 0.16.0 re­lease has a bug that breaks -fincremental on our code base. The fix al­ready landed, but to get it we’d have to build on a nightly 0.17.0 pre­re­lease build (which has break­ing lan­guage changes), along with ven­dor­ing and up­grad­ing our af­fected de­pen­den­cies to 0.17.0. We de­cided to wait for the next sta­ble re­lease in­stead.

As of the last com­mit that had Rust sources in our code base, here’s a tim­ing com­par­i­son on my Intel desk­top ma­chine run­ning Ubuntu 26 for build­ing cold (no cache, but pack­ages down­loaded lo­cally) com­pared to do­ing an in­cre­men­tal re­build af­ter mak­ing a triv­ial edit to our parser:

Note that our Zig build con­fig­u­ra­tion as of the fea­ture-par­ity com­mit was re­build­ing rarely-chang­ing ar­ti­facts on every build that we later de­cided to re­build only on de­mand. That’s why to­day’s cold builds are faster than they were back at 300K LoC, even though our lines of code have in­creased by ~50% since then.

Note that our Zig build con­fig­u­ra­tion as of the fea­ture-par­ity com­mit was re­build­ing rarely-chang­ing ar­ti­facts on every build that we later de­cided to re­build only on de­mand. That’s why to­day’s cold builds are faster than they were back at 300K LoC, even though our lines of code have in­creased by ~50% since then.

Rust 1.97 is the cur­rent sta­ble re­lease to­day, and 1.85 was the cur­rent sta­ble re­lease 487 days ago (the time our rewrite took to reach to fea­ture par­ity). So if we’d stayed on Rust for the same du­ra­tion, we could have seen our in­cre­men­tal build times de­crease from 10 sec­onds to 3.4. That’s a big jump! I re­ally ap­pre­ci­ate all the hard work that Rust con­trib­u­tors have done to im­prove build times. Eliminating 2/3 of our in­cre­men­tal build times over 18 months would have been a very wel­come change if we’d stayed on Rust, and it’s a big­ger im­prove­ment than I would have an­tic­i­pated in an 18-month pe­riod. Bravo!

As im­pres­sive as that im­prove­ment is, Zig’s 35ms is still way ahead. Not only is it 1/100th the build time of 3.4 sec­onds, it’s also in a dif­fer­ent per­for­mance cat­e­gory—and that 35ms is on a Zig code base with ~50% more lines of code than the Rust one that got 3.4s. I ex­pect Roc’s code base to keep grow­ing, and for this gap to keep grow­ing with it; I’ve never heard of any ini­tia­tive on Rust’s roadmap com­pa­ra­ble to -fincremental.

So while our de­ci­sion to re­main on sta­ble 0.16.0 (plus how many of our con­trib­u­tors run Mac lap­tops with ARM proces­sors; -fincremental only works on x86 – 64 CPUs right now) means we haven’t yet reaped the an­tic­i­pated build-time re­wards of choos­ing Zig for the rewrite, we cer­tainly have some­thing to look for­ward to in the next sta­ble Zig re­lease!

Memory Control: Zero-Parse Deserialization

Roc’s new on-disk caching sys­tem uses a tech­nique I first learned about from Zig’s com­piler, and which Casey Muratori told me is com­mon prac­tice in game pro­gram­ming. It re­lies on the happy co­in­ci­dence that if you’re or­ga­niz­ing your mem­ory in the way that runs fastest on mod­ern hard­ware any­way, you can also load it from disk di­rectly into mem­ory and start us­ing it with­out pars­ing any­thing.

Here’s how it works:

All of our com­piler data struc­tures are rep­re­sented as ar­rays with 32-bit in­dices over point­ers (and of­ten in struc­ture-of-ar­rays form).

This not only saves mem­ory and runs faster, it also means our data struc­tures can be writ­ten di­rectly to disk with­out need­ing to be se­ri­al­ized into a dif­fer­ent for­mat first.

The big­ger ben­e­fit is that this lets us de­se­ri­al­ize them back into mem­ory with­out pars­ing the on-disk bytes in any way. We load the bytes into mem­ory, do some re­lo­ca­tions to point our ex­ist­ing data struc­tures to the newly-loaded ar­rays, and we’re ready to go.

This means we de­se­ri­al­ize at the speed of load­ing the bytes from disk into mem­ory—so, ac­tu­ally I/O bound. If those bytes are al­ready in the op­er­at­ing sys­tem’s disk cache, it means we load cached work from pre­vi­ous builds at roughly the speed of mem­cpy.

When you run roc check twice in a row, the first time it caches all of its out­puts on disk us­ing this strat­egy. The sec­ond time, if the in­put source code files haven’t changed, all the parsed/​type-checked/​etc. data struc­tures jump straight from disk into mem­ory. It’s ex­tremely fast. roc test sim­i­larly caches the out­comes for tests of pure func­tions (which are de­ter­min­is­tic), and all of this is done with file-level gran­u­lar­ity, so if you change one file you’ll only be pay­ing for re­do­ing work of that file and any oth­ers that de­pend on it.

This zero-parse de­se­ri­al­iza­tion strat­egy only works be­cause we’re fol­low­ing this pro­gram­ming with­out point­ers style for all of our com­piler data struc­tures. If we in­stead used point­ers every­where (like al­most all com­pil­ers do), de­se­ri­al­iza­tion could­n’t be zero-parse.

This ap­proach has safety risks, how­ever. Similarly to how a pointer in mem­ory can point to the wrong ad­dress (e.g. lead­ing to a use-af­ter-free), any in­dex can be used as a lookup into the wrong ar­ray at run­time, at which point you end up with what­ever ran­dom bytes hap­pened to be at that lo­ca­tion. Rust’s bor­row checker is de­signed to help with pointer life­times, but it does­n’t at­tempt to an­swer the ques­tion “which in­dex goes with which ar­ray?” be­cause that has never been in scope for its de­sign.