Semble is a code search li­brary built for agents. It re­turns the ex­act code snip­pets they need in­stantly, us­ing ~98% fewer to­kens than grep+read. Indexing and search­ing a full code­base end-to-end takes un­der a sec­ond, with ~200x faster in­dex­ing and ~10x faster queries than a code-spe­cial­ized trans­former, at 99% of its re­trieval qual­ity (see bench­marks). Everything runs on CPU with no API keys, GPU, or ex­ter­nal ser­vices. Run it as an MCP server or call it from the shell via AGENTS.md and any agent (Claude Code, Cursor, Codex, OpenCode, etc.) gets in­stant ac­cess to any repo.

Quickstart

Your agent queries Semble in nat­ural lan­guage (e.g. “How is au­then­ti­ca­tion han­dled?“) and gets back only the rel­e­vant code snip­pets, with­out grep­ping or read­ing full files. Set it up as an MCP server or via AGENTS.md:

MCP (Claude Code)

Add Semble to Claude Code (requires uv):

claude mcp add sem­ble -s user — uvx –from “semble[mcp]” sem­ble

Using Codex, OpenCode, or Cursor? See MCP Server for setup in­struc­tions.

Bash / AGENTS.md

Install Semble, then add the snip­pet be­low to your AGENTS.md or CLAUDE.md:

pip in­stall sem­ble # Install with pip uv tool in­stall sem­ble # Or in­stall with uv

## Code Search

Use `semble search` to find code by de­scrib­ing what it does or nam­ing a sym­bol/​iden­ti­fier, in­stead of grep:

​```bash sem­ble search “authentication flow” ./my-project sem­ble search “save_pretrained” ./my-project sem­ble search “save model to disk” ./my-project –top-k 10 ​```

Use `semble find-re­lated` to dis­cover code sim­i­lar to a known lo­ca­tion (pass `file_path` and `line` from a prior search re­sult):

​```bash sem­ble find-re­lated src/​auth.py 42 ./my-project ​```

`path` de­faults to the cur­rent di­rec­tory when omit­ted; git URLs are ac­cepted.

If `semble` is not on `$PATH`, use `uvx –from “semble[mcp]” sem­ble` in its place.

### Workflow

1. Start with `semble search` to find rel­e­vant chunks. 2. Inspect full files only when the re­turned chunk is not enough con­text. 3. Optionally use `semble find-re­lated` with a promis­ing re­sult’s `file_path` and `line` to dis­cover re­lated im­ple­men­ta­tions. 4. Use grep only when you need ex­haus­tive lit­eral matches or quick con­fir­ma­tion of an ex­act string.

Once in­stalled, run sem­ble sav­ings to see how many to­kens Semble has saved you. Note that for sub-agent sup­port in Claude Code or Codex, you need the full Bash / AGENTS.md setup be­low.

pip in­stall –upgrade sem­ble # with pip uv tool up­grade sem­ble # with uv uv cache clean sem­ble # for MCP users (restart your MCP client af­ter)

Main Features

Fast: in­dexes an av­er­age repo in ~250 ms and an­swers queries in ~1.5 ms, all on CPU.

Accurate: NDCG@10 of 0.854 on our bench­marks, on par with code-spe­cial­ized trans­former mod­els, at a frac­tion of the size and cost.

Token-efficient: re­turns only the rel­e­vant chunks, us­ing ~98% fewer to­kens than grep+read.

Zero setup: runs on CPU with no API keys, GPU, or ex­ter­nal ser­vices re­quired.

MCP server: works with Claude Code, Cursor, Codex, OpenCode, and any other MCP-compatible agent.

Local and re­mote: pass a lo­cal path or a git URL.

MCP Server

Semble can run as an MCP server so agents can search any code­base di­rectly. Repos are cloned and in­dexed on de­mand, and in­dexes are cached for the life­time of the ses­sion. Local paths are watched for file changes and re-in­dexed au­to­mat­i­cally.

Setup

Requires uv to be in­stalled.

Requires uv to be in­stalled.

Claude Code

claude mcp add sem­ble -s user — uvx –from “semble[mcp]” sem­ble

Codex

Add to ~/.codex/config.toml:

[mcp_servers.semble] com­mand = “uvx” args = [“–from”, “semble[mcp]”, “semble”]

OpenCode

Add to ~/.opencode/config.json:

{ “mcp”: { “semble”: { “type”: “local”, “command”: [“uvx”, “–from”, “semble[mcp]”, “semble”] } } }

Cursor

Add to ~/.cursor/mcp.json (or .cursor/mcp.json in your pro­ject):

{ “mcpServers”: { “semble”: { “command”: “uvx”, “args”: [“–from”, “semble[mcp]”, “semble”] } } }

Tools

Bash / AGENTS.md

An al­ter­na­tive to MCP is to in­voke Semble via Bash. For Claude Code and Codex CLI, this is the only op­tion for sub-agents, which can­not call MCP tools di­rectly, though it can also be used along­side MCP for the top-level agent.

To add Bash sup­port, ap­pend the fol­low­ing to your AGENTS.md or CLAUDE.md:

## Code Search

Use `semble search` to find code by de­scrib­ing what it does or nam­ing a sym­bol/​iden­ti­fier, in­stead of grep:

​```bash sem­ble search “authentication flow” ./my-project sem­ble search “save_pretrained” ./my-project sem­ble search “save model to disk” ./my-project –top-k 10 ​```

Use `semble find-re­lated` to dis­cover code sim­i­lar to a known lo­ca­tion (pass `file_path` and `line` from a prior search re­sult):

​```bash sem­ble find-re­lated src/​auth.py 42 ./my-project ​```

`path` de­faults to the cur­rent di­rec­tory when omit­ted; git URLs are ac­cepted.

If `semble` is not on `$PATH`, use `uvx –from “semble[mcp]” sem­ble` in its place.

## Workflow

1. Start with `semble search` to find rel­e­vant chunks. 2. Inspect full files only when the re­turned chunk is not enough con­text. 3. Optionally use `semble find-re­lated` with a promis­ing re­sult’s `file_path` and `line` to dis­cover re­lated im­ple­men­ta­tions. 4. Use grep only when you need ex­haus­tive lit­eral matches or quick con­fir­ma­tion of an ex­act string.

Claude Code sub-agent: Claude Code also sup­ports a ded­i­cated sub-agent. Run this once in your pro­ject root:

sem­ble init # or, if sem­ble is not on $PATH: uvx –from “semble[mcp]” sem­ble init

This writes .claude/agents/semble-search.md.

CLI

Semble also ships as a stand­alone CLI. This is use­ful in scripts or any­where you want search re­sults with­out an MCP ses­sion.

# Search a lo­cal repo sem­ble search “authentication flow” ./my-project

# Search for a sym­bol or iden­ti­fier sem­ble search “save_pretrained” ./my-project

# Search a re­mote repo (cloned on de­mand) sem­ble search “save model to disk” https://​github.com/​Min­ish­Lab/​mod­el2vec

# Limit re­sults sem­ble search “save model to disk” ./my-project –top-k 10

# Find code sim­i­lar to a known lo­ca­tion sem­ble find-re­lated src/​auth.py 42 ./my-project

path de­faults to the cur­rent di­rec­tory when omit­ted; git URLs are ac­cepted. If sem­ble is not on $PATH, use uvx –from “semble[mcp]” sem­ble in its place.

sem­ble sav­ings shows how many to­kens sem­ble has saved across all your searches:

sem­ble sav­ings # sum­mary by pe­riod sem­ble sav­ings –verbose # also show break­down by call type

Semble Token Savings ════════════════════════════════════════════════════════════════ Period Calls Savings ──────────────────────────────────────────────────────────────── Today 42 [███████████████░] ~58.4k to­kens (95%) Last 7 days 287 [██████████████░░] ~312.4k to­kens (90%) All time 1.4k [██████████████░░] ~1.2M to­kens (89%)

Savings are cal­cu­lated as fol­lows: for each call, sem­ble records the to­tal char­ac­ter count of the unique files con­tain­ing re­turned chunks and the char­ac­ter count of the snip­pets re­turned. Estimated to­kens saved is (file chars − snip­pet chars) / 4 (4 chars per to­ken). This is a con­ser­v­a­tive es­ti­mate: the base­line is read­ing matched files in full, which is how cod­ing agents of­ten ex­plore un­fa­mil­iar code.

Stats are stored in ~/.semble/savings.jsonl.

Semble can also be used as a Python li­brary for pro­gram­matic ac­cess, use­ful when build­ing cus­tom tool­ing or in­te­grat­ing search di­rectly into your own code.

from sem­ble im­port SembleIndex

# Index a lo­cal di­rec­tory in­dex = SembleIndex.from_path(”./my-project”)

# Index a re­mote git repos­i­tory in­dex = SembleIndex.from_git(“https://​github.com/​Min­ish­Lab/​mod­el2vec”)

# Search the in­dex with a nat­ural-lan­guage or code query re­sults = in­dex.search(“save model to disk”, top_k=3)

# Find code sim­i­lar to a spe­cific re­sult re­lated = in­dex.find­_re­lated(re­sults[0], top_k=3)

# Each re­sult ex­poses the matched chunk re­sult = re­sults[0] re­sult.chunk.file_­path # “model2vec/model.py” re­sult.chunk.start_­line # 127 re­sult.chunk.end_­line # 150 re­sult.chunk.con­tent # “def save_pre­trained(self, path: PathLike, …”

Benchmarks

We bench­mark qual­ity and speed across ~1,250 queries over 63 repos­i­to­ries in 19 lan­guages (left), and to­ken ef­fi­ciency against grep+read at equiv­a­lent re­call lev­els (right).

The qual­ity bench­mark (left) scores re­trieval qual­ity (NDCG@10) against to­tal la­tency; sem­ble achieves 99% of the qual­ity of the 137M-parameter CodeRankEmbed Hybrid while in­dex­ing 218x faster. The to­ken ef­fi­ciency bench­mark (right) mea­sures how many to­kens each method needs to reach a given re­call level; sem­ble uses 98% fewer to­kens on av­er­age and hits 94% re­call at only 2k to­kens, while grep+read needs a full 100k con­text win­dow to reach 85%. See bench­marks for per-lan­guage re­sults, ab­la­tions, and full method­ol­ogy.

How it works

Semble splits each file into code-aware chunks us­ing tree-sit­ter, then scores every query against the chunks with two com­ple­men­tary re­triev­ers: sta­tic Model2Vec em­bed­dings us­ing the code-spe­cial­ized po­tion-code-16M model for se­man­tic sim­i­lar­ity, and BM25 for lex­i­cal matches on iden­ti­fiers and API names. The two score lists are fused with Reciprocal Rank Fusion (RRF).

After fus­ing, re­sults are reranked with a set of code-aware sig­nals:

Adaptive weight­ing. Symbol-like queries (Foo::bar, _private, ge­tUser­ById) get more lex­i­cal weight, while nat­ural-lan­guage queries stay bal­anced be­tween se­man­tic and lex­i­cal re­triev­ers.

Definition boosts. A chunk that de­fines the queried sym­bol (a class, def, func, etc.) is ranked above chunks that merely ref­er­ence it.

Identifier stems. Query to­kens are stemmed and matched against iden­ti­fier stems in a chunk, giv­ing an ad­di­tional weight to chunks that con­tain them. For ex­am­ple, query­ing parse con­fig boosts chunks con­tain­ing par­seC­on­fig, ConfigParser, or con­fig_­parser.

File co­her­ence. When mul­ti­ple chunks from the same file match the query, the file is boosted so the top re­sult re­flects broad file-level rel­e­vance rather than a sin­gle out-of-con­text chunk.

Noise penal­ties. Test files, com­pat//​legacy/ shims, ex­am­ple code, and .d.ts de­c­la­ra­tion stubs are down-ranked so canon­i­cal im­ple­men­ta­tions sur­face first.

Because the em­bed­ding model is sta­tic with no trans­former for­ward pass at query time, all of this runs in mil­lisec­onds on CPU.

License

MIT

Citing

If you use Semble in your re­search, please cite the fol­low­ing:

@software{minishlab2026semble, au­thor = {{van Dongen}, Thomas and Stephan Tulkens}, ti­tle = {Semble: Fast and Accurate Code Search for Agents}, year = {2026}, pub­lisher = {Zenodo}, doi = {10.5281/zenodo.19785932}, url = {https://​github.com/​Min­ish­Lab/​sem­ble}, li­cense = {MIT} }

Abstract

We pre­sent GenCAD, an im­age-con­di­tional CAD gen­er­a­tion model. Our model not only gen­er­ates the 3D CAD but also the en­tire pa­ra­me­ter­ized CAD com­mand his­tory, CAD pro­gram, as out­put.

The com­plex­ity of CAD data struc­tures such as bound­ary rep­re­sen­ta­tion (B-rep) makes it dif­fi­cult to train ef­fi­cient AI mod­els. Due to the ease of data avail­abil­ity, com­mon ap­proaches of­ten re­sort to rep­re­sen­ta­tions like meshes, vox­els, or point clouds, which sac­ri­fice the ac­cu­racy and mod­i­fi­a­bil­ity of true CAD mod­els that are crit­i­cal for en­gi­neer­ing tasks, man­u­fac­tur­ing and de­sign space ex­plo­ration. Here we pro­pose GenCAD, an im­age con­di­tional gen­er­a­tive model that gen­er­ates para­met­ric CAD com­mand se­quences, also known as CAD pro­grams, that can be con­verted to a 3D solid model us­ing a geom­e­try ker­nel. At the core of GenCAD, we de­velop a strong rep­re­sen­ta­tion learn­ing frame­work for mul­ti­ple modal­i­ties of com­pu­ta­tional en­gi­neer­ing de­signs.

Our pro­posed GenCAD ar­chi­tec­ture is a com­bi­na­tion of four crit­i­cal steps; 1) an au­tore­gres­sive trans­former en­coder is used for learn­ing the la­tent rep­re­sen­ta­tion of the CAD com­mand se­quences, 2) a con­trastive learn­ing-based model is used to learn the joint rep­re­sen­ta­tions of the la­tent spaces be­tween CAD com­mand se­quences and CAD-images, 3) a la­tent dif­fu­sion model that can gen­er­ate the la­tent rep­re­sen­ta­tion of CAD com­mand se­quences con­di­tioned on CAD-images, and 4) fi­nally, a de­coder model that can con­vert cad la­tents into a se­quence of para­met­ric CAD com­mands. Most im­por­tantly, GenCAD does not merely gen­er­ate a 3D solid but also the en­tire CAD pro­gram. Our work rep­re­sents a step for­ward in CAD, of­fer­ing more pre­cise and mod­i­fi­able 3D mod­el­ing from im­ages, po­ten­tially en­hanc­ing au­to­mated de­sign processes.

Former Google CEO Eric Schmidt was booed mul­ti­ple times Sunday while dis­cussing ar­ti­fi­cial in­tel­li­gence dur­ing a com­mence­ment speech at the University of Arizona.

Schmidt, who led Google for a decade, opened his re­marks by re­flect­ing on his own stu­dent years and the rise of the com­puter, — a de­vice named Time mag­a­zine’s “Person of the Year” in 1982. He traced its evo­lu­tion into the lap­top and smart­phone and its pro­lif­er­a­tion through the in­ter­net and so­cial me­dia.

While the com­puter con­nected peo­ple, “democratized knowl­edge” and lifted many out of poverty, it also car­ried a darker side, Schmidt said.

“The same plat­forms that gave every­one a voice, like you’re us­ing now, also de­graded the pub­lic square,” he said. “They re­warded out­rage. They am­pli­fied our worst in­stincts. They coarsen the way we speak to each other, and that way, and in the way that we treat each other, is in the essence of a so­ci­ety.”

Schmidt then drew a par­al­lel be­tween ar­ti­fi­cial in­tel­li­gence and the trans­for­ma­tive im­pact of the com­puter — and was im­me­di­ately met with boos.

“I know what many of you are feel­ing about that. I can hear you,” Schmidt said, ad­dress­ing the crowd as many con­tin­ued to boo him. “There is a fear … there is a fear in your gen­er­a­tion that the fu­ture has al­ready been writ­ten, that the ma­chines are com­ing, that the jobs are evap­o­rat­ing, that the cli­mate is break­ing, that pol­i­tics is frac­tured, and that you are in­her­it­ing a mess that you did not cre­ate, and I un­der­stand that fear.”

He went on to ar­gue that the fu­ture re­mains un­writ­ten and that the grad­u­at­ing class of 2026 has real power to shape how AI de­vel­ops — a claim that drew fur­ther dis­ap­proval from parts of the au­di­ence.

Schmidt urged grad­u­ates to em­brace free­dom, open de­bate, equal­ity and the will­ing­ness to en­gage with those they dis­agree with.

“If you’d let me make this point, please —” Schmidt said amid boos. “The point I’d like to make is choose a di­ver­sity of per­spec­tives, in­clud­ing the per­spec­tive of the im­mi­grant who has so of­ten been the per­son who came to this coun­try and made it bet­ter. America is at its best when we are the coun­try that am­bi­tious peo­ple want to come to. Let us not lose that.”

He closed by con­grat­u­lat­ing the class and of­fer­ing them clos­ing words. “The fu­ture is not yet fin­ished. It is now your turn to shape it.”

University of Arizona spokesper­son Mitch Zak said Schmidt was in­vited to de­liver the com­mence­ment ad­dress be­cause of his “extraordinary lead­er­ship and global con­tri­bu­tions in tech­nol­ogy, in­no­va­tion and sci­en­tific ad­vance­ment.”

“He helped lead Google’s rise into one of the world’s most in­flu­en­tial tech­nol­ogy com­pa­nies and con­tin­ues to ad­vance re­search and dis­cov­ery through ma­jor phil­an­thropic and sci­en­tific ini­tia­tives, in­clud­ing part­ner­ships that sup­port im­por­tant work at the University of Arizona,” Zak added.

Schmidt’s re­cep­tion was not an iso­lated in­ci­dent. Earlier this month, real es­tate ex­ec­u­tive Gloria Caulfield was sim­i­larly booed at a com­mence­ment speech at the University of Central Florida af­ter men­tion­ing the con­tro­ver­sial tech­nol­ogy. “The rise of ar­ti­fi­cial in­tel­li­gence is the next in­dus­trial rev­o­lu­tion,” she said as the crowd erupted in boos.

A sim­ple ap­pli­ca­tion for your .md files. All data stays on your de­vice.

You can store whole your life:

📌 Notes

📝 Documents, Projects

💚 Journal, Habits

✅ Checklists, Tasks

All in plain .md files, lo­cal-first. LLM-friendly.

Try it out: app.files.md (Beta)

My friends and I have been build­ing this pro­ject for 5 years. Hope you’ll like it!

Sponsor on GitHub 💚

Another note tak­ing app?

Maybe. But this time:

Only nec­es­sary fea­tures, re­stric­tions fos­ter cre­ativ­ity

No need to in­stall any­thing, all you need is a browser

Works of­fline

Local-first, files don’t leave your de­vice

Free and open source, you can tweak it how­ever you want

Extremely sim­ple code. One per­son or an LLM can fit the whole pro­ject in head

Portable, no build sys­tems, just open web/​in­dex.html

Optional out of the box syn­chro­niza­tion

The server is just one bi­nary (or use iCloud/​Drop­box/​Google Drive for sync)

Telegram chat­bot for on-the-go ac­cess to your files

How to use

Open app.files.md in Chrome browser

Click “Install files.md” on the right side of the ad­dress bar:

Open a lo­cal folder to per­sist changes

Occasionally hit force-re­fresh (Cmd+Shift+R) to get new up­dates.

Dump your thoughts

You can use chat to quickly dump your thoughts.

It will be syn­chro­nized across all de­vices.

Open the chat and send a mes­sage:

Choose where to save (can do later):

With this flow you can quickly save notes, jour­nal and check­lists.

Save things in the chat­bot

Open the chat, write some­thing and press Enter:

That’s it.

Telegram Bot

Other mes­sen­gers will fol­low

How to grow your knowl­edge base

Connect ideas. Let them com­pound. Think through.

I used app.files.md to grow my knowl­edge about brain and soft­ware de­vel­op­ment

I added new notes to ei­ther brain or dev fold­ers. One idea per note

I made con­nec­tions be­tween the rel­e­vant notes in the web app (type [)

Everything is con­nected, just as in our brain

I spent time trav­el­ling through the notes and think­ing it through

At some point, brain and dev notes ap­peared very re­lated

An in­ter­con­nec­tion be­tween do­mains pro­duced an in­sight

I wrote an ar­ti­cle based on that in­sight: Cognitive Load in Software Development

All this ac­tiv­ity helped me to:

Think deeply (which is very im­por­tant in the AI-age)

Think sys­tem­at­i­cally and see the big­ger pic­ture

Write in­sight­ful texts

To achieve all that, you’ll have to use your brain, not ad­vanced tem­plates or AI work­flows.

Start with no struc­ture at all, 0 fold­ers

One idea per note

Every note should be un­der­stood with­out con­text

Apply new knowl­edge im­me­di­ately, don’t save it for fu­ture self

Link re­lated notes

Revisit your notes and think through

My friends and I have been us­ing this sim­ple setup for five years, and it works well.

Second Brain?

I’ll quote I Deleted My Second Brain:

Obsidian is a bril­liant piece of soft­ware. I love it, dearly. But like any­thing, with­out re­straint, it can also be a trap. Markdown files in nested fold­ers. Plugins that track your pro­duc­tiv­ity. Graph views that sug­gest om­ni­science. There’s an il­lu­sion of mas­tery in watch­ing your notes web into con­stel­la­tions. But con­stel­la­tions are pro­jec­tions. They tell sto­ries. They do not guar­an­tee un­der­stand­ing. When I first started us­ing PKM tools, I be­lieved I was solv­ing a prob­lem of for­get­ting. Later, I be­lieved I was solv­ing a prob­lem of in­te­gra­tion. Eventually, I re­al­ized I had cre­ated a new prob­lem: de­fer­ral. The more my sys­tem grew, the more I de­ferred the work of thought to some fu­ture self who would sort, tag, dis­till, and ex­tract the gold. That self never ar­rived.

Obsidian is a bril­liant piece of soft­ware. I love it, dearly. But like any­thing, with­out re­straint, it can also be a trap. Markdown files in nested fold­ers. Plugins that track your pro­duc­tiv­ity. Graph views that sug­gest om­ni­science. There’s an il­lu­sion of mas­tery in watch­ing your notes web into con­stel­la­tions. But con­stel­la­tions are pro­jec­tions. They tell sto­ries. They do not guar­an­tee un­der­stand­ing.

When I first started us­ing PKM tools, I be­lieved I was solv­ing a prob­lem of for­get­ting. Later, I be­lieved I was solv­ing a prob­lem of in­te­gra­tion.

Eventually, I re­al­ized I had cre­ated a new prob­lem: de­fer­ral. The more my sys­tem grew, the more I de­ferred the work of thought to some fu­ture self who would sort, tag, dis­till, and ex­tract the gold.

That self never ar­rived.

The Second Brain is thrilling. Advanced guru tem­plates, plu­g­ins and AI work­flows… One wants to scrape the wis­dom of the whole in­ter­net. There’s some beauty in this neat sys­tem. Every new note brings dopamine. Second Brain gets bet­ter and bet­ter.

However, the first brain never ac­tu­ally gets smarter. And that’s an is­sue - in the AI age, your first brain is as valu­able as ever.

Use your brain to think through the notes.

Notes can pre­vent ex­pe­ri­ence

Reading and tak­ing notes can eas­ily fool us into be­liev­ing that we un­der­stand a text

We think we un­der­stand, but in re­al­ity we just know

At some point our “knowing” is so good, that we start feel­ing that we ac­tu­ally do it (or at least tried)

The worst thing is that we don’t let new ex­pe­ri­ences emerge be­cause we al­ready have knowl­edge. It’s a knowl­edge bar­rier. Life gives us op­por­tu­ni­ties to live through new ex­pe­ri­ences, but we refuse, be­cause “we al­ready know”.

Self-help through read­ing and tak­ing notes? 🧘‍

Harm caused at the emo­tional level must be healed at the emo­tional level.

Not through in­tel­lec­tual work and tak­ing notes. Reading with­out ac­tion is en­ter­tain­ment. A form of pro­cras­ti­na­tion. No amount of self-help books can heal emo­tional wounds. What can help is psy­chother­apy, re­script­ing and chair work. Meditation. Healing hap­pens by feel­ing.

When to take notes

If your goal is to:

Develop a deeper, more struc­tured un­der­stand­ing of some­thing

Do re­search

Write an ar­ti­cle or a book

Then tak­ing notes is per­fectly fine.

Files struc­ture

You don’t have to think about the struc­ture, it is pre­de­fined. Although, you’re free to use what­ever struc­ture you want.

Chat: Chat.md

Notes: brain/​Note.md, <category>/*.md

Checklists: Read.md, Watch.md, Shop.md, MyChecklist_.md

Journal: jour­nal/​2024.08 August.md

Tasks: Later.md

Habits: habits/​Ate con­sciously.md, habits/*.​md

Images: me­dia/* (png, jpg, webp, gif)

Archive: archive/*.​md

Config: con­fig.json

Scheme is also avail­able at files.md/​llms.txt. You can copy-paste it into CLAUDE.md or AGENTS.md, so that your AI agent would un­der­stand the struc­ture.

Hotkeys

Useful scripts for your files

All scripts are in cmd and can be run in­side your files di­rec­tory. Install Go first.

Add Whoop met­rics to jour­nal

Automated data bro­ker opt-out run­ner for ma­cOS. Removes your per­sonal in­for­ma­tion from 500+ peo­ple-search sites and data bro­ker data­bases on a monthly sched­ule — with CAPTCHA solv­ing, per­sis­tent state track­ing (so com­pleted opt-outs aren’t re­sub­mit­ted every run), and an iMes­sage no­ti­fi­ca­tion when done.

What it does

Each month, the script:

Searches each data bro­ker site for your name + state

Finds your spe­cific list­ing (for sites that need a pro­file URL)

Fills and sub­mits the opt-out form au­to­mat­i­cally

Solves CAPTCHAs via CapSolver (AI-powered, ~$0.001/solve)

Skips bro­kers you were al­ready re­moved from re­cently (90-day re-check win­dow)

Sends you an iMes­sage with the re­sults sum­mary

Opens any sites that re­quire man­ual ac­tion in your browser

Requirements

ma­cOS (uses launchd for sched­ul­ing and Messages for iMes­sage)

Node.js 18+

Playwright browsers in­stalled

npx play­wright in­stall chromium

Quick Start

# 1. Clone the repo git clone https://​github.com/​stephenlthorn/​auto-iden­tity-re­move.git cd auto-iden­tity-re­move

# 2. Install de­pen­den­cies npm in­stall

# 3. Run in­ter­ac­tive setup (creates con­fig.json and sched­ules the monthly job) node setup.js

# 4. Run man­u­ally any­time ./run.sh

Setup walk­through

node setup.js guides you through:

Your per­sonal info never leaves your ma­chine. con­fig.json and state.json are both git­ig­nored.

CapSolver (optional but rec­om­mended)

Some opt-out forms have re­CAPTCHA. Without CapSolver, those sites go to your man­ual list in­stead of be­ing han­dled au­to­mat­i­cally.

Sign up at cap­solver.com — free, pay-as-you-go

Add $1 – 2 of cred­its (enough for months of use at ~$0.001/solve)

Paste your API key when setup.js asks, or add it to con­fig.json:

“capsolver”: { “apiKey”: “CAP-xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx” }

Files

auto-iden­tity-re­move/ ├── setup.js ← Run once: in­ter­ac­tive setup + sched­ul­ing ├── watcher.js ← Main run­ner ├── bro­kers.js ← Broker list with opt-out strate­gies ├── run.sh ← Manual trig­ger ├── con­fig.ex­am­ple.json ← Template (copy → con­fig.json) ├── pack­age.json ├── .gitignore │ ├── con­fig.json ← YOUR per­sonal info (gitignored, cre­ated by setup.js) ├── state.json ← Opt-out his­tory / skip logic (gitignored) └── logs/ ← Per-run JSON logs (gitignored)

State track­ing

state.json tracks when each bro­ker was last suc­cess­fully opted out. The de­fault re-check win­dow is 90 days — bro­kers typ­i­cally re-add your data within that win­dow, so the script re-sub­mits when it’s time.

{ “optOuts”: { “Spokeo”: { “lastSuccess”: “2026 – 05-01T09:00:00.000Z”, “totalRuns”: 3, “detail”: “” } } }

On each run you’ll see:

✅ Removed — opt-out sub­mit­ted this run

⏭ Skipped (fresh) — re­moved re­cently, re-check not due yet

🔍 Not listed — your name was­n’t found on that site

📋 Manual needed — opened in your browser for you to han­dle

❌ Error — net­work/​time­out is­sue, will retry next run

Brokers cov­ered

Auto-removed (30+)

Generic — 500+ ad­di­tional bro­kers (auto-detected)

generic-run­ner.js han­dles the re­main­ing ~470 bro­kers from two pub­lic datasets:

For each site it tries four strate­gies in or­der:

Click a “Do Not Sell My Personal Information” but­ton

Opt out via OneTrust / TrustArc / Osano pri­vacy man­ager

Fill any generic opt-out form (email, name, state) and sub­mit

Find and record a DSAR / data re­quest link for man­ual fol­low-up

Sites re­quir­ing man­ual ac­tion are opened in your browser au­to­mat­i­cally.

Manual (opened in browser for you)

Adding more bro­kers

Edit bro­kers.js and add an en­try:

{ name: ‘NewBrokerSite’, method: ‘direct-form’, // or ‘search-form’, ‘email’, ‘manual’ optOutUrl: ’https://​ex­am­ple.com/​opt-out’, form­Fields: { ‘input[name*=“first” i]’: F, // F, L, N, E, ST, Z are from con­fig ‘input[name*=“last” i]’: L, ‘input[type=“email”]’: E, }, sub­mit­S­e­lec­tor: ‘button[type=“submit”]’, captcha­Likely: false, pri­or­ity: 2, }

PRs wel­come — es­pe­cially for bro­kers with ver­i­fied work­ing se­lec­tors.

Manual run

./run.sh

Dry-run mode — nav­i­gates to each site and fills forms but does NOT sub­mit any­thing. Good for ver­i­fy­ing what the script will do be­fore your first real run:

node watcher.js –dry-run

Or to run in the back­ground and log out­put:

./run.sh >> logs/​man­ual-run.log 2>&1 &

Uninstall / dis­able sched­ule

launchctl un­load ~/Library/LaunchAgents/com.auto-identity-remove.plist rm ~/Library/LaunchAgents/com.auto-identity-remove.plist

Is it safe to sub­mit my info to 500 opt-out forms?

A fair con­cern raised by some users: aren’t you just con­firm­ing your data to the bro­kers by fill­ing out their forms?

A few things worth know­ing:

These bro­kers al­ready have your info. You’re not re­veal­ing any­thing new — you’re us­ing the legally-re­quired re­moval mech­a­nism they’re ob­lig­ated to pro­vide.

CCPA (California) and sim­i­lar state laws re­quire bro­kers to honor opt-out re­quests. Submitting the form cre­ates a le­gal oblig­a­tion to re­move you. Doing noth­ing does not.

The script uses info you’re al­ready listed un­der — your name as it ap­pears pub­licly, your state, your email. It does­n’t add new data points.

The al­ter­na­tive is worse. Every month that passes, more bro­kers scrape and re­sell your data. Opt-outs are im­per­fect, but they work more of­ten than not.

That said: if you’re in a sit­u­a­tion where even con­firm­ing your email ad­dress to a bro­ker is a risk, this tool is not the right ap­proach. Consider a paid ser­vice that uses a proxy email.

California res­i­dents: DELETE Registry (August 2025)

California is launch­ing an of­fi­cial Delete Me opt-out reg­istry on August 1, 2025. Once reg­is­tered, data bro­kers are legally re­quired to delete your info au­to­mat­i­cally — no in­di­vid­ual form sub­mis­sions needed for par­tic­i­pat­ing bro­kers.

Register at: optoutreg­istry.oag.ca.gov (live August 1)

Recommended: Register with the CA Delete Registry first, then run this script for the bro­kers that aren’t cov­ered.

Why not just use a paid ser­vice?

Paid ser­vices like Incogni ($96/yr) or Optery ($39/yr) are ex­cel­lent and cover more bro­kers with pro­fes­sion­ally main­tained opt-out flows. This tool is for peo­ple who want full con­trol, trans­parency, and no re­cur­ring sub­scrip­tion — or who want to han­dle the gaps those ser­vices miss (Acxiom, LexisNexis, ZoomInfo, Clearbit).

Using both is the strongest ap­proach: a paid ser­vice for the bulk of bro­kers + this script for the gaps.

License

MIT

MOUNTAIN HOME AFB, Idaho — All four crew mem­bers ejected safely af­ter two Navy jets col­lided and crashed on Sunday dur­ing an air show at the Mountain Home Air Force Base, of­fi­cials said.

The col­li­sion in­volved two U.S. Navy EA-18G Growlers from the Electronic Attack Squadron 129 in Whidbey Island, Washington, said Cmdr. Amelia Umayam, spokesper­son for Naval Air Forces, U.S. Pacific Fleet.

The air­craft were per­form­ing an aer­ial demon­stra­tion when the crash hap­pened, Umayam said in a state­ment. She said the four crew mem­bers from both jets safely ejected and were be­ing eval­u­ated by med­ical per­son­nel.

Nobody at the mil­i­tary base was hurt, said Kim Sykes, mar­ket­ing di­rec­tor with Silver Wings of Idaho, which helped to plan the air show.

“Everyone is safe, and I think that’s the most im­por­tant thing,” Sykes said.

The base said in a so­cial me­dia post that it was locked down fol­low­ing the in­ci­dent.

Videos posted on­line by spec­ta­tors showed four para­chutes open­ing in the sky as the air­craft plum­meted to the ground near the base about 50 miles (80 kilo­me­ters) south of Boise.

Shane Odgen said he was film­ing the two jets as they came close to­gether. A video he cap­tured shows the two air­craft ap­pear to make con­tact and then spin in tan­dem as the crew mem­bers eject and their para­chutes open.

The planes then fall to­gether, ex­plod­ing into a fire­ball upon im­pact as the crew mem­bers drop to the ground nearby.

“I was just film­ing, think­ing they were go­ing to split apart, and that hap­pened, and I filmed the rest,” Ogden said in a text mes­sage.

He said he left soon af­ter the crash be­cause he did not want to get in the way of emer­gency re­spon­ders.

Organizers said the pop­u­lar air show, which in­cludes fly­ing demon­stra­tions and para­chute jumps, is a cel­e­bra­tion of avi­a­tion his­tory and a show­case of mod­ern mil­i­tary ca­pa­bil­i­ties.

The U.S. Air Force Thunderbirds demon­stra­tion squadron head­lined the show both days.

The National Weather Service re­ported good vis­i­bil­ity and winds gust­ing up to 29 mph around the time of the crash.

This year’s Gunfighter Skies event was the first at the base since 2018, when a hang glider died in a crash dur­ing an air show per­for­mance.

In 2003, a Thunderbirds air­craft crashed while at­tempt­ing a ma­neu­ver. The pi­lot, who was not hurt, was able to steer the plane away from the crowd and eject less than a sec­ond be­fore it hit the ground.

The air show in­dus­try has been work­ing to im­prove safety for years at the roughly 200 events held each year in the U.S.

The last fa­tal crash at an air show came in 2022 when two vin­tage mil­i­tary planes col­lided at an event in Dallas and killed six peo­ple.

John Cudahy, pres­i­dent and CEO of the International Council of Air Shows, said that there used to be an av­er­age of about two deaths a year at a U.S. air show. But over the past decade, the av­er­age has been closer to one death per year, he said.

There were no air show deaths in 2025 or 2024, and a spec­ta­tor has­n’t been killed at an air show since 1952.

“Safety-wise, we’ve en­joyed re­ally an un­prece­dented term of few ac­ci­dents,” Cudahy said.

Idaho Transportation says SH-167 is closed due to the crash and in­ves­ti­ga­tion from Simco Rd to SH-67 near the Mountain Home AFB. ITD says this is an­tic­i­pated to be a multi-day clo­sure.

Col. David Gunter, Wing Commander of the 366th Fighter Wing, re­leased the fol­low­ing state­ment on Sunday.

First and fore­most, we are in­cred­i­bly thank­ful that every­one in­volved in to­day’s in­ci­dent is safe. The ex­tra­or­di­nary pro­fes­sion­al­ism of our emer­gency re­sponse teams, in­clud­ing the city and county, al­lowed for quick re­sponse to the air­crew as well as se­cur­ing the scene to en­sure the safety of our guests, per­form­ers and com­mu­nity. And to all of our guests here to­day, I can’t tell you how much we ap­pre­ci­ated your pa­tience, trust and sup­port.

Silver Wings of Idaho, which is a vol­un­teer board work­ing along­side Mountain Home Air Force Base to help plan and sup­port the Gunfighter Skies 2026 Airshow, re­leased this state­ment.

Today was an emo­tional and dif­fi­cult day at Gunfighter Skies. While the events were not what any­one ex­pected, we are in­cred­i­bly grate­ful and re­lieved that all four pi­lots in­volved are safe. That is, and al­ways will be, the most im­por­tant thing. Our hearts are with every­one af­fected to­day, the pi­lots, their fam­i­lies, the crews, and all those who worked tire­lessly be­hind the scenes to make this event pos­si­ble. Situations like this re­mind us how quickly things can change and how much these men and women put on the line every time they take to the skies. We also want to rec­og­nize the in­cred­i­ble re­sponse from Mountain Home Air Force Base along with the many lo­cal, county, and state agen­cies that re­sponded to­day. Law en­force­ment, fire, EMS, emer­gency per­son­nel, and sup­port teams from mul­ti­ple agen­cies showed out­stand­ing pro­fes­sion­al­ism, team­work, and care through­out the sit­u­a­tion. We are be­yond thank­ful for every per­son who stepped in to help keep oth­ers safe dur­ing an in­cred­i­bly stress­ful mo­ment. To our guests, sup­port­ers, vol­un­teers, spon­sors, and com­mu­nity, thank you for your pa­tience, kind­ness, and un­der­stand­ing to­day. Even through un­ex­pected cir­cum­stances, peo­ple came to­gether and sup­ported one an­other, and that meant so much. Serving on the Silver Wings of Idaho board and help­ing bring this air­show to our com­mu­nity has truly been an honor. This week­end was filled with so much hard work, ded­i­ca­tion, and heart from count­less peo­ple be­hind the scenes. Today re­minded all of us that while air­shows in­spire and ex­cite us, safety and the peo­ple be­hind the mis­sion will al­ways come first. Tonight, we are sim­ply thank­ful that every­one made it home safe.

(Reporting from the Associated Press and Idahonews.com)

If vibecod­ing is what peo­ple say it is, the world should be drown­ing in vibecoded ar­ti­facts right now. Two years of ac­cess. Millions of peo­ple with the tools. The bar­rier sup­pos­edly fell, right? RIGHT? So where is every­thing?

Show me the ev­i­dence

Where is the vibecoded Photoshop. The vibecoded Excel. The vibecoded Maya. The vibecoded Blender. The vibecoded com­piler that com­piles it­self. The vibecoded data­base, the vibecoded OS, the vibecoded any­thing-that-re­quires-ar­chi­tec­tural-judg­ment-to-hold-to­gether. Huh?

I am not ask­ing for slop. Slop ex­ists, slop is easy, slop is not the ques­tion. I am ask­ing for the co­her­ent, com­plex, non-triv­ial things that vibecod­ing al­legedly makes ac­ces­si­ble to any­one who can prompt.

Silence. Every time. The cat­e­gory is empty.

And the ac­cusers never want to ad­dress that, be­cause ad­dress­ing it means ad­mit­ting the ac­cu­sa­tion does­n’t hold up.

Accusation with­out ev­i­dence

There are no vibecoded Photoshops be­cause vibecod­ing does not do what the rhetoric claims it does. The ac­cu­sa­tion it­self is the vibe. The ac­cuser feels that a thing must have been easy be­cause it was made with AI. They post that feel­ing as if it were a find­ing. The find­ing never has to be checked, be­cause no­body else checks it ei­ther. The ac­cu­sa­tion trav­els be­cause it feels right, not be­cause it is right. Big dif­fer­ence.

The ac­cu­sa­tion that some­one pro­duced un­ver­i­fied out­put is it­self be­ing pro­duced as un­ver­i­fied out­put. Or in other words.. The thing they ac­cuse vibecoders of is the very thing they are do­ing: No de­f­i­n­i­tion. No test. No fal­si­fi­ca­tion. Just a claim, shipped fast, never checked. The ac­cu­sa­tion is the real vibecoded con­tent. The ac­cuser is the vibecoder. They ac­cuse oth­ers of the very crime they are com­mit­ting them­selves, and most of them are un­aware of do­ing it.

The line in the sand

Their whole at­tack vec­tor is built around the premise of de­fend­ing a gate that does­n’t need de­fense, and a threat that does­n’t even ex­ist. I can prove it.

There are lev­els in this work. Level 1 is the typ­ing. Syntax, semi­colons, the years mem­o­riz­ing pointer arith­metic and which header file the func­tion lives in. Level 2 is the ver­i­fy­ing. The har­ness. The test suite. The re­flex of re­ject­ing the ninety at­tempts that al­most work and ship­ping the one that does. Level 3 is the de­cid­ing. What to build at all. Which ar­chi­tec­ture sur­vives con­tact with the real world. The three have never been the same thing. The gate was never at Level 1. The gate was at Levels 2 and 3, where the work that holds to­gether ac­tu­ally hap­pens.

AI low­ered the cost of Level 1. It did not touch Levels 2 or 3. The gate is ex­actly where it al­ways was.

The Level 2 and 3 peo­ple can see this. They look at SoulPlayer and ask about the har­ness. They look at an AI mu­sic video and ask about the tim­ing de­ci­sions or how con­sis­tency was en­forced. They are at the gate. They know where the gate is. They rec­og­nize when some­one else is also at the gate.

The ac­cusers can­not see this. They are not at the gate. They were at Level 1. Level 1 was their iden­tity, their hours, their proof of be­long­ing, their rea­son to feel at home in this pro­fes­sion. When AI made Level 1 cheap, it did not threaten the gate. It threat­ened them. Because they bet their self-worth on the layer that just got rented out.

So they call the work vibecoded. They have to. The new ar­ti­fact can­not be le­git­i­mate, be­cause if it is le­git­i­mate, then Level 1 was never the gate, and they were never at the gate ei­ther. Exactly, and I feel sorry for every­one who is solely op­er­at­ing at Level 1 and with noth­ing left to con­tribute. If AI re­vealed they were never do­ing the de­cid­ing, that they were never do­ing the ver­i­fy­ing, that they were the typ­ist all along, then they are in deep trou­ble right now.

And they are find­ing this out in pub­lic. By pan­ick­ing at the wrong things. By call­ing other peo­ple’s work vibecoded be­cause that is the only move left when the floor dis­ap­pears un­der you.

I do not feel an­gry at them. I feel sorry for them.

I will not use this ac­cu­sa­tion

I could. I have re­ceipts.

SoulPlayer has a ninety-test ver­i­fi­ca­tion har­ness de­vel­oped in Python that emits the C64 as­sem­bly and bi­nary. Four bit-iden­ti­cal ref­er­ence im­ple­men­ta­tions have to agree be­fore any­thing ships. I made six­teen AI mu­sic videos, weeks of work each, spent months cre­at­ing our own cus­tom in­fer­ence tool­chain in or­der to gain a max­i­mum of con­trol over the fi­nal frame. I have been the per­son you call when noth­ing else works, in rooms full of peo­ple who would never have called me on a good day. I know where the gate is. I have been at the gate since be­fore this pro­fes­sion called it a gate.

My demoscene back­ground alone would al­low me to turn around and call other peo­ple’s work vibecoded. I could punch down at every prompt-and-pray app on the in­ter­net and land a lot of hits.

I will not.

Because I know what the ac­cu­sa­tion costs the per­son on the re­ceiv­ing end. I have been the tar­get of “you don’t re­ally be­long here” my whole life. Neurodivergent, Tics. Freelancer-not-by-choice. Demoscene-not-industry. Artist-not-CS. Disabled. The ac­cu­sa­tion is the same shape every time. It lets the ac­cuser feel su­pe­rior with­out do­ing any work. It makes the tar­get spend their next week de­fend­ing them­selves in­stead of build­ing the next thing.

I am writ­ing this es­say right now in­stead of fin­ish­ing two other pro­jects. The ac­cu­sa­tion is al­ready win­ning, even though it has no ev­i­dence, no de­f­i­n­i­tion, no fal­si­fi­ca­tion, and no Photoshops to sup­port it. The ac­cu­sa­tion does­n’t have to be true. It just has to cost the tar­get enough time and morale that the next per­son stops shar­ing their work. The ac­cuser wants to harm and they suc­ceed usu­ally.

People are afraid to say they used AI. Not be­cause us­ing AI is shame­ful. Because the ac­cu­sa­tion is shame­ful, and the ac­cu­sa­tion is cheap to make and hard to re­fute. The shame econ­omy runs with­out any ac­tual shame­ful be­hav­ior in the sys­tem. It runs on fear. The fear of be­ing the next per­son tar­geted.

I will not feed that. I will not call work vibecoded just be­cause some­one used AI. I will not punch down on the prompt-and-pray crowd, even though they are wrong, be­cause the form of that punch is the same form that has been used against me my whole life. I rec­og­nize the move. I will not run it. And I will not duck away from the ac­cu­sa­tions ei­ther. I de­lib­er­ately chose to be trans­par­ent and vo­cal about this.

QED

So. Where are the vibecoded Photoshops? WHERE IS THE THREAT YOU MADE UP TO ATTACK ME?

I’m wait­ing. — gizmo

Credits

Carlo Rovelli is a the­o­ret­i­cal physi­cist known for his work on quan­tum grav­ity, the foun­da­tion of quan­tum me­chan­ics and the na­ture of space and time.

A fierce de­bate is rag­ing around the slip­pery no­tion of con­scious­ness. It re­traces a trot­ted pat­tern of cul­tural re­sis­tance: We hu­mans are of­ten scared by any­thing that may dis­turb our im­age of our­selves.

Famously, Darwin’s re­al­iza­tion that we have com­mon an­ces­tors with all liv­ing or­gan­isms on our planet met fe­ro­cious re­sis­tance. Many felt con­founded or de­graded by the idea of shar­ing a fam­ily tree with don­keys. The cul­tural his­tory of moder­nity is dot­ted by sim­i­lar ide­o­log­i­cal rear­guard bat­tles, wherein old world­views fight in re­treat against novel knowl­edge to save some con­cept held dear. Amid the cur­rent cul­tural back­lash against pro­gres­sive ideas, to­day’s de­bate on con­scious­ness re­flects our hu­man fears of be­long­ing to the same fam­ily as inan­i­mate mat­ter and los­ing our dear, tran­scen­dent souls.

During the Middle Ages, Western civ­i­liza­tion de­scribed hu­mans as com­posed of two dis­tinct en­ti­ties: body and soul. The body was an in­ter­con­nected bunch of mat­ter that de­cayed and died. The soul be­longed to a tran­scen­dent spir­i­tual world in­de­pen­dent from vile mat­ter. Angels were souls with­out a body and so were peo­ple af­ter their ma­te­r­ial death. The soul, taken to be im­mor­tal and cre­ated by God, was un­der­stood as the repos­i­tory of mem­o­ries, emo­tions and our sub­jec­tiv­ity. It could speak and fall in love. It was the agent of our agency; the sub­ject of our free­dom; the en­tity that bore re­spon­si­bil­ity, cul­pa­bil­ity, virtue and value; and de­served to be judged, saved or damned.

The cur­rent de­bate on con­scious­ness is in­flu­enced by our en­trenched tra­di­tional ideas of our­selves and by the long, slow ef­fort to up­date them with our new un­der­stand­ings of re­al­ity de­vel­oped over the last three cen­turies.

Despite the ar­ro­gant claims of those who say sci­ence can “explain every­thing,” most phe­nom­ena, from thun­der­storms to pro­tein fold­ing, es­cape our full un­der­stand­ing. We still can’t cure the flu or ac­cu­rately pre­dict the weather two weeks ahead. We do not know the ba­sic phys­i­cal laws of the uni­verse. And even where we are con­fi­dent that we know the ba­sic un­der­ly­ing nat­ural laws, we still can­not ac­count for what they im­ply. I am con­fi­dent that my bi­cy­cle dili­gently obeys the laws of par­ti­cle physics, yet those laws are use­less when it breaks down. To fix it, I ask a me­chanic, not a par­ti­cle physi­cist.

The func­tion­ing of our own body and brain is among the phe­nom­ena we un­der­stand the least and are cu­ri­ous about the most. This is the proper in­tel­lec­tual space where the “problem of con­scious­ness” is lo­cated. That is, con­scious­ness is hard to fig­ure out for pre­cisely the same rea­son thun­der­storms are: not be­cause we have ev­i­dence that it is not a nat­ural phe­nom­e­non, but be­cause it is a very com­pli­cated nat­ural phe­nom­e­non.

Updating the un­der­stand­ing of a phe­nom­e­non is not to deny it. Sunsets were un­der­stood in Antiquity and the Middle Ages as the de­scent of the sun in its daily mo­tion over the Earth. Today, we un­der­stand them as a re­sult of the Earth’s ro­ta­tion, which turns us to­ward its shady side, where the sun grad­u­ally be­comes no longer vis­i­ble. Such an up­date in un­der­stand­ing does not make sun­sets il­lu­sory or un­real.

Similarly, our soul won’t be­come il­lu­sory or un­real if we get a bet­ter sense of how our brain func­tions. We can still call our soul our “soul,” even if we un­der­stand our­selves bet­ter. I call it so, be­cause this no­tion — the soul — is dear to my soul.

The ‘Hard Problem Of Consciousness’

The con­scious­ness de­bate is of­ten for­mu­lated in terms used in an in­flu­en­tial talk given by a young David Chalmers in Tucson in 1994. Chalmers, a philoso­pher, dis­tin­guished two sep­a­rate “problems of con­scious­ness.” The first is the very hard prob­lem de­scribed above: un­der­stand­ing the processes in the brain that give rise to the many as­pects of our vis­i­ble be­hav­ior and our in­ner be­hav­ior that we can re­port about. Chalmers chris­tened this hard prob­lem as the “easy” prob­lem of con­scious­ness.

Then he de­clared that there is an­other dis­tinct prob­lem — why the brain’s be­hav­ior is ac­com­pa­nied by ex­pe­ri­ence at all — which he chris­tened the “hard” prob­lem of con­scious­ness. Today, this so-called “hard prob­lem” is men­tioned in all de­bates on con­scious­ness. Ac­cord­ing to many, it un­veils the very lim­its of cur­rent sci­en­tific un­der­stand­ing. Chalmers claimed that even af­ter hy­po­thet­i­cally ac­count­ing for our en­tire be­hav­ior, and for all our re­ports about our in­ner life, there would still be an “explanatory gap” be­tween brain processes and ex­pe­ri­ence.

“In the Renaissance, it was hard to ac­cept that heaven and Earth are of the same na­ture; af­ter Darwin, it was hard to ac­cept that an­i­mals and hu­mans are cousins; af­ter re­cent ad­vances in bi­ol­ogy, it is hard to ac­cept that liv­ing be­ings and inan­i­mate mat­ter are of the same na­ture.”

“In the Renaissance, it was hard to ac­cept that heaven and Earth are of the same na­ture; af­ter Darwin, it was hard to ac­cept that an­i­mals and hu­mans are cousins; af­ter re­cent ad­vances in bi­ol­ogy, it is hard to ac­cept that liv­ing be­ings and inan­i­mate mat­ter are of the same na­ture.”

The idea of this sup­posed “explanatory gap” rein­car­nates in a num­ber of re­lated forms: ex­plain­ing “qualia,” the hy­po­thet­i­cal el­e­men­tary bits of ex­pe­ri­ence; ex­plain­ing “subjectivity,” the very fact that some en­tity is ca­pa­ble of hav­ing ex­pe­ri­ence at all; or ex­plain­ing, as the philoso­pher Thomas Nagel fa­mously put it, “what is it like” to be the sub­ject of a cer­tain ex­pe­ri­ence.

I fail to make sense of the claim that there is such an “explanatory gap.” It re­gards what we would un­der­stand if we were to un­der­stand some­thing that we cur­rently do not un­der­stand. Forgive the mud­dled ques­tion, but: How can we know now what we would un­der­stand if we were to un­der­stand some­thing we do not cur­rently un­der­stand?

But this cu­ri­ous claim has been en­thu­si­as­ti­cally em­braced by crowds of thinkers, com­men­ta­tors and writ­ers across many fields and world­views, who have all jumped on the band­wagon of the “hard prob­lem.” This wide­spread em­brace is nour­ished by a stren­u­ous re­sis­tance to an idea an­tic­i­pated cen­turies ago by the philoso­pher Baruch Spinoza: that our soul could be a phe­nom­e­non of the same ba­sic na­ture as any other phe­nom­e­non in na­ture.

In the Renaissance, it was hard to ac­cept that heaven and Earth are of the same na­ture; af­ter Darwin, it was hard to ac­cept that an­i­mals and hu­mans are cousins; af­ter re­cent ad­vances in bi­ol­ogy, it is hard to ac­cept that liv­ing be­ings and inan­i­mate mat­ter are of the same na­ture.

The idea that we will never be able to un­der­stand con­scious­ness up­holds a world­view in which spirit and na­ture, sub­ject and ob­ject, form dis­tinct do­mains. Accepting that con­scious­ness may not be sep­a­rate from the phys­i­cal world — that our beloved soul could be of the same na­ture as our body and any other phe­nom­e­non of the world — is too much for many.

Seeing The World From Within It

Chalmers claims that ex­pe­ri­ence can­not be ac­counted for by sci­ence. But sci­en­tific un­der­stand­ing is not ex­tra­ne­ous to ex­pe­ri­ence; it is en­tirely about ex­pe­ri­ence. Empiricism, the ground­ing of knowl­edge in ex­pe­ri­ence, is not al­ter­na­tive to sci­ence; it is a main com­po­nent of sci­ence’s tra­di­tional con­cep­tual ground. As the Russian in­tel­lec­tual Alexander Bogdanov put it, sci­ence is the his­tor­i­cal process of a suc­cess­ful col­lec­tive or­ga­ni­za­tion of our ex­pe­ri­ence.

It is mis­lead­ing to see sci­ence, as of­ten naively por­trayed, as a di­rect ac­count of an ab­solute and ob­jec­tive world, ob­served and de­scribed from its out­side. If we think in this man­ner, we in­tro­duce du­al­ism. No sur­prise, then, that we find du­al­ism down the road: an ir­re­ducible gap be­tween sub­ject and ob­ject of knowl­edge. We have in­tro­duced it up­front.

What this view misses is the fact that we, sub­jects of knowl­edge and un­der­stand­ing, are not out­side the world. We are part of it. Our the­o­ries and knowl­edge are em­bod­ied tools to help us nav­i­gate the real world, not dis­em­bod­ied views on re­al­ity from the out­side. They are them­selves as­pects of the very world they de­scribe. Our un­der­stand­ing, like our feel­ings, per­cep­tions and ex­pe­ri­ence, is a nat­ural phe­nom­e­non. The source of the con­fu­sion about con­scious­ness is the ini­tial step: treat­ing knowl­edge, con­scious­ness and qualia as some­thing to be de­rived from a sci­en­tific pic­ture un­der­stood to be about some­thing else. In fact, the sci­en­tific pic­ture is a story about them.

Experience is not over and above the processes that hap­pen in the brain, as Chalmers as­sumed up­front. The du­al­ism be­tween a first-per­son de­scrip­tion of ex­pe­ri­ence and a third-per­son (or sci­en­tific) ac­count of the same is a nor­mal per­spec­ti­val dif­fer­ence: the same brain phe­nom­e­non as ex­pe­ri­enced by that same brain it­self, or by an­other. Ex­pe­ri­ence for both — not ev­i­dence of two dif­fer­ent kinds of re­al­ity.

“Subjective ex­pe­ri­ence,” “qualia” and “consciousness” are names of phe­nom­ena that of course ap­pear dif­fer­ently from dif­fer­ent per­spec­tives. It would be strange if they did­n’t. They af­fect the body and the brain em­body­ing them dif­fer­ently from how they af­fect some­thing in­ter­act­ing with them from the ex­te­rior. This is not due to a mys­te­ri­ous “explanatory gap.” “Red,” as a qualia, is the name of the process we gen­er­ally un­dergo when we see or re­mem­ber or think about the color red. We do not need to ex­plain why it looks red for the same rea­son that we do not have to ex­plain why the an­i­mal that we call “cat” looks like a cat. Why should we have to ex­plain why “red” looks red?

“The false ‘hard prob­lem of con­scious­ness’ as­sumes up­front that there ex­ists a meta­phys­i­cal gap be­tween mind and body. This con­tra­dicts every­thing we have learned about na­ture.”

“The false ‘hard prob­lem of con­scious­ness’ as­sumes up­front that there ex­ists a meta­phys­i­cal gap be­tween mind and body. This con­tra­dicts every­thing we have learned about na­ture.”

We do not have to de­rive a first-per­son per­spec­tive from an ob­jec­tive third-per­son view. It is the op­po­site: Any ac­count is per­spec­ti­val be­cause knowl­edge is al­ways em­bod­ied. Scientific knowl­edge is ul­ti­mately first-per­sonal. The world is real, but any ac­count of it can ex­ist only from within it. Any knowl­edge is per­spec­ti­val. Sub­jec­tiv­ity is not mys­te­ri­ous; it is just a spe­cial case of a per­spec­tive. What gen­er­ates the ap­par­ent “metaphysical gap” and “explanatory gap” is mis­tak­ing sci­en­tific pic­tures for di­rect ac­counts of an ul­ti­mate re­al­ity.

‘Philosophical Zombies’

Chalmers asks us to con­tem­plate what he calls a “philosophical zom­bie.” This is a hy­po­thet­i­cal en­tity that looks and be­haves like a hu­man in all re­spects, in­clud­ing re­port­ing emo­tions, feel­ings, dreams and ex­pe­ri­ence, yet it has no con­scious­ness. As Chalmers puts it, “There is no­body home.” This is a rhetor­i­cal trick that in­duces us to dis­tin­guish be­tween be­hav­ior and a hy­po­thet­i­cal re­al­ity ac­ces­si­ble only by in­tro­spec­tion. The very fact that a philo­soph­i­cal zom­bie could be con­ceived, Chalmers ar­gues, shows that in­ner ex­pe­ri­ence is in­trin­si­cally dis­tinct from ob­serv­able nat­ural phe­nom­ena.

But the ar­gu­ment is weak. A philo­soph­i­cal zom­bie would claim to know what sub­jec­tive ex­pe­ri­ence is; oth­er­wise, it would be em­pir­i­cally dis­tin­guish­able from a hu­man. Chalmer­s’s point is that the ex­is­tence of the hy­po­thet­i­cal, ir­re­ducible con­scious­ness of which he speaks is some­thing we can be con­vinced of only by in­tro­spec­tion. During in­tro­spec­tion, phys­i­cal processes in my brain con­vince me of my con­scious­ness. The same would the­o­ret­i­cally hap­pen in the zom­bie brain, con­vinc­ing it of hav­ing con­scious­ness as well. If this is true, can I be­lieve my own con­clu­sion of hav­ing this mys­te­ri­ous non-phys­i­cal ex­pe­ri­ence, know­ing that if I were a zom­bie, I would be con­vinced of the same with­out ac­tu­ally hav­ing it? The ar­gu­ment is self-de­feat­ing.

My hy­po­thet­i­cal, phys­i­cally iden­ti­cal zom­bie twin would be ex­actly like me — in­clud­ing in ex­pe­ri­ence. In other words, philo­soph­i­cal zom­bies are dis­tin­guish­able from or­di­nary peo­ple only by those who as­sume up­front what Chalmers seeks to prove: that there is some­thing non-phys­i­cal go­ing on in the world. They are not prov­ing any­thing; they are ex­am­ples of an un­con­vinc­ing meta­phys­i­cal pos­si­bil­ity and nos­tal­gia for the old no­tion of the tran­scen­dent soul.

The Soul Is Real & Is Part Of Nature

“Consciousness” and “experience” are names we use to de­note events that hap­pen in­side us, that make us. No ar­gu­ment con­tra­dicts the pos­si­bil­ity that what hap­pens can be equally de­scribed, us­ing other names, by a ca­pa­ble ex­ter­nal ob­server. Today, we do not have an ex­haus­tive ex­ter­nal ac­count, but this is not the same as hav­ing proof that no such ac­count is pos­si­ble.

The false “hard prob­lem of con­scious­ness” as­sumes up­front that there ex­ists a meta­phys­i­cal gap be­tween mind and body. But this con­tra­dicts every­thing we have learned about na­ture in the last cen­turies. The mind is the be­hav­ior of the brain, prop­erly de­scribed in a high-level lan­guage. Nei­ther my own ex­pe­ri­ence of my­self nor an ex­ter­nal ex­pe­ri­ence of me is pri­mary: They are two dis­tinct per­spec­tives on the same events. We do not need to as­sume that the cir­cle be­tween epis­te­mol­ogy (how we get knowl­edge) and on­tol­ogy (what ex­ists) re­quires a start­ing point. There is noth­ing wrong with its cir­cu­lar­ity: The world I ac­cess is the in­for­ma­tion I have about it, and I am part of that world.

Nor do we need to re­quire that there is any ul­ti­mate or fun­da­men­tal ac­count of re­al­ity. Any ac­count is ap­prox­i­mate, has blind spots and is re­al­ized within re­al­ity, so it is em­bod­ied in a part of that same re­al­ity. There are hinges be­tween a rep­re­sen­ta­tion and where it is em­bod­ied, and this may be a sin­gu­lar point in a rep­re­sen­ta­tion, but it is not a meta­phys­i­cal gap. It is not an ex­plana­tory gap.

So, there is no “hard prob­lem of con­scious­ness.” Our men­tal life can very well be of the same na­ture as any other phe­nom­e­non of the uni­verse. The more in­ter­est­ing chal­lenge is not to spec­u­late about a “hard prob­lem,” it is to try hard to un­der­stand more about the func­tion­ing of our brain and body with­out pos­tu­lat­ing that our soul is tran­scen­dent or dif­fer­ent in kind from the rest of na­ture.

We have souls. We have an in­ner self. We can treat our­selves as tran­scen­den­tal sub­jects in the Kantian sense. We have emo­tions and spir­i­tual life; we ex­pe­ri­ence qualia. These en­ti­ties are not ob­tained by ad­di­tion to a phys­i­cal state, but by sub­trac­tion from a com­plete phys­i­cal ac­count. Mental processes are phys­i­cal processes de­scribed in a way that cap­tures only their salient char­ac­ter­is­tics.

“It is time to give up the per­ni­cious du­al­ism in­tro­duced by the de­bate on con­scious­ness and em­brace the re­al­ity that our soul, or our spir­i­tual life, is con­sis­tent with our fun­da­men­tal physics.”

“It is time to give up the per­ni­cious du­al­ism in­tro­duced by the de­bate on con­scious­ness and em­brace the re­al­ity that our soul, or our spir­i­tual life, is con­sis­tent with our fun­da­men­tal physics.”

If we do not fall into the er­ror of du­al­ism up­front, we can safely speak of soul and emo­tions just as we speak of a kitchen table, even if the table is also a col­lec­tion of atoms. It is time to give up the per­ni­cious du­al­ism in­tro­duced by the de­bate on con­scious­ness and em­brace the re­al­ity that our soul, or our spir­i­tual life, is con­sis­tent with our fun­da­men­tal physics.

The rea­son why this pic­ture is more cred­i­ble than any du­al­ism is not that “science ex­plains every­thing” — it does­n’t — or be­cause “physics ex­plains every­thing” — it does so even less. It is be­cause of the hun­dreds of years of as­ton­ish­ing and un­ex­pected suc­cess of the sci­ences that have con­vinc­ingly shown that ap­par­ent meta­phys­i­cal gaps are never such.

Earth is not meta­phys­i­cally dif­fer­ent from the heav­ens, liv­ing be­ings are not meta­phys­i­cally dif­fer­ent from inan­i­mate mat­ter, hu­mans are not meta­phys­i­cally dif­fer­ent from other an­i­mals. The soul is not meta­phys­i­cally dif­fer­ent from the body. We are all parts of na­ture, like any­thing else in this sweet world.

Track: Main Track

Room: Janson

Day: Saturday

Start (UTC+1): 12:00

End (UTC+1): 12:50

Chat: Join the con­ver­sa­tion!

Mercurial is a Distributed Version Control System cre­ated in 2005.

The pro­ject has been con­stantly ac­tive since then, fos­ter­ing mod­ern tool­ing, in­tro­duc­ing new ideas, spawn­ing mul­ti­ple re­cent tools from its com­mu­nity, keep­ing it­self com­pet­i­tive, and with sus­tained fund­ing for its de­vel­op­ment. However nowa­days, most peo­ple we en­counter re­mem­ber Mercurial for los­ing the pop­u­lar­ity bat­tle to its sib­ling Git in the 2010s and think the pro­ject dead.

This talk con­fronts this para­dox. How did Mercurial get it­self in such a sit­u­a­tion? What can every­one learn from it? What does this mean for the fu­ture of ver­sion con­trol?

Using our first hand knowl­edge of Mercurial’s his­tory, we look at a se­lec­tion of events, con­trib­u­tor pro­files, tech­ni­cal and com­mu­nity as­pects, to see how they’ve af­fected the pro­jec­t’s course.

We will fo­cus on top­ics that we have been asked about most fre­quently, such as: * How has Mercurial weath­ered the Git storm? * Which im­pacts has Mercurial had on your life, un­be­knownst to you? * How has the in­volve­ment from be­he­moth com­pa­nies re­shaped the pro­ject? * What brings peo­ple to Mercurial in 2025?

Finally, we lever­age the knowl­edge ex­tracted from our past, to as­sess the pre­sent state of ver­sion con­trol, try to pre­dict its fu­ture, and high­light how com­mu­nity-based open-source re­mains as rel­e­vant as ever.

Speakers

Links

Video record­ing (AV1/WebM; pre­ferred) - 110.9 MB

Video record­ing (MP4; for legacy sys­tems) - 1.1 GB

Video record­ing sub­ti­tle file (VTT)

Chat room(web)

Chat room(app)

Submit Feedback

Good news, every­one! jank has a new cus­tom in­ter­me­di­ate rep­re­sen­ta­tion (IR) and we’re us­ing it to op­ti­mize jank to com­pete with the JVM. We’ll dive into more of that to­day, but first I want to say thank you to my Github spon­sors and to Clojurists Together for spon­sor­ing me this whole year. You all are help­ing a great deal. I am still search­ing for a way to con­tinue work­ing on jank full-time with an in­come which will cover rent and gro­ceries, so if you’ve not yet chipped in a spon­sor­ship, now’s a great time!

What is an in­ter­me­di­ate rep­re­sen­ta­tion (IR)?

Compilers of­ten rep­re­sent pro­grams as a more ab­stract set of in­struc­tions than a tar­get CPU in­struc­tion set can af­ford. This has a few added ben­e­fits. Firstly, the pro­gram can be rep­re­sented in a way which could later be low­ered to dif­fer­ent CPU ar­chi­tec­tures, such as x86_64 or ar­m64. Since in­ter­me­di­ate rep­re­sen­ta­tions are of­ten higher level than CPU ar­chi­tec­tures, they can gen­er­ally be more portable. Secondly, IRs can be specif­i­cally de­signed to rep­re­sent the pro­gram in a way which makes writ­ing cer­tain op­ti­miza­tions eas­ier, such as sin­gle sta­tic as­sig­ment (SSA) form. Finally, IR de­sign­ers get to choose the level of ab­strac­tion of the IR to match the se­man­tics they’re aim­ing to rep­re­sent, which can ei­ther make an IR more gen­eral or more spe­cific to a cer­tain lan­guage.

There are many com­mon pop­u­lar IRs, such as the JVM’s byte­code, the CLR’s com­mon in­ter­me­di­ate lan­guage (CIL), GCC’s GIMPLE, LLVM’s IR, and so on. Some com­pil­ers may move the pro­gram through mul­ti­ple IRs dur­ing com­pi­la­tion.

Custom IR ra­tio­nale

Historically, jank has not been an op­ti­miz­ing com­piler. We’ve del­e­gated ba­si­cally all of that work to LLVM, based on the C++ or LLVM IR which we would gen­er­ate. However, LLVM IR works at a very low level, com­pared to Clojure. It has no con­cept of Clojure’s vars, tran­sients, per­sis­tent data struc­tures, lazy se­quences, and so on. Clojure’s dy­namism is granted by a great deal of both poly­mor­phism and in­di­rec­tion, but this means LLVM has very few op­ti­miza­tion op­por­tu­ni­ties when it’s deal­ing with the LLVM IR from jank.

The op­ti­miza­tion work done pre­vi­ously on jank helped op­ti­mize its run­time, and the com­piler it­self, but less so the code be­ing com­piled by the com­piler. In the past two months, I have sought to change this.

I wanted an IR which op­er­ated at the level of Clojure’s se­man­tics. This would be much higher level than LLVM IR and even much higher level than JVM’s byte­code. Since I’m not build­ing a gen­eral vir­tual ma­chine (VM) or com­piler plat­form, I don’t need to gen­er­al­ize the IR for dif­fer­ent lan­guages. I can make jank’s IR specif­i­cally tai­lored to jank, which gives us even more power for op­ti­miza­tions. As far as I know, no Clojure di­alects have taken this step.

Custom IR de­tails

I have writ­ten a ref­er­ence for jank’s IR in the jank book here. This ref­er­ence is tar­geted at peo­ple who’re work­ing on jank it­self, since I’m mak­ing no promises on the sta­bil­ity of jank’s IR at this point. However, I will copy some of that here to il­lus­trate jank’s IR and help pro­vide a men­tal model for what’s to come. Let’s ex­am­ine this sim­ple Clojure func­tion.

(defn greet [name] (if (= “jeaye” name) (println “Are you me?!“) (println (str “Hello, ” name ”!“))))

jank’s IR is stored in mem­ory as C++ data struc­tures, but it is ren­der­able to Clojure data for de­bug­ging and test­ing. This is not full se­ri­al­iza­tion, since it can­not round-trip back into the jank com­piler from the IR, due to all of the Clang AST in­ter­nal data we have on hand. Let’s take a look at the jank IR mod­ule for this func­tion.

{:name user_­greet_82687 :lifted-vars {clojure_core_SLASH_str_82694 clo­jure.core/​str clo­jure_­core_S­LASH_print­l­n_82691 clo­jure.core/​println clo­jure_­core_S­LASH__E­Q__82689 clo­jure.core/=} :lifted-constants {const_82693 ”!” con­st_82692 “Hello, ” con­st_82690 “Are you me?!” con­st_82688 “jeaye”} :functions [{:name user_­greet_82687_1 :blocks [{:name en­try :instructions [{:name greet :op :parameter :type “jank::runtime::object_ref”} {:name name :op :parameter :type “jank::runtime::object_ref”} {:name v3 :op :literal :value “jeaye” :type “jank::runtime::obj::persistent_string_ref”} {:name v4 :op :var-deref :var clo­jure_­core_S­LASH__E­Q__82689 :type “jank::runtime::object_ref”} {:name v5 :op :dynamic-call :fn v4 :args [v3 name] :type “jank::runtime::object_ref”} {:name v7 :op :truthy :value v5 :type “bool”} {:name v8 :op :branch :condition v7 :then if0 :else else1 :merge nil :shadow nil :type “void”}]} {:name if0 :instructions [{:name v9 :op :literal :value “Are you me?!” :type “jank::runtime::obj::persistent_string_ref”} {:name v10 :op :var-deref :var clo­jure_­core_S­LASH_print­l­n_82691 :type “jank::runtime::object_ref”} {:name v11 :op :dynamic-call :fn v10 :args [v9] :type “jank::runtime::object_ref”} {:name v12 :op :ret :value v11 :type “jank::runtime::object_ref”}]} {:name else1 :instructions [{:name v13 :op :literal :value “Hello, ” :type “jank::runtime::obj::persistent_string_ref”} {:name v14 :op :literal :value ”!” :type “jank::runtime::obj::persistent_string_ref”} {:name v15 :op :var-deref :var clo­jure_­core_S­LASH_str_82694 :type “jank::runtime::object_ref”} {:name v16 :op :dynamic-call :fn v15 :args [v13 name v14] :type “jank::runtime::object_ref”} {:name v17 :op :var-deref :var clo­jure_­core_S­LASH_print­l­n_82691 :type “jank::runtime::object_ref”} {:name v18 :op :dynamic-call :fn v17 :args [v16] :type “jank::runtime::object_ref”} {:name v19 :op :ret :value v18 :type “jank::runtime::object_ref”}]}]}]}

jank’s IR is SSA-based, mean­ing that each name is only as­signed once. This makes en­tire cat­e­gories of op­ti­miza­tions much eas­ier to rea­son about. jank’s IR is also rep­re­sented as a con­trol flow graph (CFG), which is com­posed of one or more ba­sic blocks, each with ex­actly one ter­mi­nat­ing in­struc­tion (branch, jump, throw, ret, etc).

As we can see from the IR mod­ule, jank han­dles the lift­ing of vars and con­stants, and it has in­struc­tions at the level of Clojure’s se­man­tics, for deref­er­enc­ing vars, call­ing func­tions, and so on. Let’s take a look at the gen­er­ated C++ from this IR.

ex­tern “C” jank::run­time::ob­jec­t_ref user_­greet_19_1(jank::run­time::ob­jec­t_ref const greet, jank::run­time::ob­jec­t_ref name) { auto const v3(con­st_33); auto const v4(clo­jure_­core_S­LASH__E­Q__34->deref()); auto const v5(jank::run­time::dy­nam­ic_­call(v4, v3, name)); auto const v7(jank::run­time::truthy(v5)); if(v7) { auto const v9(con­st_35); auto const v10(clo­jure_­core_S­LASH_print­l­n_36->deref()); auto const v11(jank::run­time::dy­nam­ic_­call(v10, v9)); re­turn v11; } else { auto const v13(con­st_37); auto const v14(con­st_38); auto const v15(clo­jure_­core_S­LASH_str_39->deref()); auto const v16(jank::run­time::dy­nam­ic_­call(v15, v13, name, v14)); auto const v17(clo­jure_­core_S­LASH_print­l­n_36->deref()); auto const v18(jank::run­time::dy­nam­ic_­call(v17, v16)); re­turn v18; } }

If you com­pare the C++ to the IR, you can im­me­di­ately see the cor­re­la­tion. The C++ vari­ables are named to match the IR vari­ables. A var deref­er­ence just be­comes a call to ->deref() on the var. A dy­namic call just be­comes a jank::run­time::dy­nam­ic_­call. This is in­ten­tional.

Optimizing the IR

It took about six weeks to de­sign and im­ple­ment the IR, in­clud­ing re­work­ing our C++ code gen­er­a­tion to gen­er­ate from the IR in­stead of from jank’s AST. At this point, we’re not yet run­ning any op­ti­miza­tion passes on the IR. However, we have every­thing we need to start do­ing that. I wanted to pri­or­i­tize get­ting the new IR pipeline merged, rather than build­ing it out as much as pos­si­ble, since six weeks is al­ready a long time to be branched from main. Now that the IR is merged in, my ap­proach will be to pick up one bench­mark at a time and op­ti­mize it as needed un­til I’m satis­ti­fied and/​or can­not op­ti­mize it any fur­ther. Some of those op­ti­miza­tions will in­volve the IR di­rectly, while oth­ers will not.

If you’re in­ter­ested in more of the tech­ni­cal de­vel­op­ment side of this IR, there are a few videos on the jank TV YouTube chan­nel from the var­i­ous Twitch streams I did while work­ing on the IR. These videos go way into the weeds of im­ple­ment­ing things.

With the new IR in­tro­duced, let’s jump into op­ti­miz­ing our first bench­mark: re­cur­sive fi­bonacci.

Interlude

Before we pro­ceed, please con­sider sub­scrib­ing to jank’s mail­ing list. This is go­ing to be the best way to make sure you stay up to date with jank’s re­leases, jank-re­lated talks, work­shops, and so on. It’s very low traf­fic.

Optimizing re­cur­sive fi­bonacci

Our first bench­mark for this round of op­ti­miza­tion is a re­cur­sive fi­bonacci im­ple­men­ta­tion. It’s just five lines long. Our goal is to have jank be at least as fast as Clojure JVM, if not faster, but we have to earn it.

(defn fi­bonacci [n] (if (<= n 1) n (+ (fibonacci (- n 1)) (fibonacci (- n 2)))))

This may seem like a triv­ial bench­mark to op­ti­mize. You might won­der why would this be rep­re­sen­ta­tive of a real world ap­pli­ca­tion. In re­al­ity, this bench­mark cov­ers some es­sen­tial as­pects of the com­piler and run­time.

Polymorphic arith­metic and re­la­tional pred­i­cates. Basically every pro­gram crunches num­bers and needs to do so quickly.

Recursion. Many pop­u­lar al­go­rithms, es­pe­cially in Lisps, are re­cur­sive. Being able to han­dle these pat­terns ef­fi­ciently is im­por­tant.

Garbage cre­ation and col­lec­tion. The trash truck may come every week, but that does­n’t mean we should be gen­er­at­ing as much garbage as we can.

In gen­eral, the abil­ity for the lan­guage run­time to get out of the way. If we’re try­ing to cal­cu­late fi­bonacci num­bers, we don’t want any­thing show­ing up in the pro­filer which is un­re­lated to fi­bonacci num­bers.

As we op­ti­mize, through­out this post, con­sider these four cat­e­gories and how each op­ti­miza­tion we do can be cat­e­go­rized.

Baseline fi­bonacci tim­ing

We’re go­ing to use Clojure JVM to get our base­line bench­mark num­bers and then we’ll aim to beat those num­bers with jank.

Note that all num­bers in this post are mea­sured on my five year old x86_64 desk­top with an AMD Ryzen Threadripper 2950X on NixOS with OpenJDK 21. When I say “JVM” in this post, I mean OpenJDK 21.

❯ clo­jure -Sdeps ‘{:deps {criterium/criterium {:mvn/version “0.4.6”}}}’ Clojure 1.12.4 user=> (require ’[criterium.core :refer [quick-bench]]) nil

user=> (defn fi­bonacci [n] (if (<= n 1) n (+ (fibonacci (- n 1)) (fibonacci (- n 2))))) #’user/fibonacci

user=> (quick-bench (fibonacci 35))

Clojure takes about 200 mil­lisec­onds to cal­cu­late (fibonacci 35). This is our base­line!

Be care­ful of lein repl

Note that I orig­i­nally did my bench­mark­ing for Clojure in lein repl, which gives in­cred­i­bly dif­fer­ent re­sults. On my sys­tem, in­stead of 200 mil­lisec­onds, Clojure clocks in around 2,800 mil­lisec­onds in­stead! There are some notes here stat­ing that lein repl dis­ables some JVM op­ti­miza­tions which ap­par­ently play a key role here. Thank you to Kyle Cesare for point­ing this out.

Initial jank tim­ing

Starting from jank’s main a few weeks ago, we’re go­ing to use the same fi­bonacci de­f­i­n­i­tion, but we don’t have cri­terium, since that’s a li­brary for the JVM. Instead, jank has its own bench­mark­ing li­brary, shipped along with jank it­self.

(defn fi­bonacci [n] (if (<= n 1) n (+ (fibonacci (- n 1)) (fibonacci (- n 2)))))

(require ’[jank.perf]) (jank.perf/​bench­mark {:label “fib”} (fibonacci 35))

If we run this with op­ti­miza­tions en­abled and ea­ger com­pi­la­tion, we can get our ini­tial num­bers.

❯ jank run -O3 –eagerness ea­ger fib.jank

jank clocks in at 5,522 mil­lisec­onds. That’s… not fast. Not com­pared to the JVM’s 200 mil­lisec­onds.

Inlining arith­metic

To kick things off, I know that Clojure is in­lin­ing math calls and jank used to have a hacky so­lu­tion to that which was re­moved. It’s time to do this prop­erly. Clojure han­dles in­lin­ing via meta­data, since the body of a func­tion from an­other name­space is not avail­able. This is not specif­i­cally a Clojure prob­lem, since it’s also ex­actly how C and C++ work. Calls to C or C++ func­tions across trans­la­tion units won’t be in­lined un­less link time op­ti­miza­tions (LTO) are used. The only other op­tion is to move the de­f­i­n­i­tion into a header file and mark the func­tion in­line, so that every trans­la­tion unit has its own copy. In Clojure, we can achieve the same ef­fect of “putting the func­tion in a header” by chang­ing the meta­data of the var to in­clude in­lin­ing in­for­ma­tion, since any­one can read var meta­data from any­where. Let’s see an ex­am­ple of this.

(defn ^{:inline (fn [l r] (list ’cpp/jank.runtime.max l r)) :inline-arities #{2}} max ([x] x) ([l r] (cpp/jank.runtime.max l r)) ([l r & args] (let [res (cpp/jank.runtime.max l r)] (if (empty? args) res (recur res (first args) (next args))))))

Here, we have clo­jure.core/​max, which de­fines some meta­data con­tain­ing two keys: :inline and :inline-arities. The lat­ter is a set of ar­i­ties to in­line. Here, we only care about the [l r] ar­ity. The value of :inline is an ac­tual func­tion to call to get the body for that ar­ity. In the case of max, we just want to in­line a C++ call to jank::run­time::max. Later on, we’ll tell Clang to even in­line that call.

The in­lin­ing is done dur­ing analy­sis, rather than in an IR pass. When we find a func­tion call through a var, we check the var’s meta­data and call the cor­re­spond­ing :inline func­tion if pre­sent. You can think of this as a sis­ter to macro ex­pan­sion, since it works very sim­i­larly.

This style of in­lin­ing has some huge ben­e­fits. Firstly, we get to re­move the var in­tern­ing and deref­er­ence of clo­jure.core/​max. Secondly, since every Clojure func­tion needs boxed pa­ra­me­ters, if we’re work­ing with na­tive val­ues, we don’t need to box them be­fore call­ing max. Thirdly, if max re­turns an un­boxed na­tive value, we don’t need to box it to re­turn from the func­tion. This al­lows us to avoid box­ing and bet­ter prop­a­gate type in­for­ma­tion.

After adding in­line sup­port to jank’s an­a­lyzer and up­dat­ing the meta­data for all arith­metic func­tions, we can check our new bench­mark re­sults. This drops us from 5,522 mil­lisec­onds to 2,309 mil­lisec­onds. It’s nice to start with a huge win.

Eliminating ex­tra IR in­struc­tions

Next, let’s take a look at the IR for our fi­bonacci func­tion. I love in­spect­ing the IR for jank func­tions now, since is pro­vides such a nice view into how the jank com­piler sees the code.

{:name user_­fi­bonac­ci_82580 :lifted-vars {} :lifted-constants {const_82598 2 con­st_82597 1} :functions [{:name user_­fi­bonac­ci_82580_1 :blocks [{:name en­try :instructions [{:name fi­bonacci :op :parameter :type “jank::runtime::object_ref”} {:name n :op :parameter :type “jank::runtime::object_ref”} {:name v3 :op :literal :value 1 :type “jank::runtime::obj::integer_ref”} {:name v4 :op :cpp/call :value “jank::runtime::lte” :args [n v3] :type “bool”} {:name v5 :op :cpp/into-object :value v4 :type “jank::runtime::object_ref”} {:name v7 :op :truthy :value v5 :type “bool”} {:name v8 :op :branch :condition v7 :then if0 :else else1 :merge nil :shadow nil :type “void”}]} {:name if0 :instructions [{:name v9 :op :ret :value n :type “jank::runtime::object_ref”}]} {:name else1 :instructions [{:name v10 :op :literal :value 1 :type “jank::runtime::obj::integer_ref”} {:name v11 :op :cpp/call :value “jank::runtime::sub” :args [n v10] :type “jank::runtime::object_ref”} {:name v12 :op :named-recursion :fn fi­bonacci :args [v11] :type “jank::runtime::object_ref”} {:name v13 :op :literal :value 2 :type “jank::runtime::obj::integer_ref”} {:name v14 :op :cpp/call :value “jank::runtime::sub” :args [n v13] :type “jank::runtime::object_ref”} {:name v15 :op :named-recursion :fn fi­bonacci :args [v14] :type “jank::runtime::object_ref”} {:name v16 :op :cpp/call :value “jank::runtime::add” :args [v12 v15] :type “jank::runtime::object_ref”} {:name v17 :op :ret :value v16 :type “jank::runtime::object_ref”}]}]}]}

So we have three blocks in our func­tion. We start at the en­try block. We grab our pa­ra­me­ter n and the lit­eral 1 and we do our <= check, which has been in­lined as a cpp/​call in­struc­tion which re­turns bool.

{:name n :op :parameter :type “jank::runtime::object_ref”} {:name v3 :op :literal :value 1 :type “jank::runtime::obj::integer_ref”} {:name v4 :op :cpp/call :value “jank::runtime::lte” :args [n v3] :type “bool”}

We then con­vert that bool into a boxed ob­ject and check if it’s truthy so we can branch.

{:name v5 :op :cpp/into-object :value v4 :type “jank::runtime::object_ref”} {:name v7 :op :truthy :value v5 :type “bool”} {:name v8 :op :branch :condition v7 :then if0 :else else1 :merge nil :shadow nil :type “void”}

This can be op­ti­mized away, since the re­sult of our <= check (v4) was al­ready a bool. We turned it into an ob­ject just so we can turn it back into a bool. Ideally, the branch con­di­tion can just be v4 in­stead.

Before op­ti­miz­ing that, let’s fin­ish ex­am­in­ing our IR. We ei­ther branch to if0, which just re­turns our re­sult, or to else1, which then needs to do the re­cur­sion. Our else1 branch hap­pens in three steps.

; 1. Recur with `(- n 1)`. {:name v10 :op :literal :value 1 :type “jank::runtime::obj::integer_ref”} {:name v11 :op :cpp/call :value “jank::runtime::sub” :args [n v10] :type “jank::runtime::object_ref”} {:name v12 :op :named-recursion :fn fi­bonacci :args [v11] :type “jank::runtime::object_ref”}

; 2. Recur with `(- n 2)`. {:name v13 :op :literal :value 2 :type “jank::runtime::obj::integer_ref”} {:name v14 :op :cpp/call :value “jank::runtime::sub” :args [n v13] :type “jank::runtime::object_ref”} {:name v15 :op :named-recursion :fn fi­bonacci :args [v14] :type “jank::runtime::object_ref”}

; 3. Return the sum of those. {:name v16 :op :cpp/call :value “jank::runtime::add” :args [v12 v15] :type “jank::runtime::object_ref”} {:name v17 :op :ret :value v16 :type “jank::runtime::object_ref”}

That’s every­thing! So let’s elim­i­nate those ex­tra :cpp/into-object and :truthy in­struc­tions and add sup­port to our IR gen­er­a­tion for just us­ing bool val­ues di­rectly. The re­sults are that we drop from 2,309 mil­lisec­onds to 2,247 mil­lisec­onds. That’s quite mar­ginal, given our over­all mag­ni­tude. It’s nice that the IR no longer has any ex­tra­ne­ous work in it, though. We could lift those two :literal in­struc­tions for 1 so there’s just one of them, but that’s not go­ing to af­fect our per­for­mance, so I’ll leave it for now.

{:name en­try :instructions [{:name fi­bonacci :op :parameter :type “jank::runtime::object_ref”} {:name n :op :parameter :type “jank::runtime::object_ref”} {:name v3 :op :literal :value 1 :type “jank::runtime::obj::integer_ref”} {:name v4 :op :cpp/call :value “jank::runtime::lte” :args [n v3] :type “bool”} {:name v6 :op :branch :condition v4 :then if0 :else else1 :merge nil :shadow nil :type “void”}]} {:name if0 :instructions [{:name v7 :op :ret :value n :type “jank::runtime::object_ref”}]} {:name else1 :instructions [{:name v8 :op :literal :value 1 :type “jank::runtime::obj::integer_ref”} {:name v9 :op :cpp/call :value “jank::runtime::sub” :args [n v8] :type “jank::runtime::object_ref”} {:name v10 :op :named-recursion :fn fi­bonacci :args [v9] :type “jank::runtime::object_ref”} {:name v11 :op :literal :value 2 :type “jank::runtime::obj::integer_ref”} {:name v12 :op :cpp/call :value “jank::runtime::sub” :args [n v11] :type “jank::runtime::object_ref”} {:name v13 :op :named-recursion :fn fi­bonacci :args [v12] :type “jank::runtime::object_ref”} {:name v14 :op :cpp/call :value “jank::runtime::add” :args [v10 v13] :type “jank::runtime::object_ref”} {:name v15 :op :ret :value v14 :type “jank::runtime::object_ref”}]}

Optimizing nil us­age

At this point, the IR looks good and I have no other planned op­ti­miza­tions, so let’s take a look at a flame­graph to see where our time is be­ing spent. You can click this im­age to open it in a new tab and see the de­tails, if you’d like. I’ll cover the es­sen­tials here any­way.

We’re ex­pect­ing to see arith­metic, and calls to fi­bonacci, but cu­ri­ously we see a LOT of time spent in jank_nil and jank_­con­st_nil. As a Clojure di­alect, we ac­cess nil very of­ten, since we need to check if things are nil, ini­tial­ize things to nil, and lots of ex­pres­sions eval­u­ate to nil. However, that should­n’t show up in the pro­filer! It does be­cause jank cur­rently puts its nil value be­hind a func­tion for some very in-the-weeds rea­sons. C++ does­n’t guar­an­tee ini­tial­iza­tion or­der for glob­als across trans­la­tion units and any trans­la­tion units jank AOT com­piles will be full of glob­als (lifted con­stants) which will want to ini­tial­ize their val­ues to be nil. If nil is de­fined as a global in an­other trans­la­tion unit, as part of the jank run­time, we might try to use it be­fore it’s ini­tial­ized. So we’ve been work­ing around that by putting nil be­hind a func­tion called jank_nil, but ap­par­ently this has had a heavy per­for­mance cost.

jank’s boxed pointer type does­n’t al­low ini­tial­iza­tion with nullptr, since that’s not a valid value. jank sep­a­rates nil from nullptr since deref­er­enc­ing nil is well-de­fined, in Clojure, but deref­er­enc­ing nullptr is un­de­fined be­hav­ior in C++. We sim­ply just can’t do it. But I hap­pen to know that we don’t care about the de­fault con­struc­tion of the glob­als we gen­er­ate for AOT com­piled code, since we later re-ini­tial­ize them when the mod­ule is loaded. So let’s add a cus­tom con­struc­tor to our boxed pointer type which ac­tu­ally ini­tial­izes them to nullptr be­cause we know we’ll be re-ini­tial­iz­ing them with nil later. Then we can just keep jank_nil as a global value in­stead of a func­tion.

This brings us from 2,247 mil­lisec­onds to 1,400 mil­lisec­onds!

That’s a big win! This qual­i­fies as the run­time get­ting in the way of our bench­mark, so it’s nice to clear that up. We’re still over 5x slower than Clojure JVM, though, so it’s time to cut off some more huge chunks.

Why are add/​sub slow?

If we look at the rest of the flame­graph em­bed­ded above, we see the ex­pected sus­pects. Namely, add and sub. The slow­est parts of those func­tions, the flame­graph tells us, is the GC al­lo­ca­tion of new num­bers. Every sin­gle in­te­ger re­sult from adding or sub­tract­ing is a new dy­namic al­lo­ca­tion. At this point, we’re spend­ing ba­si­cally our en­tire time just al­lo­cat­ing num­bers.

You might won­der why we’re not us­ing un­boxed num­bers for this. Or you might think “Just add a type hint! Clojure sup­ports those!”. But that misses the point of this bench­mark. This bench­mark is specif­i­cally writ­ten to chal­lenge the com­piler and run­time. Let’s take a look at our Clojure code one more time and I’ll ex­plain why.

(defn fi­bonacci [n] (if (<= n 1) n (+ (fibonacci (- n 1)) (fibonacci (- n 2)))))

Here, the type of n is just go­ing to be a type-erased ob­ject. In Clojure JVM, it will be a Java Object. In jank, it’s a jank::run­time::ob­jec­t_ref. Either way, we have no idea what kind of ob­ject is in there. When we do (- n 1) and (- n 2), we still don’t know the type of n. It could be a float. It could be an in­te­ger. It could be a ra­tio. It could be a big dec­i­mal or big in­te­ger. It could be some­thing which does­n’t even sup­port arith­metic. So we need to do a whole poly­mor­phic dance to han­dle arith­metic on n. Fortunately, we know the types of 1 and 2, so we can op­ti­mize for that, but it’s not enough to un­box any of this arith­metic. The same thing ap­plies to the + call. We don’t know the re­turn type of fi­bonacci. We ei­ther re­turn n, which we don’t know the type of, or we re­turn the re­sult of + with both in­puts be­ing the re­turn from fi­bonacci, which we still don’t know the type of. So there’s noth­ing we can do here sta­t­i­cally ei­ther. This is the key as­pect that makes this bench­mark dif­fi­cult. The JVM is do­ing a much bet­ter job of it than we are, cur­rently.

Pointer tag­ging

To rem­edy this, let’s just avoid dy­namic al­lo­ca­tions for in­te­gers en­tirely. There are some well-known tricks used by lan­guage run­times to do ex­actly that and jank is­n’t yet us­ing any of them. We’ll start with the sim­plest trick and then build to a more com­plex de­sign in a later bench­mark post.

Did you know that, on 64 bit sytems, the bot­tom three bits of point­ers are ef­fec­tively un­used? This is be­cause point­ers are aligned to 64 bit ma­chine words. For ex­am­ple, an aligned pointer ex­ists at ad­dress 0, 8, 16, 24, and so on. But an aligned pointer will not ex­ist at an ad­dress which is not di­vis­i­ble by 8 bytes (64 bits). The bot­tom three bits of a pointer, 000, are for the 1′s place (lowest bit), 2′s place (middle bit), and 4′s place (highest bit). If all bits are on, 111, we have the value 7 (4 + 2 + 1). If you add one more, we get 8 and all three low­est bits go back to 0.

This is an in­cred­i­ble piece of knowl­edge, since it means that we can em­bed ex­tra in­for­ma­tion in our point­ers very eas­ily. For ex­am­ple, if we set the low­est bit to 1, we can con­vey that the pointer is not ac­tu­ally a pointer. Instead, it’s an en­coded in­te­ger. We can then use the other 63 bits to store the ac­tual in­te­ger value. This way, as long as our in­te­gers don’t need to store val­ues higher than what 63 bits can man­age, we can store them in­line with­out need­ing to do a dy­namic al­lo­ca­tion! In the case where we need all 64 bits, we can just do an al­lo­ca­tion like we nor­mally would and we’d get a nor­mal pointer (lowest three bits all 0) to a boxed in­te­ger.

To vi­su­al­ize this, a nor­mal 64 bit pointer would look like this. The high­est 61 bits are used for pointer data and the low­est three bits are 0.

pppppppp pppppppp pppppppp pppppppp pppppppp pppppppp pppppppp pppp­p000

An en­coded in­te­ger would look like this. The low­est bit is 1 and the rest is re­served for in­te­ger data. To get the ac­tual 63 bit in­te­ger, we just shift the whole thing to the right once.

xxxxxxxx xxxxxxxx xxxxxxxx xxxxxxxx xxxxxxxx xxxxxxxx xxxxxxxx xxxxxxx1

For our fi­bonacci bench­mark, and in­deed for ba­si­cally every bench­mark and real-world ap­pli­ca­tion, this is go­ing to ef­fec­tively elim­i­nate every dy­namic al­lo­ca­tion for in­te­gers. Let’s see how it af­fects our num­bers.

With tagged point­ers to op­ti­mize for 63 bit in­te­gers, jank goes from 1,400 mil­lisec­onds to 282 mil­lisec­onds. As I said, we were ba­si­cally spend­ing all of our time al­lo­cat­ing in­te­gers. Even bet­ter, this puts us within reach of Clojure’s 200 mil­lisec­onds.

Intense in­lin­ing

Let’s take a look at a new flame­graph with our lat­est changes and see how we can trim off the ex­tra fat. Again, you can click this im­age to open it in a new tab, if you’d like. Or just take my word for it.

Ideally, what we see here is just fi­bonacci and noth­ing more, since every­thing that mat­ters should have been in­lined. Instead, we see jank::run­time::add, jank::run­time::sub, and jank::run­time::lte. This means those calls have not been in­lined by Clang. Let’s try to tell Clang to al­ways in­line our arith­metic func­tions. We can do this by putting some at­trib­utes on the C++ add, sub, etc func­tions. Modern C++ syn­tax makes this easy.

tem­plate <typename L, type­name R> [[gnu::always_inline, gnu::flat­ten, gnu::hot]] auto add(L const l, R const r) { … }

Arithmetic is some­thing we al­ways want to be fast, so there’s no bet­ter thing to in­line than num­ber crunch­ing. Now we can try again. Fortunately, with a sigh of re­lief, this drops jank from 282 mil­lisec­onds to 114 mil­lisec­onds. We’re nearly twice as fast as Clojure JVM! Better yet, we’re gone from tak­ing 5,522 mil­lisec­onds to tak­ing 114 mil­lisec­onds and we’re ef­fec­tively do­ing the same work.

I once saw an in­ter­view with an artist who does chain­saw sculp­tures. The in­ter­viewer asked him how he ap­proaches sculpt­ing some­thing like a bear out of a huge log. He said “I just re­move every­thing that does­n’t look like a bear”. While that’s not a par­tic­u­larly help­ful an­swer, op­ti­miza­tion is quite sim­i­lar. When we’re pro­fil­ing and tak­ing a look at how the time is be­ing spent, we just need to re­move all of the time spent NOT do­ing the most es­sen­tial tasks. Generally, that just means fig­ur­ing out what the es­sen­tial tasks are and then fig­ur­ing out how to not do every­thing else.

What’s next

One bench­mark down, many more to go. Next, I will be re­vis­it­ing a ray tracer I wrote in Clojure a cou­ple of years ago and we will uti­lize our new IR and op­ti­mized run­time to see how fast we can push jank. This post, and this first bench­mark, is just the start. I will be spend­ing the next cou­ple of months tack­ling more and more bench­marks, of vary­ing sizes, to en­sure jank is rea­son­ably fast in every prac­ti­cal sense. This is all build­ing up to jank’s beta re­lease.

A note about the JVM ver­sus na­tive

Many folks who use jank for the first time end up say­ing some­thing along the lines of “Why is jank slow? Isn’t it writ­ten in C++?” So, firstly, jank is slow be­cause it’s un­op­ti­mized. As we can see here, jank can ab­solutely com­pete with the JVM in this mi­cro-bench­mark and I will show in fu­ture posts that we can do so in larger bench­marks, too. Secondly, you know what’s also writ­ten in C++? The JVM. jank is in­deed a minia­ture JVM, with some key dis­tinc­tions.