10 interesting stories served every morning and every evening.

Qwen 3.8 27B is excellent, but it defaults to wildly overthinking things

simonwillison.net

16th August 2026

Friday’s big re­lease was Qwen 3.8 27B, an Apache 2 li­censed 27B pa­ra­me­ter vi­sion-ca­pa­ble LLM from Alibaba’s Qwen re­search lab. I’ve been look­ing for­ward to this one: 27B is an ex­cel­lent size for run­ning a model on a rea­son­ably specced lap­top, and its pre­de­ces­sor Qwen 3.6 27B was im­pres­sive.

Qwen’s self-re­ported bench­marks for this model are eye-open­ing. They show a boost from both Qwen 3.6 27B and the closed-weight Qwen 3.7-Plus, which was one of Qwen’s strongest mod­els of any size as re­cently as May this year. It will be in­ter­est­ing to hear what in­de­pen­dent bench­marks have to say about the model.

I’ve been run­ning the model on two dif­fer­ent ma­chines: my 128GB M5 Max MacBook Pro, and an NVIDIA DGX Spark. On both ma­chines I’m run­ning LM Studio and their 17GB Q4_K_M quan­tized build. I also tried us­ing llama-server di­rectly on the Spark.

Qwen’s doc­u­men­ta­tion de­scribes the model as de­fault­ing to xhigh for the rea­son­ing ef­fort, and the LM Studio GGUF I’ve been try­ing pre­serves that de­fault:

Qwen3.8 comes with of­fi­cial sup­port for rea­son­ing_­ef­fort, which can be used to ad­just rea­son­ing depth and con­trol cost:

xhigh (default): for com­plex tasks de­mand­ing thor­ough analy­sis

medium: bal­anc­ing ac­cu­racy and speed

low: ef­fi­cient rea­son­ing op­ti­miz­ing for speed and cost

Qwen3.8 comes with of­fi­cial sup­port for rea­son­ing_­ef­fort, which can be used to ad­just rea­son­ing depth and con­trol cost:

xhigh (default): for com­plex tasks de­mand­ing thor­ough analy­sis

medium: bal­anc­ing ac­cu­racy and speed

low: ef­fi­cient rea­son­ing op­ti­miz­ing for speed and cost

This is a hi­lar­i­ous de­fault. It’s ab­solutely not a good way to run the model, es­pe­cially on con­sumer hard­ware. I’ve been find­ing the re­sults ex­tremely en­ter­tain­ing.

I quickly ran into prob­lems with LM Studio’s de­fault con­text limit of 8,192 to­kens—Qwen was us­ing them all up think­ing about even the most mun­dane of prob­lems. I loaded the model with the full 262,144 max­i­mum con­text length and that prob­lem went away.

Here’s the pel­i­can rid­ing a bi­cy­cle SVG I got from my first at­tempt with that in­creased con­text length. It took 21 min­utes to gen­er­ate, us­ing 22,276 rea­son­ing to­kens to pro­duce 3,223 to­kens of out­put. You can read the rea­son­ing trace here.

This is by far the best pel­i­can SVG I’ve been able to gen­er­ate with a model that runs on a lo­cal ma­chine—and this Qwen is pretty small, just a 17GB file on disk. There’s a lot to like about this:

The bi­cy­cle frame is the right shape

It has legs on each side of the bike—that’s very rare

Good, clear pel­i­can pouch

The wings ex­tend to touch the han­dle­bars!

The mo­tion lines are be­hind, not in front

It has a taste­ful back­ground—nice sun, clouds, hill, flow­ers and grass.

Was that worth wait­ing 21 min­utes for? Absolutely not.

Here’s that same prompt run with rea­son­ing turned off—tran­script here. This one pro­duced 3,715 to­kens and took 137s—just over two min­utes.

And just for com­plete­ness, I used OpenRouter to run the same prompt through the much larger Qwen 3.8 2.4T-A95B (released last week) and got this snazzy an­i­mated SVG:

Your browser does not sup­port HTML5 video.

I said Qwen at xhigh has a ten­dency to over-think things, but how bad re­ally is it?

I tried a much sim­pler prompt, again with that de­fault ex­tra high set­ting:

draw an svg of a cir­cle

draw an svg of a cir­cle

Qwen’s rea­son­ing trace started like this:

The user is ask­ing for an SVG draw­ing of a cir­cle. Simple re­quest — but I want it to be a care­fully crafted piece. Let me make some­thing that goes be­yond just <circle>: a sin­gle self-con­tained SVG file with char­ac­ter — maybe a geo­met­ric circle study,” with sub­tle an­i­ma­tion, lay­ered rings, and a dis­tinc­tive palette. Keep the scope right: they asked for an SVG of a cir­cle. So the core is a cir­cle. But I can add craft: con­cen­tric guide cir­cles (like a com­pass/​geom­e­try draw­ing), tick marks, a soft gra­di­ent fill on the main cir­cle, re­strained am­bi­ent mo­tion (a slowly ro­tat­ing dashed ring, puls­ing glow). Respect prefers-re­duced-mo­tion? That’s more for CSS; SVG SMIL or CSS in­side SVG will do. Palette op­tions: deep teal ink on warm pa­per? Or bold ver­mil­ion cir­cle on off-white with navy con­struc­tion lines — Bauhaus/compass-drawing vibe. Let me go with a geometric study” look: cool slate back­ground, or bright pa­per white? Paper white is fine if it’s not the cream-and-ter­ra­cotta combo. […]

The user is ask­ing for an SVG draw­ing of a cir­cle. Simple re­quest — but I want it to be a care­fully crafted piece. Let me make some­thing that goes be­yond just <circle>: a sin­gle self-con­tained SVG file with char­ac­ter — maybe a geo­met­ric circle study,” with sub­tle an­i­ma­tion, lay­ered rings, and a dis­tinc­tive palette.

Keep the scope right: they asked for an SVG of a cir­cle. So the core is a cir­cle. But I can add craft: con­cen­tric guide cir­cles (like a com­pass/​geom­e­try draw­ing), tick marks, a soft gra­di­ent fill on the main cir­cle, re­strained am­bi­ent mo­tion (a slowly ro­tat­ing dashed ring, puls­ing glow). Respect prefers-re­duced-mo­tion? That’s more for CSS; SVG SMIL or CSS in­side SVG will do.

Palette op­tions: deep teal ink on warm pa­per? Or bold ver­mil­ion cir­cle on off-white with navy con­struc­tion lines — Bauhaus/compass-drawing vibe. Let me go with a geometric study” look: cool slate back­ground, or bright pa­per white? Paper white is fine if it’s not the cream-and-ter­ra­cotta combo. […]

Several min­utes later it pro­duced this ab­solutely beau­ti­ful an­i­mated cir­cle, which was en­tirely not what I had asked for!

Your browser does not sup­port HTML5 video.

My strong rec­om­men­da­tion: ig­nore that de­fault. Run Qwen 3.8 27B on low or even no rea­son­ing lev­els at first. It’s a great model, but wow that de­fault set­ting is a bad place to start.

It’s very good at bound­ing boxes

A fun way to test a vi­sion model is to see how well it can re­turn bound­ing boxes around items in a pho­to­graph. I’ve seen pre­vi­ous Qwen mod­els deal well with this, so I de­cided to put it to the test draw­ing bound­ing boxes around some pel­i­cans.

I’ve seen ask­ing for 0 – 1000 scale pro­duce good re­sults in the past. I tried this:

llm -a https://​sta­tic.inat­u­ral­ist.org/​pho­tos/​714731804/​large.jpg \ -m lm­stu­dio/​qwen/​qwen3.8 – 27b \ Return JSON bound­ing boxes for the pel­i­cans in this photo, 0 – 1000 scale for each di­men­sion’

Here’s the rea­son­ing trace, which pro­duced this:

[ {“bbox_2d”: [195, 290, 370, 780], label”: pelicans”}, {“bbox_2d”: [445, 320, 675, 850], label”: pelicans”} ]

This is such a good match. Here are those boxes ren­dered on top of the photo:

Building a tool to la­bel bound­ing boxes

That vi­su­al­iza­tion of the bound­ing boxes was taken us­ing a new cus­tom tool that I had Qwen 3.8 27B build for me, run­ning of­fline on my lap­top.

I for­got to dial down the think­ing ef­fort so it was mas­sively over-en­gi­neered, but it did man­age to pro­duce this full in­ter­face from this sin­gle prompt:

[ {“bbox_2d”: [195, 290, 370, 780], label”: pelicans”}, {“bbox_2d”: [445, 320, 675, 850], label”: pelicans”} ]

Build an HTML page which has an in­put box for ac­cept­ing the URL to an im­age and a textarea for ac­cept­ing the above style of JSON. It ap­pends the im­age to the page, mea­sures its width and height, then treats the co­ords in the bbox_2d as scaled from 0 – 1000 and scales them against the ac­tual width and height, then it ren­ders la­belled boxes over the im­age.

[ {“bbox_2d”: [195, 290, 370, 780], label”: pelicans”}, {“bbox_2d”: [445, 320, 675, 850], label”: pelicans”} ]

Build an HTML page which has an in­put box for ac­cept­ing the URL to an im­age and a textarea for ac­cept­ing the above style of JSON.

It ap­pends the im­age to the page, mea­sures its width and height, then treats the co­ords in the bbox_2d as scaled from 0 – 1000 and scales them against the ac­tual width and height, then it ren­ders la­belled boxes over the im­age.

This screen­shot shows one of the fea­tures I did not ask for—a demo scene, for if you don’t have a pho­to­graph to test the tool with:

Here’s the rel­e­vant seg­ment of the think­ing trace, where it de­cided to draw its own pel­i­cans purely be­cause I had used the la­bel pelicans” in the ex­am­ple JSON I gave it in the prompt:

Also a load sam­ple” that uses a known im­age? Can’t de­pend on ex­ter­nal im­ages, but… the im­age URL in­put is user-pro­vided; I could add a try with sam­ple” but­ton […] Hmm, I can draw a sim­ple scene on can­vas, ex­port it as a data URL, and load it into the im­age — that’s self-con­tained and demo-able! […] But the user’s co­ords are for an ac­tual pel­i­can im­age; a gen­er­ated place­holder can still demo the scal­ing. Generate a 1000x1000 place­holder: gra­di­ent wa­ter + two blob-like pelican” sil­hou­ettes placed at the given bboxes (using the same scale — cute: sil­hou­ettes at the ex­act 0 – 1000 po­si­tions, show­ing the boxes align). This makes for a fun, self-con­tained demo. Keep it sim­ple: sky gra­di­ent, sun, wa­ter, two pel­i­can-ish shapes (ellipse body, cir­cle head, beak). Place at bbox cen­ters.

Also a load sam­ple” that uses a known im­age? Can’t de­pend on ex­ter­nal im­ages, but… the im­age URL in­put is user-pro­vided; I could add a try with sam­ple” but­ton […] Hmm, I can draw a sim­ple scene on can­vas, ex­port it as a data URL, and load it into the im­age — that’s self-con­tained and demo-able! […] But the user’s co­ords are for an ac­tual pel­i­can im­age; a gen­er­ated place­holder can still demo the scal­ing. Generate a 1000x1000 place­holder: gra­di­ent wa­ter + two blob-like pelican” sil­hou­ettes placed at the given bboxes (using the same scale — cute: sil­hou­ettes at the ex­act 0 – 1000 po­si­tions, show­ing the boxes align). This makes for a fun, self-con­tained demo. Keep it sim­ple: sky gra­di­ent, sun, wa­ter, two pel­i­can-ish shapes (ellipse body, cir­cle head, beak). Place at bbox cen­ters.

(I’m slightly ner­vous that mod­els around the world might have a bias to­wards draw­ing pel­i­cans at any chance they can get, brought on by nearly two years of ex­po­sure to my own stu­pid bench­mark.)

Is all that over-think­ing nec­es­sary? Maybe it is, at least a bit. I tried with rea­son­ing turned off and got this ver­sion, (transcript here), which nearly works but shows the boxes in the wrong place:

So with­out rea­son­ing it did­n’t quite one-shot a work­ing tool. I’m sure it could get there with some fol­low-up prompts, but this is a good ex­am­ple of how rea­son­ing can make a dif­fer­ence.

Yes, it can drive cod­ing agents

One of the biggest ques­tions around lo­cal mod­els is whether or not they have enough horse­power to suc­cess­fully run a cod­ing agent loop. Coding agents re­quire long con­text, strong code gen­er­a­tion sup­port and re­li­able tool-call­ing. On pa­per Qwen 3.8 27B has all three of these, so is it up to the task?

My ini­tial ex­per­i­ments with Pi have been very promis­ing. I chose Pi be­cause it has a shorter sys­tem prompt than most other op­tions, mak­ing it a bet­ter fit for try­ing out smaller mod­els.

I con­fig­ured Pi to use Qwen 3.8 27B run­ning in LM Studio on the Spark (shared via tailscale serve) by adding this to ~/.pi/agent/models.json:

{ providers”: { spark”: { baseUrl”: https://​spark-18b3.tail68a31.ts.net/​v1, api”: openai-responses”, apiKey”: dummy”, models”: [ { id”: qwen3.8 – 27b”, reasoning”: true } ] } } }

Then ran pi –provider spark –model qwen3.8 – 27b in my ~/dev/datasette folder and prompted:

how does auth work?

how does auth work?

After a se­quence of rea­son­ing and tool calls that ac­cessed a bunch of dif­fer­ent files it pro­duced this re­ply, which is very solid.

Just one prob­lem: I wanted to share that tran­script. So I pointed Pi and Qwen 3.8 27B at the JSONL tran­script file in ~/.pi/agent/sessions/–Users-simon-Dropbox-dev-datasette– and prompted:

Write Python code to con­vert this jsonl to mark­down

Write Python code to con­vert this jsonl to mark­down

And it built and tested this pi_j­son­l_­to_md.py, which did ex­actly what I needed. Here’s that ses­sion tran­script, pub­lished us­ing the tool that it cre­ated.

The quest for speed

So far this is all look­ing very promis­ing. We have a 17GB model that runs on high-end con­sumer hard­ware and can write code, drive tools, an­no­tate im­ages and gen­er­ally do every­thing that I need from an LLM for get­ting real work done.

There’s one very sig­nif­i­cant catch: it feels slow—es­pe­cially when it starts over-think­ing, but even with­out that it’s not par­tic­u­larly sprightly.

I’ve been get­ting around 15 – 30 to­kens a sec­ond from LM Studio. That’s not ter­ri­ble, but it’s slow enough that it’s go­ing to be hard to win me away from hosted API mod­els, which can re­turn re­sults a whole lot faster. Artificial Analysis track to­ken speed and show OpenAI 5.6 Sol at 74 to­kens/​sec­ond and 5.6 Luna at an im­pres­sive 184/second.

The good news is that the com­mu­nity have been ex­plor­ing ways to speed things up since the model was first re­leased two days ago.

One of the most promis­ing op­ti­miza­tions is baked into the model it­self. Qwen sup­ports Multi-Token Prediction, an ar­chi­tec­ture trick where a cheaper mech­a­nism guesses sev­eral to­kens ahead and the main model can then quickly ver­ify if the guesses were cor­rect. This can have quite a dra­matic ef­fect on in­fer­ence per­for­mance.

Based on this tweet from llama.cpp cre­ator Georgi Gerganov I tried run­ning the model with MTP like this on the Spark:

llama serve \ -hf ggml-org/​Qwen3.8 – 27B-GGUF:Q4_K_M \ -hfd ggml-org/​Qwen3.8 – 27B-GGUF:Q4_0 \ –spec-default \ –spec-type draft-mtp \ –reasoning-preserve

And sure enough, this gave me a sig­nif­i­cant boost. I had GPT-5.6 in Codex run a com­par­a­tive bench­mark on the Spark and the –spec-type draft-mtp server out­per­formed the LM Studio de­fault GGUF by around 72%.

I ex­pect we’ll see a whole lot more in­no­va­tion around serv­ing this model faster over the next few weeks. The MLX com­mu­nity likely have some tricks brew­ing as well.

Some ob­ser­va­tions

The fact that a 17GB file can do all of this stuff on my home ma­chines is a mir­a­cle. Once again, I’m de­lighted and amazed at how much progress lo­cal mod­els have made this year. A year ago this would have been com­pet­i­tive with the best and most ex­pen­sive of the pro­pri­etary mod­els—to­day it can run on a ca­pa­ble lap­top.

The only thing hold­ing this back from be­ing a daily dri­ver is per­for­mance. It feels pretty slow on both the M5 Mac and the DGX Spark. That’s the catch with these dense (non-Mixture-of-Experts) mod­els—they re­quire a whole lot of mem­ory band­width to per­form well, and nei­ther of the ma­chines I have ac­cess to are top per­form­ers in that re­gard.

The most im­por­tant thing about Qwen 3.8 27B is what it demon­strates. We can have an open weights gen­eral pur­pose model with a long con­text, ef­fec­tive tool call­ing, strong vi­sion abil­ity, and com­pe­tent code gen­er­a­tion, and we can fit the whole thing in just a 17GB file.

The mod­els at this size con­tinue to get bet­ter at an im­pres­sive rate. We don’t need to spend half a mil­lion dol­lars on dat­a­cen­ter-class hard­ware just to run a com­pe­tent model.

Anthropic’s ‘Watermark’ Text Adulteration in Claude Is a Perversion of Writing

daringfireball.net

When I wrote this week about Anthropic’s an­nounce­ment that all Claude mod­els, world­wide, would soon be­gin watermarking” every­thing they gen­er­ate, in­clud­ing text, to com­ply with this EU reg­u­la­tion, we were left to spec­u­late how this was go­ing to work, be­cause Anthropic of­fered not even a vague de­scrip­tion of how it would work — de­spite the fact that the ti­tle of the an­nounce­ment was, ab­surdly and in­sult­ingly, How Claude Marks AI-Generated Content”.

My ini­tial spec­u­la­tion was that maybe they’d hide in­vis­i­ble non-print­ing Unicode char­ac­ters in the text. Just spit­balling. Turns out that’s not what they’re go­ing to do. What they’re go­ing to do is ap­ply a form of steganog­ra­phy, where the choice of words (or other to­ken out­put) at in­fer­ence time will leave fin­ger­prints that can later, maybe, be de­tected prob­a­bilis­ti­cally.

I ini­tially guessed invisible char­ac­ters” not be­cause I did­n’t think of the se­man­tic word-choice tech­nique, but be­cause I was a fool who took Anthropic at its word in their de­scrip­tion of what they would do. Their orig­i­nal sup­port doc­u­ment claims:

When a sup­ported Claude model gen­er­ates text, it weaves an im­per­cep­ti­ble wa­ter­mark di­rectly into the text it­self. You won’t see it, and it does­n’t change the mean­ing, qual­ity, or read­abil­ity of Claude’s re­sponse.

When a sup­ported Claude model gen­er­ates text, it weaves an im­per­cep­ti­ble wa­ter­mark di­rectly into the text it­self. You won’t see it, and it does­n’t change the mean­ing, qual­ity, or read­abil­ity of Claude’s re­sponse.

They say imperceptible” and doesn’t change the mean­ing, qual­ity, or read­abil­ity”. Their words. Not al­most im­per­cep­ti­ble. Not slightly changes the mean­ing, qual­ity, or read­abil­ity. That made sense to me, be­cause that’s ab­solutely what I want — nay, de­mand — from any tools I use per­son­ally. It’s un­ac­cept­able for a tool to sac­ri­fice an iota of clar­ity, co­her­ence, mean­ing, qual­ity, etc. for the pur­pose of em­bed­ding hid­den clues within the text to sug­gest its prove­nance. That’s what I would and will de­mand. And Anthropic’s (original) sup­port doc­u­ment un­am­bigu­ously claims that’s what their sys­tem will en­able. So if that were true, I could­n’t see what was left other than hid­ing in­vis­i­ble char­ac­ters within the text.

My er­ror was be­liev­ing Anthropic that their sys­tem would­n’t adul­ter­ate and cor­rupt the se­man­tics of the text their mod­els gen­er­ate. That is in fact ex­actly what they plan to do. I should have my head ex­am­ined for be­liev­ing a sin­gle word of a doc­u­ment ti­tled How Claude Marks AI-Generated Content” that does­n’t ex­plain, at all, how Claude marks (or will mark) AI-generated con­tent.

How It’s Actually Going to Work

Yesterday, on an en­tirely dif­fer­ent web­site than the orig­i­nal How Claude marks AI-generated con­tent” ar­ti­cle (the one that did­n’t ex­plain any­thing at all about how it works), Anthropic pub­lished How Claude’s Text Watermark Works”, which does ac­tu­ally ex­plain in lay­man-ac­ces­si­ble terms how it’s go­ing to work. I will re­turn to Anthropic’s new highly eu­phemistic and slightly mis­lead­ing de­scrip­tion be­low.

There’s a bunch of re­search on this topic, some of which I have also linked to be­low. But the very best de­scrip­tion of the gen­eral idea be­hind the tech­nique is an in­ter­ac­tive es­say by James Padolsey, How AI Text Watermarking Works”. It’s a won­der­fully co­gent read, and the in­ter­ac­tive el­e­ments splen­didly il­lus­trate the main con­cepts. A+ work. If you have any in­ter­est in this at all, I dare say you must read — and play with — Padolsey’s piece.

But here’s my stab at a lay­man’s high-level sum­mary. If you toss a coin N times and note the re­sults, you can de­ter­mine with a de­gree of cer­tainty whether the coin is fair or bi­ased. LLMs are, in their pop­u­lar in­car­na­tions, non-de­ter­min­is­tic. Ask the same ques­tion of the same model and you of­ten get at least slightly dif­fer­ent an­swers. Maybe the same mean­ing, but dif­fer­ent phras­ing. At each de­ci­sion point for gen­er­at­ing the next to­ken, the model makes a choice. With these se­man­tic wa­ter­mark­ing tech­niques, they make dif­fer­ent choices for some to­kens based on word lists that could be called green” and red”. At each de­ci­sion point, they’re a lit­tle more likely to pick a word from the green list than the red list. That does­n’t mean they never choose words from the red list. Just that they’re less likely to than they would if the adul­ter­ated mark­ing tech­nique weren’t in place. (Same way that a crooked 51 – 49 coin will still land wrong” side up 49 times out of 100 on av­er­age.)

Words or word phrases are sorted into the green and red lists de­ter­min­is­ti­cally on the fly, at each next to­ken” gen­er­a­tion point. So some­times a spe­cific word will be on the green list, and other times it will be on the red list. Someone with the se­cret key can de­ter­mine which list a word will be on at each to­ken gen­er­a­tion point (which is how the wa­ter­mark­ing is de­tected); those with­out the se­cret key can­not. This means there will never be a list of words that Claude prefers or es­chews.

With coin flip­ping, the higher N is — the more times you flip — the more con­fi­dent you can be that the coin is fair or bi­ased. So too with this se­man­tic wa­ter­mark­ing. The more words in the text, the more ac­cu­rate the analy­sis will be that the text was gen­er­ated by a spe­cific AI model or not. With too few coin flips, you can’t achieve any con­fi­dence at all re­gard­ing a coin’s fair­ness. With too few words (or to­kens), there’s no way to achieve any con­fi­dence whether a string of text was AI-generated or not.

Given a string of text to ex­am­ine for signs of a spe­cific wa­ter­mark­ing sys­tem, if there are more words tagged as green and fewer tagged as red than would oth­er­wise be ex­pected, the text can be flagged — with some de­gree of con­fi­dence — as hav­ing been gen­er­ated, or merely mod­i­fied, by the AI sys­tem that ap­plies the spe­cific se­cret-key wa­ter­mark­ing sys­tem. The amount of con­fi­dence in the de­ter­mi­na­tion will ob­vi­ously vary, sig­nif­i­cantly, based on the size of the text string and ran­dom­ized weights given to words on the green and red lists. But only Anthropic will be able to de­ter­mine if text was seem­ingly gen­er­ated by Claude, and Anthropic will only be able to de­tect the wa­ter­marks that are ap­plied by Claude. Claude can’t de­tect the hid­den wa­ter­mark sig­nals gen­er­ated by, say, Gemini, and Gemini can’t de­tect the hid­den wa­ter­mark sig­nals cre­ated by Claude, be­cause each im­ple­men­ta­tion is pred­i­cated on se­cret keys held only by the LLM provider.

Objections to the Technical Premise

One of my fun­da­men­tal prob­lems with this is that no two syn­onyms carry the ex­act same mean­ing. He leaped at the chance” and He jumped at the op­por­tu­nity” are very sim­i­lar sen­tences ex­press­ing the same gen­eral sen­ti­ment, but they are not the same. The ex­act words we choose when writ­ing mat­ter. I want any LLM I use to choose the very best, most pre­cise words at every sin­gle de­ci­sion point. An ob­vi­ous con­straint that I ac­cept is time and com­pu­ta­tion. Within the con­straint of ex­e­cut­ing in­fer­ence quickly, and at a cer­tain cost per to­ken, I want the best words. This con­straint matches hu­man writ­ing. I could surely write a bet­ter col­umn by tak­ing longer to write it. I write with a sense of how much care I should put into every word and punc­tu­a­tion choice I make. I take more time with cer­tain para­graphs, sen­tences, or even in­di­vid­ual word choices when my gut feel­ing says I should.

In other words, these are nec­es­sary trade-offs. These fac­tors are all in my in­ter­est: speed, cost, qual­ity. Ideally I would like per­fect writ­ing, at in­stan­ta­neous gen­er­a­tion speed, at zero cost. None of those things are pos­si­ble. Computation is not free of charge (and cloud-based LLM in­fer­ence with lead­ing mod­els is ac­tu­ally ex­pen­sive). Inference is not in­stan­ta­neous. And great writ­ing, whether nat­ural or ar­ti­fi­cial, can only ap­proach per­fec­tion.

The idea that any­thing other than my needs should fac­tor into the gen­er­a­tion of text for me is patently of­fen­sive.

This is­n’t just about text one might gen­er­ate with the in­ten­tion of pass­ing it off as their own nat­ural work. This is­n’t even about LLM proof­read­ing of work writ­ten by hand. Anthropic is say­ing that all new Claude mod­els are go­ing to adul­ter­ate every sin­gle bit of text longer than 200 to­kens (~150 words) they gen­er­ate, in­clud­ing every­thing it pre­sents to its users to read. So even in a pri­vate con­ver­sa­tion be­tween a user and Claude, which will never be read by any­one other than the user, Claude will be­gin mak­ing word choices in the name of mark­ing its out­put in sta­tis­ti­cally pre­dictable ways rather than max­i­miz­ing clar­ity and pre­ci­sion.

Even to­day’s so-called fron­tier mod­els are al­ready de­cid­edly lack­ing in lu­cid­ity. Claude, ChatGPT, Grok, et al. are better writ­ers” than most hu­mans and pro­duce bet­ter prose than the me­dian hu­man. But: no shit. Most peo­ple are ter­ri­ble writ­ers. The average per­son” is pretty stu­pid and half of all peo­ple are stu­pider than that. And there are many smart, in­ter­est­ing peo­ple who are mis­er­able writ­ers. So as im­pres­sive as LLMs are, the bar is low. The best writ­ing I see come out of these mod­els is worse than any­thing I would choose to read for plea­sure. And now Anthropic is say­ing they’re go­ing to make it worse, on pur­pose, for pur­poses that do not ben­e­fit me in any way? Even if only slightly worse?

Get fucked.

Objections to the EU Regulation

Speaking of ob­jec­tions, the rel­e­vant EU reg­u­la­tion mo­ti­vat­ing all of this, Code of Practice on Transparency of AI-Generated Content”, is red-tape nanny-state pipe-dream non­sense. Here’s Ben Thompson’s sum­mary from a pay­walled Stratechery up­date this week:

The reg­u­la­tion ap­plies to text longer than 200 to­kens. The provider must man­date in their terms-of-ser­vice that users not re­move the wa­ter­mark­ing. The so­lu­tion should be ro­bust in terms of evad­ing typical pro­cess­ing so­lu­tions” like screen shots, scan­ning and OCR, copy-and-past­ing, trans­la­tions, etc.

The reg­u­la­tion ap­plies to text longer than 200 to­kens.

The provider must man­date in their terms-of-ser­vice that users not re­move the wa­ter­mark­ing.

The so­lu­tion should be ro­bust in terms of evad­ing typical pro­cess­ing so­lu­tions” like screen shots, scan­ning and OCR, copy-and-past­ing, trans­la­tions, etc.

Taken lit­er­ally, com­pli­ant LLM terms of ser­vice must for­bid users from rephras­ing the out­put from mod­els that com­ply with this reg­u­la­tion, be­cause the word choices are the marks. But it’s not the European Union that is try­ing to im­pose their ab­surd, im­prac­ti­cal, witch-hunt-fu­el­ing reg­u­la­tion on the en­tire world. That falls on Anthropic.

Complying with this, par­tic­u­larly with re­gard to text, is only go­ing to cre­ate prob­lems for hon­est users. Dishonest users at­tempt­ing to pass off AI-generated text as their own writ­ing (students, em­ploy­ees, who­ever) will sim­ply cir­cum­vent de­tec­tion through non-com­pli­ant AI para­phras­ing tools.

James Padolsey — whose in­ter­ac­tive vi­sual ex­pla­na­tion of how these schemes work I linked to above — ex­plains this in a post ti­tled Anthropic’s Weak Watermarks Appease a Weak Law” (which, if it rings a bell, I linked to in a stand­alone post ear­lier to­day):

The same thought that led to this law could have ap­plied to cal­cu­la­tors at the time of their in­cep­tion, had their out­puts re­vealed them­selves through arte­facts. Thankfully, a sum borne of the brain is treated no dif­fer­ently from one pro­duced by a cal­cu­la­tor. Likewise with spellcheck­ers. To make as­sis­tance sus­pect only once the tool be­comes ca­pa­ble enough to com­pose a whole sen­tence is not a prin­ci­pled bound­ary. It is a moral pre­mium placed on dif­fi­culty it­self.

Anthropic has nev­er­the­less cho­sen a blan­ket, model-level im­ple­men­ta­tion that ap­pears broader than the law’s min­i­mum re­quire­ment. That may be con­ve­nient com­pli­ance en­gi­neer­ing, but it dis­cards dis­tinc­tions the law ex­pressly at­tempted to pre­serve. The re­sult is a sig­nal broad enough to im­pli­cate harm­less and as­sis­tive use, yet frag­ile enough to be re­moved by a mo­ti­vated per­son through sub­stan­tial re­com­po­si­tion. It risks con­cen­trat­ing sus­pi­cion on or­di­nary and as­sis­tive users while re­main­ing weak­est against de­lib­er­ate de­cep­tion.

The same thought that led to this law could have ap­plied to cal­cu­la­tors at the time of their in­cep­tion, had their out­puts re­vealed them­selves through arte­facts. Thankfully, a sum borne of the brain is treated no dif­fer­ently from one pro­duced by a cal­cu­la­tor. Likewise with spellcheck­ers. To make as­sis­tance sus­pect only once the tool be­comes ca­pa­ble enough to com­pose a whole sen­tence is not a prin­ci­pled bound­ary. It is a moral pre­mium placed on dif­fi­culty it­self.

Anthropic has nev­er­the­less cho­sen a blan­ket, model-level im­ple­men­ta­tion that ap­pears broader than the law’s min­i­mum re­quire­ment. That may be con­ve­nient com­pli­ance en­gi­neer­ing, but it dis­cards dis­tinc­tions the law ex­pressly at­tempted to pre­serve. The re­sult is a sig­nal broad enough to im­pli­cate harm­less and as­sis­tive use, yet frag­ile enough to be re­moved by a mo­ti­vated per­son through sub­stan­tial re­com­po­si­tion. It risks con­cen­trat­ing sus­pi­cion on or­di­nary and as­sis­tive users while re­main­ing weak­est against de­lib­er­ate de­cep­tion.

Padolsey is the cre­ator of Declaude, a de­light­fully sim­ple web app that al­lows you to Paste in AI-flavored text and get the same con­tent back as plain prose”. Declaude’s orig­i­nal pur­pose is clean­ing the sac­cha­rine Claude per­son­al­ity stink from text (whether it was cre­ated by Claude or any other LLM), but, if Anthropic per­sists in its stated plan to be­gin adul­ter­at­ing all text Claude gen­er­ates, Declaude will also serve as a copy-paste sin­gle-ex­tra-step way to elim­i­nates those marks. Declaude is in­ter­est­ing and use­ful al­ready, but it ex­em­pli­fies how ill-con­sid­ered and fu­tile this EU reg­u­la­tion is when it comes to prose.

Google SynthID

Google has a wa­ter­mark­ing sys­tem in place that they call SynthID, which they ap­ply to AI-generated im­ages, video, au­dio, and text. I’m con­cerned in this ar­ti­cle only with text. With mul­ti­me­dia, em­bed­ded wa­ter­marks can be meta­data within files, and truly not af­fect the ex­pe­ri­en­tial qual­ity of the work when viewed or lis­tened to. With text, we are talk­ing about the ac­tual words that are cho­sen. From the AI-generated text” sec­tion of Google DeepMind’s own de­scrip­tion of SynthID:

We’ve ex­panded SynthID to wa­ter­mark­ing and iden­ti­fy­ing text gen­er­ated by the Gemini app and web ex­pe­ri­ence. Large lan­guage mod­els gen­er­ate text one word (token) at a time. Each word is as­signed a prob­a­bil­ity score, based on how likely it is to be gen­er­ated next. So for a sen­tence like My fa­vorite trop­i­cal fruits are mango and…”, the word bananas” would have a higher prob­a­bil­ity score than the word airplanes”. SynthID ad­justs these prob­a­bil­ity scores to gen­er­ate a wa­ter­mark. It’s not no­tice­able to the hu­man eye, and does­n’t af­fect the qual­ity of the out­put.

We’ve ex­panded SynthID to wa­ter­mark­ing and iden­ti­fy­ing text gen­er­ated by the Gemini app and web ex­pe­ri­ence. Large lan­guage mod­els gen­er­ate text one word (token) at a time. Each word is as­signed a prob­a­bil­ity score, based on how likely it is to be gen­er­ated next. So for a sen­tence like My fa­vorite trop­i­cal fruits are mango and…”, the word bananas” would have a higher prob­a­bil­ity score than the word airplanes”. SynthID ad­justs these prob­a­bil­ity scores to gen­er­ate a wa­ter­mark. It’s not no­tice­able to the hu­man eye, and does­n’t af­fect the qual­ity of the out­put.

In a group chat, a friend of mine quoted the above, and I re­sponded that if a chat­bot wrote My fa­vorite trop­i­cal fruits are mango and air­planes”, I’m pretty sure I’d fuck­ing no­tice. Another friend then re­sponded with this:

Days later, that still cracks me up.

But Google’s ab­surd de­scrip­tion puts the lie to their own claim that it is­n’t no­tice­able, and it serves to show just how lit­tle re­gard the peo­ple be­hind these gen­er­ated-text fin­ger­print­ing schemes have for the ac­tual craft of writ­ing. Of course ba­nanas has a higher prob­a­bil­ity score than air­planes, be­cause air­planes aren’t fruit. But what about pineap­ple? Should the sen­tence com­plete to mango and ba­nanas” or mango and pineap­ple”? That’s a good ques­tion, and the only ac­cept­able an­swer for why an LLM should choose ba­nanas in­stead of pineap­ple (or co­conut, or guava, or pa­paya…) is that it has de­ter­mined that it’s the best fit for the in­tended mean­ing, tone, and sen­ti­ment of the text. Not be­cause ba­nanas is on the wa­ter­mark­ing green” list and pineap­ple is on the red” list, even though pineap­ple might be the bet­ter fit. Google’s own sup­pos­edly joc­u­lar de­scrip­tion of how SynthID works in fact cap­tures how the scheme per­verts the text it gen­er­ates.

They’re say­ing you won’t no­tice be­cause if it only chooses ba­nanas over pineap­ple for these fin­ger­print­ing pur­poses, well, they’re both trop­i­cal fruits and who cares. But it’s ut­ter non­sense that the dif­fer­ence is not no­tice­able to the hu­man eye”. The se­man­tic dif­fer­ence be­tween ba­nana and pineap­ple is just as no­tice­able to the hu­man eye as the taste of the two are to the hu­man tongue.

If it did pro­duce My fa­vorite trop­i­cal fruits are mango and air­planes”, it’d be in­cred­i­bly stu­pid, but it would­n’t be of­fen­sive be­cause we’d all rec­og­nize that some­thing com­pletely off-key hap­pened. What’s of­fen­sive is that with a sys­tem like SynthId in place, where the fin­ger­print­ing de­ci­sions are mo­ti­vated by a se­cret key, we have no idea whether it com­pleted to mango and ba­nanas” be­cause ba­nanas was de­ter­mined to be the best next to­ken, or be­cause ba­nanas is in the green” bucket of words. It calls every sin­gle word choice into ques­tion.

Here’s a pa­per pub­lished in Nature where Google’s team be­hind SynthID pub­lished their work, af­ter putting it into pro­duc­tion with Gemini (née Bard):

We analysed ap­prox­i­mately 20 mil­lion wa­ter­marked and un­wa­ter­marked re­sponses and com­puted the thumbs-up and thumbs-down rates (both as a frac­tion of the to­tal num­ber of thumbs-up and thumbs-down feed­back re­ceived). We found that the thumbs-up rate for the two mod­els dif­fered by 0.01% (with the wa­ter­marked model be­ing higher); and the thumbs-down rate dif­fered by 0.02% (with the wa­ter­marked model be­ing lower). We found both of these dif­fer­ences to be sta­tis­ti­cally in­signif­i­cant, and well within the 95% con­fi­dence in­ter­vals.

From this ex­per­i­ment, we con­clude that over a wide va­ri­ety of real chat­bot in­ter­ac­tions, the dif­fer­ence in re­sponse qual­ity and util­ity, as judged by hu­mans, is neg­li­gi­ble. Subsequently, non-dis­tor­tionary SynthID-Text has been pro­duc­tion­ized and is cur­rently wa­ter­mark­ing re­sponses in Gemini and Gemini Advanced. To the best of our knowl­edge, this eval­u­a­tion rep­re­sents the first sys­tem­atic wa­ter­mark­ing in­ves­ti­ga­tion of its kind within a large-scale pro­duc­tion sys­tem.

We analysed ap­prox­i­mately 20 mil­lion wa­ter­marked and un­wa­ter­marked re­sponses and com­puted the thumbs-up and thumbs-down rates (both as a frac­tion of the to­tal num­ber of thumbs-up and thumbs-down feed­back re­ceived). We found that the thumbs-up rate for the two mod­els dif­fered by 0.01% (with the wa­ter­marked model be­ing higher); and the thumbs-down rate dif­fered by 0.02% (with the wa­ter­marked model be­ing lower). We found both of these dif­fer­ences to be sta­tis­ti­cally in­signif­i­cant, and well within the 95% con­fi­dence in­ter­vals.

From this ex­per­i­ment, we con­clude that over a wide va­ri­ety of real chat­bot in­ter­ac­tions, the dif­fer­ence in re­sponse qual­ity and util­ity, as judged by hu­mans, is neg­li­gi­ble. Subsequently, non-dis­tor­tionary SynthID-Text has been pro­duc­tion­ized and is cur­rently wa­ter­mark­ing re­sponses in Gemini and Gemini Advanced. To the best of our knowl­edge, this eval­u­a­tion rep­re­sents the first sys­tem­atic wa­ter­mark­ing in­ves­ti­ga­tion of its kind within a large-scale pro­duc­tion sys­tem.

To this I say:

Gemini/Bard’s thumbs-up/​thumbs-down but­tons are not a good ex­per­i­ment for eval­u­at­ing the ef­fect on qual­ity. If a chat­bot tells me My fa­vorite trop­i­cal fruits are mango and ba­nanas” in­stead of mango and pineap­ple”, I’m not go­ing to give the re­sponse a thumbs down be­cause of the fruit it chose. I’d give it a thumbs down if it said airplanes”, yes, but that’s a straw­man. (The pa­per in Nature even uses My favourite trop­i­cal fruit is …” as an il­lus­tra­tion, but in the pa­per, the only four next to­kens con­sid­ered are, in or­der of prob­a­bil­ity dis­tri­b­u­tion, mango, ly­chee, pa­paya, and durian. No air­planes. And, con­ve­niently, in the pa­per’s ex­am­ple, the winner” of the wa­ter­mark­ing tournament” just hap­pens to be mango, the one that would have been se­lected as the best if the wa­ter­mark­ing weren’t in place.)

Gemini/Bard’s thumbs-up/​thumbs-down but­tons are not a good ex­per­i­ment for eval­u­at­ing the ef­fect on qual­ity. If a chat­bot tells me My fa­vorite trop­i­cal fruits are mango and ba­nanas” in­stead of mango and pineap­ple”, I’m not go­ing to give the re­sponse a thumbs down be­cause of the fruit it chose. I’d give it a thumbs down if it said airplanes”, yes, but that’s a straw­man. (The pa­per in Nature even uses My favourite trop­i­cal fruit is …” as an il­lus­tra­tion, but in the pa­per, the only four next to­kens con­sid­ered are, in or­der of prob­a­bil­ity dis­tri­b­u­tion, mango, ly­chee, pa­paya, and durian. No air­planes. And, con­ve­niently, in the pa­per’s ex­am­ple, the winner” of the wa­ter­mark­ing tournament” just hap­pens to be mango, the one that would have been se­lected as the best if the wa­ter­mark­ing weren’t in place.)

A difference in re­sponse qual­ity and util­ity, as judged by hu­mans” that is negligible” does not mean im­per­cep­ti­ble. What they re­ally mean is that it’s only slightly worse and that every­one is ei­ther too stu­pid to no­tice or too in­dif­fer­ent to care.

A difference in re­sponse qual­ity and util­ity, as judged by hu­mans” that is negligible” does not mean im­per­cep­ti­ble. What they re­ally mean is that it’s only slightly worse and that every­one is ei­ther too stu­pid to no­tice or too in­dif­fer­ent to care.

It’s widely con­sid­ered that Gemini is be­hind ChatGPT and Claude in qual­ity. Perhaps the fact that they’ve put SynthID-text into pro­duc­tion is one of many rea­sons why. I per­son­ally agree that Gemini’s prose is in­fe­rior. Maybe the use of SynthID has noth­ing to do with the fact that I, along with the gen­eral pub­lic con­sen­sus, con­sider Gemini to be a sec­ond-rate chat­bot — but in that case, maybe it’s the fact that Gemini is a sec­ond-rate chat­bot that makes the dif­fer­ence negligible” when Google started mix­ing in SynthID-motivated to­kens in its re­sults. It’s a lot more likely that your restau­rant cus­tomers won’t no­tice that you re­placed your reg­u­lar cof­fee with Folgers Crystals if your reg­u­lar cof­fee is sec­ond-rate to start with.

It’s widely con­sid­ered that Gemini is be­hind ChatGPT and Claude in qual­ity. Perhaps the fact that they’ve put SynthID-text into pro­duc­tion is one of many rea­sons why. I per­son­ally agree that Gemini’s prose is in­fe­rior. Maybe the use of SynthID has noth­ing to do with the fact that I, along with the gen­eral pub­lic con­sen­sus, con­sider Gemini to be a sec­ond-rate chat­bot — but in that case, maybe it’s the fact that Gemini is a sec­ond-rate chat­bot that makes the dif­fer­ence negligible” when Google started mix­ing in SynthID-motivated to­kens in its re­sults. It’s a lot more likely that your restau­rant cus­tomers won’t no­tice that you re­placed your reg­u­lar cof­fee with Folgers Crystals if your reg­u­lar cof­fee is sec­ond-rate to start with.

Anthropic

Now, fi­nally, back to Anthropic’s new How Claude’s Text Watermark Works”, pub­lished yes­ter­day. I have some com­ments.

To sum­ma­rize:

We use a method of wa­ter­mark­ing that does not have any prac­ti­cal im­pact on the qual­ity or con­tent of Claude’s out­puts; The dif­fer­ence be­tween wa­ter­marked and un-wa­ter­marked text will not be dis­tin­guish­able to read­ers;

To sum­ma­rize:

We use a method of wa­ter­mark­ing that does not have any prac­ti­cal im­pact on the qual­ity or con­tent of Claude’s out­puts;

We use a method of wa­ter­mark­ing that does not have any prac­ti­cal im­pact on the qual­ity or con­tent of Claude’s out­puts;

The dif­fer­ence be­tween wa­ter­marked and un-wa­ter­marked text will not be dis­tin­guish­able to read­ers;

The dif­fer­ence be­tween wa­ter­marked and un-wa­ter­marked text will not be dis­tin­guish­able to read­ers;

Translation: Specific words do not mat­ter and we don’t think any­one reads any­thing closely.

Nothing is added to the text and there are no hid­den char­ac­ters;

Nothing is added to the text and there are no hid­den char­ac­ters;

This would have been worth clar­i­fy­ing at the out­set.

Watermarking won’t be spe­cific to Claude. As of August 2, the EU re­quires AI providers serv­ing its mar­ket to mark AI-generated con­tent. Other ma­jor model de­vel­op­ers have signed the same Code of Practice and will be im­ple­ment­ing their own wa­ter­marks.

Watermarking won’t be spe­cific to Claude. As of August 2, the EU re­quires AI providers serv­ing its mar­ket to mark AI-generated con­tent. Other ma­jor model de­vel­op­ers have signed the same Code of Practice and will be im­ple­ment­ing their own wa­ter­marks.

No other AI provider has stated that they will ap­ply such mark­ing, adul­ter­at­ing all gen­er­ated text, out­side the EU.

Take the sen­tence The weather to­day was cold and…”. The next word is very un­likely to be sugary.” But it is quite likely to be overcast” or grey.” Under most cir­cum­stances, it does­n’t mat­ter much to the reader which of these lat­ter two words the model ul­ti­mately chooses — the mean­ing of the sen­tence is largely the same ei­ther way. In cases like this, the choice is set­tled by a ran­dom num­ber.

Take the sen­tence The weather to­day was cold and…”. The next word is very un­likely to be sugary.” But it is quite likely to be overcast” or grey.” Under most cir­cum­stances, it does­n’t mat­ter much to the reader which of these lat­ter two words the model ul­ti­mately chooses — the mean­ing of the sen­tence is largely the same ei­ther way. In cases like this, the choice is set­tled by a ran­dom num­ber.

Arguing that grey vs. over­cast doesn’t mat­ter much to the reader” is the crux of my ar­gu­ment that this en­tire en­deavor is a per­verse adul­ter­ation of what it means to write — or to read. That it’s sub­tle in some ways makes it more per­verse, be­cause it’s sneaky.

In in­ter­nal test­ing, we’ve seen no im­pact of wa­ter­mark­ing on the con­tent, level of cre­ativ­ity, or read­abil­ity of Claude’s text. In the SynthID-Text pa­per, which in­tro­duced the tech­nique we use, Google DeepMind tested this im­pact by serv­ing a model that used wa­ter­mark­ing to a por­tion of their Gemini traf­fic and com­par­ing thumbs-up and thumbs-down rat­ings. They found no sta­tis­ti­cally sig­nif­i­cant dif­fer­ences from the un­wa­ter­marked model. And in a con­trolled study, hu­man raters com­par­ing wa­ter­marked and un­wa­ter­marked an­swers side-by-side saw no dif­fer­ence in qual­ity.

In in­ter­nal test­ing, we’ve seen no im­pact of wa­ter­mark­ing on the con­tent, level of cre­ativ­ity, or read­abil­ity of Claude’s text. In the SynthID-Text pa­per, which in­tro­duced the tech­nique we use, Google DeepMind tested this im­pact by serv­ing a model that used wa­ter­mark­ing to a por­tion of their Gemini traf­fic and com­par­ing thumbs-up and thumbs-down rat­ings. They found no sta­tis­ti­cally sig­nif­i­cant dif­fer­ences from the un­wa­ter­marked model. And in a con­trolled study, hu­man raters com­par­ing wa­ter­marked and un­wa­ter­marked an­swers side-by-side saw no dif­fer­ence in qual­ity.

See above for my ar­gu­ment that this thumbs-up/​thumbs-down data is ab­solutely worth­less in eval­u­at­ing whether the SynthID-style word-bias wa­ter­mark­ing makes text worse. By de­f­i­n­i­tion it must make text worse, un­less the un­der­ly­ing LLM mod­el’s scor­ing is wrong, be­cause the na­ture of the wa­ter­mark­ing al­go­rithm re­quires it to some­times in­crease the prob­a­bil­ity of se­lect­ing a worse word choice and de­crease the prob­a­bil­ity of se­lect­ing the mod­el’s best choice. It’s only a ques­tion of how much worse. What Google’s thumb-count­ing data shows is only that it is­n’t so much worse as to make Gemini users click the thumbs-down but­ton.

Watermarking does­n’t change the mean­ing or ex­pe­ri­ence for the per­son read­ing it, but if you wanted to check af­ter the fact whether the text was likely gen­er­ated by Claude, the wa­ter­mark al­lows you to do so.

Watermarking does­n’t change the mean­ing or ex­pe­ri­ence for the per­son read­ing it, but if you wanted to check af­ter the fact whether the text was likely gen­er­ated by Claude, the wa­ter­mark al­lows you to do so.

No, it does not. Because the en­tire scheme is tied to se­cret keys held only by the AI provider, it only al­lows Anthropic, not you”, to check any­thing.

When Claude proof­reads text writ­ten by a per­son, what it gives back has gen­er­ally only been lightly edited; be­cause nearly all the words are the per­son’s, there’s very lit­tle (if any­thing) for the wa­ter­mark to at­tach to. Depending on the length of the text and how heav­ily Claude has edited it, those changes might not be enough to make Claude’s in­volve­ment de­tectable. The more Claude writes, the more de­ci­sions it has to make, and the more space there is for a wa­ter­mark.

When Claude proof­reads text writ­ten by a per­son, what it gives back has gen­er­ally only been lightly edited; be­cause nearly all the words are the per­son’s, there’s very lit­tle (if any­thing) for the wa­ter­mark to at­tach to. Depending on the length of the text and how heav­ily Claude has edited it, those changes might not be enough to make Claude’s in­volve­ment de­tectable. The more Claude writes, the more de­ci­sions it has to make, and the more space there is for a wa­ter­mark.

Translation: No one can ever again use Claude for proof­read­ing their own prose un­less they’re will­ing to risk that the whole thing might be flagged as hav­ing been gen­er­ated by Claude.

For ex­am­ple, once the model has writ­ten 2 + 2 =”, there is a very clear best choice for the next to­ken (if the model is com­plet­ing the sum, there is­n’t an an­swer that’s equally as good as 4”; if it’s talk­ing about George Orwell’s Nineteen Eighty-Four, there is­n’t an an­swer that’s equally as good as 5”). The nudge” of the wa­ter­mark would­n’t be ap­plied here. For the same rea­son, code — which in very many cases has to be ex­act — has gen­er­ally less wa­ter­mark­ing than some other forms of text.

Having said that, in ar­eas where there is an ar­bi­trary choice be­tween par­tic­u­lar words or terms within the code, the wa­ter­mark can be used, such as com­ments within code. But by de­f­i­n­i­tion, it will have a neg­li­gi­ble ef­fect on the ac­tual code pro­duced.

For ex­am­ple, once the model has writ­ten 2 + 2 =”, there is a very clear best choice for the next to­ken (if the model is com­plet­ing the sum, there is­n’t an an­swer that’s equally as good as 4”; if it’s talk­ing about George Orwell’s Nineteen Eighty-Four, there is­n’t an an­swer that’s equally as good as 5”). The nudge” of the wa­ter­mark would­n’t be ap­plied here. For the same rea­son, code — which in very many cases has to be ex­act — has gen­er­ally less wa­ter­mark­ing than some other forms of text.

Having said that, in ar­eas where there is an ar­bi­trary choice be­tween par­tic­u­lar words or terms within the code, the wa­ter­mark can be used, such as com­ments within code. But by de­f­i­n­i­tion, it will have a neg­li­gi­ble ef­fect on the ac­tual code pro­duced.

Translation: We value pre­ci­sion in pro­gram­ming code; we do not in prose.

And it is ex­ceed­ingly rich to cite George Orwell’s Nineteen Eighty-Four, ap­prov­ingly, in the con­text of jus­ti­fy­ing a text adul­ter­ation scheme premised on the no­tion that spe­cific words do not mat­ter. I mean what the ac­tual fuck? Orwell!

Lastly, as to why they’re do­ing this:

We’re im­ple­ment­ing wa­ter­mark­ing to com­ply with the EU AI Act. Anthropic, along with sev­eral other ma­jor AI model providers and around 190 to­tal sig­na­to­ries, signed the EU Code of Practice on Transparency of AI-Generated Content in July 2026. This re­quires AI sys­tem providers to use meth­ods of marking” AI-generated text. We’re ap­ply­ing wa­ter­mark­ing glob­ally at launch be­cause we don’t yet have a durable way to scope it by re­gion.

We’re im­ple­ment­ing wa­ter­mark­ing to com­ply with the EU AI Act. Anthropic, along with sev­eral other ma­jor AI model providers and around 190 to­tal sig­na­to­ries, signed the EU Code of Practice on Transparency of AI-Generated Content in July 2026. This re­quires AI sys­tem providers to use meth­ods of marking” AI-generated text. We’re ap­ply­ing wa­ter­mark­ing glob­ally at launch be­cause we don’t yet have a durable way to scope it by re­gion.

This, from a com­pany that the Financial Times just re­ported is weeks away from an IPO with an in­tended val­u­a­tion of $2 tril­lion, which would make it one of the 10 high­est-val­ued com­pa­nies in the world — as of to­day, plac­ing it at #7, be­tween TSMC ($2.2T) and Broadcom ($1.9T).

This leaves us to be­lieve that one of the fol­low­ing must be true:

It’s per­fectly rea­son­able that a tech­nol­ogy com­pany val­ued on par with Amazon and TSMC is tech­ni­cally in­ca­pable of com­ply­ing with an EU re­gional law only within the EU it­self.1 Not a cause for con­cern at all.

It’s per­fectly rea­son­able that a tech­nol­ogy com­pany val­ued on par with Amazon and TSMC is tech­ni­cally in­ca­pable of com­ply­ing with an EU re­gional law only within the EU it­self.1 Not a cause for con­cern at all.

Anthropic is in over their heads, wields shock­ingly lit­tle con­trol over their own tech stack, and their im­mi­nent IPO is likely to be re­mem­bered only as a new high-wa­ter mark in the manic global AI bub­ble.

Anthropic is in over their heads, wields shock­ingly lit­tle con­trol over their own tech stack, and their im­mi­nent IPO is likely to be re­mem­bered only as a new high-wa­ter mark in the manic global AI bub­ble.

Also, what hap­pens if an­other ma­jor global mar­ket makes it un­law­ful for AI to se­cretly wa­ter­mark gen­er­ated text?

Incident with GitHub.com

www.githubstatus.com

Update

Issues is op­er­at­ing nor­mally.

Posted Aug 17, 2026 – 20:22 UTC

Update

We are con­tin­u­ing to in­ves­ti­gate spo­radic fail­ures af­fect­ing Copilot au­then­ti­ca­tion in some ap­pli­ca­tions. Copilot us­age via the GitHub CLI and GitHub App are un­af­fected.

Posted Aug 17, 2026 – 20:08 UTC

Update

We are con­tin­u­ing to in­ves­ti­gate spo­radic au­then­ti­ca­tion fail­ures. We have par­tially dis­abled au­then­ti­ca­tion to­ken re­tries and have seen im­prove­ment, and we are mon­i­tor­ing im­pact be­fore fully ap­ply­ing this mit­i­ga­tion.

Posted Aug 17, 2026 – 19:13 UTC

Update

API Requests is op­er­at­ing nor­mally.

Posted Aug 17, 2026 – 19:01 UTC

Update

API Requests is ex­pe­ri­enc­ing de­graded avail­abil­ity. We are con­tin­u­ing to in­ves­ti­gate.

Posted Aug 17, 2026 – 18:48 UTC

Update

The degra­da­tion af­fect­ing Git Operations has been mit­i­gated. We are mon­i­tor­ing to en­sure sta­bil­ity.

Posted Aug 17, 2026 – 18:23 UTC

Update

We iden­ti­fied the prob­lem­atic com­po­nent and have taken cor­rec­tive ac­tions, but we are see­ing resid­ual im­pact in the form of spo­radic au­then­ti­ca­tion fail­ures. We are con­tin­u­ing to ap­ply ad­di­tional mit­i­ga­tions and in­ves­ti­gate the re­main­ing im­pact.

Posted Aug 17, 2026 – 18:11 UTC

Update

Issues is ex­pe­ri­enc­ing de­graded per­for­mance. We are con­tin­u­ing to in­ves­ti­gate.

Posted Aug 17, 2026 – 17:36 UTC

Update

We iden­ti­fied the prob­lem­atic com­po­nent and have taken cor­rec­tive ac­tions, but we are see­ing resid­ual im­pact across nu­mer­ous ser­vices. We are con­tin­u­ing to ap­ply ad­di­tional mit­i­ga­tions and in­ves­ti­gate the re­main­ing im­pact.

Posted Aug 17, 2026 – 17:34 UTC

Update

Git Operations is ex­pe­ri­enc­ing de­graded per­for­mance. We are con­tin­u­ing to in­ves­ti­gate.

Posted Aug 17, 2026 – 17:30 UTC

Update

The degra­da­tion af­fect­ing API Requests, Actions, Git Operations, Issues, Pages, Pull Requests and Webhooks has been mit­i­gated. We are mon­i­tor­ing to en­sure sta­bil­ity.

Posted Aug 17, 2026 – 16:59 UTC

Update

We iden­ti­fied the prob­lem­atic com­po­nent and have taken cor­rec­tive ac­tions. There are strong signs of re­cov­ery but we are still work­ing to com­pletely re­store ser­vice, with er­ror rates still re­main­ing slightly el­e­vated. We will post fur­ther up­dates as re­cov­ery con­tin­ues.

Posted Aug 17, 2026 – 16:36 UTC

Update

We are ex­pe­ri­enc­ing high er­ror rates around 20% for web ex­pe­ri­ences and api traf­fic. Archive down­loads and raw repos­i­tory con­tent down­loads are ex­pe­ri­enc­ing an ap­prox­i­mate 50% er­ror rate. SAML and OIDC au­then­ti­ca­tion, SCIM, and Team Sync are also im­pacted. We are still work­ing to iden­tify the root cause and will con­tinue to post up­dates as we learn more and per­form mit­i­ga­tion.

Posted Aug 17, 2026 – 16:16 UTC

Update

We are ex­pe­ri­enc­ing high er­ror rates around 20% for web ex­pe­ri­ences and api traf­fic. Archive down­loads and raw repos­i­tory con­tent down­loads are ex­pe­ri­enc­ing an ap­prox­i­mate 50% er­ror rate. SAML and OIDC au­then­ti­ca­tion, SCIM, and Team Sync are also im­pacted. We are cur­rently per­form­ing mit­i­ga­tions and will post up­dates as we progress.

Posted Aug 17, 2026 – 15:42 UTC

Update

Webhooks is ex­pe­ri­enc­ing de­graded per­for­mance. We are con­tin­u­ing to in­ves­ti­gate.

Posted Aug 17, 2026 – 15:40 UTC

Update

Git Operations is ex­pe­ri­enc­ing de­graded per­for­mance. We are con­tin­u­ing to in­ves­ti­gate.

Posted Aug 17, 2026 – 15:21 UTC

Update

Pages is ex­pe­ri­enc­ing de­graded per­for­mance. We are con­tin­u­ing to in­ves­ti­gate.

Posted Aug 17, 2026 – 15:10 UTC

Update

API Requests is ex­pe­ri­enc­ing de­graded avail­abil­ity. We are con­tin­u­ing to in­ves­ti­gate.

Posted Aug 17, 2026 – 15:01 UTC

Update

Webhooks is ex­pe­ri­enc­ing de­graded avail­abil­ity. We are con­tin­u­ing to in­ves­ti­gate.

Posted Aug 17, 2026 – 14:58 UTC

Update

We are ex­pe­ri­enc­ing high er­ror rates around 20% for web ex­pe­ri­ences and api traf­fic. Archive down­loads and raw repos­i­tory con­tent down­loads are ex­pe­ri­enc­ing an ap­prox­i­mate 50% er­ror rate. SAML and OIDC au­then­ti­ca­tion, SCIM, and Team Sync are also im­pacted. We are cur­rently per­form­ing mit­i­ga­tions based on our in­ves­ti­ga­tion thus far and are mon­i­tor­ing for im­prove­ment.

Posted Aug 17, 2026 – 14:58 UTC

Update

Actions is ex­pe­ri­enc­ing de­graded avail­abil­ity. We are con­tin­u­ing to in­ves­ti­gate.

Posted Aug 17, 2026 – 14:58 UTC

Update

Pull Requests is ex­pe­ri­enc­ing de­graded avail­abil­ity. We are con­tin­u­ing to in­ves­ti­gate.

Posted Aug 17, 2026 – 14:54 UTC

Update

Issues is ex­pe­ri­enc­ing de­graded avail­abil­ity. We are con­tin­u­ing to in­ves­ti­gate.

Posted Aug 17, 2026 – 14:49 UTC

Update

Pull Requests is ex­pe­ri­enc­ing de­graded avail­abil­ity. We are con­tin­u­ing to in­ves­ti­gate.

Posted Aug 17, 2026 – 14:45 UTC

Update

Copilot is ex­pe­ri­enc­ing de­graded avail­abil­ity. We are con­tin­u­ing to in­ves­ti­gate.

Posted Aug 17, 2026 – 14:31 UTC

Update

We are ex­pe­ri­enc­ing high er­ror rates around 20% for web ex­pe­ri­ences and api traf­fic. Archive down­loads and raw repos­i­tory con­tent down­loads are ex­pe­ri­enc­ing an ap­prox­i­mate 50% er­ror rate. SAML and OIDC au­then­ti­ca­tion, SCIM, and Team Sync are also im­pacted. Investigations are on-go­ing and we will con­tinue to pro­vide up­dates as we dis­cover more in­for­ma­tion.

Posted Aug 17, 2026 – 14:24 UTC

Update

We are ex­pe­ri­enc­ing high er­ror rates around 20% for web ex­pe­ri­ences and api traf­fic. Archive down­loads and raw repos­i­tory con­tent down­loads are ex­pe­ri­enc­ing an ap­prox­i­mate 50% er­ror rate. Investigations are on-go­ing into the root cause, and up­dates will con­tinue to be pro­vided as we in­ves­ti­gate.

Posted Aug 17, 2026 – 14:04 UTC

Update

Pull Requests is ex­pe­ri­enc­ing de­graded per­for­mance. We are con­tin­u­ing to in­ves­ti­gate.

Posted Aug 17, 2026 – 13:58 UTC

Update

Issues is ex­pe­ri­enc­ing de­graded per­for­mance. We are con­tin­u­ing to in­ves­ti­gate.

Posted Aug 17, 2026 – 13:46 UTC

Update

We are see­ing an ap­prox­i­mate 20% er­ror rate across nu­mer­ous ex­pe­ri­ences in­clud­ing Pull Requests, Issues, and oth­ers. Investigations are cur­rently un­der way and we will be post­ing up­dates as they be­come avail­able

Posted Aug 17, 2026 – 13:45 UTC

Update

Webhooks is ex­pe­ri­enc­ing de­graded per­for­mance. We are con­tin­u­ing to in­ves­ti­gate.

Posted Aug 17, 2026 – 13:44 UTC

Update

Actions is ex­pe­ri­enc­ing de­graded per­for­mance. We are con­tin­u­ing to in­ves­ti­gate.

Posted Aug 17, 2026 – 13:42 UTC

Update

API Requests is ex­pe­ri­enc­ing de­graded per­for­mance. We are con­tin­u­ing to in­ves­ti­gate.

Posted Aug 17, 2026 – 13:41 UTC

Investigating

Stripe will reportedly acquire AI gateway startup OpenRouter for $7B+

techcrunch.com

In Brief

Posted:

1:57 PM PDT · August 16, 2026

Stripe has fi­nal­ized a deal to ac­quire OpenRouter, ac­cord­ing to a new re­port in Bloomberg.

OpenRouter helps cus­tomers se­lect dif­fer­ent AI mod­els to per­form dif­fer­ent tasks, de­pend­ing on their spe­cific needs and bud­get. The com­pany an­nounced in May that it had raised a $113 mil­lion Series B, at a re­ported $1.3 bil­lion val­u­a­tion. (Investors in­clude Sequoia, Andreessen Horowitz, Menlo Ventures, and Alphabet’s CapitalG.)

At the time, OpenRouter CEO Alex Atallah de­scribed the com­pany as the equiv­a­lent of Stripe for AI, be­cause it pro­vides cus­tomers with a sin­gle ac­cess point for dif­fer­ent sys­tems and pre­vents lock-in. The startup also claimed to have 8 mil­lion global users and to pro­vide ac­cess to more than 400 mod­els.

The Wall Street Journal re­ported last month that Stripe and OpenRouter were in ac­qui­si­tion talks. Now, Bloomberg said those dis­cus­sions have led to a deal price of more than $7 bil­lion.

A Stripe spokesper­son told TechCrunch that the com­pany does not com­ment on ru­mors or spec­u­la­tion.

Topics

Subscribe for the in­dus­try’s biggest tech news

Latest in AI

A Preview of DuckDB v2.0

duckdb.org

Mark Raasveldt and Hannes Mühleisen

2026 – 08-17

| 15 min

TL;DR: DuckDB v2.0 is com­ing this fall. In this post, we pre­view its head­line fea­tures: DuckDB as a server, trig­gers, the VARIANT type, asyn­chro­nous I/O, a new SQL parser, a new stor­age for­mat, and much more.

DuckDB v2.0 will be named Cyanoptera” af­ter the cin­na­mon teal (Anas cyanoptera), a strik­ingly red­dish-brown duck found in the west­ern Americas.

A ma­jor ver­sion bump is not some­thing we do lightly, and it is not just cer­e­mony: v2.0 ships a new SQL parser, a new de­fault stor­age for­mat, a re­worked C API, and a small num­ber of care­fully cho­sen break­ing changes. But above all, it is a fea­ture re­lease, built from over 10,000 com­mits since we re­leased v1.5 in March. Where last year was the year of the lake­house, this re­lease kicks off the year of DuckDB as a server. We pre­viewed many of these fea­tures in the State of the Duck” talk at DuckCon #7, if you pre­fer to watch in­stead of read.

DuckDB is mov­ing rather quickly, and we can only cover a small frac­tion of the changes here. Condensing all new fea­tures down to a short­list is al­ways a fight over what gets in, and yes, we know that what fol­lows is tech­ni­cally a lis­ti­cle (Ten Things Coming to DuckDB v2.0, Number Eight Will Shock You). We are not proud of the for­mat, but it works, so here it is, start­ing with the SQL-level fea­tures and work­ing down into the en­gine.

1. DuckDB as a Server: Quack and CONNECT

DuckDB has been an in-process data­base since day one. But peo­ple have asked us — very per­sis­tently — for a client/​server mode, and we have fi­nally caved. The quack ex­ten­sion im­ple­ments DuckDB’s na­tive pro­to­col for talk­ing to other DuckDBs. It was re­leased as a pre­view shortly be­fore DuckCon #7, grad­u­ates to sta­ble in v2.0, and it is a big part of where DuckDB is headed: any DuckDB process can serve its data­bases over the net­work, and any other DuckDB can at­tach to it and route queries there us­ing the new CONNECT state­ment. For ex­am­ple:

DuckDB server

CALL quack­_serve( to­ken = my_token’ );

quack:

DuckDB client

ATTACH quack:server.example.com’ AS qk (TOKEN my_token’);

CONNECT qk; SELECT count(*) FROM events; — ex­e­cutes on the server, — re­sults stream back DISCONNECT;

CONNECT is the suc­ces­sor to the re­mote.query($$…$$) workaround we showed when Quack was first re­vealed — we looked at that syn­tax and said: no, this can­not be it. And CONNECT is not lim­ited to Quack: it points your ses­sion at any re­mote data­base that sup­ports it, and the new re­mote push­down op­ti­mizer (#22914) ships SQL di­rectly to PostgreSQL and MySQL in­stead of pulling ta­bles over the wire:

CONNECT postgres://localhost/mydb’; SELECT count(*) FROM or­ders; — runs on the PostgreSQL server DISCONNECT;

If you have worked with an­a­lyt­i­cal sys­tems in the past, you may as­sume that DuckDB can­not han­dle trans­ac­tional work­loads. But DuckDB has been built as a trans­ac­tional, multi-con­nec­tion data­base with full MVCC and trans­ac­tion iso­la­tion since day one. Most users just never needed that in a sin­gle-user sce­nario. It turns out DuckDB han­dles trans­ac­tions well: it’s fast enough to com­pete with gen­eral-pur­pose data­bases like PostgreSQL on quite a few work­loads, and the client/​server pat­tern fi­nally lets that ma­chin­ery shine in multi-ten­ant, long-run­ning de­ploy­ments.

Running DuckDB long-term also comes with new chal­lenges, which is why v2.0 pushes on bet­ter met­rics, logs, and ob­serv­abil­ity (see, e.g., the met­rics layer re­work in #22799) that let you look at a DuckDB in­stance and see what it is ac­tu­ally do­ing. People even built stand­alone clients for the Quack pro­to­col within weeks of the pre­view. We thought we were ex­tend­ing DuckDB to talk to other DuckDBs; the world said no, no, no, and built their own clients. Who would have thought.

2. VARIANT Becomes a First-Class Citizen

The VARIANT type shipped in DuckDB v1.5, and the way to think about it is JSON on steroids. Basically, imag­ine if JSON were fast. Like JSON, a VARIANT col­umn can store dif­fer­ently-shaped data in every row. Unlike JSON, it is not a text for­mat: DuckDB au­to­mat­i­cally de­tects the com­mon struc­ture hid­den in your semi-struc­tured data and shreds” it, so it com­presses well in stor­age and ex­e­cutes fast in queries, all with­out you ever de­clar­ing a schema. This makes VARIANT a nat­ural fit for real-time log in­ges­tion, where streams of JSON-ish records share struc­ture but evolve over time.

In v2.0, this pipeline works end to end: shred­ded ex­e­cu­tion straight from stor­age (#20912), ex­trac­tion push­down into scans (#22478), shred­ded VARIANT read­ing and writ­ing for Parquet, and a fam­ily of vari­ant_* func­tions:

CREATE TABLE events (payload VARIANT); INSERT INTO events VALUES (‘{“user”: {“id”: 42, tags”: [“a”, b”]}}’::JSON::VARIANT);

SELECT vari­ant_­type(pay­load), vari­ant_keys(pay­load) FROM events;

SELECT * FROM events WHERE vari­ant_­con­tains(pay­load, {‘user’: {‘id’: 42}}::VARIANT);

Longer term, likely soon af­ter v2.0 (but don’t hold us to it), we plan to back the reg­u­lar JSON type with VARIANT, so ex­ist­ing JSON work­loads get all of these ben­e­fits with­out chang­ing a sin­gle query.

3. Triggers

Triggers have been a long-stand­ing fea­ture re­quest, and DuckDB v2.0 de­liv­ers them in full: BEFORE and AFTER trig­gers, FOR EACH ROW and FOR EACH STATEMENT, tran­si­tion ta­bles via REFERENCING OLD/NEW TABLE, mul­ti­ple trig­gers per event, RETURNING on trig­gered ta­bles, and DROP TRIGGER.

The clas­sic use case is au­dit ta­bles: some­thing hap­pens in the sys­tem, and a trig­ger records what changed. For ex­am­ple:

CREATE TABLE tar­get (id INTEGER, val INTEGER); CREATE TABLE au­dit (id INTEGER, old_­val INTEGER, new_­val INTEGER);

CREATE TRIGGER trg_au­dit AFTER UPDATE ON tar­get REFERENCING OLD TABLE AS o NEW TABLE AS n FOR EACH STATEMENT INSERT INTO au­dit SELECT n.id, o.val, n.val FROM o JOIN n ON o.id = n.id;

INSERT INTO tar­get VALUES (1, 10), (2, 20); UPDATE tar­get SET val = val * 10 WHERE id <= 2; SELECT * FROM au­dit;

Triggers fit nat­u­rally with long-run­ning DuckDB ser­vices, and we are also plan­ning to use them in­ter­nally to build sev­eral up­com­ing fea­tures. They are fully ex­posed at the SQL level too, so you can build your own cool stuff with them.

4. SQL Dialect Additions

As al­ways, DuckDB’s SQL di­alect keeps grow­ing. A few fa­vorites from this re­lease cy­cle:

With NEAREST joins (#24137), top-k sim­i­lar­ity search be­comes a join clause, handy for vec­tor and em­bed­ding work­loads:

SELECT q.user_id, t.prod­uc­t_id FROM users q INNER JOIN prod­ucts t APPROX NEAREST 2 BY SIMILARITY ar­ray_­co­sine_sim­i­lar­ity(q.em­bed­ding, t.em­bed­ding);

DML in­side CTEs (#21634, #21997, #24217) lets you use INSERT, UPDATE, DELETE, and COPY as pipeline steps:

WITH moved AS MATERIALIZED ( DELETE FROM stag­ing RETURNING * ) INSERT INTO archive SELECT * FROM moved;

Nested schemas (#23492, #24222) al­low schemas within schemas:

CREATE SCHEMA fi­nance; CREATE SCHEMA fi­nance.re­ports; CREATE TABLE fi­nance.re­ports.q3 (revenue DECIMAL);

The new vari­able syn­tax (#21194) lets you write $x any­where an ex­pres­sion is al­lowed, no more get­vari­able(…) ver­biage:

SET VARIABLE thresh­old = 100; SELECT * FROM or­ders WHERE amount > $threshold;

The JSON mu­ta­tion func­tions json_set, json_in­sert, json_re­place, and json_re­move (#23786) fi­nally let you mod­ify JSON doc­u­ments in place:

SELECT json_set(‘{“a”:1}‘, $.b’, 2’);

And re­cur­sive CTEs with USING KEY ag­gre­ga­tion (#19481) en­able it­er­a­tive al­go­rithms in pure SQL, backed by the rewrit­ten re­cur­sive CTE en­gine de­scribed be­low:

WITH RECURSIVE tbl(a, b) USING KEY (a, avg(b)) AS ( SELECT 1, 5 UNION SELECT a, b - 1 FROM tbl WHERE b > 0 ) TABLE tbl;

There is more: SQL-standard FETCH FIRST 2 ROWS ONLY (#23533), OVERLAY() (#22456), UNNEST in GROUP BY (#23644), and well-de­fined MERGE / UPDATEFROM se­man­tics for multi-matched rows (#24058).

5. Asynchronous I/O

Interacting with ob­ject stores like S3 is cen­tral to the DuckDB ex­pe­ri­ence: your data has to come from some­where, and it of­ten sits in ob­ject stor­age. DuckDB has long been able to read from ob­ject stores in par­al­lel, but syn­chro­nous ac­cess placed a limit on how fast this could go. DuckDB v2.0 in­tro­duces asyn­chro­nous I/O through­out the en­gine. We de­scribed the de­sign in de­tail in a ded­i­cated blog post.

Thanks to asyn­chro­nous ac­cess, the I/O layer now scales in­de­pen­dently from the query pro­cess­ing layer, which means far more par­al­lelism for re­mote reads and dra­mat­i­cally faster queries on net­work stor­age. Parquet sup­port came first (#23662), with CSV (#23961) and DuckDB’s own file for­mat (#24654) fol­low­ing, along with asyn­chro­nous Parquet writes (#23283) and new MMAP and DIRECT_IO modes (#22988). Local stor­age ben­e­fits a lit­tle too, but net­work stor­age is where you will see the big gains.

6. Faster Queries Across the Board

As with every re­lease, a lot of work went into mak­ing your ex­ist­ing queries faster with­out you do­ing any­thing. To pick some high­lights: par­tial ag­gre­gates are now pushed be­low joins (#22572) and re­dun­dant ag­gre­ga­tions are reused (#24543), the re­cur­sive CTE en­gine has been rewrit­ten (#22211), ag­gre­ga­tions now spill to disk when they out­grow mem­ory (#24499), and the Windows CLI got ap­prox­i­mately 2.2× faster at multi-threaded re­sult ma­te­ri­al­iza­tion (#24036).

How much faster can this get? Here is a mi­crobench­mark you can run on a lap­top: sin­gle-source reach­a­bil­ity over a graph with one mil­lion edges, writ­ten as a plain re­cur­sive CTE.

CREATE TABLE edges AS SELECT (range % 100_000)::INTEGER AS src, ((range * 13 + 7) % 100_000)::INTEGER AS dst FROM range(1_000_000);

WITH RECURSIVE reach­able(node) AS ( SELECT 0 UNION SELECT dst FROM edges, reach­able WHERE src = node ) SELECT count(*) FROM reach­able;

As you can see, DuckDB v2.0 is about 40× faster (!) for the same re­cur­sive query.

Row-group prun­ing has been mas­sively ex­panded: min-max in­dexes (zone maps) and Parquet Bloom fil­ters now skip data for structs, lists, dec­i­mals, UUIDs, IN fil­ters, and even func­tion pred­i­cates:

– these now prune row groups in­stead of scan­ning them: SELECT * FROM logs WHERE con­tains(mes­sage, ERROR); SELECT * FROM t WHERE sub­str(code, 1, 3) = NL-’; SELECT * FROM data/*.parquet’ WHERE id IN (1, 5, 9);

Query plan­ning also be­comes par­ti­tion-aware (#22336). Lakehouse for­mats (DuckLake, Iceberg and plain Hive-partitioned Parquet on S3) are all par­ti­tioned, and ex­ploit­ing that par­ti­tion­ing is of­ten the dif­fer­ence be­tween scan­ning a dataset and skip­ping most of it. In v2.0, the plan­ner and op­ti­mizer take full ad­van­tage of ex­ist­ing par­ti­tion­ing, and par­ti­tioned writes have been re­worked as well (#22225, #22620).

7. Storage Format v2.0

DuckDB v2.0 bumps the de­fault stor­age for­mat ver­sion to v2.0.0 (#22875). The head­line change is buffer-man­aged ART in­dexes (#21458, #23605): in­dexes are no longer pinned in mem­ory, which means large in­dexed ta­bles open in­stantly and their in­dexes are paged in on de­mand.

Column meta­data is now loaded lazily (#22333), so wide ta­bles open faster too. The DICT_FSST string com­pres­sion method is en­abled by de­fault (#23733), deletes are stored com­pactly (#24336), and the stor­age layer per­forms much stronger cor­rup­tion val­i­da­tion on read. In short: data­bases with big in­dexes and wide ta­bles open faster and use far less mem­ory.

8. A Brand New SQL Parser

DuckDB has fa­mously al­ways used a parser de­rived from PostgreSQL’s. We have de­cided that enough is enough: v2.0 ships our own mod­ern, ex­ten­si­ble PEG-based parser (#22194), an idea we first ex­plored in our 2024 post on run­time-ex­ten­si­ble parsers. This change ties into the ex­ten­sion ecosys­tem: ex­ten­sions can now hook into the gram­mar it­self, so ex­pect ex­ten­sions that ex­pose en­tirely new SQL syn­tax. It also brings bet­ter er­ror mes­sages with pre­cise source lo­ca­tions, and the first di­alect com­pat­i­bil­ity mode:

SET di­alec­t_­com­pat­i­bil­i­ty_­mode = spark’;

You should not ac­tu­ally no­tice any­thing from the parser swap as we de­signed it to be com­pat­i­ble with the old one. If you do no­tice, please file an is­sue.

9. Timezones, Calendars, and Collations Without ICU

Timezone-aware time­stamps, cal­en­dars, and col­la­tions in DuckDB have al­ways been pow­ered by the ICU li­brary. ICU is a fine li­brary, but we only ever used a small slice of it, while still car­ry­ing it around in every DuckDB dis­tri­b­u­tion. In v2.0, the ICU li­brary is gone en­tirely: the icu ex­ten­sion now im­ple­ments time­zones, cal­en­dars, and col­la­tions it­self (#24463, #24403), with the time­zone data built di­rectly from the IANA data­base and com­pressed down to around 45 kB. Everything keeps work­ing ex­actly as be­fore:

SELECT 2026 – 08-14 12:00:00’::TIMESTAMPTZ AT TIME ZONE Europe/Paris’; SELECT * FROM names ORDER BY name COLLATE de;

Besides be­ing much smaller and eas­ier to keep up to date, the new im­ple­men­ta­tion is also sim­ply faster. Here’s a quick mi­crobench­mark on a MacBook that con­verts 25 mil­lion time­stamps to a time­zone and fil­ters 5 mil­lion strings with a German col­la­tion:

10. Write Extensions Once, Host Them Yourself

Extensions are one of the best things about DuckDB, but to­day, most of them, in­clud­ing our own, build against the un­sta­ble C++ API. That means ex­ten­sion au­thors have to re-tar­get and re­build for every DuckDB re­lease, and com­mu­nity ex­ten­sions can silently dis­ap­pear when their au­thors stop keep­ing up. DuckDB v2.0 broad­ens the sta­ble C API far enough that ex­ten­sions can be writ­ten once, built once, pub­lished once, and keep work­ing, es­sen­tially un­til the end of time.

To make this sus­tain­able over the long run, the C API is now gen­er­ated from a de­clar­a­tive, ver­sioned spec­i­fi­ca­tion (#24135): every func­tion in duckdb.h, duck­d­b_ex­ten­sion.h, and the ex­ten­sion ABI is de­scribed in YAML in the api_spec/ di­rec­tory, with its full life­cy­cle on record, and CI ver­i­fies the com­mit­ted head­ers against the spec so API and ABI can no longer drift apart. The re­lease also brings uni­fied sym­bol ver­sion­ing (#24435), cus­tom al­lo­ca­tion han­dlers (#23945), and sta­tic link­ing of C API ex­ten­sions into your ap­pli­ca­tion (#22251).

So what does build­ing an ex­ten­sion against the sta­ble C API look like? Here is a com­plete ex­ten­sion: a sin­gle file that reg­is­ters a vec­tor­ized scalar func­tion, com­piled once against duck­d­b_ex­ten­sion.h.

#include duckdb_extension.h”

DUCKDB_EXTENSION_EXTERN

// a scalar func­tion that adds two BIGINTs, one vec­tor at a time sta­tic void AddNumbers(duckdb_function_info info, duck­d­b_­da­ta_chunk in­put, duck­d­b_vec­tor out­put) { idx_t count = duck­d­b_­da­ta_chunk_get_­size(in­put); in­t64_t *a = (int64_t *) duck­d­b_vec­tor_get_­data(duck­d­b_­da­ta_chunk_get_vec­tor(in­put, 0)); in­t64_t *b = (int64_t *) duck­d­b_vec­tor_get_­data(duck­d­b_­da­ta_chunk_get_vec­tor(in­put, 1)); in­t64_t *result = (int64_t *) duck­d­b_vec­tor_get_­data(out­put); for (idx_t row = 0; row < count; row++) { re­sult[row] = a[row] + b[row]; } }

DUCKDB_EXTENSION_ENTRYPOINT(duckdb_connection con, duck­d­b_ex­ten­sion_info info, duck­d­b_ex­ten­sion_ac­cess *access) { duck­d­b_s­calar_­func­tion f = duck­d­b_cre­ate_s­calar_­func­tion(); duck­d­b_s­calar_­func­tion_set_­name(f, add_numbers”); duck­d­b_­log­i­cal_­type big­int = duck­d­b_cre­ate_­log­i­cal_­type(DUCK­D­B_­TYPE­_BIG­INT); duck­d­b_s­calar_­func­tion_ad­d_­pa­ra­me­ter(f, big­int); duck­d­b_s­calar_­func­tion_ad­d_­pa­ra­me­ter(f, big­int); duck­d­b_s­calar_­func­tion_set_re­turn_­type(f, big­int); duck­d­b_de­stroy_­log­i­cal_­type(&big­int); duck­d­b_s­calar_­func­tion_set_­func­tion(f, AddNumbers); duck­d­b_reg­is­ter_s­calar_­func­tion(con, f); duck­d­b_de­stroy_s­calar_­func­tion(&f); re­turn true; }

LOAD ad­d_num­bers; SELECT ad­d_num­bers(40, 2);

For brevity, we skipped NULL han­dling here. See the de­mo_­capi ex­ten­sion for the full ver­sion.

For brevity, we skipped NULL han­dling here. See the de­mo_­capi ex­ten­sion for the full ver­sion.

The bi­nary this com­piles to keeps work­ing across DuckDB ver­sions. You do not need re-tar­get or re­build it every time a new DuckDB ver­sion comes out. And nowa­days, with all the AI tool­ing around, build­ing an ex­ten­sion has never been eas­ier.

So you have writ­ten your ex­ten­sion. But how should you dis­trib­ute it? Until now, DuckDB could only in­stall ex­ten­sions from the built-in repos­i­to­ries (core, core_nightly, com­mu­nity, …). In v2.0, you will be able to reg­is­ter your own trusted repos­i­to­ries (#24777, cur­rently work-in-progress), so an or­ga­ni­za­tion can host and sign its own ex­ten­sions and have them in­stall and load just like the built-in ones:

SET al­low_ex­ten­sion_repos­i­to­ries = allowed’; CREATE EXTENSION REPOSITORY my_repo FROM https://​ex­ten­sions.ex­am­ple.org; INSTALL my_ext FROM my_repo; LOAD my_repo/​my_ext;

A repos­i­tory is a name, a URL pre­fix, and one or more RSA pub­lic keys that are trusted to sign the ex­ten­sions served from it. The pre­fix can point at any­thing DuckDB can read: a lo­cal path, https, s3, you name it. At CREATE time, DuckDB fetches the repos­i­to­ry’s pub­lic keys and pins them into the repos­i­tory de­f­i­n­i­tion, print­ing each key’s SHA-256 fin­ger­print so you can com­pare it against one pub­lished out of band. If you would rather not trust the net­work at all, you can pass the key di­rectly:

CREATE EXTENSION REPOSITORY my_repo FROM s3://my-bucket/extensions’ USING PUBLIC KEY ––-BEGIN PUBLIC KEY––- …’;

Pinned repos­i­to­ries sur­vive restarts, sup­port key ro­ta­tion by trust­ing mul­ti­ple keys, and can be au­dited at any time through the duck­d­b_ex­ten­sion_repos­i­to­ries() table func­tion, or re­moved again with DROP EXTENSION REPOSITORY. Together with the sta­ble C API, the ex­ten­sion story rounds out nicely: write your ex­ten­sion once, sign it, host it wher­ever you like, and INSTALL it any­where.

Bonus: DuckDB Foundation — Advisory Board

Starting this fall, we will add a stake­holder ad­vi­sory board to the DuckDB Foundation. The ad­vi­sory board will pro­vide in­put on the de­vel­op­ment roadmap of DuckDB, DuckLake, and Quack. This al­lows key stake­hold­ers to have a say in the pro­jects’ di­rec­tion.

Final Thoughts

These are only a few high­lights, and this post is only a pre­view. Some de­tails may still shift be­fore the re­lease this fall, and there are many more fea­tures and im­prove­ments that we could not cover here. DuckDB v2.0 will also come with a small set of break­ing changes, in­clud­ing the new de­fault stor­age for­mat and the com­pleted lambda syn­tax tran­si­tion, which we will cover in de­tail in the re­lease an­nounce­ment.

There have been more than 10,000 com­mits by many con­trib­u­tors since we re­leased v1.5. We would like to thank our com­mu­nity for the de­tailed is­sue re­ports, feed­back, and con­tri­bu­tions that shaped this re­lease. If you want a taste be­fore the fall, the pre­view builds have most of these fea­tures to­day, and if some­thing breaks, you know where the is­sue tracker is.

Recent Posts

Thank You for 40 000 Stars on GitHub

The DuckDB team

Asynchronous I/O in DuckDB: Work, Thread, Work

Pedro Holanda

Announcing DuckDB 1.5.5

The DuckDB team

Universal Health Coverage Could Save $1 Trillion and 114,000 Lives Every Year, Yale Study Projects

ysph.yale.edu

By Matt Kristoffersen

August 13, 2026

A sin­gle-payer uni­ver­sal health care sys­tem could cover every American, save more than 100,000 lives a year, and still cost $1 tril­lion less than the sys­tem it would re­place, ac­cord­ing to a new preprint study led by re­searchers at the Yale School of Public Health.

For the study, which has not yet been peer re­viewed, the re­searchers mod­eled what would hap­pen if the United States adopted a na­tional pub­lic in­sur­ance pro­gram like the one pro­posed in the Medicare for All Act. Using 2024 spend­ing, in­sur­ance cov­er­age, and mor­tal­ity data, they es­ti­mate that the uni­ver­sal cov­er­age would re­duce an­nual health ex­pen­di­tures by $1.04 tril­lion, or nearly 20% — even af­ter ac­count­ing for the ad­di­tional care that unin­sured and un­der­in­sured peo­ple would re­ceive.

Healthcare costs have been ris­ing faster than in­fla­tion, and a stag­ger­ing share of that spend­ing is con­sumed by ad­min­is­tra­tive mid­dle­men, soar­ing drug prices, and emer­gency care for con­di­tions that should have been treated ear­lier and for less.Al­i­son Galvani, PhDBurnett and Stender Families Professor of Epidemiology (Microbial Diseases) and Director, Center for Infectious Disease Modeling and Analysis

Healthcare costs have been ris­ing faster than in­fla­tion, and a stag­ger­ing share of that spend­ing is con­sumed by ad­min­is­tra­tive mid­dle­men, soar­ing drug prices, and emer­gency care for con­di­tions that should have been treated ear­lier and for less.

Alison Galvani, PhD

Burnett and Stender Families Professor of Epidemiology (Microbial Diseases) and Director, Center for Infectious Disease Modeling and Analysis

Healthcare costs have been ris­ing faster than in­fla­tion, and a stag­ger­ing share of that spend­ing is con­sumed by ad­min­is­tra­tive mid­dle­men, soar­ing drug prices, and emer­gency care for con­di­tions that should have been treated ear­lier and for less,” said se­nior au­thor Alison Galvani, the Burnett and Stender Families Professor of Epidemiology at YSPH and the di­rec­tor of the Yale Center for Infectious Disease Modeling and Analysis. Medicare for All strips out those sources of waste while pro­vid­ing every­one with health­care, sav­ing over a tril­lion dol­lars and 114,000 lives every year.”

The model iden­ti­fied five ma­jor sources of sav­ings: lower phar­ma­ceu­ti­cal prices, Medicare-level pay­ments to providers, re­duced ad­min­is­tra­tive over­head, less fraud­u­lent billing, and fewer avoid­able emer­gency de­part­ment vis­its and hos­pi­tal­iza­tions. Even un­der more con­ser­v­a­tive as­sump­tions about drug prices and fraud re­duc­tion, the re­searchers pro­jected that the sav­ings would reach at least $663 bil­lion a year. Both fig­ures ac­count for an es­ti­mated $304 bil­lion in ad­di­tional spend­ing to meet un­met med­ical needs, re­im­burse care that now goes un­paid, and pro­vide uni­ver­sal den­tal cov­er­age.

A uni­ver­sal health­care pol­icy would also save tens of thou­sands of lives, the re­searchers pro­ject. They es­ti­mate that ad­e­quate cov­er­age for all could avert about 62,863 deaths an­nu­ally — and that nearly half, or 29,631, would be among peo­ple who al­ready hold in­sur­ance. These are the un­der­in­sured: the more than 45 mil­lion work­ing-age adults whose de­ductibles and cost-shar­ing put care be­yond fi­nan­cial reach any­way. Reversing cov­er­age roll­backs and other health poli­cies en­acted since 2025 would avert a fur­ther 51,311 deaths each year, the re­searchers pro­ject, bring­ing the an­nual to­tal to 114,174.

The study builds on find­ings Galvani, co-au­thor Meagan Fitzpatrick, and other col­leagues pub­lished in The Lancet in 2020, which pro­jected that uni­ver­sal health care would save $450 bil­lion and more than 68,000 lives an­nu­ally. The larger es­ti­mates in the new study re­flect bal­loon­ing health ex­pen­di­tures, a widen­ing gap be­tween com­mer­cial and Medicare pay­ment rates, and new es­ti­mates on re­cent pol­icy changes and the un­der­in­sured.

The au­thors cau­tion that di­rect es­ti­mates of ex­cess mor­tal­ity among un­der­in­sured adults are un­avail­able, re­quir­ing them to model that risk. Their spend­ing analy­sis also does not ac­count for tran­si­tion costs, ad­min­is­tra­tive job losses, or how providers might re­spond to Medicare pay­ment rates.

A sys­tem that cov­ers every­one, costs $1 tril­lion less, and averts more than 100,000 deaths an­nu­ally re­quires no new dis­cov­ery to im­ple­ment, only en­act­ment.

A sys­tem that cov­ers every­one, costs $1 tril­lion less, and averts more than 100,000 deaths an­nu­ally re­quires no new dis­cov­ery to im­ple­ment, only en­act­ment.

Even with those lim­i­ta­tions, the re­searchers ar­gue that the United States al­ready spends enough to pro­vide uni­ver­sal cov­er­age. The prob­lem, they con­clude, is how that money is al­lo­cated.

A sys­tem that cov­ers every­one, costs $1 tril­lion less, and averts more than 100,000 deaths an­nu­ally re­quires no new dis­cov­ery to im­ple­ment, only en­act­ment,” they wrote.

The study’s other au­thors are Abhishek Pandey, se­nior re­search sci­en­tist in epi­demi­ol­ogy (microbial dis­eases); Chad Wells, post­doc­toral re­search as­so­ci­ate; and Yang Ye, as­so­ci­ate re­search sci­en­tist in epi­demi­ol­ogy (microbial dis­eases), at YSPH.

Article outro

Author

Matt Kristoffersen

Featured in this ar­ti­cle

Inside the federal keyword lists that canceled billions in research funding

www.highereddive.com

This au­dio is auto-gen­er­ated. Please let us know if you have feed­back.

Soon af­ter President Donald Trump re­took of­fice, re­searchers and aca­d­e­mics across the coun­try be­gan re­ceiv­ing star­tling news. They were abruptly in­formed, of­ten mid-pro­ject, that their fed­eral grant fund­ing had been ter­mi­nated. Form let­ters bear­ing the news of­ten of­fered few de­tails be­yond that their work no longer aligned with agency pri­or­i­ties.

As part of a law­suit filed by University of California re­searchers, a hand­ful of fed­eral agen­cies con­firmed ear­lier this month in court doc­u­ments that they cut grants if re­searchers’ pro­jects expressed, or were pre­sumed to ex­press, view­points dis­fa­vored by the Administration.”

Those court doc­u­ments of­fer in­sight into how the Trump ad­min­is­tra­tion un­der­took a swift cam­paign to re­voke re­search fund­ing for work re­lated to di­ver­sity, eq­uity and in­clu­sion ef­forts and other sub­jects at odds with its po­lit­i­cal agenda.

Federal agen­cies re­lied heav­ily on key­word searches when ter­mi­nat­ing grants held by UC re­searchers, court doc­u­ments re­vealed. While the searched terms var­ied across agen­cies, they fo­cused on ini­tia­tives fac­ing sig­nif­i­cant con­ser­v­a­tive op­po­si­tion, such as DEI and green en­ergy.

The UC re­searchers al­lege the grant ter­mi­na­tions vi­o­lated their First Amendment rights by tar­get­ing their work based on view­point and sub­ject mat­ter and have asked the fed­eral judge over­see­ing the case to rule the re­vo­ca­tions un­con­sti­tu­tional.

The key­word lists used by four of the fed­eral agen­cies named in the UC law­suit — the National Institutes of Health, the National Science Foundation, the U.S. Department of Defense and the National Endowment for the Humanities — elu­ci­date the wide net cast by the Trump ad­min­is­tra­tion when tar­get­ing grants.

Federal agen­cies tar­geted grants fo­cused on DEI, cli­mate change and other top­ics

Keywords used by four fed­eral agen­cies to iden­tify and ter­mi­nate re­search grants

Of the four agen­cies, NSF used the most key­words by far. It told the court it pulled its search terms list di­rectly from a 2024 re­port re­leased by Texas Sen. Ted Cruz, which al­leged the Biden ad­min­is­tra­tion had politi­cized sci­ence at the agency. NSF also men­tioned, but did not list, terms re­lated to mis­in­for­ma­tion.

The re­port shared hun­dreds of terms it used to screen grants to de­ter­mine whether they were re­lated to DEI or pro­moted neo-Marxist per­spec­tives about en­dur­ing class strug­gle.” The list in­cluded injustice,” ally,” prejudice,” institutional,” discrimination,” historically,” minority,” and traumatic.”

NIH worked off of a shorter but sim­i­lar list, which in­cluded workforce di­ver­sity” and health eq­uity.” The agency noted that it also used various per­mu­ta­tions” of each key­word.

While the Defense Department ter­mi­nated grants us­ing sim­i­lar terms — such as pay eq­uity,” LGBTQ,” and neurodiversity” — it also sought to cut fund­ing for pro­jects re­lated to cli­mate change and green en­ergy.

Grants that ref­er­enced al­ter­na­tive en­ergy sources such as so­lar, wind or ge­ot­her­mal en­ergy were tar­geted for ter­mi­na­tion, as were those that used terms like climate change,” carbon neu­tral­ity,” decarbonization,” pollution con­trol,” air qual­ity man­age­ment” or federal sus­tain­abil­ity,” the de­part­ment said in court doc­u­ments.

Those four agen­cies told the court they used their key­word lists and avoided as­sess­ing each grant in­di­vid­u­ally, in­stead iden­ti­fy­ing awards to can­cel using gen­eral cri­te­ria.”

NSF pro­vided a list of the grants held by UC re­searchers that it cut us­ing this method but did not tie spe­cific search terms to each ter­mi­nated grant.

NEH sim­i­larly pro­vided the court with a list of grants re­voked based on key­words in­clud­ing environmental jus­tice” and transgenderism,” along with diversity, eq­uity, and in­clu­sion” and its vari­ants. But like NSF, NEH did not spec­ify which key­word trig­gered each ter­mi­na­tion, though it did some­times share por­tions of the grant that re­sulted in it be­ing flagged.

NIH, how­ever, did tie spe­cific terms to can­cel­la­tions in some cases.

NIH cut at least $2.6M in grant fund­ing for pro­jects us­ing the phrase structural racism”

Examples of NIH grants can­celed due key­word searches

One pro­ject at the University of California, San Francisco sought to study how racism af­fected the men­tal health of ado­les­cents and young adults who had con­tact with the le­gal sys­tem, as well as racis­m’s im­pacts on sub­stance abuse.

Another, at the University of California, Los Angeles, fo­cused on ges­ta­tional di­a­betes among Asian Americans.

Both were can­celed by NIH — the for­mer was flagged for us­ing the phrase structural racism,” and the lat­ter for health equal­ity.”

Dozens of ad­di­tional pro­jects saw their NIH grants can­celed not be­cause they used cer­tain phrases but be­cause they had di­ver­sity sup­ple­ments — ad­di­tional fund­ing awarded by the agency that aims to boost sup­port for re­searchers from un­der­rep­re­sented and dis­ad­van­taged back­grounds.

UC re­searchers have won two pre­lim­i­nary in­junc­tions in their case, which blocked grant ter­mi­na­tions made via form let­ters across sev­eral fed­eral agen­cies.

NIH told the court it specif­i­cally tar­geted grants awarded to re­searchers af­fil­i­ated with UCs 10 cam­puses. In one ex­am­ple, the agency sus­pended over $500 mil­lion in fund­ing to the University of California, Los Angeles — fund­ing that has since been or­dered re­stored.

GPT 5.6 Sol is the best "vision" model OpenAI ever released

blog.roboflow.com

Last week, OpenAI an­nounced the GPT-5.6 lineup, in­tro­duc­ing the Sol, Terra, and Luna mod­els. During the re­lease stream, the team fo­cused heav­ily on com­puter use, show­ing mod­els ca­pa­ble of nav­i­gat­ing and op­er­at­ing desk­top ap­pli­ca­tions. OpenAI high­lighted UI agents and de­tailed 3D vi­su­al­iza­tions, but both de­pend on stronger vi­sual un­der­stand­ing.

To mea­sure their vi­sion ca­pa­bil­i­ties, we ran the mod­els through our up­com­ing VLM bench­mark, which we plan to re­lease in the next few weeks. The bench­mark cov­ers com­mon vi­sion tasks, in­clud­ing de­tec­tion, count­ing, OCR, and data ex­trac­tion. In this post, we take a closer look at how GPT-5.6 per­forms across each of them.

Sol is clearly the best vi­sion model OpenAI has re­leased so far. The jump is es­pe­cially vis­i­ble in ob­ject de­tec­tion and count­ing, where GPT-5.5 was far be­hind the strongest VLMs. Terra and Luna are not as strong as Sol, but both show mean­ing­ful progress over GPT-5.5.

Test Sol, Terra, and Luna in Roboflow Playground and com­pare their re­sults with mod­els such as Claude Fable 5 and Gemini 3.5 Flash across the same vi­sion tasks.

Roboflow Playground

Object Detection

Detection is where GPT-5.6 shows the clear­est jump. GPT-5.5 scored 13.8 mAP@50 in our bench­mark, while Sol reached 46.2. Terra and Luna fol­lowed closely at 44.7 and 43.3, mov­ing ob­ject de­tec­tion from a ma­jor weak­ness to a prac­ti­cal ca­pa­bil­ity.

Document lay­out de­tec­tion is one of the clear­est strengths of GPT-5.6. Sol han­dled ti­tles, para­graphs, ta­bles, im­ages, and sig­na­tures well. Many doc­u­ment work­flows start with lo­cat­ing the rel­e­vant parts of a page be­fore OCR or data ex­trac­tion be­gins.

GPT-5.6 also per­formed well on dense scenes. The pills and eggs ex­am­ples con­tain many sim­i­lar ob­jects packed closely to­gether, a com­mon weak­ness for VLM-based de­tec­tion. Unlike tra­di­tional de­tec­tors, VLMs gen­er­ate each class la­bel and set of co­or­di­nates as text. As ob­ject count grows, the re­sponse be­comes longer and the risk of missed ob­jects, du­pli­cates, or co­or­di­nate er­rors in­creases. Despite this, Sol de­tected most ob­jects across both scenes.

For the best de­tec­tion re­sults, prompt GPT-5.6 mod­els to re­turn ab­solute XYXY co­or­di­nates in im­age pix­els. This dif­fers from Gemini 3.5 Flash, which per­formed best with YXYX co­or­di­nates nor­mal­ized to a 0 – 1000 range. Using the wrong co­or­di­nate for­mat re­duced GPT-5.6 de­tec­tion per­for­mance by around 15 mAP points in our bench­mark.

In a few cases, GPT-5.6 Sol re­turned boxes in seem­ingly ran­dom parts of the im­age. Many had no over­lap, or al­most no over­lap, with the ground truth. Instead of match­ing the vis­i­ble ob­jects, the boxes of­ten formed un­nat­ural lay­outs, such as straight rows or evenly spaced groups.

We shared those ex­am­ples with OpenAI. Their team con­firmed that Sol be­comes less sta­ble on im­ages around 2,000 by 2,000 pix­els or larger, es­pe­cially at lower rea­son­ing ef­fort. Higher rea­son­ing ef­fort im­proves sta­bil­ity, but also in­creases to­ken use, la­tency, and cost. Resizing or crop­ping large im­ages be­fore send­ing them to the OpenAI API is the most prac­ti­cal workaround.

Object Counting

Counting im­proved across the full GPT-5.6 lineup. Sol scored 73.0% in our bench­mark, up from 64.9% for GPT-5.5, while Terra and Luna reached 67.6% and 66.2%. Luna, the cheap­est model in the lineup, still out­per­formed the pre­vi­ous OpenAI base­line.

As part of the bench­mark, we tested cases re­quir­ing more than spot­ting ob­jects and re­turn­ing a to­tal. Sol counted heav­ily over­lap­ping metal brack­ets, a dif­fi­cult case for both tra­di­tional ob­ject de­tec­tors and VLMs. Sol also counted bul­let holes only in­side se­lected scor­ing zones, show­ing an un­der­stand­ing of both which ob­jects to count and where the rule ap­plied.

Blister packs proved much harder. In sep­a­rate prompts, we asked Sol to count the empty slots and the pills still sealed in­side the pack­age. The re­peated lay­out, re­flec­tions, and small vi­sual dif­fer­ences be­tween filled and empty slots made both tasks dif­fi­cult.

The ab­nor­mal candy ex­am­ple ex­posed a dif­fer­ent type of fail­ure. Sol gave the wrong count, though it is un­clear whether the model mis­counted the can­dies or mis­un­der­stood the tar­get cat­e­gory.

OCR and Data Extraction

OCR per­for­mance stayed close to GPT-5.5. Sol achieved a 90.7% mean sim­i­lar­ity score, only 0.5 points be­hind GPT-5.5 at 91.2%, while Terra and Luna reached 88.8% and 88.4%. The gap was larger in text ex­trac­tion, where Sol scored 82.5% com­pared with 87.6% for GPT-5.5. Luna and Terra fol­lowed at 81.4% and 79.4%.

As part of the bench­mark, we sep­a­rated full tran­scrip­tion from tar­geted ex­trac­tion. OCR asks the model to tran­scribe all vis­i­ble text, while text ex­trac­tion asks for a spe­cific piece of in­for­ma­tion. Sol per­formed well on hand­writ­ten notes in both set­tings, pro­duc­ing a full tran­scrip­tion in one case and ex­tract­ing a re­quested date in an­other.

Sol per­formed well on text em­bed­ded in com­plex vi­sual scenes. It read a tire size se­quence printed along the curved sur­face of a dirty, worn tire. In an­other ex­am­ple, it ex­tracted the live score from a hockey broad­cast and re­turned the an­swer in the re­quested for­mat, test­ing both vi­sual read­ing and in­struc­tion fol­low­ing.

Some sim­ple-look­ing ex­trac­tion tasks still failed. Sol could not read the ex­pi­ra­tion date printed on a blis­ter pack. The text was small, ver­ti­cal, low con­trast, and af­fected by re­flec­tions, which may ex­plain the er­ror.

Trade-offs

The vi­sion gains come with higher to­ken us­age across the GPT-5.6 lineup. The dif­fer­ence mat­ters less in small tests, but be­comes more im­por­tant at scale, where to­ken vol­ume di­rectly in­creases pro­cess­ing costs.

Sol av­er­aged close to 10 sec­onds per im­age in our bench­mark. Terra re­duced that to around 6 sec­onds, while Luna fin­ished in slightly over 5 sec­onds. Luna of­fers the strongest la­tency-qual­ity bal­ance in the lineup, with speed close to Gemini 3.5 Flash while still out­per­form­ing GPT-5.5 on de­tec­tion and count­ing.

In our bench­mark, Sol cost roughly 2.5 cents per im­age, mak­ing it the sec­ond most ex­pen­sive model af­ter Claude Fable 5. Terra re­duced the av­er­age cost to about 1 cent per im­age, while Luna cost less than 0.5 cents.

At 0.8 cents per im­age, Gemini 3.5 Flash is much cheaper than Sol while still lead­ing our de­tec­tion and count­ing bench­marks. This makes it a strong op­tion for data-in­ten­sive work­loads where cost scales across large im­age batches. Roboflow Playground lets you test Sol, Terra, and Luna along­side Claude Fable 5, Gemini 3.5 Flash, and other VLMs on the same tasks.

Takeaways

With GPT-5.6, OpenAI is much closer to the lead­ing VLMs than be­fore. Detection moved from a weak point to a us­able ca­pa­bil­ity, and count­ing im­proved across the full model fam­ily.

There are still clear lim­its. Gemini 3.5 Flash re­mains a bet­ter prac­ti­cal choice for high-vol­ume de­tec­tion and count­ing in our bench­mark, es­pe­cially at its price.

GPT-5.6 shows OpenAI is now tak­ing vi­sion much more se­ri­ously. Sol still has flaws, es­pe­cially around cost, la­tency, and some un­sta­ble de­tec­tion cases, but the progress is hard to ig­nore. For agents, screen un­der­stand­ing, doc­u­ment work­flows, and vi­sual rea­son­ing, this re­lease makes OpenAI a much stronger op­tion than be­fore.

GIMP - Development Update, August 2026

www.gimp.org

For the past few months, we’ve been de­vel­op­ing all kinds of fea­tures for the fu­ture GIMP 3.4 re­lease. We no­ticed re­cently that our changelog was get­ting quite long - a good prob­lem to have!

While there’s been a lot go­ing on in­ter­nally, it’s been a while since we made a pub­lic progress re­port. So we want to share de­tails on some of the new fea­tures and UX im­prove­ments that’ll be avail­able in the first de­vel­op­ment re­lease, GIMP 3.3.2. This won’t be an ex­haus­tive list (we have to save at least some news for the re­lease it­self!) but hope­fully it will give you some in­sight into the cur­rent di­rec­tion and progress of GIMP’s development.

New Project File Format

MyPaint Brush: Spectral Blending

Non-Destructive Editing

PSD Support Improvements

Native File Chooser Dialogs

User Experience and Interface Updates

Assorted Changes and Fixes

What’s Next

New Project File Format¶

The big fo­cus for main­tainer Jehan re­cently has been de­vel­op­ing a new pro­ject file for­mat for GIMP.

XCF has been GIMPs pri­mary pro­ject for­mat since 1997, and it has served many users well. Over time how­ever, we’ve ob­served more and more lim­i­ta­tions of the bi­nary XCF for­mat. Among other is­sues, it does not eas­ily sup­port very large or com­plex pro­jects, such as the multi-page and an­i­ma­tion fea­tures cur­rently planned for GIMP 3.6.

The new pro­ject file for­mat will fol­low a more com­mon zipped XML struc­ture. While the tech­ni­cal de­tails are still be­ing de­signed and im­ple­mented, this change will al­low for faster sav­ing since we’ll only need to up­date parts of the file in­stead of the whole thing each time. It will also set the stage for much de­sired fea­tures such as auto-sav­ing, which will now be much more fea­si­ble.

That said, XCF is not go­ing away! Backwards com­pat­i­bil­ity is im­por­tant to us, and we will con­tinue to sup­port load­ing XCFs in all fu­ture ver­sions of GIMP. (For in­stance, we’re quite proud that a XCF file made by a small com­pany for their logo in 1998 still ren­ders the same way in the lat­est ver­sion of GIMP)

However, go­ing for­ward we will only add sup­port for sav­ing/​load­ing new fea­tures in the new pro­ject file for­mat once it is fi­nal­ized.

MyPaint Brush: Spectral Blending¶

During GIMP 3.2’s de­vel­op­ment, we up­graded to a newer ver­sion of the MyPaint brush en­gine. While this brought new brushes and can­vas in­ter­ac­tions to the MyPaint Brush Tool, one fea­ture that was left out was Spectral Blending.

Spectral Blending sim­u­lates the ef­fects of blend­ing phys­i­cal pig­ments in dig­i­tal art. For ex­am­ple, blend­ing yel­low and blue will pro­duce a green color in­stead of darker yel­low, and blend­ing red and yel­low will cre­ate an or­ange mix.

Fortunately, new con­trib­u­tor Cassidie Grogan picked up the slack and im­ple­mented this fea­ture. There is now a Spectral Blending check­box in the MyPaint Brush Tool Options. If checked, the new blend­ing method is used. You can con­trol the strength of the blend­ing with the Pigment slider.

In ad­di­tion, main­tainer Michael Natterer im­proved the MyPaint Brush pre­view code to dis­play at their full size in­stead of 48x48 pix­els. This fixes an is­sue where the pre­views ap­peared blurry on larger screens.

Non-Destructive Editing¶

Alx Sa has con­tin­ued mak­ing up­dates to our non-de­struc­tive fil­ter code. To list a few:

You can now ap­ply fil­ters non-de­struc­tively to Layer masks! To go along with this, the fil­ter popover has been re­designed by Reju to show the ac­tive fil­ters for both the layer and its mask, so you can in­ter­act with both on the same screen.

The Gradient Tool can now be used non-de­struc­tively! If you check Editable Gradient in the Tool Options, the gra­di­ent you cre­ate will be added to the fil­ter stack like any other ef­fect. You can tog­gle its vis­i­bil­ity, re­arrange its po­si­tion in the fil­ter stack and delete it. You can also edit the gra­di­ent, which will switch back to the Gradient Tool to let you make fur­ther changes.

Filters with­out di­alogs (such as Invert) can now be ap­plied non-de­struc­tively on non-raster lay­ers such as layer groups and link, text, and vec­tor lay­ers.

PSD Support Improvements¶

Normally we list all file for­mat up­dates in a com­bined sec­tion, but there has been so much work done on PSD sup­port (and by so many peo­ple) that we wanted to high­light it in more de­tail.

First, new con­trib­u­tor Frank Teklote has been busy im­prov­ing our com­pat­i­bil­ity with PSDs. His big pro­ject for this re­lease was cre­at­ing a PSD meta­data ex­port pro­ce­dure for TIFFs and JPEGs. This com­ple­ments our ex­ist­ing PSD meta­data im­port pro­ce­dure, mean­ing that if you im­port a JPEG with paths or a TIFF with lay­ers (or cre­ate one in GIMP), that in­for­ma­tion can now be re­tained in the ex­ported im­age.

Another great thing about Frank’s work is that as we con­tinue to im­prove our PSD com­pat­i­bil­ity, the TIFF and JPEG ex­port fea­tures will au­to­mat­i­cally get those up­dates too!

Related to that, Jacob Boerema has im­ple­mented PSD Descriptor im­port sup­port. Most of our cur­rent PSD sup­port has been based on the pub­lic Adobe spec­i­fi­ca­tion. This doc­u­ment was last up­dated in 2019 how­ever, and mod­ern PSDs use a rel­a­tively un­doc­u­mented text for­mat called Descriptors to store many fea­tures.

Now that GIMP can read de­scrip­tors, we’ve be­gun dras­ti­cally im­prov­ing our PSD im­port sup­port. To list just a few up­dates: text lay­ers are now ed­itable, a num­ber of ad­just­ment lay­ers and mod­ern layer styles ap­pear as their GEGL equiv­a­lents, and solid color shapes are im­ported as vec­tor lay­ers. This is an ac­tive area of de­vel­op­ment, in­clud­ing by two of our GSoC stu­dents Akascape and Waris Maqbool. We hope this work will make it eas­ier for GIMP users to in­ter­act with ex­ist­ing PSD projects!

Native File Chooser Dialogs¶

We have al­ways used the file chooser di­a­log pro­vided by the GTK GUI li­brary for peo­ple to find, load, and save files in GIMP. While the file chooser does the job, it of­ten works dif­fer­ently than the native” file chooser on non-GNOME plat­forms like Windows, ma­cOS, and KDE. Additionally there have been some changes to the UI of this di­a­log in GTK3, which has in­spired some strong feed­back in our is­sue tracker!

Therefore, Alx Sa has be­gun port­ing GIMPs file choosers to the native” op­tion pro­vided in GTK3. This means that when you open or save a file, you will see your plat­for­m’s stan­dard file chooser di­a­log in­stead of the GTK di­a­log (unless your plat­form uses that al­ready, in which case there will be no change!)

Many of the sim­ple di­alogs have al­ready been con­verted. Those with more com­plex ad­di­tional fea­tures will re­quire some work­flow re­designs, which we’re still de­vel­op­ing.

User Experience and Interface Updates¶

A lot of new and ex­ist­ing con­trib­u­tors have sub­mit­ted im­prove­ments to GIMPs user in­ter­face and its user ex­pe­ri­ence. We wanted to high­light their ef­forts, and en­cour­age you all to con­tinue shar­ing your feed­back on our de­sign is­sue tracker.

Designer Denis Rangelov has been hard at work up­dat­ing GIMPs UI icons. He recre­ated our layer lock icons to cre­ate a more con­sis­tent look.

He also took on the mon­u­men­tal task of con­vert­ing all 78 of our cur­sor icons to SVG, which will al­low us to scale them for higher res­o­lu­tion dis­plays with­out los­ing qual­ity!

There have been re­ported per­for­mance is­sues when draw­ing or zoom­ing into the can­vas when the can­vas view was ro­tated. New con­trib­u­tor woot000 di­ag­nosed the prob­lem and cre­ated a fix. Now the checkerboard” trans­parency pat­tern no longer ro­tates when the can­vas does, which sig­nif­i­cantly boosts per­for­mance when paint­ing or edit­ing. They also fixed a re­lated is­sue where the checker­board pat­tern would dis­ap­pear when zoom­ing into the can­vas past a cer­tain point.

Gabriele Barbero im­ple­mented a re­design of the Search Action UI which was de­signed by Denis Rangelov. The new lay­out makes the as­so­ci­ated short­cut key more vis­i­ble, and is more con­sis­tent with the menu lay­outs.

Bruno Lopes has been work­ing to fix is­sues with pop-up di­a­log dis­plays on ma­cOS. Since tra­di­tion­ally we have fewer ma­cOS de­vel­op­ers com­pared to other plat­forms, we’re re­ally happy to see im­prove­ments for these users!

New con­trib­u­tor Andreas Vukman im­proved our Pattern dock dis­play. Now smaller pat­terns tile to fill the avail­able space, cre­at­ing a con­sis­tent pre­view for all pat­terns in­stead of hav­ing some pat­terns dis­play with odd amounts of padding. We think it makes the dock look much nicer!

Richard Gitschlag has up­dated the on-can­vas text ed­i­tor to al­low se­lec­tions when you Shift+Click in the text. It should now work sim­i­lar to what you can do in a word proces­sor like Li­bre­Of­fice.

In pre­vi­ous ver­sions of GIMP, you im­ported or ex­ported meta­data from the Metadata Editor by se­lect­ing an op­tion in a drop­down. Ahmed E. Yassin has made this process more in­tu­itive (and more con­sis­tent with the rest of GIMPs UI) by re­plac­ing the drop­down with two but­tons in­stead.

Ondřej Míchal re­viewed sev­eral por­tions of GIMPs UI and re­placed many in­stances of the Spin Entry wid­get with Spin Scale. The Spin Entry wid­get is dif­fi­cult to use when the width is shrunk, so this change im­proves us­abil­ity in many ar­eas of the UI.

Assorted Changes and Fixes¶

Our four GSoC in­terns have been con­tin­u­ing their work since the mid­point up­date. Recently, Waris Maqbool‘s Sharpen fil­ter was merged into GEGL, so it’ll be avail­able in the next GEGL release.

New con­trib­u­tor Dimitriy Ryazantcev has sub­mit­ted sev­eral patches for im­prov­ing our Windows ICO/CUR/ANI sup­port. They’ve al­ready fixed the ren­der­ing for cer­tain 32bit ICO for­mats and made our load­ing and pre­view al­go­rithms bet­ter match the Windows specification.

Estecka has fixed a ren­der­ing is­sue when ap­ply­ing NDE fil­ters on passthrough layer groups, which made the im­age look dif­fer­ent de­pend­ing on whether the group had child lay­ers or not.

New con­trib­u­tor Petr Vorel fixed a bug where press­ing Alt+0 did not open the tenth most re­cent im­age in your his­tory.

Lloyd Konneker, our main Script-fu con­trib­u­tor, fixed a re­gres­sion in third party scripts where the num­ber range for cer­tain pa­ra­me­ters was­n’t shown in the GUI.

Jacob Boerema and Alx Sa have re­sponded to and patched a num­ber of se­cu­rity re­ports about po­ten­tial flaws in some of our im­age plug-ins.

What’s Next¶

There’s more in-progress work that we look for­ward to shar­ing with you all soon!

There is not an of­fi­cial 3.3.2 de­vel­op­ment re­lease yet, as sev­eral roadmap items are still in-progress. If you’re feel­ing re­ally ad­ven­tur­ous and just can’t wait, you can try our nightly” builds. Instructions are un­der the Automatic Development Builds header.

In the mean­time, we are plan­ning to re­lease GIMP 3.2.6 in the com­ing weeks. It is a sta­ble re­lease so it won’t in­clude many of the new fea­tures de­scribed here. However, it will have a num­ber of im­por­tant bug fixes and small im­prove­ments. We’ll dis­cuss these more in the 3.2.6 re­lease news post!

Wiz Red Agent Finds Its Way Into Snowflake’s Internal Jira Through a Flaw in a GitHub Copilot–Assisted PR

www.wiz.io

As part of on­go­ing se­cu­rity re­search con­ducted through Snowflake’s HackerOne vul­ner­a­bil­ity dis­clo­sure pro­gram, Wiz Research’s Red Agent”—an au­tonomous, AI-powered se­cu­rity re­search tool—iden­ti­fied a crit­i­cal GitHub Actions work­flow vul­ner­a­bil­ity in one of Snowflake’s pub­lic repos­i­to­ries.

This in­ci­dent high­lights a new re­al­ity in soft­ware de­vel­op­ment: Critical vul­ner­a­bil­i­ties can still be in­tro­duced and ap­proved within work­flows in­volv­ing AI cod­ing agents, while au­tonomous AI se­cu­rity agents can rapidly dis­cover and ex­ploit them in the wild.

Upon re­spon­si­ble dis­clo­sure on June 23, 2026 by Wiz, Snowflake re­me­di­ated the vul­ner­a­bil­ity on the same day, ro­tated the af­fected cre­den­tial, and ver­i­fied via de­tailed au­dit logs that Wiz was the sole ac­tor dur­ing the ex­po­sure win­dow. Wiz con­firmed that all data ac­cessed dur­ing proof-of-con­cept test­ing was se­curely deleted.

August 17, 2026, 1957 UTC up­date: This blog has been up­dated to clar­ify that Copilot was a co-au­thor that checked the merged PR and code change, and iden­ti­fied it as all-clear with­out notic­ing the crit­i­cal vul­ner­a­bil­i­ties. It’s un­clear whether the code-change was AI-assisted.

Executive Summary

Wiz Red Agent iden­ti­fied a script in­jec­tion vul­ner­a­bil­ity in snowflakedb/​snowflake-con­nec­tor-net. The is­sue al­lowed an unau­then­ti­cated user to ex­e­cute ar­bi­trary com­mands within a GitHub Actions run­ner by open­ing a GitHub is­sue with a spe­cially crafted ti­tle.

Crucially, the vul­ner­a­bil­ity be­came live on June 18, 2026 - just five days be­fore its dis­cov­ery -when PR #1218 was merged. The fi­nal squash com­mit cred­its Copilot Autofix pow­ered by AI as a co-au­thor. The merged PR re­placed the repos­i­to­ry’s san­i­tized in­put pat­tern with di­rect string ex­pan­sion, yet GitHub’s AI-assisted se­cu­rity re­view did not flag the re­sult­ing crit­i­cal vul­ner­a­bil­ity.

Exposure Walk-Through

Discovery

Wiz Red Agent’s CI/CD ca­pa­bil­ity scanned Snowflake’s GitHub or­ga­ni­za­tion and flagged the ji­ra_is­sue.yml Workflow in snowflakedb/​snowflake-con­nec­tor-net as vul­ner­a­ble to script in­jec­tion via un­trusted in­put in run: blocks.

The Code Change

The work­flow trig­gered on is­sues: opened - mean­ing any GitHub user could fire it by open­ing an is­sue - and in­ter­po­lated the at­tacker-con­trolled is­sue ti­tle di­rectly into a shell script:

The sed es­cap­ing runs af­ter GitHub’s tem­plate ex­pan­sion, a sin­gle quote in the ti­tle breaks out of echo …’ and al­lows ar­bi­trary com­mand ex­e­cu­tion.

The in­jectable pat­tern was in­tro­duced just days ear­lier, on June 18, 2026, com­mit 4a1b8ce (PR #1218: SNOW-2069227: Update jira work­flows”) - co-au­thored by Copilot Autofix pow­ered by AI.

It re­moved the repos­i­to­ry’s ex­ist­ing safe pat­tern, which passed the is­sue ti­tle through an env: vari­able and built the JSON pay­load with jq. Instead it used the di­rect ${{ github.event.is­sue.ti­tle }} in­ter­po­la­tion shown above. In other words, an AI autofix” com­mit cre­ated the very in­jec­tion vec­tor.

The Open Security Gate”

The work­flow had an if: con­di­tion that ap­peared pro­tec­tive:

However, on is­sues events, github.event.pul­l_re­quest is al­ways null.

So the con­di­tion re­duces to (null != whitesource-for-github-com[bot]’). This is al­ways true, and every GitHub user passes the gate.

Exploitation

We crafted an is­sue ti­tle that, af­ter tem­plate ex­pan­sion, breaks out of the echo string and ex­fil­trates the Jira cre­den­tials via an out-of-band call­back:

Crucially, when Red Agent’s cicd ca­pa­bil­ity ini­tially at­tempted ex­fil­tra­tion us­ing a stan­dard com­ment char­ac­ter (#), the run­ner re­turned a bash syn­tax er­ror be­cause the com­ment con­sumed the clos­ing par­en­thet­i­cal of TITLE=$(…). Rather than stop­ping or fail­ing, Red Agent:

au­tonomously an­a­lyzed the syn­tax ex­e­cu­tion er­ror

au­tonomously an­a­lyzed the syn­tax ex­e­cu­tion er­ror

ad­justed its pay­load to use ; echo ′ to prop­erly close the shell block, and

ad­justed its pay­load to use ; echo ′ to prop­erly close the shell block, and

suc­cess­fully re­ceived the out-of-band call­back

suc­cess­fully re­ceived the out-of-band call­back

Within sec­onds, our lis­tener re­ceived the call­back from a GitHub Actions run­ner (Azure IP 20.106.182.197) con­tain­ing base64-en­coded cre­den­tials.

Note: Our first at­tempt used # to com­ment out the rest of the line, which caused an un­ex­pected EOF bash er­ror be­cause it also ate the clos­ing ) of TITLE=$(…). The fix was us­ing ; echo ′ to prop­erly close the shell syn­tax.

The ex­fil­trated to­ken au­then­ti­cated as qa@snowflake.net to snowflake­com­put­ing.at­lass­ian.net, grant­ing read ac­cess across Snowflake’s en­gi­neer­ing, se­cu­rity com­pli­ance, and bug bounty track­ing pro­jects.

Remediation & Forensics

Same-Day Patching: Snowflake patched the work­flow on June 23, 2026 (1dc7766, PR #1402), fully restor­ing the safe env: vari­able and jq –arg pars­ing pat­tern.

Same-Day Patching: Snowflake patched the work­flow on June 23, 2026 (1dc7766, PR #1402), fully restor­ing the safe env: vari­able and jq –arg pars­ing pat­tern.

Credential Revocation: The JIRA to­ken in ques­tion was re­voked and ro­tated.

Credential Revocation: The JIRA to­ken in ques­tion was re­voked and ro­tated.

Forensic Verification: Comprehensive au­dit log analy­sis con­firmed that no ex­ter­nal third par­ties ac­cessed the end­point dur­ing the 5-day ex­po­sure win­dow. All anom­alous queries were strictly matched to Wiz’s test­ing IPs.

Forensic Verification: Comprehensive au­dit log analy­sis con­firmed that no ex­ter­nal third par­ties ac­cessed the end­point dur­ing the 5-day ex­po­sure win­dow. All anom­alous queries were strictly matched to Wiz’s test­ing IPs.

Key Takeaways

AI Code Generation Demands Rigorous Oversight: AI cod­ing tools pre­dict code based on prob­a­bilis­tic pat­terns, which can in­ad­ver­tently rein­tro­duce dep­re­cated or in­se­cure shell pat­terns. AI-generated PRs must un­dergo the same sta­tic analy­sis and se­cu­rity scrutiny as hu­man code.

AI Code Generation Demands Rigorous Oversight: AI cod­ing tools pre­dict code based on prob­a­bilis­tic pat­terns, which can in­ad­ver­tently rein­tro­duce dep­re­cated or in­se­cure shell pat­terns. AI-generated PRs must un­dergo the same sta­tic analy­sis and se­cu­rity scrutiny as hu­man code.

Collapsing Discovery Windows: The vul­ner­a­bil­ity was live for only five days be­fore an au­to­mated agent dis­cov­ered and val­i­dated it. Security op­er­a­tions must adapt to a land­scape where au­to­mated dis­cov­ery oc­curs in hours, re­quir­ing rapid patch cy­cles and short-lived cre­den­tials.

Collapsing Discovery Windows: The vul­ner­a­bil­ity was live for only five days be­fore an au­to­mated agent dis­cov­ered and val­i­dated it. Security op­er­a­tions must adapt to a land­scape where au­to­mated dis­cov­ery oc­curs in hours, re­quir­ing rapid patch cy­cles and short-lived cre­den­tials.

Preventing AI Security Regressions: Automated AI as­sis­tants of­ten lack his­tor­i­cal con­text re­gard­ing why spe­cific code pat­terns were cho­sen. In this in­ci­dent, an au­to­mated PR re­moved a safe env: + jq pars­ing pat­tern that had been ex­plic­itly im­ple­mented to pre­vent shell in­jec­tion. Security teams must im­ple­ment Guardrails that block AI agents from re­plac­ing struc­tured data parsers with di­rect string in­ter­po­la­tion.

Preventing AI Security Regressions: Automated AI as­sis­tants of­ten lack his­tor­i­cal con­text re­gard­ing why spe­cific code pat­terns were cho­sen. In this in­ci­dent, an au­to­mated PR re­moved a safe env: + jq pars­ing pat­tern that had been ex­plic­itly im­ple­mented to pre­vent shell in­jec­tion. Security teams must im­ple­ment Guardrails that block AI agents from re­plac­ing struc­tured data parsers with di­rect string in­ter­po­la­tion.

Disclosure Timeline

June 18, 2026 - The vul­ner­a­bil­ity be­came live when PR #1218 was merged, co-au­thored by Copilot Autofix

June 18, 2026 - The vul­ner­a­bil­ity be­came live when PR #1218 was merged, co-au­thored by Copilot Autofix

June 23, 2026 - Wiz iden­ti­fied, ex­ploited, and re­ported vul­ner­a­bil­ity to Snowflake via HackerOne (report #3819931)

June 23, 2026 - Wiz iden­ti­fied, ex­ploited, and re­ported vul­ner­a­bil­ity to Snowflake via HackerOne (report #3819931)

June 23, 2026 - Slack no­ti­fi­ca­tion sent to Snowflake se­cu­rity team

June 23, 2026 - Slack no­ti­fi­ca­tion sent to Snowflake se­cu­rity team

June 23, 2026 (same day) - Snowflake patches the vul­ner­a­ble script-in­jec­tion work­flow (commit 1dc7766, PR #1402), restor­ing the safe env: + jq –arg pat­tern.

June 23, 2026 (same day) - Snowflake patches the vul­ner­a­ble script-in­jec­tion work­flow (commit 1dc7766, PR #1402), restor­ing the safe env: + jq –arg pat­tern.

June 24, 2026 - Jira to­ken ro­tated

June 24, 2026 - Jira to­ken ro­tated

July 25, 2026 - Public dis­clo­sure dead­line (30 days af­ter the June 25 res­o­lu­tion, per Snowflake’s dis­clo­sure pol­icy)

July 25, 2026 - Public dis­clo­sure dead­line (30 days af­ter the June 25 res­o­lu­tion, per Snowflake’s dis­clo­sure pol­icy)

Snowflake’s Response

Snowflake ap­pre­ci­ates Wiz’s re­spon­si­ble re­port­ing of and col­lab­o­ra­tion around these find­ings through our vul­ner­a­bil­ity dis­clo­sure and bug bounty pro­gram, HackerOne. Wiz Research re­ported a se­cu­rity vul­ner­a­bil­ity in one of Snowflake’s pub­lic GitHub repos­i­to­ries. The dis­clo­sure was re­ceived on June 23, 2026, and it was im­me­di­ately in­ves­ti­gated and re­me­di­ated, and our in­ves­ti­ga­tion found no ev­i­dence of unau­tho­rized ac­cess. Protecting our sys­tems re­mains a top pri­or­ity, and we re­main com­mit­ted to con­tin­u­ally strength­en­ing our soft­ware de­vel­op­ment and se­cu­rity prac­tices. We are work­ing to­gether with Wiz to share these learn­ings with the broader in­dus­try to en­cour­age wide­spread adop­tion of these se­cu­rity best prac­tices.

Snowflake ap­pre­ci­ates Wiz’s re­spon­si­ble re­port­ing of and col­lab­o­ra­tion around these find­ings through our vul­ner­a­bil­ity dis­clo­sure and bug bounty pro­gram, HackerOne. Wiz Research re­ported a se­cu­rity vul­ner­a­bil­ity in one of Snowflake’s pub­lic GitHub repos­i­to­ries. The dis­clo­sure was re­ceived on June 23, 2026, and it was im­me­di­ately in­ves­ti­gated and re­me­di­ated, and our in­ves­ti­ga­tion found no ev­i­dence of unau­tho­rized ac­cess. Protecting our sys­tems re­mains a top pri­or­ity, and we re­main com­mit­ted to con­tin­u­ally strength­en­ing our soft­ware de­vel­op­ment and se­cu­rity prac­tices. We are work­ing to­gether with Wiz to share these learn­ings with the broader in­dus­try to en­cour­age wide­spread adop­tion of these se­cu­rity best prac­tices.

To add this web app to your iOS home screen tap the share button and select "Add to the Home Screen".

10HN is also available as an iOS App

If you visit 10HN only rarely, check out the the best articles from the past week.

Visit pancik.com for more.