10 interesting stories served every morning and every evening.

Meta confirms thousands of Instagram accounts were hacked by abusing its AI chatbot

this.weekinsecurity.com

Meta is no­ti­fy­ing thou­sands of peo­ple whose Instagram ac­counts were hi­jacked dur­ing the months-long abuse of the com­pa­ny’s AI chat­bot, which hack­ers re­peat­edly tricked into tak­ing con­trol of a per­son’s ac­count.

In a new data breach no­ti­fi­ca­tion let­ter, seen by this week in se­cu­rity, Meta has re­vealed for the first time how many peo­ple had their ac­counts hi­jacked as part of the long-run­ning hack­ing cam­paign, which was dis­cov­ered ear­lier this week and first re­ported by 404 Media ($) and TechCrunch ($). The num­ber of af­fected ac­counts gives some clar­ity as to how wide­spread this hack­ing cam­paign was, and for how long it op­er­ated.

According to the data breach no­tice filed with Maine’s at­tor­ney gen­er­al’s of­fice late on Friday, Meta no­ti­fied at least 20,225 peo­ple that their ac­counts had been com­pro­mised, in­clud­ing 30 peo­ple in Maine.

The com­pro­mises al­lowed the hack­ers to take over the per­son’s en­tire Instagram and any linked ac­counts, in­clud­ing ob­tain­ing con­tact in­for­ma­tion, dates of birth, and pro­file in­for­ma­tion, as well as the abil­ity to ac­cess the per­son’s posts, di­rect mes­sages, and ac­count ac­tiv­ity, the no­tice reads.

Meta’s no­tice con­firmed that the breach re­lates to a vul­ner­a­bil­ity in an AI-assisted ac­count re­cov­ery sys­tem for Instagram,” which was ex­ploited to perform pass­word re­sets on Instagram user ac­counts.”

As pre­vi­ously re­ported, hack­ers abused a flaw in Meta’s chat­bot that al­lowed any­one to re­set the pass­word of any ac­count that did not have two-fac­tor au­then­ti­ca­tion switched on. The bug tricked the chat­bot into send­ing a ver­i­fi­ca­tion code to an email ad­dress con­trolled by the hacker, rather than the ac­count hold­er’s email ad­dress on file, sim­ply by ask­ing it. The chat­bot com­plied any­way.

The tool it­self worked prop­erly and func­tioned as in­tended; how­ever due to a bug in a sep­a­rate code path, the sys­tem did not prop­erly ver­ify that the email ad­dress pro­vided by the in­di­vid­ual re­quest­ing a pass­word re­set matched the email ad­dress as­so­ci­ated with that user’s Instagram ac­count,” said Meta in its breach no­tice.

As a re­sult, when an in­di­vid­ual pro­vided an email ad­dress not pre­vi­ously as­so­ci­ated with the ac­count, the sys­tem in­cor­rectly sent a pass­word re­set link to that unas­so­ci­ated email rather than re­ject­ing the re­quest. This al­lowed unau­tho­rized third par­ties to re­ceive a pass­word re­set link for ac­counts they did not own,” the com­pany added.

At this point, Meta says, the hack­ers could re­set some­one’s pass­word and take over their ac­count as if they were the right­ful owner.

Meta said that it is unaware” of what, if any, per­sonal in­for­ma­tion was ac­cessed dur­ing the hacks. (An email to Meta’s press line ask­ing for clar­ity on this was un­re­turned as of early Saturday.)

According to Maine’s list­ing, the hacks be­gan around April 17 and lasted un­til this week, when Meta said that it had se­cured the chat­bot. Instagram re­port­edly started no­ti­fy­ing af­fected in­di­vid­u­als ear­lier this week by send­ing a pass­word re­set no­ti­fi­ca­tion, even as some re­ported that the hacks were on­go­ing.

Meta also con­firmed in the no­tice that it alerted users to se­cure their ac­counts, say­ing it instructed im­pacted users to re­set their pass­words and re-au­then­ti­cate through se­cure, ver­i­fied chan­nels.”

Meta said that it has dis­abled the AI chat­bot for now and re­moved the code path that al­lowed the chat­bot to re­set user ac­counts, and said it’s also check­ing other chat­bots across its plat­forms to pre­vent a re­peat in­ci­dent. It’s not yet clear what cir­cum­stances led up to the chat­bot be­ing abused, but comes soon af­ter Meta laid off thou­sands of em­ploy­ees while re­ward­ing top ex­ec­u­tives with stock in­cen­tives, as the com­pany con­tin­ues to dou­ble-down on AI.

~ ~

Thank you so much for read­ing ~this week in se­cu­rity~. If you liked this ar­ti­cle, please share it! Feel free to reach out with any feed­back, ques­tions, or com­ments about this ar­ti­cle: this@weekin­se­cu­rity.com.

Pentagon raised threat of Israeli spying on U.S. to highest level, sources say

www.nbcnews.com

WASHINGTON — The Pentagon is in­creas­ingly con­cerned about Israel ramp­ing up its spy­ing on the U.S., re­cently rais­ing the coun­ter­in­tel­li­gence threat level from America’s top ally in the Middle East to the high­est level, ac­cord­ing to two U.S. of­fi­cials and one for­mer U.S. of­fi­cial.

The Pentagon’s Defense Intelligence Agency in re­cent weeks is­sued the new coun­ter­in­tel­li­gence threat as­sess­ment amid ris­ing ten­sions be­tween Israel and the U.S. over the way for­ward in the war with Iran, the of­fi­cials said. They said the DIA posted an in­ter­nal mes­sage, viewed by one of the cur­rent of­fi­cials, that raised the level for Israel to critical.”

The des­ig­na­tion stems from con­cerns within the Pentagon that Israel is mak­ing a par­tic­u­lar ef­fort to sur­veil top U.S. of­fi­cials to get in­for­ma­tion on the Trump ad­min­is­tra­tion’s in­ter­nal de­lib­er­a­tions and de­ci­sion-mak­ing on the con­flicts in the Middle East, the of­fi­cials said.

The DIA as­sess­ment in­cludes a seven-page doc­u­ment and fea­tures a chart, ac­cord­ing to one of the cur­rent U.S. of­fi­cials. The doc­u­ment says the as­sess­ment of Israel is that its abil­ity to con­duct hu­man es­pi­onage and tech­ni­cal col­lec­tion is at a critical level,” ac­cord­ing to the of­fi­cial.

It also iden­ti­fies a se­ries of spe­cific in­ci­dents that height­ened U.S. con­cerns, the of­fi­cial said.

A spokesper­son for the Israeli Embassy in Washington, D.C., said in a state­ment that it is completely false” that Israel spies on the U.S. Israel does not gather in­tel­li­gence on American en­ti­ties, let alone US gov­ern­ment of­fi­cials,” the spokesper­son said. Israel in­tel­li­gence col­lec­tion ef­forts are aimed at its en­e­mies, not its al­lies. Any claims to the con­trary are ei­ther mis­in­formed or po­lit­i­cally mo­ti­vated.”

The Pentagon de­clined to com­ment.

A White House of­fi­cial said in a state­ment, This en­tire story is false and sourced to some­one who does­n’t have any knowl­edge of what’s go­ing on.”

The Office of the Director of National Intelligence, which over­sees all the U.S. in­tel­li­gence agen­cies in­clud­ing the DIA, did not re­spond to a re­quest for com­ment.

While it is com­mon­place for al­lies and ad­ver­saries across the globe to spy on each other, the cur­rent and for­mer U.S. of­fi­cials said Israel’s re­cent ef­forts have gone well be­yond what is typ­i­cal and ex­pected es­pi­onage. The of­fi­cials did not know if a spe­cific in­ci­dent trig­gered the DIAs de­ci­sion to raise the coun­ter­in­tel­li­gence threat level.

The height­ened alert comes as President Donald Trump and Israeli Prime Minister Benjamin Netanyahu have clashed over the war with Iran and Israel’s mil­i­tary op­er­a­tions in Lebanon, in­clud­ing in a tense phone call this past week, NBC News re­ported. Trump ac­knowl­edged af­ter­ward to re­porters that he called Netanyahu crazy” dur­ing the call as ques­tions mount about whether the two coun­tries’ ob­jec­tives in the Middle East are be­gin­ning to sig­nif­i­cantly di­verge.

Since a cease­fire deal was reached in early April, Trump has been pur­su­ing a diplo­matic deal with Iran to end the war Israel and the U.S. launched on Feb. 28. Israel has pub­licly ex­pressed skep­ti­cism that Iran would abide by any ne­go­ti­ated deal. Netanyahu has pushed for a re­sump­tion of bomb­ing raids against Iran and dis­agreed with Trump, who has pressed him to scale back at­tacks against Hezbollah in Lebanon, ac­cord­ing to Western of­fi­cials.

Israel is keenly in­ter­ested in whether Trump de­cides to re­sume ma­jor com­bat op­er­a­tions against Iran or to end the con­flict, the cur­rent and for­mer U.S. of­fi­cials and out­side ex­perts said.

The most prac­ti­cal out­come for the Pentagon is that U.S. of­fi­cials will use ex­tra cau­tion when trav­el­ing to Israel or vis­it­ing with Israeli of­fi­cials, the cur­rent and for­mer U.S. of­fi­cials said. They said there did not ap­pear to be any im­pact on the high-level in­tel­li­gence-shar­ing that oc­curs on a daily ba­sis be­tween the two coun­tries, par­tic­u­larly as­so­ci­ated with the Iran war.

The U.S. al­ready takes ex­tra pre­cau­tions when vis­it­ing Israel,” one of the cur­rent U.S. of­fi­cials said. They’re well-known to ag­gres­sively col­lect.”

The U.S., like other coun­tries, main­tains elab­o­rate coun­ter­in­tel­li­gence, or spy catcher,” ef­forts to pre­vent and track es­pi­onage by for­eign ad­ver­saries as well as by al­lies and part­ners, seek­ing to safe­guard state se­crets and mon­i­tor at­tempts to re­cruit or co­erce U.S. of­fi­cials. Under U.S. law, the FBI has the lead­ing role in coun­ter­in­tel­li­gence ef­forts, but they also in­volve a range of gov­ern­ment agen­cies and the mil­i­tary.

According to cur­rent and for­mer diplo­mats and for­mer na­tional se­cu­rity of­fi­cials, Israel for years has had a rep­u­ta­tion for ag­gres­sive es­pi­onage even against the U.S., its clos­est ally. It’s a prac­tice that has long raised con­cerns among na­tional se­cu­rity and diplo­matic of­fi­cials, and U.S. in­tel­li­gence of­fi­cials closely mon­i­tor the is­sue, ac­cord­ing to ex­perts and the cur­rent and for­mer U.S. of­fi­cials.

Top U.S. of­fi­cials of­ten take ex­tra care when trav­el­ing to Israel, some­times us­ing burner phones and com­put­ers and tak­ing ex­treme cau­tion when speak­ing in ho­tel rooms dur­ing of­fi­cial trips, the cur­rent and for­mer U.S. of­fi­cials and ex­perts said.

Israel has a hy­per-ag­gres­sive in­tel­li­gence ser­vice,” said Emily Harding, vice pres­i­dent of the Defense and Security Department and di­rec­tor of the in­tel­li­gence, na­tional se­cu­rity and tech­nol­ogy pro­gram at the Center for Strategic and International Studies, a think tank in Washington. They are ex­ceed­ingly in­ter­ested in what we are up to,” Harding said of the Israelis.

In the 1980s, spy­ing by Israel caused a rift with Washington, with U.S. Navy in­tel­li­gence an­a­lyst Jonathan Pollard spend­ing 30 years in prison af­ter he was found to have sold suit­cases of top-se­cret doc­u­ments to Israel.

The U.S. also spies on its al­lies and seeks to gather in­tel­li­gence on for­eign part­ners, as ev­i­denced in 2013 by leaks from in­tel­li­gence con­trac­tor Edward Snowden.

Those leaks showed that the U.S. was eaves­drop­ping on European lead­ers, in­clud­ing then-Ger­man Chancellor Angela Merkel’s mo­bile phone, spark­ing out­rage in Berlin.

The U.S. and Israel re­main close al­lies, and the two coun­tries’ in­tel­li­gence ser­vices have forged a close work­ing re­la­tion­ship over decades. But con­cerns about pos­si­ble Israeli es­pi­onage at such a sen­si­tive mo­ment — when the two gov­ern­ments are not in full agree­ment about the war with Iran — carry the risk of un­der­min­ing trust be­tween the two coun­tries, two ad­di­tional for­mer U.S. of­fi­cials said.

rs - an accurate VHS video effect

ntsc.rs

ntsc-rs is a free, open-source video ef­fect which ac­cu­rately em­u­lates ana­log TV and VHS ar­ti­facts.

Other pop­u­lar ef­fects eye­ball the look of VHS tapes us­ing sim­ple color lookup ta­bles and over­lays. ntsc-rs uses al­go­rithms that model how NTSC trans­mis­sion and VHS en­cod­ing ac­tu­ally work, based on al­go­rithms de­vel­oped in com­pos­ite-video-sim­u­la­tor, zhuker/​ntsc, and ntscQT.

ntsc-rs is writ­ten in Rust, and is mul­ti­threaded and SIMD-accelerated. Unlike sim­i­lar ef­fects such as ntscQT, it can run in real time at much higher res­o­lu­tions than ac­tual NTSC footage.

ntsc-rs is avail­able not just as a stand­alone and web ap­pli­ca­tion, but also as a plu­gin for After Effects, Premiere, and all OpenFX-compatible soft­ware. This in­cludes DaVinci Resolve, Hitfilm, and Vegas.

LLMs are eroding my software engineering career and I don't know what to do

human-in-the-loop.bearblog.dev

06 Jun, 2026

I’m a soft­ware en­gi­neer, com­plet­ing 10 years of pro­fes­sional ex­pe­ri­ence this year. I started my ca­reer as a web fron­tend en­gi­neer (it was eas­ier for me to de­bug fron­tend code back then, so I chose that path), but shortly tran­si­tioned to (web) back­end and never looked back.

Through a se­ries of co­in­ci­dences, once I stepped into back­end de­vel­op­ment, I ended up work­ing in soft­ware de­vel­op­ment roles in the do­mains of fi­nance, book­keep­ing and pay­ment pro­cess­ing, where I had great au­ton­omy and a close and can­did re­la­tion­ship with Product Managers and stake­hold­ers.

I learnt a lot about the do­main and how to ef­fec­tively write pro­grams for it: PCI com­pli­ance, dou­ble-en­try ledgers, es­crows, rec­on­cil­i­a­tion, pay­ment life­cy­cles, bank trans­fer idem­po­tency, etc.

It was, then, ob­vi­ous that I should fo­cus my ca­reer on be­com­ing an ex­pert on that do­main to stand out as a pro­fes­sional and dif­fer­en­ti­ate my­self in a field that showed signs of an in­creas­ing need for do­main spe­cial­ists.

The first pil­lar to erode: do­main-spe­cific knowl­edge

Last year, I got hired by a com­pany in the fi­nance work­space. So far, I had worked on com­pa­nies that do have a strong pay­ment and fi­nance com­po­nent to their op­er­a­tions/​of­fer­ings, but that were not solely fi­nance-fo­cused com­pa­nies.

That com­pany also em­braced AI whole­heart­edly, so I got ChatGPT and Claude Enterprise ac­counts from day one and was en­cour­aged to use them for my re­search, ex­plo­ration, and even cod­ing, al­beit with a warn­ing that I should still re­view and own every sin­gle line that made it into pro­duc­tion.

One of my first pro­jects in­volved re­work­ing the legacy on­line pay­ment sys­tem, which was a mess. They hired me for (among other things) my pre­vi­ous ex­pe­ri­ence in build­ing that and trusted me with the task.

Different from the other com­pa­nies I had worked for so far, they wanted the Design Docs” I write be­fore cod­ing to be read­able by both en­gi­neers and prod­uct man­agers - so they should­n’t be a tech­ni­cal deep dive and more of an ar­chi­tec­tural view. I wrote my first one with min­i­mal AI as­sis­tance - I even called LLMs stochastic par­rots” at the time, a view I no longer hold - and de­liv­ered it.

I val­ued my knowl­edge and thought no LLMs could re­place it.

Then my man­ager reached out to me: even though you’re de­liv­er­ing code at a good pace, you’re tak­ing too long to de­liver those Design Docs. Are you us­ing AI? You should use more AI.

No way this will work”, I thought in my head, but agreed. The mod­els at that time were not as good as the ones we have now, but they did pro­vide a good speed-up on my writ­ing and even the de­ci­sion-mak­ing.

And then I started re­al­iz­ing: all the knowl­edge I have ac­cu­mu­lated over the years: the trade-offs be­tween im­ple­men­ta­tions, how ac­quir­ing works, how to struc­ture idem­po­tency to pre­vent dou­ble-charges, every­thing, was be­com­ing use­less. Even though the mod­els still needed some steer­ing, they could con­nect the dots on how to struc­ture such sys­tems, which was the hard­est part that only de­vel­ops in your brain af­ter years of hands-on ex­pe­ri­ence. That was my first shock.

But sure, I thought, they can do that be­cause there’s plenty of ar­ti­cles on the web on how that shit works along with all the tech­ni­cal doc­u­men­ta­tion, and we have blog posts ex­plain­ing how to ap­ply the tech­ni­cal tools to the do­main. For hu­mans, it may take a long time to learn all that, but that’s train­ing data so the mod­els can pick it up.

What the mod­els will never be good at, and that’s where hu­mans will shine, is de­bug­ging! I had ac­cu­mu­lated a good ex­pe­ri­ence de­bug­ging race con­di­tions and dis­trib­uted sys­tems in pro­duc­tion. That was my ticket to long-term em­ploy­a­bil­ity.

The sec­ond pil­lar to erode: de­bug­ging and dis­trib­uted sys­tems

So, af­ter LLMs started get­ting good at writ­ing docs and help­ing plan the ac­tual im­ple­men­ta­tions, they be­came good at cod­ing. It started in the sec­ond half of 2025 with the Claude Code hype, then Codex came and so on. Although I was us­ing LLMs for writ­ing unit tests every day be­fore that, I was­n’t trust­ing them to write the full im­ple­men­ta­tion yet.

The nat­ural next step was to in­tro­duce more AI into writ­ing code. And hon­estly, I liked it. I like ship­ping things to pro­duc­tion and see­ing users happy as much as I like cod­ing, so I was trad­ing one thing that I like for an­other one that I also like, it was fair.

LLMs were be­com­ing good at cod­ing, but it still could­n’t de­bug the mess left be­hind (by then or by the hu­mans), so I still had a role that was big­ger than steer­ing the ro­bot - a ticket to em­ploy­a­bil­ity.

Everything seemed fine.

Then came the MCPs, the agen­tic work­flows and Claude 4.5 and the sky started to fall.

Claude 4.5, to be hon­est, was­n’t that good. It solved like 60% of the bugs given a stack trace and some con­text (a Sentry link with Sentry MCP en­abled was all it took in most cases). Sometimes it gave a so­lu­tion that sounded plau­si­ble but was to­tally wrong.

This time, how­ever, I stopped doubt­ing the ma­chines. I saw bugs that in the past would eas­ily take 1 day of full-time de­bug­ging be­ing one-shot­ted by Claude Code. Of course, not all of them yet, but the pat­tern was clear.

Then came 4.6, 4.7, GPT 5.5, Opus 4.8 and the DataDog MCP… Now I have CLIs that one-shots bugs across dis­trib­uted sys­tems for me. Bugs that I could­n’t solve in the past. Bugs that would take 2 days of full-time de­bug­ging. Bugs across dis­trib­uted sys­tems that lack dis­trib­uted ob­serv­abil­ity. 90% of the bugs are one-shot­ted now, in­clud­ing bizarre race con­di­tions, un­ex­pected cor­ner-cases, third-party in­te­gra­tion is­sues, un­doc­u­mented API edge cases, every­thing. I hardly have to in­ter­vene.

Of course, I’m still em­ploy­able be­cause some­one has to re­view the code and steer the ro­bot. But I’m just an­other off-the-shelf en­gi­neer now. I have no do­main ex­per­tise that an­other Sr. en­gi­neer steer­ing an LLM can­not match. All my fi­nance and pay­ment do­main ex­per­tise, all the de­bug­ging in­tu­ition and dis­trib­uted sys­tem knowl­edge earned through hours of sweat and tears, is now prompt­able.

We were taught that gen­er­al­ists and spe­cial­ists will al­ways have their roles. But now the mar­ket is shap­ing every­one into be­com­ing a gen­er­al­ist. That’s not a bad thing per se, un­til you look un­der the eco­nom­ics of sup­ply and de­mand: if every­one is a gen­er­al­ist, the price of a gen­er­al­ist falls if there’s no de­mand to match. And we all know the de­mand is dry­ing up.

The third pil­lar, the one that has­n’t eroded yet: code qual­ity and ar­chi­tec­ture

I still have one pil­lar stand­ing, though: code qual­ity and soft­ware ar­chi­tec­ture - what’s now be­ing re­duced to be­ing called taste” 1.

Along the course of my ca­reer, I al­ways liked to refac­tor, al­ways prized good code, and ne­go­ti­ated time in the sprint for it. DDD, Hexagonal, Clean Architecture, you know all the buzz­words. I like this topic, I like to dis­cuss the trade-offs and dif­fer­ent ideas on how to shape code­bases. I re­ally like it.

This is the last pil­lar stand­ing. Except that no­body cares any­more.

Agents do a re­ally bad job at keep­ing code­bases or­ga­nized. If you don’t steer them, they’ll hit a cir­cu­lar de­pen­dency is­sue sooner than you think. Will du­pli­cate code. Add un­nec­es­sary com­ments. Mix up pure func­tions and side-ef­fects. Disregard the prin­ci­ples of SOLID.

That should keep hu­mans em­ployed, ex­cept that this skill is now be­ing re­duced to the word taste”. But it’s not just a re­nam­ing, the in­dus­try is mov­ing to a world where code or­ga­ni­za­tion is less im­por­tant.

Sure, hu­mans should steer the agent to pre­vent spaghetti code­bases with cir­cu­lar de­pen­dency graphs. We don’t want F-rated code­bases that are im­pos­si­ble to touch with­out break­ing some­thing. But a C or D? It’s now fine. Nobody needs A or B-grade code­bases any­more be­cause they’re be­ing made for LLMs, not for hu­mans to read.

I don’t want to ar­gue if this is in­her­ently good or bad. If the source code is now writ­ten for ma­chines to read and not hu­mans, it may be ac­tu­ally ok to tar­get them.

But that’s an­other pil­lar of my ex­per­tise that’s erod­ing. A good chunk of the knowl­edge I ac­cu­mu­lated on that topic is not that valu­able any­more. All the time I spent on it - read­ing books, do­ing real-world ex­er­cises, dis­cussing with other en­gi­neers, writ­ing ADRs - is be­com­ing use­less.

What now?

I’m still em­ployed and I see my­self em­ployed (at least in that com­pany) for a fore­see­able fu­ture. But I don’t know what to think about the long-term.

I spent 10 years (even more when you ac­count for non-pro­fes­sion ex­pe­ri­ence) get­ting good at things that are be­com­ing less and less valu­able. My last pil­lar of ex­per­tise is now re­duced to a taste” and will prob­a­bly won’t last long.

And I know that’s not just me. About 8 months ago there was a lay­off at my cur­rent com­pany (not re­lated to AI, ac­cord­ing to them). Some bril­liant ex-cowork­ers were laid off and are still look­ing for jobs. Most of them suf­fer from the same prob­lem I out­lined here: their do­main ex­per­tise is not enough to stand out any­more.

The com­pany is now hir­ing again for a few roles and do­main fa­mil­iar­ity is not a strong dif­fer­en­tia­tor any­more. We used to list Software Engineer - Area”. Now it’s just Software Engineer” and the team as­sign­ment comes af­ter the of­fer is ac­cepted.

Of course, this is good for bril­liant en­gi­neers that never had the chance to get deep into the do­main and now have bet­ter chances at get­ting a job, but it’s also sad to think that other bril­liant en­gi­neers that spent their lives col­lect­ing do­main knowl­edge are now com­pet­ing on the same lane.

The only way out for keep­ing my em­ploy­a­bil­ity in the long-term now seems to be shift­ing my do­main ex­per­tise to some­thing LLMs will not get good at so eas­ily. But what’s left?

I thought about go­ing back to col­lege, learn­ing Math, Statistics, ad­vanced Machine Learning and ap­ply­ing for re­search role at a fron­tier lab. Except that there are no fron­tier labs in my coun­try, the few ones that ex­ist are flood­ing with ap­pli­ca­tions and I have fam­ily mat­ters that makes mov­ing to an­other coun­try dif­fi­cult. By the time I can af­ford to make that jump, RSI may have made re­searchers ob­so­lete.

Maybe I should con­sider trans­form­ing my wood­work­ing hobby into a pro­fes­sion…

See this, this and this for ref­er­ence. Don’t take this as an en­dorse­ment of the con­tent in­side any of these posts.↩

See this, this and this for ref­er­ence. Don’t take this as an en­dorse­ment of the con­tent in­side any of these posts.↩

#ai

#llm

#software en­gi­neer­ing

Moving beyond fork() + exec()

lwn.net

fork() is a rel­a­tively ex­pen­sive sys­tem call; it must copy the en­tire process state (including mem­ory) for the child process. Many op­ti­miza­tions have been made over the years, but a fork is still a fun­da­men­tally costly op­er­a­tion. To make things worse, a fork() call is of­ten im­me­di­ately fol­lowed by an exec(), which will dis­card all of that mem­ory that was so care­fully copied for the child. Attempts (such as vfork()) have been made over the years to op­ti­mize for this case, but the pat­tern still is more ex­pen­sive than it could be.

Spawn tem­plates

Chen’s patch set takes an in­ter­est­ing ap­proach to op­ti­mize the fork() and exec() pat­tern. It is fo­cused on ap­pli­ca­tions that re­peat­edly launch processes run­ning the same ex­e­cutable; imag­ine, for ex­am­ple, a pro­gram that must run Git re­peat­edly to ob­tain in­for­ma­tion about the con­tents of a repos­i­tory. In such cases, the pro­gram could es­tab­lish a tem­plate to ac­cel­er­ate those in­vo­ca­tions, spread­ing the setup cost across mul­ti­ple op­er­a­tions. This tem­plate would be cre­ated with the spawn_tem­plate_cre­ate() sys­tem call:

struct spawn_tem­plate_cre­ate_args { __aligned_u64 flags; __s32 ex­ecfd; __u32 ex­ec_flags; __aligned_u64 file­name; /* Some fields elided */ };

int spawn_tem­plate_cre­ate(struct spawn_tem­plate_cre­ate_args *args, size_t args_­size);

This call will re­turn a file de­scrip­tor rep­re­sent­ing a tem­plate for the ex­e­cutable file, which can be spec­i­fied as ei­ther a file de­scrip­tor (execfd) or an ab­solute path (filename), but not both. To cre­ate the tem­plate, the ker­nel will open the in­di­cated file and cache a bunch of in­for­ma­tion that will al­low a process to run that file more quickly in the fu­ture.

The ap­pli­ca­tion in ques­tion may run a given ex­e­cutable many times, but each in­vo­ca­tion is dif­fer­ent in a num­ber of ways. The de­tails of a spe­cific in­vo­ca­tion must be placed into an in­stance of this struc­ture:

struct spawn_tem­plate_s­pawn_args { __aligned_u64 flags; __aligned_u64 pidfd; __aligned_u64 argv; __aligned_u64 envp; __aligned_u64 ac­tions; __aligned_u64 ac­tion­s_len; __aligned_u64 re­served[4]; };

The argv field is a pointer to the ar­gu­ment list to be passed to the pro­gram, while envp points to its en­vi­ron­ment. Changes to file de­scrip­tors and sig­nal han­dling, in­stead, are passed through ac­tions, which is a pointer to an ar­ray of:

struct spawn_tem­plate_ac­tion { __u32 type; __u32 flags; __s32 fd; __s32 newfd; __aligned_u64 arg; };

If, for ex­am­ple, file de­scrip­tor four should be closed in the child, the as­so­ci­ated spawn_tem­plate_ac­tion struc­ture would have type set to SPAWN_TEMPLATE_ACTION_CLOSE and fd set to four. Other ac­tions ex­ist for du­pli­cat­ing file de­scrip­tors, open­ing files, chang­ing the work­ing di­rec­tory, and chang­ing sig­nal han­dling.

Once the spawn_tem­plate_s­pawn_args struc­ture has been filled in, the new process can be run with:

int spawn_tem­plate_s­pawn(int tem­plate_fd, struct spawn_tem­plate_s­pawn_args *args, int args_­size);

Internally, this sys­tem call fol­lows some­thing close to the nor­mal fork()/​exec() path. Chen is care­ful to point out that all of the nor­mal checks ap­plied when ex­e­cut­ing a new file re­main in place. But the cached in­for­ma­tion in the tem­plate makes the whole process faster than it was be­fore.

How much faster? Benchmark re­sults pro­vided in the cover let­ter show an im­prove­ment of about 2%, which may not seem like a lot, but it may make a dif­fer­ence for ap­pli­ca­tions that fit the ex­pected pat­tern.

Toward posix_s­pawn()

The most de­tailed re­view of this work was posted by Mateusz Guzik, who said: This prob­lem is dear to my heart and I have been pon­der­ing it on and off for some time now. The en­tire fork + exec id­iom is ter­ri­ble and needs to be re­tired”. He pointed out that the fo­cus of the patch set was a bit strange in that it left the fork() part of the prob­lem un­touched. That is where most of the cost lies, he said, so op­ti­miza­tion ef­forts should seek to re­move it from the pic­ture. Rather than copy­ing the cur­rent process, creating a pris­tine process is the way to go”.

Christian Brauner was fa­vor­able to­ward the goal, say­ing: The idea of hav­ing a builder api for exec is­n’t all that crazy”. His sug­ges­tion, though, was that a new API should be built on top of the ex­ist­ing pidfd ab­strac­tion. Without get­ting into any de­gree of de­tail, he said that the right ap­proach would be to cre­ate an op­tion to pidfd_open() to cre­ate an empty process. A se­ries of calls to a new pidfd_­con­fig() sys­tem call would then con­fig­ure this new process as de­sired, set­ting up its en­vi­ron­ment, im­age to ex­e­cute, and more. pidfd_­con­fig() would thus be anal­o­gous to fs­con­fig().

An im­por­tant ob­jec­tive for a new in­ter­face, Brauner said, would be the abil­ity to sup­port an im­ple­men­ta­tion of posix_s­pawn() in user space. posix_s­pawn() is well suited as a re­place­ment for the fork()/​exec() pat­tern; de­vel­op­ers would likely wel­come a na­tive im­ple­men­ta­tion that is­n’t (unlike the cur­rent im­ple­men­ta­tion) hid­ing fork() and exec() un­der the cov­ers.

Chen agreed that the API as broadly sketched out by Brauner seemed bet­ter, and said that fu­ture work would be in that di­rec­tion. So there will be no spawn tem­plates in the Linux ker­nel but, if Chen’s fu­ture work comes to fruition, Linux may fi­nally gain a proper posix_s­pawn() im­ple­men­ta­tion in­stead.

Did you like this ar­ti­cle?? Subscribe now at the spe­cial dis­counted rate to get a lot more like it.

2025 - The 29th IOCCC

www.ioccc.org

Twenty Ninth International Obfuscated C Code Contest

Where to start

See be­low for links to the 2025 win­ning IOCCC en­tries.

Check out the in­dex.html web pages for each win­ning en­try. They have most of the in­for­ma­tion you need to com­pile and run the win­ning pro­gram. Take a look at the win­ning source code and try to fig­ure out how it works. You might also want to check out the au­thor’s re­marks for even more de­tails.

You may down­load all win­ning en­tries in the form of a com­pressed tar­ball for this year’s con­test.

For IOCCC29, the vol­ume and qual­ity of sub­mis­sions were at near-his­toric heights.

IOCCC28 was spec­u­lated to have at­tracted a record num­ber of sub­mis­sions due to the 4-year ab­sence, al­low­ing au­thors to re­fine their sub­mis­sions, re­sult­ing in a higher-than-usual sub­mis­sion qual­ity.

IOCCC29 was the sec­ond con­sec­u­tive con­test af­ter the 2020 – 2024 hia­tus. And yet, the num­ber of sub­mis­sions for IOCCC29 was sim­i­lar to last year’s con­test, and the over­all sub­mis­sion qual­ity re­mained high for this con­test. So per­haps the in­creased sub­mis­sion vol­ume, com­bined with a higher-than-usual sub­mis­sion qual­ity, is due to fac­tors such as im­proved web­site de­sign, in­creased so­cial me­dia pres­ence, au­thors build­ing on the ideas of past win­ning en­tries, and other fac­tors?

Starting with the close of IOCCC28, the pro­ce­dures used for clos­ing the con­test to new sub­mis­sions, the judg­ing process, se­lect­ing the win­ning en­tries, prepar­ing the up­date to the web­site, and the process to cre­ate the live show on the Our Favorite Universe were care­fully doc­u­mented. And while this doc­u­men­ta­tion re­quired ad­di­tional time as well as more ef­fort, the doc­u­men­ta­tion process re­sulted in over­all im­prove­ments to how the IOCCC is run.

A few days af­ter the pre­sen­ta­tion of the win­ning en­tries for IOCCC29 has been made on the Our Favorite Universe YouTube chan­nel. The record­ing of the main show will be di­vided up into in­di­vid­ual seg­ments. Then, each win­ning en­try will be up­dated to in­clude a link to a YouTube seg­ment un­der a new Award pre­sen­ta­tion near the top of the win­ning en­try’s in­dex.html page.

Fun chal­lenge info

We have added fun chal­lenges to this year’s win­ning en­tries com­pe­ti­tion, un­der the Judges’ re­marks” sec­tion. After you fig­ure out what a given win­ning en­try does, we en­cour­age you to at­tempt the fun chal­lenge.

Some of these chal­lenges are eas­ier than oth­ers. In some cases, you’re asked to cre­ate an al­ter­na­tive ver­sion of prog.c or a re­lated file. In some cases you are asked to pro­duce an ex­pla­na­tion about some­thing.

If the fun chal­lenge is still open (check the A fun chal­lenge” sec­tion for the given win­ning en­try), con­sider sub­mit­ting a GitHub pull re­quest as a con­tri­bu­tion.

If the fun chal­lenge is closed, but you think you have a bet­ter so­lu­tion, con­sider sub­mit­ting a GitHub pull re­quest as a con­tri­bu­tion. If the IOCCC Judges agree that you so­lu­tion is bet­ter, we will con­sider it.

If you be­lieve you have an even bet­ter (or im­prove­ment) to win­ning en­try’s fun chal­lenge, please con­sider sub­mit­ting a GitHub pull re­quest, for the IOCCC judges to con­sider.

Rules and Guidelines for this con­test

The fi­nal ver­sions of the IOCCC rules and guide­lines that were in ef­fect for this con­test were:

2025 rules ver­sion 29.15 2025 – 12-02

2025 guide­lines ver­sion 29.08 2025 – 12-02

The IOCCC rules and guide­lines for IOCCC29 rep­re­sented a sub­stan­tial rewrite over pre­vi­ous con­tests, thanks in part to a num­ber of vol­un­teers: giv­ing the IOCCC judges use­ful ed­its, text rewrites, con­sol­i­da­tion, as well as over­all im­proved or­ga­ni­za­tion.

Looking for­ward to the next con­test

We plan to open IOCCC30 to­wards the end of 2026 and have the con­test run for a sim­i­lar amount of time, clos­ing some­time to­wards the end of Q1 2027.

As we per­form the ac­tions needed to open IOCCC30, we plan to in­ter­nally doc­u­ment the process as we did dur­ing the clos­ing of IOCCC29.

About two or three weeks af­ter the IOCCC29 win­ning en­tries have been posted, and we process some of the early pull re­quests against the 2025 di­rec­tory tree, the IOCCC Judges plan to go on an IOCCC va­ca­tion.

We had in­tended to go on an IOCCC va­ca­tion af­ter re­leas­ing the win­ners of IOCCC28, but then the ef­forts to process bug fixes and en­hance­ments to the mkioc­c­cen­try repo took so much time that by the time that repo was sta­ble, it was time to open IOCCC29. Therefore, this time, we plan to wait un­til af­ter the end of our post-IOC­C­C29 IOCCC va­ca­tion be­fore work­ing on any mkioc­c­cen­try repo PRs.

While work­ing on cre­at­ing po­ten­tial write-ups for sub­mis­sions that en­tered the fi­nal round of the set of judg­ing rounds:

A few sub­mis­sions were set aside in the fi­nal round of the fi­nal set of rounds.

We gained an ad­di­tional level of ap­pre­ci­a­tion for a num­ber of the re­main­ing en­tries.

While the win­ning en­try au­thors came from lo­ca­tions of pre­vi­ous win­ning au­thors, this IOCCC29 had an au­thor - jing­p49 from a new lo­ca­tion: Taiwan.

This con­test saw a Hat trick of Hat-tricks by:

Yusuke Endoh: 2025/endoh1, 2025/endoh2, and 2025/endoh3

Nick Craig-Wood: 2025/ncw1, 2025/ncw2, and 2025/ncw3

Don Yang: 2025/yang1, 2025/yang2, and 2025/yang3

Notable and re­mark­able win­ning en­tries of IOCCC29 in­clude, but are not lim­ited to:

2025/cable - Subleq com­puter

2025/cesmoak - Black hole punch­card Fortran

2025/endoh3 - patch/​diff quine

2025/jhshrvdp - Quasi-rogue-like game

2025/jingp49 - Dr. WHO se­quence

2025/ncw1 - GameBoy em­u­la­tor

2025/tompng - Ocean sound gen­er­a­tor

2025/uellenberg - Quine pong

2025/yang2 - Zoltraak en­cod­ing

Those are just a few of the many amaz­ing win­ners of the IOCCC29, so be sure to check out the rest!

As we dis­cussed above, there were quite a few ex­cel­lent sub­mis­sions that did­n’t quite make the fi­nal cut. We truly ap­pre­ci­ate the hard work each au­thor put into their en­tries, but we’re sorry that we can’t award based solely on ef­fort.

We re­ceived many great sub­mis­sions that did­n’t quite make the cut as win­ners. If you sub­mit­ted some­thing for IOCCC29 that did­n’t win, think about pol­ish­ing your code and giv­ing it an­other shot for IOCCC30. Interestingly, more than one win­ner of IOCCC29 was ac­tu­ally an im­proved ver­sion of code that did­n’t win in a pre­vi­ous con­test.

Encouragement for those who did not win this year

We know many of you that sub­mit­ted to the IOCCC put in a ton of ef­fort into your sub­mis­sions for this year’s IOCCC. We can’t just give out awards to every­one. That would mean tak­ing away from the sub­mis­sions that we think are the best and de­serve to win.

Sometimes, a fi­nal round sub­mis­sion might be good enough to be a win­ning IOCCC en­try, only to be beaten by a sim­i­lar, but slightly bet­ter sub­mis­sion. If you think this hap­pened with your sub­mis­sion, con­sider sub­mit­ting an en­hanced ver­sion to the next IOCCC.

PLEASE DO NOT give up hope! There are some sub­mis­sions that have been sub­mit­ted with re­vi­sions, mul­ti­ple times be­fore ris­ing to the level of a win­ning IOCCC en­try. You might also want to try with a dif­fer­ent type of sub­mis­sion al­to­gether for the next IOCCC.

If you’re not plan­ning to im­prove and re­sub­mit your non-win­ning en­try for the next IOCCC, you’re wel­come to pub­lish it.

On com­pil­ing and run­ning win­ning en­tries

Some C com­pil­ers aren’t as great as they could be. If yours is­n’t work­ing well, you might want to try com­pil­ing with an up­dated ver­sion of clang and/​or gcc in­stead.

If you en­counter prob­lems in com­pil­ing and/​or run­ning the win­ning en­tries, see the FAQ on:

Compiling IOCCC en­tries

IOCCC en­try de­pen­den­cies

Problems com­pil­ing en­tries

Running IOCCC en­tries

For ad­di­tional in­for­ma­tion on how to sub­mit fixes, see the FAQ on:

How to sub­mit a fix - how to sub­mit a fix to an en­try

Update au­thor in­for­ma­tion - how to cor­rect or up­date an IOCCC au­thor’s in­for­ma­tion

For even more in­for­ma­tion

Reporting an IOCCC web­site prob­lem

Submitting a fix to the IOCCC web­site

How to con­tact the IOCCC - up-to-date con­tact de­tails

IOCCC FAQ - ad­di­tional in­for­ma­tion on the IOCCC

www.ioccc.org - the pri­mary IOCCC web­site

Winning Entries of 2025 - The 29th IOCCC

Download all win­ning en­tries from 2025

2025/ayu - IMO award

2025/cable - Best imag­i­nary em­u­la­tor

2025/cesmoak - Retro space award

2025/diels-grabsch - Best one liner

2025/dogon - Consistently con­stant award

2025/endoh1 - Most likely to daz­zle

2025/endoh2 - Most likely to shock

2025/endoh3 - Most re­silient

2025/ferguson - Opposite award

2025/howe - Most likely to in­vade

2025/jhshrvdp - Most likely to tele­port

2025/jingp49 - Who won award

2025/kurdyukov - Most likely to count

2025/mattpep - Most ob­fus­cated op­tions

2025/ncw1 - Best real em­u­la­tor

2025/ncw2 - Best frac­tional em­u­la­tor

2025/ncw3 - Best use of Unicode

2025/tompng - Most sooth­ing

2025/uellenberg - Ping pong prize

2025/yang1 - Compound prize

2025/yang2 - Most mag­i­cal word

2025/yang3 - INABIAF award

Jump to: top

zeroserve: a zero-config web server you can script with eBPF

su3.io

Disclaimer: This ar­ti­cle is co-au­thored with GPT-5.5 and Claude Opus 4.8.

ze­roserve is a small, fast, zero-con­fig HTTPS server. You hand it a tar­ball of a web­site and it serves it - over HTTP/2 and TLS 1.3, with hot re­load and a tiny res­i­dent foot­print. The twist is that you can drop eBPF pro­grams into the tar­ball and they run on every re­quest, in user­space, as sand­boxed mid­dle­ware - rewrit­ing, au­then­ti­cat­ing, and rate-lim­it­ing re­quests, or re­verse-prox­y­ing them to a back­end when you want it to act as a gate­way in front of your app.

In short:

Fast: on one core it beats ng­inx across most work­loads - small and large sta­tic files, scripted mid­dle­ware, and small-re­sponse prox­y­ing, all over HTTPS.

Efficient eBPF script­ing: scripts are JIT-compiled to na­tive code and sand­boxed in user­space, cheap enough to run on every re­quest.

Program-as-configuration: your eBPF pro­gram is the whole con­fig­u­ra­tion, de­cid­ing what hap­pens to each re­quest.

io_ur­ing through­out: every net­work and disk op­er­a­tion is sub­mit­ted through io_ur­ing.

Modern TLS in the box: TLS 1.3, HTTP/2, Encrypted Client Hello, SNI cer­tifi­cate se­lec­tion, and JA4 fin­ger­print­ing.

Simple to op­er­ate: serve a whole site from one tar­ball and hot-re­load it (and the TLS ma­te­r­ial) with a SIGHUP.

It’s meant to be an al­ter­na­tive to ng­inx and Caddy, and the de­sign bet is about con­fig­u­ra­tion. Those servers give you a de­clar­a­tive con­fig lan­guage - lo­ca­tion blocks, rewrite rules, map di­rec­tives, try_­files - and then, once the de­clar­a­tive lan­guage hits its lim­its, an op­tional script­ing run­time bolted on the side (Lua, or Caddy’s plu­g­ins). Behavior ends up split across two lay­ers: di­rec­tives that qui­etly grow their own con­trol flow, plus scripts that run some­where in the re­quest life­cy­cle you have to keep in your head.

ze­roserve col­lapses that into one thing. There is no con­fig file. The eBPF pro­gram is the con­fig­u­ra­tion - a sin­gle, or­di­nary, sand­boxed pro­gram that sees every re­quest and de­cides what hap­pens: rout­ing, head­ers, auth, rate lim­it­ing, prox­y­ing. I want the whole re­quest path in one pro­gram I can read top to bot­tom.

One tar­ball, served in place

The whole site is a sin­gle tar file. ze­roserve in­dexes it on load - build­ing a path -> byte-range map - and then serves files by is­su­ing byte-range reads against the tar­ball it­self. Nothing is ever un­packed to disk. The site lives en­tirely in that one file, so there’s no doc­u­ment root for a stray lo­ca­tion rule to ex­pose, and a de­ploy is a sin­gle atomic file swap. To pack­age a di­rec­tory:

ze­roserve –pack ./public > site.tar ze­roserve –addr 0.0.0.0:8080 site.tar

Deploying a new ver­sion is replace the tar­ball and send SIGHUP. The re­load swaps the site, the scripts, and the TLS ma­te­r­ial atom­i­cally, in the same process, with no dropped con­nec­tions:

kil­lall -SIGHUP ze­roserve

All net­work and disk I/O goes through io_ur­ing (via the monoio run­time). Each in­stance is a sin­gle-threaded event loop. That sounds like a lim­i­ta­tion, and per-process it is - but it’s the right shape when your scal­ing unit is more processes”, and it’s why many of them co­ex­ist hap­pily on one box.

Scripting with eBPF, in user­space

This is the part I find most fun. Any .c file you put un­der .zeroserve/scripts/ gets com­piled to an eBPF ob­ject at pack time (with clang and llc) and runs on every re­quest. The eBPF runs en­tirely in user­space: ze­roserve loads the byte­code into a run­time (async-ebpf) in­side its own or­di­nary, un­priv­i­leged process, so the ker­nel’s BPF sub­sys­tem and CAP_BPF stay out of it. async-ebpf JIT-compiles the byte­code to na­tive ma­chine code (it ven­dors uBPF), so your config” runs as na­tive x86 – 64.

A pointer cage does the job the ker­nel ver­i­fier nor­mally would, keep­ing the pro­gram from read­ing or writ­ing mem­ory it should­n’t: every mem­ory ac­cess in the JIT-compiled code is masked into the pro­gram’s own arena, so a stray ac­cess stays con­fined to the scrip­t’s own mem­ory.

The script runs di­rectly on ze­roserve’s sin­gle event loop. To keep one slow script from stalling every other con­nec­tion, the run­time is fully pre­emptible: a timer can in­ter­rupt JIT-compiled na­tive code mid-ex­e­cu­tion and hand con­trol back to the event loop.

The pro­gram­ming model is a chain of scripts, run in sorted file­name or­der, shar­ing a per-re­quest meta­data map. If a script calls zs_re­spond or zs_re­verse_proxy, the chain short-cir­cuits. Here’s a script that runs first and en­riches every re­quest:

#include <zeroserve.h>

ZS_ENTRY zs_u64 en­try(void) { char peer[64]; if (zs_req_peer(peer, sizeof(peer)) <= 0) zs_str­cpy(peer, unknown”);

// pub­lish val­ues for the HTML tem­plate pass zs_meta_set(ZS_STR(“vis­i­tor”), ZS_STR(peer)); // at­tach a header to *every* re­sponse: sta­tic files, zs_re­spond, prox­ied zs_meta_set(ZS_STR(“zs.re­sponse.header.x-served-by”), ZS_STR(“zeroserve-ebpf”)); re­turn 0; }

The meta­data it sets does two things. Keys un­der zs.re­sponse.header.* be­come re­sponse head­ers on every­thing. And other keys feed a tiny tem­plate pass: a <zs-meta>visitor</zs-meta> place­holder in an HTML file gets sub­sti­tuted on the way out. So you get dy­namic-ish sta­tic pages with­out a tem­plate en­gine.

The helper sur­face a script can call is broad:

Request in­spec­tion and mu­ta­tion: read the method, path, query params, head­ers, and peer ad­dress; rewrite the URI or set and re­move head­ers be­fore the re­sponse goes out.

Crypto and en­cod­ing: SHA-256, HMAC-SHA256, base64, hex, and ge­tran­dom.

JSON: parse a re­quest body, build and mu­tate a doc­u­ment tree, and re­ply with zs_j­son_re­spond.

Rate lim­it­ing: per-key to­ken buck­ets keyed on any­thing from a peer IP to an API key, with state that sur­vives hot re­loads.

AWS SigV4: signed Authorization head­ers and pre­signed URLs for talk­ing to S3 and other AWS ser­vices.

OIDC lo­gin: a com­plete re­ly­ing-party flow (Authorization Code + PKCE) that car­ries the en­tire lo­gin ses­sion in sealed XChaCha20-Poly1305 cook­ies, so you can gate a sta­tic site be­hind log in with Google” while the server stays state­less.

A dy­namic end­point is just a script that re­sponds:

ZS_ENTRY zs_u64 en­try(void) { char path[64]; zs_re­q_­path(path, sizeof(path)); if (zs_strcmp(path, /health”) != 0) re­turn 0;

zs_meta_set(ZS_STR(“zs.re­sponse.header.con­tent-type”), ZS_STR(“application/json”)); zs_re­spond(200, ZS_STR(“{"status":"ok"}\n”)); re­turn 0; }

Each script runs un­der a mem­ory-foot­print cap (256 KB by de­fault), the run­time time-slices long-run­ning scripts off the ex­ecu­tor and throt­tles the run­aways, and scripts can even call each other (zs_call) up to a bounded depth. A script that spins for­ever stalls only its own re­quest - the pre­emp­tion timer in­ter­rupts it and the server keeps serv­ing every­one else.

The TLS story un­der­neath is more com­plete than the zero-con­fig fram­ing sug­gests: TLS 1.3 only, ter­mi­nated by BoringSSL, with na­tive Encrypted Client Hello (so the real SNI never ap­pears in clear­t­ext), SNI cer­tifi­cate se­lec­tion from a di­rec­tory, JA4 client fin­ger­print­ing ex­posed to scripts, and a trans­par­ent ECH re­lay mode that byte-for-byte for­wards un­de­crypt­able hand­shakes to a real up­stream so a pro­tected name blends in be­hind a pub­lic one. That’s a lot of trans­port se­cu­rity to ship in a sin­gle zero-con­fig bi­nary.

How fast is it?

I bench­marked ze­roserve against ng­inx 1.26 and Caddy 2.11 over HTTPS on an 8-core Ryzen 7 3700X, each serv­ing the same con­tent with the same self-signed cer­tifi­cate. Because a ze­roserve in­stance is sin­gle-threaded by de­sign, the only fair com­par­i­son is per core: I pinned every server to one CPU with taskset (and held ng­inx to work­er_processes 1 and Caddy to GOMAXPROCS=1; ze­roserve is sin­gle-threaded al­ready) and drove load with wrk -t4 -c100 from other cores, tak­ing the me­dian of three 10-second runs. wrk speaks HTTP/1.1, so these are HTTP/1.1-over-TLS-1.3 num­bers with the hand­shake amor­tized across long-lived keep-alive con­nec­tions: the steady-state cost of serv­ing an al­ready-open HTTPS con­nec­tion.

Small sta­tic file (174 B) - the bread and but­ter of sta­tic sites:

ze­roserve serves small files about 17% faster than ng­inx on a sin­gle core, with a tighter tail. HTML pages, small JSON, CSS - this is the case ze­roserve is tuned for.

Large sta­tic file (100 KB):

All three are close here, with ze­roserve a hair ahead at around 780 MB/s on one core. ng­inx’s usual trump card for large files is send­file(), which splices file pages from the page cache to the socket with zero user­space copies. Under TLS that path goes un­used: the bytes have to be en­crypted in user­space any­way (short of ker­nel TLS, which all three leave off), so every server is bound by the same en­crypt-and-write loop, and ze­roserve’s io_ur­ing read-and-write path is a touch faster at it.

eBPF vs Lua

The ob­vi­ous com­par­i­son for the script­ing is ng­inx + LuaJIT (ngx_http_lua_module), the usual way to run fast code in­side a web server. So I wrote the equiv­a­lent Lua for two cases and put them head to head.

One tun­ing knob mat­ters a lot here. ze­roserve ships with a con­ser­v­a­tive de­fault: it arms the script-pre­emp­tion timer every 2 ms. Fine gran­u­lar­ity makes it quick to throt­tle a mis­be­hav­ing script, but it taxes every well-be­haved one - at the de­fault, eBPF trails ng­inx Lua on a fully dy­namic re­sponse (about 32k req/​s against 41k). Bumping –preempt-timer-interval-ms to 10 re­cov­ers ~40% of script­ing through­put and turns that around:

Per-request header-in­jec­tion mid­dle­ware (script runs, sta­tic file is still served):

Fully dy­namic JSON re­sponse:

At the 10 ms in­ter­val, tuned eBPF wins both cases. On the mid­dle­ware case - a script shap­ing an oth­er­wise-sta­tic re­sponse - it beats ng­inx Lua by about 50%, with a tighter tail. On the fully syn­thetic re­sponse it edges ng­inx’s heav­ily-tuned con­tent_­by_lua too (47k against 41k). Both en­gines com­pile to na­tive code (LuaJIT is a trac­ing JIT; async-ebpf JITs the eBPF through uBPF), and with TLS en­cryp­tion as a shared per-re­quest cost, the tuned eBPF path comes out ahead on through­put. At the 2 ms de­fault, eBPF keeps the mid­dle­ware win but gives up the syn­thetic-re­sponse lead, so I’d run pro­duc­tion scripts at 10 ms.

As a re­verse proxy

Serving files is half the job; the other half is prox­y­ing to a back­end, which is the main rea­son most peo­ple reach for ng­inx or Caddy in the first place. ze­roserve does it from a script - zs_re­verse_proxy(“http://​127.0.0.1:9000) - and keeps a pool of up­stream con­nec­tions (up to 128 per back­end, 30 s idle) and reuses them across re­quests.

Getting a fair fight here takes care: ng­inx’s fa­mous de­fault closes up­stream con­nec­tions af­ter each re­quest, so keep-alive is en­abled ex­plic­itly (keepalive 128, prox­y_http_ver­sion 1.1, and a cleared Connection header), with Caddy reusing con­nec­tions as it does by de­fault. Each proxy ter­mi­nates TLS on a sin­gle core and for­wards to a shared plain­text back­end, a sep­a­rate 2-core server that sus­tains 100k req/​s on its own, so the mea­sure­ment iso­lates the prox­y’s own over­head.

Proxying a small (174 B) re­sponse:

ze­roserve’s pooled io_ur­ing proxy leads here, about 22% ahead of ng­inx (26.5k against 21.8k) and roughly 3.4× Caddy. For the typ­i­cal proxy work­load - for­ward­ing API calls, small JSON, an app server’s HTML - ze­roserve ter­mi­nates TLS and shut­tles the re­quest to the back­end faster than the ref­er­ence im­ple­men­ta­tion.

Large bod­ies tip the bal­ance back. Proxying a 100 KB re­sponse:

Once the prox­ied body is large, ng­inx’s buffer­ing moves bytes more ef­fi­ciently and pulls ahead, with Caddy slot­ting in be­tween and ze­roserve trail­ing. If your prox­ied re­sponses are large, ng­inx is the bet­ter tool; if they’re small and nu­mer­ous, ze­roserve is faster.

Memory

Idle, a sin­gle ze­roserve in­stance sits around 15 MB PSS - more than ng­inx’s ~6 MB, less than Caddy’s ~60 MB. On its own that’s un­re­mark­able. What makes it mat­ter is that the unit is a whole process: when you run a copy per core, they all map the same bi­nary, so the code pages are shared, and each ex­tra process adds lit­tle be­yond its own work­ing set.

ze­roserve is open source on GitHub - try it your­self!

openai.com

Scientists ejected from diabetes conference for distributing journal reprints

arstechnica.com

Five lead­ing sci­en­tists were ousted from the an­nual meet­ing of the American Diabetes Association (ADA) in New Orleans on Friday. Their crime: hand­ing out copies of an ed­i­to­r­ial, pub­lished in the jour­nal Diabetes Care on April 29, sharply crit­i­ciz­ing the Trump ad­min­is­tra­tion’s on­go­ing at­tacks on sci­en­tific re­search.

Those ousted were Steven Kahn, pro­fes­sor of med­i­cine at the University of Washington and ed­i­tor-in-chief of Diabetes Care, who co-au­thored the pub­lished ed­i­to­r­ial; for­mer ADA pres­i­dent Desmond Schatz of the University of Florida, Gainesville; Aaron Kelly, pe­di­atrics proces­sor at the University of Minnesota; Justin Ryder of Northwestern University; and Irl Hirsch, also of the University of Washington. The five were hand­ing out reprints of the ed­i­to­r­ial out­side a room where NIH di­rec­tor Jay Bhattacharya had been sched­uled to speak. Bhattacharya can­celled and an­other NIH of­fi­cial spoke in his stead.

They phys­i­cally grabbed us, forced us out of the con­fer­ence cen­ter, and now are telling us we can no longer at­tend this meet­ing,” Kelly told MedPage Today, which first re­ported the in­ci­dent. They’re tak­ing our lan­yards. It re­ally has come to this in America. Censorship is real. America needs to stand up. Scientists, stand up. Physicians, stand up.”

The ADA con­firmed to MedPage Today that five reg­is­tered sci­en­tists had been re­moved from the meet­ing, claim­ing the sci­en­tists had vi­o­lated the or­ga­ni­za­tion’s code of con­duct for con­fer­ences. These at­ten­dees were es­corted out by our on­site event se­cu­rity be­cause they demon­strated be­hav­ior not con­sis­tent with this code of con­duct,” the ADA me­dia team said in a state­ment. They were re­spect­fully given the op­por­tu­nity to cease this be­hav­ior and chose not to which is why they were es­corted out.”

All at­ten­dees will con­duct them­selves in a pro­fes­sional and re­spect­ful man­ner, free from any form of dis­crim­i­na­tion, ha­rass­ment, or in­tim­i­da­tion,” the code of con­duct states. Inappropriate con­duct, in­clud­ing but not lim­ited to ha­rass­ment; threat­en­ing or un­wel­come phys­i­cal or ver­bal ac­tions; or dis­or­derly or dis­rup­tive con­duct such as protest­ing, will not be tol­er­ated.”

Major P2P issues in Israel and possibly other middle east countries

github.com

I am not sure i am in the right place but we are all out of op­tions here since we could­n’t get any help from the game or steam sup­port.

Since around the 13/03, there is a ma­jor sys­temic prob­lem in every game that uses Steam Networking for P2P games.

For ex­am­ple, in the game Street Fighter 6, when play­ing with one Israeli or an­other with PC to PC the ping be­tween play­ers are ~120ms. when play­ing with European play­ers the ping is around 60 – 80ms which means it works well, it cur­rently af­fects only Israeli play­ers when play­ing PC to PC. Since Street Fighter 6 sup­ports cross-play we have tried play­ing PC to PS5 and the ping is a flaw­less 5 – 10ms.

It cur­rently af­fects all play­ers in Israel, we have a few dozens in our com­mu­nity with sev­eral ISPs, we have of course at­tempted speak­ing to our ISPs and port for­ward­ing and we have found no net­work is­sues on their part. in other P2P games that don’t use steam net­work­ing the is­sue is not ex­is­tent(for ex­am­ple, Tekken 8).

We are cur­rently don’t know what we can do with this is­sue since noth­ing we have done have helped and 120ms is too high for it to be playable in any P2P game.

While I can’t con­firm but heard re­ports from play­ers in other mid­dle east­ern coun­tries like Egypt that they also re­port on this is­sue so it could be re­gion-wide.

To add this web app to your iOS home screen tap the share button and select "Add to the Home Screen".

10HN is also available as an iOS App

If you visit 10HN only rarely, check out the the best articles from the past week.

Visit pancik.com for more.