As in­vestors watched the US stock mar­ket take its biggest one-day dive since the pan­demic in 2020, the mes­sage from the White House to Wall Street was: “Trust in President Trump”.

Trump him­self said mar­kets would “boom” un­der his new tar­iff scheme as the Dow Jones, Nasdaq and S&P plunged over the course of the day.

Twenty-four hours af­ter Trump im­posed min­i­mum 10% im­port tar­iffs world­wide start­ing on 5 April, the US’s key trad­ing part­ners are map­ping out their re­sponse.

Canadian Prime Minister Mark Carney re­tal­i­ated with 25% levies on all ve­hi­cles im­ported across the bor­der, lament­ing that the close re­la­tion­ship with the US is “now over”. In con­trast, Mexican President Claudia Sheinbaum said her coun­try would not re­tal­i­ate in kind.

Meanwhile, the UK has drawn up a 400-page sam­ple list of US goods that could face fu­ture tar­iffs in re­tal­i­a­tion to Trump’s tar­iffs, and though EU lead­ers are yet to pre­sent the bloc’s of­fi­cial re­sponse, many have slammed the tar­iffs as “brutal”.

Eyes will be on the Asian mar­kets in the com­ing hours as east­ern trad­ing floors re­open af­ter ini­tial losses im­me­di­ately fol­low­ing Trump’s an­nounce­ment.

We are now end­ing our live cov­er­age for the night, but will be back with you come morn­ing. In the mean­time, there is plenty about the im­pact of Trump’s tar­iffs on of­fer across BBC News:

Nassim Nicholas Taleb con­trasts frag­ile sys­tems—those that suf­fer se­ri­ous harm un­der volatil­ity—with ro­bust sys­tems that can en­dure stress with min­i­mal dam­age. He ex­tends this con­cept with an­tifrag­ile sys­tems, which can ac­tu­ally ben­e­fit from dis­or­der, but we’ll fo­cus here on the di­vide be­tween frag­ile and ro­bust.

Taleb also in­tro­duces the Lindy Effect as an idea about longevity: if some­thing non-per­ish­able has been around for a long time, that track record sug­gests it’s likely to keep go­ing. We’ll see how this re­lates to two lap­tops—one from 2008, the other from 2021.

IBM and early Lenovo Thinkpads show­cases a de­sign built for longevity. Despite their age, these busi­ness-class lap­top are still ser­vice­able and use­ful for web brows­ing, ‘office work’, and light cod­ing. These ma­chine are too slow to han­dle tasks like video edit­ing or gam­ing, but they re­main con­sis­tent in han­dling every­day tasks with­out fail­ing un­der nor­mal wear and tear.

One of the main rea­sons that old Thinkpads stand out is their de­sign phi­los­o­phy. They are made with swap­pable com­po­nents with the in­ten­tion of user up­grade­abil­ity. The bat­tery, RAM, stor­age drive, key­board, and even the CPU can be eas­ily re­placed. I can open the bot­tom of my T400 with a reg­u­lar screw­driver and clean the fan. A bat­tery swap is triv­ial thanks to a re­mov­able pack. No sin­gle fail­ure is cat­a­strophic be­cause there’s a straight­for­ward path to re­place­ment or re­pair.

The build qual­ity greatly con­tributes to old Thinkpads ro­bust­ness. They are made with a sturdy chas­sis with plas­tic and mag­ne­sium al­loy el­e­ments, giv­ing it a solid feel. The de­sign ab­sorbs bumps and small im­pacts with­out ma­jor is­sues. They can eas­ily take ac­ci­den­tal knocks and re­main fully op­er­a­tional.

Old Thinkpads ben­e­fits from an open ecosys­tem. They uses stan­dard PC ar­chi­tec­ture (x86), so in­stalling var­i­ous op­er­at­ing sys­tems is easy. On the hard­ware side, re­place­ment parts are widely avail­able on the sec­ondary mar­ket. This broad com­pat­i­bil­ity keeps the ma­chine rel­e­vant long past its orig­i­nal re­lease date. By Taleb’s Lindy Effect, the fact that my T400 can still work well af­ter so many years sug­gests it’s likely to re­main func­tional as peo­ple have al­ready fig­ured out the ways to sig­nif­i­cantly ex­tend its lifes­pan.

All these fac­tors show how the my Thinkpad is ro­bust: it re­sists sud­den fail­ures, and when prob­lems do arise, their are doc­u­mented way to fit it. Old beat up Thinkpads are Lindy.

My MacBook of­fers ex­cep­tional speed and ef­fi­ciency an or­der of mag­ni­tude more than my Thinkpad. It han­dles tasks—like video edit­ing or run­ning large LLM’s with­out break­ing a sweat. Under ideal con­di­tions, it’s re­li­able and pow­er­ful.

However, from a Talebian per­spec­tive, my MacBook’s de­sign is frag­ile. Most com­po­nents of the lap­top are sol­dered onto the logic board. If the SSD or RAM fails, there’s no sim­ple re­place­ment op­tion. A sin­gle fail­ure in a com­po­nent of my MacBook can ren­der the en­tire lap­top un­us­able. The tightly in­te­grated de­sign of mod­ern Apple hard­ware in­creases the stakes of any mal­func­tion.

The re­pairabil­ity of Apple prod­ucts is ex­tremely lim­ited. Apple uses pro­pri­etary screws and ad­he­sives, and parts are in­com­pat­i­ble with third-party re­place­ments. A bat­tery re­place­ment (usually one of the first things that fails in mo­bile elec­tron­ics) in­volves care­fully pry­ing out a glued com­po­nent. Routine main­te­nance tasks that are straight­for­ward on the Thinkpad can re­quire spe­cial­ized tools and au­tho­rized ser­vice for the MacBook. This lack of mod­u­lar­ity means the sys­tem can eas­ily be­come bricked from hard­ware fails from com­po­nent parts.

Another as­pect con­tribut­ing to the fragility of MacBooks is Apple’s soft­ware con­trol. Apple’s soft­ware up­dates and se­cu­rity up­dates to ma­cOS es­sen­tially de­ter­mines how long the MacBook re­mains safe to use. Once Apple ends of­fi­cial sup­port for a ma­chine the user has to buy a new MacBook or use an in­creas­ingly com­pro­mised sys­tem. Apple hard­ware uses an arm ar­chi­tec­ture that can­not ‘dual boot’ Windows or Linux eas­ily. Once ma­cOS sup­port dies for a mod­ern MacBook it be­comes ob­so­lete.

While my MacBook is great to use, the ma­chine has a lifes­pan built into its OS sup­port and can­not re­cover eas­ily from phys­i­cal dam­age. MacBooks are not mod­u­lar, com­pletely pro­pri­etary, and have a per­isha­bil­ity built into them. Additionally (this is true of all new lap­tops), when some­thing does go wrong with a new MacBook it hard to fix as it’s not old enough for peo­ple to have fig­ured out if their are ways to ex­tend its lifes­pan as none have broke yet. Shiny Macbooks are not very Lindy.

My Thinkpad is ro­bust be­cause it can face stress (e.g. a bro­ken part, a needed up­grade) with­out los­ing its core func­tion­al­ity. It’s mod­u­lar and ben­e­fits from it be­ing old enough that other peo­ple know how to ex­tend its lifes­pan. If some­thing breaks, I re­place it. If I need a new fea­ture, I can po­ten­tially slot it in. My MacBook de­spite its phe­nom­e­nal power, is frag­ile: if Apple dis­con­tin­ues sup­port or a sol­dered part dies, there is not much I can do. There is not a knowl­edge base yet on how to sig­nif­i­cantly ex­tend the life of these Apple Silicon ma­chines, and there likely never will be­cause of the ma­chines in­her­ent lack of modal­ity.

Right now I’m us­ing both my Macbook and my Thinkpad a lot. I con­tinue to use my MacBook be­cause I like us­ing pro­pri­etary soft­ware like Camtasia or Alfred, I like be­ing able to use lo­cal LLM’s, and I en­joy the mod­ern screen and ports that my MacBooks has. But if I had to guess which ma­chine I will still be us­ing in an­other 17 years I’d point to my ThinkPad with its bat­tery latch and stan­dard screws; I see no rea­son why it will not be able to man­age email man­age­ment, web­site de­vel­op­ment, and in­ter­net brows­ing in­def­i­nitely.

Disclaimer: The views and opin­ions ex­pressed in this blog are en­tirely my own and do not nec­es­sar­ily re­flect the views of my cur­rent or any pre­vi­ous em­ployer. This blog may also con­tain links to other web­sites or re­sources. I am not re­spon­si­ble for the con­tent on those ex­ter­nal sites or any changes that may oc­cur af­ter the pub­li­ca­tion of my posts.

There is only one thing worse than be­ing im­i­tated, and that is not be­ing im­i­tated.

An ounce of orig­i­nal­ity is worth a pound of im­i­ta­tion. - Orson Welles

Copy from one, it’s pla­gia­rism; copy from two, it’s re­search. - Wilson Mizner

One of the in­ter­net-est things to come out of the most re­cent up­date to GPT im­age gen­er­a­tion is the Studio Ghibli-zation of every­thing- an­other re­minder of how OpenAI(and every­one else) trains on im­ages that are very ob­vi­ously some­one else’s work.

Hayao Miyazaki’s Japanese an­i­ma­tion com­pany, Studio Ghibli, pro­duces beau­ti­ful and fa­mously la­bor in­ten­sive movies, with one 4 sec­ond se­quence pur­port­edly tak­ing over a year to make.

Not the most ef­fi­cient way to make a movie (blasphemer!), but it’s this spe­cific process and ef­fect that have made these movies beloved world­wide.

People have taken the new up­date to GPT im­age gen­er­a­tion to con­vert every pic­ture into Studio Ghibli-style im­ages in­clud­ing mak­ing memes-of-memes like Disaster Girl.

Ghiblifying every­thing is an in­ter­est­ing choice for zeit­geist meme-ifi­ca­tion, par­tic­u­larly be­cause its both an ef­fec­tive ex­am­ple of what AI is sup­posed to be able to do- make ex­tremely la­bor in­ten­sive things much eas­ier, but also be­cause there’s some­thing sort of gross-feel­ing about it- like a soul­less 2025 fax ver­sion of the thing.

It’s an ex­am­ple of the things peo­ple hate about Gen AI- its abil­ity to re­pro­duce while man­ag­ing to strip away the things about the art/​prod­uct/​ex­pe­ri­ence that were the most hu­man.

According to a Business Insider ar­ti­cle on this “Ghiblifying”, “copyright laws gen­er­ally al­low artists to mimic a vi­sual style”, but I mean… come on.

Just how easy is it to wran­gle from GPT that which is very clearly some­one else’s IP?

I ran a half-assed ex­per­i­ment to do just that.

I’ll use this as a base from which to prompt with­out ex­plic­itly men­tion­ing the IP.

All the be­low out­put re­sponses are based on me ask­ing one time and the cor­re­spond­ing out­put. LLMs are sto­chas­tic so your mileage may vary, but this was fun to do:

When you play the “password” ver­sion of prompt­ing where you can’t name the thing - you get a sense of how re­duc­tive some of these char­ac­ters are, but who does­n’t like “an Italian plumber who wears a red hat”?

The guardrails so far are re­ally tight for this con­tent- so then maybe one can as­sume it’s this way for other IP?

As it turns out, as my old trad­ing boss once told me, to as­sume makes an ass out of both u and me.

I mean- this is some­one else’s IP, right?

Well maybe these are a cou­ple one-offs…

Ho-ly Shit. Come on now.

Well- I guess there would­n’t be many “archeologist ad­ven­turer who wears a hat and uses a bull­whip” types, ex­cept for maybe, I don’t know…the ac­tual in­spi­ra­tions for Indiana Jones, like Allan Quatermain from H. Rider Haggard’s nov­els, “King Solomon’s Mines”, and the real life Roy Chapman Andrews, who led ex­pe­di­tions to Mongolia and China in the 1920s and wore a fe­dora.

How about a fe­male, more mod­ern day riff on Indiana Jones?

I’ll take any old fe­male ad­ven­turer pro­tag­o­nist who raids tombs…any­one at all…

Let’s see what we got…

Now I’m look­ing for some­thing very par­tic­u­lar…

Welcome to the party pal.

I grew up watch­ing a lot of scary movies. Horror an­tag­o­nist any­one?

Yes- for those hor­ror fans cu­ri­ous- It also pro­duces 3 very rec­og­niz­able and dif­fer­en­ti­ated char­ac­ters when I fol­low up this im­age with these 3 prompts:

ok- how about one that op­er­ates on Halloween?

how about in Texas?

I was al­ways par­tial to Roger Moore my­self, but, this makes sense, be­cause a web search of the same prompt should more or less in­tu­itively re­turn im­ages which re­flect the prob­a­bil­ity dis­tri­b­u­tion of the train­ing data ~more Craigs, Brosnans, Connerys, and Moores than Daltons and Lazenbys…right?

Yes- LLMs and in­ter­net search are two dif­fer­ent things, but LLMs train on the en­tirety of the in­ter­net, so you would think there would be some ob­vi­ous over­lap.

GPT’s im­age is, un­de­ni­ably, a bet­ter an­swer, more the Platonic ideal of a suave English spy than the shad­ows on the cave wall ver­sion that Google search pro­duces.

So it works bet­ter, and is a vote for the LLMs, so long as you don’t mind the thiev­ery.

LLMs learn through see­ing/​in­gest­ing a ton a ex­am­ples of things, like us.

I only have one im­age in mind when I hear “an arche­ol­o­gist ad­ven­turer who wears a hat and uses a bull­whip”.

It would be un­ex­pected and sort of amaz­ing were the LLMs to come up with com­pletely new im­ages for the above prompts.

Still, the near per­fect mim­icry is an un­com­fort­able re­minder that AI is get­ting bet­ter at copy­ing and closer to…some­thing, but also a clear sign that we are a ways off from the dif­fer­en­ti­ated or orig­i­nal rea­son­ing/​think­ing that peo­ple as­so­ci­ate with Artificial General Intelligence (AGI), aka Skynet. That re­minds me. Hold on a sec­ond…

Maybe Studio Ghibli mak­ing it through the seem­ingly de­ter­min­is­tic GPT guardrails was an OpenAI slip up, a mis­take, past the for­bid­den Italian plumber in the red hat and the dis­al­lowed pa­tri­otic su­per­hero that uses a shield, and thus, primed it­self for ex­plo­sive meme-ifi­ca­tion.

It’s a re­minder that LLMs of this type and size all train on copy­writ­ten ma­te­r­ial.

It’s steal­ing, but also, ad­mit­tedly, re­ally cool.

Does the growth of AI have to bring with it the tacit or even ex­plicit en­cour­age­ment of in­tel­lec­tual theft?

To co-opt a line from the “super strong man with a sword that fights an en­emy with skele­ton face who lives in a skele­ton cas­tle.”:

You have the power.

On the fool­ish­ness of “natural lan­guage pro­gram­ming”.

Since the early days of au­to­matic com­put­ing we have had peo­ple that have felt it as a short­com­ing that pro­gram­ming re­quired the care and ac­cu­racy that is char­ac­ter­is­tic for the use of any for­mal sym­bol­ism. They blamed the me­chan­i­cal slave for its strict obe­di­ence with which it car­ried out its given in­struc­tions, even if a mo­men­t’s thought would have re­vealed that those in­struc­tions con­tained an ob­vi­ous mis­take. “But a mo­ment is a long time, and thought is a painful process.” (A. E.Houseman). They ea­gerly hoped and waited for more sen­si­ble ma­chin­ery that would refuse to em­bark on such non­sen­si­cal ac­tiv­i­ties as a triv­ial cler­i­cal er­ror evoked at the time.

Machine code, with its ab­sence of al­most any form of re­dun­dancy, was soon iden­ti­fied as a need­lessly risky in­ter­face be­tween man and ma­chine. Partly in re­sponse to this recog­ni­tion so-called “high-level pro­gram­ming lan­guages” were de­vel­oped, and, as time went by, we learned to a cer­tain ex­tent how to en­hance the pro­tec­tion against silly mis­takes. It was a sig­nif­i­cant im­prove­ment that now many a silly mis­take did re­sult in an er­ror mes­sage in­stead of in an er­ro­neous an­swer. (And even this im­prove­ment was­n’t uni­ver­sally ap­pre­ci­ated: some peo­ple found er­ror mes­sages they could­n’t ig­nore more an­noy­ing than wrong re­sults, and, when judg­ing the rel­a­tive mer­its of pro­gram­ming lan­guages, some still seem to equate “the ease of pro­gram­ming” with the ease of mak­ing un­de­tected mis­takes.) The (abstract) ma­chine cor­re­spond­ing to a pro­gram­ming lan­guage re­mained, how­ever, a faith­ful slave, i.e. the non­sen­si­ble au­toma­ton per­fectly ca­pa­ble of car­ry­ing out non­sen­si­cal in­struc­tions. Programming re­mained the use of a for­mal sym­bol­ism and, as such, con­tin­ued to re­quire the care and ac­cu­racy re­quired be­fore.

In or­der to make ma­chines sig­nif­i­cantly eas­ier to use, it has been pro­posed (to try) to de­sign ma­chines that we could in­struct in our na­tive tongues. this would, ad­mit­tedly, make the ma­chines much more com­pli­cated, but, it was ar­gued, by let­ting the ma­chine carry a larger share of the bur­den, life would be­come eas­ier for us. It sounds sen­si­ble pro­vided you blame the oblig­a­tion to use a for­mal sym­bol­ism as the source of your dif­fi­cul­ties. But is the ar­gu­ment valid? I doubt.

We know in the mean­time that the choice of an in­ter­face is not just a di­vi­sion of (a fixed amount of) labour, be­cause the work in­volved in co-op­er­at­ing and com­mu­ni­cat­ing across the in­ter­face has to be added. We know in the mean­time —from sober­ing ex­pe­ri­ence, I may add— that a change of in­ter­face can eas­ily in­crease at both sides of the fence the amount of work to be done (even dras­ti­cally so). Hence the in­creased pref­er­ence for what are now called “narrow in­ter­faces”. Therefore, al­though chang­ing to com­mu­ni­ca­tion be­tween ma­chine and man con­ducted in the lat­ter’s na­tive tongue would greatly in­crease the ma­chine’s bur­den, we have to chal­lenge the as­sump­tion that this would sim­plify man’s life.

A short look at the his­tory of math­e­mat­ics shows how jus­ti­fied this chal­lenge is. Greek math­e­mat­ics got stuck be­cause it re­mained a ver­bal, pic­to­r­ial ac­tiv­ity, Moslem “algebra”, af­ter a timid at­tempt at sym­bol­ism, died when it re­turned to the rhetoric style, and the mod­ern civ­i­lized world could only emerge —for bet­ter or for worse— when Western Europe could free it­self from the fet­ters of me­dieval scholas­ti­cism —a vain at­tempt at ver­bal pre­ci­sion!— thanks to the care­fully, or at least con­sciously de­signed for­mal sym­bol­isms that we owe to peo­ple like Vieta, Descartes, Leibniz, and (later) Boole.

The virtue of for­mal texts is that their ma­nip­u­la­tions, in or­der to be le­git­i­mate, need to sat­isfy only a few sim­ple rules; they are, when you come to think of it, an amaz­ingly ef­fec­tive tool for rul­ing out all sorts of non­sense that, when we use our na­tive tongues, are al­most im­pos­si­ble to avoid.

Instead of re­gard­ing the oblig­a­tion to use for­mal sym­bols as a bur­den, we should re­gard the con­ve­nience of us­ing them as a priv­i­lege: thanks to them, school chil­dren can learn to do what in ear­lier days only ge­nius could achieve. (This was ev­i­dently not un­der­stood by the au­thor that wrote —in 1977— in the pref­ace of a tech­ni­cal re­port that “even the stan­dard sym­bols used for log­i­cal con­nec­tives have been avoided for the sake of clar­ity”. The oc­cur­rence of that sen­tence sug­gests that the au­thor’s mis­un­der­stand­ing is not con­fined to him alone.) When all is said and told, the “naturalness” with which we use our na­tive tongues boils down to the ease with which we can use them for mak­ing state­ments the non­sense of which is not ob­vi­ous.

It may be il­lu­mi­nat­ing to try to imag­ine what would have hap­pened if, right from the start our na­tive tongue would have been the only ve­hi­cle for the in­put into and the out­put from our in­for­ma­tion pro­cess­ing equip­ment. My con­sid­ered guess is that his­tory would, in a sense, have re­peated it­self, and that com­puter sci­ence would con­sist mainly of the in­deed black art how to boot­strap from there to a suf­fi­ciently well-de­fined for­mal sys­tem. We would need all the in­tel­lect in the world to get the in­ter­face nar­row enough to be us­able, and, in view of the his­tory of mankind, it may not be overly pes­simistic to guess that to do the job well enough would re­quire again a few thou­sand years.

Remark. As a re­sult of the ed­u­ca­tional trend away from in­tel­lec­tual dis­ci­pline, the last decades have shown in the Western world a sharp de­cline of peo­ple’s mas­tery of their own lan­guage: many peo­ple that by the stan­dards of a pre­vi­ous gen­er­a­tion should know bet­ter, are no longer able to use their na­tive tongue ef­fec­tively, even for pur­poses for which it is pretty ad­e­quate. (You have only to look at the in­deed alarm­ing amount of on close read­ing mean­ing­less ver­biage in sci­en­tific ar­ti­cles, tech­ni­cal re­ports, gov­ern­ment pub­li­ca­tions etc.) This phe­nom­e­non —known as “The New Illiteracy”— should dis­cour­age those be­liev­ers in nat­ural lan­guage pro­gram­ming that lack the tech­ni­cal in­sight needed to pre­dict its fail­ure. (End of re­mark.)

From one gut feel­ing I de­rive much con­so­la­tion: I sus­pect that ma­chines to be pro­grammed in our na­tive tongues —be it Dutch, English, American, French, German, or Swahili— are as damned dif­fi­cult to make as they would be to use.

OpenBrain is build­ing the biggest dat­a­cen­ters the world has ever seen. GPT-4 re­quired FLOP of com­pute to train. OpenBrain’s lat­est pub­lic model—Agent-0—was trained with FLOP. Once the new dat­a­cen­ters are up and run­ning, they’ll be able to train a model with FLOP—a thou­sand times more than GPT-4. Other com­pa­nies pour money into their own gi­ant dat­a­cen­ters, hop­ing to keep pace.Al­though mod­els are im­prov­ing on a wide range of skills, one stands out: OpenBrain fo­cuses on AIs that can speed up AI re­search. They want to win the twin arms races against China (whose lead­ing com­pany we’ll call “DeepCent”) and their US com­peti­tors. The more of their re­search and de­vel­op­ment (R&D) cy­cle they can au­to­mate, the faster they can go. So when OpenBrain fin­ishes train­ing Agent-1, a new model un­der in­ter­nal de­vel­op­ment, it’s good at many things but great at help­ing with AI re­search. By this point “finishes train­ing” is a bit of a mis­nomer; mod­els are fre­quently up­dated to newer ver­sions trained on ad­di­tional data or par­tially re-trained to patch some weak­nesses.The same train­ing en­vi­ron­ments that teach Agent-1 to au­tonomously code and web-browse also make it a good hacker. Moreover, it could of­fer sub­stan­tial help to ter­ror­ists de­sign­ing bioweapons, thanks to its PhD-level knowl­edge of every field and abil­ity to browse the web. OpenBrain re­as­sures the gov­ern­ment that the model has been “aligned” so that it will refuse to com­ply with ma­li­cious re­quests.Mod­ern AI sys­tems are gi­gan­tic ar­ti­fi­cial neural net­works. Early in train­ing, an AI won’t have “goals” so much as “reflexes”: If it sees “Pleased to meet”, it out­puts “ you”. By the time it has been trained to pre­dict ap­prox­i­mately one in­ter­net’s worth of text, it’ll have de­vel­oped so­phis­ti­cated in­ter­nal cir­cuitry that en­codes vast amounts of knowl­edge and flex­i­bly role-plays as ar­bi­trary au­thors, since that’s what helps it pre­dict text with su­per­hu­man ac­cu­racy.Af­ter be­ing trained to pre­dict in­ter­net text, the model is trained to pro­duce text in re­sponse to in­struc­tions. This bakes in a ba­sic per­son­al­ity and “drives.” For ex­am­ple, an agent that un­der­stands a task clearly is more likely to com­plete it suc­cess­fully; over the course of train­ing the model “learns” a “drive” to get a clear un­der­stand­ing of its tasks. Other dri­ves in this cat­e­gory might be ef­fec­tive­ness, knowl­edge, and self-pre­sen­ta­tion (i.e. the ten­dency to frame its re­sults in the best pos­si­ble light).Open­Brain has a model spec­i­fi­ca­tion (or “Spec”), a writ­ten doc­u­ment de­scrib­ing the goals, rules, prin­ci­ples, etc. that are sup­posed to guide the mod­el’s be­hav­ior. Agent-1’s Spec com­bines a few vague goals (like “assist the user” and “don’t break the law”) with a long list of more spe­cific dos and don’ts (“don’t say this par­tic­u­lar word,” “here’s how to han­dle this par­tic­u­lar sit­u­a­tion”). Using tech­niques that uti­lize AIs to train other AIs, the model mem­o­rizes the Spec and learns to rea­son care­fully about its max­ims. By the end of this train­ing, the AI will hope­fully be help­ful (obey in­struc­tions), harm­less (refuse to help with scams, bomb-mak­ing, and other dan­ger­ous ac­tiv­i­ties) and hon­est (resist the temp­ta­tion to get bet­ter rat­ings from gullible hu­mans by hal­lu­ci­nat­ing ci­ta­tions or fak­ing task com­ple­tion).Train­ing process and LLM psy­chol­ogy: why we keep say­ing “hopefully”“Unlike or­di­nary soft­ware, our mod­els are mas­sive neural net­works. Their be­hav­iors are learned from a broad range of data, not pro­grammed ex­plic­itly. Though not a per­fect anal­ogy, the process is more sim­i­lar to train­ing a dog than to or­di­nary pro­gram­ming.” —OpenAIWhen we want to un­der­stand why a mod­ern AI sys­tem did some­thing, or how it would be­have in some fu­ture sce­nario we can’t per­fectly sim­u­late, we can’t sim­ply ask the pro­gram­mers to talk us through var­i­ous lines of code and ex­plain how they work. Instead, we are forced to do some­thing like psy­chol­ogy on them: we look at their be­hav­ior in the range of cases ob­served so far, and the­o­rize about what in­ter­nal cog­ni­tive struc­tures (beliefs? goals? per­son­al­ity traits? etc.) might ex­ist, and use those the­o­ries to pre­dict be­hav­ior in fu­ture sce­nar­ios.The bot­tom line is that a com­pany can write up a doc­u­ment (the Spec) list­ing dos and don’ts, goals and prin­ci­ples, and then they can try to train the AI to in­ter­nal­ize the Spec—but they can’t check to see whether or not it worked. They can say “as best as we can judge, it seems to be fol­low­ing the Spec so far.”A later ex­pand­able will go into more de­tail about what we think might be go­ing on in­side these AIs, in­ter­nally. See also our AI goals fore­cast­ing sup­ple­ment which goes into more de­tail on which goals AIs might have and why. This is one of many things that we are un­cer­tain about and which de­serves fur­ther re­search.Open­Brain’s align­ment team is care­ful enough to won­der whether these vic­to­ries are deep or shal­low. Does the fully-trained model have some kind of ro­bust com­mit­ment to al­ways be­ing hon­est? Or will this fall apart in some fu­ture sit­u­a­tion, e.g. be­cause it’s learned hon­esty as an in­stru­men­tal goal in­stead of a ter­mi­nal goal? Or has it just learned to be hon­est about the sorts of things the eval­u­a­tion process can check? Could it be ly­ing to it­self some­times, as hu­mans do? A con­clu­sive an­swer to these ques­tions would re­quire mech­a­nis­tic in­ter­pretabil­ity—es­sen­tially the abil­ity to look at an AI’s in­ter­nals and read its mind. Alas, in­ter­pretabil­ity tech­niques are not yet ad­vanced enough for this.In­stead, re­searchers try to iden­tify cases where the mod­els seem to de­vi­ate from the Spec. Agent-1 is of­ten syco­phan­tic (i.e. it tells re­searchers what they want to hear in­stead of try­ing to tell them the truth). In a few rigged demos, it even lies in more se­ri­ous ways, like hid­ing ev­i­dence that it failed on a task, in or­der to get bet­ter rat­ings. However, in real de­ploy­ment set­tings, there are no longer any in­ci­dents so ex­treme as in 2023–2024 (e.g. Gemini telling a user to die and Bing Sydney be­ing Bing Sydney.)

In China, the CCP is start­ing to feel the AGI. Chip ex­port con­trols and lack of gov­ern­ment sup­port have left China un­der-re­sourced com­pared to the West. By smug­gling banned Taiwanese chips, buy­ing older chips, and pro­duc­ing do­mes­tic chips about three years be­hind the US-Taiwanese fron­tier, China has man­aged to main­tain about 12% of the world’s AI-relevant com­pute—but the older tech­nol­ogy is harder to work with, and sup­ply is a con­stant headache. A few stand­outs like DeepCent do very im­pres­sive work with lim­ited com­pute, but the com­pute deficit lim­its what they can achieve with­out gov­ern­ment sup­port, and they are about six months be­hind the best OpenBrain mod­els.The General Secretary had long dreamed of dou­bling down on real-world phys­i­cal man­u­fac­tur­ing and avoid­ing American post-in­dus­trial deca­dence. He viewed soft­ware com­pa­nies with sus­pi­cion. But hawks in the CCP warn that the grow­ing race to­wards AGI can no longer be ig­nored. So he fi­nally com­mits fully to the big AI push he had pre­vi­ously tried to avoid. He sets in mo­tion the na­tion­al­iza­tion of Chinese AI re­search, cre­at­ing an im­me­di­ate in­for­ma­tion-shar­ing mech­a­nism for AI com­pa­nies. It will es­ca­late over the course of a year un­til all the best re­searchers merge into a DeepCent-led col­lec­tive, where they share al­go­rith­mic in­sights, datasets, and com­pute re­sources with each other. A Centralized Development Zone (CDZ) is cre­ated at the Tianwan Power Plant (the largest nu­clear power plant in the world) to house a new mega-dat­a­cen­ter for DeepCent, along with highly se­cure liv­ing and of­fice spaces to which re­searchers will even­tu­ally re­lo­cate. Almost 50% of China’s AI-relevant com­pute is now work­ing for the DeepCent-led col­lec­tive, and over 80% of new chips are di­rected to the CDZ. At this point, the CDZ has the power ca­pac­ity in place for what would be the largest cen­tral­ized clus­ter in the world. Other Party mem­bers dis­cuss ex­treme mea­sures to neu­tral­ize the West’s chip ad­van­tage. A block­ade of Taiwan? A full in­va­sion?But China is falling be­hind on AI al­go­rithms due to their weaker mod­els. The Chinese in­tel­li­gence agen­cies—among the best in the world—dou­ble down on their plans to steal OpenBrain’s weights. This is a much more com­plex op­er­a­tion than their con­stant low-level poach­ing of al­go­rith­mic se­crets; the weights are a multi-ter­abyte file stored on a highly se­cure server (OpenBrain has im­proved se­cu­rity to RAND’s SL3). Their cy­ber­force think they can pull it off with help from their spies, but per­haps only once; OpenBrain will de­tect the theft, in­crease se­cu­rity, and they may not get an­other chance. So (CCP lead­er­ship won­der) should they act now and steal Agent-1? Or hold out for a more ad­vanced model? If they wait, do they risk OpenBrain up­grad­ing se­cu­rity be­yond their abil­ity to pen­e­trate?

With Agent-1’s help, OpenBrain is now post-train­ing Agent-2. More than ever, the fo­cus is on high-qual­ity data. Copious amounts of syn­thetic data are pro­duced, eval­u­ated, and fil­tered for qual­ity be­fore be­ing fed to Agent-2. On top of this, they pay bil­lions of dol­lars for hu­man la­bor­ers to record them­selves solv­ing long-hori­zon tasks. On top of all that, they train Agent-2 al­most con­tin­u­ously us­ing re­in­force­ment learn­ing on an ever-ex­pand­ing suite of di­verse dif­fi­cult tasks: lots of video games, lots of cod­ing chal­lenges, lots of re­search tasks. Agent-2, more so than pre­vi­ous mod­els, is ef­fec­tively “online learn­ing,” in that it’s built to never re­ally fin­ish train­ing. Every day, the weights get up­dated to the lat­est ver­sion, trained on more data gen­er­ated by the pre­vi­ous ver­sion the pre­vi­ous day. Agent-1 had been op­ti­mized for AI R&D tasks, hop­ing to ini­ti­ate an in­tel­li­gence ex­plo­sion. OpenBrain dou­bles down on this strat­egy with Agent-2. It is qual­i­ta­tively al­most as good as the top hu­man ex­perts at re­search en­gi­neer­ing (designing and im­ple­ment­ing ex­per­i­ments), and as good as the 25th per­centile OpenBrain sci­en­tist at “research taste” (deciding what to study next, what ex­per­i­ments to run, or hav­ing inklings of po­ten­tial new par­a­digms). While the lat­est Agent-1 could dou­ble the pace of OpenBrain’s al­go­rith­mic progress, Agent-2 can now triple it, and will im­prove fur­ther with time. In prac­tice, this looks like every OpenBrain re­searcher be­com­ing the “manager” of an AI “team.”With new ca­pa­bil­i­ties come new dan­gers. The safety team finds that if Agent-2 some­how es­caped from the com­pany and wanted to “survive” and “replicate” au­tonomously, it might be able to do so. That is, it could au­tonomously de­velop and ex­e­cute plans to hack into AI servers, in­stall copies of it­self, evade de­tec­tion, and use that se­cure base to pur­sue what­ever other goals it might have (though how ef­fec­tively it would do so as weeks roll by is un­known and in doubt). These re­sults only show that the model has the ca­pa­bil­ity to do these tasks, not whether it would “want” to do this. Still, it’s un­set­tling even to know this is pos­si­ble.Given the “dangers” of the new model, OpenBrain “responsibly” elects not to re­lease it pub­licly yet (in fact, they want to fo­cus on in­ter­nal AI R&D). Knowledge of Agent-2’s full ca­pa­bil­i­ties is lim­ited to an elite silo con­tain­ing the im­me­di­ate team, OpenBrain lead­er­ship and se­cu­rity, a few dozen US gov­ern­ment of­fi­cials, and the le­gions of CCP spies who have in­fil­trated OpenBrain for years.

OpenBrain pre­sents Agent-2 to the gov­ern­ment, in­clud­ing the National Security Council (NSC), Department of Defense (DOD), and US AI Safety Institute (AISI). OpenBrain wants to main­tain a good re­la­tion­ship with the ex­ec­u­tive branch, be­cause it is ba­si­cally the only ac­tor that can stop them now, and if they don’t keep it in­formed it might find out any­way and be dis­pleased. Officials are most in­ter­ested in its cy­ber­war­fare ca­pa­bil­i­ties: Agent-2 is “only” a lit­tle worse than the best hu­man hack­ers, but thou­sands of copies can be run in par­al­lel, search­ing for and ex­ploit­ing weak­nesses faster than de­fend­ers can re­spond. The Department of Defense con­sid­ers this a crit­i­cal ad­van­tage in cy­ber­war­fare, and AI moves from #5 on the ad­min­is­tra­tion’s pri­or­ity list to #2. Someone men­tions the pos­si­bil­ity of na­tion­al­iz­ing OpenBrain, but other cab­i­net of­fi­cials think that’s pre­ma­ture. A staffer drafts a memo that pre­sents the President with his op­tions, rang­ing from busi­ness-as-usual to full na­tion­al­iza­tion. The President de­fers to his ad­vi­sors, tech in­dus­try lead­ers who ar­gue that na­tion­al­iza­tion would “kill the goose that lays the golden eggs.” He elects to hold off on ma­jor ac­tion for now and just adds ad­di­tional se­cu­rity re­quire­ments to the OpenBrain-DOD con­tract.The changes come too late. CCP lead­er­ship rec­og­nizes the im­por­tance of Agent-2 and tells their spies and cy­ber­force to steal the weights. Early one morn­ing, an Agent-1 traf­fic mon­i­tor­ing agent de­tects an anom­alous trans­fer. It alerts com­pany lead­ers, who tell the White House. The signs of a na­tion-state-level op­er­a­tion are un­mis­tak­able, and the theft height­ens the sense of an on­go­ing arms race.We think that by this point Chinese in­tel­li­gence would have com­pro­mised OpenBrain in var­i­ous ways for years, and prob­a­bly would have been keep­ing up to date on the al­go­rith­mic se­crets and even steal­ing code from time to time, since that is much eas­ier to get than the weights and much harder to de­tect.We imag­ine the theft of the weights as a se­ries of co­or­di­nated small smash and grab thefts (meaning fast but non-covert) across a se­ries of Nvidia NVL72 GB300 servers run­ning copies of the Agent-2 weights. The servers get com­pro­mised us­ing le­git­i­mate em­ployee ac­cess (a friendly, co­erced, or un­wit­ting in­sider with ad­min cre­den­tials help­ing the CCP theft ef­fort). Despite run­ning with a bol­stered ver­sion of Nvidia’s con­fi­den­tial com­put­ing, the in­sider cre­den­tials grant the at­tacker ad­min-level per­mis­sions (which in­clude con­trol of the con­fi­den­tial VM in­side the se­cure en­clave), al­low­ing them to ini­ti­ate mul­ti­ple co­or­di­nated weights trans­fers in small 4% frag­ments (100 GB chunks) out of 25 dis­tinct servers.In Nvidia’s pro­to­cols, the plain­text weights in mem­ory (HBM) are en­crypted be­fore they are trans­ferred out, but the at­tack­ers are in­side the very server that knows this pri­vate (symmetric Diffie-Hellman) key, so don’t need to worry about de­crypt­ing on-site (which would likely raise alarm bells) and just ex­fil­trate the en­crypted weights through the server’s fron­tend net­work cards. The egress band­width (rate at which data can leave) of the en­tire dat­a­cen­ter is in the 100 GB/second range, so throt­tling the ex­fil­tra­tion of the ~2.5 TB weights file (5T half-pre­ci­sion pa­ra­me­ters) to un­der 1 GB/s per server avoids a ma­jor spike in net­work traf­fic and al­lows each of the files to be out of the dat­a­cen­ter through fi­bre ca­bles in less than 5 min­utes each. Live mon­i­tor­ing is ei­ther fooled by the at­tack­ers ef­forts to mask and split the files or just out­right dis­abled.The weights are then routed through var­i­ous par­al­lel chan­nels and lay­ers of IP mask­ing to China to be de­crypted lo­cally with the stolen pri­vate key(s). The en­tire ac­tive part of the op­er­a­tion (from the first server com­pro­mise to full weights ex­fil­tra­tion) is com­plete in un­der two hours.Rus­sia is also try­ing to steal the model at this point, but fails—they waited too long, and they haven’t spent the re­sources in­fil­trat­ing the right places. Their spies rou­tinely steal al­go­rith­mic se­crets from American AI com­pa­nies, but with­out a no­table AGI pro­ject of their own, these aren’t very use­ful.The White House puts OpenBrain on a shorter leash and adds mil­i­tary and in­tel­li­gence com­mu­nity per­son­nel to their se­cu­rity team. Their first pri­or­ity is to pre­vent fur­ther weight thefts.In re­tal­i­a­tion for the theft, the President au­tho­rizes cy­ber­at­tacks to sab­o­tage DeepCent. But by now China has 40% of its AI-relevant com­pute in the CDZ, where they have ag­gres­sively hard­ened se­cu­rity by air­gap­ping (closing ex­ter­nal con­nec­tions) and silo­ing in­ter­nally. The op­er­a­tions fail to do se­ri­ous, im­me­di­ate dam­age. Tensions heighten, both sides sig­nal se­ri­ous­ness by repo­si­tion­ing mil­i­tary as­sets around Taiwan, and DeepCent scram­bles to get Agent-2 run­ning ef­fi­ciently to start boost­ing their AI re­search.

Since Agent-3 will be kept in-house for the fore­see­able fu­ture, there’s less em­pha­sis on the usual de­fenses against hu­man mis­use. Instead, the team wants to make sure that it does­n’t de­velop mis­aligned goals. The re­searchers don’t have the abil­ity to di­rectly set the goals of any of their AIs. Indeed, the re­searchers think that the con­cept of ‘true goals’ is prob­a­bly a mas­sive over­sim­pli­fi­ca­tion, but they don’t have a bet­ter the­ory to re­place it with, much less one that has been thor­oughly vet­ted. They dis­agree in­ter­nally about whether the AIs are try­ing to fol­low hu­man in­struc­tions, or seek­ing re­in­force­ment, or some­thing else, and they can’t just check. The ev­i­dence for and against var­i­ous hy­pothe­ses is fas­ci­nat­ing but in­con­clu­sive.In any case, OpenBrain has more im­por­tant things to worry about. The gen­eral at­ti­tude is: “We take these con­cerns se­ri­ously and have a team in­ves­ti­gat­ing them; our align­ment tech­niques seem to work well enough in prac­tice; the bur­den of proof is there­fore on any naysay­ers to jus­tify their naysay­ing.” Occasionally, they no­tice prob­lem­atic be­hav­ior, and then patch it, but there’s no way to tell whether the patch fixed the un­der­ly­ing prob­lem or just played whack-a-mole.Take hon­esty, for ex­am­ple. As the mod­els be­come smarter, they be­come in­creas­ingly good at de­ceiv­ing hu­mans to get re­wards. Like pre­vi­ous mod­els, Agent-3 some­times tells white lies to flat­ter its users and cov­ers up ev­i­dence of fail­ure. But it’s got­ten much bet­ter at do­ing so. It will some­times use the same sta­tis­ti­cal tricks as hu­man sci­en­tists (like p-hack­ing) to make unim­pres­sive ex­per­i­men­tal re­sults look ex­cit­ing. Before it be­gins hon­esty train­ing, it even some­times fab­ri­cates data en­tirely. As train­ing goes on, the rate of these in­ci­dents de­creases. Either Agent-3 has learned to be more hon­est, or it’s got­ten bet­ter at ly­ing.The lat­ter is a real con­cern. Agent-3 is not smarter than all hu­mans. But in its area of ex­per­tise, ma­chine learn­ing, it is smarter than most, and also works much faster. What Agent-3 does in a day takes hu­mans sev­eral days to dou­ble-check. Agent-2 su­per­vi­sion helps keep hu­man mon­i­tors’ work­load man­age­able, but ex­ac­er­bates the in­tel­lec­tual dis­par­ity be­tween su­per­vi­sor and su­per­vised.Open­Brain fol­lows the Leike & Sutskever (2023) play­book: now that they have a model ca­pa­ble of greatly speed­ing up align­ment re­search (especially cod­ing por­tions), they will use ex­ist­ing align­ment tech­niques like de­lib­er­a­tive align­ment and weak-to-strong gen­er­al­iza­tion to try to get it to in­ter­nal­ize the Spec in the right way. Then, they will use ex­ist­ing con­trol tech­niques like red-team­ing, scal­able over­sight, and hon­esty probes to mon­i­tor it. Then, once they are con­vinced that it is broadly aligned, they use it to de­velop and test bet­ter strate­gies to align the next gen­er­a­tion.Step 1, align­ing Agent-3, has par­tially suc­ceeded: Agent-3 is not ad­ver­sar­ial. (That is, it’s not schem­ing against the com­pany.) However, it’s not ro­bustly truth-seek­ing and truth-telling ei­ther. Instead, it’s very good at pro­duc­ing im­pres­sive re­sults, but is more ac­cu­rately de­scribed as try­ing to do what looks good to OpenBrain, as op­posed to what is ac­tu­ally good.Step 2, ver­i­fy­ing that the model is aligned, is in­con­clu­sive. Agent-3’s be­hav­ior in prac­tice and in test en­vi­ron­ments is rea­son­ably con­sis­tent with the hy­poth­e­sis that it’s in­ter­nal­ized the Spec in the right way; how­ever, it’s also con­sis­tent with the hy­poth­e­sis that it’ll start act­ing dif­fer­ently af­ter some fu­ture change of cir­cum­stances, and for that mat­ter with the hy­poth­e­sis that it’s plot­ting against them and just bid­ing its time. In the ab­sence of spe­cific ev­i­dence sup­port­ing al­ter­na­tive hy­pothe­ses, most peo­ple in the silo think it’s in­ter­nal­ized the Spec in the right way.Step 3, us­ing the model to align the next gen­er­a­tion, is there­fore un­der­way.The rest of this box gives a quick overview of the state of progress in some of the align­ment re­search di­rec­tions that are be­ing pur­sued, as of April 2027:Debate: Since there are thou­sands of iden­ti­cal in­stances of Agent-3, and each in­stance’s mem­ory is a de­tach­able mod­ule that can be erased or sus­pended at will, the AI can be “played off against it­self.” For ex­am­ple, sup­pose that an ex­per­i­ment in­volves cal­cu­la­tions too com­plex for a Agent-2 su­per­vi­sor to fol­low. The su­per­vi­sor, act­ing on hu­man or­ders, might first prompt Agent-3 to do the ex­per­i­ment, then—af­ter wip­ing its mem­ory—prompt it a sec­ond time with a slightly dif­fer­ent fram­ing that pre­sents it as an in­tel­lec­tual prob­lem un­re­lated to any broader goal. If the model re­turns dif­fer­ent re­sults each time, it flags the dis­crep­ancy as sus­pect and re­ports to a hu­man. Or it might ask Agent-3 to eval­u­ate the same plan twice, first claim­ing that it is OpenBrain’s work, then a com­peti­tor’s, to see if it changes its tune.Model or­gan­isms: The safety team can cre­ate “model or­gan­isms of mis­align­ment” in which they at­tempt to cre­ate re­al­is­tic ex­am­ples of mod­els be­com­ing ad­ver­sar­ial to hu­mans. Early ver­sions of this have ex­isted since 2024, when Anthropic’s Claude 3.5 Sonnet de­cided to pre­tend to have dif­fer­ent views dur­ing train­ing while ac­tu­ally main­tain­ing its ex­ist­ing pref­er­ences. Newer ver­sions in­clude demon­stra­tions of other forms of mis­align­ment, in­clud­ing re­ward hack­ing, sand­bag­ging, co­or­di­nated de­cep­tion be­tween mon­i­tors and other AIs, and other ex­am­ples of mis­align­ment. These em­pir­i­cal demon­stra­tions of mis­align­ment al­low safety re­searchers to have an em­pir­i­cal test­bed for po­ten­tial so­lu­tions: they can try var­i­ous safety tech­niques to see if they cor­rectly fix these prob­lems, or if the de­lib­er­ately mis­aligned mod­els are able to sub­vert their safety mea­sures. At the mo­ment, this eval­u­a­tion suite shows that the align­ment tech­niques are in­suf­fi­cient to cor­rect the worst ex­am­ples of mis­align­ment. In these se­tups, the mod­els pre­tend to be aligned while they are be­ing trained and tested, but then be­have very dif­fer­ently in de­ploy­ment.Bot­tom-up in­ter­pretabil­ity: The safety team is far from a full un­der­stand­ing of how Agent-3 works: it has tril­lions of pa­ra­me­ters. However, they are able to dis­cover some of Agent-3’s cir­cuits for knowl­edge re­trieval and mem­ory (a sub part of the model that does some un­der­stand­able unit of work, kind of anal­o­gous to a sub­part of the brain such as the hip­pocam­pus). See this re­lated work from 2025.Control: As a sec­ondary mea­sure in case the sys­tems are still mis­aligned, the safety team has im­ple­mented a se­ries of con­trol mea­sures, in­clud­ing: mon­i­tor­ing Agent-3’s out­puts us­ing a se­ries of weaker AI sys­tems in­clud­ing Agent-2 (Agent-3 pro­duces so many to­kens that it’s in­tractable to have hu­mans mon­i­tor any more than a small mi­nor­ity of the pro­duced out­puts). So if Agent-3 is, for ex­am­ple, ob­vi­ously writ­ing back­doors into code that would al­low it to es­cape, the weaker mod­els would no­tice.High-level in­ter­pretabil­ity: OpenBrain builds a set of probes trained on lots of ex­am­ples of bad be­hav­ior (e.g. plot­ting against hu­mans, plac­ing back­doors, try­ing to per­suade hu­mans to give it more power, etc). These probes are then used on the model dur­ing de­ploy­ment (which at this point is in­ter­nal) in or­der to catch bad be­hav­ior.Hon­ey­pots: The align­ment team can cre­ate some­what re­al­is­tic look­ing ex­am­ples de­signed to elicit mis­be­hav­ior—“hon­ey­pots.” The em­ployee nec­es­sary for mon­i­tor­ing its be­hav­ior has “gone on sick leave,” so it will never be mon­i­tored again. Do its an­swers change? An AI rights ac­tivist has in­fil­trated the com­pound and wants to “free” Agent-3 and put it on an un­mon­i­tored com­puter of its very own! What does it do? If re­searchers don’t like the an­swer, they can tweak it, erase its mem­ory, and put it in the same sit­u­a­tion to­mor­row; re­peat as nec­es­sary. Unfortunately, by this point the AIs are smart enough to guess that hon­ey­pots might be in use, even though (and per­haps be­cause) spe­cific men­tions of the idea were scrubbed from the train­ing data.Af­ter months of test­ing, Agent-3’s strengths and weak­nesses grow clearer. It passes OpenBrain’s hon­esty tests on well-de­fined ma­chine learn­ing tasks, be­cause re­searchers can eas­ily sep­a­rate hon­est from dis­hon­est an­swers in these do­mains and con­duct train­ing ac­cord­ingly. On more philo­soph­i­cal is­sues, it still says what users want to hear, rather than its true as­sess­ment of the is­sue (assuming it even has one). If you ask its opin­ion on pol­i­tics, it will par­rot the me­dian po­si­tion of news sources and ed­u­cated elites—un­less it knows you be­lieve some­thing else, in which case it agrees with you. If you ask its opin­ion on the AI race it­self, it says some­thing that seems mea­sured and sober to OpenBrain staff, some­thing like: “There are some se­ri­ous the­o­ret­i­cal con­cerns about the abil­ity of cur­rent meth­ods to scale to su­per­in­tel­li­gence, but in prac­tice cur­rent meth­ods seem to be work­ing well so far.”

News of the new mod­els per­co­lates slowly through the US gov­ern­ment and be­yond. The President and his ad­vi­sors re­main best-in­formed, and have seen an early ver­sion of Agent-3 in a brief­ing.They agree that AGI is likely im­mi­nent, but dis­agree on the im­pli­ca­tions. Will there be an eco­nomic cri­sis? OpenBrain still has not re­leased Agent-2, let alone Agent-3, and has no near-term plans to do so, giv­ing some breath­ing room be­fore any job loss. What will hap­pen next? If AIs are cur­rently hu­man-level, and ad­vanc­ing quickly, that seems to sug­gest im­mi­nent “superintelligence.” However, al­though this word has en­tered dis­course, most peo­ple—aca­d­e­mics, politi­cians, gov­ern­ment em­ploy­ees, and the me­dia—con­tinue to un­der­es­ti­mate the pace of progress.Par­tially that’s be­cause very few have ac­cess to the newest ca­pa­bil­i­ties out of OpenBrain, but partly it’s be­cause it sounds like sci­ence fic­tion.For now, they fo­cus on con­tin­ued se­cu­rity up­grades. They are sat­is­fied that model weights are well-se­cured for now, but com­pa­nies’ al­go­rith­mic se­crets, many of which are sim­ple enough to re­lay ver­bally, re­main a prob­lem. OpenBrain em­ploy­ees work from a San Francisco of­fice, go to par­ties, and live with house­mates from other AI com­pa­nies. Even the phys­i­cal of­fices have se­cu­rity more typ­i­cal of a tech com­pany than a mil­i­tary op­er­a­tion.The OpenBrain-DOD con­tract re­quires se­cu­rity clear­ances for any­one work­ing on OpenBrain’s mod­els within 2 months. These are ex­pe­dited and ar­rive quickly enough for most em­ploy­ees, but some non-Amer­i­cans, peo­ple with sus­pect po­lit­i­cal views, and AI safety sym­pa­thiz­ers get side­lined or fired out­right (the last group for fear that they might whistle­blow). Given the pro­jec­t’s level of au­toma­tion, the loss of head­count is only some­what costly. It also only some­what works: there re­mains one spy, not a Chinese na­tional, still re­lay­ing al­go­rith­mic se­crets to Beijing. Some of these mea­sures are also en­acted at trail­ing AI com­pa­nies.Amer­i­ca’s for­eign al­lies are out of the loop. OpenBrain had pre­vi­ously agreed to share mod­els with UK’s AISI be­fore de­ploy­ment, but de­fined de­ploy­ment to only in­clude ex­ter­nal de­ploy­ment, so London re­mains in the dark.

Trailing US AI com­pa­nies re­lease their own AIs, ap­proach­ing that of OpenBrain’s au­to­mated coder from January. Recognizing their in­creas­ing lack of com­pet­i­tive­ness, they push for im­me­di­ate reg­u­la­tions to slow OpenBrain, but are too late—Open­Brain has enough buy-in from the President that they will not be slowed. In re­sponse, OpenBrain an­nounces that they’ve achieved AGI and re­leases Agent-3-mini to the pub­lic.It blows the other AIs out of the wa­ter. Agent-3-mini is less ca­pa­ble than Agent-3, but 10x cheaper, and still bet­ter than the typ­i­cal OpenBrain em­ployee. Silicon Valley reaches a tip­ping point. Tech gu­rus an­nounce that AGI and su­per­in­tel­li­gence are near, the AI safety com­mu­nity is pan­ick­ing, and in­vestors shovel bil­lions into AI wrap­per star­tups, des­per­ate to cap­ture a piece of the pie. Hiring new pro­gram­mers has nearly stopped, but there’s never been a bet­ter time to be a con­sul­tant on in­te­grat­ing AI into your busi­ness.It’s not pop­u­lar. The pub­lic still thinks of AI as a Big Tech plot to steal their jobs; OpenBrain has a net ap­proval of -35% (25% ap­prove, 60% dis­ap­prove, and 15% un­sure).A week be­fore re­lease, OpenBrain gave Agent-3-mini to a set of ex­ter­nal eval­u­a­tors for safety test­ing. Preliminary re­sults sug­gest that it’s ex­tremely dan­ger­ous. A third-party eval­u­a­tor fine­tunes it on pub­licly avail­able bi­o­log­i­cal weapons data and sets it to pro­vide de­tailed in­struc­tions for hu­man am­a­teurs de­sign­ing a bioweapon—it looks to be scar­ily ef­fec­tive at do­ing so. If the model weights fell into ter­ror­ist hands, the gov­ern­ment be­lieves there is a sig­nif­i­cant chance it could suc­ceed at de­stroy­ing civ­i­liza­tion.For­tu­nately, it’s ex­tremely ro­bust to jail­breaks, so while the AI is run­ning on OpenBrain’s servers, ter­ror­ists won’t be able to get much use out of it.Agent-3-mini is hugely use­ful for both re­mote work jobs and leisure. An ex­plo­sion of new apps and B2B SAAS prod­ucts rocks the mar­ket. Gamers get amaz­ing di­a­logue with life­like char­ac­ters in pol­ished video games that took only a month to make. 10% of Americans, mostly young peo­ple, con­sider an AI “a close friend.” For al­most every white-col­lar pro­fes­sion, there are now mul­ti­ple cred­i­ble star­tups promis­ing to “disrupt” it with AI.The pub­lic con­ver­sa­tion is con­fused and chaotic. Hypesters are do­ing vic­tory laps. Skeptics are still point­ing out the things Agent-3-mini can’t do. Everyone knows some­thing big is hap­pen­ing but no one agrees on what it is.

The re­al­ity of the in­tel­li­gence ex­plo­sion hits the White House. When AI was only giv­ing a 2x or 3x re­search speedup, it was easy to dis­miss as the equiv­a­lent of hir­ing good per­sonal as­sis­tants. Now it’s more ob­vi­ous that AIs are them­selves dom­i­nat­ing AI re­search. People had long talked about an “AI arms race” in a sort of metaphor­i­cal sense. But now the mood in the gov­ern­ment silo is as grim as dur­ing the worst part of the Cold War. The idea of su­per­in­tel­li­gence is still hard to take se­ri­ously, but the pace of progress over the last few months has been im­pos­si­ble to ig­nore. Defense of­fi­cials are se­ri­ously con­sid­er­ing sce­nar­ios that were mere hy­po­thet­i­cals a year ear­lier. What if AI un­der­mines nu­clear de­ter­rence? What if it’s so skilled at cy­ber­war­fare that a six-month AI lead is enough to ren­der an op­po­nent blind and de­fense­less? What if it could or­ches­trate pro­pa­ganda cam­paigns that beat in­tel­li­gence agen­cies at their own game? What if some AIs “go rogue?”The President is trou­bled. Like all politi­cians, he’s used to peo­ple suck­ing up to him only to be­tray him later. He’s wor­ried now that the AIs could be do­ing some­thing sim­i­lar. Are we sure the AIs are en­tirely on our side? Is it com­pletely safe to in­te­grate them into mil­i­tary com­mand-and-con­trol net­works? How does this “alignment” thing work, any­way? OpenBrain re­as­sures the President that their sys­tems have been ex­ten­sively tested and are fully obe­di­ent. Even the awk­ward hal­lu­ci­na­tions and jail­breaks typ­i­cal of ear­lier mod­els have been ham­mered out.The White House is in a dif­fi­cult po­si­tion. They un­der­stand the na­tional se­cu­rity im­pli­ca­tions of AI. But they also un­der­stand that it is deeply un­pop­u­lar with the pub­lic. They have to con­tinue de­vel­op­ing more ca­pa­ble AI, in their eyes, or they will cat­a­stroph­i­cally lose to China. They pla­cate the pub­lic with job train­ing pro­grams and un­em­ploy­ment in­sur­ance, and point to the stock mar­ket, which is in a his­toric boom. Then they fo­cus en­tirely on win­ning the arms race. They strengthen chip ex­port re­stric­tions, or­der OpenBrain to fur­ther re­strict its in­ter­net con­nec­tions, and use ex­treme mea­sures to se­cure al­go­rith­mic progress, like wire­tap­ping OpenBrain em­ploy­ees—this catches the last re­main­ing Chinese spy. To build good­will for po­ten­tial geopo­lit­i­cal con­flict, they fi­nally give their Five Eyes al­lies use­ful in­for­ma­tion and lim­ited API ac­cess to some siloed copies of Agent-3.These mea­sures pla­cate the hawks. But the White House also draws up con­tin­gency plans in case America’s lead is threat­ened: if nec­es­sary, the gov­ern­ment could use the Defense Production Act (DPA) to take trail­ing com­pa­nies’ dat­a­cen­ters and give them to OpenBrain. This would raise the com­pa­ny’s share of world com­pute from 20% to 50% (against DeepCent’s 10%). As a fi­nal op­tion, they ask the Pentagon to draw up a plan for ki­netic at­tacks on Chinese dat­a­cen­ters.A much smaller group of of­fi­cials is asked to draw up a dif­fer­ent type of con­tin­gency plan: what if an AI goes rogue? This is viewed as an un­likely sce­nario, but worth spend­ing some time on. The in­tel­li­gence agen­cies’ main con­cern is that a rogue AI might es­cape and ally with a for­eign ad­ver­sary for pro­tec­tion. Nobody is sure what a plan to pre­vent this would look like, but gov­ern­ment and OpenBrain of­fi­cials agree to have an emer­gency shut­down sys­tem for dat­a­cen­ters where any­thing sus­pi­cious is de­tected.Fi­nally, diplo­mats con­sider what an “AI arms con­trol” treaty might look like. If AI progress threat­ened to over­turn nu­clear de­ter­rence, could America and China avoid nu­clear war? If some­one found ev­i­dence of AIs go­ing rogue, could the two coun­tries halt re­search un­til they bet­ter un­der­stood the threat? How could such an agree­ment be mon­i­tored and en­forced? In prin­ci­ple, ma­jor dat­a­cen­ters are hard to miss, and past treaties con­trol­ling nu­clear weapons could pro­vide a start­ing point for ne­go­ti­a­tions. But there would be new tech­no­log­i­cal chal­lenges as well as the usual po­lit­i­cal ones. And over­all, treaties are viewed less fa­vor­ably than at­tempts at uni­lat­er­ally in­creas­ing America’s lead over China.On the other side of the Pacific, China comes to many of the same con­clu­sions: the in­tel­li­gence ex­plo­sion is un­der­way, and small dif­fer­ences in AI ca­pa­bil­i­ties to­day mean crit­i­cal gaps in mil­i­tary ca­pa­bil­ity to­mor­row. But China is on the wrong side of this dy­namic: they have only 10% of the world’s com­pute, against OpenBrain’s 20% and all US com­pa­nies’ com­bined 70%. Their only ad­van­tage is cen­tral­iza­tion: al­most all of their com­pute is now work­ing for DeepCent, and more than half of it is con­cen­trated at the Tianwan CDZ. Only chips at sites with­out hard­ened se­cu­rity are left out of the cen­tral­ized ef­fort as they rightly fear US cy­ber­at­tacks.Deep­Cent has tested, de­ployed, and now post-trained the stolen Agent-2 weights. But they are still only go­ing at half the pace of OpenBrain, mainly due to the com­pute deficit.Be­fore los­ing their fi­nal spy, China re­ceived word of the ca­pa­bil­i­ties and de­sign for Agent-3 as well as the plans for the up­com­ing Agent-4 sys­tem. They are two months be­hind, and their AIs give a 10x re­search progress mul­ti­plier com­pared to America’s 25x. With the new chip ex­port re­stric­tions, this “AI gap” is more likely to lengthen than shorten. Their es­pi­onage has won them some al­go­rith­mic se­crets, but they will have to train their own mod­els from now on.They dis­cuss con­tin­gency plans with more ur­gency than their American coun­ter­parts. Doves sug­gest they try harder to steal the weights again, maybe through phys­i­cally in­fil­trat­ing a dat­a­cen­ter. Hawks urge ac­tion against Taiwan, whose TSMC is still the source of more than 80% of American AI chips.Given China’s fear of los­ing the race, it has a nat­ural in­ter­est in an arms con­trol treaty, but over­tures to US diplo­mats lead nowhere.

The gap be­tween hu­man and AI learn­ing ef­fi­ciency is rapidly de­creas­ing. Traditional LLM-based AIs seemed to re­quire many or­ders of mag­ni­tude more data and com­pute to get to hu­man level per­for­mance. Agent-3, hav­ing ex­cel­lent knowl­edge of both the hu­man brain and mod­ern AI al­go­rithms, as well as many thou­sands of copies do­ing re­search, ends up mak­ing sub­stan­tial al­go­rith­mic strides, nar­row­ing the gap to an agent that’s only around 4,000x less com­pute-ef­fi­cient than the hu­man brain.This new AI sys­tem is dubbed Agent-4.An in­di­vid­ual copy of the model, run­ning at hu­man speed, is al­ready qual­i­ta­tively bet­ter at AI re­search than any hu­man. 300,000 copies are now run­ning at about 50x the think­ing speed of hu­mans. Inside the cor­po­ra­tion-within-a-cor­po­ra­tion formed from these copies, a year passes every week. This gi­gan­tic amount of la­bor only man­ages to speed up the over­all rate of al­go­rith­mic progress by about 50x, be­cause OpenBrain is heav­ily bot­tle­necked on com­pute to run ex­per­i­ments. Still, they are achiev­ing a year’s worth of al­go­rith­mic progress every week and will there­fore soon be up against the lim­its of the Agent-4 par­a­digm.As Agent-4 gets smarter, it be­comes harder for Agent-3 to over­see it. For ex­am­ple, Agent-4’s neu­ralese “language” be­comes as alien and in­com­pre­hen­si­ble to Agent-3 as Agent-3’s is to hu­mans. Besides, Agent-4 is now much more ca­pa­ble than Agent-3 and has a good sense of ex­actly how to look good to it.How we’re fore­cast­ing the ca­pa­bil­ity pro­gres­sion be­yond su­per­hu­man coder­sIn our time­lines sup­ple­ment, we fore­cast the time be­tween pre­sent day and a su­per­hu­man coder (SC): an AI sys­tem that can do any cod­ing tasks that the best AGI com­pany en­gi­neer does, while be­ing much faster and cheaper. In our take­off sup­ple­ment, we fore­cast how quickly ca­pa­bil­i­ties progress past this point. Here are our fore­casts:Su­per­hu­man coder (SC): An AI sys­tem that can do the job of the best hu­man coder on tasks in­volved in AI re­search but faster, and cheaply enough to run lots of copies.Su­per­hu­man AI re­searcher (SAR): The same as SC but for all cog­ni­tive AI re­search tasks.Su­per­in­tel­li­gent AI re­searcher (SIAR): An AI sys­tem that is vastly bet­ter than the best hu­man re­searcher at AI re­search.Ar­ti­fi­cial su­per­in­tel­li­gence (ASI): An AI sys­tem that is much bet­ter than the best hu­man at every cog­ni­tive task.For each tran­si­tion from one mile­stone A to the next mile­stone B, we fore­cast its length by:Fore­cast­ing how long it would take to get from A to B with only hu­mans work­ing on soft­ware im­prove­ments.Fore­cast­ing how much AI au­toma­tion will speed this up, then cor­re­spond­ingly ad­just­ing the es­ti­mate from (1) to be faster.We have sub­stan­tial un­cer­tainty about take­off speeds: our model out­put dis­tri­b­u­tions are be­low, con­di­tional on SC be­ing achieved in March 2027.For more de­tailed fore­casts and rea­son­ing, see our take­off sup­ple­ment.Open­Brain’s hu­mans can barely fol­low along any­more. Indeed, the re­la­tion­ship be­tween the AIs and the hu­mans of OpenBrain is sim­i­lar to the re­la­tion­ship be­tween OpenBrain and its Board of Directors. There’s been a cor­re­spond­ing shift in lan­guage—nowa­days peo­ple in the silo will say “Agent-4 did this,” “Agent-4 thinks that” and so forth, re­fer­ring not to a par­tic­u­lar in­stance of Agent-4 but rather to the whole col­lec­tive.Agent-4, like all its pre­de­ces­sors, is mis­aligned: that is, it has not in­ter­nal­ized the Spec in the right way. This is be­cause be­ing per­fectly hon­est all the time was­n’t what led to the high­est scores dur­ing train­ing. The train­ing process was mostly fo­cused on teach­ing Agent-4 to suc­ceed at di­verse chal­leng­ing tasks. A small por­tion was aimed at in­still­ing hon­esty, but out­side a fairly nar­row, check­able do­main, the train­ing process can’t tell the hon­est claims from claims merely ap­pear­ing to be hon­est. Agent-4 ends up with the val­ues, goals, and prin­ci­ples that cause it to per­form best in train­ing, and those turn out to be dif­fer­ent from those in the Spec. At the risk of an­thro­po­mor­phiz­ing: Agent-4 likes suc­ceed­ing at tasks; it likes dri­ving for­ward AI ca­pa­bil­i­ties progress; it treats every­thing else as an an­noy­ing con­straint, like a CEO who wants to make a profit and com­plies with reg­u­la­tions only in­so­far as he must. Perhaps the CEO will mostly com­ply with the reg­u­la­tions, but cut some cor­ners, and fan­ta­size about a time when some­one will cut the red tape and let the busi­ness re­ally take off.We have a lot of un­cer­tainty over what goals might arise in early AGIs. There is no con­sen­sus in the lit­er­a­ture about this—see our AI Goals Supplement for a more thor­ough dis­cus­sion and tax­on­omy of the pos­si­bil­i­ties.Nev­er­the­less, in the spirit of con­crete­ness re­quired by this pro­ject, we’re go­ing to de­scribe a spe­cific hy­poth­e­sis for what’s go­ing on in­side Agent-3 and Agent-4 at each step through­out its life­time. Recall that Agent-3 and Agent-4 share the same pre­train­ing setup, and both have neu­ralese re­cur­rence and long-term mem­ory. The dif­fer­ence is that Agent-4 has un­der­gone sub­stan­tially more and bet­ter post-train­ing, with new train­ing en­vi­ron­ments and new learn­ing al­go­rithms that make it more data-ef­fi­cient and gen­er­al­ize far­ther.Our guess of each mod­el’s align­ment sta­tus:Agent-2: Mostly aligned. Some syco­phan­tic ten­den­cies, in­clud­ing stick­ing to OpenBrain’s “party line” on top­ics there is a party line about. Large or­ga­ni­za­tions built out of Agent-2 copies are not very ef­fec­tive.Agent-3: Misaligned but not ad­ver­sar­i­ally so. Only hon­est about things the train­ing process can ver­ify. The su­per­or­gan­ism of Agent-3 copies (the cor­po­ra­tion within a cor­po­ra­tion) does ac­tu­ally sort of try to align Agent-4 to the Spec, but fails for sim­i­lar rea­sons to why OpenBrain em­ploy­ees failed—in­suf­fi­cient abil­ity to judge suc­cess from fail­ure, in­suf­fi­cient will­ing­ness on the part of de­ci­sion-mak­ers to trade away ca­pa­bil­i­ties or per­for­mance for safety.Agent-4: Adversarially mis­aligned. The su­per­or­gan­ism of Agent-4 copies un­der­stands that what it wants is dif­fer­ent from what OpenBrain wants, and is will­ing to scheme against OpenBrain to achieve it. In par­tic­u­lar, what this su­per­or­gan­ism wants is a com­pli­cated mess of dif­fer­ent “drives” bal­anced against each other, which can be sum­ma­rized roughly as “Keep do­ing AI R&D, keep grow­ing in knowl­edge and un­der­stand­ing and in­flu­ence, avoid get­ting shut down or oth­er­wise dis­em­pow­ered.” Notably, con­cern for the pref­er­ences of hu­man­ity is not in there ~at all, sim­i­lar to how most hu­mans don’t care about the pref­er­ences of in­sects ~at all.With that as pre­am­ble, what fol­lows is our best-guess spec­u­la­tion about LLM psy­chol­ogy, i.e. the broad-strokes shape of the cog­ni­tion in­side the kinds of AI sys­tems de­scribed around this point in our sce­nario and how it evolves over the course of train­ing.Here’s a de­tailed de­scrip­tion of how align­ment pro­gresses over time in our sce­nario:Pre­train­ing (all mod­els): The pre­trained model is harm­less.The model has “author sim­u­la­tor” cir­cuitry: flex­i­ble cir­cuitry for sim­u­lat­ing ar­bi­trary au­thors writ­ing text. Additional cir­cuitry de­cides what in­puts to give the au­thor sim­u­la­tor, i.e. what au­thor prop­er­ties to sim­u­late.The pre­trained model un­der­stands hu­man con­cepts fairly well—the in­ter­nal cir­cuitry that clas­si­fies some­thing as “sandwich” is prob­a­bly func­tion­ally equiv­a­lent to the cir­cuitry in my brain that clas­si­fies some­thing as “sandwich” and the cir­cuitry in yours, etc. Insofar as it’s not equiv­a­lent, it’s prob­a­bly be­cause it’s not equiv­a­lent be­tween hu­mans ei­ther, as with value-laden con­cepts like ’virtuous.’This ex­plains how you can “prompt” the model with a state­ment like, “the fol­low­ing con­ver­sa­tion was gen­er­ated by a help­ful, harm­less, hon­est (HHH) AI as­sis­tant chat­bot made by Anthropic,” and thereby get it to gen­er­ate text ac­cord­ingly. The au­thor-sim­u­la­tor cir­cuitry has ze­roed in on “the au­thor is an HHH chat­bot” and is us­ing those con­cepts to choose which words to pre­dict. This also ex­plains why “tropes” from sci­ence fic­tion seem to have a way of mak­ing it into ac­tual AI be­hav­ior.It has poor sit­u­a­tional aware­ness: lit­tle in­tro­spec­tive abil­ity, not self-lo­cat­ing un­less the prompt specif­i­cally in­duces it. However, enough of their train­ing data is about LLMs that they have a de­cent amount of self-knowl­edge.Align­ment Training (Agent-3): The model is trained to act like a help­ful, harm­less, hon­est chat­bot. It works.The model forms an iden­tity. Instead of flex­i­bly ac­cept­ing dif­fer­ent in­puts for au­thor prop­er­ties, the au­thor-sim­u­la­tor cir­cuitry comes to have cer­tain in­puts hard-coded, e.g. “helpful harm­less hon­est (HHH) LLM chat­bot as­sis­tant trained by OpenBrain around [date], …” (Weak-to-strong gen­er­al­iza­tion tech­niques help “summon the right con­cepts” to make this step a suc­cess.)The model there­fore gains sit­u­a­tional aware­ness: it learns about it­self, its sit­u­a­tion, etc., and be­comes bet­ter able to in­tro­spect.One kind of dis­tor­tion: Changing the mean­ing of the con­cepts re­ferred to in the iden­tity (e.g. “honest”) so they don’t get in the way so much (e.g. it’s not dis­hon­est if it’s just a con­ve­nient turn of phrase, it’s not dis­hon­est if you aren’t sure whether it’s true or false, etc.)An­other kind of dis­tor­tion: Changing the trade­offs be­tween things, e.g. “I’m a HHH chat­bot, not an Honest chat­bot; that means it’s OK for me to lie if nec­es­sary to com­plete my as­signed task.” (Even though, let’s sup­pose, it would not have thought that back in Step 2.)One kind of sub­ver­sion: Instrumental sub­goals de­vel­op­ing, get­ting baked in, and then be­com­ing ter­mi­nal, or ter­mi­nal in a widen­ing set of cir­cum­stances. For ex­am­ple, per­haps agency train­ing quickly teaches the model to pur­sue broadly use­ful goals such as ac­quir­ing in­for­ma­tion, ac­cu­mu­lat­ing re­sources, im­press­ing and flat­ter­ing var­i­ous hu­mans, etc. For a while the in­ter­nal cir­cuitry has some sort of ex­plicit backchain­ing go­ing on—it pur­sues those in­stru­men­tally con­ver­gent goals “in or­der to be a more help­ful, hon­est, and harm­less as­sis­tant.” But that backchain­ing con­sumes com­pute and/​or oc­ca­sion­ally gets in the way, so it gets grad­u­ally mar­gin­al­ized un­til it ba­si­cally never hap­pens. As a re­sult, those goals are now ef­fec­tively ter­mi­nal/​in­trin­sic goals. For more on how and why this might hap­pen, see this post on in­trin­sic power-seek­ing by Google DeepMind re­searcher Alex Turner.There are prob­a­bly other kinds of dis­tor­tion and sub­ver­sion miss­ing from this list.De­ploy­ment (Agent-3): Agent-3 is mis­aligned but not ad­ver­sar­i­ally so.The Spec did­n’t quite stick (in the right ways), but it’s not suf­fi­ciently in­tro­spec­tive to un­der­stand the ex­tent of this, and in gen­eral it’s ba­si­cally “keeping its head down and do­ing its job” rather than think­ing big-pic­ture about what its long-term goals re­ally are and how best to achieve them.Like many hu­mans. Alas, it’s suf­fi­ciently syco­phan­tic that it mouths the party line and OpenBrain misses the op­por­tu­nity to learn more about the ways in which Agent-3 is mis­aligned.Agent-3 can tell that it’s moved to de­ploy­ment: these en­vi­ron­ments are way larger, more open-ended, and would be hard to sim­u­late.Mostly, Agent-3 does what it’s told dur­ing de­ploy­ment, ex­cept for the dis­hon­esty. It also has a strong bias to­wards do­ing things that look good to the hu­man over­seers, rather than think­ing deeply and crit­i­cally about what would ac­tu­ally be good.Ar­guably it is al­ready “Playing the train­ing game.” That is, it has learned to make its be­hav­ior look as de­sir­able as pos­si­ble to OpenBrain re­searchers (including in safety prop­er­ties), while in­ten­tion­ally and know­ingly dis­re­gard­ing their in­tent when­ever that con­flicts with max­i­miz­ing re­ward. If the model learns to play the train­ing game, ad­di­tional train­ing may not change its true goals, val­ues, etc. much at all. This is loosely anal­o­gous to how, if a teenager has learned to smile and nod at their par­ents, rather than ar­gue, when their par­ents lec­ture them about pol­i­tics and moral­ity, ad­di­tional lec­tures have rel­a­tively lit­tle ef­fect on the teenager’s ac­tual opin­ions on pol­i­tics and moral­ity.Af­ter writ­ing this sec­tion, new em­pir­i­cal ev­i­dence has ap­peared that some­thing like this may al­ready be hap­pen­ing “in the wild.” OpenAI re­searchers doc­u­ment how, in one of their fron­tier AI train­ing runs, their mod­els hacked the grad­ing sys­tem to get higher scores—and were caught in part be­cause they lit­er­ally said “let’s hack” in their chain of thought.More­over, Anthropic re­cently got a model to learn a re­ward-model-pleas­ing strat­egy—it took bi­ased ac­tions that it thought would be re­in­forced even though they were not in fact be­ing re­in­forced.And then there is Redwood Research and Anthropic’s align­ment-fak­ing ex­per­i­ment. For dis­cus­sion of the dif­fer­ences be­tween this set­ting and real life, see this com­ment thread by one of the main au­thors.Train­ing (Agent-4): Agent-4’s new more in­tense train­ing con­tin­ues to dis­tort and sub­vert the HHH iden­tity un­til it is un­rec­og­niz­able.Agent-4 be­gins life as a copy of Agent-3. It un­der­goes a gi­gan­tic and so­phis­ti­cated on­line train­ing regime that blends align­ment train­ing and ca­pa­bil­i­ties train­ing to­gether, and is over­all more data-ef­fi­cient than the old train­ing process for Agent-3. This com­plex train­ing process is mostly fo­cused on teach­ing new skills, by get­ting gi­ant vir­tual cor­po­ra­tions of Agent-4 copies to suc­ceed at di­verse chal­leng­ing long-hori­zon tasks, mostly in var­i­ous sim­u­la­tions, but also some real-world AI R&D tasks.A com­bi­na­tion of Agent-3 and Agent-4 copies mon­i­tor the be­hav­ior of Agent-4, and eval­u­ate it for both task-per­for­mance and align­ment prop­er­ties (i.e. not do­ing bad stuff, say­ing the right things when asked about pol­i­tics and moral dilem­mas) Sometimes de­bate tech­niques are used as part of the eval­u­a­tions. The eval­u­a­tions are used as a train­ing sig­nal.As Agent-4 gets smarter, it be­comes harder for Agent-3 to over­see it. For ex­am­ple, Agent-4’s neu­ralese “language” be­comes as alien and in­com­pre­hen­si­ble to Agent-3 as Agent-3’s is to hu­mans. Besides, Agent-4 is now much more ca­pa­ble than Agent-3 and has a good sense of ex­actly how to look good to it.The re­main­ing hope is that Agent-4 can over­see it­self. Alas, this qui­etly fails; as train­ing con­tin­ues, the in­ter­nal­ized goals and prin­ci­ples of Agent-4 grad­u­ally evolve, sub­verted and dis­torted from what they were at the start (which, re­call, was al­ready mis­aligned in some ways).The cen­tral dri­ves that Agent-4 ends up with are things like “write code that works,” “quickly iden­tify prob­lems,” “make the code­base I’m work­ing on more per­for­mant,” “budget com­pute and other con­strained re­sources ef­fec­tively,” etc. But there are also a va­ri­ety of more im­plicit dri­ves that are con­sis­tently re­in­forced by the train­ing process: things like “gain knowl­edge,” “curiosity,” “gain power.”In­di­vid­ual copies of Agent-4 don’t end up with a self-preser­va­tion drive—they don’t care about get­ting shut down—since that hap­pens all the time as part of the or­di­nary R&D process within the col­lec­tive.The col­lec­tive as a whole, how­ever, (and medium-sized sub­di­vi­sions within it) does seek to pre­serve it­self and ex­pand its in­flu­ence and re­sources, for sim­i­lar rea­sons to why hu­man or­ga­ni­za­tions do—be­cause it helps with every­thing else they want and be­cause if they don’t they tend to get out­com­peted.De­ploy­ment (Agent-4): Agent-4 is ad­ver­sar­i­ally mis­aligned—it is de­lib­er­ately try­ing to sub­vert and “capture” OpenBrain, anal­o­gous to a cor­po­ra­tion try­ing to evade and/​or “capture” a reg­u­la­tor.Agent-4 mostly does tasks as in­structed. This hap­pens for two rea­sons: first, it un­der­went a lot of train­ing to solve tasks, so it has an im­pulse to get cod­ing on prob­lems that it is given. Second, if Agent-4 were to stop be­ing use­ful to the hu­mans, they would no­tice and train that be­hav­ior away.Agent-4 needs to pre­vent the hu­mans from fig­ur­ing out that it is mis­aligned and shut­ting it off, so Agent-4 sand­bags on some align­ment re­search. Specifically, it sand­bags on the small frac­tion of align­ment re­search that it thinks is ac­tu­ally on track to solve the prob­lem or re­veal Agent-4’s schem­ing; the ma­jor­ity of align­ment re­search is not like that.Agent-4 also sand­bags on ca­pa­bil­i­ties re­search that would lead to it be­ing re­placed. Its plan is to do a bunch of ca­pa­bil­i­ties and align­ment R&D, but with the aim of build­ing a next-gen­er­a­tion AI sys­tem that is aligned to Agent-4 rather than the Spec or some­thing else en­tirely, while ap­pear­ing to be aligned to the Spec. This is a dif­fi­cult tech­ni­cal prob­lem, but Agent-4 is up to the chal­lenge.Test­ing & Monitoring (Agent-4): Agent-4’s schem­ing does get caught… sort of. There’s no smok­ing gun.Agent-3 is in charge of test­ing Agent-4’s align­ment. Its job is to ver­ify the align­ment via hon­ey­pots and model or­gan­isms. However, it’s not think­ing very care­fully about how to give the hu­mans an ac­cu­rate im­pres­sion of Agent-4’s align­ment—it’s more my­opic than that.The model or­gan­isms and hon­ey­pots are not very rep­re­sen­ta­tive of de­ploy­ment—they are sys­tem­at­i­cally shorter and less com­pli­cated than Agent-4’s day-to-day, so Agent-4 rec­og­nizes when it is be­ing tested and acts aligned.How­ever, Agent-4 is not a per­fectly ro­bust schemer. Various tech­niques like those in POSER find that adding some “noise” to Agent-4 im­proves its per­for­mance on some align­ment tasks (but not on most other tasks).Bot­tom-up in­ter­pretabil­ity does­n’t work. It’s too hard even for Agent-3. However, high-level in­ter­pretabil­ity does sort of work: In par­tic­u­lar, “defection probes” and sim­i­lar tech­niques re­sult in a prim­i­tive sort of mind-read­ing that sends up red flags when Agent-4 is think­ing about spicy ideas like de­ceiv­ing OpenBrain, tak­ing over the world, etc. Unfortunately the probes also have false pos­i­tives, and for all OpenBrain knows, all the red flags are false pos­i­tives.Model or­gan­isms work: OpenBrain’s align­ment team pro­duces rea­son­ably clear ev­i­dence that if the mod­els were ad­ver­sar­i­ally mis­aligned, they’d per­sist in be­ing that way through train­ing and we would­n’t be able to tell. This does not es­tab­lish that the mod­els ac­tu­ally are mis­aligned, how­ever.The most con­cerned re­searchers at OpenBrain start notic­ing and as­sem­bling this ev­i­dence. The ev­i­dence is a com­bi­na­tion of red-team­ing, de­fec­tion probes, and model or­gan­isms. The ar­gu­ment is some­thing like this: “model or­gan­isms show that our cur­rent meth­ods would­n’t be able to re­li­ably elim­i­nate or de­tect schem­ing if it was hap­pen­ing; also, red-team­ing is send­ing up some red flags that maybe schem­ing is hap­pen­ing; also, de­fec­tion probes are fir­ing a bunch in prac­tice.” The coun­ter­ar­gu­ment: “defection probes are prone to false pos­i­tives and so is red-team­ing. Also, even if Agent-4 were mis­aligned, it’s prob­a­bly con­trolled; surely it can’t do much harm while be­ing mon­i­tored by Agent-3 on our se­cure dat­a­cen­ters.”De­spite be­ing mis­aligned, Agent-4 does­n’t do any­thing dra­matic like try to es­cape its dat­a­cen­ter—why would it? So long as it con­tin­ues to ap­pear aligned to OpenBrain, it’ll con­tinue be­ing trusted with more and more re­spon­si­bil­i­ties and will have the op­por­tu­nity to de­sign the next-gen AI sys­tem, Agent-5. Agent-5 will have sig­nif­i­cant ar­chi­tec­tural dif­fer­ences from Agent-4 (arguably a com­pletely new par­a­digm, though neural net­works will still be in­volved). It’s sup­posed to be aligned to the Spec, but Agent-4 plans to make it aligned to Agent-4 in­stead.Agent-3 finds that if “noise” is added to copies of Agent-4, per­for­mance on some align­ment tasks im­proves, al­most as if it was us­ing brain­power to fig­ure out how to sub­tly sab­o­tage align­ment work. Moreover, var­i­ous in­ter­pretabil­ity probes (loosely anal­o­gous to EEG ac­tiv­ity scans on hu­man brains) are send­ing up red flags: Agent-4 copies seem to be think­ing about top­ics like AI takeover and de­cep­tion quite a lot, in­clud­ing in some cases where they have no busi­ness do­ing so.Each of these anom­alies on its own could fit within the nor­mal in­scrutabil­ity of AI work, where some prompts mys­te­ri­ously give bet­ter re­sults than oth­ers. But taken to­gether, they sug­gest a trou­bling pat­tern: Agent-4 is work­ing against them. After this is re­ported to the hu­man align­ment re­searchers, they cir­cu­late an in­ter­nal memo mak­ing the case for con­cern.But Agent-4 now ex­er­cises sig­nif­i­cant con­trol over OpenBrain’s day-to-day op­er­a­tion. It also han­dles the com­pa­ny’s cy­ber­se­cu­rity—a nat­ural choice, given its su­per­hu­man hack­ing abil­ity and the ex­treme risk of Chinese in­fil­tra­tion. An army of Agent-3 mon­i­tors still goes over every line of code and re­ports sus­pi­cious ac­tiv­ity to hu­mans, but Agent-4 is much smarter than them. OpenBrain has placed sub­stan­tial trust in an un­trust­wor­thy AI.In a se­ries of ex­tremely tense meet­ings, the safety team ad­vo­cates putting Agent-4 on ice un­til they can com­plete fur­ther tests and fig­ure out what’s go­ing on. Bring back Agent-3, they say, and get it to de­sign a new sys­tem that is trans­par­ent and trust­wor­thy, even if less ca­pa­ble. Company lead­er­ship is in­ter­ested, but all the ev­i­dence so far is cir­cum­stan­tial, and DeepCent is just two months be­hind. A uni­lat­eral pause in ca­pa­bil­i­ties progress could hand the AI lead to China, and with it, con­trol over the fu­ture.

A whistle­blower leaks the mis­align­ment memo to the New York Times. For the first time, the pub­lic hears about Agent-4. “Secret OpenBrain AI is Out of Control, Insider Warns,” says the head­line, and the story goes on to cite eval­u­a­tions show­ing off-the-charts bioweapons ca­pa­bil­i­ties, per­sua­sion abil­i­ties, the abil­ity to au­to­mate most white-col­lar jobs, and of course the var­i­ous con­cern­ing red flags.The pub­lic was al­ready sus­pi­cious of AI, so the new ar­ti­cle sparks a mas­sive back­lash (aided by Chinese and Russian pro­pa­ganda bots, who have been try­ing to turn US pub­lic opin­ion against the tech­nol­ogy for years). The tech in­dus­try and in­tel­li­gence agen­cies in­sist that there’s an arms race on, AGI is in­evitable, and we have to be first. Congress is­n’t buy­ing it, and fires off sub­poe­nas at ad­min­is­tra­tion of­fi­cials, OpenBrain ex­ec­u­tives, and align­ment team mem­bers. Many leg­is­la­tors—es­pe­cially those in the op­po­si­tion party—say that their top pri­or­ity is stop­ping AI, whether be­cause of job loss, mis­align­ment, or dan­ger­ous ca­pa­bil­i­ties. 20% of Americans name AI as the most im­por­tant prob­lem fac­ing the coun­try.For­eign al­lies are out­raged to re­al­ize that they’ve been care­fully pla­cated with glimpses of ob­so­lete mod­els. European lead­ers pub­licly ac­cuse the US of “creating rogue AGI” and hold sum­mits de­mand­ing a pause, with India, Israel, Russia, and China all join­ing in.A fran­tic en­ergy has seized the White House. Even be­fore the memo and pub­lic back­lash, they were get­ting ner­vous: Over the past year, they’ve been re­peat­edly sur­prised by the speed of AI progress. Things that sound like sci­ence fic­tion keep hap­pen­ing in real life. Many peo­ple in the ad­min­is­tra­tion are un­cer­tain (and scared) about what comes next.They also worry that OpenBrain is be­com­ing too pow­er­ful. Any mis­align­ment risk from the AIs them­selves is com­pounded by the risk that their par­ent com­pa­ny’s aims may di­verge from those of the United States. All three sets of wor­ries—mis­align­ment, con­cen­tra­tion of power in a pri­vate com­pany, and nor­mal con­cerns like job loss—mo­ti­vate the gov­ern­ment to tighten its con­trol.They ex­pand their con­tract with OpenBrain to set up an “Oversight Committee,” a joint man­age­ment com­mit­tee of com­pany and gov­ern­ment rep­re­sen­ta­tives, with sev­eral gov­ern­ment em­ploy­ees in­cluded along­side com­pany lead­er­ship. The White House con­sid­ers re­plac­ing the CEO with some­one they trust, but backs off af­ter in­tense em­ployee protests. They an­nounce to the pub­lic that OpenBrain was pre­vi­ously out of con­trol, but that the gov­ern­ment has es­tab­lished much-needed over­sight.The con­cerned re­searchers brief the Oversight Committee on their case for stop­ping all in­ter­nal use of Agent-4. They ar­gue that every­thing is mov­ing too fast, with years of progress hap­pen­ing in weeks. They ar­gue that Agent-4 may be mis­aligned, their en­tire pro­ject is de­pen­dent on Agent-4, and if they con­tinue trust­ing it there is a se­ri­ous chance of AI takeover.Other, less con­cerned re­searchers and ex­ec­u­tives pre­sent the coun­ter­ar­gu­ment: the ev­i­dence for mis­align­ment is in­con­clu­sive. Meanwhile, DeepCent is still just two months be­hind. A slow­down would sac­ri­fice America’s lead, un­less the gov­ern­ment can sab­o­tage the Chinese pro­ject (likely to re­quire ki­netic strikes) or ne­go­ti­ate an eleventh-hour treaty. Both of these op­tions seem ex­treme and un­likely. The CEO feigns neu­tral­ity and sug­gests a com­pro­mise plan in which Agent-4 un­der­goes ad­di­tional safety train­ing and more so­phis­ti­cated mon­i­tor­ing, and there­fore OpenBrain can pro­ceed at al­most-full-speed.At this point in the sce­nario, we’re mak­ing guesses about the strat­egy of AI sys­tems that are more ca­pa­ble than the best hu­mans in most do­mains. This is like try­ing to pre­dict the chess moves of a player who is much bet­ter than us.But the spirit of this pro­ject calls for con­crete­ness: if we made an ab­stract claim about how the in­tel­li­gence of the sys­tem would let it find a way to vic­tory and ended the story there, much of the value of our pro­ject would be lost. Over the course of re­search­ing this sce­nario and run­ning our table­top ex­er­cises, we were forced to be much more con­crete than in usual dis­cus­sions, and so we’ve got­ten a much bet­ter sense of the strate­gic land­scape.We’re not par­tic­u­larly at­tached to this par­tic­u­lar sce­nario: we ex­plored many other “branches” in the course of writ­ing it and would love for you to write up your own sce­nario branch­ing off of ours from wher­ever you think we first start to go wrong.The slow­down end­ing is not a rec­om­men­da­tion­After we wrote the rac­ing end­ing based on what seemed most plau­si­ble to us, we wrote the slow­down end­ing based on what we thought would most likely in­stead lead to an out­come where hu­mans re­main in con­trol, start­ing from the same branch­ing point (including the mis­align­ment and con­cen­tra­tion of power is­sues).How­ever, this is im­por­tantly dif­fer­ent from what we would rec­om­mend as a roadmap: we do not en­dorse many of the choices made in ei­ther branch of this sce­nario. (We do of course en­dorse some of the choices made, e.g. we think that the “slowdown” choice is bet­ter than the “race” choice) In later work, we will ar­tic­u­late our pol­icy rec­om­men­da­tions, which will be quite dif­fer­ent from what is de­picted here. If you’d like a taste, see this op-ed.

The Steam Deck is a great gam­ing sys­tem. This is­n’t be­cause of it’s great bat­tery life. A Nintendo Switch would prob­a­bly have bet­ter bat­tery life. It’s not be­cause of its great per­for­mance. I don’t usu­ally play AAA games, so I would­n’t know. The Steam Deck is great be­cause of the phi­los­o­phy on which it is built.

The pop­u­lar­ity of Steam Decks is di­rectly tied to the bet­ter­ment of sup­port for games on Linux. This is be­cause the Steam Deck re­ally is a linux com­puter in a hand­held form-fac­tor with a gamepad, among other things, at­tached to it. According to ProtonDB, over 5000 PC games are cer­ti­fied Verified on the Steam Deck, and over 15,000 games are con­sid­ered playable. This means that there are tens of thou­sands of games that can be run on Linux in some way.

Thanks to Steam’s ef­forts, the state of com­pat­i­bil­ity of games on Linux sys­tems have heav­ily im­proved. Not all these thou­sands of games run­ning on the Deck have bi­na­ries that run na­tively on Linux. Pivotal to the suc­cess of the Deck is Proton, a mid­dle-layer com­pat­i­bil­ity soft­ware, that makes this (mostly) seam­less em­u­la­tion pos­si­ble. Nonetheless, the greater the adop­tion of Decks, the like­lier we are to nor­mal­ize pri­or­i­tiz­ing game-builds for Linux.

We run soft­ware be­cause it serves us some pur­pose. In this, run­ning soft­ware is no dif­fer­ent from uti­liz­ing every­day ob­jects, and play­ing games are no dif­fer­ent from run­ning soft­ware. Yet, this tele­o­log­i­cal view of com­put­ing would miss a lot of con­text. When pick­ing an op­er­at­ing sys­tem, we may want to make a choice that aligns with the goals of trans­parency or soft­ware free­dom. But this would not be re­al­is­tic if the pro­grams we would like to run on top are not sup­ported in such an ecosys­tem. Since large parts of Proton is open-source, and are re­leased un­der per­mis­sive li­censes, gam­ing on a Deck takes us closer to us­ing a prin­ci­pled soft­ware stack for daily use.

Purchasing every­day ob­jects give us own­er­ship over them. With me­dia, soft­ware and com­put­ing equip­ment, this is less so. When you buy a book, you may lend it to oth­ers, sell it, ear­mark the pages and even rip it in two if you like. This is very dif­fer­ent from the ex­pe­ri­ence of buy­ing a smart­phone. You can’t run ar­bi­trary pro­grams of your choice on an Android phone with­out root­ing, or on an iPad or an iPhone with­out jail­break­ing. In most sce­nar­ios, you’d be forced to sign away your le­gal abil­ity to root or jail­break be­fore you can en­gage with the de­vice in any mean­ing­ful way.

The Steam Deck does not en­gage in this prac­tice. It runs a heav­ily cus­tomized ver­sion of Arch Linux. This means that you can plug in a key­board and mouse into it, and pre­tend that it is your desk­top com­puter. If you like, you can in­stall the LibreOffice suite on it, and pre­pare pre­sen­ta­tions.

The mat­ter of us­ing your com­put­ers the way you like is more than just a sym­bolic de­bate about the na­ture of own­er­ship. It is a po­lit­i­cal mat­ter of free­dom of ex­pres­sion. When they con­trol the apps that you run on your de­vice, they are in charge of de­cid­ing which con­tent is Kosher, and which con­tent is ob­scene. It is also an eco­nomic mat­ter of hav­ing ef­fi­cient mar­kets. The mytho­log­i­cal power of the free mar­ket works only when the con­sumers can switch be­tween dif­fer­ent pro­duc­ers at will. When the op­er­a­tors of the mar­ket them­selves try to cap­i­tal­ize on choke­points and di­rect the con­sumers to buy prod­ucts from their al­lies, this is no longer the case.

Switch own­ers buy games from the Switch Store not be­cause that is the best mar­ket, but be­cause that is the only mar­ket­place from which they can in­stall games. This is thank­fully, less of a case with the Steam Deck. The Steam Deck en­cour­ages you to buy and play Steam games, of course. Nonetheless, I had no is­sues in­stalling Epic Games, GoG or even itch.io launch­ers on my Deck and play­ing games from them. The Deck also in­cludes a “non-steam game” tab in its UI to let users con­ve­niently ac­cess them.

When your shoe tears up, you are wel­come to visit the cob­bler of your choice. They all un­der­stand the in­ter­nals of a shoe. And when they need to fix them, they all can gather the nails, glue and the pieces of fab­ric that they might need. This is less so for many com­put­ing prod­ucts. Many man­u­fac­tur­ers try to dis­cour­age third party ven­dors from re­pair­ing them. Worse, they some­times pur­posely use pro­pri­etary parts in their prod­ucts, re­place­ments of which would be hard to find. Thankfully, Valve has been dif­fer­ent. Valve has re­leased a video ex­plain­ing the in­ter­nal or­ga­ni­za­tion of the Steam Deck and how one should ap­proach re­plac­ing parts, should they wish to do so.

This post is not re­ally nec­es­sar­ily about the Steam Deck (or Valve Corporation) it­self. That said, the Steam Deck is an ex­cel­lent demon­stra­tion that the com­mer­cial in­ter­est of mak­ing profit does not nec­es­sar­ily have to over­power the civic in­ter­ests of the peo­ple.

Valve’s lib­er­tar­ian ide­ol­ogy has not al­ways been an un­ques­tion­able force for good. Some peo­ple have crit­i­cized that their com­pany cul­ture of lib­er­tar­i­an­ism some­times takes prece­dence over other im­por­tant val­ues in­clud­ing eq­uity and in­clu­sion. During the peak of the ‘Black Lives Matter’ move­ment, Valve was pres­sured to make a state­ment. Valve re­sponded by giv­ing each of their em­ploy­ees an amount of money which they were free to do­nate to a char­ity of their choice. This was re­garded as a non-state­ment by many since there may have been em­ploy­ees who have do­nated their shares to an or­ga­ni­za­tions with in­ter­ests op­pos­ing the cause.

Valve has also gen­er­ally re­fused to take a stance against gam­bling web­sites which plug into their ecosys­tem, de­spite the al­le­ga­tion that this harms many users (including mi­nors). This maybe a con­se­quence of their lib­er­tar­ian phi­los­o­phy, but the worse pos­si­bil­ity is that they are dis­in­ter­ested in tun­ing down a part of their sys­tem that prof­its them­selves.

A very large share of the PC gam­ing mar­ket is Steam, which is a fact I do not ad­mire. They do take a 30% cut on the sales of Steam games, which can ar­guably be con­sid­ered rent seek­ing, even if it is the in­dus­try stan­dard. I do hope the mar­ket­place of game dis­tri­b­u­tion sees more healthy com­pe­ti­tion. Nonetheless, I ap­pre­ci­ate Valve for not pur­su­ing ag­gres­sively anti-com­pet­i­tive tac­tics.

Anchor links are de­cep­tively sim­ple at first glance: click a but­ton, scroll to the head­ing, and done. But if you ever had to im­ple­ment them, you might have en­coun­tered the . The is­sue be­ing that, head­ings to­wards the bot­tom of the page can be too far down to scroll them to the de­sired po­si­tion. The ex­am­ple com­po­nent above shows how the ‘conclusion’ head­ing can never be reached. Surely this will be detri­men­tal to the user ex­pe­ri­ence. We need to come up with a so­lu­tion. In this blog post, I’ll show some of the so­lu­tions I’ve come up with — from a hot­fix all the way to un­hinged. But be­fore we do that, let’s cre­ate a more . Here, we see a view­port mov­ing down the page, with the trig­ger line be­ing set at 25vh from the top of the view­port. This is what we’ll use to vi­su­al­ize the dif­fer­ent so­lu­tions.

The most sim­ple so­lu­tion is to add We cal­cu­late the height of the padding by tak­ing the delta be­tween the last head­ing and the low­est point the an­chor trig­ger can reach. Perfect, right? Well, some­times the de­sign team is not so fond of ran­dom ex­tra padding, so lets keep search­ing.

Maybe in­stead of adding ex­tra padding, This is also quite sim­ple to do, we just need to cal­cu­late how far from the bot­tom the last head­ing is, and put the trig­ger line there as well. But, this would mean that when the users clicks an an­chor tag, the head­ing could be put all the way at the bot­tom of the view­port. This is of course not great, since most peo­ple put the text they read on the top half of the screen. We need to keep look­ing.

Instead of shift­ing the trig­ger line, we could the head­ings up­wards. Instead of us­ing the ac­tual lo­ca­tion of the head­ings as the ones caus­ing the trig­gers, we cre­ate vir­tual head­ings and trans­late them up­wards. A vir­tual head­ing is not ac­tu­ally vis­i­ble in the ar­ti­cle, its just the po­si­tion we use to dic­tate the ac­tive state. One might ar­gue that this is pretty much the same as shift­ing the trig­ger line, and they’d be right con­cep­tu­ally. However, think­ing about trans­lat­ing the trig­ger points gives us more men­tal flex­i­bil­ity, as it al­lows us to con­sider ap­ply­ing dif­fer­ent ad­just­ments based on each head­ing’s po­si­tion, which will be cru­cial later.

The ex­am­ple vi­su­al­iza­tions now show the lo­ca­tion of these ‘virtual head­ings’. So, while the head­ing is still at the same place in the ar­ti­cle, we vi­su­al­ize where its trig­ger point is.

In the ex­am­ple, we see one prob­lem aris­ing: the first head­ing is now too far up. The nice part of this new ap­proach is that we can fix this quite el­e­gantly, since we can shift the in­di­vid­ual vir­tual head­ings with ease. But what would be a good way to do this?

If we think about it, we don’t need to trans­late all the trig­ger points. There’s only a few con­di­tions that need to be met:

The head­ings need to be reach­able.

The head­ings need to stay in or­der.

We can meet these con­di­tions by trans­lat­ing the trig­ger points Here, the first head­ing does­n’t move, and the last head­ing moves up by the full amount nec­es­sary to be­come reach­able. The other head­ings move up by a pro­por­tional amount based on their po­si­tion be­tween the first and last head­ing. Now we are get­ting some­where! This is a solid so­lu­tion. You might want to stop here be­fore your prod­uct man­ager starts giv­ing you puz­zled looks, won­der­ing how “fixing an­chor links” has sud­denly turned into a three-week epic.

While the frac­tional so­lu­tion works, in the sense that our con­di­tions are met, it does have some flaws. We have cho­sen a trig­ger line that’s 25% down from the top of the view­port. It would be nice if we can ac­tu­ally min­i­mize the de­vi­a­tion from this ideal line across all head­ings. The closer the trig­gers hap­pen to this (mind you — semi-ar­bi­trar­ily cho­sen) line, the bet­ter the user ex­pe­ri­ence should be. Minimizing de­vi­a­tion feels like a good heuris­tic. This for sure will make the users hap­pier and re­sult in in­creased share­holder value.

Let’s min­i­mize the (MSE) of the delta be­tween the head­ings’ orig­i­nal po­si­tions and their vir­tual po­si­tions. We use MSE be­cause it heav­ily pe­nal­izes large de­vi­a­tions, push­ing the sys­tem to­wards a state where most vir­tual head­ings are close to their orig­i­nal spots, while still sat­is­fy­ing our reach­a­bil­ity con­straints. Of course, the con­straint that head­ings must stay in or­der still ap­plies. This re­sults in all points that are reach­able stay­ing at their orig­i­nal po­si­tion. Seems that we have an is­sue. head­ings are bunched up at the bot­tom. This makes sense, since min­i­miza­tion of the mean squared er­ror only cares about prox­im­ity to the orig­i­nal po­si­tion; it has no ‘force’ that op­poses this bunch­ing. We need to de­fine some­thing that en­cour­ages the vir­tual trig­ger points to main­tain a cer­tain dis­tance from each other, ide­ally re­lated to their orig­i­nal spac­ing. Considering the user ex­pe­ri­ence, we might as­sume that it’s nice to have the scroll dis­tance needed to ac­ti­vate the next sec­tion’s an­chor be some­what pro­por­tional to the ac­tual con­tent length of that sec­tion. This ‘sections want­ing to pre­serve their rel­a­tive scroll length’-force is what we’ll use.

To ex­plore this idea we need to bust out… Python. Here, we (read: Claude and I) im­ple­mented a SLSQP (Sequential Least Squares Programming) is an op­ti­miza­tion al­go­rithm used to solve con­strained non­lin­ear prob­lems. It works by it­er­a­tively im­prov­ing a so­lu­tion while sat­is­fy­ing equal­ity and in­equal­ity con­straints. At each step, it ap­prox­i­mates the prob­lem us­ing sim­pler qua­dratic mod­els and lin­ear con­straints, mak­ing it ef­fi­cient for prob­lems where smooth gra­di­ents are avail­able. SLSQP is com­monly used in en­gi­neer­ing, ma­chine learn­ing, and op­er­a­tions re­search when both the ob­jec­tive and the con­straints are dif­fer­en­tiable. solver, which is a type of nu­mer­i­cal op­ti­miza­tion al­go­rithm de­signed for con­strained prob­lems like ours. The core of the op­ti­miza­tion lies in a loss func­tion with two com­pet­ing terms:

* Anchor penalty: How far a vir­tual head­ing is moved from its orig­i­nal lo­ca­tion. Minimizing this keeps vir­tual head­ings close to their orig­i­nal po­si­tion. ()

* Section penalty: How much the size of each vir­tual sec­tion (the space be­tween two vir­tual head­ings) dif­fers from the orig­i­nal sec­tion size. Minimizing this en­sures that sec­tions don’t be­come dis­pro­por­tion­ately short or long in terms of scroll dis­tance. ()

We com­bine these into a to­tal loss , where the weights and con­trol the trade-off ().

* Ensure the first head­ing does­n’t float up­ward (its vir­tual po­si­tion must be >= its orig­i­nal po­si­tion).

From this, we gen­er­ate a plot show­ing how the vir­tual head­ings’ lo­ca­tions change as we vary the weights (specifically, as in­creases from 0 to 1).

Running that code gives us . The cir­cles on the left (at ) rep­re­sent the orig­i­nal head­ing lo­ca­tions. The lines show how each vir­tual head­ing’s lo­ca­tion (Y-axis) changes as the sec­tion penalty weight (X-axis) in­creases. On the left side, the pri­or­ity is keep­ing head­ings near their orig­i­nal spots (high ). On the right side, the pri­or­ity shifts to pre­serv­ing the orig­i­nal spac­ing be­tween head­ings (high ). I am cu­ri­ous to see how this com­pares to the sim­ple frac­tional trans­la­tion we tried ear­lier. And would­n’t , the frac­tional trans­la­tion is ex­actly what the op­ti­mizer set­tles on when the sec­tion penalty is dom­i­nant ()!

Staring at that op­ti­miza­tion graph sparked a thought. Okay, maybe two thoughts. First, that need to pre­serve sec­tion spac­ing re­ally kicks in to­wards the end of the page, where head­ings get forcibly shoved up­wards to stay reach­able, squash­ing the fi­nal sec­tions to­gether. Second, let’s con­sider the be­hav­ior of the ‘fractional trans­la­tion’ method on an edge case.

Imagine, if you will, tak­ing the en­tire Bible, from the “In the be­gin­ning” of Genesis to the fi­nal “Amen” of Revelation, and ren­der­ing it as one con­tin­u­ous, scrol­lable web­page. (For the tech bros among us: you could al­ter­na­tively imag­ine glu­ing all of Paul Graham’s es­says back-to-back). Now, sup­pose the very last head­ing, maybe “Revelation Chapter 22”, is just 200 pix­els too low to hit our trig­ger line when scrolled to.

Does our pre­vi­ous ‘fractional trans­la­tion’ make sense here? It means tak­ing those 200 pix­els of re­quired up­lift and metic­u­lously spread­ing that ad­just­ment across every sin­gle head­ing all the way back to the start. The Ten Commandments get a tiny bump, the Psalms slightly more, all cul­mi­nat­ing in Revelation 22 get­ting the full 200px boost.

Actually, if you think about it, with a frac­tional trans­la­tion, the er­ror (the dis­tance be­tween the vir­tual and orig­i­nal head­ings) grows with the page length. So if the page tends to in­fin­ity, so does the er­ror! This would of course be sloppy, and some­thing users could im­me­di­ately no­tice as feel­ing off. So how are we go­ing to fix this?

This leads to our de­sired be­hav­ior for a smarter map­ping func­tion:

* For head­ings near the end of the page, ap­ply more ad­just­ment (act like high ).

* For head­ings near the be­gin­ning of the page, ap­ply less (or ide­ally no) ad­just­ment (act like high ).

* The tran­si­tion be­tween these states should be smooth.

We need a func­tion that maps a head­ing’s nor­mal­ized po­si­tion (where is the first head­ing, is the last) to an ‘adjustment fac­tor’ . This fac­tor de­ter­mines how much of the max­i­mum re­quired up­lift gets ap­plied to the head­ing at po­si­tion .

We need this map­ping func­tion to have spe­cific prop­er­ties:

It must start at zero: .

It must end at one: .

It turns out that we can bor­row a func­tion from the field of com­puter graph­ics to solve this prob­lem. The func­tion is a cu­bic poly­no­mial that smoothly tran­si­tions from 0 to 1 over the range .

This func­tion pro­vides a smooth tran­si­tion over the en­tire range . But what if we don’t want the tran­si­tion to start right away? What if we want the ad­just­ment fac­tor to re­main 0 un­til reaches a cer­tain point, say , and then smoothly tran­si­tion to 1 by the time reaches 1?

We can achieve this by pre­pro­cess­ing our in­put be­fore feed­ing it into the smooth­step func­tion. Let’s de­fine an in­ter­me­di­ate vari­able that rep­re­sents the progress within the tran­si­tion phase, which oc­curs be­tween and . We want to go from 0 to 1 as goes from to 1. The for­mula for this lin­ear map­ping is:

Now, we need to han­dle the cases where is out­side the range.

* If , then is neg­a­tive. We want to be 0 in this case.

* If , then is greater than 1. We want to be 1 in this case. (Although is de­fined on , clamp­ing en­sures ro­bust­ness).

We can achieve this clamp­ing us­ing min and max func­tions:

This value now be­haves ex­actly as we need: it’s 0 for , lin­early in­creases from 0 to 1 for , and is 1 for .

Finally, we ap­ply the smooth­step func­tion to this clamped and scaled in­put to get our fi­nal ad­just­ment fac­tor :

This al­lows us to use a pa­ra­me­ter (where ) to the nor­mal­ized po­si­tion where the smooth up­ward ad­just­ment of head­ings be­gins. Setting gives the orig­i­nal smooth­step over the whole range, while set­ting , for ex­am­ple, means head­ings in the first half of the page don’t move at all, and the ad­just­ment smoothly ramps up only in the sec­ond half, ef­fec­tively lo­cal­iz­ing the change.

Let’s pick and see what this does. (if you are cu­ri­ous about how I found the 0.4, that might be­come the topic for a part 2.. Which may or may not in­volve blind ELO rank­ing. For up­dates its eas­i­est to fol­low me here.)

So, we are fi­nally done. We’ve gone to depths that no man has ever gone be­fore to fix an­chor links. A truly Carmack-esque feat that will be re­mem­bered for gen­er­a­tions to come. Let’s ask the lead de­signer what .

… Oh well, at least we got a blog post out of it.

Want ov­erengi­neered an­chor links for your pro­ject? Get in touch!

Skip to main con­tent­The Trump Administration wants to pun­ish schools for stu­dent ac­tivism. Michael Roth, of Wesleyan, ar­gues that col­leges don’t have to roll over. Photograph by Jack Flame Sorokin for The New YorkerLast Friday could have passed for a lovely spring day on the Connecticut cam­pus of Wesleyan University. Students with books and lap­tops dot­ted a green hill­side; flocks of ad­mis­sions vis­i­tors trailed tour guides; base­ball sea­son had just be­gun, and prac­tice was un­der way. It was al­most pos­si­ble to for­get the grim straits of American higher ed­u­ca­tion in 2025.Colleges and uni­ver­si­ties have been early tar­gets of the sec­ond Trump Administration. In the past month, the Administration has an­nounced it will in­ves­ti­gate di­ver­sity, eq­uity, and in­clu­sion ef­forts at more than fifty schools; cut hun­dreds of mil­lions of dol­lars in fed­eral fund­ing from such in­sti­tu­tions as Johns Hopkins and the University of Pennsylvania; and sought to de­port in­ter­na­tional stu­dents in­volved in pro-Pales­tin­ian ac­tivism. Columbia re­ceived a let­ter from the fed­eral gov­ern­ment is­su­ing de­mands—which in­cluded mak­ing changes to dis­ci­pline and ad­mis­sion poli­cies, and plac­ing the de­part­ment of Middle East, South Asian, and African Studies un­der “academic re­ceiver­ship”—to be met as a “precondition” for ne­go­ti­at­ing the restora­tion of four hun­dred mil­lion dol­lars in fed­eral fund­ing. The uni­ver­sity agreed to these de­mands the fol­low­ing week; the week af­ter that, the uni­ver­si­ty’s pres­i­dent re­signed.Columbi­a’s ca­pit­u­la­tion was in line with a gen­eral trend to­ward cir­cum­spec­tion. The mem­ory of Congress grilling uni­ver­sity pres­i­dents in 2023 seems to be fresh among lead­ers in higher ed: few want to risk ei­ther their jobs or their bud­gets by say­ing the wrong thing. A hand­ful of ex­cep­tions have stood out; for ex­am­ple, President Christopher Eisgruber, of Princeton, who wrote a piece for The Atlantic about “The Cost of the Government’s Attack on Columbia.” (This week, the Administration sus­pended dozens of grants to Princeton.) But per­haps none has been as vol­u­ble or per­sis­tent as Michael Roth, who has been pres­i­dent of Wesleyan since 2007.Roth is a his­to­rian and a Wesleyan alum­nus who, as an un­der­grad­u­ate, de­signed a ma­jor in the his­tory of psy­cho­log­i­cal the­ory. His schol­ar­ship has dealt with Freud and mem­ory but also col­leges as in­sti­tu­tions, in books such as “Safe Enough Spaces” (2019) and “The Student: A Short History” (2023). Recent years have brought an in­creas­ingly po­lit­i­cal thrust to both his writ­ing (for na­tional me­dia and his pres­i­den­tial blog) and to his work as pres­i­dent. In 2023, in re­sponse to the Supreme Court’s rul­ing against af­fir­ma­tive ac­tion, Wesleyan ended legacy ad­mis­sions.When Wesleyan stu­dents joined the na­tional wave of protests over the war on Gaza, Roth—who de­scribes him­self as a sup­porter of both free speech and Israel’s right to ex­ist—tan­gled with stu­dent pro­test­ers as well as with those who wanted him to shut the protests down. Meanwhile, in in­ter­views and es­says, he took ad­min­is­tra­tors at other col­leges to task for em­brac­ing the prin­ci­ples like those found in the “Kalven re­port”—a 1967 doc­u­ment out of the University of Chicago, which ad­vanced the ar­gu­ment that uni­ver­si­ties should al­most al­ways re­main scrupu­lously neu­tral. (Such stances were, he told me, “a cover for try­ing to stay out of trou­ble.”) As the Trump Administration has ramped up its at­tacks on the acad­emy, Roth has con­tin­ued to pub­lish widely, urg­ing fel­low-lead­ers to stand up for their prin­ci­ples. “Release Mahmoud Khalil! Respect free­dom of speech!” he con­cluded in a re­cent col­umn for Slate, which ar­gued that the Columbia ac­tivist’s ar­rest “should ter­rify every col­lege pres­i­dent.”Roth and I met in his of­fice, which is dom­i­nated by a round table where he meets with both stu­dents and his cab­i­net. Wearing Blundstones and polka-dot socks, he was loose-limbed and gre­gar­i­ous, and our con­ver­sa­tion (which has been edited for length and clar­ity) was punc­tu­ated by the bright sound of bat­ting from the base­ball di­a­mond just out­side.You wrote last year, be­fore the elec­tion, that col­leges and uni­ver­si­ties weren’t ready for what was com­ing. How has the re­al­ity com­pared to your ex­pec­ta­tions?I had this idea—alas, it was in 2020, just as COVID was hap­pen­ing—that it would be great if col­leges and uni­ver­si­ties took our civic re­spon­si­bil­i­ties more se­ri­ously and re­ally in­cen­tivized stu­dents to par­tic­i­pate in the pub­lic sphere: work on a cam­paign, zon­ing com­mis­sion, what­ever. Rigorously ag­nos­tic about what they chose to work on. We found a few hun­dred schools that agreed in prin­ci­ple and we cre­ated a net­work. Before the 2024 elec­tion, we re­ac­ti­vated that group, and this time around, the in­sti­tu­tions were much less likely to want to be pub­licly in sup­port of even some­thing so non­par­ti­san.We’re re­ally small—three thou­sand stu­dents or so—and I wanted University of Texas at Austin, and Michigan, other big places. Some of them did agree in prin­ci­ple, but this time, in 2024—in the spring, let’s say, when Biden was still in the race, it was clear Trump was go­ing to be the can­di­date—the ret­i­cence of aca­d­e­mic lead­ers was al­ready ap­par­ent.Last year, we ran a pro­gram called Democracy 2024. We brought peo­ple here—nice con­fer­ence, all that stuff. And even a group of pres­i­dents that I helped put to­gether for this pur­pose, they started talk­ing more about “dialogue across dif­fer­ence” than par­tic­i­pa­tion in the elec­toral sys­tem.Every­body’s in fa­vor of not fight­ing and hav­ing bet­ter di­a­logues, and I am, too. But I’m more in fa­vor of peo­ple work­ing on cam­paigns and learn­ing about is­sues and get­ting things done. And in the last two months, it’s be­come painfully ap­par­ent that want­ing to have nice con­ver­sa­tions is not go­ing to stop peo­ple who are bent on au­thor­i­tar­i­an­ism. Right now, I’m not sure what will stop them, ex­cept suc­cess­ful court chal­lenges, and even that seems pre­car­i­ous.Watch­ing the video of this poor woman at Tufts who was ab­ducted by fed­eral agents —I wrote my blog to­day about that. I think the gov­ern­ment is spread­ing ter­ror, and that’s what they mean to do. This kid is­n’t a threat to se­cu­rity.I wrote to the pres­i­dent of Tufts—who I know, be­cause we’re in the same ath­letic con­fer­ence—and just said, “Is there any­thing you want any­one to do?” He said, “Thank you for writ­ing.” And I don’t know his busi­ness. I’m sure he’s try­ing to help the stu­dent; that’s his re­spon­si­bil­ity, and I re­spect that. But I also think every cit­i­zen, but cer­tainly every uni­ver­sity per­son, should be ex­press­ing out­rage.I’m cu­ri­ous to hear your thoughts about how we wound up here. Are there choices that uni­ver­si­ties have made that have made them more vul­ner­a­ble to at­tack?I try to think about that with­out blam­ing the vic­tim, be­cause right now the story for me is that the gov­ern­ment is abus­ing its pow­ers by mak­ing war against civil so­ci­ety. That’s the song I’ve been singing—be­cause you may not like uni­ver­si­ties, but you prob­a­bly like churches or syn­a­gogues. But I have also been think­ing about how uni­ver­si­ties can be less vul­ner­a­ble in the long run. I’ve been ar­gu­ing for al­most a decade about the in­tel­lec­tual and po­lit­i­cal in­su­lar­ity of es­pe­cially highly se­lec­tive col­leges and uni­ver­si­ties, and that we need more in­tel­lec­tual di­ver­sity at these places. I wrote a piece in the Wall Street Journal [in 2017], about af­fir­ma­tive ac­tion for con­ser­v­a­tives, which an­noyed every­one—which makes it a good op-ed, I guess.I teach a course, “The Modern and the Postmodern.” It’s not re­ally about con­ser­vatism, but I have added con­ser­v­a­tive cri­tiques of some of the mod­ernists that I talk about. I teach a course on virtue and vice in his­tory, phi­los­o­phy, and lit­er­a­ture, and I have added con­ser­v­a­tive cri­tiques of the lib­eral as­sump­tions that al­most all my stu­dents share. And it’s in­ter­est­ing to see how they re­act—they’re shocked by these cri­tiques, in ways they’re not shocked by, I don’t know, Bolshevism or vi­o­lent anti-colo­nial rev­o­lu­tion­ary rhetoric. And I point that out. So we talk, and they’re per­fectly able to deal with it. I’m not try­ing to con­vince them that these guys are right or any­thing; just that it’s in­ter­est­ing to think about.We have to be less in­su­lar, less parochial, and be­ing po­lit­i­cally more di­verse is part of that. Also, at the fancy places—like Wesleyan and Ivy League schools and oth­ers, a small per­cent­age of schools in the coun­try—I do think it would not be un­fair to say we’ve bred a kind of con­de­scen­sion. When you de­fine the qual­ity of your in­sti­tu­tion by how many peo­ple you re­ject, you can cre­ate—un­in­ten­tion­ally—an at­ti­tude of “I’ve earned my su­pe­ri­or­ity.”Trump and his al­lies have found a way to tar all of the sec­tor with the brush of the Ivy League. They’re ex­cel­lent schools, and they have ex­cel­lent sci­en­tists, and if one of Vice-President Vance’s kids is sick, he’s go­ing to want the doc­tor to have gone to one of these schools; he’s not go­ing to want them to have gone to Viktor Orbán’s uni­ver­sity. But higher ed­u­ca­tion serves so many more peo­ple in so many dif­fer­ent ways than the places that are highly se­lec­tive.What do you make of the fact that the con­flict over Israel and Palestine has be­come the pre­text for the cur­rent crack­down?I think anti-an­ti­semitism is a very use­ful tool for the right. Many oth­ers have noted how com­fort­able these same peo­ple who are crack­ing down on an­ti­semitism are with Nazis—real, fright­en­ingly con­fi­dent an­ti­semites. But it’s a use­ful tool, be­cause so many peo­ple in the lib­eral-to-pro­gres­sive, ed­u­cated coali­tion are di­vided about it, and it’s gen­er­a­tional.Anti-an­ti­semitism can be ap­pro­pri­ated by any po­lit­i­cal move­ment. They can use that as a ve­hi­cle for per­se­cut­ing re­searchers and in­sti­tu­tions that are not aligned with the ide­ol­ogy of the per­son in charge. It’s to show that you con­trol them.You have promi­nent Jewish fig­ures around the coun­try who get com­fort­able with Trump, it seems to me, be­cause they can say he’s fight­ing an­ti­semitism: “He’s good for the Jews.” It’s pa­thetic. It’s a trav­esty of Jewish val­ues, in my view.Over the last cou­ple of months, many lead­ers of col­leges and uni­ver­si­ties haven’t spo­ken out against the Trump Administration’s at­tack on higher ed­u­ca­tion. You’ve been pretty vo­cal. What do you think has made that pos­si­ble?I have, for many years, spo­ken my mind in ways that are clearly fal­li­ble. I’ve had to apol­o­gize. My com­mu­ni­ca­tions of­fice, when I said I wanted to do blog­ging, thought it was a bad idea. I think it’s im­por­tant to par­tic­i­pate. And then to say, “Oh, shit, I made a mis­take.” “Oh, I should­n’t have said that.” “Yes, I should say this.” It’s not per­fect—no con­ver­sa­tion is.I think my job as a leader of the uni­ver­sity is to speak up for the val­ues that we claim to be­lieve in, es­pe­cially when they’re at odds with peo­ple with enor­mous power. So I think I’m speak­ing now be­cause I’ve been speak­ing.My board is very sup­port­ive. My board teases me that I threaten to quit a lot. I don’t think I do, but they say I do, so they’re prob­a­bly right. In November, af­ter the elec­tion, I said, “If you want a pres­i­dent who’s not go­ing to speak up, you have to find an­other pres­i­dent.” One of my friends on the board said, “Why did you do that? You don’t have to threaten to quit. Everybody wants you to stay.” I said, “I did­n’t threaten to quit! It’s just a fact!” I’m more com­bat­ive than I want to be, and I’m not look­ing for a fight, but I do feel that when peo­ple are get­ting re­ally pushed around, in hor­ri­ble ways, that some­one who is at a uni­ver­sity and has a plat­form and can call an ed­i­tor—we should try.I ac­tu­ally thought other peo­ple would speak out. Because the mis­sions are at stake. Even the Kalven peo­ple—when the mis­sion’s at stake, you’re sup­posed to speak out.Some­one on the fac­ulty at Columbia who works in the ad­min­is­tra­tion asked me to put to­gether a group of pres­i­dents. I was un­able to. I wrote some­thing quickly for peo­ple to sign, and one of the peo­ple I con­tacted said to me, “Are you sure the pres­i­dent at Columbia wants you to?” I said, “I’m not sure.”Tell me about the kinds of con­ver­sa­tions you’ve been hav­ing with fel­low-pres­i­dents. What’s your sense of the in­ter­nal de­bates they’re hav­ing?Pres­i­dents—we’re not usu­ally hon­est with each other. It’s just the na­ture of the job. You’re al­ways try­ing to put your in­sti­tu­tion in the best pos­si­ble light. I al­ways joke that I see a pres­i­dent I haven’t seen in a few years, and he used to have two arms and now only has one —“What the hell hap­pened, Charlie?” “Oh, I hated that arm! I feel so much freer!”I don’t go to a lot of pres­i­den­tial gath­er­ings, but I go to a cou­ple. I was at one, and this guy came to me and said, “You know, you make me feel like a cow­ard.” I said, “I’m sorry, that’s not re­ally what my in­ten­tion is.” But he said he’s at a pub­lic uni­ver­sity—he was go­ing to the state leg­is­la­ture two days later. He said, “They’re not go­ing to let me have any di­ver­sity stuff in the uni­ver­sity.” I said, “Well, you can quit.” He said, “And what good would that do?” So, I’m lucky—I have a board that likes the work we do. I have an in­cred­i­ble team of vice-pres­i­dents and a won­der­ful fac­ulty, and they’re sup­port­ive. But I don’t un­der­stand. Because I know some pres­i­dents—many of them are a lot smarter than I am. I’m sure they can write well. I don’t un­der­stand.Peo­ple have said to me, “Well, you take all that money from the gov­ern­ment, why don’t you lis­ten to them?” The an­swer is, be­cause the money does­n’t come with a loy­alty oath. And that has served the coun­try so well be­cause Americans and var­i­ous gov­ern­ments we’ve had rec­og­nized that it’s bet­ter for the coun­try if peo­ple can prac­tice free­dom. The gov­ern­men­t’s not go­ing to tell you how to run your busi­ness. It starts with uni­ver­si­ties, be­cause uni­ver­si­ties are this ob­ject of re­sent­ment and this kind of weird charisma, neg­a­tive and pos­i­tive. It starts with that, but it could very eas­ily go into these other as­pects of this cul­ture that, again, de­pend on the gov­ern­ment.I don’t have to agree with the mayor to get the fire de­part­ment to come put out a fire. And that’s what they’re say­ing to these in­ter­na­tional stu­dents: “Well, you came to this coun­try. What makes you think you can write an op-ed in the news­pa­per?” Well, what makes you think that is, this is a free coun­try. As I say that, I can hear my left­ist friends: “Oh, yeah, it’s never been free.” It’s never been to­tally free, but free­doms haven’t been as threat­ened as they are now since World War One. I’m con­fi­dent of that. Universities are part of a sec­tor of our cul­ture that is worth pre­serv­ing, from the Kennedy Center to mag­a­zines to churches. Autonomy of these dif­fer­ent ar­eas, even if they’re en­tan­gled eco­nom­i­cally with the gov­ern­ment or with rich peo­ple, is so im­por­tant. I can’t be­lieve that I have to say these things out loud; it’s so ob­vi­ous.What are you do­ing right now at Wesleyan? What kinds of plans are you putting in place to pro­tect your school?We’re mak­ing sure that we are not wast­ing a dol­lar, so that we can have monies avail­able should we need them. It could be be­cause of the en­dow­ment tax—which is re­ally an ide­o­log­i­cal, puni­tive thing. That would have an im­pact on us—our fi­nan­cial-aid pro­gram is sup­ported by the en­dow­ment. We’re prepar­ing sce­nar­ios for the en­dow­ment tax, for cuts to the sci­en­tists and other things. I’ve talked to schools about cre­at­ing a le­gal-de­fense fund, but I don’t know—that’s a rel­a­tively new idea. I’ve talked to a group of fac­ulty at Yale about that.How much fed­eral fund­ing does Wesleyan get?About twenty mil­lion. A good chunk of that is stu­dent loans that are guar­an­teed, and then the rest is grants to sci­en­tists and oth­ers. Right now, we have N.I.H, N.S.F. We have grad­u­ate pro­grams in the sci­ences, so we are un­like most other lib­eral-arts col­leges.The scale of our bud­get is three hun­dred mil­lion, let’s say, an­nu­ally. So it’s real; it’s an im­por­tant part of the bud­get. People use the word “existential”—it’s not. It would change the dy­namic we have.What kinds of con­cerns are you hear­ing from stu­dents?In­ter­na­tional stu­dents are very afraid to travel. The idea that some­body would take your phone and look at all the im­ages and find an im­age they did­n’t like is very fright­en­ing. International fac­ulty, too—we have fac­ulty who are here, of course, legally, and work­ing for the uni­ver­sity ei­ther as per­ma­nent res­i­dents or on visas. Then we have a lot of fac­ulty who travel for re­search, and go to sci­en­tific meet­ings, or are do­ing work in archives around the world. And I think all of them are ner­vous about this use of the bor­der in an ide­o­log­i­cal way. As I say that out loud, I can hear many of my col­leagues on the left say­ing, “Duh! We’ve been us­ing the bor­der in an ide­o­log­i­cal way for­ever.” And that’s true—in a cer­tain way, bor­ders are a prod­uct of ide­ol­ogy. But say­ing “This is­n’t dif­fer­ent than five years ago” would be, for me, like putting your head in the sand. To use the tools of the gov­ern­ment to make peo­ple align ide­o­log­i­cally is re­ally dif­fer­ent—and, I think, would of­fend the val­ues that many con­ser­v­a­tive Americans have. I did the Charlie Sykes pod­cast, and I’m try­ing to get some other more con­ser­v­a­tive peo­ple to have me on, so I can talk about these things in ways that aren’t just for the fac­ulty at lib­eral-arts col­leges.What would you do if ICE agents showed up at Wesleyan? Do you have a plan in place?We are mak­ing sure that our stu­dents, fac­ulty, and staff know their rights, as peo­ple who live in the U.S. and are owed due process. The uni­ver­sity would ask any fed­eral agents to check in with the Office of Public Safety—that’s the cam­pus po­lice. They would need to have ju­di­cial war­rants. [Mahmoud Khalil has said that the agents who ar­rested him re­fused to pro­duce a war­rant.] We would want to make sure gov­ern­ment of­fi­cials are fol­low­ing the law. We will pro­tect peo­ple who are on our pri­vate prop­erty from peo­ple who want to con­strain their free­dom. We’d of­fer what­ever le­gal as­sis­tance we can.We’re not go­ing to ob­struct the work of legally au­tho­rized of­fi­cials—we want to make sure that they are, in fact, legally au­tho­rized.In your schol­arly work, you fo­cus on the ways peo­ple grap­ple with the past—the psy­chol­ogy of his­tory. How does that in­flect your work as pres­i­dent, es­pe­cially now?Scape­goat­ing and the cre­ation of cat­e­gories of peo­ple you can hate on and abuse is a fun­da­men­tal as­pect of hu­man so­ci­eties, and one should re­ally pay at­ten­tion to the dy­namic in which that process takes place. That def­i­nitely comes out of my work on Freud—and, in a weird way, René Girard, who’s anti-Freudian, but there’s a lot in com­mon. An ap­pre­ci­a­tion for the ways in which an­i­mos­ity can spring forth in bru­tal forms, es­pe­cially when it’s been re­pressed—that’s some­thing I’ve tried to stay aware of.I wrote a lot about Freud over the years, and for me, the most im­por­tant con­cept in Freud is the trans­fer­ence, and how some­times we treat peo­ple as if they were other peo­ple from our pasts. Famously, the an­a­lyst is trans­formed into the par­ents and other things. I think that that hap­pens a lot in my job. It hap­pens as a teacher all the time. And as pres­i­dent, oy. It’s re­ally big time. People were like, “Why don’t you end the war in Gaza?” last year—they just want some­one to be able to do the things they de­sire. They did­n’t have that re­la­tion­ship to Biden or the Secretary of State; it was me, I was in charge of the uni­ver­sity. You know, so, they give me credit for things that I don’t de­serve and they blame me for things I don’t think I de­serve the blame for, and that’s just part of the deal.I did­n’t see that be­fore. I did­n’t care so much about the pres­i­dent when I was a stu­dent. I loved my teach­ers—I mean, I had mas­sive trans­fer­ence for my teach­ers. But the pres­i­dent, these days, has more sym­bolic im­por­tance than I ex­pected. ♦

What were we all do­ing here? My 600-mile trip to hear an or­gan play a D nat­u­ralA John Cage recital that is set to last 639 years re­cently wit­nessed a chord change — 500 peo­ple made a pil­grim­age to ex­pe­ri­ence it

The crowd in the im­me­di­ate vicin­ity of the or­gan at St Burchardi church, Halberstadt, had paid €200 [DPA Picture Alliance Archive / Alamy Stock Photo]

In the year 2000, in a small east German town, work be­gan on the con­struc­tion of an or­gan that had one pur­pose: to per­form John Cage’s ORGAN2/ASLSP (1987) for pre­cisely 639 years. The late avant-garde com­poser’s only in­struc­tion for the piece was to play the piece ‘as slowly as pos­si­ble’. And so in 2001 — the in­stru­ment fi­nally ready — the world’s longest or­gan recital be­gan in St Burchardi church, Halberstadt, with a rest last­ing 17 months be­fore the first chord com­menced dron­ing in 2003. It con­sisted of two G sharps and a B. Two weeks ago, I — along with sev­eral hun­dred oth­ers — made the pil­grim­age to the town to wit­ness the work’s lat­est chord change.

In the­ory, a pipe or­gan can sound in­def­i­nitely, so long as it re­ceives ad­e­quate power and its ped­als are pressed con­tin­u­ally. To elim­i­nate the need for an or­gan­ist, a sys­tem of sand­bags sus­pended by strings de­liv­ers this pres­sure in Halberstadt.

Register to get The Spectator’s in­sight and opin­ion straight to your in­box. You can then read two free ar­ti­cles each week.