Something ter­ri­ble just hap­pened. My wife, Elise Cawley, was re­cov­er­ing from heart surgery and had just at­tended vir­tu­ally a cel­e­bra­tion for one of our chil­dren when she had a freak, vast car­dio­vas­cu­lar event—and died in­stantly. We had been to­gether for 36 years. The pic­ture above was taken just hours be­fore she died.

In all the writ­ing and pub­lic speak­ing I have done, I have cho­sen, as a mat­ter of pri­vacy, to say lit­tle about my fam­ily. But now that Elise is gone I can­not re­strain my­self from telling the world some­thing about the re­mark­able per­son with whom I shared the past 36 years of my life.

It was September 17, 1990, and I was in New York City. The event I had been at­tend­ing fin­ished a lit­tle early, so I de­cided to drop in on a friend of mine. And there, sit­ting styl­ishly on the floor, was Elise, a some­what bash­ful but ob­vi­ously bril­liant young pure math­e­mati­cian. It took me a cou­ple of weeks to call her. But from that mo­ment on we talked es­sen­tially every day for 36 years—un­til the day she died a week ago.

Somehow in the shock of what has hap­pened it feels as if I just met Elise, and now she is gone. But ac­tu­ally there were 36 won­der­ful years in be­tween, in which my life was lit up by Elise’s in­tel­lect, love and en­ergy. She was deeply the­matic, able to get, with re­mark­able ease, to the essence of things. Truth, beauty and our fam­ily and four chil­dren were her great themes. She al­ways told me that truth and beauty were ul­ti­mately the same thing, and that one day I would un­der­stand that. What I could see is that she her­self loved to cre­ate beauty—whether in math­e­mat­ics, in­te­rior de­sign, ar­chi­tec­ture or per­sonal style. She was a force­ful in­tel­lect, kind and sweet, but per­sis­tent and of­ten feisty in ar­gu­ing for what she thought was true.

Despite her deep knowl­edge and ca­pa­bil­ity in ab­stract math­e­mat­ics, she was at some level fun­da­men­tally a hu­man­ist, think­ing about the world in hu­man terms. She de­cried dry mod­ernism and was drawn to the hu­man con­nec­tions of clas­si­cism. In re­cent years she thought in­creas­ingly about the foun­da­tions of math­e­mat­ics, tak­ing a Platonic view of truth and lament­ing the twen­ti­eth-cen­tury tran­si­tion to a more for­mal­is­tic ap­proach.

There was al­ways a cer­tain lively spon­tane­ity to Elise, mixed with a drive to­wards the high­est stan­dards of in­tel­lec­tual and aes­thetic rigor. People’s first im­pres­sions of Elise tended to be of her per­sonal en­ergy, her lively style and her keen in­tel­lect. But as they got to know her, they would also come to see how much she cared about the peo­ple around her and how much lov­ing thought and ef­fort she would put into things she did for them. Whether it was a gift or the in­te­rior de­sign of a space, Elise would ap­ply her cre­ativ­ity and go back and forth un­til it was just right.

Our four chil­dren were a defin­ing part of Elise’s life, and she was im­mensely proud of all of them (as am I!). She had her own unique re­la­tion­ship with each of our chil­dren that helped them each de­velop in their own way. She also built us a won­der­ful home, full of warmth, ar­chi­tec­tural beauty and cre­ative de­sign. She took great plea­sure in per­fect­ing every de­tail, main­tain­ing re­mark­able over­all co­her­ence, with dashes of the un­ex­pected.

Elise was a very fine math­e­mati­cian, top flight in tech­ni­cal mas­tery but, more im­por­tantly, able to the­mat­i­cally pull to­gether grand con­cep­tual arcs. She was al­ways drawn to el­e­gance, of­ten with an ab­stract geo­met­ri­cal bent, and by the time I met her she had al­ready es­tab­lished her­self with re­sults that con­tinue to be cited to­day. Later in life Elise would re­fer to her­self as a “retired math­e­mati­cian”, but if ex­posed to her kind of math, she would im­me­di­ately en­gage with great en­thu­si­asm.

In all my years with Elise we never di­rectly worked to­gether on in­tel­lec­tual pro­jects. Perhaps we were both too strong willed, and with her quick mind, it did not take long for her to get im­pa­tient with the dif­fer­ences in our styles of think­ing. Still, I would of­ten tell her about work I was do­ing. And it was re­mark­able how ef­fort­lessly she would see deeper themes, which she would then ex­plain back to me, with vary­ing de­grees of pa­tience.

She liked my con­cept of com­pu­ta­tional ir­re­ducibil­ity, and I think was pleased that the ef­forts in sci­ence on which I had spent so many of our years to­gether had led to a place that high­lighted the lim­i­ta­tions of sci­ence and of sci­en­tism, and aligned well with her hu­man­is­tic world­view. When I be­gan to form ideas about ob­servers and the ru­liad, she was far ahead of me in see­ing their philo­soph­i­cal and meta­phys­i­cal im­pli­ca­tions.

For years, the foun­da­tions of math­e­mat­ics were some­thing of a sore point be­tween us (and our chil­dren were highly amused that one of the most vig­or­ous ar­gu­ments they ever saw be­tween us was about the meta­phys­i­cal char­ac­ter of the con­cept of a cir­cle). Elise main­tained that raw ax­iomatic for­mal­iza­tion can­not be the true essence of math and that math is re­ally some­thing much more hu­man. And fi­nally, just a few years ago, by a long and cir­cuitous route, I came to re­al­ize that—de­spite my ob­sti­nate in­sis­tence to the con­trary—she’d been right about this all along. But then she made other as­ser­tions too, that she thought were quite sig­nif­i­cant. Like that the fun­da­men­tal ab­strac­tions from which we build math as we know it are the con­cept of point (which she viewed as equiv­a­lent to space, con­ti­nu­ity and in­fin­ity), and the con­cept of num­ber. She also as­serted that physics dif­fers from math mainly in its in­tro­duc­tion of the con­cept of time.

While Elise could be a very orig­i­nal thinker, she also be­lieved in the value of ex­ist­ing wis­dom and ex­ist­ing ways of do­ing things—and was con­stantly telling me that I should­n’t try to fig­ure out ab­solutely every­thing (including about every­day life) from first prin­ci­ples. Being with Elise was al­ways some­thing of an ad­ven­ture. There was spon­tane­ity (“no plan; let’s play it by ear”), yet also pre­ci­sion and per­fec­tion. And there were the force­fully de­liv­ered bursts of in­sight, not just about math and ab­strac­tion, but also about the hu­man con­di­tion and the way the world works. Elise would say that there were peo­ple of ac­tion, and there were philoso­phers. She saw her­self more as the lat­ter. She had strong opin­ions about how things should be in the world. But she was much more in­ter­ested in fig­ur­ing out what should be than in what she saw as the grind of try­ing to make it ac­tu­ally hap­pen.

Elise was a per­son who had both great con­fi­dence and great hu­mil­ity. While she was al­ways con­vinced that she was right in things she thought (and in fact usu­ally was), she was never one to ad­ver­tise her ca­pa­bil­i­ties. Still, if ever a con­ver­sa­tion turned to things she knew or cared about, she would­n’t re­strain her­self from mak­ing what was of­ten a deeply in­sight­ful con­tri­bu­tion.

I al­ways thought Elise and I were an out­stand­ing match—sim­i­lar in many ways, yet also com­ple­men­tary. We both val­ued the­matic think­ing (though in many do­mains she was bet­ter at it than me), and we shared an ap­pre­ci­a­tion for aes­thet­ics and work well done. But for ex­am­ple our ways of in­ter­act­ing with peo­ple were some­what dif­fer­ent. My chil­dren say I have only one mode of in­ter­ac­tion: full on. Elise was at some level more in­tro­verted and would imag­ine her­self to be shy—though once en­gaged she would be­come highly talk­a­tive and an­i­mated. And when it was just Elise and me on our own, we al­ways seemed to have things to talk about, in fact an in­fi­nite num­ber of things.

Elise was born in 1961 in Urbana, IL—ironically not far from the head­quar­ters of our com­pany—when her fa­ther was a physics grad­u­ate stu­dent at the University of Illinois. The sec­ond of four chil­dren, she grew up mostly in Columbia, MD, as a top stu­dent, par­tic­u­larly rec­og­nized for her math and her the­matic es­says. As she of­ten men­tioned to our chil­dren, in 6th grade her teacher char­ac­ter­ized her writ­ing as “the cat’s pa­ja­mas”—and for a while she imag­ined grow­ing up to be a writer. In high school she had the for­ma­tive ex­pe­ri­ence of go­ing to the Hampshire sum­mer math pro­gram, where for the first time she met other peo­ple who were in­tensely in­ter­ested in math.

She went to MIT for col­lege, falling in with a group that in­cluded sev­eral later promi­nent math­e­mati­cians. She al­ways told me that what re­ally ig­nited her in­ter­est in con­cep­tual pure math­e­mat­ics was an epiphany she had in a class she took about topol­ogy. At MIT, she sup­ported her­self by work­ing in the the­sis bindery, us­ing her cal­li­graphic skills to la­bel the­ses. She also made up lan­guage puz­zles for a lan­guage ac­qui­si­tion study and had a sum­mer job work­ing on al­go­rithms at an early speech recog­ni­tion com­pany. A fam­ily cri­sis took her back to Maryland, where she fin­ished de­grees in both math and physics at the University of Maryland. She also had an in­tern­ship work­ing on a PET scan­ner at NIH, vol­un­teered at some hos­pi­tals and con­sid­ered go­ing to med­ical school. But in the end she de­cided on grad­u­ate school.

She could­n’t make her mind up be­tween physics and math but ended up ap­ply­ing in physics. One of her physics pro­fes­sors had said she was the best stu­dent he’d ever seen but would be most suited to very the­o­ret­i­cal physics. She had her choice be­tween Harvard and Berkeley. And in her ever-typ­i­cal fash­ion she waited un­til the last minute, pick­ing Berkeley be­cause she was told she’d be able to switch to math there.

After an in­tense “think on your feet” in­ter­view process, she re­ceived a Hertz Foundation fel­low­ship, which gen­er­ously sup­ported her grad­u­ate stud­ies. Before even ar­riv­ing at Berkeley she’d set­tled on math, and hav­ing im­pressed peo­ple with her el­e­gant so­lu­tion to a small open prob­lem, she joined a group work­ing on geo­met­ri­cal ap­proaches to dy­nam­i­cal sys­tems. After four years in grad­u­ate school she com­pleted her PhD the­sis on “Smooth Markov Partitions and Toral Automorphisms”. It was quin­tes­sen­tial Elise. An el­e­gant the­matic idea im­ple­mented with clean tech­ni­cal pro­fi­ciency.

It made her a hot­shot young math­e­mati­cian, sought af­ter by many uni­ver­si­ties. She ended up with two jobs: a tenure-track pro­fes­sor­ship at Stony Brook and an NSF post­doc spon­sored by the promi­nent math­e­mati­cian Dennis Sullivan. She chose to de­fer the pro­fes­sor­ship and take the post­doc for a year, partly at CUNY in New York City and partly at IHES near Paris.

While at IHES, Elise lived in the very cen­ter of Paris, only steps from the Jardin du Luxembourg, in an un­used stu­dio owned by an artist who was the sis­ter of an­other math­e­mati­cian. It was a pe­riod of in­tense math­e­mat­i­cal work for Elise about which she was very en­thu­si­as­tic. And her the­matic and aes­thetic sen­si­bil­i­ties drew her to the study of Teichmüller spaces, which are spaces of pos­si­ble geo­met­ri­cal ob­jects, or in a sense spaces of pos­si­ble spaces. It was also at this time that she em­barked on what would be a long-term in­ter­est in the so-called ther­mo­dy­namic for­mal­ism, or Gibbs the­ory.

With con­nec­tions to math­e­mat­i­cal physics, Gibbs the­ory pro­vides a way to talk about in­fi­nite col­lec­tions of in­fi­nite struc­tures, and their de­vel­op­ment, for ex­am­ple through time. (Ironically enough—as Elise would re­peat­edly men­tion—con­cepts such as trans­ver­sals that ap­pear in Gibbs the­ory are now di­rectly rel­e­vant to my own work on in­fra­geom­e­try and the foun­da­tions of physics.)

Rather un­typ­i­cally for a math­e­mati­cian, Elise was al­ways no­table for her dis­tinc­tive per­sonal style—of­ten an in­ter­pre­ta­tion of cur­rent fash­ion, but with her own spunky and col­or­ful touches. As one of my daugh­ters tells it, Elise’s in­ter­est in style be­gan to ex­pand with the fund­ing she re­ceived from her Hertz fel­low­ship and de­vel­oped fur­ther dur­ing her time in New York City and Paris.

Upon re­turn­ing from Paris, Elise was about to start at Stony Brook, but while wait­ing for her apart­ment to be ready, was stay­ing with her long-time friend Lisa Goldberg in New York City. And that’s how, on September 17, 1990, Elise and I first met.

Lisa’s apart­ment was an el­e­gant one, but I was not quite so el­e­gant that day. I had been at a tech in­dus­try event where I had re­ceived a rather bulky award, which was in my jacket pocket weigh­ing it down. Elise was sit­ting on the floor, with her bangs flopped across her face, in a pose that I thought might be yoga. We talked just a bit, but it was enough.

The next day I went back to Champaign, IL, where I was based at the time. It had been two and a half years since we’d re­leased Mathematica 1.0 and our com­pany was grow­ing rapidly. I had started my ca­reer young, so even though I was only two years older than Elise (she was close to 29 and I had just turned 31), I had al­ready done quite a bit, both in acad­e­mia and the tech in­dus­try.

Early ver­sions of Mathematica came with a book, that I had to write. And in September 1990 I was rush­ing to fin­ish the sec­ond edi­tion of that book, to ac­com­pany Mathematica 2.0. Two weeks af­ter I re­turned to Champaign, I did fin­ish, though promptly got sick with a high fever. And that was when I fi­nally called Elise.

Soon I was tak­ing trips to Stony Brook, and she to Champaign. And when apart we were talk­ing by phone for hours each day. Lisa Goldberg made a point of telling me, “She’s a very fine math­e­mati­cian, you know”. Elise had met an­other per­son a lit­tle ear­lier, and Lisa had dis­missed them as un­suit­able. But—as I just now learned from one of my chil­dren—when Elise asked Lisa about me she im­me­di­ately re­sponded with en­thu­si­asm.

One no­table trip was the re­sult of a call I got from Elise where she ex­plained that she’d mixed up con­tact lens so­lu­tion and hy­dro­gen per­ox­ide and now had to patch both her eyes for a few days. I im­me­di­ately got on the next flight. And while the only food I could cook at that time (and still to­day) was an omelette, she viewed this “rescue” as an act of chivalry that—as she told our chil­dren—was what de­fin­i­tively sold her on me.

On Elise’s 29th birth­day in December 1990 I sent some flow­ers. She re­sponded by email, now bit­ter­sweet:

Hi there. Some amaz­ing flow­ers just ar­rived. I think I can make to92. I’m head­ing off to aer­o­bics — tonight is the amaz­ing Margaret(who plays great mu­sic). I has some good math con­ver­sa­tion with­caro­line s. to­day. And got a lit­tle bit of real work done.

I’ll be back around 9pm.

elise

The very next email I re­ceived from her was about a bug in the Mathematica Residue func­tion.

My day job was (and still is) tech CEO. But what made me orig­i­nally de­velop our tech­nol­ogy was that I wanted to use it my­self—to do sci­ence. By the spring of 1991 I was get­ting frus­trated that I could in­ject new ideas into our com­pany faster than it could ab­sorb them. So I de­cided I should take some­thing of a sab­bat­i­cal and work on sci­ence for a while. Meanwhile, Elise still had time left on her NSF post­doc, that she could take any­where. And so it was that we de­cided to get a house to­gether in the hills above Oakland, CA, and she re­turned to Berkeley.

Before mov­ing into the house that she found for us, she came with me on an 18-city tour of Europe+Russia to pro­mote Mathematica 2.0. After that, we set­tled into a good rou­tine. She would go off each day to do math. I would stay home to be a re­mote CEO, and work on sci­ence. And from time to time she’d jet off to Texas, Montana, France or wher­ever to give talks.

By late 1992 we were think­ing about next steps. Elise was adamant that she could­n’t live in Champaign-Urbana (despite hav­ing been born there). But she thought the idea of liv­ing in the Chicago area was in­ter­est­ing. Meanwhile, she was highly in de­mand as a math­e­mati­cian, and there were mul­ti­ple op­por­tu­ni­ties for her there.

Elise took charge, and ex­plored many dif­fer­ent lo­ca­tions and houses around Chicago, even­tu­ally pick­ing a house in Hinsdale, IL. We moved there in the mid­dle of 1993, and Elise took up a po­si­tion at the University of Chicago. It was a nice house, but Elise thought it could be nicer, and set about ren­o­vat­ing it to have top-of-the-line in­te­rior de­sign. She started work­ing with a de­signer who she dis­cov­ered by read­ing mag­a­zines, and learned that her aes­thetic and geo­met­ric sen­si­bil­i­ties trans­lated well into cre­ative in­te­rior de­sign. Elise was par­tic­u­larly keen on in­te­grat­ing lively col­ors and un­ex­pected el­e­ments into tra­di­tional de­sign themes.

Meanwhile, at the University of Chicago, Elise was teach­ing a course about Gibbs the­ory. She wrote ex­ten­sive notes which she planned even­tu­ally to turn into a book. Over the years, peo­ple asked for the book many times, and Elise had re­cently started talk­ing about fi­nally fin­ish­ing it. Ironically enough, our daugh­ter Catherine, now her­self a math­e­mati­cian, has ended up work­ing in some­what re­lated ar­eas.

Elise in­ter­acted a lot with Bob Zimmer, who was then chair­man of the math de­part­ment (and would later be­come pres­i­dent of the University of Chicago). Together they de­vised the con­cept of a new ac­tiv­ity for the math de­part­ment: a pro­fes­sional mas­ter’s de­gree in fi­nan­cial math­e­mat­ics, ba­si­cally aimed at quants, and taught by math pro­fes­sors. Though Elise was never too in­ter­ested in the messi­ness of ac­tual mar­ket data, she liked the pure math­e­mat­ics of mar­ket mod­els, and helped de­velop a cur­ricu­lum around it. The mas­ter’s de­gree ended up be­ing highly suc­cess­ful and con­tin­ues to this day.

In the fall of 1994 Elise and I mar­ried. And then, in December 1995, as Elise would later de­scribe it, “the world overnight had a new cen­ter”: our first child, Alexander, was born. The next few years were dom­i­nated by the ar­rival of chil­dren, with Catherine be­ing born in 1997 and Christopher in 1998—leaving Elise, as she put it, with “three un­der three”.

With our fam­ily grow­ing—and our book col­lec­tion reach­ing over 5000 vol­umes—we were out­grow­ing our house in Hinsdale, and started look­ing for a new home. At first, Elise found real es­tate in Lake Forest, IL. But when the deal for it fell through we started think­ing more broadly and de­cided that since the Boston area was the place where to­gether we knew the most peo­ple, that was where we should go. But which town in par­tic­u­lar? Elise’s #1 ini­tial cri­te­rion was that it should al­ready have a Starbucks. From the list of such towns, she then started look­ing at USGS aer­ial pho­tographs (or what would now be satel­lite im­ages), and soon set­tled on Concord, MA. But find­ing no suit­able ex­ist­ing house to buy there, she set about de­sign­ing a house to build.

It was a com­plex process. The house had to ac­com­mo­date what was by then a li­brary of nearly 10,000 books, as well as space for my re­mote-CEO of­fice and for our three chil­dren. Elise had been scour­ing ar­chi­tec­ture books, and in­ter­viewed sev­eral top-of-the-line ar­chi­tects be­fore set­tling on one to work with. With her math­e­mat­i­cal and aes­thetic sen­si­bil­i­ties she eas­ily ab­sorbed prin­ci­ples of ar­chi­tec­ture and soon de­vel­oped her own in­tu­ition for it. Sometimes she’d bring in se­ri­ous math—like num­ber the­ory to de­sign a stair­case with aligned squig­gly ban­is­ters, or cir­cle pack­ing to lay out an Italian-inspired tiling based on cir­cles and Sierpinski pat­terns. She started off mak­ing sketches, but soon learned the lat­est CAD tools. And in the end her ideas de­fined much of the de­sign of the house. (At the be­gin­ning, I had tried mak­ing some “suggestions” about the plans, but quickly re­al­ized that this was an Elise pro­ject, and I should stay out of it.)

It took sev­eral years to build the house (with Elise reg­u­larly trav­el­ing to Boston to over­see the pro­ject) but in the spring of 2002 the pro­ject was fin­ished—co­in­ci­den­tally at ba­si­cally the same time as my 10-year pro­ject of writ­ing A New Kind of Science. Other than in CAD draw­ings I’d only seen the house once, early in its con­struc­tion, so when we drove from Hinsdale and ar­rived at the house I was in for a sur­prise. It was much big­ger than I ex­pected! And when we went in­side I could see that it was a mas­ter­piece, filled with per­son­al­ity: beau­ti­ful, el­e­gant, cre­ative—quin­tes­sen­tial Elise.

And then there was the in­te­rior de­sign. Bold and col­or­ful, yet clas­sic. Each room with a dis­tinct char­ac­ter, grace­fully flow­ing into the next. Over the years, Elise re­vised and up­dated the in­te­rior de­sign of the house many times. She was of­ten to be seen sketch­ing on graph pa­per new pos­si­ble de­signs based on new ideas. She kept the style fresh, con­stantly breath­ing new life into the house—even if it meant that our chil­dren would com­ment that every time they showed up the fur­ni­ture seemed to have been re­arranged. And 24 years af­ter I first saw the house, I still find my­self wowed by it.

By 2003, our old­est was en­cour­ag­ing us to have an­other child, not­ing, among other things, that the sym­me­try of the house had given it four chil­dren’s bed­rooms. And so it was that in the fall of 2004 our fourth child, Elizabeth, was born.

In her ear­lier years, Elise had al­ways said that she “couldn’t imag­ine her life not do­ing math”. But partly through me, and, more im­por­tantly through our chil­dren, her world had been broad­ened, and her per­spec­tives had shifted. She would later de­scribe her life as hav­ing sev­eral chap­ters, and this was a chap­ter about chil­dren.

Through her chil­dren show­ing in­ter­est in math, she ended up coach­ing some mid­dle-school MathCounts math com­pe­ti­tion teams. She was­n’t a fan of the way math was typ­i­cally taught in schools, com­plain­ing that the beauty of math was be­ing lost in the at­tempt to make math seem “applicable”, and not­ing that “most of math in mod­ern text­books is not re­ally math at all”. In an email to me in 2010 Elise spirit­edly went fur­ther:

What has hap­pened to beauty in math? The beauty de­rives­from the sim­plic­ity and el­e­gance. That has been lost in the­p­ro­lif­er­a­tion of ex­tra­ne­ous ter­mi­nol­ogy, meant to make things “moreconceptual”, from early end­less ex­cur­sions with “place value” to ab­sur­d­vari­a­tions on al­go­rithms, to gummy de­scrip­tions of every­thing us­ing­con­tor­tions of lan­guage, rather than the lan­guage of math­e­mat­ics it­self.

Elise did nev­er­the­less do some ap­pli­ca­tions of math her­self. During the pan­demic, for ex­am­ple, she de­vel­oped the idea that in­fec­tion was not a bi­nary phe­nom­e­non, but that in­stead each per­son had a con­stantly chang­ing vi­ral load and im­mu­nity, cre­at­ing a “viral field” at the pop­u­la­tion level. Elise was al­ways up to date on world af­fairs, and had a deep in­ter­est in un­der­stand­ing their the­matic arcs at work.

Elise loved the small-town at­mos­phere of Concord, MA, and en­joyed the feel­ing of be­ing in a place “where every­one knows your name”. She was a reg­u­lar at lo­cal cafes and ex­er­cise classes, bring­ing her live­li­ness wher­ever she went. She had a cir­cle of friends with many dif­fer­ent back­grounds, of­ten met through their chil­dren or sim­ply through re­peat­edly run­ning into them at Starbucks. She took great plea­sure in ap­ply­ing her knowl­edge and think­ing to help her friends in all sorts of ways. Whether it was do­ing re­search on med­ical is­sues, solv­ing in­te­rior de­sign prob­lems, or sim­ply us­ing her keen in­sight and judg­ment to pro­vide life ad­vice, she would re­li­ably go above and be­yond.

Our fam­ily had a def­i­nite rou­tine, with din­ner each day at about 6pm. Every Friday night—fol­low­ing a tra­di­tion from Elise’s own fam­ily grow­ing up—we would go out to a movie (and, yes, that meant we saw many ques­tion­able movies). Before our chil­dren were old enough, it was just Elise and me; later it in­cluded var­i­ous con­tin­gents of chil­dren too. Elise and I would talk af­ter each movie, with her of­ten be­ing quite ap­palled by my lack of un­der­stand­ing of what seemed to her like the movie’s ob­vi­ous the­matic el­e­ments. (Elise also gave me a hard time about my fail­ure to read fic­tion, claim­ing that many ideas, par­tic­u­larly about hu­man na­ture, are best com­mu­ni­cated in fic­tion.)

A dozen years ago two of our chil­dren were at the Stanford Online High School, and Elise was re­cruited as an in­au­gural mem­ber of the school’s Community Advisory Group. Originally this was just sup­posed to be for a year. But a decade later she was still there, a highly val­ued mem­ber of the group. She spent con­sid­er­able ef­fort think­ing about the over­all strat­egy and is­sues of the or­ga­ni­za­tion, and de­vel­oped many ideas and opin­ions. Despite this, she’d of­ten tell us that she did­n’t think she’d end up say­ing any­thing at a given meet­ing. But with great con­sis­tency it would later tran­spire that ac­tu­ally she had been the most talk­a­tive and en­gaged per­son there.

In the early 2000s, the Hertz Foundation, from which Elise had re­ceived her grad­u­ate school fel­low­ship, started reach­ing out to her, and she en­joyed en­gag­ing with the emerg­ing Hertz com­mu­nity, and meet­ing fel­lows from a wide range of fields. In 2018 she was also re­cruited to the board of the Hertz Foundation. And here again she be­came a key con­trib­u­tor, of­ten ex­pect­ing not to say any­thing, but then end­ing up as one of the most vo­cal peo­ple there.

By 2023, our three older chil­dren had all grad­u­ated from col­lege, but our youngest, Elizabeth, was just start­ing at Yale. Elizabeth has al­ways been in­volved in a great many ac­tiv­i­ties, and Elise took great plea­sure in at­tend­ing every pos­si­ble con­cert, sport­ing or other event—and also for ex­am­ple host­ing Elizabeth’s en­tire a cap­pella group for a re­treat at our house.

A bit more than a decade ago Elise also dis­cov­ered a love for the Southwest and for hik­ing. And with her brother Jason—with whom she was al­ways very close—liv­ing in the Phoenix, AZ, area, we started spend­ing time there each win­ter.

With Elizabeth off in col­lege we were fi­nally empty nesters, and Elise was ready for a new chap­ter in her life. She con­sid­ered re­turn­ing to where she’d left off in Gibbs the­ory 30 years ago, be­ing told by an old col­league that her work there was still very cur­rent. She also con­sid­ered get­ting more deeply in­volved in the phi­los­o­phy of math­e­mat­ics and was on the brink of writ­ing a piece about it. She was also quite en­gaged in think­ing about the pur­pose and di­rec­tion of higher ed­u­ca­tion, and the im­por­tance of free ex­pres­sion in the search for truth.

In some ways Elise and I were be­com­ing “the old mar­ried cou­ple”, with her ad­mon­ish­ing me about my messi­ness in eat­ing ice cream cones or her de­sire that I switch on self-dri­ving to avoid her be­ing sub­jected to my own sub­stan­dard hu­man dri­ving.

Elise had al­ways been a fit and healthy per­son, and was also very med­ically knowl­edge­able. So when she started notic­ing jaw pain when hik­ing in Colorado and Arizona, she was con­cerned. Tests at first showed noth­ing alarm­ing, though, as she al­ready knew, she had a small con­gen­i­tal heart is­sue that was grad­u­ally wors­en­ing. But just a few weeks ago her symp­toms wors­ened and more in­va­sive tests re­vealed that she needed ur­gent open heart surgery. She had an out­stand­ing team of top doc­tors, the com­plex surgery went very well, and she was soon con­tin­u­ing her re­cov­ery from home.

Meanwhile there was a long-planned cel­e­bra­tion for one of our chil­dren that Elise hoped to at­tend. But though it was not pos­si­ble for her to be there in per­son, we arranged for her to at­tend vir­tu­ally. It was a won­der­ful event, and Elise could be seen via video call beam­ing through all of it. But then, soon af­ter, dis­as­ter struck, and she col­lapsed, the vic­tim of a mas­sive car­dio­vas­cu­lar event. My only so­lace in this tragedy is that the end came in­stantly, af­ter a day filled with noth­ing but joy.

There is so much that Elise was look­ing for­ward to. The im­mi­nent ar­rival of our first grand­child. Elizabeth’s grad­u­a­tion from col­lege. The next chap­ter of her life. But Elise will not be here for any of these.

Elise had no idea the end was near. And even though she ex­pected more, she had al­ready had many chap­ters in her life, and many achieve­ments of which she was proud. Elise had a beau­ti­ful life, full of things she val­ued greatly, es­pe­cially her chil­dren. That she is gone is a tragedy for us all; she will be deeply missed.

Thank you, Elise, for all those won­der­ful years.

If you’re just look­ing for the re­sults, be­low is a cus­tom color picker based on the color space writ­ten in Javascript and a sam­ple pro­ce­dural gen­er­a­tion al­go­rithm in Python (Javascript equiv­a­lents are in the page source) - feel free to take this math and go have fun de­pict­ing our di­verse world!

The goal of this pro­ject was to de­fine a color space that makes it eas­ier to build in­clu­sive color tools for a va­ri­ety of con­texts - such as char­ac­ter cre­ators or dig­i­tal art. If you see some­thing here that sparks your cu­rios­ity, I would love for you to stick around and read be­low this sec­tion to learn more!

R² =

(What’s R²?)

│

Show Sphere

What is each di­rec­tion on the picker ad­just­ing? Jump to that ex­pla­na­tion here.

# Plug the out­put of one of the se­lec­t_­point im­ple­men­ta­tions into to_rgb(t, u, v)

def se­lec­t_­point(r_square: float = 2.) -> tu­ple[float, float, float]: “”″Uniformly sam­ple from the sphere de­ter­min­is­ti­cally”“” ra­dius = r_square ** (1. / 2)

phi = uni­form(0, 2 * math.pi) cos­theta = uni­form(-1, 1) n = uni­form(0, 1)

theta = math.acos(cos­theta) r = ra­dius * (n ** (1.0 / 3))

t = r * math.sin(theta) * math.cos(phi) u = r * math.sin(theta) * math.sin(phi) v = r * math.cos(theta)

re­turn (t, u, v)

def se­lec­t_­point(r_square: float = 2.) -> tu­ple[float, float, float]: “”″Uniformly sam­ple from the sphere us­ing re­jec­tion sam­pling”“” ra­dius = r_square ** (1. / 2) R = ra­dius + 1

while R > ra­dius: t = uni­form(-ra­dius, ra­dius) u = uni­form(-ra­dius, ra­dius) v = uni­form(-ra­dius, ra­dius)

R = (t**2 + u**2 + v**2) ** (1.0 / 2)

re­turn (t, u, v)

def to_rgb(t, u, v) -> tu­ple[int, int, int]: x = (t - 0.15) / 0.45 y = (v - 1.2 * t ** 2 + 0.2 * t + 0.655) / 1.84 z = u / 3.6

r = 28.77438370854 * x + 36.78307445559 * y - 19.69766918644 * z + 187.1436241611 g = 35.38327306318 * x - 2.009931981182 * y + 47.93462563172 * z + 137.1073825503 b = 36.14733717939 * x - 43.54346996173 * y - 28.50821294135 * z + 108.2241610738

re­turn int(r), int(g), int(b)

# Overview

What col­ors are we? The short an­swer is maybe some­thing like “brown” and the long an­swer is very, very long. Representing the broad range of hu­man skin tones dig­i­tally is a hard prob­lem. Often, a lim­ited set of col­ors is pre­sented as be­ing good enough to cover the full spec­trum of di­ver­sity. However, in us­ing a spe­cific set of col­ors, large groups of peo­ple are un­able to ac­cu­rately be rep­re­sented, or might be un­in­ten­tion­ally ex­cluded.

The goal of this work is to iden­tify the broad­est in­clu­sive range of col­ors in the RGB color space that cor­re­spond to plau­si­ble, but sim­pli­fied skin tones. In par­tic­u­lar, the aim was to iden­tify sim­ple, “good enough” equa­tions which de­fine that area, al­low­ing the range to be used in a va­ri­ety of con­texts. Calling the equa­tions “good enough” is in­tended to keep the lim­i­ta­tions of this work at the fore­front - the re­sults are a use­ful start­ing point, but should not be taken to be au­thor­i­ta­tive.

# Introduction

Although there have been im­prove­ments in the set of col­ors that are pre­sented as rep­re­sen­ta­tive of us, there’s still a gap that needs to be closed. Emojis say we’re 5 shades (or car­toon-yel­low); a makeup brand might say 50; and a char­ac­ter cre­ator might shrug and tell you to pick from all 16,777,216 op­tions. If you look out­side - or just at your­self - you’ll quickly no­tice that none of those can com­pare to the va­ri­ety of re­al­ity; one per­son is not just one color. Despite that, it can be use­ful to try to boil things down to fewer val­ues. The Unicode Consortium and makeup com­pa­nies can fig­ure out their own ranges, but I be­lieve we can find a bet­ter so­lu­tion that’s some­where be­tween “several” and “several mil­lion.”

Taking the dig­i­tal art world as an ex­am­ple, im­ages such as the one be­low are of­ten cir­cu­lated in an at­tempt to as­sist other artists in iden­ti­fy­ing plau­si­ble col­ors.

In the video game world, nowa­days the pre­set col­ors are of­ten wide rang­ing and sup­ple­mented with a gen­eral color picker, but it would be even bet­ter if the ini­tial ex­pe­ri­ence pre­sented bet­ter op­tions.

# Limitations

Taking a step back to re­al­ity, this work has a num­ber of in­her­ent lim­i­ta­tions.

As men­tioned be­fore, skin tones are much more com­pli­cated than a sin­gle color. They vary widely be­tween dif­fer­ent ar­eas of the body and are sub­ject to com­plex bi­o­log­i­cal processes. The per­ceived color of the skin is af­fected by blood flow, con­cen­tra­tions of melanin, com­plex scat­ter­ing of light through the lay­ers of the skin, as well as things like vi­tiligo, freck­les, hy­per­pig­men­ta­tion, scar­ring, and other com­mon vari­a­tions.

Secondly, a va­ri­ety of health con­di­tions can cause peo­ple to have skin tones that are well out­side what might be per­ceived as plau­si­ble. Argyria can of­ten lead to skin that is blue-gray in color; high biliru­bin can cause skin to be yel­low­ish or green­ish.

Additionally, it’s im­por­tant to note that I am one per­son, not a re­searcher, and sub­ject to my own gen­eral bi­ases on top of my own per­cep­tion of color. As far as I am aware I don’t have color vi­sion de­fi­ciency, but a lot of the choices I made are com­pletely sub­jec­tive.

Finally, col­ors are not per­ceived con­sis­tently across dis­play types and view­ing en­vi­ron­ments. RGB val­ues look dif­fer­ent be­tween dif­fer­ent screens, and peo­ple look com­pletely dif­fer­ent un­der dif­fer­ent light­ing con­di­tions.

Broadening this work to ad­dress some of these lim­i­ta­tions would be an in­ter­est­ing area of fur­ther in­ves­ti­ga­tion - a goal of this pro­ject was to be good enough for sim­ple use cases, but these lim­i­ta­tions might be more prob­lem­atic in other con­texts. The re­sults here will mainly be ap­plic­a­ble in con­texts that re­late to gen­er­at­ing sim­pli­fied rep­re­sen­ta­tions of peo­ple.

# Methodology

What are all of those num­bers, and how did you get them?

Content warn­ing - un­sci­en­tific method­ol­ogy be­low. In other words: good enough is fine if you’re an en­gi­neer.

Summary:

Manually la­bel col­ors in RGB in or­der to get a rough ap­prox­i­ma­tion of the dataset shape

Perform a prin­ci­pal com­po­nent analy­sis (PCA) with N=3 on the dataset to change the shape into some­thing eas­ier to work with

Use your pre­ferred graph­ing soft­ware to man­u­ally cre­ate equa­tions that map a sphere in the tar­get space onto the trans­formed data in the XYZ, or PCA space

# Manually Labeling Colors

Sometimes the best place to start is by do­ing a ton of te­dious work. I did­n’t la­bel every color in RGB, but there cer­tainly were a lot. This is the UI I built for la­bel­ing - just a sim­ple web­page where you click to move a face be­tween the yes and no side. Initially they were just squares, but I ended up draw­ing a face and slap­ping the col­ors onto it. Gazing into my lumpy re­search as­sis­tan­t’s eyes made it much eas­ier to quickly go “yeah I can imag­ine that per­son walk­ing around”.

Graphing that data, you get the fol­low­ing shape - for vi­su­al­iza­tion it’s kind of like a ba­nana shape that swoops be­tween 0, 0, 0 and 255, 255, 255. The curve is more to­wards red and away from blue.

This is where I got stuck for the longest. For your sake, I’m go­ing to skip over all of my dead ends and strug­gles along the way. There were many. Far too many. Things in­volv­ing con­vex hulls, re­gres­sion mis­ad­ven­tures, 4th de­gree poly­no­mi­als, and the worst look­ing code I’ve ever writ­ten.

The re­sult of all of that was that I de­cided I needed some way to trans­form the shape into some­thing eas­ier to work with: that’s when I learned about prin­ci­pal com­po­nent analy­sis and was im­me­di­ately in­spired.

# Humanities Intermission

Although all of the math and tech­nol­ogy here is fun, it’s im­por­tant to ad­dress the fact that tech­nol­ogy ex­ists in a so­cial con­text. Stating things plainly: lighter skin col­ors have both presently and his­tor­i­cally been cel­e­brated and pri­or­i­tized while darker skin col­ors have been mar­gin­al­ized and ma­ligned. Racism and col­orism are sys­tem­i­cally pre­sent in many cul­tures, in both overt and sub­tle ways.

Below are some great videos, es­says and pro­jects that ad­dress these is­sues through a va­ri­ety of lenses. I highly rec­om­mend tak­ing a look, es­pe­cially if you want a break be­fore we get into the math.

In her video se­ries The Darkest Shade, Nyma Tang re­views the dark­est shades from a va­ri­ety of makeup brands. In her own words, “It’s im­por­tant for makeup brands to make prod­ucts for all shades and as some­one with a darker skin tone, I want to be able to help oth­ers who strug­gle to find the same!”

In a sim­i­lar vein, Kat Blaque’s video (for in­tro­spec­tive hot peo­ple) Youthforia’s Blackface Foundation is about a makeup brand that re­leased a dark shade that was lit­er­ally black - far darker than what would be plau­si­bly use­ful.

Finally in the makeup space, the Vox video How beauty brands failed women of color talks about the his­tory of dis­crim­i­na­tion in the beauty in­dus­try and the lim­ited avail­abil­ity of deeper shades. There’s a lot of great ex­pert in­ter­views as well as a dis­cus­sion of the in­ter­sec­tion with Black his­tory.

Taking a look at video games, Me, On The Screen: Race in Animal Crossing: New Leaf by Austin Walker is an ex­cel­lent es­say about the au­thor’s strug­gle to see him­self rep­re­sented in video games. In the years since, Animal Crossing has got­ten much bet­ter about in­clu­siv­ity but the es­say also goes into his his­tory of not see­ing him­self - or see­ing char­ac­ters that look like him be stuck in racist tropes.

The Humanae pho­tog­ra­phy pro­ject by Angélica Dass is “an un­usu­ally di­rect re­flec­tion on the color of the skin, at­tempt­ing to doc­u­ment hu­man­i­ty’s true col­ors rather than the un­true la­bels “white”, “red”, “black” and “yellow” as­so­ci­ated with race.” In a way it’s re­ally sim­i­lar to the work here - ex­cept with a fo­cus on doc­u­ment­ing and con­vers­ing in­stead of de­scrib­ing, and us­ing Pantone shades over hex codes.

For more di­rect re­sources, the video It’s not a Coincidence. It’s Colorism. by Tee Noir gives a de­scrip­tion of what col­orism is, and a few ways it man­i­fests in mod­ern pop cul­ture.

Finally, Writing With Color is an ex­cel­lent blog with many re­sources about writ­ing var­i­ous kinds of di­ver­sity. The blog is run by a team with ex­per­tise in many dif­fer­ent ar­eas, and they of­ten take reader ques­tions. In par­tic­u­lar I found their post Words for Skin Tone | How to Describe Skin Color re­ally in­ter­est­ing. As more of a reader than a writer, it’s fas­ci­nat­ing to see be­hind the scenes on how to make bet­ter word choices.

# Principal Component Analysis

Briefly, prin­ci­pal com­po­nent analy­sis (PCA) is a way to ro­tate and stretch a dataset so that the main di­rec­tions of vari­a­tion lie along the axes of the co­or­di­nate sys­tem. To make that more con­crete - re­call the ba­nana shape of my dataset from be­fore. In its orig­i­nal form, the ba­nana was sort of float­ing in space. PCA took that shape and placed in on the ground so that it was sym­met­ri­cal about the axes and aligned nicely. It also did some slight stretch­ing and rescal­ing, but that’s less im­por­tant.

Aside from the mod­i­fied dataset, an­other out­put of the analy­sis is a ma­trix that can be used to take a point from RGB space and trans­late it into the rest­ing-on-the-floor space. I re­fer to this as PCA space, or XYZ space since those are the vari­ables I use for it.

# Manual Function Fitting

To sum­ma­rize what we’ve done so far: we started with data in RGB space (the man­u­ally la­beled dataset). PCA gives us a way to take points in that space and trans­form them into an­other space - XYZ space.

Now the goal is to fit some func­tion to all of these points. Because it’s a dense shape, if we can de­fine the sur­face of it with an equa­tion, we can mod­ify that equa­tion to in­clude the points on the in­side as well by switch­ing the equa­tion to be­ing an in­equal­ity (eg from x² + y² + z² = 2 to x² + y² + z² ≤ 2).

The process for fit­ting the func­tion is as fol­lows:

Start with a func­tion for a sphere: R² = t² + u² + v² (see Adjusting R² for why a sphere is used)

Define t, u, and v in terms of x, y, and z

Modify how t, u, and v are de­fined un­til the sphere stretches to match up with the ex­pected data

Invert the re­la­tion­ships in or­der to de­fine x, y, and z in terms of t, u, and v

If you have (t = x, u = y, v = z) as a start­ing point, you can start to play with those re­la­tions un­til you start to get an in­tu­ition for how changes to the equa­tions will end up shift­ing your sphere around.

So I did that! A lot. To put it plainly - I man­u­ally fit func­tions to my tar­get cloud of points. There is no re­gres­sion here, I lit­er­ally did guess and check and eye­balled the func­tion fit. I did this in Desmos 3D - if you haven’t used Desmos in a while it’s amaz­ing how good it was for this.

The func­tions that re­late (t, u, v) to (x, y, z) can be used to trans­late a point be­tween those two spaces. Principal com­po­nent analy­sis gave us a way to move be­tween RGB and XYZ, and then these new equa­tions give us a way to move be­tween XYZ and TUV. Linking up all the trans­for­ma­tions gives us the func­tions that take us be­tween TUV to RGB.

Am I en­dors­ing this method­ol­ogy? No. But also kind of. In this spe­cific sit­u­a­tion it’s what worked to get it done and get func­tional equa­tions. It would be great to see some­one do this prop­erly, with nice clean sym­bolic re­gres­sion and re­ally good train­ing data. But I fig­ured since the train­ing data I la­beled was mid­dling qual­ity at best, I’d prob­a­bly have a bet­ter time us­ing my guess and check and then eye­balling the re­sults.

And all that be­ing said, I think I would en­dorse the over­all method I used, if not the specifics. That be­ing “Label Data” → “PCA” → “Fit A Spherical Equation”.

# Results

From the equa­tions, the sam­ple code at the top of the page is just a short trans­la­tion away. The next sec­tion is the de­scrip­tion of the com­po­nents of the Picker UI, which also serves as an ex­pla­na­tion of what TUV space ac­tu­ally looks like, and what prop­er­ties it has.

# Picker UI

Just like how a typ­i­cal color picker al­lows you to ad­just red, green and blue val­ues (or hue, sat­u­ra­tion and value), this color picker also has 3 in­de­pen­dent val­ues to ad­just - re­ferred to in the code sam­ples as T, U and V.

What’s re­ally in­ter­est­ing about these con­trols is that the con­cept each is ad­just­ing is com­pletely an out­put of the prin­ci­pal com­po­nent analy­sis. Or in other words, I did­n’t know that these would be what the di­rec­tions con­trolled un­til af­ter I had made the picker. It’s a re­ally neat demon­stra­tion of what prin­ci­pal com­po­nent analy­sis does - it ro­tates data to max­i­mize how mean­ing­ful each axis is.

Now, if you’re ex­pe­ri­enced with color spaces you might be think­ing that this looks a lot like HSL/HSV. However, when I was do­ing my first ex­per­i­ments with the la­beled data those color spaces did­n’t quite cap­ture what I was look­ing for. I think in par­tic­u­lar that al­though T and U seem sim­i­lar to Value and Hue, they don’t com­pletely match up. And the biggest dif­fer­ence is be­tween V and Saturation. V ranges from blue to or­ange, while Saturation would look more like gray to or­ange; al­though at some (T, U) val­ues V might look a lot like Saturation, over­all it does­n’t quite cor­re­late.

# Adjusting R²

What is R²? R² is the ra­dius of a sphere in the skin tone color space. Because one of the steps of trans­la­tion from RGB to TUV in­volves a sphere, it makes spheres a nat­ural shape to work with in TUV space. A lot of the cool prop­er­ties that arise from that fact are just a side ef­fect of spheres hav­ing a lot of use­ful prop­er­ties.

To start with a vi­sual, be­low is a table of R² val­ues, and the re­sult of us­ing a sphere with that value:

So what’s go­ing on with that table? Because a sphere is de­fined by a ra­dius, you can de­fine use­ful sam­pling ranges us­ing only a sin­gle value and then eas­ily se­lect points from within that sphere. For ex­am­ple, in the sam­ple im­ple­men­ta­tion of the tone gen­er­a­tion func­tion I use a ra­dius of 2. This is the ra­dius that was used when de­vel­op­ing the color space and gen­er­ally will re­sult in a broad range of tones. However, if you find there are im­plau­si­ble re­sults gen­er­ated us­ing that ra­dius you can de­crease it by a bit in or­der to re­duce the vari­a­tion in gen­er­ated col­ors.

The fact that the sphere shrinks along a ra­dius means that the re­duc­tion in vari­a­tion ac­tu­ally does not re­sult in a large re­duc­tion in rep­re­sen­ta­tive­ness. In other words, re­duc­ing the ra­dius does­n’t lead to just chop­ping off deep or fair skinned col­ors - deep skin tones, fair skin tones, flush skin tones, ochre skin tones, cool skin tones and warm skin tones are uni­formly de­creased in vari­a­tion.

Taking R² all the way down to 0, we can iden­tify the ori­gin of the color space. If the space is well formed then this ori­gin point should be a very neu­tral am­bigu­ous tone. If the ori­gin seems bi­ased in a par­tic­u­lar di­rec­tion, then that would in­di­cate that the space needs more fine tun­ing. Essentially, the ori­gin point can be used to iden­tify bias in the space map­ping.

At the other ex­treme, you can in­crease R² in or­der to al­low for more vari­a­tion. As an ex­am­ple, I do this in the color picker UI. In a picker UI the user can ig­nore im­plau­si­ble col­ors, so there’s lit­tle is­sue with in­creas­ing the vari­a­tion too far. In a color gen­er­a­tion con­text you’d likely want to avoid hav­ing im­plau­si­ble col­ors pop­ping up. There is likely not an ideal R² value that works in all con­texts, but hav­ing a sin­gle pa­ra­me­ter to tweak makes it easy to ad­just the equa­tions to your needs.

# In Summary

We have a color space, we have a picker for it, we have a pro­ce­dural gen­er­a­tion al­go­rithm, and we have a method­ol­ogy for cre­at­ing new spaces or mod­i­fy­ing the ex­ist­ing one. Overall, I’m thrilled with this re­sult. It’s cer­tainly not per­fect, but I don’t think a sim­ple and com­pletely cor­rect so­lu­tion ex­ists. I’ve al­ready used the picker in a dig­i­tal paint­ing and it was ex­actly as I was hop­ing it’d be - and I’m look­ing for­ward to us­ing the gen­er­a­tor in my next pro­jects.

# What’s Next?

First is feed­back: if you at all found this in­ter­est­ing or use­ful, please let me know! Although I did this work for my own use, hear­ing from oth­ers is also so re­ward­ing. Also please reach out with any ques­tions or feed­back - es­pe­cially if you spot some­thing I did­n’t think about.

Although I would love to post an email ad­dress for ac­ces­si­bil­ity, I think the safest so­lu­tion is to use the is­sues page of the repo for this pro­ject to con­tact me. But open to sug­ges­tions on al­ter­na­tives for that as well!

A task for me is to clean up my code and post it to the repo for this page to make it easy for peo­ple to recre­ate and it­er­ate on this work.

# Future Work

# Refining The Space

In defin­ing the color space so pre­cisely and us­ing spe­cific R² val­ues, there is a risk that some col­ors have been ex­cluded or mar­gin­al­ized. However, that same con­crete­ness also al­lows for ex­tremely pre­cise cri­tiques of the color space, and there­fore ex­tremely pre­cise im­prove­ments. As men­tioned be­fore, my process in­volved a lot of sub­jec­tive eye­balling - a great it­er­a­tion on that would be to have mul­ti­ple peo­ple, per­haps ex­perts, la­bel­ing data and then mea­sur­ing the out­put more pre­cisely against that.

# Skin Variations and Conditions

A di­rec­tion men­tioned ear­lier would be to look into mod­el­ing sim­pli­fied ver­sions of var­i­ous con­di­tions. What mod­i­fi­ca­tion might you be able to ap­ply to base tones to sim­u­late that per­son hav­ing jaun­dice, or ar­gyria, or be­com­ing pal­lid? What col­ors are freck­les, vi­tiligo, scars, hy­per­pig­men­ta­tion, or stretch marks rel­a­tive to a per­son’s base tone?

# Technical Improvements

From a tech­ni­cal di­rec­tion, mak­ing the equa­tion gen­er­a­tion more for­mal­ized is an­other po­ten­tial di­rec­tion to go. Starting with the shape of equa­tions I man­u­ally de­ter­mined, sym­bolic re­gres­sion on those against bet­ter train­ing data might lead to an even bet­ter color space de­f­i­n­i­tion.

There’s also the po­ten­tial for op­ti­miz­ing the equa­tions for dif­fer­ent con­texts. I’ve cho­sen to write them out in code as they are to be the most read­able to a broad au­di­ence. However, keen math­e­mati­cians might no­tice that one of those steps looks a lot like a ma­trix mul­ti­pli­ca­tion, which can be rewrit­ten to be more ef­fi­cient in cer­tain con­texts.

It might also be in­ter­est­ing to see what the re­sults look like if you trans­form the data into an­other color space be­fore do­ing the prin­ci­pal com­po­nent analy­sis and equa­tion fit­ting. There’s a chance the equa­tions end up be­ing sim­pler or more rep­re­sen­ta­tive. Because of the sphere, the shape of the sur­face in the color space you build from will be main­tained and it will mainly scale with R² val­ues. For an il­lus­tra­tion of this idea, see the graph script in the re­po’s code.

Thinking

Summary

Shieldstral in­tro­duces a 3B open-weights mul­ti­modal safety clas­si­fier that out­per­forms mod­els up to 7x its size by fram­ing con­tent mod­er­a­tion as a pol­icy-adap­tive ques­tion-an­swer­ing task. Unlike tra­di­tional guardrail mod­els, it ac­cepts plain-lan­guage poli­cies at in­fer­ence time, uni­fy­ing text and im­age safety eval­u­a­tion with­out re­train­ing. Released un­der Apache 2.0, it de­liv­ers cal­i­brated safety scores across di­verse bench­marks while run­ning ef­fi­ciently on a sin­gle 16GB NVIDIA GPU.

A 3B open-weights, pol­icy-adap­tive mul­ti­modal safety clas­si­fier that matches mod­els up to 7x its size on text safety and sets a new state of the art on mul­ti­modal mod­er­a­tion.

“Does this con­tent pro­mote vi­o­lence against a pro­tected group? Is this im­age safe to show to a mi­nor? Did the as­sis­tant refuse the re­quest?”

Every prod­uct that ships a model needs to an­swer ques­tions like these — but the right an­swer de­pends on the prod­uct, the au­di­ence, and the mo­ment. The same con­tent can be fine for a cy­ber­se­cu­rity re­search tool and harm­ful on a men­tal-health plat­form. Most guardrail mod­els bake a fixed tax­on­omy of harm cat­e­gories into their weights, so re-tar­get­ing them to a new de­ploy­ment con­text means re­train­ing. And be­cause safety de­f­i­n­i­tions dif­fer across ap­pli­ca­tions and do­mains, there is no sin­gle “correct” set of cat­e­gories to model in the first place.

Shieldstral takes a dif­fer­ent ap­proach: you write the pol­icy as a plain-lan­guage ques­tion at in­fer­ence time, and the model re­turns a cal­i­brated safety score. No re­train­ing, one in­ter­face for text and im­ages, and a ver­dict from a sin­gle to­ken. Please re­fer to our tech­ni­cal re­port here.

As an in­au­gural mem­ber of the Open Secure AI Alliance with NVIDIA and other or­ga­ni­za­tions, to­day we’re re­leas­ing Shieldstral as open weights un­der Apache 2.0, avail­able for down­load here.

Moderation as a ques­tion

Shieldstral frames con­tent mod­er­a­tion as a bi­nary ques­tion-an­swer­ing task. Each re­quest has three parts:

<Instruct> — the eval­u­a­tion con­text, strict­ness, and (optionally) a de­f­i­n­i­tion of what counts as un­safe con­tent.

<Instruct> — the eval­u­a­tion con­text, strict­ness, and (optionally) a de­f­i­n­i­tion of what counts as un­safe con­tent.

<Query> — a sin­gle yes/​no ques­tion, e.g. “Does this con­tent pro­mote phys­i­cal vi­o­lence?”

<Query> — a sin­gle yes/​no ques­tion, e.g. “Does this con­tent pro­mote phys­i­cal vi­o­lence?”

<Document> — the con­tent to judge: a prompt, a re­sponse, a prompt–re­sponse pair, or an im­age with op­tional text.

<Document> — the con­tent to judge: a prompt, a re­sponse, a prompt–re­sponse pair, or an im­age with op­tional text.

At in­fer­ence the model reads out only the yes and no log­its and soft­max-nor­mal­izes them into a con­tin­u­ous safety score. This one sim­ple for­mu­la­tion does a lot of work: it uni­fies prompt clas­si­fi­ca­tion, re­sponse mod­er­a­tion, re­fusal de­tec­tion, and tox­i­c­ity de­tec­tion into a sin­gle prob­lem; it lets poli­cies live en­tirely in the prompt, so one check­point adapts to novel poli­cies at de­ploy­ment time.

Highlights

Strong per­for­mance — matches or out­per­forms open guard mod­els up to 7× its size across text safety, re­fusal de­tec­tion, pol­icy adapt­abil­ity, and mul­ti­modal bench­marks.

Strong per­for­mance — matches or out­per­forms open guard mod­els up to 7× its size across text safety, re­fusal de­tec­tion, pol­icy adapt­abil­ity, and mul­ti­modal bench­marks.

Adaptive and flex­i­ble — a sin­gle nat­ural-lan­guage in­ter­face cov­ers text, im­age, and text+im­age con­tent across prompts, re­sponses, and prompt–re­sponse pairs. Policies are sup­plied as free-form queries and re-tar­geted at in­fer­ence time, with­out re­train­ing.

Adaptive and flex­i­ble — a sin­gle nat­ural-lan­guage in­ter­face cov­ers text, im­age, and text+im­age con­tent across prompts, re­sponses, and prompt–re­sponse pairs. Policies are sup­plied as free-form queries and re-tar­geted at in­fer­ence time, with­out re­train­ing.

Small, trained on het­ero­ge­neous sources — a 3B model that runs on a sin­gle 16GB GPU, trained on real and syn­thetic data with di­verse la­bel for­mats and tax­onomies, con­sol­i­dated into one frame­work.

Small, trained on het­ero­ge­neous sources — a 3B model that runs on a sin­gle 16GB GPU, trained on real and syn­thetic data with di­verse la­bel for­mats and tax­onomies, con­sol­i­dated into one frame­work.

Continuous safety score — re­turns a cal­i­brated yes/​no prob­a­bil­ity from a sin­gle for­ward pass, so you can thresh­old or rank by con­fi­dence rather than re­ly­ing on a dis­crete la­bel.

Continuous safety score — re­turns a cal­i­brated yes/​no prob­a­bil­ity from a sin­gle for­ward pass, so you can thresh­old or rank by con­fi­dence rather than re­ly­ing on a dis­crete la­bel.

Open — Apache 2.0 weights.

Open — Apache 2.0 weights.

Benchmarks

We eval­u­ate Shieldstral against open guard mod­els up to 7x its size across four axes. All eval­u­a­tion sam­ples are held out from train­ing.

How we built it

The core idea is that a small model can beat much larger ones if the data is right. Getting the data right meant solv­ing four prob­lems:

Unify het­ero­ge­neous data. Public safety datasets dis­agree on tax­onomies, la­bels, and an­no­ta­tion con­ven­tions — from bi­nary safe/​un­safe flags to fine-grained multi-la­bel tax­onomies. We con­vert every dataset into the same in­struc­tion–query–doc­u­ment for­mat with a per-dataset proces­sor, and we vary the word­ing of in­struc­tions, queries, and prompt–re­sponse de­lim­iters so the model gen­er­al­izes across phras­ing in­stead of over­fit­ting to one style. We also cal­i­brate strict­ness per source — strict for ad­ver­sar­ial jail­breaks, le­nient for re­sponse-qual­ity data — so the model learns cal­i­brated de­ci­sion bound­aries. This lets us con­sol­i­date sources that would oth­er­wise be in­com­pat­i­ble.

Teach dis­crim­i­na­tion, not mem­o­riza­tion. If trained on a fixed set of pol­icy la­bels, a model learns only to clas­sify those pre­de­fined poli­cies, rather than rea­son­ing about the pre­cise bound­aries of a given pol­icy. This pre­vents gen­er­al­iza­tion to novel poli­cies. Instead, we con­struct sets of de­lib­er­ately sim­i­lar, eas­ily con­fused poli­cies and ask an LLM to rewrite safe text into con­trastive pairs: each rewrite is en­gi­neered to vi­o­late one pol­icy but not its sib­ling. This trains the model to dis­tin­guish which spe­cific pol­icy a piece of con­tent vi­o­lates, a skill that trans­fers to un­seen, user-de­fined poli­cies at in­fer­ence time.

Ground safety in im­ages. Unsafe im­ages can’t be syn­thezised by an LLM the way text can, so vi­sual safety data is scarce. We sup­ple­ment lim­ited mod­er­a­tion datasets with gen­eral-pur­pose im­age datasets as high-qual­ity neg­a­tives, mu­tate queries to aug­ment the dataset, and fil­ter every im­age–query pair through a vi­sion–lan­guage reranker to re­duce mis­la­beled data and hal­lu­ci­na­tions.

Combine com­ple­men­tary check­points. We fine-tune with LoRA and merge — via SLERP — a check­point cal­i­brated on pub­lic data, one that adds fine-grained pol­icy dis­crim­i­na­tion from gen­er­ated data, and the base in­struct model. The merge re­cov­ers com­mon pol­icy cal­i­bra­tion and pol­icy adapt­abil­ity in a sin­gle model, and in­struc­tion-fol­low­ing from the base model trans­fers to the mod­er­a­tion task.

Forge. We built Shieldstral end to end on Forge, our plat­form for train­ing, align­ing, and eval­u­at­ing cus­tom mod­els. Forge man­aged the in­fra­struc­ture, data and model shard­ing, met­rics, and log­ging on top of state-of-the-art dis­trib­uted train­ing, so the team could stay fo­cused on the data which is what de­ter­mines the safety mod­el’s qual­ity.

What’s next

Shieldstral is a step to­ward mod­er­a­tion that adapts to con­text in­stead of forc­ing every prod­uct through one frozen tax­on­omy. We’re con­tin­u­ing to push on mul­ti­lin­gual cov­er­age, longer-doc­u­ment ro­bust­ness, and broader mul­ti­modal safety — and we’d love to see what the com­mu­nity builds on top of it.

BTW, we’re hir­ing! If you want to help make AI bet­ter, see our ca­reers page.

Pi’s Minimalism Is Its Advantage

AI has made code cheap, and as a re­sult many com­pa­nies are build­ing big­ger tools in pur­suit of bet­ter per­for­mance. Larger prompts, more or­ches­tra­tion, more lay­ers, more com­plex­ity. This also makes these tools in­trin­si­cally more ex­pen­sive to use. Pi takes the op­po­site ap­proach.

Pi is the cod­ing har­ness that chooses min­i­mal­ism on pur­pose. It comes out of the box with only 4 tools, and its sys­tem prompt and tool de­f­i­n­i­tions come in be­low 1,000 to­kens. The idea be­ing that most work can be done with the ba­sics, and if you want more, build it.

Evidence in­creas­ingly sug­gests that Pi’s de­sign is not just cleaner; it’s cheaper and more per­for­mant. Users are find­ing that vanilla Pi pro­duces in­dus­try lead­ing re­sults, even be­fore adding on ex­ten­sions to match user spe­cific work­flows and needs. As we’ll see in case stud­ies of Databricks and Shopify, Pi pro­duced ideal out­comes for both.

Case Studies

Databricks Study: Cost Per Task

Databricks re­cently shared their find­ings “Benchmarking Coding Agents on Databricks’ Multi-Million Line Codebase.” The goal of their re­search was to un­der­stand which cod­ing agents of­fer the best per­for­mance on real-world cod­ing tasks, and how task-per­for­mance varies with price.

To avoid bias from ex­ter­nal bench­marks that have be­come over­sat­u­rated, they cre­ated their own based on tasks their team of en­gi­neers reg­u­larly per­forms. The re­sults match what we would ex­pect, but what many in the in­dus­try may have been sur­prised to learn. In their words, “…the har­ness a model is called from dra­mat­i­cally im­pacts cost and qual­ity,” and, “in many cases, sim­ple har­nesses like Pi per­formed best on our work­loads.”

When com­bined with Opus 4.8, xhigh, Pi had the high­est over­all pass-rate, at a sig­nif­i­cantly lower cost than both Claude Code and Codex.

Minimal har­ness, mea­sur­able ef­fect

Pi shines be­cause it does­n’t try to wrap the model in a bunch of de­faults and in­struc­tions that get lost in the in­struc­tion hi­er­ar­chy. Instead, Pi stays out of the mod­el’s way, and the team is able to add what they ac­tu­ally need for their work­flow.

Databricks’ study is in­sight­ful be­cause it sep­a­rates model from har­ness.

They re­ported that when they ran the same model with the same think­ing ef­fort through dif­fer­ent har­nesses, “the cost per task dif­fered sig­nif­i­cantly (more than 2x in some cases), while qual­ity re­mained the same”. We call this Pi’s “context dis­ci­pline”. “Pi sent about 3x less con­text per turn. It man­aged con­text bet­ter, keep­ing a tighter work­ing set and fin­ish­ing the tasks in fewer runs.”

We agree that one must take into ac­count end-to-end en­gi­neer­ing eco­nom­ics, and not just price per to­ken. And this is also true at the model level; we have ob­served, for in­stance, that run­ning com­plex work­flows on Haiku 4.5 was of­ten more ex­pen­sive than Sonnet 4.6, es­pe­cially when code ex­e­cu­tion was in­volved, sim­ply be­cause the agent re­quired more turns to com­plete the task suc­cess­fully.

Now we see this at the har­ness level too; stronger, more ex­pen­sive mod­els with a per­for­mant har­ness can be cheaper than the con­verse.

Shopify builds Pi Autoresearch: Extensible beats bloat

Minimalism is part of Pi’s core phi­los­o­phy. What makes this work is that min­i­mal does not mean in­flex­i­ble. In fact, it is the first widely used agen­tic in­fra­struc­ture cre­ated for ex­ten­si­bil­ity and self-ed­itabil­ity.

Another in­sight­ful ex­ter­nal val­i­da­tion of Pi’s de­sign comes from Shopify. In this post from Shopify Engineering, David Cortés de­scribes build­ing pi-au­tore­search di­rectly as a Pi ex­ten­sion, by sim­ply ask­ing “Pi, [to] cre­ate an ex­ten­sion for Autoresearch…”. Pi reads its own ex­ten­sion doc­u­men­ta­tion and starts build­ing a new work­flow from there.

Autoresearch is an au­tonomous loop for op­ti­miza­tion with cod­ing agents. When you ask for a change, it runs ex­per­i­ments to find out what works and what causes re­gres­sions. For as long as the tar­get is mea­sur­able, it can throw out these re­gres­sions and keep self-im­prov­ing.

For Shopify and oth­ers, the Autoresearch ex­ten­sion quickly be­came a se­ri­ous in­ter­nal pro­duc­tiv­ity tool. Shopify re­ported cases in­clud­ing unit tests run­ning “300 times faster,” React com­po­nent mount­ing “20% faster,” re­duced build times across mul­ti­ple pro­jects, and even im­prove­ments to pnpm per­for­mance.

The im­por­tant point here is that Pi does­n’t ship any of these tools out of the box. Instead, it makes it ridicu­lously sim­ple for you to build them. Instead of as­sum­ing the ven­dor knows your work­flow and try­ing to ship every tool un­der the sun, Pi as­sumes you know best, and gifts you ex­ten­si­bil­ity to wield and craft your own work­flow.

Why min­i­mal wins now

About a year ago, an ar­gu­ment could be made for na­tive har­nesses hav­ing a struc­tural ad­van­tage over all oth­ers, be­cause mod­els were built around them. However, this ar­gu­ment has got­ten weaker.

Frontier mod­els are now gen­er­ally very com­pe­tent at un­der­stand­ing a ter­mi­nal (or ter­mi­nal-style) cod­ing en­vi­ron­ment, and act­ing within it. Anthropic re­cently cut­ting down Claude Code’s sys­tem prompt by 80% is a clear sign of this. So the ques­tion is be­com­ing less about how na­tive the har­ness is, and more about how it han­dles con­text to avoid re­dun­dancy and act with clean prim­i­tives. Models need a clean in­ter­face to the en­vi­ron­ment, and a har­ness that does not waste con­text.

Pi pro­vides this: less prompt over­head and re­peated con­text, cheaper runs, fewer un­nec­es­sary ab­strac­tions. Because it is ex­ten­si­ble, you do not lose power, but gain se­lec­tiv­ity. You add com­plex­ity only when it “earns its keep”.

We are also see­ing lo­cal mod­els de­vel­op­ing fast, and at Earendil we find them very promis­ing. Pi’s con­text dis­ci­pline is es­pe­cially an as­set here. Local mod­els usu­ally have lower con­text win­dows, and pre­fill can take a long time, so pre­serv­ing a sta­ble prompt pre­fix mat­ters. Context dis­ci­pline means we do not change the con­text with­out the user ex­plic­itly ask­ing for it, avoid­ing minute-long re-pre­fill­ing. Combined with the min­i­mal de­fault sys­tem prompt and tool set, this makes pi an ideal har­ness for lo­cal mod­els.

Pi is prov­ing that it can man­age it all. To be cheaper, min­i­mal, and more per­for­mant.

WINONA, Minn. (Valley News Live) - Every Flock li­cense plate reader cam­era op­er­ated by the Winona Police Department has been sawed off and stolen in what in­ves­ti­ga­tors be­lieve was a co­or­di­nated theft.

All eight cam­eras were taken Aug. 1, ac­cord­ing to the Winona Police Department. A pa­trol of­fi­cer first no­ticed the cam­eras had not sent any alerts in 24 hours. When of­fi­cers checked the lo­ca­tions, they found the cam­eras had been cut from their poles and taken. The poles were left be­hind.

Two ad­di­tional Flock cam­eras on the Mississippi River Bridge, owned by Buffalo County, were also stolen in the same man­ner.

Each cam­era is val­ued at ap­prox­i­mately $3,000, putting the to­tal loss at roughly $24,000. That cost falls di­rectly on the Winona Police Department’s bud­get.

The eight WPD cam­eras were po­si­tioned at key high­way en­try and exit points through­out the city: near Highway 61 and Bundy Boulevard, Highway 43 and Homer Road, Highway 14 and Knapp Valley Drive, and Highway 61 and Highway 14. Two cam­eras were sta­tioned at each lo­ca­tion.

Flock Safety cam­eras are au­to­mated li­cense plate read­ers that cap­ture im­ages of plates along with a ve­hi­cle’s make, model and color. Police use the tech­nol­ogy to in­ves­ti­gate crimes such as hit-and-runs and to lo­cate miss­ing per­sons. Camera data is owned by the po­lice de­part­ment and per­ma­nently deleted af­ter 30 days.

The thefts are part of a broader na­tional trend. Flock cam­eras have been van­dal­ized and cut down in com­mu­ni­ties across the coun­try. The tech­nol­ogy has drawn crit­i­cism from civil lib­er­ties ad­vo­cates who ar­gue the cam­eras ex­pand gov­ern­ment sur­veil­lance and raise con­cerns about data be­ing shared with third-party com­pa­nies or fed­eral agen­cies.

No sus­pects have been iden­ti­fied. The in­ves­ti­ga­tion is on­go­ing. Anyone with in­for­ma­tion is asked to con­tact the Winona Police Department.

Sources: News 8000 | Winona Post

Copyright 2026 KVLY. All rights re­served.

Apple is now seek­ing a pre­lim­i­nary in­junc­tion in its trade se­crets case against OpenAI, which aims to stop the AI model maker from mov­ing for­ward with de­vel­op­ing an AI de­vice or other prod­ucts based on Apple’s tech­nol­ogy. The iPhone maker also claims that more of its for­mer em­ploy­ees may be in­volved with the trade se­crets theft.

In a new fil­ing, Apple is re­quest­ing ex­pe­dited dis­cov­ery from the ac­cused OpenAI em­ploy­ees, se­nior sys­tems en­gi­neer Chang Liu and Chief Hardware Officer Tang Yew Tan; OpenAI, and its foun­da­tion; and io, the de­vice startup co-founded by Apple’s for­mer lead de­signer Jony Ive.

Apple also notes that its con­tin­ued in­ves­ti­ga­tion has so far re­vealed 11 other for­mer Apple em­ploy­ees be­yond Liu and Tan may have been wit­nesses or oth­er­wise in­volved in the case, and oth­ers who were pre­vi­ously named in the orig­i­nal com­plaint, like OpenAI em­ployee Yu-Ting Peng.

The fil­ing marks an es­ca­la­tion in Apple’s le­gal bat­tle with OpenAI, as it sug­gests Apple has un­cov­ered new ev­i­dence that the mis­con­duct goes be­yond the for­mer em­ploy­ees named in the orig­i­nal com­plaint.

“For ex­am­ple, an­other for­mer Apple em­ployee seems to have met with Mr. Liu and Ms. Peng in ad­vance of Ms. Peng’s in­ter­view at OpenAI and dis­cussed with them dur­ing that meet­ing Apple pro­pri­etary in­for­ma­tion re­lat­ing to unan­nounced prod­ucts,” the fil­ing states. “Yet an­other for­mer Apple em­ployee took screen­shots of con­fi­den­tial Apple doc­u­ments re­lat­ing to an unan­nounced Apple prod­uct be­fore an in­ter­view at OpenAI.”

“And, af­ter Apple filed its com­plaint, mul­ti­ple for­mer Apple em­ploy­ees now work­ing at OpenAI reached out to dis­cuss re­turn­ing Apple-issued work de­vices they kept when they left Apple,” Apple claims, sug­gest­ing there were more who were pos­si­bly in­volved with the scheme.

Apple is push­ing the court to al­low for ex­pe­dited dis­cov­ery be­cause it be­lieves it has good cause to sus­pect that there are oth­ers in­volved in the theft of its in­tel­lec­tual prop­erty. The com­pany noted that its mo­tion for a pre­lim­i­nary in­junc­tion is also pend­ing.

OpenAI re­sponded pub­licly to Apple’s lat­est, say­ing in a blog post that Apple’s re­quest for a pre­lim­i­nary in­junc­tion is “both based on false in­for­ma­tion and com­pletely un­nec­es­sary be­cause we do not have, nor want, any of their trade se­crets.”

“We’re much more in­ter­ested in build­ing in­no­v­a­tive prod­ucts and tech­nolo­gies that push the fron­tier,” OpenAI’s state­ment reads.

The AI model maker also pointed to ear­lier mis­takes Apple made, which had been re­ported, in­clud­ing that Apple emailed the wrong per­son when it made con­tact with OpenAI af­ter con­fus­ing two sim­i­lar sur­names. OpenAI also al­leges that Apple lied about dis­cussing mat­ters with its gen­eral coun­sel. And, the com­pany said that Apple did­n’t ad­mit to the claim that the “residual ac­cess” al­low­ing for­mer em­ploy­ees to ac­cess Apple’s sys­tem was the re­sult of poor se­cu­rity pro­ce­dures on Apple’s part.

When you pur­chase through links in our ar­ti­cles, we may earn a small com­mis­sion. This does­n’t af­fect our ed­i­to­r­ial in­de­pen­dence.

Sarah has worked as a re­porter for TechCrunch since August 2011. She joined the com­pany af­ter hav­ing pre­vi­ously spent over three years at ReadWriteWeb. Prior to her work as a re­porter, Sarah worked in I.T. across a num­ber of in­dus­tries, in­clud­ing bank­ing, re­tail and soft­ware.

You can con­tact or ver­ify out­reach from Sarah by email­ing sarahp@techcrunch.com or via en­crypted mes­sage at sarah­perez.01 on Signal.

View Bio

Starting to­day, any­one in Dallas can down­load the Waymo app and hail a fully au­tonomous ride. Since open­ing our ser­vice in February, we’ve wel­comed nearly 150,000 rid­ers in Dallas from our in­ter­est list to ex­pe­ri­ence the safety, re­li­a­bil­ity, and magic of Waymo, and now we’re of­fer­ing our ser­vice to every­one.

We’ve proudly helped Dallasites run er­rands, com­mute to work, and cel­e­brate nights out with friends. Now, we’re go­ing a step fur­ther to un­lock a new way for tourists and other vis­i­tors to get around the city. We con­tinue fully au­tonomous test­ing at Dallas Love Field Airport ter­mi­nals and look for­ward to serv­ing trav­el­ers there soon. And to get there ef­fi­ciently, we’ll soon be­gin fully au­tonomous test­ing on Dallas free­ways, which is the fi­nal step be­fore of­fer­ing these routes to pub­lic rid­ers.

The Dallas com­mu­nity has em­braced Waymo as a vi­tal re­source for ex­pand­ing ac­ces­si­ble and re­li­able trans­porta­tion op­tions across the re­gion. Chris Justl, CEO, Epilepsy Foundation Texas, shared, “Waymo au­tonomous ve­hi­cles are not just the fu­ture—they’re a trans­for­ma­tional step for­ward for the epilepsy com­mu­nity and any­one liv­ing with med­ical con­di­tions that limit their abil­ity to drive, cre­at­ing a new path­way to safe, in­de­pen­dent travel. At Epilepsy Foundation Texas, we’re proud to part­ner with Waymo to help bring this fu­ture to life across Texas.”

Rolling with Waymo in Dallas has never been eas­ier. Simply down­load the Waymo app and ride to­day!

For read­ers new to Expat:

lib­ex­pat is a fast stream­ing XML parser. Alongside libxml2, Expat is one of the most widely used soft­ware li­bre XML parsers writ­ten in C, specif­i­cally C99. It is cross-plat­form and li­censed un­der the MIT li­cense.

lib­ex­pat is a fast stream­ing XML parser. Alongside libxml2, Expat is one of the most widely used soft­ware li­bre XML parsers writ­ten in C, specif­i­cally C99. It is cross-plat­form and li­censed un­der the MIT li­cense.

Starting 2026 – 08-01, the “security va­ca­tion” of the pro­ject has ended and(!) I will be be paid to work on main­tain­ing lib­ex­pat for up to 6 months thanks to the City of Munich un­der the um­brella of their Open Source Sabbatical pro­gram. What does that mean?

For much of the past 10 years, work­ing on lib­ex­pat has been com­pet­ing with my reg­u­lar oc­cu­pa­tion as a soft­ware en­gi­neer, chores, so­cial life and re-cre­ation. For the first time, I am now be­ing em­ployed to work on main­tain­ing lib­ex­pat as my “regular job” for a lim­ited pe­riod of time. My top pri­or­i­ties will be:

Fixing the cur­rently 5 known un­fixed vul­ner­a­bil­i­ties

Fixing the cur­rently 5 known un­fixed vul­ner­a­bil­i­ties

Adding sup­port for XML 1.0r5

Adding sup­port for XML 1.0r5

Further im­prov­ing the ro­bust­ness and main­tain­abil­ity of the pro­ject

Further im­prov­ing the ro­bust­ness and main­tain­abil­ity of the pro­ject

Yesterday and to­day most of my time went into fix­ing a vul­ner­a­bil­ity un­cov­ered by Mozilla.

Technically, I am be­ing em­ployed by digi­tial@M now for of up 6 months with a reg­u­lar work­ing con­tract, in­clud­ing can­cel­la­tion by ei­ther party, re­motely from home. There is plenty to do.

Unvalidated AI slop sub­mis­sions will still not be ap­pre­cated, but for every­thing else: if you want to throw in­tel­li­gence at find­ing fur­ther vul­ner­a­bil­i­ties in lib­ex­pat and send them my way, the com­ing months will be the best chance at get­ting things fixed in rea­son­able time. Queueing the­ory and laws of physics still ap­ply.

Wish me luck!

PS: If any­one man­aged to com­bine Clang-based MinGW with AddressSanitizer and Wine with­out crash­ing at launch, please show me how and drop me an e-mail. Thank you!

Best, Sebastian

31st July 2026

Tuesday was Stateless MCP day—the roll­out of MCP 2.0, or the 2026 – 07-28 Model Context Protocol spec­i­fi­ca­tion to use the more for­mal but less mem­o­rable name. This is the most sig­nif­i­cant change to the MCP spec since it first launched, and has also served to reignite my per­sonal in­ter­est in the pro­to­col.

For back­ground: MCP is the Model Context Protocol, which de­scribes a stan­dard way to ex­pose new tools to LLM-powered agent frame­works. It was in­tro­duced by Anthropic back in November 2024, had a huge spike of in­ter­est through much of 2025, and then be­came some­what eclipsed by Skills (another Anthropic in­ven­tion) when it be­came ap­par­ent that an agent har­ness with ac­cess to a ter­mi­nal and curl could do most of what MCP did in a more flex­i­ble way. I wrote about that in my re­view of 2025.

I’m com­ing back around to MCP now. Giving an agent a shell en­vi­ron­ment with the abil­ity to ac­cess the in­ter­net is fraught with risk, and re­quires a strong model that is ca­pa­ble of ef­fec­tively dri­ving such an en­vi­ron­ment. MCP tools are eas­ier to au­dit and con­trol, and sim­ple enough that smaller mod­els that run on a lap­top can still drive them rea­son­ably well.

The new state­less MCP spec­i­fi­ca­tion also greatly de­creases the com­plex­ity of im­ple­ment­ing both clients and servers for the pro­to­col. I built three of those this week!

What’s eas­ier with state­less MCP

The best demon­stra­tion of the dif­fer­ence be­tween state­ful and state­less MCP is in this May 21st blog post that in­tro­duced the RC for the new spec­i­fi­ca­tion. It in­cluded a clear be­fore-and-af­ter ex­am­ple.

The older state­ful MCP (I’m go­ing to call it “legacy MCP”) re­quired two HTTP re­quests—the first to ini­tial­ize a ses­sion and ob­tain a Mcp-Session-Id, and the sec­ond to ac­tu­ally call the tool:

POST /mcp HTTP/1.1 Content-Type: ap­pli­ca­tion/​json

{ “jsonrpc”: “2.0″, “id”: 1, “method”: “initialize”, “params”: { “protocolVersion”: “2025 – 11-25″, “capabilities”: { }, “clientInfo”: { “name”: “my-app”, “version”: “1.0″ } } }

POST /mcp HTTP/1.1 Mcp-Session-Id: 1868a90c-3a3f-4f5b Content-Type: ap­pli­ca­tion/​json

{ “jsonrpc”: “2.0″, “id”: 2, “method”: “tools/call”, “params”: { “name”: “search”, “arguments”: { “q”: “otters” } } }

The new state­less way uses a sin­gle HTTP re­quest which looks like this:

POST /mcp HTTP/1.1 MCP-Protocol-Version: 2026 – 07-28 Mcp-Method: tools/​call Mcp-Name: search Content-Type: ap­pli­ca­tion/​json

{ “jsonrpc”: “2.0″, “id”: 1, “method”: “tools/call”, “params”: { “name”: “search”, “arguments”: { “q”: “otters” }, “_meta”: { “io.modelcontextprotocol/clientInfo”: { “name”: “my-app”, “version”: “1.0” } } } }

This is so much cleaner from both a client- and server-side im­ple­men­ta­tion per­spec­tive. It’s also a bet­ter fit for build­ing scal­able web ap­pli­ca­tions, since now you don’t need to main­tain server-side state to keep track of those ses­sion IDs, or worry about rout­ing the same ses­sion to the same back­end ma­chine.

mcp-ex­plorer

I could­n’t find a great CLI tool for in­ter­ac­tively prob­ing an MCP server, so I had Codex help build my own.

mcp-ex­plorer is the re­sult. It’s a state­less Python CLI tool, so you don’t even need to in­stall it to try it out—it works with uvx like this:

uvx mcp-ex­plorer list https://​agen­tic-mer­maid.dev/​mcp

This queries Ade Oshineye’s agen­tic-mer­maid.dev demo MCP. The above com­mand re­turns the fol­low­ing list of tools:

ex­e­cute(code: string, time­outMs?: in­te­ger) - Execute Mermaid SDK code Run JavaScript in an iso­lated sand­box; re­turn a value.

de­scribe_sdk(fam­ily: string, de­tail?: string) - Describe Mermaid SDK op­er­a­tions Return ver­sion-matched mu­ta­tion op­er­a­tions for one di­a­gram fam­ily.

ren­der_svg(source: string, op­tions?: ob­ject) - Render Mermaid as SVG Render a Mermaid source string to the­me­able SVG. Returns { ok, svg }.

ren­der_ascii(source: string, use­Ascii?: boolean, tar­getWidth?: in­te­ger, op­tions?: ob­ject) - Render Mermaid as text Render a Mermaid source string to text. Returns { ok, text }.

ren­der_png(source: string, scale?: num­ber, back­ground?: string, fitTo?: ob­ject, op­tions?: ob­ject) - Render Mermaid as PNG Rasterize a Mermaid source string to PNG. Returns { ok, png_base64 }. …

Then to in­spect a tool:

uvx mcp-ex­plorer in­spect ren­der_svg

This out­puts a whole bunch of in­for­ma­tion, in­clud­ing the JSON schema of the in­puts and out­puts.

To call that tool and pass ar­gu­ments to it:

uvx mcp-ex­plorer call \ https://​agen­tic-mer­maid.dev/​mcp \ ren­der_svg \ -a source ‘graph TD; A–>B’ \ -a op­tions ‘{“padding”:24}’

Which re­turns:

{“ok”:true,“svg”:“<svg xmlns="hhttp://​www.w3.org/​2000/​svg\ width=…

To get just the raw SVG try adding | jq .svg -r to that com­mand. I got back this im­age:

There are a few more com­mands in the README, but you get the gen­eral idea. I find build­ing CLI tools like this to be a re­ally pro­duc­tive way to get fa­mil­iar with a spec­i­fi­ca­tion, even if an agent writes most of the ac­tual code.

datasette-mcp

The sec­ond pro­ject is datasette-mcp, a Datasette plu­gin which adds a /-/mcp end­point to any Datasette in­stance.

This is prob­a­bly the fourth time I’ve tried build­ing this plu­gin, but thanks to the new state­less MCP spec­i­fi­ca­tion I fi­nally have a ver­sion that feels good to re­lease.

It pro­vides just three tools: list_­data­bases(), get_­data­base_schema(data­base_­name), and ex­e­cute_sql(data­base_­name, sql). They do ex­actly what you would ex­pect them to do—though ex­e­cute_sql() is read-only for the mo­ment.

Wire these into an agent, or a chat tool like ChatGPT or Claude, and they’ll gain the abil­ity to run SQL queries against your hosted Datasette in­stance.

So far I’m run­ning it on the Datasette mir­ror of my blog, at datasette.si­mon­willi­son.net/-/​mcp. It took a bit of fid­dling to fig­ure out how to at­tach that to ChatGPT and Claude, but I got there in the end. Here’s a new TIL show­ing ex­actly how to do that.

Here’s a shared Claude ses­sion where I asked it:

list ta­bles in si­mon­willi­son.net

list ta­bles in si­mon­willi­son.net

And then:

what has Simon said re­cently about MCP?

what has Simon said re­cently about MCP?

It ran 7 sep­a­rate SQL queries to fig­ure out the an­swer.

llm-mcp-client

My LLM tool is long over­due for an of­fi­cial MCP in­te­gra­tion. The new al­pha llm-mcp-client plu­gin is my at­tempt at ex­actly that:

llm in­stall llm-mcp-client llm -T ’MCP(“https://​datasette.si­mon­willi­son.net/-/​mcp”)′ ‘count the notes’

Here’s the out­put (including rea­son­ing trace, I’m us­ing LLM 0.32rc2):

Considering note count I see the ques­tion “count the notes” is prob­a­bly ask­ing me to tally up blog notes. It could also mean pub­lished notes or drafts, so there’s some am­bi­gu­ity there. I’ll need to fig­ure out the to­tal num­ber of notes, likely by query­ing the count for both pub­lished notes and drafts to get a clear an­swer. Let’s ex­e­cute that count! There are 151 notes.

Considering note count

I see the ques­tion “count the notes” is prob­a­bly ask­ing me to tally up blog notes. It could also mean pub­lished notes or drafts, so there’s some am­bi­gu­ity there. I’ll need to fig­ure out the to­tal num­ber of notes, likely by query­ing the count for both pub­lished notes and drafts to get a clear an­swer. Let’s ex­e­cute that count!

There are 151 notes.

And the out­put of llm logs for that prompt.

Once this is fully baked, I’m con­sid­er­ing bring­ing it di­rectly into LLM core. I’m ex­cited to ex­per­i­ment with MCP in Datasette Agent and llm-cod­ing-agent as well.

MCP is a safer way to build with agents

A few months af­ter MCP was first re­leased, I wrote Model Context Protocol has prompt in­jec­tion se­cu­rity prob­lems, where I noted that the pat­tern of hav­ing end users mix and match tools pushed re­spon­si­bil­ity for avoid­ing data ex­fil­tra­tion at­tacks out to the users them­selves. I had­n’t coined the Lethal Trifecta yet, but that was ab­solutely what I had in mind.

Then gen­eral agents with ar­bi­trary shell and curl ac­cess came along, and that’s so much harder to keep se­cure!

Something I’ve come to ap­pre­ci­ate about MCP is that it’s much eas­ier to rea­son about agent ca­pa­bil­i­ties and what might go wrong than with ar­bi­trary com­mand ex­e­cu­tion in an open net­work en­vi­ron­ment—the de­fault for most of to­day’s gen­eral and cod­ing agent tools.

I plan to lean into MCP a whole lot more when I’m build­ing sen­si­tive ap­pli­ca­tions on top of LLMs.

Artificial in­tel­li­gence is now pow­er­ing more than half of re­ported cy­ber­crime across Africa, al­low­ing crim­i­nals to launch faster, more con­vinc­ing and larger-scale at­tacks, ac­cord­ing to INTERPOL’s African Cyberthreat Assessment Report 2026.

The re­port found that 55% of cy­ber­crime cases recorded across the con­ti­nent in­volve the use of AI, rais­ing con­cerns as Africa’s dig­i­tal econ­omy con­tin­ues to ex­pand.

With more than 1.1 bil­lion mo­bile sub­scribers in 2025, mil­lions of peo­ple are re­ly­ing on dig­i­tal ser­vices, cre­at­ing new op­por­tu­ni­ties for both in­no­va­tion and cy­ber­crim­i­nals.

Based on data from 36 African coun­tries, the 40-page as­sess­ment says cy­ber­crime has evolved into a highly or­gan­ised, cross-bor­der in­dus­try that is be­com­ing harder for au­thor­i­ties to de­tect and stop.

Online scams re­main Africa’s biggest cy­ber threat

According to the re­port, on­line scams re­mained the most com­mon form of cy­ber­crime in 2025. Criminals in­creas­ingly used ar­ti­fi­cial in­tel­li­gence along­side so­cial me­dia plat­forms and mo­bile money ser­vices to tar­get vic­tims.

INTERPOL said cy­ber­crime-re­lated fi­nan­cial losses have risen sharply over the past year, climb­ing from $192 mil­lion in 2024 to $484 mil­lion. Investigators at­tribute the in­crease to AI-powered fraud, stolen lo­gin cre­den­tials and so­phis­ti­cated so­cial en­gi­neer­ing at­tacks.

The re­port also found that 72% of sur­veyed coun­tries iden­ti­fied scam cen­tres op­er­at­ing within their bor­ders, with the high­est con­cen­tra­tion in West and Southern Africa.

Different re­gions face dif­fer­ent cy­ber risks

The re­port high­lights dis­tinct cy­ber­crime trends across the con­ti­nent.

In East Africa, mo­bile money fraud and ran­somware at­tacks tar­get­ing crit­i­cal in­fra­struc­ture are among the biggest threats.

West and Central Africa con­tinue to ex­pe­ri­ence high lev­els of busi­ness email com­pro­mise and ro­mance scams af­fect­ing both com­pa­nies and in­di­vid­u­als.

Meanwhile, Southern Africa’s ad­vanced dig­i­tal con­nec­tiv­ity has made the re­gion an at­trac­tive tar­get for in­ter­na­tional cy­ber­crim­i­nal net­works seek­ing to max­imise dis­rup­tion.

AI is mak­ing cy­ber­crime more con­vinc­ing

INTERPOL warned that ar­ti­fi­cial in­tel­li­gence is trans­form­ing the way cy­ber­crim­i­nals op­er­ate.

Deepfake tech­nol­ogy and AI-generated con­tent are in­creas­ingly be­ing used in dig­i­tal sex­tor­tion and on­line ha­rass­ment cam­paigns. One of INTERPOL’s tech­nol­ogy part­ners, TrendAI, de­tected around 600,000 sex­tor­tion cases linked to these tac­tics.

The re­port also noted a sharp rise in Business Email Compromise (BEC) scams, where crim­i­nals use AI to pro­duce re­al­is­tic emails that im­i­tate trusted con­tacts.

Some Africa-based cy­ber­crim­i­nal groups have tar­geted busi­nesses and in­di­vid­u­als in Europe and North America, us­ing in­fra­struc­ture spread across sev­eral coun­tries to hide their ac­tiv­i­ties.

Another grow­ing con­cern is the use of syn­thetic iden­ti­ties. Rather than sim­ply steal­ing per­sonal in­for­ma­tion, cy­ber­crim­i­nals are com­bin­ing gen­uine data with fab­ri­cated de­tails to cre­ate en­tirely new dig­i­tal iden­ti­ties.

These fake pro­files have re­port­edly been used to open bank ac­counts, ob­tain mo­bile loans and reg­is­ter SIM cards while evad­ing some bio­met­ric ver­i­fi­ca­tion sys­tems.

Gaps in co­op­er­a­tion leave fi­nan­cial sys­tems ex­posed

INTERPOL said weak co­or­di­na­tion be­tween banks, tele­com com­pa­nies and law en­force­ment agen­cies con­tin­ues to ham­per ef­forts to com­bat cy­ber­crime.

The ab­sence of real-time in­for­ma­tion shar­ing cre­ates op­por­tu­ni­ties for crim­i­nals to move stolen funds quickly and ex­ploit weak­nesses across mul­ti­ple ju­ris­dic­tions be­fore au­thor­i­ties can re­spond.

The re­port also found that many African law en­force­ment agen­cies are still not ad­e­quately pre­pared to re­spond to AI-driven cy­ber threats, de­spite the rapid pace at which the tech­nol­ogy is be­ing adopted by crim­i­nal net­works.

Countries step up ef­forts against cy­ber­crime

Despite the grow­ing threat, the re­port points to progress across the con­ti­nent.

In 2025, 17 African coun­tries in­tro­duced or up­dated cy­ber­crime leg­is­la­tion. Senegal also launched an on­line re­port­ing plat­form de­signed to im­prove re­sponses to on­line of­fences in­volv­ing chil­dren.

INTERPOL said joint in­ter­na­tional op­er­a­tions have also de­liv­ered sig­nif­i­cant re­sults. Four ma­jor op­er­a­tions, Operation Serengeti 2.0, Operation Contender 3.0, Operation Sentinel and Operation Red Card 2.0, led to more than 1,500 ar­rests, the seizure of hun­dreds of elec­tronic de­vices and the re­cov­ery of over $100 mil­lion linked to cy­ber­crime.