This page is a free and open-source wiki about hard­ware hack­ing!

The goal of HardBreak (https://​www.hard­break.wiki/) is to col­lect knowl­edge about Hardware Hacking / IoT hack­ing in one place. There are many great blogs about Hardware Hacking, but it is a rather un­pleas­ant ex­pe­ri­ence to search through mul­ti­ple blogs in dif­fer­ent for­mats to find the in­for­ma­tion you need. HardBreak aims to or­ga­nize all in­for­ma­tion in one ac­ces­si­ble and easy-to-use plat­form.

🎉 We just launched our HardBreak Discord Server! 🎉

* Want to dis­cuss hard­ware hack­ing and IoT se­cu­rity

* Share the pro­ject you are work­ing on

* Have feed­back or re­quests for new con­tent on our wiki

Come be a part of our grow­ing com­mu­nity of hard­ware hack­ers⚡

We strongly en­cour­age any­one in­ter­ested to con­tribute their knowl­edge and in­sights. By shar­ing your dis­cov­er­ies or im­prov­ing ex­ist­ing con­tent, you help build a valu­able re­source for every­one.

* Help us keep the con­tent ac­cu­rate—if you no­tice an er­ror, please re­port it so we can cor­rect it quickly! Reach out on LinkedIn or Twitter

Check out our Contribution Guide for a step-by-step tu­to­r­ial to mak­ing your first pull re­quest!

Since I last wrote about WordPress, things have gone off the rails. This af­ter a brief pe­riod when things were bliss­fully quiet. Matt Mullenweg stopped com­ment­ing for a while, though his com­pany had launched WP Engine Tracker — a site for track­ing WordPress-driven web­sites that moved away from WP Engine. I think this is a bit gauche, but it seems like fair mar­ket­ing given every­thing that’s go­ing on. It should be noted that many sites are leav­ing for Pressable — owned by Mullenweg’s com­pany, Automattic — because of a sweet­heart deal.

But the drama ramped up quickly af­ter WP Engine won a pre­lim­i­nary in­junc­tion against Automattic on December 10th. The in­junc­tion re­quired that WP Engine be al­lowed to ac­cess WordPress.org re­sources, and that Automattic stop in­ter­fer­ing with WP Engine plu­g­ins, while the trial moves for­ward. Ernie Smith wrote an ex­cel­lent piece with more de­tails on out­come of the in­junc­tion, in­clud­ing a note about Mullenweg quit­ting a com­mu­nity Slack in­stance with a hammy mes­sage. Mullenweg com­plied with the in­junc­tion, though the “loyalty test check­box” text was changed to a still-re­quired note about pineap­ple on pizza.

On December 20th, Mullenweg an­nounced that WordPress.org would be on hol­i­day break for an un­spec­i­fied amount of time. In a post on the WordPress.org blog, he again men­tioned be­ing “compelled to pro­vide free la­bor and ser­vices to WP Engine thanks to the suc­cess of their ex­pen­sive lawyers”. He also in­vited peo­ple to fund le­gal at­tacks against him by sign­ing up for WP Engine ser­vices, and hoped to have the “time, en­ergy, and money to re­open all of this some­time in the new year”.

This was the first time WordPress.org had ever gone on break, and it was an­other in­stance of Mullenweg us­ing a core part of the WordPress com­mu­nity to send a mes­sage. WordPress.org re­turned to ser­vice on January 4th, but plu­gin and theme up­dates weren’t be­ing re­viewed un­til then. I’m all for giv­ing vol­un­teers time off, but this came as a sur­prise to the com­mu­nity and there was ini­tially no in­di­ca­tion when the break would end. Mullenweg’s “woe is me” lan­guage around maybe, pos­si­bly, be­ing able to find the re­sources to re­open a core piece of WordPress in­fra­struc­ture did­n’t help things. It fur­ther ce­mented that Matt Mullenweg’s cur­rent mood is an im­por­tant func­tion of whether or not the WordPress com­mu­nity op­er­ates smoothly.

While WordPress.org was on hia­tus, Mullenweg also reached out to the WPDrama com­mu­nity on Reddit, ask­ing what drama he should cre­ate in 2025. A cou­ple of years ago, this sort of thing would have been some tongue-in-cheek fun from a per­son who’s al­ways been a bit of a goof. These days it comes off a bit dif­fer­ently.

Then yes­ter­day hap­pened. Automattic an­nounced that it would re­strict its con­tri­bu­tions to the open source ver­sion of WordPress. The com­pany would now only put in about 45 hours a week to­tal — down from nearly 4,000 a week — so as to match the es­ti­mated hourly con­tri­bu­tions of WP Engine. This ac­tion is blamed on the “the le­gal at­tacks started by WP Engine and funded by Silver Lake”, which I think is a gross mis­char­ac­ter­i­za­tion. WP Engine def­i­nitely did not start this.

Automattic noted it would fo­cus its open source hours on “security and crit­i­cal up­dates”. The other hours would be redi­rected to for-profit pro­jects like WordPress.com. This means that the com­mu­nity will be ex­pected to take up the slack if it wants WordPress to im­prove. I worry that 45 hours a week is­n’t enough time to keep WordPress se­cure and bug-free. Hopefully oth­ers will step up, here.

But you know what? In a nor­mal world, hav­ing the com­mu­nity tak­ing the lead would be fine. I’d be all for it! The prob­lem is that Mullenweg has fi­nal say over some very im­por­tant parts of the WordPress com­mu­nity. He also seems re­cently to be act­ing more child­ishly and im­pul­sively than usual.

Another thing that came to light yes­ter­day was that the WordPress Sustainability com­mit­tee was shut­tered af­ter a core mem­ber, Thijs Buijs, stepped down. In a post on the WordPress Slack, Buijs cited the “2025 drama” post on Reddit as the rea­son he was leav­ing, and called for a change in WordPress com­mu­nity lead­er­ship. In re­sponse, Mullenweg re­sponded in part with “[t]oday I learned that we have a sus­tain­abil­ity team”, and closed the chan­nel.

The WordPress Sustainability Team had four core mem­bers, and 11 peo­ple who had con­tributed on GitHub. As far as I can tell, they were all com­mu­nity mem­bers, and none were Automattic em­ploy­ees. Even if it was­n’t pro­duc­ing amaz­ing re­sults, I can’t see what harm it was do­ing. The sin was point­ing out some­thing stu­pid that Mullenweg did, and hav­ing a mem­ber want­ing change. The op­tics, es­pe­cially given cur­rent world events, are def­i­nitely not great. The wildest part of this to me is that there’s video of Mullenweg — live on stage at Word Camp Europe in 2022 — re­quest­ing the cre­ation of the Slack chan­nel he was turn­ing off. Guess that slipped his mind.

All of this bodes poorly for the open source ver­sion of WordPress. I think it’s per­fectly fair for Automattic to switch gears and fo­cus on for-profit pro­jects — it’s a com­pany af­ter all. The prob­lem is that there’s a void be­ing left. Automattic had, for bet­ter or worse, lead the de­vel­op­ment of both the com­mer­cial and open source pieces of the WordPress ecosys­tem. Now it seems like the com­mu­nity needs to take over, but Mullenweg still holds all the keys.

In the an­nounce­ment, Automattic said that WordPress.com would be up­dated to be more in line with the open source ver­sion of WordPress. This also makes sense to me, as WordPress.com has al­ways been a weird ver­sion of the soft­ware. Of course, hav­ing slight dif­fer­ences to the core WordPress ex­pe­ri­ence is the ar­gu­ment that Mullenweg ini­tially used to call WP Engine a “cancer”, but who’s keep­ing track? I’d also like to point out again that Automattic in­vested in WP Engine in 2011. It also ac­quired Pressable in 2016, likely be­cause it was a host­ing ser­vice that of­fered a “real” ver­sion of WordPress, un­like WordPress.com.

It’s hard to see how to move for­ward from here. I think the best bet would be for peo­ple to rally around new com­mu­nity-dri­ven in­fra­struc­ture. This would likely re­quire a fork of WordPress, though, and that’s go­ing to be a messy. The cur­rent open source ver­sion of WordPress re­lies on the sites and ser­vices Mullenweg con­trols. Joost de Valk, the orig­i­nal cre­ator of an ex­tremely pop­u­lar SEO plu­gin, wrote a blog post with some thoughts on the mat­ter. I’m hop­ing that more promi­nent peo­ple in the com­mu­nity step up like this, and that some way for­ward can be found.

In the mean­time, if you’re a WordPress de­vel­oper, you may want to look into some other op­tions.

Update: Moments af­ter post­ing this, I was pointed to a story on TechCrunch about Mullenweg de­ac­ti­vat­ing the WordPress.org ac­counts of users plan­ning a “fork”. This af­ter he pre­vi­ously pro­moted (though in a slightly mock­ing way) the idea of fork­ing open source soft­ware. In both cases, the peo­ple he men­tioned weren’t ac­tu­ally plan­ning forks, but mus­ing about fu­ture ways for­ward for WordPress. Mullenweg framed the ac­count de­ac­ti­va­tions as giv­ing peo­ple the push they need to get started. Remember that WordPress.org ac­counts are re­quired to sub­mit themes, plu­g­ins, or core code to the WordPress pro­ject. These re­cent events re­ally make it seem like you’re no longer wel­come to con­tribute to WordPress if you ques­tion Matt Mullenweg.

Every morn­ing I get up and check what ma­li­cious pack­ages my de­tec­tor had found the night be­fore.   It’s like some­one check­ing their fish­ing nets to see what fish they caught.

As I was look­ing at last nights ma­li­cious pack­ages I no­ticed some­thing strange:  Someone from Snyk had de­ployed sev­eral pack­ages to NPM.  Even weirder, the names of those pack­ages ap­peared to show they were tar­get­ing Cursor, the hot new AI cod­ing com­pany.

These pack­ages were de­ployed by an NPM user named sn4k-s3c.  The pack­ages are named things like “cursor-retreival”, “cursor-always-local” and “cursor-shadow-workspace”.

If you in­stall any of these pack­ages they will col­lect data about your sys­tem and send it to an at­tacker con­trolled web ser­vice.

13 Jan 2025

Here’s my fluid sim­u­la­tion pen­dant, a hand­crafted piece of jew­ellery run­ning a re­al­time FLIP fluid sim­u­la­tion. The en­clo­sure is gold plated, and the dis­play is pro­tected by a watch glass.

Watch the fol­low­ing video to ex­pe­ri­ence the nar­rated de­sign and con­struc­tion:

I pro­duced the first pen­dant in March 2024, and then sev­eral more over the next few months. I now have a small hand­ful of pen­dants, and if you’d re­ally, re­ally like to own one, then while stocks last, a few of them are for sale.

The mo­ti­va­tion and ini­tial de­sign is de­scribed ex­ten­sively in the youtube video, so I won’t re­peat my­self too much here, but in short, fol­low­ing the vol­u­met­ric dis­play an­i­ma­tions, I’ve been look­ing to im­ple­ment a re­al­time fluid sim­u­la­tion that ul­ti­mately could cre­ate a 3D vir­tual snow­globe. Progress has been made on that front, but along the way, we came up with the Simsim con­cept, upon which this pen­dant is based.

While work­ing on it, I came up with a bunch of cool new de­vel­op­ments that led to sev­eral other pro­jects, which I’ll post in due time, but this is the pro­ject that spawned them. Not just the fluid sim­u­la­tion stuff, but also the un­ex­pected ben­e­fits of a di­ag­o­nal char­lieplexed dis­play.

The FLIP sim­u­la­tion is based on the work of Matthias Müller, check out his web­site Ten Minute Physics and par­tic­u­larly the “How to write a FLIP Water Simulator” tu­to­r­ial. It ex­plains FLIP bet­ter than I can here.

My fluid sim­u­la­tion is not a di­rect port, but a re-im­ple­men­ta­tion fol­low­ing the tu­to­r­ial.

The hard­ware con­sists of an STM32L432KC (ARM Cortex-M4 with FPU, over­clocked to 100MHz), an ADXL362 ul­tra-low power ac­celerom­e­ter, an MCP73832 charge con­troller for the LiR2450 bat­tery, a TPS7A02 reg­u­la­tor (crazy low power) and a TPS3839 su­per­vi­sor. It all comes to­gether on a four-layer, 0.8mm PCB.

The key take­aways from this pro­ject are as fol­lows:

Diagonal char­lieplex­ing per­mits rout­ing with half the num­ber of vias as com­pared to a con­ven­tional ma­trix. For small-pitch LED dis­plays, the num­ber of vias is of­ten the lim­it­ing fac­tor, so this makes a huge dif­fer­ence. The arrange­ment also places LEDs with the same net end-to-end, so a sol­der bridge does­n’t im­pact per­for­mance. One could even in­ten­tion­ally squish the LEDs closer to­gether with this in mind.

DMA in cir­cu­lar mode can run a dis­play ma­trix with zero over­head. This can also be used, with some wran­gling of two DMA streams, to run a char­lieplexed ma­trix with no over­head too.

A lookup table is re­quired to map LEDs to the cor­re­spond­ing pix­els, but this means that there is no ad­di­tional cost to chang­ing that map­ping. In other words, we can con­nect any sig­nal of the ma­trix to any pin of the mi­cro­con­troller port, which makes rout­ing sur­pris­ingly easy, if a lit­tle un­con­ven­tional.

Running a larger dis­play di­rect from GPIO or­di­nar­ily suf­fers from bright­ness is­sues as the on-re­sis­tance of the out­put FETs lim­its the cur­rent. But char­lieplex­ing nec­es­sar­ily il­lu­mi­nates only one pixel at a time, which means the ef­fect of on-re­sis­tance is fac­tored out, as it dims all pix­els evenly. You can even con­trol the bright­ness of the dis­play by vary­ing the volt­age to the mi­crochip. With this in mind, we could solve the prob­lem for a con­ven­tional ma­trix by forcibly il­lu­mi­nat­ing just one pixel at a time, in­stead of a whole row. Normally this would in­crease the over­head of run­ning the dis­play, but if it’s han­dled en­tirely by DMA, that gets fac­tored out too. Pretty cool!

Again, I fol­lowed the Ten Minute Physics tu­to­ri­als for this, but in or­der to un­der­stand what’s go­ing on, I re-im­ple­mented it, try­ing as much as pos­si­ble not to look di­rectly at the source code. This was a re­ally fun jour­ney.

In the Eulerian fluid sim­u­la­tion, the move­ment of the fluid is en­acted through a process called ad­vec­tion. In FLIP, we don’t do this step, and in­stead let the par­ti­cles’ mo­tion carry the fluid about.

There are some places where not enough de­tail was given in the videos, and in those cases I did peek at the other source code. One such ex­am­ple is the par­ti­cle col­li­sions. Initially I did­n’t bother with do­ing a hash­grid, as we’re start­ing on a desk­top com­puter and that’s an op­ti­mi­sa­tion that can be left un­til later. But even with naive col­li­sions, what ac­tu­ally hap­pens when par­ti­cles col­lide? They are pushed apart with an im­pulse in­versely pro­por­tional to their dis­tance, but the sta­bil­ity of the sim­u­la­tion de­pends on get­ting that im­pulse right, much in the same way that the “overrelaxation” step of solv­ing for in­com­press­ibil­ity sort of mag­i­cally cor­rects things. It did oc­cur to me that if we’re solv­ing for in­com­press­ibil­ity, and the euler­ian grid ve­loc­i­ties are trans­ferred back to the par­ti­cles, why are ex­plicit col­li­sions be­tween par­ti­cles even needed?

I can con­firm that with­out the col­li­sions step, the whole fluid col­lapses into an over­lap­ping mess, so the step cer­tainly is­n’t su­per­flu­ous.

There’s a lot of over­head (both com­pu­ta­tional, and men­tally) to the hash­grid. In my fi­nal code I had a switch be­tween naive col­li­sions and hash­grid col­li­sions, and the dif­fer­ence is re­mark­able — even at a tiny size of 8x8, the hash­grid al­go­rithm pro­vides a sig­nif­i­cant speedup.

I cre­ated a vast num­ber of bizarre not-quite fluid sim­u­la­tions along the way. With the par­ti­cles ren­dered, most of them looked like some vari­a­tion of frogspawn. I have a col­lec­tion of odd screen­shots where I’ve for­got­ten what ex­actly was go­ing on at the time, I don’t re­mem­ber what led to this dough­nut…

The ex­am­ple from Ten Minute Physics has a small er­ror in the bound­ary con­di­tion for the left edge, mean­ing that the fluid never comes to rest. I spot­ted the mis­take (and vaguely con­sid­ered send­ing a pull re­quest but for­got). With the bound­aries work­ing cor­rectly, the whole mass of fluid and par­ti­cles still does­n’t quite stop mov­ing, as there’s no vis­cos­ity or fric­tion ex­cept for the walls, but it does sort of con­geal in a way that re­minds me of crys­tal struc­tures, of­ten with dis­lo­ca­tions or grain bound­aries be­tween groups of reg­u­larly spaced par­ti­cles. Here’s an early screen­shot:

Here we see a sort of tri­force emerg­ing from the fluid be­ing pushed into a cor­ner:

The yel­low plot is of den­sity (number of par­ti­cles over­lap­ping each grid cell) which helped cre­ate some re­ally cool vi­su­al­i­sa­tions. If we bunch up the fluid into one cor­ner, then sud­denly change grav­ity to point into the other cor­ner, we get a shock wave as it crashes against the walls.

It only lasts for a frame or two, blink and you’ll miss it, but I find that cir­cu­lar wave­front of dense par­ti­cles to be a re­ally pleas­ing ef­fect, es­pe­cially as it emerged some­what or­gan­i­cally from the rules of FLIP. You can even see the shock­wave di­rectly in the par­ti­cle po­si­tions if you freeze-frame at the right mo­ment. Here I just missed the cor­ner and two com­pet­ing shock­waves are prop­a­gat­ing:

Barely a fort­night af­ter post­ing the Simsim con­cept, I pro­duced the Simsimsim demo, as an in­ter­nal test­ing tool to de­ter­mine how low we could make the LED den­sity and still call it a fluid.

It also gives me a rough es­ti­mate of how much RAM is re­quired when we port it to bare metal. A lot of prob­lems can be solved by adding an­other lookup table, but as the di­am­e­ter in­creases the nec­es­sary RAM for all those ta­bles blows up very quickly. The STM32L432KC has 64KB of RAM — not a lot, but a di­am­e­ter of 16 needs only 26KB. It’s the kind of thing that’s def­i­nitely worth check­ing be­fore you com­mit to hard­ware though.

These demos, along with the source code to the pen­dant it­self, are not yet re­leased to the pub­lic, but I plan to do that at some point in fu­ture. I fig­ured that at least for a short while, given that I’ve put some of the pen­dants up for sale, there should be some mys­tery be­hind their ex­act op­er­a­tion, but all will be re­vealed in time.

Before even start­ing on the PCB, I had to con­vince my­self that the char­lieplexed dis­play pat­tern was go­ing to work. There are a large num­ber of char­lieplexed dis­play rout­ing op­tions out there. I found ref­er­ences to an arrange­ment called “twistyplexing” which op­ti­mises for hand-sol­dered cir­cuits. But as the in­cred­i­ble ben­e­fits of­fered by a di­ag­o­nal criss-cross started to be­come ap­par­ent, I be­gan to ques­tion my­self and fig­ured I needed to pro­duce a work­ing pro­to­type.

This small dis­play took bloody ages to put to­gether. A laser-cut piece of card is used to hold the LEDs in place, and a sou­venir from MCH2022 sup­ports the con­trivance.

I am re­minded of a pa­per I once read but can’t find for the life of me, about the dis­cov­ery of cer­tain knots. The Ashley book of knots, which serves as a kind of knot gospel, lists a huge num­ber of knots that have been in use for cen­turies. But when a math­e­mat­i­cal no­ta­tion was used to de­scribe the topol­ogy of these knots, it was dis­cov­ered that some of them are, in fact, iden­ti­cal, at least from a topo­log­i­cal point of view. They were listed sep­a­rately, be­cause with­out a rig­or­ous lan­guage to de­scribe them, men­tally reshuf­fling one knot into an­other form is ex­tremely dif­fi­cult.

I sup­pose things aren’t quite as sim­ple as all that. If you dress a knot in­cor­rectly, that can im­pact its per­for­mance, so ar­guably a reshuf­fled knot could have dif­fer­ent ap­pli­ca­tions, but you’d think vari­a­tions of the same knot would at least be grouped to­gether. The point I’m mak­ing is that the ca­pac­ity of a hu­man brain to ma­nip­u­late knot­work is sur­pris­ingly lim­ited.

We wired up our 8x9 ma­trix to the L432 dev board. The blue wire on the right is the BOOT0 pin of the chip, which is­n’t bro­ken out, but was needed way back for de­bug­ging the flash synth.

The pain of sol­der­ing this thing to­gether with enamel wire con­vinced me to quickly throw a PCB to­gether. Just a generic, di­ag­o­nally-routed, char­lieplexed LED arrange­ment to avoid hav­ing to do this again. Consider that some not-so-sub­tle fore­shad­ow­ing. But in the heat of the mo­ment, noth­ing beats a hand-wired pro­to­type to con­firm your idea with­out de­lay.

I got our FLIP sim­u­la­tion run­ning on the L432, first as a teeny 8x8 square, then as the up­per left cor­ner of an imag­i­nary fluid pen­dant.

When we think about pos­si­ble rout­ings for a char­lieplexed dis­play, and how this al­ter­na­tive rout­ing mode was hid­ing in plain sight, it makes us won­der how many other rout­ing op­tions are out there and yet to be dis­cov­ered. The temp­ta­tion is to try and use a com­puter al­go­rithm to seek out new pos­si­bil­i­ties.

Autorouters are, in gen­eral, fa­mously bad, at least out­side of ba­sic sit­u­a­tions. That may change in the near fu­ture when peo­ple no doubt throw huge neural net­works at the prob­lem. But for au­torout­ing to even be an op­tion we need to ac­cu­rately de­scribe the con­straints. Such as, cer­tain com­po­nents need to be in cer­tain places. A more im­por­tant con­straint is that the rout­ing needs to match the netlist (schematic). In our case, that schematic is not fixed: thanks to the lookup table, we don’t care which dis­play net ends up at what GPIO pin. But the search space is much big­ger than that. We want to con­strain all the LEDs to the grid of the dis­play, but we don’t care which LED goes at what po­si­tion. The lookup table for a “conventional” char­lieplexed ma­trix bares no re­sem­blance to the di­ag­o­nal criss-cross ver­sion.

Constraints re­duce the search space, and with enough con­straints, solv­ing it via brute force be­comes pos­si­ble. The con­straints for what we want to au­to­mate here are so broad, so much less con­strain­ing than in other cir­cuit de­signs, that I don’t think we’ll be au­torout­ing a bet­ter char­lieplexed dis­play any­time soon. There are even fur­ther “unconstraints” I haven’t men­tioned. In one of the fol­low-up pro­jects, space was so tight that I ended up rout­ing some of the tracks through the pads of un­used GPIO on that mi­cro­con­troller, thus break­ing one of the im­plicit con­straints that dif­fer­ent nets should­n’t touch each other. It’s triv­ial to tri-state those pins in the firmware for the chip, but know­ing that’s on the cards ex­pands our search space even fur­ther.

As a re­minder, the “conventional” char­lieplexed ma­trix fol­lows a pat­tern like this:

Adding la­bels along both edges is con­cep­tu­ally help­ful, but only one edge is strictly nec­es­sary as con­nec­tions can be made along the di­ag­o­nal. This arrange­ment needs at least one via per LED.

The di­ag­o­nal arrange­ment, af­ter squish­ing into some­thing re­sem­bling a cir­cle, looks like this:

Of a pos­si­ble 240 LEDs on 16 GPIO pins, we only need 216 to fill out our dis­play, but one edge has to be pieced to­gether from the miss­ing cor­ners. I did this some­what ar­bi­trar­ily, delet­ing un­used LEDs and shift­ing the near­est ones into place. In the process we ended up with LEDs that don’t match the pat­tern. This came back to bite me later, as a sol­der bridge on that edge is now a prob­lem, and guess where a sol­der bridge oc­curred… With a bit more thought­ful place­ment, I think we could have pro­duced the whole dis­play with end-to-end con­nec­tions mak­ing it im­mune to sol­der­ing mis­takes.

The dis­play is sup­posed to be a cir­cle, but ended up as an oc­ta­gon by co­in­ci­dence. For the Simsimsim pro­gram, I just placed LEDs based on dis­tance to the cen­tre, a cir­cle from first prin­ci­ples. It hap­pens that at 16, de­pend­ing on whether you round-down or round-to-cen­tre, you can end up with an oc­ta­gon. The op­tion to delete some of the cor­ner LEDs and make the dis­play more round was con­sid­ered, and re­jected.

The first PCB de­sign for the pen­dant, on the whole, was eas­ier than I ex­pected. We’re not boost­ing the cur­rent from the GPIO so there re­ally is­n’t all that much that needs to go on the back, and the re­duced num­ber of vias makes things so much eas­ier. We also have a good amount of space to shove those vias about as needed.

Rounding the in­ter­nal lay­ers of a PCB is both com­i­cal and, given the ex­is­tence of an ex­cel­lent track-round­ing plu­gin, com­pul­sory!

I added the panel man­u­ally, specif­i­cally be­cause I need a way to hold the board in the pick-and-place ma­chine. You can ask the board house to pan­elise it for you, but then you’re at the whims of their arrange­ment, and the com­po­nent po­si­tion file will need ad­just­ment to match. I do the mouse-bites man­u­ally, just place two drill holes on the bor­der, a ring of stop mask to mark the edge of the pcb. I keep in mind that the nor­mal end­mill for rout­ing these boards is 2mm di­am­e­ter. It’s fairly easy to round most of the in­ter­nal edges, but on the mouse-bites where we’re mat­ing to an­other curved out­line, there’s no easy way in ki­cad to fil­let those. So far, I’ve not had any com­plaints with or­der­ing boards like this, I guess it’s very clear what is de­sired.

The quar­ter-arcs be­tween each mouse-bite are a real pain to make in ki­cad. In ret­ro­spect I prob­a­bly should have drawn it in some­thing else and im­ported a DXF. Within ki­cad, I first placed the cir­cu­lar out­line of the board, then in­ter­sected it with the sup­port arms of the panel. I zoomed way in, ad­justed the end of the in­ter­sect­ing lines to meet the cir­cle ex­actly, then used those as start and end points for the arc. Quite te­dious re­ally.

For the amulet I used a bent bit of wire to press against the bat­tery. Functional but a bit lame. The cor­rect search term, if you want to find PCB-mounted gold-plated spring ter­mi­nals, is “RFI shield fin­ger”.

I later up­graded the spring tab to a larger one with more travel but did­n’t bother to up­date the 3D model.

We have plenty of room around the LiR2450 coin cell. The orig­i­nal de­sign was a bit smaller, but I em­biggened it while strug­gling to source the mag­netic charg­ing con­nec­tor. Back in 2023, a bunch of very cheap “smart rings” ap­peared on aliex­press, some of which used a 4mm mag­netic charg­ing con­nec­tor. I orig­i­nally or­dered them be­cause I wanted to steal the curved bat­tery, which un­til now was not pos­si­ble to buy in small quan­ti­ties. But the ap­peal of a mag­netic charg­ing con­nec­tor for this pen­dant was un­de­ni­able, if I could source it.

Searching for 4mm mag­netic charg­ing con­nec­tor, or vari­a­tions upon that theme, re­turned a bunch of con­nec­tors that pro­truded far fur­ther into the case than I was happy with. They were clearly in­tended as through-mount com­po­nents, whereas the ring had a to­tal thick­ness of maybe 3mm.

At the time of writ­ing, the range of con­nec­tors avail­able has in­creased dra­mat­i­cally, and the 4mm con­nec­tor I spent so long search­ing for does now ap­pear in the first page of re­sults, so per­haps it was sim­ply a newly launched prod­uct that had­n’t been in­dexed yet. The part num­ber, for fu­ture ref­er­ence, is cx-4mm-jz from WNRE.

Incidentally, the charg­ing ca­bles for the dif­fer­ent 4mm con­nec­tors are not com­pat­i­ble. Even though they have the same po­lar­ity, even the same mag­netic po­lar­ity, the ca­ble from one does­n’t stick prop­erly to the other. I think the larger con­nec­tor has a stronger mag­netic field, and a cor­re­spond­ingly weaker field on the ca­ble, pos­si­bly to re­duce the chance of it short­ing out on every­thing metal in the vicin­ity.

Huge thanks to Martin for giv­ing me free use of his met­al­work­ing equip­ment.

The video shows the process in full, es­sen­tially just bor­ing out some brass and mak­ing a few grooves. Producing a snap-back was def­i­nitely quicker than ma­chin­ing a fine thread. I did­n’t have any clear di­men­sions for how a snap-back should be built, and made some ed­u­cated guesses. The re­sult­ing snap-back test piece did snap to­gether, but was a lit­tle loose, it did­n’t hold it­self un­der ten­sion. However, the ad­di­tion of an O-ring com­pletely took up the slack, and gave us a wa­ter­tight seal in the process. The O-ring means the re­quired tol­er­ances are way more re­laxed.

Filming the process was in­ter­est­ing as Martin’s lathe is smaller than the big Colchester I’m used to. It’s a Hardinge lathe, and my mag­netic tri­pod is too large to be use­ful. Also, the head­stock has no flat sur­faces.

A cou­ple of magic arms to the res­cue.

Positioning the cam­era this way is very im­prac­ti­cal, and makes the process take so much longer. Ideally, we could have one big arm hold­ing it, but the vi­bra­tions of the lathe mo­tor mean that if it is­n’t sup­ported from mul­ti­ple po­si­tions, the footage ends up all wob­bly. If, as I hope, I get to do some more pro­jects on this lathe, I’ll have to come up with some­thing bet­ter.

In an ear­lier edit of the video, I went on a bit of a tan­gent about be­ing be­tween work­shops and even­tu­ally cut that whole bit, re­plac­ing it with some (hopefully less ag­gra­vat­ing) pi­ano mu­sic. There’s noth­ing ex­ag­ger­ated in the rant, but it’s not re­ally the feel­ing I wanted to con­vey with a nice met­al­work­ing mon­tage.

But for com­plete­ness, let me clar­ify here that the hack­space is where most of my met­al­work­ing has pre­vi­ously taken place, and in 2022 the hack­space closed as it was (not for the first time) evicted from the premises. At the time of writ­ing, af­ter much de­lay the hack­space has now moved into a new lo­ca­tion, but at the time of this pro­ject I was stumped. I even­tu­ally be­friended Martin who, as part of his busi­ness, has a well-equipped met­al­work­ing shop.

My first re­ac­tion upon us­ing this lathe is that all of the tools and equip­ment I’ve used in the past have been rub­bish. The hack­space ma­chines aren’t bad qual­ity, but they were pretty well worn even be­fore they were do­nated to the space. Much of the value in a metal work­shop is in the tool­ing, which is an­other area where the hack­space falls short.

Unlike the hack­space, Martin’s work­shop is used in a com­mer­cial set­ting, which puts the equip­ment on a com­pletely dif­fer­ent level. At the same time, my us­age of the work­shop, in ad­di­tion to be­ing lim­ited to busi­ness hours and only when the ma­chines aren’t oth­er­wise in use, is a favour, and one that, if we’re hon­est, I sus­pect was granted at least in part due to my youtube sub­scriber count. Which is fan­tas­tic in a way, fi­nally we have some gen­uine cre­den­tials, but re­ally life would be so much eas­ier if I had a de­cent work­shop of my own.

Unfortunately, I live in London and in London our houses are tiny and the rent is ex­tor­tion­ate. Even if I could af­ford the equip­ment, I sim­ply haven’t got the space for it. My Chinese mini-lathe is, quite frankly, a piece of crap. The hack­space ma­chines are a step up, but they’re still the bot­tom rung.

We lie to our­selves about weighted train­ing shoes, that learn­ing on bad equip­ment makes us bet­ter over­all… but it’s non­sense! Anyone can see that if you have ac­cess to good tools and equip­ment you can do bet­ter things, more eas­ily.

I be­lieve that ac­cess to a lathe is a fun­da­men­tal hu­man right, and the en­tire no­tion of the hack­space limp­ing along on a trickle of do­na­tions is ab­hor­rent to me. This is some­where where the gov­ern­ment should just step in. I mean, for­get so­cialised health­care, I want so­cialised work­shops! The vast ma­jor­ity of peo­ple can’t jus­tify own­ing big milling ma­chines and lathes, even if they can af­ford it, but at those times when they are needed, they should be there, for all.

Anyway, that all seemed a bit too whiny and po­lit­i­cal ver­sus the type of con­tent I want to pro­duce, so to the writ­ten word it was del­e­gated, and let’s quickly move on with the fun stuff.

Partway through the met­al­work­ing I de­cided to pro­duce a sec­ond pen­dant with a watch glass (“crystal”) cov­er­ing the dis­play. Watch glasses are avail­able in al­most all di­am­e­ters, with plenty of op­tions for thick­ness, whether it’s flat or curved, etc. Something I strug­gled to find any info about is the di­men­sional tol­er­ances, or in fact any di­men­sions at all, for the metal part it’s pressed into.

A small test piece was cut. I chose a glass di­am­e­ter of 27.5mm. I imag­ine that some­where out there are some in­struc­tions on how to build watches to ac­com­mo­date such glasses, but here all I could find was about re­plac­ing the glass dur­ing a watch re­pair. Between the glass and the metal sits a gas­ket. With a gas­ket thick­ness of 0.45mm, our to­tal di­am­e­ter would be 28.4mm.

The glass did in fact press in beau­ti­fully, with just the right amount of force. The test di­am­e­ter seemed per­fect. The cracked glass did­n’t hap­pen un­til later, when I be­came over­con­fi­dent in how easy it was to press the glass in with­out the spe­cial tool. Sometimes it’s good to test these lim­its to cal­i­brate our judge­ment.

Anyway, those glass flats are very cheap, so noth­ing lost re­ally, and I pur­sued the sec­ond pen­dant case with aban­don. When it came to bore the 28.4mm re­cess on the front, or­di­nar­ily it would be an or­deal to re-chuck the part af­ter part­ing off. But Martin’s ex­ten­sive col­lec­tion of soft col­lets turned it into a non-is­sue. The idea with soft col­lets is that you ma­chine them to cus­tom fit the part you’re work­ing on, but ap­par­ently by co­in­ci­dence, sev­eral of the soft col­lets on the shelf matched my 28mm in­ter­nal di­am­e­ter al­ready.

The milling process was un­event­ful and I think I was overly cau­tious. For sub­se­quent pen­dants, I raced through this part in no time. The plas­tic ar­bour in a square col­let-holder worked very well (and was reusable for later pen­dants).

It’s a pretty pleas­ing lit­tle slot. The O-ring adds some de­f­i­n­i­tion to the snap-back, I like the look of it, turn­ing it into a fea­ture. The al­ter­na­tive would be try­ing to hide it, but that would need much tighter tol­er­ances, and also make open­ing it more dif­fi­cult as there’d be no room to in­sert a case knife.

The jump rings were sol­dered us­ing “hard” brass sol­der. Jewellers use mul­ti­ple dif­fer­ent grades of sol­der, al­though even “soft” jew­ellers’ sol­der is harder than nor­mal elec­tron­ics sol­der. The idea is that you can sol­der the first joints with the hard­est (highest melt­ing point) sol­der, and sub­se­quent joints with softer sol­der, with­out the risk of the ear­lier joints melt­ing.

My gut feel­ing is that I prob­a­bly could have done both joints with the hard­est sol­der, but for all of the pen­dants so far I used soft sol­der for the sec­ond joint. This is elec­tron­ics sol­der, and fol­low­ing some dis­parag­ing com­ments on a pre­vi­ous pro­ject, I used lead-free sol­der. This was meant to be a sub­tle joke, as the brass al­loy has a sig­nif­i­cant quan­tity of lead in it any­way.

I would later re­alise that this lead-free sol­der is com­pletely in­com­pat­i­ble with gold plat­ing, it just bounces off the sur­face. I don’t think it’s a bad look as such, but in the past, the sil­ver-lead sol­der was able to ac­cept elec­tro­plat­ing with­out any trou­ble. For the first pen­dants I made the fil­let as tiny as pos­si­ble, but later be­came con­cerned that there might be gaps that would com­pro­mise the over­all seal, and made the fil­lets more promi­nent.

I brush plated these parts with gold. For the first two pen­dants I re­ally did a poor job of prepar­ing the sur­face, and the tool­marks were em­bossed by the plat­ing process. I was some­what in a hurry to fin­ish as the PCBs were due to ar­rive very shortly.

Sometimes you get these brown splodges dur­ing plat­ing. I’m not en­tirely sure what causes it.

An ex­pected part of the plat­ing process is to pol­ish the parts af­ter­ward with a very fine abra­sive (jeweller’s rouge). This re­moves such marks, and gives us a per­fect fin­ish, or at least it would be if not for the tool marks.

We def­i­nitely rushed this part, but who cares, the cir­cuit boards had ar­rived. First off the pick-and-place ma­chine:

More sol­der bridges than I’d an­tic­i­pated. I was us­ing a dif­fer­ent grade of sol­der paste than usual (just to ex­per­i­ment) and I think it would have ben­e­fited from smaller aper­tures in the sten­cil. The 0402 LEDs are quite small, and touch­ing up bridges be­tween them is quite tricky. As men­tioned, the ma­jor­ity of them have no im­pact on the per­for­mance as end-to-end LEDs are mostly the same net, all bar the few on the edge where I repo­si­tioned them. However, it de­tracts from the look, so I felt com­pelled to re­move such mis­takes from be­ing on show.

One over­sight is that I did­n’t break out the re­set pin of the mi­cro­con­troller. The dis­play uses all of port A, and the SWDIO/SWCLK de­bug lines also ex­ist on port A. For gen­eral use, that’s not go­ing to mat­ter as we can dis­able them in soft­ware. But for de­vel­op­ment, this makes it im­pos­si­ble to flash new soft­ware onto the board. The trick is to re­set the chip right be­fore you pro­gram it, but here that means adding a bodge wire.

The first cir­cuit board be­came my dev board. At this stage I still had­n’t sourced the de­sired mag­net con­nec­tor, but the older mag­net con­nec­tor proved the charg­ing cir­cuit worked. With a 3D printed coin-cell holder, I was able to power the dev board from the bat­tery too.

The bus keeper on the ac­celerom­e­ter’s in­ter­rupt line caused a few (of sev­eral) dis­play glitches. A re­sis­tor was bodged onto the track which helped a bit.

Ultimately I added a diode there, which com­pletely fixed the is­sue. At one point I re­placed that re­sis­tor with an LED, which had the added bonus of let­ting me see the sig­nal as I nudged the board.

I gave a quick mon­tage of some of the other bodges I ap­plied to the cir­cuit in the video, mostly try­ing to en­sure that the cir­cuit could­n’t be tricked into a soft-lock. The bat­tery un­der­volt­age de­tec­tion be­ing in soft­ware made our cir­cuit sim­pler, but made me suf­fi­ciently un­easy that the next re­vi­sion of the PCB did it in hard­ware. The beauty of the sim­ple case de­sign, with no but­tons and no easy way to open it, comes with a cer­tain amount of para­noia. I re­ally wanted a way to re­set the chip if needed, so the cir­cuit lis­ten­ing for the charg­ing con­nec­tor was con­structed.

Some datasheets/​de­vboards sug­gest al­ways hav­ing a 100n cap on the re­set line of an STM32 chip. I’m not sure whether it’s needed, but I think I’d rather have a pre­dictable, rea­son­able ca­pac­i­tance on the pin than a small amount of un­known ca­pac­i­tance. The pulse of con­nect­ing the charger passes through a small ca­pac­i­tor on the base of an NPN tran­sis­tor, which am­pli­fies the cur­rent to pull down the re­set pin. Not shown in the pic­ture above, I added a 100n cap to the re­set line as rec­om­mended, and a pull­down re­sis­tor on the tran­sis­tor side of the sig­nal cap, to try and max­imise the pulse volt­age when the charger con­nects.

It’s very easy to short the charg­ing con­nec­tor as it ap­proaches, and in that case the poly­fuse heats up and drops the out­put volt­age. Even though it starts charg­ing as the volt­age re­turns, the slowly ris­ing power is not enough to trig­ger the re­set cir­cuit. I de­cided that this is fine, be­cause if you re­ally need to re­set it, just con­nect the mag­net end of the ca­ble first, then plug in the USB. And, in the re­design of the PCB, the hard­ware un­der­volt­age de­tec­tion means that re­set­ting the chip should never be a re­quire­ment, this cir­cuit is just there to pla­cate my para­noia.

Jumping ahead some­what, we epox­ied the mag­net con­nec­tor in place, and also filled the lit­tle hole for the charg­ing LED with epoxy too. Under the mi­cro­scope, there’s a tiny menis­cus of resin there.

The cir­cuit board is sol­dered to the case in a few strate­gic lo­ca­tions, for me­chan­i­cal sta­bil­ity, and to make elec­tri­cal con­tact with the bat­tery ground.

After a few as­sem­blies and dis­as­sem­blies, the foam pads were cor­rected, the spring pin was re­placed with a taller one, the over­all dis­play bright­ness was re­duced, and the var­i­ous other dis­play glitches were cor­rected.

It is some­what scary to close up the back and have no ac­cess to any­thing ex­cept that charge con­nec­tor. The only in­put to the de­vice is the ac­celerom­e­ter data. I had planned to ac­ti­vate the “deep sleep” mode by spin­ning the pen­dant on the end of a chain, which would be very easy to de­tect com­pared to most ges­tures, just the Y co­or­di­nate be­yond a thresh­old for some amount of time. It would be swish if that was how to wake it up as well. Unfortunately, that would mean more com­plex wakeup logic. We’d need to wake up, then check the con­di­tion and go back to sleep if it was­n’t met.

But I was quite pleased to think of sim­ply in­creas­ing the thresh­old of the ac­celerom­e­ter’s move­ment de­tec­tion in­ter­rupt. By set­ting it to 6g, it will be un­likely to wake up ac­ci­den­tally, but it’s easy enough to shake it back to life. Shake-to-wake. This is a great so­lu­tion be­cause it uses no more power than the reg­u­lar sleep.

But we can do bet­ter!

Before as­sem­bling the sec­ond pen­dant with the promised watch glass, I wanted to re­vise the PCB, in­cor­po­rat­ing our re­set cir­cuit, the wakeup line diode, and the hard­ware su­per­vi­sor chip.

In the same se­ries as the in­cred­i­ble TPS7A02, the TPS3839 sup­ply volt­age mon­i­tor chip comes with some sim­i­larly im­pres­sive spec­i­fi­ca­tions, and in the same tiny pack­age. Its sup­ply cur­rent of 150nA may sound a lot com­pared to the 25nA of the reg­u­la­tor, but then you take a step back and re­alise both of these num­bers are ba­si­cally zero. The coin cell has a ca­pac­ity of 120mAh, so even a 1000nA would take over 13 years to drain it. And on such timescales ex­trap­o­lat­ing does­n’t re­ally make any sense, due to self-dis­charge, non­lin­ear ef­fects and so on.

That reg­u­la­tor is es­sen­tially an ideal com­po­nent, both the dropout and the qui­es­cent cur­rent are in­fin­i­tes­i­mally small. But it is quite a bit more ex­pen­sive than a nor­mal reg­u­la­tor, by which I mean it costs maybe $1.

I chose the su­per­vi­sor to cut off at 3.08V, which is con­ser­v­a­tive enough that if the bat­tery gets this low, we still have enough ca­pac­ity to sit on a shelf for a few years with­out hurt­ing the chem­istry. A lot of lithium pro­tec­tion cir­cuits cut out at 2.5V, in fact you might be won­der­ing why I’m not us­ing an off-the-shelf Li-ion pro­tec­tion cir­cuit, of which there are many. It’s gen­er­ally ac­cepted that you should­n’t let the open-cir­cuit volt­age of a lithium bat­tery drop be­low 3.0V, but in use, if there’s a load on the bat­tery, the volt­age at the ter­mi­nals is lower than the open-cir­cuit volt­age. For this rea­son, most bat­ter­ies set their un­der­volt­age pro­tec­tion at 2.5V, to stop it kick­ing in too soon when the bat­tery has a heavy load.

In the case of our ~10mA load, and my rea­son­ing above about not want­ing to run it to­tally flat, a 3.08V thresh­old makes per­fect sense. Protection cir­cuits usu­ally add a cou­ple of low-volt­age mos­fets in the path to the bat­tery, which them­selves will eat some of the power. My plan was to con­nect the su­per­vi­sor chip to the en­able pin of the reg­u­la­tor. I had tried ear­lier to pulse the en­able pin of the reg­u­la­tor when the charg­ing ca­ble is con­nected with no suc­cess. I later fig­ured out that there are two ver­sions of the TPS7A02: the TPS7A0233DQNR, and the TPS7A0233PDQNR. The P ver­sion has an “active dis­charge” cir­cuit for when the reg­u­la­tor is dis­abled. The non-P ver­sion sim­ply lets the mi­crochip, and its power sup­ply ca­pac­i­tors, sit there un­til they run down. If the mi­cro­con­troller is in deep sleep, that could take a sig­nif­i­cant time.

I might have been cre­at­ing an ar­ti­fi­cially fast run­down as I var­ied the volt­age to the cir­cuit, but I found that the non-P ver­sion of the reg­u­la­tor I was orig­i­nally us­ing caused a num­ber of prob­lems when the su­per­vi­sor kicked in. There may or may not be some brownout mon­i­tor­ing cir­cuitry on the STM32, but it was def­i­nitely pos­si­ble to latch the cir­cuit into an­other soft­lock by wav­ing the volt­age around the thresh­old. I or­dered the P-version of the reg­u­la­tor, swapped it over with the heat gun, and the prob­lems went away.

The two X2SON parts were placed prob­a­bly too close to­gether, that made re­work more dif­fi­cult than it needed to be.

It’s not like I was try­ing to mass pro­duce these things, but as­sem­bling sev­eral at once is more ef­fi­cient on the odd chance that a third or fourth pen­dant might hap­pen.

Discussions on X, LinkedIn, YouTube. Also: Meet AI Engineers in per­son! Applications clos­ing soon for at­tend­ing and spon­sor­ing AI Engineer Summit NYC, Feb 20-21.

The picks from all the speak­ers in our Best of 2024 se­ries catches you up for 2024, but since we wrote about run­ning Paper Clubs, we’ve been asked many times for a read­ing list to rec­om­mend for those start­ing from scratch at work or with friends. We started with the 2023 a16z Canon, but it needs a 2025 up­date and a prac­ti­cal fo­cus.

Here we cu­rate “required reads” for the AI en­gi­neer. Our de­sign goals are:

* tell you why this pa­per mat­ters in­stead of just name drop with­out help­ful con­text

* be very prac­ti­cal for the AI Engineer; no time wasted on Attention is All You Need, bc 1) every­one else al­ready starts there, 2) most won’t re­ally need it at work

We ended up pick­ing 5 “papers” per sec­tion for:

You can both use and learn a lot from other LLMs, this is a vast topic.

We cov­ered many of these in Benchmarks 101 and Benchmarks 201, while our Carlini, LMArena, and Braintrust episodes cov­ered pri­vate, arena, and prod­uct evals (read LLM-as-Judge and the Applied LLMs es­say). Benchmarks are linked to Datasets.

Note: The GPT3 pa­per (“Language Models are Few-Shot Learners”) should al­ready have in­tro­duced In-Context Learning (ICL) - a close cousin of prompt­ing. We also con­sider prompt in­jec­tions re­quired knowl­edge — Lilian Weng, Simon W.

Section 3 is one area where read­ing dis­parate pa­pers may not be as use­ful as hav­ing more prac­ti­cal guides - we rec­om­mend Lilian Weng, Eugene Yan, and Anthropic’s Prompt Engineering Tutorial and AI Engineer Workshop.

RAG is the bread and but­ter of AI Engineering at work in 2024, so there are a LOT of in­dus­try re­sources and prac­ti­cal ex­pe­ri­ence you will be ex­pected to have. LlamaIndex (course) and LangChain (video) have per­haps in­vested the most in ed­u­ca­tional re­sources. You should also be fa­mil­iar with the peren­nial RAG vs Long Context de­bate.

We cov­ered many of the 2024 SOTA agent de­signs at NeurIPS, and you can find more read­ings in the UC Berkeley LLM Agents MOOC. Note that we skipped bikeshed­ding agent de­f­i­n­i­tions, but if you re­ally need one, you could use mine.

CodeGen is an­other field where much of the fron­tier has moved from re­search to in­dus­try and prac­ti­cal en­gi­neer­ing ad­vice on code­gen and code agents like Devin are only found in in­dus­try blog­posts and talks rather than re­search pa­pers.

Much fron­tier VLM work these days is no longer pub­lished (the last we re­ally got was GPT4V sys­tem card and de­riv­a­tive pa­pers). We rec­om­mend hav­ing work­ing ex­pe­ri­ence with vi­sion ca­pa­bil­i­ties of 4o (including fine­tun­ing 4o vi­sion), Claude 3.5 Sonnet/Haiku, Gemini 2.0 Flash, and o1. Others: Pixtral, Llama 3.2, Moondream, QVQ.

We do rec­om­mend di­ver­si­fy­ing from the big labs here for now - try Daily, Livekit, Vapi, Assembly, Deepgram, Fireworks, Cartesia, Elevenlabs etc. See the State of Voice 2024. While NotebookLM’s voice model is not pub­lic, we got the deep­est de­scrip­tion of the mod­el­ing process that we know of.

With Gemini 2.0 also be­ing na­tively voice and vi­sion mul­ti­modal, the Voice and Vision modal­i­ties are on a clear path to merg­ing in 2025 and be­yond.

We also highly rec­om­mend fa­mil­iar­ity with ComfyUI (upcoming episode). Text Diffusion, Music Diffusion, and au­tore­gres­sive im­age gen­er­a­tion are niche but ris­ing.

We rec­om­mend go­ing thru the Unsloth note­books and HuggingFace’s How to fine-tune open LLMs for more on the full process. This is ob­vi­ously an end­lessly deep rab­bit hole that, at the ex­treme, over­laps with the Research Scientist track.

This list will seem in­tim­i­dat­ing and you will fall off the wagon a few times. Just get back on it. We’ll up­date with more thru 2025 to keep it cur­rent. You can make up your own ap­proach but you can use our How To Read Papers In An Hour as a guide if that helps. Many folks also chimed in with ad­vice here.

* If you’re look­ing to go thru this with new friends, reader Krispin has started a dis­cord here: https://​app.dis­cuna.com/​in­vite/​ai_en­gi­neer and you’re also ofc wel­come to join the Latent Space dis­cord.

* Reader Niels has started a notes blog where he is pulling out high­lights: https://​niels-ole.com/​2025/​01/​05/​notes-on-the-2025-ai-en­gi­neer-read­ing-list

Did we miss any­thing ob­vi­ous? It’s quite pos­si­ble. Please com­ment be­low and we’ll up­date with credit to help the com­mu­nity.

Thanks to Eugene Yan and Vibhu Sapra for great sug­ges­tions to this list.

Data in­form­ing the GDD mod­el­ing es­ti­mates for this study, in­clud­ing from LMICs (low- and mid­dle-in­come coun­tries), were col­lected be­tween 1980 and 2018 from GDD con­sor­tium mem­bers and pub­licly avail­able sources in the form of di­etary in­take sur­veys. If na­tion­ally rep­re­sen­ta­tive sur­veys were not avail­able for a coun­try, we also con­sid­ered na­tional sur­veys with­out rep­re­sen­ta­tive sam­pling, fol­lowed by re­gional, ur­ban or rural sur­veys, and fi­nally large lo­cal co­horts, pro­vided that se­lec­tion and mea­sure­ment bi­ases were not ap­par­ent lim­i­ta­tions (for ex­am­ple, ex­clud­ing stud­ies fo­cused on a se­lected pop­u­la­tion with a spe­cific dis­ease, a cer­tain pro­fes­sion or fol­low­ing a par­tic­u­lar di­etary pat­tern). For coun­tries with no sur­veys iden­ti­fied, other sources of po­ten­tial data were con­sid­ered, in­clud­ing the WHO Infobase, the STEP data­base and house­hold bud­get sur­vey data. As of August 2021, we iden­ti­fied and re­trieved 1,634 el­i­gi­ble sur­vey years of data from pub­lic and pri­vate sources. Of these, 1,224 were checked, stan­dard­ized and in­cluded in the GDD model, in­clud­ing 450 sur­veys in­form­ing SSB in­take es­ti­mates12.

Most sur­veys iden­ti­fied were pri­vately held or, if pub­lic, not avail­able in rel­e­vant for­mat for GDD mod­el­ing (for ex­am­ple, not jointly strat­i­fied by age, sex, ed­u­ca­tion, and ur­ban or rural sta­tus). We thus re­lied al­most en­tirely on di­rect con­sor­tium mem­ber con­tacts for each sur­vey to pro­vide us with ex­po­sure data di­rectly. Roles and re­spon­si­bil­i­ties of GDD con­sor­tium mem­bers were de­ter­mined and agreed upon be­fore data shar­ing as part of a stan­dard­ized data shar­ing agree­ment. The draft man­u­script was shared with all GDD con­sor­tium mem­bers be­fore sub­mis­sion for peer re­view, and all mem­bers are in­cluded as coau­thors of this work. We en­dorse the Nature Portfolio jour­nals’ guid­ance on LMIC au­thor­ship and in­clu­sion and are com­mit­ted to the in­clu­sion of re­searchers from LMICs in pub­li­ca­tions from the GDD. We share the GDD data with the en­tire con­sor­tium, en­cour­age au­thors from LMICs to take the lead on analy­ses and pa­pers, and can pro­vide tech­ni­cal and writ­ing sup­port to LMIC au­thors. For more de­tails on the col­lab­o­ra­tive GDD data col­lec­tion process, please visit our web­site at https://​www.glob­aldietary­data­base.org/​meth­ods/​sum­mary-meth­ods-and-data-col­lec­tion.

This re­search is lo­cally rel­e­vant to all coun­tries in­cluded, given that it dis­ag­gre­gates find­ings na­tion­ally and sub­na­tion­ally by key de­mo­graphic fac­tors such as age, sex, ed­u­ca­tion level and ur­ban­ic­ity, pro­vid­ing de­ci­sion-mak­ers with the CVD and di­a­betes risk as­so­ci­ated with SSB in­takes over time.

This mod­el­ing in­ves­ti­ga­tion was ex­empt from eth­i­cal re­view board ap­proval be­cause it was based on pub­lished data and na­tion­ally rep­re­sen­ta­tive, de-iden­ti­fied datasets, with­out per­son­ally iden­ti­fi­able in­for­ma­tion. Individual sur­veys un­der­went eth­i­cal re­view board ap­proval re­quired for the ap­plic­a­ble lo­cal con­text.

A CRA mod­el14 es­ti­mated the num­bers, pro­por­tions and un­cer­tainty of global T2D and CVD in­ci­dence, DALYs and mor­tal­ity at­trib­ut­able to in­take of SSBs among adults aged 20+ years. Importantly, the CRA frame­work does not use eco­logic cor­re­la­tions to es­ti­mate risk, but in­cor­po­rates in­de­pen­dently de­rived in­put pa­ra­me­ters and their un­cer­tain­ties on so­ciode­mo­graph­ics, pop­u­la­tion size, risk fac­tor (that is, SSBs) their mul­ti­vari­able-ad­justed es­ti­mated eti­o­logic ef­fects on dis­ease based on ex­ter­nal stud­ies, and back­ground dis­ease in­ci­dence, mor­tal­ity and DALYs14. These pa­ra­me­ters are en­tered into the model to es­ti­mate the dis­ease bur­dens and their un­cer­tain­ties. Specifically for this in­ves­ti­ga­tion, we lever­aged in­put data and cor­re­spond­ing un­cer­tainty in 184 coun­tries in­clud­ing (1) pop­u­la­tion SSB in­take dis­tri­b­u­tions based on in­di­vid­ual-level sur­vey data from the GDD (https://​www.glob­aldietary­data­base.org/)7,12,13; (2) op­ti­mal SSB in­take lev­els from pre­vi­ous analy­ses67; (3) di­rect age-ad­justed eti­o­logic ef­fects of SSBs on T2D, is­chemic heart dis­ease and is­chemic stroke ad­justed for BMI2,68,69,70, and of weight gain on T2D15, is­chemic heart dis­ease16 and is­chemic stroke15 from pre­vi­ous meta-analy­ses and pooled analy­ses of prospec­tive co­horts, as well as lin­ear, BMI-stratified ef­fects of SSBs on weight gain or loss17; (4) pop­u­la­tion over­weight (BMI ≥ 25 kg m−2) and un­der­weight (BMI −2) dis­tri­b­u­tions from the (non-communicable dis­ease) NCD Risk Factor Collaboration (NCD-RisC)71; (5) to­tal T2D, is­chemic heart dis­ease, and is­chemic stroke in­ci­dence, DALYs and mor­tal­ity es­ti­mate dis­tri­b­u­tions from the GBD study72,73; and (6) pop­u­la­tion de­mo­graphic data from the United Nations Population Division74,75 and the Barro and Lee Educational Attainment Dataset 201376, as pre­vi­ously re­port­ed31 (Supplementary Table 7).

Bias and re­li­a­bil­ity were ad­dressed in each of the in­de­pen­dent data sources used in our model. The GDD se­lected na­tional and sub­na­tional di­etary sur­veys with­out ap­par­ent mea­sure­ment or se­lec­tion bi­as­es7, and lever­aged a Bayesian model to in­cor­po­rate dif­fer­ences in data com­pa­ra­bil­ity and sam­pling un­cer­tainty. In GBD, bias ad­just­ment of un­der­ly­ing rates of T2D and CVD not specif­i­cally meet­ing the gold-stan­dard de­f­i­n­i­tion of these causes was done us­ing net­work meta-re­gres­sion be­fore es­ti­ma­tion in DisMod, while im­plau­si­ble or un­spec­i­fied causes of death were re­dis­trib­uted to valid un­der­ly­ing causes of death us­ing re­clas­si­fi­ca­tion al­go­rithm­s73,77. Etiologic ef­fects were ob­tained from pub­lished meta-analy­ses or pooled analy­ses of prospec­tive co­horts and ran­dom­ized con­trol tri­als in­clud­ing mul­ti­vari­able ad­just­ment for age, sex, BMI and other risk fac­tors to re­duce bias from con­found­ing2,68,69,70. Studies with in­creased risk of bias such as ret­ro­spec­tive or cross-sec­tional stud­ies were ex­clud­ed2. Underlying adi­pos­ity rates were ob­tained from the NCD-RisC, which used na­tional or sub­na­tional sur­veys that col­lected mea­sured height and weight data to avoid bias in self-re­ported data71.

The GBD study uses a dif­fer­ent ap­proach to di­etary as­sess­ment, pri­mar­ily re­ly­ing on ad­justed United Nations (UN) and FAO na­tional per capita avail­abil­ity of sugar as pri­mary data to es­ti­mate SSBs, with a lim­ited set of in­di­vid­ual-level di­etary sur­veys (N = 44). In com­par­i­son, the GDD uses a much more com­pre­hen­sive data­base of largely in­di­vid­ual-level di­etary sur­veys to es­ti­mate SSB in­take (N = 450), with other data (such as UN FAO sugar) used as co­vari­ates rather than as pri­mary data. Thus, in ad­di­tion to novel strat­i­fi­ca­tion by ed­u­ca­tional level and area of res­i­dence, the GDD di­etary es­ti­mates may be more valid and in­for­ma­tive. Our in­ves­ti­ga­tion lever­ages pub­lished diet–dis­ease eti­o­logic ef­fects from ex­ten­sive meta-analy­ses iden­ti­fied through re­views con­ducted by our team, in­cludes both di­rect and BMI-mediated ef­fects, and in­cor­po­rates new data on preva­lence of over­weight and obe­sity from the NCD-RisC. Our study also es­ti­mates in­ci­dent cases, which is not a mea­sure re­ported in pre­vi­ous global stud­ies.

Compared with our pre­vi­ous 2010 es­ti­mates3, our pre­sent in­ves­ti­ga­tion in­cludes ma­jor ex­pan­sion of in­di­vid­ual-level di­etary sur­veys and global cov­er­age through 2018; up­dated mod­el­ing meth­ods, co­vari­ates and val­i­da­tion to im­prove es­ti­mates of stra­tum-spe­cific mean in­takes and un­cer­tainty; in­clu­sion of up­dated di­etary and dis­ease data that are jointly strat­i­fied sub­na­tion­ally by age, sex, ed­u­ca­tion level, and ur­ban or rural res­i­dence; and up­dated SSB eti­o­logic ef­fect es­ti­mates on T2D, is­chemic stroke and is­chemic heart dis­ease. This pre­sent analy­sis fo­cused on adults aged 20+ years given the low rates of T2D and CVD glob­ally at younger ages.

The GDD sys­tem­at­i­cally searched for and com­piled rep­re­sen­ta­tive data on in­di­vid­ual-level di­etary in­takes from na­tional sur­veys and sub­na­tional sur­veys7,12. The fi­nal GDD model in­cor­po­rated 1,224 di­etary sur­veys rep­re­sent­ing 185 coun­tries from 7 world re­gions and 99.0% of the global pop­u­la­tion in 2020. Of these, 450 sur­veys re­ported data on SSBs, to­tal­ing 2.9 mil­lion in­di­vid­u­als from 118 coun­tries rep­re­sent­ing 86.8% of the global pop­u­la­tion. Most sur­veys were na­tion­ally or sub­na­tion­ally rep­re­sen­ta­tive (94.2%), col­lected at the in­di­vid­ual level (84.7%), and in­cluded es­ti­mates in both ur­ban and rural area of res­i­dence (61.6%). Further de­tails on char­ac­ter­is­tics of sur­veys with data on SSBs, in­clud­ing avail­abil­ity of sur­veys per world re­gion, are avail­able in Supplementary Table 1. The world re­gion clas­si­fi­ca­tion used in our study was based on group­ings that are likely to have con­sis­tent ex­po­sures to dis­ease risk and rates of dis­ease out­comes, and this or sim­i­lar clas­si­fi­ca­tions have been pre­vi­ously used by our team and oth­er­s73. Countries in­cluded in each world re­gion are listed in Supplementary Table 2. Global, re­gional and na­tional es­ti­mates among the 30 most pop­u­lous coun­tries, by pop­u­la­tion char­ac­ter­is­tics in 2020, are avail­able in Supplementary Tables 3 and 4.

SSBs were de­fined as any bev­er­ages with added sug­ars and ≥50 kcal per 8 oz serv­ing, in­clud­ing com­mer­cial or home­made bev­er­ages, soft drinks, en­ergy drinks, fruit drinks, punch, lemon­ade and aguas fres­cas. This de­f­i­n­i­tion ex­cluded 100% fruit and veg­etable juices, noncaloric ar­ti­fi­cially sweet­ened drinks and sweet­ened milk. All in­cluded sur­veys used this de­f­i­n­i­tion. We used an av­er­age sugar con­tent per SSB serv­ing, an as­sump­tion that prob­a­bly has lit­tle in­flu­ence on large-scale de­mo­graphic es­ti­mates such as these but could be a prob­lem for more fo­cused lo­cal stud­ies. Home-sweetened teas and cof­fees (which of­ten would have less than 50 kcal per serv­ing) were not ex­plic­itly ex­cluded from the SSB de­f­i­n­i­tion at the time of data col­lec­tion, but to­tal tea and cof­fee in­take were sep­a­rately col­lected in the di­etary sur­veys and by the GDD as sep­a­rate vari­ables. Compared with soda and other in­dus­trial SSBs, 100% fruit juices and sugar-sweet­ened milk, cof­fee and tea have shown in­con­sis­tent ev­i­dence for health ef­fects, and were there­fore ex­cluded from our de­f­i­n­i­tion of SSBs2,9. Differences in health ef­fects may re­late to ad­di­tional nu­tri­ents in those drinks, such as cal­cium, vi­t­a­min D, fats, and pro­tein in milk, caf­feine and polyphe­nols in cof­fee and tea, and fiber and vi­t­a­mins in 100% juice, or to dif­fer­ences in the ra­pid­ity of con­sump­tion and/​or drink­ing pat­terns of these bev­er­ages. Notably, each of these other bev­er­ages is also gen­er­ally ex­cluded in pol­icy and sur­veil­lance ef­forts around SSBs12. At high in­takes, al­co­holic bev­er­ages have been as­so­ci­ated with T2D and CVD in prospec­tive co­horts and genome-wide as­so­ci­a­tion stud­ies78,79. However, the ef­fect of al­co­holic bev­er­ages on T2D and CVD dif­fers from the ef­fect of SSBs on these dis­eases, and thus, al­co­hol and SSB should be an­a­lyzed sep­a­rate­ly2,79,80. Moreover, the ex­clu­sion of al­co­holic bev­er­ages en­sures com­pa­ra­bil­ity across di­verse pop­u­la­tions, given vari­a­tions in al­co­hol con­sump­tion due to re­li­gious and cul­tural fac­tors81. Regulatory short­com­ings in la­bel­ing 100% fruit and veg­etable juices may have led to un­der­es­ti­ma­tions in SSB in­take and at­trib­ut­able bur­dens for cer­tain pop­u­la­tion­s82,83.

For our pre­sent analy­sis, we up­dated SSB in­take es­ti­mates for 2020 us­ing sim­i­lar method­ol­ogy as pre­vi­ously re­port­ed12, but with up­dated food avail­abil­ity data re­leased by FAO for 2014–2020 as co­vari­ates. Because FAO up­dated its method­ol­ogy for these new es­ti­mates, the FAO es­ti­mates from this pe­riod ver­sus their es­ti­mates from ear­lier years are not di­rectly com­pa­ra­ble (for ex­am­ple, a ‘step change’ in FAO es­ti­mates was noted com­par­ing 2013 ver­sus 2014 data for most coun­tries). To ac­count for this and re­tain the rel­a­tive rank­ing be­tween na­tions, we cal­cu­lated a na­tion-spe­cific ad­just­ment fac­tor for each FAO co­vari­ate, based on the ra­tio of that na­tion’s 2013 ver­sus 2014 data, and ap­plied this to each na­tion’s FAO es­ti­mates from 2014 to 2020.

A Bayesian model with a nested hi­er­ar­chi­cal struc­ture (with ran­dom ef­fects by coun­try and re­gion) was used to es­ti­mate the mean con­sump­tion level of SSBs and its sta­tis­ti­cal un­cer­tainty for each of 264 pop­u­la­tion strata across 185 coun­tries from 1990 through 2020, in­cor­po­rat­ing and ad­dress­ing dif­fer­ences in data com­pa­ra­bil­ity and sam­pling un­cer­tain­ty12,84. The model then es­ti­mated in­takes jointly strat­i­fied by age (22 age cat­e­gories from 0 to 6 months through 95+ years), sex (female, male), ed­u­ca­tion (≤6 years of ed­u­ca­tion, >6 years to 12 years, >12 years) and ur­ban­ic­ity (urban, rural). Although this analy­sis fo­cuses only on adults aged 20+ years, the model used all age data to gen­er­ate the strata es­ti­mates.

Of the 188 coun­tries with sur­vey data, 3 were dropped from the GDD es­ti­ma­tion model ow­ing to un­avail­abil­ity of FAO food avail­abil­ity data (Andorra, Democratic People’s Republic of Korea and Somalia), an im­por­tant co­vari­ate in the es­ti­ma­tion model. Uncertainty of each stra­tum-spe­cific es­ti­mate was quan­ti­fied us­ing 4,000 it­er­a­tions to de­ter­mine pos­te­rior dis­tri­b­u­tions of mean in­take jointly by coun­try, year and so­ciode­mo­graphic sub­group. The me­dian in­take and the 95% UI for each stra­tum were com­puted at the 50th, 2.5th and 97.5th per­centiles of the 4,000 draws, re­spec­tively.

Global, re­gional, na­tional and within-coun­try pop­u­la­tion sub­group in­takes of SSBs and their un­cer­tainty were cal­cu­lated as pop­u­la­tion-weighted av­er­ages us­ing all 4,000 pos­te­rior es­ti­mates for each of the 264 de­mo­graphic strata in each coun­try–year. Population weights for each year were de­rived from the United Nations Population Division74,75, sup­ple­mented with data for ed­u­ca­tion and ur­ban or rural sta­tus from a pre­vi­ous study85. Intakes were cal­cu­lated as 8 oz (248 g) serv­ings per week. For our pre­sent analy­sis, GDD SSB es­ti­mates were col­lapsed for adults aged 85+ years us­ing the 4,000 sim­u­la­tions cor­re­spond­ing to the stra­tum-level in­take data de­rived from the Bayesian model. In this study, re­gres­sion-based meth­ods were used to es­ti­mate the stan­dard de­vi­a­tion cor­re­spond­ing to each es­ti­mated, stra­tum-spe­cific mean from the di­etary sur­vey in­put data. These mean–stan­dard de­vi­a­tion pairs were then used to gen­er­ate gamma dis­tri­b­u­tion pa­ra­me­ters for usual di­etary in­take as de­tailed in the fol­low­ing sec­tion.

Dietary in­takes can­not be neg­a­tive, and the usual in­take dis­tri­b­u­tions tend to be skewed to the right86,87. Gamma dis­tri­b­u­tions were shown to be more ap­pro­pri­ate than nor­mal dis­tri­b­u­tions for SSBs based on the analy­sis of GDD in­put data (for ex­am­ple, NHANES data) in a pre­vi­ous study88 and other re­search on as­sess­ment of pop­u­la­tion di­etary in­take89,90, as it is non­neg­a­tive and in­cludes a wide range of shapes with vary­ing de­grees of skew­ness91. Standard de­vi­a­tion (s.d.) needed to be ob­tained to con­struct the gamma dis­tri­b­u­tion of in­takes. Parameters for gamma dis­tri­b­u­tion were gen­er­ated us­ing the mean es­ti­mate from the GDD es­ti­ma­tion model and the es­ti­mated s.d. for the mean es­ti­mate from 1,000 sim­u­la­tions.

Stratum-level GDD in­put sur­vey data were used to fit a lin­ear re­gres­sion of the s.d. of in­take on mean in­take (both ad­justed for to­tal en­ergy). To de­ter­mine the ap­pro­pri­ate trans­for­ma­tion of the in­put data used for fit­ting the lin­ear re­gres­sion, scat­ter plots of en­ergy-ad­justed means ver­sus en­ergy-ad­justed s.d. were cre­ated. Using this ap­proach, we con­cluded that a nat­ural log trans­for­ma­tion for both mean and s.d. was most ap­pro­pri­ate. We also ex­plored ex­clud­ing de­mo­graphic and health sur­veys, house­hold sur­veys and out­lier data ow­ing to the po­ten­tial un­re­li­a­bil­ity of such sur­veys for es­ti­mat­ing s.d., but de­ter­mined that no one di­etary as­sess­ment method con­tributed un­evenly to the ob­served lin­ear trend. Thus, all avail­able data were in­cluded, al­low­ing for the largest pos­si­ble sam­ple size and great­est gen­er­al­iz­abil­ity. We also in­ves­ti­gated whether the log mean and log s.d. re­la­tion­ship dif­fered by world re­gion, but did not find strong ev­i­dence for such het­ero­gene­ity. A re­gres­sion model was used for each in­di­vid­ual diet fac­tor to cal­cu­late the s.d.:

where i refers to each sur­vey stra­tum, Y is the nat­ural log of the s.d. of stra­tum-spe­cific in­take, x is the nat­ural log of the mean of stra­tum-spe­cific in­take and ε is the ran­dom er­ror that fol­lows N(0, σ2).

Estimates for β and β were used to pre­dict 1,000 ln(s.d.) val­ues cor­re­spond­ing to 1,000 it­er­a­tions (k) of the pre­dicted mean in­take for each pop­u­la­tion stra­tum (j) us­ing Monte Carlo sim­u­la­tions.

in which \(\widehat{{X}_{{jk}}}\) is the kth sam­ple draw of the pos­te­rior dis­tri­b­u­tion for mean in­take for pop­u­la­tion stra­tum j. We prop­a­gated un­cer­tainty from the model es­ti­mates, as well as vari­a­tion within the sam­pling data it­self, by ran­domly draw­ing from a t-dis­tri­b­u­tion with n − 1 de­grees of free­dom us­ing the fol­low­ing equa­tion:

in which \(\hat{\sigma }\) is the es­ti­mate for σ, n is the num­ber of sur­vey strata, \({t}_{k}^{n-1}\) is the kth sam­ple drawn from a t-dis­tri­b­u­tion with n − 1 de­grees of free­dom and \(\widehat{{s.d.}_{{jk}}}\) is the kth sam­ple draw of the pre­dicted s.d. dis­tri­b­u­tion for pop­u­la­tion stra­tum j.

The pos­te­rior dis­tri­b­u­tions for each stra­tum-spe­cific s.d. were used to gen­er­ate 1,000 cor­re­spond­ing shape and rate gamma pa­ra­me­ters for the dis­tri­b­u­tion of usual in­take, a pri­mary in­put in the CRA model, us­ing the fol­low­ing equa­tions:

The di­rect risk es­ti­mates be­tween SSB in­take and T2D, is­chemic heart dis­ease and is­chemic stroke were ob­tained from pub­lished sys­tem­atic re­views and ev­i­dence grad­ing, based on pub­lished meta-analy­ses of prospec­tive co­hort stud­ies and ran­dom­ized con­trolled tri­als in­clud­ing mul­ti­vari­able ad­just­ment for age, sex, BMI and other risk fac­tors to re­duce bias from con­found­ing (Supplementary Table 8)2,68,69,70. The meth­ods and re­sults for the re­view, iden­ti­fi­ca­tion and as­sess­ment of ev­i­dence for the SSB–disease re­la­tion­ships have been de­scribed2,67. Briefly, ev­i­dence for each SSB–disease re­la­tion­ship was first eval­u­ated by grad­ing the qual­ity of ev­i­dence ac­cord­ing to nine dif­fer­ent Bradford Hill cri­te­ria for cau­sa­tion: strength, con­sis­tency, tem­po­ral­ity, co­her­ence, speci­ficity, anal­ogy, plau­si­bil­ity, bi­o­log­i­cal gra­di­ent and ex­per­i­ment. This ev­i­dence grad­ing was com­pleted in­de­pen­dently and in du­pli­cate by two ex­pert in­ves­ti­ga­tors. Based on these as­sess­ments, prob­a­ble or con­vinc­ing ev­i­dence was de­ter­mined in­de­pen­dently and in du­pli­cate, in ac­cor­dance with the cri­te­ria of the FAO and World Health Organization92 and with con­sid­er­a­tion of con­sis­tency with sim­i­lar cri­te­ria of the World Cancer Research Fund and the American Institute for Cancer Research93. SSBs had at least prob­a­ble as­so­ci­a­tion for di­rect eti­o­logic ef­fects (BMI in­de­pen­dent) on T2D, is­chemic heart dis­ease and is­chemic stroke risk, as well as on weight gain. See Supplementary Table 9 for fur­ther de­tails on the ev­i­dence grad­ing cri­te­ria and re­sults of this eval­u­a­tion. All SSB–disease es­ti­mates were stan­dard­ized from the orig­i­nally re­ported 250 ml serv­ing size to 8 oz serv­ings (248 g), the unit used in our analy­sis.

Given that these stud­ies ad­justed for BMI, we sep­a­rately as­sessed the BMI-mediated ef­fects (BMI change in kg m−2) based on pooled analy­ses from long-term prospec­tive co­hort stud­ies of changes in diet and changes in BMI (Supplementary Table 8)17. Specifically, we used data from three sep­a­rate prospec­tive co­hort stud­ies: the Nurses’ Health Study (1986–2006), in­volv­ing 50,422 women with 20 years of fol­low-up; the Nurses’ Health Study II (1991–2003), in­clud­ing 47,898 women with 12 years of fol­low-up; and the Health Professionals Follow-up Study (1986–2006) with 22,557 men with 20 years of fol­low-up. Participants in­cluded in these analy­ses were ini­tially free of obe­sity (that is, BMI −2) or chronic dis­eases and had com­plete base­line data on weight and lifestyle habits. The as­so­ci­a­tions be­tween SSBs and weight gain were es­ti­mated sep­a­rately for over­weight and obese (BMI ≥ 25 kg m−2) and non-over­weight adults (BMI −2), given ob­served ef­fect mod­i­fi­ca­tion by base­line BMI sta­tus17. We used lin­ear re­gres­sion with ro­bust vari­ance, ac­count­ing for within-per­son re­peated mea­sures, to as­sess the in­de­pen­dent re­la­tion­ships be­tween changes in SSB in­take and changes in BMI over 4 year pe­ri­ods. Women who be­came preg­nant dur­ing fol­low-up were ex­cluded from the analy­sis. BMI-mediated ef­fects did not specif­i­cally dif­fer­en­ti­ate be­tween over­weight and obe­sity, which could have led to an un­der­es­ti­ma­tion in the BMI-mediated ef­fects among adults with obe­sity.

To ex­am­ine the BMI-mediated as­so­ci­a­tions, we as­sessed the im­pact of dif­fer­ences in BMI on the risk of T2D, is­chemic heart dis­ease and is­chemic stroke (Supplementary Table 8)15,16. These re­la­tion­ships were ob­tained from pooled analy­ses of mul­ti­ple co­hort stud­ies in­ves­ti­gat­ing the quan­ti­ta­tive ef­fects of BMI on T2D15, is­chemic heart dis­ease16 and is­chemic stroke15. The risk es­ti­mates were trans­formed from the orig­i­nally re­ported 5 kg m−2 to 1 kg m−2.

Age-specific rel­a­tive risks were cal­cu­lated for each SSB–disease eti­o­logic re­la­tion­ship based on ev­i­dence show­ing de­creas­ing pro­por­tional ef­fects of meta­bolic risk fac­tors on car­diometa­bolic dis­ease in­ci­dence at older ages (for ex­am­ple, due to other com­pet­ing risk fac­tors)15,67. The age-spe­cific rel­a­tive risks were cal­cu­lated based on the age at event and were as­sumed to have a log-lin­ear age as­so­ci­a­tion, al­though the true age re­la­tion­ship may dif­fer.

To cal­cu­late the age at event for each SSB–disease pair, we ob­tained rel­e­vant data from the orig­i­nal stud­ies. This in­cluded the av­er­age age at base­line in years, the fol­low-up time in years, the type of fol­low-up time re­ported (for ex­am­ple, max­i­mum, me­dian or mean) and the study weight for each study in each meta-analy­sis (Supplementary Tables 10–12 and Supplementary Data 4). In cases in which the age at base­line was re­ported as a range rather than as the av­er­age, we used the cen­tral value to es­ti­mate the mean. If fol­low-up time to events was not re­ported, we es­ti­mated it based on the du­ra­tion of the study. For stud­ies that re­ported max­i­mum fol­low-up time, we es­ti­mated the mean time to event as half of the max­i­mum fol­low-up, and for stud­ies that re­ported mean or me­dian fol­low-up times, as two-thirds of the mean or me­dian fol­low-up. The un­weighted mean age at event for each study was cal­cu­lated by sum­ming the mean age at base­line and the ap­pro­pri­ate mean time to event, and the weighted mean age at event for the meta-analy­sis as the weighted age at event across all stud­ies. In cases in which spe­cific stud­ies were ex­cluded from the meta-analy­sis ow­ing to lim­i­ta­tions in study qual­ity, or when the meta-analy­sis was con­ducted for mul­ti­ple out­comes, the weights were ad­justed ac­cord­ingly. When study weights were not re­ported, we as­signed equal weights to each study when cal­cu­lat­ing the mean over­all age at event.

Given lim­ited ev­i­dence of sig­nif­i­cant ef­fect mod­i­fi­ca­tion by sex, we in­cor­po­rated sim­i­lar pro­por­tional ef­fects of risk fac­tors by sex67. In pre­vi­ous re­search, we eval­u­ated the pro­por­tional dif­fer­ences in rel­a­tive risk for key diet-re­lated car­diometa­bolic risk fac­tors, in­clud­ing sys­tolic blood pres­sure, BMI, fast­ing plasma glu­cose and to­tal cho­les­terol, across six 10 year age groups from 25–34 years to 75+ years67. Given sim­i­lar­i­ties across these four risk fac­tors, the mean pro­por­tional dif­fer­ences in rel­a­tive risk across all risk fac­tors were ap­plied to the SSB–disease rel­a­tive risks. In this study, we dis­ag­gre­gated the mean pro­por­tional dif­fer­ences into 14 5 year age groups from 20–24 years to 85+ years. This was achieved by lin­early scal­ing be­tween each 10 year mean pro­por­tional dif­fer­ence in log rel­a­tive risk, an­chor­ing at the cal­cu­lated mean age at event for each SSB–disease.

We used Monte Carlo sim­u­la­tions to es­ti­mate the un­cer­tainty in the age-dis­trib­uted log rel­a­tive risk, sam­pling from the dis­tri­b­u­tion of log rel­a­tive risk at the age at event. On the ba­sis of 1,000 sim­u­la­tions, we used the 2.5th and 97.5th per­centiles to de­rive the 95% UI.

Prevalence of over­weight (BMI ≥ 25 kg m−2) and un­der­weight (BMI −2) in each coun­try–year–age–sex–ur­ban­ic­ity stra­tum and their un­cer­tainty was ob­tained from the NCD-RisC. The NCD-RisC col­lected data from 1,820 pop­u­la­tion-based stud­ies en­com­pass­ing na­tional, re­gional and global trends in mean BMI, with mea­sure­ments of height and weight taken from over 97 mil­lion adult­s71,94. Surveys were ex­cluded if they re­lied solely on self-re­port data, fo­cused on spe­cific sub­sets of the pop­u­la­tion or in­volved preg­nancy. The NCD-RisC used a Bayesian hi­er­ar­chi­cal model to es­ti­mate age-spe­cific mean BMI and preva­lence of over­weight and obe­sity by coun­try, year and sex. The model in­cor­po­rated data-dri­ven fixed ef­fects to ac­count for dif­fer­ences in BMI by rural and ur­ban area of res­i­dence. A Gibbs sam­pling Markov Chain Monte Carlo al­go­rithm was used to fit the model, pro­duc­ing 5,000 sam­ples of the pos­te­rior dis­tri­b­u­tions of the model pa­ra­me­ters. These sam­ples were then used to gen­er­ate pos­te­rior dis­tri­b­u­tions of mean BMI and preva­lence of over­weight and obe­sity for each stra­tum. Estimates were age stan­dard­ized us­ing age weights from the WHO stan­dard pop­u­la­tion. Weighting was also used at the global, re­gional and na­tional lev­els, tak­ing into ac­count the re­spec­tive age-spe­cific pop­u­la­tion pro­por­tions by coun­try, year and sex. The es­ti­mates of mean BMI and over­weight and obe­sity preva­lence were pre­sented along with their re­spec­tive 95% cred­i­ble in­ter­vals, rep­re­sent­ing the un­cer­tainty around the es­ti­mates. To fur­ther strat­ify the NCD-RisC over­weight and obe­sity preva­lence es­ti­mates by ed­u­ca­tion level and ur­ban­ic­ity, we as­sumed that the preva­lence did not vary across dif­fer­ent ed­u­ca­tion lev­els or be­tween ur­ban and rural res­i­dences. In ad­di­tion, it was as­sumed that these es­ti­mates re­mained con­stant be­tween 2016 and 2020 (as NCD-RisC re­ports only through 2016, but this CRA analy­sis as­sesses es­ti­mates for 2020), a con­ser­v­a­tive as­sump­tion that prob­a­bly un­der­es­ti­mates the preva­lence of over­weight and obe­sity and, thus, SSB-attributable bur­dens.

The op­ti­mal in­take level of SSBs served as the coun­ter­fac­tual in our CRA mod­el­ing analy­sis, al­low­ing the quan­tifi­ca­tion of im­pacts of SSBs on dis­ease risk at the pop­u­la­tion level. We de­ter­mined the op­ti­mal in­take level based on prob­a­ble or con­vinc­ing ev­i­dence for ef­fects of SSBs on car­diometa­bolic out­comes. The method­ol­ogy for defin­ing the op­ti­mal in­take level has been de­scribed67. Briefly, it was de­ter­mined pri­mar­ily based on dis­ease risk (observed con­sump­tion lev­els as­so­ci­ated with low­est dis­ease risk in meta-analy­ses) with fur­ther con­sid­er­a­tions of fea­si­bil­ity (observed na­tional mean con­sump­tion lev­els in na­tion­ally rep­re­sen­ta­tive sur­veys world­wide)95,96, and con­sis­tency with ex­ist­ing ma­jor food-based di­etary guide­li­nes97,98. The term ‘optimal in­take’ can be con­sid­ered an­a­lyt­i­cally anal­o­gous to what has been re­ferred to as the ‘theoretical min­i­mum risk ex­po­sure lev­el’ in other analy­ses99,100. We pre­fer the for­mer term as it is more rel­e­vant to di­etary risks, which can serve as a bench­mark for quan­ti­fy­ing dis­ease risk, in­form­ing di­etary guid­ance and in­form­ing pol­icy pri­or­i­ties.

The es­ti­mates of un­der­ly­ing car­diometa­bolic dis­ease bur­dens at global, re­gional and na­tional lev­els were ob­tained from the GBD 2021. The GBD col­lected data from cen­suses, house­hold sur­veys, civil reg­is­tra­tion, vi­tal sta­tis­tics and other rel­e­vant records to es­ti­mate in­ci­dence, preva­lence, mor­tal­ity, years lived with dis­abil­ity (YLDs), years of life lost (YLLs) and DALYs for 371 dis­eases and in­juries73. These es­ti­mates were strat­i­fied by 204 coun­tries and ter­ri­to­ries, 23 age groups and sex, yearly from 1990 to 2021. For this analy­sis, we used GBD es­ti­mates of in­ci­dence, mor­tal­ity and DALYs for T2D, is­chemic heart dis­ease and is­chemic stroke for 1990 and 2020. The GBD de­fined T2D as fast­ing plasma glu­cose greater than or equal to 126 mg dl−1 (7 mmol l−1) or re­port­ing the use of di­a­betes med­ica­tion73. Estimated cases of type 1 di­a­betes were sub­tracted from the over­all di­a­betes cases at the most strat­i­fied level of age, sex, lo­ca­tion and year to es­ti­mate T2D cases. Ischemic heart dis­ease was es­ti­mated in the GBD as the ag­gre­gate of my­ocar­dial in­frac­tion (heart at­tack), angina (chest pain) or is­chemic car­diomy­opa­thy (heart fail­ure due to is­chemic heart dis­ease). Ischemic stroke was de­fined as rapidly de­vel­op­ing clin­i­cal signs of (usually fo­cal) cere­bral func­tion dis­tur­bance last­ing over 24 h, or lead­ing to death, ac­cord­ing to the WHO cri­te­rion of sud­den oc­clu­sion of ar­ter­ies sup­ply­ing the brain due to a throm­bus101.

GBD mor­tal­ity es­ti­mates were gen­er­ated us­ing the Cause of Death Ensemble Model frame­work, which in­cor­po­rated var­i­ous mod­els in­clud­ing dif­fer­ent sets of co­vari­ates test­ing the pre­dic­tive va­lid­ity, and gen­er­at­ing cause-spe­cific mor­tal­ity es­ti­mates73,102,103. Cause of Death Ensemble Model es­ti­mates were scaled among all causes such that the sum of cause-spe­cific deaths did not ex­ceed all-cause mor­tal­ity. YLLs were cal­cu­lated as the prod­uct of the num­ber of deaths for each cause by age, sex, lo­ca­tion and year times the stan­dard life ex­pectancy. Life ex­pectancy was first de­com­posed by cause of death, lo­ca­tion and year to rep­re­sent the cause-spe­cific ef­fects on life ex­pectan­cy102. Then, the sum across age groups was taken to es­ti­mate the im­pact of a given cause on the at-birth life ex­pectancy from 1990 to 2021. Incidence was mod­eled us­ing DisMod, a meta-re­gres­sion tool that used epi­demi­o­logic data to es­ti­mate the oc­cur­rence dis­ease within a pop­u­la­tion and de­ter­mines whether cases re­main preva­lent, go into re­mis­sion or re­sult in death. YLDs were cal­cu­lated by split­ting the preva­lence of each cause into mu­tu­ally ex­clu­sive se­quela, each de­fined by a health state; each health state was then weighted by the cor­re­spond­ing dis­abil­ity weight73. Finally, DALYs were cal­cu­lated as the sum of YLLs and YLDs.

The GBD pro­vides un­der­ly­ing dis­ease es­ti­mates at global, re­gional and na­tional lev­els for 1990 to 2021, jointly strat­i­fied by age and sex. Extensive pre­vi­ous ev­i­dence shows that T2D and CVD out­comes vary by ed­u­ca­tional at­tain­ment and ur­ban­ic­i­ty104,105,106,107,108,109,110,111,112,113,114,115,116,117,118,119,120,121,122. We fur­ther strat­i­fied the 1990 and 2020 GBD es­ti­mates by ed­u­ca­tion level (low, medium, high) and area of res­i­dency (urban, rural) to ex­am­ine po­ten­tial vari­a­tions in risk within these sub­pop­u­la­tions and to align with the de­mo­graphic and GDD di­etary data strat­i­fi­ca­tions avail­able. This ap­proach re­quired as­sump­tions on dis­tri­b­u­tions of dis­ease bur­dens by these de­mo­graphic fac­tors and po­ten­tially un­der­es­ti­mated un­cer­tainty in our re­sults strat­i­fied by these fac­tors.

To strat­ify the GBD es­ti­mates, we con­ducted a search of sci­en­tific lit­er­a­ture to iden­tify re­cent meta-analy­sis, pooled analy­ses and large sur­veys eval­u­at­ing the as­so­ci­a­tion be­tween ed­u­ca­tional at­tain­ment and ur­ban­ic­ity with the risk of T2D and CVD. Because we hy­poth­e­sized that coun­try in­come level was a po­ten­tial ef­fect mod­i­fier for the re­la­tion­ships of ed­u­ca­tional at­tain­ment and ur­ban­ic­ity with T2D and CVD risk, we fur­ther col­lected and col­lated risk es­ti­mates strat­i­fied by coun­try in­come level. We lim­ited our analy­sis to stud­ies ad­just­ing only for age and sex, when pos­si­ble, to avoid the at­ten­u­at­ing ef­fects of ad­just­ing for ad­di­tional co­vari­ates104,105,106,107,108,109,110,111,112,113,114,115,116,117,118,119,120,121,122.

We con­ducted fixed-ef­fects meta-analy­sis of col­lated ef­fect sizes (associations be­tween ed­u­ca­tion or ur­ban­ic­ity and dis­ease rates), strat­i­fied by coun­try in­come level. Published es­ti­mates were stan­dard­ized to high ver­sus low ed­u­ca­tion level, matched as closely as pos­si­ble to the GDD de­f­i­n­i­tions (low: 0–6 years of ed­u­ca­tion; high: >12 years of ed­u­ca­tion), as well as to ur­ban ver­sus rural res­i­dence. We pooled es­ti­mates within stud­ies when (1) mul­ti­ple es­ti­mates were re­ported for dif­fer­ent CVD out­comes, (2) sep­a­rate es­ti­mates were pro­vided for men and women, (3) es­ti­mates were re­ported for dif­fer­ent lo­ca­tions (except by coun­try in­come) or (4) an in­ter­me­di­ate cat­e­gory matched our de­f­i­n­i­tions for ed­u­ca­tion level or area of res­i­dence. The char­ac­ter­is­tics of the stud­ies used to cal­cu­late the ef­fect es­ti­mates, in­clud­ing their orig­i­nal and cal­cu­lated ef­fect sizes, can be found in Supplementary Data 5 and 6 for ed­u­ca­tion level and area of res­i­dence, re­spec­tively.

We con­ducted a sep­a­rate fixed-ef­fect meta-analy­sis for the re­la­tion­ship of ed­u­ca­tion or ur­ban­ic­ity to T2D and CVD, strat­i­fied by coun­try in­come level. We dis­trib­uted the cen­tral es­ti­mate of our meta-an­a­lyzed risk es­ti­mate equally for high ver­sus low ed­u­ca­tion, and ur­ban ver­sus rural res­i­dence, by tak­ing its square root and in­verse square root (Supplementary Table 13). This ap­proach as­sumed sim­i­lar dif­fer­ences from high to medium ed­u­ca­tion as from medium to low ed­u­ca­tion. We also ex­plored dis­trib­ut­ing the cen­tral es­ti­mate by in­cor­po­rat­ing in­for­ma­tion on the ac­tual dis­tance (for ex­am­ple, grade years) from high to medium ed­u­ca­tion and medium to low ed­u­ca­tion, when such in­for­ma­tion was avail­able. As the re­sults did not ap­pre­cia­bly dif­fer, we used the square root and in­verse square root ap­proach to main­tain con­sis­tency across stud­ies, par­tic­u­larly given het­ero­gene­ity in cat­e­go­riza­tions of ed­u­ca­tion lev­els. The fi­nal cal­cu­lated ef­fect es­ti­mates for the as­so­ci­a­tion be­tween ed­u­ca­tion level and area of res­i­dence with T2D and CVD, by in­come coun­try level, can be found in Supplementary Table 13.

The T2D, is­chemic heart dis­ease and is­chemic stroke es­ti­mates for each year–coun­try–age–sex stra­tum (mean and 95% UI) were mul­ti­plied by their re­spec­tive pop­u­la­tion pro­por­tion, ed­u­ca­tion ef­fect and ur­ban ef­fect. This process cre­ated six de novo strata with the raw (unscaled) fully pro­por­tioned bur­den es­ti­mates and their un­cer­tainty. The global pop­u­la­tion pro­por­tions for each year were de­rived from the United Nations Population Division75, sup­ple­mented with data on ed­u­ca­tion at­tain­ment from a pre­vi­ous study76. Finally, to pre­vent un­der- or over­es­ti­ma­tion of the ab­solute num­ber of T2D, is­chemic heart dis­ease and is­chemic stroke cases glob­ally, the raw fully pro­por­tioned bur­den es­ti­mates were scaled to match the to­tal bur­den es­ti­mate for each stra­tum. This scal­ing en­sured that the over­all bur­den es­ti­mates re­mained con­sis­tent. Supplementary Table 14 pro­vides a fic­ti­tious, il­lus­tra­tive ex­am­ple of how 1,000 T2D cases in a sin­gle age–sex pop­u­la­tion stra­tum (low-income coun­try) in a given year were dis­ag­gre­gated into the 6 finer ed­u­ca­tion–ur­ban­ic­ity strata us­ing the cen­tral es­ti­mate of the meta-an­a­lyzed ed­u­ca­tion and ur­ban ef­fects. The pop­u­la­tion pro­por­tioned only bur­den es­ti­mates is also pro­vided as a com­par­i­son. While un­cer­tainty was in­cor­po­rated in all the mod­el­ing pa­ra­me­ters, we were un­able to in­clude un­cer­tainty in the strat­i­fi­ca­tion of T2D and CVD cases by ed­u­ca­tional at­tain­ment and ur­ban or rural res­i­dence as rig­or­ous data to do so were not avail­able.

The CRA frame­work in­cor­po­rated the data in­puts and their un­cer­tainty to es­ti­mate the ab­solute num­ber, rate (per mil­lion adults 20+ years) and pro­por­tion of T2D, is­chemic heart dis­ease and is­chemic stroke cases at­trib­ut­able to in­take of SSBs in 1990 and 2020 (Supplementary Fig. 18). For each stra­tum, the model cal­cu­lated the per­cent­age (population at­trib­ut­able frac­tion (PAF)) of to­tal T2D, is­chemic heart dis­ease and is­chemic stroke in­ci­dence, mor­tal­ity and DALYs at­trib­ut­able to in­take of SSBs. For BMI-mediated ef­fects, the model con­sid­ered the as­so­ci­a­tions be­tween ob­served SSB in­takes and changes in BMI at the stra­tum level. This as­so­ci­a­tion was weighted by the preva­lence of over­weight (BMI ≥ 25), nor­mal weight (BMI >18.5 to

Given that sum­ming di­rect and BMI-mediated PAFs would over­es­ti­mate the com­bined ef­fect, for each dis­ease stra­tum (that is, coun­try–year–age–sex–ed­u­ca­tion–res­i­dence), the PAF was cal­cu­lated us­ing pro­por­tional mul­ti­pli­ca­tion of the di­rect and BMI-mediated PAFs as fol­lows:

The re­sult­ing PAF was then mul­ti­plied by the cor­re­spond­ing num­ber of dis­ease cases to cal­cu­late the at­trib­ut­able bur­den in each stra­tum. Findings were eval­u­ated glob­ally, re­gion­ally and na­tion­ally, and by spe­cific pop­u­la­tion sub­groups of age, sex, ed­u­ca­tion and ur­ban­ic­ity. The re­sults are pre­sented as pro­por­tional at­trib­ut­able bur­den (percentage of cases) and at­trib­ut­able rate (per one mil­lion adults). This rep­re­sen­ta­tion of the pro­por­tional mul­ti­pli­ca­tion for a sin­gle risk fac­tor (that is, SSBs) is equiv­a­lent to the for­mula com­monly re­ported for sev­eral risk fac­tors: \({\rm{PAF}}=1-\,\mathop{\prod}\nolimits_{i=1}^{n}1-{\rm{PAF}}_{i}\)

The PAF for­mula is used to quan­tify the bur­den of dis­ease at­trib­ut­able to a par­tic­u­lar ex­po­sure. It in­volves com­par­ing the dis­ease cases as­so­ci­ated with the ob­served ex­po­sure lev­els in the pop­u­la­tion to a coun­ter­fac­tual sce­nario with an op­ti­mal in­take dis­tri­b­u­tion, given a known eti­o­logic ex­po­sure–dis­ease risk re­la­tion­ship.

In this analy­sis, we aimed to es­ti­mate the bur­den of in­ci­dence, mor­tal­ity and DALYs for T2D, is­chemic heart dis­ease and is­chemic stroke at­trib­ut­able to in­take of SSBs.

The PAF for­mula used is as fol­lows:

where P(x) is the usual SSB in­take dis­tri­b­u­tion in a spe­cific pop­u­la­tion stra­tum, as­sumed to fol­low a gamma dis­tri­b­u­tion as used in pre­vi­ous analy­ses3,31,88; RR(x) is the age-spe­cific rel­a­tive risk func­tion for T2D or CVD risk; and m is the max­i­mum ex­po­sure level.

where β is the stra­tum-spe­cific change in log rel­a­tive risk per unit of ex­po­sure, x is the cur­rent ex­po­sure level and y(x) is the op­ti­mal ex­po­sure level. y(x) is de­fined to be \({F}_{\rm{optimal}}({F}_{x}^{-1}\left(x\right))\), where F is the cu­mu­la­tive dis­tri­b­u­tion func­tion of the op­ti­mal in­take and \({F}_{x}^{-1}\) is the in­verse cu­mu­la­tive dis­tri­b­u­tion func­tion of the cur­rent ex­po­sure dis­tri­b­u­tion. Implicit in how we char­ac­ter­ize the rel­a­tive risk func­tion are cer­tain as­sump­tions, in­clud­ing a lin­ear re­la­tion­ship be­tween the log rel­a­tive risk (beta) and the unit of ex­po­sure. This model as­sumes that no fur­ther risk is as­so­ci­ated with ex­po­sure be­yond the op­ti­mal in­take level, and that both x and the op­ti­mal in­take level for an in­di­vid­ual at ex­po­sure level x are the qth quan­tile of their re­spec­tive dis­tri­b­u­tions (the ob­served ex­po­sure dis­tri­b­u­tion and the op­ti­mal in­take dis­tri­b­u­tion, re­spec­tively).

In prac­tice, sim­ple nu­mer­i­cal in­te­gra­tion us­ing Riemann sums can be used to com­pute the in­te­grals in the PAF for­mu­la88.

n cat­e­gories are de­ter­mined by di­vid­ing the ex­po­sure range (chosen here to be 0, \({F}_{x}^{-1}\left.\left(\varPhi \left(-6\right)\right)\right)\) into 121 in­ter­vals, each of length 0.1 when con­verted to the stan­dard nor­mal scale (except for the first one). Φ is de­fined as the cu­mu­la­tive dis­tri­b­u­tion func­tion of the stan­dard nor­mal dis­tri­b­u­tion (N(0,1)). More pre­cisely, the range of ex­po­sure groups i can be de­scribed as:

The as­so­ci­a­tion of change in BMI with change in SSB in­take was as­sessed in three pooled US co­horts us­ing mul­ti­vari­ate lin­ear re­gres­sion ac­count­ing for within-per­son re­peated mea­sures, as de­scribed in an ear­lier study17. Separate lin­ear re­la­tion­ships were es­ti­mated for un­der­weight (BMI 17. Because in­di­vid­u­als with obe­sity were ex­cluded in these pre­vi­ous analy­ses, we used the risk es­ti­mate for in­di­vid­u­als with over­weight for in­di­vid­u­als with obe­sity, which could un­der­es­ti­mate the full ef­fects of SSB on weight change.

To as­sess the BMI-mediated ef­fects of SSB in­take on in­ci­dence, mor­tal­ity and DALYs of T2D, is­chemic heart dis­ease and is­chemic stroke, we first cal­cu­lated the mo­not­o­nic ef­fect of SSB in­take on BMI change for each pop­u­la­tion stra­tum by weight­ing the base­line BMI-specific ef­fect by the re­spec­tive preva­lence of un­der­weight, nor­mal weight and over­weight (including obe­sity) within each stra­tum. We ob­tained over­weight and un­der­weight pop­u­la­tion dis­tri­b­u­tions from the NCD-RisC71 and cal­cu­lated the preva­lence of nor­mal weight as 1 mi­nus the sum of these preva­lences71. The NCD-RisC es­ti­mates go up to 2016, and thus, for our 2020 analy­sis, we used data from 2016 as a proxy for 2020. Given in­creas­ing adi­pos­ity glob­ally, this as­sump­tion could re­sult in un­der­es­ti­ma­tion of dis­ease bur­dens due to SSBs in 2020. We as­sumed that in­di­vid­u­als with un­der­weight did not ex­pe­ri­ence in­creased risk of T2D, is­chemic heart dis­ease or is­chemic stroke with in­creased con­sump­tion of SSBs. As such, the mo­not­o­nic ef­fect for this pop­u­la­tion seg­ment was set at 0:

We then es­ti­mated the BMI-mediated log(RR) by mul­ti­ply­ing the log(RR) per unit in­crease in BMI and the SSB-to-BMI ef­fect (associated in­crease in BMI per one-unit-as­so­ci­ated in­crease in SSB in­take).

We used Monte Carlo sim­u­la­tions to quan­tify the un­cer­tainty around the PAF es­ti­mate. In this cal­cu­la­tion, we in­cor­po­rated un­cer­tainty of mul­ti­ple key pa­ra­me­ters, in­clud­ing the usual in­take dis­tri­b­u­tion of SSBs in each stra­tum; un­der­ly­ing T2D, CVD and DALY bur­den es­ti­mates in each stra­tum; the eti­o­logic es­ti­mates (RR) for SSB–BMI, SSB–T2D and SSB–CVD re­la­tion­ships; and the preva­lence of in­di­vid­u­als with un­der­weight, nor­mal weight or over­weight in each stra­tum. For each SSB–disease com­bi­na­tion and stra­tum, we drew ran­domly 1,000 times from the re­spec­tive prob­a­bil­ity dis­tri­b­u­tions. This in­cluded draw­ing ran­domly from the nor­mal dis­tri­b­u­tion of the es­ti­mate of dis­ease-spe­cific changes in the log(RR) of BMI-mediated and di­rect eti­o­logic ef­fects for a one-unit in­crease in SSB in­take, the pos­te­rior dis­tri­b­u­tions for shape and rate pa­ra­me­ters for usual di­etary in­take and the nor­mal dis­tri­b­u­tion of the es­ti­mate for the preva­lence of un­der­weight, nor­mal weight and over­weight. Draws of pro­por­tions that were less than 0 or greater than 1 were trun­cated at 0 or 1, re­spec­tively, and draws of mean in­take that were 0 or less were trun­cated at 0.00001. Each set of ran­dom draws was used to cal­cu­late the PAFs and, mul­ti­plied by the stra­tum-spe­cific dis­ease rates, the as­so­ci­ated ab­solute at­trib­ut­able dis­ease bur­den. Corresponding 95% UIs were de­rived from the 2.5th and 97.5th per­centiles of 1,000 es­ti­mated mod­els.

We as­sessed na­tional-level find­ings by SDI in 1990 and 2020. The SDI is a com­pos­ite mea­sure of a na­tion’s de­vel­op­ment based on fac­tors such as in­come per capita, ed­u­ca­tional at­tain­ment and fer­til­ity rates18.

To com­pare es­ti­mates across dif­fer­ent years (1990 and 2020), we cal­cu­lated dif­fer­ences for ab­solute and pro­por­tional bur­dens from 1990 to 2020 (that is, 2020 mi­nus 1990). We per­formed this cal­cu­la­tion for each sim­u­la­tion re­sult­ing in a dis­tri­b­u­tion of dif­fer­ences, and we re­port the me­dian and 95% UIs for each dif­fer­ence. We did not for­mally stan­dard­ize com­par­isons over time by age or sex. This de­ci­sion was made to en­sure that find­ings would re­flect the ac­tual pop­u­la­tion dif­fer­ences in at­trib­ut­able bur­dens that are rel­e­vant to pol­icy de­ci­sions. However, we also per­formed analy­ses strat­i­fied by age and sex, tak­ing into ac­count changes in these de­mo­graph­ics over time. All analy­ses were con­ducted us­ing R sta­tis­ti­cal soft­ware, R ver­sion 4.4.0 (ref. 123) on the Tufts High Performance Cluster.

Further in­for­ma­tion on re­search de­sign is avail­able in the Nature Portfolio Reporting Summary linked to this ar­ti­cle.

A/B test­ing is used far too of­ten, for some­thing that per­forms so badly. It is de­fec­tive by de­sign: Segment users into two groups. Show the A group the old, tried and true stuff. Show the B group the new whiz-bang de­sign with the big­ger but­tons and slightly dif­fer­ent copy. After a while, take a look at the stats and fig­ure out which group presses the but­ton more of­ten. Sounds good, right? The prob­lem is star­ing you in the face. It is the same dilemma faced by re­searchers ad­min­is­ter­ing drug stud­ies. During drug tri­als, you can only give half the pa­tients the life sav­ing treat­ment. The oth­ers get sugar wa­ter. If the treat­ment works, group B lost out. This sac­ri­fice is made to get good data. But it does­n’t have to be this way.

In re­cent years, hun­dreds of the bright­est minds of mod­ern civ­i­liza­tion have been hard at work not cur­ing can­cer. Instead, they have been re­fin­ing tech­niques for get­ting you and me to click on ban­ner ads. It has been work­ing. Both Google and Microsoft are fo­cus­ing on us­ing more in­for­ma­tion about vis­i­tors to pre­dict what to show them. Strangely, any­thing bet­ter than A/B test­ing is ab­sent from main­stream tools, in­clud­ing Google Analytics, and Google Website op­ti­mizer. I hope to change that by rais­ing aware­ness about bet­ter tech­niques.

With a sim­ple 20-line change to how A/B test­ing works, that you can im­ple­ment to­day, you can al­ways do bet­ter than A/B test­ing — some­times, two or three times bet­ter. This method has sev­eral good points:

It can rea­son­ably han­dle more than two op­tions at once.. Eg, A, B, C, D, E, F, G, �

New op­tions can be added or re­moved at any time.

But the most en­tic­ing part is that you can set it and for­get it. If your time is re­ally worth $1000/hour, you re­ally don’t have time to go back and check how every change you made is do­ing and pick op­tions. You don’t have time to write ram­bling blog en­tries about how you got your site re­designed and changed this and that and it worked or it did­n’t work. Let the al­go­rithm do its job. This 20 lines of code au­to­mat­i­cally finds the best choice quickly, and then uses it un­til it stops be­ing the best choice.

The multi-armed ban­dit prob­lem takes its ter­mi­nol­ogy from a casino. You are faced with a wall of slot ma­chines, each with its own lever. You sus­pect that some slot ma­chines pay out more fre­quently than oth­ers. How can you learn which ma­chine is the best, and get the most coins in the fewest tri­als?

Like many tech­niques in ma­chine learn­ing, the sim­plest strat­egy is hard to beat. More com­pli­cated tech­niques are worth con­sid­er­ing, but they may eke out only a few hun­dredths of a per­cent­age point of per­for­mance. One strat­egy that has been shown to per­form well time af­ter time in prac­ti­cal prob­lems is the ep­silon-greedy method. We al­ways keep track of the num­ber of pulls of the lever and the amount of re­wards we have re­ceived from that lever. 10% of the time, we choose a lever at ran­dom. The other 90% of the time, we choose the lever that has the high­est ex­pec­ta­tion of re­wards.

def choose(): if math.ran­dom() < 0.1: # ex­plo­ration! # choose a ran­dom lever 10% of the time. else: # ex­ploita­tion! # for each lever, # cal­cu­late the ex­pec­ta­tion of re­ward. # This is the num­ber of tri­als of the lever di­vided by the to­tal re­ward # given by that lever. # choose the lever with the great­est ex­pec­ta­tion of re­ward. # in­cre­ment the num­ber of times the cho­sen lever has been played. # store test data in re­dis, choice in ses­sion key, etc..

def re­ward(choice, amount): # add the re­ward to the to­tal for the given lever.

Why does this work?

Let’s say we are choos­ing a colour for the “Buy now!” but­ton. The choices are or­ange, green, or white. We ini­tial­ize all three choices to 1 win out of 1 try. It does­n’t re­ally mat­ter what we ini­tial­ize them too, be­cause the al­go­rithm will adapt. So when we start out, the in­ter­nal test data looks like this.

Then a web site vis­i­tor comes along and we have to show them a but­ton. We choose the first one with the high­est ex­pec­ta­tion of win­ning. The al­go­rithm thinks they all work 100% of the time, so it chooses the first one: or­ange. But, alas, the vis­i­tor does­n’t click on the but­ton.

Another vis­i­tor comes along. We def­i­nitely won’t show them or­ange, since we think it only has a 50% chance of work­ing. So we choose Green. They don’t click. The same thing hap­pens for sev­eral more vis­i­tors, and we end up cy­cling through the choices. In the process, we re­fine our es­ti­mate of the click through rate for each op­tion down­wards.

But sud­denly, some­one clicks on the or­ange but­ton! Quickly, the browser makes an Ajax call to our re­ward func­tion $.ajax(url:“/reward?testname=buy-button”); and our code up­dates the re­sults:

When our in­tre­pid web de­vel­oper sees this, he scratches his head. What the F*? The or­ange but­ton is the worst choice. Its font is tiny! The green but­ton is ob­vi­ously the bet­ter one. All is lost! The greedy al­go­rithm will al­ways choose it for­ever now!

But wait, let’s see what hap­pens if Orange is re­ally the sub­op­ti­mal choice. Since the al­go­rithm now be­lieves it is the best, it will al­ways be shown. That is, un­til it stops work­ing well. Then the other choices start to look bet­ter.

After many more vis­its, the best choice, if there is one, will have been found, and will be shown 90% of the time. Here are some re­sults based on an ac­tual web site that I have been work­ing on. We also have an es­ti­mate of the click through rate for each choice.

Edit: What about the ran­dom­iza­tion?

I have not dis­cussed the ran­dom­iza­tion part. The ran­dom­iza­tion of 10% of tri­als forces the al­go­rithm to ex­plore the op­tions. It is a trade-off be­tween try­ing new things in hopes of some­thing bet­ter, and stick­ing with what it knows will work. There are sev­eral vari­a­tions of the ep­silon-greedy strat­egy. In the ep­silon-first strat­egy, you can ex­plore 100% of the time in the be­gin­ning and once you have a good sam­ple, switch to pure-greedy. Alternatively, you can have it de­crease the amount of ex­plo­ration as time passes. The ep­silon-greedy strat­egy that I have de­scribed is a good bal­ance be­tween sim­plic­ity and per­for­mance. Learning about the other al­go­rithms, such as UCB, Boltzmann Exploration, and meth­ods that take con­text into ac­count, is fas­ci­nat­ing, but op­tional if you just want some­thing that works.

Wait a minute, why is­n’t every­body do­ing this?

Statistics are hard for most peo­ple to un­der­stand. People dis­trust things that they do not un­der­stand, and they es­pe­cially dis­trust ma­chine learn­ing al­go­rithms, even if they are sim­ple. Mainstream tools don’t sup­port this, be­cause then you’d have to ed­u­cate peo­ple about it, and about sta­tis­tics, and that is hard. Some com­mon ob­jec­tions might be:

Showing the dif­fer­ent op­tions at dif­fer­ent rates will skew the re­sults. (No it won’t. You al­ways have an es­ti­mate of the click through rate for each choice)

This won’t adapt to change. (Your vis­i­tors prob­a­bly don’t change. But if you re­ally want to, in the re­ward func­tion, mul­ti­ply the old re­ward value by a for­get­ting fac­tor)

This won’t han­dle chang­ing sev­eral things at once that de­pend on each-other. (Agreed. Neither will A/B test­ing.)

I won’t know what the click is worth for 30 days so how can I re­ward it?

Skip to con­tent

We read every piece of feed­back, and take your in­put very se­ri­ously.

Include my email ad­dress so I can be con­tacted

Use saved searches to fil­ter your re­sults more quickly

To see all avail­able qual­i­fiers, see our doc­u­men­ta­tion.

Sign up

You signed in with an­other tab or win­dow. Reload to re­fresh your ses­sion.

You signed out in an­other tab or win­dow. Reload to re­fresh your ses­sion.

You switched ac­counts on an­other tab or win­dow. Reload to re­fresh your ses­sion.

Notifications

You must be signed in to change no­ti­fi­ca­tion set­tings

161 com­mits

to mas­ter

since this re­lease

This tag was signed with the com­mit­ter’s ver­i­fied sig­na­ture.

We are ex­cited to an­nounce the re­lease of OpenZFS 2.3.0.

RAIDZ Expansion (#15022): Add new de­vices to an ex­ist­ing RAIDZ pool, in­creas­ing stor­age ca­pac­ity with­out down­time.

Direct IO (#10018): Allows by­pass­ing the ARC for reads/​writes, im­prov­ing per­for­mance in sce­nar­ios like NVMe de­vices where caching may hin­der ef­fi­ciency.

JSON (#16217): Optional JSON out­put for the most used com­mands.

Long names (#15921): Support for file and di­rec­tory names up to 1023 char­ac­ters.

Thank you to all 134 con­trib­u­tors who par­tic­i­pated in this re­lease cy­cle

[behlendorf1@pip zfs]$ git short­log -s zfs-2.2.0..zfs-2.3.0

4 Ahelenia Ziemiańska

2 Akash B

12 Alan Somers

1 Alek Pinchuk

144 Alexander Motin

6 Allan Jude

1 Alphan Yılmaz

26 Ameer Hamza

2 Andrea Righi

2 Andrew Innes

2 Andrew Turner

1 Andrew Walker

1 Andy Fiddaman

2 Benda Xu

1 Benjamin Sherman

1 Bojan Novković

8 Brian Atkinson

61 Brian Behlendorf

4 Brooks Davis

2 Cameron Harr

1 ChenHao Lu

1 Chris Davidson

1 Chris Peredun

9 Chunwei Chen

14 Coleman Kane

1 Colin Percival

3 Dag-Erling Smørgrav

2 Daniel Berlin

1 Daniel Perry

1 Dennis R. Friedrichsen

1 Derek Schrock

1 Dex Wood

3 Dimitry Andric

13 Don Brady

4 Edmund Nadolski

2 Fabian-Gruenbichler

4 George Amanakis

5 George Melikov

3 George Wilson

6 Gionatan Danti

1 Gleb Smirnoff

1 Gordon Tetlow

1 Ilkka Sovanto

1 Ivan Volosyuk

1 JKDingwall

1 James Reilly

1 Jaron Kent-Dobias

2 Jason King

2 Jason Lee

1 Jessica Clarke

4 Jitendra Patidar

1 John Wren Kennedy

1 Jose Luis Duran

1 José Luis Salvador Rufo

1 Justin Gottula

4 Kay Pedersen

1 Kent Ross

1 Kevin Greene

1 Kevin Jin

1 Lalufu

1 Laura Hild

1 Mariusz Zaborski

17 Mark Johnston

3 Mart Frauenlob

9 Martin Matuška

1 Martin Wagner

4 Mateusz Guzik

7 Mateusz Piotrowski

1 Matthew Ahrens

1 Matthew Heller

1 Mauricio Faria de Oliveira

1 Maxim Filimonov

2 MigeljanImeri

1 Olivier Certner

14 Paul Dagnelie

6 Pavel Snajdr

6 Pawel Jakub Dawidek

1 Peter Doherty

Webtop - Alpine, Ubuntu, Fedora, and Arch based con­tain­ers con­tain­ing full desk­top en­vi­ron­ments in of­fi­cially sup­ported fla­vors ac­ces­si­ble via any mod­ern web browser.

We utilise the docker man­i­fest for multi-plat­form aware­ness. More in­for­ma­tion is avail­able from docker here and our an­nounce­ment here.

Simply pulling lscr.io/​lin­uxserver/​webtop:lat­est should re­trieve the cor­rect im­age for your arch, but you can also pull spe­cific arch im­ages via tags.

The ar­chi­tec­tures sup­ported by this im­age are:

This im­age pro­vides var­i­ous ver­sions that are avail­able via tags. Please read the de­scrip­tions care­fully and ex­er­cise cau­tion when us­ing un­sta­ble or de­vel­op­ment tags.

The Webtop can be ac­cessed at:

Modern GUI desk­top apps have is­sues with the lat­est Docker and syscall com­pat­i­bil­ity, you can use Docker with the –security-opt sec­comp=un­con­fined set­ting to al­low these syscalls on hosts with older Kernels or lib­sec­comp

By de­fault this con­tainer has no au­then­ti­ca­tion and the op­tional en­vi­ron­ment vari­ables CUSTOM_USER and PASSWORD to en­able ba­sic http auth via the em­bed­ded NGINX server should only be used to lo­cally se­cure the con­tainer from un­wanted ac­cess on a lo­cal net­work. If ex­pos­ing this to the Internet we rec­om­mend putting it be­hind a re­verse proxy, such as SWAG, and en­sur­ing a se­cure au­then­ti­ca­tion so­lu­tion is in place. From the web in­ter­face a ter­mi­nal can be launched and it is con­fig­ured for pass­word­less sudo, so any­one with ac­cess to it can in­stall and run what­ever they want along with prob­ing your lo­cal net­work.

This con­tainer is based on Docker Baseimage KasmVNC which means there are ad­di­tional en­vi­ron­ment vari­ables and run con­fig­u­ra­tions to en­able or dis­able spe­cific func­tion­al­ity.

The en­vi­ron­ment vari­able LC_ALL can be used to start this con­tainer in a dif­fer­ent lan­guage than English sim­ply pass for ex­am­ple to launch the Desktop ses­sion in French LC_ALL=fr_FR. UTF-8. Some lan­guages like Chinese, Japanese, or Korean will be miss­ing fonts needed to ren­der prop­erly known as cjk fonts, but oth­ers may ex­ist and not be in­stalled in­side the con­tainer de­pend­ing on what un­der­ly­ing dis­tri­b­u­tion you are run­ning. We only en­sure fonts for Latin char­ac­ters are pre­sent. Fonts can be in­stalled with a mod on startup.

To in­stall cjk fonts on startup as an ex­am­ple pass the en­vi­ron­ment vari­ables (Alpine base):

The web in­ter­face has the op­tion for “IME Input Mode” in Settings which will al­low non eng­lish char­ac­ters to be used from a non en_US key­board on the client. Once en­abled it will per­form the same as a lo­cal Linux in­stal­la­tion set to your lo­cale.

For ac­cel­er­ated apps or games, ren­der de­vices can be mounted into the con­tainer and lever­aged by ap­pli­ca­tions us­ing:

The DRINODE en­vi­ron­ment vari­able can be used to point to a spe­cific GPU. Up to date in­for­ma­tion can be found here

Nvidia sup­port is not com­pat­i­ble with Alpine based im­ages as Alpine lacks Nvidia dri­vers

Nvidia sup­port is avail­able by lever­ag­ing Zink for OpenGL sup­port. This can be en­abled with the fol­low­ing run flags:

The com­pose syn­tax is slightly dif­fer­ent for this as you will need to set nvidia as the de­fault run­time:

And to as­sign the GPU in com­pose:

If you run sys­tem na­tive in­stal­la­tions of soft­ware IE sudo apt-get in­stall filezilla and then up­grade or de­stroy/​re-cre­ate the con­tainer that soft­ware will be re­moved and the con­tainer will be at a clean state. For some users that will be ac­cept­able and they can up­date their sys­tem pack­ages as well us­ing sys­tem na­tive com­mands like apt-get up­grade. If you want Docker to han­dle up­grad­ing the con­tainer and re­tain your ap­pli­ca­tions and set­tings we have cre­ated proot-apps which al­low portable ap­pli­ca­tions to be in­stalled to per­sis­tent stor­age in the user’s $HOME di­rec­tory and they will work in a con­fined Docker en­vi­ron­ment out of the box. These ap­pli­ca­tions and their set­tings will per­sist up­grades of the base con­tainer and can be mounted into dif­fer­ent fla­vors of KasmVNC based con­tain­ers on the fly. This can be achieved from the com­mand line with:

PRoot Apps is in­cluded in all KasmVNC based con­tain­ers, a list of lin­uxserver.io sup­ported ap­pli­ca­tions is lo­cated HERE.

It is pos­si­ble to in­stall ex­tra pack­ages dur­ing con­tainer start us­ing uni­ver­sal-pack­age-in­stall. It might in­crease start­ing time sig­nif­i­cantly. PRoot is pre­ferred.

To help you get started cre­at­ing a con­tainer from this im­age you can ei­ther use docker-com­pose or the docker cli.

Containers are con­fig­ured us­ing pa­ra­me­ters passed at run­time (such as those above). These pa­ra­me­ters are sep­a­rated by a colon and in­di­cate re­spec­tively. For ex­am­ple, -p 8080:80 would ex­pose port 80 from in­side the con­tainer to be ac­ces­si­ble from the host’s IP on port 8080 out­side the con­tainer.

You can set any en­vi­ron­ment vari­able from a file by us­ing a spe­cial prepend FILE__.

As an ex­am­ple:

Will set the en­vi­ron­ment vari­able MYVAR based on the con­tents of the /run/secrets/mysecretvariable file.

For all of our im­ages we pro­vide the abil­ity to over­ride the de­fault umask set­tings for ser­vices started within the con­tain­ers us­ing the op­tional -e UMASK=022 set­ting. Keep in mind umask is not chmod it sub­tracts from per­mis­sions based on it’s value it does not add. Please read up here be­fore ask­ing for sup­port.

When us­ing vol­umes (-v flags), per­mis­sions is­sues can arise be­tween the host OS and the con­tainer, we avoid this is­sue by al­low­ing you to spec­ify the user PUID and group PGID.

Ensure any vol­ume di­rec­to­ries on the host are owned by the same user you spec­ify and any per­mis­sions is­sues will van­ish like magic.

In this in­stance PUID=1000 and PGID=1000, to find yours use id your_user as be­low:

We pub­lish var­i­ous Docker Mods to en­able ad­di­tional func­tion­al­ity within the con­tain­ers. The list of Mods avail­able for this im­age (if any) as well as uni­ver­sal mods that can be ap­plied to any one of our im­ages can be ac­cessed via the dy­namic badges above.

* To mon­i­tor the logs of the con­tainer in re­al­time:

Most of our im­ages are sta­tic, ver­sioned, and re­quire an im­age up­date and con­tainer recre­ation to up­date the app in­side. With some ex­cep­tions (noted in the rel­e­vant readme.md), we do not rec­om­mend or sup­port up­dat­ing apps in­side the con­tainer. Please con­sult the Application Setup sec­tion above to see if it is rec­om­mended for the im­age.

Below are the in­struc­tions for up­dat­ing con­tain­ers:

* You can also re­move the old dan­gling im­ages:

* Recreate a new con­tainer with the same docker run pa­ra­me­ters as in­structed above (if mapped cor­rectly to a host folder, your /config folder and set­tings will be pre­served)

* You can also re­move the old dan­gling im­ages:

If you want to make lo­cal mod­i­fi­ca­tions to these im­ages for de­vel­op­ment pur­poses or just to cus­tomize the logic:

The ARM vari­ants can be built on x86_64 hard­ware and vice versa us­ing lscr.io/​lin­uxserver/​qemu-sta­tic

Once reg­is­tered you can de­fine the dock­er­file to use with -f Dockerfile.aarch64.

To help with de­vel­op­ment, we gen­er­ate this de­pen­dency graph.

* 26.09.24: - Swap from fire­fox to chromium on Alpine im­ages.

* 29.12.23: - Rebase Alpine to 3.19 and swap back to Firefox.

* 05.02.22: - Rebase KDE Ubuntu to Jammy, add new doc­u­men­ta­tion for up­dated gclient, stop rec­om­mend­ing priv mode.