Donations are a key part of what keeps F-Droid in­de­pen­dent and re­li­able and our lat­est hard­ware up­date is a di­rect re­sult of your sup­port. Thanks to do­na­tions from our in­cred­i­ble com­mu­nity, F-Droid has re­placed one of its most crit­i­cal pieces of in­fra­struc­ture, our core server hard­ware. It was over­due for a re­fresh, and now we are happy to give you an up­date on the new server and how it im­pacts the pro­ject.

This up­grade touches a core part of the in­fra­struc­ture that builds and pub­lishes apps for the main F-Droid repos­i­tory. If the server is slow, every­thing down­stream gets slower too. If it is healthy, the en­tire ecosys­tem ben­e­fits.

This server re­place­ment took a bit longer than we would have liked. The biggest rea­son is that sourc­ing re­li­able parts right now is gen­uinely hard. Ongoing global trade ten­sions have made sup­ply chains un­pre­dictable, and that hit the spe­cific com­po­nents we needed. We had to wait for quotes, re­view, re­plan, and wait again when quotes turned out to have un­ex­pected long waits, be­fore we fi­nally man­aged to re­ceive hard­ware that met our re­quire­ments.

Even with the de­lays, the pri­or­ity never changed. We were look­ing for the right server set up for F-Droid, built to last for the long haul.

Another im­por­tant part of this story is where the server lives and how it is man­aged. F-Droid is not hosted in just any data cen­ter where com­mod­ity hard­ware is man­aged by some un­known staff. We worked out a spe­cial arrange­ment so that this server is phys­i­cally held by a long time con­trib­u­tor with a proven track record of se­curely host­ing ser­vices. We can con­trol it re­motely, we know ex­actly where it is, and we know who has ac­cess. That level of trans­parency and trust is not com­mon in in­fra­struc­ture, but it is cen­tral to how we think about re­silience and stew­ard­ship.

This was not the eas­i­est path, and it re­quired care­ful co­or­di­na­tion and ne­go­ti­a­tion. But we are glad we did it this way. It fits our val­ues and our threat model, and it keeps the pro­ject grounded in real peo­ple rather than anony­mous sys­tems.

The pre­vi­ous server was 12 year old hard­ware and had been run­ning for about five years. In in­fra­struc­ture terms, that is a life­time. It served F-Droid well, but it was reach­ing the point where speed and main­te­nance over­head were be­com­ing a daily bur­den.

The new sys­tem is al­ready show­ing a huge im­prove­ment. Stats of the run­ning cy­cles from the last two months sug­gest it can han­dle the full build and pub­lish ac­tions much faster than be­fore. E.g. this year, be­tween January and September, we pub­lished up­dates once every 3 or 4 days, that got down to once every 2 days in October, to every day in November and it’s reach­ing twice a day in December. (You can see this in the fre­quency of in­dex pub­lish­ing af­ter October 18, 2025 in our f-droid.org trans­parency

log). That ex­tra ca­pac­ity gives us more breath­ing room and helps shorten the gap be­tween when apps are up­dated and when those up­dates reach users. We can now build all the auto-up­dated

apps in the

(UTC) morn­ing in one cy­cle, and all the newly in­cluded apps, fixed apps and man­u­ally up­dated apps, through the day, in the evening cy­cle.

We are be­ing care­ful here, be­cause real world in­fra­struc­ture al­ways comes with sur­prises. But the per­for­mance gains are real, and they are ex­cit­ing.

This up­grade ex­ists be­cause of com­mu­nity sup­port, pooled over time, turned into real in­fra­struc­ture, ben­e­fit­ing every­one who re­lies on F-Droid.

A faster server does not just make our lives eas­ier. It helps de­vel­op­ers get timely builds. It re­duces main­te­nance risk. It strength­ens the health of the en­tire repos­i­tory.

So thank you. Every do­na­tion, whether large or small, is part of how this pro­ject stays re­li­able, in­de­pen­dent, and aligned with free soft­ware val­ues.

Weather has al­ways sig­nif­i­cantly in­flu­enced my life. When I was a young ath­lete, know­ing the fore­cast in ad­vance would have al­lowed me to bet­ter plan my train­ing ses­sions. As I grew older, I could choose whether to go to school on my mo­tor­cy­cle or, for safety rea­sons, have my grand­fa­ther drive me. And it was him, my grand­fa­ther, who was my go-to me­te­o­rol­o­gist. He fol­lowed all weather pat­terns and fore­casts, a rem­nant of his child­hood in the coun­try­side and his life on the move. It’s to him that I ded­i­cate FediMeteo.

The idea for FediMeteo started al­most by chance while I was check­ing the hol­i­day weather fore­cast to plan an out­ing. Suddenly, I thought how nice it would be to re­ceive reg­u­lar weather up­dates for my city di­rectly in my time­line. After re­flect­ing for a few min­utes, I reg­is­tered a do­main and started plan­ning.

The choice of op­er­at­ing sys­tem was al­most au­to­matic. The idea was to sep­a­rate in­stances by coun­try, and FreeBSD jails are one of the most use­ful tools for this pur­pose.

I ini­tially thought the pro­ject would gen­er­ate lit­tle in­ter­est. I was wrong. After all, weather af­fects many of our lives, di­rectly or in­di­rectly. So I de­cided to struc­ture every­thing in this way:

* I would use a test VPS to see how things would go. The VPS was a small VM on a German provider with 4 shared cores, 4GB of RAM, 120GB of SSD disk space, and a 1Gbit/sec in­ter­net con­nec­tion and now is a 4 euro per month VPS in Milano, Italy - 4 shared cores, 8 GB RAM and 75GB disk space.

* I would sep­a­rate var­i­ous coun­tries into dif­fer­ent in­stances, for both man­age­ment and se­cu­rity rea­sons, as well as to have the pos­si­bil­ity of re­lo­cat­ing just some of them if needed.

* Weather data would come from a re­li­able and open-source friendly source. I nar­rowed it down to two op­tions: wttr.in and Open-Meteo, two so­lu­tions I know and that have al­ways given me re­li­able re­sults.

* I would pay close at­ten­tion to ac­ces­si­bil­ity: fore­casts would be in lo­cal lan­guages, con­sultable via text browsers, with emo­jis to give an idea even to those who don’t speak lo­cal lan­guages, and every­thing would be ac­ces­si­ble with­out JavaScript or other re­quire­ments. One’s mother tongue is al­ways more “familiar” than a sec­ond lan­guage, even if you’re flu­ent.

* I would man­age every­thing ac­cord­ing to Unix phi­los­o­phy: small pieces work­ing to­gether. The more years pass, the more I un­der­stand how valu­able this ap­proach is.

* The soft­ware cho­sen to man­age the in­stances is snac. Snac em­bod­ies my phi­los­o­phy of min­i­mal and ef­fec­tive soft­ware, per­fect for this pur­pose. It pro­vides clear web pages for those who want to con­sult via the web, “speaks” the ActivityPub pro­to­col per­fectly, pro­duces RSS feeds for each user (i.e., city), has ex­tremely low RAM and CPU con­sump­tion, com­piles in sec­onds, and is sta­ble. The de­vel­oper is an ex­tremely help­ful and pos­i­tive per­son, and in my opin­ion, this car­ries equal weight as every­thing else.

* I would do it for my­self. If there was no in­ter­est, I would have kept it run­ning any­way, with­out ex­pand­ing it. So no anx­i­ety or fear of fail­ure.

I started set­ting up the first “pieces” dur­ing the days around Christmas 2024. The scheme was clear: each jail would han­dle every­thing in­ter­nally. A Python script would down­load data, city by city, and pro­duce mark­down. The city co­or­di­nates would be cal­cu­lated via the geopy li­brary and passed to wttr.in and Open-Meteo. No data would be stored lo­cally. This ap­proach gives the abil­ity to process all cities to­gether. Just pass the city and coun­try to the script, and the mark­down would be served. At that point, snac comes into play: with­out the need to use ex­ter­nal util­i­ties, the “snac note” com­mand al­lows post­ing from stdin by spec­i­fy­ing the in­stance di­rec­tory and the user to post from. No need to make API calls with ex­ter­nal util­i­ties, hav­ing to man­age API keys, per­mis­sions, etc.

To sim­plify things, I first struc­tured the jail for Italy. I made a list of the main cities, nor­mal­iz­ing them. For ex­am­ple, La Spezia be­came la_spezia. Forlì, with an ac­cent, be­came forli - this for max­i­mum com­pat­i­bil­ity since each city would be a snac user. I then cre­ated a script that takes this list and cre­ates snac users via “snac ad­duser.” At that point, af­ter cre­at­ing all the users, the script would mod­ify the JSON of each user to con­vert the city name to up­per­case, in­sert the bio (a stan­dard text), ac­ti­vate the “bot” flag, and set the avatar, which was the same for all users at the time. This script is also able to add a new city: just run the script with the (normalized) name of the city, and it will add it - also adding it to the “cities.txt” file, so it will be up­dated in the next weather up­date cy­cle.

I then cre­ated the heart of the ser­vice. A Python ap­pli­ca­tion (initially only in Italian, then mul­ti­lin­gual, sep­a­rat­ing the op­er­a­tional part from the text) able to re­ceive (via com­mand line) the name of a city and a coun­try code (corresponding to the file with texts in the lo­cal lan­guage). The script de­ter­mines the co­or­di­nates and then, us­ing API calls, re­quests the cur­rent weather con­di­tions, those for the next 12 hours, and the next 7 days. I con­ducted ex­per­i­ments with both wttr.in and Open-Meteo, and both gave good re­sults. However, I set­tled on Open-Meteo be­cause, for my uses, it has al­ways pro­vided very re­li­able re­sults. This ap­pli­ca­tion di­rectly pro­vides an out­put in Markdown since snac sup­ports it, at least par­tially.

The cities.txt file is also cru­cial for up­dates. I cre­ated a script - post.sh, in pure sh, that scrolls through all cities, and for each one, launches the FediMeteo ap­pli­ca­tion and pub­lishes its out­put us­ing snac di­rectly via com­mand line. Once the job is fin­ished, it makes a call to my in­stance of Uptime-Kuma, which keeps an eye on the sit­u­a­tion. In case of fail­ure, the mon­i­tor­ing will alert me that there have been no re­cent up­dates, and I can check.

At this point, the sys­tem cron takes care of launch­ing post.sh every 6 hours. The re­quests are se­ri­al­ized, so the cities will up­date one at a time, and the posts will be sent to fol­low­ers.

After list­ing all Italian provin­cial cap­i­tals, I started test­ing every­thing. It worked per­fectly. Of course, I had to make some ad­just­ments at all lev­els. For ex­am­ple, one of the prob­lems en­coun­tered was that snac did not set the lan­guage of the posts, and some users could have missed them. The de­vel­oper was very quick and, as soon as I ex­posed the prob­lem, im­me­di­ately mod­i­fied the pro­gram so that the post could keep the sys­tem lan­guage, set as an en­vi­ron­ment vari­able in the sh script.

After two days, I de­cided to start adding other coun­tries and an­nounce the pro­ject. And the an­nounce­ment was un­ex­pect­edly well re­ceived: there were many boosts, and peo­ple started ask­ing me to add their cities or coun­tries. I tried to do what I could, within the lim­its of my phys­i­cal con­di­tion, as in those days, I had the flu that kept me at home with a fever and ill­ness for sev­eral days. I started adding many coun­tries in the heart of Europe, trans­lat­ing the main in­di­ca­tions into lo­cal lan­guages but main­tain­ing emo­jis so that every­thing would be un­der­stand­able even to those who don’t speak the lo­cal lan­guage. There were some small prob­lems re­ported by some users. One of them: not all weather con­di­tions had been trans­lated, so some­times they ap­peared in Italian - as well as er­rors. In bilin­gual coun­tries, I tried to in­clude all lo­cal lan­guages. Sometimes, un­for­tu­nately, mak­ing mis­takes as I en­coun­tered dy­nam­ics un­known to me or dif­fi­cult to in­ter­pret. For ex­am­ple, in Ireland, fore­casts were pub­lished in Irish, but it was pointed out to me that not every­one speaks it, so I mod­i­fied and pub­lished in English.

The turn­ing point was when FediFollows (@FediFollows@social.growyourown.ser­vices - who also man­ages the site Fedi Directory) started pub­lish­ing the list of coun­tries and cities, high­light­ing the pro­ject. Many peo­ple be­came aware of FediMeteo and started fol­low­ing the var­i­ous ac­counts, the var­i­ous cities. And from here came re­quests to add new coun­tries and some new in­for­ma­tion, such as wind speed. Moreover, I was asked (rightly, to avoid flood­ing time­lines) to pub­lish posts as un­listed - this way, fol­low­ers would see the posts, but they would­n’t fill lo­cal time­lines. Snac did­n’t sup­port this, but again, the snac dev came to my res­cue in a few hours.

But with new coun­tries came new chal­lenges. For ex­am­ple, in my orig­i­nal im­ple­men­ta­tion, all units of mea­sure­ment were in met­ric/​dec­i­mal/​Cel­sius - and this does­n’t adapt well to re­al­i­ties like the USA. Moreover, fo­cus­ing on Europe, al­most all coun­tries were lo­cated in a sin­gle time­zone, while for larger coun­tries (such as Australia, USA, Canada, etc.), this is to­tally dif­fer­ent. So I started de­vel­op­ing a more com­plete and global ver­sion and, in the mean­time, added al­most all of Europe. The new ver­sion would have to be back­ward com­pat­i­ble, would have to take into ac­count time­zone dif­fer­ences for each city, dif­fer­ent mea­sure­ments (e.g., de­grees C and F), as well as, ini­tially more dif­fi­cult part, be­ing able to sep­a­rate cities with the same name based on states or provinces. I had al­ready seen a sim­i­lar prob­lem with the im­ple­men­ta­tion of sup­port for Germany, so it had to be ad­dressed prop­erly.

The orig­i­nal goal was to have a VPS for each con­ti­nent, but I soon re­al­ized that thanks to the qual­ity of snac’s code and FreeBSD’s ef­fi­cient man­age­ment, even keep­ing coun­tries in sep­a­rate jails, the load did­n’t in­crease much. So I de­cided to chal­lenge my­self and the lim­its of the eco­nom­i­cal 4 eu­ros per month VPS. That is, to in­sert as much as pos­si­ble un­til see­ing what the lim­its were. Limits that, to date, I have not yet reached. I would also soon ex­haust the avail­able API calls for Open-Meteo’s free ac­counts, so I tried to con­tact the team and ex­plain every­thing. I was pos­i­tively sur­prised to read that they ap­pre­ci­ated the pro­ject and pro­vided me with a ded­i­cated API key.

Compatible with my free time, I man­aged to com­plete the richer and more com­plete ver­sion of my Python pro­gram. I’m not a pro­fes­sional dev, I’m more ori­ented to­wards sys­tems, so the code is prob­a­bly quite poor in the eyes of an ex­pert dev. But, in the end, it just needs to take an in­put and give me an out­put. It’s not a dae­mon, it’s not a ser­vice that re­sponds on the net­work. For that, snac takes care of it.

So I de­cided to start with a very im­por­tant launch: the USA and Canada. A non-triv­ial part was iden­ti­fy­ing the main cities in or­der to cover, state by state, all the ter­ri­tory. In the end, I iden­ti­fied more than 1200 cities. A num­ber that, by it­self, ex­ceeded the sum of all other coun­tries (at that time). And the pro­gram, now, is able to take an in­put with a sep­a­ra­tor (two un­der­scores: __) be­tween city and state. In this way, it’s pos­si­ble to per­fectly un­der­stand the dif­fer­ences be­tween city and state: new_y­ork__new_y­ork is an ex­am­ple I like to make, but there are many.

The launch of the USA was in­ter­est­ing: de­spite hav­ing had many pre­vi­ous re­quests, the re­cep­tion was ini­tially quite luke­warm, to my ex­treme sur­prise. The num­ber of fol­low­ers in Canada, in a few hours, far ex­ceeded that of the USA. On the con­trary, the coun­try with the most fol­low­ers (in a few days, more than 1000) was Germany. Followed by the UK - which I ex­pected would have been the first.

The VPS held up well. Except for the mo­ments when FediFollows launched (after fix­ing some FreeBSD tun­ing, the ser­vice slowed slightly but did­n’t crash), the load re­mained ex­tremely low. So I con­tin­ued to ex­pand: Japan, Australia, New Zealand, etc.

At the time of the last up­date of this ar­ti­cle (30 December 2025), the sup­ported coun­tries are 38: Argentina, Australia, Austria, Belgium, Brazil, Bulgaria, Canada, Croatia, Czechia, Denmark, Estonia, Finland, France, Germany, Greece, Hungary, India, Ireland, Italy, Japan, Latvia, Lithuania, Malta, Mexico, Netherlands, New Zealand, Norway, Poland, Portugal, Romania, Slovakia, Slovenia, Spain, Sweden, Switzerland, Taiwan, the United Kingdom, and the United States of America (with more re­gions com­ing soon!).

Direct fol­low­ers in the Fediverse are around 7,707 and grow­ing daily, ex­clud­ing those who fol­low hash­tags or cities via RSS, whose num­ber I can’t es­ti­mate. However, a quick look at the logs sug­gests there are many more.

The cities cur­rently cov­ered are 2937 - grow­ing based on new coun­tries and re­quests.

There have been some prob­lems. The most se­ri­ous, by my fault, was the API key leak: I had left a de­bug code ac­tive and, the first time Open-Meteo had prob­lems, the er­ror mes­sage also in­cluded the API call - in­clud­ing the API key. Some users re­ported it to me (others just mocked) and I fixed the code and im­me­di­ately re­ported every­thing to the Open-Meteo team, who kindly gave me a new API Key and de­ac­ti­vated the old one.

A fur­ther prob­lem was re­lated to geopy. It makes a call to Nominatim to de­ter­mine co­or­di­nates. One of the times Nominatim did­n’t re­spond, my pro­gram was­n’t able to de­ter­mine the po­si­tion and went into er­ror. I solved this by in­tro­duc­ing co­or­di­nate caching: now the pro­gram, the first time it en­coun­ters a city, re­quests and saves the co­or­di­nates. If pre­sent, they will be used in the fu­ture with­out mak­ing a new re­quest via geopy. This is both lighter on their servers and faster and safer for us.

And the VPS? It has no prob­lems and is sur­pris­ingly fast and ef­fec­tive. FreeBSD 14.3-RELEASE, BastilleBSD to man­age the jails. Currently, there are 39 jails - one for haproxy, the FediMeteo web­site, so ng­inx, and the snac in­stance for FediMeteo an­nounce­ments and sup­port - the other 38 for the in­di­vid­ual in­stances. Each of them, there­fore, has its au­tonomous ZFS dataset. Every 15 min­utes, there is a lo­cal snap­shot of all datasets. Every hour, the home­page is re­gen­er­ated: a small script cal­cu­lates the num­ber of fol­low­ers (counting, in­stance by in­stance, the fol­low­ers of in­di­vid­ual cities, since I don’t pub­lish ex­cept in ag­gre­gate to avoid pos­si­ble tri­an­gu­la­tions and pri­vacy leaks of users). Every hour, more­over, an ex­ter­nal backup is made via zfs-au­to­backup (on en­crypted at rest dataset), and once a day, a fur­ther backup is made in my dat­a­cen­ter, on disks en­crypted with geli. The oc­cu­pied RAM is 501 MB (yes, ex­actly: 501 MB), which rises slightly when up­dates are in progress. Updates nor­mally oc­cur every 6 hours. I have tried, as much as pos­si­ble, to space them out to avoid over­loads in time­lines (or on the server it­self). Only for the USA, I added a sleep of 5 sec­onds be­tween one city and an­other, to give snac the op­por­tu­nity to bet­ter or­ga­nize the send­ing of mes­sages. It prob­a­bly would­n’t be nec­es­sary, with the cur­rent num­bers, but bet­ter safe than sorry. In this way, the USA is processed in about 2 and a half hours, but the other jails (thus coun­tries) can work au­tonomously and send their up­dates.

The av­er­age load of the VPS (taking as ref­er­ence both the last 24 hours and the last two weeks) is about 25%, as it rises to 70/75% when up­dates oc­cur for larger in­stances (such as the USA), or when it is an­nounced by FediFollows. Otherwise, it is on av­er­age less than 10%. So, the VPS still has huge mar­gin, and new in­stances, with new na­tions, will still be in­side it.

This ar­ti­cle, al­though in some parts very con­ver­sa­tional, aims to demon­strate how it’s pos­si­ble to build solid, valid, and ef­fi­cient so­lu­tions with­out the need to use ex­pen­sive and com­plex ser­vices. Moreover, this is the demon­stra­tion of how it’s pos­si­ble to have your on­line pres­ence with­out the need to put your data in the hands of third par­ties or with­out nec­es­sar­ily hav­ing to re­sort to com­plex stacks. Sometimes, less is more.

The suc­cess of this pro­ject demon­strates, once again, that my grand­fa­ther was right: weather fore­casts in­ter­est every­one. He wor­ried about my health and, thanks to his con­cerns, we spent time to­gether. In the same way, I see many fol­low­ers and friends talk­ing to me or among them­selves about the weather, their ex­pe­ri­ences, what hap­pens. Again, in my life, weather fore­casts have helped so­cial­ity and so­cial­iza­tion.

Libsodium is now 13 years old!

I started that pro­ject to pur­sue Dan Bernstein’s de­sire to make cryp­tog­ra­phy sim­ple to use. That meant ex­pos­ing a lim­ited set of high-level func­tions and pa­ra­me­ters, pro­vid­ing a sim­ple API, and writ­ing doc­u­men­ta­tion for users, not cryp­tog­ra­phers. Libsodium’s goal was to ex­pose APIs to per­form op­er­a­tions, not low-level func­tions. Users should­n’t even have to know or care about what al­go­rithms are used in­ter­nally. This is how I’ve al­ways viewed lib­sodium.

Never break­ing the APIs is also some­thing I’m ob­sessed with. APIs may not be great, and if I could start over from scratch, I would have made them very dif­fer­ent, but as a de­vel­oper, the best APIs are not the most beau­ti­fully de­signed ones, but the ones that you don’t have to worry about be­cause they don’t change and up­grades don’t re­quire any changes in your ap­pli­ca­tion ei­ther. Libsodium started from the NaCl API, and still ad­heres to it.

These APIs ex­posed high-level func­tions, but also some lower-level func­tions that high-level func­tions wrap or de­pend on. Over the years, peo­ple started us­ing these low-level func­tions di­rectly. Libsodium started to be used as a toolkit of al­go­rithms and low-level prim­i­tives.

That made me sad, es­pe­cially since it is clearly doc­u­mented that only APIs from builds with –enable-minimal are guar­an­teed to be tested and sta­ble. But af­ter all, it makes sense. When build­ing cus­tom pro­to­cols, hav­ing a sin­gle portable li­brary with a con­sis­tent in­ter­face for dif­fer­ent func­tions is far bet­ter than im­port­ing mul­ti­ple de­pen­den­cies, each with their own APIs and some­times in­com­pat­i­bil­i­ties be­tween them.

That’s a lot of code to main­tain. It in­cludes fea­tures and tar­get plat­forms I don’t use but try to sup­port for the com­mu­nity. I also main­tain a large num­ber of other open source pro­jects.

Still, the se­cu­rity track record of lib­sodium is pretty good, with zero CVEs in 13 years even though it has got­ten a lot of scrutiny.

However, while re­cently ex­per­i­ment­ing with adding sup­port for batch sig­na­tures, I no­ticed in­con­sis­tent re­sults with code orig­i­nally writ­ten in Zig. The cul­prit was a check that was pre­sent in a func­tion in Zig, but that I for­got to add in lib­sodium.

The func­tion cryp­to_­core_ed25519_is_­valid_­point(), a low-level func­tion used to check if a given el­lip­tic curve point is valid, was sup­posed to re­ject points that aren’t in the main cryp­to­graphic group, but some points were slip­ping through.

Edwards25519 is like a spe­cial math­e­mat­i­cal play­ground where cryp­to­graphic op­er­a­tions hap­pen.

It is used in­ter­nally for Ed25519 sig­na­tures, and in­cludes mul­ti­ple sub­groups of dif­fer­ent sizes (order):

* Order L: the “main sub­group” (L = ~2^252 points) where all op­er­a­tions are ex­pected to hap­pen

* Order 2L, 4L, 8L: very large, but not prime or­der sub­groups

The val­i­da­tion func­tion was de­signed to re­ject points not in the main sub­group. It prop­erly re­jected points in the small-or­der sub­groups, but not points in the mixed-or­der sub­groups.

To check if a point is in the main sub­group (the one of or­der L), the func­tion mul­ti­plies it by L. If the or­der is L, mul­ti­ply­ing any point by L gives the iden­tity point (the math­e­mat­i­cal equiv­a­lent of zero). So, the code does the mul­ti­pli­ca­tion and checks that we ended up with the iden­tity point.

Points are rep­re­sented by co­or­di­nates. In the in­ter­nal rep­re­sen­ta­tion used here, there are three co­or­di­nates: X, Y, and Z. The iden­tity point is rep­re­sented in­ter­nally with co­or­di­nates where X = 0 and Y = Z. Z can be any­thing de­pend­ing on pre­vi­ous op­er­a­tions; it does­n’t have to be 1.

The old code only checked X = 0. It for­got to ver­ify Y = Z. This meant some in­valid points (where X = 0 but Y ≠ Z af­ter the mul­ti­pli­ca­tion) were in­cor­rectly ac­cepted as valid.

Concretely: take any main-sub­group point Q (for ex­am­ple, the out­put of cryp­to_­core_ed25519_ran­dom) and add the or­der-2 point (0, -1), or equiv­a­lently negate both co­or­di­nates. Every such Q + (0, -1) would have passed val­i­da­tion be­fore the fix, even though it’s not in the main sub­group.

The fix is triv­ial and adds the miss­ing check:

Now it prop­erly ver­i­fies both con­di­tions: X must be zero and Y must equal Z.

You may be af­fected if you:

* Use a point re­lease <= 1.0.20 or a ver­sion of lib­sodium re­leased be­fore December 30, 2025.

* Use cryp­to_­core_ed25519_is_­valid_­point() to val­i­date points from un­trusted sources

* Implement cus­tom cryp­tog­ra­phy us­ing arith­metic over the Edwards25519 curve

But don’t panic. Most users are not af­fected.

None of the high-level APIs (crypto_sign_*) are af­fected; they don’t even use or need that func­tion. Scalar mul­ti­pli­ca­tion us­ing cryp­to_s­calar­mult_ed25519 won’t leak any­thing even if the pub­lic key is not on the main sub­group. And pub­lic keys cre­ated with the reg­u­lar cryp­to_sign_key­pair and cryp­to_sign_seed_key­pair func­tions are guar­an­teed to be on the cor­rect sub­group.

Support for the Ristretto255 group was added to lib­sodium in 2019 specif­i­cally to solve co­fac­tor-re­lated is­sues. With Ristretto255, if a point de­codes, it’s safe. No fur­ther val­i­da­tion is re­quired.

If you im­ple­ment cus­tom cryp­to­graphic schemes do­ing arith­metic over a fi­nite field group, us­ing Ristretto255 is rec­om­mended. It’s eas­ier to use, and as a bonus, low-level op­er­a­tions will run faster than over Edwards25519.

If you can’t up­date lib­sodium and need an ap­pli­ca­tion-level workaround, use the fol­low­ing func­tion:

This is­sue was fixed im­me­di­ately af­ter dis­cov­ery. All sta­ble pack­ages re­leased af­ter December 30, 2025 in­clude the fix:

A new point re­lease is also go­ing to be tagged.

If lib­sodium is use­ful to you, please keep in mind that it is main­tained by one per­son, for free, in time I could spend with my fam­ily or on other pro­jects. The best way to help the pro­ject would be to con­sider spon­sor­ing it, which helps me ded­i­cate more time to im­prov­ing it and mak­ing it great for every­one, for many more years to come.

MegaLag’s December 2024 video in­tro­duced 18 mil­lion view­ers to se­ri­ous ques­tions about Honey, the widely-used browser shop­ping plug-in—in par­tic­u­lar, whether Honey abides by the rules set by af­fil­i­ate net­works and mer­chants, and whether Honey takes com­mis­sions that should flow to other af­fil­i­ates.  I wrote in January that I thought Honey was out of line.  In par­tic­u­lar, I pointed out the con­tracts that limit when and how Honey may pre­sent af­fil­i­ate links, and I ap­plied those con­tracts to the be­hav­ior MegaLag doc­u­mented.  Honey was plainly break­ing the rules.

As it turns out, Honey’s mis­con­duct is con­sid­er­ably worse than MegaLag, I, or oth­ers knew.  When Honey is con­cerned that a user may be a tester—a “network qual­ity” em­ployee, a mer­chan­t’s af­fil­i­ate man­ager, an af­fil­i­ate, or an en­thu­si­ast—Honey de­signs its soft­ware to honor stand down in full.  But when Honey feels con­fi­dent that it’s be­ing used by an or­di­nary user, Honey de­fies stand down rules.  Multiple meth­ods sup­port these con­clu­sions: I ex­tracted source code from Honey’s browser plu­gin and stud­ied it at length, plus I ran Honey through a packet snif­fer to col­lect its con­fig files, and I cross-checked all of this with ac­tual app be­hav­ior.  Details be­low.  MegaLag tested too, and has a new video with his up­dated as­sess­ment.

Behaving bet­ter when it thinks it’s be­ing tested, Honey fol­lows in Volkswagen’s “Dieselgate” foot­steps.  Like Volkswagen, the cover-up is ar­guably worse than the un­der­ly­ing con­duct.  Facing the al­le­ga­tions MegaLag pre­sented last year, Honey could try to de­fend pre­sent­ing its af­fil­i­ate links willy-nilly—ar­gue users want this, claim to be sav­ing users money, sug­gest that net­work rules don’t ap­ply or don’t mean what they say.  But these new al­le­ga­tions are more dif­fi­cult to de­fend.  Designing its soft­ware to per­form dif­fer­ently when un­der test, Honey re­veals know­ing what the rules re­quire and know­ing they’d be in trou­ble if caught.  Hiding from testers re­veals that Honey wanted to pre­sent af­fil­i­ate links as widely as pos­si­ble, de­spite the rules, so long as it does­n’t get caught.  It’s not a good look.  Affiliates, mer­chants, and net­works should be fu­ri­ous.

The ba­sic bar­gain of af­fil­i­ate mar­ket­ing is that a pub­lisher pre­sents a link to a user, who clicks, browses, and buys.  If the user makes a pur­chase, com­mis­sion flows to the pub­lisher whose link was last clicked.

Shopping plu­g­ins and other client-side soft­ware un­der­mine the ba­sic bar­gain of af­fil­i­ate mar­ket­ing.  If a pub­lisher puts soft­ware on a user’s com­puter, that soft­ware can mon­i­tor where the user browses, pre­sent its af­fil­i­ate link, and al­ways (appear to) be “last”—even if it had min­i­mal role in in­flu­enc­ing the cus­tomer’s pur­chase de­ci­sion.

Affiliate net­works and mer­chants es­tab­lished rules to re­store and pre­serve the bar­gain be­tween what we might call “web af­fil­i­ates” ver­sus soft­ware af­fil­i­ates.  One, a user has to ac­tu­ally click a soft­ware af­fil­i­ate’s link; decades ago, auto-clicks were com­mon, but that’s long-since banned (yet nonethe­less rou­tine from “adware”-style browser plu­g­ins—  ex­am­ple).  Two, soft­ware must “stand down”—must not even show its link to users—when some prior web af­fil­i­ate P has al­ready re­ferred a user to a given mer­chant.  This re­flects a bal­anc­ing of in­ter­ests: P wants a rea­son­able op­por­tu­nity for the user to make a pur­chase, so P can get paid.  If a shop­ping plu­gin could al­ways pre­sent its of­fer, the shop­ping plu­gin would claim the com­mis­sion that P had fairly earned.  Meanwhile P would­n’t get suf­fi­cient pay­ment for its ef­fort—and might switch to pro­mot­ing some other mer­chant with rules P sees as more fa­vor­able.  Merchants and net­works need to main­tain a bal­ance in or­der to at­tract and re­tain web af­fil­i­ates, which are un­der­stood to send traf­fic that’s sub­stan­tially in­cre­men­tal (customers who would­n’t have pur­chased any­way), whereas shop­ping plu­g­ins of­ten take credit for non­in­cre­men­tal pur­chases.  So if a mer­chant is un­sure, it has good rea­son to err on the side of web af­fil­i­ates.

All of this was known and un­der­stood lit­er­ally decades ago.  Stand-down rules were first es­tab­lished in 2002.  Since then, they’ve been in­creas­ingly rou­tine, and over­all have be­come clearer and bet­ter en­forced.  Crucially, mer­chants and net­works in­clude stand-down rules in their con­tracts, mak­ing this not just a prin­ci­ple and a norm, but a bind­ing con­trac­tual oblig­a­tion.

How can Honey tell when a user may be a tester?  Honey’s code and con­fig files show that they’re us­ing four cri­te­ria:

* New ac­counts. If an ac­count is less than 30 days old, Honey con­cludes the user might be a tester, so it dis­ables its pro­hib­ited be­hav­ior.

* Low earn­ings-to-date. In gen­eral, un­der Honey’s cur­rent rules, if an ac­count has less than 65,000 points of Honey earn­ing, Honey con­cludes the user might be a tester, so it dis­ables its pro­hib­ited be­hav­ior.  Since 1,000 points can be re­deemed for $10 of gift cards, this thresh­old re­quires hav­ing earned $650 worth of points.  That sounds like a high re­quire­ment, and it is.  But it’s ac­tu­ally rel­a­tively new: As of June 2022, there was no points re­quire­ment for most mer­chants, and for mer­chants in Rakuten Advertising, the re­quire­ment was just 501 points (about $5 of points).  (Details be­low.)

* Honey pe­ri­od­i­cally checks a server-side black­list.  The server can con­di­tion its de­ci­sion on any fac­tor known to the server, in­clud­ing the user’s Honey ID and cookie, or IP ad­dress in­side a ge­ofence or on a ban list.  Suppose the user has sub­mit­ted prior com­plaints about Honey, as pro­fes­sional testers fre­quently do.  Honey can black­list the user ID, cookie, and IP or IP range.  Then any fur­ther re­quests from that user, cookie, or IP will be treated as high-risk, and Honey dis­ables its pro­hib­ited be­hav­ior.

* Affiliate in­dus­try cook­ies. Honey checks whether a user has cook­ies in­di­cat­ing hav­ing logged into key af­fil­i­ate in­dus­try tools, in­clud­ing the CJ, Rakuten Advertising, and Awin dash­boards.  If the user has such a cookie, the user is par­tic­u­larly likely to be a tester, so Honey dis­ables its pro­hib­ited be­hav­ior.

If even one of these fac­tors in­di­cates a user is high-risk, Honey hon­ors stand-down.  But if all four pass, then Honey ig­nores stand-down rules and pre­sents its af­fil­i­ate links re­gard­less of a prior web pub­lish­er’s role and re­gard­less of stand-down rules.  This is­n’t a prob­a­bilis­tic or un­cer­tain dis­hon­or­ing of stand-down (as plain­tiffs posited in lit­i­ga­tion against Honey).  Rather, Honey’s ac­tions are de­ter­min­is­tic: If a high-risk fac­tor hits, Honey will com­pletely and in every in­stance honor stand-down; and if no such fac­tor hits, then Honey will com­pletely and in every in­stance dis­honor stand-down (meaning, pre­sent its link de­spite net­works’ rules).

These cri­te­ria in­di­cate Honey’s at­tempt to ob­struct and frankly frus­trate testers.  In my ex­pe­ri­ence from two decades of test­ing af­fil­i­ate mis­con­duct, it is rou­tine for a tester to in­stall a new shop­ping plu­gin on a new PC, cre­ate a new ac­count, and check for im­me­di­ate wrong­do­ing.  By al­ways stand­ing down on new ac­counts (

The re­quire­ment of achiev­ing suf­fi­ciently many points sim­i­larly stymies many testers.  Over the years, a hand­ful of my clients granted a bud­get for test pur­chases.  But most did not.  If I wanted to per­form test pur­chases, I would have needed to spend my own money.  Meanwhile, there’s no proper rea­son why a shop­ping app should stand down dif­fer­ently for users with points ver­sus new users, so if testers as­sumed good faith by Honey, they’d have no rea­son to build up a points his­tory.  This cri­te­ria cer­tainly pre­vented testers from un­cov­er­ing what Honey was do­ing.

The in­ter­ac­tion of ac­count age, points his­tory, and black­list fur­ther stymies test­ing.  Once a tester re­ports a spe­cific ob­ser­va­tion of mis­con­duct, Honey can black­list the tester’s ac­count—which will still do most things nor­mally, but will be­gin to stand down even though un­banned ac­counts don’t stand down in the same cir­cum­stances.  A tester who sus­pected this, would have to cre­ate a new ac­count, wait 30 days, and build up a siz­able points bal­ance to see what Honey is ac­tu­ally do­ing for other users.  That’s a gen­uine bar­rier, im­ped­ing even a tester who cor­rectly pro­poses this hy­poth­e­sis.

Honey’s de­tec­tion of af­fil­i­ate in­dus­try cook­ies is par­tic­u­larly re­veal­ing and, in my view, dev­as­tat­ingly in­crim­i­nat­ing.  Maybe Honey could con­coct a story about why it stands down less for es­tab­lished ac­counts and for high-points-earn­ers.  It would be a tall tale, but maybe Honey could con­vince some peo­ple.  In con­trast, it is in­con­ceiv­able why a shop­ping plu­gin should care whether a user pre­vi­ously logged into an af­fil­i­ate dash­board.  There’s  only one rea­son to check: to hide from testers.  But that re­veals Honey’s in­tent and Honey’s knowl­edge that its con­duct is pro­hib­ited.

Multiple forms of ev­i­dence sup­port my find­ing of Honey de­tect­ing testers.  First, con­sider hands-on test­ing.  With a stan­dard test ac­count with few or no points, Honey hon­ored stand-down.  See video 1.  But when I tricked the Honey plu­gin into think­ing I had tens of thou­sands of points (details be­low about how I did this), Honey popped up de­spite stand-down rules.  See video 2.  I re­peated this test over mul­ti­ple days, as to mul­ti­ple mer­chants.  The find­ing was the same every time.  The only thing I changed be­tween the “video 1” tests and “video 2” tests was the num­ber of points sup­pos­edly as­so­ci­ated with my ac­count.

To demon­strate Honey check­ing for af­fil­i­ate in­dus­try cook­ies, I added a step to my test sce­nario. With Honey tricked into think­ing I had am­ple points, same as video 2, I be­gan a test run by log­ging into a CJ por­tal used by af­fil­i­ates.  In all other re­spects, my test run was the same as video 2.  Seeing the CJ por­tal cookie, Honey stood down.  See video 3.

Some might ask whether the find­ings in the prior sec­tion could be co­in­ci­dence.  Maybe Honey just hap­pened to open in some sce­nar­ios and not oth­ers.  Maybe I’m as­crib­ing in­ten­tion­al­ity to acts that are just co­in­ci­dence.  Let me of­fer two re­sponses to this hy­poth­e­sis.  One, my find­ings are re­peat­able, coun­ter­ing any claim of co­in­ci­dence.  Second, sep­a­rate from hands-on test­ing, three sep­a­rate types of tech­ni­cal analy­sis—con­fig files, teleme­try, and source code—all con­firm the ac­cu­racy of the prior sec­tion.

Honey re­trieves its con­fig­u­ra­tion set­tings from JSON files on a Honey server. Honey’s core stand-down con­fig­u­ra­tion is in stand­down-rules.json, while the se­lec­tive stand-down—de­clin­ing to stand down ac­cord­ing to the cri­te­ria de­scribed above—is in the sep­a­rate con­fig file ssd.json.  Here’s the con­tents of ssd.json as of October 22, 2025, with // com­ments added by me

{“ssd”: {

“base”: {

“gca”: 1, //enable af­fil­i­ate con­sole cookie check

“bl”: 1, //enable black­list check

“uP”: 65000, //min points to dis­able stand­down

“adb”: 26298469858850

“affiliates”: [“https://​www.cj.com”, “https://​www.link­share”, “https://​www.rakuten.com”, “https://​ui.awin.com”, “https://​www.swag­bucks.com”], //affiliate con­sole cookie do­mains to check

“LS”: { //override points thresh­old for LinkShare mer­chants

“uP”: 5001

“PAYPAL”: {

“uL”: 1,

“uP”: 5000001,

“adb”: 26298469858850

“ex”: { //ssd ex­cep­tions

“7555272277853494990”: { //TJ Maxx

“uP”: 5001

“7394089402903213168”: { //booking.com

“uL”: 1,

“adb”: 120000,

“uP”: 1001

“243862338372998182”: { //kayosports

“uL”: 0,

“uP”: 100000

“314435911263430900”: {

“adb”: 26298469858850

“315283433846717691”: {

“adb”: 26298469858850

“GA”: [“CONTID”, “s_vi”, “_ga”, “networkGroup”, “_gid”] //which cook­ies to check on af­fil­i­ate con­sole cookie do­mains

On its own, the ssd con­fig file is not a model of clar­ity.  But source code (discussed be­low) re­veals the mean­ing of ab­bre­vi­a­tions in ssd.  uP (yellow) refers to user points—the min­i­mum num­ber of points a user must have in or­der for Honey to dis­honor stand-down.  Note the cur­rent base (default) re­quire­ment of uP user points at least 65,000 (green), though the sub­se­quent sec­tion LS sets a lower thresh­old of just 5001 for mer­chants on the Rakuten Advertising (LinkShare) net­work.  bl set to 1 in­structs the Honey plu­gin to stand down if the server-side black­list so in­structs.

Meanwhile, the af­fil­i­ates and ex GA data struc­tures (blue), es­tab­lish the af­fil­i­ate in­dus­try cookie checks men­tioned above.  The “affiliates” en­try lists do­main where cook­ies are to be checked.  The ex GA data struc­ture lists which cookie is to be checked for each do­main.  Though these are pre­sented as two one-di­men­sional lists, Honey’s code ac­tu­ally checks them in con­junc­tion — checks the first-listed af­fil­i­ate net­work do­main for the first-listed cookie, then the sec­ond, and so forth.  One might ask why Honey stored the do­main names and cookie names in two sep­a­rate one-di­men­sional lists, rather than in a two-di­men­sional list, name-value pair, or sim­i­lar.  The ob­vi­ous an­swer is that Honey’s ap­proach kept the do­main names more dis­tant from the cook­ies on those do­mains, mak­ing its ac­tions that much harder for testers to no­tice even if they got as far as this con­fig file.

The rest of ex (red) sets ex­cep­tions to the stan­dard (“base”) ssd.  This lists five spe­cific ecom­merce sites (each ref­er­enced with an 18-digit ID num­ber pre­vi­ously as­signed by Honey) with ad­justed ssd set­tings.  For Booking.com and Kayosports, the ssd ex­cep­tions set even higher points re­quire­ments to can­cel stand­down (120,000 and 100,000 points, re­spec­tively), which I in­ter­pret as re­sponse to com­plaints from those sites.

Honey’s teleme­try is de­light­fully ver­bose and, frankly, easy to un­der­stand, in­clud­ing English ex­pla­na­tions of what data is be­ing col­lected and why.  Perhaps Google de­manded im­prove­ments as part of ap­prov­ing Honey’s sub­mis­sion to Chrome Web Store.  (Google en­forces what it calls “strict guide­lines” for col­lect­ing user data.  Rule 12: data col­lec­tion must be “necessary for a user-fac­ing fea­ture.”  The English ex­pla­na­tions are most con­sis­tent with seek­ing to show Google that Honey’s data col­lec­tion is proper and ar­guably nec­es­sary.)  Meanwhile, Honey sub­mit­ted much the same code to Apple as an iPhone app, and Apple is known to be quite strict in its app re­view.  Whatever the rea­son, Honey teleme­try re­veals some im­por­tant as­pects of what it is do­ing and why.

When a user with few points gets a stand-down, Honey re­ports that in teleme­try with the JSON data struc­ture “method”:”suspend”.  Meanwhile, the nearby JSON vari­able state gives the spe­cific ssd re­quire­ment that the user did­n’t sat­isfy—in my video 1: “state”:”uP:5001” re­port­ing that, in this test run, my Honey app had less than 5001 points, and the ssd logic there­fore de­cided to stand down.  See video 1 at 0:37-0:41, or screen­shots be­low for con­ve­nience.  (My net­work trac­ing tool con­verted the teleme­try from plain­text to a JSON tree for read­abil­ity.)

When I gave my­self more points (video 2), state in­stead re­ported ssd—in­di­cat­ing that all ssd cri­te­ria were sat­is­fied, and Honey pre­sented its of­fer and did not stand down.  See video 2 at 0:32.

Finally, when I browsed an af­fil­i­ate net­work con­sole and al­lowed its cookie to be placed on my PC, Honey teleme­try re­ported “state”:“gca”.  Like video 1, the state value re­ports that ssd cri­te­ria were not sat­is­fied, in this case be­cause the gca (affiliate dash­board cookie) re­quire­ment was trig­gered, caus­ing ssd to de­cide to stand down.  See video 3 at 1:04-1:14.

In each in­stance, the teleme­try matched iden­ti­fiers from the con­fig file (ssd, uP, gca).  And as I changed from one test run to an­other, the teleme­try trans­mis­sions tracked my un­der­stand­ing of Honey’s op­er­a­tion.  Readers can check this in my videos: After Honey does or does­n’t stand down, I opened Fiddler to show what Honey re­ported in teleme­try, in each in­stance in one con­tin­u­ous video take.

As a browser ex­ten­sion, Honey pro­vides client-side code in JavaScript.  Google’s Code Readability Requirements al­low mini­fi­ca­tion—re­mov­ing white­space, short­en­ing vari­able and func­tion names.  Honey’s code is sub­stan­tial—af­ter dem­i­ni­fi­ca­tion, more than 1.5 mil­lion lines.  But a dili­gent an­a­lyst can still find what’s rel­e­vant.  In fact the rel­e­vant parts are clus­tered to­gether, and eas­ily found via searches for ob­vi­ous string such as “ssd”.

In a sur­pris­ing twist, Honey in one in­stance re­leased some­thing ap­proach­ing orig­i­nal code to Apple as  an iPhone app.  In par­tic­u­lar, Honey in­cluded sourceMap­pin­gURL meta­data that al­lows an an­a­lyst to re­cover orig­i­nal func­tion names and vari­able names.  (Instructions.)  That re­lease was from a mo­ment in time, and Honey sub­se­quently made re­vi­sions.  But where that code is sub­stan­tially the same as the code cur­rently in use, I pre­sent the un­ob­fus­cated ver­sion for read­ers’ con­ve­nience.  Here’s how it works:

re­turn e.next = 7, fetch(“”.con­cat(“https://​s.join­honey.com”, “/ck/alive”));

If the kill­switch re­turns “alive”, Honey sets the bl value to 0:

c = S().then((function(e) {

e && “alive” === e.is && (o.bl = 0)

The ssd logic later checks this vari­able bl, among oth­ers, to de­cide whether to can­cel stand­down.

The core ssd logic is in a long func­tion called R() which runs an in­fi­nite loop with a switch syn­tax to pro­ceed through a se­ries of num­bered cases.

func­tion(e) {

for (;;) switch (e.prev = e.next) {

Focusing on the sec­tions rel­e­vant to the be­hav­ior de­scribed above: Honey makes sure the user’s email ad­dress does­n’t in­clude the string “test”, and checks whether the user is on the kill­switch black­list.

if (r.email && r.email.match(“test”) && (o.bl = 0), !r.isLoggedIn || t) {

e.next = 7;

break

Honey com­putes the age of the user’s ac­count by sub­tract­ing the ac­count cre­ation date (r.created) from the cur­rent time:

case 8:

o.uL = r.is­LoggedIn ? 1 : 0, o.uA = Date.now() - r.cre­ated;

Honey checks for the most re­cent time a re­source was blocked by an ad blocker:

case 20:

re­turn p = e.sent, l && a.A.getAdbTab(l) ? o.adb = a.A.getAdbTab(l) : a.A.get­State().re­source­Last­Blocke­dAt > 0 ? o.adb = a.A.get­State().re­source­Last­Blocke­dAt : o.adb = 0

Honey checks whether any of the af­fil­i­ate do­mains listed in the ssd af­fil­i­ates data struc­ture has the con­sole cookie named in the GA data struc­ture.

m = p.ex && p.ex.GA || []

g = i().map(p.ssd && p.ssd.af­fil­i­ates, (function(e) {

re­turn f += 1, u.A.get({

name: m[f], //cookie name from GA ar­ray

url: e //domain to be checked

}).then((function(e) {

e && (o.gca = 0) //if cookie found, set gca to 0

Then the com­par­i­son func­tion P() com­pares each re­trieved or cal­cu­lated value to the thresh­old from ssd.json.  The fun­da­men­tal logic is that if any re­trieved or cal­cu­lated value (received in vari­able e be­low) is less than the thresh­old t from ssd, the ssd logic will honor stand­down.  In con­trast, if all four val­ues ex­ceed the thresh­old, ssd will can­cel the stand­down.  If this func­tion elects to honor stand­down, the re­turn value gives the name of the rule (a) and the thresh­old (s) that caused the de­ci­sion (yellow high­light­ing).  If this func­tion elects to dis­honor stand­down, it re­turns “ssd” (red) (which is the func­tion’s de­fault if not over­rid­den by the logic that fol­l­lows).  This yields the state= val­ues I showed in teleme­try and pre­sented in screen­shots and videos above.

func­tion P(e, t) {

var r = “ssd”;

re­turn Object.entries(t).forEach((function(t) {

var n, o, i = (o = 2, _(n = t) || b(n, o) || y(n, o) || g()),

a = i[0], // field name (e.g., uP, gca, adb)

s = i[1]; // thresh­old value from ssd.json

“adb” === a && (s = s > Date.now() ? s : Date.now() - s), // spe­cial han­dling for adb time­stamps

void 0 !== e[a] && e[a] < s && (r = “”.concat(a, ”:“).concat(s))

})), r

Reviewing both con­fig files and code, I was in­trigued to see eBay called out for greater pro­tec­tions than oth­ers.  Where Honey stands down for other mer­chant and net­works for 3,600 sec­onds (one hour), eBay gets 86,400 sec­onds (24 hours).

“regex”: “^https?\\:\\/\\/rover\\.ebay((?![\\?\\&]pub=5575133559).)*$”,

“provider”: “LS”,

“overrideBl”: true,

“ttl”: 86400

Hey every­one, I have been stay­ing at a ho­tel for a while. It’s one of those mod­ern ones with smart TVs and other con­nected good­ies. I got cu­ri­ous and opened Wireshark, as any tin­kerer would do.

I was very sur­prised to see a huge amount of UDP traf­fic on port 2046. I looked it up but the re­sults were far from use­ful. This was­n’t a stan­dard port, so I would have to fig­ure it out man­u­ally.

At first, I sus­pected that the data might be a tele­vi­sion stream for the TVs, but the packet length seemed too small, even for a sin­gle video frame.

This ar­ti­cle is also avail­able in French.

The UDP pack­ets weren’t sent to my IP and I was­n’t do­ing ARP spoof­ing, so these pack­ets were sent to every­one. Upon closer in­spec­tion, I found out that these were Multicast pack­ets. This ba­si­cally means that the pack­ets are sent once and re­ceived by mul­ti­ple de­vices si­mul­ta­ne­ously. Another thing I no­ticed was the fact that all of those pack­ets were the same length (634 bytes).

I de­cided to write a Python script to save and an­a­lyze this data. First of all, here’s the code I used to re­ceive Multicast pack­ets. In the fol­low­ing code, 234.0.0.2 is the IP I got from Wireshark.

im­port socket

im­port struct

s = socket.socket(socket.AF_INET, socket.SOCK­_D­GRAM, socket.IP­PRO­TO_UDP)

s.set­sock­opt(socket.SOL_­SOCKET, socket.SO_REUSE­ADDR, 1)

s.bind((‘’, 2046))

mreq = struct.pack(“4sl”, socket.in­et_a­ton(“234.0.0.2″), socket.IN­AD­DR_ANY)

s.set­sock­opt(socket.IP­PRO­TO_IP, socket.IP_AD­D_MEM­BER­SHIP, mreq)

while True:

data = s.recv(2048)

print(data)

On top of this, I also used bi­nascii to con­vert this to hex in or­der make read­ing the bytes eas­ier. After watch­ing thou­sands of these pack­ets scroll through the con­sole, I no­ticed that the first ~15 bytes were the same. These bytes prob­a­bly in­di­cate the pro­to­col and the packet/​com­mand ID but I only re­ceived the same one so I could­n’t in­ves­ti­gate those.

It also took me an em­bar­rass­ingly long time to see the string LAME3.91UUUUUUU at the end of the pack­ets. I sus­pected this was MPEG com­pressed au­dio data, but sav­ing one packet as test.mp3 failed to played with mplayer and the file util­ity only iden­ti­fied this as test.mp3: data. There was ob­vi­ously data in this packet and file should know when it sees MPEG Audio data, so I de­cided to write an­other Python script to save the packet data with off­sets. This way it would save the file test1 skip­ping 1 byte from the packet, test2 skip­ping 2 bytes and so on. Here’s the code I used and the re­sult.

data = s.recv(2048)

for i in range(25):

open(“test{}”.for­mat(i), “wb+“).write(data[i:])

After this, I ran file test* and voilà! Now we know we have to skip 8 bytes to get to the MPEG Audio data.

$ file test*

test0: data

test1: UNIF v-16624417 for­mat NES ROM im­age

test10: UNIF v-763093498 for­mat NES ROM im­age

test11: UNIF v-1093499874 for­mat NES ROM im­age

test12: data

test13: TTComp archive, bi­nary, 4K dic­tio­nary

test14: data

test15: data

test16: UNIF v-1939734368 for­mat NES ROM im­age

test17: UNIF v-1198759424 for­mat NES ROM im­age

test18: UNIF v-256340894 for­mat NES ROM im­age

test19: UNIF v-839862132 for­mat NES ROM im­age

test2: UNIF v-67173804 for­mat NES ROM im­age

test20: data

test21: data

test22: data

test23: DOS ex­e­cutable (COM, 0x8C-variant)

test24: COM ex­e­cutable for DOS

test3: UNIF v-1325662462 for­mat NES ROM im­age

test4: data

test5: data

test6: data

test7: data

test8: MPEG ADTS, layer III, v1, 192 kbps, 44.1 kHz, JntStereo

test9: UNIF v-2078407168 for­mat NES ROM im­age

while True:

data = s.recv(2048)

sys.std­out.buffer.write(data[8:])

Now all we need to do is con­tin­u­ously read pack­ets, skip the first 8 bytes, write them to a file and it should play per­fectly.

But what was this au­dio? Was this a sneak­ily placed bug that lis­tened to me? Was it some­thing re­lated to the smart TVs in my room? Something re­lated to the ho­tel sys­tems? Only one way to find out.

$ python3 lis­ten_2046.py > test.mp3

* wait a lit­tle to get a record­ing *

^C

$ mplayer test.mp3

MPlayer (C) 2000-2016 MPlayer Team

224 au­dio & 451 video codecs

Playing test.mp3.

libav­for­mat ver­sion 57.25.100 (external)

Audio only file for­mat de­tected.

Starting play­back…

A: 3.9 (03.8) of 13.0 (13.0) 0.7%

What the hell? I can’t be­lieve I spent time for this. It’s just el­e­va­tor mu­sic. It is played in the ho­tel cor­ri­dors around the el­e­va­tors. Oh well, at least I can lis­ten to it from my room now.

Lol, how do nerds lis­ten to el­e­va­tor mu­sic? Nice ar­ti­cle.

I know that it’s been al­most a decade since then but did you save the el­e­va­tor mu­sic? Kinda cu­ri­ous about it hon­estly.

This is fas­ci­nat­ing. Someone in the thread asked if the el­e­va­tor con­trols would be on that port and I can re­port there is no way. That au­dio is ging to be from a 3rd party ser­vice like Muzak. Nobody wants to man­age the roy­alty sys­tem ex­cept for spe­cial­ized com­pa­nies. But do we have any idea who? Mood Media? ActiveAire? Myndstream?

You can use `binwalk’ to search for file sig­na­tures (not quite the same ones as lib­magic, but sim­i­lar ones) at any off­set in the file. As an up­side, it also de­tects things like copy­right strings and other help­ful stuff.

Nice! Question: was the TV con­nected to an in­ter­face on your lap­top as a proxy? Else how did you know about the mul­ti­cast pack­ets?

What made you think to try an off­set? Is that a com­mon way these files are for­mat­ted?

RESPONDING TO: or in­ter­jec­tions of mys­te­ri­ous, echoed “voices from the past” at var­i­ous in­ter­vals…Is there some­body there?…What are you do­ing?…This can’t be real…the sound of a wa­ter­melon be­ing smashed…re­verbed laugh­ter…a quiet child’s voice… CREATE A TOOL TO SPOOF THE MULTICAST. BE SPOOKED COMPLAIN DEMAND A REFUND!

as it was UDP pro­to­col, that would be in­ter­est­ing if you did the source spoof­ing for some fun

now the im­me­di­ate ques­tion i get is can you trans­mit your own pack­ets and change what mu­sic plays?

What a great ar­ti­cle. Love the writ­ing style and the di­gestible deep dive into re­vers­ing.

Sir, this is re­ally in­spi­ra­tional.

Lol all for the el­e­va­tor mu­sic. Was thinki g smart TV was snoop­ing on every­one

Well done and very en­ter­tain­ing. A good read and learn.

as most things in life what seems mys­te­ri­ous it’s usu­ally pretty dis­ap­point­ing. Love this ar­ti­cle!

You sir, are an in­spi­ra­tion!

Music on hold / el­e­va­tor mu­sic is usu­ally sent as mul­ti­cast hence why you and the el­e­va­tor speaker is get­ting it.

Would have been even fun­nier if one of the songs was Rick Astley’s “Never Gonna Give You Up”.

How long all this took?

I’m re­spect­fully adding a vote for you post­ing that sweet el­e­va­tor mu­sic file.

I won­der if you could broad­cast your own mu­sic on the same port?

What made you think to save the data with off­sets? Is that rep­re­sen­ta­tive of the to­tal time of the el­e­va­tor mu­sic?

Time to start stream­ing some black metal into the el­e­va­tors.

So glad there is ev­i­dence that I’m not the only one who would have gone down this rab­bit hole!

Gold! Thanks for the write up, and the ride.

Appreciated this ar­ti­cle wanted to say Thank you for shar­ing.

Next step: Send out some mp3′s with the same for­mat to the same mul­ti­cast ad­dress. Kick that ho­tel party up a notch.

…or in­ter­jec­tions of mys­te­ri­ous, echoed “voices from the past” at var­i­ous in­ter­vals…Is there some­body there?…What are you do­ing?…This can’t be real…the sound of a wa­ter­melon be­ing smashed…re­verbed laugh­ter…a quiet child’s voice…

Could be lots of fun

Looks like you missed out on a lot of free NES ROMs there!

For those ask­ing for a link for the el­e­va­tor mu­sic: https://​www.youtube.com/​watch?v=xN­jyG8S4_kI

Nice. Who would have thought? I would­n’t have ex­pected that you could re­con­sti­tute the mp3 by strip­ping N bytes and con­cate­nat­ing the rest. Maybe some­day I’ll find a use for that. Thanks.

rep­toid clone of john wayne and elvis on 2023-02-23 17:25:41

now who woulda thought that they’d do it that way haha

now spoof it ;)

please now post the el­e­va­tor’s mu­sic you dumped

Last week, I up­dated our pric­ing lim­its. One JSON file. The back­end started en­forc­ing the new caps, the fron­tend dis­played them cor­rectly, the mar­ket­ing site showed them on the pric­ing page, and our docs re­flected the change—all from a sin­gle com­mit.

No sync is­sues. No “wait, which repo has the cur­rent pric­ing?” No de­ploy co­or­di­na­tion across three teams. Just one change, every­where, in­stantly.

At Kasava, our en­tire plat­form lives in a sin­gle repos­i­tory. Not just the code—every­thing:

kasava/ # 5,470+ files TypeScript files

├── fron­tend/ # Next.js 16 + React 19 ap­pli­ca­tion

│ └── src/

│ ├── app/ # 25+ route di­rec­to­ries

│ └── com­po­nents/ # 45+ com­po­nent di­rec­to­ries

├── back­end/ # Cloudflare Workers API

│ └── src/

│ ├── ser­vices/ # 55+ busi­ness logic ser­vices

│ └── work­flows/ # Mastra AI work­flows

├── web­site/ # Marketing site (kasava.ai)

├── docs/ # Public doc­u­men­ta­tion (Mintlify)

├── docs-in­ter­nal/ # 12+ ar­chi­tec­ture docs & specs

├── mar­ket­ing/

│ ├── blogs/ # Blog pipeline (drafts → re­view → pub­lished)

│ ├── in­vestor-deck/ # Next.js site show­ing in­vest­ment pro­posal

│ └── email/ # MJML tem­plates for Loops.so cam­paigns

├── ex­ter­nal/

│ ├── chrome-ex­ten­sion/ # WXT + React bug cap­ture tool

│ ├── google-docs-ad­don/ # @helper AI as­sis­tant (Apps Script)

│ └── google-cloud-func­tions/

│ ├── tree-sit­ter-ser­vice/ # AST pars­ing for 10+ lan­guages

│ └── mob­bin-re­search-ser­vice/

├── scripts/ # Deployment & in­te­gra­tion test­ing

├── in­fra-tester/ # Integration test har­ness

└── github-sim­u­la­tor/ # Mock GitHub API for lo­cal dev

This is­n’t about ab­stract philoso­phies on de­sign pat­terns for ‘how we should work.’ It’s about ve­loc­ity in an era where prod­ucts change fast and con­text mat­ters.

AI is all about con­text. And this monorepo is our com­pany—not just the prod­uct.

When our AI tools help us write doc­u­men­ta­tion, they have im­me­di­ate ac­cess to the ac­tual code be­ing doc­u­mented. When we up­date our mar­ket­ing web­site, the AI can ver­ify claims against the real im­ple­men­ta­tion. When we write blog posts like this one, the AI can fact-check every code ex­am­ple, every num­ber, every ar­chi­tec­tural claim against the source of truth.

* Documentation up­dates faster be­cause the AI sees code changes and sug­gests doc up­dates in the same con­text

* Website up­dates faster be­cause pric­ing, fea­tures, and ca­pa­bil­i­ties are pulled from the same con­fig files that power the app

* Blog posts ship faster be­cause the AI can run self-ref­er­en­tial checks—val­i­dat­ing that our “5,470+ TypeScript files” claim is ac­cu­rate by ac­tu­ally count­ing them

* Nothing goes out of sync be­cause there’s only one source of truth, and AI has ac­cess to all of it

When you ask Claude to “update the pric­ing page to re­flect the new lim­its,” it can:

Check the fron­tend that dis­plays them

Flag any blog posts that might men­tion out­dated num­bers

All in one con­ver­sa­tion. All in one repos­i­tory.

This is what “AI-native de­vel­op­ment” ac­tu­ally means: struc­tur­ing your work so AI can be max­i­mally help­ful, not fight­ing against frag­men­ta­tion.

Everything-as-code means every­thing ships the same way: git push. Want to up­date the web­site pric­ing page? git push. New blog post ready to go live? git push. Fix a typo in the docs? git push. Deploy a back­end fea­ture? git push.

No sep­a­rate CMSs to log into. No WordPress ad­min pan­els. No wait­ing for mar­ket­ing tools to sync. No “can some­one with Contentful ac­cess up­date this?” The same Git work­flow that ships code also ships con­tent, doc­u­men­ta­tion, and mar­ket­ing. Everyone on the team can ship any­thing, and it all goes through the same re­view process, the same CI/CD, the same au­dit trail.

This uni­for­mity re­moves fric­tion and re­moves ex­cuses. Shipping be­comes mus­cle mem­ory.

When a back­end API changes, the fron­tend type de­f­i­n­i­tions up­date in the same com­mit. When we add a new fea­ture, the doc­u­men­ta­tion can ship along­side it. No ver­sion mis­matches. No “which ver­sion of the API does this fron­tend need?”

AI can see and val­i­date the en­tire change in con­text.

When we ask Claude to add a fea­ture, it does­n’t just write back­end code. It sees the fron­tend that will con­sume it, the docs that need up­dat­ing, and the mar­ket­ing site that might ref­er­ence it. All in one view. All in one con­ver­sa­tion.

Real ex­am­ple from our code­base—adding Asana in­te­gra­tion:

com­mit: “feat: add Asana in­te­gra­tion”

├── back­end/​src/​ser­vices/​AsanaSer­vice.ts

├── back­end/​src/​routes/​api/​in­te­gra­tions/​asana.ts

├── fron­tend/​src/​com­po­nents/​in­te­gra­tions/​asana/

├── fron­tend/​src/​app/​in­te­gra­tions/​asana/

├── docs/​in­te­gra­tions/​asana.mdx

└── web­site/​src/​app/​in­te­gra­tions/​page.tsx

One PR. One re­view. One merge. Everything ships to­gether.

We have a sin­gle billing-plans.json that de­fines all plan lim­its and fea­tures:

// fron­tend/​src/​con­fig/​billing-plans.json (also copied to web­site/​src/​con­fig/)

“plans”: {

“free”: { “limits”: { “repositories”: 1, “aiChatMessagesPerDay”: 10 } },

“starter”: {

“limits”: { “repositories”: 10, “aiChatMessagesPerDay”: 100 }

“professional”: {

“limits”: { “repositories”: 50, “aiChatMessagesPerDay”: 1000 }

The back­end en­forces these lim­its. The fron­tend dis­plays them in set­tings. The mar­ket­ing web­site shows them on the pric­ing page. When we change a limit, one JSON up­date prop­a­gates every­where—no “the web­site says 50 re­pos but the app shows 25” bugs.

And AI val­i­dates all of it. When we up­date billing-plans.json, we can ask Claude to ver­ify that the back­end, fron­tend, and web­site are all con­sis­tent. It reads all three im­ple­men­ta­tions and con­firms they match—or tells us what needs fix­ing.

Renaming a func­tion? Your IDE finds all us­ages across fron­tend, back­end, docs ex­am­ples, and blog code snip­pets. One find-and-re­place. One com­mit.

* Search: Find any­thing with one grep

fron­tend/ # Customer-facing Next.js app

├── src/

│ ├── app/ # Next.js 15 App Router

│ │ ├── an­a­lyt­ics/ # Semantic com­mit analy­sis

│ │ ├── bug-re­ports/ # AI-powered bug track­ing

│ │ ├── chat/ # AI as­sis­tant in­ter­face

│ │ ├── code-search/ # Semantic code search

│ │ ├── dash­board/ # Main dash­board

│ │ ├── google-docs-as­sis­tant/

│ │ ├── in­te­gra­tions/ # GitHub, Linear, Jira, Asana

│ │ ├── prd/ # PRD man­age­ment

│ │ └── … # 25+ route di­rec­to­ries to­tal

│ ├── com­po­nents/ # 45+ com­po­nent di­rec­to­ries

│ │ ├── ai-el­e­ments/ # AI-specific UI

│ │ ├── bug-re­ports/ # Bug track­ing UI

│ │ ├── dash­board/ # Dashboard wid­gets

│ │ ├── google-docs/ # Google Docs in­te­gra­tion

│ │ ├── on­board­ing/ # User on­board­ing flow

│ │ └── ui/ # shadcn/​ui base com­po­nents

│ ├── mas­tra/ # Frontend Mastra in­te­gra­tion

│ └── lib/ # SDK, util­i­ties, hooks

back­end/ # Cloudflare Workers API

├── src/

│ ├── routes/ # Hono API end­points

│ ├── ser­vices/ # 55+ busi­ness logic ser­vices

│ ├── work­flows/ # Mastra AI work­flows

│ │ ├── steps/ # Reusable work­flow steps

│ │ └── RepositoryIndexingWorkflow.ts

│ ├── db/ # Drizzle ORM schema

│ ├── durable-ob­jects/ # Stateful edge com­put­ing

│ ├── work­ers/ # Queue con­sumers

│ └── mas­tra/ # AI agents and tools

These two talk to each other con­stantly. Having them in the same repo means:

Many de­vel­op­ers want to start a side pro­ject but aren’t sure what to build. The in­ter­net is full of ideas that are ba­sic and dull.

Here’s our list of 73 pro­ject ideas to in­spire you. We have cho­sen pro­jects that teach a lot and are fun to build.

Build a BitTorrent client that can down­load files us­ing the BitTorrent pro­to­col. You can start with sin­gle-file tor­rents. This is a great way to learn how P2P net­work­ing works.

Read the of­fi­cial BitTorrent spec­i­fi­ca­tion here.

Build a pro­gram that solves Wordle. This can be a great les­son on in­for­ma­tion the­ory and en­tropy. You’ll also get hands-on ex­pe­ri­ence at op­ti­miz­ing com­pu­ta­tions.

This YouTube video will get you started.

Implement Optimal Transport from scratch to morph one face into an­other while pre­serv­ing iden­tity and struc­ture. You’ll ap­ply lin­ear pro­gram­ming to a real prob­lem.

Here are some OT re­sources and a pa­per which pro­poses a so­lu­tion.

Create a spread­sheet with sup­port for cell ref­er­ences, sim­ple for­mu­las, and live up­dates. You’ll learn about de­pen­dency graphs, pars­ing, and re­ac­tive UI de­sign.

The founder of the GRID spread­sheet en­gine shares some in­sights here.

Build a light­weight con­tainer run­time from scratch with­out Docker. You’ll learn about ker­nel name­spaces, ch­root, process iso­la­tion, and more.

Read this to un­der­stand how con­tain­ers work.

Build a sys­tem that uses Euclid’s pos­tu­lates to de­rive geo­met­ric proofs and vi­su­al­ize the steps. You’ll learn sym­bolic rep­re­sen­ta­tion, rule sys­tems, logic en­gines, and proof the­ory.

Google uses a crawler to nav­i­gate web pages and save their con­tents. By build­ing one, you’ll learn how web search works. It’s also great prac­tice for sys­tem de­sign.

You can make your own list of sites and cre­ate a search en­gine on a topic of your in­ter­est.

Build a DNS server that lis­tens for queries, parses pack­ets, re­solves do­mains, and caches re­sults. Learn more about low-level net­work­ing, UDP, TCP, and the in­ter­net.

Start with how DNS works and dive into the DNS packet for­mat.

Build a game where play­ers con­nect two ac­tors through shared cred­its with other ac­tors, and re­veal the op­ti­mal path at the end. You’ll learn how to deal with mas­sive graphs.

Explore how to cre­ate fast graphs, and then try Landmark Labelling for supreme per­for­mance.

Implement the RAFT pro­to­col from scratch to sup­port dis­trib­uted com­put­ing. Learn con­sen­sus, fail­ure re­cov­ery, and how to build fault-tol­er­ant dis­trib­uted sys­tems.

Visit this page for the RAFT pa­per and other re­sources.

Design a pro­gram from scratch that cre­ates sat­is­fy­ing cross­words with ad­justable dif­fi­culty. You’ll learn pro­ce­dural gen­er­a­tion, con­straint prop­a­ga­tion, and dif­fi­culty mod­el­ing.

For ex­am­ple, you can im­ple­ment the Wave Function Collapse al­go­rithm ex­plained here.

Bitcask is an ef­fi­cient em­bed­ded key-value store de­signed to han­dle pro­duc­tion-grade traf­fic. Building this will im­prove your un­der­stand­ing of data­bases and ef­fi­cient stor­age.

You can im­ple­ment this short pa­per.

Apps like Shazam ex­tract unique fea­tures from au­dio. This fin­ger­print is then used to match and iden­tify sounds. You’ll need to learn hash-based lookups and a bit of sig­nal pro­cess­ing.

Here’s a de­tailed post with every­thing you need to know.

Recreate the in­dus­try-chang­ing game us­ing SDL, and add some story el­e­ments, NPC in­ter­ac­tions, and lev­els. It’ll be a per­fect in­tro to game de­vel­op­ment.

This video will set you up.

Implement an al­go­rithm from scratch to com­pare two text files or pro­grams. This will in­volve dy­namic pro­gram­ming and ap­pli­ca­tion of graph tra­ver­sal.

Here’s the clas­sic pa­per be­hind Myers’ diff, used in Git for years.

Generate UML class di­a­grams from source code with sup­port for re­la­tion­ships like in­her­i­tance. You’ll vi­su­al­ize ob­ject-ori­ented code and learn how to parse with ASTs.

Write your own en­coder/​de­coder for the BMP im­age for­mat and build a tiny viewer for it. You’ll learn bi­nary pars­ing, im­age en­cod­ing, and how to work with pixel buffers and head­ers.

The Wikipedia ar­ti­cle is a good place to start.

Build a FUSE filesys­tem for Linux from scratch, with in­dex­ing, file meta­data, and caching. You’ll have to op­ti­mize data struc­tures for stor­age and per­for­mance.

This ar­ti­cle talks about the con­cepts used in filesys­tems.

Write the qubit and quan­tum gates from scratch. Use them to sim­u­late a cir­cuit for a quan­tum al­go­rithm like Bernstein-Vazirani or Simon’s al­go­rithm.

Read this short pa­per for the es­sen­tials with­out any fluff.

Write a video player that de­codes H.264/H.265 us­ing ffm­peg, and sup­ports cast­ing lo­cal files to smart de­vices. Learn packet buffer­ing, dis­cov­ery pro­to­cols, and stream en­cod­ing.

Get started with this ar­ti­cle.

Build a Redis clone from scratch that sup­ports ba­sic com­mands, RDB per­sis­tence, replica sync, streams, and trans­ac­tions. You’ll get to deep dive into sys­tems pro­gram­ming.

You can use the of­fi­cial Redis docs as a guide.

Build a client-side video ed­i­tor that runs in the browser with­out up­load­ing files to a server. Learn how to work with WASM, and why peo­ple love us­ing it for high per­for­mance tasks.

Visit the of­fi­cial WebAssembly site to get started.

This is a rite of pas­sage. You’ll get hands-on ex­pe­ri­ence with en­cryp­tion, to­ken ex­pi­ra­tion, re­fresh flows, and how to man­age user ses­sions se­curely.

Implement user­name and pass­word auth. Then man­age ses­sions with JWT or ses­sion IDs.

You have used it in searches and other places where you write text. Implement a so­lu­tion that sug­gests the right words, and then op­ti­mize heav­ily for speed.

This YouTube video gives an idea of the im­ple­men­ta­tion process.

Build a sim­ple SQL en­gine that reads .db files, uses in­dexes and ex­e­cutes queries. It’s a deep dive into how real-world data­bases are built and run ef­fi­ciently.

You need to un­der­stand B-trees and how SQLite stores data on disk.

Remove back­ground sounds from au­dio files. You’ll learn sig­nal pro­cess­ing and de­nois­ing tech­niques used in GPS, mouse in­put, sen­sors, ob­ject track­ing, etc.

You can use a tech­nique like Kalman Filtering to do this.

Design a file shar­ing app with sync, cloud stor­age, and ba­sic p2p fea­tures that can scale to some ex­tent. You’ll get prac­tice in cloud ar­chi­tec­ture and back­end de­sign.

This ar­ti­cle dives into the sys­tem de­sign.

Build a map en­gine to in­dex roads, ter­rain (rivers, moun­tains), places (shops, land­marks), and ar­eas (cities, states). Learn spa­tial in­dex­ing, range queries, and zoom-level ab­strac­tions.

Start by im­ple­ment­ing an R-tree from scratch by fol­low­ing the orig­i­nal pa­per.

Use Natural Earth and GeoFabrik datasets to pop­u­late your map en­gine.

Recreate a city’s road net­work, sim­u­late traf­fic us­ing real open data, and de­sign an im­proved ver­sion. Tackle an NP-hard op­ti­miza­tion prob­lem with real con­straints.

In some cases, na­ture has long solved what we call hard. Implement SMA or ACO here.

Develop a de­cen­tral­ized col­lab­o­ra­tive text ed­i­tor. Similar to Google Docs, but with­out any cen­tral server. Use CRDTs to man­age con­cur­rent ed­its and en­sure even­tual con­sis­tency.

Use ropes, gap buffers, or piece ta­bles to build a fast text buffer op­ti­mized for ef­fi­cient edit­ing.

Read this ar­ti­cle on de­sign­ing data struc­tures for such apps.

Evolve work­ing mod­els of ma­chin­ery us­ing only prim­i­tive me­chan­i­cal parts and con­straints. You’ll learn about ge­netic al­go­rithms, fit­ness func­tions, and physics sim­u­la­tion.

You can de­sign bridges, cars, clocks, cal­cu­la­tors, cat­a­pults, and more. NASA used GAs to de­sign an an­tenna for their space mis­sion.

This YouTube video shows how in­ter­est­ing evo­lu­tion­ary de­sign can get.

Create a server from scratch that sup­ports HTTP re­quests, sta­tic files, rout­ing, and re­verse prox­y­ing. Learn socket pro­gram­ming and how web servers work.

This page will get you started.

Estimate a depth (disparity) map from a stereo im­age pair us­ing Markov Random Fields. You’ll learn about com­puter vi­sion, graph­i­cal mod­els, and in­fer­ence tech­niques.

Start with the Middlebury Dataset and this ar­ti­cle on be­lief prop­a­ga­tion for stereo match­ing.

Build a min­i­mal Git with core fea­tures like init, com­mit, diff, log, and branch­ing. Learn how ver­sion con­trol works us­ing con­tent-ad­dress­able stor­age, hashes, and trees.

Check out Write your­self a Git for an overview of git in­ter­nals.

Build a Unix de­bug­ger with step­ping, break­points, and mem­ory in­spec­tion. You’ll learn low-level sys­tems pro­gram­ming and process con­trol.

This ar­ti­cle dis­cusses the in­ter­nal struc­ture of GDB.

Build a deep learn­ing frame­work from scratch with a ten­sor class, au­to­grad, ba­sic lay­ers, and op­ti­miz­ers. Grasp the in­ter­nals of back­prop­a­ga­tion and gra­di­ent de­scent.

Start by build­ing a sim­ple 3-layer feed­for­ward NN (multilayer per­cep­tron) with your frame­work.

Andrej Karpathy ex­plains the ba­sic con­cepts in this YouTube Video.

Build a Chess app from scratch, where users can play against each other or your own UCI en­gine. This pro­ject of­fers a blend of al­go­rithms, UI, game logic, and AI.

You can go one step fur­ther and make the en­gine play it­self to im­prove like AlphaZero and Leela.

You can start with the rules and the chess pro­gram­ming wiki.

Build a fast search en­gine from scratch for the Wikipedia dump with typo tol­er­ance and se­man­tic rank­ing, and fuzzy queries. You’ll learn in­dex­ing, to­k­eniza­tion, and rank­ing al­go­rithms.

This ar­ti­cle of­fers a good in­tro­duc­tion to the ba­sics of in­for­ma­tion re­trieval.

Build a caching sys­tem to avoid re­dun­dant fetches for sta­tic as­sets. You’ll learn web caching, log analy­sis, and how to use prob­a­bilis­tic data struc­tures in a real set­ting.

You can use this dataset con­tain­ing two mon­th’s worth of HTTP re­quests to the NASA server.

This ar­ti­cle in­tro­duces some of the key con­cepts.

Build a short-video app with in­fi­nite scroll, so­cial graphs of friends and subs, and a tai­lored feed. You’ll learn ef­fi­cient pre­load­ing, knowl­edge graphs, and be­hav­ioral sig­nals.

Implement NTP from scratch to build a back­ground ser­vice that syncs sys­tem time with time servers. You’ll learn dae­mon de­sign and the in­ter­nals of net­work time sync.

Implement HyperLogLog from scratch to pro­vide an­a­lyt­ics on num­ber of users en­gag­ing with hash­tags in real time. You’ll learn some key con­cepts around big data sys­tems.

Write a query plan­ner that rewrites SQL queries for bet­ter per­for­mance. You’ll learn cost es­ti­ma­tion, join re­order­ing, and in­dex se­lec­tion.

Implement an en­crypted vot­ing sys­tem for anonymity. Use zero-knowl­edge proofs to ver­ify re­sults.

For ex­am­ple, this pa­per at­tempts to de­fine such a pro­to­col.

Build a mesh VPN where nodes re­lay traf­fic with­out cen­tral servers. You’ll learn NAT tra­ver­sal, en­crypted tun­nel­ing, and de­cen­tral­ized rout­ing.

Build a file archiver that com­presses, bun­dles, and en­crypts your files. Implement com­pres­sion and en­cryp­tion al­go­rithms from scratch. Benchmark your per­for­mance against zip.

You can re­fer to the of­fi­cial .zip spec­i­fi­ca­tion.

Build a ba­sic ray tracer to ren­der 3D scenes with spheres, planes, and lights. This will be great prac­tice in writ­ing clean ab­strac­tions and op­ti­miz­ing per­for­mance-heavy code.

You can re­fer to the Ray Tracing in One Weekend ebook.

Create your own lan­guage. It is best to start with an in­ter­preted lan­guage that does not need a com­plier. Design your own gram­mar, parser, and an eval­u­a­tion en­gine.

Crafting Interpreters is by far the best re­source you can re­fer to.

Recreate WhatsApp with chats, groups, his­tory, en­cryp­tion, no­ti­fi­ca­tions, and re­ceipts. You’ll get prac­tice at build­ing a pro­duc­tion-grade app with an API, data store, and se­cu­rity.

You can draw in­spi­ra­tion from this sys­tem de­sign ap­proach.

Build a ser­vice to pro­vide routes for a fleet of ve­hi­cles with lim­ited ca­pac­ity to de­liver Amazon pack­ages. You’ll learn to op­ti­mize rout­ing un­der con­straints.

Hey folks, I got a lot of feed­back from var­i­ous meet­ings on the pro­posed LLVM AI con­tri­bu­tion pol­icy, and I made some sig­nif­i­cant changes based on that feed­back. The cur­rent draft pro­posal fo­cuses on the idea of re­quir­ing a hu­man in the loop who un­der­stands their con­tri­bu­tion well enough to an­swer ques­tions about it dur­ing re­view. The idea here is that con­trib­u­tors are not al­lowed to of­fload the work of val­i­dat­ing LLM tool out­put to main­tain­ers. I’ve mostly re­moved the Fedora pol­icy in an ef­fort to move from the vague no­tion of “owning the con­tri­bu­tion” to a more ex­plicit “contributors have to re­view their con­tri­bu­tions and be pre­pared to an­swer ques­tions about them”. Contributors should never find them­selves in the po­si­tion of say­ing “I don’t know, an LLM did it”. I felt the change here was sig­nif­i­cant, and de­served a new thread.

From an in­for­mal show of hands at the round table at the US LLVM de­vel­oper meet­ing, most con­trib­u­tors (or at least the sub­set with the re­sources and in­ter­est in at­tend­ing this round table in per­son) are in­ter­ested in us­ing LLM as­sis­tance to in­crease their pro­duc­tiv­ity, and I re­ally do want to en­able them to do so, while also mak­ing sure we give main­tain­ers a use­ful pol­icy tool for push­ing back against un­wanted con­tri­bu­tions.

I’ve up­dated the PR, and I’ve pasted the mark­down be­low as well, but you can also view it on GitHub.

LLVM’s pol­icy is that con­trib­u­tors can use what­ever tools they would like to

craft their con­tri­bu­tions, but there must be a hu­man in the loop.

Contributors must read and re­view all LLM-generated code or text be­fore they

ask other pro­ject mem­bers to re­view it. The con­trib­u­tor is al­ways the au­thor

and is fully ac­count­able for their con­tri­bu­tions. Contributors should be

suf­fi­ciently con­fi­dent that the con­tri­bu­tion is high enough qual­ity that ask­ing

for a re­view is a good use of scarce main­tainer time, and they should be able

to an­swer ques­tions about their work dur­ing re­view.

We ex­pect that new con­trib­u­tors will be less con­fi­dent in their con­tri­bu­tions,

and our guid­ance to them is to start with small con­tri­bu­tions that they can

fully un­der­stand to build con­fi­dence. We as­pire to be a wel­com­ing com­mu­nity

that helps new con­trib­u­tors grow their ex­per­tise, but learn­ing in­volves tak­ing

small steps, get­ting feed­back, and it­er­at­ing. Passing main­tainer feed­back to an

LLM does­n’t help any­one grow, and does not sus­tain our com­mu­nity.

Contributors are ex­pected to be trans­par­ent and la­bel con­tri­bu­tions that

con­tain sub­stan­tial amounts of tool-gen­er­ated con­tent. Our pol­icy on

la­belling is in­tended to fa­cil­i­tate re­views, and not to track which parts of

LLVM are gen­er­ated. Contributors should note tool us­age in their pull re­quest

de­scrip­tion, com­mit mes­sage, or wher­ever au­thor­ship is nor­mally in­di­cated for

the work. For in­stance, use a com­mit mes­sage trailer like Assisted-by: . This trans­parency helps the com­mu­nity de­velop best prac­tices

and un­der­stand the role of these new tools.

An im­por­tant im­pli­ca­tion of this pol­icy is that it bans agents that take ac­tion

in our dig­i­tal spaces with­out hu­man ap­proval, such as the GitHub @claude

agent. Similarly, au­to­mated re­view tools that

pub­lish com­ments with­out hu­man re­view are not al­lowed. However, an opt-in

re­view tool that keeps a hu­man in the loop is ac­cept­able un­der this pol­icy.

As an­other ex­am­ple, us­ing an LLM to gen­er­ate doc­u­men­ta­tion, which a con­trib­u­tor

man­u­ally re­views for cor­rect­ness, ed­its, and then posts as a PR, is an ap­proved

use of tools un­der this pol­icy.

This pol­icy in­cludes, but is not lim­ited to, the fol­low­ing kinds of

con­tri­bu­tions:

Code, usu­ally in the form of a pull re­quest

The rea­son for our “human-in-the-loop” con­tri­bu­tion pol­icy is that pro­cess­ing

patches, PRs, RFCs, and com­ments to LLVM is not free — it takes a lot of

main­tainer time and en­ergy to re­view those con­tri­bu­tions! Sending the

un­re­viewed out­put of an LLM to open source pro­ject main­tain­ers ex­tracts work

from them in the form of de­sign and code re­view, so we call this kind of

con­tri­bu­tion an “extractive con­tri­bu­tion”.

Our golden rule is that a con­tri­bu­tion should be worth more to the pro­ject

than the time it takes to re­view it. These ideas are cap­tured by this quote

from the book Working in Public by Nadia Eghbal:

“When at­ten­tion is be­ing ap­pro­pri­ated, pro­duc­ers need to weigh the costs and

ben­e­fits of the trans­ac­tion. To as­sess whether the ap­pro­pri­a­tion of at­ten­tion

is net-pos­i­tive, it’s use­ful to dis­tin­guish be­tween ex­trac­tive and

non-ex­trac­tive con­tri­bu­tions. Extractive con­tri­bu­tions are those where the

mar­ginal cost of re­view­ing and merg­ing that con­tri­bu­tion is greater than the

mar­ginal ben­e­fit to the pro­jec­t’s pro­duc­ers. In the case of a code

con­tri­bu­tion, it might be a pull re­quest that’s too com­plex or un­wieldy to

re­view, given the po­ten­tial up­side.” — Nadia Eghbal

Prior to the ad­vent of LLMs, open source pro­ject main­tain­ers would of­ten re­view

any and all changes sent to the pro­ject sim­ply be­cause post­ing a change for

re­view was a sign of in­ter­est from a po­ten­tial long-term con­trib­u­tor. While new

tools en­able more de­vel­op­ment, it shifts ef­fort from the im­ple­men­tor to the

re­viewer, and our pol­icy ex­ists to en­sure that we value and do not squan­der

main­tainer time.

Reviewing changes from new con­trib­u­tors is part of grow­ing the next gen­er­a­tion

of con­trib­u­tors and sus­tain­ing the pro­ject. We want the LLVM pro­ject to be

wel­com­ing and open to as­pir­ing com­piler en­gi­neers who are will­ing to in­vest

time and ef­fort to learn and grow, be­cause grow­ing our con­trib­u­tor base and

re­cruit­ing new main­tain­ers helps sus­tain the pro­ject over the long term. Being

open to con­tri­bu­tions and lib­er­ally grant­ing com­mit ac­cess

is a big part of how LLVM has grown and suc­cess­fully been adopted all across

the in­dus­try. We there­fore au­to­mat­i­cally post a greet­ing com­ment to pull

re­quests from new con­trib­u­tors and en­cour­age main­tain­ers to spend their time to

help new con­trib­u­tors learn.

If a main­tainer judges that a con­tri­bu­tion is ex­trac­tive (i.e. it does­n’t

com­ply with this pol­icy), they should copy-paste the fol­low­ing re­sponse to

re­quest changes, add the ex­trac­tive la­bel if ap­plic­a­ble, and re­frain from

fur­ther en­gage­ment:

This PR ap­pears to be ex­trac­tive, and re­quires ad­di­tional jus­ti­fi­ca­tion for

why it is valu­able enough to the pro­ject for us to re­view it. Please see

our de­vel­oper pol­icy on AI-generated con­tri­bu­tions:

http://​llvm.org/​docs/​AIToolPol­icy.html

Other re­view­ers should use the la­bel to pri­or­i­tize their re­view time.

The best ways to make a change less ex­trac­tive and more valu­able are to re­duce

its size or com­plex­ity or to in­crease its use­ful­ness to the com­mu­nity. These

fac­tors are im­pos­si­ble to weigh ob­jec­tively, and our pro­ject pol­icy leaves this

de­ter­mi­na­tion up to the main­tain­ers of the pro­ject, i.e. those who are do­ing

the work of sus­tain­ing the pro­ject.

If a con­trib­u­tor re­sponds but does­n’t make their change mean­ing­fully less

ex­trac­tive, main­tain­ers should es­ca­late to the rel­e­vant mod­er­a­tion or ad­min

team for the space (GitHub, Discourse, Discord, etc) to lock the con­ver­sa­tion.

Artificial in­tel­li­gence sys­tems raise many ques­tions around copy­right that have

yet to be an­swered. Our pol­icy on AI tools is sim­i­lar to our copy­right pol­icy:

Contributors are re­spon­si­ble for en­sur­ing that they have the right to

con­tribute code un­der the terms of our li­cense, typ­i­cally mean­ing that ei­ther

they, their em­ployer, or their col­lab­o­ra­tors hold the copy­right. Using AI tools

to re­gen­er­ate copy­righted ma­te­r­ial does not re­move the copy­right, and

con­trib­u­tors are re­spon­si­ble for en­sur­ing that such ma­te­r­ial does not ap­pear in

their con­tri­bu­tions. Contributions found to vi­o­late this pol­icy will be re­moved

just like any other of­fend­ing con­tri­bu­tion.

Here are some ex­am­ples of con­tri­bu­tions that demon­strate how to ap­ply

the prin­ci­ples of this pol­icy:

This PR con­tains a proof from Alive2, which is a strong sig­nal of

value and cor­rect­ness.

This gen­er­ated doc­u­men­ta­tion was re­viewed for cor­rect­ness by a

hu­man be­fore be­ing posted.

Skip to con­tent

Secure your code as you build

We read every piece of feed­back, and take your in­put very se­ri­ously.

Include my email ad­dress so I can be con­tacted

Use saved searches to fil­ter your re­sults more quickly

To see all avail­able qual­i­fiers, see our doc­u­men­ta­tion.

Sign up

You signed in with an­other tab or win­dow. Reload to re­fresh your ses­sion.

You signed out in an­other tab or win­dow. Reload to re­fresh your ses­sion.

You switched ac­counts on an­other tab or win­dow. Reload to re­fresh your ses­sion.

Notifications

You must be signed in to change no­ti­fi­ca­tion set­tings

Notifications

You must be signed in to change no­ti­fi­ca­tion set­tings

There was an er­ror while load­ing. .

Lower is bet­ter. Build with zig build -Doptimize=ReleaseFast.

Build with zig build -Doptimize=ReleaseFast for best per­for­mance.

zig build # Build li­brary and CLI

zig build test # Run tests

const zpdf = @import(“zpdf”);

pub fn main() !void {

var gpa = std.heap. GeneralPurposeAllocator(.{}){};

de­fer _ = gpa.deinit();

const al­lo­ca­tor = gpa.al­lo­ca­tor();

const doc = try zpdf.Doc­u­ment.open(al­lo­ca­tor, “file.pdf”);

de­fer doc.close();

var buf: [4096]u8 = un­de­fined;

var writer = std.fs.File.std­out().writer(&buf);

de­fer writer.in­ter­face.flush() catch {};

for (0..doc.pages.items.len) |page_num| {

try doc.ex­tract­Text(page_num, &writer.interface);

zpdf ex­tract doc­u­ment.pdf # Extract all pages to std­out

zpdf ex­tract -p 1-10 doc­u­ment.pdf # Extract pages 1-10

zpdf ex­tract -o out.txt doc­u­ment.pdf # Output to file

zpdf ex­tract –reading-order doc.pdf # Use vi­sual read­ing or­der (experimental)

zpdf info doc­u­ment.pdf # Show doc­u­ment info

zpdf bench doc­u­ment.pdf # Run bench­mark

im­port zpdf

with zpdf.Doc­u­ment(“file.pdf”) as doc:

print(doc.page_­count)

# Single page

text = doc.ex­trac­t_­page(0)

# All pages (parallel by de­fault)

al­l_­text = doc.ex­trac­t_all()

# Reading or­der ex­trac­tion (experimental)

or­dered_­text = doc.ex­trac­t_all(read­ing_or­der=True)

# Page info

info = doc.get_­page_info(0)

print(f”{info.width}x{info.height}“)

zig build -Doptimize=ReleaseFast

PYTHONPATH=python python3 ex­am­ples/​ba­sic.py

src/

├── root.zig # Document API and core types

├── capi.zig # C ABI ex­ports for FFI

├── parser.zig # PDF ob­ject parser

├── xref.zig # XRef table/​stream pars­ing

├── page­tree.zig # Page tree res­o­lu­tion

├── de­com­press.zig # Stream de­com­pres­sion fil­ters

├── en­cod­ing.zig # Font en­cod­ing and CMap pars­ing

├── in­ter­preter.zig # Content stream in­ter­preter

├── simd.zig # SIMD string op­er­a­tions

└── main.zig # CLI

python/​zpdf/ # Python bind­ings (cffi)

ex­am­ples/ # Usage ex­am­ples

*ToUnicode/CID: Works when CMap is em­bed­ded di­rectly.

**pdfium re­quires multi-process for par­al­lelism (forked be­fore thread sup­port).

There was an er­ror while load­ing. Please re­load this page.

You can’t per­form that ac­tion at this time.