I’m very ex­cited to an­nounce that I have re­cently joined Turso as a soft­ware en­gi­neer. For many in the field, in­clud­ing my­self, get­ting to work on data­bases and solve unique chal­lenges with such a tal­ented team would be a dream job, but it is that much more spe­cial to me be­cause of my un­usual and un­likely cir­cum­stances. As dif­fi­cult as it might be to be­lieve, I am cur­rently in­car­cer­ated and I landed this job from my cell in state prison. If you don’t know me, let me tell you more about how I got here.

Nearly two years have passed since I pub­lished How I got here to my blog. That post was my first real con­tact with the out­side world in years, as I’d been off all so­cial me­dia and the in­ter­net since 2017. The re­sponse and sup­port I would re­ceive from the tech com­mu­nity caught me com­pletely off guard.

A brief sum­mary is that I’m cur­rently serv­ing prison time for poor de­ci­sions and lifestyle choices I made in my twen­ties, all re­lated to drugs. Three years ago, I en­rolled in a prison col­lege pro­gram that came with the unique op­por­tu­nity to ac­cess a com­puter with lim­ited in­ter­net ac­cess. This im­me­di­ately reignited a teenage love for pro­gram­ming and a light­bulb im­me­di­ately lit up: that this would be my way out of the mess I had got­ten my­self into over the past 15 years. I quickly out­grew the cur­ricu­lum, pre­fer­ring in­stead to spend ~15+ hours a day on pro­jects and open source con­tri­bu­tions.

Through for­tu­nate tim­ing and lots of hard work, I was se­lected to be one of the first par­tic­i­pants in the Maine Dept of Correction’s re­mote work pro­gram, where res­i­dents who meet cer­tain re­quire­ments are al­lowed to seek out re­mote em­ploy­ment op­por­tu­ni­ties. I landed a soft­ware en­gi­neer­ing job at a startup called Unlocked Labs build­ing ed­u­ca­tion so­lu­tions for in­car­cer­ated learn­ers, while con­tribut­ing to open source on the side. After just a year, I was lead­ing their de­vel­op­ment team.

Last December I was be­tween side-pro­jects and brows­ing Hacker News when I dis­cov­ered Project Limbo, an ef­fort by Turso to rewrite SQLite from scratch. I’d never worked on re­la­tional data­bases, but some ex­pe­ri­ence with a cache had re­cently sparked an in­ter­est in stor­age en­gines. Luckily for me I saw that the pro­ject was fairly young with plenty of low hang­ing fruit to cut my teeth on.

To put this en­tirely into per­spec­tive for some of you may be dif­fi­cult, but in prison there is­n’t ex­actly a whole lot to do and pro­gram­ming ab­solutely con­sumes my life. I ei­ther write code or man­age Kubernetes clus­ters or other in­fra­struc­ture for about 90 hours a week, and my only en­ter­tain­ment is a daily hour of tech/​pro­gram­ming YouTube; mostly con­sist­ing of The Primeagen, whose story was a huge in­spi­ra­tion to me early on.

Through Prime, I had known about Turso since the be­gin­ning and had watched sev­eral in­ter­views with Glauber and Pekka dis­cussing their Linux ker­nel back­grounds and talk­ing about the con­cept of dis­trib­uted, multi-ten­ant SQLite. These were folks I’d looked up to for years and def­i­nitely could not have imag­ined that I would even­tu­ally be in any po­si­tion to be con­tribut­ing mean­ing­fully to such an am­bi­tious pro­ject of theirs. So need­less to say, for those first PR’s, just the thought of a ker­nel main­tainer re­view­ing my code had made me quite ner­vous.

Helping build Limbo quickly be­came my new ob­ses­sion. I split my time be­tween my job and div­ing deep into SQLite source code, aca­d­e­mic pa­pers on data­base in­ter­nals, and Andy Pavlo’s CMU lec­tures. I was ac­tive on the Turso Discord but I don’t think I con­sid­ered whether any­one was aware that one of the top con­trib­u­tors was do­ing so from a prison cell. My story and in­for­ma­tion are linked on my GitHub, but it’s sub­tle enough where you could miss it if you did­n’t read the whole pro­file. A cou­ple months later, I got a Discord mes­sage from Glauber in­tro­duc­ing him­self and ask­ing if we could meet.

In January, Glauber’s tweet about our in­ter­ac­tion caught the at­ten­tion of The Primeagen, and he ended up read­ing my blog post on his stream, bring­ing a whole lot of new at­ten­tion to it.

To this day I re­ceive semi-reg­u­lar emails ei­ther from de­vel­op­ers, col­lege kids or oth­ers who maybe have ei­ther gone through ad­dic­tion or sim­i­lar cir­cum­stances, or just want to reach out for ad­vice on how to best start con­tribut­ing to open source or op­ti­mize their learn­ing path.

I’m in­cred­i­bly proud to be an ex­am­ple to oth­ers of how far hard work, de­ter­mi­na­tion and dis­ci­pline will get you, and will be for­ever grate­ful for the op­por­tu­ni­ties given to me by the Maine Dept of Corrections to even be able to work hard in the first place, and to Unlocked Labs for giv­ing me a chance and hir­ing me at a time when most as­suredly no-one else would.

I’m also in­cred­i­bly proud to an­nounce that I am now work­ing for Turso full time, some­thing I would never have dreamed would be pos­si­ble just a few years ago, I’m very ex­cited to be a part of the team and to get to help build the mod­ern evo­lu­tion of SQLite.

Although some re­cent bad news from the court means that I won’t be com­ing home as early as my fam­ily and I had hoped, my only choice is to view this as a bless­ing and for the next 10 months, will in­stead just be able to con­tinue to ded­i­cate time and fo­cus to ad­vanc­ing my ca­reer at such a level that just would­n’t be pos­si­ble oth­er­wise.

Thank you to every­one who has taken the time to reach out over the past cou­ple years, to my team at Unlocked Labs, and es­pe­cially my par­ents. Thanks to Turso for the op­por­tu­nity and to all the other com­pa­nies with fair chance hir­ing poli­cies who be­lieve that peo­ple de­serve a sec­ond chance. This jour­ney has been to­tally sur­real and every day I am still in awe of how far my life has come from the life I lived even just a few years ago.

Thanks to war, geopol­i­tics, and cli­mate change, Europe will have more fre­quent and more se­vere in­ter­net

dis­rup­tions

in the very near fu­ture. Governments and busi­nesses need to pre­pare for cat­a­strophic loss of com­mu­ni­ca­tions. Unfortunately, the nec­es­sary changes are risky and ex­pen­sive, which means they won’t do it un­til a

cri­sis is al­ready here. However, small groups of vol­un­teers with a lit­tle bit of time and money can pro­vide cru­cial ini­tial lead­er­ship to boot­strap re­cov­ery.

An Internet Resiliency Club is a group of in­ter­net ex­perts who can com­mu­ni­cate with each other across a few kilo­me­ters with­out any cen­tral­ized in­fra­struc­ture us­ing cheap, low-power, un­li­censed

LoRa ra­dios and open source

Meshtastic text mes­sag­ing soft­ware. These vol­un­teer groups can use their ra­dios, tech­ni­cal skills, and per­sonal con­nec­tions with other ex­perts to re­store in­ter­net con­nec­tiv­ity.

This page is a quick-start guide to form­ing your own Internet Resiliency Club. You can also join a mail­ing

list for gen­eral ques­tions and dis­cus­sion about in­ter­net re­siliency clubs:

I am Valerie Aurora, a sys­tems soft­ware en­gi­neer with 25 years of ex­pe­ri­ence in open source soft­ware, op­er­at­ing sys­tems, net­work­ing, file sys­tems, and vol­un­teer or­ga­niz­ing. When I moved from San Francisco to Amsterdam in 2023, I started look­ing for ways to give back to my new home. In ad­di­tion to sys­tems con­sult­ing, I am a spe­cial rap­por­teur for the EU’s Cyber Resilience Act, serve as a RIPE Meeting pro­gram com­mit­tee mem­ber, and speak at European tech­ni­cal con­fer­ences.

One of my night­mares is wak­ing up one morn­ing and dis­cov­er­ing that the power is out, the in­ter­net is down, my cell phone does­n’t work, and when I turn on the emer­gency ra­dio (if you have one), all you hear is “Swan Lake” on re­peat.

As a re­cent im­mi­grant to Amsterdam, I be­gan to re­al­ize that this night­mare was in­creas­ingly likely. Russia reg­u­larly knocks out com­mu­ni­ca­tions and power in Ukraine, us­ing both bombs and hack­ers. In 2022, German wind­mills were dis­abled by mal­ware aimed at Ukraine. Dubious tankers con­tinue to “accidentally” drag their an­chors and cut un­der­sea ca­bles in the Baltic. The head of NATO ad­vised every­one to keep three days of sup­plies at home.

What made me fi­nally take ac­tion is watch­ing a video cre­ated by

Ukrainian IXP

1-IX to teach other European coun­tries what Ukrainian in­ter­net op­er­a­tors have learned about hard­en­ing and re­pair­ing in­ter­net in­fra­struc­ture lead­ing up to and fol­low­ing the 2022 Russian in­va­sion. The prac­ti­cal re­al­i­ties of keep­ing net­works op­er­at­ing dur­ing war were sober­ing: build­ing camoflouged router rooms with 3 days of gen­er­a­tor power, re­plac­ing ac­tive fiber op­tic ca­ble with pas­sive, get­ting mil­i­tary ser­vice ex­emp­tions for their per­son­nel, etc.. You can watch the most re­cent ver­sion, “Network Resilience: Experiences of sur­vival and de­vel­op­ment

dur­ing the war in

Ukraine”, a 30 minute pre­sen­ta­tion at RIPE 90.

Unfortunately, the gov­ern­ment of the Netherlands is not fol­low­ing Ukraine’s lead. Bert Hubert’s blog post

de­scribes the Netherlands’ cloud-based “emergency com­mu­ni­ca­tions” sys­tem, which will def­i­nitely not work in any emer­gency that af­fects power or in­ter­net con­nec­tiv­ity.

I have asked many Dutch net­work op­er­a­tors if there is any of­fi­cial plan for the com­mu­ni­ca­tions equiv­a­lent of a “black start” of the elec­tri­cal grid. If there is one, it is­n’t be­ing shared with the peo­ple who will have to im­ple­ment it.

The fi­nal piece of the idea came from a class I took on Crisis

Engineering from Layer Aleph, on how or­ga­ni­za­tions fac­ing an ex­is­ten­tial cri­sis ei­ther swiftly trans­form them­selves into a more func­tional form, or they fail and be­come even more dys­func­tional. Our class’s first ques­tion was, “How do you con­vince an or­ga­ni­za­tion that a cri­sis is com­ing and they need to pre­pare for it?”

Their an­swer was both de­press­ing and free­ing: “You can’t. All you can do is be pre­pared with tools and a plan for when the cri­sis ar­rives. That’s when the or­ga­ni­za­tion will lis­ten.”

I started think­ing about what I could per­son­ally do with­out any help from gov­ern­ment or busi­nesses. What if I could or­ga­nize a group of vol­un­teer net­work­ing ex­perts who could com­mu­ni­cate with­out any cen­tral­ized in­fra­struc­ture? We could ef­fec­tively boot­strap com­mu­ni­ca­tions re­cov­ery with just a few vol­un­teers and some cheap hard­ware.

Initially I looked into ham ra­dio, but it is just too ex­pen­sive, dif­fi­cult, and power-hun­gry to be prac­ti­cal. Then Alexander Yurtchenko told me about LoRa (Long Range) ra­dio and Meshtastic, a cheap, low-power method of send­ing text mes­sages across a few kilo­me­ters.

After a few months of part-time re­search and or­ga­niz­ing, the Amsterdam Internet Resiliency Club was born. This page ex­ists to make it eas­ier for other peo­ple to start Internet Resiliency Clubs in their area.

The ev­i­dence that Internet Resiliency Clubs are nec­es­sary keeps grow­ing. Since I started this pro­ject, the city of Amsterdam an­nounced that it is plan­ning for three weeks with­out elec­tric­ity. Spain and Portugal lost power for most of a day. The U. S. re-elected Donald Trump, who may at some point re­al­ize that he can hold Europe hostage by threat­en­ing to cut off ac­cess to U.S.-owned in­ter­net ser­vices like AWS and Microsoft Exchange. Simultaneously, large parts of Dutch gov­ern­ment are mi­grat­ing to email hosted by Microsoft, and ma­jor Dutch tech­nol­ogy firms con­tinue to mi­grate to AWS and Microsoft Azure.

If you and I don’t do this, dear reader, no one will.

How to form an Internet Resiliency Club:

* Collect a group of in­ter­net-y peo­ple within ~10 km of each other

* Decide how to com­mu­ni­cate nor­mally (Signal, Matrix, email, etc.)

* Buy every­one LoRa (Long Range) ra­dios and a power­bank with trickle charge

If you work for a in­ter­net in­fra­struc­ture com­pany, you can sug­gest giv­ing in­ter­ested em­ploy­ees a LoRa ra­dio, a mo­bile phone power­bank, and maybe even a small so­lar panel for their per­sonal use (perhaps as part of an an­nual gift or bonus).

LoRa ra­dios have sev­eral ad­van­tages for use in emer­gency com­mu­ni­ca­tions:

* low-power (< 1W, can power with an or­di­nary mo­bile phone power­bank)

* can send text mes­sages across sev­eral line-of-sight hops (several kms)

* can con­nect via Bluetooth or WiFi to phones/​com­put­ers

* many ur­ban ar­eas have a good Meshtastic net­work al­ready

Amateur ham ra­dio can trans­mit at higher band­width for longer dis­tances, but re­quires ex­ten­sive train­ing, li­cens­ing, larger an­ten­nas, and more power. Ideally, both would be avail­able in an emer­gency.

With a LoRa ra­dio run­ning the Meshtastic firmware, any­one can send text mes­sages to any­one else with a Meshtastic node as long as it takes three or fewer for­wards from other Meshtastic nodes to get from source to des­ti­na­tion (usually around ~10 km but highly de­pen­dent on lo­cal ter­rain and weather).

Specifically, LoRa is a pro­pri­etary tech­nique for send­ing low bit-rate ra­dio mes­sages (~1 - 25 kbps) us­ing very low power (< 1W), de­rived from chirp spread spec­trum tech­niques. Meshtastic is open source firmware for LoRa ra­dios that uses a flood-for­ward mesh pro­to­col to send mes­sage across up to three line-of-sight hops be­tween LoRa nodes run­ning Meshtastic.

LoRa ra­dios are for sale on­line. The cheap­est ver­sions are de­vel­op­ment boards, in­tended for com­pa­nies to use while build­ing a prod­uct, of­ten with­out bat­ter­ies, cases, or good an­ten­nas. To use them, you must con­nect to them from a phone or com­puter, ei­ther over Bluetooth via the Meshtastic app or over WiFi us­ing a web browser. The more ex­pen­sive sys­tems may in­clude an en­clo­sure, bat­tery, so­lar panel, larger screen, key­board, etc. Some can be used with­out an ad­di­tional phone or com­puter.

LoRa ra­dios use rel­a­tively lit­tle power, of­ten in the range of 100 - 200 mA. A nor­mal mo­bile phone power bank with a ca­pac­ity of 10000 - 20000 mAh can power a LoRa ra­dio ap­prox­i­mately 2 - 8 days, de­pend­ing on chipset, time spent trans­mit­ting, whether WiFi or Bluetooth are in use, etc. The power­bank should sup­port “trickle charg­ing”; with­out this, many power­banks will stop sup­ply­ing power be­cause the power draw of many LoRa ra­dios is so low that the power­bank thinks noth­ing is con­nected and stops sup­ply­ing power.

LoRa ra­dios can be pow­ered by di­rectly plug­ging them into a small so­lar panel with USB out­put, or by charg­ing a bat­tery used by the LoRa ra­dio. A small fold­ing 800 cm^2 so­lar panel gen­er­at­ing 15w with a 5W/500 mA max out­put is suf­fi­cient to power many LoRa ra­dios. With this small of a setup, you don’t need fuses, charge con­trollers, buck/​boost con­vert­ers, or any­thing other than the so­lar panel and an op­tional mo­bile phone power bank.

LoRa ra­dios are avail­able in a huge range of ca­pa­bil­i­ties and fea­tures. For an Internet Resiliency Club, we rec­om­mend one of:

IMPORTANT: Never turn on a LoRa de­vice with­out an an­tenna at­tached! The power sent to the an­tenna can de­stroy the de­vice if there is no an­tenna at­tached to ra­di­ate it.

Note: While many LoRa de­vices have USB-C ports, they of­ten don’t im­ple­ment USB-C PD (Power Delivery) and won’t charge their bat­tery cor­rectly on USB-C to USB-C ca­bles. Use a USB-A to USB-C ca­ble (often sup­plied with the de­vice).

If you have more time than money, try the lat­est Heltec V3, cur­rently one of the cheap­est boards avail­able at around €20. It has a postage stamp-sized OLED screen, a cou­ple of tiny but­tons, WiFi/Bluetooth, and USB-C in­put/​power (but use a USB-A to USB-C ca­ble). Received mes­sages are dis­played on the OLED and can be cy­cled through with tiny but­tons. Sending mes­sages re­quires con­nect­ing to it via WiFi or Bluetooth.

It has no case, but the lit­tle plas­tic box it comes in can eas­ily be turned into one with a sharp pen knife. It also has no bat­tery, but it is a good idea to have a sep­a­rate power bank any­way since you need a work­ing phone or com­puter to send mes­sages. It has no GPS.

The Meshtastic

page on this

board

in­cludes links to pur­chase from in Europe. I bought mine from

TinyTronics.

If you have more money than time, I rec­om­mend the LILYGO T-Echo, a sim­ple small low-power ready-to-use hand­held de­vice for about €80. It has ~3cm square e-ink dis­play, a case with a few but­tons, Bluetooth, GPS, and about a day’s worth of bat­tery. Input/output/charging is via USB-C (but use a USB-A to USB-C ca­ble). Received mes­sages are dis­played on the e-ink screen and can be cy­cled through with the but­tons. Sending mes­sages re­quires con­nect­ing with an­other de­vice via Bluetooth.

The Meshtastic

page on this

board

in­cludes links to pur­chase from in Europe. I bought mine from

TinyTronics.

If you want a stand­alone de­vice that does­n’t re­quire a sep­a­rate phone or com­puter to send mes­sages, the LILYGO

T-Deck

in­cludes a key­board, track­ball, and touch screen for about €70 - 80, de­pend­ing on whether it in­cludes a case and whether the an­tenna is in­ter­nal or ex­ter­nal. It has about 8 hours of bat­tery. I’m not a fan be­cause the screen and key­board aren’t as good as the one on your phone and take ex­tra bat­tery to run. It is of­ten out of stock, es­pe­cially if you’re look­ing for a case and ex­ter­nal an­tenna.

The Meshtastic page on this

board

in­cludes links to pur­chase from in Europe.

Most of the an­ten­nas that ship with eval­u­a­tion boards are not very good. One op­tion for an up­grade if you’re us­ing the rec­om­mended 868 MHz net­work is the Taoglas

TI.08. A.

IMPORTANT: Never turn on a LoRa de­vice with­out an an­tenna at­tached! The power sent to the an­tenna can de­stroy the de­vice if there is no an­tenna at­tached to ra­di­ate it.

Some boards ship with Meshtastic al­ready in­stalled, but it’s un­doubt­edly sev­eral months out of date. Flashing LoRa boards is rel­a­tively easy; it can be as sim­ple as us­ing the Meshtastic web

browser

flasher

(requires Chrome or Edge) or drag­ging and drop­ping a file into a mounted USB drive pre­sented by the de­vice. A com­mand line tool us­ing a se­r­ial in­ter­face is also an op­tion, but may re­quire some fid­dling with a Python env.

In Europe, two fre­quen­cies are avail­able for use by LoRa: 868 MHz and

433

MHz. 868 MHz is the most pop­u­lar for Meshtastic users in

Europe. Several mo­dem pre­sets are avail­able; use the de­fault mode LONG_FAST un­less you have a spe­cific rea­son not to.

LoRa has

chan­nels, a stream of mes­sages us­ing the same en­cryp­tion key and chan­nel name. Each de­vice is con­fig­ured with a de­fault pri­mary chan­nel shared by all Meshtastic nodes. You can also con­fig­ure sec­ondary chan­nels that can only be ac­cessed by nodes with the same key and chan­nel name. Choose an en­cryp­tion key and chan­nel name for a shared sec­ondary chan­nel. You can share a QR code to con­fig­ure a de­vice with the ap­pro­pri­ate chan­nels and set­tings.

The best time to learn how to work to­gether with a group of peo­ple is be­fore a cri­sis, not dur­ing it. Crisis en­gi­neer­ing tells us that a team is more likely to be suc­cess­ful if every­one has al­ready worked to­gether.

Since this is a vol­un­teer group, “working” to­gether has to be fun. Invite your group to do fun things to­gether, chang­ing up what ac­tiv­ity you are do­ing, where it is lo­cated, and what time it is held so that a wide va­ri­ety of peo­ple can par­tic­i­pate.

If you have more ques­tions or sug­ges­tions, please join our mail­ing

list:

Many peo­ple helped me with Internet Resiliency Club:

My ear­li­est mem­ory of the ocean is of a trop­i­cal la­goon. Ammonites rose and fell in the warm wa­ter col­umn, oc­ca­sion­ally pro­pelling them­selves for­wards, their curled ram’s horn shells sur­pris­ingly stream­lined in the wa­ter.

This trop­i­cal la­goon was in fact in my imag­i­na­tion, fired as I ex­plored the old lime­stone quarry near my child­hood home in Leicester, some 60 miles from the coast.

For a small boy in the 1930s this was a mar­vel­lous place for ad­ven­tures, and the knowl­edge that mil­lions of years ago it would have been a warm and wild la­goon only in­creased its ap­peal. Here I could spend days search­ing for trea­sure buried in rocks laid down in an­cient trop­i­cal seas. Holding the fos­sils of long-dead sea crea­tures that I had chipped out of the rock, know­ing my eyes were the first ever to see them, ig­nited my cu­rios­ity. I would spend much of the rest of my life won­der­ing what lived be­low the sur­face of the ocean.

I have been for­tu­nate enough to live for nearly 100 years. During this time we have dis­cov­ered more about our ocean than in any other span of hu­man his­tory. Marine sci­ence has re­vealed nat­ural won­ders a young boy in the 1930s could never have imag­ined. New tech­nol­ogy has al­lowed us to film wildlife be­hav­iour I could only have dreamt of record­ing in the early stages of my ca­reer, and we have changed the ocean so pro­foundly that the next 100 years could ei­ther wit­ness a mass ex­tinc­tion of ocean life or a spec­tac­u­lar re­cov­ery.

To date we have done such a good job of telling the sto­ries of demise and col­lapse that many of us can all too eas­ily pic­ture a fu­ture ocean of bleached reefs, tur­tles chok­ing on plas­tic, sewage plumes, jel­ly­fish swarms and ghost towns where fish­ing vil­lages were once full of life. There may be much to fear in the near fu­ture, yet it could also be the most ex­cit­ing time to be alive.

We know al­ready that the ocean can re­cover. Mangroves and kelp forests can re­grow, whales can re­turn and dy­ing coastal com­mu­ni­ties can flour­ish once again.

Attenborough pre­pares for a dive while film­ing Life on Earth, 1979. The se­ries was watched by 500 mil­lion peo­ple world­wide

We now un­der­stand how to fix many of the biggest prob­lems we face as a species, and we have cen­turies of progress to draw on for in­spi­ra­tion. Indeed, in the past 100 years alone we have dra­mat­i­cally re­duced in­fant mor­tal­ity, sup­pressed many of our most feared dis­eases, in­creased ac­cess to ed­u­ca­tion and health­care, ac­quired sci­en­tific knowl­edge that has trans­formed our un­der­stand­ing of the world and co-op­er­ated on global is­sues to a de­gree never seen be­fore.

Young chil­dren play­ing on a beach to­day will live through per­haps the most con­se­quen­tial time for the hu­man species in the past 10,000 years. They will grow up to see how this story ends, to see how our choices play out. If we use our great dis­cov­er­ies, ap­ply our unique minds and di­rect our un­par­al­leled com­mu­ni­ca­tion and prob­lem-solv­ing skills to restor­ing our ocean, then those chil­dren will bring their own into a world where the biggest chal­lenges our species has ever faced have al­ready been nav­i­gated.

They will wit­ness decades of re­cov­ery and restora­tion. They will see shoals of fish, roosts of seabirds and pods of whales be­yond any­thing any­one alive has ever laid eyes upon. They will ex­pe­ri­ence the re­birth of coastal com­mu­ni­ties and the turn­ing point in the sta­bil­i­sa­tion of our cli­mate. But more than that, they will live in a world where our species, the most in­tel­li­gent to ex­ist on Earth, has moved be­yond try­ing to rule the waves and in­stead has learnt to thrive along­side the great­est wilder­ness of all.

I will not see how that story ends but, af­ter a life­time of ex­plor­ing our planet, I re­main con­vinced that the more peo­ple en­joy and un­der­stand the nat­ural world, the greater our hope of sav­ing both it and our­selves be­comes. With that in mind, here are some of my favourite ocean ex­pe­ri­ences, which I hope will in­spire you to look be­yond the shore and be­neath the waves.

Filming a blue whale in the Gulf of California, 2001

A blue whale cruises off the coast of Sri Lanka

Blue whales are per­fectly adapted for ocean voy­ag­ing. Their pow­er­ful yet stream­lined bod­ies en­able them to travel un­seen for thou­sands of miles each year. But in cer­tain places, and at cer­tain times, they come close to the shore in or­der to give birth and to suckle their young.

One such place and time is the Gulf of California dur­ing the win­ter months. It was there that I went with a team from the BBC to try and film a blue whale for a se­ries called The Life of Mammals. Even to­day, no one would de­scribe the blue whale as easy to film. But al­most 25 years ago it was far more chal­leng­ing. There were no drones to launch within sec­onds from a boat; nor were there satel­lite tags to alert you to a tracked whale’s lo­ca­tion. We had to rely on spot­ters on the shore and hope that a light plane guided by them could fly to the right place in time to cap­ture an aer­ial view of a whale swim­ming along­side our boat.

The be­gin­ning of the new mil­len­nium was only 15 years af­ter the ban on com­mer­cial whal­ing had been agreed. So the to­tal num­ber of blue whales in the ocean was ap­prox­i­mately 5,000 — only 2 per cent of their nat­ural level.

David Attenborough gets wet as a whale sur­faces close to his boat in The Life of Mammals, which aired in 2002-3

To these dif­fi­cul­ties we added a chal­lenge of our own. The shot we wanted was one in which, as I was speak­ing in a small in­flat­able launch, a whale would break the sur­face along­side me so that both it and I ap­peared in the same frame, and thus give as vivid an idea as pos­si­ble of just how gi­gan­tic it is.

Early one morn­ing we left har­bour and headed for the bay. Our pi­lot guide in his slow-fly­ing air­craft ap­peared over­head and cir­cled sev­eral hun­dred feet above the ocean. He had ex­plained to us how he could dis­tin­guish the spout of a blue whale from that of other species — it shoots up to 30 feet in a rel­a­tively straight jet, and the height, vol­ume and sheer power make it hang in the air for far longer than a spout made by any other kind of whale.

Once he had spot­ted one, he would tell us on the ra­dio which way to go in the hope that we could catch up with it be­fore it dived again. After sev­eral at­tempts we man­aged to do that. As soon as we were within 20 yards of it, we pushed a small in­flat­able launch over the side. I jumped in, tied my­self on, and within sec­onds we were above the whale as it cruised 20 feet or so be­low the sur­face.

“It’s a blue whale,” I shouted ex­cit­edly over the noise of our out­board en­gine, and a great spout of wa­ter shot into the air and fell, drench­ing me. It was one of the most thrilling mo­ments of my life.

• The David Attenborough lo­ca­tions that could in­spire your next hol­i­day

Drifting in a wet­suit above a sub­ma­rine for­est in south­ern California, I found my­self along­side one of the most bliss­ful of crea­tures. On its back, all four paws tucked into its body fur for warmth, gen­tly rolling in a man­ner that brought to mind a swad­dled new­born baby, lay a south­ern sea ot­ter. They were once seen linked to­gether in rafts hun­dreds strong, but this one was alone and seemed to be quite un­con­cerned by my clumsy at­tempts to float nearby.

I was no more than 200 yards off­shore, prepar­ing to record a piece to cam­era on the wildlife of the Pacific coast of North America. If I looked to­wards shore I could see houses and the odd car, yet if I looked down I felt I was in a wilder­ness. The for­est be­neath my ot­ter com­pan­ion and me was one of gi­ant kelp, each frond an­chored by a hold­fast to a rock on the sea floor some 150 feet be­low. I only had a snorkel, so those depths were out of reach.

But not for my neigh­bour. Periodically it dived down be­yond my sight. A sea ot­ter’s hind paws are fully webbed and rea­son­ably flat, so al­though they are ca­pa­ble of mov­ing fast on land, they are also ef­fec­tive divers. They can close their nos­trils and ears, and their lungs are so big that not only can they float with­out any ef­fort but they can also re­main un­der­wa­ter for about four min­utes at a time. This ot­ter was div­ing in one of the rich­est ma­rine en­vi­ron­ments on the planet, so find­ing food was no prob­lem.

The sea ot­ter sud­denly reap­peared be­side me. It had used its sen­si­tive whiskers and front paws to lo­cate and col­lect a clam from the sea floor. Once back on the sur­face and float­ing on its back, it pro­duced both a clam and a rock from a pouch of skin un­der its forelegs. I watched, cap­ti­vated by the prac­tised skill with which it bal­anced the rock on its belly and then smashed the clamshell re­peat­edly against it un­til the shell broke apart. Sea ot­ters are one of the few species that, like hu­man be­ings, reg­u­larly use tools.

They eat a wide range of the in­hab­i­tants of the kelp for­est but one is of par­tic­u­lar im­por­tance to them — sea urchins. In a healthy, bal­anced kelp for­est sea urchins play a key role, act­ing like a kind of kelp gar­dener. They gnaw away at the al­gae grow­ing on the rocks and in do­ing so cre­ate pits that en­able the kelp to an­chor their hold­fast. Left unchecked, the urchins can de­stroy such a for­est by eat­ing the hold­fasts that keep the kelp in place.

The im­por­tance of the sea ot­ter was re­vealed when al­most 200 years of hunt­ing brought them to the verge of ex­tinc­tion. Unusually for a ma­rine mam­mal, sea ot­ters don’t have blub­ber, so they were not tar­geted by hu­mans in the way that seals were, for the ex­trac­tion of oil.

In lieu of blub­ber, how­ever, they have the thick­est fur of any mam­mal, a dou­ble-lay­ered pelt that en­ables them to keep warm in these frigid seas, and in the 18th and 19th cen­turies they were hunted for that in their thou­sands. As a con­se­quence, the global pop­u­la­tion fell from 150,000-300,000 to fewer than 2,000 in­di­vid­u­als.

Urchin num­bers ex­ploded and as a re­sult many kelp forests all but van­ished, tak­ing with them much of the other life that used the forests for food or shel­ter. The del­i­cate bal­ance of this com­plex sys­tem was dev­as­tat­ingly dis­rupted by the tar­geted re­moval of a sin­gle species.

But promis­ingly, this process can also hap­pen in re­verse. Since hunt­ing sea ot­ters was banned in the early 1900s, num­bers have slowly re­cov­ered across sig­nif­i­cant parts of their old habi­tats. Recovery is not yet com­plete, but where it has hap­pened the ef­fects on the kelp forests are of­ten spec­tac­u­lar. As the ot­ters feast on the urchins, the kelp gets some respite. Being so fast-grow­ing, it quickly be­gins to pro­vide habi­tat that at­tracts other species, in­clud­ing other urchin eaters.

Capuchin mon­keys use ‘sheer in­tel­li­gence’ to sur­vive in the man­grove for­est

While mak­ing The Life of Mammals I spent sev­eral weeks in a man­grove for­est hop­ing to film the be­hav­iour of two sep­a­rate troops of ex­tremely in­tel­li­gent mon­keys — ca­puchins. We wanted to show how sheer in­tel­li­gence en­abled dif­fer­ent mon­key species to thrive in a va­ri­ety of dif­fi­cult con­di­tions — and it is fair to say that none of us ex­pected to be film­ing it in an ocean habi­tat! But we had read sci­en­tific stud­ies of the way ca­puchins har­vested shell­fish at low tide in a man­grove for­est in Costa Rica, and it sounded an in­ter­est­ing way to be­gin the pro­gramme.

The ca­puchin is a par­tic­u­larly clever species of mon­key. Capuchins are of­ten de­scribed sim­ply as “inquisitive”, but when watch­ing them at close range for a pe­riod of time you re­alise that, much like our­selves, they are able to imag­ine the fu­ture and plan how to deal with the prob­lems it will bring — ex­actly the char­ac­ter­is­tics re­quired to ex­ploit the com­plex world of man­groves.

We could­n’t hope to track the ca­puchins in the man­groves; they moved through the tan­gles of aer­ial roots much faster than we could. But we found a suit­ably open area well stocked with crabs, clams and oys­ters, and waited. Eventually a troop of ca­puchins ar­rived. Some of the braver ones plunged their hands into holes in the mud. The suc­cess­ful ones pulled out crabs, the un­suc­cess­ful quickly with­drew in pain! It was fas­ci­nat­ing to watch. But the be­hav­iour we re­ally wanted to film was the way in which they lo­cated and ate clams.

The troop moved with the ebb and flow of the tides. Each day the muddy ground would be ex­posed ap­prox­i­mately 50 min­utes later than the pre­vi­ous day, and the mon­keys ad­justed their move­ments ac­cord­ingly. By the time we had been film­ing for a few days, they took lit­tle no­tice of us and al­lowed us to get close and film as they dug in the mud and lo­cated clams.

The shell­fish clamp the two halves of their shell so tightly that even a hu­man can’t open them with­out a knife or sim­i­lar tool. But the ca­puchins have worked out their own way of get­ting at a clam’s flesh. Having col­lected one, they take it to a con­ve­nient branch and start knock­ing it, over and over again. Eventually the clam gets so tired it re­laxes its mus­cle and the ca­puchin is able to prise it open.

Ever since I was a boy I had been thrilled by pic­tures of the Great Barrier Reef ’s mul­ti­coloured, in­fi­nitely var­ied colonies of coral and its is­lands thronged by im­mense num­bers of breed­ing seabirds. I had al­ways yearned to see this won­der with my own eyes. This was my chance.

With the help of Vince, the ac­quain­tance of a friend, I sailed north­wards from Cairns up the reef, stop­ping to in­ves­ti­gate any is­land or reef that par­tic­u­larly at­tracted us, un­til on the 14th day we reached Raine Island — the reef’s north­ern limit.

The is­land was said to have one of the biggest and most var­ied colonies of breed­ing seabirds to be found any­where on the reef. It was also the world’s largest breed­ing site for green tur­tles.

Happily for us, there were clouds of seabirds as nu­mer­ous and dense as I have ever seen. The most abun­dant were two species of tern — the noddy and the white-capped. There were three species of booby — the com­mon, the brown and the red-footed. But for me the most im­pres­sive and cer­tainly least fa­mil­iar were the frigate birds: glossy black, with six-foot wingspans and long, deeply forked tails.

On our first walk around the is­land we saw per­haps twenty curv­ing tracks weav­ing through the sand hills. I did­n’t know at the time, but sci­en­tists now be­lieve that every year more than 60,000 fe­male green tur­tles travel im­mense dis­tances to get to this one small and re­mote is­land. The few here now were just end-of-sea­son strag­glers.

Attenborough film­ing the se­ries Life on Earth in 1978 — a three-year pro­ject that raised the bar for nat­ural his­tory film-mak­ing

Each morn­ing we found sev­eral that had dragged them­selves far enough in­land to be be­yond the reach of a high tide and were now dig­ging with pow­er­ful swishes of their fore-flip­pers. Every now and then they swiv­elled slightly so that the holes they were cre­at­ing were cir­cu­lar. When one fe­male was down in the sand by about 12 inches, she started to use her hind flip­pers as well, un­til fi­nally the top of her shell was vir­tu­ally level with the sur­face of the sand. Then, us­ing just one hind flip­per at a time, she be­gins to widen the down­ward pas­sage to cre­ate an egg cham­ber.

All the time she weeps to clear sand from her eyes. She gasps, mak­ing great breathy ex­ha­la­tions, and fol­lows each one with a sud­den in­take of air, as though she is still in the sea and prepar­ing to make an­other dive. The nest hole must not be too deep, for the eggs will need the warmth of the sun if they are to de­velop. But it must, nonethe­less, be deep enough to be be­yond the reach of preda­tors. Then she lays a hun­dred or so eggs, fills in the hole and re­turns to the sea. A sin­gle fe­male may re­peat this ex­haust­ing process half a dozen times dur­ing a sin­gle breed­ing sea­son.

The sheer quan­tity of hatch­lings that emerge on the beaches of Raine Island in a sin­gle sea­son is hard to imag­ine. But great num­bers are es­sen­tial be­cause only one out of every thou­sand hatch­lings is likely to reach ma­tu­rity. Within min­utes of ap­pear­ing on the sur­face of the sand, most are eaten by birds. Those that do reach the wa­ter are then at­tacked by ma­rine preda­tors. Only a tiny mi­nor­ity reach the rel­a­tive safety of the open ocean.

I did­n’t get to dive in the shal­low, warm wa­ters of the Great Barrier Reef un­til 1957. I was so taken aback by the spec­ta­cle be­fore me that I mo­men­tar­ily for­got to breathe. I could have spent days swim­ming above it and never tired of the colours, the move­ment, the in­ter­ac­tions. It is life at its most mes­meris­ing.

Nothing can pre­pare you for ac­tu­ally see­ing so many dif­fer­ent species, all with their own way of over­com­ing life’s tri­als, some­how fit­ting to­gether in an ecosys­tem so vivid and vi­brant. Even though we know that a trop­i­cal rain­for­est har­bours ex­tra­or­di­nary an­i­mal di­ver­sity, you see rel­a­tively lit­tle of it on a sin­gle walk. Yet on that half-hour dive I saw more species of an­i­mals than I could have be­gun to count, let alone iden­tify.

© Sir David Attenborough and Colin Butfield, 2025. Extracted from Ocean: Earth’s Last Wilderness by Sir David Attenborough and Colin Butfield (John Murray Press £28), pub­lished on Thursday. Order at times­book­shop.co.uk

Times+ mem­bers can win a signed copy of Oceans. Visit thetimes.com/​time­s­plus to find out more

Previously I have cov­ered a rel­a­tively ob­scure now-re­moved place­holder string in Android that dou­bles as an easter egg, the fic­ti­tious car­rier by the name of El Telco Loco. But this time it is about meth­ods and other parts of the pub­licly fac­ing Android API that may gen­er­ally be more hu­mourous than they are use­ful. Easter eggs, jokes, what­ever you want to call them, that are vis­i­ble to Android app de­vel­op­ers rather than reg­u­lar users.

While it may ini­tially look like a joke when it’s de­scribed as re­turn­ing true if the UI is “currently be­ing messed with by a mon­key” with­out any fur­ther elab­o­ra­tion in the doc­u­men­ta­tion, this is prob­a­bly the one in the list with the most use­ful­ness at­tached to it.

It is re­fer­ring to the UI Exerciser Monkey, which is a de­vel­oper tool for Android that sim­u­lates ran­dom se­quences of user in­put in or­der to stress-test apps. So this method will re­turn a boolean of whether the Monkey is cur­rently run­ning or not.

The in­tro­duc­tion of such a method to de­tect the us­age of the Monkey ap­pears to have an ori­gin in some­thing that hap­pened dur­ing Android’s de­vel­op­ment, as per a quote from the book Androids: The Team that Built the Android Operating System:

One day I walked into the mon­key lab to hear a voice say, ’911 -What’s your emer­gency?” That sit­u­a­tion re­sulted in Dianne adding a new func­tion to the API, is­User­A­Mon­key() which is used to gate ac­tions that mon­keys should­n’t take dur­ing tests (including di­al­ing the phone and re­set­ting the de­vice).

Indeed, when feed­ing ran­dom and in­her­ently un­pre­dictable in­put into an app, you would want to have some way of lock­ing away por­tions of an app that may have un­in­tended real-world con­se­quences such as call­ing emer­gency ser­vices. As such, is­User­A­Mon­key was im­ple­mented and later made its way into the pub­lic API in Android 2.2 Froyo (API 8).

This one is more of a joke. The de­vel­oper doc­u­men­ta­tion says it is “used to de­ter­mine whether the user mak­ing this call is sub­ject to tele­por­ta­tions”, which in it­self is likely a ref­er­ence to a hid­den col­umn in the Chrome task man­ager that shows how many goats a browser process has tele­ported.

It was first in­tro­duced in Android 4.2 (API 17), and orig­i­nally just re­turned false. However in Android 5.0 Lollipop (API 21) it was changed to “automatically iden­tify goats us­ing ad­vanced goat recog­ni­tion tech­nol­ogy”. The game Goat Simulator had re­leased ear­lier that year and was made avail­able for Android in September dur­ing Lollipop’s de­vel­op­ment, so this method was changed to in­stead de­tect the pres­ence of the Android ver­sion of Goat Simulator be­ing in­stalled on the de­vice:

Later in Android 11 (API 30), it was changed such that apps tar­get­ting API 30 and above will once again al­ways re­turn false when the method is called. According to the de­vel­oper doc­u­men­ta­tion this was made to “protect goat pri­vacy”.

Android 11 is also the ver­sion where the QUERY_ALL_PACKAGES per­mis­sion was in­tro­duced, mean­ing that apps tar­get­ting Android 11 would not be able to query for in­for­ma­tion of other apps through the PackageManager with­out this per­mis­sion. So it makes sense to also wall off this method in or­der to not leak any in­for­ma­tion about other apps in­stalled on an user’s de­vice, even as a joke.

This con­stant refers to a de­vice pol­icy added in Android 6 Marshmallow (API 23) which re­stricts the user from hav­ing “fun”. The de­scrip­tion given in the de­vel­oper doc­u­men­ta­tion is, iron­i­cally, amus­ing and re­minds me of some­thing GLaDOS would prob­a­bly say:

Specifies if the user is not al­lowed to have fun. In some cases, the de­vice owner may wish to pre­vent the user from ex­pe­ri­enc­ing amuse­ment or joy while us­ing the de­vice.

This is in fact a real de­vice pol­icy that a de­vice owner may change to re­strict what users of the de­vice is able to do with it. And third-par­ties can then hook into this to dis­able fea­tures of their app that are deemed “too fun”. I don’t know if any third-party apps ac­tu­ally make use of it, but in the Android sys­tem it is used to dis­able the Android ver­sion easter egg that shows up when press­ing the ver­sion la­bel in the set­tings.

Considering that “fun” easter eggs like the Google Chrome “No in­ter­net” Dinosaur minigame end up be­ing dis­trac­tions that e.g. schools want to dis­able for en­rolled de­vices (see Chromium is­sue #41159706), maybe the Android ver­sion easter egg could very much be a dis­trac­tion de­pend­ing on the ver­sion.

The Chronometer class had a new method by the name of is­The­Fi­nal­Count­down added to it in Android 8 Oreo (API 26). When called, it will send an Intent to open the YouTube video for The Final Countdown by Europe.

No re­ally. That’s what it does:

This con­stant was added in Android 2.3 Gingerbread (API 8) and is used to de­scribe a de­vice that sup­ports track­ing 5 si­mul­ta­ne­ous touch in­puts, with the name be­ing a ref­er­ence to Jazz hands.

According to the de­vel­oper doc­u­men­ta­tion, WTF stands for “What a Terrible Failure” (sure…), and is in­tended to log things that should never hap­pen. It logs the mes­sage at the as­ser­tion level.

This is a method with an oddly hu­mourous in­for­mal name, which was likely caused by some de­vel­oper com­ing up blank on nam­ing it and has now ended up in the pub­lic Android API, be­ing added in Android 3.0 Honeycomb (API 11). It gets called by an AppWidgetHost when ad­vanc­ing the views in­side of the AdapterViewFlipper ob­ject.

Indeed, nam­ing things is one of the two hard prob­lems in com­puter sci­ence, the other be­ing off-by-one er­rors and cache in­val­i­da­tion.

The Android Binder sys­tem is used for per­form­ing IPC and trans­ac­tions are dis­tin­guished us­ing dif­fer­ent types, one of them be­ing… TWEET_TRANSACTION. It was added in Android 3.2 Honeycomb (API 13) and claims to be used to send a tweet to a tar­get ob­ject.

It does not ac­tu­ally do any­thing, let alone send a tweet. The doc­u­ment men­tions that mes­sages have a limit of 130 char­ac­ters, ref­er­enc­ing Twitter’s old mes­sage char­ac­ter limit.

In a sim­i­lar fash­ion, a new trans­ac­tion type by the name of LIKE_TRANSACTION was added in Android 4.0.3 ICS (API 15). It’s used to tell an app that the caller likes it, there is no counter to keep track of the amount of likes but it is claimed that send­ing such trans­ac­tions will im­prove the ap­p’s self-es­teem.

I do have to ad­mit I did­n’t know what a Tricorder is, but it ap­pears to be a fic­tional de­vice from Star Trek and the con­stant was “added” in Android 1.0 (meaning it likely was pre­sent since be­fore Android’s first of­fi­cial re­lease).

The SENSOR_* con­stants in SensorManager have since then been dep­re­cated in API level 15 in favour of the Sensor class, which does not in­clude any equiv­a­lent ref­er­ence to the Tricorder. Unfortunate.

The SensorManager class has a lot of con­stants which store the grav­i­ta­tional ve­loc­ity of var­i­ous bod­ies in our so­lar sys­tem rang­ing from GRAVITY_SUN to GRAVITY_PLUTO. While whether any of these out­side of GRAVITY_EARTH is use­ful in any real-world sce­nar­ios is de­bat­able, there are some that are ac­tu­ally just jokes.

GRAVITY_DEATH_STAR_I stores the grav­ity of the first Death Star in SI units (referred to as Empire units). This ap­pears to be a Star Wars ref­er­ence.

GRAVITY_THE_ISLAND stores the grav­ity of “the is­land”. Apparently this is a ref­er­ence to The Island in the 2004 TV show Lost.

Last one, and this one is par­tic­u­larly crazy. Did you know there is a hid­den tag in­side the Android view lay­out sys­tem by the name of ? Because that is a thing:

It makes any chil­dren that is wrapped in­side of it blink, like the old HTML tag. This one ap­pears to be com­pletely un­doc­u­mented in the Android Developer ref­er­ence, but was added in a com­mit in 2011 with the ti­tle “Improve LayoutInflater’s com­pli­ance” (right…) and is still pre­sent in the AOSP mas­ter branch.

Whether you should ac­tu­ally use it is de­bat­able how­ever.

Did you find the blog post to be in­for­ma­tive, amus­ing or oth­er­wise in­ter­est­ing? All blog posts are writ­ten by a hu­man who would ap­pre­ci­ate a do­na­tion

if you got some value out of this piece of writ­ing.

Nanonets-OCR-s is a pow­er­ful, state-of-the-art im­age-to-mark­down OCR model that goes far be­yond tra­di­tional text ex­trac­tion. It trans­forms doc­u­ments into struc­tured mark­down with in­tel­li­gent con­tent recog­ni­tion and se­man­tic tag­ging, mak­ing it ideal for down­stream pro­cess­ing by Large Language Models (LLMs).

Nanonets-OCR-s is packed with fea­tures de­signed to han­dle com­plex doc­u­ments with ease:

* LaTeX Equation Recognition: Automatically con­verts math­e­mat­i­cal equa­tions and for­mu­las into prop­erly for­mat­ted LaTeX syn­tax. It dis­tin­guishes be­tween in­line ($…$) and dis­play ($$…$$) equa­tions.

* Intelligent Image Description: Describes im­ages within doc­u­ments us­ing struc­tured tags, mak­ing them di­gestible for LLM pro­cess­ing. It can de­scribe var­i­ous im­age types, in­clud­ing lo­gos, charts, graphs and so on, de­tail­ing their con­tent, style, and con­text.

* Signature Detection & Isolation: Identifies and iso­lates sig­na­tures from other text, out­putting them within a tag. This is cru­cial for pro­cess­ing le­gal and busi­ness doc­u­ments.

* Watermark Extraction: Detects and ex­tracts wa­ter­mark text from doc­u­ments, plac­ing it within a tag.

* Smart Checkbox Handling: Converts form check­boxes and ra­dio but­tons into stan­dard­ized Unicode sym­bols (☐, ☑, ☒) for con­sis­tent and re­li­able pro­cess­ing.

* Complex Table Extraction: Accurately ex­tracts com­plex ta­bles from doc­u­ments and con­verts them into both mark­down and HTML table for­mats.

from PIL im­port Image

from trans­form­ers im­port AutoTokenizer, AutoProcessor, AutoModelForImageTextToText

mod­el_­path = “nanonets/Nanonets-OCR-s”

model = AutoModelForImageTextToText.from_pretrained(

mod­el_­path,

torch_d­type=“auto”,

de­vice_map=“auto”,

at­tn_im­ple­men­ta­tion=“flash_at­ten­tion_2”

model.eval()

to­k­enizer = AutoTokenizer.from_pretrained(model_path)

proces­sor = AutoProcessor.from_pretrained(model_path)

def ocr_­page_with­_­nanonet­s_s(im­age_­path, model, proces­sor, max_new_­to­kens=4096):

prompt = “”″Extract the text from the above doc­u­ment as if you were read­ing it nat­u­rally. Return the ta­bles in html for­mat. Return the equa­tions in LaTeX rep­re­sen­ta­tion. If there is an im­age in the doc­u­ment and im­age cap­tion is not pre­sent, add a small de­scrip­tion of the im­age in­side the tag; oth­er­wise, add the im­age cap­tion in­side . Watermarks should be wrapped in brack­ets. Ex: . Page num­bers should be wrapped in brack­ets. Ex: or . Prefer us­ing ☐ and ☑ for check boxes.“”″

im­age = Image.open(image_path)

mes­sages = [

{“role”: “system”, “content”: “You are a help­ful as­sis­tant.“},

{“role”: “user”, “content”: [

{“type”: “image”, “image”: f”file://{​im­age_­path}“},

{“type”: “text”, “text”: prompt},

text = proces­sor.ap­ply_chat_tem­plate(mes­sages, to­k­enize=False, ad­d_­gen­er­a­tion_prompt=True)

in­puts = proces­sor(text=[text], im­ages=[im­age], padding=True, re­turn_ten­sors=“pt”)

in­puts = in­puts.to(model.de­vice)

out­put_ids = model.gen­er­ate(**in­puts, max_new_­to­kens=max_new_­to­kens, do_sam­ple=False)

gen­er­at­ed_ids = [output_ids[len(input_ids):] for in­put_ids, out­put_ids in zip(in­puts.in­put_ids, out­put_ids)]

out­put_­text = proces­sor.batch_de­code(gen­er­at­ed_ids, skip_spe­cial_­to­kens=True, clean_up­_­to­k­eniza­tion_­spaces=True)

re­turn out­put_­text[0]

im­age_­path = “/path/to/your/document.jpg”

re­sult = ocr_­page_with­_­nanonet­s_s(im­age_­path, model, proces­sor, max_new_­to­kens=15000)

print(re­sult)

vllm serve nanonets/​Nanonets-OCR-s

from ope­nai im­port OpenAI

im­port base64

client = OpenAI(api_key=“123”, base_url=“http://​lo­cal­host:8000/​v1”)

model = “nanonets/Nanonets-OCR-s”

def en­code_im­age(im­age_­path):

with open(im­age_­path, “rb”) as im­age_­file:

re­turn base64.b64en­code(im­age_­file.read()).de­code(“utf-8”)

def ocr_­page_with­_­nanonet­s_s(img_base64):

re­sponse = client.chat.com­ple­tions.cre­ate(

model=model,

mes­sages=[

“role”: “user”,

“content”: [

“type”: “image_url”,

“image_url”: {“url”: f”data:im­age/​png;base64,{img_base64}“},

“type”: “text”,

“text”: “Extract the text from the above doc­u­ment as if you were read­ing it nat­u­rally. Return the ta­bles in html for­mat. Return the equa­tions in LaTeX rep­re­sen­ta­tion. If there is an im­age in the doc­u­ment and im­age cap­tion is not pre­sent, add a small de­scrip­tion of the im­age in­side the tag; oth­er­wise, add the im­age cap­tion in­side . Watermarks should be wrapped in brack­ets. Ex: . Page num­bers should be wrapped in brack­ets. Ex: or . Prefer us­ing ☐ and ☑ for check boxes.”,

tem­per­a­ture=0.0,

max_­to­kens=15000

re­turn re­sponse.choices[0].mes­sage.con­tent

test_img_­path = “/path/to/your/document.jpg”

img_base64 = en­code_im­age(test_img_­path)

print(ocr_­page_with­_­nanonet­s_s(img_base64))

pip in­stall do­cext

python -m do­cext.app.app –model_name host­ed_vllm/​nanonets/​Nanonets-OCR-s

@misc{Nanonets-OCR-S,

ti­tle={Nanonets-OCR-S: A model for trans­form­ing doc­u­ments into struc­tured mark­down with in­tel­li­gent con­tent recog­ni­tion and se­man­tic tag­ging},

au­thor={Sou­vik Mandal and Ashish Talewar and Paras Ahuja and Prathamesh Juvatkar},

year={2025},

Isaac Newton was never en­tirely happy with his law of uni­ver­sal grav­i­ta­tion. For decades af­ter pub­lish­ing it in 1687, he sought to un­der­stand how, ex­actly, two ob­jects were able to pull on each other from afar. He and oth­ers came up with sev­eral me­chan­i­cal mod­els, in which grav­ity was not a pull, but a push. For ex­am­ple, space might be filled with un­seen par­ti­cles that bom­bard the ob­jects on all sides. The ob­ject on the left ab­sorbs the par­ti­cles com­ing from the left, the one on the right ab­sorbs those com­ing from the right, and the net ef­fect is to push them to­gether.

Those the­o­ries never quite worked, and Albert Einstein even­tu­ally pro­vided a deeper ex­pla­na­tion of grav­ity as a dis­tor­tion of space and time. But Einstein’s ac­count, called gen­eral rel­a­tiv­ity, cre­ated its own puz­zles, and he him­self rec­og­nized that it could not be the fi­nal word. So the idea that grav­ity is a col­lec­tive ef­fect — not a fun­da­men­tal force, but the out­come of swarm be­hav­ior on a finer scale — still com­pels physi­cists.

Earlier this year, a team of the­o­ret­i­cal physi­cists put for­ward what might be con­sid­ered a mod­ern ver­sion of those 17th-century me­chan­i­cal mod­els. “There’s some kind of gas or some ther­mal sys­tem out there that we can’t see di­rectly,” said Daniel Carney of Lawrence Berkeley National Laboratory, who led the ef­fort. “But it’s ran­domly in­ter­act­ing with masses in some way, such that on av­er­age you see all the nor­mal grav­ity things that you know about: The Earth or­bits the sun, and so forth.”

This pro­ject is one of the many ways that physi­cists have sought to un­der­stand grav­ity, and per­haps the bendy space-time con­tin­uum it­self, as emer­gent from deeper, more mi­cro­scopic physics. Carney’s line of think­ing, known as en­tropic grav­ity, pegs that deeper physics as es­sen­tially just the physics of heat. It says grav­ity re­sults from the same ran­dom jig­gling and mix­ing up of par­ti­cles — and the at­ten­dant rise of en­tropy, loosely de­fined as dis­or­der — that gov­erns steam boil­ers, car en­gines and re­frig­er­a­tors.

Attempts at mod­el­ing grav­ity as a con­se­quence of ris­ing en­tropy have cropped up now and again for sev­eral decades. Entropic grav­ity is very much a mi­nor­ity view. But it’s one that won’t die, and even de­trac­tors are loath to dis­miss it al­to­gether. The new model has the virtue of be­ing ex­per­i­men­tally testable — a rar­ity when it comes to the­o­ries about the mys­te­ri­ous un­der­pin­nings of the uni­ver­sal at­trac­tion.

What makes Einstein’s the­ory of grav­ity so re­mark­able is not just that it works (and does so with sub­lime math­e­mat­i­cal beauty), but that it be­trays its own in­com­plete­ness. General rel­a­tiv­ity pre­dicts that stars can col­lapse to form black holes, and that, at the cen­ters of these ob­jects, grav­ity be­comes in­fi­nitely strong. There, the space-time con­tin­uum tears open like an over­loaded gro­cery bag, and the the­ory is un­able to say what comes next. Furthermore, gen­eral rel­a­tiv­ity has un­canny par­al­lels to heat physics, even though not a sin­gle ther­mal con­cept went into its de­vel­op­ment. It pre­dicts that black holes only grow, never shrink, and only swal­low, never dis­gorge. Such ir­re­versibil­ity is char­ac­ter­is­tic of the flow of heat. When heat flows, en­ergy takes a more ran­dom­ized or dis­or­dered form; once it does, it is un­likely to re­order it­self spon­ta­neously. Entropy quan­ti­fies this growth of dis­or­der.

Indeed, when physi­cists used quan­tum me­chan­ics to study what hap­pens in the dis­torted space-time around a black hole, they find that black holes give off en­ergy like any hot body. Because heat is the ran­dom mo­tion of par­ti­cles, these ther­mal ef­fects sug­gest to many re­searchers that black holes, and the space-time con­tin­uum in gen­eral, ac­tu­ally con­sist of some kind of par­ti­cles or other mi­cro­scopic com­po­nents.

Following the clues from black holes, physi­cists have pur­sued mul­ti­ple ap­proaches to un­der­stand­ing how space-time emerges from more mi­cro­scopic com­po­nents. The lead­ing ap­proach takes off from what’s known as the holo­graphic prin­ci­ple. It says the emer­gence of space-time works a bit like an or­di­nary holo­gram. Just as a holo­gram evokes a sense of depth from a wavy pat­tern etched onto a flat sur­face, pat­terns in the mi­cro­scopic com­po­nents of the uni­verse may give rise to an­other spa­tial di­men­sion. This new di­men­sion is curved, so that grav­ity arises or­gan­i­cally.

Entropic grav­ity, in­tro­duced in a fa­mous 1995 pa­per by the the­o­ret­i­cal physi­cist Ted Jacobson of the University of Maryland, takes a re­lated but dis­tinct tack. Previously, physi­cists had started with Einstein’s the­ory and de­rived its heat­like con­se­quences. But Jacobson went the other way. He started from the as­sump­tion that space-time has ther­mal prop­er­ties and used these to de­rive the equa­tions of gen­eral rel­a­tiv­ity. His work con­firmed that there’s some­thing sig­nif­i­cant about the par­al­lels be­tween grav­ity and heat.

“He turned black hole ther­mo­dy­nam­ics on its head,” Carney said. “I’ve been mys­ti­fied by this re­sult for my en­tire adult life.”

How might grav­i­ta­tional at­trac­tion arise out of more mi­cro­scopic com­po­nents? Inspired by Jacobson’s ap­proach, Carney and his co-au­thors — Manthos Karydas, Thilo Scharnhorst, Roshni Singh and Jacob Taylor — put for­ward two mod­els.

In the first, space is filled with a crys­talline grid of quan­tum par­ti­cles, or qubits. Each has an ori­en­ta­tion, like a com­pass nee­dle. These qubits will align them­selves with a nearby ob­ject that pos­sesses mass and ex­ert a force on that ob­ject. “If you put a mass some­where in the lat­tice, it causes all of the qubits nearby to get po­lar­ized — they all try to go in the same di­rec­tion,” Carney said.

By re­ori­ent­ing the nearby qubits, a mas­sive ob­ject cre­ates a pocket of high or­der in the grid of oth­er­wise ran­domly ori­ented qubits. If you place two masses into the lat­tice, you cre­ate two such pock­ets of or­der. High or­der means low en­tropy. But the sys­tem’s nat­ural ten­dency is to max­i­mize en­tropy. So, as the masses re­align the qubits and the qubits in turn buf­fet the masses, the net ef­fect will be to squash the masses closer to­gether to con­tain the or­der­li­ness to a smaller re­gion. It will ap­pear that the two masses are at­tract­ing each other grav­i­ta­tion­ally when in fact the qubits are do­ing all the work. And just as Newton’s law dic­tates, the ap­par­ent at­trac­tion di­min­ishes with the square of the dis­tance be­tween the masses.

The sec­ond model does away with the grid. Massive ob­jects still re­side within space and are acted upon by qubits, but now those qubits do not oc­cupy any par­tic­u­lar lo­ca­tion and could in fact be far away. Carney said this fea­ture is in­tended to cap­ture the non­lo­cal­ity of Newtonian grav­ity: Every ob­ject in the uni­verse acts on every other ob­ject to some de­gree.

Each qubit in the model is able to store some en­ergy; the amount de­pends on the dis­tance be­tween the masses. When they are far apart, a qubit’s en­ergy ca­pac­ity is high, so the to­tal en­ergy of the sys­tem can fit in just a few qubits. But if the masses are closer to­gether, the en­ergy ca­pac­ity of each qubit drops, so the to­tal en­ergy has to be spread over more qubits. The lat­ter sit­u­a­tion cor­re­sponds to a higher en­tropy, so the nat­ural ten­dency of the sys­tem is to push the masses to­gether, again in keep­ing with Newtonian grav­ity.

Carney cau­tioned that both mod­els are ad hoc. There’s no in­de­pen­dent ev­i­dence for these qubits, and he and his col­leagues had to fine-tune the strength and di­rec­tion of the force ex­erted by them. One might ask whether this is any im­prove­ment over tak­ing grav­ity to be fun­da­men­tal. “It ac­tu­ally seems to re­quire a pe­cu­liar en­gi­neered-look­ing in­ter­ac­tion to get this to work,” Carney said.

And what works is just Newton’s law of grav­ity, not the full ap­pa­ra­tus of Einstein’s the­ory, where grav­ity is equiv­a­lent to the cur­va­ture of space-time. For Carney, the mod­els are just a proof of prin­ci­ple — a demon­stra­tion that it is at least pos­si­ble for swarm be­hav­ior to ex­plain grav­i­ta­tional at­trac­tion — rather than a re­al­is­tic model for how the uni­verse works. “The on­tol­ogy of all of this is neb­u­lous,” he said.

Mark Van Raamsdonk, a physi­cist at the University of British Columbia, is doubt­ful that the mod­els re­ally rep­re­sent a proof of prin­ci­ple. A prac­ti­tioner of holog­ra­phy, the lead­ing ap­proach to emer­gent space-time, Van Raamsdonk notes that the new en­tropic mod­els don’t have any of the qual­i­ties that make grav­ity spe­cial, such as the fact that you feel no grav­i­ta­tional force when you’re freely falling through space-time. “Their con­struc­tion does­n’t re­ally have any­thing to do with grav­ity,” he said.

Furthermore, the mod­els dwell on the one as­pect of grav­ity that physi­cists think they al­ready un­der­stand. Newton’s law arises nat­u­rally out of Einstein’s the­ory when grav­ity is com­par­a­tively fee­ble, as it is on Earth. It’s where grav­ity gets strong, as in black holes, that it gets weird, and the en­tropic model has noth­ing to say about that. “The real chal­lenge in grav­i­ta­tional physics is un­der­stand­ing its strong-cou­pling, strong-field regime,” said Ramy Brustein, a the­o­rist at Ben-Gurion University who said he used to be sym­pa­thetic to en­tropic grav­ity but has cooled on the idea.

Proponents of en­tropic grav­ity re­spond that physi­cists should­n’t be so sure about how grav­ity be­haves when it is weak. If grav­ity is in­deed a col­lec­tive ef­fect of qubits, the Newtonian force law rep­re­sents a sta­tis­ti­cal av­er­age, and the mo­ment-by-mo­ment ef­fect will bounce around that av­er­age. “You have to go to very weak fields, be­cause then these fluc­tu­a­tions might be­come ob­serv­able,” said Erik Verlinde of the University of Amsterdam, who ar­gued for en­tropic grav­ity in a much-dis­cussed 2010 pa­per and has con­tin­ued to de­velop the idea.

Carney thinks the main ben­e­fit of the new mod­els is that they prompt con­cep­tual ques­tions about grav­ity and open up new ex­per­i­men­tal di­rec­tions.

Suppose a mas­sive body is in a quan­tum com­bi­na­tion, or “superposition,” of be­ing in two dif­fer­ent lo­ca­tions. Will its grav­i­ta­tional field like­wise be in a su­per­po­si­tion, pulling on falling bod­ies in two dif­fer­ent di­rec­tions? The new en­tropic-grav­ity mod­els pre­dict that the qubits will act on the mas­sive body to snap it out of its Schrödinger’s cat–like predica­ment.

This sce­nario con­nects to the much-fret­ted-over ques­tion of wave func­tion col­lapse — which asks how it is that mea­sur­ing a quan­tum sys­tem in su­per­po­si­tion causes its mul­ti­ple pos­si­ble states to be­come a sin­gle def­i­nite state. Some physi­cists have sug­gested that this col­lapse is caused by some in­trin­sic ran­dom­ness in the uni­verse. These pro­pos­als dif­fer in de­tail from Carney’s but have sim­i­lar testable con­se­quences. They pre­dict that an iso­lated quan­tum sys­tem will even­tu­ally col­lapse of its own ac­cord, even if it’s never mea­sured or oth­er­wise af­fected from with­out. “The same ex­per­i­men­tal se­tups could, in prin­ci­ple, be used to test both,” said Angelo Bassi of the University of Trieste, who has led the ef­fort to per­form such ex­per­i­ments, al­ready rul­ing out some col­lapse mod­els.

For all his doubts, Van Raamsdonk agrees that the en­tropic-grav­ity ap­proach is worth a try. “Since it has­n’t been es­tab­lished that ac­tual grav­ity in our uni­verse arises holo­graph­i­cally, it’s cer­tainly valu­able to ex­plore other mech­a­nisms by which grav­ity might arise,” he said. And if this long-shot the­ory does work out, physi­cists will need to up­date the artist Gerry Mooney’s fa­mous grav­ity poster, which reads: “Gravity. It is­n’t just a good idea. It’s the law.” Perhaps grav­ity is not, in fact, a law, just a sta­tis­ti­cal ten­dency.

For more than a cen­tury the tele­phone ex­change has formed the back­bone of our telecom­mu­ni­ca­tions sys­tem. A vast ar­ray of more than 5,500 mostly non­de­script build­ings sit un­no­ticed on city, town or vil­lage streets, and qui­etly link up more than 254 mil­lion kilo­me­tres of ca­bles and wires — keep­ing peo­ple in the UK con­nected to each other and the rest of the world.

Since the first tele­phone ex­change was es­tab­lished in London in 1879 with just eight sub­scribers, these anony­mous look­ing build­ings have spread the length and breadth of the UK — from the small­est on the re­mote Shetland Isle of Papa Stour, with just 14 homes, to the largest in Oldham, Manchester, serv­ing more than 45,000.

But the re­cent ex­plo­sive growth in new dig­i­tal fi­bre based ser­vices means the ma­jor­ity of these iconic com­mu­ni­ca­tion hubs will soon route their last ever call.

The ad­vent of tiny but pow­er­ful mi­cro­proces­sors and glass fi­bres, thin­ner than the width of a hu­man hair, only need a tiny frac­tion of the space taken up by miles of cop­per wires and bulky racks of switch­ing ma­chin­ery to run the old cop­per based phone net­work or Public Switched Telephone Network (PSTN).

This seis­mic shift means that to­day we’re able to pro­vide fi­bre broad­band ser­vices to the en­tire coun­try from just 1,000 ‘super dig­i­tal ex­changes’ or Openreach Handover Points (OHPs).

Sadly, this spells the be­gin­ning of the end for the re­main­ing 4,600 ex­changes used to sup­port tra­di­tional cop­per based phone and broad­band voice ser­vices. And these cop­per cus­tomers are dwin­dling fast as peo­ple mi­grate to faster more ef­fi­cient fi­bre

Openreach is now con­sult­ing with its com­mu­ni­ca­tion provider (CP) cus­tomers — like Sky, Vodafone, TalkTalk and BT, who use our net­work to con­nect their own cus­tomers — about how to close these ‘legacy’ ex­changes over the next decade or so.

This will be a ma­jor un­der­tak­ing with sev­eral mil­lion ser­vices to be mi­grated, and the im­por­tance of en­sur­ing vul­ner­a­ble cus­tomers and the UK’s Critical National Infrastructure providers are pro­tected along the way.  So we’re plan­ning it in stages — with the first 103 ex­changes to close by December 2030. These have some of the high­est run­ning costs so there’s a clear ad­van­tage in tar­get­ing them first.  Most of the re­main­ing 4,500 ex­changes will likely fol­low in the early 2030s.

The seek­able for­mat splits com­pressed data into a se­ries of in­de­pen­dent “frames”, each com­pressed in­di­vid­u­ally, so that de­com­pres­sion of a sec­tion in the mid­dle of an archive only re­quires zstd to de­com­press at most a frame’s worth of ex­tra data, in­stead of the en­tire archive.

Zeekstd makes ad­di­tions to the seek­able for­mat by im­ple­ment­ing an up­dated ver­sion of the

spec­i­fi­ca­tion, how­ever, it is fully com­pat­i­ble with the

ini­tial ver­sion of the seek­able for­mat.

A seek­able Encoder will start new frames au­to­mat­i­cally at 2MiB of un­com­pressed data. See

EncodeOptions to change this and other com­pres­sion pa­ra­me­ters.

use std::{fs::File, io};

use zeek­std::En­coder;

fn main() -> zeek­std::Re­sult

By de­fault, the seek­able Decoder de­com­presses every­thing, from the first to the last frame, but can also be con­fig­ured to de­com­press only spe­cific frames.

use std::{fs::File, io};

use zeek­std::De­coder;

fn main() -> zeek­std::Re­sult

This repo also con­tains a CLI tool that uses the li­brary.

* The zstd C li­brary is un­der a dual BSD/GPLv2 li­cense.