Capture a web page as it ap­pears now for use as a trusted ci­ta­tion in the fu­ture.

Computer chips that cram bil­lions of elec­tronic de­vices into a few square inches have pow­ered the dig­i­tal econ­omy and trans­formed the world. Scientists may be on the cusp of launch­ing a sim­i­lar tech­no­log­i­cal rev­o­lu­tion — this time us­ing light.

In a sig­nif­i­cant ad­vance to­ward that goal, National Institute of Standards and Technology (NIST) sci­en­tists and col­lab­o­ra­tors have pi­o­neered a way to make in­te­grated cir­cuits for light by de­posit­ing com­plex pat­terns of spe­cial­ized ma­te­ri­als onto sil­i­con wafers. These so-called pho­ton­ics chips use op­ti­cal de­vices such as lasers, wave­guides, fil­ters and switches to shut­tle light around and process in­for­ma­tion. The new ad­vance could pro­vide a big boost for emerg­ing tech­nolo­gies such as ar­ti­fi­cial in­tel­li­gence, quan­tum com­put­ers and op­ti­cal atomic clocks.

Making cir­cuitry for light as pow­er­ful and ubiq­ui­tous as cir­cuitry for elec­trons is one of to­day’s tech­no­log­i­cal fron­tiers, says Scott Papp, a NIST physi­cist whose group led the re­search, pub­lished this week in Nature. “We’re learn­ing to make com­plex cir­cuits with many func­tions, cut­ting across many ap­pli­ca­tion ar­eas.”

When it comes to in­for­ma­tion trans­fer and pro­cess­ing, light can do things that elec­tric­ity can’t. Photons — par­ti­cles of light — are far zip­pier than elec­trons at work­ing their way through cir­cuits.

Laser light is also es­sen­tial for con­trol­ling pow­er­ful, emerg­ing quan­tum tech­nolo­gies such as op­ti­cal atomic clocks and quan­tum com­put­ers.

But sev­eral hur­dles re­main be­fore in­te­grated pho­ton­ics can truly hit its stride. One in­volves lasers. High-quality, com­pact and ef­fi­cient lasers ex­ist in only a few wave­lengths, or col­ors, of light. For ex­am­ple, semi­con­duc­tor lasers are very good at gen­er­at­ing in­frared light with a wave­length of 980 nanome­ters, or bil­lionths of a me­ter — a color just out­side the range of hu­man vi­sion.

Emerging tech­nolo­gies such as op­ti­cal atomic clocks and quan­tum com­put­ers need laser light in many other col­ors as well. The lasers that pro­duce those col­ors are big, costly and power-hun­gry, ef­fec­tively con­fin­ing these quan­tum tech­nolo­gies to a hand­ful of spe­cial-pur­pose labs.

By in­te­grat­ing lasers into cir­cuits on chips, sci­en­tists hope to help quan­tum tech­nolo­gies be­come cheaper and more portable, so they can start to ful­fill their vast promise.

The new NIST pho­ton­ics chip is a bit like a layer cake. NIST physi­cists Papp and Grant Brodnik, along with col­leagues, started with a stan­dard wafer of sil­i­con coated with sil­i­con diox­ide (glass) and lithium nio­bate, a so-called non­lin­ear ma­te­r­ial that can change the color of light com­ing into it.

The re­searchers then added pieces of metal to elec­tri­cally con­trol how the cir­cuits con­vert one color of light to oth­ers. The sci­en­tists also cre­ated other metal-lithium nio­bate in­ter­faces that al­lowed them to rapidly turn light on and off within the cir­cuits — a cru­cial abil­ity for data pro­cess­ing and high-speed rout­ing.

The ic­ing on the cake, so to speak, was a sec­ond non­lin­ear ma­te­r­ial called tan­ta­lum pen­tox­ide, or tan­tala. Tantala can trans­form light in ways that feel like magic, tak­ing in a sin­gle laser color and putting out the full rain­bow of vis­i­ble light col­ors plus a wide range of in­frared wave­lengths. Papp and col­leagues have spent years de­vel­op­ing tech­niques to fab­ri­cate cir­cuits out of tan­tala with­out heat­ing it up, al­low­ing the ma­te­r­ial to be de­posited onto other ma­te­ri­als with­out dam­ag­ing them.

By pat­tern­ing the dif­fer­ent ma­te­ri­als on top of each other in a three-di­men­sional stack, the re­searchers pro­duced a sin­gle chip that ef­fi­ciently routes light be­tween lay­ers. That al­lowed them to merge the light-ma­nip­u­lat­ing wiz­ardry of tan­tala with the con­trol­la­bil­ity of lithium nio­bate. The new tech­nique “allows seam­less in­te­gra­tion,” says Brodnik. “The real power is that tan­tala can be added to ex­ist­ing cir­cuitry.”

Ultimately, the re­searchers were able to fit roughly 50 fin­ger­nail-sized chips con­tain­ing 10,000 pho­tonic cir­cuits, each out­putting a unique color, onto a wafer roughly the size of a beer coaster. “We can cre­ate all these dif­fer­ent col­ors, just by de­sign­ing cir­cuits,” says Papp.

Quantum tech­nolo­gies such as clocks and com­put­ers could be among the biggest ben­e­fi­cia­ries of in­te­grated pho­ton­ics. These de­vices of­ten use ar­rays of atoms to store and process in­for­ma­tion. For each type of atom, physi­cists need lasers tai­lored to the atom’s in­ter­nal quan­tum en­ergy lev­els. For ex­am­ple, ru­bid­ium atoms, com­monly used in quan­tum com­put­ers and clocks, re­spond to red light with a wave­length of 780 nanome­ters. Strontium atoms, an­other pop­u­lar choice, “see” blue light at 461 nanome­ters. Shine other col­ors on the atoms and noth­ing hap­pens.

The bulky, costly and com­pli­cated lasers needed to pro­duce these be­spoke col­ors have been a ma­jor hin­drance to get­ting quan­tum com­put­ers and op­ti­cal clocks out of the lab and into the field, where they could have big im­pacts. Cheap, low-power, portable op­ti­cal clocks, for ex­am­ple, could help pre­dict vol­canic erup­tions and earth­quakes, of­fer an al­ter­na­tive to GPS for po­si­tion­ing and nav­i­ga­tion, and help sci­en­tists in­ves­ti­gate sci­en­tific mys­ter­ies such as the na­ture of dark mat­ter. Quantum com­put­ers could of­fer new ways to study the physics and chem­istry of drugs and ma­te­ri­als.

Integrated pho­tonic cir­cuits aren’t just for quan­tum. Papp be­lieves NIST’s pho­ton­ics chips could help ef­fi­ciently shut­tle sig­nals be­tween the spe­cial­ized chips used by tech firms, po­ten­tially mak­ing AI-based tools more pow­er­ful and ef­fi­cient. Tech com­pa­nies are also in­ter­ested in us­ing pho­ton­ics to im­prove vir­tual re­al­ity dis­plays.

While NIST’s chips aren’t yet ready for mass pro­duc­tion, the tech­nique used to cre­ate them pro­vides a path for­ward, Papp and Brodnik say. The NIST sci­en­tists col­lab­o­rated with ex­perts at Octave Photonics, a Louisville, Colorado-based startup com­pany founded by for­mer NIST re­searchers that’s now work­ing to scale up the tech­nol­ogy.

“When you see the chip glow­ing in the lab, tak­ing in in­vis­i­ble light and mak­ing all this vis­i­ble light in one in­te­grated chip — it’s ob­vi­ous how many po­ten­tial ap­pli­ca­tions there could be,” says Papp.

UPDATE–Vercel, a widely used cloud plat­form for de­vel­op­ing and de­ploy­ing apps, has dis­closed a breach of its in­ter­nal sys­tems, and says a “limited sub­set of cus­tomers” is af­fected.

The in­ci­dent came to light on Sunday and the com­pany says it has brought in an in­ci­dent re­sponse provider to in­ves­ti­gate the in­tru­sion. The com­pany rec­om­mends that cus­tomers check ac­tiv­ity logs for sus­pi­cious ac­tiv­ity and also ro­tate en­vi­ron­men­tal vari­ables as a pre­cau­tion. Vercek also sug­gests that cus­tomers use its sen­si­tive en­vi­ron­men­tal vari­ables fea­ture to mark things such as API keys as sen­si­tive, which then causes Vercel to store them in an un­read­able for­mat.

Vercel said the in­tru­sion was re­lated to the com­pro­mise of a third-party app.

“Our in­ves­ti­ga­tion has re­vealed that the in­ci­dent orig­i­nated from a third-party AI tool whose Google Workspace OAuth app was the sub­ject of a broader com­pro­mise, po­ten­tially af­fect­ing hun­dreds of its users across many or­ga­ni­za­tions,” the com­pany said.

Vercel did not iden­tify the app but in­cluded IOCs the iden­ti­fier for it. Given that the in­tru­sion orig­i­nated with a third-party app, there may well be other re­lated in­ci­dents emerg­ing in the com­ing hours or days.

“We’ve iden­ti­fied a se­cu­rity in­ci­dent that in­volved unau­tho­rized ac­cess to cer­tain in­ter­nal Vercel sys­tems. We are ac­tively in­ves­ti­gat­ing, and we have en­gaged in­ci­dent re­sponse ex­perts to help in­ves­ti­gate and re­me­di­ate. We have no­ti­fied law en­force­ment and will up­date this page as the in­ves­ti­ga­tion pro­gresses,” the com­pany said in a state­ment.

“At this time, we have iden­ti­fied a lim­ited sub­set of cus­tomers that were im­pacted and are en­gag­ing with them di­rectly.”

Vercel pro­vides a wide range of ser­vices for de­vel­op­ers and en­ter­prises, and has a num­ber of of­fer­ings that are fo­cused on agen­tic AI work­loads.

Vercel did not spec­ify which of its sys­tems were com­pro­mised or how many of its cus­tomers are af­fected.

This story was up­dated on April 19 to add in­for­ma­tion about the source of the in­tru­sion.

Pausing a game is so com­mon that I doubt many of us ever re­ally think about it. Maybe a pause menu has a cool song, or maybe you’re play­ing an al­ways-on­line game that fea­tures a pause menu that does­n’t ac­tu­ally pause any­thing. In those cases, you might mo­men­tar­ily con­tem­plate the act of paus­ing a video game. Those are the rare ex­cep­tions. Normally, we all just pause and un­pause with­out a sec­ond thought. It’s just ex­pected that most games will let you pause the ac­tion.

But how does that ac­tu­ally work? How do de­vel­op­ers ac­tu­ally let you pause a game?

I asked de­vel­op­ers on so­cial me­dia to tell me how they make a game pause, and the an­swers I got were all over the place. Many devs said that most mod­ern game en­gines sup­port paus­ing, and it should­n’t cause too many is­sues as long as you don’t screw any­thing up while mak­ing the game. But, as you might ex­pect, game de­vel­op­ment is weird and com­pli­cated and messy, and that means some­times paus­ing a game in­volves ma­nip­u­lat­ing time.

“In Waves of Steel, paus­ing slows the game speed down to 0.000000001 times nor­mal speed,” ex­plained game de­vel­oper ‪Chris Weisiger‬ on BlueSky. “In other words, it’d take about three years of real-time for one sec­ond of game time to pass. I did this be­cause I heard that Unity has spe­cial be­hav­ior for when game­speed is 0, which I wanted to avoid.”

“As a hob­by­ist in Unreal, I do some­thing a lit­tle stu­pid,” said dev Tommy Hanusa on so­cial me­dia. “I set the timescale to .000001 so that I can let the player/​tester eject from the pause and fly around (with an ap­pro­pri­ately ridicu­lous speed of like 5000000) in case they want to show me some­thing.”

Many other devs told me that they just set the game’s timescale to 0 when you hit pause and make sure that cer­tain func­tions, like the menu UI, ig­nore that com­mand and still work as ex­pected.

Another as­pect of paus­ing a game that I had­n’t con­sid­ered was that there are dif­fer­ent kinds of pauses. For ex­am­ple, hit­ting start might pause a game and bring up the pause menu. But what if you dis­con­nect a con­troller? What if you open the game’s in­ven­tory? What if you hit the guide but­ton on an Xbox and pop out to the guide? These are dif­fer­ent kinds of pauses, and some games have a whole bunch of them.

“I worked on var­i­ous games at Frontier, in­clud­ing Kinectimals on the Xbox 360,” ex­plained game dev Andrew Gillett via email. “I was­n’t di­rectly in­volved with this part of the game, but I re­call there were some­thing like seven dif­fer­ent lev­els of ‘pause.’ For ex­am­ple, the game should pause if the Kinect cam­era is dis­con­nected, and this is a dif­fer­ent kind of pause than when the user has brought up the Xbox sys­tem menu.”

Dreamless on BlueSky ex­plained that these dif­fer­ent kinds of pauses could some­times cause headaches for devs.

“I re­mem­ber in the Xbox/PS2 era we’d do a pause for nor­mal game­play,” said Dreamless. “With ex­cep­tions like can’t pause dur­ing QTEs & etc. Then, when it was time to ship, we’d read the [Technical Requirements Checklists] and have to go back and add a spe­cial pause for when you un­plug the con­troller. The two pauses would con­flict and cause bugs.”

Perhaps my fa­vorite pause method in­volves devs freez­ing time and then tak­ing a screen­shot of the game which the game then uses as the back­ground be­hind the pause menu UI, let­ting them get up to all sorts of nasty busi­ness be­hind that im­age, like not ren­der­ing en­e­mies or even mov­ing the player to an empty room.

“Usually, I will…take a screen­shot of the game­play at the point the game is paused and then draw that un­der what­ever pause screen menu while also no longer draw­ing the ac­tual ob­jects,” said game dev DW O’Boyle. “This is mostly just to free up some mem­ory, but it is­n’t re­ally nec­es­sary for the style of games I make.”

“In most of the Vlambeer games and Minit / Disc Room,” said de­vel­oper Jan Willem Nijman, “I take a screen­shot (with the UI dis­abled), then ei­ther jump to a com­pletely dif­fer­ent empty room or de­ac­ti­vate every­thing…with that screen­shot as the back­ground, [and] on un­pause jump back [to the game]. Sometimes there’s a 1-frame de­lay be­cause that screen­shot needs the UI dis­abled.”

When some­one replied that this trick al­ways felt “hacky” to them, Nijman said that in every game they’ve worked on, you’ll find “a healthy dose of hack­y­ness.”

My big take­away from all of these re­sponses is that, gen­er­ally speak­ing, paus­ing a game is­n’t the most com­pli­cated fea­ture to get work­ing in a pro­ject. However, you still need to be mind­ful of how you im­ple­ment it, and do proper amounts of test­ing if your game has quirks that might cause is­sues when you start paus­ing game time.

Developer Caliban Darklock told me on BlueSky that a lot of game mak­ers screw up adding a pause func­tion early on in their de­vel­op­ment ca­reer, which can lead to prob­lems, but can also be a very im­por­tant learn­ing mo­ment.

“The first time I im­ple­mented ‘pause’ in a game, I had every sin­gle game ob­ject check­ing whether the game was paused in every sin­gle frame, which de­graded per­for­mance across the whole game,” said Darklock. “Now all my ob­jects are arranged in a hi­er­ar­chy, and only one ob­ject at the top checks if the game is paused.”

“Most de­vel­op­ers do a hor­ri­ble, sloppy night­mare job the first time they im­ple­ment this, and then they know bet­ter for the rest of their lives.”

We’ve iden­ti­fied a se­cu­rity in­ci­dent that in­volved unau­tho­rized ac­cess to cer­tain in­ter­nal Vercel sys­tems. We are ac­tively in­ves­ti­gat­ing, and we have en­gaged in­ci­dent re­sponse ex­perts to help in­ves­ti­gate and re­me­di­ate. We have no­ti­fied law en­force­ment and will up­date this page as the in­ves­ti­ga­tion pro­gresses.

At this time, we have iden­ti­fied a lim­ited sub­set of cus­tomers that were im­pacted and are en­gag­ing with them di­rectly.

Our ser­vices re­main op­er­a­tional, and we will con­tinue to up­date this page with new in­for­ma­tion.

We are tak­ing ac­tions to pro­tect Vercel sys­tems and cus­tomers.

Our in­ves­ti­ga­tion is on­go­ing. In the mean­time, here are best prac­tices you can fol­low for peace of mind:

* Review the ac­tiv­ity log for your ac­count and en­vi­ron­ments for sus­pi­cious ac­tiv­ity.

* Review and ro­tate en­vi­ron­ment vari­ables. Environment vari­ables marked as “sensitive” in Vercel are stored in a man­ner that pre­vents them from be­ing read, and we cur­rently do not have ev­i­dence that those val­ues were ac­cessed. However, if any of your en­vi­ron­ment vari­ables con­tain se­crets (API keys, to­kens, data­base cre­den­tials, sign­ing keys) that were not marked as sen­si­tive, those val­ues should be treated as po­ten­tially ex­posed and ro­tated as a pri­or­ity.

* Take ad­van­tage of the sen­si­tive en­vi­ron­ment vari­ables fea­ture go­ing for­ward, so that se­cret val­ues are pro­tected from be­ing read in the fu­ture.

For sup­port ro­tat­ing your se­crets or other tech­ni­cal sup­port, con­tact us through ver­cel.com/​help.

Our in­ves­ti­ga­tion has re­vealed that the in­ci­dent orig­i­nated from a third-party AI tool whose Google Workspace OAuth app was the sub­ject of a broader com­pro­mise, po­ten­tially af­fect­ing hun­dreds of its users across many or­ga­ni­za­tions.

We are pub­lish­ing the fol­low­ing IOC to sup­port the wider com­mu­nity in the in­ves­ti­ga­tion and vet­ting of po­ten­tial ma­li­cious ac­tiv­ity in their en­vi­ron­ments. We rec­om­mend that Google Workspace Administrators and Google Account own­ers check for us­age of this app im­me­di­ately.

What is Antithesis?

How Antithesis works

How we’re dif­fer­ent

Problems we solve

Security ap­proach

Distributed sys­tems re­li­a­bil­ity glos­sary

Cost of out­ages white pa­per

Deterministic sim­u­la­tion test­ing primer

Property-based test­ing primer

Autonomous test­ing primer

Techniques to im­prove soft­ware test­ing

Catalog of re­li­a­bil­ity prop­er­ties for key-value data­s­tores

Catalog of re­li­a­bil­ity prop­er­ties for blockchains

Test ACID-compliance with a Ring test

What are skiplists good for?

A while back, I joined Phil Eaton’s book club on The Art of Multiprocessor Programming, and the topic of skiplists came up.

For most of my ca­reer, skiplists had al­ways seemed like a niche data struc­ture, with a ra­bid cult fol­low­ing but not a whole ton of ap­plic­a­bil­ity to my life. Then six or so years ago, we en­coun­tered a prob­lem at Antithesis that seemed in­tractable un­til it turned out that a gen­er­al­iza­tion of skiplists was ex­actly what we needed.

Before I tell you about that, though, let me ex­plain what skiplists are (feel free to skip ahead if you al­ready know them well).

A skiplist is a ran­dom­ized data struc­ture that’s ba­si­cally a drop-in re­place­ment for a bi­nary search tree with the same in­ter­face and the same as­ymp­totic com­plex­ity on each of its op­er­a­tions. Some peo­ple like them be­cause you can pro­duce rel­a­tively sim­ple and un­der­stand­able lock-free con­cur­rent im­ple­men­ta­tions, and oth­ers like them as a mat­ter of taste, or be­cause they en­joy lis­ten­ing to bands that you’ve to­tally never heard of.

In im­ple­men­ta­tion terms, you can think of them roughly as linked lists plus “express lanes”:

You start with a ba­sic linked list, and then add a hi­er­ar­chy of linked lists with pro­gres­sively fewer nodes in them. In the ex­am­ple above, the nodes in the higher-level lists are cho­sen prob­a­bilis­ti­cally, with each node hav­ing a 50% chance of be­ing pro­moted to the next level.1

This helps with search, be­cause you can use the higher-level lists to skip more quickly to the node you want:

For (much) more on skiplists, see The Ubiquitous Skiplist.

Here we’ve found the node with an ID of 38 by start­ing at the top level and work­ing down­wards. At each level we ad­vance un­til the next node would have an ID that’s too high, then jump down a level.

In a reg­u­lar linked list of n nodes, find­ing a node would take O(n) time, be­cause you’re walk­ing through the nodes one by one. Skiplists let you jump lev­els, with each level halv­ing the num­ber of nodes you need to check, so you end up find­ing the node in O(log n) time.

This is all very nice, but af­ter read­ing about this data struc­ture I lit­er­ally never thought about it again, un­til one day we en­coun­tered the fol­low­ing prob­lem at Antithesis…

Antithesis runs cus­tomers’ soft­ware many times to look for bugs. Each time, our fuzzer in­jects dif­fer­ent faults and tells your test­ing code to make dif­fer­ent ran­dom de­ci­sions. Over many runs, these choices cre­ate a branch­ing tree of time­lines: each path from root to leaf rep­re­sents one se­quence of choices the fuzzer made and what hap­pened as a re­sult.

There were a lot of queries that we wanted to do which ba­si­cally amounted to fold op­er­a­tions up or down this tree. For ex­am­ple, given a par­tic­u­lar log mes­sage, what’s the unique his­tory of events that led to it? (Walk up the par­ent point­ers from that node to the root.)

The trou­ble was that the amount of data out­put by the soft­ware we were test­ing was so huge, we had to throw it all into an an­a­lytic data­base, and at the time we were us­ing Google BigQuery. Analytic data­bases are op­ti­mized for scan­ning mas­sive amounts of data in par­al­lel to com­pute ag­gre­gate re­sults. The trade­off is that they’re slow at point lookups, where you fetch a spe­cific row by its ID.

This mat­ters, be­cause the nat­ural way to rep­re­sent a tree in a data­base is with par­ent point­ers — each node is a row in the table, with a par­en­t_id col­umn point­ing to its par­ent. To an­swer a ques­tion like “show me the his­tory lead­ing to this log mes­sage”, you’d need to walk up the tree one node at a time: look up the node, get its par­ent ID, look up the par­ent node, and so on. Each step is a point lookup. In an OLTP data­base de­signed for point lookups, that’s fine.2 But in BigQuery, ba­si­cally every op­er­a­tion re­sults in a full table scan, which means even the most ba­sic queries would end up do­ing O(depth) reads over your en­tire data set. Yikes!

I mean, not ac­tu­ally, but it’s less bad..

One al­ter­na­tive would have been to split the data: store just the tree struc­ture (the par­ent point­ers) in a data­base that’s good at point lookups, and keep the bulk data in BigQuery. But this ap­proach would have cre­ated other prob­lems. Every in­sert would need to write to both sys­tems, and since we want to an­a­lyze the data on­line (while new writes are stream­ing in) keep­ing the two data­bases con­sis­tent would re­quire some­thing like two-phase com­mit (2PC). I pre­fer not to in­vent new 2PC prob­lems where I don’t need them. And any­way, at the time BigQuery had very loose con­sis­tency se­man­tics, so it’s not even clear that keep­ing the two sys­tems in sync would have been pos­si­ble.

Skiplists to the res­cue! Or rather, a weird thing we in­vented called a “skiptree”…

Well, it’s like a skiplist, but it’s a tree.

More help­fully, here’s an ex­am­ple:

You have a level-0 tree, and then a hi­er­ar­chy of trees above it. Each tree has roughly 50% of the nodes of the level be­low (the re­moved nodes are shown with grey dot­ted lines on the di­a­gram).

If you pick any path from the root to a leaf, the nodes along that path — to­gether with their ap­pear­ances in the higher-level trees — form a skiplist. So a skip­tree is re­ally just a bunch of skiplists shar­ing struc­ture, one for every root-to-leaf path in the tree.

To store the skip­tree, you cre­ate a SQL table for each level: tree0, tree1, and so on. Each table has a row for every node in that tree. Instead of hav­ing a sin­gle par­en­t_id col­umn, it has a col­umn for the clos­est an­ces­tor node in the tree above (we’ll call that nex­t_lev­el_an­ces­tor) and an­other col­umn (call it an­ces­tors_­be­tween) with a list of all nodes be­tween the cur­rent node and the next-level an­ces­tor.

For the di­a­gram above, tree0 would look like this:

As an ex­am­ple, take the row for node H. Node H’s par­ent is D, which is not in tree1. D’s par­ent B is also not in tree1, but B’s par­ent A is, so nex­t_lev­el_an­ces­tor is A. Then an­ces­tors_­be­tween stores B and D.

The higher-level ta­bles work the same way:

You can use these ta­bles to find the an­ces­tors of a node by chain­ing to­gether JOINs, work­ing your way up the ta­bles.

For ex­am­ple, to find all an­ces­tors of node I, start at table0. The nex­t_lev­el_an­ces­tor col­umn tells you to JOIN on node C in table1, col­lect­ing node G from the an­ces­tors_­be­tween col­umn on the way. Then in table1 you find that the nex­t_lev­el_an­ces­tor is node A, with no other nodes to col­lect on the way. Node A is the root of the tree so you’re now done: the to­tal list of an­ces­tors is [G, C, A]. In a deeper tree you’d keep go­ing by look­ing in tree2, tree3 and so on.

Hey! Now we can find an­ces­tors with a sin­gle non-re­cur­sive SQL query with a fixed num­ber of JOINs. We just had to do… 40 or so JOINs.3

Best of all, at the time BigQuery’s pric­ing charged you for the amount of data scanned, rather than for com­pute, and the geo­met­ric dis­tri­b­u­tion of table sizes meant that each of these queries only cost twice a nor­mal table scan.4

The num­ber of skip lev­els was pre­cisely cho­sen to gen­er­ate a num­ber of joins just un­der the BigQuery plan­ner’s hard-coded limit.

Of course, there were dis­ad­van­tages, like the SQL it­self. The tex­tual size of these queries was of­ten mea­sured in the kilo­bytes. But what do I look like, a cave­man? We did­n’t write the SQL. We wrote a com­piler in JavaScript that gen­er­ated it. And that is how most test prop­er­ties in Antithesis were eval­u­ated for the first six years of the com­pany, un­til we fi­nally wrote our own an­a­lytic data­base that could do ef­fi­cient tree-shaped queries.5

I’m sure it cost Google a whole lot more.

Later I dis­cov­ered that a skip­tree is closely re­lated to a real data struc­ture called a skip graph, a dis­trib­uted data struc­ture based on skiplists. Which just goes to show that there is noth­ing new un­der the sun. Whatever crazy idea you have, there’s a good chance some other crazy per­son has al­ready done it. Moral of the story: you never know when an ex­otic data struc­ture will save you a lot of time and money.

Migrating from BigQuery to Pangolin (our in-house tree data­base) was what en­abled us to launch our new pre-ob­serv­abil­ity fea­ture last year.

Also, while Andy Pavlo is cor­rect that a well-writ­ten tree will al­ways trounce a skiplist, the great thing about skiplists is that a to­tally naive im­ple­men­ta­tion has ad­e­quate per­for­mance. That comes in handy when you’re writ­ing them in, say, SQL.

Thank you to Phil Eaton for sug­gest­ing that we write this up.

You made it to the end! Grab some stick­ers

Place them any­where and watch the com­pli­ments com­pile.

Get free stick­ers

You made it to the end! Grab some stick­ers

Place them any­where and watch the com­pli­ments com­pile.

Get free stick­ers

You made it to the end!

Get free stick­ers

You made it to the end! Grab some stick­ers

Place them any­where and watch the com­pli­ments com­pile.

Get free stick­ers

On April 17, en­gi­neers at NASA’s Jet Propulsion Laboratory (JPL) in Southern California sent com­mands to shut down an in­stru­ment aboard Voyager 1 called the Low-energy Charged Particles ex­per­i­ment, or LECP. The nu­clear-pow­ered space­craft is run­ning low on power, and turn­ing off the LECP is con­sid­ered the best way to keep hu­man­i­ty’s first in­ter­stel­lar ex­plorer go­ing.

Mission en­gi­neers at NASA’s Jet Propulsion Laboratory in Southern California turned off the Low-energy Charged Particles ex­per­i­ment aboard Voyager 1 on April 17, 2026.

The LECP has been op­er­at­ing al­most with­out in­ter­rup­tion since Voyager 1 launched in 1977 — al­most 49 years. It mea­sures low-en­ergy charged par­ti­cles, in­clud­ing ions, elec­trons, and cos­mic rays orig­i­nat­ing from our so­lar sys­tem and galaxy. The in­stru­ment has pro­vided crit­i­cal data about the struc­ture of the in­ter­stel­lar medium, de­tect­ing pres­sure fronts and re­gions of vary­ing par­ti­cle den­sity in the space be­yond our he­lios­phere. The twin Voyagers are the only space­craft that are far enough from Earth to pro­vide this in­for­ma­tion.

Like Voyager 2, Voyager 1 re­lies on a ra­dioiso­tope ther­mo­elec­tric gen­er­a­tor, a de­vice that con­verts heat from de­cay­ing plu­to­nium into elec­tric­ity. Both probes lose about 4 watts of power each year. After al­most a half-cen­tury in space, power mar­gins have grown ra­zor thin, re­quir­ing the team to con­serve en­ergy by shut­ting off heaters and in­stru­ments while mak­ing sure the space­craft don’t get so cold that their fuel lines freeze.

During a rou­tine, planned roll ma­neu­ver on Feb. 27, Voyager 1’s power lev­els fell un­ex­pect­edly. Mission en­gi­neers knew any ad­di­tional drop in power could trig­ger the space­craft’s un­der­volt­age fault pro­tec­tion sys­tem, which would shut down com­po­nents on its own to safe­guard the probe, re­quir­ing re­cov­ery by the flight team — a lengthy process that car­ries its own risks.

The Voyager team needed to act first.

“While shut­ting down a sci­ence in­stru­ment is not any­body’s pref­er­ence, it is the best op­tion avail­able,” said Kareem Badaruddin, Voyager mis­sion man­ager at JPL. “Voyager 1 still has two re­main­ing op­er­at­ing sci­ence in­stru­ments — one that lis­tens to plasma waves and one that mea­sures mag­netic fields. They are still work­ing great, send­ing back data from a re­gion of space no other hu­man-made craft has ever ex­plored. The team re­mains fo­cused on keep­ing both Voyagers go­ing for as long as pos­si­ble.”

The choice of which in­stru­ment to turn off next was­n’t made in the heat of the mo­ment. Years ago, the Voyager sci­ence and en­gi­neer­ing teams sat down to­gether and agreed on the or­der in which they would shut off parts of the space­craft while en­sur­ing the mis­sion can con­tinue to con­duct its unique sci­ence. Of the 10 iden­ti­cal sets of in­stru­ments that each space­craft car­ries, seven have been shut off so far. For Voyager 1, the LECP was next on that list. The team shut off the LECP on Voyager 2 in March 2025.

Because Voyager 1 is more than 15 bil­lion miles (25 bil­lion kilo­me­ters) from Earth, the se­quence of com­mands to shut down the in­stru­ment will take 23 or so hours to reach the space­craft, and the shut­down process it­self will take about three hours and 15 min­utes to com­plete. One part of the LECP — a small mo­tor that spins the sen­sor in a cir­cle to scan in all di­rec­tions — will re­main on. It uses lit­tle power (0.5 watts), and keep­ing it run­ning gives the team the best chance of be­ing able to turn the in­stru­ment back on some­day if they find ex­tra power.

Engineers are con­fi­dent that shut­ting down the LECP will give Voyager 1 about a year of breath­ing room. They are us­ing the time to fi­nal­ize a more am­bi­tious en­ergy-sav­ing fix for both Voyagers they call “the Big Bang,” which is de­signed to fur­ther ex­tend Voyager op­er­a­tions. The idea is to swap out a group of pow­ered de­vices all at once — hence the nick­name — turn­ing some things off and re­plac­ing them with lower-power al­ter­na­tives to keep the space­craft warm enough to con­tinue gath­er­ing sci­ence data.

The team will im­ple­ment the Big Bang on Voyager 2 first, which has a lit­tle more power to spare and is closer to Earth, mak­ing it the safer test sub­ject. Tests are planned for May and June 2026. If they go well, the team will at­tempt the same fix on Voyager 1 no sooner than July. If it works, there is even a chance that Voyager 1’s LECP could be switched back on.

Police lured the man to a meet­ing and ar­rested him af­ter ac­cess­ing a pri­vate WhatsApp group with col­leagues

Police lured the man to a meet­ing and ar­rested him af­ter ac­cess­ing a pri­vate WhatsApp group with col­leagues

Police ac­cessed the closed WhatsApp group chat, saved the ev­i­dence and told the man to come to a meet­ing be­fore ar­rest­ing him. The of­fend­ing im­age showed smoke ris­ing above a build­ing af­ter the March 2026 strikes and had only been shared in the pri­vate group chat. He re­mains in de­ten­tion on charges in­clud­ing pub­lish­ing in­for­ma­tion deemed harm­ful to state in­ter­ests, the max­i­mum sen­tence of which is two years. Read more: Dubai ’arrests sur­vivors of Iranian drone strike af­ter they sent im­ages of ex­plo­sion af­ter­math to loved ones’Read more: British hol­i­day­maker, 60, ar­rested in Dubai for ‘filming mis­siles’

Radha Stirling, chief ex­ec­u­tive of London-based ad­vo­cacy group Detained in Dubai, said Dubai po­lice had “explicitly con­firmed they are con­duct­ing elec­tronic sur­veil­lance op­er­a­tions ca­pa­ble of de­tect­ing pri­vate WhatsApp mes­sages.“She said peo­ple were be­ing tracked, iden­ti­fied, and ar­rested not for pub­lic state­ments, but for pri­vate ex­changes be­tween col­leagues.“’Com­pa­nies like WhatsApp must an­swer ur­gent ques­tions about user pri­vacy.” she added.

Ms Stirling con­tin­ued: “If pri­vate com­mu­ni­ca­tions can be de­tected and used as the ba­sis for ar­rest by over­reach­ing or hy­per­sen­si­tive states, users world­wide need clar­ity on how their data is be­ing ac­cessed.” The po­lice re­port said au­thor­i­ties learned of the ma­te­ri­al’s ex­is­tence “’through elec­tronic mon­i­tor­ing op­er­a­tions”.A spe­cial team from the Electronic and Cybercrime Department was told to find the ac­count holder who shared the video. The air­line worker was tracked down, lured to a meet­ing and ar­rested by po­lice.The case was then es­ca­lated to State Security Prosecution. He re­mains in de­ten­tion.

The UAE gov­ern­ment owns ma­jor­ity hold­ings in tele­com com­pa­nies Etisalat and Du. This gives se­cu­rity ser­vices the power to ob­serve all com­mu­ni­ca­tions on their net­works. The Arab state has also used the Israeli-developed soft­ware Pegasus which al­lows agents to lis­ten into pri­vate calls and read mes­sages, even if they are shared on en­crypted apps like WhatsApp,.The spy­ware can in­fect a de­vice even with­out the user ac­ti­vat­ing a link - such as via a WhatsApp call, even if it is­n’t an­swered.Once in­side, it can ac­cess all WhatsApp mes­sages, lo­gos and con­tacts.Ms Stirling said other tourists, air­line crew and res­i­dents have re­ported be­ing de­tained for send­ing, re­ceiv­ing or keep­ing con­tent even when they did not share it.

This doc­u­ment is pub­lished un­der the Creative Commons

CC BY-NC-SA 4.0 li­cense. This li­cense en­ables reusers to dis­trib­ute, remix, adapt, and build upon the ma­te­r­ial in any medium or for­mat for non­com­mer­cial pur­poses only, and only so long as at­tri­bu­tion is given to the cre­ator. If you remix, adapt, or build upon the ma­te­r­ial, you must li­cense the mod­i­fied ma­te­r­ial un­der iden­ti­cal terms.

This doc­u­ment con­tains no LLM-generated text — zero, zip, nada. (Yes, I do use em-dashes and semi­colons; yes, I have writ­ten every one my­self.)

For er­rors no­ticed or other sug­ges­tions, please file is­sues on

Common-Lisp. Net’s GitLab in­stance or

GitHub.

In a dra­matic mid­dle-of-the-night stand off, a bi­par­ti­san set of law­mak­ers push­ing for true re­form and pri­vacy pro­tec­tions for Americans bought us some more time to fight! They are hold­ing out for, at a min­i­mum, the re­quire­ment of an ac­tual prob­a­ble cause war­rant for FBI ac­cess to in­for­ma­tion col­lected un­der the mass spy­ing pro­gram known as 702.

A reau­tho­riza­tion with vir­tu­ally no changes was de­feated be­cause a core group of law­mak­ers held strong; they know that peo­ple are hun­gry for real re­form that pro­tects the pri­vacy of our com­mu­ni­ca­tions. We now have a 10-day ex­ten­sion to con­tinue to push Congress to pass a real re­form bill.

The Lawmakers ral­lied late Thursday night to re­ject a pro­posed amend­ment that made ges­tures at pri­vacy pro­tec­tions, but it would not have im­proved on the sta­tus quo and would have reau­tho­rized Section 702 for five more years to boot.

Section 702 is rife with prob­lems, loop­holes, and com­pli­ance is­sues that need fix­ing. The National Security Agency col­lects full con­ver­sa­tions be­ing con­ducted by and with tar­gets over­seas — in­clud­ing by and with Americans in the U. S. –  and stores them in mas­sive data­bases. The NSA then al­lows other agen­cies, in­clud­ing the Federal Bureau of Investigation, to ac­cess un­told amounts of that in­for­ma­tion. In turn, the FBI takes a “finders keep­ers” ap­proach to this data: they rea­son that since it’s al­ready col­lected un­der one law, it’s OK for them to see it.

Under cur­rent prac­tice, the FBI can query and even read the U. S. side of that com­mu­ni­ca­tion with­out a war­rant. What’s more, vic­tims of this sur­veil­lance  won’t even know and have very few ways of find­ing out that their com­mu­ni­ca­tions have been sur­veilled. EFF and other civil lib­er­ties ad­vo­cates have been try­ing for years to know when data col­lected through Section 702 is used as ev­i­dence against them.

Reforming Section 702 is even more ur­gent be­cause of rev­e­la­tions hinted at by Senator Ron Wyden’s pub­lic state­ments con­cern­ing a “secret in­ter­pre­ta­tion” of the law that en­ables sur­veil­lance of Americans, and a pub­lic  “Dear Colleague” let­ter he sent to fel­low Senators about FBI abuse of Section 702.

That’s right—the way the gov­ern­ment con­ducts mass sur­veil­lance is so se­cret and un­ac­count­able even the way they in­ter­pret the law is clas­si­fied.

“In many cases these will be law-abid­ing Americans hav­ing per­fectly le­git­i­mate, of­ten sen­si­tive, con­ver­sa­tions,” Wyden wrote. “These Americans could in­clude jour­nal­ists, for­eign aid work­ers, peo­ple with fam­ily mem­bers over­seas - even women try­ing to get abor­tion med­ica­tion from an over­seas provider. Congress has an oblig­a­tion to pro­tect our coun­try from for­eign threats and pro­tect the rights of these and other Americans.”

We have 10 days to make it clear to Congress: 702 needs real re­forms. Not a blan­ket  reau­tho­riza­tion. Not lip ser­vice to change. Real re­form.