Skip to main con­tent

Ask the pub­lish­ers to re­store ac­cess to 500,000+ books.

8 Days Left: The year is al­most over—help us fin­ish strong in 2025!

Please Don’t Scroll Past This

Can you chip in? As an in­de­pen­dent non­profit, the Internet Archive is fight­ing for uni­ver­sal ac­cess to qual­ity in­for­ma­tion. We build and main­tain all our own sys­tems, but we don’t charge for ac­cess, sell user in­for­ma­tion, or run ads. We’d be deeply grate­ful if you’d join the one in a thou­sand users that sup­port us fi­nan­cially.

We un­der­stand that not every­one can do­nate right now, but if you can af­ford to con­tribute this Thursday, we promise it will be put to good use. Our re­sources are cru­cial for knowl­edge lovers every­where—so if you find all these bits and bytes use­ful, please pitch in.

Please Don’t Scroll Past This The Internet Archive is work­ing to keep the record straight by record­ing gov­ern­ment web­sites, news pub­li­ca­tions, his­tor­i­cal doc­u­ments, and more. If you find our li­brary use­ful, please pitch in.

Remind Me

By sub­mit­ting, you agree to re­ceive donor-re­lated emails from the Internet Archive. Your pri­vacy is im­por­tant to us. We do not sell or trade your in­for­ma­tion with any­one.

An icon used to rep­re­sent a menu that can be

tog­gled by in­ter­act­ing with this icon.

An il­lus­tra­tion of an open book.

An il­lus­tra­tion of two cells of a film

strip.

An il­lus­tra­tion of an au­dio speaker.

An il­lus­tra­tion of two pho­tographs.

An il­lus­tra­tion of a per­son’s head and chest.

An il­lus­tra­tion of a hor­i­zon­tal line over an up

point­ing ar­row.

Search the his­tory of more than 1 tril­lion web pages.

Capture a web page as it ap­pears now for use as a trusted ci­ta­tion in the fu­ture.

Internet Archive’s in-browser video “theater” re­quires JavaScript to be en­abled.

It ap­pears your browser does not have it turned on.

Please see your browser set­tings for this fea­ture.

Sharyn Alfonsi’s “Inside CECOT” for 60 Minutes, which was cen­sored by Bari Weiss, as it ap­peared on Canada’s Global TV app.

To see all avail­able qual­i­fiers, see our doc­u­men­ta­tion.

We read every piece of feed­back, and take your in­put very se­ri­ously.

Secure your code as you build

To see all avail­able qual­i­fiers, see our doc­u­men­ta­tion.

We read every piece of feed­back, and take your in­put very se­ri­ously.

Secure your code as you build

You signed in with an­other tab or win­dow. Reload to re­fresh your ses­sion.

You signed out in an­other tab or win­dow. Reload to re­fresh your ses­sion.

You switched ac­counts on an­other tab or win­dow. Reload to re­fresh your ses­sion.

uv in­stalls pack­ages faster than pip by an or­der of mag­ni­tude. The usual ex­pla­na­tion is “it’s writ­ten in Rust.” That’s true, but it does­n’t ex­plain much. Plenty of tools are writ­ten in Rust with­out be­ing no­tably fast. The in­ter­est­ing ques­tion is what de­sign de­ci­sions made the dif­fer­ence.

Charlie Marsh’s Jane Street talk and a Xebia en­gi­neer­ing deep-dive cover the tech­ni­cal de­tails well. The in­ter­est­ing parts are the de­sign de­ci­sions: stan­dards that en­able fast paths, things uv drops that pip sup­ports, and op­ti­miza­tions that don’t re­quire Rust at all.

pip’s slow­ness is­n’t a fail­ure of im­ple­men­ta­tion. For years, Python pack­ag­ing re­quired ex­e­cut­ing code to find out what a pack­age needed.

The prob­lem was setup.py. You could­n’t know a pack­age’s de­pen­den­cies with­out run­ning its setup script. But you could­n’t run its setup script with­out in­stalling its build de­pen­den­cies. PEP 518 in 2016 called this out ex­plic­itly: “You can’t ex­e­cute a setup.py file with­out know­ing its de­pen­den­cies, but cur­rently there is no stan­dard way to know what those de­pen­den­cies are in an au­to­mated fash­ion with­out ex­e­cut­ing the setup.py file.”

This chicken-and-egg prob­lem forced pip to down­load pack­ages, ex­e­cute un­trusted code, fail, in­stall miss­ing build tools, and try again. Every in­stall was po­ten­tially a cas­cade of sub­process spawns and ar­bi­trary code ex­e­cu­tion. Installing a source dis­tri­b­u­tion was es­sen­tially curl | bash with ex­tra steps.

The fix came in stages:

* PEP 518 (2016) cre­ated pypro­ject.toml, giv­ing pack­ages a place to de­clare build de­pen­den­cies with­out code ex­e­cu­tion. The TOML for­mat was bor­rowed from Rust’s Cargo, which makes a Rust tool re­turn­ing to fix Python pack­ag­ing feel less like co­in­ci­dence.

* PEP 517 (2017) sep­a­rated build fron­tends from back­ends, so pip did­n’t need to un­der­stand se­tup­tools in­ter­nals.

* PEP 621 (2020) stan­dard­ized the [project] table, so de­pen­den­cies could be read by pars­ing TOML rather than run­ning Python.

* PEP 658 (2022) put pack­age meta­data di­rectly in the Simple Repository API, so re­solvers could fetch de­pen­dency in­for­ma­tion with­out down­load­ing wheels at all.

PEP 658 went live on PyPI in May 2023. uv launched in February 2024. uv could be fast be­cause the ecosys­tem fi­nally had the in­fra­struc­ture to sup­port it. A tool like uv could­n’t have shipped in 2020. The stan­dards weren’t there yet.

Other ecosys­tems fig­ured this out ear­lier. Cargo has had sta­tic meta­data from the start. npm’s pack­age.json is de­clar­a­tive. Python’s pack­ag­ing stan­dards fi­nally bring it to par­ity.

Speed comes from elim­i­na­tion. Every code path you don’t have is a code path you don’t wait for.

uv’s com­pat­i­bil­ity doc­u­men­ta­tion is a list of things it does­n’t do:

No .egg sup­port. Eggs were the pre-wheel bi­nary for­mat. pip still han­dles them; uv does­n’t even try. The for­mat has been ob­so­lete for over a decade.

No pip.conf. uv ig­nores pip’s con­fig­u­ra­tion files en­tirely. No pars­ing, no en­vi­ron­ment vari­able lookups, no in­her­i­tance from sys­tem-wide and per-user lo­ca­tions.

No byte­code com­pi­la­tion by de­fault. pip com­piles .py files to .pyc dur­ing in­stal­la­tion. uv skips this step, shav­ing time off every in­stall. You can opt in if you want it.

Virtual en­vi­ron­ments re­quired. pip lets you in­stall into sys­tem Python by de­fault. uv in­verts this, re­fus­ing to touch sys­tem Python with­out ex­plicit flags. This re­moves a whole cat­e­gory of per­mis­sion checks and safety code.

Stricter spec en­force­ment. pip ac­cepts mal­formed pack­ages that tech­ni­cally vi­o­late pack­ag­ing specs. uv re­jects them. Less tol­er­ance means less fall­back logic.

Ignoring re­quires-python up­per bounds. When a pack­age says it re­quires python, uv ig­nores the up­per bound and only checks the lower. This re­duces re­solver back­track­ing dra­mat­i­cally since up­per bounds are al­most al­ways wrong. Packages de­clare python be­cause they haven’t tested on Python 4, not be­cause they’ll ac­tu­ally break. The con­straint is de­fen­sive, not pre­dic­tive.

First-index wins by de­fault. When mul­ti­ple pack­age in­dexes are con­fig­ured, pip checks all of them. uv picks from the first in­dex that has the pack­age, stop­ping there. This pre­vents de­pen­dency con­fu­sion at­tacks and avoids ex­tra net­work re­quests.

Each of these is a code path pip has to ex­e­cute and uv does­n’t.

Some of uv’s speed comes from Rust. But not as much as you’d think. Several key op­ti­miza­tions could be im­ple­mented in pip to­day:

HTTP range re­quests for meta­data. Wheel files are zip archives, and zip archives put their file list­ing at the end. uv tries PEP 658 meta­data first, falls back to HTTP range re­quests for the zip cen­tral di­rec­tory, then full wheel down­load, then build­ing from source. Each step is slower and riskier. The de­sign makes the fast path cover 99% of cases. None of this re­quires Rust.

Parallel down­loads. pip down­loads pack­ages one at a time. uv down­loads many at once. Any lan­guage can do this.

Global cache with hardlinks. pip copies pack­ages into each vir­tual en­vi­ron­ment. uv keeps one copy glob­ally and uses hardlinks (or copy-on-write on filesys­tems that sup­port it). Installing the same pack­age into ten venvs takes the same disk space as one. Any lan­guage with filesys­tem ac­cess can do this.

Python-free res­o­lu­tion. pip needs Python run­ning to do any­thing, and in­vokes build back­ends as sub­processes to get meta­data from legacy pack­ages. uv parses TOML and wheel meta­data na­tively, only spawn­ing Python when it hits a setup.py-only pack­age that has no other op­tion.

PubGrub re­solver. uv uses the PubGrub al­go­rithm, orig­i­nally from Dart’s pub pack­age man­ager. Both pip and PubGrub use back­track­ing, but PubGrub ap­plies con­flict-dri­ven clause learn­ing from SAT solvers: when it hits a dead end, it an­a­lyzes why and skips sim­i­lar dead ends later. This makes it faster on com­plex de­pen­dency graphs and bet­ter at ex­plain­ing fail­ures. pip could adopt PubGrub with­out rewrit­ing in Rust.

Zero-copy de­se­ri­al­iza­tion. uv uses rkyv to de­se­ri­al­ize cached data with­out copy­ing it. The data for­mat is the in-mem­ory for­mat. Libraries like FlatBuffers achieve this in other lan­guages, but rkyv in­te­grates tightly with Rust’s type sys­tem.

Thread-level par­al­lelism. Python’s GIL forces par­al­lel work into sep­a­rate processes, with IPC over­head and data copy­ing. Rust can par­al­lelize across threads na­tively, shar­ing mem­ory with­out se­ri­al­iza­tion bound­aries. This mat­ters most for res­o­lu­tion, where the solver ex­plores many ver­sion com­bi­na­tions.

No in­ter­preter startup. Every time pip spawns a sub­process, it pays Python’s startup cost. uv is a sin­gle sta­tic bi­nary with no run­time to ini­tial­ize.

Compact ver­sion rep­re­sen­ta­tion. uv packs ver­sions into u64 in­te­gers where pos­si­ble, mak­ing com­par­i­son and hash­ing fast. Over 90% of ver­sions fit in one u64. This is mi­cro-op­ti­miza­tion that com­pounds across mil­lions of com­par­isons.

These are real ad­van­tages. But they’re smaller than the ar­chi­tec­tural wins from drop­ping legacy sup­port and ex­ploit­ing mod­ern stan­dards.

uv is fast be­cause of what it does­n’t do, not be­cause of what lan­guage it’s writ­ten in. The stan­dards work of PEP 518, 517, 621, and 658 made fast pack­age man­age­ment pos­si­ble. Dropping eggs, pip.conf, and per­mis­sive pars­ing made it achiev­able. Rust makes it a bit faster still.

pip could im­ple­ment par­al­lel down­loads, global caching, and meta­data-only res­o­lu­tion to­mor­row. It does­n’t, largely be­cause back­wards com­pat­i­bil­ity with fif­teen years of edge cases takes prece­dence. But it means pip will al­ways be slower than a tool that starts fresh with mod­ern as­sump­tions.

Other pack­age man­agers could learn from this: sta­tic meta­data, no code ex­e­cu­tion to dis­cover de­pen­den­cies, and the abil­ity to re­solve every­thing up­front be­fore down­load­ing. Cargo and npm have op­er­ated this way for years. If your ecosys­tem re­quires run­ning ar­bi­trary code to find out what a pack­age needs, you’ve al­ready lost.

Christmas is of­ten re­garded as a time for good­will, but one young UK cou­ple’s act of kind­ness 50 years ago changed their lives for­ever.

On 23 December 1975, Rob Parsons and his wife Dianne were prepar­ing for Christmas at their Cardiff home when they heard a knock at the door.

On their doorstep stood a man with a bin bag con­tain­ing his pos­ses­sions in his right hand and a frozen chicken in his left.

Rob stud­ied the man’s face and vaguely re­mem­bered him as Ronnie Lockwood, some­one he would oc­ca­sion­ally see at Sunday School as a boy and who he was told to be kind to as he was a “bit dif­fer­ent”.

“I said ‘Ronnie, what’s with the chicken?’ He said ‘somebody gave it to me for Christmas’. And then I said two words that changed all of our lives.

“And I’m not ex­actly sure why I said them. I said come in.”

People ex­am­in­ing doc­u­ments re­leased by the Department of Justice in the Jeffrey Epstein case dis­cov­ered that some of the file redac­tion can be un­done with Photoshop tech­niques, or by sim­ply high­light­ing text to paste into a word pro­cess­ing file.

Un-redacted text from these doc­u­ments be­gan cir­cu­lat­ing through so­cial me­dia on Monday evening. An ex­hibit in a civil case in the Virgin Islands against Darren K Indyke and Richard D Kahn, two ex­ecu­tors of Epstein’s es­tate, con­tains redacted al­le­ga­tions ex­plain­ing how Epstein and his as­so­ci­ates had fa­cil­i­tated the sex­ual abuse of chil­dren. The ex­hibit was the sec­ond amended com­plaint in the state case against Indyke and Kahn.

In sec­tion 85, the redacted por­tion states: “Between September 2015 and June 2019, Indyke signed (FAC) for over $400,000 made payable to young fe­male mod­els and ac­tresses, in­clud­ing a for­mer Russian model who re­ceived over $380,000 through monthly pay­ments of $8,333 made over a pe­riod of more than three and a half years un­til the mid­dle of 2019.”

Prosecutors in the Virgin Islands set­tled its civil sex-traf­fick­ing case against Epstein’s es­tate, Indyke and Kahn in 2022 for $105m, plus one half of the pro­ceeds from the sale of Little St James, the is­land on which Epstein resided and on which many of his crimes oc­curred. The jus­tice de­part­ment press re­lease an­nounc­ing the set­tle­ment did not in­clude an ad­mis­sion of li­a­bil­ity.

Indyke, an at­tor­ney who rep­re­sented Epstein for decades, has not been crim­i­nally in­dicted by fed­eral au­thor­i­ties. He was hired by the Parlatore Law Group in 2022, be­fore the jus­tice de­part­ment set­tled the Epstein case. That firm rep­re­sents the de­fense sec­re­tary, Pete Hegseth, and pre­vi­ously rep­re­sented Donald Trump in his de­fense against charges stem­ming from the dis­cov­ery of clas­si­fied gov­ern­ment doc­u­ments stored at Trump’s Florida es­tate. Calls and email seek­ing com­ment from Indyke and the Parlatore Law Group have not yet been re­turned.

Trump has re­peat­edly de­nied any knowl­edge of or in­volve­ment in Epstein’s crim­i­nal ac­tiv­i­ties and any wrong­do­ing.

Other sec­tions fur­ther al­lege how Epstein’s en­ter­prise con­cealed crimes.

“Defendants also at­tempted to con­ceal their crim­i­nal sex traf­fick­ing and abuse, con­duct by pay­ing large sums of money to par­tic­i­pant-wit­nesses, in­clud­ing by pay­ing for their at­tor­neys’ fees and case costs in lit­i­ga­tion re­lated to this con­duct,” reads one redacted pas­sage.

“Epstein also threat­ened harm to vic­tims and helped re­lease dam­ag­ing sto­ries about them to dam­age their cred­i­bil­ity when they tried to go pub­lic with their sto­ries of be­ing traf­ficked and sex­u­ally abused. Epstein also in­structed one or more Epstein Enterprise par­tic­i­pant-wit­nesses to de­stroy ev­i­dence rel­e­vant to on­go­ing court pro­ceed­ings in­volv­ing Defendants’ crim­i­nal sex traf­fick­ing and abuse con­duct.”

Redactions of sec­tions 184 through 192 of the doc­u­ment de­scribe prop­erty taxes paid by com­pa­nies in­cor­po­rated by Epstein on prop­er­ties that were not on the bal­ance sheet for those firms.

“For in­stance, Cypress’s Balance Sheet as of December 31, 2018 did not re­flect any as­sets other than cash of $18,824. Further, Cypress re­ported only $301 in ex­penses for the year ended December 31, 2018, de­spite it pay­ing $106,394.60 in Santa Fe prop­erty taxes on November 6, 2018,” reads one redacted pas­sage.

“Similarly, in 2017, Cypress re­ported as its only as­set cash in the amount of $29,736 and ex­penses of $150, de­spite it pay­ing $55,770.41 and $113,679.56 in Santa Fe prop­erty taxes dur­ing 2017.”

The Epstein Files Transparency Act signed into law last month per­mits the Department of Justice “to with­hold cer­tain in­for­ma­tion such as the per­sonal in­for­ma­tion of vic­tims and ma­te­ri­als that would jeop­ar­dize an ac­tive fed­eral in­ves­ti­ga­tion”.

It was un­clear how prop­erty ma­te­r­ial com­plies with the redac­tion stan­dard un­der the law. An in­quiry to the Department of Justice has not yet been an­swered.

I’ve been read­ing Lord of the Rings for two months and I’m just at the end of the first part. It’s not be­cause I’m not en­joy­ing it. It’s one of the most en­joy­able read­ing ex­pe­ri­ences I can re­mem­ber.

From the be­gin­ning, I’ve read the whole thing aloud. I’ve found read­ing aloud help­ful for stay­ing en­gaged — lim­it­ing my­self to mouth-speed rather than eye-speed means I won’t rush, miss im­por­tant de­tails, and then lose in­ter­est, which has al­ways been a prob­lem for me.

At first I was anx­ious to read a 1,500-page book this way, be­cause it would take so long. But, as some­one pointed out to me, if I’m en­joy­ing it, why would I want to be done with it sooner?

So I tried slow­ing down even more, and dis­cov­ered some­thing. I slowed to a pace that felt al­most ab­surd, treat­ing each sen­tence as though it might be a par­tic­u­larly im­por­tant one. I gave each one maybe triple the usual time and at­ten­tion, ig­nor­ing the fact that there are hun­dreds of pages to go.

This leisurely pace made Middle-Earth blos­som be­fore my eyes. When I paused af­ter each comma, and let each sen­tence ring for a small mo­ment af­ter the pe­riod, the events of the story reached me with more weight and strength. That ex­tra time gave space for Tolkien’s im­ages and moods to prop­a­gate in my mind, which they did au­to­mat­i­cally.

Some part of me still wanted to rush and get on with it, to make good time, to gloss over the songs and lore to get to Moria and Mount Doom and the other mar­quee mo­ments of the story. But the more I ig­nored that im­pulse, the bet­ter the ex­pe­ri­ence got.

By of­fer­ing the book about triple the usual amount of at­ten­tive­ness, I was get­ting about triple the sto­ry­ness (i.e. mean­ing, en­gage­ment, lit­er­ary plea­sure). Whatever the thing is that I’m seek­ing when I pick up a novel in the first place, there’s much more of it avail­able at this pace.

This ef­fect re­minded me of a para­dox around eat­ing I rec­og­nized long ago. When you slow down your eat­ing speed, say to half or a third your de­fault speed, you get much more en­joy­ment out of a smaller amount of food. The ex­tra at­ten­tion given to each bite al­lows more of the “good stuff,” what­ever that is ex­actly, to reach you.

What’s para­dox­i­cal is that it’s pre­cisely the seek­ing of that “good stuff” that nor­mally dri­ves me to eat so quickly, and miss most of what I’m seek­ing. When you try to bar­rel ahead to ac­cess the good stuff quicker, you get less of it in the end. Slow down and much more of it is re­leased.

And it’s re­leased au­to­mat­i­cally, in both read­ing and eat­ing. You don’t have to search it out. The good stuff (the mean­ing in the text, the plea­sure in the eat­ing) just rises up to meet you in that ex­tra time you give it. Slowing down, and of­fer­ing more time to the act of con­sump­tion, im­me­di­ately in­creases read­ing com­pre­hen­sion (and eat­ing com­pre­hen­sion).

Both are anal­o­gous to slow­ing down while you vac­uum a car­pet. If you pass the vac­uum head too quickly, you miss half the dirt. Slow down, and you can hear how much more grit is sent skit­ter­ing up the tube. The suc­tion and bris­tles are work­ing, but they need more time to do their work fully, to draw up the deeper-ly­ing stuff.

It seems that my de­fault con­sump­tion speeds for read­ing and eat­ing (and maybe every­thing else) re­duce the re­wards of those things sig­nif­i­cantly, un­der­min­ing the point of do­ing ei­ther.

Part of it is my own im­pa­tience. But I also sus­pect that mod­ern liv­ing, with its in­fi­nite sup­ply of con­sum­ables, tends to push our rate-of-in­take di­als too high. I’m not go­ing to run out of books, or snacks, or op­por­tu­ni­ties to learn some­thing. There’s al­ways more, so not every crust of bread or printed page needs to be ap­pre­ci­ated fully.

Internally though, the mind is jug­gling like Lucy and Ethel on the con­veyor belt at the choco­late fac­tory. Our re­cep­tors for mean­ing and ap­pre­ci­a­tion, like the vac­uum head, need more time to do their full work, to make all the con­nec­tions they’re de­signed to make.

It might sound like I’m just of­fer­ing clichés — less is more, stop and smell the roses, take your time — and I guess I am. But clichés suf­fer the same is­sue: they are of­ten pro­found in­sights, con­sumed and passed on too rapidly for their real mean­ing to reg­is­ter any­more. You re­ally should stop and smell roses, as you know if you’re in the habit of do­ing that.

At least see what hap­pens when you re­duce your con­sump­tion speed — of any­thing, but es­pe­cially books, in­for­ma­tion, and food — by a half, or two thirds. Notice that (1) some­thing in you re­ally wants to plow through at the high­est vi­able set­ting, and (2) how much more of the re­ward is re­leased when you slow down any­way.

As far as I can tell, al­most every­thing be­comes more sat­is­fy­ing when you give it more time and in­ten­tion, even things like check­ing the mail­box or writ­ing a shop­ping list.

Slowing down your rate of con­sump­tion will in­evitably change what you want to con­sume. Reading throw­away news ar­ti­cles or AI slop with great care and at­ten­tion is only go­ing to show you how empty of value it is. Reading dense writ­ing in inky old books, crafted for your mind by great mas­ters, be­comes eas­ier with­out the rushed pace, and the mean­ing just blooms out of it.

Same with food. Try to sa­vor a cheap, waxy “chocolate” bar, or a bag of store-brand cheese puffs, and you dis­cover a harsh taste that you don’t want to look at too closely. Enjoy a home­made pas­try with great at­ten­tion, and dis­cover there’s even more in it than you re­al­ized.

Mass pro­duc­tion is good in so many ways, but the faster we tend to con­sume its fruits, the more we end up seek­ing things for their glossy, can­died sur­faces. The more we go for these sur­face-level re­wards, the more the cul­ture fo­cuses on of­fer­ing only that part — such as TikTok videos, processed food, CGI-forward movies, and pub­lic dis­course in the form of un­ex­am­ined talk­ing points.

Who knows how far we’ve drifted from the best modes of con­sum­ing the things we value. Once some­thing be­comes a norm, it seems like an ap­pro­pri­ate stan­dard, no mat­ter how much has been lost. Apparently, read­ing silently and alone was un­usual un­til as late as the 18th cen­tury. Certainly sit-down meals and cook­ing at home were.

I don’t mean to sound like a scold. Let’s say none of this is morally good or bad. It’s just that in so much of what we do, we could be get­ting much more of the part of it that we re­ally seek — but it’s only avail­able at slower speeds.

If you’re cu­ri­ous, try con­sum­ing things more slowly, so slowly it seems silly to oth­ers — say a third your ha­bit­ual speed — and see what rises up to meet you.

Recently I opened a dis­cus­sion fo­rum for Raptitude read­ers who want to give some­thing up for the month of December (alcohol, so­cial me­dia, snacks, etc).

It’s been a real suc­cess, and many peo­ple want to do some­thing sim­i­lar in January. If you want to quit some­thing, or just give it up for a month, you’re in­vited to join.

Follow this link at the end of this post to get an in­vite.

Flock

Flock Exposed Its AI-Powered Cameras to the Internet. We Tracked Ourselves

Flock left at least 60 of its peo­ple-track­ing Condor PTZ cam­eras live stream­ing and ex­posed to the open in­ter­net.

I am stand­ing on the cor­ner of Harris Road and Young Street out­side of the Crossroads Business Park in Bakersfield, California, look­ing up at a Flock sur­veil­lance cam­era bolted high above a traf­fic sig­nal. On my phone, I am watch­ing my­self in real time as the cam­era records and livestreams me—with­out any pass­word or lo­gin—to the open in­ter­net. I wan­der into the in­ter­sec­tion, stare at the cam­era and wave. On the livestream, I can see my­self clearly. Hundreds of miles away, my col­leagues are re­motely watch­ing me too through the ex­posed feed. Flock left livestreams and ad­min­is­tra­tor con­trol pan­els for at least 60 of its AI-enabled Condor cam­eras around the coun­try ex­posed to the open in­ter­net, where any­one could watch them, down­load 30 days worth of video archive, and change set­tings, see log files, and run di­ag­nos­tics. Un­like many of Flock’s cam­eras, which are de­signed to cap­ture li­cense plates as peo­ple drive by, Flock’s Condor cam­eras are pan-tilt-zoom (PTZ) cam­eras de­signed to record and track peo­ple, not ve­hi­cles. Condor cam­eras can be set to au­to­mat­i­cally zoom in on peo­ple’s faces as they walk through a park­ing lot, down a pub­lic street, or play on a play­ground, or they can be con­trolled man­u­ally, ac­cord­ing to mar­ket­ing ma­te­r­ial on Flock’s web­site. We watched Condor cam­eras zoom in on a woman walk­ing her dog on a bike path in sub­ur­ban Atlanta; a cam­era fol­lowed a man walk­ing through a Macy’s park­ing lot in Bakersfield; sur­veil chil­dren swing­ing on a swingset at a play­ground; and film high-res video of peo­ple sit­ting at a stop­light in traf­fic. In one case, we were able to watch a man rollerblade down Brookhaven, Georgia’s Peachtree Creek Greenway bike path. The Flock cam­era zoomed in on him and tracked him as he rolled past. Minutes later, he showed up on an­other ex­posed cam­era livestream fur­ther down the bike path. The cam­er­a’s res­o­lu­tion was good enough that we were able to see that, when he stopped be­neath one of the cam­eras, he was watch­ing rollerblad­ing videos on his phone.The ex­po­sure was ini­tially dis­cov­ered by YouTuber and tech­nol­o­gist Benn Jordan and was shared with se­cu­rity re­searcher Jon “GainSec” Gaines, who re­cently found nu­mer­ous vul­ner­a­bil­i­ties in sev­eral other mod­els of Flock’s au­to­mated li­cense plate reader (ALPR) cam­eras. They shared the de­tails of what they found  with me, and I ver­i­fied many of the de­tails seen in the ex­posed por­tals by dri­ving to Bakersfield to walk in front of two cam­eras there while I watched my­self on the livestream. I also pulled Flock’s con­tracts with cities for Condor cam­eras, pulled de­tails from com­pany pre­sen­ta­tions about the tech­nol­ogy, and ge­olo­cated a hand­ful of the cam­eras to cities and towns across the United States. Jordan also filmed him­self in front of sev­eral of the cam­eras on the Peachtree Creek Greenway bike path. Jordan said he and Gaines dis­cov­ered many of the ex­posed cam­eras with Shodan, an in­ter­net of things search en­gine that re­searchers reg­u­larly use to iden­tify im­prop­erly se­cured de­vices. Af­ter find­ing links to the feed, “immediately, we were just with­out any user­name, with­out any pass­word, we were just see­ing every­thing from play­grounds to park­ing lots with peo­ple, Christmas shop­ping and un­load­ing their stuff into cars,” Jordan told me in an in­ter­view. “I think it was like the first time that I ac­tu­ally got like im­me­di­ately scared … I think the one that af­fected me most was as play­ground. You could see un­at­tended kids, and that’s some­thing I want peo­ple to know about so they can un­der­stand how dan­ger­ous this is.” In a YouTube video about his re­search, Jordan said he was able to use footage pulled from the ex­posed feed to iden­tify spe­cific peo­ple us­ing open source in­ves­ti­ga­tion tools in or­der to show how triv­ially an ex­po­sure like this could be abused.

This post is for paid mem­bers only

Become a paid mem­ber for un­lim­ited ad-free ac­cess to ar­ti­cles, bonus pod­cast con­tent, and more.

Subscribe

Sign up for free ac­cess to this post

Free mem­bers get ac­cess to posts like this one along with an email round-up of our week’s sto­ries.

Subscribe

Already have an ac­count? Sign in

More like this

Flock Uses Overseas Gig Workers to Build its Surveillance AI

Flock ac­ci­den­tally ex­posed train­ing ma­te­ri­als and a panel which tracked what its AI an­no­ta­tors were work­ing on. It showed that Flock, which has cam­eras in thou­sands of U.S. com­mu­ni­ties, is us­ing work­ers in the Philippines to re­view and clas­sify footage.

Cops Used Flock to Monitor No Kings Protests Around the Country

A mas­sive cache of Flock lookups col­lated by the Electronic Frontier Foundation (EFF) shows as many as 50 fed­eral, state, and lo­cal agen­cies used Flock dur­ing protests over the last year.

“Most dri­vers are un­aware that San Jose’s Police Department is track­ing their lo­ca­tions and do not know all that their saved lo­ca­tion data can re­veal about their pri­vate lives and ac­tiv­i­ties.”

Why I Quit Streaming And Got Back Into Cassettes

In the age of Spotify and AI slop, tapes re­mind us what we’re miss­ing when we stop tak­ing risks.

Podcast: We Tracked Ourselves with Exposed Flock Cameras

How we tracked our­selves with ex­posed Flock cam­eras; a year in re­view; and our per­sonal rec­om­men­da­tions on all sorts of things.

iCloud, Mega, and as a tor­rent. Archivists have up­loaded the 60 Minutes episode Bari Weiss spiked.

We are pleased to an­nounce the re­lease of Ruby 4.0.0. Ruby 4.0 in­tro­duces “Ruby Box” and “ZJIT”, and adds many im­prove­ments.

Ruby Box is a new (experimental) fea­ture to pro­vide sep­a­ra­tion about de­f­i­n­i­tions. Ruby Box is en­abled when an en­vi­ron­ment vari­able RUBY_BOX=1 is spec­i­fied. The class is Ruby::Box.

Definitions loaded in a box are iso­lated in the box. Ruby Box can iso­late/​sep­a­rate mon­key patches, changes of global/​class vari­ables, class/​mod­ule de­f­i­n­i­tions, and loaded na­tive/​ruby li­braries from other boxes.

* Run test cases in box to pro­tect other tests when the test case uses mon­key patches to over­ride some­thing

* Run web app boxes in par­al­lel to ex­e­cute blue-green de­ploy­ment on an app server in a Ruby process

* Run web app boxes in par­al­lel to eval­u­ate de­pen­dency up­dates for a cer­tain pe­riod of time by check­ing re­sponse diff us­ing Ruby code

* Used as the foun­da­tion (low-level) API to im­ple­ment kind of “package” (high-level) API (it is not de­signed yet)

For the de­tail of “Ruby Box”, see Ruby::Box. [Feature #21311] [Misc #21385]

ZJIT is a new just-in-time (JIT) com­piler, which is de­vel­oped as the next gen­er­a­tion of YJIT. You need Rust 1.85.0 or newer to build Ruby with ZJIT sup­port, and ZJIT is en­abled when –zjit is spec­i­fied.

We’re build­ing a new com­piler for Ruby be­cause we want to both raise the per­for­mance ceil­ing (bigger com­pi­la­tion unit size and SSA IR) and en­cour­age more out­side con­tri­bu­tion (by be­com­ing a more tra­di­tional method com­piler). See our blog post for more de­tails.

ZJIT is faster than the in­ter­preter, but not yet as fast as YJIT. We en­cour­age you to ex­per­i­ment with ZJIT, but maybe hold off on de­ploy­ing it in pro­duc­tion for now. Stay tuned for Ruby 4.1 ZJIT.

Ractor, Ruby’s par­al­lel ex­e­cu­tion mech­a­nism, has re­ceived sev­eral im­prove­ments. A new class, Ractor::Port, was in­tro­duced to ad­dress is­sues re­lated to mes­sage send­ing and re­ceiv­ing (see our blog post). Additionally, Ractor.shareable_proc makes it eas­ier to share Proc ob­jects be­tween Ractors.

On the per­for­mance side, many in­ter­nal data struc­tures have been im­proved to sig­nif­i­cantly re­duce con­tention on a global lock, un­lock­ing bet­ter par­al­lelism. Ractors also now share less in­ter­nal data, re­sult­ing in less CPU cache con­tention when run­ning in par­al­lel.

Ractor was first in­tro­duced in Ruby 3.0 as an ex­per­i­men­tal fea­ture. We aim to re­move its “experimental” sta­tus next year.

*nil no longer calls nil.to_a, sim­i­lar to how **nil does not call nil.to_hash. [Feature #21047]

Logical bi­nary op­er­a­tors (||, &&, and and or) at the be­gin­ning of a line con­tinue the pre­vi­ous line, like flu­ent dot. The fol­low­ing code ex­am­ples are equal:

We only list stdlib changes that are no­table fea­ture changes.

Other changes are listed in the fol­low­ing sec­tions. We also listed re­lease his­tory from the pre­vi­ous bun­dled ver­sion that is Ruby 3.4.0 if it has GitHub re­leases.

The fol­low­ing bun­dled gems are pro­moted from de­fault gems.

The fol­low­ing de­fault gem is added.

The fol­low­ing de­fault gems are up­dated.

The fol­low­ing bun­dled gems are up­dated.

Ruby 4.0 bun­dled RubyGems and Bundler ver­sion 4. see the fol­low­ing links for de­tails.

Dropped sup­port for MSVC ver­sions older than 14.0 (_MSC_VER 1900).

This means Visual Studio 2015 or later is now re­quired.

* Dropped sup­port for MSVC ver­sions older than 14.0 (_MSC_VER 1900).

This means Visual Studio 2015 or later is now re­quired.

The fol­low­ing meth­ods were re­moved from Ractor due to the ad­di­tion of Ractor::Port:

Process::Status#& and Process::Status#>> have been re­moved. They were dep­re­cated in Ruby 3.3. [Bug #19868]

rb_­path_check has been re­moved. This func­tion was used for

$SAFE path check­ing which was re­moved in Ruby 2.7, and was al­ready dep­re­cated. [Feature #20971]

A back­trace for ArgumentError of “wrong num­ber of ar­gu­ments” now in­clude the re­ceiver’s class or mod­ule name (e.g., in Foo#bar

in­stead of in bar). [Bug #21698]

Backtraces no longer dis­play in­ter­nal frames. These meth­ods now ap­pear as if it is in the Ruby source file, con­sis­tent with other C-implemented meth­ods. [Bug #20968]

ruby -e ‘[1].fetch_values(42)’

:211:in ‘Array#fetch’: in­dex 42 out­side of ar­ray bounds: -1…1 (IndexError)

from :211:in ‘block in Array#fetch_values’

from

$ ruby -e ‘[1].fetch_values(42)’

-e:1:in ‘Array#fetch_values’: in­dex 42 out­side of ar­ray bounds: -1…1 (IndexError)

from -e:1:in ′

CGI li­brary is re­moved from the de­fault gems. Now we only pro­vide cgi/​es­cape for the fol­low­ing meth­ods:

With the move of Set from stdlib to core class, set/​sort­ed_set.rb has been re­moved, and SortedSet is no longer an au­toloaded con­stant. Please in­stall the sort­ed_set gem and re­quire ‘sorted_set’ to use SortedSet. [Feature #21287]

The de­fault be­hav­ior of au­to­mat­i­cally set­ting the Content-Type header

to ap­pli­ca­tion/​x-www-form-ur­len­coded for re­quests with a body

(e.g., POST, PUT) when the header was not ex­plic­itly set has been

re­moved. If your ap­pli­ca­tion re­lied on this au­to­matic de­fault, your

re­quests will now be sent with­out a Content-Type header, po­ten­tially

break­ing com­pat­i­bil­ity with cer­tain servers.

[GH-net-http #205]

* The de­fault be­hav­ior of au­to­mat­i­cally set­ting the Content-Type header

to ap­pli­ca­tion/​x-www-form-ur­len­coded for re­quests with a body

(e.g., POST, PUT) when the header was not ex­plic­itly set has been

re­moved. If your ap­pli­ca­tion re­lied on this au­to­matic de­fault, your

re­quests will now be sent with­out a Content-Type header, po­ten­tially

break­ing com­pat­i­bil­ity with cer­tain servers.

[GH-net-http #205]

rb_thread­_fd_­close is dep­re­cated and now a no-op. If you need to ex­pose

file de­scrip­tors from C ex­ten­sions to Ruby code, cre­ate an IO in­stance

us­ing RUBY_IO_MODE_EXTERNAL and use rb_io_­close(io) to close it (this

also in­ter­rupts and waits for all pend­ing op­er­a­tions on the IO

in­stance). Directly clos­ing file de­scrip­tors does not in­ter­rupt pend­ing

op­er­a­tions, and may lead to un­de­fined be­hav­iour. In other words, if two

IO ob­jects share the same file de­scrip­tor, clos­ing one does not af­fect

the other. [Feature #18455]

* rb_thread­_fd_­close is dep­re­cated and now a no-op. If you need to ex­pose

file de­scrip­tors from C ex­ten­sions to Ruby code, cre­ate an IO in­stance

us­ing RUBY_IO_MODE_EXTERNAL and use rb_io_­close(io) to close it (this

also in­ter­rupts and waits for all pend­ing op­er­a­tions on the IO

in­stance). Directly clos­ing file de­scrip­tors does not in­ter­rupt pend­ing

op­er­a­tions, and may lead to un­de­fined be­hav­iour. In other words, if two

IO ob­jects share the same file de­scrip­tor, clos­ing one does not af­fect

the other. [Feature #18455]

rb_thread­_­cal­l_with­_gvl now works with or with­out the GVL.

This al­lows gems to avoid check­ing ru­by_thread­_has_gvl_p.

Please still be dili­gent about the GVL. [Feature #20750]

* rb_thread­_­cal­l_with­_gvl now works with or with­out the GVL.

This al­lows gems to avoid check­ing ru­by_thread­_has_gvl_p.

Please still be dili­gent about the GVL. [Feature #20750]

A C API for Set has been added. The fol­low­ing meth­ods are sup­ported: [Feature #21459]

A C API for Set has been added. The fol­low­ing meth­ods are sup­ported: [Feature #21459]

* Class#new (ex. Object.new) is faster in all cases, but es­pe­cially when pass­ing key­word ar­gu­ments. This has also been in­te­grated into YJIT and ZJIT. [Feature #21254]

* GC heaps of dif­fer­ent size pools now grow in­de­pen­dently, re­duc­ing mem­ory us­age when only some pools con­tain long-lived ob­jects

* GC sweep­ing is faster on pages of large ob­jects

* “Generic ivar” ob­jects (String, Array, TypedData, etc.) now use a new in­ter­nal “fields” ob­ject for faster in­stance vari­able ac­cess

* The GC avoids main­tain­ing an in­ter­nal id2ref table un­til it is first used, mak­ing ob­jec­t_id al­lo­ca­tion and GC sweep­ing faster

* ob­jec­t_id and hash are faster on Class and Module ob­jects

* Random, Enumerator::Product, Enumerator::Chain, Addrinfo,

StringScanner, and some in­ter­nal ob­jects are now write-bar­rier pro­tected,

which re­duces GC over­head.

A lot of work has gone into mak­ing Ractors more sta­ble, per­for­mant, and us­able. These im­prove­ments bring Ractor im­ple­men­ta­tion closer to leav­ing ex­per­i­men­tal sta­tus.

* Performance im­prove­ments

Frozen strings and the sym­bol table in­ter­nally use a lock-free hash set [Feature #21268]

Class (and generic ivar) in­stance vari­able ac­cess is faster and avoids lock­ing

CPU cache con­tention is avoided in ob­ject al­lo­ca­tion by us­ing a per-rac­tor counter

CPU cache con­tention is avoided in xmal­loc/​xfree by us­ing a thread-lo­cal counter

* Frozen strings and the sym­bol table in­ter­nally use a lock-free hash set [Feature #21268]

* Class (and generic ivar) in­stance vari­able ac­cess is faster and avoids lock­ing

* CPU cache con­tention is avoided in ob­ject al­lo­ca­tion by us­ing a per-rac­tor counter

* CPU cache con­tention is avoided in xmal­loc/​xfree by us­ing a thread-lo­cal counter