We’re ex­cited to see the first press re­views go live for the Framework Laptop and the first or­ders land on your doorsteps to­day! With the FTC unan­i­mously vot­ing to en­force the Right to Repair just yes­ter­day, our tim­ing could­n’t be bet­ter for de­liv­er­ing a great, high per­for­mance, easy to re­pair prod­uct. There is a ton of amaz­ing ma­te­r­ial to read and watch, with more com­ing in the next weeks. Some of our fa­vorite quotes so far are:

“A poster child for the right-to-re­pair move­ment, Framework’s mod­u­lar lap­top is one of the smartest de­signs I’ve seen in a long time.”

“It’s the ul­ti­mate Right to Repair lap­top.”

“The Framework Laptop is more than just [a] worth­while ex­per­i­ment in mod­u­lar­ity, it’s also a great lap­top.”

Reviewers loved the free­dom to re­pair and up­grade, the Expansion Card sys­tem, CPU per­for­mance, key­board feel, we­b­cam qual­ity, and more.  Of course, in­side of Framework, we grav­i­tate to­wards the crit­i­cal feed­back that points us to where to do bet­ter.  We take every bit of feed­back se­ri­ously, and we want your thoughts as you start us­ing your Framework Laptop.  This lets us know where to fo­cus for fu­ture im­prove­ments, whether that is for firmware up­dates, mod­ules, or next prod­ucts.  A won­der­ful thing about our prod­uct phi­los­o­phy is that im­prove­ments can go into re­place­ment parts and up­grades that every ex­ist­ing user can pick up and swap to, rather than need­ing to wait around and pay for an en­tirely new prod­uct.

We’re grate­ful to each of you who have or­dered al­ready, and we’re look­ing for­ward to get­ting your Framework Laptop to you.  Batch 1 pre-or­ders for July de­liv­ery con­tinue to ship out from our ware­house each day.  We’ll start Batch 2 ship­ments for August de­liv­ery soon af­ter.  We have a small num­ber of Batch 2 Framework Laptop and Framework Laptop DIY Edition units cur­rently avail­able for sale, with just a fully re­fund­able $100 de­posit due to­day.  If you pre-or­der now, some of you will be able to re­ceive your or­der within 3-4 weeks.

As proud as we are of the Framework Laptop (and we’re ex­tremely proud!), the great­est thing we have cre­ated over the last 18 months is the team that built it.  It takes an in­cred­i­ble team to build an ex­cel­lent prod­uct this com­plex and de­liver it on time.  We’re hir­ing on all fronts to con­tinue de­vel­op­ing the Framework Laptop ecosys­tem and ini­ti­ate our next cat­e­gories.  Let us know if you know any­one who may be in­ter­ested in help­ing us build prod­ucts that are bet­ter for peo­ple and the planet.

How can com­put­ers learn to rec­og­nize birds from sounds? The Cornell Lab of Ornithology and the Chemnitz University of Technology are try­ing to find an an­swer to this ques­tion. Our re­search is mainly fo­cused on the de­tec­tion and clas­si­fi­ca­tion of avian sounds us­ing ma­chine learn­ing — we want to as­sist ex­perts and cit­i­zen sci­en­tist in their work of mon­i­tor­ing and pro­tect­ing our birds. BirdNET is a re­search plat­form that aims at rec­og­niz­ing birds by sound at scale. We sup­port var­i­ous hard­ware and op­er­at­ing sys­tems such as Arduino mi­cro­con­trollers, the Raspberry Pi, smart­phones, web browsers, work­sta­tion PCs, and even cloud ser­vices. BirdNET is a cit­i­zen sci­ence plat­form as well as an analy­sis soft­ware for ex­tremely large col­lec­tions of au­dio. BirdNET aims to pro­vide in­no­v­a­tive tools for con­ser­va­tion­ists, bi­ol­o­gists, and bird­ers alike.

This page fea­tures some of our pub­lic demon­stra­tions, in­clud­ing a live stream demo, a demo for the analy­sis of au­dio record­ings, an Android and iOS app, and its vi­su­al­iza­tion of sub­mis­sions. All demos are based on an ar­ti­fi­cial neural net­work we call BirdNET. We are con­stantly im­prov­ing the fea­tures and per­for­mance of our demos — please make sure to check back with us reg­u­larly.

We are cur­rently fea­tur­ing 984 of the most com­mon species of North America and Europe. We will add more species and more re­gions in the near fu­ture. Click here for the list of sup­ported species.

TLDR: read the pa­per and view the gallery of pretty Perl pro­grams.

In this pa­per, we aim to an­swer a long-stand­ing open prob­lem in the pro­gram­ming lan­guages com­mu­nity: is it pos­si­ble to smear paint on the wall with­out cre­at­ing valid Perl?

We an­swer this ques­tion in the af­fir­ma­tive: it is pos­si­ble to smear paint on the wall with­out cre­at­ing a valid Perl pro­gram. We em­ploy an em­pir­i­cal ap­proach, us­ing op­ti­cal char­ac­ter recog­ni­tion (OCR) soft­ware, which finds that merely 93% of paint splat­ters parse as valid Perl. We an­a­lyze the prop­er­ties of paint-splat­ter Perl pro­grams, and pre­sent seven ex­am­ples of paint splat­ters which are not valid Perl pro­grams.

Accepted for pub­li­ca­tion at SIGBOVIK 2019, held April 1st 2019 in Pittsburgh. Winner of a Unwitting Participation Ribbon, “an un­wel­come brand we’ve af­fixed to each pa­per de­ter­mined af­ter care­ful scrutiny to have in­cluded a gen­uine ar­ti­fact, thereby fur­ther­ing the ad­mirable causes of open sci­ence and fruit­ful pro­cras­ti­na­tion.”

Read it on Google Docs or down­load a PDF. Or grab the en­tire SIGBOVIK 2019 pro­ceed­ings; I’m on page 174.

Here’s all the paint splat­ters on a sin­gle page, along with the valid Perl source code cor­re­spond­ing to each. “Not valid” is writ­ten in red for those im­ages which did not parse as valid Perl pro­grams. If dif­fer­ent OCR set­tings rec­og­nized mul­ti­ple valid Perl pro­grams, I chose the one that seemed the most “interesting”, ac­cord­ing to my own aes­thetic sense.

Here’s a tar­ball of 100 paint-splat­ter im­ages that were used as the main dataset for this pa­per.

There are a few paint splat­ter Perl pro­grams that I did­n’t rec­og­nize as “interesting” un­til af­ter the SIGBOVIK sub­mis­sion dead­line. For ex­am­ple, this splat­ter is rec­og­nized by OCR as the string lerz­fi­jglp­Fiji-j, which eval­u­ates to the num­ber 0 in Perl:

The im­age be­low is rec­og­nized as the string -*?, which also eval­u­ates to the num­ber 0 in Perl:

Another sur­pris­ing pro­gram is shown be­low; OCR rec­og­nizes this im­age as the string ;i;c;;#\\?z{;?;;fn’:.;, which eval­u­ates to the string c in Perl:

Finally, this im­age is rec­og­nized as the string ;E,‘__’, which eval­u­ates to the string E__ in Perl:

Computing de­vices are won­der­ful; they surely must be, since so many of us have so many of them. The pro­lif­er­a­tion of com­put­ers leads di­rectly to a fa­mil­iar prob­lem, though: the files we want are al­ways on the wrong ma­chine. One so­lu­tion is syn­chro­niza­tion ser­vices that keep a set of files up to date across a mul­ti­tude of ma­chines; a num­ber of com­pa­nies have cre­ated suc­cess­ful com­mer­cial of­fer­ings based on such ser­vices. Some of us, though, are stub­bornly re­sis­tant to the idea of plac­ing our data in the hands of cor­po­ra­tions and their pro­pri­etary sys­tems. For those of us who would rather stay in con­trol of our data, sys­tems like Syncthing of­fer a pos­si­ble so­lu­tion.

The core idea be­hind syn­chro­niza­tion sys­tems is es­sen­tially the same for all of them: given a list of di­rec­to­ries and a list of sys­tems, en­sure that those di­rec­to­ries have the same con­tents on each sys­tem. If a file is added on one, it is copied out to the rest; mod­i­fi­ca­tions and dele­tions are (usually) prop­a­gated as well. The trou­ble is al­ways in the de­tails, though; from fid­dly setup pro­ce­dures to data cor­rup­tion and se­cu­rity prob­lems, there are a lot of ways in which syn­chro­niza­tion can go wrong. So users have to put a lot of trust in these sys­tems; open source code is an im­por­tant step to­ward that goal, but it is also nec­es­sary to be­lieve that the de­vel­op­ers in­volved have thought care­fully through the is­sues.

When the north­ern-hemi­sphere sum­mer sets in and fresh news be­comes

rel­a­tively scarce, the op­por­tu­nity arises to fi­nally get around to

check­ing out an in­ter­est­ing soft­ware pro­ject or two.

Your ed­i­tor, thus, has been duly play­ing around with the Syncthing 1.17.0

re­lease,

ob­tained from the Fedora and CentOS (EPEL) repos­i­to­ries. Starting the

sys­tem on Fedora is just a mat­ter of run­ning the com­mand

to start the syn­chro­niza­tion dae­mon

and try­ing to not be dis­mayed at the vol­ume of log data gen­er­ated. The

EPEL pack­age, in­stead, ap­pears to in­stall Syncthing as a sys­temd user

ser­vice, mean­ing that all that was needed was to log into the sys­tem and

the dae­mon started au­to­mat­i­cally.

The dae­mon is man­aged through an in­ter­nal web server that shows up, by de­fault, on port 8384; see the ex­am­ple im­age on the right. There is ini­tially no au­then­ti­ca­tion re­quired; users will likely want to fix that as one of the first things they do. The web server is, by de­fault, only ac­ces­si­ble via the loop­back in­ter­face; a change to a con­fig­u­ra­tion file can make it avail­able to the Internet as a whole, but that sounds like a dar­ing thing to do even with au­then­ti­ca­tion en­abled. An al­ter­na­tive for gain­ing ac­cess to the web in­ter­face on a re­mote ma­chine, as sug­gested in the doc­u­men­ta­tion, is to set up an SSH tun­nel from the lo­cal sys­tem.

When it starts for the first time, Syncthing gen­er­ates a “device ID” iden­ti­fy­ing the lo­cal sys­tem; it looks like this:

Setting up syn­chro­niza­tion be­tween two ma­chines re­sem­bles the Bluetooth pair­ing process; it is done by pro­vid­ing each side with the de­vice ID be­long­ing to the other. Use of copy-and-paste is ad­vis­able here. Alternatively, if both sys­tems are on the same lo­cal net, they will dis­cover each other through broad­casts and ask (through the man­age­ment in­ter­face) whether a con­nec­tion should be es­tab­lished.

After a con­nec­tion be­tween sys­tems is made, users must tell Syncthing which di­rec­to­ries should be syn­chro­nized; that is a mat­ter of set­ting up fold­ers and shar­ing them with any or all of the known re­mote sys­tems (which Syncthing calls “devices”). Once the share has been ac­cepted on the re­mote end, file changes will be prop­a­gated back and forth. When pos­si­ble, Syncthing re­quests file-change no­ti­fi­ca­tions from the ker­nel; that leads to rel­a­tively fast prop­a­ga­tion times.

There are a lot of op­tions that can be set to con­trol shar­ing. Sharing can be made one-way, for ex­am­ple, so that a par­tic­u­lar sys­tem might cre­ate files and send them out with­out ac­cept­ing changes from the other sys­tems. One es­pe­cially in­ter­est­ing (though new and “beta”) fea­ture is the abil­ity to share

files to

spe­cific sys­tems in en­crypted form. If one sys­tem is, for ex­am­ple, a cloud server that is used pri­mar­ily for backup or dis­tri­b­u­tion pur­poses, it can be given en­crypted data that it can­not read. Any other sys­tem in the shar­ing net­work that has the cor­rect pass­word will be able to read those files, though. There are also var­i­ous ways of han­dling ver­sion­ing, which keeps older ver­sions of files around when one sys­tem changes them.

It’s worth not­ing that, while it is pos­si­ble to con­fig­ure a set of Syncthing clients all con­nected to a cen­tral server, noth­ing in Syncthing re­quires that sort of ar­chi­tec­ture. Systems can be con­nected in any way that seems to make sense. If a sys­tem finds that it needs files that have al­ready prop­a­gated to mul­ti­ple con­nected peers, it can re­ceive the needed data in blocks, BitTorrent-style, from whichever sys­tem can pro­vide it first.

Interestingly, nei­ther host names nor IP ad­dresses are in­volved in any stage of the con­fig­u­ra­tion process — by de­fault, at least; the sys­tems find each other based only on the de­vice ID re­gard­less of which net­works they are at­tached to. This, clearly, re­quires some third-party help. The Syncthing pro­ject runs a set of “discovery” servers that will help sys­tems find each other based on their de­vice IDs. There is also a set of “relay servers” that can re­lay data in sit­u­a­tions where the sys­tems in­volved can­not reach each other di­rectly — when they are both be­hind NAT fire­walls, for ex­am­ple.

Some thought has clearly gone into the se­cu­rity im­pli­ca­tions of this ar­chi­tec­ture. Data only goes through re­lay servers if there is no al­ter­na­tive, for ex­am­ple, and it is en­crypted at the end­points. But there is still some in­for­ma­tion that a hos­tile dis­cov­ery or re­lay server could ob­tain that might worry some users. For any­body who is truly wor­ried, the code for both types of server is avail­able; any­body can set up pri­vate servers and con­fig­ure their Syncthing in­stances to use only those.

According to the doc­u­men­ta­tion, de­vice IDs need not be kept se­cret, since an af­fir­ma­tive ac­tion is re­quired on both sides to set up a con­nec­tion. One might won­der whether an at­tacker might try to set up a sys­tem with a tar­get’s de­vice ID and thus gain ac­cess to the man­aged files. That ID, though, is es­sen­tially a pub­lic key, and the con­nec­tion process in­volves prov­ing pos­ses­sion of the as­so­ci­ated pri­vate key, so such an at­tack should not be pos­si­ble. This page

de­scribes de­vice IDs in more de­tail.

Perhaps the most com­mon use of syn­chro­niza­tion on to­day’s net is copy­ing pho­tos from a phone hand­set to a cen­tral server. Since Android phones, at least, are Linux-based, one need only set up a nor­mal shell en­vi­ron­ment on it and put Syncthing there to achieve this goal; the process should­n’t take more than a day or so. Or one could just in­stall the Android app, which is avail­able

on F-Droid and the Google Play Store as well. This app, shown on the right, comes with a folder for the cam­era (set for send-only shar­ing) con­fig­ured out of the box, so it is just a mat­ter of set­ting up the peers. And, lest one worry about typ­ing one of those de­vice IDs with an on-screen key­board, the app can read the QR code that the web in­ter­face will help­fully pro­vide, eas­ing that process con­sid­er­ably.

One slightly sur­pris­ing be­hav­ior is that the app asks for lo­ca­tion per­mis­sion, which does­n’t seem like some­thing it would need. That per­mis­sion is needed to de­ter­mine which WiFi net­work (if any) the phone is on, which is use­ful for the fea­ture con­fig­ur­ing when syn­chro­niza­tion should (and should not) be per­formed. Users of me­tered WiFi ser­vices may want to use this mech­a­nism to avoid syn­chro­niza­tion when it could cost them money. In the ab­sence of this per­mis­sion, the app will, by de­fault, per­form syn­chro­niza­tion when­ever it is con­nected to any WiFi net­work.

One need not look far to find com­plaints from users that the Android app drains the bat­tery quickly. Your ed­i­tor has not ob­served this be­hav­ior in a lim­ited amount of test­ing; it is pos­si­ble that the worst prob­lems have al­ready been fixed.

The pro­ject states that “security is one of the pri­mary pro­ject goals”, and the de­vel­op­ers do ap­pear to have put some thought into the is­sue. Encryption is used in the right places, cer­tifi­cates are ver­i­fied, etc. A quick CVE search turns up two en­tries over the last four years, one

of which en­abled the over­writ­ing of ar­bi­trary files. Exploiting that vul­ner­a­bil­ity would re­quire first gain­ing con­trol of one of the ma­chines in the shar­ing net­work, at which point the bat­tle is likely lost any­way. It does not seem that any sort of for­mal se­cu­rity au­dit has been done, but the Syncthing de­vel­op­ers are at least mak­ing the right kinds of noises.

With re­gard to re­li­a­bil­ity, it is not hard to search for (and find) var­i­ous

scary

sto­ries

from users who have lost data with Syncthing. It seems that many of those prob­lems are the re­sult of op­er­a­tor er­ror; if you set up a sys­tem and al­low it to delete all your data, it may even­tu­ally con­clude that you want it to do ex­actly that. Synchronization can be amaz­ingly ef­fi­cient at prop­a­gat­ing mis­takes. Use of ver­sion­ing can help, as can avoid­ing the use of two-way syn­chro­niza­tion when­ever pos­si­ble. Syncthing does­n’t seem

like it has a lot of data-los­ing bugs, but back­ups are al­ways a good idea.

Syncthing has been sync­ing things since at least 2013, when the first com­mit ap­pears in its Git repos­i­tory; LWN looked at it in 2014. The pro­ject is writ­ten mostly in Go, and is dis­trib­uted un­der the Mozilla Public License. The cur­rent Syncthing re­lease is 1.18.0; it came out on July 6 — while this ar­ti­cle was be­ing writ­ten. The pro­ject shows a nearly monthly re­lease ca­dence in the last year; 1.7.0

was re­leased on July 7, 2020. There have been 728 non-merge com­mits to the Syncthing repos­i­tory over the last year from 40 developers; the top three de­vel­op­ers (Simon Frei, Jakob Borg, and Jesse Lucas) ac­count for just over 76% of of those com­mits. The pro­ject is thus not swarm­ing with de­vel­op­ers, but it ap­pears healthy enough for now.

A com­pany called Kastelo of­fers sup­port sub­scrip­tions for Syncthing and pro­vides sig­nif­i­cant re­sources for Syncthing de­vel­op­ment. The com­pany also is part of the Syncthing Foundation which, in turn, man­ages the pro­jec­t’s in­fra­struc­ture and makes grants for de­vel­op­ment pro­jects.

All told, Syncthing leaves a fa­vor­able im­pres­sion. The de­vel­op­ers seem to have done the work to cre­ate a sys­tem that is ca­pa­ble, re­li­able, se­cure, and which per­forms rea­son­ably well. But they have also done the work to make it all easy to set up and make use of — the place where a lot of free-soft­ware pro­jects seem to fall down. It is an ap­peal­ing tool for any­body want­ing to take con­trol of their data syn­chro­niza­tion and repli­ca­tion needs.

(Log in to post com­ments)

As a young man, I wanted to help make a new medium that would be a step for­ward from Gutenberg’s in­ven­tion hun­dreds of years be­fore.

By build­ing a Library of Everything in the dig­i­tal age, I thought the op­por­tu­nity was not just to make it avail­able to every­body in the world, but to make it bet­ter–smarter than pa­per. By us­ing com­put­ers, we could make the Library not just search­able, but or­ga­ni­z­able; make it so that you could nav­i­gate your way through mil­lions, and maybe even­tu­ally bil­lions of web pages.

The first step was to make com­put­ers that worked for large col­lec­tions of rich me­dia. The next was to cre­ate a net­work that could tap into com­put­ers all over the world: the Arpanet that be­came the Internet. Next came aug­mented in­tel­li­gence, which came to be called search en­gines. I then helped build WAIS–Wide Area Information Server–that helped pub­lish­ers get on­line to an­chor this new and open sys­tem, which came to be en­veloped by the World Wide Web.

By 1996, it was time to start build­ing the li­brary.

This li­brary would have all the pub­lished works of hu­mankind. This li­brary would be avail­able not only to those who could pay the $1 per minute that LexusNexus charged, or only at the most elite uni­ver­si­ties. This would be a li­brary avail­able to any­body, any­where in the world. Could we take the role of a li­brary a step fur­ther, so that every­one’s writ­ings could be in­cluded–not only those with a New York book con­tract? Could we build a mul­ti­me­dia archive that con­tains not only writ­ings, but also songs, recipes, games, and videos? Could we make it pos­si­ble for any­one to learn about their grand­mother in a hun­dred years’ time?

From the be­gin­ning, the Internet Archive had to be a non­profit be­cause it con­tains every­body else’s things. Its mo­tives had to be trans­par­ent. It had to last a long time.

In Silicon Valley, the goal is to find a prof­itable exit, ei­ther through ac­qui­si­tion or IPO, and go off to do your next thing. That was never my goal. The goal of the Internet Archive is to cre­ate a per­ma­nent mem­ory for the Web that can be lever­aged to make a new Global Mind. To find pat­terns in the data over time that would pro­vide us with new in­sights, well be­yond what you could do with a search en­gine.  To be not only a his­tor­i­cal ref­er­ence but a liv­ing part of the pulse of the Internet.

My fa­vorite things from the early era of the Web were the dream­ers.

In the early Web, we saw peo­ple try­ing to make a more de­mo­c­ra­tic sys­tem work. People tried to make pub­lish­ing more in­clu­sive.

We also saw the other parts of hu­man­ity: the pornog­ra­phers, the scam­mers, the spam­mers, and the trolls. They, too, saw the op­por­tu­nity to re­al­ize their dreams in this new world. At the end of the day, the Internet and the World Wide Web–it’s just us. It’s just a his­tory of hu­mankind. And it has been an ex­per­i­ment in shar­ing and open­ness.

The World Wide Web at its best is a mech­a­nism for peo­ple to share what they know, al­most al­ways for free, and to find one’s com­mu­nity no mat­ter where you are in the world.

Over the next 25 years, we have a very dif­fer­ent chal­lenge. It’s solv­ing some of the big prob­lems with the Internet that we’re see­ing now. Will this be our medium or will it be theirs? Will it be for a small con­trol­ling set of or­ga­ni­za­tions or will it be a com­mon good, a pub­lic re­source?

So many of us trust the Web to find recipes, how to re­pair your lawn­mower, where to buy new shoes, who to date. Trust is per­haps the most valu­able as­set we have, and squan­der­ing that trust will be a global dis­as­ter.

We may not have achieved Universal Access to All Knowledge yet, but we still can.

In an­other 25 years, we can have writ­ings from not a hun­dred mil­lion peo­ple, but from a bil­lion peo­ple, pre­served for­ever. We can have com­pen­sa­tion sys­tems that aren’t dri­ven by ad­ver­tis­ing mod­els that en­rich only a few.

We can have a world with many win­ners, with peo­ple par­tic­i­pat­ing, find­ing com­mu­ni­ties of like-minded peo­ple they can learn from all over the world.  We can cre­ate an Internet where we feel in con­trol.

I be­lieve we can build this fu­ture to­gether. You have al­ready helped the Internet Archive build this fu­ture. Over the last 25 years, we’ve amassed bil­lions of pages, 70 petabytes of data to of­fer to the next gen­er­a­tion. Let’s of­fer it to them in new and ex­cit­ing ways. Let’s be the builders and dream­ers of the next twenty-five years.

See a time­line of Key Moments in Access to Knowledge, videos & an in­vi­ta­tion to our 25th Anniversary Virtual Celebration at an­niver­sary.archive.org.

I’ve spent nearly twenty years blog­ging, giv­ing talks and re­leas­ing open source code. It’s been fan­tas­tic for my ca­reer, and a huge amount of work. But here’s a use­ful se­cret: you don’t have to put very much work at all into pub­lic cre­ativ­ity in or­der to stand out as a job can­di­date.

I’ve in­ter­viewed hun­dreds of peo­ple, and screened hun­dreds more re­sumes—mostly for tech roles in San Francisco, an ex­tremely com­pet­i­tive job mar­ket.

The vast ma­jor­ity of can­di­dates have lit­tle to no ev­i­dence of cre­ativ­ity in pub­lic at all. The same is true for many of the best en­gi­neers I have worked with.

As a hir­ing man­ager, this means you have to learn how to source can­di­dates and in­ter­view ef­fec­tively: you don’t want to miss out on a great en­gi­neer just be­cause they spent all of their en­ergy mak­ing great prod­ucts for prior em­ploy­ers rather than blog­ging, speak­ing and cod­ing in pub­lic.

But as a can­di­date, this means you can give your­self a big ad­van­tage in terms of stand­ing out from the crowd with a rel­a­tively small amount of work.

Start a blog. Post an in­ter­est­ing tech­ni­cal ar­ti­cle to it once or twice a year—some­thing you’ve learned, or a bug you’ve fixed, or a prob­lem you’ve solved. After a few years stop both­er­ing en­tirely, but leave the blog on­line some­where.

Build a small per­sonal pro­ject and put the code on GitHub. Accompany it with a README with a de­tailed de­scrip­tion of the pro­ject and screen­shots of it in ac­tion—al­most no-one does this, it only takes a few hours ex­tra and it mas­sively in­creases the im­pact your pro­ject will have on hir­ing man­agers who are check­ing you out.

That’s it. One or two blog posts. Maybe a GitHub repos­i­tory. Believe it or not, if you are up against a bunch of other can­di­dates (especially ear­lier on in your ca­reer) they likely won’t have any­thing like that. You will jump straight to the top of the hir­ing man­ager’s men­tal list, maybe with­out them even notic­ing.

There’s plenty more you can do if you want to put the ef­fort in: build an au­di­ence on Twitter, start a newslet­ter, make videos, give talks (ideally that get recorded and pub­lished on­line), re­lease open source pack­ages, pub­lish TILs—but hon­estly if your goal is to get through the in­ter­view process more eas­ily you will very quickly hit the law of di­min­ish­ing re­turns.

If you’re go­ing to do that stuff do it be­cause you want to de­velop those skills and share with and learn from the world—don’t just do it be­cause you think it’s a crit­i­cal path to be­ing hired.

This post started out as a Twitter thread:

If you want to stand out from other can­di­dates, hav­ing even one piece of writ­ing or pub­lished piece of code that shows some­thing you’ve built is a great way to do that https://​t.co/​QfYEWx­fIet- Simon Willison (@simonw) July 17, 2021

Today, I have some ex­cit­ing news that I wanted to share with the broader HashiCorp com­mu­nity: I am tran­si­tion­ing into a new role as a full-time in­di­vid­ual con­trib­u­tor and off of the HashiCorp ex­ec­u­tive team.

I founded HashiCorp in 2012 and served as CEO un­til 2016, and then be­came one of our CTOs un­til to­day. After nearly 10 years of learn­ing and grow­ing as an ex­ec­u­tive, I’m ready and ex­cited to step back into a full-time en­gi­neer­ing po­si­tion and look for­ward to mak­ing a mean­ing­ful im­pact with this new role. This change has been years in plan­ning and is now pos­si­ble thanks to HashiCorp’s ma­tu­rity and the ex­cel­lent lead­er­ship team we now have in place.

I feel the best way to ex­plain more de­tails about this change is to sim­ply share the email I wrote to our em­ploy­ees, so I’ve in­cluded that be­low. I look for­ward to con­tin­u­ing to talk with you all as part of our com­mu­nity as I pur­sue my new role.

Here’s my email to our em­ploy­ees:

Beginning to­day, I’ll be tran­si­tion­ing into a new role here at HashiCorp as an in­di­vid­ual con­trib­u­tor fo­cused on en­gi­neer­ing. This is some­thing I’m very ex­cited about and that the ex­ec­u­tive team and I have planned for a long time. I’d like to take the op­por­tu­nity in this email to fill in the back­ground about this change and de­scribe what this means for me and the com­pany go­ing for­ward.

Ever since found­ing HashiCorp, I’ve felt it’s im­por­tant to build a com­pany where I’m not re­quired for day-to-day op­er­a­tions and where other lead­ers can step in at the right phase. I be­lieve this is nec­es­sary for a com­pany to be re­silient and long-last­ing. This mind­set could be seen as early as 2016 when Armon [Dadgar] and I proac­tively sought out a CEO to suc­ceed us and lead the com­pany for­ward. In our search we found Dave [McJannet] and he was, and con­tin­ues to be, the right leader for the next phases of the com­pany. At that time, I stepped into a new role as CTO, one that has been bet­ter suited to me for the last few years.

There are also per­sonal el­e­ments to this de­ci­sion. I founded HashiCorp as an en­gi­neer pas­sion­ate about in­fra­struc­ture tool­ing. But as a founder, my role at times has had to ex­pand well be­yond and away from that. That’s the price of be­ing a founder: you do what­ever is nec­es­sary of you, even if there are parts of the role that don’t par­tic­u­larly mo­ti­vate you. And over the course of nearly a decade build­ing HashiCorp into a multi-bil­lion dol­lar com­pany, I’ve con­tin­u­ously reaf­firmed that I’m still an en­gi­neer at heart and I’m ready to more of­fi­cially get back to fo­cus­ing on that.

HashiCorp is once again at a place where I feel the com­pany and I are ready for me to step into a new role. The HashiCorp lead­er­ship team is strong and I trust them com­pletely. The busi­ness is healthy and I have com­plete op­ti­mism about the fu­ture.

In my new role, I will be an in­di­vid­ual con­trib­u­tor ei­ther on spe­cific prod­uct teams or ex­plor­ing other ideas within HashiCorp. I will con­tinue to par­tic­i­pate with lead­er­ship in cer­tain big-pic­ture plan­ning such as ma­jor prod­uct plans, mes­sag­ing for HashiConf keynotes, and other strate­gic de­ci­sions. I’ll con­tinue to be a full-time HashiCorp em­ployee. However, as I step into this role, I will of­fi­cially no longer be a mem­ber of the ex­ec­u­tive team and will there­fore no longer have ac­cess to ex­ec­u­tive meet­ings, plans, or de­ci­sions.

Coinciding with my role change, I am also step­ping down from the HashiCorp board. While we’ve de­cided to match the tim­ing, I’d like to stress that this was a de­ci­sion we made years ago and is not re­lated at all to my chang­ing role. As a startup ma­tures into a later stage, it is ex­pected for the board to have more in­de­pen­dent mem­bers and it is typ­i­cal for only one founder to be on a late-stage board along­side the CEO. I fully trust Armon and Dave to rep­re­sent the em­ploy­ees, my­self in­cluded.

While I will no longer be on the ex­ec­u­tive team or board, I will re­main a pas­sion­ate and ac­tive HashiCorp em­ployee and en­gi­neer­ing leader. I’m very ex­cited to be able to ded­i­cate more of my time to­wards prod­uct and en­gi­neer­ing chal­lenges and to spend more of my day-to-day time work­ing di­rectly along­side mem­bers in those or­ga­ni­za­tions. My ex­pe­ri­ence, voice, and ex­per­tise re­main fully avail­able to every­one in the com­pany.

I’m in­cred­i­bly proud that as an ex­ec­u­tive, I helped HashiCorp grow from noth­ing to nearly 1,500 em­ploy­ees with a val­u­a­tion of over $5 bil­lion. Looking for­ward, the com­pany has a huge op­por­tu­nity, and I’m ex­cited to con­tinue work­ing with you all to build and grow the com­pany and make an im­pact in my new role.

We’d Rather Have the Iceberg Than the Ship

I re­cently bought a house in Madison, Wisconsin, but be­fore I did I went through a se­ries of ad­ven­tures in dif­fer­ent parts of the state ex­plor­ing the prop­erty mar­ket. I was ini­tially in­ter­ested in Wisconsin be­cause I have ex­tended fam­ily there and they en­cour­aged me to look at Appleton. They de­scribed it as a safe, pros­per­ous, and grow­ing com­mu­nity pop­u­lated by good peo­ple. There’s a his­toric down­town, Lawrence University, the med­ical cen­ter, and it’s the county seat. As proof of its bright fu­ture they also men­tioned Amazon is build­ing a gi­ant ful­fill­ment cen­ter out by the air­port. I found a real es­tate agent, de­scribed what I was look­ing for, and she be­gan send­ing me links to avail­able prop­er­ties. What I re­quested was a build­ing with a bit of ar­chi­tec­tural char­ac­ter in a walk­a­ble neigh­bor­hood near civic ameni­ties, quite pos­si­bly within a bi­cy­cle ride of the uni­ver­sity, pub­lic parks, restau­rants, and shops. I asked for a du­plex, think­ing I would be buy­ing a place that I would co-own with fam­ily mem­bers. They could have their half, I could have mine. Win win.These are the places she sug­gested in­stead. I looked at the fil­ter she had set on the list­ings and she had elim­i­nated any prop­erty that was built be­fore 1990. That guar­an­teed the homes on of­fer would all be out on the sub­ur­ban fringe along the high­ways in­stead of in town. When I asked her about this she was con­fused and set about ed­u­cat­ing me on the prop­erty mar­ket in Appleton.She ex­plained that you don’t want an older home be­cause they have too much de­ferred main­te­nance. Bringing them up to a mod­ern stan­dard is too ex­pen­sive rel­a­tive to their re­sale value. Taxes are too high in old neigh­bor­hoods so you want to buy across mu­nic­i­pal lines out­side the older city lim­its. You get more house for less money with lower taxes in the newer de­vel­op­ments. The schools are much bet­ter in the newer ar­eas, and peo­ple shop for school dis­tricts more than they shop for a house it­self. While Appleton is a very safe lit­tle city, crime is al­ways a big­ger prob­lem at the core com­pared to the edges. You have to think of the chil­dren. And a newer home on a larger lot is a bet­ter in­vest­ment be­cause that’s what qual­ity buy­ers want. Older homes don’t ap­pre­ci­ate, they de­cline.What she was telling me was no doubt true from her per­spec­tive. She re­flected the val­ues not just of Appleton, but most of America and the peo­ple who choose to live in these places. She was­n’t wrong. But aside from the fact that I did­n’t care for any of these homes and was never go­ing to buy in these lo­ca­tions, I re­al­ized the truth of the Appleton model. Thirty years from now all the new homes she’s sell­ing will slip into the “old” cat­e­gory and will grad­u­ally fes­ter as taxes rise and the mid­dle class mi­grates to new green­field de­vel­op­ments. These older places (the homes be­ing built to­day) will then be pop­u­lated by lower class peo­ple with fewer re­sources and less sta­tus thereby re­in­forc­ing the per­cep­tion that it’s best to move on if at all pos­si­ble. These are fun­gi­ble, for­get­table, dis­pos­able places that rapidly age and are then left to qui­etly de­cay. There’s also the prob­lem of ag­ing com­mer­cial real es­tate. Over the last few gen­er­a­tions we’ve wit­nessed the whole­sale aban­don­ment of Main Street in fa­vor of strip cen­ters, big box stores, and re­gional shop­ping malls. Now these re­tail cen­ters are get­ting old and are be­ing squeezed hard by on­line com­merce and home de­liv­ery. The Amazon ful­fill­ment ware­house that’s so ag­gres­sively sub­si­dized by lo­cal gov­ern­ment and her­alded as “growth” out on the edge of the metro­plex is ef­fec­tively re­plac­ing the dried husk of the Fox River Mall and var­i­ous lesser shop­ping plazas. This is­n’t an ad­di­tive process. It’s slash and burn ur­ban­ism and it does­n’t end well.The “good news” is these places were built fast and cheap and were never meant to last any­way. They have a con­struc­tion lifes­pan ex­quis­itely tuned to match the amor­ti­za­tion and de­pre­ci­a­tion sched­ules of the fi­nan­cial un­der­writ­ers. As soon as they stop spin­ning off cash flow and tax ad­van­tages to dis­tant in­vestors they’re toast. Then they be­come some­one else’s prob­lem. If the value of the dirt is high enough some­one will re­de­velop the prop­erty into some­thing dif­fer­ent, but only with the ap­pro­pri­ate sweet­en­ers from “business friendly” lo­cal gov­ern­ment, mean­ing heavy sub­si­dies and tax hol­i­days. These deals tend to be a hostage ne­go­ti­a­tion.Within the con­tin­u­ous greater Appleton re­gion for ten miles in any di­rec­tion are dozens of smaller mu­nic­i­pal­i­ties that have been en­gulfed by sub­ur­ban de­vel­op­ment for most of a cen­tury. Little ves­tiges of once pros­per­ous towns linger in di­min­ished form em­bed­ded in the sprawl. These towns were built along barge canals, small hy­dro­elec­tric dams, and pro­duc­tive mills. Many of the sur­viv­ing build­ings have been par­tially oc­cu­pied or com­pletely va­cant for decades. Half the orig­i­nal struc­tures were so de­val­ued that they were torn down and re­placed with sur­face park­ing lots. A num­ber of them are used as low grade stor­age fa­cil­i­ties or bud­get con­sign­ment shops.Out of cu­rios­ity I looked at how much one of these build­ings costs. This win­ter this place with a de­funct Chinese restau­rant on the ground floor and a va­cant two bed­room apart­ment up­stairs was of­fered at $85,000. The cost of the restau­rant equip­ment alone, the in­dus­trial stoves, deep fry­ers, walk-in re­frig­er­a­tors, stain­less steel work ta­bles, and fire sup­press­ing vent hoods would have been close to that price. The build­ing had been on the mar­ket for a very long time with no bids. It even­tu­ally sold a few months ago for $65,000. For com­par­i­son, here’s a re­view of a $65,000 lux­ury 2021 Ram pickup truck.Some­one re­moved the restau­rant in­te­rior and opened a col­lectibles shop that sells base­ball cards. This shop ac­ti­vates the store­front and keeps the place warm so I’m all for it. But un­less this en­ter­prise does most of its sales via the in­ter­webs it’s go­ing to strug­gle to sur­vive. Fortunately the over­head is shock­ingly low. It will all keep tick­ing along un­til the build­ing needs a big ticket item like a struc­tural re­pair to a fail­ing brick wall or cracked foun­da­tion. Then it starts down the road to a park­ing lot con­ver­sion. Back when this town was at its peak a cen­tury ago each of these va­cant parcels had mul­ti­ple build­ings on them. They’re all gone now. I called the lo­cal busi­ness im­prove­ment dis­trict of­fice as well as the town plan­ner and chat­ted with peo­ple in var­i­ous de­part­ments. They gave the usual sales pitch about how they’re cre­at­ing a vi­brant re­tail Main Street with his­toric char­ac­ter. They men­tioned the new dec­o­ra­tive lamp posts, flower beds, fes­tive flags, and sea­sonal cel­e­bra­tions. “Did you see our ea­gle sculp­ture?” I got a flash­back to Colerain Township, Ohio. Blighted build­ings had been re­placed with a new fire sta­tion, mid­dle school, city of­fices, and a new po­lice sta­tion. And there are plans for in­fill de­vel­op­ment once fi­nanc­ing and enough va­cant parcels are ag­gre­gated to ac­com­mo­date mod­ern build­ings. (AKA Texas Doughnuts.) My sense is that they’re jump­ing on the city plan­ning fash­ion band­wagon twenty years too late.What I see for the fu­ture of this spot is a vari­a­tion on things I’ve seen else­where around the coun­try. A sliver of Ye Olde Towne is pre­served for sen­ti­men­tal rea­sons and turned into a de facto themed strip mall. In this lo­ca­tion, with the pre­vail­ing cul­ture and econ­omy… I don’t see this be­ing much more than a mediocre col­lec­tion of cos­metic patches that will limp along half assed un­til the next fad ar­rives in an­other twenty years. It’s not a ter­ri­ble sit­u­a­tion, but it’s not “transformative” or “catalytic” ei­ther. I gave these of­fi­cials my usual blunt as­sess­ment. They did­n’t like my tone.So here’s the big pic­ture. All of America’s in­sti­tu­tions are fo­cused ex­clu­sively on churn. Crank out new stuff, sell it fast, cash out, and move on to the next pro­ject. Blighted neigh­bor­hoods aren’t an ac­ci­dent. They’re baked in to every facet of how we do every­thing. Successful in­di­vid­u­als and savvy in­vestors know this in­stinc­tively and keep mov­ing every five or ten years to the next new bet­ter place. This is also true of mu­nic­i­pal of­fi­cials and pri­vate con­sul­tants who con­tin­u­ally hop­scotch from job to job leav­ing be­hind dis­tricts that have peaked in fa­vor of ones that are still grow­ing. I’d love for peo­ple to stop pre­tend­ing oth­er­wise and just be hon­est about the sit­u­a­tion. You get a re­ally good run for a few decades. Then things slowly turn to crap as the vinyl sid­ing and syn­thetic stucco start to peel off. We’re go­ing to con­tinue to do this un­til we sim­ply can’t any­more for one rea­son or an­other. Then we’ll have no choice but to start re-in­hab­it­ing the dregs that were left be­hind. Some places will be more wor­thy of sal­va­tion than oth­ers. Shrug.I’m re­minded of Elizabeth Bishop’s poem “The Imaginary Iceberg” which be­gins, “We’d rather have the ice­berg than the ship, al­though it meant the end of travel.” That’s how I feel about the North American de­vel­op­ment pat­tern. It’s ephemeral. In the long arc of his­tory no one will miss any of these places. Future gen­er­a­tions will be busy do­ing en­tirely dif­fer­ent things with the land­scape they’ll in­herit. I’ve made my peace with that re­al­ity.

P{{ padded­PageStr }} 

Hacker News in the style of Teletext 

Type the num­ber of the page to go to 

The Zip file for­mat is now 32 years old. You’d think be­ing 32 years old the for­mat would be well doc­u­mented. Unfortunately it’s not.

I have a feel­ing this is like many file for­mats. They aren’t de­signed, rather the de­vel­oper just makes it up as they go. If it gets pop­u­lar other peo­ple want to read and/​or write them. They ei­ther try to re­verse en­gi­neer the for­mat OR they ask for specs. Even if the de­vel­oper writes specs they of­ten for­get all the as­sump­tions their orig­i­nal pro­gram makes. Those are not writ­ten down and hence the spec is in­com­plete. Zip is such a for­mat.

Zip claims its for­mat is doc­u­mented in a file called APPNOTE. TXT which can be found here.

The short ver­sion is, a zip file con­sists of records, each record starts with some 4 byte marker that gen­er­ally takes the form

0x50, 0x4B, ??, ??

Where the 0x50, 0x4B are the let­ters PK stand­ing for “Phil Katz”, the per­son who made the zip for­mat. The two ?? are bytes that iden­tify the type of the record. Examples

0x50 0x4b 0x03 0x04 // a lo­cal file record

0x50 0x4b 0x01 0x02 // a cen­tral di­rec­tory file record

0x50 0x4b 0x06 0x06 // an end of cen­tral di­rec­tory record

Records do NOT fol­low any stan­dard pat­tern. To read or even skip a record you must know its for­mat. What I mean is there are sev­eral other for­mats that fol­low some con­ven­tion like each record id is fol­lowed by the length of the record. So, if you see an id, and you don’t un­der­stand it, you just read the length, skip that many bytes (*), and you’ll be at the next id. Examples of this type in­clude most video con­tainer for­mats, jpgs, tiff, pho­to­shop files, wav files, and many oth­ers.

(*) some for­mats re­quire round­ing the length up to the near­est mul­ti­ple of 4 or 16.

Zip does NOT do this. If you see an id and you don’t know how that type of record’s con­tent is struc­tured there is no way to know how many bytes to skip.

APPNOTE. TXT says the fol­low­ing things

4.1.9 ZIP files MAY be streamed, split into seg­ments (on fixed or on re­mov­able me­dia) or “self-extracting”. Self-extracting ZIP files MUST in­clude ex­trac­tion code for a tar­get plat­form within the ZIP file.

4.3.1 A ZIP file MUST con­tain an “end of cen­tral di­rec­tory record”. A ZIP file con­tain­ing only an “end of cen­tral di­rec­tory record” is con­sid­ered an empty ZIP file. Files MAY be added or re­placed within a ZIP file, or deleted. A ZIP file MUST have only one “end of cen­tral di­rec­tory record”. Other records de­fined in this spec­i­fi­ca­tion MAY be used as needed to sup­port stor­age re­quire­ments for in­di­vid­ual ZIP files.

4.3.2 Each file placed into a ZIP file MUST be pre­ceded by a “local file header” record for that file. Each “local file header” MUST be ac­com­pa­nied by a cor­re­spond­ing “central di­rec­tory header” record within the cen­tral di­rec­tory sec­tion of the ZIP file.

4.3.3 Files MAY be stored in ar­bi­trary or­der within a ZIP file. A ZIP file MAY span mul­ti­ple vol­umes or it MAY be split into user-de­fined seg­ment sizes. All val­ues MUST be stored in lit­tle-en­dian byte or­der un­less oth­er­wise spec­i­fied in this doc­u­ment for a spe­cific data el­e­ment.

[local file header 1]

[encryption header 1]

[file data 1]

[data de­scrip­tor 1]

[local file header n]

[encryption header n]

[file data n]

[data de­scrip­tor n]

[archive de­cryp­tion header]

[archive ex­tra data record]

[central di­rec­tory header 1]

[central di­rec­tory header n]

[zip64 end of cen­tral di­rec­tory record]

[zip64 end of cen­tral di­rec­tory lo­ca­tor]

[end of cen­tral di­rec­tory record]

lo­cal file header sig­na­ture 4 bytes (0x04034b50)

ver­sion needed to ex­tract 2 bytes

gen­eral pur­pose bit flag 2 bytes

com­pres­sion method 2 bytes

last mod file time 2 bytes

last mod file date 2 bytes

crc-32 4 bytes

com­pressed size 4 bytes

un­com­pressed size 4 bytes

file name length 2 bytes

ex­tra field length 2 bytes

file name (variable size)

ex­tra field (variable size)

Immediately fol­low­ing the lo­cal header for a file SHOULD be placed the com­pressed or stored data for the file. If the file is en­crypted, the en­cryp­tion header for the file SHOULD be placed af­ter the lo­cal header and be­fore the file data. The se­ries of [local file header][en­cryp­tion header] [file data][data de­scrip­tor] re­peats for each file in the  .ZIP archive.

Zero-byte files, di­rec­to­ries, and other file types that con­tain no con­tent MUST NOT in­clude file data.

[central di­rec­tory header 1]

[central di­rec­tory header n]

[digital sig­na­ture]

cen­tral file header sig­na­ture 4 bytes (0x02014b50)

ver­sion made by 2 bytes

ver­sion needed to ex­tract 2 bytes

gen­eral pur­pose bit flag 2 bytes

com­pres­sion method 2 bytes

last mod file time 2 bytes

last mod file date 2 bytes

crc-32 4 bytes

com­pressed size 4 bytes

un­com­pressed size 4 bytes

file name length 2 bytes

ex­tra field length 2 bytes

file com­ment length 2 bytes

disk num­ber start 2 bytes

in­ter­nal file at­trib­utes 2 bytes

ex­ter­nal file at­trib­utes 4 bytes

rel­a­tive off­set of lo­cal header 4 bytes

file name (variable size)

ex­tra field (variable size)

file com­ment (variable size)

end of cen­tral dir sig­na­ture 4 bytes (0x06054b50)

num­ber of this disk 2 bytes

num­ber of the disk with the

start of the cen­tral di­rec­tory 2 bytes

to­tal num­ber of en­tries in the

cen­tral di­rec­tory on this disk 2 bytes

to­tal num­ber of en­tries in

the cen­tral di­rec­tory 2 bytes

size of the cen­tral di­rec­tory 4 bytes

off­set of start of cen­tral

di­rec­tory with re­spect to

the start­ing disk num­ber 4 bytes

.ZIP file com­ment length 2 bytes

.ZIP file com­ment (variable size)

There are other de­tails in­volv­ing en­cryp­tion, larger files, op­tional data, but for the pur­poses of this post this is all we need. We need one more piece of info, how to make a self ex­tract­ing archive.

To do so we could look back to ZIP2EXE.exe which shipped with pkzip in 1989 and see what it does but it’s eas­ier look at Info-Zip to see what hap­pens.

How do I make a DOS (or other non-na­tive) self-ex­tract­ing archive un­der Unix?

The pro­ce­dure is ba­si­cally de­scribed in the UnZipSFX man page. First grab the ap­pro­pri­ate UnZip bi­nary dis­tri­b­u­tion for your tar­get plat­form (DOS, Windows, OS/2, etc.), as de­scribed above; we’ll as­sume DOS in the fol­low­ing ex­am­ple. Then ex­tract the UnZipSFX stub from the dis­tri­b­u­tion and prepend as if it were a na­tive Unix stub:

> un­zip un­z552x3.exe un­zipsfx.exe // ex­tract the DOS SFX stub

> cat un­zipsfx.exe yourzip.zip > your­DOSzip.exe // cre­ate the SFX archive

> zip -A your­DOSzip.exe // fix up in­ter­nal off­sets

> That’s it. You can still test, up­date and delete en­tries from the archive; it’s a fully func­tional zip­file.

So given all of that let’s go over some prob­lems.

This is un­de­fined by the spec.

Scan from the front, when you see an id for a record do the ap­pro­pri­ate thing.

Scan from the back, find the end-of-cen­tral-di­rec­tory-record and then use it to read through the cen­tral di­rec­tory, only look­ing at things the cen­tral di­rec­tory ref­er­ences.

Scanning from the back is how the orig­i­nal pkun­zip works. For one it means if you ask for some sub­set of files it can jump di­rectly to the data you need in­stead of hav­ing to scan the en­tire zip file. This was es­pe­cially im­por­tant if the zip file spanned mul­ti­ple floppy disks.

But, 4.1.9 says you can stream zip files. How is that pos­si­ble? What if there is some lo­cal file record that is not ref­er­enced by the cen­tral di­rec­tory? Is that valid? This is un­de­fined.

Files MAY be added or re­placed within a ZIP file, or deleted.

Okay? That sug­gests the cen­tral di­rec­tory might not ref­er­ence all the files in the zip file be­cause oth­er­wise this state­ment about files be­ing added, re­placed, or delete has no point to be in the spec.