10 interesting stories served every morning and every evening.

1 1,282 shares, 65 trendiness, words and minutes reading time

The Framework Laptop is now shipping, and press reviews are up!

We’re ex­cited to see the first press re­views go live for the Framework Laptop and the first or­ders land on your doorsteps to­day! With the FTC unan­i­mously vot­ing to en­force the Right to Repair just yes­ter­day, our tim­ing could­n’t be bet­ter for de­liv­er­ing a great, high per­for­mance, easy to re­pair prod­uct. There is a ton of amaz­ing ma­te­r­ial to read and watch, with more com­ing in the next weeks. Some of our fa­vorite quotes so far are:

A poster child for the right-to-re­pair move­ment, Framework’s mod­u­lar lap­top is one of the smartest de­signs I’ve seen in a long time.”

It’s the ul­ti­mate Right to Repair lap­top.”

The Framework Laptop is more than just [a] worth­while ex­per­i­ment in mod­u­lar­ity, it’s also a great lap­top.”

Reviewers loved the free­dom to re­pair and up­grade, the Expansion Card sys­tem, CPU per­for­mance, key­board feel, we­b­cam qual­ity, and more.  Of course, in­side of Framework, we grav­i­tate to­wards the crit­i­cal feed­back that points us to where to do bet­ter.  We take every bit of feed­back se­ri­ously, and we want your thoughts as you start us­ing your Framework Laptop.  This lets us know where to fo­cus for fu­ture im­prove­ments, whether that is for firmware up­dates, mod­ules, or next prod­ucts.  A won­der­ful thing about our prod­uct phi­los­o­phy is that im­prove­ments can go into re­place­ment parts and up­grades that every ex­ist­ing user can pick up and swap to, rather than need­ing to wait around and pay for an en­tirely new prod­uct.

We’re grate­ful to each of you who have or­dered al­ready, and we’re look­ing for­ward to get­ting your Framework Laptop to you.  Batch 1 pre-or­ders for July de­liv­ery con­tinue to ship out from our ware­house each day.  We’ll start Batch 2 ship­ments for August de­liv­ery soon af­ter.  We have a small num­ber of Batch 2 Framework Laptop and Framework Laptop DIY Edition units cur­rently avail­able for sale, with just a fully re­fund­able $100 de­posit due to­day.  If you pre-or­der now, some of you will be able to re­ceive your or­der within 3-4 weeks.

As proud as we are of the Framework Laptop (and we’re ex­tremely proud!), the great­est thing we have cre­ated over the last 18 months is the team that built it.  It takes an in­cred­i­ble team to build an ex­cel­lent prod­uct this com­plex and de­liver it on time.  We’re hir­ing on all fronts to con­tinue de­vel­op­ing the Framework Laptop ecosys­tem and ini­ti­ate our next cat­e­gories.  Let us know if you know any­one who may be in­ter­ested in help­ing us build prod­ucts that are bet­ter for peo­ple and the planet.


Read the original on frame.work »

2 528 shares, 48 trendiness, words and minutes reading time

BirdNET – The easiest way to identify birds by sound.

How can com­put­ers learn to rec­og­nize birds from sounds? The Cornell Lab of Ornithology and the Chemnitz University of Technology are try­ing to find an an­swer to this ques­tion. Our re­search is mainly fo­cused on the de­tec­tion and clas­si­fi­ca­tion of avian sounds us­ing ma­chine learn­ing — we want to as­sist ex­perts and cit­i­zen sci­en­tist in their work of mon­i­tor­ing and pro­tect­ing our birds. BirdNET is a re­search plat­form that aims at rec­og­niz­ing birds by sound at scale. We sup­port var­i­ous hard­ware and op­er­at­ing sys­tems such as Arduino mi­cro­con­trollers, the Raspberry Pi, smart­phones, web browsers, work­sta­tion PCs, and even cloud ser­vices. BirdNET is a cit­i­zen sci­ence plat­form as well as an analy­sis soft­ware for ex­tremely large col­lec­tions of au­dio. BirdNET aims to pro­vide in­no­v­a­tive tools for con­ser­va­tion­ists, bi­ol­o­gists, and bird­ers alike.

This page fea­tures some of our pub­lic demon­stra­tions, in­clud­ing a live stream demo, a demo for the analy­sis of au­dio record­ings, an Android and iOS app, and its vi­su­al­iza­tion of sub­mis­sions. All demos are based on an ar­ti­fi­cial neural net­work we call BirdNET. We are con­stantly im­prov­ing the fea­tures and per­for­mance of our demos — please make sure to check back with us reg­u­larly.

We are cur­rently fea­tur­ing 984 of the most com­mon species of North America and Europe. We will add more species and more re­gions in the near fu­ture. Click here for the list of sup­ported species.


Read the original on birdnet.cornell.edu »

3 399 shares, 30 trendiness, words and minutes reading time

93% of Paint Splatters are Valid Perl Programs

TLDR: read the pa­per and view the gallery of pretty Perl pro­grams.

In this pa­per, we aim to an­swer a long-stand­ing open prob­lem in the pro­gram­ming lan­guages com­mu­nity: is it pos­si­ble to smear paint on the wall with­out cre­at­ing valid Perl?

We an­swer this ques­tion in the af­fir­ma­tive: it is pos­si­ble to smear paint on the wall with­out cre­at­ing a valid Perl pro­gram. We em­ploy an em­pir­i­cal ap­proach, us­ing op­ti­cal char­ac­ter recog­ni­tion (OCR) soft­ware, which finds that merely 93% of paint splat­ters parse as valid Perl. We an­a­lyze the prop­er­ties of paint-splat­ter Perl pro­grams, and pre­sent seven ex­am­ples of paint splat­ters which are not valid Perl pro­grams.

Accepted for pub­li­ca­tion at SIGBOVIK 2019, held April 1st 2019 in Pittsburgh. Winner of a Unwitting Participation Ribbon, an un­wel­come brand we’ve af­fixed to each pa­per de­ter­mined af­ter care­ful scrutiny to have in­cluded a gen­uine ar­ti­fact, thereby fur­ther­ing the ad­mirable causes of open sci­ence and fruit­ful pro­cras­ti­na­tion.”

Read it on Google Docs or down­load a PDF. Or grab the en­tire SIGBOVIK 2019 pro­ceed­ings; I’m on page 174.

Here’s all the paint splat­ters on a sin­gle page, along with the valid Perl source code cor­re­spond­ing to each. Not valid” is writ­ten in red for those im­ages which did not parse as valid Perl pro­grams. If dif­fer­ent OCR set­tings rec­og­nized mul­ti­ple valid Perl pro­grams, I chose the one that seemed the most interesting”, ac­cord­ing to my own aes­thetic sense.

Here’s a tar­ball of 100 paint-splat­ter im­ages that were used as the main dataset for this pa­per.

There are a few paint splat­ter Perl pro­grams that I did­n’t rec­og­nize as interesting” un­til af­ter the SIGBOVIK sub­mis­sion dead­line. For ex­am­ple, this splat­ter is rec­og­nized by OCR as the string lerz­fi­jglp­Fiji-j, which eval­u­ates to the num­ber 0 in Perl:

The im­age be­low is rec­og­nized as the string -*?, which also eval­u­ates to the num­ber 0 in Perl:

Another sur­pris­ing pro­gram is shown be­low; OCR rec­og­nizes this im­age as the string ;i;c;;#\\?z{;?;;fn’:.;, which eval­u­ates to the string c in Perl:

Finally, this im­age is rec­og­nized as the string ;E,‘__’, which eval­u­ates to the string E__ in Perl:


Read the original on www.mcmillen.dev »

4 358 shares, 38 trendiness, words and minutes reading time

Syncing all the things [LWN.net]

Computing de­vices are won­der­ful; they surely must be, since so many of us have so many of them. The pro­lif­er­a­tion of com­put­ers leads di­rectly to a fa­mil­iar prob­lem, though: the files we want are al­ways on the wrong ma­chine. One so­lu­tion is syn­chro­niza­tion ser­vices that keep a set of files up to date across a mul­ti­tude of ma­chines; a num­ber of com­pa­nies have cre­ated suc­cess­ful com­mer­cial of­fer­ings based on such ser­vices. Some of us, though, are stub­bornly re­sis­tant to the idea of plac­ing our data in the hands of cor­po­ra­tions and their pro­pri­etary sys­tems. For those of us who would rather stay in con­trol of our data, sys­tems like Syncthing of­fer a pos­si­ble so­lu­tion.

The core idea be­hind syn­chro­niza­tion sys­tems is es­sen­tially the same for all of them: given a list of di­rec­to­ries and a list of sys­tems, en­sure that those di­rec­to­ries have the same con­tents on each sys­tem. If a file is added on one, it is copied out to the rest; mod­i­fi­ca­tions and dele­tions are (usually) prop­a­gated as well. The trou­ble is al­ways in the de­tails, though; from fid­dly setup pro­ce­dures to data cor­rup­tion and se­cu­rity prob­lems, there are a lot of ways in which syn­chro­niza­tion can go wrong. So users have to put a lot of trust in these sys­tems; open source code is an im­por­tant step to­ward that goal, but it is also nec­es­sary to be­lieve that the de­vel­op­ers in­volved have thought care­fully through the is­sues.

When the north­ern-hemi­sphere sum­mer sets in and fresh news be­comes

rel­a­tively scarce, the op­por­tu­nity arises to fi­nally get around to

check­ing out an in­ter­est­ing soft­ware pro­ject or two.

Your ed­i­tor, thus, has been duly play­ing around with the Syncthing 1.17.0


ob­tained from the Fedora and CentOS (EPEL) repos­i­to­ries. Starting the

sys­tem on Fedora is just a mat­ter of run­ning the com­mand

to start the syn­chro­niza­tion dae­mon

and try­ing to not be dis­mayed at the vol­ume of log data gen­er­ated. The

EPEL pack­age, in­stead, ap­pears to in­stall Syncthing as a sys­temd user

ser­vice, mean­ing that all that was needed was to log into the sys­tem and

the dae­mon started au­to­mat­i­cally.

The dae­mon is man­aged through an in­ter­nal web server that shows up, by de­fault, on port 8384; see the ex­am­ple im­age on the right. There is ini­tially no au­then­ti­ca­tion re­quired; users will likely want to fix that as one of the first things they do. The web server is, by de­fault, only ac­ces­si­ble via the loop­back in­ter­face; a change to a con­fig­u­ra­tion file can make it avail­able to the Internet as a whole, but that sounds like a dar­ing thing to do even with au­then­ti­ca­tion en­abled. An al­ter­na­tive for gain­ing ac­cess to the web in­ter­face on a re­mote ma­chine, as sug­gested in the doc­u­men­ta­tion, is to set up an SSH tun­nel from the lo­cal sys­tem.

When it starts for the first time, Syncthing gen­er­ates a device ID iden­ti­fy­ing the lo­cal sys­tem; it looks like this:

Setting up syn­chro­niza­tion be­tween two ma­chines re­sem­bles the Bluetooth pair­ing process; it is done by pro­vid­ing each side with the de­vice ID be­long­ing to the other. Use of copy-and-paste is ad­vis­able here. Alternatively, if both sys­tems are on the same lo­cal net, they will dis­cover each other through broad­casts and ask (through the man­age­ment in­ter­face) whether a con­nec­tion should be es­tab­lished.

After a con­nec­tion be­tween sys­tems is made, users must tell Syncthing which di­rec­to­ries should be syn­chro­nized; that is a mat­ter of set­ting up fold­ers and shar­ing them with any or all of the known re­mote sys­tems (which Syncthing calls devices”). Once the share has been ac­cepted on the re­mote end, file changes will be prop­a­gated back and forth. When pos­si­ble, Syncthing re­quests file-change no­ti­fi­ca­tions from the ker­nel; that leads to rel­a­tively fast prop­a­ga­tion times.

There are a lot of op­tions that can be set to con­trol shar­ing. Sharing can be made one-way, for ex­am­ple, so that a par­tic­u­lar sys­tem might cre­ate files and send them out with­out ac­cept­ing changes from the other sys­tems. One es­pe­cially in­ter­est­ing (though new and beta”) fea­ture is the abil­ity to share

files to

spe­cific sys­tems in en­crypted form. If one sys­tem is, for ex­am­ple, a cloud server that is used pri­mar­ily for backup or dis­tri­b­u­tion pur­poses, it can be given en­crypted data that it can­not read. Any other sys­tem in the shar­ing net­work that has the cor­rect pass­word will be able to read those files, though. There are also var­i­ous ways of han­dling ver­sion­ing, which keeps older ver­sions of files around when one sys­tem changes them.

It’s worth not­ing that, while it is pos­si­ble to con­fig­ure a set of Syncthing clients all con­nected to a cen­tral server, noth­ing in Syncthing re­quires that sort of ar­chi­tec­ture. Systems can be con­nected in any way that seems to make sense. If a sys­tem finds that it needs files that have al­ready prop­a­gated to mul­ti­ple con­nected peers, it can re­ceive the needed data in blocks, BitTorrent-style, from whichever sys­tem can pro­vide it first.

Interestingly, nei­ther host names nor IP ad­dresses are in­volved in any stage of the con­fig­u­ra­tion process — by de­fault, at least; the sys­tems find each other based only on the de­vice ID re­gard­less of which net­works they are at­tached to. This, clearly, re­quires some third-party help. The Syncthing pro­ject runs a set of discovery” servers that will help sys­tems find each other based on their de­vice IDs. There is also a set of relay servers” that can re­lay data in sit­u­a­tions where the sys­tems in­volved can­not reach each other di­rectly — when they are both be­hind NAT fire­walls, for ex­am­ple.

Some thought has clearly gone into the se­cu­rity im­pli­ca­tions of this ar­chi­tec­ture. Data only goes through re­lay servers if there is no al­ter­na­tive, for ex­am­ple, and it is en­crypted at the end­points. But there is still some in­for­ma­tion that a hos­tile dis­cov­ery or re­lay server could ob­tain that might worry some users. For any­body who is truly wor­ried, the code for both types of server is avail­able; any­body can set up pri­vate servers and con­fig­ure their Syncthing in­stances to use only those.

According to the doc­u­men­ta­tion, de­vice IDs need not be kept se­cret, since an af­fir­ma­tive ac­tion is re­quired on both sides to set up a con­nec­tion. One might won­der whether an at­tacker might try to set up a sys­tem with a tar­get’s de­vice ID and thus gain ac­cess to the man­aged files. That ID, though, is es­sen­tially a pub­lic key, and the con­nec­tion process in­volves prov­ing pos­ses­sion of the as­so­ci­ated pri­vate key, so such an at­tack should not be pos­si­ble. This page

de­scribes de­vice IDs in more de­tail.

Perhaps the most com­mon use of syn­chro­niza­tion on to­day’s net is copy­ing pho­tos from a phone hand­set to a cen­tral server. Since Android phones, at least, are Linux-based, one need only set up a nor­mal shell en­vi­ron­ment on it and put Syncthing there to achieve this goal; the process should­n’t take more than a day or so. Or one could just in­stall the Android app, which is avail­able

on F-Droid and the Google Play Store as well. This app, shown on the right, comes with a folder for the cam­era (set for send-only shar­ing) con­fig­ured out of the box, so it is just a mat­ter of set­ting up the peers. And, lest one worry about typ­ing one of those de­vice IDs with an on-screen key­board, the app can read the QR code that the web in­ter­face will help­fully pro­vide, eas­ing that process con­sid­er­ably.

One slightly sur­pris­ing be­hav­ior is that the app asks for lo­ca­tion per­mis­sion, which does­n’t seem like some­thing it would need. That per­mis­sion is needed to de­ter­mine which WiFi net­work (if any) the phone is on, which is use­ful for the fea­ture con­fig­ur­ing when syn­chro­niza­tion should (and should not) be per­formed. Users of me­tered WiFi ser­vices may want to use this mech­a­nism to avoid syn­chro­niza­tion when it could cost them money. In the ab­sence of this per­mis­sion, the app will, by de­fault, per­form syn­chro­niza­tion when­ever it is con­nected to any WiFi net­work.

One need not look far to find com­plaints from users that the Android app drains the bat­tery quickly. Your ed­i­tor has not ob­served this be­hav­ior in a lim­ited amount of test­ing; it is pos­si­ble that the worst prob­lems have al­ready been fixed.

The pro­ject states that security is one of the pri­mary pro­ject goals”, and the de­vel­op­ers do ap­pear to have put some thought into the is­sue. Encryption is used in the right places, cer­tifi­cates are ver­i­fied, etc. A quick CVE search turns up two en­tries over the last four years, one

of which en­abled the over­writ­ing of ar­bi­trary files. Exploiting that vul­ner­a­bil­ity would re­quire first gain­ing con­trol of one of the ma­chines in the shar­ing net­work, at which point the bat­tle is likely lost any­way. It does not seem that any sort of for­mal se­cu­rity au­dit has been done, but the Syncthing de­vel­op­ers are at least mak­ing the right kinds of noises.

With re­gard to re­li­a­bil­ity, it is not hard to search for (and find) var­i­ous



from users who have lost data with Syncthing. It seems that many of those prob­lems are the re­sult of op­er­a­tor er­ror; if you set up a sys­tem and al­low it to delete all your data, it may even­tu­ally con­clude that you want it to do ex­actly that. Synchronization can be amaz­ingly ef­fi­cient at prop­a­gat­ing mis­takes. Use of ver­sion­ing can help, as can avoid­ing the use of two-way syn­chro­niza­tion when­ever pos­si­ble. Syncthing does­n’t seem

like it has a lot of data-los­ing bugs, but back­ups are al­ways a good idea.

Syncthing has been sync­ing things since at least 2013, when the first com­mit ap­pears in its Git repos­i­tory; LWN looked at it in 2014. The pro­ject is writ­ten mostly in Go, and is dis­trib­uted un­der the Mozilla Public License. The cur­rent Syncthing re­lease is 1.18.0; it came out on July 6 — while this ar­ti­cle was be­ing writ­ten. The pro­ject shows a nearly monthly re­lease ca­dence in the last year; 1.7.0

was re­leased on July 7, 2020. There have been 728 non-merge com­mits to the Syncthing repos­i­tory over the last year from 40 developers; the top three de­vel­op­ers (Simon Frei, Jakob Borg, and Jesse Lucas) ac­count for just over 76% of of those com­mits. The pro­ject is thus not swarm­ing with de­vel­op­ers, but it ap­pears healthy enough for now.

A com­pany called Kastelo of­fers sup­port sub­scrip­tions for Syncthing and pro­vides sig­nif­i­cant re­sources for Syncthing de­vel­op­ment. The com­pany also is part of the Syncthing Foundation which, in turn, man­ages the pro­jec­t’s in­fra­struc­ture and makes grants for de­vel­op­ment pro­jects.

All told, Syncthing leaves a fa­vor­able im­pres­sion. The de­vel­op­ers seem to have done the work to cre­ate a sys­tem that is ca­pa­ble, re­li­able, se­cure, and which per­forms rea­son­ably well. But they have also done the work to make it all easy to set up and make use of — the place where a lot of free-soft­ware pro­jects seem to fall down. It is an ap­peal­ing tool for any­body want­ing to take con­trol of their data syn­chro­niza­tion and repli­ca­tion needs.

(Log in to post com­ments)


Read the original on lwn.net »

5 352 shares, 0 trendiness, words and minutes reading time

Reflections as the Internet Archive turns 25

As a young man, I wanted to help make a new medium that would be a step for­ward from Gutenberg’s in­ven­tion hun­dreds of years be­fore.

By build­ing a Library of Everything in the dig­i­tal age, I thought the op­por­tu­nity was not just to make it avail­able to every­body in the world, but to make it bet­ter–smarter than pa­per. By us­ing com­put­ers, we could make the Library not just search­able, but or­ga­ni­z­able; make it so that you could nav­i­gate your way through mil­lions, and maybe even­tu­ally bil­lions of web pages.

The first step was to make com­put­ers that worked for large col­lec­tions of rich me­dia. The next was to cre­ate a net­work that could tap into com­put­ers all over the world: the Arpanet that be­came the Internet. Next came aug­mented in­tel­li­gence, which came to be called search en­gines. I then helped build WAIS–Wide Area Information Server–that helped pub­lish­ers get on­line to an­chor this new and open sys­tem, which came to be en­veloped by the World Wide Web.

By 1996, it was time to start build­ing the li­brary.

This li­brary would have all the pub­lished works of hu­mankind. This li­brary would be avail­able not only to those who could pay the $1 per minute that LexusNexus charged, or only at the most elite uni­ver­si­ties. This would be a li­brary avail­able to any­body, any­where in the world. Could we take the role of a li­brary a step fur­ther, so that every­one’s writ­ings could be in­cluded–not only those with a New York book con­tract? Could we build a mul­ti­me­dia archive that con­tains not only writ­ings, but also songs, recipes, games, and videos? Could we make it pos­si­ble for any­one to learn about their grand­mother in a hun­dred years’ time?

From the be­gin­ning, the Internet Archive had to be a non­profit be­cause it con­tains every­body else’s things. Its mo­tives had to be trans­par­ent. It had to last a long time.

In Silicon Valley, the goal is to find a prof­itable exit, ei­ther through ac­qui­si­tion or IPO, and go off to do your next thing. That was never my goal. The goal of the Internet Archive is to cre­ate a per­ma­nent mem­ory for the Web that can be lever­aged to make a new Global Mind. To find pat­terns in the data over time that would pro­vide us with new in­sights, well be­yond what you could do with a search en­gine.  To be not only a his­tor­i­cal ref­er­ence but a liv­ing part of the pulse of the Internet.

My fa­vorite things from the early era of the Web were the dream­ers.

In the early Web, we saw peo­ple try­ing to make a more de­mo­c­ra­tic sys­tem work. People tried to make pub­lish­ing more in­clu­sive.

We also saw the other parts of hu­man­ity: the pornog­ra­phers, the scam­mers, the spam­mers, and the trolls. They, too, saw the op­por­tu­nity to re­al­ize their dreams in this new world. At the end of the day, the Internet and the World Wide Web–it’s just us. It’s just a his­tory of hu­mankind. And it has been an ex­per­i­ment in shar­ing and open­ness.

The World Wide Web at its best is a mech­a­nism for peo­ple to share what they know, al­most al­ways for free, and to find one’s com­mu­nity no mat­ter where you are in the world.

Over the next 25 years, we have a very dif­fer­ent chal­lenge. It’s solv­ing some of the big prob­lems with the Internet that we’re see­ing now. Will this be our medium or will it be theirs? Will it be for a small con­trol­ling set of or­ga­ni­za­tions or will it be a com­mon good, a pub­lic re­source?

So many of us trust the Web to find recipes, how to re­pair your lawn­mower, where to buy new shoes, who to date. Trust is per­haps the most valu­able as­set we have, and squan­der­ing that trust will be a global dis­as­ter.

We may not have achieved Universal Access to All Knowledge yet, but we still can.

In an­other 25 years, we can have writ­ings from not a hun­dred mil­lion peo­ple, but from a bil­lion peo­ple, pre­served for­ever. We can have com­pen­sa­tion sys­tems that aren’t dri­ven by ad­ver­tis­ing mod­els that en­rich only a few.

We can have a world with many win­ners, with peo­ple par­tic­i­pat­ing, find­ing com­mu­ni­ties of like-minded peo­ple they can learn from all over the world.  We can cre­ate an Internet where we feel in con­trol.

I be­lieve we can build this fu­ture to­gether. You have al­ready helped the Internet Archive build this fu­ture. Over the last 25 years, we’ve amassed bil­lions of pages, 70 petabytes of data to of­fer to the next gen­er­a­tion. Let’s of­fer it to them in new and ex­cit­ing ways. Let’s be the builders and dream­ers of the next twenty-five years.

See a time­line of Key Moments in Access to Knowledge, videos & an in­vi­ta­tion to our 25th Anniversary Virtual Celebration at an­niver­sary.archive.org.


Read the original on blog.archive.org »

6 344 shares, 15 trendiness, words and minutes reading time

It doesn’t take much public creativity to stand out as a job candidate

I’ve spent nearly twenty years blog­ging, giv­ing talks and re­leas­ing open source code. It’s been fan­tas­tic for my ca­reer, and a huge amount of work. But here’s a use­ful se­cret: you don’t have to put very much work at all into pub­lic cre­ativ­ity in or­der to stand out as a job can­di­date.

I’ve in­ter­viewed hun­dreds of peo­ple, and screened hun­dreds more re­sumes—mostly for tech roles in San Francisco, an ex­tremely com­pet­i­tive job mar­ket.

The vast ma­jor­ity of can­di­dates have lit­tle to no ev­i­dence of cre­ativ­ity in pub­lic at all. The same is true for many of the best en­gi­neers I have worked with.

As a hir­ing man­ager, this means you have to learn how to source can­di­dates and in­ter­view ef­fec­tively: you don’t want to miss out on a great en­gi­neer just be­cause they spent all of their en­ergy mak­ing great prod­ucts for prior em­ploy­ers rather than blog­ging, speak­ing and cod­ing in pub­lic.

But as a can­di­date, this means you can give your­self a big ad­van­tage in terms of stand­ing out from the crowd with a rel­a­tively small amount of work.

Start a blog. Post an in­ter­est­ing tech­ni­cal ar­ti­cle to it once or twice a year—some­thing you’ve learned, or a bug you’ve fixed, or a prob­lem you’ve solved. After a few years stop both­er­ing en­tirely, but leave the blog on­line some­where.

Build a small per­sonal pro­ject and put the code on GitHub. Accompany it with a README with a de­tailed de­scrip­tion of the pro­ject and screen­shots of it in ac­tion—al­most no-one does this, it only takes a few hours ex­tra and it mas­sively in­creases the im­pact your pro­ject will have on hir­ing man­agers who are check­ing you out.

That’s it. One or two blog posts. Maybe a GitHub repos­i­tory. Believe it or not, if you are up against a bunch of other can­di­dates (especially ear­lier on in your ca­reer) they likely won’t have any­thing like that. You will jump straight to the top of the hir­ing man­ager’s men­tal list, maybe with­out them even notic­ing.

There’s plenty more you can do if you want to put the ef­fort in: build an au­di­ence on Twitter, start a newslet­ter, make videos, give talks (ideally that get recorded and pub­lished on­line), re­lease open source pack­ages, pub­lish TILs—but hon­estly if your goal is to get through the in­ter­view process more eas­ily you will very quickly hit the law of di­min­ish­ing re­turns.

If you’re go­ing to do that stuff do it be­cause you want to de­velop those skills and share with and learn from the world—don’t just do it be­cause you think it’s a crit­i­cal path to be­ing hired.

This post started out as a Twitter thread:

If you want to stand out from other can­di­dates, hav­ing even one piece of writ­ing or pub­lished piece of code that shows some­thing you’ve built is a great way to do that https://​t.co/​QfYEWx­fIet- Simon Willison (@simonw) July 17, 2021


Read the original on simonwillison.net »

7 343 shares, 12 trendiness, words and minutes reading time

Mitchell's New Role at HashiCorp

Today, I have some ex­cit­ing news that I wanted to share with the broader HashiCorp com­mu­nity: I am tran­si­tion­ing into a new role as a full-time in­di­vid­ual con­trib­u­tor and off of the HashiCorp ex­ec­u­tive team.

I founded HashiCorp in 2012 and served as CEO un­til 2016, and then be­came one of our CTOs un­til to­day. After nearly 10 years of learn­ing and grow­ing as an ex­ec­u­tive, I’m ready and ex­cited to step back into a full-time en­gi­neer­ing po­si­tion and look for­ward to mak­ing a mean­ing­ful im­pact with this new role. This change has been years in plan­ning and is now pos­si­ble thanks to HashiCorp’s ma­tu­rity and the ex­cel­lent lead­er­ship team we now have in place.

I feel the best way to ex­plain more de­tails about this change is to sim­ply share the email I wrote to our em­ploy­ees, so I’ve in­cluded that be­low. I look for­ward to con­tin­u­ing to talk with you all as part of our com­mu­nity as I pur­sue my new role.

Here’s my email to our em­ploy­ees:

Beginning to­day, I’ll be tran­si­tion­ing into a new role here at HashiCorp as an in­di­vid­ual con­trib­u­tor fo­cused on en­gi­neer­ing. This is some­thing I’m very ex­cited about and that the ex­ec­u­tive team and I have planned for a long time. I’d like to take the op­por­tu­nity in this email to fill in the back­ground about this change and de­scribe what this means for me and the com­pany go­ing for­ward.

Ever since found­ing HashiCorp, I’ve felt it’s im­por­tant to build a com­pany where I’m not re­quired for day-to-day op­er­a­tions and where other lead­ers can step in at the right phase. I be­lieve this is nec­es­sary for a com­pany to be re­silient and long-last­ing. This mind­set could be seen as early as 2016 when Armon [Dadgar] and I proac­tively sought out a CEO to suc­ceed us and lead the com­pany for­ward. In our search we found Dave [McJannet] and he was, and con­tin­ues to be, the right leader for the next phases of the com­pany. At that time, I stepped into a new role as CTO, one that has been bet­ter suited to me for the last few years.

There are also per­sonal el­e­ments to this de­ci­sion. I founded HashiCorp as an en­gi­neer pas­sion­ate about in­fra­struc­ture tool­ing. But as a founder, my role at times has had to ex­pand well be­yond and away from that. That’s the price of be­ing a founder: you do what­ever is nec­es­sary of you, even if there are parts of the role that don’t par­tic­u­larly mo­ti­vate you. And over the course of nearly a decade build­ing HashiCorp into a multi-bil­lion dol­lar com­pany, I’ve con­tin­u­ously reaf­firmed that I’m still an en­gi­neer at heart and I’m ready to more of­fi­cially get back to fo­cus­ing on that.

HashiCorp is once again at a place where I feel the com­pany and I are ready for me to step into a new role. The HashiCorp lead­er­ship team is strong and I trust them com­pletely. The busi­ness is healthy and I have com­plete op­ti­mism about the fu­ture.

In my new role, I will be an in­di­vid­ual con­trib­u­tor ei­ther on spe­cific prod­uct teams or ex­plor­ing other ideas within HashiCorp. I will con­tinue to par­tic­i­pate with lead­er­ship in cer­tain big-pic­ture plan­ning such as ma­jor prod­uct plans, mes­sag­ing for HashiConf keynotes, and other strate­gic de­ci­sions. I’ll con­tinue to be a full-time HashiCorp em­ployee. However, as I step into this role, I will of­fi­cially no longer be a mem­ber of the ex­ec­u­tive team and will there­fore no longer have ac­cess to ex­ec­u­tive meet­ings, plans, or de­ci­sions.

Coinciding with my role change, I am also step­ping down from the HashiCorp board. While we’ve de­cided to match the tim­ing, I’d like to stress that this was a de­ci­sion we made years ago and is not re­lated at all to my chang­ing role. As a startup ma­tures into a later stage, it is ex­pected for the board to have more in­de­pen­dent mem­bers and it is typ­i­cal for only one founder to be on a late-stage board along­side the CEO. I fully trust Armon and Dave to rep­re­sent the em­ploy­ees, my­self in­cluded.

While I will no longer be on the ex­ec­u­tive team or board, I will re­main a pas­sion­ate and ac­tive HashiCorp em­ployee and en­gi­neer­ing leader. I’m very ex­cited to be able to ded­i­cate more of my time to­wards prod­uct and en­gi­neer­ing chal­lenges and to spend more of my day-to-day time work­ing di­rectly along­side mem­bers in those or­ga­ni­za­tions. My ex­pe­ri­ence, voice, and ex­per­tise re­main fully avail­able to every­one in the com­pany.

I’m in­cred­i­bly proud that as an ex­ec­u­tive, I helped HashiCorp grow from noth­ing to nearly 1,500 em­ploy­ees with a val­u­a­tion of over $5 bil­lion. Looking for­ward, the com­pany has a huge op­por­tu­nity, and I’m ex­cited to con­tinue work­ing with you all to build and grow the com­pany and make an im­pact in my new role.


Read the original on www.hashicorp.com »

8 268 shares, 28 trendiness, words and minutes reading time

We’d Rather Have the Iceberg Than the Ship — GS

We’d Rather Have the Iceberg Than the Ship

I re­cently bought a house in Madison, Wisconsin, but be­fore I did I went through a se­ries of ad­ven­tures in dif­fer­ent parts of the state ex­plor­ing the prop­erty mar­ket. I was ini­tially in­ter­ested in Wisconsin be­cause I have ex­tended fam­ily there and they en­cour­aged me to look at Appleton. They de­scribed it as a safe, pros­per­ous, and grow­ing com­mu­nity pop­u­lated by good peo­ple. There’s a his­toric down­town, Lawrence University, the med­ical cen­ter, and it’s the county seat. As proof of its bright fu­ture they also men­tioned Amazon is build­ing a gi­ant ful­fill­ment cen­ter out by the air­port. I found a real es­tate agent, de­scribed what I was look­ing for, and she be­gan send­ing me links to avail­able prop­er­ties. What I re­quested was a build­ing with a bit of ar­chi­tec­tural char­ac­ter in a walk­a­ble neigh­bor­hood near civic ameni­ties, quite pos­si­bly within a bi­cy­cle ride of the uni­ver­sity, pub­lic parks, restau­rants, and shops. I asked for a du­plex, think­ing I would be buy­ing a place that I would co-own with fam­ily mem­bers. They could have their half, I could have mine. Win win.These are the places she sug­gested in­stead. I looked at the fil­ter she had set on the list­ings and she had elim­i­nated any prop­erty that was built be­fore 1990. That guar­an­teed the homes on of­fer would all be out on the sub­ur­ban fringe along the high­ways in­stead of in town. When I asked her about this she was con­fused and set about ed­u­cat­ing me on the prop­erty mar­ket in Appleton.She ex­plained that you don’t want an older home be­cause they have too much de­ferred main­te­nance. Bringing them up to a mod­ern stan­dard is too ex­pen­sive rel­a­tive to their re­sale value. Taxes are too high in old neigh­bor­hoods so you want to buy across mu­nic­i­pal lines out­side the older city lim­its. You get more house for less money with lower taxes in the newer de­vel­op­ments. The schools are much bet­ter in the newer ar­eas, and peo­ple shop for school dis­tricts more than they shop for a house it­self. While Appleton is a very safe lit­tle city, crime is al­ways a big­ger prob­lem at the core com­pared to the edges. You have to think of the chil­dren. And a newer home on a larger lot is a bet­ter in­vest­ment be­cause that’s what qual­ity buy­ers want. Older homes don’t ap­pre­ci­ate, they de­cline.What she was telling me was no doubt true from her per­spec­tive. She re­flected the val­ues not just of Appleton, but most of America and the peo­ple who choose to live in these places. She was­n’t wrong. But aside from the fact that I did­n’t care for any of these homes and was never go­ing to buy in these lo­ca­tions, I re­al­ized the truth of the Appleton model. Thirty years from now all the new homes she’s sell­ing will slip into the old” cat­e­gory and will grad­u­ally fes­ter as taxes rise and the mid­dle class mi­grates to new green­field de­vel­op­ments. These older places (the homes be­ing built to­day) will then be pop­u­lated by lower class peo­ple with fewer re­sources and less sta­tus thereby re­in­forc­ing the per­cep­tion that it’s best to move on if at all pos­si­ble. These are fun­gi­ble, for­get­table, dis­pos­able places that rapidly age and are then left to qui­etly de­cay. There’s also the prob­lem of ag­ing com­mer­cial real es­tate. Over the last few gen­er­a­tions we’ve wit­nessed the whole­sale aban­don­ment of Main Street in fa­vor of strip cen­ters, big box stores, and re­gional shop­ping malls. Now these re­tail cen­ters are get­ting old and are be­ing squeezed hard by on­line com­merce and home de­liv­ery. The Amazon ful­fill­ment ware­house that’s so ag­gres­sively sub­si­dized by lo­cal gov­ern­ment and her­alded as growth” out on the edge of the metro­plex is ef­fec­tively re­plac­ing the dried husk of the Fox River Mall and var­i­ous lesser shop­ping plazas. This is­n’t an ad­di­tive process. It’s slash and burn ur­ban­ism and it does­n’t end well.The good news” is these places were built fast and cheap and were never meant to last any­way. They have a con­struc­tion lifes­pan ex­quis­itely tuned to match the amor­ti­za­tion and de­pre­ci­a­tion sched­ules of the fi­nan­cial un­der­writ­ers. As soon as they stop spin­ning off cash flow and tax ad­van­tages to dis­tant in­vestors they’re toast. Then they be­come some­one else’s prob­lem. If the value of the dirt is high enough some­one will re­de­velop the prop­erty into some­thing dif­fer­ent, but only with the ap­pro­pri­ate sweet­en­ers from business friendly” lo­cal gov­ern­ment, mean­ing heavy sub­si­dies and tax hol­i­days. These deals tend to be a hostage ne­go­ti­a­tion.Within the con­tin­u­ous greater Appleton re­gion for ten miles in any di­rec­tion are dozens of smaller mu­nic­i­pal­i­ties that have been en­gulfed by sub­ur­ban de­vel­op­ment for most of a cen­tury. Little ves­tiges of once pros­per­ous towns linger in di­min­ished form em­bed­ded in the sprawl. These towns were built along barge canals, small hy­dro­elec­tric dams, and pro­duc­tive mills. Many of the sur­viv­ing build­ings have been par­tially oc­cu­pied or com­pletely va­cant for decades. Half the orig­i­nal struc­tures were so de­val­ued that they were torn down and re­placed with sur­face park­ing lots. A num­ber of them are used as low grade stor­age fa­cil­i­ties or bud­get con­sign­ment shops.Out of cu­rios­ity I looked at how much one of these build­ings costs. This win­ter this place with a de­funct Chinese restau­rant on the ground floor and a va­cant two bed­room apart­ment up­stairs was of­fered at $85,000. The cost of the restau­rant equip­ment alone, the in­dus­trial stoves, deep fry­ers, walk-in re­frig­er­a­tors, stain­less steel work ta­bles, and fire sup­press­ing vent hoods would have been close to that price. The build­ing had been on the mar­ket for a very long time with no bids. It even­tu­ally sold a few months ago for $65,000. For com­par­i­son, here’s a re­view of a $65,000 lux­ury 2021 Ram pickup truck.Some­one re­moved the restau­rant in­te­rior and opened a col­lectibles shop that sells base­ball cards. This shop ac­ti­vates the store­front and keeps the place warm so I’m all for it. But un­less this en­ter­prise does most of its sales via the in­ter­webs it’s go­ing to strug­gle to sur­vive. Fortunately the over­head is shock­ingly low. It will all keep tick­ing along un­til the build­ing needs a big ticket item like a struc­tural re­pair to a fail­ing brick wall or cracked foun­da­tion. Then it starts down the road to a park­ing lot con­ver­sion. Back when this town was at its peak a cen­tury ago each of these va­cant parcels had mul­ti­ple build­ings on them. They’re all gone now. I called the lo­cal busi­ness im­prove­ment dis­trict of­fice as well as the town plan­ner and chat­ted with peo­ple in var­i­ous de­part­ments. They gave the usual sales pitch about how they’re cre­at­ing a vi­brant re­tail Main Street with his­toric char­ac­ter. They men­tioned the new dec­o­ra­tive lamp posts, flower beds, fes­tive flags, and sea­sonal cel­e­bra­tions. Did you see our ea­gle sculp­ture?” I got a flash­back to Colerain Township, Ohio. Blighted build­ings had been re­placed with a new fire sta­tion, mid­dle school, city of­fices, and a new po­lice sta­tion. And there are plans for in­fill de­vel­op­ment once fi­nanc­ing and enough va­cant parcels are ag­gre­gated to ac­com­mo­date mod­ern build­ings. (AKA Texas Doughnuts.) My sense is that they’re jump­ing on the city plan­ning fash­ion band­wagon twenty years too late.What I see for the fu­ture of this spot is a vari­a­tion on things I’ve seen else­where around the coun­try. A sliver of Ye Olde Towne is pre­served for sen­ti­men­tal rea­sons and turned into a de facto themed strip mall. In this lo­ca­tion, with the pre­vail­ing cul­ture and econ­omy… I don’t see this be­ing much more than a mediocre col­lec­tion of cos­metic patches that will limp along half assed un­til the next fad ar­rives in an­other twenty years. It’s not a ter­ri­ble sit­u­a­tion, but it’s not transformative” or catalytic” ei­ther. I gave these of­fi­cials my usual blunt as­sess­ment. They did­n’t like my tone.So here’s the big pic­ture. All of America’s in­sti­tu­tions are fo­cused ex­clu­sively on churn. Crank out new stuff, sell it fast, cash out, and move on to the next pro­ject. Blighted neigh­bor­hoods aren’t an ac­ci­dent. They’re baked in to every facet of how we do every­thing. Successful in­di­vid­u­als and savvy in­vestors know this in­stinc­tively and keep mov­ing every five or ten years to the next new bet­ter place. This is also true of mu­nic­i­pal of­fi­cials and pri­vate con­sul­tants who con­tin­u­ally hop­scotch from job to job leav­ing be­hind dis­tricts that have peaked in fa­vor of ones that are still grow­ing. I’d love for peo­ple to stop pre­tend­ing oth­er­wise and just be hon­est about the sit­u­a­tion. You get a re­ally good run for a few decades. Then things slowly turn to crap as the vinyl sid­ing and syn­thetic stucco start to peel off. We’re go­ing to con­tinue to do this un­til we sim­ply can’t any­more for one rea­son or an­other. Then we’ll have no choice but to start re-in­hab­it­ing the dregs that were left be­hind. Some places will be more wor­thy of sal­va­tion than oth­ers. Shrug.I’m re­minded of Elizabeth Bishop’s poem The Imaginary Iceberg” which be­gins, We’d rather have the ice­berg than the ship, al­though it meant the end of travel.” That’s how I feel about the North American de­vel­op­ment pat­tern. It’s ephemeral. In the long arc of his­tory no one will miss any of these places. Future gen­er­a­tions will be busy do­ing en­tirely dif­fer­ent things with the land­scape they’ll in­herit. I’ve made my peace with that re­al­ity.


Read the original on www.granolashotgun.com »

9 268 shares, 12 trendiness, words and minutes reading time


P{{ padded­PageStr }} 

Hacker News in the style of Teletext 

Type the num­ber of the page to go to 


Read the original on glynnbird.github.io »

10 235 shares, 9 trendiness, words and minutes reading time

How not to design a file format.

The Zip file for­mat is now 32 years old. You’d think be­ing 32 years old the for­mat would be well doc­u­mented. Unfortunately it’s not.

I have a feel­ing this is like many file for­mats. They aren’t de­signed, rather the de­vel­oper just makes it up as they go. If it gets pop­u­lar other peo­ple want to read and/​or write them. They ei­ther try to re­verse en­gi­neer the for­mat OR they ask for specs. Even if the de­vel­oper writes specs they of­ten for­get all the as­sump­tions their orig­i­nal pro­gram makes. Those are not writ­ten down and hence the spec is in­com­plete. Zip is such a for­mat.

Zip claims its for­mat is doc­u­mented in a file called APPNOTE. TXT which can be found here.

The short ver­sion is, a zip file con­sists of records, each record starts with some 4 byte marker that gen­er­ally takes the form

0x50, 0x4B, ??, ??

Where the 0x50, 0x4B are the let­ters PK stand­ing for Phil Katz”, the per­son who made the zip for­mat. The two ?? are bytes that iden­tify the type of the record. Examples

0x50 0x4b 0x03 0x04 // a lo­cal file record

0x50 0x4b 0x01 0x02 // a cen­tral di­rec­tory file record

0x50 0x4b 0x06 0x06 // an end of cen­tral di­rec­tory record

Records do NOT fol­low any stan­dard pat­tern. To read or even skip a record you must know its for­mat. What I mean is there are sev­eral other for­mats that fol­low some con­ven­tion like each record id is fol­lowed by the length of the record. So, if you see an id, and you don’t un­der­stand it, you just read the length, skip that many bytes (*), and you’ll be at the next id. Examples of this type in­clude most video con­tainer for­mats, jpgs, tiff, pho­to­shop files, wav files, and many oth­ers.

(*) some for­mats re­quire round­ing the length up to the near­est mul­ti­ple of 4 or 16.

Zip does NOT do this. If you see an id and you don’t know how that type of record’s con­tent is struc­tured there is no way to know how many bytes to skip.

APPNOTE. TXT says the fol­low­ing things

4.1.9 ZIP files MAY be streamed, split into seg­ments (on fixed or on re­mov­able me­dia) or self-extracting”. Self-extracting ZIP files MUST in­clude ex­trac­tion code for a tar­get plat­form within the ZIP file.

4.3.1 A ZIP file MUST con­tain an end of cen­tral di­rec­tory record”. A ZIP file con­tain­ing only an end of cen­tral di­rec­tory record” is con­sid­ered an empty ZIP file. Files MAY be added or re­placed within a ZIP file, or deleted. A ZIP file MUST have only one end of cen­tral di­rec­tory record”. Other records de­fined in this spec­i­fi­ca­tion MAY be used as needed to sup­port stor­age re­quire­ments for in­di­vid­ual ZIP files.

4.3.2 Each file placed into a ZIP file MUST be pre­ceded by a local file header” record for that file. Each local file header” MUST be ac­com­pa­nied by a cor­re­spond­ing central di­rec­tory header” record within the cen­tral di­rec­tory sec­tion of the ZIP file.

4.3.3 Files MAY be stored in ar­bi­trary or­der within a ZIP file. A ZIP file MAY span mul­ti­ple vol­umes or it MAY be split into user-de­fined seg­ment sizes. All val­ues MUST be stored in lit­tle-en­dian byte or­der un­less oth­er­wise spec­i­fied in this doc­u­ment for a spe­cific data el­e­ment.

[local file header 1]

[encryption header 1]

[file data 1]

[data de­scrip­tor 1]

[local file header n]

[encryption header n]

[file data n]

[data de­scrip­tor n]

[archive de­cryp­tion header]

[archive ex­tra data record]

[central di­rec­tory header 1]

[central di­rec­tory header n]

[zip64 end of cen­tral di­rec­tory record]

[zip64 end of cen­tral di­rec­tory lo­ca­tor]

[end of cen­tral di­rec­tory record]

lo­cal file header sig­na­ture 4 bytes (0x04034b50)

ver­sion needed to ex­tract 2 bytes

gen­eral pur­pose bit flag 2 bytes

com­pres­sion method 2 bytes

last mod file time 2 bytes

last mod file date 2 bytes

crc-32 4 bytes

com­pressed size 4 bytes

un­com­pressed size 4 bytes

file name length 2 bytes

ex­tra field length 2 bytes

file name (variable size)

ex­tra field (variable size)

Immediately fol­low­ing the lo­cal header for a file SHOULD be placed the com­pressed or stored data for the file. If the file is en­crypted, the en­cryp­tion header for the file SHOULD be placed af­ter the lo­cal header and be­fore the file data. The se­ries of [local file header][en­cryp­tion header] [file data][data de­scrip­tor] re­peats for each file in the  .ZIP archive.

Zero-byte files, di­rec­to­ries, and other file types that con­tain no con­tent MUST NOT in­clude file data.

[central di­rec­tory header 1]

[central di­rec­tory header n]

[digital sig­na­ture]

cen­tral file header sig­na­ture 4 bytes (0x02014b50)

ver­sion made by 2 bytes

ver­sion needed to ex­tract 2 bytes

gen­eral pur­pose bit flag 2 bytes

com­pres­sion method 2 bytes

last mod file time 2 bytes

last mod file date 2 bytes

crc-32 4 bytes

com­pressed size 4 bytes

un­com­pressed size 4 bytes

file name length 2 bytes

ex­tra field length 2 bytes

file com­ment length 2 bytes

disk num­ber start 2 bytes

in­ter­nal file at­trib­utes 2 bytes

ex­ter­nal file at­trib­utes 4 bytes

rel­a­tive off­set of lo­cal header 4 bytes

file name (variable size)

ex­tra field (variable size)

file com­ment (variable size)

end of cen­tral dir sig­na­ture 4 bytes (0x06054b50)

num­ber of this disk 2 bytes

num­ber of the disk with the

start of the cen­tral di­rec­tory 2 bytes

to­tal num­ber of en­tries in the

cen­tral di­rec­tory on this disk 2 bytes

to­tal num­ber of en­tries in

the cen­tral di­rec­tory 2 bytes

size of the cen­tral di­rec­tory 4 bytes

off­set of start of cen­tral

di­rec­tory with re­spect to

the start­ing disk num­ber 4 bytes

.ZIP file com­ment length 2 bytes

.ZIP file com­ment (variable size)

There are other de­tails in­volv­ing en­cryp­tion, larger files, op­tional data, but for the pur­poses of this post this is all we need. We need one more piece of info, how to make a self ex­tract­ing archive.

To do so we could look back to ZIP2EXE.exe which shipped with pkzip in 1989 and see what it does but it’s eas­ier look at Info-Zip to see what hap­pens.

How do I make a DOS (or other non-na­tive) self-ex­tract­ing archive un­der Unix?

The pro­ce­dure is ba­si­cally de­scribed in the UnZipSFX man page. First grab the ap­pro­pri­ate UnZip bi­nary dis­tri­b­u­tion for your tar­get plat­form (DOS, Windows, OS/2, etc.), as de­scribed above; we’ll as­sume DOS in the fol­low­ing ex­am­ple. Then ex­tract the UnZipSFX stub from the dis­tri­b­u­tion and prepend as if it were a na­tive Unix stub:

> un­zip un­z552x3.exe un­zipsfx.exe // ex­tract the DOS SFX stub

> cat un­zipsfx.exe yourzip.zip > your­DOSzip.exe // cre­ate the SFX archive

> zip -A your­DOSzip.exe // fix up in­ter­nal off­sets

> That’s it. You can still test, up­date and delete en­tries from the archive; it’s a fully func­tional zip­file.

So given all of that let’s go over some prob­lems.

This is un­de­fined by the spec.

Scan from the front, when you see an id for a record do the ap­pro­pri­ate thing.

Scan from the back, find the end-of-cen­tral-di­rec­tory-record and then use it to read through the cen­tral di­rec­tory, only look­ing at things the cen­tral di­rec­tory ref­er­ences.

Scanning from the back is how the orig­i­nal pkun­zip works. For one it means if you ask for some sub­set of files it can jump di­rectly to the data you need in­stead of hav­ing to scan the en­tire zip file. This was es­pe­cially im­por­tant if the zip file spanned mul­ti­ple floppy disks.

But, 4.1.9 says you can stream zip files. How is that pos­si­ble? What if there is some lo­cal file record that is not ref­er­enced by the cen­tral di­rec­tory? Is that valid? This is un­de­fined.

Files MAY be added or re­placed within a ZIP file, or deleted.

Okay? That sug­gests the cen­tral di­rec­tory might not ref­er­ence all the files in the zip file be­cause oth­er­wise this state­ment about files be­ing added, re­placed, or delete has no point to be in the spec.


Read the original on games.greggman.com »

To add this web app to your iOS home screen tap the share button and select "Add to the Home Screen".

10HN is also available as an iOS App

If you visit 10HN only rarely, check out the the best articles from the past week.

If you like 10HN please leave feedback and share

Visit pancik.com for more.