10 interesting stories served every morning and every evening.

1 424 shares, 40 trendiness, 3644 words and 29 minutes reading time


Published: July 1, 2020


The smartest per­son I’ve ever known had a habit that, as a teenager, I found strik­ing. After he’d prove a the­o­rem, or solve a prob­lem, he’d go back and con­tinue think­ing about the prob­lem and try to fig­ure out dif­fer­ent proofs of the same thing. Sometimes he’d spend hours on a prob­lem he’d al­ready solved.

I had the op­po­site ten­dency: as soon as I’d reached the end of the proof, I’d stop since I’d gotten the an­swer”.

Afterwards, he’d come out with three or four proofs of the same thing, plus some ex­pla­na­tion of why each proof is con­nected some­how. In this way, he got a much deeper un­der­stand­ing of things than I did.

I con­cluded that what we call intelligence’ is as much about virtues such as hon­esty, in­tegrity, and brav­ery, as it is about raw in­tel­lect’.

Intelligent peo­ple sim­ply aren’t will­ing to ac­cept an­swers that they don’t un­der­stand — no mat­ter how many other peo­ple try to con­vince them of it, or how many other peo­ple be­lieve it, if they aren’t able to con­vince them selves of it, they won’t ac­cept it.

Importantly, this is a software’ trait & is in­de­pen­dent of more hardware’ traits such as pro­cess­ing speed, work­ing mem­ory, and other such things.

Moreover, I have no­ticed that these hardware’ traits vary greatly in the smartest peo­ple I know — some are re­mark­ably quick thinkers, cal­cu­la­tors, read­ers, whereas oth­ers are slow’. The soft­ware traits, though, they all have in com­mon — and can, with ef­fort, be learned.

What this means is that you can in­ter­nal­ize good in­tel­lec­tual habits that, in ef­fect, increase your in­tel­li­gence”. Intelligence’ is not fixed.


This qual­ity of not stop­ping at an un­sat­is­fac­tory an­swer” de­serves some ex­am­i­na­tion.

One com­po­nent of it is en­ergy: think­ing hard takes ef­fort, and it’s much eas­ier to just stop at an an­swer that seems to make sense, than to pur­sue every­thing that you don’t quite get down an end­less, and rapidly pro­lif­er­at­ing, se­ries of rab­bit holes.

It’s also so easy to think that you un­der­stand some­thing, when you ac­tu­ally don’t. So even fig­ur­ing out whether you un­der­stand some­thing or not re­quires you to at­tack the thing from mul­ti­ple an­gles and test your own un­der­stand­ing.

This re­quires a lot of in­trin­sic mo­ti­va­tion, be­cause it’s so hard; so most peo­ple sim­ply don’t do it.

The Nobel Prize win­ner William Shockley was fond of talk­ing about the will to think”:

Motivation is at least as im­por­tant as method for the se­ri­ous thinker, Shockley be­lieved…the es­sen­tial el­e­ment for suc­cess­ful work in any field was the will to think”. This was a phrase he learned from the nu­clear physi­cist Enrico Fermi and never for­got. In these four words,” Shockley wrote later, [Fermi] dis­tilled the essence of a very sig­nif­i­cant in­sight: A com­pe­tent thinker will be re­luc­tant to com­mit him­self to the ef­fort that te­dious and pre­cise think­ing de­mands — he will lack the will to think’ — un­less he has the con­vic­tion that some­thing worth­while will be done with the re­sults of his ef­forts.” The dis­ci­pline of com­pe­tent think­ing is im­por­tant through­out life… (source)But it’s not just en­ergy. You have to be able to mo­ti­vate your­self to spend large quan­ti­ties of en­ergy on a prob­lem, which means on some level that not un­der­stand­ing some­thing — or hav­ing a bug in your think­ing — both­ers you a lot. You have the drive, the will to know.

Related to this is hon­esty, or in­tegrity: a sort of com­pul­sive un­will­ing­ness, or in­abil­ity, to lie to your­self. Feynman said that the first rule of sci­ence is that you do not fool your­self, and you are the eas­i­est per­son to fool. It is uniquely easy to lie to your­self be­cause there is no ex­ter­nal force keep­ing you hon­est; only you can run the con­stant loop of ask­ing do I re­ally un­der­stand this?”.

(This is why writ­ing is im­por­tant. It’s harder to fool your­self that you un­der­stand some­thing when you sit down to write about it and it comes out all dis­jointed and con­fused. Writing forces clar­ity.)


The physi­cist Michael Faraday be­lieved noth­ing with­out be­ing able to ex­per­i­men­tally demon­strate it him­self, no mat­ter how te­dious the demon­sta­tion.

Simply hear­ing or read­ing of such things was never enough for Faraday. When as­sess­ing the work of oth­ers, he al­ways had to re­peat, and per­haps ex­tend, their ex­per­i­ments. It be­came a life­long habit—his way of es­tab­lish­ing own­er­ship over an idea. Just as he did count­less times later in other set­tings, he set out to demon­strate this new phe­nom­e­non to his own sat­is­fac­tion. When he had saved enough money to buy the ma­te­ri­als, he made a bat­tery from seven cop­per half­pen­nies and seven discs cut from a sheet of zinc, in­ter­leaved with pieces of pa­per soaked in salt wa­ter. He fixed a cop­per wire to each end plate, dipped the other ends of the wires in a so­lu­tion of Epsom salts (magnesium sul­fate), and watched. (source)Understanding some­thing re­ally deeply is con­nected to our phys­i­cal in­tu­ition. A sim­ple words based” un­der­stand­ing can only go so far. Visualizing some­thing, in three di­men­sions, can help you with a con­crete hook” that your brain can grasp onto and use as a model; un­der­stand­ing then has a phys­i­cal con­text that it can take place in”.

This is why Jesus speaks in para­bles through­out the New Testament — in ways that stick with you long af­ter you’ve read them — rather than just stat­ing the ab­stract prin­ci­ple. Are not two spar­rows sold for a cent? And yet not one of them will fall to the ground apart from your Father.” can stick with you for­ever in a way that God watches over all liv­ing be­ings” will not.

Faraday, again, had this qual­ity in spades — the book makes clear that this is partly be­cause he was bad at math­e­mat­ics and thus un­der­stood every­thing through the medium of ex­per­i­ments, and con­trasts this with the French sci­en­tists (such as Ampere) who un­der­stood every­thing in a highly ab­stract way.

But Faraday’s phys­i­cal in­tu­ition led him to some of the most cru­cial dis­cov­er­ies in all of sci­ence:

Much as he ad­mired Ampère’s work, Faraday be­gan to de­velop his own views on the na­ture of the force be­tween a cur­rent-car­ry­ing wire and the mag­netic nee­dle it de­flected. Ampère’s math­e­mat­ics (which he had no rea­son to doubt) showed that the mo­tion of the mag­netic nee­dle was the re­sult of re­pul­sions and at­trac­tions be­tween it and the wire. But, to Faraday, this seemed wrong, or, at least, the wrong way around. What hap­pened, he felt, was that the wire in­duced a cir­cu­lar force in the space around it­self, and that every­thing else fol­lowed from this. The next step beau­ti­fully il­lus­trates Faraday’s ge­nius. Taking Sarah’s four­teen-year-old brother George with him down to the lab­o­ra­tory, he stuck an iron bar mag­net into hot wax in the bot­tom of a basin and, when the wax had hard­ened, filled the basin with mer­cury un­til only the top of the mag­net was ex­posed. He dan­gled a short length of wire from an in­su­lated stand so that its bot­tom end dipped in the mer­cury, and then he con­nected one ter­mi­nal of a bat­tery to the top end of the wire and the other to the mer­cury. The wire and the mer­cury now formed part of a cir­cuit that would re­main un­bro­ken even if the bot­tom end of the wire moved. And move it did—in rapid cir­cles around the mag­net! (source)

Being able to gen­er­ate these con­crete ex­am­ples, even when you’re not phys­i­cally do­ing ex­per­i­ments, is im­po­ratant.

I re­cently saw this strik­ing rep­re­sen­ta­tion of the bag of words” model in NLP. If you were read­ing this in the usual dry math­e­mat­i­cal way these things are rep­re­sented, and then forced your­self to come up with a vi­su­al­iza­tion like this, then you’d be much fur­ther on your way to re­ally grasp­ing the thing.

Conversely, if you’re not com­ing up with vi­su­als like this, and your un­der­stand­ing of the thing re­mains on the level of equa­tions or ab­stract con­cepts, you prob­a­bly do not un­der­stand the con­cept deeply and should dig fur­ther.

Another qual­ity I have no­ticed in very in­tel­li­gent peo­ple is be­ing un­afraid to look stu­pid.

Malcolm Gladwell on his fa­ther:

My fa­ther has zero in­tel­lec­tual in­se­cu­ri­ties… It has never crossed his mind to be con­cerned that the world thinks he’s an id­iot. He’s not in that game. So if he does­n’t un­der­stand some­thing, he just asks you. He does­n’t care if he sounds fool­ish. He will ask the most ob­vi­ous ques­tion with­out any sort of con­cern about it… So he asks lots and lots of dumb, in the best sense of that word, ques­tions. He’ll say to some­one, I don’t un­der­stand. Explain that to me.’ He’ll just keep ask­ing ques­tions un­til he gets it right, and I grew up lis­ten­ing to him do this in every con­ceiv­able set­ting. If my fa­ther had met Bernie Madoff, he would never have in­vested money with him be­cause he would have said, I don’t un­der­stand’ a hun­dred times. I don’t un­der­stand how that works’, in this kind of dumb, slow voice. I don’t un­der­stand, sir. What is go­ing on?’Most peo­ple are not will­ing to do this — look­ing stu­pid takes courage, and some­times it’s eas­ier to just let things slide. It is strik­ing how many sit­u­a­tions I am in where I start ask­ing ba­sic ques­tions, feel guilty for slow­ing the group down, and it turns out that no­body un­der­stood what was go­ing on to be­gin with (often peo­ple mes­sage me pri­vately say­ing that they’re re­lieved I asked), but I was the only one who ac­tu­ally spoke up and asked about it.

This is a habit. It’s easy to pick up. And it makes you smarter.


I re­mem­ber be­ing taught cal­cu­lus at school and get­ting stuck on the dy/dx” no­ta­tion (aka Leibniz no­ta­tion) for cal­cu­lus.

The dy/dx” just looked like a frac­tion, it looked like we were do­ing di­vi­sion, but we weren’t ac­tu­ally do­ing di­vi­sion. dy/dx” does­n’t mean dy” di­vided by dx”, it means the value of an in­fin­i­tes­i­mal change in y with re­spect to an in­fin­i­tes­i­mal change in x”, and I did­n’t see how you could break this thing apart as though it was sim­ple di­vi­sion.

At one point the proof of the fun­da­men­tal the­o­rem of cal­cu­lus in­volved mul­ti­ply­ing out a poly­no­mial, and along the way you could can­cel out dy*dx” be­cause both of these quan­ti­ties are in­fin­i­tes­i­mal, so in ef­fect this can be can­celled out”. This rea­son­ing did not make sense.

The proof” of the chain rule we were given looked like this.

(Amusingly, you can even get cor­rect re­sults us­ing in­valid math­e­mat­ics, like this. Even though this is clearly in­valid, it does­n’t feel far off the valid” proof of the chain rule I was taught.)

It turns out that my mis­giv­ings were right, and that the Leibniz no­ta­tion is ba­si­cally just a con­ve­nient short­hand and that you more or less can treat those things as if” they are frac­tions, but the proof is su­per com­pli­cated etc. Moreover, the Leibniz short­hand is ac­tu­ally far more pow­er­ful and eas­ier to work with than Newton’s func­tions-based short­hand, which is why main­land Europe got way ahead of England (which stuck with Newton’s no­ta­tion) in cal­cu­lus. And then all of the log­i­cal prob­lems did­n’t re­ally get sorted out un­til Riemann came along 200 years later and for­mu­lated cal­cu­lus in terms of lim­its. But all of that went over my head in high school.

At the time, I was in­fu­ri­ated by these in­ad­e­quate proofs, but I was un­der time pres­sure to just learn the op­er­a­tions so that I could an­swer exam ques­tions be­cause the class needed to move onto the next thing.

And since you ac­tu­ally can an­swer the exam ques­tions and me­chan­i­cally per­form cal­cu­lus op­er­a­tions with­out ever deeply un­der­stand­ing cal­cu­lus, it’s much eas­ier to just get by and do the exam with­out re­ally ques­tion­ing the con­cepts deeply — which is in fact what hap­pens for most peo­ple. (See my es­say on ed­u­ca­tion.)

How many peo­ple ac­tu­ally go back and try and un­der­stand this, or other such top­ics, in a deeper way? Very few. Moreover, the meta’ les­son is: don’t ques­tion it too deeply, you’ll fall be­hind. Just learn the al­go­rithm, plug in the num­bers, and pass your ex­ams. Speed is of the essence. In this way, school kills the will to un­der­stand­ing” in peo­ple.

My coun­ter­vail­ing ad­vice to peo­ple try­ing to un­der­stand some­thing is: go slow. Read slowly, think slowly, re­ally spend time pon­der­ing the thing. Start by think­ing about the ques­tion your­self be­fore read­ing a bunch of stuff about it. A week or a month of con­tin­u­ous pon­der­ing about a ques­tion will get you sur­pris­ingly far.

And you’ll have a se­man­tic men­tal framework’ in your brain on which to then hang all the great things you learn from your read­ing, which makes it more likely that you’ll re­tain that stuff as well. I read some­where that Bill Gates struc­tures his fa­mous reading weeks” around an out­line of im­por­tant ques­tions he’s thought about and bro­ken down into pieces. e.g. he’ll think about water scarcity” and then break it down into ques­tions like how much wa­ter is there in the world?”, where does ex­ist­ing drink­ing wa­ter come from?”, how do you turn ocean wa­ter into drink­ing wa­ter”, etc., and only then will he pick read­ing to ad­dress those ques­tions.

This method is far more ef­fec­tive than just read­ing ran­dom things and let­ting them pass through you.


The best thing I have read on re­ally un­der­stand­ing things is the Sequences, es­pe­cially the sec­tion on Noticing Confusion.

There are some mantra-like ques­tions it can be help­ful to ask as you’re think­ing through things. Some ex­am­ples:

But what ex­actly is X? What is it? (h/t Laura Deming’s post)Why must X be true? Why does this have to be the case? What is the sin­gle, fun­da­men­tal rea­son? Do I re­ally be­lieve that this is true, deep down? Would I bet a large amount of money on it with a friend?


Two para­bles:

First, Ezra Pound’s para­ble of Agassiz, from his ABC of Reading” (incidentally one of the most un­der­rated books about lit­er­a­ture). I’ve pre­served his quirky for­mat­tingNo man is equipped for mod­ern think­ing un­til he has un­der­stood the anec­dote of Agassiz and the fish:

A post-grad­u­ate stu­dent equipped with ho­n­ours and diplo­mas went to Agassiz to re­ceive the fi­nal and fin­ish­ing touches.

The great man of­fered him a small fish and told him to de­scribe it.

Post-Graduate Student: That’s only a sun-fish”

Agassiz: I know that. Write a de­scrip­tion of it.”

After a few min­utes the stu­dent re­turned with the de­scrip­tion of the Ichthus Heliodiplodokus, or what­ever term is used to con­ceal the com­mon sun­fish from vul­gar knowl­edge, fam­ily of Heliichterinkus, etc., as found in text­books of the sub­ject.

Agassiz again told the stu­dent to de­scribe the fish.

The stu­dent pro­duced a four-page es­say.

Agassiz then told him to look at the fish. At the end of the three weeks the fish was in an ad­vanced state of de­com­po­si­tion, but the stu­dent knew some­thing about it. The sec­ond, one of my fa­vorite pas­sages from Zen and the Art of Motorcycle Maintenance”:

He’d been hav­ing trou­ble with stu­dents who had noth­ing to say. At first he thought it was lazi­ness but later it be­came ap­par­ent that it was­n’t. They just could­n’t think of any­thing to say.

One of them, a girl with strong-lensed glasses, wanted to write a five-hun­dred­word es­say about the United States. He was used to the sink­ing feel­ing that comes from state­ments like this, and sug­gested with­out dis­par­age­ment that she nar­row it down to just Bozeman.

When the pa­per came due she did­n’t have it and was quite up­set. She had tried and tried but she just could­n’t think of any­thing to say.

He had al­ready dis­cussed her with her pre­vi­ous in­struc­tors and they’d con­firmed his im­pres­sions of her. She was very se­ri­ous, dis­ci­plined and hard­work­ing, but ex­tremely dull. Not a spark of cre­ativ­ity in her any­where. Her eyes, be­hind the thick-lensed glasses, were the eyes of a drudge. She was­n’t bluff­ing him, she re­ally could­n’t think of any­thing to say, and was up­set by her in­abil­ity to do as she was told.

It just stumped him. Now he could­n’t think of any­thing to say. A si­lence oc­curred, and then a pe­cu­liar an­swer: Narrow it down to the main street of Bozeman.” It was a stroke of in­sight.

She nod­ded du­ti­fully and went out. But just be­fore her next class she came back in real dis­tress, tears this time, dis­tress that had ob­vi­ously been there for a long time. She still could­n’t think of any­thing to say, and could­n’t un­der­stand why, if she could­n’t think of any­thing about all of Bozeman, she should be able to think of some­thing about just one street.

He was fu­ri­ous. You’re not look­ing!” he said. A mem­ory came back of his own dis­missal from the University for hav­ing too much to say. For every fact there is an in­fin­ity of hy­pothe­ses. The more you look the more you see. She re­ally was­n’t look­ing and yet some­how did­n’t un­der­stand this.

He told her an­grily, Narrow it down to the front of one build­ing on the main street of Bozeman. The Opera House. Start with the up­per left-hand brick.”

Her eyes, be­hind the thick-lensed glasses, opened wide. She came in the next class with a puz­zled look and handed him a five- thou­sand-word es­say on the front of the Opera House on the main street of Bozeman, Montana. I sat in the ham­burger stand across the street,” she said, and started writ­ing about the first brick, and the sec­ond brick, and then by the third brick it all started to come and I could­n’t stop. They thought I was crazy, and they kept kid­ding me, but here it all is. I don’t un­der­stand it.”

Neither did he, but on long walks through the streets of town he thought about it and con­cluded she was ev­i­dently stopped with the same kind of block­age that had par­a­lyzed him on his first day of teach­ing. She was blocked be­cause she was try­ing to re­peat, in her writ­ing, things she had al­ready heard, just as on the first day he had tried to re­peat things he had al­ready de­cided to say. She could­n’t think of any­thing to write about Bozeman be­cause she could­n’t re­call any­thing she had heard worth re­peat­ing. She was strangely un­aware that she could look and see freshly for her­self, as she wrote, with­out pri­mary re­gard for what had been said be­fore. The nar­row­ing down to one brick de­stroyed the block­age be­cause it was so ob­vi­ous she had to do some orig­i­nal and di­rect see­ing.

The point of both of these para­bles: noth­ing beats di­rect ex­pe­ri­ence. Get the data your­self. This is why I wanted to an­a­lyze the coro­n­avirus genome di­rectly, for ex­am­ple. You de­velop some ba­sis in re­al­ity by get­ting some first-hand data, and rea­son­ing up from there, ver­sus start­ing with some­body else’s lossy com­pres­sion of a messy, evolv­ing phe­nom­e­non and then won­der­ing why events keep sur­pris­ing you.

People who have not ex­pe­ri­enced the thing are un­likely to be gen­er­at­ing truth. More likely, they’re resur­fac­ing cached thoughts and nar­ra­tives. Reading pop­u­lar sci­ence books or news ar­ti­cles is not a sub­sti­tute for un­der­stand­ing, and may make you stu­pider, by fill­ing your mind with nar­ra­tives and sto­ries that don’t rep­re­sent your own syn­the­sis.

Even if you can’t ex­pe­ri­ence the thing di­rectly, try go­ing for in­for­ma­tion-dense sources with high amounts of de­tail and facts, and then rea­son up from those facts. On for­eign pol­icy, read books pub­lished by uni­ver­sity presses — not The Atlantic or The Economist or what­ever. You can read those af­ter you’ve de­vel­oped a model of the thing your­self, against which you can judge the pop­u­lar nar­ra­tives.

Another thing the para­ble about the bricks tells us: un­der­stand­ing is not a bi­nary yes/no”. It has lay­ers of depth. My friend un­der­stood Pythagoras’s the­o­rem far more deeply than I did; he could prove it six dif­fer­ent ways and had sim­ply thought about it for longer.

The sim­plest things can re­ward close study. Michael Nielsen has a nice ex­am­ple of this — the equals sign:

I first re­ally ap­pre­ci­ated this af­ter read­ing an es­say by the math­e­mati­cian Andrey Kolmogorov. You might sup­pose a great math­e­mati­cian such as Kolmogorov would be writ­ing about some very com­pli­cated piece of math­e­mat­ics, but his sub­ject was the hum­ble equals sign: what made it a good piece of no­ta­tion, and what its de­fi­cien­cies were. Kolmogorov dis­cussed this in lov­ing de­tail, and made many beau­ti­ful points along the way, e.g., that the in­ven­tion of the equals sign helped make pos­si­ble no­tions such as equa­tions (and al­ge­braic ma­nip­u­la­tions of equa­tions).

Prior to read­ing the es­say I thought I un­der­stood the equals sign. Indeed, I would have been of­fended by the sug­ges­tion that I did not. But the es­say showed con­vinc­ingly that I could un­der­stand the equals sign much more deeply. (link)The pho­tog­ra­pher Robert Capa ad­vised be­gin­ning pho­tog­ra­phers: If your pic­tures aren’t good enough, you’re not close enough”. (This is good fic­tion writ­ing ad­vice, by the way.)

It is also good ad­vice for un­der­stand­ing things. When in doubt, go closer.

Thanks to Jose-Luis Ricon for read­ing a draft of this es­say.

Follow me on Twitter: @nabeelqu


Read the original on nabeelqu.co »

2 348 shares, 18 trendiness, 0 words and 0 minutes reading time

Linux kernel in-tree Rust support


Read the original on lore.kernel.org »

3 337 shares, 27 trendiness, 166 words and 2 minutes reading time

The Hard Parts — Martin Kleppmann’s talks


A talk at Hydra, on­line (originally planned to be in Moscow, Russia), 06 Jul 2020

Conflict-free Replicated Data Types (CRDTs) are an in­creas­ingly pop­u­lar fam­ily of al­go­rithms for op­ti­mistic repli­ca­tion. They al­low data to be con­cur­rently up­dated on sev­eral repli­cas, even while those repli­cas are of­fline, and pro­vide a ro­bust way of merg­ing those up­dates back into a con­sis­tent state. CRDTs are used in geo-repli­cated data­bases, multi-user col­lab­o­ra­tion soft­ware, dis­trib­uted pro­cess­ing frame­works, and var­i­ous other sys­tems.

However, while the ba­sic prin­ci­ples of CRDTs are now quite well known, many chal­leng­ing prob­lems are lurk­ing be­low the sur­face. It turns out that CRDTs are easy to im­ple­ment badly. Many pub­lished al­go­rithms have anom­alies that cause them to be­have strangely in some sit­u­a­tions. Simple im­ple­men­ta­tions of­ten have ter­ri­ble per­for­mance, and mak­ing the per­for­mance good is chal­leng­ing.

In this talk Martin goes be­yond the in­tro­duc­tory ma­te­r­ial on CRDTs, and dis­cusses some of the hard-won lessons from years of re­search on mak­ing CRDTs work in prac­tice.


Read the original on martin.kleppmann.com »

4 240 shares, 18 trendiness, 2524 words and 21 minutes reading time

Testing Firefox more efficiently with machine learning – Mozilla Hacks

A browser is an in­cred­i­bly com­plex piece of soft­ware. With such enor­mous com­plex­ity, the only way to main­tain a rapid pace of de­vel­op­ment is through an ex­ten­sive CI sys­tem that can give de­vel­op­ers con­fi­dence that their changes won’t in­tro­duce bugs. Given the scale of our CI, we’re al­ways look­ing for ways to re­duce load while main­tain­ing a high stan­dard of prod­uct qual­ity. We won­dered if we could use ma­chine learn­ing to reach a higher de­gree of ef­fi­ciency.

At Mozilla we have around 50,000 unique test files. Each con­tain many test func­tions. These tests need to run on all our sup­ported plat­forms (Windows, Mac, Linux, Android) against a va­ri­ety of build con­fig­u­ra­tions (PGO, de­bug, ASan, etc.), with a range of run­time pa­ra­me­ters (site iso­la­tion, WebRender, multi-process, etc.).

While we don’t test against every pos­si­ble com­bi­na­tion of the above, there are still over 90 unique con­fig­u­ra­tions that we do test against. In other words, for each change that de­vel­op­ers push to the repos­i­tory, we could po­ten­tially run all 50k tests 90 dif­fer­ent times. On an av­er­age work day we see nearly 300 pushes (including our test­ing branch). If we sim­ply ran every test on every con­fig­u­ra­tion on every push, we’d run ap­prox­i­mately 1.35 bil­lion test files per day! While we do throw money at this prob­lem to some ex­tent, as an in­de­pen­dent non-profit or­ga­ni­za­tion, our bud­get is fi­nite.

So how do we keep our CI load man­age­able? First, we rec­og­nize that some of those ninety unique con­fig­u­ra­tions are more im­por­tant than oth­ers. Many of the less im­por­tant ones only run a small sub­set of the tests, or only run on a hand­ful of pushes per day, or both. Second, in the case of our test­ing branch, we rely on our de­vel­op­ers to spec­ify which con­fig­u­ra­tions and tests are most rel­e­vant to their changes. Third, we use an in­te­gra­tion branch.

Basically, when a patch is pushed to the in­te­gra­tion branch, we only run a small sub­set of tests against it. We then pe­ri­od­i­cally run every­thing and em­ploy code sher­iffs to fig­ure out if we missed any re­gres­sions. If so, they back out the of­fend­ing patch. The in­te­gra­tion branch is pe­ri­od­i­cally merged to the main branch once every­thing looks good.

A sub­set of the tasks we run on a sin­gle mozilla-cen­tral push. The full set of tasks were too hard to dis­tin­guish when scaled to fit in a sin­gle im­age.

These meth­ods have served us well for many years, but it turns out they’re still very ex­pen­sive. Even with all of these op­ti­miza­tions our CI still runs around 10 com­pute years per day! Part of the prob­lem is that we have been us­ing a naive heuris­tic to choose which tasks to run on the in­te­gra­tion branch. The heuris­tic ranks tasks based on how fre­quently they have failed in the past. The rank­ing is un­re­lated to the con­tents of the patch. So a push that mod­i­fies a README file would run the same tasks as a push that turns on site iso­la­tion. Additionally, the re­spon­si­bil­ity for de­ter­min­ing which tests and con­fig­u­ra­tions to run on the test­ing branch has shifted over to the de­vel­op­ers them­selves. This wastes their valu­able time and tends to­wards over-se­lec­tion of tests.

About a year ago, we started ask­ing our­selves: how can we do bet­ter? We re­al­ized that the cur­rent im­ple­men­ta­tion of our CI re­lies heav­ily on hu­man in­ter­ven­tion. What if we could in­stead cor­re­late patches to tests us­ing his­tor­i­cal re­gres­sion data? Could we use a ma­chine learn­ing al­go­rithm to fig­ure out the op­ti­mal set of tests to run? We hy­poth­e­sized that we could si­mul­ta­ne­ously save money by run­ning fewer tests, get re­sults faster, and re­duce the cog­ni­tive bur­den on de­vel­op­ers. In the process, we would build out the in­fra­struc­ture nec­es­sary to keep our CI pipeline run­ning ef­fi­ciently.

The main pre­req­ui­site to a ma­chine-learn­ing-based so­lu­tion is col­lect­ing a large and pre­cise enough re­gres­sion dataset. On the sur­face this ap­pears easy. We al­ready store the sta­tus of all test ex­e­cu­tions in a data ware­house called ActiveData. But in re­al­ity, it’s very hard to do for the rea­sons be­low.

Since we only run a sub­set of tests on any given push (and then pe­ri­od­i­cally run all of them), it’s not al­ways ob­vi­ous when a re­gres­sion was in­tro­duced. Consider the fol­low­ing sce­nario:

It is easy to see that the Test A” fail­ure was re­gressed by Patch 2, as that’s where it first started fail­ing. However with the Test B” fail­ure, we can’t re­ally be sure. Was it caused by Patch 2 or 3? Now imag­ine there are 8 patches in be­tween the last PASS and the first FAIL. That adds a lot of un­cer­tainty!

Intermittent (aka flaky) fail­ures also make it hard to col­lect re­gres­sion data. Sometimes tests can both pass and fail on the same code­base for all sorts of dif­fer­ent rea­sons. It turns out we can’t be sure that Patch 2 re­gressed Test A” in the table above af­ter all! That is un­less we re-run the fail­ure enough times to be sta­tis­ti­cally con­fi­dent. Even worse, the patch it­self could have in­tro­duced the in­ter­mit­tent fail­ure in the first place. We can’t as­sume that just be­cause a fail­ure is in­ter­mit­tent that it’s not a re­gres­sion.

The writ­ers of this post hav­ing a hard time.

In or­der to solve these prob­lems, we have built quite a large and com­pli­cated set of heuris­tics to pre­dict which re­gres­sions are caused by which patch. For ex­am­ple, if a patch is later backed out, we check the sta­tus of the tests on the back­out push. If they’re still fail­ing, we can be pretty sure the fail­ures were not due to the patch. Conversely, if they start pass­ing we can be pretty sure that the patch was at fault.

Some fail­ures are clas­si­fied by hu­mans. This can work to our ad­van­tage. Part of the code sher­if­f’s job is an­no­tat­ing fail­ures (e.g. intermittent” or fixed by com­mit” for fail­ures fixed at some later point). These clas­si­fi­ca­tions are a huge help find­ing re­gres­sions in the face of miss­ing or in­ter­mit­tent tests. Unfortunately, due to the sheer num­ber of patches and fail­ures hap­pen­ing con­tin­u­ously, 100% ac­cu­racy is not at­tain­able. So we even have heuris­tics to eval­u­ate the ac­cu­racy of the clas­si­fi­ca­tions!

Another trick for han­dling miss­ing data is to back­fill miss­ing tests. We se­lect tests to run on older pushes where they did­n’t ini­tially run, for the pur­pose of find­ing which push caused a re­gres­sion. Currently, sher­iffs do this man­u­ally. However, there are plans to au­to­mate it in cer­tain cir­cum­stances in the fu­ture.

We also need to col­lect data about the patches them­selves, in­clud­ing files mod­i­fied and the diff.  This al­lows us to cor­re­late with the test fail­ure data. In this way, the ma­chine learn­ing model can de­ter­mine the set of tests most likely to fail for a given patch.

Collecting data about patches is way eas­ier, as it is to­tally de­ter­min­is­tic. We it­er­ate through all the com­mits in our Mercurial repos­i­tory, pars­ing patches with our rust-parsep­atch pro­ject and an­a­lyz­ing source code with our rust-code-analy­sis pro­ject.

Now that we have a dataset of patches and as­so­ci­ated tests (both passes and fail­ures), we can build a train­ing set and a val­i­da­tion set to teach our ma­chines how to se­lect tests for us.

90% of the dataset is used as a train­ing set, 10% is used as a val­i­da­tion set. The split must be done care­fully. All patches in the val­i­da­tion set must be pos­te­rior to those in the train­ing set. If we were to split ran­domly, we’d leak in­for­ma­tion from the fu­ture into the train­ing set, caus­ing the re­sult­ing model to be bi­ased and ar­ti­fi­cially mak­ing its re­sults look bet­ter than they ac­tu­ally are.

For ex­am­ple, con­sider a test which had never failed un­til last week and has failed a few times since then. If we train the model with a ran­domly picked train­ing set, we might find our­selves in the sit­u­a­tion where a few fail­ures are in the train­ing set and a few in the val­i­da­tion set. The model might be able to cor­rectly pre­dict the fail­ures in the val­i­da­tion set, since it saw some ex­am­ples in the train­ing set.

In a real-world sce­nario though, we can’t look into the fu­ture. The model can’t know what will hap­pen in the next week, but only what has hap­pened so far. To eval­u­ate prop­erly, we need to pre­tend we are in the past, and fu­ture data (relative to the train­ing set) must be in­ac­ces­si­ble.

Visualization of our split be­tween train­ing and val­i­da­tion set.

We train an XGBoost model, us­ing fea­tures from both test, patch, and the links be­tween them, e.g:

* In the past, how of­ten did this test fail when the same files were touched?

* How far in the di­rec­tory tree are the source files from the test files?

* How of­ten in the VCS his­tory were the source files mod­i­fied to­gether with the test files?

The in­put to the model is a tu­ple (TEST, PATCH), and the la­bel is a bi­nary FAIL or NOT FAIL. This means we have a sin­gle model that is able to take care of all tests. This ar­chi­tec­ture al­lows us to ex­ploit the com­mon­al­i­ties be­tween test se­lec­tion de­ci­sions in an easy way. A nor­mal multi-la­bel model, where each test is a com­pletely sep­a­rate la­bel, would not be able to ex­trap­o­late the in­for­ma­tion about a given test and ap­ply it to an­other com­pletely un­re­lated test.

Given that we have tens of thou­sands of tests, even if our model was 99.9% ac­cu­rate (which is pretty ac­cu­rate, just one er­ror every 1000 eval­u­a­tions), we’d still be mak­ing mis­takes for pretty much every patch! Luckily the cost as­so­ci­ated with false pos­i­tives (tests which are se­lected by the model for a given patch but do not fail) is not as high in our do­main, as it would be if say, we were try­ing to rec­og­nize faces for polic­ing pur­poses. The only price we pay is run­ning some use­less tests. At the same time we avoided run­ning hun­dreds of them, so the net re­sult is a huge sav­ings!

As de­vel­op­ers pe­ri­od­i­cally switch what they are work­ing on the dataset we train on evolves. So we cur­rently re­train the model every two weeks.

After we have cho­sen which tests to run, we can fur­ther im­prove the se­lec­tion by choos­ing where the tests should run. In other words, the set of con­fig­u­ra­tions they should run on. We use the dataset we’ve col­lected to iden­tify re­dun­dant con­fig­u­ra­tions for any given test. For in­stance, is it re­ally worth run­ning a test on both Windows 7 and Windows 10? To iden­tify these re­dun­dan­cies, we use a so­lu­tion sim­i­lar to fre­quent item­set min­ing:

Collect fail­ure sta­tis­tics for groups of tests and con­fig­u­ra­tions

Calculate the support” as the num­ber of pushes in which both X and Y failed over the num­ber of pushes in which they both run

Calculate the confidence” as the num­ber of pushes in which both X and Y failed over the num­ber of pushes in which they both run and only one of the two failed.

We only se­lect con­fig­u­ra­tion groups where the sup­port is high (low sup­port would mean we don’t have enough proof) and the con­fi­dence is high (low con­fi­dence would mean we had many cases where the re­dun­dancy did not ap­ply).

Once we have the set of tests to run, in­for­ma­tion on whether their re­sults are con­fig­u­ra­tion-de­pen­dent or not, and a set of ma­chines (with their as­so­ci­ated cost) on which to run them; we can for­mu­late a math­e­mat­i­cal op­ti­miza­tion prob­lem which we solve with a mixed-in­te­ger pro­gram­ming solver. This way, we can eas­ily change the op­ti­miza­tion ob­jec­tive we want to achieve with­out in­va­sive changes to the op­ti­miza­tion al­go­rithm. At the mo­ment, the op­ti­miza­tion ob­jec­tive is to se­lect the cheap­est con­fig­u­ra­tions on which to run the tests.

A ma­chine learn­ing model is only as use­ful as a con­sumer’s abil­ity to use it. To that end, we de­cided to host a ser­vice on Heroku us­ing ded­i­cated worker dynos to ser­vice re­quests and Redis Queues to bridge be­tween the back­end and fron­tend. The fron­tend ex­poses a sim­ple REST API, so con­sumers need only spec­ify the push they are in­ter­ested in (identified by the branch and top­most re­vi­sion). The back­end will au­to­mat­i­cally de­ter­mine the files changed and their con­tents us­ing a clone of mozilla-cen­tral.

Depending on the size of the push and the num­ber of pushes in the queue to be an­a­lyzed, the ser­vice can take sev­eral min­utes to com­pute the re­sults. We there­fore en­sure that we never queue up more than a sin­gle job for any given push. We cache re­sults once com­puted. This al­lows con­sumers to kick off a query asyn­chro­nously, and pe­ri­od­i­cally poll to see if the re­sults are ready.

We cur­rently use the ser­vice when sched­ul­ing tasks on our in­te­gra­tion branch. It’s also used when de­vel­op­ers run the spe­cial mach try auto com­mand to test their changes on the test­ing branch. In the fu­ture, we may also use it to de­ter­mine which tests a de­vel­oper should run lo­cally.

Sequence di­a­gram de­pict­ing the com­mu­ni­ca­tion be­tween the var­i­ous ac­tors in our in­fra­struc­ture.

From the out­set of this pro­ject, we felt it was cru­cial that we be able to run and com­pare ex­per­i­ments, mea­sure our suc­cess and be con­fi­dent that the changes to our al­go­rithms were ac­tu­ally an im­prove­ment on the sta­tus quo. There are ef­fec­tively two vari­ables that we care about in a sched­ul­ing al­go­rithm:

The amount of re­sources used (measured in hours or dol­lars).

The re­gres­sion de­tec­tion rate. That is, the per­cent­age of in­tro­duced re­gres­sions that were caught di­rectly on the push that caused them. In other words, we did­n’t have to rely on a hu­man to back­fill the fail­ure to fig­ure out which push was the cul­prit.

sched­uler ef­fec­tive­ness = 1000 * re­gres­sion de­tec­tion rate / hours per push

The higher this met­ric, the more ef­fec­tive a sched­ul­ing al­go­rithm is. Now that we had our met­ric, we in­vented the con­cept of a shadow sched­uler”. Shadow sched­ulers are tasks that run on every push, which shadow the ac­tual sched­ul­ing al­go­rithm. Only rather than ac­tu­ally sched­ul­ing things, they out­put what they would have sched­uled had they been the de­fault. Each shadow sched­uler may in­ter­pret the data re­turned by our ma­chine learn­ing ser­vice a bit dif­fer­ently. Or they may run ad­di­tional op­ti­miza­tions on top of what the ma­chine learn­ing model rec­om­mends.

Finally we wrote an ETL to query the re­sults of all these shadow sched­ulers, com­pute the sched­uler ef­fec­tive­ness met­ric of each, and plot them all in a dash­board. At the mo­ment, there are about a dozen dif­fer­ent shadow sched­ulers that we’re mon­i­tor­ing and fine-tun­ing to find the best pos­si­ble out­come. Once we’ve iden­ti­fied a win­ner, we make it the de­fault al­go­rithm. And then we start the process over again, cre­at­ing fur­ther ex­per­i­ments.

The early re­sults of this pro­ject have been very promis­ing. Compared to our pre­vi­ous so­lu­tion, we’ve re­duced the num­ber of test tasks on our in­te­gra­tion branch by 70%! Compared to a CI sys­tem with no test se­lec­tion, by al­most 99%! We’ve also seen pretty fast adop­tion of our mach try auto tool, sug­gest­ing a us­abil­ity im­prove­ment (since de­vel­op­ers no longer need to think about what to se­lect). But there is still a long way to go!

We need to im­prove the mod­el’s abil­ity to se­lect con­fig­u­ra­tions and de­fault to that. Our re­gres­sion de­tec­tion heuris­tics and the qual­ity of our dataset needs to im­prove. We have yet to im­ple­ment us­abil­ity and sta­bil­ity fixes to mach try auto.

And while we can’t make any promises, we’d love to pack­age the model and ser­vice up in a way that is use­ful to or­ga­ni­za­tions out­side of Mozilla. Currently, this ef­fort is part of a larger pro­ject that con­tains other ma­chine learn­ing in­fra­struc­ture orig­i­nally cre­ated to help man­age Mozilla’s Bugzilla in­stance. Stay tuned!

If you’d like to learn more about this pro­ject or Firefox’s CI sys­tem in gen­eral, feel free to ask on our Matrix chan­nel, #firefox-ci:mozilla.org.


Read the original on hacks.mozilla.org »

5 240 shares, 30 trendiness, 99 words and 1 minutes reading time

Do you know how much your computer can do in a second?

Let’s find out how well you know com­put­ers! All of these pro­grams have a vari­able NUMBER in them. Your mis­sion: guess how big NUMBER needs to get be­fore the pro­gram takes 1 sec­ond to run.

You don’t need to guess ex­actly: they’re all be­tween 1 and a bil­lion. Just try to guess the right or­der of mag­ni­tude! A few notes:

* If the an­swer is 38,000, both 10,000 and 100,000 are

con­sid­ered cor­rect an­swers. The goal is to not be wrong by

more than 10x :)

* We know com­put­ers have dif­fer­ent disk & net­work & CPU

speeds! We’re try­ing to get you to tell the dif­fer­ence

be­tween code that can run 10 times/​s and 100000 times/​s. A

newer com­puter won’t make your code run 1000x faster :)

* That said, all this was run on a new lap­top with a fast

SSD and a sketchy net­work con­nec­tion. The C code was com­piled with gcc -O2.

Good luck! We were sur­prised by a lot of these. We’ll be anony­mously col­lect­ing your an­swers, so ex­pect some graphs in the fu­ture! =D


Read the original on computers-are-fast.github.io »

6 237 shares, 27 trendiness, 112 words and 1 minutes reading time

Powerful new route planner that prefers greenery and can generate round trip routes of a specified distance

Our rout­ing al­go­rithm prefers paths that go through parks, forests or by wa­ter, and avoids busy roads wher­ever pos­si­ble.

Here’s a few things you can do with Trail Router:

Manually cre­ate your own point-to-point route that prefers na­ture.

Choose whether you’d pre­fer routes that in­volve na­ture, well-lit streets or a lack of hills.

We’re not per­fect! Help im­prove Trail Router by re­port­ing an is­sue here.

A road is un­safe for run­ners ( See FAQ

Thank you for re­port­ing an is­sue!

We will re­view your is­sue and be in touch via email. Please note that cor­rec­tions to map­ping data can take a cou­ple of weeks to fil­ter through.


Read the original on trailrouter.com »

7 186 shares, 19 trendiness, 249 words and 2 minutes reading time

Scientists Say You Can Cancel the Noise but Keep Your Window Open

Dr. Lam ex­plained that in places like Singapore, we want to keep the win­dows open as much as pos­si­ble” to re­duce the use of car­bon-in­ten­sive air-con­di­tion­ers and to pre­vent buildup of stale air that can pose health risks for some peo­ple.

But with win­dows open, the con­stant din from city traf­fic, trains, jets pass­ing over­head and con­struc­tion equip­ment can rat­tle apart­ments. The Anti-Noise Control Window, as it is called, is the sonic equiv­a­lent of shut­ting a win­dow.

With any sound, the best way to re­duce it is at the source, like a gun’s si­lencer. So the re­searchers treated the win­dow aper­ture it­self as the noise source, be­cause most noise en­ters a room that way.

The sys­tem uses a mi­cro­phone out­side the win­dow to de­tect the re­peat­ing sound waves of the of­fend­ing noise source, which is reg­is­tered by a com­puter con­troller. That in turn de­ci­phers the proper wave fre­quency needed to neu­tral­ize the sound, which is trans­mit­ted to the ar­ray of speak­ers on the in­side of the win­dow frame.

The speak­ers then emit the proper anti” waves, which can­cel out the in­com­ing waves, and there you have it: near bliss­ful si­lence.

If you sit in the room, you get that same feel­ing like when you flick on the switch of noise-can­cel­ing ear­phones,” Dr. Lam said, splay­ing his hands to de­note the calm­ing ef­fect.

The sys­tem is best at at­ten­u­at­ing the au­di­ble blasts from the types of steady noise sources found within the op­ti­mal fre­quency range.


Read the original on www.nytimes.com »

8 184 shares, 8 trendiness, 77 words and 1 minutes reading time


Darwin is the Open Source op­er­at­ing sys­tem from Apple that forms the ba­sis for Mac OS X and PureDarwin. PureDarwin is a com­mu­nity pro­ject that aims to make Darwin more us­able (some peo­ple think of it as the in­for­mal suc­ces­sor to OpenDarwin).

One cur­rent goal of this pro­ject is to pro­vide a use­ful bootable ISO/VM of Darwin 10.x

Come Join our fo­rum over at https://​www.pd-devs.org/

See the Wiki for more in­for­ma­tion.


Read the original on github.com »

9 170 shares, 40 trendiness, 0 words and 0 minutes reading time

We’ve de­tected that JavaScript is dis­abled in your browser. Would you like to pro­ceed to legacy Twitter?


Read the original on twitter.com »

10 159 shares, 12 trendiness, 1850 words and 16 minutes reading time

Why Are Toys Such a Bad Business?

This is the once-a-week free edi­tion of The Diff, the newslet­ter about in­flec­tions in fi­nance and tech­nol­ogy. The free edi­tion goes out to 8,221 sub­scribers, up 211 week-over-week. This week’s sub­scribers-only posts:

* The UK as a Science Hub is an up­date on Boris Johnson’s plan (or, if you pre­fer, Dominic Cumming’s scheme) to make Britain a sci­en­tific pow­er­house. The out­lines of the plan aren’t new, but the op­por­tu­nity is.

* The Equity Risk Premium at 0% Interest looks at the im­pli­ca­tions of low real rates for tech com­pa­nies. In equi­lib­rium, low rates are good for eq­ui­ties be­cause they raise the pre­sent value of fu­ture cash flows. But an­other way of say­ing this is that, in fi­nan­cial terms, low rates mean the fu­ture hap­pens all at once.

* Globalization: A Toy Story) is a pre­quel to to­day’s note, dis­cussing the his­tory of Hong Kong’s toy in­dus­try. Hong Kong’s toy in­dus­try was ba­si­cally nonex­is­tent in 1945, the biggest in the world by 1972, and con­sis­tently lost share to China from the 80s on­ward. It’s a case study in how glob­al­iza­tion works.

* The Depressing Bull Thesis for Rocket Mortgage is a writeup of Rocket, the largest mort­gage orig­i­na­tor in the US, which re­cently filed to go pub­lic. Fewer red flags than ex­pected, but it’s partly dri­ven by a dire fi­nan­cial bet.

* Why Are Toys Such a Bad Business?

* V-Shaped Recovery is here… just not evenly dis­trib­uted.

Early-stage in­vestors some­times use the heuris­tic that if a prod­uct gets de­rided as a toy, it’s worth in­vest­ing in. That model would have got­ten you into PCs in the 70s, the Internet in the early 90s, so­cial net­works when the good ones were pri­vately-held, cryp­tocur­ren­cies, and drones. The risk is in­vest­ing in ac­tual toy com­pa­nies, which is usu­ally a ter­ri­ble de­ci­sion. Hasbro stock has­n’t done any­thing for half a decade, and Mattel trades where it did in the early 90s. JAKKS Pacific has de­stroyed most of its share­hold­ers’ wealth, and Funko is work­ing on the same.

This is not a new phe­nom­e­non, ei­ther. The biggest toy com­pany in the US in the 50s was Louis Marx & Company, whose founder made the cover of Time. Sales de­clined slightly over the next decade, and faster af­ter that; the com­pany was bank­rupt in 1980. Coleco rode the Cabbage Patch Kids trend in the mid-80s—in 1985, they had the high­est re­turn on eq­uity of any com­pany in the Fortune 500—but they were bank­rupt by 1988.

The record is no bet­ter for re­tail­ers. Toys R Us is bank­rupt, of course, and they fol­lowed FAO Shwarz, KB Toys, Right Start, and Zany Brainy.

The toy in­dus­try has not been kind to in­vestors, at any level.

There are a few rea­sons, and a few rel­e­vant lessons.

First, the toy busi­ness op­er­ates on an an­nual cy­cle. Historically, about 40% of toy sales hap­pen dur­ing the hol­i­day sea­son, and about half of those were in the two weeks be­fore Christmas. (That’s a dated sta­tis­tic, from about twenty years ago. Discretionary re­tail sales in gen­eral have got­ten spikier, since more shop­pers are used to fast, free ship­ping. With Amazon Prime, the Christmas shop­ping sea­son starts on December 22nd or so.) 84% of US toy sales come from China, trans­ported by a mix of ships and air freight, so they need to be or­dered months in ad­vance.

And they have to be mar­keted: while cheap toys can com­pete on price, the higher-mar­gin ones only get sold when there’s an ef­fec­tive ad cam­paign. Mattel cre­ated this model (and over­turned Louis Marx’s price-first ap­proach) when they spent their en­tire net worth on a one-year spon­sor­ship of The Mickey Mouse Club in 1955.

TV ad cam­paigns, too, tend to be pur­chased in ad­vance. About half of TV ad spend­ing is al­lo­cated to the up­fronts—booked March through May to be de­liv­ered by the end of the year.

This locks toy com­pa­nies into a chal­leng­ing bet. Every year, they have to a) pre­dict trends, b) in­vent them, and c) com­mit cap­i­tal to them. All with­out know­ing how the rest of the year will turn out. Since toy trends ex­ist, but don’t last for very long, they have to in­vent new prod­ucts every year—but the tech­no­log­i­cal state of the art does­n’t ad­vance very fast. It has all the volatil­ity of tech, with­out the progress.

A hand­ful of com­pa­nies have made se­ri­ous money in toys, or, rather, in toy-like or toy-ad­ja­cent busi­nesses. The video game in­dus­try has gen­er­ally done well. Disney turns a profit. And Games Workshop has a nice lit­tle busi­ness. (I wrote up in The Diff in April—note that I’ve since sold the stock, just for val­u­a­tion rea­sons) . Lego, too, is a great busi­ness, worth an es­ti­mated $15bn.

What these com­pa­nies have in com­mon is that they es­cape the de­mo­graphic trap that toy man­u­fac­tur­ers are locked into. Every year, there’s a new co­hort of six-year-olds, and they need some­thing that a) did­n’t ex­ist last year, but b) ap­peals to time­less six-year-old sen­si­bil­i­ties. They don’t have much brand loy­alty, be­cause a year later the same toy is a toy for lit­tle kids. Each of these suc­cess­ful com­pa­nies beats that in a dif­fer­ent way:

* Video games’ av­er­age age has trended older over time, so in­stead of mar­ket­ing to more trend-sen­si­tive young peo­ple, they’re mar­ket­ing to more dol­lar-in­sen­si­tive not-so-young peo­ple.

* Disney has a gen­er­a­tional loop, of which toys are a small part. Movies and stream­ing video get kids hooked on Disney char­ac­ters, which can be mon­e­tized at much higher dol­lar val­ues through their parks. (See my writeup here for much more.)

* Games Workshop and Lego have a very healthy prod­uct dy­namic: the ones you al­ready own are an eco­nomic com­ple­ment to the ones you buy. And Lego clearly de­signs their mar­ket­ing around hit­ting two gen­er­a­tions, too: at the Lego Store, $30 Rise of Skywalker-themed Lego sets are at a kids’ eye level. The $800 set based on the orig­i­nal tril­ogy is po­si­tioned at an adult’s eye level.

These com­pa­nies have some­thing else in com­mon: they own their core in­tel­lec­tual prop­erty. Video game pub­lish­ers do make games based on su­per­heroes and sports leagues, and Lego cer­tainly has branded sets, but the core of each busi­ness is IP owned by the com­pany it­self; the li­censed prod­ucts are a lu­cra­tive side busi­ness: it’s much eas­ier for a video game com­pany to re-skin char­ac­ters than for a movie com­pany to start a video game stu­dio. As a case study, one pop­u­lar game was orig­i­nally in­tended to be set in the Game of Thrones uni­verse, but ended up us­ing in-house IP in­stead. Disney, of course, sells toys based on its own char­ac­ters. And while some Lego sets are as­so­ci­ated with out­side brands at the point of pur­chase, they in­evitably end up be­ing fun­gi­ble with other Legos.

Because it’s a hit-dri­ven in­dus­try, toy com­pa­nies that suc­ceed can be im­mensely prof­itable for a while. The prob­lem is that the dif­fer­ence be­tween a cul­tural land­mark and a fad is vis­i­ble af­ter a decade or so, while the de­ci­sion of how much to or­der and how much to spend on mar­ket­ing has to hap­pen every year re­gard­less. So toy com­pa­nies with a hit prod­uct in year N tend to be bank­rupt com­pa­nies writ­ing down the value of their in­ven­tory to ~$0 in year N+3 or N+5.

You should­n’t have to take big risks to make big re­turns. So when data from Citibank shows that art has out­per­formed the S&P by 180% since 2000 with the least volatil­ity of any ma­jor as­set class—we’re in­clined to no­tice.

The ul­tra-wealthy have in­vested in art for cen­turies, to the tune of over $1.7 tril­lion in to­tal value—so why can’t the rest of us?

Masterworks lets any­one in­vest in paint­ings by some of the most suc­cess­ful artists in his­tory like Banksy, Warhol, Basquiat, and more, in just a few clicks. The only catch? There’s cur­rently a back­log of over 25,000 of peo­ple ap­ply­ing for mem­ber­ship, but you can skip the wait­list by sign­ing up to­day.*

… just not evenly dis­trib­uted. And not likely to last. In Japan, Uniqlo ex­pects Japanese sales to be up 25% Y/Y in their August quar­ter, af­ter a 15% de­cline last quar­ter. Their re­gional es­ti­mates are very much virus-dri­ven, with op­ti­mism in China and pes­simism in coun­tries see­ing a sec­ond wave, or, in the USs case, a 1.5th wave. And world­wide PC ship­ments grew in Q2, mostly due to the one-time build-out of home of­fices and Zoom-based schools. In China, auto sales were up 10% in Q2 ($), and China’s cop­per smelt­ing is also ris­ing ($).

Inventory re­stock­ing used to be a sig­nif­i­cant dri­ver of GDP growth: when the econ­omy slowed, com­pa­nies had too much in­ven­tory on hand, and had to cut jobs to work through the ex­cess. Once they ran out, they had to re­hire fast. Now, sup­ply and de­mand for man­u­fac­tur­ing are lo­cated in dif­fer­ent places (with dif­fer­ent poli­cies), and com­pa­nies are more averse to hold­ing in­ven­tory for long pe­ri­ods, so this model is­n’t as de­scrip­tive or pre­dic­tive as it once was. When the peo­ple get­ting fired are the ones pro­vid­ing de­mand, it’s easy for a re­ces­sion to feed on it­self, and easy for a re­cov­ery to boot­strap it­self, too. When those groups are in dif­fer­ent coun­tries, and when the swings in in­ven­tory are more muted, it’s less of a fac­tor, lead­ing to fewer re­ces­sions but much slower re­bounds.

The Indonesion gov­ern­ment is strapped for cash, and needs to spend heav­ily to mit­i­gate the ef­fects of Covid-19. But the gov­ern­ment is not great at col­lect­ing taxes (taxes are 11-12% of GDP. For com­par­i­son, Mexico and the Netherlands have sim­i­lar-sized economies, and col­lect 16% and 39%, re­spec­tively). But tech com­pa­nies are great at col­lect­ing taxes on on­line com­merce, and tend to charge close to the Laffer peak. So Indonesia is out­sourc­ing tax­a­tion to them ($) by im­pos­ing a 10% value-added tax on large Internet com­pa­nies. Google, Facebook, and Netflix have built their own tax col­lec­tion” ap­pa­ra­tus, and are bet­ter at catch­ing tax-evaders and charg­ing the right amount. As it turns out, tax-farm­ing was­n’t a ter­ri­ble idea, just a few cen­turies early

In other tax news, Chinese main­lan­ders work­ing in Hong Kong sud­denly owe the main­land’s 45% tax rates rather than Hong Kong’s 15%. China seems to al­ter­nate—on a daily ba­sis—be­tween want­ing Hong Kong to be a fi­nan­cial cen­ter they con­trol and want­ing to use their con­trol to end Hong Kong’s sta­tus as a fi­nan­cial cen­ter.

The dol­lar, by virtue of be­ing the world’s most-used cur­rency, is the cur­rency that least rep­re­sents how cur­ren­cies work. Since it’s a re­serve cur­rency, dol­lars are de­manded by peo­ple who don’t earn them or spend them, but who know they’ll need them, so the US has less con­trol over the value of its money than any other place. To para­phrase John Connally, it’s every­one’s cur­rency but America’s prob­lem.

For ex­am­ple, there’s no way the US could get away with this ($):

Africa’s most pop­u­lous na­tion has long main­tained sev­eral ex­change rates. In ad­di­tion to the in­ter­bank and black-mar­ket rates, there are of­fi­cial rates for con­sumers want­ing dol­lars for school and med­ical fees abroad, for Muslims mak­ing the pil­grim­age to Saudi Arabia, and for peo­ple wish­ing to buy hard cur­rency at ex­change bu­reaux.

That’s a very clever setup. Smaller coun­tries can use a tiered ex­change-rate sys­tem to mod­er­ately en­cour­age or dis­cour­age cer­tain be­hav­iors, or to dole out fa­vors to par­tic­u­lar groups. Dollars are so liq­uid, and used in so many places, that the US has to take a more bi­nary ap­proach, of al­low­ing or ban­ning trans­ac­tions; taxes get routed around.

Alpha Architect has a neg­a­tive view on trea­suries, ar­gu­ing that they’re not a good di­ver­si­fier and that yields are too low to jus­tify own­ing them. The piece goes into de­tail on how trea­suries func­tion as in­sur­ance (not al­ways!), and how they’re mostly owned by price-in­sen­si­tive buy­ers like reg­u­lated in­sur­ance com­pa­nies and cen­tral banks. All true. But the most im­por­tant line in the piece is: Okay, Treasuries Aren’t Compelling: What Are My Alternatives? Answer: Nothing.” Investments are al­ways ex­pressed in rel­a­tive terms. In an ag­ing world, we should­n’t ex­pect any­thing to be cheap be­cause there’s so much de­mand for sav­ings.

Chinese CSI 300 in­dex dropped 1.8% in the last ses­sion, and it’s now up only 14% since late June. Bloomberg pro­files the wild mar­ket, with plenty of pull quotes from new in­vestors (“There’s no way I can lose,” sounds like a clas­sic bull mar­ket line, but there’s a tinge of des­per­a­tion there). One com­pany, QuantumCTek, rose 1,000% in its IPO ($).

2K games is try­ing to push video game prices above the de facto ceil­ing of $60/copy. Video game prices have been de­clin­ing in real terms, in part be­cause of cheaper man­u­fac­tur­ing and dis­tri­b­u­tion. As the video game in­dus­try gets more ma­ture, pre­dict­ing sales for any given ti­tle gets eas­ier, which en­cour­ages pub­lish­ers to in­vest more in pro­duc­tion. So cost de­fla­tion in one part of the mar­ket is off­set by cost in­fla­tion in an­other.


Read the original on diff.substack.com »

To add this web app to your iOS home screen tap the share button and select "Add to the Home Screen".

10HN is also available as an iOS App

If you visit 10HN only rarely, check out the the best articles from the past week.

If you like 10HN please leave feedback and share

Visit pancik.com for more.