In Masters of Doom, a book about the game de­vel­op­ment com­pany id Software and its in­flu­ence on pop­u­lar cul­ture, David Kushner re­flected on the un­con­ven­tional work­ing style of the com­pa­ny’s ace coder, John Carmack.

To in­crease his pro­duc­tiv­ity and find a break from dis­trac­tion while work­ing on his break­through Quake en­gine, Carmack adopted an ag­gres­sive tac­tic — he be­gan grad­u­ally shift­ing the start of his work­day.

Eventually, he was start­ing his pro­gram­ming in the evening and fin­ish­ing just be­fore dawn.

These un­in­ter­rupted stretches of si­lence, iso­la­tion, and deep work al­lowed Carmack to rein­vent gam­ing with the world’s first light­ning-fast 3D game en­gine.

While Carmack’s sched­ule may have made it harder for the rest of his team to reach him at times, the value he pro­duced while work­ing at his full cog­ni­tive ca­pac­ity far out­weighed that in­con­ve­nience.

The ‘Carmacks’ of the world — those whose work in­volves writ­ing code, get­ting cre­ative, and prob­lem-solv­ing — op­er­ate on what tech in­vestor Paul Graham refers to as the maker sched­ule. In his 2009 es­say ti­tled “Maker’s Schedule, Manager’s Schedule”, he ar­gued that peo­ple who make things op­er­ate on a dif­fer­ent sched­ule from those who man­age things.

Managers’ days are “cut into one-hour in­ter­vals. You can block off sev­eral hours for a sin­gle task if you need to, but by de­fault, you change what you’re do­ing every hour.”Mak­ers, on the other hand, “generally pre­fer to use time in units of half a day at least. You can’t write or pro­gram well in units of an hour. That’s barely enough time to get started.”

For man­agers, in­ter­rup­tions in the form of meet­ings, phone calls, and Slack no­ti­fi­ca­tions are nor­mal. For some­one on the maker sched­ule, how­ever, even the slight­est dis­trac­tion can have a dis­rup­tive ef­fect.

Research shows that it takes as long as 30 min­utes for mak­ers to get into the flow and we can’t sim­ply switch from one task to an­other. Instead, it changes the whole mode in which we work and this con­stant con­text switch­ing pre­vents our brains from fully en­gag­ing the task at hand. A study con­ducted by Gloria Marks, a Professor of Informatics at the University of California, re­vealed that it takes us an av­er­age of 23 min­utes and 15 sec­onds to re­fo­cus on a task af­ter an in­ter­rup­tion, and even when we do, we ex­pe­ri­ence a de­crease in pro­duc­tiv­ity.

A sin­gle standup meet­ing can, there­fore, blow a whole af­ter­noon by break­ing it into two pieces each too small to do any­thing sub­stan­tial in. And if you know your work is go­ing to be in­ter­rupted, why bother start­ing any­thing am­bi­tious?

Working in an open of­fice ren­ders us even more vul­ner­a­ble.

Separately, man­agers and mak­ers work fine. Friction hap­pens when they meet. And since most pow­er­ful peo­ple op­er­ate on the man­ager sched­ule, they’re in a po­si­tion to force every­one to adapt to their sched­ule, po­ten­tially wreck­ing the mak­ers’ pro­duc­tiv­ity.

And the pre­dictable re­sult is — al­most no or­ga­ni­za­tions to­day sup­port maker sched­ules.

The rea­sons why most man­agers fail to ac­com­mo­date the mak­ers and their sched­ule are quite straight­for­ward.

Instant mes­sag­ing tools like Slack trans­formed the way we com­mu­ni­cate at work, em­pow­er­ing the man­agers to col­lab­o­rate with mak­ers at their con­ve­nience. The work style these tools en­able fits the man­agers’ sched­ule so neatly, that they of­ten don’t see the costs to the maker. Immediate re­sponse be­comes the im­plicit ex­pec­ta­tion, with barely any bar­ri­ers or re­stric­tions in place.

And in the ab­sence of bar­ri­ers, con­ve­nience al­ways wins.

The rea­son why many man­agers fail to see and ad­dress this prob­lem is that they are used to look­ing at com­mu­ni­ca­tion and as­sume it’s a good thing. Because they see ac­tiv­ity. People are at­tend­ing meet­ings, talk­ing to each other, the on­line pres­ence in­di­ca­tors are bright green. Clearly, a lot of work is hap­pen­ing!

At the same time, real work is not get­ting done. Meaningful work is usu­ally done qui­etly and in soli­tude.

Most mak­ers don’t have the lev­els of con­trol and au­ton­omy nec­es­sary to block out half a day with­out any calls or meet­ings. So in­stead of push­ing the is­sue with the man­age­ment, we try to com­pen­sate by at­tempt­ing to mul­ti­task — un­for­tu­nately, that rarely works. Building con­text can take hours, and con­text switch­ing be­tween com­mu­ni­ca­tion and cre­ative work only kills the qual­ity of both.

Being busy feels like work to us, but it’s not the work that needs to be done.

In many com­pa­nies, the choice that the mak­ers face is that be­tween cav­ing to the man­agers and sac­ri­fic­ing their deep work time and pro­duc­tiv­ity — or of­fend­ing peo­ple.

But there are smarter com­pro­mises.

The first tech­nique that Paul Graham rec­om­mends to sim­u­late the man­ager’s sched­ule within the mak­er’s is “office hours”.

Office hours are chunks of time that mak­ers set aside for meet­ings, while the rest of the time they are free to go into a Do Not Disturb mode. Managers get their (brief) face time with the mak­ers on their team, while mak­ers get long stretches of time to get stuff done.

During his time as a tech­ni­cal lead at Buffer, Harrison Harnisch de­cided to ap­ply this con­cept to his sched­ule, split­ting his week up, and set­ting clear ex­pec­ta­tions about how a day should be treated. On Mondays and Fridays, he fo­cused solely on col­lab­o­rat­ing with his team, while re­serv­ing the rest of the week for heads-down cod­ing.

We have adopted a sim­i­lar sched­ule at Nuclino, re­serv­ing sev­eral days per week for our “maker time” while work­ing from home. It does­n’t mean that we ig­nore all mes­sages and only look up from our work when some­thing is on fire — but the gen­eral ex­pec­ta­tion is that it’s okay to not be im­me­di­ately avail­able to your team­mates when you are fo­cus­ing on your work.

“It is im­por­tant to note that deep work time can be in­ter­rupted by things that are both ur­gent and im­por­tant. However, treat­ing every ques­tion as ur­gent is likely to do more harm than good.”

It’s a nat­ural knee-jerk re­ac­tion for many man­agers to sched­ule a meet­ing when­ever a de­ci­sion needs to be made. Most of the time, such meet­ings quickly morph into ad hoc group brain­storm­ing ses­sions that may feel pro­duc­tive be­cause of all the talk­ing, but at the end of the day yield no tan­gi­ble re­sults, dis­rupt­ing every­one’s work for no good rea­son.

On the days that are re­served for col­lab­o­ra­tion, it does not al­ways need to hap­pen syn­chro­nously. Nor does it have to be face-to-face for it to be mean­ing­ful and pro­duc­tive.

Instead, com­mu­ni­ca­tion can hap­pen at a qui­eter asyn­chro­nous fre­quency in the form of thought­ful, writ­ten dis­cus­sions rather than soul-suck­ing meet­ings or er­ratic one-line-at-a-time chat mes­sages.

“People think it’s ef­fi­cient to dis­trib­ute in­for­ma­tion all at the same time to a bunch of peo­ple around a room. But it’s ac­tu­ally a lot less ef­fi­cient than dis­trib­ut­ing it asyn­chro­nously by writ­ing it up and send­ing it out and let­ting peo­ple ab­sorb it when they’re ready to so it does­n’t break their days into smaller bits.”

In our ex­pe­ri­ence, the best way to pre­vent a use­less meet­ing is to write up our goals and thoughts first. Despite work­ing in the same of­fice, our team at Nuclino has con­verted nearly all of our meet­ings into asyn­chro­nously writ­ten re­ports.

Not only does that pre­serve a de­tailed log — every meet­ing and pro­ject we’ve ever had is neatly doc­u­mented — it also helps every team mem­ber have a say, prop­erly ex­press their thoughts, and ab­sorb the in­put of oth­ers at a time and pace that is con­ve­nient for them.

A lot of the in­ter­rup­tions hap­pen be­cause peo­ple have repet­i­tive ques­tions and can’t find the an­swers on their own. If the is­sue is a blocker, hav­ing to wait till the “office hours” start can be frus­trat­ing.

The most straight­for­ward way to ad­dress this is to build a team knowl­edge base. Not only does that min­i­mize the num­ber of repet­i­tive ques­tions bounced around the of­fice, it al­lows new team mem­bers to ba­si­cally on­board them­selves.

But at the end of the day, it’s a mat­ter of cul­ture. None of these rules would work if the man­age­ment fails to see that mak­ers need to fol­low a dif­fer­ent sched­ule — and to make an ef­fort to re­spect it.

The truth is, though there is a time and place for syn­chro­nous, in­stant, and face-to-face com­mu­ni­ca­tion, that time is not all the time. In fact, very few things are ur­gent enough to jus­tify the po­ten­tial cost of an in­ter­rup­tion. Most are triv­ial. And while the of­fices we work in and the col­lab­o­ra­tion tools we use may nudge us to adopt the ASAP cul­ture, be­ing al­ways avail­able and keep­ing busy are not sus­tain­able sub­sti­tutes for chal­leng­ing, thought­ful work.

Keep calm and fol­low the no­hello rule.

Google has be­gun test­ing their up­com­ing ex­ten­sion man­i­fest V3 in the the lat­est Chrome Canary build, and with this ini­tial ‘alpha’ re­lease, de­vel­op­ers can be­gin test­ing their ex­ten­sions un­der the up­com­ing spec­i­fi­ca­tion.

In a post to the Chromium Extensions Google group, Simeon Vincent, a Google Developer Advocate for Chrome Extensions, stated that as of October 31st a de­vel­oper pre­view of the ex­ten­sion man­i­fest v3 is now avail­able in the Chrome 80 Canary build.

“Think of it as an early al­pha. The “dev pre­view” is the first op­por­tu­nity for ex­ten­sions de­vel­op­ers to start ex­per­i­ment­ing with a work-in-progress ver­sion of the MV3 plat­form.

We’re far from fin­ished with the im­ple­men­ta­tion work on the MV3 plat­form, so first and fore­most ex­pect changes.

As for what’s chang­ing, the four big-ticket items in MV3 are:

The declarativeNetRequest API has al­ready been avail­able for ex­per­i­men­ta­tion in Chrome Canary and we’re con­tin­u­ing to it­er­ate on it’s ca­pa­bil­i­ties

As part of this launch, Google has cre­ated a Mi­grat­ing to Manifest V3 guide that de­vel­op­ers can use to mi­grate their ex­ist­ing ex­ten­sions.

The most con­tro­ver­sial as­pect of the ex­ten­sion man­i­fest v3 is the up­com­ing changes to the we­bRe­quest API. In v3, Google has changed the API so that ex­ten­sions can only mon­i­tor browser con­nec­tions, but not mod­ify any of the con­tent be­fore it’s dis­played.

Instead Google wants de­vel­op­ers to use the de­clar­a­tiveN­e­tRe­quest API, which has the browser, not the ex­ten­sion, strip con­tent or re­sources from a vis­ited web sites.  This API, though, has a limit of 30,000 rules that can be cre­ated.

Unfortunately, this change will break pop­u­lar ad block­ers such as uBlock Origin, which rely on the orig­i­nal func­tion­al­ity of the we­bRe­quest API and need more rules than are avail­able in the de­clar­a­tiveN­e­tRe­quest API.

If you are us­ing the cur­rent Chrome Canary build you can test the new changes by cre­at­ing your own ex­ten­sion and set­ting its man­i­fest ver­sion to 3.

BrowserNative.com, who first re­ported about the de­vel­oper pre­view launch, shared with BleepingComputer a test ex­ten­sion that can be used to test the new changes.

For ex­am­ple, in the ex­ten­sion man­i­fest.json file be­low, the ver­sion has been set to 3 and it’s us­ing a back­ground.scripts call.

As back­ground.scripts is no longer sup­ported, try­ing to load the ex­ten­sion will gen­er­ate an er­ror, as shown be­low, that states you need to “use the “background.service_worker” key in­stead”.

If you switch the ex­ten­sion to use a ser­vice_­worker in­stead then the ex­ten­sion loads prop­erly into Google Chrome.

All ex­ten­sion de­vel­op­ers should con­sult the mi­gra­tion guide to make sure their ex­ten­sions will work prop­erly with the up­com­ing man­i­fest v3 changes.

While they are now in pre­view, Google ex­pects the man­i­fest v3 to go live in 2020 with the v2 end of life to be de­ter­mined in the fu­ture.

August 2007

(This is a talk I gave at the last

Y Combinator din­ner of the sum­mer.

Usually we don’t have a speaker at the last din­ner; it’s more of

a party. But it seemed worth spoil­ing the at­mos­phere if I could

save some of the star­tups from

pre­ventable deaths. So at the last minute I cooked up this rather

grim talk. I did­n’t mean this as an es­say; I wrote it down

be­cause I only had two hours be­fore din­ner and think fastest while

writ­ing.)

A cou­ple days ago I told a re­porter that we ex­pected about a third of the com­pa­nies we funded to suc­ceed. Actually I was be­ing con­ser­v­a­tive. I’m hop­ing it might be as much as a half. Wouldn’t it be amaz­ing if we could achieve a 50% suc­cess rate?

Another way of say­ing that is that half of you are go­ing to die. Phrased that way, it does­n’t sound good at all. In fact, it’s kind of weird when you think about it, be­cause our de­f­i­n­i­tion of suc­cess is that the founders get rich. If half the star­tups we fund suc­ceed, then half of you are go­ing to get rich and the other half are go­ing to get noth­ing.

If you can just avoid dy­ing, you get rich. That sounds like a joke, but it’s ac­tu­ally a pretty good de­scrip­tion of what hap­pens in a typ­i­cal startup. It cer­tainly de­scribes what hap­pened in Viaweb. We avoided dy­ing till we got rich.

It was re­ally close, too. When we were vis­it­ing Yahoo to talk about be­ing ac­quired, we had to in­ter­rupt every­thing and bor­row one of their con­fer­ence rooms to talk down an in­vestor who was about to back out of a new fund­ing round we needed to stay alive. So even in the mid­dle of get­ting rich we were fight­ing off the grim reaper.

You may have heard that quote about luck con­sist­ing of op­por­tu­nity meet­ing prepa­ra­tion. You’ve now done the prepa­ra­tion. The work you’ve done so far has, in ef­fect, put you in a po­si­tion to get lucky: you can now get rich by not let­ting your com­pany die. That’s more than most peo­ple have. So let’s talk about how not to die.

We’ve done this five times now, and we’ve seen a bunch of star­tups die. About 10 of them so far. We don’t know ex­actly what hap­pens when they die, be­cause they gen­er­ally don’t die loudly and hero­ically. Mostly they crawl off some­where and die.

For us the main in­di­ca­tion of im­pend­ing doom is when we don’t hear from you. When we haven’t heard from, or about, a startup for a cou­ple months, that’s a bad sign. If we send them an email ask­ing what’s up, and they don’t re­ply, that’s a re­ally bad sign. So far that is a 100% ac­cu­rate pre­dic­tor of death.

Whereas if a startup reg­u­larly does new deals and re­leases and ei­ther sends us mail or shows up at YC events, they’re prob­a­bly go­ing to live.

I re­al­ize this will sound naive, but maybe the link­age works in both di­rec­tions. Maybe if you can arrange that we keep hear­ing from you, you won’t die.

That may not be so naive as it sounds. You’ve prob­a­bly no­ticed that hav­ing din­ners every Tuesday with us and the other founders causes you to get more done than you would oth­er­wise, be­cause every din­ner is a mini Demo Day. Every din­ner is a kind of a dead­line. So the mere con­straint of stay­ing in reg­u­lar con­tact with us will push you to make things hap­pen, be­cause oth­er­wise you’ll be em­bar­rassed to tell us that you haven’t done any­thing new since the last time we talked.

If this works, it would be an amaz­ing hack. It would be pretty cool if merely by stay­ing in reg­u­lar con­tact with us you could get rich. It sounds crazy, but there’s a good chance that would work.

A vari­ant is to stay in touch with other YC-funded star­tups. There is now a whole neigh­bor­hood of them in San Francisco. If you move there, the peer pres­sure that made you work harder all sum­mer will con­tinue to op­er­ate.

When star­tups die, the of­fi­cial cause of death is al­ways ei­ther run­ning out of money or a crit­i­cal founder bail­ing. Often the two oc­cur si­mul­ta­ne­ously. But I think the un­der­ly­ing cause is usu­ally that they’ve be­come de­mor­al­ized. You rarely hear of a startup that’s work­ing around the clock do­ing deals and pump­ing out new fea­tures, and dies be­cause they can’t pay their bills and their ISP un­plugs their server.

Startups rarely die in mid key­stroke. So keep typ­ing!

If so many star­tups get de­mor­al­ized and fail when merely by hang­ing on they could get rich, you have to as­sume that run­ning a startup can be de­mor­al­iz­ing. That is cer­tainly true. I’ve been there, and that’s why I’ve never done an­other startup. The low points in a startup are just un­be­liev­ably low. I bet even Google had mo­ments where things seemed hope­less.

Knowing that should help. If you know it’s go­ing to feel ter­ri­ble some­times, then when it feels ter­ri­ble you won’t think “ouch, this feels ter­ri­ble, I give up.” It feels that way for every­one. And if you just hang on, things will prob­a­bly get bet­ter. The metaphor peo­ple use to de­scribe the way a startup feels is at least a roller coaster and not drown­ing. You don’t just sink and sink; there are ups af­ter the downs.

Another feel­ing that seems alarm­ing but is in fact nor­mal in a startup is the feel­ing that what you’re do­ing is­n’t work­ing. The rea­son you can ex­pect to feel this is that what you do prob­a­bly won’t work. Startups al­most never get it right the first time. Much more com­monly you launch some­thing, and no one cares. Don’t as­sume when this hap­pens that you’ve failed. That’s nor­mal for star­tups. But don’t sit around do­ing noth­ing. Iterate.

I like Paul Buchheit’s sug­ges­tion of try­ing to make some­thing that at least some­one re­ally loves. As long as you’ve made some­thing that a few users are ec­sta­tic about, you’re on the right track. It will be good for your morale to have even a hand­ful of users who re­ally love you, and star­tups run on morale. But also it will tell you what to fo­cus on. What is it about you that they love? Can you do more of that? Where can you find more peo­ple who love that sort of thing? As long as you have some core of users who love you, all you have to do is ex­pand it. It may take a while, but as long as you keep plug­ging away, you’ll win in the end. Both Blogger and Delicious did that. Both took years to suc­ceed. But both be­gan with a core of fa­nat­i­cally de­voted users, and all Evan and Joshua had to do was grow that core in­cre­men­tally.

Wufoo is on the same tra­jec­tory now.

So when you re­lease some­thing and it seems like no one cares, look more closely. Are there zero users who re­ally love you, or is there at least some lit­tle group that does? It’s quite pos­si­ble there will be zero. In that case, tweak your prod­uct and try again. Every one of you is work­ing on a space that con­tains at least one win­ning per­mu­ta­tion some­where in it. If you just keep try­ing, you’ll find it.

Let me men­tion some things not to do. The num­ber one thing not to do is other things. If you find your­self say­ing a sen­tence that ends with “but we’re go­ing to keep work­ing on the startup,” you are in big trou­ble. Bob’s go­ing to grad school, but we’re go­ing to keep work­ing on the startup. We’re mov­ing back to Minnesota, but we’re go­ing to keep work­ing on the startup. We’re tak­ing on some con­sult­ing pro­jects, but we’re go­ing to keep work­ing on the startup. You may as well just trans­late these to “we’re giv­ing up on the startup, but we’re not will­ing to ad­mit that to our­selves,” be­cause that’s what it means most of the time. A startup is so hard that work­ing on it can’t be pre­ceded by “but.”

In par­tic­u­lar, don’t go to grad­u­ate school, and don’t start other pro­jects. Distraction is fa­tal to star­tups. Going to (or back to) school is a huge pre­dic­tor of death be­cause in ad­di­tion to the dis­trac­tion it gives you some­thing to say you’re do­ing. If you’re only do­ing a startup, then if the startup fails, you fail. If you’re in grad school and your startup fails, you can say later “Oh yeah, we had this startup on the side when I was in grad school, but it did­n’t go any­where.”

You can’t use eu­phemisms like “didn’t go any­where” for some­thing that’s your only oc­cu­pa­tion. People won’t let you.

One of the most in­ter­est­ing things we’ve dis­cov­ered from work­ing on Y Combinator is that founders are more mo­ti­vated by the fear of look­ing bad than by the hope of get­ting mil­lions of dol­lars. So if you want to get mil­lions of dol­lars, put your­self in a po­si­tion where fail­ure will be pub­lic and hu­mil­i­at­ing.

When we first met the founders of

Octopart, they seemed very smart, but not a great bet to suc­ceed, be­cause they did­n’t seem es­pe­cially com­mit­ted. One of the two founders was still in grad school. It was the usual story: he’d drop out if it looked like the startup was tak­ing off. Since then he has not only dropped out of grad school, but ap­peared full length in

Newsweek with the word “Billionaire” printed across his chest. He just can­not fail now. Everyone he knows has seen that pic­ture. Girls who dissed him in high school have seen it. His mom prob­a­bly has it on the fridge. It would be un­think­ably hu­mil­i­at­ing to fail now. At this point he is com­mit­ted to fight to the death.

I wish every startup we funded could ap­pear in a Newsweek ar­ti­cle de­scrib­ing them as the next gen­er­a­tion of bil­lion­aires, be­cause then none of them would be able to give up. The suc­cess rate would be 90%. I’m not kid­ding.

When we first knew the Octoparts they were light­hearted, cheery guys. Now when we talk to them they seem grimly de­ter­mined. The elec­tronic parts dis­trib­u­tors are try­ing to squash them to keep their mo­nop­oly pric­ing. (If it strikes you as odd that peo­ple still or­der elec­tronic parts out of thick pa­per cat­a­logs in 2007, there’s a rea­son for that. The dis­trib­u­tors want to pre­vent the trans­parency that comes from hav­ing prices on­line.) I feel kind of bad that we’ve trans­formed these guys from light­hearted to grimly de­ter­mined. But that comes with the ter­ri­tory. If a startup suc­ceeds, you get mil­lions of dol­lars, and you don’t get that kind of money just by ask­ing for it. You have to as­sume it takes some amount of pain.

And how­ever tough things get for the Octoparts, I pre­dict they’ll suc­ceed. They may have to morph them­selves into some­thing to­tally dif­fer­ent, but they won’t just crawl off and die. They’re smart; they’re work­ing in a promis­ing field; and they just can­not give up.

All of you guys al­ready have the first two. You’re all smart and work­ing on promis­ing ideas. Whether you end up among the liv­ing or the dead comes down to the third in­gre­di­ent, not giv­ing up.

So I’ll tell you now: bad shit is com­ing. It al­ways is in a startup. The odds of get­ting from launch to liq­uid­ity with­out some kind of dis­as­ter hap­pen­ing are one in a thou­sand. So don’t get de­mor­al­ized. When the dis­as­ter strikes, just say to your­self, ok, this was what Paul was talk­ing about. What did he say to do? Oh, yeah. Don’t give up.

You’re writ­ing soft­ware that processes data, and it works fine when you test it on a small sam­ple file. But when you load the real data, your pro­gram crashes.

The prob­lem is that you don’t have enough mem­ory—if you have 16GB of RAM, you can’t load a 100GB file. At some point the op­er­at­ing sys­tem will run out of mem­ory, fail to al­lo­cate, and there goes your pro­gram.

So what can you do? You could spin up a Big Data clus­ter—all you’ll need to do is:

* In many cases, learn a com­pletely new API and rewrite all your code.

This can be ex­pen­sive and frus­trat­ing; luck­ily, in many cases it’s also un­nec­es­sary.

You need a so­lu­tion that’s sim­ple and easy: pro­cess­ing your data on a sin­gle com­puter, with min­i­mal setup, and as much as pos­si­ble us­ing the same li­braries you’re al­ready us­ing.

And much of the time you can ac­tu­ally do that, us­ing a set of tech­niques that are some­times called “out-of-core com­pu­ta­tion”.

* Why you need RAM at all.

* The eas­i­est way to process data that does­n’t fit in mem­ory: spend­ing some money.

* The three ba­sic soft­ware tech­niques for han­dling too much data: com­pres­sion, chunk­ing, and in­dex­ing.

Followup ar­ti­cles will then show you how to ap­ply these tech­niques to par­tic­u­lar li­braries like NumPy and Pandas.

Before we move on to talk­ing about so­lu­tions, let’s clar­ify why the prob­lem ex­ists at all. Your com­put­er’s mem­ory (RAM) lets you read and write data, but so does your hard drive—so why does your com­puter need RAM at all? Disk is cheaper than RAM, so it can usu­ally fit all your data, so why can’t your code just limit it­self to read­ing and writ­ing from disk?

In the­ory, that can work. However, even the more mod­ern and fast solid-state hard dri­ves (SSDs) are much, much slower than RAM:

If you want fast com­pu­ta­tion, data has to fit in RAM, oth­er­wise your code may run as much as 150× times more slowly.

The eas­i­est so­lu­tion to not hav­ing enough RAM is to throw money at the prob­lem. You can ei­ther buy a com­puter or rent a vir­tual ma­chine (VM) in the cloud with lots more mem­ory than most lap­tops. In November 2019, with min­i­mal search­ing and very lit­tle price com­par­i­son, I found that you can:

* Buy a Thinkpad M720 Tower, with 6 cores and 64GB RAM, for $1074.

* Rent a VM in the cloud, with 64 cores and 432GB RAM, for $3.62/hour.

These are just num­bers I found with min­i­mal work, and with a lit­tle more re­search you can prob­a­bly do even bet­ter.

If spend­ing some money on hard­ware will make your data fit into RAM, that is of­ten the cheap­est so­lu­tion: your time is pretty ex­pen­sive, af­ter all.

Sometimes, how­ever, it’s in­suf­fi­cient.

For ex­am­ple, if you’re run­ning many data pro­cess­ing jobs, over a pe­riod of time, cloud com­put­ing may be the nat­ural so­lu­tion, but also an ex­pen­sive one. At one job the com­pute cost for the soft­ware I was work­ing on would have used up all our pro­jected rev­enue for the prod­uct, in­clud­ing the all-im­por­tant rev­enue needed to pay my salary.

If buy­ing/​rent­ing more RAM is­n’t suf­fi­cient or pos­si­ble, the next step is to fig­ure out how to re­duce mem­ory us­age by chang­ing your soft­ware.

Compression means us­ing a dif­fer­ent rep­re­sen­ta­tion for your data, in a way that uses less mem­ory. There are two forms of com­pres­sion:

* Lossless: The data you’re stor­ing has the ex­act same in­for­ma­tion as the orig­i­nal data.

* Lossy: The data you’re stor­ing loses some of the de­tails in the orig­i­nal data, but in a way that ide­ally does­n’t im­pact the re­sults of your cal­cu­la­tion very much.

Just to be clear, I’m not talk­ing about a ZIP or gzip file, since those typ­i­cally in­volve com­pres­sion on disk. To process the data from a ZIP file you will typ­i­cally un­com­press it as part of load­ing the files into mem­ory. So that’s not go­ing to help.

What you need is com­pres­sion of rep­re­sen­ta­tion in mem­ory.

For ex­am­ple, let’s say your data has two val­ues, and will only ever have those two val­ues: “AVAILABLE” and “UNAVAILABLE”. Instead of stor­ing them as a string with ~10 bytes or more per en­try, you could store them as a boolean, True or False, which you could store in 1 byte. You might even get the rep­re­sen­ta­tion down to the sin­gle bit nec­es­sary to rep­re­sent a boolean, re­duc­ing mem­ory us­age by an­other fac­tor of 8.

Chunking is use­ful when you need to process all the data, but don’t need to load all the data into mem­ory at once. Instead you can load it into mem­ory in chunks, pro­cess­ing the data one chunk at time (or as we’ll dis­cuss in a fu­ture ar­ti­cle, mul­ti­ple chunks in par­al­lel).

Let’s say, for ex­am­ple, that you want to find the largest word in a book. You could load all the data into mem­ory at once:

But since in our case the book does­n’t fit into mem­ory, you could in­stead load the book page by page:

You are us­ing much less mem­ory, since you only have one page of the book in mem­ory at any given time. And you still get the same an­swer in the end.

Indexing is use­ful when you only need to use a sub­set of the data, and you ex­pect to be load­ing dif­fer­ent sub­sets of the data at dif­fer­ent times.

You could solve this use case with chunk­ing: load all the data every time, and just fil­ter out the data you don’t care about. But that’s slow, since you need to load lots of ir­rel­e­vant data.

If you only need part of the data, in­stead of chunk­ing you are bet­ter off us­ing an in­dex, a sum­mary of the data that tells you where to find the data you care about.

Imagine you want to only read the parts of the book that talk about aard­varks. If you used chunk­ing, you would read the whole book, page by page, look­ing for aard­varks—but that would take quite a while.

Or, you can go to the end of the book, where the book’s in­dex is, and find the en­try for “Aardvarks”. It might tell you to read pages 7, 19, and 120-123. So now you can read those pages, and those pages only, which is much faster.

This works be­cause the in­dex is much smaller than the full book, so load­ing the in­dex into mem­ory to lookup the rel­e­vant data is much eas­ier.

The sim­plest and most com­mon way to im­ple­ment in­dex­ing is by nam­ing files in a di­rec­tory:

If you want the data for March 2019, you just load 2019-Mar.csv—no need to load data for February, July, or any other month.

The eas­i­est so­lu­tion to lack of RAM is spend­ing money to get more RAM. But if that is­n’t pos­si­ble or suf­fi­cient in your case, you will one way or an­other find­ing your­self us­ing com­pres­sion, chunk­ing, or in­dex­ing.

These same tech­niques ap­pear in many dif­fer­ent soft­ware pack­ages and tools.

Even Big Data sys­tems are built on these tech­niques: us­ing mul­ti­ple com­put­ers to process chunks of the data, for ex­am­ple.

In fol­low-up ar­ti­cles I will show you to how to ap­ply these tech­niques with spe­cific li­braries and tools: NumPy, Pandas, and even ZIP files. If you want to read these ar­ti­cles as they come out, sign up for my newslet­ter in the form be­low.

Google is en­gaged with one of the U. S.’s largest health-care sys­tems on a pro­ject to col­lect and crunch the de­tailed per­sonal-health in­for­ma­tion of mil­lions of peo­ple across 21 states.

The ini­tia­tive, code-named “Project Nightingale,” ap­pears to be the biggest ef­fort yet by a Silicon Valley gi­ant to gain a toe­hold in the health-care in­dus­try through the han­dling of pa­tients’ med­ical data. Amazon.com Inc., Apple Inc. and Microsoft Corp. are also ag­gres­sively push­ing into health care, though they haven’t yet struck deals of…

This web­site re­quires JavaScript to func­tion prop­erly. If you dis­abled JavaScript, please en­able it and re­fresh this page.

Another year, an­other pre­dicted new-car re­li­a­bil­ity rank­ing from non-profit con­sumer prod­ucts re­searchers at Consumer Reports. And this year, the do­mes­tic au­tomak­ers aren’t look­ing so good, tak­ing up 11 of the bot­tom 12 spots. Here’s a look at re­li­a­bil­ity rank­ings for all the brands.

Each year, Consumer Reports sends a ques­tion­naire to its mem­bers to learn about is­sues that they’ve had with their cars over the past year, and the sever­ity of those is­sues. In their lat­est sur­vey, CR got info on over 500,000 ve­hi­cles from model years 2000 to 2018, gath­er­ing in­for­ma­tion about prob­lems that own­ers have had in twelve key ar­eas, in­clud­ing en­gine in­ter­nals, ac­ces­sory drive, cool­ing sys­tem, trans­mis­sion in­ter­nals, dri­ve­train, fuel sys­tem, elec­tri­cal sys­tem, brakes, cli­mate con­trol, ex­haust, in-car elec­tron­ics, and oth­ers.

With typ­i­cally 200 to 400 sam­ples from each model, Consumer Reports’ lat­est data shows every American au­tomaker in the bot­tom half of the pre­dicted new-car re­li­a­bil­ity rank­ings.

At the top are Lexus and Toyota once again, with the GX as the most re­li­able Lexus and the Prius C as the most re­li­able Toyota. Subaru, Kia, and Infiniti have also stayed in the top six spots, though Mazda made a huge leap from 12 to num­ber three. That leap, CR says, comes cour­tesy of Mazda “[working] out prob­lems that plagued the CX-9 SUV and MX-5 Miata road­ster.”

German brands are in the mid­dle, with Audi, BMW, and Mini com­ing it at num­bers seven through nine, Porsche at 11, VW at 16, and Mercedes at 17. Korean brands Hyundai and Genesis—as well as Japanese brands Acura, Nissan, and Honda—are also in the mid­dle there, ranked at 10, 12, 13, 14, and 15, re­spec­tively.

Honda’s re­li­a­bil­ity rank­ing dropped the most of these brands—down by six thanks to an Odyssey that al­legedly has in­fo­tain­ment and door lock is­sues, yield­ing “much-worse-than-average re­li­a­bil­ity.” In ad­di­tion, the CR-V and Accord’s rat­ings are down to just “average” thanks in part to “infotainment sys­tem and in­te­rior rat­tles,” and the Clarity line of cars ap­par­ently has “electronic glitches” that bring it to “much-worse-than-average.”

But on the plus side for Honda, its lux­ury sis­ter brand, Acura, is up six spots thanks to re­cently worked-out trans­mis­sion and in­fo­tain­ment prob­lem, CR says.

Left at the bot­tom are the Americans, with the ex­cep­tion of Swedish brand Volvo, which is lit­er­ally at the very bot­tom, ranked 29, in part be­cause of al­leged in­fo­tain­ment is­sues with the XC60 and XC90, as well as “complaints about en­gine knock­ing or ping­ing” on the S90

Ford sits at num­ber 18 and the once-mighty Buick brand dropped a whop­ping 11 spots to 19. Consumer Reports at­trib­utes the gi­ant drop of GM’s mid-level lux­ury brand to the Enclave’s nine-speed trans­mis­sion woes which con­tributed to its “much-worse-than-average rat­ing.” Other Buicks like the LaCrosse, Encore, and Envision were rated av­er­age, CR says.

Lincoln, Dodge, Jeep, Chevy, and Chrysler are ranked be­tween 20 and 24, while GMC, Ram, Tesla, and Cadillac are in po­si­tions 25 to 28. The American brands that dropped the most were Chrysler (down seven), Tesla (down six) and Chevrolet (down five).

CR says Chrysler’s Pacifica was one that con­tributed to the drop in rank­ings thanks to the mini­van’s in­fo­tain­ment and trans­mis­sion is­sues.

Update Oct. 24, 2018 8:25 P. M: A Fiat Chrysler spokesper­son pro­vided the fol­low­ing com­ment on the new rank­ings:

“The qual­ity and re­li­a­bil­ity of our ve­hi­cles is of the great­est im­por­tance to all of us here at FCA US. We are in con­stant com­mu­ni­ca­tion with our cus­tomers, ad­dress­ing their feed­back in an ef­fort to con­tin­u­ously im­prove the qual­ity of our ve­hi­cles. In ad­di­tion, our teams are ag­gres­sively work­ing to­ward so­lu­tions to any con­cerns to en­sure com­plete cus­tomer sat­is­fac­tion. We en­cour­age peo­ple to ex­pe­ri­ence our lineup for them­selves, and we thank our loyal cus­tomers who con­tinue to love their ve­hi­cles as they rec­om­mend them to their fam­ily and friends.”

As for why Tesla dropped six spots, CR notes the Model S’s re­ported sus­pen­sion and door han­dle is­sues, as well as the Model X re­main­ing at “much worse than av­er­age” thanks in part to the in­fo­tain­ment screen and the Falcon Doors.

We reached out to Tesla, and their spokesper­son pro­vided the fol­low­ing com­ment on about the Model S sus­pen­sion com­plaints:

The sus­pen­sion is­sues that some Model S cus­tomers ex­pe­ri­enced pri­mar­ily in 2017 were due to a sup­plier-re­lated is­sue that did not pose any threat to ve­hi­cle safety or dri­vabil­ity, and pre­sented it­self only when the car was parked. The is­sue has al­ready been ad­dressed for cus­tomer ve­hi­cles in the field and re­solved at the source with fun­da­men­tal de­sign im­prove­ments. In ad­di­tion, there was an un­re­lated false ser­vice alert that some cus­tomers re­ceived re­gard­ing their sus­pen­sion in 2018, and it was fixed for all cus­tomers via an over-the-air soft­ware up­date within two weeks of be­ing re­ported. Suspension is­sues for Model S have im­proved 65% since last year, and we con­tinue to make fur­ther im­prove­ments.

As for the re­ported is­sues with the Model X, Tesla says it’s made changes to fix them:

While the ear­li­est pro­duc­tion Model X cars en­coun­tered some qual­ity in­con­sis­ten­cies, this is sim­ply not a con­cern for Model X cars be­ing built to­day, and it has­n’t been one for quite a while. In fact, the qual­ity of brand-new Model X ve­hi­cles to­day is 3.5 times bet­ter than the qual­ity of brand new Model X cars from 2015. And, we con­tinue to im­prove the re­li­a­bil­ity of cars al­ready on the road via over-the-air soft­ware up­dates and proac­tive ser­vice bul­letins. This proac­tive ap­proach to im­proved re­li­a­bil­ity is one of the rea­sons why Tesla is the high­est rated car brand among con­sumers, ac­cord­ing to Consumer Reports.”

The rest of the com­pa­ny’s state­ment reads:

“Not only are our cars the safest and best per­form­ing ve­hi­cles avail­able to­day, but we take feed­back from our cus­tomers very se­ri­ously and quickly im­ple­ment im­prove­ments any time we hear about is­sues. That’s just one of the rea­sons why Model S has been ranked num­ber one on Consumer Reports’ owner sat­is­fac­tion sur­vey every year since 2013, which was the first year Tesla was in­cluded in their re­port.

As for Chevy, it looks like the new Traverse—with the same trans is­sues as the Enclave—didn’t help any with its “much-worse-than-average re­li­a­bil­ity.” A General Motors rep­re­sen­ta­tive pro­vided the fol­low­ing state­ment:

We are com­mit­ted to pro­vid­ing our cus­tomers high-qual­ity prod­ucts and re­main fo­cused on launch­ing with ex­cel­lence. Most of our brands con­tinue to main­tain or im­prove rel­a­tive to in­dus­try av­er­age. As al­ways, we are in­ter­ested in ob­tain­ing the sur­vey data to bet­ter un­der­stand our per­for­mance and where we can im­prove.

You can learn more about how Consumer Reports com­piles this list here, and you can watch the video above. You can also check out CR’s press re­lease for analy­sis on the new rank­ings. Whether you agree with the non-prof­it’s method­ol­ogy, the re­al­ity is that Consumer Reports’ find­ings play a ma­jor role in shap­ing car buy­ers’ opin­ions, and also in how au­tomak­ers ac­tu­ally go about de­sign­ing their ve­hi­cles.

This post has been up­dated with com­ment from a Fiat Chrysler spokesper­son.

Renaissance Technologies LLC is an American hedge fund firm based in East Setauket, New York,[5] on Long Island, which spe­cial­izes in sys­tem­atic trad­ing us­ing quan­ti­ta­tive mod­els de­rived from math­e­mat­i­cal and sta­tis­ti­cal analy­ses. The com­pany was founded in 1982 by James Simons, an award-win­ning math­e­mati­cian and for­mer Cold War code breaker.

In 1988, the firm es­tab­lished its most prof­itable port­fo­lio, the Medallion Fund, which used an im­proved and ex­panded form of Leonard Baum’s math­e­mat­i­cal mod­els, im­proved by al­ge­braist James Ax, to ex­plore cor­re­la­tions from which they could profit. Simons and Ax started a hedge fund and named it Medallion in honor of the math awards that they had won.[6][7]

Renaissance’s flag­ship Medallion fund, which is run mostly for fund em­ploy­ees,[8] is famed for the best record in in­vest­ing his­tory, re­turn­ing more than 66 per­cent an­nu­alised be­fore fees and 39 per­cent af­ter fees over a 30-year span from 1988 to 2018.[5][9] Renaissance of­fers two port­fo­lios to out­side in­vestors—Re­nais­sance Institutional Equities Fund (RIEF) and Renaissance Institutional Diversified Alpha (RIDA).[10]

Simons ran Renaissance un­til his re­tire­ment in late 2009.[11] The com­pany is now run by Peter Brown (after Robert Mercer re­signed), both of them were com­puter sci­en­tists spe­cial­iz­ing in com­pu­ta­tional lin­guis­tics who joined Renaissance in 1993 from IBM Research.[1][10][12] Simons con­tin­ues to play a role at the firm as non-ex­ec­u­tive chair­man and re­mains in­vested in its funds, par­tic­u­larly the se­cre­tive and con­sis­tently prof­itable black-box strat­egy known as Medallion.[13] Because of the suc­cess of Renaissance in gen­eral and Medallion in par­tic­u­lar, Simons has been de­scribed as the best money man­ager on earth.[14][15][16] By October 2015, Renaissance had roughly $65 billion worth of as­sets un­der man­age­ment, most of which be­long to em­ploy­ees of the firm.[17]

James Simons founded Renaissance Technologies fol­low­ing a decade as the Chair of the Department of Mathematics at Stony Brook University. Simons is a 1976 re­cip­i­ent of the Oswald Veblen Prize of the American Mathematical Society, which is geom­e­try’s high­est honor.[18] He is known in the sci­en­tific com­mu­nity for his work, Chern–Simons the­ory, which is fun­da­men­tal in mod­ern the­o­ret­i­cal physics, in­clud­ing ad­vanced the­o­ries of how in­vis­i­ble fields like those of grav­ity in­ter­act with mat­ter to pro­duce every­thing from su­per­strings to black holes.[19]

The firm uses quan­ti­ta­tive trad­ing, where staff tap data in its petabyte-scale data ware­house to as­sess sta­tis­ti­cal prob­a­bil­i­ties for the di­rec­tion of se­cu­ri­ties prices in any given mar­ket. Staff at­tribute the breadth of data on events pe­riph­eral to fi­nan­cial and eco­nomic phe­nom­ena that Renaissance takes into ac­count, and the fir­m’s abil­ity to ma­nip­u­late amounts of data by de­ploy­ing scal­able tech­no­log­i­cal ar­chi­tec­tures for com­pu­ta­tion and ex­e­cu­tion.[20] In many ways, Renaissance Technologies, along with a few other firms, has been syn­the­siz­ing ter­abytes of data daily and ex­tract­ing in­for­ma­tion sig­nals from petabytes of data for al­most two decades now, well be­fore big data and data an­a­lyt­ics caught the imag­i­na­tion of main­stream tech­nol­ogy.[21]

For more than twenty years, the fir­m’s Renaissance Technologies hedge fund, which trades in mar­kets around the world, has em­ployed com­plex math­e­mat­i­cal mod­els to an­a­lyze and ex­e­cute trades, many of them au­to­mated. The firm uses com­puter-based mod­els to pre­dict price changes in eas­ily traded fi­nan­cial in­stru­ments. These mod­els are based on an­a­lyz­ing as much data as can be gath­ered, then look­ing for non-ran­dom move­ments to make pre­dic­tions. Some also at­tribute the fir­m’s per­for­mance to em­ploy­ing fi­nan­cial sig­nal pro­cess­ing tech­niques such as pat­tern recog­ni­tion. The book The Quants de­scribes the hir­ing of speech recog­ni­tion ex­perts, many from IBM, in­clud­ing the cur­rent lead­ers of the firm.

Renaissance em­ploys spe­cial­ists with non-fi­nan­cial back­grounds, in­clud­ing math­e­mati­cians, physi­cists, sig­nal pro­cess­ing ex­perts and sta­tis­ti­cians. The fir­m’s lat­est fund is the Renaissance Institutional Equities Fund (RIEF).[22] RIEF has his­tor­i­cally trailed the fir­m’s bet­ter-known Medallion fund, a sep­a­rate fund that con­tains only the per­sonal money of the fir­m’s ex­ec­u­tives.[23]

In a 2013 ar­ti­cle in The Daily Telegraph, jour­nal­ist Sarfraz Manzoor de­scribed Renaissance staff as math ge­niuses run­ning Wall Street.[16]

“Of his 200 em­ploy­ees, en­sconced in a fortress-like build­ing in un­fash­ion­able Long Island, New York, a third have PhDs, not in fi­nance, but in fields like physics, math­e­mat­ics and sta­tis­tics. Renaissance has been called “the best physics and math­e­mat­ics de­part­ment in the world” and, ac­cord­ing to Weatherall, “avoids hir­ing any­one with even the slight­est whiff of Wall Street bona fides.

Renaissance is a firm run by and for sci­en­tists, em­ploy­ing prefer­ably those with non-fi­nan­cial back­grounds for quan­ti­ta­tive fi­nance re­search like math­e­mati­cians, sta­tis­ti­cians, pure and ex­per­i­men­tal physi­cists, as­tronomers, and com­puter sci­en­tists. Wall Street ex­pe­ri­ence is frowned on and a flair for sci­ence is prized.[19] It is a widely held be­lief within Renaissance that the herd­like men­tal­ity among busi­ness school grad­u­ates is to blame for poor in­vestor re­turns.[6] Renaissance en­gages roughly 150 re­searchers and com­puter pro­gram­mers, half of whom have PhDs in sci­en­tific dis­ci­plines, at its 50-acre East Setauket cam­pus in Long Island, New York, which is near the State University of New York at Stony Brook.[24] Mathematician Isadore Singer re­ferred to Renaissance’s East Setauket of­fice as the best physics and math­e­mat­ics de­part­ment in the world.[25]

The fir­m’s ad­min­is­tra­tive and back-of­fice func­tions are han­dled from its Manhattan of­fice in New York City. The firm is se­cre­tive about the work­ings of its busi­ness and very lit­tle is known about them.[26] The firm is known for its abil­ity to re­cruit and re­tain sci­en­tific types, for hav­ing a per­son­nel turnover that is nearly non-ex­is­tent,[27] and for re­quir­ing its re­searchers to agree to in­tel­lec­tual prop­erty oblig­a­tions by sign­ing non-com­pete and non-dis­clo­sure agree­ments.[28]

In 1978 Simons left acad­e­mia and started a hedge fund man­age­ment firm called Monemetrics in a Long Island strip mall. The firm pri­mar­ily traded cur­ren­cies at the start. It did not oc­cur to Simons at first to ap­ply math­e­mat­ics to his busi­ness, but he grad­u­ally re­al­ized that it should be pos­si­ble to make math­e­mat­i­cal mod­els of the data he was col­lect­ing.

Monemetrics’ name was changed to Renaissance Technologies in 1982. Simons started re­cruit­ing some of the math­e­mati­cians and data-mod­el­ing types from his days at the Institute for Defense Analyses (IDA) and Stony Brook University. His first re­cruit was Leonard Baum, a crypt­an­a­lyst from IDA who was also the co-au­thor of the Baum–Welch al­go­rithm. When Baum aban­doned the idea of trad­ing with math­e­mat­i­cal mod­els and took to fun­da­men­tal trad­ing, Simons brought in al­ge­braist James Ax from Cornell University. Ax ex­panded Baum’s mod­els for trad­ing cur­ren­cies to cover any com­mod­ity fu­ture and sub­se­quently Simons set up Ax with his own trad­ing ac­count, Axcom Ltd., which even­tu­ally gave birth to the prof­itable fund — Medallion. During the 1980s, Ax and his re­searchers im­proved on Baum’s mod­els and used them to ex­plore cor­re­la­tions from which they could profit.

From 2001 through 2013, the fund’s worst year was a 21 per­cent gain, af­ter sub­tract­ing fees. Medallion reaped a 98.2 per­cent gain in 2008, the year the Standard & Poor’s 500 Index lost 38.5 per­cent.

In 1988 Renaissance es­tab­lished its most fa­mous and prof­itable port­fo­lio, the Medallion fund, which used an im­proved and ex­panded form of Leonard Baum’s math­e­mat­i­cal mod­els im­proved by pi­o­neer­ing al­ge­braist James Ax to ex­plore cor­re­la­tions from which they could profit. Simons and Ax started a hedge fund and named it Medallion in honor of the math awards that they had won.[6][7] The math­e­mat­i­cal mod­els the com­pany de­vel­oped worked bet­ter and bet­ter each year, and by 1988, Simons had de­cided to base the com­pa­ny’s trades en­tirely on the mod­els.[6][7]

By April 1989, peak-to-trough losses had mounted to about 30%. Ax had ac­counted for such a draw­down in his mod­els and pushed to keep trad­ing. Simons wanted to stop to re­search what was go­ing on. After a brief stand­off, Simons pulled rank and Ax left. Simons turned to Elwyn Berlekamp to run Medallion from Berkeley, California. A con­sul­tant for Axcom whom Simons had first met at the IDA, Berlekamp had bought out most of Ax’s stake in Axcom and be­came its CEO. He worked with Sandor Straus, Jim Simons and an­other con­sul­tant, Henry Laufer, to over­haul Medallion’s trad­ing sys­tem dur­ing a six-month stretch. In 1990, Berlekamp led Medallion to a 55.9% gain, net of fees — and then re­turned to teach­ing math at University of California, Berkeley af­ter sell­ing out to Jim Simons at six times the price for which he had bought his Axcom in­ter­ests 16 months ear­lier. Straus took the reins of Medallion’s re­vamped trad­ing sys­tem and Medallion re­turned 39.4% in 1991, 34% in 1992 and 39.1% in 1993, ac­cord­ing to Medallion an­nual re­ports.[6][29]

The Medallion fund is con­sid­ered to be one of the most suc­cess­ful hedge funds ever. It has av­er­aged a 71.8% an­nual re­turn, be­fore fees, from 1994 through mid-2014.[30] The fund has been closed to out­side in­vestors since 1993[31] and is avail­able only to cur­rent and past em­ploy­ees and their fam­i­lies. The firm bought out the last in­vestor in the Medallion fund in 2005 and the in­vestor com­mu­nity has not seen its re­turns since then.[8] About 100 of Renaissance’s 275 or so em­ploy­ees are what it calls “qualified pur­chasers”, mean­ing they gen­er­ally have at least $5 million in as­sets to in­vest. The re­main­ing are “accredited in­vestors”, gen­er­ally worth at least $1 million.[30]

Since 1988, his flag­ship Medallion fund has gen­er­ated av­er­age an­nual re­turns of 66% be­fore charg­ing hefty in­vestor fees—39% af­ter fees—rack­ing up trad­ing gains of more than $100 bil­lion. No one in the in­vest­ment world comes close. Warren Buffett, George Soros, Peter Lynch, Steve Cohen, and Ray Dalio all fall short.— ‘The Man Who Solved the Market: How Jim Simons Launched the Quant Revolution’ by Gregory Zuckerman 2019

By the year 2000, the com­puter-dri­ven Medallion fund had made an av­er­age of 34% a year af­ter fees since 1988.[14] Simons ran Renaissance un­til his re­tire­ment in late 2009.[11] Between January 1993 and April 2005, Medallion only had 17 monthly losses and out of 49 quar­ters in the same time pe­riod, Medallion only posted three quar­terly losses. Between 1989-2005 Medallion had only one year show­ing a loss: 1989.[32]

[Renaissance] won the [Labor Department]’s per­mis­sion to put pieces of Medallion in­side Roth IRAs. That means no taxes — ever — on the fu­ture earn­ings of a fund that av­er­aged a 71.8 per­cent an­nual re­turn, be­fore fees, from 1994 through mid-2014.

Renaissance Technologies ter­mi­nated its 401(k) re­tire­ment plan in 2010 and em­ploy­ees ac­count bal­ances were put into Individual Retirement Accounts.[30] Contributions could be made to a stan­dard Individual Retirement Accounts and then con­verted to a Roth IRA re­gard­less of in­come.[33] By 2012 Renaissance was granted a spe­cial ex­emp­tion by the United States Labor Department al­low­ing em­ploy­ees to in­vest their re­tire­ment money in Medallion ar­gu­ing that Medallion had con­sis­tently out­per­formed their old 401(k) plan. In 2013 Renaissance’s IRA plans had 259 par­tic­i­pants whose $86.6 million con­tri­bu­tion grew to $153 million that year with­out fees or an­nual taxes.[30] Renaissance set up a new 401(k) plan and in November 2014 the Labor Department al­lowed that plan to be in­vested in Medallion as well.[30]

In 2005 Renaissance Institutional Equities Fund (RIEF) was cre­ated.[22] RIEF has his­tor­i­cally trailed the fir­m’s bet­ter-known Medallion fund, a sep­a­rate fund that only con­tains the per­sonal money of the fir­m’s ex­ec­u­tives.[23] Renaissance also of­fers two Renaissance Institutional Diversified Alpha (RIDA) to out­siders.[10] Simons ran Renaissance un­til his re­tire­ment in late 2009.[11] Renaissance Institutional Equities Fund had dif­fi­culty with the higher volatil­ity en­vi­ron­ment that per­sisted through­out the end of the sum­mer of 2007. According to an ar­ti­cle in Bloomberg in August 2007,[34]

James Simons’s $29 billion Renaissance Institutional Equities Fund fell 8.7% in August 2007 when his com­puter mod­els used to buy and sell stocks were over­whelmed by se­cu­ri­ties’ price swings. The two-year-old quan­ti­ta­tive, or ‘quant’, hedge fund now has de­clined 7.4 per­cent for the year. Simons said other hedge funds have been forced to sell po­si­tions, short-cir­cuit­ing sta­tis­ti­cal mod­els based on the re­la­tion­ships among se­cu­ri­ties.

On 25 September 2008, Renaissance wrote a com­ment let­ter to the Securities and Exchange Commission, dis­cour­ag­ing them from im­ple­ment­ing a rule change that would have per­mit­ted the pub­lic to ac­cess in­for­ma­tion re­gard­ing in­sti­tu­tional in­vestors’ short po­si­tions, as they can cur­rently do with long po­si­tions. The com­pany cited a num­ber of rea­sons for this, in­clud­ing the fact that “institutional in­vestors may al­ter their trad­ing ac­tiv­ity to avoid pub­lic dis­clo­sure”.[35]

In July 2014 Renaissance Technologies was in­cluded in a larger in­ves­ti­ga­tion un­der­taken by Carl Levin and the Permanent Subcommittee on Investigations on tax eva­sion by wealthy in­di­vid­u­als.[5] The fo­cus of the tax avoid­ance in­ves­ti­ga­tion was Renaissance’s trad­ing strat­egy — which in­volved trans­ac­tions with banks such as Barclays Plc and Deutsche Bank AG — through which prof­its con­verted from rapid trad­ing were con­verted into lower-taxed, long-term cap­i­tal gains.[5] The strat­egy was also ques­tioned by the Internal Revenue Service (IRS).[5] The higher rates for the five years un­der in­ves­ti­ga­tion would have been 44.4 per­cent, as com­pared to 35 per­cent, whereas the lower rate was 15 per­cent, as com­pared to 23.8 per­cent.[5]

The IRS con­tend[ed] that the arrange­ment Renaissance’s Medallion fund had with the banks, in which the fund owned op­tion con­tracts rather than the un­der­ly­ing fi­nan­cial in­stru­ments, is a ruse and that the fund in­vestors owe taxes at the higher rate. Because Medallion could claim that it owned just one as­set — the op­tion — and held it for more than a year, in­vestors could de­clare their gains to be long-term in­vest­ments.

According to the Center for Responsive Politics, Renaissance is the top fi­nan­cial firm con­tribut­ing to fed­eral cam­paigns in the 2016 elec­tion cy­cle, do­nat­ing $33,108,000 by July.[36] By com­par­i­son, over that same pe­riod sixth ranked Soros Fund Management has con­tributed $13,238,551.[36] Renaissance’s man­agers have also been ac­tive in the 2016 cy­cle, con­tribut­ing nearly $30 mil­lion by June, with Mercer rank­ing as the #1 in­di­vid­ual fed­eral donor, largely to Republicans, and Simons ranked #5, largely to Democrats.[37] They were top donors to the pres­i­den­tial cam­paigns of Hillary Clinton[38] and Donald Trump.[39]

During the 2016 cam­paign cy­cle Simons con­tributed $26,277,450, rank­ing as the 5th largest in­di­vid­ual con­trib­u­tor. Simons di­rected all but $25,000 of his funds to­wards lib­eral can­di­dates. Robert Mercer con­tributed $25,059,300, rank­ing as the 7th largest in­di­vid­ual con­trib­u­tor. Robert Mercer di­rected all funds con­tributed to­wards con­ser­v­a­tive can­di­dates.

Since 1990 Renaissance has con­tributed $59,081,152 to fed­eral cam­paigns and since 2001 has spent $3,730,000 on lob­by­ing.[40]

These de­vices al­low you to con­nect, con­trol and mon­i­tor Live with a range of in­no­v­a­tive tech­nolo­gies and com­mu­ni­ca­tion pro­to­cols. Use LEGO® MINDSTORMS® EV3, Arduino, or lit­tleBits™ to con­nect up sen­sors, lights or mo­tors, open your sound world up to the web through JSON-based APIs, or con­vert OSC data to MIDI data. The list of in­put and out­put pos­si­bil­i­ties for mu­sic & sound cre­ation with Live is al­most end­less.

These de­vices al­low you to con­nect, con­trol and mon­i­tor Live with a range of in­no­v­a­tive tech­nolo­gies and com­mu­ni­ca­tion pro­to­cols. Use LEGO® MINDSTORMS® EV3, Arduino, or lit­tleBits™ to con­nect up sen­sors, lights or mo­tors, open your sound world up to the web through JSON-based APIs, or con­vert OSC data to MIDI data. The list of in­put and out­put pos­si­bil­i­ties for mu­sic & sound cre­ation with Live is al­most end­less.

The Pack con­sists of 11 Max for Live de­vices: a toolkit for ex­plo­ration, or to open up in Max and adapt to your own needs (Max pro­gram­ming knowl­edge is re­quired for this!). Some de­vices demon­strate how you can use each pro­to­col to cap­ture dif­fer­ent types of data.

You can get the pack us­ing the down­load but­ton to the right or by fork­ing our repos­i­tory on GitHub.

Here’s more on how you can use each de­vice.