What ORMs have taught me: just learn SQL

I’ve come to the con­clu­sion that, for me, ORMs are more detri­ment than ben­e­fit. In short, they can be used to nicely aug­ment work­ing with SQL in a pro­gram, but they should not re­place it.

Some back­ground: For the past 30 months I’ve been work­ing with code that has to in­ter­face with Postgres and to some ex­tent, SQLite. Most of that has been with SQLAlchemy (which I quite like) and Hibernate

(which I don’t). I’ve worked with ex­ist­ing code and data mod­els, as well as de­sign­ing my own. Most of the data is event-based stor­age (“timelines”) with a heavy em­pha­sis on cre­at­ing re­ports.

Much has been writ­ten about the Object/Relational Impedance Mismatch. It’s hard to ap­pre­ci­ate it un­til you live it. Neward, in his

well known es­say, lays out many co­gent rea­sons why ORMs turn into quag­mires. In my ex­pe­ri­ence, I’ve had to deal di­rectly with a fair num­ber of them: en­tity iden­tity is­sues, dual-schema prob­lem, data

re­trieval mech­a­nism con­cern, and the par­tial-ob­ject prob­lem. I want to talk briefly about my ex­pe­ri­ences with these is­sues and add one of my own.

Perhaps the most sub­ver­sive is­sue I’ve had with ORMs is “attribute creep” or “wide ta­bles”, that is, ta­bles that just keep ac­cru­ing at­trib­utes. As much as I’d like to avoid it, some­times it be­comes nec­es­sary (although things like Postgres’ hstore can help). For ex­am­ple, a client may be pro­vid­ing you with lots of data that they want at­tached to re­ports based on var­i­ous busi­ness logic. Furthermore, you don’t have much in­sight into this data; you’re just schlep­ping it around.

This in and of it­self is­n’t a ter­ri­ble thing in a data­base. It be­comes a real pain point with an ORM. Specifically, the prob­lem starts to show up in any query that uses the en­tity di­rectly to cre­ate the query. You may have a Hibernate query like so early on in the pro­ject.

query(Foo.class).add(Re­stric­tion.eq(“x”, value))

This may be fine when Foo has five at­trib­utes, but be­comes a data fire hose when it has a hun­dred. This is the equiv­a­lent of us­ing SELECT *, which is usu­ally say­ing more than what is in­tended. ORMs, how­ever, en­cour­age this use and of­ten make writ­ing pre­cise pro­jec­tions as te­dious as they are in SQL. (I have op­ti­mized such queries by adding the ap­pro­pri­ate pro­jec­tion and re­duced the run time from min­utes to sec­onds; all the time was spent trans­lat­ing the data­base row into a Java ob­ject.)

Which leads to an­other bad ex­pe­ri­ence: the per­ni­cious use of for­eign keys. In the ORMs I’ve used, links be­tween classes are rep­re­sented in the data model as for­eign keys which, if not con­fig­ured care­fully, re­sult in a large num­ber of joins when re­triev­ing the ob­ject. (A re­cent count of one such table in my work re­sulted in over 600 at­trib­utes and 14 joins to ac­cess a sin­gle ob­ject, us­ing the pre­ferred query method­ol­ogy.)

Attribute creep and ex­ces­sive use of for­eign keys shows me is that in or­der to use ORMs ef­fec­tively, you still need to know SQL. My con­tention with ORMs is that, if you need to know SQL, just use SQL since it pre­vents the need to know how non-SQL gets trans­lated to SQL.

Knowing how to write SQL be­comes even more im­por­tant when you at­tempt to ac­tu­ally write queries us­ing an ORM. This is es­pe­cially im­por­tant when ef­fi­ciency is a con­cern.

From what I’ve seen, un­less you have a re­ally sim­ple data model (that is, you never do joins), you will be bend­ing over back­wards to fig­ure out how to get an ORM to gen­er­ate SQL that runs ef­fi­ciently. Most of the time, it’s more ob­fus­cated than ac­tual SQL.

And if you elect to keep the query sim­ple, you end up do­ing a lot of work in the code that could be done in the data­base faster. Window

func­tions are rel­a­tively ad­vanced SQL that is painful to write with ORMs. Not writ­ing them into the query likely means you will be trans­fer­ring a lot of ex­tra data from the data­base to your ap­pli­ca­tion.

In these cases, I’ve elected to write queries us­ing a tem­plat­ing sys­tem and de­scribe the ta­bles us­ing the ORM. I get the con­ve­nience of an ap­pli­ca­tion level de­scrip­tion of the table with di­rect use of SQL. It’s a lot less trou­ble than any­thing else I’ve used so far.

This one seems to be one of those un­avoid­able re­dun­dan­cies. If you try to get rid of it, you only make more prob­lems or add ex­ces­sive com­plex­ity.

The prob­lem is that you end up hav­ing a data de­f­i­n­i­tion in two places: the data­base and your ap­pli­ca­tion. If you keep the de­f­i­n­i­tion en­tirely in the ap­pli­ca­tion, you end up hav­ing to write the SQL Data Definition Language (DDL) with the ORM code, which is the same com­pli­ca­tion as writ­ing ad­vanced queries in the ORM. If you keep it in the data­base, you will prob­a­bly want a rep­re­sen­ta­tion in the ap­pli­ca­tion for con­ve­nience and to pre­vent too much “string typ­ing”.

I much pre­fer to keep the data de­f­i­n­i­tion in the data­base and read it into the ap­pli­ca­tion. It does­n’t solve the prob­lem, but it makes it more man­age­able. I’ve found that re­flec­tion tech­niques to get the data de­f­i­n­i­tion are not worth it and I suc­cumb to man­ag­ing the re­dun­dancy of data defini­tons in two places.

But the damn mi­gra­tion is­sue is a real kick in the teeth: chang­ing the model is no big deal in the ap­pli­ca­tion, but a real pain in the data­base. After all, data­bases are per­sis­tent whereas ap­pli­ca­tion data is not. ORMs sim­ply get in the way here be­cause they don’t help man­age data mi­gra­tion at all. I work on the prin­ci­ple that the data­base’s data de­f­i­n­i­tions aren’t things you should ma­nip­u­late in the ap­pli­ca­tion. Instead, ma­nip­u­late the re­sults of queries. That is, the queries are your API to the data­base. So in­stead of think­ing about ob­jects, I think about func­tions with re­turn types.

Thus, one is forced to ask, should you use an ORM for any­thing but con­ve­nience in mak­ing queries?

Dealing with en­tity iden­ti­ties is one of those things that you have to keep in mind at all times when work­ing with ORMs, forc­ing you to write for two sys­tems while only have the ex­pres­siv­ity of one.

When you have for­eign keys, you re­fer to re­lated iden­ti­ties with an iden­ti­fier. In your ap­pli­ca­tion, “identifier” takes on var­i­ous mean­ings, but usu­ally it’s the mem­ory lo­ca­tion (a pointer). In the data­base, it’s the state of the ob­ject it­self. These two things don’t re­ally get along be­cause you can re­ally only use data­base iden­ti­fiers in the data­base (the ul­ti­mate des­ti­na­tion of the data you’re work­ing with).

What this re­sults in is hav­ing to ma­nip­u­late the ORM to get a data­base iden­ti­fier by man­u­ally flush­ing the cache or do­ing a par­tial com­mit to get the ac­tual data­base iden­ti­fier.

I can’t even call this a leaky ab­strac­tion be­cause the work “leak” im­plies small amounts of the con­tents es­cap­ing rel­a­tive to the source.

Something that Neward al­ludes to is the need for de­vel­op­ers to han­dle trans­ac­tions. Transactions are dy­nam­i­cally scoped, which is a pow­er­ful but mostly ne­glected con­cept in pro­gram­ming lan­guages due to the con­fu­sion they cause if overused. This leads to a lot of boil­er­plate code with ex­cep­tion han­dlers and a care­ful con­sid­er­a­tion of where trans­ac­tion bound­aries should oc­cur. It also makes you pass ses­sion ob­jects around to any func­tion/​method that might have to com­mu­ni­cate with the data­base.

The con­cept of a trans­ac­tion trans­lates poorly to ap­pli­ca­tions due to their re­liance on con­text based on time. As men­tioned, dy­namic scop­ing is one way to use this in a pro­gram, but it is at odds with lex­i­cal scop­ing, the dom­i­nant par­a­digm. Thus, you must take great care to know about the “when” of a trans­ac­tion when writ­ing code that works with data­bases and can make mod­u­lar­ity tricky (“Here’s a use­ful func­tion that will only work in cer­tain con­texts”).

Where do I see my­self go­ing?

At this point, I’m start­ing to ques­tion the wis­dom be­hind the out­right re­jec­tion of stored pro­ce­dures. It sounds hereti­cal, but it may work for my use cases. (And hey, with the ad­vent of “devops”, the di­vide be­tween the de­vel­oper and the data­base ad­min­is­tra­tor is ba­si­cally non-ex­is­tent.)

I’ve found my­self think­ing about the data­base as just an­other data type that has an API: the queries. The queries re­turn val­ues of some type, which are rep­re­sented as some ob­ject in the pro­gram. By mov­ing away from think­ing of the ob­jects in my ap­pli­ca­tion as some­thing to be stored in a data­base (the rai­son d’être for ORMs) and in­stead think­ing of the data­base as a (large and com­plex) data type, I’ve found work­ing with a data­base from an ap­pli­ca­tion to be much sim­pler. And won­der­ing why I did­n’t see it ear­lier.

Regardless of whether I find that stored pro­ce­dures aren’t ac­tu­ally that evil or whether I keep us­ing tem­plated SQL, I do know one thing: I won’t fall into the “ORMs make it easy” trap. They are an ac­cept­able way to rep­re­sent a data de­f­i­n­i­tion, but a poor way to write queries and a bad way to store ob­ject state. If you’re us­ing an RDBMS, bite the bul­let and learn SQL.

Send com­ments to woz­niak.ca with the lo­cal part be­ing com­ment

What you’ll find on Do It Yurtself! Thanks for check­ing it out. Go build a yurt!

Videos by: - Jason Rayne - Octave Zangs

Photos by: - Bryan Aulick - Jory Block

Please make sure your browser sup­ports JavaScript and cook­ies and that you are not block­ing them from load­ing. For more in­for­ma­tion you can re­view our Terms of Service and Cookie Policy.

Progress against big prob­lems — the fo­cus of our work at Our World in Data — is pos­si­ble be­cause of new ideas: hu­man­ity started to win the war against in­fec­tious dis­eases when sci­en­tists started to un­der­stand that it is mi­cro­scopic pathogens that in­fect and kill us. The germ the­ory of dis­ease was just the first step; based on this in­sight oth­ers fol­lowed with more ideas — a myr­iad of new tech­nolo­gies, but also pub­lic health in­ter­ven­tions. It was only once we un­der­stood the ori­gins of in­fec­tious dis­eases that we re­al­ized the im­por­tance of now seem­ingly ob­vi­ous ideas, such as sup­ply­ing house­holds with clean wa­ter.1

The same is true for progress against other prob­lems: the ideas that sparked tech­no­log­i­cal in­no­va­tion made so­ci­eties richer, and break­throughs in the de­vel­op­ment of agri­cul­tural tech­nol­ogy made it pos­si­ble that while the world pop­u­la­tion in­creased rapidly hunger de­creased at the same time.

The list of in­no­va­tions we need is long: clean and cheap en­ergy, bet­ter crops, in­ter­ven­tions to help against the dis­eases that shorten and im­pair our lives. This, and much more be­sides, is needed to make progress against the big prob­lems we face. But while the de­mand for in­no­va­tion is large, its sup­ply is lim­ited.

Those who have bril­liant ideas are of­ten able to make a good liv­ing from them for them­selves. But be­yond their pri­vate re­turns big in­sights and dis­cov­er­ies ben­e­fit so­ci­ety as a whole. The very best ideas — think about peni­cillin, syn­thetic fer­til­iz­ers, or vac­cines — ben­e­fit every per­son on the planet.

For both rea­sons — per­sonal op­por­tu­nity and so­cial ben­e­fit — we should make sure that all tal­ented peo­ple have the chance to de­velop new ideas. But this is not the case.

’Who be­comes an in­ven­tor in America?’ — is the ti­tle of a study by Alex Bell, Raj Chetty, Xavier Jaravel, Neviana Petkova, and John Van Reenen pub­lished in 2019.2

It is based on an im­pres­sively com­pre­hen­sive dataset. The au­thors stud­ied the lives of 1.2 mil­lion in­ven­tors in the US from birth to adult­hood.3

The au­thors wanted to an­swer the ques­tion: what is it that mat­ters for whether a child grows up to be­come an in­ven­tor? They stud­ied two as­pects: per­sonal abil­ity at a young age and the cir­cum­stances in which a child grows up.

To study abil­ity they matched a sub­set of their data on in­ven­tors with records of their math scores back when they were third graders.4 To study the role that cir­cum­stances play the re­search team matched the in­ven­tor dataset with tax record data that al­lowed them to study the so­cio-eco­nomic sit­u­a­tion they grew up in.

Skills mat­ter a lot. Those who were good at maths at a young age were much more likely to be­come in­ven­tors later in life.

The find­ings on how much cir­cum­stances mat­ter are just as clear. Whether chil­dren grew up to be­come in­ven­tors later in life de­pends on the en­vi­ron­ment they grew up in.

The cir­cum­stances that mat­ter are to a large ex­tent eco­nomic: Those who grew up in richer fam­i­lies were much more likely to be­come in­ven­tors later in life. These dif­fer­ences are big. The study il­lus­trates this by com­par­ing stu­dents from poor and high-in­come fam­i­lies. The in­come thresh­old is high: the au­thors chose a cut­off that sep­a­rates the bot­tom 80% of American fam­i­lies from the rest. Of the high­est scor­ing 5% of stu­dents, those from high-in­come fam­i­lies were more than twice as likely to be­come in­ven­tors as those at lower in­comes.

The study fur­ther shows that very spe­cific cir­cum­stances play an ad­di­tional big role: Thanks to the very large dataset that in­cludes in­for­ma­tion from very dif­fer­ent re­gions within the coun­try, the re­searchers show that it is ex­po­sure to in­no­va­tors dur­ing child­hood that mat­ters. Children who grew up in an area, or bet­ter yet a fam­ily, with many in­ven­tors were more likely to fol­low their path and be­come in­no­va­tors them­selves. Children look for role mod­els, and when they are ex­posed to sci­ence and in­no­va­tion some end up fol­low­ing in their idol’s foot­steps.

Skills and cir­cum­stances mat­ter. What is tragic is when chil­dren have the abil­ity, but were de­nied the cir­cum­stances that would have helped them to ful­fill their po­ten­tial. The au­thors coined the term ‘Lost Einsteins’ for these chil­dren.

The re­searchers also stud­ied why is it that far fewer girls than boys grew up to be­come in­ven­tors. Differences in skills at a young age are not the ex­pla­na­tion: test scores of boys and girls were very sim­i­lar ac­cord­ing to the study.5 Instead the re­searchers find that tal­ented girls were much less likely to be­come in­no­va­tors than equally tal­ented boys and show that it is the ex­po­sure to role mod­els that can ex­plain these gen­der dif­fer­ences. Those girls who are ex­posed to women who are in­ven­tors are more likely to grow up to be in­ven­tors them­selves. The lack of role mod­els in the older gen­er­a­tion brings about a lack of role mod­els in the younger gen­er­a­tion. The gen­der gap in in­no­va­tors is still large. Only 18% of US in­ven­tors born in 1980 are women. To ex­tend the au­thors’ metaphor: even more com­mon than ‘Lost Einsteins’ are ‘Lost Marie Curies’.

The dis­ad­van­tage of some chil­dren comes at a cost to all of us. Overall, the re­searchers es­ti­mate that if high-abil­ity chil­dren from dis­ad­van­taged groups would ben­e­fit from the same cir­cum­stances as the best-off groups, there would be four times as many in­ven­tors in the US as there are to­day.

And we are not just los­ing in­ven­tors: we all los­ing out on teach­ers, artists, writ­ers, po­lit­i­cal lead­ers, sci­en­tists, painters, thinkers, mu­si­cians and the many other ways in which tal­ented young peo­ple can en­rich the world of all of us when they find them­selves in a sup­port­ing en­vi­ron­ment.

The study on in­ven­tors does not only shed light on the loss of in­ven­tors in our world to­day, but also help to un­der­stand the his­tory of in­no­va­tion. The world surely saw many ex­cep­tion­ally tal­ented peo­ple in our long his­tory, but for mil­len­nia they grew up in poverty, hunger, poor health, and with lit­tle ac­cess to ed­u­ca­tion. Very few bright chil­dren in our long his­tory lived in cir­cum­stances that made it pos­si­ble for them to ful­fill their po­ten­tial. Lost Marie Curies and Einsteins are not rare; there were many thou­sands of them through­out hu­man­i­ty’s his­tory.

At var­i­ous times some were lucky enough to en­joy bet­ter liv­ing con­di­tions and de­velop ideas that al­tered their so­ci­ety’s cir­cum­stances and cul­ture. Some of his­to­ry’s many ex­cep­tional peo­ple were for­tu­nate to be born in the past civ­i­liza­tions of an­cient Greece, China, Rome, or the Indus-valley and were able to de­velop artis­tic, sci­en­tific, or tech­no­log­i­cal in­no­va­tions that had a last­ing legacy. But for the vast ma­jor­ity, the daily strug­gle in the ex­tremely poor pre-growth economies was hard. Personal de­vel­op­ment was not easy in times when most peo­ple were re­quired to work in agri­cul­ture to pro­duce enough food, many were sick and un­der­nour­ished, many lived as slaves, and even very ba­sic ed­u­ca­tional skills — like read­ing and writ­ing — were a priv­i­lege of the very rich.

But two cen­turies ago the trickle of in­no­va­tors be­came a stream when, for the first time in his­tory, liv­ing con­di­tions started to im­prove for large parts of the pop­u­la­tion in en­tire world re­gions. The Industrial Revolution first changed Northwestern Europe where health im­proved, and in­comes in­creased. North America and Oceania fol­lowed soon af­ter. At the start of the 20th cen­tury life ex­pectancy in these re­gions had in­creased to 45 to 55 years, while the rest of the world re­mained with a life ex­pectancy be­tween 23 and 30. Only in the bet­ter-off re­gions did the av­er­age adult have some for­mal ed­u­ca­tion.

It was this im­prove­ment in liv­ing con­di­tions for the masses that was key to en­able fur­ther eco­nomic growth and im­prove­ments in liv­ing con­di­tions. It started a vir­tu­ous cir­cle be­tween ideas lead­ing to bet­ter liv­ing con­di­tions and bet­ter liv­ing con­di­tions en­abling bet­ter ideas. The sci­en­tific rev­o­lu­tion pow­ered the in­dus­trial rev­o­lu­tion and in­dus­tri­al­iza­tion in turn raised liv­ing con­di­tions and en­abled more peo­ple to con­tribute to sci­ence and in­no­va­tion.

The global in­equal­ity in liv­ing con­di­tions that emerged is re­flected in the big in­no­va­tions of the time. Penicillin, syn­thetic fer­til­iz­ers, and vac­cines were largely de­vel­oped in those places where chil­dren grew up in the best con­di­tions. The huge ma­jor­ity of Nobel Prize win­ners in the first half of the 20th cen­tury came from the rich­est re­gions on the planet — Europe and North America.6

Just as it is very hard for a child from a dis­ad­van­taged back­ground in the US to be­come an in­ven­tor, the same was true for bil­lions of chil­dren who grew up in poor liv­ing con­di­tions through­out his­tory. Children with great po­ten­tial ended up liv­ing a life in poverty, work­ing as farm­ers and strug­gling to feed their fam­i­lies and them­selves.

Being a great artist, sci­en­tist or in­ven­tor is of course not the only way to live a ful­filled life and con­tribute to so­ci­ety, but it is tragic when chil­dren who had the abil­ity were pre­vented from reach­ing their po­ten­tial. Tragic for them­selves, but also for all of us who missed out on their work that could have en­riched our lives, given us a bet­ter un­der­stand­ing of our world, and got­ten us closer to so­lu­tions of prob­lems that we all share.

Life is about much more than so­lu­tions to prob­lems and we are not only miss­ing out on in­no­va­tions, but also art, writ­ing, re­search, mu­sic — the en­tire breath of hu­man cre­ativ­ity that flour­ishes when tal­ent and ded­i­ca­tion come to­gether with en­abling cir­cum­stances. Our losses are unimag­in­able.

Maybe it is dif­fi­cult to imag­ine what the life of a tal­ented child could have looked like if he or she had not been born in a poor English vil­lage in the Middle Ages. But the same is true for dif­fer­ent places in our world to­day. In this con­text it might be eas­ier to imag­ine how liv­ing con­di­tions mat­ter.

What would have been the chances for Steve Jobs if he was born in the Central African Republic? And what would have been pos­si­ble for Taylor Swift if she grew up in Papua New Guinea in­stead of Pennsylvania and Nashville? And the other way around: What could the world’s poor­est chil­dren grow up to be if they had the cir­cum­stances of Steve Jobs or Taylor Swift?

What would our world look like if all tal­ented chil­dren had a good chance to be­come in­no­va­tors and artists in­stead of grow­ing up hun­gry, in poor health, and with poor ed­u­ca­tion?

Surely we all are miss­ing out on many of the most tal­ented peo­ple in the world. The US study showed that not be­long­ing to the rich­est 20% of American fam­i­lies gives chil­dren a much worse chance at be­ing an in­no­va­tor. Income in­equal­ity in the world is so large that Americans who are poorer than 90% of other Americans are richer than more than 70% of the world pop­u­la­tion.7

It is not just in terms of eco­nomic cir­cum­stances that the ma­jor­ity of chil­dren find them­selves worse off than chil­dren in America. Children at school-en­try age can ex­pect 16.5 years of ed­u­ca­tion in the US, but only 5 years on av­er­age in poor coun­tries. And es­ti­mates from the lat­est global data show that 60 mil­lion chil­dren of pri­mary-school age do not at­tend school at all, and those who go to school in poor coun­tries also learn much less. For a child grow­ing up in a poor place it is ex­tremely hard to ob­tain the level of ed­u­ca­tion an av­er­age per­son gets in the best-off places.

And the cited re­search sug­gests that it is not just in­come and ac­cess to ed­u­ca­tion that mat­ter. If the role model ef­fect that the US study em­pha­sized ex­ists every­where, then poor liv­ing con­di­tions are com­pounded by the fact that most chil­dren in most places are ex­posed to very few role mod­els in re­search and in­no­va­tion.

The map be­low shows the global dis­tri­b­u­tion of re­search and de­vel­op­ment, here given as the share of peo­ple em­ployed in re­search and de­vel­op­ment. The hotspots are six coun­tries in which more than 6000 per mil­lion in­hab­i­tants work in re­search and de­vel­op­ment: South Korea, Singapore, Israel and three Scandinavian coun­tries (Sweden, Finland, and Denmark). 6000 per mil­lion means that 1 out of 1667 peo­ple works in R&D. Even in the best-off coun­tries, very few fo­cus on in­no­va­tion.

As the his­tor­i­cal dis­cus­sion above would sug­gest there is a strong cor­re­la­tion be­tween eco­nomic de­vel­op­ment and re­search and de­vel­op­ment. In some poor coun­tries less than 30 per mil­lion are en­gaged in R&D.

What is true in the US, and what has been true through­out his­tory, is true for the world to­day: sci­ence, re­search, and de­vel­op­ment can flour­ish where large parts of the pop­u­la­tion are rel­a­tively well off.

If all coun­tries in the world had the same con­cen­tra­tion of re­searchers as rich coun­tries, there would be around three-times as many re­searchers in the world; if the en­tire world had the con­cen­tra­tion of the top six coun­tries there would be more than five-times higher.8

The cir­cum­stances for those re­searchers dif­fer widely too. Spending on R&D dif­fers be­tween 4.25% of GDP in Israel and South Korea to be­low 0.1% in many poor coun­tries.

Finding so­lu­tions to big prob­lems has al­ways been hard. And there is ev­i­dence that sug­gests that it is get­ting harder: Michael Nielsen and Patrick Collison (here) and also Scott Alexander re­cently ar­gued that while the world is cer­tainly ded­i­cat­ing much larger re­sources — both peo­ple and fund­ing — to sci­ence, re­search and de­vel­op­ment, the rate of progress in physics, med­i­cine, and other im­por­tant fields kept at best con­stant over the last cen­tury.9

The world’s most lim­ited re­source is not oil, iron, or coal, but the po­ten­tial of the hu­man mind.

We lack treat­ments for a large range of dis­eases; we need to de­car­bonize the global econ­omy; we should trans­form our food sys­tem to one that feeds the world while re­duc­ing its im­pact on the en­vi­ron­ment that we rely on; if any­one has the ideas to solve these prob­lems, every­one can ben­e­fit from them at the same time. Someone liv­ing in Europe or America ben­e­fits if some­one in India or Africa be­comes bet­ter off and in­vents that med­ical drug, the tech­nol­ogy to pro­duce cleaner en­ergy, or those new crops.

The de­mand for new ideas from which we would all ben­e­fit is enor­mous, but be­cause op­por­tu­ni­ties are lim­ited, the sup­ply of new ideas is lim­ited.

Talent and abil­ity are every­where, but op­por­tu­nity is not.

There are 2 bil­lion chil­dren un­der the age of 15. Focussing our ef­forts on im­prov­ing the con­di­tions in which the young gen­er­a­tion grows up is a big chal­lenge, but also a fan­tas­tic op­por­tu­nity. All who are in power to­day will be gone soon and the gen­er­a­tion who is grow­ing up now will take the lead. All will go through the ed­u­ca­tion sys­tem we of­fer them to­day; whether they can re­al­ize their po­ten­tial de­pends on the cir­cum­stances in health, shel­ter, food, pros­per­ity, free­dom, and ed­u­ca­tion that we can pro­vide for them.

Improving liv­ing con­di­tions is our moral duty, but be­yond that it is also the way to in­crease the sup­ply of much needed cre­ativ­ity and in­no­va­tion. This means that there is a self­ish rea­son for all of us to do what we can to sup­port global de­vel­op­ment. Better ed­u­ca­tion and im­prov­ing cir­cum­stances in which chil­dren grow up can al­low them to reach their po­ten­tial. Historically we have seen how im­prov­ing cir­cum­stances can lead to in­no­va­tion which feeds back to im­prov­ing liv­ing con­di­tions. These pos­i­tive feed­back loops have changed many places around the world and are needed to solve the big prob­lems that we all face. This is why it is so very im­por­tant to con­tinue the pos­i­tive de­vel­op­ments of the last decades with more chil­dren sur­viv­ing, more chil­dren grow­ing up free of the worst poverty, and more chil­dren be­ing bet­ter ed­u­cated than ever be­fore. If I am op­ti­mistic about the fu­ture of the world and progress against the world’s prob­lems it is be­cause of this.

What you can do — and also: this ar­ti­cle as a Kurz Gesagt video:

As I ar­gued in the ar­ti­cle, I be­lieve that the main rea­son why we should be in­ter­ested in im­prov­ing global liv­ing con­di­tions is a moral one: we need to do what is pos­si­ble to al­low every­one to live a life free of poverty, free of hunger, and free of pre­ma­ture death.

– Givewell.org stud­ies where your do­na­tion can make the biggest dif­fer­ence against the world’s large prob­lems.

– 80000hours.org helps you choose a ca­reer in which you can have the biggest pos­i­tive im­pact to make progress against the world’s largest prob­lems.

But in ad­di­tion to the moral ar­gu­ment, there is also a self­ish ar­gu­ment: be­cause we all ben­e­fit from every­one else’s ideas, we should all have an in­ter­est in en­abling every­one to live a good life. Last year I worked with the great Kurz Gesagt team on this video that is based on the ideas of this ar­ti­cle.

Everyone al­ways talks about how much money there is in pol­i­tics. This is the wrong fram­ing. The right fram­ing is Ansolabehere et al’s: why is there so lit­tle money in pol­i­tics? But Ansolabehere fo­cuses on elec­tions, and the mys­tery is wider than that.

Sure, dur­ing the 2018 elec­tion, can­di­dates, par­ties, PACs, and out­siders com­bined spent about $5 bil­lion — $2.5 bil­lion on Democrats, $2 bil­lion on Republicans, and $0.5 bil­lion on third par­ties. And al­though that sounds like a lot of money to you or me, on the na­tional scale, it’s puny. The US al­mond in­dus­try earns $12 bil­lion per year. Americans spent about 2.5x as much on al­monds as on can­di­dates last year.

But also, what about lob­by­ing? Open Secrets re­ports $3.5 bil­lion in lob­by­ing spend­ing in 2018. Again, sounds like a lot. But when we add $3.5 bil­lion in lob­by­ing to the $5 bil­lion in elec­tion spend­ing, we only get $8.5 bil­lion — still less than al­monds.

What about think tanks? Based on num­bers dis­cussed in this post, I es­ti­mate that the bud­get for all US think tanks, lib­eral and con­ser­v­a­tive com­bined, is prob­a­bly around $500 mil­lion per year. Again, an amount of money that I wish I had. But add it to the to­tal, and we’re only at $9 bil­lion. Still less than al­monds!

What about po­lit­i­cal ac­tivist or­ga­ni­za­tions? The National Rifle Association, the two-ton go­rilla of ad­vo­cacy groups, has a yearly bud­get of $400 mil­lion. The ACLU is a lit­tle smaller, at $234 mil­lion. AIPAC is $80 mil­lion. The NAACP is $24 mil­lion. None of them are any­where close to the first-per­son shooter video game “Overwatch”, which made $1 bil­lion last year. And when we add them all to the to­tal, we’re still less than al­monds.

Add up all US spend­ing on can­di­dates, PACs, lob­by­ing, think tanks, and ad­vo­cacy or­ga­ni­za­tions — lib­eral and con­ser­v­a­tive com­bined — and we’re still $2 bil­lion short of what we spend on al­monds each year. In fact, we’re still less than Elon Musk’s per­sonal for­tune; Musk could per­son­ally fund the en­tire US po­lit­i­cal ecosys­tem on both sides for a whole two-year elec­tion cy­cle.

But let’s go fur­ther.

According to this ar­ti­cle, Mic.com sold for less than $5 mil­lion. Mashable sold for less than $50 mil­lion. The whole Gawker net­work (plus some other stuff in­clud­ing the Onion) sold for $50 mil­lion. There are some hints that Vox is worth a high-eight-digit to low-nine-digit amount of money. The Washington Post was sold for $250 mil­lion in 2013 (though it’s prob­a­bly worth more now). These prop­er­ties seem to be priced en­tirely as cash cows — based on their abil­ity to make money through sub­scrip­tions or ads. The ex­tra value of us­ing them for po­lit­i­cal in­flu­ence seems to be priced around zero, and this price seems to be cor­rect based on how lit­tle money is spent on po­lit­i­cal causes.

Or: Jacobin spends a lot of time ad­vo­cat­ing so­cial­ism. The Economist spends a lot of time ad­vo­cat­ing lib­er­al­ism. First Things spends a lot of time ad­vo­cat­ing con­ser­vatism. They all have one thing in com­mon: pay­walls. How could this be ef­fi­cient? There are mil­lions of peo­ple who fol­low all of these philoso­phies and re­ally want to spread them. And there are other peo­ple who have ded­i­cated their lives to pro­duc­ing great sto­ries and es­says ad­vo­cat­ing and ex­plain­ing these philoso­phies — but peo­ple have to pay $29.99 for a sub­scrip­tion to read their work? Why do ide­olo­gies make peo­ple pay to read their pro­pa­ganda?

Maybe the most ex­treme ex­am­ple here is Tumblr.com, which re­cently sold for $3 mil­lion, ie the cost of a medium-sized house in San Francisco. Tumblr has 400 mil­lion monthly vis­i­tors, and at least tens of mil­lions of ac­tive users. These peo­ple talk pol­i­tics all the time, usu­ally of a far-left va­ri­ety. Nobody thinks that one of the cen­tral po­lit­i­cal dis­cus­sion plat­forms of the far-left is worth more than $3 mil­lion? Nobody on the right wants to shut it down? Nobody on the left wants to pre­vent that from hap­pen­ing? Nobody with a weird idio­syn­cratic agenda thinks be­ing able to pro­mote, cen­sor, or ad­ver­tise dif­fer­ent top­ics on a site with tens of mil­lions of po­lit­i­cally en­gaged peo­ple is at all in­ter­est­ing?

The low level of money in pol­i­tics should be re­ally sur­pris­ing for three rea­sons.

First, we should ex­pect or­di­nary peo­ple to do­nate more to pol­i­tics. A lot of the or­di­nary peo­ple I know care a lot about pol­i­tics. In many of the events they care about most, like the pres­i­den­tial pri­maries, small do­na­tions mat­ter a lot — just wit­ness Tom Steyer beg­ging for small do­na­tions de­spite be­ing a bil­lion­aire. If every American do­nated $25 to some can­di­date they sup­ported, elec­tion spend­ing would sur­pass the al­mond in­dus­try. But this is­n’t even close to hap­pen­ing. Bernie Sanders is rightly fa­mous for get­ting un­usu­ally many small do­na­tions from or­di­nary peo­ple. It’s not clear ex­actly how much he’s re­ceived, but it looks like about $50 mil­lion to­tal. This sounds like a lot of money, but if you use polls to es­ti­mate how many sup­port­ers he has, it looks like each sup­porter has on av­er­age given him $2. This is a nice to­ken ges­ture, but surely less than these peo­ple’s yearly al­mond bud­get.

Second, we should ex­pect the rich to do­nate more to pol­i­tics. Many politi­cians want to tax bil­lion­aires; bil­lion­aires pre­sum­ably want to pre­vent that from hap­pen­ing. Or wealthy peo­ple might just have hon­estly-held po­lit­i­cal opin­ions of their own. As rich as Elon Musk is, he’s only one of five hun­dred bil­lion­aires, and some of the oth­ers are even richer. So how come the amount of money in pol­i­tics is so much less than many in­di­vid­ual bil­lion­aires’ per­sonal for­tunes?

Third, we should ex­pect big cor­po­ra­tions to do­nate more to pol­i­tics. Post Citizens United, cor­po­ra­tions can sup­pos­edly put as much money into pol­i­tics as they want. And they should want a lot. The gov­ern­ment reg­u­lates cor­po­ra­tions, so hav­ing friendly politi­cians in power can mean life or death for en­tire in­dus­tries. Suppose hos­tile gov­ern­ment reg­u­la­tion could de­crease Exxon Mobil’s rev­enues 5% — you would think Exxon Mobil would be will­ing to spend 4% of its rev­enue to pre­vent this. But Exxon makes $280 bil­lion per year. 4% of its rev­enue would al­ready be larger than the whole US po­lit­i­cal ecosys­tem! In fact, ac­cord­ing to Exxon’s own records, they only spend about $1 mil­lion per cy­cle. While they’re prob­a­bly hid­ing some­thing, they could­n’t hide do­na­tions the size of the whole rest of the po­lit­i­cal ecosys­tem, so it’s still pretty mys­te­ri­ous.

I think there are in­di­vid­ual fac­tors af­fect­ing all of these. As men­tioned be­fore, elec­tions have spend­ing lim­its (however in­con­sis­tently en­forced) and may not be tractable to money. Think tanks may be more tal­ent-lim­ited than fund­ing-lim­ited. Media prop­er­ties may be lim­ited by the opin­ions of their jour­nal­ists and sub­scribers (the Washington Post could­n’t pivot to be­ing a con­ser­v­a­tive out­let with­out get­ting com­pletely dif­fer­ent em­ploy­ees and cus­tomers). Tumblr has al­ready proven un­able to cen­sor its users with­out spark­ing a mass ex­o­dus. These is­sues are prob­a­bly re­spon­si­ble for part of the un­der­pric­ing. But it still seems sur­pris­ing.

In his pa­per on elec­tions, Ansolabehere fo­cuses on the cor­po­rate per­spec­tive. He ar­gues that money nei­ther makes a can­di­date much more likely to win, nor buys much in­flu­ence with a can­di­date who does win. Corporations know this, which is why they don’t bother spend­ing more. Most re­search (plus the 2016 re­sults) con­firms that money has lit­tle ef­fect on vic­tory, so maybe this is true. But it would also have to be true that lob­by­ing, the NRA, the me­dia, etc don’t af­fect pol­i­tics very much, which seems like a harder sell.

That leaves the Bernie Sanders sup­port­ers. Even if money does­n’t af­fect pol­i­tics, Sanders sup­port­ers seem like about the least likely peo­ple to be­lieve that. I think here we have to go back to the same ex­pla­na­tion I give in Does Class Warfare Have A Free Rider Problem? People just can’t co­or­di­nate. If every­one who cared about home­less­ness do­nated $100 to the prob­lem, home­less­ness would be solved. Nobody does this, be­cause they know that no­body else is go­ing to do it, and their $100 is just go­ing to feel like a tiny drop in the ocean that does­n’t change any­thing. People know that a sin­gle per­son can’t make a dif­fer­ence, so they don’t want to spend any money, so no money gets spent. This is true for or­di­nary peo­ple, but it’s also true for bil­lion­aires and greedy cor­po­ra­tions. No sin­gle greedy cor­po­ra­tion wants to pony up the money to change the laws to fa­vor greedy cor­po­ra­tions all on its own, while its com­peti­tors lie back and free-ride on its hard work. So they ba­si­cally do­nate to­ken amounts and do noth­ing. By all ac­counts the Koch broth­ers ac­tu­ally be­lieved in every­thing they were do­ing, and they had to, be­cause you could­n’t make bil­lion­aires spend Koch-brothers-like lev­els of time and money out of self-in­ter­est.

In this model, the dif­fer­ence be­tween pol­i­tics and al­monds is that if you spend $2 on al­monds, you get $2 worth of al­monds. In pol­i­tics, if you spend $2 on Bernie Sanders, you get noth­ing, un­less mil­lions of other peo­ple also spend their $2 on him. People are great at spend­ing money on di­rect con­sump­tion goods, and ter­ri­ble at spend­ing money on co­or­di­na­tion prob­lems.

I don’t want more money in pol­i­tics. But the same fac­tors that keep money out of pol­i­tics keep it out of char­ity too.

The pol­i­tics case is in­ter­est­ing be­cause it’s so ob­vi­ous. Nobody’s go­ing to cyn­i­cally de­clare “Oh, peo­ple don’t re­ally care who wins the elec­tion, they just pre­tend to.” It’s co­or­di­na­tion prob­lems! It has to be!

So when I hear sto­ries like that Americans could end home­less­ness by redi­rect­ing the money they spend on Christmas dec­o­ra­tions, I don’t think that’s be­cause they’re evil or hyp­o­crit­i­cal or don’t re­ally care about the is­sue. I think they would if they could but the co­or­di­na­tion prob­lem gets in the way.

This is one rea­son I’m so gung ho about peo­ple pledg­ing to do­nate 10% of their in­come to char­ity. It mows through these kinds of prob­lems. I may not be a great per­son. But I spend more each year on the things I con­sider most im­por­tant than I do on al­monds, and this is the kind of thing that does­n’t hap­pen nat­u­rally. It’s the kind of thing where I have to force my­self to ig­nore the feel­ing of “just a drop in the ocean”, ig­nore whether I feel like other peo­ple are free-rid­ing on me, and just do it. Pledging to do­nate money (and then fig­ur­ing out what to do with it later) en­sures I will take that ef­fort, and not end up with re­vealed pref­er­ences that seem ridicu­lous in light of my val­ues.

An in­ter­ac­tive cheat­sheet tool for the com­mand-line so that you’ll never say the fol­low­ing again:

— How to run that com­mand again?

— Oh, it’s not in my bash his­tory

— Geez, it’s al­most what I wanted but I need to change some args

navi al­lows you to browse through cheat­sheets (that you may write your­self or down­load from main­tain­ers) and ex­e­cute com­mands, prompt­ing for ar­gu­ment val­ues.

brew in­stall denisidoro/​tools/​navi

git clone http://​github.com/​denisidoro/​navi /opt/navi

cd /opt/navi

sudo make in­stall

# in­stall fzf: https://​github.com/​june­gunn/​fzf

navi

Head to this play­ground for pre­view­ing navi.

* to in­crease dis­cov­er­abil­ity, by find­ing com­mands given key­words or de­scrip­tions;

* to pre­vent you from run­ning aux­il­iar com­mands, copy­ing the re­sult into the clip­board and then past­ing into the orig­i­nal com­mand;

* to eas­ily share one-lin­ers with oth­ers so that they don’t need to fig­ure out how to write the com­mands;

* to im­prove ter­mi­nal us­age as a whole.

Sure, you can find au­to­com­pleters out there for all your fa­vorite com­mands. However, they are very spe­cific and each one may of­fer a dif­fer­ent learn­ing curve.

Or you can launch a browser and search for in­struc­tions on Google, but that takes some time.

navi, on the other hand, in­tends to be a gen­eral pur­pose plat­form for book­mark­ing any com­mand at a very low cost.

In this case, you need to pass the di­rec­tory which con­tains .cheat files as in:

navi –dir “/folder/with/cheats”

Alternatively, you can set an en­vi­ron­ment vari­able in your .bashrc-like file:

ex­port NAVI_DIR=“/folder/with/cheats”

Feel free to fork this pro­ject and open a PR for me to in­clude your con­tri­bu­tions.

* lines start­ing with % should con­tain tags which will be added to any com­mand in a given file;

* lines start­ing with # should be de­scrip­tions of com­mands;

* lines start­ing with $ should con­tain com­mands that gen­er­ate sug­ges­tion val­ues for a given ar­gu­ment;

* all the other non-empty lines are con­sid­ered as ex­e­cutable com­mands.

For ex­am­ple, this is a valid .cheat file:

% git, code

# Change branch

git check­out

For ad­vanced us­age, please re­fer to the files in /cheats.

In The Legend of Zelda Ocarina of Time, navi is a char­ac­ter that pro­vides Link with a va­ri­ety of clues to help him solve puz­zles and progress in his quest.

Want cli­mate news in your in­box? Sign up here for Climate Fwd:, our email newslet­ter.

Greenhouse gas emis­sions from com­mer­cial air travel are grow­ing at a faster clip than pre­dicted in pre­vi­ous, al­ready dire, pro­jec­tions, ac­cord­ing to new re­search — putting pres­sure on air­line reg­u­la­tors to take stronger ac­tion as they pre­pare for a sum­mit next week.

The United Nations avi­a­tion body fore­casts that air­plane emis­sions of car­bon diox­ide, a ma­jor green­house gas, will reach just over 900 mil­lion met­ric tons in 2018, and then triple by 2050.

But the new re­search, from the International Council on Clean Transportation, found that emis­sions from global air travel may be in­creas­ing more than 1.5 times as fast as the U. N. es­ti­mate. The re­searchers an­a­lyzed nearly 40 mil­lion flights around the world last year.

“Airlines, for all in­tents and pur­poses, are be­com­ing more fuel ef­fi­cient. But we’re see­ing de­mand out­strip any of that,” said Brandon Graver, who led the new study. “The cli­mate chal­lenge for avi­a­tion is worse than any­one ex­pected.”

We’ve been harp­ing for a while on the stark eco­nomic di­vides that de­fine American life in the Donald Trump years.

To be sure, racial and cul­tural re­sent­ment have been the prime fac­tors of the Trump back­lash, but it’s also clear that the two par­ties speak for and to dra­mat­i­cally dif­fer­ent seg­ments of the American econ­omy. Where Republican ar­eas of the coun­try rely on lower-skill, lower-pro­duc­tiv­ity “traditional” in­dus­tries like man­u­fac­tur­ing and re­source ex­trac­tion, Democratic, mostly ur­ban dis­tricts con­tain large con­cen­tra­tions of the na­tion’s higher-skill, higher-tech pro­fes­sional and dig­i­tal ser­vices.

Yet now comes an­other wrin­kle to the story. Not only are red and blue America ex­pe­ri­enc­ing two dif­fer­ent economies, but those economies are di­verg­ing fast. In fact, rad­i­cal change is trans­form­ing the two par­ties’ economies in real time. Which is a key take­away of a new data analy­sis—pub­lished to­day—that we de­vel­oped with the Wall Street Journal’s Aaron Zitner and Dante Chinni.

What do the new num­bers show ex­actly? Based on stan­dard eco­nomic data linked to re­cent con­gres­sional dis­trict out­comes that we have tracked over time, the Journal/Brookings analy­sis de­picts above all the ex­treme pace in which the economies of the two par­ties’ dis­tricts are chang­ing in this decade.

Some of the change is al­ready fa­mil­iar based on how the map of con­gres­sional vote out­comes has evolved in the last decade:

In the 111th Congress in 2008, Democratic-voting, of­ten-ur­ban dis­tricts en­com­passed 39% of U. S. land area com­pared with the 61% ex­panse of Republican dis­tricts. By the 116th Congress, just ten years later, the Democratic share had fallen to 20% of the map, with Republicans’ ex­panse ris­ing to 80% of the na­tion’s land area. Certainly, those num­bers point to eco­nomic change, al­though they also re­flect ger­ry­man­der­ing and the low pop­u­la­tion den­sity of rural ar­eas.

In 2008, Democratic-voting, of­ten-ur­ban dis­tricts en­com­passed 39% of U. S. land area com­pared with the 61% ex­panse of Republican dis­tricts. By the 116th Congress, just ten years later, the Democratic share had fallen to 20% of the map.

Turn to our eco­nomic and de­mo­graphic data, how­ever, and it’s clear that a se­ries of gen­uine, pen­e­trat­ing shifts have been hap­pen­ing at warp speed through the last decade. These shifts are mas­sively al­ter­ing the two par­ties’ eco­nomic iden­ti­ties.

For one thing, the two par­ties have in just 10 years gone from near-par­ity on pros­per­ity and in­come mea­sures to stark, fast-mov­ing di­ver­gence.

With their out­put surg­ing as a re­sult of the big-city tilt of the decade’s “winner-take-most” econ­omy, Democratic dis­tricts have seen their me­dian house­hold in­come soar in a decade—from $54,000 in 2008 to $61,000 in 2018. By con­trast, the in­come level in Republican dis­tricts be­gan slightly higher in 2008, but then de­clined from $55,000 to $53,000.

Underlying these changes have been eye-pop­ping shifts in eco­nomic per­for­mance. Democratic-voting dis­tricts have seen their GDP per seat grow by a third since 2008, from $35.7 bil­lion to $48.5 bil­lion a seat, whereas Republican dis­tricts saw their out­put slightly de­cline from $33.2 bil­lion to $32.6 bil­lion.

Looking deeper, it’s clear that big shifts in in­dus­try ge­og­ra­phy and com­po­si­tion are dri­ving the par­ties’ changes of iden­tity. Look at the ma­trix of 10-year trends de­picted here:

Democratic dis­tricts, for ex­am­ple, have grown sig­nif­i­cantly more dy­namic in the last decade. Overall, “blue” ter­ri­to­ries have seen their pro­duc­tiv­ity climb from $118,000 per worker in 2008 to $139,000 in 2018 as re­cent de­mo­graphic changes and elec­toral sort­ing en­sured they be­came bet­ter ed­u­cated and more ur­ban. Republican-district pro­duc­tiv­ity, by con­trast, re­mains stuck at about $110,000, re­flect­ing only slight im­prove­ments of bach­e­lor’s de­gree at­tain­ment and Republicans’ in­creas­ingly non-metro do­main.

Relatedly, and equally strik­ing, Democratic dis­tricts are rapidly in­creas­ing their dom­i­nance of the na­tion’s ur­ban-tilt­ing pro­fes­sional and dig­i­tal ser­vices em­ploy­ment while ced­ing their his­tor­i­cal, more rural shares of man­u­fac­tur­ing and agri­cul­ture-min­ing ac­tiv­ity. Just since 2008, Democratic dis­tricts’ share of pro­fes­sional and dig­i­tal ser­vices em­ploy­ment surged from 63.7% to 71.1%, while their share of the na­tion’s man­u­fac­tur­ing and ex­trac­tive ac­tiv­i­ties shrunk from 53.8% to 43.6% and 46.1% to 39.5%, re­spec­tively. Conversely, Republican dis­tricts—fail­ing as a group to gain trac­tion in the new sec­tors—have re­verted to more “traditional” ones. GOP dis­tricts’ pro­fes­sional and dig­i­tal em­ploy­ment fell from 36.3% to 28.9% of the to­tal in just 10 years, for ex­am­ple, while their shares of man­u­fac­tur­ing and agri­cul­ture-min­ing in­creased from 46.2% to 56.4% and 53.9% to 60.5%, re­spec­tively.

Democratic dis­tricts have seen their me­dian house­hold in­come soar in a decade—from $54,000 in 2008 to $61,000 in 2018. By con­trast, the in­come level in Republican dis­tricts be­gan slightly higher in 2008, but then de­clined from $55,000 to $53,000.

Related to these shifts have been dra­matic de­mo­graphic changes. In just a decade, Democratic-voting dis­tricts have be­come strik­ingly bet­ter ed­u­cated and more di­verse. For ex­am­ple, Democratic-voting dis­tricts have seen their share of adults with at least a bach­e­lor’s de­gree rise from 28.4% in 2008 to 35.5%. For their part, Republican dis­tricts have barely in­creased their bach­e­lor’s de­gree at­tain­ment be­yond 26.6% and have mean­while be­come no­tably whiter and older. Today, there­fore, nei­ther party rep­re­sents the same types of places it did just 10 years ago. As such, the Democratic Party is now an­chored in the na­tion’s boom­ing, but highly un­equal, metro ar­eas, while the GOP re­lies on ag­ing and eco­nom­i­cally stag­nant man­u­fac­tur­ing-re­liant rural and ex­ur­ban com­mu­ni­ties.

What might these di­vides look like in the fu­ture? It’s hard to imag­ine the cur­rent ex­treme shifts go­ing much far­ther. The con­cen­tra­tion of more than 70% of the na­tion’s pro­fes­sional and dig­i­tal ser­vices econ­omy in the ter­ri­tory of one party would seem to reg­is­ter an al­most un­sus­tain­able de­gree of po­lar­iza­tion.

At the same time, there are few signs of any com­ing re­ver­sal of the decade’s di­ver­gence. Instead, the cur­rent eco­nomic trends un­der­lie the cur­rent party di­vide and re­in­force it. For at least the fore­see­able fu­ture, there­fore, the na­tion seems des­tined to strug­gle with ex­treme eco­nomic, ter­ri­to­r­ial, and po­lit­i­cal di­vides in which the two par­ties talk al­most en­tirely past each other on the most im­por­tant eco­nomic and so­cial is­sues, like in­no­va­tion, im­mi­gra­tion, and ed­u­ca­tion be­cause they rep­re­sent starkly sep­a­rate and di­verg­ing worlds. Not only do the two par­ties ad­here to very dif­fer­ent views, but they in­habit in­creas­ingly dif­fer­ent economies and en­vi­ron­ments.

“Blue” ter­ri­to­ries have seen their pro­duc­tiv­ity climb from $118,000 per worker in 2008 to $139,000 in 2018. Republican-district pro­duc­tiv­ity, by con­trast, re­mains stuck at about $110,000.

And yet, per­haps there are pos­si­ble meet­ing points. Maybe the ur­ban, di­verse, tech-ori­ented pro­fes­sional party and the ex­ur­ban-rural party of older white males will find com­mon cause on the “future of work” at a time of spread­ing dis­lo­ca­tion from au­toma­tion and the spread of pre­car­i­ous work in of­fices as well as fac­to­ries and ware­houses. Perhaps Democrats wor­ried about cli­mate change and Republicans weary of storm dam­age in the Midwest and along the Gulf Coast will be able to work to­gether on re­sponses to ex­treme weather. And for that mat­ter maybe the two camps will soon them­selves be­come wor­ried enough about to­day’s mas­sive re­gional and eco­nomic dis­par­i­ties to en­gage.

Or maybe not. It could be, in­stead, that the na­tion is only head­ing deeper into a pe­riod of con­tin­ued stand-off, where eco­nomic change re­in­forces cul­tural back­lash, and both ger­ry­man­dered dis­tricts and the Constitution’s al­lo­ca­tion of two sen­a­tors for every state prop up the po­lit­i­cal power of a de­clin­ing frac­tion of the econ­omy. If that’s how the pol­i­tics of the decade’s chang­ing econ­omy are go­ing to play out, then po­lit­i­cal an­a­lyst Ron Brownstein may be right that the na­tion may be head­ing deeper into an era of sus­tained tur­bu­lence that “pits what America has been against what it is be­com­ing.”

As HCL em­ploy­ees who work at Google like Johanne pre­pare to vote to join us Tuesday, they’re shar­ing their sto­ries of why unions are needed in the tech in­dus­try. RT to wish them luck! #1u #USWUnity pic.twit­ter.com/​bZYt0RaKcf

— United Steelworkers (@steelworkers) September 20, 2019