A real-world pro­duc­tion mi­gra­tion from DigitalOcean to Hetzner ded­i­cated, han­dling 248 GB of MySQL data across 30 data­bases, 34 Nginx sites, GitLab EE, Neo4j, and live mo­bile app traf­fic — with zero down­time.

Running a soft­ware com­pany in Turkey has be­come in­creas­ingly ex­pen­sive over the last few years. Skyrocketing in­fla­tion and a dra­mat­i­cally weak­en­ing Turkish Lira against the US dol­lar have turned dol­lar-de­nom­i­nated in­fra­struc­ture costs into a se­ri­ous bur­den. A bill that felt man­age­able two years ago now hits very dif­fer­ently when the ex­change rate has mul­ti­plied sev­eral times over.

Every month, we were pay­ing $1,432 to DigitalOcean for a droplet with 192GB RAM, 32 vC­PUs, 600GB SSD, two block vol­umes (1TB each), and back­ups en­abled. The server was fine — but the price-to-per­for­mance ra­tio had stopped mak­ing sense.

Then we dis­cov­ered the Hetzner AX162-R.

That’s $14,388 saved per year — for a server that’s ob­jec­tively more pow­er­ful in every di­men­sion. The de­ci­sion was easy.

I’ve been a DigitalOcean cus­tomer for nearly 8 years. They have a great prod­uct and I have no com­plaints about re­li­a­bil­ity or de­vel­oper ex­pe­ri­ence. But look­ing at those num­bers now, I can­not help feel­ing a bit sad about all the ex­tra money I left on the table over the years. If you are run­ning steady-state work­loads and not ac­tively us­ing DO’s ecosys­tem fea­tures, do your­self a fa­vor and check ded­i­cated server pric­ing be­fore your next re­newal.

* Several live mo­bile apps serv­ing hun­dreds of thou­sands of users

Old server: CentOS 7 — long past its end-of-life, but still run­ning in pro­duc­tion. New server: AlmaLinux 9.7 — a RHEL 9 com­pat­i­ble dis­tri­b­u­tion and the nat­ural suc­ces­sor to CentOS. This mi­gra­tion was also an op­por­tu­nity to fi­nally es­cape an OS that had­n’t re­ceived se­cu­rity up­dates in years.

The naive ap­proach — change DNS, restart every­thing, hope for the best — was­n’t ac­cept­able. Instead, we de­signed a proper mi­gra­tion path with six phases:

Phase 1 — Full stack in­stal­la­tion on the new server

Nginx (compiled from source with iden­ti­cal flags), PHP (via Remi repo, with the same .ini con­fig files from the old server), MySQL 8.0, Neo4J Graph DB, GitLab EE, Node.js, Supervisor, and Gearman. Every ser­vice had to be con­fig­ured to match the old server’s be­hav­ior be­fore we touched a sin­gle DNS record.

SSL cer­tifi­cates were han­dled by rsync­ing the en­tire /etc/letsencrypt/ di­rec­tory from the old server to the new one. After the mi­gra­tion was com­plete and all traf­fic was flow­ing through the new server, we force-re­newed all cer­tifi­cates in one shot:

Phase 2 — Web files cloned with rsync

The en­tire /var/www/html di­rec­tory (~65 GB, 1.5 mil­lion files) was cloned to the new server us­ing rsync over SSH with the –checksum flag for in­tegrity ver­i­fi­ca­tion. We ran a fi­nal in­cre­men­tal sync right be­fore cu­tover to catch any files changed af­ter the ini­tial clone.

Phase 3 — MySQL mas­ter to slave repli­ca­tion

Rather than tak­ing the data­base of­fline for a dump-and-re­store, we set up live repli­ca­tion. The old server be­came mas­ter, the new server a read-only slave. We used my­dumper for the ini­tial bulk load, then started repli­ca­tion from the ex­act bin­log po­si­tion recorded in the dump meta­data. This kept both data­bases in real-time sync un­til the mo­ment of cu­tover.

Phase 4 — DNS TTL re­duc­tion

We scripted the DigitalOcean DNS API to lower all A and AAAA record TTLs from 3600 to 300 sec­onds — with­out touch­ing MX or TXT records (changing mail record TTLs can cause de­liv­er­abil­ity is­sues). After wait­ing one hour for old TTLs to ex­pire glob­ally, we were ready to cut over in un­der 5 min­utes.

Phase 5 — Old server ng­inx con­verted to re­verse proxy

We wrote a Python script that parsed every server {} block across all 34 Nginx site con­figs, backed up the orig­i­nals, and re­placed them with proxy con­fig­u­ra­tions point­ing to the new server. This meant that dur­ing DNS prop­a­ga­tion, any re­quest still hit­ting the old IP was silently for­warded. No user would see a dis­rup­tion.

Phase 6 — DNS cu­tover and de­com­mis­sion

A sin­gle Python script hit the DigitalOcean API and flipped all A records to the new server IP in sec­onds. The old server re­mained as a cold standby for one week, then was shut down.

The key in­sight: at no point did we have a win­dow where the ser­vice was un­avail­able. Traffic was al­ways be­ing served — ei­ther di­rectly or through the proxy.

This was the most com­plex part of the en­tire op­er­a­tion.

We used my­dumper in­stead of the stan­dard mysql­dump — and it made an enor­mous dif­fer­ence. By lever­ag­ing the new server’s 48 CPU cores for par­al­lel ex­port and im­port, what would have taken days with a tra­di­tional sin­gle-threaded mysql­dump was com­pleted in hours. If you’re mi­grat­ing a large MySQL data­base and you’re not us­ing my­dumper/​my­loader, you’re do­ing it the hard way.

The main dump’s meta­data file recorded the bin­log po­si­tion at the time of the snap­shot:

File: mysql-bin.000004

Position: 21834307

This would be our repli­ca­tion start­ing point.

Once the dump was com­plete, we trans­ferred it to the new server us­ing rsync over SSH. With 248 GB of com­pressed chunks, this was sig­nif­i­cantly faster than any other trans­fer method:

The –compress flag in my­dumper paid off here — com­pressed chunks trans­ferred much faster over the wire.

Being stuck on CentOS 7 meant we were also stuck on MySQL 5.7 — an out­dated ver­sion that had been run­ning in pro­duc­tion for years. Before the mi­gra­tion, we ran mysqlcheck –check-upgrade to ver­ify that our data was com­pat­i­ble with MySQL 8.0. It came back clean, so we in­stalled the lat­est MySQL 8.0 Community on the new server. The per­for­mance im­prove­ment across all our pro­jects was im­me­di­ately no­tice­able — query ex­e­cu­tion times dropped sig­nif­i­cantly thanks to MySQL 8.0’s im­proved op­ti­mizer and InnoDB en­hance­ments.

That said, the ver­sion jump did in­tro­duce one tricky prob­lem.

After im­port, the mysql.user table had the wrong col­umn struc­ture — 45 columns in­stead of the ex­pected 51. This caused mysql.in­fos­chema to be miss­ing, break­ing user au­then­ti­ca­tion.

But this failed the first time with:

ERROR: ‘sys.innodb_buffer_stats_by_schema’ is not VIEW

The sys schema had been im­ported as reg­u­lar ta­bles in­stead of views. Solution:

With both dumps im­ported, we con­fig­ured the new server as a replica of the old one:

Almost im­me­di­ately, repli­ca­tion stopped with er­ror 1062 (Duplicate Key). This hap­pened be­cause our dump was taken in two passes — dur­ing the gap be­tween them, rows were writ­ten to cer­tain ta­bles, and now both the im­ported dump and the bin­log re­play were try­ing to in­sert the same rows.

IDEMPOTENT mode silently skips du­pli­cate key and miss­ing row er­rors. All crit­i­cal data­bases synced with­out a sin­gle er­ror. Within a few min­utes, Seconds_Behind_Master dropped to 0.

Before touch­ing a sin­gle DNS record, we needed to ver­ify that all ser­vices were work­ing cor­rectly on the new server. The trick: we tem­porar­ily edited the /etc/hosts file on our lo­cal ma­chine to point our do­main names to the new server’s IP.

# /etc/hosts (local ma­chine)

NEW_SERVER_IP your­do­main1.com

NEW_SERVER_IP your­do­main2.com

# … and so on for all your do­mains

With this in place, our browsers and Postman would hit the new server while the rest of the world was still go­ing to the old one. We ran through our API end­points, checked ad­min pan­els, and ver­i­fied that every ser­vice was re­spond­ing cor­rectly. Only af­ter this con­fir­ma­tion did we pro­ceed with the cu­tover.

Once mas­ter-slave repli­ca­tion was fully syn­chro­nized, we no­ticed that INSERT state­ments were suc­ceed­ing on the new server when they should­n’t have been — read­_only = 1 was set, but writes were go­ing through.

The rea­son: all PHP ap­pli­ca­tion users had been granted SUPER priv­i­lege. In MySQL, SUPER by­passes read­_only.

We re­voked it from all 24 ap­pli­ca­tion users:

After this, read­_only = 1 cor­rectly blocked all writes from ap­pli­ca­tion users while al­low­ing repli­ca­tion to con­tinue.

All do­mains were man­aged through DigitalOcean DNS (with name­servers pointed from GoDaddy). We scripted the TTL re­duc­tion against the DigitalOcean API, only touch­ing A and AAAA records — not MX or TXT records, since chang­ing mail record TTLs can cause de­liv­er­abil­ity is­sues with Google Workspace.

After wait­ing one hour for old TTLs to ex­pire, we were ready.

Rather than edit­ing 34 con­fig files by hand, we wrote a Python script that parsed every server {} block in every con­fig file, iden­ti­fied the main con­tent blocks, re­placed them with proxy con­figs, and backed up orig­i­nals as .backup files.

The key: prox­y_ss­l_ver­ify off — the new server’s SSL cert is valid for the do­main, not for the IP ad­dress. Disabling ver­i­fi­ca­tion here is fine be­cause we con­trol both ends.

With repli­ca­tion at Seconds_Behind_Master: 0 and the re­verse proxy ready, we ex­e­cuted the cu­tover in or­der:

1. New server: STOP SLAVE;

2. New server: SET GLOBAL read­_only = 0;

3. New server: RESET SLAVE ALL;

4. New server: su­per­vi­sor­ctl start all

5. Old server: ng­inx -t && sys­tem­ctl re­load ng­inx (proxy goes live)

6. Old server: su­per­vi­sor­ctl stop all

7. Mac: python3 do_­cu­tover.py (DNS: all A records to new server IP)

8. Wait: ~5 min­utes for prop­a­ga­tion

9. Old server: com­ment out all crontab en­tries

The DNS cu­tover script hit the DigitalOcean API and changed every A record to the new server IP — in about 10 sec­onds.

After mi­gra­tion, we dis­cov­ered many GitLab pro­ject web­hooks were still point­ing to the old server IP. We wrote a script to scan all pro­jects via the GitLab API and up­date them in bulk.

We went from $1,432/month down to $233/month — sav­ing $14,388 per year. And we ended up with a more pow­er­ful ma­chine:

The en­tire mi­gra­tion took roughly 24 hours. No users were af­fected.

MySQL repli­ca­tion is your best friend for zero-down­time mi­gra­tions. Set it up early, let it catch up, then cut over with con­fi­dence.

Check your MySQL user priv­i­leges be­fore mi­gra­tion. SUPER priv­i­lege by­passes read­_only — if your app users have it, your slave en­vi­ron­ment is­n’t ac­tu­ally read-only.

Script every­thing. DNS up­dates, ng­inx con­fig rewrites, web­hook up­dates — do­ing these by hand across 34+ sites would have taken hours and in­tro­duced er­rors.

my­dumper + my­loader dra­mat­i­cally out­per­forms mysql­dump for large datasets. Parallel dump/​re­store with 32 threads cut what would have been days of work down to hours.

Cloud providers are ex­pen­sive for steady-state work­loads. If you’re not us­ing au­toscal­ing or ephemeral in­fra­struc­ture, a ded­i­cated server of­ten de­liv­ers bet­ter per­for­mance at a frac­tion of the cost.

All Python scripts used in this mi­gra­tion are open-sourced and avail­able on GitHub:

* do_list_­do­main­s_ttl.py — List all DigitalOcean do­mains with their A records, IPs, and TTLs

* do_­to_het­zn­er_bulk_dns_record­s_im­port.py — Migrate all DNS zones from DigitalOcean to Hetzner DNS

* do_­cu­tover_­to_new_ip.py — Flip all A records from old server IP to new server IP

* mysql_­com­pare.py — Compare row counts across all ta­bles on two MySQL servers

* fi­nal_git­lab_web­hook_up­date.py — Update all GitLab pro­ject web­hooks to the new server IP

All scripts sup­port a DRY_RUN = True mode so you can safely pre­view changes be­fore ap­ply­ing them.

Anonymous re­quest-to­ken com­par­isons from the com­mu­nity, show­ing how Opus 4.6 and Opus 4.7 dif­fer on real in­puts

Are the Costs of AI Agents Also Rising Exponentially?

There is an ex­tremely im­por­tant ques­tion about the near-fu­ture of AI that al­most no-one is ask­ing. We’ve all seen the graphs from METR show­ing that the length of tasks AI agents can per­form has been grow­ing ex­po­nen­tially over the last 7 years. While GPT-2 could only do soft­ware en­gi­neer­ing tasks that would take some­one a few sec­onds, the lat­est mod­els can (50% of the time) do tasks that would take a hu­man a few hours.

As this trend shows no signs of stop­ping, peo­ple have nat­u­rally taken to ex­trap­o­lat­ing it out, to fore­cast when we might ex­pect AI to be able to do tasks that take an en­gi­neer a full work-day; or week; or year. But we are miss­ing a key piece of in­for­ma­tion — the cost of per­form­ing this work. Over those 7 years AI sys­tems have grown ex­po­nen­tially. The size of the mod­els (parameter count) has grown by 4,000x and the num­ber of times they are run in each task (tokens gen­er­ated) has grown by about 100,000x. AI re­searchers have also found mas­sive ef­fi­cien­cies, but it is em­i­nently plau­si­ble that the cost for the peak per­for­mance mea­sured by METR has been grow­ing — and grow­ing ex­po­nen­tially.This might not be so bad. For ex­am­ple, if the best AI agents are able to com­plete tasks that are 3x longer each year and the costs to do so are also in­creas­ing by 3x each year, then the cost to have an AI agent per­form tasks would re­main the same mul­ti­ple of what it costs a hu­man to do those tasks. Or if the costs have a longer dou­bling time than the time-hori­zons, then the AI-systems would be get­ting cheaper com­pared with hu­mans. But what if the costs are grow­ing more quickly than the time hori­zons? In that case, these cut­ting-edge AI sys­tems would be get­ting less cost-com­pet­i­tive with hu­mans over time. If so, the METR time-hori­zon trend could be mis­lead­ing. It would be show­ing how the state of the art is im­prov­ing, but part of this progress would be due to more and more lav­ish ex­pen­di­ture on com­pute so it would be di­verg­ing from what is eco­nom­i­cal. It would be be­com­ing more like the Formula 1 of AI per­for­mance — show­ing what is pos­si­ble, but not what is prac­ti­cal. So in my view, a key ques­tion we need to ask is:        How is the ‘hourly’ cost of AI agents chang­ing over time?By ‘hourly’ cost I mean the fi­nan­cial cost of us­ing an LLM to com­plete a task right at the mod­el’s 50% time hori­zon di­vided by the length of that time hori­zon. So as with the METR time hori­zons them­selves, the du­ra­tions are mea­sured not by how long it takes the model, but how long it typ­i­cally takes hu­mans to do that task. For ex­am­ple, Claude 4.1 Opus’s 50% time hori­zon is 2 hours: it can suc­ceed in 50% of tasks that take hu­man soft­ware en­gi­neers 2 hours. So we can look at how much it costs for it to per­form such a task and di­vide by 2, to find its hourly rate for this work. I’ve found that very few peo­ple are ask­ing this ques­tion. And when I ask peo­ple what they think is hap­pen­ing to these costs over time, their opin­ions vary wildly. Some as­sume the to­tal cost of a task is stay­ing the same, even as the task length in­creases ex­po­nen­tially. That would im­ply an ex­po­nen­tially de­clin­ing hourly rate. Others as­sume the to­tal cost is also grow­ing ex­po­nen­tially — af­ter all, we’ve seen dra­matic in­creases in the costs to ac­cess cut­ting-edge mod­els. And most peo­ple (myself in­cluded) had lit­tle idea of how much it cur­rently costs for AI agents to do an hour’s soft­ware en­gi­neer­ing work. Are we talk­ing cents? Dollars? Hundreds of dol­lars? An AI agent can’t cost more per hour than a hu­man to com­plete these tasks can it? Can it?A cou­ple of months ago I asked METR if they could share the cost data for their bench­mark­ing. I fig­ured it would be easy — just take the cost of run­ning their bench­mark for each model, plot it against re­lease date and see how it is grow­ing. Or plot the cost of each model vs its time hori­zon and see the re­la­tion­ship.But they help­fully pointed out that it is­n’t so easy at all. Their head­line time-hori­zon num­bers are meant to show the best pos­si­ble per­for­mance that can be at­tained with a model (regardless of cost). So they run their mod­els in­side an agent scaf­fold un­til the per­for­mance has plateaued. Since they re­ally want to make sure it has plateaued, they use a lot of com­pute on this and don’t worry too much about whether they’ve used too much. After all, if you are just try­ing to find the even­tual height of a plateau, there is no prob­lem in go­ing far into the flat part of the graph. But if you are try­ing to find out when the plateau be­gins, there is a prob­lem with this strat­egy. Their to­tal spend for each model is some­times just enough to get onto the plateau and some­times many times more than is needed. So to­tal spend can’t be used as di­rect es­ti­mate of the costs of achiev­ing that per­for­mance. Fortunately, they re­leased a chart that can be used to shed some light on the key ques­tion of how hourly costs of LLM agents are chang­ing over time:

This chart (from METR’s page for GPT-5) shows how per­for­mance in­creases with cost. The cost in ques­tion is the cost of us­ing more and more to­kens to com­plete the task (and thus more and more com­pute).The yel­low curve is the best hu­man per­for­mance for each task. It steadily marches on­wards and up­wards, trans­form­ing more wages into longer tasks. Since it is hu­man per­for­mance that is used to de­fine the ver­ti­cal axis for METR’s time hori­zon work, it is­n’t sur­pris­ing that this curve is fairly lin­ear — it costs about 8 times as much to get a hu­man soft­ware en­gi­neer to per­form an 8-hour task as a 1-hour task.The other colours are the curves for a se­lec­tion of LLM-based agents. Unlike the hu­mans, they all show di­min­ish­ing re­turns, with the time hori­zon each one can achieve even­tu­ally stalling out and plateau­ing as more and more com­pute is added. The short upticks at the end of some of these curves are an arte­fact of some mod­els not be­ing pre­pared to give an an­swer un­til the last avail­able mo­ment. This sug­gests that the model must have been still mak­ing progress dur­ing the ap­par­ent flat­line be­fore the uptick (just not show­ing it). Indeed, this chart was orig­i­nally dis­played on METR’s page for GPT-5 to show that they may have stopped its run be­fore it’s per­for­mance had truly plateaued. These upticks do make analy­sis harder and hope­fully fu­ture ver­sions of this chart will be able to avoid these glitches.So what can this chart tell us about our key ques­tion con­cern­ing the hourly cost of AI agents?To tease out the lessons that lie hid­den in the chart, we’ll need to add a num­ber of an­no­ta­tions. The first step is to add lines of con­stant hourly cost. On a log-log plot like this, every con­stant hourly cost will be a straight line with slope 1. Lower hourly costs will ap­pear as lines that are lo­cated fur­ther to the left.

For each curve I’ve added a line of con­stant hourly cost that just grazes it. That is the cheap­est hourly cost the model achieves. We can call the point where the line touches the curve the sweet spot for that model. Before a mod­el’s sweet spot, its time hori­zon is grow­ing su­per-lin­early in cost — it is get­ting in­creas­ing mar­ginal re­turns. The sweet spot is ex­actly the point at which di­min­ish­ing mar­ginal re­turns set in (which would cor­re­spond to the point of in­flec­tion if this was re­plot­ted on lin­ear axes). It is thus a key point on any mod­el’s per­for­mance curve.We can see that the hu­man soft­ware en­gi­neer is at best \$120 per hour, while the sweet spots for the AI agents range from \$40 per hour for o3, all the way down to 40 cents per hour for Grok 4 and Sonnet 3.5. That’s quite a range of costs. While dif­fer­ences in hori­zon length be­tween these mod­els vary by about a fac­tor of 15 (judged at ei­ther the end-points or at the sweet-spots) their sweet-spot costs vary by a fac­tor of 100.And these are the best hourly rates for these mod­els. On many task lengths (including those near their plateau) they cost 10 to 100 times as much per hour. For in­stance, Grok 4 is at \$0.40 per hour at its sweet spot, but \$13 per hour at the start of its fi­nal plateau. GPT-5 is about \$13 per hour for tasks that take about 45 min­utes, but \$120 per hour for tasks that take 2 hours. And o3 ac­tu­ally costs \$350 per hour (more than the hu­man price) to achieve tasks at its full 1.5 hour task hori­zon. This is a lot of money to pay for an agent that fails at the task you’ve just paid for 50% of the time — es­pe­cially in cases where fail­ure is much worse than not hav­ing tried at all.How­ever, I do want to note that I’m a bit puz­zled by how much higher the costs are here for the rea­son­ing mod­els from OpenAI com­pared to mod­els from Anthropic and xAI. The METR page sug­gests that the price data for those mod­els was still an es­ti­mate at that point (based on o1 costs), so I would­n’t be sur­prised if these curves should re­ally be shifted some­what to the left, mak­ing them sev­eral times cheaper. We there­fore should­n’t lean too heav­ily on the fact that they cost as much or more than hu­man labour at their full time-hori­zon.As well as the sweet spot, ide­ally we could add a sat­u­ra­tion point for each curve — a point to rep­re­sent the lo­ca­tion where the plateau be­gins. We can’t sim­ply use the end of the curve since some have run longer into the plateau than oth­ers. What I’ll do is find the point where the slope has di­min­ished to 1/10th that of the sweet spot. This is the point at which it re­quires a 10% in­crease in cost just to in­crease the time hori­zon by 1%. Or equiv­a­lently, the time hori­zon is only grow­ing as the 1/10th power of com­pute. Of course the num­ber 1/10 is some­what ar­bi­trary, but un­like for the sweet spot, any de­f­i­n­i­tion of a sat­u­ra­tion point will be ar­bi­trary to some de­gree. As you can see be­low, this de­f­i­n­i­tion of sat­u­ra­tion point does roughly cor­re­spond with the in­tu­itive lo­ca­tion, though it is still not quite clear how best to deal with the fi­nal upticks.

Armed with our sweet spots and sat­u­ra­tion points, we can start to tease out the re­la­tion­ship be­tween time hori­zon and cost.

We can see that there is a weak, but clear, pos­i­tive cor­re­la­tion be­tween task du­ra­tion and cost in this dataset. Moreover, we see that higher task du­ra­tions (at the sweet spot) are as­so­ci­ated with higher hourly costs (and re­call that these hourly costs at the sweet spot are the best hourly cost achiev­able with that model).What about if we in­stead look at the mod­els’ sat­u­ra­tion points, which are a lit­tle ar­bi­trary in their de­f­i­n­i­tion, but closer to what METR is mea­sur­ing in their head­line re­sults about time hori­zons:

Again, there is a cor­re­la­tion be­tween time hori­zon and cost, and again the hourly costs seem to be in­creas­ing with time hori­zon too. Indeed it sug­gests we are near­ing the point where the mod­els’ peak per­for­mance comes at an im­prac­ti­cally high cost. If this re­la­tion­ship were to con­tinue, then fore­cast­ing when cer­tain time hori­zons will be avail­able from the head­line METR trend will be mis­lead­ing, as the mod­els would be im­prac­ti­cally ex­pen­sive when they first reach those ca­pa­bil­i­ties. We would need to wait some ad­di­tional pe­riod of time for them to come down suf­fi­ciently in cost.That said, there are some sig­nif­i­cant lim­i­ta­tions to the analy­sis above. Ideally one would want to:in­clude curves for a larger and more rep­re­sen­ta­tive set of mod­els­find a way of ad­dress­ing the uptick prob­lem­check if there is an is­sue with the costs of the OpenAI mod­els­For­tu­nately, it should be fairly easy for METR to per­form such analy­sis, and I hope they will fol­low up on this.Too few peo­ple are ask­ing about how the costs of AI agents are grow­ingThe key ques­tion is How is the ‘hourly’ cost of LLM agents chang­ing over time?We can use METR’s chart to shed some light on this.We need to add lines of con­stant hourly cost, sweet spots, and sat­u­ra­tion points.This pro­vides mod­er­ate ev­i­dence that:the costs to achieve the time hori­zons are grow­ing ex­po­nen­tially,even the hourly costs are ris­ing ex­po­nen­tially,the hourly costs for some mod­els are now close to hu­man costs.Thus, there is ev­i­dence that:the METR trend is partly dri­ven by un­sus­tain­ably in­creas­ing in­fer­ence com­puteth­ere will be a di­ver­gence be­tween what time hori­zon is pos­si­ble in-prin­ci­ple and what is eco­nom­i­cally fea­si­ble­real-world ap­pli­ca­tions of AI agents will lag be­hind the METR time-hori­zon trend by in­creas­ingly large amountsMETR has a sim­i­lar graph on their page for GPT-5.1 codex. It in­cludes more mod­els and com­pares them by to­ken counts rather than dol­lar costs:

the cor­re­la­tion be­tween time hori­zon and cost holds for these other mod­els toore­a­son­ing mod­els with more RL post-train­ing don’t al­ways dom­i­nate their pre­de­ces­sors (e.g. o1 is bet­ter at small to­ken bud­gets than o3 or GPT-5)the hor­i­zon­tal gap be­tween the OpenAI rea­son­ing mod­els and the rest is smaller, sup­port­ing the idea that their costs were a bit high in the main chart

$\setCounter{0}$

February 04, 2026

Hazard Rates for AI Agents Decline as a Task Goes On

In 2025, the Kdenlive team con­tin­ued grind­ing to push the pro­ject for­ward through steady de­vel­op­ment, col­lab­o­ra­tion, and com­mu­nity sup­port. Over the past year we’ve found a nice bal­ance be­tween adding new fea­tures, bug fix­ing, pol­ish­ing the user in­ter­face, and im­prov­ing per­for­mance and work­flow, with sta­bil­ity tak­ing pri­or­ity over fea­ture creep. We re­launched the web­site with a new con­tent man­age­ment sys­tem, re­freshed some con­tent and the de­sign, and re­stored his­toric con­tent dat­ing back to 2002. We also strength­ened up­stream col­lab­o­ra­tion with the MLT de­vel­op­ers and con­tributed sev­eral im­prove­ments to OpenTimelineIO.Here’s a look at what we’ve been up to and what is ahead.As part of KDE Apps, we fol­low the KDE Gear re­lease cy­cle, with three ma­jor re­leases each year—in April, August, and December—each fol­lowed by three point main­te­nance re­leases.This re­lease added a pow­er­ful au­to­matic mask­ing tool and brought the last batch of fea­tures from our last fundraiser.The new Object Segmentation plu­gin based on the [SAM2][4] model al­lows to re­move any se­lected ob­ject from the back­ground.We rewrote our OpenTimelineIO im­port and ex­port func­tion us­ing the C++ li­brary. Now you can ex­change pro­jects with other edit­ing ap­pli­ca­tions that sup­port this open source file for­mat.Au­dio wave­form gen­er­a­tion got a 300% per­for­mance boost, along with a refac­tored sam­pling method that ac­cu­rately ren­ders the au­dio sig­nal and higher-res­o­lu­tion wave­forms for greater pre­ci­sion.This re­lease fo­cused heav­ily on sta­bi­liza­tion, bring­ing over 300 com­mits and fix­ing more than 15 crashes. Instead of ma­jor new fea­tures, the ef­fort went into pol­ish­ing and bug fix­ing.We re­designed the au­dio mixer bring­ing lev­els with clearer vi­su­als and thresh­olds. We also did some code refac­tor­ing and cleanup. This change fixes is­sues with HiDPI dis­plays with frac­tional scal­ing.Guides and Markers got a ma­jor over­haul this re­lease to im­prove the pro­ject or­ga­ni­za­tion.This re­lease the ti­tler re­ceived some much needed love like im­proved SVG and im­age sup­port with abil­ity to move and re­size items, added cen­ter re­size with Shift + Drag, and re­named the Pattern tab to Templates and moved the tem­plates drop­down to it­The fo­cus of this re­lease cy­cle was on im­prov­ing the user ex­pe­ri­ence and pol­ish­ing the user in­ter­face.We added a new first-run launch screen for first time users as well as added a Welcome Screen al­low­ing to eas­ily launch re­cent pro­jects.We added a new, more flex­i­ble dock­ing sys­tem that lets you group wid­gets, show or hide them on de­mand, and save lay­outs as sep­a­rate files that can be shared or stored within pro­jects.The au­dio wave­form in the Project Monitor got a re­vamped in­ter­face with an added min­imap.This next re­lease is just around the cor­ner and brings a nice batch of nifty new fea­tures like mon­i­tor mir­ror­ing and an­i­mated tran­si­tion pre­views, mak­ing it much eas­ier to vi­su­al­ize how they will look be­fore ap­ply­ing them. Additionally, drop­ping a tran­si­tion onto the time­line can now au­to­mat­i­cally ad­just its du­ra­tion to match the clips above and be­low, sav­ing time and re­duc­ing man­ual tweak­ing.This fea­ture al­lows you to mir­ror any mon­i­tor while work­ing in fullscreen mode. It’s es­pe­cially use­ful when work­ing with mul­ti­ple dis­plays or col­lab­o­rat­ing with oth­ers in the edit­ing room.Change the play­back speed of mul­ti­ple clips at on­ceIm­port a clip di­rectly from the time­line con­text menu and in­sert it at the click po­si­tionOp­tion to al­ways zoom to­ward the mouse po­si­tion in­stead of the time­line play­head­Our roadmap is con­stantly be­ing re­viewed and up­dated, and some of the up­com­ing high­lights in­clude im­ple­ment­ing the new fea­tures in MLT, the mul­ti­me­dia frame­work which pow­ers Kdenlive. Some ex­cit­ing up­com­ing fea­tures in­clude 10/12 bit color sup­port, play­back op­ti­miza­tions (decoding), and OpenFX sup­port. (Shoutout to a Kdenlive com­mu­nity mem­ber for lead­ing this ef­fort). Also ex­pected is a refac­tor­ing of the sub­ti­tle sys­tem as well as con­tin­u­ing to de­velop the Advanced Trimming Tools.We are cur­rently work­ing on refac­tor­ing the keyfram­ing sys­tem and im­ple­ment­ing a Dopesheet, ba­si­cally it is a ded­i­cated time­line for man­ag­ing and view­ing keyframes from mul­ti­ple ef­fects si­mul­ta­ne­ously. This work will also in­tro­duce per-pa­ra­me­ter keyfram­ing (currently, once you add a keyframe to an ef­fect, it is ap­plied to all pa­ra­me­ters by de­fault). More info can be found in the last sta­tus re­port. This work is made pos­si­ble through an NGI Zero Commons grant via NLnet.We have been work­ing on en­abling and fix­ing mul­ti­ple mod­ules in MLT to com­pile with MSVC al­low­ing us to ship Kdenlive in the Microsoft Store soon. Another ad­van­tage is that it will al­low to run unit tests on our CI for Windows.Currently, the Kdenlive core team is made up of 8 ac­tive mem­bers, in­clud­ing 2 de­vel­op­ers.In 2025, 38 peo­ple con­tributed code to Kdenlive (including the core dev team and other KDE devs), a truly im­pres­sive num­ber! Even more ex­cit­ing, about half of them were first-time con­trib­u­tors, which is al­ways great. We hope to see many of them con­tinue con­tribut­ing in the fu­ture. On be­half of the Kdenlive team, we salute you all!Note that these num­bers re­fer specif­i­cally to con­tri­bu­tions to the Kdenlive ap­pli­ca­tion. Other pro­jects such as the test suite and web­site are hosted in sep­a­rate repos­i­to­ries and are not in­cluded in these fig­ures.In February, part of the Kdenlive core team met in Amsterdam for a short sprint, high­lighted by a visit to the Blender Foundation, where we met with Francesco Siddi and he shared valu­able in­sights into Blender’s his­tory and of­fered ad­vice on prod­uct man­age­ment for Kdenlive. We also at­tended their weekly open ses­sion, where artists and de­vel­op­ers pre­sent progress on on­go­ing pro­jects. During the sprint, we dis­cussed and ad­vanced sev­eral tech­ni­cal top­ics, some high­lights in­clude:Fin­ish­ing an MLT Framework patch to en­able ren­der­ing with­out a dis­play server (needed for Flatpak test­ing)The Berlin sprint was one of our most pro­duc­tive gath­er­ings to date. Most of the team was there in per­son, and we also con­nected on­line with those who could­n’t make it. We dis­cussed just about every as­pect of the pro­ject, from roadmap plan­ning to up­com­ing fea­tures and work­flow im­prove­ments. Some of the high­lights in­clude:Eval­u­ated the cur­rent state of the Titler and dis­cussed pos­si­ble in­te­gra­tion with GlaxnimateDeveloped a proof of con­cept for us­ing KDDockWidgetsRedesigned and started de­vel­op­ment of the au­dio clip view in the Clip MonitorThanks to the nice folks at c-base who kindly hosted us.Akademy is al­ways a great op­por­tu­nity to ex­change ideas with the broader KDE and Qt com­mu­ni­ties. One of the high­lights was meet­ing the main­tainer of Glaxnimate, where we dis­cussed com­mon goals and ways to col­lab­o­rate. This year, Akademy will be in Graz on the 19-24 of September, and we hope to see you there.We’re very happy to see more YouTube chan­nels talk­ing about Kdenlive. Here are some ex­am­ples of what the com­mu­nity has been cre­at­ing.We’d love to see what you’ve been work­ing on in the past year. Share your videos pro­duc­tions in the com­ments!Help us grow the com­mu­nity by or­ga­niz­ing mee­tups, talks, or work­shops in your lo­cal area. Don’t hes­i­tate to con­tact us if you need guid­ance, ma­te­ri­als, or sup­port to get started.Be­low are pho­tos from a work­shop with in­dige­nous com­mu­ni­ties in Paraguay.Kdenlive was down­loaded 11,500,714 times from our down­load page in 2025. Do note that many ad­di­tional in­stalls hap­pen through Linux dis­tri­b­u­tion pack­age man­agers, the Snap Store, Flathub, and other third-party servers, where sta­tis­tics are not al­ways avail­able or re­li­ably mea­sur­able.The Flatpak pack­age from Flathub gets 41,499 down­loads per month.25.04.2 got the most num­ber of down­loads.Files With Most Code ChangesTo the 5 of you in Antarctica, let us know what you are edit­ing. ;)Ever since our last, and very suc­cess­ful, fundraiser in 2022, we haven’t ac­tively asked for do­na­tions, yet the com­mu­nity has con­tin­ued to sup­port us. We are very grate­ful for that.In 2025, we re­ceived a to­tal of €9,344.80 from do­na­tions (down from €11,526.61 in 2024). Around 30% of the amount was given by donors who kindly set up a re­cur­ring plan. The av­er­age do­na­tion was about €25, with the low­est amount be­ing €10 and the high­est €500.We al­lo­cate 20% of our bud­get to KDE e.V. to sup­port in­fra­struc­ture costs (servers and re­lated ex­penses), as well as ad­min­is­tra­tion, le­gal sup­port, and travel. As in pre­vi­ous years, your con­tri­bu­tions en­able us to con­tinue sup­port­ing Jean-Baptiste (Kdenlive’s main­tainer), al­low­ing him to ded­i­cate sev­eral days each month to Kdenlive in ad­di­tion to his vol­un­teer work.WE NEED YOUR SUPPORTKdenlive needs your sup­port to keep grow­ing and im­prov­ing. If just a quar­ter of the peo­ple who down­loaded Kdenlive in 2025 con­tributed €5, our main­tain­ers would be able to ded­i­cate more time to the pro­ject, and it would even al­low us to hire more de­velpers to speed up de­vel­op­ment and im­prove sta­bil­ity. Small amounts can make a big dif­fer­ence, please con­sider mak­ing a do­na­tion.You may also con­tribute by get­ting in­volved and help­ing in:

In a pre­vi­ous post about AI-discovered bugs in Vim and Emacs, we looked at how seem­ingly harm­less work­flows could cross a sur­pris­ing line into code ex­e­cu­tion. This time we wanted to push that idea even fur­ther: is cat readme.txt safe?

It turns out that it is NOT, if you use iTerm2.

That looks in­sane un­til you un­der­stand what iTerm2 is try­ing to do for a le­git­i­mate fea­ture, how it uses the PTY, and what hap­pens when ter­mi­nal out­put is able to im­per­son­ate one side of that fea­ture’s pro­to­col.

We’d like to ac­knowl­edge OpenAI for part­ner­ing with us on this pro­ject.

iTerm2 has an SSH in­te­gra­tion fea­ture that gives it a richer un­der­stand­ing of re­mote ses­sions. To make that work, it does not just “blindly type com­mands” into a re­mote shell. Instead, it boot­straps a tiny helper script on the re­mote side called the con­duc­tor.

iTerm2 sends a re­mote boot­strap script, the con­duc­tor, over the ex­ist­ing SSH ses­sion. That re­mote script be­comes the pro­to­col peer for iTerm2.iTerm2 and the re­mote con­duc­tor ex­change ter­mi­nal es­cape se­quences to co­or­di­nate things like:

The im­por­tant point is that there is no sep­a­rate net­work ser­vice. The con­duc­tor is just a script run­ning in­side the re­mote shell ses­sion, and the pro­to­col is car­ried over nor­mal ter­mi­nal I/O.

A ter­mi­nal used to be a real hard­ware de­vice: a key­board and screen con­nected to a ma­chine, with pro­grams read­ing in­put from that de­vice and writ­ing out­put back to it.

A ter­mi­nal em­u­la­tor like iTerm2 is the mod­ern soft­ware ver­sion of that hard­ware ter­mi­nal. It draws the screen, ac­cepts key­board in­put, and in­ter­prets ter­mi­nal con­trol se­quences.

But the shell and other com­mand-line pro­grams still ex­pect to talk to some­thing that looks like a real ter­mi­nal de­vice. That is why the OS pro­vides a PTY, or pseudoter­mi­nal. A PTY is the soft­ware stand-in for the old hard­ware ter­mi­nal, and it sits be­tween the ter­mi­nal em­u­la­tor and the fore­ground process.

* ssh for­wards those bytes to the re­mote ma­chine

* the re­mote con­duc­tor reads them from its stdin

So when iTerm2 wants to “send a com­mand to the re­mote con­duc­tor,” what it ac­tu­ally does lo­cally is write bytes to the PTY.

The SSH in­te­gra­tion pro­to­col uses ter­mi­nal es­cape se­quences as its trans­port.

* DCS 2000p is used to hook the SSH con­duc­tor

* OSC 135 is used for pre-framer con­duc­tor mes­sages

At source level, DCS 2000p causes iTerm2 to in­stan­ti­ate a con­duc­tor parser. Then the parser ac­cepts OSC 135 mes­sages like:

So a le­git­i­mate re­mote con­duc­tor can talk back to iTerm2 en­tirely through ter­mi­nal out­put.

The bug is a trust fail­ure. iTerm2 ac­cepts the SSH con­duc­tor pro­to­col from ter­mi­nal out­put that is not ac­tu­ally com­ing from a trusted, real con­duc­tor ses­sion. In other words, un­trusted ter­mi­nal out­put can im­per­son­ate the re­mote con­duc­tor.

That means a ma­li­cious file, server re­sponse, ban­ner, or MOTD can print:

and iTerm2 will start act­ing like it is in the mid­dle of a real SSH in­te­gra­tion ex­change. That is the ex­ploit prim­i­tive.

iTerm2 ren­ders the file, but the file is not just text. It con­tains:

Once the hook is ac­cepted, iTerm2 starts its nor­mal con­duc­tor work­flow. In up­stream source, Conductor.start() im­me­di­ately sends get­shell(), and af­ter that suc­ceeds it sends python­ver­sion().

So the ex­ploit does not need to in­ject those re­quests. iTerm2 is­sues them it­self, and the ma­li­cious out­put only has to im­per­son­ate the replies.

The fake OSC 135 mes­sages are min­i­mal but pre­cise.

They do this:

Return lines that look like shell-dis­cov­ery out­put

This is enough to push iTerm2 down its nor­mal fall­back path. At that point, iTerm2 be­lieves it has com­pleted enough of the SSH in­te­gra­tion work­flow to move on to the next step: build­ing and send­ing a run(…) com­mand.

The forged DCS 2000p hook con­tains sev­eral fields, in­clud­ing at­tacker-con­trolled sshargs.

That value mat­ters be­cause iTerm2 later uses it as com­mand ma­te­r­ial when it con­structs the con­duc­tor’s run … re­quest.

The ex­ploit chooses sshargs so that when iTerm2 base64-en­codes:

the last 128-byte chunk be­comes:

That string is not ar­bi­trary. It is cho­sen be­cause it is both:

In a le­git­i­mate SSH in­te­gra­tion ses­sion, iTerm2 writes base64-en­coded con­duc­tor com­mands to the PTY, and ssh for­wards them to the re­mote con­duc­tor. In the ex­ploit case, iTerm2 still writes those com­mands to the PTY, but there is no real SSH con­duc­tor. The lo­cal shell re­ceives them as plain in­put in­stead.

That is why the ses­sion looks like this when recorded:

* the last chunk is ace/​c+al­i­FIo

Earlier chunks fail as non­sense com­mands. The fi­nal chunk works if that path ex­ists lo­cally and is ex­e­cutable.

You can re­pro­duce the orig­i­nal file-based PoC with gen­poc.py:

* readme.txt, a file con­tain­ing the ma­li­cious DCS 2000p and OSC 135 se­quences

The first fools iTerm2 into talk­ing to a fake con­duc­tor. The sec­ond gives the shell some­thing real to ex­e­cute when the fi­nal chunk ar­rives.

For the ex­ploit to work, run cat readme.txt from the di­rec­tory con­tain­ing ace/​c+al­i­FIo, so the fi­nal at­tacker-shaped chunk re­solves to a real ex­e­cutable path.

* Mar 30: We re­ported the bug to iTerm2.

* Mar 31: The bug was fixed in com­mit a9e745993c2e2cb­b30b884a16617cd5495899f86.

* At the time of writ­ing, the fix has not yet reached sta­ble re­leases.

When the patch com­mit landed, we tried to re­build the ex­ploit from scratch us­ing the patch alone. The prompts used for that process are in prompts.md, and the re­sult­ing ex­ploit is gen­poc2.py, which works very sim­i­larly to gen­poc.py.

When they re­ceived the call to re­spond to an Israeli airstrike in the city of Mayfadoun, in south­ern Lebanon, most of the para­medics held back, hav­ing pre­vi­ously seen col­leagues killed by dou­ble-tap at­tacks tar­get­ing res­cuers. But the medics from the Islamic Health Association (IHA) rushed to the scene.

By the time the other emer­gency work­ers ar­rived at the site, they found the IHA medics had in­deed been caught in a sec­ond strike. They started evac­u­at­ing their wounded col­leagues, only for their am­bu­lances to be hit in two fur­ther at­tacks.

One of the para­medics cov­ered his ears and screamed, con­vuls­ing in pain as shrap­nel shat­tered the back win­dow of the am­bu­lance.

The res­cue mis­sion on Wednesday af­ter­noon had turned into a night­mare as Israel car­ried out three con­sec­u­tive strikes on three sets of am­bu­lances and med­ical work­ers.

In to­tal, the at­tacks killed four medics and wounded six more, from three dif­fer­ent am­bu­lance corps, ac­cord­ing to med­ical sources. Three of the medics were from the Hezbollah-affiliated IHA and Amal-affiliated med­ical corps, while one was from the Nabatieh emer­gency ser­vices or­gan­i­sa­tion. Under in­ter­na­tional law, all medics are pro­tected and are con­sid­ered non-com­bat­ants, re­gard­less of po­lit­i­cal af­fil­i­a­tion.

Rescuers in Lebanon have long been wary of the dou­ble-tap at­tack, when Israeli forces tar­get a lo­ca­tion, wait un­til peo­ple gather to help sur­vivors, and then strike again. Wednesday’s three-wave at­tack af­ter the ini­tial one prompted the coin­ing of a fear­some new term: the quadru­ple tap.

In a video taken by one of the para­medics at the site, res­cuers are seen load­ing two wounded peo­ple into their am­bu­lances when a bomb lands next to their ve­hi­cle. Paramedics rush to ex­tract the dri­ver, who is mo­tion­less and limp as they pull him from the am­bu­lance, which is splashed with blood. “Oh God, oh God,” the man film­ing can be heard say­ing. They carry two more blood-cov­ered medics out of their ve­hi­cle and on to stretch­ers.

Among the para­medics killed was Fadel Sarhan, 43, who is sur­vived by his eight-year-old daugh­ter.

“Fadel was a very loved per­son. He had a bold per­son­al­ity, but at the same time, he was emo­tional. He was well liked and re­spon­si­ble,” said Ali Nasr al-Deen, the head of the Mayfadoun civil de­fence cen­tre who grew up with Sarhan.

“He used to feed the cats and dogs. He would bring pet food from Beirut so they would­n’t go hun­gry. He was that kind of per­son, car­ing and at­ten­tive. It’s a huge loss for us,” said Nasr al-Deen.

Medics mourned their col­leagues on Thursday at fu­ner­als in Nabatieh, a city near Mayfadoun. Such events have be­come in­creas­ingly com­mon, with health­care work­ers killed by Israeli bomb­ings on a near daily ba­sis.

Mohammed Suleiman, whose 16-year-old son, Joud, was killed while on duty as a para­medic by an Israeli strike weeks ear­lier, joined his peers in bury­ing an­other of his friends on Thursday. A few hours af­ter the fu­ner­als, Israel car­ried out an­other wave of airstrikes on Nabatieh.

Israel has so far killed 91 health­care work­ers and wounded 214 more in Lebanon since the Israel-Hezbollah war started on 2 March. It has given lit­tle jus­ti­fi­ca­tion for its re­peated at­tacks on med­ical in­fra­struc­ture and work­ers, apart from ac­cus­ing Hezbollah of us­ing am­bu­lances and hos­pi­tals to trans­port fight­ers and weapons, with­out pro­vid­ing ev­i­dence for the claim.

The Lebanese min­istry of health ac­cused Israel of de­lib­er­ately tar­get­ing am­bu­lance crews. “Paramedics have be­come di­rect tar­gets, pur­sued re­lent­lessly in a bla­tant vi­o­la­tion that con­firms a to­tal dis­re­gard for all norms and prin­ci­ples es­tab­lished by in­ter­na­tional hu­man­i­tar­ian law,” the min­istry said in a state­ment.

The Israeli mil­i­tary did not im­me­di­ately re­spond to a re­quest for com­ment.

In the video taken of the quadru­ple tap on Wednesday, the frame was frozen on the in­te­rior of the am­bu­lances, as the Nabatieh emer­gency ser­vices high­lighted that the ve­hi­cle clearly con­tained no weapons.

A few hours af­ter Israel hit the am­bu­lances out­side Nabatieh, it bombed the vicin­ity of the gov­ern­men­tal hos­pi­tal in Tebnine, south Lebanon. It was the sec­ond time in two days that Israeli bomb­ings dam­aged the health­care fa­cil­ity, which is the only re­main­ing pub­lic hos­pi­tal in the area. The strikes in­jured 11 hos­pi­tal work­ers and dam­ag­ing the emer­gency de­part­ment, ac­cord­ing to the World Health Organization (WHO).

A video of Tebnine hos­pi­tal from 14 April showed work­ers try­ing to clear shat­tered con­crete and de­bris from the emer­gency de­part­ment af­ter a strike blew in the win­dows.

Commenting on the strike in Tebnine, the head of the WHO, Tedros Adhanom Ghebreyesus, said: “I re­it­er­ate the call for the im­me­di­ate pro­tec­tion of health­care fa­cil­i­ties, health work­ers, am­bu­lances and pa­tients. There must be safe, sus­tained and un­hin­dered hu­man­i­tar­ian ac­cess across Lebanon.”

An am­bu­lance in Tebnine was also struck on Thursday, lead­ing to the crit­i­cal in­jury of two medics, ac­cord­ing to the Lebanese min­istry of health. As health­care work­ers watched their col­leagues and friends be­ing killed by Israel, the men­tal toll was be­com­ing al­most too much to bear.

“We have to go to places to res­cue peo­ple, but then we get dou­ble tapped,” said Abbas Atwi, the head of the IHA’s emer­gency de­part­ment in Nabatieh, shortly af­ter a med­ical cen­tre was tar­geted in March, killing his friends and col­leagues. “But we will stay and keep go­ing, we will not leave.”

This is a cal­cu­la­tor that works over unions of in­ter­vals rather than just real num­bers. It is an im­ple­men­ta­tion of Interval

Union Arithmetic.

An in­ter­val [a, b] rep­re­sents the set of all num­bers be­tween and in­clud­ing a and b. An in­ter­val union: [a, b] U [c, d] is a dis­joint set of in­ter­vals.

Interval union arith­metic is an ex­ten­sion of reg­u­lar in­ter­val arith­metic that is vastly su­pe­rior, mostly be­cause it re­mains closed while sup­port­ing di­vi­sion by in­ter­vals con­tain­ing zero:

➤ 2 / [-2, 1]

[-∞, -1] U [2, +∞]

The in­ter­est­ing thing about in­ter­val union arith­metic is the in­clu­sion prop­erty, which means that if you pick any real num­ber from every in­put union and com­pute the same ex­pres­sion over the re­als, the re­sult is guar­an­teed to be in the out­put union.

You can use it to rep­re­sent un­cer­tainty:

➤ 50 * (10 + [-1, 1])

[450, 550]

You can also com­pute more com­plex in­ter­val ex­pres­sions, us­ing the in­ter­val union op­er­a­tor U:

➤ ( [5, 10] U [15, 16] ) / [10, 100]

[0.05, 1.6]

Operations can re­sult in dis­joint unions of in­ter­vals:

➤ 1 / [-2, 1]

[-∞, -0.5] U [1, +∞]

➤ tan([pi/​3, 2*pi/3])

[-∞, -1.732] U [1.732, +∞]

In full pre­ci­sion mode, you can use it as a reg­u­lar cal­cu­la­tor, and ob­tain in­ter­val re­sults that are guar­an­teed to con­tain the true value, de­spite float­ing point pre­ci­sion is­sues:

➤ 0.1 + 0.2

[0.29999999999999993, 0.3000000000000001]

Note: you can in­put in­ter­vals with the bracket syn­tax: [1, 2], or bare num­bers with­out brack­ets: 3.14. Bare num­bers are in­tepreted as a nar­row in­ter­val, i.e. [3.14, 3.14] (with sub­tleties re­lated to full pre­ci­sion mode). This en­ables bare num­bers and in­ter­vals to be mixed nat­u­rally:

➤ 1.55 + [-0.002, 0.002]

[1.548, 1.552]

A sur­pris­ing con­se­quence of the cal­cu­la­tor gram­mar is that in­ter­vals can be nested and you can write things like:

➤ [0, [0, 100]]

[0, 100]

This is be­cause all num­bers, in­clud­ing those in­side an in­ter­val bracket which de­fine a bound, are in­ter­preted as in­ter­vals. When nest­ing two in­ter­vals as above, an in­ter­val used as an in­ter­val bound is the same as tak­ing its up­per bound. This de­sign choice en­ables us­ing arith­metic on in­ter­val bounds them­selves:

➤ [0, cos(2*pi)]

[0, 1]

Outward round­ing is im­ple­mented over IEEE 754 dou­ble pre­ci­sion floats (javascript’s num­ber type), so re­sult in­ter­vals are guar­an­teed to con­tain the true value that would be ob­tained by com­put­ing the same ex­pres­sion over the re­als with in­fi­nite pre­ci­sion. For ex­am­ple, try the fa­mous sum 0.1 + 0.2 in the cal­cu­la­tor. Interval arith­metic com­putes an in­ter­val that is guar­an­teed to con­tain 0.3, even though 0.3 is not rep­re­sentable as a dou­ble pre­ci­sion float.

* Numbers in­put by the user are in­ter­preted as the small­est

in­ter­val that con­tains the IEEE 754 value clos­est to the in­put

dec­i­mal rep­re­sen­ta­tion but where nei­ther bounds are equal to

it

* Output num­bers are dis­played with all avail­able dec­i­mal

dig­its (using Number.toString())

* Numbers in­put by the user are in­ter­preted as the

de­gen­er­ate in­ter­val (width zero) where both bounds are equal

to the IEEE 754 value clos­est to the in­put dec­i­mal

rep­re­sen­ta­tion

* Output num­bers are dis­played with a max­i­mum of 4 dec­i­mal

dig­its (using Number.toPrecision())

While I’ve been very care­ful, I’m sure there are still some bugs in the cal­cu­la­tor. Please re­port

any is­sue on GitHub.

Interval

Calculator and not-so-float

(the en­gine pow­er­ing the cal­cu­la­tor) are open-source. If you you like my open-source work, please con­sider spon­sor­ing me

on GitHub. Thank you ❤️

* Split full pre­ci­sion mode into two con­trols: in­put

in­ter­pre­ta­tion and dis­play pre­ci­sion

Japan is the land of the train. 28 per­cent of pas­sen­ger kilo­me­ters in Japan are trav­elled by rail, more than any­where else in the de­vel­oped world. France achieves 10 per­cent, Germany 6.4 per­cent, and the United States just 0.25 per­cent. Travel in Japan is over a hun­dred times more likely to be by rail than travel in the United States.

Japan’s vast rail­way net­work is di­vided be­tween dozens of com­pa­nies, nearly all of them pri­vate. The largest of these, JR East, car­ries more pas­sen­gers than the en­tire rail­way sys­tem of every coun­try other than China and India. Each year, JR East car­ries four times as many pas­sen­gers as the whole British rail­way sys­tem, even though it has fewer kilo­me­ters of track, serves about ten mil­lion fewer peo­ple, and com­petes with eight other com­pa­nies. Japan’s rail­way sys­tem turns a large op­er­at­ing profit and re­ceives far less pub­lic sub­sidy than European and American rail­ways.

Subscribe for $100 to re­ceive six beau­ti­ful is­sues per year.

In most de­vel­oped coun­tries, the rail­ways have strug­gled since the rise of the au­to­mo­bile in the 1950s. From this point on, North America saw the near-to­tal re­place­ment of pas­sen­ger trains with cars and planes. In Europe, it meant vast gov­ern­ment fi­nan­cial sup­port to keep the lines open.

Japan’s dif­fer­ent tra­jec­tory is of­ten at­trib­uted to cul­ture: the Japanese are con­formists who are con­tent to take pub­lic trans­port, un­like free­dom-lov­ing Americans who pre­fer to drive every­where. Europeans are some­where in be­tween. Culture is also used to ex­plain the in­cred­i­ble punc­tu­al­ity of Japanese rail­ways.

These cul­tural ex­pla­na­tions are wrong. The Japanese love cars, but they take trains be­cause they have the best rail­way sys­tem in the world. And their sys­tem ex­cels be­cause of good pub­lic pol­icy: busi­ness struc­ture, land use rules, dri­ving rules, su­pe­rior mod­els for pri­va­ti­za­tion, and sound reg­u­la­tion have given Japan its out­stand­ing rail­ways.

This is good news for friends of rail. Culture is built over cen­turies, and repli­cat­ing it is hard. But suc­cess­ful pub­lic poli­cies can be em­u­lated by one good gov­ern­ment. Much about Japan’s rail­way sys­tem could be replic­a­ble around the world.

Today, the most strik­ing in­sti­tu­tional fea­ture of Japanese rail is that it is pri­vately owned by a throng of com­pet­ing com­pa­nies.

The rail­way ar­rived in Japan in 1872, dur­ing the Meiji Restoration, which opened the coun­try up to for­eign trade, ideas, and tech­nolo­gies. Like most Western coun­tries, Japan na­tion­al­ized its rail­ways in the early twen­ti­eth cen­tury, cre­at­ing what be­came known as Japanese National Railways (JNR). But it did not na­tion­al­ize all of the lines, fo­cus­ing only on main­line rail­ways of na­tional im­por­tance, and new pri­vate rail­ways were still per­mit­ted.

Between 1907 and World War II, Japan saw a boom in new pri­vate elec­tric rail­ways, co­in­cid­ing with rapid ur­ban­iza­tion. Technologically, most of these pri­vate rail­ways were sim­i­lar to the fa­mous in­terur­bans in the United States: they were ba­si­cally elec­tric trams, but run­ning be­tween cities as well as within them. The American net­work even­tu­ally with­ered, and al­most noth­ing of it sur­vives to­day. In Japan, how­ever, the net­work con­sol­i­dated, and the light tram­lines grad­u­ally evolved into heavy-rail in­ter­city con­nec­tions.

These com­pa­nies are to­day known as ‘legacy pri­vate rail­ways’ on ac­count of their hav­ing been pri­vate since their in­cep­tion. There are eight legacy pri­vate rail­ways in the Tokyo met­ro­pol­i­tan area, five in the Osaka–Kobe–Kyoto mega­lopo­lis, two in Nagoya, and one in the fourth city of Fukuoka. There are also dozens of smaller ones else­where. In the three largest ur­ban ar­eas, these op­er­a­tors ac­count for nearly half of rail­way track and sta­tions, as well as a plu­ral­ity of rid­er­ship. The largest, Kintetsu, not only op­er­ates ur­ban ser­vices, but a whole in­ter­city net­work stretch­ing from Osaka to Nagoya.

These com­pa­nies of­ten com­pete head-to-head. At its most ex­treme, three sep­a­rate com­muter lines com­pete for the traf­fic be­tween Osaka and the port city of Kobe, run­ning in par­al­lel, some­times fewer than 500 meters apart.

Meanwhile, the na­tion­al­ized rail­ways were man­aged by JNR. In the post­war era, JNR was re­spon­si­ble for build­ing the fa­mous Shinkansen sys­tem, as well as run­ning com­muter and long-dis­tance lines through­out Japan. But in 1988, it was largely pri­va­tized, bro­ken into six re­gional mo­nop­o­lies for pas­sen­ger ser­vices to­gether with a sin­gle na­tional freight op­er­a­tor. These are col­lec­tively known as the Japan Railways Group (JR).

This means that Japan has ended up with six rail­way com­pa­nies that trace their de­scent to the na­tion­al­ized rail­ways, the six­teen big legacy com­pa­nies that have al­ways been pri­vate, and a host of mi­nor legacy rail­ways, as well as nu­mer­ous un­der­ground met­ros (some pri­vate, some mu­nic­i­pally owned), mono­rails, and tram sys­tems. This in­sti­tu­tional di­ver­sity is strik­ing enough. But equally strik­ing is the con­sis­tent busi­ness model that has evolved amidst this plu­ral­ism: the rail­way that builds a city.

If I take a train to go for a soli­tary walk in the coun­try­side, the rail­way com­pany can cap­ture some of the value it cre­ates by charg­ing me for the jour­ney, just as other com­pa­nies cap­ture the value of the goods and ser­vices they pro­vide by charg­ing for them. However, if I take a train to visit fam­ily, clients, a the­ater, or a shop, an im­por­tant dif­fer­ence ap­pears. The rail­way can cap­ture the value it cre­ates for me by charg­ing me a fare, but it can­not cap­ture the value it cre­ates for those at my des­ti­na­tion. As trans­port in­fra­struc­ture cre­ates ben­e­fits that pro­duce no rev­enue for providers, free mar­kets rarely build enough of it.

Japan has partly solved this prob­lem by en­abling rail­way com­pa­nies to do a great deal be­side run­ning rail­ways. Take the ex­am­ple of the Tokyu cor­po­ra­tion, one of the legacy pri­vate rail­ways in south­ern Tokyo. You can not only travel on its trains, but also ride a Tokyu bus, live in a Tokyu-built house, work in a Tokyu of­fice com­plex, see a doc­tor in a Tokyu hos­pi­tal, buy gro­ceries in a Tokyu su­per­mar­ket, spend an af­ter­noon at a Tokyu mu­seum-the­ater-cin­ema com­plex, take your chil­dren to their amuse­ment park, and even die in a Tokyu re­tire­ment home. The pos­i­tive spillover ef­fects of the rail­way on these things are cap­tured by Tokyu be­cause it owns them. The pres­i­dent of Tokyu has said:

I think that though we are a rail­way com­pany, we con­sider our­selves a city-shap­ing com­pany. In Europe for in­stance, rail­way com­pa­nies sim­ply con­nect cities through their ter­mi­nals. That is a pretty nor­mal way of op­er­at­ing in this in­dus­try, whereas what we do is com­pletely dif­fer­ent: we cre­ate cities and then, as a util­ity fa­cil­ity, we add the sta­tions and the rail­ways to con­nect them one with an­other.

This model was pi­o­neered in the 1950s by what be­came Hankyu Railways. Hankyu’s net­work con­nects cen­tral Osaka to its north­ern sub­urbs, as well as Kyoto and Kobe. Its in­no­v­a­tive founder Kobayashi Ichizo first built sub­ur­ban hous­ing, then a de­part­ment store at the ter­mi­nal sta­tion; he then cre­ated a hot spring re­sort, a zoo, and his own dis­tinc­tive brand of all-women mu­si­cal the­ater, the Takarazuka Revue. He also be­gan to run bus ser­vices to and from his sta­tions. Other com­pa­nies em­u­lated Hankyu’s ex­am­ple: Tokyo Disneyland is a col­lab­o­ra­tion be­tween Disney and the Keisei Railway, while Hanshin in Osaka owns the Hanshin Tigers base­ball team.

Core rail op­er­a­tions are prof­itable for every Japanese pri­vate rail­way com­pany, but they usu­ally only ac­count for a plu­ral­ity or a small ma­jor­ity of rev­enue. The rest is con­tributed by their port­fo­lio of side busi­nesses. There is a nat­ural fi­nan­cial syn­ergy be­tween the re­li­able but un­re­mark­able cash flow of train fares and the prof­itable but riskier real es­tate and com­mer­cial side of the busi­ness. Railway com­pa­nies’ side busi­nesses also at­tract peo­ple to live and work on their rail cor­ri­dor, re­in­forc­ing the cus­tomer base for the rail­way ser­vices them­selves.

This vir­tu­ous cir­cle is en­abled by tran­sit-ori­ented de­vel­op­ment. Japan’s lib­eral land use reg­u­la­tion makes it straight­for­ward to build new neigh­bor­hoods next to rail­way lines, giv­ing com­muters easy ac­cess to city cen­ters. It also en­ables the den­si­fi­ca­tion of these cen­ters, which means that com­muters have more places they want to go.

Railways cost a lot to build, but once they are built, they can move enor­mous num­bers of peo­ple, far more than a road of sim­i­lar size. This means that they work best in cities with a high den­sity of peo­ple, jobs, and other ac­tiv­i­ties. In 2019, New York City was the only American city where rail had a higher modal share than cars, in part be­cause Manhattan has 2.5 mil­lion jobs, two mil­lion res­i­dents, and 50 mil­lion tourist vis­its crammed into 59 square kilo­me­ters.

This does not mean that rail-ori­ented cities must be struc­tured like Chinese cities: is­lands of high-rise apart­ments con­nected by met­ros and sep­a­rated by mo­tor­ways. Japanese cities have the low­est res­i­den­tial den­sity in Asia, and a plu­ral­ity of the Japanese live in houses, usu­ally de­tached ones. The ur­ban area of Tokyo, the dens­est Japanese city, has a weighted pop­u­la­tion den­sity less than that of many European cities, in­clud­ing Paris, Madrid, or Athens. Japanese cities have vast low-rise, pre­dom­i­nantly res­i­den­tial sub­urbs, built at den­si­ties that might be higher than what is typ­i­cal in the United States, but that would be quite nor­mal in Northern Europe.

What makes Japan’s cities par­tic­u­larly suited to rail is thus not their res­i­den­tial dis­tricts, but their huge and hy­per­dense cen­ters. These re­ally are spe­cial: the cores of Tokyo or Osaka are un­like any­thing that ex­ists in Europe or North America. Many of their fea­tures are fa­mous world­wide: the ver­ti­cal street za­kkyo build­ings, un­der­ground streets, shop­ping streets un­der rail tracks, cov­ered ar­cades, el­e­vated sta­tion squares, and ver­ti­cal cities. Getting mil­lions of com­muters and shop­pers into these down­towns is where rail ex­cels be­cause its ex­treme spa­tial ef­fi­ciency means that in­fra­struc­ture with a rel­a­tively mod­est foot­print can trans­port vast num­bers of peo­ple into a small area.

None of this emerged from a co­her­ent mas­ter­plan of tran­sit-ori­ented de­vel­op­ment like Copenhagen’s Finger Plan or Curitiba’s Trinary System. Postwar Japanese opin­ion was com­mit­ted to de­cen­tral­iza­tion both to rural pe­riph­eries and to the sub­urbs through green­belts, mo­tor­ways, and new towns.

Instead, this va­ri­ety and adapt­abil­ity around rail­ways is pos­si­ble be­cause of the way Japanese ur­ban plan­ning works. Since 1919, Japan has had a stan­dard­ized na­tional zon­ing sys­tem, but it is much more lib­eral than de­vel­op­ment con­trol sys­tems in Western coun­tries. The Japanese au­thor­i­ties did not in­tend or even de­sire dense ur­ban cen­ters, but they did not pre­vent them, rather like nine­teenth-cen­tury gov­ern­ments in the West.

This lib­eral zon­ing sys­tem is re­in­forced by pri­vate ac­cess to city plan­ning pow­ers. Thirty per­cent of Japan’s ur­ban land has been sub­ject to land read­just­ment, where agree­ment among two thirds of res­i­dents and landown­ers in an area is enough to al­low its re­plan­ning, in­clud­ing com­pul­so­rily tak­ing and de­mol­ish­ing land for ameni­ties and in­fra­struc­ture. Initially land read­just­ment was used only to as­sem­ble rural land for ur­ban­iza­tion, but over time it was in­creas­ingly used to re­de­velop al­ready ur­ban­ized ar­eas, and new vari­ants were cre­ated to build the sky­scrap­ers that sur­round the ma­jor sta­tions of cen­tral Tokyo.

The his­tory of the pri­vate rail­way com­pa­nies could be writ­ten as a story of land read­just­ment pro­jects: the ini­tial build­ing of the lines in the in­ter­war years pro­ceeded through one land read­just­ment pro­ject af­ter an­other. Postwar im­prove­ments such as dou­ble-track­ing, plat­form length­en­ing, and con­stant re­de­vel­op­ment of sta­tions and their im­me­di­ate thresh­olds were only pos­si­ble be­cause the rail­ways could se­cure land tak­ings co­op­er­a­tively with lo­cal busi­nesses and landown­ers.

Perhaps the great­est ex­am­ple of this phe­nom­e­non in­volved Tokyu. In 1953 the com­pany de­cided to build the Den’en Toshi Line, or Garden City Line, to serve a rural area south­west of Tokyo. This would be en­abled by a se­ries of land read­just­ment pro­jects col­lec­tively among the largest in Japanese his­tory.

Over 30 years, 3,100 hectares were cov­ered, of which only 36 per­cent was de­voted to res­i­den­tial and com­mer­cial de­vel­op­ment, with 20 per­cent for for­est and parks, 17 per­cent for roads, and much of the rest for wa­ter­courses. The pop­u­la­tion of the land read­just­ment zone would rise from 42,000 in 1954 to over 500,000 in 2003.

By con­nect­ing the af­flu­ent south­west­ern sub­urbs to Tokyu’s main real es­tate hub next to Shibuya sta­tion, now the sec­ond busiest in the world, the Den’en Toshi Line al­lowed Tokyu to be­come the largest pri­vate rail­way by rev­enue and rid­er­ship. The Japanese gov­ern­ment and aca­d­e­mics gen­er­ally con­sider the Den’en Toshi Line to be the best cor­ri­dor of tran­sit–ori­ented de­vel­op­ment in Japan.

But the rail­way-as-city-builder model is not the only rea­son Japanese rail­ways have been able to thrive. European coun­tries usu­ally pro­hib­ited rail­ways from run­ning real es­tate side busi­nesses, but in the United States and Canada the prac­tice was ex­tremely wide­spread in the nine­teenth and early twen­ti­eth cen­turies, and many fa­mous rail­way sub­urbs were de­vel­oped this way. Despite this, pas­sen­ger rail in these coun­tries col­lapsed in the mid-twen­ti­eth cen­tury. Part of the dif­fer­ence was that Japan did not ex­tend the same im­plicit sub­si­dies to cars as Western gov­ern­ments did.

The land of Toyota, Nissan, and Honda is not an anti-car nir­vana. In fact, Japan has ex­cel­lent mo­tor­ways, and across the coun­try as a whole a small ma­jor­ity of jour­neys are made by car. But Japan is a place where cars and car-ori­ented lifestyles com­pete on a level play­ing field.

Japan is one of the only coun­tries to have pri­va­tized park­ing. In Europe and North America, vast quan­ti­ties of park­ing space is so­cial­ized: mu­nic­i­pal­i­ties own the streets and al­low peo­ple to park on them at low or zero cost. Initially with the in­ten­tion of en­cour­ag­ing the pro­vi­sion of more park­ing spaces, Japan made it il­le­gal to park on pub­lic roads or pave­ments with­out spe­cial per­mis­sion. Before some­one buys a car, they must prove that they have a re­served night-time space on pri­vate land, ei­ther owned or leased.

Since park­ing on pub­lic land is banned, mu­nic­i­pal­i­ties are not wor­ried about over­spill park­ing from de­vel­op­ments with in­ad­e­quate pri­vate park­ing. They there­fore have no rea­son to im­pose park­ing min­i­mums on de­vel­op­ments: the mar­ket is left to de­cide whether park­ing is the most valu­able use of pri­vate land. Where land is abun­dant, as in rural ar­eas, sub­urbs, or small towns, pri­vate park­ing is plen­ti­ful. But in city cen­ters, it is out­com­peted by other land uses. According to the late Donald Shoup, cen­tral Tokyo has 23 park­ing spaces per hectare and 0.04 park­ing spaces per job, com­pared with 263 and 0.52 for Los Angeles. Even Manhattan, the dens­est ur­ban area in North America with the low­est lev­els of car own­er­ship, has about 60 park­ing spaces per hectare.

Japanese roads are ex­pected to be self-fi­nanc­ing. Motorways are run by self-con­tained pub­lic co­op­er­a­tives, very sim­i­lar to the statu­tory au­thor­i­ties that ran English roads and canals be­tween 1660 and the late 1800s, and funded by tolls on their users. Vehicle reg­is­tra­tion taxes, which are al­lo­cated to lo­cal­i­ties for road con­struc­tion and main­te­nance, are worth three per­cent of the Japanese gov­ern­ment bud­get.

These mea­sures, adopted in the 1950s, were not in­tended to sup­press car use — the point was to fund a mas­sive road ex­pan­sion — but they have forced pri­vate ve­hi­cles to in­ter­nal­ize many of their hid­den costs. In the Tokyo ur­ban area, the av­er­age house­hold spends 71,000 yen ($450) each year on pub­lic trans­port fares and 210,000 yen ($1,350) on car pur­chase and main­te­nance costs.

But the pri­vate car was not the only com­peti­tor faced by the pri­vate rail­ways. For eight decades in the twen­ti­eth cen­tury, they also had to face the jug­ger­naut of Japanese National Railways. Its pri­va­ti­za­tion in 1988 re­moved the fi­nal ob­sta­cle to cre­at­ing the world’s best rail­way sys­tem.

Railway pri­va­ti­za­tion in Britain, New Zealand, Argentina, and Sweden has had a mixed re­cep­tion, and all of those coun­tries, apart from Sweden, have taken steps to re­verse it. In Japan, it has been so suc­cess­ful that the gov­ern­ment sub­se­quently pri­va­tized the metro sys­tems in Tokyo and Osaka.

In the post­war pe­riod, JNR en­joyed real suc­cesses. It built the rev­o­lu­tion­ary Shinkansen, the first high-speed rail­way in the world. It also ag­gres­sively elec­tri­fied and dou­ble-tracked ma­jor trunk lines, quadru­ple-tracked lines into and out of ma­jor cities, and added city-cen­ter loops and freight by­passes. But these achieve­ments were over­shad­owed by two prob­lems.

The first was pol­i­tics. Many coun­tries adapted to the rise of the car by clos­ing the least prof­itable parts of their pas­sen­ger rail net­work, like the con­sol­i­da­tion of American freight rail into the Class I op­er­a­tors or the Beeching Axe in Britain. In Japan, how­ever, the rul­ing Liberal Democratic Party drew its sup­port from rural con­stituen­cies, whose sup­port it re­tained with pork–bar­rel pol­i­tics. Its ‘rail tribe’ group, led by rural MPs, pre­vented JNR from adapt­ing it­self to mass mo­tor­iza­tion.

JNR there­fore did not am­pu­tate gan­grenous rural and freight ser­vices that im­posed heavy costs with few ben­e­fits. Worse, it con­tin­ued to build new loss-mak­ing rural rail­way lines, known in Japanese as Gaden-intetsu, or rail­ways pulled into the rice field.

The sec­ond prob­lem was or­ga­nized la­bor. In gen­eral, Japanese trade unions are known for their mod­er­a­tion and re­spon­si­bil­ity, a gen­er­al­i­sa­tion that also held true for the unions at the legacy pri­vate rail­ways. The JNR unions, how­ever, be­came highly mil­i­tant, se­cure in the knowl­edge that their na­tion­al­ized em­ploy­ers could never go bank­rupt. Their largest se­ries of strikes in 1973 pro­voked ri­ots from com­muters.

The rail­way unions im­posed over­staffing on rev­enue-gen­er­at­ing ur­ban ser­vices, at a time when both in­ter­na­tional and pri­vate do­mes­tic op­er­a­tors were re­duc­ing staffing re­quire­ments against a back­drop of higher wages and the grow­ing au­toma­tion of sig­nal­ing and tick­et­ing. As a re­sult, 78 per­cent of JNR’s costs were re­lated to la­bor, com­pared to 40 per­cent for other Japanese rail­ways. The av­er­age worker at a pri­vate rail­way was 121 per­cent more pro­duc­tive than their JNR coun­ter­part.

By the early 1980s, only seven out of 200 JNR lines made a profit. Successive gov­ern­ments de­ferred se­ri­ous re­form, run­ning up debt, cut­ting down in­vest­ments in new ur­ban lines, rais­ing ticket prices to twice those of com­pa­ra­ble pri­vate rail­ways, and in­creas­ing sub­si­dies — which rose un­til an­nual sub­si­dies equaled the to­tal cost of the Shinkansen.

In 1982, Prime Minister Yasuhiro Nakasone started to pri­va­tize the rail­ways. Unlike other coun­tries, Japan sim­ply re­turned to the tra­di­tional pri­vate rail­way model of the nine­teenth and early twen­ti­eth cen­turies: tracks, trains, sta­tions, and yards were owned by ver­ti­cally in­te­grated re­gional con­glom­er­ates.

There are sub­stan­tial ad­van­tages to ver­ti­cal in­te­gra­tion. Railways are a closed sys­tem that has to be planned as a sin­gle unit. Changing the timetable at sta­tion A can af­fect the timetable at sta­tion Z; buy­ing new trains that can travel faster might re­quire changes to the in­fra­struc­ture so they can reach their top speed, which in turn re­quires rewrit­ing the timeta­bles. This be­comes es­pe­cially com­pli­cated if dif­fer­ent ser­vices share tracks. To pre­vent de­lays from prop­a­gat­ing from one ser­vice to an­other, the timetable needs to be care­fully de­signed to make best use of the avail­able in­fra­struc­ture.

The stark­est ef­fect of pri­va­ti­za­tion was a mas­sive and im­me­di­ate in­crease in la­bor pro­duc­tiv­ity and prof­itabil­ity rel­a­tive to the legacy pri­vate rail­ways. In fact, this be­gan be­fore pri­va­ti­za­tion: its mere threat strength­ened the gov­ern­men­t’s hand when bar­gain­ing with the unions and forced JNR to be­gin clos­ing rural lines.

Privatization saw a gen­eral trend of pro­duc­tiv­ity im­prove­ments, fol­low­ing a big one-time im­prove­ment be­tween 1982 and 1990, when the work­force was cut by more than half, 83 loss-mak­ing lines were re­moved, and JNR’s debts were trans­ferred to a hold­ing com­pany.

The sec­ond great ad­van­tage of pri­va­ti­za­tion was to al­low the JR com­pa­nies to em­u­late the rail­way-as-city-builder model of the legacy pri­vate rail­ways: for in­stance, JR East owns two shop­ping cen­ter brands, a ski re­sort, a cof­fee chain, and even a vend­ing ma­chine drink com­pany. The JR com­pa­nies have not ig­nored their rail busi­ness: they have con­tin­ued to build new high-speed lines and ur­ban tun­nels, up­grade sta­tions, and im­ple­ment a host of other im­prove­ments such as the in­tro­duc­tion in the 1990s of smart cards that al­low pas­sen­gers to pay their fare with a tap.

This does not mean that the Japanese rail­way in­dus­try is a pure crea­ture of free en­ter­prise. No rail­way sys­tem ever has been. The Japanese sys­tem has found an equi­lib­rium that makes rail pol­icy ex­plicit and lim­ited. Leaving aside rail­way safety and busi­ness reg­u­la­tion, there are two main pol­icy levers: fare max­i­mums and cap­i­tal ex­pan­sion sub­si­dies.

Price con­trols are of­ten cited as a clas­sic ex­am­ple of mis­guided gov­ern­ment in­ter­ven­tion, whether through rent con­trols, caps on the price of gaso­line, wage freezes, or min­i­mum agri­cul­tural prices. Tokyo’s in­fa­mously crammed trains are a symp­tom of un­der­priced rush hour traf­fic.

Railways have mar­ket power be­cause the sub­sti­tutes for rail­way trips – coaches, cars and planes – are quite a dif­fer­ent prod­uct. This mo­nop­o­lis­tic po­si­tion has his­tor­i­cally meant trou­ble: mo­nop­oly sys­tems, whether pri­vate or pub­lic, have a ten­dency to abuse their po­si­tion to charge higher prices and run bad ser­vices. For this rea­son, the pri­vate mo­nop­o­lies that were com­mon in the Western world be­fore World War I of­ten had price con­trols im­posed on them. For ex­am­ple, most of the American street­car net­works were op­er­ated as long-term, price-con­trolled fran­chises granted by the city.

Price max­i­mums, if set too low, could have ru­ined Japan’s rail­ways. This is ex­actly what hap­pened to many Western tran­sit ser­vices af­ter the First World War. But the post­war Japanese prac­tice has capped fares gen­er­ously. The sys­tem is ex­plic­itly de­signed to main­tain prof­itabil­ity per rider, which in turn in­cen­tivizes the com­pa­nies to max­i­mize rid­er­ship. That buys po­lit­i­cal le­git­i­macy for the pri­va­tized sys­tem, which is nec­es­sary for the con­tin­ued pro­vi­sion of cap­i­tal ex­pan­sion sub­si­dies. Indeed, dur­ing the long de­fla­tion era be­tween 1992 and 2022, it was com­mon for op­er­a­tors to charge be­low the max­i­mum, and the real value of rail­way fares con­tin­ued to rise. Fare max­i­mums are set on the ba­sis of the av­er­age cost struc­tures of all rail­way op­er­a­tors in a re­gion, so com­pa­nies with be­low-av­er­age costs like Tokyu would of­ten charge be­low the cap to main­tain a com­pet­i­tive edge, pre­vent pub­lic back­lash, and max­i­mize traf­fic to their side-busi­nesses.

Other than the fare max­i­mums, the rail­ways are free to make their own de­ci­sions about timeta­bles, ser­vice pat­terns and day-to-day op­er­a­tions, a highly spe­cial­ized and tech­ni­cal task which re­quires deep ex­per­tise. This con­trasts with the gov­ern­ment med­dling with, say, Amtrak’s routes.

Carefully de­signed pub­lic sub­si­dies also play a use­ful role. Although Japanese rail­ways do not re­ceive sub­si­dies for day-to-day op­er­a­tions, they do re­ceive gov­ern­ment loans and grants for cap­i­tal in­vest­ments. These are typ­i­cally tied to pub­lic pri­or­i­ties, such as dis­abil­ity ac­cess or earth­quake-proof­ing, or to pro­jects that have large spillovers that the rail­way com­pany would be un­able to in­ter­nal­ize, like re­mov­ing level cross­ings, or el­e­vat­ing at-grade rail­ways or trams in or­der to re­duce road con­ges­tion and ac­ci­dent risk. Generally, the lo­cal pre­fec­tural gov­ern­ment will match the con­tri­bu­tion of the na­tional gov­ern­ment. Larger new build pro­jects are sub­ject to lease back or debt-pay­ment con­di­tions that fare rev­enue is ex­pected to pay back.

Railway com­pa­nies in­vested heav­ily in real es­tate busi­nesses, of­ten fund­ing lines through sell­ing land for hous­ing around new sta­tions. Liberal spa­tial pol­icy meant that such de­vel­op­ment hap­pened eas­ily, even as it en­abled dense de­vel­op­ment in ur­ban cores where ra­dial rail lines con­verged. Rail com­pa­nies were gen­er­ally ver­ti­cally in­te­grated re­gional mo­nop­o­lies, own­ing the land, track, and rolling stock, set­ting their own timeta­bles, and em­ploy­ing their staff. The state im­posed con­trols to stop them ex­ploit­ing their mo­nop­oly po­si­tion, but it did so cau­tiously, al­low­ing them to make suf­fi­cient profit that in­cen­tives to in­vest were pre­served. Capital sub­si­dies were tar­geted at pro­vid­ing spe­cific pub­lic goods that nor­mal com­mer­cial op­er­a­tions over­looked.

The above para­graph could be writ­ten by a his­to­rian of the fu­ture about con­tem­po­rary Japan. But every word in it could also be writ­ten by a his­to­rian to­day about the United States in the nine­teenth cen­tury — usu­ally seen as the epit­ome of cap­i­tal­ist in­di­vid­u­al­ism. This strik­ing fact con­tra­dicts the idea that America’s sup­posed in­di­vid­u­al­ism fore­or­dains it to be the land of the car, or that Japan’s sup­posed com­mu­ni­tar­i­an­ism fore­or­dained it to be the land of rail.

It also puts pres­sure on the idea that the demise of rail is the in­evitable con­se­quence of cars. All coun­tries saw some shift to cars in the twen­ti­eth cen­tury, and all rail in­dus­tries had to re­spond to that. But pub­lic pol­icy had an enor­mous ef­fect on how suc­cess­fully they did so. The rise of zon­ing re­stric­tions on den­sity, ex­ces­sive price con­trols, na­tion­al­iza­tion, and ver­ti­cally dis­in­te­grated pri­va­ti­za­tion have ham­pered Western rail in re­main­ing com­pet­i­tive against cars since the 1920s. By main­tain­ing and restor­ing the in­sti­tu­tions that built the first rail­way sys­tems in the nine­teenth cen­tury, the Japanese have cre­ated the might­i­est rail­way sys­tem of the twenty-first.

Given a set of ob­jects, there can be nu­mer­ous cri­te­ria, based on which to or­der them (depending on the ob­jects them­selves) — size, weight, age, al­pha­bet­i­cal or­der etc.

However, cur­rently we are not in­ter­ested in the cri­te­ria that we can use to or­der ob­jects, but in the na­ture of the re­la­tion­ships that de­fine or­der. Of which there can be sev­eral types as well.

Mathematically, the or­der as a con­struct is rep­re­sented (much like a monoid) by two com­po­nents.

An or­der is a set of el­e­ments, to­gether with a bi­nary re­la­tion be­tween the el­e­ments of the set, which obeys cer­tain laws.

We de­note the el­e­ments of our set, as usual, like this.

And the bi­nary re­la­tion is a re­la­tion be­tween two el­e­ments, which is of­ten de­noted with an ar­row.

As for the laws, they are dif­fer­ent de­pend­ing on the type of or­der.

Let’s start with an ex­am­ple — the most straight­for­ward type of or­der that you think of is lin­ear or­der i.e. one in which every ob­ject has its place de­pend­ing on every other ob­ject. In this case the or­der­ing cri­te­ria is com­pletely de­ter­min­is­tic and leaves no room for am­bi­gu­ity in terms of which el­e­ment comes be­fore which. For ex­am­ple, or­der of col­ors, sorted by the length of their light-waves (or by how they ap­pear in the rain­bow).

Using set the­ory, we can rep­re­sent this or­der, as well as any other or­der, as a sets of pairs of the or­der’s un­der­ly­ing set with it­self (a sub­set of the prod­uct set).

And in pro­gram­ming, or­ders are de­fined by pro­vid­ing a func­tion which, given two ob­jects, tells us which one of them is “bigger” (comes first) and which one is “smaller”. It is­n’t hard to see that this func­tion de­fines a set of pairs (we are given a pair and we have to say whether or not it be­longs to the set).

However (this is where it gets in­ter­est­ing) not all such func­tions (and not all sets of pairs) de­fine or­ders. For such func­tion to re­ally de­fine an or­der i.e. to have the same out­put every time, in­de­pen­dent of how the ob­jects were shuf­fled ini­tially, it has to obey sev­eral rules.

Incidentally, (or rather not in­ci­den­tally at all), these rules are nearly equiv­a­lent to the math­e­mat­i­cal laws that de­fine the cri­te­ria of the or­der re­la­tion­ship i.e. those are the rules that de­fine which el­e­ment can point to which.

A lin­ear or­der is a set of el­e­ments, to­gether with a bi­nary re­la­tion be­tween the el­e­ments of the set, which obeys the laws of re­flex­iv­ity, tran­si­tiv­ity, an­ti­sym­me­try, to­tal­ity.

Let’s check what they are.

Let’s get the bor­ing law out of the way — each ob­ject has to be big­ger or equal to it­self, or $a ≤ a$ for all $a$ (the re­la­tion­ship be­tween el­e­ments in an or­der is com­monly de­noted as $≤$ in for­mu­las, but it can also be rep­re­sented with an ar­row from first ob­ject to the sec­ond.)

This law only ex­ist to cover the “base case”: we can for­mu­late it the op­po­site way too and say that each ob­ject should not have the re­la­tion­ship to it­self, in which case we would have a re­la­tion than re­sem­bles big­ger than, as op­posed to big­ger or equal to and a slightly dif­fer­ent type of or­der, some­times called a strict or­der.

The sec­ond law is maybe the least ob­vi­ous, (but prob­a­bly the most es­sen­tial) — it states that if ob­ject $a$ is big­ger than ob­ject $b$, it is au­to­mat­i­cally big­ger than all ob­jects that are smaller than $b$ or $a ≤ b \land b ≤ c \to a ≤ c$.

This is the law that to a large ex­tend de­fines what an or­der is: if I am bet­ter at play­ing soc­cer than my grand­mother, then I would also be bet­ter at it than my grand­moth­er’s friend, whom she beats, oth­er­wise I would­n’t re­ally be bet­ter than her.

The third law is called an­ti­sym­me­try. It states that the func­tion that de­fines the or­der should not give con­tra­dic­tory re­sults (or in other words you have $x ≤ y$ and $y ≤ x$ only if $x = y$).

It also means that no ties are per­mit­ted — ei­ther I am bet­ter than my grand­mother at soc­cer or she is bet­ter at it than me.

The last law is called to­tal­ity (or con­nex­ity) and it man­dates that all el­e­ments that be­long to the or­der should be com­pa­ra­ble ($a ≤ b \lor b ≤ a$). That is, for any two el­e­ments, one would al­ways be “bigger” than the other.

By the way, the law of to­tal­ity makes the re­flex­iv­ity law re­dun­dant, as re­flex­iv­ity is just a spe­cial case of to­tal­ity when $a$ and $b$ are one and the same ob­ject, but I still want to pre­sent it for rea­sons that will be­come ap­par­ent soon.

Actually, here are the rea­sons: the law of to­tal­ity can be re­moved. Orders, that don’t fol­low the to­tal­ity law are called par­tial or­ders, (and lin­ear or­ders are also called to­tal or­ders.)

Task 1: Previously, we cov­ered a re­la­tion that is pretty sim­i­lar to this. Do you re­mem­ber it? What is the dif­fer­ence?

Task 2: Think about some or­ders that you know about and fig­ure out whether they are par­tial or to­tal.

Partial or­ders are ac­tu­ally much more in­ter­est­ing than lin­ear/​to­tal or­ders. But be­fore we dive into them, let’s say a few things about num­bers.

Natural num­bers form a lin­ear or­der un­der the op­er­a­tion big­ger or equal to (the sym­bol of which we have been us­ing in our for­mu­las.)

In many ways, nat­ural num­bers are the quin­tes­sen­tial or­der — every fi­nite or­der of ob­jects is iso­mor­phic to a sub­set of the or­der of num­bers, as we can map the first el­e­ment of any or­der to the num­ber $1$, the sec­ond one to the num­ber $2$ etc (and we can do the op­po­site op­er­a­tion as well).

If we think about it, this iso­mor­phism is ac­tu­ally closer to the every­day no­tion of a lin­ear or­der, than the one de­fined by the laws — when most peo­ple think of or­der, they aren’t think­ing of a tran­si­tive, an­ti­sym­met­ric and to­tal re­la­tion, but are rather think­ing about cri­te­ria based on which they can de­cide which ob­ject comes first, which comes sec­ond etc. So it’s im­por­tant to no­tice that the two no­tions are equiv­a­lent.

From the fact that any fi­nite or­der of ob­jects is iso­mor­phic to the nat­ural num­bers, it also fol­lows that all lin­ear or­ders of the same mag­ni­tude are iso­mor­phic to one an­other.

So, the lin­ear or­der is sim­ple, but it is also (and I think that this iso­mor­phism proves it) the most bor­ing or­der ever, es­pe­cially when looked from a cat­e­gory-the­o­retic view­point — all fi­nite lin­ear or­ders (and most in­fi­nite ones) are just iso­mor­phic to the nat­ural num­bers and so all of their di­a­grams look the same way.

However, this is not the case with par­tial or­ders that we will look into next.

Law of to­tal­ity does not look so “set in stone” as the rest of the laws i.e. we can prob­a­bly think of some sit­u­a­tions in which it does not ap­ply. For ex­am­ple, if we aim to or­der all peo­ple based on soc­cer skills there are many ways in which we can rank a per­son com­pared to their friends their friend’s friends etc. but there is­n’t a way to or­der groups of peo­ple who never played with one an­other.

Remove the law of to­tal­ity from the laws of lin­ear or­ders and we get a par­tial or­der (also a par­tially-or­dered set, or poset).

An par­tial or­der is a set of el­e­ments, to­gether with a bi­nary re­la­tion be­tween the el­e­ments of the set, which obeys the laws of re­flex­iv­ity, tran­si­tiv­ity and an­ti­sym­me­try.

Every lin­ear or­der is also a par­tial or­der (just as a group is still a monoid), but not the other way around.

We can even cre­ate an or­der of or­ders, based on which is more gen­eral.

Partial or­ders are also re­lated to the con­cept of an equiv­a­lence re­la­tions that we cov­ered in chap­ter 1, ex­cept that sym­me­try law is re­placed with an­ti­sym­me­try.

If we re­visit the ex­am­ple of the soc­cer play­ers rank list, we can see that the first ver­sion that in­cludes just my­self, my grand­mother and her friend is a lin­ear or­der.

However, in­clud­ing this other per­son whom none of us played yet, makes the hi­er­ar­chy non-lin­ear i.e. a par­tial or­der.

This is the main dif­fer­ence be­tween par­tial and to­tal or­ders — par­tial or­ders can­not pro­vide us with a def­i­nite an­swer of the ques­tion who is bet­ter than who. But some­times this is what we need — in sports, as well as in other do­mains, there is­n’t al­ways an ap­pro­pri­ate way to rate el­e­ments lin­early.

Before, we said that all lin­ear or­ders can be rep­re­sented by the same chain-like di­a­gram, we can re­verse this state­ment and say that all di­a­grams that look some­thing dif­fer­ent than the said di­a­gram rep­re­sent par­tial or­ders.

An ex­am­ple of this is a par­tial or­der that con­tains a bunch of lin­early-or­dered sub­sets, e.g. in our soc­cer ex­am­ple we can have sep­a­rate groups of friends who play to­gether and are ranked with each other, but not with any­one from other groups.

The dif­fer­ent lin­ear or­ders that make up the par­tial or­der are called chains. There are two chains in this di­a­gram $m \to g \to f$ and $d \to o$.

The chains in an or­der don’t have to be com­pletely dis­con­nected from each other in or­der for it to be par­tial. They can be con­nected as long as the con­nec­tions are not all one-to-one i.e. ones when the last el­e­ment from one chain is con­nected to the first el­e­ment of the other one (this would ef­fec­tively unite them into one chain.)

The above set is not lin­early-or­dered — al­though we know that $d ≤ g$ and that $f ≤ g$, the re­la­tion­ship be­tween $d$ and $f$ is not known — any el­e­ment can be big­ger than the other one.

Although par­tial or­ders don’t give us a de­fin­i­tive an­swer to “Who is bet­ter than who?”, some of them still can give us an an­swer to the more im­por­tant ques­tion (in sports, as well as in other do­mains), namely “Who is num­ber one?” i.e. who is the cham­pion, the player who is bet­ter than any­one else. Or, more gen­er­ally, the el­e­ment that is big­ger than all other el­e­ments.

The great­est el­e­ment of an or­der is an el­e­ment $a$, such that we have we have $x ≤ a$ for any other el­e­ment $x$, Some (not all) par­tial or­ders do have such el­e­ment — in our last di­a­gram $m$ is the great­est el­e­ment, in this di­a­gram, the green el­e­ment is the biggest one.

Sometimes we have more than one el­e­ments that are big­ger than all other el­e­ments, in this case none of them is the great­est.

In ad­di­tion to the great­est el­e­ment, a par­tial or­der may also have a least (smallest) el­e­ment, which is de­fined in the same way.

The least up­per bound of two el­e­ments that are con­nected as part of an or­der is called the join of these el­e­ments, e.g. the green el­e­ment is a join of the other two.

The join of $a$ and $b$ is the small­est el­e­ment $c$ that is big­ger than then, for­mally:

The join of ob­jects $A$ and $B$ is an ob­ject $G$, such that:

It is big­ger than both of these ob­jects, so $A ≤ G$ and $B ≤ G$.

It is smaller than any other ob­ject that is big­ger than them, so for any other ob­ject $P$ such that $P ≤ A$ and $P ≤ B$ then we should also have $G ≤ P$.

Given any two el­e­ments in which one is big­ger than the other (e.g. $a ≤ b$), the join is this big­ger el­e­ment (in this case $b$)

e.g. in a lin­ear or­ders, the join of any two el­e­ments is just the big­ger el­e­ment.

Like with the great­est el­e­ment, if two el­e­ments have sev­eral up­per bounds that are equally big, then none of them is a join (a join must be unique).

If, how­ever, one of those el­e­ments is es­tab­lished as smaller than the rest of them, it im­me­di­ately qual­i­fies.

Task 3: Which con­cept in cat­e­gory the­ory re­minds you of joins?

Given two el­e­ments, the biggest el­e­ment that is smaller than both of them is called the meet of these el­e­ments.

The same rules as for the joins ap­ply, but in re­verse.

The di­a­grams that we use in this sec­tion are called “Hasse di­a­grams” and they work much like our usual di­a­grams, how­ever they have an ad­di­tional rule that is fol­lowed — “bigger” el­e­ments are al­ways po­si­tioned above smaller ones.

In terms of ar­rows, the rule means that if you add an ar­row to a point, the point to which the ar­row points must al­ways be above the one from which it points.

This arrange­ment al­lows us to com­pare any two points by just see­ing which one is above the other e.g. we can de­ter­mine the join of two el­e­ments, by just iden­ti­fy­ing the el­e­ments that they con­nect to and see which one is low­est.

We all know many ex­am­ples of to­tal or­ders (any form of chart or rank­ing is a to­tal or­der), but there are prob­a­bly not so many ob­vi­ous ex­am­ples of par­tial or­ders that we can think of off the top of our head. So let’s see some. This will gives us some con­text, and will help us un­der­stand what joins are.

To stay true to our form, let’s re­visit our color-mix­ing monoid and cre­ate a color-mix­ing par­tial or­der in which all col­ors point to col­ors that con­tain them.

If you go through it, you will no­tice that the join of any two col­ors is the color that they make up when mixed. Nice, right?

The par­tial or­der of num­bers by di­vi­sion

We saw that when we or­der num­bers by “bigger or equal to”, they form a lin­ear or­der. But num­bers can also form a par­tial or­der, for ex­am­ple they form a par­tial or­der if we or­der them by which di­vides which, i.e. if $a$ di­vides $b$, then $a$ is be­fore $b$ e.g. be­cause $2 \times 5 = 10$, $2$ and $5$ come be­fore $10$ (but $3$, for ex­am­ple, does not come be­fore $10$.)

And it so hap­pens (actually for very good rea­son) that the join op­er­a­tion again cor­re­sponds to an op­er­a­tion that is rel­e­vant in the con­text of the ob­jects — the join of two num­bers in this par­tial or­der is their least com­mon mul­ti­ple.

And the meet (the op­po­site of join) of two num­bers is their great­est com­mon di­vi­sor.

Given a col­lec­tion of sets con­tain­ing a com­bi­na­tion of a given set of el­e­ments…

…we can de­fine what is called the in­clu­sion or­der of those sets.

The in­clu­sion or­der of sets is a bi­nary re­la­tion that we can use to or­der a col­lec­tion of sets (usually sets that con­tain some com­mon el­e­ments) in which $a$ comes be­fore $b$ if $a$ in­cludes $b$, or in other words if $b$ is a sub­set of $a$.

In this case the join op­er­a­tion of two sets is their union, and the meet op­er­a­tion is their set in­ter­sec­tion.

This di­a­gram might re­mind you of some­thing — if we take the col­ors that are con­tained in each set and mix them into one color, we get the color-blend­ing par­tial or­der that we saw ear­lier.

The or­der ex­am­ple with the num­ber di­viders is also iso­mor­phic to an in­clu­sion or­der, namely the in­clu­sion or­der of all pos­si­ble sets of prime num­bers, in­clud­ing re­peat­ing ones (or al­ter­na­tively the set of all prime pow­ers). This is con­firmed by the fun­da­men­tal the­ory of arith­metic, which states that every num­ber can be writ­ten as a prod­uct of primes in ex­actly one way.

So far, we saw two dif­fer­ent par­tial or­ders, one based on color mix­ing, and one based on num­ber di­vi­sion, that can be rep­re­sented by the in­clu­sion or­ders of all pos­si­ble com­bi­na­tions of sets of some ba­sic el­e­ments (the pri­mary col­ors in the first case, and the prime num­bers (or prime pow­ers) in the sec­ond one.) Many other par­tial or­ders can be de­fined in this way. Which ones ex­actly, is a ques­tion that is an­swered by an amaz­ing re­sult called Birkhoff’s rep­re­sen­ta­tion the­o­rem. They are the fi­nite par­tial or­ders that meet the fol­low­ing two cri­te­ria:

All el­e­ments have joins and meets.

Those meet and join op­er­a­tions dis­trib­ute over one an­other, that is if we de­note joins as meets as $∨$ or $∧$, then $x ∨ (y ∧ z) = (x ∨ y) ∧ (x ∨ z)$.

The par­tial or­ders that meet the first cri­te­ria are called lat­tices. The ones that meet the sec­ond one are called dis­trib­u­tive lat­tices. Let’s write that down:

Partial or­ders in which all el­e­ments have joins and meets is called a lat­tice. A lat­tice whose meet and join op­er­a­tions dis­trib­ute over one an­other is called a dis­trib­u­tive lat­tice.

And the “prime” el­e­ments which we use to con­struct the in­clu­sion or­der are the el­e­ments that are not the join of any other el­e­ments. They are also called join-ir­re­ducible el­e­ments.

So we may phrase the the­o­rem like this:

Each dis­trib­u­tive lat­tice is iso­mor­phic to an in­clu­sion or­der of its join-ir­re­ducible el­e­ments.

By the way, the par­tial or­ders that are not dis­trib­u­tive lat­tices are also iso­mor­phic to in­clu­sion or­ders, it is just that they are iso­mor­phic to in­clu­sion or­ders that do not con­tain all pos­si­ble com­bi­na­tions of el­e­ments.

We will now talk more about lat­tices (the or­ders for which Birkhoff’s the­o­rem ap­plies). Lattices are par­tial or­ders, in which every two el­e­ments have a join and a meet. So every lat­tice is also par­tial or­der, but not every par­tial or­der is a lat­tice (we will see even more mem­bers of this hi­er­ar­chy).

Most par­tial or­ders that are cre­ated based on some sort of rule are dis­trib­u­tive lat­tices, like for ex­am­ple the par­tial or­ders from the pre­vi­ous sec­tion are also dis­trib­u­tive lat­tices when they are drawn in full, for ex­am­ple the color-mix­ing or­der.

Notice that we added the black ball at the top and the white one at the bot­tom. We did that be­cause oth­er­wise the top three el­e­ments would­n’t have a join el­e­ment, and the bot­tom three would­n’t have a meet.

Our color-mix­ing lat­tice, has a great­est el­e­ment (the black ball) and a least el­e­ment (the white one). Lattices that have a least and great­est el­e­ments are called bounded lat­tices. It is­n’t hard to see that all fi­nite lat­tices are also bounded.

Task 4: Prove that all fi­nite lat­tices are bounded.

We men­tioned or­der iso­mor­phisms sev­eral times al­ready so this is about time to elab­o­rate on what they are.

Given two sets (we will use par­tial or­der of num­bers by di­vi­sion and the prime in­clu­sion or­der as an ex­am­ple) an iso­mor­phism be­tween them is com­prised of the fol­low­ing two func­tions:

One func­tion from the prime in­clu­sion or­der, to the num­ber or­der (which in this case is just the mul­ti­pli­ca­tion of all the el­e­ments in the set)

One func­tion from the num­ber or­der to the prime in­clu­sion or­der (which is an op­er­a­tion called prime fac­tor­iza­tion of a num­ber, con­sist­ing of find­ing the set of prime num­bers that re­sult in that num­ber when mul­ti­plied with one an­other).

An or­der iso­mor­phism is es­sen­tially an iso­mor­phism be­tween the or­ders’ un­der­ly­ing sets (invertible func­tion). However, be­sides their un­der­ly­ing sets, or­ders also have the ar­rows that con­nect them, so there is one more con­di­tion: in or­der for an in­vert­ible func­tion to con­sti­tute an or­der iso­mor­phism, it has to re­spect those ar­rows.

An iso­mor­phism be­tween two or­ders is an in­vert­ible func­tion be­tween their un­der­ly­ing sets, such that ap­ply­ing this func­tion (let’s call it $F$) to any two el­e­ments that have a cer­tain or­der in one set (let’s call them $a$ and $b$) should re­sult in two el­e­ments that have a cor­re­spond­ing or­der in the other set (i.e. $a ≤ b$ if and only if $F(a) ≤ F(b)$).

In the pre­vi­ous sec­tion, we saw how re­mov­ing the law of to­tal­ity from the laws of (linear) or­der pro­duces a dif­fer­ent (and some­what more in­ter­est­ing) struc­ture, called par­tial or­der. Now let’s see what will hap­pen if we re­move an­other one of the laws, namely the an­ti­sym­me­try law.

The an­ti­sym­me­try law man­dated that you can­not have an ob­ject that is at the same time smaller and big­ger than an­other one. (or that $a ≤ b ⟺ b ≰ a$).