10 interesting stories served every morning and every evening.




1 591 shares, 71 trendiness

From $1,432 to $233/month With Zero Downtime

A real-world pro­duc­tion mi­gra­tion from DigitalOcean to Hetzner ded­i­cated, han­dling 248 GB of MySQL data across 30 data­bases, 34 Nginx sites, GitLab EE, Neo4j, and live mo­bile app traf­fic — with zero down­time.

Running a soft­ware com­pany in Turkey has be­come in­creas­ingly ex­pen­sive over the last few years. Skyrocketing in­fla­tion and a dra­mat­i­cally weak­en­ing Turkish Lira against the US dol­lar have turned dol­lar-de­nom­i­nated in­fra­struc­ture costs into a se­ri­ous bur­den. A bill that felt man­age­able two years ago now hits very dif­fer­ently when the ex­change rate has mul­ti­plied sev­eral times over.

Every month, we were pay­ing $1,432 to DigitalOcean for a droplet with 192GB RAM, 32 vC­PUs, 600GB SSD, two block vol­umes (1TB each), and back­ups en­abled. The server was fine — but the price-to-per­for­mance ra­tio had stopped mak­ing sense.

Then we dis­cov­ered the Hetzner AX162-R.

That’s $14,388 saved per year — for a server that’s ob­jec­tively more pow­er­ful in every di­men­sion. The de­ci­sion was easy.

I’ve been a DigitalOcean cus­tomer for nearly 8 years. They have a great prod­uct and I have no com­plaints about re­li­a­bil­ity or de­vel­oper ex­pe­ri­ence. But look­ing at those num­bers now, I can­not help feel­ing a bit sad about all the ex­tra money I left on the table over the years. If you are run­ning steady-state work­loads and not ac­tively us­ing DOs ecosys­tem fea­tures, do your­self a fa­vor and check ded­i­cated server pric­ing be­fore your next re­newal.

* Several live mo­bile apps serv­ing hun­dreds of thou­sands of users

Old server: CentOS 7 — long past its end-of-life, but still run­ning in pro­duc­tion. New server: AlmaLinux 9.7 — a RHEL 9 com­pat­i­ble dis­tri­b­u­tion and the nat­ural suc­ces­sor to CentOS. This mi­gra­tion was also an op­por­tu­nity to fi­nally es­cape an OS that had­n’t re­ceived se­cu­rity up­dates in years.

The naive ap­proach — change DNS, restart every­thing, hope for the best — was­n’t ac­cept­able. Instead, we de­signed a proper mi­gra­tion path with six phases:

Phase 1 — Full stack in­stal­la­tion on the new server

Nginx (compiled from source with iden­ti­cal flags), PHP (via Remi repo, with the same .ini con­fig files from the old server), MySQL 8.0, Neo4J Graph DB, GitLab EE, Node.js, Supervisor, and Gearman. Every ser­vice had to be con­fig­ured to match the old server’s be­hav­ior be­fore we touched a sin­gle DNS record.

SSL cer­tifi­cates were han­dled by rsync­ing the en­tire /etc/letsencrypt/ di­rec­tory from the old server to the new one. After the mi­gra­tion was com­plete and all traf­fic was flow­ing through the new server, we force-re­newed all cer­tifi­cates in one shot:

Phase 2 — Web files cloned with rsync

The en­tire /var/www/html di­rec­tory (~65 GB, 1.5 mil­lion files) was cloned to the new server us­ing rsync over SSH with the –checksum flag for in­tegrity ver­i­fi­ca­tion. We ran a fi­nal in­cre­men­tal sync right be­fore cu­tover to catch any files changed af­ter the ini­tial clone.

Phase 3 — MySQL mas­ter to slave repli­ca­tion

Rather than tak­ing the data­base of­fline for a dump-and-re­store, we set up live repli­ca­tion. The old server be­came mas­ter, the new server a read-only slave. We used my­dumper for the ini­tial bulk load, then started repli­ca­tion from the ex­act bin­log po­si­tion recorded in the dump meta­data. This kept both data­bases in real-time sync un­til the mo­ment of cu­tover.

Phase 4 — DNS TTL re­duc­tion

We scripted the DigitalOcean DNS API to lower all A and AAAA record TTLs from 3600 to 300 sec­onds — with­out touch­ing MX or TXT records (changing mail record TTLs can cause de­liv­er­abil­ity is­sues). After wait­ing one hour for old TTLs to ex­pire glob­ally, we were ready to cut over in un­der 5 min­utes.

Phase 5 — Old server ng­inx con­verted to re­verse proxy

We wrote a Python script that parsed every server {} block across all 34 Nginx site con­figs, backed up the orig­i­nals, and re­placed them with proxy con­fig­u­ra­tions point­ing to the new server. This meant that dur­ing DNS prop­a­ga­tion, any re­quest still hit­ting the old IP was silently for­warded. No user would see a dis­rup­tion.

Phase 6 — DNS cu­tover and de­com­mis­sion

A sin­gle Python script hit the DigitalOcean API and flipped all A records to the new server IP in sec­onds. The old server re­mained as a cold standby for one week, then was shut down.

The key in­sight: at no point did we have a win­dow where the ser­vice was un­avail­able. Traffic was al­ways be­ing served — ei­ther di­rectly or through the proxy.

This was the most com­plex part of the en­tire op­er­a­tion.

We used my­dumper in­stead of the stan­dard mysql­dump — and it made an enor­mous dif­fer­ence. By lever­ag­ing the new server’s 48 CPU cores for par­al­lel ex­port and im­port, what would have taken days with a tra­di­tional sin­gle-threaded mysql­dump was com­pleted in hours. If you’re mi­grat­ing a large MySQL data­base and you’re not us­ing my­dumper/​my­loader, you’re do­ing it the hard way.

The main dump’s meta­data file recorded the bin­log po­si­tion at the time of the snap­shot:

File: mysql-bin.000004

Position: 21834307

This would be our repli­ca­tion start­ing point.

Once the dump was com­plete, we trans­ferred it to the new server us­ing rsync over SSH. With 248 GB of com­pressed chunks, this was sig­nif­i­cantly faster than any other trans­fer method:

The –compress flag in my­dumper paid off here — com­pressed chunks trans­ferred much faster over the wire.

Being stuck on CentOS 7 meant we were also stuck on MySQL 5.7 — an out­dated ver­sion that had been run­ning in pro­duc­tion for years. Before the mi­gra­tion, we ran mysqlcheck –check-upgrade to ver­ify that our data was com­pat­i­ble with MySQL 8.0. It came back clean, so we in­stalled the lat­est MySQL 8.0 Community on the new server. The per­for­mance im­prove­ment across all our pro­jects was im­me­di­ately no­tice­able — query ex­e­cu­tion times dropped sig­nif­i­cantly thanks to MySQL 8.0’s im­proved op­ti­mizer and InnoDB en­hance­ments.

That said, the ver­sion jump did in­tro­duce one tricky prob­lem.

After im­port, the mysql.user table had the wrong col­umn struc­ture — 45 columns in­stead of the ex­pected 51. This caused mysql.in­fos­chema to be miss­ing, break­ing user au­then­ti­ca­tion.

But this failed the first time with:

ERROR: sys.innodb_buffer_stats_by_schema’ is not VIEW

The sys schema had been im­ported as reg­u­lar ta­bles in­stead of views. Solution:

With both dumps im­ported, we con­fig­ured the new server as a replica of the old one:

Almost im­me­di­ately, repli­ca­tion stopped with er­ror 1062 (Duplicate Key). This hap­pened be­cause our dump was taken in two passes — dur­ing the gap be­tween them, rows were writ­ten to cer­tain ta­bles, and now both the im­ported dump and the bin­log re­play were try­ing to in­sert the same rows.

IDEMPOTENT mode silently skips du­pli­cate key and miss­ing row er­rors. All crit­i­cal data­bases synced with­out a sin­gle er­ror. Within a few min­utes, Seconds_Behind_Master dropped to 0.

Before touch­ing a sin­gle DNS record, we needed to ver­ify that all ser­vices were work­ing cor­rectly on the new server. The trick: we tem­porar­ily edited the /etc/hosts file on our lo­cal ma­chine to point our do­main names to the new server’s IP.

# /etc/hosts (local ma­chine)

NEW_SERVER_IP your­do­main1.com

NEW_SERVER_IP your­do­main2.com

# … and so on for all your do­mains

With this in place, our browsers and Postman would hit the new server while the rest of the world was still go­ing to the old one. We ran through our API end­points, checked ad­min pan­els, and ver­i­fied that every ser­vice was re­spond­ing cor­rectly. Only af­ter this con­fir­ma­tion did we pro­ceed with the cu­tover.

Once mas­ter-slave repli­ca­tion was fully syn­chro­nized, we no­ticed that INSERT state­ments were suc­ceed­ing on the new server when they should­n’t have been — read­_only = 1 was set, but writes were go­ing through.

The rea­son: all PHP ap­pli­ca­tion users had been granted SUPER priv­i­lege. In MySQL, SUPER by­passes read­_only.

We re­voked it from all 24 ap­pli­ca­tion users:

After this, read­_only = 1 cor­rectly blocked all writes from ap­pli­ca­tion users while al­low­ing repli­ca­tion to con­tinue.

All do­mains were man­aged through DigitalOcean DNS (with name­servers pointed from GoDaddy). We scripted the TTL re­duc­tion against the DigitalOcean API, only touch­ing A and AAAA records — not MX or TXT records, since chang­ing mail record TTLs can cause de­liv­er­abil­ity is­sues with Google Workspace.

After wait­ing one hour for old TTLs to ex­pire, we were ready.

Rather than edit­ing 34 con­fig files by hand, we wrote a Python script that parsed every server {} block in every con­fig file, iden­ti­fied the main con­tent blocks, re­placed them with proxy con­figs, and backed up orig­i­nals as .backup files.

The key: prox­y_ss­l_ver­ify off — the new server’s SSL cert is valid for the do­main, not for the IP ad­dress. Disabling ver­i­fi­ca­tion here is fine be­cause we con­trol both ends.

With repli­ca­tion at Seconds_Behind_Master: 0 and the re­verse proxy ready, we ex­e­cuted the cu­tover in or­der:

1. New server: STOP SLAVE;

2. New server: SET GLOBAL read­_only = 0;

3. New server: RESET SLAVE ALL;

4. New server: su­per­vi­sor­ctl start all

5. Old server: ng­inx -t && sys­tem­ctl re­load ng­inx (proxy goes live)

6. Old server: su­per­vi­sor­ctl stop all

7. Mac: python3 do_­cu­tover.py (DNS: all A records to new server IP)

8. Wait: ~5 min­utes for prop­a­ga­tion

9. Old server: com­ment out all crontab en­tries

The DNS cu­tover script hit the DigitalOcean API and changed every A record to the new server IP — in about 10 sec­onds.

After mi­gra­tion, we dis­cov­ered many GitLab pro­ject web­hooks were still point­ing to the old server IP. We wrote a script to scan all pro­jects via the GitLab API and up­date them in bulk.

We went from $1,432/month down to $233/month — sav­ing $14,388 per year. And we ended up with a more pow­er­ful ma­chine:

The en­tire mi­gra­tion took roughly 24 hours. No users were af­fected.

MySQL repli­ca­tion is your best friend for zero-down­time mi­gra­tions. Set it up early, let it catch up, then cut over with con­fi­dence.

Check your MySQL user priv­i­leges be­fore mi­gra­tion. SUPER priv­i­lege by­passes read­_only — if your app users have it, your slave en­vi­ron­ment is­n’t ac­tu­ally read-only.

Script every­thing. DNS up­dates, ng­inx con­fig rewrites, web­hook up­dates — do­ing these by hand across 34+ sites would have taken hours and in­tro­duced er­rors.

my­dumper + my­loader dra­mat­i­cally out­per­forms mysql­dump for large datasets. Parallel dump/​re­store with 32 threads cut what would have been days of work down to hours.

Cloud providers are ex­pen­sive for steady-state work­loads. If you’re not us­ing au­toscal­ing or ephemeral in­fra­struc­ture, a ded­i­cated server of­ten de­liv­ers bet­ter per­for­mance at a frac­tion of the cost.

All Python scripts used in this mi­gra­tion are open-sourced and avail­able on GitHub:

* do_list_­do­main­s_ttl.py — List all DigitalOcean do­mains with their A records, IPs, and TTLs

* do_­to_het­zn­er_bulk_dns_record­s_im­port.py — Migrate all DNS zones from DigitalOcean to Hetzner DNS

* do_­cu­tover_­to_new_ip.py — Flip all A records from old server IP to new server IP

* mysql_­com­pare.py — Compare row counts across all ta­bles on two MySQL servers

* fi­nal_git­lab_web­hook_up­date.py — Update all GitLab pro­ject web­hooks to the new server IP

All scripts sup­port a DRY_RUN = True mode so you can safely pre­view changes be­fore ap­ply­ing them.

...

Read the original on isayeter.com »

2 341 shares, 62 trendiness

Anthropic Token Cost Calculator

Anonymous re­quest-to­ken com­par­isons from the com­mu­nity, show­ing how Opus 4.6 and Opus 4.7 dif­fer on real in­puts

...

Read the original on tokens.billchambers.me »

3 292 shares, 12 trendiness

Are the Costs of AI Agents Also Rising Exponentially? — Toby Ord

Are the Costs of AI Agents Also Rising Exponentially?

There is an ex­tremely im­por­tant ques­tion about the near-fu­ture of AI that al­most no-one is ask­ing. We’ve all seen the graphs from METR show­ing that the length of tasks AI agents can per­form has been grow­ing ex­po­nen­tially over the last 7 years. While GPT-2 could only do soft­ware en­gi­neer­ing tasks that would take some­one a few sec­onds, the lat­est mod­els can (50% of the time) do tasks that would take a hu­man a few hours.

As this trend shows no signs of stop­ping, peo­ple have nat­u­rally taken to ex­trap­o­lat­ing it out, to fore­cast when we might ex­pect AI to be able to do tasks that take an en­gi­neer a full work-day; or week; or year. But we are miss­ing a key piece of in­for­ma­tion — the cost of per­form­ing this work. Over those 7 years AI sys­tems have grown ex­po­nen­tially. The size of the mod­els (parameter count) has grown by 4,000x and the num­ber of times they are run in each task (tokens gen­er­ated) has grown by about 100,000x. AI re­searchers have also found mas­sive ef­fi­cien­cies, but it is em­i­nently plau­si­ble that the cost for the peak per­for­mance mea­sured by METR has been grow­ing — and grow­ing ex­po­nen­tially.This might not be so bad. For ex­am­ple, if the best AI agents are able to com­plete tasks that are 3x longer each year and the costs to do so are also in­creas­ing by 3x each year, then the cost to have an AI agent per­form tasks would re­main the same mul­ti­ple of what it costs a hu­man to do those tasks. Or if the costs have a longer dou­bling time than the time-hori­zons, then the AI-systems would be get­ting cheaper com­pared with hu­mans. But what if the costs are grow­ing more quickly than the time hori­zons? In that case, these cut­ting-edge AI sys­tems would be get­ting less cost-com­pet­i­tive with hu­mans over time. If so, the METR time-hori­zon trend could be mis­lead­ing. It would be show­ing how the state of the art is im­prov­ing, but part of this progress would be due to more and more lav­ish ex­pen­di­ture on com­pute so it would be di­verg­ing from what is eco­nom­i­cal. It would be be­com­ing more like the Formula 1 of AI per­for­mance — show­ing what is pos­si­ble, but not what is prac­ti­cal. So in my view, a key ques­tion we need to ask is:        How is the hourly’ cost of AI agents chang­ing over time?By hourly’ cost I mean the fi­nan­cial cost of us­ing an LLM to com­plete a task right at the mod­el’s 50% time hori­zon di­vided by the length of that time hori­zon. So as with the METR time hori­zons them­selves, the du­ra­tions are mea­sured not by how long it takes the model, but how long it typ­i­cally takes hu­mans to do that task. For ex­am­ple, Claude 4.1 Opus’s 50% time hori­zon is 2 hours: it can suc­ceed in 50% of tasks that take hu­man soft­ware en­gi­neers 2 hours. So we can look at how much it costs for it to per­form such a task and di­vide by 2, to find its hourly rate for this work. I’ve found that very few peo­ple are ask­ing this ques­tion. And when I ask peo­ple what they think is hap­pen­ing to these costs over time, their opin­ions vary wildly. Some as­sume the to­tal cost of a task is stay­ing the same, even as the task length in­creases ex­po­nen­tially. That would im­ply an ex­po­nen­tially de­clin­ing hourly rate. Others as­sume the to­tal cost is also grow­ing ex­po­nen­tially — af­ter all, we’ve seen dra­matic in­creases in the costs to ac­cess cut­ting-edge mod­els. And most peo­ple (myself in­cluded) had lit­tle idea of how much it cur­rently costs for AI agents to do an hour’s soft­ware en­gi­neer­ing work. Are we talk­ing cents? Dollars? Hundreds of dol­lars? An AI agent can’t cost more per hour than a hu­man to com­plete these tasks can it? Can it?A cou­ple of months ago I asked METR if they could share the cost data for their bench­mark­ing. I fig­ured it would be easy — just take the cost of run­ning their bench­mark for each model, plot it against re­lease date and see how it is grow­ing. Or plot the cost of each model vs its time hori­zon and see the re­la­tion­ship.But they help­fully pointed out that it is­n’t so easy at all. Their head­line time-hori­zon num­bers are meant to show the best pos­si­ble per­for­mance that can be at­tained with a model (regardless of cost). So they run their mod­els in­side an agent scaf­fold un­til the per­for­mance has plateaued. Since they re­ally want to make sure it has plateaued, they use a lot of com­pute on this and don’t worry too much about whether they’ve used too much. After all, if you are just try­ing to find the even­tual height of a plateau, there is no prob­lem in go­ing far into the flat part of the graph. But if you are try­ing to find out when the plateau be­gins, there is a prob­lem with this strat­egy. Their to­tal spend for each model is some­times just enough to get onto the plateau and some­times many times more than is needed. So to­tal spend can’t be used as di­rect es­ti­mate of the costs of achiev­ing that per­for­mance. Fortunately, they re­leased a chart that can be used to shed some light on the key ques­tion of how hourly costs of LLM agents are chang­ing over time:

This chart (from METRs page for GPT-5) shows how per­for­mance in­creases with cost. The cost in ques­tion is the cost of us­ing more and more to­kens to com­plete the task (and thus more and more com­pute).The yel­low curve is the best hu­man per­for­mance for each task. It steadily marches on­wards and up­wards, trans­form­ing more wages into longer tasks. Since it is hu­man per­for­mance that is used to de­fine the ver­ti­cal axis for METRs time hori­zon work, it is­n’t sur­pris­ing that this curve is fairly lin­ear — it costs about 8 times as much to get a hu­man soft­ware en­gi­neer to per­form an 8-hour task as a 1-hour task.The other colours are the curves for a se­lec­tion of LLM-based agents. Unlike the hu­mans, they all show di­min­ish­ing re­turns, with the time hori­zon each one can achieve even­tu­ally stalling out and plateau­ing as more and more com­pute is added. The short upticks at the end of some of these curves are an arte­fact of some mod­els not be­ing pre­pared to give an an­swer un­til the last avail­able mo­ment. This sug­gests that the model must have been still mak­ing progress dur­ing the ap­par­ent flat­line be­fore the uptick (just not show­ing it). Indeed, this chart was orig­i­nally dis­played on METRs page for GPT-5 to show that they may have stopped its run be­fore it’s per­for­mance had truly plateaued. These upticks do make analy­sis harder and hope­fully fu­ture ver­sions of this chart will be able to avoid these glitches.So what can this chart tell us about our key ques­tion con­cern­ing the hourly cost of AI agents?To tease out the lessons that lie hid­den in the chart, we’ll need to add a num­ber of an­no­ta­tions. The first step is to add lines of con­stant hourly cost. On a log-log plot like this, every con­stant hourly cost will be a straight line with slope 1. Lower hourly costs will ap­pear as lines that are lo­cated fur­ther to the left.

For each curve I’ve added a line of con­stant hourly cost that just grazes it. That is the cheap­est hourly cost the model achieves. We can call the point where the line touches the curve the sweet spot for that model. Before a mod­el’s sweet spot, its time hori­zon is grow­ing su­per-lin­early in cost — it is get­ting in­creas­ing mar­ginal re­turns. The sweet spot is ex­actly the point at which di­min­ish­ing mar­ginal re­turns set in (which would cor­re­spond to the point of in­flec­tion if this was re­plot­ted on lin­ear axes). It is thus a key point on any mod­el’s per­for­mance curve.We can see that the hu­man soft­ware en­gi­neer is at best \$120 per hour, while the sweet spots for the AI agents range from \$40 per hour for o3, all the way down to 40 cents per hour for Grok 4 and Sonnet 3.5. That’s quite a range of costs. While dif­fer­ences in hori­zon length be­tween these mod­els vary by about a fac­tor of 15 (judged at ei­ther the end-points or at the sweet-spots) their sweet-spot costs vary by a fac­tor of 100.And these are the best hourly rates for these mod­els. On many task lengths (including those near their plateau) they cost 10 to 100 times as much per hour. For in­stance, Grok 4 is at \$0.40 per hour at its sweet spot, but \$13 per hour at the start of its fi­nal plateau. GPT-5 is about \$13 per hour for tasks that take about 45 min­utes, but \$120 per hour for tasks that take 2 hours. And o3 ac­tu­ally costs \$350 per hour (more than the hu­man price) to achieve tasks at its full 1.5 hour task hori­zon. This is a lot of money to pay for an agent that fails at the task you’ve just paid for 50% of the time — es­pe­cially in cases where fail­ure is much worse than not hav­ing tried at all.How­ever, I do want to note that I’m a bit puz­zled by how much higher the costs are here for the rea­son­ing mod­els from OpenAI com­pared to mod­els from Anthropic and xAI. The METR page sug­gests that the price data for those mod­els was still an es­ti­mate at that point (based on o1 costs), so I would­n’t be sur­prised if these curves should re­ally be shifted some­what to the left, mak­ing them sev­eral times cheaper. We there­fore should­n’t lean too heav­ily on the fact that they cost as much or more than hu­man labour at their full time-hori­zon.As well as the sweet spot, ide­ally we could add a sat­u­ra­tion point for each curve — a point to rep­re­sent the lo­ca­tion where the plateau be­gins. We can’t sim­ply use the end of the curve since some have run longer into the plateau than oth­ers. What I’ll do is find the point where the slope has di­min­ished to 1/10th that of the sweet spot. This is the point at which it re­quires a 10% in­crease in cost just to in­crease the time hori­zon by 1%. Or equiv­a­lently, the time hori­zon is only grow­ing as the 1/10th power of com­pute. Of course the num­ber 1/10 is some­what ar­bi­trary, but un­like for the sweet spot, any de­f­i­n­i­tion of a sat­u­ra­tion point will be ar­bi­trary to some de­gree. As you can see be­low, this de­f­i­n­i­tion of sat­u­ra­tion point does roughly cor­re­spond with the in­tu­itive lo­ca­tion, though it is still not quite clear how best to deal with the fi­nal upticks.

Armed with our sweet spots and sat­u­ra­tion points, we can start to tease out the re­la­tion­ship be­tween time hori­zon and cost.

We can see that there is a weak, but clear, pos­i­tive cor­re­la­tion be­tween task du­ra­tion and cost in this dataset. Moreover, we see that higher task du­ra­tions (at the sweet spot) are as­so­ci­ated with higher hourly costs (and re­call that these hourly costs at the sweet spot are the best hourly cost achiev­able with that model).What about if we in­stead look at the mod­els’ sat­u­ra­tion points, which are a lit­tle ar­bi­trary in their de­f­i­n­i­tion, but closer to what METR is mea­sur­ing in their head­line re­sults about time hori­zons:

Again, there is a cor­re­la­tion be­tween time hori­zon and cost, and again the hourly costs seem to be in­creas­ing with time hori­zon too. Indeed it sug­gests we are near­ing the point where the mod­els’ peak per­for­mance comes at an im­prac­ti­cally high cost. If this re­la­tion­ship were to con­tinue, then fore­cast­ing when cer­tain time hori­zons will be avail­able from the head­line METR trend will be mis­lead­ing, as the mod­els would be im­prac­ti­cally ex­pen­sive when they first reach those ca­pa­bil­i­ties. We would need to wait some ad­di­tional pe­riod of time for them to come down suf­fi­ciently in cost.That said, there are some sig­nif­i­cant lim­i­ta­tions to the analy­sis above. Ideally one would want to:in­clude curves for a larger and more rep­re­sen­ta­tive set of mod­els­find a way of ad­dress­ing the uptick prob­lem­check if there is an is­sue with the costs of the OpenAI mod­els­For­tu­nately, it should be fairly easy for METR to per­form such analy­sis, and I hope they will fol­low up on this.Too few peo­ple are ask­ing about how the costs of AI agents are grow­ingThe key ques­tion is How is the hourly’ cost of LLM agents chang­ing over time?We can use METRs chart to shed some light on this.We need to add lines of con­stant hourly cost, sweet spots, and sat­u­ra­tion points.This pro­vides mod­er­ate ev­i­dence that:the costs to achieve the time hori­zons are grow­ing ex­po­nen­tially,even the hourly costs are ris­ing ex­po­nen­tially,the hourly costs for some mod­els are now close to hu­man costs.Thus, there is ev­i­dence that:the METR trend is partly dri­ven by un­sus­tain­ably in­creas­ing in­fer­ence com­puteth­ere will be a di­ver­gence be­tween what time hori­zon is pos­si­ble in-prin­ci­ple and what is eco­nom­i­cally fea­si­ble­real-world ap­pli­ca­tions of AI agents will lag be­hind the METR time-hori­zon trend by in­creas­ingly large amountsMETR has a sim­i­lar graph on their page for GPT-5.1 codex. It in­cludes more mod­els and com­pares them by to­ken counts rather than dol­lar costs:

the cor­re­la­tion be­tween time hori­zon and cost holds for these other mod­els toore­a­son­ing mod­els with more RL post-train­ing don’t al­ways dom­i­nate their pre­de­ces­sors (e.g. o1 is bet­ter at small to­ken bud­gets than o3 or GPT-5)the hor­i­zon­tal gap be­tween the OpenAI rea­son­ing mod­els and the rest is smaller, sup­port­ing the idea that their costs were a bit high in the main chart

$\setCounter{0}$

February 04, 2026

Hazard Rates for AI Agents Decline as a Task Goes On

...

Read the original on www.tobyord.com »

4 290 shares, 32 trendiness

State of Kdenlive

In 2025, the Kdenlive team con­tin­ued grind­ing to push the pro­ject for­ward through steady de­vel­op­ment, col­lab­o­ra­tion, and com­mu­nity sup­port. Over the past year we’ve found a nice bal­ance be­tween adding new fea­tures, bug fix­ing, pol­ish­ing the user in­ter­face, and im­prov­ing per­for­mance and work­flow, with sta­bil­ity tak­ing pri­or­ity over fea­ture creep. We re­launched the web­site with a new con­tent man­age­ment sys­tem, re­freshed some con­tent and the de­sign, and re­stored his­toric con­tent dat­ing back to 2002. We also strength­ened up­stream col­lab­o­ra­tion with the MLT de­vel­op­ers and con­tributed sev­eral im­prove­ments to OpenTimelineIO.Here’s a look at what we’ve been up to and what is ahead.As part of KDE Apps, we fol­low the KDE Gear re­lease cy­cle, with three ma­jor re­leases each year—in April, August, and December—each fol­lowed by three point main­te­nance re­leases.This re­lease added a pow­er­ful au­to­matic mask­ing tool and brought the last batch of fea­tures from our last fundraiser.The new Object Segmentation plu­gin based on the [SAM2][4] model al­lows to re­move any se­lected ob­ject from the back­ground.We rewrote our OpenTimelineIO im­port and ex­port func­tion us­ing the C++ li­brary. Now you can ex­change pro­jects with other edit­ing ap­pli­ca­tions that sup­port this open source file for­mat.Au­dio wave­form gen­er­a­tion got a 300% per­for­mance boost, along with a refac­tored sam­pling method that ac­cu­rately ren­ders the au­dio sig­nal and higher-res­o­lu­tion wave­forms for greater pre­ci­sion.This re­lease fo­cused heav­ily on sta­bi­liza­tion, bring­ing over 300 com­mits and fix­ing more than 15 crashes. Instead of ma­jor new fea­tures, the ef­fort went into pol­ish­ing and bug fix­ing.We re­designed the au­dio mixer bring­ing lev­els with clearer vi­su­als and thresh­olds. We also did some code refac­tor­ing and cleanup. This change fixes is­sues with HiDPI dis­plays with frac­tional scal­ing.Guides and Markers got a ma­jor over­haul this re­lease to im­prove the pro­ject or­ga­ni­za­tion.This re­lease the ti­tler re­ceived some much needed love like im­proved SVG and im­age sup­port with abil­ity to move and re­size items, added cen­ter re­size with Shift + Drag, and re­named the Pattern tab to Templates and moved the tem­plates drop­down to it­The fo­cus of this re­lease cy­cle was on im­prov­ing the user ex­pe­ri­ence and pol­ish­ing the user in­ter­face.We added a new first-run launch screen for first time users as well as added a Welcome Screen al­low­ing to eas­ily launch re­cent pro­jects.We added a new, more flex­i­ble dock­ing sys­tem that lets you group wid­gets, show or hide them on de­mand, and save lay­outs as sep­a­rate files that can be shared or stored within pro­jects.The au­dio wave­form in the Project Monitor got a re­vamped in­ter­face with an added min­imap.This next re­lease is just around the cor­ner and brings a nice batch of nifty new fea­tures like mon­i­tor mir­ror­ing and an­i­mated tran­si­tion pre­views, mak­ing it much eas­ier to vi­su­al­ize how they will look be­fore ap­ply­ing them. Additionally, drop­ping a tran­si­tion onto the time­line can now au­to­mat­i­cally ad­just its du­ra­tion to match the clips above and be­low, sav­ing time and re­duc­ing man­ual tweak­ing.This fea­ture al­lows you to mir­ror any mon­i­tor while work­ing in fullscreen mode. It’s es­pe­cially use­ful when work­ing with mul­ti­ple dis­plays or col­lab­o­rat­ing with oth­ers in the edit­ing room.Change the play­back speed of mul­ti­ple clips at on­ceIm­port a clip di­rectly from the time­line con­text menu and in­sert it at the click po­si­tionOp­tion to al­ways zoom to­ward the mouse po­si­tion in­stead of the time­line play­head­Our roadmap is con­stantly be­ing re­viewed and up­dated, and some of the up­com­ing high­lights in­clude im­ple­ment­ing the new fea­tures in MLT, the mul­ti­me­dia frame­work which pow­ers Kdenlive. Some ex­cit­ing up­com­ing fea­tures in­clude 10/12 bit color sup­port, play­back op­ti­miza­tions (decoding), and OpenFX sup­port. (Shoutout to a Kdenlive com­mu­nity mem­ber for lead­ing this ef­fort). Also ex­pected is a refac­tor­ing of the sub­ti­tle sys­tem as well as con­tin­u­ing to de­velop the Advanced Trimming Tools.We are cur­rently work­ing on refac­tor­ing the keyfram­ing sys­tem and im­ple­ment­ing a Dopesheet, ba­si­cally it is a ded­i­cated time­line for man­ag­ing and view­ing keyframes from mul­ti­ple ef­fects si­mul­ta­ne­ously. This work will also in­tro­duce per-pa­ra­me­ter keyfram­ing (currently, once you add a keyframe to an ef­fect, it is ap­plied to all pa­ra­me­ters by de­fault). More info can be found in the last sta­tus re­port. This work is made pos­si­ble through an NGI Zero Commons grant via NLnet.We have been work­ing on en­abling and fix­ing mul­ti­ple mod­ules in MLT to com­pile with MSVC al­low­ing us to ship Kdenlive in the Microsoft Store soon. Another ad­van­tage is that it will al­low to run unit tests on our CI for Windows.Currently, the Kdenlive core team is made up of 8 ac­tive mem­bers, in­clud­ing 2 de­vel­op­ers.In 2025, 38 peo­ple con­tributed code to Kdenlive (including the core dev team and other KDE devs), a truly im­pres­sive num­ber! Even more ex­cit­ing, about half of them were first-time con­trib­u­tors, which is al­ways great. We hope to see many of them con­tinue con­tribut­ing in the fu­ture. On be­half of the Kdenlive team, we salute you all!Note that these num­bers re­fer specif­i­cally to con­tri­bu­tions to the Kdenlive ap­pli­ca­tion. Other pro­jects such as the test suite and web­site are hosted in sep­a­rate repos­i­to­ries and are not in­cluded in these fig­ures.In February, part of the Kdenlive core team met in Amsterdam for a short sprint, high­lighted by a visit to the Blender Foundation, where we met with Francesco Siddi and he shared valu­able in­sights into Blender’s his­tory and of­fered ad­vice on prod­uct man­age­ment for Kdenlive. We also at­tended their weekly open ses­sion, where artists and de­vel­op­ers pre­sent progress on on­go­ing pro­jects. During the sprint, we dis­cussed and ad­vanced sev­eral tech­ni­cal top­ics, some high­lights in­clude:Fin­ish­ing an MLT Framework patch to en­able ren­der­ing with­out a dis­play server (needed for Flatpak test­ing)The Berlin sprint was one of our most pro­duc­tive gath­er­ings to date. Most of the team was there in per­son, and we also con­nected on­line with those who could­n’t make it. We dis­cussed just about every as­pect of the pro­ject, from roadmap plan­ning to up­com­ing fea­tures and work­flow im­prove­ments. Some of the high­lights in­clude:Eval­u­ated the cur­rent state of the Titler and dis­cussed pos­si­ble in­te­gra­tion with GlaxnimateDeveloped a proof of con­cept for us­ing KDDockWidgetsRedesigned and started de­vel­op­ment of the au­dio clip view in the Clip MonitorThanks to the nice folks at c-base who kindly hosted us.Akademy is al­ways a great op­por­tu­nity to ex­change ideas with the broader KDE and Qt com­mu­ni­ties. One of the high­lights was meet­ing the main­tainer of Glaxnimate, where we dis­cussed com­mon goals and ways to col­lab­o­rate. This year, Akademy will be in Graz on the 19-24 of September, and we hope to see you there.We’re very happy to see more YouTube chan­nels talk­ing about Kdenlive. Here are some ex­am­ples of what the com­mu­nity has been cre­at­ing.We’d love to see what you’ve been work­ing on in the past year. Share your videos pro­duc­tions in the com­ments!Help us grow the com­mu­nity by or­ga­niz­ing mee­tups, talks, or work­shops in your lo­cal area. Don’t hes­i­tate to con­tact us if you need guid­ance, ma­te­ri­als, or sup­port to get started.Be­low are pho­tos from a work­shop with in­dige­nous com­mu­ni­ties in Paraguay.Kdenlive was down­loaded 11,500,714 times from our down­load page in 2025. Do note that many ad­di­tional in­stalls hap­pen through Linux dis­tri­b­u­tion pack­age man­agers, the Snap Store, Flathub, and other third-party servers, where sta­tis­tics are not al­ways avail­able or re­li­ably mea­sur­able.The Flatpak pack­age from Flathub gets 41,499 down­loads per month.25.04.2 got the most num­ber of down­loads.Files With Most Code ChangesTo the 5 of you in Antarctica, let us know what you are edit­ing. ;)Ever since our last, and very suc­cess­ful, fundraiser in 2022, we haven’t ac­tively asked for do­na­tions, yet the com­mu­nity has con­tin­ued to sup­port us. We are very grate­ful for that.In 2025, we re­ceived a to­tal of €9,344.80 from do­na­tions (down from €11,526.61 in 2024). Around 30% of the amount was given by donors who kindly set up a re­cur­ring plan. The av­er­age do­na­tion was about €25, with the low­est amount be­ing €10 and the high­est €500.We al­lo­cate 20% of our bud­get to KDE e.V. to sup­port in­fra­struc­ture costs (servers and re­lated ex­penses), as well as ad­min­is­tra­tion, le­gal sup­port, and travel. As in pre­vi­ous years, your con­tri­bu­tions en­able us to con­tinue sup­port­ing Jean-Baptiste (Kdenlive’s main­tainer), al­low­ing him to ded­i­cate sev­eral days each month to Kdenlive in ad­di­tion to his vol­un­teer work.WE NEED YOUR SUPPORTKdenlive needs your sup­port to keep grow­ing and im­prov­ing. If just a quar­ter of the peo­ple who down­loaded Kdenlive in 2025 con­tributed €5, our main­tain­ers would be able to ded­i­cate more time to the pro­ject, and it would even al­low us to hire more de­velpers to speed up de­vel­op­ment and im­prove sta­bil­ity. Small amounts can make a big dif­fer­ence, please con­sider mak­ing a do­na­tion.You may also con­tribute by get­ting in­volved and help­ing in:

...

Read the original on kdenlive.org »

5 283 shares, 11 trendiness

Even "cat readme.txt" is not safe

In a pre­vi­ous post about AI-discovered bugs in Vim and Emacs, we looked at how seem­ingly harm­less work­flows could cross a sur­pris­ing line into code ex­e­cu­tion. This time we wanted to push that idea even fur­ther: is cat readme.txt safe?

It turns out that it is NOT, if you use iTerm2.

That looks in­sane un­til you un­der­stand what iTerm2 is try­ing to do for a le­git­i­mate fea­ture, how it uses the PTY, and what hap­pens when ter­mi­nal out­put is able to im­per­son­ate one side of that fea­ture’s pro­to­col.

We’d like to ac­knowl­edge OpenAI for part­ner­ing with us on this pro­ject.

iTerm2 has an SSH in­te­gra­tion fea­ture that gives it a richer un­der­stand­ing of re­mote ses­sions. To make that work, it does not just blindly type com­mands” into a re­mote shell. Instead, it boot­straps a tiny helper script on the re­mote side called the con­duc­tor.

iTerm2 sends a re­mote boot­strap script, the con­duc­tor, over the ex­ist­ing SSH ses­sion. That re­mote script be­comes the pro­to­col peer for iTerm2.iTerm2 and the re­mote con­duc­tor ex­change ter­mi­nal es­cape se­quences to co­or­di­nate things like:

The im­por­tant point is that there is no sep­a­rate net­work ser­vice. The con­duc­tor is just a script run­ning in­side the re­mote shell ses­sion, and the pro­to­col is car­ried over nor­mal ter­mi­nal I/O.

A ter­mi­nal used to be a real hard­ware de­vice: a key­board and screen con­nected to a ma­chine, with pro­grams read­ing in­put from that de­vice and writ­ing out­put back to it.

A ter­mi­nal em­u­la­tor like iTerm2 is the mod­ern soft­ware ver­sion of that hard­ware ter­mi­nal. It draws the screen, ac­cepts key­board in­put, and in­ter­prets ter­mi­nal con­trol se­quences.

But the shell and other com­mand-line pro­grams still ex­pect to talk to some­thing that looks like a real ter­mi­nal de­vice. That is why the OS pro­vides a PTY, or pseudoter­mi­nal. A PTY is the soft­ware stand-in for the old hard­ware ter­mi­nal, and it sits be­tween the ter­mi­nal em­u­la­tor and the fore­ground process.

* ssh for­wards those bytes to the re­mote ma­chine

* the re­mote con­duc­tor reads them from its stdin

So when iTerm2 wants to send a com­mand to the re­mote con­duc­tor,” what it ac­tu­ally does lo­cally is write bytes to the PTY.

The SSH in­te­gra­tion pro­to­col uses ter­mi­nal es­cape se­quences as its trans­port.

* DCS 2000p is used to hook the SSH con­duc­tor

* OSC 135 is used for pre-framer con­duc­tor mes­sages

At source level, DCS 2000p causes iTerm2 to in­stan­ti­ate a con­duc­tor parser. Then the parser ac­cepts OSC 135 mes­sages like:

So a le­git­i­mate re­mote con­duc­tor can talk back to iTerm2 en­tirely through ter­mi­nal out­put.

The bug is a trust fail­ure. iTerm2 ac­cepts the SSH con­duc­tor pro­to­col from ter­mi­nal out­put that is not ac­tu­ally com­ing from a trusted, real con­duc­tor ses­sion. In other words, un­trusted ter­mi­nal out­put can im­per­son­ate the re­mote con­duc­tor.

That means a ma­li­cious file, server re­sponse, ban­ner, or MOTD can print:

and iTerm2 will start act­ing like it is in the mid­dle of a real SSH in­te­gra­tion ex­change. That is the ex­ploit prim­i­tive.

iTerm2 ren­ders the file, but the file is not just text. It con­tains:

Once the hook is ac­cepted, iTerm2 starts its nor­mal con­duc­tor work­flow. In up­stream source, Conductor.start() im­me­di­ately sends get­shell(), and af­ter that suc­ceeds it sends python­ver­sion().

So the ex­ploit does not need to in­ject those re­quests. iTerm2 is­sues them it­self, and the ma­li­cious out­put only has to im­per­son­ate the replies.

The fake OSC 135 mes­sages are min­i­mal but pre­cise.

They do this:

Return lines that look like shell-dis­cov­ery out­put

This is enough to push iTerm2 down its nor­mal fall­back path. At that point, iTerm2 be­lieves it has com­pleted enough of the SSH in­te­gra­tion work­flow to move on to the next step: build­ing and send­ing a run(…) com­mand.

The forged DCS 2000p hook con­tains sev­eral fields, in­clud­ing at­tacker-con­trolled sshargs.

That value mat­ters be­cause iTerm2 later uses it as com­mand ma­te­r­ial when it con­structs the con­duc­tor’s run … re­quest.

The ex­ploit chooses sshargs so that when iTerm2 base64-en­codes:

the last 128-byte chunk be­comes:

That string is not ar­bi­trary. It is cho­sen be­cause it is both:

In a le­git­i­mate SSH in­te­gra­tion ses­sion, iTerm2 writes base64-en­coded con­duc­tor com­mands to the PTY, and ssh for­wards them to the re­mote con­duc­tor. In the ex­ploit case, iTerm2 still writes those com­mands to the PTY, but there is no real SSH con­duc­tor. The lo­cal shell re­ceives them as plain in­put in­stead.

That is why the ses­sion looks like this when recorded:

* the last chunk is ace/​c+al­i­FIo

Earlier chunks fail as non­sense com­mands. The fi­nal chunk works if that path ex­ists lo­cally and is ex­e­cutable.

You can re­pro­duce the orig­i­nal file-based PoC with gen­poc.py:

* readme.txt, a file con­tain­ing the ma­li­cious DCS 2000p and OSC 135 se­quences

The first fools iTerm2 into talk­ing to a fake con­duc­tor. The sec­ond gives the shell some­thing real to ex­e­cute when the fi­nal chunk ar­rives.

For the ex­ploit to work, run cat readme.txt from the di­rec­tory con­tain­ing ace/​c+al­i­FIo, so the fi­nal at­tacker-shaped chunk re­solves to a real ex­e­cutable path.

* Mar 30: We re­ported the bug to iTerm2.

* Mar 31: The bug was fixed in com­mit a9e745993c2e2cb­b30b884a16617cd5495899f86.

* At the time of writ­ing, the fix has not yet reached sta­ble re­leases.

When the patch com­mit landed, we tried to re­build the ex­ploit from scratch us­ing the patch alone. The prompts used for that process are in prompts.md, and the re­sult­ing ex­ploit is gen­poc2.py, which works very sim­i­larly to gen­poc.py.

...

Read the original on blog.calif.io »

6 277 shares, 5 trendiness

Israel escalates attacks on medics in Lebanon with deadly ‘quadruple tap’

When they re­ceived the call to re­spond to an Israeli airstrike in the city of Mayfadoun, in south­ern Lebanon, most of the para­medics held back, hav­ing pre­vi­ously seen col­leagues killed by dou­ble-tap at­tacks tar­get­ing res­cuers. But the medics from the Islamic Health Association (IHA) rushed to the scene.

By the time the other emer­gency work­ers ar­rived at the site, they found the IHA medics had in­deed been caught in a sec­ond strike. They started evac­u­at­ing their wounded col­leagues, only for their am­bu­lances to be hit in two fur­ther at­tacks.

One of the para­medics cov­ered his ears and screamed, con­vuls­ing in pain as shrap­nel shat­tered the back win­dow of the am­bu­lance.

The res­cue mis­sion on Wednesday af­ter­noon had turned into a night­mare as Israel car­ried out three con­sec­u­tive strikes on three sets of am­bu­lances and med­ical work­ers.

In to­tal, the at­tacks killed four medics and wounded six more, from three dif­fer­ent am­bu­lance corps, ac­cord­ing to med­ical sources. Three of the medics were from the Hezbollah-affiliated IHA and Amal-affiliated med­ical corps, while one was from the Nabatieh emer­gency ser­vices or­gan­i­sa­tion. Under in­ter­na­tional law, all medics are pro­tected and are con­sid­ered non-com­bat­ants, re­gard­less of po­lit­i­cal af­fil­i­a­tion.

Rescuers in Lebanon have long been wary of the dou­ble-tap at­tack, when Israeli forces tar­get a lo­ca­tion, wait un­til peo­ple gather to help sur­vivors, and then strike again. Wednesday’s three-wave at­tack af­ter the ini­tial one prompted the coin­ing of a fear­some new term: the quadru­ple tap.

In a video taken by one of the para­medics at the site, res­cuers are seen load­ing two wounded peo­ple into their am­bu­lances when a bomb lands next to their ve­hi­cle. Paramedics rush to ex­tract the dri­ver, who is mo­tion­less and limp as they pull him from the am­bu­lance, which is splashed with blood. Oh God, oh God,” the man film­ing can be heard say­ing. They carry two more blood-cov­ered medics out of their ve­hi­cle and on to stretch­ers.

Among the para­medics killed was Fadel Sarhan, 43, who is sur­vived by his eight-year-old daugh­ter.

Fadel was a very loved per­son. He had a bold per­son­al­ity, but at the same time, he was emo­tional. He was well liked and re­spon­si­ble,” said Ali Nasr al-Deen, the head of the Mayfadoun civil de­fence cen­tre who grew up with Sarhan.

He used to feed the cats and dogs. He would bring pet food from Beirut so they would­n’t go hun­gry. He was that kind of per­son, car­ing and at­ten­tive. It’s a huge loss for us,” said Nasr al-Deen.

Medics mourned their col­leagues on Thursday at fu­ner­als in Nabatieh, a city near Mayfadoun. Such events have be­come in­creas­ingly com­mon, with health­care work­ers killed by Israeli bomb­ings on a near daily ba­sis.

Mohammed Suleiman, whose 16-year-old son, Joud, was killed while on duty as a para­medic by an Israeli strike weeks ear­lier, joined his peers in bury­ing an­other of his friends on Thursday. A few hours af­ter the fu­ner­als, Israel car­ried out an­other wave of airstrikes on Nabatieh.

Israel has so far killed 91 health­care work­ers and wounded 214 more in Lebanon since the Israel-Hezbollah war started on 2 March. It has given lit­tle jus­ti­fi­ca­tion for its re­peated at­tacks on med­ical in­fra­struc­ture and work­ers, apart from ac­cus­ing Hezbollah of us­ing am­bu­lances and hos­pi­tals to trans­port fight­ers and weapons, with­out pro­vid­ing ev­i­dence for the claim.

The Lebanese min­istry of health ac­cused Israel of de­lib­er­ately tar­get­ing am­bu­lance crews. Paramedics have be­come di­rect tar­gets, pur­sued re­lent­lessly in a bla­tant vi­o­la­tion that con­firms a to­tal dis­re­gard for all norms and prin­ci­ples es­tab­lished by in­ter­na­tional hu­man­i­tar­ian law,” the min­istry said in a state­ment.

The Israeli mil­i­tary did not im­me­di­ately re­spond to a re­quest for com­ment.

In the video taken of the quadru­ple tap on Wednesday, the frame was frozen on the in­te­rior of the am­bu­lances, as the Nabatieh emer­gency ser­vices high­lighted that the ve­hi­cle clearly con­tained no weapons.

A few hours af­ter Israel hit the am­bu­lances out­side Nabatieh, it bombed the vicin­ity of the gov­ern­men­tal hos­pi­tal in Tebnine, south Lebanon. It was the sec­ond time in two days that Israeli bomb­ings dam­aged the health­care fa­cil­ity, which is the only re­main­ing pub­lic hos­pi­tal in the area. The strikes in­jured 11 hos­pi­tal work­ers and dam­ag­ing the emer­gency de­part­ment, ac­cord­ing to the World Health Organization (WHO).

A video of Tebnine hos­pi­tal from 14 April showed work­ers try­ing to clear shat­tered con­crete and de­bris from the emer­gency de­part­ment af­ter a strike blew in the win­dows.

Commenting on the strike in Tebnine, the head of the WHO, Tedros Adhanom Ghebreyesus, said: I re­it­er­ate the call for the im­me­di­ate pro­tec­tion of health­care fa­cil­i­ties, health work­ers, am­bu­lances and pa­tients. There must be safe, sus­tained and un­hin­dered hu­man­i­tar­ian ac­cess across Lebanon.”

An am­bu­lance in Tebnine was also struck on Thursday, lead­ing to the crit­i­cal in­jury of two medics, ac­cord­ing to the Lebanese min­istry of health. As health­care work­ers watched their col­leagues and friends be­ing killed by Israel, the men­tal toll was be­com­ing al­most too much to bear.

We have to go to places to res­cue peo­ple, but then we get dou­ble tapped,” said Abbas Atwi, the head of the IHAs emer­gency de­part­ment in Nabatieh, shortly af­ter a med­ical cen­tre was tar­geted in March, killing his friends and col­leagues. But we will stay and keep go­ing, we will not leave.”

...

Read the original on www.theguardian.com »

7 274 shares, 15 trendiness

Interval Calculator

This is a cal­cu­la­tor that works over unions of in­ter­vals rather than just real num­bers. It is an im­ple­men­ta­tion of Interval

Union Arithmetic.

An in­ter­val [a, b] rep­re­sents the set of all num­bers be­tween and in­clud­ing a and b. An in­ter­val union: [a, b] U [c, d] is a dis­joint set of in­ter­vals.

Interval union arith­metic is an ex­ten­sion of reg­u­lar in­ter­val arith­metic that is vastly su­pe­rior, mostly be­cause it re­mains closed while sup­port­ing di­vi­sion by in­ter­vals con­tain­ing zero:

➤ 2 / [-2, 1]

[-∞, -1] U [2, +∞]

The in­ter­est­ing thing about in­ter­val union arith­metic is the in­clu­sion prop­erty, which means that if you pick any real num­ber from every in­put union and com­pute the same ex­pres­sion over the re­als, the re­sult is guar­an­teed to be in the out­put union.

You can use it to rep­re­sent un­cer­tainty:

➤ 50 * (10 + [-1, 1])

[450, 550]

You can also com­pute more com­plex in­ter­val ex­pres­sions, us­ing the in­ter­val union op­er­a­tor U:

➤ ( [5, 10] U [15, 16] ) / [10, 100]

[0.05, 1.6]

Operations can re­sult in dis­joint unions of in­ter­vals:

➤ 1 / [-2, 1]

[-∞, -0.5] U [1, +∞]

➤ tan([pi/​3, 2*pi/3])

[-∞, -1.732] U [1.732, +∞]

In full pre­ci­sion mode, you can use it as a reg­u­lar cal­cu­la­tor, and ob­tain in­ter­val re­sults that are guar­an­teed to con­tain the true value, de­spite float­ing point pre­ci­sion is­sues:

➤ 0.1 + 0.2

[0.29999999999999993, 0.3000000000000001]

Note: you can in­put in­ter­vals with the bracket syn­tax: [1, 2], or bare num­bers with­out brack­ets: 3.14. Bare num­bers are in­tepreted as a nar­row in­ter­val, i.e. [3.14, 3.14] (with sub­tleties re­lated to full pre­ci­sion mode). This en­ables bare num­bers and in­ter­vals to be mixed nat­u­rally:

➤ 1.55 + [-0.002, 0.002]

[1.548, 1.552]

A sur­pris­ing con­se­quence of the cal­cu­la­tor gram­mar is that in­ter­vals can be nested and you can write things like:

➤ [0, [0, 100]]

[0, 100]

This is be­cause all num­bers, in­clud­ing those in­side an in­ter­val bracket which de­fine a bound, are in­ter­preted as in­ter­vals. When nest­ing two in­ter­vals as above, an in­ter­val used as an in­ter­val bound is the same as tak­ing its up­per bound. This de­sign choice en­ables us­ing arith­metic on in­ter­val bounds them­selves:

➤ [0, cos(2*pi)]

[0, 1]

Outward round­ing is im­ple­mented over IEEE 754 dou­ble pre­ci­sion floats (javascript’s num­ber type), so re­sult in­ter­vals are guar­an­teed to con­tain the true value that would be ob­tained by com­put­ing the same ex­pres­sion over the re­als with in­fi­nite pre­ci­sion. For ex­am­ple, try the fa­mous sum 0.1 + 0.2 in the cal­cu­la­tor. Interval arith­metic com­putes an in­ter­val that is guar­an­teed to con­tain 0.3, even though 0.3 is not rep­re­sentable as a dou­ble pre­ci­sion float.

* Numbers in­put by the user are in­ter­preted as the small­est

in­ter­val that con­tains the IEEE 754 value clos­est to the in­put

dec­i­mal rep­re­sen­ta­tion but where nei­ther bounds are equal to

it

* Output num­bers are dis­played with all avail­able dec­i­mal

dig­its (using Number.toString())

* Numbers in­put by the user are in­ter­preted as the

de­gen­er­ate in­ter­val (width zero) where both bounds are equal

to the IEEE 754 value clos­est to the in­put dec­i­mal

rep­re­sen­ta­tion

* Output num­bers are dis­played with a max­i­mum of 4 dec­i­mal

dig­its (using Number.toPrecision())

While I’ve been very care­ful, I’m sure there are still some bugs in the cal­cu­la­tor. Please re­port

any is­sue on GitHub.

Interval

Calculator and not-so-float

(the en­gine pow­er­ing the cal­cu­la­tor) are open-source. If you you like my open-source work, please con­sider spon­sor­ing me

on GitHub. Thank you ❤️

* Split full pre­ci­sion mode into two con­trols: in­put

in­ter­pre­ta­tion and dis­play pre­ci­sion

...

Read the original on victorpoughon.github.io »

8 244 shares, 29 trendiness

Why Japan has such good railways

Japan is the land of the train. 28 per­cent of pas­sen­ger kilo­me­ters in Japan are trav­elled by rail, more than any­where else in the de­vel­oped world. France achieves 10 per­cent, Germany 6.4 per­cent, and the United States just 0.25 per­cent. Travel in Japan is over a hun­dred times more likely to be by rail than travel in the United States.

Japan’s vast rail­way net­work is di­vided be­tween dozens of com­pa­nies, nearly all of them pri­vate. The largest of these, JR East, car­ries more pas­sen­gers than the en­tire rail­way sys­tem of every coun­try other than China and India. Each year, JR East car­ries four times as many pas­sen­gers as the whole British rail­way sys­tem, even though it has fewer kilo­me­ters of track, serves about ten mil­lion fewer peo­ple, and com­petes with eight other com­pa­nies. Japan’s rail­way sys­tem turns a large op­er­at­ing profit and re­ceives far less pub­lic sub­sidy than European and American rail­ways.

Subscribe for $100 to re­ceive six beau­ti­ful is­sues per year.

In most de­vel­oped coun­tries, the rail­ways have strug­gled since the rise of the au­to­mo­bile in the 1950s. From this point on, North America saw the near-to­tal re­place­ment of pas­sen­ger trains with cars and planes. In Europe, it meant vast gov­ern­ment fi­nan­cial sup­port to keep the lines open.

Japan’s dif­fer­ent tra­jec­tory is of­ten at­trib­uted to cul­ture: the Japanese are con­formists who are con­tent to take pub­lic trans­port, un­like free­dom-lov­ing Americans who pre­fer to drive every­where. Europeans are some­where in be­tween. Culture is also used to ex­plain the in­cred­i­ble punc­tu­al­ity of Japanese rail­ways.

These cul­tural ex­pla­na­tions are wrong. The Japanese love cars, but they take trains be­cause they have the best rail­way sys­tem in the world. And their sys­tem ex­cels be­cause of good pub­lic pol­icy: busi­ness struc­ture, land use rules, dri­ving rules, su­pe­rior mod­els for pri­va­ti­za­tion, and sound reg­u­la­tion have given Japan its out­stand­ing rail­ways.

This is good news for friends of rail. Culture is built over cen­turies, and repli­cat­ing it is hard. But suc­cess­ful pub­lic poli­cies can be em­u­lated by one good gov­ern­ment. Much about Japan’s rail­way sys­tem could be replic­a­ble around the world.

Today, the most strik­ing in­sti­tu­tional fea­ture of Japanese rail is that it is pri­vately owned by a throng of com­pet­ing com­pa­nies.

The rail­way ar­rived in Japan in 1872, dur­ing the Meiji Restoration, which opened the coun­try up to for­eign trade, ideas, and tech­nolo­gies. Like most Western coun­tries, Japan na­tion­al­ized its rail­ways in the early twen­ti­eth cen­tury, cre­at­ing what be­came known as Japanese National Railways (JNR). But it did not na­tion­al­ize all of the lines, fo­cus­ing only on main­line rail­ways of na­tional im­por­tance, and new pri­vate rail­ways were still per­mit­ted.

Between 1907 and World War II, Japan saw a boom in new pri­vate elec­tric rail­ways, co­in­cid­ing with rapid ur­ban­iza­tion. Technologically, most of these pri­vate rail­ways were sim­i­lar to the fa­mous in­terur­bans in the United States: they were ba­si­cally elec­tric trams, but run­ning be­tween cities as well as within them. The American net­work even­tu­ally with­ered, and al­most noth­ing of it sur­vives to­day. In Japan, how­ever, the net­work con­sol­i­dated, and the light tram­lines grad­u­ally evolved into heavy-rail in­ter­city con­nec­tions.

These com­pa­nies are to­day known as legacy pri­vate rail­ways’ on ac­count of their hav­ing been pri­vate since their in­cep­tion. There are eight legacy pri­vate rail­ways in the Tokyo met­ro­pol­i­tan area, five in the Osaka–Kobe–Kyoto mega­lopo­lis, two in Nagoya, and one in the fourth city of Fukuoka. There are also dozens of smaller ones else­where. In the three largest ur­ban ar­eas, these op­er­a­tors ac­count for nearly half of rail­way track and sta­tions, as well as a plu­ral­ity of rid­er­ship. The largest, Kintetsu, not only op­er­ates ur­ban ser­vices, but a whole in­ter­city net­work stretch­ing from Osaka to Nagoya.

These com­pa­nies of­ten com­pete head-to-head. At its most ex­treme, three sep­a­rate com­muter lines com­pete for the traf­fic be­tween Osaka and the port city of Kobe, run­ning in par­al­lel, some­times fewer than 500 meters apart.

Meanwhile, the na­tion­al­ized rail­ways were man­aged by JNR. In the post­war era, JNR was re­spon­si­ble for build­ing the fa­mous Shinkansen sys­tem, as well as run­ning com­muter and long-dis­tance lines through­out Japan. But in 1988, it was largely pri­va­tized, bro­ken into six re­gional mo­nop­o­lies for pas­sen­ger ser­vices to­gether with a sin­gle na­tional freight op­er­a­tor. These are col­lec­tively known as the Japan Railways Group (JR).

This means that Japan has ended up with six rail­way com­pa­nies that trace their de­scent to the na­tion­al­ized rail­ways, the six­teen big legacy com­pa­nies that have al­ways been pri­vate, and a host of mi­nor legacy rail­ways, as well as nu­mer­ous un­der­ground met­ros (some pri­vate, some mu­nic­i­pally owned), mono­rails, and tram sys­tems. This in­sti­tu­tional di­ver­sity is strik­ing enough. But equally strik­ing is the con­sis­tent busi­ness model that has evolved amidst this plu­ral­ism: the rail­way that builds a city.

If I take a train to go for a soli­tary walk in the coun­try­side, the rail­way com­pany can cap­ture some of the value it cre­ates by charg­ing me for the jour­ney, just as other com­pa­nies cap­ture the value of the goods and ser­vices they pro­vide by charg­ing for them. However, if I take a train to visit fam­ily, clients, a the­ater, or a shop, an im­por­tant dif­fer­ence ap­pears. The rail­way can cap­ture the value it cre­ates for me by charg­ing me a fare, but it can­not cap­ture the value it cre­ates for those at my des­ti­na­tion. As trans­port in­fra­struc­ture cre­ates ben­e­fits that pro­duce no rev­enue for providers, free mar­kets rarely build enough of it.

Japan has partly solved this prob­lem by en­abling rail­way com­pa­nies to do a great deal be­side run­ning rail­ways. Take the ex­am­ple of the Tokyu cor­po­ra­tion, one of the legacy pri­vate rail­ways in south­ern Tokyo. You can not only travel on its trains, but also ride a Tokyu bus, live in a Tokyu-built house, work in a Tokyu of­fice com­plex, see a doc­tor in a Tokyu hos­pi­tal, buy gro­ceries in a Tokyu su­per­mar­ket, spend an af­ter­noon at a Tokyu mu­seum-the­ater-cin­ema com­plex, take your chil­dren to their amuse­ment park, and even die in a Tokyu re­tire­ment home. The pos­i­tive spillover ef­fects of the rail­way on these things are cap­tured by Tokyu be­cause it owns them. The pres­i­dent of Tokyu has said:

I think that though we are a rail­way com­pany, we con­sider our­selves a city-shap­ing com­pany. In Europe for in­stance, rail­way com­pa­nies sim­ply con­nect cities through their ter­mi­nals. That is a pretty nor­mal way of op­er­at­ing in this in­dus­try, whereas what we do is com­pletely dif­fer­ent: we cre­ate cities and then, as a util­ity fa­cil­ity, we add the sta­tions and the rail­ways to con­nect them one with an­other.

This model was pi­o­neered in the 1950s by what be­came Hankyu Railways. Hankyu’s net­work con­nects cen­tral Osaka to its north­ern sub­urbs, as well as Kyoto and Kobe. Its in­no­v­a­tive founder Kobayashi Ichizo first built sub­ur­ban hous­ing, then a de­part­ment store at the ter­mi­nal sta­tion; he then cre­ated a hot spring re­sort, a zoo, and his own dis­tinc­tive brand of all-women mu­si­cal the­ater, the Takarazuka Revue. He also be­gan to run bus ser­vices to and from his sta­tions. Other com­pa­nies em­u­lated Hankyu’s ex­am­ple: Tokyo Disneyland is a col­lab­o­ra­tion be­tween Disney and the Keisei Railway, while Hanshin in Osaka owns the Hanshin Tigers base­ball team.

Core rail op­er­a­tions are prof­itable for every Japanese pri­vate rail­way com­pany, but they usu­ally only ac­count for a plu­ral­ity or a small ma­jor­ity of rev­enue. The rest is con­tributed by their port­fo­lio of side busi­nesses. There is a nat­ural fi­nan­cial syn­ergy be­tween the re­li­able but un­re­mark­able cash flow of train fares and the prof­itable but riskier real es­tate and com­mer­cial side of the busi­ness. Railway com­pa­nies’ side busi­nesses also at­tract peo­ple to live and work on their rail cor­ri­dor, re­in­forc­ing the cus­tomer base for the rail­way ser­vices them­selves.

This vir­tu­ous cir­cle is en­abled by tran­sit-ori­ented de­vel­op­ment. Japan’s lib­eral land use reg­u­la­tion makes it straight­for­ward to build new neigh­bor­hoods next to rail­way lines, giv­ing com­muters easy ac­cess to city cen­ters. It also en­ables the den­si­fi­ca­tion of these cen­ters, which means that com­muters have more places they want to go.

Railways cost a lot to build, but once they are built, they can move enor­mous num­bers of peo­ple, far more than a road of sim­i­lar size. This means that they work best in cities with a high den­sity of peo­ple, jobs, and other ac­tiv­i­ties. In 2019, New York City was the only American city where rail had a higher modal share than cars, in part be­cause Manhattan has 2.5 mil­lion jobs, two mil­lion res­i­dents, and 50 mil­lion tourist vis­its crammed into 59 square kilo­me­ters.

This does not mean that rail-ori­ented cities must be struc­tured like Chinese cities: is­lands of high-rise apart­ments con­nected by met­ros and sep­a­rated by mo­tor­ways. Japanese cities have the low­est res­i­den­tial den­sity in Asia, and a plu­ral­ity of the Japanese live in houses, usu­ally de­tached ones. The ur­ban area of Tokyo, the dens­est Japanese city, has a weighted pop­u­la­tion den­sity less than that of many European cities, in­clud­ing Paris, Madrid, or Athens. Japanese cities have vast low-rise, pre­dom­i­nantly res­i­den­tial sub­urbs, built at den­si­ties that might be higher than what is typ­i­cal in the United States, but that would be quite nor­mal in Northern Europe.

What makes Japan’s cities par­tic­u­larly suited to rail is thus not their res­i­den­tial dis­tricts, but their huge and hy­per­dense cen­ters. These re­ally are spe­cial: the cores of Tokyo or Osaka are un­like any­thing that ex­ists in Europe or North America. Many of their fea­tures are fa­mous world­wide: the ver­ti­cal street za­kkyo build­ings, un­der­ground streets, shop­ping streets un­der rail tracks, cov­ered ar­cades, el­e­vated sta­tion squares, and ver­ti­cal cities. Getting mil­lions of com­muters and shop­pers into these down­towns is where rail ex­cels be­cause its ex­treme spa­tial ef­fi­ciency means that in­fra­struc­ture with a rel­a­tively mod­est foot­print can trans­port vast num­bers of peo­ple into a small area.

None of this emerged from a co­her­ent mas­ter­plan of tran­sit-ori­ented de­vel­op­ment like Copenhagen’s Finger Plan or Curitiba’s Trinary System. Postwar Japanese opin­ion was com­mit­ted to de­cen­tral­iza­tion both to rural pe­riph­eries and to the sub­urbs through green­belts, mo­tor­ways, and new towns.

Instead, this va­ri­ety and adapt­abil­ity around rail­ways is pos­si­ble be­cause of the way Japanese ur­ban plan­ning works. Since 1919, Japan has had a stan­dard­ized na­tional zon­ing sys­tem, but it is much more lib­eral than de­vel­op­ment con­trol sys­tems in Western coun­tries. The Japanese au­thor­i­ties did not in­tend or even de­sire dense ur­ban cen­ters, but they did not pre­vent them, rather like nine­teenth-cen­tury gov­ern­ments in the West.

This lib­eral zon­ing sys­tem is re­in­forced by pri­vate ac­cess to city plan­ning pow­ers. Thirty per­cent of Japan’s ur­ban land has been sub­ject to land read­just­ment, where agree­ment among two thirds of res­i­dents and landown­ers in an area is enough to al­low its re­plan­ning, in­clud­ing com­pul­so­rily tak­ing and de­mol­ish­ing land for ameni­ties and in­fra­struc­ture. Initially land read­just­ment was used only to as­sem­ble rural land for ur­ban­iza­tion, but over time it was in­creas­ingly used to re­de­velop al­ready ur­ban­ized ar­eas, and new vari­ants were cre­ated to build the sky­scrap­ers that sur­round the ma­jor sta­tions of cen­tral Tokyo.

The his­tory of the pri­vate rail­way com­pa­nies could be writ­ten as a story of land read­just­ment pro­jects: the ini­tial build­ing of the lines in the in­ter­war years pro­ceeded through one land read­just­ment pro­ject af­ter an­other. Postwar im­prove­ments such as dou­ble-track­ing, plat­form length­en­ing, and con­stant re­de­vel­op­ment of sta­tions and their im­me­di­ate thresh­olds were only pos­si­ble be­cause the rail­ways could se­cure land tak­ings co­op­er­a­tively with lo­cal busi­nesses and landown­ers.

Perhaps the great­est ex­am­ple of this phe­nom­e­non in­volved Tokyu. In 1953 the com­pany de­cided to build the Den’en Toshi Line, or Garden City Line, to serve a rural area south­west of Tokyo. This would be en­abled by a se­ries of land read­just­ment pro­jects col­lec­tively among the largest in Japanese his­tory.

Over 30 years, 3,100 hectares were cov­ered, of which only 36 per­cent was de­voted to res­i­den­tial and com­mer­cial de­vel­op­ment, with 20 per­cent for for­est and parks, 17 per­cent for roads, and much of the rest for wa­ter­courses. The pop­u­la­tion of the land read­just­ment zone would rise from 42,000 in 1954 to over 500,000 in 2003.

By con­nect­ing the af­flu­ent south­west­ern sub­urbs to Tokyu’s main real es­tate hub next to Shibuya sta­tion, now the sec­ond busiest in the world, the Den’en Toshi Line al­lowed Tokyu to be­come the largest pri­vate rail­way by rev­enue and rid­er­ship. The Japanese gov­ern­ment and aca­d­e­mics gen­er­ally con­sider the Den’en Toshi Line to be the best cor­ri­dor of tran­sit–ori­ented de­vel­op­ment in Japan.

But the rail­way-as-city-builder model is not the only rea­son Japanese rail­ways have been able to thrive. European coun­tries usu­ally pro­hib­ited rail­ways from run­ning real es­tate side busi­nesses, but in the United States and Canada the prac­tice was ex­tremely wide­spread in the nine­teenth and early twen­ti­eth cen­turies, and many fa­mous rail­way sub­urbs were de­vel­oped this way. Despite this, pas­sen­ger rail in these coun­tries col­lapsed in the mid-twen­ti­eth cen­tury. Part of the dif­fer­ence was that Japan did not ex­tend the same im­plicit sub­si­dies to cars as Western gov­ern­ments did.

The land of Toyota, Nissan, and Honda is not an anti-car nir­vana. In fact, Japan has ex­cel­lent mo­tor­ways, and across the coun­try as a whole a small ma­jor­ity of jour­neys are made by car. But Japan is a place where cars and car-ori­ented lifestyles com­pete on a level play­ing field.

Japan is one of the only coun­tries to have pri­va­tized park­ing. In Europe and North America, vast quan­ti­ties of park­ing space is so­cial­ized: mu­nic­i­pal­i­ties own the streets and al­low peo­ple to park on them at low or zero cost. Initially with the in­ten­tion of en­cour­ag­ing the pro­vi­sion of more park­ing spaces, Japan made it il­le­gal to park on pub­lic roads or pave­ments with­out spe­cial per­mis­sion. Before some­one buys a car, they must prove that they have a re­served night-time space on pri­vate land, ei­ther owned or leased.

Since park­ing on pub­lic land is banned, mu­nic­i­pal­i­ties are not wor­ried about over­spill park­ing from de­vel­op­ments with in­ad­e­quate pri­vate park­ing. They there­fore have no rea­son to im­pose park­ing min­i­mums on de­vel­op­ments: the mar­ket is left to de­cide whether park­ing is the most valu­able use of pri­vate land. Where land is abun­dant, as in rural ar­eas, sub­urbs, or small towns, pri­vate park­ing is plen­ti­ful. But in city cen­ters, it is out­com­peted by other land uses. According to the late Donald Shoup, cen­tral Tokyo has 23 park­ing spaces per hectare and 0.04 park­ing spaces per job, com­pared with 263 and 0.52 for Los Angeles. Even Manhattan, the dens­est ur­ban area in North America with the low­est lev­els of car own­er­ship, has about 60 park­ing spaces per hectare.

Japanese roads are ex­pected to be self-fi­nanc­ing. Motorways are run by self-con­tained pub­lic co­op­er­a­tives, very sim­i­lar to the statu­tory au­thor­i­ties that ran English roads and canals be­tween 1660 and the late 1800s, and funded by tolls on their users. Vehicle reg­is­tra­tion taxes, which are al­lo­cated to lo­cal­i­ties for road con­struc­tion and main­te­nance, are worth three per­cent of the Japanese gov­ern­ment bud­get.

These mea­sures, adopted in the 1950s, were not in­tended to sup­press car use — the point was to fund a mas­sive road ex­pan­sion — but they have forced pri­vate ve­hi­cles to in­ter­nal­ize many of their hid­den costs. In the Tokyo ur­ban area, the av­er­age house­hold spends 71,000 yen ($450) each year on pub­lic trans­port fares and 210,000 yen ($1,350) on car pur­chase and main­te­nance costs.

But the pri­vate car was not the only com­peti­tor faced by the pri­vate rail­ways. For eight decades in the twen­ti­eth cen­tury, they also had to face the jug­ger­naut of Japanese National Railways. Its pri­va­ti­za­tion in 1988 re­moved the fi­nal ob­sta­cle to cre­at­ing the world’s best rail­way sys­tem.

Railway pri­va­ti­za­tion in Britain, New Zealand, Argentina, and Sweden has had a mixed re­cep­tion, and all of those coun­tries, apart from Sweden, have taken steps to re­verse it. In Japan, it has been so suc­cess­ful that the gov­ern­ment sub­se­quently pri­va­tized the metro sys­tems in Tokyo and Osaka.

In the post­war pe­riod, JNR en­joyed real suc­cesses. It built the rev­o­lu­tion­ary Shinkansen, the first high-speed rail­way in the world. It also ag­gres­sively elec­tri­fied and dou­ble-tracked ma­jor trunk lines, quadru­ple-tracked lines into and out of ma­jor cities, and added city-cen­ter loops and freight by­passes. But these achieve­ments were over­shad­owed by two prob­lems.

The first was pol­i­tics. Many coun­tries adapted to the rise of the car by clos­ing the least prof­itable parts of their pas­sen­ger rail net­work, like the con­sol­i­da­tion of American freight rail into the Class I op­er­a­tors or the Beeching Axe in Britain. In Japan, how­ever, the rul­ing Liberal Democratic Party drew its sup­port from rural con­stituen­cies, whose sup­port it re­tained with pork–bar­rel pol­i­tics. Its rail tribe’ group, led by rural MPs, pre­vented JNR from adapt­ing it­self to mass mo­tor­iza­tion.

JNR there­fore did not am­pu­tate gan­grenous rural and freight ser­vices that im­posed heavy costs with few ben­e­fits. Worse, it con­tin­ued to build new loss-mak­ing rural rail­way lines, known in Japanese as Gaden-intetsu, or rail­ways pulled into the rice field.

The sec­ond prob­lem was or­ga­nized la­bor. In gen­eral, Japanese trade unions are known for their mod­er­a­tion and re­spon­si­bil­ity, a gen­er­al­i­sa­tion that also held true for the unions at the legacy pri­vate rail­ways. The JNR unions, how­ever, be­came highly mil­i­tant, se­cure in the knowl­edge that their na­tion­al­ized em­ploy­ers could never go bank­rupt. Their largest se­ries of strikes in 1973 pro­voked ri­ots from com­muters.

The rail­way unions im­posed over­staffing on rev­enue-gen­er­at­ing ur­ban ser­vices, at a time when both in­ter­na­tional and pri­vate do­mes­tic op­er­a­tors were re­duc­ing staffing re­quire­ments against a back­drop of higher wages and the grow­ing au­toma­tion of sig­nal­ing and tick­et­ing. As a re­sult, 78 per­cent of JNRs costs were re­lated to la­bor, com­pared to 40 per­cent for other Japanese rail­ways. The av­er­age worker at a pri­vate rail­way was 121 per­cent more pro­duc­tive than their JNR coun­ter­part.

By the early 1980s, only seven out of 200 JNR lines made a profit. Successive gov­ern­ments de­ferred se­ri­ous re­form, run­ning up debt, cut­ting down in­vest­ments in new ur­ban lines, rais­ing ticket prices to twice those of com­pa­ra­ble pri­vate rail­ways, and in­creas­ing sub­si­dies — which rose un­til an­nual sub­si­dies equaled the to­tal cost of the Shinkansen.

In 1982, Prime Minister Yasuhiro Nakasone started to pri­va­tize the rail­ways. Unlike other coun­tries, Japan sim­ply re­turned to the tra­di­tional pri­vate rail­way model of the nine­teenth and early twen­ti­eth cen­turies: tracks, trains, sta­tions, and yards were owned by ver­ti­cally in­te­grated re­gional con­glom­er­ates.

There are sub­stan­tial ad­van­tages to ver­ti­cal in­te­gra­tion. Railways are a closed sys­tem that has to be planned as a sin­gle unit. Changing the timetable at sta­tion A can af­fect the timetable at sta­tion Z; buy­ing new trains that can travel faster might re­quire changes to the in­fra­struc­ture so they can reach their top speed, which in turn re­quires rewrit­ing the timeta­bles. This be­comes es­pe­cially com­pli­cated if dif­fer­ent ser­vices share tracks. To pre­vent de­lays from prop­a­gat­ing from one ser­vice to an­other, the timetable needs to be care­fully de­signed to make best use of the avail­able in­fra­struc­ture.

The stark­est ef­fect of pri­va­ti­za­tion was a mas­sive and im­me­di­ate in­crease in la­bor pro­duc­tiv­ity and prof­itabil­ity rel­a­tive to the legacy pri­vate rail­ways. In fact, this be­gan be­fore pri­va­ti­za­tion: its mere threat strength­ened the gov­ern­men­t’s hand when bar­gain­ing with the unions and forced JNR to be­gin clos­ing rural lines.

Privatization saw a gen­eral trend of pro­duc­tiv­ity im­prove­ments, fol­low­ing a big one-time im­prove­ment be­tween 1982 and 1990, when the work­force was cut by more than half, 83 loss-mak­ing lines were re­moved, and JNRs debts were trans­ferred to a hold­ing com­pany.

The sec­ond great ad­van­tage of pri­va­ti­za­tion was to al­low the JR com­pa­nies to em­u­late the rail­way-as-city-builder model of the legacy pri­vate rail­ways: for in­stance, JR East owns two shop­ping cen­ter brands, a ski re­sort, a cof­fee chain, and even a vend­ing ma­chine drink com­pany. The JR com­pa­nies have not ig­nored their rail busi­ness: they have con­tin­ued to build new high-speed lines and ur­ban tun­nels, up­grade sta­tions, and im­ple­ment a host of other im­prove­ments such as the in­tro­duc­tion in the 1990s of smart cards that al­low pas­sen­gers to pay their fare with a tap.

This does not mean that the Japanese rail­way in­dus­try is a pure crea­ture of free en­ter­prise. No rail­way sys­tem ever has been. The Japanese sys­tem has found an equi­lib­rium that makes rail pol­icy ex­plicit and lim­ited. Leaving aside rail­way safety and busi­ness reg­u­la­tion, there are two main pol­icy levers: fare max­i­mums and cap­i­tal ex­pan­sion sub­si­dies.

Price con­trols are of­ten cited as a clas­sic ex­am­ple of mis­guided gov­ern­ment in­ter­ven­tion, whether through rent con­trols, caps on the price of gaso­line, wage freezes, or min­i­mum agri­cul­tural prices. Tokyo’s in­fa­mously crammed trains are a symp­tom of un­der­priced rush hour traf­fic.

Railways have mar­ket power be­cause the sub­sti­tutes for rail­way trips – coaches, cars and planes – are quite a dif­fer­ent prod­uct. This mo­nop­o­lis­tic po­si­tion has his­tor­i­cally meant trou­ble: mo­nop­oly sys­tems, whether pri­vate or pub­lic, have a ten­dency to abuse their po­si­tion to charge higher prices and run bad ser­vices. For this rea­son, the pri­vate mo­nop­o­lies that were com­mon in the Western world be­fore World War I of­ten had price con­trols im­posed on them. For ex­am­ple, most of the American street­car net­works were op­er­ated as long-term, price-con­trolled fran­chises granted by the city.

Price max­i­mums, if set too low, could have ru­ined Japan’s rail­ways. This is ex­actly what hap­pened to many Western tran­sit ser­vices af­ter the First World War. But the post­war Japanese prac­tice has capped fares gen­er­ously. The sys­tem is ex­plic­itly de­signed to main­tain prof­itabil­ity per rider, which in turn in­cen­tivizes the com­pa­nies to max­i­mize rid­er­ship. That buys po­lit­i­cal le­git­i­macy for the pri­va­tized sys­tem, which is nec­es­sary for the con­tin­ued pro­vi­sion of cap­i­tal ex­pan­sion sub­si­dies. Indeed, dur­ing the long de­fla­tion era be­tween 1992 and 2022, it was com­mon for op­er­a­tors to charge be­low the max­i­mum, and the real value of rail­way fares con­tin­ued to rise. Fare max­i­mums are set on the ba­sis of the av­er­age cost struc­tures of all rail­way op­er­a­tors in a re­gion, so com­pa­nies with be­low-av­er­age costs like Tokyu would of­ten charge be­low the cap to main­tain a com­pet­i­tive edge, pre­vent pub­lic back­lash, and max­i­mize traf­fic to their side-busi­nesses.

Other than the fare max­i­mums, the rail­ways are free to make their own de­ci­sions about timeta­bles, ser­vice pat­terns and day-to-day op­er­a­tions, a highly spe­cial­ized and tech­ni­cal task which re­quires deep ex­per­tise. This con­trasts with the gov­ern­ment med­dling with, say, Amtrak’s routes.

Carefully de­signed pub­lic sub­si­dies also play a use­ful role. Although Japanese rail­ways do not re­ceive sub­si­dies for day-to-day op­er­a­tions, they do re­ceive gov­ern­ment loans and grants for cap­i­tal in­vest­ments. These are typ­i­cally tied to pub­lic pri­or­i­ties, such as dis­abil­ity ac­cess or earth­quake-proof­ing, or to pro­jects that have large spillovers that the rail­way com­pany would be un­able to in­ter­nal­ize, like re­mov­ing level cross­ings, or el­e­vat­ing at-grade rail­ways or trams in or­der to re­duce road con­ges­tion and ac­ci­dent risk. Generally, the lo­cal pre­fec­tural gov­ern­ment will match the con­tri­bu­tion of the na­tional gov­ern­ment. Larger new build pro­jects are sub­ject to lease back or debt-pay­ment con­di­tions that fare rev­enue is ex­pected to pay back.

Railway com­pa­nies in­vested heav­ily in real es­tate busi­nesses, of­ten fund­ing lines through sell­ing land for hous­ing around new sta­tions. Liberal spa­tial pol­icy meant that such de­vel­op­ment hap­pened eas­ily, even as it en­abled dense de­vel­op­ment in ur­ban cores where ra­dial rail lines con­verged. Rail com­pa­nies were gen­er­ally ver­ti­cally in­te­grated re­gional mo­nop­o­lies, own­ing the land, track, and rolling stock, set­ting their own timeta­bles, and em­ploy­ing their staff. The state im­posed con­trols to stop them ex­ploit­ing their mo­nop­oly po­si­tion, but it did so cau­tiously, al­low­ing them to make suf­fi­cient profit that in­cen­tives to in­vest were pre­served. Capital sub­si­dies were tar­geted at pro­vid­ing spe­cific pub­lic goods that nor­mal com­mer­cial op­er­a­tions over­looked.

The above para­graph could be writ­ten by a his­to­rian of the fu­ture about con­tem­po­rary Japan. But every word in it could also be writ­ten by a his­to­rian to­day about the United States in the nine­teenth cen­tury — usu­ally seen as the epit­ome of cap­i­tal­ist in­di­vid­u­al­ism. This strik­ing fact con­tra­dicts the idea that America’s sup­posed in­di­vid­u­al­ism fore­or­dains it to be the land of the car, or that Japan’s sup­posed com­mu­ni­tar­i­an­ism fore­or­dained it to be the land of rail.

It also puts pres­sure on the idea that the demise of rail is the in­evitable con­se­quence of cars. All coun­tries saw some shift to cars in the twen­ti­eth cen­tury, and all rail in­dus­tries had to re­spond to that. But pub­lic pol­icy had an enor­mous ef­fect on how suc­cess­fully they did so. The rise of zon­ing re­stric­tions on den­sity, ex­ces­sive price con­trols, na­tion­al­iza­tion, and ver­ti­cally dis­in­te­grated pri­va­ti­za­tion have ham­pered Western rail in re­main­ing com­pet­i­tive against cars since the 1920s. By main­tain­ing and restor­ing the in­sti­tu­tions that built the first rail­way sys­tems in the nine­teenth cen­tury, the Japanese have cre­ated the might­i­est rail­way sys­tem of the twenty-first.

...

Read the original on worksinprogress.co »

9 227 shares, 8 trendiness

Slop Cop

...

Read the original on awnist.com »

10 208 shares, 15 trendiness

Category Theory Illustrated

Given a set of ob­jects, there can be nu­mer­ous cri­te­ria, based on which to or­der them (depending on the ob­jects them­selves) — size, weight, age, al­pha­bet­i­cal or­der etc.

However, cur­rently we are not in­ter­ested in the cri­te­ria that we can use to or­der ob­jects, but in the na­ture of the re­la­tion­ships that de­fine or­der. Of which there can be sev­eral types as well.

Mathematically, the or­der as a con­struct is rep­re­sented (much like a monoid) by two com­po­nents.

An or­der is a set of el­e­ments, to­gether with a bi­nary re­la­tion be­tween the el­e­ments of the set, which obeys cer­tain laws.

We de­note the el­e­ments of our set, as usual, like this.

And the bi­nary re­la­tion is a re­la­tion be­tween two el­e­ments, which is of­ten de­noted with an ar­row.

As for the laws, they are dif­fer­ent de­pend­ing on the type of or­der.

Let’s start with an ex­am­ple — the most straight­for­ward type of or­der that you think of is lin­ear or­der i.e. one in which every ob­ject has its place de­pend­ing on every other ob­ject. In this case the or­der­ing cri­te­ria is com­pletely de­ter­min­is­tic and leaves no room for am­bi­gu­ity in terms of which el­e­ment comes be­fore which. For ex­am­ple, or­der of col­ors, sorted by the length of their light-waves (or by how they ap­pear in the rain­bow).

Using set the­ory, we can rep­re­sent this or­der, as well as any other or­der, as a sets of pairs of the or­der’s un­der­ly­ing set with it­self (a sub­set of the prod­uct set).

And in pro­gram­ming, or­ders are de­fined by pro­vid­ing a func­tion which, given two ob­jects, tells us which one of them is bigger” (comes first) and which one is smaller”. It is­n’t hard to see that this func­tion de­fines a set of pairs (we are given a pair and we have to say whether or not it be­longs to the set).

However (this is where it gets in­ter­est­ing) not all such func­tions (and not all sets of pairs) de­fine or­ders. For such func­tion to re­ally de­fine an or­der i.e. to have the same out­put every time, in­de­pen­dent of how the ob­jects were shuf­fled ini­tially, it has to obey sev­eral rules.

Incidentally, (or rather not in­ci­den­tally at all), these rules are nearly equiv­a­lent to the math­e­mat­i­cal laws that de­fine the cri­te­ria of the or­der re­la­tion­ship i.e. those are the rules that de­fine which el­e­ment can point to which.

A lin­ear or­der is a set of el­e­ments, to­gether with a bi­nary re­la­tion be­tween the el­e­ments of the set, which obeys the laws of re­flex­iv­ity, tran­si­tiv­ity, an­ti­sym­me­try, to­tal­ity.

Let’s check what they are.

Let’s get the bor­ing law out of the way — each ob­ject has to be big­ger or equal to it­self, or $a ≤ a$ for all $a$ (the re­la­tion­ship be­tween el­e­ments in an or­der is com­monly de­noted as $≤$ in for­mu­las, but it can also be rep­re­sented with an ar­row from first ob­ject to the sec­ond.)

This law only ex­ist to cover the base case”: we can for­mu­late it the op­po­site way too and say that each ob­ject should not have the re­la­tion­ship to it­self, in which case we would have a re­la­tion than re­sem­bles big­ger than, as op­posed to big­ger or equal to and a slightly dif­fer­ent type of or­der, some­times called a strict or­der.

The sec­ond law is maybe the least ob­vi­ous, (but prob­a­bly the most es­sen­tial) — it states that if ob­ject $a$ is big­ger than ob­ject $b$, it is au­to­mat­i­cally big­ger than all ob­jects that are smaller than $b$ or $a ≤ b \land b ≤ c \to a ≤ c$.

This is the law that to a large ex­tend de­fines what an or­der is: if I am bet­ter at play­ing soc­cer than my grand­mother, then I would also be bet­ter at it than my grand­moth­er’s friend, whom she beats, oth­er­wise I would­n’t re­ally be bet­ter than her.

The third law is called an­ti­sym­me­try. It states that the func­tion that de­fines the or­der should not give con­tra­dic­tory re­sults (or in other words you have $x ≤ y$ and $y ≤ x$ only if $x = y$).

It also means that no ties are per­mit­ted — ei­ther I am bet­ter than my grand­mother at soc­cer or she is bet­ter at it than me.

The last law is called to­tal­ity (or con­nex­ity) and it man­dates that all el­e­ments that be­long to the or­der should be com­pa­ra­ble ($a ≤ b \lor b ≤ a$). That is, for any two el­e­ments, one would al­ways be bigger” than the other.

By the way, the law of to­tal­ity makes the re­flex­iv­ity law re­dun­dant, as re­flex­iv­ity is just a spe­cial case of to­tal­ity when $a$ and $b$ are one and the same ob­ject, but I still want to pre­sent it for rea­sons that will be­come ap­par­ent soon.

Actually, here are the rea­sons: the law of to­tal­ity can be re­moved. Orders, that don’t fol­low the to­tal­ity law are called par­tial or­ders, (and lin­ear or­ders are also called to­tal or­ders.)

Task 1: Previously, we cov­ered a re­la­tion that is pretty sim­i­lar to this. Do you re­mem­ber it? What is the dif­fer­ence?

Task 2: Think about some or­ders that you know about and fig­ure out whether they are par­tial or to­tal.

Partial or­ders are ac­tu­ally much more in­ter­est­ing than lin­ear/​to­tal or­ders. But be­fore we dive into them, let’s say a few things about num­bers.

Natural num­bers form a lin­ear or­der un­der the op­er­a­tion big­ger or equal to (the sym­bol of which we have been us­ing in our for­mu­las.)

In many ways, nat­ural num­bers are the quin­tes­sen­tial or­der — every fi­nite or­der of ob­jects is iso­mor­phic to a sub­set of the or­der of num­bers, as we can map the first el­e­ment of any or­der to the num­ber $1$, the sec­ond one to the num­ber $2$ etc (and we can do the op­po­site op­er­a­tion as well).

If we think about it, this iso­mor­phism is ac­tu­ally closer to the every­day no­tion of a lin­ear or­der, than the one de­fined by the laws — when most peo­ple think of or­der, they aren’t think­ing of a tran­si­tive, an­ti­sym­met­ric and to­tal re­la­tion, but are rather think­ing about cri­te­ria based on which they can de­cide which ob­ject comes first, which comes sec­ond etc. So it’s im­por­tant to no­tice that the two no­tions are equiv­a­lent.

From the fact that any fi­nite or­der of ob­jects is iso­mor­phic to the nat­ural num­bers, it also fol­lows that all lin­ear or­ders of the same mag­ni­tude are iso­mor­phic to one an­other.

So, the lin­ear or­der is sim­ple, but it is also (and I think that this iso­mor­phism proves it) the most bor­ing or­der ever, es­pe­cially when looked from a cat­e­gory-the­o­retic view­point — all fi­nite lin­ear or­ders (and most in­fi­nite ones) are just iso­mor­phic to the nat­ural num­bers and so all of their di­a­grams look the same way.

However, this is not the case with par­tial or­ders that we will look into next.

Law of to­tal­ity does not look so set in stone” as the rest of the laws i.e. we can prob­a­bly think of some sit­u­a­tions in which it does not ap­ply. For ex­am­ple, if we aim to or­der all peo­ple based on soc­cer skills there are many ways in which we can rank a per­son com­pared to their friends their friend’s friends etc. but there is­n’t a way to or­der groups of peo­ple who never played with one an­other.

Remove the law of to­tal­ity from the laws of lin­ear or­ders and we get a par­tial or­der (also a par­tially-or­dered set, or poset).

An par­tial or­der is a set of el­e­ments, to­gether with a bi­nary re­la­tion be­tween the el­e­ments of the set, which obeys the laws of re­flex­iv­ity, tran­si­tiv­ity and an­ti­sym­me­try.

Every lin­ear or­der is also a par­tial or­der (just as a group is still a monoid), but not the other way around.

We can even cre­ate an or­der of or­ders, based on which is more gen­eral.

Partial or­ders are also re­lated to the con­cept of an equiv­a­lence re­la­tions that we cov­ered in chap­ter 1, ex­cept that sym­me­try law is re­placed with an­ti­sym­me­try.

If we re­visit the ex­am­ple of the soc­cer play­ers rank list, we can see that the first ver­sion that in­cludes just my­self, my grand­mother and her friend is a lin­ear or­der.

However, in­clud­ing this other per­son whom none of us played yet, makes the hi­er­ar­chy non-lin­ear i.e. a par­tial or­der.

This is the main dif­fer­ence be­tween par­tial and to­tal or­ders — par­tial or­ders can­not pro­vide us with a def­i­nite an­swer of the ques­tion who is bet­ter than who. But some­times this is what we need — in sports, as well as in other do­mains, there is­n’t al­ways an ap­pro­pri­ate way to rate el­e­ments lin­early.

Before, we said that all lin­ear or­ders can be rep­re­sented by the same chain-like di­a­gram, we can re­verse this state­ment and say that all di­a­grams that look some­thing dif­fer­ent than the said di­a­gram rep­re­sent par­tial or­ders.

An ex­am­ple of this is a par­tial or­der that con­tains a bunch of lin­early-or­dered sub­sets, e.g. in our soc­cer ex­am­ple we can have sep­a­rate groups of friends who play to­gether and are ranked with each other, but not with any­one from other groups.

The dif­fer­ent lin­ear or­ders that make up the par­tial or­der are called chains. There are two chains in this di­a­gram $m \to g \to f$ and $d \to o$.

The chains in an or­der don’t have to be com­pletely dis­con­nected from each other in or­der for it to be par­tial. They can be con­nected as long as the con­nec­tions are not all one-to-one i.e. ones when the last el­e­ment from one chain is con­nected to the first el­e­ment of the other one (this would ef­fec­tively unite them into one chain.)

The above set is not lin­early-or­dered — al­though we know that $d ≤ g$ and that $f ≤ g$, the re­la­tion­ship be­tween $d$ and $f$ is not known — any el­e­ment can be big­ger than the other one.

Although par­tial or­ders don’t give us a de­fin­i­tive an­swer to Who is bet­ter than who?”, some of them still can give us an an­swer to the more im­por­tant ques­tion (in sports, as well as in other do­mains), namely Who is num­ber one?” i.e. who is the cham­pion, the player who is bet­ter than any­one else. Or, more gen­er­ally, the el­e­ment that is big­ger than all other el­e­ments.

The great­est el­e­ment of an or­der is an el­e­ment $a$, such that we have we have $x ≤ a$ for any other el­e­ment $x$, Some (not all) par­tial or­ders do have such el­e­ment — in our last di­a­gram $m$ is the great­est el­e­ment, in this di­a­gram, the green el­e­ment is the biggest one.

Sometimes we have more than one el­e­ments that are big­ger than all other el­e­ments, in this case none of them is the great­est.

In ad­di­tion to the great­est el­e­ment, a par­tial or­der may also have a least (smallest) el­e­ment, which is de­fined in the same way.

The least up­per bound of two el­e­ments that are con­nected as part of an or­der is called the join of these el­e­ments, e.g. the green el­e­ment is a join of the other two.

The join of $a$ and $b$ is the small­est el­e­ment $c$ that is big­ger than then, for­mally:

The join of ob­jects $A$ and $B$ is an ob­ject $G$, such that:

It is big­ger than both of these ob­jects, so $A ≤ G$ and $B ≤ G$.

It is smaller than any other ob­ject that is big­ger than them, so for any other ob­ject $P$ such that $P ≤ A$ and $P ≤ B$ then we should also have $G ≤ P$.

Given any two el­e­ments in which one is big­ger than the other (e.g. $a ≤ b$), the join is this big­ger el­e­ment (in this case $b$)

e.g. in a lin­ear or­ders, the join of any two el­e­ments is just the big­ger el­e­ment.

Like with the great­est el­e­ment, if two el­e­ments have sev­eral up­per bounds that are equally big, then none of them is a join (a join must be unique).

If, how­ever, one of those el­e­ments is es­tab­lished as smaller than the rest of them, it im­me­di­ately qual­i­fies.

Task 3: Which con­cept in cat­e­gory the­ory re­minds you of joins?

Given two el­e­ments, the biggest el­e­ment that is smaller than both of them is called the meet of these el­e­ments.

The same rules as for the joins ap­ply, but in re­verse.

The di­a­grams that we use in this sec­tion are called Hasse di­a­grams” and they work much like our usual di­a­grams, how­ever they have an ad­di­tional rule that is fol­lowed — bigger” el­e­ments are al­ways po­si­tioned above smaller ones.

In terms of ar­rows, the rule means that if you add an ar­row to a point, the point to which the ar­row points must al­ways be above the one from which it points.

This arrange­ment al­lows us to com­pare any two points by just see­ing which one is above the other e.g. we can de­ter­mine the join of two el­e­ments, by just iden­ti­fy­ing the el­e­ments that they con­nect to and see which one is low­est.

We all know many ex­am­ples of to­tal or­ders (any form of chart or rank­ing is a to­tal or­der), but there are prob­a­bly not so many ob­vi­ous ex­am­ples of par­tial or­ders that we can think of off the top of our head. So let’s see some. This will gives us some con­text, and will help us un­der­stand what joins are.

To stay true to our form, let’s re­visit our color-mix­ing monoid and cre­ate a color-mix­ing par­tial or­der in which all col­ors point to col­ors that con­tain them.

If you go through it, you will no­tice that the join of any two col­ors is the color that they make up when mixed. Nice, right?

The par­tial or­der of num­bers by di­vi­sion

We saw that when we or­der num­bers by bigger or equal to”, they form a lin­ear or­der. But num­bers can also form a par­tial or­der, for ex­am­ple they form a par­tial or­der if we or­der them by which di­vides which, i.e. if $a$ di­vides $b$, then $a$ is be­fore $b$ e.g. be­cause $2 \times 5 = 10$, $2$ and $5$ come be­fore $10$ (but $3$, for ex­am­ple, does not come be­fore $10$.)

And it so hap­pens (actually for very good rea­son) that the join op­er­a­tion again cor­re­sponds to an op­er­a­tion that is rel­e­vant in the con­text of the ob­jects — the join of two num­bers in this par­tial or­der is their least com­mon mul­ti­ple.

And the meet (the op­po­site of join) of two num­bers is their great­est com­mon di­vi­sor.

Given a col­lec­tion of sets con­tain­ing a com­bi­na­tion of a given set of el­e­ments…

…we can de­fine what is called the in­clu­sion or­der of those sets.

The in­clu­sion or­der of sets is a bi­nary re­la­tion that we can use to or­der a col­lec­tion of sets (usually sets that con­tain some com­mon el­e­ments) in which $a$ comes be­fore $b$ if $a$ in­cludes $b$, or in other words if $b$ is a sub­set of $a$.

In this case the join op­er­a­tion of two sets is their union, and the meet op­er­a­tion is their set in­ter­sec­tion.

This di­a­gram might re­mind you of some­thing — if we take the col­ors that are con­tained in each set and mix them into one color, we get the color-blend­ing par­tial or­der that we saw ear­lier.

The or­der ex­am­ple with the num­ber di­viders is also iso­mor­phic to an in­clu­sion or­der, namely the in­clu­sion or­der of all pos­si­ble sets of prime num­bers, in­clud­ing re­peat­ing ones (or al­ter­na­tively the set of all prime pow­ers). This is con­firmed by the fun­da­men­tal the­ory of arith­metic, which states that every num­ber can be writ­ten as a prod­uct of primes in ex­actly one way.

So far, we saw two dif­fer­ent par­tial or­ders, one based on color mix­ing, and one based on num­ber di­vi­sion, that can be rep­re­sented by the in­clu­sion or­ders of all pos­si­ble com­bi­na­tions of sets of some ba­sic el­e­ments (the pri­mary col­ors in the first case, and the prime num­bers (or prime pow­ers) in the sec­ond one.) Many other par­tial or­ders can be de­fined in this way. Which ones ex­actly, is a ques­tion that is an­swered by an amaz­ing re­sult called Birkhoff’s rep­re­sen­ta­tion the­o­rem. They are the fi­nite par­tial or­ders that meet the fol­low­ing two cri­te­ria:

All el­e­ments have joins and meets.

Those meet and join op­er­a­tions dis­trib­ute over one an­other, that is if we de­note joins as meets as $∨$ or $∧$, then $x ∨ (y ∧ z) = (x ∨ y) ∧ (x ∨ z)$.

The par­tial or­ders that meet the first cri­te­ria are called lat­tices. The ones that meet the sec­ond one are called dis­trib­u­tive lat­tices. Let’s write that down:

Partial or­ders in which all el­e­ments have joins and meets is called a lat­tice. A lat­tice whose meet and join op­er­a­tions dis­trib­ute over one an­other is called a dis­trib­u­tive lat­tice.

And the prime” el­e­ments which we use to con­struct the in­clu­sion or­der are the el­e­ments that are not the join of any other el­e­ments. They are also called join-ir­re­ducible el­e­ments.

So we may phrase the the­o­rem like this:

Each dis­trib­u­tive lat­tice is iso­mor­phic to an in­clu­sion or­der of its join-ir­re­ducible el­e­ments.

By the way, the par­tial or­ders that are not dis­trib­u­tive lat­tices are also iso­mor­phic to in­clu­sion or­ders, it is just that they are iso­mor­phic to in­clu­sion or­ders that do not con­tain all pos­si­ble com­bi­na­tions of el­e­ments.

We will now talk more about lat­tices (the or­ders for which Birkhoff’s the­o­rem ap­plies). Lattices are par­tial or­ders, in which every two el­e­ments have a join and a meet. So every lat­tice is also par­tial or­der, but not every par­tial or­der is a lat­tice (we will see even more mem­bers of this hi­er­ar­chy).

Most par­tial or­ders that are cre­ated based on some sort of rule are dis­trib­u­tive lat­tices, like for ex­am­ple the par­tial or­ders from the pre­vi­ous sec­tion are also dis­trib­u­tive lat­tices when they are drawn in full, for ex­am­ple the color-mix­ing or­der.

Notice that we added the black ball at the top and the white one at the bot­tom. We did that be­cause oth­er­wise the top three el­e­ments would­n’t have a join el­e­ment, and the bot­tom three would­n’t have a meet.

Our color-mix­ing lat­tice, has a great­est el­e­ment (the black ball) and a least el­e­ment (the white one). Lattices that have a least and great­est el­e­ments are called bounded lat­tices. It is­n’t hard to see that all fi­nite lat­tices are also bounded.

Task 4: Prove that all fi­nite lat­tices are bounded.

We men­tioned or­der iso­mor­phisms sev­eral times al­ready so this is about time to elab­o­rate on what they are.

Given two sets (we will use par­tial or­der of num­bers by di­vi­sion and the prime in­clu­sion or­der as an ex­am­ple) an iso­mor­phism be­tween them is com­prised of the fol­low­ing two func­tions:

One func­tion from the prime in­clu­sion or­der, to the num­ber or­der (which in this case is just the mul­ti­pli­ca­tion of all the el­e­ments in the set)

One func­tion from the num­ber or­der to the prime in­clu­sion or­der (which is an op­er­a­tion called prime fac­tor­iza­tion of a num­ber, con­sist­ing of find­ing the set of prime num­bers that re­sult in that num­ber when mul­ti­plied with one an­other).

An or­der iso­mor­phism is es­sen­tially an iso­mor­phism be­tween the or­ders’ un­der­ly­ing sets (invertible func­tion). However, be­sides their un­der­ly­ing sets, or­ders also have the ar­rows that con­nect them, so there is one more con­di­tion: in or­der for an in­vert­ible func­tion to con­sti­tute an or­der iso­mor­phism, it has to re­spect those ar­rows.

An iso­mor­phism be­tween two or­ders is an in­vert­ible func­tion be­tween their un­der­ly­ing sets, such that ap­ply­ing this func­tion (let’s call it $F$) to any two el­e­ments that have a cer­tain or­der in one set (let’s call them $a$ and $b$) should re­sult in two el­e­ments that have a cor­re­spond­ing or­der in the other set (i.e. $a ≤ b$ if and only if $F(a) ≤ F(b)$).

In the pre­vi­ous sec­tion, we saw how re­mov­ing the law of to­tal­ity from the laws of (linear) or­der pro­duces a dif­fer­ent (and some­what more in­ter­est­ing) struc­ture, called par­tial or­der. Now let’s see what will hap­pen if we re­move an­other one of the laws, namely the an­ti­sym­me­try law.

The an­ti­sym­me­try law man­dated that you can­not have an ob­ject that is at the same time smaller and big­ger than an­other one. (or that $a ≤ b ⟺ b ≰ a$).

...

Read the original on abuseofnotation.github.io »

To add this web app to your iOS home screen tap the share button and select "Add to the Home Screen".

10HN is also available as an iOS App

If you visit 10HN only rarely, check out the the best articles from the past week.

If you like 10HN please leave feedback and share

Visit pancik.com for more.