10 interesting stories served every morning and every evening.

Meta blocks human rights accounts from reaching audiences in Saudi Arabia and the UAE

www.alqst.org

The un­der­signed or­gan­i­sa­tions con­demn Meta’s re­cent de­ci­sion to re­strict the Facebook and Instagram ac­counts of in­de­pen­dent NGOs, re­searchers, and civil so­ci­ety fig­ures from reach­ing au­di­ences in Saudi Arabia and the United Arab Emirates (UAE). This is the lat­est in a pat­tern of ma­jor tech­nol­ogy com­pa­nies, in­clud­ing Meta, act­ing as en­force­ment arms for re­pres­sive gov­ern­ments in the Gulf. We call on Meta to up­hold its hu­man rights re­spon­si­bil­i­ties and en­sure that users’ con­tent is not ar­bi­trar­ily re­stricted.

Since 30 April 2026, Gulf-focused NGOs ALQST for Human Rights and Democratic Diwan, as well as Saudi re­searcher Abdullah Alaoudh and hu­man rights de­fender Yahya Assiri, have had their Facebook ac­counts ren­dered unavailable” in Saudi Arabia at the re­quest of the Saudi gov­ern­ment, in a form of geo-blocking”. Similar re­stric­tions have been im­posed in the UAE, in­clud­ing against an aca­d­e­mic. According to Meta’s pub­licly avail­able con­tent re­stric­tion re­ports, over 100 Facebook pages and Instagram ac­counts have been re­stricted since March 2026. This fol­lows a sim­i­lar pat­tern on X (formerly Twitter). Most re­cently, the Saudi gov­ern­ment re­quested that a num­ber of X ac­counts be­long­ing to promi­nent Saudi ac­tivists be geo-blocked. As of the pub­li­ca­tion date (20 May), X had not com­plied.

The un­der­signed or­gan­i­sa­tions con­sider these mea­sures ar­bi­trary, dis­crim­i­na­tory, and a di­rect vi­o­la­tion of the right to free­dom of ex­pres­sion and ac­cess to in­for­ma­tion. Affected users were no­ti­fied that Meta acted in re­sponse to a local le­gal re­quire­ment” or a request from a gov­ern­ment” (i.e. Saudi Arabia and the UAE), demon­strat­ing the com­pa­ny’s will­ing­ness to com­ply with de­mands from state au­thor­i­ties that rou­tinely sup­press, sur­veil, and crim­i­nalise on­line ex­pres­sion.

Meta’s no­ti­fi­ca­tions cite com­pli­ance with local laws”, and its re­ports spec­ify the cy­ber­crime laws of Saudi Arabia and the UAE. Authorities in Saudi Arabia and the UAE have long re­lied on dra­con­ian cy­ber­crime and coun­tert­er­ror­ism leg­is­la­tion to si­lence dis­sent and re­strict free­dom of ex­pres­sion on­line. Count­less ac­tivists and peace­ful crit­ics have been ar­rested, tried and sen­tenced for ex­press­ing crit­i­cal opin­ions in on­line pub­li­ca­tions or on so­cial me­dia, in­clud­ing Facebook and X.

The reports further cite that the con­tent al­legedly vi­o­lat­ing cy­ber­crime laws in­cludes reporting on re­gional geopo­lit­i­cal con­flicts and se­cu­rity de­vel­op­ments.” Since the United States and Israel launched strikes on Iran on 28 February 2026, Gulf gov­ern­ments have moved swiftly to tighten the in­for­ma­tion en­vi­ron­ment even fur­ther, con­trol­ling what their pop­u­la­tions can see, say, and share about the at­tacks un­fold­ing in their coun­tries.

Meta claims to con­duct hu­man rights due dili­gence re­views be­fore com­ply­ing with gov­ern­ment re­quests. We ask Meta to dis­close what that re­view looked like for ALQSTs page (and other af­fected ac­counts), who con­ducted it, what stan­dards were ap­plied, and how the com­pany con­cluded that re­strict­ing a hu­man rights or­gan­i­sa­tion at the re­quest of a gov­ern­ment that im­pris­ons peo­ple for so­cial me­dia posts is com­pat­i­ble with its own stated com­mit­ments.

Meanwhile, au­thor­i­ties in both coun­tries ex­ert near to­tal con­trol over on­line in­for­ma­tion, and rou­tinely block web­sites and in­ter­net ac­counts that might be of pub­lic in­ter­est, such as the web­sites of ALQST (blocked in Saudi Arabia since 2015), the Gulf Centre for Human Rights (blocked in Saudi Arabia and the UAE since 2015) and oth­ers that might con­tain con­tent crit­i­cal of the gov­ern­ment or that ad­vo­cate for hu­man rights and democ­racy in the coun­try. In this highly re­pres­sive con­text, Meta has a height­ened re­spon­si­bil­ity to up­hold free­dom of ex­pres­sion and pro­tect hu­man rights de­fend­ers, es­pe­cially from censorship de­mands from gov­ern­ments or their prox­ies” as Meta pub­licly states in its hu­man rights pol­icy. Instead of pre­serv­ing the free flow of in­for­ma­tion, Meta has fur­ther re­stricted ac­cess to it.

Ironically, Meta’s no­ti­fi­ca­tions state that the com­pany con­ducted le­gal as­sess­ments be­fore­hand and took into ac­count hu­man rights im­pli­ca­tions.” Given the well-doc­u­mented pat­tern of dig­i­tal re­pres­sion by Saudi and UAE au­thor­i­ties, such claims are dif­fi­cult to rec­on­cile. The UN Guiding Principles on Business and Human Rights ex­pect com­pa­nies to as­sess whether gov­ern­ment re­quests are con­sis­tent with in­ter­na­tional hu­man rights stan­dards be­fore com­ply­ing and to be trans­par­ent about how they reached their con­clu­sions. The un­der­signed or­gan­i­sa­tions there­fore call on Meta to:

Publish the full le­gal re­quests re­ceived from Saudi and UAE au­thor­i­ties along with the hu­man rights as­sess­ments Meta claims to have con­ducted;

Restore full ac­cess to all af­fected ac­counts im­me­di­ately;

Commit to no­ti­fy­ing af­fected users with spe­cific de­tails about which con­tent trig­gered the re­stric­tion and un­der which law; and

Explain what role, if any, Meta’s re­gional of­fices in the Gulf played in pro­cess­ing these re­quests.

Signatories:

Access Now

ALQST for Human Rights

American Committee for Middle East Rights (ACMER)

DAWN

De|Center

Digital Action

Electronic Frontier Foundation

Gulf Centre for Human Rights (GCHR)

HuMENA for Human Rights and Civic Engagement

MENA Rights Group

Skyline International for Human Rights (SIHR)

SMEX

Adieu Visa et Mastercard : 130 millions d'Européens basculent vers un paiement 100 % souverain dès 2026

www.lesnumeriques.com

En ce mo­ment :

Publicité, votre con­tenu con­tinue ci-dessous

Publicité

L’Europe rè­gle ses comptes

Publié le 15/05/26 à 19h35

Nos réseaux :

Suivez-nous

Ajoutez nous à vos fa­voris Google

2

L’Europe lance en­fin sa ri­poste tech­nologique face à l’hégé­monie améri­caine. Cinq géants du paiement mo­bile con­ti­nen­tal vi­en­nent de sceller une al­liance his­torique pour uni­fier leurs réseaux. Dès l’an­née prochaine, les trans­ac­tions quo­ti­di­ennes de mil­lions d’u­til­isa­teurs s’af­franchi­ront des cir­cuits tra­di­tion­nels transat­lan­tiques pour cir­culer sur une in­fra­struc­ture stricte­ment eu­ropéenne et in­dépen­dante.

21

Visa et Mastercard, maîtres in­con­testés du paiement en Europe, font dé­sor­mais face à une of­fen­sive co­or­don­née de cinq cham­pi­ons na­tionaux.

© Olgsera

Le paysage ban­caire eu­ropéen s’ap­prête à vivre un séisme. Des ac­teurs ma­jeurs comme Bizum en Espagne, Bancomat en Italie, MB WAY au Portugal et Vipps MobilePay dans les pays du Nord s’u­nis­sent of­fi­cielle­ment à l’ini­tia­tive française Wero. Cette force de frappe n’a rien de sym­bol­ique car elle s’ap­puie sur une base solide de 130 mil­lions d’u­til­isa­teurs déjà ac­t­ifs. En con­nec­tant ces écosys­tèmes na­tionaux, l’Eu­rope ne se con­tente plus de cri­ti­quer la dom­i­na­tion de Visa et Mastercard, elle con­struit une al­ter­na­tive ca­pa­ble de traiter des mil­liards de trans­ac­tions an­nuelles sans que la moin­dre don­née ne tran­site par les serveurs des États-Unis.

L’interopérabilité to­tale comme clé de la lib­erté numérique

L’Europe dis­pose de l’in­fra­struc­ture, de l’échelle et de la vi­sion néces­saires pour of­frir une al­ter­na­tive eu­ropéenne sou­veraine, ro­buste et fi­able dans le do­maine des paiements.

L’Europe dis­pose de l’in­fra­struc­ture, de l’échelle et de la vi­sion néces­saires pour of­frir une al­ter­na­tive eu­ropéenne sou­veraine, ro­buste et fi­able dans le do­maine des paiements.

Le pro­jet s’ap­puie sur la créa­tion d’un hub cen­tral d’in­teropéra­bil­ité, géré par une en­tité com­mune que les parte­naires établiront au pre­mier se­mes­tre 2026. Cette plate­forme tech­nique per­me­t­tra aux dif­férents sys­tèmes de di­a­loguer en­tre eux, sans que les util­isa­teurs n’aient à changer leurs habi­tudes. Un Français util­isant Wero pourra ainsi trans­férer de l’ar­gent à un ami es­pag­nol sur Bizum, avec la même sim­plic­ité qu’un paiement do­mes­tique.

Les cinq parte­naires de l’al­liance eu­ropéenne des paiements, qui représen­tent déjà 130 mil­lions d’u­til­isa­teurs à tra­vers 13 pays.

© EPI

Un cal­en­drier de dé­ploiement am­bitieux vers une au­tonomie com­plète

Le dé­ploiement se fera pro­gres­sive­ment. Les vire­ments en­tre par­ti­c­uliers seront disponibles dès 2026 sur l’ensem­ble des treize pays cou­verts — d’An­dorre à la Suède. Les paiements en ligne et en ma­g­a­sin suiv­ront en 2027. À terme, cette coali­tion cou­vrira 72 % de la pop­u­la­tion de l’U­nion eu­ropéenne et de la Norvège.

La sou­veraineté eu­ropéenne des paiements n’est pas une vi­sion, mais une réal­ité en de­venir.

La sou­veraineté eu­ropéenne des paiements n’est pas une vi­sion, mais une réal­ité en de­venir.

L’alliance EuroPA, qui con­necte déjà l’Es­pagne, le Portugal, l’I­talie et Andorre depuis mars 2025, sert de pro­to­type. Six mil­lions d’eu­ros y ont tran­sité en un an, sans cam­pagne pro­mo­tion­nelle par­ti­c­ulière. Un sig­nal en­cour­ageant pour cette am­bi­tion d’au­tonomie eu­ropéenne dans un secteur longtemps dom­iné par des géants ex­tra-con­ti­nen­taux — une préoc­cu­pa­tion que Christine Lagarde avait déjà martelée en avril 2025.

Suivez toute l’ac­tu­al­ité des Numériques sur Google Actualités et sur la chaîne WhatsApp des Numériques

Envie de faire en­core plus d’é­conomies ? Découvrez nos codes promo sélec­tion­nés pour vous.

openai.com

VICTORY! Tennessee man jailed 37 days for Trump meme wins $835,000 settlement after First Amendment lawsuit

www.fire.org

FIRE plain­tiff Larry Bushart and his wife Leanne.

NASHVILLE, May 20, 2026 — After spend­ing 37 days in jail for noth­ing more than post­ing a meme, re­tired Tennessee law en­force­ment of­fi­cer Larry Bushart has won a sub­stan­tial set­tle­ment from the county and sher­iff be­hind his ar­rest.

Represented by the Foundation for Individual Rights and Expression and Phillips & Phillips, PLLC, Larry Bushart filed a fed­eral civil rights law­suit last December against Sheriff Nick Weems, Investigator Jason Morrow, and Perry County, Tennessee, for vi­o­lat­ing his con­sti­tu­tional rights in re­tal­i­a­tion for his pro­tected speech.

Today, the par­ties an­nounced in a joint state­ment that Larry will re­ceive $835,000 in ex­change for dis­miss­ing his com­plaint.

I am pleased my First Amendment rights have been vin­di­cated,” said Larry. The peo­ple’s free­dom to par­tic­i­pate in civil dis­course is cru­cial to a healthy democ­racy. I am look­ing for­ward to mov­ing on and spend­ing time with my fam­ily.”

After the September 2025 as­sas­si­na­tion of con­ser­v­a­tive ac­tivist Charlie Kirk, Larry com­mented on a Facebook post pro­mot­ing a vigil in nearby Perry County by shar­ing a meme that ac­cu­rately quoted Donald Trump’s state­ment af­ter a school shoot­ing:  We have to get over it.”

That meme — which Larry did­n’t cre­ate or al­ter — in­cluded a ref­er­ence to the 2024 school shoot­ing at Perry High School in Perry, Iowa. But that did not stop Weems from seek­ing and ob­tain­ing a war­rant for Larry’s ar­rest, based on the ab­surd no­tion that the meme could be in­ter­preted as a threat against Perry County High School in Tennessee. Video of the ar­rest shows Larry in­formed the ar­rest­ing of­fi­cer (not a de­fen­dant in the case) that he had never made a threat.

No one should be hauled off to jail in the dark of night over a harm­less meme just be­cause the au­thor­i­ties dis­agree with its mes­sage,” said FIRE se­nior at­tor­ney Adam Steinbaugh. We’re pleased that Larry has been com­pen­sated for this in­jus­tice, but lo­cal law en­force­ment never should have forced him to en­dure this or­deal in the first place.”

COURTESY PHOTOS OF LARRY FOR MEDIA USE

Weems ad­mit­ted in a later in­ter­view that he knew at the time of the ar­rest that Larry’s Facebook post was a pre-ex­ist­ing meme that re­ferred to an ac­tual shoot­ing that took place in a dif­fer­ent state, over 500 miles away. But Weems and Morrow left out that ex­tremely im­por­tant con­text from their war­rant ap­pli­ca­tion. Not that it should have mat­tered; the Supreme Court has long held that heated po­lit­i­cal rhetoric is fully pro­tected by the First Amendment.

Larry spent over a month be­hind bars on a $2 mil­lion bond. Perry County re­leased him from jail only af­ter his plight went vi­ral na­tion­wide and prompted out­rage. During his stay in jail, Larry lost his post-re­tire­ment job and missed his an­niver­sary — as well as the birth of his grand­child. After his re­lease, he teamed up with FIRE to hold those who vi­o­lated his con­sti­tu­tional rights ac­count­able.

Larry was just one of hun­dreds of Americans cen­sored for on­line speech af­ter Kirk’s as­sas­si­na­tion. Elsewhere in Tennessee, FIRE also rep­re­sents Mon­ica Meeks, a life­long pub­lic ser­vant who the state fired solely for a Facebook post crit­i­ciz­ing Kirk. And ear­lier this year, Austin Peay State University settled a law­suit filed by a pro­fes­sor fired sim­ply for cit­ing Kirk’s own words on gun vi­o­lence.

It’s in times of tur­moil and height­ened ten­sions that our na­tional com­mit­ment to free speech is tested the most,” said FIRE staff at­tor­ney Cary Davis. When gov­ern­ment of­fi­cials fail that test, the Constitution ex­ists to hold them ac­count­able. Our hope is that Larry’s set­tle­ment sends a mes­sage to law en­force­ment across the coun­try: Respect the First Amendment to­day, or be pre­pared to pay the price to­mor­row.”

The Foundation for Individual Rights and Expression (FIRE) is a non­par­ti­san, non­profit or­ga­ni­za­tion ded­i­cated to de­fend­ing and sus­tain­ing the in­di­vid­ual rights of all Americans to free speech and free thought — the most es­sen­tial qual­i­ties of lib­erty. FIRE ed­u­cates Americans about the im­por­tance of these in­alien­able rights, pro­motes a cul­ture of re­spect for these rights, and pro­vides the means to pre­serve them.

CONTACT:Alex Griswold, Communications Campaign Manager, FIRE: 215 – 717-3473; me­dia@fire.org

Qwen Studio

qwen.ai

Railway Service Disruption — Resolved | Railway Status

status.railway.com

Railway Service Disruption

Major Outage

Dashboard (railway.com)Login - GitHubLogin - GoogleDNSTrafficBuild Machines (GCP)Image Registry (GCP)TCP Proxy (GCP)Southeast Asia (GCP / Singapore)US East (GCP / Virginia, USA)US West (GCP / Oregon, USA)EU West (GCP / Amsterdam, Netherlands)EU West (GCP / Amsterdam, Netherlands)US West (GCP / Oregon, USA)US East (GCP / Virginia, USA)Southeast Asia (GCP / Singapore)TCP Proxy (Metal)US East (Metal / Virginia, USA)Southeast Asia (Metal / Singapore)EU West (Metal / Amsterdam, Netherlands)US West (Metal / California, USA)MetricsLogsSoutheast Asia (Metal / Singapore)US East (Metal / Virginia, USA)US West (Metal / California, USA)EU West (Metal / Amsterdam, Netherlands)Build Machines (Metal)Image Registry (Metal)Central Station (station.railway.com)

May 20, 07:57 UTC

Earlier up­dates

Monitoring

Railway ser­vices have fully re­cov­ered. Some work­loads may still need a re­de­ploy, we’re au­to­mat­i­cally re­de­ploy­ing any we de­tect as un­healthy. If your ser­vice is­n’t re­spond­ing cor­rectly, please trig­ger a re­de­ploy from the dash­board or CLI.We’re sorry for the dis­rup­tion. A de­tailed post­mortem will fol­low once we’ve con­firmed sta­bil­ity.

May 20, 06:14 UTC

May 20, 04:58 UTC

Identified

More work­loads are com­ing back on­line. Some users may still ex­pe­ri­ence in­ter­mit­tent is­sues dur­ing the re­cov­ery. Non-enterprise de­ploys re­main paused; en­ter­prise de­ploys are un­af­fected.

May 20, 03:05 UTC

Identified

We are see­ing grad­ual re­cov­ery on Railway metal work­loads. To en­sure things re­main sta­ble as we ramp back up, we are tem­porar­ily throt­tling all non-en­ter­prise builds to avoid over­whelm­ing our build in­fra­struc­ture.

May 20, 01:41 UTC

Identified

We have re­cov­ered our com­pute on Google Cloud, but ser­vices are un­able to start be­cause of on­go­ing net­work­ing is­sues on Google Cloud’s side. We are en­gaged with Google Cloud sup­port to re­solve this and will post the next up­date as soon as we have more to share.

May 20, 01:34 UTC

Identified

We are con­tin­u­ing to work on restor­ing ac­cess to our work­loads on Google Cloud. In par­al­lel, our in­fra­struc­ture team is eval­u­at­ing al­ter­na­tive paths to bring af­fected ser­vices back on­line. We do not have an ETA at this time and will con­tinue post­ing up­dates as we have them.

May 20, 01:23 UTC

Identified

We are work­ing to re­store the Google Cloud in­fra­struc­ture that pow­ers our dash­board, API, and in­ter­nal net­work’s con­trol plane. We are in di­rect con­tact with Google Cloud’s sup­port team. We do not have an ETA at this time.

May 20, 00:37 UTC

Identified

Google Cloud has blocked our ac­count, mak­ing some Railway ser­vices un­avail­able. We have es­ca­lated this di­rectly with Google.The Railway Platform team has since con­firmed ac­cess to Google Cloud and is work­ing on restor­ing ac­cess to all work­loads. We have ac­cess to some of our Google Cloud–hosted in­fra­struc­ture and are work­ing to re­store the rest of the ser­vice. We apol­o­gize for the dis­rup­tion.

May 19, 23:37 UTC

Identified

We have iden­ti­fied the cause of the dis­rup­tion. Access to our up­stream cloud provider has been re­stored and we are work­ing on a fix.The Railway dash­board and ser­vices run­ning on our cloud in­fra­struc­ture may con­tinue to be af­fected un­til the fix is de­ployed.

May 19, 22:43 UTC

Investigating

We are in­ves­ti­gat­ing a wide­spread ser­vice dis­rup­tion af­fect­ing Railway. Users may be ex­pe­ri­enc­ing er­rors in­clud­ing no healthy up­stream”, unconditional drop over­load”, lo­gin fail­ures, and in­abil­ity to ac­cess the dash­board. We are ac­tively in­ves­ti­gat­ing and will pro­vide up­dates as soon as pos­si­ble.

May 19, 22:29 UTC

GitHub confirms breach of 3,800 repos via malicious VSCode extension

www.bleepingcomputer.com

GitHub has con­firmed that roughly 3,800 in­ter­nal repos­i­to­ries were breached af­ter one of its em­ploy­ees in­stalled a ma­li­cious VS Code ex­ten­sion.

The com­pany has since re­moved the un­named tro­janized ex­ten­sion from the VS Code mar­ket­place and has se­cured the com­pro­mised de­vice.

Yesterday we de­tected and con­tained a com­pro­mise of an em­ployee de­vice in­volv­ing a poi­soned VS Code ex­ten­sion. We re­moved the ma­li­cious ex­ten­sion ver­sion, iso­lated the end­point, and be­gan in­ci­dent re­sponse im­me­di­ately,” the com­pany said.

Our cur­rent as­sess­ment is that the ac­tiv­ity in­volved ex­fil­tra­tion of GitHub-internal repos­i­to­ries only. The at­tack­er’s cur­rent claims of ~3,800 repos­i­to­ries are di­rec­tion­ally con­sis­tent with our in­ves­ti­ga­tion so far.”

This comes af­ter GitHub told BleepingComputer on Tuesday evening that it was in­ves­ti­gat­ing claims of unau­tho­rized ac­cess to its in­ter­nal repos­i­to­ries and added that it has no ev­i­dence that cus­tomer data stored out­side the af­fected re­pos has been af­fected.

While GitHub has yet to at­tribute the breach, the TeamPCP hacker group claimed ac­cess to GitHub source code and ~4,000 re­pos of pri­vate code” on the Breached cy­ber­crime fo­rum on Tuesday, ask­ing for at least $50,000 for the stolen data.

As al­ways this is not a ran­som, We do not care about ex­tort­ing Github, 1 buyer and we shred the data on our end, it looks like our re­tire­ment is soon so if no buyer is found we will leak it free,” the cy­ber­crim­i­nals said. If you are in­ter­ested. Send your of­fers to the com­mu­ni­ca­tions be­low, we are not in­ter­ested in un­der 50k, the best of­fer will get it.”

​TeamPCP was pre­vi­ously linked to mas­sive sup­ply chain at­tacks tar­get­ing de­vel­oper code plat­forms, in­clud­ing GitHub, PyPI, NPM, and Docker, and, more re­cently, to the Mini Shai-Hulud” sup­ply chain cam­paign(which also im­pacted two OpenAI em­ploy­ees).

​VS Code ex­ten­sions are plu­g­ins that can be in­stalled from the VS Code Marketplace (the of­fi­cial store for add-ons for Microsoft’s code ed­i­tor) to add fea­tures or in­te­grate tools into the ed­i­tor.

This is­n’t the first time a tro­janized VS Code ex­ten­sion has been spot­ted on the mar­ket­place, as mul­ti­ple other ma­li­cious ex­ten­sions with mil­lions of in­stalls have been used to steal de­vel­oper cre­den­tials and other sen­si­tive data over the last sev­eral years.

For in­stance, last year, VSCode ex­ten­sions with 9 mil­lion in­stalls were pulled over se­cu­rity risks, and 10 more, pos­ing as le­git­i­mate de­vel­op­ment tools, in­fected users with the XMRig cryp­tominer.

Later in the year, a ma­li­cious ex­ten­sion with ba­sic ran­somware ca­pa­bil­i­ties snuck onto the VS Code mar­ket­place af­ter a threat ac­tor named WhiteCobra flooded it with 24 crypto-steal­ing ex­ten­sions.

More re­cently, in January, two ma­li­cious ex­ten­sions ad­ver­tised as AI-based cod­ing as­sis­tants with 1.5 mil­lion in­stalls ex­fil­trated data from com­pro­mised de­vel­oper sys­tems to servers in China.

GitHub’s cloud-based plat­form is now used by over 4 mil­lion or­ga­ni­za­tions (including 90% of the Fortune 100) and more than 180 mil­lion de­vel­op­ers who con­tribute to over 420 mil­lion code repos­i­to­ries.

The Validation Gap: Automated Pentesting Answers One Question. You Need Six.

Automated pen­test­ing tools de­liver real value, but they were built to an­swer one ques­tion: can an at­tacker move through the net­work? They were not built to test whether your con­trols block threats, your de­tec­tion rules fire, or your cloud con­figs hold.

This guide cov­ers the 6 sur­faces you ac­tu­ally need to val­i­date.

Download Now

Everything in C is undefined behavior

blog.habets.se

If he had been a pro­gram­mer, Cardinal Richelieu would have said Give me six lines writ­ten by the hand of the most ex­pert C pro­gram­mer in the world, and I will find enough in them to trig­ger un­de­fined be­hav­ior”.

Nobody can write cor­rect C, or C++. And I say that as some­one who’s writ­ten C and C++ on an al­most daily ba­sis for about 30 years. I lis­ten to C++ pod­casts. I watch C++ con­fer­ence talks. I en­joy read­ing and writ­ing C++.

C++ has served us well, but it’s 2026, and the en­vi­ron­ment of 1985 (C++) or 1972 (C) is not the en­vi­ron­ment of to­day.

I’m def­i­nitely not the first to say this. I re­mem­ber read­ing a post by some­one promi­nent about a decade ago say­ing that a good case can be made that use of C++ is a SOX vi­o­la­tion. And while I was not on­board with the rest of their rant (nor their con­fu­sion about its” vs it’s”), I never dis­agreed about that point.

With time I found it to be more and more true. WAY more things are un­de­fined be­hav­ior (UB) than you’d ex­pect.

Everyone knows that dou­ble-free, use af­ter free, ac­cess­ing out­side the bounds of an ob­ject (e.g. ar­ray), and ac­cess­ing unini­tial­ized mem­ory is UB. After all, C & C++ are not mem­ory safe lan­guages. And yet we as an in­dus­try seem to be un­able to stop mak­ing even those mis­takes over and over.

But there’s more. More sub­tle. More il­log­i­cal.

It’s not about op­ti­miza­tions

Some peo­ple seem to think that as long as they don’t com­pile with op­ti­miza­tions turned on, un­de­fined be­hav­ior can’t hurt them. They be­lieve that the com­piler is some­how be­ing de­lib­er­ately hos­tile, go­ing AHA! UB! I can do what­ever I want here!”, and with­out op­ti­miza­tions turned on it won’t.

This is in­cor­rect.

UB does­n’t mean that the com­piler can take ad­van­tage of your slop­pi­ness. UB means that the com­piler can as­sume that your code is valid. It means that the in­ten­tion of your code that’s oh so ob­vi­ous when read by a hu­man, does­n’t even have a way to be ex­pressed be­tween com­piler stages or mod­ules.

UB means that the com­piler does­n’t even have to im­ple­ment some spe­cial cases in its code gen­er­a­tion, be­cause they can’t hap­pen”.

The com­piler, and re­ally the un­der­ly­ing hard­ware too, is play­ing a game of tele­phone with your UB in­ten­tions. It may end up with what you wanted, but there’s no guar­an­tee for now or in the fu­ture.

UB is every­where

The fol­low­ing is not an at­tempt at enu­mer­at­ing all the UB in the world. It’s merely mak­ing the case that UB is every­where, and if no­body can do it right, how is it even fair to blame the pro­gram­mer? My point is that ALL non­triv­ial C and C++ code has UB.

Accessing an ob­ject which is not cor­rectly aligned

As an ex­am­ple of this, take this code:

int foo(const int* p) { re­turn *p; }

If this func­tion is called with a pointer not cor­rectly aligned (probably mean­ing on an ad­dress that’s a mul­ti­ple of sizeof(int), but who knows), this is UB. C23 6.3.2.3.

On Linux Alpha, in some cases this would merely trap to the ker­nel, which would soft­ware em­u­late what you in­tended. In other cases it would (probably) crash your pro­gram with a SIGBUS.

On SPARC it would cause a SIGBUS.

Sure, on x86/​amd64 (henceforth just x86”) this is likely fine. Hell, it’s prob­a­bly even an atomic read. x86 is fa­mously ex­tremely for­giv­ing about cache co­herency sub­tleties.

So here we have three cases:

ker­nel gave a help­ing hand (Alpha for some loads)

crash (other Alpha loads, and SPARC)

not a prob­lem (x86)

What about ARM, RISC-V, and oth­ers? What about fu­ture ar­chi­tec­tures? A fu­ture ar­chi­tec­ture could even have spe­cial int-pointer reg­is­ters that do not pop­u­late the low­est bits, be­cause such point­ers can­not ex­ist.

Even if it works, maybe the com­piler one day changes from us­ing one load in­struc­tion to an­other, and sud­denly that’s no longer fixed up by the ker­nel.

Because the com­piler is not ob­lig­ated to gen­er­ate as­sem­bly in­struc­tions that work on un­aligned point­ers. Because it’s UB.

Or how about this:

void set_it(std::atomic<int>* p) { p->store(123); } int get_it(std::atomic<int>* p) { re­turn p->load(); }

Is this op­er­a­tion atomic when the ob­ject is not cor­rectly aligned? That’s the wrong ques­tion to ask. Mu, unask the ques­tion. It’s UB. (but also yes, in prac­tice this can eas­ily be an atom­ic­ity prob­lem)

If you want to get even more con­vinced, you can try think­ing about what hap­pens if an ob­ject you thought you were read­ing atom­i­cally spans pages. But don’t think too much about it, or you may con­clude that it’s fine”. It’s not. It’s UB.

Actually, it was UB even be­fore that

Don’t blame the foo() func­tion, above. The act of deref­er­enc­ing the pointer was­n’t the prob­lem. Merely cre­at­ing the pointer was enough to be a prob­lem.

Example:

bool parse_­packet(const uin­t8_t* bytes) { const int* mag­ic_intp = (const int*)bytes; // UB! int mag­ic_raw = foo(mag­ic_intp); // Probably crashes on SPARC. int magic = ntohl(mag­ic_raw); // this is fine, at least. […] }

That cast is the prob­lem, not foo().

It’s per­fectly valid for the com­piler to as­sign spe­cific mean­ing, such as garbage col­lec­tion or se­cu­rity tag­ging bits, to the lower bits of an int*.

isxdigit() on char in­put

bool bar(char ch) { re­turn isxdigit(ch); }

isxdigit() is a sim­ple func­tion that takes a char­ac­ter and re­turns 1 if it’s a hex digit. 0 – 9 or a-f. It can also take the value EOF. Uh, ok. What value is EOF? Per C23 7.4p1 we know it’s an int, and we can in­fer that it’s not rep­re­sentable by un­signed char.

isxdigit() there­fore takes an int, not a char. All val­ues of char fit in­side int, so we should be fine. Casting from char to int fits, so per sec­tion 6.3.1.3 we’re fine, right?

No. Because if bar() is called with a value other than 0 – 127, and on your ar­chi­tec­ture char is signed (implementation de­fined, per 6.2.5, para­graph 20 in C23), then the in­te­ger value ends up neg­a­tive.

And the fol­low­ing is a valid im­ple­men­ta­tion of isxdigit(), that would cause a read of who-knows-what mem­ory. It could even be I/O mapped mem­ory, trig­ger­ing things to hap­pen that is more than merely get­ting a ran­dom value or crash. It could cause the mo­tor to start. Less likely in an ap­pli­ca­tion run­ning in a desk­top op­er­at­ing sys­tem than in an em­bed­ded sys­tem, sure. But there are user space net­work dri­vers (for per­for­mance), so even user space won’t pro­tect you.

int isxdigit(int c) { if (c == EOF) { re­turn false; } re­turn some_ar­ray[c]; }

Casting from float to int

int mil­lisec­onds(float sec­onds) { int tmp = (int)(seconds * 1000.0); /* WRONG */ re­turn tmp + 1; /* WRONG sep­a­rately (signed over­flow is UB) */ }

When a fi­nite value of real float­ing type is con­verted to an in­te­ger type[…]If the value of the in­te­gral part can­not be rep­re­sented by the in­te­ger type, the be­hav­ior is un­de­fined. — 6.3.1.4

And, by omis­sion, it’s also UB if the float is a non-fi­nite value.

So how do you com­pare a float to INT_MAX? Do you cast the float to int? No, that’s the UB you want to avoid. So you cast INT_MAX to float? How do you know it can be rep­re­sented ex­actly? Maybe cast­ing INT_MAX to float rounds to a value not rep­re­sentable in int, and your com­par­i­son be­comes non-rep­re­sen­ta­tive?

Maybe the fol­low­ing works? You’ll miss out on rep­re­sent­ing some re­ally high val­ues, but maybe that’s OK?

int mil­lisec­onds(float sec­onds) { const float ftmp = sec­onds * 1000.0f; if (!isfinite(ftmp)) { // or other er­ror re­port­ing. re­turn 0; } if ((float)(INT_MIN + 1000) > ftmp) { // or other er­ror re­port­ing. re­turn 0; } if ((float)(INT_MAX - 1000) < ftmp) { // or other er­ror re­port­ing. re­turn 0; } // Now safe to con­vert. const int tmp = (int)ftmp; if (INT_MAX == tmp) { // or other er­ror re­port­ing. re­turn 0; } // Now safe to add. re­turn tmp + 1; }

I just wanted to con­vert a float to an int. :-(

I bet there’s lots of code out there that take a value in sec­onds, and con­vert it to in­te­ger mil­lisec­onds, by just mul­ti­ply­ing and cast­ing.

Object at ad­dress zero

Most pro­gram­mers won’t have to deal with this, but I don’t think there’s any C stan­dards com­pli­ant way in prac­tice to put an ob­ject at ad­dress zero. This can come up in OS ker­nel and em­bed­ded cod­ing.

By 6.3.2.3 an in­te­ger con­stant zero (which is con­vert­ible to a pointer) and nullptr are the null pointer con­stant” (which I’ll just call NULL). C does­n’t spec­ify that the ac­tual pointer NULL points addr ma­chine ad­dress zero, be­cause the C stan­dard only talks of the C ab­stract ma­chine, not about hard­ware.

All C guar­an­tees is that if you com­pare NULL to zero you’ll see them equal. But for all you know that’s be­cause the zero is con­verted to the na­tive plat­for­m’s NULL, which hap­pens to be 0xffff.

It also ex­plic­itly says that deref­er­enc­ing a null pointer, no mat­ter what the value, is un­de­fined be­hav­ior. It’s the ex­am­ple of UB un­der 3.4.3.

This also means that you can’t as­sume that mem­set(&ptr, 0, sizeof(ptr)); will cre­ate a NULL pointer! You can­not ini­tial­ize your structs this way and as­sume mem­ber point­ers are NULL! And this does ap­ply to most pro­gram­mers.

And yes, some his­toric ma­chines used non-zero NULL point­ers.

But let’s say you have a mod­ern ma­chine, where NULL is a pointer to ad­dress zero, and you ac­tu­ally have an ob­ject there.

Again, C 6.3.2.3 says that NULL com­pares un­equal to any ob­ject or func­tion”. So this is UB:

void (*func_ptr)() = NULL; func_ptr();

C says there is no func­tion there”. For all you know the com­piler has no in­ter­nal way to even ex­press your in­ten­tion here. You may ar­gue that but surely it’ll just emit a call in­struc­tion to the bit pat­tern of all ze­roes? Nothing else seems rea­son­able.

What is all ze­roes”, though? On 16bit x86, is it 0000:0000? Is it CS:0000?

Variable ar­gu­ments and types (e.g. printf with %ld in­stead of %lld)

This is UB:

ex­ecl(“/​bin/​sh”, sh”, -c”, date”, NULL); /* WRONG */ ex­ecl(“/​bin/​sh”, sh”, -c”, date”, 0); /* WRONG */

This is not:

ex­ecl(“/​bin/​sh”, sh”, -c”, date”, (char*)NULL);

Because the ar­gu­ment needs to be a pointer, and the NULL macro may be mis­in­ter­preted as an in­te­ger zero.

Similarly, this is UB:

uin­t64_t blah = 123; printf(“%ld\n”, blah); /* WRONG */

It needs to be:

uin­t64_t blah = 123; printf(“%”PRI­u64″\n”, blah);

So how do you print an uid_t? Well, you could cast them to uint­max_t and print them us­ing PRIuMAX. But is uid_t even un­signed? Oh well, worst case you get a non­sense value printed in­stead of -1, I guess.

Divide by zero is UB

Sure, you prob­a­bly knew this. But did you con­sider the se­cu­rity as­pects of it? It’s not rare for the de­nom­i­na­tor to come from un­trusted in­put.

And there’s so much more. The C23 stan­dard con­tains 283 uses of the word undefined”. And that’s not even in­clud­ing the things that are un­de­fined by omis­sion.

Bonus non-UB

Nobody can ap­ply in­te­ger pro­mo­tion rules at code skim­ming speeds. Nobody.

This post is al­ready long enough, but as a start:

un­signed char a = 0xff; un­signed char b = 1; un­signed char zero = 0; bool over­flowed = (a + b) == zero; // over­flowed is set to zero, not one.

un­signed char a = 0x80; uin­t64_t b = a << 24; // Bonus UB(?) // b is now 18446744071562067968 (ffffffff80000000), not 2147483648 (0x80000000). // even with all our vari­ables un­signed.

LLMs are bet­ter than us at this

Point an LLM at ANY C code, ask­ing it to find UB, and it will. And it’ll be right al­most all the time, nowa­days.

I felt a bit bad af­ter it cor­rectly found ones in my code, so I thought I’d point it at the ma­ture and pedan­ti­cally writ­ten OpenBSD. I just picked the first tool I could think of, find, and it spit out a bunch.

I sent the pro­ject a patch for an out of bounds write (and also for a non-UB logic bug). I did­n’t send them patches for the UB that was left and right, partly be­cause the OpenBSD pro­ject has not been very re­cep­tive in the past for bug re­ports, my sense of this is prob­a­bly fine, in prac­tice”, and that if OpenBSD wants to weed out UB from their code base, then that’s a ma­jor pro­ject that should be done in a bet­ter way than me just be­ing the mid­dle man be­tween the LLM and them for a patch here and there.

So what do we do now?

We can’t just throw away our C and C++ code bases. But leav­ing them in­her­ently bro­ken is also not an op­tion.

We need some way of fix­ing UB at scale, with­out com­mit­ting AI slop nor over­whelm­ing hu­man re­view­ers.

This too is not a new opin­ion, nor a great rev­e­la­tion.

But yes, writ­ing C or C++ in 2026 with­out an LLM su­per­vis­ing you for UB should prob­a­bly be seen as a SOX vi­o­la­tion, and just plain ir­re­spon­si­ble. If OpenBSD peo­ple can’t find these prob­lems given 30+ years, what chance do the rest of us have?

It may not scale to large code bases, but for my own pro­jects I’ve asked the LLM to find UB, if nec­es­sary ex­plain it, and fix it. And then stare at the out­put un­til I can con­firm the is­sue and the fix.

A prob­lem with this is that in or­der to con­firm the find­ings, you’ll need an ex­pert hu­man. But gen­er­ally ex­pert hu­mans are busy do­ing other things. This is jan­i­tor work, but too sub­tle to leave to the ju­nior pro­gram­mers who have tra­di­tion­ally been as­signed jan­i­tor work.

No way to parse in­te­gers in C

Integer han­dling is bro­ken

UB in the Linux ker­nel

Map of Metal

mapofmetal.com

An in­ter­ac­tive overview of Metal his­tory and the in­flu­en­tial bands that helped shaped the gen­res we know to­day.

To con­tinue you need to en­able JavaScript.

Incident Report: May 19, 2026- GCP Account Suspension

blog.railway.com

Table of Contents

Impact

Incident Timeline

What Happened?

Preventative Measures

🚅

This re­port re­flects what we know at time of pub­li­ca­tion and may be up­dated pend­ing Google Cloud’s in­ter­nal re­view.

Railway ex­pe­ri­enced a plat­form-wide ser­vice dis­rup­tion due to Google Cloud in­cor­rectly plac­ing our ac­count in a sus­pended sta­tus. This re­sulted in a tem­po­rary loss of ser­vice for all GCP hosted in­fra­struc­ture. This in­fra­struc­ture sup­ports our dash­board, API, and pieces of our net­work in­fra­struc­ture. As cached net­work routes ex­pired, the out­age ex­tended be­yond GCP to af­fect all Railway work­loads.

Below, we walk through what hap­pened, how we re­sponded, and what we’re do­ing to pre­vent a sim­i­lar in­ci­dent in the fu­ture.

Impact

On May 19, 2026 be­tween 22:20 UTC and ap­prox­i­mately 06:14 UTC on May 20 (~8 hours), Railway ex­pe­ri­enced a plat­form-wide out­age af­ter Google Cloud sus­pended ser­vices on our pro­duc­tion ac­count. This took our API, con­trol plane and data­bases of­fline, along with com­pute in­fra­struc­ture hosted on Google Cloud.

Users im­me­di­ately ex­pe­ri­enced 503 er­rors on the dash­board and API, in­clud­ing no healthy up­stream” and unconditional drop over­load” mes­sages, and were un­able to log in. All work­loads hosted on Google Cloud com­pute were taken of­fline.

While work­loads on our own Railway Metal and AWS burst-cloud en­vi­ron­ments re­mained up, Railway’s edge prox­ies rely on a Google Cloud-hosted con­trol plane API to pop­u­late their rout­ing ta­bles, caus­ing the out­age to cas­cade be­yond Google Cloud. As the route caches ex­pired, these other work­loads be­came un­reach­able, re­sult­ing in re­turn­ing 404 er­rors as the net­work con­trol plane could no longer re­solve routes to ac­tive in­stances. At peak im­pact, all Railway work­loads across all re­gions were ren­dered un­reach­able.

As we re­cov­ered our Google Cloud en­vi­ron­ment, builds and de­ploy­ments were blocked plat­form-wide while we re­stored the in­di­vid­ual ser­vices. Once the en­tirety of our in­fra­struc­ture was re­stored, a sig­nif­i­cant back­log of queued de­ploys was grad­u­ally drained to avoid over­whelm­ing the plat­form. In par­al­lel, GitHub be­gan rate-lim­it­ing Railway’s OAuth and web­hook in­te­gra­tions, tem­porar­ily block­ing lo­gins and builds. The vol­ume of these calls in­creased as a re­sult of our caches be­ing cleared from the Google Cloud out­age. As a side ef­fect, Terms-of-service ac­cep­tance records were also re­set, prompt­ing users to re-ac­cept on their next visit to the dash­board.

We take full re­spon­si­bil­ity for the ar­chi­tec­tural de­ci­sions that al­lowed a sin­gle up­stream provider ac­tion to cas­cade into a plat­form-wide out­age, and de­tail be­low what hap­pened, how we re­cov­ered, and the changes we are mak­ing to pre­vent this from hap­pen­ing again.

Incident Timeline

May 19, 22:10 UTC - Our au­to­mated mon­i­tor­ing de­tected API health check fail­ures and paged our on-calls, who started in­ves­ti­gat­ing the is­sue.

May 19, 22:11 UTC - Dashboard re­turn­ing 503 er­rors. Users un­able to log in.

May 19, 22:19 UTC - Root cause iden­ti­fied: Google Cloud Platform has sus­pended Railway’s pro­duc­tion ac­count.

May 19, 22:22 UTC - P0 ticket filed with Google Cloud. Railway’s GCP ac­count man­ager en­gaged di­rectly.

May 19, 22:29 UTC - Incident de­clared.

May 19, 22:29 UTC - GCP ac­count ac­cess re­stored. All com­pute in­stances re­mained stopped and per­sis­tent disks in­ac­ces­si­ble.

May 19, 22:35 UTC - Cached net­work routes be­gan ex­pir­ing; work­loads on Railway Metal and AWS be­gan re­turn­ing 404 er­rors as the net­work­ing could no longer re­solve routes.

May 19, 23:09 UTC - First per­sis­tent disk comes back on­line.

May 19, 23:54 UTC - All per­sis­tent disks re­stored to ready state. Network still down.

May 20, 00:39 UTC - Disks con­firmed ready. Recovery blocked on Google Cloud net­work­ing restora­tion.

May 20, 01:30 UTC - Compute in­stances be­gan re­cov­er­ing.

May 20, 01:38 UTC - Edge traf­fic be­ing served again. Networking re­stored.

May 20, 01:57 UTC - Orchestration and build in­fra­struc­ture re­stored. Deploys tem­porar­ily paused to pre­vent over­whelm­ing sys­tems as queued work at­tempted to ex­e­cute si­mul­ta­ne­ously.

May 20, 02:04 UTC - Compute hosts be­ing brought back on­line in­cre­men­tally.

May 20, 02:47 UTC - GitHub be­gan rate-lim­it­ing Railway’s OAuth and web­hook in­te­gra­tions; some users un­able to log in, builds blocked.

May 20, 02:55 UTC - Dashboard ac­ces­si­ble again.

May 20, 03:59 UTC - Deployments be­gin­ning to process again across all tiers.

May 20, 04:00 UTC - API, dash­board, and OAuth end­points con­firmed op­er­a­tional. Remaining work­loads con­tin­u­ing to re­store.

May 20, 06:14 UTC - Incident moved to mon­i­tor­ing.

May 20, 07:58 UTC - Incident is re­solved.

What Happened?

At 22:20 UTC on May 19, Google Cloud placed Railway’s pro­duc­tion ac­count into a sus­pended sta­tus in­cor­rectly, as part of an au­to­mated ac­tion. This ac­tion ex­tended to many ac­counts within Google Cloud. As this was a plat­form-wide ac­tion, there was no proac­tive out­reach to in­di­vid­ual cus­tomers prior to the re­stric­tion.

This sus­pended sta­tus dis­abled our GCP re­lated in­fra­struc­ture, which sup­ports the Railway Dashboard, API and parts of our Network in­fra­struc­ture, along with ad­di­tional burst-com­pute in­fra­struc­ture hosted on Google Cloud.

Railway’s con­trol plane is a set of a core de­pen­den­cies that serves the dash­board, processes builds and de­ploy­ments, and pop­u­lates the rout­ing ta­bles used by our edge. The im­pact was im­me­di­ate for all work­loads on Google Cloud.

Railway’s edge prox­ies main­tain a cache of rout­ing ta­bles from the net­work con­trol plane, which is hosted within Google Cloud. While that cache held, work­loads on Railway Metal and AWS con­tin­ued to serve traf­fic. Once the cache ex­pired, the edge could no longer re­solve routes to ac­tive in­stances, and work­loads across all re­gions, in­clud­ing Metal and AWS, be­gan re­turn­ing 404 er­rors. This caused the net­work out­age im­pact to cas­cade be­yond Google Cloud, into these re­gions as well, even though the work­loads them­selves re­mained on­line.

Railway’s in­fra­struc­ture is de­signed for high avail­abil­ity. Our data­bases run across mul­ti­ple avail­abil­ity zones, and our net­work uses re­dun­dant con­nec­tions be­tween AWS, GCP, and Railway Metal. However, restor­ing ac­count ac­cess did not re­store these in­di­vid­ual ser­vices. Persistent disks, com­pute in­stances, and net­work­ing all re­quired sep­a­rate re­cov­ery. Due to the na­ture of this re­cov­ery process, the out­age was ex­tended by sev­eral hours. Disks were re­stored to a ready state by 23:54 UTC, but core net­work­ing and edge rout­ing did not fully re­store un­til ap­prox­i­mately 01:30 UTC on May 20. (We are await­ing con­fir­ma­tion to see if this de­lay and as­so­ci­ated er­rors were on Google’s side)

As net­work­ing was re­stored, re­cov­ery of Railway core ser­vices and val­i­da­tion of end user work­loads pro­ceeded layer by layer. To pre­vent over­whelm­ing our build sys­tems we tem­porar­ily paused de­ploys, and grad­u­ally al­lowed them to re­sume. In par­al­lel to our core sys­tem re­cov­ery, GitHub be­gan rate-lim­it­ing Railway’s OAuth and web­hook in­te­gra­tions, due to the vol­ume and burst na­ture of all re­tried re­quests, tem­porar­ily block­ing user lo­gins and builds.

By ap­prox­i­mately 04:00 UTC on May 20, the API, dash­board, and OAuth end­points were con­firmed op­er­a­tional, with re­main­ing work­loads con­tin­u­ing to re­store.

Preventative Measures

Railway’s net­work con­trol plane is de­signed for re­silience. It is a multi-AZ, multi-zone con­trol plane which can tol­er­ate the loss of mul­ti­ple ma­chines and com­po­nents, while still func­tion­ing with zero user im­pact. This has been tested in both stag­ing as well as live traf­fic (prior to its roll­out a few months ago).

We have in­vested in re­siliency as a re­sult of prior in­ci­dents which have as­sisted us in deal­ing with the im­pact. A prior ex­am­ple of these lessons was Railway be­ing able to grace­fully re­cover user GitHub in­stal­la­tions with­out trig­ger­ing sec­ondary rate-lim­its.

However, many have asked over mul­ti­ple fo­rums, how could Railway have a sin­gle de­pen­dency that would af­fect all cus­tomer work­loads?

Railway’s net­work is a mesh ring, built up of high avail­abil­ity fiber in­ter­con­nects be­tween Metal <> GCP <> AWS. However, in this ring, there was still a hard de­pen­dency on work­load dis­cov­er­abil­ity be­ing tied to the net­work con­trol plane API that was hosted on the ma­chines run­ning in Google Cloud. This meant that de­spite the mesh con­tin­u­ing to op­er­ate for an hour, when the route cache ex­pired, the mesh failed to re-pop­u­late the rout­ing ta­bles.

We are im­me­di­ately work­ing on re­mov­ing this de­pen­dency, mak­ing this a true mesh. This means that if any of the in­ter­con­nects go out, there is al­ways a path be­tween the clouds.

As a re­sult of this, we will be ex­tend­ing the high avail­abil­ity data­base shards across AWS and Metal. In the fu­ture, should all in­stances in a par­tic­u­lar cloud dis­ap­pear in­stantly, data­base quo­rum will keep every­thing run­ning and im­me­di­ately failover any no longer run­ning work­loads.

Finally, we are in plan­ning to re­move Google Cloud ser­vices from our data plane’s hot path, and keep­ing them only for sec­ondary/​failover. This is in par­al­lel to im­ple­ment­ing a new ar­chi­tec­ture for our data plane (enabling con­nec­tiv­ity to hosts), and our con­trol plane (which pow­ers the dash­board you use to ac­cess and man­age Railway). These ar­chi­tec­ture up­grades will en­sure that our core ser­vices, es­pe­cially user fac­ing com­po­nents, are not de­pen­dent on any one ven­dor or plat­form.

Railway owns our ven­dor choices, and we ul­ti­mately own this one. Your cus­tomers don’t care whether the fail­ure was Google or Railway; they see your prod­uct. Your up­time is our re­spon­si­bil­ity, and we’ll keep de­liv­er­ing on it.

To add this web app to your iOS home screen tap the share button and select "Add to the Home Screen".

10HN is also available as an iOS App

If you visit 10HN only rarely, check out the the best articles from the past week.

Visit pancik.com for more.