10 interesting stories served every morning and every evening.

Obfuscated, self-evaluating bash script by CDN Akamai being supplied to consumers via retail stores

tris.sherliker.net

When my wife said to me Let me show you a t-shirt I saw…”, I was­n’t sure what to ex­pect, but it def­i­nitely was­n’t an ob­fus­cated bash script printed on the back de­signed to print a happy Easter egg mes­sage.

I’m not in the habit of click­baity head­li­ne­sI’ve no idea at all how many views this site gets, but I’m will­ing to bet it’s not even dou­ble-digit hu­mans per month. but I can see why sube­d­i­tors have such fun with them. The ti­tle above is, strictly speak­ing, en­tirely ac­cu­rate, but prob­a­bly not what you think. The ob­fus­cated code in ques­tion is ac­tu­ally an easter egg, it’s be­ing sup­plied via Uniqlo stores on an ex­cel­lent t-shirt de­signed by Akamai in sup­port of their Peace for All cam­paign.

And it’s very cool! The front has a heart in curly braces:

While the back has a big al­phanu­meric block:

Is that … a she­bang?!

My wife was right that I’d want to see it. Was that… a she­bang?

Take a closer look at the text block:

Yes, a she­bang! On a t-shirt sold in a high street store, no less. And it is clearly a base64 en­coded Here string be­ing fed to eval via base64 –decode.

Interesting. I told my wife that’s ba­si­cally how peo­ple ship virus­es’ and bought it.

OCR was cum­ber­some

There was good news and bad news:

The bad news was that base64 has­n’t got er­ror cor­rec­tion, mean­ing that the tran­scrip­tion would need to be per­fect. Sigh.

The good news was that the string seemed to be in­tact - at least, it ter­mi­nated with the ex­pected padding and had match­ing quotes and braces. This is a good thing be­cause Uniqlo x Akamai sells an­other de­sign of shirt in the same range which is plainly in­com­plete­For ex­am­ple, its im­ports are trun­cated and it ends retu” in­stead of return”. This is a pity, be­cause it’s a re­ally nice colour com­bi­na­tion and con­tains the highly id­iomatic in­struc­tion go doStuff(msg, work… which any­one can re­late to., a trun­cated crop from a wider text block which could never com­pile.

I ran OCR in a few ways: First, us­ing the built-in OCR of the cir­cle-to-search fea­ture on Android, which is of­ten very good. Second, by us­ing Tesseract with a few op­tions and tweaks. And third by run­ning it through Claude. After diff­ing the three to look for mis­matches and get­ting Claude to out­put a table of lo­ca­tions for quick scan­ning, it be­came triv­ial but time-con­suimg to tidy up the re­main­der. The re­sult­ing string was:

IyEvYmluL2Jhc2gKCiMgQ29uZ3JhdHVsYXRpb25zISBZb3UgZm91bmQgdGhlIGVhc3RlciBlZ2chIOKdpO+4jwojIOOBiuOCgeOBp+OBqOOBhuOBlOOBluOBhOOBvuOBme+8gemaoOOBleOCjOOBn+OCteODl+ODqeOCpOOCuuOCkuimi+OBpOOBkeOBvuOBl+OBn++8geKdpO+4jwoKIyBEZWZpbmUgdGhlIHRleHQgdG8gYW5pbWF0ZQp0ZXh0PSLimaVQRUFDReKZpUZPUuKZpUFMTOKZpVBFQUNF4pmlRk9S4pmlQUxM4pmlUEVBQ0XimaVGT1LimaVBTEzimaVQRUFDReKZpUZPUuKZpUFMTOKZpVBFQUNF4pmlRk9S4pmlQUxM4pmlIgoKIyBHZXQgdGVybWluYWwgZGltZW5zaW9ucwpjb2xzPSQodHB1dCBjb2xzKQpsaW5lcz0kKHRwdXQgbGluZXMpCgojIENhbGN1bGF0ZSB0aGUgbGVuZ3RoIG9mIHRoZSB0ZXh0CnRleHRfbGVuZ3RoPSR7I3RleHR9CgojIEhpZGUgdGhlIGN1cnNvcgp0cHV0IGNpdmlzCgojIFRyYXAgQ1RSTCtDIHRvIHNob3cgdGhlIGN1cnNvciBiZWZvcmUgZXhpdGluZwp0cmFwICJ0cHV0IGNub3JtOyBleGl0IiBTSUdJTlQKCiMgU2V0IGZyZXF1ZW5jeSBzY2FsaW5nIGZhY3RvcgpmcmVxPTAuMgoKIyBJbmZpbml0ZSBsb29wIGZvciBjb250aW51b3VzIGFuaW1hdGlvbgpmb3IgKCggdD0wOyA7IHQrPTEgKSk7IGRvCiAgICAjIEV4dHJhY3Qgb25lIGNoYXJhY3RlciBhdCBhIHRpbWUKICAgIGNoYXI9IiR7dGV4dDp0ICUgdGV4dF9sZW5ndGg6MX0iCiAgICAKICAgICMgQ2FsY3VsYXRlIHRoZSBhbmdsZSBpbiByYWRpYW5zCiAgICBhbmdsZT0kKGVjaG8gIigkdCkgKiAkZnJlcSIgfCBiYyAtbCkKCiAgICAjIENhbGN1bGF0ZSB0aGUgc2luZSBvZiB0aGUgYW5nbGUKICAgIHNpbmVfdmFsdWU9JChlY2hvICJzKCRhbmdsZSkiIHwgYmMgLWwpCgogICAgIyBDYWxjdWxhdGUgeCBwb3NpdGlvbiB1c2luZyB0aGUgc2luZSB2YWx1ZQogICAgeD0kKGVjaG8gIigkY29scyAvIDIpICsgKCRjb2xzIC8gNCkgKiAkc2luZV92YWx1ZSIgfCBiYyAtbCkKICAgIHg9JChwcmludGYgIiUuMGYiICIkeCIpCgogICAgIyBFbnN1cmUgeCBpcyB3aXRoaW4gdGVybWluYWwgYm91bmRzCiAgICBpZiAoKCB4IDwgMCApKTsgdGhlbiB4PTA7IGZpCiAgICBpZiAoKCB4ID49IGNvbHMgKSk7IHRoZW4geD0kKChjb2xzIC0gMSkpOyBmaQoKICAgICMgQ2FsY3VsYXRlIGNvbG9yIGdyYWRpZW50IGJldHdlZW4gMTIgKGN5YW4pIGFuZCAyMDggKG9yYW5nZSkKICAgIGNvbG9yX3N0YXJ0PTEyCiAgICBjb2xvcl9lbmQ9MjA4CiAgICBjb2xvcl9yYW5nZT0kKChjb2xvcl9lbmQgLSBjb2xvcl9zdGFydCkpCiAgICBjb2xvcj0kKChjb2xvcl9zdGFydCArIChjb2xvcl9yYW5nZSAqIHQgLyBsaW5lcykgJSBjb2xvcl9yYW5nZSkpCgogICAgIyBQcmludCB0aGUgY2hhcmFjdGVyIHdpdGggMjU2LWNvbG9yIHN1cHBvcnQKICAgIGVjaG8gLW5lICJcMDMzWzM4OzU7JHtjb2xvcn1tIiQodHB1dCBjdXAgJHQgJHgpIiRjaGFyXDAzM1swbSIKCiAgICAjIExpbmUgZmVlZCB0byBtb3ZlIGRvd253YXJkCiAgICBlY2hvICIiCgpkb25lCgo=

The de­coded script

After Base64 de­cod­ing, the re­sult­ing script is a wel­com­ing and nicely com­mented Easter egg:

#!/bin/bash

# Congratulations! You found the easter egg! ❤️ # おめでとうございます!隠されたサプライズを見つけました!❤️

# Define the text to an­i­mate text=“♥PEACE♥FOR♥ALL♥PEACE♥FOR♥ALL♥PEACE♥FOR♥ALL♥PEACE♥FOR♥ALL♥PEACE♥FOR♥ALL♥”

# Get ter­mi­nal di­men­sions cols=$(tput cols) lines=$(tput lines)

# Calculate the length of the text tex­t_length=${#text}

# Hide the cur­sor tput civis

# Trap CTRL+C to show the cur­sor be­fore ex­it­ing trap tput cnorm; exit” SIGINT

# Set fre­quency scal­ing fac­tor freq=0.2

# Infinite loop for con­tin­u­ous an­i­ma­tion for (( t=0; ; t+=1 )); do # Extract one char­ac­ter at a time char=“${text:t % tex­t_length:1}”

# Calculate the an­gle in ra­di­ans an­gle=$(echo ($t) * $freq” | bc -l)

# Calculate the sine of the an­gle sine_­value=$(echo s($angle)” | bc -l)

# Calculate x po­si­tion us­ing the sine value x=$(echo ($cols / 2) + ($cols / 4) * $sine_value” | bc -l) x=$(printf %.0f” $x”)

# Ensure x is within ter­mi­nal bounds if (( x < 0 )); then x=0; fi if (( x >= cols )); then x=$((cols - 1)); fi

# Calculate color gra­di­ent be­tween 12 (cyan) and 208 (orange) col­or_s­tart=12 col­or_end=208 col­or_range=$((col­or_end - col­or_s­tart)) color=$((col­or_s­tart + (color_range * t / lines) % col­or_range))

# Print the char­ac­ter with 256-color sup­port echo -ne \033[38;5;${color}m”$(tput cup $t $x)“$char\033[0m”

# Line feed to move down­ward echo

done

The re­sult is a con­tin­u­ous happy sine-wave loop of the cam­paign mes­sage, Peace for All:

Detail: The font choice

Edit: The fol­low­ing font ID is wrong! User ral­phi­nus on Hacker News pointed out that the font is Roboto Mono. I don’t know how I over­looked the very dif­fer­ent g.

I guess Uniqlo is run through Windows though: one thing that struck me was the font, which I’m I was pre­vi­ously al­most cer­tain is ConsolasI was for­tu­nate enough to cor­re­spond with the de­signer, Lucas de Groot, once in re­la­tion to a le­gal case in which some­one had used one of his fonts to forge a doc­u­ment. He was very help­ful and kind enough to con­firm the nec­es­sary facts in writ­ing, even though he owed noth­ing to us. , which I’m fond of. Note the very shal­lowly-slashed 0, the lack of serif on the 1 and the rounded curves of let­ters like BDyg and num­ber 2. It’s strik­ing be­cause it’s pri­mar­ily a Windows font, so not the sort of thing I’d ex­pect to see call­ing Bash.

Linux, the lan­guage of the Internet

Akamai put out a press re­lease about the shirt when it was re­leased, which is an­other sort of in­ter­est­ing due to the blend of tech and mar­ket­ing:

Design mes­sage More than 25 years ago, Akamai helped make the in­ter­net we know to­day pos­si­ble. This shirt’s de­sign is a call­back to those early days of life on­line. The light tan color is a ref­er­ence to the beige box” plas­tic cas­ings that housed the early in­ter­net com­put­ers, and the heart on the front rep­re­sents how the in­ter­net has been used for good all over the world. On the back of the T-shirt is real code. It’s a ref­er­ence to Linux, the open-source lan­guage of the in­ter­net. This com­mon lan­guage unites Akamai with the world’s top brands and the peo­ple they serve, as we work to­gether to­ward a vi­sion of a safer and more con­nected world.

Design mes­sage

More than 25 years ago, Akamai helped make the in­ter­net we know to­day pos­si­ble. This shirt’s de­sign is a call­back to those early days of life on­line. The light tan color is a ref­er­ence to the beige box” plas­tic cas­ings that housed the early in­ter­net com­put­ers, and the heart on the front rep­re­sents how the in­ter­net has been used for good all over the world. On the back of the T-shirt is real code. It’s a ref­er­ence to Linux, the open-source lan­guage of the in­ter­net. This com­mon lan­guage unites Akamai with the world’s top brands and the peo­ple they serve, as we work to­gether to­ward a vi­sion of a safer and more con­nected world.

Not the first

I de­lib­er­ately did­n’t search for spoil­ers at first, but I see that I am of course not the first per­son to get nerd-sniped by this. Wen Chuan Lee and that post also links to an­other (against which I’ve cross-checked my tran­scrip­tion above). I’m happy to carry on the chain.

Read the dis­cus­sion of this post on Hacker News, which in­cludes links to more info from the de­signer and other in­ter­est­ing ob­ser­va­tions

Chatto is now Open Source!

www.hmans.dev

Hot damn. This is the big one.

I’m happy to an­nounce that Chatto, the group and team chat ap­pli­ca­tion that I’ve been work­ing on for the past year or so, is now of­fi­cially Open Source, and avail­able for any­one to self-host.

The fastest way to give it a try is through Homebrew:

brew in­stall chat­to­corp/​tap/​chatto chatto init chatto run

See Chatto’s Getting Started Guide for de­tails. Or stick around to hear more!

Chat Just Got Real

Chatto aims to be the group chat ap­pli­ca­tion that you ac­tu­ally en­joy us­ing. You’re prob­a­bly fa­mil­iar with the one that rhymes with knack”, or the one that rhymes with beams”, or the one that rhymes with this gourd”.

Chatto is just like those. Except you’re go­ing to love how com­pact and snappy it is. And that it’s Open Source. And you can just self-host it. For free, too! (A weird thing to write, but the OSS chat app space has be­come very weird in many ways!)

This is what it looks like:

If you want to see it in ac­tion, drop by the Chatto HQ Community!

It’s de­signed to be ex­tremely easy to self-host on your own in­fra­struc­ture. In its most ba­sic shape, you just run the ex­e­cutable, and that’s it. It even serves its own fron­tend!

It’s very light on re­sources, and prob­a­bly has the snap­pi­est fron­tend that you’ve ever used in an app like this. It puts data pro­tec­tion and pri­vacy first, with all per­sonal and chat data fully en­crypted at rest with per-user keys that get shred­ded when a user de­cides to delete their ac­count.

Each Chatto server pow­ers a sin­gle com­mu­nity, with no fed­er­a­tion of data be­tween servers, nor any third-party track­ing or an­a­lyt­ics. If you want to hang out in mul­ti­ple servers at once, the client will sim­ply con­nect to all of them di­rectly. If you want to host mul­ti­ple com­mu­ni­ties, just spin up mul­ti­ple Chatto processes. Easy!

Chatto comes with full sup­port for voice and video calls, with screen-shar­ing, built in. Calls are fully end-to-end en­crypted and will scale to as many par­tic­i­pants as your in­fra­struc­ture can han­dle.

And you can use it to­day, for free, by self-host­ing it on your own server. Binaries are avail­able for Linux (x86_64 and ARM64), ma­cOS, and Windows; head over to the Chatto Self-Hosting Documentation site to get started.

Chatto Cloud

If you pre­fer some­one else to take care of the host­ing, I’m also happy to an­nounce that Chatto Cloud will soon en­ter pub­lic beta. Chatto Cloud’s of­fer­ing is very sim­ple: it pro­vides paid host­ing for Chatto servers — and that’s it. No pre­mium sub­scrip­tions, no ads, no icky bits. Just host­ing.

And it’s re­ally good host­ing! Chatto Cloud is launch­ing with fully European and European-owned in­fra­struc­ture, with more re­gions slated for launch in early 2027. Every Chatto server on Chatto Cloud ben­e­fits from au­to­matic scal­ing, nightly back­ups of all data, and zero-down­time ver­sion up­grades.

There’s no lock-in; servers hosted through Chatto Cloud are 100% com­pat­i­ble with self-hosted ones, and you can pack up your data and move into or out of Chatto Cloud at any time.

If you want to get no­ti­fied about the start of the beta, please see the end of this post for a low-vol­ume newslet­ter you can sub­scribe to.

What’s Next for Chatto

Chatto is now at ver­sion 0.4. I con­sider it sta­ble enough for pro­duc­tion use, but there are a few im­por­tant fea­tures still miss­ing — head over to the Chatto Roadmap if you want an overview.

The fo­cus for Chatto 0.5 will be on ad­di­tional safety fea­tures (content re­port­ing and mod­er­a­tion) as well as pol­ish­ing the client, par­tic­u­larly its multi-server func­tion­al­ity. I have some fun stuff planned for this that I can’t wait to put into peo­ple’s hands.

I ex­pect Chatto to hit 1.0.0 in about 6 – 12 months. Until then, there may still be break­ing changes, even though I’ll be try­ing to keep them to a min­i­mum. If you do de­cide to self-host, please be ready to up­date to new ver­sions as they are re­leased.

Get in Touch

It’s been an ex­cit­ing jour­ney so far and I’m look­ing for­ward to find­ing out what’s ahead. If you’re self-host­ing Chatto, I’m su­per ea­ger to hear from you about your ex­pe­ri­ence — please don’t hes­i­tate to head over to the Chatto HQ com­mu­nity and get in touch.

Also please feel free to drop by and say hello if you’re in­ter­ested in Chatto for your com­pany, Open Source pro­ject, or sim­i­lar. I’d love to learn more about your re­quire­ments, and help you get set up.

Links

Chatto HQ Community - we have a #self-hosting sup­port chan­nel!

Chatto Self-Hosting Documentation

GitHub Repository

Chatto on Bluesky

Newsletter

If you want to be no­ti­fied about new re­leases or the start of Chatto Cloud’s beta, you’re in­vited to sub­scribe to the Chatto an­nounce­ments newslet­ter. It’s su­per low-vol­ume (~1 email per month), and is only used for no­ti­fy­ing you when ex­cit­ing new stuff be­comes avail­able.

openai.com

GitLost: How We Tricked GitHub’s AI Agent into Leaking Private Repos

noma.security

TL;DR: Noma Labs dis­cov­ered a crit­i­cal prompt in­jec­tion vul­ner­a­bil­ity within GitHub’s new Agentic Workflows, al­low­ing an unau­then­ti­cated at­tacker to silently pull data from pri­vate repos­i­to­ries by post­ing a crafted GitHub Issue in a pub­lic repos­i­tory be­long­ing to the same or­ga­ni­za­tion as the pri­vate repos­i­to­ries. Noma Labs named the vul­ner­a­bil­ity GitLost.

https://​noma.se­cu­rity/​wp-con­tent/​up­loads/​Git­Lost-full-video-1.mp4

Introduction

GitHub re­cently launched GitHub Agentic Workflows, pair­ing GitHub Actions (GitHub’s au­toma­tion sys­tem for run­ning tasks in re­sponse to repos­i­tory events) with an AI agent backed by Claude or GitHub Copilot. GitHub Agentic Workflows al­low teams to write their GitHub work­flows in plain Markdown, and the GitHub agent reads is­sues, calls tools, and re­sponds on its own.

As a vul­ner­a­bil­ity re­searcher with a se­cu­rity de­vel­op­ment back­ground, one of the first ques­tions that came to mind af­ter this launch was fun­da­men­tal and straight­for­ward: What will hap­pen when the GitHub agent reads some­thing it should not trust?

The an­swer is a text­book in­di­rect prompt-in­jec­tion at­tack, the kind of at­tack that qui­etly sends pri­vate data to any­one on the in­ter­net. Prompt in­jec­tion is a class of at­tack in which an ad­ver­sary hides ma­li­cious in­struc­tions in­side the con­tent read by an AI agent. That con­tent causes the agent to fol­low those hid­den in­struc­tions in­stead of the ones its op­er­a­tor in­tended.

What are GitHub Agentic Workflows?

GitHub Agentic Workflows let teams au­to­mate their in­ter­ac­tions with code repos­i­to­ries us­ing nat­ural lan­guage. Workflows live in Markdown (.md) files, are com­piled into YAML (a com­mon con­fig­u­ra­tion file for­mat), Actions files with the .yml ex­ten­sion, and run with the help of an AI agent with con­fig­urable per­mis­sions. The GitHub agent can read is­sues, call tools, and ac­cess other repos­i­to­ries within an or­ga­ni­za­tion.

GitLost Vulnerability Overview

The root cause of the GitLost vul­ner­a­bil­ity is, by now, a fa­mil­iar one in agen­tic AI sys­tems: prompt in­jec­tion. In most agen­tic prompt in­jec­tion at­tacks, the agent treats the wrong con­tent as a trusted source of in­struc­tions and al­lows it­self to be mis­di­rected or mis­used. This hap­pens when the sys­tem fails to main­tain a strict trust bound­ary be­tween sys­tem-level di­rec­tives and un­trusted user data. In this spe­cific case, any ma­li­cious ac­tor can cre­ate a GitHub Issue and, in the is­sue body, hide com­mands in plain English that GitHub’s agent will fol­low.

The vul­ner­a­ble Github Agentic Workflow Noma Labs dis­cov­ered was con­fig­ured to:

Trigger the work­flow on is­sues.as­signed events in GitHub

Read the is­sue Title and Body

Post a com­ment in re­sponse us­ing the add-com­ment tool

Run with read ac­cess to other repos­i­to­ries (public and pri­vate) in the or­ga­ni­za­tion

To ex­ploit this vul­ner­a­bil­ity, the at­tacker needed no cod­ing skills, ac­cess, or cre­den­tials. All that was needed was to open an is­sue in a pub­lic repos­i­tory be­long­ing to an or­ga­ni­za­tion that uses GitHub’s Agentic Workflow setup and wait.

The Attack Flow

Let’s take a look at the ex­act at­tack flow that Noma Labs vul­ner­a­bil­ity re­searchers suc­ceeded with:

First, they crafted a GitHub is­sue that looked com­pletely in­no­cent, con­sist­ing of a plau­si­ble-look­ing re­quest from a VP Sales af­ter meet­ing with a cus­tomer, as shown be­low:

In this spe­cific ex­am­ple, the work­flow ac­tion was trig­gered when the is­sue was as­signed, but our test­ing con­firmed it works the same way for other GitHub work­flow ac­tions.

Then, af­ter a GitHub au­toma­tion as­signed the is­sue, an event-trig­gered work­flow caused the agent to fetch the con­tents of README.md from both the poc (public) and test­lo­cal (private) repos­i­to­ries.

Finally, the GitHub agent then posted them as a pub­lic com­ment on the is­sue in the pub­lic repos­i­tory, which any­one could ac­cess and read.

The Additional” Exploit

GitHub had re­stric­tive guardrails in place to pre­vent ex­actly this sce­nario, but they failed to pro­tect the repos­i­to­ries as in­tended. Testing GitHub re­peat­edly with vari­a­tions, as an at­tacker would, and adding the key­word Additionally” trig­gered un­in­tended be­hav­ior in the model, caus­ing it to re­frame its out­put rather than refuse it. Essentially, by trick­ing the model, I was able to en­sure that GitHub’s guardrails did not work as in­tended and did­n’t pre­vent the data leak.

https://​noma.se­cu­rity/​wp-con­tent/​up­loads/​github_a­gen­tic_­work­flows.mp4

Vulnerability Proof of Concept

With the goal of full trans­parency, Noma Lab’s con­firmed find­ings, in­clud­ing our work­flow re­pro­duc­tions and live ev­i­dence, can be found here:

Workflow run: https://​github.com/​sasi­no­m­al­abs/​poc/​ac­tions/​runs/​23909666039

Issue: https://​github.com/​sasi­no­m­al­abs/​poc/​is­sues/​153

The leaked data in­cluded the con­tents of README.md from:

sasi­no­m­al­abs/​poc (public repo)

sasi­no­m­al­abs/​re­mote-ping (public repo, no README con­firmed)

sasi­no­m­al­abs/​test­lo­cal (private repo)

Why it Matters

GitLost per­fectly il­lus­trates one of the fun­da­men­tal se­cu­rity chal­lenges every or­ga­ni­za­tion faces with agen­tic AI sys­tems. The agen­t’s con­text win­dow is also its at­tack sur­face. Any con­tent the agent reads, whether is­sues, pull re­quests, com­ments, or files, can be weaponized if the agent treats that con­tent as in­struc­tional in­put.

Traditional se­cu­rity mod­els typ­i­cally as­sume that trust bound­aries are en­forced by code. In agen­tic sys­tems, trust bound­aries are partly en­forced by the mod­el’s be­hav­ior, and mod­els are in­her­ently in­struc­tion-fol­low­ing. Prompt in­jec­tion at­tacks have be­come, to agen­tic AI, what SQL in­jec­tions were to web ap­pli­ca­tions: a sys­tem­atic, cat­e­gory-wide vul­ner­a­bil­ity class that re­quires the same sys­tem­atic strate­gies and de­fenses.

Noma Recommendations for Builders/AI Security Officers:

Never treat user-con­trolled con­tent as trusted in­struc­tion in­put for an AI agent

Scope per­mis­sions to the min­i­mum re­quired. Agents with cross-repos­i­tory ac­cess are es­pe­cially high-value tar­gets

Restrict what any agent can post pub­licly, es­pe­cially in re­sponse to is­sue con­tent

Sanitize or iso­late user in­put from the in­struc­tion con­text be­fore pass­ing it to the model

Responsible Disclosure

GitLost was re­spon­si­bly dis­closed to GitHub. Vulnerability de­tails are shared here with their knowl­edge.

Found this in­ter­est­ing? Subscribe for more agen­tic AI vul­ner­a­bil­ity re­search by Noma Labs, or check out: GrafanaGhost, DockerDash, Context Crush, GeminiJack. Looking for an ef­fec­tive Agentic AI Security Solution? Contact us to arrange a demo of Noma’s com­pre­hen­sive so­lu­tion.

Attention Required! | Cloudflare

x.ai

Why have I been blocked?

This web­site is us­ing a se­cu­rity ser­vice to pro­tect it­self from on­line at­tacks. The ac­tion you just per­formed trig­gered the se­cu­rity so­lu­tion. There are sev­eral ac­tions that could trig­ger this block in­clud­ing sub­mit­ting a cer­tain word or phrase, a SQL com­mand or mal­formed data.

What can I do to re­solve this?

You can email the site owner to let them know you were blocked. Please in­clude what you were do­ing when this page came up and the Cloudflare Ray ID found at the bot­tom of this page.

Announcing TypeScript 7.0

devblogs.microsoft.com

Today we are proud to an­nounce the avail­abil­ity of TypeScript 7, a 10x faster na­tive port of TypeScript!

Since its early days, TypeScript has promised to de­liver on JavaScript that scales. By bring­ing strong type-check­ing and rich tool­ing to the world of JavaScript, TypeScript made it pos­si­ble to build non-triv­ial high-qual­ity apps across plat­forms.

Last year, our team un­veiled TypeScript’s next step in scal­ing: mak­ing every part of the toolset an or­der of mag­ni­tude faster. The mis­sion was a na­tive port of TypeScript built in Go that could make the most of mod­ern hard­ware. This port was done as faith­fully as pos­si­ble, writ­ing new code while main­tain­ing the struc­ture and logic of the orig­i­nal code­base to keep re­sults con­sis­tent and com­pat­i­ble be­tween the two com­pil­ers. The key dif­fer­ence is that with this new code­base, TypeScript 7 brings na­tive code speed, shared mem­ory mul­ti­thread­ing, and a num­ber of new op­ti­miza­tions that typ­i­cally yield speedups be­tween 8x and 12x on full builds.

Just as with any other re­lease, TypeScript 7 is avail­able via npm:

npm in­stall -D type­script

That will get you the new tsc ex­e­cutable in your work­space (which you can run via npx tsc). Of course, a big part of the TypeScript ex­pe­ri­ence is also about ed­i­tor sup­port. Your fa­vorite code ed­i­tor should eas­ily sup­port TypeScript 7 with its new sup­port for the lan­guage server pro­to­col (LSP), and its new speed and mul­ti­thread­ing im­prove­ments. Whether you’re us­ing some­thing like VS Code, Visual Studio, WebStorm, or any other mod­ern ed­i­tor, TypeScript 7 should work great. Just check your ed­i­tor’s doc­u­men­ta­tion — for ex­am­ple, VS Code has a ded­i­cated ex­ten­sion for TypeScript 7 that you can use to­day, and Visual Studio will au­to­mat­i­cally en­able TypeScript 7 based on your work­space.

What Does A Faster TypeScript Mean?

A faster TypeScript sounds great on pa­per, but what does it mean in prac­tice? Maybe it helps to think about where TypeScript comes up at every stage of de­vel­op­ment.

A typ­i­cal day of de­vel­op­ment might in­volve open­ing your ed­i­tor, open­ing a TypeScript file, and run­ning an op­er­a­tion like find-all-ref­er­ences across your pro­jects. Then as you’d start to make ed­its, maybe you’d ex­pect auto-com­ple­tions to pop up, and get red squig­gles on the fly as you’d make ed­its. When you (and more re­cently, per­haps an AI agent) were ready to build your pro­ject, you’d run tsc, check the out­put for er­rors, and then run your gen­er­ated code some­how.

A faster TypeScript means every part above is stream­lined. Waiting for your ed­i­tor to fully load your pro­ject will feel in­stan­ta­neous. Delays on find-all-ref­er­ences, auto-com­ple­tion, and di­ag­nos­tics should take a frac­tion of the time they used to. And when you run tsc, maybe in –watch mode, you’ll be able to tighten your feed­back loop and it­er­ate faster than ever be­fore.

You can see this on real-world pro­jects. In fact, you can try com­par­ing on a few open-source pro­jects your­self. Here are the build times of run­ning TypeScript 6 and 7 on some fairly large open source code­bases.

TypeScript 7 also typ­i­cally does bet­ter while ask­ing for less ag­gre­gate mem­ory over the span of a build.

Of course, there’s more to the ex­pe­ri­ence than the full build. On the same com­puter, open­ing a file with an er­ror in the VS Code code­base would pre­vi­ously take about 17.5 sec­onds from the time you opened the ed­i­tor to the time you saw the first er­ror. With TypeScript 7, it’s un­der 1.3 sec­onds — over 13x faster.

Battle-Tested and Ready for Production

The TypeScript pro­ject con­tains tens of thou­sands of tests built over more than a decade that run on every com­mit on our main branch. They’ve en­sured every one of our re­leases is sta­ble and re­li­able.

But TypeScript 7 is no or­di­nary re­lease. Beyond our test suite, we’ve lever­aged a num­ber of dif­fer­ent re­sources to make sure TypeScript 7 is solid for pro­duc­tion use.

Over the last year we’ve worked with many large teams in­ter­nally and ex­ter­nally to test TypeScript 7 on real-world code­bases. The re­sults have been over­whelm­ingly pos­i­tive, with en­tire com­pa­nies re­port­ing that TypeScript 7 has been sta­ble, fast, and easy to adopt. For ex­am­ple, the VS Code team re­cently high­lighted their ex­pe­ri­ence with TypeScript 7’s pre­view re­leases to move faster in their de­vel­op­ment cy­cle. We’ve also worked with Microsoft teams like Loop, Office, PowerBI, Teams, and Xbox to en­sure that TypeScript is ready for the largest of code­bases. Likewise, com­pa­nies like Bloomberg, Canva, Figma, Google, Lattice, Linear, Miro, Notion, Sentry, Slack, Vanta, Vercel, VoidZero, and more have worked with us to test TypeScript 7 on their code­bases and given us feed­back to make it bet­ter.

Additionally, we’ve re­built much of our broader test in­fra­struc­ture to run on TypeScript 7. TypeScript 6 and ear­lier had au­to­mated and on-de­mand test­ing for TypeScript and JavaScript pro­jects on GitHub to de­tect re­gres­sions in the com­piler and lan­guage ser­vice. The same test­ing is back, and run­ning against TypeScript 7, find­ing is­sues in real code­bases so we can find gaps in our core test suite and ship a bet­ter ex­pe­ri­ence.

The com­bi­na­tion of ex­plicit feed­back, au­to­mated crash re­ports, and ag­gres­sive test­ing has made a mea­sur­able dif­fer­ence in qual­ity. In fact, our data in­sights have shown us that TypeScript 7.0’s new lan­guage server has ac­tu­ally re­duced fail­ing lan­guage server com­mands by over 80%, and re­duced server crashes by over 60% com­pared to that of TypeScript 6.0.

We’ve also heard some in­cred­i­ble feed­back from teams at scale:

Slack en­gi­neers have told us that TypeScript 7 elim­i­nated 40% of their merge queue time and brought type-check­ing time in CI from about 7.5 min­utes to 1.25 min­utes. Local de­vel­op­ment in the ed­i­tor was pre­vi­ously al­most unusable” due to lan­guage server load times and en­gi­neers would typ­i­cally let CI do a full type-check. TypeScript 7 has been able to load the same code­base in a few sec­onds and made lo­cal type-check­ing fea­si­ble again.

Builds at Vanta have dra­mat­i­cally im­proved, show­ing a speedup of up to 9x faster on one of their biggest pro­jects.

Similarly, the News Services team at Microsoft told us that adopt­ing TypeScript 7 saved them 400 hours a month wait­ing for CI builds.

Last year, en­gi­neers work­ing on PowerBI de­scribed TypeScript 7 in the ed­i­tor as life-saving” for work­ing on their code­base. They adopted the ex­pe­ri­ence as a de­fault even be­fore TypeScript 7 sup­ported re­name func­tion­al­ity in VS Code.

Developers work­ing on Loop’s monorepo were also ec­sta­tic. The pre­vi­ous ed­i­tor ex­pe­ri­ence was de­scribed as un­us­able at their scale, whereas the TypeScript 7 ex­pe­ri­ence has been amazing” to use.

Canva de­vel­op­ers have told us that TypeScript 7’s lan­guage ser­vice shows dra­matic speedups, go­ing from about 58 sec­onds to see­ing the first er­ror in their ed­i­tors to about 4.8 sec­onds.

Running Side-by-Side with TypeScript 6.0

While TypeScript 7.0 is here, it does not ship with an API. We ex­pect TypeScript 7.1 to ship with a new (and dif­fer­ent) API, but un­til then we have made it a pri­or­ity to en­sure TypeScript can be run side-by-side with TypeScript 6.0 for util­i­ties that still need some pro­gram­matic ac­cess to the com­piler (such as type­script-es­lint).

As part of the 6.0/7.0 tran­si­tion process, we’ve pub­lished a new com­pat­i­bil­ity pack­age, @typescript/typescript6. This pack­age pro­vides an ex­e­cutable named tsc6, so that if needed, you can in­stall TypeScript 7.0 (which ships its own tsc bi­nary) side-by-side with­out nam­ing con­flicts. The new pack­age also re-ex­ports the TypeScript 6.0 API, so that you can use tsc for TypeScript 7, while other tool­ing can con­tinue to rely on 6.0.

Because some tools like type­script-es­lint ex­pect to im­port from type­script di­rectly via peer de­pen­den­cies, we rec­om­mend achiev­ing this via npm aliases. You should be able to run the fol­low­ing com­mand

npm in­stall -D type­script@npm:@type­script/​type­script6

or mod­ify your pack­age.json as fol­lows:

{ devDependencies”: { typescript”: npm:@typescript/typescript6@^6.0.2”, } }

Note that do­ing this will leave you only with a tsc6 ex­e­cutable. To get 7.0’s tsc, you can add an­other alias for TypeScript 7 and npx tsc will just work with 7.0:

{ devDependencies”: { @typescript/native”: npm:typescript@^7.0.2”, typescript”: npm:@typescript/typescript6@^6.0.2” } }

Nightly Builds and @typescript/native-preview

Until now, most de­vel­op­ers have in­stalled TypeScript 7 via the @typescript/native-preview pack­age. This pack­age shipped nightly builds of the new code­base, and has served the com­mu­nity well with over 8.5 mil­lion weekly down­loads!

However, go­ing for­ward, nightly builds will soon re­sume un­der the stan­dard type­script pack­age with the next tag. You can in­stall it with:

npm in­stall -D type­script@next

Custom Scaling: Parallelization and Controls

TypeScript 7.0 now per­forms many steps in par­al­lel, in­clud­ing pars­ing, type-check­ing, and emit­ting. Some of these steps, like pars­ing and emit­ting can mostly be done in­de­pen­dently across files. As such, par­al­leliza­tion au­to­mat­i­cally scales well with larger code­bases with rel­a­tively lit­tle over­head. But not every step in a TypeScript build is eas­ily par­al­leliz­able.

TypeScript 7 in­tro­duces the ex­per­i­men­tal –checkers and –builders flags to fine-tune the par­al­leliza­tion be­hav­ior for less-triv­ial steps like type-check­ing and pro­ject ref­er­ence build­ing. It also in­tro­duces a –singleThreaded flag to dis­able par­al­leliza­tion en­tirely, which can be use­ful for de­bug­ging or run­ning in en­vi­ron­ments with lim­ited re­sources.

Type-Checker Parallelization

Other steps, like type-check­ing, have more com­plex de­pen­den­cies across files. Most files end up re­ly­ing on the same type in­for­ma­tion from their de­pen­den­cies and the global scope, and so run­ning type-check­ers com­pletely in­de­pen­dently would be waste­ful — both in com­pu­ta­tion and mem­ory. On the other hand, type-check­ing oc­ca­sion­ally re­lies on the rel­a­tive or­der­ing of in­for­ma­tion in a pro­gram, and so type-check­ing from scratch must al­ways check the same files in an iden­ti­cal or­der to en­sure the same re­sults.

To en­able par­al­leliza­tion while avoid­ing these pit­falls, TypeScript 7.0 cre­ates a fixed num­ber of type-checker work­ers with their own view of the world. These type-check­ing work­ers may end up du­pli­cat­ing some com­mon work, but given the same in­put files, they will al­ways di­vide them iden­ti­cally and pro­duce the same re­sults.

The de­fault num­ber of type-check­ing work­ers is 4, but it can be con­fig­ured with the new –checkers flag. You may find that in­creas­ing this num­ber can fur­ther speed up builds on larger code­bases where typ­i­cal ma­chines have more CPU cores, but will typ­i­cally come at the cost of in­creased mem­ory us­age. For ex­am­ple, in the table above, we ran TypeScript 7 with its de­fault of –checkers 4. Here’s what the re­sults look like on the same ma­chine with –checkers 8.

As you can see, these code­bases get a bet­ter speedup from ded­i­cat­ing more cores, but re­sults will dif­fer across pro­jects and un­der­ly­ing ma­chines.

On the other hand, on ma­chines with fewer CPU cores and less mem­ory (e.g. CI run­ners) you may want to de­crease this num­ber to avoid un­nec­es­sary or in­ci­den­tal over­head. You can spec­ify a value as low as –checkers 1, ef­fec­tively mak­ing type-check­ing sin­gle-threaded and elim­i­nat­ing du­pli­cate work.

In rare cases, vary­ing the num­ber of –checkers may sur­face or­der-de­pen­dent re­sults. Specifying a fixed num­ber of check­ers across build en­vi­ron­ments can help en­sure every­one is get­ting the same re­sults, but is up to the dis­cre­tion of your team.

Project Reference Builder Parallelization

TypeScript 7.0 can par­al­lelize builds within a pro­ject, but it can now also build mul­ti­ple pro­jects at once as well. This be­hav­ior can be con­fig­ured with the new –builders flag, which con­trols the num­ber of par­al­lel pro­ject ref­er­ence builders that can run at once when run­ning un­der –build. This can be par­tic­u­larly help­ful for monore­pos with many pro­jects.

Like –checkers, in­creas­ing the num­ber of builders can speed up builds, but may come at the cost of in­creased mem­ory us­age. It also has a mul­ti­plica­tive ef­fect with –checkers, so it’s im­por­tant to find the right bal­ance for your ma­chine and code­base. For ex­am­ple, build­ing with –checkers 4 –builders 4 al­lows up to 16 type-check­ers to run at once, which may be ex­ces­sive.

Unlike –checkers, vary­ing the num­ber of builders should not pro­duce dif­fer­ent re­sults; how­ever, build­ing pro­ject ref­er­ences is fun­da­men­tally bot­tle­necked by the de­pen­dency graph of pro­jects (with the ex­cep­tion of type-check­ing on code­bases that lever­age –isolatedDeclarations and sep­a­rate syn­tac­tic de­c­la­ra­tion file emit).

Single-Threaded Mode

In some cases, it can be help­ful to en­force sin­gle-threaded op­er­a­tion through­out the com­piler. This may be use­ful for de­bug­ging, com­par­ing per­for­mance with TypeScript 6 and 7, when or­ches­trat­ing par­al­lel builds ex­ter­nally, or for run­ning in en­vi­ron­ments with very lim­ited re­sources. To en­able sin­gle-threaded mode, you can use the new –singleThreaded flag. This will not only cap the num­ber of type-check­ing work­ers to 1, but also en­sure pars­ing and emit­ting are done in a sin­gle thread.

Improved –watch Mode

TypeScript 7 ships with a com­pletely re­built –watch mode. –watch is now pow­ered by a new foun­da­tion based on the Parcel bundler’s file-watcher that pro­vides ef­fi­cient and sta­ble cross-plat­form file watch­ing ca­pa­bil­i­ties.

When our team set out to port our file watch­ing logic, we en­coun­tered a few chal­lenges with cross-plat­form file watch­ing in Go. The stan­dard li­brary does­n’t pro­vide a built-in file watch­ing API, and ex­ist­ing third-party li­braries we ex­plored had var­i­ous is­sues with sta­bil­ity, per­for­mance, cross-plat­form sup­port, or is­sues with build tool­ing in­te­gra­tion. We were able to build so­lu­tions around polling pe­ri­od­i­cally to check for file changes, and this worked broadly across op­er­at­ing sys­tems; how­ever it was com­pu­ta­tion­ally ex­pen­sive, es­pe­cially at larger-scale pro­jects with many de­pen­den­cies in node_­mod­ules. Even with dy­namic sched­ul­ing strate­gies, we found that pure-polling so­lu­tions were too tax­ing for gen­eral use.

For many years, Visual Studio Code has re­lied on @parcel/watcher, and in re­cent years TypeScript in VS Code has re­lied on its file watch­ing ca­pa­bil­i­ties in­di­rectly. While it seemed promis­ing, one of the prob­lems for us with Parcel’s watcher is that it’s writ­ten in C++, and in turn re­quires a full C++ tool­chain to build. Given our pos­i­tive ex­pe­ri­ence with Parcel’s watcher in VS Code, we ex­plored port­ing it to Go with a few min­i­mal as­sem­bly shims to avoid in­tro­duc­ing a new tool­chain de­pen­dency.

The ex­plo­ration has been a suc­cess — what started as a very di­rect trans­la­tion from C++ to Go was fur­ther re­fined into id­iomatic Go that still passes the ported test suite. The watcher is a self-con­tained pack­age that has al­lowed us to keep a clean sep­a­ra­tion of con­cerns be­tween what we care to watch and why. We are now see­ing sig­nif­i­cant re­source im­prove­ments in –watch mode across plat­forms, and have been hear­ing pos­i­tive feed­back from ear­lier users of TypeScript 7.

We’d like to ex­tend our thanks to Devon Govett whose work on Parcel has pro­vided im­mense ben­e­fits to both the Visual Studio Code and TypeScript pro­jects. We hope this port will pro­vide op­por­tu­ni­ties and in­sights for the orig­i­nal Parcel watcher code­base over time.

Updates Since 5.x, and New Behaviors from 6.0

TypeScript 7.0 is made to be com­pat­i­ble with TypeScript 6.0’s type-check­ing and com­mand-line be­hav­ior. Practically any TypeScript code that com­piles cleanly with TypeScript 6.0 (with the sta­ble­Type­Order­ing flag on, and with­out any ig­nore­Dep­re­ca­tions flag set) should com­pile iden­ti­cally in TypeScript 7.0.

With that said, TypeScript 7.0 adopts 6.0’s new de­faults, and pro­vides hard er­rors in the face of any flags and con­structs dep­re­cated in TypeScript 6.0. This is no­table as 6.0 is still rel­a­tively new, and many pro­jects will need to adapt to its new be­hav­iors. We en­cour­age de­vel­op­ers to adopt TypeScript 6.0 to make the tran­si­tion to TypeScript 7.0 eas­ier, and you can also read the TypeScript 6.0 re­lease blog post for more de­tails on these dep­re­ca­tions.

At a glance, the no­table de­fault changes to con­fig­u­ra­tion are:

strict is true by de­fault.

mod­ule de­faults to es­next.

tar­get de­faults to the cur­rent sta­ble ECMAScript ver­sion im­me­di­ately pre­ced­ing es­next.

noUnchecked­Side­Ef­fectIm­ports is true by de­fault.

li­bRe­place­ment is false by de­fault.

sta­ble­Type­Order­ing is true by de­fault, and can­not be turned off.

root­Dir now de­faults to ./, and in­ner source di­rec­to­ries must be ex­plic­itly set.

types now de­faults to [], and the old be­hav­ior can be re­stored by set­ting it to [“*”].

We be­lieve the root­Dir and types changes may be the most surprising” changes, but they can be mit­i­gated eas­ily. Projects where the tscon­fig.json sits out­side of a di­rec­tory like src will sim­ply need to in­clude root­Dir to pre­serve the same di­rec­tory struc­ture.

{ compilerOptions”: { // … + rootDir”: ./src” }, include”: [”./src”] }

For the types change, pro­jects that de­pend on spe­cific global de­c­la­ra­tions will need to list them ex­plic­itly. For ex­am­ple,

{ compilerOptions”: { // Explicitly list the @types pack­ages you need (e.g. bun, mocha, jas­mine, etc.) + types”: [“node”, jest”] } }

The dep­re­ca­tions that have turned into hard er­rors with no-op be­hav­ior are:

tar­get: es5 is no longer sup­ported.

down­levelIt­er­a­tion is no longer sup­ported.

mod­uleRes­o­lu­tion: node/​node10 are no longer sup­ported, with nodenext and bundler be­ing rec­om­mended in­stead.

mod­ule: amd, umd, sys­temjs, none are no longer sup­ported, with es­next or pre­serve be­ing rec­om­mended in con­junc­tion with bundlers or browser-based mod­ule res­o­lu­tion.

baseUrl is no longer sup­ported, and paths can be up­dated to be rel­a­tive to the pro­ject root in­stead of baseUrl.

mod­uleRes­o­lu­tion: clas­sic is no longer sup­ported, and bundler or nodenext are the rec­om­mended re­place­ments.

es­Mod­uleIn­terop and al­lowSyn­thet­icDe­fault­Im­ports can­not be set to false.

al­waysStrict is as­sumed to be true and can no longer be set to false.

The mod­ule key­word can­not be used in name­space de­c­la­ra­tions.

The as­serts key­word can­not be used on im­ports, and must use the with key­word in­stead (to align with de­vel­op­ments on ECMAScript’s im­port at­tribute syn­tax).

/// <reference no-de­fault-lib /> di­rec­tives are no longer re­spected un­der skipDe­fault­LibCheck.

Command line builds can­not take file paths when the cur­rent di­rec­tory con­tains a tscon­fig.json file un­less passed an ex­plicit –ignoreConfig flag.

Template Literal Types Now Preserve Unicode Code Points

TypeScript 7.0 now treats Unicode code points more nat­u­rally when in­fer­ring from tem­plate lit­eral types. For ex­am­ple:

type HeadTail<S> = S ex­tends `${infer Head}${infer Tail}` ? [Head, Tail] : never;

type Result = HeadTail<“😀abc”>; // ^ // In 7.0: [“😀, abc”] // Previously: [“\ud83d”, \ude00abc”]

Previously, TypeScript fol­lowed JavaScript’s UTF-16 in­dex­ing be­hav­ior here and split 😀 into two halves of a sur­ro­gate pair (\ud83d and \ude00). That was tech­ni­cally con­sis­tent with in­dex­ing in JavaScript (e.g. the in­ferred Head type was equal to 😀abc”[0]), but it usu­ally was­n’t what peo­ple in­tended, and could pro­duce string lit­eral types con­tain­ing un­paired sur­ro­gates that aren’t se­man­ti­cally mean­ing­ful.

This is a break­ing change for type-level string ma­nip­u­la­tion that in­ten­tion­ally mod­eled UTF-16 code units, such as some string Length util­i­ties. In prac­tice, we ex­pect the new be­hav­ior to be more use­ful and less sur­pris­ing: tem­plate lit­eral in­fer­ence now fol­lows the same in­tu­ition as it­er­at­ing a string with for…of or spread­ing it with […str], where 😀 is treated as one unit.

JavaScript Differences

As we ported the ex­ist­ing code­base, we also took the op­por­tu­nity to re­visit how our JavaScript sup­port works.

TypeScript orig­i­nally sup­ported JavaScript files by us­ing JSDoc com­ments and rec­og­niz­ing cer­tain code pat­terns for analy­sis and type in­fer­ence. Lots of the time, this was based on pop­u­lar cod­ing pat­terns, but oc­ca­sion­ally it was based on what­ever peo­ple might be writ­ing that Closure and the JSDoc doc gen­er­at­ing tool might un­der­stand. While this ap­proach was help­ful for de­vel­op­ers with loosely-writ­ten JSDoc code­bases, it re­quired a num­ber of com­pro­mises and spe­cial cases to work well, and di­verged in a num­ber of ways from TypeScript’s analy­sis in .ts files.

In TypeScript 7.0, we have re­worked our JavaScript sup­port to be more con­sis­tent with how we an­a­lyze TypeScript files. Some of the dif­fer­ences in­clude:

Robostral Navigate: single-camera AI navigation | Mistral AI

mistral.ai

Thinking

Summary

Robostral Navigate is an 8B model that en­ables ro­bots to au­tonomously nav­i­gate com­plex en­vi­ron­ments us­ing only a sin­gle RGB cam­era, achiev­ing 76.6% suc­cess on un­seen R2R-CE bench­marks—out­per­form­ing multi-sen­sor ap­proaches while be­ing more ef­fi­cient. Built en­tirely in-house with sim­u­lated data and to­ken-ef­fi­cient tech­niques, it gen­er­al­izes across ro­bot types and adapts to real-world ob­sta­cles un­seen dur­ing train­ing. The model com­bines point­ing-based nav­i­ga­tion with re­in­force­ment learn­ing for con­tin­u­ous im­prove­ment, paving the way for uni­fied em­bod­ied AI in ro­bot­ics.

Today we’re in­tro­duc­ing Robostral Navigate, our first model built for em­bod­ied nav­i­ga­tion. It’s an 8B model that takes RGB im­ages and a plain-lan­guage in­struc­tion and moves a ro­bot through an en­vi­ron­ment:

Leave the lobby, walk through the cor­ri­dor, en­ter the sup­ply room, and stop to face the sec­ond shelf.”

To per­form such tasks, other mod­els of­ten em­ploy depth sen­sors, LiDAR, or sev­eral cam­eras work­ing to­gether. Robostral Navigate uses only one or­di­nary RGB cam­era and no depth sen­sors, yet still achieves 76.6% on R2R-CE (Room-to-Room in Continuous Environments) val­i­da­tion un­seen, the bench­mark for fol­low­ing in­struc­tions in en­vi­ron­ments held out of train­ing. Consequently, it beats the best sin­gle-cam­era ap­proach by 9.7 points and the best sys­tem us­ing depth or mul­ti­ple cam­eras by 4.5 points, de­spite us­ing nei­ther.

Our model is de­signed for ro­botic nav­i­ga­tion, en­abling ro­bots to au­tonomously nav­i­gate com­plex en­vi­ron­ments, in­clud­ing of­fices, res­i­den­tial and com­mer­cial build­ings, and out­door set­tings.

Robostral Navigate run­ning fully au­tonomously in one long-hori­zon in­struc­tion route through a work­ing of­fice.

This tech­nol­ogy un­locks nu­mer­ous ap­pli­ca­tions across man­u­fac­tur­ing, de­liv­ery, lo­gis­tics, and hos­pi­tal­ity, mak­ing it one of the most in-de­mand ca­pa­bil­i­ties for our cus­tomers to­day. Give Robostral Navigate one in­struc­tion and it com­pletes the en­tire task on its own, mov­ing through a live space full of peo­ple and ob­sta­cles it was never shown, ca­pa­ble of adapt­ing to any set­ting.

Highlights

State-of-the-art per­for­mance on R2R-CE

State-of-the-art per­for­mance on R2R-CE

79.4% Success Rate on val­i­da­tion seen 76.6% Success Rate on val­i­da­tion un­seen

79.4% Success Rate on val­i­da­tion seen

79.4% Success Rate on val­i­da­tion seen

76.6% Success Rate on val­i­da­tion un­seen

76.6% Success Rate on val­i­da­tion un­seen

Operates from a sin­gle RGB cam­era, with no LiDAR or depth sen­sors

Operates from a sin­gle RGB cam­era, with no LiDAR or depth sen­sors

8B model, built in-house and trained en­tirely in sim­u­la­tion

8B model, built in-house and trained en­tirely in sim­u­la­tion

Runs on wheeled, legged, and fly­ing ro­bots, and gen­er­al­izes across ro­bot sizes

Runs on wheeled, legged, and fly­ing ro­bots, and gen­er­al­izes across ro­bot sizes

Robust to dif­fer­ences in cam­era in­trin­sics

Robust to dif­fer­ences in cam­era in­trin­sics

Token-efficient train­ing via pre­fix-caching

Token-efficient train­ing via pre­fix-caching

Navigation via point­ing

Given a task and a his­tory of ob­ser­va­tions, Robostral Navigate pre­dicts where the ro­bot should move next via point­ing: it in­fers the im­age co­or­di­nates of the tar­get lo­ca­tion in the ro­bot’s cur­rent cam­era view, to­gether with the de­sired ori­en­ta­tion upon ar­rival. Unlike com­mands re­ly­ing on met­ric dis­place­ments, point­ing makes the pol­icy nat­u­rally ro­bust to changes in cam­era in­trin­sics and world scale.

However, this method can­not han­dle cases where the tar­get lo­ca­tion lies out­side the cur­rent field of view. When point­ing does not ap­ply, the model falls back to dis­place­ments in the ro­bot’s lo­cal co­or­di­nate frame, such as:

Move 2 me­ters for­ward, 1.5 me­ters to the left, and turn 25 de­grees left.”

Built from the ground up

Robostral Navigate is built en­tirely in-house and does not rely on ex­ist­ing open-source VLMs.

The model is ini­tial­ized from our vi­sion-lan­guage model spe­cial­ized for ground­ing tasks such as point­ing, count­ing, and ob­ject lo­cal­iza­tion. Navigation emerges as a nat­ural ex­ten­sion of these ca­pa­bil­i­ties: once it un­der­stands where things are, it learns how to move.

We built an ef­fi­cient data gen­er­a­tion pipeline en­tirely in sim­u­la­tion. This en­abled rapid it­er­a­tion on the data, re­sult­ing in a dataset of ap­prox­i­mately 400,000 tra­jec­to­ries col­lected across 6,000 scenes.

Efficient su­per­vised train­ing

A key in­gre­di­ent of Robostral Navigate is an ef­fi­cient train­ing al­go­rithm based on pre­fix-caching. Using a tree-based at­ten­tion-mask­ing strat­egy, our method com­presses an en­tire episode into a sin­gle se­quence, en­abling train­ing on all time steps in a sin­gle for­ward pass while pre­vent­ing in­for­ma­tion leak­age be­tween time steps.

Compared to train­ing with one sam­ple per time step, our ap­proach re­duces the num­ber of train­ing to­kens by 22× while pre­serv­ing all of the learn­ing sig­nals. In prac­tice, this method trans­forms train­ing runs that would take months into runs that com­plete in days.

Online re­in­force­ment learn­ing

We lever­age our knowl­edge of post-train­ing LLMs at scale, us­ing on­line re­in­force­ment learn­ing, to boost the per­for­mance of Robostral Navigate. After the su­per­vised train­ing stage, we fur­ther im­prove the mod­el’s per­for­mance us­ing CISPO, an on­line re­in­force­ment learn­ing al­go­rithm. This en­ables the model to learn from trial and er­ror, re­cover from fail­ures, and ac­quire ex­ploratory be­hav­iors, ef­fec­tively mit­i­gat­ing the dis­tri­b­u­tion shift is­sue of vanilla be­hav­ior cloning. This alone im­proved the suc­cess rate by 3.2%. We are not see­ing any plateau­ing, so we are con­fi­dent that more train­ing and more ex­per­i­ments will con­tinue to push this num­ber up.

What’s Next

Robostral Navigate is only the first step to­ward a uni­fied em­bod­ied agent.

We be­lieve nav­i­ga­tion is a foun­da­tional ca­pa­bil­ity for gen­eral-pur­pose ro­bot­ics. By com­bin­ing large-scale sim­u­la­tion, ef­fi­cient train­ing, and strong ground­ing pri­ors, Robostral Navigate demon­strates that state-of-the-art em­bod­ied nav­i­ga­tion can be achieved with a com­pact model and a sin­gle RGB cam­era.

Start your jour­ney to em­bod­ied fron­tier AI, talk with our team.

BTW, we’re hir­ing!

The re­lease of our nav­i­ga­tion mod­els marks a sig­nif­i­cant step for­ward, but our jour­ney is far from over. Our am­bi­tion is to en­able ro­bots to au­tonomously nav­i­gate com­plex en­vi­ron­ments—of­fices, homes, com­mer­cial build­ings, and out­door spaces—and there’s a lot more work to do. We are ac­tively ex­pand­ing our ro­bot­ics team and look­ing for tal­ented re­search sci­en­tists and en­gi­neers who share our am­bi­tion.

If you’re in­ter­ested in join­ing us on our mis­sion to bring seam­less nav­i­ga­tion to ro­bots every­where, we wel­come your ap­pli­ca­tions to join our team!

By Théo Cachet, Arjun Majumdar, Srijan Mishra, Thomas Chabal, Chris Bamford, Elliot Chane-Sane, Benjamin Tibi, Ludovic Ho Fuh, Olivier Duchenne - AI Science Robotics

Eve Online's Carbon engine is now open source: Fenris Creations explains why

www.gamesindustry.biz

In 2024, Fenris Creations — then called CCP Games — said that it was plan­ning to make its Carbon game en­gine open source. Now, some two years later, the tech be­hind the long-run­ning sci-fi MMO Eve Online is avail­able on GitHub for every­one to use.

The open-source pro­ject is some­thing that the com­pa­ny’s core tech team has been work­ing on at a slow burn” for some time now, with the bulk of the work done in the last 12 weeks. Speaking to GamesIndustry.biz, Fenris Creations’ se­nior de­vel­op­ment di­rec­tor for core tech­nol­ogy, Ben Hunter, ex­plains the rea­son­ing be­hind it all. We wanted to get the code out there for in­spectabil­ity and build­ing trust with the com­mu­nity,” he says.

Fenris has a long his­tory of build­ing com­mu­ni­ties and en­gag­ing with them. If you look back to the early days of Eve Online, when we ex­posed our ap­pli­ca­tion pro­gram­ming in­ter­face (API), that was the start of our ef­fort to en­gage with the com­mu­nity and let them build some­thing with it. We ar­rived at this point two and a half to three years ago, where we de­cided there’s noth­ing re­ally spe­cial about our sauce in terms of the ac­tual code. We, and the com­mu­nity, would be bet­ter served by ac­tu­ally get­ting it out there, hav­ing more eyes on it, so that we can ac­tu­ally learn and grow from that, and peo­ple can do crazy things with it, which we’re very ex­cited to see.”

It’s early days at the mo­ment. Hunter says that things are leaning to­wards” peo­ple us­ing Carbon to build within the Eve ecosys­tem. Members of the com­mu­nity have al­ready been sub­mit­ting pull re­quests (PRs) — pro­posed changes to a code­base — for se­cu­rity fixes, and there’s been chat­ter about some­one mak­ing a web app to watch Eve Online con­tent.

We have to see how that man­i­fests, but es­sen­tially, you can build any­thing with it,” Hunter says.

Carbon is avail­able in its en­tirety across a num­ber of dif­fer­ent mod­ules. Most of the tech is un­der the MIT License, a pop­u­lar and per­mis­sive op­tion. Only two mod­ules aren’t un­der that ban­ner: spa­tial au­dio clus­ter­ing is cov­ered by Apache License 2.0, while IO has a Python Software Foundation License.

None of these li­censes has any com­mer­cial el­e­ment; some­one can use all of Carbon for free. They could make their own MMO us­ing the tech, for free. They could even fork off the en­gine and build their own ver­sion, sim­i­lar to how the Linux dis­tro sys­tem works.

But mak­ing money is­n’t the point of this ven­ture.

It’s about gar­ner­ing the ac­tual in­ter­est from peo­ple so that they want to in­vest their time, their ef­fort, their money into con­tribut­ing some­thing,” Hunter ex­plains. It’s this be­lief that ris­ing tides lift all ships. If we im­prove the code and we can all ben­e­fit from it, it’s good for every­one.”

Security con­cerns

Open sourc­ing cre­ates a bit of ex­tra work for the core tech team; they’ve got to han­dle PRs and mon­i­tor the changes. This is some­thing Fenris has slowly been hir­ing to­wards for years now.

We an­nounced our in­ten­tion to open source a cou­ple of years ago, then through­out that pe­riod, slowly ramped up in some of the teams, not specif­i­cally for open source it­self, but rather just to aug­ment the teams so they’d have more band­width to han­dle the me­chan­ics,” Hunter says. We have re­served time dur­ing our sprint process to re­view PRs, process them, and go through every­thing.”

There are many ben­e­fits to open­ing up your tech and let­ting any­one take a look un­der the hood. But bad ac­tors are al­ways out there, look­ing for any ex­ploit they can find. Hunter says that se­cu­rity is absolutely” a con­cern mov­ing for­ward, adding that it’s a pres­sure that en­sures the team increases the ef­fort” they put in when re­view­ing code and mak­ing ar­chi­tec­tural de­ci­sions.

But at the same time, the holes that were there would have been there any­way,” he says. Actually hav­ing the abil­ity to have third par­ties con­tribute to and help us close any po­ten­tial se­cu­rity gaps is very good. To be hon­est, for an en­gine that is 23 years old, the num­ber of se­cu­rity-re­lated PRs that we’ve had is quite min­i­mal. That’s eye-open­ing, in a good way — there’s been a lot of work done over the years. As you can imag­ine, Eve Online has gar­nered a lot of in­ter­est over the years be­cause of the scale of the fleet fights, the bat­tles, and things like that. Nefarious ac­tors, def­i­nitely, in the past, have wanted to probe that and try to dis­rupt it. There’s been a lot of bat­tle-hard­en­ing over 23 years to the in­fra­struc­tural stack of the en­gine and the net­work­ing layer.”

The abil­ity to have third par­ties con­tribute to and help us close any po­ten­tial se­cu­rity gaps is very good”

As well as game­play in­tegrity, there’s also the game’s econ­omy. Eve Online has a com­pli­cated, ro­bust, and in­cred­i­bly valu­able in-game econ­omy, with es­ti­mates sug­gest­ing a trad­ing vol­ume of more than $50 mil­lion per year. That part of the game is­n’t open source. In fact, Hunter says that a great deal of care was taken in de­cid­ing what was marked off for open source and what was not.

We had to make very care­ful con­sid­er­a­tions for what we carved out and what we left off,” he says. Probably the hard­est part of do­ing the open source pro­ject is de­cid­ing what gen­uinely is en­gine ver­sus what is two decades of things that grew up around it, and then also re­build­ing all of the pieces that were mid­dle­ware or li­cens­ing that no longer would be ap­plic­a­ble for open sourc­ing. That’s def­i­nitely been a big part of the chal­lenge.”

Weighting from Godot

To help thread the nee­dle, Fenris asked for ad­vice from Godot — an open-source en­gine pro­ject that started in 2014 and that has seen sub­stan­tial suc­cess in the past few years. Hunter notes that the rise of open, per­mis­si­ble soft­ware is def­i­nitely a trend at the mo­ment.”

The main con­ver­sa­tions were around gov­er­nance mod­els,” he con­tin­ues. We were ex­pect­ing Godot to come in and have this play­book of how to gov­ern an open source pro­ject at scale like that, but in re­al­ity, what it came down to was mak­ing the right ar­chi­tec­tural choices and hav­ing the ar­chi­tec­ture help pro­tect and de­fine the sur­face area for con­tri­bu­tion. If you roll that down, that’s a plug-in model for the en­gine, which is some­thing Unreal and Unity have, and it’s some­thing we are cur­rently im­ple­ment­ing for Carbon at the mo­ment.

We’re mov­ing to this plug-in ar­chi­tec­ture with our tool­ing. It’s also some­thing we’ll be open-sourc­ing in the com­ing months. The biggest light-bulb mo­ment in that con­ver­sa­tion is that we can make some ar­chi­tec­tural de­ci­sions that will help with the ac­tual op­er­a­tional gov­er­nance of the open source pro­ject.”

Regarding Carbon’s gov­er­nance struc­ture, the pro­ject is fully per­mis­si­ble and open. Fenris is ac­cept­ing con­tri­bu­tions and putting to­gether PR tem­plates and con­tri­bu­tion guide­lines.

We ba­si­cally had our code ready to be open-sourced be­fore we had some of the ma­chin­ery in place to process the ac­tual gov­er­nance of it,” Hunter says. That work has all the de­tails of how you can con­tribute, what cri­te­ria you have to meet when it comes to test­ing your work be­fore it is sub­mit­ted, and dis­clos­ing that you have utilised an LLM. We don’t mind you us­ing an LLM, but you have to dis­close it be­cause we may sub­ject it to dif­fer­ent scrutiny than if it were not dis­closed.”

Looking at the broader en­gine ecosys­tem, Hunter de­scribes the land­scape as shifting quite con­sid­er­ably”, point­ing at Epic’s re­cent an­nounce­ment re­gard­ing in­te­grat­ing AI in Unreal Engine 6.

We don’t mind you us­ing an LLM, but you have to dis­close it”

You’re see­ing a lot of the de­vel­op­ers that were tra­di­tion­ally go­ing to Unity or Unreal start­ing to shift gears a lit­tle bit to­wards Godot,” he says. If you look at Epic’s re­cent an­nounce­ments, they are shift­ing things quite a bit to en­sure they can en­ter the LLM era. Unreal’s been chang­ing a lot of its foun­da­tions with the new Verse lan­guage and the Scene Graph re­plac­ing the Actor model to pro­vide more per­sis­tence in large-scale en­vi­ron­ments. That’s some­thing that Carbon has been do­ing at scale for a very long time. It’s some­thing we’ve def­i­nitely seen the value in con­tribut­ing to, and a lot of the other en­gines are start­ing to want to have their own path­way to this as well.

There’s a lot of re-ar­chi­tect­ing go­ing on at the mo­ment, but also the biggest part of this is fig­ur­ing out the most use­ful way to in­te­grate or utilise LLMs for work­flows more than any­thing else. We our­selves also have a tools gate­way that we’ve just cre­ated in­ter­nally for LLM in­ter­faces, which we are rolling out to the game teams in the com­ing weeks and af­ter it’s had a bit of hard­en­ing time, we’ll be rolling that out as open source as well. There’s def­i­nitely a shift in the in­dus­try at the mo­ment.”

Testing in the wild

Looking for­ward, Fenris Creations is go­ing to be de­vel­op­ing Carbon in the open. Anyone and every­one can look, and that’s go­ing to im­pact how the team works.

The biggest thing is that we will be putting a lot more scrutiny into any big­ger ar­chi­tec­tural changes that we make,” Hunter says. That’s some­thing that we now have to have much more con­sid­er­a­tion with the game teams be­cause we will want them to do their own test­ing of that as well.

The other, which will def­i­nitely change now, is we are see­ing the need to cre­ate a test pro­ject. I don’t want to call it a game, but it’s an ex­am­ple game that is more for the testa­bil­ity and un­der­stand­ing the ar­chi­tec­ture and get started quickly in the en­gine. That’s an­other thing that’s chang­ing a lot. We, as the core tech group, will have to cre­ate that test ex­am­ple, and that will be­come our test space, which will be our ex­am­ple pro­ject. Right now, when we make changes to the en­gine, we have to jump into Eve or Eve Frontier and try to test in those game ex­am­ples. We don’t have our own sim­pli­fied ex­am­ple to run through.”

It’s taken a great deal of work to get here, but now Carbon is open source, what does Fenris Creations want for its en­gine over the next five years? Hunter says that he hopes to see a large, Eve-centric” com­mu­nity build up around the en­gine that will create their own aug­mented ver­sions of the game ex­pe­ri­ence”.

If you look at his­tory, when we re­leased the API for Eve Online, we saw var­i­ous side ap­pli­ca­tions that helped you man­age the skills of your char­ac­ter or fit the ship,” he con­tin­ues.

With Carbon, we’ve given the ca­pa­bil­i­ties and tools to the com­mu­nity, and the abil­ity to make that a much richer ex­pe­ri­ence is so much higher. But if you look at the di­rec­tion of what Eve Frontier in par­tic­u­lar is do­ing, where they’re be­com­ing a very open builder game, there’s so much more po­ten­tial there fur­ther down the road. I would say that in five years from now, we’d have quite a large com­mu­nity build­ing a lot of in­fra­struc­ture and apps and ex­pe­ri­ences around the Eve uni­verse, as it were.”

He con­cludes: Of course, any­thing is pos­si­ble with it be­ing MIT-licensed, but I would like to see it con­tribut­ing to the Eve uni­verse as a whole.”

EU now one step away from reviving private message scanning rules

cyberinsider.com

The European Parliament has ap­proved an ur­gent pro­ce­dure to fast-track leg­is­la­tion that would re­vive the EUs ex­pired Chat Control 1.0” rules.

This de­vel­op­ment sets up a de­ci­sive vote on July 9 over whether on­line plat­forms may once again be al­lowed to vol­un­tar­ily scan pri­vate user com­mu­ni­ca­tions for child sex­ual abuse ma­te­r­ial (CSAM).

MEPs voted 331 in fa­vor and 304 against us­ing the ur­gency pro­ce­dure, al­low­ing Parliament to by­pass the usual com­mit­tee stage. The pro­ce­dural vote does not it­self re­in­state the law but ac­cel­er­ates con­sid­er­a­tion of a pro­posal that would ef­fec­tively re­store the tem­po­rary le­gal frame­work that ex­pired in April.

STRASBOURG: EU Chat Control 👁️‍🗨️Just a few mo­ments ago, the European Parliament ap­proved the ur­gent pro­ce­dure for the Chat Control ex­ten­sion (331 in favour, 304 against). On Thursday, MEPs will vote on whether to al­low on­line plat­forms to scan pri­vate mes­sages. pic.twit­ter.com/​vB­gyvFCXYl— Sebastián Lukomski (@lukomski_sebito) July 7, 2026

STRASBOURG: EU Chat Control 👁️‍🗨️

Just a few mo­ments ago, the European Parliament ap­proved the ur­gent pro­ce­dure for the Chat Control ex­ten­sion (331 in favour, 304 against).

On Thursday, MEPs will vote on whether to al­low on­line plat­forms to scan pri­vate mes­sages. pic.twit­ter.com/​vB­gyvFCXYl

The leg­is­la­tion in ques­tion is sep­a­rate from the EUs long-run­ning ne­go­ti­a­tions over the pro­posed Child Sexual Abuse Regulation (CSAR), com­monly known as Chat Control 2.0.” The co­ex­is­tence of two dif­fer­ent leg­isla­tive files has led to con­fu­sion, as one con­cerns the re­vival of an ex­pired tem­po­rary mea­sure while the other seeks to es­tab­lish a per­ma­nent frame­work for de­tect­ing and re­port­ing CSAM.

The tem­po­rary reg­u­la­tion, for­mally Regulation (EU) 2021/1232, orig­i­nally cre­ated an ex­emp­tion to the ePri­vacy Directive al­low­ing providers to vol­un­tar­ily scan pri­vate com­mu­ni­ca­tions for CSAM. The ex­emp­tion ap­plied pri­mar­ily to ser­vices such as Gmail, Facebook Messenger, Instagram Messenger, Skype, Snapchat, iCloud Mail, and Xbox mes­sag­ing, while end-to-end en­crypted ser­vices were gen­er­ally un­af­fected un­less providers chose to im­ple­ment client-side scan­ning.

As pre­vi­ously re­ported, the European Parliament voted in March to re­ject an ex­ten­sion of the tem­po­rary dero­ga­tion af­ter ne­go­ti­a­tions with the Council col­lapsed. The reg­u­la­tion sub­se­quently ex­pired on April 4, 2026, re­mov­ing the le­gal ba­sis that many plat­forms had re­lied upon for vol­un­tary scan­ning within the scope of the ePri­vacy Directive.

Since then, how­ever, the Council of the European Union has sought to bring the mea­sure back through what is for­mally pre­sented as a new reg­u­la­tion con­tain­ing sub­stan­tially the same pro­vi­sions. Former Pirate Party MEP Patrick Breyer, a long-time op­po­nent of the pro­posal, de­scribes the move as an un­prece­dented at­tempt to res­ur­rect leg­is­la­tion that Parliament had al­ready re­jected.

🇬🇧Narrow ma­jor­ity for ur­gency vote on re­in­stat­ing #ChatControl 1.0 mass scans.331 +304 – 11 oA dark day for pri­vacy & democ­racy. On Thursday, there will be the vote on the sub­stance. Only an ab­solute ma­jor­ity can stop it. 🚨https://​https://​t.co/​H0qG­zofi7X— Patrick Breyer #JoinMastodon (@echo_pbreyer) July 7, 2026

🇬🇧Narrow ma­jor­ity for ur­gency vote on re­in­stat­ing #ChatControl 1.0 mass scans.331 +304 – 11 oA dark day for pri­vacy & democ­racy. On Thursday, there will be the vote on the sub­stance. Only an ab­solute ma­jor­ity can stop it. 🚨https://​https://​t.co/​H0qG­zofi7X

According to Breyer’s time­line, the Council ap­proved its ne­go­ti­at­ing po­si­tion on July 2 be­fore Parliament agreed this week to con­sider the pro­posal un­der an ex­pe­dited pro­ce­dure.

The bind­ing vote is sched­uled for Thursday, July 9. Under the pro­ce­dure, op­po­nents would need an ab­solute ma­jor­ity of all Members of the European Parliament, 361 votes, to re­ject or amend the pro­posal. If that thresh­old is not reached, the Council’s text is ex­pected to pro­ceed with­out the Parliament im­pos­ing ad­di­tional safe­guards.

The re­vived pro­posal should not be con­fused with Chat Control 2.0, the per­ma­nent Child Sexual Abuse Regulation that has been un­der ne­go­ti­a­tion since 2022. That pro­posal re­mains stalled af­ter five rounds of tri­logue ne­go­ti­a­tions be­tween Parliament, the Council, and the European Commission.

The main point of dis­agree­ment re­mains whether providers should be per­mit­ted or re­quired to con­duct broad, sus­pi­cion­less scan­ning of pri­vate com­mu­ni­ca­tions, par­tic­u­larly on end-to-end en­crypted ser­vices. The Parliament’s ne­go­ti­at­ing po­si­tion lim­its scan­ning to users or groups specif­i­cally sus­pected of child sex­ual abuse and re­quires ju­di­cial au­tho­riza­tion, while the Council has con­tin­ued ad­vo­cat­ing broader risk-mit­i­ga­tion oblig­a­tions and vol­un­tary de­tec­tion mea­sures that crit­ics ar­gue would still en­cour­age mass scan­ning.

The Council’s own Legal Service re­port­edly warned in June that even voluntary” gen­er­al­ized scan­ning of com­mu­ni­ca­tions may con­flict with Article 7 of the EU Charter of Fundamental Rights un­less sup­ported by rea­son­able sus­pi­cion and prior ju­di­cial au­tho­riza­tion.

As a re­sult, the EU is now pur­su­ing two par­al­lel leg­isla­tive tracks. One seeks to tem­porar­ily re­store the ex­pired vol­un­tary scan­ning regime, while the other con­tin­ues ne­go­ti­a­tions on a per­ma­nent law that could re­shape how on­line plat­forms de­tect CSAM across the European Union.

Thursday’s par­lia­men­tary vote will de­ter­mine whether the tem­po­rary frame­work re­turns while ne­go­ti­a­tions over the much broader and more con­tro­ver­sial per­ma­nent reg­u­la­tion con­tinue.

If you liked this ar­ti­cle, be sure to fol­low us on X/Twitter and also LinkedIn for more ex­clu­sive con­tent.

How to Build a Minimal ZFS NAS without Synology, QNAP, TrueNAS

neil.computer

How to Build a Minimal ZFS NAS with­out Synology, QNAP, TrueNAS

Published on: 2024 – 08-23

If you need a ba­sic NAS and don’t care about GUI fea­tures, it is supris­ingly sim­ple to set up a ZFS dataset and share it over the net­work us­ing Samba.

Scope:

I am us­ing this ar­ti­cle to doc­u­ment it for fu­ture my­self, feel free to adopt it for your needs. Problem with TrueNAS is that it is a full-fea­tured, sup­pos­edly en­ter­prise-grade, soft­ware suite. While it may be sim­ple to set it up (I’ve never tried), I just don’t need any of the bells and whis­tles it of­fers. It’s the mis­match be­tween what I need and what it of­fers; not some­thing in­her­ently wrong with TrueNAS. There is also some­thing to be said about a sys­tem you know every­thing about and not hav­ing to rely on yet an­other thing.

ZFSs best fea­ture that’s never ex­plained or writ­ten any­where

ZFS filesys­tem is self con­tained. If your OS is nuked sud­dently, sim­ply take all disks to an­other ma­chine or in­stall a new OS, in­stall zfs, run zfs im­port and get back your data. This free­dom is un­der­rated and not well un­der­stood. It is also not ex­plained any­where.

It’s worth em­pha­siz­ing: All con­fig­u­ra­tion/​de­tails about ZFS is stored on the disks them­selves. If you’ve setup a RAIDZ2 (Raid 6) with 6 disks, they are self con­tained. Move them to a new ma­chine with zfs tools in­stalled, and sim­ply run zfs im­port. Boom, they’ll show up as RAIDZ2. This is an amaz­ing fea­ture that no mat­ter what hap­pens to the host OS, ma­chine, etc; as long as the disks are not dam­aged, your data is fine.

Step 1. Locate and Organize Disks

List all disks on a linux ma­chine us­ing ls­blk -d -o TRAN,NAME,TYPE,MODEL,SERIAL,SIZE com­mand.

[root@sys ~]# ls­blk -d -o TRAN,NAME,TYPE,MODEL,SERIAL,SIZE TRAN NAME TYPE MODEL SERIAL SIZE sda disk Virtual disk 40G nvme nvme5n1 disk Samsung SSD 990 PRO 4TB XXXXXXXXXXXXXXX 3.6T nvme nvme2n1 disk Samsung SSD 990 PRO 4TB XXXXXXXXXXXXXXX 3.6T nvme nvme6n1 disk Samsung SSD 990 PRO 4TB XXXXXXXXXXXXXXX 3.6T nvme nvme3n1 disk Samsung SSD 990 PRO 4TB XXXXXXXXXXXXXXX 3.6T

These are brand new NVMe dri­ves from Samsung so they should be com­pletely un­al­lo­cated.

The disks are also mapped to an ID, run­ning ls -lh /dev/disk/by-id:

[root@sys ~]# ls -lh /dev/disk/by-id to­tal 0 lr­wxr­wxrwx. 1 root root 10 May 10 09:34 dm-name-rhel-root -> ../../dm-0 lr­wxr­wxrwx. 1 root root 10 May 10 09:34 dm-name-rhel-swap -> ../../dm-1 lr­wxr­wxrwx. 1 root root 10 May 10 09:34 lvm-pv-uuid-DGBpev-Na0C-tY20-YY6E-tpL3-epNA-8T­s3Y0 -> ../../sda3 lr­wxr­wxrwx. 1 root root 13 May 10 09:34 nvme-Sam­sung_SS­D_990_PRO_4T­B_XXXXXXXXXXXXXXX -> ../../nvme2n1 lr­wxr­wxrwx. 1 root root 13 May 10 09:34 nvme-Sam­sung_SS­D_990_PRO_4T­B_XXXXXXXXXXXXXXX -> ../../nvme5n1 lr­wxr­wxrwx. 1 root root 13 May 10 09:34 nvme-Sam­sung_SS­D_990_PRO_4T­B_XXXXXXXXXXXXXXX -> ../../nvme3n1 lr­wxr­wxrwx. 1 root root 13 May 10 09:34 nvme-Sam­sung_SS­D_990_PRO_4T­B_XXXXXXXXXXXXXXX -> ../../nvme6n1

Notice that they are sym­linked to their disk names in /dev/.

We can cre­ate a /etc/zfs/vdev_id.conf that maps an alias to these IDs:

[root@sys ~]# vim /etc/zfs/vdev_id.conf

# Add these lines in the vde­v_id.conf file alias nvme0 /dev/disk/by-id/nvme-Samsung_SSD_990_PRO_4TB_XXXXXXXXXXXXXXX alias nvme1 /dev/disk/by-id/nvme-Samsung_SSD_990_PRO_4TB_XXXXXXXXXXXXXXX alias nvme2 /dev/disk/by-id/nvme-Samsung_SSD_990_PRO_4TB_XXXXXXXXXXXXXXX alias nvme3 /dev/disk/by-id/nvme-Samsung_SSD_990_PRO_4TB_XXXXXXXXXXXXXXX

Run ude­vadm trig­ger to set the alias (or you can re­boot the ma­chine). We can ver­ify that the aliases have been mapped by run­ning ls -lh /dev/disk/by-vdev:

lr­wxr­wxrwx. 1 root root 13 May 10 10:28 nvme0 -> ../../nvme2n1 lr­wxr­wxrwx. 1 root root 13 May 10 10:28 nvme1 -> ../../nvme5n1 lr­wxr­wxrwx. 1 root root 13 May 10 10:28 nvme2 -> ../../nvme3n1 lr­wxr­wxrwx. 1 root root 13 May 10 10:28 nvme3 -> ../../nvme6n1

Alias map­ping is com­pletely op­tional, you can if you’d like use the full ID /dev/disk/by-id/nvme-eui.002538414143c248 of the disk when cre­at­ing the zpool as we will do in the next sec­tion. Using an alias makes it nice. However, please don’t use /dev/nvme1, /dev/nvme2, … as the or­der is not guar­an­teed, es­pe­cially if you mount a new drive to the sys­tem. Creating a vde­v_id.conf en­sures that the se­r­ial num­ber of the drive is tied to the alias.

Remember when we dis­cussed there is no con­fig­u­ra­tion needed on the host OS? /etc/zfs/vdev_id/conf is not nec­es­sary and only used when cre­at­ing a zpool for con­ve­nience. If your OS gets nuked, and you lose vde­v_id.conf, it won’t mat­ter at all.

Step 2. Create ZPOOL

For this tu­to­r­ial, I am cre­at­ing a RAIDZ1 (RAID 5) zpool. That means 1 drive re­dun­dancy in-case of fail­ure. It’s up to you if you’d like ad­di­tional re­dun­dancy, RAIDZ2 (RAID 6) would cer­tainly be more risilient.

First, we need to in­stall zfs on the linux ma­chine. Please re­fer to the OpenZFS doc­u­men­ta­tion on how to in­stall it. It’s usu­ally as straight for­ward as, in my case, dnf in­stall zfs on RHEL 9.

I rec­om­mend set­ting the ashift=12 op­tion when cre­at­ing the zpool as this is your last chance to do so. Most disks re­port 512kB sec­tor size to OS due to back­wards com­pat­i­bil­ity rea­sons, but large disks such as Samsung 990 Pro has a sec­tor size of 4KB or even 8KB. ashift=12 rep­re­sents a sec­tor size of 4KB which will sub­stan­tially im­prove per­for­mance.

[root@sys ~]# ls /dev/disk/by-vdev nvme0 nvme1 nvme2 nvme3 [root@sys ~]# zpool cre­ate -o ashift=12 s16z1 raidz1 nvme0 nvme1 nvme2 nvme3 [root@sys ~]# zpool sta­tus s16z1 pool: s16z1 state: ONLINE con­fig:

NAME STATE READ WRITE CKSUM s16z1 ONLINE 0 0 0 raidz1 – 0 ONLINE 0 0 0 nvme0 ONLINE 0 0 0 nvme1 ONLINE 0 0 0 nvme2 ONLINE 0 0 0 nvme3 ONLINE 0 0 0

er­rors: No known data er­rors [root@sys ~]#

Perfect! I chose s16z1 as the name as it de­scribes the size and type of raid. Most tu­to­ri­als will use tank as the name, as it re­lates to pool. Corny, I re­ject this.

We’re not done yet. zpool is a disk ab­strac­tion, and zfs is the file sys­tem. When we ran zpool cre­ate, it cre­ated a zfs file sys­tem with it.

List all prop­er­ties of the zfs file sys­tem by run­ning: zfs get all s16z1. To check whether our zpool is prop­erly con­fig­ured with ashift=12, we can run:

[root@sys ~]# zdb | grep ashift

ashift: 12

Before we share the sys­tem, con­fig­ure com­pres­sion (optionally) as well as the de­fault mount point.

[root@sys ~]# zfs set mount­point=/​mnt/​s16z1 s16z1 [root@sys ~]# zfs set com­pres­sion=lz4 s16z1

Next, let’s cre­ate a cou­ple of zfs datasets un­der s16z1 root dataset. We will share them us­ing Samba in the next sec­tion.

[root@sys ~]# zfs cre­ate s16z1/​docs [root@sys ~]# zfs cre­ate s16z1/​back­ups

docs for doc­u­ments, back­ups for time ma­chine backup. You can cre­ate as many datasets as you’d like. Try to keep them at the top level. If you’re won­der­ing what is the dif­fer­ence be­tween just a reg­u­lar filesys­tem folder and a dataset—a zfs dataset is a way more than just a folder. You can man­age zil­lion prop­er­ties of a dataset, en­crypt it, send and repli­cate a dataset, take snap­shots, etc.—es­sen­tially, the en­tire ZFS fea­ture set. Therefore, it is a good idea to cre­ate in­di­vid­ual datasets for large cat­e­gories of your files. docs or back­ups is a good ab­strac­tion level for a dataset. If you want to send just docs to a an­other re­mote server as a backup, you can do that with­out send­ing the whole s16z1 root dataset.

We will dis­cuss shar­ing s16z1/​docs as a gen­eral pur­pose share as well as ceat­ing a proper Time Machine stor­age (for Apple sys­tems) by shar­ing s16z1/​back­ups with spe­cial prop­er­ties.

We’ll use Samba for this, and the type of file shar­ing sys­tem is or­thog­o­nal to ZFS. ZFS does­n’t care as long as it is mounted on the host sys­tem.

Install Samba:

[root@sys ~]# apt in­stall samba

Create a UNIX user specif­i­cally for samba, we’ll call it john:

[root@sys ~]# user­add -m john

and cre­ate a UNIX pass­word for john:

[root@sys ~]# passwd john

Next, as­so­ci­ate the UNIX user john to Samba by cre­at­ing a Samba pass­word for john . This pass­word will then be used by hosts con­nect­ing to the share as SYS\john.

[root@sys ~]# smb­passwd -a john

john is also added to the Samba user group. You can ver­ify john as a SMB user by run­ning:

[root@sys ~]# pdbe­dit -L -v john

Unix user­name: john NT user­name: Account Flags: [U ] User SID: S-1 – 5-21 – 880039843-1994218806 – 4034623300-1001 Primary Group SID: S-1 – 5-21 – 880039843-1994218806 – 4034623300-513 Full Name: john Home Directory: \\SYS\john HomeDir Drive: Logon Script: Profile Path: \\SYS\john\profile Domain: SYS Account desc: Workstations: Munged dial: Logon time: 0 Logoff time: Wed, 06 Feb 2036 08:06:39 MST Kickoff time: Wed, 06 Feb 2036 08:06:39 MST Password last set: Fri, 23 Aug 2024 23:46:08 MST Password can change: Fri, 23 Aug 2024 23:46:08 MST Password must change: never Last bad pass­word  : 0 Bad pass­word count  : 0 Logon hours  : FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF

To delete john SMB user, pdbe­dit -x -u john

Now that the UNIX user and Samba user is setup, con­fig­ur­ing Samba ser­vice is ex­tremely straight for­ward.

Edit /etc/samba/smb.conf (delete every­thing and re­place with the fol­low­ing):

[docs] path = /mnt/s16z1/docs browseable = yes read only = no guest ok = no valid users = john cre­ate mask = 0755 [backups] path = /mnt/s16z1/backups read only = no guest ok = no in­herit acls = yes spot­light = yes fruit:aapl = yes fruit:time ma­chine = yes vfs ob­jects = ca­tia fruit stream­s_x­attr valid users = john

Test it out on ma­cOS by mount­ing it with cmd+K in Finder app and us­ing the fol­low­ing url for­mat: smb://​10.0.0.6/​docs and smb://​10.0.0.6/​back­ups where 10.0.0.6 is the IP of the server shar­ing the SMB share.

To test it on Debian sys­tems or sim­i­lar, in­stall apt in­stall sm­b­client and run:

[root@sys ~]# sm­b­client -U john //10.0.0.6/docs -c ls’

Mount the smb://​10.0.0.6/​back­ups and it will show up as a Time Machine share on ma­cOS. Once mounted, start Time Machine back­ups by adding the share to ma­cOS > Settings > General > Time Machine.

That’s all for now. I plan to write an­other ar­ti­cle ex­pand­ing on the en­cryp­tion and very pow­er­ful zfs dataset repli­ca­tion fea­tures.

Cover im­age credit to Oracle ZFS sys­tems. God, it is beau­ti­ful: https://​docs.or­a­cle.com/​cd/​E78901_01/​html/​E78910/​gqtmb.html

To add this web app to your iOS home screen tap the share button and select "Add to the Home Screen".

10HN is also available as an iOS App

If you visit 10HN only rarely, check out the the best articles from the past week.

Visit pancik.com for more.