Annie Sexton
Annie Sexton is a Developer Educator at ngrok with a passion for nerd-sniping developers. She also has over a decade of experience working at PaaS companies such as Heroku, Render, and Fly.io.
10 interesting stories served every morning and every evening.
Annie Sexton
Annie Sexton is a Developer Educator at ngrok with a passion for nerd-sniping developers. She also has over a decade of experience working at PaaS companies such as Heroku, Render, and Fly.io.
For help please visit help.ft.com. We apologise for any inconvenience.
The following information can help our support team to resolve this issue.
A white nationalist with neo-Nazi links who hurled racist abuse at the Indian prime minister during his visit last month is being paid by Facebook’s parent company, Meta, to produce content.
An ABC NEWS Verify investigation into several controversial Australian pages on Facebook has found that some are benefiting directly from social media monetisation programs run by Meta.
Hugo Lennon is a far-right agitator who was formally moved on by Victoria Police for shouting profanities and racist abuse at Narendra Modi at a Melbourne hotel.
He has been receiving revenue through Facebook’s “Content Monetization” program since September 2025.
White nationalist Hugo Lennon being confronted by Victoria Police after yelling racist abuse at India’s prime minister. (Supplied)
Mr Lennon has been photographed with known neo-Nazis and has interacted with them online.
His recent Facebook videos include his outburst at the Indian PM, and others referencing white supremacist concepts like replacement theory and remigration.
The Noticer, a far-right Australian news website that regularly promotes white supremacist and neo-Nazi ideologies, has been making money through the same Facebook revenue program since November 2025.
There is a short period from January 11 to February 15 when it stopped earning money for unknown reasons.
The Noticer is currently suspended from its X account with a notice that the account is “temporarily unavailable because it violates the X Hateful Profile Policy”.
A page promoting the anti-immigration group “March for Australia”, which also has close ties to neo-Nazis, registered for the program in December 2025.
Monica Smit, the founder of the anti-vaccine and anti-lockdown group Reignite Democracy Australia, joined Facebook’s content monetisation program in September 2025.
She appears to have been earning advertising revenue through Facebook from as far back as 2017.
Her page features vaccine misinformation and promotes the sale of “radiation protection” bracelets through her personal website.
Hugo Lennon, Monica Smit, The Noticer and March for Australia did not respond to requests for comment.
Facebook told ABC NEWS Verify “it is not Meta’s role to police offensiveness”. (Reuters: Dado Ruvic/illustration)
Invitation only
Facebook distributed nearly $US3 billion ($4.27 billion) to an estimated 16.2 million monetised accounts in 2025.
Meta has published regular disclosures relating to these “partner-publishers” since 2019.
Tech policy and accountability non-profit What To Fix collates this raw data into a searchable archive that was used by ABC NEWS Verify for this investigation.
According to Facebook, the monetisation scheme that Hugo Lennon, Monica Smit, March for Australia and The Noticer appear to be profiting from is “an invitation-only programme that lets you earn money from the performance of your eligible public reels, photos, stories and text posts.”
What To Fix executive director Victoire Rio said it meant that, in effect, Facebook was in a business relationship with these people and organisations.
“If we consider that Meta is in a direct commercial relationship with its publishers, insofar that they’re paying them royalties and that they have a monetisation agreement with them, then you could argue that they are liable for the content that gets produced by their business partner in some ways.”
Ms Rio said she was no longer surprised by some of the pages Meta was choosing to monetise.
“We’ve actually encountered accounts affiliated with sanctioned entities that were part of those programs,” she said.
“It’s quite common that we see accounts that are very clearly in violation of their policies getting through.
“I think the question here really is: is the quality of the enforcement of those policies that they have.”
RDA founder Monica Smit delivers a speech outside Daniel Andrews’ electorate office on Tuesday. (Facebook)
Policy Violation
The pages examined by ABC NEWS Verify all appeared at times to be in direct violation of Facebook’s own policies around content monetisation.
They state that “debated social issues” such as race “may face reduced or restricted monetisation”, while “misleading medical information” is “ineligible to monetise”.
Independent right-wing extremism researcher Kaz Ross said paying controversial creators was a deliberate strategy from Meta.
“Their financial model is to reward content creators who get engagement, and as we know, the best way of getting engagement is to produce rage bait, extremist material, aggravating material: anything that will get people upset and arguing is the best way to get engagement,” Dr Ross said.
Meta did not answer specific questions about these pages and their content but provided a general statement.
“We have clear policies in place which anyone using our monetisation tools must comply with.“When Content Monetisation Pages violate our Community Standards, we apply penalties such as temporarily or permanently disabling their ability to earn on our platforms.“Creators or publishers who repeatedly breach these policies will be removed from our platforms.”
“We have clear policies in place which anyone using our monetisation tools must comply with.
“When Content Monetisation Pages violate our Community Standards, we apply penalties such as temporarily or permanently disabling their ability to earn on our platforms.
“Creators or publishers who repeatedly breach these policies will be removed from our platforms.”
The company also said it was, “important to distinguish between speech that is offensive and content that could potentially lead to offline violence”.
“Such speech may be offensive to many, but it is not Meta’s role to police offensiveness,” it said.
‘Any amount is too much’
It is unclear exactly how much money these specific creators have made from their Facebook pages as this information is not made public by Meta.
Dr Ross said any amount was too much.
“It’s morally bankrupt to do that, to know that these people are producing content that’s in breach of your own regulations, yet you’re actually not just allowing them, but you’re encouraging them by paying them,” she said.
Meta is also one of the few social media giants that discloses any information about who it is paying. Other platforms, like YouTube, TikTok and Twitter, have very little transparency in this space.
“I think in general, Meta has been more forthcoming with transparency and that’s something that should be lauded,” What To Fix’s Victoire Rio said.
“It’s still a very low bar, and the question here is to really push for all platforms to at least meet that bar and if not actually go higher,′ she said.
It’s 2020. You’re the most senior person on your team, in charge of code quality and architecture. You’ve set up good engineering practices, you thoroughly review PRs from people who are less experienced than you and work hard to maintain a healthy codebase.
Then at some point, you go on holiday. When you come back, the codebase is a mess. Everyone merged each other’s PRs without really paying much attention, someone added a bunch of new tables to the database to denormalise it because it was easier and they added serverless or Kafka to the stack without any solid evidence that they needed either.
It’s okay. You can fix this.
Fast forward to 2026. You haven’t been on holiday. It’s just a normal Monday morning. You make yourself a nice coffee, open your computer and find yourself with 7 PRs to review. You open the first one: +24506 – 3938 lines, accompanied by some AI-generated description of what they’re supposed to do. Somehow, your team has made more changes since Friday than they used to make while you were away for a few weeks.
AI removed the speed limit
AI makes projects with weak engineering culture fail much faster.
There used to be a time when people sat down and talked about how they’d do something. Now they can just prompt an agent for a few hours and open a PR.
The most tragic aspect of this way of working is that, to the untrained eye, it works.
If you pull the branch and test it, you’ll probably get something somewhat functional. So what do they do? They keep going. Again and again. Until the project reaches a point where no one knows how anything works.
Just like someone buying a new luxury car on a credit card. You don’t see the debt. You just see the car that looks great.
But then users start to report a weird bug. It’s the 4th time your team has been trying to fix it. I mean… asking AI to fix it. Unfortunately, it seems like not even Fable can figure it out.
You go talk to the person who worked on this feature.
“So where does the data come from?”
“Hmm… actually I don’t know. Let me ask Claude.”
You sit next to each other watching an endless wall of text appear on the screen. Neither of you has any idea whether any of it is true but Claude seems very confident.
“Let’s just turn on ultracode and ask it to double-check?”
This one will take a while. You start talking about the latest drama on X.
You finally get an answer back.
“Does this make any sense to you?”
“I’m not sure.”
“Didn’t you build this like… last week?”
Silence.
This project has become so convoluted, with so many layers and services, that no one on your team could possibly start to understand what’s going on.
So, what do you do?
Fixing it would require such a colossal amount of work that it would be impossible to even start justifying it to anyone in management.
And what are you even thinking about? It would end up in the exact same state again in just a few months anyway.
“Let’s just ask Claude to fix it.”
“Okay. I’ll create a loop and goal so it doesn’t stop until it’s checked that everything works.”
“Sounds good”
“Actually, I ran out of Fable usage for today so I’ll run it tomorrow”
You grab another coffee and walk back to your computer. You now have 13 PRs left to review. You see something you don’t quite understand, so you message the person who wrote it.
“Why are we doing this here?”
They send you a link. It’s a Claude conversation.
Somewhere in that conversation, buried between Claude confidently recommending one architecture, apologising, changing its mind, your coworker asking it to reconsider again and another 15 rounds of changes, is apparently the design decision behind this code.
“Which part should I read?”
“Probably all of it.”
Does this sound familiar?
Whenever I talk about this, someone eventually tells me that nobody ever fully understood large systems anyway. It’s true.
You were never expected to understand every service and every database. But at least someone did and would explain it to you.
Now they ask an LLM because they don’t actually know themselves.
You can’t afford bad engineers anymore
In every team, there are competent people who make the project possible. There are also people who essentially make it harder for everyone else. And now anyone can produce more code in a day than they used to in a year.
In the story above, everyone is failing:
The engineer opening a 25,000-line PR should have stopped the agent long before it got there. They should have understood what it was doing, broken the work into smaller pieces and questioned every new abstraction it introduced.
The person reviewing it should have refused to review something that large instead of giving in.
The person adding Kafka should have been able to explain exactly why it was needed.
The person who built the feature should have been able to explain where the data came from without sending a link to a Claude conversation.
But what’s the problem then? Just use AI to fix it. Well, it’s not that easy…
Before anyone jumps on this, none of this means technical debt is always bad. The important part is that you know it’s a shortcut.
Anyway, reverting a bad decision is hard. Very hard.
For example, how long would it take an LLM to add a bunch of tables and columns to the database? 10 minutes?
But once you start storing data there, you can’t just remove them. You have to come up with a migration plan, make sure you don’t disrupt the system because people are paying to use this every day. You have to think about what you’ll do if the migration fails. Make sure you don’t end up with orphaned foreign keys. It’s just so much harder to fix. Even with the best model you can get.
And while you’re fixing it, more PRs keep coming in. More code, more abstractions, more decisions. A person can generate 20,000 lines of code in an afternoon, but you still have to sit there and understand what those lines actually do.
By the time you’ve untangled one bad decision, five more have been merged.
The new AI economy
Of course, bad engineers were always a liability.
It has been like this for decades, well before OpenAI or Anthropic existed. Bad decisions compounded, unnecessary complexity accumulated and teams ended up maintaining systems nobody really understood.
The difference is that there used to be a limit to how fast you could do it.
Today, implementation is cheap. You are paid to make good decisions. To build software that will scale while managing complexity.
Ask yourself why companies are paying six-figure salaries for engineers in London or San Francisco in the first place.
If all they needed was someone who could turn a specification into working code, why were they paying that much when they could already get it done cheaply elsewhere?
Why are the tech companies claiming that “software is solved” still paying top salaries to attract the best people they can?
My bet is that AI pushes salaries further apart. To be employable, there’s a bar you have to clear and that bar is whatever the current best model du jour can do.
Good engineers have become more valuable because AI lets them move much faster. They don’t need as many people around them just to do the implementation work anymore.
At the same time, bad engineers have become much more expensive to hire.
I wrote about this before when I said the vibe coder career path is doomed.
You need to contribute beyond what everyone already gets by giving an agent a prompt.
If you lack the judgment required to evaluate the LLM’s recommendation, asking for more judgment doesn’t solve the problem.
At some point, someone still has to know what is going on. And that’s the most valuable person on the team.
The people who don’t will become much cheaper to hire or get replaced entirely while the money gets funnelled towards an increasingly smaller number of people who can actually be trusted.
I don’t think this is going to be limited to software engineering either. I believe the same thing is going to happen across most knowledge work. AI will make the best people much more productive and the bad ones almost impossible to hire. Before, there was a good chance someone would catch their bad decisions before they went too far. Now they can make changes faster than anyone around them can realistically review or understand them.
At the end of last year, our uptime was pretty shaky. You can see this trend on our status page, and that instability continued into the new year. Many of these outages were caused by a single bug, deep in SQLite. It took months of intense forensics to track it down.
Now we’re in summer, we’re confident that we’ve found the bug, that we understand it—and more importantly, that we’ve fixed it.
We know our customers expect Tailscale to be a reliable service, and for several months we didn’t live up to that promise. That’s disruptive, and we’re sorry. We’re publishing this blog post to explain what went wrong, how we responded, and how we ultimately helped to uncover a long-standing bug in the heart of the SQLite database.
Tailscale’s database architecture
While our clients interact with our control plane as a single public endpoint (controlplane.tailscale.com), internally, our control plane is split into a series of coordination servers (or “shards”). Each tailnet lives on one internal shard at a time, but can migrate seamlessly from one to another. These shards are an internal implementation detail: you don’t know what shard your tailnet is on, and you never need to.
Each shard has an SQLite database that holds all the information about the tailnets on that shard. A single Go process exclusively accesses that database, and serves the control plane for those tailnets. This single-writer design is exactly how SQLite is meant to be used.
We’ve used SQLite as our primary database since 2022, and we chose it because it’s well-known, reliable, and widely used. SQLite is “boring technology”—in a good way. Many companies use SQLite in much larger deployments without issue, and we expected the same stress-free usage.
In our current backup pipeline, we take a complete snapshot of the database every few minutes, then upload the entire SQLite file to an S3 bucket. We’d been running this setup without incident since early 2023.
Fast forward to August last year, when a data pipeline that reads those S3 backups reported an error in one of our databases. We ran SQLite’s PRAGMA integrity_check command against the backup, and found it was indeed corrupted. SQLite corruption is possible, but it’s highly unusual and not something you should encounter in normal operation. We repaired the affected database, and investigated the cause, but to no avail.
When operating at scale, even rare events can occur with some frequency, so we should have been unsurprised when it happened again—and again, and again, and again. In total, we faced 19 separate instances of database corruption over six months before we finally resolved the underlying bug.
When you hear the phrase “database corruption”, it’s natural to worry about data loss. Because our control plane only handles configuration data, these databases contain metadata about your tailnet and devices, but never your private encryption keys or network traffic. In the earliest incidents, the recovery process meant a handful of newly added devices or configuration changes didn’t persist, and a small amount of metadata had to be re-entered.
Whenever corruption occurred, we had to stop the control plane process on the shard while we repaired or restored the database. This was painful for tailnets on that shard, because their entire control plane disappeared during that recovery window. In the early incidents, that downtime was over an hour, but we gradually sped up the recovery process over subsequent incidents.
Each tailnet is a mesh network, where devices make peer-to-peer WireGuard® connections to each other. When a device joins the tailnet, it has to get a list of other devices from the control plane before it can establish new connections—so if a device came online during the SQLite downtime, it couldn’t connect. While the database was being repaired, devices already online remained connected to each other, but they couldn’t learn about changes to the network. Those tailnets also temporarily lost access to the web-based admin console and the Tailscale API.
There’s also a broader impact on trust. We post a global incident on our status page even when only a small number of tailnets are affected. Many people saw a status page event for an incident that didn’t affect them. Indeed, the majority of shards and tailnets were never involved in a database corruption incident! Nonetheless, repeated downtime erodes trust, whether or not you’re directly affected.
From the very first instance of corruption, we knew this was a serious threat to our reliability, and we threw a lot of engineering time at the problem—but the fix wasn’t easy.
Trying to find the fault
This bug resisted all our initial attempts to find it.
We looked at recent changes, but there weren’t any that seemed relevant. Nobody had been working on our low-level code that interacts with SQLite, because it had all been written years ago and presented no issues up until that point. We re-reviewed all of that code with a fine-toothed comb to look for previously missed bugs, but we didn’t find anything that would cause the corruption we were seeing.
We looked for common factors between corruption incidents, but we couldn’t find any. It wasn’t tied to a single shard, or customer, or tailnet feature, or time of day, or load level. We were at a loss for what might be triggering the behaviour.
This lack of reliable trigger conditions meant we couldn’t reproduce the bug synthetically. Instead, we had to rely on deploying passive, forensic telemetry in our live environment to catch the corruption red-handed. Gathering live diagnostics for a database issue is the last thing we wanted to do, but we had no choice.
As an additional complication, the corruption didn’t occur on a regular schedule. Sometimes incidents would be hours apart, other times weeks. This made it difficult to predict progress or plan further work, because we were never sure when we’d get our next diagnostic dump. We had a six-week period between October and December when there were no corruption incidents, before they returned as an unwelcome Christmas present.
Because this wouldn’t be a quick or easy fix, we reached out to the SQLite developers for a professional support contract. This was a great decision. It gave us direct access to their deep expertise and experience, and we had many detailed technical conversations about our architecture and our incidents.
Between Tailscale engineering and the SQLite core developers, we mapped out several theories for what might be causing the corruption—including broken POSIX locks on close(), mismanaging memory owned by SQLite, or accidentally using SQLite from multiple threads while disabling thread safety. After every incident, we gathered more data, added more diagnostics, and systematically ruled out these theories. We were gradually converging on the true bug.
The transactions that didn’t bark
While we were investigating the root cause, we still had a live platform to run. We took aggressive steps to automate recovery and minimize downtime:
Configuring our control plane shards to hard-stop immediately upon encountering corruption
Deploying an automated backup monitor that continuously ran PRAGMA integrity_check over our backups
Improving our runbooks and on-call training
These efforts cut our response time to under an hour—and then we discovered an unexpected clue.
We wanted a way to restore service that didn’t involve rolling back to the last known-good backup (which would lose a lot of data) or repairing the known-corrupted database (which was potentially risky).
To do this, we built a transaction logging pipeline. We streamed every SQL statement that modified the database to a separate log file. Because SQLite is a single-writer database with serialisable transactions, our transaction history was completely linear and deterministic. (This wouldn’t be true in a multi-writer database like Postgres or MySQL.) Replaying those transactions against the latest known-good backup should restore the database to its most recent state, safely bypassing the corruption.
This pipeline worked, but then it did something even better: it gave us a clue.
In two incidents, our transaction logs failed to replay cleanly. Upon closer inspection, we discovered that data written and committed by one transaction was inexplicably invisible to later transactions. A write had vanished into thin air without raising an error. That should be impossible!
The writing on the WAL
As these incidents were ongoing, the SQLite developers had been developing a new debugging tool. For a while, we’d suspected that the bug was somewhere in the checkpoint process. They were building a new tool to give better visibility into what was happening during checkpoints.
To understand what this tool found, we need to briefly explain how SQLite checkpoints work.
A SQLite database is made of a series of “pages”, tiny blocks of information. When you update the database, some of those pages need to be replaced with new pages with the updated information.
For better performance and greater concurrency, we run SQLite with Write-Ahead Logging, which means new pages aren’t written directly to the database file. Instead, they’re written to the “write-ahead log” or “WAL file”.
New pages can’t be written to the WAL file indefinitely; at some point they have to be copied back to the main database file. This process is called “checkpointing”.
In most deployments, SQLite itself decides when to do a checkpoint, and the process is invisible to the end user and developer. In our control plane, we take manual control of the checkpoint process so we can run fast and consistent backups. This non-standard approach seemed suspicious as we steadily eliminated potential causes.
One clue was that during corruption incidents, our metrics showed that SQLite would report copying more pages from the WAL file than were actually available. If there are 10 pages in the WAL file and 20 pages get copied to the database, something is clearly wrong.
To understand what was happening during these faulty checkpoints, the SQLite developers created a new debugging tool for the virtual filesystem layer.
SQLite is split into several layers. The top layer is the parser and code generator, which converts SQL statements into SQLite’s internal data structures. These data structures get passed to the pager, which splits them into the individual pages to be written to disk. Actually writing them to disk is handled by the OS interface, or “virtual filesystem”. Currently SQLite has two mainstream virtual filesystem implementations—Unix and Windows.
If you’re interested in a deeper dive on these internals, I recommend this lecture by Richard Hipp, the primary author of SQLite.
This approach allows you to replace different layers with different implementations, or wrap an existing layer to get more information. To help diagnose our problem, the SQLite developers created a wrapper around the virtual filesystem that writes additional tracing information and logs about changes to the database. This wrapper is called the tmstmpvfs shim, and the source code is available in the SQLite public repository.
We deployed the shim into our live environment, and waited for the next corruption to occur. Fortunately, we didn’t have to wait long.
The WAL-Reset bug
After our next corruption incident, the additional logs from the new tmstmpvfs shim allowed the SQLite developers to find and fix the bug: a rare data race in the SQLite source code between a checkpoint and a write transaction.
In particular, if a write occurs at a specific time during a checkpoint, the checkpointing process gets confused—it thinks some of the pages have been copied from the WAL into the main database file, but they haven’t. Those pages never get written to the database file, and that data is permanently lost. The database file becomes corrupt, because other pages which reference those pages—such as an index—are written to the database.
The SQLite developers named this the “WAL-Reset bug”, and they estimate it was present in SQLite for at least 16 years. It could exist that long because it was rare—so rare, the SQLite developers had to add code to deliberately trigger it in their testing environments. Their fix adds an additional check to the checkpointing function which detects when the WAL has been reset by another thread.
They confirmed that this bug caused all of the baffling behaviour we’d seen. It explained the corruption, the transaction logs that wouldn’t apply cleanly, and the inconsistent checkpoint statistics. They also explained why we were more likely to hit the bug than other SQLite users: we take manual control of the checkpointing process, and we checkpoint very aggressively. Even a bug triggered by a rare condition was bound to hit us eventually.
This was an exciting moment. After months of confusion and uncertainty, we finally had a plausible theory for why the corruption was occurring, and a fix we could deploy to prevent it.
The SQLite developers released the fix as SQLite 3.52.0, and we prepared to deploy it as soon as it was available.
Fixed, with a false alarm
We rolled out SQLite 3.52.0 carefully—first to a few canary shards, then, when we saw it running smoothly, we deployed it to the rest of the control plane.
Our backup monitor promptly turned red, and reported corruption in 13 different databases. This was extremely alarming, but we followed our recovery procedures to fix all the supposed corruption, and everything was happy. It turned out these databases had not suffered real corruption, but were subject to a second problem in the version of SQLite.
We shared our errors with the SQLite developers, which uncovered a bug in SQLite related to stale expression indexes. If you create an index on a computed value, and then the computation changes, the index will contain mismatched values, which gets reported as corruption by PRAGMA integrity_check.
In our case, we were storing some high-precision timestamps as text, converting them to a floating-point number in a VIRTUAL generated column, and the SQLite 3.52.0 release that fixed our data race also made an optimisation that subtly changed the rounding behaviour for text-to-floating-point conversions. Our canary shards didn’t have any timestamps that triggered the changed rounding behaviour, so we missed this in our phased rollout.
Because this change caused false corruption warnings, the SQLite developers withdrew the 3.52.0 release and instead published 3.51.3, which only contained a fix for the WAL-Reset bug.
We fixed the issue on our side by reducing the precision of our timestamps to integer seconds; text-to-integer conversions are unambiguous. Meanwhile, the SQLite developers created an automated, self-healing index feature in 3.53.0, which prevents the stale expression index problem.
Party time!
With the fix rolled out to our entire control plane, we were ready to declare victory, but we were still cautious. An absence of corruption incidents doesn’t mean things are fixed—we’d already had one six-week period of deceptive calm.
We wanted positive proof that this data race was actively occurring in our production environment. Now that we understood the cause of the bug—a collision between a write transaction and a WAL-reset—we patched our SQLite driver to log a warning when these two operations overlap. If the warning fired but the database remained uncorrupted, we’d know the fix had saved us from a potential corruption incident.
We deployed the warning, and we waited. And we waited. And waited. And waited. As weeks slipped by, we began to wonder why we didn’t see it. Was the warning broken? Was our theory wrong? Was the true bug still lurking in the darkness?
Then, two months later, the alert we were waiting for finally fired:
This alert proved that the precise conditions for the WAL-Reset bug do occur in our production environment, which means it was the likely culprit for our six months of shaky uptime.
Since that weirdly joyous alert fired, we’ve run for another four months without any database incidents, as of this writing. Finally, we could breathe a sigh of relief.
Off the well-trodden path
Nobody wanted us to spend six months looking for bugs in SQLite. This was an immensely frustrating experience for both our customers and staff, and we’re all glad to put this instability behind us.
This investigation is a useful reminder: running boring technology in a non-standard way is a risk. The common paths and standard configurations are incredibly well-tested and reliable. Most people use SQLite in a standard configuration and never face this sort of issue. Everything we were doing was a public, documented, supported configuration—but by taking manual control of the checkpointing process and running at our own aggressive pace, we stepped off the well-trodden operational path.
Resolving these incidents was a massive, cross-functional effort involving dozens of people—including Tailscale’s engineering and support teams, and the core maintainers of SQLite. It is to all of their credit that the impact of these incidents was not much worse.
We know that repeated downtime erodes trust, no matter how many people are affected, and we’re grateful to our customers for their patience and support while we chased this down.
Frustrating as this period was, we’re left in a stronger position than we were before. The long-standing bug in SQLite has been patched, and we fixed dozens of other incidental issues that we spotted while looking for it. We funded the open-source SQLite VFS shim that helped isolate the race condition almost immediately, and will help track down similar bugs in the future. Finally, we’ve refined our database backup and recovery processes, and live-tested them over a dozen times.
Hopefully there won’t be another database incident like this—but if there is, we’ll be ready.
Today, the Mojo language officially reaches 1.0: a milestone the language has been building toward since its first release in 2023. Mojo has grown into a general-purpose language with a vibrant developer community writing their own libraries, tools, and applications on top of it. With Mojo 1.0, developers can now build for the long-term on a stable, production-ready language foundation.
Mojo 1.0: A stable foundation for ecosystem growth
Modular has rapidly evolved the Mojo language through extensive internal use. But that pace of progress has come with a tradeoff: frequent changes have made it difficult for the community to maintain long-term projects.
As we stated when we first announced the path to Mojo 1.0, its primary goal is to provide a stable foundation developers can build on. We are making that commitment today because Mojo is ready: it is no longer just a language we are developing; it is a language we rely on every day in production as the foundation of our commercial infrastructure, MAX and Modular Cloud.
Importantly, Mojo 1.0 does not mark the end of the language’s evolution, but it is an important milestone on a longer journey. During the 1.x timeframe, changes should primarily be additive, giving developers confidence that the language will not continually shift beneath them. Breaking changes may still be made, but will be managed with care, following the standards of how mature languages (e.g. C++) evolve over time.
Yet, this milestone belongs just as much to our incredible community as it does to us. Since we open-sourced the standard library, nearly 200 contributors have landed more than 1,100 pull requests, changing over 200,000 lines of code, and more than a thousand others have filed issues that shaped the language. To every developer who filed an issue, opened a pull request, wrote a language proposal, or built a package: thank you for being the architects of this language alongside us.
Mojo improvements in 26.5
Much of this release is focused on completing the work required for Mojo 1.0 — a throughline across our last several releases as we’ve worked to make the language more consistent, predictable, and approachable.
Where Mojo offered multiple ways to express the same idea, we’ve converged on one. Variables are now consistently declared with var, closures have been unified, there is a single Pointer type, and a number of renamings have made the Mojo lexicon more precise and consistent.
This release completes that final round of language simplification and cleanup, giving Mojo 1.0 the stable, coherent foundation we want developers to be able to build on for years to come.
Beyond this foundational work, Mojo 1.0 also includes several new features and improvements since the last beta release:
Mojo now supports Python-style “lambda” syntax for inline closures.
The Mojo LSP server is far more stable and reliable, greatly improving your everyday experience with VS Code and other editors.
The Mojo AI Skills are now “1.0 ready”, covering new project creation, GPU programming, porting from other languages, etc.
Mojo now diagnoses memory safety problems involving reference invalidation, e.g. noticing when List.append invalidates a reference into the list.
“where” clauses are more consistently used across the standard library, and allow a descriptive message to make failures more actionable.
These are only a few of the highlights. See the full Mojo changelog on mojolang.org for the complete list of changes.
Where Mojo goes from here
Mojo 1.0 is a major milestone, but there’s so much more we are planning for the language. Mojo has already established itself as a powerful language for writing high-performance code across modern CPUs, GPUs, and accelerators. The next phase of its evolution is to broaden that foundation and make Mojo a truly great general-purpose systems programming language.
That means continuing to invest in the core language and developer experience, with major capabilities ahead including a robust asynchronous programming model, pattern matching and unions, and much more. You can see what we are working toward in the Mojo roadmap.
Finally, we will continue to progressively open-source more of the Mojo language, as well as components in MAX that we have built with it. Our commitment remains unchanged — we will open source the Mojo compiler and toolchain in 2026.
MAX enhancements in 26.5
While Mojo 1.0 is the highlight of this release, 26.5 brings improvements to MAX, too.
Installing MAX is now easier: use max[“serve”] and max[“benchmark”] (max-serve and max-benchmark with conda) to install only the dependencies you need, or max[“all”] to install everything. The modular package will be retired in 26.6.
MAX also adds support for two new model families: GLM-5.2 and Nemotron-H, both hybrid Mamba-2 models. And Kimi 2.5 now works with Module V3, our streamlined model-authoring path.
Last, our collection of open source agent skills is a great way to get started with this release. We’ve used these skills internally to speed up full model lifecycle bring-up, and they’ve picked up 7.2K+ downloads through skills.sh.
For the full list of updates, see the MAX changelog.
Get started with 26.5 and Mojo 1.0
Install or upgrade to get started in minutes:
bash
uv pip install –upgrade mojo
uv pip install max[all]
Mojo changelog
MAX changelog
mojolang.org
GitHub
Modular forum
1.0 is just the beginning, and we’ll share more on our plans for Mojo, MAX, and open source at ModCon on August 18th in San Francisco. Tune in virtually via the livestream or join the in-person waitlist.
🌑 Totality begins in …
📹 Reaches first webcam in …
made by jonty
So while I work with police departments regularly, I think it is critically important that technology be used reasonably.
While this may be off-putting to some of my clients, I worked with the Institute for Justice as an expert witness in their trial Schmidt v City of Norfolk. (Any opinions herein are my own and not those of IJ, to be clear.) The gist of that case was whether searches of historically cached ALPR data (automated-license-plate-reader) constituted an illegal search.1
The judge ruled against plaintiffs in that case. Here is a quote from the judgment:
Consistent with Plaintiffs’ claims in this case and controlling precedent involving mass surveillance in public spaces, ALPR surveillance could become too intrusive and run afoul of [constitutional privacy standards] at some point. But when? While a definitive answer to that question is elusive, what is readily apparent to this Court is that, at least in Norfolk, Virginia, the answer is: not today.
Consistent with Plaintiffs’ claims in this case and controlling precedent involving mass surveillance in public spaces, ALPR surveillance could become too intrusive and run afoul of [constitutional privacy standards] at some point. But when? While a definitive answer to that question is elusive, what is readily apparent to this Court is that, at least in Norfolk, Virginia, the answer is: not today.
The important point to note about this quote is “not today”. This will be a long winded post, but to try to keep it simple:
I think cameras will become ubiquitous in the foreseeable future. So the question is not if this data will require a warrant, it is when. It is going to happen eventually under current case law.
I think cameras are good, and can be used to reduce crime in a cost effective manner.
There is a difference between active flags (e.g. this car is stolen and it pings the PD when it drives past a camera) vs historical searches (e.g. look to see where license plate XYZ1000 was the last 30 days).
Requiring a warrant for historical searches will not seriously impede police investigations.
The current status quo of not retaining data is VERY BAD; it does not prevent illegal searches, and currently limits the utility of actually using that data for legitimate investigations.
Current standards to prevent abuse of the searching ALPR data systems are laughable.
Long story short in my opinion everyone would be better off if states just mandated warrant procedures through state statutes.
Current Legal Standards
To try to not get too much into the weeds of what historically constitutes a search, I think the easiest place to start is via Carpenter vs US. So current US case law requires police departments to obtain a warrant to request cellular providers provide law enforcement with cell phone tower pings (cell-site location information, CSLI).
This deviated from historical precedent in requiring a warrant mainly because it was private companies that had the information. Before Carpenter, mostly it was argued you did not have a reasonable expectation of privacy if a private company could access the same data. The court in Carpenter basically made a determination that cell phone data was so comprehensive it justified a different standard — that you could track the whole of a person’s movements with the detailed CSLI data. And that this level of invasiveness violated a reasonable person’s expectation of privacy. Even if Google had all that info, you did not expect them to give it away.
This opinion was reaffirmed with the recent Chatrie decision (for geofence warrants, e.g. give me a ping for all cell phones in area X and datetime-range Y). Another relevant decision to be aware of is also Beautiful Struggle v Baltimore, in which searching historical aerial imagery via drones also constituted a search.
So this is why I am saying the question is when, not if, ALPR data will require a warrant. If a city happened to have a camera on literally every intersection (which I think will happen in the future), under current case law it would clearly be the same situation as you have for your cell phone data.
Cameras are Good
To be brief, again I mostly work with police departments in my career and was a former crime analyst. I do think ALPR cameras are good investments, mainly because they are cheap enough to have a reasonable return on investment. (Note I do not think this about all police tech, I am particularly critical of the price tag for acoustic-gun-shot-detection.)
So ALPRs are well under $3,000 per camera. The machine learning models, camera, and computation necessary to flag a plate when it passes can easily fit on current cell phones. (The harder part is powering the phone and protecting it from the elements.) ALPRs for the most part just take static images and then extract out the license plate (and for some vendors extract out additional information, like car make and color).
The overall evidence that ALPRs reduce crime is pretty meh at the moment (see my slides at a Wake Libertarian talk I did in 2024), but because they are so cheap they really only need to increase a few arrests per camera to likely have a positive return on investment.
It is pretty hand-wavy, as we do not have estimates for the value of increased clearances I find persuasive. But I think saying “I would pay $500 to help solve one case” is on the low side if anything. So a single camera if it helps catch just a handful of crimes a year is likely in my opinion to be a positive ROI.
I think cameras in all public spaces are going to happen. Imagine Ring comes out with a nicer camera system for homeowners that has more comprehensive views around your house and is just as cheap. And we will ultimately be safer for it. So even for folks advocating that cities do not pay for Flock, this is coming anyway in the near future.
Historical Searches vs Active Flags
ALPRs have been around a long time. The first ones I worked with at Troy, NY when I was an analyst were in-car cameras. Basically a go pro attached to the window that alerted when an officer drove by a stolen plate.
While ALPRs initial use was always pitched as this active flagging of stolen vehicles, they were used right away to retroactively search the historical locations of plates. They had a log of every plate, lat/lon, and timestamp of when that car passed a camera.
So imagine you are conducting an investigation of Joe Schmo, you know his license plate, and then you can type in his plate and see where his car passed a camera. Based on this information, same as CSLI data, you can basically trace where Joe went, where he repeatedly visited, where he likely slept, etc. (The first time I used this at Troy, we figured out a particular individual we were actively investigating was living with his girlfriend for example. I was honestly amazed how densely filled in the map was of hits for a single plate based on the in-car cameras.)
You technically do not need to cache any data at all to accomplish this “flag a stolen vehicle” (or any other scenario where you are actively looking for a specific license plate). There are legitimate scenarios though where ALPR searches for recent data in a real time context can be very helpful.
One of the more common examples — someone robs a gas station, and they drove a vehicle. You don’t know the plate, but can look at the images that passed by the fixed location ALPRs in the time range, and then especially if you have a car description from the gas station attendant can figure out the plate associated with the vehicle.
To be clear I am not a lawyer, but in my opinion I think exigent circumstances make searching a few minutes of cached ALPR location data totally reasonable. In practice, New Hampshire’s 3 minute data retention is far too short. I could see arguments for several hours (imagine “I found a dead body on the side of the road”, that requires more time for it to be reported.) But we are meandering into the territory where it is not an active emergency “need to find someone who may have a gun and hurt people” that would justify those exigent circumstances. Those are the scenarios where getting a warrant is reasonable (no different than a geofence warrant if you do not have a plate and want to just search what cars passed by a camera within a certain date-time window, or no different than a CSLI warrant if you have an active suspect and want to search for a specific license plate).
Most states are retaining ALPR data for longer periods. While the Norfolk case was ongoing, Virginia set a standard across the state at 21 days. Before that it was up to the individual agency. It varies state by state, but states often mandate data retention around 30 days, or leave it up to the discretion of the police department.
Deleting Data does not prevent abuses
These data retention statutes are argued as a mechanism to prevent abuse. They do not accomplish this.
If you look through the cases in which officers abused the system to search for individuals, all of them searched for specific plates over-and-over again, sometimes hundreds of times.
If you retain data for 20 days, you can just go and do a search every 20 days, keep notes on the data as you so wish, and then do another search 20 days later. Getting rid of old data, in-and-of-itself, does nothing to prevent that abuse. In fact if someone is actively stalking a person, you would expect them to regularly do searches, seeing where their victim is going on a regular basis while they have access to the system.
Simultaneously, deleting data does prevent its legitimate use in long term law enforcement investigations. It is totally normal for a murder investigation to take more than 30 days to identify a suspect. Gosh, sure would be nice to be able to then query the ALPR data to show whether a person was in the vicinity of the murder. Simultaneously it could be used by the defense for exculpatory purposes (which assuredly would take longer than 30 days).
So folks advocating for deleting data as a mechanism to prevent abuse are making things worse. It does not prevent abuse, and limits the utility of ALPR for historical investigations. The only way data retention by itself prevents abuse is if you do not cache data at all (like in New Hampshire), and only use ALPRs for the active alert situation.
What Smart Regulation Looks Like
One of the reasons I say that the current standards to prevent abuse are laughable is that data retention policies and internal PD policies on when the data should be searched have been in place in most departments for years (if not a decade) at this point. The examples where searching ALPR data to stalk an intimate partner were obviously not prevented via data retention policies.
Alas, my suggestion that some data is cached for real time investigations (longer than 3 minutes), and that a warrant should be required outside of this window, does not prevent that type of abuse either. Most departments have in place reasons why a search can be conducted, and some states have specific statutes identifying impermissible reasons for conducting searches. In the Norfolk IJ case, officers, when entering a reason for a search (which was often omitted), sometimes supplied reasons that appeared prima facie illegal, such as “protest”.
Departments, even if they have a standard to do internal audits, often do not follow them. It took Tyler Dukes asking Raleigh PD for their audit results for them to even conduct their first audit.
This is a long standing problem for PDs, not just with ALPRs, but also with searching criminal history illegally. IJ collating a dozen cases of arrests of ALPR misuse across the country is not evidence these systems are working, as it is likely the case that only the most egregious abuses are ever caught.
In addition to creating state statutes to mandate that a warrant be used for historical ALPR searches, states should, at a minimum, have clear punishments for illegal searches. These should include at a minimum losing your job, and being banned from accessing the system forever. When I was a crime analyst in New York (and ditto for when I worked at DCJS), this was the standard for misusing the criminal history search database.
If there is a standard for just retaining active search data for less than 24 hours, it does present a potential simple check that should be flagged — if a specific plate or specific camera is searched twice within 2 days, it should be flagged to review more closely. Flock does have their own system to identify suspicious search history.
The bigger issue to me though is who is doing the reviewing. It does not make sense to put this on vendors, and PDs just have not seriously devoted resources to this, even in response to public criticism. This audit mechanism should be delegated to a third party, either a specific group in the state attorney general’s office, or a state criminal justice agency (like DCJS in New York).
So that of course needs to be explicitly set by state statute as well. Who is doing the auditing?
My focus so far has been on abuses via police departments themselves, but smart regulation should also specify auditing of the vendors themselves, as well as punishments if they fail to meet data standards. (I am not thinking so much TEMPEST attacks here, but more so “I left an unauthenticated endpoint willy nilly on the internet”.)
Indeed, many of the requirements I am suggesting are likely already on the books; the problem is that the entity responsible for auditing is often unspecified or lacks the resources to do the work. (Also it is often unclear what the punishments are for failing to abide by statutes. That also needs to be specifically stated.)
The Future
So while I hope (although I have no expectation) that my blog post can somehow influence current standards across the country, I think it is important to keep in mind surveillance not just as the world exists now, but how it may look in the foreseeable future.
I think states should just pull the band aid off and create statutes that require a warrant to search the historical ALPR data. (And this makes data sharing between agencies mostly moot, the real time searches only need to be done within your own jurisdiction.) Like I said at the beginning, the current case law on being able to reconstruct the whole of a person’s movements (which I think is quite reasonable) will eventually be met if the ALPR cameras become dense enough. So states can either create the statutes to dictate that a warrant is necessary themselves, or eventually have the court system thrust it upon them.
In a world filled with privately owned cameras in public spaces, I think these suggestions are still relevant. So similar to Carpenter for CSLI data, and Chatrie for geofence warrants, there should just be warrant standards for historically searching any surveillance footage. There need be no special distinction between ALPR data (public or private) or video cameras.
Even if the groups calling for the banning of Flock cameras get their way, this does not stop private owners from collecting the data. So banning Flock, by itself, does not prevent abuse of searching private cameras. Again I think it is better to just let the government retain the data (same as private vendors will retain the data), and have consistent warrant standards for police to obtain that historical data.
This, of course, is a burden to detectives. I believe that trade-off in protecting our personal liberties while still allowing police effective means to investigate cases is a reasonable one.
There are some technicalities between whether just collecting the data is a search (which was the scenario in the Norfolk case) or whether doing an active search (e.g. an officer querying the system for license plate ABC1234). The Norfolk case was the former, but for this post I am focusing on officers actually searching the data (the latter scenario).↩︎
There are some technicalities between whether just collecting the data is a search (which was the scenario in the Norfolk case) or whether doing an active search (e.g. an officer querying the system for license plate ABC1234). The Norfolk case was the former, but for this post I am focusing on officers actually searching the data (the latter scenario).↩︎
Why have I been blocked?
This website is using a security service to protect itself from online attacks. The action you just performed triggered the security solution. There are several actions that could trigger this block including submitting a certain word or phrase, a SQL command or malformed data.
What can I do to resolve this?
You can email the site owner to let them know you were blocked. Please include what you were doing when this page came up and the Cloudflare Ray ID found at the bottom of this page.
To add this web app to your iOS home screen tap the share button and select "Add to the Home Screen".
10HN is also available as an iOS App
If you visit 10HN only rarely, check out the the best articles from the past week.
Visit pancik.com for more.