The URL short­ener that makes your links look as sus­pi­cious as pos­si­ble.

Normal links are too trust­wor­thy. Make them creepy.

Please see the be­low state­ment in re­gards to the le­gal nasty­grams:

This web­site is a joke. Redirect pages are in place to in­form the user of what web­site they are be­ing di­rected to. It is not de­signed to “weaken global cy­ber­se­cu­rity hy­giene” and does not fa­cil­i­tate phish­ing.

This web­site does not vi­o­late any known laws, poli­cies, or rules, to the best of the au­thor’s knowl­edge.

Valid con­cerns brought up in your let­ter have been ad­dressed. Going for­ward, I would greatly ap­pre­ci­ate if you use the sup­port email in “Report Issue” to dis­cuss con­cerns or prob­lems with this ser­vice, rather than send­ing le­gal threats.

Palantir is work­ing on a tool for Immigration and Customs Enforcement (ICE) that pop­u­lates a map with po­ten­tial de­por­ta­tion tar­gets, brings up a dossier on each per­son, and pro­vides a “confidence score” on the per­son’s cur­rent ad­dress, 404 Media has learned. ICE is us­ing it to find lo­ca­tions where lots of peo­ple it might de­tain could be based.

The find­ings, based on in­ter­nal ICE ma­te­r­ial ob­tained by 404 Media, pub­lic pro­cure­ment records, and re­cent sworn tes­ti­mony from an ICE of­fi­cial, show the clear­est link yet be­tween the tech­no­log­i­cal in­fra­struc­ture Palantir is build­ing for ICE and the agen­cy’s ac­tiv­i­ties on the ground. The tool re­ceives peo­ples’ ad­dresses from the Department of Health and Human Services (HHS) among a range of other sources, ac­cord­ing to the ma­te­r­ial.

The news comes af­ter Department of Homeland Security (DHS) head Kristi Noem said the agency is send­ing hun­dreds more fed­eral agents to Minneapolis amid wide­spread protests against the agency. Last week ICE of­fi­cer Jonathan Ross shot and killed 37 year old U. S. cit­i­zen Renee Nicole Good. During Operation Metro Surge, which DHS calls the “largest im­mi­gra­tion op­er­a­tion ever,” im­mi­gra­tion agents have sur­rounded rideshare dri­vers and used pep­per spray on high school stu­dents.

When CC Wei vis­ited Cupertino last August, he had bad news for his largest client. Apple would need to ac­qui­esce to the largest price rise in years, TSMC’s CEO told its ex­ec­u­tives.

Tim Cook and his team took the news on the chin. Wei had been telegraph­ing hikes in earn­ings calls over the past few quar­ters, and the Taiwanese chip mak­er’s ris­ing gross mar­gins were tes­ta­ment to its in­creas­ing pric­ing power.

That was­n’t the worst news, my sources tell me.

Apple, which once held a dom­i­nant po­si­tion on TSMC’s cus­tomer list, now needs to fight for pro­duc­tion ca­pac­ity. With the con­tin­u­ing AI boom, and each GPU from clients like Nvidia and AMD tak­ing up a larger foot­print per wafer, the iPhone mak­er’s chip de­signs are no longer guar­an­teed a place among TSMC’s al­most two dozen fabs.

What Wei prob­a­bly did­n’t tell Cook is that Apple may no longer be his largest client.

According to Culpium analy­sis and dis­cus­sions with sources in the sup­ply chain, Nvidia likely took top spot in at least one or two quar­ters of last year. “We don’t dis­cuss that,” Chief Financial Officer Wendell Huang told Culpium Thursday when asked about the change in client rank­ings.

Final data will be un­veiled in a few months when TSMC re­leases its an­nual re­port — which in­cludes rev­enue from its top clients — but there’s every chance that Apple’s lead for the full year nar­rowed sig­nif­i­cantly and may have even fallen be­low Nvidia’s. If it did­n’t hap­pen in 2025, then it’s al­most cer­tain to do so in 2026, my sources tell me.

TSMC’s rev­enue climbed 36% last year to $122 bil­lion, it re­ported Thursday. Nvidia’s sales for the fis­cal year through January 2026 is set to climb 62% while Apple’s prod­uct rev­enue — which ex­cludes ser­vices — is on track to grow just 3.6% for the 12-months to December 2025, ac­cord­ing to Culpium es­ti­mates based on earn­ings re­ports and com­pany guid­ance.

Apple’s role as the pri­mary dri­ver of TSMC rev­enue growth ended five years ago. In 2018 TSMC sales would have even fallen if not for in­cre­men­tal pur­chases by Apple that year. Now, the Cupertino com­pany is post­ing low sin­gle-digit rev­enue growth while Nvidia is sky­rock­et­ing.

The rea­son for this change is two-fold, and pretty ob­vi­ous: AI is dri­ving mas­sive de­mand for high-pow­ered chips, while the smart­phone boom has plateaued.

TSMC’s sales from high-per­for­mance com­put­ing, which in­cludes AI chips, climbed 48% last year on top of 58% growth the year be­fore. Smartphone rev­enue climbed just 11%, slower than 23% in the prior year. That trend will con­tinue this year, and for the fore­see­able fu­ture.

Revenue in 2026 will rise close to 30%, yet cap­i­tal ex­pen­di­ture will climb around 32% to a record of some­where be­tween $52 bil­lion and $56 bil­lion, TSMC said Thursday. Longer term, growth will av­er­age 25% in the five years through 2029 yet the AI seg­ment will climb an av­er­age of 55% or more over the same pe­riod, the com­pany said. That’s higher than a prior fore­cast for a mid-40 per­cent fig­ure.

The ul­ti­mate flex for TSMC came Thursday when it showed off not only record rev­enue and net in­come, but a gross mar­gin ap­proach­ing that of soft­ware mak­ers and fa­b­less chip de­sign­ers. In the December quar­ter, that fig­ure was an as­tound­ing 62.3%, 280 ba­sis points higher than the prior pe­riod. If not for its over­seas fabs (Arizona and Japan) gross mar­gin would have been even higher.

There are two caveats that are im­por­tant. First, while smart­phone proces­sors are the largest por­tion of chips bought by Apple, they’re not the only type. Processors for Macs come un­der HPC, while it also has a strong lineup of cus­tom chips used in ac­ces­sories which fall un­der dig­i­tal con­sumer elec­tron­ics. Second, Nvidia is­n’t the only HPC client. AMD is a ma­jor buyer of ca­pac­ity for its own GPUs while Amazon and Google are on the grow­ing list of cus­tomers de­vel­op­ing in-house AI chips.

Put an­other way, Apple’s chip cat­a­log is broader and more var­ied, while Nvidia’s lineup is more con­cen­trated around a huge num­ber of wafers at, or near, lead­ing-edge. It’s for these rea­sons that Apple will re­main im­por­tant for at least an­other decade.

In the near-term, how­ever, TSMC’s tech­nol­ogy roadmap cou­pled with broader in­dus­try trends fa­vor Nvidia, AMD and their ilk, mean­ing Apple may need to keep fight­ing for ca­pac­ity over the next year or two.

TSMC is al­ready pro­duc­ing chips in vol­ume at 2 nanome­ter (called N2), cur­rently its most ad­vanced node, with Apple a ma­jor buyer. But in the sec­ond half of this year it’s set to ramp up both a new vari­ant called N2P as well as a new node called A16.

The com­pa­ny’s busi­ness model is a lit­tle quirky. Instead of re­pur­pos­ing an ex­ist­ing fac­tory for new tech­nol­ogy, TSMC just builds a new one. This en­sures no in­ter­rup­tion to out­put and al­lows it to squeeze the most out of old tools and processes. In gen­eral, this means any new ca­pac­ity that TSMC builds is for a new node. As a re­sult, it has nu­mer­ous fabs still churn­ing out chips on tech­nol­ogy that’s a decade older or more.

In TSMC CEO CC Wei’s words A16, with Super Power Rail, is “best for HPC with com­plex sig­nal routes.” SPR is TSMC’s ver­sion of back­side power, a newer ap­proach de­signed to sep­a­rate a chip’s sig­nal from its power sup­ply. Intel is also de­vel­op­ing this tech­nol­ogy, and many be­lieve it’ll be the key to the US com­pa­ny’s prospects at steal­ing foundry share from its Taiwan ri­val.

After that, TSMC has A14 which it ex­pects to bring into vol­ume pro­duc­tion around 2028. Some call this the next full node af­ter N2, la­bel­ing A16 as not a “full node.” In truth, all of these names are as much mar­ket­ing terms as they are tech­nol­ogy des­ig­na­tors. Nevertheless, as SemiAnalysis re­cently wrote in a fab­u­lous re­port on the TSMC-Apple re­la­tion­ship, the bal­ance will shift back to Apple be­cause A14 is de­signed “for both mo­bile and HPC from the start.”

More im­por­tantly, what Apple of­fers is sta­bil­ity. Nvidia has been a client for a lot longer than Apple, but broadly speak­ing it’s a bit niche. Right now that “niche” is the hottest prod­uct on the planet, but niche it is. Apple, on the other hand, has prod­ucts be­ing made in no fewer than a dozen TSMC fabs. Even if Nvidia did over­take Apple by pur­chases, the breadth of its man­u­fac­tur­ing foot­print at TSMC is nowhere near as large.

This dis­tinc­tion may not mat­ter now, but it prob­a­bly will at some point. The AI boom won’t last for­ever. The bub­ble may burst, or it may slowly de­flate, but the growth tra­jec­tory will surely flat­ten and that means de­mand for lead­ing-edge AI chips will fall.

Wei knows this, which is why he’s ex­pand­ing both quickly yet cau­tiously. “I am also very ner­vous,” he said at the com­pa­ny’s in­vestor con­fer­ence on Thursday in Taipei. “If we did­n’t do it care­fully, it would be a big dis­as­ter for TSMC for sure.”

The chip gi­ant has re­cently come un­der fire, in­clud­ing from noted an­a­lyst Benedict Evans, for be­ing “unwilling/unable to ex­pand ca­pac­ity fast enough to meet Nvidia’s book.” I think this is wrong, and un­fair.

“The risk of un­der-in­vest­ing is sig­nif­i­cantly greater than the risk of over-in­vest­ing,” Evans cited Google CEO Sundar Pichai as say­ing back in 2Q 2024, as if to make the point. TSMC and Alphabet, Google’s par­ent, have ap­prox­i­mately the same gross mar­gin. But their busi­ness mod­els could­n’t be more dif­fer­ent. Nvidia’s fi­nan­cials are also un­like TSMC’s. Their re­spec­tive capex strate­gies need to re­flect this risk.

Alphabet’s cap­i­tal in­ten­sity, cal­cu­lated as ac­qui­si­tions of prop­erty, plant & equip­ment di­vided by rev­enue, was just 15% for full-year 2024. TSMC’s is more than dou­ble that at over 33%. More im­por­tantly, de­pre­ci­a­tion — which is where the cost of capex is re­flected in earn­ings — was just 10% of Alphabet’s cost of rev­enue. For TSMC, this fig­ure is more than four times higher at 45%.

At Nvidia, which is a tier-one buyer of TSMC’s out­put, the data is more stark. Capital in­ten­sity was just 2.5% for 2024, while de­pre­ci­a­tion was only 5.7% of the cost of rev­enue. As a fa­b­less chip­maker, it can en­joy gross mar­gins of over 70%. Its only real risk is hold­ing ex­cess in­ven­tory. Even then, it could have writ­ten off its en­tire in­ven­tory at the end of October and still main­tain a gross mar­gin ap­proach­ing that of its chief sup­plier. What’s more, nei­ther of these clients have any­where near the cus­tomer-con­cen­tra­tion risk of TSMC.

The com­plaint that TSMC could and should build faster ig­nores the fact that it’s the one left hold­ing the baby if a down­turn comes and de­mand falls. It takes two to three years to build a new fab, Wei ex­plained, so the com­pany must skate where the puck is go­ing with­out think­ing too much about where it’s been. “Even if we spend 52 to 56 bil­lion this year, the con­tri­bu­tion this year is none,“ Wei said Thursday. Its ma­jor cost, buy­ing equip­ment, re­mains on the books no mat­ter what rev­enue it brings in for the quar­ter.

For the best part of a decade, Apple was the one dri­ving TSMC’s need to keep spend­ing on new fa­cil­i­ties. Today it’s Nvidia, and Jensen Huang is start­ing to wield more power than Tim Cook. But nei­ther has to bother with the ex­pen­sive busi­ness of ac­tu­ally man­u­fac­tur­ing semi­con­duc­tors, merely the has­sle of beg­ging CC Wei for wafers.

For such clients, the foundry’s ca­pac­ity is a fixed cost that they need­n’t worry about. Which is pre­cisely why eight of the world’s ten largest com­pa­nies turn to TSMC to make their chips, and in re­turn the Taiwanese gi­ant gets to reap the re­wards dur­ing boom times like this.

Last year China in­stalled more than half of all wind and so­lar added glob­ally. In May alone, it added enough re­new­able en­ergy to power Poland, in­stalling so­lar pan­els at a rate of roughly 100 every sec­ond.

The mas­sive build­out is hap­pen­ing across the coun­try, from crowded east­ern cities in­creas­ingly topped by rooftop so­lar pan­els to re­mote west­ern deserts where colos­sal wind farms sprawl across the land­scape.

“From the ground, it’s hard to grasp the scale of these power plants,” said Chinese pho­tog­ra­pher Weimin Chu. “But when you rise into the air, you can see the geom­e­try, the rhythm — and their re­la­tion­ship with the moun­tains, the desert, the sea.”

Chu has spent three years cap­tur­ing the shift un­der­way us­ing drones to pho­to­graph power plants from over­head. His work, which draws from the vi­sual lan­guage of tra­di­tional Chinese ink paint­ings, was fea­tured last year in an award-win­ning ex­hi­bi­tion, pre­sented by Greenpeace. A se­lec­tion of those pho­tos is re­pro­duced here.

“I started out just shoot­ing land­scapes,” Chu said. “But when I trav­eled to places like Guizhou, Yunnan, and Qinghai in 2022, I kept see­ing wind farms and so­lar power plants ap­pear in my cam­era frame. I re­al­ized this is the story of our time — and al­most no one is doc­u­ment­ing it in a sys­tem­atic way.”

“Internal ICE ma­te­r­ial and tes­ti­mony from an of­fi­cial ob­tained by 404 Media pro­vides the clear­est link yet be­tween the tech­no­log­i­cal in­fra­struc­ture Palantir is build­ing for ICE and the agen­cy’s ac­tiv­i­ties on the ground.”

“Internal ICE ma­te­r­ial and tes­ti­mony from an of­fi­cial ob­tained by 404 Media pro­vides the clear­est link yet be­tween the tech­no­log­i­cal in­fra­struc­ture Palantir is build­ing for ICE and the agen­cy’s ac­tiv­i­ties on the ground.”

This is racial pro­fil­ing on a grand scale:

It ap­par­ently looks a lot like Google Maps, but de­signed to show the rich­ness of an area for “targets”, pop­u­lated in part by den­sity of im­mi­grants. And then you can dig in:

The Nazis could only dream of hav­ing such a ca­pa­bil­ity.

Imagine work­ing for this com­pany, on this prod­uct. Every day, you go into work, in what I as­sume is a beau­ti­ful of­fice with pine fur­ni­ture and a well-stocked kitchen, and you build soft­ware that will help to de­port peo­ple us­ing what you know are ex­tra­ju­di­cial means with­out due process. You prob­a­bly have OKRs. There are cus­tomer calls with ICE. Every two-week sprint, you take on tasks that help make this en­gine bet­ter.

What do you tell your­self? What do you tell your fam­ily?

Are you on board with this agenda, or do you tell your­self you need the job to pay rent? To get health­care?

You re­ceive stock as part of your pay pack­age. It’s go­ing up! You can use it to buy a home, or to build a com­fort­able re­tire­ment, or some com­bi­na­tion of the two.

Your co-work­ers are val­ues aligned and work hard. They’re tal­ented and smart. Man, you might think to your­self, I love work­ing with this team.

Or, you might think, man, I’ve got to find an­other job.

Either way, you’re proud of your prod­uct work. You’re happy to take the salary, the free lunches, the espresso. And re­gard­less of how you feel about it, the thing you do every day is pow­er­ing an armed force that is kid­nap­ping peo­ple on the street and shoot­ing civil­ians, that shot a mother in the face, that is tar­get­ing peo­ple to dis­ap­pear us­ing a beau­ti­ful, mod­ern map in­ter­face.

Yes, you, who are think­ing about not hir­ing a tech­ni­cal writer this year or, worse, erased one or more tech­ni­cal writ­ing po­si­tions last year be­cause of AI. You, who are buy­ing into the promise of docs en­tirely au­thored by LLMs with­out ex­pert over­sight or guid­ance. You, who un­loaded the weight of docs on your de­vs’ shoul­ders, as if it was a triv­ial chore.

You are mak­ing a big mis­take. But you can still undo the dam­age.

It’s been a com­pli­cated year, 2025. When even Andrej Karpathy, one of OpenAI’s founders, ad­mits, in a fit of Oppenheimerian guilt, to feel­ing lost, you know that no one holds the key to the fu­ture. You flail and dance around these new totems made of words, which are nei­ther in­tel­li­gent nor con­scious, pre­tend­ing they can re­place hu­mans while, in fact, they’re lit­tle more than glo­ri­fied tools.

You might think that the plau­si­ble taste of AI prose is all you need to give your prod­ucts a voice. You paste code into a field and some­thing that re­sem­bles docs comes out af­ter a few min­utes. Like a stu­dent ea­ger to turn home­work in, you might be tempted to con­tent your­self with docs the­atre, think­ing that it’ll earn you a good grade. It won’t, be­cause docs aren’t just ar­ti­facts.

You keep us­ing that word. I do not think it means what you think it means

When you say “docs”, you’re care­ful to fo­cus on the out­put, omit­ting the process. Perhaps you don’t know how docs are pro­duced. You’ve for­got­ten, or per­haps never knew, that docs are prod­uct truth; that with­out them, soft­ware be­comes un­us­able, be­cause soft­ware is never done, is never ob­vi­ous, and is never sim­ple. Producing those docs re­quires tech writ­ers.

Tech writ­ers go to great lengths to get the in­for­ma­tion they need. They write so that your au­di­ence can un­der­stand. They hunger for clar­ity and mean­ing and im­pact. They power through weeks full of dead­lines, chas­ing prod­uct news, be­cause with­out their re­port­ing, most prod­ucts would­n’t thrive; some would­n’t even ex­ist. Their docs aren’t a byprod­uct: they tie the prod­uct to­gether.

An LLM can’t do all that, be­cause it can’t feel the pain of your users. It can’t put it­self into their shoes. It lacks the kind of em­pa­thy that’s be­hind great help con­tent. It does not, in fact, have any em­pa­thy at all, be­cause it can­not care. You need folks who will care, be­cause con­tent is a hairy beast that can only be tamed by agents made of flesh and ca­pa­ble of emo­tions: hu­mans.

You can’t gen­er­ate docs on au­topi­lot. Let me tell you why.

First, AI-generated docs are not in­tel­li­gent. They not only make up things in sub­tle ways: They lack vi­sion. Even if you fed them mil­lions of to­kens, they could­n’t de­velop a docs strat­egy, de­cide what not to doc­u­ment, or struc­ture con­tent for reuse. And they fail to cap­ture the ten­sion, the caveats, the edge cases, the feel­ing of un­fin­ished­ness that only some­one who cares can feel. Without that ground­ing, docs are hol­low.

Second, li­a­bil­ity does­n’t van­ish just be­cause AI wrote it. When docs cause harm through wrong in­struc­tions, some­one will be held re­spon­si­ble. It won’t be the model. You can’t de­pose an LLM. You can’t fire it. You can’t point at it in court when a cus­tomer’s data evap­o­rates be­cause your GenAI run­book told them to run the wrong com­mand. That some­one will be you, or some­one who re­ports to you.

Third, even your fa­vorite AI must RTFM. All your Claude Skills, Cursor rules, all the se­man­tic tag­ging that makes RAG work, is tech­ni­cal writ­ing un­der a new name: con­text cu­ra­tion. You fired or did­n’t hire the peo­ple who cre­ate high-qual­ity con­text and then won­dered why your AI tools pro­duce slop. You can’t aug­ment what is­n’t there. The writ­ers you let go were the sup­ply chain for the in­tel­li­gence you’re now bet­ting on.

It’s not all bad news: Marvelous things can hap­pen if you pro­vide your writ­ers with AI tools and train­ing while you pro­tect the qual­ity of your con­tent through an AI pol­icy. I’ve de­scribed the ideal end state in My day as an aug­mented tech­ni­cal writer in 2030, a vi­sion of the fu­ture where writ­ers or­ches­trate, edit, and pub­lish docs to­gether with AI agents. This is al­ready hap­pen­ing be­fore our eyes.

Productivity gains are real when you un­der­stand that aug­men­ta­tion is bet­ter than re­plac­ing hu­mans, a re­al­ity even AWS’ CEO, Matt Garman, ac­knowl­edged. Read how I’m us­ing AI as a tech­ni­cal writer. I’m not alone: Follow Tom Johnson, CT Smith, and Sarah Deaton, and dis­cover how tech writ­ers are build­ing tools through AI to bet­ter ap­ply it to docs.

Develop an AI strat­egy for docs to­gether with tech writ­ers, and give them time and re­sources to ex­per­i­ment with AI. Tech writ­ers are re­source­ful by na­ture: they’ve spent ca­reers do­ing more with less, op­ti­miz­ing work­flows, find­ing clever so­lu­tions to im­pos­si­ble quests. Give them the tools and a bit of run­way, and they’ll fig­ure out how to make AI work for the docs, not in­stead of them.

Reconsider the po­si­tions you did not open. Or the writ­ers you let go. Reconsider the as­sump­tion that AI has solved a prob­lem that, at its core, is deeply hu­man and re­quires not only con­cate­nat­ing words, but also chas­ing sub­ject-mat­ter ex­perts and un­der­stand­ing the sub­tleties of prod­uct mo­tions, among many other things.

Technical writ­ers aren’t a lux­ury. They are the peo­ple who trans­late what you’ve built into some­thing oth­ers can use. Without them, you’re ship­ping a prod­uct that can’t speak for it­self, or that lies. Your prod­uct needs to speak. AI can gen­er­ate noise ef­fec­tively and in­fi­nitely, but only a tech­ni­cal writer can cre­ate the sig­nal.

Don’t choose the noise. Get them back. Get them on­board.

Thanks to Tiffany Hrabusa, Casey Smith, and Anna Urbiztondo for their re­views of early drafts and for their en­cour­age­ment. Thanks to my part­ner, Valentina, for help­ing me im­prove this piece and for sug­gest­ing to wait a bit be­fore hit­ting Publish. And a heart­felt thank you to the tech writ­ing com­mu­nity and its won­der­ful hu­man be­ings.

For a stand­alone ver­sion of this let­ter, use https://​passo.uno/​re­con­sider/.

Today Raspberry Pi launched their new $130 AI HAT+ 2 which in­cludes a Hailo 10H and 8 GB of LPDDR4X RAM.

With that, the Hailo 10H is ca­pa­ble of run­ning LLMs en­tirely stand­alone, free­ing the Pi’s CPU and sys­tem RAM for other tasks. The chip runs at a max­i­mum of 3W, with 40 TOPS of INT8 NPU in­fer­ence per­for­mance in ad­di­tion to the equiv­a­lent 26 TOPS INT4 ma­chine vi­sion per­for­mance on the ear­lier AI HAT with Hailo 8.

In prac­tice, it’s not as amaz­ing as it sounds.

You still can’t up­grade the RAM on the Pi, but at least this way if you do have a need for an AI co­proces­sor, you don’t have to eat up the Pi’s mem­ory to run things on it.

And it’s a lot cheaper and more com­pact than run­ning an eGPU on a Pi. In that sense, it’s more use­ful than the silly NPUs Microsoft forces into their ‘AI PCs’.

But it’s still a so­lu­tion in search of a prob­lem, in all but the most niche of use cases.

Besides feel­ing like I’m liv­ing in the world of the Turbo Encabulator every time I’m test­ing AI hard­ware, I find the mar­ket­ing of these things to be very vague, and the ap­pli­ca­tions not very broad.

For ex­am­ple, the Hailo 10H is ad­ver­tised as be­ing used for a Fujitsu demo of au­to­matic shrink de­tec­tion for a self-check­out.

That’s cer­tainly not a worth­less use case, but it’s not some­thing I’ve ever needed to do. I have a feel­ing this board is meant more for de­vel­op­ment, for peo­ple who want to de­ploy the 10H in other de­vices, rather than as a to­tal so­lu­tion to prob­lems in­di­vid­ual Pi own­ers need to solve.

Especially when it comes to the head­line fea­ture: run­ning in­fer­ence, like with LLMs.

I also pub­lished a video with all the in­for­ma­tion in this blog post, but if you en­joy text more than video, scroll on past—it does­n’t of­fend me!

I ran every­thing on an 8 gig Pi 5, so I could get an ap­ples-to-ap­ples com­par­i­son, run­ning the same mod­els on the Pi’s CPU as I did on the AI HAT’s NPU.

They both have the same 8GB LPDDR4X RAM con­fig­u­ra­tion, so ide­ally, they’d have sim­i­lar per­for­mance.

I tested every model Hailo put out so far, and com­pared them, Pi 5 ver­sus Hailo 10H:

The Hailo is only close, re­ally, on Qwen2.5 Coder 1.5B.

It is slightly more ef­fi­cient in most cases:

But look­ing more closely at power draw, we can see why the Hailo does­n’t keep up:

The Pi’s CPU is al­lowed to max out it’s power lim­its (10W on the SoC), which are a lot higher than the Hailo’s (3W).

So power holds it back, but the 8 gigs of RAM holds back the LLM use case (vs just run­ning on the Pi’s CPU) the most. The Pi 5 can be bought in up to a 16 GB con­fig­u­ra­tion. That’s as much as you get in de­cent con­sumer graph­ics cards.

Because of that, many quan­tized medium-size mod­els tar­get 10-12 GB of RAM us­age (leaving space for con­text, which eats up an­other 2+ GB of RAM).

A cou­ple weeks ago, ByteShape got Qwen3 30B A3B Instruct to fit on a 16GB Pi 5. Now this post is­n’t about LLMs, but the short of it is they found a novel way to com­press the model to fit in 10 GB of RAM.

A lit­tle bit of qual­ity is lost, but like a JPEG, it’s still good enough to ace all the con­trived tests (like build­ing a TODO list app, or sort­ing a com­plex list) that the tiny mod­els I ran on the Hailo 10H did­n’t com­plete well (see the video ear­lier in this post for de­tails).

To test the 30B model, I in­stalled llama.cpp fol­low­ing this guide from my blog, and down­loaded the com­pressed model.

I asked it to gen­er­ate a sin­gle page TODO list app, and it’s still not a speed de­mon (this is a Pi CPU with LPDDR4x RAM we’re talk­ing about), but af­ter a lit­tle while, it gave me this:

It met all my re­quire­ments:

* I can type in as many items as I want

* I can drag them around to re­arrange them

* I can check off items and they go to the bot­tom of the list…

It’s hon­estly crazy how many small tasks you can do even with free lo­cal mod­els… even on a Pi. Natural Language Programming was just a dream back when I started my ca­reer.

Besides be­ing an­gry Google, OpenAI, Anthropic and all these other com­pa­nies are con­sum­ing all the world’s money and re­sources do­ing this stuff—not to men­tion de­stroy­ing the ca­reers of thou­sands of ju­nior de­vel­op­ers—it is kinda neat to see NLP work for very tightly de­fined ex­am­ples.

But I don’t think this HAT is the best choice to run lo­cal, pri­vate LLMs (at least not as a pri­mary goal).

What it is good for, is vi­sion pro­cess­ing. But the orig­i­nal AI HAT was good for that too!

In my test­ing, Hailo’s hailo-rpi5-ex­am­ples were not yet up­dated for this new HAT, and even if I spec­i­fied the Hailo 10H man­u­ally, model files would not load, or I ran into er­rors once the board was de­tected.

But Raspberry Pi’s mod­els ran, so I tested them with a Camera Module 3:

I pointed it over at my desk, and it was able to pick out things like my key­board, my mon­i­tor (which it thought was a TV), my phone, and even the mouse tucked away in the back.

It all ran quite fast—and 10x faster than on the Pi’s CPU—but the prob­lem is I can do the same thing with the orig­i­nal AI HAT ($110)—or the AI Camera ($70).

If you just need vi­sion pro­cess­ing, I would stick with one of those.

The head­line fea­ture of the AI HAT+ 2 is the abil­ity to run in a ‘mixed’ mode, where it can process ma­chine vi­sion (frames from a cam­era or video feed), while also run­ning in­fer­ence (like an LLM or text-to-speech).

Unfortunately, when I tried run­ning two mod­els si­mul­ta­ne­ously, I ran into seg­men­ta­tion faults or ‘device not ready’, and lack­ing any work­ing ex­am­ples from Hailo, I had to give up on get­ting that work­ing in time for this post.

Just like the orig­i­nal AI HAT, there’s some grow­ing pains.

It seems like with most hard­ware with “AI” in the name, it’s hard­ware-first, then soft­ware comes later—if it comes at all. At least with Raspberry Pi’s track record, the soft­ware does come, it’s just… of­ten the so­lu­tions are only use­ful in tiny niche use cases.

8 GB of RAM is use­ful, but it’s not quite enough to give this HAT an ad­van­tage over just pay­ing for the big­ger 16GB Pi with more RAM, which will be more flex­i­ble and run mod­els faster.

The main use case for this HAT might be in power-con­strained ap­pli­ca­tions where you need both vi­sion pro­cess­ing and in­fer­enc­ing. But even there… it’s hard to say “yes, buy this thing”, be­cause for just a few more watts, the Pi could achieve bet­ter per­for­mance for in­fer­ence in tan­dem with the $70 AI Camera or the $110 AI HAT+ for the vi­sion pro­cess­ing.

Outside of run­ning tiny LLMs in less than 10 watts, maybe the idea is you use the AI HAT+ 2 as a de­vel­op­ment kit for de­sign­ing de­vices us­ing the 10H like self-check­out scan­ners (which might not even run on a Pi)? I’m not sure.

Handy is a cross-plat­form desk­top ap­pli­ca­tion built with Tauri (Rust + React/TypeScript) that pro­vides sim­ple, pri­vacy-fo­cused speech tran­scrip­tion. Press a short­cut, speak, and have your words ap­pear in any text field—all with­out send­ing your voice to the cloud.

Handy was cre­ated to fill the gap for a truly open source, ex­ten­si­ble speech-to-text tool. As stated on handy.com­puter:

* Free: Accessibility tool­ing be­longs in every­one’s hands, not be­hind a pay­wall

* Open Source: Together we can build fur­ther. Extend Handy for your­self and con­tribute to some­thing big­ger

* Private: Your voice stays on your com­puter. Get tran­scrip­tions with­out send­ing au­dio to the cloud

* Simple: One tool, one job. Transcribe what you say and put it into a text box

Handy is­n’t try­ing to be the best speech-to-text app—it’s try­ing to be the most fork­able one.

Press a con­fig­urable key­board short­cut to start/​stop record­ing (or use push-to-talk mode)

Speak your words while the short­cut is ac­tive

Release and Handy processes your speech us­ing Whisper

Get your tran­scribed text pasted di­rectly into what­ever app you’re us­ing

The process is en­tirely lo­cal:

* Silence is fil­tered us­ing VAD (Voice Activity Detection) with Silero

* Transcription uses your choice of mod­els:

Whisper mod­els (Small/Medium/Turbo/Large) with GPU ac­cel­er­a­tion when avail­able

* Whisper mod­els (Small/Medium/Turbo/Large) with GPU ac­cel­er­a­tion when avail­able

Download the lat­est re­lease from the re­leases page or the web­site

* Frontend: React + TypeScript with Tailwind CSS for the set­tings UI

* Core Libraries:

Handy in­cludes an ad­vanced de­bug mode for de­vel­op­ment and trou­bleshoot­ing. Access it by press­ing:

This pro­ject is ac­tively be­ing de­vel­oped and has some known is­sues. We be­lieve in trans­parency about the cur­rent state:

* Whisper mod­els crash on cer­tain sys­tem con­fig­u­ra­tions (Windows and Linux)

* Does not af­fect all sys­tems - is­sue is con­fig­u­ra­tion-de­pen­dent

If you ex­pe­ri­ence crashes and are a de­vel­oper, please help to fix and pro­vide de­bug logs!

* If you ex­pe­ri­ence crashes and are a de­vel­oper, please help to fix and pro­vide de­bug logs!

* Requires wtype or dotool for text in­put to work cor­rectly (see Linux Notes be­low for in­stal­la­tion)

For re­li­able text in­put on Linux, in­stall the ap­pro­pri­ate tool for your dis­play server:

* X11: Install xdo­tool for both di­rect typ­ing and clip­board paste short­cuts

* Wayland: Install wtype (preferred) or dotool for text in­put to work cor­rectly

* dotool setup: Requires adding your user to the in­put group: sudo user­mod -aG in­put $USER (then log out and back in)

Without these tools, Handy falls back to enigo which may have lim­ited com­pat­i­bil­ity, es­pe­cially on Wayland.

The record­ing over­lay is dis­abled by de­fault on Linux (Overlay Position: None) be­cause cer­tain com­pos­i­tors treat it as the ac­tive win­dow. When the over­lay is vis­i­ble it can steal fo­cus, which pre­vents Handy from past­ing back into the ap­pli­ca­tion that trig­gered tran­scrip­tion. If you en­able the over­lay any­way, be aware that clip­board-based past­ing might fail or end up in the wrong win­dow.

If you are hav­ing trou­ble with the app, run­ning with the en­vi­ron­ment vari­able WEBKIT_DISABLE_DMABUF_RENDERER=1 may help

You can man­age global short­cuts out­side of Handy and still con­trol the app via sig­nals. Sending SIGUSR2 to the Handy process tog­gles record­ing on/​off, which lets Wayland win­dow man­agers or other hotkey dae­mons keep own­er­ship of key­bind­ings. Example (Sway):

bindsym $mod+o exec pkill -USR2 -n handy

pkill here sim­ply de­liv­ers the sig­nal—it does not ter­mi­nate the process.

The fol­low­ing are rec­om­men­da­tions for run­ning Handy on your own ma­chine. If you don’t meet the sys­tem re­quire­ments, the per­for­mance of the ap­pli­ca­tion may be de­graded. We are work­ing on im­prov­ing the per­for­mance across all kinds of com­put­ers and hard­ware.

We’re ac­tively work­ing on sev­eral fea­tures and im­prove­ments. Contributions and feed­back are wel­come!

* Adding de­bug log­ging to a file to help di­ag­nose is­sues

* A rewrite of global short­cut han­dling for MacOS, and po­ten­tially other OS’s too.

* Cleanup and refac­tor set­tings sys­tem which is be­com­ing bloated and messy

If you’re be­hind a proxy, fire­wall, or in a re­stricted net­work en­vi­ron­ment where Handy can­not down­load mod­els au­to­mat­i­cally, you can man­u­ally down­load and in­stall them. The URLs are pub­licly ac­ces­si­ble from any browser.

Navigate to the About sec­tion

Copy the “App Data Directory” path shown there, or use the short­cuts:

Inside your app data di­rec­tory, cre­ate a mod­els folder if it does­n’t al­ready ex­ist:

# ma­cOS/​Linux

mkdir -p ~/Library/Application\ Support/com.pais.handy/models

# Windows (PowerShell)

New-Item -ItemType Directory -Force -Path “$env:APPDATA\com.pais.handy\models”

Download the mod­els you want from be­low

Simply place the .bin file di­rectly into the mod­els di­rec­tory:

Place the ex­tracted di­rec­tory into the mod­els folder

The di­rec­tory must be named ex­actly as fol­lows:

Final struc­ture should look like:

* For Parakeet mod­els, the ex­tracted di­rec­tory name must match ex­actly as shown above

* Do not re­name the .bin files for Whisper mod­els—use the ex­act file­names from the down­load URLs

* After plac­ing the files, restart Handy to de­tect the new mod­els

Your man­u­ally in­stalled mod­els should now ap­pear as “Downloaded”

Select the model you want to use and test tran­scrip­tion

Test thor­oughly on your tar­get plat­form

Submit a pull re­quest with clear de­scrip­tion of changes

Join the dis­cus­sion - reach out at con­tact@handy.com­puter

The goal is to cre­ate both a use­ful tool and a foun­da­tion for oth­ers to build upon—a well-pat­terned, sim­ple code­base that serves the com­mu­nity.

* Whisper by OpenAI for the speech recog­ni­tion model

“Your search for the right speech-to-text tool can end here—not be­cause Handy is per­fect, but be­cause you can make it per­fect for you.”