10 interesting stories served every morning and every evening.




1 826 shares, 44 trendiness

HigherOrderCO/Bend: A massively parallel, high-level programming language

Unlike low-level al­ter­na­tives like CUDA and Metal, Bend has the feel­ing and fea­tures of ex­pres­sive lan­guages like Python and Haskell, in­clud­ing fast ob­ject al­lo­ca­tions, higher-or­der func­tions with full clo­sure sup­port, un­re­stricted re­cur­sion, even con­tin­u­a­tions. Yet, it runs on mas­sively par­al­lel hard­ware like GPUs, with near-lin­ear speedup based on core count, and zero ex­plicit par­al­lel an­no­ta­tions: no thread spawn­ing, no locks, mu­texes, atom­ics. Bend is pow­ered by the HVM2 run­time.

Currently not work­ing on Windows, please use WSL2 as a workaround.

First, in­stall Rust nightly. Then, in­stall both HVM2 and Bend with:

cargo +nightly in­stall hvm

cargo +nightly in­stall bend-lang

Finally, write some Bend file, and run it with one of these com­mands:

bend run

You can also com­pile Bend to stand­alone C/CUDA files with gen-c and

gen-cu, for max­i­mum per­for­mance. But keep in mind our code gen is still on its in­fancy, and is nowhere as ma­ture as SOTA com­pil­ers like GCC and GHC.

To write par­al­lel pro­grams in Bend, all you have to do is… noth­ing. Other than not mak­ing it in­her­ently se­quen­tial! For ex­am­ple, the ex­pres­sion:

(((1 + 2) + 3) + 4)

Can not run in par­al­lel, be­cause +4 de­pends on +3 which de­pends on (1+2). But the fol­low­ing ex­pres­sion:

((1 + 2) + (3 + 4))

Can run in par­al­lel, be­cause (1+2) and (3+4) are in­de­pen­dent; and it will, per Bend’s fun­da­men­tal pledge:

Everything that can run in par­al­lel, will run in par­al­lel.

For a more com­plete ex­am­ple, con­sider:

# Sorting Network = just ro­tate trees!

def sort(d, s, tree):

switch d:

case 0:

re­turn tree

case _:

(x,y) = tree

lft = sort(d-1, 0, x)

rgt = sort(d-1, 1, y)

re­turn rots(d, s, lft, rgt)

# Rotates sub-trees (Blue/Green Box)

def rots(d, s, tree):

switch d:

case 0:

re­turn tree

case _:

(x,y) = tree

re­turn down(d, s, warp(d-1, s, x, y))

This

file

im­ple­ments a bitonic sorter with

im­mutable tree ro­ta­tions. It is not the kind of al­go­rithm you’d ex­pect to run fast on GPUs. Yet, since it uses a di­vide-and-con­quer ap­proach, which is

in­her­ently par­al­lel, Bend will run it multi-threaded. Some bench­marks:

That’s a 57x speedup by do­ing noth­ing. No thread spawn­ing, no ex­plicit man­age­ment of locks, mu­texes. We just asked Bend to run our pro­gram on RTX, and it did. Simple as that.

Bend is­n’t lim­ited to a spe­cific par­a­digm, like ten­sors or ma­tri­ces. Any con­cur­rent sys­tem, from shaders to Erlang-like ac­tor mod­els can be em­u­lated on Bend. For ex­am­ple, to ren­der im­ages in real time, we could sim­ply al­lo­cate an im­mutable tree on each frame:

# given a shader, re­turns a square im­age

def ren­der(depth, shader):

bend d = 0, i = 0:

when d < depth:

color = (fork(d+1, i*2+0), fork(d+1, i*2+1))

else:

width = depth / 2

color = shader(i % width, i / width)

re­turn color

# given a po­si­tion, re­turns a color

# for this demo, it just busy loops

def de­mo_shader(x, y):

bend i = 0:

when i < 5000:

color = fork(i + 1)

else:

color = 0x000001

re­turn color

# ren­ders a 256x256 im­age us­ing de­mo_shader

def main:

re­turn ren­der(16, de­mo_shader)

And it would ac­tu­ally work. Even in­volved al­go­rithms par­al­lelize well on Bend. Long-distance com­mu­ni­ca­tion is per­formed by global beta-re­duc­tion (as per the

Interaction Calculus), and syn­chro­nized cor­rectly and ef­fi­ciently by

HVM2′s atomic linker.

Bend is de­vel­oped by HigherOrderCO.com - join our Discord!

...

Read the original on github.com »

2 614 shares, 70 trendiness

ChatGPT can talk, but OpenAI employees sure can’t

Editor’s note, May 17, 2024, 11:20 pm ET: This story has been up­dated to in­clude a post-pub­li­ca­tion state­ment from OpenAI.

On Monday, OpenAI an­nounced ex­cit­ing new prod­uct news: ChatGPT can now talk like a hu­man.

It has a cheery, slightly in­gra­ti­at­ing fem­i­nine voice that sounds im­pres­sively non-ro­botic, and a bit fa­mil­iar if you’ve seen a cer­tain 2013 Spike Jonze film. Her,” tweeted OpenAI CEO Sam Altman, ref­er­enc­ing the movie in which a man falls in love with an AI as­sis­tant voiced by Scarlett Johansson.

But the prod­uct re­lease of ChatGPT 4o was quickly over­shad­owed by much big­ger news out of OpenAI: the res­ig­na­tion of the com­pa­ny’s co-founder and chief sci­en­tist, Ilya Sutskever, who also led its su­per­align­ment team, as well as that of his co-team leader Jan Leike (who we put on the Future Perfect 50 list last year).

The res­ig­na­tions did­n’t come as a to­tal sur­prise. Sutskever had been in­volved in the board­room re­volt that led to Altman’s tem­po­rary fir­ing last year, be­fore the CEO quickly re­turned to his perch. Sutskever pub­licly re­gret­ted his ac­tions and backed Altman’s re­turn, but he’s been mostly ab­sent from the com­pany since, even as other mem­bers of OpenAI’s pol­icy, align­ment, and safety teams have de­parted.

But what has re­ally stirred spec­u­la­tion was the ra­dio si­lence from for­mer em­ploy­ees. Sutskever posted a pretty typ­i­cal res­ig­na­tion mes­sage, say­ing I’m con­fi­dent that OpenAI will build AGI that is both safe and ben­e­fi­cial…I am ex­cited for what comes next.”

Leike … did­n’t. His res­ig­na­tion mes­sage was sim­ply: I re­signed.” After sev­eral days of fer­vent spec­u­la­tion, he ex­panded on this on Friday morn­ing, ex­plain­ing that he was wor­ried OpenAI had shifted away from a safety-fo­cused cul­ture.

Questions arose im­me­di­ately: Were they forced out? Is this de­layed fall­out of Altman’s brief fir­ing last fall? Are they re­sign­ing in protest of some se­cret and dan­ger­ous new OpenAI pro­ject? Speculation filled the void be­cause no one who had once worked at OpenAI was talk­ing.

It turns out there’s a very clear rea­son for that. I have seen the ex­tremely re­stric­tive off-board­ing agree­ment that con­tains nondis­clo­sure and non-dis­par­age­ment pro­vi­sions for­mer OpenAI em­ploy­ees are sub­ject to. It for­bids them, for the rest of their lives, from crit­i­ciz­ing their for­mer em­ployer. Even ac­knowl­edg­ing that the NDA ex­ists is a vi­o­la­tion of it.

If a de­part­ing em­ployee de­clines to sign the doc­u­ment, or if they vi­o­late it, they can lose all vested eq­uity they earned dur­ing their time at the com­pany, which is likely worth mil­lions of dol­lars. One for­mer em­ployee, Daniel Kokotajlo, who posted that he quit OpenAI due to los­ing con­fi­dence that it would be­have re­spon­si­bly around the time of AGI,” has con­firmed pub­licly that he had to sur­ren­der what would have likely turned out to be a huge sum of money in or­der to quit with­out sign­ing the doc­u­ment.

While nondis­clo­sure agree­ments aren’t un­usual in highly com­pet­i­tive Silicon Valley, putting an em­ploy­ee’s al­ready-vested eq­uity at risk for de­clin­ing or vi­o­lat­ing one is. For work­ers at star­tups like OpenAI, eq­uity is a vi­tal form of com­pen­sa­tion, one that can dwarf the salary they make. Threatening that po­ten­tially life-chang­ing money is a very ef­fec­tive way to keep for­mer em­ploy­ees quiet.

OpenAI did not re­spond to a re­quest for com­ment in time for ini­tial pub­li­ca­tion. After pub­li­ca­tion, an OpenAI spokesper­son sent me this state­ment: We have never can­celed any cur­rent or for­mer em­ploy­ee’s vested eq­uity nor will we if peo­ple do not sign a re­lease or nondis­par­age­ment agree­ment when they exit.”

Sources close to the com­pany I spoke to told me that this rep­re­sented a change in pol­icy as they un­der­stood it. When I asked the OpenAI spokesper­son if that state­ment rep­re­sented a change, they replied, This state­ment re­flects re­al­ity.”

All of this is highly ironic for a com­pany that ini­tially ad­ver­tised it­self as OpenAI — that is, as com­mit­ted in its mis­sion state­ments to build­ing pow­er­ful sys­tems in a trans­par­ent and ac­count­able man­ner.

OpenAI long ago aban­doned the idea of open-sourc­ing its mod­els, cit­ing safety con­cerns. But now it has shed the most se­nior and re­spected mem­bers of its safety team, which should in­spire some skep­ti­cism about whether safety is re­ally the rea­son why OpenAI has be­come so closed.

OpenAI has spent a long time oc­cu­py­ing an un­usual po­si­tion in tech and pol­icy cir­cles. Their re­leases, from DALL-E to ChatGPT, are of­ten very cool, but by them­selves they would hardly at­tract the near-re­li­gious fer­vor with which the com­pany is of­ten dis­cussed.

What sets OpenAI apart is the am­bi­tion of its mis­sion: to en­sure that ar­ti­fi­cial gen­eral in­tel­li­gence — AI sys­tems that are gen­er­ally smarter than hu­mans — ben­e­fits all of hu­man­ity.” Many of its em­ploy­ees be­lieve that this aim is within reach; that with per­haps one more decade (or even less) — and a few tril­lion dol­lars — the com­pany will suc­ceed at de­vel­op­ing AI sys­tems that make most hu­man la­bor ob­so­lete.

Which, as the com­pany it­self has long said, is as risky as it is ex­cit­ing.

Superintelligence will be the most im­pact­ful tech­nol­ogy hu­man­ity has ever in­vented, and could help us solve many of the world’s most im­por­tant prob­lems,” a re­cruit­ment page for Leike and Sutskever’s team at OpenAI states. But the vast power of su­per­in­tel­li­gence could also be very dan­ger­ous, and could lead to the dis­em­pow­er­ment of hu­man­ity or even hu­man ex­tinc­tion. While su­per­in­tel­li­gence seems far off now, we be­lieve it could ar­rive this decade.”

Naturally, if ar­ti­fi­cial su­per­in­tel­li­gence in our life­times is pos­si­ble (and ex­perts are di­vided), it would have enor­mous im­pli­ca­tions for hu­man­ity. OpenAI has his­tor­i­cally po­si­tioned it­self as a re­spon­si­ble ac­tor try­ing to tran­scend mere com­mer­cial in­cen­tives and bring AGI about for the ben­e­fit of all. And they’ve said they are will­ing to do that even if that re­quires slow­ing down de­vel­op­ment, miss­ing out on profit op­por­tu­ni­ties, or al­low­ing ex­ter­nal over­sight.

We don’t think that AGI should be just a Silicon Valley thing,” OpenAI co-founder Greg Brockman told me in 2019, in the much calmer pre-Chat­GPT days. We’re talk­ing about world-al­ter­ing tech­nol­ogy. And so how do you get the right rep­re­sen­ta­tion and gov­er­nance in there? This is ac­tu­ally a re­ally im­por­tant fo­cus for us and some­thing we re­ally want broad in­put on.”

OpenAI’s unique cor­po­rate struc­ture — a capped-profit com­pany ul­ti­mately con­trolled by a non­profit — was sup­posed to in­crease ac­count­abil­ity. No one per­son should be trusted here. I don’t have su­per-vot­ing shares. I don’t want them,” Altman as­sured Bloomberg’s Emily Chang in 2023. The board can fire me. I think that’s im­por­tant.” (As the board found out last November, it could fire Altman, but it could­n’t make the move stick. After his fir­ing, Altman made a deal to ef­fec­tively take the com­pany to Microsoft, be­fore be­ing ul­ti­mately re­in­stated with most of the board re­sign­ing.)

But there was no stronger sign of OpenAI’s com­mit­ment to its mis­sion than the promi­nent roles of peo­ple like Sutskever and Leike, tech­nol­o­gists with a long his­tory of com­mit­ment to safety and an ap­par­ently gen­uine will­ing­ness to ask OpenAI to change course if needed. When I said to Brockman in that 2019 in­ter­view, You guys are say­ing, We’re go­ing to build a gen­eral ar­ti­fi­cial in­tel­li­gence,’” Sutskever cut in. We’re go­ing to do every­thing that can be done in that di­rec­tion while also mak­ing sure that we do it in a way that’s safe,” he told me.

Their de­par­ture does­n’t her­ald a change in OpenAI’s mis­sion of build­ing ar­ti­fi­cial gen­eral in­tel­li­gence — that re­mains the goal. But it al­most cer­tainly her­alds a change in OpenAI’s in­ter­est in safety work; the com­pany has­n’t an­nounced who, if any­one, will lead the su­per­align­ment team.

And it makes it clear that OpenAI’s con­cern with ex­ter­nal over­sight and trans­parency could­n’t have run all that deep. If you want ex­ter­nal over­sight and op­por­tu­ni­ties for the rest of the world to play a role in what you’re do­ing, mak­ing for­mer em­ploy­ees sign ex­tremely re­stric­tive NDAs does­n’t ex­actly fol­low.

This con­tra­dic­tion is at the heart of what makes OpenAI pro­foundly frus­trat­ing for those of us who care deeply about en­sur­ing that AI re­ally does go well and ben­e­fits hu­man­ity. Is OpenAI a buzzy, if mid­size tech com­pany that makes a chatty per­sonal as­sis­tant, or a tril­lion-dol­lar ef­fort to cre­ate an AI god?

The com­pa­ny’s lead­er­ship says they want to trans­form the world, that they want to be ac­count­able when they do so, and that they wel­come the world’s in­put into how to do it justly and wisely.

But when there’s real money at stake — and there are as­tound­ing sums of real money at stake in the race to dom­i­nate AI — it be­comes clear that they prob­a­bly never in­tended for the world to get all that much in­put. Their process en­sures for­mer em­ploy­ees — those who know the most about what’s hap­pen­ing in­side OpenAI — can’t tell the rest of the world what’s go­ing on.

The web­site may have high-minded ideals, but their ter­mi­na­tion agree­ments are full of hard-nosed legalese. It’s hard to ex­er­cise ac­count­abil­ity over a com­pany whose for­mer em­ploy­ees are re­stricted to say­ing I re­signed.”

ChatGPT’s new cute voice may be charm­ing, but I’m not feel­ing es­pe­cially en­am­ored.

A ver­sion of this story orig­i­nally ap­peared in the Future Perfect newslet­ter. Sign up here!

...

Read the original on www.vox.com »

3 476 shares, 23 trendiness

what happens to a game when pi is not 3.14159…

We all know that the value of pi is a con­stant with a par­tic­u­lar im­mutable value. Anyone who has done any graph­i­cal pro­gram­ming also knows that vi­sual ren­der­ing re­lies not just on pi but trigonom­e­try more broadly as well as other math­e­mat­i­cal tech­niques. If we look into the source code of the first per­son shooter Doom we find that the value of pi used in the game is wrong. In this talk I will ex­plore what hap­pens when we sub­tly and not so sub­tly break math in the source.

Doom is a well known clas­sic first per­son shooter game with source code re­leased un­der the GPL in 1999. In this talk I will be­gin by ex­plor­ing what hap­pens to the game when we make the value of pi even more wrong. What about when we change other trigono­met­ric func­tions and con­stants to in­cor­rect val­ues? How will our fa­mil­iar un­der­stand­ing and abil­ity to tra­verse this vir­tual world change when we do this. Are there any in­ter­est­ing gam­ing pos­si­bil­i­ties with non-Eu­clid­ean geome­tries? A brief seg­way will cover some op­ti­miza­tion tricks made to en­able the game to run well on hard­ware avail­able at the time. At the end I will pro­vide a link to other games and pub­lic source code repos­i­to­ries that also use an in­cor­rect value of pi. Pointers will also be pro­vided to al­low the au­di­ence to com­pile their own in­cor­rect math ver­sion of the game.

...

Read the original on media.ccc.de »

4 421 shares, 22 trendiness

x.com

...

Read the original on x.com »

5 368 shares, 17 trendiness

Seeing Cartoons from a New Perspective

Hand-drawn scenes are not 3D con­sis­tent, so we cre­ate Toon3D to re­cover cam­era poses and dense geom­e­try! We do this with a piece­wise-rigid de­for­ma­tion op­ti­miza­tion at hand-la­beled key­points and us­ing monoc­u­lar depth as a prior. Now we can in­ter­po­late novel views never be­fore seen! Press the but­ton to move the cam­eras be­tween two view­points! Note that we re­con­struct the scenes with more than two hand-drawn im­ages, but this demo shows

a smooth tran­si­tion be­tween just two of the in­puts views.

We pro­pose Toon3D. In this work, we re­cover the un­der­ly­ing 3D struc­ture of non-geo­met­ri­cally con­sis­tent scenes. We fo­cus our analy­sis on hand-drawn im­ages from car­toons and anime. Many car­toons are cre­ated by artists with­out a 3D ren­der­ing en­gine, which means that any new im­age of a scene is hand-drawn. The hand-drawn im­ages are usu­ally faith­ful rep­re­sen­ta­tions of the world, but only in a qual­i­ta­tive sense, since it is dif­fi­cult for hu­mans to draw mul­ti­ple per­spec­tives of an ob­ject or scene 3D con­sis­tently. Nevertheless, peo­ple can eas­ily per­ceive 3D scenes from in­con­sis­tent in­puts! In this work, we cor­rect for 2D draw­ing in­con­sis­ten­cies to re­cover a plau­si­ble 3D struc­ture such that the newly warped draw­ings are con­sis­tent with each other. Our pipeline con­sists of a user-friendly an­no­ta­tion tool, cam­era pose es­ti­ma­tion, and im­age de­for­ma­tion to re­cover a dense struc­ture. Our method warps im­ages to obey a per­spec­tive cam­era model, en­abling our aligned re­sults to be plugged into novel-view syn­the­sis re­con­struc­tion meth­ods to ex­pe­ri­ence car­toons from view­points never drawn be­fore.

(Left) We first re­cover cam­era poses and aligned point clouds. (Right) Then we ini­tial­ize Gaussians from our dense point cloud and op­ti­mize Gaussian Splatting with the re­cov­ered cam­eras. Our method has depth reg­u­lar­iza­tion and is built on Nerfstudio. Here we show fly-through ren­ders of our scenes.

Here is the gallery of all our scenes. Can you guess which is which? Click to re­veal names.

We first pre­dict the depth of each im­age with Marigold

and ob­tain can­di­date tran­sient masks with SAM. We then la­bel im­ages with the Toon3D Labeler to ob­tain cor­re­spon­dences and mark tran­sient re­gions. We op­ti­mize cam­era poses and warp im­ages to ob­tain cal­i­brated, per­spec­tive cam­eras. Finally, we can ini­tial­ize Gaussians with the aligned dense point cloud and run re­fine­ment.

Here you can see the two ma­jor steps of our method. The sparse align­ment video shows rough cam­era pa­ra­me­ter es­ti­ma­tion. The dense align­ment video shows var­i­ous lay­ers used in the method (e.g., cam­eras, sparse cor­re­spon­dences, warp­ing meshes, etc.) and how they align in 3D.

We re­con­struct in­side the Rick and Morty house by la­bel­ing be­tween walls and ceil­ings to con­nect the rooms. In the first video, we show the point cloud & cam­eras and our cus­tom la­bel­ing in­ter­face. In the sec­ond video, you can scrub the slider to see a walk­through in­side the house! The clos­est cam­er­a’s im­age is shown in the bot­tom right cor­ner.

We can re­con­struct scenes from few im­ages and with large view­point changes. Where COLMAP may fail, we can in­ter­vene with the Toon3D Labeler to ob­tain hu­man-la­beled cor­re­spon­dences. Here we show a fly-through ren­der­ing for two rooms (“Living room” and Bedroom 2″) of this Airbnb list­ing.

Cartoons are hand-drawn so we need to warp the im­ages to be 3D con­sis­tent. The first item is a video that shows the warp tak­ing place dur­ing align­ment op­ti­miza­tion. The next two items are im­ages which show the orig­i­nal and warped draw­ings, as well as the over­lap be­tween the two. Blurry re­gions in­di­cate where a lot of warp oc­cured.

We can re­con­struct paint­ings with Toon3D even though the paint­ings are hand-drawn. We pre­dict the depth of each im­age, then align and warp point clouds. Finally we use Gaussian re­fine­ment to cre­ate the video shown be­low.

...

Read the original on toon3d.studio »

6 276 shares, 13 trendiness

Thinking out loud about 2nd-gen Email

Note: This is just me, think­ing out loud; you ab­solutely do not need to think that I have care­fully thought this through, or that this is a good idea. With ex­pec­ta­tions set as low as pos­si­ble, let’s con­tinue.

There are many old pieces of tech still in use, but there’s one that grinds my gears every time I try to use it: Email.

For users, email works pretty well. Sometimes it sends too many emails to Junk, but Email is old, re­li­able, easy to un­der­stand, and rel­a­tively easy to search. It’s a good sys­tem, and I’m not ea­ger to re­place it with Slack any­time soon.

However… the back­end for email, is a mess. In es­ca­lat­ing or­der (and we” is used in a very im­pre­cise, broad hand-wav­ing sense for tech­nol­o­gists):

Many things in Email have no spec; even ba­sic things. For ex­am­ple: When you re­ply, are you re­ply­ing at the top of the mes­sage, or the bot­tom? It might even be a po­lit­i­cal ques­tion, de­pend­ing on who you ask. This has been worked around by email clients ba­si­cally guess­ing the or­der and rarely even show­ing the email’s orig­i­nal text, putting lay­ers be­tween the user and the ac­tual mes­sage.

What HTML are you al­lowed to put in an email? Well… it de­pends. When there’s no spec, and Microsoft Outlook abuses the Microsoft Word HTML ren­derer, it gets ugly. There’s no guar­an­tee the re­ceiver even has an HTML ren­derer, and then it’s even more ugly.

How do you en­crypt an email, from the provider it­self? Well, we in­vented this thing called OpenPGP, but al­most no­body used it. Then it turned out to have ma­jor flaws.

We could­n’t al­ways make sure emails were au­then­tic. So we in­vented SPF.

Then it turned out SPF did­n’t fix every­thing. So we in­vented DKIM.

Then it turned out DKIM did­n’t fix every­thing. So we in­vented DMARC.

And then it turns out DKIM it­self has ma­jor flaws that also by­pass DMARC by ex­ten­sion.

Also, be­cause we threw on yet an­other layer called BIMI, which it­self re­lies on DMARC, when DMARC re­lies on DKIM, and DKIM has flaws, then it’s ex­ceed­ingly bad for users.

Even if your sender has DMARC, 68.2% of records have their pol­icy set to p=none. This tells DMARC to ba­si­cally… do noth­ing! (Sure, it tech­ni­cally speak­ing no­ti­fies the do­main owner, but it’s clear the do­main own­ers don’t care.)

Did I men­tion all of the above, plus ag­gres­sive anti-spam poli­cies, makes self-host­ing email in­sanely dif­fi­cult?

Last, but not least, there’s the inane jug­gling of IP rep­u­ta­tion. Some IP ad­dresses are cleaner” than other ad­dresses, es­pe­cially on shared sys­tems like SendGrid or AWS SES. This makes sign­ing up for a mass-mail­ing ac­count, for what­ever rea­son, messy; and causes count­less sur­prise in­stances of le­git­i­mate emails go­ing to Junk. Combine that with IP Address de­ple­tion, and the num­ber of mostly clean ad­dresses is shrink­ing over time.

My gut re­ac­tion to the above is that we’ve got a lousy spec, with decades of cruft and un­of­fi­cial spec, and we aren’t that great at se­cur­ing it, or mak­ing sure mes­sages are au­then­tic. So… could we do bet­ter?

Your ini­tial re­ac­tion might be: That would be point­less, be­cause not every­one would opt into it, and it would break com­pat­i­bil­ity all over the place. My thought is… that’s not nec­es­sar­ily a given. Imagine this:

We cre­ate a new DNS record, called MX2. Most email ser­vices, then, would have an MX2 and MX record. Older ser­vices only have MX.

If an an­cient, 20 year old email client, tries to send a mes­sage — it finds the MX record and sends the mes­sage just like nor­mal. A mod­ern client sees the MX2 and sends the mes­sage there if it ex­ists; oth­er­wise, it falls back to MX.

From there, the email ser­vices which im­ple­ment MX2 would pub­lish a pub­lic date, on which all mes­sages sent to them by the old MX record, will be au­to­mat­i­cally sent to Junk. If just Microsoft and Google alone agreed on such a date, that would be 40% of global email traf­fic.

If the above looks slightly fa­mil­iar, it’s be­cause this strat­egy al­ready worked, in a sense, with the tran­si­tion from HTTP to HTTPS. We threw away a multi-decade-old pro­to­col, for a new and more se­cure one. We set browsers to au­to­mat­i­cally up­grade the con­nec­tion wher­ever pos­si­ble, and now warn users about in­se­cure con­nec­tions when ac­cess­ing HTTP (especially on lo­gin pages). Nevertheless — users can still visit HTTP pages, an­cient browsers still work on HTTP, but most web­sites have got­ten the memo and up­graded to HTTPS any­way.

The in­cen­tive to up­grad­ing to MX2 would be sim­ple: Your mes­sages, while they still would ar­rive, would go to Junk au­to­mat­i­cally past the pub­licly posted date. No busi­ness wants that, even if users are al­ready trained to ex­pect, and act, like that can hap­pen. Thus, the in­cen­tive to up­grade with­out truly break­ing any day-to-day com­pat­i­bil­ity.

Personally, I think that such a tran­si­tion could go even faster than the HTTP to HTTPS tran­si­tion. Self-hosted email is not very pop­u­lar in part be­cause of the com­plex­ity of the cur­rent email sys­tem, so be­tween Microsoft, Google, Amazon, Zoho, GoDaddy, Gandhi, Wix, Squarespace, MailChimp, SparkPost, and SendGrid — you have most of the email mar­ket cov­ered for the US; any­one not in the above list would quickly fold. The rel­a­tive cen­tral­iza­tion of email, iron­i­cally, makes a mass up­grade to email much more achiev­able.

What would a 2nd-gen email pri­or­i­tize then? Everyone has dif­fer­ent pri­or­i­ties, but I’d per­son­ally sug­gest the fol­low­ing which would hope­fully win a broad enough con­sen­sus if this idea goes any­where (though ex­perts, of which I am not one, would have plenty of their own ideas):

A stan­dard­ized HTML spec­i­fi­ca­tion for email; com­plete with a test suite for con­for­mance. Or, maybe we just de­clare a ver­sion of the HTML5 spec to be of­fi­cially bind­ing and that’s the end of it.

Headers for email chain pref­er­ences, or other email-spe­cific pref­er­ences (i.e. Is this email chain a top-re­ply chain, or a bot­tom-re­ply chain? The client should­n’t need to guess, or worse, ig­nore it.)

If an email has a rich, HTML view; it should be re­quired to come with a text-only, non-HTML copy of the body as well; for ac­ces­si­bil­ity, com­pat­i­bil­ity, and pri­vacy rea­sons.

All MX2 records must have a pub­lic key em­bed­ded in the record. To send an email from the do­main:

– A hash of the email con­tent, and all head­ers, is cre­ated.

– This hash is then en­crypted with the pri­vate key, cor­re­spond­ing to the record’s pub­lic key.

– This header is then added to the email, as the only per­mit­ted un­trusted header.

– When an email is re­ceived, the header con­tain­ing the hash is de­crypted with the DNS pub­lic key, and the rest of the email is checked against the hash for in­tegrity and au­then­tic­ity.

Point #4 is a lot like DKIM and DMARC right now, ex­cept:

– There would al­ways be an au­to­matic re­ject pol­icy (p=reject) . Currently, only 19.6% of email ser­vices which even have DKIM are this strin­gent.

– If head­ers do need to be added to an email, the spec can care­fully de­fine carve-outs for where un­trusted data can go (i.e. if the spam fil­ter wanted to add a header).

– There also could be stan­dard­ized carve-outs for, say, ap­pend­ing un­trusted data from the re­ceiv­ing server to a mes­sage body (i.e. your busi­ness could add data to the body’s top or bot­tom in­di­cat­ing that the mes­sage from an ex­ter­nal re­cip­i­ent and you have le­gal oblig­a­tions, but your email client can also clearly show that this was not part of the orig­i­nal mes­sage and is not signed).

– As such, the sign­ing would not need to work around email com­pat­i­bil­ity to such an ex­tent as DKIM, re­duc­ing the like­li­hood of crit­i­cal flaws.

By sim­pli­fy­ing the stack to the above, elim­i­nat­ing SPF, DKIM, and DMARC (and their re­spec­tive con­fig­u­ra­tion op­tions), and stan­dard­iz­ing on one record (MX2) for the fu­ture, run­ning your own self-hosted email stack would be­come much eas­ier. Additionally, the ad­di­tional au­then­tic­ity ver­i­fi­ca­tions would hope­fully al­low spam fil­ters to be sig­nif­i­cantly less ag­gres­sive by au­then­ti­cat­ing against do­mains in­stead of IPs.

Point #6 is the biggest change — we’re no longer au­then­ti­cat­ing, or car­ing, about the IP Address that’s send­ing the email. Every email can and al­ways would be ver­i­fied against the do­main us­ing MX2 records and the pub­lic keys in them. Send a fake spam email? It does­n’t have a sig­na­ture, so it gets tossed with­out any heuris­tics. Send a real spam email? Block that do­main when there’s com­plaints. Go af­ter the reg­is­trar (or treat do­mains be­long­ing to that reg­is­trar as sus­pi­cious) if needed. This would mostly elim­i­nate the need for IP rep­u­ta­tion by re­plac­ing it with do­main rep­u­ta­tion — which, at least to me, is a far su­pe­rior stan­dard with more un­der­stand­able and con­trol­lable out­comes(1).

Clients which im­ple­ment MX2 can, op­tion­ally, have an up­dated en­cryp­tion scheme to re­place OpenPGP. Something like Apple’s Contact Key Verification. Hopefully there would be for­ward se­crecy this time.

If you have got great coun­ter­ar­gu­ments, let me hear them.

(1) This would, per­haps, be the one and only new fea­ture” we could ad­ver­tise to users. Not get­ting emails? You can just type in the name of the web­site, and al­ways re­ceive the emails.

Edit 1, for clar­i­fi­ca­tion: For bulk senders, there would be mul­ti­ple MX2 records on the do­main, each con­tain­ing a pub­lic key for every au­tho­rized sender. One of those records would have a marker in­di­cat­ing it as suit­able for in­com­ing mail.

Gabriel Sieben is a soft­ware de­vel­oper from St. Paul, MN, who en­joys ex­per­i­ment­ing with com­put­ers and loves to share his var­i­ous tech­nol­ogy-re­lated pro­jects. He owns and runs this blog, and is a tra­di­tional Catholic. In his free time (when not mess­ing with com­put­ers), he en­joys hik­ing, fish­ing, and board games.

View more posts

...

Read the original on gabrielsieben.tech »

7 226 shares, 12 trendiness

A Trial HIV Vaccine Triggered Elusive and Essential Antibodies in Humans

DURHAM, N. C. — An HIV vac­cine can­di­date de­vel­oped at the Duke Human Vaccine Institute trig­gered low lev­els of an elu­sive type of broadly neu­tral­iz­ing HIV an­ti­bod­ies among a small group of peo­ple en­rolled in a 2019 clin­i­cal trial.

The find­ing, re­ported May 17 in the jour­nal Cell, not only pro­vides proof that a vac­cine can elicit these an­ti­bod­ies to fight di­verse strains of HIV, but that it can also ini­ti­ate the process within weeks, set­ting in mo­tion an es­sen­tial im­mune re­sponse.

The vac­cine can­di­date tar­gets an area on the HIV-1 outer en­ve­lope called the mem­brane prox­i­mal ex­ter­nal re­gion (MPER), which re­mains sta­ble even as the virus mu­tates. Antibodies against this sta­ble re­gion in the HIV outer coat can block in­fec­tion by many dif­fer­ent cir­cu­lat­ing strains of HIV.

This work is a ma­jor step for­ward as it shows the fea­si­bil­ity of in­duc­ing an­ti­bod­ies with im­mu­niza­tions that neu­tral­ize the most dif­fi­cult strains of HIV,” said se­nior au­thor Bar­ton F. Haynes, M. D., di­rec­tor of the Duke Human Vaccine Institute (DHVI). Our next steps are to in­duce more po­tent neu­tral­iz­ing an­ti­bod­ies against other sites on HIV to pre­vent virus es­cape. We are not there yet, but the way for­ward is now much clearer.”

The re­search team an­a­lyzed data from a phase 1 clin­i­cal trial of a vac­cine can­di­date de­vel­oped by Haynes and S. Munir Alam, Ph. D., at DHVI.

Twenty healthy, HIV-negative peo­ple en­rolled in the trial. Fifteen par­tic­i­pants re­ceived two of four planned doses of the in­ves­ti­ga­tional vac­cine, and five re­ceived three doses.

After just two im­mu­niza­tions, the vac­cine had a 95% serum re­sponse rate and a 100% blood CD4+ T-cell re­sponse rate — two key mea­sure­ments that demon­strated strong im­mune ac­ti­va­tion. Most of the serum re­sponses mapped to the por­tion of the virus tar­geted by the vac­cine.

Importantly, broadly neu­tral­iz­ing an­ti­bod­ies were in­duced af­ter just two doses.

The trial was halted when one par­tic­i­pant ex­pe­ri­enced a non-life-threat­en­ing al­ler­gic re­ac­tion, sim­i­lar to rare in­ci­dences re­ported with COVID-19 vac­ci­na­tions. The team in­ves­ti­gated the cause of the event, which was likely from an ad­di­tive.

To get a broadly neu­tral­iz­ing an­ti­body, a se­ries of events needs to hap­pen, and it typ­i­cally takes sev­eral years post-in­fec­tion,” said lead au­thor Wilton Williams, Ph. D., as­so­ci­ate pro­fes­sor in Duke’s Department of Surgery and mem­ber of DHVI. The chal­lenge has al­ways been to recre­ate the nec­es­sary events in a shorter space of time us­ing a vac­cine. It was very ex­cit­ing to see that, with this vac­cine mol­e­cule, we could ac­tu­ally get neu­tral­iz­ing an­ti­bod­ies to emerge within weeks.”

Other fea­tures of the vac­cine were also promis­ing, most no­tably how the cru­cial im­mune cells re­mained in a state of de­vel­op­ment that al­lowed them to con­tinue ac­quir­ing mu­ta­tions, so they could evolve along with the ever-chang­ing virus.

The re­searchers said there is more work to be done to cre­ate a more ro­bust re­sponse, and to tar­get more re­gions of the virus en­ve­lope. A suc­cess­ful HIV vac­cine will likely have at least three com­po­nents, all aimed at dis­tinct re­gions of the virus.

Ultimately, we will need to hit all the sites on the en­ve­lope that are vul­ner­a­ble so that the virus can­not es­cape,” Haynes said. But this study demon­strates that broadly neu­tral­iz­ing an­ti­bod­ies can in­deed be in­duced in hu­mans by vac­ci­na­tion. Now that we know that in­duc­tion is pos­si­ble, we can repli­cate what we have done here with im­muno­gens that tar­get the other vul­ner­a­ble sites on the virus en­ve­lope.”

In ad­di­tion to Haynes and Williams, study au­thors in­clude S. Munir Alam, Gilad Ofek, Nathaniel Erdmann, David Montefiori, Michael S. Seaman, Kshitij Wagh, Bette Korber, Robert J. Edwards, Katayoun Mansouri, Amanda Eaton, Derek W. Cain, Mitchell Martin, Robert Parks, Maggie Barr, Andrew Foulger, Kara Anasti, Parth Patel, Salam Sammour, Ruth J. Parsons, Xiao Huang, Jared Lindenberger, Susan Fetics, Katarzyna Janowska, Aurelie Niyongabo, Benjamin M. Janus, Anagh Astavans, Christopher B. Fox, Ipsita Mohanty, Tyler Evangelous, Yue Chen, Madison Berry, Helene Kirshner, Elizabeth Van Itallie, Kevin Saunders, Kevin Wiehe, Kristen W. Cohen, M. Juliana McElrath, Lawrence Corey, Priyamvada Acharya, Stephen R. Walsh, and Lindsey R. Baden.

The study re­ceived fund­ing sup­port from the National Institute of Allergy and Infectious Diseases, part of the National Institutes of Health (AI100645, AI144371, AI170752), and from the Bill & Melinda Gates Foundation (OPP1094352/INV-007688).

The con­tent of this press re­lease is solely the re­spon­si­bil­ity of DHVI and does not nec­es­sar­ily rep­re­sent the of­fi­cial views of the National Institutes of Health.

...

Read the original on corporate.dukehealth.org »

8 208 shares, 12 trendiness

EquityVal

Please be as de­scrip­tive as pos­si­ble. For best re­sults in­clude de­tails like:

The page you ex­pe­ri­enced the is­sue on

The ac­tion you were tak­ing

...

Read the original on www.useequityval.com »

9 192 shares, 13 trendiness

google/wuffs: Wrangling Untrusted File Formats Safely

Wuffs is a mem­ory-safe pro­gram­ming lan­guage (and a stan­dard li­brary

writ­ten in that lan­guage) for Wrangling Untrusted File Formats Safely. Wrangling in­cludes pars­ing, de­cod­ing and en­cod­ing. Example file for­mats in­clude im­ages, au­dio, video, fonts and com­pressed archives.

Per its bench­marks and other linked-to blog posts:

* It can de­code bzip2 1.3x faster than /usr/bin/bzcat

(C).

* It can de­code de­flate up to 1.4x faster than zlib-the-li­brary (C).

* It can de­code GIF 2x-6x faster than giflib” (C), image/gif” (Go) and

gif” (Rust).

* It can de­code PNG 1.2x-2.7x faster than libpng” (C), image/png” (Go) and

png”

(Rust).

Wuffs’ goal is to pro­duce soft­ware li­braries that are as safe as Go or Rust, roughly speak­ing, but as fast as C, and that can be used any­where C li­braries are used. This in­cludes very large C/C++ pro­jects, such as pop­u­lar web browsers and op­er­at­ing sys­tems (using that term to in­clude desk­top and mo­bile user in­ter­faces, not just the ker­nel).

Wuffs the Library is avail­able as tran­spiled C code. Other C/C++ pro­jects can use that li­brary with­out

re­quir­ing the Wuffs the Language tool­chain. Those pro­jects can use Wuffs the Library like us­ing any other third party C li­brary. It’s just not hand-writ­ten C.

However, un­like hand-writ­ten C, Wuffs the Language is safe with re­spect to buffer over­flows, in­te­ger arith­metic over­flows and null pointer deref­er­ences. A key dif­fer­ence be­tween Wuffs and other mem­ory-safe lan­guages is that all such

checks are done at com­pile time, not at run time. If it com­piles, it is safe, with re­spect to those three bug classes.

The trade-off in aim­ing for both safety and speed is that Wuffs pro­grams take longer for a pro­gram­mer to write, as they have to ex­plic­itly an­no­tate their

pro­grams with proofs of safety. A state­ment like x += 1 un­sur­pris­ingly means to in­cre­ment the vari­able x by 1. However, in Wuffs, such a state­ment is a com­pile time er­ror un­less the com­piler can also prove that x is not the max­i­mal value of x’s type (e.g. x is not 255 if x is a base.u8), as the in­cre­ment would oth­er­wise over­flow. Similarly, an in­te­ger arith­metic ex­pres­sion like x / y is a com­pile time er­ror un­less the com­piler can also prove that y is not zero.

Wuffs is not a gen­eral pur­pose pro­gram­ming lan­guage. It is for writ­ing

li­braries, not pro­grams. Wuffs code is her­metic

and can only com­pute (e.g. con­vert compressed bytes” to decompressed bytes”).

It can­not make any syscalls (e.g. it has no am­bi­ent au­thor­ity to read your files), im­ply­ing that it can­not al­lo­cate or free mem­ory (and is there­fore triv­ially safe against things like mem­ory leaks, use-af­ter-frees and dou­ble-frees).

It pro­duces Sans I/O style li­braries (but C li­braries, not Python), mean­ing that they are ag­nos­tic to function

col­ors’. They can be com­bined with syn­chro­nous or asyn­chro­nous I/O, as the li­brary caller (not li­brary im­ple­men­ta­tion) is re­spon­si­ble for the ac­tual I/O.

The idea is­n’t to write your whole pro­gram in Wuffs, only the parts that are

both per­for­mance-con­scious and se­cu­rity-con­scious. For ex­am­ple, while tech­ni­cally pos­si­ble, it is un­likely that a Wuffs com­piler would be worth writ­ing en­tirely in Wuffs.

The /std/lzw/decode_lzw.wuffs file is a good ex­am­ple. The Wuffs the Language doc­u­ment has more in­for­ma­tion on how it dif­fers from other lan­guages in the C fam­ily.

For ex­am­ple, mak­ing this one-line edit to the LZW codec leads to a com­pile time er­ror. wuffs gen fails to gen­er­ate the C code, i.e. fails to com­pile (transpile) the Wuffs code to C code:

diff –git a/​std/​lzw/​de­code_lzw.wuffs b/​std/​lzw/​de­code_lzw.wuffs

in­dex f878c5e..f10d­cee 100644

–- a/​std/​lzw/​de­code_lzw.wuffs

+++ b/​std/​lzw/​de­code_lzw.wuffs

@@ -98,7 +98,7 @@ pub func lzw_de­coder.de­code?(dst ptr buf1, src ptr buf1, sr­c_­fi­nal bool)() {

in.dst.write?(x:s)

if use_save_­code {

- this.suf­fixes[save_­code] = c as u8

+ this.suf­fixes[save_­code] = (c + 1) as u8

this.pre­fixes[save_­code] = pre­v_­code as u16

In com­par­i­son, this two-line edit will com­pile (but the does it de­code GIF cor­rectly” tests then fail):

diff –git a/​std/​lzw/​de­code_lzw.wuffs b/​std/​lzw/​de­code_lzw.wuffs

in­dex f878c5e..b43443d 100644

–- a/​std/​lzw/​de­code_lzw.wuffs

+++ b/​std/​lzw/​de­code_lzw.wuffs

@@ -97,8 +97,8 @@ pub func lzw_de­coder.de­code?(dst ptr buf1, src ptr buf1, sr­c_­fi­nal bool)() {

// type check­ing, bounds check­ing and code gen­er­a­tion for it).

in.dst.write?(x:s)

- if use_save_­code {

- this.suf­fixes[save_­code] = c as u8

+ if use_save_­code and (c < 200) {

+ this.suf­fixes[save_­code] = (c + 1) as u8

this.pre­fixes[save_­code] = pre­v_­code as u16

* lang holds the Go li­braries that im­ple­ment Wuffs the Language: to­k­enizer,

AST, parser, ren­derer, etc. The Wuffs tools are writ­ten in Go, but as

men­tioned above, Wuffs tran­spiles to C code, and Go is not nec­es­sar­ily

in­volved if all you want is to use the C edi­tion of Wuffs.

* lib holds other Go li­braries, not spe­cific to Wuffs the Language per se.

* cmd holds Wuffs the Language’ com­mand line tools, also writ­ten in Go.

* re­lease holds the re­leases (e.g. in their C form) of Wuffs the Library.

* test holds the reg­u­lar tests for Wuffs the Library.

* fuzz holds the fuzz tests for Wuffs the Library.

* ex­am­ple holds ex­am­ple pro­grams for Wuffs the Library.

* hello-wuffs-c holds an ex­am­ple pro­gram for Wuffs the Language.

The Note di­rec­tory also con­tains var­i­ous short ar­ti­cles.

* Bindings for Go, Rust and other lan­guages are tracked as is­sue

#38.

Version 0.3 (April 2023) is the lat­est sta­ble ver­sion. Stable means that its API won’t change any fur­ther, but be­ing a version 0.x” means that:

* It will not have long term sup­port.

The com­piler un­doubt­edly has bugs. Assertion check­ing needs more rigor, es­pe­cially around side ef­fects and alias­ing, and be­ing suf­fi­ciently well spec­i­fied to al­low al­ter­na­tive im­ple­men­ta­tions. Lots of de­tail needs work, but the broad brush­strokes are there.

Nonetheless, Wuffs’ GIF de­coder has shipped in the Google Chrome web browser

since June

2021

(milestone M93). See also the ridiculously

fast” tweet al­ready men­tioned above.

The CONTRIBUTING.md file con­tains in­struc­tions on how to file the Contributor License Agreement be­fore send­ing any pull re­quests (PRs). Of course, if you’re new to the pro­ject, it’s usu­ally best to dis­cuss any pro­pos­als and reach con­sen­sus be­fore send­ing your first PR.

This soft­ware is dis­trib­uted un­der the terms of both the MIT li­cense and the Apache License (Version 2.0).

This is not an of­fi­cial Google prod­uct, it is just code that hap­pens to be owned by Google.

Tony is an arse-kick­ing wom­bat who loves play­ing

full-for­ward and hates buffer over­flows.

...

Read the original on github.com »

10 174 shares, 14 trendiness

metaskills/experts: Experts.js is the easiest way to create and deploy OpenAI's Assistants and link them together as Tools to create advanced Multi AI Agent Systems with expanded memory and attention to detail.

Experts.js is the eas­i­est way to cre­ate and de­ploy OpenAI’s Assistants and link them to­gether as Tools to cre­ate a Panel of Experts sys­tem with ex­panded mem­ory and at­ten­tion to de­tail.

The new Assistants API from OpenAI sets a new in­dus­try stan­dard, sig­nif­i­cantly ad­vanc­ing be­yond the widely adopted Chat Completions API. It rep­re­sents a ma­jor leap in the us­abil­ity of AI agents and the way en­gi­neers in­ter­act with LLMs. Paired with the cut­ting-edge GPT-4o model, Assistants can now ref­er­ence at­tached files & im­ages as knowl­edge sources within a man­aged con­text win­dow called a Thread. Unlike Custom GPTs, Assistants sup­port in­struc­tions up to 256,000 char­ac­ters, in­te­grate with 128 tools, and uti­lize the in­no­v­a­tive Vector Store API for ef­fi­cient file search on up to 10,000 files per as­sis­tant.

Experts.js aims to sim­plify the us­age of this new API by re­mov­ing the com­plex­ity of man­ag­ing Run ob­jects and al­low­ing Assistants to be linked to­gether as Tools.

const thread = Thread.create();

const as­sis­tant = await MyAssistant.create();

const out­put = await as­sis­tant.ask(“Say hello.”, thread.id);

con­sole.log(out­put) // Hello

More im­por­tantly, Experts.js in­tro­duces Assistants as Tools, en­abling the cre­ation of Multi AI Agent Systems. Each Tool is an LLM-backed Assistant that can take on spe­cial­ized roles or ful­fill com­plex tasks on be­half of their par­ent Assistant or Tool. Allowing for com­plex or­ches­tra­tion work­flows or chore­o­graph­ing a se­ries of tightly knit tasks. Shown here is an ex­am­ple of a com­pany as­sis­tant with a prod­uct cat­a­log tool which it­self has a LLM backed tool to cre­ate OpenSearch queries.

Install via npm. Usage is very sim­ple, there are only three ob­jects to im­port.

npm in­stall ex­perts

im­port { Assistant, Tool, Thread } from experts”;

* Assistants - The main ob­ject that rep­re­sents an AI agent.

* Tools - An Assistant that can be used by other Assistants.

The con­struc­tor of our Assistant fa­cade ob­ject re­quires a name, de­scrip­tion, and in­struc­tions. The third ar­gu­ment is a set of op­tions which di­rectly maps to all the re­quest body op­tions out­lined in the cre­ate as­sis­tant doc­u­men­ta­tion. All ex­am­ples in Experts.js are writ­ten in ES6 classes for sim­plic­ity. The de­fault model is gpt-4o.

class MyAssistant ex­tends Assistant {

con­struc­tor() {

const name = My Assistant”;

const de­scrip­tion = …”;

const in­struc­tions = …”

su­per(name, de­scrip­tion, in­struc­tions, {

model: gpt-4-turbo”,

tools: [{ type: file_search” }],

tem­per­a­ture: 0.1,

tool_re­sources: {

file_search: {

vec­tor_­s­tore_ids: [process.env.VECTOR_STORE_ID],

const as­sis­tant = await MyAssistant.create();

* Find or cre­ate your as­sis­tant by name.

The ask() func­tion is a sim­ple in­ter­face to ask or in­struct your as­sis­tant(s). It re­quires a mes­sage and a thread iden­ti­fier. More on Threads be­low. The mes­sage can be a string or na­tive OpenAI mes­sage ob­ject. This is where Experts.js re­ally shines. You never have to man­age Run ob­jects or their Run Steps di­rectly.

const out­put = await as­sis­tant.ask(”…”, threa­dID)

const out­put = await as­sis­tant.ask({ role: user”, con­tent: …” }, threa­dID);

Normal OpenAI tools and func­tion call­ing are sup­ported via our con­struc­tors op­tions ob­ject via tools and tool_re­sources. Experts.js also sup­ports adding Assistants as Tools. More in­for­ma­tion on us­ing Assistants as Tools can be found in the next sec­tion. Use the ad­dAs­sis­tant­Tool func­tion to add an Assistant as a Tool.

class MainAssistant ex­tends Assistant {

con­struc­tor() {

const name = Company Assistant;

const de­scrip­tion = …”;

const in­struc­tions = …”;

su­per(name, de­scrip­tion, in­struc­tions);

this.ad­dAs­sis­tant­Tool(Prod­uct­sTools);

By de­fault all Experts.js lever­ages the Assistants Streaming Events. These al­low your ap­pli­ca­tions to re­ceive text, im­age, and tool out­puts via OpenAI’s server-send events. We lever the ope­nai-node stream helpers and sur­face these and more so you can be in con­trol of all events in your agen­tic ap­pli­ca­tions.

const as­sis­tant = await MainAssistant.create();

as­sis­tant.on(“textDelta”, (delta, _snapshot) => {

process.std­out.write(delta.value)

All ope­nai-node stream­ing events are sup­ported via our Assistant’s on() func­tion. The avail­able event names are: event, textDelta, textDone, im­age­File­Done, tool­CallDelta, run­Step­Done, tool­Call­Done, and end

If your lis­ten­ers need to per­form work in an async fash­ion, such as redi­rect­ing tool out­puts, con­sider us­ing our ex­ten­sions to these events. They are called in this or­der af­ter the Run has been com­pleted. The avail­able async event names are: textDoneA­sync, im­age­File­DoneA­sync, run­Step­DoneA­sync, tool­Call­DoneA­sync, and en­dA­sync.

If you want to lazily standup ad­di­tional re­sources when an as­sis­tan­t’s cre­ate() func­tion is called, im­ple­ment the be­for­eInit() func­tion in your class. This is an async method that will be called be­fore the as­sis­tant is cre­ated.

async be­for­eInit() {

await this.#cre­ate­File­Search();

All Assistant events re­ceive an ex­tra Experts’js meta­data ar­gu­ment. An ob­ject that con­tains the Run’s stream. This al­lows you to use the ope­nai-node’s helper func­tions such as cur­rentEvent, fi­nalMes­sages, etc.

as­sis­tant.on(“en­dA­sync”, async (metadata) => {

await meta­data.stream.fi­nalMes­sages();

Using an Assistant as a Tool is cen­tral fo­cal point of the Experts.js frame­work. Tools are a sub­class of Assistant and en­cap­su­late the in­ter­face for their par­ent ob­jects. In this way Experts.js tools are reusable com­po­nents in your agen­tic ar­chi­tec­ture. Our ex­am­ples il­lus­trate a ba­sic mes­sage pass­ing pat­tern, for brevity. You should lever­age all of OpenAI’s tool and func­tion call­ing fea­tures to their fullest.

class EchoTool ex­tends Tool {

con­struc­tor() {

const name = Echo Tool”;

const de­scrip­tion = Echo”;

const in­struc­tions = Echo the same text back to the user”;

su­per(name, de­scrip­tion, in­struc­tions, {

par­entsTools: [

type: function”,

func­tion: {

name: EchoTool.toolName,

de­scrip­tion: de­scrip­tion,

pa­ra­me­ters: {

type: object”,

prop­er­ties: { mes­sage: { type: string” } },

re­quired: [“message”],

As such, Tool class names are im­por­tant and help OpenAI’s mod­els de­cide which tool to call. So pick a good name for your tool class. For ex­am­ple, ProductsOpenSearchTool will be prod­uct­s_open_search and clearly helps the model in­fer along with the tool’s de­scrip­tion what role it per­forms.

Tools are added to your Assistant via the ad­dAs­sis­tant­Tool func­tion. This func­tion will add the tool to the as­sis­tan­t’s tools ar­ray and up­date the as­sis­tan­t’s con­fig­u­ra­tion.

class MainAssistant ex­tends Assistant {

con­struc­tor() {

const name = Company Assistant;

const de­scrip­tion = …”;

const in­struc­tions = …”;

su­per(name, de­scrip­tion, in­struc­tions);

this.ad­dAs­sis­tant­Tool(EchoTool);

Your Tool as­sis­tant re­sponse will au­to­mat­i­cally be sub­mit­ted as the out­put for the par­ent Assistant or Tool.

By de­fault Tools are backed by an LLM model and per­form all the same life­cy­cles events, runs, etc as Assistants. However, you can cre­ate a Tool that does not use any of the core Assistant’s fea­tures by set­ting the llm op­tion to false. When do­ing so, you must im­ple­ment the ask() func­tion in your Tool. The re­turn value will be sub­mit­ted as the tool’s out­put.

class AnswerTwoTool ex­tends Tool {

con­struc­tor() {

su­per(name, de­scrip­tion, in­struc­tions, {

llm: false,

par­entsTools: […],

async ask(mes­sage) {

re­turn …;

In com­plex work­flows, a LLM backed Tool can be used to con­vert hu­man or other LLM in­struc­tions into ex­e­cutable code and the re­sult of that code (not the LLM out­put) would need to be sub­mit­ted for your Tool’s par­en­t’s out­puts. For ex­am­ple, the ProductsOpenSearchTool could con­vert mes­sages into OpenSearch queries, ex­e­cute them, and re­turn the re­sults. Sub classes can im­ple­ment the an­swered() func­tion to con­trol the out­put. In this case, the out­put would be an OpenSearch query and the tools out­puts now con­tain the re­sults of that LLM-generated query.

async an­swered(out­put) {

const args = JSON.parse(output);

re­turn await this.opensearch­Query(args);

Alternatively, LLM backed Tools could opt to redi­rect their own tool out­puts back to their par­ent Assistant or Tool. Thus ig­nor­ing the LLM out­put. This also al­lows for all of a Tools tool out­puts to be sub­mit­ted as the par­en­t’s out­put. More on why this is im­por­tant in the prod­uct cat­a­log ex­am­ple be­low.

class ProductsTool ex­tends Tool {

con­struc­tor() {

su­per(name, de­scrip­tion, in­struc­tions, {

tem­per­a­ture: 0.1,

tools: [{ type: code_interpreter” }],

out­puts: tools”,

par­entsTools: […],

this.ad­dAs­sis­tant­Tool(Prod­uct­sOpenSearch­Tool);

this.on(“im­age­File­DoneA­sync”, this.im­age­File­DoneA­sync.bind(this));

OpenAI’s Assistants API in­tro­duces a new re­source called Threads which mes­sages & files are stored within. Essentially, threads are a man­aged con­text win­dow (memory) for your agents. Creating a new thread with Experts.js is as easy as:

...

Read the original on github.com »

To add this web app to your iOS home screen tap the share button and select "Add to the Home Screen".

10HN is also available as an iOS App

If you visit 10HN only rarely, check out the the best articles from the past week.

If you like 10HN please leave feedback and share

Visit pancik.com for more.