Hoppa till hu­vudin­nehåll

Physical but­tons are in­creas­ingly rare in mod­ern cars. Most man­u­fac­tur­ers are switch­ing to touch­screens — which per­form far worse in a test car­ried out by Vi Bilägare. The dri­ver in the worst-per­form­ing car needs four times longer to per­form sim­ple tasks than in the best-per­form­ing car.

Inspiration for the screen-heavy in­te­ri­ors in mod­ern cars comes from smart­phones and tablets. Designers want a ”clean” in­te­rior with min­i­mal switchgear, and the fi­nan­cial de­part­ment wants to lower the cost. Instead of de­vel­op­ing, man­u­fac­tur­ing and keep­ing phys­i­cal but­tons in stock for years to come, car man­u­fac­tur­ers are keen on in­te­grat­ing more func­tions into a dig­i­tal screen which can be up­dated over time.

So in what way have these screens af­fected safety? Vi Bilägare gath­ered eleven mod­ern cars from dif­fer­ent man­u­fac­tur­ers at an air­field och mea­sured the time needed for a dri­ver to per­form dif­fer­ent sim­ple tasks, such as chang­ing the ra­dio sta­tion or ad­just­ing the cli­mate con­trol. At the same time, the car was dri­ven at 110 km/​h (68 mph). We also in­vited an ”old-school” car with­out a touch­screen, a 17-year-old Volvo V70, for com­par­i­son.

Introducing the Electronic Materials Office Altar I — an ul­tra-low pro­file, wire­less me­chan­i­cal key­board

forged

in pre­mium, sus­tain­able ma­te­ri­als.

Altar I fo­cuses on one idea: Less, but bet­ter. A pre­ci­sion ma­chined alu­minium

mono­body. Slim, tac­tile, me­chan­i­cal switches. Typographically bal­anced. Bluetooth. Quietly un­der­stated.

And it looks good on a desk.

Altar I is cur­rently in pri­vate, in­vite only beta. To gain ac­cess,

add your email be­low. No spam — only info on pric­ing, avail­abil­ity and an in­vi­ta­tion to

pur­chase when avail­able.

Crunchy PostgresIntegrated high avail­abil­ity PostgreSQL so­lu­tion for en­ter­prises with al­ways on re­quire­ments. Customers

Today I’m ex­cited to in­tro­duce a new place for devs to pol­ish their Postgres skills, a Postgres play­ground from Crunchy Data. What is the play­ground? Put sim­ply it is:

With canned datasets you can load

Guided tu­to­ri­als to fol­low along to learn about the power of Postgres

Wait?!?!? Postgres in the browser? Yep. You can jump right over to our play­ground and check out some of the tu­to­ri­als right away. Or read on to learn a bit more on what you can do.

Postgres is awe­some, which is­n’t re­ally a se­cret around these parts. But know­ing Postgres is awe­some is just the first piece, we want it to be easy and ac­ces­si­ble for every­one. We want you to be able to take full ad­van­tage of things like rich datatypes such a JSONB, Arrays, and UUIDs. Leverage the rich set of built-in func­tions and in­dex types. And be able to query your data­base with power to get the in­sights you need from your data.

All of the above is why we started com­pil­ing our own col­lec­tion of Postgres tips from our team to share with you all. Today we’re go­ing a step fur­ther with the play­ground where you can run Postgres in your browser and fol­low along with some tu­to­ri­als to level up your Postgres skills.

At launch our play­ground in­cludes tu­to­ri­als for:

Often times the gap in try­ing/​learn­ing some­thing in Postgres is hav­ing a good tan­gi­ble ex­am­ple. The play­ground makes that eas­ier by load­ing a dataset then guid­ing you step by step through an ex­er­cise lever­ag­ing that dataset in a prac­ti­cal way. Whether it’s just the ba­sics of in­ter­act­ing with the Postgres CLI with psql , im­prov­ing your query­ing skills with SQL, or dig­ging into per­for­mance analy­sis we want some­thing for every­one to be able to level up your skills. Our guided tu­to­ri­als fo­cus on prac­ti­cal uses and ex­am­ples as op­posed to purely aca­d­e­mic de­f­i­n­i­tions. One ex­am­ple you’ll find a real world sce­nario in com­put­ing week-over-week dif­fer­ences lever­ag­ing CTEs and win­dow func­tions.

Some of the tech­ni­cal de­tails

We’ll be do­ing a fol­low-up post on how the play­ground all came to be, but to give you a few of the ques­tions you’re prob­a­bly won­der­ing off­hand.

Yes, this is WASM and run­ning fully in your web browser. Due to browser sand­box­ing there is no way to con­nect di­rectly to the Postgres in­stance be­yond the em­bed­ded psql in­ter­face that we es­tab­lish for you. The cur­rent con­fig­u­ra­tion al­lo­cates 512MB of mem­ory for your Postgres in­stance, we may make this more con­fig­urable in the fu­ture. It’s in the browser, hence if you re­fresh you’re go­ing to get a fresh in­stance, we haven’t cre­ated any per­sis­tence lay­ers (yet). And fi­nally, it’s early, you may run into some weird states if you do we ask that you send us what you were do­ing when it hap­pened and then at­tempt to re­fresh.

This is just the be­gin­ning. We’ll be con­tin­u­ing to add other tu­to­ri­als and fea­tures to this area. If you want to stay tuned for up­dates we en­cour­age you to sub­scribe to our newslet­ter. And let us know you’re feed­back @crunchydata and what you’d like to see im­proved. Get more de­liv­ered to your in­boxDo not fill this out please: Do not fill this out please, it will be pre-filled:

Subscribe to the Crunchy Data Newsletter to re­ceive Postgres con­tent every month. Do not fill this out please: Do not fill this out please, it will be pre-filled:

This site uses cook­ies for us­age an­a­lyt­ics to im­prove our ser­vice. By con­tin­u­ing to browse this site, you agree to this use. Learn more

Last week I pub­lished a re­port on the risks of mo­bile apps us­ing in-app browsers. Some apps, like Instagram and Facebook, in­ject JavaScript code into third party web­sites that cause po­ten­tial se­cu­rity and pri­vacy risks to the user.

I was so happy to see the ar­ti­cle fea­tured by ma­jor me­dia out­lets across the globe, like TheGuardian and The Register, gen­er­ated a over a mil­lion im­pres­sions on Twitter, and was ranked #1 on HackerNews for more than 12 hours. After read­ing through the replies and DMs, I saw a com­mon ques­tion across the com­mu­nity:

“How can I ver­ify what apps do in their we­b­views?”

Introducing InAppBrowser.com, a sim­ple tool to list the JavaScript com­mands ex­e­cuted by the iOS app ren­der­ing the page.

To try this this tool your­self:

Open an app you want to an­a­lyze

Share the url https://​In­App­Browser.com some­where in­side the app (e.g. send a DM to a friend, or post to your feed)

Tap on the link in­side the app to open it

Read the re­port on the screen

I started us­ing this tool to an­a­lyze the most pop­u­lar iOS apps that have their own in-app browser. Below are the re­sults I’ve found.

For this analy­sis I have ex­cluded all third party iOS browsers (Chrome, Brave, etc.), as they use JavaScript to of­fer some of their func­tion­al­ity, like a pass­word man­ager. Apple re­quires all third party iOS browsers apps to use the Safari ren­der­ing en­gine WebKit.

Important Note: This tool can’t de­tect all JavaScript com­mands ex­e­cuted, as well as does­n’t show any track­ing the app might do us­ing na­tive code (like cus­tom ges­ture recog­nis­ers). More de­tails on this be­low.

InAppBrowser.com is de­signed for every­body to ver­ify for them­selves what apps are do­ing in­side their in-app browsers. I have de­cided to open source the code used for this analy­sis, you can check it out on GitHub. This al­lows the com­mu­nity to up­date and im­prove this script over time.

* Option to open in de­fault browser: Does the app pro­vide a but­ton to open the cur­rently shown link in the de­fault browser?

* Modify page: Does the app in­ject JavaScript code into third party web­sites to mod­ify its con­tent? This in­cludes adding track­ing code (like in­puts, text se­lec­tions, taps, etc.), in­ject­ing ex­ter­nal JavaScript files, as well as cre­at­ing new HTML el­e­ments.

* Fetch meta­data: Does the app run JavaScript code to fetch web­site meta­data? This is a harm­less thing to do, and does­n’t cause any real se­cu­rity or pri­vacy risks.

* JS: A link to the JavaScript code that I was able to de­tect. Disclaimer: There might be other code ex­e­cuted. The code might not be a 100% ac­cu­rate rep­re­sen­ta­tion of all JS com­mands.

Click on the Yes or None on the above table to see a screen­shot of the app.

Important: Just be­cause an app in­jects JavaScript into ex­ter­nal web­sites, does­n’t mean the app is do­ing any­thing ma­li­cious. There is no way for us to know the full de­tails on what kind of data each in-app browser col­lects, or how or if the data is be­ing trans­ferred or used. This pub­li­ca­tion is stat­ing the JavaScript com­mands that get ex­e­cuted by each app, as well as de­scrib­ing what ef­fect each of those com­mands might have. For more back­ground on the risks of in-app browsers, check out last week’s pub­li­ca­tion.

Even if some of the apps above have green check­marks, they might use the new WKContentWorld iso­lated JavaScript, which I’ll de­scribe be­low.

When you open any link on the TikTok iOS app, it’s opened in­side their in-app browser. While you are in­ter­act­ing with the web­site, TikTok sub­scribes to all key­board in­puts (including pass­words, credit card in­for­ma­tion, etc.) and every tap on the screen, like which but­tons and links you click.

* TikTok iOS sub­scribes to every key­stroke (text in­puts) hap­pen­ing on third party web­sites ren­dered in­side the TikTok app. This can in­clude pass­words, credit card in­for­ma­tion and other sen­si­tive user data. (keypress and key­down). We can’t know what TikTok uses the sub­scrip­tion for, but from a tech­ni­cal per­spec­tive, this is the equiv­a­lent of in­stalling a key­log­ger on third party web­sites.

* TikTok iOS sub­scribes to every tap on any but­ton, link, im­age or other com­po­nent on web­sites ren­dered in­side the TikTok app.

* TikTok iOS uses a JavaScript func­tion to get de­tails about the el­e­ment the user clicked on, like an im­age (document.elementFromPoint)

Here is a list of all JavaScript com­mands I was able to de­tect.

The com­pany con­firmed those fea­tures ex­ist in the code, but said TikTok is not us­ing them.

“Like other plat­forms, we use an in-app browser to pro­vide an op­ti­mal user ex­pe­ri­ence, but the Javascript code in ques­tion is used only for de­bug­ging, trou­bleshoot­ing and per­for­mance mon­i­tor­ing of that ex­pe­ri­ence — like check­ing how quickly a page loads or whether it crashes,” spokesper­son Maureen Shanahan said in a state­ment.

The above state­ment con­firms my find­ings. TikTok in­jects code into third party web­sites through their in-app browsers that be­haves like a key­log­ger. However claims it’s not be­ing used.

Last week’s post talked about how Meta in­jects the pcm.js script onto third party web­sites. Meta claimed they only in­ject the script to re­spect the user’s ATT choice, and ad­di­tional “security and user fea­tures”.

The code in ques­tion al­lows us to re­spect peo­ple’s pri­vacy choices by help­ing ag­gre­gate events (such as mak­ing a pur­chase on­line) from pix­els al­ready on web­sites, be­fore those events are used for ad­ver­tis­ing or mea­sure­ment pur­poses.

After im­prov­ing the JavaScript de­tec­tion, I now found some ad­di­tional com­mands Instagram ex­e­cutes:

* Instagram iOS sub­scribes to every tap on any but­ton, link, im­age or other com­po­nent on ex­ter­nal web­sites ren­dered in­side the Instagram app.

* Instagram iOS sub­scribes to every time the user se­lects a UI el­e­ment (like a text field) on third party web­sites ren­dered in­side the Instagram app.

Here is a list of all JavaScript com­mands I was able to de­tect.

Note on sub­scrib­ing: When I talk about “App sub­scribes to”, I mean that the app sub­scribes to the JavaScript events of that type (e.g. all taps). There is no way to ver­ify what hap­pens with the data.

Since iOS 14.3 (December 2020), Apple in­tro­duced the sup­port of run­ning JavaScript code in the con­text of a spec­i­fied frame and con­tent world. JavaScript com­mands ex­e­cuted us­ing this ap­proach can still fully ac­cess the third party web­site, but can’t be de­tected by the web­site it­self (in this case a tool like InAppBrowser.com).

Use a WKContentWorld ob­ject as a name­space to sep­a­rate your ap­p’s web en­vi­ron­ment from the en­vi­ron­ment of in­di­vid­ual web­pages or scripts you ex­e­cute. Content worlds help pre­vent is­sues that oc­cur when two scripts mod­ify en­vi­ron­ment vari­ables in con­flict­ing ways. […] Changes you make to the DOM are vis­i­ble to all script code, re­gard­less of con­tent world.

This new sys­tem was ini­tially built so that web­site op­er­a­tors can’t in­ter­fere with JavaScript code of browser plu­g­ins, and to make fin­ger­print­ing more dif­fi­cult. As a user, you can check the source code of any browser plu­gin, as you are in con­trol over the browser it­self. However with in-app browsers we don’t have a re­li­able way to ver­ify all the code that is ex­e­cuted.

So when Meta or TikTok want to hide the JavaScript com­mands they ex­e­cute on third party web­sites, all they’d need to do is to up­date their JavaScript run­ner:

For ex­am­ple, Firefox for iOS al­ready uses the new WKContentWorld sys­tem. Due to the open source na­ture of Firefox and Google Chrome for iOS it’s easy for us as a com­mu­nity to ver­ify noth­ing sus­pi­cious is hap­pen­ing.

Especially af­ter the pub­lic­ity of last week’s post, as well as this one, tech com­pa­nies that still use cus­tom in-app browsers will very quickly up­date to use the new WKContentWorld iso­lated JavaScript sys­tem, so their code be­comes un­de­tectable to us.

Hence, it be­comes more im­por­tant than ever to find a so­lu­tion to end the use of cus­tom in-app browsers for show­ing third party con­tent.

There are many valid rea­sons to use an in-app browser, par­tic­u­larly when an app ac­cesses its own web­sites to com­plete spe­cific trans­ac­tions. For ex­am­ple, an air­line app might not have the seat se­lec­tion im­ple­mented na­tively for their whole air­plane fleet. Instead they might choose to reuse the web-in­ter­face they al­ready have. If they weren’t able to in­ject cook­ies or JavaScript com­mands in­side their we­b­view, the user would have to re-lo­gin while us­ing the app, just so they can se­lect their seat. Shoutout to Venmo, which uses their own in-app browser for all their in­ter­nal web­sites (e.g. Terms of Service), but as soon as you tap on an ex­ter­nal link, they au­to­mat­i­cally tran­si­tion over to SFSafariViewController.

However, there are data pri­vacy & in­tegrity is­sues when you use in-app browsers to visit non-first party web­sites, such as how Instagram and TikTok show all ex­ter­nal web­sites in­side their app. More im­por­tantly, those apps rarely of­fer an op­tion to use a stan­dard browser as de­fault, in­stead of the in-app browser. And in some cases (like TikTok), there is no but­ton to open the cur­rently shown page in the de­fault browser.

The apps be­low fol­low Apple’s rec­om­men­da­tion of us­ing Safari or SFSafariViewController for view­ing ex­ter­nal web­sites. More con­text on SFSafariViewController in the orig­i­nal ar­ti­cle.

All apps that use SFSafariViewController or Default Browser are on the safe side, and there is no way for apps to in­ject any code onto web­sites, even with the new WKContentWorld sys­tem.

As a user of an app

Most in-app browsers have a way to open the cur­rently shown web­site in Safari. As soon as you land in­side an in-app browser, use the Open in Browser fea­ture to switch to a safer browser. If that but­ton is­n’t avail­able, you will have to copy & paste the URL to open the link in the browser of your choice. If the app makes it dif­fi­cult to even do that, you can tap & hold a link on the web­site and then use the Copy fea­ture, which can be a lit­tle tricky to get right.

TikTok does­n’t have a but­ton to open web­sites in the de­fault browser.

Update: According to some tweets, some­times there is a way to open web­sites in the de­fault browser.

If you’re at a com­pany where you have an in-app browser, use it only for your own pages and open all ex­ter­nal links in the user’s de­fault browser. Additionally, pro­vide a set­ting to let users choose a de­fault browser over an in-app browser ex­pe­ri­ence. Unfortunately, these types of changes rarely get pri­or­i­tized over fea­tures that move met­rics in­side of tech or­ga­ni­za­tions. However, it’s so im­por­tant for peo­ple to ed­u­cate oth­ers on their team, and their man­agers, about the pos­i­tive im­pact of mak­ing bet­ter se­cu­rity and pri­vacy de­ci­sions for the user. These changes can be trans­par­ently mar­keted to users as an op­por­tu­nity to build fur­ther trust.

It’s im­por­tant to call out how much move­ment there’s been in the pri­vacy of data space, but it’s un­clear how many of these changes have been mo­tion vs. true progress for the in­dus­try and the user.

“Many tech com­pa­nies take heat for ‘abusing their users’ pri­va­cy’, when in fact they try to bal­ance out busi­ness pri­or­i­ties, great user ex­pe­ri­ences, and en­sur­ing they are re­spect­ing pri­vacy and user data. It’s clear why com­pa­nies were mo­ti­vated to pro­vide an in-app ex­pe­ri­ence for ex­ter­nal web­sites in the first place.

With the lat­est tech­nol­ogy, com­pa­nies can start to pro­vide a smooth ex­pe­ri­ence for the user, while re­spect­ing their pri­vacy. It’s pos­si­ble for iOS or Android de­vel­op­ers to move the pri­vacy stan­dards and re­spon­si­bil­ity to Apple & Google (e.g. stricter app re­views, more per­mis­sion screens, etc.), how­ever this is a much larger con­ver­sa­tion where com­pa­nies need to work to­gether to de­fine what stan­dards should ex­ist. We can’t have one or two com­pa­nies set the di­rec­tion for the en­tire in­dus­try, since a so­lu­tion needs to work for the large ma­jor­ity of com­pa­nies. Otherwise, we’re left in a world where com­pa­nies are forced to get cre­ative on find­ing ways to track ad­di­tional user data from any source pos­si­ble, or de­fine their own stan­dards of what’s best for user pri­vacy, ul­ti­mately hurt­ing the con­sumer and the prod­uct ex­pe­ri­ence.”

Technology-wise App-Bound Domains seems to be an ex­cel­lent new WebKit fea­ture mak­ing it pos­si­ble for de­vel­op­ers to of­fer a safer in-app brows­ing ex­pe­ri­ence when us­ing WKWebView. As an app de­vel­oper, you can de­fine which do­mains your app can ac­cess (your own), and you won’t be able to con­trol third party pages any more. To dis­able the pro­tec­tion, a user would have to ex­plic­itly dis­able it in the iOS set­tings app. However, at the time of writ­ing, this sys­tem is not yet en­abled by de­fault.

* Can in-app browsers read every­thing I do on­line? Yes, if you are brows­ing through their in-app browser they tech­ni­cally can.

* Do the apps above ac­tu­ally steal my pass­words, ad­dress and credit card num­bers? No! I wanted to show­case that bad ac­tors could get ac­cess to this data with this ap­proach. As shown in the past, if it’s pos­si­ble for a com­pany to get ac­cess to data legally and for free, with­out ask­ing the user for per­mis­sion, they will track it.

* How can I pro­tect my­self? Whenever you open a link from any app, see if the app of­fers a way to open the cur­rently shown web­site in your de­fault browser. During this analy­sis, every app be­sides TikTok of­fered a way to do this.

* Are com­pa­nies do­ing this on pur­pose? Building your own in-app browser takes a non-triv­ial time to pro­gram and main­tain, sig­nif­i­cantly more than just us­ing the pri­vacy and user-friendly al­ter­na­tive that’s al­ready been built into the iPhone for the past 7 years. Most likely there is some mo­ti­va­tion there for the com­pany to track your ac­tiv­i­ties on those web­sites.

* I opened InAppBrowser.com in­side an app, and it does­n’t show any com­mands. Am I safe? No! First of all, the web­site only checks for one of many hun­dreds of at­tack vec­tors: JavaScript in­jec­tion from the app it­self. And even for those, as of December 2020, app de­vel­op­ers can com­pletely hide the JavaScript com­mands they ex­e­cute, there­fore there is no way for us to ver­ify what is ac­tu­ally hap­pen­ing un­der the hood.

Many FreeBSD ports and apps, Linux apps, and even triv­ial Darwin & ma­cOS bi­na­ries can run on ravynOS. And we’re ac­tively work­ing on im­prov­ing it!

Many FreeBSD ports and apps, Linux apps, and even triv­ial Darwin & ma­cOS bi­na­ries can run on ravynOS. And we’re ac­tively work­ing on im­prov­ing it!

We’re build­ing fa­mil­iar apps na­tive to ravynOS, pack­aged as portable  .app bun­dles and built with our core tech­nolo­gies of Cocoa, Qt, and Mach. These are guar­an­teed to work and fit seam­lessly into your desk­top.

FreeBSD ports that sup­port Wayland con­tinue to work on ravynOS right out of the box, or with mi­nor changes.

Many Linux ap­pli­ca­tions that sup­port Wayland are ex­pected to work with FreeBSD’s Linux em­u­la­tion layer.

Trivial ma­cOS and Darwin ap­pli­ca­tions may run di­rectly on ravynOS. This is an ac­tive area of work and re­search. AppKit-based source code may build and run na­tively.

From Day 1 to Gear Making Guru

Ultimate Guide and Essential Tools

Show me the way!

Opt In for New Patterns

I know what you’re think­ing…. an­other “old guard” geezer post­ing about how bad things are with noth­ing but anec­do­tal ev­i­dence that this is ac­tu­ally the case. Before you judge me, please watch the video.  I don’t be­lieve un­hap­pi­ness is un­rea­son­able when one is forced to ac­cept con­cepts such as “throw-away so­ci­ety” and “poor value / high over­all own­er­ship costs” and “bad own­er­ship ex­pe­ri­ence” in the name of “progress.”

The back-story to this video is as fol­lows.  A cou­ple years ago a lo­cal gen­tle­man brought his 1947 GE fridge to me for rewiring and re-gas­ket­ing. That job went well and he has been us­ing that fridge ever since as a bev­er­age fridge.

Fast-forward to last week, and his main kitchen fridge stopped work­ing. Without warn­ing, it started click­ing on and off on the com­pres­sor over­load, as if the com­pres­sor was stuck. I nor­mally don’t work on mod­ern ap­pli­ances; how­ever this is lo­cal and the owner was able to bring it over in the back of his pickup with lit­tle ef­fort on his part. So I agreed to look at it.

The com­pres­sor would not start. I tried boost­ing with higher volt­ages, adding ca­pac­i­tors etc. and noth­ing would break it free.  So, I ended up putting in a sal­vaged com­pres­sor to get his go­ing.

Here is a video where I tear it down and in­spect it for root cause of fail­ure. Again, I need to re­peat that I am not say­ing every­thing “old” is bet­ter just be­cause it’s “old.”  I am say­ing that this com­pres­sor, and its ap­pli­ca­tion, show clear en­gi­neer­ing choices made, which sac­ri­ficed its life span in the name of some mod­icum of en­ergy sav­ings.  These mar­ginal de­sign choices are not the only way to get an ef­fi­cient unit, since the fridge com­pres­sors from the 1940′s and 1950′s era were very ef­fi­cient, while hav­ing con­sis­tently longer lives. This failed unit is purely an ex­am­ple of do­ing just enough to get by un­til it is some­one else’s prob­lem. When it be­comes a prob­lem, it goes to the dump where all the foamed-to­gether plas­tic parts will not be fea­si­ble to sep­a­rate nor re­cy­cle.

It was­n’t for noth­ing

Several friends and fam­ily mem­bers have asked me why I spend so much of my free time de­vel­op­ing Guix. I don’t al­ways have a good an­swer, but af­ter six years I’m start­ing to get a sense of it.

First of all, it’s fun. It’s fun to make my own op­er­at­ing sys­tem. I can de­sign it ex­actly how I want it, from the in­ner work­ings to the user in­ter­face.

It’s also re­ward­ing. While I haven’t been as ac­tive on the patch tracker as I’d like lately, it gives a sense of awe and ac­com­plish­ment to aid new con­trib­u­tors from be­gin­ners to “professionals”. It’s also some­thing of a dopamine rush to per­form huge mi­gra­tions like switch­ing to a new C com­piler or Python ver­sion up­grades (these ac­tiv­i­ties feel more like an ad­dic­tion than a hobby, but re­ward­ing nonethe­less).

The work feels im­por­tant. Guix oc­cu­pies an un­der­served niche in the op­er­at­ing sys­tem “market”. Namely one that of­fers full re­pro­ducibil­ity, trans­parency, and prove­nance, of every pack­age, and every op­er­at­ing sys­tem gen­er­a­tion. It also in­te­grates a neat script­ing lan­guage where you can string to­gether any pack­age into one de­ter­min­is­tic script. Think of it like a shell script where every com­mand is lazily in­stan­ti­ated by the pack­age man­ager on de­mand. You don’t even have to use Scheme, you can write in your fa­vorite lan­guage and run it through guix shell and get the same de­ter­min­is­tic guar­an­tees. If you note the Guix re­vi­sion (as given by guix de­scribe), you can use guix time-ma­chine to re­pro­duce the ex­act same en­vi­ron­ment in the fu­ture, bit by bit (at least in the­ory).

Guix is pure. It’s re­fresh­ing to know that I can take any pack­age and reuse its source code for my own pur­poses with­out hav­ing to worry about com­mer­cial re­stric­tions or re­dis­tri­b­u­tion. It’s re­as­sur­ing to know that all pack­ages are boot­strapped and not just some ran­dom bi­nary blobs. Guix is also im­mensely hack­able, it’s easy to add ex­tra pack­ages or ser­vices with­out sub­mit­ting them, ei­ther by cre­at­ing a chan­nel or “inlining” straight in the OS con­fig­u­ra­tion.

But most im­por­tantly, Guix saved my life. When I first started con­tribut­ing to Guix I did not have any di­rec­tion in life, any sense of pur­pose or be­long­ing. I’d been us­ing GNU/Linux for a long time, been a “power user” of Debian, Gentoo, and NixOS, but did not con­tribute much to any ei­ther be­cause pack­ag­ing was too much work, or I did­n’t en­joy the com­mu­nity, or I found the pack­ag­ing stan­dards “too low”. With Guix I fi­nally felt home, it ticked all the boxes, the lan­guage is a joy to work with (even though it felt weird in the be­gin­ning), and it also des­per­ately needed de­vel­op­ers.

I know some are dis­ap­pointed that I have pri­or­i­tized a hobby over do­ing “normal” things like chas­ing an ex­cit­ing ca­reer or strength­en­ing my so­cial net­work. But with­out Guix I would still be a con­fused al­co­holic with daddy is­sues crash­ing on some­ones couch. I have now been with Guix for longer than I held any job, and am fi­nally feel­ing some sta­bil­ity in life; and a de­sire to ex­plore other cre­ative out­lets such as blog­ging (hi!), cook­ing, grow­ing sus­tain­able foods; and healthy ac­tiv­i­ties like climb­ing and ski­ing.

None of this would be pos­si­ble with­out Guix, and the (occasionally re­luc­tant) sup­port from my awe­some friends and fam­ily. So thank you. It was­n’t for noth­ing. Rather the op­po­site.

As a pay­ment op­er­a­tions startup, ac­count­ing prin­ci­ples are core to our work. We have seen them im­ple­mented at scale at some of the largest fin­techs and mar­ket­places. Yet, ac­count­ing seems like an ar­cane topic when you’re start­ing out. This HackerNews thread, for ex­am­ple, is rather rep­re­sen­ta­tive of the state of con­fu­sion around the topic.

Over the years, there have been many ac­count­ing for de­vel­oper guides pub­lished (two great ones are Martin Blais’ and Martin Kleppmans’), but we wanted to add our two cents to the dis­cus­sion. In our ex­pe­ri­ence, a con­cepts-first ap­proach to ex­plain­ing ac­count­ing comes in handy when you are de­sign­ing sys­tems that move or touch money.

This post is di­vided into two parts. Part I cov­ers the foun­da­tional ac­count­ing prin­ci­ples. Part II will show how to bring it all to­gether by walk­ing through how to build a Venmo clone.

This guide is de­signed for de­vel­op­ers that work on ap­pli­ca­tions that han­dle money in any way. You may work at a fin­tech com­pany (the data you han­dle is money), or per­haps you are re­spon­si­ble for man­ag­ing the fin­tech in­te­gra­tions in your startup (you han­dle data and money). We think that every en­gi­neer that builds or main­tains such sys­tems ben­e­fits from know­ing the core prin­ci­ples of ac­count­ing.

Double-entry sys­tems are more re­li­able at track­ing money than any other vi­able al­ter­na­tive. As a pay­ments in­fra­struc­ture com­pany, we of­ten get to see the ar­chi­tec­ture of some of the most suc­cess­ful soft­ware com­pa­nies. One de­sign is con­stant: they use dou­ble-en­try ac­count­ing in their code. Some build their ap­pli­ca­tions from the start with ac­count­ing con­cepts in mind, but in most cases, com­pa­nies be­gin in­cor­po­rat­ing these con­cepts only af­ter their orig­i­nal code starts caus­ing prob­lems.

When soft­ware fails to track money prop­erly, it does so in a num­ber of com­mon pat­terns. The most com­mon fail­ure mode is soft­ware ac­ci­den­tally cre­at­ing or de­stroy­ing records of funds. This leads to all sorts of in­con­sis­ten­cies. Every de­vel­oper we know has hor­ror sto­ries about ex­plain­ing to their fi­nance team why a cus­tomer is owed money or what caused a pay­out to have an un­ex­pected amount. Internal records dif­fer­ing from bank state­ments, rec­on­cil­i­a­tion en­gines gone awry, bal­ances that don’t make sense given a set of trans­ac­tions—these are all prob­lems that can be mit­i­gated with dou­ble-en­try ac­count­ing. For more ev­i­dence that dou­ble-en­try sys­tems are a good stan­dard for scal­able ap­pli­ca­tions, see the sto­ries of Uber, Square, and Airbnb.

The core prin­ci­ple of dou­ble-en­try ac­count­ing is that every trans­ac­tion should record both where the money came from and what the money was used for. This guide ex­plains why that is and how it works.

An ac­count is a seg­re­gated pool of value. The eas­i­est anal­ogy here is your own bank check­ing ac­count: money that a bank is hold­ing on your be­half, clearly de­mar­cated as yours. Any dis­crete bal­ance can be an ac­count: from a user’s bal­ance on Venmo to the an­nual de­fense spend­ing of the United States. Accounts gen­er­ally cor­re­late with the bal­ances you want to track.

In ac­count­ing, ac­counts have types. More on this later.

Transactions are atomic events that af­fect ac­count bal­ances. Transactions are com­posed of en­tries. A trans­ac­tion has at least two en­tries, each of which cor­re­sponds to one ac­count.

Let’s use a sim­ple Venmo trans­fer as an ex­am­ple. Jim is send­ing $50 to Mary:

The en­tries in this trans­ac­tion tell which ac­counts were af­fected. If each user’s bal­ance is set up as an ac­count, a trans­ac­tion can si­mul­ta­ne­ously write an en­try against each ac­count.

Now, let’s ex­pand this model with more ac­counts and a hand­ful of ad­di­tional events:

Here I have a ledger—a log of events with mon­e­tary im­pact. We of­ten see de­vel­op­ers mu­tat­ing bal­ances di­rectly rather than com­put­ing a bal­ance from a log of trans­ac­tions. This is sub­op­ti­mal.

While mu­tat­ing a bal­ance di­rectly is more ef­fi­cient and sim­pler to im­ple­ment, it’s more ac­cu­rate to store im­mutable trans­ac­tions and al­ways com­pute bal­ances from those trans­ac­tions. Mutating bal­ances di­rectly cre­ates a sys­tem that is prone to er­rors, as it be­comes non-triv­ial to de­tect and rec­on­cile in­ac­cu­ra­cies.

Notice how each trans­ac­tion has mul­ti­ple en­tries. Each en­try be­longs to a trans­ac­tion and an ac­count. By com­par­ing en­tries side by side, one can clearly see where the money came from and what it was used for. Double-entry en­sures that, as trans­ac­tions are logged, sources and uses of funds are clearly shown, and bal­ances can be re­con­structed as of any date:

This core idea—one trans­ac­tion, at least two en­tries, one rep­re­sent­ing the source and the other rep­re­sent­ing the use of funds—is one of the foun­da­tional ideas of dou­ble-en­try ac­count­ing [1]. We’ll ex­pand more on this later.

As men­tioned be­fore, an­other big in­no­va­tion of ac­count­ing was giv­ing types to ac­counts. The two types we will cover here are debit nor­mal and credit nor­mal.

* Accounts that rep­re­sent funds you own, or uses of money, are debit nor­mal ac­counts.

* Accounts that rep­re­sent funds you owe, or sources of money, are credit nor­mal ac­counts.

Let’s il­lus­trate that with a sim­ple table with two columns, the right side list­ing credit nor­mal ac­counts and the left side list­ing debit nor­mal ac­counts. We will place ac­counts that track uses of funds on the debit nor­mal side and ac­counts that track sources of funds on the credit nor­mal side.

Examples of uses of funds are as­sets and ex­penses. Buying in­ven­tory, mak­ing an in­vest­ment, ac­quir­ing phys­i­cal prop­erty, and so on. The term “use” here is broadly de­fined: let­ting cash sit in a bank ac­count is a use of funds, as well as sell­ing on credit to some­one else (you are ef­fec­tively ‘using’ the money you’d get on a sale by ex­tend­ing them credit). The ac­counts that rep­re­sent these bal­ances are all debit nor­mal ac­counts.

Conversely, sources of funds—such as li­a­bil­i­ties, eq­uity, or rev­enue—can mean bank loans, in­vestors’ cap­i­tal, ac­cu­mu­lated prof­its, or in­come. “Source” is broadly de­fined here, too: if you are buy­ing on credit, for in­stance, that is a “source” of money for you in the sense that it pre­vents you from spend­ing money right now. Accounts that rep­re­sent these bal­ances are credit nor­mal ac­counts.

Here is a handy table with the dif­fer­ent ac­count types:

Some of the guides we men­tioned at the be­gin­ning of this post ad­vise de­vel­op­ers to “save the con­fu­sion and flush out deb­its and cred­its from your mind.” We do rec­og­nize that deb­its and cred­its can be chal­leng­ing to grasp, but we think fully mas­ter­ing these con­cepts is im­por­tant when cre­at­ing trans­ac­tion han­dling rules.

Part of the con­fu­sion is that “debits” and “credits” are of­ten used as verbs: to debit or to credit an ac­count. Debits and cred­its can also re­fer to en­tries, for ex­am­ple:

This sam­ple trans­ac­tion has two en­tries: we’re deb­it­ing our cash ac­count and cred­it­ing our eq­uity ac­count for $1M. Save for a few spe­cial sit­u­a­tions, ac­count­ing sys­tems only log pos­i­tive num­bers. The ef­fect on bal­ances will de­pend on whether the en­try is on the “debit side“ or “credit side.”

Debits and cred­its are a short­hand for the ex­pected ef­fects on ac­counts, de­pend­ing on their type. A credit en­try will al­ways in­crease the bal­ance of a credit nor­mal ac­count and de­crease the bal­ance of a debit nor­mal ac­count. Put dif­fer­ently:

Let’s con­tinue to model out a few trans­ac­tions to drive this point home. Let’s use a fic­ti­tious startup called Modern Bagelry—an eCom­merce store for pre­mium bagels.

In this ex­am­ple, we will use four ac­counts: Cash and Inventory (both debit nor­mal ac­counts) as well as Equity and Loans (both credit nor­mal ac­counts). Let’s say this ledger starts on a day T, and we are mea­sur­ing time in days.

A com­mon mis­con­cep­tion is that one ac­count needs to de­crease while an­other needs to in­crease. However, they can both in­crease or de­crease in tan­dem, de­pend­ing on the debit and credit en­tries in the trans­ac­tion and the ac­count types. In the first trans­ac­tion cash in­creases be­cause it’s a debit en­try in a debit nor­mal ac­count (cash); eq­uity also in­creases be­cause it’s a credit en­try in a credit nor­mal ac­count (equity). Conversely, in the last trans­ac­tion both bal­ances de­crease be­cause we are adding a debit en­try into a credit nor­mal ac­count (loans) and a credit en­try into a debit nor­mal ac­count (cash).

Tracking sources and uses of funds in a sin­gle ledger with dou­ble-en­try is help­ful to clearly show clearly that bal­ances match.

Let’s say we are ag­gre­gat­ing the bal­ances for each ac­count in the ex­am­ple above right af­ter each trans­ac­tion takes place:

A sys­tem of ac­counts will bal­ance as long as the bal­ance on debit nor­mal ac­counts equals the bal­ance on credit nor­mal ac­counts. The end­ing bal­ances of Cash ($1.22M) and Inventory ($250k) sum to $1.47M. That is equal to the sum of end­ing bal­ances of Equity ($1M) and Loans ($470k). It is said that our ac­counts in this ex­am­ple are bal­anced. Not match­ing would mean the sys­tem cre­ated or lost money out of noth­ing.

We’ll walk through how to im­ple­ment these con­cepts in more de­tail in our next post.

Let’s re­cap the prin­ci­ples we re­viewed so far:

* A ledger is a time­stamped log of events that have a mon­e­tary im­pact.

* An ac­count is a dis­crete pool of value that rep­re­sents a bal­ance you want to track.

* A trans­ac­tion is an event recorded in the ledger.

* Transactions must have two or more en­tries.

* Entries be­long to a ledger trans­ac­tion and also be­long to an ac­count.

* Accounts can be clas­si­fied as credit nor­mal or debit nor­mal.

* Entries can be added onto the ledger “on the debit side” or “on the credit side.” Debits and cred­its re­fer to how a given en­try will af­fect an ac­coun­t’s bal­ance:Deb­its—or en­tries on the debit side—in­crease the bal­ance of debit nor­mal ac­counts, while cred­its de­crease it.Cred­its—or en­tries on the credit side—in­crease the bal­ance of credit nor­mal ac­counts, while deb­its de­crease it.

* Debits—or en­tries on the debit side—in­crease the bal­ance of debit nor­mal ac­counts, while cred­its de­crease it.

* Credits—or en­tries on the credit side—in­crease the bal­ance of credit nor­mal ac­counts, while deb­its de­crease it.

* If the sum of bal­ances of all credit nor­mal ac­counts matches the sum of bal­ances of all debit nor­mal ac­counts in a sin­gle ledger, it is said that the ledger is bal­anced. This is an as­sur­ance of con­sis­tency and that all money is prop­erly ac­counted for.

In part two, we’ll add some com­plex­ity to trans­ac­tion struc­tures, and we’ll bring every­thing to­gether by do­ing a walk­through of how to build a Venmo clone.

If you are a de­vel­oper who works with money, Modern Treasury Ledgers sim­pli­fies the process of build­ing a dou­ble-en­try sys­tem. Reach out to us here if we can be help­ful.

TMC youth wing stage protest against cen­tral agen­cies over crack­down on party lead­ers

TMC youth wing stage protest against cen­tral agen­cies over crack­down on party lead­ers

Diljit Dosanjh and his an­tics to keep the fans en­ter­tained