10 interesting stories served every morning and every evening.

1 1,119 shares, 47 trendiness, 1455 words and 15 minutes reading time

Why procrastination is about managing emotions, not time

Why pro­cras­ti­na­tion is about man­ag­ing emo­tions, not timeAd­dress the real rea­sons you pro­cras­ti­nate and you’re more likely to start achiev­ing your goals. Like many writ­ers, I’m a supreme ex­pert at pro­cras­ti­na­tion. When I ought to be work­ing on an as­sign­ment, with the clock tick­ing to­wards my dead­line, I’ll sit there watch­ing point­less po­lit­i­cal in­ter­views or box­ing high­lights on YouTube (cat videos aren’t my thing). At its worst I can al­most be­gin to feel a lit­tle crazy – you need to be work­ing, I say to my­self, so what on Earth are you do­ing?

According to tra­di­tional think­ing — still es­poused by uni­ver­sity coun­selling cen­tres around the world, such as the University of Manchester in the UK and the University of Rochester in the US — I, along with my fel­low pro­cras­ti­na­tors, have a time man­age­ment prob­lem. By this view, I haven’t fully ap­pre­ci­ated how long my as­sign­ment is go­ing to take and I’m not pay­ing enough at­ten­tion to how much time I’m cur­rently wast­ing on cyberloafing’. With bet­ter sched­ul­ing and a bet­ter grip on time, so the logic goes, I will stop pro­cras­ti­nat­ing and get on with my work.

Increasingly, how­ever, psy­chol­o­gists are re­al­is­ing this is wrong. Experts like Tim Pychyl at Carleton University in Canada and his col­lab­o­ra­tor Fuschia Sirois at the University of Sheffield in the UK have pro­posed that pro­cras­ti­na­tion is an is­sue with man­ag­ing our emo­tions, not our time. The task we’re putting off is mak­ing us feel bad – perhaps it’s bor­ing, too dif­fi­cult or we’re wor­ried about fail­ing – and to make our­selves feel bet­ter in the mo­ment, we start do­ing some­thing else, like watch­ing videos.Chronic pro­cras­ti­na­tion is linked with men­tal and phys­i­cal health costs, from de­pres­sion and anx­i­ety to car­dio­vas­cu­lar dis­ease (Credit: Alamy)This fresh per­spec­tive on pro­cras­ti­na­tion is be­gin­ning to open up ex­cit­ing new ap­proaches to re­duc­ing the habit; it could even help you im­prove your own ap­proach to work. Self-change of any of sort is not a sim­ple thing, and it typ­i­cally fol­lows the old adage of two steps for­ward and one step back,” says Pychyl. All of this said, I am con­fi­dent that any­one can learn to stop pro­cras­ti­nat­ing.”

One of the first in­ves­ti­ga­tions to in­spire the emo­tional view of pro­cras­ti­na­tion was pub­lished in the early 2000s by re­searchers at Case Western Reserve University in Ohio. They first prompted peo­ple to feel bad (by ask­ing them to read sad sto­ries) and showed that this in­creased their in­cli­na­tion to pro­cras­ti­nate by do­ing puz­zles or play­ing video games in­stead of prepar­ing for the in­tel­li­gence test they knew was com­ing. Subsequent stud­ies by the same team showed low mood only in­creases pro­cras­ti­na­tion if en­joy­able ac­tiv­i­ties are avail­able as a dis­trac­tion, and only if peo­ple be­lieve they can change their moods. One study used mood-freezing can­dles’ to trick some vol­un­teers into think­ing their low mood was frozen and, in this case, they did­n’t bother pro­cras­ti­nat­ing.

The emo­tional reg­u­la­tion the­ory of pro­cras­ti­na­tion makes in­tu­itive sense. In my case, it’s not that I don’t re­alise how long my as­sign­ment will take (I know I need to be work­ing on it right now) or that I haven’t sched­uled enough time for my YouTube view­ing – in fact, I don’t re­ally even want to watch those videos, I’m just drawn to them as a way of avoid­ing the dis­com­fort of knuck­ling down to work. In the psy­chol­o­gists’ jar­gon, I’m pro­cras­ti­nat­ing to achieve a short-term pos­i­tive hedonic shift’, at the cost of my longer-term goals.Pro­cras­ti­na­tion — while ef­fec­tively dis­tract­ing in the short-term — can lead to guilt, which ul­ti­mately com­pounds the ini­tial stress­The emo­tional reg­u­la­tion view of pro­cras­ti­na­tion also helps ex­plain some strange mod­ern phe­nom­ena, like the fad for watch­ing on­line cat videos which have at­tracted bil­lions of views on YouTube. A sur­vey of thou­sands of peo­ple by Jessica Myrick at the Media School at Indiana University con­firmed pro­cras­ti­na­tion as a com­mon mo­tive for view­ing the cat videos and that watch­ing them led to a boost in pos­i­tive mood. It’s not that peo­ple had­n’t ad­e­quately sched­uled time for watch­ing the videos; of­ten they were only watch­ing the clips to make them­selves feel bet­ter when they should be do­ing some­thing else less fun.

Myrick’s re­search also high­lighted an­other emo­tional as­pect to pro­cras­ti­na­tion. Many of those sur­veyed felt guilty af­ter watch­ing the cat videos. This speaks to how pro­cras­ti­na­tion is a mis­guided emo­tional reg­u­la­tion strat­egy. While it might bring short-term re­lief, it only stores up prob­lems for later. In my own case, by de­lay­ing my work I just end up feel­ing even more stressed, not to men­tion the gath­er­ing clouds of guilt and frus­tra­tion.

It’s per­haps lit­tle won­der that re­search by Fuschia Sirois has shown chronic pro­cras­ti­na­tion — that is, be­ing in­clined to pro­cras­ti­nate on a reg­u­lar, long-term ba­sis — is as­so­ci­ated with a host of ad­verse men­tal and phys­i­cal health con­se­quences, in­clud­ing anx­i­ety and de­pres­sion, poor health such as colds and flu, and even more se­ri­ous con­di­tions like car­dio­vas­cu­lar dis­ease.Re­searchers say pro­cras­ti­nat­ing helps us feel bet­ter when cer­tain tasks fill us with neg­a­tive emo­tions — if they are too dif­fi­cult or bor­ing, say (Credit: Getty Images)Sirois be­lieves pro­cras­ti­na­tion has these ad­verse con­se­quences through two routes – first, it’s stress­ful to keep putting off im­por­tant tasks and fail­ing to ful­fil your goals, and sec­ond, the pro­cras­ti­na­tion of­ten in­volves de­lay­ing im­por­tant health be­hav­iours, such as tak­ing up ex­er­cise or vis­it­ing the doc­tor. Over time high stress and poor health be­hav­iours are well known to have a syn­er­gis­tic and cu­mu­la­tive ef­fect on health that can in­crease risk for a num­ber of se­ri­ous and chronic health con­di­tions such as heart dis­ease, di­a­betes, arthri­tis, and even can­cer,” she says.

All of this means that over­com­ing pro­cras­ti­na­tion could have a ma­jor pos­i­tive im­pact on your life. Sirois says her re­search sug­gests that decreasing a ten­dency to chron­i­cally pro­cras­ti­nate by one point [on a five-point pro­cras­ti­na­tion scale] would also po­ten­tially mean that your risk for hav­ing poor heart health would re­duce by 63%”.

On a pos­i­tive note, if pro­cras­ti­na­tion is an emo­tional reg­u­la­tion is­sue, this of­fers im­por­tant clues for how to ad­dress it most ef­fec­tively. An ap­proach based on Acceptance and Commitment Therapy or ACT, an off-shoot of Cognitive Behavioural Therapy, seems es­pe­cially apt. ACT teaches the ben­e­fits of psychological flex­i­bil­i­ty’ – that is, be­ing able to tol­er­ate un­com­fort­able thoughts and feel­ings, stay­ing in the pre­sent mo­ment in spite of them, and pri­ori­tis­ing choices and ac­tions that help you get closer to what you most value in life.

Relevant here is cut­ting edge re­search that’s shown stu­dents who pro­cras­ti­nate more tend to score higher on psy­cho­log­i­cal in­flex­i­bil­ity. That is, they’re dom­i­nated by their psy­cho­log­i­cal re­ac­tions, like frus­tra­tion and worry, at the ex­pense of their life val­ues; high scor­ers agree with state­ments like I’m afraid of my feel­ings’ and My painful ex­pe­ri­ences and mem­o­ries make it dif­fi­cult for me to live a life that I would val­ue’. Those who pro­cras­ti­nate more also score lower on committed ac­tion’, which de­scribes how much a per­son per­sists with ac­tions and be­hav­iours in pur­suit of their goals. Low scor­ers tend to agree with state­ments like If I feel dis­tressed or dis­cour­aged, I let my com­mit­ments slide’.Re­search shows that once the first step is made to­wards a task, fol­low­ing through be­comes eas­ier­ACT trains peo­ple both to in­crease their psy­cho­log­i­cal flex­i­bil­ity (for ex­am­ple, through mind­ful­ness) and their com­mit­ted ac­tion (for ex­am­ple, by find­ing cre­ative ways to pur­sue goals that serve their val­ues — what mat­ters most to them in life), and pre­lim­i­nary re­search  in­volv­ing stu­dents has been promis­ing, with ACT prov­ing more ef­fec­tive than CBT in one trial over the longer-term.

Of course, most of us prob­a­bly won’t have the op­tion of sign­ing up to an ACT course any time soon — and in any case we’re bound to keep putting off look­ing for one — so how can we go about ap­ply­ing these prin­ci­ples to­day? When some­one fi­nally recog­nises that pro­cras­ti­na­tion is­n’t a time man­age­ment prob­lem but is in­stead an emo­tion reg­u­la­tion prob­lem, then they are ready to em­brace my favourite tip,” says Pychyl.

The next time you’re tempted to pro­cras­ti­nate, make your fo­cus as sim­ple as What’s the next ac­tion – a sim­ple next step – I would take on this task if I were to get started on it now?’”. Doing this, he says, takes your mind off your feel­ings and onto eas­ily achiev­able ac­tion. Our re­search and lived ex­pe­ri­ence show very clearly that once we get started, we’re typ­i­cally able to keep go­ing. Getting started is every­thing.”

Dr Christian Jarrettis a se­nior ed­i­tor at Aeon mag­a­zine. His next book, about per­son­al­ity change, will be pub­lished in 2021.


Read the original on www.bbc.com »

2 973 shares, 29 trendiness, 927 words and 7 minutes reading time

Leaving Google

After ~12.5 years at Google and ~10 years work­ing on Go (#golang), it’s time for me to do some­thing new. Tomorrow is my last day at Google.

Working at Google and on Go has been a high­light of my ca­reer. Go re­ally made pro­gram­ming fun for me again, and I’ve had fun help­ing make it. I want to thank Rob Pike for let­ting me work on Go full time (instead of just as a dis­trac­tion on painfully long gBus rides) as well as Russ Cox and Ian Lance Taylor and Robert Griesemer and oth­ers for all the pa­tience while I learned my way around. I’ve loved hack­ing on var­i­ous pack­ages and sys­tems with the team and mem­bers of the com­mu­nity, giv­ing a bunch of talks, hang­ing out in Denver, Sydney, MTV, NYC, at FOSDEM and other meet-ups, etc. While I’ve learned a bunch while work­ing on Go, more ex­cit­ingly I dis­cov­ered many things that I did­n’t know I did­n’t know, and it was a joy watch­ing the whole team and com­mu­nity work their (to me) magic.

I’ll still be around the Go com­mu­nity, but less, and dif­fer­ently. My @golang.org email will con­tinue to work and please con­tinue to mail me or copy me on GitHub (@bradfitz), es­pe­cially for some­thing bro­ken that might be my fault.

In some­what chrono­log­i­cal or­der, but not en­tirely:

Worked on the Social Graph API, con­tin­u­ally ex­tract­ing and FOAF and other such

se­man­tic so­cial links be­tween pages, in­dex­ing it all, and ex­port­ing it over a pub­lic API. It’s since been shut down,

but I learned a lot about Google in­dex­ing, pro­duc­tion (Borg, BigTable, MapReduce), got up to speed on Google-used

lan­guages & style (C++, Java, Python, Sawzall).

Web-ified the Google open source CLA process; it pre­vi­ously in­volved fax­ing and stuff, which no­body wanted to do.

Worked on Gmail’s back­end for a bit, specif­i­cally its ad­dress book back­end.

Integrated per­sonal ad­dress book search into Google’s main search. (So search­ing google.com for some­body in your con­tacts

would show your ad­dress book en­try for them.) It dark launched as an ex­per­i­ment, but never went live be­cause at the time

we did­n’t have enough SSDs in enough data cen­ters to meet la­tency bud­gets. (I re­mem­ber meet­ings with var­i­ous teams

draw­ing Gannt charts a few mil­lisec­onds wide, show­ing RPCs and la­tency bud­gets… blew my mind at the time.)

Rewrote mem­cached (which I wrote pre-Google) in Google C++

and us­ing Google’s RPC sys­tem and then added mem­cache sup­port

to App Engine. The Google mem­cache server con­tin­ues to be

used by many teams.

Fixed a bunch of per­for­mance bugs in Android when it first came out, got in­vited to join the team.

Worked on the Android frame­work team do­ing per­for­mance analy­sis and tool­ing and fixes.

(some of the com­mits)

Did a bunch of logs pro­cess­ing

on nightly up­loaded per­for­mance sam­ples from Android team & Googler dog­food phones and iden­ti­fied prob­lem­atic code & stack traces & RPCs

caus­ing UI jank.

Worked on a dis­trib­uted build sys­tem for Android that let you do make -j9999 of ar­bi­trary code out over

a clus­ter of ma­chine build­ing in a cus­tom FUSE filesys­tem that watched what got writ­ten and com­mu­ni­cated back to the co­or­di­na­tor

server. It was in­ter­nal code, but its spirit lives on in han­wen/​ter­mite, a Go im­ple­men­ta­tion writ­ten


Started writ­ing in Go for var­i­ous Android analy­sis tasks.

Started writ­ing in Go for per­sonal pro­jects (Perkeep, then named Camlistore)

Started send­ing changes to Go to fix/​add to the Go stan­dard li­brary. (starting May 5, 2010 with some os. Chtimes


http stuff)

Invited to join the Go team.

Worked on most of the Go stan­dard li­brary. Primary au­thor of net/​http, data­base/​sql, os/​exec, Go’s build/​test CI sys­tem, etc.

20% pro­ject: co-au­thor of PubSubHubBub, which be­came WebSub.

Rewrote the Google down­load server from C++ to Go. This was one

of (or the?) first pro­duc­tion Go ser­vice at Google, and in­volved plenty of de­pen­dency work to make it hap­pen. It also showed

var­i­ous short­com­ings in the Go run­time which are long since fixed.

Gave many Go talks at var­i­ous con­fer­ences.

Wrote Go’s HTTP/2 im­ple­men­ta­tion, client and server, in Go 1.6.

8 man­agers, IIRC. (not at once, for­tu­nately; Eight bosses?” Eight.” Eight, Bob”)

15 Go re­leases (and sev­eral be­fore Go 1)

See go/​brad­fitz for the in­ter­nal ver­sion of this doc­u­ment. (It’s ap­prox­i­mately the same but with a bit more stuff I can’t or don’t want to share pub­licly.)

Why are you leav­ing?

Little bored. Not learn­ing as much as I used to. I’ve been do­ing the same thing too long and need a change. It’d be nice to pri­mar­ily work in Go rather than work on Go.

When I first joined Google it was a chaotic first cou­ple years while I learned Google’s in­ter­nal code­base, build sys­tem, a bunch of new lan­guages, Borg, Bigtable, etc. Then I joined Android it was fun/​learn­ing chaos again. Go was the same when I joined and it was a new, fast-mov­ing ex­per­i­ment. Now Go is very pop­u­lar, sta­ble and, while there’s a lot to do, things–of­ten nec­es­sar­ily–move pretty slowly. Moving slowly is fine, and hy­per-spe­cial­iz­ing in small cor­ners of Go makes sense at scale (few per­cent im­prove­ments add up!), but I want to build some­thing new again.

I don’t want to get stuck in a com­fort­able rut. (And Google cer­tainly is com­fort­able, ex­cept for open floor plans.)

TBA. But build­ing some­thing new.


Read the original on bradfitz.com »

3 753 shares, 31 trendiness, 335 words and 3 minutes reading time

cesar featuring A Pimp Named Slickback on Twitter

Skip to con­tent

earth or some­where close to it

You can add lo­ca­tion in­for­ma­tion to your Tweets, such as your city or pre­cise lo­ca­tion, from the web and via third-party ap­pli­ca­tions. You al­ways have the op­tion to delete your Tweet lo­ca­tion his­tory. Learn more

Public · Anyone can fol­low this list

Private · Only you can ac­cess this list

Here’s the URL for this Tweet. Copy it to eas­ily share with friends.

Add this Tweet to your web­site by copy­ing the code be­low. Learn more

Add this video to your web­site by copy­ing the code be­low. Learn more

Hmm, there was a prob­lem reach­ing the server.

By em­bed­ding Twitter con­tent in your web­site or app, you are agree­ing to the Twitter Developer Agreement and Developer Policy.

Why you’re see­ing this ad

Not on Twitter? Sign up, tune into the things you care about, and get up­dates as they hap­pen.

» See SMS short codes for other coun­tries

This time­line is where you’ll spend most of your time, get­ting in­stant up­dates about what mat­ters to you.

Hover over the pro­file pic and click the Following but­ton to un­fol­low any ac­count.

When you see a Tweet you love, tap the heart — it lets the per­son who wrote it know you shared the love.

Add your thoughts about any Tweet with a Reply. Find a topic you’re pas­sion­ate about, and jump right in.

Get in­stant in­sight into what peo­ple are talk­ing about now.

Get more of what you love

Follow more ac­counts to get in­stant up­dates about top­ics you care about.

See the lat­est con­ver­sa­tions about any topic in­stantly.

Catch up in­stantly on the best sto­ries hap­pen­ing as they un­fold.

This is why i use ad block­ers and a pi-hole server…….. re­quest map of ny­times.pic.twit­ter.com/​yf9Sv4­daQH

Thanks. Twitter will use this to make your time­line bet­ter.

Loading seems to be tak­ing a while.

Twitter may be over ca­pac­ity or ex­pe­ri­enc­ing a mo­men­tary hic­cup. Try again or visit Twitter Status for more in­for­ma­tion.


Read the original on twitter.com »

4 710 shares, 22 trendiness, 2423 words and 22 minutes reading time

Leaked Documents Expose the Secretive Market for Your Web Browsing Data

An an­tivirus pro­gram used by hun­dreds of mil­lions of peo­ple around the world is sell­ing highly sen­si­tive web brows­ing data to many of the world’s biggest com­pa­nies, a joint in­ves­ti­ga­tion by Motherboard and PCMag has found. Our re­port re­lies on leaked user data, con­tracts, and other com­pany doc­u­ments that show the sale of this data is both highly sen­si­tive and is in many cases sup­posed to re­main con­fi­den­tial be­tween the com­pany sell­ing the data and the clients pur­chas­ing it.

The doc­u­ments, from a sub­sidiary of the an­tivirus gi­ant Avast called Jumpshot, shine new light on the se­cre­tive sale and sup­ply chain of peo­ples’ in­ter­net brows­ing his­to­ries. They show that the Avast an­tivirus pro­gram in­stalled on a per­son’s com­puter col­lects data, and that Jumpshot repack­ages it into var­i­ous dif­fer­ent prod­ucts that are then sold to many of the largest com­pa­nies in the world. Some past, pre­sent, and po­ten­tial clients in­clude Google, Yelp, Microsoft, McKinsey, Pepsi, Home Depot, Condé Nast, Intuit, and many oth­ers. Some clients paid mil­lions of dol­lars for prod­ucts that in­clude a so-called All Clicks Feed,” which can track user be­hav­ior, clicks, and move­ment across web­sites in highly pre­cise de­tail.

Avast claims to have more than 435 mil­lion ac­tive users per month, and Jumpshot says it has data from 100 mil­lion de­vices. Avast col­lects data from users that opt-in and then pro­vides that to Jumpshot, but mul­ti­ple Avast users told Motherboard they were not aware Avast sold brows­ing data, rais­ing ques­tions about how in­formed that con­sent is.

The data ob­tained by Motherboard and PCMag in­cludes Google searches, lookups of lo­ca­tions and GPS co­or­di­nates on Google Maps, peo­ple vis­it­ing com­pa­nies’ LinkedIn pages, par­tic­u­lar YouTube videos, and peo­ple vis­it­ing porn web­sites. It is pos­si­ble to de­ter­mine from the col­lected data what date and time the anonymized user vis­ited YouPorn and PornHub, and in some cases what search term they en­tered into the porn site and which spe­cific video they watched.

Do you know about any other com­pa­nies sell­ing data? We’d love to hear from you. Using a non-work phone or com­puter, you can con­tact Joseph Cox se­curely on Signal on +44 20 8133 5190, Wickr on joseph­cox, OTR chat on jf­cox@jab­ber.ccc.de, or email joseph.cox@vice.com.

Although the data does not in­clude per­sonal in­for­ma­tion such as users’ names, it still con­tains a wealth of spe­cific brows­ing data, and ex­perts say it could be pos­si­ble to deanonymize cer­tain users.

In a press re­lease from July, Jumpshot claims to be the only com­pany that un­locks walled gar­den data” and seeks to provide mar­keters with deeper vis­i­bil­ity into the en­tire on­line cus­tomer jour­ney.” Jumpshot has pre­vi­ously dis­cussed some of its clients pub­licly. But other com­pa­nies men­tioned in Jumpshot doc­u­ments in­clude Expedia, IBM, Intuit, which makes TurboTax, Loreal, and Home Depot. Employees are in­structed not to talk pub­licly about Jumpshot’s re­la­tion­ships with these com­pa­nies.

It’s very gran­u­lar, and it’s great data for these com­pa­nies, be­cause it’s down to the de­vice level with a time­stamp,” the source said, re­fer­ring to the speci­ficity and sen­si­tiv­ity of the data be­ing sold. Motherboard granted the source anonymity to speak more can­didly about Jumpshot’s processes.

Until re­cently, Avast was col­lect­ing the brows­ing data of its cus­tomers who had in­stalled the com­pa­ny’s browser plu­gin, which is de­signed to warn users of sus­pi­cious web­sites. Security re­searcher and AdBlock Plus cre­ator Wladimir Palant pub­lished a blog post in October show­ing that Avast har­vest user data with that plu­gin. Shortly af­ter, browser mak­ers Mozilla, Opera, and Google re­moved Avast’s and sub­sidiary AVGs ex­ten­sions from their re­spec­tive browser ex­ten­sion stores. Avast had pre­vi­ously ex­plained this data col­lec­tion and shar­ing in a blog and fo­rum post in 2015. Avast has since stopped send­ing brows­ing data col­lected by these ex­ten­sions to Jumpshot, Avast said in a state­ment to Motherboard and PCMag.

However, the data col­lec­tion is on­go­ing, the source and doc­u­ments in­di­cate. Instead of har­vest­ing in­for­ma­tion through soft­ware at­tached to the browser, Avast is do­ing it through the anti-virus soft­ware it­self. Last week, months af­ter it was spot­ted us­ing its browser ex­ten­sions to send data to Jumpshot, Avast be­gan ask­ing its ex­ist­ing free an­tivirus con­sumers to opt-in to data col­lec­tion, ac­cord­ing to an in­ter­nal doc­u­ment.

If they opt-in, that de­vice be­comes part of the Jumpshot Panel and all browser-based in­ter­net ac­tiv­ity will be re­ported to Jumpshot,” an in­ter­nal prod­uct hand­book reads. What URLs did these de­vices visit, in what or­der and when?” it adds, sum­maris­ing what ques­tions the prod­uct may be able to an­swer.

Senator Ron Wyden, who in December asked Avast why it was sell­ing users’ brows­ing data, said in a state­ment, It is en­cour­ag­ing that Avast has ended some of its most trou­bling prac­tices af­ter en­gag­ing con­struc­tively with my of­fice. However I’m con­cerned that Avast has not yet com­mit­ted to delet­ing user data that was col­lected and shared with­out the opt-in con­sent of its users, or to end the sale of sen­si­tive in­ter­net brows­ing data. The only re­spon­si­ble course of ac­tion is to be fully trans­par­ent with cus­tomers go­ing for­ward, and to purge data that was col­lected un­der sus­pect con­di­tions in the past.”

Despite Avast cur­rently ask­ing users to opt back into the data col­lec­tion via a pop-up in the an­tivirus soft­ware, mul­ti­ple Avast users said they did not know that Avast was sell­ing brows­ing data.

I was not aware of this,” Keith, a user of the free Avast an­tivirus prod­uct who only pro­vided their first name, told Motherboard. That sounds scary. I usu­ally say no to data track­ing,” they said, adding that they haven’t yet seen the new opt-in pop-up from Avast.

Did not know that they did that :(,” an­other free Avast an­tivirus user said in a Twitter di­rect mes­sage.

Motherboard and PCMag con­tacted over two dozen com­pa­nies men­tioned in in­ter­nal doc­u­ments. Only a hand­ful re­sponded to ques­tions ask­ing what they do with data based on the brows­ing his­tory of Avast users.

We some­times use in­for­ma­tion from third-party providers to help im­prove our busi­ness, prod­ucts and ser­vices. We re­quire these providers to have the ap­pro­pri­ate rights to share this in­for­ma­tion with us. In this case, we re­ceive anonymized au­di­ence data, which can­not be used to iden­tify in­di­vid­ual cus­tomers,” a Home Depot spokesper­son wrote in an emailed state­ment.

Microsoft de­clined to com­ment on the specifics of why it pur­chased prod­ucts from Jumpshot, but said that it does­n’t have a cur­rent re­la­tion­ship with the com­pany. A Yelp spokesper­son wrote in an email, In 2018, as part of a re­quest for in­for­ma­tion by an­titrust au­thor­i­ties, Yelp’s pol­icy team was asked to es­ti­mate the im­pact of Google’s an­ti­com­pet­i­tive be­hav­ior on the lo­cal search mar­ket­place. Jumpshot was en­gaged on a one-time ba­sis to gen­er­ate a re­port of anonymized, high-level trend data which val­i­dated other es­ti­mates of Google’s si­phon­ing of traf­fic from the web. No PII was re­quested or ac­cessed.”

Every search. Every click. Every buy. On every site.”

Southwest Airlines said it had dis­cus­sions with Jumpshot but did­n’t reach an agree­ment with the com­pany. IBM said it did not have a record of be­ing a client, and Altria said it is not work­ing with Jumpshot, al­though did­n’t spec­ify if it did so pre­vi­ously. Sephora said it has not worked with Jumpshot. Google did not re­spond to a re­quest for com­ment.

On its web­site and in press re­leases, Jumpshot names Pepsi, and con­sult­ing gi­ants Bain & Company and McKinsey as clients.

As well as Expedia, Intuit, and Loreal, other com­pa­nies which are not al­ready men­tioned in pub­lic Jumpshot an­nounce­ments in­clude cof­fee com­pany Keurig, YouTube pro­mo­tion ser­vice vidIQ, and con­sumer in­sights firm Hitwise. None of those com­pa­nies re­sponded to a re­quest for com­ment.

On its web­site, Jumpshot lists some pre­vi­ous case stud­ies for us­ing its brows­ing data. Magazine and dig­i­tal me­dia gi­ant Condé Nast, for ex­am­ple, used Jumpshot’s prod­ucts to see whether the me­dia com­pa­ny’s ad­ver­tise­ments re­sulted in more pur­chases on Amazon and else­where. Condé Nast did not re­spond to a re­quest for com­ment.

Jumpshot sells a va­ri­ety of dif­fer­ent prod­ucts based on data col­lected by Avast’s an­tivirus soft­ware in­stalled on users’ com­put­ers. Clients in the in­sti­tu­tional fi­nance sec­tor of­ten buy a feed of the top 10,000 do­mains that Avast users are vis­it­ing to try and spot trends, the prod­uct hand­book reads.

Another Jumpshot prod­uct is the com­pa­ny’s so-called All Click Feed.” It al­lows a client to buy in­for­ma­tion on all of the clicks Jumpshot has seen on a par­tic­u­lar do­main, like Amazon.com, Walmart.com, Target.com, BestBuy.com, or Ebay.com.

In a tweet sent last month in­tended to en­tice new clients, Jumpshot noted that it col­lects Every search. Every click. Every buy. On every site” [emphasis Jumpshot’s.]

Jumpshot’s data could show how some­one with Avast an­tivirus in­stalled on their com­puter searched for a prod­uct on Google, clicked on a link that went to Amazon, and then maybe added an item to their cart on a dif­fer­ent web­site, be­fore fi­nally buy­ing a prod­uct, the source who pro­vided the doc­u­ments ex­plained.

One com­pany that pur­chased the All Clicks Feed is New York-based mar­ket­ing firm Omnicom Media Group, ac­cord­ing to a copy of its con­tract with Jumpshot. Omnicom paid Jumpshot $2,075,000 for ac­cess to data in 2019, the con­tract shows. It also in­cluded an­other prod­uct called Insight Feed” for 20 dif­fer­ent do­mains. The fee for data in 2020 and then 2021 is listed as $2,225,000 and $2,275,000 re­spec­tively, the doc­u­ment adds.

Jumpshot gave Omnicom ac­cess to all click feeds from 14 dif­fer­ent coun­tries around the world, in­clud­ing the U. S., England, Canada, Australia, and New Zealand. The prod­uct also in­cludes the in­ferred gen­der of users based on brows­ing be­hav­ior,” their in­ferred age, and the en­tire URL string” but with per­son­ally iden­ti­fi­able in­for­ma­tion (PII) re­moved, the con­tract adds.

Omnicom did not re­spond to mul­ti­ple re­quests for com­ment.

According to the Omnicom con­tract, the device ID of each user is hashed, mean­ing the com­pany buy­ing the data should not be able to iden­tify who ex­actly is be­hind each piece of brows­ing ac­tiv­ity. Instead, Jumpshot’s prod­ucts are sup­posed to give in­sights to com­pa­nies who may want to see what prod­ucts are par­tic­u­larly pop­u­lar, or how ef­fec­tive an ad cam­paign is work­ing.

What we don’t do is re­port on the Jumpshot Device ID that ex­e­cuted the clicks to pro­tect against the tri­an­gu­la­tion of PII,” one in­ter­nal Jumpshot doc­u­ment reads.

But Jumpshot’s data may not be to­tally anony­mous. The in­ter­nal prod­uct hand­book says that de­vice IDs do not change for each user, unless a user com­pletely unin­stalls and re­in­stalls the se­cu­rity soft­ware.” Numerous ar­ti­cles and aca­d­e­mic stud­ies have shown how it is pos­si­ble to un­mask peo­ple us­ing so-called anonymized data. In 2006, New York Times re­porters were able to iden­tify a spe­cific per­son from a cache of sup­pos­edly anony­mous search data that AOL pub­licly re­leased. Although the tested data was more fo­cused on so­cial me­dia links, which Jumpshot redacts some­what, a 2017 study from Stanford University found it was pos­si­ble to iden­tify peo­ple from anony­mous web brows­ing data.

De-identification has shown to be a very fail­ure-prone process. There are so many ways it can go wrong,” Günes Acar, who stud­ies large-scale in­ter­net track­ing at the Computer Security and Industrial Cryptography re­search group at the Department of Electrical Engineering of the Katholieke Universiteit Leuven, said.

De-anonymization be­comes a greater con­cern when con­sid­er­ing how the even­tual end-users of Jumpshot’s data could com­bine it with their own data.

Most of the threats posed by de-anonymiza­tion—where you are iden­ti­fy­ing peo­ple—comes from the abil­ity to merge the in­for­ma­tion with other data,” Acar said. A set of Jumpshot data ob­tained by Motherboard and PCMag shows how each vis­ited URL comes with a pre­cise time­stamp down to the mil­lisec­ond, which could al­low a com­pany with its own bank of cus­tomer data to see one user vis­it­ing their own site, and then fol­low them across other sites in the Jumpshot data.

It’s al­most im­pos­si­ble to de-iden­tify data,” Eric Goldman, a pro­fes­sor at the Santa Clara University School of Law, said. When they promise to de-iden­tify the data, I don’t be­lieve it.”

Motherboard and PCMag asked Avast a se­ries of de­tailed ques­tions about how it pro­tects user anonymity as well as de­tails on some of the com­pa­ny’s con­tracts. Avast did not an­swer most of the ques­tions but wrote in a state­ment, Because of our ap­proach, we en­sure that Jumpshot does not ac­quire per­sonal iden­ti­fi­ca­tion in­for­ma­tion, in­clud­ing name, email ad­dress or con­tact de­tails, from peo­ple us­ing our pop­u­lar free an­tivirus soft­ware.”

Users have al­ways had the abil­ity to opt out of shar­ing data with Jumpshot. As of July 2019, we had al­ready be­gun im­ple­ment­ing an ex­plicit opt-in choice for all new down­loads of our AV, and we are now also prompt­ing our ex­ist­ing free users to make an ex­plicit choice, a process which will be com­pleted in February 2020,” it said, adding that the com­pany com­plies with the California Consumer Privacy Act (CCPA) and Europe’s General Data Protection Regulation (GDPR) across its en­tire global user base.

We have a long track record of pro­tect­ing users’ de­vices and data against mal­ware, and we un­der­stand and take se­ri­ously the re­spon­si­bil­ity to bal­ance user pri­vacy with the nec­es­sary use of data,” the state­ment added.

It’s al­most im­pos­si­ble to de-iden­tify data.”

When PCMag in­stalled Avast’s an­tivirus prod­uct for the first time this month, the soft­ware did ask if they wanted to opt-in to data col­lec­tion.

If you al­low it, we’ll pro­vide our sub­sidiary Jumpshot Inc. with a stripped and de-iden­ti­fied data set de­rived from your brows­ing his­tory for the pur­pose of en­abling Jumpshot to an­a­lyze mar­kets and busi­ness trends and gather other valu­able in­sights,” the opt-in mes­sage read. The pop-up did not go into de­tail on how Jumpshot then uses this brows­ing data, how­ever.

The data is fully de-iden­ti­fied and ag­gre­gated and can­not be used to per­son­ally iden­tify or tar­get you. Jumpshot may share ag­gre­gated in­sights with its cus­tomers,” the pop-up added.

Just a few days ago, the Twitter ac­count for Avast sub­sidiary AVG tweeted, Do you re­mem­ber the last time you cleaned your #browser his­tory? Storing your brows­ing his­tory for a long time can take up mem­ory on your de­vice and can put your pri­vate info at risk.”

Update: This piece has been up­dated to in­clude a re­sponse from Sephora.


Read the original on www.vice.com »

5 686 shares, 23 trendiness, 93 words and 1 minutes reading time

Here are some terms to mute on Twitter to clean your timeline up a bit.

What would you like to do?

Embed this gist in your web­site.

Clone with Git or check­out with SVN us­ing the repos­i­to­ry’s web ad­dress.

Here are some terms to mute on Twitter to clean your time­line up a bit.

Sign up for free

to join this con­ver­sa­tion on GitHub.

Already have an ac­count?

Sign in to com­ment

You can’t per­form that ac­tion at this time.

You signed in with an­other tab or win­dow. Reload to re­fresh your ses­sion.

You signed out in an­other tab or win­dow. Reload to re­fresh your ses­sion.


Read the original on gist.github.com »

6 636 shares, 23 trendiness, 917 words and 8 minutes reading time

Monty Python stars pay tribute to Terry Jones

Monty Python stars have led the trib­utes to their co-star Terry Jones, who has died at the age of 77.

The Welsh ac­tor and writer played a va­ri­ety of char­ac­ters in the iconic com­edy group’s Flying Circus TV se­ries, and di­rected sev­eral of their films.

He died on Tuesday, four years af­ter con­tract­ing a rare form of de­men­tia known as FTD.

David Walliams and Simon Pegg were among other co­me­di­ans who re­mem­bered him.

Fellow Python star Sir Michael Palin de­scribed Jones as one of the fun­ni­est writer-per­form­ers of his gen­er­a­tion”.

In a tweet, John Cleese said he was a man of so many tal­ents and such end­less en­thu­si­asm”.

Eric Idle, an­other mem­ber of the highly in­flu­en­tial com­edy troupe, re­called the many laughs [and] mo­ments of to­tal hi­lar­ity” they shared.

It’s too sad if you knew him, but if you did­n’t you will al­ways smile at the many won­der­fully funny mo­ments he gave us,” he went on.

Terry Gilliam, with whom Jones di­rected the group’s film The Holy Grail in 1975, de­scribed his fel­low Python as a brilliant, con­stantly ques­tion­ing, icon­o­clas­tic, right­eously ar­gu­men­ta­tive and an­gry but out­ra­geously funny and gen­er­ous and kind hu­man be­ing”.

One could never hope for a bet­ter friend,” he said.

Palin added: Terry was one of my clos­est, most val­ued friends. He was kind, gen­er­ous, sup­port­ive and pas­sion­ate about liv­ing life to the full.

He was far more than one of the fun­ni­est writer-per­form­ers of his gen­er­a­tion, he was the com­plete Renaissance co­me­dian - writer, di­rec­tor, pre­sen­ter, his­to­rian, bril­liant chil­dren’s au­thor, and the warmest, most won­der­ful com­pany you could wish to have.”

Screenwriter Charlie Brooker posted: RIP the ac­tual ge­nius Terry Jones. Far too many bril­liant mo­ments to choose from.”

David Walliams thanked his com­edy hero for a life­time of laugh­ter”.

Simon Pegg - who acted in Jones’ fi­nal film as di­rec­tor, 2015′s Absolutely Anything - said: Terry was a sweet, gen­tle, funny man who was a joy to work with and im­pos­si­ble not to love.”

And co­me­dian Eddie Izzard told BBC News: It’s a tragedy - the good go too early. Monty Python changed the face of world com­edy. It will live for­ever. It’s a ter­ri­ble loss.”

Shane Allen, BBC con­troller of com­edy com­mis­sion­ing, wrote that it was a sad day to lose an ab­solute ti­tan of British com­edy” and one of the found­ing fa­thers of the most in­flu­en­tial and pi­o­neer­ing com­edy en­sem­bles of all time”.

Jones was born in Colwyn Bay and went on to study at Oxford University, where he met his fu­ture Python pal Palin in the Oxford Revue - a stu­dent com­edy group.

Alongside Palin, Idle and the likes of David Jason, he ap­peared in the BBC chil­dren’s satir­i­cal sketch show Do Not Adjust Your Set, which would set the tem­plate for their work to come with Python.

He wrote and starred in Monty Python’s Flying Circus TV show and the com­edy col­lec­tive’s films, as a range of much-loved char­ac­ters. These in­cluded Arthur Two Sheds” Jackson, Cardinal Biggles of the Spanish Inquisition and Mr Creosote.

In ad­di­tion to di­rect­ing The Holy Grail with Gilliam, Jones took sole di­rec­to­r­ial charge of 1979′s Life of Brian and The Meaning of Life in 1983.

Cleese said: Of his many achieve­ments, for me the great­est gift he gave us all was his di­rec­tion of Life of Brian. Perfection.”

Beyond Monty Python, he wrote the screen­play for the 1986 film Labyrinth, star­ring David Bowie.

Monty Python’s Flying Circus, the ground­break­ing com­edy se­ries that made Jones and his fel­low cast mem­bers in­ter­na­tional stars, first aired on BBC One in October 1969.

Surreal, an­ar­chic and bawdily ir­rev­er­ent, the show’s blend of live-ac­tion sketches and an­i­mated in­ter­ludes mocked both broad­cast­ing con­ven­tions and so­ci­etal norms.

Jones and Palin had met at Oxford, while Cleese, Graham Chapman and Eric Idle stud­ied at Cambridge. After uni­ver­sity, they took part in var­i­ous com­edy shows be­fore form­ing Monty Python with US-born an­i­ma­tor Terry Gilliam.

After four se­ries, the troupe moved to the big screen to make Arthurian spoof Monty Python and the Holy Grail and Monty Python’s Life of Brian, a con­tro­ver­sial par­ody of Biblical epics.

Monty Python’s The Meaning of Life, their fi­nal film as a col­lec­tive, re­turned to the orig­i­nal se­ries’ sketch-based for­mat.

The sur­viv­ing mem­bers re­united pe­ri­od­i­cally af­ter Chapman’s death in 1989, most no­tably for a run of live shows at the O2 in London in 2014.

* Now, you lis­ten here! He’s not the Messiah. He’s a very naughty boy!” - as Brian’s mother in Monty Python’s Life of Brian

* I’m alive, I’m alive!” - as the naked her­mit who gives away the lo­ca­tion of a hid­ing Brian in Life of Brian

* I shall use my largest scales” - as Sir Belvedere, who over­sees a witch trial in Monty Python and the Holy Grail

* What, the cur­tains?” - as Prince Herbert, who is told One day, lad, all this will be yours” in Holy Grail

* Spam, spam, spam, spam, spam, spam, spam” - as the greasy spoon wait­ress in a Monty Python sketch

The state­ment from Jones’ fam­ily noted his uncompromising in­di­vid­u­al­ity, re­lent­less in­tel­lect and ex­tra­or­di­nary hu­mour [that] has given plea­sure to count­less mil­lions across six decades”.

Over the past few days his wife, chil­dren, ex­tended fam­ily and many close friends have been con­stantly with Terry as he gen­tly slipped away at his home in north London.

His work with Monty Python, his books, films, tele­vi­sion pro­grammes, po­ems and other work will live on for­ever, a fit­ting legacy to a true poly­math.”

The fam­ily thanked Jones’ wonderful med­ical pro­fes­sion­als and car­ers for mak­ing the past few years not only bear­able but of­ten joy­ful”.

They said: We hope that this dis­ease will one day be erad­i­cated en­tirely. We ask that our pri­vacy be re­spected at this sen­si­tive time and give thanks that we lived in the pres­ence of an ex­tra­or­di­nar­ily tal­ented, play­ful and happy man liv­ing a truly au­then­tic life, in his words Lovingly frosted with glu­cose.’”

* FTD is an un­com­mon type of de­men­tia that mainly af­fects the front and sides of the brain

* The ini­tial symp­toms are, in most cases, changes in be­hav­iour, men­tal abil­ity and lan­guage, which will get worse over time

* Language prob­lems can in­clude loss of vo­cab­u­lary, rep­e­ti­tion and for­get­ting the mean­ing of com­mon words

* FTD is caused by ab­nor­mal pro­tein clumps form­ing in­side the brain cells, which are thought to dam­age the cells and stop them work­ing. Why this hap­pens is not fully un­der­stood

* Although there are cur­rently no cures or treat­ments to slow FTD down, there are treat­ments that can help con­trol some symp­toms

Follow us on Facebook, or on Twitter @BBCNewsEnts. If you have a story sug­ges­tion email en­ter­tain­ment.news@bbc.co.uk.


Read the original on www.bbc.com »

7 632 shares, 24 trendiness, 534 words and 6 minutes reading time

Access to Wikipedia restored in Turkey after more than two and a half years

Today, on Wikipedia’s 19th birth­day, the Wikimedia Foundation has re­ceived re­ports that ac­cess to Wikipedia in Turkey is ac­tively be­ing re­stored.* This lat­est de­vel­op­ment fol­lows a 26 December 2019 rul­ing by the Constitutional Court of Turkey that the more than two and a half year block im­posed by the Turkish gov­ern­ment was un­con­sti­tu­tional. Earlier to­day, the Turkish Constitutional Court made the full text of that rul­ing avail­able to the pub­lic, and shortly af­ter, we re­ceived re­ports that ac­cess was re­stored to Wikipedia.

We are thrilled that the peo­ple of Turkey will once again be able to par­tic­i­pate in the largest global con­ver­sa­tion about the cul­ture and his­tory of Turkey on­line and con­tinue to make Wikipedia a vi­brant source of in­for­ma­tion about Turkey and the world.

We are thrilled to be re­united with the peo­ple of Turkey,” said Katherine Maher, Executive Director of the Wikimedia Foundation. At Wikimedia we are com­mit­ted to pro­tect­ing every­one’s fun­da­men­tal right to ac­cess in­for­ma­tion. We are ex­cited to share this im­por­tant mo­ment with our Turkish con­trib­u­tor com­mu­nity on be­half of knowl­edge-seek­ers every­where.”

We are ac­tively re­view­ing the full text of the rul­ing by the Constitutional Court of Turkey. In the mean­time, our case be­fore the European Court of Human Rights is still be­ing con­sid­ered by the Court. We filed a pe­ti­tion in the European Court of Human Rights in spring of last year, and in July, the Court granted our case pri­or­ity sta­tus. We will con­tinue to ad­vo­cate for strong pro­tec­tions for free ex­pres­sion on­line in Turkey and around the world.

Wikipedia is a global free knowl­edge re­source writ­ten and edited by peo­ple around the world. Because of this open edit­ing model, Wikipedia is also a re­source every­one can be a part of ac­tively shap­ing  — adding knowl­edge about their cul­ture, coun­try, in­ter­ests, stud­ies, and more through Wikipedia’s ar­ti­cles. Volunteers work to­gether to write ar­ti­cles about many dif­fer­ent top­ics rang­ing from his­tory, pop cul­ture, sci­ence, sports, and more us­ing re­li­able sources to ver­ify the facts. It is through this col­lec­tive process of writ­ing, dis­cus­sion, and de­bate that Wikipedia be­comes more neu­tral, more com­pre­hen­sive, and more rep­re­sen­ta­tive of the world’s knowl­edge.

More than 85 per­cent of the ar­ti­cles on Wikipedia are in lan­guages other than English, which in­cludes the Turkish Wikipedia’s more than 335,000 ar­ti­cles, writ­ten by Turkish-speaking vol­un­teers for Turkish-speaking peo­ple.

In the time that the block was in ef­fect, we heard from stu­dents, teach­ers, pro­fes­sion­als and more in Turkey about how the block had im­pacted their daily lives. For many stu­dents, the block had oc­curred just days be­fore their fi­nal ex­ams. On so­cial me­dia, mem­bers of the in­ter­na­tional vol­un­teer Wikipedia ed­i­tor com­mu­nity and count­less in­di­vid­u­als shared mes­sages of sup­port with #WeMissTurkey and their de­sire to once again col­lab­o­rate with the peo­ple of Turkey on Wikipedia.

With the de­ci­sion to­day, our ed­i­tors in Turkey will once again be able to fully par­tic­i­pate in shar­ing and con­tribut­ing to free knowl­edge on­line.

* We have re­ceived re­ports that sev­eral in­ter­net ser­vice providers in Turkey, de­pend­ing on the lo­ca­tion, have re­stored ac­cess to Wikipedia in Turkey, with some still in the process of restor­ing ac­cess. We will keep this state­ment up­dated as fur­ther ac­cess is re­stored.


Read the original on wikimediafoundation.org »

8 609 shares, 23 trendiness, 0 words and 0 minutes reading time

Draw all roads in any city at once


Read the original on anvaka.github.io »

9 606 shares, 24 trendiness, 0 words and 0 minutes reading time

Float Toy

Click on a cell be­low to tog­gle bit val­ues. Use this to build in­tu­ition for the IEEE float­ing-point for­mat. See Wikipedia for de­tails on the half-pre­ci­sion, sin­gle-pre­ci­sion and dou­ble-pre­ci­sion float­ing-point for­mats.


Read the original on evanw.github.io »

10 593 shares, 22 trendiness, 3657 words and 29 minutes reading time

Building personal search infrastructure for your knowledge and code

7. Searching in code

What do I want

11. Future and my holy grail of search

Having in­for­ma­tion in the dig­i­tal form, col­lect­ing and writ­ing notes is in­cred­i­bly valu­able.

Our brains are good at as­so­ci­a­tions, pat­tern match­ing and cre­ative think­ing, not stor­ing ar­rays of struc­tured data, and ex­ter­nal mem­ory is one of the main think­ing hacks com­put­ers aid us with.

However this in­for­ma­tion is not so use­ful if you can’t ac­cess and search it quickly. Instant search changes the way you think. Ever got sense of flow while work­ing through some prob­lem, and try­ing dif­fer­ent things from Stackoverflow or doc­u­men­ta­tion?

These days, if you have de­cent con­nec­tion, you are sec­onds away from find­ing al­most any pub­lic knowl­edge in the in­ter­net. However, there is an­other as­pect of in­for­ma­tion: per­sonal and spe­cific to your needs, work and hob­bies. It’s your todo list, your pri­vate notes, books you are read­ing. Of course, it’s not that well in­te­grated with the out­side world, hence the tool­ing and ex­pe­ri­ence of in­ter­act­ing with it is very dif­fer­ent.

To find some­thing from my Messenger his­tory with a friend, I need to be on­line, open Facebook, nav­i­gate to search and use the in­ter­face Facebook’s em­ploy­ees thought con­ve­nient (spoiler: it sucks)

It’s my in­for­ma­tion, some­thing that came out from my brain. Why can’t I have it avail­able any­where, any­time, pre­sented the way I pre­fer?

To find some­thing in my Kobo ebook, I need to reach my de­vice phys­i­cally and type the query us­ing the vir­tual key­board (yep, e-ink lag!). Not a very pleas­ant ex­pe­ri­ence.

It’s some­thing I own and have read. Why does it have to be so hard?

Such things are pretty frus­trat­ing to me, so I’ve been work­ing on mak­ing them eas­ier. Search has to be in­cre­men­tal, fast and as con­ve­nient to use as pos­si­ble. I’ll be shar­ing some of work­flows, tricks and thoughts in this post.

The post is geared to­wards us­ing Emacs and Org-mode, but hope­fully you’ll find some use­ful tricks for your cur­rent tools and work­flow even if you don’t. There is (almost) noth­ing in­her­ently spe­cial about Emacs, I’m sure you can achieve sim­i­lar work­flows in other mod­ern text ed­i­tors given they are flex­i­ble enough.

Note: through­out the post I will link to my emacs con­fig snip­pets. To pre­vent code ref­er­ences from stal­ing, I use perma­links, but check mas­ter branch as well in case of patches or more com­ments in code.

* my per­sonal notes, tasks and knowl­edge repos­i­tory (this blog in­cluded)

By per­sonal in­for­ma­tion I re­fer to things like todo list, per­sonal wiki or what­ever you use to store in­for­ma­tion rel­e­vant to your life.

For the most part, I keep things in Org mode, and I use Emacs to work with it. Apart from reg­u­lar means of plain­text search (I’ll write about it later), for me it’s im­por­tant to search over tags:

org-tags-view is avail­able by de­fault and an easy way to run sim­ple tag searches

It’s a rel­a­tively new pack­age, with new query syn­tax as the main fea­ture, which is much eas­ier to use and re­mem­ber than builtin Org query syn­tax: com­par­i­son.

I mainly use these com­mands:

helm-org-ql for in­cre­men­tal search in the cur­rent buffer

org-ql-search in­ter­ac­tively prompts you for the search tar­get, sort and group­ing

Another no­table men­tion is org-ri­fle, which is an en­try based search, pre­sent­ing head­ings along with the matched con­tent in Helm buffer. However as the au­thor men­tioned it might be ob­so­leted by org-ql soon.

Here are some typ­i­cal work­flows with my org-mode:

I see an in­ter­est­ing ar­ti­cle or think of some­thing which would be good to share with a friend, but at the mo­ment it’s not quite a good time to send it. I can just cap­ture it and at­tach a tag (e.g. or ). That way next time we chat I can just look up things un­der their tag and send them.

It works the other way around as well: imag­ine they sent me a link or asked me to do some­thing, but I can’t do it im­me­di­ately. I have a spe­cial script that con­verts chat mes­sages into todo items and au­to­mat­i­cally at­taches the cor­re­spond­ing tag. I write more about it here.

Unfortunately, I can’t just sit and write com­pre­hen­si­ble texts with­out prepa­ra­tion. Typically I have thoughts on the topic now and then, which I just note down and mark with the tag. When I feel it’s time to pre­pare the post, I can just search by the tag (e.g. tags:search for this post), and re­file the items into the file with the post draft.

As I men­tioned, I find hav­ing to switch to the browser, wait till the web­site loads and cope with crappy search im­ple­men­ta­tions very dis­tract­ing and frus­trat­ing.

What is more, of­ten you don’t even re­mem­ber whether ex­actly you were dis­cussing some­thing: on Telegram or Facebook or Reddit? So hav­ing a sin­gle point of en­try to your in­for­ma­tion and uni­fied search over all of your stuff is ex­tremely help­ful.

For in­stant mes­sag­ing, I’m us­ing plain­text mir­rors, so chat his­tory is al­ways avail­able in plain­text on my com­put­ers:

Telegram mes­sages

Didn’t bother with org-mode be­cause files would be too huge and there is­n’t much struc­ture any­way.

Vkontakte mes­sages

Sadly ex­port tool stopped work­ing be­cause of API re­stric­tions, but I’m not us­ing VK much any­more ei­ther. At least I got his­toric mes­sages.

Most ser­vices where I can com­ment, write or leave an­no­ta­tion, I’m mir­ror­ing as org-mode. I write about it in de­tail here: part I, part II.

That gives me source data for a search en­gine over any­thing I’ve ever:

All these files are ei­ther non-Org or some­what heavy for struc­tured Org-mode search. In ad­di­tion, I have many old files from my pre-orgmode era when I was us­ing Gitit or Zim.

To search over them, I’m us­ing Emacs and Ripgrep (you can read why later):


runs rip­grep against my/​search-tar­gets vari­able con­tains paths to notes, chat logs, Orger out­puts etc.

The in­ter­est­ing bit about my/​search is –my/one-off-helm-follow-mode call. It’s a some­what hor­ri­ble hack that au­to­mat­i­cally en­ables helm-fol­low mode so you don’t have to press C-c C-f every time you in­voke helm.

Finally, to make sure I can in­voke search in an in­stance, I’m us­ing a global key­bind­ing.

Here’s a demo gif (5Mb) of us­ing this to search greg egan’ in my knowl­edge repos­i­tory. You can see that as a re­sult, I’m get­ting my Kobo high­lights (), my read­ing list () and even some video ()!

Recoll is an in­dexer that runs as dae­mon (or a reg­u­lar cron job) and a full text search tool.

It sup­ports many for­mats and other fea­tures, so I sug­gest check­ing them out for your­self.

Even though I in­dex all my doc­u­ments, I find it quicker to run grep I de­scribed above to search in plain­text. So for me, Recoll is mostly for search­ing and quickly jump­ing to re­sults in PDFs and EPUBs (see screen­shot).

There is helm-rec­oll Emacs mod­ule, but I found it a bit awk­ward to use, and Recoll GUI feels sig­nif­i­cantly su­pe­rior. Basically only thing helm-rec­oll does is pre­sent­ing you list of file­names that match your query. It feels that it should be straight­for­ward to mod­ify the mod­ule and in­te­grate ab­stract, snip­pets and other things you can query Recoll for.

Considering I don’t need use Recoll it too of­ten, I just gave up on helm-rec­oll and us­ing GUI.

I’m also run­ning a Web UI on my VPS, so I can use it from my phone, or po­ten­tially from other com­put­ers.

Recoll’s dis­tin­guish­ing fea­tures are proper search query lan­guage

and re­al­time, in­o­tify based in­dex­ing. I don’t have that much data yet to ben­e­fit mas­sively from proper search queries, but I can see that it could be po­ten­tially use­ful in fu­ture as amount of per­sonal data grows.

Most of my notes and knowl­edge repos­i­tory are plain­text, so it is eas­ily and con­tin­u­ously shared on my phone via Dropbox/Syncthing.

Since us­ing Emacs on Android is hardly a mean­ing­ful ex­pe­ri­ence, I’m work­ing around it by us­ing other apps.

I can’t rec­om­mend it enough, it’s got many things done right, very fast and the code is ex­tremely read­able and well tested so it’s easy to con­tribute.

It has its own small query lan­guage (at the time org-ql did­n’t ex­ist).

You can save search queries, which ends up be­ing pretty sim­i­lar to cus­tom Org-mode agen­das. Searches can be dis­played as per­sis­tent wid­gets, e.g. I find con­ve­nient to have a phone screen ded­i­cated to Buy’ search (t.buy query) or Do at work’ search (t.@work query).

As I de­scribed above, I keep few saved search queries for some friends so I can re­call what I wanted to dis­cuss with them.

Docsearch is a not very well-known tool (e.g. zero search re­sults on Reddit or Twitter), but I don’t know any al­ter­na­tives for it.

It’s a full­text in­dex­ing and search app for plain­text files, but ap­par­ently it even sup­ports EPUBs and PDFs.

Here’s how matches list looks. Screenshots on Google Play give a pretty good idea what the app does.

I find it con­ve­nient for quick search over things that are not im­ported in Orgzly, e.g. chat logs (Telegram, VK) and huge org-mode files I de­scribed above.

It’s a bit back­wards in terms of UI (even though I like that it’s com­pact and func­tional), but main down­side is it’s not open­source. I’d be ex­tremely happy to re­place this with some open source ap­pli­ca­tion, so please let me know if you know one!

On the rare oc­ca­sions when I need to search in pdfs or books (which I don’t sync on my phone) , I just use Recoll Web UI that I’m self­host­ing.

If you’re read­ing this at all, chances you’re quite good at us­ing web search al­ready. Gwern got a good writeup on the sub­ject.

Knowing how to com­pose a search query is one thing, but nav­i­gat­ing to the ser­vice, wait­ing till it loads, mov­ing to search­box takes pre­cious time. Many peo­ple for­get about cus­tom search en­gines. Here are ones I’m us­ing:

Some of these ob­vi­ous, some de­serve sep­a­rate men­tion:

red­dit con­tains vast amounts of (somewhat cu­rated) hu­man knowl­edge

Google search of­ten gives du­bi­ous and not very mean­ing­ful re­sults on cer­tain top­ics (e.g. prod­uct re­views, ex­er­cise, di­et­ing). On red­dit, you’d at least find real peo­ple shar­ing their hon­est and real opin­ions. Chances are that if a link is good, you would find it on on red­dit any­way.

twit­ter is sim­i­lar: there is cer­tainly more spam there, but some­times it’s in­ter­est­ing to type a link or blog post ti­tle in twit­ter search to see how real peo­ple re­acted.

That has lim­ited util­ity, e.g. does­n’t work with politi­cized con­tent, but if the topic of in­ter­est is rare, could be very use­ful.

* pin­board is an awe­some source of cu­rated con­tent as well

Next, I find it very con­ve­nient to have some code doc­u­men­ta­tion avail­able lo­cally. First, it helps when you’re on wonky in­ter­net or just of­fline for what­ever rea­son. Second, it’s feels re­ally fast, even if you’re on fiber.

Here’s what I’m us­ing for that:

Sadly, the ex­ten­sion men­tioned above does­n’t work with schema for some rea­son, so to add it in Firefox, you can use the method de­scribed here, it’s as easy as adding a book­mark.

Recently I ran into de­v­docs.io, it’s us­ing your browser’s of­fline stor­age to cache the doc­u­men­ta­tion. I’m still get­ting used to it, but it’s amaz­ing how faster it is than jump­ing to doc­u­men­ta­tion on­line. You can use it with mul­ti­ple lan­guages, you just type the search en­gine pre­fix first, and then lan­guage pre­fix (e.g. dd cpp em­place_back).

Finally, it may be con­ve­nient to set up en­gine-mode in Emacs, or search-en­gine layer in Spacemacs. It lets you in­voke a browser search di­rectly from Emacs (e.g. SPC s G to do google search). I find it con­ve­nient when I need to search many things in bulk.

I find it con­ve­nient to en­able highlight all’ for search within a page.

When I used Chrome, one thing that an­noyed me was that it pop­u­lates search en­gines au­to­mat­i­cally, and there is not way to dis­able it.

There is a nice open source ex­ten­sion that pre­vents Chrome from do­ing it.

TLDR: I tried dif­fer­ent ex­ist­ing code search tools, was dis­ap­pointed and ended up us­ing Emacs + Ripgrep.

I’ve got lots of per­sonal pro­jects, ex­per­i­ments, data pro­cess­ing and backup scripts on my com­puter. I also tend to cre­ate a git repos­i­tory at a slight­est op­por­tu­nity pri­mar­ily as a means of code backup/​roll­back and progress track­ing, but of­ten it re­sults in ac­tual pro­jects, so I would need a repos­i­tory any­way. Naturally, these repos­i­to­ries end up scat­tered across the whole filesys­tem, mak­ing it tricky to re­mem­ber where I’ve put the code or that it even ex­isted in the first place.

It’s very con­ve­nient to have some sort of code search en­gine if you’re in a sim­i­lar sit­u­a­tion to mine for mul­ti­ple rea­sons:

For in­stance, I want to re­move some un­used func­tion or refac­tor some­thing in my pack­age, which is a Python li­brary to ac­cess my per­sonal data. It’s used in lots of scripts or dash­boards that run in Cron every day.

I could just go for it, re­move the func­tion and hope noth­ing fails, but if it does then I’d have to deal with fix­ing it again. It’s frus­trat­ing and I’d rather search for func­tion us­ages in all of my code and make sure it’s ac­tu­ally safe to re­move.

When you’re get­ting fa­mil­iar to some new li­brary or frame­work, you of­ten end up googling how to solve prob­lems twice. Sometimes you re­mem­ber solv­ing the prob­lem you’ve al­ready had, but don’t quite re­call where.

For in­stance for me, such li­brary is Sqlalchemy. It’s very con­ve­nient for han­dling data­bases, but I only need it in­fre­quently, so can never re­mem­ber how to work with it. Reading doc­u­men­ta­tion all over again is not very help­ful be­cause I’ve got very few use­cases and queries that are spe­cific to my pur­poses.

If I can search for sqlalchemy in my code, it shows every repos­i­tory where I used it so I can quickly copy bit of code I’m in­ter­ested at.

It hap­pens that I re­mem­ber writ­ing code for some pur­pose, but don’t quite re­call where I put it. Even if you keep all your re­pos in the same lo­ca­tion, you might for­get how you named it.

Full text search, how­ever, al­lows to find it if you re­mem­ber some com­ments or class/​func­tion names.

However good is li­brary’s doc­u­men­ta­tion, some­times it just is­n’t cov­er­ing your typ­i­cal needs. If you’re a power user, docs are al­most never enough and you end up read­ing the code to bend the li­brary into do­ing what you want.

For me such li­braries are Org mode or Hakyll, so I of­ten had to search in their code on Github. Searching on Github how­ever is quite awk­ward. It’s slow, it’s not in­cre­men­tal and lacks nav­i­ga­tion.

If I have a lo­cal clone of the repos­i­tory on my disk, I can search over it in an in­stant (without hav­ing it opened in the first place) and use fa­mil­iar tools (e.g. IDE) for nav­i­ga­tion.

At the time, I was just us­ing re­cur­sive grep and then open­ing some of re­sults in vim to re­fine. That’s a pretty pa­thetic work­flow.

run against code on my filesys­tem

Just any source files, so it would­n’t have to fetch repos­i­to­ries from Github and keep them some­where sep­a­rately.

Ideally, in­o­tify-based, but any means of re­fresh­ing search in­dex with­out hav­ing to com­mit first would be nice.

se­man­tic search in de­f­i­n­i­tions/​vari­ables etc with fall­back to sim­ple search if the lan­guage is­n’t sup­ported

So, I wanted some code search and in­dex­ing tool that could watch over all the source file on my filesys­tem and let me search through them.

It sounds as a fairly straight­for­ward wish, but to my sur­prise, none of ex­ist­ing pro­jects I found and tried do the job:

Lets you in­dex Github/Bitbucket/Gitlab re­pos etc, but the process for adding lo­cal repos­i­to­ries is ex­tremely te­dious. Also ap­par­ently, it clones repos­i­to­ries first so it’s not ex­actly re­al­time in­dex­ing.


Read the original on beepb00p.xyz »

To add this web app to your iOS home screen tap the share button and select "Add to the Home Screen".

10HN is also available as an iOS App

If you visit 10HN only rarely, check out the the best articles from the past week.

If you like 10HN please leave feedback and share

Visit pancik.com for more.