10 interesting stories served every morning and every evening.
Sometimes when I’m bored, I like to look at the list of macOS Bash commands. Here’s some commands that I found interesting:
If you store your secrets in the Keychain (and you should!), you can access them programmatically using security.
security find-internet-password -s “https://example.com”
I found this useful for writing automated scripts that used locally-stored credentials.
Bonus tip: If you are using 1Password, there is a 1Password CLI that you can use to access your 1Password items from the command line.
If you want to open a file from the terminal, you can use the open command.
open file.txt
This will open the file in the default application for that file type, as if you had double-clicked it in the Finder.
pbcopy and pbpaste are command-line utilities that allow you to copy and paste text to the pasteboard (what other operating systems might call the “clipboard”).
pbcopy takes whatever was given in the standard input, and places it in the pasteboard.
echo “Hello, world!” | pbcopy;
pbpaste takes whatever is in the pasteboard and prints it to the standard output.
pbpaste>> Hello, world!
This is very useful for getting data from files into the browser, or other GUI applications.
If you work with servers a lot, it can be useful to know the current time in UTC, when e.g. looking at server logs.
This is a one-liner in the terminal:
date -u
Alternatively, you can use
TZ=UTC date
If you want to run an Internet speedtest, you can run one directly from the terminal with
networkQuality # Note the capital “Q”!
If you are want to keep your Mac from sleeping, you can run caffeinate in the terminal.
caffeinate
caffeinate will keep your Mac awake until you stop it, e.g. by pressing Ctrl+C. caffeinate used to be a third-party tool, but it is now built-in to macOS.
I use this mostly to prevent my Mac from sleeping when I am running a server.
If you need to generate a UUID, you can use the uuidgen command.
uuidgen
By default uuidgen outputs a UUID in uppercase. You can combine this with tr and pbcopy to copy the UUID to the clipboard in lowercase.
uuidgen | tr ‘[:upper:]’ ‘[:lower:]’ | pbcopy
I use this a lot when writing unit tests that require IDs.
* mdfind: Spotlight search, but in the terminal. I generally use Spotlight itself (or rather the excellent Raycast). Link
* say: This command makes your Mac speak the text you give it. Link
* screencapture: This command allows you to take screenshots and save them to a file. I prefer using cmd-shift-5 for this. Link
* networksetup: This command allows you to configure your network settings programmatically. I found its API very intimidating, and so I haven’t really used it much. Link
...
Read the original on weiyen.net »
A title drop is when a character in a movie says the title of the movie they’re in. Here’s a large-scale analysis of 73,921 movies from the last 80 years on how often, when and maybe even why that happens.
I’m sure you all know the part of the movie where one of the characters says the actual title of the movie
and you’re like
The overall meta-ness of this is - of course - nothing new. And filmmakers and scriptwriters have been doing it since the dawn of the medium itself*. It’s known in film speak as a title drop.
Consequently, there’s tons of examples throughout movie history that range from the iconic (see Back to the Future’s above)
via the eccentric,
the very much self-aware
But how common are these title drops really? Has this phenomenon gained momentum over time with our postmodern culture becoming ever more meta? Can we predict anything about the quality of a film based on how many times its title is mentioned? And what does a movie title mean, anyway?
There have been analyses
and oh so so many listicles
of the title drop phenomenon before, but they are small and anecdotal. Here’s the first extensive analysis of title drops for a dataset of 73,921 movies that amount to roughly 61% of movies on IMDb with at least 100 user votes*. I’m looking at movies released between 1940 and 2023. Special thanks go to my friends at OpenSubtitles.com for providing this data!
I started out with two datasets: 89,242 (English) movie subtitles from OpenSubtitles.com
and metadata for 121,797 movies from IMDb. After joining them and filtering them for broken subtitle files I was left with a total of 73,921 subtitled movies. With that out of the way, I realized that the tougher task was still ahead of me: answering the question what even was a title drop?
The naïve approach is - of course - to simply look for the movie’s name anywhere in the subtitles. Which is a fantastic approach for movies like Back to the Future with a nice unique title:
But this quickly breaks down if we look at movies like E or I *, which lead to way too many matches.
We also run into problems with every movie that is a sequel (Rocky III, Hot Tub Time Machine 2) since none of the characters will add the sequel number to character names/oversized bathing equipment. Similarly, the rise of the colon
in movie titles would make for some very awkward dialogue (LUKE: “Gosh Mr. Kenobi, it’s almost like we’re in the middle of some Star Wars Episode Four: A New Hope!“).
(See also the He Didn’t Say That
meme.)
So I applied a few rules to my title matching in the dialogue. Leading ‘The’, ‘An’ and ’A’s and special characters like dashes are ignored, sequel numbers both Arabic and Roman are dropped (along with ‘Episode…’, ‘Part…’ etc.) and titles containing a colon are split and either side counts as a title drop. So for The Lord of the Rings: The Fellowship of the Ring
either “Lord of the Rings” or “Fellowship of the Ring” would count as title drops (feel free to hover over the visualizations to explore the matches)!
With the data cleaning out of the way, let’s get down to business!
Alright, so here’s the number you’ve all been waiting for (drumroll):
36.5% - so about a third - of movies have at least one title drop during their runtime.
Also, there’s a total of 277,668 title drops for all 26,965 title-dropping movies which means that there’s an average of 10.3 title drops per movie that title drops. If they do it, they really go for it.
So who are the most excessive offenders in mentioning their titles over the course of the film? The overall star when it comes to fiction only came out last year: it’s Barbie by Greta Gerwig with an impressive 267 title drops within its 1 hour and 54 minutes runtime, clocking in at a whopping 2.34 BPM (Barbies Per Minute).
On the non-fiction side of documentaries the winner is Mickey: The Story of a Mouse
with 309 title drops in only 90 minutes, so 3.43 Mickeys Per Minute!
What’s interesting about the (Fiction) list here is that it’s pretty international: only two of the top ten movies come from Hollywood, 6 are from India, one from Indonesia and one from Turkey. So it’s definitely an international phenomenon.
Looking at the top ten list you might have noticed this little icon
signifying a movie where the data says it’s named after one of its characters*.
Unsurprisingly, movies named after one of their characters have an average of 24.7 title drops, more than twice as much as the usual 10.3. Protagonists have a tendency to pop up repeatedly in a film, so their names usually do the same.
Similarly, movies named after a protagonist have a title drop rate of 88.5%
while only 34.2% of other movies drop their titles.
A note on the data here
This is the more experimental part of the analysis. To figure out if a movie was named after its
protagonist I’ve used
IMDb’s Principals Dataset
that lists character names for the first couple of actors and compared that to the movie’s title.
This approach yields reliable results, but of course misses movies when the character the movie
is named after does not appear on that list. So you might find movies that miss the
‘Named’
icon even though they’re clearly named after a character.
Special characters in the title and character name are also challenging: for example, Tosun Pasa which actually has a ş character in its title - wrong on IMDb (Pasa) as well as the subtitles
(Pasha) - or WALL·E with the challenging · in the middle: Even
though there are mentions of “Wall-E” in the subtitles, the script - looking for “WALL·E” - wouldn’t
detect it. (I’ve fixed both of these films manually - but there might be more!)
Titles or surnames also usually prevent being counted as title drops according to our definitions.
Michael The Brave,
King Lear or Barry Lyndon might mention a character’s name (‘Michael’, ‘Lear’, ‘Barry’) but leave out the title or surname
- so zero drops.
Nevertheless, there do exist named films where you would expect a title drop which doesn’t come!
Examples are:
Anyway - back to the analysis!
An interesting category are movies named after a character that only have a single title drop - making it all the more meaningful?
Title-drop connoisseurs might sneer at this point and well-actually us that a “real” title drop should only happen once in a film. That there’s this one memorable (or cringe-y) scene where the protagonist looks directly at the camera and declares the title of the film with as much pathos as they can muster. Or as a nice send-off in the last spoken line.
Such single drops happen surprisingly often:
11.3% of all movies do EXACTLY ONE title drop during their runtime.
Which means that there’s about twice as many movies having multiple title drops than single ones.
In the single drop case it is more likely that the filmmakers were adding a title drop very consciously.
Single drops often happen in a key scene and explain the movie’s title: what mysterious fellowship the first Lord of the Rings is named after. Or that the audience waiting for some dark knight to show up must simply accept that it’s been the Batman all along.
One suspicion I had was that the very meta act of having a character speak the name of the movie they’re in would be something gaining more and more traction over the last two or three decades.
And indeed, if we look at the average number of movies with title drops over the decades we can see that there’s a certain upwards trend. The 1960s and 1970s seemed to be most averse to mentioning their title in the film, while it’s become more common-place over the last years.
If we dig deeper, this growth over the decades comes with a clearer explanation: splitting up movies by single- and multi-title drops shows that while the tendency of movies to drop their title exactly once keeps more or less steady, the number of multi-drop films is on the rise.
Your explanation for this (More movies are being named after their protagonists? Movies are more productified so brand recognition becomes an important concern?) is probably as good as mine 🤷
Another question I wanted to answer was if a high number of title drops was a sign of a bad movie. Think of all the trashy slasher and horror movies about Meth Marmots and Killer Ballerinas - wouldn’t their characters in the sparse dialogues constantly mention the title for brand recognition and all that?
Interestingly though, there’s no strong connection between film quality (expressed as IMDb rating (YMMV)) and the probability of title-dropping.
An aspect that certainly does have an impact on the probability of a title drop though is the genre of a film.
If you think back to the discussion about names in titles from earlier, genres like Biography and other non-fiction genres like Sport and History - almost by definition - mention their subject in both the title and throughout the film.
Accordingly, the probability of a title drop varies wildly by genre. Non-fiction films have a strong tendency towards title-dropping, while more fiction-oriented genres like Crime, Romance and War don’t.
Finally, we can ask the question: what even is a movie title?
I couldn’t find a complete classification in the scientific literature (“What’s in a name? The art of movie titling”
by Ingrid Haidegger comes the closest). Movie titles are an interesting case, since they have to work as a description of a product, a marketing instrument, but also as the title of a piece of art.
Consequently, it’s a field ripe with opinions, science and experimentation
and listicles.
The most extensive classification of media titles in general I could find is TVTropes’ Title Tropes list
which lists over 180 (!) different types of tropes alone. Some of those tropes are:
While naming a movie is a very creative task and pretty successfully defies classification, we can still look at the overall shape of movie titles and see if that has any impact on the number of title drops.
One such simple aspect is the length of the title itself. As you would expect there’s a negative correlation (if only a slight one*) between the length of a title and the number of title drops it does.
Still, there are some fun examples for reaaaaally
long movie titles that nevertheless do at least one title drop:
And while these previous examples only drops parts from before or after the colon, this next specimen actually does an impressive full title drop:
And with that, we’re done with the overarching analysis! Feel free to drop us an e-mail
or follow up on X/X, Bluesky
or Mastodon
if you have comments, questions, praise ❤️
Oh, and one more thing:
If you’re curious, here’s the full dataset for you to explore!
...
Read the original on www.titledrops.net »
We invite you to download raw JunoCam images posted here and do your own image processing on them. Be creative! Anything from cropping to color enhancing to collaging is fair game. Then upload your creations here.
Please refrain from direct use of any official NASA or Juno mission logos in your work, as this confuses what is officially sanctioned by NASA and by the Juno Project.
We invite you to download raw JunoCam images posted here and do your own image processing on them. Be creative! Anything from cropping to color enhancing to collaging is fair game. Then upload your creations here.
Please refrain from direct use of any official NASA or Juno mission logos in your work, as this confuses what is officially sanctioned by NASA and by the Juno Project.
We ask that you refrain from posting any patently offensive, political, or inappropriate images. Let’s keep it clean and fun for everyone of any age! Remember, this section is moderated so inappropriate content will be rejected. But creativity and curiosity in the scientific spirit and the adventure of space exploration is highly encouraged and we look forward to seeing Jupiter through not only JunoCam’s eyes, but your own. Have at it!
...
Read the original on www.missionjuno.swri.edu »
My coworker and I enjoy having debates about whether the American economy is in the express lane to collapse or cruising in the good times (I’m really fun at parties). For the two-and-a-half years I’ve worked at my current company, one of us has been a bull while the other has been a bear. I’ll let you guess which one is me.
Monday, November 4, when I walked into the office he brought up a podcast he had been listening to on the drive to work: All-In. I had never heard of it, but I guess it’s a group of four venture capitalists that talk about politics, current events, and the economy.
My coworker surfaced a point that the podcasters had made in the opening segment of last week’s episode: 85% of the past quarter’s economic growth came from government spending. I was stunned. I had in my mind that government spending composed something like 30%-40% of GDP thanks to Matt Yglesias’ recent tirades about how imports don’t subtract from GDP, resurfacing the macro-101 equation:
My coworker showed me the first few minutes of the podcast, where they flash this chart after noting the economy grew by 2.8% in Q3, and one of the hosts, Chamath Palihapitiya, describes what he sees as going on:
This is where you can get a little confused by data. Jason, this is net outlays. And that’s different from total gross government spending, which also includes QE… So just to be clear about what’s happening, 85% of this quarter’s GDP was induced by the government. If you sub it out, so take 2.8% and multiply it by 0.15, that is the true growth X the United States government that exists in the United States economy today. Sacks, your thoughts here on the GDP, obviously looks pretty good for Biden-Harris to have all these stats going in their favor, but there is the caveat obviously about the government spending in there.
“This is where you can get a little confused about data” yeah, okay big guy. Let’s see who is confused here.
Putting aside the comment about quantitative easing, which feels irrelevant, I left his office and went straight to the Department of Commerce’s website, where the Bureau of Economic Analysis publishes GDP estimates. The third-quarter advance estimate table 2 provides us the information we’re looking for, and, in fact, is the source of Chamath’s graph.
You can see the Macro-101 equation recreated here. All of these subcategories (personal consumption + investment + net exports + government consumption) add up to 2.8% (2.82% to be precise) of Q3 GDP growth.
0.85% of the total 2.82% GDP growth is from government spending. Meaning that 0.85% / 2.82% = 30.1% of Q3 GDP growth came from government spending, not 85%.
If you look closely at Chamath’s chart, you can tell that he’s using this exact data source to develop his gross misinterpretation of the data.
So, Chamath’s thesis that “if you back out the percentage of government consumption that is included in GDP, you start to see a very different picture, which is that over the last two and a half years, all of the economic gains under the Biden administration have largely been through government consumption” is total hogwash. The claim that makes up the entire talking point of this initial segment of the show is a misreading of the data that I, a random nonexpert guy, noticed and disproved in ten minutes of research and writing this up.
Looking at government expenditures as a proportion of GDP over time, you can see that the current period is nothing new—in fact, it’s typical for the post-Great Recession era, roughly in line with government spending from the late Obama years through Trump’s presidency, pre-COVID.
Was this gross incompetence or purposeful deception? I’m not sure. But I know that I won’t be tuning in for the next episode of All-In to find out. I will not fall prey to Gell-Mann Amnesia. In my first and only 15 minutes of watching, Chamath’s confidence in making this false claim, coupled with his co-hosts’ complete lack of critical pushback, suggests to me that these kinds of mistakes happen often enough to where these guys’ content isn’t worth consuming.
...
Read the original on passingtime.substack.com »
On its flight to the International Space Station, Dragon executes a series of burns that position the vehicle progressively closer to the station before it performs final docking maneuvers, followed by pressurization of the vestibule, hatch opening, and crew ingress.
On its flight to the International Space Station, Dragon executed a series of burns that positioned the vehicle progressively closer to the station before it performed final docking maneuvers, followed by pressurization of the vestibule, hatch opening, and crew ingress.
On its flight to the International Space Station, Dragon executes a series of burns that position the vehicle progressively closer to the station before it performs final docking maneuvers, followed by pressurization of the vestibule, hatch opening, and crew ingress.
On its flight to the International Space Station, Dragon executed a series of burns that positioned the vehicle progressively closer to the station before it performed final docking maneuvers, followed by pressurization of the vestibule, hatch opening, and crew ingress.
Falcon 9’s first stage lofts Dragon to orbit. Falcon 9’s first and second stage separate. Second stage accelerates Dragon to orbital velocity.
Dragon separates from Falcon 9’s second stage and performs initial orbit activation and checkouts of propulsion, life support, and thermal control systems.
Dragon performs delta-velocity orbit raising maneuvers to catch up with the International Space Station.
Dragon establishes a communication link with the International Space Station and performs its final orbit raising delta-velocity burn.
Dragon establishes relative navigation to the International Space Station and arrives along the docking axis, initiating an autonomous approach.
Dragon performs final approach and docks with the International Space Station, followed by pressurization, hatch open, and crew ingress.
...
Read the original on www.spacex.com »
Please turn on JavaScript in your browser and refresh the page to view its content.
...
Read the original on security.apple.com »
Although I have a good gig as a full professor at Iowa State University, I’ve daydreamed about learning a trade — something that required both my mind and my hands.
So in 2018, I started night courses in welding at Des Moines Area Community College. For three years, I studied different types of welding and during the day worked on a book about the communication between welding teachers and students. I wasn’t the only woman who became interested in trades work during this time. Recognizing the good pay and job security, U. S. women have moved in greater numbers into skilled trades such as welding and fabrication within the past 10 years.
From 2017 to 2022, the number of women in trades rose from about 241,000 to nearly 354,000. That’s an increase of about 47%. Even so, women still constitute just 5.3% of welders in the United States.
When I received my diploma in welding in May 2022, I’d already found the place I wanted to work: Howe’s Welding and Metal Fabrication. I’d met the owner, Jim Howe, when I visited his three-man shop in Ames, Iowa, in January 2022 for research on a second book about communication in skilled trades.
Howe’s shop focuses on repairs and one-off fabrication, not large-scale production of single items. Under Howe’s tutelage, I’ve fabricated skis for the machines that make the rumble strips in the road, shepherd’s hooks for bird feeders, fence poles and stainless-steel lampshade frames. I’ve repaired trailers, wheelchair ramps, office chairs and lawn mowers.
Both my experience at Howe’s and my research at nine other fabrication facilities in Iowa have shown me that — at least for the time being — tradeswomen must find workarounds for commonly encountered challenges. Some of these challenges are physical. These could include being unable to easily reach or move necessary material and tools. Or they could be emotional, such as encountering sexism. As I explore in my forthcoming book, “Learning Skilled Trades in the Workplace,” this is true even in a welcoming environment like Howe’s shop, where I work with a supportive and helpful boss and co-workers.
Being a tradeswoman means being scrutinized for competence. One of the tradeswomen I interviewed for the book told me this story about being tested by more experienced tradesmen:
“I remember them tacking together a couple of pieces of metal for me and saying, ‘Okay, I want you to weld a six millimeter weld here and an eight millimeter weld here,’ and I was so nervous because these are the guys that I’m going to work with, and I just was so nervous and I laid down the welds and put my hood up and the guy goes, ‘Well, goddamn, bitch can weld,’ and I was like, ‘Oh my god, thank god.’”
I’ve felt this same scrutiny from Howe’s customers. Once, two customers watched me as I used the ironworker to punch ovals in rectangular tubing. I had to step on the pedal to lower the punch, find the indentation of the spot to punch, hold a combination square against the metal to ensure the oblong shape was parallel to the tubing’s edge, step on the pedal and pull the stripper toward me.
I could feel my legs turn to jelly as I performed the steps and — as I perceived it — represented the trade competence of all womankind. I’m resentful of these silent evaluations, particularly when I’m learning something new and trying to keep all my fingers.
The standards established by the Occupational Safety and Health Administration, or OSHA, don’t necessarily account for all the physicality of trades work. On the day Jim told me to bend 20 pieces of ½-inch round stock, I had to use all my weight to pull the Hossfeld bender’s arm to make the S shapes.
The 20 S hooks would hang on a bar and hold the 18 come-alongs that Jim had accumulated. Tired after I’d finished all the bending, I sighed as Jim told me to hang all the come-alongs on a mobile rack he had bought at auction for just this purpose.
I had to squat to pick each one up and use my legs and then arms to lift each to a newly made hook. But I didn’t complain. Stoicism is a workaround to credibility.
My interactions with Howe’s customers have been peppered with low-grade sexism. Trying to determine the reason for my presence, one customer asked me, “Are you the new secretary?”
Another man commented on my appearance, comparing me to my co-worker: “You’re better looking than the guy I talked to before.” Such harassment remains common for tradeswomen and ranges from mild, to violent, to just plain creepy, as when one man, paying his bill at the front desk, whispered, “Your hands are dirty.”
Women in trades have reported encounters with customers who doubted their competence and who refused to deal with them, seeking a man instead.
Some customers at Howe’s fit this pattern. I’ve noticed that if I’m at the front desk with a male co-worker, men will often look past me and address them, even though I’m older and, as far as they know, more experienced. Other customers like to tell me how to do my job.
One man, watching me while I cut 8-foot lengths of tubing for him, told me that I could simply hook my tape measure over the saw blade and subtract ⅛-inch to find the correct length. Piqued after I explained why his method wouldn’t work for a precise measurement, he responded by quizzing me on something I wasn’t likely to know: the purpose of the black diamonds on my tape measure.
The man in the audience at the academic conference who wants to lecture rather than ask a question of the woman who is the speaker has become a trope. The pontificating metal-shop customer should be, too. Like other tradeswomen, I’ve learned to work around unwanted comments, including uninvited conversations with men bent on signaling their expertise.
My soon-to-be-published book doesn’t focus solely or even mostly on my experiences as a woman in a welding and fabrication shop. Rather, it looks at the nonlinear process of learning skilled trades — a process that is, for tradeswomen, sometimes frustrated by scrutiny, physical challenges and sexism, which require workarounds.
Nevertheless, along this journey, I’ve leaned on the strength of the tradeswomen before me. Although these women have been “alone in a crowd,” they’ve consistently worked around challenges toward broader and deeper expertise.
...
Read the original on theconversation.com »
Nintendo has confirmed that the successor to the Nintendo Switch will be backward compatible with the Nintendo Switch.
In a post on X, a message from Nintendo president Shuntaro Furukawa also announced that further information about the successor to the Nintendo Switch would come “at a later date.”
“This is Furukawa,” the message reads. “At today’s Corporate Management Policy Briefing, we announced that Nintendo Switch software will also be playable on the successor to Nintendo Switch.
“Nintendo Switch Online will be available on the successor to Nintendo Switch as well. Further information about the successor to Nintendo Switch, including its compatibility with Nintendo Switch, will be announced at a later date.”
The post also confirmed that Nintendo Switch Online would be available on the successor console. No further details on its implementation were announced.
Earlier today, Nintendo reiterated it still intends to announce its next console hardware before the end of its current fiscal year, which concludes on March 31, 2025.
President Shuntaro Furukawa made the comments during an online press conference on Tuesday, following the publication of Nintendo’s latest earnings results, but the executive did not add any additional details.
According to a report, developers have reportedly been briefed not to expect Nintendo’s next console to launch before April 2025.
“No developer I’ve spoken to expects it to be launching this financial year,” said GI.biz journalist Chris Dring. “In fact, they’ve been told not to expect it in the [current] financial year. A bunch of people I spoke to hope it’s out in April or May time, still early next year, not late.
“I don’t think any of us wants a late launch for Switch 2 because we all want a new Nintendo console, everyone gets very excited for it, and we don’t want that crunch of Grand Theft Auto 6 and Switch and all that kind of stuff on top of each other.”
Having launched in March 2017, Switch is in its eighth year on the market. In July, it surpassed the Famicom as the Nintendo console with the longest lifespan before being replaced.
...
Read the original on www.videogameschronicle.com »
/ Sign up for Verge Deals to get deals on products we’ve tested sent to your inbox weekly.
...
Read the original on www.theverge.com »
During my first semester as a computer science graduate student at Princeton, I took COS 402: Artificial Intelligence. Toward the end of the semester there was a lecture about neural networks. This was in the fall of 2008, and I got the distinct impression—both from that lecture and the textbook—that neural networks had become a backwater.
Neural networks had delivered some impressive results in the late 1980s and early 1990s. But then progress stalled. By 2008, many researchers had moved on to mathematically elegant approaches such as support vector machines.
I didn’t know it at the time, but a team at Princeton—in the same computer science building where I was attending lectures—was working on a project that would upend the conventional wisdom and demonstrate the power of neural networks. That team, led by Prof. Fei-Fei Li, wasn’t working on a better version of neural networks. They were hardly thinking about neural networks at all.
Rather, they were creating a new image dataset that would be far larger than any that had come before: 14 million images, each labeled with one of nearly 22,000 categories.
Li tells the story of ImageNet in her recent memoir, The Worlds I See. As she worked on the project, she faced a lot of skepticism from friends and colleagues.
“I think you’ve taken this idea way too far,” a mentor told her a few months into the project in 2007. “The trick is to grow with your field. Not to leap so far ahead of it.”
It wasn’t just that building such a large dataset was a massive logistical challenge. People doubted the machine learning algorithms of the day would benefit from such a vast collection of images.
“Pre-ImageNet, people did not believe in data,” Li said in a September interview at the Computer History Museum. “Everyone was working on completely different paradigms in AI with a tiny bit of data.”
Ignoring negative feedback, Li pursued the project for more than two years. It strained her research budget and the patience of her graduate students. When she took a new job at Stanford in 2009, she took several of those students—and the ImageNet project—with her to California.
ImageNet received little attention for the first couple of years after its release in 2009. But in 2012, a team from the University of Toronto trained a neural network on the ImageNet dataset, achieving unprecedented performance in image recognition. That groundbreaking AI model, dubbed AlexNet after lead author Alex Krizhevsky, kicked off the deep learning boom that has continued until the present day.
AlexNet would not have succeeded without the ImageNet dataset. AlexNet also would not have been possible without a platform called CUDA that allowed Nvidia’s graphics processing units (GPUs) to be used in non-graphics applications. Many people were skeptical when Nvidia announced CUDA in 2006.
So the AI boom of the last 12 years was made possible by three visionaries who pursued unorthodox ideas in the face of widespread criticism. One was Geoffrey Hinton, a University of Toronto computer scientist who spent decades promoting neural networks despite near-universal skepticism. The second was Jensen Huang, the CEO of Nvidia, who recognized early that GPUs could be useful for more than just graphics.
The third was Fei-Fei Li. She created an image dataset that seemed ludicrously large to most of her colleagues. But it turned out to be essential for demonstrating the potential of neural networks trained on GPUs.
A neural network is a network of thousands, millions, or even billions of neurons. Each neuron is a mathematical function that produces an output based on a weighted average of its inputs.
Suppose you want to create a network that can identify handwritten decimal digits like the number two in the red square above. Such a network would take in an intensity value for each pixel in an image and output a probability distribution over the ten possible digits—0, 1, 2, and so forth.
To train such a network, you first initialize it with random weights. Then you run it on a sequence of example images. For each image, you train the network by strengthening the connections that push the network toward the right answer (in this case, a high probability value for the “2” output) and weakening connections that push toward a wrong answer (a low probability for “2” and high probabilities for other digits). If trained on enough example images, the model should start to predict a high probability for “2” when shown a two—and not otherwise.
In the late 1950s, scientists started to experiment with basic networks that had a single layer of neurons. However, their initial enthusiasm cooled as they realized that such simple networks lacked the expressive power required for complex computations.
Deeper networks—those with multiple layers—had the potential to be more versatile. But in the 1960s, no one knew how to train them efficiently. This was because changing a parameter somewhere in the middle of a multi-layer network could have complex and unpredictable effects on the output.
So by the time Hinton began his career in the 1970s, neural networks had fallen out of favor. Hinton wanted to study them, but he struggled to find an academic home to do so. Between 1976 and 1986, Hinton spent time at four different research institutions: Sussex University, the University of California San Diego (UCSD), a branch of the UK Medical Research Council, and finally Carnegie Mellon, where he became a professor in 1982.
In a landmark 1986 paper, Hinton teamed up with two of his former colleagues at UCSD, David Rumelhart and Ronald Williams, to describe a technique called backpropagation for efficiently training deep neural networks.
Their idea was to start with the final layer of the network and work backwards. For each connection in the final layer, the algorithm computes a gradient—a mathematical estimate of whether increasing the strength of that connection would push the network toward the right answer. Based on these gradients, the algorithm adjusts each parameter in the model’s final layer.
The algorithm then propagates these gradients backwards to the second-to-last layer. A key innovation here is a formula—based on the chain rule from high school calculus—for computing the gradients in one layer based on gradients in the following layer. Using these new gradients, the algorithm updates each parameter in the second-to-last layer of the model. Then the gradients get propagated backwards to the third-to-last layer and the whole process repeats once again.
The algorithm only makes small changes to the model in each round of training. But as the process is repeated over thousands, millions, billions, or even trillions of training examples, the model gradually becomes more accurate.
Hinton and his colleagues weren’t the first to discover the basic idea of backpropagation. But their paper popularized the method. As people realized it was now possible to train deeper networks, it triggered a new wave of enthusiasm for neural networks.
Hinton moved to the University of Toronto in 1987 and began attracting young researchers who wanted to study neural networks. One of the first was the French computer scientist Yann LeCun, who did a year-long postdoc with Hinton before moving to Bell Labs in 1988.
Hinton’s backpropagation algorithm allowed LeCun to train models deep enough to perform well on real-world tasks like handwriting recognition. By the mid-1990s, LeCun’s technology was working so well that banks started to use it for processing checks.
“At one point, LeCun’s creation read more than 10 percent of all checks deposited in the United States,” wrote Cade Metz in his 2022 book Genius Makers.
But when LeCun and other researchers tried to apply neural networks to larger and more complex images, it didn’t go well. Neural networks once again fell out of fashion, and some researchers who had focused on neural networks moved on to other projects.
Hinton never stopped believing that neural networks could outperform other machine learning methods. But it would be many years before he’d have access to enough data and computing power to prove his case.
The brains of every personal computer is a central processing unit (CPU). These chips are designed to perform calculations in order, one step at a time. This works fine for conventional software like Windows and Office. But some video games require so many calculations that they strain the capabilities of CPUs. This is especially true of games like Quake, Call of Duty, and Grand Theft Auto that render three-dimensional worlds many times per second.
So gamers rely on GPUs to accelerate performance. Inside a GPU are many execution units—essentially tiny CPUs—packaged together on a single chip. During gameplay, different execution units draw different areas of the screen. This parallelism enables better image quality and higher frame rates than would be possible with a CPU alone.
Nvidia invented the GPU in 1999 and has dominated the market ever since. By the mid-2000s, Nvidia CEO Jensen Huang suspected that the massive computing power inside a GPU would be useful for applications beyond gaming. He hoped scientists could use it for compute-intensive tasks like weather simulation or oil exploration.
So in 2006, Nvidia announced the CUDA platform. CUDA allows programmers to write “kernels,” short programs designed to run on a single execution unit. Kernels allow a big computing task to be split up into bite-sized chunks that can be processed in parallel. This allows certain kinds of calculations to be completed far faster than with a CPU alone.
But there was little interest in CUDA when it was first introduced, wrote Steven Witt in the New Yorker last year:
When CUDA was released, in late 2006, Wall Street reacted with dismay. Huang was bringing supercomputing to the masses, but the masses had shown no indication that they wanted such a thing.“They were spending a fortune on this new chip architecture,” Ben Gilbert, the co-host of “Acquired,” a popular Silicon Valley podcast, said. “They were spending many billions targeting an obscure corner of academic and scientific computing, which was not a large market at the time—certainly less than the billions they were pouring in.”Huang argued that the simple existence of CUDA would enlarge the supercomputing sector. This view was not widely held, and by the end of 2008 Nvidia’s stock price had declined by seventy per cent…Downloads of CUDA hit a peak in 2009, then declined for three years. Board members worried that Nvidia’s depressed stock price would make it a target for corporate raiders.
Huang wasn’t specifically thinking about AI or neural networks when he created the CUDA platform. But it turned out that Hinton’s backpropagation algorithm could easily be split up into bite-sized chunks. And so training neural networks turned out to be a killer app for CUDA.
According to Witt, Hinton was quick to recognize the potential of CUDA:
In 2009, Hinton’s research group used Nvidia’s CUDA platform to train a neural network to recognize human speech. He was surprised by the quality of the results, which he presented at a conference later that year. He then reached out to Nvidia. “I sent an e-mail saying, ‘Look, I just told a thousand machine-learning researchers they should go and buy Nvidia cards. Can you send me a free one?’ ” Hinton told me. “They said no.”
Despite the snub, Hinton and his graduate students, Alex Krizhevsky and Ilya Sutskever, obtained a pair of Nvidia GTX 580 GPUs for the AlexNet project. Each GPU had 512 execution units, allowing Krizhevsky and Sutskever to train a neural network hundreds of times faster than would be possible with a CPU. This speed allowed them to train a larger model—and to train it on many more training images. And they would need all that extra computing power to tackle the massive ImageNet dataset.
Fei-Fei Li wasn’t thinking about either neural networks or GPUs as she began a new job as a computer science professor at Princeton in January of 2007. While earning her PhD at Caltech, she had built a dataset called Caltech 101 that had 9,000 images across 101 categories.
That experience had taught her that computer vision algorithms tended to perform better with larger and more diverse training datasets. Not only had Li found her own algorithms performed better when trained on Caltech 101, other researchers started training their models using Li’s dataset and comparing their performance to one another. This turned Caltech 101 into a benchmark for the field of computer vision.
So when she got to Princeton, Li decided to go much bigger. She became obsessed with an estimate by vision scientist Irving Biederman that the average person recognizes roughly 30,000 different kinds of objects. Li started to wonder if it would be possible to build a truly comprehensive image dataset—one that included every kind of object people commonly encounter in the physical world.
A Princeton colleague told Li about WordNet, a massive database that attempted to catalog and organize 140,000 words. Li called her new dataset ImageNet, and she used WordNet as a starting point for choosing categories. She eliminated verbs and adjectives as well as intangible nouns like “truth.” That left a list of 22,000 countable objects, ranging from ambulance to zucchini.
She planned to take the same approach she’d taken with the Caltech 101 dataset: use Google’s image search to find candidate images, then have a human being verify them. For the Caltech 101 dataset, Li had done this herself over the course of a few months. This time she would need more help. She planned to hire dozens of Princeton undergraduates to help her choose and label images.
But even after heavily optimizing the labeling process—for example, pre-downloading candidate images so they’re instantly available for students to review—Li and her graduate student, Jia Deng, calculated it would take more than 18 years to select and label millions of images.
The project was saved when Li learned about Amazon Mechanical Turk, a crowdsourcing platform Amazon had launched a couple of years earlier. Not only was AMT’s international workforce more affordable than Princeton undergraduates, the platform was far more flexible and scalable. Li’s team could hire as many people as they needed, on demand, and pay them only as long as they had work available.
AMT cut the time needed to complete ImageNet down from 18 to two years. Li writes that her lab spent two years “on the knife-edge of our finances” as they struggled to complete the ImageNet project. But they had enough funds to pay three people to look at each of the 14 million images in the final data set.
ImageNet was ready for publication in 2009, and Li submitted it to the Conference on Computer Vision and Pattern Recognition, which was held in Miami that year. Their paper was accepted, but it didn’t get the kind of recognition Li hoped for.
“ImageNet was relegated to a poster session,” Li writes. “This meant that we wouldn’t be presenting our work in a lecture hall to an audience at a predetermined time, but would instead be given space on the conference floor to prop up a large-format print summarizing the project in hopes that passersby might stop and ask questions… After so many years of effort, this just felt anticlimactic.”
To generate public interest, Li turned ImageNet into a competition. Realizing that the full dataset might be too unwieldy to distribute to dozens of contestants, she created a much smaller (but still massive) dataset with 1,000 categories and 1.4 million images.
The first year’s competition in 2010 generated a healthy amount of interest, with 11 teams participating. The winning entry was based on support vector machines. Unfortunately, Li writes, it was “only a slight improvement over cutting-edge work found elsewhere in our field.”
The second year of the ImageNet competition attracted fewer entries than the first. The winning entry in 2011 was another support vector machine, and it just barely improved on the performance of the 2010 winner. Li started to wonder if the critics had been right. Maybe “ImageNet was too much for most algorithms to handle.”
“For two years running, well-worn algorithms had exhibited only incremental gains in capabilities, while true progress seemed all but absent,” Li writes. “If ImageNet was a bet, it was time to start wondering if we’d lost.”
But when Li reluctantly staged the competition a third time in 2012, the results were totally different. Geoff Hinton’s team was the first to submit a model based on a deep neural network. And its top-5 accuracy was 85 percent—10 percentage points better than the 2011 winner.
Li’s initial reaction was incredulity: “Most of us saw the neural network as a dusty artifact encased in glass and protected by velvet ropes.”
The ImageNet winners were scheduled to be announced at the European Conference on Computer Vision in Florence, Italy. Li, who had a baby at home in California, was planning to skip the event. But when she saw how well AlexNet had done on her dataset, she realized this moment would be too important to miss: “I settled reluctantly on a twenty-hour slog of sleep deprivation and cramped elbow room.”
On an October day in Florence, Alex Krizhevsky presented his results to a standing-room-only crowd of computer vision researchers. Fei-Fei Li was in the audience. So was Yann LeCun.
Cade Metz reports that after the presentation, LeCun stood up and called AlexNet “an unequivocal turning point in the history of computer vision. This is proof.”
The success of AlexNet vindicated Hinton’s faith in neural networks, but it was arguably an even bigger vindication for LeCun.
AlexNet was a convolutional neural network, a type of neural network that LeCun had developed 20 years earlier to recognize handwritten digits on checks. (For more details on how CNNs work, see the in-depth explainer I wrote for Ars Technica in 2018.) Indeed, there were few architectural differences between AlexNet and LeCun’s image recognition networks from the 1990s.
AlexNet was simply far larger. In a 1998 paper, LeCun described a document recognition network with seven layers and 60,000 trainable parameters. AlexNet had eight layers, but these layers had 60 million trainable parameters.
LeCun could not have trained a model that large in the early 1990s because there were no computer chips with as much processing power as a 2012-era GPU. Even if LeCun had managed to build a big enough supercomputer, he would not have had enough images to train it properly. Collecting those images would have been hugely expensive in the years before Google and Amazon Mechanical Turk.
And this is why Fei-Fei Li’s work on ImageNet was so consequential. She didn’t invent convolutional networks or figure out how to make them run efficiently on GPUs. But she provided the training data that large neural networks needed to reach their full potential.
The technology world immediately recognized the importance of AlexNet. Hinton and his students formed a shell company with the goal to be “acquihired” by a big tech company. Within months, Google purchased the company for $44 million. Hinton worked at Google for the next decade while retaining his academic post in Toronto. Ilya Sutskever spent a few years at Google before becoming a cofounder of OpenAI.
AlexNet also made Nvidia GPUs the industry standard for training neural networks. In 2012, the market valued Nvidia at less than $10 billion. Today, Nvidia is one of the most valuable companies in the world, with a market capitalization north of $3 trillion. That high valuation is driven mainly by overwhelming demand for GPUs like the H100 that are optimized for training neural networks.
“That moment was pretty symbolic to the world of AI because three fundamental elements of modern AI converged for the first time,” Li said in a September interview at the Computer History Museum. “The first element was neural networks. The second element was big data, using ImageNet. And the third element was GPU computing.”
Today leading AI labs believe the key to progress in AI is to train huge models on vast data sets. Big technology companies are in such a hurry to build the data centers required to train larger models that they’ve started to lease out entire nuclear power plants to provide the necessary power.
You can view this as a straightforward application of the lessons of AlexNet. But I wonder if we ought to draw the opposite lesson from AlexNet: that it’s a mistake to become too wedded to conventional wisdom.
“Scaling laws” have had a remarkable run in the 12 years since AlexNet, and perhaps we’ll see another generation or two of impressive results as the leading labs scale up their foundation models even more.
But we should be careful not to let the lessons of AlexNet harden into dogma. I think there’s at least a chance that scaling laws will run out of steam in the next few years. And if that happens, we’re going to need a new generation of stubborn nonconformists to notice that the old approach isn’t working and try something different.
...
Read the original on www.understandingai.org »
To add this web app to your iOS home screen tap the share button and select "Add to the Home Screen".
10HN is also available as an iOS App
If you visit 10HN only rarely, check out the the best articles from the past week.
If you like 10HN please leave feedback and share
Visit pancik.com for more.