Watch: Cruise ships chopped in half are a li­cense to print money

Watch: Cruise ships chopped in half are a li­cense to print money

It’s mas­sively prof­itable for cruise op­er­a­tors to hack their ships in half and stick an ex­tra sec­tion in to lengthen them

It’s mas­sively prof­itable for cruise op­er­a­tors to hack their ships in half and stick an ex­tra sec­tion in to lengthen them

The new sec­tion slides in, com­plete with fur­ni­ture and fit­tings that might oth­er­wise be im­pos­si­ble to load on board

That ex­tra sec­tion costs some­where around US$80 mil­lion. It does­n’t look like much, but it rad­i­cally trans­forms the ship’s money-mak­ing ca­pa­bil­ity

A su­per-pre­cise cut right through the mid­dle of the ship

View gallery - 4 im­ages

Slicing huge cruise ships in half, then weld­ing in an ex­tra seg­ment to lengthen them, is more or less a li­cense to print money for cruise op­er­a­tors — so this ‘jumboization’ surgery is be­com­ing very com­mon. Let’s take a look at how it’s done. Humanity, it seems, can’t get enough of cruise ships. In 1990, ac­cord­ing to Cruise Market Watch, the global cruise in­dus­try treated some 3.7 mil­lion pas­sen­gers to its fa­mil­iar regime of buf­fet food, all-in­clu­sive child su­per­vi­sion, shuf­fle­board, plen­ti­ful liquor and wink­ing en­ter­tain­ers. In 2024, that num­ber’s track­ing closer to 30 mil­lion.So the ships are get­ting big­ger, as ev­i­denced by Royal Caribbean Group’s Icon of the Seas, launched this January. This gaudy, tee­ter­ing be­he­moth is de­signed to carry just un­der 10,000 peo­ple, in­clud­ing crew. It’s just un­der 1,200 ft (365 m) long, 159 ft (48 m) wide, and it’s stacked no less than 20 decks high, with a tech­ni­colour wa­ter fun park, ‘Aquadome,’ seven swim­ming pools and the in­dus­try’s tallest wa­ter­fall piled on top. Good lord:But this tow­er­ing leisure-fest cost around US$2 bil­lion to build, and took around two and a half years to launch from the first steel cut­ting. The cruise ship in­dus­try can’t build new ships fast enough to sat­isfy the ram­pant de­mand — and build­ing new ships also means you have to train new crews.

The cheaper, eas­ier way for op­er­a­tors to ex­pand their car­ry­ing ca­pac­ity and jack up prof­its is to make ex­ist­ing ships big­ger, as it turns out. For an av­er­age of around $80 mil­lion, and just a cou­ple of months out of ser­vice, op­er­a­tors can chop an ex­ist­ing ship down the mid­dle, slide in a new slice that’s de­signed to fit per­fectly, weld it to­gether, and come away with enough ex­tra pre­mium cab­ins to pay off the whole op­er­a­tion within a few years. That’s not to men­tion the op­por­tu­nity for a new paint job, big­ger deck pools or en­gine up­grades while the ship’s up on blocks — and HR only needs to train a small per­cent­age of ex­tra staff to add to an ex­ist­ing crew. The re­sult: with a much smaller out­lay and a neg­li­gi­ble gap in ser­vice, op­er­a­tors can make an ex­ist­ing boat much more prof­itable. ‘Jumboization,’ as the process is known, cer­tainly is­n’t a new idea, or unique to cruise ships. Indeed, the term it­self ap­pears to have been coined just af­ter World War II, at which point ship­builders were al­ready do­ing it at enor­mous scale to lengthen war­ships, in some of the most com­plex en­gi­neer­ing work ever at­tempted in the slide rule era. Nowadays, it’s a well-trod­den path, with cer­tain ship­yards spe­cial­iz­ing in the process of adding 80-130 ft (24-40 m) to a given cruise ship. The process takes the best part of a year if you in­clude the mea­sure, de­sign and build of the new slice of ship — but re­mark­ably, just a mat­ter of weeks for the ac­tual ma­jor surgery it­self. How it’s done­First, you need the new chunk of ship ready to roll. Engineers pore over the ini­tial de­sign plans, and then go on board to make hun­dreds of mea­sure­ments; clearly, it’s got to be pre­cise.This done, they de­sign the new sec­tion. Every one of the thou­sands of pipes, wires, ca­bles and ven­ti­la­tion chan­nels that’ll be cut are de­signed into the new sec­tion, ready to line up per­fectly at both ends when the new joins are made. In this way, ex­ist­ing func­tion­al­ity can be main­tained — and if the ex­tra cab­ins place too much de­mand on ex­ist­ing sys­tems, they can be up­graded to han­dle their new loads. Then, they go ahead and build the new sec­tion, of­ten com­plete with in­te­rior fit-outs, at a ship­yard fa­cil­ity, leav­ing the ends open. Sometimes, as in the video be­low of the MSC Armonia, these open sec­tions are ac­tu­ally chris­tened and launched into the wa­ter in their own right, and towed to the dry dock where the surgery is sched­uled, like float­ing apart­ment blocks. Then, the ship is brought in, and pre­cisely po­si­tioned above an ar­ray of “skid shoes” — lit­tle ship-lift­ing sleds po­si­tioned on metal tracks pre-bolted to the floor of the dry dock. Each of these lit­tle sleds is a hy­draulic jack ca­pa­ble of lift­ing as much as 1,000 tons — and each is also ca­pa­ble of ap­ply­ing hy­draulic lat­eral force to move it along the tracks. There might be up­wards of 50 of these skid shoes in­volved in a given ship-stretch­ing op­er­a­tion. With the ship po­si­tioned over the skid shoes, the dock is emp­tied, and the ship comes down to rest on its new mo­bile sup­ports. Then, a work crew that could ex­tend into the hun­dreds gets to work chop­ping the ship in half. Laser guides are used to en­sure mil­lime­ter-level pre­ci­sion, since it’s crit­i­cal that the edges meet per­fectly when the new sec­tion comes in. And of course, care is taken to plan which bits get cut in which or­der, be­cause of the enor­mous weight in­volved and the as­so­ci­ated struc­tural pres­sures that might re­sult.

A su­per-pre­cise cut right through the mid­dle of the ship

The cut­ting job it­self is split be­tween au­to­mated cut­ting ma­chines, which han­dle the straight sur­faces like the side walls and floors, and skilled la­bor­ers for curved or com­plex shapes, but both the ma­chines and the work­ers use com­mon acety­lene blow­torches to cut through the metal.When the whole ship’s been cut right through (memories of a par­tic­u­larly hor­ri­fy­ing Three Body Problem scene any­one?) the skid shoes be­gin the co-or­di­nated process of pulling the two halves apart, far enough for the new slice to be moved in on its own team of skid shoes. This is also a fine op­por­tu­nity to load the ex­ist­ing ship seg­ments up with large fit­ments and items that would oth­er­wise be im­pos­si­ble to get into the lower decks.The whole ship can be chopped in half and sep­a­rated in about two days.

The new sec­tion slides in, com­plete with fur­ni­ture and fit­tings that might oth­er­wise be im­pos­si­ble to load on board

Once that’s all done, the edges of the stem, mid­dle and stern sec­tions are lined up to per­fec­tion us­ing the lift­ing and mov­ing ca­pa­bil­i­ties of the skid shoes, and then it’s time to start con­nect­ing things back to­gether. The sur­faces are brought to­gether, and the weld­ing be­gins.A com­bi­na­tion of elec­trode and stick weld­ing is used — and here’s where the ex­treme cut­ting pre­ci­sion re­ally comes into play, be­cause the max­i­mum dis­tance that can be cov­ered be­tween the two sur­faces is only a few mil­lime­ters. The closer, the bet­ter — and of course, there’s re­ally no mar­gin for er­ror with the wa­ter­tight hull and the struc­tural sup­ports that’ll hold the mas­sive ship to­gether in rough seas. With the hull, decks and struc­ture all joined to­gether, it’s then time to re­con­nect all the thou­sands of ca­bles, ducts, wires and other fit­ments, while in­te­rior teams scurry to make the joins seam­less on the in­side of the ship, and an­other squad be­gins to give the whole ship a paint job that’ll cover up the welds and scars on the ex­te­rior. The ship­yard’s re­spon­si­bil­ity is struc­tural in­tegrity and sea­wor­thi­ness — but the op­er­a­tor also has to thor­oughly test every sin­gle elec­tri­cal, me­chan­i­cal and hy­draulic sys­tem on the ship to make sure every­thing has been re­con­nected prop­erly. They also need to run sea tri­als to prove the strength and han­dling of the new, longer ship.The whole op­er­a­tion, from when the ship ar­rives at the dry dock to the point where it’s ap­proved to be sent back out into ser­vice, can take just nine weeks. If you in­clude the time spent build­ing the new sec­tion, as well as all the pre-plan­ning and en­gi­neer­ing, the whole pro­ject might run out to about nine months.

That ex­tra sec­tion costs some­where around US$80 mil­lion. It does­n’t look like much, but it rad­i­cally trans­forms the ship’s money-mak­ing ca­pa­bil­ity

As I said, this is­n’t a new process; Wikipedia lists 21 cruise ships that have been length­ened in this way since 1977, and in­deed the mighty Seawise Giant — the longest self-pro­pelled ship in his­tory at a stag­ger­ing 1,504 ft (458 m) end to end — owes some of its prodi­gious length to the jum­boiza­tion tech­nique.But the scale of these op­er­a­tions, the tech­niques in­volved and the fi­nan­cial in­cen­tives be­hind them have made it a fas­ci­nat­ing rab­bit hole for me. I hope you’ve en­joyed it too!And if you wanna dive even deeper, take a look at this fan­tas­tic mini-doco from Spark:

Jumboization - Modifying Ships To Make Them Even Bigger [4K] | Heavy Lift | Spark

View gallery - 4 im­ages

Loz leads the New Atlas team as Editorial Director, af­ter nearly two decades as one of our most ver­sa­tile writ­ers. He’s also proven him­self as a pho­tog­ra­pher, video­g­ra­pher, pre­sen­ter, pro­ducer and pod­cast en­gi­neer. A grad­u­ate in Psychology, for­mer busi­ness an­a­lyst and tour­ing mu­si­cian, he’s cov­ered just about every­thing for New Atlas, con­cen­trat­ing lately on clean en­ergy, AI, hu­manoid ro­bot­ics, next-gen air­craft, and the odd bit of mu­sic, mo­tor­cy­cles and au­to­mo­tive.

Old codger yells at soft­ware, part the lat­est.

The story so far

A great amount of my writ­ing on this site re­volves around com­plain­ing about mod­ern user in­ter­face de­sign. Some peo­ple agree with me, some don’t. Most prob­a­bly don’t care at all. That’s fine. What I find in­ter­est­ing is that many (but not all!) peo­ple who dis­agree ei­ther pre­sent ex­tremely spe­cific non-ar­gu­ment nit­picks like “Something in Windows 3.1 was bad, too!” and ig­nores the big ques­tions posed, such as whether it’s ac­tu­ally a good idea shov­ing every­thing into a sin­gle ham­burger menu, or if mix­ing touch and desk­top par­a­digms wildly be­tween and some­times within pro­grams is ac­tu­ally ben­e­fi­cial to end users. Others - de­spite my ef­forts to the con­trary - use sweep­ing dis­missals of the kind “you just don’t like flat de­sign.” That’s true - I don’t like flat de­sign, but many of my ar­gu­ments have noth­ing what­so­ever to do with aes­thet­ics.

Well, that’s what writ­ing on the net is like. But I shall not de­spair, nor shall I be si­lenced! Allow me, for a few mo­ments, to fo­cus on a very de­tailed ex­am­ple that’s got noth­ing to do with flat­ness, but rather with how to ac­cess core pro­gram func­tion­al­ity.

It’s worth men­tion­ing that I agree that the mod­ern de­sign par­a­digm prob­a­bly is friendly to be­gin­ner users in many ways. But at some point, peo­ple stop be­ing be­gin­ners. People who use com­put­ers sev­eral hours per day, per­form­ing a wide va­ri­ety of tasks in many dif­fer­ent pro­grams, should also be taken in to ac­count when de­sign­ing soft­ware. As such, my cri­tique comes from the point of what’s usu­ally called a “power user”. It’s also worth con­sid­er­ing that the more an in­ter­face hides, the less it of­fers by way of op­por­tu­ni­ties for a user to grow and learn.

For full trans­parency: It’s no se­cret that I don’t par­tic­u­larly en­joy the Gnome desk­top. When I in­stalled Ubuntu on my work lap­top about a year ago, I wanted to see how long I could get by us­ing Gnome. The an­swer was “about five min­utes”. When I de­cided to switch the de­fault desk­top back­ground im­age to a solid color, I dis­cov­ered that the only avail­able op­tion in the con­fig­u­ra­tion tool was to se­lect im­ages. Sure, I could google a so­lu­tion to the prob­lem, which in­volved a pretty lengthy com­mand line in­can­ta­tion, but de­priv­ing the user of a sim­ple op­tion to se­lect a back­ground color was more than my pa­tience could han­dle.

I’ve writ­ten some pretty harsh cri­tique about Gnome be­fore, but I feel this is a pro­ject that does de­serve cri­tique. It’s not just the de­fault desk­top en­vi­ron­ment in most ma­jor Linux dis­tri­b­u­tions, it’s also a pro­ject that’s very vo­cal - opin­ion­ated, as the say­ing goes - about how to do things. A few ex­am­ples might be in or­der:

“Our soft­ware is built to be us­able by every­one. We care deeply about user ex­pe­ri­ence.”

“Software should be struc­turally and aes­thet­i­cally el­e­gant.”

“People’s at­ten­tion is pre­cious. We pride our­selves in be­ing dis­trac­tion free.”

“The ‘traditional desk­top’ is dead, and it’s not com­ing back. (…) Instead of try­ing to bring back old con­cepts like menu bars or sta­tus icons, in­vent some­thing bet­ter from first prin­ci­ples.”

Bold words, in­deed! Let’s see how they fare in the real world. Recently, in an­other sud­den fit of open-mind­ed­ness, I de­cided to per­form some light file or­ga­niz­ing us­ing Gnome Files. This is­n’t just any old Gnome ap­pli­ca­tion: the file man­ager is a cen­tral point of any desk­top en­vi­ron­ment, along with win­dow man­age­ment and a de­cent set of UI wid­gets. Surely the Gnome pro­ject will have spent a good amount of time and ef­fort on mak­ing Gnome Files a flag­ship ap­pli­ca­tion, show­cas­ing their su­pe­rior UI phi­los­o­phy from its ab­solutely best side.

Well, it cer­tainly does­n’t look too shabby. Clean, el­e­gant, even min­i­mal­ist. A calm­ing ap­pear­ance. Sure, it’s a flat de­sign, but I can clearly make out the dif­fer­ent UI el­e­ments and get a good idea of what’s click­able or not - or so I’m led to be­lieve. There’s just one prob­lem - I’d pre­fer a list view in­stead of the large icons. How do I change it?

Note the many dis­tinct-look­ing icons in the tool­bar. From left to right we have three stacked dots, three stacked dots and lines, and three stacked lines. Pretty ob­vi­ous what they’re all about, right? Anyways, this drop­down menu with the tooltip text “View Options” is prob­a­bly a good start.

Alas, that menu only shows var­i­ous sort op­tions. This makes me think “Sort Options” would have been a bet­ter tooltip text. Then again, I’m not much for rein­vent­ing every­thing from first prin­ci­ples, so what do I know?

What’s in the “Main Menu”, then? Nothing that has to do with a list view, I’m afraid - though there are op­tions here I’d ex­pect to find in a menu called “View Options”, such as Icon Size and Show Hidden Files.

There’s a “Folder Menu” too, but that does­n’t help me much, ei­ther.

Ah! There it is! The “View Options” drop­down is ac­tu­ally a split but­ton, with a tog­gle part and a drop­down part. That’s sort of fine I guess, but I’d ex­pect the tog­gle op­tions to also be listed in the drop­down por­tion of the wid­get. If not, why group them to­gether like this? Why not split them into two clearly sep­a­rate but­tons? This hon­estly took me a good while to fig­ure out - and I’d like to think I’m nei­ther ex­tremely stu­pid nor par­tic­u­larly in­ex­pe­ri­enced with com­put­ers.

Yes, it re­ally was this hard for me to find the list view tog­gle. The “View Options” drop­down did­n’t con­tain view op­tions, but rather sort op­tions, and I did­n’t re­al­ize it was a split but­ton with two com­pletely dif­fer­ent func­tions. I was fur­ther con­fused when other ac­tual view op­tions were, in fact, only avail­able in the “Main Menu”. To me, this does­n’t feel like “structural el­e­gance”. By now, my calm was re­placed with frus­tra­tion.

I ac­tu­ally re­sorted to the built-in help func­tion when look­ing for a way to switch to the list view. I searched for “list view”, but did­n’t find any­thing im­me­di­ately help­ful. Even though I brought up the help win­dow us­ing Gnome Files’ menu op­tion for this, search re­sults were listed for many other ap­pli­ca­tions. Search re­sult num­ber nine, “Browse files and fold­ers”, was at least re­lated to Gnome Files - but ap­peared af­ter things like “Manage vol­umes and par­ti­tions” and “Edit con­tact de­tails”. The only en­try I could find while man­u­ally brows­ing the help that men­tioned “List View” was about what I could do when the list view had al­ready been se­lected. I tried my best, but could­n’t find any in­for­ma­tion on how to ac­tu­ally ac­ti­vate it.

While brows­ing the help, I no­ticed some­thing else: friv­o­lous tooltips. Tooltips are a great in­ven­tion and can be very use­ful, but a lot of pro­grams nowa­days seem to just sprin­kle them every­where, with no par­tic­u­lar thought as to whether they’re ac­tu­ally mean­ing­ful or not. Gnome Help is one such case.

Here, I’m just rest­ing my mouse pointer some­where, as one might do when read­ing through a bunch of click­able items. Up pops a tooltip that not only com­pletely cov­ers the ti­tle of the next item, it also has the ex­act same text as the header of the item it be­longs to. Why? Perhaps it’s part of the “distraction free ex­pe­ri­ence”.

The same goes for the left hand side tool­bar in Gnome Files. Yes, I as­sumed that “Recent” and “Starred” here, in the con­text of Gnome Files, are about re­cent and starred files and not some­thing else. Granted, some of these tooltips show a full path, but hon­estly - if I’ve added some­thing to this bar, I prob­a­bly know what it is and where it’s lo­cated. A tooltip would be use­ful if two items have the same name - but why show them every­where else? To me, at least, it’s very an­noy­ing that I can’t rest my mouse pointer some­where with­out hav­ing point­less in­for­ma­tion pop­ping up that cov­ers what I’m re­ally in­ter­ested in look­ing at. In fact, I’d say this be­hav­ior teaches me as a user that tooltips are point­less and ir­ri­tat­ing, mak­ing me in­stinc­tively ig­nore them even when they might be use­ful.

Navigating us­ing the Gnome Files UI is fine, for the most part. I’m un­used to the but­ton lo­ca­tions, and I miss a but­ton for go­ing one level up, to the par­ent di­rec­tory. There are but­tons for go­ing back and for­ward in the nav­i­ga­tion his­tory, but that’s not the same thing. Clicking on a di­rec­tory in the lo­ca­tion bar will take you there, but it’s much eas­ier to misclick and thus less con­ve­nient.

When search­ing for a “parent di­rec­tory” but­ton, I no­ticed a few strange be­hav­iors. Clicking on di­rec­tory names in the lo­ca­tion bar is a nice fea­ture. However, many file man­agers will also let the user di­rectly en­ter a path here. In fact, it looks a lot like a text box (such as the Folder name wid­get in the screen­shot above) but no mat­ter where I click in it, I can­not ac­ti­vate a nor­mal edit­ing mode.

It seems this edit­ing mode can only be ac­ti­vated us­ing a key­board short­cut, Ctrl-L, which is­n’t im­me­di­ately ap­par­ent - or, to be frank, very log­i­cal. It looks like a text box, and it is in­deed a text box - not just all the time. Maybe I’m not sup­posed to think of it as a text box, but the de­sign lan­guage here is­n’t ex­actly su­per clear on what kind of in­put I’m deal­ing with, so I’m work­ing from as­sump­tions based on hav­ing used com­put­ers and GUI file man­agers for 36 years - which is some­thing de­vel­op­ers and de­sign­ers should take into ac­count when build­ing in­ter­faces.

Having no mouse-dri­ven way of ac­ti­vat­ing a fairly com­mon UI task feels like a step back in dis­cov­er­abil­ity. In fact, I thought the fea­ture had­n’t been im­ple­mented un­til I googled it. To be fair, it listed in the Keyboard Shortcuts win­dow, but call me old school for ex­pect­ing GUI el­e­ments to be ac­ces­si­ble with the mouse.

The short­cuts list is three pages long. It’s got a search func­tion, which is nice - if you know what you’re look­ing for. For ex­am­ple, Gnome Help calls the lo­ca­tion bar “the path bar”, but search­ing for “path” gives zero re­sults. If you don’t know what to search for, there’s no table of con­tents or other way of quickly view­ing the var­i­ous cat­e­gories of short­cuts. You have to ex­am­ine each of the pages and hope to find what you’re look­ing for.

Upon reach­ing page three, I learned that there’s even a key­board short­cut for open­ing the key­board short­cuts win­dow - but un­like some other short­cuts, it’s not shown next to its cor­re­spond­ing GUI en­try in the Main Menu. Inconsistent and con­fus­ing.

I’m not sure why, but the Gnome pro­ject has a strong aver­sion to menu bars. The key­board short­cuts win­dow may be “reinventing from first prin­ci­ples”, but to me this in­ven­tion looks like a flawed rein­ter­pre­ta­tion of a tra­di­tional menu bar. A menu bar pro­vides a way of ex­pos­ing cat­e­go­rized pro­gram fea­tures in a fa­mil­iar and al­ways-vis­i­ble UI, lists their key­board short­cuts in a con­sis­tent man­ner and it lets the user in­ter­act with the op­tion im­me­di­ately upon find­ing it.

In Gnome Files, we’re in­stead given a hand­ful of fea­tures scat­tered across the UI. Hidden fea­tures (accessible solely through key­board shorcuts) can only be learned by brows­ing what is best de­scribed as a non-in­ter­ac­tive menu of the kind you’d find printed on pa­per in a restau­rant. This brows­ing brings a con­sid­er­able con­text switch be­cause the win­dow is modal, so you can’t keep it open while ex­per­i­ment­ing. You have to find what you’re look­ing for, note the key­board short­cut, close the short­cuts win­dow, and then in­voke the fea­ture. Keyboard short­cuts aren’t bad, but in a mouse dri­ven en­vi­ron­ment, hav­ing no way of find­ing and in­vok­ing fea­tures through the GUI is lim­it­ing and con­fus­ing. And why force users to mem­o­rize a short­cut for some­thing they might only do oc­ca­sion­ally?

In short, this feels like an af­ter­thought rather than a rev­o­lu­tion­ary new, ef­fi­cient ap­proach to op­er­at­ing a GUI.

When try­ing to ac­ti­vate path edit­ing us­ing the mouse, my vig­or­ous click­ing just made the Gnome Files win­dow move around. Since the win­dow has no real ti­tle bar, it must be moved by click­ing-and-drag­ging in the top part of the win­dow. This is also where the tool­bar re­sides, which means click-drag­ging on UI con­trols that al­ready have a com­pletely dif­fer­ent func­tion. You can use the search func­tion by click­ing on the search icon - but you can also click the search icon and start drag­ging the win­dow around.

There’s a lo­ca­tion his­tory avail­able by ei­ther con­text-click­ing or per­form­ing a “long click” on the back- and for­ward but­tons, but noth­ing about the but­tons in­di­cate this (and the fea­ture does­n’t seem to have a key­board short­cut), which in­tro­duces fur­ther am­bi­gu­ity re­gard­ing the click-drag be­hav­ior. Confusingly, per­form­ing a long click on other items does­n’t bring up their con­text menu.

Activating win­dows with the mouse is even worse: If I want to bring a win­dow to the front of the stack, I have to search for a non-click­able area, lest I ac­ti­vate the search fea­ture, skip up­wards in the path, switch from list view back to icon view or ac­ci­den­tally ac­cess some other pro­gram fea­ture. This in­tro­duces ex­treme am­bi­gu­ity that makes me as a user feel in­se­cure and un­com­fort­able: a sim­ple mouse slip can give the most sur­pris­ing re­sults. The user is con­di­tioned to be mind­ful of this, which in­creases the cog­ni­tive load for very sim­ple and com­mon win­dow op­er­a­tions.

Another cu­ri­ous idio­syn­crasy of for­go­ing win­dow ti­tle bars is that right-click­ing on for ex­am­ple the search icon ac­tu­ally brings up a win­dow man­age­ment menu, with op­tions for clos­ing, max­i­miz­ing and min­i­miz­ing the win­dow. This pops up when right click­ing any­where in the top part of the win­dow. Except when right click­ing on a di­rec­tory name in the lo­ca­tion bar - that will in­stead bring up a com­pletely dif­fer­ent menu, with var­i­ous di­rec­tory ac­tions. Except if you click on the cur­rent di­rec­tory, which won’t bring up a con­text menu. Except some­times it does, but then it’s the win­dow ac­tions menu. Which is also what hap­pens if you miss the con­text-click­able area of an­other di­rec­tory by just a pixel, even if the pointer is still in­side the lo­ca­tion bar.

You can, how­ever, mid­dle click on any di­rec­tory name - - to open it in a new tab. Which is also one of the op­tions avail­able in the con­text menu.

Gnome Files, or rather GTK 4, uses hid­den scroll bars. I’m us­ing what I as­sume to be the de­fault set­tings and the de­fault GTK 4 theme as sup­plied by Debian, be­cause I haven’t changed any­thing (except the font). I per­son­ally don’t like hid­den scroll bars, be­cause hid­ing the scroll bar also hides in­for­ma­tion not only about what I can do with the GUI it­self, but also about where I’m cur­rently po­si­tioned in E. G. a file list­ing or text doc­u­ment.

Moving the mouse about in Gnome Files re­veals the scroll bar, as de­picted in the first pic­ture above. It’s very small and very hard to see be­cause of the low con­trast, but it’s there. But - Ah-hah! - the scroll bar is ac­tu­ally hid­den in two steps! When mov­ing the mouse pointer to it, it grows and be­comes more vis­i­ble, as in the sec­ond pic­ture above. But when it does, it also moves the en­tire width of its orig­i­nal size to the left, mean­ing that my mouse pointer is now point­ing at… noth­ing. Thanks, Gnomebama.

The Gnome Files UI feels hap­haz­ard, in­co­her­ent and some­times even dan­ger­ous:

Menu names and their con­tents are con­fus­ing, with “View Options” ac­tu­ally be­ing sort op­tions. Actual view op­tions are in­stead sprin­kled across other parts of the UI.

Some com­mon fea­tures are only ac­ces­si­ble - and dis­cov­er­able - through key­board short­cuts. The key­board short­cuts list­ing is non-in­ter­ac­tive, modal, and in­curs a sub­stan­tial men­tal con­text switch.

Widgets don’t be­have quite as you’d ex­pect when com­pared to other, sim­i­lar, mod­ern ap­pli­ca­tions.

Even within the closed world of Gnome Files, wid­get looks can be de­cep­tive.

Tooltips are ei­ther mis­lead­ing, or com­i­cally un­in­for­ma­tive and thus an­noy­ingly dis­tract­ing.

Moving win­dows by click­ing on icons that al­ready have a spe­cific func­tion feels un­in­tu­itive and in­tro­duces an un­nec­es­sary risk of misclick­ing.

To ac­ti­vate a win­dow with­out also ac­ti­vat­ing a pro­gram fea­ture, time must be spent search­ing for a non-in­ter­ac­tive area of the UI.

Context-clicking in the top part of the win­dow gives spu­ri­ous and un­pre­dictable re­sults.

In the de­fault theme, scroll bars jump away from their orig­i­nal po­si­tion when you point at them.

Searching and then brows­ing the built-in help for “list view” did­n’t ac­tu­ally help me find out how to en­able the list view. The only page I could find that con­tains the ex­act phrase ap­pears as #15 among the search re­sults.

Nomenclature dif­fers be­tween the help texts and the ac­tual GUI.

The many var­i­ous in­con­sis­ten­cies listed above makes it hard for me as a user to con­struct men­tal mod­els and pat­terns for mak­ing ef­fi­cient and re­li­able as­sump­tions about the UI. I of­ten have to dou­ble check things and look for fea­tures in the key­board short­cuts win­dow, since I don’t know if they’ve been deigned a menu en­try or tool­bar icon. Ambiguous mouse be­hav­ior in­tro­duces un­nec­es­sary cog­ni­tive load, mak­ing it hard to build con­fi­dence even when per­form­ing ba­sic tasks.

Designing user in­ter­faces is hard. Writing soft­ware is hard. Designing great in­ter­faces and writ­ing great soft­ware is even harder. No mat­ter how hard you try, you can’t please every­one. Functionally, Gnome Files lets me do file man­age­ment. I could, if I had no other op­tions, prob­a­bly get used to its UI idio­syn­crasies. But as I’ve hope­fully demon­strated above, there are many things about Gnome Files - a cen­tral part of the Gnome desk­top - that can be con­sid­ered ob­jec­tively bad from a UI de­sign stand­point.

This does­n’t mean that there aren’t other bad pro­grams or that pre­vi­ous de­sign par­a­digms were per­fec­tion in­car­nate. Neither does it mean that every lit­tle bit of Gnome Files is worth­less, or that Gnome and Gnome Files are unique in em­ploy­ing these pat­ters (but that’s hardly a com­fort). But should­n’t this new de­sign par­a­digm pro­duce some­thing bet­ter? Isn’t that its whole rai­son d’être? After ten-fif­teen years of the old main­stream desk­top par­a­digm, we ar­rived at Windows 95 - which was­n’t per­fect, but pretty damn con­sis­tent, pre­dictable and re­li­able. After ten-fif­teen years of this new par­a­digm, we in­stead have con­stant re­designs and re­work­ings of so­lu­tions to prob­lems we al­ready know how to solve.

Examining these new so­lu­tions sug­gests to me that maybe pro­po­nents of the new UI par­a­digm should be a bit more hum­ble in their ap­proach to tried and tested pat­terns. Nearly all of the crit­i­cisms ex­am­ined above al­ready have well known so­lu­tions that have been honed for a pe­riod of decades. For ex­am­ple: Having ac­tual win­dow ti­tle bars, con­sis­tently list­ing key­board short­cuts in menus, con­sis­tently cat­e­go­riz­ing menus and op­tions, and us­ing a richer de­sign lan­guage. Old does­n’t au­to­mat­i­cally mean worse, just as new does­n’t au­to­mat­i­cally mean bet­ter.

My per­sonal con­clu­sion is that if this is the re­sult of in­vent­ing “something bet­ter from first prin­ci­ples” to cre­ate “structurally el­e­gant” and “distraction free” soft­ware “usable by every­one”, I guess I’m just not “everyone”. And that would be fine, if it was­n’t for the fact that these days, I’m of­ten left with no choice but us­ing pro­grams with UI:s like these.

Academic jour­nals, archives, and repos­i­to­ries are see­ing an in­creas­ing num­ber of ques­tion­able re­search pa­pers clearly pro­duced us­ing gen­er­a­tive AI. They are of­ten cre­ated with widely avail­able, gen­eral-pur­pose AI ap­pli­ca­tions, most likely ChatGPT, and mimic sci­en­tific writ­ing. Google Scholar eas­ily lo­cates and lists these ques­tion­able pa­pers along­side rep­utable, qual­ity-con­trolled re­search. Our analy­sis of a se­lec­tion of ques­tion­able GPT-fabricated sci­en­tific pa­pers found in Google Scholar shows that many are about ap­plied, of­ten con­tro­ver­sial top­ics sus­cep­ti­ble to dis­in­for­ma­tion: the en­vi­ron­ment, health, and com­put­ing. The re­sult­ing en­hanced po­ten­tial for ma­li­cious ma­nip­u­la­tion of so­ci­ety’s ev­i­dence base, par­tic­u­larly in po­lit­i­cally di­vi­sive do­mains, is a grow­ing con­cern.

Where are ques­tion­able pub­li­ca­tions pro­duced with gen­er­a­tive pre-trained trans­form­ers (GPTs) that can be found via Google Scholar pub­lished or de­posited?

What are the main char­ac­ter­is­tics of these pub­li­ca­tions in re­la­tion to pre­dom­i­nant sub­ject cat­e­gories?

How are these pub­li­ca­tions spread in the re­search in­fra­struc­ture for schol­arly com­mu­ni­ca­tion?

How is the role of the schol­arly com­mu­ni­ca­tion in­fra­struc­ture chal­lenged in main­tain­ing pub­lic trust in sci­ence and ev­i­dence through in­ap­pro­pri­ate use of gen­er­a­tive AI?

A sam­ple of sci­en­tific pa­pers with signs of GPT-use found on Google Scholar was re­trieved, down­loaded, and an­a­lyzed us­ing a com­bi­na­tion of qual­i­ta­tive cod­ing and de­scrip­tive sta­tis­tics. All pa­pers con­tained at least one of two com­mon phrases re­turned by con­ver­sa­tional agents that use large lan­guage mod­els (LLM) like OpenAI’s ChatGPT. Google Search was then used to de­ter­mine the ex­tent to which copies of ques­tion­able, GPT-fabricated pa­pers were avail­able in var­i­ous repos­i­to­ries, archives, ci­ta­tion data­bases, and so­cial me­dia plat­forms.

Roughly two-thirds of the re­trieved pa­pers were found to have been pro­duced, at least in part, through undis­closed, po­ten­tially de­cep­tive use of GPT. The ma­jor­ity (57%) of these ques­tion­able pa­pers dealt with pol­icy-rel­e­vant sub­jects (i.e., en­vi­ron­ment, health, com­put­ing), sus­cep­ti­ble to in­flu­ence op­er­a­tions. Most were avail­able in sev­eral copies on dif­fer­ent do­mains (e.g., so­cial me­dia, archives, and repos­i­to­ries).

Two main risks arise from the in­creas­ingly com­mon use of GPT to (mass-)produce fake, sci­en­tific pub­li­ca­tions. First, the abun­dance of fab­ri­cated “studies” seep­ing into all ar­eas of the re­search in­fra­struc­ture threat­ens to over­whelm the schol­arly com­mu­ni­ca­tion sys­tem and jeop­ar­dize the in­tegrity of the sci­en­tific record. A sec­ond risk lies in the in­creased pos­si­bil­ity that con­vinc­ingly sci­en­tific-look­ing con­tent was in fact de­ceit­fully cre­ated with AI tools and is also op­ti­mized to be re­trieved by pub­licly avail­able aca­d­e­mic search en­gines, par­tic­u­larly Google Scholar. However small, this pos­si­bil­ity and aware­ness of it risks un­der­min­ing the ba­sis for trust in sci­en­tific knowl­edge and poses se­ri­ous so­ci­etal risks.

The use of ChatGPT to gen­er­ate text for aca­d­e­mic pa­pers has raised con­cerns about re­search in­tegrity. Discussion of this phe­nom­e­non is on­go­ing in ed­i­to­ri­als, com­men­taries, opin­ion pieces, and on so­cial me­dia (Bom, 2023; Stokel-Walker, 2024; Thorp, 2023). There are now sev­eral lists of pa­pers sus­pected of GPT mis­use, and new pa­pers are con­stantly be­ing added. While many le­git­i­mate uses of GPT for re­search and aca­d­e­mic writ­ing ex­ist (Huang & Tan, 2023; Kitamura, 2023; Lund et al., 2023), its un­de­clared use—be­yond proof­read­ing—has po­ten­tially far-reach­ing im­pli­ca­tions for both sci­ence and so­ci­ety, but es­pe­cially for their re­la­tion­ship. It, there­fore, seems im­por­tant to ex­tend the dis­cus­sion to one of the most ac­ces­si­ble and well-known in­ter­me­di­aries be­tween sci­ence, but also cer­tain types of mis­in­for­ma­tion, and the pub­lic, namely Google Scholar, also in re­sponse to the le­git­i­mate con­cerns that the dis­cus­sion of gen­er­a­tive AI and mis­in­for­ma­tion needs to be more nu­anced and em­pir­i­cally sub­stan­ti­ated  (Simon et al., 2023).

Google Scholar, https://​scholar.google.com, is an easy-to-use aca­d­e­mic search en­gine. It is avail­able for free, and its in­dex is ex­ten­sive (Gusenbauer & Haddaway, 2020). It is also of­ten touted as a cred­i­ble source for aca­d­e­mic lit­er­a­ture and even rec­om­mended in li­brary guides, by me­dia and in­for­ma­tion lit­er­acy ini­tia­tives, and fact check­ers (Tripodi et al., 2023). However, Google Scholar lacks the trans­parency and ad­her­ence to stan­dards that usu­ally char­ac­ter­ize ci­ta­tion data­bases. Instead, Google Scholar uses au­to­mated crawlers, like Google’s web search en­gine (Martín-Martín et al., 2021), and the in­clu­sion cri­te­ria are based on pri­mar­ily tech­ni­cal stan­dards, al­low­ing any in­di­vid­ual au­thor—with or with­out sci­en­tific af­fil­i­a­tion—to up­load pa­pers to be in­dexed (Google Scholar Help, n.d.). It has been shown that Google Scholar is sus­cep­ti­ble to ma­nip­u­la­tion through ci­ta­tion ex­ploits (Antkare, 2020) and by pro­vid­ing ac­cess to fake sci­en­tific pa­pers (Dadkhah et al., 2017). A large part of Google Scholar’s in­dex con­sists of pub­li­ca­tions from es­tab­lished sci­en­tific jour­nals or other forms of qual­ity-con­trolled, schol­arly lit­er­a­ture. However, the in­dex also con­tains a large amount of gray lit­er­a­ture, in­clud­ing stu­dent pa­pers, work­ing pa­pers, re­ports, preprint servers, and aca­d­e­mic net­work­ing sites, as well as ma­te­r­ial from so-called “questionable” aca­d­e­mic jour­nals, in­clud­ing pa­per mills. The search in­ter­face does not of­fer the pos­si­bil­ity to fil­ter the re­sults mean­ing­fully by ma­te­r­ial type, pub­li­ca­tion sta­tus, or form of qual­ity con­trol, such as lim­it­ing the search to peer-re­viewed ma­te­r­ial.

To un­der­stand the oc­cur­rence of ChatGPT (co-)authored work in Google Scholar’s in­dex, we scraped it for pub­li­ca­tions, in­clud­ing one of two com­mon ChatGPT re­sponses (see Appendix A) that we en­coun­tered on so­cial me­dia and in me­dia re­ports (DeGeurin, 2024). The re­sults of our de­scrip­tive sta­tis­ti­cal analy­ses showed that around 62% did not de­clare the use of GPTs. Most of these GPT-fabricated pa­pers were found in non-in­dexed jour­nals and work­ing pa­pers, but some cases in­cluded re­search pub­lished in main­stream sci­en­tific jour­nals and con­fer­ence pro­ceed­ings. More than half (57%) of these GPT-fabricated pa­pers con­cerned pol­icy-rel­e­vant sub­ject ar­eas sus­cep­ti­ble to in­flu­ence op­er­a­tions. To avoid in­creas­ing the vis­i­bil­ity of these pub­li­ca­tions, we ab­stained from ref­er­enc­ing them in this re­search note. However, we have made the data avail­able in the Harvard Dataverse repos­i­tory.

The pub­li­ca­tions were re­lated to three is­sue ar­eas—health (14.5%), en­vi­ron­ment (19.5%) and com­put­ing (23%)—with key terms such “healthcare,” “COVID-19,” or “infection”for health-re­lated pa­pers, and “analysis,” “sustainable,” and “global” for en­vi­ron­ment-re­lated pa­pers. In sev­eral cases, the pa­pers had ti­tles that strung to­gether gen­eral key­words and buzz­words, thus al­lud­ing to very broad and cur­rent re­search. These terms in­cluded “biology,” “telehealth,” “climate pol­icy,” “diversity,” and “disrupting,” to name just a few.  While the study’s scope and de­sign did not in­clude a de­tailed analy­sis of which parts of the ar­ti­cles in­cluded fab­ri­cated text, our dataset did con­tain the sur­round­ing sen­tences for each oc­cur­rence of the sus­pi­cious phrases that formed the ba­sis for our search and sub­se­quent se­lec­tion. Based on that, we can say that the phrases oc­curred in most sec­tions typ­i­cally found in sci­en­tific pub­li­ca­tions, in­clud­ing the lit­er­a­ture re­view, meth­ods, con­cep­tual and the­o­ret­i­cal frame­works, back­ground, mo­ti­va­tion or so­ci­etal rel­e­vance, and even dis­cus­sion. This was con­firmed dur­ing the joint cod­ing, where we read and dis­cussed all ar­ti­cles. It be­came clear that not just the text re­lated to the tell­tale phrases was cre­ated by GPT, but that al­most all ar­ti­cles in our sam­ple of ques­tion­able ar­ti­cles likely con­tained traces of GPT-fabricated text every­where.

Generative pre-trained trans­form­ers (GPTs) can be used to pro­duce texts that mimic sci­en­tific writ­ing. These texts, when made avail­able on­line—as we demon­strate—leak into the data­bases of aca­d­e­mic search en­gines and other parts of the re­search in­fra­struc­ture for schol­arly com­mu­ni­ca­tion. This de­vel­op­ment ex­ac­er­bates prob­lems that were al­ready pre­sent with less so­phis­ti­cated text gen­er­a­tors (Antkare, 2020; Cabanac & Labbé, 2021). Yet, the pub­lic re­lease of ChatGPT in 2022, to­gether with the way Google Scholar works, has in­creased the like­li­hood of lay peo­ple (e.g., me­dia, politi­cians, pa­tients, stu­dents) com­ing across ques­tion­able (or even en­tirely GPT-fabricated) pa­pers and other prob­lem­atic re­search find­ings. Previous re­search has em­pha­sized that the abil­ity to de­ter­mine the value and sta­tus of sci­en­tific pub­li­ca­tions for lay peo­ple is at stake when mis­lead­ing ar­ti­cles are passed off as rep­utable (Haider & Åström, 2017) and that sys­tem­atic lit­er­a­ture re­views risk be­ing com­pro­mised (Dadkhah et al., 2017). It has also been high­lighted that Google Scholar, in par­tic­u­lar, can be and has been ex­ploited for ma­nip­u­lat­ing the ev­i­dence base for po­lit­i­cally charged is­sues and to fuel con­spir­acy nar­ra­tives (Tripodi et al., 2023). Both con­cerns are likely to be mag­ni­fied in the fu­ture, in­creas­ing the risk of what we sug­gest call­ing ev­i­dence hack­ing—the strate­gic and co­or­di­nated ma­li­cious ma­nip­u­la­tion of so­ci­ety’s ev­i­dence base.

The au­thor­ity of qual­ity-con­trolled re­search as ev­i­dence to sup­port leg­is­la­tion, pol­icy, pol­i­tics, and other forms of de­ci­sion-mak­ing is un­der­mined by the pres­ence of un­de­clared GPT-fabricated con­tent in pub­li­ca­tions pro­fess­ing to be sci­en­tific. Due to the large num­ber of archives, repos­i­to­ries, mir­ror sites, and shadow li­braries to which they spread, there is a clear risk that GPT-fabricated, ques­tion­able pa­pers will reach au­di­ences even af­ter a pos­si­ble re­trac­tion. There are con­sid­er­able tech­ni­cal dif­fi­cul­ties in­volved in iden­ti­fy­ing and trac­ing com­puter-fab­ri­cated pa­pers (Cabanac & Labbé, 2021; Dadkhah et al., 2023; Jones, 2024), not to men­tion pre­vent­ing and curb­ing their spread and up­take.

However, as the rise of the so-called anti-vaxx move­ment dur­ing the COVID-19 pan­demic and the on­go­ing ob­struc­tion and de­nial of cli­mate change show, re­tract­ing er­ro­neous pub­li­ca­tions of­ten fu­els con­spir­a­cies and in­creases the fol­low­ing of these move­ments rather than stop­ping them. To il­lus­trate this mech­a­nism, cli­mate de­niers fre­quently ques­tion es­tab­lished sci­en­tific con­sen­sus by point­ing to other, sup­pos­edly sci­en­tific, stud­ies that sup­port their claims. Usually, these are poorly ex­e­cuted, not peer-re­viewed, based on ob­so­lete data, or even fraud­u­lent (Dunlap & Brulle, 2020). A sim­i­lar strat­egy is suc­cess­ful in the al­ter­na­tive epis­temic world of the global anti-vac­ci­na­tion move­ment (Carrion, 2018) and the per­sis­tence of flawed and ques­tion­able pub­li­ca­tions in the sci­en­tific record al­ready poses sig­nif­i­cant prob­lems for health re­search, pol­icy, and law­mak­ers, and thus for so­ci­ety as a whole (Littell et al., 2024). Considering that a per­son’s sup­port for “doing your own re­search” is as­so­ci­ated with in­creased mis­trust in sci­en­tific in­sti­tu­tions (Chinn & Hasell, 2023), it will be of ut­most im­por­tance to an­tic­i­pate and con­sider such back­fir­ing ef­fects al­ready when de­sign­ing a tech­ni­cal so­lu­tion, when sug­gest­ing in­dus­try or le­gal reg­u­la­tion, and in the plan­ning of ed­u­ca­tional mea­sures.

Solutions should be based on si­mul­ta­ne­ous con­sid­er­a­tions of tech­ni­cal, ed­u­ca­tional, and reg­u­la­tory ap­proaches, as well as in­cen­tives, in­clud­ing so­cial ones, across the en­tire re­search in­fra­struc­ture. Paying at­ten­tion to how these ap­proaches and in­cen­tives re­late to each other can help iden­tify points and mech­a­nisms for dis­rup­tion. Recognizing fraud­u­lent aca­d­e­mic pa­pers must hap­pen along­side un­der­stand­ing how they reach their au­di­ences and what rea­sons there might be for some of these pa­pers suc­cess­fully “sticking around.” A pos­si­ble way to mit­i­gate some of the risks as­so­ci­ated with GPT-fabricated schol­arly texts find­ing their way into aca­d­e­mic search en­gine re­sults would be to pro­vide fil­ter­ing op­tions for facets such as in­dexed jour­nals, gray lit­er­a­ture, peer-re­view, and sim­i­lar on the in­ter­face of pub­licly avail­able aca­d­e­mic search en­gines. Furthermore, eval­u­a­tion tools for in­dexed jour­nals could be in­te­grated into the graph­i­cal user in­ter­faces and the crawlers of these aca­d­e­mic search en­gines. To en­able ac­count­abil­ity, it is im­por­tant that the in­dex (database) of such a search en­gine is pop­u­lated ac­cord­ing to cri­te­ria that are trans­par­ent, open to scrutiny, and ap­pro­pri­ate to the work­ings of  science and other forms of aca­d­e­mic re­search. Moreover, con­sid­er­ing that Google Scholar has no real com­peti­tor, there is a strong case for es­tab­lish­ing a freely ac­ces­si­ble, non-spe­cial­ized aca­d­e­mic search en­gine that is not run for com­mer­cial rea­sons but for rea­sons of pub­lic in­ter­est. Such mea­sures, to­gether with ed­u­ca­tional ini­tia­tives aimed par­tic­u­larly at pol­i­cy­mak­ers, sci­ence com­mu­ni­ca­tors, jour­nal­ists, and other me­dia work­ers, will be cru­cial to re­duc­ing the pos­si­bil­i­ties for and ef­fects of ma­li­cious ma­nip­u­la­tion or ev­i­dence hack­ing. It is im­por­tant not to pre­sent this as a tech­ni­cal prob­lem that ex­ists only be­cause of AI text gen­er­a­tors but to re­late it to the wider con­cerns in which it is em­bed­ded. These range from a largely dys­func­tional schol­arly pub­lish­ing sys­tem (Haider & Åström, 2017) and acad­e­mi­a’s “publish or per­ish” par­a­digm to Google’s near-mo­nop­oly and ide­o­log­i­cal bat­tles over the con­trol of in­for­ma­tion and ul­ti­mately knowl­edge. Any in­ter­ven­tion is likely to have sys­temic ef­fects; these ef­fects need to be con­sid­ered and as­sessed in ad­vance and, ide­ally, fol­lowed up on.

Our study fo­cused on a se­lec­tion of pa­pers that were eas­ily rec­og­niz­able as fraud­u­lent. We used this rel­a­tively small sam­ple as a mag­ni­fy­ing glass to ex­am­ine, de­lin­eate, and un­der­stand a prob­lem that goes be­yond the scope of the sam­ple it­self, which how­ever points to­wards larger con­cerns that re­quire fur­ther in­ves­ti­ga­tion. The work of on­go­ing whistle­blow­ing ini­tia­tives, re­cent me­dia re­ports of jour­nal clo­sures (Subbaraman, 2024), or GPT-related changes in word use and writ­ing style (Cabanac et al., 2021; Stokel-Walker, 2024) sug­gest that we only see the tip of the ice­berg. There are al­ready more so­phis­ti­cated cases (Dadkhah et al., 2023) as well as cases in­volv­ing fab­ri­cated im­ages (Gu et al., 2022). Our analy­sis shows that ques­tion­able and po­ten­tially ma­nip­u­la­tive GPT-fabricated pa­pers per­me­ate the re­search in­fra­struc­ture and are likely to be­come a wide­spread phe­nom­e­non. Our find­ings un­der­line that the risk of fake sci­en­tific pa­pers be­ing used to ma­li­ciously ma­nip­u­late ev­i­dence (see Dadkhah et al., 2017) must be taken se­ri­ously. Manipulation may in­volve un­de­clared au­to­matic sum­maries of texts, in­clu­sion in lit­er­a­ture re­views, ex­plicit sci­en­tific claims, or the con­ceal­ment of er­rors in stud­ies so that they are dif­fi­cult to de­tect in peer re­view. However, the mere pos­si­bil­ity of these things hap­pen­ing is a sig­nif­i­cant risk in its own right that can be strate­gi­cally ex­ploited and will have ram­i­fi­ca­tions for trust in and per­cep­tion of sci­ence. Society’s meth­ods of eval­u­at­ing sources and the foun­da­tions of me­dia and in­for­ma­tion lit­er­acy are un­der threat and pub­lic trust in sci­ence is at risk of fur­ther ero­sion, with far-reach­ing con­se­quences for so­ci­ety in deal­ing with in­for­ma­tion dis­or­ders. To ad­dress this mul­ti­fac­eted prob­lem, we first need to un­der­stand why it ex­ists and pro­lif­er­ates.

Finding 1: 139 GPT-fabricated, ques­tion­able pa­pers were found and listed as reg­u­lar re­sults on the Google Scholar re­sults page. Non-indexed jour­nals dom­i­nate.

Most ques­tion­able pa­pers we found were in non-in­dexed jour­nals or were work­ing pa­pers, but we did also find some in es­tab­lished jour­nals, pub­li­ca­tions, con­fer­ences, and repos­i­to­ries. We found a to­tal of 139 pa­pers with a sus­pected de­cep­tive use of ChatGPT or sim­i­lar LLM ap­pli­ca­tions (see Table 1). Out of these, 19 were in in­dexed jour­nals, 89 were in non-in­dexed jour­nals, 19 were stu­dent pa­pers found in uni­ver­sity data­bases, and 12 were work­ing pa­pers (mostly in preprint data­bases). Table 1 di­vides these pa­pers into cat­e­gories. Health and en­vi­ron­ment pa­pers made up around 34% (47) of the sam­ple. Of these, 66% were pre­sent in non-in­dexed jour­nals.

Finding 2: GPT-fabricated, ques­tion­able pa­pers are dis­sem­i­nated on­line, per­me­at­ing the re­search in­fra­struc­ture for schol­arly com­mu­ni­ca­tion, of­ten in mul­ti­ple copies. Applied top­ics with prac­ti­cal im­pli­ca­tions dom­i­nate.

The 20 pa­pers con­cern­ing health-re­lated is­sues are dis­trib­uted across 20 unique do­mains, ac­count­ing for 46 URLs. The 27 pa­pers deal­ing with en­vi­ron­men­tal is­sues can be found across 26 unique do­mains, ac­count­ing for 56 URLs.  Most of the iden­ti­fied pa­pers ex­ist in mul­ti­ple copies and have al­ready spread to sev­eral archives, repos­i­to­ries, and so­cial me­dia. It would be dif­fi­cult, or im­pos­si­ble, to re­move them from the sci­en­tific record.

As ap­par­ent from Table 2, GPT-fabricated, ques­tion­able pa­pers are seep­ing into most parts of the on­line re­search in­fra­struc­ture for schol­arly com­mu­ni­ca­tion. Platforms on which iden­ti­fied pa­pers have ap­peared in­clude ResearchGate, ORCiD, Journal of Population Therapeutics and Clinical Pharmacology (JPTCP), Easychair, Frontiers, the Institute of Electrical and Electronics Engineer (IEEE), and X/Twitter. Thus, even if they are re­tracted from their orig­i­nal source, it will prove very dif­fi­cult to track, re­move, or even just mark them up on other plat­forms. Moreover, un­less reg­u­lated, Google Scholar will en­able their con­tin­ued and most likely un­la­beled dis­cov­er­abil­ity.

A word rain vi­su­al­iza­tion (Centre for Digital Humanities Uppsala, 2023), which com­bines word promi­nences through TF-IDF scores with se­man­tic sim­i­lar­ity of the full texts of our sam­ple of GPT-generated ar­ti­cles that fall into the “Environment” and “Health” cat­e­gories, re­flects the two cat­e­gories in ques­tion. However, as can be seen in Figure 1, it also re­veals over­lap and sub-ar­eas. The y-axis shows word promi­nences through word po­si­tions and font sizes, while the x-axis in­di­cates se­man­tic sim­i­lar­ity. In ad­di­tion to a cer­tain amount of over­lap, this re­veals sub-ar­eas, which are best de­scribed as two dis­tinct events within the word rain. The event on the left bun­dles terms re­lated to the de­vel­op­ment and man­age­ment of health and health­care with “challenges,” “impact,” and “potential of ar­ti­fi­cial in­tel­li­gence”emerg­ing as se­man­ti­cally re­lated terms. Terms re­lated to re­search in­fra­struc­tures, en­vi­ron­men­tal, epis­temic, and tech­no­log­i­cal con­cepts are arranged fur­ther down in the same event (e.g., “system,” “climate,” “understanding,” “knowledge,” “learning,” “education,” “sustainable”). A sec­ond dis­tinct event fur­ther to the right bun­dles terms as­so­ci­ated with fish farm­ing and aquatic med­i­c­i­nal plants, high­light­ing the pres­ence of an aqua­cul­ture clus­ter.  Here, the promi­nence of groups of terms such as “used,” “model,” “-based,” and “traditional” sug­gests the pres­ence of ap­plied re­search on these top­ics. The two events mak­ing up the word rain vi­su­al­iza­tion, are linked by a less dom­i­nant but over­lap­ping clus­ter of terms re­lated to “energy” and “water.”

The bar chart of the terms in the pa­per sub­set (see Figure 2) com­ple­ments the word rain vi­su­al­iza­tion by de­pict­ing the most promi­nent terms in the full texts along the y-axis. Here, word promi­nences across health and en­vi­ron­ment pa­pers are arranged de­scend­ingly, where val­ues out­side paren­the­ses are TF-IDF val­ues (relative fre­quen­cies) and val­ues in­side paren­the­ses are raw term fre­quen­cies (absolute fre­quen­cies).

Finding 3: Google Scholar pre­sents re­sults from qual­ity-con­trolled and non-con­trolled ci­ta­tion data­bases on the same in­ter­face, pro­vid­ing un­fil­tered ac­cess to GPT-fabricated ques­tion­able pa­pers.

Google Scholar’s cen­tral po­si­tion in the pub­licly ac­ces­si­ble schol­arly com­mu­ni­ca­tion in­fra­struc­ture, as well as its lack of stan­dards, trans­parency, and ac­count­abil­ity in terms of in­clu­sion cri­te­ria, has po­ten­tially se­ri­ous im­pli­ca­tions for pub­lic trust in sci­ence. This is likely to ex­ac­er­bate the al­ready-known po­ten­tial to ex­ploit Google Scholar for ev­i­dence hack­ing (Tripodi et al., 2023) and will have im­pli­ca­tions for any at­tempts to re­tract or re­move fraud­u­lent pa­pers from their orig­i­nal pub­li­ca­tion venues. Any so­lu­tion must con­sider the en­tirety of the re­search in­fra­struc­ture for schol­arly com­mu­ni­ca­tion and the in­ter­play of dif­fer­ent ac­tors, in­ter­ests, and in­cen­tives.

We searched and scraped Google Scholar us­ing the Python li­brary Scholarly (Cholewiak et al., 2023) for pa­pers that in­cluded spe­cific phrases known to be com­mon re­sponses from ChatGPT and sim­i­lar ap­pli­ca­tions with the same un­der­ly­ing model (GPT3.5 or GPT4): “as of my last knowl­edge up­date” and/​or “I don’t have ac­cess to real-time data” (see Appendix A). This fa­cil­i­tated the iden­ti­fi­ca­tion of pa­pers that likely used gen­er­a­tive AI to pro­duce text, re­sult­ing in 227 re­trieved pa­pers. The pa­pers’ bib­li­o­graphic in­for­ma­tion was au­to­mat­i­cally added to a spread­sheet and down­loaded into Zotero.

We em­ployed mul­ti­ple cod­ing (Barbour, 2001) to clas­sify the pa­pers based on their con­tent. First, we jointly as­sessed whether the pa­per was sus­pected of fraud­u­lent use of ChatGPT (or sim­i­lar) based on how the text was in­te­grated into the pa­pers and whether the pa­per was pre­sented as orig­i­nal re­search out­put or the AI tool’s role was ac­knowl­edged. Second, in an­a­lyz­ing the con­tent of the pa­pers, we con­tin­ued the mul­ti­ple cod­ing by clas­si­fy­ing the fraud­u­lent pa­pers into four cat­e­gories iden­ti­fied dur­ing an ini­tial round of analy­sis—health, en­vi­ron­ment, com­put­ing, and oth­ers—and then de­ter­min­ing which sub­jects were most af­fected by this is­sue (see Table 1). Out of the 227 re­trieved pa­pers, 88 pa­pers were writ­ten with le­git­i­mate and/​or de­clared use of GPTs (i.e., false pos­i­tives, which were ex­cluded from fur­ther analy­sis), and 139 pa­pers were writ­ten with un­de­clared and/​or fraud­u­lent use (i.e., true pos­i­tives, which were in­cluded in fur­ther analy­sis). The mul­ti­ple cod­ing was con­ducted jointly by all au­thors of the pre­sent ar­ti­cle, who col­lab­o­ra­tively coded and cross-checked each oth­er’s in­ter­pre­ta­tion of the data si­mul­ta­ne­ously in a shared spread­sheet file. This was done to sin­gle out cod­ing dis­crep­an­cies and set­tle cod­ing dis­agree­ments, which in turn en­sured method­olog­i­cal thor­ough­ness and an­a­lyt­i­cal con­sen­sus (see Barbour, 2001). Redoing the cat­e­gory cod­ing later based on our es­tab­lished cod­ing sched­ule, we achieved an in­ter­coder re­li­a­bil­ity (Cohen’s kappa) of 0.806 af­ter erad­i­cat­ing ob­vi­ous dif­fer­ences.

The rank­ing al­go­rithm of Google Scholar pri­or­i­tizes highly cited and older pub­li­ca­tions (Martín-Martín et al., 2016). Therefore, the po­si­tion of the ar­ti­cles on the search en­gine re­sults pages was not par­tic­u­larly in­for­ma­tive, con­sid­er­ing the rel­a­tively small num­ber of re­sults in com­bi­na­tion with the re­cency of the pub­li­ca­tions. Only the query “as of my last knowl­edge up­date” had more than two search en­gine re­sult pages. On those, ques­tion­able ar­ti­cles with un­de­clared use of GPTs were evenly dis­trib­uted across all re­sult pages (min: 4, max: 9, mode: 8), with the pro­por­tion of un­de­clared use be­ing slightly higher on av­er­age on later search re­sult pages.

To un­der­stand how the pa­pers mak­ing fraud­u­lent use of gen­er­a­tive AI were dis­sem­i­nated on­line, we pro­gram­mat­i­cally searched for the pa­per ti­tles (with ex­act string match­ing) in Google Search from our lo­cal IP ad­dress (see Appendix B) us­ing the google­search–python li­brary(Vikra­ma­ditya, 2020). We man­u­ally ver­i­fied each search re­sult to fil­ter out false pos­i­tives—re­sults that were not re­lated to the pa­per—and then com­piled the most promi­nent URLs by field. This en­abled the iden­ti­fi­ca­tion of other plat­forms through which the pa­pers had been spread. We did not, how­ever, in­ves­ti­gate whether copies had spread into SciHub or other shadow li­braries, or if they were ref­er­enced in Wikipedia.

We used de­scrip­tive sta­tis­tics to count the preva­lence of the num­ber of GPT-fabricated pa­pers across top­ics and venues and top do­mains by sub­ject. The pan­das soft­ware li­brary for the Python pro­gram­ming lan­guage (The pan­das de­vel­op­ment team, 2024) was used for this part of the analy­sis. Based on the mul­ti­ple cod­ing, pa­per oc­cur­rences were counted in re­la­tion to their cat­e­gories, di­vided into in­dexed jour­nals, non-in­dexed jour­nals, stu­dent pa­pers, and work­ing pa­pers. The schemes, sub­do­mains, and sub­di­rec­to­ries of the URL strings were fil­tered out while top-level do­mains and sec­ond-level do­mains were kept, which led to nor­mal­iz­ing do­main names. This, in turn, al­lowed the count­ing of do­main fre­quen­cies in the en­vi­ron­ment and health cat­e­gories. To dis­tin­guish word promi­nences and mean­ings in the en­vi­ron­ment and health-re­lated GPT-fabricated ques­tion­able pa­pers, a se­man­ti­cally-aware word cloud vi­su­al­iza­tion was pro­duced through the use of a word rain (Centre for Digital Humanities Uppsala, 2023) for full-text ver­sions of the pa­pers. Font size and y-axis po­si­tions in­di­cate word promi­nences through TF-IDF scores for the en­vi­ron­ment and health pa­pers (also vi­su­al­ized in a sep­a­rate bar chart with raw term fre­quen­cies in paren­the­ses), and words are po­si­tioned along the x-axis to re­flect se­man­tic sim­i­lar­ity (Skeppstedt et al., 2024), with an English Word2vec skip gram model space (Fares et al., 2017). An English stop word list was used, along with a man­u­ally pro­duced list in­clud­ing terms such as “https,” “volume,” or “years.”

Antkare, I. (2020). Ike Antkare, his pub­li­ca­tions, and those of his dis­ci­ples. In M. Biagioli & A. Lippman (Eds.), Gaming the met­rics (pp. 177–200). The MIT Press. https://​doi.org/​10.7551/​mit­press/​11087.003.0018

Barbour, R. S. (2001). Checklists for im­prov­ing rigour in qual­i­ta­tive re­search: A case of the tail wag­ging the dog? BMJ, 322(7294), 1115–1117. https://​doi.org/​10.1136/​bmj.322.7294.1115

Bom, H.-S. H. (2023). Exploring the op­por­tu­ni­ties and chal­lenges of ChatGPT in aca­d­e­mic writ­ing: A round­table dis­cus­sion. Nuclear Medicine and Molecular Imaging, 57(4), 165–167. https://​doi.org/​10.1007/​s13139-023-00809-2

Cabanac, G., & Labbé, C. (2021). Prevalence of non­sen­si­cal al­go­rith­mi­cally gen­er­ated pa­pers in the sci­en­tific lit­er­a­ture. Journal of the Association for Information Science and Technology, 72(12), 1461–1476. https://​doi.org/​10.1002/​asi.24495

Cabanac, G., Labbé, C., & Magazinov, A. (2021). Tortured phrases: A du­bi­ous writ­ing style emerg­ing in sci­ence. Evidence of crit­i­cal is­sues af­fect­ing es­tab­lished jour­nals. arXiv. https://​doi.org/​10.48550/​arXiv.2107.06751

Carrion, M. L. (2018). “You need to do your re­search”: Vaccines, con­testable sci­ence, and ma­ter­nal epis­te­mol­ogy. Public Understanding of Science, 27(3), 310–324. https://​doi.org/​10.1177/​0963662517728024

Chinn, S., & Hasell, A. (2023). Support for “doing your own re­search” is as­so­ci­ated with COVID-19 mis­per­cep­tions and sci­en­tific mis­trust. Harvard Kennedy School (HSK) Misinformation Review, 4(3). https://​doi.org/​10.37016/​mr-2020-117

Cholewiak, S. A., Ipeirotis, P., Silva, V., & Kannawadi, A. (2023). SCHOLARLY: Simple ac­cess to Google Scholar au­thors and ci­ta­tion us­ing Python (1.5.0) [Computer soft­ware]. https://​doi.org/​10.5281/​zen­odo.5764801

Dadkhah, M., Lagzian, M., & Borchardt, G. (2017). Questionable pa­pers in ci­ta­tion data­bases as an is­sue for lit­er­a­ture re­view. Journal of Cell Communication and Signaling, 11(2), 181–185. https://​doi.org/​10.1007/​s12079-016-0370-6

Dadkhah, M., Oermann, M. H., Hegedüs, M., Raman, R., & Dávid, L. D. (2023). Detection of fake pa­pers in the era of ar­ti­fi­cial in­tel­li­gence. Diagnosis, 10(4), 390–397. https://​doi.org/​10.1515/​dx-2023-0090

Dunlap, R. E., & Brulle, R. J. (2020). Sources and am­pli­fiers of cli­mate change de­nial. In D. C. Holmes & L. M. Richardson (Eds.), Research hand­book on com­mu­ni­cat­ing cli­mate change (pp. 49–61). Edward Elgar Publishing. https://​doi.org/​10.4337/​9781789900408.00013

Fares, M., Kutuzov, A., Oepen, S., & Velldal, E. (2017). Word vec­tors, reuse, and replic­a­bil­ity: Towards a com­mu­nity repos­i­tory of large-text re­sources. In J. Tiedemann & N. Tahmasebi (Eds.), Proceedings of the 21st Nordic Conference on Computational Linguistics (pp. 271–276). Association for Computational Linguistics. https://​aclan­thol­ogy.org/​W17-0237

Google Scholar Help. (n.d.). Inclusion guide­lines for web­mas­ters. https://​scholar.google.com/​intl/​en/​scholar/​in­clu­sion.html

Gusenbauer, M., & Haddaway, N. R. (2020). Which aca­d­e­mic search sys­tems are suit­able for sys­tem­atic re­views or meta-analy­ses? Evaluating re­trieval qual­i­ties of Google Scholar, PubMed, and 26 other re­sources. Research Synthesis Methods, 11(2), 181–217.  https://​doi.org/​10.1002/​jrsm.1378

Haider, J., & Åström, F. (2017). Dimensions of trust in schol­arly com­mu­ni­ca­tion: Problematizing peer re­view in the af­ter­math of John Bohannon’s “Sting” in sci­ence. Journal of the Association for Information Science and Technology, 68(2), 450–467. https://​doi.org/​10.1002/​asi.23669

Huang, J., & Tan, M. (2023). The role of ChatGPT in sci­en­tific com­mu­ni­ca­tion: Writing bet­ter sci­en­tific re­view ar­ti­cles. American Journal of Cancer Research, 13(4), 1148–1154. https://​www.ncbi.nlm.nih.gov/​pmc/​ar­ti­cles/​PM­C10164801/

Jones, N. (2024). How jour­nals are fight­ing back against a wave of ques­tion­able im­ages. Nature, 626(8000), 697–698. https://​doi.org/​10.1038/​d41586-024-00372-6

Kitamura, F. C. (2023). ChatGPT is shap­ing the fu­ture of med­ical writ­ing but still re­quires hu­man judg­ment. Radiology, 307(2), e230171. https://​doi.org/​10.1148/​ra­diol.230171

Littell, J. H., Abel, K. M., Biggs, M. A., Blum, R. W., Foster, D. G., Haddad, L. B., Major, B., Munk-Olsen, T., Polis, C. B., Robinson, G. E., Rocca, C. H., Russo, N. F., Steinberg, J. R., Stewart, D. E., Stotland, N. L., Upadhyay, U. D., & Ditzhuijzen, J. van. (2024). Correcting the sci­en­tific record on abor­tion and men­tal health out­comes. BMJ, 384, e076518. https://​doi.org/​10.1136/​bmj-2023-076518

Lund, B. D., Wang, T., Mannuru, N. R., Nie, B., Shimray, S., & Wang, Z. (2023). ChatGPT and a new aca­d­e­mic re­al­ity: Artificial Intelligence-written re­search pa­pers and the ethics of the large lan­guage mod­els in schol­arly pub­lish­ing. Journal of the Association for Information Science and Technology, 74(5), 570–581. https://​doi.org/​10.1002/​asi.24750

Martín-Martín, A., Orduna-Malea, E., Ayllón, J. M., & Delgado López-Cózar, E. (2016). Back to the past: On the shoul­ders of an aca­d­e­mic search en­gine gi­ant. Scientometrics, 107, 1477–1487. https://​doi.org/​10.1007/​s11192-016-1917-2

Martín-Martín, A., Thelwall, M., Orduna-Malea, E., & Delgado López-Cózar, E. (2021). Google Scholar, Microsoft Academic, Scopus, Dimensions, Web of Science, and OpenCitations’ COCI: A mul­ti­dis­ci­pli­nary com­par­i­son of cov­er­age via ci­ta­tions. Scientometrics, 126(1), 871–906. https://​doi.org/​10.1007/​s11192-020-03690-4

Simon, F. M., Altay, S., & Mercier, H. (2023). Misinformation re­loaded? Fears about the im­pact of gen­er­a­tive AI on mis­in­for­ma­tion are overblown. Harvard Kennedy School (HKS) Misinformation Review, 4(5). https://​doi.org/​10.37016/​mr-2020-127

Skeppstedt, M., Ahltorp, M., Kucher, K., & Lindström, M. (2024). From word clouds to Word Rain: Revisiting the clas­sic word cloud to vi­su­al­ize cli­mate change texts. Information Visualization, 23(3), 217–238. https://​doi.org/​10.1177/​14738716241236188

Stokel-Walker, C. (2024, May 1.). AI Chatbots Have Thoroughly Infiltrated Scientific Publishing. Scientific American. https://​www.sci­en­tifi­camer­i­can.com/​ar­ti­cle/​chat­bots-have-thor­oughly-in­fil­trated-sci­en­tific-pub­lish­ing/

Subbaraman, N. (2024, May 14). Flood of fake sci­ence forces mul­ti­ple jour­nal clo­sures: Wiley to shut­ter 19 more jour­nals, some tainted by fraud. The Wall Street Journal. https://​www.wsj.com/​sci­ence/​aca­d­e­mic-stud­ies-re­search-pa­per-mills-jour­nals-pub­lish­ing-f5a3d4bc

Thorp, H. H. (2023). ChatGPT is fun, but not an au­thor. Science, 379(6630), 313–313. https://​doi.org/​10.1126/​sci­ence.adg7879

Tripodi, F. B., Garcia, L. C., & Marwick, A. E. (2023). ‘Do your own re­search’: Affordance ac­ti­va­tion and dis­in­for­ma­tion spread. Information, Communication & Society, 27(6), 1212–1228. https://​doi.org/​10.1080/​1369118X.2023.2245869

This re­search has been sup­ported by Mistra, the Swedish Foundation for Strategic Environmental Research, through the re­search pro­gram Mistra Environmental Communication (Haider, Ekström, Rödl) and the Marcus and Amalia Wallenberg Foundation [2020.0004] (Söderström).

The re­search de­scribed in this ar­ti­cle was car­ried out un­der Swedish leg­is­la­tion. According to the rel­e­vant EU and Swedish leg­is­la­tion (2003:460) on the eth­i­cal re­view of re­search in­volv­ing hu­mans (“Ethical Review Act”), the re­search re­ported on here is not sub­ject to au­tho­riza­tion by the Swedish Ethical Review Authority (“etikprövningsmyndigheten”) (SRC, 2017).

This is an open ac­cess ar­ti­cle dis­trib­uted un­der the terms of the Creative Commons Attribution License, which per­mits un­re­stricted use, dis­tri­b­u­tion, and re­pro­duc­tion in any medium, pro­vided that the orig­i­nal au­thor and source are prop­erly cred­ited.

All data needed to repli­cate this study are avail­able at the Harvard Dataverse: https://​doi.org/​10.7910/​DVN/​WU­VD8X

The au­thors wish to thank two anony­mous re­view­ers for their valu­able com­ments on the ar­ti­cle man­u­script as well as the ed­i­to­r­ial group of Harvard Kennedy School (HKS) Misinformation Review for their thought­ful feed­back and in­put.

The al­go­rith­mic knowl­edge gap within and be­tween coun­tries: Implications for com­bat­ting mis­in­for­ma­tion

While un­der­stand­ing how so­cial me­dia al­go­rithms op­er­ate is es­sen­tial to pro­tect one­self from mis­in­for­ma­tion, such un­der­stand­ing is of­ten un­evenly dis­trib­uted. This study ex­plores the al­go­rith­mic knowl­edge gap both within and be­tween coun­tries, us­ing na­tional sur­veys in the United States (N = 1,415), the United Kingdom (N = 1,435), South Korea (N = 1,798), and Mexico (N = 784).

Keep Reading

Stochastic lies: How LLM-powered chat­bots deal with Russian dis­in­for­ma­tion about the war in Ukraine

Research on dig­i­tal mis­in­for­ma­tion has turned its at­ten­tion to large lan­guage mod­els (LLMs) and their han­dling of sen­si­tive po­lit­i­cal top­ics. Through an AI au­dit, we an­a­lyze how three LLM-powered chat­bots (Perplexity, Google Bard, and Bing Chat) gen­er­ate con­tent in re­sponse to the prompts linked to com­mon Russian dis­in­for­ma­tion nar­ra­tives about the war in Ukraine.

Keep Reading

AI from The Basement: My lat­est side pro­ject, a ded­i­cated LLM server pow­ered by 8x RTX 3090 Graphic Cards, boast­ing a to­tal of 192GB of VRAM. I built this with run­ning Meta’s Llamma-3.1 405B in mind.

This blog­post was orig­i­nally posted on my LinkedIn pro­file in July 2024.

Backstory: Sometime in March I found my­self strug­gling to keep up with the mere 48GB of VRAM I had been re­ly­ing on for al­most a year in my LLMs ex­per­i­men­ta­tions. So, in a geeky-yet-styl­ish way, I de­cided to spend my money to build this thing of beauty. Questions swirled: Which CPU/Platform to buy? Does mem­ory speed re­ally mat­ter? And why the more PCIe Lanes we have the bet­ter? Why 2^n num­ber of GPUs mat­ter in multi-GPU node setup (Tensor Parallelism, any­one?) How many GPUs, and how can I get all the VRAM in the world? Why are Nvidia cards so ex­pen­sive and why did­n’t I in­vest in their stock ear­lier? What in­fer­ence en­gine to use (hint: it’s not just llama.cpp and not al­ways the most well-doc­u­mented op­tion)?

After so many hours of re­search, I de­cided on the fol­low­ing plat­form:

* Asrock Rack ROMED8-2T moth­er­board with 7x PCIe 4.0x16 slots and 128 lanes of PCIe

* A mere trio of 1600-watt power sup­ply units to keep every­thing run­ning smoothly

* 8x RTX 3090 GPUs with 4x NVLinks, en­abling a blis­ter­ing 112GB/s data trans­fer rate be­tween each pair

Now that I kinda have every­thing in or­der, I’m work­ing on a se­ries of blog posts that will cover the en­tire jour­ney, from build­ing this be­he­moth to avoid­ing costly pit­falls. Topics will in­clude:

* The chal­lenges of as­sem­bling this sys­tem: from drilling holes in metal frames and adding 30amp 240volt break­ers, to bend­ing CPU socket pins (don’t try this at home, kids!).

* Why PCIe Risers suck and the im­por­tance of us­ing SAS Device Adapters, Redrivers, and Retimers for er­ror-free PCIe con­nec­tions.

* NVLink speeds, PCIe lanes band­width and VRAM trans­fer speeds, and Nvidia’s de­ci­sion to block P2P na­tive PCIe band­width on the soft­ware level.

* Benchmarking in­fer­ence en­gines like TensorRT-LLM, vLLM, and Aphrodite Engine, all of which sup­port Tensor Parallelism.

* Training and fine-tun­ing your own LLM.

P. S. I’m sit­ting here star­ing at those GPUs, and I just can’t help but think how wild tech progress has been. I re­mem­ber be­ing so ex­cited to get a 60GB HDD back in 2004. I mean, all the movies and games I could store?! Fast for­ward 20 years, and now I’ve got more than triple that stor­age ca­pac­ity in just one ma­chine’s graphic cards… It makes me think, what will we be do­ing in an­other 20 years?!

Anyway, that’s why I’m do­ing this pro­ject. I wanna help cre­ate some of the cool stuff that’ll be around in the fu­ture. And who knows, maybe some­one will look back on my work and be like “haha, re­mem­ber when we thought 192GB of VRAM was a lot?”

Sometimes I just want to put pix­els on a screen. I don’t want to think about SDL this or OpenGL that—I just want to draw my pixel buffer and be done.

fen­ster, a tiny 2D can­vas li­brary by Serge Zaitsev, does just that. It’s a tiny drop-in header-only C/C++ file that weighs no more than 400 LOC of pretty read­able code. It works with WinAPI, Cocoa, and X11. And it han­dles key­board and mouse in­put, too!

Sometimes I want to do just a lit­tle more than draw pix­els—maybe have a menu, some but­tons, ren­der text—and I don’t want to com­pletely DIY but I still don’t want to think about SDL.

Fortunately, mi­croui by rxi ex­ists and han­dles the trans­la­tion from GUI el­e­ments into a sim­ple re­tar­getable draw­ing byte­code. It’s sim­i­larly a small, drop-in li­brary, weigh­ing only 1500 LOC.

Unfortunately, the demo pro­gram uses SDL as a back­end for the byte­code. I’d been mean­ing to see if I could in­stead use fen­ster but un­der­stand­ing what a “quad” was or what “glScissor” did seemed in­tim­i­dat­ing. The pro­ject went nowhere.

Then, as usual, Kartik and I had a small ar­gu­ment and that re­sulted in us cre­at­ing the fen­ster back­end for mi­croui! I sent him a skele­ton to show what I wanted to do and he did most of the heavy lift­ing for the OpenGL-like parts.

The re­sult is a less than 250 LOC file that binds mi­croui to fen­ster. It’s in­spired by the SDL ren­derer demo, but with a cou­ple of added func­tions to ab­stract away keys and mouse but­tons. It’s hacky and there’s some stuff we still don’t un­der­stand, but it works! And by “works” I mean draws the ex­pected demo win­dows, han­dles mouse hover and click, and han­dles key­board in­put.

* How to de­ter­mine when to ren­der from the tex­ture and when from the pro­vided

draw­ing com­mand’s color

* Mod keys like so that, for ex­am­ple, + ren­ders

Check it out here. It’s de­signed to all be dropped di­rectly into your pro­ject.

This blog is open source.

See an er­ror? Go ahead and

pro­pose a change.

He built the glider, in­ci­den­tally, with a gift of $5 sent to him by an American Civil War vet­eran af­ter a school es­say he’d writ­ten about Robert E. Lee was pub­lished in the lo­cal pa­per.  The war, af­ter all, had ended only 44 years ear­lier.

In 1946, by which time he’d be­come a no­table writer of sci­ence fic­tion, he pub­lished a story called ‘A Logic named Joe’, which de­scribed a global com­puter net­work with servers and ter­mi­nals, that starts giv­ing peo­ple the in­for­ma­tion that it thinks they ought to know as op­posed to wait­ing for them to search for it - the Singularity, if you like, or maybe just Alexa. He also, as I re­call, pre­dicted re­al­ity TV some­where.

And yet, de­spite pre­dict­ing half of our world, as a fa­ther in the 1950s he could not imag­ine why his daugh­ter - my mother - wanted to work.

This is­n’t an un­com­mon ob­ser­va­tion - plenty of peo­ple have pointed out that vin­tage scifi is full of rock­et­ships but all the pi­lots are men. 1950s scifi shows 1950s so­ci­ety, but with ro­bots. Mean­while, the in­ter­stel­lar lin­ers have pa­per tick­ets, that you queue up to buy. With fun­da­men­tal tech­nol­ogy change, we don’t so much get our pre­dic­tions wrong as make pre­dic­tions about the wrong things. (And, of course, we now have nei­ther trol­leys nor per­sonal glid­ers.)

I was re­minded of this photo re­cently when I came across a RAND ‘long-range fore­cast­ing’ study, from 1964. The au­thors polled a range of ex­perts on what the key de­vel­op­ments in com­ing decades would be and when they’d hap­pen. Fields ad­dressed in­cluded space flight and med­i­cine, but the most in­ter­est­ing in this con­text is what was then called ‘automation’ (the past tended to de­scribe as ‘automatic’ what we would now call ‘computers’). The dou­ble-page spread be­low shows the con­clu­sions (click to en­large).

Singletons look­ing to shack up with their soul­mates on­line have re­lied on two key routes in the past decade or so: take your chance on dat­ing apps, or be­friend as many mu­tu­als as pos­si­ble on so­cial me­dia, in the hope that you find the one.

But some have found a third way, us­ing ser­vices such as Goodreads and Strava to meet part­ners with whom they hope to spend the rest of their lives. Those cou­ples proved to be trend­set­ters. So-called hobby apps — built around ac­tivites such as run­ning, read­ing or movie-go­ing — are hav­ing a mo­ment, and not just for love.

It’s all part of a broader move­ment as peo­ple grow tired of the “digital town square” of­fered on Twitter/X and other so­cial me­dia plat­forms. At a time when many are aban­don­ing Elon Musk’s so­cial net­work over his at­ti­tude to “free speech” (which some see as “amplifying hate”), com­pet­ing apps such as Bluesky and Threads are hav­ing a resur­gence in users.

Whereas some users are switch­ing to Twitter repli­cas, oth­ers are seek­ing refuge in apps that promise to con­nect them to peo­ple with whom they have com­mon in­ter­ests. Running app Strava has seen user num­bers grow 20% in a year, ac­cord­ing to dig­i­tal mar­ket in­tel­li­gence firm Sensor Tower. That suc­cess has led it to add a mes­sag­ing tool for users to keep in touch, along­side doc­u­ment­ing their work­outs. Knitting so­cial net­work Ravelry, which is ac­cessed through a num­ber of third-party apps, has more than 9 mil­lion users. Goodreads has clocked up more than 150 mil­lion mem­bers.

Letterboxd, a film com­pletist’s dream app, where you can tick off the lat­est movies you’ve seen, and re­view and rate them, along­side other cinephiles and the oc­ca­sional fa­mous ac­tor or di­rec­tor, has gone from hav­ing 1.8 mil­lion users world­wide in March 2020 to more than 14 mil­lion users this sum­mer. The app has grown its monthly ac­tive user­base 55% in a year, ac­cord­ing to Sensor Tower.

“We re­ally work hard on the tone and voice of every­thing we do, from com­mu­nity pol­icy through to ed­i­to­r­ial through to our so­cial, to guide folks in terms of how we want them to be around Letterboxd,” says Gemma Gracewood, the ap­p’s ed­i­tor-in-chief. “We talk about movies.”

And that’s re­fresh­ing in a world where pol­i­tics and cul­ture wars are be­ing pushed at us through al­go­rithms. “Social me­dia users have been turn­ing to­wards niche apps and spaces for a long time,” says Jess Maddox, as­sis­tant pro­fes­sor in dig­i­tal me­dia at the University of Alabama. “Paradoxically, as ma­jor plat­forms such as Twitter/X, YouTube, TikTok and Instagram push more al­go­rith­mi­cally cu­rated feeds, users may be less ex­posed to the con­tent they want to see.”

The cosy na­ture of hobby apps, and the way they’re set up to share pas­sions and pas­times, means they’re an al­to­gether gen­tler place than the rough-and-tum­ble racism you can en­counter on X with an er­rant tap. “It’s a way for peo­ple to con­nect via com­mon in­ter­ests,” says Dr Carolina Are, a so­cial me­dia re­searcher at the Centre for Digital Citizens at Northumbria University. It all means that the apps can spend less time, ef­fort and money on con­tent mod­er­a­tion — as­sum­ing that ci­vil­ity will be supreme — and in­stead fo­cus on mak­ing the over­all ex­pe­ri­ence bet­ter.

“The thing about Letterboxd is there is­n’t a ‘central town square’ like there is on X; it’s a very sin­gle-chan­nel con­ver­sa­tion,” says Gracewood. Comments hap­pen in-line — sim­i­lar to those on the Guardian and Observer web­sites — mean­ing that it’s less pos­si­ble to per­for­ma­tively re­post con­tent into a main feed in or­der to en­cour­age a pile-on. Similar sit­u­a­tions ex­ist on plat­forms such as Goodreads and Strava, where it’s pos­si­ble to com­mu­ni­cate with and mes­sage oth­ers, but not to pub­licly shame them eas­ily.

Because hobby apps are nicer places to ex­ist, peo­ple spend more time on them — and they can even­tu­ally turn into ser­vices that are more than ad­ver­tised. That in­cludes find­ing like-minded peo­ple with whom you’d want to spend your time ro­man­ti­cally.

One rea­son that peo­ple may be start­ing to find love on apps not ex­plic­itly de­signed for that pur­pose is be­cause the ex­pec­ta­tions are lower — and as such, the at­mos­phere is less sex­u­ally charged. “Dating apps seem like a dat­ing su­per­mar­ket, and some­thing you have to do if you want to have some kind of con­nec­tion,” says Are.

She points out that while dat­ing apps are try­ing to shed their shal­low rep­u­ta­tion as places to hook up, they still lead with gi­ant pic­tures of users to gauge com­pat­i­bil­ity. “A lot of peo­ple are be­com­ing quite dis­il­lu­sioned with the fact you’re judged on looks,” she says. “In gen­eral, there is a bit of a dis­il­lu­sion­ment with plat­form-fa­cil­i­tated dat­ing cul­ture, be­cause it seems very im­per­sonal. It’s all fa­cil­i­tated by an al­go­rithm. And it seems not to serve peo­ple very well.”

Hobby apps’ gain is dat­ing apps’ loss, based on re­cent fi­nan­cial fig­ures from Match Group, the com­pany that op­er­ates the best-known dat­ing ser­vices, in­clud­ing Tinder and Hinge. From an October 2021 peak of more than $175 a share, Match is now trad­ing at nearer $36 a share. The firm an­nounced job cuts of 6% in July due to dwin­dling pay­ing users.

But the rot is­n’t lim­ited to the big beasts in the game. An analy­sis of the top 200 dat­ing and so­cial con­nec­tion apps by Deutsche Bank — en­ti­tled Dating: The Dating Debate — Have We Hit Saturation Levels? — sug­gests that global down­loads have plateaued.

It also helps that hobby apps feel like a more co­he­sive, kinder com­mu­nity. That’s not just be­cause the peo­ple are kinder: Letterboxd has a set of mod­er­a­tors who are tasked with tak­ing a “zero tol­er­ance” ap­proach to overt or coded hate speech, racism, ho­mo­pho­bia, white su­premacy, trans­pho­bia or any other mar­gin­al­is­ing at­ti­tudes.

Letterboxd has fewer than 10 staff mod­er­at­ing con­tent, says Gracewood, and gen­er­ally they don’t need to in­ter­vene of­ten. “I can’t speak to whether we’ve ben­e­fited from cul­tural and mis­sion shifts at other so­cial me­dia plat­forms, but I can say that from day one, we have al­ways been very, very con­cerned with what cre­at­ing a com­mu­nity on­line looks like, and how to keep it feel­ing free, good and nice.”

Whether that light-touch ap­proach com­pared with so­cial me­dia apps — TikTok em­ploys 40,000 con­tent mod­er­a­tors world­wide, while Meta re­port­edly has 15,000 — will last is yet to be seen. “It seems like every app is born, is­n’t mod­er­ated, then some­thing bad hap­pens and it gets heav­ily mod­er­ated,” says Are. “So maybe they [hobby apps] will have that tra­jec­tory as well.”

Chris Stokel-Walker is the au­thor of TikTok Boom: China’s Dynamite App and the Superpower Race for Social Media (Canbury Press, £9.99). To sup­port the Guardian and Observer, or­der your copy at guardian­book­shop.com. Delivery charges may ap­ply

When I peer into the far reaches of sci­ence fic­tional imag­i­na­tion, way out be­yond the easy extrap­o­la­tions and con­sen­sus fu­tures, be­yond the Blade Runners and the Star Treks, the name that looms largest is Iain M. Banks.

For those unfa­miliar with his work in this genre, I’ll tell you a lit­tle bit about the Culture nov­els and rec­om­mend a read­ing ap­proach. Then, for Banks be­gin­ners and de­voted Culturephiles alike, I’ll ex­plain why his fu­ture means so much to me.

What is the Culture? A civ­i­liza­tion. An agree­ment. The sub­ject of a collec­tion of books, writ­ten across decades, which of­fer clues and sug­ges­tions, glances and re­flec­tions. A big part of the fun of read­ing those books is assem­bling your own mo­saic. Here’s mine:

The Culture is a space­far­ing, free­wheeling ad­mix­ture of an­ar­chism and so­cial­ism. In most ways, it promises its cit­i­zens rad­i­cal, breath­taking free­dom … but in a few other ways, it re­quires their sub­mis­sion — to super­human sys­tems of plan­ning and man­u­fac­ture, the Culture’s in­ef­fa­ble Minds.

The Culture is a utopia: a fu­ture you might ac­tu­ally want to live in. It of­fers a co­her­ent po­lit­i­cal vi­sion. This is­n’t sub­tle or al­le­gor­i­cal; on the page, cit­i­zens of the Culture very fre­quently artic­u­late and de­fend their val­ues. (Their enthu­siasm for their own pol­i­tics is consid­ered an­noy­ing by most other civ­i­liza­tions.)

Coherent po­lit­i­cal vi­sion does­n’t re­quire a lot, just some sense of this is what we ought to do, yet it is ab­sent from plenty of sci­ence fic­tion that dwells only in the realm of the cau­tion­ary tale.

I don’t have much pa­tience left for that genre. I mean … we have been, at this point, am­ply cau­tioned.

Vision, on the other hand: I can’t get enough.

The Culture nov­els aren’t con­nected by an over­ar­ching plot, and there is no canon­i­cal read­ing or­der. For all my ap­pre­ci­a­tion: I have not even read all of them! If you search on­line, you’ll find plenty of pro­posed ap­proaches.

Here is mine, which is un­ortho­dox; call it a recipe for en­joy­ing the Culture. It pro­ceeds in three stages:

I very strongly be­lieve new read­ers ought to start with Player of Games. It is a capti­vating novel in its own right, and its intro­duc­tion to the Culture is smooth, al­most stealthy. It’s also the book I started with, and ob­vi­ously It Worked for Me, so I can’t help but rec­om­mend the same on-ramp.

For your sec­ond foray, you can choose ba­si­cally at ran­dom. I like Matter and Surface Detail. I do not like Consider Phlebas.

Here is the un­ortho­dox part: I don’t think it’s nec­es­sary, or even de­sir­able, to have read more than a cou­ple of Culture nov­els be­fore turn­ing to A Few Notes on the Culture, the post from Iain M. Banks that just … lays it all out there.

A Few Notes on the Culture is, for me, THE thrilling Culture doc­u­ment. It helps that it’s this odd sort of web samiz­dat — you are al­ways read­ing a mir­rored copy on some ran­dom web­site. The orig­i­nal was posted to a Usenet news­group in 1994!

I should say, I don’t gen­er­ally love “raw world­build­ing” of this kind — RPG source­book ma­te­r­ial. This doc­u­ment is a bril­liant ex­cep­tion, be­cause the ideas are so big, so fresh, and so confi­dently artic­u­lated; and of course be­cause it’s Iain M. Banks be­hind them, his voice inim­itable, wry and win­ning.

Why not sim­ply be­gin with A Few Notes on the Culture, if it’s so great? Well, it IS raw world­build­ing, and even the best ex­em­plar of that genre ben­e­fits from nar­ra­tive con­text. Read it on its own, and it’s a wonky thought ex­per­i­ment. Read it af­ter a cou­ple of nov­els, and it’s a back­stage pass.

You ought to meet a char­ac­ter or two — hear from a few of the rol­lick­ing Minds, learn their won­der­ful names — be­fore you go be­hind the cur­tain.

There are, in sci­ence fic­tion, sev­eral close peers to Iain M. Banks, at least in terms of the scale of their sto­ry­telling. I think in partic­ular of Olaf Stapledon, his Last and First Men, which gal­lops across mil­lions of years; and of Cixin Liu, his se­ries start­ing with The Three-Body Problem, which bumps up against the death of the uni­verse. I like both of these au­thors, but/​and their fu­tures are cold and grim. You would­n’t call ei­ther one utopia.

So, I suppose it’s not just the scale of Iain M. Banks’s sto­ries that I want to praise, but their warmth. His megas­truc­tures over­flow with ap­peal­ing char­ac­ters pur­su­ing inter­esting pro­jects. Their voices are ironic and funny.

There’s no utopia with­out irony and hu­mor; this fact re­ally nar­rows the field.

In my novel Mr. Penumbra’s 24-Hour Bookstore, the am­bi­tious and bril­liant Kat Potente asks:

Kat straight­ens her shoul­ders. “Okay, we’re go­ing to play. To start, imag­ine the fu­ture. The good fu­ture. No nu­clear bombs. Pretend you’re a sci­ence fic­tion writer.”

“Go fur­ther. What’s the good fu­ture af­ter that?”

“Star Trek. Transporters. You can go any­where.”

Kat shakes her head. “It’s re­ally hard. And that’s, what, a thou­sand years? What comes af­ter that? What could pos­si­bly come af­ter that? Imag­i­na­tion runs out. But it makes sense, right? We prob­a­bly just imag­ine things based on what we al­ready know, and we run out of analo­gies in the thirty-first cen­tury.”

I’m try­ing hard to imag­ine an av­er­age day in the year 3012. I can’t even come up with a half-de­cent scene. Will peo­ple live in build­ings? Will they wear clothes? My imag­i­na­tion is al­most phys­i­cally strain­ing.

Fingers of thought are rak­ing the space be­hind the cush­ions, look­ing for loose ideas, find­ing noth­ing.

I have of­ten de­scribed imag­i­na­tion as a mus­cle — one that, like any other mus­cle, can be de­vel­oped.

Steady ex­po­sure to Star Trek gives you a mi­nor work­out, for sure; the bump of an imag­i­na­tive bi­cep. But there’s much fur­ther to go. You can read your way into some of it, and some of it, you have to dream up for your­self.

Even among elite ath­letes, there must be ti­tans: Schwarzeneg­gers strid­ing across the stage. (I con­jure body­builders be­cause I like the idea of these imag­i­na­tive mus­cles BULGING.) Iain M. Banks, who died in 2013, way too young, was Mr. Universe. Here was a great writer, sure; but here was an imag­i­na­tion un­matched.

He sim­ply pushed fur­ther and thought big­ger.

For me, the Culture is the stan­dard, so, for a long time, the chal­lenge has been im­plicit: can you, Sloan, imag­ine on that scale? And not just tech­ni­cally, but hu­manely — with warmth and irony?

I don’t get to Culture scale in Moonbound, but the plan — and there is a plan — is to ratchet up book by book, so my no­tional se­ries can cul­mi­nate in a feat that takes se­ri­ously the scale of the uni­verse, as we now un­der­stand it.

It’s eas­ier to write the de­feat than the vic­tory, is­n’t it? Easier to write the fail­ure than the suc­cess. For some rea­son, the suc­cess seems like it might be … bor­ing.

Iain M. Banks shows us the Culture har­ness­ing mat­ter and en­ergy on incred­ible scales. He tells us that cit­i­zens of the Culture live for hun­dreds of years; that death is gen­er­ally a choice. In these books, !

At first glance, this seems fa­tal to plot. Endless en­ergy, im­mor­tal­ity … aren’t these the GOALS of the story? Are we just talk­ing about heaven here? Heaven: which ought to be oc­cluded, un­know­able, un­sayable. Heaven: be­cause it’s bor­ing.

Turns out, no, it’s not bor­ing at all. Plot gal­lops on, even at the outer lim­its of mat­ter and en­ergy. Even at the far reaches of free­dom, the sto­ries are only just be­gin­ning.

In the writerly ret­i­cence to dra­ma­tize abun­dance, I detect hu­mil­ity — it re­quires se­ri­ous imag­i­na­tive mus­cle, be­yond what most folks are work­ing with — and I detect also cow­ardice. What if my utopia is­n’t good enough? What if I say, “this is gonna be so great”, and read­ers re­ply, “eh, does­n’t sound that great”?

Easier to con­jure some tyrant ma­chines, some fast-spread­ing plagues. Everybody can agree on those.

Plenty of read­ers might in­deed read the Culture nov­els and say, “eh, does­n’t sound that great”. The point is, there is some­thing here to in­spect, and con­sider, and, sure, even re­ject. In these nov­els, Iain M. Banks hoists the imag­i­na­tive bur­den. He twirls it in the air. His mus­cles bulge. It’s amaz­ing to be­hold.

P. S. Culturephiles will note I’ve men­tioned only obiquely the Culture’s great­est aes­thetic bounty, the names of its ships. That’s be­cause they are ac­tu­ally not funny or inter­esting un­til you step in­side the magic cir­cle of the books, and be­gin to in­tuit the rules of the game. Of course, af­ter you’ve read one or two Culture nov­els, learn­ing new names be­comes a large and grow­ing frac­tion of the fun!

Having your photo taken at the Appalachian Trail Conservancy (ATC) head­quar­ters in Harpers Ferry, West Virginia, has be­come a stan­dard rit­ual for those hik­ers in­tend­ing on walk­ing the en­tire A. T. One of the func­tions of the ATC, as the lead or­ga­ni­za­tion in man­ag­ing and pro­tect­ing the A.T., is to main­tain the of­fi­cial 2,000-miler reg­istry of all those who have com­pleted the A.T. Therefore, hav­ing a photo taken here makes many hik­ers feel as though their hikes have gained of­fi­cial recog­ni­tion.

Thanks to a gen­er­ous grant from the Quimby Family Foundation and the work of ded­i­cated vol­un­teers of the A. T. Museum un­der the lead­er­ship of Terry Harley Wilson, both scanned ver­sions of the ear­lier Polaroid pho­tos and the more re­cent dig­i­tal pho­tos can now be viewed on-line from any­where in the world. When the A.T. Museum opened in Pennsylvania’s Pine Grove Furnace State Park in 2010, dig­i­tal ver­sions of the pho­tos be­came avail­able there too. (read ex­panded text)