Journal tags: language

Bhí me i Londain an deireadh seachtaine seo caite mar gheall ar an Gaeltacht cois Tamaise. Cúpla lá iontach ba ea iad!

Bhí na ranganna ar siúl Dé Sathairn agus Dé Domhnaigh, ceithre huaire an chloig gach lá, i gColáiste na Rí. Bhí ceithre leibhéal ann—tosathóiri, meanleibhéal-iseal, meanleibhéal-ard, agus an ardleibhéal. Bhí gach rang lán le foghlaimeoirí.

Roghnaigh mé an rang meanleibhéal-ard agus bhí an leibhéal foirfe. D’fhreastail Jessica ar an rang tosathóirí agus dúirt sí go raibh a mhúinteor iontach deas freisin.

Bhraith sé aisteach a bheith ag labhairt Gaeilge i lár na phriomhcathair Shasana, ach bhain mé go leor sult as!

Roimh na ranganna, bhí imeachta ar siúl ar an Embasáid na hÉireann ar an tráthnóna Dé hAoine; taifeadadh beo ar an bpodchraoladh How To Gael le Louis Cantillon agus Doireann ní Ghlacáin. Éistim leis an podchraoladh, mar sin thapaigh mé an deis iad a fheiceáll beo. Mná cliste agus greanmhar is ea iad!

Bhí imeachta eile ar siúl ar an tráthnóna De Sathairn ach ní raibh mé ann. Chuaigh mé go dtí an teach tabhairne Brendan The Navigator i Highgate—i bhfad ó croílár na caithreach!—mar gheall go raibh seisiún ceoil ann. Seisiún iontach iontach deas a bhí ann le daoine fáiltiúil agus go leor poirt áille.

Beidh mé ar ais!

In the latest issue of Matthias’s excellent Own Your Web series, he describes the recent betrayal by Google:

The search engine no longer says “here, go read what this person wrote.” It now says “here, I’ve already read it for you.” The contract is broken.

He’s absolutely right.

But…

Have you ever clicked on a result from a search engine? Unless you’re lucky enough to land on a nice personal website, you’re more than likely to be confronted with pop-ups to allow tracking, or a desparate plea to subscribe to a newsletter, or just rubbish ads all accompanied by a slow page loading somewhere in the mix.

Don’t get me wrong. I’m not saying that what Google is doing is okay. But let’s not pretend that everything indexed by Google is just fine and dandy for people to visit.

And of course the main reason why websites are so terrible is because they’ve tied their business model to heaps of behavioral advertising driven by invasive tracking courtesy of …Google.

This reminds me of AMP. Remember Google AMP? It was a terrible solution to a real problem. Web pages were (and still are) bloated and slow. The correct solution would be to encourage people to fix that, but instead Google mandated a proprietary format for your content that had to be hosted on their servers.

AMP was a disaster, both in practical terms and in the reputational damage it did to Google’s developer relations.

Now they’re doing it again, powerwashing away any goodwill they ever had with site owners. Now Google doesn’t even send search engine traffic to the websites that host the ads that Google encouraged people to put on every page.

It’s almost as if Google is a company so large and with so many competing interests that it now suffers from an incurable split personality disorder.

Personally I think they’re missing a trick. They should be using “AI” summaries as a stick.

If your site is slow, or filled with user-hostile annoyances then it should be cockblocked by a hallucinated summary. But a nice fast respectful website? Send the traffic their way! Everyone wins—users, site owners, Google, the World Wide Web.

Could you imagine how quickly this would revolutionise the world of search engine optimisation? They’ve always told us that we should make websites for humans in order to get good Google juice. This would be a way of making it come true, without any of the over-engineered woefulness of AMP.

It’ll never happen of course. But I can dream.

Nothing can travel faster than light. And if you manage to travel close to the speed of light, things get weird.

Technically, we all experience time differently depending on how fast or slow we’re moving. But the differences are so imperceptible as to be non-existent. That’s how we can describe events as being “simultaneous”, even though according to Einstein’s theory of relativity, there’s no such thing.

It’s thanks to these small relativistic effects that GPS works. But when you approach the speed of light—or get close to something very massive—then the large-scale relativistic effects kick in.

If you travel close to the speed of light for a short time, it will seem like a much longer time to everyone you left behind. This is the twin paradox, which isn’t really a parodox at all, just time dilation in action.

There are some coincidental parallels to this kind of time dilation in old folk tales.

The Japanese tale of Urashima Tarō tells of a fisherman who rescues a sea turtle and is rewarded with a relaxing few days in an underwater kingdom, only to find that when he returns home to his village, 300 years have passed.

The Irish tale of Oisín describes the warrior’s journey to Tir na nÓg, the land of youth. He spends three years there but when he returns to Éire to see his old fighting comrades from the Fianna, 300 years have passed.

This story gives us a wonderfully poetic turn of phrase that’s still used today. The closest English equivalent is “Billy no mates”, a rather cruel term to describe someone with no friends. In Irish, we say:

Mar Oisín i ndiadh na Fianna

Like Oisín after the Fianna.

People talk about the effectiveness (or lack thereof) of large language models as though all tasks are comparable. But it strikes me that there are three broad categories of work that large language models are applied to:

1. Compression.
2. Transformation.
3. Expansion.

Compression is when you feed a large language model something big that you want to make small. Summarise this book. Give me the gist of this meeting. Large language models are generally pretty good at this, which makes sense given that they themselves are kind of like compressed artifacts.

Transformation is when large language models convert from one format into another. Turn this audio into text. Turn this jumble of data into structured JSON. A large language model can handle these tasks pretty well. There’ll probably be a few errors so make sure that’s not a deal-breaker.

Expansion is when you give a large language model a prompt to generate something from scratch. An image. A presentation. An email. A poem. This is where slop lives. The output inevitably betrays its origins, glistening with a sheen of mediocrity.

Laurie spotted this three-way split a while back:

Is what you’re doing taking a large amount of text and asking the LLM to convert it into a smaller amount of text? Then it’s probably going to be great at it. If you’re asking it to convert into a roughly equal amount of text it will be so-so. If you’re asking it to create more text than you gave it, forget about it.

I hope that when the bubble finally bursts, we’ll see the surviving large language models put to work on the first two categories. The boring stuff. The work that’s tedious for humans.

But tedious is as tedious does. Something I consider drudgery might be the very thing that gives you life. Like Giles says:

I have a feeling that everyone likes using AI tools to try doing someone else’s profession. They’re much less keen when someone else uses it for their profession.

The big exception seems to be programming. Apparently there are plenty of coders who never before expressed an interest in being managers who are now happily hanging up their coding spurs in favour being the overseer of non-human workers.

It’s a reasonable outlook. It could even be considered a user-centred approach. Users don’t care about the elegance of your code; they care about accomplishing their tasks.

Programming is something of an exception to the efficacy of large language models in general. Instead of relying on the subjectivity of painting, poetry, or prose, programming can be objectively tested. Throw enough money at the worst people in the world and they’ll give you tokens you can use to get the machines to test their own output. So you can get a large language model to create something reasonably good from scratch as long as that something is code.

If you had asked me about the threat model of large language models two years ago, I probably would’ve been worried for artists, writers, and musicians. I thought that software had enough inherent complexity to be relatively safe.

Now my opinion has completely reversed. Software is almost certainly the killer app for large language models.

I think the artists, writers, and musicians will be okay, or at least as okay as they ever were. It turns out that humans like things made by other humans.

And y’know what? If I had to choose which endeavour I’d rather see automated away—programming or art—it’s no competition.

Don’t get me wrong—it would be nice if everyone got paid for doing what they enjoy. It’s just that I’m okay with software engineers not being at the front of that line.

I remember when I first started getting paid money to make websites. “Really?” I thought, “Someone is willing to pay me to do something I’d do anyway?” I kept waiting for the jig to be up. Instead I saw my profession grow and expand.

Perhaps there’s a long-overdue compression happening.

Or maybe it’s more like a transformation.

If you wanted to make a really crude approximation of project management, you could say there are two main styles: waterfall and agile.

It’s not as simple as that by any means. And the two aren’t really separate things; agile came about as a response to the failures of waterfall. But if we’re going to stick with crude approximations, here we go:

• In a waterfall process, you define everything up front and then execute.
• In an agile process, you start executing and then adjust based on what you learn.

So crude! Much approximation!

It only recently struck me that the agile approach is basically a cybernetic system.

Cybernetics is pretty much anything that involves feedback. If it’s got inputs and outputs that are connected in some way, it’s probably cybernetic. Politics. Finance. Your YouTube recommendations. Every video game you’ve ever played. You. Every living thing on the planet. That’s cybernetics.

Fun fact: early on in the history of cybernetics, a bunch of folks wanted to get together at an event to geek about this stuff. But they knew that if they used the word “cybernetics” to describe the event, Norbert Wiener would show up and completely dominate proceedings. So they invented a new alias for the same thing. They coined the term “artificial intelligence”, or AI for short.

Yes, ironically the term “AI” was invented in order to repel a Reply Guy. Now it’s Reply Guy catnip. In today’s AI world, everyone’s a Norbert Wiener.

The thing that has the Wieners really excited right now in the world of programming is the idea of agentic AI. In this set-up, you don’t do any of the actual coding. Instead you specify everything up front and then have a team of artificial agents execute your plan.

That’s right; it’s a return to waterfall. But that’s not as crazy as it sounds. Waterfall was wasteful because execution was expensive and time-consuming. Now that execution is relatively cheap (you pay a bit of money to line the pockets of the worst people in exchange for literal tokens), you can afford to throw some spaghetti at the wall and see if it sticks.

But you lose the learning. The idea of a cybernetic system like, say, agile development, is that you try something, learn from it, and adjust accordingly. You remember what worked. You remember what didn’t. That’s learning.

Outsourcing execution to machines makes a lot of sense.

I’m not so sure it makes sense to outsource learning.

I don’t like magic.

I’m not talking about acts of prestidigitation and illusion. I mean the kind of magic that’s used to market technologies. It’s magic. It just works. Don’t think about it.

I’ve written about seamless and seamful design before. Seamlessness is often touted as the ultimate goal of UX—“don’t make me think!”—but it comes with a price. That price is the reduction of agency.

When it comes to front-end development, my distrust of magic tips over into being a complete control freak.

I don’t like using code that I haven’t written and understood myself. Sometimes its unavoidable. I use two JavaScript libraries on The Session. One for displaying interactive maps and another for generating sheet music. As dependencies go, they’re very good but I still don’t like the feeling of being dependant on anything I don’t fully understand.

I can’t stomach the idea of using npm to install client-side JavaScript (which then installs more JavaScript, which in turn is dependant on even more JavaScript). It gives me the heebie-jeebies. I’m kind of astonished that most front-end developers have normalised doing daily trust falls with their codebases.

While I’m mistrustful of libraries, I’m completely allergic to frameworks.

Often I don’t distinguish between libraries and frameworks but the distinction matters here. Libraries are bits of other people’s code that I call from my code. Frameworks are other people’s code that call bits of my code.

Think of React. In order to use it, you basically have to adopt its idioms, its approach, its syntax. It’s a deeper level of dependency than just dropping in a regular piece of JavaScript.

I’ve always avoided client-side React because of its direct harm to end users (over-engineered bloated sites that take way longer to load than they need to). But the truth is that I also really dislike the extra layer of abstraction it puts between me and the browser.

Now, whenever there’s any talk about abstractions someone inevitably points out that, when it comes to computers, there’s always some layer of abstraction. If you’re not writing in binary, you don’t get to complain about an extra layer of abstraction making you uncomfortable.

I get that. But I still draw a line. When it comes to front-end development, that line is for me to stay as close as I can to raw HTML, CSS, and JavaScript. After all, that’s what users are going to get in their browsers.

My control freakery is not typical. It’s also not a very commercial or pragmatic attitude.

Over the years, I’ve stopped doing front-end development for client projects at work. Partly that’s because I’m pretty slow; it makes more sense to give the work to a better, faster developer. But it’s also because of my aversion to React. Projects came in where usage of React was a foregone conclusion. I wouldn’t work on those projects.

I mention this to point out that you probably shouldn’t adopt my inflexible mistrustful attitude if you want a career in front-end development.

Fortunately for me, front-end development still exists outside of client work. I get to have fun with my own website and with The Session. Heck, they even let me build the occasional hand-crafted website for a Clearleft event. I get to do all that the long, hard stupid way.

Meanwhile in the real world, the abstractions are piling up. Developers can now use large language models to generate code. Sometimes the code is good. Sometimes its not. You should probably check it before using it. But some developers just YOLO it straight to production.

That gives me the heebie-jeebies, but then again, so did npm. Is it really all that different? With npm you dialled up other people’s code directly. With large language models, they first slurp up everyone’s code (like, the whole World Wide Web), run a computationally expensive process of tokenisation, and then give you the bit you need when you need it. In a way, large language model coding tools are like a turbo-charged npm with even more layers of abstraction.

It’s not for me but I absolutely understand why it can work in a pragmatic commercial environment. Like Alice said:

Knitting is the future of coding. Nobody knits because they want a quick or cheap jumper, they knit because they love the craft. This is the future of writing code by hand. You will do it because you find it satisfying but it will be neither the cheapest or quickest way to write software.

But as Dave points out:

And so now we have these “magic words” in our codebases. Spells, essentially. Spells that work sometimes. Spells that we cast with no practical way to measure their effectiveness. They are prayers as much as they are instructions.

I shudder!

But again, this too is nothing new. We’ve all seen those codebases that contain mysterious arcane parts that nobody dares touch. coughWebpackcough. The issue isn’t with the code itself, but with the understanding of the code. If the understanding of the code was in one developer’s head, and that person has since left, the code is dangerous and best left untouched.

This, as you can imagine, is a maintenance nightmare. That’s where I’ve seen the real cost of abstractions. Abstractions often really do speed up production, but you pay the price in maintenance later on. If you want to understand the codebase, you must first understand the abstractions used in the codebase. That’s a lot to document, and let’s face it, documentation is the first casuality of almost every project.

So perhaps my aversion to abstraction in general—and large language models in particular—is because I tend to work on long-term projects. This website and The Session have lifespans measured in decades. For these kinds of projects, maintenance is a top priority.

Large language model coding tools truly are magic.

I don’t like magic.

I find Brian Eno to be a fascinating chap. His music isn’t my cup of tea, but I really enjoy hearing his thoughts on art, creativity, and culture.

I’ve always loved this short piece he wrote about singing with other people. I’ve passed that link onto multiple people who have found a deep joy in singing with a choir:

Singing aloud leaves you with a sense of levity and contentedness. And then there are what I would call “civilizational benefits.” When you sing with a group of people, you learn how to subsume yourself into a group consciousness because a capella singing is all about the immersion of the self into the community. That’s one of the great feelings — to stop being me for a little while and to become us. That way lies empathy, the great social virtue.

Then there’s the whole Long Now thing, a phrase that originated with him:

I noticed that this very local attitude to space in New York paralleled a similarly limited attitude to time. Everything was exciting, fast, current, and temporary. Enormous buildings came and went, careers rose and crashed in weeks. You rarely got the feeling that anyone had the time to think two years ahead, let alone ten or a hundred. Everyone seemed to be passing through. It was undeniably lively, but the downside was that it seemed selfish, irresponsible and randomly dangerous. I came to think of this as “The Short Now”, and this suggested the possibility of its opposite - “The Long Now”.

I was listening to my Huffduffer feed recently, where I had saved yet another interview with Brian Eno. Sure enough, there was plenty of interesting food for thought, but the bit that stood out to me was relevant to, of all things, prototyping:

I have an architect friend called Rem Koolhaas. He’s a Dutch architect, and he uses this phrase, “the premature sheen.” In his architectural practice, when they first got computers and computers were first good enough to do proper renderings of things, he said everything looked amazing at first.

You could construct a building in half an hour on the computer, and you’d have this amazing-looking thing, but, he said, “It didn’t help us make good buildings. It helped us make things that looked like they might be good buildings.”

I went to visit him one day when they were working on a big new complex for some place in Texas, and they were using matchboxes and pens and packets of tissues. It was completely analog, and there was no sense at all that this had any relationship to what the final product would be, in terms of how it looked.

It meant that what you were thinking about was: How does it work? What do we want it to be like to be in that place? You started asking the important questions again, not: What kind of facing should we have on the building or what color should the stone be?

I keep thinking about that insight: “It didn’t help us make good buildings. It helped us make things that looked like they might be good buildings.”

Substitute the word “buildings” for whatever output is supposedly being revolutionised by generative models today. Websites. Articles. Public policy.

England is criss-crossed by routes that were originally laid down by the Romans. When it came time to construct modern roads, it often made sense to use these existing routes rather than trying to designate entirely new ones. So some of the roads in England are like an early kind of desire path.

Desire paths are something of a cliché in the UX world. They’re the perfect metaphor for user-centred design; instead of trying to make people take a pre-defined route, let them take the route that’s easiest for them and then codify that route.

This idea was enshrined into the very design principles of HTML as “pave the cowpaths”:

When a practice is already widespread among authors, consider adopting it rather than forbidding it or inventing something new.

Ireland never had any Roman roads. But it’s always had plenty of cowpaths.

The Irish word for cow is bó.

The Irish word for road is bóthar, which literally means “cowpath”.

The cowpaths were paved in both the landscape and the language.

On the last day of UX London this year, I was sitting and chatting with Rachel Coldicutt who was going to be giving the closing keynote. Inevitably the topic of converstation worked its way ’round to “AI”. I remember Rachel having a good laugh when I summarised my overall feeling:

I kind of wish I could go into suspended animation and be woken up when all this is over and things have settled down one way or another.

I still feel that way. Like Gina, I’d welcome a measured approach to this technology. As Anil puts it:

Technologies like LLMs have utility, but the absurd way they’ve been over-hyped, the fact they’re being forced on everyone, and the insistence on ignoring the many valid critiques about them make it very difficult to focus on legitimate uses where they might add value.

I very much look forward to using language models (probably small and local) to automate genuinely tedious tasks. That’s a very different vision to what the slopagandists are pushing. Or, like Paul Ford says:

Make it boring. That’s what’s interesting.

Fortunately, my cryosleep-awakening probably isn’t be too far off. You can smell it in the air, that whiff of a bubble about to burst. And while it will almost certainly be messy, it’s long overdue.

Paul Ford again:

I’ve felt so alienated from tech over the past couple of years. Part of it is the craven authoritarianism. It dampens the mood. But another part is the monolithic narrative—the fact that we live in a world where there seem to be only a few companies, only a few stories going at any time, and everything reduces to politics. God, please let it end.

When I talk about large language models, I make sure to call them large language models, not “AI”. I know it’s a lost battle, but the terminology matters to me.

The term “AI” can encompass everything from a series of if/else statements right up to Skynet and HAL 9000. I’ve written about this naming collision before.

It’s not just that the term “AI” isn’t useful, it’s so broad as to be actively duplicitous. While talking about one thing—like, say, large language models—you can point to a completely different thing—like, say, machine learning or computer vision—and claim that they’re basically the same because they’re both labelled “AI”.

If a news outlet runs a story about machine learning in the context of disease prevention or archeology, the headline will inevitably contain the phrase “AI”. That story will then gleefully be used by slopagandists looking to inflate the usefulness of large language models.

Conflating these different technologies is the fallacy at the heart of Robin Sloan’s faulty logic:

If these machines churn through all media, and then, in their deployment, discover several superconductors and cure all cancers, I’d say, okay … we’re good.

John Scalzi recently wrote:

“AI” is mostly a marketing phrase for a bunch of different processes and tools which in a different era would have been called “machine learning” or “neural networks” or something else now horribly unsexy.

But I’ve noticed something recently. More than once I’ve seen genuinely-useful services refer to their technology as “traditional machine learning”.

First off, I find that endearing. Like machine learning is akin to organic farming or hand-crafted furniture.

Secondly, perhaps it points to a severing of the ways between machine learning and large language models.

Up until now it may have been mutually benificial for them to share the same marketing term, but with the bubble about to burst, anything to do with large language models might become toxic by association, including the term “AI”. Hence the desire to shake the large-language model grifters from the coattails of machine learning and computer vision.

After spending a week immersed in the language and the landscape of Glencolmcille, Jessica and I were headed to Miltown Malbay for the annual Willie Clancy music week.

I could only get us accommodation from the Monday onwards so we had a weekend in between Donegal and Clare. We decided to spend it in Galway.

We hadn’t booked any travel from Glencolmcille to Galway and that worked out fine. We ended up getting a lift from a fellow student (and fellow blogger) heading home to Limerick.

Showing up in Galway on a busy Saturday afternoon was quite the change after the peace and quiet of Glencolmcille. But we dove right in and enjoyed a weekend of good food and music.

But I missed speaking Irish. So on the Sunday afternoon we made a trip out to Spiddal for lunch just so we could say a few words as Gaeilge.

We also got some practice in every morning getting coffee at the Plámás cafe. You get a ten-cent discount for ordering in Irish. What a lovely little piece of behaviour design—a nice gentle nudge!

From Galway we made our way down to Miltown Malbay where the Willie Clancy festival was in full swing. We were staying out in Spanish Point, so we could escape the madness of the town each evening. Mind you, there was plenty going at the Armada hotel too.

The hotel was something of an extravagance but it was worth it—we had a beautiful view on to the beach at Spanish Point and our room was tucked away far from the wild shenanigans in the hotel bar (not to mention the céilís on the other side of the hotel!).

I have to admit, I got quite overwhelmed the first day I ventured into Miltown proper. It’s easy to have a constant state of FOMO, constantly searching for the best session. But once I calmed down and accepted the situation, I had a lovely time at some really nice sessions.

Last time we were in Miltown Malbay was three years ago …and three years before that. Maybe we’ll be back in another three years.

I don’t know, though. It kind of felt like going to the South By Southwest after it got crazy big and the host town could no longer bear the weight of the event.

Still, I thoroughly enjoyed our two-week excursion down a stretch of the Wild Atlantic Way from Donegal to Galway to Clare.

I had never been to Donegal before my trip to Glencolmcille to spend a week there learning Irish.

I had heard it’s beautiful there. But pictures don’t really do it justice. When our bus was winding its way down into the valley, it looked breathtaking, laid out before us like a green haven where we’d spend the week immersed in the language as well as the landscape.

The reason I say that pictures don’t do it justice is that the light is constantly changing, like in the Lake District or the Dingle peninsula. The beauty is formed of equal parts geography and meteorology.

We had a day to explore before the language courses begin. We strolled along the beach. We walked down winding paths to find ancient burial tombs and standing stones.

Then it was time to knuckle down and learn Irish.

Oideas Gael provides seven levels of learning, increasing in experience. Jessica went in at level one and I was amazed by how much she had picked up by the end of the week. I figured I’d go in at level three or maybe four, but after hearing a description of all the levels, I actually decided to try level five.

It turned out to be just right. There was lots to learn, and I definitely need to make sure I keep working on it, but the teacher was great and my classmates were lovely.

Tar éis an cursa, tá níos mó ealois agam, tá níos mó taithí agam, ach an rud is tábhachtaí, tá níos mó féin-mhuinín agam. After the course, I have more knowledge, I have more experience, but most importantly, I have more self-confidence.

And after a day of learning Irish, it was nice to unwind in the evening with a pint in the local pub, where there was also a session every single night. Not only were the musicians top-notch, they were also very welcoming to this blow-in mandolin player.

All in all, it was a wonderful and fulfilling week.

Beidh mé ar ais arís! I’ll be back again!

I’ve been taking some time off after UX London. That was a big project I was working towards all year and it went great, so I think I’ve earned a reward for myself.

My reward is to head off to Ireland to immerse myself in the language and music. A week at an Irish language school in Donegal followed by a week at an Irish music festival in Clare, with a little weekend in Galway in between.

First I had to get to Donegal. My plan was: fly from Gatwick to Dublin; get the train from Dublin to Sligo; spend the night in Sligo; take a couple of buses to get to my destination in Donegal.

I fell at the first hurdle.

I consider myself a fairly seasoned traveller at this point so I’m kicking myself that I somehow messed up the time of that flight to Dublin. I showed up after the bag check had closed. That’s when I realised I was off by an hour.

The next available flight to Dublin wasn’t until late in the evening. Jessica and I contemplated spending all day waiting for that, then spending the night in Dublin, and then doing all the overland travel the next day.

But we didn’t do that. We went to Belfast instead. As it turned out, we had a great evening there at a lovely piping session that only happens on the last Friday of the month—the very day we were there. It was meant to be.

The next day we got the train to Derry, then a bus to Letterkenny, and then eventually another bus to Donegal town (the first one just didn’t show up—probably because Donegal were playing a semi-final match at the time), and finally the bus from Donegal town to Glencolmcille.

I had never been to Donegal before. Everyone always goes on about how beautiful it is. They are not wrong. The closer we got to Glencolmcille, the more our breath was literally taken away by the stunning landscape.

So here we are. We’re both doing Irish language classes. It’s all very challenging and very rewarding at the same time.

Best of all, we’re doing it in this unbelievably beautiful place.

This is the just the start of my little odyssey on the west coast of Ireland and it’s already absolutely wonderful …apart from that unexpectedly bumpy start.

I don’t use large language models. My objection to using them is ethical. I know how the sausage is made.

I wanted to clarify that. I’m not rejecting large language models because they’re useless. They can absolutely be useful. I just don’t think the usefulness outweighs the ethical issues in how they’re trained.

Molly White came to the same conclusion:

The benefits, though extant, seem to pale in comparison to the costs.

Rich has similar thoughts:

What I do know is that I find LLMs useful on occasion, but every time I use one I die a little inside.

I genuinely look forward to being able to use a large language model with a clear conscience. Such a model would need to be trained ethically. When we get a free-range organic large language model I’ll be the first in line to use it. Until then, I’ll abstain. Remember:

You don’t get companies to change their behaviour by rewarding them for it. If you really want better behaviour from the purveyors of generative tools, you should be boycotting the current offerings.

Still, in anticipation of an ethical large language model someday becoming reality, I think it’s good for me to have an understanding of which tasks these tools are good at.

Prototyping seems like a good use case. My general attitude to prototyping is the exact opposite to my attitude to production code; use absolutely any tool you want and prioritise speed over quality.

When it comes to coding in general, I think Laurie is really onto something when he says:

Is what you’re doing taking a large amount of text and asking the LLM to convert it into a smaller amount of text? Then it’s probably going to be great at it. If you’re asking it to convert into a roughly equal amount of text it will be so-so. If you’re asking it to create more text than you gave it, forget about it.

In other words, despite what the hype says, these tools are far better at transforming than they are at generating.

Iris Meredith goes deeper into this distinction between transformative and compositional work:

Compositionality relies (among other things) on two core values or functions: choice and precision, both of which are antithetical to LLM functioning.

My own take on this is that transformative work is often the drudge work—take this data dump and convert it to some other format; take this mock-up and make a disposable prototype. I want my tools to help me with that.

But compositional work that relies on judgement, taste, and choice? Not only would I not use a large language model for that, it’s exactly the kind of work that I don’t want to automate away.

Transformative work is done with broad brushstrokes. Compositional work is done with a scalpel.

Large language models are big messy brushes, not scalpels.

One persistent piece of slopaganda you’ll hear is this:

“It’s just a tool. What matters is how you use it.”

This isn’t a new tack. The same justification has been applied to many technologies.

Leaving aside Kranzberg’s first law, large language models are the very antithesis of a neutral technology. They’re imbued with bias and political decisions at every level.

There’s the obvious problem of where the training data comes from. It’s stolen. Everyone knows this, but some people would rather pretend they don’t know how the sausage is made.

But if you set aside how the tool is made, it’s still just a tool, right? A building is still a building even if it’s built on stolen land.

Except with large language models, the training data is just the first step. After that you need to traumatise an underpaid workforce to remove the most horrifying content. Then you build an opaque black box that end-users have no control over.

Take temperature, for example. That’s the degree of probability a large language model uses for choosing the next token. Dial the temperature too low and the tool will parrot its training data too closely, making it a plagiarism machine. Dial the temperature too high and the tool generates what we kindly call “hallucinations”.

Either way, you have no control over that dial. Someone else is making that decision for you.

A large language model is as neutral as an AK-47.

I understand why people want to feel in control of the tools they’re using. I know why people will use large language models for some tasks—brainstorming, rubber ducking—but strictly avoid them for any outputs intended for human consumption.

You could even convince yourself that a large language model is like a bicycle for the mind. In truth, a large language model is more like one of those hover chairs on the spaceship in WALL·E.

Large language models don’t amplify your creativity and agency. Large language models stunt your creativity and rob you of agency.

When someone applies a large language model it is an example of tool use. But the large language model isn’t the tool.

Also sprach Wittgenstein:

Die Grenzen meiner Sprache bedeuten die Grenzen meiner Welt.

Or in English, thus spoke Wittgenstein:

The limits of my language mean the limits of my world.

Language and thinking are intertwined. I’m not saying there’s anything to the strong form of the Sapir-Whorf hypothesis but I think George Lakoff is onto something when he talks about political language.

There’s literal political language like saying “tax relief”—framing taxation as something burdensome that needs to be relieved. But our everyday language has plenty of framing devices that might subconsciously influence our thinking.

When it comes to technology, our framing of new technologies often comes from previous technologies. As a listener to a show, you might find yourself being encouraged to “tune in again next week” when you may never have turned a radio dial in your entire life.

In the early days of the web we used a lot of language from print. John Allsopp wrote about this in his classic article A Dao Of Web Design:

The web is a new medium, although it has emerged from the medium of printing, whose skills, design language and conventions strongly influence it. Yet it is often too shaped by that from which it sprang.

One outdated piece of language on the web is a framing device in two senses: “above the fold”. It’s a conceptual framing device that comes straight from print where newspapers were literally folded in half. It’s a literal framing device that puts the important content at the top of the page.

But there is no fold. We pretended that everyone’s screens were 640 by 480 pixels. Then we pretended that everyone’s screens were 800 by 600 pixels. But we never really knew. It was all a consensual hallucination. Even before mobile devices showed up there was never a single fold.

Even if you know that there’s no literal page fold on the web, using the phrase “above the fold” is still insidiously unhelpful.

So what’s the alternative? Well, James has what I think is an excellent framing:

The landing zone.

It’s the bit of the page where people first show up. It doesn’t have a defined boundary. The landing zone isn’t something separate to the rest of the page; the content landing zone merges into the rest of the content.

You don’t know where the landing zone ends, and that’s okay. It’s better than okay. It encourages you design in a way that still prioritises the most important content but without fooling yourself into thinking there’s some invisible boundary line.

Next time you’re discussing the design of a web page—whether it’s with a colleague or a client—try talking about the landing zone.

I have little understanding for people using large language models to generate slop; words and images that nobody asked for.

I have more understanding for people using large language models to generate code. Code isn’t the thing in the same way that words or images are; code is the thing that gets you to the thing.

And if a large language model hallucinates some code, you’ll find out soon enough:

With code you get a powerful form of fact checking for free. Run the code, see if it works.

But I want to push back on one justification I see repeatedly about using large language models to write code. Here’s Craig:

There are many moral and ethical issues with using LLMs, but building software feels like one of the few truly ethically “clean”(er) uses (trained on open source code, etc.)

That’s not how this works. Yes, the large language models are trained on lots of code (most of it open source), but they’re not only trained on that. That’s on top of everything else; all the stolen books, all the unpaid creative work of others.

Even Robin Sloan, who first says:

I think the case of code is especially clear, and, for me, basically settled.

…goes on to acknowledge:

But, again, it’s important to say: the code only works because of Everything. Take that data away, train a model using GitHub alone, and you’ll get a far less useful tool.

When large language models are trained on domain-specific data, it’s always in addition to the mahoosive amount of content they’ve already stolen. It’s that mohoosive amount of content—not the domain-specific data—that enables them to parse your instructions.

(Note that I’m being very delibarate in saying “parse”, not “understand.” Though make no mistake, I’m astonished at how good these tools are at parsing instructions. I say that as someone who tried to write natural language parsers for text-only adventure games back in the 1980s.)

So, sure, go ahead and use large language models to write code. But don’t fool yourself into thinking that it’s somehow ethical.

What I said here applies to code too:

If you’re going to use generative tools powered by large language models, don’t pretend you don’t know how your sausage is made.

The Wikimedia Foundation, stewards of the finest projects on the web, have written about the hammering their servers are taking from the scraping bots that feed large language models.

Our infrastructure is built to sustain sudden traffic spikes from humans during high-interest events, but the amount of traffic generated by scraper bots is unprecedented and presents growing risks and costs.

Drew DeVault puts it more bluntly, saying Please stop externalizing your costs directly into my face:

Over the past few months, instead of working on our priorities at SourceHut, I have spent anywhere from 20-100% of my time in any given week mitigating hyper-aggressive LLM crawlers at scale.

And no, a robots.txt file doesn’t help.

If you think these crawlers respect robots.txt then you are several assumptions of good faith removed from reality. These bots crawl everything they can find, robots.txt be damned.

Free and open source projects are particularly vulnerable. FOSS infrastructure is under attack by AI companies:

LLM scrapers are taking down FOSS projects’ infrastructure, and it’s getting worse.

You try to do the right thing by making knowledge and tools freely available. This is how you get repaid. AI bots are destroying Open Access:

There’s a war going on on the Internet. AI companies with billions to burn are hard at work destroying the websites of libraries, archives, non-profit organizations, and scholarly publishers, anyone who is working to make quality information universally available on the internet.

My own experience with The Session bears this out.

Ars Technica has a piece on this: Open source devs say AI crawlers dominate traffic, forcing blocks on entire countries .

So does MIT Technology Review: AI crawler wars threaten to make the web more closed for everyone.

When we talk about the unfair practices and harm done by training large language models, we usually talk about it in the past tense: how they were trained on other people’s creative work without permission. But this is an ongoing problem that’s just getting worse.

The worst of the internet is continuously attacking the best of the internet. This is a distributed denial of service attack on the good parts of the World Wide Web.

If you’re using the products powered by these attacks, you’re part of the problem. Don’t pretend it’s cute to ask ChatGPT for something. Don’t pretend it’s somehow being technologically open-minded to continuously search for nails to hit with the latest “AI” hammers.

If you’re going to use generative tools powered by large language models, don’t pretend you don’t know how your sausage is made.

Dan wrote an interesting post with a somewhat clickbaity title; This Competition Exposed How AI is Reshaping Design:

I watched two designers go head-to-head in a high-speed battle to create the best landing page in 45 minutes. One was a seasoned pro. The other was a non-designer using AI.

If you can ignore the title (and the fact that Dan still actively posts on Twitter; something I find very hard to ignore), then there’s a really thoughtful analysis in there.

It’s less about one platform or tool vs. another more than it is a commentary on how design happens, and whether or not that’s changing in a significant way.

In particular, there’s a very revealing graph that shows the pros and cons of both approaches.

There’s no doubt about it, using a generative large language model helped a non-designer to get past the blank page. But it was less useful in subsequent iterations that rely on decision-making:

I’ve said it before and I’ll say it again: design is deciding. The best designers are the best deciders.

Dan finishes by saying that what he’d really like to see is an experienced designer/decider using these tools to turbo-boost their process:

AI raises the floor for non-designers. But can it raise the ceiling for designers?

Meanwhile, Matt has been writing about Vibe-designing. Matt is an experienced designer, but he’s not experienced with Figma. He’s found that he can work around that using a large language model:

Where in the past 30 years I might have had to cajole a more technically adept colleague into making something through sketches, gesticulating and making sound effects – I open up a Claude window and start what-iffing.

The “vibe” part of the equation often defaults to the mean, which is not a surprise when you think about what you’re asking to help is a staggeringly-massive machine for producing generally-unsurprising satisfactory answers quickly. So, you look at the output as a basis for the next sketch, and the next sketch and quickly, together, you move to something more novel as a result.

Interesting! Just as Dan insisted, the important work is making the decision and moving on to the next stage. If the actual outputs at each stage are mediocre, that seems to be okay, as long as they’re just good enough to inform a go/no-go decision.

This certainly seems more centaur-like than the usual boring uses of large language models to simply do what people are already doing.

Rich gets at something similar when he talks about using large language models for prototyping, where it’s okay if the code is kind of shitty:

If all you need is crappy code to try out a concept or a solution, then an LLM might well enable you (the designer) to do that.

Mind you, even if you do end up finding useful and appropriate ways to use these tools, you’re still using a tool built on exploitation and unfairness:

It’s hard (and reckless) to ignore the heartfelt and cogent perspective laid out by Miriam on the role of AI companies in the current geopolitical crisis:

When eugenics-obsessed billionaires try to sell me a new toy, I don’t ask how many keystrokes it will save me at work. It’s impossible for me to discuss the utility of a thing when I fundamentally disagree with the purpose of it.

A couple of days ago I linked to a post by Robin Sloan called Is it okay?, saying:

Robin takes a fair and balanced look at the ethics of using large language models.

That’s how it came across to me: fair and balanced.

Robin’s central question is whether the current crop of large language models might one day lead to life-saving super-science, in which case, doesn’t that outweigh the damage they’re doing to our collective culture?

Baldur wrote a response entitled Knowledge tech that’s subtly wrong is more dangerous than tech that’s obviously wrong. (Or, where I disagree with Robin Sloan).

Baldur pointed out that one side of the scale that Robin is attempting to balance is based on pure science fiction:

There is no path from language modelling to super-science.

Robin responded pointing out that some things that we currently have would have seemed like science fiction a few years ago, right?

Well, no. Baldur debunks that in a post called Now I’m disappointed.

(By the way, can I just point out how great it is to see a blog-to-blog conversation like this, regardless of how much they might be in disagreement.)

Baldur kept bringing the receipts. That’s when it struck me that Robin’s stance is largely based on vibes, whereas Baldur’s viewpoint is informed by facts on the ground.

In a way, they’ve got something in common. They’re both advocating for an interpretation of the precautionary principle, just from completely opposite ends.

Robin’s stance is that if these tools one day yield amazing scientific breakthroughs then that’s reason enough to use them today. It’s uncomfortably close to the reasoning of the effective accelerationist nutjobs, but in a much milder form.

Baldur’s stance is that because of the present harms being inflicted by current large language models, we should be slamming on the brakes. If anything, the harms are going to multiply, not magically reduce.

I have to say, Robin’s stance doesn’t look nearly as fair and balanced as I initially thought. I’m on Team Baldur.

Michelle also weighs in, pointing out the flaw in Robin’s thinking:

AI isn’t LLMs. Or not just LLMs. It’s plausible that AI (or more accurately, Machine Learning) could be a useful scientific tool, particularly when it comes to making sense of large datasets in a way no human could with any kind of accuracy, and many people are already deploying it for such purposes. This isn’t entirely without risk (I’ll save that debate for another time), but in my opinion could feasibly constitute a legitimate application of AI.

LLMs are not this.

In other words, we’ve got a language collision:

We call them “AI”, we look at how much they can do today, and we draw a straight line to what we know of “AI” in our science fiction.

This ridiculous situation could’ve been avoided if we had settled on a more accurate buzzword like “applied statistics” instead of “AI”.

There’s one other flaw in Robin’s reasoning. I don’t think it follows that future improvements warrant present use. Quite the opposite:

The logic is completely backwards! If large language models are going to improve their ethical shortcomings (which is debatable, but let’s be generous), then that’s all the more reason to avoid using the current crop of egregiously damaging tools.

You don’t get companies to change their behaviour by rewarding them for it. If you really want better behaviour from the purveyors of generative tools, you should be boycotting the current offerings.

Anyway, this back-and-forth between Robin and Baldur (and Michelle) was interesting. But it all pales in comparison to the truth bomb that Miriam dropped in her post Tech continues to be political:

When eugenics-obsessed billionaires try to sell me a new toy, I don’t ask how many keystrokes it will save me at work. It’s impossible for me to discuss the utility of a thing when I fundamentally disagree with the purpose of it.

Boom!

Maybe we should consider the beliefs and assumptions that have been built into a technology before we embrace it? But we often prefer to treat each new toy as as an abstract and unmotivated opportunity. If only the good people like ourselves would get involved early, we can surely teach everyone else to use it ethically!

You know what? I could quote every single line. Just go read the whole thing. Please.