Every time I read about some study measuring the productivity gain of AI-assisted development, I raise an eyebrow. I find them uninteresting and useful only for the online fights between “there is no AI productivity for software developers” and “Claude Code increased my productivity by a billion percent.” The problem is that the landscape is so varied that, honestly, you can stress the data to prove both assertions.

My personal opinion is that many papers on the subject are just messy, and often they do not test at all how people using AI efficiently for work are actually using it. There are many issues I identified: they give too-easy problems, they impose arbitrary time constraints, they give problems to people not familiar with the framework they have to use (a big no no), and, surprisingly, I still see a lot of studies where the test subjects use “copy-pasting code from the web chat interface back and forth.” That is a way to work with AI that I think every developer stopped using in 2024.

But setting this aside, my main problem is that they never measure a specific class of task: the tasks that I would not have even started without AI assistants. We can discuss “this AI agent increases productivity by 2%” or “5%” or “20%” for as long as we want, but the reality is that many software and features I made in the last year would not exist at all without AI agents. I have folders with a sea of tools, automations, and single-use scripts that I would have never started.

So how could I measure that? For me, that is an increase of infinite percent, because they didn’t help me complete a project faster; they are the reason some projects exist. In some sense, it is a kind of Pascal’s wager. How should I redact the claim “AI agents improve developer output by only 2%” when I have concrete evidence on my hard drive of things that would have remained annotated in my notebook for all eternity?

Maybe it is my ADHD talking. Probably other people function differently and can do everything by just deciding to. But for me, lowering the activation moat that blocks me from starting to work on something is 100% worth it, even if I end up doing 80% of the work by hand.

It is a personal thing. I know that. I also understand if you work differently. And that’s the point. I saw developers using these tools in wildly different ways, for very different purposes, and with very different outcomes, and I don’t think it’s possible or fair to reduce that to a universal percentage number in some random study.

I vented my concerns on Mastodon before. Multiple times. But I was moderately optimistic.

Now, after years, I have to say that little changed. Almost all the technical issues are still there, but they never were my main concern. What bothered me was Mastodon’s sociology and the “instances” dynamic.

A lot of people are unrelentingly annoying. Having a Ph.D. in Artificial Intelligence (even if video game–related) was often enough to have people scolding me for no reason. But even that I could tolerate.

But then there are the instances and their admins: feudal castles ruled by feudal lords, fighting or allying with each other, enacting arbitrary rules even more arbitrarily enforced.

The other day, mastodon.social deleted a post by a guy I know because it criticized (legitimately, and by stating factual truths) another instance. Recently, another instance added a rule prohibiting talking badly about other instances. They said it is to protect the quality of the Fediverse. That’s funny, because I constantly get false and deranged posts on the Explore tab, and I don’t think that criticizing an instance admin is the problem with Mastodon’s quality.

In any case, that’s the last straw. I refuse to be part of such infantile power play.

Mastodon’s problem is not a technology issue; it is a people issue.

P.S. This is not about the Fediverse as a whole or ActivityPub

I just want to say that this is a Mastodon-specific issue. The ActivityPub protocol is, obviously, not responsible for this. Probably other services of the Fediverse (Lemmy, Pixelfed, etc.) have the same problems. I don’t know. It didn’t look that way to me, but maybe it is just because they are small in comparison.

And, of course, there are exceptions. There are instances and admins that genuinely care about Mastodon’s mission. I know them. It is a shame they have to be affected by this state of affairs.

But I am too old for this. I have little time for things that are no longer interesting or fun.