Stop and think about what this means: the two models that predict constant merge rates over the latter half of the plot are more accurate than the linear growth trend. This corroborates what we eyeballed in the plots: the merge rate has not increased in the latter half of this plot. This means llms have not improved in their programming abilities for over a year. Isn’t that wild? Why is nobody talking about this?

This is really fascinating. Something to note is that the sample size is narrow focused on experienced developers with particular famous open source projects (average 5 years and 1,500 commits on the project in question).

In my own job I also have a novel, long-term situation so I can really sympathize with the prime slowdowns they identify. AI just doesn’t cut it.

One paper that caught my attention a bit ago was Selective attention in hypothesis-driven data analysis by Itai Yanai and Martin Lercher. In their study, students who were given specific hypotheses to test were much less likely to notice an obvious “gorilla in the data” compared to students who explored the data freely.