Percentiles are a statistics concept, but you often see them mentioned in software engineering. Like this, this, and this. If you’re wondering what t…

No Starch Press link, with coupon: https://nostarch.com/learnbayes

It is true that, as Fisher points out, with enough samples you are almost guaranteed to reject the null hypothesis. That’s why we tell students to consider both p values (which you could think of as a form of quality control on the dataset) and variance explained. Loftus and Loftus make the point nicely: p tells you if you have enough samples and any effect to consider, variance explained tells you if it’s worth pursuing. Both are useful guides to a thoughtful analysis. In addition, I’d make a case for thinking about the scientific significance and importance of the hypothesis and the Bayesian prior. And to put a positive spin on this, given how easy it is to get small p values, big ones are pretty much a red flag to stop the analysis and go and do something more productive instead.