Psych-ICC.R is a R script that reads two sets of scores from a JSON file and calculates the Intraclass Correlation Coefficient (ICC) to compare them. Results are saved in a text document. Designed for psychological research.
The main program where the calculation takes place. Change INSERT-YOUR-FILE-HERE.JSON to the JSON file containing the ratings.
The sample JSON file containing ratings from human_rater and ai_rater. Replace the ratings with your own.
The text file that contains the calculations, e.g.,
Intraclass Correlation Coefficient (ICC)
----------------------------------------
Model: twoway
Type: agreement
Subjects: 149
Raters: 2
ICC = 0.569
95% CI [0.236, 0.745]
p = 0.000915
May your ICCs be high and your p-values low!