This dissertation consists of three main chapters covering two areas of application: development of normative standards for cognitive testing and heterogeneity test for standardized mean differences in meta-analysis.
Chapter 1 is a published work, describing the utility of the R package called test2norm. We developed this publicly available package for conversion of uncorrected (raw) neurocognitive test scores into demographically adjusted standard scores using a regression-based method. Demographic corrections help health providers detect an individual’s performance below what is expected for their demographic characteristics (e.g., age). We describe, in detail, the process of generating regression-based normative standards, provide practical recommendations for application of these standards, and issue instructions on how to use the test2norm package, complete with examples.
Chapters 2 and 3 explore tests for heterogeneity between standardized mean differences (δ ) in meta-analysis for studies with small groups (≤10). In chapter 2, through simulations, we estimate Type I error for heterogeneity tests with a test statistic calculated based on four different estimators of variance for δ , including a pooled unbiased variance estimator, not previously used. The simulations were run assuming different conditions, varying the group sizes, the number of studies, and the size of δ . We show that, under certain conditions, the asymptotic estimator meant for studies with large groups performs well, even when the group sizes are as small as 3. We show when the test based on this estimator breaks down, and provide recommendations for its use.
In chapter 3, we propose a new estimator for variance of δ , called “unbiased pooled leave-one-out estimator,” and compare the performance of the test based on this estimator to the tests explored in chapter 2. Using simulations, we estimate the Type I error and power of the proposed test under different test conditions. We show that, like in the case of the asymptotic estimator, the performance of the pooled unbiased leave-one-out estimator is dependent on the conditions of testing, including the size of δ and the number of studies included in the test.
Chapter 4 summarizes our findings, provides recommendations, and lays out a path for future research.