The structure of neural systems is fundamental to their function and is the product of both genes and the environment. Elucidating how and why brains take their particular form is a step toward understanding what makes us human and would improve the diagnosis and treatment of heritable neuropsychiatric disorders with characteristic brain pathology. Statistical analysis of anatomical and genetic variation within species provides a powerful strategy for understanding the structural layout of brains, and variations to this approach were applied in three studies. The first study examined how continuously growing teleost fish compensate for potentially increasing visual conduction delays along lengthening optic nerves. We found that as fish grew retinal ganglion cell axons in the optic nerve enlarged so that the shape of the axon diameter distribution remained constant, potentially simplifying visual computation problems in the brain. The second study investigated the relationship between skull and brain morphology and genetic background among individuals of European ancestry. Skull size and shape as well as frontotemporal cortical surface area varied continuously along a NW-SE axis in Europe. This trend was consistent with known historical population movements and could not be attributed to brains scaling with body size. The third study applied a genome-wide analysis to identify single nucleotide polymorphisms (SNPs) associated with the scaling of visual cortical surface area in humans. In two independent cohorts, a common SNP was significantly associated with an increased occipital proportion of cortex as a function of total cortical surface area. In summary, allometric scaling and statistical genetic analyses provided insight into the scalable architecture of the fish visual system and genetic contributions to human cortical morphology. Identifying genes that contribute to normal cortical architecture provides a first step toward understanding genetic mechanisms that underlie cortical pathology in a host of heritable neuropsychiatric disorders