Born in Portsmouth, New Hampshire, Thomas Bailey Aldrich moved with his father to New Orleans, Louisiana at the age of three. He remained there until age thirteen, when his father's impending death prompted Aldrich's return to New Hampshire and his mother's household (Parker). At age sixteen, Aldrich started working as a clerk for his uncle, Charles Frost: "While working over the books of the firm, his mind was often busy with themes outside of the commission house, all leading towards a literary career" (Hemstreet 218).