The movie was based upon L.P. Hartley's novel of the same name. The opening line of the novel has become somewhat well-known: "The past is a foreign country: they do things differently there." That same line--spoken by the voice-over narrator--opens this movie.
Regarding filming in Norfolk, director Joseph Losey said in an interview, "Norfolk helped me a lot because Norfolk hasn't changed. Most of the costumes were genuine. We made very few others, and we all lived in the house. They wore the costumes all the time, and ate, as well as acted in their costumes. Once you've got the exact house, accessories, costumes, something then springs to life."
Deborah Kerr originally was cast as Mrs. Maudsley but withdrew after director Joseph Losey left her waiting several hours in a hotel for his return call. Margaret Leighton eventually got the part.
In an interview with critic John Russell Taylor, screenwriter Harold Pinter said, "Now what I find most exciting about the subject is the role of time: the annihilation of time by the man's return to the scene of his childhood experience." Pinter told interviewer Michel Ciment, "I am fascinated by the concept of time, and by the power the cinema has to suddenly reveal the meaning of a whole life from the age of twelve to sixty."