Producer Ismail Merchant has said of this movie in his memoirs 'My Passage from India' (2002): "We tossed various ideas around and finally settled on the subject of collecting Indian miniature paintings, something that [director James Ivory (Jim)] had been doing for twenty years since his interest had been aroused while he was making The Sword and the Flute (1959). Jim's interest in the subject had aroused mine. I knew little about Indian miniature painting until we made The Householder (1963), when Jim would spend all his free time visiting dealers, and I tagged along. We would go into tiny shops where [there were] shady dealers or perhaps they just looked shady [because they] sat behind shuttered windows. They would bring out musty-smelling bundles, cloth bags tied with string that they would carefully unwrap to reveal miniature treasures. Gradually I got hooked and began collecting in a modest way. Jim has a scholarly knowledge of these paintings. I don't. I go by instinct. If something speaks to me, I buy it".
The art deco royal regal castle in India featured in the film was the Umaid Bhawan Palace in the city Jodhpur in the Indian state of Rajastan.
During principal photography a large fire broke out on set which blew out from just a small fire being required for a scene. Producer Ismail Merchant has said of the fire in his memoirs 'My Passage from India' (2002): "Bapji [aka Gaj Singh II, the then current Maharaja of Jodhpur] was enormously hospitable and helpful to us and behaved with the utmost courtesy when we almost burned his palace down during filming. We prepared bundles of rags soaked in kerosene, enough bundles for several takes if necessary. This was an interior scene, so we took the precaution of having a few fire extinguishers at hand, not so much because we expected to set the palace alight, but because we needed to put out the fire after shooting the scene. Unfortunately, on the first take the flames spread from one kerosene-soaked bundle to the next, and before we even had a chance to react, the fire took hold. The flames leaped out of the window, and the exterior shot looks as though the whole palace was on fire. Everyone in the unit was shocked. We had come to shoot a film at this historic site and were about to leave it a pile of ashes. Some fast-thinking assistants on the crew grabbed the extinguishers and eventually managed to put the fire out, but not before the room suffered quite a lot of damage. [Director James Ivory] felt we had behaved badly, and he saw no difference between us and the characters in the film who were intent on raiding the palace of its treasures no matter what the cost. Bapji, on the other hand, regarded this as nothing more than an unfortunate accident and, generously, dismissed the incident".
The high majority of the film's financing was funded by the London Weekend Television (LWT) outfit.
The screenplay for the movie was still being written by scriptwriter Ruth Prawer Jhabvala when principal photography on the picture commenced.