The volcano in this film was built from scratch on farmland outside Fallbrook, California. Each shot required packing with hundreds of pounds of explosives and a carefully-orchestrated "eruption" to be filmed by the cameraman seated on the front skids of a helicopter. One eruption went off a little early and nearly took out the chopper, burning off the cameraman's eyebrows and some of his hair. Because the eruptions looked so good, this ersatz volcano provided stock footage for other films, commercials, etc., for decades.
In his autobiography "Sun and Shadow", Jean-Pierre Aumont, who played Jacques, spoke of scheduling conflicts between Spencer Tracy and Frank Sinatra: "[Tracy], a genial man who was not well at the time, couldn't work past the morning. The problem was that Sinatra would only work in the afternoon. In the morning he hired a private plane and hopped from island to island trying to convince the startled inhabitants to vote for [John F. Kennedy] in the next presidential election. Around two o'clock he returned, exhausted, at the precise moment when Tracy was retiring for the day to his rooms. How, in these conditions, the scenes between Tracy and Sinatra were shot is a mystery to me."
The meaning and relevance of this film's title is that its literary origins are said to have derived from a proverb that says, "It is hard for a man to be brave when he knows he is going to meet the devil at four o'clock". Moreover, in the film's story, the team of rescuer adventurers goes out to save lepers on the mountain after the volcano has erupted and the island evacuated. They must return by 4:00 o'clock the next day, which is the deadline for when the last schooner will depart the island.
At a negative cost of over $5.7 million, it was the most expensive movie Columbia had ever made up to that date.
Richard Widmark has said that Spencer Tracy told him that a broomstick was used as Frank Sinatra's stand-in during Sinatra's absences, marking his position in a scene, the brush end representing Sinatra's head. Tracy had to act opposite it whilst a script girl read Sinatra's lines.