Fred Astaire had sung "They Can't Take That Away from Me" to Ginger Rogers previously in Shall We Dance (1937), but they had never danced to it. Rogers suggested that they use the song again (this time dancing), and so it was included.
This was the tenth and final film to co-star Fred Astaire and Ginger Rogers, their first in ten years since The Story of Vernon and Irene Castle (1939). Also, it was their only film together in color, and the only one they made for a studio (MGM) other than RKO.
Originally planned to star Fred Astaire and Judy Garland after their success together in Easter Parade (1948), but when Garland's addiction to drugs spiraled out of control, producer Arthur Freed replaced her with Ginger Rogers. The original title of the film was supposed to be "You Made Me Love You", after a hit song of Garland's.
Fred Astaire's wife Phyllis was widely rumored to have forbidden Fred to kiss Ginger Rogers in their first nine films together. To end the gossip, in The Barkleys of Broadway (1949), Fred and Ginger performed a kiss that was the longest recorded on a Hollywood film to that date.
Oscar Levant performs one of the fastest renditions of "Saber Dance" ever recorded.