When Davey packs up to leave his apartment, his two bags are not the same two bags he has at the end of the film.
Vinnie is driving a Buick at the beginning of the film but a Plymouth later when Davey jumps into the car at the stop light.
Before Davey jumps through the loft window, he is wearing light socks and later dark socks. When he lands on the sidewalk, he is clearly wearing white socks.
When Vinnie drives Davey to the loft, they take a lift up to at least the second floor. However, when Davey jumps through the loft window, he falls only one story.
When Davey drops the gun the close up shows Vinnie picking it up. When the shot switches to a longer view he is once again reaching for the Luger.
Pennsylvania Station was electrified, all trains entering and leaving would not be pulled by steam engines. The sounds of steam engines chugging about during the station sequence are a goof. Pennsylvania Railroad trains had GG1 electric locomotives.
Davey escapes from the loft by leaping through a closed window. When he lands in the street, not nearly enough glass lands with him.
When Davey is reading the letter on the subway, the narration of the letter is obviously very different from what is written on the page.