Media upload with pre-signed URL #3159
Replies: 4 comments
|
Hey Jan Bücker (@janbuecker), |
|
I am not sure about the traffic improvements. We have to download the files afterwards to generate thumbnails or analyst the uploaded file. I don't really see here any benefit 😅 |
|
Oliver Skroblin (@OliverSkroblin) ok Soner (@shyim) Yes, but this is true for images only. You could even delegate the generation of thumbnails to another service and work with events. In case of s3, you can automatically trigger a lambda on upload and handle thumbnails there. Inter-VPC traffic is much cheaper than through the internet. It’s totally fine to have it all on the app server but with this change you enable such possibilities described above. |
Uh oh!
There was an error while loading. Please reload this page.
Effort: low
Description:
Media files that are being uploaded, always pass the controller and the backend copies the stream to the desired path in the file system. That means, that every file crosses the application server in a whole. In scenarios, where you have an object storage like AWS S3, you double the traffic, because the user uploads the file to the application server, which then uploads the file to the remote file system.
Many object storage services offer "pre-signed URLs". You call the service to generate a one-time token based on parameters like the path and get a URL including that token in return. You can then use that URL to let the user upload the file directly to the remote file system and skip the traffic to the application server.
Benefits
Break strategy
For file systems with no support for pre-signed URLs (e.g. local), you could provide a controller endpoint, which simulates this behavior and another one, which generates the pre-signed URL. So in turn, deprecate the old endpoint and remove it with Version 6.x.
All reactions