'Slim' docker container #1036
Replies: 2 comments 1 reply
-
|
The NVIDIA CUDA drivers are several GB each, and we support the largest set of hardware possible with the default docker image. My recollection is that you should be able to get down to a few GB docker image for a single target, although I have not validated that recently. Have you tried building the Dockerfile here: https://github.com/postgresml/postgresml/blob/master/docker/Dockerfile It may be worth building your image on base ubuntu instead of nvidia/cuda, to see how much you can reduce bloat. This is possible, but somewhat laborious since NVIDIA does not open source their drivers. |
Beta Was this translation helpful? Give feedback.
-
|
Thanks for the response.
Yes I did try the build myself and it comes out the same.
The Nvidia driver size rings a bell, I'll try the base Ubuntu as suggested although, after testing yesterday, it's going to be a serious hit on performance... :(
Sent from Outlook for Android<https://aka.ms/AAb9ysg>
…________________________________
From: Montana Low ***@***.***>
Sent: Saturday, September 30, 2023 3:27:15 AM
To: postgresml/postgresml ***@***.***>
Cc: Mike Reynolds ***@***.***>; Author ***@***.***>
Subject: Re: [postgresml/postgresml] 'Slim' docker container (Discussion #1036)
The NVIDIA CUDA drivers are several GB each, and we support the largest set of hardware possible with the default docker image. My recollection is that you should be able to get down to a few GB docker image for a single target, although I have not validated that recently.
Have you tried building the Dockerfile here: https://github.com/postgresml/postgresml/blob/master/docker/Dockerfile
It may be worth building your image on base ubuntu instead of nvidia/cuda, to see how much you can reduce bloat. This is possible, but somewhat laborious since NVIDIA does not open source their drivers.
—
Reply to this email directly, view it on GitHub<#1036 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AUHPLEXKW2FDBAKATFFIDPTX457QHANCNFSM6AAAAAA5MOK7MU>.
You are receiving this because you authored the thread.Message ID: ***@***.***>
|
Beta Was this translation helpful? Give feedback.
-
|
If you dig through the git history you can probably find our old docker file that installed the Nvidia drivers in the base Ubuntu image, so that you keep GPU acceleration. |
Beta Was this translation helpful? Give feedback.
-
Are there any plans to make a slimmer docker container, as it currently runs at ~15Gb?
Not being a Python person I love the idea of having vector embedding support directly within the database, but 15Gb is prohibitive for deploying on Kubernetes and I cannot tell if or what can be cleaned out of the image post building.
Many thanks.
Beta Was this translation helpful? Give feedback.
All reactions