Repository navigation
Question: reduce size of docker image #316
Description
Activity
This is a known issue. I'm not sure if ~300MB is is possible, since the base CentOS 8 image is already ~239MB, see for example:
$ podman pull centos:8 $ podman images REPOSITORY TAG IMAGE ID CREATED SIZE quay.io/centos/centos 8 5d0da3dc9764 6 weeks ago 239 MB
Of course, any further reduction in size would be great. I've considered changing the base image to another distro, but haven't yet found a better alternative. For example, libvips in Debian 10 is still on version 8.7.4 due to their policy of never updating packages (except for security fixes and critical bugs) because they prioritize stability. Alpine is a very popular rolling release system, but unfortunately only has a default thread stack size of 128k, which can lead to stack overflows in the dependencies of libvips (see for e.g. issue libvips/libvips#1287).
Note that PR #276 tried to reduce the size of the image by using a multi-stage build, but this was not merged for several reasons:
- DNF (CentOS/RHEL/Fedora default package manager) is not aware of the copied dependencies.
- Loadable modules were not copied along to the final image (i.e. the dependencies in
/usr/lib64/vips-modules-x.y/*and/usr/lib64/ImageMagick-x.y.z/modules-Q16/*). - It was difficult to maintain.
A better approach to reduce the image size would be to remove the
-develdependencies, the build directory, and the DNF cache after configuring and building the CMake project. For example, I saw that someone did this on one of the forks:
8349d64...palamccc:5.xLet's tag this as an enhancement. Very happy to accept a PR, if you're able.
Reacted by Giau Tran MinhSize of image after remove remove the
-develdependencies reduced to 1.07GB (only 50MB). I think there are still a lot of files we can remove (e.g: gcc cache during build phase).REPOSITORY TAG IMAGE ID CREATED SIZE weserv/images latest 35914800957a 6 seconds ago 1.07GBThe issue libvips/libvips#1287 has been resolved by allow to set
VIPS_MIN_STACK_SIZEto change stack size. Is any help to switch to Alpine?I able to build docker image to reduce size to 451MB. Can you double-check any issue with it?
REPOSITORY TAG IMAGE ID CREATED SIZE weserv/images alpine 969a998f93ce 7 seconds ago 451MB weserv/images latest 35914800957a About an hour ago 1.07GB
FROM alpine:3.13.6 RUN apk add --update-cache \ --repository https://dl-3.alpinelinux.org/alpine/edge/testing/ \ --repository https://dl-3.alpinelinux.org/alpine/edge/main \ build-base cmake git \ libjpeg-turbo \ openssl openssl-dev \ vips vips-dev # Copy the contents of this repository to the container COPY . /var/www/imagesweserv WORKDIR /var/www/imagesweserv/build # Build CMake-based project ARG NGINX_VERSION=1.21.3 RUN cmake .. \ -DCMAKE_BUILD_TYPE=Release \ -DBUILD_TOOLS=ON \ -DNGX_VERSION=$NGINX_VERSION \ -DCUSTOM_NGX_FLAGS="--prefix=/usr/share/nginx;\ --sbin-path=/usr/sbin/nginx;\ --modules-path=/usr/lib64/nginx/modules;\ --conf-path=/etc/nginx/nginx.conf;\ --error-log-path=/var/log/nginx/error.log;\ --http-log-path=/var/log/nginx/access.log;\ --http-client-body-temp-path=/var/lib/nginx/tmp/client_body;\ --http-proxy-temp-path=/var/lib/nginx/tmp/proxy;\ --http-fastcgi-temp-path=/var/lib/nginx/tmp/fastcgi;\ --http-uwsgi-temp-path=/var/lib/nginx/tmp/uwsgi;\ --http-scgi-temp-path=/var/lib/nginx/tmp/scgi;\ --pid-path=/run/nginx.pid;\ --lock-path=/run/lock/subsys/nginx;\ --user=nginx;\ --group=nginx" \ && make -j"$(nproc)" \ && cd .. \ && rm -Rf build /var/cache/* # Cleanup build dependencies RUN apk del build-base cmake git openssl-dev vips-dev WORKDIR /var/www/imagesweserv RUN addgroup -S nginx && adduser -S nginx -G nginx # Ensure nginx directories exist RUN mkdir -m 700 /var/lib/nginx \ && mkdir -m 700 /var/lib/nginx/tmp \ # Forward request and error logs to docker log collector && ln -sf /dev/stdout /var/log/nginx/weserv-access.log \ && ln -sf /dev/stderr /var/log/nginx/weserv-error.log \ # Copy nginx configuration to the appropriate location && cp /var/www/imagesweserv/ngx_conf/*.conf /etc/nginx EXPOSE 80 ENTRYPOINT ["nginx", "-g", "daemon off;"]
I'm not sure if
VIPS_MIN_STACK_SIZEhelps in all scenarios, for example a thread spawned by an external dependency would not honor this env variable. Another thing to consider is that Alpine packages are built with-Osby default (optimized for binary size) whereas CentOS/RHEL/Fedora uses,-O2(optimized for performance). libvips from Remi's RPM repository also enables auto-vectorization, which would speed things up even more.Given this, I'm a bit reluctant to switch the pre-built Docker image to Alpine Linux, but you're always free to use an other Linux distro of your choice.
Can you double-check any issue with it?
I will review PR #317 in the course of next week.
Reacted by Giau Tran MinhCan we compile
libvipsfrom source code for alpine?https://github.com/felixbuenemann/vips-alpine/blob/master/Dockerfile#L5-L30
- added a commit that references this issue
on Oct 31, 2021 With commit a9488c4 and cee1837 I now see:
$ podman images REPOSITORY TAG IMAGE ID CREATED SIZE localhost/weserv/images custom 61c3cce52b29 3 seconds ago 588 MB localhost/weserv/images alpine b04aa046a324 4 minutes ago 96.4 MB localhost/weserv/images latest 325490d9ea82 6 minutes ago 622 MB quay.io/centos/centos 8 5d0da3dc9764 6 weeks ago 239 MB docker.io/library/alpine 3.14 14119a10abf4 2 months ago 5.87 MB
The
weserv/images:customtag is based on the experimental RPMS repo from https://rpms.weserv.nl, see comment #298 (comment) on how to use that.Can we compile
libvipsfrom source code for alpine?I prefer to install libvips through the standard package managers, this ensures that it can be easily updated in a running container and avoids some maintenance.
FWIW, the released image on the GitHub Container Registry is even smaller 🎉
$ docker images REPOSITORY TAG IMAGE ID CREATED SIZE ghcr.io/weserv/images 5.x e13dfa2278ad 15 minutes ago 595MB
Reacted by Giau Tran MinhThe results are amazing. Thank you very much!
Hi @kleisauke, I just build alpine in local and facing an error.
giautm ➜ ~/develop $ docker run -p 8080:80 --name=imagesweserv giautm/weserv-images:alpine-3.14 nginx: [emerg] mkdir() "/var/cache/nginx/fastcgi_temp" failed (2: No such file or directory)My bad, new docker requires a volume to mount
docker run -p 8080:80 --name=imagesweserv -v $(pwd)/nginx-cache:/var/cache/nginx -v $(pwd)/nginx-pid:/var/run giautm/weserv-images:alpine-3.14
Those host volumes mount points shouldn't be necessary, commit 82771fc fixes this.
After further testing, I noticed that AVIF and GIF output doesn't work in the Alpine based Docker image, the former needs further investigation, the latter is due to the missing ImageMagick dependency. The upcoming libvips 8.12 has a
gifsaveoperation that uses cgif, so IM should then not be necessary (for saving GIF images at least).For now, I can recommend disabling these savers in the Alpine based Docker image by using a custom nginx configuration that includes:
weserv_savers jpg png webp tiff json; # Disable AVIF and GIF outputReacted by Giau Tran Minh
Current size of docker image is quite large. Can we reduce size to ~ 300MB for faster pull/push when deploy?