Why are the encoded-video and upload directories taking up space? #12698

ppskj178 · 2024-09-15T11:42:05Z

ppskj178
Sep 15, 2024

The bug

The problem is not resolved and you open an issue.
#12543

root@jellyfin-immich:~/library# du -sh *
58G     encoded-video
1.0K    library
1.0K    profile
21G     thumbs
65M     upload

The encoded-video folder is getting bigger and bigger even though I turned off video transcoding and no transcode jobs are happening in Jobs.

What should you check?

The OS that Immich Server is running on

Ubuntu 22.04.4 LTS

Version of Immich Server

v1.115.0

Version of Immich Mobile App

just server problem

Platform with the issue

Server
Web
Mobile

Your docker-compose.yml content

root@jellyfin-immich:~# cat docker-compose.yml 
#
# WARNING: Make sure to use the docker-compose.yml of the current release:
#
# https://github.com/immich-app/immich/releases/latest/download/docker-compose.yml
#
# The compose file on main may not be compatible with the latest release.
#

name: immich

services:
  immich-server:
    container_name: immich_server
    image: ghcr.io/immich-app/immich-server:${IMMICH_VERSION:-release}
    # extends:
    #   file: hwaccel.transcoding.yml
    #   service: cpu # set to one of [nvenc, quicksync, rkmpp, vaapi, vaapi-wsl] for accelerated transcoding
    devices:
      - /dev/dri:/dev/dri
    volumes:
      - ${UPLOAD_LOCATION}:/usr/src/app/upload
      - /etc/localtime:/etc/localtime:ro
      - /data/photo:/data/photo
    env_file:
      - .env
    ports:
      - 2283:3001
    depends_on:
      - redis
      - database
    restart: always

  immich-machine-learning:
    container_name: immich_machine_learning
    # For hardware acceleration, add one of -[armnn, cuda, openvino] to the image tag.
    # Example tag: ${IMMICH_VERSION:-release}-cuda
    image: ghcr.io/immich-app/immich-machine-learning:${IMMICH_VERSION:-release}
    # extends: # uncomment this section for hardware acceleration - see https://immich.app/docs/features/ml-hardware-acceleration
    #   file: hwaccel.ml.yml
    #   service: cpu # set to one of [armnn, cuda, openvino, openvino-wsl] for accelerated inference - use the `-wsl` version for WSL2 where applicable
    volumes:
      - model-cache:/cache
    env_file:
      - .env
    restart: always

  redis:
    container_name: immich_redis
    image: docker.io/redis:6.2-alpine@sha256:328fe6a5822256d065debb36617a8169dbfbd77b797c525288e465f56c1d392b
    healthcheck:
      test: redis-cli ping || exit 1
    restart: always

  database:
    container_name: immich_postgres
    image: docker.io/tensorchord/pgvecto-rs:pg14-v0.2.0@sha256:90724186f0a3517cf6914295b5ab410db9ce23190a2d9d0b9dd6463e3fa298f0
    environment:
      POSTGRES_PASSWORD: ${DB_PASSWORD}
      POSTGRES_USER: ${DB_USERNAME}
      POSTGRES_DB: ${DB_DATABASE_NAME}
      POSTGRES_INITDB_ARGS: '--data-checksums'
    volumes:
      - ${DB_DATA_LOCATION}:/var/lib/postgresql/data
    healthcheck:
      test: pg_isready --dbname='${DB_DATABASE_NAME}' --username='${DB_USERNAME}' || exit 1; Chksum="$$(psql --dbname='${DB_DATABASE_NAME}' --username='${DB_USERNAME}' --tuples-only --no-align --command='SELECT COALESCE(SUM(checksum_failures), 0) FROM pg_stat_database')"; echo "checksum failure count is $$Chksum"; [ "$$Chksum" = '0' ] || exit 1
      interval: 5m
      start_interval: 30s
      start_period: 5m
    command: ["postgres", "-c" ,"shared_preload_libraries=vectors.so", "-c", 'search_path="$$user", public, vectors', "-c", "logging_collector=on", "-c", "max_wal_size=2GB", "-c", "shared_buffers=512MB", "-c", "wal_compression=on"]
    restart: always

volumes:
  model-cache:

Your .env content

root@jellyfin-immich:~# cat .env 
# You can find documentation for all the supported env variables at https://immich.app/docs/install/environment-variables

# The location where your uploaded files are stored
UPLOAD_LOCATION=./library
# The location where your database files are stored
DB_DATA_LOCATION=./postgres

# To set a timezone, uncomment the next line and change Etc/UTC to a TZ identifier from this list: https://en.wikipedia.org/wiki/List_of_tz_database_time_zones#List
# TZ=Etc/UTC

# The Immich version to use. You can pin this to a specific version like "v1.71.0"
IMMICH_VERSION=v1.115.0

# Connection secret for postgres. You should change it to a random password
DB_PASSWORD=postgres

# The values below this line do not need to be changed
###################################################################################
DB_USERNAME=postgres
DB_DATABASE_NAME=immich

Reproduction steps

just use immich

Relevant log output

no logs

Additional information

No response

danieldietzler · 2024-09-15T11:48:07Z

danieldietzler
Sep 15, 2024
Maintainer

Do you have live/motion photos?

3 replies

ppskj178 Sep 15, 2024
Author

yes

ppskj178 Sep 15, 2024
Author

but very little
never 58G

ppskj178 Sep 16, 2024
Author

Is there anything else I can check?

ppskj178 · 2024-09-15T12:14:09Z

ppskj178
Sep 15, 2024
Author

I don't know if this log will help, but
I always get error logs on the server.

immich_server            |         "owner": true
immich_server            |     },
immich_server            |     "withDeleted": false
immich_server            | }
immich_server            |     at /usr/src/app/node_modules/typeorm/entity-manager/EntityManager.js:652:39
immich_server            |     at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
immich_server            |     at async LibraryService.handleQueueAssetRefresh (/usr/src/app/dist/services/library.service.js:502:25)
immich_server            |     at async /usr/src/app/dist/services/job.service.js:148:36
immich_server            |     at async Worker.processJob (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:394:28)
immich_server            |     at async Worker.retryIfFailed (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:581:24)
immich_server            | [Nest] 7  - 09/15/2024, 12:00:07 PM   ERROR [Microservices:JobService] Object:
immich_server            | {
immich_server            |   "id": "86fb818d-7286-410e-b2a2-971fece07139",
immich_server            |   "refreshModifiedFiles": true,
immich_server            |   "refreshAllFiles": false
immich_server            | }
immich_server            | 
immich_server            | [Nest] 7  - 09/15/2024, 12:00:08 PM     LOG [Microservices:LibraryService] Refreshing library 82300211-acb1-49f6-b0dc-c999eb0c5ccd
immich_server            | [Nest] 7  - 09/15/2024, 12:00:08 PM     LOG [Microservices:LibraryService] Refreshing library 3fa58c71-fe13-4054-9a11-f4c1ac6d8ced
immich_server            | [Nest] 7  - 09/15/2024, 12:00:08 PM     LOG [Microservices:LibraryService] Refreshing library 0eae7c67-4a5f-44f8-aeab-85a3b66459a2
immich_server            | [Nest] 7  - 09/15/2024, 12:00:11 PM     LOG [Microservices:LibraryService] Refreshing library 7fee2573-4ee6-4cad-8b4e-56225b4fbf4d
immich_server            | [Nest] 7  - 09/15/2024, 12:00:12 PM   ERROR [Microservices:JobService] Unable to run job handler (library/library-refresh): EntityNotFoundError: Could not find any entity of type "LibraryEntity" matching: {
immich_server            |     "where": {
immich_server            |         "id": "66b378b2-2b1f-475b-ae51-8d8b1233c618"
immich_server            |     },
immich_server            |     "relations": {
immich_server            |         "owner": true
immich_server            |     },
immich_server            |     "withDeleted": false
immich_server            | }
immich_server            | [Nest] 7  - 09/15/2024, 12:00:12 PM   ERROR [Microservices:JobService] EntityNotFoundError: Could not find any entity of type "LibraryEntity" matching: {
immich_server            |     "where": {
immich_server            |         "id": "66b378b2-2b1f-475b-ae51-8d8b1233c618"
immich_server            |     },
immich_server            |     "relations": {
immich_server            |         "owner": true
immich_server            |     },
immich_server            |     "withDeleted": false
immich_server            | }
immich_server            |     at /usr/src/app/node_modules/typeorm/entity-manager/EntityManager.js:652:39
immich_server            |     at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
immich_server            |     at async LibraryService.handleQueueAssetRefresh (/usr/src/app/dist/services/library.service.js:502:25)
immich_server            |     at async /usr/src/app/dist/services/job.service.js:148:36
immich_server            |     at async Worker.processJob (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:394:28)
immich_server            |     at async Worker.retryIfFailed (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:581:24)
immich_server            | [Nest] 7  - 09/15/2024, 12:00:12 PM   ERROR [Microservices:JobService] Object:
immich_server            | {
immich_server            |   "id": "66b378b2-2b1f-475b-ae51-8d8b1233c618",
immich_server            |   "refreshModifiedFiles": true,
immich_server            |   "refreshAllFiles": false
immich_server            | }
immich_server            | 
immich_server            | [Nest] 7  - 09/15/2024, 12:00:12 PM     LOG [Microservices:LibraryService] Refreshing library 3b90de40-84ad-487a-aee6-b9a24743fcd5
immich_server            | [Nest] 7  - 09/15/2024, 12:00:12 PM     LOG [Microservices:LibraryService] Finished queueing online check of 3008 assets for library 82300211-acb1-49f6-b0dc-c999eb0c5ccd
immich_server            | [Nest] 7  - 09/15/2024, 12:00:12 PM   ERROR [Microservices:JobService] Unable to run job handler (library/library-refresh): EntityNotFoundError: Could not find any entity of type "LibraryEntity" matching: {
immich_server            |     "where": {
immich_server            |         "id": "e234b208-1f0a-4c6a-9449-a57f3b6f6c08"
immich_server            |     },
immich_server            |     "relations": {
immich_server            |         "owner": true
immich_server            |     },
immich_server            |     "withDeleted": false
immich_server            | }
immich_server            | [Nest] 7  - 09/15/2024, 12:00:12 PM   ERROR [Microservices:JobService] EntityNotFoundError: Could not find any entity of type "LibraryEntity" matching: {
immich_server            |     "where": {
immich_server            |         "id": "e234b208-1f0a-4c6a-9449-a57f3b6f6c08"
immich_server            |     },
immich_server            |     "relations": {
immich_server            |         "owner": true
immich_server            |     },
immich_server            |     "withDeleted": false
immich_server            | }
immich_server            |     at /usr/src/app/node_modules/typeorm/entity-manager/EntityManager.js:652:39
immich_server            |     at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
immich_server            |     at async LibraryService.handleQueueAssetRefresh (/usr/src/app/dist/services/library.service.js:502:25)
immich_server            |     at async /usr/src/app/dist/services/job.service.js:148:36
immich_server            |     at async Worker.processJob (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:394:28)
immich_server            |     at async Worker.retryIfFailed (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:581:24)
immich_server            | [Nest] 7  - 09/15/2024, 12:00:12 PM   ERROR [Microservices:JobService] Object:
immich_server            | {
immich_server            |   "id": "e234b208-1f0a-4c6a-9449-a57f3b6f6c08",
immich_server            |   "refreshModifiedFiles": true,
immich_server            |   "refreshAllFiles": false
immich_server            | }
immich_server            |

0 replies

bo0tzz · 2024-09-15T12:21:39Z

bo0tzz
Sep 15, 2024
Maintainer

Please post your external library settings.

8 replies

ppskj178 Sep 15, 2024
Author

My external libraries are mounted as NFS and for some reason they are often disconnected.

ppskj178 Sep 16, 2024
Author

Is there anything else I can check?

danieldietzler Sep 16, 2024
Maintainer

The DB isn't on a network mount?

ppskj178 Sep 16, 2024
Author

The DB isn't on a network mount?

db is local
i do docker compose up in home dir
and mount nfs path /data/photo

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Why are the encoded-video and upload directories taking up space? #12698

{{title}}

Replies: 3 comments 11 replies

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{title}}

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

Select a reply

Why are the encoded-video and upload directories taking up space? #12698

ppskj178 Sep 15, 2024

The bug

The OS that Immich Server is running on

Version of Immich Server

Version of Immich Mobile App

Platform with the issue

Your docker-compose.yml content

Your .env content

Reproduction steps

Relevant log output

Additional information

Replies: 3 comments · 11 replies

danieldietzler Sep 15, 2024 Maintainer

ppskj178 Sep 15, 2024 Author

ppskj178 Sep 15, 2024 Author

ppskj178 Sep 16, 2024 Author

ppskj178 Sep 15, 2024 Author

bo0tzz Sep 15, 2024 Maintainer

ppskj178 Sep 15, 2024 Author

ppskj178 Sep 15, 2024 Author

ppskj178 Sep 16, 2024 Author

danieldietzler Sep 16, 2024 Maintainer

ppskj178 Sep 16, 2024 Author

ppskj178
Sep 15, 2024

Replies: 3 comments 11 replies

danieldietzler
Sep 15, 2024
Maintainer

ppskj178 Sep 15, 2024
Author

ppskj178 Sep 15, 2024
Author

ppskj178 Sep 16, 2024
Author

ppskj178
Sep 15, 2024
Author

bo0tzz
Sep 15, 2024
Maintainer

ppskj178 Sep 15, 2024
Author

ppskj178 Sep 15, 2024
Author

ppskj178 Sep 16, 2024
Author

danieldietzler Sep 16, 2024
Maintainer

ppskj178 Sep 16, 2024
Author