[Bug] immich microservice memory leak kills host #9414

Dunky-Z · 2024-05-13T02:21:09Z

The bug

Today, I encountered the same issue #5283 . After bulk importing around 10,000 photos(By setting the external library), I suddenly couldn't remotely connect to my NAS host. Initially, I suspected that the ML service was causing high memory usage during facial recognition, so I forcibly restarted the NAS and reconfigured concurrency settings, setting all jobs to run on a single thread. I didn't start all the jobs at once; instead, I sequentially ran FACE DETECTION and then GENERATE THUMBNAILS. There were no abnormalities during the FACE DETECTION process, indicating that the issue wasn't caused by the ML operations. After completing all FACE DETECTIONS, I initiated the GENERATE THUMBNAILS task, which ran slowly due to the large image sizes. Without continuous monitoring, upon returning to check on the NAS, I found it was unreachable remotely, with the system log displaying the following error messages:

[17452.279485] Out of memory: Killed process 24344 (immich_microser) total-vm:14375660kB, anon-rss:8827768kB, file-rss:0kB, shmem-rss:0kB, UID:0 pgtables:23516kB oom_score_adj:0
[17653.570542] Out of memory: Killed process 24972 (immich_microser) total-vm:13119600kB, anon-rss:8825796kB, file-rss:0kB, shmem-rss:0kB, UID:0 pgtables:20652kB oom_score_adj:0
[17848.345248] Out of memory: Killed process 26162 (immich_microser) total-vm:19086052kB, anon-rss:8709080kB, file-rss:0kB, shmem-rss:0kB, UID:0 pgtables:31688kB oom_score_adj:0
[17932.668353] Out of memory: Killed process 27348 (immich_microser) total-vm:13774836kB, anon-rss:8759228kB, file-rss:0kB, shmem-rss:0kB, UID:0 pgtables:22648kB oom_score_adj:0
[18001.034451] Out of memory: Killed process 27902 (immich_microser) total-vm:14749840kB, anon-rss:9111036kB, file-rss:0kB, shmem-rss:0kB, UID:0 pgtables:25104kB oom_score_adj:0

This suggests that there might be a memory leak issue within the immich_microser service. I would greatly appreciate any advice on how to address this problem.

The OS that Immich Server is running on

Debian(OMV)

Version of Immich Server

OpenMediaVault 5

Version of Immich Mobile App

v1.103.1

Platform with the issue

Server
Web
Mobile

Your docker-compose.yml content

#
# WARNING: Make sure to use the docker-compose.yml of the current release:
#
# https://github.com/immich-app/immich/releases/latest/download/docker-compose.yml
#
# The compose file on main may not be compatible with the latest release.
#

name: immich

services:
  immich-server:
    container_name: immich_server
    image: ghcr.io/immich-app/immich-server:${IMMICH_VERSION:-release}
    command: ['start.sh', 'immich']
    volumes:
      - ${UPLOAD_LOCATION}:/usr/src/app/upload
      - /etc/localtime:/etc/localtime:ro
      - "/srv/share/album:/mnt/media/share/album:ro"
    env_file:
      - .env
    ports:
      - 2283:3001
    depends_on:
      - redis
      - database
    restart: always

  immich-microservices:
    container_name: immich_microservices
    image: ghcr.io/immich-app/immich-server:${IMMICH_VERSION:-release}
    # extends: # uncomment this section for hardware acceleration - see https://immich.app/docs/features/hardware-transcoding
    #   file: hwaccel.transcoding.yml
    #   service: cpu # set to one of [nvenc, quicksync, rkmpp, vaapi, vaapi-wsl] for accelerated transcoding
    command: ['start.sh', 'microservices']
    volumes:
      - ${UPLOAD_LOCATION}:/usr/src/app/upload
      - /etc/localtime:/etc/localtime:ro
      - "/srv/share/album:/mnt/media/share/album:ro"
    env_file:
      - .env
    depends_on:
      - redis
      - database
    restart: always

  immich-machine-learning:
    container_name: immich_machine_learning
    # For hardware acceleration, add one of -[armnn, cuda, openvino] to the image tag.
    # Example tag: ${IMMICH_VERSION:-release}-cuda
    image: ghcr.io/immich-app/immich-machine-learning:${IMMICH_VERSION:-release}
    # extends: # uncomment this section for hardware acceleration - see https://immich.app/docs/features/ml-hardware-acceleration
    #   file: hwaccel.ml.yml
    #   service: cpu # set to one of [armnn, cuda, openvino, openvino-wsl] for accelerated inference - use the `-wsl` version for WSL2 where applicable
    volumes:
      - "/srv/appdata/immich/immich-cache:/cache"
    env_file:
      - .env
    restart: always

  redis:
    container_name: immich_redis
    image: registry.hub.docker.com/library/redis:6.2-alpine@sha256:51d6c56749a4243096327e3fb964a48ed92254357108449cb6e23999c37773c5
    restart: always

  database:
    container_name: immich_postgres
    image: registry.hub.docker.com/tensorchord/pgvecto-rs:pg14-v0.2.0@sha256:90724186f0a3517cf6914295b5ab410db9ce23190a2d9d0b9dd6463e3fa298f0
    environment:
      POSTGRES_PASSWORD: ${DB_PASSWORD}
      POSTGRES_USER: ${DB_USERNAME}
      POSTGRES_DB: ${DB_DATABASE_NAME}
    volumes:
      - "/srv/appdata/immich/immich-postgresql/data:/var/lib/postgresql/data"
    restart: always

Your .env content

# You can find documentation for all the supported env variables at https://immich.app/docs/install/environment-variables

# The location where your uploaded files are stored
UPLOAD_LOCATION=./library

# The Immich version to use. You can pin this to a specific version like "v1.71.0"
IMMICH_VERSION=release

# Connection secret for postgres. You should change it to a random password
DB_PASSWORD=postgres

# The values below this line do not need to be changed
###################################################################################
DB_HOSTNAME=immich_postgres
DB_USERNAME=postgres
DB_DATABASE_NAME=immich

REDIS_HOSTNAME=immich_redis

Reproduction steps

1. Add an external library that contains the over 10000 pics
2. Wait for immich to generate thumbnails for the photo

Relevant log output

No response

Additional information

No response

The text was updated successfully, but these errors were encountered:

mertalev · 2024-05-13T04:09:39Z

How much RAM does the server have, and how much is used just after starting thumbnail generation? An increase in memory usage for some period of time during thumbnail generation is normal because of memory fragmentation, but it should plateau after a certain point. If you have any RAW images, they will amplify this effect since they require much more memory.

Dunky-Z · 2024-05-13T04:58:46Z

Thank you for your response. The server has a total of 16GB of RAM, and its typical daily memory usage hovers around 6GB. After initiating thumbnail generation, it doesn't immediately consume all memory but gradually fills it up until the program crashes. Your mention of RAW images did remind me that the majority of my gallery consists of RAW files. Is there a way to prevent excessive memory usage in this scenario? I have tried limiting the microservice using Cgroups, but when the limit is exceeded, the microservice restarts. Since I bind the Cgroup settings to the PID, once the microservice restarts, PID changed, the previous Cgroup configurations become ineffective.

Dunky-Z · 2024-05-13T05:01:21Z

Given the server's average performance, I have configured all my JOBS to use a single thread, and only one JOB is executed at a time. In theory, processing a single RAW file shouldn't require such an excessive amount of memory.

mertalev · 2024-05-13T05:18:40Z

Hmm, that much of an increase is unexpected. It could be related to the issue behind #6542, or possibly #4391. Are there any errors in the microservices logs before the OOM error?

As far as limiting memory usage in the meantime, you can set a limit through Docker, which will force the container to be restarted after reaching a certain usage.

Dunky-Z · 2024-05-13T13:02:29Z

Below are some logs from the microservices; perhaps they might provide some insight:

[Nest] 7  - 05/13/2024, 11:34:38 AM     LOG [EventRepository] WebSocket server initialized.
[Nest] 7  - 05/13/2024, 11:35:06 AM   ERROR [JobService] Failed to execute job handler (thumbnailGeneration/generate-preview): Error: The input file contains an unsupported image format
[Nest] 7  - 05/13/2024, 11:35:06 AM   ERROR [JobService] Error: The input file contains an unsupported image format
    at Sharp.toFile (/usr/src/app/node_modules/sharp/lib/output.js:89:19)
    at MediaRepository.resize (/usr/src/app/dist/repositories/media.repository.js:76:14)
    at MediaService.generateThumbnail (/usr/src/app/dist/services/media.service.js:156:48)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async MediaService.handleGeneratePreview (/usr/src/app/dist/services/media.service.js:134:29)
    at async /usr/src/app/dist/services/job.service.js:149:36
    at async Worker.processJob (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:394:28)
    at async Worker.retryIfFailed (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:581:24)
[Nest] 7  - 05/13/2024, 11:35:06 AM   ERROR [JobService] Affected Job ID:
{
  "id": "5802ddbb-4e01-47fb-b21b-73bc0361f9e8"
}

[Nest] 7  - 05/13/2024, 11:35:07 AM   ERROR [JobService] Failed to execute job handler (thumbnailGeneration/generate-preview): Error: ffprobe exited with code 1
ffprobe version 6.0.1-Jellyfin Copyright (c) 2007-2023 the FFmpeg developers
  Built with gcc 12 (Debian 12.2.0-14)
  Configuration: --prefix=/usr/lib/jellyfin-ffmpeg --target-os=linux --extra-version=Jellyfin --disable-doc --disable-ffplay --disable-ptx-compression --disable-static --disable-libxcb --disable-sdl2 --disable-xlib --enable-lto --enable-gpl --enable-version3 --enable-shared --enable-gmp --enable-gnutls --enable-chromaprint --enable-opencl --enable-libdrm --enable-libass --enable-libfreetype --enable-libfribidi --enable-libfontconfig --enable-libbluray --enable-libmp3lame --enable-libopus --enable-libtheora --enable-libvorbis --enable-libopenmpt --enable-libdav1d --enable-libsvtav1 --enable-libwebp --enable-libvpx --enable-libx264 --enable-libx265 --enable-libzvbi --enable-libzimg --enable-libfdk-aac --arch=amd64 --enable-libshaderc --enable-libplacebo --enable-vulkan --enable-vaapi --enable-amf --enable-libvpl --enable-ffnvcodec --enable-cuda --enable-cuda-llvm --enable-cuvid --enable-nvdec --enable-nvenc
  libavutil      58.  2.100 / 58.  2.100
  libavcodec     60.  3.100 / 60.  3.100
  libavformat    60.  3.100 / 60.  3.100
  libavdevice    60.  1.100 / 60.  1.100
  libavfilter     9.  3.100 /  9.  3.100
  libswscale      7.  1.100 /  7.  1.100
  libswresample   4. 10.100 /  4. 10.100
  libpostproc    57.  1.100 / 57.  1.100
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x36da0190180] Invalid sample size -1008
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x36da0190180] Error reading header
upload/upload/98a61f7d-3957-465a-b946-79697c36d3b1/59/5d/595d905e-3bde-49ef-a699-17b677622b3c.mp4: Invalid data found when processing input

[Nest] 7  - 05/13/2024, 11:35:07 AM   ERROR [JobService] Error: ffprobe exited with code 1
ffprobe version 6.0.1-Jellyfin Copyright (c) 2007-2023 the FFmpeg developers
  Built with gcc 12 (Debian 12.2.0-14)
  Configuration: --prefix=/usr/lib/jellyfin-ffmpeg --target-os=linux --extra-version=Jellyfin --disable-doc --disable-ffplay --disable-ptx-compression --disable-static --disable-libxcb --disable-sdl2 --disable-xlib --enable-lto --enable-gpl --enable-version3 --enable-shared --enable-gmp --enable-gnutls --enable-chromaprint --enable-opencl --enable-libdrm --enable-libass --enable-libfreetype --enable-libfribidi --enable-libfontconfig --enable-libbluray --enable-libmp3lame --enable-libopus --enable-libtheora --enable-libvorbis --enable-libopenmpt --enable-libdav1d --enable-libsvtav1 --enable-libwebp --enable-libvpx --enable-libx264 --enable-libx265 --enable-libzvbi --enable-libzimg --enable-libfdk-aac --arch=amd64 --enable-libshaderc --enable-libplacebo --enable-vulkan --enable-vaapi --enable-amf --enable-libvpl --enable-ffnvcodec --enable-cuda --enable-cuda-llvm --enable-cuvid --enable-nvdec --enable-nvenc
  libavutil      58.  2.100 / 58.  2.100
  libavcodec     60.  3.100 / 60.  3.100
  libavformat    60.  3.100 / 60.  100
  libavdevice    60.  1.100 / 60.  1.100
  libavfilter     9.  3.100 /  9.  3.100
  libswscale      7.  1.100 /  7.  1.100
  libswresample   4. 10.100 /  4. 10.100
  libpostproc    57.  1.100 / 57.  1.100
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x36da0190180] Invalid sample size -1008
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x36da0190180] Error reading header
upload/upload/98a61f7d-3957-465a-b946-79697c36d3b1/59/5d/595d905e-3bde-49ef-a699-17b677622b3c.mp4: Invalid data found when processing input

    at ChildProcess.<anonymous> (/usr/src/app/node_modules/fluent-ffmpeg/lib/ffprobe.js:233:22)
    at ChildProcess.emit (node:events:518:28)
    at ChildProcess._handle.onexit (node:internal/child_process:294:12)
[Nest] 7  - 05/13/2024, 11:35:07 AM   ERROR [JobService] Affected Job ID:
{
  "id": "ee18e5e8-5430-4fcb-aa10-ef4170e2b30e"
}

Limiting the container's resource usage has indeed helped me; it prevents my server from crashing. However, upon examining the microservices' logs, I noticed frequent restarts, indicating that the microservices are indeed exceeding the memory limit. Even after setting a 4GB memory limit, it's still insufficient.

Here's the relevant part of my Docker Compose configuration:

  immich-microservices:
    container_name: immich_microservices
    image: altran1502/immich-server:${IMMICH_VERSION:-release}
    command: ['start.sh', 'microservices']
    volumes:
      - ${UPLOAD_LOCATION}:/usr/src/app/upload
      - "/root/sharedfolder/syncthing/Photo_Album:/mnt/media/Photo_Album:ro"
      - /etc/localtime:/etc/localtime:ro
    devices:
      - /dev/dri:/dev/dri
    env_file:
      - .env
    depends_on:
      - redis
      - database
    restart: always
    deploy:
      resources:
        limits:
          cpus: '2'
          memory: 4G

[Nest] 7  - 05/13/2024, 11:35:19 AM     LOG [EventRepository] Initialized websocket server
[Nest] 7  - 05/13/2024, 11:35:33 AM     LOG [EventRepository] Initialized websocket server
[Nest] 7  - 05/13/2024, 11:35:45 AM     LOG [EventRepository] Initialized websocket server
[Nest] 7  - 05/13/2024, 11:35:58 AM     LOG [EventRepository] Initialized websocket server
[Nest] 7  - 05/13/2024, 11:36:11 AM     LOG [EventRepository] Initialized websocket server
[Nest] 7  - 05/13/2024, 11:36:53 AM     LOG [EventRepository] Initialized websocket server
[Nest] 7  - 05/13/2024, 11:37:06 AM     LOG [EventRepository] Initialized websocket server
[Nest] 7  - 05/13/2024, 11:37:19 AM     LOG [EventRepository] Initialized websocket server
[Nest] 7  - 05/13/2024, 11:37:34 AM     LOG [EventRepository] Initialized websocket server
[Nest] 7  - 05/13/2024, 11:37:47 AM     LOG [EventRepository] Initialized websocket server

I'm still hoping to identify and resolve the root cause entirely. If more information is needed, please let me know.

On a side note, how can I persistently save Immich logs? The log path isn't mentioned in the user documentation, so I haven't mapped any volumes, resulting in log loss upon container restarts.

Thank you very much！

ceebu · 2024-05-17T16:02:34Z

I have the same problem (i5 8th gen server, running OMV 6) - am also Looking for a easy way to save immich logs -

RazerProof · 2024-05-22T13:44:07Z

I can also confirm this issue is affecting the installation on my NAS. Limiting memory causes frequent restarts of the microservices container. I also tried setting the Generate Thumbnails Concurrency to 1, but it didn't help. I found the NAS started overutalising the drives as they were at 100% utilisation. I imagine it was paging out memory to disk; CPU utilization was around 40%. I have increased the RAM in the NAS to 32GB, which has solved the problem for me, and the microservices container now runs constantly, with no restarting at all. The microservices container stabilised at around 28GB, the NAS memory resource monitor shows almost all of that usage is cache memory, the container limit is set to 4GB.
After restarting the container, it takes about 15-20min to stabilise at this level. Disk usage is now around 40% and CPU is around 80%, generating thumbnails at around 30/min. This is with no other jobs running.
I have around 230k assets in the external library, 50%jpg, 50%RAW. I also tried this in docker on a server (TrueNAS scale) and experienced the same issue and outcome.
Overall, I love the way Immich is going. It is going to be a great product. Well done to everyone involved, and thanks for the great work.

mertalev · 2024-05-22T13:59:47Z

Thanks for the detailed info!

How many of those assets are RAW, if any?
Do you have /tmp configured to be in-memory?
Is the increase completely linear, or are there spikes?
Any errors in the logs?

RazerProof · 2024-05-22T14:14:05Z

How many of those assets are RAW, if any? 50% of the images are RAW around 50GB each.
Do you have /tmp configured to be in-memory? No /tmp configured.
Is the increase completely linear, or are there spikes? Mostly linear overlayed with a small saw tooth shape.

I just restarted the container. That smooth section is unusual, and it will go back to the sawtooth.
Any errors in the logs? No Errors, running smoothly.

RazerProof · 2024-05-22T14:38:36Z

Below is an example of the errors I was receiving on the NAS when 4GB was installed and getting constant restarts of the microservices container.

mertalev · 2024-05-22T14:51:42Z

I made a test image for microservices with a possible fix: ghcr.io/immich-app/immich-server:pr-9665. Would you be able to change your image to that and see if it affects RAM usage?

RazerProof · 2024-05-22T15:13:55Z

Not good news I am afraid. The immich_microservices fails.

immich_microservices
date stream content
2024/05/23 01:09:52 stderr Microservices worker exited with code 1

2024/05/23 01:09:52 stderr }

2024/05/23 01:09:52 stderr routine: 'parserOpenTable'

2024/05/23 01:09:52 stderr line: '1381',
2024/05/23 01:09:52 stderr file: 'parse_relation.c',
2024/05/23 01:09:52 stderr constraint: undefined,
2024/05/23 01:09:52 stderr dataType: undefined,
2024/05/23 01:09:52 stderr column: undefined,
2024/05/23 01:09:52 stderr table: undefined,
2024/05/23 01:09:52 stderr schema: undefined,
2024/05/23 01:09:52 stderr where: undefined,
2024/05/23 01:09:52 stderr internalQuery: undefined,
2024/05/23 01:09:52 stderr internalPosition: undefined,
2024/05/23 01:09:52 stderr position: '128',
2024/05/23 01:09:52 stderr hint: undefined,
2024/05/23 01:09:52 stderr detail: undefined,
2024/05/23 01:09:52 stderr code: '42P01',
2024/05/23 01:09:52 stderr severity: 'ERROR',
2024/05/23 01:09:52 stderr length: 113,
2024/05/23 01:09:52 stderr },
2024/05/23 01:09:52 stderr routine: 'parserOpenTable'

2024/05/23 01:09:52 stderr line: '1381',
2024/05/23 01:09:52 stderr file: 'parse_relation.c',
2024/05/23 01:09:52 stderr constraint: undefined,
2024/05/23 01:09:52 stderr dataType: undefined,
2024/05/23 01:09:52 stderr column: undefined,
2024/05/23 01:09:52 stderr table: undefined,
2024/05/23 01:09:52 stderr schema: undefined,
2024/05/23 01:09:52 stderr where: undefined,
2024/05/23 01:09:52 stderr internalQuery: undefined,
2024/05/23 01:09:52 stderr internalPosition: undefined,
2024/05/23 01:09:52 stderr position: '128',
2024/05/23 01:09:52 stderr hint: undefined,
2024/05/23 01:09:52 stderr detail: undefined,
2024/05/23 01:09:52 stderr code: '42P01',
2024/05/23 01:09:52 stderr severity: 'ERROR',
2024/05/23 01:09:52 stderr length: 113,

mertalev · 2024-05-22T15:23:23Z

Hmm, that error is about connecting to Postgres, not related to thumbnail generation.

RazerProof · 2024-05-22T15:25:08Z

Yeah... Something may have gone wrong with the deployment. Let me do some more testing, and maybe get some sleep :-).

mertalev · 2024-05-22T17:29:55Z

So sorry! I based that branch off of the latest release, but it turns out that main gets merged into it anyway when the image is built. The error is probably because of that.

You can either wait for the next release to get things back up or restore from a backup. (It's also possible to mess with it more to get it back up, but I think these options are safer.)

RazerProof · 2024-05-22T18:02:43Z

Thanks for letting me know and I can confirm I am getting the same result.

It is working, and there are no errors in the log; it's just hungry :-)
I will wait for the next release and re-test
Thanks for all your great work.

ThisIsCyreX · 2024-06-12T20:31:18Z

Same issue here, worked great for some time until I noticed the slow ssh and docker container restarting on its own.
I re-created all thumbnails and thought maybe that will help, The job ran without issue always ~1gig memory. From 200k, the last 60 or so thumbnails are the problem I guess. Fills the memory and the log says this:

[Nest] 18 - 06/12/2024, 9:53:14 PM ERROR [Api:ExceptionsHandler~wx0506ns] Connection terminated due to connection timeout
Error: Connection terminated due to connection timeout
at Connection.<anonymous> (/usr/src/app/node_modules/pg/lib/client.js:132:73)
at Object.onceWrapper (node:events:633:28)
at Connection.emit (node:events:519:28)
at Socket.<anonymous> (/usr/src/app/node_modules/pg/lib/connection.js:63:12)
at Socket.emit (node:events:519:28)
at TCP.<anonymous> (node:net:338:12)
at TCP.callbackTrampoline (node:internal/async_hooks:130:17)
Detected CPU Cores: 6
Starting api worker
Starting microservices worker
[Nest] 7 - 06/12/2024, 9:53:44 PM LOG [Microservices:EventRepository] Initialized websocket server
[Nest] 7 - 06/12/2024, 9:53:45 PM LOG [Microservices:SystemConfigService] LogLevel=verbose (set via system config)

(long!) (click me) And this is the log after starting thumbnails generation "missing".

[Nest] 17 - 06/12/2024, 10:08:57 PM DEBUG [Api:LoggingInterceptor~qzqsrgdk] PUT /api/jobs/thumbnailGeneration 200 13.94ms 192.168.0.201
[Nest] 17 - 06/12/2024, 10:08:57 PM VERBOSE [Api:LoggingInterceptor~qzqsrgdk] {"command":"start","force":false}
[Nest] 17 - 06/12/2024, 10:08:57 PM DEBUG [Api:LoggingInterceptor~ayp6dm8q] GET /api/jobs 200 4.07ms 192.168.0.201
[Nest] 7 - 06/12/2024, 10:08:58 PM VERBOSE [Microservices:PersonService] Cropping face for person: 0cb04746-b03c-4029-9f92-047160d1d2f2
[Nest] 7 - 06/12/2024, 10:08:58 PM ERROR [Microservices:JobService] Unable to run job handler (thumbnailGeneration/generate-person-thumbnail): Error: Input file contains unsupported image format
[Nest] 7 - 06/12/2024, 10:08:58 PM ERROR [Microservices:JobService] Error: Input file contains unsupported image format
at Sharp.toFile (/usr/src/app/node_modules/sharp/lib/output.js:89:19)
at MediaRepository.generateThumbnail (/usr/src/app/dist/repositories/media.repository.js:69:14)
at PersonService.handleGeneratePersonThumbnail (/usr/src/app/dist/services/person.service.js:429:36)
at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
at async /usr/src/app/dist/services/job.service.js:148:36
at async Worker.processJob (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:394:28)
at async Worker.retryIfFailed (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:581:24)
[Nest] 7 - 06/12/2024, 10:08:58 PM ERROR [Microservices:JobService] Object:
{
"id": "0cb04746-b03c-4029-9f92-047160d1d2f2"
}

    all values are hidden

[Nest] 7 - 06/12/2024, 10:08:58 PM VERBOSE [Microservices:PersonService] Cropping face for person: 53791cd7-3656-4372-b7c8-c247704fe454
[Nest] 7 - 06/12/2024, 10:08:58 PM ERROR [Microservices:JobService] Unable to run job handler (thumbnailGeneration/generate-person-thumbnail): Error: Input file contains unsupported image format
[Nest] 7 - 06/12/2024, 10:08:58 PM ERROR [Microservices:JobService] Error: Input file contains unsupported image format
at Sharp.toFile (/usr/src/app/node_modules/sharp/lib/output.js:89:19)
at MediaRepository.generateThumbnail (/usr/src/app/dist/repositories/media.repository.js:69:14)
at PersonService.handleGeneratePersonThumbnail (/usr/src/app/dist/services/person.service.js:429:36)
at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
at async /usr/src/app/dist/services/job.service.js:148:36
at async Worker.processJob (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:394:28)
at async Worker.retryIfFailed (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:581:24)
[Nest] 7 - 06/12/2024, 10:08:58 PM ERROR [Microservices:JobService] Object:
{
"id": "53791cd7-3656-4372-b7c8-c247704fe454"
}

    all values are hidden

[Nest] 7 - 06/12/2024, 10:08:58 PM VERBOSE [Microservices:PersonService] Cropping face for person: ecb5278a-f33d-4493-98f8-28bba1e390eb
[Nest] 7 - 06/12/2024, 10:08:58 PM ERROR [Microservices:JobService] Unable to run job handler (thumbnailGeneration/generate-person-thumbnail): Error: Input file contains unsupported image format
[Nest] 7 - 06/12/2024, 10:08:58 PM ERROR [Microservices:JobService] Error: Input file contains unsupported image format
at Sharp.toFile (/usr/src/app/node_modules/sharp/lib/output.js:89:19)
at MediaRepository.generateThumbnail (/usr/src/app/dist/repositories/media.repository.js:69:14)
at PersonService.handleGeneratePersonThumbnail (/usr/src/app/dist/services/person.service.js:429:36)
at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
at async /usr/src/app/dist/services/job.service.js:148:36
at async Worker.processJob (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:394:28)
at async Worker.retryIfFailed (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:581:24)
[Nest] 7 - 06/12/2024, 10:08:58 PM ERROR [Microservices:JobService] Object:
{
"id": "ecb5278a-f33d-4493-98f8-28bba1e390eb"
}

    all values are hidden

[Nest] 7 - 06/12/2024, 10:08:58 PM VERBOSE [Microservices:PersonService] Cropping face for person: ec72944f-2ad1-4529-b565-1e58e2a73727
[Nest] 7 - 06/12/2024, 10:08:58 PM ERROR [Microservices:JobService] Unable to run job handler (thumbnailGeneration/generate-person-thumbnail): Error: Input file contains unsupported image format
[Nest] 7 - 06/12/2024, 10:08:58 PM ERROR [Microservices:JobService] Error: Input file contains unsupported image format
at Sharp.toFile (/usr/src/app/node_modules/sharp/lib/output.js:89:19)
at MediaRepository.generateThumbnail (/usr/src/app/dist/repositories/media.repository.js:69:14)
at PersonService.handleGeneratePersonThumbnail (/usr/src/app/dist/services/person.service.js:429:36)
at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
at async /usr/src/app/dist/services/job.service.js:148:36
at async Worker.processJob (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:394:28)
at async Worker.retryIfFailed (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:581:24)
[Nest] 7 - 06/12/2024, 10:08:58 PM ERROR [Microservices:JobService] Object:
{
"id": "ec72944f-2ad1-4529-b565-1e58e2a73727"
}

    all values are hidden

[Nest] 7 - 06/12/2024, 10:08:58 PM VERBOSE [Microservices:PersonService] Cropping face for person: a4b18b05-1b5f-429a-b478-528451246073
[Nest] 7 - 06/12/2024, 10:08:58 PM ERROR [Microservices:JobService] Unable to run job handler (thumbnailGeneration/generate-person-thumbnail): Error: Input file contains unsupported image format
[Nest] 7 - 06/12/2024, 10:08:58 PM ERROR [Microservices:JobService] Error: Input file contains unsupported image format
at Sharp.toFile (/usr/src/app/node_modules/sharp/lib/output.js:89:19)
at MediaRepository.generateThumbnail (/usr/src/app/dist/repositories/media.repository.js:69:14)
at PersonService.handleGeneratePersonThumbnail (/usr/src/app/dist/services/person.service.js:429:36)
at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
at async /usr/src/app/dist/services/job.service.js:148:36
at async Worker.processJob (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:394:28)
at async Worker.retryIfFailed (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:581:24)
[Nest] 7 - 06/12/2024, 10:08:58 PM ERROR [Microservices:JobService] Object:
{
"id": "a4b18b05-1b5f-429a-b478-528451246073"
}

    all values are hidden

[Nest] 7 - 06/12/2024, 10:08:58 PM VERBOSE [Microservices:PersonService] Cropping face for person: 54013ed0-c3d7-49a5-84f7-3f8ee6879fc3
[Nest] 7 - 06/12/2024, 10:08:59 PM ERROR [Microservices:JobService] Unable to run job handler (thumbnailGeneration/generate-person-thumbnail): Error: Input file contains unsupported image format
[Nest] 7 - 06/12/2024, 10:08:59 PM ERROR [Microservices:JobService] Error: Input file contains unsupported image format
at Sharp.toFile (/usr/src/app/node_modules/sharp/lib/output.js:89:19)
at MediaRepository.generateThumbnail (/usr/src/app/dist/repositories/media.repository.js:69:14)
at PersonService.handleGeneratePersonThumbnail (/usr/src/app/dist/services/person.service.js:429:36)
at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
at async /usr/src/app/dist/services/job.service.js:148:36
at async Worker.processJob (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:394:28)
at async Worker.retryIfFailed (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:581:24)
[Nest] 7 - 06/12/2024, 10:08:59 PM ERROR [Microservices:JobService] Object:
{
"id": "54013ed0-c3d7-49a5-84f7-3f8ee6879fc3"
}

    all values are hidden

[Nest] 7 - 06/12/2024, 10:08:59 PM VERBOSE [Microservices:PersonService] Cropping face for person: 19d6ceaa-1899-480f-ad2c-879dcbd5a048
[Nest] 7 - 06/12/2024, 10:08:59 PM ERROR [Microservices:JobService] Unable to run job handler (thumbnailGeneration/generate-person-thumbnail): Error: Input file contains unsupported image format
[Nest] 7 - 06/12/2024, 10:08:59 PM ERROR [Microservices:JobService] Error: Input file contains unsupported image format
at Sharp.toFile (/usr/src/app/node_modules/sharp/lib/output.js:89:19)
at MediaRepository.generateThumbnail (/usr/src/app/dist/repositories/media.repository.js:69:14)
at PersonService.handleGeneratePersonThumbnail (/usr/src/app/dist/services/person.service.js:429:36)
at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
at async /usr/src/app/dist/services/job.service.js:148:36
at async Worker.processJob (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:394:28)
at async Worker.retryIfFailed (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:581:24)
[Nest] 7 - 06/12/2024, 10:08:59 PM ERROR [Microservices:JobService] Object:
{
"id": "19d6ceaa-1899-480f-ad2c-879dcbd5a048"
}

    all values are hidden

[Nest] 7 - 06/12/2024, 10:08:59 PM VERBOSE [Microservices:PersonService] Cropping face for person: be6cd6d9-f992-4d13-8fa0-ea8eba655075
[Nest] 7 - 06/12/2024, 10:08:59 PM ERROR [Microservices:JobService] Unable to run job handler (thumbnailGeneration/generate-person-thumbnail): Error: Input file contains unsupported image format
[Nest] 7 - 06/12/2024, 10:08:59 PM ERROR [Microservices:JobService] Error: Input file contains unsupported image format
at Sharp.toFile (/usr/src/app/node_modules/sharp/lib/output.js:89:19)
at MediaRepository.generateThumbnail (/usr/src/app/dist/repositories/media.repository.js:69:14)
at PersonService.handleGeneratePersonThumbnail (/usr/src/app/dist/services/person.service.js:429:36)
at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
at async /usr/src/app/dist/services/job.service.js:148:36
at async Worker.processJob (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:394:28)
at async Worker.retryIfFailed (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:581:24)
[Nest] 7 - 06/12/2024, 10:08:59 PM ERROR [Microservices:JobService] Object:
{
"id": "be6cd6d9-f992-4d13-8fa0-ea8eba655075"
}

    all values are hidden

[Nest] 7 - 06/12/2024, 10:08:59 PM VERBOSE [Microservices:PersonService] Cropping face for person: 20f02f50-6da6-4ff5-9d2a-af5b95701b92
[Nest] 7 - 06/12/2024, 10:08:59 PM ERROR [Microservices:JobService] Unable to run job handler (thumbnailGeneration/generate-person-thumbnail): Error: Input file contains unsupported image format
[Nest] 7 - 06/12/2024, 10:08:59 PM ERROR [Microservices:JobService] Error: Input file contains unsupported image format
at Sharp.toFile (/usr/src/app/node_modules/sharp/lib/output.js:89:19)
at MediaRepository.generateThumbnail (/usr/src/app/dist/repositories/media.repository.js:69:14)
at PersonService.handleGeneratePersonThumbnail (/usr/src/app/dist/services/person.service.js:429:36)
at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
at async /usr/src/app/dist/services/job.service.js:148:36
at async Worker.processJob (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:394:28)
at async Worker.retryIfFailed (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:581:24)
[Nest] 7 - 06/12/2024, 10:08:59 PM ERROR [Microservices:JobService] Object:
{
"id": "20f02f50-6da6-4ff5-9d2a-af5b95701b92"
}

    all values are hidden

[Nest] 7 - 06/12/2024, 10:08:59 PM VERBOSE [Microservices:PersonService] Cropping face for person: 6f697b7a-8687-4806-b079-0cb1016cd6aa
[Nest] 7 - 06/12/2024, 10:08:59 PM ERROR [Microservices:JobService] Unable to run job handler (thumbnailGeneration/generate-person-thumbnail): Error: Input file contains unsupported image format
[Nest] 7 - 06/12/2024, 10:08:59 PM ERROR [Microservices:JobService] Error: Input file contains unsupported image format
at Sharp.toFile (/usr/src/app/node_modules/sharp/lib/output.js:89:19)
at MediaRepository.generateThumbnail (/usr/src/app/dist/repositories/media.repository.js:69:14)
at PersonService.handleGeneratePersonThumbnail (/usr/src/app/dist/services/person.service.js:429:36)
at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
at async /usr/src/app/dist/services/job.service.js:148:36
at async Worker.processJob (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:394:28)
at async Worker.retryIfFailed (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:581:24)
[Nest] 7 - 06/12/2024, 10:08:59 PM ERROR [Microservices:JobService] Object:
{
"id": "6f697b7a-8687-4806-b079-0cb1016cd6aa"
}

    all values are hidden

[Nest] 7 - 06/12/2024, 10:08:59 PM VERBOSE [Microservices:PersonService] Cropping face for person: f3e99d65-8834-49f4-82fc-750f290031cb
[Nest] 7 - 06/12/2024, 10:09:00 PM ERROR [Microservices:JobService] Unable to run job handler (thumbnailGeneration/generate-person-thumbnail): Error: Input file contains unsupported image format
[Nest] 7 - 06/12/2024, 10:09:00 PM ERROR [Microservices:JobService] Error: Input file contains unsupported image format
at Sharp.toFile (/usr/src/app/node_modules/sharp/lib/output.js:89:19)
at MediaRepository.generateThumbnail (/usr/src/app/dist/repositories/media.repository.js:69:14)
at PersonService.handleGeneratePersonThumbnail (/usr/src/app/dist/services/person.service.js:429:36)
at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
at async /usr/src/app/dist/services/job.service.js:148:36
at async Worker.processJob (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:394:28)
at async Worker.retryIfFailed (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:581:24)
[Nest] 7 - 06/12/2024, 10:09:00 PM ERROR [Microservices:JobService] Object:
{
"id": "f3e99d65-8834-49f4-82fc-750f290031cb"
}

    all values are hidden

[Nest] 7 - 06/12/2024, 10:09:00 PM VERBOSE [Microservices:PersonService] Cropping face for person: ff83f386-7187-4798-9734-4b4b37611eb5
[Nest] 7 - 06/12/2024, 10:09:00 PM ERROR [Microservices:JobService] Unable to run job handler (thumbnailGeneration/generate-person-thumbnail): Error: Input file contains unsupported image format
[Nest] 7 - 06/12/2024, 10:09:00 PM ERROR [Microservices:JobService] Error: Input file contains unsupported image format
at Sharp.toFile (/usr/src/app/node_modules/sharp/lib/output.js:89:19)
at MediaRepository.generateThumbnail (/usr/src/app/dist/repositories/media.repository.js:69:14)
at PersonService.handleGeneratePersonThumbnail (/usr/src/app/dist/services/person.service.js:429:36)
at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
at async /usr/src/app/dist/services/job.service.js:148:36
at async Worker.processJob (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:394:28)
at async Worker.retryIfFailed (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:581:24)
[Nest] 7 - 06/12/2024, 10:09:00 PM ERROR [Microservices:JobService] Object:
{
"id": "ff83f386-7187-4798-9734-4b4b37611eb5"
}

    all values are hidden

[Nest] 7 - 06/12/2024, 10:09:00 PM VERBOSE [Microservices:PersonService] Cropping face for person: ecbef7f5-9122-4e06-9414-af4c61484e82
[Nest] 7 - 06/12/2024, 10:09:00 PM ERROR [Microservices:JobService] Unable to run job handler (thumbnailGeneration/generate-person-thumbnail): Error: Input file contains unsupported image format
[Nest] 7 - 06/12/2024, 10:09:00 PM ERROR [Microservices:JobService] Error: Input file contains unsupported image format
at Sharp.toFile (/usr/src/app/node_modules/sharp/lib/output.js:89:19)
at MediaRepository.generateThumbnail (/usr/src/app/dist/repositories/media.repository.js:69:14)
at PersonService.handleGeneratePersonThumbnail (/usr/src/app/dist/services/person.service.js:429:36)
at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
at async /usr/src/app/dist/services/job.service.js:148:36
at async Worker.processJob (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:394:28)
at async Worker.retryIfFailed (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:581:24)
[Nest] 7 - 06/12/2024, 10:09:00 PM ERROR [Microservices:JobService] Object:
{
"id": "ecbef7f5-9122-4e06-9414-af4c61484e82"
}

    all values are hidden

[Nest] 7 - 06/12/2024, 10:09:00 PM VERBOSE [Microservices:PersonService] Cropping face for person: 8a194a27-3d52-43ba-8ac0-145e51f1765d
[Nest] 7 - 06/12/2024, 10:09:00 PM ERROR [Microservices:JobService] Unable to run job handler (thumbnailGeneration/generate-person-thumbnail): Error: Input file contains unsupported image format
[Nest] 7 - 06/12/2024, 10:09:00 PM ERROR [Microservices:JobService] Error: Input file contains unsupported image format
at Sharp.toFile (/usr/src/app/node_modules/sharp/lib/output.js:89:19)
at MediaRepository.generateThumbnail (/usr/src/app/dist/repositories/media.repository.js:69:14)
at PersonService.handleGeneratePersonThumbnail (/usr/src/app/dist/services/person.service.js:429:36)
at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
at async /usr/src/app/dist/services/job.service.js:148:36
at async Worker.processJob (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:394:28)
at async Worker.retryIfFailed (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:581:24)
[Nest] 7 - 06/12/2024, 10:09:00 PM ERROR [Microservices:JobService] Object:
{
"id": "8a194a27-3d52-43ba-8ac0-145e51f1765d"
}

    all values are hidden

[Nest] 7 - 06/12/2024, 10:09:00 PM VERBOSE [Microservices:PersonService] Cropping face for person: 5d393247-39d9-4823-8a85-d14e6d9e4230
[Nest] 7 - 06/12/2024, 10:09:00 PM ERROR [Microservices:JobService] Unable to run job handler (thumbnailGeneration/generate-person-thumbnail): Error: Input file contains unsupported image format
[Nest] 7 - 06/12/2024, 10:09:00 PM ERROR [Microservices:JobService] Error: Input file contains unsupported image format
at Sharp.toFile (/usr/src/app/node_modules/sharp/lib/output.js:89:19)
at MediaRepository.generateThumbnail (/usr/src/app/dist/repositories/media.repository.js:69:14)
at PersonService.handleGeneratePersonThumbnail (/usr/src/app/dist/services/person.service.js:429:36)
at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
at async /usr/src/app/dist/services/job.service.js:148:36
at async Worker.processJob (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:394:28)
at async Worker.retryIfFailed (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:581:24)
[Nest] 7 - 06/12/2024, 10:09:00 PM ERROR [Microservices:JobService] Object:
{
"id": "5d393247-39d9-4823-8a85-d14e6d9e4230"
}

    all values are hidden

[Nest] 7 - 06/12/2024, 10:09:00 PM VERBOSE [Microservices:PersonService] Cropping face for person: b567edac-0341-4f8a-915c-2cb0724bf1aa
[Nest] 7 - 06/12/2024, 10:09:00 PM ERROR [Microservices:JobService] Unable to run job handler (thumbnailGeneration/generate-person-thumbnail): Error: Input file contains unsupported image format
[Nest] 7 - 06/12/2024, 10:09:00 PM ERROR [Microservices:JobService] Error: Input file contains unsupported image format
at Sharp.toFile (/usr/src/app/node_modules/sharp/lib/output.js:89:19)
at MediaRepository.generateThumbnail (/usr/src/app/dist/repositories/media.repository.js:69:14)
at PersonService.handleGeneratePersonThumbnail (/usr/src/app/dist/services/person.service.js:429:36)
at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
at async /usr/src/app/dist/services/job.service.js:148:36
at async Worker.processJob (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:394:28)
at async Worker.retryIfFailed (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:581:24)
[Nest] 7 - 06/12/2024, 10:09:00 PM ERROR [Microservices:JobService] Object:
{
"id": "b567edac-0341-4f8a-915c-2cb0724bf1aa"
}

    all values are hidden

[Nest] 7 - 06/12/2024, 10:09:00 PM VERBOSE [Microservices:PersonService] Cropping face for person: f92d4193-8e6a-4e12-817a-7f00bfa9c66b
[Nest] 7 - 06/12/2024, 10:09:00 PM ERROR [Microservices:JobService] Unable to run job handler (thumbnailGeneration/generate-person-thumbnail): Error: Input file contains unsupported image format
[Nest] 7 - 06/12/2024, 10:09:00 PM ERROR [Microservices:JobService] Error: Input file contains unsupported image format
at Sharp.toFile (/usr/src/app/node_modules/sharp/lib/output.js:89:19)
at MediaRepository.generateThumbnail (/usr/src/app/dist/repositories/media.repository.js:69:14)
at PersonService.handleGeneratePersonThumbnail (/usr/src/app/dist/services/person.service.js:429:36)
at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
at async /usr/src/app/dist/services/job.service.js:148:36
at async Worker.processJob (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:394:28)
at async Worker.retryIfFailed (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:581:24)
[Nest] 7 - 06/12/2024, 10:09:00 PM ERROR [Microservices:JobService] Object:
{
"id": "f92d4193-8e6a-4e12-817a-7f00bfa9c66b"
}

    all values are hidden

[Nest] 7 - 06/12/2024, 10:09:00 PM VERBOSE [Microservices:PersonService] Cropping face for person: 508e39a4-09c7-4f32-9b33-4c825873cb7b
[Nest] 7 - 06/12/2024, 10:09:00 PM ERROR [Microservices:JobService] Unable to run job handler (thumbnailGeneration/generate-person-thumbnail): Error: Input file contains unsupported image format
[Nest] 7 - 06/12/2024, 10:09:00 PM ERROR [Microservices:JobService] Error: Input file contains unsupported image format
at Sharp.toFile (/usr/src/app/node_modules/sharp/lib/output.js:89:19)
at MediaRepository.generateThumbnail (/usr/src/app/dist/repositories/media.repository.js:69:14)
at PersonService.handleGeneratePersonThumbnail (/usr/src/app/dist/services/person.service.js:429:36)
at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
at async /usr/src/app/dist/services/job.service.js:148:36
at async Worker.processJob (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:394:28)
at async Worker.retryIfFailed (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:581:24)
[Nest] 7 - 06/12/2024, 10:09:00 PM ERROR [Microservices:JobService] Object:
{
"id": "508e39a4-09c7-4f32-9b33-4c825873cb7b"
}

    all values are hidden

[Nest] 7 - 06/12/2024, 10:09:00 PM VERBOSE [Microservices:PersonService] Cropping face for person: ca4a4949-f97d-4764-8166-7268df99eb67
[Nest] 7 - 06/12/2024, 10:09:00 PM ERROR [Microservices:JobService] Unable to run job handler (thumbnailGeneration/generate-person-thumbnail): Error: Input file contains unsupported image format
[Nest] 7 - 06/12/2024, 10:09:00 PM ERROR [Microservices:JobService] Error: Input file contains unsupported image format
at Sharp.toFile (/usr/src/app/node_modules/sharp/lib/output.js:89:19)
at MediaRepository.generateThumbnail (/usr/src/app/dist/repositories/media.repository.js:69:14)
at PersonService.handleGeneratePersonThumbnail (/usr/src/app/dist/services/person.service.js:429:36)
at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
at async /usr/src/app/dist/services/job.service.js:148:36
at async Worker.processJob (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:394:28)
at async Worker.retryIfFailed (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:581:24)
[Nest] 7 - 06/12/2024, 10:09:00 PM ERROR [Microservices:JobService] Object:
{
"id": "ca4a4949-f97d-4764-8166-7268df99eb67"
}

    all values are hidden

[Nest] 7 - 06/12/2024, 10:09:00 PM VERBOSE [Microservices:PersonService] Cropping face for person: 06d5111c-ada4-4415-a8fb-849f835db426
[Nest] 7 - 06/12/2024, 10:09:00 PM ERROR [Microservices:JobService] Unable to run job handler (thumbnailGeneration/generate-person-thumbnail): Error: Input file contains unsupported image format
[Nest] 7 - 06/12/2024, 10:09:00 PM ERROR [Microservices:JobService] Error: Input file contains unsupported image format
at Sharp.toFile (/usr/src/app/node_modules/sharp/lib/output.js:89:19)
at MediaRepository.generateThumbnail (/usr/src/app/dist/repositories/media.repository.js:69:14)
at PersonService.handleGeneratePersonThumbnail (/usr/src/app/dist/services/person.service.js:429:36)
at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
at async /usr/src/app/dist/services/job.service.js:148:36
at async Worker.processJob (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:394:28)
at async Worker.retryIfFailed (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:581:24)
[Nest] 7 - 06/12/2024, 10:09:00 PM ERROR [Microservices:JobService] Object:
{
"id": "06d5111c-ada4-4415-a8fb-849f835db426"
}

    all values are hidden

[Nest] 7 - 06/12/2024, 10:09:00 PM VERBOSE [Microservices:PersonService] Cropping face for person: 5f73132f-b003-452b-b81d-dcf0b45cddff
[Nest] 7 - 06/12/2024, 10:09:01 PM ERROR [Microservices:JobService] Unable to run job handler (thumbnailGeneration/generate-person-thumbnail): Error: Input file contains unsupported image format
[Nest] 7 - 06/12/2024, 10:09:01 PM ERROR [Microservices:JobService] Error: Input file contains unsupported image format
at Sharp.toFile (/usr/src/app/node_modules/sharp/lib/output.js:89:19)
at MediaRepository.generateThumbnail (/usr/src/app/dist/repositories/media.repository.js:69:14)
at PersonService.handleGeneratePersonThumbnail (/usr/src/app/dist/services/person.service.js:429:36)
at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
at async /usr/src/app/dist/services/job.service.js:148:36
at async Worker.processJob (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:394:28)
at async Worker.retryIfFailed (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:581:24)
[Nest] 7 - 06/12/2024, 10:09:01 PM ERROR [Microservices:JobService] Object:
{
"id": "5f73132f-b003-452b-b81d-dcf0b45cddff"
}

    all values are hidden

[Nest] 7 - 06/12/2024, 10:09:01 PM VERBOSE [Microservices:PersonService] Cropping face for person: eb1a0657-3ac0-4095-b55c-7a35a4e38e1f
[Nest] 7 - 06/12/2024, 10:09:01 PM ERROR [Microservices:JobService] Unable to run job handler (thumbnailGeneration/generate-person-thumbnail): Error: Input file contains unsupported image format
[Nest] 7 - 06/12/2024, 10:09:01 PM ERROR [Microservices:JobService] Error: Input file contains unsupported image format
at Sharp.toFile (/usr/src/app/node_modules/sharp/lib/output.js:89:19)
at MediaRepository.generateThumbnail (/usr/src/app/dist/repositories/media.repository.js:69:14)
at PersonService.handleGeneratePersonThumbnail (/usr/src/app/dist/services/person.service.js:429:36)
at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
at async /usr/src/app/dist/services/job.service.js:148:36
at async Worker.processJob (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:394:28)
at async Worker.retryIfFailed (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:581:24)
[Nest] 7 - 06/12/2024, 10:09:01 PM ERROR [Microservices:JobService] Object:
{
"id": "eb1a0657-3ac0-4095-b55c-7a35a4e38e1f"
}

    all values are hidden

[Nest] 7 - 06/12/2024, 10:09:01 PM VERBOSE [Microservices:PersonService] Cropping face for person: 1a61f0cc-6f19-4acb-a92e-0b4c4995652b
[Nest] 7 - 06/12/2024, 10:09:01 PM ERROR [Microservices:JobService] Unable to run job handler (thumbnailGeneration/generate-person-thumbnail): Error: Input file contains unsupported image format
[Nest] 7 - 06/12/2024, 10:09:01 PM ERROR [Microservices:JobService] Error: Input file contains unsupported image format
at Sharp.toFile (/usr/src/app/node_modules/sharp/lib/output.js:89:19)
at MediaRepository.generateThumbnail (/usr/src/app/dist/repositories/media.repository.js:69:14)
at PersonService.handleGeneratePersonThumbnail (/usr/src/app/dist/services/person.service.js:429:36)
at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
at async /usr/src/app/dist/services/job.service.js:148:36
at async Worker.processJob (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:394:28)
at async Worker.retryIfFailed (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:581:24)
[Nest] 7 - 06/12/2024, 10:09:01 PM ERROR [Microservices:JobService] Object:
{
"id": "1a61f0cc-6f19-4acb-a92e-0b4c4995652b"
}

    all values are hidden

[Nest] 7 - 06/12/2024, 10:09:01 PM VERBOSE [Microservices:PersonService] Cropping face for person: 08937eff-0454-4e8a-939b-875409fb313e
[Nest] 7 - 06/12/2024, 10:09:01 PM ERROR [Microservices:JobService] Unable to run job handler (thumbnailGeneration/generate-person-thumbnail): Error: Input file contains unsupported image format
[Nest] 7 - 06/12/2024, 10:09:01 PM ERROR [Microservices:JobService] Error: Input file contains unsupported image format
at Sharp.toFile (/usr/src/app/node_modules/sharp/lib/output.js:89:19)
at MediaRepository.generateThumbnail (/usr/src/app/dist/repositories/media.repository.js:69:14)
at PersonService.handleGeneratePersonThumbnail (/usr/src/app/dist/services/person.service.js:429:36)
at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
at async /usr/src/app/dist/services/job.service.js:148:36
at async Worker.processJob (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:394:28)
at async Worker.retryIfFailed (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:581:24)
[Nest] 7 - 06/12/2024, 10:09:01 PM ERROR [Microservices:JobService] Object:
{
"id": "08937eff-0454-4e8a-939b-875409fb313e"
}

    all values are hidden

[Nest] 7 - 06/12/2024, 10:09:01 PM VERBOSE [Microservices:PersonService] Cropping face for person: 725e89a9-bd9d-4225-86bd-8e928a1d84b4
[Nest] 7 - 06/12/2024, 10:09:01 PM ERROR [Microservices:JobService] Unable to run job handler (thumbnailGeneration/generate-person-thumbnail): Error: Input file contains unsupported image format
[Nest] 7 - 06/12/2024, 10:09:01 PM ERROR [Microservices:JobService] Error: Input file contains unsupported image format
at Sharp.toFile (/usr/src/app/node_modules/sharp/lib/output.js:89:19)
at MediaRepository.generateThumbnail (/usr/src/app/dist/repositories/media.repository.js:69:14)
at PersonService.handleGeneratePersonThumbnail (/usr/src/app/dist/services/person.service.js:429:36)
at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
at async /usr/src/app/dist/services/job.service.js:148:36
at async Worker.processJob (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:394:28)
at async Worker.retryIfFailed (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:581:24)
[Nest] 7 - 06/12/2024, 10:09:01 PM ERROR [Microservices:JobService] Object:
{
"id": "725e89a9-bd9d-4225-86bd-8e928a1d84b4"
}

    all values are hidden

[Nest] 7 - 06/12/2024, 10:09:01 PM VERBOSE [Microservices:PersonService] Cropping face for person: 79d6c1dc-fdb3-44cc-be49-59f5e2d98906
[Nest] 7 - 06/12/2024, 10:09:01 PM ERROR [Microservices:JobService] Unable to run job handler (thumbnailGeneration/generate-person-thumbnail): Error: Input file contains unsupported image format
[Nest] 7 - 06/12/2024, 10:09:01 PM ERROR [Microservices:JobService] Error: Input file contains unsupported image format
at Sharp.toFile (/usr/src/app/node_modules/sharp/lib/output.js:89:19)
at MediaRepository.generateThumbnail (/usr/src/app/dist/repositories/media.repository.js:69:14)
at PersonService.handleGeneratePersonThumbnail (/usr/src/app/dist/services/person.service.js:429:36)
at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
at async /usr/src/app/dist/services/job.service.js:148:36
at async Worker.processJob (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:394:28)
at async Worker.retryIfFailed (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:581:24)
[Nest] 7 - 06/12/2024, 10:09:01 PM ERROR [Microservices:JobService] Object:
{
"id": "79d6c1dc-fdb3-44cc-be49-59f5e2d98906"
}

    all values are hidden

[Nest] 7 - 06/12/2024, 10:09:01 PM VERBOSE [Microservices:PersonService] Cropping face for person: eb243f39-206d-4e9b-bc09-1669f4fc4e1f
[Nest] 7 - 06/12/2024, 10:09:01 PM ERROR [Microservices:JobService] Unable to run job handler (thumbnailGeneration/generate-person-thumbnail): Error: Input file contains unsupported image format
[Nest] 7 - 06/12/2024, 10:09:01 PM ERROR [Microservices:JobService] Error: Input file contains unsupported image format
at Sharp.toFile (/usr/src/app/node_modules/sharp/lib/output.js:89:19)
at MediaRepository.generateThumbnail (/usr/src/app/dist/repositories/media.repository.js:69:14)
at PersonService.handleGeneratePersonThumbnail (/usr/src/app/dist/services/person.service.js:429:36)
at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
at async /usr/src/app/dist/services/job.service.js:148:36
at async Worker.processJob (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:394:28)
at async Worker.retryIfFailed (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:581:24)
[Nest] 7 - 06/12/2024, 10:09:01 PM ERROR [Microservices:JobService] Object:
{
"id": "eb243f39-206d-4e9b-bc09-1669f4fc4e1f"
}

    all values are hidden

[Nest] 7 - 06/12/2024, 10:09:02 PM VERBOSE [Microservices:PersonService] Cropping face for person: fcb1d0c1-abe6-4cd4-b508-489997e24bd7
[Nest] 7 - 06/12/2024, 10:09:02 PM ERROR [Microservices:JobService] Unable to run job handler (thumbnailGeneration/generate-person-thumbnail): Error: Input file contains unsupported image format
at MediaRepository.generateThumbnail (/usr/src/app/dist/repositories/media.repository.js:69:14)
at PersonService.handleGeneratePersonThumbnail (/usr/src/app/dist/services/person.service.js:429:36)
at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
at async /usr/src/app/dist/services/job.service.js:148:36
at async Worker.processJob (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:394:28)
at async Worker.retryIfFailed (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:581:24)
[Nest] 7 - 06/12/2024, 10:09:02 PM ERROR [Microservices:JobService] Object:
{
"id": "fcb1d0c1-abe6-4cd4-b508-489997e24bd7"
}

    all values are hidden

[Nest] 7 - 06/12/2024, 10:09:02 PM VERBOSE [Microservices:PersonService] Cropping face for person: 838ba6d8-758e-479e-82db-29443bd280a6
[Nest] 7 - 06/12/2024, 10:09:02 PM ERROR [Microservices:JobService] Unable to run job handler (thumbnailGeneration/generate-person-thumbnail): Error: Input file contains unsupported image format
[Nest] 7 - 06/12/2024, 10:09:02 PM ERROR [Microservices:JobService] Error: Input file contains unsupported image format
at Sharp.toFile (/usr/src/app/node_modules/sharp/lib/output.js:89:19)
at MediaRepository.generateThumbnail (/usr/src/app/dist/repositories/media.repository.js:69:14)
at PersonService.handleGeneratePersonThumbnail (/usr/src/app/dist/services/person.service.js:429:36)
at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
at async /usr/src/app/dist/services/job.service.js:148:36
at async Worker.processJob (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:394:28)
at async Worker.retryIfFailed (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:581:24)
[Nest] 7 - 06/12/2024, 10:09:02 PM ERROR [Microservices:JobService] Object:
{
"id": "838ba6d8-758e-479e-82db-29443bd280a6"
}

    all values are hidden

[Nest] 7 - 06/12/2024, 10:09:02 PM VERBOSE [Microservices:PersonService] Cropping face for person: 11e18daa-c5e6-4caa-8f02-e5eadca5586f
[Nest] 7 - 06/12/2024, 10:09:02 PM ERROR [Microservices:JobService] Unable to run job handler (thumbnailGeneration/generate-person-thumbnail): Error: Input file contains unsupported image format
[Nest] 7 - 06/12/2024, 10:09:02 PM ERROR [Microservices:JobService] Error: Input file contains unsupported image format
at Sharp.toFile (/usr/src/app/node_modules/sharp/lib/output.js:89:19)
at MediaRepository.generateThumbnail (/usr/src/app/dist/repositories/media.repository.js:69:14)
at PersonService.handleGeneratePersonThumbnail (/usr/src/app/dist/services/person.service.js:429:36)
at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
at async /usr/src/app/dist/services/job.service.js:148:36
at async Worker.processJob (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:394:28)
at async Worker.retryIfFailed (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:581:24)
[Nest] 7 - 06/12/2024, 10:09:02 PM ERROR [Microservices:JobService] Object:
{
"id": "11e18daa-c5e6-4caa-8f02-e5eadca5586f"
}

    all values are hidden

[Nest] 7 - 06/12/2024, 10:09:02 PM VERBOSE [Microservices:PersonService] Cropping face for person: 2d2f2e02-f921-4c08-8e4d-e70158e87ed2
[Nest] 7 - 06/12/2024, 10:09:02 PM ERROR [Microservices:JobService] Unable to run job handler (thumbnailGeneration/generate-person-thumbnail): Error: Input file contains unsupported image format
[Nest] 7 - 06/12/2024, 10:09:02 PM ERROR [Microservices:JobService] Error: Input file contains unsupported image format
at Sharp.toFile (/usr/src/app/node_modules/sharp/lib/output.js:89:19)
at MediaRepository.generateThumbnail (/usr/src/app/dist/repositories/media.repository.js:69:14)
at PersonService.handleGeneratePersonThumbnail (/usr/src/app/dist/services/person.service.js:429:36)
at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
at async /usr/src/app/dist/services/job.service.js:148:36
at async Worker.processJob (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:394:28)
at async Worker.retryIfFailed (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:581:24)
[Nest] 7 - 06/12/2024, 10:09:02 PM ERROR [Microservices:JobService] Object:
{
"id": "2d2f2e02-f921-4c08-8e4d-e70158e87ed2"
}

    all values are hidden

[Nest] 7 - 06/12/2024, 10:09:02 PM VERBOSE [Microservices:PersonService] Cropping face for person: e2cef1bb-e999-4197-b1da-61d9f3d467c9
[Nest] 7 - 06/12/2024, 10:09:02 PM ERROR [Microservices:JobService] Unable to run job handler (thumbnailGeneration/generate-person-thumbnail): Error: Input file contains unsupported image format
[Nest] 7 - 06/12/2024, 10:09:02 PM ERROR [Microservices:JobService] Error: Input file contains unsupported image format
at Sharp.toFile (/usr/src/app/node_modules/sharp/lib/output.js:89:19)
at MediaRepository.generateThumbnail (/usr/src/app/dist/repositories/media.repository.js:69:14)
at PersonService.handleGeneratePersonThumbnail (/usr/src/app/dist/services/person.service.js:429:36)
at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
at async /usr/src/app/dist/services/job.service.js:148:36
at async Worker.processJob (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:394:28)
at async Worker.retryIfFailed (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:581:24)
[Nest] 7 - 06/12/2024, 10:09:02 PM ERROR [Microservices:JobService] Object:
{
"id": "e2cef1bb-e999-4197-b1da-61d9f3d467c9"
}

    all values are hidden

[Nest] 7 - 06/12/2024, 10:09:02 PM VERBOSE [Microservices:PersonService] Cropping face for person: af0ef861-1d72-40a0-82ed-3981267a84d6
[Nest] 7 - 06/12/2024, 10:09:02 PM ERROR [Microservices:JobService] Unable to run job handler (thumbnailGeneration/generate-person-thumbnail): Error: Input file contains unsupported image format
[Nest] 7 - 06/12/2024, 10:09:02 PM ERROR [Microservices:JobService] Error: Input file contains unsupported image format
at Sharp.toFile (/usr/src/app/node_modules/sharp/lib/output.js:89:19)
at MediaRepository.generateThumbnail (/usr/src/app/dist/repositories/media.repository.js:69:14)
at PersonService.handleGeneratePersonThumbnail (/usr/src/app/dist/services/person.service.js:429:36)
at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
at async /usr/src/app/dist/services/job.service.js:148:36
at async Worker.processJob (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:394:28)
at async Worker.retryIfFailed (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:581:24)
[Nest] 7 - 06/12/2024, 10:09:02 PM ERROR [Microservices:JobService] Object:
{
"id": "af0ef861-1d72-40a0-82ed-3981267a84d6"
}

    all values are hidden

[Nest] 7 - 06/12/2024, 10:09:02 PM VERBOSE [Microservices:PersonService] Cropping face for person: 535a21c9-b0c5-4fdc-8cbd-bed848adb6a4
[Nest] 7 - 06/12/2024, 10:09:03 PM ERROR [Microservices:JobService] Unable to run job handler (thumbnailGeneration/generate-person-thumbnail): Error: Input file contains unsupported image format
[Nest] 7 - 06/12/2024, 10:09:03 PM ERROR [Microservices:JobService] Error: Input file contains unsupported image format
at Sharp.toFile (/usr/src/app/node_modules/sharp/lib/output.js:89:19)
at MediaRepository.generateThumbnail (/usr/src/app/dist/repositories/media.repository.js:69:14)
at PersonService.handleGeneratePersonThumbnail (/usr/src/app/dist/services/person.service.js:429:36)
at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
at async /usr/src/app/dist/services/job.service.js:148:36
at async Worker.processJob (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:394:28)
at async Worker.retryIfFailed (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:581:24)
[Nest] 7 - 06/12/2024, 10:09:03 PM ERROR [Microservices:JobService] Object:
{
"id": "535a21c9-b0c5-4fdc-8cbd-bed848adb6a4"
}

    all values are hidden

[Nest] 7 - 06/12/2024, 10:09:03 PM VERBOSE [Microservices:PersonService] Cropping face for person: ed51c8d6-9eaf-4639-937d-e5ac2aeedebe
[Nest] 17 - 06/12/2024, 10:09:03 PM DEBUG [Api:LoggingInterceptor~4ss0inwu] GET /api/jobs 200 8.11ms 192.168.0.201
[Nest] 7 - 06/12/2024, 10:09:03 PM ERROR [Microservices:JobService] Unable to run job handler (thumbnailGeneration/generate-person-thumbnail): Error: Input file contains unsupported image format
[Nest] 7 - 06/12/2024, 10:09:03 PM ERROR [Microservices:JobService] Error: Input file contains unsupported image format
at Sharp.toFile (/usr/src/app/node_modules/sharp/lib/output.js:89:19)
at MediaRepository.generateThumbnail (/usr/src/app/dist/repositories/media.repository.js:69:14)
at PersonService.handleGeneratePersonThumbnail (/usr/src/app/dist/services/person.service.js:429:36)
at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
at async /usr/src/app/dist/services/job.service.js:148:36
at async Worker.processJob (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:394:28)
at async Worker.retryIfFailed (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:581:24)
[Nest] 7 - 06/12/2024, 10:09:03 PM ERROR [Microservices:JobService] Object:
{
"id": "ed51c8d6-9eaf-4639-937d-e5ac2aeedebe"
}

    all values are hidden

[Nest] 7 - 06/12/2024, 10:09:03 PM VERBOSE [Microservices:PersonService] Cropping face for person: bf1f8bf9-34b4-4681-b64a-6128e6bfa6e6
[Nest] 7 - 06/12/2024, 10:09:03 PM ERROR [Microservices:JobService] Unable to run job handler (thumbnailGeneration/generate-person-thumbnail): Error: Input file contains unsupported image format
[Nest] 7 - 06/12/2024, 10:09:03 PM ERROR [Microservices:JobService] Error: Input file contains unsupported image format
at Sharp.toFile (/usr/src/app/node_modules/sharp/lib/output.js:89:19)
at MediaRepository.generateThumbnail (/usr/src/app/dist/repositories/media.repository.js:69:14)
at PersonService.handleGeneratePersonThumbnail (/usr/src/app/dist/services/person.service.js:429:36)
at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
at async /usr/src/app/dist/services/job.service.js:148:36
at async Worker.processJob (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:394:28)
at async Worker.retryIfFailed (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:581:24)
[Nest] 7 - 06/12/2024, 10:09:03 PM ERROR [Microservices:JobService] Object:
{
"id": "bf1f8bf9-34b4-4681-b64a-6128e6bfa6e6"
}

    all values are hidden

[Nest] 7 - 06/12/2024, 10:09:03 PM VERBOSE [Microservices:PersonService] Cropping face for person: d5505bd9-26c6-4070-b1cc-e5dc4848367b
[Nest] 7 - 06/12/2024, 10:09:03 PM ERROR [Microservices:JobService] Unable to run job handler (thumbnailGeneration/generate-person-thumbnail): Error: Input file contains unsupported image format
[Nest] 7 - 06/12/2024, 10:09:03 PM ERROR [Microservices:JobService] Error: Input file contains unsupported image format
at Sharp.toFile (/usr/src/app/node_modules/sharp/lib/output.js:89:19)
at MediaRepository.generateThumbnail (/usr/src/app/dist/repositories/media.repository.js:69:14)
at PersonService.handleGeneratePersonThumbnail (/usr/src/app/dist/services/person.service.js:429:36)
at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
at async /usr/src/app/dist/services/job.service.js:148:36
at async Worker.processJob (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:394:28)
at async Worker.retryIfFailed (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:581:24)
[Nest] 7 - 06/12/2024, 10:09:03 PM ERROR [Microservices:JobService] Object:
{
"id": "d5505bd9-26c6-4070-b1cc-e5dc4848367b"
}

    all values are hidden

[Nest] 7 - 06/12/2024, 10:09:03 PM VERBOSE [Microservices:PersonService] Cropping face for person: ebec2899-ff9d-4cac-bf95-1c61ece5ebd8
[Nest] 7 - 06/12/2024, 10:09:04 PM ERROR [Microservices:JobService] Unable to run job handler (thumbnailGeneration/generate-person-thumbnail): Error: Input file contains unsupported image format
[Nest] 7 - 06/12/2024, 10:09:04 PM ERROR [Microservices:JobService] Error: Input file contains unsupported image format
at Sharp.toFile (/usr/src/app/node_modules/sharp/lib/output.js:89:19)
at MediaRepository.generateThumbnail (/usr/src/app/dist/repositories/media.repository.js:69:14)
at PersonService.handleGeneratePersonThumbnail (/usr/src/app/dist/services/person.service.js:429:36)
at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
at async /usr/src/app/dist/services/job.service.js:148:36
at async Worker.processJob (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:394:28)
at async Worker.retryIfFailed (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:581:24)
[Nest] 7 - 06/12/2024, 10:09:04 PM ERROR [Microservices:JobService] Object:
{
"id": "ebec2899-ff9d-4cac-bf95-1c61ece5ebd8"
}

    all values are hidden

[Nest] 7 - 06/12/2024, 10:09:04 PM VERBOSE [Microservices:PersonService] Cropping face for person: 48ea0af7-6174-42bf-bce6-2785918319c0
[Nest] 7 - 06/12/2024, 10:09:04 PM ERROR [Microservices:JobService] Unable to run job handler (thumbnailGeneration/generate-person-thumbnail): Error: Input file contains unsupported image format
[Nest] 7 - 06/12/2024, 10:09:04 PM ERROR [Microservices:JobService] Error: Input file contains unsupported image format
at Sharp.toFile (/usr/src/app/node_modules/sharp/lib/output.js:89:19)
at MediaRepository.generateThumbnail (/usr/src/app/dist/repositories/media.repository.js:69:14)
at PersonService.handleGeneratePersonThumbnail (/usr/src/app/dist/services/person.service.js:429:36)
at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
at async /usr/src/app/dist/services/job.service.js:148:36
at async Worker.processJob (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:394:28)
at async Worker.retryIfFailed (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:581:24)
[Nest] 7 - 06/12/2024, 10:09:04 PM ERROR [Microservices:JobService] Object:
{
"id": "48ea0af7-6174-42bf-bce6-2785918319c0"
}

    all values are hidden

[Nest] 7 - 06/12/2024, 10:09:04 PM VERBOSE [Microservices:PersonService] Cropping face for person: 86b21a30-b861-42d1-aa76-389b0a3d19a6
[Nest] 7 - 06/12/2024, 10:09:04 PM ERROR [Microservices:JobService] Unable to run job handler (thumbnailGeneration/generate-person-thumbnail): Error: Input file contains unsupported image format
[Nest] 7 - 06/12/2024, 10:09:04 PM ERROR [Microservices:JobService] Error: Input file contains unsupported image format
at Sharp.toFile (/usr/src/app/node_modules/sharp/lib/output.js:89:19)
at MediaRepository.generateThumbnail (/usr/src/app/dist/repositories/media.repository.js:69:14)
at PersonService.handleGeneratePersonThumbnail (/usr/src/app/dist/services/person.service.js:429:36)
at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
at async /usr/src/app/dist/services/job.service.js:148:36
at async Worker.processJob (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:394:28)
at async Worker.retryIfFailed (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:581:24)
[Nest] 7 - 06/12/2024, 10:09:04 PM ERROR [Microservices:JobService] Object:
{
"id": "86b21a30-b861-42d1-aa76-389b0a3d19a6"
}

    all values are hidden

[Nest] 7 - 06/12/2024, 10:09:04 PM VERBOSE [Microservices:PersonService] Cropping face for person: 6069389a-aea3-49be-87da-d7bbf061ec47
[Nest] 7 - 06/12/2024, 10:09:04 PM ERROR [Microservices:JobService] Unable to run job handler (thumbnailGeneration/generate-person-thumbnail): Error: Input file contains unsupported image format
[Nest] 7 - 06/12/2024, 10:09:04 PM ERROR [Microservices:JobService] Error: Input file contains unsupported image format
at Sharp.toFile (/usr/src/app/node_modules/sharp/lib/output.js:89:19)
at MediaRepository.generateThumbnail (/usr/src/app/dist/repositories/media.repository.js:69:14)
at PersonService.handleGeneratePersonThumbnail (/usr/src/app/dist/services/person.service.js:429:36)
at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
at async /usr/src/app/dist/services/job.service.js:148:36
at async Worker.processJob (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:394:28)
at async Worker.retryIfFailed (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:581:24)
[Nest] 7 - 06/12/2024, 10:09:04 PM ERROR [Microservices:JobService] Object:
{
"id": "6069389a-aea3-49be-87da-d7bbf061ec47"
}

    all values are hidden

[Nest] 7 - 06/12/2024, 10:09:04 PM VERBOSE [Microservices:PersonService] Cropping face for person: 37e889c6-f83d-47ca-8aeb-c3aae4e1a39c
[Nest] 7 - 06/12/2024, 10:09:04 PM ERROR [Microservices:JobService] Unable to run job handler (thumbnailGeneration/generate-person-thumbnail): Error: Input file contains unsupported image format
[Nest] 7 - 06/12/2024, 10:09:04 PM ERROR [Microservices:JobService] Error: Input file contains unsupported image format
at Sharp.toFile (/usr/src/app/node_modules/sharp/lib/output.js:89:19)
at MediaRepository.generateThumbnail (/usr/src/app/dist/repositories/media.repository.js:69:14)
at PersonService.handleGeneratePersonThumbnail (/usr/src/app/dist/services/person.service.js:429:36)
at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
at async /usr/src/app/dist/services/job.service.js:148:36
at async Worker.processJob (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:394:28)
at async Worker.retryIfFailed (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:581:24)
[Nest] 7 - 06/12/2024, 10:09:04 PM ERROR [Microservices:JobService] Object:
{
"id": "37e889c6-f83d-47ca-8aeb-c3aae4e1a39c"
}

    all values are hidden

[Nest] 7 - 06/12/2024, 10:09:04 PM VERBOSE [Microservices:PersonService] Cropping face for person: ae7dc9ec-e9c9-4b01-8fa7-bde50b375ec5
[Nest] 7 - 06/12/2024, 10:09:04 PM ERROR [Microservices:JobService] Unable to run job handler (thumbnailGeneration/generate-person-thumbnail): Error: Input file contains unsupported image format
[Nest] 7 - 06/12/2024, 10:09:04 PM ERROR [Microservices:JobService] Error: Input file contains unsupported image format
at Sharp.toFile (/usr/src/app/node_modules/sharp/lib/output.js:89:19)
at MediaRepository.generateThumbnail (/usr/src/app/dist/repositories/media.repository.js:69:14)
at PersonService.handleGeneratePersonThumbnail (/usr/src/app/dist/services/person.service.js:429:36)
at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
at async /usr/src/app/dist/services/job.service.js:148:36
at async Worker.processJob (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:394:28)
at async Worker.retryIfFailed (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:581:24)
[Nest] 7 - 06/12/2024, 10:09:04 PM ERROR [Microservices:JobService] Object:
{
"id": "ae7dc9ec-e9c9-4b01-8fa7-bde50b375ec5"
}

    all values are hidden

[Nest] 7 - 06/12/2024, 10:09:04 PM VERBOSE [Microservices:PersonService] Cropping face for person: 3d957272-2c16-43c8-82dc-992ca635d045
[Nest] 7 - 06/12/2024, 10:09:04 PM ERROR [Microservices:JobService] Unable to run job handler (thumbnailGeneration/generate-person-thumbnail): Error: Input file contains unsupported image format
[Nest] 7 - 06/12/2024, 10:09:04 PM ERROR [Microservices:JobService] Error: Input file contains unsupported image format
at Sharp.toFile (/usr/src/app/node_modules/sharp/lib/output.js:89:19)
at MediaRepository.generateThumbnail (/usr/src/app/dist/repositories/media.repository.js:69:14)
at PersonService.handleGeneratePersonThumbnail (/usr/src/app/dist/services/person.service.js:429:36)
at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
at async /usr/src/app/dist/services/job.service.js:148:36
at async Worker.processJob (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:394:28)
at async Worker.retryIfFailed (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:581:24)
[Nest] 7 - 06/12/2024, 10:09:04 PM ERROR [Microservices:JobService] Object:
{
"id": "3d957272-2c16-43c8-82dc-992ca635d045"
}

    all values are hidden

[Nest] 7 - 06/12/2024, 10:09:04 PM VERBOSE [Microservices:PersonService] Cropping face for person: 04a4c2c8-326b-4d53-adbe-758b49252eb2
[Nest] 7 - 06/12/2024, 10:09:04 PM ERROR [Microservices:JobService] Unable to run job handler (thumbnailGeneration/generate-person-thumbnail): Error: Input file contains unsupported image format
[Nest] 7 - 06/12/2024, 10:09:04 PM ERROR [Microservices:JobService] Error: Input file contains unsupported image format
at Sharp.toFile (/usr/src/app/node_modules/sharp/lib/output.js:89:19)
at MediaRepository.generateThumbnail (/usr/src/app/dist/repositories/media.repository.js:69:14)
at PersonService.handleGeneratePersonThumbnail (/usr/src/app/dist/services/person.service.js:429:36)
at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
at async /usr/src/app/dist/services/job.service.js:148:36
at async Worker.processJob (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:394:28)
at async Worker.retryIfFailed (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:581:24)
[Nest] 7 - 06/12/2024, 10:09:04 PM ERROR [Microservices:JobService] Object:
{
"id": "04a4c2c8-326b-4d53-adbe-758b49252eb2"
}

    all values are hidden

[Nest] 7 - 06/12/2024, 10:09:04 PM VERBOSE [Microservices:PersonService] Cropping face for person: bd4fef80-0592-4ce9-8f6f-436852757295
[Nest] 7 - 06/12/2024, 10:09:04 PM ERROR [Microservices:JobService] Unable to run job handler (thumbnailGeneration/generate-person-thumbnail): Error: Input file contains unsupported image format
[Nest] 7 - 06/12/2024, 10:09:04 PM ERROR [Microservices:JobService] Error: Input file contains unsupported image format
at Sharp.toFile (/usr/src/app/node_modules/sharp/lib/output.js:89:19)
at MediaRepository.generateThumbnail (/usr/src/app/dist/repositories/media.repository.js:69:14)
at PersonService.handleGeneratePersonThumbnail (/usr/src/app/dist/services/person.service.js:429:36)
at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
at async /usr/src/app/dist/services/job.service.js:148:36
at async Worker.processJob (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:394:28)
at async Worker.retryIfFailed (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:581:24)
[Nest] 7 - 06/12/2024, 10:09:04 PM ERROR [Microservices:JobService] Object:
{
"id": "bd4fef80-0592-4ce9-8f6f-436852757295"
}

    all values are hidden

[Nest] 7 - 06/12/2024, 10:09:04 PM VERBOSE [Microservices:PersonService] Cropping face for person: 3356c7bc-c53e-492c-b382-3aed27f6403d
[Nest] 7 - 06/12/2024, 10:09:04 PM ERROR [Microservices:JobService] Unable to run job handler (thumbnailGeneration/generate-person-thumbnail): Error: Input file contains unsupported image format
[Nest] 7 - 06/12/2024, 10:09:04 PM ERROR [Microservices:JobService] Error: Input file contains unsupported image format
at Sharp.toFile (/usr/src/app/node_modules/sharp/lib/output.js:89:19)
at MediaRepository.generateThumbnail (/usr/src/app/dist/repositories/media.repository.js:69:14)
at PersonService.handleGeneratePersonThumbnail (/usr/src/app/dist/services/person.service.js:429:36)
at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
at async /usr/src/app/dist/services/job.service.js:148:36
at async Worker.processJob (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:394:28)
at async Worker.retryIfFailed (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:581:24)
[Nest] 7 - 06/12/2024, 10:09:04 PM ERROR [Microservices:JobService] Object:
{
"id": "3356c7bc-c53e-492c-b382-3aed27f6403d"
}

    all values are hidden

[Nest] 7 - 06/12/2024, 10:09:04 PM VERBOSE [Microservices:PersonService] Cropping face for person: b98c0512-2cb8-4c2f-a290-eda3280fbc28
[Nest] 7 - 06/12/2024, 10:09:05 PM ERROR [Microservices:JobService] Unable to run job handler (thumbnailGeneration/generate-person-thumbnail): Error: Input file contains unsupported image format
[Nest] 7 - 06/12/2024, 10:09:05 PM ERROR [Microservices:JobService] Error: Input file contains unsupported image format
at Sharp.toFile (/usr/src/app/node_modules/sharp/lib/output.js:89:19)
at MediaRepository.generateThumbnail (/usr/src/app/dist/repositories/media.repository.js:69:14)
at PersonService.handleGeneratePersonThumbnail (/usr/src/app/dist/services/person.service.js:429:36)
at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
at async /usr/src/app/dist/services/job.service.js:148:36
at async Worker.processJob (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:394:28)
at async Worker.retryIfFailed (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:581:24)
[Nest] 7 - 06/12/2024, 10:09:05 PM ERROR [Microservices:JobService] Object:
{
"id": "b98c0512-2cb8-4c2f-a290-eda3280fbc28"
}

    all values are hidden

[Nest] 7 - 06/12/2024, 10:09:05 PM VERBOSE [Microservices:PersonService] Cropping face for person: c5c7dc6e-dd22-446f-b1ee-0470056488d4
[Nest] 7 - 06/12/2024, 10:09:05 PM ERROR [Microservices:JobService] Unable to run job handler (thumbnailGeneration/generate-person-thumbnail): Error: Input file contains unsupported image format
[Nest] 7 - 06/12/2024, 10:09:05 PM ERROR [Microservices:JobService] Error: Input file contains unsupported image format
at Sharp.toFile (/usr/src/app/node_modules/sharp/lib/output.js:89:19)
at MediaRepository.generateThumbnail (/usr/src/app/dist/repositories/media.repository.js:69:14)
at PersonService.handleGeneratePersonThumbnail (/usr/src/app/dist/services/person.service.js:429:36)
at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
at async /usr/src/app/dist/services/job.service.js:148:36
at async Worker.processJob (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:394:28)
at async Worker.retryIfFailed (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:581:24)
[Nest] 7 - 06/12/2024, 10:09:05 PM ERROR [Microservices:JobService] Object:
{
"id": "c5c7dc6e-dd22-446f-b1ee-0470056488d4"
}

    all values are hidden

[Nest] 7 - 06/12/2024, 10:09:05 PM VERBOSE [Microservices:PersonService] Cropping face for person: 9c4e283e-e6b8-4050-ad2e-040e3d5cf131
[Nest] 7 - 06/12/2024, 10:09:05 PM ERROR [Microservices:JobService] Unable to run job handler (thumbnailGeneration/generate-person-thumbnail): Error: Input file contains unsupported image format
[Nest] 7 - 06/12/2024, 10:09:05 PM ERROR [Microservices:JobService] Error: Input file contains unsupported image format
at Sharp.toFile (/usr/src/app/node_modules/sharp/lib/output.js:89:19)
at MediaRepository.generateThumbnail (/usr/src/app/dist/repositories/media.repository.js:69:14)
at PersonService.handleGeneratePersonThumbnail (/usr/src/app/dist/services/person.service.js:429:36)
at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
at async /usr/src/app/dist/services/job.service.js:148:36
at async Worker.processJob (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:394:28)
at async Worker.retryIfFailed (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:581:24)
[Nest] 7 - 06/12/2024, 10:09:05 PM ERROR [Microservices:JobService] Object:
{
"id": "9c4e283e-e6b8-4050-ad2e-040e3d5cf131"
}

    all values are hidden

[Nest] 7 - 06/12/2024, 10:09:05 PM VERBOSE [Microservices:PersonService] Cropping face for person: f7260144-2011-4df4-8eb2-cccf2fde55d0
[Nest] 7 - 06/12/2024, 10:09:05 PM ERROR [Microservices:JobService] Unable to run job handler (thumbnailGeneration/generate-person-thumbnail): Error: Input file contains unsupported image format
[Nest] 7 - 06/12/2024, 10:09:05 PM ERROR [Microservices:JobService] Error: Input file contains unsupported image format
at Sharp.toFile (/usr/src/app/node_modules/sharp/lib/output.js:89:19)
at MediaRepository.generateThumbnail (/usr/src/app/dist/repositories/media.repository.js:69:14)
at PersonService.handleGeneratePersonThumbnail (/usr/src/app/dist/services/person.service.js:429:36)
at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
at async /usr/src/app/dist/services/job.service.js:148:36
at async Worker.processJob (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:394:28)
at async Worker.retryIfFailed (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:581:24)
[Nest] 7 - 06/12/2024, 10:09:05 PM ERROR [Microservices:JobService] Object:
{
"id": "f7260144-2011-4df4-8eb2-cccf2fde55d0"
}

    all values are hidden

[Nest] 7 - 06/12/2024, 10:09:05 PM VERBOSE [Microservices:PersonService] Cropping face for person: a0f19a39-b18f-449a-b88d-4434fab6efc3
[Nest] 7 - 06/12/2024, 10:09:05 PM ERROR [Microservices:JobService] Unable to run job handler (thumbnailGeneration/generate-person-thumbnail): Error: Input file contains unsupported image format
[Nest] 7 - 06/12/2024, 10:09:05 PM ERROR [Microservices:JobService] Error: Input file contains unsupported image format
at Sharp.toFile (/usr/src/app/node_modules/sharp/lib/output.js:89:19)
at MediaRepository.generateThumbnail (/usr/src/app/dist/repositories/media.repository.js:69:14)
at PersonService.handleGeneratePersonThumbnail (/usr/src/app/dist/services/person.service.js:429:36)
at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
at async /usr/src/app/dist/services/job.service.js:148:36
at async Worker.processJob (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:394:28)
at async Worker.retryIfFailed (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:581:24)
[Nest] 7 - 06/12/2024, 10:09:05 PM ERROR [Microservices:JobService] Object:
{
"id": "a0f19a39-b18f-449a-b88d-4434fab6efc3"
}

    all values are hidden

[Nest] 7 - 06/12/2024, 10:09:05 PM VERBOSE [Microservices:PersonService] Cropping face for person: 98a84581-ff5d-4849-bbd8-91470f8d8d98
[Nest] 7 - 06/12/2024, 10:09:05 PM ERROR [Microservices:JobService] Unable to run job handler (thumbnailGeneration/generate-person-thumbnail): Error: Input file contains unsupported image format
[Nest] 7 - 06/12/2024, 10:09:05 PM ERROR [Microservices:JobService] Error: Input file contains unsupported image format
at Sharp.toFile (/usr/src/app/node_modules/sharp/lib/output.js:89:19)
at MediaRepository.generateThumbnail (/usr/src/app/dist/repositories/media.repository.js:69:14)
at PersonService.handleGeneratePersonThumbnail (/usr/src/app/dist/services/person.service.js:429:36)
at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
at async /usr/src/app/dist/services/job.service.js:148:36
at async Worker.processJob (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:394:28)
at async Worker.retryIfFailed (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:581:24)
[Nest] 7 - 06/12/2024, 10:09:05 PM ERROR [Microservices:JobService] Object:
{
"id": "98a84581-ff5d-4849-bbd8-91470f8d8d98"
}

    all values are hidden

[Nest] 7 - 06/12/2024, 10:09:05 PM VERBOSE [Microservices:PersonService] Cropping face for person: 9e34737f-6bd6-4ea1-bd0b-fd37b28346ee
[Nest] 7 - 06/12/2024, 10:09:05 PM ERROR [Microservices:JobService] Unable to run job handler (thumbnailGeneration/generate-person-thumbnail): Error: Input file contains unsupported image format
[Nest] 7 - 06/12/2024, 10:09:05 PM ERROR [Microservices:JobService] Error: Input file contains unsupported image format
at Sharp.toFile (/usr/src/app/node_modules/sharp/lib/output.js:89:19)
at MediaRepository.generateThumbnail (/usr/src/app/dist/repositories/media.repository.js:69:14)
at PersonService.handleGeneratePersonThumbnail (/usr/src/app/dist/services/person.service.js:429:36)
at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
at async /usr/src/app/dist/services/job.service.js:148:36
at async Worker.processJob (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:394:28)
at async Worker.retryIfFailed (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:581:24)
[Nest] 7 - 06/12/2024, 10:09:05 PM ERROR [Microservices:JobService] Object:
{
"id": "9e34737f-6bd6-4ea1-bd0b-fd37b28346ee"
}

    all values are hidden

[Nest] 7 - 06/12/2024, 10:09:05 PM VERBOSE [Microservices:PersonService] Cropping face for person: 779f63f0-ef25-4b40-bc4c-c6cb8198e51f
[Nest] 7 - 06/12/2024, 10:09:05 PM ERROR [Microservices:JobService] Unable to run job handler (thumbnailGeneration/generate-person-thumbnail): Error: Input file contains unsupported image format
[Nest] 7 - 06/12/2024, 10:09:05 PM ERROR [Microservices:JobService] Error: Input file contains unsupported image format
at Sharp.toFile (/usr/src/app/node_modules/sharp/lib/output.js:89:19)
at MediaRepository.generateThumbnail (/usr/src/app/dist/repositories/media.repository.js:69:14)
at PersonService.handleGeneratePersonThumbnail (/usr/src/app/dist/services/person.service.js:429:36)
at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
at async /usr/src/app/dist/services/job.service.js:148:36
at async Worker.processJob (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:394:28)
at async Worker.retryIfFailed (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:581:24)
[Nest] 7 - 06/12/2024, 10:09:05 PM ERROR [Microservices:JobService] Object:
{
"id": "779f63f0-ef25-4b40-bc4c-c6cb8198e51f"
}

    all values are hidden

[Nest] 7 - 06/12/2024, 10:09:05 PM VERBOSE [Microservices:PersonService] Cropping face for person: 94e2af6b-aa00-49ae-97a9-758b18e8fde1
[Nest] 7 - 06/12/2024, 10:09:05 PM ERROR [Microservices:JobService] Unable to run job handler (thumbnailGeneration/generate-person-thumbnail): Error: Input file contains unsupported image format
[Nest] 7 - 06/12/2024, 10:09:05 PM ERROR [Microservices:JobService] Error: Input file contains unsupported image format
at Sharp.toFile (/usr/src/app/node_modules/sharp/lib/output.js:89:19)
at MediaRepository.generateThumbnail (/usr/src/app/dist/repositories/media.repository.js:69:14)
at PersonService.handleGeneratePersonThumbnail (/usr/src/app/dist/services/person.service.js:429:36)
at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
at async /usr/src/app/dist/services/job.service.js:148:36
at async Worker.processJob (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:394:28)
at async Worker.retryIfFailed (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:581:24)
[Nest] 7 - 06/12/2024, 10:09:05 PM ERROR [Microservices:JobService] Object:
{
"id": "94e2af6b-aa00-49ae-97a9-758b18e8fde1"
}

    all values are hidden

[Nest] 7 - 06/12/2024, 10:09:05 PM VERBOSE [Microservices:PersonService] Cropping face for person: bbdc1a7d-b637-4335-8583-678ba4433cf8
[Nest] 7 - 06/12/2024, 10:09:06 PM ERROR [Microservices:JobService] Unable to run job handler (thumbnailGeneration/generate-person-thumbnail): Error: Input file contains unsupported image format
[Nest] 7 - 06/12/2024, 10:09:06 PM ERROR [Microservices:JobService] Error: Input file contains unsupported image format
at Sharp.toFile (/usr/src/app/node_modules/sharp/lib/output.js:89:19)
at MediaRepository.generateThumbnail (/usr/src/app/dist/repositories/media.repository.js:69:14)
at PersonService.handleGeneratePersonThumbnail (/usr/src/app/dist/services/person.service.js:429:36)
at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
at async /usr/src/app/dist/services/job.service.js:148:36
at async Worker.processJob (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:394:28)
at async Worker.retryIfFailed (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:581:24)
[Nest] 7 - 06/12/2024, 10:09:06 PM ERROR [Microservices:JobService] Object:
{
"id": "bbdc1a7d-b637-4335-8583-678ba4433cf8"
}

    all values are hidden

[Nest] 7 - 06/12/2024, 10:09:06 PM VERBOSE [Microservices:PersonService] Cropping face for person: b42fd151-b76c-4447-956a-0aba0ff77c42
[Nest] 7 - 06/12/2024, 10:09:06 PM ERROR [Microservices:JobService] Unable to run job handler (thumbnailGeneration/generate-person-thumbnail): Error: Input file contains unsupported image format
[Nest] 7 - 06/12/2024, 10:09:06 PM ERROR [Microservices:JobService] Error: Input file contains unsupported image format
at Sharp.toFile (/usr/src/app/node_modules/sharp/lib/output.js:89:19)
at MediaRepository.generateThumbnail (/usr/src/app/dist/repositories/media.repository.js:69:14)
at PersonService.handleGeneratePersonThumbnail (/usr/src/app/dist/services/person.service.js:429:36)
at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
at async /usr/src/app/dist/services/job.service.js:148:36
at async Worker.processJob (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:394:28)
at async Worker.retryIfFailed (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:581:24)
[Nest] 7 - 06/12/2024, 10:09:06 PM ERROR [Microservices:JobService] Object:
{
"id": "b42fd151-b76c-4447-956a-0aba0ff77c42"
}

    all values are hidden

[Nest] 7 - 06/12/2024, 10:09:06 PM VERBOSE [Microservices:PersonService] Cropping face for person: e9bdfff7-9667-4c4b-aa12-65cbf98a9bd8
[Nest] 7 - 06/12/2024, 10:09:06 PM ERROR [Microservices:JobService] Unable to run job handler (thumbnailGeneration/generate-person-thumbnail): Error: Input file contains unsupported image format
[Nest] 7 - 06/12/2024, 10:09:06 PM ERROR [Microservices:JobService] Error: Input file contains unsupported image format
at Sharp.toFile (/usr/src/app/node_modules/sharp/lib/output.js:89:19)
at MediaRepository.generateThumbnail (/usr/src/app/dist/repositories/media.repository.js:69:14)
at PersonService.handleGeneratePersonThumbnail (/usr/src/app/dist/services/person.service.js:429:36)
at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
at async /usr/src/app/dist/services/job.service.js:148:36
at async Worker.processJob (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:394:28)
at async Worker.retryIfFailed (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:581:24)
[Nest] 7 - 06/12/2024, 10:09:06 PM ERROR [Microservices:JobService] Object:
{
"id": "e9bdfff7-9667-4c4b-aa12-65cbf98a9bd8"
}

    all values are hidden

[Nest] 7 - 06/12/2024, 10:09:06 PM VERBOSE [Microservices:PersonService] Cropping face for person: d2b5ea1c-8a6c-4303-bacf-c0ec01e6800f
[Nest] 7 - 06/12/2024, 10:09:06 PM ERROR [Microservices:JobService] Unable to run job handler (thumbnailGeneration/generate-person-thumbnail): Error: Input file contains unsupported image format
[Nest] 7 - 06/12/2024, 10:09:06 PM ERROR [Microservices:JobService] Error: Input file contains unsupported image format
at Sharp.toFile (/usr/src/app/node_modules/sharp/lib/output.js:89:19)
at MediaRepository.generateThumbnail (/usr/src/app/dist/repositories/media.repository.js:69:14)
at PersonService.handleGeneratePersonThumbnail (/usr/src/app/dist/services/person.service.js:429:36)
at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
at async /usr/src/app/dist/services/job.service.js:148:36
at async Worker.processJob (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:394:28)
at async Worker.retryIfFailed (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:581:24)
[Nest] 7 - 06/12/2024, 10:09:06 PM ERROR [Microservices:JobService] Object:
{
"id": "d2b5ea1c-8a6c-4303-bacf-c0ec01e6800f"
}

    all values are hidden

[Nest] 7 - 06/12/2024, 10:09:06 PM VERBOSE [Microservices:PersonService] Cropping face for person: de62bc16-f43d-430f-ac36-f769eab2f129
[Nest] 7 - 06/12/2024, 10:09:06 PM ERROR [Microservices:JobService] Unable to run job handler (thumbnailGeneration/generate-person-thumbnail): Error: Input file contains unsupported image format
[Nest] 7 - 06/12/2024, 10:09:06 PM ERROR [Microservices:JobService] Error: Input file contains unsupported image format
at Sharp.toFile (/usr/src/app/node_modules/sharp/lib/output.js:89:19)
at MediaRepository.generateThumbnail (/usr/src/app/dist/repositories/media.repository.js:69:14)
at PersonService.handleGeneratePersonThumbnail (/usr/src/app/dist/services/person.service.js:429:36)
at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
at async /usr/src/app/dist/services/job.service.js:148:36
at async Worker.processJob (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:394:28)
at async Worker.retryIfFailed (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:581:24)
[Nest] 7 - 06/12/2024, 10:09:06 PM ERROR [Microservices:JobService] Object:
{
"id": "de62bc16-f43d-430f-ac36-f769eab2f129"
}

    all values are hidden

[Nest] 7 - 06/12/2024, 10:09:06 PM VERBOSE [Microservices:PersonService] Cropping face for person: ad96190a-23f0-43f4-bada-833898e87178
[Nest] 7 - 06/12/2024, 10:09:06 PM ERROR [Microservices:JobService] Unable to run job handler (thumbnailGeneration/generate-person-thumbnail): Error: Input file contains unsupported image format
[Nest] 7 - 06/12/2024, 10:09:06 PM ERROR [Microservices:JobService] Error: Input file contains unsupported image format
at Sharp.toFile (/usr/src/app/node_modules/sharp/lib/output.js:89:19)
at MediaRepository.generateThumbnail (/usr/src/app/dist/repositories/media.repository.js:69:14)
at PersonService.handleGeneratePersonThumbnail (/usr/src/app/dist/services/person.service.js:429:36)
at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
at async /usr/src/app/dist/services/job.service.js:148:36
at async Worker.processJob (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:394:28)
at async Worker.retryIfFailed (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:581:24)
[Nest] 7 - 06/12/2024, 10:09:06 PM ERROR [Microservices:JobService] Object:
{
"id": "ad96190a-23f0-43f4-bada-833898e87178"
}

    all values are hidden

[Nest] 7 - 06/12/2024, 10:09:06 PM VERBOSE [Microservices:PersonService] Cropping face for person: 4a5d85d2-561f-4a99-854c-3474cc231a1d
[Nest] 7 - 06/12/2024, 10:09:06 PM ERROR [Microservices:JobService] Unable to run job handler (thumbnailGeneration/generate-person-thumbnail): Error: Input file contains unsupported image format
[Nest] 7 - 06/12/2024, 10:09:06 PM ERROR [Microservices:JobService] Error: Input file contains unsupported image format
at Sharp.toFile (/usr/src/app/node_modules/sharp/lib/output.js:89:19)
at MediaRepository.generateThumbnail (/usr/src/app/dist/repositories/media.repository.js:69:14)
at PersonService.handleGeneratePersonThumbnail (/usr/src/app/dist/services/person.service.js:429:36)
at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
at async /usr/src/app/dist/services/job.service.js:148:36
at async Worker.processJob (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:394:28)
at async Worker.retryIfFailed (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:581:24)
[Nest] 7 - 06/12/2024, 10:09:06 PM ERROR [Microservices:JobService] Object:
{
"id": "4a5d85d2-561f-4a99-854c-3474cc231a1d"
}

    all values are hidden

[Nest] 7 - 06/12/2024, 10:09:06 PM VERBOSE [Microservices:PersonService] Cropping face for person: 9483894c-8385-42de-bb6b-c23e34e57207
[Nest] 7 - 06/12/2024, 10:09:06 PM ERROR [Microservices:JobService] Unable to run job handler (thumbnailGeneration/generate-person-thumbnail): Error: Input file contains unsupported image format
[Nest] 7 - 06/12/2024, 10:09:06 PM ERROR [Microservices:JobService] Error: Input file contains unsupported image format
at Sharp.toFile (/usr/src/app/node_modules/sharp/lib/output.js:89:19)
at MediaRepository.generateThumbnail (/usr/src/app/dist/repositories/media.repository.js:69:14)
at PersonService.handleGeneratePersonThumbnail (/usr/src/app/dist/services/person.service.js:429:36)
at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
at async /usr/src/app/dist/services/job.service.js:148:36
at async Worker.processJob (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:394:28)
at async Worker.retryIfFailed (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:581:24)
[Nest] 7 - 06/12/2024, 10:09:06 PM ERROR [Microservices:JobService] Object:
{
"id": "9483894c-8385-42de-bb6b-c23e34e57207"
}

    all values are hidden

[Nest] 7 - 06/12/2024, 10:09:06 PM VERBOSE [Microservices:PersonService] Cropping face for person: f7451031-a63d-47f5-87e4-bc2c964ee8e3
[Nest] 7 - 06/12/2024, 10:09:06 PM ERROR [Microservices:JobService] Unable to run job handler (thumbnailGeneration/generate-person-thumbnail): Error: Input file contains unsupported image format
[Nest] 7 - 06/12/2024, 10:09:06 PM ERROR [Microservices:JobService] Error: Input file contains unsupported image format
at Sharp.toFile (/usr/src/app/node_modules/sharp/lib/output.js:89:19)
at MediaRepository.generateThumbnail (/usr/src/app/dist/repositories/media.repository.js:69:14)
at PersonService.handleGeneratePersonThumbnail (/usr/src/app/dist/services/person.service.js:429:36)
at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
at async /usr/src/app/dist/services/job.service.js:148:36
at async Worker.processJob (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:394:28)
at async Worker.retryIfFailed (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:581:24)
[Nest] 7 - 06/12/2024, 10:09:06 PM ERROR [Microservices:JobService] Object:
{
"id": "f7451031-a63d-47f5-87e4-bc2c964ee8e3"
}

    all values are hidden

[Nest] 17 - 06/12/2024, 10:09:08 PM DEBUG [Api:LoggingInterceptor~6uebmnw9] GET /api/jobs 200 4.70ms 192.168.0.201
[Nest] 17 - 06/12/2024, 10:09:14 PM DEBUG [Api:LoggingInterceptor~7jn37qgz] GET /api/jobs 200 232.43ms 192.168.0.201
[Nest] 17 - 06/12/2024, 10:09:20 PM DEBUG [Api:LoggingInterceptor~b6lww6fn] GET /api/jobs 200 34.84ms 192.168.0.201
[Nest] 17 - 06/12/2024, 10:09:25 PM DEBUG [Api:LoggingInterceptor~a46gxvui] GET /api/jobs 200 8.58ms 192.168.0.201
[Nest] 17 - 06/12/2024, 10:09:31 PM DEBUG [Api:LoggingInterceptor~wvofbvcj] GET /api/jobs 200 21.43ms 192.168.0.201
[Nest] 17 - 06/12/2024, 10:09:37 PM DEBUG [Api:LoggingInterceptor~wg55esg8] GET /api/jobs 200 9.99ms 192.168.0.201
[Nest] 17 - 06/12/2024, 10:09:42 PM DEBUG [Api:LoggingInterceptor~xowj7g7j] GET /api/jobs 200 23.94ms 192.168.0.201
[Nest] 17 - 06/12/2024, 10:09:48 PM DEBUG [Api:LoggingInterceptor~csry0p5s] GET /api/jobs 200 3.87ms 192.168.0.201
[Nest] 17 - 06/12/2024, 10:09:53 PM DEBUG [Api:LoggingInterceptor~umxx7vdj] GET /api/jobs 200 8.63ms 192.168.0.201
[...]
[Nest] 17 - 06/12/2024, 10:23:12 PM DEBUG [Api:LoggingInterceptor~jojnqmwq] GET /api/jobs 200 45.54ms 192.168.0.201
[Nest] 17 - 06/12/2024, 10:23:18 PM DEBUG [Api:LoggingInterceptor~rvgz32em] GET /api/jobs 200 9.83ms 192.168.0.201
[Nest] 17 - 06/12/2024, 10:23:23 PM DEBUG [Api:LoggingInterceptor~kjnbrvop] GET /api/jobs 200 20.24ms 192.168.0.201
[Nest] 17 - 06/12/2024, 10:23:28 PM DEBUG [Api:LoggingInterceptor~7624em7n] GET /api/jobs 200 19.44ms 192.168.0.201
[Nest] 17 - 06/12/2024, 10:23:34 PM DEBUG [Api:LoggingInterceptor~2upxhdlv] GET /api/jobs 200 137.55ms 192.168.0.201
[Nest] 17 - 06/12/2024, 10:23:41 PM DEBUG [Api:LoggingInterceptor~vfz4qwoa] GET /api/jobs 200 55.67ms 192.168.0.201
[Nest] 17 - 06/12/2024, 10:23:51 PM DEBUG [Api:LoggingInterceptor~w99x2sew] GET /api/jobs 200 53.48ms 192.168.0.201
[Nest] 17 - 06/12/2024, 10:23:56 PM DEBUG [Api:LoggingInterceptor~nd53zvbj] GET /api/jobs 200 15.28ms 192.168.0.201
[Nest] 17 - 06/12/2024, 10:24:03 PM DEBUG [Api:LoggingInterceptor~jxfcr0ab] GET /api/jobs 200 28.12ms 192.168.0.201
[Nest] 17 - 06/12/2024, 10:24:09 PM DEBUG [Api:LoggingInterceptor~0cha5l9h] GET /api/jobs 200 204.41ms 192.168.0.201
[Nest] 17 - 06/12/2024, 10:24:15 PM DEBUG [Api:LoggingInterceptor~er8mru1v] GET /api/jobs 200 82.89ms 192.168.0.201
[Nest] 17 - 06/12/2024, 10:24:21 PM DEBUG [Api:LoggingInterceptor~cna77vxp] GET /api/jobs 200 28.74ms 192.168.0.201
[Nest] 17 - 06/12/2024, 10:24:28 PM DEBUG [Api:LoggingInterceptor~cxe8u9ex] GET /api/jobs 200 123.74ms 192.168.0.201
[Nest] 17 - 06/12/2024, 10:24:33 PM DEBUG [Api:LoggingInterceptor~zkdv7em3] GET /api/jobs 200 6.36ms 192.168.0.201
[Nest] 17 - 06/12/2024, 10:24:39 PM DEBUG [Api:LoggingInterceptor~09ewiqa4] GET /api/jobs 200 4.34ms 192.168.0.201
Container stopped
Container started
Detected CPU Cores: 6

The job will stay for ca 10 minutes on active 1; waiting: 0, fill the memory without posting anything, except my client viewing the page.

/edit: I removed a read-only external library and now the job is fine again. Need to check what asset(s) causes the leak.
/edit2: Found one PSD file that crashed my server. 208 MP, 11792 x 17649, 47,5 MiB - Was 2gig before redacting (but still crashing). I can upload it to immich, but the thumbnail job won't finish.
File: immich-stuck-thumbnail_PSD.zip

DRIgnazGortngschirl · 2024-07-11T19:52:55Z

I recently uploaded approximately 93MB (~180 files) and noticed that as soon as the images were uploaded and I started the jobs, the memory usage spiked. When the face detection job began, my memory usage jumped from about 600MB to 4GB, which was my maximum at that time. I increased the memory allocation, thinking that Buffalo-L might need more RAM. I kept increasing it until I started receiving error logs. Prior to this, there were no error logs, likely because the memory filled up so quickly that the logs couldn't be written. Once the memory was fully utilized, the CPU usage also surged to 100%.

immich_machine_learning  | [07/11/24 19:27:57] INFO     Booting worker with pid: 54
immich_machine_learning  | [07/11/24 19:28:00] INFO     Started server process [54]
immich_machine_learning  | [07/11/24 19:28:00] INFO     Waiting for application startup.
immich_machine_learning  | [07/11/24 19:28:00] INFO     Created in-memory cache with unloading after 300s
immich_machine_learning  |                              of inactivity.
immich_machine_learning  | [07/11/24 19:28:00] INFO     Initialized request thread pool with 16 threads.
immich_machine_learning  | [07/11/24 19:28:00] INFO     Application startup complete.
immich_machine_learning  | [07/11/24 19:28:00] INFO     Loading detection model 'buffalo_l' to memory
immich_machine_learning  | [07/11/24 19:28:00] INFO     Setting execution providers to
immich_machine_learning  |                              ['CPUExecutionProvider'], in descending order of
immich_machine_learning  |                              preference
immich_machine_learning  | [07/11/24 19:28:01] INFO     Loading recognition model 'buffalo_l' to memory
immich_machine_learning  | [07/11/24 19:28:01] INFO     Setting execution providers to
immich_machine_learning  |                              ['CPUExecutionProvider'], in descending order of
immich_machine_learning  |                              preference
immich_machine_learning  | [07/11/24 19:28:26] ERROR    Worker (pid:54) was sent SIGKILL! Perhaps out of
immich_machine_learning  |                              memory?
immich_server            | [Nest] 7  - 07/11/2024, 7:28:26 PM   ERROR [Microservices:JobService] Unable to run job handler (faceDetection/face-detection): Error: Machine learning request to "http://immich-machine-learning:3003" failed with SocketError: other side closed
immich_server            | [Nest] 7  - 07/11/2024, 7:28:26 PM   ERROR [Microservices:JobService] Error: Machine learning request to "http://immich-machine-learning:3003" failed with SocketError: other side closed
immich_server            |     at /usr/src/app/dist/repositories/machine-learning.repository.js:19:19
immich_server            |     at async MachineLearningRepository.predict (/usr/src/app/dist/repositories/machine-learning.repository.js:18:21)
immich_server            |     at async MachineLearningRepository.detectFaces (/usr/src/app/dist/repositories/machine-learning.repository.js:33:26)
immich_server            |     at async PersonService.handleDetectFaces (/usr/src/app/dist/services/person.service.js:275:52)
immich_server            |     at async /usr/src/app/dist/services/job.service.js:148:36
immich_server            |     at async Worker.processJob (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:394:28)
immich_server            |     at async Worker.retryIfFailed (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:581:24)
immich_server            | [Nest] 7  - 07/11/2024, 7:28:26 PM   ERROR [Microservices:JobService] Object:
immich_server            | {
immich_server            |   "id": "be1f37e0-0390-46da-aab5-1f0864c35941"
immich_server            | }
immich_server            |
immich_machine_learning  | [07/11/24 19:28:26] INFO     Booting worker with pid: 76
immich_machine_learning  | [07/11/24 19:28:29] INFO     Started server process [76]
immich_machine_learning  | [07/11/24 19:28:29] INFO     Waiting for application startup.
immich_machine_learning  | [07/11/24 19:28:29] INFO     Created in-memory cache with unloading after 300s
immich_machine_learning  |                              of inactivity.
immich_machine_learning  | [07/11/24 19:28:29] INFO     Initialized request thread pool with 16 threads.
immich_machine_learning  | [07/11/24 19:28:29] INFO     Application startup complete.
immich_machine_learning  | [07/11/24 19:28:29] INFO     Loading detection model 'buffalo_l' to memory
immich_machine_learning  | [07/11/24 19:28:29] INFO     Setting execution providers to
immich_machine_learning  |                              ['CPUExecutionProvider'], in descending order of
immich_machine_learning  |                              preference
immich_machine_learning  | [07/11/24 19:28:30] INFO     Loading recognition model 'buffalo_l' to memory
immich_machine_learning  | [07/11/24 19:28:30] INFO     Setting execution providers to
immich_machine_learning  |                              ['CPUExecutionProvider'], in descending order of
immich_machine_learning  |                              preference
immich_machine_learning  | [07/11/24 19:28:51] ERROR    Worker (pid:76) was sent SIGKILL! Perhaps out of
immich_machine_learning  |                              memory?
immich_server            | [Nest] 7  - 07/11/2024, 7:28:51 PM   ERROR [Microservices:JobService] Unable to run job handler (faceDetection/face-detection): Error: Machine learning request to "http://immich-machine-learning:3003" failed with SocketError: other side closed
immich_server            | [Nest] 7  - 07/11/2024, 7:28:51 PM   ERROR [Microservices:JobService] Error: Machine learning request to "http://immich-machine-learning:3003" failed with SocketError: other side closed
immich_server            |     at /usr/src/app/dist/repositories/machine-learning.repository.js:19:19
immich_server            |     at async MachineLearningRepository.predict (/usr/src/app/dist/repositories/machine-learning.repository.js:18:21)
immich_server            |     at async MachineLearningRepository.detectFaces (/usr/src/app/dist/repositories/machine-learning.repository.js:33:26)
immich_server            |     at async PersonService.handleDetectFaces (/usr/src/app/dist/services/person.service.js:275:52)
immich_server            |     at async /usr/src/app/dist/services/job.service.js:148:36
immich_server            |     at async Worker.processJob (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:394:28)
immich_server            |     at async Worker.retryIfFailed (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:581:24)
immich_server            | [Nest] 7  - 07/11/2024, 7:28:51 PM   ERROR [Microservices:JobService] Object:
immich_server            | {
immich_server            |   "id": "c66e8bd6-24e7-45b8-8ba6-0e6634577e14"
immich_server            | }
immich_server            |
immich_machine_learning  | [07/11/24 19:28:51] INFO     Booting worker with pid: 98
immich_machine_learning  | [07/11/24 19:28:53] INFO     Started server process [98]
immich_machine_learning  | [07/11/24 19:28:53] INFO     Waiting for application startup.
immich_machine_learning  | [07/11/24 19:28:53] INFO     Created in-memory cache with unloading after 300s
immich_machine_learning  |                              of inactivity.
immich_machine_learning  | [07/11/24 19:28:53] INFO     Initialized request thread pool with 16 threads.
immich_machine_learning  | [07/11/24 19:28:53] INFO     Application startup complete.
immich_machine_learning  | [07/11/24 19:28:53] INFO     Loading detection model 'buffalo_l' to memory
immich_machine_learning  | [07/11/24 19:28:53] INFO     Setting execution providers to
immich_machine_learning  |                              ['CPUExecutionProvider'], in descending order of
immich_machine_learning  |                              preference
immich_machine_learning  | [07/11/24 19:28:54] INFO     Loading recognition model 'buffalo_l' to memory
immich_machine_learning  | [07/11/24 19:28:54] INFO     Setting execution providers to
immich_machine_learning  |                              ['CPUExecutionProvider'], in descending order of
immich_machine_learning  |                              preference
immich_server            | [Nest] 7  - 07/11/2024, 7:29:13 PM   ERROR [Microservices:JobService] Unable to run job handler (faceDetection/face-detection): Error: Machine learning request to "http://immich-machine-learning:3003" failed with SocketError: other side closed
immich_server            | [Nest] 7  - 07/11/2024, 7:29:13 PM   ERROR [Microservices:JobService] Error: Machine learning request to "http://immich-machine-learning:3003" failed with SocketError: other side closed
immich_server            |     at /usr/src/app/dist/repositories/machine-learning.repository.js:19:19
immich_server            |     at async MachineLearningRepository.predict (/usr/src/app/dist/repositories/machine-learning.repository.js:18:21)
immich_server            |     at async MachineLearningRepository.detectFaces (/usr/src/app/dist/repositories/machine-learning.repository.js:33:26)
immich_server            |     at async PersonService.handleDetectFaces (/usr/src/app/dist/services/person.service.js:275:52)
immich_server            |     at async /usr/src/app/dist/services/job.service.js:148:36
immich_server            |     at async Worker.processJob (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:394:28)
immich_server            |     at async Worker.retryIfFailed (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:581:24)
immich_server            | [Nest] 7  - 07/11/2024, 7:29:13 PM   ERROR [Microservices:JobService] Object:
immich_server            | {
immich_server            |   "id": "fa08cb36-ad88-44c7-ad8b-24950987bf3c"
immich_server            | }
immich_server            |
immich_machine_learning  | [07/11/24 19:29:13] ERROR    Worker (pid:98) was sent SIGKILL! Perhaps out of
immich_machine_learning  |                              memory?
immich_machine_learning  | [07/11/24 19:29:13] INFO     Booting worker with pid: 114
immich_machine_learning  | [07/11/24 19:29:16] INFO     Started server process [114]
immich_machine_learning  | [07/11/24 19:29:16] INFO     Waiting for application startup.
immich_machine_learning  | [07/11/24 19:29:16] INFO     Created in-memory cache with unloading after 300s
immich_machine_learning  |                              of inactivity.
immich_machine_learning  | [07/11/24 19:29:16] INFO     Initialized request thread pool with 16 threads.
immich_machine_learning  | [07/11/24 19:29:16] INFO     Application startup complete.
immich_machine_learning  | [07/11/24 19:29:16] INFO     Loading detection model 'buffalo_l' to memory
immich_machine_learning  | [07/11/24 19:29:16] INFO     Setting execution providers to
immich_machine_learning  |                              ['CPUExecutionProvider'], in descending order of
immich_machine_learning  |                              preference
immich_machine_learning  | [07/11/24 19:29:17] INFO     Loading recognition model 'buffalo_l' to memory
immich_machine_learning  | [07/11/24 19:29:17] INFO     Setting execution providers to
immich_machine_learning  |                              ['CPUExecutionProvider'], in descending order of
immich_machine_learning  |                              preference
immich_server            | [Nest] 7  - 07/11/2024, 7:29:42 PM   ERROR [Microservices:JobService] Unable to run job handler (faceDetection/face-detection): Error: Machine learning request to "http://immich-machine-learning:3003" failed with SocketError: other side closed
immich_server            | [Nest] 7  - 07/11/2024, 7:29:42 PM   ERROR [Microservices:JobService] Error: Machine learning request to "http://immich-machine-learning:3003" failed with SocketError: other side closed
immich_server            |     at /usr/src/app/dist/repositories/machine-learning.repository.js:19:19
immich_server            |     at async MachineLearningRepository.predict (/usr/src/app/dist/repositories/machine-learning.repository.js:18:21)
immich_server            |     at async MachineLearningRepository.detectFaces (/usr/src/app/dist/repositories/machine-learning.repository.js:33:26)
immich_server            |     at async PersonService.handleDetectFaces (/usr/src/app/dist/services/person.service.js:275:52)
immich_server            |     at async /usr/src/app/dist/services/job.service.js:148:36
immich_server            |     at async Worker.processJob (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:394:28)
immich_server            |     at async Worker.retryIfFailed (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:581:24)
immich_server            | [Nest] 7  - 07/11/2024, 7:29:42 PM   ERROR [Microservices:JobService] Object:
immich_server            | {
immich_server            |   "id": "3c01f06f-01aa-481f-9703-0d7e67d41bbd"
immich_server            | }
immich_server            |
immich_machine_learning  | [07/11/24 19:29:42] ERROR    Worker (pid:114) was sent SIGKILL! Perhaps out of
immich_machine_learning  |                              memory?
immich_machine_learning  | [07/11/24 19:29:42] INFO     Booting worker with pid: 136
immich_redis             | 1:M 11 Jul 2024 19:29:44.072 * 10000 changes in 60 seconds. Saving...
immich_redis             | 1:M 11 Jul 2024 19:29:44.072 * Background saving started by pid 22
immich_redis             | 22:C 11 Jul 2024 19:29:44.089 * DB saved on disk
immich_redis             | 22:C 11 Jul 2024 19:29:44.089 * RDB: 0 MB of memory used by copy-on-write
immich_redis             | 1:M 11 Jul 2024 19:29:44.190 * Background saving terminated with success
immich_machine_learning  | [07/11/24 19:29:45] INFO     Started server process [136]
immich_machine_learning  | [07/11/24 19:29:45] INFO     Waiting for application startup.
immich_machine_learning  | [07/11/24 19:29:45] INFO     Created in-memory cache with unloading after 300s
immich_machine_learning  |                              of inactivity.
immich_machine_learning  | [07/11/24 19:29:45] INFO     Initialized request thread pool with 16 threads.
immich_machine_learning  | [07/11/24 19:29:45] INFO     Application startup complete.

jrasm91 · 2024-09-10T04:53:05Z

We changed some settings that should have helped. Is this still an issue?

kdybicz · 2024-09-10T08:10:22Z

On the latest release version server crashed without obvious error:

2024/09/10 09:35:55,stderr,api worker exited with code null
2024/09/10 09:35:42,stderr,
2024/09/10 09:35:42,stderr,}
2024/09/10 09:35:42,stderr,  "id": "74013933-6496-422e-a1ba-d7e98c6332dc"
2024/09/10 09:35:42,stderr,{
2024/09/10 09:35:42,stderr,"�[Nest] 7  - �09/10/2024, 7:35:41 AM �  ERROR� �[Microservices:JobService]� �Object:�"
2024/09/10 09:35:42,stderr,    at async Worker.retryIfFailed (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:581:24)�
2024/09/10 09:35:42,stderr,    at async Worker.processJob (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:394:28)
2024/09/10 09:35:42,stderr,    at async /usr/src/app/dist/services/job.service.js:148:36
2024/09/10 09:35:42,stderr,    at async MediaService.handleGeneratePreview (/usr/src/app/dist/services/media.service.js:141:29)
2024/09/10 09:35:42,stderr,    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
2024/09/10 09:35:42,stderr,    at MediaService.generateThumbnail (/usr/src/app/dist/services/media.service.js:176:48)
2024/09/10 09:35:42,stderr,    at MediaRepository.generateThumbnail (/usr/src/app/dist/repositories/media.repository.js:69:14)
2024/09/10 09:35:42,stderr,    at Sharp.toFile (/usr/src/app/node_modules/sharp/lib/output.js:90:19)
2024/09/10 09:35:42,stderr,"�[Nest] 7  - �09/10/2024, 7:35:41 AM �  ERROR� �[Microservices:JobService]� �Error: Input file contains unsupported image format"
2024/09/10 09:35:42,stderr,"�[Nest] 7  - �09/10/2024, 7:35:41 AM �  ERROR� �[Microservices:JobService]� �Unable to run job handler (thumbnailGeneration/generate-preview): Error: Input file contains unsupported image format�"
2024/09/10 09:35:41,stderr,
2024/09/10 09:35:41,stderr,}
...
2024/09/10 09:35:38,stderr,  "id": "3f35526b-ce3d-40a2-b54e-6a5b19773b3d"
2024/09/10 09:35:38,stderr,{
2024/09/10 09:35:38,stderr,"�[Nest] 7  - �09/10/2024, 7:35:38 AM �  ERROR� �[Microservices:JobService]� �Object:�"
2024/09/10 09:35:38,stderr,    at async Worker.retryIfFailed (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:581:24)�
2024/09/10 09:35:38,stderr,    at async Worker.processJob (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:394:28)
2024/09/10 09:35:38,stderr,    at async /usr/src/app/dist/services/job.service.js:148:36
2024/09/10 09:35:38,stderr,    at async MetadataService.handleMetadataExtraction (/usr/src/app/dist/services/metadata.service.js:194:40)
2024/09/10 09:35:38,stderr,    at async MetadataService.exifData (/usr/src/app/dist/services/metadata.service.js:467:23)
2024/09/10 09:35:38,stderr,    at async Object.stat (node:internal/fs/promises:1032:18)
2024/09/10 09:35:38,stderr,"�[Nest] 7  - �09/10/2024, 7:35:38 AM �  ERROR� �[Microservices:JobService]� �Error: ENOENT: no such file or directory, stat '/usr/src/app/external/Upload/2024/05/IMG_20240509_183102.jpg'"
2024/09/10 09:35:38,stderr,"�[Nest] 7  - �09/10/2024, 7:35:38 AM �  ERROR� �[Microservices:JobService]� �Unable to run job handler (metadataExtraction/metadata-extraction): Error: ENOENT: no such file or directory, stat '/usr/src/app/external/Upload/2024/05/IMG_20240509_183102.jpg'�"
2024/09/10 09:35:38,stderr,
2024/09/10 09:35:38,stderr,}
...
2024/09/10 09:35:30,stderr,"�[Nest] 17  - �09/10/2024, 7:34:07 AM �  ERROR� �[Api:LibraryService]� �Library watcher for library 82616fd4-b0d7-4a16-98bd-a845f000f014 encountered error: Error: ENOSPC: System limit for number of file watchers reached, watch '/usr/src/app/external/#recycle/something/@eaDir'�"
2024/09/10 09:35:30,stderr,"�[Nest] 17  - �09/10/2024, 7:33:35 AM �  ERROR� �[Api:LibraryService]� �Library watcher for library 82616fd4-b0d7-4a16-98bd-a845f000f014 encountered error: Error: ENOSPC: System limit for number of file watchers reached, watch '/usr/src/app/external/2017/something else/@eaDir/cn-tower.jpeg@SynoEAStream'�
...

danieldietzler · 2024-09-21T17:48:26Z

This looks like some images are just borked. You don't run into out of memory issues anymore though, correct?

jrasm91 · 2024-09-22T13:31:21Z

@kdybicz

System limit for number of file watchers reached, watch '/usr/src/app/external/#recycle/something/@eaDir

kdybicz · 2024-09-22T19:01:37Z

@jrasm91 I would love to try to exclude #recycle and @eaDir, if that's what you're suggesting, but... web UI doesn't load. Just tested latest version, still having same issues.

Edit: Scratch that, I've just double checked and looks like I've already added those exclusion patterns:

**/@eaDir/**
**/#recycle/**

🤔

Edit 2: Just after web UI started I paused all of the jobs and forced the removal of offline files, and even though the service stopped logging errors related to files not found or broken, it still crashed a minute later with:

microservices worker error: Error [ERR_WORKER_OUT_OF_MEMORY]: Worker terminated due to reaching memory limit: JS heap out of memory

danieldietzler · 2024-09-22T19:57:55Z

The latest release adds those as default exclusion patterns, that's why they're there :D

kdybicz · 2024-09-22T20:53:34Z

@danieldietzler latest release adds **/@eaDir/** and **/._*, not **/#recycle/** and others I've added manually.

Nevertheless, there must be some other gotcha regarding re-scans and/or offline/missing files, as I just did the classic "turn it off and on again" by nuking my old configuration and processing the library from scratch, and so far so good 🤞🏼

jrasm91 · 2024-09-26T23:36:47Z

The 116 release that came out today also includes significant changes to the library scanning implementation, so you should test and let us know how it goes.

etnoy · 2024-10-08T08:09:13Z

Closing this for now, please let us know if this still is an issue. This code has been improved and hopefully it's fixed

yuhuan417 · 2024-10-19T08:50:55Z

I also encountered this issue on a photo library of 100,000 images, where the memory usage of immich-machine-learning would spike and keep increasing. So, I performed the following steps to identify the problem.

I ran the tasks one by one and found that facial detection was causing the issue. To make it easier to pinpoint the problem, I set the memory limit of immich-machine-learning to 2G and observed from the immich-server logs which photos would cause issues when the program crashed. Some photos, which were not large in size, would cause problems with a high probability (strangely, not 100%). I found one of them, a 2M jpg file, which was a group photo of 20 people. So, I speculated that the memory usage for facial recognition is related to the number of faces.

Upon closer examination of recognition.py, I noticed that facial recognition was done in batch prediction mode. I guessed that there was an issue with the underlying code that prevented memory from being released in this mode. I tried hardcoding "self.batch = self.model_format == ModelFormat.ONNX" to "self.batch = False", and the problem disappeared.

I'm not sure if this solution would work only in my specific environment. My machine is an N4505 with 8G of memory, and I'm using openvino for hardware acceleration.

kdybicz · 2024-10-19T12:17:46Z

That's an interesting observation and might suggest why I still see my memory-limiter server crash without any clear reason 🤔

alextran1502 · 2024-10-19T12:33:53Z

@mertalev can you replicate what @yuhuan417 is seeing?

mertalev · 2024-10-19T15:20:34Z

Yes, I've tested the same thing with @1-tempest and came to the conclusion that OpenVINO allocates completely separate memory for each number of faces it encounters in an image, probably for speed reasons. This means the memory to process 3 faces is completely separate from the memory for 4 faces, etc., and this memory is cached for quick reuse.

On @1-tempest's processor, batching didn't have any positive effect on performance. On the new 155H processor, it had a 16% impact for 5 faces. The latter is notable and would likely be a bigger gap with discrete Arc cards.

I think it makes sense to limit the batch size here in general, and conservatively set it to 1 for OpenVINO by default.

lscorcia · 2024-10-21T01:55:05Z

I tried hardcoding "self.batch = self.model_format == ModelFormat.ONNX" to "self.batch = False", and the problem disappeared.

+1. I can confirm that the suggestion finally fixed the persistent out of memory issue on my Synology NAS. Now memory is nice and stable around 1.5 GB for the immich_machine_learning container during face detection / recognition. Awesome find!

Dunky-Z changed the title ~~immich microservice memory leak kills host~~ [Bug] immich microservice memory leak kills host May 13, 2024

mertalev mentioned this issue May 22, 2024

fix(server): disable sharp cache #9665

Closed

etnoy closed this as completed Oct 8, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Bug] immich microservice memory leak kills host #9414

[Bug] immich microservice memory leak kills host #9414

Dunky-Z commented May 13, 2024 •

edited

Loading

mertalev commented May 13, 2024

Dunky-Z commented May 13, 2024

Dunky-Z commented May 13, 2024

mertalev commented May 13, 2024

Dunky-Z commented May 13, 2024

ceebu commented May 17, 2024

RazerProof commented May 22, 2024 •

edited

Loading

mertalev commented May 22, 2024

RazerProof commented May 22, 2024 •

edited

Loading

RazerProof commented May 22, 2024

mertalev commented May 22, 2024

RazerProof commented May 22, 2024

mertalev commented May 22, 2024

RazerProof commented May 22, 2024

mertalev commented May 22, 2024

RazerProof commented May 22, 2024

ThisIsCyreX commented Jun 12, 2024 •

edited

Loading

DRIgnazGortngschirl commented Jul 11, 2024

jrasm91 commented Sep 10, 2024

kdybicz commented Sep 10, 2024

danieldietzler commented Sep 21, 2024

jrasm91 commented Sep 22, 2024

kdybicz commented Sep 22, 2024 •

edited

Loading

danieldietzler commented Sep 22, 2024

kdybicz commented Sep 22, 2024 •

edited

Loading

jrasm91 commented Sep 26, 2024

etnoy commented Oct 8, 2024

yuhuan417 commented Oct 19, 2024

kdybicz commented Oct 19, 2024

alextran1502 commented Oct 19, 2024

mertalev commented Oct 19, 2024 •

edited

Loading

lscorcia commented Oct 21, 2024

[Bug] immich microservice memory leak kills host #9414

[Bug] immich microservice memory leak kills host #9414

Comments

Dunky-Z commented May 13, 2024 • edited Loading

The bug

The OS that Immich Server is running on

Version of Immich Server

Version of Immich Mobile App

Platform with the issue

Your docker-compose.yml content

Your .env content

Reproduction steps

Relevant log output

Additional information

mertalev commented May 13, 2024

Dunky-Z commented May 13, 2024

Dunky-Z commented May 13, 2024

mertalev commented May 13, 2024

Dunky-Z commented May 13, 2024

ceebu commented May 17, 2024

RazerProof commented May 22, 2024 • edited Loading

mertalev commented May 22, 2024

RazerProof commented May 22, 2024 • edited Loading

RazerProof commented May 22, 2024

mertalev commented May 22, 2024

RazerProof commented May 22, 2024

mertalev commented May 22, 2024

RazerProof commented May 22, 2024

mertalev commented May 22, 2024

RazerProof commented May 22, 2024

ThisIsCyreX commented Jun 12, 2024 • edited Loading

DRIgnazGortngschirl commented Jul 11, 2024

jrasm91 commented Sep 10, 2024

kdybicz commented Sep 10, 2024

danieldietzler commented Sep 21, 2024

jrasm91 commented Sep 22, 2024

kdybicz commented Sep 22, 2024 • edited Loading

danieldietzler commented Sep 22, 2024

kdybicz commented Sep 22, 2024 • edited Loading

jrasm91 commented Sep 26, 2024

etnoy commented Oct 8, 2024

yuhuan417 commented Oct 19, 2024

kdybicz commented Oct 19, 2024

alextran1502 commented Oct 19, 2024

mertalev commented Oct 19, 2024 • edited Loading

lscorcia commented Oct 21, 2024

Dunky-Z commented May 13, 2024 •

edited

Loading

RazerProof commented May 22, 2024 •

edited

Loading

RazerProof commented May 22, 2024 •

edited

Loading

ThisIsCyreX commented Jun 12, 2024 •

edited

Loading

kdybicz commented Sep 22, 2024 •

edited

Loading

kdybicz commented Sep 22, 2024 •

edited

Loading

mertalev commented Oct 19, 2024 •

edited

Loading