Skip to content

[Bug]: Errors in server logs while deploying a service with gRPC workers #4300

Description

@un-def

Steps to reproduce

Create a service (the same configuration as in #4240 issue):

type: service
name: qwen38-27b-agentic-smg

groups:
  - replicas: 1
    python: "3.12"
    commands:
      - pip install smg
      - smg launch --host 0.0.0.0 --port 8000 --enable-igw --policy cache_aware --model-path Qwen/Qwen3.8-27B
    router:
      type: sglang
    resources:
      gpu: 0

  - replicas: 1
    image: vllm/vllm-openai:qwen38
    commands:
      - pip install smg-grpc-servicer
      - python3 -m vllm.entrypoints.grpc_server --host 0.0.0.0 --port 8000 --model Qwen/Qwen3.8-27B-FP8
    resources:
      gpu: H100:80GB

port: 8000
model:
  type: chat
  name: Qwen/Qwen3.8-27B
  format: openai
  prefix: /v1

Actual behaviour

There are two errors in the server logs:

  1. Parent instance <RunModel at 0x78af662b9e40> is not bound to a Session; deferred load operation of attribute 'run_name' cannot proceed – occurs once
  2. Could not fetch server_info for worker http://10.0.0.3:8000: ReadError('') – repeats until the worker is ready

Expected behaviour

No response

dstack version

0.22.1

Server logs

DEBUG    dstack._internal.server.background.pipeline_tasks.base:357 Processing service_router_worker_sync item
         648bb027-2a5c-447e-80c7-a8e0b391bc37
ERROR    dstack._internal.server.background.pipeline_tasks.base:361 Unexpected exception when processing item
         Traceback (most recent call last):
           File "/home/def/dev/dstack/src/dstack/_internal/server/background/pipeline_tasks/base.py", line 359, in start
             await self.process(item)
           File "/home/def/dev/dstack/src/dstack/_internal/server/utils/tracing.py", line 26, in wrapper
             result = await f(*args, **kwargs)
           File "/home/def/dev/dstack/src/dstack/_internal/server/background/pipeline_tasks/service_router_worker_sync.py", line
         269, in process
             await sync_router_workers_for_run_model(run_for_sync)
           File "/home/def/dev/dstack/src/dstack/_internal/server/services/runs/router_worker_sync.py", line 643, in
         sync_router_workers_for_run_model
             fmt(run_model),
           File "/home/def/dev/dstack/src/dstack/_internal/server/services/logging.py", line 26, in fmt
             return fmt_entity("run", model.id, model.run_name)
           File "/home/def/dev/dstack/.venv/lib/python3.10/site-packages/sqlalchemy/orm/attributes.py", line 569, in __get__
             return self.impl.get(state, dict_)  # type: ignore
           File "/home/def/dev/dstack/.venv/lib/python3.10/site-packages/sqlalchemy/orm/attributes.py", line 1096, in get
             value = self._fire_loader_callables(state, key, passive)
           File "/home/def/dev/dstack/.venv/lib/python3.10/site-packages/sqlalchemy/orm/attributes.py", line 1129, in
         _fire_loader_callables
             return callable_(state, passive)
           File "/home/def/dev/dstack/.venv/lib/python3.10/site-packages/sqlalchemy/orm/strategies.py", line 585, in __call__
             return strategy._load_for_state(state, passive)
           File "/home/def/dev/dstack/.venv/lib/python3.10/site-packages/sqlalchemy/orm/strategies.py", line 543, in
         _load_for_state
             raise orm_exc.DetachedInstanceError(
         sqlalchemy.orm.exc.DetachedInstanceError: Parent instance <RunModel at 0x78af662b9e40> is not bound to a Session; deferred
         load operation of attribute 'run_name' cannot proceed (Background on this error at: https://sqlalche.me/e/20/bhk3)
DEBUG    dstack._internal.server.background.pipeline_tasks.base:364 Processed service_router_worker_sync item
         648bb027-2a5c-447e-80c7-a8e0b391bc37 in 0.022

----------------------------------

ERROR    dstack._internal.server.services.runs.router_worker_sync:421 Could not fetch server_info for worker http://10.0.0.3:8000:
         ReadError('')
         Traceback (most recent call last):
           File "/home/def/dev/dstack/.venv/lib/python3.10/site-packages/httpx/_transports/default.py", line 101, in
         map_httpcore_exceptions
             yield
           File "/home/def/dev/dstack/.venv/lib/python3.10/site-packages/httpx/_transports/default.py", line 394, in
         handle_async_request
             resp = await self._pool.handle_async_request(req)
           File "/home/def/dev/dstack/.venv/lib/python3.10/site-packages/httpcore/_async/connection_pool.py", line 256, in
         handle_async_request
             raise exc from None
           File "/home/def/dev/dstack/.venv/lib/python3.10/site-packages/httpcore/_async/connection_pool.py", line 236, in
         handle_async_request
             response = await connection.handle_async_request(
           File "/home/def/dev/dstack/.venv/lib/python3.10/site-packages/httpcore/_async/connection.py", line 103, in
         handle_async_request
             return await self._connection.handle_async_request(request)
           File "/home/def/dev/dstack/.venv/lib/python3.10/site-packages/httpcore/_async/http11.py", line 136, in
         handle_async_request
             raise exc
           File "/home/def/dev/dstack/.venv/lib/python3.10/site-packages/httpcore/_async/http11.py", line 106, in
         handle_async_request
             ) = await self._receive_response_headers(**kwargs)
           File "/home/def/dev/dstack/.venv/lib/python3.10/site-packages/httpcore/_async/http11.py", line 177, in
         _receive_response_headers
             event = await self._receive_event(timeout=timeout)
           File "/home/def/dev/dstack/.venv/lib/python3.10/site-packages/httpcore/_async/http11.py", line 217, in _receive_event
             data = await self._network_stream.read(
           File "/home/def/dev/dstack/.venv/lib/python3.10/site-packages/httpcore/_backends/anyio.py", line 32, in read
             with map_exceptions(exc_map):
           File "/home/def/.local/share/uv/python/cpython-3.10-linux-x86_64-gnu/lib/python3.10/contextlib.py", line 153, in
         __exit__
             self.gen.throw(typ, value, traceback)
           File "/home/def/dev/dstack/.venv/lib/python3.10/site-packages/httpcore/_exceptions.py", line 14, in map_exceptions
             raise to_exc(exc) from exc
         httpcore.ReadError

Additional information

No response

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

bugSomething isn't working

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions