Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Bug: Deserialization of list[Model[T]] not working in v5 #1375

Closed
chrisgoddard opened this issue Apr 16, 2024 · 3 comments · Fixed by #1377
Closed

Bug: Deserialization of list[Model[T]] not working in v5 #1375

chrisgoddard opened this issue Apr 16, 2024 · 3 comments · Fixed by #1377
Labels
bug Something isn't working

Comments

@chrisgoddard
Copy link

Describe the bug
I just upgraded to v5 and all of my subscribers that consume from a redis list broke. I use a Pydantic model with a generic, kind of like this:

T = typing.TypeVar('T', bound=BaseModel | dict)
class Event(BaseModel, typing.Generic[T]):
    uid: str
    data: T

I now for a consumer like this:

@broker.subscriber(
    list=ListSub(
        QUEUE_OBJECTS_WRITE,
        batch=True,
        max_records=10000,
        polling_interval=30
    )
)
async def write_object_stream(
    events: list[Event[ObjectRecord]],
    ...

I get an error like this:

events.0
  Input should be a valid dictionary or instance of Event[ObjectRecord] [type=model_type, input_value='{"uid":"01HVMGPZC8S6RVP4...e.\\"}"},"metadata":{}}', input_type=str]
    For further information visit https://errors.pydantic.dev/2.7/v/model_type
events.1
  Input should be a valid dictionary or instance of Event[ObjectRecord] [type=model_type, input_value='{"uid":"01HVMGPZC812XW2C...false}"},"metadata":{}}', input_type=str]
    For further information visit https://errors.pydantic.dev/2.7/v/model_type
events.2
  Input should be a valid dictionary or instance of Event[ObjectRecord] [type=model_type, input_value='{"uid":"01HVMGPZCGBZ5EPW...l.\\"}"},"metadata":{}}', input_type=str]
    For further information visit https://errors.pydantic.dev/2.7/v/model_type
events.3

The messages are coming through as strings so it's not correctly parsing the JSON before passing them to the Pydantic model.

Has anything changed in the way batch consumers are configured?

I've had to roll back to 0.4.7 but I'd quite like to leverage the StopApplication exception so I'd love to resolve this.

Thanks!
@chrisgoddard chrisgoddard added the bug Something isn't working label Apr 16, 2024
Lancetnik added a commit that referenced this issue Apr 17, 2024
@Lancetnik
fix (#1375): correct Redis Batch message serialization
157e2a5
@Lancetnik Lancetnik mentioned this issue Apr 17, 2024
Merged
13 tasks
@Lancetnik
Copy link
Member

Sorry, our tests were missed this behavior for some reason. Now it is fixed and tests are added. Wait for patch a bit.

github-merge-queue bot pushed a commit that referenced this issue Apr 17, 2024
@Lancetnik
Fix/1375 (#1377)
a8d3142 
* chore: update dependencies

* fix (#1375): correct Redis Batch message serialization

* fix (#1376): fix Redis connection options priority

* tests: fix Redis connection tests (correct ping usage
@chrisgoddard
Copy link
Author

All good! Thanks for the quick turnaround!

@chrisgoddard
Copy link
Author

Confirm it's working as expected.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Something isn't working
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Fix/1375 airtai/faststream
2 participants
@chrisgoddard @Lancetnik