Feature/generation result #92
Conversation
There was a problem hiding this comment.
Looks great; thank you for working on this @LeonNissen ! 🚀
Happy to see this merged once all CI elements are passing & the comments are addressed.
|
|
||
| async let generate: AsyncThrowingStream<LLMLocalGenerateState, Error> = generate() | ||
|
|
||
| for try await state in try await generate { |
There was a problem hiding this comment.
Wondering if we could use a map/compactmap or something similar here instead of a continuation and for loop. Also applies to the code below.
|
Closing this PR as it was mostly used to gather performance data for local inference, not ready to merge as is. |
|
I think some of the additions could be nice if we can easily enable apps to get all the elements as a client application (optional, e.g. with an overload or a different method). Maybe we can find an other way to do this if you think this wasn't the best way to do this @philippzagar? |
Name of the PR
♻️ Current situation & Problem
Link any open issues or pull requests (PRs) related to this PR. Please ensure that all non-trivial PRs are first tracked and discussed in an existing GitHub issue or discussion.
⚙️ Release Notes
Add a bullet point list summary of the feature and possible migration guides if this is a breaking change so this section can be added to the release notes.
Include code snippets that provide examples of the feature implemented or links to the documentation if it appends or changes the public interface.
📚 Documentation
Please ensure that you properly document any additions in conformance to Spezi Documentation Guide.
You can use this section to describe your solution, but we encourage contributors to document your reasoning and changes using in-line documentation.
✅ Testing
Please ensure that the PR meets the testing requirements set by CodeCov and that new functionality is appropriately tested.
This section describes important information about the tests and why some elements might not be testable.
📝 Code of Conduct & Contributing Guidelines
By submitting creating this pull request, you agree to follow our Code of Conduct and Contributing Guidelines: