✨ feat: Add RHDH Store Manager agent with RAG knowledge base#16
Draft
durandom wants to merge 2 commits into
Draft
✨ feat: Add RHDH Store Manager agent with RAG knowledge base#16durandom wants to merge 2 commits into
durandom wants to merge 2 commits into
Conversation
Introduces a comprehensive agent for Red Hat Developer Hub's plugin ecosystem management with extensive knowledge base integration and Jira toolkit support. Features: - Store Manager agent with 8 core use cases (discovery, migration, certification, lifecycle, support, releases, metadata, coordination) - RAG knowledge base with 24 files (9 MD, 1 PDF, 14 CSV) covering plugin documentation, catalogs, and release schedules - Jira toolkit integration for real-time issue tracking (RHIDP, RHDHPLAN projects) - Comprehensive test suite with 10 validation scenarios covering knowledge retrieval, CSV parsing, and Jira integration - Detailed documentation with 58+ test questions and testing guide Technical implementation: - AgentConfigurator pattern with JiraToolkitConfig for token management - Knowledge base auto-loading via _get_knowledge_config() - Hybrid search (vector + keyword) with LanceDB persistence - Plugin system registration via pyproject.toml entry point Documentation: - docs/agents/store-manager.md - Agent overview and capabilities - docs/agents/store-manager-test-questions.md - 58+ test questions across 8 use cases - docs/agents/store-manager-comprehensive-test.md - Test execution guide - docs/testing-guide.md - General testing best practices Configuration: - STORE_MANAGER_JIRA_API_TOKEN environment variable for Jira access - knowledge/ directory added to .gitignore for local knowledge bases
Fixes missing knowledge bases in deployed containers by explicitly copying them during the container build process. Changes: - Containerfile: Add COPY commands for examples/ and knowledge/ directories - .containerignore: Add explicit includes for knowledge directories (!examples/knowledge/, !knowledge/) - .gitignore: Narrow exclusion from knowledge/ to knowledge/store-manager/ (allows demo agent knowledge to be tracked) - Reorganize: Move demo agent knowledge from examples/knowledge/ to knowledge/demo-agent/ Technical details: - Knowledge directories were git-ignored but needed in container runtime - Total size impact: ~5.3 MB (demo: 72 KB, store-manager: 5.2 MB) - LanceDB vector indexes are generated at runtime in /app/tmp/lancedb/ - Source documents (MD, PDF, CSV) are now bundled in the container image This allows agents to access their knowledge bases: - Demo agent: knowledge/demo-agent/ (3 MD files) - Store Manager agent: knowledge/store-manager/ (24 files: 9 MD, 1 PDF, 14 CSV)
Contributor
There was a problem hiding this comment.
Pull request overview
This PR adds a comprehensive RHDH Store Manager agent to help Red Hat Developer Hub users navigate the plugin ecosystem using Retrieval-Augmented Generation (RAG). The agent provides guidance across 8 key use cases including plugin discovery, migration, certification, lifecycle management, support boundaries, release planning, metadata management, and team coordination.
Key Changes:
- New Store Manager agent with RAG knowledge base and Jira integration
- Enhanced knowledge management system with improved async event loop handling for LiteLLM proxy compatibility
- Comprehensive test suite with 477 lines covering unit and integration testing
- Complete documentation including agent guide, test scenarios, and testing guide
Reviewed changes
Copilot reviewed 12 out of 14 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
uv.lock |
Added aiofiles and duckdb dependencies; removed beautifulsoup4 and requests ( |
pyproject.toml |
Added new dependencies and registered store-manager agent entry point |
src/agentllm/agents/store_manager_agent.py |
New agent wrapper following established three-class pattern |
src/agentllm/agents/store_manager_agent_configurator.py |
Comprehensive configurator with RAG, CSV tools, and optional Jira integration |
src/agentllm/knowledge/manager.py |
Enhanced async document loading with thread pool support for LiteLLM proxy |
tests/test_store_manager_agent.py |
Extensive test suite with 13 unit tests and 10 parametrized integration scenarios |
proxy_config.yaml |
Added Store Manager model configuration with mode metadata |
.env.secrets.template |
Added STORE_MANAGER_JIRA_API_TOKEN configuration with setup instructions |
docs/agents/store-manager.md |
Comprehensive agent documentation covering all capabilities and troubleshooting |
docs/agents/store-manager-test-questions.md |
58+ test questions organized by use case |
docs/agents/store-manager-comprehensive-test.md |
Detailed guide for running and interpreting comprehensive test scenarios |
docs/testing-guide.md |
Quick reference for running parametrized test scenarios |
src/agentllm/agents/demo_agent_configurator.py |
Minor formatting improvements (multi-line function calls) |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| toolkits = [] | ||
|
|
||
| # Add CSV toolkit for plugin metadata | ||
| csv_path = Path("knowledge/store-manager/store-manager-docs/marketplace plugin metadata_sheets/working_sheet.csv") |
There was a problem hiding this comment.
[nitpick] The CSV path is hardcoded to a very specific location: knowledge/store-manager/store-manager-docs/marketplace plugin metadata_sheets/working_sheet.csv. This path contains spaces ("marketplace plugin metadata_sheets") and is deeply nested.
Consider:
- Using a configuration variable or constant at the module level to make this path more discoverable and easier to update
- Documenting this path requirement in the agent's setup documentation
- Adding a more informative error message if the file is not found, guiding users on where to place it
This will improve maintainability and make it easier for users to set up the agent correctly.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds a new RHDH Store Manager agent that helps Red Hat Developer Hub users navigate the extensive plugin ecosystem using Retrieval-Augmented Generation (RAG).
Key additions:
Changes:
store_manager_agent.py,store_manager_agent_configurator.py): Full plugin system integration with RAG knowledge baseknowledge/manager.py): Enhanced RAG system with better caching and persistenceproxy_config.yamlandpyproject.tomlfor new agentTest Plan
nox -s proxyandnox -s dev_local_proxy)docs/agents/store-manager-comprehensive-test.mdRelated
Part of RHDH agent ecosystem. Follows plugin architecture patterns from existing agents (GitHub PR Prioritization, Sprint Reviewer).