Skip to content

fulvian/devstream

Folders and files

NameName
Last commit message
Last commit date

Latest commit

Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 

DevStream: Integrated Task Management & Cross-Session Memory System

Version License Python 3.11+ Poetry Code style: black Linting: ruff Type checking: mypy CI Status

A revolutionary framework that transforms user-AI interaction by enforcing a structured workflow based on granular tasks with persistent semantic memory for Claude Code.

🎯 Key Features

Task-Forced Workflow

  • Every user interaction must flow through task creation
  • Granular micro-tasks: max 10 minutes, 256K tokens
  • Structured plans: Goals β†’ Phases β†’ Micro-tasks
  • Automatic validation to maintain granularity

Cross-Session Semantic Memory

  • Vector embeddings with Ollama embeddinggemma
  • Hybrid search semantic + keyword with weighted scoring
  • Automatic context injection from relevant memory
  • Incremental learning from every completed task

Virtual Agent Team

  • Architect: System design and technology selection
  • Coder: Implementation and debugging
  • Reviewer: Quality assurance and testing
  • Documenter: Technical writing and API docs

Advanced Hook System

  • Event-driven: Hooks on user interaction, task completion, context limits
  • Auto-enforcement: Enforces task-based workflow
  • Memory automation: Automatic save and retrieval

πŸ›  Technology Stack

  • Python 3.11+ with complete type hints
  • SQLite + FTS5 for storage and full-text search
  • Ollama + embeddinggemma for local semantic embeddings
  • Pydantic v2 for data validation and configuration
  • Poetry for dependency management
  • AsyncIO for non-blocking operations

πŸ“š Development Methodology

DevStream uses a research-driven methodology with 4 fundamental phases:

  1. 🎯 Discussion & Analysis - Align on objectives and technical specifications
  2. πŸ“‹ Structured Division - Breakdown into phases and granular micro-tasks
  3. πŸ” Context7 Research - Best practices and technical documentation
  4. βœ… Verification & Testing - Rigorous testing and complete validation

Quick Reference

# Complete workflow
Discuss β†’ Divide β†’ Research β†’ Implement β†’ Test β†’ Validate

# Main tools
- Context7: Research and best practices
- TodoWrite: Granular task management
- Testing: 95%+ coverage target

Complete details: See CLAUDE.md and CLAUDE.quick-reference.md

πŸš€ Quick Start

Prerequisites

# Python 3.11+
python --version

# Poetry for dependency management
curl -sSL https://install.python-poetry.org | python3 -

# Ollama for embedding generation
curl -fsSL https://ollama.ai/install.sh | sh
ollama serve
ollama pull nomic-embed-text

Installation

βœ… PRODUCTION-READY: Zero Manual Configuration Required

# Clone repository
git clone <repository-url>
cd devstream

# Run automated installation (configures EVERYTHING)
./install.sh

# CRITICAL: Restart Claude Code to load hook configuration
# (hooks will NOT work until restart)

# Verify installation
./scripts/verify-install.py
make check-env

What the Installation Script Does

The install.sh script provides zero-configuration setup:

βœ… Creates Python 3.11+ virtual environment (.devstream/) βœ… Installs all dependencies (cchooks, aiohttp, structlog, etc.) βœ… Builds MCP server (if Node.js installed) βœ… Configures hook files (chmod +x for all hooks) βœ… AUTO-CONFIGURES ~/.claude/settings.json ← CRITICAL βœ… Creates database directory (data/) βœ… Verifies complete setup

No manual configuration needed!

Context7 MCP Setup (Optional but Recommended)

Context7 provides up-to-date library documentation for research-driven development (used in DevStream RESEARCH phase).

Steps:

  1. Get API Key: Visit https://context7.com and sign up for a free account

  2. Configure: Add your API key to .env.devstream:

    echo "CONTEXT7_API_KEY=your_api_key_here" >> .env.devstream
  3. Verify: Restart Claude Code and test:

    # Context7 will be automatically configured by post-install.py
    # Test with: mcp__context7__resolve-library-id tool in Claude Code

Note: Context7 integration is optional but highly recommended for DevStream's research-driven workflow. It enables automatic documentation retrieval for libraries during the RESEARCH phase of the 7-step workflow.

Troubleshooting

If hooks don't work after installation:

# 1. Verify configuration
./scripts/validate-config.py

# 2. Manually re-configure hooks (if needed)
./scripts/post-install.py

# 3. Check hook logs
tail -50 ~/.claude/logs/devstream/pre_tool_use.log
tail -50 ~/.claude/logs/devstream/post_tool_use.log

First Use

# Open project in Claude Code
# Hooks activate automatically after restart

# Verify hooks are working (create test file, check logs)
tail -f ~/.claude/logs/devstream/post_tool_use.log

# Search semantic memory
# (Use MCP tools in Claude Code)

πŸ“‹ Development Workflow

Development Environment Setup

# Complete development setup
make dev

# Install pre-commit hooks
make install-hooks

# Check environment
make check-env

Development Commands

# Code quality
make format           # Format with black + isort
make lint            # Run ruff + mypy + bandit
make check           # Format + lint

# Testing
make test            # All tests
make test-unit       # Unit tests only
make test-integration # Integration tests
make test-coverage   # Coverage report

# Database
make db-init         # Initialize database
make db-migrate      # Run migrations
make db-reset        # Reset database (⚠️ destroys data)

# Documentation
make docs            # Serve docs locally
make docs-build      # Build static docs

Testing Strategy

# Unit tests (fast, mocked)
pytest tests/unit -v

# Integration tests (requires Ollama)
pytest tests/integration -m "requires_ollama" -v

# End-to-end tests (requires Docker)
pytest tests/e2e -m "requires_docker" -v

# Performance benchmarks
pytest tests/ --benchmark-only

πŸ“– Architecture

Project Structure

devstream/
β”œβ”€β”€ src/devstream/           # Main package
β”‚   β”œβ”€β”€ core/               # Core abstractions
β”‚   β”œβ”€β”€ database/           # Database layer
β”‚   β”œβ”€β”€ memory/             # Memory system
β”‚   β”œβ”€β”€ tasks/              # Task management
β”‚   β”œβ”€β”€ hooks/              # Hook system
β”‚   └── cli/                # CLI interface
β”œβ”€β”€ tests/                  # Test suite
β”œβ”€β”€ config/                 # Configuration templates
β”œβ”€β”€ docs/                   # Documentation
└── scripts/                # Utility scripts

Main Components

  1. Database Layer (devstream.database)

    • SQLite with FTS5 and optional vector search
    • Async connection management
    • Schema migrations
  2. Memory System (devstream.memory)

    • Ollama integration for embeddings
    • Hybrid search (semantic + keyword)
    • Context assembly and injection
  3. Task Management (devstream.tasks)

    • InterventionPlan β†’ Phase β†’ MicroTask hierarchy
    • Agent dispatcher with capability matching
    • Task state tracking
  4. Hook System (devstream.hooks)

    • Event-driven automation
    • Claude Code integration
    • Configurable hook chains

βš™οΈ Configuration

Environment Variables

# Database
DEVSTREAM_DB_PATH=./data/devstream.db
DEVSTREAM_DB_ENABLE_VECTOR_SEARCH=false

# Ollama
DEVSTREAM_OLLAMA_ENDPOINT=http://localhost:11434
DEVSTREAM_OLLAMA_EMBEDDING_MODEL=nomic-embed-text

# Memory
DEVSTREAM_MEMORY_SIMILARITY_THRESHOLD=0.7
DEVSTREAM_MEMORY_MAX_CONTEXT_TOKENS=2000

# Tasks
DEVSTREAM_TASK_MAX_DURATION_MINUTES=10
DEVSTREAM_TASK_FORCE_CREATION=true

Configuration Files

  • config/development.yml - Development settings
  • config/production.yml - Production settings
  • config/testing.yml - Test environment
# Load configuration
poetry run devstream --config config/development.yml

πŸ§ͺ Testing

Test Categories

  • Unit Tests (tests/unit/) - Fast, isolated, mocked
  • Integration Tests (tests/integration/) - Real Ollama + SQLite
  • End-to-End Tests (tests/e2e/) - Full workflow testing

Test Markers

# Run specific test categories
pytest -m "unit"                    # Unit tests only
pytest -m "integration"             # Integration tests
pytest -m "requires_ollama"         # Tests needing Ollama
pytest -m "slow"                    # Long-running tests
pytest -m "not slow"                # Skip slow tests

Mocking Strategy

  • Ollama: Mock embedding generation for unit tests
  • Database: In-memory SQLite for fast tests
  • File System: Temporary directories
  • Time: Freezegun for time-dependent tests

πŸ“ˆ Performance

Benchmarks

# Run performance benchmarks
make test-performance

# Memory profiling
make memory-profile

# CPU profiling
make profile

Optimization Guidelines

  • Database: Use WAL mode, appropriate indexes
  • Memory: Batch embedding generation, context caching
  • Search: Hybrid search with relevance thresholds
  • Tasks: Parallel micro-task execution when possible

πŸ” Debugging

Debug Tools

# Show current configuration
make debug-config

# Check dependency tree
make debug-deps

# Environment information
make debug-env

# Database inspection
sqlite3 data/devstream.db ".schema"

Logging

Structured logging with flexible configuration:

import structlog

logger = structlog.get_logger()
logger.info("Task completed", task_id="123", duration_ms=1500)

🀝 Contributing

Code Quality Standards

  • Type Hints: All parameters and return types
  • Docstrings: Google-style docstrings
  • Testing: 90%+ coverage for new code
  • Linting: Automatic pre-commit hooks

Development Process

  1. Fork repository and create feature branch
  2. Implement changes with test coverage
  3. Run make check for code quality
  4. Submit pull request with detailed description

Commit Convention

Use Conventional Commits:

feat: add semantic search with Ollama integration
fix: resolve memory leak in embedding cache
docs: update configuration examples
test: add integration tests for hook system

πŸ“š Documentation

Generate Documentation

# Serve locally with hot reload
make docs

# Build static documentation
make docs-build

🚒 Production Deployment

Docker

# Build production image
docker build -t devstream:latest .

# Run with Docker Compose
docker-compose up -d

Environment Variables

# Production settings
DEVSTREAM_ENVIRONMENT=production
DEVSTREAM_DEBUG=false
DEVSTREAM_DB_PATH=/data/devstream.db
DEVSTREAM_OLLAMA_ENDPOINT=http://ollama:11434

Monitoring

  • Structured Logging: JSON logs for aggregation
  • Performance Metrics: Task completion times, memory usage
  • Health Checks: Database connectivity, Ollama availability
  • Alerts: Failed tasks, memory threshold exceeded

πŸ“„ License

MIT License - see LICENSE file for details.

πŸ™ Acknowledgments

  • Claude Code Team for the vision of task-forced workflow
  • Ollama Project for local embedding capabilities
  • SQLite Team for FTS5 and performance optimizations
  • Python Community for excellent typing and async support

Note: This system represents a paradigm shift in AI development tool interaction, transforming Claude Code from a reactive tool to a proactive project management system with permanent memory.

About

Claude Code proactive framework: task-forced workflow + semantic memory + AI agent team

Resources

Code of conduct

Contributing

Security policy

Stars

Watchers

Forks

Packages

No packages published

Contributors 2

  •  
  •