This project transforms complex water quality datasets from Oklahoma's Blue Thumb volunteer monitoring program into accessible, interactive visualizations that help communicate stream health across Oklahoma's watersheds. With data from 370+ monitoring sites, the dashboard provides comprehensive statewide coverage enhanced by AI-powered assistance and automated cloud processing.
- Python 3.12+ - Data processing and analysis
- Dash & Plotly - Interactive web dashboard and visualizations
- SQLite - Normalized database schema for comprehensive monitoring data
- Pandas - Data manipulation and analysis
- Bootstrap - Responsive UI components
- Vertex AI (Gemini 2.0) - AI-powered stream health chatbot with document grounding
- Cloud Functions - Serverless data processing and synchronization
- Cloud Storage - Database hosting with automated backups
- Cloud Scheduler - Automated daily data updates
- ArcGIS API - Survey123 integration for real-time data collection
- Real-time visualization of all 370+ monitoring sites across Oklahoma
- Parameter-based color coding for immediate status assessment
- Active site filtering to focus on currently monitored locations
- Click-to-navigate functionality for detailed site analysis
- Expert Knowledge: Trained on Blue Thumb documentation and stream health science
- Context-Aware: Provides tab-specific guidance and answers
- Multi-Source: Combines grounded knowledge with real-time search capabilities
- Interactive Chat: Available on every tab with persistent conversation history
- Automated Daily Sync: Processes new Survey123 submissions at 6 AM Central
- Smart Data Processing: Handles range-based measurements and validation
- Backup Management: Automatic database backups before each update
- Error Handling: Comprehensive logging and monitoring
- Cost-Efficient: <$1/year operational costs
- Time series visualization of key water quality parameters
- Reference threshold highlighting (normal, caution, poor conditions)
- Multi-site comparison capabilities
- Seasonal filtering and trend analysis
- Parameter-specific educational explanations
- Fish community integrity scoring over time
- Macroinvertebrate bioassessment results statewide
- Species diversity metrics and trends
- Detailed biological metrics for scientific review
- Interactive species galleries with identification guides
- Physical stream condition scoring across Oklahoma watersheds
- Habitat quality trends over monitoring periods
- Component-level habitat metrics breakdown
- Watershed-scale habitat comparisons
├── app.py
├── requirements.txt
├── cloud_functions/
│ └── survey123_sync/ # Automated data synchronization
│ ├── main.py
│ ├── chemical_processor.py
│ ├── deploy.sh
│ └── requirements.txt
├── database/ # Database schema and connection utilities
│ ├── db_schema.py
│ └── reset_database.py
├── data_processing/ # Comprehensive data cleaning and processing pipeline
│ ├── data_loader.py
│ ├── site_processing.py
│ ├── chemical_processing.py
│ ├── fish_processing.py
│ ├── macro_processing.py
│ └── habitat_processing.py
├── callbacks/ # Interactive dashboard logic
│ ├── chatbot_callbacks.py
│ ├── chemical_callbacks.py
│ ├── biological_callbacks.py
│ └── habitat_callbacks.py
├── layouts/
│ ├── components/
│ │ └── chatbot.py
│ └── tabs/
├── visualizations/
├── data/
│ ├── raw/ # Original CSV data files
│ ├── interim/ # Cleaned and validated data
│ └── processed/ # Database-ready outputs
│ └── chatbot_data/ # AI knowledge base content
├── text/
└── assets/
- Python 3.12+
- Git
- Google Cloud SDK (for cloud features)
-
Clone the repository
git clone https://github.com/yourusername/Blue-Thumb-Dashboard.git cd Blue-Thumb-Dashboard -
Create virtual environment
python -m venv .venv source .venv/bin/activate # On Windows: .venv\Scripts\activate
-
Install dependencies
pip install -r requirements.txt
-
Load the monitoring data
python -m database.reset_database
-
Start the dashboard
python app.py
-
Open your browser Navigate to http://127.0.0.1:8050
- ETL Architecture: Comprehensive processes for multiple data types across 370+ sites
- Real-time Integration: Automated Survey123 form processing with ArcGIS API
- Data Validation: duplicate detection and quality assurance
- Scalable Design: Cloud-native architecture for production deployment
- Contextual Assistance: Tab-aware chatbot providing relevant stream health guidance
- Knowledge Grounding: Responses based on authoritative Blue Thumb documentation
- Intelligent Fallback: Google Search integration for comprehensive coverage
- Natural Interaction: Conversational interface with typing indicators and message history
- Comprehensive Test Suite: 700+ tests ensuring reliability across all components
- Automated CI/CD: Continuous integration with quality checks
- Data Validation: Multi-layer validation ensuring data integrity
- Performance Monitoring: Real-time tracking of system performance
- 370+ Monitoring Sites: Comprehensive statewide water quality coverage
- Multi-Parameter Analysis: Chemical, biological, and habitat assessment integration
- Educational Outreach: Public-facing dashboard promoting stream health awareness
- Automated Processing: Daily data updates reducing manual intervention by 100%
- AI Enhancement: Intelligent assistance improving user engagement and understanding
- Real-time Webhooks: Immediate processing of new submissions
- Advanced Analytics: Machine learning for trend prediction and anomaly detection
- Mobile Optimization: Progressive web app capabilities
- Multi-State Expansion: Framework for other volunteer monitoring programs
- Weather Integration: Precipitation correlation analysis
- API Development: Public API for researchers and third-party applications
This dashboard uses data from the Blue Thumb Volunteer Stream Monitoring Program, administered by the Oklahoma Conservation Commission. Blue Thumb trains citizen volunteers to collect standardized water quality data, creating one of the most comprehensive stream monitoring datasets in Oklahoma with over 370 active and historical monitoring sites.
- Blue Thumb Program - Oklahoma Conservation Commission
- Volunteer Monitors - Citizens collecting water quality data across 370+ sites statewide
Built with ❤️ for Oklahoma's streams and the volunteers who protect them