A practical guide for real-world language-based AI applications for resource-constrained devices with industry standards in mind.
- Start with the Edge AI Engineering: a practical guide covering core concepts of the entire Edge AI MLOps stack with industry blueprints.
- Then read this: The Next AI Frontier is at the Edge
- Related work: Edge Audio | Edge Vision
The goal of this guide is to provide resources for building, optimizing, and deploying language-based AI applications at the edge, through hands-on examples including practical notebooks and real-world use cases across key industries.
Industry Blueprints
- Autonomous Systems
- Healthcare & Medical Imaging*
- Retail & Consumer Analytics
- Security & Surveillance
- Agriculture & Precision Farming
- Manufacturing & Quality Control
- Smart Cities & Urban Planning
Edge Optimization Lab: techniques and tools for maximizing performance and efficiency of language models on edge hardware
- Model Quantization
- Pruning Techniques
- Federated Learning
- Compiler Targets (ONNX, TVM)
- Hardware-Specific Optimizations (Jetson, Raspberry Pi, Microcontrollers)
Production Pipelines: guides and templates for robust, scalable edge language AI operations
- CI/CD for Language Models on the Edge
- Monitoring (Drift Detection, Edge Metrics Dashboard)
- OTA Updates
- Edge Security (Secure Boot, Data Encryption, Threat Detection, Privacy-Preserving language, Adversarial Robustness, Device Hardening, Compliance)
Reference Architectures: blueprints for edge language hardware and system design
- Microphone Array Setups
- Edge Server Specs
- IoT Connectivity
- Edge-Cloud Hybrid Models
Integration
- Notebooks (hands-on deep dives)
- Companion Resources
- Industry-Specific Stardards
├── edge-ai-engineering/
│ ├── introduction-to-edge-ai.md
│ ├── edge-ai-architectures.md
│ ├── model-optimization-techniques.md
│ ├── hardware-acceleration.md
│ ├── edge-deployment-strategies.md
│ ├── real-time-processing.md
│ ├── privacy-and-security.md
│ ├── edge-ai-frameworks.md
│ └── benchmarking-and-performance.md
├── industry-blueprints/
│ ├── autonomous-systems/
│ │ ├── voice-command-recognition-tflite.md
│ │ ├── natural-language-vehicle-control.md
│ │ └── edge-language-agents-drone.md
│ ├── healthcare/
│ │ ├── clinical-text-analysis-edge.md
│ │ ├── patient-conversation-models.md
│ │ └── medical-language-agents.md
│ ├── retail-consumer-analytics/
│ │ ├── sentiment-analysis-edge.md
│ │ ├── chatbot-instore-assistance.md
│ │ └── personalized-recommendation-edge.md
│ ├── security-surveillance/
│ │ ├── voice-biometric-authentication.md
│ │ ├── call-center-language-intent-detection.md
│ │ └── edge-threat-intelligence-llm.md
│ ├── agriculture-precision-farming/
│ │ ├── voice-command-machinery-control.md
│ │ ├── farmer-assistant-chatbots.md
│ │ └── natural-language-report-generation.md
│ ├── manufacturing/
│ │ ├── voice-operated-maintenance.md
│ │ ├── defect-report-llm.md
│ │ └── predictive-language-agents.md
│ └── smart-cities/
│ ├── multilingual-public-announcements.md
│ ├── emergency-alert-llm.md
│ └── citizen-feedback-analysis.md
├── edge-optimization-lab/
│ ├── model-quantization/
│ │ ├── int8-quantization-for-llm.md
│ │ └── post-training-quantization-llm.md
│ ├── pruning-techniques/
│ │ ├── structured-pruning-llms.md
│ │ └── adaptive-pruning-edge.md
│ ├── federated-learning/
│ │ ├── privacy-preserving-llm.md
│ │ └── distributed-fine-tuning.md
│ ├── compiler-targets/
│ │ ├── onnx-runtime-for-llms.md
│ │ └── tvm-compiler-usage.md
│ └── hardware-specific-optimization/
│ ├── nvidia-jetson-llm-optimization.md
│ ├── intel-openvino-llm.md
│ ├── raspberry-pi-llm.md
│ └── microcontroller-tinyml-language.md
├── production-pipelines/
│ ├── ci-cd-for-edge.md
│ ├── monitoring/
│ │ ├── drift-detection-llm.md
│ │ └── edge-llm-metrics-dashboard.md
│ ├── ota-updates.md
│ └── edge-security/
│ ├── secure-boot-implementation.md
│ ├── data-encryption-edge.md
│ ├── threat-detection/
│ │ ├── anomaly-detection-text.md
│ │ └── adversarial-attack-defense.md
│ ├── privacy-preserving-llm/
│ │ ├── federated-learning-techniques.md
│ │ └── differential-privacy.md
│ ├── model-security/
│ │ └── adversarial-robustness.md
│ ├── edge-device-hardening/
│ │ ├── secure-deployment.md
│ │ └── secure-communication.md
│ └── industry-compliance/
│ ├── regulatory-standards.md
│ └── ethical-ai-guidelines.md
├── reference-architectures/
│ ├── language-model-hardware.md
│ ├── edge-server-specs.md
│ ├── iot-connectivity.md
│ └── edge-cloud-hybrid-models.md
└── integration/
├── cs-notebook-redirects.md
├── companion-resources.md
└── industry-specific-regulations.md
Important
This project uses a submodule edge-ai-engineering located in lab/edge-ai-engineering.
Please initialize submodules after cloning the repository: git submodule update --init --recursive
- Clone this repository:
git clone https://github.com/afondiel/edge-language.git- Explore the Edge AI Engineering for foundational knowledge.
- Dive into Industry Blueprints for hands-on, sector-specific language AI guides.
- Use the Edge Optimization Lab and Production Pipeline for deployment and scaling.
See CONTRIBUTING.md for guidelines on how to contribute, report issues, or suggest new blueprints.
Distributed under the MIT License. See LICENSE for more information.
Technical Guides:
Curated Lists:
- awesomesmol - An awesome list of "small but mighty" models and resources.
- Edge-AI Model Zoo - A list of production-ready models for resource-constrained devices.
Books: