This Git Organization has been established to develop distributed inference and model optimization technologies for large-scale AI models in heterogeneous AI semiconductor environments. Our goal is to maximize the performance and power efficiency of AI services by effectively utilizing various types of AI semiconductors (GPUs, NPUs, etc.).
- Performance Optimization: Optimize inference speed and throughput of AI models in heterogeneous AI semiconductor environments.
- Power Efficiency Enhancement: Minimize the energy consumption of AI services to improve performance per watt.
- Hardware Utilization Maximization: Develop technologies that comprehensively leverage the strengths of AI semiconductors with different characteristics.
- Open Source Ecosystem Contribution: Contribute to the advancement of the field by open-sourcing the developed technologies.
This project focuses on the development of the following core technologies:
- Distributed Inference Framework for Heterogeneous AI Semiconductors:
- Efficient task distribution and synchronization technologies among diverse AI semiconductors.
- Optimized data communication and batching strategies.
- Model Optimization Technologies:
- Model lightweighting techniques such as Quantization and Pruning, optimized for heterogeneous semiconductor environments.
- Model compilation and deployment pipelines tailored to specific semiconductor characteristics.
- Performance & Power Consumption Measurement/Evaluation Tools:
- Tools for quantitatively measuring and validating the performance (inference speed, throughput) and power consumption/efficiency of the developed technologies.
Existing AI services often face limitations in scalability and power efficiency due to their reliance on specific hardware (primarily GPUs). This project aims to overcome these challenges by leveraging heterogeneous semiconductors, presenting clear advantages in terms of energy consumption and performance metrics during inference compared to existing methods.
This project welcomes collaboration with various institutions and experts.
- We plan to proceed with service verification utilizing specific verification devices (e.g., NPU-equipped devices) through collaboration with SK Hynix, and a related letter of intent for cooperation has been secured.
- We welcome participation from researchers, developers, and companies interested in contributing to technology development and verification.
For inquiries or collaboration proposals, please contact jsseok@etri.re.kr.
© 2025 HADIS. All rights reserved.