vision-follow-car

中文 | English

This project serves as a hands-on learning project for AI, covering the full pipeline from data collection and model training to deployment on embedded hardware.

A monocular vision-based auto-follow toy car system. End-to-end pipeline from data collection and model training to onboard inference and MCU control.

This project implements a complete vision-follow vehicle system using a single camera for perception, a Jetson Orin Nano for real-time inference, and a DlaBoard_B (STM32F407VET6) control board for low-level control. The system consists of three subprojects that run on different hardware:

• Training — Server-side data processing and model training
• Inference — Jetson Orin Nano for camera capture and neural network inference
• Control — STM32 firmware for steering, throttle, brake, and serial communication

                    +------------------+
                    |   CSI Camera     |
                    |   (IMX219)       |
                    +--------+---------+
                             |
                             v
+------------------+   +-----+------+   +------------------+
| vision-follow-   |   |  Jetson    |   | vision-follow-   |
| car-training     |   | Orin Nano  |   | car-mcu          |
| (Server)         |   |            |   | (STM32)          |
|                  |   | Inference  |   |                  |
| Data / Train /   |-->| & Control  |-->| Steering /       |
| Export           |   | Output     |   | Throttle / Brake |
+------------------+   +------------+   +------------------+

Data flow: Images → Jetson (inference) → Control commands → MCU (execution).

• Compute: Jetson Orin Nano
• MCU board: DlaBoard_B (STM32F407VET6)
• Camera: CSI camera (IMX219)
• Actuators: Steering mechanism, brake actuator

See docs/hardware.md for details and the DlaBoard_B specification for the control board.

Buy the DlaBoard_B control board: China (Taobao) https://item.taobao.com/item.htm?id=1063856776213 · International https://shop.daliangauto.com/

1. Data collection — Human-driven driving, images + control signals recorded
2. Data cleaning — Invalid samples removed, paired image–control data prepared
3. Model training — Vision model trained on server
4. Model export — Model exported for inference
5. Jetson deployment — Model loaded on Jetson for real-time inference
6. MCU execution — Jetson sends control commands to STM32 for actuation

See docs/pipeline.md for the full workflow.

Demo video and screenshots will be added here.

MIT License