Skip to content

Sonny-Inkai/TNI_AI_ENGINEER

Folders and files

NameName
Last commit message
Last commit date

Latest commit

Β 

History

43 Commits
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 

Repository files navigation

πŸš€ SmartAlert - AI Video Management System (VMS)

SmartAlert is a mini-alert pipeline designed to ingest AI-detected events (face recognition, intrusion, etc.), store them efficiently, and notify operators in real-time via WebSockets.

πŸ› οΈ Tech Stack

  • Backend: FastAPI, SQLAlchemy 2.0 (Async), Alembic, Redis (Pub/Sub), PostgreSQL.
  • Frontend: Next.js 15 (App Router), TypeScript, TanStack Query v5.
  • Infrastructure: Docker Compose, MinIO (Object Storage), Qdrant (Vector DB).

I. How to run

🏁 Getting Started (Cold Start)

  1. Clone the repository and ensure Docker is running.
  2. Start all services:
docker compose up --build -d
  1. Verify Health:
  • Open API backend swagger UI:
http://localhost:8000/docs
  • Open Frontend:
http://localhost:3000

II. How to Test:

I have provided sample images in the assets/ directory. Use the following curl commands to simulate different AI events. These commands use Multipart Form-Data to send both the event metadata and a snapshot image. I have organized the testing process to match the project requirements. Please ensure all services are running via Docker before proceeding

Note: Ensure you are in the project root directory where the assets/ folder is located before running these commands.

πŸ”΄ Tier 1: Core Functionalities (Mandatory)

1.1 AI Event Ingestion (POST /events)

Simulate 5 different types of AI detections using curl. These commands include metadata and timestamps.

A. INTRUSION

curl -X 'POST' 'http://localhost:8000/api/v1/events/' \
  -F 'data={"camera_id": "CAM_01", "camera_name": "Main Gate", "event_type": "INTRUSION", "confidence": 0.98}' \
  -F 'image=@assets/1.jpeg'

B. FACE_RECOGNIZED

curl -X 'POST' 'http://localhost:8000/api/v1/events/' \
  -F 'data={"camera_id": "CAM_02", "camera_name": "Lobby", "event_type": "FACE_RECOGNIZED", "confidence": 0.92, "metadata": {"person_name": "John Doe"}}' \
  -F 'image=@assets/2.jpeg'

C. ANPR

curl -X 'POST' 'http://localhost:8000/api/v1/events/' \
  -F 'data={"camera_id": "CAM_03", "camera_name": "Parking", "event_type": "ANPR", "confidence": 0.85, "metadata": {"plate": "30A-12345"}}' \
  -F 'image=@assets/3.jpeg'

D. LOITERING

curl -X 'POST' 'http://localhost:8000/api/v1/events/' \
  -F 'data={"camera_id": "CAM_01", "camera_name": "Main Gate", "event_type": "LOITERING", "confidence": 0.75}' \
  -F 'image=@assets/4.jpeg'

E. CROWD

curl -X 'POST' 'http://localhost:8000/api/v1/events/' \
  -F 'data={"camera_id": "CAM_04", "camera_name": "Warehouse", "event_type": "CROWD", "confidence": 0.88}' \
  -F 'image=@assets/5.jpeg'

1.2 Querying & Filtering (GET /events)

Verify the listing, pagination, and filtering logic.

# Filter by camera and event type
curl -X 'GET' 'http://localhost:8000/api/v1/events/?camera_id=CAM_01&event_type=INTRUSION&page=1&page_size=10'

# Get 24h Statistics
curl -X 'GET' 'http://localhost:8000/api/v1/events/stats'

🟑 Tier 2: Quality & Depth

2.1 Image Snapshot Verification (MinIO)

Verify that images sent via curl are stored in MinIO.

  • Check the snapshot_url in the API response.
  • Access the MinIO Console at http://localhost:9000 to view files in the snapshots bucket.

2.3 Event Acknowledgement (PATCH)

Acknowledge an event to clear it from the pending list from frondent, click button and see how it set.


🟒 Tier 3: AI Integration

3.1 Face Similarity Search (Tier 3.1)

To test face re-identification, we ingest 5 events with specific 128-dimensional vectors. Three events share a "Vector 1" (all 1.0s) and two share a "Vector 0" (all 0.0s).

Open Qdrant to see to it save

http://localhost:6333/dashboard#/collections/face_events#points

A. Ingest 3 events with "Vector 1" (Simulating Person A)

# Event 1 - Person A
curl -X 'POST' 'http://localhost:8000/api/v1/events/' \
  -F 'data={"camera_id": "CAM_01", "camera_name": "Lobby", "event_type": "FACE_RECOGNIZED", "confidence": 0.99, "metadata": {"person_name": "Person A - Shot 1"}, "face_vector": [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0]}' \
  -F 'image=@assets/2.jpeg'

# Event 2 - Person A
curl -X 'POST' 'http://localhost:8000/api/v1/events/' \
  -F 'data={"camera_id": "CAM_02", "camera_name": "Lobby", "event_type": "FACE_RECOGNIZED", "confidence": 0.98, "metadata": {"person_name": "Person A - Shot 2"}, "face_vector": [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0]}' \
  -F 'image=@assets/2.jpeg'

# Event 3 - Person A
curl -X 'POST' 'http://localhost:8000/api/v1/events/' \
  -F 'data={"camera_id": "CAM_03", "camera_name": "Lobby", "event_type": "FACE_RECOGNIZED", "confidence": 0.97, "metadata": {"person_name": "Person A - Shot 3"}, "face_vector": [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0]}' \
  -F 'image=@assets/2.jpeg'

B. Ingest 2 events with "Vector 0" (Simulating Person B)

# Event 4 - Person B
curl -X 'POST' 'http://localhost:8000/api/v1/events/' \
  -F 'data={"camera_id": "CAM_04", "camera_name": "Lobby", "event_type": "FACE_RECOGNIZED", "confidence": 0.95, "metadata": {"person_name": "Person B - Shot 1"}, "face_vector": [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0]}' \
  -F 'image=@assets/2.jpeg'

# Event 5 - Person B
curl -X 'POST' 'http://localhost:8000/api/v1/events/' \
  -F 'data={"camera_id": "CAM_05", "camera_name": "Lobby", "event_type": "FACE_RECOGNIZED", "confidence": 0.94, "metadata": {"person_name": "Person B - Shot 2"}, "face_vector": [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0]}' \
  -F 'image=@assets/2.jpeg'

C. Perform Similarity Search Search for Person A using a query vector of all 1.0s.

curl -X 'POST' 'http://localhost:8000/api/v1/events/face-search' \
  -H 'Content-Type: application/json' \
  -d '{"query_vector": [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0], "top_k": 5}'

Expected Result: Person A's events should appear at the top with high similarity scores.

3.2 Anomaly Flagging (Burst Detection)

Trigger the "Burst Detection" logic (6+ events in 60s).

  1. Run the loop:
for i in {1..6}; do
  curl -X 'POST' 'http://localhost:8000/api/v1/events/' \
    -F 'data={"camera_id": "CAM_01", "camera_name": "Main Gate", "event_type": "INTRUSION", "confidence": 0.98}' \
    -F 'image=@assets/1.jpeg'
  sleep 1
done

  1. Verify flagged anomalies:
curl -X 'GET' 'http://localhost:8000/api/v1/events/anomalies'

2.4 Rate Limiting

for i in {1..62}; do
  curl -i -X POST "http://localhost:8000/api/v1/events/" \
    -F 'data={"camera_id": "CAM-01", "camera_name": "Gate", "event_type": "INTRUSION", "confidence": 0.98}' \
    -s -o /dev/null -w "%{http_code}\n"
done
curl -i -X POST "http://localhost:8000/api/v1/events/" \
  -F 'data={"camera_id": "CAM-01", "camera_name": "Gate", "event_type": "INTRUSION", "confidence": 0.98}'

πŸ— Architecture Decisions

  • Service Layer Pattern: Decoupled business logic from API route handlers to ensure high testability, maintainability, and reusability across different parts of the system.
  • Redis-backed Real-time Notifications: Leveraged Redis Pub/Sub as a message broker for WebSocket broadcasting to ensure the system can scale horizontally across multiple API worker instances.
  • Asynchronous Anomaly Detection: Utilized FastAPI's BackgroundTasks to handle "Burst Detection" (Tier 3.2) asynchronously, ensuring that complex event analysis does not block the primary ingestion response.
  • Multi-stage Docker Builds: Implemented a multi-stage Dockerfile architecture to separate the build-time dependencies from the runtime environment, successfully reducing the production image size from ~900MB to ~180MB.
  • Modern Tooling with uv: Adopted uv as the primary package manager for lightning-fast dependency resolution and guaranteed environment consistency through deterministic uv.lock files.

πŸš€ What I Would Improve Next

  • Full Authentication (T2.2): Implement JWT-based authentication and Role-Based Access Control (RBAC) to restrict access to sensitive event data and configuration endpoints.
  • Enhanced Test Coverage: Expand the current pytest suite to include integration tests for WebSocket broadcasting and end-to-end (E2E) testing for the frontend dashboard.
  • Durable Event Streaming: Transition from Redis Pub/Sub to a persistent message queue like RabbitMQ or Apache Kafka to ensure zero data loss for critical security alerts during high-traffic bursts.
  • Advanced UI Features: Add interactive data visualizations (charts/graphs) for event statistics and integrate live camera stream previews using HLS or WebRTC.
  • Edge Integration: Incorporate actual ONNX or TensorRT models directly into the pipeline instead of relying on mocked vector data for the similarity search feature.

About

CV project

Topics

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

 
 
 

Contributors