A production-ready background job processing REST API built with BullMQ, Redis, Express, and TypeScript. Supports asynchronous email sending via Ethereal test SMTP and PDF/CSV report generation with real-time job status tracking, Swagger documentation, and Bull Board monitoring.

• Email Queue - Asynchronous email sending with Nodemailer; uses a real SMTP server when configured, otherwise falls back to Ethereal test SMTP with preview URLs
• Report Queue - PDF and CSV report generation from structured data using PDFKit and json2csv, resilient to rows with missing or inconsistent keys
• Report Download - Stream completed PDF/CSV reports via a dedicated download endpoint
• Real-Time Job Status Tracking - Poll job state, progress percentage, results, and failure reasons
• Swagger Documentation - Interactive API docs with request/response examples at /api-docs
• Bull Board Dashboard - Visual job monitoring and management panel at /admin/queues
• Security Hardened - Helmet, CORS, rate limiting, input validation (Zod), HPP protection
• Retry & Backoff - Automatic retries with exponential backoff on job failures
• Graceful Shutdown - Clean process termination with worker draining
• Render-Ready - One-click deployment with render.yaml blueprint

⚡ View Live Demo

• Node.js + TypeScript - Type-safe runtime with ES2022 features and strict mode
• Express.js - Minimal and flexible web framework for REST API
• BullMQ - High-performance Redis-backed job queue for Node.js
• Redis (ioredis) - In-memory data store used as message broker and job persistence
• Nodemailer + Ethereal - Email sending with test SMTP for safe development
• PDFKit - PDF document generation for report exports
• json2csv - JSON to CSV conversion for tabular report exports
• Zod - TypeScript-first schema validation for request payloads
• swagger-jsdoc + swagger-ui-express - Auto-generated OpenAPI documentation
• @bull-board/express - Real-time visual dashboard for queue monitoring
• Helmet - Secure HTTP headers middleware
• CORS + HPP - Cross-origin resource sharing and HTTP parameter pollution protection
• express-rate-limit - API rate limiting (100 requests per 15 minutes)
• express-basic-auth - Basic authentication for Bull Board in production
• Winston - Structured logging with multiple transports

• Node.js >= 18.0.0
• Redis server running locally or remotely

1. Clone the repository:

git clone https://github.com/serkanbyx/job-queue-api.git
cd job-queue-api

2. Install dependencies:

npm install

3. Configure environment variables:

cp .env.example .env

4. Start Redis (if not already running):

Using Docker:

docker run -d --name redis -p 6379:6379 redis:alpine

Using Homebrew (macOS):

brew services start redis

Using WSL/Ubuntu:

sudo service redis-server start

5. Start the development server:

npm run dev

6. Open your browser and navigate to http://localhost:3000/api-docs to explore the API.

npm run build
npm start

1. Start the server and navigate to the Swagger UI at /api-docs
2. Submit an email job by sending a POST request to /api/jobs/email with recipient, subject, and body
3. Submit a report job by sending a POST request to /api/jobs/report with type (pdf/csv), title, and data array
4. Track job progress by polling /api/jobs/:queue/:id/status with the returned job ID
5. Monitor all queues visually through the Bull Board at /admin/queues

Client Request → Express (Validation + Security) → BullMQ Queue → Redis
                                                         ↓
                                                   BullMQ Worker
                                                         ↓
                                              Service (Email / Report)
                                                         ↓
                                              Result stored in Redis
                                                         ↓
                                          Client polls /status endpoint

When an email job is submitted, it gets added to the email queue with 3 retry attempts and exponential backoff (2s base). The email worker (concurrency: 5, rate limit: 10 jobs/sec) processes the job using Nodemailer. If SMTP_HOST is configured, it sends through the real SMTP server; otherwise it falls back to an Ethereal test account and returns a preview URL for verification.

Report jobs support PDF and CSV formats. The report worker (concurrency: 3) generates documents using PDFKit or json2csv and saves them to the reports/ directory. Each job supports up to 10,000 data objects.

curl -X POST http://localhost:3000/api/jobs/email \
  -H "Content-Type: application/json" \
  -d '{
    "to": "user@example.com",
    "subject": "Welcome!",
    "body": "<h1>Hello</h1><p>Welcome to our platform.</p>"
  }'

Response:

{
  "message": "Email job queued successfully",
  "jobId": "1",
  "queue": "email"
}

curl -X POST http://localhost:3000/api/jobs/report \
  -H "Content-Type: application/json" \
  -d '{
    "type": "pdf",
    "title": "Monthly Sales Report",
    "data": [
      { "name": "Product A", "quantity": 150, "revenue": 4500 },
      { "name": "Product B", "quantity": 89, "revenue": 2670 }
    ]
  }'

Response:

{
  "message": "Report job queued successfully",
  "jobId": "1",
  "queue": "report"
}

curl -OJ http://localhost:3000/api/jobs/report/1/download

Returns the generated PDF or CSV file once the report job has completed. Responds with 409 if the report is not ready yet and 404 if the job or file does not exist.

curl http://localhost:3000/api/jobs/email/1/status

Response:

{
  "jobId": "1",
  "queue": "email",
  "state": "completed",
  "progress": 100,
  "result": {
    "messageId": "<abc123@ethereal.email>",
    "previewUrl": "https://ethereal.email/message/..."
  },
  "failedReason": null,
  "timestamp": "2026-03-05T12:00:00.000Z",
  "processedOn": "2026-03-05T12:00:01.000Z",
  "finishedOn": "2026-03-05T12:00:02.000Z"
}

src/
├── config/
│   ├── env.ts              # Environment validation (Zod)
│   ├── redis.ts            # Redis connection config
│   ├── swagger.ts          # Swagger/OpenAPI setup
│   └── bullBoard.ts        # Bull Board dashboard setup
├── queues/
│   ├── emailQueue.ts       # Email queue definition & options
│   └── reportQueue.ts      # Report queue definition & options
├── workers/
│   ├── emailWorker.ts      # Email processing worker
│   └── reportWorker.ts     # Report generation worker
├── routes/
│   ├── jobRoutes.ts        # Job submission & status endpoints
│   └── healthRoutes.ts     # Health check endpoint
├── middlewares/
│   ├── security.ts         # Helmet, CORS, rate-limit, HPP
│   └── validate.ts         # Zod validation middleware
├── schemas/
│   ├── emailSchema.ts      # Email payload schema
│   └── reportSchema.ts     # Report payload schema
├── services/
│   ├── emailService.ts     # Nodemailer + Ethereal integration
│   └── reportService.ts    # PDFKit + json2csv generation
├── utils/
│   └── logger.ts           # Winston logger configuration
├── app.ts                  # Express app setup & middleware
└── server.ts               # Entry point & graceful shutdown

• Helmet - Sets secure HTTP headers (XSS, CSP, HSTS, etc.)
• CORS - Configurable origin whitelist via CORS_ORIGINS
• Rate Limiting - 100 requests per 15 minutes on job submission routes
• HPP - HTTP Parameter Pollution protection
• Zod Validation - Strict input validation on all request payloads
• Bull Board Auth - Basic authentication protection in production
• No Secrets in Code - All credentials managed via environment variables

1. Push your code to a GitHub repository
2. Go to Render Dashboard
3. Click New > Blueprint
4. Connect your repository
5. Render will auto-detect render.yaml and provision both the web service and Redis instance
6. Set the required environment variables in the Render dashboard

1. Fork the repository
2. Create your feature branch (git checkout -b feat/amazing-feature)
3. Commit your changes using semantic format:
   • feat: - New feature
   • fix: - Bug fix
   • refactor: - Code refactoring
   • docs: - Documentation changes
   • chore: - Maintenance tasks
4. Push to the branch (git push origin feat/amazing-feature)
5. Open a Pull Request

This project is licensed under the MIT License - see the LICENSE file for details.

Serkanby

• Website: serkanbayraktar.com
• GitHub: @Serkanbyx
• Email: serkanbyx1@gmail.com

• Open an Issue
• Email: serkanbyx1@gmail.com
• Website: serkanbayraktar.com

⭐ If you like this project, don't forget to give it a star!