Skip to content

fenix-hub/u18n

Folders and files

NameName
Last commit message
Last commit date

Latest commit

Β 

History

21 Commits
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 

u18n - Universal Translation Microservice

A high-performance, scalable translation microservice built with Flask and ArgosTranslate, designed for ease of use and integration into any multilingual application.

The microservice provides a robust translation API built on Flask and ArgosTranslate, with features like rate limiting, request throttling, and flexible configuration. The implementation follows best practices with proper error handling, middleware design, and comprehensive testing.

The service can handle various input and output formats, making it easy to integrate with different client applications. It's Docker-ready for easy deployment and includes health checks for monitoring.

🌟 Features

  • Multiple Translation Pairs: Support for various language pairs (en-es, es-en, en-fr, fr-en, en-de, de-en)
  • Rate Limiting: Protect your service with configurable rate limiting
  • Request Throttling: Control concurrent request load
  • Multiple Input/Output Formats: JSON and plain text support
  • Docker Ready: Easy deployment with Docker and Docker Compose
  • Configurable: YAML config files and environment variable overrides
  • Metrics: Request tracking with detailed headers
  • Health Checks: Built-in monitoring endpoint
  • Comprehensive API: Well-documented RESTful endpoints

πŸ“‹ API Endpoints

Health Check

GET /health

Returns service status and installed language packages.

Translation

POST /translate

Translates text from source language to target language.

Input Formats:

  • JSON body:
{
  "text": "Hello world",
  "source": "en",
  "target": "es",
  "outputFormat": "json"
}
  • Form data with parameters: text, source, target, outputFormat

Output Formats:

  • JSON:
{
  "translated": "Hola mundo",
  "source": "en",
  "target": "es",
  "original": "Hello world"
}
  • Plain text: Hola mundo

Configuration

GET /config

Returns current service configuration.

PUT /config

Updates service configuration.

πŸš€ Getting Started

Prerequisites

  • Docker and Docker Compose
  • Python 3.9+ (for development without Docker)

Running with Docker

  1. Clone the repository:
git clone https://github.com/fenix-hub/u18n.git
cd u18n
  1. Start the service using Docker Compose:
docker-compose up -d
  1. The service will be available at http://localhost:5000

Running Locally for Development

  1. Create a virtual environment:
python -m venv venv
source venv/bin/activate  # On Windows: venv\Scripts\activate
  1. Install dependencies:
pip install -r requirements.txt
  1. Run the service:
FLASK_APP=app/main.py python -m flask run

βš™οΈ Configuration

Configuration can be provided through:

  1. Default values (built into the application)
  2. YAML configuration file (config/default_config.yml)
  3. Environment variables (highest priority)

Environment Variables

Variable Description
CONFIG_PATH Path to YAML config file
RATE_LIMIT_ENABLED Enable/disable rate limiting (true/false)
RATE_LIMIT_RPM Requests per minute limit
RATE_LIMIT_BURST Maximum burst size
THROTTLING_ENABLED Enable/disable request throttling (true/false)
THROTTLING_CONCURRENT Maximum concurrent requests
MAX_CHARS_PER_REQUEST Character limit per translation request
AVAILABLE_PACKAGES Comma-separated list of language pairs (e.g., "en-es,es-en")
LOGGING_LEVEL Logging level (e.g., DEBUG, INFO, WARNING, ERROR)
LOGGING_FORMAT Python logging format string

Sample Configuration

logging:
  level: "INFO"
  format: "%(asctime)s - PID:%(process)d - %(name)s - %(levelname)s - %(message)s"

rate_limit:
  requests_per_minute: 60
  burst: 10
  enabled: true

throttling:
  concurrent_requests: 5
  enabled: true

translation:
  max_chars_per_request: 5000
  available_packages:
    - "en-es"
    - "es-en"
    - "en-fr"
    - "fr-en"
    - "en-de"
    - "de-en"
    - "it-en"
    - "en-it"
  fallback_response: "Translation service unavailable. Please try again later."

formats:
  input:
    - "json"
    - "text"
  output:
    - "json"
    - "text"

πŸ“Š Response Headers

The service adds informative headers to responses:

Rate Limiting Headers

  • X-RateLimit-Limit: Maximum requests per minute
  • X-RateLimit-Remaining: Remaining tokens
  • X-RateLimit-Reset: Seconds until full reset
  • Retry-After: Seconds to wait when rate limited

Throttling Headers

  • X-Throttle-Limit: Maximum concurrent requests
  • X-Throttle-Usage: Current usage
  • X-Throttle-Remaining: Available slots

Translation Headers

  • X-Translation-Characters: Character count of original text

πŸ§ͺ Testing

Run tests with pytest:

pytest

Run with coverage report:

pytest --cov=app tests/

πŸ”§ Architecture

The project follows a modular architecture with separate components for:

  • API Layer: RESTful endpoints and request handling
  • Service Layer: Business logic implementation
  • Configuration Management: Dynamic config loading
  • Middleware: Rate limiting and request throttling
  • Utilities: Helper functions and decorators

Project Structure

translation-service/
β”œβ”€β”€ Dockerfile                      # Docker image definition
β”œβ”€β”€ docker-compose.yml              # Docker composition for easy deployment
β”œβ”€β”€ requirements.txt                # Python dependencies
β”œβ”€β”€ README.md                       # Project documentation
β”œβ”€β”€ config/                         # Configuration management
β”‚   β”œβ”€β”€ __init__.py
β”‚   β”œβ”€β”€ default_config.yml          # Default configuration values
β”‚   └── config_loader.py            # Configuration loading logic
β”œβ”€β”€ app/                            # Application code
β”‚   β”œβ”€β”€ __init__.py                 # Logging configuration
β”‚   β”œβ”€β”€ main.py                     # Application entry point
β”‚   β”œβ”€β”€ api/                        # API endpoints
β”‚   β”‚   β”œβ”€β”€ __init__.py
β”‚   β”‚   β”œβ”€β”€ routes.py               # Route definitions
β”‚   β”‚   └── middleware.py           # Request middleware
β”‚   β”œβ”€β”€ services/                   # Business logic services
β”‚   β”‚   β”œβ”€β”€ __init__.py
β”‚   β”‚   β”œβ”€β”€ translation_service.py  # Translation logic
β”‚   β”‚   β”œβ”€β”€ rate_limiter.py         # Rate limiting service
β”‚   β”‚   └── request_throttler.py    # Request throttling service
β”‚   └── utils/                      # Utility functions
β”‚       β”œβ”€β”€ __init__.py
β”‚       └── helpers.py              # Helper decorators and functions
└── tests/                          # Test suite
    β”œβ”€β”€ __init__.py
    β”œβ”€β”€ test_api.py                 # API tests
    β”œβ”€β”€ test_translation.py         # Translation service tests
    └── test_config.py              # Configuration tests

πŸ“– About ArgosTranslate

This service uses ArgosTranslate, an open-source neural machine translation library that allows for offline translation. It supports multiple language pairs and uses lightweight models suitable for containerized deployments.

πŸ“ Performance Considerations

  • Model Download: Language models are downloaded on first use, which may cause initial delays
  • Memory Usage: Consider container memory limits based on the number of language pairs loaded
  • Caching: API responses are not cached; consider implementing a caching layer for frequent translations
  • Scaling: For high-volume scenarios, deploy multiple instances behind a load balancer

πŸ›‘οΈ Security Notes

  • The service does not implement authentication; consider adding API keys or OAuth2
  • Request rates are tracked by IP address; implement user-based tracking for multi-user scenarios
  • Add HTTPS for production deployments

🀝 Contributing

Contributions are welcome! Please feel free to submit a Pull Request.

  1. Fork the repository
  2. Create your feature branch (git checkout -b feature/amazing-feature)
  3. Commit your changes (git commit -m 'Add some amazing feature')
  4. Push to the branch (git push origin feature/amazing-feature)
  5. Open a Pull Request

πŸ“„ License

This project is licensed under the MIT License - see the LICENSE file for details.

About

A high-performance, scalable translation microservice built with Flask and ArgosTranslate, designed for ease of use and integration into any multilingual application.

Topics

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

 
 
 

Contributors