An open-source Document Parser application that leverages:

• 🚀 FastAPI for the backend API
• 🧠 llama-index-core, Kotaemon, Markitdown for document understanding and parsing
• Celery for asynchronous task processing
• 🔧 Fully Dockerized environment with development and production modes

Built for intelligent document parsing at scale with modular design and API-first architecture.

• ⚙️ Installation
• 🔐 Environment Setup
• 🚀 Running the Application
• 🐳 Run All Services with Docker
• 🛠 Makefile Commands
• 🤝 Contributing
• 📄 License

git clone https://github.com/maowragvn-ai/document-task-parser.git
cd document-task-parser

• Unix/macOS:

  python3 -m venv venv
  source venv/bin/activate

• Windows:

  python -m venv venv
  .\venv\Scripts\activate

pip install -r requirements.txt

1. Copy the example environment file:

cp .env.example .env

2. Update .env with your API keys and database credentials:

GOOGLE_GENAI_USE_VERTEXAI=FALSE
GOOGLE_API_KEY=
TAVILY_API_KEY=

## Local dev DB
DB_USER=myuser
DB_PASSWORD=1
DB_HOST=postgres
DB_PORT=5432
DB_NAME=dvp_database

## Redis broker URL
CELERY_BROKER_URL=redis://redis:6379/0

celery -A src.celery_worker worker --loglevel=info

• PostgreSQL: localhost:5432
• Redis: localhost:6379

💡 Use init_db.sh to initialize the database if needed.

./init_db.sh

uvicorn app_fastapi:app --host 0.0.0.0 --port 8000 --reload

• API Docs: http://localhost:8000/docs

streamlit run app_streamlit.py --server.port=8501 --server.address=0.0.0.0

• UI: http://localhost:8501

alembic revision --autogenerate -m "message"
alembic upgrade head

docker compose -f docker-compose.dev.yaml build

docker compose -f docker-compose.dev.yaml up

• FastAPI backend: http://localhost:8000

• Streamlit UI: http://localhost:8501

• PostgreSQL: http://localhost:5432

• Redis: http://localhost:6379

• Healthcheck for Backend: http://localhost:8000/health

• Healthcheck for all services: make healthcheck-dev

make db-migrate-dev
make db-upgrade-dev

docker-compose down

docker compose -f docker-compose.prod.yaml build

docker compose -f docker-compose.prod.yaml up -d

• Healthcheck for Backend: http://localhost:8000/health
• Healthcheck for all services: make healthcheck-prod

make db-migrate-prod
make db-upgrade-prod

make build-dev        # Build dev images
make up-dev           # Start dev containers
make deploy-dev       # Rebuild and start containers
make healthcheck-dev  # Check service health
make db-migrate-dev   # Create DB migration with timestamp
make db-upgrade-dev   # Apply latest DB migration
make bash-dev         # Open shell inside backend container
make logs-dev         # Show logs

make build-prod
make up-prod
make deploy-prod
make healthcheck-prod
make db-migrate-prod
make db-upgrade-prod
make bash-prod
make logs-prod

• Amazone AWS s3 storage: Integrate with Amazon S3 for document storage.
• Streamlit UI: Enhance the Streamlit UI for a more user-friendly experience.
• Testing: Add comprehensive testing for the application.
• CI/CD: Set up CI/CD pipelines for automated testing and deployment.
• Monitoring: Implement monitoring and logging for better observability.
• Security: Enhance security measures, including authentication and authorization.

• FastAPI
• Streamlit
• Celery
• Alembic
• LLama Index
• Google Gen AI

• kotaemon
• Markitdown

We welcome contributions of all kinds! Feel free to fork the repo, submit issues, or open a pull request.

This project is licensed under the MIT License.