This started off as a dumping ground for experiments using the new Micrososft Agent Framework (MAF) to see if MAF would work offline. The initial PoC involved Ollama (with Qwen3:8b), the MAF code and a Docling MCP server to parse out a PDF (an RFP) and generate answers. I then added basic RAG (using Chromadb) to store the extracted text. This developed into the direction I am now taking of building PoC's for both sides of an RFP factory (having been on both sides of the RFP process many, many times I know how painful both sides of the process can be).

The factory analogy has been used as I imagine these agents working in the background, e.g. via CRON jobs or running a a daemon, monitoring folders for documents and automatically creating the responses or assessments and dropping the drafts into a folder for further human processing.

The thinking behind this is that it will truly be a factory, a self contained Python script that is woken up by a cron job (with a shell script wrapper), that looks in two folders for an RFP to process and company information, it then builds an RFP response into a Word document which it saves - useful for sales or pursuit teams to reduce the amount of time spent on preparing RFP responses.

• Get rid of the async error messages.
• Add RAG (Chroma) for input data to ground questions.
• Add Word document creation to store output.
• Pass files in as parameters (with defaults).
• Try larger Qwen models (14b runs just fine - needed context window modified).
• Improved prompts to reduce instruction and user prompt overlap (got better responses and less reasoning).
• Built version to extract RFP contents into JSON and then loop through the individual JSON sections and develop responses (replicates what I do manually with GenAI's), all consolidated into a final Word document. This worked really well and was a major enhancement.
  • Keep JSON as intermediate format as it's entriely possible the RFP reponse could feed into an automated assessor that would ingest the JSON and assess it (easier to assess than parsing a Word document).
• RAG ingestion by Agent is now more predictible - giving one agent the responsibility of ingestion into RAG of the Company info AND JSON extraction of the requirements was too unpredictable (still hallucinating though).
• Increase context window size with 130k (128,000 tokens) model and have improved the responses as now we have multiple chunks being found and processed from the RAG retrieval so response quality has improved. Problem was likely that the context window was too small to process the docling converted company ionfo document - with the document being processed within the context window there is more data to chunck and therefore more responses to the RAG query and data for the prompts. It wasn't an hallucination problem but a context window and data problem.

At this point I copied the code from the public repo to RFP-Responder and built the MAO version as a solution, i.e. ready for phase 2.

• Split the Agents up into different agents and setup a controller Agent to use A2A to call the individual Agents with specific, tightly controlled scopes of work.
  • https://learn.microsoft.com/en-us/agent-framework/user-guide/agents/agent-types/a2a-agent?pivots=programming-language-python
  • Flask based for async i/o (amongst other things)
  • COMPLETED: See agents/ directory for multi-agent implementation
• QA Agent ✅ COMPLETED
  • ✅ Added QA agent with client persona-based evaluation (healthcare, government, financial, default)
  • ✅ Can pull in customer strategic plans and context documents to inform QA review
    • ✅ Reviews RFP responses from client perspective with structured feedback
    • ✅ Provides scores (1-5), strengths, weaknesses, red/green flags, improvement suggestions
    • ✅ Configurable personas via YAML files in agents/prompts/client_personas/
    • 📝 Self-learning/prompt optimization: Deferred to future phase (current MVP uses human-in-the-loop feedback)
• RFP Revision Agent ✅ COMPLETED
  • ✅ AI-powered response improvement based on QA feedback
  • ✅ Selectively revises sections with scores < 5/5
  • ✅ Addresses weaknesses, red flags, and incorporates improvement suggestions
  • ✅ Queries RAG to add specific details from company knowledge base
  • ✅ Produces revised Word document ready for final delivery
• Persistent ChromaDB ✅ COMPLETED
  • ✅ Shared persistent vector database across all agents
  • ✅ Eliminates in-memory database isolation issues
  • ✅ Auto-recovery from collection resets
• Add front end for file uploads, progress monitoring and download
• Change the upload to upload three required files: 1. RFP file, 2. Company background file, 3. Client background file (with hover tooltips explaining purpose of each file)
• Add QA persona selection to frontend - Users can now select from Healthcare, Government, Financial, or Default personas for quality assurance review
• Production - Security: Understand attack surface, for example, injecting prompts into source documents that are retrieved by RAG (add Security Agent or functionality to the Agent that does retreving to look for possible prompt injections) ✅ COMPLETED
  • ✅ Added Security Agent (Agent-Sec-01) - Prompt Security Sentinel on port 5007
  • ✅ Two-phase analysis: per-node scanning + full-structure distributed payload detection
  • ✅ Detects 7 threat vectors: direct instruction override, roleplay/virtualization, obfuscation/smuggling, payload splitting, context window escape, indirect injection, many-shot flooding
  • ✅ Orchestrator aborts workflow if malicious content detected, preserves security audit report in download package
  • ✅ Integrated into pipeline between RFP extraction and response generation
• Production - UI and Agent Separation: Have UI only responsible for accepting documents and displaying responses. Have Agents run completely separately ingesting documents and producing responses (security and scaling). ✅ COMPLETED
• Production - User Separation: PostgreSQL-backed job management with per-user isolation ✅ COMPLETED
• Production - Move from calling via API's to MAF Flows and add DevUI (https://www.youtube.com/live/FQtZCKWjARI?si=wRJc-AKjoNd3E5aP)
• Production - Review whether we have implemented A2A within MAF (MAF makes it easy to put everything into one Python file but I wanmt is separate for tuning and scaling).
• Production - Authentication for users and Agent to Agent
• Production - Authorisation for users (move from basic apache authentication to production grade solution - external system integration?)
• Production - Make safe for multiple users ✅ COMPLETED
  • ✅ Per-job RAG isolation - Each job uses its own ChromaDB collection (rfp_knowledge_{job_id[:8]})
  • ✅ PostgreSQL job queue with FOR UPDATE SKIP LOCKED for safe multi-orchestrator concurrency
  • ✅ Backend/orchestrator separation - Backend handles uploads, orchestrator processes jobs independently
  • ✅ Job-scoped RAG collections automatically cleaned up after processing
  • ✅ Concurrency guard (MAX_ORCHESTRATOR_INSTANCES) to prevent resource exhaustion
  • ✅ Per-agent step tracking in job_steps table with timing data
• Observability - OpenTel (https://www.youtube.com/watch?v=3yS-G-NEBu8)
• Can I improve the RAG retrieval even more (IBM - OpenRAG).
• Add 6th Agent to reproduce the manual ChatGPT step to create the final document.
• Instead of consuming company information and adding RAG overhead - what about fine-tuning a TLM like Gemma3:270M (or use the new Function Gemma to be the orchestrator agent - small and fast)
  • https://docs.unsloth.ai/models/functiongemma
• Work on async mode for each of the JSON question calls to improve performance (wonder how many I could launch?)
• Try larger Qwen models - qwen3:30b may be a bit too large with any large enough context window for my current server, although I suspect we will get a better quality RFP respnse from this larger model).
  • Qwen3:30b simply didn't work - was only using 0%-5%-10% of GPU and running forever on CPU. Swapped to OpenAI API for gpt-5.2 and much, much better.
• Automate: Develop cron shell script to monitor folder for PDF's, process the PDF and generate response (whether that is drafting a response or assessing a rubric).
• Migrate to vllm for multi-GPU support (apparently Ollama also supports this).
• Add email support to send out the responses by email.
• Add web front end and database backend (Supabase?) - upload Company info, RFP, Client Info and select QA profiles from drop down, need place to pull down results from - including all JSON audit and Word file. Need database to cope with Organisation and levels. How to seperate different organisations - different deployments? What cloud infra do we deploy on?
• Add self improving support - after the QA agent has run and has findings have it analyse the prompts for the previous agents and suggest improvements (high risk/friction activity)

Initially I didn't want to capture the reasoning information in the Word document but quickly changed my mind as:

• Prompt performance: Watching the Ollama logs and seeing the number and duration of the inferences I could see, combined with the reasoning output, that there is an opportunity to improve the performance by making the prompts clearer.
• Transparency: For the person receiving the output the reasoning provides some insight into the models reasoning process.

I optimized the prompt as much as possible and the 8B model was giving good answers. I upgraded to the Qwen:14B model and it made a massive difference - thought more clearly and when it encountered errors provided much clearer explanations of the errors and selected tools more clearly.

I was getting Docling failures and it looked like the model was calling Docling with an incorrect max_size format. Added the following to the agent prompt to solve that.

Do NOT pass numeric values for max_size, either omit max_size entirely or pass it as a string, e.g. "100"; if you are unsure, omit max_size so the default is used.

For version 4b11400 I worked on forcing the agent to load the RFP into the vector database but it didn't do a very good job at all of extracting the requirements from the vector database - it did a much better job when it seemed to just process the Docling markdown. Modified to remove the RFP load into Chroma and on ensuring only the company information was stored in the vector database. This may have worked better to store the JSON in the vector database after processing the RFP with some form of prefix to identify it as a requirement - I can't imagine this being an issue unless the RFP was absolutely huge.

It became apparent, even after upgrading to the 14B model that the approach of answering all the RFP requirements in one go by an agent wasn't going to work (maybe a function of the context window size of the smaller models). Took a different approach of using the model to extract the requirements and then process them one at a time (an approach I have had a lot of success with when manually responding to RFP's and Grant Requests using models) - this improved the quality of the output dramatically and it also, as expected, took longer to generate a document as the Agent was being called for each JSON element (RFP requirement).

Noticed this in the Ollama logs: 'Nov 02 22:57:57 ollama[1011]: time=2025-11-02T22:57:57.174Z level=WARN source=runner.go:159 msg="truncating input prompt" limit=4096 prompt=7268 keep=4 new=4096'

• Didn't seem able to use things like extra_body in the Python code for the Ollama setup to be able to modify the context window size.
• Found this article: ollama/ollama#8099
• Applied these changes - including changing the model name in the Python script and worked much better.
• Also saw the same message with the 14B model, used same approach to fix.
• Later increased the context window size to 128,000 tokens which improved the responses - due to more data being available in the context window form the docling server to ingest into the RAG data store.

   		(venv) ubuntu:~/MAF$ ollama run qwen3:8b
		>>> /set parameter num_ctx 40960
		Set parameter 'num_ctx' to '40960'
		>>> /save qwen3:8b-40k
		Created new model 'qwen3:8b-40k'
		>>> /bye
		(venv) ubuntu:~/MAF$ ollama list
		NAME                 ID              SIZE      MODIFIED
		qwen3:8b-40k         b891e3e3f240    5.2 GB    7 seconds ago
		qwen3:8b             500a1f067a9f    5.2 GB    59 minutes ago
		gemma3:270m           e7d36fb2c3b3    291 MB    4 weeks ago
		gemma3-doc:latest    f3ad5bc8c220    291 MB    2 months ago

	(venv) ubuntu@:~/MAF$ ollama show qwen3:8b-40k
  	Model
    	architecture        qwen3
    	parameters          8.2B
    	context length      40960
    	embedding length    4096
    	quantization        Q4_K_M

  	Capabilities
    	completion
    	tools
    	thinking

  	Parameters
    	num_ctx           40960

NOTE: num_ctx is not present in the base model for some reason.

The RFP Responder system has been refactored into a multi-agent architecture using the Microsoft Agent Framework with Agent-to-Agent (A2A) communication. This provides better separation of concerns, scalability, and maintainability.

The system consists of 7 agents communicating via REST APIs:

1. Orchestrator Agent (Port 5000) - Coordinates the workflow
2. RFP-Extractor Agent (Port 5001) - Extracts RFP requirements to JSON
3. CompanyInfo-Extractor Agent (Port 5002) - Ingests company info and customer context into ChromaDB
4. RFP-Prep Agent (Port 5003) - Generates responses using RAG
5. QA Agent (Port 5005) - Reviews responses from client perspective using configurable personas
6. RFP-Revision Agent (Port 5006) - Improves responses based on QA feedback
7. Security Agent (Port 5007) - Prompt Security Sentinel (Agent-Sec-01) - Scans extracted RFP requirements for prompt injection attacks, jailbreaks, and malicious context manipulations

The system uses PostgreSQL for job management, replacing the file-based manifest system for production use:

• Job queue: Jobs are registered with status waiting and claimed by orchestrator runners using FOR UPDATE SKIP LOCKED
• Step tracking: Each pipeline step (ingestion, extraction, security scan, preparation, QA, revision) is tracked in the job_steps table
• Output tracking: Generated files are recorded in job_outputs with metadata
• Concurrency: Multiple orchestrator job runners can process jobs simultaneously
• RAG isolation: Each job uses its own ChromaDB collection (rfp_knowledge_{job_id[:8]}) for safe parallel processing

See database/schema.sql for the full schema and PostgreSQL Deployment for setup instructions.

1. Ollama running with qwen3:14b-130k model (or qwen3:14b-40k)
2. Docling MCP server on port 8000
3. Python dependencies: pip install -r requirements.txt
4. PostgreSQL 14+ with the rfp_responder database configured
5. Apache web server configured with authentication (for web UI)

The system requires a PostgreSQL database for job management and multi-orchestrator support.

# Ubuntu/Debian
sudo apt update
sudo apt install postgresql postgresql-contrib

# RHEL/CentOS/Amazon Linux
sudo dnf install postgresql-server postgresql-contrib
sudo postgresql-setup --initdb
sudo systemctl start postgresql
sudo systemctl enable postgresql

# Connect as the postgres superuser
sudo -u postgres psql

# Create the application database and user
CREATE USER rfp_app WITH PASSWORD 'your_secure_password_here';
CREATE DATABASE rfp_responder OWNER rfp_app;
\c rfp_responder
GRANT ALL PRIVILEGES ON SCHEMA public TO rfp_app;
\q

# Apply the schema file (creates tables, indexes, functions, triggers)
sudo -u postgres psql -d rfp_responder -f database/schema.sql

# Or if using the rfp_app user directly
PGPASSWORD='your_secure_password_here' psql -h localhost -U rfp_app -d rfp_responder -f database/schema.sql

This creates:

• jobs table - One row per RFP submission with status tracking, file paths, orchestrator assignment
• job_steps table - Per-agent step tracking with timing (company_info_ingest, rfp_extraction, rfp_preparation, qa_review, rfp_revision)
• job_outputs table - Output file tracking with metadata
• Indexes for user job listing, orchestrator job claiming, and status queries
• Helper functions: claim_next_job(), complete_job(), fail_job(), register_job()
• Trigger for automatic updated_at timestamp management

Both the backend and orchestrator job runner need database credentials:

# Add to .env or export directly
export DB_HOST=localhost
export DB_PORT=5432
export DB_NAME=rfp_responder
export DB_USER=rfp_app
export DB_PASSWORD=your_secure_password_here

PGPASSWORD='your_secure_password_here' psql -h localhost -U rfp_app -d rfp_responder -c "\dt"

# Expected output:
#  Schema |    Name     | Type  | Owner
# --------+-------------+-------+---------
#  public | job_outputs | table | rfp_app
#  public | job_steps   | table | rfp_app
#  public | jobs        | table | rfp_app

For production deployments:

# Edit pg_hba.conf to restrict connections
# (typically at /etc/postgresql/16/main/pg_hba.conf)
# Only allow local connections and specific hosts:
# local   rfp_responder   rfp_app                 scram-sha-256
# host    rfp_responder   rfp_app   127.0.0.1/32  scram-sha-256

# Edit postgresql.conf for tuning:
# listen_addresses = 'localhost'     # Only listen locally
# max_connections = 100              # Adjust based on pool sizes
# shared_buffers = 256MB             # 25% of available RAM
# work_mem = 4MB
# maintenance_work_mem = 64MB

# Restart PostgreSQL after changes
sudo systemctl restart postgresql

The RFP Responder includes a web-based interface for easy file uploads and monitoring.

Required Files: The web UI requires three specific files to be uploaded:

1. RFP Document - The RFP or Proposal you are responding to (contains requirements, questions, and deliverables)
2. Company Information - Your company's capabilities, experience, team details, and case studies (used to answer RFP questions)
3. Client Background - Client's strategic direction and values (typically strategic plan, impact report, or mission statement to ensure responses align with client priorities)

Workflow:

1. Navigate to the web UI
2. Upload all three required files (hover over the ℹ️ icon on each card for detailed explanations)
3. Select a QA Evaluation Persona (optional) - Choose from Healthcare, Government, Financial, or Default perspectives for quality assurance review
4. Click "Start Processing" (enabled only when all three files are selected)
5. Monitor real-time processing status
6. Download the results ZIP file when complete

Features:

• Visual feedback showing which files have been uploaded
• QA Persona Selection - Choose industry-specific evaluation criteria (Healthcare, Government, Financial Services, or Generic)
• Real-time progress monitoring with auto-refresh
• Secure user-scoped job management
• Resume capability using job GUID

See frontend/README.md for detailed web UI documentation.

./start_agents.sh

The job runner is a separate process that polls PostgreSQL for waiting jobs and dispatches them to the agent pipeline:

# Start the job runner (separate from agents)
python orchestrator/job_runner.py

# With custom options
python orchestrator/job_runner.py \
  --orchestrator-id orch-01 \
  --poll-interval 5 \
  --max-instances 2

Option 1: Command Line (Basic - Three Files)

python agents/orchestrator_agent.py \
  /path/to/rfp_document.pdf \
  /path/to/company_info.pdf \
  /path/to/client_background.pdf \
  /output/directory

Option 2: Command Line (With QA Review)

python agents/orchestrator_agent.py \
  /path/to/rfp_document.pdf \
  /path/to/company_info.pdf \
  /path/to/client_background.pdf \
  /output/directory \
  healthcare

Available QA personas: healthcare, government, financial, default

Option 3: REST API (Full Configuration)

curl -X POST http://localhost:5000/orchestrate \
  -H "Content-Type: application/json" \
  -d '{
    "rfp_file": "/path/to/rfp_document.pdf",
    "company_info_file": "/path/to/company_info.pdf",
    "client_background_file": "/path/to/client_background.pdf",
    "output_dir": "/output/directory",
    "client_persona": "healthcare",
    "enable_qa_review": true
  }'

File Purposes:

• RFP Document: The proposal being responded to (requirements, questions, deliverables)
• Company Information: Your company's capabilities, experience, team details, case studies
• Client Background: Client's strategic plan, values, and priorities for alignment

./stop_agents.sh

• {rfp_name}_requirements.json - Extracted RFP requirements
• {rfp_name}_requirements.audit.json - Extraction audit trail
• {rfp_name}_security_audit.json - Security scan results (prompt injection detection)
• {rfp_name}_answers.json - Generated responses
• {rfp_name}_answers.docx - Final Word document (ready for review)
• {rfp_name}_qa_feedback.json - QA review feedback (if QA enabled)
• {rfp_name}_answers_revised.json - Revised responses (if QA enabled)
• {rfp_name}_answers_revised.docx - Revised Word document (if QA enabled)

• ARCHITECTURE.md - System architecture, diagrams, scaling strategies
• IMPLEMENTATION_SUMMARY.md - Implementation details and code statistics
• agents/README.md - Agent architecture and communication flow
• backend/README.md - Backend API and PostgreSQL integration
• frontend/README.md - Web UI documentation
• apache/README.md - Apache reverse proxy setup
• database/schema.sql - PostgreSQL schema with helper functions

• Responder-PoC.py - Original monolithic implementation (Phase 1)

• agents/orchestrator_agent.py - Workflow orchestrator
• agents/rfp_extractor_agent.py - RFP extraction agent
• agents/company_info_extractor_agent.py - Company info ingestion agent
• agents/rfp_prep_agent.py - Response generation agent
• agents/qa_agent.py - Quality assurance review agent
• agents/rfp_revision_agent.py - Response revision agent
• agents/security_agent.py - Prompt injection detection agent (Agent-Sec-01)
• shared/rag_manager.py - Shared RAG database manager
• shared/document_utils.py - Document conversion utilities
• shared/persona_loader.py - Persona loader for QA agent
• shared/config.py - Shared configuration

• database/schema.sql - PostgreSQL schema (tables, indexes, functions)
• orchestrator/job_runner.py - Standalone job runner process
• backend/database.py - PostgreSQL connection pool and helpers

• start_agents.sh - Start all agents
• stop_agents.sh - Stop all agents

• requirements.txt - Python dependencies
• .env.example - Example environment configuration
• backend/.env.example - Backend environment configuration