A real-time image captioning and visual question answering (VQA) system. This project uses computer vision and NLP to generate descriptive captions for images and answer user questions about them.

Mini CLIP-style CXR↔report retrieval with prompt ablation, token-length check, and interpretability

Fine-tuned BLIP model on Flickr8k for multimodal image captioning (vision + language).

Resource-aware X-CLIP baseline on Cholec80: FP16 + grad-accum training and evaluation for surgical video –text localization

This repository hosts the code for Jan Hadl's Master Thesis at TU Wien: GS-VQA, a zero-shot VQA pipeline that uses VLMs for visual perception and ASP for symbolic reasoning.

Cross-lingual evaluation of CLIP on Japanese vs English memes — revealing 7.4% performance gap and sarcasm detection failure

A benchmark for measuring whether multimodal assistants update to current context instead of staying anchored to prior context. 50 scenarios, three channel design (audio, camera, ground truth), cross family LLM as judge by default.

Mamba for Vision, Perception and Action

PyTorch code for Finding in NAACL 2022 paper "Probing the Role of Positional Information in Vision-Language Models".

Real-time AI that sees, understands, and talks about what it sees - like a visual brain.

Evaluation Framework for Multimodal RAG Systems

Tiny PyTorch adapter bridging visual features into language-token embeddings.

CLIP zero-shot image-text grounding, cross-modal attention visualization, and Stable Diffusion prompt-conditioned generation. VLM/VLA research.

Lightweight multimodal RAG toolkit with citation-focused context building

Fine-tuning CLIP and SigLIP on domain-specific datasets with zero-shot evaluation

Research-grade app for multimodal traffic safety analysis from live streams and recorded video.

This repository contains a spatial understanding test suite for vision-language models

Kaggle-runnable phrase-to-CT slice retrieval (OpenCLIP+FAISS). Portfolio-scale, teachable demo for foundation models in oncology

ComfyUI node for single-image captioning using frontier multimodal models via OpenRouter and Replicate, with cost estimation and fallback support.

Vision-Language Model for Automated Radiology Report Generation — ViT encoder + GPT decoder with cross-attention, SCST reward training, and hallucination detection