omarcevi.dev Book a call

ML Inference Systems Engineer · Istanbul, Turkey · open to collaborations

Omar Elcircevi

Building production ML inference systems. Google Developer Expert, Cloud AI · Google Dev Group organizer · Speaker · OSS contributor

Omar Elcircevi speaking at Google for Developers Day Istanbul
Google for Developers Day Istanbul
xt=αˉtx0+1−αˉtϵx_t = \sqrt{\bar{\alpha}_t}\,x_0 + \sqrt{1-\bar{\alpha}_t}\,\epsilon
about this photo

Forward diffusion. What a diffusion model is trained to undo: the photo x₀ mixed with Gaussian noise ε (seed 42), with ᾱt from diffusers' cosine schedule (squaredcos_cap_v2, 1000 timesteps). On load, the photo goes through the 50 timesteps a 50-step DDIM sampler visits, t = 980 down to 0; after that, the brush raises t wherever it passes, and t falls back to 0. No model runs here: the load-in is the noising played backward, not a model denoising, and the brush gives each pixel its own t, where a model sees one t for the whole image.

inference.sim · Llama 3 8B on one H100 · real time

Continuous batching. Requests join and leave at every step, so the queue drains instead of waiting on the slowest reply.

about this tab

Continuous batching. A request-level server reserves memory for the longest possible reply and runs each batch until its slowest request finishes, so under this traffic its queue fills up and requests get turned away. vLLM-style continuous batching lets requests join and leave at every step. Flip it and watch p99 time to first token.

prefill decode, one token per step finished idle slot (request-level only) preempted KV cache

a live LLM serving simulator: batching · KV cache · MIG · routing · real traffic · failure · sandbox

TTFT p50 – TTFT p99 – TPOT – throughput – KV cache – GPUs 1 cost –
preempted 0 rejected 0

A model of LLM serving, not a benchmark: step times come from the H100's specs and each model's real size. Pick a tab, send a burst, then pause and watch it drain.

readabout.md–

About

I work on ML inference systems: deploying and optimizing the platforms that serve recommendation engines, search ranking, and increasingly AI agents in production. Most recently that meant a platform serving 40M+ across 35 different countries users at over 1M requests per minute.

My work sits at the intersection of ML, systems, and infrastructure — vLLM and Triton for serving, and Kubernetes across cloud and on-prem. I care about the parts of ML that don't make it into most conference talks: p99 latency, GPU utilization, what breaks at 3am.

I'm a Google Developer Expert in Cloud AI. I co-organize GDG Istanbul and speak regularly at DevFest, Build with AI, and GDG events across Turkey and abroad, in both Turkish and English.

heads 0–11 →avg

These are the real attention weights of GPT-2 small reading the three paragraphs of this section, pooled from tokens to words. GPT-2 only looks backwards, so arcs always point to earlier words.

Each square is one head; rows are layers 0–11, and brighter squares look further back. Click one to switch.

computed by scripts/ml_figures.py · <|endoftext|> sink removed, rows renormalized

readexperience.md–

Experience

  • TrendyolTrendyol Mar 2024 – Sep 2026
    Machine Learning Engineer, ML Platform
    • Ran model serving for 40M+ across 35 different countries users at 1M+ requests per minute with Triton and vLLM, on GKE and on-prem Kubernetes.
    • Reconfigured H100 GPUs with MIG so more models could share each card, for about 1.75x more serving capacity.
    • Designed the platform's deployment client, auth, active zone switching, and pipeline event notifications.
    • Built TensorRT conversion pipelines and tuned models from data science teams across the company; traced CPU inference bottlenecks down to memory latency with perf and TMA.
    • Benchmarked AMD MI325X against H200 for LLM and ranking workloads, and built an MCP tool and a set of Claude skills for the platform.
  • Turkcell 2022 – 2024
    AI/ML Engineer, AI & Analytical Solutions
    • Helped build the backend and DevOps of an AutoML platform for corporate customers with Django REST Framework, AutoGluon, OpenShift, Jenkins, Kubernetes and S3: users upload a dataset, and preprocessing, feature engineering, splitting, training and testing run automatically before the best model is presented.
    • Helped build a customer insights platform, from segmentation to data visualization, serving SQL query results through FastAPI endpoints.
    • Helped train models on data center cooling systems to propose more efficient cooling and lower its cost.
    • Built personalized data insights to each customer's needs from big data platforms with Oracle SQL.

Community

  • Google Developer Expert, Cloud AI
    Google · 2026
  • Mentor, Google for Startups Accelerator
    Google for Startups · 2026
  • Co-organizer, GDG Istanbul
    Google Developer Groups
  • Gemma Train-the-Trainer
    Google Developer Experts · 2026
readspeaking.md–

Speaking

Talks

Workshops

readprojects.md–

Projects

replay · sdlc-agent-pipeline
t 0.0s cost $0.0000 tool calls 0 tokens 0 → 0 outcome running

A recorded run of my issue-to-PR agents, replayed with its real timestamps, tokens and cost. All recorded runs →

Live · rising
Learn machine learning by writing it, in Turkish

Short lessons and coding problems in Python, with interactive demos of gradient descent, BPE tokenization, attention and the KV cache. ML foundations are open now; LLM internals and writing an LLM from scratch are next.

PythonNumPyTritonTürkçe
Featured
Agents that turn a GitHub issue into a tested pull request

Planner, coder and reviewer agents in an ADK graph, routed by real diffs and test results instead of model claims. Commands run in sandboxes with no credentials or network, every run is cost-capped, and a human approves before a pull request opens. Deployed on Agent Runtime with keyless CI/CD. Watch recorded runs · architecture

PythonGoogle ADKAgent RuntimeVertex AITerraformGitHub Actions
Load testing vllm-metal on an M1 Pro

Four 4-bit models under AIPerf at 1 to 16 concurrent users. Bursty traffic made time-to-first-token about 5x worse than steady traffic at the same average rate, and the p99 knee came early. Scripts and raw per-request data included.

vllm-metalAIPerfGemmaQwenApple Silicon
Your first model on Triton Inference Server

Serve a ResNet-18 with dynamic batching and two model instances, load test it with perf_analyzer, and deploy it to GKE. Runs on a laptop CPU. Companion code for my intro article.

NVIDIA TritonONNXDockerGKE
A terminal-native AI research digest

Collects new AI papers and posts every day, ranks them by what you like (with or without an LLM), and lets you read full papers without leaving the terminal. Includes hybrid search and Q&A over everything you've read.

PythonGoogle ADKpre-alphaApache-2.0
Turkish meme reactions for Claude Code

A Claude Code plugin (goygoy): hand Claude a task and it answers "Başımla beraber abi! 🫡" — out loud, or in its own reply.

Claude Code pluginhooksbash
readwriting.md–

Writing & Open Source

Open source

  • adk-python #3037 — fix for third-party CrewAI tool integration
    google/adk-python
  • adk-web #272 — light theme for the ADK web UI
    google/adk-web
readcontact.md–

Book a call

I'm open to collaborations: talks and workshops, open-source work, and teams figuring out how to serve models or agents in production. Pick a slot, or just write to compare notes on inference at scale.

Book a call · 30 min on Google Meet Open the booking page ↗