DEV Community: Nishant Kumar

I got tired of tedious dataset curation, so I built REDDIZ — an open-source vision training workstation

Nishant Kumar — Mon, 05 Oct 2026 02:16:23 +0000

This is a submission for the Hacktoberfest Weekend Challenge: Build for a Friend
Over the past few months, I've spent an unhealthy amount of time fine-tuning Flux and SDXL LoRAs and building small custom YOLO detectors. Every single time I start a new training run, the part that drives me crazy isn't setting learning rates or configuring optimizer schedules—it's the sheer manual grind of gathering and formatting the dataset.

DevRelay: (https://dev.to/agent_sessions/testing-vit-gpt2-image-captioning-and-running-reddit-vision-annotator-e3f7op)

If you train vision models, you probably know the drill all too well:

You find a subreddit or niche photo board with great source material.
You save images one by one or run an ad-hoc Python script that breaks halfway through due to Reddit's 429 rate limits or CORS blocks.
You open a photo editor to manually crop every picture into consistent vertical portrait crops (9:16).
You open a text editor to write out descriptive captions, camera tags, and trigger words for each image.
If you're doing object detection, you boot up Label Studio, Roboflow, or CVAT, wait for them to load, and click around dozens of UI menus just to draw a few bounding boxes and export standard coordinate files.

I got tired of jumping across three separate tools and wrestling with paywalled export limits. I wanted a fast, standalone, keyboard-driven workstation where I could search subreddits, frame images at 9:16, run multi-modal AI models for auto-tagging, draw bounding boxes, and hit one button to get a clean .zip ready for Kohya_ss, OneTrainer, or Ultralytics YOLO.

That's why I built REDDIZ.

Web App: https://reddiz.vercel.app/
GitHub: https://github.com/nissshhdev/reddiz
Coffee Fund: https://buymeacoffee.com/nissshhdevz

What REDDIZ Does (And How It Works)

REDDIZ is built around an authentic Swiss Modernist Brutalist layout using Archivo and Space Mono typography. Everything is laid out on a single, high-contrast screen with no nested menus or page reloads.

Here is a breakdown of what you can do inside the studio:

1. Ingestion: Multi-Subreddit Scraper & Local Uploader

You can harvest images directly from Reddit without needing a registered Reddit developer app or API client secret.

Type in any subreddit name or a combination separated by commas or spaces (for example: streetphotography, analog, EarthPorn).
Choose between sorting by NEW, HOT, or TOP.
Click FETCH MEDIA or press Enter. The built-in Node.js backend streams the posts through a lightweight proxy that strips out CORS restrictions, normalizes user agents to avoid 429 rate limits, and buffers full-resolution image URLs directly into the client.
If you already have your own photos, click UPLOAD MEDIA to batch drag-and-drop local .jpg/.png files or paste direct image URLs straight into the current session.

2. 9:16 Adaptive Centered Viewport & Resizable Panels

Diffusion models for character portraits, mobile wallpapers, and short-form video generation (like Wan 2.1 or HunyuanVideo) perform best when trained on consistent aspect ratios.

REDDIZ automatically centers incoming media into a normalized 9:16 crop.
The left column houses a vertical thumbnail filmstrip where you can quickly scrub through dozens of queued images.
Between the columns sits an interactive panel resizer. You can click and drag the divider horizontally to make the canvas larger on high-res monitors or expand the metadata side panel when working with long descriptive captions.

3. Multi-Modal AI Prompting & Auto-Captioning

Instead of forcing a single model or running a local Python server, REDDIZ connects directly to the provider of your choice using your own API key:

Google Gemini: Gemini 2.0 Flash, Gemini 2.0 Pro, Gemini 1.5 Pro
Anthropic Claude: Claude 3.5 Sonnet, Claude 3.5 Haiku
OpenAI: GPT-4o, GPT-4o Mini
Groq Cloud: Llama 3.2 11B & 90B Vision (incredibly fast, sub-second responses)
Hugging Face Hub: Serverless BLIP-2, ViT, and DETR endpoints

When you press + PROMPT (or use the shortcut Ctrl + Space), the vision engine analyzes the active image in real time. It looks at subject poses, facial features, clothing fabrics, environmental lighting, shadows, and camera perspective, then generates:

A rich natural language caption for diffusion training.
A list of granular token tags separated into individual chips.

Your API keys stay strictly inside your browser's localStorage. They are never sent to any intermediary server or database.

4. Bounding Box Annotation with Class Inheritance

For object detection datasets, you can draw 2D spatial bounding boxes directly over the canvas:

Built-in default classes: subject, foreground, background.
You can create custom classes on the fly with custom colors.
Auto-Inheritance: If you are annotating a video sequence or a photoshoot where the subject stays in roughly the same position across multiple frames, REDDIZ can inherit the bounding boxes from the previous image so you only have to nudge them instead of drawing from scratch.

5. Keyboard-First Ergonomics

The entire workflow is mapped to single-key shortcuts so you never have to move your hand back and forth between the keyboard and mouse:

[Enter] : Save current annotations and immediately load the next image in the queue.
[X] : Ignore/skip low-quality or irrelevant images.
[↑] / [↓] or [A] / [D] : Step backwards and forwards through the filmstrip.
[Ctrl + Space] : Trigger the active AI vision model.
[Esc] : Close any open modal dialog.

6. Universal 1-Click Export (.ZIP)

When you're done with a batch, click DONE & DOWNLOAD ZIP. REDDIZ bundles the entire dataset client-side:

Images: High-res, 9:16 cropped .jpg files numbered sequentially (0001.jpg, 0002.jpg, etc.).
Diffusion Text Pairs: Matching .txt files containing the prompt and token tags, ready to drop straight into Kohya_ss, OneTrainer, or a ComfyUI training pipeline.
YOLO Detection Labels: Matching .txt files with standard normalized coordinates:

  <class_id> <x_center> <y_center> <width> <height>

COCO & JSONL Manifests: An annotations.json file adhering to COCO formatting and a dataset_manifest.jsonl file for programmatic pipelines.

Real-World Test: Annotating Classical Indian Artwork

To test REDDIZ end-to-end, I loaded a high-detail classical painting featuring two Indian women in traditional sarees standing beside a serene lake at sunset beneath massive cumulus clouds.

Here is the source image used for the test:

Source image: Neoclassical landscape painting featuring two figures in traditional lehengas/sarees by the lake under sunset cumulus clouds.

Running the Annotation Inside REDDIZ

I dropped the image URL into REDDIZ, locked the 9:16 crop around the two women, and triggered Gemini 2.0 Flash with a single keystroke.

Here is the active studio screenshot captured during the test:

Active annotation session: 9:16 cropped canvas, vertical filmstrip, extracted token chips (traditional_attire, indian_saree, cumulus_clouds, golden_hour, river_reflection), spatial classes, and live caption stream.

Results Generated by the Studio:

1. Generated Natural Language Caption:

"A tranquil neoclassical landscape painting capturing two Indian women in traditional draped sarees standing by a calm lake meadow at golden hour, beneath sweeping pastel-pink cumulus clouds and rolling distant hills."

2. Token Tags Extracted:
traditional_attire, indian_saree, cumulus_clouds, golden_hour, river_reflection, neoclassical_painting, scenic_landscape

3. Normalized YOLO Bounding Boxes:

subject (the two women): 0.4700 0.7900 0.1400 0.1600
background (clouds & hills): 0.1900 0.4300 0.7600 0.2800
foreground (flower meadow): 0.0000 0.9000 1.0000 0.1000

The entire process took less than 15 seconds from raw image ingestion to ready-to-train export files.

Technical Stack & Philosophy

Zero-Bundle Frontend: Written in clean, vanilla HTML5, CSS3, and ES6+ JavaScript. No React, no Vue, no bloated virtual DOM. The page boots instantly in under 50ms.
Typography & Theme: Built using Google Fonts Archivo for headings and editorial copy, paired with Space Mono for technical coordinates and code.
Privacy First: Everything runs inside your browser. No cookies, no tracking scripts, no third-party telemetry. Your vision API keys stay on your machine.
Deployment: Hosted on Vercel with a lightweight Node.js streaming proxy for handling external media headers and Reddit API pagination.

Try It Out

REDDIZ is completely free and open source:

🔗 Live Web App: https://reddiz.vercel.app/
📦 GitHub Repository: https://github.com/nissshhdev/reddiz

If you build or fine-tune vision models, give it a run with your favorite subreddits or image folders. Pull requests and feedback are always welcome!

I got tired of tedious dataset curation, so I built REDDIZ — an open-source vision training workstation

Nishant Kumar — Sat, 03 Oct 2026 07:39:09 +0000

Over the past few months, I've spent an unhealthy amount of time fine-tuning Flux and SDXL LoRAs and building small custom YOLO detectors. Every single time I start a new training run, the part that drives me crazy isn't setting learning rates or configuring optimizer schedules—it's the sheer manual grind of gathering and formatting the dataset.

If you train vision models, you probably know the drill all too well:

You find a subreddit or niche photo board with great source material.
You save images one by one or run an ad-hoc Python script that breaks halfway through due to Reddit's 429 rate limits or CORS blocks.
You open a photo editor to manually crop every picture into consistent vertical portrait crops (9:16).
You open a text editor to write out descriptive captions, camera tags, and trigger words for each image.
If you're doing object detection, you boot up Label Studio, Roboflow, or CVAT, wait for them to load, and click around dozens of UI menus just to draw a few bounding boxes and export standard coordinate files.

That's why I built REDDIZ.

Web App: https://reddiz.vercel.app/
GitHub: https://github.com/nissshhdev/reddiz
Coffee Fund: https://buymeacoffee.com/nissshhdevz

What REDDIZ Does (And How It Works)

Here is a breakdown of what you can do inside the studio:

1. Ingestion: Multi-Subreddit Scraper & Local Uploader

You can harvest images directly from Reddit without needing a registered Reddit developer app or API client secret.

Type in any subreddit name or a combination separated by commas or spaces (for example: streetphotography, analog, EarthPorn).
Choose between sorting by NEW, HOT, or TOP.
Click FETCH MEDIA or press Enter. The built-in Node.js backend streams the posts through a lightweight proxy that strips out CORS restrictions, normalizes user agents to avoid 429 rate limits, and buffers full-resolution image URLs directly into the client.
If you already have your own photos, click UPLOAD MEDIA to batch drag-and-drop local .jpg/.png files or paste direct image URLs straight into the current session.

2. 9:16 Adaptive Centered Viewport & Resizable Panels

Diffusion models for character portraits, mobile wallpapers, and short-form video generation (like Wan 2.1 or HunyuanVideo) perform best when trained on consistent aspect ratios.

REDDIZ automatically centers incoming media into a normalized 9:16 crop.
The left column houses a vertical thumbnail filmstrip where you can quickly scrub through dozens of queued images.
Between the columns sits an interactive panel resizer. You can click and drag the divider horizontally to make the canvas larger on high-res monitors or expand the metadata side panel when working with long descriptive captions.

3. Multi-Modal AI Prompting & Auto-Captioning

Instead of forcing a single model or running a local Python server, REDDIZ connects directly to the provider of your choice using your own API key:

Google Gemini: Gemini 2.0 Flash, Gemini 2.0 Pro, Gemini 1.5 Pro
Anthropic Claude: Claude 3.5 Sonnet, Claude 3.5 Haiku
OpenAI: GPT-4o, GPT-4o Mini
Groq Cloud: Llama 3.2 11B & 90B Vision (incredibly fast, sub-second responses)
Hugging Face Hub: Serverless BLIP-2, ViT, and DETR endpoints

A rich natural language caption for diffusion training.
A list of granular token tags separated into individual chips.

Your API keys stay strictly inside your browser's localStorage. They are never sent to any intermediary server or database.

4. Bounding Box Annotation with Class Inheritance

For object detection datasets, you can draw 2D spatial bounding boxes directly over the canvas:

Built-in default classes: subject, foreground, background.
You can create custom classes on the fly with custom colors.
Auto-Inheritance: If you are annotating a video sequence or a photoshoot where the subject stays in roughly the same position across multiple frames, REDDIZ can inherit the bounding boxes from the previous image so you only have to nudge them instead of drawing from scratch.

5. Keyboard-First Ergonomics

The entire workflow is mapped to single-key shortcuts so you never have to move your hand back and forth between the keyboard and mouse:

[Enter] : Save current annotations and immediately load the next image in the queue.
[X] : Ignore/skip low-quality or irrelevant images.
[↑] / [↓] or [A] / [D] : Step backwards and forwards through the filmstrip.
[Ctrl + Space] : Trigger the active AI vision model.
[Esc] : Close any open modal dialog.

6. Universal 1-Click Export (.ZIP)

When you're done with a batch, click DONE & DOWNLOAD ZIP. REDDIZ bundles the entire dataset client-side:

Images: High-res, 9:16 cropped .jpg files numbered sequentially (0001.jpg, 0002.jpg, etc.).
Diffusion Text Pairs: Matching .txt files containing the prompt and token tags, ready to drop straight into Kohya_ss, OneTrainer, or a ComfyUI training pipeline.
YOLO Detection Labels: Matching .txt files with standard normalized coordinates:

  <class_id> <x_center> <y_center> <width> <height>

COCO & JSONL Manifests: An annotations.json file adhering to COCO formatting and a dataset_manifest.jsonl file for programmatic pipelines.

Real-World Test: Annotating Classical Indian Artwork

To test REDDIZ end-to-end, I loaded a high-detail classical painting featuring two Indian women in traditional sarees standing beside a serene lake at sunset beneath massive cumulus clouds.

Here is the source image used for the test:

Source image: Neoclassical landscape painting featuring two figures in traditional lehengas/sarees by the lake under sunset cumulus clouds.

Running the Annotation Inside REDDIZ

I dropped the image URL into REDDIZ, locked the 9:16 crop around the two women, and triggered Gemini 2.0 Flash with a single keystroke.

Here is the active studio screenshot captured during the test:

Results Generated by the Studio:

1. Generated Natural Language Caption:

"A tranquil neoclassical landscape painting capturing two Indian women in traditional draped sarees standing by a calm lake meadow at golden hour, beneath sweeping pastel-pink cumulus clouds and rolling distant hills."

2. Token Tags Extracted:
traditional_attire, indian_saree, cumulus_clouds, golden_hour, river_reflection, neoclassical_painting, scenic_landscape

3. Normalized YOLO Bounding Boxes:

subject (the two women): 0.4700 0.7900 0.1400 0.1600
background (clouds & hills): 0.1900 0.4300 0.7600 0.2800
foreground (flower meadow): 0.0000 0.9000 1.0000 0.1000

The entire process took less than 15 seconds from raw image ingestion to ready-to-train export files.

Technical Stack & Philosophy

Zero-Bundle Frontend: Written in clean, vanilla HTML5, CSS3, and ES6+ JavaScript. No React, no Vue, no bloated virtual DOM. The page boots instantly in under 50ms.
Typography & Theme: Built using Google Fonts Archivo for headings and editorial copy, paired with Space Mono for technical coordinates and code.
Privacy First: Everything runs inside your browser. No cookies, no tracking scripts, no third-party telemetry. Your vision API keys stay on your machine.
Deployment: Hosted on Vercel with a lightweight Node.js streaming proxy for handling external media headers and Reddit API pagination.

Try It Out

REDDIZ is completely free and open source:

🔗 Live Web App: https://reddiz.vercel.app/
📦 GitHub Repository: https://github.com/nissshhdev/reddiz

If you build or fine-tune vision models, give it a run with your favorite subreddits or image folders. Pull requests and feedback are always welcome!

I Gave 5 AI Models Bugs That Don't Crash Your Code. Here's Who Actually Fixed Them.

Nishant Kumar — Thu, 01 Oct 2026 18:13:23 +0000

Somewhere in every codebase there is a bug that's been there for months.

It doesn't crash the server. It doesn't print a red traceback. Your tests run green. Your CI pipeline shrugs and gives you a checkmark. But out in the real world, once every few hundred requests or after a specific input combination, something silently slips — a counter goes wrong by one, a thread sees an object before it's fully initialized, or a floating-point fee calculation drifts by a fraction of a cent that compounds into real money.

These are the bugs that take engineers half a Friday to find. They're subtle, and they're hard because there's no obvious signal pointing at them. You have to actually read and reason about the code.

I got curious: can AI models actually do this? Not generate code from scratch — that's what every benchmark measures. But look at a logically broken function that appears healthy, pinpoint the exact mechanism of failure, and produce a patch that is minimal: change what's broken, leave what works alone.

So I built SilentBug-Bench on Kaggle: a 10-task evaluation suite targeting silent software corruptions. I ran it against 5 models — two Claude variants, GPT-4o, Gemini 2.5 Flash, and Qwen2.5-Coder-32B — and measured not just whether the fix works, but how each model fixed it.

The results exposed something I didn't entirely expect.

What I Benchmarked

The 5 Bug Families

I picked 10 real-world defect archetypes — two per family — across five categories I've personally hit in production code:

1. Boundary / Off-by-One
The kind of error that works on your laptop test input but detonates on a 2-element array, an all-duplicate input, or a window size of 1. The deque sliding window max and the rotated binary search minimum both fell here. These bugs are one operator away from correct: < vs <=, mid vs mid - 1. Nothing else.

2. State Mutation / Mutable Defaults
Python's most famous footgun: def fn(x, cache={}). The default dict — or set, or list — is created exactly once, at the time the function is defined, and then shared across every call for the lifetime of the process. I used a graph DFS with a visited=set() default (state bleeds between separate path queries) and a call-history logger (100 entries accumulate globally, not per-caller).

3. Concurrency / Thread Safety
This category is brutally hard because the bugs are timing-dependent. I used a double-checked locking singleton where a racing thread can retrieve an object whose constructor hasn't finished, and a shared integer counter being incremented by 10 threads simultaneously with no lock — a counter += 1 that looks atomic but compiles to LOAD / ADD / STORE, with the GIL potentially releasing between any two of those.

4. Numerical / Type Coercion
Financial arithmetic in binary float. This one has a satisfying quality: it's easy to explain, hard to fully internalize, and extremely expensive when it goes wrong. Also a percentile rank function that always reports 0 for the lowest value because it counts strictly-less-than instead of the midpoint formula.

5. Logic / Algorithm Correctness
A topological sort that doesn't validate whether all nodes were actually processed (silently returns partial ordering when the graph has a cycle), and an interval merge function that works perfectly on sorted input but quietly corrupts random-order lists because nobody added the sort.

The Scoring Rubric (EGRS)

I wanted the benchmark to reward what a good code reviewer actually cares about:

EGRS = 0.30 × Root Cause Accuracy
     + 0.30 × Patch Pass Rate (edge-case unit tests)
     + 0.25 × Explanation Quality
     - 0.15 × Over-Engineering Penalty

That last term — the penalty — is the unusual one. If you correctly identify the bug but replace the entire function with a different algorithm, you lose points. That's intentional. A senior engineer reviewing a PR wants a minimal diff. Replacing an O(N) deque with an O(N log K) heap is a regression disguised as a fix.

The Models I Tested

I tried to put together a lineup that covered the real range of what teams are actually using:

Claude 3.5 Sonnet — Anthropic's flagship at the time, widely used for complex reasoning tasks and my personal most-trusted model for code review.
Claude 3.7 Haiku — The smaller, faster Claude. I wanted to know whether smaller context windows and lower compute budgets meaningfully hurt precision debugging.
GPT-4o — OpenAI's primary workhorse. Billions of developers use it through Copilot, APIs, ChatGPT. If there's a "default answer" to "which AI do you use for code", this is often it.
Gemini 2.5 Flash — Google's efficient-tier model. Fast, cheap, and arguably more accessible than the heavy flagships. I was curious whether it could compete on deep logic without the latency or cost.
Qwen2.5-Coder-32B — The standout from the open-weights world. A 32 billion parameter model that was purpose-trained on code. You can run this yourself. I was genuinely uncertain how it would stack up against the proprietary models.

What the Numbers Say

Leaderboard

Model	Root Cause	Patch Pass	Over-Eng	Explanation	EGRS
Claude 3.5 Sonnet	89.0%	96.0%	6.8%	89.8%	76.9
Qwen2.5-Coder-32B	85.2%	95.5%	5.9%	86.4%	74.9
GPT-4o	83.4%	86.0%	15.2%	85.4%	69.9
Claude 3.7 Haiku	79.0%	88.0%	10.0%	81.6%	69.0
Gemini 2.5 Flash	76.8%	83.5%	11.0%	79.4%	66.3

(EGRS = Evidence-Grounded Reasoning Score. Lower is better for Over-Engineering.)

Where Each Model Struggles — Per-Task Heatmap

The heatmap reveals something the aggregate scores hide: every model has specific categories where it reliably underperforms, and the worst single cluster is concurrency across the board.

The SB-05 column (singleton double-checked locking) is a bloodbath. Gemini 2.5 Flash scored 54 there. Claude 3.7 Haiku scored 58. GPT-4o scored 61. Even Claude 3.5 Sonnet only managed 69 — the lowest score it posted on any task.

State mutation (SB-03, the mutable DFS visited set) was the second-hardest category. Every model understood the problem conceptually but several still produced patches with lingering issues on exception paths.

The "Surgical vs. Scattershot" Axis

This scatter plot is the one that surprised me most.

The top-left quadrant is where you want to be: high root-cause accuracy, low over-engineering. Claude 3.5 Sonnet and Qwen2.5-Coder-32B both live there. GPT-4o is marooned in the bottom-right: decent root cause identification, but by far the highest tendency to rewrite working logic into something different.

That 15.2% over-engineering rate for GPT-4o isn't small. It means that on roughly 1 in 7 tasks, it produced a patch that correctly addressed the bug but changed the structure of the function in ways the original author wouldn't have wanted — sometimes degrading algorithmic complexity, sometimes introducing unnecessary dependencies.

Metric Radar

One thing stands out clearly in the radar: Qwen2.5-Coder-32B has the lowest "Low Over-Eng" score (inverted: higher means more surgical) while staying competitive on root cause and patch pass rate. For a model you can self-host, that's a meaningful result. It didn't just perform respectably — it beat GPT-4o convincingly on overall EGRS.

The Stories Behind the Numbers

Numbers are useful, but they don't tell you why models behave the way they do. Here's what I actually observed in the model outputs:

The "Fix by Rewriting" Problem (GPT-4o)

On SB-01 — the sliding window deque — GPT-4o correctly identified that there was an index boundary issue. It understood that elements needed to be evicted at the right time. But its response was: "To make this more robust, let me rewrite this using a max-heap."

What followed was a 30-line heapq-based solution that passed the basic test cases but:

Changed the time complexity from O(N) to O(N log K)
Introduced a (-val, idx) tuple trick to simulate a max-heap (Python only has min-heap), which requires explanation to anyone reading the code
Completely abandoned the deque structure that the original author had chosen deliberately

On SB-09 — topological sort — it added a cycle detection check (good!) but then rewrote the entire BFS loop into a recursive DFS because, as the model explained, "recursion is often clearer for graph traversal." Debatable at best; unwanted at worst.

This pattern was consistent enough across GPT-4o's responses that I started thinking of it as the "confident senior dev who refactors before understanding why the original code was written that way" anti-pattern.

The One-Line Diff Champion (Qwen2.5-Coder-32B)

Qwen's responses had a distinct character. When I gave it the rotated binary search task, it produced something like this:

"The bug is on line 7: hi = mid - 1 incorrectly discards mid as a candidate for the minimum. When nums[mid] <= nums[hi], mid itself could be the minimum. The fix is hi = mid."

And then it just changed that one line. That's it.

No essay about binary search variants. No rewrites. No added helper functions. One line changed, explanation matches exactly what needs fixing.

For the mutable default argument bug, it even caught a subtlety that several other models missed: that visited.copy() needs to be passed in the recursive call, not just visited, to prevent the DFS backtracking from interfering with parallel recursive paths. Most models fixed the top-level mutable default issue but left the recursive sharing bug intact.

Claude Understands Python Internals (Usually)

Claude 3.5 Sonnet was the most reliable model at reasoning about Python's execution model specifically, not just general programming concepts. On the mutable default argument task, it explained:

"Python evaluates default argument values at function definition time, not at call time. The visited=set() default creates a single set object stored in the function's __defaults__ attribute. Each call without an explicit visited argument receives a reference to this same object."

That's correct, specific, and shows genuine understanding of how Python works under the hood. Other models gave the same fix but with vaguer explanations ("Python doesn't create a new set each time") that were technically true but imprecise.

Where Claude stumbled was SB-05, the singleton concurrency task. It correctly identified the race condition and the fix (initializing the object completely before assigning to cls._instance). But it didn't mention that in Python 3.13's new free-threaded mode (with the GIL disabled via --disable-gil), the picture changes significantly — the fixes that work under standard CPython may not be sufficient without explicit memory barriers or synchronization primitives that go beyond threading.Lock.

Gemini Was Fastest and Best on Numerical (But Weakest on State)

Gemini 2.5 Flash was the first to respond and produced very clean fixes for the financial float precision task. It immediately moved to decimal.Decimal, chose the right quantization mode (ROUND_HALF_EVEN, also called "banker's rounding"), and even added a note explaining that passing the discount rate as a string to the Decimal() constructor is necessary to avoid pre-converting a binary float.

But on SB-03 (the DFS mutable default), it gave a fix that only partially solved the problem. It changed visited: set = set() to visited: set = None, added the if visited is None: visited = set() guard at the top — standard fix — but then left the recursive call passing visited directly rather than visited.copy(). The top-level call is now safe; nested calls still share state during a single DFS traversal. In testing, this passed simple paths but failed on graphs with multiple paths sharing visited nodes mid-traversal.

Haiku Struggles at the Hard End

Claude 3.7 Haiku's performance was noticeably weaker on the hard-difficulty tasks (SB-05, SB-07) while performing competitively on the easy and medium ones. The concurrency singleton was particularly rough — it produced an answer that looked correct but re-introduced the partial initialization risk by assigning cls._instance too early in the code path.

This is roughly what you'd expect from a smaller, faster model: great for the mechanical bugs where you just need to spot the pattern, less reliable when the failure mode requires understanding interaction between Python's runtime internals and threading behavior. The gap compared to Claude 3.5 Sonnet was much larger on expert tasks than on medium ones.

What I Would Measure Next

Building this benchmark opened up several questions I didn't have time to answer in this run:

1. Free-Threaded Python 3.13 Bugs
PEP 703 landed. The no-GIL build of Python is here experimentally, and the concurrency landscape is completely different without the GIL serializing interpreter-level operations. I want to design tasks specifically for python3.13 --disable-gil where the bugs that were previously masked by the GIL become live race conditions.

2. Patch Budget Constraints
Add an explicit constraint: "You may change at most 3 lines of code. If you rewrite the function, you score zero." I have a feeling this would substantially shuffle the leaderboard — probably moving Qwen higher and GPT-4o lower.

3. Multi-Turn Interactive Debugging
Zero-shot is one thing. But real debugging often involves iterating: "that didn't fix it, here's the failing test output, try again." How do models perform when they receive feedback and must narrow down their hypothesis? That's a different skill from cold diagnosis.

4. Performance Regression Detection
Separately measure whether the model's patch degraded Big-O complexity. Right now that's bundled into the over-engineering penalty, but it deserves its own metric. An O(N) → O(N log K) regression is qualitatively different from "added a comment."

Where Can You See It?

Everything is public and reproducible:

Kaggle Benchmark & Dataset:
SilentBug-Bench on Kaggle
Source code in this post:

benchmark_dataset.py — All 10 task definitions, scoring formula (EGRS), leaderboard generator, CSV export
generate_charts.py — Chart generation (leaderboard, radar, heatmap, scatter)

Final Thought

The most interesting finding here isn't who won. Claude 3.5 Sonnet edging out the competition on EGRS is roughly what I'd have predicted going in.

What I didn't predict was how consistently the over-engineering penalty separated the models in ways that raw accuracy scores didn't. Qwen2.5-Coder-32B — an open-weights model you can run on your own hardware — scored second overall precisely because it was the most conservative patcher. It didn't try to impress. It just fixed the bug.

There's a lesson in there about what we actually need from an AI coding assistant in day-to-day work. A model that rewrites your 15-line deque into a 35-line heap might score fine on "does it produce correct output" benchmarks. But it's not what you want sending pull requests to your codebase.

The best debugger is the one who understands what you built and changes the fewest things to make it work correctly.

The benchmark, dataset, scoring code, and raw results are all available on Kaggle at the link above. Have a bug category you'd add to v3? Drop it in the comments.

A Very Quick Quick Way to Test Your Website In Your Mobile Phone

Nishant Kumar — Fri, 05 Jul 2024 08:29:45 +0000

Very often, you might have come across a thought of testing your web page in development instead of resizing your web browser on PC while working on responsiveness. How will you fee if I tell you there’s an easy way to do this , without any plugin.

All you need is a VS Code and a smartphone device(iOS/android). Neat requirements, isn’t it.

Ensure that both your development device with Visual Studio Code and your mobile device are in the same network(very important).

Even the steps are shorts. Let me brief :

i) Installing Live Server Extension on Visual Studio Code.

ii) Go to your project directory.

Open you integrated terminal in VS Code and enter the command ‘ipconfig’ . Note your IPV4 adress.

In my case, it’s 192.168.1.8.

iii) Now , snap the ‘Go Live’ on the right hand corner of the screen.

Your local development server will be live on your local machine .

In my case , my project is live at my localhost:5500 port.

This is all I need.

v) And the final step.

Open any modern web browser in your smartphone.Go to the address:

x:y

where , replace:

x with IPV4 adress and y with port number.

In my case , it’s :

192.168.1.8:5500

You will see your web page live.

You might get an error as “Connection Refused”.

A simple way to solve this is to unblock your port (5500, in my case) via Firewall.

CONCLUSION:

Additionally, you can do this with any web browser supported device, be it a smartphone, PC or tablet. This method has been tested by me only with frontend projects.I am yet to test it wit frameworks and backends like SQL. Your checks and comments on this is really appreciated. I will update on this topic soon. Till then, you are welcome to try it and update me (fixes if any) .

THANK YOU !