Skip to content

feat: add visual-verification tools (verify_object_grounded + ortho-diag screenshot)#230

Open
obselate wants to merge 1 commit into
ahujasid:mainfrom
obselate:feat/visual-verification
Open

feat: add visual-verification tools (verify_object_grounded + ortho-diag screenshot)#230
obselate wants to merge 1 commit into
ahujasid:mainfrom
obselate:feat/visual-verification

Conversation

@obselate

@obselate obselate commented Apr 19, 2026

Copy link
Copy Markdown

Summary

When the MCP is used by a model (or a person driving it remotely), there is no deterministic way to confirm a visual claim. object.location.z and object.dimensions lie after asymmetric mesh edits — e.g., a hemisphere built by deleting the bottom of a sphere keeps the original origin, so location.z - dimensions.z / 2 is 15cm off from the actual base in world space. Displace modifiers on the ground vary z per position, so "put it on z=0" leaves a visible gap or intersection. The result is a feedback loop where the caller reports "fixed" and the user sees the bug.

This PR adds two complementary tools, aimed at the same failure mode from different angles.

1. verify_object_grounded(object_name, ground_name, slice_height=1.0, max_samples=500)

Samples vertices from the object's lower slice — world z within slice_height of its lowest vertex — and raycasts straight down against the ground's evaluated mesh via BVH. Returns min/max/median/mean gap in meters:

{
  "object_name": "Tree.Pine01",
  "ground_name": "Ground",
  "samples_tested": 142,
  "samples_hit": 140,
  "samples_missed": 2,
  "min_gap": -0.042,
  "max_gap": 0.31,
  "median_gap": 0.11,
  "mean_gap": 0.13,
  "slice_height": 1.0,
  "hint": "mixed: tilted or uneven ground; base partly above and partly below"
}

Deterministic, no model vision. Evaluated geometry means Displace on the ground and armature / shape-key deformation on the object are honored automatically. slice_height is configurable because canopy-dominated meshes (pines, etc.) have most of their verts high up — a bounded z-slice is more reliable than a percentile of sorted verts.

2. get_viewport_screenshot(..., target_object, view, distance_factor, ortho_padding)

When target_object is set, creates a throwaway orthographic camera framed on the target's world bbox, aimed from one of six named axes (front, back, left, right, top, bottom), and renders a clean image. The previous scene.camera, render resolution, output path, and image format are saved and restored after the render — the user's setup is untouched. The temp camera and its data block are removed with do_unlink=True so no orphans accumulate.

Default path (no target_object) is unchanged: the existing viewport screenshot with overlays.

Together these cover both sides of verification:

  • verify_object_grounded when you want a cheap, deterministic "is it touching?"
  • ortho-diag get_viewport_screenshot when the question is broader — floating, wrong rotation, wrong color, wrong material.

Test plan

  • verify_object_grounded('Cube', 'Plane') with a default cube on a plane → min_gap ≈ 0, hint = "grounded or intersecting".
  • Raise the cube 0.5m → min_gap ≈ 0.5, hint starts with "floating".
  • Sink the cube -0.1m → min_gap ≈ -0.1, hint = "grounded or intersecting" (all gaps below the 1cm threshold).
  • Tilt the cube so one corner intersects ground → hint = "mixed".
  • verify_object_grounded('Missing', 'Plane') → error message includes "not found".
  • verify_object_grounded('Cube', 'EmptyObject') → error message includes "is not a mesh".
  • get_viewport_screenshot(target_object='Cube', view='front') → returns a clean ortho render with the cube centered; user's viewport and scene camera unchanged.
  • Same with view='top', view='right', etc. — renders from the correct axis.
  • Unknown view value → error listing valid choices.
  • Default get_viewport_screenshot() call (no target_object) still returns the viewport-with-overlays screenshot as before.
  • After the diag render, no _MCPDiagCam object or camera data block remains in the scene.

Notes

Downstream tracking bead: Blender-tqy. Covers both "shapes" mentioned in the issue description as complementary tools rather than either/or.

Summary by CodeRabbit

  • New Features
    • Added object grounding verification tool that validates spatial relationships and provides gap statistics between objects.
    • Extended viewport screenshot capability to render targeted orthographic diagnostics of specific objects with configurable view angle, distance, and padding.

…iag screenshot)

Callers that claim a visual change is applied (grounded, aligned,
recolored) have no deterministic way to confirm it. obj.location /
obj.dimensions lie after asymmetric mesh edits, and position math
ignores Displace modifiers on the ground. The result: a feedback
loop where the model reports 'fixed' while the user sees the bug.

Two complementary additions:

1. verify_object_grounded(object_name, ground_name, slice_height,
   max_samples) — samples vertices from the object's lower slice
   (world z <= zmin + slice_height) and raycasts straight down onto
   the ground's evaluated BVH. Returns min/max/median/mean vertical
   gap in meters (positive = floating, negative = intersecting) with
   a coarse 'hint' classification. Uses evaluated geometry so
   Displace modifiers and armature/shape-key deformation are honored.

2. get_viewport_screenshot(target_object, view, distance_factor,
   ortho_padding) — when target_object is set, creates a throwaway
   orthographic camera framed on the target's world bbox from one
   of six named axes (front/back/left/right/top/bottom) and renders
   a clean image. Useful when the question is 'what does it actually
   look like' rather than 'what does my math say.' Default behavior
   (target_object=None) is unchanged: viewport screenshot with
   overlays.

Render state (scene.camera, render resolution, output path, image
format) is saved and restored around the temp-camera render so the
user's setup is untouched. The temp camera and its camera data are
removed via do_unlink=True to avoid orphan accumulation.
@coderabbitai

coderabbitai Bot commented Apr 19, 2026

Copy link
Copy Markdown
📝 Walkthrough

Walkthrough

Extended the MCP server with orthographic diagnostic rendering capabilities and object grounding verification tools. The get_viewport_screenshot method now accepts parameters to render diagnostic orthographics of specific objects, while a new verify_object_grounded method validates object placement against ground surfaces using BVH raycasting.

Changes

Cohort / File(s) Summary
Orthographic Diagnostic Rendering
addon.py, src/blender_mcp/server.py
Extended get_viewport_screenshot with optional target_object, view, distance_factor, and ortho_padding parameters. Added _render_ortho_diag method to render temporary orthographic cameras with computed centering and scaling for target objects. Updated server-side wrapper to conditionally pass diagnostic rendering parameters.
Object Grounding Verification
addon.py, src/blender_mcp/server.py
Added verify_object_grounded method to validate object placement against ground surfaces via BVH-based raycasting from object vertices. Samples vertices from a Z-slice, raycasts downward, and aggregates gap statistics (min/max/median/mean, hit/miss counts, categorical hints). Registered new MCP command handler for remote invocation.

Sequence Diagram(s)

sequenceDiagram
    actor Client
    participant Server as Blender MCP Server
    participant Addon as addon.py
    participant Blender as Blender Engine

    rect rgb(0, 100, 200, 0.5)
    Note over Client,Addon: Orthographic Diagnostic Rendering Flow
    Client->>Server: get_viewport_screenshot(target_object="Cube", view="front")
    Server->>Addon: send_command("get_viewport_screenshot", target_object="Cube", ...)
    Addon->>Blender: Locate object in scene
    Addon->>Blender: Compute evaluated bounding box (world space)
    Addon->>Blender: Create temporary orthographic camera
    Addon->>Blender: Render with write_still to filepath
    Addon->>Blender: Remove temporary camera/data
    Addon->>Server: Return {filepath, dimensions, target, view}
    Server->>Client: Screenshot metadata + image path
    end

    rect rgb(100, 0, 100, 0.5)
    Note over Client,Addon: Object Grounding Verification Flow
    Client->>Server: verify_object_grounded(object_name="Box", ground_name="Floor")
    Server->>Addon: send_command("verify_object_grounded", object_name="Box", ...)
    Addon->>Blender: Locate both objects, validate are meshes
    Addon->>Blender: Build BVH from ground object geometry
    Addon->>Blender: Sample object vertices from lower Z slice
    Addon->>Blender: Raycast downward to ground surface
    Addon->>Blender: Aggregate gap statistics (min/max/median/mean)
    Addon->>Server: Return {gap_stats, hit_count, miss_count, hint}
    Server->>Client: Grounding verification result
    end
Loading

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

Possibly related PRs

Suggested labels

Review effort 2/5

Poem

🐰 Whiskers twitch with diagnostic glee,
Objects rendered orthographically!
Raycasts dance 'cross ground so true,
Verifying placement through and through.
Blender's vision, crystal clear,
Grounded objects, never to fear!

🚥 Pre-merge checks | ✅ 3
✅ Passed checks (3 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly and concisely summarizes the two main features added: verify_object_grounded and ortho-diag screenshot capability for visual verification.
Docstring Coverage ✅ Passed Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@qodo-code-review

Copy link
Copy Markdown

Review Summary by Qodo

Add visual-verification tools for grounding and diagnostic rendering

✨ Enhancement

Grey Divider

Walkthroughs

Description
• Add verify_object_grounded() tool to measure vertical gaps between object bases and ground
  meshes using raycasting
• Extend get_viewport_screenshot() with orthographic diagnostic rendering for specific objects
  from six named view angles
• Both tools use evaluated geometry to honor Displace modifiers, shape keys, and armature
  deformations
• Provide deterministic visual verification without relying on position math that fails after
  asymmetric mesh edits
Diagram
flowchart LR
  A["User calls verify_object_grounded"] --> B["Sample lower-slice vertices"]
  B --> C["Raycast down onto ground BVH"]
  C --> D["Return gap statistics + hint"]
  E["User calls get_viewport_screenshot with target_object"] --> F["Create temp ortho camera"]
  F --> G["Frame on object bbox from view axis"]
  G --> H["Render clean image"]
  H --> I["Restore render state + cleanup"]
Loading

Grey Divider

File Changes

1. addon.py ✨ Enhancement +207/-2

Add grounding verification and ortho diagnostic rendering

• Register verify_object_grounded handler in command dispatcher
• Extend get_viewport_screenshot() signature with target_object, view, distance_factor,
 ortho_padding parameters
• Add _render_ortho_diag() method to create temporary orthographic camera, render from six named
 axes, and restore render state
• Implement verify_object_grounded() method using BVHTree raycasting to measure vertical gaps with
 statistical analysis

addon.py


2. src/blender_mcp/server.py ✨ Enhancement +88/-12

Expose visual-verification tools via MCP interface

• Extend get_viewport_screenshot() MCP tool signature with optional target object and view
 parameters
• Pass new parameters to Blender addon when target_object is provided
• Add new verify_object_grounded() MCP tool that wraps the Blender addon method
• Update docstrings to document orthographic diagnostic rendering and grounding verification
 workflows

src/blender_mcp/server.py


Grey Divider

Qodo Logo

@qodo-code-review

qodo-code-review Bot commented Apr 19, 2026

Copy link
Copy Markdown

Code Review by Qodo

🐞 Bugs (3) 📘 Rule violations (0) 📎 Requirement gaps (0)

Grey Divider


Action required

1. max_samples cap/crash bug 🐞 Bug ≡ Correctness
Description
verify_object_grounded() can crash with ZeroDivisionError when max_samples=0 and often exceeds the
intended max_samples cap due to using floor-division for the downsample step. This can lead to hard
failures or unexpectedly large numbers of raycasts (slow tool runs).
Code

addon.py[R576-578]

+        if len(slice_verts) > max_samples:
+            step = max(1, len(slice_verts) // max_samples)
+            slice_verts = slice_verts[::step]
Evidence
The downsampling logic divides by max_samples without validating it, and uses floor division so
counts between (max_samples, 2*max_samples) produce step=1 (no downsampling), violating the
documented ‘cap on raycasts’.

addon.py[576-578]
src/blender_mcp/server.py[388-392]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

### Issue description
`verify_object_grounded()` intends `max_samples` to cap raycasts, but:
- `max_samples=0` causes `ZeroDivisionError`.
- The current step calculation uses floor division, so it frequently fails to downsample when `len(slice_verts)` is only slightly larger than `max_samples`.

### Issue Context
This tool may be called with arbitrary parameters via MCP, so it must be robust to edge cases and must reliably respect the `max_samples` contract.

### Fix Focus Areas
- addon.py[576-578]

### Implementation notes
- Validate inputs early (e.g., `if max_samples <= 0: return {"error": "max_samples must be > 0"}`; similarly consider `slice_height > 0`).
- Compute a downsample step using ceiling division so the resulting list length is guaranteed to be `<= max_samples`, e.g.:
 - `step = math.ceil(len(slice_verts) / max_samples)`
 - then `slice_verts = slice_verts[::step]`
- Optionally, if exact `<= max_samples` is required even after slicing, truncate (`slice_verts = slice_verts[:max_samples]`).

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools



Remediation recommended

2. Full vertex list allocated 🐞 Bug ➹ Performance
Description
verify_object_grounded() builds a full world-space list of all object vertices before
filtering/downsampling, which can be extremely expensive for high-poly meshes even when max_samples
is small. This can freeze Blender during tool execution.
Code

addon.py[R569-575]

+        obj_mw = obj_eval.matrix_world
+        world_verts = [obj_mw @ v.co for v in obj_eval.data.vertices]
+        if not world_verts:
+            return {"error": f"Object '{object_name}' has no vertices"}
+
+        zmin = min(v.z for v in world_verts)
+        slice_verts = [v for v in world_verts if v.z <= zmin + slice_height]
Evidence
The code materializes world_verts for every vertex (obj_eval.data.vertices) and only later
reduces to a slice and applies max_samples, so large meshes pay the full transform+allocation cost
regardless of max_samples.

addon.py[569-575]
addon.py[576-579]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

### Issue description
`verify_object_grounded()` currently computes world-space coordinates for *all* vertices up front (`world_verts = [...]`). For dense meshes this can consume significant time/memory and negate the value of `max_samples`.

### Issue Context
The function ultimately needs only vertices within a bottom z-slice and then only up to `max_samples` raycasts.

### Fix Focus Areas
- addon.py[569-579]

### Implementation notes
Consider restructuring to avoid a full list allocation, for example:
1) First pass: compute `zmin` without storing all vertices (iterate vertices and track minimum transformed z).
2) Second pass: collect slice vertices, but cap memory by either:
  - reservoir sampling up to `max_samples`, or
  - collecting then applying a corrected downsample step + truncate.
3) Keep the current output fields the same.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools


3. Fixed ray origin offset 🐞 Bug ☼ Reliability
Description
verify_object_grounded() raycasts from z=v.z+1000, which can start below the ground surface for
deeply buried objects or large-scale scenes, causing widespread misses and a misleading 'No ground
samples hit' error. This breaks grounding checks in valid scene configurations.
Code

addon.py[R582-589]

+        for v in slice_verts:
+            origin = mathutils.Vector((v.x, v.y, v.z + 1000.0))
+            direction = mathutils.Vector((0, 0, -1))
+            hit_loc, hit_normal, hit_idx, hit_dist = bvh.ray_cast(origin, direction)
+            if hit_loc is None:
+                missed += 1
+                continue
+            gaps.append(v.z - hit_loc.z)
Evidence
Ray origins are computed using a constant +1000 offset independent of the ground’s world-space
height/bounds. If the ground surface is higher than v.z + 1000, rays originate under the ground
and will not intersect it when cast downward.

addon.py[582-589]
addon.py[591-596]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

### Issue description
Raycasts start from `v.z + 1000.0` regardless of scene scale or ground height. In cases where the ground surface is above that origin (e.g., object is buried by >1000 units), raycasts miss and the tool returns an incorrect failure.

### Issue Context
The tool already has access to `ground_eval` and can compute a safe world-space z to start from.

### Fix Focus Areas
- addon.py[564-567]
- addon.py[582-589]

### Implementation notes
- Compute a ray start height based on the ground’s evaluated world-space bounding box, e.g. compute `ground_zmax_world` from `ground_eval.bound_box` transformed by `ground_eval.matrix_world`.
- Use `origin = Vector((v.x, v.y, ground_zmax_world + margin))` (margin like 1.0m or `slice_height + 1.0`) and keep direction `(0,0,-1)`.
- Keep the existing miss/error reporting, but it should now represent true misses (outside bounds / empty ground), not ray-origin artifacts.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools


Grey Divider

ⓘ The new review experience is currently in Beta. Learn more

Grey Divider

Qodo Logo

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🧹 Nitpick comments (2)
addon.py (1)

574-589: Minor nits: validate slice_height, use _ for unused ray_cast returns.

  • slice_height <= 0 is not guarded. With 0 you'll only capture vertices at exactly zmin (often a single point, noisy median/hint); negative values silently return the empty-gaps error branch. Consider clamping to a small positive epsilon or returning a clearer validation error.
  • hit_normal, hit_idx, hit_dist are unused on line 585 (RUF059) — prefix with _ to make intent explicit.
♻️ Suggested tweaks
-        zmin = min(v.z for v in world_verts)
-        slice_verts = [v for v in world_verts if v.z <= zmin + slice_height]
+        if slice_height <= 0:
+            return {"error": f"slice_height must be positive (got {slice_height})"}
+        zmin = min(v.z for v in world_verts)
+        slice_verts = [v for v in world_verts if v.z <= zmin + slice_height]
@@
-            hit_loc, hit_normal, hit_idx, hit_dist = bvh.ray_cast(origin, direction)
+            hit_loc, _normal, _idx, _dist = bvh.ray_cast(origin, direction)
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@addon.py` around lines 574 - 589, Validate and guard slice_height before
using it: ensure slice_height > 0 (either raise a ValueError or clamp
small/non-positive values to a tiny epsilon) so zmin + slice_height isn't
effectively equal to zmin or negative; then continue using zmin and slice_verts
as before. Also update the bvh.ray_cast return handling in the loop to mark
unused return values explicitly (e.g., use _, _ ,_ for hit_normal, hit_idx,
hit_dist) and keep hit_loc for the gap calculation.
src/blender_mcp/server.py (1)

362-404: Optional: normalize the error return shape to JSON.

On the happy path the tool returns a JSON document, but on exception it returns a bare string like "Error verifying grounding: ...". Callers that parse the response as JSON will fail on errors. Consider returning json.dumps({"error": ...}) for symmetry, or at minimum match the convention used elsewhere consistently. Matches existing tool style here, so it's optional.

♻️ Optional refactor
     except Exception as e:
         logger.error(f"Error verifying grounding: {str(e)}")
-        return f"Error verifying grounding: {str(e)}"
+        return json.dumps({"error": f"Error verifying grounding: {e}"})
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@src/blender_mcp/server.py` around lines 362 - 404, The exception path in
verify_object_grounded returns a plain string but the success path returns JSON;
change the error return to a JSON string (e.g. json.dumps({"error": str(e)})) so
callers can always parse JSON, and keep the existing logger.error call
(logger.error(f"Error verifying grounding: {e}")) for diagnostics; update the
except block in verify_object_grounded (which calls get_blender_connection and
blender.send_command) to return the JSON-encoded error object instead of a bare
string.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@addon.py`:
- Around line 485-526: The camera data-object and scene object are created
before the try block which can leak bpy data-blocks if creation or linking
fails; move the calls to bpy.data.cameras.new("_MCPDiagCam"),
bpy.data.objects.new("_MCPDiagCam", cam_data) and
scene.collection.objects.link(cam_obj) into the try so that cam_data and cam_obj
are only created inside the protected block, and ensure the existing finally
still removes cam_obj and cam_data (using bpy.data.objects.remove and
bpy.data.cameras.remove with do_unlink=True) only if those variables are
non-None to avoid orphan leaks when creation/linking raised an exception.
- Line 508: The docstring/usage for the parameter named format is misleading:
scene.render.image_settings.file_format expects Blender enum names (e.g., "PNG",
"JPEG"), not lowercase aliases like "jpg"; update the docstring to state that
format must be a Blender image enum value OR implement normalization inside the
function by mapping common aliases (e.g., "jpg" -> "JPEG", "jpeg" -> "JPEG",
"tif" -> "TIFF", "png" -> "PNG") before assigning
scene.render.image_settings.file_format (use the format variable and the
scene.render.image_settings.file_format target to locate the assignment).

---

Nitpick comments:
In `@addon.py`:
- Around line 574-589: Validate and guard slice_height before using it: ensure
slice_height > 0 (either raise a ValueError or clamp small/non-positive values
to a tiny epsilon) so zmin + slice_height isn't effectively equal to zmin or
negative; then continue using zmin and slice_verts as before. Also update the
bvh.ray_cast return handling in the loop to mark unused return values explicitly
(e.g., use _, _ ,_ for hit_normal, hit_idx, hit_dist) and keep hit_loc for the
gap calculation.

In `@src/blender_mcp/server.py`:
- Around line 362-404: The exception path in verify_object_grounded returns a
plain string but the success path returns JSON; change the error return to a
JSON string (e.g. json.dumps({"error": str(e)})) so callers can always parse
JSON, and keep the existing logger.error call (logger.error(f"Error verifying
grounding: {e}")) for diagnostics; update the except block in
verify_object_grounded (which calls get_blender_connection and
blender.send_command) to return the JSON-encoded error object instead of a bare
string.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 431a30aa-d712-4abf-8108-cfe97bcd16af

📥 Commits

Reviewing files that changed from the base of the PR and between 7636d13 and 90abc6e.

📒 Files selected for processing (2)
  • addon.py
  • src/blender_mcp/server.py

Comment thread addon.py
Comment thread addon.py
Comment thread addon.py
MickeyBadBad added a commit to MickeyBadBad/atelier-mcp that referenced this pull request Apr 27, 2026
…rchies

When called on an EMPTY (typical Sketchfab/GLB import root with multi-mesh
children parented to it), aggregate vertices from all descendant meshes
instead of erroring out. Answers "is the imported model grounded?" in one
call without forcing the caller to find the right leaf mesh.

Verified on the test scene's Bench empty (children: footer_metal_0,
screw_screw_0, wood_wood_0, wood.004_wood_0): 48/48 samples hit, median
gap 4mm — matches manually traversing to find the lowest mesh, but now
returns the list of meshes actually sampled in `sampled_meshes` for
diagnostic transparency.

Found while testing the original PR ahujasid#230 cherry-pick — most real assets
in interior-design workflows are imported as hierarchies, so single-mesh
constraint was a hard limit in practice.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant