feat: add visual-verification tools (verify_object_grounded + ortho-diag screenshot)#230
feat: add visual-verification tools (verify_object_grounded + ortho-diag screenshot)#230obselate wants to merge 1 commit into
Conversation
…iag screenshot) Callers that claim a visual change is applied (grounded, aligned, recolored) have no deterministic way to confirm it. obj.location / obj.dimensions lie after asymmetric mesh edits, and position math ignores Displace modifiers on the ground. The result: a feedback loop where the model reports 'fixed' while the user sees the bug. Two complementary additions: 1. verify_object_grounded(object_name, ground_name, slice_height, max_samples) — samples vertices from the object's lower slice (world z <= zmin + slice_height) and raycasts straight down onto the ground's evaluated BVH. Returns min/max/median/mean vertical gap in meters (positive = floating, negative = intersecting) with a coarse 'hint' classification. Uses evaluated geometry so Displace modifiers and armature/shape-key deformation are honored. 2. get_viewport_screenshot(target_object, view, distance_factor, ortho_padding) — when target_object is set, creates a throwaway orthographic camera framed on the target's world bbox from one of six named axes (front/back/left/right/top/bottom) and renders a clean image. Useful when the question is 'what does it actually look like' rather than 'what does my math say.' Default behavior (target_object=None) is unchanged: viewport screenshot with overlays. Render state (scene.camera, render resolution, output path, image format) is saved and restored around the temp-camera render so the user's setup is untouched. The temp camera and its camera data are removed via do_unlink=True to avoid orphan accumulation.
📝 WalkthroughWalkthroughExtended the MCP server with orthographic diagnostic rendering capabilities and object grounding verification tools. The Changes
Sequence Diagram(s)sequenceDiagram
actor Client
participant Server as Blender MCP Server
participant Addon as addon.py
participant Blender as Blender Engine
rect rgb(0, 100, 200, 0.5)
Note over Client,Addon: Orthographic Diagnostic Rendering Flow
Client->>Server: get_viewport_screenshot(target_object="Cube", view="front")
Server->>Addon: send_command("get_viewport_screenshot", target_object="Cube", ...)
Addon->>Blender: Locate object in scene
Addon->>Blender: Compute evaluated bounding box (world space)
Addon->>Blender: Create temporary orthographic camera
Addon->>Blender: Render with write_still to filepath
Addon->>Blender: Remove temporary camera/data
Addon->>Server: Return {filepath, dimensions, target, view}
Server->>Client: Screenshot metadata + image path
end
rect rgb(100, 0, 100, 0.5)
Note over Client,Addon: Object Grounding Verification Flow
Client->>Server: verify_object_grounded(object_name="Box", ground_name="Floor")
Server->>Addon: send_command("verify_object_grounded", object_name="Box", ...)
Addon->>Blender: Locate both objects, validate are meshes
Addon->>Blender: Build BVH from ground object geometry
Addon->>Blender: Sample object vertices from lower Z slice
Addon->>Blender: Raycast downward to ground surface
Addon->>Blender: Aggregate gap statistics (min/max/median/mean)
Addon->>Server: Return {gap_stats, hit_count, miss_count, hint}
Server->>Client: Grounding verification result
end
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~20 minutes Possibly related PRs
Suggested labels
Poem
🚥 Pre-merge checks | ✅ 3✅ Passed checks (3 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Review Summary by QodoAdd visual-verification tools for grounding and diagnostic rendering
WalkthroughsDescription• Add verify_object_grounded() tool to measure vertical gaps between object bases and ground meshes using raycasting • Extend get_viewport_screenshot() with orthographic diagnostic rendering for specific objects from six named view angles • Both tools use evaluated geometry to honor Displace modifiers, shape keys, and armature deformations • Provide deterministic visual verification without relying on position math that fails after asymmetric mesh edits Diagramflowchart LR
A["User calls verify_object_grounded"] --> B["Sample lower-slice vertices"]
B --> C["Raycast down onto ground BVH"]
C --> D["Return gap statistics + hint"]
E["User calls get_viewport_screenshot with target_object"] --> F["Create temp ortho camera"]
F --> G["Frame on object bbox from view axis"]
G --> H["Render clean image"]
H --> I["Restore render state + cleanup"]
File Changes1. addon.py
|
Code Review by Qodo
1. max_samples cap/crash bug
|
There was a problem hiding this comment.
Actionable comments posted: 2
🧹 Nitpick comments (2)
addon.py (1)
574-589: Minor nits: validateslice_height, use_for unused ray_cast returns.
slice_height <= 0is not guarded. With 0 you'll only capture vertices at exactlyzmin(often a single point, noisy median/hint); negative values silently return the empty-gaps error branch. Consider clamping to a small positive epsilon or returning a clearer validation error.hit_normal,hit_idx,hit_distare unused on line 585 (RUF059) — prefix with_to make intent explicit.♻️ Suggested tweaks
- zmin = min(v.z for v in world_verts) - slice_verts = [v for v in world_verts if v.z <= zmin + slice_height] + if slice_height <= 0: + return {"error": f"slice_height must be positive (got {slice_height})"} + zmin = min(v.z for v in world_verts) + slice_verts = [v for v in world_verts if v.z <= zmin + slice_height] @@ - hit_loc, hit_normal, hit_idx, hit_dist = bvh.ray_cast(origin, direction) + hit_loc, _normal, _idx, _dist = bvh.ray_cast(origin, direction)🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@addon.py` around lines 574 - 589, Validate and guard slice_height before using it: ensure slice_height > 0 (either raise a ValueError or clamp small/non-positive values to a tiny epsilon) so zmin + slice_height isn't effectively equal to zmin or negative; then continue using zmin and slice_verts as before. Also update the bvh.ray_cast return handling in the loop to mark unused return values explicitly (e.g., use _, _ ,_ for hit_normal, hit_idx, hit_dist) and keep hit_loc for the gap calculation.src/blender_mcp/server.py (1)
362-404: Optional: normalize the error return shape to JSON.On the happy path the tool returns a JSON document, but on exception it returns a bare string like
"Error verifying grounding: ...". Callers that parse the response as JSON will fail on errors. Consider returningjson.dumps({"error": ...})for symmetry, or at minimum match the convention used elsewhere consistently. Matches existing tool style here, so it's optional.♻️ Optional refactor
except Exception as e: logger.error(f"Error verifying grounding: {str(e)}") - return f"Error verifying grounding: {str(e)}" + return json.dumps({"error": f"Error verifying grounding: {e}"})🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@src/blender_mcp/server.py` around lines 362 - 404, The exception path in verify_object_grounded returns a plain string but the success path returns JSON; change the error return to a JSON string (e.g. json.dumps({"error": str(e)})) so callers can always parse JSON, and keep the existing logger.error call (logger.error(f"Error verifying grounding: {e}")) for diagnostics; update the except block in verify_object_grounded (which calls get_blender_connection and blender.send_command) to return the JSON-encoded error object instead of a bare string.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@addon.py`:
- Around line 485-526: The camera data-object and scene object are created
before the try block which can leak bpy data-blocks if creation or linking
fails; move the calls to bpy.data.cameras.new("_MCPDiagCam"),
bpy.data.objects.new("_MCPDiagCam", cam_data) and
scene.collection.objects.link(cam_obj) into the try so that cam_data and cam_obj
are only created inside the protected block, and ensure the existing finally
still removes cam_obj and cam_data (using bpy.data.objects.remove and
bpy.data.cameras.remove with do_unlink=True) only if those variables are
non-None to avoid orphan leaks when creation/linking raised an exception.
- Line 508: The docstring/usage for the parameter named format is misleading:
scene.render.image_settings.file_format expects Blender enum names (e.g., "PNG",
"JPEG"), not lowercase aliases like "jpg"; update the docstring to state that
format must be a Blender image enum value OR implement normalization inside the
function by mapping common aliases (e.g., "jpg" -> "JPEG", "jpeg" -> "JPEG",
"tif" -> "TIFF", "png" -> "PNG") before assigning
scene.render.image_settings.file_format (use the format variable and the
scene.render.image_settings.file_format target to locate the assignment).
---
Nitpick comments:
In `@addon.py`:
- Around line 574-589: Validate and guard slice_height before using it: ensure
slice_height > 0 (either raise a ValueError or clamp small/non-positive values
to a tiny epsilon) so zmin + slice_height isn't effectively equal to zmin or
negative; then continue using zmin and slice_verts as before. Also update the
bvh.ray_cast return handling in the loop to mark unused return values explicitly
(e.g., use _, _ ,_ for hit_normal, hit_idx, hit_dist) and keep hit_loc for the
gap calculation.
In `@src/blender_mcp/server.py`:
- Around line 362-404: The exception path in verify_object_grounded returns a
plain string but the success path returns JSON; change the error return to a
JSON string (e.g. json.dumps({"error": str(e)})) so callers can always parse
JSON, and keep the existing logger.error call (logger.error(f"Error verifying
grounding: {e}")) for diagnostics; update the except block in
verify_object_grounded (which calls get_blender_connection and
blender.send_command) to return the JSON-encoded error object instead of a bare
string.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: 431a30aa-d712-4abf-8108-cfe97bcd16af
📒 Files selected for processing (2)
addon.pysrc/blender_mcp/server.py
…rchies When called on an EMPTY (typical Sketchfab/GLB import root with multi-mesh children parented to it), aggregate vertices from all descendant meshes instead of erroring out. Answers "is the imported model grounded?" in one call without forcing the caller to find the right leaf mesh. Verified on the test scene's Bench empty (children: footer_metal_0, screw_screw_0, wood_wood_0, wood.004_wood_0): 48/48 samples hit, median gap 4mm — matches manually traversing to find the lowest mesh, but now returns the list of meshes actually sampled in `sampled_meshes` for diagnostic transparency. Found while testing the original PR ahujasid#230 cherry-pick — most real assets in interior-design workflows are imported as hierarchies, so single-mesh constraint was a hard limit in practice.
Summary
When the MCP is used by a model (or a person driving it remotely), there is no deterministic way to confirm a visual claim.
object.location.zandobject.dimensionslie after asymmetric mesh edits — e.g., a hemisphere built by deleting the bottom of a sphere keeps the original origin, solocation.z - dimensions.z / 2is 15cm off from the actual base in world space. Displace modifiers on the ground vary z per position, so "put it on z=0" leaves a visible gap or intersection. The result is a feedback loop where the caller reports "fixed" and the user sees the bug.This PR adds two complementary tools, aimed at the same failure mode from different angles.
1.
verify_object_grounded(object_name, ground_name, slice_height=1.0, max_samples=500)Samples vertices from the object's lower slice — world z within
slice_heightof its lowest vertex — and raycasts straight down against the ground's evaluated mesh via BVH. Returns min/max/median/mean gap in meters:{ "object_name": "Tree.Pine01", "ground_name": "Ground", "samples_tested": 142, "samples_hit": 140, "samples_missed": 2, "min_gap": -0.042, "max_gap": 0.31, "median_gap": 0.11, "mean_gap": 0.13, "slice_height": 1.0, "hint": "mixed: tilted or uneven ground; base partly above and partly below" }Deterministic, no model vision. Evaluated geometry means Displace on the ground and armature / shape-key deformation on the object are honored automatically.
slice_heightis configurable because canopy-dominated meshes (pines, etc.) have most of their verts high up — a bounded z-slice is more reliable than a percentile of sorted verts.2.
get_viewport_screenshot(..., target_object, view, distance_factor, ortho_padding)When
target_objectis set, creates a throwaway orthographic camera framed on the target's world bbox, aimed from one of six named axes (front,back,left,right,top,bottom), and renders a clean image. The previousscene.camera, render resolution, output path, and image format are saved and restored after the render — the user's setup is untouched. The temp camera and its data block are removed withdo_unlink=Trueso no orphans accumulate.Default path (no
target_object) is unchanged: the existing viewport screenshot with overlays.Together these cover both sides of verification:
verify_object_groundedwhen you want a cheap, deterministic "is it touching?"get_viewport_screenshotwhen the question is broader — floating, wrong rotation, wrong color, wrong material.Test plan
verify_object_grounded('Cube', 'Plane')with a default cube on a plane →min_gap≈ 0,hint= "grounded or intersecting".min_gap≈ 0.5,hintstarts with "floating".min_gap≈ -0.1,hint= "grounded or intersecting" (all gaps below the 1cm threshold).hint= "mixed".verify_object_grounded('Missing', 'Plane')→ error message includes "not found".verify_object_grounded('Cube', 'EmptyObject')→ error message includes "is not a mesh".get_viewport_screenshot(target_object='Cube', view='front')→ returns a clean ortho render with the cube centered; user's viewport and scene camera unchanged.view='top',view='right', etc. — renders from the correct axis.viewvalue → error listing valid choices.get_viewport_screenshot()call (notarget_object) still returns the viewport-with-overlays screenshot as before._MCPDiagCamobject or camera data block remains in the scene.Notes
Downstream tracking bead:
Blender-tqy. Covers both "shapes" mentioned in the issue description as complementary tools rather than either/or.Summary by CodeRabbit