Skip to content

fix(so_follower): make the kinematics joints guard reachable - #4915

Open
varun-projects wants to merge 1 commit into
huggingface:mainfrom
varun-projects:fix/kinematics-empty-joints-guard
Open

varun-projects wants to merge 1 commit into
huggingface:mainfrom
varun-projects:fix/kinematics-empty-joints-guard

Conversation

@varun-projects

@varun-projects varun-projects commented Oct 10, 2026 •

Copy link
Copy Markdown

Summary / Motivation

The four kinematics steps in robot_kinematic_processor.py build q_raw from the observation's <motor>.pos keys and then guard it with a condition that can never hold:

q_raw = np.array(
    [float(v) for k, v in observation.items() if isinstance(k, str) and k.endswith(".pos")],
    dtype=float,
)
if q_raw is None:          # np.array(...) is never None
    raise ValueError("Joints observation is require for computing robot kinematics")

So an observation carrying no joint keys reaches the body. Reproduced on main (no hardware):

GripperVelocityToJoint       -> IndexError: index -1 is out of bounds for axis 0 with size 0
InverseKinematicsEEToJoints  -> empty array handed to the solver
InverseKinematicsRLStep      -> empty array handed to the solver
EEReferenceAndDelta          -> empty array handed to FK

GripperVelocityToJoint is the clearest: q_raw[-1] + delta inside np.clip raises an IndexError that mentions neither joints, nor observations, nor which key was missing — while the class documents a ValueError for exactly this case.

That input is reachable rather than hypothetical. compute_forward_kinematics_joints_to_ee pops every <motor>.pos key, so a pipeline that maps the observation to EE space before one of these steps leaves q_raw empty; so does a policy observation keyed observation.state.

There is also a quieter failure: with 5 of 6 joint keys present, GripperVelocityToJoint integrates the wrong joint — q_raw[-1] is now wrist_roll, not gripper — and commands a bogus gripper position with no error at all. This PR does not change that (it depends on the ordering TODO at the same line, which is #4442's territory); it only makes the empty case loud.

Related issues

What changed

  • A new _assert_joints_present(q_raw, observation) helper raises the documented ValueError when the observation carried no joint positions, and names the keys that were present so the caller can see what it passed.
  • All four steps call it in place of the dead if q_raw is None check, rather than repeating the guard inline.
  • The helper keeps the is None case too, which EEReferenceAndDelta still needs: there q_raw can legitimately come from complementary_data["IK_solution"].
  • The message keeps the "Joints observation" prefix, so the existing fix(so_follower): check the observation for None before copying it #4255 test still matches, and fixes its "is require" typo.

How was this tested (or how to run locally)

New test_observation_without_joint_keys_raises_value_error, parametrised over all four steps, mirroring the existing test_missing_observation_raises_value_error block directly above it. No placo, no robot, no mocks — kinematics=None is enough, because the guard must fire before the solver is touched.

# before
FAILED test_observation_without_joint_keys_raises_value_error[ee_reference_and_delta]
FAILED test_observation_without_joint_keys_raises_value_error[inverse_kinematics_ee_to_joints]
FAILED test_observation_without_joint_keys_raises_value_error[gripper_velocity_to_joint]
FAILED test_observation_without_joint_keys_raises_value_error[inverse_kinematics_rl_step]
4 failed

# after
$ uv run pytest tests/robots/test_robot_kinematic_processor.py -q
10 passed            # 6 pre-existing + 4 new

$ uv run pytest tests/robots tests/teleoperators -q
147 passed, 4 skipped

Quality hooks on the changed files (ruff format, ruff, typos, pyupgrade, bandit, mypy, gitleaks) all pass.

Checklist (required before merge)

Reviewer notes

  • Worth checking: whether any caller today relies on an empty q_raw reaching the solver. I believe not — it produces an IndexError inside RobotKinematics.inverse_kinematics's joint loop anyway — but you would know of any in-flight work that depends on it.
  • The error message includes the observation's keys. If you would rather keep exception messages terse, I am happy to drop that.
  • The EEReferenceAndDelta case is the one asymmetry in the diff; I kept its is None branch deliberately rather than unifying all four, because IK_solution can genuinely be None there.
  • There is a stale open PR (Fix TypeError in Eagle2 text validation and AttributeError in kinematic processor #3037) touching these same functions; it only moves a .copy() and fixes the typo, and does not add an emptiness guard. Flagging it for conflict awareness.

AI assistance disclosure

Per AI_POLICY.md: found and written with AI assistance. I verified it myself: confirmed all four dead q_raw is None checks on main, executed each step with an EE-space observation to capture the actual IndexError/solver failures quoted above, confirmed compute_forward_kinematics_joints_to_ee pops the joint keys, and confirmed EEReferenceAndDelta can receive None from complementary data. Tests were run locally and verified to fail before the fix and pass after.

The four kinematics steps build `q_raw` from the observation's `<motor>.pos`
keys with `np.array(...)`, then guard it with `if q_raw is None`, which an
ndarray never is. An observation that carries no joint keys therefore reached
the body: `GripperVelocityToJoint` raised
`IndexError: index -1 is out of bounds for axis 0 with size 0` from inside
`np.clip`, and the IK steps handed an empty array to the solver.

That input is reachable, not hypothetical: `ForwardKinematicsJointsToEE*` pops
every `<motor>.pos` key, so a pipeline that maps the observation to EE space
before one of these steps leaves `q_raw` empty, as does a policy observation
keyed `observation.state`.

Check emptiness instead and name the keys that were found, so the documented
ValueError is surfaced. `EEReferenceAndDelta` keeps its `is None` check, since
there `q_raw` can come from `complementary_data["IK_solution"]`.

Follows huggingface#4255, which made the adjacent `observation is None` guard reachable
for the same reason.

The guard is factored into one `_assert_joints_present()` helper rather than
repeated at each of the four call sites.
@github-actions github-actions Bot added tests Problems with test coverage, failures, or improvements to testing robots Issues concerning robots HW interfaces labels Oct 10, 2026
@JonnyFaeth

Copy link
Copy Markdown

I checked three additional EEReferenceAndDelta cases on 82e11db: an empty observation, mixed non-string/EE keys, and an observation containing a .pos key for a motor outside motor_names.

Using a solver spy that raises if called, all three reach the solver on main b9cb121. On this PR they raise the intended ValueError before forward_kinematics is called. Combined with the existing kinematics tests, 13 cases pass on Windows CPU.

The unrelated-motor case also confirms the guard checks the selected joint array, not merely whether any .pos key exists. This uses a mocked solver; it does not validate numerical kinematics or robot operation. AI disclosure: the additional tests and this review were prepared with substantial assistance from OpenAI Codex.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

robots Issues concerning robots HW interfaces tests Problems with test coverage, failures, or improvements to testing

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants