Repository navigation
fix(so_follower): make the kinematics joints guard reachable - #4915
varun-projects wants to merge 1 commit into
Conversation
The four kinematics steps build `q_raw` from the observation's `<motor>.pos` keys with `np.array(...)`, then guard it with `if q_raw is None`, which an ndarray never is. An observation that carries no joint keys therefore reached the body: `GripperVelocityToJoint` raised `IndexError: index -1 is out of bounds for axis 0 with size 0` from inside `np.clip`, and the IK steps handed an empty array to the solver. That input is reachable, not hypothetical: `ForwardKinematicsJointsToEE*` pops every `<motor>.pos` key, so a pipeline that maps the observation to EE space before one of these steps leaves `q_raw` empty, as does a policy observation keyed `observation.state`. Check emptiness instead and name the keys that were found, so the documented ValueError is surfaced. `EEReferenceAndDelta` keeps its `is None` check, since there `q_raw` can come from `complementary_data["IK_solution"]`. Follows huggingface#4255, which made the adjacent `observation is None` guard reachable for the same reason. The guard is factored into one `_assert_joints_present()` helper rather than repeated at each of the four call sites.
|
I checked three additional EEReferenceAndDelta cases on 82e11db: an empty observation, mixed non-string/EE keys, and an observation containing a .pos key for a motor outside motor_names. Using a solver spy that raises if called, all three reach the solver on main b9cb121. On this PR they raise the intended ValueError before forward_kinematics is called. Combined with the existing kinematics tests, 13 cases pass on Windows CPU. The unrelated-motor case also confirms the guard checks the selected joint array, not merely whether any .pos key exists. This uses a mocked solver; it does not validate numerical kinematics or robot operation. AI disclosure: the additional tests and this review were prepared with substantial assistance from OpenAI Codex. |
Summary / Motivation
The four kinematics steps in
robot_kinematic_processor.pybuildq_rawfrom the observation's<motor>.poskeys and then guard it with a condition that can never hold:So an observation carrying no joint keys reaches the body. Reproduced on
main(no hardware):GripperVelocityToJointis the clearest:q_raw[-1] + deltainsidenp.clipraises anIndexErrorthat mentions neither joints, nor observations, nor which key was missing — while the class documents aValueErrorfor exactly this case.That input is reachable rather than hypothetical.
compute_forward_kinematics_joints_to_eepops every<motor>.poskey, so a pipeline that maps the observation to EE space before one of these steps leavesq_rawempty; so does a policy observation keyedobservation.state.There is also a quieter failure: with 5 of 6 joint keys present,
GripperVelocityToJointintegrates the wrong joint —q_raw[-1]is nowwrist_roll, notgripper— and commands a bogus gripper position with no error at all. This PR does not change that (it depends on the ordering TODO at the same line, which is #4442's territory); it only makes the empty case loud.Related issues
observation is Noneguard reachable for precisely the same reason — its test docstring reads "the guard below them was unreachable and an AttributeError escaped instead." This is the same fix shape for the neighbouring condition..posordering assumption, deliberately untouched here.What changed
_assert_joints_present(q_raw, observation)helper raises the documentedValueErrorwhen the observation carried no joint positions, and names the keys that were present so the caller can see what it passed.if q_raw is Nonecheck, rather than repeating the guard inline.is Nonecase too, whichEEReferenceAndDeltastill needs: thereq_rawcan legitimately come fromcomplementary_data["IK_solution"]."Joints observation"prefix, so the existing fix(so_follower): check the observation for None before copying it #4255 test still matches, and fixes its "is require" typo.How was this tested (or how to run locally)
New
test_observation_without_joint_keys_raises_value_error, parametrised over all four steps, mirroring the existingtest_missing_observation_raises_value_errorblock directly above it. No placo, no robot, no mocks —kinematics=Noneis enough, because the guard must fire before the solver is touched.Quality hooks on the changed files (ruff format, ruff, typos, pyupgrade, bandit, mypy, gitleaks) all pass.
Checklist (required before merge)
pre-commit runon the changed files — all hooks pass)tests/robots,tests/teleoperators). Fullpytestnot run: CPU-only sandbox.Reviewer notes
q_rawreaching the solver. I believe not — it produces anIndexErrorinsideRobotKinematics.inverse_kinematics's joint loop anyway — but you would know of any in-flight work that depends on it.EEReferenceAndDeltacase is the one asymmetry in the diff; I kept itsis Nonebranch deliberately rather than unifying all four, becauseIK_solutioncan genuinely beNonethere..copy()and fixes the typo, and does not add an emptiness guard. Flagging it for conflict awareness.AI assistance disclosure
Per AI_POLICY.md: found and written with AI assistance. I verified it myself: confirmed all four dead
q_raw is Nonechecks onmain, executed each step with an EE-space observation to capture the actualIndexError/solver failures quoted above, confirmedcompute_forward_kinematics_joints_to_eepops the joint keys, and confirmedEEReferenceAndDeltacan receiveNonefrom complementary data. Tests were run locally and verified to fail before the fix and pass after.