Lê Văn Hưng – MSSV 20210417
Human-robot handover with prior-to-pass soft/rigid object classification
via tactile glove
1. Introduction
A handover is a basic tool for many human-to-human collaboration tasks, it is a sequence of
coordinated joint actions that help a passer directly transfer an object to a receiver. Replacing a
passer or receiver with an autonomous robot would bring great opportunities for robotic
assistance. However, it is a challenging task. Human–robot handover (HRH) is the object of this
paper.
2. State of the art
This section provides an overview of the current technologies for the pre-handover phase and for
tactile object detection, these two aspects are central for developing methods. The pre-handover
phase always preceded the handover phase and includes two main approaches which are the
online approach and the offline approach. Tactile gloves aim to provide information through
their sensors, it can be stretchable and flexible to precisely cover the human hand.
3. The proposed HRH pipeline
This section demonstrates the block diagram of the proposed HRH pipeline which includes 5
stages. The first stage is Object classification, the classifier recognizes if the object picked up is
soft or rigid. The next stage is Pose estimation, which contains determining the handover
location, but changes based on the object position (which means applying an online approach).
The third is Trajectory calculation, when the handover location has been determined based on the
object position, the manipulator will plan a trajectory from its current configuration to a point
that is close to the position with proper orientation. The fourth is Handover, the grasping phase
starts when the robot hand has reached the object transfer located ion, the physical transfer
occurs when the receiver makes the first contact with the object. The final stage is Retreat, the
robot calculates the trajectory to return to default configuration by a motion planning algorithm,
then moves its end effector above a box and drops the object, two boxes are present for soft and
rigid objects.
4. Experimental setup
This section describes the hardware and software components of the experimental setup. For
robot and vision sensors, the manipulator used a torque-controlled Franka Emika Panda. This
robot is suitable for safe physical human–robot interaction. The tactile glove was interfaced with
the computer via ROS, it can measure pressure distribution and mechanical vibrations at the
points of contact. The objects are chosen from 12 deformable objects and 12 stiff objects of
different sizes.
5. DL for tactile object classification
This section provides a description of the case study of the training and testing of Deep Learning
(DL) models for soft/rigid object classification. In the experimental procedure, experiments were
conducted with four students and, each participant holds the same 18 objects, there were 360
trials (9 objects×4 people×10 grasps) per class in total. For the DL models and Classification
Accuracy, the training dataset result was split into two separate subsets, for training and
development. The result for sensing modalities shows some differences compared to previous
research, a CNN model was built. As expected, the performance in the case of pressure sensors
only was still superior.
6. Experimental results on HRH
The experiment for each participant and each object includes three phases. Thresholds
Determination results are shown in Table 2, which reports the percentage of experiments that all
rigid or soft objects either dropped or deformed. The results of the pre-handover object
classification during the HRH experiments are shown in Table 3. As expected, the accuracy in
Table 1 is higher than in Table 3. Quantitative results on collaboration fluency noticed that the
robot hand uses a constant total force when holding the object. Finally the Qualitative results, the
goal was to determine the overall perception of the HRH experiment.
7. Conclusions and outlook
This paper discussed an HRH pipeline with the use of a tactile glove to classify rigid or
deformable objects before handover. The results on 8 participants and 24 objects indicate that the
proposed approach has an opportunity for real-world HRH situations. The questionnaire results
provide a positive perception of the HRH process. Therefore, it is definitely possible to execute
HRH with different robot grasping forces for soft and rigid objects.
