Files
linkerhand_retarget_ros2/src/camera_retarget_l20
2026-07-21 15:25:16 +08:00
..
2026-07-20 10:48:48 +08:00
2026-07-20 10:48:48 +08:00
2026-07-20 10:48:48 +08:00
2026-07-21 15:25:16 +08:00
2026-07-20 10:48:48 +08:00
2026-07-20 10:48:48 +08:00
2026-07-20 10:48:48 +08:00
2026-07-20 10:48:48 +08:00
2026-07-20 10:48:48 +08:00
2026-07-20 10:48:48 +08:00

Left O6/G20/L20 camera retargeting

Chinese guides:

This package bridges a webcam to physical left LinkerHand O6, G20, or L20 hands. It can drive an O6 and a G20 from the same detected human hand:

camera -> MediaPipe -> shared calibrated hand features -> O6 6-value + G20 20-value JointState

G20 and L20 both expose 20 position values, but they do not use the same CAN driver or the same left-hand joint directions. Select the matching model all the way from URDF mapping to the SDK driver.

Hardware output is disabled by default. G20 control also requires an explicit fully-open human-hand calibration before the E key can arm real control.

Physical left O6 and G20 together

The combined launch owns both CAN interfaces and isolates every SDK topic under the model name. Stop manually started linker_hand_sdk processes first, and make sure can0 and can1 are both UP at 1 Mbps:

ros2 launch camera_retarget_l20 teleop_o6_g20_left.launch.py

The fixed hardware assignment is G20 on can0 and O6 on can1. The relevant topics are:

/g20/cb_left_hand_control_cmd       20 positions
/g20/cb_left_hand_state             20 positions
/o6/cb_left_hand_control_cmd         6 positions
/o6/cb_left_hand_state               6 positions
/camera_o6_g20_retarget/g20/target_joint_cmd
/camera_o6_g20_retarget/o6/target_joint_cmd
/camera_o6_g20_retarget/status
/camera_o6_g20_retarget/debug

Real output remains disabled after startup. Dual mode uses speed 60, ema_alpha=0.55, and a 16-command-unit frame limit for normal responsive motion. It uses the complete measured G20 open/fist flexion endpoints and the full protocol range for G20 finger splay; O6 flexion is also allowed across the full 0..255 protocol range. These endpoint changes do not remove the per-frame rate limit, calibration gate, or tracking/driver timeout protection. Keep the physical hands clear and complete all three poses before pressing E. Each key starts an asynchronous capture: hold the requested pose for 1.5 seconds until at least 30 stable frames have been accepted. The service response means that capture started; completion/progress is reported on the status and debug topics:

  1. C: fully open hand with the thumb spread outward (zero pose).
  2. F: normal fist (finger and thumb flexion range).
  3. P: touch the thumb and index fingertips.

Press D to stop sending commands while holding the last pose. Equivalent service calls are:

ros2 service call /camera_o6_g20_retarget/calibrate_open std_srvs/srv/Trigger "{}"
ros2 service call /camera_o6_g20_retarget/calibrate_fist std_srvs/srv/Trigger "{}"
ros2 service call /camera_o6_g20_retarget/calibrate_pinch std_srvs/srv/Trigger "{}"
ros2 service call /camera_o6_g20_retarget/enable std_srvs/srv/SetBool "{data: true}"

Calibration is intentionally in-memory only. C/F/P must be repeated after every node restart. A capture waits up to eight seconds, rejects unstable windows using robust per-feature statistics, and never replaces the previous valid pose when validation fails. Starting any calibration automatically disarms real output and holds the current robot pose.

The direct robot mapping is loaded from config/direct_mapping_profiles.yml. This profile owns command lengths, joint names, measured open/fist endpoints, flexion weights, all four thumb opposition targets, fixed channels, and filter defaults. Existing ROS mapping parameters remain runtime overrides and are logged when they differ from the profile. The profile schema is model-oriented so a future L10 can reuse the same feature/adapter path after its physical endpoints have been measured.

The optional debug JSON contains raw bends and fingertip distances, normalized closure, the confirmed pinch target, blend strength, and raw versus filtered O6/G20 commands:

ros2 topic echo /camera_o6_g20_retarget/status
ros2 topic echo /camera_o6_g20_retarget/debug

If either driver state is absent for one second, or camera tracking is lost, the node commands the vendor open pose on both outputs and disarms real control. O6 maps thumb bend/side-swing and one bend actuator per finger; only the G20 has independent four-finger side-swing outputs.

Dependencies

Use a Python 3.12 virtual environment that can still see ROS 2 Jazzy packages. MediaPipe 0.10.21 is pinned because the newer package no longer exposes the legacy solutions.hands API used by this node. The dex-retargeting project requires NumPy 2, so install MediaPipe without dependency resolution:

cd /home/lxp/projects/linkerhand_retarget_ros2
python3 -m venv --system-site-packages .venv
source .venv/bin/activate
python -m pip install -e /home/lxp/projects/dex-retargeting
python -m pip install --no-deps mediapipe==0.10.21
source /opt/ros/jazzy/setup.bash
python -m colcon build --symlink-install --packages-select camera_retarget_l20 linker_hand_ros2_sdk
source install/setup.bash

On hosts where python3 -m venv reports that ensurepip is unavailable, create the environment with --without-pip, then use the system pip's --python option for the two installation commands:

python3 -m venv --without-pip --system-site-packages .venv
python3 -m pip --python .venv/bin/python install -e /home/lxp/projects/dex-retargeting
python3 -m pip --python .venv/bin/python install --no-deps mediapipe==0.10.21

The logged-in user must have camera access. Add lxp to the video group once, then log out and back in:

sudo usermod -aG video lxp

G20 preview only

ros2 launch camera_retarget_l20 camera_g20_left.launch.py

The target command can be inspected without moving hardware:

ros2 topic echo /camera_g20_retarget/target_joint_cmd
ros2 topic echo /camera_g20_retarget/status

Physical left G20

Stop any manually started linker_hand_sdk first; only one driver may own can0. Keep the hand clear of people and objects. The combined launch starts the G20 driver at speed 30 and moves it to the SDK startup pose, but camera output remains locked:

ros2 launch camera_retarget_l20 teleop_g20_left.launch.py

The default closing range is 95%. It can be reduced for a conservative test:

ros2 launch camera_retarget_l20 teleop_g20_left.launch.py max_closure:=0.60

Hold the real left hand fully open and steadily in view until the preview says hand=OK. Press C, keep the pose stable until the 1.5-second capture completes, then press E to enable control. The equivalent service calls are:

ros2 service call /camera_g20_retarget/calibrate_open std_srvs/srv/Trigger "{}"
ros2 service call /camera_g20_retarget/enable std_srvs/srv/SetBool "{data: true}"

Disable immediately with D or:

ros2 service call /camera_g20_retarget/enable std_srvs/srv/SetBool "{data: false}"

The G20 defaults limit closing travel to 95%, use speed 30 for both driver startup and camera motion, limit every frame to 8 command units, and open/disable the hand if tracking is lost for 0.35 seconds. Root and tip flexion is mapped from human joint angles. P calibrates the contact distance; at runtime the nearest of the index/middle/ring/pinky fingertips is selected and only that finger receives pinch coupling. Calibrated four-finger side-to-side motion maps to G20 channels 6 through 9. Splay confidence automatically fades toward the calibrated open pose when a curled finger is nearly perpendicular to the palm. Thumb CMC flexion is coupled to thumb-tip flexion. In dual C/F/P mode the C pose is the maximum outward thumb endpoint and the selected thumb/fingertip distance controls opposition. A new target must remain closest for three frames before it is selected; a margin and release threshold prevent index, middle, ring, and pinky from flickering between targets. Precision pinch is a continuous blend toward the configured robot pinch pose. A four-finger power grasp gradually suppresses this override from 65% closure and uses the four finger F endpoints at 90%. It no longer forces the thumb closed: thumb flexion uses the two measured thumb bends, while thumb side-swing follows the C-to-F thumb side-angle calibration. Legacy single-G20 mode retains its geometry-based side-swing without four-finger thumb assistance. G20 flexion no longer uses a common lower command or a special command-90 thumb clamp. Each of the ten flexion motors is interpolated independently between the measured open pose and the measured fist pose. In particular, a full dual-mode fist targets 50 on the thumb base, 144 on the thumb tip, and 0 on the eight index-through-pinky flexion channels. Single-G20 launches retain their configurable 95% closure multiplier.

L20 compatibility

The original L20 entry points remain available:

ros2 launch camera_retarget_l20 camera_l20_left.launch.py
ros2 launch camera_retarget_l20 teleop_l20_left.launch.py