Initial snapshot: hand motion pipeline (Dyn-HaMR + dex-retargeting + SPIDER)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

third_party/spider/docs/workflows/workflow-dexmachina.md

@@ -0,0 +1,175 @@
+# Workflow: DexMachina (Genesis)
+
+The DexMachina workflow integrates SPIDER with [Genesis](https://genesis.github.io/) simulator and [DexMachina](https://github.com/MandiZhao/dexmachina) framework for dexterous manipulation with downstream RL training.
+
+## Quick Start (Reproducible Setup)
+
+### 1. Download Data
+
+All required data is hosted on HuggingFace. Download it into the spider directory:
+
+```bash
+sudo apt install git-lfs
+git lfs install
+cd /path/to/spider
+git clone https://huggingface.co/datasets/retarget/retarget_example example_datasets
+```
+
+### 2. Install DexMachina Environment
+
+```bash
+# Create conda environment
+conda create -n dexmachina python=3.10
+conda activate dexmachina
+
+# Install PyTorch
+pip install torch==2.5.1
+
+# Clone and install the Genesis fork (use the 'older' branch for API compatibility)
+git clone https://github.com/MandiZhao/Genesis.git
+cd Genesis
+git checkout older
+pip install -e .
+pip install libigl==2.5.1  # Required: fixes igl.signed_distance API mismatch
+cd ..
+
+# Clone and install rl_games fork
+git clone https://github.com/MandiZhao/rl_games.git
+cd rl_games
+pip install -e .
+cd ..
+
+# Clone and install dexmachina
+git clone https://github.com/MandiZhao/dexmachina.git
+cd dexmachina
+pip install -e .
+cd ..
+
+# Install additional packages for video export and RL training
+pip install moviepy==1.0.3 gymnasium ray seaborn
+```
+
+### 3. Install SPIDER (Minimal)
+
+Install SPIDER without MuJoCo Warp (only need optimization components):
+
+```bash
+cd /path/to/spider
+conda activate dexmachina
+
+# Install SPIDER without dependencies (DexMachina has its own)
+pip install --ignore-requires-python --no-deps -e .
+# Install other dependencies
+pip install loguru tyro hydra-core rerun-sdk==0.26.2
+```
+
+### 4. Copy Pre-Generated Data to DexMachina Assets
+
+Pre-generated inspire_hand data (7 clips) is stored in the HuggingFace dataset so you don't need to re-run the retargeting pipeline.
+
+```
+example_datasets/raw/dexmachina/
+  contact_retarget/inspire_hand/s01/   # contact mapping (.npy) + videos (.mp4)
+  retargeted/inspire_hand/s01/         # retargeted kinematics (.pt)
+  retargeter_results/inspire_hand/s01/ # retargeter results (.npy)
+```
+
+Clips: `box_use_01`, `ketchup_use_01`, `ketchup_use_02`, `laptop_use_01`, `mixer_use_01`, `notebook_use_01`, `waffleiron_use_01`
+
+After installing DexMachina, copy the data into its assets directory:
+
+```bash
+DEXMACHINA_ASSETS=$(python -c "from dexmachina.asset_utils import get_asset_path; print(get_asset_path(''))")
+SPIDER_DATA=/path/to/spider/example_datasets/raw/dexmachina
+
+cp -r $SPIDER_DATA/retargeted/inspire_hand/        $DEXMACHINA_ASSETS/retargeted/inspire_hand/
+cp -r $SPIDER_DATA/contact_retarget/inspire_hand/   $DEXMACHINA_ASSETS/contact_retarget/inspire_hand/
+cp -r $SPIDER_DATA/retargeter_results/inspire_hand/  $DEXMACHINA_ASSETS/retargeter_results/inspire_hand/
+```
+
+If the data is missing at runtime, the error message will print the exact copy commands for your paths.
+
+### 5. Run SPIDER
+
+```bash
+conda activate dexmachina
+cd /path/to/spider
+
+# Run with default config (inspire hand, box-30-230 task)
+python examples/run_dexmachina.py
+```
+
+This will:
+1. Initialize Genesis environment with DexMachina task
+2. Load reference motion from task
+3. Optimize trajectory with Sampling
+4. Save optimized trajectory and video
+
+Output is saved to `example_datasets/processed/arctic/inspire_hand/bimanual/{task}/{data_id}/`:
+- `trajectory_dexmachina.npz` --- trajectory data
+- `trajectory_dexmachina.mp4` --- video
+
+## Evaluate Trajectories
+
+After running `run_dexmachina.py`, evaluate object tracking metrics (position, rotation, articulation distances) with:
+
+```bash
+conda activate dexmachina
+cd /path/to/spider
+
+# Evaluate all tasks under the default data directory
+python spider/postprocess/evaluate_dexmachina.py
+
+# Evaluate specific tasks
+python spider/postprocess/evaluate_dexmachina.py --tasks box-30-230 laptop-30-230
+
+# Use a different dataset/robot configuration
+python spider/postprocess/evaluate_dexmachina.py --dataset_name arctic --robot_type inspire_hand --embodiment_type bimanual
+```
+
+The script uses the same dataset path resolution as `config.py` (`dataset_dir/processed/{dataset_name}/{robot_type}/{embodiment_type}/{task}/{data_id}/`). You can also pass `--data_dir` to override with an explicit path.
+
+## Compare SPIDER vs RL (Multi-Seed)
+
+To compare SPIDER (MPC) against RL rollouts across multiple seeds, use `--compare` mode. This loads both `trajectory_dexmachina.npz` (SPIDER) and `rollout_rl.npz` (RL) from each `{task}/{seed}/` directory and prints mean +/- std tables.
+
+```bash
+# Compare across seeds 0-4 (default), print Markdown tables
+python spider/postprocess/evaluate_dexmachina.py --compare
+
+# Compare with LaTeX tables and per-seed detail
+python spider/postprocess/evaluate_dexmachina.py --compare --latex --detail
+
+# Compare specific tasks / seeds
+python spider/postprocess/evaluate_dexmachina.py --compare --tasks box-30-230 laptop-30-230 --seeds 0 1 2
+```
+
+## Configuration
+
+Key parameters in `examples/config/dexmachina.yaml`:
+
+| Parameter | Default | Description |
+|-----------|---------|-------------|
+| `task` | `box-30-230` | Object task (format: `object_name-start_frame-end_frame`) |
+| `robot_type` | `inspire_hand` | Dexterous hand model |
+| `embodiment_type` | `bimanual` | Hand configuration |
+| `num_samples` | `1024` | Parallel CEM samples (Genesis envs) |
+| `max_num_iterations` | `32` | CEM iterations per MPC step |
+| `horizon` | `0.40` | Planning horizon (seconds) |
+| `ctrl_dt` | `0.20` | Control timestep |
+
+## Common Issues
+
+1. **Missing DexMachina data**: Copy pre-generated data from `example_datasets/raw/dexmachina/` into the DexMachina assets directory (see "Copy Pre-Generated Data" section above).
+
+2. **Genesis branch compatibility**: You must use the `older` branch of the Genesis fork (`git checkout older`). The `main` and `0122` branches have upstream merges that break API compatibility with DexMachina (e.g., `set_dofs_kp` input shape broadcasting, `inertial_quat` None handling).
+
+3. **`igl.signed_distance` unpacking error** (`ValueError: too many values to unpack`): Install `libigl==2.5.1` explicitly. Newer versions of libigl change the return signature.
+
+4. **`moviepy.editor` not found**: Install `moviepy==1.0.3` (not the newer 2.x which removed `moviepy.editor`). This is needed for video export.
+
+5. **torch weights loading error**: If failed to load trajectory from dexmachina, ensure the data is loaded with `torch.load(data_fname, weights_only=False)`.
+
+6. **numpy version**: Use `numpy==1.26.4` to avoid compatibility issues with Genesis and taichi.
+
+7. **First run is slow**: The first run compiles taichi kernels (JIT) which can appear to hang for several minutes with no output. Subsequent runs use cached kernels and are faster.

third_party/spider/docs/workflows/workflow-hdmi.md

@@ -0,0 +1,46 @@
+# Workflow: HDMI (Humanoid-Object Interaction)
+
+The HDMI workflow integrates SPIDER with [HDMI (Humanoid Manipulation)](https://github.com/lecar-lab/hdmi) framework for humanoid robot manipulation with downstream sim2real RL training.
+
+## Prerequisites
+
+### Install HDMI Environment
+
+Follow the [official HDMI installation guide](https://github.com/lecar-lab/hdmi):
+
+```bash
+# Clone HDMI repository
+git clone https://github.com/lecar-lab/hdmi.git
+cd hdmi
+
+# Install with uv (recommended)
+uv sync
+
+# Install SPIDER into HDMI environment
+uv pip install --no-deps -e /path/to/spider
+```
+
+## Running HDMI Workflow
+
+### Basic Example
+
+```bash
+cd /path/to/spider
+
+# Run with default config (G1 humanoid, suitcase task)
+python examples/run_hdmi.py
+# OR refer to
+./examples/run_hdmi.sh
+```
+
+### Configuration Options
+
+You can enable `mjlab` native viewer by setting `viewer` to `hdmi`.
+
+## Train RL
+
+Write the motion as one HDMI reference motion by running the following command:
+
+```bash
+python examples/postprocess/read_to_hdmi.py
+```

third_party/spider/docs/workflows/workflow-maniptrans.md

@@ -0,0 +1,227 @@
+# Workflow: ManipTrans (IsaacGym)
+
+SPIDER MPC for dexterous hand-object manipulation using [IsaacGym](https://developer.nvidia.com/isaac-gym) and [HO-Tracker-Baseline](https://github.com/KailinLi/HO-Tracker-Baseline) (ManipTrans).
+
+## Quick Start (Reproducible Setup)
+
+### 1. Download Data
+
+All required data is hosted on HuggingFace. Download it into the spider directory:
+
+```bash
+sudo apt install git-lfs
+git lfs install
+cd /path/to/spider
+git clone https://huggingface.co/datasets/retarget/retarget_example example_datasets
+```
+
+This downloads `example_datasets/` containing:
+- `raw/maniptrans/HO-Tracker/` --- 16 trajectory demos (8 left-hand, 8 right-hand)
+- `raw/maniptrans/assets/` --- pre-trained base imitator checkpoints
+- `raw/maniptrans/rl_runs/` --- 16 trained RL baseline checkpoints
+- `raw/maniptrans/retargeting/` --- optional retargeting initialization
+- `processed/maniptrans/` --- pre-computed rollouts, videos, and metrics (160 runs)
+
+### 2. Create Data Symlink
+
+ManipTrans uses relative paths (`data/HO-Tracker/...`). Create a symlink:
+
+```bash
+cd /path/to/spider
+ln -sf example_datasets/raw/maniptrans data
+```
+
+### 3. Install Dependencies
+
+Requires Python 3.8 + PyTorch 1.13.1 + IsaacGym (older than SPIDER's default Python 3.12+):
+
+```bash
+# Create conda environment
+conda create -n maniptrans python=3.8
+conda activate maniptrans
+
+# Install PyTorch 1.13.1 with CUDA 11.7
+pip install torch==1.13.1+cu117 torchvision==0.14.1+cu117 --extra-index-url https://download.pytorch.org/whl/cu117
+
+# Install IsaacGym (requires NVIDIA GPU)
+cd /path/to/isaacgym/python && pip install -e .
+
+# Install pytorch3d (prebuilt wheel for py38 + cu117 + pytorch 1.13.1)
+pip install pytorch3d==0.7.3 -f https://dl.fbaipublicfiles.com/pytorch3d/packaging/wheels/py38_cu117_pyt1131/download.html
+
+# Install additional dependencies
+pip install git+https://github.com/otaheri/chamfer_distance.git
+pip install git+https://github.com/otaheri/bps_torch.git
+pip install git+https://github.com/lixiny/manotorch.git
+pip install numpy==1.23.5 gym==0.23.1 opencv-python-headless scikit-learn smplx trimesh
+
+# Clone and install HO-Tracker-Baseline
+cd /path/to
+git clone https://github.com/KailinLi/HO-Tracker-Baseline.git
+cd HO-Tracker-Baseline
+touch requirements.txt  # Create empty file (prevents pip install error)
+pip install -e .
+
+# Install SPIDER (minimal, no-deps to avoid conflicts)
+cd /path/to/spider
+pip install --ignore-requires-python --no-deps -e .
+pip install loguru mujoco hydra-core omegaconf imageio[ffmpeg]
+```
+
+### 4. Set Up HO-Tracker-Baseline Data
+
+HO-Tracker-Baseline also needs access to the trajectory data and imitator checkpoints. Point its data paths to spider's `example_datasets/`:
+
+```bash
+# Link HO-Tracker data
+cd /path/to/HO-Tracker-Baseline
+rm -f data/HO-Tracker
+ln -sf /path/to/spider/example_datasets/raw/maniptrans/HO-Tracker data/HO-Tracker
+
+# Link retargeting data
+rm -rf data/retargeting/HO-Tracker
+ln -sf /path/to/spider/example_datasets/raw/maniptrans/retargeting/HO-Tracker data/retargeting/HO-Tracker
+
+# Copy imitator checkpoints
+cp /path/to/spider/example_datasets/raw/maniptrans/assets/imitator_*.pth assets/
+```
+
+### 5. Patch HO-Tracker-Baseline for Compatibility
+
+The `dexhandimitator` module may fail to import in some environments. Patch `maniptrans_envs/lib/envs/tasks/__init__.py` to make it optional:
+
+```python
+# Replace:
+from .dexhandimitator import DexHandImitatorRHEnv, DexHandImitatorLHEnv
+
+# With:
+try:
+    from .dexhandimitator import DexHandImitatorRHEnv, DexHandImitatorLHEnv
+except ImportError:
+    DexHandImitatorRHEnv = None
+    DexHandImitatorLHEnv = None
+```
+
+### 6. Run SPIDER
+
+```bash
+conda activate maniptrans
+cd /path/to/spider
+export LD_LIBRARY_PATH="/path/to/miniconda3/envs/maniptrans/lib:${LD_LIBRARY_PATH:-}"
+
+# Default: 0f900@0, inspire hand, left hand
+python examples/run_maniptrans.py
+
+# Different trajectory:
+python examples/run_maniptrans.py task=81a95@5
+
+# Right hand:
+python examples/run_maniptrans.py task=07bb1@1 embodiment_type=right
+
+# Fewer iterations (faster):
+python examples/run_maniptrans.py max_num_iterations=16
+```
+
+Output is saved to `example_datasets/processed/maniptrans/inspire/{side}/{task}/{data_id}/`:
+- `rollout_isaac.npz` --- trajectory data
+- `rollout_isaac.mp4` --- video (when `save_video=true`)
+- `metrics_isaac.json` --- evaluation metrics
+
+## Available Trajectories
+
+16 trajectories from HO-Tracker dataset (8 per hand):
+
+| Side | Task IDs |
+|------|----------|
+| Left | `0f900@0`, `81a95@5`, `0f4a7@1`, `70ca2@0`, `39e5e@5`, `380c7@0`, `2ddc8@2`, `0b3a0@6` |
+| Right | `07bb1@1`, `235ac@0`, `07301@0`, `49793@0`, `01857@10`, `031b7@0`, `03ac9@0`, `155cb@0` |
+
+## Configuration
+
+Key parameters in `examples/config/maniptrans.yaml`:
+
+| Parameter | Default | Description |
+|-----------|---------|-------------|
+| `task` | `0f900@0` | HO-Tracker data sequence |
+| `embodiment_type` | `left` | `left` or `right` |
+| `num_samples` | `1024` | Parallel CEM samples (IsaacGym envs) |
+| `max_num_iterations` | `32` | CEM iterations per MPC step |
+| `horizon` | `0.6` | Planning horizon (seconds) |
+| `ctrl_dt` | `0.2` | Control timestep |
+| `seed` | `0` | Random seed |
+| `data_id` | `0` | Output subdirectory index |
+
+## RL Baseline Evaluation
+
+Trained RL checkpoints are included for all 16 trajectories. To evaluate:
+
+```bash
+cd /path/to/HO-Tracker-Baseline
+
+# Example: evaluate left-hand trajectory
+python main/rl/train.py \
+    task=ResDexHand dexhand=inspire side=lh \
+    headless=true num_envs=256 test=true \
+    +forceFullLength=true seed=0 \
+    rolloutStateInit=false randomStateInit=false \
+    "dataIndices=[0f900@0]" \
+    checkpoint=/path/to/spider/example_datasets/raw/maniptrans/rl_runs/baseline_0f900@0_inspire_lh/nn/baseline_0f900@0.pth \
+    actionsMovingAverage=0.4 \
+    rh_base_model_checkpoint=assets/imitator_rh_inspire.pth \
+    lh_base_model_checkpoint=assets/imitator_lh_inspire.pth
+```
+
+Side tags: `lh` for left-hand trajectories, `rh` for right-hand trajectories.
+
+## Evaluation Metrics
+
+Four metrics computed against ground-truth demo data:
+
+| Metric | Description |
+|--------|-------------|
+| `e_t` | Object translation error (cm) |
+| `e_r` | Object rotation error (degrees, geodesic) |
+| `e_j` | Joint position error (cm) |
+| `e_ft` | Fingertip position error (cm) |
+
+Run evaluation across all trajectories:
+
+```bash
+# Single-seed evaluation (data_id=0 only)
+python spider/postprocess/evaluate_maniptrans.py
+
+# Multi-seed evaluation (all data_ids, mean +/- std)
+python spider/postprocess/evaluate_maniptrans.py --multiseed
+```
+
+## Common Issues
+
+1. **DexHandFactory registration failure** (`KeyError: 'inspire_rh'`): This is a circular import issue. SPIDER includes a fix in `spider/simulators/maniptrans.py` that lazily triggers hand auto-registration. Ensure you are using the latest SPIDER code.
+
+2. **`dexhandimitator` import error**: Apply the patch in step 5 above to make this import optional.
+
+3. **pytorch3d installation**: Use the prebuilt wheel for your PyTorch/CUDA combination. Building from source requires matching CUDA toolkit version (not just driver) and compatible GCC. The command in step 3 uses prebuilt wheels for py38 + cu117 + pytorch 1.13.1.
+
+4. **`np.float` deprecation** (`AttributeError: module 'numpy' has no attribute 'float'`): Install `numpy==1.23.5` which still supports the deprecated aliases.
+
+5. **Missing `requirements.txt` in HO-Tracker-Baseline**: Run `touch requirements.txt` in the HO-Tracker-Baseline root before `pip install -e .`.
+
+6. **`imageio[ffmpeg]` required for video**: Install with `pip install imageio[ffmpeg]` for video recording support.
+
+7. **`LD_LIBRARY_PATH` must include conda env lib**: Without this, IsaacGym may fail to find shared libraries. Set `export LD_LIBRARY_PATH="/path/to/miniconda3/envs/maniptrans/lib:${LD_LIBRARY_PATH:-}"`.
+
+## Important Notes
+
+### Python 3.8 Compatibility
+
+ManipTrans requires Python 3.8 + PyTorch 1.13.1 + IsaacGym. SPIDER handles this via:
+- `from __future__ import annotations` in `spider/config.py` (for `tuple[float, float]` syntax)
+- `hasattr(torch, "compile")` guards in `spider/optimizers/sampling.py`
+
+### Action Space
+
+SPIDER optimizes 18-dim residual actions (6 wrist force/torque + 12 DOF positions). The base imitator provides nominal tracking actions. These combine in the env's `pre_physics_step` to produce 36-dim actions for the full base+residual architecture.
+
+### ManipTrans Config Fields
+
+ManipTrans-specific fields (e.g., `maniptrans_episode_length`) are **not** part of the SPIDER `Config` dataclass. In `examples/run_maniptrans.py`, they are extracted from the Hydra config and attached as attributes after `Config` construction.

third_party/spider/docs/workflows/workflow-mjwp.md

@@ -0,0 +1,317 @@
+# Workflow: MuJoCo Warp (MJWP)
+
+The MuJoCo Warp (MJWP) workflow is SPIDER's default and most versatile pipeline, supporting both dexterous hand and humanoid robot retargeting with high-performance physics simulation.
+
+## Overview
+
+The MJWP workflow uses MuJoCo with Warp backend for GPU-accelerated physics simulation. It provides the complete pipeline from raw human motion data to physics-based optimized robot trajectories.
+
+**Supported Embodiments:**
+- Dexterous hands (Allegro, Inspire, Shadow, Xhand, etc.)
+- Humanoid robots (G1, H1, T1, etc.)
+- Bimanual configurations
+
+
+## Workflow Steps
+
+```mermaid
+graph LR
+    A[Raw Data] --> B[Process Dataset]
+    B --> C[Decompose Mesh]
+    C --> D[Detect Contact]
+    D --> E[Generate Scene]
+    E --> F[IK Retargeting]
+    F --> G[Physics Optimization]
+    G --> H[Output Trajectory]
+```
+
+### Step 1: Setup Environment Variables
+
+Define task parameters:
+
+```bash
+export TASK=pick_spoon_bowl
+export HAND_TYPE=bimanual
+export DATA_ID=0
+export ROBOT_TYPE=allegro
+export DATASET_NAME=oakink
+```
+
+### Step 2: Process Dataset
+
+Convert raw human motion data to standardized format:
+
+```bash
+uv run spider/process_datasets/oakink.py \
+  --task=${TASK} \
+  --embodiment-type=${HAND_TYPE} \
+  --data-id=${DATA_ID}
+```
+
+**What it does:**
+- Reads raw motion capture data (MANO poses, object poses)
+- Extracts hand kinematics (joint positions, wrist poses)
+- Saves processed data in standardized format
+
+**Output:**
+```
+example_datasets/processed/oakink/mano/bimanual/pick_spoon_bowl/0/
+├── trajectory_keypoint.npz
+└── info.json
+```
+
+::: tip Dataset-Specific Processors
+Replace `oakink.py` with the appropriate processor for your dataset:
+- `gigahand.py` - GigaHand dataset
+- `hot3d.py` - Hot3D dataset
+- `fair_fre.py` - FAIR FRE dataset
+:::
+
+### Step 3: Decompose Object Mesh
+
+Create convex decomposition for efficient collision detection:
+
+```bash
+uv run spider/preprocess/decompose_fast.py \
+  --task=${TASK} \
+  --dataset-name=${DATASET_NAME} \
+  --data-id=${DATA_ID} \
+  --embodiment-type=${HAND_TYPE}
+```
+
+If you want higher quality convex decomposition, you can use the following command:
+```bash
+uv run spider/preprocess/decompose.py \
+  --task=${TASK} \
+  --dataset-name=${DATASET_NAME} \
+  --data-id=${DATA_ID} \
+  --embodiment-type=${HAND_TYPE}
+```
+
+**What it does:**
+- Loads object mesh from dataset
+- Performs convex decomposition using V-HACD or CoACD
+- Saves decomposed mesh components
+
+**Output:**
+```
+example_datasets/processed/oakink/assets/objects/spoon/convex/
+├── 0.obj
+├── 1.obj
+├── 2.obj
+└── visual.obj
+```
+
+::: warning
+Convex decomposition quality significantly affects simulation accuracy. Complex objects may need manual tuning of decomposition parameters.
+:::
+
+### Step 4: Detect Contact (Optional)
+
+Identify hand-object contact points for contact-aware retargeting:
+
+```bash
+uv run spider/preprocess/detect_contact.py \
+  --task=${TASK} \
+  --dataset-name=${DATASET_NAME} \
+  --data-id=${DATA_ID} \
+  --embodiment-type=${HAND_TYPE}
+```
+
+**What it does:**
+- Analyzes hand-object proximity throughout trajectory
+- Identifies contact points on hand surface
+- Saves contact information for trajectory optimization
+
+**Output:**
+- Updates `trajectory_keypoint.npz` with contact data
+
+::: tip
+Contact detection helps maintain grasp stability during retargeting, especially for manipulation tasks.
+:::
+
+### Step 5: Generate Scene
+
+Create MuJoCo scene XML with robot and objects:
+
+```bash
+uv run spider/preprocess/generate_xml.py \
+  --task=${TASK} \
+  --dataset-name=${DATASET_NAME} \
+  --data-id=${DATA_ID} \
+  --embodiment-type=${HAND_TYPE} \
+  --robot-type=${ROBOT_TYPE}
+```
+
+**What it does:**
+- Loads robot URDF/XML configuration
+- Creates scene with objects from dataset
+- Sets up collision pairs and contact parameters
+- Configures camera and lighting
+
+**Output:**
+```
+example_datasets/processed/oakink/allegro/bimanual/pick_spoon_bowl/0/
+└── scene.xml
+```
+
+**Scene structure:**
+```xml
+<mujoco>
+  <compiler angle="radian" meshdir="assets"/>
+  <option timestep="0.01"/>
+
+  <!-- Robot -->
+  <worldbody>
+    <body name="left_hand">...</body>
+    <body name="right_hand">...</body>
+
+    <!-- Objects -->
+    <body name="spoon">...</body>
+    <body name="bowl">...</body>
+  </worldbody>
+
+  <!-- Contact pairs -->
+  <contact>
+    <pair geom1="left_index_tip" geom2="spoon"/>
+    ...
+  </contact>
+</mujoco>
+```
+
+### Step 6: Inverse Kinematics (IK)
+
+A minimum IK scripts based on mink can be used by:
+
+```bash
+uv run spider/preprocess/ik_fast.py \
+  --task=${TASK} \
+  --dataset-name=${DATASET_NAME} \
+  --data-id=${DATA_ID} \
+  --embodiment-type=${HAND_TYPE} \
+  --robot-type=${ROBOT_TYPE}
+```
+
+You can also use the original IK script with mujoco by:
+
+```bash
+uv run spider/preprocess/ik.py \
+  --task=${TASK} \
+  --dataset-name=${DATASET_NAME} \
+  --data-id=${DATA_ID} \
+  --embodiment-type=${HAND_TYPE} \
+  --robot-type=${ROBOT_TYPE} \
+  --open-hand
+```
+
+**What it does:**
+- Loads MANO hand poses
+- Solves inverse kinematics for robot joints
+- Retargets hand poses to robot configuration
+- Initializes with open hand pose (`--open-hand`)
+
+**Output:**
+```
+example_datasets/processed/oakink/allegro/bimanual/pick_spoon_bowl/0/
+└── trajectory_kinematic.npz
+```
+
+**Data format:**
+```python
+# trajectory_kinematic.npz contains:
+qpos: [T, nq]  # Joint positions
+qvel: [T, nv]  # Joint velocities (zero for kinematic)
+ctrl: [T, nu]  # Control targets (same as qpos)
+time: [T]      # Timestamps
+```
+
+::: tip IK Options
+- `--open-hand`: Start IK from open hand pose (recommended)
+- `--use-contacts`: Use detected contacts in IK optimization
+- `--smooth-qvel`: Add velocity smoothness term
+:::
+
+### Step 7: Physics-Based Optimization
+
+Optimize trajectory with physics constraints using DIAL-MPC:
+
+```bash
+uv run examples/run_mjwp.py \
+  +override=${DATASET_NAME} \
+  task=${TASK} \
+  data_id=${DATA_ID} \
+  robot_type=${ROBOT_TYPE} \
+  embodiment_type=${HAND_TYPE}
+```
+
+**What it does:**
+- Loads kinematic trajectory as reference
+- Sets up parallel physics simulation (GPU)
+- Runs sampling-based optimization (DIAL-MPC)
+- Tracks reference while maintaining physics feasibility
+
+**Real-time visualization** shows:
+- Current robot state
+- Reference trajectory (transparent)
+- Optimization metrics
+- Contact forces
+
+**Output:**
+```
+example_datasets/processed/oakink/allegro/bimanual/pick_spoon_bowl/0/
+├── trajectory_mjwp.npz      # Optimized trajectory
+├── visualization_mjwp.mp4   # Rendered video
+└── metrics.json             # Success metrics
+```
+
+## Augmentation Methods
+
+The original `mjwp_eq` pipeline is no longer supported due to latest mujoco warp release make it unstable.
+The newer guidance methods are supported by the `mjwp_act` (e.g. `configs/override/oakink_act.yaml`) configuration instead. You will also need to add the `--act-scene` flag when running the `generate_xml.py` and `ik.py` scripts to ensure the necessary assets and logic are generated properly. This approach will provide the fair comparison you are looking for. Related assets is also available in our huggingface repo.
+
+## Dataset-Specific Examples
+
+### OakInk Dataset
+
+```bash
+export TASK=pick_spoon_bowl
+export DATASET_NAME=oakink
+export DATA_ID=0
+
+# Full workflow
+uv run spider/process_datasets/oakink.py --task=${TASK} --embodiment-type=bimanual --data-id=${DATA_ID}
+uv run spider/preprocess/decompose_fast.py --task=${TASK} --dataset-name=${DATASET_NAME} --data-id=${DATA_ID} --embodiment-type=bimanual
+uv run spider/preprocess/generate_xml.py --task=${TASK} --dataset-name=${DATASET_NAME} --data-id=${DATA_ID} --embodiment-type=bimanual --robot-type=allegro
+uv run spider/preprocess/ik.py --task=${TASK} --dataset-name=${DATASET_NAME} --data-id=${DATA_ID} --embodiment-type=bimanual --robot-type=allegro --open-hand
+uv run examples/run_mjwp.py +override=oakink task=${TASK} data_id=${DATA_ID} robot_type=allegro embodiment_type=bimanual
+```
+
+### GigaHand Dataset
+
+```bash
+export TASK=p36-tea
+export DATASET_NAME=gigahand
+export DATA_ID=10
+
+# Full workflow
+uv run spider/process_datasets/gigahand.py --task=${TASK} --embodiment-type=bimanual --data-id=${DATA_ID}
+uv run spider/preprocess/decompose_fast.py --task=${TASK} --dataset-name=${DATASET_NAME} --data-id=${DATA_ID} --embodiment-type=bimanual
+uv run spider/preprocess/generate_xml.py --task=${TASK} --dataset-name=${DATASET_NAME} --data-id=${DATA_ID} --embodiment-type=bimanual --robot-type=xhand
+uv run spider/preprocess/ik.py --task=${TASK} --dataset-name=${DATASET_NAME} --data-id=${DATA_ID} --embodiment-type=bimanual --robot-type=xhand --open-hand
+uv run examples/run_mjwp.py +override=gigahand task=${TASK} data_id=${DATA_ID} robot_type=xhand embodiment_type=bimanual
+```
+
+### Hot3D Dataset
+
+```bash
+export TASK=P0001_4bf4e21a-obj96945373046044
+export DATASET_NAME=hot3d
+export DATA_ID=0
+
+# Full workflow
+uv run spider/process_datasets/hot3d.py --task=${TASK} --embodiment-type=right --data-id=${DATA_ID}
+uv run spider/preprocess/decompose_fast.py --task=${TASK} --dataset-name=${DATASET_NAME} --data-id=${DATA_ID} --embodiment-type=right
+uv run spider/preprocess/generate_xml.py --task=${TASK} --dataset-name=${DATASET_NAME} --data-id=${DATA_ID} --embodiment-type=right --robot-type=xhand
+uv run spider/preprocess/ik.py --task=${TASK} --dataset-name=${DATASET_NAME} --data-id=${DATA_ID} --embodiment-type=right --robot-type=xhand --open-hand
+uv run examples/run_mjwp.py +override=hot3d task=${TASK} data_id=${DATA_ID} robot_type=xhand embodiment_type=right
+```