liyang
ae28d55f81
Source: RGB-D -> Dyn-HaMR -> L20 retargeting -> FoundationPose -> reference repair -> SPIDER, documented in docs/PIPELINE_LATEST.md and docs/SETUP_AND_WEIGHTS.md. Adds FoundationPose and nvdiffrast upstream snapshots, requirements/pipeline_venv.txt and the FoundationPose weight manifest/downloader. Assets (Git LFS): weights/ (WiLoR detector, HandFlow denoiser, UniDepth-L), FoundationPose checkpoints, HaMeR checkpoint, Dyn-HaMR HMP model and BMC constraints, L20 URDF/meshes, the 20260915_171525 D405 recording and the two box CADs. MANO models are not redistributed (third_party/hamer/_DATA/data/mano/README.txt). Environments, caches and run outputs excluded. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
39 lines
2.0 KiB
Markdown
39 lines
2.0 KiB
Markdown
# FoundationPose local setup
|
|
|
|
Validated 2026-09-16 for model-based RGB-D inference using an existing STL.
|
|
|
|
- Source: `third_party/FoundationPose`, upstream commit `a1b694b83e633c2cb6115b9063d940a687759392`.
|
|
- Interpreter: existing `.venv/bin/python`; reused Torch 2.7.1+cu128 and PyTorch3D.
|
|
- CUDA toolkit: existing `.cuda/usr/local/cuda-12.8`; GPU architecture 8.6.
|
|
- Installed upstream Python requirements, pybind11, ninja, and locally built nvdiffrast 0.4.0.
|
|
- Compiled `third_party/FoundationPose/mycpp/build/mycpp.cpython-311-x86_64-linux-gnu.so`.
|
|
- Weights and mirror provenance: `third_party/FoundationPose/weights/download_manifest.json`.
|
|
|
|
## Run the verified smoke check
|
|
|
|
From the HandFlow root, in a shell with NVIDIA GPU access:
|
|
|
|
```bash
|
|
source scripts/foundationpose_env.sh
|
|
python scripts/verify_foundationpose_setup.py
|
|
```
|
|
|
|
The environment script clears ROS-related Python/library overrides and selects the existing project environment/toolkit. It does not install Torch or CUDA.
|
|
|
|
The check exercises actual CUDA rasterization, loads both pretrained networks and executes their forward passes, and initializes FoundationPose with `docs/上半.stl`, including compiled C++ rotation clustering. Results: `output/foundationpose_setup/runtime_verification.json`; installation/build logs are in the same directory.
|
|
|
|
This does not validate pose accuracy or contact constraints on the recorded video. Optional BundleSDF `mycuda`/NeRF extensions are not required for the existing-STL path and were not built.
|
|
|
|
## Rebuild compiled modules
|
|
|
|
```bash
|
|
source scripts/foundationpose_env.sh
|
|
python -m pip install --no-build-isolation --no-deps ./third_party/nvdiffrast
|
|
cmake -S third_party/FoundationPose/mycpp -B third_party/FoundationPose/mycpp/build \
|
|
-DCMAKE_BUILD_TYPE=Release \
|
|
-Dpybind11_DIR="$FP_ROOT/.venv/lib/python3.11/site-packages/pybind11/share/cmake/pybind11" \
|
|
-DPYTHON_EXECUTABLE="$FP_ROOT/.venv/bin/python" \
|
|
-DPython_EXECUTABLE="$FP_ROOT/.venv/bin/python"
|
|
cmake --build third_party/FoundationPose/mycpp/build -j2
|
|
```
|