Externalize model payloads and document reproducibility checks

This commit is contained in:
Codex
2026-10-29 16:24:50 +08:00
parent 0aa2f49ab8
commit 56ca27ef57
161 changed files with 858 additions and 682 deletions
+50
View File
@@ -41,3 +41,53 @@ CLAUDE.md
/third_party/Dyn-HaMR/dyn-hamr/optim/BMC/joint_angles.npy
/third_party/FoundationPose/weights/2024-01-11-20-02-45/model_best.pth
/third_party/FoundationPose/weights/2023-10-28-18-33-37/model_best.pth
# External model/statistical resources and shards (download locally; never recommit)
/third_party/hamer/_DATA/hamer_ckpts/checkpoints/hamer.ckpt
/third_party/hamer/_DATA/hamer_ckpts/checkpoints/hamer.ckpt.part-*
/weights/unidepth-v2-vitl14/model.safetensors
/weights/unidepth-v2-vitl14/model.safetensors.part-*
/weights/handflow_denoiser.pt
/weights/handflow_denoiser.pt.part-*
/weights/detector.pt
/weights/detector.pt.part-*
/third_party/Dyn-HaMR/_DATA/hmp_model/results/model/optimizer.pth
/third_party/Dyn-HaMR/_DATA/hmp_model/results/model/optimizer.pth.part-*
/third_party/Dyn-HaMR/_DATA/hmp_model/results/model/local_encoder.pth
/third_party/Dyn-HaMR/_DATA/hmp_model/results/model/local_encoder.pth.part-*
/third_party/Dyn-HaMR/_DATA/hmp_model/results/model/nemf.pth
/third_party/Dyn-HaMR/_DATA/hmp_model/results/model/nemf.pth.part-*
/third_party/Dyn-HaMR/dyn-hamr/optim/BMC/joint_angles.npy
/third_party/Dyn-HaMR/dyn-hamr/optim/BMC/joint_angles.npy.part-*
/third_party/FoundationPose/weights/2024-01-11-20-02-45/model_best.pth
/third_party/FoundationPose/weights/2024-01-11-20-02-45/model_best.pth.part-*
/third_party/FoundationPose/weights/2023-10-28-18-33-37/model_best.pth
/third_party/FoundationPose/weights/2023-10-28-18-33-37/model_best.pth.part-*
/third_party/Dyn-HaMR/_DATA/hmp_model/mean-neutral-128-30fps.pt
/third_party/Dyn-HaMR/_DATA/hmp_model/mean-neutral-128-30fps.pt.part-*
/third_party/Dyn-HaMR/_DATA/hmp_model/results/model/global_encoder.pth
/third_party/Dyn-HaMR/_DATA/hmp_model/results/model/global_encoder.pth.part-*
/third_party/Dyn-HaMR/_DATA/hmp_model/results/model/loss.pkl
/third_party/Dyn-HaMR/_DATA/hmp_model/results/model/loss.pkl.part-*
/third_party/Dyn-HaMR/_DATA/hmp_model/std-neutral-128-30fps.pt
/third_party/Dyn-HaMR/_DATA/hmp_model/std-neutral-128-30fps.pt.part-*
/third_party/Dyn-HaMR/dyn-hamr/optim/BMC/CONVEX_HULLS.npy
/third_party/Dyn-HaMR/dyn-hamr/optim/BMC/CONVEX_HULLS.npy.part-*
/third_party/Dyn-HaMR/dyn-hamr/optim/BMC/PHI_max.npy
/third_party/Dyn-HaMR/dyn-hamr/optim/BMC/PHI_max.npy.part-*
/third_party/Dyn-HaMR/dyn-hamr/optim/BMC/PHI_min.npy
/third_party/Dyn-HaMR/dyn-hamr/optim/BMC/PHI_min.npy.part-*
/third_party/Dyn-HaMR/dyn-hamr/optim/BMC/bone_len_max.npy
/third_party/Dyn-HaMR/dyn-hamr/optim/BMC/bone_len_max.npy.part-*
/third_party/Dyn-HaMR/dyn-hamr/optim/BMC/bone_len_min.npy
/third_party/Dyn-HaMR/dyn-hamr/optim/BMC/bone_len_min.npy.part-*
/third_party/Dyn-HaMR/dyn-hamr/optim/BMC/curvatures_max.npy
/third_party/Dyn-HaMR/dyn-hamr/optim/BMC/curvatures_max.npy.part-*
/third_party/Dyn-HaMR/dyn-hamr/optim/BMC/curvatures_min.npy
/third_party/Dyn-HaMR/dyn-hamr/optim/BMC/curvatures_min.npy.part-*
/third_party/hamer/_DATA/data/mano_mean_params.npz
/third_party/hamer/_DATA/data/mano_mean_params.npz.part-*
/weights/normalization_stats.npz
/weights/normalization_stats.npz.part-*
/third_party/hamer/_DATA/data/mano/*.pkl
/third_party/Dyn-HaMR/_DATA/data/mano/*.pkl
+53 -65
View File
@@ -1,99 +1,87 @@
# Hand Motion Pipeline
从 D405 RGB-D 视频重建人手动作,将动作重定向到 L20 左右手,估计物体 6D 位姿,再进行手物配准、接触参考修复和 SPIDER 物理优化。
从 D405 RGB-D 视频重建人手动作,重定向到 L20 左右手,估计物体 6D 位姿,再进行配准、接触参考修复和 SPIDER 物理优化。
```text
RGB-D 视频 + 内参 + 物体 CAD
→ 相机运动估计 + Dyn-HaMR 人手重建
→ dex-retargeting + L20 左右手模型 → 手指关节与腕部轨迹
→ FoundationPose 物体 6D 跟踪 → 手物配准
→ 接触目标 / 碰撞修复 / 桌面约束 / 平滑投影 → 运动学参考
→ SPIDER / MuJoCo Warp → 自由物体物理验证、审计和回放
RGB-D + 相机内参 + 物体 CAD
→ RGB-D 里程计 + Dyn-HaMR 人手重建
→ dex-retargeting + L20 → 关节与腕部轨迹
→ FoundationPose 6D 跟踪 → 手物配准
→ 接触目标 / 碰撞修复 / 桌面约束 / 平滑投影
→ SPIDER / MuJoCo Warp 物理优化与审计
```
当前示例为 `20260915_171525` 的双手红蓝盒子录制。链路各阶段已有代码和执行说明;参考轨迹与物理结果分别验收。现有实验尚未复现稳定夹持、抬升和翻转,也未验证真机。HandFlow / ViPE 单目入口和旧瓶子、HDF5 示例作为独立分支保留。
示例是 `20260915_171525` 的 352 帧双手红蓝盒子录制。主流程各阶段有代码与命令,当前仍固定部分序列名、帧数和物体映射。现有实验未实现稳定夹持、抬升和翻转,未验证真机。
## 部署入口
## 获取代码、示例和模型
**完整 RGB-D 流程按 [安装依赖与资源](docs/SETUP_AND_WEIGHTS.md) 配置,再按 [逐阶段执行说明](docs/PIPELINE_LATEST.md) 运行。** 根目录 `requirements.txt`、`setup_env.sh` 和 `setup_vipe_env.sh` 服务旧 HandFlow / ViPE 分支,不会安装完整流程。
依赖分成四个环境,避免 MuJoCo 与上游依赖互相覆盖:
| 环境 | 角色 | Python |
|---|---|---|
| `.venv` | FoundationPose、RGB-D 处理、配准、参考修复、渲染 | 3.11 |
| `third_party/Dyn-HaMR/.dynhamr` | Dyn-HaMR 人手重建 | 3.10 |
| `.dex` | L20 重定向和 HDF5 导出 | 3.11 |
| `.spider` | SPIDER 物理优化与验证 | 新部署按上游使用 ≥ 3.12 |
需要 NVIDIA GPU、与 Torch 匹配的 CUDA toolkit / nvcc,以及 C++ 编译和离屏渲染依赖。具体版本和安装顺序见安装文档;现有版本记录来自开发机,尚未完成干净机器的从零安装复现。
## 获取资源与准备示例
**代码与示例资产通过 Git 获取;预训练权重和模型统计数据由使用者从上游获取,不再随当前版本分发。** MANO 需按官方条件申请;HaMeR checkpoint 包含 MANO 参数项,不能只检查是否提交了独立 `.pkl`。
```bash
git lfs install
GIT_LFS_SKIP_SMUDGE=1 git clone https://gitea.robotquan.com/liyang/hand-motion-pipeline.git
GIT_LFS_SKIP_SMUDGE=1 git clone --depth 1 https://gitea.robotquan.com/liyang/hand-motion-pipeline.git
cd hand-motion-pipeline
git lfs pull
python3 scripts/large_files.py assemble
git lfs pull # 示例视频/深度、CAD、机器人网格等当前版本资产
python3 scripts/check_pipeline_resources.py --prepare-example --recording-only
python3 scripts/external_resources.py list --scope all
```
大于平台单请求限制的权重以分片存储,`assemble` 会按清单校验 SHA-256。资源检查脚本不依赖 GPU 包;`--prepare-example` 为录制中多嵌套的一层创建相对链接,保留原视频与深度文件。
| 资源 | 云端状态 | 接收方操作 |
|---|---|---|
| 主流程权重、L20 URDF / 网格、双盒示例录制 | 已通过 Git LFS 管理 | `git lfs pull`,再组装权重 |
| 两个盒子 CAD | 已包含 | 使用 `docs/上半.stl`、`docs/下半.stl` |
| MANO 人手模型 | **未上传** | 按 MANO 官方获取条件自行准备,见下方 |
| Python 环境、CUDA 编译产物、缓存、`output/` | 未包含 | 在目标机器安装、构建和生成 |
| ViPE 单目分支的额外模型缓存 | 未完整包含 | 仅使用该分支时按安装文档准备 |
### MANO 模型
仓库不分发 `MANO_RIGHT.pkl` / `MANO_LEFT.pkl`。请从 [MANO 官方网站](https://mano.is.tue.mpg.de/) 按其条件取得,放到:
```text
third_party/hamer/_DATA/data/mano/MANO_RIGHT.pkl
third_party/hamer/_DATA/data/mano/MANO_LEFT.pkl # 使用独立左手模型的分支需要
```
Dyn-HaMR 的 `_DATA/data/mano` 已通过相对软链接指向该目录。默认资源检查要求右手模型;需要独立左手模型时加 `--require-left-mano`。`MANO_ROOT` 用于支持该变量的入口,不能替代 Dyn-HaMR 固定目录或其软链接。
按 [外部模型获取说明](docs/EXTERNAL_RESOURCES.md) 下载、解压或生成资源,恢复指定相对路径;再检查:
```bash
python3 scripts/external_resources.py check
python3 scripts/external_resources.py check --verify-sha256
python3 scripts/check_pipeline_resources.py
```
该命令检查输入布局、缺失文件、LFS 指针和部分清单大小,不加载模型或执行推理。运行环境仍须完成安装文档中的 GPU 冒烟检查。
这些检查不加载 pickle、不自动联网、不替代模型前向和流程验证。参考哈希对应开发快照,部分权重此前来自镜像,未与官方文件独立比对;重新生成的 BMC 统计可能不同。
## 路径配置与更换数据
| 资源 | 获取入口 | 目标目录 |
|---|---|---|
| MANO | [官方申请](https://mano.is.tue.mpg.de/) | `third_party/hamer/_DATA/data/mano/` |
| HaMeR | [上游下载脚本](https://github.com/geopavlakos/hamer/blob/main/fetch_demo_data.sh) | `third_party/hamer/_DATA/` |
| HMP | [Dyn-HaMR 资源脚本](https://github.com/ZhengdiYu/Dyn-HaMR/blob/main/scripts/prepare.sh) | `third_party/Dyn-HaMR/_DATA/hmp_model/` |
| BMC | [上游数据与计算步骤](https://github.com/MengHao666/Hand-BMC-pytorch) | `third_party/Dyn-HaMR/dyn-hamr/optim/BMC/` |
| WiLoR 检测器 | [模型目录](https://huggingface.co/spaces/rolpotamias/WiLoR/tree/main/pretrained_models) | `weights/detector.pt` |
| FoundationPose | [官方安装与权重](https://github.com/NVlabs/FoundationPose#installation) | `third_party/FoundationPose/weights/` |
项目可以放在任意目录,无需复制开发者的主目录、下载目录或 Python 环境:
`configs/external_resources.json` 提供下载入口、文件清单、参考大小与 SHA-256。旧 `large_files.py assemble` 只管理另行明确允许分发的分片,不能获取这些外部模型。
## 安装与运行
先按 [四环境安装说明](docs/SETUP_AND_WEIGHTS.md) 配置,再按 [逐阶段命令](docs/PIPELINE_LATEST.md) 运行。根目录 `requirements.txt`、`setup_env.sh`、`setup_vipe_env.sh` 仅服务旧 HandFlow / ViPE 单目分支。
| 环境 | 角色 | Python |
|---|---|---|
| `.venv` | FoundationPose、RGB-D、配准、修复、渲染 | 3.11 |
| `third_party/Dyn-HaMR/.dynhamr` | 人手重建 | 3.10 |
| `.dex` | L20 重定向、HDF5 导出 | 3.11 |
| `.spider` | SPIDER 物理优化 | 上游新部署要求 ≥3.12;历史验证用 3.11 兼容环境 |
需要 NVIDIA GPU、匹配的 Torch/CUDA toolkit 和编译/离屏渲染依赖。路径从项目根目录定位,无需开发者的主目录或旧环境:
```bash
source configs/project_env.example.sh
# 若 nvcc 不在 PATH 中,先设置自己机器的 CUDA toolkit:
# export CUDA_HOME=/path/to/cuda-toolkit
# nvcc 不在 PATH 时:export CUDA_HOME=/path/to/your/cuda-toolkit
source scripts/foundationpose_env.sh
```
项目根目录自动定位;CUDA 头文件从当前主 Python 环境查找。`CUDA_HOME`、`HANDFLOW_PYTHON` 和显式设置的 GPU 编译架构可按目标机器覆盖。两个旧视频脚本现在要求调用者提供输入:
## 已验证范围
```bash
python scripts/run_handflow_15886123.py --input /path/to/source.mp4
python scripts/export_camera_delivery.py --input /path/to/source.mp4
```
2026-10-29 的隔离源码与依赖副本中,已通过:352 帧 RGB-D 预处理、HaMeR 一次 GPU 前向、8 帧 L20 重定向、FoundationPose C++ 构建/光栅化/网络前向/CAD 初始化、SPIDER 两世界 60 步接触仿真。
主流程仍固定部分示例序列名、帧数、颜色和 CAD 映射。更换视频按 [新数据适配说明](docs/PIPELINE_LATEST.md#3-换一段新视频要改什么) 修改,不能将其当作任意物体的一键工具。上游源码还保留部分示例路径,范围见 [路径与迁移说明](docs/PORTABILITY.md)。
**这不是从零安装或完整串联通过的声明。** 验证复用了本地已有依赖和合法取得的模型;未新跑完整 Dyn-HaMR 时序拟合→接触修复→L20 SPIDER 优化→HDF5。验收软件可运行与抓取成功分别记录,见 [验证记录](docs/VALIDATION_20261029.md)。
## 其他文档与源码包
## 许可与其他说明
- [FoundationPose 构建与验证](docs/FOUNDATIONPOSE_SETUP.md)
- [集成结构与旧瓶子分支](docs/INTEGRATED_PROJECT.md)
- [重定向方法、验证与限制](docs/retargeting_workflow_and_lessons_zh.md)
- [HDF5 接口约定](docs/HDF5_REQUIREMENTS.md)
根 [LICENSE](LICENSE) 是原 HandFlow 的 MIT。第三方代码保留各自许可;FoundationPose、当前 nvdiffrast 快照、SPIDER 存在非商业使用限制。模型、数据和资产的条件不能由代码许可推定,见 [第三方说明](THIRD_PARTY_NOTICES.md)。L20 资产与标定的精确上游版本、示例数据许可仍需维护者核对,不把整个资源集合标成统一 MIT。
当前提交移除模型载荷不等于清除了旧提交或服务端 LFS。历史清理状态见 [发布与历史边界](docs/EXTERNAL_RESOURCES.md#历史版本)。
- [路径与新数据适配](docs/PORTABILITY.md)
- [FoundationPose 构建](docs/FOUNDATIONPOSE_SETUP.md)
- [旧瓶子与单目分支](docs/INTEGRATED_PROJECT.md)
- [HDF5 接口](docs/HDF5_REQUIREMENTS.md)
- [HandFlow 原始说明](docs/HANDFLOW_UPSTREAM_README.md)
`python3 scripts/package_source_release.py` 导出的源码包与当前 Git LFS 仓库不同:源码包不携带权重、L20 资产、用户录制、运行环境和实验输出。`UPSTREAM_SOURCES.json`、`SHA256SUMS`、`EXTERNAL_ASSETS.json` 等为原源码快照的来源与资源记录,后续提交以 Git 历史和 `configs/large_files.json` 为准。
保留 [HandFlow 原 LICENSE](LICENSE) 及各第三方目录 LICENSE;资源使用条件由对应提供方规定。
发布前运行 `python3 -m unittest discover -s tests -v` 和 `python3 scripts/check_release_policy.py`,防止把外部模型或分片重新提交。
+34
View File
@@ -0,0 +1,34 @@
# 第三方代码、模型与资产说明
根 `LICENSE` 保留原 HandFlow 的 MIT 授权。第三方子目录中的 LICENSE 继续适用于对应代码;不能把整个依赖、模型和数据集合统一改为 MIT。以下是主要组件清单,不是对所有传递依赖的完整法律审计。
| 组件 | 本仓库许可依据 | 说明 |
|---|---|---|
| HandFlow | [LICENSE](LICENSE) | 原作者 MIT 声明保留 |
| Dyn-HaMR | [许可证](third_party/Dyn-HaMR/LICENSE) | 文件为 MIT;README 中关于科研用途的描述与文件措辞存在差异,模型/数据仍应查各来源 |
| HaMeR | [许可证](third_party/hamer/LICENSE.md) | 代码 MIT 不代表 MANO 数据可以再分发;checkpoint 改为外部获取 |
| ViPE | [许可证](third_party/vipe/LICENSE)、[第三方说明](third_party/vipe/THIRD_PARTY_LICENSES.md) | Apache-2.0 及其第三方条件;可选单目分支 |
| FoundationPose | [许可证](third_party/FoundationPose/LICENSE) | 当前源码包含非商业研究/评估使用限制,保留完整许可与版权声明 |
| nvdiffrast 当前快照 | [许可证](third_party/nvdiffrast/LICENSE.txt) | 当前快照为 NVIDIA 自定义许可并有限定用途,不由后续版本许可替代 |
| SPIDER | [许可证](third_party/spider/LICENSE) | CC BY-NC,须署名且限制商业用途 |
| Hand-BMC 实现 | [许可证](third_party/Dyn-HaMR/third-party/Hand-BMC-pytorch-main/LICENSE) | 实现 MIT;上游混合关节数据集各有自己的条款 |
| DROID-SLAM | [许可证](third_party/Dyn-HaMR/third-party/DROID-SLAM/LICENSE) | BSD-3-Clause;不用于主 RGB-D 里程计路径 |
| Configer 等运行时依赖 | 各安装包许可证 | Configer 的安装元数据声明 AGPL;安装时记录所有直接与传递依赖许可,不以本项目 MIT 覆盖 |
## 模型和统计数据
当前版本不分发预训练模型权重、MANO 参数文件/均值参数及 BMC 统计文件或其分片。接收者按 [外部资源说明](docs/EXTERNAL_RESOURCES.md) 自行从提供者获取或生成。MANO 有独立注册及禁止再分发条款;HaMeR checkpoint 中的 MANO 参数项也纳入外部资源。
不要从“公开下载”“有 SHA256”“代码是 MIT”推断模型允许再次分发。FoundationPose 同名模型的新旧发行版可能有不同许可,需匹配实际文件。
## L20、示例和 CAD 的待核对项
当前保留的 L20 模型和标定文件没有独立许可文件。厂商 [官方 URDF 仓库](https://github.com/linker-bot/linkerhand-urdf) 声明 Apache-2.0,但尚未将本项目中的 g20 URDF、网格和标定 JSON 与精确上游 commit 逐项对应。维护者应确认来源、保留 LICENSE/NOTICE 并说明本地修改后再对资产许可作出承诺。
RGB-D 示例及盒子 CAD 的作者、公开使用条件也需明确。根 MIT 不自动覆盖来源未明确的资产;本次更新不宣称已完成所有资源的许可核准。
## 版本与历史
`UPSTREAM_SOURCES.json`、`EXTERNAL_ASSETS.json`、`SHA256SUMS` 等来自早期源码快照,是来源记录,不是当前版本的依赖 lock 或当前文件全量校验清单。当前源代码变更以 Git commit 为准,外部模型清单以 `configs/external_resources.json` 为准。
本次提交不重写旧 Git 历史、不删除服务端历史 LFS 对象。需要彻底处理历史资源时,必须另行迁移历史引用并清理服务端存储;见 [历史边界](docs/EXTERNAL_RESOURCES.md#历史版本)。
+213
View File
@@ -0,0 +1,213 @@
{
"schema_version": 1,
"policy": "Pretrained weights and model/statistical data are acquired by each user from upstream. Listing a URL or reference hash does not grant redistribution rights. Source-code licenses and model/data licenses are separate.",
"hash_boundary": "Hashes identify the tested development snapshot, not independent proof of official provenance; MANO has no shared hash. BMC regeneration may produce different values.",
"resources": [
{
"id": "hamer",
"name": "HaMeR checkpoint and MANO mean parameters",
"scope": "rgbd",
"source_url": "https://github.com/geopavlakos/hamer/blob/main/fetch_demo_data.sh",
"download_url": "https://drive.google.com/uc?id=1mv7CUAnm73oKsEEG1xE3xH2C_oqcFSzT",
"instructions": "Download the upstream demo package, extract locally, then place its _DATA/hamer_ckpts and _DATA/data/mano_mean_params.npz under third_party/hamer/_DATA/. Obtain MANO separately. Do not upload the checkpoint or its shards; it contains MANO parameter buffers.",
"files": [
{
"path": "third_party/hamer/_DATA/hamer_ckpts/checkpoints/hamer.ckpt",
"bytes": 2689536166,
"sha256": "e5cc06f294d88a92dee24e603480aab04de532b49f0e08200804ee7d90e16f53"
},
{
"path": "third_party/hamer/_DATA/data/mano_mean_params.npz",
"bytes": 1178,
"sha256": "efc0ec58e4a5cef78f3abfb4e8f91623b8950be9eff8b8e0dbb0d036ebc63988"
}
]
},
{
"id": "mano",
"name": "MANO licensed hand model",
"scope": "rgbd",
"source_url": "https://mano.is.tue.mpg.de/",
"license_url": "https://mano.is.tue.mpg.de/license.html",
"instructions": "Register and obtain the model from the provider. Put MANO_RIGHT.pkl under third_party/hamer/_DATA/data/mano/. Some optional branches also need MANO_LEFT.pkl. No redistribution.",
"files": [
{
"path": "third_party/hamer/_DATA/data/mano/MANO_RIGHT.pkl"
}
]
},
{
"id": "hmp",
"name": "Dyn-HaMR HMP motion prior",
"scope": "rgbd",
"source_url": "https://github.com/ZhengdiYu/Dyn-HaMR/blob/main/scripts/prepare.sh",
"download_url": "https://drive.google.com/uc?id=1_rp_SMxZA6Gjm6Z49M6hXam9kjMD6-GQ",
"instructions": "Download hmp_model.zip from upstream; extract hmp_model/ under third_party/Dyn-HaMR/_DATA/. Preserve upstream terms.",
"files": [
{
"path": "third_party/Dyn-HaMR/_DATA/hmp_model/results/model/optimizer.pth",
"bytes": 541530298,
"sha256": "ea48b2b6d4a8bebff41b72a6f034db5506b488c8b4f114709c58b67953d2ce8d"
},
{
"path": "third_party/Dyn-HaMR/_DATA/hmp_model/results/model/local_encoder.pth",
"bytes": 161190688,
"sha256": "89db49f42b98986121ff0b2152b89c045f6d66945434bf38e6f21981e70ea82b"
},
{
"path": "third_party/Dyn-HaMR/_DATA/hmp_model/results/model/nemf.pth",
"bytes": 92620900,
"sha256": "cbb9d5bae07de807c13f02128b18e1b7b5f31e6626bdda16b800e636753004ff"
},
{
"path": "third_party/Dyn-HaMR/_DATA/hmp_model/mean-neutral-128-30fps.pt",
"bytes": 5834,
"sha256": "f895ea3e9913dcf0daeab06c770d44bdcd1e780a887f2ef8046157688fd0f431"
},
{
"path": "third_party/Dyn-HaMR/_DATA/hmp_model/results/model/global_encoder.pth",
"bytes": 16944962,
"sha256": "53ea55a05bad9816e087d9296b4aa40cc5133139417156e3faffdc3ff9957f54"
},
{
"path": "third_party/Dyn-HaMR/_DATA/hmp_model/results/model/loss.pkl",
"bytes": 11964704,
"sha256": "85d6b88d2aa847462c6f73797a29e66c12c77797583b31f174a5f8ee9d817e28"
},
{
"path": "third_party/Dyn-HaMR/_DATA/hmp_model/std-neutral-128-30fps.pt",
"bytes": 5756,
"sha256": "e1da9a50afbc1015e9fc846edbd4457a46e3594d85265020ed20ba61546a126a"
}
]
},
{
"id": "bmc",
"name": "Hand biomechanical statistics",
"scope": "rgbd",
"source_url": "https://github.com/MengHao666/Hand-BMC-pytorch",
"download_url": "https://drive.google.com/file/d/1_wV8QjmsVCMBEBhm56gFA2XTyU8VEHzk/view",
"instructions": "Obtain upstream joints data under its dataset terms and follow the upstream calculation scripts. Put the eight resulting .npy files under third_party/Dyn-HaMR/dyn-hamr/optim/BMC/. The existing _DATA/BMC relative link exposes them to both fitting stages. Reference hashes describe the development snapshot; recalculated statistics may differ. There is no verified direct download for the exact precomputed snapshot.",
"files": [
{
"path": "third_party/Dyn-HaMR/dyn-hamr/optim/BMC/joint_angles.npy",
"bytes": 90116528,
"sha256": "92c96d1899aa290542cf31af322ba53811afd914258f1367fb41e2aeaa6e0b62"
},
{
"path": "third_party/Dyn-HaMR/dyn-hamr/optim/BMC/CONVEX_HULLS.npy",
"bytes": 4897,
"sha256": "013177b2fdbdbbb834d5f3daab5e291f66c67193e2f52efed5247c8865edfec3"
},
{
"path": "third_party/Dyn-HaMR/dyn-hamr/optim/BMC/PHI_max.npy",
"bytes": 160,
"sha256": "9b9e95091c47381fa872b93b28d28c093dbcc1576cc617eb29f23ce53648da99"
},
{
"path": "third_party/Dyn-HaMR/dyn-hamr/optim/BMC/PHI_min.npy",
"bytes": 160,
"sha256": "ce403440680ecfdfc23ee7dae42fb6a0fdfac1919053458d476e589bab63710c"
},
{
"path": "third_party/Dyn-HaMR/dyn-hamr/optim/BMC/bone_len_max.npy",
"bytes": 208,
"sha256": "26f9ea48a090100fb2f70f688d3fad46dacfbbf365e5f8b6a483b6a971a7fa67"
},
{
"path": "third_party/Dyn-HaMR/dyn-hamr/optim/BMC/bone_len_min.npy",
"bytes": 208,
"sha256": "f91a46e2df3f805095aa7b20d5294bc30b9d83d76e4f5a4e7fd1f1ed3bc692fa"
},
{
"path": "third_party/Dyn-HaMR/dyn-hamr/optim/BMC/curvatures_max.npy",
"bytes": 160,
"sha256": "e576ffc657cd50ca0b1a5292bd4e44639b960a68e4eb8bd758cb502620c48880"
},
{
"path": "third_party/Dyn-HaMR/dyn-hamr/optim/BMC/curvatures_min.npy",
"bytes": 160,
"sha256": "31cc5766a93f4b018959e4b18cc93c309b3d0d8627bd5cf10f8d3cd429549b37"
}
]
},
{
"id": "detector",
"name": "WiLoR detector",
"scope": "rgbd",
"source_url": "https://huggingface.co/spaces/rolpotamias/WiLoR/tree/main/pretrained_models",
"download_url": "https://huggingface.co/spaces/rolpotamias/WiLoR/resolve/main/pretrained_models/detector.pt",
"instructions": "Download detector.pt to weights/detector.pt. Keep the existing relative detector symlink in Dyn-HaMR; no second copy is needed.",
"files": [
{
"path": "weights/detector.pt",
"bytes": 53582271,
"sha256": "5ef3df44e42d2db52d4ffe91f83a22ce9925e2acc9abebf453f2c5d22e380033"
}
]
},
{
"id": "foundationpose",
"name": "FoundationPose original research scorer/refiner",
"scope": "rgbd",
"source_url": "https://github.com/NVlabs/FoundationPose#installation",
"download_url": "https://drive.google.com/drive/folders/1DFezOAD0oD1BblsXVxqDsl8fj0qzB82i",
"instructions": "Obtain both run folders, each with config.yml and model_best.pth, from the official research release. Put them under third_party/FoundationPose/weights/. Reference hashes for model weights describe the previously used mirror and have not been independently compared with the official distribution.",
"files": [
{
"path": "third_party/FoundationPose/weights/2024-01-11-20-02-45/model_best.pth",
"bytes": 190229389,
"sha256": "81924d384bf5c26c646ee4783104982ae3d1e049c181c36641b6a7aeae494c26"
},
{
"path": "third_party/FoundationPose/weights/2023-10-28-18-33-37/model_best.pth",
"bytes": 68220109,
"sha256": "774700586ddc435d408fc01c9809c43e151232936369dfbea0f0f964ba471d60"
},
{
"path": "third_party/FoundationPose/weights/2023-10-28-18-33-37/config.yml",
"bytes": 708,
"sha256": "28a6ba94a33230ee5fc3c51939486281578b0972542bd9e38ca6123e75605686"
},
{
"path": "third_party/FoundationPose/weights/2024-01-11-20-02-45/config.yml",
"bytes": 778,
"sha256": "a79db4de3b95885dd5ae86833b37b8698a75dad81e87d1086cd50b2fcd8dda3f"
}
]
},
{
"id": "handflow",
"name": "Optional HandFlow monocular branch",
"scope": "optional",
"source_url": "https://huggingface.co/mxxu00/HandFlow",
"instructions": "Download handflow_denoiser.pt and normalization_stats.npz into weights/; review the upstream model card.",
"files": [
{
"path": "weights/handflow_denoiser.pt",
"bytes": 667907131,
"sha256": "2fbc4e1fa7a60f469a6ac94933a6e6dc8a86a0e1fc13bd7cd81c430c79acfcda"
},
{
"path": "weights/normalization_stats.npz",
"bytes": 2008,
"sha256": "7313334e6b9537fa57ec9763e83f36dfd4998e7d1de09aa3fb21c5bfca8e92c4"
}
]
},
{
"id": "unidepth",
"name": "Optional UniDepth monocular branch",
"scope": "optional",
"source_url": "https://huggingface.co/lpiccinelli/unidepth-v2-vitl14",
"instructions": "Download the model to weights/unidepth-v2-vitl14/model.safetensors and retain associated upstream configuration and terms.",
"files": [
{
"path": "weights/unidepth-v2-vitl14/model.safetensors",
"bytes": 1415383604,
"sha256": "ba73d3de735302ccc64a50f1e557122050c4b1893e6060b28dba05d6af3e67c6"
}
]
}
]
}
+9 -12
View File
@@ -1,30 +1,27 @@
{
"source": "community mirror gpue/foundationpose-weights; official Google Drive quota exceeded",
"verification": "size and mirror LFS SHA256; not independently compared to official files",
"source": "Official research release: https://github.com/NVlabs/FoundationPose#installation",
"verification": "Reference hashes from the previously tested mirror; not independently matched to official payloads. Obtain files from upstream under its terms; no automatic mirror download.",
"files": [
{
"path": "2023-10-28-18-33-37/config.yml",
"size": 708,
"sha256": "28a6ba94a33230ee5fc3c51939486281578b0972542bd9e38ca6123e75605686",
"url": "https://hf-mirror.com/gpue/foundationpose-weights/resolve/main/2023-10-28-18-33-37/config.yml"
"sha256": "28a6ba94a33230ee5fc3c51939486281578b0972542bd9e38ca6123e75605686"
},
{
"path": "2023-10-28-18-33-37/model_best.pth",
"size": 68220109,
"sha256": "774700586ddc435d408fc01c9809c43e151232936369dfbea0f0f964ba471d60",
"url": "https://hf-mirror.com/gpue/foundationpose-weights/resolve/main/2023-10-28-18-33-37/model_best.pth"
"sha256": "774700586ddc435d408fc01c9809c43e151232936369dfbea0f0f964ba471d60"
},
{
"path": "2024-01-11-20-02-45/config.yml",
"size": 778,
"sha256": "a79db4de3b95885dd5ae86833b37b8698a75dad81e87d1086cd50b2fcd8dda3f",
"url": "https://hf-mirror.com/gpue/foundationpose-weights/resolve/main/2024-01-11-20-02-45/config.yml"
"sha256": "a79db4de3b95885dd5ae86833b37b8698a75dad81e87d1086cd50b2fcd8dda3f"
},
{
"path": "2024-01-11-20-02-45/model_best.pth",
"size": 190229389,
"sha256": "81924d384bf5c26c646ee4783104982ae3d1e049c181c36641b6a7aeae494c26",
"url": "https://hf-mirror.com/gpue/foundationpose-weights/resolve/main/2024-01-11-20-02-45/model_best.pth"
"sha256": "81924d384bf5c26c646ee4783104982ae3d1e049c181c36641b6a7aeae494c26"
}
]
}
],
"download_url": "https://drive.google.com/drive/folders/1DFezOAD0oD1BblsXVxqDsl8fj0qzB82i"
}
+2 -62
View File
@@ -1,65 +1,5 @@
{
"chunk_bytes": 50331648,
"files": [
{
"path": "third_party/hamer/_DATA/hamer_ckpts/checkpoints/hamer.ckpt",
"bytes": 2689536166,
"sha256": "e5cc06f294d88a92dee24e603480aab04de532b49f0e08200804ee7d90e16f53",
"parts": 54
},
{
"path": "weights/unidepth-v2-vitl14/model.safetensors",
"bytes": 1415383604,
"sha256": "ba73d3de735302ccc64a50f1e557122050c4b1893e6060b28dba05d6af3e67c6",
"parts": 29
},
{
"path": "weights/handflow_denoiser.pt",
"bytes": 667907131,
"sha256": "2fbc4e1fa7a60f469a6ac94933a6e6dc8a86a0e1fc13bd7cd81c430c79acfcda",
"parts": 14
},
{
"path": "weights/detector.pt",
"bytes": 53582271,
"sha256": "5ef3df44e42d2db52d4ffe91f83a22ce9925e2acc9abebf453f2c5d22e380033",
"parts": 2
},
{
"path": "third_party/Dyn-HaMR/_DATA/hmp_model/results/model/optimizer.pth",
"bytes": 541530298,
"sha256": "ea48b2b6d4a8bebff41b72a6f034db5506b488c8b4f114709c58b67953d2ce8d",
"parts": 11
},
{
"path": "third_party/Dyn-HaMR/_DATA/hmp_model/results/model/local_encoder.pth",
"bytes": 161190688,
"sha256": "89db49f42b98986121ff0b2152b89c045f6d66945434bf38e6f21981e70ea82b",
"parts": 4
},
{
"path": "third_party/Dyn-HaMR/_DATA/hmp_model/results/model/nemf.pth",
"bytes": 92620900,
"sha256": "cbb9d5bae07de807c13f02128b18e1b7b5f31e6626bdda16b800e636753004ff",
"parts": 2
},
{
"path": "third_party/Dyn-HaMR/dyn-hamr/optim/BMC/joint_angles.npy",
"bytes": 90116528,
"sha256": "92c96d1899aa290542cf31af322ba53811afd914258f1367fb41e2aeaa6e0b62",
"parts": 2
},
{
"path": "third_party/FoundationPose/weights/2024-01-11-20-02-45/model_best.pth",
"bytes": 190229389,
"sha256": "81924d384bf5c26c646ee4783104982ae3d1e049c181c36641b6a7aeae494c26",
"parts": 4
},
{
"path": "third_party/FoundationPose/weights/2023-10-28-18-33-37/model_best.pth",
"bytes": 68220109,
"sha256": "774700586ddc435d408fc01c9809c43e151232936369dfbea0f0f964ba471d60",
"parts": 2
}
]
"files": [],
"note": "Pretrained weights are now external. See configs/external_resources.json and docs/EXTERNAL_RESOURCES.md. This manifest is only for future explicitly redistributable assets."
}
+52
View File
@@ -0,0 +1,52 @@
# 外部模型获取与本地校验
本项目采用代码公开、模型由接收者向上游获取的方式。下载地址和参考哈希仅用于定位资源,不授予再分发权。第三方源码按各自 LICENSE 保留;非商业限制不等同于禁止再分发。
```bash
python3 scripts/external_resources.py list --scope all
python3 scripts/external_resources.py check # 主 RGB-D 所需模型
python3 scripts/external_resources.py check --scope all # 加可选单目分支
python3 scripts/external_resources.py check --verify-sha256 # 对照开发快照
```
工具使用 Python 标准库,不自动下载、不读取 pickle 内容、不改变已有文件。缺资源时返回非零并显示上游入口。所有路径相对项目根目录。
## 获取与放置
| 组 | 上游入口 / 下载地址 | 放置方式 |
|---|---|---|
| MANO | [官网](https://mano.is.tue.mpg.de/);[许可](https://mano.is.tue.mpg.de/license.html) | 自行申请后放 `third_party/hamer/_DATA/data/mano/MANO_RIGHT.pkl`;需要独立左手模型的分支再放 `MANO_LEFT.pkl` |
| HaMeR | [官方下载脚本](https://github.com/geopavlakos/hamer/blob/main/fetch_demo_data.sh);[模型包](https://drive.google.com/uc?id=1mv7CUAnm73oKsEEG1xE3xH2C_oqcFSzT) | 本地解压,把 `_DATA/hamer_ckpts/` 与 `_DATA/data/mano_mean_params.npz` 放到 `third_party/hamer/_DATA/`;MANO 独立取得 |
| HMP | [Dyn-HaMR 脚本](https://github.com/ZhengdiYu/Dyn-HaMR/blob/main/scripts/prepare.sh);[hmp_model.zip](https://drive.google.com/uc?id=1_rp_SMxZA6Gjm6Z49M6hXam9kjMD6-GQ) | 解压 `hmp_model/` 到 `third_party/Dyn-HaMR/_DATA/`,保留包内目录结构 |
| BMC | [上游说明](https://github.com/MengHao666/Hand-BMC-pytorch);[joints.zip](https://drive.google.com/file/d/1_wV8QjmsVCMBEBhm56gFA2XTyU8VEHzk/view) | 按数据集条款取得关节数据,运行 `calculate_bmc.py`、`calculate_convex_hull.py`;把 8 个 `.npy` 放到 `third_party/Dyn-HaMR/dyn-hamr/optim/BMC/` |
| WiLoR | [模型目录](https://huggingface.co/spaces/rolpotamias/WiLoR/tree/main/pretrained_models);[detector.pt](https://huggingface.co/spaces/rolpotamias/WiLoR/resolve/main/pretrained_models/detector.pt) | 保存为 `weights/detector.pt`;Dyn-HaMR 检测器路径已有相对链接,无需第二份复制 |
| FoundationPose | [官方安装说明](https://github.com/NVlabs/FoundationPose#installation);[官方权重目录](https://drive.google.com/drive/folders/1DFezOAD0oD1BblsXVxqDsl8fj0qzB82i) | 将 `2023-10-28-18-33-37/`、`2024-01-11-20-02-45/` 放到 `third_party/FoundationPose/weights/`,每个目录都有 `config.yml` 和 `model_best.pth` |
| HandFlow,可选 | [模型库](https://huggingface.co/mxxu00/HandFlow) | `weights/handflow_denoiser.pt`、`weights/normalization_stats.npz` |
| UniDepth,可选 | [模型库](https://huggingface.co/lpiccinelli/unidepth-v2-vitl14) | `weights/unidepth-v2-vitl14/model.safetensors`,并保留上游所需配置 |
BMC 八个文件为 `bone_len_max.npy`、`bone_len_min.npy`、`CONVEX_HULLS.npy`、`curvatures_max.npy`、`curvatures_min.npy`、`joint_angles.npy`、`PHI_max.npy`、`PHI_min.npy`。现有 `_DATA/BMC` 链接指向该目录。**没有验证过与本项目开发快照完全相同的预计算 BMC 直接下载地址**;上表给的是上游原始数据及生成方法。重算结果可能与参考大小/哈希不同,应明确记录数据版本、参数和新的校验值,不能伪报同一快照。
HaMeR checkpoint 中存在 `mano.v_template`、`mano.shapedirs`、`mano.J_regressor` 等序列化参数键,因此当前版本也不分发该 checkpoint 的分片。不能通过改后缀或只排除独立 MANO `.pkl` 宣称已排除模型数据。
FoundationPose 的旧参考哈希来自此前测试使用的社区镜像,未独立与官方权重比对。兼容入口 `scripts/download_foundationpose_weights.py` 现在只显示官方入口,或用 `--check` 检查本地文件,不再自动访问社区镜像。不同 NVIDIA 发行版可能采用不同模型许可,不能把新版同名模型的许可直接套到旧文件。
## L20 与示例资产
L20 URDF/网格/标定、RGB-D 示例和 CAD 仍是当前 Git LFS 资产。厂商 [官方 URDF 仓库](https://github.com/linker-bot/linkerhand-urdf) 声明 Apache-2.0,但当前官方 `linkerhand_l20_*.urdf` 与项目中的 `linkerhand_g20_*.urdf` 命名不同,尚未建立逐文件来源和标定映射。
维护者需核对精确来源版本、差异、LICENSE/NOTICE 和标定数据许可;不能直接用最新版官方模型替换后声称同一重定向资产。自采示例和 CAD 的作者及公开许可也需明确。见 [第三方说明](../THIRD_PARTY_NOTICES.md)。
## 历史版本
本次普通提交只改变当前版本:不再跟踪预训练权重、模型/统计数据及其 48 MiB 分片,并加入 `.gitignore` 与发布检查。它不会重写既有分支历史,也不会删除服务端历史 LFS 对象。
旧提交仍可能访问旧资源。若要彻底停止历史资源分发,维护者还需单独制定历史迁移、引用清理、备份与服务端 LFS 清理方案。本次没有执行强制推送或服务端对象删除。新接收者建议 `git clone --depth 1`,只拉当前版本资产。
## 发布检查
```bash
python3 -m unittest discover -s tests -v
python3 scripts/check_release_policy.py
```
检查针对 Git 当前索引,能识别外部文件、同路径 `.part-NNN` 分片和 MANO `.pkl`。不把工作树检查等同于完整历史或任意二进制的授权审计。
+1 -1
View File
@@ -7,7 +7,7 @@ Validated 2026-09-16 for model-based RGB-D inference using an existing STL.
- CUDA toolkit: existing `.cuda/usr/local/cuda-12.8`; GPU architecture 8.6.
- Installed upstream Python requirements, pybind11, ninja, and locally built nvdiffrast 0.4.0.
- Compiled `third_party/FoundationPose/mycpp/build/mycpp.cpython-311-x86_64-linux-gnu.so`.
- Weights and mirror provenance: `third_party/FoundationPose/weights/download_manifest.json`.
- Obtain weights from the official research-release entry in [EXTERNAL_RESOURCES.md](EXTERNAL_RESOURCES.md). Current Git does not distribute model payloads. Reference hashes retain historical mirror provenance; no independent official-file match is claimed.
## Run the verified smoke check
+3 -1
View File
@@ -1,5 +1,7 @@
# 最新流程(2026-09-17):RGB-D 双手视频 → Dyn-HaMR → L20 重定向 → FoundationPose → 参考修复 → SPIDER 物理
**以下结果是开发机历史实验记录;2026-10-29 的隔离验证只覆盖部分组件,未重跑完整链路,见 [验证范围](VALIDATION_20261029.md)。**
本文是 `docs/20260915_171525`(D405 头戴 RGB-D,352 帧,双手 + 红/蓝两个盒子)这条数据上**实际跑通并交付**的链路,按执行顺序给出每一步的脚本、输入、输出、关键参数和验证文件。旧的单手 / 瓶子链路见 [INTEGRATED_PROJECT.md](INTEGRATED_PROJECT.md),此处不再重复。RL 跟踪训练线不在本文范围。
**当前状态**
@@ -10,7 +12,7 @@
## 0. 总览
首次克隆先按 [安装与资源说明](SETUP_AND_WEIGHTS.md) 拉取并组装 LFS 资源,准备 MANO。云端录制多嵌套了一层,可运行以下命令创建兼容的相对链接;数据不移动、不覆盖:
首次克隆先按 [安装与资源说明](SETUP_AND_WEIGHTS.md) 拉取示例资产,并按 [外部模型说明](EXTERNAL_RESOURCES.md) 自行取得各模型与 BMC 数据。云端录制多嵌套了一层,可运行以下命令创建兼容的相对链接;数据不移动、不覆盖:
```bash
python3 scripts/check_pipeline_resources.py --prepare-example --recording-only
+4
View File
@@ -27,3 +27,7 @@
## 验证范围
`scripts/check_pipeline_resources.py` 仅使用 Python 标准库,检查录制布局、模型文件存在性、未拉取的 LFS 指针和部分权重大小;不反序列化 MANO,不加载网络,不验证位姿精度或物理抓取。`large_files.py assemble` 另行校验分片权重 SHA-256。GPU 扩展、网络前向和物理冒烟检查分别按安装文档执行,完整抓取按每段实验结果验收。
## 2026-10-29 资源策略与验证范围
预训练权重和统计数据改为 [外部获取](EXTERNAL_RESOURCES.md),不能再靠 LFS/assemble 恢复所有模型。独立源码与依赖副本的组件验证见 [记录](VALIDATION_20261029.md);不是干净安装或完整串联通过的声明。不要通过 `.pth` 复用开发者的其他环境来完成部署。
+36 -91
View File
@@ -1,6 +1,6 @@
# 安装依赖、环境与权重(2026-09-17)
# 安装依赖、环境与外部模型(2026-10-29)
部署指引更新:2026-10-27。本文是完整 RGB-D 流程的安装入口;根目录 `requirements.txt`、`setup_env.sh`、`setup_vipe_env.sh` 仅用于旧 HandFlow / ViPE 分支。先按第 4.0 节准备资源,再安装四个环境并执行第 6 节检查。
部署指引更新:2026-10-29。本文是完整 RGB-D 流程的安装入口;根目录 `requirements.txt`、`setup_env.sh`、`setup_vipe_env.sh` 仅用于旧 HandFlow / ViPE 分支。先按第 4 节准备外部资源,再安装四个环境并执行第 6 节检查。
适用:源码快照 `hand-motion-pipeline-20260917`(由 `scripts/package_source_release.py` 生成)。链路为 **RGB-D → Dyn-HaMR → L20 重定向 → FoundationPose → 参考修复 → SPIDER**,执行顺序与每步命令见 [PIPELINE_LATEST.md](PIPELINE_LATEST.md)。本文按本机(2026-09-17)实际环境核对;**没有在干净机器上从零复现安装**,版本表是观测值,不是跨平台 lock。
@@ -10,7 +10,7 @@
|---|---|
| OS / GPU / 驱动 | Ubuntu 22.04.5,RTX 3080 Laptop 16 GB,NVIDIA 驱动 595.91 |
| CUDA toolkit(nvcc) | 12.8,解包在项目内 `.cuda/usr/local/cuda-12.8`(用户级,不需要 sudo;`scripts/foundationpose_env.sh` 把 `CUDA_HOME` 指向它)。任何能提供 nvcc 12.8 的安装方式都可以,Torch 是 cu128 版所以 toolkit 主版本要对上 |
| apt 包 | `build-essential cmake ninja-build ffmpeg libeigen3-dev libosmesa6-dev libegl1-mesa-dev libgl1-mesa-dev libglfw3-dev` |
| apt 包 | `build-essential cmake ninja-build ffmpeg libeigen3-dev libosmesa6-dev libegl1-mesa-dev libgl1-mesa-dev libglfw3-dev libboost-system-dev libboost-program-options-dev` |
| 渲染 | 离屏 `MUJOCO_GL=osmesa`(默认);交互回放 `glfw`;Dyn-HaMR 可视化用 EGL |
`scripts/foundationpose_env.sh` 会清空 ROS 注入的 `PYTHONPATH / LD_LIBRARY_PATH`,默认选择项目 `.venv`,从这个环境发现 NVIDIA 包头文件。toolkit 优先使用调用者设置的 `CUDA_HOME`,其次使用项目内 toolkit,再从 `PATH` 的 `nvcc` 定位;缺失时明确报错。GPU 编译架构默认由 PyTorch 检测,显式设置的 `TORCH_CUDA_ARCH_LIST` 会保留。无需复制开发者的 Python 环境。**所有 `.venv` 脚本都先 `source` 它**。
@@ -33,10 +33,10 @@
```bash
python3.11 -m venv .venv
.venv/bin/python -m pip install torch==2.7.1 torchvision==0.22.1 --index-url https://download.pytorch.org/whl/cu128
.venv/bin/python -m pip install -r requirements/pipeline_venv.txt
.venv/bin/python -m pip install -c requirements/torch_main_constraints.txt -r requirements/pipeline_venv.txt
# nvcc 不在 PATH 中时,先 export CUDA_HOME=/path/to/your/cuda-toolkit
source scripts/foundationpose_env.sh # 默认使用 .venv/bin/python,检查本机 nvcc
python -m pip install --no-build-isolation "git+https://github.com/facebookresearch/pytorch3d.git@stable" # 观测 0.7.9
python -m pip install --no-build-isolation "git+https://github.com/facebookresearch/pytorch3d.git@33824be3cbc87a7dd1db0f6a9a9de9ac81b2d0ba" # 观测 0.7.9
python -m pip install --no-build-isolation --no-deps ./third_party/nvdiffrast # 快照 0.4.0,需 nvcc
# FoundationPose C++ 旋转聚类模块(pybind11),产物 third_party/FoundationPose/mycpp/build/mycpp.cpython-311-*.so
cmake -S third_party/FoundationPose/mycpp -B third_party/FoundationPose/mycpp/build -DCMAKE_BUILD_TYPE=Release \
@@ -51,22 +51,35 @@ python scripts/verify_foundationpose_setup.py # GPU 光栅化 + 两个网络
- BundleSDF 的 `mycuda` / NeRF 扩展**不需要**(只跑模型法 STL 路径),未编译。
- HandFlow / ViPE 单目分支也用这个环境,但本链路不需要它们(见第 4.4 节)。详细说明:[FOUNDATIONPOSE_SETUP.md](FOUNDATIONPOSE_SETUP.md)。
### 3.2 Dyn-HaMR `.dynhamr`
### 3.2 Dyn-HaMR `.dynhamr`(RGB-D / YOLO 路径)
**不要照搬旧 `third_party/Dyn-HaMR/scripts/install_pip.sh` 来安装本链路。** 它为 Torch 1.13/cu117 的上游路径准备,torch-scatter wheel 索引也绑定该旧运行时;DROID-SLAM 扩展不用于这里的 RGB-D 相机路径。旧脚本仅保留为历史分支入口,并已修正 Linux 下 `hamer` 目录大小写。
下面是根据 2026-10-29 隔离依赖副本整理的**候选安装步骤,尚未从零实装验收**。不要把它当成完整依赖 lock 或已通过的安装证明:
```bash
python3.10 -m venv third_party/Dyn-HaMR/.dynhamr
third_party/Dyn-HaMR/.dynhamr/bin/python -m pip install torch==2.7.0 torchvision==0.22.0 --index-url https://download.pytorch.org/whl/cu128
# 其余依赖按 third_party/Dyn-HaMR/scripts/install_pip.sh 逐条执行(跳过它里面的 torch 行;该脚本钉的是旧版 torch)
third_party/Dyn-HaMR/.dynhamr/bin/python -m pip install -c requirements/torch_dynhamr_constraints.txt -r requirements/dynhamr_rgbd_observed.txt
# 先安装上面的 NumPy/setuptools,再构建两个纯 Python 来源依赖:
third_party/Dyn-HaMR/.dynhamr/bin/python -m pip install --no-build-isolation --no-deps \
'git+https://github.com/nghorbani/configer@8cd1e3e556d9697298907800a743e120be57ac36' \
'git+https://github.com/mattloper/chumpy@580566eafc9ac68b2614b64d6f7aaa84eebb70da'
# 若启用 run_vis=True 的 PyTorch3D 渲染,需在此 Python 3.10 环境单独构建,不能借用 .venv 的 .so:
third_party/Dyn-HaMR/.dynhamr/bin/python -m pip install --no-build-isolation \
'git+https://github.com/facebookresearch/pytorch3d.git@33824be3cbc87a7dd1db0f6a9a9de9ac81b2d0ba'
```
Dyn-HaMR 自带 `third-party/hamer`(含 ViTPose)与 `third-party/Hand-BMC-pytorch-main`,都在快照里;运行时 `PYTHONPATH` 由 `scripts/run_dynhamr_rgbd_20260915.py` 设置,不需要手动 export。本快照对上游的修改(`run_opt.py`、`vis/tools.py`、`vis/viewer.py` 等 6 处 + 新增 `HMP/windowed.py`)已包含在源码里,见 `UPSTREAM_SOURCES.json` 的 `local_changes`。
本集成的 HaMeR `run.py` 使用 YOLO,不调用 Detectron2/ViTPose。源码随仓库保留,`run_dynhamr_rgbd_20260915.py` 会设置自己项目中的 `PYTHONPATH`,无需把其他环境的 site-packages 写入 `.pth`。可选的旧 ViTPose/DROID 分支仍需其原有依赖。
来源 commit 已固定到可追踪值;编译兼容性和完整安装仍需在目标机器验证。候选清单不能替代本文第 6 节检查和完整流程运行。测试时应单独记录 Python 版本、每个新安装包、所有源码补丁及退出码。
### 3.3 重定向 `.dex`
```bash
python3.11 -m venv .dex
.dex/bin/python -m pip install torch==2.7.1 torchvision==0.22.1 --index-url https://download.pytorch.org/whl/cu128
.dex/bin/python -m pip install -r requirements/retarget.txt # dex-retargeting 0.5.0、mujoco 3.5.0、h5py 等
.dex/bin/python -m pip install -c requirements/torch_main_constraints.txt -r requirements/retarget.txt # dex-retargeting 0.5.0、mujoco 3.5.0、h5py 等
```
### 3.4 SPIDER `.spider`
@@ -84,98 +97,28 @@ env PYTHONPATH= LD_LIBRARY_PATH= .spider/bin/python scripts/verify_spider_instal
## 4. 权重与外部资源
### 4.0 从代码托管平台克隆(权重已随仓库,Git LFS)
仓库 `https://gitea.robotquan.com/liyang/hand-motion-pipeline` 通过 Git LFS 带了下表中"本链路需要"的全部权重、L20 模型与录制数据(MANO 除外)。服务器单次请求上限 50 MB,超过的 10 个文件以 48 MiB 分片 `*.part-NNN` 存放,清单在 `configs/large_files.json`。克隆后先拼回:
当前版本不再通过 Git/LFS 分发预训练权重、MANO 相关参数和 BMC 统计数据。模型由接收方自行向提供者获取;文件清单、上游下载入口、目标路径和参考哈希见 [EXTERNAL_RESOURCES.md](EXTERNAL_RESOURCES.md) 与 [configs/external_resources.json](../configs/external_resources.json)。
```bash
git lfs install
GIT_LFS_SKIP_SMUDGE=1 git clone https://gitea.robotquan.com/liyang/hand-motion-pipeline.git
GIT_LFS_SKIP_SMUDGE=1 git clone --depth 1 https://gitea.robotquan.com/liyang/hand-motion-pipeline.git
cd hand-motion-pipeline
git lfs pull # 约 6.3 GB
python3 scripts/large_files.py assemble # 拼回 hamer.ckpt / UniDepth / HandFlow / HMP / FoundationPose / detector,逐个 sha256 校验
git lfs pull # 当前版本的录制、CAD、机器人网格等资产
python3 scripts/check_pipeline_resources.py --prepare-example --recording-only
mkdir -p output
```
资源检查命令会为嵌套录制创建缺失的相对链接,将 `docs/20260915_171525/20260915_171525/{color.mp4,depth/}` 暴露到脚本读取的外层目录;已有文件不覆盖、源数据不移动。标准布局不改动;未拉取的 LFS 指针和缺失深度帧会报错。
`third_party/Dyn-HaMR/_DATA/` 保留资源相对软链接,MANO 目标目录尚无模型。LFS 拉取与组装后,还需按第 4.1 节准备 MANO,在目标机器安装环境、构建扩展。`assemble --clean` 仅是可选的本地磁盘整理操作,日常部署可保留分片。
**当前 Git LFS 仓库与源码压缩包是两种交付方式。** 仓库已包含主流程权重、L20、录制与 CAD;`package_source_release.py` 导出的源码包仅含代码、配置、文档和小 CAD,其他资源需另行恢复。下表列资源位置与来源,不表示 Git LFS 接收方必须重复下载。`EXTERNAL_ASSETS.json` 是原源码包资源记录,当前分片校验以 `configs/large_files.json` 为准。
| 资源 | 大小 | 放置位置 | 来源 | 本链路需要 |
|---|---|---|---|---|
| Dyn-HaMR 数据包 `_DATA/`(`hamer_ckpts/`、`data/mano/`、`hmp_model/`、`BMC/`) | 787 MB | `third_party/Dyn-HaMR/_DATA/` | 上游 `scripts/prepare.sh`(Google Drive)+ HaMeR `fetch_demo_data.sh` | 是 |
| `MANO_RIGHT.pkl`,以及需要独立左手模型时的 `MANO_LEFT.pkl` | — | `third_party/hamer/_DATA/data/mano/`(Dyn-HaMR 软链接共用) | [MANO 官网](https://mano.is.tue.mpg.de/) 自行取得;未上传 | 是(左手文件按分支准备) |
| WiLoR 手检测器 `detector.pt` | 52 MB | `weights/detector.pt` + `third_party/Dyn-HaMR/third-party/hamer/pretrained_models/detector.pt` | [WiLoR HF Space](https://huggingface.co/spaces/rolpotamias/WiLoR/tree/main/pretrained_models) | 是 |
| FoundationPose 权重:`2023-10-28-18-33-37/`(refiner,68 MB)、`2024-01-11-20-02-45/`(scorer,190 MB),各含 `config.yml + model_best.pth` | 247 MB | `third_party/FoundationPose/weights/` | `scripts/download_foundationpose_weights.py`(清单 `configs/foundationpose_weights_manifest.json`,SHA-256 校验) | 是 |
| L20 灵巧手模型:`L20/{LEFT,RIGHT}/linkerhand_g20_{left,right}.urdf` + `meshes/` + 标定 JSON | 36 MB | `third_party/l20_assets/` | 模型提供方交付;左右手各自的 URDF,不是镜像 | 是 |
| 录制数据 `color.mp4` + `depth/*.png` + `intrinsics.json` | 116 MB | `docs/20260915_171525/` | 自采(D405 头戴) | 是 |
| 盒子 CAD `上半.stl`(红)、`下半.stl`(蓝) | 104 KB | `docs/` | **已随源码包** | 是 |
| HandFlow 权重(`handflow_denoiser.pt`、`normalization_stats.npz`、UniDepth) | 2 GB | `weights/` | [mxxu00/HandFlow](https://huggingface.co/mxxu00/HandFlow)、HF `lpiccinelli/unidepth-v2-vitl14` | 否(HandFlow 单目分支) |
| HaMeR 独立副本 `_DATA/` | 2.6 GB | `third_party/hamer/_DATA/` | HaMeR `fetch_demo_data.sh` | 否(HandFlow 分支) |
| ViPE 模型缓存(SAM、DeAOT、GroundingDINO、UniDepth、DROID、GeoCalib、Depth-Anything) | ~4 GB | `.torch_cache/`、`.hf_cache/` | 见 4.4 | 否(单目无深度视频才需要) |
| 瓶子参数化模型 | 156 KB | `third_party/bottle_model_parametric/` | 项目附件 | 否(旧单手链路) |
### 4.1 Dyn-HaMR 数据包、MANO、检测器
**Git LFS 部署无需重新运行上游下载脚本覆盖已有资源。** 完成第 4.0 节后,将按 MANO 官方条件取得的模型放入共享目录:
```text
third_party/hamer/_DATA/data/mano/MANO_RIGHT.pkl
third_party/hamer/_DATA/data/mano/MANO_LEFT.pkl # 使用独立左手模型的分支需要
```
`third_party/Dyn-HaMR/_DATA/data/mano` 是指向这里的相对软链接,默认 `MANO_ROOT` 也指向这里。修改 `MANO_ROOT` 只影响支持该变量的入口,不能替代 Dyn-HaMR 固定目录或其软链接。模型不在 Git / LFS 中,无需上传开发者主目录或完整 Python 环境。
```bash
python3 scripts/external_resources.py list --scope all
# 按显示的上游入口取得文件,恢复清单中的相对目录,然后:
python3 scripts/external_resources.py check
python3 scripts/external_resources.py check --verify-sha256
python3 scripts/check_pipeline_resources.py
# 使用独立左手模型的分支:
python3 scripts/check_pipeline_resources.py --require-left-mano
```
以下下载方式只用于不含资源的源码包部署或缺失资源恢复;先保留已有资源,按需执行:
MANO 请自行申请放入 `third_party/hamer/_DATA/data/mano/`;Dyn-HaMR 的相对软链接共用该目录。HaMeR checkpoint 本身含 MANO 参数项,因此也改成从上游获取。BMC 需要按上游数据集条件下载原始关节数据后生成统计文件,未验证与开发快照完全相同的预计算数据直链。
```bash
cd third_party/Dyn-HaMR && bash scripts/prepare.sh && cd ../.. # 需要 gdown;下载后核对 _DATA 内四个子目录都非空
# MANO:按上方说明放到 third_party/hamer/_DATA/data/mano/
mkdir -p weights third_party/Dyn-HaMR/third-party/hamer/pretrained_models
curl -fL --retry 3 -C - 'https://huggingface.co/spaces/rolpotamias/WiLoR/resolve/main/pretrained_models/detector.pt' -o weights/detector.pt
cp weights/detector.pt third_party/Dyn-HaMR/third-party/hamer/pretrained_models/detector.pt
```
FoundationPose 请从官方研究版入口取得两个 run 目录;旧哈希来自此前使用的社区镜像,未独立与官方文件比对。`python3 scripts/download_foundationpose_weights.py` 只显示官方入口,`--check` 检查本地文件,不再自动下载社区镜像。
`prepare.sh` 还会拉旧的 DROID 资源,可按脚本逐项取舍。不能用"脚本退出码 0"代替 checkpoint 存在性检查。
`large_files.py assemble` 只对 `configs/large_files.json` 中仍列出的可分发资产有效;当前模型分片已移出该清单。旧 Git 提交和服务端 LFS 对象没有通过本次普通提交清除,详见 [历史版本说明](EXTERNAL_RESOURCES.md#历史版本)。
### 4.2 FoundationPose 权重
Git LFS 拉取并组装成功后可跳过本节下载;以下用于源码包部署或缺失文件恢复。
```bash
source scripts/foundationpose_env.sh
python scripts/download_foundationpose_weights.py
```
官方 Google Drive 在制作快照时配额超限,清单里的 URL 是社区 Hugging Face 镜像 `gpue/foundationpose-weights`,按镜像 LFS SHA-256 校验,**未与官方文件独立比对**。有官方链接时替换清单里的 `url` 即可,哈希不变。
### 4.3 L20 模型与录制数据
Git LFS 仓库已包含左右手 URDF、网格、标定文件和示例录制。先拉取 LFS,再执行第 4.0 节示例准备命令。只有源码包部署需要单独恢复这些资源,原资源哈希见 `EXTERNAL_ASSETS.json`。自己的录制使用 `docs/<seq>/{color.mp4,depth/,intrinsics.json}` 布局,并按 PIPELINE 适配序列、帧数和 CAD 映射。
### 4.4 可选:HandFlow 与 ViPE 分支
只在跑单目 RGB 视频(没有深度、没有里程计)时需要。本链路的相机位姿来自 `prepare_rgbd_20260915.py` 的 RGB-D 里程计,只借用 ViPE 的**文件格式**喂给 Dyn-HaMR,不运行 ViPE。
- HandFlow:`hf download mxxu00/HandFlow handflow_denoiser.pt normalization_stats.npz --local-dir weights`
- ViPE 权重(本快照 `configs/pipeline/default.yaml` 所需):SAM ViT-B(`.torch_cache/hub/sam/`)、DeAOT(`.torch_cache/hub/aot/`)、GroundingDINO(HF `ShilongLiu/GroundingDINO`)、BERT(HF `google-bert/bert-base-uncased`)、UniDepth-L(HF `lpiccinelli/unidepth-v2-vitl14`)、DROID(`.torch_cache/hub/droid_slam/droid.pth`)、GeoCalib、Depth-Anything 系列。做法:设置 `configs/project_env.example.sh` 的缓存变量,联网跑一段短视频让它按需下载,全部就绪后再开 `HF_HUB_OFFLINE=1`。搬缓存要连 `snapshots/blobs/refs` 一起搬。
### 4.5 下载慢或中断
```bash
curl -L --range 0-1048575 --max-time 15 -o /dev/null -w 'HTTP=%{http_code} speed=%{speed_download} bytes/s\n' '<URL>'
```
Hugging Face 可换 `hf-mirror.com` 测速(第三方镜像)。支持 Range 时用 `curl -C -` 或 `aria2c -c -x 16 -s 16 --min-split-size=10M` 续传。拿到 HTML 错误页时文件名再对也不算成功。
只用单目分支时,还需按各提供方条件准备 ViPE 缓存(SAM、DeAOT、GroundingDINO、BERT、UniDepth、DROID、GeoCalib、Depth-Anything 等);RGB-D 路径不运行 ViPE。联网下载成功后再设置离线标志,迁移 HF 缓存时保留 snapshots/blobs/refs 结构。不要把离线启动失败误当作模型已准备完成。
## 5. 路径与环境变量
@@ -230,6 +173,8 @@ git remote add origin <仓库地址> && git push -u origin main
- 没有干净机器的端到端安装复现记录;版本表是本机观测。
- SPIDER 在本机是 Python 3.11 兼容安装,上游声明 ≥ 3.12。
- FoundationPose 权重来自社区镜像,未与官方文件比对。
- 参考 FoundationPose 哈希来自历史社区镜像,当前分发方式为使用者从官方获取,尚未独立核对官方文件。
- 脚本序列名写死为 `20260915_171525`;`PORTABILITY_REPORT.json` 列出全部此类行。
- 交付的是运动学参考与 SPIDER 物理审计结果;**未验证真机**,抓取未复现(见 PIPELINE_LATEST.md 现状)。
2026-10-29 的组件运行验证与未验证项目见 [VALIDATION_20261029.md](VALIDATION_20261029.md)。
+35
View File
@@ -0,0 +1,35 @@
# 2026-10-29 组件运行验证
审核源码基线:`0aa2f49ab8ebafe9fdef816ceeb871407abbc29e`。验证机器为 Ubuntu / RTX 3080 Laptop 16 GB,驱动 595.99.02。检查目标是代码可运行和输出结构正确,不要求物体抓取成功。
## 验证方式
独立源码副本使用四个独立解释器。为避免重复下载 Torch/CUDA,依赖通过本机已有包的复制/硬链接快照提供;没有在解释器搜索路径中借用开发者原环境的 site-packages。基础 Python、系统库、驱动仍来自同一台机器,MANO 在本机私下提供,没有提交。
**这不是从零 pip 安装的验证,也不是完整流程串联验证。** 新增的候选安装清单尚未实装;PyTorch3D 重新固定的源码 commit 尚未重新在四个环境构建,不能倒推其兼容性已验收。
## 实际结果
| 检查 | 结果 | 边界 |
|---|---|---|
| 匿名仓库 / LFS 批量元数据 | 551 个旧版 LFS 对象均可返回 download action | 元数据存在不等于全部 payload 重新下载;当前模型资源已在本次更新中外移 |
| 示例与 CAD 内容 | 视频、352 张深度图、2 个 CAD,共 355 个 LFS payload SHA256 与 OID 一致 | 不证明数据许可或测量真值 |
| 原组装权重 | 10 个本地权重大小与 SHA256 匹配开发清单 | 复用本地文件,不证明官方来源一致或再分发权 |
| RGB-D 预处理 | 原脚本完成 352 帧,写出相机与检查报告 | 里程计估计,不是外部位姿真值 |
| HaMeR / Dyn-HaMR | `data=video_vipe` 入口解析;一次 GPU HaMeR 前向输出 `[1,21,3]` 关节与 `[1,778,3]` 顶点,数值有限 | 零图块前向;未跑时序拟合或运动先验 |
| L20 重定向 | 8 帧真实保留左手数据,限位/mimic 检查通过,FK 差约 `1.55e-13 m` | 输入来自先前保留结果,不是本次新跑 Dyn-HaMR 输出 |
| FoundationPose | mycpp 重新 CMake 编译;GPU 光栅化、两网络前向、CAD 初始化通过 | 尚未对录制帧运行 6D 估计/跟踪 |
| SPIDER | 两个 GPU 世界、60 步接触仿真通过 | 球体测试,非 L20 手物优化;用历史 Python 3.11 兼容依赖,非从零 Python 3.12 安装 |
后续仓库变更的资源指引和发布检查由 `tests/` 验证;运行 `python3 -m unittest discover -s tests -v`。机器可读的组件摘要见 [validation/20261029.json](validation/20261029.json)。
## 未验收项
- 四环境的从零安装,尤其 SPIDER Python ≥3.12 路径。
- Dyn-HaMR 时序拟合与 HMP 运动先验。
- 录制视频的 FoundationPose 6D 估计/跟踪。
- 在新任务中串联手物配准、接触/碰撞修复、L20 SPIDER 优化与 HDF5 导出。
- 完整模型与数据许可、完整 Git 历史/二进制秘密扫描。
- 抓取/抬升成功、真机验证、力学标定。
建议后续加统一短流程 smoke 配置:真实模型和输入、少帧/少迭代/少物理步数;逐阶段要求退出码为 0、输出字段与形状正确、数值有限、场景和 HDF5 可加载。抓取成功作为另外的任务指标。
+60
View File
@@ -0,0 +1,60 @@
{
"date": "2026-10-29",
"source_baseline": "0aa2f49ab8ebafe9fdef816ceeb871407abbc29e",
"method": "Independent source/interpreters with reused installed dependency payloads; shared host Python/system libraries/GPU driver. No Torch/CUDA download.",
"clean_install_verified": false,
"full_pipeline_verified": false,
"checks": {
"sample_lfs_payload_sha256": 355,
"local_assembled_weights_sha256": 10,
"rgbd_frames": 352,
"hamer_forward": {
"passed": true,
"joints": [
1,
21,
3
],
"vertices": [
1,
778,
3
],
"input": "zero crop"
},
"l20_retarget": {
"passed": true,
"frames": 8,
"side": "left",
"input": "previously retained local sequence",
"limit_violation_rad": 0,
"mimic_error_rad": 0,
"pinocchio_mujoco_fk_error_m": 1.547373340571312e-13
},
"foundationpose": {
"passed": true,
"scope": [
"fresh C++ build",
"GPU rasterization",
"score/refine forward",
"CAD estimator initialization"
]
},
"spider": {
"passed": true,
"worlds": 2,
"steps": 60,
"scene": "sphere contact, not hand/object optimization",
"python": "3.11 compatibility dependency snapshot"
}
},
"not_verified": [
"clean dependency install",
"SPIDER clean Python 3.12 install",
"Dyn-HaMR temporal/prior fitting",
"recorded-video 6D estimation",
"full hand/object contact/physics/HDF5 chain",
"grasp success",
"hardware"
]
}
+39
View File
@@ -0,0 +1,39 @@
# Candidate RGB-D/YOLO dependency recipe from the isolated 2026-10-29 snapshot.
# Not a clean-install lock. Torch is installed separately with torch_dynhamr_constraints.txt.
# Configer/chumpy source commits are installed separately after NumPy/setuptools.
# This path does not use DROID-SLAM, torch-scatter, Detectron2 or ViTPose.
numpy==2.2.6
scipy==1.15.3
pandas==2.3.3
ultralytics==8.1.34
hydra-core==1.3.6
hydra-colorlog==1.2.0
hydra-submitit-launcher==1.2.0
omegaconf==2.3.1
smplx==0.1.28
timm==0.4.9
pytorch-lightning==2.6.6
torchmetrics==0.11.4
yacs==0.1.8
opencv-python==5.0.0.93
open3d==0.19.0
pyrender==0.1.45
setuptools==59.5.0
wheel
dill==0.4.1
motmetrics==1.4.0
einops==0.8.2
scikit-image==0.25.2
joblib==1.6.0
gdown==6.2.0
tensorboard==2.21.0
rich==14.3.4
webdataset==1.0.2
pyrootutils==1.0.4
imageio-ffmpeg
matplotlib
PyYAML
tqdm
psutil
trimesh
h5py
+2 -2
View File
@@ -4,8 +4,8 @@
#
# Install order:
# 1. torch==2.7.1 torchvision==0.22.1 --index-url https://download.pytorch.org/whl/cu128 (must match driver/CUDA)
# 2. pip install -r requirements/pipeline_venv.txt
# 3. pip install --no-build-isolation git+https://github.com/facebookresearch/pytorch3d.git@stable (needs nvcc; observed 0.7.9)
# 2. pip install -c requirements/torch_main_constraints.txt -r requirements/pipeline_venv.txt
# 3. pip install --no-build-isolation git+https://github.com/facebookresearch/pytorch3d.git@33824be3cbc87a7dd1db0f6a9a9de9ac81b2d0ba (needs nvcc; observed 0.7.9)
# 4. pip install --no-build-isolation --no-deps ./third_party/nvdiffrast (needs nvcc; snapshot 0.4.0)
# 5. build FoundationPose mycpp (see docs/SETUP_AND_WEIGHTS.md)
numpy==2.4.4
@@ -0,0 +1,2 @@
torch==2.7.0+cu128
torchvision==0.22.0+cu128
+3
View File
@@ -0,0 +1,3 @@
# Install Torch first from the cu128 index; keep dependency resolution on this runtime.
torch==2.7.1+cu128
torchvision==0.22.1+cu128
+30 -3
View File
@@ -7,6 +7,10 @@ This never downloads models, overwrites existing paths, or loads pickle files.
import argparse
import json
from pathlib import Path
import sys
sys.path.insert(0, str(Path(__file__).resolve().parent))
from external_resources import load_resources
ROOT = Path(__file__).resolve().parents[1]
LFS_HEADER = b'version https://git-lfs.github.com/spec/v1'
@@ -85,18 +89,41 @@ def check_assets(root, require_left=False):
for checkpoint in ('2023-10-28-18-33-37', '2024-01-11-20-02-45'):
required.extend(f'third_party/FoundationPose/weights/{checkpoint}/{name}' for name in ('config.yml', 'model_best.pth'))
errors = []
external = {}
external_entries = {}
if (root / 'configs/external_resources.json').is_file():
for group in load_resources(root):
if group['scope'] == 'rgbd':
for entry in group['files']:
external[entry['path']] = group
external_entries[entry['path']] = entry
required = list(dict.fromkeys([*required, *external]))
by_id = {group['id']: group for group in load_resources(root)}
aliases = {
'third_party/Dyn-HaMR/third-party/hamer/pretrained_models/detector.pt': 'detector',
'third_party/Dyn-HaMR/_DATA/BMC/joint_angles.npy': 'bmc',
}
for path, group_id in aliases.items():
if group_id in by_id:
external[path] = by_id[group_id]
manifest_path = root / 'configs/large_files.json'
manifest = json.loads(manifest_path.read_text()) if manifest_path.is_file() else {'files': []}
expected = {entry['path']: entry['bytes'] for entry in manifest['files']}
expected.update({path: entry['bytes'] for path, entry in external_entries.items() if 'bytes' in entry})
for relative in required:
path = root / relative
if not path.is_file() or path.stat().st_size == 0:
action = 'Provide your MANO model; see docs/SETUP_AND_WEIGHTS.md.' if path.suffix == '.pkl' else 'Run git lfs pull and scripts/large_files.py assemble.'
if relative in external:
group = external[relative]
action = f"Obtain locally from {group['source_url']}; see docs/EXTERNAL_RESOURCES.md."
else:
action = 'Provide your MANO model; see docs/EXTERNAL_RESOURCES.md.' if path.suffix == '.pkl' else 'Run git lfs pull for redistributable assets.'
errors.append(f'Missing/empty {relative}. {action}')
elif is_pointer(path):
errors.append(f'Git LFS pointer: {relative}; run git lfs pull and assemble.')
action = f"Obtain from {external[relative]['source_url']}" if relative in external else 'Run git lfs pull'
errors.append(f'Git LFS pointer: {relative}; {action}.')
elif relative in expected and path.stat().st_size != expected[relative]:
errors.append(f'Wrong size: {relative}; reassemble and verify SHA-256 with scripts/large_files.py assemble.')
errors.append(f'Wrong reference size: {relative}; check configs/external_resources.json and run external_resources.py check --verify-sha256. Regenerated BMC data may differ; review explicitly.')
for side in ('LEFT', 'RIGHT'):
directory = root / f'third_party/l20_assets/L20/{side}/meshes'
meshes = list(directory.rglob('*.stl')) + list(directory.rglob('*.STL'))
+30
View File
@@ -0,0 +1,30 @@
#!/usr/bin/env python3
"""Reject model files and their shards in the current Git index; does not scan history."""
import json
from pathlib import Path
import subprocess
ROOT = Path(__file__).resolve().parents[1]
def forbidden_paths(tracked, resources):
files = {entry['path'] for group in resources for entry in group['files']
if not entry['path'].endswith('/config.yml')}
return [name for name in tracked
if name in files or any(name.startswith(path + '.part-') for path in files)
or Path(name).name in {'MANO_RIGHT.pkl', 'MANO_LEFT.pkl'}]
def main():
resources = json.loads((ROOT / 'configs/external_resources.json').read_text())['resources']
tracked = subprocess.check_output(['git', 'ls-files', '-z'], cwd=ROOT).decode().split('\0')
bad = forbidden_paths(tracked, resources)
for name in bad:
print(f'FAIL externally acquired resource is tracked: {name}')
if not bad:
print('PASS current Git index excludes external model payloads and shards; historical commits/LFS objects are not checked or removed.')
return int(bool(bad))
if __name__ == '__main__':
raise SystemExit(main())
+24 -31
View File
@@ -1,33 +1,26 @@
"""Download and verify the FoundationPose checkpoints listed in configs/foundationpose_weights_manifest.json.
#!/usr/bin/env python3
"""Show official FoundationPose sources or check local files; never download implicitly."""
import argparse
from external_resources import ROOT, check_resources, select_resources
Files go to third_party/FoundationPose/weights/<run>/{config.yml, model_best.pth}; resumable (Range requests), SHA-256 checked.
The manifest URLs point at a community Hugging Face mirror (official Google Drive quota was exceeded when the snapshot was made);
replace `url` entries with the official links if you have them. Usage: .venv/bin/python scripts/download_foundationpose_weights.py
"""
import hashlib, json, time
from pathlib import Path
import requests
ROOT = Path(__file__).resolve().parents[1]
manifest = json.loads((ROOT / 'configs/foundationpose_weights_manifest.json').read_text())
dest = ROOT / 'third_party/FoundationPose/weights'
for item in manifest['files']:
path = dest / item['path']; path.parent.mkdir(parents=True, exist_ok=True); tmp = path.with_suffix(path.suffix + '.part')
if path.exists() and path.stat().st_size == item['size'] and hashlib.sha256(path.read_bytes()).hexdigest() == item['sha256']:
print('ok ', item['path']); continue
for attempt in range(5):
try:
offset = tmp.stat().st_size if tmp.exists() else 0
print(f'download {item["path"]} from byte {offset}', flush=True)
with requests.get(item['url'], headers={'Range': f'bytes={offset}-{item["size"] - 1}'}, stream=True, timeout=(20, 60)) as r:
r.raise_for_status()
if offset and r.status_code != 206: offset = 0
with tmp.open('ab' if offset else 'wb') as f:
for chunk in r.iter_content(1024 * 1024): f.write(chunk)
if tmp.stat().st_size == item['size']: tmp.replace(path); break
except requests.RequestException as e:
print('retry:', type(e).__name__, flush=True); time.sleep(3)
assert path.exists() and path.stat().st_size == item['size'], f'incomplete: {path}'
digest = hashlib.sha256(path.read_bytes()).hexdigest(); assert digest == item['sha256'], f'sha256 mismatch: {path}'
print('verified', item['path'], flush=True)
(dest / 'download_manifest.json').write_text(json.dumps(manifest, indent=2)); print('all FoundationPose weights present in', dest)
def main():
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument('--check', action='store_true')
parser.add_argument('--verify-sha256', action='store_true')
args = parser.parse_args()
if args.check:
rows = check_resources(ROOT, group_ids=['foundationpose'], verify_sha256=args.verify_sha256)
for row in rows:
print(f"{row['status']:16} {row['path']}")
return int(any(row['status'] != 'present' for row in rows))
group = select_resources(ROOT, group_ids=['foundationpose'])[0]
print(group['source_url'])
print(group['download_url'])
print(group['instructions'])
print('No files downloaded. After acquiring the files: python3 scripts/download_foundationpose_weights.py --check')
return 0
if __name__ == '__main__':
raise SystemExit(main())
+98
View File
@@ -0,0 +1,98 @@
#!/usr/bin/env python3
"""List upstream model sources or check locally acquired files; never download models."""
import argparse
import hashlib
import json
from pathlib import Path
ROOT = Path(__file__).resolve().parents[1]
LFS_HEADER = b'version https://git-lfs.github.com/spec/v1'
def load_resources(root):
data = json.loads((root / 'configs/external_resources.json').read_text())
for group in data['resources']:
for entry in group['files']:
relative = Path(entry['path'])
if relative.is_absolute() or '..' in relative.parts:
raise ValueError(f'Unsafe resource path: {relative}')
return data['resources']
def select_resources(root, scope='rgbd', group_ids=None):
groups = load_resources(root)
ids = set(group_ids or [])
unknown = ids - {g['id'] for g in groups}
if unknown:
raise ValueError(f'Unknown resource groups: {sorted(unknown)}')
return [g for g in groups if (g['id'] in ids if ids else scope == 'all' or g['scope'] == 'rgbd')]
def file_status(path, entry, verify_sha256=False):
if not path.is_file():
return 'missing'
if path.stat().st_size == 0:
return 'empty'
with path.open('rb') as stream:
if stream.read(len(LFS_HEADER)) == LFS_HEADER:
return 'lfs_pointer'
if 'bytes' in entry and path.stat().st_size != entry['bytes']:
return 'size_mismatch'
if verify_sha256 and entry.get('sha256'):
digest = hashlib.sha256()
with path.open('rb') as stream:
for block in iter(lambda: stream.read(8 * 1024 * 1024), b''):
digest.update(block)
if digest.hexdigest() != entry['sha256']:
return 'sha256_mismatch'
return 'present'
def check_resources(root, scope='rgbd', group_ids=None, verify_sha256=False):
rows = []
for group in select_resources(root, scope, group_ids):
for entry in group['files']:
rows.append(dict(group=group['id'], path=entry['path'],
status=file_status(root / entry['path'], entry, verify_sha256),
source_url=group['source_url']))
return rows
def main():
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument('command', choices=['list', 'check'])
parser.add_argument('--scope', choices=['rgbd', 'all'], default='rgbd')
parser.add_argument('--group', action='append', help='Select a group; repeat to select several.')
parser.add_argument('--verify-sha256', action='store_true', help='Compare with the reference snapshot, including large weights.')
parser.add_argument('--json', action='store_true')
args = parser.parse_args()
try:
if args.command == 'list':
groups = select_resources(ROOT, args.scope, args.group)
if args.json:
print(json.dumps(groups, indent=2))
else:
for group in groups:
print(f"[{group['id']}] {group['name']}\n {group['source_url']}")
if group.get('download_url'):
print(f" Download: {group['download_url']}")
print(f" {group['instructions']}")
for entry in group['files']:
print(f" -> {entry['path']}")
return 0
rows = check_resources(ROOT, args.scope, args.group, args.verify_sha256)
if args.json:
print(json.dumps(rows, indent=2))
else:
for row in rows:
print(f"{row['status']:16} {row['path']}")
if row['status'] != 'present':
print(f" Obtain from: {row['source_url']}")
print('Scope: local files only; model loading and full-pipeline execution require separate checks.')
return int(any(row['status'] != 'present' for row in rows))
except (OSError, ValueError, KeyError) as exc:
parser.exit(2, f'{exc}\n')
if __name__ == '__main__':
raise SystemExit(main())
+10 -2
View File
@@ -1,4 +1,6 @@
"""Split / reassemble files larger than the hosting platform's 50 MB per-request limit.
"""Historical chunk helper for explicitly redistributable large assets.
Current pretrained models are external: see scripts/external_resources.py and docs/EXTERNAL_RESOURCES.md.
The Gitea server rejects any single upload above 50 MB (HTTP 413), so weights above that size are stored in
the repository as 48 MB chunks <file>.part-000, .part-001, ... (each chunk is a Git LFS object) together with
@@ -49,6 +51,9 @@ def split(paths):
def assemble(clean=False):
m = load(); bad = 0
if not m['files']:
print('No bundled chunks to assemble. Obtain external models via scripts/external_resources.py list; see docs/EXTERNAL_RESOURCES.md.')
return
for e in m['files']:
dst = ROOT / e['path']; parts = [dst.parent / f'{dst.name}.part-{i:03d}' for i in range(e['parts'])]
if dst.exists() and dst.stat().st_size == e['bytes'] and sha256(dst) == e['sha256']:
@@ -68,7 +73,10 @@ def assemble(clean=False):
def check():
for e in load()['files']:
files = load()['files']
if not files:
print('No bundled chunk entries. Check externally acquired models with scripts/external_resources.py check.')
for e in files:
dst = ROOT / e['path']
ok = dst.exists() and dst.stat().st_size == e['bytes'] and sha256(dst) == e['sha256']
print('ok ' if ok else 'BAD ', e['path'])
+65
View File
@@ -0,0 +1,65 @@
"""External resources fail clearly; release guards catch model shards."""
import hashlib
import json
from pathlib import Path
import sys
import tempfile
import unittest
sys.path.insert(0, str(Path(__file__).resolve().parents[1] / 'scripts'))
from external_resources import check_resources, load_resources, select_resources
from check_release_policy import forbidden_paths
from check_pipeline_resources import check_assets
class ExternalResourcesTests(unittest.TestCase):
def setUp(self):
self.temp = tempfile.TemporaryDirectory()
self.addCleanup(self.temp.cleanup)
self.root = Path(self.temp.name)
(self.root / 'configs').mkdir()
self.groups = [dict(id='test', name='test', scope='rgbd', source_url='https://example.org/provider',
files=[dict(path='weights/test.pt', bytes=4, sha256=hashlib.sha256(b'good').hexdigest())]),
dict(id='optional', name='optional', scope='optional', source_url='https://example.org/optional',
files=[dict(path='weights/optional.pt')])]
self.write_manifest()
def write_manifest(self):
(self.root / 'configs/external_resources.json').write_text(json.dumps({'resources': self.groups}))
def test_missing_resource_has_upstream_url_and_affects_pipeline_check(self):
row = check_resources(self.root)[0]
self.assertEqual(row['status'], 'missing')
self.assertEqual(row['source_url'], 'https://example.org/provider')
self.assertTrue(any('weights/test.pt' in x and row['source_url'] in x for x in check_assets(self.root)))
def test_pointer_and_same_size_corruption_are_rejected(self):
path = self.root / 'weights/test.pt'
path.parent.mkdir()
path.write_bytes(b'version https://git-lfs.github.com/spec/v1\n')
self.assertEqual(check_resources(self.root)[0]['status'], 'lfs_pointer')
path.write_bytes(b'evil')
self.assertEqual(check_resources(self.root, verify_sha256=True)[0]['status'], 'sha256_mismatch')
path.write_bytes(b'good')
self.assertEqual(check_resources(self.root, verify_sha256=True)[0]['status'], 'present')
def test_optional_groups_are_explicit(self):
self.assertEqual(len(check_resources(self.root)), 1)
self.assertEqual(len(check_resources(self.root, scope='all')), 2)
self.assertEqual(select_resources(self.root, group_ids=['optional'])[0]['id'], 'optional')
with self.assertRaises(ValueError):
select_resources(self.root, group_ids=['unknown'])
def test_manifest_cannot_escape_root(self):
self.groups[0]['files'][0]['path'] = '../outside.pt'
self.write_manifest()
with self.assertRaises(ValueError):
load_resources(self.root)
def test_index_guard_detects_full_models_shards_and_mano(self):
tracked = ['weights/test.pt', 'weights/test.pt.part-000', 'other/MANO_LEFT.pkl', 'docs/sample.mp4']
self.assertEqual(forbidden_paths(tracked, self.groups), tracked[:3])
if __name__ == '__main__':
unittest.main()
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
+3 -1
View File
@@ -1,5 +1,7 @@
#!/usr/bin/env bash
set -e
# Legacy upstream Torch 1.13/cu117 environment only.
# Do not use this script for the integrated RGB-D/cu128 path; see docs/SETUP_AND_WEIGHTS.md.
echo "Creating virtual environment"
python3.10 -m venv .dynhamr
@@ -25,6 +27,6 @@ python setup.py install
cd ../..
# install HaMeR
cd third-party/HaMeR
cd third-party/hamer
pip install -e .[all]
pip install -v -e third-party/ViTPose
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.

Some files were not shown because too many files have changed in this diff Show More