liyang
ae28d55f81
Source: RGB-D -> Dyn-HaMR -> L20 retargeting -> FoundationPose -> reference repair -> SPIDER, documented in docs/PIPELINE_LATEST.md and docs/SETUP_AND_WEIGHTS.md. Adds FoundationPose and nvdiffrast upstream snapshots, requirements/pipeline_venv.txt and the FoundationPose weight manifest/downloader. Assets (Git LFS): weights/ (WiLoR detector, HandFlow denoiser, UniDepth-L), FoundationPose checkpoints, HaMeR checkpoint, Dyn-HaMR HMP model and BMC constraints, L20 URDF/meshes, the 20260915_171525 D405 recording and the two box CADs. MANO models are not redistributed (third_party/hamer/_DATA/data/mano/README.txt). Environments, caches and run outputs excluded. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
18 lines
6.5 KiB
Python
18 lines
6.5 KiB
Python
"""Evidence report for the corrected SPIDER free-object collision run."""
|
|
from pathlib import Path
|
|
import json,hashlib,numpy as np,cv2,mujoco
|
|
ROOT=Path(__file__).resolve().parents[1];O=ROOT/'output/spider_dynamics_fix_20260915';p=O/'datasets/processed/current/l20/bimanual/boxes';v=json.loads((O/'physics_validation.json').read_text());old=json.loads((ROOT/'output/collision_fix_20260915/physics_validation.json').read_text());probe=json.loads((O/'gpu_contact_probe.json').read_text());replay=json.loads((O/'control_replay.json').read_text());reset=json.loads((O/'state_restore_check.json').read_text());surface=json.loads((O/'dynamic_surface_validation.json').read_text());a=np.load(O/'physics_motion.npz');m=mujoco.MjModel.from_xml_path(str(p/'scene_act.xml'))
|
|
qnames=[m.joint(j).name for j in range(m.njnt)];unames=[m.actuator(j).name for j in range(m.nu)];np.savetxt(O/'spider_trajectory.csv',np.c_[a['time'],a['qpos'],a['ctrl']],delimiter=',',header=','.join(['time_s']+qnames+['ctrl_'+n for n in unames]),comments='')
|
|
video=O/'spider_collision_rollout.mp4';cap=cv2.VideoCapture(str(video));fps=cap.get(cv2.CAP_PROP_FPS);n=0
|
|
while True:
|
|
ok,frame=cap.read()
|
|
if not ok:break
|
|
n+=1
|
|
cap.release();integrity={'decoded_frames':n,'fps':fps,'expected_frames':352,'passed':n==352 and abs(fps-30)<1e-6};(O/'video_integrity.json').write_text(json.dumps(integrity,indent=2));assert integrity['passed']
|
|
status='碰撞小于 1 mm 门槛通过' if v['max_penetration_mm']<1 else '碰撞小于 1 mm 门槛未通过'
|
|
lines=['# Spider 动态碰撞继续修复','',f'状态：**{status}；自由物体抓取跟踪 20 mm 门槛：{v["grasp_tracking_20mm_passed"]}。**','',f'完整 {v["steps"]} 步 / {v["duration_s"]:.2f} 秒，所有执行状态有限。输入 `docs/20260915_171525`，两只 L20、红盒和蓝盒。物体执行器在真实执行阶段保持零增益；没有把物体固定到手上。','', '## 完整动态结果','', '| 指标 | 上轮 | 本轮 |','|---|---:|---:|',f'| 最大手盒穿透/mm | {old["max_penetration_mm"]:.3f} | {v["max_penetration_mm"]:.3f} |',f'| 穿透超过 1 mm 的物理步数 | {old["steps_over_1mm"]} | {v["steps_over_1mm"]} |',f'| 红盒有效观测段质心平均误差/mm | {old["upper"]["observed_centroid_error_mean_mm"]:.1f} | {v["upper"]["observed_centroid_error_mean_mm"]:.1f} |',f'| 蓝盒有效观测段质心平均误差/mm | {old["lower"]["observed_centroid_error_mean_mm"]:.1f} | {v["lower"]["observed_centroid_error_mean_mm"]:.1f} |', '', '低穿透不代表抓取成功。物体位姿跟踪仍需单独通过验证；几何接触目标没有触觉真值，手指接触保持和抓取质量尚未解决。', '', '## 已修复的代码问题','', '- 虚拟辅助伺服力改为 `kp*ctrl - kp*q - kd*qvel`，并完整保存和恢复辅助增益。','- 候选轨迹重采样复制包含当前步的控制前缀，避免状态与控制历史不一致（当前任务默认关闭此功能，属于潜在路径修复）。','- 接触奖励实际乘入配置权重；双手状态按含被动关节的自由度划分。','- 新候选批次同步约束求解初值，避免遗留失败候选的 warmstart。','- 零物体辅助断言检查全部仿射偏置项。','', '## GPU 接触数值问题','', '旧的 `solimp=.999 .9999 .0001`、`solref=.005 1` 在当前 GPU 模型中导致蓝盒接触桌面后弹飞；同控制 CPU 重放未出现该早期爆炸。短段对照改为 `solimp=.9 .95 .001 .5 2`、`solref=.02 1`，并以 2 mm 提前量激活接触约束。碰撞网格没有扩大；检测使用实际几何距离。提前生效是数值缓冲，不能视为真实接触位置精度。','', '| 同一短段控制的参数对照 | 最大手盒穿透/mm | 最大物体质心位移/mm |','|---|---:|---:|']
|
|
for x in probe:
|
|
lines.append(f'| {x["name"]} | {x["max_penetration_mm"]:.3f} | {x["max_object_com_displacement_mm"]:.2f} |')
|
|
lines+=['','## 验证与边界','', '- 5 项回归测试通过：伺服平衡/恢复、控制重采样因果性、双手自由度索引、奖励权重和批次 warmstart 同步。',f'- GPU 状态恢复专项：仅状态恢复差异 {reset["state_only_max_qpos_difference"]:.3g}；状态和参数恢复差异 {reset["state_and_parameters_max_qpos_difference"]:.3g}。这些是下一步 qpos 混合米/弧度的最大差异。',f'- 独立重放 {replay["steps"]} 步，首步最大 qpos 差异 {replay["gpu_first_step_max_qpos_difference"]:.3g}，全段最大差异 {replay["gpu_max_qpos_difference"]:.3g}（混合米/弧度）。长段仍不能视为严格复现的控制轨迹。','- 每个保存的物理状态由 CPU 独立碰撞检测审计；凸分解几何仍是近似，未验证连续两步之间或手部自碰撞。',f'- 原始手网格表面独立采样：每帧 {surface["hand_surface_samples_per_frame"]} 点，共 {surface["sampled_frames"]} 帧；红盒最大侵入 {surface["upper"]["max_inside_mm"]:.3f} mm，蓝盒 {surface["lower"]["max_inside_mm"]:.3f} mm。不是完整网格相交证明。',f'- 最大关节限位超出 {v["max_joint_limit_violation_rad"]:.4f} rad；该动态轨迹不应直接用于硬件。','- 物体质量 0.12 kg、摩擦和执行器力限制均为仿真假设。','- 蓝盒第 177 帧起缺少可靠 FoundationPose 观测，沿用保持最后有效位姿的参考，不代表测到了遮挡后的运动。','- FoundationPose 原始位姿、有效标记和运动学防穿透参考沿用上一轮；`reference_inherited_*` 仅为继承证据。','', '## 输出','', '- `spider_collision_rollout.mp4`：原视频 / 防穿透运动学参考 / 本轮真实自由物体动态结果，352 帧已完整解码。','- `physics_validation.json`、`physics_motion.npz`：全物理步穿透与跟踪审计。','- `spider_trajectory.csv`：关节轨迹及控制。','- `gpu_contact_probe.json`：匹配控制的参数对照。','- `control_replay.npz/json`、`state_restore_check.json`：重放及状态恢复检查。','- `spider_fixes.patch`：上游本地代码修复；`hard_contact_attempt/`、`stable_short/`：失败和短段对照存档。','', '## 复现','', '```bash','.spider/bin/python tests/test_spider_dynamics_contract.py','SPIDER_TASK_OUT="$PWD/output/spider_dynamics_fix_20260915" .spider/bin/python scripts/run_yesterday_spider.py','SPIDER_TASK_OUT="$PWD/output/spider_dynamics_fix_20260915" .venv/bin/python scripts/audit_collision_physics.py','.spider/bin/python scripts/check_spider_dynamics_replay.py','SPIDER_TASK_OUT="$PWD/output/spider_dynamics_fix_20260915" .venv/bin/python scripts/render_collision_fix.py --physics','.venv/bin/python scripts/report_spider_dynamics_fix.py','```','']
|
|
(O/'README.md').write_text('\n'.join(lines));print(status);print(integrity)
|