From ae28d55f81e2ebffddc0046f1acac58c7ab2c2e4 Mon Sep 17 00:00:00 2001 From: liyang Date: Thu, 17 Sep 2026 11:27:46 +0800 Subject: [PATCH] Update to 2026-09-17 pipeline snapshot; add weights, L20 assets and recording via Git LFS Source: RGB-D -> Dyn-HaMR -> L20 retargeting -> FoundationPose -> reference repair -> SPIDER, documented in docs/PIPELINE_LATEST.md and docs/SETUP_AND_WEIGHTS.md. Adds FoundationPose and nvdiffrast upstream snapshots, requirements/pipeline_venv.txt and the FoundationPose weight manifest/downloader. Assets (Git LFS): weights/ (WiLoR detector, HandFlow denoiser, UniDepth-L), FoundationPose checkpoints, HaMeR checkpoint, Dyn-HaMR HMP model and BMC constraints, L20 URDF/meshes, the 20260915_171525 D405 recording and the two box CADs. MANO models are not redistributed (third_party/hamer/_DATA/data/mano/README.txt). Environments, caches and run outputs excluded. Co-Authored-By: Claude Opus 5 (1M context) --- .gitattributes | 17 + .gitignore | 37 +- EXCLUDED_FILES.json | 5396 ++++++++++++++++ EXTERNAL_ASSETS.json | 1927 ++++++ PORTABILITY_REPORT.json | 412 +- README.md | 13 +- RELEASE_VALIDATION.json | 10 +- SHA256SUMS | 200 +- UPSTREAM_SOURCES.json | 16 +- configs/foundationpose_weights_manifest.json | 30 + configs/large_files.json | 65 + configs/project_env.example.sh | 16 + .../20260915_171525/20260915_171525/color.mp4 | 3 + .../20260915_171525/depth/000000.png | 3 + .../20260915_171525/depth/000001.png | 3 + .../20260915_171525/depth/000002.png | 3 + .../20260915_171525/depth/000003.png | 3 + .../20260915_171525/depth/000004.png | 3 + .../20260915_171525/depth/000005.png | 3 + .../20260915_171525/depth/000006.png | 3 + .../20260915_171525/depth/000007.png | 3 + .../20260915_171525/depth/000008.png | 3 + .../20260915_171525/depth/000009.png | 3 + .../20260915_171525/depth/000010.png | 3 + .../20260915_171525/depth/000011.png | 3 + .../20260915_171525/depth/000012.png | 3 + .../20260915_171525/depth/000013.png | 3 + .../20260915_171525/depth/000014.png | 3 + .../20260915_171525/depth/000015.png | 3 + .../20260915_171525/depth/000016.png | 3 + .../20260915_171525/depth/000017.png | 3 + .../20260915_171525/depth/000018.png | 3 + .../20260915_171525/depth/000019.png | 3 + .../20260915_171525/depth/000020.png | 3 + .../20260915_171525/depth/000021.png | 3 + .../20260915_171525/depth/000022.png | 3 + .../20260915_171525/depth/000023.png | 3 + .../20260915_171525/depth/000024.png | 3 + .../20260915_171525/depth/000025.png | 3 + .../20260915_171525/depth/000026.png | 3 + .../20260915_171525/depth/000027.png | 3 + .../20260915_171525/depth/000028.png | 3 + .../20260915_171525/depth/000029.png | 3 + .../20260915_171525/depth/000030.png | 3 + .../20260915_171525/depth/000031.png | 3 + .../20260915_171525/depth/000032.png | 3 + .../20260915_171525/depth/000033.png | 3 + .../20260915_171525/depth/000034.png | 3 + .../20260915_171525/depth/000035.png | 3 + .../20260915_171525/depth/000036.png | 3 + .../20260915_171525/depth/000037.png | 3 + .../20260915_171525/depth/000038.png | 3 + .../20260915_171525/depth/000039.png | 3 + .../20260915_171525/depth/000040.png | 3 + .../20260915_171525/depth/000041.png | 3 + .../20260915_171525/depth/000042.png | 3 + .../20260915_171525/depth/000043.png | 3 + .../20260915_171525/depth/000044.png | 3 + .../20260915_171525/depth/000045.png | 3 + .../20260915_171525/depth/000046.png | 3 + .../20260915_171525/depth/000047.png | 3 + .../20260915_171525/depth/000048.png | 3 + .../20260915_171525/depth/000049.png | 3 + .../20260915_171525/depth/000050.png | 3 + .../20260915_171525/depth/000051.png | 3 + .../20260915_171525/depth/000052.png | 3 + .../20260915_171525/depth/000053.png | 3 + .../20260915_171525/depth/000054.png | 3 + .../20260915_171525/depth/000055.png | 3 + .../20260915_171525/depth/000056.png | 3 + .../20260915_171525/depth/000057.png | 3 + .../20260915_171525/depth/000058.png | 3 + .../20260915_171525/depth/000059.png | 3 + .../20260915_171525/depth/000060.png | 3 + .../20260915_171525/depth/000061.png | 3 + .../20260915_171525/depth/000062.png | 3 + .../20260915_171525/depth/000063.png | 3 + .../20260915_171525/depth/000064.png | 3 + .../20260915_171525/depth/000065.png | 3 + .../20260915_171525/depth/000066.png | 3 + .../20260915_171525/depth/000067.png | 3 + .../20260915_171525/depth/000068.png | 3 + .../20260915_171525/depth/000069.png | 3 + .../20260915_171525/depth/000070.png | 3 + .../20260915_171525/depth/000071.png | 3 + .../20260915_171525/depth/000072.png | 3 + .../20260915_171525/depth/000073.png | 3 + .../20260915_171525/depth/000074.png | 3 + .../20260915_171525/depth/000075.png | 3 + .../20260915_171525/depth/000076.png | 3 + .../20260915_171525/depth/000077.png | 3 + .../20260915_171525/depth/000078.png | 3 + .../20260915_171525/depth/000079.png | 3 + .../20260915_171525/depth/000080.png | 3 + .../20260915_171525/depth/000081.png | 3 + .../20260915_171525/depth/000082.png | 3 + .../20260915_171525/depth/000083.png | 3 + .../20260915_171525/depth/000084.png | 3 + .../20260915_171525/depth/000085.png | 3 + .../20260915_171525/depth/000086.png | 3 + .../20260915_171525/depth/000087.png | 3 + .../20260915_171525/depth/000088.png | 3 + .../20260915_171525/depth/000089.png | 3 + .../20260915_171525/depth/000090.png | 3 + .../20260915_171525/depth/000091.png | 3 + .../20260915_171525/depth/000092.png | 3 + .../20260915_171525/depth/000093.png | 3 + .../20260915_171525/depth/000094.png | 3 + .../20260915_171525/depth/000095.png | 3 + .../20260915_171525/depth/000096.png | 3 + .../20260915_171525/depth/000097.png | 3 + .../20260915_171525/depth/000098.png | 3 + .../20260915_171525/depth/000099.png | 3 + .../20260915_171525/depth/000100.png | 3 + .../20260915_171525/depth/000101.png | 3 + .../20260915_171525/depth/000102.png | 3 + .../20260915_171525/depth/000103.png | 3 + .../20260915_171525/depth/000104.png | 3 + .../20260915_171525/depth/000105.png | 3 + .../20260915_171525/depth/000106.png | 3 + .../20260915_171525/depth/000107.png | 3 + .../20260915_171525/depth/000108.png | 3 + .../20260915_171525/depth/000109.png | 3 + .../20260915_171525/depth/000110.png | 3 + .../20260915_171525/depth/000111.png | 3 + .../20260915_171525/depth/000112.png | 3 + .../20260915_171525/depth/000113.png | 3 + .../20260915_171525/depth/000114.png | 3 + .../20260915_171525/depth/000115.png | 3 + .../20260915_171525/depth/000116.png | 3 + .../20260915_171525/depth/000117.png | 3 + .../20260915_171525/depth/000118.png | 3 + .../20260915_171525/depth/000119.png | 3 + .../20260915_171525/depth/000120.png | 3 + .../20260915_171525/depth/000121.png | 3 + .../20260915_171525/depth/000122.png | 3 + .../20260915_171525/depth/000123.png | 3 + .../20260915_171525/depth/000124.png | 3 + .../20260915_171525/depth/000125.png | 3 + .../20260915_171525/depth/000126.png | 3 + .../20260915_171525/depth/000127.png | 3 + .../20260915_171525/depth/000128.png | 3 + .../20260915_171525/depth/000129.png | 3 + .../20260915_171525/depth/000130.png | 3 + .../20260915_171525/depth/000131.png | 3 + .../20260915_171525/depth/000132.png | 3 + .../20260915_171525/depth/000133.png | 3 + .../20260915_171525/depth/000134.png | 3 + .../20260915_171525/depth/000135.png | 3 + .../20260915_171525/depth/000136.png | 3 + .../20260915_171525/depth/000137.png | 3 + .../20260915_171525/depth/000138.png | 3 + .../20260915_171525/depth/000139.png | 3 + .../20260915_171525/depth/000140.png | 3 + .../20260915_171525/depth/000141.png | 3 + .../20260915_171525/depth/000142.png | 3 + .../20260915_171525/depth/000143.png | 3 + .../20260915_171525/depth/000144.png | 3 + .../20260915_171525/depth/000145.png | 3 + .../20260915_171525/depth/000146.png | 3 + .../20260915_171525/depth/000147.png | 3 + .../20260915_171525/depth/000148.png | 3 + .../20260915_171525/depth/000149.png | 3 + .../20260915_171525/depth/000150.png | 3 + .../20260915_171525/depth/000151.png | 3 + .../20260915_171525/depth/000152.png | 3 + .../20260915_171525/depth/000153.png | 3 + .../20260915_171525/depth/000154.png | 3 + .../20260915_171525/depth/000155.png | 3 + .../20260915_171525/depth/000156.png | 3 + .../20260915_171525/depth/000157.png | 3 + .../20260915_171525/depth/000158.png | 3 + .../20260915_171525/depth/000159.png | 3 + .../20260915_171525/depth/000160.png | 3 + .../20260915_171525/depth/000161.png | 3 + .../20260915_171525/depth/000162.png | 3 + .../20260915_171525/depth/000163.png | 3 + .../20260915_171525/depth/000164.png | 3 + .../20260915_171525/depth/000165.png | 3 + .../20260915_171525/depth/000166.png | 3 + .../20260915_171525/depth/000167.png | 3 + .../20260915_171525/depth/000168.png | 3 + .../20260915_171525/depth/000169.png | 3 + .../20260915_171525/depth/000170.png | 3 + .../20260915_171525/depth/000171.png | 3 + .../20260915_171525/depth/000172.png | 3 + .../20260915_171525/depth/000173.png | 3 + .../20260915_171525/depth/000174.png | 3 + .../20260915_171525/depth/000175.png | 3 + .../20260915_171525/depth/000176.png | 3 + .../20260915_171525/depth/000177.png | 3 + .../20260915_171525/depth/000178.png | 3 + .../20260915_171525/depth/000179.png | 3 + .../20260915_171525/depth/000180.png | 3 + .../20260915_171525/depth/000181.png | 3 + .../20260915_171525/depth/000182.png | 3 + .../20260915_171525/depth/000183.png | 3 + .../20260915_171525/depth/000184.png | 3 + .../20260915_171525/depth/000185.png | 3 + .../20260915_171525/depth/000186.png | 3 + .../20260915_171525/depth/000187.png | 3 + .../20260915_171525/depth/000188.png | 3 + .../20260915_171525/depth/000189.png | 3 + .../20260915_171525/depth/000190.png | 3 + .../20260915_171525/depth/000191.png | 3 + .../20260915_171525/depth/000192.png | 3 + .../20260915_171525/depth/000193.png | 3 + .../20260915_171525/depth/000194.png | 3 + .../20260915_171525/depth/000195.png | 3 + .../20260915_171525/depth/000196.png | 3 + .../20260915_171525/depth/000197.png | 3 + .../20260915_171525/depth/000198.png | 3 + .../20260915_171525/depth/000199.png | 3 + .../20260915_171525/depth/000200.png | 3 + .../20260915_171525/depth/000201.png | 3 + .../20260915_171525/depth/000202.png | 3 + .../20260915_171525/depth/000203.png | 3 + .../20260915_171525/depth/000204.png | 3 + .../20260915_171525/depth/000205.png | 3 + .../20260915_171525/depth/000206.png | 3 + .../20260915_171525/depth/000207.png | 3 + .../20260915_171525/depth/000208.png | 3 + .../20260915_171525/depth/000209.png | 3 + .../20260915_171525/depth/000210.png | 3 + .../20260915_171525/depth/000211.png | 3 + .../20260915_171525/depth/000212.png | 3 + .../20260915_171525/depth/000213.png | 3 + .../20260915_171525/depth/000214.png | 3 + .../20260915_171525/depth/000215.png | 3 + .../20260915_171525/depth/000216.png | 3 + .../20260915_171525/depth/000217.png | 3 + .../20260915_171525/depth/000218.png | 3 + .../20260915_171525/depth/000219.png | 3 + .../20260915_171525/depth/000220.png | 3 + .../20260915_171525/depth/000221.png | 3 + .../20260915_171525/depth/000222.png | 3 + .../20260915_171525/depth/000223.png | 3 + .../20260915_171525/depth/000224.png | 3 + .../20260915_171525/depth/000225.png | 3 + .../20260915_171525/depth/000226.png | 3 + .../20260915_171525/depth/000227.png | 3 + .../20260915_171525/depth/000228.png | 3 + .../20260915_171525/depth/000229.png | 3 + .../20260915_171525/depth/000230.png | 3 + .../20260915_171525/depth/000231.png | 3 + .../20260915_171525/depth/000232.png | 3 + .../20260915_171525/depth/000233.png | 3 + .../20260915_171525/depth/000234.png | 3 + .../20260915_171525/depth/000235.png | 3 + .../20260915_171525/depth/000236.png | 3 + .../20260915_171525/depth/000237.png | 3 + .../20260915_171525/depth/000238.png | 3 + .../20260915_171525/depth/000239.png | 3 + .../20260915_171525/depth/000240.png | 3 + .../20260915_171525/depth/000241.png | 3 + .../20260915_171525/depth/000242.png | 3 + .../20260915_171525/depth/000243.png | 3 + .../20260915_171525/depth/000244.png | 3 + .../20260915_171525/depth/000245.png | 3 + .../20260915_171525/depth/000246.png | 3 + .../20260915_171525/depth/000247.png | 3 + .../20260915_171525/depth/000248.png | 3 + .../20260915_171525/depth/000249.png | 3 + .../20260915_171525/depth/000250.png | 3 + .../20260915_171525/depth/000251.png | 3 + .../20260915_171525/depth/000252.png | 3 + .../20260915_171525/depth/000253.png | 3 + .../20260915_171525/depth/000254.png | 3 + .../20260915_171525/depth/000255.png | 3 + .../20260915_171525/depth/000256.png | 3 + .../20260915_171525/depth/000257.png | 3 + .../20260915_171525/depth/000258.png | 3 + .../20260915_171525/depth/000259.png | 3 + .../20260915_171525/depth/000260.png | 3 + .../20260915_171525/depth/000261.png | 3 + .../20260915_171525/depth/000262.png | 3 + .../20260915_171525/depth/000263.png | 3 + .../20260915_171525/depth/000264.png | 3 + .../20260915_171525/depth/000265.png | 3 + .../20260915_171525/depth/000266.png | 3 + .../20260915_171525/depth/000267.png | 3 + .../20260915_171525/depth/000268.png | 3 + .../20260915_171525/depth/000269.png | 3 + .../20260915_171525/depth/000270.png | 3 + .../20260915_171525/depth/000271.png | 3 + .../20260915_171525/depth/000272.png | 3 + .../20260915_171525/depth/000273.png | 3 + .../20260915_171525/depth/000274.png | 3 + .../20260915_171525/depth/000275.png | 3 + .../20260915_171525/depth/000276.png | 3 + .../20260915_171525/depth/000277.png | 3 + .../20260915_171525/depth/000278.png | 3 + .../20260915_171525/depth/000279.png | 3 + .../20260915_171525/depth/000280.png | 3 + .../20260915_171525/depth/000281.png | 3 + .../20260915_171525/depth/000282.png | 3 + .../20260915_171525/depth/000283.png | 3 + .../20260915_171525/depth/000284.png | 3 + .../20260915_171525/depth/000285.png | 3 + .../20260915_171525/depth/000286.png | 3 + .../20260915_171525/depth/000287.png | 3 + .../20260915_171525/depth/000288.png | 3 + .../20260915_171525/depth/000289.png | 3 + .../20260915_171525/depth/000290.png | 3 + .../20260915_171525/depth/000291.png | 3 + .../20260915_171525/depth/000292.png | 3 + .../20260915_171525/depth/000293.png | 3 + .../20260915_171525/depth/000294.png | 3 + .../20260915_171525/depth/000295.png | 3 + .../20260915_171525/depth/000296.png | 3 + .../20260915_171525/depth/000297.png | 3 + .../20260915_171525/depth/000298.png | 3 + .../20260915_171525/depth/000299.png | 3 + .../20260915_171525/depth/000300.png | 3 + .../20260915_171525/depth/000301.png | 3 + .../20260915_171525/depth/000302.png | 3 + .../20260915_171525/depth/000303.png | 3 + .../20260915_171525/depth/000304.png | 3 + .../20260915_171525/depth/000305.png | 3 + .../20260915_171525/depth/000306.png | 3 + .../20260915_171525/depth/000307.png | 3 + .../20260915_171525/depth/000308.png | 3 + .../20260915_171525/depth/000309.png | 3 + .../20260915_171525/depth/000310.png | 3 + .../20260915_171525/depth/000311.png | 3 + .../20260915_171525/depth/000312.png | 3 + .../20260915_171525/depth/000313.png | 3 + .../20260915_171525/depth/000314.png | 3 + .../20260915_171525/depth/000315.png | 3 + .../20260915_171525/depth/000316.png | 3 + .../20260915_171525/depth/000317.png | 3 + .../20260915_171525/depth/000318.png | 3 + .../20260915_171525/depth/000319.png | 3 + .../20260915_171525/depth/000320.png | 3 + .../20260915_171525/depth/000321.png | 3 + .../20260915_171525/depth/000322.png | 3 + .../20260915_171525/depth/000323.png | 3 + .../20260915_171525/depth/000324.png | 3 + .../20260915_171525/depth/000325.png | 3 + .../20260915_171525/depth/000326.png | 3 + .../20260915_171525/depth/000327.png | 3 + .../20260915_171525/depth/000328.png | 3 + .../20260915_171525/depth/000329.png | 3 + .../20260915_171525/depth/000330.png | 3 + .../20260915_171525/depth/000331.png | 3 + .../20260915_171525/depth/000332.png | 3 + .../20260915_171525/depth/000333.png | 3 + .../20260915_171525/depth/000334.png | 3 + .../20260915_171525/depth/000335.png | 3 + .../20260915_171525/depth/000336.png | 3 + .../20260915_171525/depth/000337.png | 3 + .../20260915_171525/depth/000338.png | 3 + .../20260915_171525/depth/000339.png | 3 + .../20260915_171525/depth/000340.png | 3 + .../20260915_171525/depth/000341.png | 3 + .../20260915_171525/depth/000342.png | 3 + .../20260915_171525/depth/000343.png | 3 + .../20260915_171525/depth/000344.png | 3 + .../20260915_171525/depth/000345.png | 3 + .../20260915_171525/depth/000346.png | 3 + .../20260915_171525/depth/000347.png | 3 + .../20260915_171525/depth/000348.png | 3 + .../20260915_171525/depth/000349.png | 3 + .../20260915_171525/depth/000350.png | 3 + .../20260915_171525/depth/000351.png | 3 + .../20260915_171525/intrinsics.json | 1789 ++++++ docs/20260915_171525/intrinsics.json | 1789 ++++++ docs/20260916_104026/intrinsics.json | 4494 +++++++++++++ docs/FOUNDATIONPOSE_SETUP.md | 38 + docs/INTEGRATED_PROJECT.md | 2 + docs/PIPELINE_LATEST.md | 273 + docs/SETUP_AND_WEIGHTS.md | 205 + docs/上半.stl | 3 + docs/下半.stl | 3 + requirements/pipeline_venv.txt | 47 + requirements/runtime_observed.json | 66 + requirements/spider_installed_local.txt | 51 + requirements/spider_resolved.txt | 557 ++ scripts/align_collision_floor.py | 8 + scripts/audit_collision_fix.py | 42 + scripts/audit_collision_physics.py | 16 + scripts/audit_red_depth_metrics.py | 24 + scripts/audit_smooth_table_result.py | 34 + scripts/audit_spider_dynamic_surfaces.py | 18 + scripts/audit_yesterday_geometry.py | 24 + scripts/build_box_collision_v2.py | 24 + scripts/build_palm_collision_v2.py | 11 + scripts/check_spider_dynamics_replay.py | 19 + scripts/check_spider_state_restore.py | 19 + scripts/compare_red_hand_depth.py | 75 + scripts/configure_collision_physics.py | 14 + scripts/configure_spider_stable_contact.py | 9 + scripts/configure_spider_task.py | 29 + scripts/constrain_rgbd_hand.py | 70 + scripts/correct_dynhamr_rgbd.py | 80 + scripts/decompose_yesterday_boxes.py | 13 + scripts/diagnose_dynhamr_rgbd.py | 58 + scripts/diagnose_yesterday_hand_depth.py | 129 + scripts/download_foundationpose_weights.py | 33 + scripts/evaluate_yesterday_spider.py | 56 + scripts/export_camera_delivery.py | 214 + scripts/export_dynhamr_bimanual_rgbd.py | 18 + scripts/fit_rgbd_objects.py | 93 + scripts/fix_yesterday_hand_collision.py | 118 + scripts/foundationpose_env.sh | 17 + scripts/interpolate_collision_reference.py | 18 + scripts/l20_calibrated.py | 9 +- scripts/large_files.py | 86 + scripts/package_source_release.py | 20 +- scripts/play_compare.py | 84 + scripts/play_spider_result.py | 60 + scripts/prepare_rgbd_20260915.py | 53 + scripts/prepare_yesterday_spider.py | 129 + scripts/probe_collision_dynamics.py | 22 + scripts/probe_spider_gpu_contact.py | 21 + scripts/red_box_distance.py | 18 + scripts/refit_rgbd_objects_outline.py | 60 + scripts/refit_rgbd_objects_tracked.py | 94 + scripts/register_hands_rgbd.py | 122 + scripts/render_collision_fix.py | 23 + scripts/render_corrected_dynhamr_rgbd.py | 15 + scripts/render_foundationpose_red.py | 39 + scripts/render_object_alignment_overlay.py | 15 + scripts/render_red_depth_comparison.py | 39 + scripts/render_rgbd_l20_objects.py | 94 + scripts/repair_red_box_topology.py | 14 + scripts/replay_yesterday_spider_gpu.py | 19 + scripts/report_collision_fix.py | 12 + scripts/report_spider_dynamics_fix.py | 17 + scripts/report_yesterday_spider.py | 21 + scripts/retarget_l20_video.py | 10 +- scripts/rl_track_reference.py | 194 + scripts/run_dynhamr_rgbd_20260915.py | 49 + scripts/run_red_handflow_comparison.py | 15 + scripts/run_red_mirrored_handflow.py | 16 + scripts/run_rgbd_20260915.py | 47 + scripts/run_yesterday_spider.py | 38 + scripts/smooth_object_reference.py | 200 + scripts/smooth_project_reference.py | 114 + scripts/smooth_refitted_objects.py | 35 + scripts/smooth_table_constrained.py | 96 + scripts/stabilize_l20_temporal.py | 7 +- scripts/track_red_box_foundationpose.py | 76 + scripts/track_yesterday_boxes.py | 52 + scripts/unmirror_left_rgbd.py | 20 + scripts/verify_fix_interpolation.py | 45 + scripts/verify_foundationpose_setup.py | 32 + scripts/verify_rgbd_bimanual.py | 69 + third_party/Dyn-HaMR/_DATA/BMC | 1 + third_party/Dyn-HaMR/_DATA/data/mano | 1 + .../Dyn-HaMR/_DATA/data/mano_mean_params.npz | 1 + .../Dyn-HaMR/_DATA/hamer_ckpts/checkpoints | 1 + .../_DATA/hamer_ckpts/dataset_config.yaml | 42 + .../_DATA/hamer_ckpts/model_config.yaml | 111 + .../_DATA/hmp_model/mean-neutral-128-30fps.pt | 3 + .../results/model/global_encoder.pth | 3 + .../results/model/local_encoder.pth.part-000 | 3 + .../results/model/local_encoder.pth.part-001 | 3 + .../results/model/local_encoder.pth.part-002 | 3 + .../results/model/local_encoder.pth.part-003 | 3 + .../_DATA/hmp_model/results/model/loss.pkl | 3 + .../hmp_model/results/model/nemf.pth.part-000 | 3 + .../hmp_model/results/model/nemf.pth.part-001 | 3 + .../results/model/optimizer.pth.part-000 | 3 + .../results/model/optimizer.pth.part-001 | 3 + .../results/model/optimizer.pth.part-002 | 3 + .../results/model/optimizer.pth.part-003 | 3 + .../results/model/optimizer.pth.part-004 | 3 + .../results/model/optimizer.pth.part-005 | 3 + .../results/model/optimizer.pth.part-006 | 3 + .../results/model/optimizer.pth.part-007 | 3 + .../results/model/optimizer.pth.part-008 | 3 + .../results/model/optimizer.pth.part-009 | 3 + .../results/model/optimizer.pth.part-010 | 3 + .../_DATA/hmp_model/std-neutral-128-30fps.pt | 3 + .../dyn-hamr/optim/BMC/CONVEX_HULLS.npy | 3 + .../Dyn-HaMR/dyn-hamr/optim/BMC/PHI_max.npy | 3 + .../Dyn-HaMR/dyn-hamr/optim/BMC/PHI_min.npy | 3 + .../dyn-hamr/optim/BMC/bone_len_max.npy | 3 + .../dyn-hamr/optim/BMC/bone_len_min.npy | 3 + .../dyn-hamr/optim/BMC/curvatures_max.npy | 3 + .../dyn-hamr/optim/BMC/curvatures_min.npy | 3 + .../optim/BMC/joint_angles.npy.part-000 | 3 + .../optim/BMC/joint_angles.npy.part-001 | 3 + third_party/Dyn-HaMR/dyn-hamr/vis/tools.py | 7 +- third_party/Dyn-HaMR/dyn-hamr/vis/viewer.py | 2 + .../hamer/pretrained_models/detector.pt | 1 + third_party/FoundationPose/LICENSE | 94 + third_party/FoundationPose/Utils.py | 1022 +++ third_party/FoundationPose/build_all.sh | 8 + third_party/FoundationPose/build_all_conda.sh | 23 + .../bundlesdf/config_linemod.yml | 105 + .../FoundationPose/bundlesdf/config_ycbv.yml | 105 + .../bundlesdf/mycuda/__init__.py | 0 .../bundlesdf/mycuda/bindings.cpp | 20 + .../FoundationPose/bundlesdf/mycuda/common.cu | 313 + .../FoundationPose/bundlesdf/mycuda/common.h | 30 + .../FoundationPose/bundlesdf/mycuda/setup.py | 41 + .../mycuda/torch_ngp_grid_encoder/LICENSE | 21 + .../torch_ngp_grid_encoder/bindings.cpp | 8 + .../mycuda/torch_ngp_grid_encoder/grid.py | 158 + .../torch_ngp_grid_encoder/gridencoder.cu | 500 ++ .../torch_ngp_grid_encoder/gridencoder.h | 15 + .../FoundationPose/bundlesdf/nerf_helpers.py | 475 ++ .../FoundationPose/bundlesdf/nerf_runner.py | 1232 ++++ .../FoundationPose/bundlesdf/run_nerf.py | 115 + third_party/FoundationPose/bundlesdf/tool.py | 132 + third_party/FoundationPose/check_env.py | 122 + third_party/FoundationPose/datareader.py | 613 ++ .../FoundationPose/docker/run_container.sh | 3 + third_party/FoundationPose/environment.yml | 17 + third_party/FoundationPose/estimater.py | 270 + .../learning/datasets/h5_dataset.py | 220 + .../learning/datasets/pose_dataset.py | 135 + .../learning/models/network_modules.py | 138 + .../learning/models/refine_network.py | 93 + .../learning/models/score_network.py | 90 + .../learning/training/predict_pose_refine.py | 296 + .../learning/training/predict_score.py | 227 + .../learning/training/training_config.py | 102 + .../FoundationPose/mycpp/CMakeLists.txt | 25 + .../FoundationPose/mycpp/include/Utils.h | 46 + .../FoundationPose/mycpp/src/Utils.cpp | 29 + .../mycpp/src/app/pybind_api.cpp | 77 + .../FoundationPose/offscreen_renderer.py | 79 + third_party/FoundationPose/readme.md | 226 + third_party/FoundationPose/requirements.txt | 43 + third_party/FoundationPose/run_demo.py | 79 + third_party/FoundationPose/run_linemod.py | 149 + third_party/FoundationPose/run_ycb_video.py | 149 + .../weights/2023-10-28-18-33-37/config.yml | 39 + .../model_best.pth.part-000 | 3 + .../model_best.pth.part-001 | 3 + .../weights/2024-01-11-20-02-45/config.yml | 41 + .../model_best.pth.part-000 | 3 + .../model_best.pth.part-001 | 3 + .../model_best.pth.part-002 | 3 + .../model_best.pth.part-003 | 3 + .../weights/download_manifest.json | 30 + third_party/bottle_model_parametric/README.md | 16 + .../bottle_model_parametric/assets/bottle.obj | 3 + .../assets/shoulder.obj | 3 + .../bottle_model_parametric/bottle.glb | Bin 0 -> 19252 bytes .../bottle_model_parametric/bottle.stl | 3 + .../bottle_model_parametric/bottle.xml | 17 + .../bottle_model_parametric/preview.png | 3 + .../bottle_model_parametric/profile.npz | 3 + .../bottle_model_parametric/validation.json | 17 + .../robots/hands/leap_hand/LICENSE.txt | 10 + .../hands/leap_hand/leap_hand_right.urdf | 812 +++ .../leap_hand/meshes/collision/white_tip.obj | 3 + .../meshes/collision/white_tip_thumb.obj | 3 + .../hands/leap_hand/meshes/visual/dip.obj | 3 + .../leap_hand/meshes/visual/fingertip.obj | 3 + .../leap_hand/meshes/visual/mcp_joint.obj | 3 + .../leap_hand/meshes/visual/palm_lower.obj | 3 + .../hands/leap_hand/meshes/visual/pip.obj | 3 + .../leap_hand/meshes/visual/thumb_dip.obj | 3 + .../meshes/visual/thumb_fingertip.obj | 3 + .../leap_hand/meshes/visual/thumb_pip.obj | 3 + third_party/hamer/_DATA/data/mano/README.txt | 1 + .../hamer/_DATA/data/mano_mean_params.npz | 3 + .../checkpoints/hamer.ckpt.part-000 | 3 + .../checkpoints/hamer.ckpt.part-001 | 3 + .../checkpoints/hamer.ckpt.part-002 | 3 + .../checkpoints/hamer.ckpt.part-003 | 3 + .../checkpoints/hamer.ckpt.part-004 | 3 + .../checkpoints/hamer.ckpt.part-005 | 3 + .../checkpoints/hamer.ckpt.part-006 | 3 + .../checkpoints/hamer.ckpt.part-007 | 3 + .../checkpoints/hamer.ckpt.part-008 | 3 + .../checkpoints/hamer.ckpt.part-009 | 3 + .../checkpoints/hamer.ckpt.part-010 | 3 + .../checkpoints/hamer.ckpt.part-011 | 3 + .../checkpoints/hamer.ckpt.part-012 | 3 + .../checkpoints/hamer.ckpt.part-013 | 3 + .../checkpoints/hamer.ckpt.part-014 | 3 + .../checkpoints/hamer.ckpt.part-015 | 3 + .../checkpoints/hamer.ckpt.part-016 | 3 + .../checkpoints/hamer.ckpt.part-017 | 3 + .../checkpoints/hamer.ckpt.part-018 | 3 + .../checkpoints/hamer.ckpt.part-019 | 3 + .../checkpoints/hamer.ckpt.part-020 | 3 + .../checkpoints/hamer.ckpt.part-021 | 3 + .../checkpoints/hamer.ckpt.part-022 | 3 + .../checkpoints/hamer.ckpt.part-023 | 3 + .../checkpoints/hamer.ckpt.part-024 | 3 + .../checkpoints/hamer.ckpt.part-025 | 3 + .../checkpoints/hamer.ckpt.part-026 | 3 + .../checkpoints/hamer.ckpt.part-027 | 3 + .../checkpoints/hamer.ckpt.part-028 | 3 + .../checkpoints/hamer.ckpt.part-029 | 3 + .../checkpoints/hamer.ckpt.part-030 | 3 + .../checkpoints/hamer.ckpt.part-031 | 3 + .../checkpoints/hamer.ckpt.part-032 | 3 + .../checkpoints/hamer.ckpt.part-033 | 3 + .../checkpoints/hamer.ckpt.part-034 | 3 + .../checkpoints/hamer.ckpt.part-035 | 3 + .../checkpoints/hamer.ckpt.part-036 | 3 + .../checkpoints/hamer.ckpt.part-037 | 3 + .../checkpoints/hamer.ckpt.part-038 | 3 + .../checkpoints/hamer.ckpt.part-039 | 3 + .../checkpoints/hamer.ckpt.part-040 | 3 + .../checkpoints/hamer.ckpt.part-041 | 3 + .../checkpoints/hamer.ckpt.part-042 | 3 + .../checkpoints/hamer.ckpt.part-043 | 3 + .../checkpoints/hamer.ckpt.part-044 | 3 + .../checkpoints/hamer.ckpt.part-045 | 3 + .../checkpoints/hamer.ckpt.part-046 | 3 + .../checkpoints/hamer.ckpt.part-047 | 3 + .../checkpoints/hamer.ckpt.part-048 | 3 + .../checkpoints/hamer.ckpt.part-049 | 3 + .../checkpoints/hamer.ckpt.part-050 | 3 + .../checkpoints/hamer.ckpt.part-051 | 3 + .../checkpoints/hamer.ckpt.part-052 | 3 + .../checkpoints/hamer.ckpt.part-053 | 3 + .../hamer/_DATA/hamer_ckpts/model_config.yaml | 111 + .../l20_assets/L20/LEFT/g20_left_G20.json | 5602 +++++++++++++++++ .../L20/LEFT/linkerhand_g20_left.urdf | 1285 ++++ .../L20/LEFT/meshes/hand_base_link.STL | 3 + .../L20/LEFT/meshes/index_distal.STL | 3 + .../L20/LEFT/meshes/index_metacarpals.STL | 3 + .../L20/LEFT/meshes/index_middle.STL | 3 + .../L20/LEFT/meshes/index_proximal.STL | 3 + .../L20/LEFT/meshes/middle_distal.STL | 3 + .../L20/LEFT/meshes/middle_metacarpals.STL | 3 + .../L20/LEFT/meshes/middle_middle.STL | 3 + .../L20/LEFT/meshes/middle_proximal.STL | 3 + .../L20/LEFT/meshes/pinky_distal.STL | 3 + .../L20/LEFT/meshes/pinky_metacarpals.STL | 3 + .../L20/LEFT/meshes/pinky_middle.STL | 3 + .../L20/LEFT/meshes/pinky_proximal.STL | 3 + .../L20/LEFT/meshes/ring_distal.STL | 3 + .../L20/LEFT/meshes/ring_metacarpals.STL | 3 + .../L20/LEFT/meshes/ring_middle.STL | 3 + .../L20/LEFT/meshes/ring_proximal.STL | 3 + .../L20/LEFT/meshes/thumb_distal.STL | 3 + .../L20/LEFT/meshes/thumb_metacarpals.STL | 3 + .../LEFT/meshes/thumb_metacarpals_base1.STL | 3 + .../LEFT/meshes/thumb_metacarpals_base2.STL | 3 + .../L20/LEFT/meshes/thumb_proximal.STL | 3 + .../l20_assets/L20/RIGHT/g20_right_G20.json | 5602 +++++++++++++++++ .../L20/RIGHT/linkerhand_g20_right.urdf | 1285 ++++ .../L20/RIGHT/linkerhand_g20_right_mj.urdf | 1289 ++++ .../L20/RIGHT/meshes/hand_base_link.STL | 3 + .../L20/RIGHT/meshes/index_distal.STL | 3 + .../L20/RIGHT/meshes/index_metacarpals.STL | 3 + .../L20/RIGHT/meshes/index_middle.STL | 3 + .../L20/RIGHT/meshes/index_proximal.STL | 3 + .../L20/RIGHT/meshes/middle_distal.STL | 3 + .../L20/RIGHT/meshes/middle_metacarpals.STL | 3 + .../L20/RIGHT/meshes/middle_middle.STL | 3 + .../L20/RIGHT/meshes/middle_proximal.STL | 3 + .../L20/RIGHT/meshes/pinky_distal.STL | 3 + .../L20/RIGHT/meshes/pinky_metacarpals.STL | 3 + .../L20/RIGHT/meshes/pinky_middle.STL | 3 + .../L20/RIGHT/meshes/pinky_proximal.STL | 3 + .../L20/RIGHT/meshes/ring_distal.STL | 3 + .../L20/RIGHT/meshes/ring_metacarpals.STL | 3 + .../L20/RIGHT/meshes/ring_middle.STL | 3 + .../L20/RIGHT/meshes/ring_proximal.STL | 3 + .../L20/RIGHT/meshes/thumb_distal.STL | 3 + .../L20/RIGHT/meshes/thumb_metacarpals.STL | 3 + .../RIGHT/meshes/thumb_metacarpals_base1.STL | 3 + .../RIGHT/meshes/thumb_metacarpals_base2.STL | 3 + .../L20/RIGHT/meshes/thumb_proximal.STL | 3 + third_party/nvdiffrast/Dockerfile | 20 + third_party/nvdiffrast/LICENSE.txt | 97 + third_party/nvdiffrast/MANIFEST.in | 21 + third_party/nvdiffrast/README.md | 49 + .../nvdiffrast/csrc/common/antialias.cu | 558 ++ .../nvdiffrast/csrc/common/antialias.h | 50 + third_party/nvdiffrast/csrc/common/common.cpp | 60 + third_party/nvdiffrast/csrc/common/common.h | 263 + .../csrc/common/cudaraster/CudaRaster.hpp | 63 + .../csrc/common/cudaraster/impl/Buffer.cpp | 94 + .../csrc/common/cudaraster/impl/Buffer.hpp | 55 + .../csrc/common/cudaraster/impl/Constants.hpp | 73 + .../common/cudaraster/impl/CudaRaster.cpp | 79 + .../csrc/common/cudaraster/impl/Defs.hpp | 90 + .../common/cudaraster/impl/PrivateDefs.hpp | 153 + .../common/cudaraster/impl/RasterImpl.cpp | 370 ++ .../common/cudaraster/impl/RasterImpl.hpp | 102 + .../cudaraster/impl/RasterImpl_kernel.cu | 37 + .../nvdiffrast/csrc/common/framework.h | 28 + .../nvdiffrast/csrc/common/interpolate.cu | 276 + .../nvdiffrast/csrc/common/interpolate.h | 49 + .../nvdiffrast/csrc/common/rasterize.cu | 279 + .../nvdiffrast/csrc/common/rasterize.h | 60 + .../nvdiffrast/csrc/common/texture.cpp | 104 + third_party/nvdiffrast/csrc/common/texture.h | 78 + .../nvdiffrast/csrc/common/texture_kernel.cu | 1156 ++++ .../nvdiffrast/csrc/torch/torch_antialias.cpp | 243 + .../nvdiffrast/csrc/torch/torch_bindings.cpp | 73 + .../csrc/torch/torch_interpolate.cpp | 250 + .../nvdiffrast/csrc/torch/torch_rasterize.cpp | 265 + .../nvdiffrast/csrc/torch/torch_texture.cpp | 718 +++ .../nvdiffrast/csrc/torch/torch_types.h | 47 + third_party/nvdiffrast/docs/index.html | 1033 +++ third_party/nvdiffrast/nvdiffrast/__init__.py | 11 + .../nvdiffrast/nvdiffrast/torch/__init__.py | 10 + .../nvdiffrast/nvdiffrast/torch/ops.py | 561 ++ third_party/nvdiffrast/pyproject.toml | 31 + .../nvdiffrast/samples/data/NOTICE.txt | 225 + third_party/nvdiffrast/samples/torch/cube.py | 202 + third_party/nvdiffrast/samples/torch/earth.py | 205 + .../nvdiffrast/samples/torch/envphong.py | 227 + third_party/nvdiffrast/samples/torch/pose.py | 290 + .../nvdiffrast/samples/torch/triangle.py | 30 + third_party/nvdiffrast/samples/torch/util.py | 120 + third_party/nvdiffrast/setup.py | 55 + third_party/spider/examples/run_mjwp.py | 5 +- .../spider/spider/optimizers/sampling.py | 4 +- third_party/spider/spider/simulators/mjwp.py | 21 +- weights/detector.pt.part-000 | 3 + weights/detector.pt.part-001 | 3 + weights/handflow_denoiser.pt.part-000 | 3 + weights/handflow_denoiser.pt.part-001 | 3 + weights/handflow_denoiser.pt.part-002 | 3 + weights/handflow_denoiser.pt.part-003 | 3 + weights/handflow_denoiser.pt.part-004 | 3 + weights/handflow_denoiser.pt.part-005 | 3 + weights/handflow_denoiser.pt.part-006 | 3 + weights/handflow_denoiser.pt.part-007 | 3 + weights/handflow_denoiser.pt.part-008 | 3 + weights/handflow_denoiser.pt.part-009 | 3 + weights/handflow_denoiser.pt.part-010 | 3 + weights/handflow_denoiser.pt.part-011 | 3 + weights/handflow_denoiser.pt.part-012 | 3 + weights/handflow_denoiser.pt.part-013 | 3 + weights/normalization_stats.npz | 3 + weights/unidepth-v2-vitl14/config.json | 211 + .../model.safetensors.part-000 | 3 + .../model.safetensors.part-001 | 3 + .../model.safetensors.part-002 | 3 + .../model.safetensors.part-003 | 3 + .../model.safetensors.part-004 | 3 + .../model.safetensors.part-005 | 3 + .../model.safetensors.part-006 | 3 + .../model.safetensors.part-007 | 3 + .../model.safetensors.part-008 | 3 + .../model.safetensors.part-009 | 3 + .../model.safetensors.part-010 | 3 + .../model.safetensors.part-011 | 3 + .../model.safetensors.part-012 | 3 + .../model.safetensors.part-013 | 3 + .../model.safetensors.part-014 | 3 + .../model.safetensors.part-015 | 3 + .../model.safetensors.part-016 | 3 + .../model.safetensors.part-017 | 3 + .../model.safetensors.part-018 | 3 + .../model.safetensors.part-019 | 3 + .../model.safetensors.part-020 | 3 + .../model.safetensors.part-021 | 3 + .../model.safetensors.part-022 | 3 + .../model.safetensors.part-023 | 3 + .../model.safetensors.part-024 | 3 + .../model.safetensors.part-025 | 3 + .../model.safetensors.part-026 | 3 + .../model.safetensors.part-027 | 3 + .../model.safetensors.part-028 | 3 + 761 files changed, 55598 insertions(+), 59 deletions(-) create mode 100644 .gitattributes create mode 100644 configs/foundationpose_weights_manifest.json create mode 100644 configs/large_files.json create mode 100644 configs/project_env.example.sh create mode 100644 docs/20260915_171525/20260915_171525/color.mp4 create mode 100644 docs/20260915_171525/20260915_171525/depth/000000.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000001.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000002.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000003.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000004.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000005.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000006.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000007.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000008.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000009.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000010.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000011.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000012.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000013.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000014.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000015.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000016.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000017.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000018.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000019.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000020.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000021.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000022.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000023.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000024.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000025.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000026.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000027.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000028.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000029.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000030.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000031.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000032.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000033.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000034.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000035.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000036.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000037.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000038.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000039.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000040.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000041.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000042.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000043.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000044.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000045.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000046.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000047.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000048.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000049.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000050.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000051.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000052.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000053.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000054.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000055.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000056.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000057.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000058.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000059.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000060.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000061.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000062.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000063.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000064.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000065.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000066.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000067.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000068.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000069.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000070.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000071.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000072.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000073.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000074.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000075.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000076.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000077.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000078.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000079.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000080.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000081.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000082.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000083.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000084.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000085.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000086.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000087.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000088.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000089.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000090.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000091.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000092.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000093.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000094.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000095.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000096.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000097.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000098.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000099.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000100.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000101.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000102.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000103.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000104.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000105.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000106.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000107.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000108.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000109.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000110.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000111.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000112.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000113.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000114.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000115.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000116.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000117.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000118.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000119.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000120.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000121.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000122.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000123.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000124.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000125.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000126.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000127.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000128.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000129.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000130.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000131.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000132.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000133.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000134.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000135.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000136.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000137.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000138.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000139.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000140.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000141.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000142.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000143.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000144.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000145.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000146.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000147.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000148.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000149.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000150.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000151.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000152.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000153.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000154.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000155.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000156.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000157.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000158.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000159.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000160.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000161.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000162.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000163.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000164.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000165.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000166.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000167.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000168.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000169.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000170.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000171.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000172.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000173.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000174.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000175.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000176.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000177.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000178.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000179.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000180.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000181.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000182.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000183.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000184.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000185.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000186.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000187.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000188.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000189.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000190.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000191.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000192.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000193.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000194.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000195.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000196.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000197.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000198.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000199.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000200.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000201.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000202.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000203.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000204.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000205.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000206.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000207.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000208.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000209.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000210.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000211.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000212.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000213.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000214.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000215.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000216.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000217.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000218.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000219.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000220.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000221.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000222.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000223.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000224.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000225.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000226.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000227.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000228.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000229.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000230.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000231.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000232.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000233.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000234.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000235.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000236.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000237.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000238.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000239.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000240.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000241.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000242.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000243.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000244.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000245.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000246.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000247.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000248.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000249.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000250.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000251.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000252.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000253.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000254.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000255.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000256.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000257.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000258.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000259.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000260.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000261.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000262.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000263.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000264.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000265.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000266.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000267.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000268.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000269.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000270.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000271.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000272.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000273.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000274.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000275.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000276.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000277.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000278.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000279.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000280.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000281.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000282.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000283.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000284.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000285.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000286.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000287.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000288.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000289.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000290.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000291.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000292.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000293.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000294.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000295.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000296.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000297.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000298.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000299.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000300.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000301.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000302.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000303.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000304.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000305.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000306.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000307.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000308.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000309.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000310.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000311.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000312.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000313.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000314.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000315.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000316.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000317.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000318.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000319.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000320.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000321.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000322.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000323.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000324.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000325.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000326.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000327.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000328.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000329.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000330.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000331.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000332.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000333.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000334.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000335.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000336.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000337.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000338.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000339.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000340.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000341.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000342.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000343.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000344.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000345.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000346.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000347.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000348.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000349.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000350.png create mode 100644 docs/20260915_171525/20260915_171525/depth/000351.png create mode 100644 docs/20260915_171525/20260915_171525/intrinsics.json create mode 100644 docs/20260915_171525/intrinsics.json create mode 100644 docs/20260916_104026/intrinsics.json create mode 100644 docs/FOUNDATIONPOSE_SETUP.md create mode 100644 docs/PIPELINE_LATEST.md create mode 100644 docs/SETUP_AND_WEIGHTS.md create mode 100644 docs/上半.stl create mode 100644 docs/下半.stl create mode 100644 requirements/pipeline_venv.txt create mode 100644 requirements/runtime_observed.json create mode 100644 requirements/spider_installed_local.txt create mode 100644 requirements/spider_resolved.txt create mode 100644 scripts/align_collision_floor.py create mode 100644 scripts/audit_collision_fix.py create mode 100644 scripts/audit_collision_physics.py create mode 100644 scripts/audit_red_depth_metrics.py create mode 100644 scripts/audit_smooth_table_result.py create mode 100644 scripts/audit_spider_dynamic_surfaces.py create mode 100644 scripts/audit_yesterday_geometry.py create mode 100644 scripts/build_box_collision_v2.py create mode 100644 scripts/build_palm_collision_v2.py create mode 100644 scripts/check_spider_dynamics_replay.py create mode 100644 scripts/check_spider_state_restore.py create mode 100644 scripts/compare_red_hand_depth.py create mode 100644 scripts/configure_collision_physics.py create mode 100644 scripts/configure_spider_stable_contact.py create mode 100644 scripts/configure_spider_task.py create mode 100644 scripts/constrain_rgbd_hand.py create mode 100644 scripts/correct_dynhamr_rgbd.py create mode 100644 scripts/decompose_yesterday_boxes.py create mode 100644 scripts/diagnose_dynhamr_rgbd.py create mode 100644 scripts/diagnose_yesterday_hand_depth.py create mode 100644 scripts/download_foundationpose_weights.py create mode 100644 scripts/evaluate_yesterday_spider.py create mode 100644 scripts/export_camera_delivery.py create mode 100644 scripts/export_dynhamr_bimanual_rgbd.py create mode 100644 scripts/fit_rgbd_objects.py create mode 100644 scripts/fix_yesterday_hand_collision.py create mode 100644 scripts/foundationpose_env.sh create mode 100644 scripts/interpolate_collision_reference.py create mode 100644 scripts/large_files.py create mode 100644 scripts/play_compare.py create mode 100644 scripts/play_spider_result.py create mode 100644 scripts/prepare_rgbd_20260915.py create mode 100644 scripts/prepare_yesterday_spider.py create mode 100644 scripts/probe_collision_dynamics.py create mode 100644 scripts/probe_spider_gpu_contact.py create mode 100644 scripts/red_box_distance.py create mode 100644 scripts/refit_rgbd_objects_outline.py create mode 100644 scripts/refit_rgbd_objects_tracked.py create mode 100644 scripts/register_hands_rgbd.py create mode 100644 scripts/render_collision_fix.py create mode 100644 scripts/render_corrected_dynhamr_rgbd.py create mode 100644 scripts/render_foundationpose_red.py create mode 100644 scripts/render_object_alignment_overlay.py create mode 100644 scripts/render_red_depth_comparison.py create mode 100644 scripts/render_rgbd_l20_objects.py create mode 100644 scripts/repair_red_box_topology.py create mode 100644 scripts/replay_yesterday_spider_gpu.py create mode 100644 scripts/report_collision_fix.py create mode 100644 scripts/report_spider_dynamics_fix.py create mode 100644 scripts/report_yesterday_spider.py create mode 100644 scripts/rl_track_reference.py create mode 100644 scripts/run_dynhamr_rgbd_20260915.py create mode 100644 scripts/run_red_handflow_comparison.py create mode 100644 scripts/run_red_mirrored_handflow.py create mode 100644 scripts/run_rgbd_20260915.py create mode 100644 scripts/run_yesterday_spider.py create mode 100644 scripts/smooth_object_reference.py create mode 100644 scripts/smooth_project_reference.py create mode 100644 scripts/smooth_refitted_objects.py create mode 100644 scripts/smooth_table_constrained.py create mode 100644 scripts/track_red_box_foundationpose.py create mode 100644 scripts/track_yesterday_boxes.py create mode 100644 scripts/unmirror_left_rgbd.py create mode 100644 scripts/verify_fix_interpolation.py create mode 100644 scripts/verify_foundationpose_setup.py create mode 100644 scripts/verify_rgbd_bimanual.py create mode 120000 third_party/Dyn-HaMR/_DATA/BMC create mode 120000 third_party/Dyn-HaMR/_DATA/data/mano create mode 120000 third_party/Dyn-HaMR/_DATA/data/mano_mean_params.npz create mode 120000 third_party/Dyn-HaMR/_DATA/hamer_ckpts/checkpoints create mode 100644 third_party/Dyn-HaMR/_DATA/hamer_ckpts/dataset_config.yaml create mode 100644 third_party/Dyn-HaMR/_DATA/hamer_ckpts/model_config.yaml create mode 100644 third_party/Dyn-HaMR/_DATA/hmp_model/mean-neutral-128-30fps.pt create mode 100644 third_party/Dyn-HaMR/_DATA/hmp_model/results/model/global_encoder.pth create mode 100644 third_party/Dyn-HaMR/_DATA/hmp_model/results/model/local_encoder.pth.part-000 create mode 100644 third_party/Dyn-HaMR/_DATA/hmp_model/results/model/local_encoder.pth.part-001 create mode 100644 third_party/Dyn-HaMR/_DATA/hmp_model/results/model/local_encoder.pth.part-002 create mode 100644 third_party/Dyn-HaMR/_DATA/hmp_model/results/model/local_encoder.pth.part-003 create mode 100644 third_party/Dyn-HaMR/_DATA/hmp_model/results/model/loss.pkl create mode 100644 third_party/Dyn-HaMR/_DATA/hmp_model/results/model/nemf.pth.part-000 create mode 100644 third_party/Dyn-HaMR/_DATA/hmp_model/results/model/nemf.pth.part-001 create mode 100644 third_party/Dyn-HaMR/_DATA/hmp_model/results/model/optimizer.pth.part-000 create mode 100644 third_party/Dyn-HaMR/_DATA/hmp_model/results/model/optimizer.pth.part-001 create mode 100644 third_party/Dyn-HaMR/_DATA/hmp_model/results/model/optimizer.pth.part-002 create mode 100644 third_party/Dyn-HaMR/_DATA/hmp_model/results/model/optimizer.pth.part-003 create mode 100644 third_party/Dyn-HaMR/_DATA/hmp_model/results/model/optimizer.pth.part-004 create mode 100644 third_party/Dyn-HaMR/_DATA/hmp_model/results/model/optimizer.pth.part-005 create mode 100644 third_party/Dyn-HaMR/_DATA/hmp_model/results/model/optimizer.pth.part-006 create mode 100644 third_party/Dyn-HaMR/_DATA/hmp_model/results/model/optimizer.pth.part-007 create mode 100644 third_party/Dyn-HaMR/_DATA/hmp_model/results/model/optimizer.pth.part-008 create mode 100644 third_party/Dyn-HaMR/_DATA/hmp_model/results/model/optimizer.pth.part-009 create mode 100644 third_party/Dyn-HaMR/_DATA/hmp_model/results/model/optimizer.pth.part-010 create mode 100644 third_party/Dyn-HaMR/_DATA/hmp_model/std-neutral-128-30fps.pt create mode 100644 third_party/Dyn-HaMR/dyn-hamr/optim/BMC/CONVEX_HULLS.npy create mode 100644 third_party/Dyn-HaMR/dyn-hamr/optim/BMC/PHI_max.npy create mode 100644 third_party/Dyn-HaMR/dyn-hamr/optim/BMC/PHI_min.npy create mode 100644 third_party/Dyn-HaMR/dyn-hamr/optim/BMC/bone_len_max.npy create mode 100644 third_party/Dyn-HaMR/dyn-hamr/optim/BMC/bone_len_min.npy create mode 100644 third_party/Dyn-HaMR/dyn-hamr/optim/BMC/curvatures_max.npy create mode 100644 third_party/Dyn-HaMR/dyn-hamr/optim/BMC/curvatures_min.npy create mode 100644 third_party/Dyn-HaMR/dyn-hamr/optim/BMC/joint_angles.npy.part-000 create mode 100644 third_party/Dyn-HaMR/dyn-hamr/optim/BMC/joint_angles.npy.part-001 create mode 120000 third_party/Dyn-HaMR/third-party/hamer/pretrained_models/detector.pt create mode 100644 third_party/FoundationPose/LICENSE create mode 100644 third_party/FoundationPose/Utils.py create mode 100644 third_party/FoundationPose/build_all.sh create mode 100755 third_party/FoundationPose/build_all_conda.sh create mode 100644 third_party/FoundationPose/bundlesdf/config_linemod.yml create mode 100644 third_party/FoundationPose/bundlesdf/config_ycbv.yml create mode 100644 third_party/FoundationPose/bundlesdf/mycuda/__init__.py create mode 100644 third_party/FoundationPose/bundlesdf/mycuda/bindings.cpp create mode 100644 third_party/FoundationPose/bundlesdf/mycuda/common.cu create mode 100644 third_party/FoundationPose/bundlesdf/mycuda/common.h create mode 100644 third_party/FoundationPose/bundlesdf/mycuda/setup.py create mode 100644 third_party/FoundationPose/bundlesdf/mycuda/torch_ngp_grid_encoder/LICENSE create mode 100644 third_party/FoundationPose/bundlesdf/mycuda/torch_ngp_grid_encoder/bindings.cpp create mode 100644 third_party/FoundationPose/bundlesdf/mycuda/torch_ngp_grid_encoder/grid.py create mode 100644 third_party/FoundationPose/bundlesdf/mycuda/torch_ngp_grid_encoder/gridencoder.cu create mode 100644 third_party/FoundationPose/bundlesdf/mycuda/torch_ngp_grid_encoder/gridencoder.h create mode 100644 third_party/FoundationPose/bundlesdf/nerf_helpers.py create mode 100644 third_party/FoundationPose/bundlesdf/nerf_runner.py create mode 100644 third_party/FoundationPose/bundlesdf/run_nerf.py create mode 100644 third_party/FoundationPose/bundlesdf/tool.py create mode 100755 third_party/FoundationPose/check_env.py create mode 100644 third_party/FoundationPose/datareader.py create mode 100644 third_party/FoundationPose/docker/run_container.sh create mode 100644 third_party/FoundationPose/environment.yml create mode 100644 third_party/FoundationPose/estimater.py create mode 100644 third_party/FoundationPose/learning/datasets/h5_dataset.py create mode 100644 third_party/FoundationPose/learning/datasets/pose_dataset.py create mode 100644 third_party/FoundationPose/learning/models/network_modules.py create mode 100644 third_party/FoundationPose/learning/models/refine_network.py create mode 100644 third_party/FoundationPose/learning/models/score_network.py create mode 100644 third_party/FoundationPose/learning/training/predict_pose_refine.py create mode 100644 third_party/FoundationPose/learning/training/predict_score.py create mode 100644 third_party/FoundationPose/learning/training/training_config.py create mode 100644 third_party/FoundationPose/mycpp/CMakeLists.txt create mode 100644 third_party/FoundationPose/mycpp/include/Utils.h create mode 100644 third_party/FoundationPose/mycpp/src/Utils.cpp create mode 100644 third_party/FoundationPose/mycpp/src/app/pybind_api.cpp create mode 100644 third_party/FoundationPose/offscreen_renderer.py create mode 100644 third_party/FoundationPose/readme.md create mode 100644 third_party/FoundationPose/requirements.txt create mode 100644 third_party/FoundationPose/run_demo.py create mode 100644 third_party/FoundationPose/run_linemod.py create mode 100644 third_party/FoundationPose/run_ycb_video.py create mode 100644 third_party/FoundationPose/weights/2023-10-28-18-33-37/config.yml create mode 100644 third_party/FoundationPose/weights/2023-10-28-18-33-37/model_best.pth.part-000 create mode 100644 third_party/FoundationPose/weights/2023-10-28-18-33-37/model_best.pth.part-001 create mode 100644 third_party/FoundationPose/weights/2024-01-11-20-02-45/config.yml create mode 100644 third_party/FoundationPose/weights/2024-01-11-20-02-45/model_best.pth.part-000 create mode 100644 third_party/FoundationPose/weights/2024-01-11-20-02-45/model_best.pth.part-001 create mode 100644 third_party/FoundationPose/weights/2024-01-11-20-02-45/model_best.pth.part-002 create mode 100644 third_party/FoundationPose/weights/2024-01-11-20-02-45/model_best.pth.part-003 create mode 100644 third_party/FoundationPose/weights/download_manifest.json create mode 100644 third_party/bottle_model_parametric/README.md create mode 100644 third_party/bottle_model_parametric/assets/bottle.obj create mode 100644 third_party/bottle_model_parametric/assets/shoulder.obj create mode 100644 third_party/bottle_model_parametric/bottle.glb create mode 100644 third_party/bottle_model_parametric/bottle.stl create mode 100644 third_party/bottle_model_parametric/bottle.xml create mode 100644 third_party/bottle_model_parametric/preview.png create mode 100644 third_party/bottle_model_parametric/profile.npz create mode 100644 third_party/bottle_model_parametric/validation.json create mode 100644 third_party/dex-assets/robots/hands/leap_hand/LICENSE.txt create mode 100644 third_party/dex-assets/robots/hands/leap_hand/leap_hand_right.urdf create mode 100644 third_party/dex-assets/robots/hands/leap_hand/meshes/collision/white_tip.obj create mode 100644 third_party/dex-assets/robots/hands/leap_hand/meshes/collision/white_tip_thumb.obj create mode 100644 third_party/dex-assets/robots/hands/leap_hand/meshes/visual/dip.obj create mode 100644 third_party/dex-assets/robots/hands/leap_hand/meshes/visual/fingertip.obj create mode 100644 third_party/dex-assets/robots/hands/leap_hand/meshes/visual/mcp_joint.obj create mode 100644 third_party/dex-assets/robots/hands/leap_hand/meshes/visual/palm_lower.obj create mode 100644 third_party/dex-assets/robots/hands/leap_hand/meshes/visual/pip.obj create mode 100644 third_party/dex-assets/robots/hands/leap_hand/meshes/visual/thumb_dip.obj create mode 100644 third_party/dex-assets/robots/hands/leap_hand/meshes/visual/thumb_fingertip.obj create mode 100644 third_party/dex-assets/robots/hands/leap_hand/meshes/visual/thumb_pip.obj create mode 100644 third_party/hamer/_DATA/data/mano/README.txt create mode 100644 third_party/hamer/_DATA/data/mano_mean_params.npz create mode 100644 third_party/hamer/_DATA/hamer_ckpts/checkpoints/hamer.ckpt.part-000 create mode 100644 third_party/hamer/_DATA/hamer_ckpts/checkpoints/hamer.ckpt.part-001 create mode 100644 third_party/hamer/_DATA/hamer_ckpts/checkpoints/hamer.ckpt.part-002 create mode 100644 third_party/hamer/_DATA/hamer_ckpts/checkpoints/hamer.ckpt.part-003 create mode 100644 third_party/hamer/_DATA/hamer_ckpts/checkpoints/hamer.ckpt.part-004 create mode 100644 third_party/hamer/_DATA/hamer_ckpts/checkpoints/hamer.ckpt.part-005 create mode 100644 third_party/hamer/_DATA/hamer_ckpts/checkpoints/hamer.ckpt.part-006 create mode 100644 third_party/hamer/_DATA/hamer_ckpts/checkpoints/hamer.ckpt.part-007 create mode 100644 third_party/hamer/_DATA/hamer_ckpts/checkpoints/hamer.ckpt.part-008 create mode 100644 third_party/hamer/_DATA/hamer_ckpts/checkpoints/hamer.ckpt.part-009 create mode 100644 third_party/hamer/_DATA/hamer_ckpts/checkpoints/hamer.ckpt.part-010 create mode 100644 third_party/hamer/_DATA/hamer_ckpts/checkpoints/hamer.ckpt.part-011 create mode 100644 third_party/hamer/_DATA/hamer_ckpts/checkpoints/hamer.ckpt.part-012 create mode 100644 third_party/hamer/_DATA/hamer_ckpts/checkpoints/hamer.ckpt.part-013 create mode 100644 third_party/hamer/_DATA/hamer_ckpts/checkpoints/hamer.ckpt.part-014 create mode 100644 third_party/hamer/_DATA/hamer_ckpts/checkpoints/hamer.ckpt.part-015 create mode 100644 third_party/hamer/_DATA/hamer_ckpts/checkpoints/hamer.ckpt.part-016 create mode 100644 third_party/hamer/_DATA/hamer_ckpts/checkpoints/hamer.ckpt.part-017 create mode 100644 third_party/hamer/_DATA/hamer_ckpts/checkpoints/hamer.ckpt.part-018 create mode 100644 third_party/hamer/_DATA/hamer_ckpts/checkpoints/hamer.ckpt.part-019 create mode 100644 third_party/hamer/_DATA/hamer_ckpts/checkpoints/hamer.ckpt.part-020 create mode 100644 third_party/hamer/_DATA/hamer_ckpts/checkpoints/hamer.ckpt.part-021 create mode 100644 third_party/hamer/_DATA/hamer_ckpts/checkpoints/hamer.ckpt.part-022 create mode 100644 third_party/hamer/_DATA/hamer_ckpts/checkpoints/hamer.ckpt.part-023 create mode 100644 third_party/hamer/_DATA/hamer_ckpts/checkpoints/hamer.ckpt.part-024 create mode 100644 third_party/hamer/_DATA/hamer_ckpts/checkpoints/hamer.ckpt.part-025 create mode 100644 third_party/hamer/_DATA/hamer_ckpts/checkpoints/hamer.ckpt.part-026 create mode 100644 third_party/hamer/_DATA/hamer_ckpts/checkpoints/hamer.ckpt.part-027 create mode 100644 third_party/hamer/_DATA/hamer_ckpts/checkpoints/hamer.ckpt.part-028 create mode 100644 third_party/hamer/_DATA/hamer_ckpts/checkpoints/hamer.ckpt.part-029 create mode 100644 third_party/hamer/_DATA/hamer_ckpts/checkpoints/hamer.ckpt.part-030 create mode 100644 third_party/hamer/_DATA/hamer_ckpts/checkpoints/hamer.ckpt.part-031 create mode 100644 third_party/hamer/_DATA/hamer_ckpts/checkpoints/hamer.ckpt.part-032 create mode 100644 third_party/hamer/_DATA/hamer_ckpts/checkpoints/hamer.ckpt.part-033 create mode 100644 third_party/hamer/_DATA/hamer_ckpts/checkpoints/hamer.ckpt.part-034 create mode 100644 third_party/hamer/_DATA/hamer_ckpts/checkpoints/hamer.ckpt.part-035 create mode 100644 third_party/hamer/_DATA/hamer_ckpts/checkpoints/hamer.ckpt.part-036 create mode 100644 third_party/hamer/_DATA/hamer_ckpts/checkpoints/hamer.ckpt.part-037 create mode 100644 third_party/hamer/_DATA/hamer_ckpts/checkpoints/hamer.ckpt.part-038 create mode 100644 third_party/hamer/_DATA/hamer_ckpts/checkpoints/hamer.ckpt.part-039 create mode 100644 third_party/hamer/_DATA/hamer_ckpts/checkpoints/hamer.ckpt.part-040 create mode 100644 third_party/hamer/_DATA/hamer_ckpts/checkpoints/hamer.ckpt.part-041 create mode 100644 third_party/hamer/_DATA/hamer_ckpts/checkpoints/hamer.ckpt.part-042 create mode 100644 third_party/hamer/_DATA/hamer_ckpts/checkpoints/hamer.ckpt.part-043 create mode 100644 third_party/hamer/_DATA/hamer_ckpts/checkpoints/hamer.ckpt.part-044 create mode 100644 third_party/hamer/_DATA/hamer_ckpts/checkpoints/hamer.ckpt.part-045 create mode 100644 third_party/hamer/_DATA/hamer_ckpts/checkpoints/hamer.ckpt.part-046 create mode 100644 third_party/hamer/_DATA/hamer_ckpts/checkpoints/hamer.ckpt.part-047 create mode 100644 third_party/hamer/_DATA/hamer_ckpts/checkpoints/hamer.ckpt.part-048 create mode 100644 third_party/hamer/_DATA/hamer_ckpts/checkpoints/hamer.ckpt.part-049 create mode 100644 third_party/hamer/_DATA/hamer_ckpts/checkpoints/hamer.ckpt.part-050 create mode 100644 third_party/hamer/_DATA/hamer_ckpts/checkpoints/hamer.ckpt.part-051 create mode 100644 third_party/hamer/_DATA/hamer_ckpts/checkpoints/hamer.ckpt.part-052 create mode 100644 third_party/hamer/_DATA/hamer_ckpts/checkpoints/hamer.ckpt.part-053 create mode 100644 third_party/hamer/_DATA/hamer_ckpts/model_config.yaml create mode 100644 third_party/l20_assets/L20/LEFT/g20_left_G20.json create mode 100644 third_party/l20_assets/L20/LEFT/linkerhand_g20_left.urdf create mode 100644 third_party/l20_assets/L20/LEFT/meshes/hand_base_link.STL create mode 100644 third_party/l20_assets/L20/LEFT/meshes/index_distal.STL create mode 100644 third_party/l20_assets/L20/LEFT/meshes/index_metacarpals.STL create mode 100644 third_party/l20_assets/L20/LEFT/meshes/index_middle.STL create mode 100644 third_party/l20_assets/L20/LEFT/meshes/index_proximal.STL create mode 100644 third_party/l20_assets/L20/LEFT/meshes/middle_distal.STL create mode 100644 third_party/l20_assets/L20/LEFT/meshes/middle_metacarpals.STL create mode 100644 third_party/l20_assets/L20/LEFT/meshes/middle_middle.STL create mode 100644 third_party/l20_assets/L20/LEFT/meshes/middle_proximal.STL create mode 100644 third_party/l20_assets/L20/LEFT/meshes/pinky_distal.STL create mode 100644 third_party/l20_assets/L20/LEFT/meshes/pinky_metacarpals.STL create mode 100644 third_party/l20_assets/L20/LEFT/meshes/pinky_middle.STL create mode 100644 third_party/l20_assets/L20/LEFT/meshes/pinky_proximal.STL create mode 100644 third_party/l20_assets/L20/LEFT/meshes/ring_distal.STL create mode 100644 third_party/l20_assets/L20/LEFT/meshes/ring_metacarpals.STL create mode 100644 third_party/l20_assets/L20/LEFT/meshes/ring_middle.STL create mode 100644 third_party/l20_assets/L20/LEFT/meshes/ring_proximal.STL create mode 100644 third_party/l20_assets/L20/LEFT/meshes/thumb_distal.STL create mode 100644 third_party/l20_assets/L20/LEFT/meshes/thumb_metacarpals.STL create mode 100644 third_party/l20_assets/L20/LEFT/meshes/thumb_metacarpals_base1.STL create mode 100644 third_party/l20_assets/L20/LEFT/meshes/thumb_metacarpals_base2.STL create mode 100644 third_party/l20_assets/L20/LEFT/meshes/thumb_proximal.STL create mode 100644 third_party/l20_assets/L20/RIGHT/g20_right_G20.json create mode 100644 third_party/l20_assets/L20/RIGHT/linkerhand_g20_right.urdf create mode 100644 third_party/l20_assets/L20/RIGHT/linkerhand_g20_right_mj.urdf create mode 100644 third_party/l20_assets/L20/RIGHT/meshes/hand_base_link.STL create mode 100644 third_party/l20_assets/L20/RIGHT/meshes/index_distal.STL create mode 100644 third_party/l20_assets/L20/RIGHT/meshes/index_metacarpals.STL create mode 100644 third_party/l20_assets/L20/RIGHT/meshes/index_middle.STL create mode 100644 third_party/l20_assets/L20/RIGHT/meshes/index_proximal.STL create mode 100644 third_party/l20_assets/L20/RIGHT/meshes/middle_distal.STL create mode 100644 third_party/l20_assets/L20/RIGHT/meshes/middle_metacarpals.STL create mode 100644 third_party/l20_assets/L20/RIGHT/meshes/middle_middle.STL create mode 100644 third_party/l20_assets/L20/RIGHT/meshes/middle_proximal.STL create mode 100644 third_party/l20_assets/L20/RIGHT/meshes/pinky_distal.STL create mode 100644 third_party/l20_assets/L20/RIGHT/meshes/pinky_metacarpals.STL create mode 100644 third_party/l20_assets/L20/RIGHT/meshes/pinky_middle.STL create mode 100644 third_party/l20_assets/L20/RIGHT/meshes/pinky_proximal.STL create mode 100644 third_party/l20_assets/L20/RIGHT/meshes/ring_distal.STL create mode 100644 third_party/l20_assets/L20/RIGHT/meshes/ring_metacarpals.STL create mode 100644 third_party/l20_assets/L20/RIGHT/meshes/ring_middle.STL create mode 100644 third_party/l20_assets/L20/RIGHT/meshes/ring_proximal.STL create mode 100644 third_party/l20_assets/L20/RIGHT/meshes/thumb_distal.STL create mode 100644 third_party/l20_assets/L20/RIGHT/meshes/thumb_metacarpals.STL create mode 100644 third_party/l20_assets/L20/RIGHT/meshes/thumb_metacarpals_base1.STL create mode 100644 third_party/l20_assets/L20/RIGHT/meshes/thumb_metacarpals_base2.STL create mode 100644 third_party/l20_assets/L20/RIGHT/meshes/thumb_proximal.STL create mode 100644 third_party/nvdiffrast/Dockerfile create mode 100644 third_party/nvdiffrast/LICENSE.txt create mode 100644 third_party/nvdiffrast/MANIFEST.in create mode 100644 third_party/nvdiffrast/README.md create mode 100644 third_party/nvdiffrast/csrc/common/antialias.cu create mode 100644 third_party/nvdiffrast/csrc/common/antialias.h create mode 100644 third_party/nvdiffrast/csrc/common/common.cpp create mode 100644 third_party/nvdiffrast/csrc/common/common.h create mode 100644 third_party/nvdiffrast/csrc/common/cudaraster/CudaRaster.hpp create mode 100644 third_party/nvdiffrast/csrc/common/cudaraster/impl/Buffer.cpp create mode 100644 third_party/nvdiffrast/csrc/common/cudaraster/impl/Buffer.hpp create mode 100644 third_party/nvdiffrast/csrc/common/cudaraster/impl/Constants.hpp create mode 100644 third_party/nvdiffrast/csrc/common/cudaraster/impl/CudaRaster.cpp create mode 100644 third_party/nvdiffrast/csrc/common/cudaraster/impl/Defs.hpp create mode 100644 third_party/nvdiffrast/csrc/common/cudaraster/impl/PrivateDefs.hpp create mode 100644 third_party/nvdiffrast/csrc/common/cudaraster/impl/RasterImpl.cpp create mode 100644 third_party/nvdiffrast/csrc/common/cudaraster/impl/RasterImpl.hpp create mode 100644 third_party/nvdiffrast/csrc/common/cudaraster/impl/RasterImpl_kernel.cu create mode 100644 third_party/nvdiffrast/csrc/common/framework.h create mode 100644 third_party/nvdiffrast/csrc/common/interpolate.cu create mode 100644 third_party/nvdiffrast/csrc/common/interpolate.h create mode 100644 third_party/nvdiffrast/csrc/common/rasterize.cu create mode 100644 third_party/nvdiffrast/csrc/common/rasterize.h create mode 100644 third_party/nvdiffrast/csrc/common/texture.cpp create mode 100644 third_party/nvdiffrast/csrc/common/texture.h create mode 100644 third_party/nvdiffrast/csrc/common/texture_kernel.cu create mode 100644 third_party/nvdiffrast/csrc/torch/torch_antialias.cpp create mode 100644 third_party/nvdiffrast/csrc/torch/torch_bindings.cpp create mode 100644 third_party/nvdiffrast/csrc/torch/torch_interpolate.cpp create mode 100644 third_party/nvdiffrast/csrc/torch/torch_rasterize.cpp create mode 100644 third_party/nvdiffrast/csrc/torch/torch_texture.cpp create mode 100644 third_party/nvdiffrast/csrc/torch/torch_types.h create mode 100644 third_party/nvdiffrast/docs/index.html create mode 100644 third_party/nvdiffrast/nvdiffrast/__init__.py create mode 100644 third_party/nvdiffrast/nvdiffrast/torch/__init__.py create mode 100644 third_party/nvdiffrast/nvdiffrast/torch/ops.py create mode 100644 third_party/nvdiffrast/pyproject.toml create mode 100644 third_party/nvdiffrast/samples/data/NOTICE.txt create mode 100644 third_party/nvdiffrast/samples/torch/cube.py create mode 100644 third_party/nvdiffrast/samples/torch/earth.py create mode 100644 third_party/nvdiffrast/samples/torch/envphong.py create mode 100644 third_party/nvdiffrast/samples/torch/pose.py create mode 100644 third_party/nvdiffrast/samples/torch/triangle.py create mode 100644 third_party/nvdiffrast/samples/torch/util.py create mode 100644 third_party/nvdiffrast/setup.py create mode 100644 weights/detector.pt.part-000 create mode 100644 weights/detector.pt.part-001 create mode 100644 weights/handflow_denoiser.pt.part-000 create mode 100644 weights/handflow_denoiser.pt.part-001 create mode 100644 weights/handflow_denoiser.pt.part-002 create mode 100644 weights/handflow_denoiser.pt.part-003 create mode 100644 weights/handflow_denoiser.pt.part-004 create mode 100644 weights/handflow_denoiser.pt.part-005 create mode 100644 weights/handflow_denoiser.pt.part-006 create mode 100644 weights/handflow_denoiser.pt.part-007 create mode 100644 weights/handflow_denoiser.pt.part-008 create mode 100644 weights/handflow_denoiser.pt.part-009 create mode 100644 weights/handflow_denoiser.pt.part-010 create mode 100644 weights/handflow_denoiser.pt.part-011 create mode 100644 weights/handflow_denoiser.pt.part-012 create mode 100644 weights/handflow_denoiser.pt.part-013 create mode 100644 weights/normalization_stats.npz create mode 100644 weights/unidepth-v2-vitl14/config.json create mode 100644 weights/unidepth-v2-vitl14/model.safetensors.part-000 create mode 100644 weights/unidepth-v2-vitl14/model.safetensors.part-001 create mode 100644 weights/unidepth-v2-vitl14/model.safetensors.part-002 create mode 100644 weights/unidepth-v2-vitl14/model.safetensors.part-003 create mode 100644 weights/unidepth-v2-vitl14/model.safetensors.part-004 create mode 100644 weights/unidepth-v2-vitl14/model.safetensors.part-005 create mode 100644 weights/unidepth-v2-vitl14/model.safetensors.part-006 create mode 100644 weights/unidepth-v2-vitl14/model.safetensors.part-007 create mode 100644 weights/unidepth-v2-vitl14/model.safetensors.part-008 create mode 100644 weights/unidepth-v2-vitl14/model.safetensors.part-009 create mode 100644 weights/unidepth-v2-vitl14/model.safetensors.part-010 create mode 100644 weights/unidepth-v2-vitl14/model.safetensors.part-011 create mode 100644 weights/unidepth-v2-vitl14/model.safetensors.part-012 create mode 100644 weights/unidepth-v2-vitl14/model.safetensors.part-013 create mode 100644 weights/unidepth-v2-vitl14/model.safetensors.part-014 create mode 100644 weights/unidepth-v2-vitl14/model.safetensors.part-015 create mode 100644 weights/unidepth-v2-vitl14/model.safetensors.part-016 create mode 100644 weights/unidepth-v2-vitl14/model.safetensors.part-017 create mode 100644 weights/unidepth-v2-vitl14/model.safetensors.part-018 create mode 100644 weights/unidepth-v2-vitl14/model.safetensors.part-019 create mode 100644 weights/unidepth-v2-vitl14/model.safetensors.part-020 create mode 100644 weights/unidepth-v2-vitl14/model.safetensors.part-021 create mode 100644 weights/unidepth-v2-vitl14/model.safetensors.part-022 create mode 100644 weights/unidepth-v2-vitl14/model.safetensors.part-023 create mode 100644 weights/unidepth-v2-vitl14/model.safetensors.part-024 create mode 100644 weights/unidepth-v2-vitl14/model.safetensors.part-025 create mode 100644 weights/unidepth-v2-vitl14/model.safetensors.part-026 create mode 100644 weights/unidepth-v2-vitl14/model.safetensors.part-027 create mode 100644 weights/unidepth-v2-vitl14/model.safetensors.part-028 diff --git a/.gitattributes b/.gitattributes new file mode 100644 index 0000000..30039c4 --- /dev/null +++ b/.gitattributes @@ -0,0 +1,17 @@ +# Large binaries go through Git LFS +*.pt filter=lfs diff=lfs merge=lfs -text +*.pth filter=lfs diff=lfs merge=lfs -text +*.ckpt filter=lfs diff=lfs merge=lfs -text +*.safetensors filter=lfs diff=lfs merge=lfs -text +*.pkl filter=lfs diff=lfs merge=lfs -text +*.npz filter=lfs diff=lfs merge=lfs -text +*.npy filter=lfs diff=lfs merge=lfs -text +*.STL filter=lfs diff=lfs merge=lfs -text +*.stl filter=lfs diff=lfs merge=lfs -text +*.obj filter=lfs diff=lfs merge=lfs -text +*.ply filter=lfs diff=lfs merge=lfs -text +*.png filter=lfs diff=lfs merge=lfs -text +*.mp4 filter=lfs diff=lfs merge=lfs -text +*.mkv filter=lfs diff=lfs merge=lfs -text +*.tar.gz filter=lfs diff=lfs merge=lfs -text +*.part-[0-9][0-9][0-9] filter=lfs diff=lfs merge=lfs -text diff --git a/.gitignore b/.gitignore index 6a12a9b..0273d5a 100644 --- a/.gitignore +++ b/.gitignore @@ -1,14 +1,11 @@ -# Source snapshot: runtime exclusions anchored to repository root. -/output/ -/results/ -/logs/ -/weights/ -/dist/ +# Environments, toolchains and caches are never committed (see docs/SETUP_AND_WEIGHTS.md) /.venv/ /venv/ /.dex/ /.spider/ +/third_party/Dyn-HaMR/.dynhamr/ /.cuda/ +/.cache/ /.hf_cache/ /.torch_cache/ /.spider_cache/ @@ -16,3 +13,31 @@ /.env __pycache__/ *.pyc +*.egg-info/ +/third_party/*/build/ +/third_party/FoundationPose/mycpp/build/ +*.part +*.aria2 +# Run outputs and packaged releases +/output/ +/results/ +/logs/ +/dist/ +# Licensed models that every user must obtain themselves (MANO) +/third_party/hamer/_DATA/data/mano/*.pkl +# Local tooling +.claude/ +CLAUDE.md +.vscode/ +.idea/ +# Reassembled from *.part-NNN chunks by scripts/large_files.py (never committed whole: server limit 50 MB/request) +/third_party/hamer/_DATA/hamer_ckpts/checkpoints/hamer.ckpt +/weights/unidepth-v2-vitl14/model.safetensors +/weights/handflow_denoiser.pt +/weights/detector.pt +/third_party/Dyn-HaMR/_DATA/hmp_model/results/model/optimizer.pth +/third_party/Dyn-HaMR/_DATA/hmp_model/results/model/local_encoder.pth +/third_party/Dyn-HaMR/_DATA/hmp_model/results/model/nemf.pth +/third_party/Dyn-HaMR/dyn-hamr/optim/BMC/joint_angles.npy +/third_party/FoundationPose/weights/2024-01-11-20-02-45/model_best.pth +/third_party/FoundationPose/weights/2023-10-28-18-33-37/model_best.pth diff --git a/EXCLUDED_FILES.json b/EXCLUDED_FILES.json index 746424c..2b5a945 100644 --- a/EXCLUDED_FILES.json +++ b/EXCLUDED_FILES.json @@ -1,24 +1,76 @@ [ + { + "path": "scripts/__pycache__/audit_collision_fix.cpython-310.pyc", + "reason": "runtime/data/cache directory" + }, + { + "path": "scripts/__pycache__/audit_collision_physics.cpython-310.pyc", + "reason": "runtime/data/cache directory" + }, { "path": "scripts/__pycache__/audit_l20_jitter.cpython-311.pyc", "reason": "runtime/data/cache directory" }, + { + "path": "scripts/__pycache__/audit_spider_dynamic_surfaces.cpython-310.pyc", + "reason": "runtime/data/cache directory" + }, + { + "path": "scripts/__pycache__/audit_yesterday_geometry.cpython-310.pyc", + "reason": "runtime/data/cache directory" + }, + { + "path": "scripts/__pycache__/build_box_collision_v2.cpython-310.pyc", + "reason": "runtime/data/cache directory" + }, { "path": "scripts/__pycache__/build_l20_full_replay.cpython-310.pyc", "reason": "runtime/data/cache directory" }, + { + "path": "scripts/__pycache__/build_palm_collision_v2.cpython-310.pyc", + "reason": "runtime/data/cache directory" + }, { "path": "scripts/__pycache__/build_parametric_bottle.cpython-311.pyc", "reason": "runtime/data/cache directory" }, + { + "path": "scripts/__pycache__/check_spider_dynamics_replay.cpython-310.pyc", + "reason": "runtime/data/cache directory" + }, + { + "path": "scripts/__pycache__/check_spider_state_restore.cpython-310.pyc", + "reason": "runtime/data/cache directory" + }, + { + "path": "scripts/__pycache__/configure_collision_physics.cpython-310.pyc", + "reason": "runtime/data/cache directory" + }, + { + "path": "scripts/__pycache__/configure_spider_stable_contact.cpython-310.pyc", + "reason": "runtime/data/cache directory" + }, { "path": "scripts/__pycache__/evaluate_spider_l20.cpython-311.pyc", "reason": "runtime/data/cache directory" }, + { + "path": "scripts/__pycache__/evaluate_yesterday_spider.cpython-310.pyc", + "reason": "runtime/data/cache directory" + }, { "path": "scripts/__pycache__/export_handflow_dex.cpython-310.pyc", "reason": "runtime/data/cache directory" }, + { + "path": "scripts/__pycache__/fix_yesterday_hand_collision.cpython-310.pyc", + "reason": "runtime/data/cache directory" + }, + { + "path": "scripts/__pycache__/interpolate_collision_reference.cpython-310.pyc", + "reason": "runtime/data/cache directory" + }, { "path": "scripts/__pycache__/l20_bottle_physics.cpython-311.pyc", "reason": "runtime/data/cache directory" @@ -59,6 +111,22 @@ "path": "scripts/__pycache__/prepare_spider_l20.cpython-311.pyc", "reason": "runtime/data/cache directory" }, + { + "path": "scripts/__pycache__/prepare_yesterday_spider.cpython-310.pyc", + "reason": "runtime/data/cache directory" + }, + { + "path": "scripts/__pycache__/probe_spider_gpu_contact.cpython-310.pyc", + "reason": "runtime/data/cache directory" + }, + { + "path": "scripts/__pycache__/red_box_distance.cpython-311.pyc", + "reason": "runtime/data/cache directory" + }, + { + "path": "scripts/__pycache__/render_collision_fix.cpython-310.pyc", + "reason": "runtime/data/cache directory" + }, { "path": "scripts/__pycache__/render_l20.cpython-310.pyc", "reason": "runtime/data/cache directory" @@ -79,6 +147,22 @@ "path": "scripts/__pycache__/replay_spider_l20_gpu.cpython-311.pyc", "reason": "runtime/data/cache directory" }, + { + "path": "scripts/__pycache__/replay_yesterday_spider_gpu.cpython-310.pyc", + "reason": "runtime/data/cache directory" + }, + { + "path": "scripts/__pycache__/report_collision_fix.cpython-310.pyc", + "reason": "runtime/data/cache directory" + }, + { + "path": "scripts/__pycache__/report_spider_dynamics_fix.cpython-310.pyc", + "reason": "runtime/data/cache directory" + }, + { + "path": "scripts/__pycache__/report_yesterday_spider.cpython-310.pyc", + "reason": "runtime/data/cache directory" + }, { "path": "scripts/__pycache__/retarget_l20_video.cpython-310.pyc", "reason": "runtime/data/cache directory" @@ -99,6 +183,10 @@ "path": "scripts/__pycache__/run_spider_l20_contact.cpython-311.pyc", "reason": "runtime/data/cache directory" }, + { + "path": "scripts/__pycache__/run_yesterday_spider.cpython-310.pyc", + "reason": "runtime/data/cache directory" + }, { "path": "scripts/__pycache__/smooth_l20_offline.cpython-311.pyc", "reason": "runtime/data/cache directory" @@ -107,6 +195,14 @@ "path": "scripts/__pycache__/stabilize_l20_temporal.cpython-310.pyc", "reason": "runtime/data/cache directory" }, + { + "path": "scripts/__pycache__/track_red_box_foundationpose.cpython-311.pyc", + "reason": "runtime/data/cache directory" + }, + { + "path": "scripts/__pycache__/verify_fix_interpolation.cpython-310.pyc", + "reason": "runtime/data/cache directory" + }, { "path": "scripts/__pycache__/verify_l20_bottle.cpython-311.pyc", "reason": "runtime/data/cache directory" @@ -115,6 +211,4994 @@ "path": "scripts/__pycache__/verify_video_retarget.cpython-310.pyc", "reason": "runtime/data/cache directory" }, + { + "path": "docs/20260915_171525/color.mp4", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000000.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000001.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000002.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000003.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000004.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000005.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000006.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000007.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000008.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000009.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000010.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000011.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000012.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000013.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000014.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000015.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000016.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000017.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000018.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000019.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000020.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000021.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000022.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000023.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000024.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000025.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000026.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000027.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000028.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000029.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000030.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000031.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000032.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000033.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000034.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000035.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000036.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000037.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000038.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000039.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000040.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000041.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000042.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000043.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000044.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000045.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000046.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000047.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000048.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000049.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000050.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000051.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000052.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000053.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000054.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000055.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000056.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000057.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000058.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000059.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000060.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000061.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000062.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000063.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000064.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000065.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000066.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000067.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000068.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000069.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000070.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000071.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000072.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000073.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000074.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000075.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000076.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000077.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000078.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000079.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000080.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000081.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000082.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000083.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000084.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000085.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000086.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000087.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000088.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000089.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000090.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000091.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000092.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000093.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000094.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000095.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000096.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000097.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000098.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000099.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000100.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000101.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000102.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000103.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000104.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000105.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000106.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000107.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000108.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000109.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000110.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000111.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000112.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000113.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000114.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000115.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000116.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000117.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000118.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000119.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000120.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000121.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000122.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000123.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000124.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000125.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000126.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000127.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000128.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000129.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000130.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000131.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000132.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000133.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000134.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000135.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000136.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000137.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000138.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000139.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000140.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000141.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000142.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000143.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000144.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000145.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000146.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000147.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000148.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000149.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000150.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000151.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000152.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000153.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000154.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000155.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000156.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000157.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000158.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000159.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000160.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000161.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000162.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000163.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000164.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000165.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000166.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000167.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000168.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000169.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000170.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000171.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000172.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000173.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000174.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000175.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000176.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000177.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000178.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000179.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000180.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000181.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000182.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000183.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000184.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000185.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000186.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000187.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000188.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000189.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000190.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000191.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000192.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000193.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000194.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000195.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000196.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000197.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000198.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000199.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000200.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000201.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000202.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000203.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000204.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000205.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000206.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000207.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000208.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000209.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000210.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000211.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000212.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000213.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000214.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000215.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000216.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000217.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000218.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000219.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000220.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000221.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000222.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000223.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000224.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000225.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000226.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000227.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000228.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000229.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000230.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000231.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000232.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000233.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000234.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000235.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000236.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000237.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000238.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000239.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000240.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000241.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000242.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000243.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000244.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000245.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000246.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000247.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000248.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000249.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000250.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000251.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000252.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000253.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000254.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000255.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000256.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000257.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000258.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000259.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000260.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000261.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000262.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000263.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000264.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000265.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000266.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000267.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000268.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000269.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000270.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000271.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000272.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000273.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000274.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000275.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000276.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000277.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000278.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000279.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000280.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000281.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000282.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000283.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000284.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000285.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000286.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000287.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000288.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000289.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000290.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000291.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000292.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000293.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000294.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000295.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000296.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000297.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000298.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000299.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000300.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000301.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000302.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000303.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000304.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000305.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000306.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000307.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000308.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000309.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000310.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000311.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000312.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000313.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000314.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000315.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000316.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000317.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000318.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000319.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000320.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000321.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000322.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000323.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000324.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000325.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000326.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000327.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000328.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000329.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000330.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000331.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000332.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000333.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000334.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000335.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000336.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000337.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000338.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000339.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000340.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000341.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000342.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000343.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000344.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000345.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000346.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000347.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000348.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000349.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000350.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260915_171525/depth/000351.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/color.mp4", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000000.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000001.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000002.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000003.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000004.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000005.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000006.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000007.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000008.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000009.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000010.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000011.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000012.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000013.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000014.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000015.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000016.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000017.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000018.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000019.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000020.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000021.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000022.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000023.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000024.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000025.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000026.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000027.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000028.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000029.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000030.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000031.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000032.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000033.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000034.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000035.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000036.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000037.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000038.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000039.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000040.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000041.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000042.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000043.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000044.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000045.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000046.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000047.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000048.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000049.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000050.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000051.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000052.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000053.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000054.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000055.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000056.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000057.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000058.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000059.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000060.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000061.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000062.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000063.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000064.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000065.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000066.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000067.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000068.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000069.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000070.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000071.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000072.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000073.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000074.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000075.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000076.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000077.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000078.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000079.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000080.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000081.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000082.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000083.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000084.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000085.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000086.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000087.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000088.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000089.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000090.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000091.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000092.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000093.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000094.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000095.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000096.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000097.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000098.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000099.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000100.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000101.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000102.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000103.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000104.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000105.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000106.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000107.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000108.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000109.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000110.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000111.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000112.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000113.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000114.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000115.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000116.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000117.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000118.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000119.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000120.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000121.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000122.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000123.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000124.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000125.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000126.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000127.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000128.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000129.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000130.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000131.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000132.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000133.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000134.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000135.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000136.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000137.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000138.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000139.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000140.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000141.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000142.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000143.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000144.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000145.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000146.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000147.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000148.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000149.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000150.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000151.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000152.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000153.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000154.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000155.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000156.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000157.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000158.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000159.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000160.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000161.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000162.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000163.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000164.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000165.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000166.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000167.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000168.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000169.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000170.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000171.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000172.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000173.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000174.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000175.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000176.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000177.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000178.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000179.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000180.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000181.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000182.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000183.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000184.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000185.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000186.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000187.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000188.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000189.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000190.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000191.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000192.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000193.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000194.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000195.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000196.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000197.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000198.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000199.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000200.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000201.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000202.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000203.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000204.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000205.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000206.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000207.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000208.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000209.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000210.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000211.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000212.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000213.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000214.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000215.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000216.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000217.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000218.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000219.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000220.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000221.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000222.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000223.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000224.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000225.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000226.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000227.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000228.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000229.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000230.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000231.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000232.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000233.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000234.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000235.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000236.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000237.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000238.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000239.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000240.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000241.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000242.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000243.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000244.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000245.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000246.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000247.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000248.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000249.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000250.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000251.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000252.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000253.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000254.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000255.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000256.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000257.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000258.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000259.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000260.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000261.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000262.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000263.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000264.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000265.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000266.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000267.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000268.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000269.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000270.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000271.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000272.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000273.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000274.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000275.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000276.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000277.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000278.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000279.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000280.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000281.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000282.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000283.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000284.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000285.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000286.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000287.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000288.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000289.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000290.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000291.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000292.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000293.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000294.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000295.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000296.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000297.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000298.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000299.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000300.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000301.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000302.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000303.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000304.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000305.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000306.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000307.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000308.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000309.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000310.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000311.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000312.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000313.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000314.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000315.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000316.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000317.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000318.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000319.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000320.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000321.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000322.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000323.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000324.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000325.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000326.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000327.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000328.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000329.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000330.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000331.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000332.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000333.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000334.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000335.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000336.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000337.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000338.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000339.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000340.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000341.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000342.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000343.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000344.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000345.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000346.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000347.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000348.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000349.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000350.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000351.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000352.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000353.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000354.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000355.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000356.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000357.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000358.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000359.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000360.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000361.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000362.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000363.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000364.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000365.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000366.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000367.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000368.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000369.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000370.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000371.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000372.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000373.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000374.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000375.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000376.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000377.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000378.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000379.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000380.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000381.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000382.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000383.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000384.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000385.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000386.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000387.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000388.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000389.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000390.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000391.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000392.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000393.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000394.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000395.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000396.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000397.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000398.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000399.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000400.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000401.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000402.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000403.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000404.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000405.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000406.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000407.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000408.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000409.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000410.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000411.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000412.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000413.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000414.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000415.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000416.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000417.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000418.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000419.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000420.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000421.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000422.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000423.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000424.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000425.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000426.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000427.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000428.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000429.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000430.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000431.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000432.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000433.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000434.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000435.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000436.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000437.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000438.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000439.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000440.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000441.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000442.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000443.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000444.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000445.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000446.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000447.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000448.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000449.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000450.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000451.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000452.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000453.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000454.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000455.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000456.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000457.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000458.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000459.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000460.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000461.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000462.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000463.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000464.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000465.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000466.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000467.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000468.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000469.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000470.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000471.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000472.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000473.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000474.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000475.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000476.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000477.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000478.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000479.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000480.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000481.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000482.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000483.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000484.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000485.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000486.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000487.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000488.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000489.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000490.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000491.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000492.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000493.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000494.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000495.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000496.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000497.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000498.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000499.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000500.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000501.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000502.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000503.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000504.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000505.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000506.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000507.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000508.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000509.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000510.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000511.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000512.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000513.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000514.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000515.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000516.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000517.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000518.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000519.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000520.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000521.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000522.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000523.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000524.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000525.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000526.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000527.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000528.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000529.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000530.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000531.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000532.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000533.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000534.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000535.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000536.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000537.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000538.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000539.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000540.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000541.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000542.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000543.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000544.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000545.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000546.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000547.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000548.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000549.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000550.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000551.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000552.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000553.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000554.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000555.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000556.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000557.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000558.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000559.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000560.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000561.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000562.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000563.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000564.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000565.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000566.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000567.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000568.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000569.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000570.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000571.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000572.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000573.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000574.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000575.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000576.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000577.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000578.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000579.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000580.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000581.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000582.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000583.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000584.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000585.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000586.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000587.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000588.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000589.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000590.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000591.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000592.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000593.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000594.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000595.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000596.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000597.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000598.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000599.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000600.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000601.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000602.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000603.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000604.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000605.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000606.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000607.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000608.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000609.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000610.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000611.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000612.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000613.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000614.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000615.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000616.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000617.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000618.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000619.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000620.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000621.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000622.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000623.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000624.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000625.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000626.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000627.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000628.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000629.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000630.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000631.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000632.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000633.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000634.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000635.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000636.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000637.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000638.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000639.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000640.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000641.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000642.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000643.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000644.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000645.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000646.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000647.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000648.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000649.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000650.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000651.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000652.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000653.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000654.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000655.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000656.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000657.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000658.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000659.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000660.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000661.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000662.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000663.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000664.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000665.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000666.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000667.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000668.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000669.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000670.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000671.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000672.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000673.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000674.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000675.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000676.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000677.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000678.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000679.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000680.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000681.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000682.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000683.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000684.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000685.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000686.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000687.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000688.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000689.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000690.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000691.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000692.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000693.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000694.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000695.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000696.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000697.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000698.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000699.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000700.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000701.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000702.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000703.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000704.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000705.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000706.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000707.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000708.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000709.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000710.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000711.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000712.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000713.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000714.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000715.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000716.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000717.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000718.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000719.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000720.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000721.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000722.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000723.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000724.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000725.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000726.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000727.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000728.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000729.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000730.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000731.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000732.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000733.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000734.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000735.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000736.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000737.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000738.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000739.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000740.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000741.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000742.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000743.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000744.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000745.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000746.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000747.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000748.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000749.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000750.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000751.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000752.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000753.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000754.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000755.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000756.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000757.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000758.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000759.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000760.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000761.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000762.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000763.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000764.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000765.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000766.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000767.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000768.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000769.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000770.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000771.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000772.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000773.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000774.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000775.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000776.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000777.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000778.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000779.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000780.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000781.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000782.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000783.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000784.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000785.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000786.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000787.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000788.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000789.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000790.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000791.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000792.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000793.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000794.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000795.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000796.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000797.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000798.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000799.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000800.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000801.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000802.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000803.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000804.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000805.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000806.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000807.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000808.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000809.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000810.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000811.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000812.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000813.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000814.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000815.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000816.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000817.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000818.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000819.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000820.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000821.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000822.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000823.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000824.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000825.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000826.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000827.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000828.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000829.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000830.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000831.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000832.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000833.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000834.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000835.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000836.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000837.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000838.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000839.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000840.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000841.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000842.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000843.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000844.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000845.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000846.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000847.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000848.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000849.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000850.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000851.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000852.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000853.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000854.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000855.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000856.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000857.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000858.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000859.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000860.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000861.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000862.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000863.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000864.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000865.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000866.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000867.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000868.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000869.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000870.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000871.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000872.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000873.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000874.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000875.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000876.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000877.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000878.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000879.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000880.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000881.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000882.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000883.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000884.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000885.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000886.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000887.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000888.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000889.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000890.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000891.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "docs/20260916_104026/depth/000892.png", + "reason": "non-source file; restore separately when needed" + }, { "path": "model/__pycache__/__init__.cpython-311.pyc", "reason": "runtime/data/cache directory" @@ -5759,5 +10843,317 @@ { "path": "third_party/spider/spider/assets/robots/xhand/assets/right_hand_thumb_rota_tip.STL", "reason": "non-source file; restore separately when needed" + }, + { + "path": "third_party/FoundationPose/.gitignore", + "reason": "vendored snapshot: omit nested Git filters and metadata" + }, + { + "path": "third_party/FoundationPose/assets/ar_maze_c.mp4", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "third_party/FoundationPose/assets/bop.jpg", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "third_party/FoundationPose/assets/cvpr_review.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "third_party/FoundationPose/assets/demo.jpg", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "third_party/FoundationPose/assets/demo_driller.jpg", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "third_party/FoundationPose/assets/intro.jpg", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "third_party/FoundationPose/assets/robot_mustard.mp4", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "third_party/FoundationPose/assets/train_data_vis.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "third_party/FoundationPose/assets/ycbv_tracking_c.mp4", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "third_party/FoundationPose/docker/dockerfile", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "third_party/nvdiffrast/csrc/common/cudaraster/impl/BinRaster.inl", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "third_party/nvdiffrast/csrc/common/cudaraster/impl/CoarseRaster.inl", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "third_party/nvdiffrast/csrc/common/cudaraster/impl/FineRaster.inl", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "third_party/nvdiffrast/csrc/common/cudaraster/impl/TriangleSetup.inl", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "third_party/nvdiffrast/csrc/common/cudaraster/impl/Util.inl", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "third_party/nvdiffrast/csrc/torch/torch_common.inl", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "third_party/nvdiffrast/docs/img/cube.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "third_party/nvdiffrast/docs/img/earth.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "third_party/nvdiffrast/docs/img/envphong.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "third_party/nvdiffrast/docs/img/logo.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "third_party/nvdiffrast/docs/img/pipe_cube.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "third_party/nvdiffrast/docs/img/pipe_earth.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "third_party/nvdiffrast/docs/img/pipe_envphong.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "third_party/nvdiffrast/docs/img/pose.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "third_party/nvdiffrast/docs/img/spot_aa.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "third_party/nvdiffrast/docs/img/spot_crop1.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "third_party/nvdiffrast/docs/img/spot_crop2.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "third_party/nvdiffrast/docs/img/spot_diff1.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "third_party/nvdiffrast/docs/img/spot_diff2.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "third_party/nvdiffrast/docs/img/spot_peel1.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "third_party/nvdiffrast/docs/img/spot_peel2.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "third_party/nvdiffrast/docs/img/spot_st.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "third_party/nvdiffrast/docs/img/spot_tex.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "third_party/nvdiffrast/docs/img/spot_texture.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "third_party/nvdiffrast/docs/img/spot_texw.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "third_party/nvdiffrast/docs/img/spot_tri.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "third_party/nvdiffrast/docs/img/spot_uv.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "third_party/nvdiffrast/docs/img/teaser.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "third_party/nvdiffrast/docs/img/teaser1.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "third_party/nvdiffrast/docs/img/teaser2.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "third_party/nvdiffrast/docs/img/teaser3.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "third_party/nvdiffrast/docs/img/teaser4.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "third_party/nvdiffrast/docs/img/teaser5.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "third_party/nvdiffrast/docs/img/thumb.jpg", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "third_party/nvdiffrast/docs/img/tri.png", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "third_party/nvdiffrast/samples/data/cube_c.npz", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "third_party/nvdiffrast/samples/data/cube_d.npz", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "third_party/nvdiffrast/samples/data/cube_p.npz", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "third_party/nvdiffrast/samples/data/earth.npz", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "third_party/nvdiffrast/samples/data/envphong.npz", + "reason": "non-source file; restore separately when needed" + }, + { + "path": "third_party/nvdiffrast/build/lib.linux-x86_64-cpython-311/_nvdiffrast_c.cpython-311-x86_64-linux-gnu.so", + "reason": "build artifact or partial download" + }, + { + "path": "third_party/nvdiffrast/build/lib.linux-x86_64-cpython-311/nvdiffrast/__init__.py", + "reason": "build artifact or partial download" + }, + { + "path": "third_party/nvdiffrast/build/lib.linux-x86_64-cpython-311/nvdiffrast/torch/__init__.py", + "reason": "build artifact or partial download" + }, + { + "path": "third_party/nvdiffrast/build/lib.linux-x86_64-cpython-311/nvdiffrast/torch/ops.py", + "reason": "build artifact or partial download" + }, + { + "path": "third_party/nvdiffrast/build/temp.linux-x86_64-cpython-311/.ninja_deps", + "reason": "build artifact or partial download" + }, + { + "path": "third_party/nvdiffrast/build/temp.linux-x86_64-cpython-311/.ninja_log", + "reason": "build artifact or partial download" + }, + { + "path": "third_party/nvdiffrast/build/temp.linux-x86_64-cpython-311/build.ninja", + "reason": "build artifact or partial download" + }, + { + "path": "third_party/nvdiffrast/build/temp.linux-x86_64-cpython-311/csrc/common/antialias.o", + "reason": "build artifact or partial download" + }, + { + "path": "third_party/nvdiffrast/build/temp.linux-x86_64-cpython-311/csrc/common/common.o", + "reason": "build artifact or partial download" + }, + { + "path": "third_party/nvdiffrast/build/temp.linux-x86_64-cpython-311/csrc/common/cudaraster/impl/Buffer.o", + "reason": "build artifact or partial download" + }, + { + "path": "third_party/nvdiffrast/build/temp.linux-x86_64-cpython-311/csrc/common/cudaraster/impl/CudaRaster.o", + "reason": "build artifact or partial download" + }, + { + "path": "third_party/nvdiffrast/build/temp.linux-x86_64-cpython-311/csrc/common/cudaraster/impl/RasterImpl.o", + "reason": "build artifact or partial download" + }, + { + "path": "third_party/nvdiffrast/build/temp.linux-x86_64-cpython-311/csrc/common/cudaraster/impl/RasterImpl_kernel.o", + "reason": "build artifact or partial download" + }, + { + "path": "third_party/nvdiffrast/build/temp.linux-x86_64-cpython-311/csrc/common/interpolate.o", + "reason": "build artifact or partial download" + }, + { + "path": "third_party/nvdiffrast/build/temp.linux-x86_64-cpython-311/csrc/common/rasterize.o", + "reason": "build artifact or partial download" + }, + { + "path": "third_party/nvdiffrast/build/temp.linux-x86_64-cpython-311/csrc/common/texture.o", + "reason": "build artifact or partial download" + }, + { + "path": "third_party/nvdiffrast/build/temp.linux-x86_64-cpython-311/csrc/common/texture_kernel.o", + "reason": "build artifact or partial download" + }, + { + "path": "third_party/nvdiffrast/build/temp.linux-x86_64-cpython-311/csrc/torch/torch_antialias.o", + "reason": "build artifact or partial download" + }, + { + "path": "third_party/nvdiffrast/build/temp.linux-x86_64-cpython-311/csrc/torch/torch_bindings.o", + "reason": "build artifact or partial download" + }, + { + "path": "third_party/nvdiffrast/build/temp.linux-x86_64-cpython-311/csrc/torch/torch_interpolate.o", + "reason": "build artifact or partial download" + }, + { + "path": "third_party/nvdiffrast/build/temp.linux-x86_64-cpython-311/csrc/torch/torch_rasterize.o", + "reason": "build artifact or partial download" + }, + { + "path": "third_party/nvdiffrast/build/temp.linux-x86_64-cpython-311/csrc/torch/torch_texture.o", + "reason": "build artifact or partial download" + }, + { + "path": "third_party/nvdiffrast/nvdiffrast.egg-info/PKG-INFO", + "reason": "build artifact or partial download" + }, + { + "path": "third_party/nvdiffrast/nvdiffrast.egg-info/SOURCES.txt", + "reason": "build artifact or partial download" + }, + { + "path": "third_party/nvdiffrast/nvdiffrast.egg-info/dependency_links.txt", + "reason": "build artifact or partial download" + }, + { + "path": "third_party/nvdiffrast/nvdiffrast.egg-info/requires.txt", + "reason": "build artifact or partial download" + }, + { + "path": "third_party/nvdiffrast/nvdiffrast.egg-info/top_level.txt", + "reason": "build artifact or partial download" } ] diff --git a/EXTERNAL_ASSETS.json b/EXTERNAL_ASSETS.json index 3fff277..3503693 100644 --- a/EXTERNAL_ASSETS.json +++ b/EXTERNAL_ASSETS.json @@ -3,6 +3,126 @@ "path": "third_party/l20_assets", "bundled": false, "files": [ + { + "path": "third_party/l20_assets/L20/LEFT/g20_left_G20.json", + "bytes": 111699, + "sha256": "28fed01a4d9bac3ae5ddbc1a3a158382afb9cd334774b7ad554e90ed8e8145a3" + }, + { + "path": "third_party/l20_assets/L20/LEFT/linkerhand_g20_left.urdf", + "bytes": 28677, + "sha256": "b8ef22e436ab311fb61f87091b72ae717821092471d628d0ea9475b5088daa5d" + }, + { + "path": "third_party/l20_assets/L20/LEFT/meshes/hand_base_link.STL", + "bytes": 3504384, + "sha256": "f5fa64265908b92a132e18fff27dc212ec21ba4872806386846374d5c91fc70a" + }, + { + "path": "third_party/l20_assets/L20/LEFT/meshes/index_distal.STL", + "bytes": 993184, + "sha256": "130639135efbb5cd2773c18b481fcea45afe3bfd5dc8279a28a18a6b518779db" + }, + { + "path": "third_party/l20_assets/L20/LEFT/meshes/index_metacarpals.STL", + "bytes": 153484, + "sha256": "60bc3fc552cbdc4a53fbde3e11a06fc8b0b52a0ee6ea8b730b481a6eb54cfe92" + }, + { + "path": "third_party/l20_assets/L20/LEFT/meshes/index_middle.STL", + "bytes": 732584, + "sha256": "b3e46d945996e52bf0758b80f7aa1c2b99bca503b4c5b3c350e6850f78b05152" + }, + { + "path": "third_party/l20_assets/L20/LEFT/meshes/index_proximal.STL", + "bytes": 881084, + "sha256": "5a561d159861170e0bc91d6a4ecb634611bcc497bb2e60440d829e5ae0b9b7f7" + }, + { + "path": "third_party/l20_assets/L20/LEFT/meshes/middle_distal.STL", + "bytes": 993184, + "sha256": "a6143d000a38c7b60231b55b9e209161c7726a0410f5a7e5c48c0f41f067a006" + }, + { + "path": "third_party/l20_assets/L20/LEFT/meshes/middle_metacarpals.STL", + "bytes": 153484, + "sha256": "592ce620ebba1e75ce109cf70654dc8dc2d9bb41315b6dfc902b3ed3c703f43a" + }, + { + "path": "third_party/l20_assets/L20/LEFT/meshes/middle_middle.STL", + "bytes": 732584, + "sha256": "09b1b9a5bf9017d86dc543a214d263acc32c2ed4c3a474116ccdb8ab163cce89" + }, + { + "path": "third_party/l20_assets/L20/LEFT/meshes/middle_proximal.STL", + "bytes": 881084, + "sha256": "115d5c2027a656d6765339a477effc251a6fb9cd26dfc56474e6b63fac64aa43" + }, + { + "path": "third_party/l20_assets/L20/LEFT/meshes/pinky_distal.STL", + "bytes": 993184, + "sha256": "2d31ac67d0c2573758f0da2de4c3b773494a262845cc63659f2816bfd1fda10a" + }, + { + "path": "third_party/l20_assets/L20/LEFT/meshes/pinky_metacarpals.STL", + "bytes": 153484, + "sha256": "95a7614a7495032fdec6d5bf39df04a5fec85fbe1032ea8ffad5e09e59fe287a" + }, + { + "path": "third_party/l20_assets/L20/LEFT/meshes/pinky_middle.STL", + "bytes": 732584, + "sha256": "5a67a4840294f79a64b143e1cba9131811342df8c735f817708dd106699c6480" + }, + { + "path": "third_party/l20_assets/L20/LEFT/meshes/pinky_proximal.STL", + "bytes": 881084, + "sha256": "5fce82ec63ecba86c656edbba8b17c5b44d51aa78df0ddb9657a465f89872206" + }, + { + "path": "third_party/l20_assets/L20/LEFT/meshes/ring_distal.STL", + "bytes": 993184, + "sha256": "c82b0efd00e191a019927cf5590135f8b4d9b2c9228abfb68b12f3e0257c42aa" + }, + { + "path": "third_party/l20_assets/L20/LEFT/meshes/ring_metacarpals.STL", + "bytes": 153484, + "sha256": "c90a028bf7a5da221a92d886a65cde79cf5641ff82eccd65bd6b982e17df912d" + }, + { + "path": "third_party/l20_assets/L20/LEFT/meshes/ring_middle.STL", + "bytes": 732584, + "sha256": "191ddee09524ed6f7ebd447cf30fb85f894969ba161fc8c0059b6e97f70319f1" + }, + { + "path": "third_party/l20_assets/L20/LEFT/meshes/ring_proximal.STL", + "bytes": 881084, + "sha256": "6a49d3ee8360028022c2ba7435ffb35fb19ffaaa3c73ddba5da16b106d86857c" + }, + { + "path": "third_party/l20_assets/L20/LEFT/meshes/thumb_distal.STL", + "bytes": 1115884, + "sha256": "0812eea8d86179c70f00070e3142595e1b5cea231a5c606c78744041d7acd9ba" + }, + { + "path": "third_party/l20_assets/L20/LEFT/meshes/thumb_metacarpals.STL", + "bytes": 1074384, + "sha256": "7bcb58b906bcff02d8de04f0ac90dbe6bf318cdbda52988787808b50f491d8f7" + }, + { + "path": "third_party/l20_assets/L20/LEFT/meshes/thumb_metacarpals_base1.STL", + "bytes": 778384, + "sha256": "fa50f9021ae593bb37b95472a999215574ae56f63fdc5ff6edfd74b298826059" + }, + { + "path": "third_party/l20_assets/L20/LEFT/meshes/thumb_metacarpals_base2.STL", + "bytes": 227384, + "sha256": "316ab10d2dae65040a78f85182635ea3b7b1f57859ef0b568ccee58d88d58129" + }, + { + "path": "third_party/l20_assets/L20/LEFT/meshes/thumb_proximal.STL", + "bytes": 689584, + "sha256": "e41ea39ce8cb6673fc33b3e33e0fe2af9e82158006fccf6355ee3e49bfc5db8b" + }, { "path": "third_party/l20_assets/L20/RIGHT/g20_right_G20.json", "bytes": 111584, @@ -246,5 +366,1812 @@ "sha256": "d28876cc3b674323aa7d8a628a4240cda2757c61263b05e86a66280035a2689d" } ] + }, + { + "path": "third_party/FoundationPose/weights", + "bundled": false, + "files": [ + { + "path": "third_party/FoundationPose/weights/2023-10-28-18-33-37/config.yml", + "bytes": 708, + "sha256": "28a6ba94a33230ee5fc3c51939486281578b0972542bd9e38ca6123e75605686" + }, + { + "path": "third_party/FoundationPose/weights/2023-10-28-18-33-37/model_best.pth", + "bytes": 68220109, + "sha256": "774700586ddc435d408fc01c9809c43e151232936369dfbea0f0f964ba471d60" + }, + { + "path": "third_party/FoundationPose/weights/2024-01-11-20-02-45/config.yml", + "bytes": 778, + "sha256": "a79db4de3b95885dd5ae86833b37b8698a75dad81e87d1086cd50b2fcd8dda3f" + }, + { + "path": "third_party/FoundationPose/weights/2024-01-11-20-02-45/model_best.pth", + "bytes": 190229389, + "sha256": "81924d384bf5c26c646ee4783104982ae3d1e049c181c36641b6a7aeae494c26" + }, + { + "path": "third_party/FoundationPose/weights/download_manifest.json", + "bytes": 1330, + "sha256": "d2fcde6f355b0d5cb98439fedb88e300f15169c341cad89cf8ef27df0cc8ef8f" + } + ] + }, + { + "path": "docs/20260915_171525", + "bundled": false, + "files": [ + { + "path": "docs/20260915_171525/color.mp4", + "bytes": 3156391, + "sha256": "193a2a2d2a21cb3abbd01c6d6f93b597d2cb501a64d1ef11a91c193bdf3fc574" + }, + { + "path": "docs/20260915_171525/depth/000000.png", + "bytes": 331266, + "sha256": "e5f9706b5468bbd36488b7e31165a2535db6f356de48bff710320c4dc0277d61" + }, + { + "path": "docs/20260915_171525/depth/000001.png", + "bytes": 326373, + "sha256": "66d6551cc0a0dd460870a9d6579d9832fbbd94228495d4c03fd15803774ced18" + }, + { + "path": "docs/20260915_171525/depth/000002.png", + "bytes": 333345, + "sha256": "06dfb4a84d52347a0dddbaa1c92c6e620c5b2bf45fc81abe44ec5246a0005cde" + }, + { + "path": "docs/20260915_171525/depth/000003.png", + "bytes": 326474, + "sha256": "701266a4316b77e66f979fa36711d4a1a842474bd4dcad9218232085de171b63" + }, + { + "path": "docs/20260915_171525/depth/000004.png", + "bytes": 329915, + "sha256": "a3083806b610b69cad602b7f6a28302acfc2f0b123afcd076fb4f3295cdfcd2f" + }, + { + "path": "docs/20260915_171525/depth/000005.png", + "bytes": 339924, + "sha256": "601cd0d8e35b0565e5bb2066d141d4327b82e2825a590b8d3c333314b8f984bf" + }, + { + "path": "docs/20260915_171525/depth/000006.png", + "bytes": 337000, + "sha256": "65e68e409915199b8f119f25b3fc69ba8d7c3abae022f2f04b312d9bf5152add" + }, + { + "path": "docs/20260915_171525/depth/000007.png", + "bytes": 329906, + "sha256": "39e5eb6755cb2c7509a2f4d07e4585b4bf10a0082316fce3645e8014c67352d8" + }, + { + "path": "docs/20260915_171525/depth/000008.png", + "bytes": 331489, + "sha256": "5d584c27357fab708a3359729df3a2060d4ad1d3a348d3125bff246265381b52" + }, + { + "path": "docs/20260915_171525/depth/000009.png", + "bytes": 326222, + "sha256": "a61a18b86e9a7cb2e7528282a40691b7b9c1d4f2fd067fdf85aa38ed65e8fb61" + }, + { + "path": "docs/20260915_171525/depth/000010.png", + "bytes": 335454, + "sha256": "be7955d81fdde9221c5f188629350ea315fb22857da98b9aa48eabac235a293f" + }, + { + "path": "docs/20260915_171525/depth/000011.png", + "bytes": 326890, + "sha256": "5f49e2edb35cfc5676b1c1f056f550e20d765c6efa2c4a040ff3b8aa9dff401b" + }, + { + "path": "docs/20260915_171525/depth/000012.png", + "bytes": 330689, + "sha256": "f0ab4e8b2344ee982a1557ad392685c3053ee8ef12d32004de909cfe883bf46c" + }, + { + "path": "docs/20260915_171525/depth/000013.png", + "bytes": 327180, + "sha256": "77933d558e8c8a27f19a48792b980fbd631f5a0977a2fc0fccf9d28d007ab0c9" + }, + { + "path": "docs/20260915_171525/depth/000014.png", + "bytes": 331345, + "sha256": "967cfdd6d33b378eb07ccff9c614c0a082790fef9037c4bec60a0e446c92708a" + }, + { + "path": "docs/20260915_171525/depth/000015.png", + "bytes": 327139, + "sha256": "a4e9f571254d564480ab4a2a2d6012c1eda074bb8d11f0246133420d1591e0f5" + }, + { + "path": "docs/20260915_171525/depth/000016.png", + "bytes": 330244, + "sha256": "dfaf6068e58fe15a2ecc185d0eb53cf0f21f988dd16e73e0dbd5c126dbfd8ae5" + }, + { + "path": "docs/20260915_171525/depth/000017.png", + "bytes": 331516, + "sha256": "92f63819bda39da2b612465fb443b8c765469a42023706cecdceeec2d2dc963b" + }, + { + "path": "docs/20260915_171525/depth/000018.png", + "bytes": 333072, + "sha256": "6579777cc31d05264dbdba69bc1f0a105a37ba106b7135ffb38c905e8b5d72ed" + }, + { + "path": "docs/20260915_171525/depth/000019.png", + "bytes": 326299, + "sha256": "958dc2da6603e9c05bde446c421794e4c62bdd9f26b07c0c45bd27512713f551" + }, + { + "path": "docs/20260915_171525/depth/000020.png", + "bytes": 327184, + "sha256": "44b163c6722b435145c15b451f5456b59da84cf4419d715314e6a719fe661f4c" + }, + { + "path": "docs/20260915_171525/depth/000021.png", + "bytes": 326112, + "sha256": "641522f5934f6a3a03ff5e36ae951d5e94acc1ffdc4d96c67ef46fb91212e404" + }, + { + "path": "docs/20260915_171525/depth/000022.png", + "bytes": 329445, + "sha256": "4183534310ec4b18bcae5ca7d9ffad74b159d2b62b0e4a552fcad22ffc5f3970" + }, + { + "path": "docs/20260915_171525/depth/000023.png", + "bytes": 338203, + "sha256": "59a622da9103d09249e8a9efb3be59ec80e446cc875239774d4e82d971f7c31d" + }, + { + "path": "docs/20260915_171525/depth/000024.png", + "bytes": 333433, + "sha256": "56ae6fa1362290f76af19fc11e5071b622ec69f43dd84026c663ad2758681ba2" + }, + { + "path": "docs/20260915_171525/depth/000025.png", + "bytes": 331027, + "sha256": "743994ac05a57fbba2bea51bdd83ca2a0e43b68fdd641b15d3c516d29048dd56" + }, + { + "path": "docs/20260915_171525/depth/000026.png", + "bytes": 341677, + "sha256": "b28b5de138f22716b05ee9f92d2d0faa68ba616a65c528c20d76e555d5813d49" + }, + { + "path": "docs/20260915_171525/depth/000027.png", + "bytes": 332620, + "sha256": "f8c180c69ac7e8478a086031b3171cba17e3cfba4b4c5467e2e054b6c0000ad7" + }, + { + "path": "docs/20260915_171525/depth/000028.png", + "bytes": 331938, + "sha256": "f89d8c0653640e426b0a772784250574682453accfc0f1416283248fba0ddca0" + }, + { + "path": "docs/20260915_171525/depth/000029.png", + "bytes": 335152, + "sha256": "40ab0356f2ca071177d2d6f907b1f4885fa0c5e8131821d84c63cd3730573441" + }, + { + "path": "docs/20260915_171525/depth/000030.png", + "bytes": 331486, + "sha256": "dcbfda275e1992969902513d5948a3a9afb9e36040302516b91071d5a6dfe1f1" + }, + { + "path": "docs/20260915_171525/depth/000031.png", + "bytes": 341661, + "sha256": "acb6d92b24fd6c13ba15e56ce18f5d391ff01b51506b0f932db268235d50c522" + }, + { + "path": "docs/20260915_171525/depth/000032.png", + "bytes": 334524, + "sha256": "fa8a9bbdd33ba63c4f284aeed4bf2c656530a976d800fc66dd61ff82ff9dd005" + }, + { + "path": "docs/20260915_171525/depth/000033.png", + "bytes": 325228, + "sha256": "52d4326fd1d0176f9de524b5389b1e8af291390e391d9d359db2c6ed9651b2e2" + }, + { + "path": "docs/20260915_171525/depth/000034.png", + "bytes": 332585, + "sha256": "9c01ea73ad178ce0fefb3c26502c8a1924bb2895bc7b82130d5a66c36a53fe30" + }, + { + "path": "docs/20260915_171525/depth/000035.png", + "bytes": 328117, + "sha256": "df41b9c352fdba396ce141a6d84392fc0095d0b951e91ca8fe9acdda88d4d307" + }, + { + "path": "docs/20260915_171525/depth/000036.png", + "bytes": 332159, + "sha256": "5ec1425eee81d9355d2ad5c4dfc538a5571e281bf5e263ad102e989aefb21d40" + }, + { + "path": "docs/20260915_171525/depth/000037.png", + "bytes": 336691, + "sha256": "b43ceca8caa8cb8cf5d45936f8c4ad6a5919199476354dce210ae68c24c866ce" + }, + { + "path": "docs/20260915_171525/depth/000038.png", + "bytes": 342192, + "sha256": "d50f3eb7a5d33056541a76afc11a4b611d99d68e2a7465a09b8ab8f581a6856f" + }, + { + "path": "docs/20260915_171525/depth/000039.png", + "bytes": 336540, + "sha256": "3b0653dee7bc22bba9e2067ba2a1e50c274d356b037b223f904cd41dbf19752a" + }, + { + "path": "docs/20260915_171525/depth/000040.png", + "bytes": 335002, + "sha256": "42812c72fcb8fe2385c4584f54e531a6cd582868d44e0d5c21453319603114cf" + }, + { + "path": "docs/20260915_171525/depth/000041.png", + "bytes": 335619, + "sha256": "bd01821bc3f52a23b0dd3339a0ec490d13b3b53b69087ec1238553bf624158fb" + }, + { + "path": "docs/20260915_171525/depth/000042.png", + "bytes": 338929, + "sha256": "a45e14914111ce945e5325217277c04f3f795bf1fa4f10a1383b8b15e7467ecf" + }, + { + "path": "docs/20260915_171525/depth/000043.png", + "bytes": 332470, + "sha256": "2df8a0ed1261456ab75fdf2d4779eb0d4429d63750428ce6f06aa5dd675af5c6" + }, + { + "path": "docs/20260915_171525/depth/000044.png", + "bytes": 335036, + "sha256": "2ce8a8f5048b7dc72a60e25f925defd2c80f1b151e24d0b059595d412fad0481" + }, + { + "path": "docs/20260915_171525/depth/000045.png", + "bytes": 340777, + "sha256": "a9af9592509d54887300ccb558ccccbfb05c114104a4286b2a4f41d6f4f9319e" + }, + { + "path": "docs/20260915_171525/depth/000046.png", + "bytes": 330649, + "sha256": "b76274e0d3b18afeeab773e1503d134e3b6d2b6233af22c337b761495333f8c1" + }, + { + "path": "docs/20260915_171525/depth/000047.png", + "bytes": 343123, + "sha256": "f907bb8432b89af4e63ae12a4f97375d0e810e70c464bdadb63072ed54d89a00" + }, + { + "path": "docs/20260915_171525/depth/000048.png", + "bytes": 342273, + "sha256": "d0f3ef53acb5884a07392a20b87942230da66466a6af3363fa6b8d390531c166" + }, + { + "path": "docs/20260915_171525/depth/000049.png", + "bytes": 335981, + "sha256": "3e0145c3c22d7873472dbcf1e2cc0db6273422717a65b1685449c2fd6a789acf" + }, + { + "path": "docs/20260915_171525/depth/000050.png", + "bytes": 348761, + "sha256": "c674028588ea096dcbc7160c6c8443073e83bb7158118e0a8824ecfcd4fe4629" + }, + { + "path": "docs/20260915_171525/depth/000051.png", + "bytes": 341958, + "sha256": "b56cd177e53fd9c27dbd1261e12925b307e908ab4928ca9becdd54fc757516bd" + }, + { + "path": "docs/20260915_171525/depth/000052.png", + "bytes": 333214, + "sha256": "e19f40f9f51ddbe3b9bb65de251467bb9af51b1c40a0a1113fa8ec596a44b3d6" + }, + { + "path": "docs/20260915_171525/depth/000053.png", + "bytes": 339096, + "sha256": "cbaa0a654811d7472635caeee7b3d3ae2fc67ad27c3ca5b927972090937f96d2" + }, + { + "path": "docs/20260915_171525/depth/000054.png", + "bytes": 326073, + "sha256": "d6ec7afa7a5e9d71bb4d8b39d33b5afd2e6b0cabe33fe64c77a3292a81bc4ca4" + }, + { + "path": "docs/20260915_171525/depth/000055.png", + "bytes": 337254, + "sha256": "1130e9c7a1062c41174bcbf182d79653b84a491b507406fa83d4ae302c399197" + }, + { + "path": "docs/20260915_171525/depth/000056.png", + "bytes": 332765, + "sha256": "989719f9a46e574f713fc39fb4a3033623cbbfad3a24ca2c45a9aa0c446f0888" + }, + { + "path": "docs/20260915_171525/depth/000057.png", + "bytes": 344836, + "sha256": "7cb7de2844bea35e9acadeece51db96aa9f88070b73f185f1d8de0ec6224de3d" + }, + { + "path": "docs/20260915_171525/depth/000058.png", + "bytes": 340157, + "sha256": "029968602be3e470ca97520b083cf7ba91a5f10c90bd0439d09f4eb7edc1dc0c" + }, + { + "path": "docs/20260915_171525/depth/000059.png", + "bytes": 341314, + "sha256": "d8aebda30cd613e27bae9dc364f0b472fbcb1d0950fb1c3a4b23e1cdfa297fda" + }, + { + "path": "docs/20260915_171525/depth/000060.png", + "bytes": 341714, + "sha256": "9e02d40ae890f00e324dc155053af75bd0d144615cc316c663e7c77ba00a8091" + }, + { + "path": "docs/20260915_171525/depth/000061.png", + "bytes": 339118, + "sha256": "f65d8122250cfeca31b80b9c7ca9ed87f66c97b9953a42aca81579a39c48d792" + }, + { + "path": "docs/20260915_171525/depth/000062.png", + "bytes": 342295, + "sha256": "75ce0b9a0f77dbb678df72d666afe39326d9b46b9e54bf77e984261ae2cddd5e" + }, + { + "path": "docs/20260915_171525/depth/000063.png", + "bytes": 337709, + "sha256": "7a9aee11918e46b5a805add841e90b149b8618b3408dfff4aa00bc27eabaeb9f" + }, + { + "path": "docs/20260915_171525/depth/000064.png", + "bytes": 344887, + "sha256": "0425b9dd443e287b8e9c6346dc25e6318cdeb03700c5666baf35ac7793bdb6a1" + }, + { + "path": "docs/20260915_171525/depth/000065.png", + "bytes": 336785, + "sha256": "17e0b3a44dfa09cca0d5e7ff981abe0024df5c77bac3ae08982c8c272e28a4c1" + }, + { + "path": "docs/20260915_171525/depth/000066.png", + "bytes": 336035, + "sha256": "6fcd6ad1b5acd87225f11b95a23773a81e3da096bb1ffefec6cb8c59f31931eb" + }, + { + "path": "docs/20260915_171525/depth/000067.png", + "bytes": 341775, + "sha256": "ee26f48c93d869f7f73dc6ba32f143b24eaaa61f8443d4970352dcca53de33f7" + }, + { + "path": "docs/20260915_171525/depth/000068.png", + "bytes": 348384, + "sha256": "defb668f2c4a20a2667fe796b6482d5ccb8593818f23897bf3ccf1cf19efd933" + }, + { + "path": "docs/20260915_171525/depth/000069.png", + "bytes": 342108, + "sha256": "b5c62c90f22c02e128bfe6ddb29afcfb76402fad5dacb565c8fb4540eac0c27d" + }, + { + "path": "docs/20260915_171525/depth/000070.png", + "bytes": 344598, + "sha256": "d52abf14c1f9ed4815a83cb2a0624934e21d7f47ca79bad5ce5463f598c4e78e" + }, + { + "path": "docs/20260915_171525/depth/000071.png", + "bytes": 342147, + "sha256": "12efc6cf8bd85820da459e535b2f00aba8788567c663a29ec56ea955779b9484" + }, + { + "path": "docs/20260915_171525/depth/000072.png", + "bytes": 344530, + "sha256": "ae75372391ac1eda41fa979dc8eccfea9f75a3a06d3667837df7532516af8140" + }, + { + "path": "docs/20260915_171525/depth/000073.png", + "bytes": 340358, + "sha256": "ff16f7ac7cab5f1adabb53b148aae0746ff11c727804de3b772897c0ac95a38e" + }, + { + "path": "docs/20260915_171525/depth/000074.png", + "bytes": 336446, + "sha256": "6a6cc5eeccf6a32c82ce0adc008c59f8fe54aecfa07004d4a0605833c4351690" + }, + { + "path": "docs/20260915_171525/depth/000075.png", + "bytes": 343955, + "sha256": "cc80b0c331c511c439113a1c6816af062f1385c27832b690d663ff725f0b111b" + }, + { + "path": "docs/20260915_171525/depth/000076.png", + "bytes": 342131, + "sha256": "545a06b3908560deb7ce3ec70147591af66e5c4aa812fe3c08d4f82167d05b41" + }, + { + "path": "docs/20260915_171525/depth/000077.png", + "bytes": 345644, + "sha256": "987b6cd4b4b6daaf2d245a5ef9ca228ec516d896c548c16bf1f6a05a0c4abd59" + }, + { + "path": "docs/20260915_171525/depth/000078.png", + "bytes": 337762, + "sha256": "44a1576e6c9a31f11ef495b347e642e1dd996384f933fcaa93a7bdb6a162dba1" + }, + { + "path": "docs/20260915_171525/depth/000079.png", + "bytes": 344684, + "sha256": "ff4637db4a6854b48fc353612b7d5fefe2187aa81b7bf050caf4f8f9f965dd3f" + }, + { + "path": "docs/20260915_171525/depth/000080.png", + "bytes": 337731, + "sha256": "3d1ce17b64cda19a55776f9cf229e593e8f232a8c28bd00b705e7fc0b15bf87f" + }, + { + "path": "docs/20260915_171525/depth/000081.png", + "bytes": 337952, + "sha256": "149e93fe2233afee85b45044cceabfb8b03c3c108a046af1266d98a874180ed9" + }, + { + "path": "docs/20260915_171525/depth/000082.png", + "bytes": 334106, + "sha256": "3362e7659c127dde4250d415da36f28696c33a89de8061dd33cd096c0becce37" + }, + { + "path": "docs/20260915_171525/depth/000083.png", + "bytes": 338294, + "sha256": "d9403ba0fe1f555ce20d2fcc18b1649327d4d2f0eb04c5fe4ffc781b7152f1ce" + }, + { + "path": "docs/20260915_171525/depth/000084.png", + "bytes": 337404, + "sha256": "f88a4f2f2091deb0e50bdac94fff55c9cc4363bd4aebe7431557746add5a9dff" + }, + { + "path": "docs/20260915_171525/depth/000085.png", + "bytes": 337328, + "sha256": "0538e157d6cf9cbf8c3c3c9721dae6d73614a37e9afdcaab4b8a1f6dfb7c1feb" + }, + { + "path": "docs/20260915_171525/depth/000086.png", + "bytes": 338501, + "sha256": "54fa06a48ca9bdb8a6e594e24fdc78657b0268e9bc2de38f126e1674924fc9a3" + }, + { + "path": "docs/20260915_171525/depth/000087.png", + "bytes": 336120, + "sha256": "426f04ea0b462918dd45685753b4523db5cf0e7682aca0b3a903bee37be329e7" + }, + { + "path": "docs/20260915_171525/depth/000088.png", + "bytes": 338196, + "sha256": "d082cd718230c3376314d0098c3b1a9bf13cfe86cea2ca9384df3e30ab0dc567" + }, + { + "path": "docs/20260915_171525/depth/000089.png", + "bytes": 338891, + "sha256": "143e55a6c11ac32b1d842dc0815d48ff7878016d4d05eaa86134b8cac21034e7" + }, + { + "path": "docs/20260915_171525/depth/000090.png", + "bytes": 333833, + "sha256": "0f86513052829a4ad544d387f32aa5c40e98c875b93e413b9487bf30196cceca" + }, + { + "path": "docs/20260915_171525/depth/000091.png", + "bytes": 334017, + "sha256": "9cfae37f3cc45db512ebb0451b3ee43adc13106479bce38e41e5bd7dc7439fd3" + }, + { + "path": "docs/20260915_171525/depth/000092.png", + "bytes": 329420, + "sha256": "995765b62caeb5418b231fd3201669a8db51c2729c8e8d45c4e1d53b2fec377b" + }, + { + "path": "docs/20260915_171525/depth/000093.png", + "bytes": 333784, + "sha256": "c2570283d7cdd53c80ba5d02dfe76766e85125b7756226ab168d54b95f7ab0f6" + }, + { + "path": "docs/20260915_171525/depth/000094.png", + "bytes": 333203, + "sha256": "4a925673510cdaf292c69b2589ad0417801f969e41c02c1133883f639573bc89" + }, + { + "path": "docs/20260915_171525/depth/000095.png", + "bytes": 340919, + "sha256": "ee4a2a3307b25563c356da6180efd83fee6d76b69fa9a933e924f2a1769f3408" + }, + { + "path": "docs/20260915_171525/depth/000096.png", + "bytes": 337998, + "sha256": "8a3c5c09b3ad41458b144851cd3787bbd15b7a6251f243917d9a92f2f9c6ee76" + }, + { + "path": "docs/20260915_171525/depth/000097.png", + "bytes": 336044, + "sha256": "62fe9788fc3395435c40008d88f8db28da25dc2123e5fe110eff33961fe435a6" + }, + { + "path": "docs/20260915_171525/depth/000098.png", + "bytes": 329311, + "sha256": "9122d0ba6b2c859670f05f68e704cdfbebeb6aba96f1ad9d4b4e9a0285bc2ced" + }, + { + "path": "docs/20260915_171525/depth/000099.png", + "bytes": 334617, + "sha256": "41c794c03350ae4c200a5cac75558bf8598e3f5d2f6072165f7c7559ef684584" + }, + { + "path": "docs/20260915_171525/depth/000100.png", + "bytes": 336937, + "sha256": "b5daa853560b760de82d2e11f7ccda2dbf6dafa1de27d3aad2ed295a9582cf90" + }, + { + "path": "docs/20260915_171525/depth/000101.png", + "bytes": 336975, + "sha256": "b74ae94294247e108c15ba271d121136640d1bf8467c0026d1852e57bd9d1d8a" + }, + { + "path": "docs/20260915_171525/depth/000102.png", + "bytes": 338134, + "sha256": "5ed88198723c0e17897199c0e90e668057c45e833cd0d7af777a64bdb143c051" + }, + { + "path": "docs/20260915_171525/depth/000103.png", + "bytes": 339782, + "sha256": "a465e88ff39a99df0bf6c13352647c7dd29e546e9ad466371777c2b99097331a" + }, + { + "path": "docs/20260915_171525/depth/000104.png", + "bytes": 332104, + "sha256": "9fd61cb4491ee8325aaa2f2b1885e0a61a96fd3eb66b7d4a4b6c5f92882ea487" + }, + { + "path": "docs/20260915_171525/depth/000105.png", + "bytes": 346092, + "sha256": "539ce0a84ebbb334b05268a9f87625decb2d7dee5cc5a97b38db823c1db845ab" + }, + { + "path": "docs/20260915_171525/depth/000106.png", + "bytes": 328037, + "sha256": "5737f89d60c57bdcb5f9340d7d33a7fbfd17a8a8b0a395398e9bd4a7e7bb66a1" + }, + { + "path": "docs/20260915_171525/depth/000107.png", + "bytes": 333564, + "sha256": "4b2681dd8b6091c04a1e8f13a5544a3635032743af40fb440adb8a591332f1e1" + }, + { + "path": "docs/20260915_171525/depth/000108.png", + "bytes": 333119, + "sha256": "5a9cb00d3456c14a05a229e047400f2f6424290d07d68bf503badda8e842e84e" + }, + { + "path": "docs/20260915_171525/depth/000109.png", + "bytes": 336228, + "sha256": "990daafbdd8f8169dbc81b960fc06a5c4718aae31ab55a6cb317981f0f154e8e" + }, + { + "path": "docs/20260915_171525/depth/000110.png", + "bytes": 329866, + "sha256": "105e49da61ea042c233cb62452ca3a1ede830c10e120a7a3d2c446866d07eadf" + }, + { + "path": "docs/20260915_171525/depth/000111.png", + "bytes": 323454, + "sha256": "0b90f71bc87ae35b6f96cc1ba303d054f917be80e0e04f71e4da38ad4c7a146b" + }, + { + "path": "docs/20260915_171525/depth/000112.png", + "bytes": 329176, + "sha256": "b49df8da4c58c4bf3e41c41178c3b98b60adf0ca0d76204033994c03d180572b" + }, + { + "path": "docs/20260915_171525/depth/000113.png", + "bytes": 337220, + "sha256": "8c854d12055460bfcba00407215e6d37f09b2ff86d9073cfcef2fd398b1520eb" + }, + { + "path": "docs/20260915_171525/depth/000114.png", + "bytes": 331532, + "sha256": "4ace3d4b7380bd9682b41c4a2af1715b97c7361a2791fabbacb5e6e956cb78c5" + }, + { + "path": "docs/20260915_171525/depth/000115.png", + "bytes": 330859, + "sha256": "7a74fb71b6c35a98f128a9f28406a3be3fdac131bbe35d37e479588920589781" + }, + { + "path": "docs/20260915_171525/depth/000116.png", + "bytes": 333296, + "sha256": "0b55ed1c6e9aa7528e39122f74bca3116640e86d755c90dc583f98df8dcb651c" + }, + { + "path": "docs/20260915_171525/depth/000117.png", + "bytes": 337283, + "sha256": "59878fca8df4b80a7e414e8e3ec88dfa90a5db4e7e668c0090b45d90e4ba036b" + }, + { + "path": "docs/20260915_171525/depth/000118.png", + "bytes": 340141, + "sha256": "daad3ac6cc1fed70c55cc51462c76438035d4a1c0d3bd407e87e69c67ae39ad0" + }, + { + "path": "docs/20260915_171525/depth/000119.png", + "bytes": 332045, + "sha256": "a6fb0bf9d9c13452f75a39638b0ea88bfb815df5ed44b37e3631ff6a733e089d" + }, + { + "path": "docs/20260915_171525/depth/000120.png", + "bytes": 331659, + "sha256": "abefc4e99ea18b92df0b9ef999779f1d69ba17a37e5692185bcd72254b1276bf" + }, + { + "path": "docs/20260915_171525/depth/000121.png", + "bytes": 336383, + "sha256": "af20279f1db8a701ef61ae2b919411898d015c4ea508dac8dc50cc841dc08287" + }, + { + "path": "docs/20260915_171525/depth/000122.png", + "bytes": 332920, + "sha256": "95b3ade515ab9b8dd6323ef3fac1299a1122b57a92989ad34d33d48f314fef5a" + }, + { + "path": "docs/20260915_171525/depth/000123.png", + "bytes": 337071, + "sha256": "d8b13e6be85aba0a9c8036a1858210f414767b216c215d1ea1d34da8fa597050" + }, + { + "path": "docs/20260915_171525/depth/000124.png", + "bytes": 327884, + "sha256": "29a3ec69c0f5ad7ea3d09dd05fcd700839db8cbcae105ebdffe79d7c246155bf" + }, + { + "path": "docs/20260915_171525/depth/000125.png", + "bytes": 333056, + "sha256": "4317f191c861c7ad189f9221d6802c88a35e3d7ae474563510e1c76f1a5ad49d" + }, + { + "path": "docs/20260915_171525/depth/000126.png", + "bytes": 335081, + "sha256": "56a7d825a8cceda65dc861304bc3ca114301b29bd1c6e8975618c94d6f8ea1fd" + }, + { + "path": "docs/20260915_171525/depth/000127.png", + "bytes": 329630, + "sha256": "225c6cb7d8ca6d783dfef8838719979f2769199dddf014d7a21dbe3fd326f1b8" + }, + { + "path": "docs/20260915_171525/depth/000128.png", + "bytes": 331393, + "sha256": "f3ac280fbbbd3b8be844fb7edab58e014acbdf06901d10451fb2a5f716c400fa" + }, + { + "path": "docs/20260915_171525/depth/000129.png", + "bytes": 333954, + "sha256": "b7c72dfa35451b2aaf9e099d8072cc7fed46eca79a7b15ac677592f2f431cf64" + }, + { + "path": "docs/20260915_171525/depth/000130.png", + "bytes": 330604, + "sha256": "2f0ed06e43e43345a1d03bbddc98d0abde4a130fdb13c918ca64076e8371fed1" + }, + { + "path": "docs/20260915_171525/depth/000131.png", + "bytes": 335143, + "sha256": "4027a883d9c6dd104d0cf3c7f96f9002e494c9bafb8d8c0804f4d8445a8ccd34" + }, + { + "path": "docs/20260915_171525/depth/000132.png", + "bytes": 327054, + "sha256": "a1ee3982c24a0555c0480dc7508e2a68b9665a84194c0dd166ff91f534a15c26" + }, + { + "path": "docs/20260915_171525/depth/000133.png", + "bytes": 327517, + "sha256": "c2ba3f1a066a9f9d7a739e083082bd3fc25f18b0858c1a4dec08d72b06fe06bd" + }, + { + "path": "docs/20260915_171525/depth/000134.png", + "bytes": 324287, + "sha256": "48e2080bcda77ad467a90526566e29dfac31368f71c5c11e4a9595c03c35a601" + }, + { + "path": "docs/20260915_171525/depth/000135.png", + "bytes": 325639, + "sha256": "0d4e3d47960b9642c9aaee1cbb9e52e57005a5a7ae3f7e41aa53a4217ad37360" + }, + { + "path": "docs/20260915_171525/depth/000136.png", + "bytes": 329695, + "sha256": "b76e25f161507c0266d642015815c82f96c05e3cce6144327adf057cb1a3cf35" + }, + { + "path": "docs/20260915_171525/depth/000137.png", + "bytes": 341721, + "sha256": "e7621aa9a4fdab5ec4d1a8f6a212f211f2700b2959ca6a4672a72387a1c1871b" + }, + { + "path": "docs/20260915_171525/depth/000138.png", + "bytes": 331182, + "sha256": "0c2241e7a7da5f8492d936e22b5b8f7180713af917f0e93be22afabe6f54cac5" + }, + { + "path": "docs/20260915_171525/depth/000139.png", + "bytes": 331074, + "sha256": "569e754145f71a825efceee82b9d1e2cb6970eaca97483e0a4475d5f393c9f8d" + }, + { + "path": "docs/20260915_171525/depth/000140.png", + "bytes": 334671, + "sha256": "68789365b613905a7726b04b026d04a16a3beecd189f1e0de70e58dde2767761" + }, + { + "path": "docs/20260915_171525/depth/000141.png", + "bytes": 328554, + "sha256": "14fb38ad1c180798921dcdbd53e72d43f3c8d3ff7187561273abbf1e81255415" + }, + { + "path": "docs/20260915_171525/depth/000142.png", + "bytes": 334442, + "sha256": "40530938d432287661c70347f64ff0d43ff800dfbcf2ab08b5b8a75fd9868912" + }, + { + "path": "docs/20260915_171525/depth/000143.png", + "bytes": 324431, + "sha256": "1b45dd4c8b91800677b2e6c132df60acac9d14d46201a0c8be23c3238941812d" + }, + { + "path": "docs/20260915_171525/depth/000144.png", + "bytes": 333249, + "sha256": "56d550a0076c30ef14231394a3742014aadb9c53b3fe3408fd081f0138a19594" + }, + { + "path": "docs/20260915_171525/depth/000145.png", + "bytes": 326788, + "sha256": "110c43d218c4c8c67a8e9cafd93e7e70e0f1428ba7be6a80308970daaa46b2e3" + }, + { + "path": "docs/20260915_171525/depth/000146.png", + "bytes": 334412, + "sha256": "7a3ce862a6e83b97b1d934478965e50470619260f8aec2dd0fd7a50d2ee551b9" + }, + { + "path": "docs/20260915_171525/depth/000147.png", + "bytes": 331338, + "sha256": "8f1f1ae1ce7228dcf2bfc3691e44c765298daa37c7d5d80f20a2832b5765ef60" + }, + { + "path": "docs/20260915_171525/depth/000148.png", + "bytes": 333531, + "sha256": "2441cb8f3b19209d61d5437934dba9eb337d551d64957be09ac0a226d94839d4" + }, + { + "path": "docs/20260915_171525/depth/000149.png", + "bytes": 332892, + "sha256": "c8394057b3529e5829ac8cd3cab79e64308315dd7bbcd3c3a271d09d937b5246" + }, + { + "path": "docs/20260915_171525/depth/000150.png", + "bytes": 327617, + "sha256": "5df514a33644f066a35ccb95b656f6386548a9a1fd1726cdb7f22b201771979e" + }, + { + "path": "docs/20260915_171525/depth/000151.png", + "bytes": 333403, + "sha256": "e9df5956e4ab238b8f03a50701228802fc453ad463e85bcc2dae3118016316cd" + }, + { + "path": "docs/20260915_171525/depth/000152.png", + "bytes": 337265, + "sha256": "c82c5c1972627860e81ca4acf0ca55813c04b3e7cdf65c94374e3651209030a3" + }, + { + "path": "docs/20260915_171525/depth/000153.png", + "bytes": 330260, + "sha256": "f9454bad7b6899cc45fa7cab7812bd023c81e8ecd8beba4c177d28e853c25460" + }, + { + "path": "docs/20260915_171525/depth/000154.png", + "bytes": 336906, + "sha256": "7e1aea1a6c3487b3dbd96205b4d6360a09f0d977d1046822235b01614f862438" + }, + { + "path": "docs/20260915_171525/depth/000155.png", + "bytes": 332993, + "sha256": "d3ca6691693496b50d8a0cdbf78922d473054ef7bd28301280371bea8becd515" + }, + { + "path": "docs/20260915_171525/depth/000156.png", + "bytes": 336608, + "sha256": "179758c652ddb616b8a562f5da09e33367f953ad0c9da05ac522a189d24e3f50" + }, + { + "path": "docs/20260915_171525/depth/000157.png", + "bytes": 326314, + "sha256": "ecfe568145679367dc076cdc3538d386968cd6cb9c177560d12c8092a351f2ab" + }, + { + "path": "docs/20260915_171525/depth/000158.png", + "bytes": 326150, + "sha256": "68ff6807da5c5fe746833a1f5b17805d343c0b7a036c59f2b09acf21158c1269" + }, + { + "path": "docs/20260915_171525/depth/000159.png", + "bytes": 334963, + "sha256": "c3c60d4308ec25e06d78eb0ade2097a54354d9a0789716f0b542ff652b64e40a" + }, + { + "path": "docs/20260915_171525/depth/000160.png", + "bytes": 342649, + "sha256": "3620426ee5812a40659c085df34cc0730e599ba4b86c81e3fd90d14442bceb98" + }, + { + "path": "docs/20260915_171525/depth/000161.png", + "bytes": 328437, + "sha256": "04bc5561e345ba7db3cc723159541b8fea6f866e60c5c3035ad6e58b395553d9" + }, + { + "path": "docs/20260915_171525/depth/000162.png", + "bytes": 332938, + "sha256": "2e7aa7be91d87a5923c5bb4a3e9d8afc85a8675b6fa0d395b1679bcf636e4520" + }, + { + "path": "docs/20260915_171525/depth/000163.png", + "bytes": 335648, + "sha256": "b79a0279a095e8cd29437883b0f3f042919eb2c177fb790c2263037df066a5a2" + }, + { + "path": "docs/20260915_171525/depth/000164.png", + "bytes": 335803, + "sha256": "82a67d40f49451c8bb75c76f3f5df7afb452b0129d18ac6f0f2c7b9801899e07" + }, + { + "path": "docs/20260915_171525/depth/000165.png", + "bytes": 333527, + "sha256": "c3cc0d25e0b0b4aedc3c6816563f3d5d811f8adcbc5d4bb1b6d7e2ccf7ba33a6" + }, + { + "path": "docs/20260915_171525/depth/000166.png", + "bytes": 335795, + "sha256": "4be22868201e9ad409b15af3871be21d006a76d20f268dad283286ec37a70174" + }, + { + "path": "docs/20260915_171525/depth/000167.png", + "bytes": 332407, + "sha256": "d5bbc4b92a27949747055f3fbaa1f91d74af7aa26bd24a67e4383deca083a00e" + }, + { + "path": "docs/20260915_171525/depth/000168.png", + "bytes": 336554, + "sha256": "0fb3263982ea79350f9ff7687a45237b14a43bf94e477492e1653e52d769ea0d" + }, + { + "path": "docs/20260915_171525/depth/000169.png", + "bytes": 328142, + "sha256": "4cfb900dcfffdf79038776d36739492970f7895c80e9d916dc1e80c2ae8ce74a" + }, + { + "path": "docs/20260915_171525/depth/000170.png", + "bytes": 331689, + "sha256": "e1a27988f69653da5c1cc23d94c8f21c4804871cb8b29d726d8256ee3da5dda4" + }, + { + "path": "docs/20260915_171525/depth/000171.png", + "bytes": 326437, + "sha256": "87ba4832c3ff416e5b329b6fb42d61717500a6e0771feb42eb0acbf8a21ef4a9" + }, + { + "path": "docs/20260915_171525/depth/000172.png", + "bytes": 319090, + "sha256": "d5e8614d0dcb778966ea249347df41f867b771c3da8c8c7fc2192eae5caf073b" + }, + { + "path": "docs/20260915_171525/depth/000173.png", + "bytes": 325674, + "sha256": "49b7117c55c095488a668d05bc2eabd0a4909630735f70498b3c336020479d8c" + }, + { + "path": "docs/20260915_171525/depth/000174.png", + "bytes": 336349, + "sha256": "35c4190e641a076dd91b96d2a948e632872b6616421c8efbee538b85a2d68e7e" + }, + { + "path": "docs/20260915_171525/depth/000175.png", + "bytes": 333999, + "sha256": "684dbac39da6dad5b1f51c526e15337a4f98599287f2bd19c8c30ba54796765d" + }, + { + "path": "docs/20260915_171525/depth/000176.png", + "bytes": 328702, + "sha256": "4c5bee6e09ade833bb80f0f2e17feb2048db506569bb78a1f22203d5ef13f24a" + }, + { + "path": "docs/20260915_171525/depth/000177.png", + "bytes": 326273, + "sha256": "620697d0d357cb6409a461ebe8491b34968494f4086256ebc35e95d03c48514f" + }, + { + "path": "docs/20260915_171525/depth/000178.png", + "bytes": 324540, + "sha256": "af1374bfbc6ec9bea6b75b52f6a9a38c75ba0bd150a9a1310913f5a69a8152cb" + }, + { + "path": "docs/20260915_171525/depth/000179.png", + "bytes": 328422, + "sha256": "84eb63913c4d3411ec2f68b3b60ddd064aaee01a47001d9e2cd6da2f3becf652" + }, + { + "path": "docs/20260915_171525/depth/000180.png", + "bytes": 320641, + "sha256": "55d2c4e118744c48848a8977cf5673e0e5dc5878f371260f20e4addc59ed23d8" + }, + { + "path": "docs/20260915_171525/depth/000181.png", + "bytes": 331661, + "sha256": "a5be18cec3a9f364844a73ee4aae4b1e3b194c7da7c45be486db2827363a575c" + }, + { + "path": "docs/20260915_171525/depth/000182.png", + "bytes": 330312, + "sha256": "d78b47a60ae6c5a59709eb6bd3f84bc54fdf110e36f70dec44869205b65fdeeb" + }, + { + "path": "docs/20260915_171525/depth/000183.png", + "bytes": 331977, + "sha256": "b6f0e32e252e8c7d5c056efb671aaed7d9b90325ae3804fdec3b8547f8b0cfc3" + }, + { + "path": "docs/20260915_171525/depth/000184.png", + "bytes": 318716, + "sha256": "45b17ab5b776d3dbd0cc9690786313f70bedea97b69ce6efce26c751e9b14aef" + }, + { + "path": "docs/20260915_171525/depth/000185.png", + "bytes": 329445, + "sha256": "0b6e3354594762099e56e88bc79f80c08375b3ead32d8330cad5b72d638e5090" + }, + { + "path": "docs/20260915_171525/depth/000186.png", + "bytes": 334010, + "sha256": "72b5aeee7201afe7682fa2c543d92df677b908ff92e3899f7d956e99a4ca20d5" + }, + { + "path": "docs/20260915_171525/depth/000187.png", + "bytes": 331826, + "sha256": "95ade8a5003788509ab92f91b86d568c37aba79bd9599bb0ace23cbe711db778" + }, + { + "path": "docs/20260915_171525/depth/000188.png", + "bytes": 330885, + "sha256": "97cf3b3db850b944edc6f32201b11229003d678ca1a4169a6e66ea079f1483ee" + }, + { + "path": "docs/20260915_171525/depth/000189.png", + "bytes": 322507, + "sha256": "22fc6749ac321bc54bd0c642d832cfad2bec0a3e7dfd127bb7385c8c118aa798" + }, + { + "path": "docs/20260915_171525/depth/000190.png", + "bytes": 330000, + "sha256": "885cf8695bcdfc156d60b30a18c145b109137013f0bcc9d785edcc5117d1dfe3" + }, + { + "path": "docs/20260915_171525/depth/000191.png", + "bytes": 343382, + "sha256": "0c3a880d5618d2af4557337c71c8ac72004ab0ae1ef04ce1a44dabeb81d26fc3" + }, + { + "path": "docs/20260915_171525/depth/000192.png", + "bytes": 324812, + "sha256": "c8a773d7ccffcd74f283e3f9b17042b1d4386b66ffe3281182409485e0829cc9" + }, + { + "path": "docs/20260915_171525/depth/000193.png", + "bytes": 323324, + "sha256": "da598b244eb5b3741b35e00830ffae166f81d8042ea5d0480c12d86ab50f22fd" + }, + { + "path": "docs/20260915_171525/depth/000194.png", + "bytes": 339008, + "sha256": "cb933266fd689dd60950becd989f60e8725dde63279b657499fd2c6d130d765c" + }, + { + "path": "docs/20260915_171525/depth/000195.png", + "bytes": 333033, + "sha256": "ac809cff381215c478f5e55c439815afc22eac071ea8c8f40a82cd643f59f0b9" + }, + { + "path": "docs/20260915_171525/depth/000196.png", + "bytes": 331635, + "sha256": "404949199a71bb01c793495e251a1c540187004beabc90cb8e2386a1cf4fb84c" + }, + { + "path": "docs/20260915_171525/depth/000197.png", + "bytes": 330943, + "sha256": "1ce0b4d1161166eb35f49a9ab3c1b8f278409277f03e1ebc4865d96f7f2743d1" + }, + { + "path": "docs/20260915_171525/depth/000198.png", + "bytes": 338409, + "sha256": "6ba0ce35421979fb133f38b02579012301eee9a071fdc50ac0742da33a2b5af9" + }, + { + "path": "docs/20260915_171525/depth/000199.png", + "bytes": 338236, + "sha256": "864bf3e21bc996b46bcce2ee7a2a432a2e1181755bcca0a22224f92877912927" + }, + { + "path": "docs/20260915_171525/depth/000200.png", + "bytes": 334525, + "sha256": "56bf11a5d167025f138038ec7733fe02f4346dabb13d99a158347d15cadc7774" + }, + { + "path": "docs/20260915_171525/depth/000201.png", + "bytes": 335526, + "sha256": "60fecce3f6cc986333114fc0ac1a088c90f2342486f901bef9f19a2062bc950f" + }, + { + "path": "docs/20260915_171525/depth/000202.png", + "bytes": 344306, + "sha256": "9146d7262de4b84c9bab033ab61cecd4abd16f3a1b47f98ec5e94beacb0905e7" + }, + { + "path": "docs/20260915_171525/depth/000203.png", + "bytes": 339786, + "sha256": "5520d111021710a690e96e6d734afe9d5bc8f7bba7ff5173ac2134991c2f5b65" + }, + { + "path": "docs/20260915_171525/depth/000204.png", + "bytes": 337361, + "sha256": "a6334587987d68389c382d810837c615ddf3f84232c663511ae343ff36b20041" + }, + { + "path": "docs/20260915_171525/depth/000205.png", + "bytes": 336850, + "sha256": "eb02f67b2504968829906041addeb6d914dbf9ea3b61093f07aa593dbb9707cd" + }, + { + "path": "docs/20260915_171525/depth/000206.png", + "bytes": 340725, + "sha256": "5f295a9b904f76217a24d035b0b32a3232521f44b561962e437b4b671bc12b71" + }, + { + "path": "docs/20260915_171525/depth/000207.png", + "bytes": 342439, + "sha256": "5d1194787e445fca060618f4c13e1247d984310b49f6015318314391a536e741" + }, + { + "path": "docs/20260915_171525/depth/000208.png", + "bytes": 334014, + "sha256": "9d7403ce70f8026603738c77a6b7a6e10b1cd77b42ff3bfb441b35afe993d835" + }, + { + "path": "docs/20260915_171525/depth/000209.png", + "bytes": 331616, + "sha256": "3e29c24996653c2b27c8033a50c0fb4959b4eb45dc66ae3ccd2e41ed5ff2fc14" + }, + { + "path": "docs/20260915_171525/depth/000210.png", + "bytes": 327568, + "sha256": "5adba0ed6635a0f681bdf72a917b3821e7cbb19b8a1734e0d555f11c3b2e8ecc" + }, + { + "path": "docs/20260915_171525/depth/000211.png", + "bytes": 337133, + "sha256": "b0f7f84cf0974b2e59ad9f9fb36056ee0a80de52f85f455e58cb73bc4eefba93" + }, + { + "path": "docs/20260915_171525/depth/000212.png", + "bytes": 325900, + "sha256": "4ee8c34a504c43ec282e3f408fd323a08f20659dcac43e0484bad4d071b673ed" + }, + { + "path": "docs/20260915_171525/depth/000213.png", + "bytes": 341307, + "sha256": "877acb79bfadfd564c4b7b3a2a1de2bed2d8b599738f7a6351eeab7793d5702f" + }, + { + "path": "docs/20260915_171525/depth/000214.png", + "bytes": 341173, + "sha256": "a6b042f9eeb1c16d08a5c11429c01f3101652a618681286fd7f68c16ed8dfd5d" + }, + { + "path": "docs/20260915_171525/depth/000215.png", + "bytes": 333597, + "sha256": "73981702aee5f16011413265314fd4297e75e2727ec2db88bf3a1d8167040e6f" + }, + { + "path": "docs/20260915_171525/depth/000216.png", + "bytes": 332194, + "sha256": "52ee371a8fd80c45442fddf34377af815cdbf4ea1e814319d6ee9d6a4ef450b6" + }, + { + "path": "docs/20260915_171525/depth/000217.png", + "bytes": 347559, + "sha256": "d14467c53eef4abf99faff3a87f520ed302e8fa2f78dbb8c2a687faa313391d4" + }, + { + "path": "docs/20260915_171525/depth/000218.png", + "bytes": 342045, + "sha256": "81a972cc58db4ba15e2737657c6838d58bb69e358c715e97950e6cc260dc48d4" + }, + { + "path": "docs/20260915_171525/depth/000219.png", + "bytes": 322261, + "sha256": "dabafb355f1e8befb9a07c3a0205c12ef1ebea92b18afab58b6cbd41c6c4877f" + }, + { + "path": "docs/20260915_171525/depth/000220.png", + "bytes": 336617, + "sha256": "78134b82bd1b1a87e5bfcabf6e96a234be050c4f3da605a118e98a50464600f4" + }, + { + "path": "docs/20260915_171525/depth/000221.png", + "bytes": 332607, + "sha256": "37601beb55e7bfd27c349a17fc8f9ed355948c4e96c963c119e90b3fc9ae1de4" + }, + { + "path": "docs/20260915_171525/depth/000222.png", + "bytes": 336830, + "sha256": "c6df5946e843dae471dd40723b3ba5693336951336c3c475165274620646e381" + }, + { + "path": "docs/20260915_171525/depth/000223.png", + "bytes": 331512, + "sha256": "600574fd2dbd1e7129b870ad25402dbfda4fa8b3bc6d1f09815e74d36a6df100" + }, + { + "path": "docs/20260915_171525/depth/000224.png", + "bytes": 327698, + "sha256": "7ca113ae3077fe9f1fbd9ced052d7aeff3da51291c86dbca581cf11b237fdadf" + }, + { + "path": "docs/20260915_171525/depth/000225.png", + "bytes": 327685, + "sha256": "693abb7195e480cdddd5551ef77984b8317bddc3ac9fed336ab32b4818ade9ea" + }, + { + "path": "docs/20260915_171525/depth/000226.png", + "bytes": 333812, + "sha256": "23a54b45ca29388877429a4fb974da2caee3f990d137fa562880f948ad60c44e" + }, + { + "path": "docs/20260915_171525/depth/000227.png", + "bytes": 329480, + "sha256": "ab04f1894cf3e19c433f40c85b5cc76735aed3cfb64fd7bd01ab394d0f481e0a" + }, + { + "path": "docs/20260915_171525/depth/000228.png", + "bytes": 333126, + "sha256": "da53ad7b9de8ac61732c0ce4fdb2536db8ea5c6af2018f28a3c410b9c44e22a0" + }, + { + "path": "docs/20260915_171525/depth/000229.png", + "bytes": 337457, + "sha256": "307c37b530cae97e9e9511df2ed21fee95637d88bd4d2c05c98fa9a6514d0c78" + }, + { + "path": "docs/20260915_171525/depth/000230.png", + "bytes": 321232, + "sha256": "aed1f75d3c3c04209aa6a82a335e4f73c0eeb9fa2efb93c9067b467d4fe439c6" + }, + { + "path": "docs/20260915_171525/depth/000231.png", + "bytes": 327770, + "sha256": "eea25cfe4152d633a274f6c305fd5067df3dbb8a2242046f349335dbd1f91d2d" + }, + { + "path": "docs/20260915_171525/depth/000232.png", + "bytes": 324600, + "sha256": "7d60387c58065d7f7f8dcc2716730656e6535784c6f7161b4bab291b529152b4" + }, + { + "path": "docs/20260915_171525/depth/000233.png", + "bytes": 332118, + "sha256": "d6d7784885b1114914e42d6ef3f80c7c9ee72e154fe5ddf4203dfbc529b0e464" + }, + { + "path": "docs/20260915_171525/depth/000234.png", + "bytes": 331965, + "sha256": "19f2951bb89d1fd6a02dd5afd3bc8d749921e787f0e43821fe8c9188dc2fdfb0" + }, + { + "path": "docs/20260915_171525/depth/000235.png", + "bytes": 333739, + "sha256": "8b517747551107930dead16c2bd2c592c15e68a73d4dd1cc261cf353415e02e3" + }, + { + "path": "docs/20260915_171525/depth/000236.png", + "bytes": 330192, + "sha256": "ff1db182687a831b9ef343025d5bebbc56cd6bf5941c27b1f98cc818b21123e2" + }, + { + "path": "docs/20260915_171525/depth/000237.png", + "bytes": 336667, + "sha256": "4b84253cbee33b2cce9eccb2db0389add9400329640020f16f784e7cdd1af8e1" + }, + { + "path": "docs/20260915_171525/depth/000238.png", + "bytes": 329451, + "sha256": "236e1b1a5d559ceab56793c634318af842dea16c7c4dc426438e9664af10393f" + }, + { + "path": "docs/20260915_171525/depth/000239.png", + "bytes": 329714, + "sha256": "711a4296de42740eedb87ea9814a0a4ae9d43524167564daebdfbfd91ea4568d" + }, + { + "path": "docs/20260915_171525/depth/000240.png", + "bytes": 337275, + "sha256": "ebbf2258d1ce523d4992863fd6f9ab9fa987daf9a45d386fa21d57855cb7aa7a" + }, + { + "path": "docs/20260915_171525/depth/000241.png", + "bytes": 332832, + "sha256": "bd5ade8df29ea663621fd8a43b44be1161cddcc5bbb276c017a1751b19ef877e" + }, + { + "path": "docs/20260915_171525/depth/000242.png", + "bytes": 336182, + "sha256": "8a2ee760f8e5c7f6d1170ac5fb974d40d06e606c72355f047d4e4ec0746f2313" + }, + { + "path": "docs/20260915_171525/depth/000243.png", + "bytes": 323470, + "sha256": "9e776f037bbd5814562114ec96ec90bb0b5fee59e42526d4c68b3d40fad677b9" + }, + { + "path": "docs/20260915_171525/depth/000244.png", + "bytes": 327958, + "sha256": "ca9bbdeb12a45f35a41a42a1f57f071b4f5f6c49bbd52a22af95ce1902d42c0d" + }, + { + "path": "docs/20260915_171525/depth/000245.png", + "bytes": 332216, + "sha256": "359a74eed588d5e66aa0fe17bb1bb8f7cccb2e78367d5c6d128f52d003fb4c86" + }, + { + "path": "docs/20260915_171525/depth/000246.png", + "bytes": 333155, + "sha256": "052ea9ad871c2b200da85793d5590b2529cc695172138492a301c0fb006b6166" + }, + { + "path": "docs/20260915_171525/depth/000247.png", + "bytes": 319274, + "sha256": "0d5ac745ba5920724a1cb5443aa55760f56d7e543732f021880ca39c857d65a5" + }, + { + "path": "docs/20260915_171525/depth/000248.png", + "bytes": 335742, + "sha256": "722caac2b6e968a6dd716d6b22cfeea797586504ec50b80c0dcc5dee8c38af86" + }, + { + "path": "docs/20260915_171525/depth/000249.png", + "bytes": 331237, + "sha256": "c524f2ad0e4e4019a15f1dadd9c3fde27fd25570c2272e325a28de2d90689a5b" + }, + { + "path": "docs/20260915_171525/depth/000250.png", + "bytes": 334409, + "sha256": "cbe4c6d7de9578279de97b0821450c782e944a7f33e5c442bd67a6bb24ddbb91" + }, + { + "path": "docs/20260915_171525/depth/000251.png", + "bytes": 341568, + "sha256": "14559aa8c915afd754a084f4261bc7991a55f98cf98085c4baf1fd54cebc7e34" + }, + { + "path": "docs/20260915_171525/depth/000252.png", + "bytes": 332619, + "sha256": "aaa5cd93168d48db4c67c31fc98dbddc255135d3aa53b44b37825a2f7820b421" + }, + { + "path": "docs/20260915_171525/depth/000253.png", + "bytes": 335338, + "sha256": "8c3ef01bfdbfa35c1a97c5622e01d39814762efb57d239bd43f1c88e9f04e962" + }, + { + "path": "docs/20260915_171525/depth/000254.png", + "bytes": 338771, + "sha256": "011cea48efdce6ffb171c20de3d7d44c80467bfcce472189286724829597065e" + }, + { + "path": "docs/20260915_171525/depth/000255.png", + "bytes": 332064, + "sha256": "36dc94d612ffe20f90379844886ee22f20ba2947c0155a7bd85fe51de57b81ea" + }, + { + "path": "docs/20260915_171525/depth/000256.png", + "bytes": 326230, + "sha256": "e2f371d180190bc0a70c18226cb4f06958fed3f4e62d6db8136b17841683808f" + }, + { + "path": "docs/20260915_171525/depth/000257.png", + "bytes": 321664, + "sha256": "02542e9973982c59e7adc8af2a2afb3afb99af34a0821d4d1df95b0f78e3339f" + }, + { + "path": "docs/20260915_171525/depth/000258.png", + "bytes": 340132, + "sha256": "7239744a88161b87335703e544dab435eaaa5c90c17c7c9c8dccaf7d30fba5e4" + }, + { + "path": "docs/20260915_171525/depth/000259.png", + "bytes": 337472, + "sha256": "bd562dc769bc3572ed4627ef008400024c3eb2ea2e098fb1494b6ecad8e715aa" + }, + { + "path": "docs/20260915_171525/depth/000260.png", + "bytes": 334334, + "sha256": "dac018cc520c89d8abf571f6514fb6893a4a1b2fbef05644cd4a8b5fdc8054b2" + }, + { + "path": "docs/20260915_171525/depth/000261.png", + "bytes": 328342, + "sha256": "31a653934882423daf578f64ca679c7e4858a28b9ae3a1690b8f3a472a557bea" + }, + { + "path": "docs/20260915_171525/depth/000262.png", + "bytes": 326430, + "sha256": "e3ca0769cb354e53228e00184d3bd14fac9716d189e071f375c636cf5c7791eb" + }, + { + "path": "docs/20260915_171525/depth/000263.png", + "bytes": 330371, + "sha256": "395490dfffad705620d2e1f3050e75859db66b0e27e81e6276acbdbc9a2d4ce5" + }, + { + "path": "docs/20260915_171525/depth/000264.png", + "bytes": 322898, + "sha256": "59872066522eb3159a8951fd2bc3736722932afc016026e39d6e42cc75b9b82a" + }, + { + "path": "docs/20260915_171525/depth/000265.png", + "bytes": 319610, + "sha256": "c3e6d2daffdaeeca925dee96d5140c327d7b1b0189931eeb4bda2a75508a991f" + }, + { + "path": "docs/20260915_171525/depth/000266.png", + "bytes": 319250, + "sha256": "751c34d61257af531287c5f6982e834b4f8049c1c0315eecf2e5ba1b6b022293" + }, + { + "path": "docs/20260915_171525/depth/000267.png", + "bytes": 327744, + "sha256": "c2a7a53be65dcade64119d88c36d89d3b03ddea64b990c97c4701c80420b9170" + }, + { + "path": "docs/20260915_171525/depth/000268.png", + "bytes": 326219, + "sha256": "5b2c76e2b4f373df8e4f63e106893adc26118cf26d8cb6392c434d1eb68154f4" + }, + { + "path": "docs/20260915_171525/depth/000269.png", + "bytes": 338857, + "sha256": "1642f9d854951cb6d3fc931870dad59aa25d807acc08c9c9d2f1bc64b3301346" + }, + { + "path": "docs/20260915_171525/depth/000270.png", + "bytes": 331363, + "sha256": "b8364209adff7a1cebb33918d76e7fb1691936dc59ce9c47dd5bf6fa684a5158" + }, + { + "path": "docs/20260915_171525/depth/000271.png", + "bytes": 332042, + "sha256": "1ad036269a3562abd20a06ee6249a6bc2ba55aea4a2c3243512fe704f9c1ab45" + }, + { + "path": "docs/20260915_171525/depth/000272.png", + "bytes": 336032, + "sha256": "812d6d3ca1489337a3cd798a857e7ddd407896ca2bd86b7151b16cf650a8d66b" + }, + { + "path": "docs/20260915_171525/depth/000273.png", + "bytes": 322749, + "sha256": "880a1a1f6906b2fdb64a99cc1cd92ec0e1ef88bb318bbabf77948eeb1f1ef138" + }, + { + "path": "docs/20260915_171525/depth/000274.png", + "bytes": 331069, + "sha256": "fd2e9b90f311d1631b6592409cb2018240673bbcd291cb00a880345cf1f7bf4a" + }, + { + "path": "docs/20260915_171525/depth/000275.png", + "bytes": 335024, + "sha256": "9004ec42dae72f2d61a8765d70351ae6dee8c6b29df9025e92ad92d73bd1412b" + }, + { + "path": "docs/20260915_171525/depth/000276.png", + "bytes": 337313, + "sha256": "8bea5306675a7c5f718aed32107409b6d62ace8bc2703076bc3239fabdc3224d" + }, + { + "path": "docs/20260915_171525/depth/000277.png", + "bytes": 343624, + "sha256": "e2097cb40a17c8059ed4681c268094535af971bd647daefb3725839c72aefb9d" + }, + { + "path": "docs/20260915_171525/depth/000278.png", + "bytes": 333550, + "sha256": "7f43cef00330403f320fff6cb111b490ab648652b46cee2b4d9bfeb263310a3d" + }, + { + "path": "docs/20260915_171525/depth/000279.png", + "bytes": 326901, + "sha256": "31d76ed50ec2ea472c73b1f72c316cf84e2a4c302502104f11d8f6fcedb0a266" + }, + { + "path": "docs/20260915_171525/depth/000280.png", + "bytes": 334886, + "sha256": "13bc4839c69efbcc418bede819e27390d80d53e1746a376203b845e8372db64d" + }, + { + "path": "docs/20260915_171525/depth/000281.png", + "bytes": 332117, + "sha256": "deec20393ef5a735940d61b793ef5c8619562188904417694810d24c01edf0b5" + }, + { + "path": "docs/20260915_171525/depth/000282.png", + "bytes": 334899, + "sha256": "759cc9c14a22af046f7299cb18b972de430a07c71ce61060bbf57b04faa5d951" + }, + { + "path": "docs/20260915_171525/depth/000283.png", + "bytes": 326890, + "sha256": "e329ca785b07f2e96c18b53cba4ef86568d175ee3c5e611573be05de9a8e37fb" + }, + { + "path": "docs/20260915_171525/depth/000284.png", + "bytes": 330777, + "sha256": "a5dd297c4c19af14d8853fca7eea1e049dd390be39cb2bd52a0333a532e43917" + }, + { + "path": "docs/20260915_171525/depth/000285.png", + "bytes": 325576, + "sha256": "0ea667806ff4e49bee816d1fb5237ba27a9c412847066a6552fa6d26ad43d423" + }, + { + "path": "docs/20260915_171525/depth/000286.png", + "bytes": 330853, + "sha256": "f4ed0c93662e92bcf722a8043374459d22efe6c15890fdded84b174d04123ace" + }, + { + "path": "docs/20260915_171525/depth/000287.png", + "bytes": 326788, + "sha256": "bab27cc50fefae0a0eb50a24e53fea52aa56d49c7c2e9d307dfa06583b7e83cd" + }, + { + "path": "docs/20260915_171525/depth/000288.png", + "bytes": 327659, + "sha256": "65b5b72680e14589cd0a8d9bb4be9a93790433bdebd8cf961b6c9bfe4bbaadcf" + }, + { + "path": "docs/20260915_171525/depth/000289.png", + "bytes": 334124, + "sha256": "d5fba23195c742b3fb1d847ff00757d992e21fcc35577ee9d1910c969e3c3f2b" + }, + { + "path": "docs/20260915_171525/depth/000290.png", + "bytes": 331380, + "sha256": "fc507789dcba87ab2143879ae3569238589b6538298848ea9447eba73e71070d" + }, + { + "path": "docs/20260915_171525/depth/000291.png", + "bytes": 332057, + "sha256": "07033b696d82ed6dcda2f0e60ef8cbc16b968b3bbc0e86bb019dabb78da99914" + }, + { + "path": "docs/20260915_171525/depth/000292.png", + "bytes": 330093, + "sha256": "2333b232984cc70ccf21d363f4c47b1860e050709bc9aa54bc27de12dbea2dbe" + }, + { + "path": "docs/20260915_171525/depth/000293.png", + "bytes": 328845, + "sha256": "7cbd13d2780da9190fc6af4316ff99e68c2ac4d1b352f281bb2a4f50d8e92fce" + }, + { + "path": "docs/20260915_171525/depth/000294.png", + "bytes": 328371, + "sha256": "ae20bb35b7dadb076d2470ba821a11a35abd910dc7c79b8de20c4ad194828dd1" + }, + { + "path": "docs/20260915_171525/depth/000295.png", + "bytes": 327450, + "sha256": "7c53b6c280fb30035220fb48f479826947a6493afa85089e8697671b32f08297" + }, + { + "path": "docs/20260915_171525/depth/000296.png", + "bytes": 329625, + "sha256": "dff13ad8c2d2c1b156fb79bfa2c820d8e88e6bf85dc2b9e385ba2f367cabf2cf" + }, + { + "path": "docs/20260915_171525/depth/000297.png", + "bytes": 322076, + "sha256": "3431c5364ba1231deec6ea56dc34eb924982c3952ae0879fa52927acb18ee83e" + }, + { + "path": "docs/20260915_171525/depth/000298.png", + "bytes": 329949, + "sha256": "b36d62df68fad7daf95240483658f9411968b07b1f2725ffea45e095b60705a9" + }, + { + "path": "docs/20260915_171525/depth/000299.png", + "bytes": 325152, + "sha256": "7e75ed8dbbeb3f03858c6014a3c67666ff207e0a4fe13c31bca9a52314c3f4df" + }, + { + "path": "docs/20260915_171525/depth/000300.png", + "bytes": 330020, + "sha256": "4942442a2f2ded49394b298d5504c9cab617a423b250dfa7b329a383165a1d33" + }, + { + "path": "docs/20260915_171525/depth/000301.png", + "bytes": 330022, + "sha256": "71cb044829cb0c0feeb6f7748a46123bf72adc1d15e7f312b765848a86a22ad9" + }, + { + "path": "docs/20260915_171525/depth/000302.png", + "bytes": 322514, + "sha256": "cfaf553a5dea3dea6ddd1dac8a8e30614b020c6e55154a122bad49d617d8e16f" + }, + { + "path": "docs/20260915_171525/depth/000303.png", + "bytes": 325997, + "sha256": "81f4b3a84852bd80ce767e8d9383654f409c9441d5de606ea22e835bbe5800d0" + }, + { + "path": "docs/20260915_171525/depth/000304.png", + "bytes": 332747, + "sha256": "5913e7cf0d8945d624f49663f8a5bc83b2fd122ab38286a286c5163ad1b760ec" + }, + { + "path": "docs/20260915_171525/depth/000305.png", + "bytes": 330702, + "sha256": "75ef76833977ab6acb36a686e2a1414f0f4cea95bfb993b310e5743438899400" + }, + { + "path": "docs/20260915_171525/depth/000306.png", + "bytes": 327900, + "sha256": "b0fc6ce3eb3c417444617243062ef8d5016b7f8b5f26761d3f3bb6fefe9f81d6" + }, + { + "path": "docs/20260915_171525/depth/000307.png", + "bytes": 326971, + "sha256": "16cf95a68eed4fd9a7a8ae0b4838738f02726c0fcc9adc506716e92843110844" + }, + { + "path": "docs/20260915_171525/depth/000308.png", + "bytes": 331966, + "sha256": "6ddec5e15f14221e80ca8ee017fb5bf048bb8cfda4e1e609bba7622fdffa435f" + }, + { + "path": "docs/20260915_171525/depth/000309.png", + "bytes": 328742, + "sha256": "04d25956149ffad5dc60113a7166666e2a31e5b269f771a38e4fac3b53ba10f9" + }, + { + "path": "docs/20260915_171525/depth/000310.png", + "bytes": 330508, + "sha256": "180a00564dc140de4cb9c16e4802028daf60469ae95c99c839ffab4142b19f54" + }, + { + "path": "docs/20260915_171525/depth/000311.png", + "bytes": 329038, + "sha256": "4aaf0cfedd9b2b3a51121942187e22ff066c2c400113e8f7a61b4af57fabfaa3" + }, + { + "path": "docs/20260915_171525/depth/000312.png", + "bytes": 335017, + "sha256": "ff10a4c59a121d4cdf77b117167f3d98f971518b9127cfe2f219e27781cb14f1" + }, + { + "path": "docs/20260915_171525/depth/000313.png", + "bytes": 332092, + "sha256": "e5fcbd206c21a42e0c4e8678422d2f9ae1f3d32fb97425e655c6117de234c74b" + }, + { + "path": "docs/20260915_171525/depth/000314.png", + "bytes": 323779, + "sha256": "c1417f9d14e5f0e782440cbc9654fcc5980e948c7c8fabed43c4317eba1e3b3b" + }, + { + "path": "docs/20260915_171525/depth/000315.png", + "bytes": 328895, + "sha256": "45d804a058dc41a1c956c79700d0baf1205c1a8a23b3551f3624b6a0a6f49d32" + }, + { + "path": "docs/20260915_171525/depth/000316.png", + "bytes": 327468, + "sha256": "be7d37a78f72984e850fc6fe335c18a2f55c133c821935c1d55d1d864f8f1611" + }, + { + "path": "docs/20260915_171525/depth/000317.png", + "bytes": 331047, + "sha256": "5d1569d0dd152af35746cf0a7ca84398db6120d4bbe06c3a376a87b9dd221339" + }, + { + "path": "docs/20260915_171525/depth/000318.png", + "bytes": 321328, + "sha256": "8e0550993dbf2b8558256f40184f7ea674ac10c8abdca69023429104dafe346b" + }, + { + "path": "docs/20260915_171525/depth/000319.png", + "bytes": 318392, + "sha256": "2e2ec2772e531409b8ad6beb2b30314c6098e3efdd856df85f5eca5f093a0cca" + }, + { + "path": "docs/20260915_171525/depth/000320.png", + "bytes": 325051, + "sha256": "bf776149cccd8f8a8b9403511d260e7a5175a5b7b252c412dbccfb4dbec78a9c" + }, + { + "path": "docs/20260915_171525/depth/000321.png", + "bytes": 335457, + "sha256": "9a159dd60d33223da7d407ed066a47016fc810e92abf3a4e0311873d794fb0cd" + }, + { + "path": "docs/20260915_171525/depth/000322.png", + "bytes": 335865, + "sha256": "07a955a200ec84f6bc6149491ba29f100e86172132a7c9a1bf1dd3c2a01ffcf8" + }, + { + "path": "docs/20260915_171525/depth/000323.png", + "bytes": 334435, + "sha256": "988dfd08293650d0c6124f44f483cff0407295cb339fab771da09e30a1377afa" + }, + { + "path": "docs/20260915_171525/depth/000324.png", + "bytes": 317218, + "sha256": "b5e0c72559c068837710aa3318850686bed7f347197b9e103eee389904fe58b3" + }, + { + "path": "docs/20260915_171525/depth/000325.png", + "bytes": 324662, + "sha256": "d13dcbc86da7b838272fdb9611a847696d0213cdf99f1ea04503e7c147a996c5" + }, + { + "path": "docs/20260915_171525/depth/000326.png", + "bytes": 327252, + "sha256": "92f2a9f8acd6a8ab79e98f4a5c5624fbdf04b0cb63a6e493590e99ff9dc16eba" + }, + { + "path": "docs/20260915_171525/depth/000327.png", + "bytes": 332721, + "sha256": "f3b1994d8b64f73c556a9e5df9c7e0c815960277db138131df265293ff4fa67f" + }, + { + "path": "docs/20260915_171525/depth/000328.png", + "bytes": 323638, + "sha256": "f777b4656b9bf0019905290b2d6f5b4975469446d7d2fb8a0ccedfd129b217c8" + }, + { + "path": "docs/20260915_171525/depth/000329.png", + "bytes": 326794, + "sha256": "7fdd7519f4284dd450036052611cd60d9a72bd39779f3d70a0ef95f17c8e6a44" + }, + { + "path": "docs/20260915_171525/depth/000330.png", + "bytes": 329987, + "sha256": "d1a43a05a854454b91db1b1dfbd6c67831a626a82ba3a27d7c3b9b14e22d04e2" + }, + { + "path": "docs/20260915_171525/depth/000331.png", + "bytes": 332902, + "sha256": "dd7156847883a030e7381938058c32f61d0abeb8eb1f3490137991db41172387" + }, + { + "path": "docs/20260915_171525/depth/000332.png", + "bytes": 329779, + "sha256": "c7053df7ba27713f93a993f6f2dcd0e3d7acb2b80d245d91a5fb7a4f17e74700" + }, + { + "path": "docs/20260915_171525/depth/000333.png", + "bytes": 332116, + "sha256": "637d707a7a8388d2e553e2a3d0e817518d65302804015234e833052d8aee7895" + }, + { + "path": "docs/20260915_171525/depth/000334.png", + "bytes": 329420, + "sha256": "23e42d749561d9798c4bcfc3969d878fa28baeaa2908349824be1b8a51130a3c" + }, + { + "path": "docs/20260915_171525/depth/000335.png", + "bytes": 330580, + "sha256": "ab2dea38b4730fe74f3e9bbddfb176bb1cb76ca4c15cc4750ff9f8ceef003752" + }, + { + "path": "docs/20260915_171525/depth/000336.png", + "bytes": 328374, + "sha256": "3940f19b35dc1099349c0667485130727abe0b32cf2cb5bf7397c5fb39288ede" + }, + { + "path": "docs/20260915_171525/depth/000337.png", + "bytes": 328365, + "sha256": "d76dfd9f7773254d6a15ac20ab7e540e60ac5ce473745bbe8a6531ab5f7adc32" + }, + { + "path": "docs/20260915_171525/depth/000338.png", + "bytes": 327146, + "sha256": "35612b40c0e4e11ca9ab7696e06f9cca5c9fbfbbe518702050e2294a18a3239b" + }, + { + "path": "docs/20260915_171525/depth/000339.png", + "bytes": 330624, + "sha256": "69bc46fe638e910b9be0a3f822446584a0be33e07e5a255dc6824aa097022e77" + }, + { + "path": "docs/20260915_171525/depth/000340.png", + "bytes": 331051, + "sha256": "0e6bf2a3c7c2d0e953d11a6f58f49b348e5fb23287ed02e09ed27f413ebb765e" + }, + { + "path": "docs/20260915_171525/depth/000341.png", + "bytes": 323655, + "sha256": "fa2c9f2d80b16237fa484417ffbd3b5d02b0d1705d2855471929f01ede0ad29e" + }, + { + "path": "docs/20260915_171525/depth/000342.png", + "bytes": 328375, + "sha256": "2162aa7645957283c9f9e022df78fcb7cea4cd4588be88c0c2a04267e8d2548f" + }, + { + "path": "docs/20260915_171525/depth/000343.png", + "bytes": 333077, + "sha256": "59c290c85b2a5f17c3cfa4b88f15c3daa1d774b530d9c8426c0412a74fe2dcc6" + }, + { + "path": "docs/20260915_171525/depth/000344.png", + "bytes": 332233, + "sha256": "df673135d2be0eda1a0e8c9bbba559076feed8ea0c4a44ec4f87f2834e7d0f3d" + }, + { + "path": "docs/20260915_171525/depth/000345.png", + "bytes": 328274, + "sha256": "2e2eef06a2f68f6451e3dd8e8e762059ddd5a5e2afda07299546da327c042b88" + }, + { + "path": "docs/20260915_171525/depth/000346.png", + "bytes": 330038, + "sha256": "bc7b5361b13831583cca62417e2d4897b483e22cdef71e72504fac5d0f3f0e7d" + }, + { + "path": "docs/20260915_171525/depth/000347.png", + "bytes": 332398, + "sha256": "8a8bb25eceb85f464a014b457ab37615588483255edfd4f98652e62ebe58bcfe" + }, + { + "path": "docs/20260915_171525/depth/000348.png", + "bytes": 322567, + "sha256": "e8422df1e3bd2c4eec94836ff8580efafa3db4baed27a39fe0dda8ba14edee39" + }, + { + "path": "docs/20260915_171525/depth/000349.png", + "bytes": 326033, + "sha256": "7333300cb76c455d821180acd2bae4a16f1678f185cc77ea71630ecec877ab49" + }, + { + "path": "docs/20260915_171525/depth/000350.png", + "bytes": 324664, + "sha256": "6c772539e3a25e7f20ca32f005197c5f6deb046c96294df2853d87ff0e9f9475" + }, + { + "path": "docs/20260915_171525/depth/000351.png", + "bytes": 330515, + "sha256": "47fd6c87904aad9d37b93d0a10794e5093ce841dd48b97546ee852aaa68d8749" + }, + { + "path": "docs/20260915_171525/intrinsics.json", + "bytes": 22949, + "sha256": "aa0eac9afe553a30e5f18024a1c2bc31811675acda4679b24222aede4ca1b64b" + } + ] } ] diff --git a/PORTABILITY_REPORT.json b/PORTABILITY_REPORT.json index 39c8dee..e59ee96 100644 --- a/PORTABILITY_REPORT.json +++ b/PORTABILITY_REPORT.json @@ -14,6 +14,146 @@ "line": 47, "kind": "machine path or fixed example identifier" }, + { + "path": "scripts/compare_red_hand_depth.py", + "line": 11, + "kind": "machine path or fixed example identifier" + }, + { + "path": "scripts/constrain_rgbd_hand.py", + "line": 13, + "kind": "machine path or fixed example identifier" + }, + { + "path": "scripts/constrain_rgbd_hand.py", + "line": 14, + "kind": "machine path or fixed example identifier" + }, + { + "path": "scripts/correct_dynhamr_rgbd.py", + "line": 11, + "kind": "machine path or fixed example identifier" + }, + { + "path": "scripts/correct_dynhamr_rgbd.py", + "line": 12, + "kind": "machine path or fixed example identifier" + }, + { + "path": "scripts/correct_dynhamr_rgbd.py", + "line": 13, + "kind": "machine path or fixed example identifier" + }, + { + "path": "scripts/correct_dynhamr_rgbd.py", + "line": 77, + "kind": "machine path or fixed example identifier" + }, + { + "path": "scripts/diagnose_dynhamr_rgbd.py", + "line": 13, + "kind": "machine path or fixed example identifier" + }, + { + "path": "scripts/diagnose_dynhamr_rgbd.py", + "line": 52, + "kind": "machine path or fixed example identifier" + }, + { + "path": "scripts/diagnose_yesterday_hand_depth.py", + "line": 1, + "kind": "machine path or fixed example identifier" + }, + { + "path": "scripts/diagnose_yesterday_hand_depth.py", + "line": 17, + "kind": "machine path or fixed example identifier" + }, + { + "path": "scripts/diagnose_yesterday_hand_depth.py", + "line": 18, + "kind": "machine path or fixed example identifier" + }, + { + "path": "scripts/diagnose_yesterday_hand_depth.py", + "line": 19, + "kind": "machine path or fixed example identifier" + }, + { + "path": "scripts/evaluate_yesterday_spider.py", + "line": 45, + "kind": "machine path or fixed example identifier" + }, + { + "path": "scripts/export_camera_delivery.py", + "line": 43, + "kind": "machine path or fixed example identifier" + }, + { + "path": "scripts/export_camera_delivery.py", + "line": 45, + "kind": "machine path or fixed example identifier" + }, + { + "path": "scripts/export_camera_delivery.py", + "line": 46, + "kind": "machine path or fixed example identifier" + }, + { + "path": "scripts/export_camera_delivery.py", + "line": 47, + "kind": "machine path or fixed example identifier" + }, + { + "path": "scripts/export_camera_delivery.py", + "line": 48, + "kind": "machine path or fixed example identifier" + }, + { + "path": "scripts/export_camera_delivery.py", + "line": 80, + "kind": "machine path or fixed example identifier" + }, + { + "path": "scripts/export_camera_delivery.py", + "line": 165, + "kind": "machine path or fixed example identifier" + }, + { + "path": "scripts/export_camera_delivery.py", + "line": 205, + "kind": "machine path or fixed example identifier" + }, + { + "path": "scripts/export_dynhamr_bimanual_rgbd.py", + "line": 4, + "kind": "machine path or fixed example identifier" + }, + { + "path": "scripts/export_dynhamr_bimanual_rgbd.py", + "line": 5, + "kind": "machine path or fixed example identifier" + }, + { + "path": "scripts/export_dynhamr_bimanual_rgbd.py", + "line": 6, + "kind": "machine path or fixed example identifier" + }, + { + "path": "scripts/export_dynhamr_bimanual_rgbd.py", + "line": 11, + "kind": "machine path or fixed example identifier" + }, + { + "path": "scripts/export_dynhamr_bimanual_rgbd.py", + "line": 16, + "kind": "machine path or fixed example identifier" + }, + { + "path": "scripts/export_dynhamr_bimanual_rgbd.py", + "line": 17, + "kind": "machine path or fixed example identifier" + }, { "path": "scripts/export_dynhamr_dex.py", "line": 15, @@ -39,6 +179,16 @@ "line": 110, "kind": "machine path or fixed example identifier" }, + { + "path": "scripts/fit_rgbd_objects.py", + "line": 12, + "kind": "machine path or fixed example identifier" + }, + { + "path": "scripts/foundationpose_env.sh", + "line": 14, + "kind": "machine path or fixed example identifier" + }, { "path": "scripts/l20_bottle_physics.py", "line": 13, @@ -66,7 +216,22 @@ }, { "path": "scripts/package_source_release.py", - "line": 60, + "line": 63, + "kind": "machine path or fixed example identifier" + }, + { + "path": "scripts/package_source_release.py", + "line": 115, + "kind": "machine path or fixed example identifier" + }, + { + "path": "scripts/package_source_release.py", + "line": 118, + "kind": "machine path or fixed example identifier" + }, + { + "path": "scripts/prepare_rgbd_20260915.py", + "line": 12, "kind": "machine path or fixed example identifier" }, { @@ -74,6 +239,106 @@ "line": 61, "kind": "machine path or fixed example identifier" }, + { + "path": "scripts/prepare_yesterday_spider.py", + "line": 10, + "kind": "machine path or fixed example identifier" + }, + { + "path": "scripts/prepare_yesterday_spider.py", + "line": 14, + "kind": "machine path or fixed example identifier" + }, + { + "path": "scripts/prepare_yesterday_spider.py", + "line": 15, + "kind": "machine path or fixed example identifier" + }, + { + "path": "scripts/refit_rgbd_objects_outline.py", + "line": 7, + "kind": "machine path or fixed example identifier" + }, + { + "path": "scripts/refit_rgbd_objects_outline.py", + "line": 8, + "kind": "machine path or fixed example identifier" + }, + { + "path": "scripts/refit_rgbd_objects_outline.py", + "line": 11, + "kind": "machine path or fixed example identifier" + }, + { + "path": "scripts/refit_rgbd_objects_outline.py", + "line": 20, + "kind": "machine path or fixed example identifier" + }, + { + "path": "scripts/refit_rgbd_objects_tracked.py", + "line": 7, + "kind": "machine path or fixed example identifier" + }, + { + "path": "scripts/refit_rgbd_objects_tracked.py", + "line": 8, + "kind": "machine path or fixed example identifier" + }, + { + "path": "scripts/refit_rgbd_objects_tracked.py", + "line": 11, + "kind": "machine path or fixed example identifier" + }, + { + "path": "scripts/refit_rgbd_objects_tracked.py", + "line": 39, + "kind": "machine path or fixed example identifier" + }, + { + "path": "scripts/register_hands_rgbd.py", + "line": 9, + "kind": "machine path or fixed example identifier" + }, + { + "path": "scripts/register_hands_rgbd.py", + "line": 22, + "kind": "machine path or fixed example identifier" + }, + { + "path": "scripts/register_hands_rgbd.py", + "line": 23, + "kind": "machine path or fixed example identifier" + }, + { + "path": "scripts/render_collision_fix.py", + "line": 13, + "kind": "machine path or fixed example identifier" + }, + { + "path": "scripts/render_corrected_dynhamr_rgbd.py", + "line": 3, + "kind": "machine path or fixed example identifier" + }, + { + "path": "scripts/render_corrected_dynhamr_rgbd.py", + "line": 4, + "kind": "machine path or fixed example identifier" + }, + { + "path": "scripts/render_corrected_dynhamr_rgbd.py", + "line": 10, + "kind": "machine path or fixed example identifier" + }, + { + "path": "scripts/render_corrected_dynhamr_rgbd.py", + "line": 11, + "kind": "machine path or fixed example identifier" + }, + { + "path": "scripts/render_foundationpose_red.py", + "line": 8, + "kind": "machine path or fixed example identifier" + }, { "path": "scripts/render_l20.py", "line": 13, @@ -99,6 +364,46 @@ "line": 15, "kind": "machine path or fixed example identifier" }, + { + "path": "scripts/render_object_alignment_overlay.py", + "line": 4, + "kind": "machine path or fixed example identifier" + }, + { + "path": "scripts/render_object_alignment_overlay.py", + "line": 6, + "kind": "machine path or fixed example identifier" + }, + { + "path": "scripts/render_red_depth_comparison.py", + "line": 21, + "kind": "machine path or fixed example identifier" + }, + { + "path": "scripts/render_rgbd_l20_objects.py", + "line": 11, + "kind": "machine path or fixed example identifier" + }, + { + "path": "scripts/render_rgbd_l20_objects.py", + "line": 13, + "kind": "machine path or fixed example identifier" + }, + { + "path": "scripts/report_collision_fix.py", + "line": 8, + "kind": "machine path or fixed example identifier" + }, + { + "path": "scripts/report_spider_dynamics_fix.py", + "line": 13, + "kind": "machine path or fixed example identifier" + }, + { + "path": "scripts/report_yesterday_spider.py", + "line": 10, + "kind": "machine path or fixed example identifier" + }, { "path": "scripts/resume_dynhamr_2047635068.py", "line": 41, @@ -134,6 +439,21 @@ "line": 19, "kind": "machine path or fixed example identifier" }, + { + "path": "scripts/run_dynhamr_rgbd_20260915.py", + "line": 10, + "kind": "machine path or fixed example identifier" + }, + { + "path": "scripts/run_dynhamr_rgbd_20260915.py", + "line": 12, + "kind": "machine path or fixed example identifier" + }, + { + "path": "scripts/run_dynhamr_rgbd_20260915.py", + "line": 32, + "kind": "machine path or fixed example identifier" + }, { "path": "scripts/run_handflow_15886123.py", "line": 9, @@ -144,11 +464,51 @@ "line": 23, "kind": "machine path or fixed example identifier" }, + { + "path": "scripts/run_red_handflow_comparison.py", + "line": 4, + "kind": "machine path or fixed example identifier" + }, + { + "path": "scripts/run_red_mirrored_handflow.py", + "line": 4, + "kind": "machine path or fixed example identifier" + }, + { + "path": "scripts/run_rgbd_20260915.py", + "line": 1, + "kind": "machine path or fixed example identifier" + }, + { + "path": "scripts/run_rgbd_20260915.py", + "line": 15, + "kind": "machine path or fixed example identifier" + }, + { + "path": "scripts/run_rgbd_20260915.py", + "line": 16, + "kind": "machine path or fixed example identifier" + }, { "path": "scripts/smooth_l20_offline.py", "line": 8, "kind": "machine path or fixed example identifier" }, + { + "path": "scripts/smooth_object_reference.py", + "line": 5, + "kind": "machine path or fixed example identifier" + }, + { + "path": "scripts/smooth_object_reference.py", + "line": 21, + "kind": "machine path or fixed example identifier" + }, + { + "path": "scripts/smooth_refitted_objects.py", + "line": 5, + "kind": "machine path or fixed example identifier" + }, { "path": "scripts/stabilize_l20_temporal.py", "line": 19, @@ -159,6 +519,31 @@ "line": 20, "kind": "machine path or fixed example identifier" }, + { + "path": "scripts/track_red_box_foundationpose.py", + "line": 26, + "kind": "machine path or fixed example identifier" + }, + { + "path": "scripts/track_yesterday_boxes.py", + "line": 10, + "kind": "machine path or fixed example identifier" + }, + { + "path": "scripts/track_yesterday_boxes.py", + "line": 11, + "kind": "machine path or fixed example identifier" + }, + { + "path": "scripts/unmirror_left_rgbd.py", + "line": 5, + "kind": "machine path or fixed example identifier" + }, + { + "path": "scripts/unmirror_left_rgbd.py", + "line": 13, + "kind": "machine path or fixed example identifier" + }, { "path": "scripts/verify_dex_video.py", "line": 6, @@ -169,6 +554,11 @@ "line": 6, "kind": "machine path or fixed example identifier" }, + { + "path": "scripts/verify_rgbd_bimanual.py", + "line": 7, + "kind": "machine path or fixed example identifier" + }, { "path": "third_party/hamer/hamer/configs/datasets_eval.yaml", "line": 4, @@ -433,5 +823,25 @@ "path": "third_party/spider/spider/simulators/maniptrans.py", "line": 1317, "kind": "machine path or fixed example identifier" + }, + { + "path": "third_party/FoundationPose/bundlesdf/run_nerf.py", + "line": 18, + "kind": "machine path or fixed example identifier" + }, + { + "path": "third_party/FoundationPose/docker/run_container.sh", + "line": 3, + "kind": "machine path or fixed example identifier" + }, + { + "path": "third_party/FoundationPose/estimater.py", + "line": 19, + "kind": "machine path or fixed example identifier" + }, + { + "path": "third_party/FoundationPose/estimater.py", + "line": 69, + "kind": "machine path or fixed example identifier" } ] diff --git a/README.md b/README.md index c9b4766..512f3db 100644 --- a/README.md +++ b/README.md @@ -1,15 +1,20 @@ # Hand Motion Pipeline:Dyn-HaMR · dex-retargeting · SPIDER -在 HandFlow 工作区中集成单目人手重建、L20 右手重定向、接触优化实验和 HDF5 轨迹导出。 +在 HandFlow 工作区中集成 RGB-D / 单目人手重建(Dyn-HaMR、HandFlow)、L20 左右手重定向、FoundationPose 物体跟踪、手物参考修复、SPIDER 接触优化实验和 HDF5 轨迹导出。 ```text -视频 → Dyn-HaMR / HandFlow + ViPE → 人手世界轨迹 - → dex-retargeting + L20 模型约束 → 机器人腕部与关节轨迹 - → SPIDER 接触引导实验 / 完整运动学回放 → HDF5 +D405 RGB-D 视频 → 静态背景里程计 → Dyn-HaMR 双手世界轨迹(也可 HandFlow / ViPE 单目分支) + → dex-retargeting + L20 左右手 URDF → 机器人腕部与关节轨迹 + → FoundationPose 物体 6D 跟踪 → 手物配准 → 碰撞修复 / 桌面约束 / 平滑投影 → 运动学参考 + → SPIDER 接触引导物理滚动(MuJoCo Warp)→ 审计 / 回放 / HDF5 ``` ## 从这里开始 +- **[最新流程(RGB-D 双手 → Dyn-HaMR → L20 重定向 → FoundationPose → 参考修复 → SPIDER)](docs/PIPELINE_LATEST.md)** ← 当前交付链路,按此执行 +- [安装依赖、环境与权重下载](docs/SETUP_AND_WEIGHTS.md) ← 部署先看这个 +- 克隆本仓库后:`git lfs pull && python3 scripts/large_files.py assemble`(大于 50 MB 的权重是分片存放的,见 SETUP §4.0) +- [环境变量配置模板](configs/project_env.example.sh) - [集成结构、环境、运行顺序与上传步骤](docs/INTEGRATED_PROJECT.md) - [重定向方法、验证与限制](docs/retargeting_workflow_and_lessons_zh.md) - [HDF5 原始接口要求](docs/HDF5_REQUIREMENTS.md):原文为左手;当前导出为明确标记的右手变体。 diff --git a/RELEASE_VALIDATION.json b/RELEASE_VALIDATION.json index 03f4c7d..685e655 100644 --- a/RELEASE_VALIDATION.json +++ b/RELEASE_VALIDATION.json @@ -1,10 +1,10 @@ { - "local_python_syntax_checked": 67, + "local_python_syntax_checked": 131, "shell_syntax": "PASS", - "source_files": 7000, - "bytes": 59113672, - "portability_findings": 87, - "excluded_files": 1440, + "source_files": 7166, + "bytes": 60649184, + "portability_findings": 169, + "excluded_files": 2789, "end_to_end_reconstruction_rerun": false, "remote_upload_performed": false } diff --git a/SHA256SUMS b/SHA256SUMS index c9b2704..d6159d5 100644 --- a/SHA256SUMS +++ b/SHA256SUMS @@ -1,17 +1,26 @@ -3908af8172c09292676e4251e92047bba6affe8b93b10678931abaddfb40d9c4 .gitignore -b57bb497baac2331336ed0fce0a88323a22170d807e2a6a3e93acc617fa1b586 EXCLUDED_FILES.json -c65bf89cc6efa1599cbc14d5a6a77c1ae9720b2a0858f8e73c14ccc16cd3f518 EXTERNAL_ASSETS.json +873737027b95bcdeef4817b8f7d4c32a058d8dee45a7edf90fc76f80ecbb7657 .gitignore +7ebd2fc278d77c6169155e2500dc7ca94c71c3c61728d2fc9d8f074927551a16 EXCLUDED_FILES.json +707506e2394bce2e12605a8f5035c8fdab043b4067e717987b615e3afb47b1ef EXTERNAL_ASSETS.json 3b94f458ee76f45c5bc05d9bc809ee6549f4abc1e289e20672e4094f06313dbe LICENSE -b70239228c5cbe4fe2d777965a1b255e09b54f26640a74e2470acd8def6f82e6 PORTABILITY_REPORT.json -21d1923ff692d12a7f2500846b9cd9b5d58eae5c91f051dbff92c3ae038affb8 README.md -c586facbab36165c37b4cb036e82470481e9b6155835715de9c2977207bec956 RELEASE_VALIDATION.json -bfea3bb258141bb73420d96dd05c9e7ccb5fe9155d696f024bb9de2feaa04ed9 UPSTREAM_SOURCES.json +f87632519750fe27a73c6ee63a6b75fa04f0b6928d5d6dfbdd02b7a7f4e607f7 PORTABILITY_REPORT.json +04c14678d19c1ab9a2128f05241fe9961a77957e30dd2e8d966d78ee1e4cff37 README.md +3a24bd92822f65a39a94cfc35fb747183f74acf9b559e9d5c8a3c5d14ff1b93f RELEASE_VALIDATION.json +616d982f0edba541db1f62aad6d520b2335829c6392d3204695836bc1e59a012 UPSTREAM_SOURCES.json +d2fcde6f355b0d5cb98439fedb88e300f15169c341cad89cf8ef27df0cc8ef8f configs/foundationpose_weights_manifest.json 14715c6825d04287291e06981f64303b769d206964ae9debf234fe2ef9a98183 configs/inference.yaml b5e4ecc2254e1908f7407e593e485300dda703d8ea6e1f243b1acad249733186 configs/model.yaml +9d2d05e1a8a72415d256ca744beb468597efc741008171cfe9c5173be9cd9000 configs/project_env.example.sh +aa0eac9afe553a30e5f18024a1c2bc31811675acda4679b24222aede4ca1b64b docs/20260915_171525/intrinsics.json +204823cb5b87c98e625e4c842aada4a85807e0cd44e3646ebf7ecac17425c169 docs/20260916_104026/intrinsics.json +42129bbc83c4c180b7a95f20e18113ed5fcb5cae4878a62d0dd882fca2fc75e1 docs/FOUNDATIONPOSE_SETUP.md d8c2768879ceea116540ff2cbdf8499fddad6884e64df58c62d3da15e79d2601 docs/HANDFLOW_UPSTREAM_README.md 98636beba160b543d8417b182c173d382029c53a5545320d00e009861b744869 docs/HDF5_REQUIREMENTS.md -a20a00e6066cb0ce04393d15df3274293d2dc4771837dc16a3f2a2c9606b8396 docs/INTEGRATED_PROJECT.md +8fa55652f588f0c7dbcbd38b2460bf90171eae7d1f3ff620a4a87a6d400fa475 docs/INTEGRATED_PROJECT.md +9ee1cafaad4f9610dc9b869144f30cfbb2a947997c27cd8fac4a1e022195444b docs/PIPELINE_LATEST.md +986421f1253f32f6b9d4d59204bc28c9fa1dd7d74a291cc0ea931c36d216795a docs/SETUP_AND_WEIGHTS.md 989281fd4b51192c0be950c374adb562ce6c0dcbb5e571720d20f588fdc76787 docs/retargeting_workflow_and_lessons_zh.md +d0565299f32944770e867a3c5714ed0e14468764efbd19a475a5bcae581c3ff5 docs/上半.stl +ec647027c0f3a0827eb7cf6fc343f4d8adc53f9ecb7df662c8fc5035ed30ad1c docs/下半.stl eab65b1dabd13e5ebc5e1cf90bdd2221d1854e7ccae585418f7310119e100292 model/__init__.py 7dd467f04165d5a89ba8e11e8e2c08e4fc493f0f5a09052de0d3350eea2a7df3 model/feature_extractors/__init__.py 65fbf3a3aeaf8fb36f9b875bc1694874cd562ec536ac71e598dd2454d7e6aab9 model/feature_extractors/condition_builder.py @@ -29,49 +38,118 @@ d9a9c64c40ea17bc1131fd94e309f708be02b629f57db587d1031be16f71636f model/flow_mat e8d1cfb7824dab7b173b81bd87bbc50c0eecf036a58de9f9919fd23b21bf9cbc model/pose_embedding.py e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 preprocessing/__init__.py 545b76ce9c91517c8825a036b8f7fab88c8cd09dc1d4ebeedff2aafd09364b92 preprocessing/hamer_crop_and_skeleton.py +0dbdc5b0b5e6222902a8e1a4dee36709c43ca3599cd5b60ad85fe210d2d2b0f5 requirements/pipeline_venv.txt 176ac5e797423af4c48d00516cc6fc335938a5905565d1bb57a4d2d6f5059d9d requirements/retarget.txt 26d3363149bfef66f3d35c479e31842c8dfdfd13cf1ccc909d17069a23fc75c9 requirements/retarget_observed.json +b5b7186c6d03bee37a4c2b7738f37150d91f478c83962020734b4188db41efc1 requirements/runtime_observed.json +45e7086637ef4b033dec04c1747387dc21c8fe4a544e080b3d8ec702a8870326 requirements/spider_installed_local.txt +edf81ff4975fb401202ed5407ea6237c8f6c4a0a7dd02184ee1f1a725beb72d1 requirements/spider_resolved.txt 8aeff3bc690acd9aa27413483887599d465306993ea60f6cb9597c95506a5e9a requirements.txt +77fe5e9712c4a6671fda5d0e3facd9ddbee2380d6d884c166913ed84acb88785 scripts/align_collision_floor.py +33fff827d0be3f7242255df859463375b2a4f11edc4063dbb5de9159ff37ca38 scripts/audit_collision_fix.py +8d0053cfa1de3c34090962c14aadd71339cb73267f87e116dacff64c5229c033 scripts/audit_collision_physics.py df0df48089c32d087c008d2e9ae561dab90fc4753253ae262ffaaf7569cd9799 scripts/audit_l20_jitter.py +46a55e4dc11478872eec54a582705e72dbb0174918a6e5fc94ce8f4d18c3c26c scripts/audit_red_depth_metrics.py +a33eee4cfa2064d971c008347b64045103630a20c8c6de177831769976cc6cdd scripts/audit_smooth_table_result.py +e3b858a7249f29e39758923485c7b9970824395c72af38da2c8bbf0d95c70aad scripts/audit_spider_dynamic_surfaces.py +923f7cb6ea62b7cc07163d062aeefda8e36fa1568b17e877eb545a6660c3c279 scripts/audit_yesterday_geometry.py +29c0e6c228e6dea0a38869d922312ed917e3709b40e3e8cd5500d21b12e885e5 scripts/build_box_collision_v2.py 7cc6785277d5d8edd10f8412f792ea09d22c01f6b02904c3f16373a151e9dc40 scripts/build_l20_full_replay.py +8f39db6ebb228ea67db6238db824c00e4f3e4aa0f508e14dff8542534dd1404d scripts/build_palm_collision_v2.py e12f5e37bb87c107779ca1e909bf26276148b1afd53d6246fda3a381deda8209 scripts/build_parametric_bottle.py +7e3422e3a0e79961002ba8d916cce4bed5fed9dd36eddb44a2242e05e2d06db6 scripts/check_spider_dynamics_replay.py +8f8069b9c26e386e54ee0f0fc15e64fc779e3baee32c11461529a9beb3d03090 scripts/check_spider_state_restore.py +263bdcf9634c0bfb2cea59b0d13ba73b8c8f3bd73bc8abb81e27507bc7b1bec5 scripts/compare_red_hand_depth.py +de28804aefe62b9199ba410869c643eab23966a67e9af519e183518e941ba58a scripts/configure_collision_physics.py +fe7b1a3e6d89ffc930218b5646a1abc10014f800f1dd51414a41957cda3c19a0 scripts/configure_spider_stable_contact.py +a874d2c88584b4d479e7a8a2be704b7ba9fb0cf81caf27dc3db31bbc3f8a0515 scripts/configure_spider_task.py +509f08f66851ff9f828b36f28cbece11c76921f594ac9f194d656a33b8a2e168 scripts/constrain_rgbd_hand.py +9a068ccc14111f1d707a53e1a57f23308c1077167aac6103604d1a3f765bf3e0 scripts/correct_dynhamr_rgbd.py +b3b56d92131afbd6d277893d525f7d20f87e9cbf2f3974304354e70c10efa700 scripts/decompose_yesterday_boxes.py 98465a8829aa3d2e3a24276c29518cf067e0ff3eeef875eb9d43bb0a0f76f66f scripts/demo.py +1f2267066a4556d3d1d1357fd5a5f477e9da4dc32fcfba0f3f569f9c492c3311 scripts/diagnose_dynhamr_rgbd.py +743e6642542ff876f87c2ee4ac5ef95514614dab03aac911afc86832804db692 scripts/diagnose_yesterday_hand_depth.py +7f0bee7bb08ea1296fca0e3b3631011188588018ab3cc10cafc913f33bc8a17c scripts/download_foundationpose_weights.py fbff07de02cda520b4c37843346167e625c3326c26bbf60aaaec38c3cdf73d08 scripts/evaluate_spider_l20.py +8655cfcdf7e7a2d4ef9a34c00ec74c00e282ef6e020e44ae1e5db046b2fb8080 scripts/evaluate_yesterday_spider.py +af24a21a248617ed9d9d9a2ae0cf79b3ac6ea05ff70bf5778154ab36f5d0ce54 scripts/export_camera_delivery.py +8bbeed8b9665a2dc616f9b821f5952a45ac6cb0a1c91dc86b7551d85ed481066 scripts/export_dynhamr_bimanual_rgbd.py b465844a0b9084e38062c48ba1afce988085a331c076b296716f6a9fa116e1ca scripts/export_dynhamr_dex.py 6c160caaf8e5ab42746284d99808470e8e25932ab0ff43e37f6d41de6f736143 scripts/export_handflow_dex.py bf1d95f8c8cae52debba49e90967b844b56e3a0a4990d5434e3364415b1a6125 scripts/export_l20_right_hdf5.py +b37abece30e8af630a0ed1a597d1093d0c281b08682555e10e08d75978367e99 scripts/fit_rgbd_objects.py +0938adcd65c5c76d01c96d4d49406b421748bd1bb47a80097fc6917a08f957bf scripts/fix_yesterday_hand_collision.py +d7ec9022b9bceee8383265d5de754abe775314afc0ab2097699e8c5cb72c221b scripts/foundationpose_env.sh +e75b99aecd07b68d3faa9aed2ba3806ac4bbfa0de60ec90e439ca234e6667158 scripts/interpolate_collision_reference.py c67f3786c0d154064a8c5d6af974a60bf25238b5aa078e8fba2e6bc95e3e348f scripts/l20_bottle_physics.py d6634364c3a149532b9dfa7055f0bfabe22f61c3428ae2b5713dec6e582125f1 scripts/l20_bottle_sequence.py -f27fdff03ba67ceb45c4a69578eda00bc66cd9747b90d42fb1e5bc2af2ecf05a scripts/l20_calibrated.py +ef95d1b95593540ad055f574130be5495f09dc4e5d919cd170ce8c5ad4e654ec scripts/l20_calibrated.py 15b7e11a315d170e40a313714675b1218e07c06c1582cbf013612574753321fe scripts/l20_consistent_optimizer.py e252d0dcb40de4f2032562a172a569b0af8d393f947c40682eef98c17b188eb6 scripts/l20_contact.py 2717e755fcd7de74ab40cbccd746a74264dd1d8a383d853c45adcdabb58ffa18 scripts/l20_kinematic_grasp_check.py 9e19684d33c591e00e1a5401c89c3d1458544112bd9fc02a1642c8c0d7900d3d scripts/l20_model_source.py -a008c8b4b555a77a120094a078fc59425fb555c565d24104844214885a51ba08 scripts/package_source_release.py +348a0aa4522f8b2f4b09cd3facb5507145bdcc469ed003c16899d81d4ff39132 scripts/package_source_release.py +34bca5d2c52a580f7b55ada435aa6971bf17501b1c5155d82685346939999ac4 scripts/play_compare.py 837400e8f78b380b9debc6bfee7fbf2ce16361cb6e354d6c00bb676708aef6af scripts/play_l20_full.py 9694a21fd1d0bfdc3a6de08b640a48218bf6f60deac8e9c7a82d823345a516b3 scripts/play_l20_motion.py +80562d33ea266e91de6a8371cd29227bb43929ba95f4a6df7cbdf7a305f21832 scripts/play_spider_result.py +8f9f67965de231b6b8ae6131f3ad71979672ec0c852eb60588eb06f4c23a5f52 scripts/prepare_rgbd_20260915.py 0825f6e1f54b2e8a18a8eb4a920f03854fc43ef661730e8eaac76175d8108045 scripts/prepare_spider_l20.py 17e40993a99e91fa14e7e40365fc497c2827aaee4d96a6f12d21f89733c6b231 scripts/prepare_vipe_cache.py +a1ee716f05683ae20860e79b9fc790bfa555319637143ff11abdf5a1ffcd22b8 scripts/prepare_yesterday_spider.py +a647bb7ae66e706b071b04a0d33db6cc5a3db711cf86e74483d66e38e688c6bd scripts/probe_collision_dynamics.py +3235a0a855759504db3e99559ae613d9b8f7b1a69378fbbf645d3ea07645676a scripts/probe_spider_gpu_contact.py +596a80c6c5e172cab40b3eed7086380e06f0bb014fd0583df311444099e0f2ba scripts/red_box_distance.py +9a7b7816f9a22d8afe1a56f10411daa4ae903c4a1da858a5ed4f3fbc5d645d81 scripts/refit_rgbd_objects_outline.py +79e51b5274dcaf543a1fc024d043ce36e45922ebf9faa0d9e769fdc558f4d603 scripts/refit_rgbd_objects_tracked.py +17a7fac470ad82274f452362eefcfa533cb6fede15d74d55b9dd539d158973d5 scripts/register_hands_rgbd.py +a9ab14e114acdc823c5feef966bb0578d197b83d6a06f356764d9d11d8b8d3d4 scripts/render_collision_fix.py +63af9a907cf82b65cd623fd327ae9fc59166410554c2a7c9808756ae0f5072f3 scripts/render_corrected_dynhamr_rgbd.py +baf33de4914c8537489f1999348fe452288cf7aae0b808bbe6eb49fec4fefc4e scripts/render_foundationpose_red.py 399c059b001daceff5cd1e71f37e7f6413862664ef1b038e26734cae905be5c5 scripts/render_l20.py 42cb059829f89d818f3b73a24536b7eea02f9850d2a7fcf686b5921c1623b459 scripts/render_l20_bottle.py 5362cd29bbf58db9053eb1705260c675fce37f644383709eabcd05225b73c863 scripts/render_l20_jitter_comparison.py 3127e1993c76cac3dbf108409a3724f06612786948eb075a8a07558bb4308000 scripts/render_l20_stabilization.py +de8c375699f1a9cdaf953c247257d77d677df3d3d9611d778bce34215dc13fe2 scripts/render_object_alignment_overlay.py +ac11d0f80f146092081a3767fc1e51b546f0c637c8cc92336b4cec38591a88af scripts/render_red_depth_comparison.py +3d068a78ef5ccbc526595867fe2c54a50cd29353601b3ce5c1b1276e694b96d5 scripts/render_rgbd_l20_objects.py 0d6bd3c8933b1c1c2de192aa6d10025a763525f73fe2e907ec5b86b7dfeed1a5 scripts/repair_l20_bottle.py +0629762e56be19aec0022f3edaf9f43a4fe3f61eb973ebff6ffb3b3c172058b6 scripts/repair_red_box_topology.py 0fed31f73fd7a82f15eca9252bf4f4f8aa5a0e7d84a09741512aec7a6d7b81da scripts/replay_spider_l20_gpu.py +92418d25ad828ae3ad6e612998fb2bf745d23498cefd7c7589583ff4fd012b5f scripts/replay_yesterday_spider_gpu.py +dd9f88792397a954cdfd4315eee84913bf15159483a0171dde6e5e8d38fbda90 scripts/report_collision_fix.py +e85c24af32cfb02b347bfb5a1fbfb2bc0e7306e9e9b754837c36d60204c1df70 scripts/report_spider_dynamics_fix.py +f98ae1a94334b56e590c8d89658238ea72417d2a5030ffd905f21303bcdaca0e scripts/report_yesterday_spider.py cfe8274247ea3cbcab63951f9ee2cfed38e60766f69a207ed4fb8448716df857 scripts/requirements-vipe-compat.txt 1e8805996465d5765a4a297dee056d6ae2c2ef5553b0c22845775768d0773994 scripts/resume_dynhamr_2047635068.py -69ffaa3edf31c56b43780c6c5a0ce2a9a11dc4a62b8d945f0b67902465e87249 scripts/retarget_l20_video.py +01cb15e8355d6c2eb362c5ce5d56b34710afbc712a06072c7436c14b2646ea54 scripts/retarget_l20_video.py +3c1dc6765f285a2cbfd81fb2de8b8485ecc1ea5cb775e25cca6470d2e3c197df scripts/rl_track_reference.py 1706fb0e5dec9df754c3f3227d0d85b976de3d238e4d25c9b68b2346f99cafa0 scripts/run_dex_2047635068.sh e8269bbd0b7754812ef1d73b2c955fe9331c737385f27b7fecaff07e0d15ab41 scripts/run_dex_l20.py 043f43019ff78cccc316824f435ee4060b3ac90e2821fd963077a22b936c2ada scripts/run_dex_leap.py +b3d7f1978780f269865fb72324326dd49686e0363ff8510c3d10fdcf0188a270 scripts/run_dynhamr_rgbd_20260915.py 59cc88ded2acf3fd8b9bd6ecb9c10897a068342758d636b210a5a04661371b48 scripts/run_handflow_15886123.py +c52590b2acd3ecee248aa14f285880f4482daee75c8d0fda298c9f46feb8200e scripts/run_red_handflow_comparison.py +12f6e2b93975c0398b2be86c0d3ce68ffe8532390f95210ab643bb7a14478c96 scripts/run_red_mirrored_handflow.py +4ae360c9a66bfd0a20a152a3c227531a6f4bd8f9ff066a06f9804947135ba524 scripts/run_rgbd_20260915.py 69d0929da639d008ef9a1be515fb2de322fc0ee5546f1ce7077aa1574128f291 scripts/run_spider.sh 14e00e9f72295e1251b3d6bfefdfb807169dd30274be6de490ecf09f8d028867 scripts/run_spider_l20_contact.py +0ccce522536bb0e71f658c1d874607cf4bf4911595eef057008a69edd6dd7f2a scripts/run_yesterday_spider.py f5e25d93e5e0a703f69a4db02bb745eb5caf20663f9df5ee050f40d1f0bbbaac scripts/smooth_l20_offline.py -43cc11db3bc06b07f8e1164ec4b4435b9275e9fe7f87cc1cf0532dafcbee592a scripts/stabilize_l20_temporal.py +15c3bb09dd9d6e962f98aee7ba4784a59fce48deffa2b142308d947db1b628ea scripts/smooth_object_reference.py +0d51c1cc2f0a212737b713ad45349cb75de05a75b1f919a432b82467483e6a6b scripts/smooth_project_reference.py +a5594ec93c7f2154af44d46c0e8a29990e710a1169aa2dd2e86c83bf6a893314 scripts/smooth_refitted_objects.py +1678676bc174d93b83c7278ee01eed97093728cb55eb9d5308589f9aa23588e6 scripts/smooth_table_constrained.py +22aadb33132c844c3bc4264303a1183df2e6336bd9d9b39cd6dc99ecc5dcdc55 scripts/stabilize_l20_temporal.py +30a5bfa30e0119052fd4e88863789ed5635624c897037815686c09eef0f12ca6 scripts/track_red_box_foundationpose.py +bacdae6ba8a4fa60612c8624d3dccd18175d6d006c3879e536a6d7eba5df2112 scripts/track_yesterday_boxes.py +fcf07dd63502cb6a45f93c130d476c43b35f796c625659242e977d70d45e20e9 scripts/unmirror_left_rgbd.py 8fb458b018a2ff1e5078934af09ef344c7d0859bd4c9c94b5054d7fa35fb6db8 scripts/verify_dex_video.py +3231c2cbf71e4e583c745083255c920c3ff299907e9fd421e28f0126e0a463bd scripts/verify_fix_interpolation.py +8507f5ba9101de8457780cb684ff2e890a38cf287555e63096c440dd7921c9e6 scripts/verify_foundationpose_setup.py 032abf435dcd0b96c1b15a3d0277329ed1037ae28cf970716af84a74d93a4497 scripts/verify_l20_bottle.py dcb976ca19f407fec64cf6c516046a6aa6352573e555d481da45c84c24bff027 scripts/verify_l20_full_replay.py +d20e3453c3e3b00ef3672426e0c3bf1b26bb8f79de99c74aa52dbfcf7b581ac6 scripts/verify_rgbd_bimanual.py 7c233690c90a2063a761083ea5bb1564c9a2a91eaab17be84c004a4e7a82f0da scripts/verify_spider_install.py a5301cfa4a1689a2b93ae712f022827630e6c4f88e295ffee51232bc85422cb5 scripts/verify_video_retarget.py 7996490fdf261b8ce7e916d77e878a60d88a45705cb77f6c5801068c58743414 setup_env.sh @@ -155,8 +233,8 @@ abab41c939bf736853870c84e4b65ecf4a304dec9be52323c86050fb44d9d010 third_party/Dy 839619b1e4cef259e659e822af020ff6a1304da1768d38a46f61306f7d88ab29 third_party/Dyn-HaMR/dyn-hamr/vis/fig_specs.py 521033d1a22f112e589a93b867786849ff7ae3a18e7f9430f6ef9d035588a712 third_party/Dyn-HaMR/dyn-hamr/vis/output.py 5643269902c71627403e422acfdc932eb9e4c5a733b0b9cb32a7535665f1cba6 third_party/Dyn-HaMR/dyn-hamr/vis/render.py -074269f6475b128357de0652a4cf8a99a08d71a5d940afc89fc952529036d762 third_party/Dyn-HaMR/dyn-hamr/vis/tools.py -99b1eb2852eef5018f169fb1d5d759acf50557b38b985330321f51002a255193 third_party/Dyn-HaMR/dyn-hamr/vis/viewer.py +c3c6c7a4ef2ea8a4a99bb9a92811eba9af4f7d11407391cbd191cf94e2734b88 third_party/Dyn-HaMR/dyn-hamr/vis/tools.py +85869a812e2b8a7885f8a3de176fac1ce6f4d13f20d9be00b311898ac6313139 third_party/Dyn-HaMR/dyn-hamr/vis/viewer.py 48debdbd133d80faa5d4adbe92d6a86780c9aebaf87c4a4d3894d0c75122985a third_party/Dyn-HaMR/requirements.txt a460b0aab572c9c8e03cda7c009dc7679eae507d61d5b95086c17eb03ecdef2b third_party/Dyn-HaMR/scripts/convert_videos_to_gif.py 1f8545a9db2dc863737482b871bf2ebd1befa9c660783d02fe07c7471d7bc1c3 third_party/Dyn-HaMR/scripts/install_conda.sh @@ -5072,6 +5150,49 @@ ebab16d0301ffaa0dc8611482965d04e6dfd01f55c293fcfc8332b729c362729 third_party/Dy 459aae68c2e7d76cc84ff75e0b9643680dee6b1f82261901f5c39809e6e981a5 third_party/Dyn-HaMR/third-party/vipe/vipe/utils/morph.py d786d4fdc5aac33a6a6a7ce8fa26629795fa13d6510051279d55d5b4c7e75fed third_party/Dyn-HaMR/third-party/vipe/vipe/utils/viser.py e71048085605bef6d2a5ec6f92e4dfb4eeffc0f2d57977a94c8b5e1c7d9cc6e9 third_party/Dyn-HaMR/third-party/vipe/vipe/utils/visualization.py +0de51abc6b73e24724858239efc42cfeada5d9e7c911cf7eff077bd9a23dec62 third_party/FoundationPose/LICENSE +ab77518df7a6c41bef82cb32579d06ccff5d435ee5ad716a8c1fef0c72f05ad4 third_party/FoundationPose/Utils.py +9bec5b73299871c8cd9fba5eec07b97d49f98a159d36b10ea7eac7ee821f414b third_party/FoundationPose/build_all.sh +28cd230a4599e59764144d850e3fd502349a2deab4c8e8407367f67fa0e309d2 third_party/FoundationPose/build_all_conda.sh +2474a9d31111b84fdf4c40fc6dfe31a7c2850d5029b00cdf0ad67b1e31cca8ec third_party/FoundationPose/bundlesdf/config_linemod.yml +312b37d4fedfde999fb130894b06e5d7149fabfeb808eaf18e0e583cdce72e3b third_party/FoundationPose/bundlesdf/config_ycbv.yml +e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 third_party/FoundationPose/bundlesdf/mycuda/__init__.py +911a33242afbbc31cd6423943e552eafc5228f22101c9dd639b7c17176b44bc9 third_party/FoundationPose/bundlesdf/mycuda/bindings.cpp +8ac9153b22e5526e7252f36e7e27849dd632da903e01c1d8a3610b1c0922259d third_party/FoundationPose/bundlesdf/mycuda/common.cu +d872ffb94c58e611f83bb272c888a79a9ffa577b90d886fe580bede093e53858 third_party/FoundationPose/bundlesdf/mycuda/common.h +2f8cadb97ceaa0e0202d918c606afc7185384f7dcae7f24e09577d1abdc3a347 third_party/FoundationPose/bundlesdf/mycuda/setup.py +2767ebd056b9458a9b1d6952f08cc4008e8565e68a88c02c8b6a994ad435f6a9 third_party/FoundationPose/bundlesdf/mycuda/torch_ngp_grid_encoder/LICENSE +5936432e36d7a366088d1804d84062c08cb80ffe97e880441f9d1462f56a7fc6 third_party/FoundationPose/bundlesdf/mycuda/torch_ngp_grid_encoder/bindings.cpp +0eb89d4b59d65d29b3296b0fbfc6ac179ecde7e660e38ec90c5b6afec5b13209 third_party/FoundationPose/bundlesdf/mycuda/torch_ngp_grid_encoder/grid.py +24b83e080d88970a8f241d634cf800ae9d284d25521a12c1f34a98be9c2ea214 third_party/FoundationPose/bundlesdf/mycuda/torch_ngp_grid_encoder/gridencoder.cu +4668e78b479d03e4a8862edadbf7005b46c4da082adc39e2685d36b97c2de0ef third_party/FoundationPose/bundlesdf/mycuda/torch_ngp_grid_encoder/gridencoder.h +7478760cd5469b3b4579f9e8cb880a66d98ee9e9a2d41873b7f8f045a8d6378c third_party/FoundationPose/bundlesdf/nerf_helpers.py +7aeafd1cd0641bb7e0949043c21248bfa84830c486b5f6c89b5674b3c6e0fc6e third_party/FoundationPose/bundlesdf/nerf_runner.py +f9866dcbef0d31afca227ddf034cf6666a516c04b61df9790d9a8dc60b7ec853 third_party/FoundationPose/bundlesdf/run_nerf.py +a63668025a6fda90009d369b48444de1649d6cbbee66b754576ef84600f01a16 third_party/FoundationPose/bundlesdf/tool.py +b40240a60d44000588e02d1585ce1d78a1d84aa14e5f50d5dd40c62fd373e8dc third_party/FoundationPose/check_env.py +c32f24c0f051a45a2683521f52fd848e9a0064304c7f1cf8b56104be8a0b4fd4 third_party/FoundationPose/datareader.py +d8f42dbef0b121e2f5982aa6e62e6616bcc7b5afb8121a66b37c31f3ebc7e1f7 third_party/FoundationPose/docker/run_container.sh +4a4617b6477f31ed2305cad2d7018e305c18c66a29d5fbd80f9a29cb3f789303 third_party/FoundationPose/environment.yml +73a40474d11ab9d9dc873e101b5eef370c0bd78f2874ab1b5148dac402496ad0 third_party/FoundationPose/estimater.py +c582f46c65f9787d7c7d4376f64900e10c87fb4d569f9eca55f05daeec395484 third_party/FoundationPose/learning/datasets/h5_dataset.py +27c5312108718611d7d82bb601e58d59b2c4f0b93b75ab02bee652e810a3e60e third_party/FoundationPose/learning/datasets/pose_dataset.py +29a4875a35150c662a3604e88a7f102d2fa4de9d026a8ee07151e09e788a2e26 third_party/FoundationPose/learning/models/network_modules.py +c024375d544352fbcb94698d792ec6fd017008ca5d093e71e747727ea59db709 third_party/FoundationPose/learning/models/refine_network.py +6cea9a7683def72b92e8c8328884e48fb6ca057ff480efab6d1328686b3ce926 third_party/FoundationPose/learning/models/score_network.py +a210f46868ab77631fdb264043356a435d06b1294929b304b0922e23c839d64a third_party/FoundationPose/learning/training/predict_pose_refine.py +97f2f6f74fbe60be6a630d515db5142d78045f1150adb26dca56d93cffd3bbde third_party/FoundationPose/learning/training/predict_score.py +da0e1265e9b4c35624b599029050631d62fb201ad6c838e3b4d26af99adc27b2 third_party/FoundationPose/learning/training/training_config.py +53aee630d869187ba47addf86142e211361774096d2543e4017460937884fb8f third_party/FoundationPose/mycpp/CMakeLists.txt +c5a9d3af06625c3bb966fcb514a71787964224aa546cfbbdaa7d5a71b303562a third_party/FoundationPose/mycpp/include/Utils.h +466878738286fd787038c28d80ea801ef4e6756d5e38e551b456b6f121a17d17 third_party/FoundationPose/mycpp/src/Utils.cpp +8567ae667cfe7657414effdd9995a76680052b865b09a6d4e37f2034f3fe3730 third_party/FoundationPose/mycpp/src/app/pybind_api.cpp +b219ee8decaff6ce6944cefe07926bab0a51e912c10421f7db0765b866fcf647 third_party/FoundationPose/offscreen_renderer.py +77dff29225aa2e53b48b7444afa5e25a1573f22a1eef497c61da212511dc086f third_party/FoundationPose/readme.md +943c5ef618210ce2d95aa8123a62f550d4a0a0080c6a32d5d542f1c0a291fa87 third_party/FoundationPose/requirements.txt +ff7fa1caee76e36ecc3a5ab6abe86f20ad96e3a98dda42be6d41666bcdc7c31b third_party/FoundationPose/run_demo.py +9f5f7648314cdd0df5581e441455a3c25f083c4a90bf2b45a4c08b1e3fa2259f third_party/FoundationPose/run_linemod.py +e7aacd1433c43678d7627edc9853f46780376167c1f3e0f02f22534e6ae3653b third_party/FoundationPose/run_ycb_video.py 04b9984691905594ec1757ec97f4cd939d87bea33f9aefc4bf9841b835392757 third_party/hamer/LICENSE.md 8c794047f7d965650799e13bf9337edc47e1cfbce61fbf337cdca11bcb91b18c third_party/hamer/README.md d2de2bdba9e0f78e020ec73cad41b5d95b9b824b00693e4c159e310c48cb5559 third_party/hamer/demo.py @@ -6547,6 +6668,51 @@ d77d2d3191ce9ead4746ec96950fd4e70f4366f143f940e1f51c06d5a5dd3720 third_party/ha ce164f71e951e6a423ab4de7ae67a0baab18a4e8e96d6442f21f7332f46ba4cf third_party/hamer/third-party/ViTPose/tools/webcam/webcam_apis/webcam_runner.py abd988f8ef00739213c548efe27e08da22e72d9de4aee39da24386b2e83d3e12 third_party/hamer/train.py f2bd4bcb167c7e4dfd9cca708c552372097d61afa881d9abc1a8b7c358a1710d third_party/hamer/vitpose_model.py +1e92afb622930d0d474296106734de5928d3836ac2c9ae3bd0d078bfbd9c83f6 third_party/nvdiffrast/Dockerfile +ce20436c2730611757d3c7ed2b4d9bef8315a8a0af0540ff8baa5dce7a94f5da third_party/nvdiffrast/LICENSE.txt +cd50f73d16a653c13677382328df6674357ad387811ee6356ee36cbc86880507 third_party/nvdiffrast/MANIFEST.in +2c53380264d463e2866c04d507404abd9060c5cb9aa02478690f132796537a87 third_party/nvdiffrast/README.md +820afbe9bb181f6e0f748e95881c753a0780f67e44c8da9239c81e43031956e2 third_party/nvdiffrast/csrc/common/antialias.cu +bb2566dc30b50abd8c604d6f8bc1a3e2a6fa2f9b3cb46a048972ba0978c24adc third_party/nvdiffrast/csrc/common/antialias.h +853052945a2b22272f09b305c38dda3d0e636c2a268f2d2f76f036be5dbfe47e third_party/nvdiffrast/csrc/common/common.cpp +9f426db8dfe79e62b2f6c7ca26068f56ad3a6d2120c288be494b9c1e2cee4a83 third_party/nvdiffrast/csrc/common/common.h +98a4c7e60b110b58f6875404a3ee6f36b9f6fabf17a0e46c11ae68cf74b58a67 third_party/nvdiffrast/csrc/common/cudaraster/CudaRaster.hpp +024b53503530a9cb46c8c5789224168526ff63e40df71a974cce3959fbb520e6 third_party/nvdiffrast/csrc/common/cudaraster/impl/Buffer.cpp +26dad7ee75b7b65959132ef6c148f608f775a07f437a5fdcf365f3acecda58e9 third_party/nvdiffrast/csrc/common/cudaraster/impl/Buffer.hpp +9b5bc875062479d6942df4af75d2f30f730dbd7e5a3758e5d2a3d407acd172e4 third_party/nvdiffrast/csrc/common/cudaraster/impl/Constants.hpp +550430abc3dc842a53507e4701dc30065316809318d187bf9405f54f784067b0 third_party/nvdiffrast/csrc/common/cudaraster/impl/CudaRaster.cpp +de4bc5732071af6df2206be091107d3992f4114ac0b120b7957d8ead0e574767 third_party/nvdiffrast/csrc/common/cudaraster/impl/Defs.hpp +ad3552438fbbdcb7797646da48bf3367c50227d4cb1d4a614b7ddaa98c10fadb third_party/nvdiffrast/csrc/common/cudaraster/impl/PrivateDefs.hpp +38b85912f1139f684f45b5f717c4e2404ba55c6ce4498574fcdd9a8d23176cab third_party/nvdiffrast/csrc/common/cudaraster/impl/RasterImpl.cpp +1d2554a62849b98038e3601652c552d6631e0108cf8aeacff5639877b9139117 third_party/nvdiffrast/csrc/common/cudaraster/impl/RasterImpl.hpp +b067421e0f742ab848a004bb098f7064935e5ed7f440222700579b2b777d62e1 third_party/nvdiffrast/csrc/common/cudaraster/impl/RasterImpl_kernel.cu +69beed905070b631773a8eb69658bd0c4a516919b04b42310eef4278393aec9c third_party/nvdiffrast/csrc/common/framework.h +a2ff0f402a0aeaa82346e28088d3b020910932625d4f0212026e25d2da723e41 third_party/nvdiffrast/csrc/common/interpolate.cu +4da693612c29f8ec70987df8b70da76e8f0980263ea647b5ff73dd5cf3ccf2e2 third_party/nvdiffrast/csrc/common/interpolate.h +286e85b2a0c59342a22e0f6e53d612418bc5c7f1d2c679255171342f335c690c third_party/nvdiffrast/csrc/common/rasterize.cu +70542df84dd30984bbadd78beb4715c7593b3b630b980dafa7430e324aaa47f9 third_party/nvdiffrast/csrc/common/rasterize.h +1368a918fd8b55946ed9f503b41f928117a278db8814487e3aaa9deba0e3e6c0 third_party/nvdiffrast/csrc/common/texture.cpp +c9c4404c85724d3b9839ecdcfff16c71b3ff67fe1e3eec4bd166f50c274eee5d third_party/nvdiffrast/csrc/common/texture.h +f54788e2df76490f4bfbc74a129d00ac31d6d954859be18d193b12a712d363ff third_party/nvdiffrast/csrc/common/texture_kernel.cu +0155510bb5d3c22852aac910f448cb0070e9eb24fdd8f8f173bad9bcc5cb3be9 third_party/nvdiffrast/csrc/torch/torch_antialias.cpp +448ddf58b557d99a28a87382cab0698dfcfc77e7d83dfe3916ee30968ca1dd9e third_party/nvdiffrast/csrc/torch/torch_bindings.cpp +6111f5631d57678c2d7ee1927ee299a977678afd9f1b1fcb76ac7dcb40e130e4 third_party/nvdiffrast/csrc/torch/torch_interpolate.cpp +1bb4b1f5502cb0b827a93e152c02fd1c22aad3cb05748ecb0e73d8d813106278 third_party/nvdiffrast/csrc/torch/torch_rasterize.cpp +146526d5c8639ac19355082893d5916607bdddbf47f88c4c28ef4d4a237846c3 third_party/nvdiffrast/csrc/torch/torch_texture.cpp +37ed6a10f724dbfb17d6945d22ab171d073bd1ae83d3378a2a52b1c47c750058 third_party/nvdiffrast/csrc/torch/torch_types.h +2e20d0bb443cc934fc03a2e4f53aa1e763a6f7c9224b84f3c8ae1123a0d78739 third_party/nvdiffrast/docs/index.html +1b569050844b7deacdf78ba4fea254ae74e35a4c062f812f08a4c7453f53ac19 third_party/nvdiffrast/nvdiffrast/__init__.py +61f7b47e6402c5db311a546dd02bc672544a1db5e0e8dcd81faaed7d0729bf36 third_party/nvdiffrast/nvdiffrast/torch/__init__.py +135f7b64bd8eb7698491b8224cdb737faf7672b558e6c5c841e596401424cabf third_party/nvdiffrast/nvdiffrast/torch/ops.py +485c87bc1f0d8f58cf2d76f958bd88f646a1a666fa6e2215242be76db700cc66 third_party/nvdiffrast/pyproject.toml +5b24e04c152f4dc1e8109adfecd87b484de168b70b8800ae90f2488b4f1f8fb1 third_party/nvdiffrast/samples/data/NOTICE.txt +79883d6111248c371cc21a8d2960834d0197546d2c9335a402c1e3ef131f2807 third_party/nvdiffrast/samples/torch/cube.py +2843227d3c313eda2c72472fe4d5deb4e7afdcbb2572bdf844175922d7ad031d third_party/nvdiffrast/samples/torch/earth.py +77c67b32d99c34b8aea1ba47ba6025e1ca5fcc49b592495b28a752f26819ae3b third_party/nvdiffrast/samples/torch/envphong.py +894df74752972a36518688d276aca037d5cd228dbfcd1fc938a7760f04871b2d third_party/nvdiffrast/samples/torch/pose.py +b6744a6a76a9f5d5700240b7185cac7ba7b1e7aefc664226c979934397375af6 third_party/nvdiffrast/samples/torch/triangle.py +7b17c0eabed8b029ae8646dda0f8b18dd78ebe1096c6841055fccebead3b099d third_party/nvdiffrast/samples/torch/util.py +12d02cbc23dc7496f94f62edf0136996303980e3ccd7884f2193d74eecf8091d third_party/nvdiffrast/setup.py e7a8f99a0db1fd4d800a28a1bbde315913f5633c4502a128753dc9e026c9dd91 third_party/spider/.github/workflows/deploy-docs.yml 2f8dd1e972a28624d016699b3df32bc2dd73399870074ce9b7759b65cd2d8238 third_party/spider/.vscode/extensions.json a945d05d8830cd825c8263c7868d6495141c0dddaa02e303668ad007996251a1 third_party/spider/.vscode/launch.json @@ -6598,7 +6764,7 @@ f067a12d6a940a271cb61cb2958a5eefb54348b36a0a746766543bef6765c975 third_party/sp d5a41860851c1e7d25caba3761875b757c7899608926983c1ccf7a02a990f401 third_party/spider/examples/run_hdmi.py b53d3a90e71e98035c8361367d53159b64b742fec3a985a98e98a019630bacc7 third_party/spider/examples/run_hdmi.sh c40d709d373b9e555b313841bf4880ac469b6a2140d383776ae8fc27894248d9 third_party/spider/examples/run_maniptrans.py -e3d053406cd51232e3293136ac266498f0e2c77e9e054f379468cd9f4f6b99f6 third_party/spider/examples/run_mjwp.py +d65050da9478deeb9d98c0a608872d93784702159e96cdbcbf8c78aee308a141 third_party/spider/examples/run_mjwp.py 56584869816735d774693269e4c094300ac8b0620f6bc17f049652d5fbd4ebaf third_party/spider/examples/run_mjwp_fast.py 5e67cca2e054c4d2e7a95453542b25e2e203c30d4cc23495baa2eeaf6d738ddb third_party/spider/examples/tutorial.ipynb e24eb7ea68256a56f6d7f7723ff2fae6626957e1ce2cd95ef687f1d6b916ae84 third_party/spider/pyproject.toml @@ -6675,7 +6841,7 @@ c6f3bc70b31da3e65b4d93255e3bbdec333ed989e91d42cff665dc6ab5262e41 third_party/sp e28d0f45bbfb8bd89ed378547c75e9c3a6731b628b2938e6106d0ee71b48dc33 third_party/spider/spider/math.py 9a91b818e261d148accf7b4edb3c9326709e37d109818aa9e57fa5d6b8a1b935 third_party/spider/spider/mujoco_utils.py 2d3199ff3c1c2e94d42a94308cb11da2ca23fbb49d3f23d6d37fb9fb494630ad third_party/spider/spider/optimizers/__init__.py -d4df207522e0ac5656d44763dfcb91982d01a3559da783b42c272bee9ee22940 third_party/spider/spider/optimizers/sampling.py +ffd2536751c466b4f6c2cbf8eb5b9236a393ea1f627e0ba4d40a2ef90848e452 third_party/spider/spider/optimizers/sampling.py b76a8e277daf5a75c92ed9cc146579a7885d209bb4c554a47a99c375fa77e77e third_party/spider/spider/optimizers/sampling_fast.py 924ae2fbd559bf964989ed1cadce64f4962f03f4dba68e1ca142d3d97662ec08 third_party/spider/spider/postprocess/__init__.py 26aa4cb33ced809b07813f27cdbab231ed0b3dd240b5322ef3ca228d095766a3 third_party/spider/spider/postprocess/add_auc.py @@ -6713,7 +6879,7 @@ fdb97e69024f2acdf902896d82a7378720eace998ba6edd042c5266b6719e80f third_party/sp af889f40ae71b2cdd5a8c705fdc77f133c1bc115f753a3e2f0d58490ea503238 third_party/spider/spider/simulators/isaac.py d2fa891db989182dc4f83f1bf73694610d59341d5d6894bcba44562fe00bdbff third_party/spider/spider/simulators/maniptrans.py 8af27b1e6b876f8ede9d657840dea63f65e83816a93da55bade476e41fe46977 third_party/spider/spider/simulators/maniptrans_test.py -b414105e5fd276e92ea347a69b529cd21a65a59a198cb2b40406ebdf7644ed66 third_party/spider/spider/simulators/mjwp.py +9cf471565802d55500bc8990c1e2117e963169b6d0242bf38df1c28d0ccbcc81 third_party/spider/spider/simulators/mjwp.py 08e5dbe18ea495625aeabee6351cc2a81fb458da6a85fa11638bb1b16461b2a8 third_party/spider/spider/simulators/mjwp_eq.py 9746ccde567081321d0d549e471c6445ca7669fdbfa0b460988e744fdb69a1c1 third_party/spider/spider/simulators/mjwp_test.py c76e1502bb50a2280eac7e673fe3891444a348a20860bbf79d12f35a2c28d0eb third_party/spider/spider/tasks/__init__.py diff --git a/UPSTREAM_SOURCES.json b/UPSTREAM_SOURCES.json index 8a1c0fa..266473c 100644 --- a/UPSTREAM_SOURCES.json +++ b/UPSTREAM_SOURCES.json @@ -30,7 +30,7 @@ "path": "third_party/Dyn-HaMR", "commit": "fa9cd7412c205fd15ee4139c8caacf79bf6167e6", "origin": "https://github.com/ZhengdiYu/Dyn-HaMR.git", - "local_changes": "M dyn-hamr/HMP/fitting.py\n M dyn-hamr/data/vidproc.py\n M dyn-hamr/optim/base_scene.py\n M dyn-hamr/run_opt.py\n?? dyn-hamr/HMP/windowed.py\n?? third-party/Hand-BMC-pytorch-main/", + "local_changes": "M dyn-hamr/HMP/fitting.py\n M dyn-hamr/data/vidproc.py\n M dyn-hamr/optim/base_scene.py\n M dyn-hamr/run_opt.py\n M dyn-hamr/vis/tools.py\n M dyn-hamr/vis/viewer.py\n?? dyn-hamr/HMP/windowed.py\n?? third-party/Hand-BMC-pytorch-main/", "snapshot": "working tree content; tracked files plus source additions" }, { @@ -65,7 +65,21 @@ "path": "third_party/spider", "commit": "71238456bf97a7eeb3d0471aa31974e2d404d4ae", "origin": "https://github.com/facebookresearch/spider.git", + "local_changes": "M examples/run_mjwp.py\n M spider/optimizers/sampling.py\n M spider/simulators/mjwp.py", + "snapshot": "working tree content; tracked files plus source additions" + }, + { + "path": "third_party/FoundationPose", + "commit": "a1b694b83e633c2cb6115b9063d940a687759392", + "origin": "https://github.com/NVlabs/FoundationPose.git", "local_changes": "", "snapshot": "working tree content; tracked files plus source additions" + }, + { + "path": "third_party/nvdiffrast", + "commit": "253ac4fcea7de5f396371124af597e6cc957bfae", + "origin": "https://github.com/NVlabs/nvdiffrast.git", + "local_changes": "?? build/\n?? nvdiffrast.egg-info/", + "snapshot": "working tree content; tracked files plus source additions" } ] diff --git a/configs/foundationpose_weights_manifest.json b/configs/foundationpose_weights_manifest.json new file mode 100644 index 0000000..a226022 --- /dev/null +++ b/configs/foundationpose_weights_manifest.json @@ -0,0 +1,30 @@ +{ + "source": "community mirror gpue/foundationpose-weights; official Google Drive quota exceeded", + "verification": "size and mirror LFS SHA256; not independently compared to official files", + "files": [ + { + "path": "2023-10-28-18-33-37/config.yml", + "size": 708, + "sha256": "28a6ba94a33230ee5fc3c51939486281578b0972542bd9e38ca6123e75605686", + "url": "https://hf-mirror.com/gpue/foundationpose-weights/resolve/main/2023-10-28-18-33-37/config.yml" + }, + { + "path": "2023-10-28-18-33-37/model_best.pth", + "size": 68220109, + "sha256": "774700586ddc435d408fc01c9809c43e151232936369dfbea0f0f964ba471d60", + "url": "https://hf-mirror.com/gpue/foundationpose-weights/resolve/main/2023-10-28-18-33-37/model_best.pth" + }, + { + "path": "2024-01-11-20-02-45/config.yml", + "size": 778, + "sha256": "a79db4de3b95885dd5ae86833b37b8698a75dad81e87d1086cd50b2fcd8dda3f", + "url": "https://hf-mirror.com/gpue/foundationpose-weights/resolve/main/2024-01-11-20-02-45/config.yml" + }, + { + "path": "2024-01-11-20-02-45/model_best.pth", + "size": 190229389, + "sha256": "81924d384bf5c26c646ee4783104982ae3d1e049c181c36641b6a7aeae494c26", + "url": "https://hf-mirror.com/gpue/foundationpose-weights/resolve/main/2024-01-11-20-02-45/model_best.pth" + } + ] +} \ No newline at end of file diff --git a/configs/large_files.json b/configs/large_files.json new file mode 100644 index 0000000..9aa7711 --- /dev/null +++ b/configs/large_files.json @@ -0,0 +1,65 @@ +{ + "chunk_bytes": 50331648, + "files": [ + { + "path": "third_party/hamer/_DATA/hamer_ckpts/checkpoints/hamer.ckpt", + "bytes": 2689536166, + "sha256": "e5cc06f294d88a92dee24e603480aab04de532b49f0e08200804ee7d90e16f53", + "parts": 54 + }, + { + "path": "weights/unidepth-v2-vitl14/model.safetensors", + "bytes": 1415383604, + "sha256": "ba73d3de735302ccc64a50f1e557122050c4b1893e6060b28dba05d6af3e67c6", + "parts": 29 + }, + { + "path": "weights/handflow_denoiser.pt", + "bytes": 667907131, + "sha256": "2fbc4e1fa7a60f469a6ac94933a6e6dc8a86a0e1fc13bd7cd81c430c79acfcda", + "parts": 14 + }, + { + "path": "weights/detector.pt", + "bytes": 53582271, + "sha256": "5ef3df44e42d2db52d4ffe91f83a22ce9925e2acc9abebf453f2c5d22e380033", + "parts": 2 + }, + { + "path": "third_party/Dyn-HaMR/_DATA/hmp_model/results/model/optimizer.pth", + "bytes": 541530298, + "sha256": "ea48b2b6d4a8bebff41b72a6f034db5506b488c8b4f114709c58b67953d2ce8d", + "parts": 11 + }, + { + "path": "third_party/Dyn-HaMR/_DATA/hmp_model/results/model/local_encoder.pth", + "bytes": 161190688, + "sha256": "89db49f42b98986121ff0b2152b89c045f6d66945434bf38e6f21981e70ea82b", + "parts": 4 + }, + { + "path": "third_party/Dyn-HaMR/_DATA/hmp_model/results/model/nemf.pth", + "bytes": 92620900, + "sha256": "cbb9d5bae07de807c13f02128b18e1b7b5f31e6626bdda16b800e636753004ff", + "parts": 2 + }, + { + "path": "third_party/Dyn-HaMR/dyn-hamr/optim/BMC/joint_angles.npy", + "bytes": 90116528, + "sha256": "92c96d1899aa290542cf31af322ba53811afd914258f1367fb41e2aeaa6e0b62", + "parts": 2 + }, + { + "path": "third_party/FoundationPose/weights/2024-01-11-20-02-45/model_best.pth", + "bytes": 190229389, + "sha256": "81924d384bf5c26c646ee4783104982ae3d1e049c181c36641b6a7aeae494c26", + "parts": 4 + }, + { + "path": "third_party/FoundationPose/weights/2023-10-28-18-33-37/model_best.pth", + "bytes": 68220109, + "sha256": "774700586ddc435d408fc01c9809c43e151232936369dfbea0f0f964ba471d60", + "parts": 2 + } + ] +} diff --git a/configs/project_env.example.sh b/configs/project_env.example.sh new file mode 100644 index 0000000..dea0326 --- /dev/null +++ b/configs/project_env.example.sh @@ -0,0 +1,16 @@ +#!/usr/bin/env bash +# Source from any directory: source /path/to/project/configs/project_env.example.sh +export HANDFLOW_PROJECT_ROOT="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")/.." && pwd)" +export TORCH_HOME="$HANDFLOW_PROJECT_ROOT/.torch_cache" +export HF_HOME="$HANDFLOW_PROJECT_ROOT/.hf_cache" +export WARP_CACHE_PATH="$HANDFLOW_PROJECT_ROOT/.spider_cache/warp" +export TORCHINDUCTOR_CACHE_DIR="$HANDFLOW_PROJECT_ROOT/.spider_cache/torchinductor" +export VIPE_ROOT="$HANDFLOW_PROJECT_ROOT/third_party/vipe" +export VIPE_PYTHON="$HANDFLOW_PROJECT_ROOT/.venv/bin/python" +export HAMER_CKPT="$HANDFLOW_PROJECT_ROOT/third_party/hamer/_DATA/hamer_ckpts/checkpoints/hamer.ckpt" +export MANO_ROOT="$HANDFLOW_PROJECT_ROOT/third_party/hamer/_DATA/data/mano" +export DETECTOR_CKPT="$HANDFLOW_PROJECT_ROOT/weights/detector.pt" +export HANDFLOW_NORMALIZATION_STATS="$HANDFLOW_PROJECT_ROOT/weights/normalization_stats.npz" +export MUJOCO_GL="${MUJOCO_GL:-osmesa}" +# Offline flags intentionally unset here: populate caches before offline inference. +# Dyn-HaMR also uses its own _DATA tree and Hydra paths; see SETUP_AND_WEIGHTS.md. diff --git a/docs/20260915_171525/20260915_171525/color.mp4 b/docs/20260915_171525/20260915_171525/color.mp4 new file mode 100644 index 0000000..0240cd3 --- /dev/null +++ b/docs/20260915_171525/20260915_171525/color.mp4 @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:193a2a2d2a21cb3abbd01c6d6f93b597d2cb501a64d1ef11a91c193bdf3fc574 +size 3156391 diff --git a/docs/20260915_171525/20260915_171525/depth/000000.png b/docs/20260915_171525/20260915_171525/depth/000000.png new file mode 100644 index 0000000..92c7af4 --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000000.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:e5f9706b5468bbd36488b7e31165a2535db6f356de48bff710320c4dc0277d61 +size 331266 diff --git a/docs/20260915_171525/20260915_171525/depth/000001.png b/docs/20260915_171525/20260915_171525/depth/000001.png new file mode 100644 index 0000000..b0c669a --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000001.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:66d6551cc0a0dd460870a9d6579d9832fbbd94228495d4c03fd15803774ced18 +size 326373 diff --git a/docs/20260915_171525/20260915_171525/depth/000002.png b/docs/20260915_171525/20260915_171525/depth/000002.png new file mode 100644 index 0000000..9087a5e --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000002.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:06dfb4a84d52347a0dddbaa1c92c6e620c5b2bf45fc81abe44ec5246a0005cde +size 333345 diff --git a/docs/20260915_171525/20260915_171525/depth/000003.png b/docs/20260915_171525/20260915_171525/depth/000003.png new file mode 100644 index 0000000..64927a8 --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000003.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:701266a4316b77e66f979fa36711d4a1a842474bd4dcad9218232085de171b63 +size 326474 diff --git a/docs/20260915_171525/20260915_171525/depth/000004.png b/docs/20260915_171525/20260915_171525/depth/000004.png new file mode 100644 index 0000000..e7139ac --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000004.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:a3083806b610b69cad602b7f6a28302acfc2f0b123afcd076fb4f3295cdfcd2f +size 329915 diff --git a/docs/20260915_171525/20260915_171525/depth/000005.png b/docs/20260915_171525/20260915_171525/depth/000005.png new file mode 100644 index 0000000..ad85f44 --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000005.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:601cd0d8e35b0565e5bb2066d141d4327b82e2825a590b8d3c333314b8f984bf +size 339924 diff --git a/docs/20260915_171525/20260915_171525/depth/000006.png b/docs/20260915_171525/20260915_171525/depth/000006.png new file mode 100644 index 0000000..7a7efe3 --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000006.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:65e68e409915199b8f119f25b3fc69ba8d7c3abae022f2f04b312d9bf5152add +size 337000 diff --git a/docs/20260915_171525/20260915_171525/depth/000007.png b/docs/20260915_171525/20260915_171525/depth/000007.png new file mode 100644 index 0000000..d5e3864 --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000007.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:39e5eb6755cb2c7509a2f4d07e4585b4bf10a0082316fce3645e8014c67352d8 +size 329906 diff --git a/docs/20260915_171525/20260915_171525/depth/000008.png b/docs/20260915_171525/20260915_171525/depth/000008.png new file mode 100644 index 0000000..1df510f --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000008.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:5d584c27357fab708a3359729df3a2060d4ad1d3a348d3125bff246265381b52 +size 331489 diff --git a/docs/20260915_171525/20260915_171525/depth/000009.png b/docs/20260915_171525/20260915_171525/depth/000009.png new file mode 100644 index 0000000..bb2ad0e --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000009.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:a61a18b86e9a7cb2e7528282a40691b7b9c1d4f2fd067fdf85aa38ed65e8fb61 +size 326222 diff --git a/docs/20260915_171525/20260915_171525/depth/000010.png b/docs/20260915_171525/20260915_171525/depth/000010.png new file mode 100644 index 0000000..916f400 --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000010.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:be7955d81fdde9221c5f188629350ea315fb22857da98b9aa48eabac235a293f +size 335454 diff --git a/docs/20260915_171525/20260915_171525/depth/000011.png b/docs/20260915_171525/20260915_171525/depth/000011.png new file mode 100644 index 0000000..3a60a0a --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000011.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:5f49e2edb35cfc5676b1c1f056f550e20d765c6efa2c4a040ff3b8aa9dff401b +size 326890 diff --git a/docs/20260915_171525/20260915_171525/depth/000012.png b/docs/20260915_171525/20260915_171525/depth/000012.png new file mode 100644 index 0000000..6846d4b --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000012.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:f0ab4e8b2344ee982a1557ad392685c3053ee8ef12d32004de909cfe883bf46c +size 330689 diff --git a/docs/20260915_171525/20260915_171525/depth/000013.png b/docs/20260915_171525/20260915_171525/depth/000013.png new file mode 100644 index 0000000..0938fad --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000013.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:77933d558e8c8a27f19a48792b980fbd631f5a0977a2fc0fccf9d28d007ab0c9 +size 327180 diff --git a/docs/20260915_171525/20260915_171525/depth/000014.png b/docs/20260915_171525/20260915_171525/depth/000014.png new file mode 100644 index 0000000..17b89ed --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000014.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:967cfdd6d33b378eb07ccff9c614c0a082790fef9037c4bec60a0e446c92708a +size 331345 diff --git a/docs/20260915_171525/20260915_171525/depth/000015.png b/docs/20260915_171525/20260915_171525/depth/000015.png new file mode 100644 index 0000000..28ae951 --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000015.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:a4e9f571254d564480ab4a2a2d6012c1eda074bb8d11f0246133420d1591e0f5 +size 327139 diff --git a/docs/20260915_171525/20260915_171525/depth/000016.png b/docs/20260915_171525/20260915_171525/depth/000016.png new file mode 100644 index 0000000..b3f8c7f --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000016.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:dfaf6068e58fe15a2ecc185d0eb53cf0f21f988dd16e73e0dbd5c126dbfd8ae5 +size 330244 diff --git a/docs/20260915_171525/20260915_171525/depth/000017.png b/docs/20260915_171525/20260915_171525/depth/000017.png new file mode 100644 index 0000000..945e38d --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000017.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:92f63819bda39da2b612465fb443b8c765469a42023706cecdceeec2d2dc963b +size 331516 diff --git a/docs/20260915_171525/20260915_171525/depth/000018.png b/docs/20260915_171525/20260915_171525/depth/000018.png new file mode 100644 index 0000000..7dc34a3 --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000018.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:6579777cc31d05264dbdba69bc1f0a105a37ba106b7135ffb38c905e8b5d72ed +size 333072 diff --git a/docs/20260915_171525/20260915_171525/depth/000019.png b/docs/20260915_171525/20260915_171525/depth/000019.png new file mode 100644 index 0000000..0a7d638 --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000019.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:958dc2da6603e9c05bde446c421794e4c62bdd9f26b07c0c45bd27512713f551 +size 326299 diff --git a/docs/20260915_171525/20260915_171525/depth/000020.png b/docs/20260915_171525/20260915_171525/depth/000020.png new file mode 100644 index 0000000..5e13bca --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000020.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:44b163c6722b435145c15b451f5456b59da84cf4419d715314e6a719fe661f4c +size 327184 diff --git a/docs/20260915_171525/20260915_171525/depth/000021.png b/docs/20260915_171525/20260915_171525/depth/000021.png new file mode 100644 index 0000000..47f5fd1 --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000021.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:641522f5934f6a3a03ff5e36ae951d5e94acc1ffdc4d96c67ef46fb91212e404 +size 326112 diff --git a/docs/20260915_171525/20260915_171525/depth/000022.png b/docs/20260915_171525/20260915_171525/depth/000022.png new file mode 100644 index 0000000..2760424 --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000022.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:4183534310ec4b18bcae5ca7d9ffad74b159d2b62b0e4a552fcad22ffc5f3970 +size 329445 diff --git a/docs/20260915_171525/20260915_171525/depth/000023.png b/docs/20260915_171525/20260915_171525/depth/000023.png new file mode 100644 index 0000000..6b69a6b --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000023.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:59a622da9103d09249e8a9efb3be59ec80e446cc875239774d4e82d971f7c31d +size 338203 diff --git a/docs/20260915_171525/20260915_171525/depth/000024.png b/docs/20260915_171525/20260915_171525/depth/000024.png new file mode 100644 index 0000000..a195d01 --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000024.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:56ae6fa1362290f76af19fc11e5071b622ec69f43dd84026c663ad2758681ba2 +size 333433 diff --git a/docs/20260915_171525/20260915_171525/depth/000025.png b/docs/20260915_171525/20260915_171525/depth/000025.png new file mode 100644 index 0000000..44de281 --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000025.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:743994ac05a57fbba2bea51bdd83ca2a0e43b68fdd641b15d3c516d29048dd56 +size 331027 diff --git a/docs/20260915_171525/20260915_171525/depth/000026.png b/docs/20260915_171525/20260915_171525/depth/000026.png new file mode 100644 index 0000000..ca475aa --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000026.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:b28b5de138f22716b05ee9f92d2d0faa68ba616a65c528c20d76e555d5813d49 +size 341677 diff --git a/docs/20260915_171525/20260915_171525/depth/000027.png b/docs/20260915_171525/20260915_171525/depth/000027.png new file mode 100644 index 0000000..334e2e5 --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000027.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:f8c180c69ac7e8478a086031b3171cba17e3cfba4b4c5467e2e054b6c0000ad7 +size 332620 diff --git a/docs/20260915_171525/20260915_171525/depth/000028.png b/docs/20260915_171525/20260915_171525/depth/000028.png new file mode 100644 index 0000000..1570b4a --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000028.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:f89d8c0653640e426b0a772784250574682453accfc0f1416283248fba0ddca0 +size 331938 diff --git a/docs/20260915_171525/20260915_171525/depth/000029.png b/docs/20260915_171525/20260915_171525/depth/000029.png new file mode 100644 index 0000000..66a2665 --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000029.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:40ab0356f2ca071177d2d6f907b1f4885fa0c5e8131821d84c63cd3730573441 +size 335152 diff --git a/docs/20260915_171525/20260915_171525/depth/000030.png b/docs/20260915_171525/20260915_171525/depth/000030.png new file mode 100644 index 0000000..1d28f5f --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000030.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:dcbfda275e1992969902513d5948a3a9afb9e36040302516b91071d5a6dfe1f1 +size 331486 diff --git a/docs/20260915_171525/20260915_171525/depth/000031.png b/docs/20260915_171525/20260915_171525/depth/000031.png new file mode 100644 index 0000000..e1f4a31 --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000031.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:acb6d92b24fd6c13ba15e56ce18f5d391ff01b51506b0f932db268235d50c522 +size 341661 diff --git a/docs/20260915_171525/20260915_171525/depth/000032.png b/docs/20260915_171525/20260915_171525/depth/000032.png new file mode 100644 index 0000000..8db667d --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000032.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:fa8a9bbdd33ba63c4f284aeed4bf2c656530a976d800fc66dd61ff82ff9dd005 +size 334524 diff --git a/docs/20260915_171525/20260915_171525/depth/000033.png b/docs/20260915_171525/20260915_171525/depth/000033.png new file mode 100644 index 0000000..42b5a6f --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000033.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:52d4326fd1d0176f9de524b5389b1e8af291390e391d9d359db2c6ed9651b2e2 +size 325228 diff --git a/docs/20260915_171525/20260915_171525/depth/000034.png b/docs/20260915_171525/20260915_171525/depth/000034.png new file mode 100644 index 0000000..9810c7a --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000034.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:9c01ea73ad178ce0fefb3c26502c8a1924bb2895bc7b82130d5a66c36a53fe30 +size 332585 diff --git a/docs/20260915_171525/20260915_171525/depth/000035.png b/docs/20260915_171525/20260915_171525/depth/000035.png new file mode 100644 index 0000000..9546ed8 --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000035.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:df41b9c352fdba396ce141a6d84392fc0095d0b951e91ca8fe9acdda88d4d307 +size 328117 diff --git a/docs/20260915_171525/20260915_171525/depth/000036.png b/docs/20260915_171525/20260915_171525/depth/000036.png new file mode 100644 index 0000000..063e244 --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000036.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:5ec1425eee81d9355d2ad5c4dfc538a5571e281bf5e263ad102e989aefb21d40 +size 332159 diff --git a/docs/20260915_171525/20260915_171525/depth/000037.png b/docs/20260915_171525/20260915_171525/depth/000037.png new file mode 100644 index 0000000..a36b40d --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000037.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:b43ceca8caa8cb8cf5d45936f8c4ad6a5919199476354dce210ae68c24c866ce +size 336691 diff --git a/docs/20260915_171525/20260915_171525/depth/000038.png b/docs/20260915_171525/20260915_171525/depth/000038.png new file mode 100644 index 0000000..b5fc103 --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000038.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:d50f3eb7a5d33056541a76afc11a4b611d99d68e2a7465a09b8ab8f581a6856f +size 342192 diff --git a/docs/20260915_171525/20260915_171525/depth/000039.png b/docs/20260915_171525/20260915_171525/depth/000039.png new file mode 100644 index 0000000..1b70052 --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000039.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:3b0653dee7bc22bba9e2067ba2a1e50c274d356b037b223f904cd41dbf19752a +size 336540 diff --git a/docs/20260915_171525/20260915_171525/depth/000040.png b/docs/20260915_171525/20260915_171525/depth/000040.png new file mode 100644 index 0000000..897e31f --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000040.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:42812c72fcb8fe2385c4584f54e531a6cd582868d44e0d5c21453319603114cf +size 335002 diff --git a/docs/20260915_171525/20260915_171525/depth/000041.png b/docs/20260915_171525/20260915_171525/depth/000041.png new file mode 100644 index 0000000..9c7c39a --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000041.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:bd01821bc3f52a23b0dd3339a0ec490d13b3b53b69087ec1238553bf624158fb +size 335619 diff --git a/docs/20260915_171525/20260915_171525/depth/000042.png b/docs/20260915_171525/20260915_171525/depth/000042.png new file mode 100644 index 0000000..27493a8 --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000042.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:a45e14914111ce945e5325217277c04f3f795bf1fa4f10a1383b8b15e7467ecf +size 338929 diff --git a/docs/20260915_171525/20260915_171525/depth/000043.png b/docs/20260915_171525/20260915_171525/depth/000043.png new file mode 100644 index 0000000..3ec0056 --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000043.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:2df8a0ed1261456ab75fdf2d4779eb0d4429d63750428ce6f06aa5dd675af5c6 +size 332470 diff --git a/docs/20260915_171525/20260915_171525/depth/000044.png b/docs/20260915_171525/20260915_171525/depth/000044.png new file mode 100644 index 0000000..0e52a68 --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000044.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:2ce8a8f5048b7dc72a60e25f925defd2c80f1b151e24d0b059595d412fad0481 +size 335036 diff --git a/docs/20260915_171525/20260915_171525/depth/000045.png b/docs/20260915_171525/20260915_171525/depth/000045.png new file mode 100644 index 0000000..d36e6bf --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000045.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:a9af9592509d54887300ccb558ccccbfb05c114104a4286b2a4f41d6f4f9319e +size 340777 diff --git a/docs/20260915_171525/20260915_171525/depth/000046.png b/docs/20260915_171525/20260915_171525/depth/000046.png new file mode 100644 index 0000000..2ce1519 --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000046.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:b76274e0d3b18afeeab773e1503d134e3b6d2b6233af22c337b761495333f8c1 +size 330649 diff --git a/docs/20260915_171525/20260915_171525/depth/000047.png b/docs/20260915_171525/20260915_171525/depth/000047.png new file mode 100644 index 0000000..2f25fad --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000047.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:f907bb8432b89af4e63ae12a4f97375d0e810e70c464bdadb63072ed54d89a00 +size 343123 diff --git a/docs/20260915_171525/20260915_171525/depth/000048.png b/docs/20260915_171525/20260915_171525/depth/000048.png new file mode 100644 index 0000000..bf661a6 --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000048.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:d0f3ef53acb5884a07392a20b87942230da66466a6af3363fa6b8d390531c166 +size 342273 diff --git a/docs/20260915_171525/20260915_171525/depth/000049.png b/docs/20260915_171525/20260915_171525/depth/000049.png new file mode 100644 index 0000000..15ee398 --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000049.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:3e0145c3c22d7873472dbcf1e2cc0db6273422717a65b1685449c2fd6a789acf +size 335981 diff --git a/docs/20260915_171525/20260915_171525/depth/000050.png b/docs/20260915_171525/20260915_171525/depth/000050.png new file mode 100644 index 0000000..5c4baff --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000050.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:c674028588ea096dcbc7160c6c8443073e83bb7158118e0a8824ecfcd4fe4629 +size 348761 diff --git a/docs/20260915_171525/20260915_171525/depth/000051.png b/docs/20260915_171525/20260915_171525/depth/000051.png new file mode 100644 index 0000000..a67d5dc --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000051.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:b56cd177e53fd9c27dbd1261e12925b307e908ab4928ca9becdd54fc757516bd +size 341958 diff --git a/docs/20260915_171525/20260915_171525/depth/000052.png b/docs/20260915_171525/20260915_171525/depth/000052.png new file mode 100644 index 0000000..d0231d3 --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000052.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:e19f40f9f51ddbe3b9bb65de251467bb9af51b1c40a0a1113fa8ec596a44b3d6 +size 333214 diff --git a/docs/20260915_171525/20260915_171525/depth/000053.png b/docs/20260915_171525/20260915_171525/depth/000053.png new file mode 100644 index 0000000..66c1f15 --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000053.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:cbaa0a654811d7472635caeee7b3d3ae2fc67ad27c3ca5b927972090937f96d2 +size 339096 diff --git a/docs/20260915_171525/20260915_171525/depth/000054.png b/docs/20260915_171525/20260915_171525/depth/000054.png new file mode 100644 index 0000000..89f34a0 --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000054.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:d6ec7afa7a5e9d71bb4d8b39d33b5afd2e6b0cabe33fe64c77a3292a81bc4ca4 +size 326073 diff --git a/docs/20260915_171525/20260915_171525/depth/000055.png b/docs/20260915_171525/20260915_171525/depth/000055.png new file mode 100644 index 0000000..94bffe0 --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000055.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:1130e9c7a1062c41174bcbf182d79653b84a491b507406fa83d4ae302c399197 +size 337254 diff --git a/docs/20260915_171525/20260915_171525/depth/000056.png b/docs/20260915_171525/20260915_171525/depth/000056.png new file mode 100644 index 0000000..019fcdb --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000056.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:989719f9a46e574f713fc39fb4a3033623cbbfad3a24ca2c45a9aa0c446f0888 +size 332765 diff --git a/docs/20260915_171525/20260915_171525/depth/000057.png b/docs/20260915_171525/20260915_171525/depth/000057.png new file mode 100644 index 0000000..267a610 --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000057.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:7cb7de2844bea35e9acadeece51db96aa9f88070b73f185f1d8de0ec6224de3d +size 344836 diff --git a/docs/20260915_171525/20260915_171525/depth/000058.png b/docs/20260915_171525/20260915_171525/depth/000058.png new file mode 100644 index 0000000..9fbbe7e --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000058.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:029968602be3e470ca97520b083cf7ba91a5f10c90bd0439d09f4eb7edc1dc0c +size 340157 diff --git a/docs/20260915_171525/20260915_171525/depth/000059.png b/docs/20260915_171525/20260915_171525/depth/000059.png new file mode 100644 index 0000000..fd83f6f --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000059.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:d8aebda30cd613e27bae9dc364f0b472fbcb1d0950fb1c3a4b23e1cdfa297fda +size 341314 diff --git a/docs/20260915_171525/20260915_171525/depth/000060.png b/docs/20260915_171525/20260915_171525/depth/000060.png new file mode 100644 index 0000000..a16eb4c --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000060.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:9e02d40ae890f00e324dc155053af75bd0d144615cc316c663e7c77ba00a8091 +size 341714 diff --git a/docs/20260915_171525/20260915_171525/depth/000061.png b/docs/20260915_171525/20260915_171525/depth/000061.png new file mode 100644 index 0000000..8f76684 --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000061.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:f65d8122250cfeca31b80b9c7ca9ed87f66c97b9953a42aca81579a39c48d792 +size 339118 diff --git a/docs/20260915_171525/20260915_171525/depth/000062.png b/docs/20260915_171525/20260915_171525/depth/000062.png new file mode 100644 index 0000000..e923591 --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000062.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:75ce0b9a0f77dbb678df72d666afe39326d9b46b9e54bf77e984261ae2cddd5e +size 342295 diff --git a/docs/20260915_171525/20260915_171525/depth/000063.png b/docs/20260915_171525/20260915_171525/depth/000063.png new file mode 100644 index 0000000..f7b43a1 --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000063.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:7a9aee11918e46b5a805add841e90b149b8618b3408dfff4aa00bc27eabaeb9f +size 337709 diff --git a/docs/20260915_171525/20260915_171525/depth/000064.png b/docs/20260915_171525/20260915_171525/depth/000064.png new file mode 100644 index 0000000..2fdd2ca --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000064.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:0425b9dd443e287b8e9c6346dc25e6318cdeb03700c5666baf35ac7793bdb6a1 +size 344887 diff --git a/docs/20260915_171525/20260915_171525/depth/000065.png b/docs/20260915_171525/20260915_171525/depth/000065.png new file mode 100644 index 0000000..31f1586 --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000065.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:17e0b3a44dfa09cca0d5e7ff981abe0024df5c77bac3ae08982c8c272e28a4c1 +size 336785 diff --git a/docs/20260915_171525/20260915_171525/depth/000066.png b/docs/20260915_171525/20260915_171525/depth/000066.png new file mode 100644 index 0000000..f2ff83c --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000066.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:6fcd6ad1b5acd87225f11b95a23773a81e3da096bb1ffefec6cb8c59f31931eb +size 336035 diff --git a/docs/20260915_171525/20260915_171525/depth/000067.png b/docs/20260915_171525/20260915_171525/depth/000067.png new file mode 100644 index 0000000..6cbf703 --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000067.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:ee26f48c93d869f7f73dc6ba32f143b24eaaa61f8443d4970352dcca53de33f7 +size 341775 diff --git a/docs/20260915_171525/20260915_171525/depth/000068.png b/docs/20260915_171525/20260915_171525/depth/000068.png new file mode 100644 index 0000000..2c2a188 --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000068.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:defb668f2c4a20a2667fe796b6482d5ccb8593818f23897bf3ccf1cf19efd933 +size 348384 diff --git a/docs/20260915_171525/20260915_171525/depth/000069.png b/docs/20260915_171525/20260915_171525/depth/000069.png new file mode 100644 index 0000000..478408a --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000069.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:b5c62c90f22c02e128bfe6ddb29afcfb76402fad5dacb565c8fb4540eac0c27d +size 342108 diff --git a/docs/20260915_171525/20260915_171525/depth/000070.png b/docs/20260915_171525/20260915_171525/depth/000070.png new file mode 100644 index 0000000..3352847 --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000070.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:d52abf14c1f9ed4815a83cb2a0624934e21d7f47ca79bad5ce5463f598c4e78e +size 344598 diff --git a/docs/20260915_171525/20260915_171525/depth/000071.png b/docs/20260915_171525/20260915_171525/depth/000071.png new file mode 100644 index 0000000..ad17f1e --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000071.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:12efc6cf8bd85820da459e535b2f00aba8788567c663a29ec56ea955779b9484 +size 342147 diff --git a/docs/20260915_171525/20260915_171525/depth/000072.png b/docs/20260915_171525/20260915_171525/depth/000072.png new file mode 100644 index 0000000..3b4c046 --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000072.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:ae75372391ac1eda41fa979dc8eccfea9f75a3a06d3667837df7532516af8140 +size 344530 diff --git a/docs/20260915_171525/20260915_171525/depth/000073.png b/docs/20260915_171525/20260915_171525/depth/000073.png new file mode 100644 index 0000000..7fc27d1 --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000073.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:ff16f7ac7cab5f1adabb53b148aae0746ff11c727804de3b772897c0ac95a38e +size 340358 diff --git a/docs/20260915_171525/20260915_171525/depth/000074.png b/docs/20260915_171525/20260915_171525/depth/000074.png new file mode 100644 index 0000000..9d3317a --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000074.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:6a6cc5eeccf6a32c82ce0adc008c59f8fe54aecfa07004d4a0605833c4351690 +size 336446 diff --git a/docs/20260915_171525/20260915_171525/depth/000075.png b/docs/20260915_171525/20260915_171525/depth/000075.png new file mode 100644 index 0000000..d95d952 --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000075.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:cc80b0c331c511c439113a1c6816af062f1385c27832b690d663ff725f0b111b +size 343955 diff --git a/docs/20260915_171525/20260915_171525/depth/000076.png b/docs/20260915_171525/20260915_171525/depth/000076.png new file mode 100644 index 0000000..49b4ea0 --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000076.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:545a06b3908560deb7ce3ec70147591af66e5c4aa812fe3c08d4f82167d05b41 +size 342131 diff --git a/docs/20260915_171525/20260915_171525/depth/000077.png b/docs/20260915_171525/20260915_171525/depth/000077.png new file mode 100644 index 0000000..ef85e41 --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000077.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:987b6cd4b4b6daaf2d245a5ef9ca228ec516d896c548c16bf1f6a05a0c4abd59 +size 345644 diff --git a/docs/20260915_171525/20260915_171525/depth/000078.png b/docs/20260915_171525/20260915_171525/depth/000078.png new file mode 100644 index 0000000..9fd6865 --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000078.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:44a1576e6c9a31f11ef495b347e642e1dd996384f933fcaa93a7bdb6a162dba1 +size 337762 diff --git a/docs/20260915_171525/20260915_171525/depth/000079.png b/docs/20260915_171525/20260915_171525/depth/000079.png new file mode 100644 index 0000000..37e45fe --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000079.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:ff4637db4a6854b48fc353612b7d5fefe2187aa81b7bf050caf4f8f9f965dd3f +size 344684 diff --git a/docs/20260915_171525/20260915_171525/depth/000080.png b/docs/20260915_171525/20260915_171525/depth/000080.png new file mode 100644 index 0000000..b40ef63 --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000080.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:3d1ce17b64cda19a55776f9cf229e593e8f232a8c28bd00b705e7fc0b15bf87f +size 337731 diff --git a/docs/20260915_171525/20260915_171525/depth/000081.png b/docs/20260915_171525/20260915_171525/depth/000081.png new file mode 100644 index 0000000..62a5be2 --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000081.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:149e93fe2233afee85b45044cceabfb8b03c3c108a046af1266d98a874180ed9 +size 337952 diff --git a/docs/20260915_171525/20260915_171525/depth/000082.png b/docs/20260915_171525/20260915_171525/depth/000082.png new file mode 100644 index 0000000..dc988b5 --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000082.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:3362e7659c127dde4250d415da36f28696c33a89de8061dd33cd096c0becce37 +size 334106 diff --git a/docs/20260915_171525/20260915_171525/depth/000083.png b/docs/20260915_171525/20260915_171525/depth/000083.png new file mode 100644 index 0000000..aeaf47c --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000083.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:d9403ba0fe1f555ce20d2fcc18b1649327d4d2f0eb04c5fe4ffc781b7152f1ce +size 338294 diff --git a/docs/20260915_171525/20260915_171525/depth/000084.png b/docs/20260915_171525/20260915_171525/depth/000084.png new file mode 100644 index 0000000..26ea397 --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000084.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:f88a4f2f2091deb0e50bdac94fff55c9cc4363bd4aebe7431557746add5a9dff +size 337404 diff --git a/docs/20260915_171525/20260915_171525/depth/000085.png b/docs/20260915_171525/20260915_171525/depth/000085.png new file mode 100644 index 0000000..047060c --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000085.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:0538e157d6cf9cbf8c3c3c9721dae6d73614a37e9afdcaab4b8a1f6dfb7c1feb +size 337328 diff --git a/docs/20260915_171525/20260915_171525/depth/000086.png b/docs/20260915_171525/20260915_171525/depth/000086.png new file mode 100644 index 0000000..f8edf29 --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000086.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:54fa06a48ca9bdb8a6e594e24fdc78657b0268e9bc2de38f126e1674924fc9a3 +size 338501 diff --git a/docs/20260915_171525/20260915_171525/depth/000087.png b/docs/20260915_171525/20260915_171525/depth/000087.png new file mode 100644 index 0000000..1c7d52a --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000087.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:426f04ea0b462918dd45685753b4523db5cf0e7682aca0b3a903bee37be329e7 +size 336120 diff --git a/docs/20260915_171525/20260915_171525/depth/000088.png b/docs/20260915_171525/20260915_171525/depth/000088.png new file mode 100644 index 0000000..77c9ce8 --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000088.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:d082cd718230c3376314d0098c3b1a9bf13cfe86cea2ca9384df3e30ab0dc567 +size 338196 diff --git a/docs/20260915_171525/20260915_171525/depth/000089.png b/docs/20260915_171525/20260915_171525/depth/000089.png new file mode 100644 index 0000000..ef4599b --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000089.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:143e55a6c11ac32b1d842dc0815d48ff7878016d4d05eaa86134b8cac21034e7 +size 338891 diff --git a/docs/20260915_171525/20260915_171525/depth/000090.png b/docs/20260915_171525/20260915_171525/depth/000090.png new file mode 100644 index 0000000..0365eb8 --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000090.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:0f86513052829a4ad544d387f32aa5c40e98c875b93e413b9487bf30196cceca +size 333833 diff --git a/docs/20260915_171525/20260915_171525/depth/000091.png b/docs/20260915_171525/20260915_171525/depth/000091.png new file mode 100644 index 0000000..a7e676c --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000091.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:9cfae37f3cc45db512ebb0451b3ee43adc13106479bce38e41e5bd7dc7439fd3 +size 334017 diff --git a/docs/20260915_171525/20260915_171525/depth/000092.png b/docs/20260915_171525/20260915_171525/depth/000092.png new file mode 100644 index 0000000..e5552ec --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000092.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:995765b62caeb5418b231fd3201669a8db51c2729c8e8d45c4e1d53b2fec377b +size 329420 diff --git a/docs/20260915_171525/20260915_171525/depth/000093.png b/docs/20260915_171525/20260915_171525/depth/000093.png new file mode 100644 index 0000000..13bd2d2 --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000093.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:c2570283d7cdd53c80ba5d02dfe76766e85125b7756226ab168d54b95f7ab0f6 +size 333784 diff --git a/docs/20260915_171525/20260915_171525/depth/000094.png b/docs/20260915_171525/20260915_171525/depth/000094.png new file mode 100644 index 0000000..6895361 --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000094.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:4a925673510cdaf292c69b2589ad0417801f969e41c02c1133883f639573bc89 +size 333203 diff --git a/docs/20260915_171525/20260915_171525/depth/000095.png b/docs/20260915_171525/20260915_171525/depth/000095.png new file mode 100644 index 0000000..8608f69 --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000095.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:ee4a2a3307b25563c356da6180efd83fee6d76b69fa9a933e924f2a1769f3408 +size 340919 diff --git a/docs/20260915_171525/20260915_171525/depth/000096.png b/docs/20260915_171525/20260915_171525/depth/000096.png new file mode 100644 index 0000000..aab5954 --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000096.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:8a3c5c09b3ad41458b144851cd3787bbd15b7a6251f243917d9a92f2f9c6ee76 +size 337998 diff --git a/docs/20260915_171525/20260915_171525/depth/000097.png b/docs/20260915_171525/20260915_171525/depth/000097.png new file mode 100644 index 0000000..248f90d --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000097.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:62fe9788fc3395435c40008d88f8db28da25dc2123e5fe110eff33961fe435a6 +size 336044 diff --git a/docs/20260915_171525/20260915_171525/depth/000098.png b/docs/20260915_171525/20260915_171525/depth/000098.png new file mode 100644 index 0000000..2035d37 --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000098.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:9122d0ba6b2c859670f05f68e704cdfbebeb6aba96f1ad9d4b4e9a0285bc2ced +size 329311 diff --git a/docs/20260915_171525/20260915_171525/depth/000099.png b/docs/20260915_171525/20260915_171525/depth/000099.png new file mode 100644 index 0000000..cb79a75 --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000099.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:41c794c03350ae4c200a5cac75558bf8598e3f5d2f6072165f7c7559ef684584 +size 334617 diff --git a/docs/20260915_171525/20260915_171525/depth/000100.png b/docs/20260915_171525/20260915_171525/depth/000100.png new file mode 100644 index 0000000..535fb10 --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000100.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:b5daa853560b760de82d2e11f7ccda2dbf6dafa1de27d3aad2ed295a9582cf90 +size 336937 diff --git a/docs/20260915_171525/20260915_171525/depth/000101.png b/docs/20260915_171525/20260915_171525/depth/000101.png new file mode 100644 index 0000000..60237a3 --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000101.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:b74ae94294247e108c15ba271d121136640d1bf8467c0026d1852e57bd9d1d8a +size 336975 diff --git a/docs/20260915_171525/20260915_171525/depth/000102.png b/docs/20260915_171525/20260915_171525/depth/000102.png new file mode 100644 index 0000000..b196b3d --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000102.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:5ed88198723c0e17897199c0e90e668057c45e833cd0d7af777a64bdb143c051 +size 338134 diff --git a/docs/20260915_171525/20260915_171525/depth/000103.png b/docs/20260915_171525/20260915_171525/depth/000103.png new file mode 100644 index 0000000..199f382 --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000103.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:a465e88ff39a99df0bf6c13352647c7dd29e546e9ad466371777c2b99097331a +size 339782 diff --git a/docs/20260915_171525/20260915_171525/depth/000104.png b/docs/20260915_171525/20260915_171525/depth/000104.png new file mode 100644 index 0000000..b877eeb --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000104.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:9fd61cb4491ee8325aaa2f2b1885e0a61a96fd3eb66b7d4a4b6c5f92882ea487 +size 332104 diff --git a/docs/20260915_171525/20260915_171525/depth/000105.png b/docs/20260915_171525/20260915_171525/depth/000105.png new file mode 100644 index 0000000..a6d785b --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000105.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:539ce0a84ebbb334b05268a9f87625decb2d7dee5cc5a97b38db823c1db845ab +size 346092 diff --git a/docs/20260915_171525/20260915_171525/depth/000106.png b/docs/20260915_171525/20260915_171525/depth/000106.png new file mode 100644 index 0000000..de24932 --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000106.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:5737f89d60c57bdcb5f9340d7d33a7fbfd17a8a8b0a395398e9bd4a7e7bb66a1 +size 328037 diff --git a/docs/20260915_171525/20260915_171525/depth/000107.png b/docs/20260915_171525/20260915_171525/depth/000107.png new file mode 100644 index 0000000..002ba9e --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000107.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:4b2681dd8b6091c04a1e8f13a5544a3635032743af40fb440adb8a591332f1e1 +size 333564 diff --git a/docs/20260915_171525/20260915_171525/depth/000108.png b/docs/20260915_171525/20260915_171525/depth/000108.png new file mode 100644 index 0000000..becb90f --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000108.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:5a9cb00d3456c14a05a229e047400f2f6424290d07d68bf503badda8e842e84e +size 333119 diff --git a/docs/20260915_171525/20260915_171525/depth/000109.png b/docs/20260915_171525/20260915_171525/depth/000109.png new file mode 100644 index 0000000..eb0c4e7 --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000109.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:990daafbdd8f8169dbc81b960fc06a5c4718aae31ab55a6cb317981f0f154e8e +size 336228 diff --git a/docs/20260915_171525/20260915_171525/depth/000110.png b/docs/20260915_171525/20260915_171525/depth/000110.png new file mode 100644 index 0000000..03efc42 --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000110.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:105e49da61ea042c233cb62452ca3a1ede830c10e120a7a3d2c446866d07eadf +size 329866 diff --git a/docs/20260915_171525/20260915_171525/depth/000111.png b/docs/20260915_171525/20260915_171525/depth/000111.png new file mode 100644 index 0000000..2303d2e --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000111.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:0b90f71bc87ae35b6f96cc1ba303d054f917be80e0e04f71e4da38ad4c7a146b +size 323454 diff --git a/docs/20260915_171525/20260915_171525/depth/000112.png b/docs/20260915_171525/20260915_171525/depth/000112.png new file mode 100644 index 0000000..a912c64 --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000112.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:b49df8da4c58c4bf3e41c41178c3b98b60adf0ca0d76204033994c03d180572b +size 329176 diff --git a/docs/20260915_171525/20260915_171525/depth/000113.png b/docs/20260915_171525/20260915_171525/depth/000113.png new file mode 100644 index 0000000..77dd504 --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000113.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:8c854d12055460bfcba00407215e6d37f09b2ff86d9073cfcef2fd398b1520eb +size 337220 diff --git a/docs/20260915_171525/20260915_171525/depth/000114.png b/docs/20260915_171525/20260915_171525/depth/000114.png new file mode 100644 index 0000000..95b6341 --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000114.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:4ace3d4b7380bd9682b41c4a2af1715b97c7361a2791fabbacb5e6e956cb78c5 +size 331532 diff --git a/docs/20260915_171525/20260915_171525/depth/000115.png b/docs/20260915_171525/20260915_171525/depth/000115.png new file mode 100644 index 0000000..9012c07 --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000115.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:7a74fb71b6c35a98f128a9f28406a3be3fdac131bbe35d37e479588920589781 +size 330859 diff --git a/docs/20260915_171525/20260915_171525/depth/000116.png b/docs/20260915_171525/20260915_171525/depth/000116.png new file mode 100644 index 0000000..ef45b9e --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000116.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:0b55ed1c6e9aa7528e39122f74bca3116640e86d755c90dc583f98df8dcb651c +size 333296 diff --git a/docs/20260915_171525/20260915_171525/depth/000117.png b/docs/20260915_171525/20260915_171525/depth/000117.png new file mode 100644 index 0000000..199f3d8 --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000117.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:59878fca8df4b80a7e414e8e3ec88dfa90a5db4e7e668c0090b45d90e4ba036b +size 337283 diff --git a/docs/20260915_171525/20260915_171525/depth/000118.png b/docs/20260915_171525/20260915_171525/depth/000118.png new file mode 100644 index 0000000..38a7c45 --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000118.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:daad3ac6cc1fed70c55cc51462c76438035d4a1c0d3bd407e87e69c67ae39ad0 +size 340141 diff --git a/docs/20260915_171525/20260915_171525/depth/000119.png b/docs/20260915_171525/20260915_171525/depth/000119.png new file mode 100644 index 0000000..2913ed8 --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000119.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:a6fb0bf9d9c13452f75a39638b0ea88bfb815df5ed44b37e3631ff6a733e089d +size 332045 diff --git a/docs/20260915_171525/20260915_171525/depth/000120.png b/docs/20260915_171525/20260915_171525/depth/000120.png new file mode 100644 index 0000000..d26b00d --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000120.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:abefc4e99ea18b92df0b9ef999779f1d69ba17a37e5692185bcd72254b1276bf +size 331659 diff --git a/docs/20260915_171525/20260915_171525/depth/000121.png b/docs/20260915_171525/20260915_171525/depth/000121.png new file mode 100644 index 0000000..8c6891b --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000121.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:af20279f1db8a701ef61ae2b919411898d015c4ea508dac8dc50cc841dc08287 +size 336383 diff --git a/docs/20260915_171525/20260915_171525/depth/000122.png b/docs/20260915_171525/20260915_171525/depth/000122.png new file mode 100644 index 0000000..a89c476 --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000122.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:95b3ade515ab9b8dd6323ef3fac1299a1122b57a92989ad34d33d48f314fef5a +size 332920 diff --git a/docs/20260915_171525/20260915_171525/depth/000123.png b/docs/20260915_171525/20260915_171525/depth/000123.png new file mode 100644 index 0000000..a78a4d5 --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000123.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:d8b13e6be85aba0a9c8036a1858210f414767b216c215d1ea1d34da8fa597050 +size 337071 diff --git a/docs/20260915_171525/20260915_171525/depth/000124.png b/docs/20260915_171525/20260915_171525/depth/000124.png new file mode 100644 index 0000000..8a8202e --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000124.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:29a3ec69c0f5ad7ea3d09dd05fcd700839db8cbcae105ebdffe79d7c246155bf +size 327884 diff --git a/docs/20260915_171525/20260915_171525/depth/000125.png b/docs/20260915_171525/20260915_171525/depth/000125.png new file mode 100644 index 0000000..a36ddf4 --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000125.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:4317f191c861c7ad189f9221d6802c88a35e3d7ae474563510e1c76f1a5ad49d +size 333056 diff --git a/docs/20260915_171525/20260915_171525/depth/000126.png b/docs/20260915_171525/20260915_171525/depth/000126.png new file mode 100644 index 0000000..f2f218c --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000126.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:56a7d825a8cceda65dc861304bc3ca114301b29bd1c6e8975618c94d6f8ea1fd +size 335081 diff --git a/docs/20260915_171525/20260915_171525/depth/000127.png b/docs/20260915_171525/20260915_171525/depth/000127.png new file mode 100644 index 0000000..441b464 --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000127.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:225c6cb7d8ca6d783dfef8838719979f2769199dddf014d7a21dbe3fd326f1b8 +size 329630 diff --git a/docs/20260915_171525/20260915_171525/depth/000128.png b/docs/20260915_171525/20260915_171525/depth/000128.png new file mode 100644 index 0000000..b7c94cb --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000128.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:f3ac280fbbbd3b8be844fb7edab58e014acbdf06901d10451fb2a5f716c400fa +size 331393 diff --git a/docs/20260915_171525/20260915_171525/depth/000129.png b/docs/20260915_171525/20260915_171525/depth/000129.png new file mode 100644 index 0000000..689abfa --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000129.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:b7c72dfa35451b2aaf9e099d8072cc7fed46eca79a7b15ac677592f2f431cf64 +size 333954 diff --git a/docs/20260915_171525/20260915_171525/depth/000130.png b/docs/20260915_171525/20260915_171525/depth/000130.png new file mode 100644 index 0000000..36c12be --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000130.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:2f0ed06e43e43345a1d03bbddc98d0abde4a130fdb13c918ca64076e8371fed1 +size 330604 diff --git a/docs/20260915_171525/20260915_171525/depth/000131.png b/docs/20260915_171525/20260915_171525/depth/000131.png new file mode 100644 index 0000000..3987720 --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000131.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:4027a883d9c6dd104d0cf3c7f96f9002e494c9bafb8d8c0804f4d8445a8ccd34 +size 335143 diff --git a/docs/20260915_171525/20260915_171525/depth/000132.png b/docs/20260915_171525/20260915_171525/depth/000132.png new file mode 100644 index 0000000..de217b9 --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000132.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:a1ee3982c24a0555c0480dc7508e2a68b9665a84194c0dd166ff91f534a15c26 +size 327054 diff --git a/docs/20260915_171525/20260915_171525/depth/000133.png b/docs/20260915_171525/20260915_171525/depth/000133.png new file mode 100644 index 0000000..c896a8d --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000133.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:c2ba3f1a066a9f9d7a739e083082bd3fc25f18b0858c1a4dec08d72b06fe06bd +size 327517 diff --git a/docs/20260915_171525/20260915_171525/depth/000134.png b/docs/20260915_171525/20260915_171525/depth/000134.png new file mode 100644 index 0000000..aab705f --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000134.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:48e2080bcda77ad467a90526566e29dfac31368f71c5c11e4a9595c03c35a601 +size 324287 diff --git a/docs/20260915_171525/20260915_171525/depth/000135.png b/docs/20260915_171525/20260915_171525/depth/000135.png new file mode 100644 index 0000000..a867452 --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000135.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:0d4e3d47960b9642c9aaee1cbb9e52e57005a5a7ae3f7e41aa53a4217ad37360 +size 325639 diff --git a/docs/20260915_171525/20260915_171525/depth/000136.png b/docs/20260915_171525/20260915_171525/depth/000136.png new file mode 100644 index 0000000..a1926ef --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000136.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:b76e25f161507c0266d642015815c82f96c05e3cce6144327adf057cb1a3cf35 +size 329695 diff --git a/docs/20260915_171525/20260915_171525/depth/000137.png b/docs/20260915_171525/20260915_171525/depth/000137.png new file mode 100644 index 0000000..e1ccfd2 --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000137.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:e7621aa9a4fdab5ec4d1a8f6a212f211f2700b2959ca6a4672a72387a1c1871b +size 341721 diff --git a/docs/20260915_171525/20260915_171525/depth/000138.png b/docs/20260915_171525/20260915_171525/depth/000138.png new file mode 100644 index 0000000..33f3887 --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000138.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:0c2241e7a7da5f8492d936e22b5b8f7180713af917f0e93be22afabe6f54cac5 +size 331182 diff --git a/docs/20260915_171525/20260915_171525/depth/000139.png b/docs/20260915_171525/20260915_171525/depth/000139.png new file mode 100644 index 0000000..6f6f17d --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000139.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:569e754145f71a825efceee82b9d1e2cb6970eaca97483e0a4475d5f393c9f8d +size 331074 diff --git a/docs/20260915_171525/20260915_171525/depth/000140.png b/docs/20260915_171525/20260915_171525/depth/000140.png new file mode 100644 index 0000000..5748005 --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000140.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:68789365b613905a7726b04b026d04a16a3beecd189f1e0de70e58dde2767761 +size 334671 diff --git a/docs/20260915_171525/20260915_171525/depth/000141.png b/docs/20260915_171525/20260915_171525/depth/000141.png new file mode 100644 index 0000000..1bc0844 --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000141.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:14fb38ad1c180798921dcdbd53e72d43f3c8d3ff7187561273abbf1e81255415 +size 328554 diff --git a/docs/20260915_171525/20260915_171525/depth/000142.png b/docs/20260915_171525/20260915_171525/depth/000142.png new file mode 100644 index 0000000..b32b84b --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000142.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:40530938d432287661c70347f64ff0d43ff800dfbcf2ab08b5b8a75fd9868912 +size 334442 diff --git a/docs/20260915_171525/20260915_171525/depth/000143.png b/docs/20260915_171525/20260915_171525/depth/000143.png new file mode 100644 index 0000000..ad541fe --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000143.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:1b45dd4c8b91800677b2e6c132df60acac9d14d46201a0c8be23c3238941812d +size 324431 diff --git a/docs/20260915_171525/20260915_171525/depth/000144.png b/docs/20260915_171525/20260915_171525/depth/000144.png new file mode 100644 index 0000000..83ac615 --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000144.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:56d550a0076c30ef14231394a3742014aadb9c53b3fe3408fd081f0138a19594 +size 333249 diff --git a/docs/20260915_171525/20260915_171525/depth/000145.png b/docs/20260915_171525/20260915_171525/depth/000145.png new file mode 100644 index 0000000..e262af4 --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000145.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:110c43d218c4c8c67a8e9cafd93e7e70e0f1428ba7be6a80308970daaa46b2e3 +size 326788 diff --git a/docs/20260915_171525/20260915_171525/depth/000146.png b/docs/20260915_171525/20260915_171525/depth/000146.png new file mode 100644 index 0000000..2a1cd62 --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000146.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:7a3ce862a6e83b97b1d934478965e50470619260f8aec2dd0fd7a50d2ee551b9 +size 334412 diff --git a/docs/20260915_171525/20260915_171525/depth/000147.png b/docs/20260915_171525/20260915_171525/depth/000147.png new file mode 100644 index 0000000..29edce0 --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000147.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:8f1f1ae1ce7228dcf2bfc3691e44c765298daa37c7d5d80f20a2832b5765ef60 +size 331338 diff --git a/docs/20260915_171525/20260915_171525/depth/000148.png b/docs/20260915_171525/20260915_171525/depth/000148.png new file mode 100644 index 0000000..b8892c8 --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000148.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:2441cb8f3b19209d61d5437934dba9eb337d551d64957be09ac0a226d94839d4 +size 333531 diff --git a/docs/20260915_171525/20260915_171525/depth/000149.png b/docs/20260915_171525/20260915_171525/depth/000149.png new file mode 100644 index 0000000..c8dda61 --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000149.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:c8394057b3529e5829ac8cd3cab79e64308315dd7bbcd3c3a271d09d937b5246 +size 332892 diff --git a/docs/20260915_171525/20260915_171525/depth/000150.png b/docs/20260915_171525/20260915_171525/depth/000150.png new file mode 100644 index 0000000..3cca76b --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000150.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:5df514a33644f066a35ccb95b656f6386548a9a1fd1726cdb7f22b201771979e +size 327617 diff --git a/docs/20260915_171525/20260915_171525/depth/000151.png b/docs/20260915_171525/20260915_171525/depth/000151.png new file mode 100644 index 0000000..e1cc137 --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000151.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:e9df5956e4ab238b8f03a50701228802fc453ad463e85bcc2dae3118016316cd +size 333403 diff --git a/docs/20260915_171525/20260915_171525/depth/000152.png b/docs/20260915_171525/20260915_171525/depth/000152.png new file mode 100644 index 0000000..e773820 --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000152.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:c82c5c1972627860e81ca4acf0ca55813c04b3e7cdf65c94374e3651209030a3 +size 337265 diff --git a/docs/20260915_171525/20260915_171525/depth/000153.png b/docs/20260915_171525/20260915_171525/depth/000153.png new file mode 100644 index 0000000..dafc3b4 --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000153.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:f9454bad7b6899cc45fa7cab7812bd023c81e8ecd8beba4c177d28e853c25460 +size 330260 diff --git a/docs/20260915_171525/20260915_171525/depth/000154.png b/docs/20260915_171525/20260915_171525/depth/000154.png new file mode 100644 index 0000000..bbad091 --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000154.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:7e1aea1a6c3487b3dbd96205b4d6360a09f0d977d1046822235b01614f862438 +size 336906 diff --git a/docs/20260915_171525/20260915_171525/depth/000155.png b/docs/20260915_171525/20260915_171525/depth/000155.png new file mode 100644 index 0000000..6d2d74c --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000155.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:d3ca6691693496b50d8a0cdbf78922d473054ef7bd28301280371bea8becd515 +size 332993 diff --git a/docs/20260915_171525/20260915_171525/depth/000156.png b/docs/20260915_171525/20260915_171525/depth/000156.png new file mode 100644 index 0000000..38c7d61 --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000156.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:179758c652ddb616b8a562f5da09e33367f953ad0c9da05ac522a189d24e3f50 +size 336608 diff --git a/docs/20260915_171525/20260915_171525/depth/000157.png b/docs/20260915_171525/20260915_171525/depth/000157.png new file mode 100644 index 0000000..b4f9748 --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000157.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:ecfe568145679367dc076cdc3538d386968cd6cb9c177560d12c8092a351f2ab +size 326314 diff --git a/docs/20260915_171525/20260915_171525/depth/000158.png b/docs/20260915_171525/20260915_171525/depth/000158.png new file mode 100644 index 0000000..ccf5f29 --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000158.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:68ff6807da5c5fe746833a1f5b17805d343c0b7a036c59f2b09acf21158c1269 +size 326150 diff --git a/docs/20260915_171525/20260915_171525/depth/000159.png b/docs/20260915_171525/20260915_171525/depth/000159.png new file mode 100644 index 0000000..1d9e6bb --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000159.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:c3c60d4308ec25e06d78eb0ade2097a54354d9a0789716f0b542ff652b64e40a +size 334963 diff --git a/docs/20260915_171525/20260915_171525/depth/000160.png b/docs/20260915_171525/20260915_171525/depth/000160.png new file mode 100644 index 0000000..f4230e1 --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000160.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:3620426ee5812a40659c085df34cc0730e599ba4b86c81e3fd90d14442bceb98 +size 342649 diff --git a/docs/20260915_171525/20260915_171525/depth/000161.png b/docs/20260915_171525/20260915_171525/depth/000161.png new file mode 100644 index 0000000..fd7998b --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000161.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:04bc5561e345ba7db3cc723159541b8fea6f866e60c5c3035ad6e58b395553d9 +size 328437 diff --git a/docs/20260915_171525/20260915_171525/depth/000162.png b/docs/20260915_171525/20260915_171525/depth/000162.png new file mode 100644 index 0000000..2bcc183 --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000162.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:2e7aa7be91d87a5923c5bb4a3e9d8afc85a8675b6fa0d395b1679bcf636e4520 +size 332938 diff --git a/docs/20260915_171525/20260915_171525/depth/000163.png b/docs/20260915_171525/20260915_171525/depth/000163.png new file mode 100644 index 0000000..86594e9 --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000163.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:b79a0279a095e8cd29437883b0f3f042919eb2c177fb790c2263037df066a5a2 +size 335648 diff --git a/docs/20260915_171525/20260915_171525/depth/000164.png b/docs/20260915_171525/20260915_171525/depth/000164.png new file mode 100644 index 0000000..2f3524a --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000164.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:82a67d40f49451c8bb75c76f3f5df7afb452b0129d18ac6f0f2c7b9801899e07 +size 335803 diff --git a/docs/20260915_171525/20260915_171525/depth/000165.png b/docs/20260915_171525/20260915_171525/depth/000165.png new file mode 100644 index 0000000..e032079 --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000165.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:c3cc0d25e0b0b4aedc3c6816563f3d5d811f8adcbc5d4bb1b6d7e2ccf7ba33a6 +size 333527 diff --git a/docs/20260915_171525/20260915_171525/depth/000166.png b/docs/20260915_171525/20260915_171525/depth/000166.png new file mode 100644 index 0000000..2c64192 --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000166.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:4be22868201e9ad409b15af3871be21d006a76d20f268dad283286ec37a70174 +size 335795 diff --git a/docs/20260915_171525/20260915_171525/depth/000167.png b/docs/20260915_171525/20260915_171525/depth/000167.png new file mode 100644 index 0000000..baef7f8 --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000167.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:d5bbc4b92a27949747055f3fbaa1f91d74af7aa26bd24a67e4383deca083a00e +size 332407 diff --git a/docs/20260915_171525/20260915_171525/depth/000168.png b/docs/20260915_171525/20260915_171525/depth/000168.png new file mode 100644 index 0000000..d7387b4 --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000168.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:0fb3263982ea79350f9ff7687a45237b14a43bf94e477492e1653e52d769ea0d +size 336554 diff --git a/docs/20260915_171525/20260915_171525/depth/000169.png b/docs/20260915_171525/20260915_171525/depth/000169.png new file mode 100644 index 0000000..df47924 --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000169.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:4cfb900dcfffdf79038776d36739492970f7895c80e9d916dc1e80c2ae8ce74a +size 328142 diff --git a/docs/20260915_171525/20260915_171525/depth/000170.png b/docs/20260915_171525/20260915_171525/depth/000170.png new file mode 100644 index 0000000..4b6dad8 --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000170.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:e1a27988f69653da5c1cc23d94c8f21c4804871cb8b29d726d8256ee3da5dda4 +size 331689 diff --git a/docs/20260915_171525/20260915_171525/depth/000171.png b/docs/20260915_171525/20260915_171525/depth/000171.png new file mode 100644 index 0000000..4a9d697 --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000171.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:87ba4832c3ff416e5b329b6fb42d61717500a6e0771feb42eb0acbf8a21ef4a9 +size 326437 diff --git a/docs/20260915_171525/20260915_171525/depth/000172.png b/docs/20260915_171525/20260915_171525/depth/000172.png new file mode 100644 index 0000000..e1f1965 --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000172.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:d5e8614d0dcb778966ea249347df41f867b771c3da8c8c7fc2192eae5caf073b +size 319090 diff --git a/docs/20260915_171525/20260915_171525/depth/000173.png b/docs/20260915_171525/20260915_171525/depth/000173.png new file mode 100644 index 0000000..623e4a5 --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000173.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:49b7117c55c095488a668d05bc2eabd0a4909630735f70498b3c336020479d8c +size 325674 diff --git a/docs/20260915_171525/20260915_171525/depth/000174.png b/docs/20260915_171525/20260915_171525/depth/000174.png new file mode 100644 index 0000000..078f481 --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000174.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:35c4190e641a076dd91b96d2a948e632872b6616421c8efbee538b85a2d68e7e +size 336349 diff --git a/docs/20260915_171525/20260915_171525/depth/000175.png b/docs/20260915_171525/20260915_171525/depth/000175.png new file mode 100644 index 0000000..e8bfb52 --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000175.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:684dbac39da6dad5b1f51c526e15337a4f98599287f2bd19c8c30ba54796765d +size 333999 diff --git a/docs/20260915_171525/20260915_171525/depth/000176.png b/docs/20260915_171525/20260915_171525/depth/000176.png new file mode 100644 index 0000000..4d33b52 --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000176.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:4c5bee6e09ade833bb80f0f2e17feb2048db506569bb78a1f22203d5ef13f24a +size 328702 diff --git a/docs/20260915_171525/20260915_171525/depth/000177.png b/docs/20260915_171525/20260915_171525/depth/000177.png new file mode 100644 index 0000000..ce19a6e --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000177.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:620697d0d357cb6409a461ebe8491b34968494f4086256ebc35e95d03c48514f +size 326273 diff --git a/docs/20260915_171525/20260915_171525/depth/000178.png b/docs/20260915_171525/20260915_171525/depth/000178.png new file mode 100644 index 0000000..27709f6 --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000178.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:af1374bfbc6ec9bea6b75b52f6a9a38c75ba0bd150a9a1310913f5a69a8152cb +size 324540 diff --git a/docs/20260915_171525/20260915_171525/depth/000179.png b/docs/20260915_171525/20260915_171525/depth/000179.png new file mode 100644 index 0000000..60d1c9f --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000179.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:84eb63913c4d3411ec2f68b3b60ddd064aaee01a47001d9e2cd6da2f3becf652 +size 328422 diff --git a/docs/20260915_171525/20260915_171525/depth/000180.png b/docs/20260915_171525/20260915_171525/depth/000180.png new file mode 100644 index 0000000..614d211 --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000180.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:55d2c4e118744c48848a8977cf5673e0e5dc5878f371260f20e4addc59ed23d8 +size 320641 diff --git a/docs/20260915_171525/20260915_171525/depth/000181.png b/docs/20260915_171525/20260915_171525/depth/000181.png new file mode 100644 index 0000000..ccc4214 --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000181.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:a5be18cec3a9f364844a73ee4aae4b1e3b194c7da7c45be486db2827363a575c +size 331661 diff --git a/docs/20260915_171525/20260915_171525/depth/000182.png b/docs/20260915_171525/20260915_171525/depth/000182.png new file mode 100644 index 0000000..17bfde1 --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000182.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:d78b47a60ae6c5a59709eb6bd3f84bc54fdf110e36f70dec44869205b65fdeeb +size 330312 diff --git a/docs/20260915_171525/20260915_171525/depth/000183.png b/docs/20260915_171525/20260915_171525/depth/000183.png new file mode 100644 index 0000000..a000359 --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000183.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:b6f0e32e252e8c7d5c056efb671aaed7d9b90325ae3804fdec3b8547f8b0cfc3 +size 331977 diff --git a/docs/20260915_171525/20260915_171525/depth/000184.png b/docs/20260915_171525/20260915_171525/depth/000184.png new file mode 100644 index 0000000..894ed12 --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000184.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:45b17ab5b776d3dbd0cc9690786313f70bedea97b69ce6efce26c751e9b14aef +size 318716 diff --git a/docs/20260915_171525/20260915_171525/depth/000185.png b/docs/20260915_171525/20260915_171525/depth/000185.png new file mode 100644 index 0000000..3d5f5ff --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000185.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:0b6e3354594762099e56e88bc79f80c08375b3ead32d8330cad5b72d638e5090 +size 329445 diff --git a/docs/20260915_171525/20260915_171525/depth/000186.png b/docs/20260915_171525/20260915_171525/depth/000186.png new file mode 100644 index 0000000..1503747 --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000186.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:72b5aeee7201afe7682fa2c543d92df677b908ff92e3899f7d956e99a4ca20d5 +size 334010 diff --git a/docs/20260915_171525/20260915_171525/depth/000187.png b/docs/20260915_171525/20260915_171525/depth/000187.png new file mode 100644 index 0000000..58d97fb --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000187.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:95ade8a5003788509ab92f91b86d568c37aba79bd9599bb0ace23cbe711db778 +size 331826 diff --git a/docs/20260915_171525/20260915_171525/depth/000188.png b/docs/20260915_171525/20260915_171525/depth/000188.png new file mode 100644 index 0000000..5b8c62c --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000188.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:97cf3b3db850b944edc6f32201b11229003d678ca1a4169a6e66ea079f1483ee +size 330885 diff --git a/docs/20260915_171525/20260915_171525/depth/000189.png b/docs/20260915_171525/20260915_171525/depth/000189.png new file mode 100644 index 0000000..378223d --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000189.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:22fc6749ac321bc54bd0c642d832cfad2bec0a3e7dfd127bb7385c8c118aa798 +size 322507 diff --git a/docs/20260915_171525/20260915_171525/depth/000190.png b/docs/20260915_171525/20260915_171525/depth/000190.png new file mode 100644 index 0000000..1f18154 --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000190.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:885cf8695bcdfc156d60b30a18c145b109137013f0bcc9d785edcc5117d1dfe3 +size 330000 diff --git a/docs/20260915_171525/20260915_171525/depth/000191.png b/docs/20260915_171525/20260915_171525/depth/000191.png new file mode 100644 index 0000000..776c5c9 --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000191.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:0c3a880d5618d2af4557337c71c8ac72004ab0ae1ef04ce1a44dabeb81d26fc3 +size 343382 diff --git a/docs/20260915_171525/20260915_171525/depth/000192.png b/docs/20260915_171525/20260915_171525/depth/000192.png new file mode 100644 index 0000000..514d306 --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000192.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:c8a773d7ccffcd74f283e3f9b17042b1d4386b66ffe3281182409485e0829cc9 +size 324812 diff --git a/docs/20260915_171525/20260915_171525/depth/000193.png b/docs/20260915_171525/20260915_171525/depth/000193.png new file mode 100644 index 0000000..9125cd9 --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000193.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:da598b244eb5b3741b35e00830ffae166f81d8042ea5d0480c12d86ab50f22fd +size 323324 diff --git a/docs/20260915_171525/20260915_171525/depth/000194.png b/docs/20260915_171525/20260915_171525/depth/000194.png new file mode 100644 index 0000000..2e9f431 --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000194.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:cb933266fd689dd60950becd989f60e8725dde63279b657499fd2c6d130d765c +size 339008 diff --git a/docs/20260915_171525/20260915_171525/depth/000195.png b/docs/20260915_171525/20260915_171525/depth/000195.png new file mode 100644 index 0000000..099930a --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000195.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:ac809cff381215c478f5e55c439815afc22eac071ea8c8f40a82cd643f59f0b9 +size 333033 diff --git a/docs/20260915_171525/20260915_171525/depth/000196.png b/docs/20260915_171525/20260915_171525/depth/000196.png new file mode 100644 index 0000000..3bc1e53 --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000196.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:404949199a71bb01c793495e251a1c540187004beabc90cb8e2386a1cf4fb84c +size 331635 diff --git a/docs/20260915_171525/20260915_171525/depth/000197.png b/docs/20260915_171525/20260915_171525/depth/000197.png new file mode 100644 index 0000000..c052c0a --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000197.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:1ce0b4d1161166eb35f49a9ab3c1b8f278409277f03e1ebc4865d96f7f2743d1 +size 330943 diff --git a/docs/20260915_171525/20260915_171525/depth/000198.png b/docs/20260915_171525/20260915_171525/depth/000198.png new file mode 100644 index 0000000..2cc3375 --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000198.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:6ba0ce35421979fb133f38b02579012301eee9a071fdc50ac0742da33a2b5af9 +size 338409 diff --git a/docs/20260915_171525/20260915_171525/depth/000199.png b/docs/20260915_171525/20260915_171525/depth/000199.png new file mode 100644 index 0000000..e1a4fa7 --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000199.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:864bf3e21bc996b46bcce2ee7a2a432a2e1181755bcca0a22224f92877912927 +size 338236 diff --git a/docs/20260915_171525/20260915_171525/depth/000200.png b/docs/20260915_171525/20260915_171525/depth/000200.png new file mode 100644 index 0000000..254930d --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000200.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:56bf11a5d167025f138038ec7733fe02f4346dabb13d99a158347d15cadc7774 +size 334525 diff --git a/docs/20260915_171525/20260915_171525/depth/000201.png b/docs/20260915_171525/20260915_171525/depth/000201.png new file mode 100644 index 0000000..702175e --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000201.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:60fecce3f6cc986333114fc0ac1a088c90f2342486f901bef9f19a2062bc950f +size 335526 diff --git a/docs/20260915_171525/20260915_171525/depth/000202.png b/docs/20260915_171525/20260915_171525/depth/000202.png new file mode 100644 index 0000000..ee7edce --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000202.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:9146d7262de4b84c9bab033ab61cecd4abd16f3a1b47f98ec5e94beacb0905e7 +size 344306 diff --git a/docs/20260915_171525/20260915_171525/depth/000203.png b/docs/20260915_171525/20260915_171525/depth/000203.png new file mode 100644 index 0000000..14d0925 --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000203.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:5520d111021710a690e96e6d734afe9d5bc8f7bba7ff5173ac2134991c2f5b65 +size 339786 diff --git a/docs/20260915_171525/20260915_171525/depth/000204.png b/docs/20260915_171525/20260915_171525/depth/000204.png new file mode 100644 index 0000000..b3959e3 --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000204.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:a6334587987d68389c382d810837c615ddf3f84232c663511ae343ff36b20041 +size 337361 diff --git a/docs/20260915_171525/20260915_171525/depth/000205.png b/docs/20260915_171525/20260915_171525/depth/000205.png new file mode 100644 index 0000000..7c4bf71 --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000205.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:eb02f67b2504968829906041addeb6d914dbf9ea3b61093f07aa593dbb9707cd +size 336850 diff --git a/docs/20260915_171525/20260915_171525/depth/000206.png b/docs/20260915_171525/20260915_171525/depth/000206.png new file mode 100644 index 0000000..9579281 --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000206.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:5f295a9b904f76217a24d035b0b32a3232521f44b561962e437b4b671bc12b71 +size 340725 diff --git a/docs/20260915_171525/20260915_171525/depth/000207.png b/docs/20260915_171525/20260915_171525/depth/000207.png new file mode 100644 index 0000000..f0f0970 --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000207.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:5d1194787e445fca060618f4c13e1247d984310b49f6015318314391a536e741 +size 342439 diff --git a/docs/20260915_171525/20260915_171525/depth/000208.png b/docs/20260915_171525/20260915_171525/depth/000208.png new file mode 100644 index 0000000..3ae5cfb --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000208.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:9d7403ce70f8026603738c77a6b7a6e10b1cd77b42ff3bfb441b35afe993d835 +size 334014 diff --git a/docs/20260915_171525/20260915_171525/depth/000209.png b/docs/20260915_171525/20260915_171525/depth/000209.png new file mode 100644 index 0000000..242b42c --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000209.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:3e29c24996653c2b27c8033a50c0fb4959b4eb45dc66ae3ccd2e41ed5ff2fc14 +size 331616 diff --git a/docs/20260915_171525/20260915_171525/depth/000210.png b/docs/20260915_171525/20260915_171525/depth/000210.png new file mode 100644 index 0000000..ab0dd8f --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000210.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:5adba0ed6635a0f681bdf72a917b3821e7cbb19b8a1734e0d555f11c3b2e8ecc +size 327568 diff --git a/docs/20260915_171525/20260915_171525/depth/000211.png b/docs/20260915_171525/20260915_171525/depth/000211.png new file mode 100644 index 0000000..c63dd4b --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000211.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:b0f7f84cf0974b2e59ad9f9fb36056ee0a80de52f85f455e58cb73bc4eefba93 +size 337133 diff --git a/docs/20260915_171525/20260915_171525/depth/000212.png b/docs/20260915_171525/20260915_171525/depth/000212.png new file mode 100644 index 0000000..f6c3659 --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000212.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:4ee8c34a504c43ec282e3f408fd323a08f20659dcac43e0484bad4d071b673ed +size 325900 diff --git a/docs/20260915_171525/20260915_171525/depth/000213.png b/docs/20260915_171525/20260915_171525/depth/000213.png new file mode 100644 index 0000000..33a578e --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000213.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:877acb79bfadfd564c4b7b3a2a1de2bed2d8b599738f7a6351eeab7793d5702f +size 341307 diff --git a/docs/20260915_171525/20260915_171525/depth/000214.png b/docs/20260915_171525/20260915_171525/depth/000214.png new file mode 100644 index 0000000..0cbb49f --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000214.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:a6b042f9eeb1c16d08a5c11429c01f3101652a618681286fd7f68c16ed8dfd5d +size 341173 diff --git a/docs/20260915_171525/20260915_171525/depth/000215.png b/docs/20260915_171525/20260915_171525/depth/000215.png new file mode 100644 index 0000000..0aa7d97 --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000215.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:73981702aee5f16011413265314fd4297e75e2727ec2db88bf3a1d8167040e6f +size 333597 diff --git a/docs/20260915_171525/20260915_171525/depth/000216.png b/docs/20260915_171525/20260915_171525/depth/000216.png new file mode 100644 index 0000000..05ec9ca --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000216.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:52ee371a8fd80c45442fddf34377af815cdbf4ea1e814319d6ee9d6a4ef450b6 +size 332194 diff --git a/docs/20260915_171525/20260915_171525/depth/000217.png b/docs/20260915_171525/20260915_171525/depth/000217.png new file mode 100644 index 0000000..489d179 --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000217.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:d14467c53eef4abf99faff3a87f520ed302e8fa2f78dbb8c2a687faa313391d4 +size 347559 diff --git a/docs/20260915_171525/20260915_171525/depth/000218.png b/docs/20260915_171525/20260915_171525/depth/000218.png new file mode 100644 index 0000000..7ca16ef --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000218.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:81a972cc58db4ba15e2737657c6838d58bb69e358c715e97950e6cc260dc48d4 +size 342045 diff --git a/docs/20260915_171525/20260915_171525/depth/000219.png b/docs/20260915_171525/20260915_171525/depth/000219.png new file mode 100644 index 0000000..0ca1825 --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000219.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:dabafb355f1e8befb9a07c3a0205c12ef1ebea92b18afab58b6cbd41c6c4877f +size 322261 diff --git a/docs/20260915_171525/20260915_171525/depth/000220.png b/docs/20260915_171525/20260915_171525/depth/000220.png new file mode 100644 index 0000000..f726cf6 --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000220.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:78134b82bd1b1a87e5bfcabf6e96a234be050c4f3da605a118e98a50464600f4 +size 336617 diff --git a/docs/20260915_171525/20260915_171525/depth/000221.png b/docs/20260915_171525/20260915_171525/depth/000221.png new file mode 100644 index 0000000..ad16eb3 --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000221.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:37601beb55e7bfd27c349a17fc8f9ed355948c4e96c963c119e90b3fc9ae1de4 +size 332607 diff --git a/docs/20260915_171525/20260915_171525/depth/000222.png b/docs/20260915_171525/20260915_171525/depth/000222.png new file mode 100644 index 0000000..3e8e53c --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000222.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:c6df5946e843dae471dd40723b3ba5693336951336c3c475165274620646e381 +size 336830 diff --git a/docs/20260915_171525/20260915_171525/depth/000223.png b/docs/20260915_171525/20260915_171525/depth/000223.png new file mode 100644 index 0000000..98e3a88 --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000223.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:600574fd2dbd1e7129b870ad25402dbfda4fa8b3bc6d1f09815e74d36a6df100 +size 331512 diff --git a/docs/20260915_171525/20260915_171525/depth/000224.png b/docs/20260915_171525/20260915_171525/depth/000224.png new file mode 100644 index 0000000..1831a42 --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000224.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:7ca113ae3077fe9f1fbd9ced052d7aeff3da51291c86dbca581cf11b237fdadf +size 327698 diff --git a/docs/20260915_171525/20260915_171525/depth/000225.png b/docs/20260915_171525/20260915_171525/depth/000225.png new file mode 100644 index 0000000..f8c0413 --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000225.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:693abb7195e480cdddd5551ef77984b8317bddc3ac9fed336ab32b4818ade9ea +size 327685 diff --git a/docs/20260915_171525/20260915_171525/depth/000226.png b/docs/20260915_171525/20260915_171525/depth/000226.png new file mode 100644 index 0000000..57eb151 --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000226.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:23a54b45ca29388877429a4fb974da2caee3f990d137fa562880f948ad60c44e +size 333812 diff --git a/docs/20260915_171525/20260915_171525/depth/000227.png b/docs/20260915_171525/20260915_171525/depth/000227.png new file mode 100644 index 0000000..b7cbda3 --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000227.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:ab04f1894cf3e19c433f40c85b5cc76735aed3cfb64fd7bd01ab394d0f481e0a +size 329480 diff --git a/docs/20260915_171525/20260915_171525/depth/000228.png b/docs/20260915_171525/20260915_171525/depth/000228.png new file mode 100644 index 0000000..6642620 --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000228.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:da53ad7b9de8ac61732c0ce4fdb2536db8ea5c6af2018f28a3c410b9c44e22a0 +size 333126 diff --git a/docs/20260915_171525/20260915_171525/depth/000229.png b/docs/20260915_171525/20260915_171525/depth/000229.png new file mode 100644 index 0000000..00a8fce --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000229.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:307c37b530cae97e9e9511df2ed21fee95637d88bd4d2c05c98fa9a6514d0c78 +size 337457 diff --git a/docs/20260915_171525/20260915_171525/depth/000230.png b/docs/20260915_171525/20260915_171525/depth/000230.png new file mode 100644 index 0000000..2c3b84a --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000230.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:aed1f75d3c3c04209aa6a82a335e4f73c0eeb9fa2efb93c9067b467d4fe439c6 +size 321232 diff --git a/docs/20260915_171525/20260915_171525/depth/000231.png b/docs/20260915_171525/20260915_171525/depth/000231.png new file mode 100644 index 0000000..4daaa32 --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000231.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:eea25cfe4152d633a274f6c305fd5067df3dbb8a2242046f349335dbd1f91d2d +size 327770 diff --git a/docs/20260915_171525/20260915_171525/depth/000232.png b/docs/20260915_171525/20260915_171525/depth/000232.png new file mode 100644 index 0000000..2f9051c --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000232.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:7d60387c58065d7f7f8dcc2716730656e6535784c6f7161b4bab291b529152b4 +size 324600 diff --git a/docs/20260915_171525/20260915_171525/depth/000233.png b/docs/20260915_171525/20260915_171525/depth/000233.png new file mode 100644 index 0000000..7704a7c --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000233.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:d6d7784885b1114914e42d6ef3f80c7c9ee72e154fe5ddf4203dfbc529b0e464 +size 332118 diff --git a/docs/20260915_171525/20260915_171525/depth/000234.png b/docs/20260915_171525/20260915_171525/depth/000234.png new file mode 100644 index 0000000..9813f8c --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000234.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:19f2951bb89d1fd6a02dd5afd3bc8d749921e787f0e43821fe8c9188dc2fdfb0 +size 331965 diff --git a/docs/20260915_171525/20260915_171525/depth/000235.png b/docs/20260915_171525/20260915_171525/depth/000235.png new file mode 100644 index 0000000..accd750 --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000235.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:8b517747551107930dead16c2bd2c592c15e68a73d4dd1cc261cf353415e02e3 +size 333739 diff --git a/docs/20260915_171525/20260915_171525/depth/000236.png b/docs/20260915_171525/20260915_171525/depth/000236.png new file mode 100644 index 0000000..1b299f8 --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000236.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:ff1db182687a831b9ef343025d5bebbc56cd6bf5941c27b1f98cc818b21123e2 +size 330192 diff --git a/docs/20260915_171525/20260915_171525/depth/000237.png b/docs/20260915_171525/20260915_171525/depth/000237.png new file mode 100644 index 0000000..b799312 --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000237.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:4b84253cbee33b2cce9eccb2db0389add9400329640020f16f784e7cdd1af8e1 +size 336667 diff --git a/docs/20260915_171525/20260915_171525/depth/000238.png b/docs/20260915_171525/20260915_171525/depth/000238.png new file mode 100644 index 0000000..f3a3056 --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000238.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:236e1b1a5d559ceab56793c634318af842dea16c7c4dc426438e9664af10393f +size 329451 diff --git a/docs/20260915_171525/20260915_171525/depth/000239.png b/docs/20260915_171525/20260915_171525/depth/000239.png new file mode 100644 index 0000000..55de346 --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000239.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:711a4296de42740eedb87ea9814a0a4ae9d43524167564daebdfbfd91ea4568d +size 329714 diff --git a/docs/20260915_171525/20260915_171525/depth/000240.png b/docs/20260915_171525/20260915_171525/depth/000240.png new file mode 100644 index 0000000..821634b --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000240.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:ebbf2258d1ce523d4992863fd6f9ab9fa987daf9a45d386fa21d57855cb7aa7a +size 337275 diff --git a/docs/20260915_171525/20260915_171525/depth/000241.png b/docs/20260915_171525/20260915_171525/depth/000241.png new file mode 100644 index 0000000..ab6d613 --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000241.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:bd5ade8df29ea663621fd8a43b44be1161cddcc5bbb276c017a1751b19ef877e +size 332832 diff --git a/docs/20260915_171525/20260915_171525/depth/000242.png b/docs/20260915_171525/20260915_171525/depth/000242.png new file mode 100644 index 0000000..0635f31 --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000242.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:8a2ee760f8e5c7f6d1170ac5fb974d40d06e606c72355f047d4e4ec0746f2313 +size 336182 diff --git a/docs/20260915_171525/20260915_171525/depth/000243.png b/docs/20260915_171525/20260915_171525/depth/000243.png new file mode 100644 index 0000000..07ca347 --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000243.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:9e776f037bbd5814562114ec96ec90bb0b5fee59e42526d4c68b3d40fad677b9 +size 323470 diff --git a/docs/20260915_171525/20260915_171525/depth/000244.png b/docs/20260915_171525/20260915_171525/depth/000244.png new file mode 100644 index 0000000..7501c8f --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000244.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:ca9bbdeb12a45f35a41a42a1f57f071b4f5f6c49bbd52a22af95ce1902d42c0d +size 327958 diff --git a/docs/20260915_171525/20260915_171525/depth/000245.png b/docs/20260915_171525/20260915_171525/depth/000245.png new file mode 100644 index 0000000..2c10012 --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000245.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:359a74eed588d5e66aa0fe17bb1bb8f7cccb2e78367d5c6d128f52d003fb4c86 +size 332216 diff --git a/docs/20260915_171525/20260915_171525/depth/000246.png b/docs/20260915_171525/20260915_171525/depth/000246.png new file mode 100644 index 0000000..8e8487b --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000246.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:052ea9ad871c2b200da85793d5590b2529cc695172138492a301c0fb006b6166 +size 333155 diff --git a/docs/20260915_171525/20260915_171525/depth/000247.png b/docs/20260915_171525/20260915_171525/depth/000247.png new file mode 100644 index 0000000..be0c548 --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000247.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:0d5ac745ba5920724a1cb5443aa55760f56d7e543732f021880ca39c857d65a5 +size 319274 diff --git a/docs/20260915_171525/20260915_171525/depth/000248.png b/docs/20260915_171525/20260915_171525/depth/000248.png new file mode 100644 index 0000000..108a7a8 --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000248.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:722caac2b6e968a6dd716d6b22cfeea797586504ec50b80c0dcc5dee8c38af86 +size 335742 diff --git a/docs/20260915_171525/20260915_171525/depth/000249.png b/docs/20260915_171525/20260915_171525/depth/000249.png new file mode 100644 index 0000000..fba3de1 --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000249.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:c524f2ad0e4e4019a15f1dadd9c3fde27fd25570c2272e325a28de2d90689a5b +size 331237 diff --git a/docs/20260915_171525/20260915_171525/depth/000250.png b/docs/20260915_171525/20260915_171525/depth/000250.png new file mode 100644 index 0000000..519ad23 --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000250.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:cbe4c6d7de9578279de97b0821450c782e944a7f33e5c442bd67a6bb24ddbb91 +size 334409 diff --git a/docs/20260915_171525/20260915_171525/depth/000251.png b/docs/20260915_171525/20260915_171525/depth/000251.png new file mode 100644 index 0000000..8b7fba5 --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000251.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:14559aa8c915afd754a084f4261bc7991a55f98cf98085c4baf1fd54cebc7e34 +size 341568 diff --git a/docs/20260915_171525/20260915_171525/depth/000252.png b/docs/20260915_171525/20260915_171525/depth/000252.png new file mode 100644 index 0000000..2895764 --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000252.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:aaa5cd93168d48db4c67c31fc98dbddc255135d3aa53b44b37825a2f7820b421 +size 332619 diff --git a/docs/20260915_171525/20260915_171525/depth/000253.png b/docs/20260915_171525/20260915_171525/depth/000253.png new file mode 100644 index 0000000..e8a7428 --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000253.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:8c3ef01bfdbfa35c1a97c5622e01d39814762efb57d239bd43f1c88e9f04e962 +size 335338 diff --git a/docs/20260915_171525/20260915_171525/depth/000254.png b/docs/20260915_171525/20260915_171525/depth/000254.png new file mode 100644 index 0000000..25112ad --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000254.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:011cea48efdce6ffb171c20de3d7d44c80467bfcce472189286724829597065e +size 338771 diff --git a/docs/20260915_171525/20260915_171525/depth/000255.png b/docs/20260915_171525/20260915_171525/depth/000255.png new file mode 100644 index 0000000..fb25509 --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000255.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:36dc94d612ffe20f90379844886ee22f20ba2947c0155a7bd85fe51de57b81ea +size 332064 diff --git a/docs/20260915_171525/20260915_171525/depth/000256.png b/docs/20260915_171525/20260915_171525/depth/000256.png new file mode 100644 index 0000000..1196075 --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000256.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:e2f371d180190bc0a70c18226cb4f06958fed3f4e62d6db8136b17841683808f +size 326230 diff --git a/docs/20260915_171525/20260915_171525/depth/000257.png b/docs/20260915_171525/20260915_171525/depth/000257.png new file mode 100644 index 0000000..9496e8f --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000257.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:02542e9973982c59e7adc8af2a2afb3afb99af34a0821d4d1df95b0f78e3339f +size 321664 diff --git a/docs/20260915_171525/20260915_171525/depth/000258.png b/docs/20260915_171525/20260915_171525/depth/000258.png new file mode 100644 index 0000000..c849b9b --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000258.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:7239744a88161b87335703e544dab435eaaa5c90c17c7c9c8dccaf7d30fba5e4 +size 340132 diff --git a/docs/20260915_171525/20260915_171525/depth/000259.png b/docs/20260915_171525/20260915_171525/depth/000259.png new file mode 100644 index 0000000..31b7902 --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000259.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:bd562dc769bc3572ed4627ef008400024c3eb2ea2e098fb1494b6ecad8e715aa +size 337472 diff --git a/docs/20260915_171525/20260915_171525/depth/000260.png b/docs/20260915_171525/20260915_171525/depth/000260.png new file mode 100644 index 0000000..5f42c72 --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000260.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:dac018cc520c89d8abf571f6514fb6893a4a1b2fbef05644cd4a8b5fdc8054b2 +size 334334 diff --git a/docs/20260915_171525/20260915_171525/depth/000261.png b/docs/20260915_171525/20260915_171525/depth/000261.png new file mode 100644 index 0000000..c64ad55 --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000261.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:31a653934882423daf578f64ca679c7e4858a28b9ae3a1690b8f3a472a557bea +size 328342 diff --git a/docs/20260915_171525/20260915_171525/depth/000262.png b/docs/20260915_171525/20260915_171525/depth/000262.png new file mode 100644 index 0000000..dd23b22 --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000262.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:e3ca0769cb354e53228e00184d3bd14fac9716d189e071f375c636cf5c7791eb +size 326430 diff --git a/docs/20260915_171525/20260915_171525/depth/000263.png b/docs/20260915_171525/20260915_171525/depth/000263.png new file mode 100644 index 0000000..be05101 --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000263.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:395490dfffad705620d2e1f3050e75859db66b0e27e81e6276acbdbc9a2d4ce5 +size 330371 diff --git a/docs/20260915_171525/20260915_171525/depth/000264.png b/docs/20260915_171525/20260915_171525/depth/000264.png new file mode 100644 index 0000000..5c44773 --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000264.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:59872066522eb3159a8951fd2bc3736722932afc016026e39d6e42cc75b9b82a +size 322898 diff --git a/docs/20260915_171525/20260915_171525/depth/000265.png b/docs/20260915_171525/20260915_171525/depth/000265.png new file mode 100644 index 0000000..b24890e --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000265.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:c3e6d2daffdaeeca925dee96d5140c327d7b1b0189931eeb4bda2a75508a991f +size 319610 diff --git a/docs/20260915_171525/20260915_171525/depth/000266.png b/docs/20260915_171525/20260915_171525/depth/000266.png new file mode 100644 index 0000000..d49afc9 --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000266.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:751c34d61257af531287c5f6982e834b4f8049c1c0315eecf2e5ba1b6b022293 +size 319250 diff --git a/docs/20260915_171525/20260915_171525/depth/000267.png b/docs/20260915_171525/20260915_171525/depth/000267.png new file mode 100644 index 0000000..3461b0c --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000267.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:c2a7a53be65dcade64119d88c36d89d3b03ddea64b990c97c4701c80420b9170 +size 327744 diff --git a/docs/20260915_171525/20260915_171525/depth/000268.png b/docs/20260915_171525/20260915_171525/depth/000268.png new file mode 100644 index 0000000..46ecd26 --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000268.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:5b2c76e2b4f373df8e4f63e106893adc26118cf26d8cb6392c434d1eb68154f4 +size 326219 diff --git a/docs/20260915_171525/20260915_171525/depth/000269.png b/docs/20260915_171525/20260915_171525/depth/000269.png new file mode 100644 index 0000000..539f0a1 --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000269.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:1642f9d854951cb6d3fc931870dad59aa25d807acc08c9c9d2f1bc64b3301346 +size 338857 diff --git a/docs/20260915_171525/20260915_171525/depth/000270.png b/docs/20260915_171525/20260915_171525/depth/000270.png new file mode 100644 index 0000000..62e6c34 --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000270.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:b8364209adff7a1cebb33918d76e7fb1691936dc59ce9c47dd5bf6fa684a5158 +size 331363 diff --git a/docs/20260915_171525/20260915_171525/depth/000271.png b/docs/20260915_171525/20260915_171525/depth/000271.png new file mode 100644 index 0000000..6c7d30b --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000271.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:1ad036269a3562abd20a06ee6249a6bc2ba55aea4a2c3243512fe704f9c1ab45 +size 332042 diff --git a/docs/20260915_171525/20260915_171525/depth/000272.png b/docs/20260915_171525/20260915_171525/depth/000272.png new file mode 100644 index 0000000..37562c7 --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000272.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:812d6d3ca1489337a3cd798a857e7ddd407896ca2bd86b7151b16cf650a8d66b +size 336032 diff --git a/docs/20260915_171525/20260915_171525/depth/000273.png b/docs/20260915_171525/20260915_171525/depth/000273.png new file mode 100644 index 0000000..d74675e --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000273.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:880a1a1f6906b2fdb64a99cc1cd92ec0e1ef88bb318bbabf77948eeb1f1ef138 +size 322749 diff --git a/docs/20260915_171525/20260915_171525/depth/000274.png b/docs/20260915_171525/20260915_171525/depth/000274.png new file mode 100644 index 0000000..8f245a5 --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000274.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:fd2e9b90f311d1631b6592409cb2018240673bbcd291cb00a880345cf1f7bf4a +size 331069 diff --git a/docs/20260915_171525/20260915_171525/depth/000275.png b/docs/20260915_171525/20260915_171525/depth/000275.png new file mode 100644 index 0000000..9347a7a --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000275.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:9004ec42dae72f2d61a8765d70351ae6dee8c6b29df9025e92ad92d73bd1412b +size 335024 diff --git a/docs/20260915_171525/20260915_171525/depth/000276.png b/docs/20260915_171525/20260915_171525/depth/000276.png new file mode 100644 index 0000000..e95c375 --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000276.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:8bea5306675a7c5f718aed32107409b6d62ace8bc2703076bc3239fabdc3224d +size 337313 diff --git a/docs/20260915_171525/20260915_171525/depth/000277.png b/docs/20260915_171525/20260915_171525/depth/000277.png new file mode 100644 index 0000000..05b4687 --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000277.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:e2097cb40a17c8059ed4681c268094535af971bd647daefb3725839c72aefb9d +size 343624 diff --git a/docs/20260915_171525/20260915_171525/depth/000278.png b/docs/20260915_171525/20260915_171525/depth/000278.png new file mode 100644 index 0000000..5a8318a --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000278.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:7f43cef00330403f320fff6cb111b490ab648652b46cee2b4d9bfeb263310a3d +size 333550 diff --git a/docs/20260915_171525/20260915_171525/depth/000279.png b/docs/20260915_171525/20260915_171525/depth/000279.png new file mode 100644 index 0000000..261e59c --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000279.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:31d76ed50ec2ea472c73b1f72c316cf84e2a4c302502104f11d8f6fcedb0a266 +size 326901 diff --git a/docs/20260915_171525/20260915_171525/depth/000280.png b/docs/20260915_171525/20260915_171525/depth/000280.png new file mode 100644 index 0000000..8da915c --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000280.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:13bc4839c69efbcc418bede819e27390d80d53e1746a376203b845e8372db64d +size 334886 diff --git a/docs/20260915_171525/20260915_171525/depth/000281.png b/docs/20260915_171525/20260915_171525/depth/000281.png new file mode 100644 index 0000000..8a18634 --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000281.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:deec20393ef5a735940d61b793ef5c8619562188904417694810d24c01edf0b5 +size 332117 diff --git a/docs/20260915_171525/20260915_171525/depth/000282.png b/docs/20260915_171525/20260915_171525/depth/000282.png new file mode 100644 index 0000000..59e3017 --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000282.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:759cc9c14a22af046f7299cb18b972de430a07c71ce61060bbf57b04faa5d951 +size 334899 diff --git a/docs/20260915_171525/20260915_171525/depth/000283.png b/docs/20260915_171525/20260915_171525/depth/000283.png new file mode 100644 index 0000000..07e1cca --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000283.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:e329ca785b07f2e96c18b53cba4ef86568d175ee3c5e611573be05de9a8e37fb +size 326890 diff --git a/docs/20260915_171525/20260915_171525/depth/000284.png b/docs/20260915_171525/20260915_171525/depth/000284.png new file mode 100644 index 0000000..f7116c1 --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000284.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:a5dd297c4c19af14d8853fca7eea1e049dd390be39cb2bd52a0333a532e43917 +size 330777 diff --git a/docs/20260915_171525/20260915_171525/depth/000285.png b/docs/20260915_171525/20260915_171525/depth/000285.png new file mode 100644 index 0000000..a3704ab --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000285.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:0ea667806ff4e49bee816d1fb5237ba27a9c412847066a6552fa6d26ad43d423 +size 325576 diff --git a/docs/20260915_171525/20260915_171525/depth/000286.png b/docs/20260915_171525/20260915_171525/depth/000286.png new file mode 100644 index 0000000..c5ed385 --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000286.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:f4ed0c93662e92bcf722a8043374459d22efe6c15890fdded84b174d04123ace +size 330853 diff --git a/docs/20260915_171525/20260915_171525/depth/000287.png b/docs/20260915_171525/20260915_171525/depth/000287.png new file mode 100644 index 0000000..5762445 --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000287.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:bab27cc50fefae0a0eb50a24e53fea52aa56d49c7c2e9d307dfa06583b7e83cd +size 326788 diff --git a/docs/20260915_171525/20260915_171525/depth/000288.png b/docs/20260915_171525/20260915_171525/depth/000288.png new file mode 100644 index 0000000..5307750 --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000288.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:65b5b72680e14589cd0a8d9bb4be9a93790433bdebd8cf961b6c9bfe4bbaadcf +size 327659 diff --git a/docs/20260915_171525/20260915_171525/depth/000289.png b/docs/20260915_171525/20260915_171525/depth/000289.png new file mode 100644 index 0000000..e620db6 --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000289.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:d5fba23195c742b3fb1d847ff00757d992e21fcc35577ee9d1910c969e3c3f2b +size 334124 diff --git a/docs/20260915_171525/20260915_171525/depth/000290.png b/docs/20260915_171525/20260915_171525/depth/000290.png new file mode 100644 index 0000000..b9cf140 --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000290.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:fc507789dcba87ab2143879ae3569238589b6538298848ea9447eba73e71070d +size 331380 diff --git a/docs/20260915_171525/20260915_171525/depth/000291.png b/docs/20260915_171525/20260915_171525/depth/000291.png new file mode 100644 index 0000000..394f015 --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000291.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:07033b696d82ed6dcda2f0e60ef8cbc16b968b3bbc0e86bb019dabb78da99914 +size 332057 diff --git a/docs/20260915_171525/20260915_171525/depth/000292.png b/docs/20260915_171525/20260915_171525/depth/000292.png new file mode 100644 index 0000000..c5b00aa --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000292.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:2333b232984cc70ccf21d363f4c47b1860e050709bc9aa54bc27de12dbea2dbe +size 330093 diff --git a/docs/20260915_171525/20260915_171525/depth/000293.png b/docs/20260915_171525/20260915_171525/depth/000293.png new file mode 100644 index 0000000..60e1673 --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000293.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:7cbd13d2780da9190fc6af4316ff99e68c2ac4d1b352f281bb2a4f50d8e92fce +size 328845 diff --git a/docs/20260915_171525/20260915_171525/depth/000294.png b/docs/20260915_171525/20260915_171525/depth/000294.png new file mode 100644 index 0000000..eee5ac7 --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000294.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:ae20bb35b7dadb076d2470ba821a11a35abd910dc7c79b8de20c4ad194828dd1 +size 328371 diff --git a/docs/20260915_171525/20260915_171525/depth/000295.png b/docs/20260915_171525/20260915_171525/depth/000295.png new file mode 100644 index 0000000..1361d27 --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000295.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:7c53b6c280fb30035220fb48f479826947a6493afa85089e8697671b32f08297 +size 327450 diff --git a/docs/20260915_171525/20260915_171525/depth/000296.png b/docs/20260915_171525/20260915_171525/depth/000296.png new file mode 100644 index 0000000..18ad1ff --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000296.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:dff13ad8c2d2c1b156fb79bfa2c820d8e88e6bf85dc2b9e385ba2f367cabf2cf +size 329625 diff --git a/docs/20260915_171525/20260915_171525/depth/000297.png b/docs/20260915_171525/20260915_171525/depth/000297.png new file mode 100644 index 0000000..7db70d1 --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000297.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:3431c5364ba1231deec6ea56dc34eb924982c3952ae0879fa52927acb18ee83e +size 322076 diff --git a/docs/20260915_171525/20260915_171525/depth/000298.png b/docs/20260915_171525/20260915_171525/depth/000298.png new file mode 100644 index 0000000..e48ecaa --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000298.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:b36d62df68fad7daf95240483658f9411968b07b1f2725ffea45e095b60705a9 +size 329949 diff --git a/docs/20260915_171525/20260915_171525/depth/000299.png b/docs/20260915_171525/20260915_171525/depth/000299.png new file mode 100644 index 0000000..1382724 --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000299.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:7e75ed8dbbeb3f03858c6014a3c67666ff207e0a4fe13c31bca9a52314c3f4df +size 325152 diff --git a/docs/20260915_171525/20260915_171525/depth/000300.png b/docs/20260915_171525/20260915_171525/depth/000300.png new file mode 100644 index 0000000..9be3231 --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000300.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:4942442a2f2ded49394b298d5504c9cab617a423b250dfa7b329a383165a1d33 +size 330020 diff --git a/docs/20260915_171525/20260915_171525/depth/000301.png b/docs/20260915_171525/20260915_171525/depth/000301.png new file mode 100644 index 0000000..8ead351 --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000301.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:71cb044829cb0c0feeb6f7748a46123bf72adc1d15e7f312b765848a86a22ad9 +size 330022 diff --git a/docs/20260915_171525/20260915_171525/depth/000302.png b/docs/20260915_171525/20260915_171525/depth/000302.png new file mode 100644 index 0000000..e535908 --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000302.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:cfaf553a5dea3dea6ddd1dac8a8e30614b020c6e55154a122bad49d617d8e16f +size 322514 diff --git a/docs/20260915_171525/20260915_171525/depth/000303.png b/docs/20260915_171525/20260915_171525/depth/000303.png new file mode 100644 index 0000000..43ccaf5 --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000303.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:81f4b3a84852bd80ce767e8d9383654f409c9441d5de606ea22e835bbe5800d0 +size 325997 diff --git a/docs/20260915_171525/20260915_171525/depth/000304.png b/docs/20260915_171525/20260915_171525/depth/000304.png new file mode 100644 index 0000000..78ef10e --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000304.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:5913e7cf0d8945d624f49663f8a5bc83b2fd122ab38286a286c5163ad1b760ec +size 332747 diff --git a/docs/20260915_171525/20260915_171525/depth/000305.png b/docs/20260915_171525/20260915_171525/depth/000305.png new file mode 100644 index 0000000..e2edddf --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000305.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:75ef76833977ab6acb36a686e2a1414f0f4cea95bfb993b310e5743438899400 +size 330702 diff --git a/docs/20260915_171525/20260915_171525/depth/000306.png b/docs/20260915_171525/20260915_171525/depth/000306.png new file mode 100644 index 0000000..c09c6c0 --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000306.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:b0fc6ce3eb3c417444617243062ef8d5016b7f8b5f26761d3f3bb6fefe9f81d6 +size 327900 diff --git a/docs/20260915_171525/20260915_171525/depth/000307.png b/docs/20260915_171525/20260915_171525/depth/000307.png new file mode 100644 index 0000000..d8cc28c --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000307.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:16cf95a68eed4fd9a7a8ae0b4838738f02726c0fcc9adc506716e92843110844 +size 326971 diff --git a/docs/20260915_171525/20260915_171525/depth/000308.png b/docs/20260915_171525/20260915_171525/depth/000308.png new file mode 100644 index 0000000..368fa5b --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000308.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:6ddec5e15f14221e80ca8ee017fb5bf048bb8cfda4e1e609bba7622fdffa435f +size 331966 diff --git a/docs/20260915_171525/20260915_171525/depth/000309.png b/docs/20260915_171525/20260915_171525/depth/000309.png new file mode 100644 index 0000000..69c7e35 --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000309.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:04d25956149ffad5dc60113a7166666e2a31e5b269f771a38e4fac3b53ba10f9 +size 328742 diff --git a/docs/20260915_171525/20260915_171525/depth/000310.png b/docs/20260915_171525/20260915_171525/depth/000310.png new file mode 100644 index 0000000..ff7b750 --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000310.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:180a00564dc140de4cb9c16e4802028daf60469ae95c99c839ffab4142b19f54 +size 330508 diff --git a/docs/20260915_171525/20260915_171525/depth/000311.png b/docs/20260915_171525/20260915_171525/depth/000311.png new file mode 100644 index 0000000..e9e14f2 --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000311.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:4aaf0cfedd9b2b3a51121942187e22ff066c2c400113e8f7a61b4af57fabfaa3 +size 329038 diff --git a/docs/20260915_171525/20260915_171525/depth/000312.png b/docs/20260915_171525/20260915_171525/depth/000312.png new file mode 100644 index 0000000..3073799 --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000312.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:ff10a4c59a121d4cdf77b117167f3d98f971518b9127cfe2f219e27781cb14f1 +size 335017 diff --git a/docs/20260915_171525/20260915_171525/depth/000313.png b/docs/20260915_171525/20260915_171525/depth/000313.png new file mode 100644 index 0000000..e1f025d --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000313.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:e5fcbd206c21a42e0c4e8678422d2f9ae1f3d32fb97425e655c6117de234c74b +size 332092 diff --git a/docs/20260915_171525/20260915_171525/depth/000314.png b/docs/20260915_171525/20260915_171525/depth/000314.png new file mode 100644 index 0000000..c9707b6 --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000314.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:c1417f9d14e5f0e782440cbc9654fcc5980e948c7c8fabed43c4317eba1e3b3b +size 323779 diff --git a/docs/20260915_171525/20260915_171525/depth/000315.png b/docs/20260915_171525/20260915_171525/depth/000315.png new file mode 100644 index 0000000..50c6610 --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000315.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:45d804a058dc41a1c956c79700d0baf1205c1a8a23b3551f3624b6a0a6f49d32 +size 328895 diff --git a/docs/20260915_171525/20260915_171525/depth/000316.png b/docs/20260915_171525/20260915_171525/depth/000316.png new file mode 100644 index 0000000..ab23067 --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000316.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:be7d37a78f72984e850fc6fe335c18a2f55c133c821935c1d55d1d864f8f1611 +size 327468 diff --git a/docs/20260915_171525/20260915_171525/depth/000317.png b/docs/20260915_171525/20260915_171525/depth/000317.png new file mode 100644 index 0000000..00337b6 --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000317.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:5d1569d0dd152af35746cf0a7ca84398db6120d4bbe06c3a376a87b9dd221339 +size 331047 diff --git a/docs/20260915_171525/20260915_171525/depth/000318.png b/docs/20260915_171525/20260915_171525/depth/000318.png new file mode 100644 index 0000000..b6dc75c --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000318.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:8e0550993dbf2b8558256f40184f7ea674ac10c8abdca69023429104dafe346b +size 321328 diff --git a/docs/20260915_171525/20260915_171525/depth/000319.png b/docs/20260915_171525/20260915_171525/depth/000319.png new file mode 100644 index 0000000..407717f --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000319.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:2e2ec2772e531409b8ad6beb2b30314c6098e3efdd856df85f5eca5f093a0cca +size 318392 diff --git a/docs/20260915_171525/20260915_171525/depth/000320.png b/docs/20260915_171525/20260915_171525/depth/000320.png new file mode 100644 index 0000000..6608b36 --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000320.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:bf776149cccd8f8a8b9403511d260e7a5175a5b7b252c412dbccfb4dbec78a9c +size 325051 diff --git a/docs/20260915_171525/20260915_171525/depth/000321.png b/docs/20260915_171525/20260915_171525/depth/000321.png new file mode 100644 index 0000000..57a5e9c --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000321.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:9a159dd60d33223da7d407ed066a47016fc810e92abf3a4e0311873d794fb0cd +size 335457 diff --git a/docs/20260915_171525/20260915_171525/depth/000322.png b/docs/20260915_171525/20260915_171525/depth/000322.png new file mode 100644 index 0000000..7aa3d85 --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000322.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:07a955a200ec84f6bc6149491ba29f100e86172132a7c9a1bf1dd3c2a01ffcf8 +size 335865 diff --git a/docs/20260915_171525/20260915_171525/depth/000323.png b/docs/20260915_171525/20260915_171525/depth/000323.png new file mode 100644 index 0000000..0a69085 --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000323.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:988dfd08293650d0c6124f44f483cff0407295cb339fab771da09e30a1377afa +size 334435 diff --git a/docs/20260915_171525/20260915_171525/depth/000324.png b/docs/20260915_171525/20260915_171525/depth/000324.png new file mode 100644 index 0000000..60984b7 --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000324.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:b5e0c72559c068837710aa3318850686bed7f347197b9e103eee389904fe58b3 +size 317218 diff --git a/docs/20260915_171525/20260915_171525/depth/000325.png b/docs/20260915_171525/20260915_171525/depth/000325.png new file mode 100644 index 0000000..44c55b2 --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000325.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:d13dcbc86da7b838272fdb9611a847696d0213cdf99f1ea04503e7c147a996c5 +size 324662 diff --git a/docs/20260915_171525/20260915_171525/depth/000326.png b/docs/20260915_171525/20260915_171525/depth/000326.png new file mode 100644 index 0000000..0d2a7c7 --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000326.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:92f2a9f8acd6a8ab79e98f4a5c5624fbdf04b0cb63a6e493590e99ff9dc16eba +size 327252 diff --git a/docs/20260915_171525/20260915_171525/depth/000327.png b/docs/20260915_171525/20260915_171525/depth/000327.png new file mode 100644 index 0000000..360befc --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000327.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:f3b1994d8b64f73c556a9e5df9c7e0c815960277db138131df265293ff4fa67f +size 332721 diff --git a/docs/20260915_171525/20260915_171525/depth/000328.png b/docs/20260915_171525/20260915_171525/depth/000328.png new file mode 100644 index 0000000..4aed202 --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000328.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:f777b4656b9bf0019905290b2d6f5b4975469446d7d2fb8a0ccedfd129b217c8 +size 323638 diff --git a/docs/20260915_171525/20260915_171525/depth/000329.png b/docs/20260915_171525/20260915_171525/depth/000329.png new file mode 100644 index 0000000..ddcb6fd --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000329.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:7fdd7519f4284dd450036052611cd60d9a72bd39779f3d70a0ef95f17c8e6a44 +size 326794 diff --git a/docs/20260915_171525/20260915_171525/depth/000330.png b/docs/20260915_171525/20260915_171525/depth/000330.png new file mode 100644 index 0000000..c432bf3 --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000330.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:d1a43a05a854454b91db1b1dfbd6c67831a626a82ba3a27d7c3b9b14e22d04e2 +size 329987 diff --git a/docs/20260915_171525/20260915_171525/depth/000331.png b/docs/20260915_171525/20260915_171525/depth/000331.png new file mode 100644 index 0000000..5c405ab --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000331.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:dd7156847883a030e7381938058c32f61d0abeb8eb1f3490137991db41172387 +size 332902 diff --git a/docs/20260915_171525/20260915_171525/depth/000332.png b/docs/20260915_171525/20260915_171525/depth/000332.png new file mode 100644 index 0000000..cc7b77a --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000332.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:c7053df7ba27713f93a993f6f2dcd0e3d7acb2b80d245d91a5fb7a4f17e74700 +size 329779 diff --git a/docs/20260915_171525/20260915_171525/depth/000333.png b/docs/20260915_171525/20260915_171525/depth/000333.png new file mode 100644 index 0000000..424bf05 --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000333.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:637d707a7a8388d2e553e2a3d0e817518d65302804015234e833052d8aee7895 +size 332116 diff --git a/docs/20260915_171525/20260915_171525/depth/000334.png b/docs/20260915_171525/20260915_171525/depth/000334.png new file mode 100644 index 0000000..fc394a0 --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000334.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:23e42d749561d9798c4bcfc3969d878fa28baeaa2908349824be1b8a51130a3c +size 329420 diff --git a/docs/20260915_171525/20260915_171525/depth/000335.png b/docs/20260915_171525/20260915_171525/depth/000335.png new file mode 100644 index 0000000..4666d98 --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000335.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:ab2dea38b4730fe74f3e9bbddfb176bb1cb76ca4c15cc4750ff9f8ceef003752 +size 330580 diff --git a/docs/20260915_171525/20260915_171525/depth/000336.png b/docs/20260915_171525/20260915_171525/depth/000336.png new file mode 100644 index 0000000..1306a31 --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000336.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:3940f19b35dc1099349c0667485130727abe0b32cf2cb5bf7397c5fb39288ede +size 328374 diff --git a/docs/20260915_171525/20260915_171525/depth/000337.png b/docs/20260915_171525/20260915_171525/depth/000337.png new file mode 100644 index 0000000..ce38a8f --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000337.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:d76dfd9f7773254d6a15ac20ab7e540e60ac5ce473745bbe8a6531ab5f7adc32 +size 328365 diff --git a/docs/20260915_171525/20260915_171525/depth/000338.png b/docs/20260915_171525/20260915_171525/depth/000338.png new file mode 100644 index 0000000..2be5f83 --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000338.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:35612b40c0e4e11ca9ab7696e06f9cca5c9fbfbbe518702050e2294a18a3239b +size 327146 diff --git a/docs/20260915_171525/20260915_171525/depth/000339.png b/docs/20260915_171525/20260915_171525/depth/000339.png new file mode 100644 index 0000000..f8530cb --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000339.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:69bc46fe638e910b9be0a3f822446584a0be33e07e5a255dc6824aa097022e77 +size 330624 diff --git a/docs/20260915_171525/20260915_171525/depth/000340.png b/docs/20260915_171525/20260915_171525/depth/000340.png new file mode 100644 index 0000000..54fcd33 --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000340.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:0e6bf2a3c7c2d0e953d11a6f58f49b348e5fb23287ed02e09ed27f413ebb765e +size 331051 diff --git a/docs/20260915_171525/20260915_171525/depth/000341.png b/docs/20260915_171525/20260915_171525/depth/000341.png new file mode 100644 index 0000000..ec20c00 --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000341.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:fa2c9f2d80b16237fa484417ffbd3b5d02b0d1705d2855471929f01ede0ad29e +size 323655 diff --git a/docs/20260915_171525/20260915_171525/depth/000342.png b/docs/20260915_171525/20260915_171525/depth/000342.png new file mode 100644 index 0000000..21be703 --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000342.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:2162aa7645957283c9f9e022df78fcb7cea4cd4588be88c0c2a04267e8d2548f +size 328375 diff --git a/docs/20260915_171525/20260915_171525/depth/000343.png b/docs/20260915_171525/20260915_171525/depth/000343.png new file mode 100644 index 0000000..98f945b --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000343.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:59c290c85b2a5f17c3cfa4b88f15c3daa1d774b530d9c8426c0412a74fe2dcc6 +size 333077 diff --git a/docs/20260915_171525/20260915_171525/depth/000344.png b/docs/20260915_171525/20260915_171525/depth/000344.png new file mode 100644 index 0000000..3eea744 --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000344.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:df673135d2be0eda1a0e8c9bbba559076feed8ea0c4a44ec4f87f2834e7d0f3d +size 332233 diff --git a/docs/20260915_171525/20260915_171525/depth/000345.png b/docs/20260915_171525/20260915_171525/depth/000345.png new file mode 100644 index 0000000..9df0de4 --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000345.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:2e2eef06a2f68f6451e3dd8e8e762059ddd5a5e2afda07299546da327c042b88 +size 328274 diff --git a/docs/20260915_171525/20260915_171525/depth/000346.png b/docs/20260915_171525/20260915_171525/depth/000346.png new file mode 100644 index 0000000..8cef8d7 --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000346.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:bc7b5361b13831583cca62417e2d4897b483e22cdef71e72504fac5d0f3f0e7d +size 330038 diff --git a/docs/20260915_171525/20260915_171525/depth/000347.png b/docs/20260915_171525/20260915_171525/depth/000347.png new file mode 100644 index 0000000..c533101 --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000347.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:8a8bb25eceb85f464a014b457ab37615588483255edfd4f98652e62ebe58bcfe +size 332398 diff --git a/docs/20260915_171525/20260915_171525/depth/000348.png b/docs/20260915_171525/20260915_171525/depth/000348.png new file mode 100644 index 0000000..06cf8a8 --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000348.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:e8422df1e3bd2c4eec94836ff8580efafa3db4baed27a39fe0dda8ba14edee39 +size 322567 diff --git a/docs/20260915_171525/20260915_171525/depth/000349.png b/docs/20260915_171525/20260915_171525/depth/000349.png new file mode 100644 index 0000000..20b0fa5 --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000349.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:7333300cb76c455d821180acd2bae4a16f1678f185cc77ea71630ecec877ab49 +size 326033 diff --git a/docs/20260915_171525/20260915_171525/depth/000350.png b/docs/20260915_171525/20260915_171525/depth/000350.png new file mode 100644 index 0000000..0adbaec --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000350.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:6c772539e3a25e7f20ca32f005197c5f6deb046c96294df2853d87ff0e9f9475 +size 324664 diff --git a/docs/20260915_171525/20260915_171525/depth/000351.png b/docs/20260915_171525/20260915_171525/depth/000351.png new file mode 100644 index 0000000..9bcca9e --- /dev/null +++ b/docs/20260915_171525/20260915_171525/depth/000351.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:47fd6c87904aad9d37b93d0a10794e5093ce841dd48b97546ee852aaa68d8749 +size 330515 diff --git a/docs/20260915_171525/20260915_171525/intrinsics.json b/docs/20260915_171525/20260915_171525/intrinsics.json new file mode 100644 index 0000000..753fe18 --- /dev/null +++ b/docs/20260915_171525/20260915_171525/intrinsics.json @@ -0,0 +1,1789 @@ +{ + "fx": 437.34423828125, + "fy": 436.91009521484375, + "cx": 427.2064208984375, + "cy": 246.0908660888672, + "depth_scale_m": 9.999999747378752e-05, + "aligned_to": "color", + "frames": 352, + "width": 848, + "height": 480, + "fps_nominal": 30, + "fps_actual": 29.97, + "save_depth": true, + "time_source": "device hardware clock, global_time disabled", + "timestamp_domain": "timestamp_domain.hardware_clock", + "timestamps_ms": [ + 511179.702, + 511213.04600000003, + 511246.39, + 511279.734, + 511313.07800000004, + 511346.422, + 511379.766, + 511413.11, + 511446.454, + 511479.798, + 511513.142, + 511546.48600000003, + 511579.83, + 511613.174, + 511646.517, + 511679.86100000003, + 511713.205, + 511746.549, + 511779.893, + 511813.237, + 511846.581, + 511879.925, + 511913.26900000003, + 511946.613, + 511979.957, + 512013.30100000004, + 512046.645, + 512079.988, + 512113.332, + 512146.67600000004, + 512180.02, + 512213.364, + 512246.708, + 512280.052, + 512313.396, + 512346.74, + 512380.08400000003, + 512413.428, + 512446.772, + 512480.11600000004, + 512513.45900000003, + 512546.803, + 512580.147, + 512613.49100000004, + 512646.835, + 512680.179, + 512713.523, + 512746.867, + 512780.211, + 512813.555, + 512846.89900000003, + 512880.243, + 512913.587, + 512946.931, + 512980.27400000003, + 513013.618, + 513046.962, + 513080.306, + 513113.65, + 513146.994, + 513180.338, + 513213.68200000003, + 513247.026, + 513280.37, + 513313.71400000004, + 513347.058, + 513380.402, + 513413.745, + 513447.08900000004, + 513480.433, + 513513.777, + 513547.121, + 513580.465, + 513613.809, + 513647.153, + 513680.49700000003, + 513713.841, + 513747.185, + 513780.52900000004, + 513813.873, + 513847.216, + 513880.56, + 513913.90400000004, + 513947.248, + 513980.592, + 514013.936, + 514047.28, + 514080.624, + 514113.968, + 514147.31200000003, + 514180.656, + 514214.0, + 514247.344, + 514280.688, + 514314.031, + 514347.375, + 514380.719, + 514414.063, + 514447.407, + 514480.751, + 514514.09500000003, + 514547.439, + 514580.783, + 514614.12700000004, + 514647.471, + 514680.815, + 514714.159, + 514747.50200000004, + 514780.846, + 514814.19, + 514847.534, + 514880.878, + 514914.222, + 514947.566, + 514980.91000000003, + 515014.254, + 515047.598, + 515080.94200000004, + 515114.286, + 515147.63, + 515180.973, + 515214.31700000004, + 515247.661, + 515281.005, + 515314.349, + 515347.693, + 515381.037, + 515414.381, + 515447.72500000003, + 515481.069, + 515514.413, + 515547.757, + 515581.101, + 515614.445, + 515647.788, + 515681.132, + 515714.476, + 515747.82, + 515781.164, + 515814.50800000003, + 515847.852, + 515881.196, + 515914.54000000004, + 515947.884, + 515981.228, + 516014.572, + 516047.916, + 516081.259, + 516114.603, + 516147.947, + 516181.291, + 516214.635, + 516247.979, + 516281.32300000003, + 516314.667, + 516348.011, + 516381.35500000004, + 516414.699, + 516448.043, + 516481.387, + 516514.73000000004, + 516548.074, + 516581.418, + 516614.762, + 516648.106, + 516681.45, + 516714.794, + 516748.13800000004, + 516781.482, + 516814.826, + 516848.17, + 516881.514, + 516914.858, + 516948.202, + 516981.545, + 517014.889, + 517048.233, + 517081.577, + 517114.92100000003, + 517148.265, + 517181.609, + 517214.95300000004, + 517248.297, + 517281.641, + 517314.985, + 517348.329, + 517381.673, + 517415.016, + 517448.36, + 517481.704, + 517515.048, + 517548.392, + 517581.73600000003, + 517615.08, + 517648.424, + 517681.768, + 517715.112, + 517748.456, + 517781.8, + 517815.14400000003, + 517848.487, + 517881.831, + 517915.175, + 517948.51900000003, + 517981.863, + 518015.207, + 518048.55100000004, + 518081.895, + 518115.239, + 518148.583, + 518181.927, + 518215.271, + 518248.615, + 518281.95900000003, + 518315.302, + 518348.646, + 518381.99, + 518415.33400000003, + 518448.678, + 518482.022, + 518515.36600000004, + 518548.71, + 518582.054, + 518615.398, + 518648.742, + 518682.086, + 518715.43, + 518748.773, + 518782.117, + 518815.461, + 518848.805, + 518882.14900000003, + 518915.493, + 518948.837, + 518982.181, + 519015.525, + 519048.869, + 519082.213, + 519115.55700000003, + 519148.901, + 519182.244, + 519215.588, + 519248.93200000003, + 519282.276, + 519315.62, + 519348.96400000004, + 519382.308, + 519415.652, + 519448.996, + 519482.34, + 519515.684, + 519549.028, + 519582.37200000003, + 519615.716, + 519649.059, + 519682.403, + 519715.74700000003, + 519749.091, + 519782.435, + 519815.77900000004, + 519849.123, + 519882.467, + 519915.811, + 519949.155, + 519982.499, + 520015.843, + 520049.18700000003, + 520082.53, + 520115.874, + 520149.218, + 520182.56200000003, + 520215.906, + 520249.25, + 520282.594, + 520315.938, + 520349.282, + 520382.626, + 520415.97000000003, + 520449.314, + 520482.658, + 520516.001, + 520549.34500000003, + 520582.689, + 520616.033, + 520649.37700000004, + 520682.721, + 520716.065, + 520749.409, + 520782.753, + 520816.097, + 520849.441, + 520882.78500000003, + 520916.129, + 520949.473, + 520982.816, + 521016.16000000003, + 521049.504, + 521082.848, + 521116.19200000004, + 521149.536, + 521182.88, + 521216.224, + 521249.568, + 521282.912, + 521316.256, + 521349.60000000003, + 521382.944, + 521416.287, + 521449.631, + 521482.97500000003, + 521516.319, + 521549.663, + 521583.007, + 521616.351, + 521649.695, + 521683.039, + 521716.38300000003, + 521749.727, + 521783.071, + 521816.41500000004, + 521849.75800000003, + 521883.102, + 521916.446, + 521949.79000000004, + 521983.134, + 522016.478, + 522049.822, + 522083.166, + 522116.51, + 522149.854, + 522183.19800000003, + 522216.542, + 522249.886, + 522283.23000000004, + 522316.57300000003, + 522349.917, + 522383.261, + 522416.60500000004, + 522449.949, + 522483.293, + 522516.637, + 522549.981, + 522583.325, + 522616.669, + 522650.01300000004, + 522683.357, + 522716.701, + 522750.044, + 522783.38800000004, + 522816.732, + 522850.076, + 522883.42 + ], + "depth_timestamps_ms": [ + 511179.398, + 511212.742, + 511246.086, + 511279.43, + 511312.77400000003, + 511346.118, + 511379.462, + 511412.806, + 511446.15, + 511479.494, + 511512.838, + 511546.18200000003, + 511579.526, + 511612.869, + 511646.213, + 511679.55700000003, + 511712.901, + 511746.245, + 511779.58900000004, + 511812.933, + 511846.277, + 511879.621, + 511912.965, + 511946.309, + 511979.653, + 512012.99700000003, + 512046.34, + 512079.684, + 512113.028, + 512146.37200000003, + 512179.716, + 512213.06, + 512246.40400000004, + 512279.748, + 512313.092, + 512346.436, + 512379.78, + 512413.124, + 512446.468, + 512479.81200000003, + 512513.155, + 512546.499, + 512579.843, + 512613.18700000003, + 512646.531, + 512679.875, + 512713.219, + 512746.563, + 512779.907, + 512813.251, + 512846.59500000003, + 512879.939, + 512913.283, + 512946.626, + 512979.97000000003, + 513013.314, + 513046.658, + 513080.00200000004, + 513113.346, + 513146.69, + 513180.034, + 513213.378, + 513246.722, + 513280.066, + 513313.41000000003, + 513346.754, + 513380.097, + 513413.441, + 513446.78500000003, + 513480.129, + 513513.473, + 513546.81700000004, + 513580.161, + 513613.505, + 513646.849, + 513680.193, + 513713.537, + 513746.881, + 513780.22500000003, + 513813.569, + 513846.912, + 513880.256, + 513913.60000000003, + 513946.944, + 513980.288, + 514013.632, + 514046.976, + 514080.32, + 514113.664, + 514147.00800000003, + 514180.352, + 514213.696, + 514247.04000000004, + 514280.38300000003, + 514313.727, + 514347.071, + 514380.41500000004, + 514413.759, + 514447.103, + 514480.447, + 514513.791, + 514547.135, + 514580.479, + 514613.82300000003, + 514647.167, + 514680.511, + 514713.854, + 514747.19800000003, + 514780.542, + 514813.886, + 514847.23000000004, + 514880.574, + 514913.918, + 514947.262, + 514980.606, + 515013.95, + 515047.294, + 515080.63800000004, + 515113.982, + 515147.326, + 515180.669, + 515214.01300000004, + 515247.357, + 515280.701, + 515314.045, + 515347.389, + 515380.733, + 515414.077, + 515447.42100000003, + 515480.765, + 515514.109, + 515547.45300000004, + 515580.797, + 515614.14, + 515647.484, + 515680.82800000004, + 515714.172, + 515747.516, + 515780.86, + 515814.204, + 515847.548, + 515880.892, + 515914.23600000003, + 515947.58, + 515980.924, + 516014.268, + 516047.61100000003, + 516080.955, + 516114.299, + 516147.643, + 516180.987, + 516214.331, + 516247.675, + 516281.01900000003, + 516314.363, + 516347.707, + 516381.05100000004, + 516414.395, + 516447.739, + 516481.083, + 516514.42600000004, + 516547.77, + 516581.114, + 516614.458, + 516647.802, + 516681.146, + 516714.49, + 516747.83400000003, + 516781.178, + 516814.522, + 516847.86600000004, + 516881.21, + 516914.554, + 516947.897, + 516981.24100000004, + 517014.585, + 517047.929, + 517081.273, + 517114.617, + 517147.961, + 517181.305, + 517214.64900000003, + 517247.993, + 517281.337, + 517314.681, + 517348.025, + 517381.368, + 517414.712, + 517448.056, + 517481.4, + 517514.744, + 517548.088, + 517581.43200000003, + 517614.776, + 517648.12, + 517681.46400000004, + 517714.808, + 517748.152, + 517781.496, + 517814.84, + 517848.183, + 517881.527, + 517914.871, + 517948.215, + 517981.559, + 518014.903, + 518048.24700000003, + 518081.591, + 518114.935, + 518148.27900000004, + 518181.623, + 518214.967, + 518248.311, + 518281.65400000004, + 518314.998, + 518348.342, + 518381.686, + 518415.03, + 518448.374, + 518481.718, + 518515.06200000003, + 518548.406, + 518581.75, + 518615.094, + 518648.438, + 518681.782, + 518715.125, + 518748.469, + 518781.813, + 518815.157, + 518848.501, + 518881.84500000003, + 518915.189, + 518948.533, + 518981.87700000004, + 519015.221, + 519048.565, + 519081.909, + 519115.253, + 519148.597, + 519181.94, + 519215.284, + 519248.628, + 519281.972, + 519315.316, + 519348.66000000003, + 519382.004, + 519415.348, + 519448.69200000004, + 519482.036, + 519515.38, + 519548.724, + 519582.068, + 519615.411, + 519648.755, + 519682.099, + 519715.443, + 519748.787, + 519782.131, + 519815.47500000003, + 519848.819, + 519882.163, + 519915.507, + 519948.851, + 519982.195, + 520015.539, + 520048.882, + 520082.226, + 520115.57, + 520148.914, + 520182.25800000003, + 520215.602, + 520248.946, + 520282.29000000004, + 520315.634, + 520348.978, + 520382.322, + 520415.666, + 520449.01, + 520482.354, + 520515.697, + 520549.041, + 520582.385, + 520615.729, + 520649.07300000003, + 520682.417, + 520715.761, + 520749.10500000004, + 520782.449, + 520815.793, + 520849.137, + 520882.481, + 520915.825, + 520949.168, + 520982.512, + 521015.856, + 521049.2, + 521082.544, + 521115.88800000004, + 521149.232, + 521182.576, + 521215.92, + 521249.264, + 521282.608, + 521315.952, + 521349.29600000003, + 521382.639, + 521415.983, + 521449.327, + 521482.67100000003, + 521516.015, + 521549.359, + 521582.70300000004, + 521616.047, + 521649.391, + 521682.735, + 521716.079, + 521749.423, + 521782.767, + 521816.11100000003, + 521849.454, + 521882.798, + 521916.142, + 521949.48600000003, + 521982.83, + 522016.174, + 522049.518, + 522082.862, + 522116.206, + 522149.55, + 522182.89400000003, + 522216.238, + 522249.582, + 522282.925, + 522316.26900000003, + 522349.613, + 522382.957, + 522416.30100000004, + 522449.645, + 522482.989, + 522516.333, + 522549.677, + 522583.021, + 522616.365, + 522649.70900000003, + 522683.053, + 522716.396, + 522749.74, + 522783.08400000003, + 522816.428, + 522849.772, + 522883.11600000004 + ], + "color_frame_numbers": [ + 6178, + 6179, + 6180, + 6181, + 6182, + 6183, + 6184, + 6185, + 6186, + 6187, + 6188, + 6189, + 6190, + 6191, + 6192, + 6193, + 6194, + 6195, + 6196, + 6197, + 6198, + 6199, + 6200, + 6201, + 6202, + 6203, + 6204, + 6205, + 6206, + 6207, + 6208, + 6209, + 6210, + 6211, + 6212, + 6213, + 6214, + 6215, + 6216, + 6217, + 6218, + 6219, + 6220, + 6221, + 6222, + 6223, + 6224, + 6225, + 6226, + 6227, + 6228, + 6229, + 6230, + 6231, + 6232, + 6233, + 6234, + 6235, + 6236, + 6237, + 6238, + 6239, + 6240, + 6241, + 6242, + 6243, + 6244, + 6245, + 6246, + 6247, + 6248, + 6249, + 6250, + 6251, + 6252, + 6253, + 6254, + 6255, + 6256, + 6257, + 6258, + 6259, + 6260, + 6261, + 6262, + 6263, + 6264, + 6265, + 6266, + 6267, + 6268, + 6269, + 6270, + 6271, + 6272, + 6273, + 6274, + 6275, + 6276, + 6277, + 6278, + 6279, + 6280, + 6281, + 6282, + 6283, + 6284, + 6285, + 6286, + 6287, + 6288, + 6289, + 6290, + 6291, + 6292, + 6293, + 6294, + 6295, + 6296, + 6297, + 6298, + 6299, + 6300, + 6301, + 6302, + 6303, + 6304, + 6305, + 6306, + 6307, + 6308, + 6309, + 6310, + 6311, + 6312, + 6313, + 6314, + 6315, + 6316, + 6317, + 6318, + 6319, + 6320, + 6321, + 6322, + 6323, + 6324, + 6325, + 6326, + 6327, + 6328, + 6329, + 6330, + 6331, + 6332, + 6333, + 6334, + 6335, + 6336, + 6337, + 6338, + 6339, + 6340, + 6341, + 6342, + 6343, + 6344, + 6345, + 6346, + 6347, + 6348, + 6349, + 6350, + 6351, + 6352, + 6353, + 6354, + 6355, + 6356, + 6357, + 6358, + 6359, + 6360, + 6361, + 6362, + 6363, + 6364, + 6365, + 6366, + 6367, + 6368, + 6369, + 6370, + 6371, + 6372, + 6373, + 6374, + 6375, + 6376, + 6377, + 6378, + 6379, + 6380, + 6381, + 6382, + 6383, + 6384, + 6385, + 6386, + 6387, + 6388, + 6389, + 6390, + 6391, + 6392, + 6393, + 6394, + 6395, + 6396, + 6397, + 6398, + 6399, + 6400, + 6401, + 6402, + 6403, + 6404, + 6405, + 6406, + 6407, + 6408, + 6409, + 6410, + 6411, + 6412, + 6413, + 6414, + 6415, + 6416, + 6417, + 6418, + 6419, + 6420, + 6421, + 6422, + 6423, + 6424, + 6425, + 6426, + 6427, + 6428, + 6429, + 6430, + 6431, + 6432, + 6433, + 6434, + 6435, + 6436, + 6437, + 6438, + 6439, + 6440, + 6441, + 6442, + 6443, + 6444, + 6445, + 6446, + 6447, + 6448, + 6449, + 6450, + 6451, + 6452, + 6453, + 6454, + 6455, + 6456, + 6457, + 6458, + 6459, + 6460, + 6461, + 6462, + 6463, + 6464, + 6465, + 6466, + 6467, + 6468, + 6469, + 6470, + 6471, + 6472, + 6473, + 6474, + 6475, + 6476, + 6477, + 6478, + 6479, + 6480, + 6481, + 6482, + 6483, + 6484, + 6485, + 6486, + 6487, + 6488, + 6489, + 6490, + 6491, + 6492, + 6493, + 6494, + 6495, + 6496, + 6497, + 6498, + 6499, + 6500, + 6501, + 6502, + 6503, + 6504, + 6505, + 6506, + 6507, + 6508, + 6509, + 6510, + 6511, + 6512, + 6513, + 6514, + 6515, + 6516, + 6517, + 6518, + 6519, + 6520, + 6521, + 6522, + 6523, + 6524, + 6525, + 6526, + 6527, + 6528, + 6529 + ], + "depth_frame_numbers": [ + 6192, + 6193, + 6194, + 6195, + 6196, + 6197, + 6198, + 6199, + 6200, + 6201, + 6202, + 6203, + 6204, + 6205, + 6206, + 6207, + 6208, + 6209, + 6210, + 6211, + 6212, + 6213, + 6214, + 6215, + 6216, + 6217, + 6218, + 6219, + 6220, + 6221, + 6222, + 6223, + 6224, + 6225, + 6226, + 6227, + 6228, + 6229, + 6230, + 6231, + 6232, + 6233, + 6234, + 6235, + 6236, + 6237, + 6238, + 6239, + 6240, + 6241, + 6242, + 6243, + 6244, + 6245, + 6246, + 6247, + 6248, + 6249, + 6250, + 6251, + 6252, + 6253, + 6254, + 6255, + 6256, + 6257, + 6258, + 6259, + 6260, + 6261, + 6262, + 6263, + 6264, + 6265, + 6266, + 6267, + 6268, + 6269, + 6270, + 6271, + 6272, + 6273, + 6274, + 6275, + 6276, + 6277, + 6278, + 6279, + 6280, + 6281, + 6282, + 6283, + 6284, + 6285, + 6286, + 6287, + 6288, + 6289, + 6290, + 6291, + 6292, + 6293, + 6294, + 6295, + 6296, + 6297, + 6298, + 6299, + 6300, + 6301, + 6302, + 6303, + 6304, + 6305, + 6306, + 6307, + 6308, + 6309, + 6310, + 6311, + 6312, + 6313, + 6314, + 6315, + 6316, + 6317, + 6318, + 6319, + 6320, + 6321, + 6322, + 6323, + 6324, + 6325, + 6326, + 6327, + 6328, + 6329, + 6330, + 6331, + 6332, + 6333, + 6334, + 6335, + 6336, + 6337, + 6338, + 6339, + 6340, + 6341, + 6342, + 6343, + 6344, + 6345, + 6346, + 6347, + 6348, + 6349, + 6350, + 6351, + 6352, + 6353, + 6354, + 6355, + 6356, + 6357, + 6358, + 6359, + 6360, + 6361, + 6362, + 6363, + 6364, + 6365, + 6366, + 6367, + 6368, + 6369, + 6370, + 6371, + 6372, + 6373, + 6374, + 6375, + 6376, + 6377, + 6378, + 6379, + 6380, + 6381, + 6382, + 6383, + 6384, + 6385, + 6386, + 6387, + 6388, + 6389, + 6390, + 6391, + 6392, + 6393, + 6394, + 6395, + 6396, + 6397, + 6398, + 6399, + 6400, + 6401, + 6402, + 6403, + 6404, + 6405, + 6406, + 6407, + 6408, + 6409, + 6410, + 6411, + 6412, + 6413, + 6414, + 6415, + 6416, + 6417, + 6418, + 6419, + 6420, + 6421, + 6422, + 6423, + 6424, + 6425, + 6426, + 6427, + 6428, + 6429, + 6430, + 6431, + 6432, + 6433, + 6434, + 6435, + 6436, + 6437, + 6438, + 6439, + 6440, + 6441, + 6442, + 6443, + 6444, + 6445, + 6446, + 6447, + 6448, + 6449, + 6450, + 6451, + 6452, + 6453, + 6454, + 6455, + 6456, + 6457, + 6458, + 6459, + 6460, + 6461, + 6462, + 6463, + 6464, + 6465, + 6466, + 6467, + 6468, + 6469, + 6470, + 6471, + 6472, + 6473, + 6474, + 6475, + 6476, + 6477, + 6478, + 6479, + 6480, + 6481, + 6482, + 6483, + 6484, + 6485, + 6486, + 6487, + 6488, + 6489, + 6490, + 6491, + 6492, + 6493, + 6494, + 6495, + 6496, + 6497, + 6498, + 6499, + 6500, + 6501, + 6502, + 6503, + 6504, + 6505, + 6506, + 6507, + 6508, + 6509, + 6510, + 6511, + 6512, + 6513, + 6514, + 6515, + 6516, + 6517, + 6518, + 6519, + 6520, + 6521, + 6522, + 6523, + 6524, + 6525, + 6526, + 6527, + 6528, + 6529, + 6530, + 6531, + 6532, + 6533, + 6534, + 6535, + 6536, + 6537, + 6538, + 6539, + 6540, + 6541, + 6542, + 6543 + ], + "host_arrival_monotonic_ns": [ + 791630160462, + 791662625688, + 791697053890, + 791730869230, + 791763895729, + 791796555406, + 791830521225, + 791862881403, + 791897824571, + 791929241674, + 791963682450, + 791996707457, + 792029222226, + 792062988574, + 792094975938, + 792131033968, + 792162627702, + 792195778833, + 792227934844, + 792261165117, + 792295959511, + 792327331981, + 792362175626, + 792395061453, + 792429581072, + 792462485560, + 792496156022, + 792530332787, + 792562442368, + 792596949440, + 792628721725, + 792663729927, + 792695923458, + 792729364153, + 792762453392, + 792799796231, + 792849068761, + 792869790966, + 792897532782, + 792928295419, + 792965354228, + 792996352584, + 793027776506, + 793063557568, + 793095284822, + 793130649862, + 793162471659, + 793196599146, + 793229558748, + 793263009971, + 793296433271, + 793328107917, + 793362161792, + 793397643937, + 793429381633, + 793462432244, + 793496952622, + 793527726907, + 793564266842, + 793595809004, + 793628377153, + 793660807320, + 793694445569, + 793728463812, + 793762095521, + 793803464198, + 793829943390, + 793863227607, + 793895757731, + 793928186026, + 793962655249, + 793994804170, + 794028386026, + 794061792766, + 794094995990, + 794128698456, + 794161992398, + 794196112997, + 794230678084, + 794264006566, + 794298032626, + 794331020383, + 794363994246, + 794397289572, + 794431273454, + 794464646377, + 794496822431, + 794529246199, + 794562103051, + 794596403089, + 794632399879, + 794662224818, + 794697013061, + 794729044474, + 794762654780, + 794797565383, + 794829951578, + 794864813428, + 794896425203, + 794932450860, + 794965430889, + 794997553682, + 795030635088, + 795063963156, + 795096905714, + 795129064673, + 795161383836, + 795196967135, + 795230515114, + 795264160349, + 795296408395, + 795329402387, + 795362717780, + 795397541668, + 795430012007, + 795463844538, + 795496921604, + 795532979607, + 795564286363, + 795598019781, + 795631532165, + 795664974405, + 795697614200, + 795729730368, + 795763803304, + 795797759796, + 795831234134, + 795863144081, + 795899871848, + 795930476260, + 795966606854, + 795997690608, + 796030718515, + 796064040830, + 796096840892, + 796130363954, + 796162599328, + 796195542681, + 796229497810, + 796263675800, + 796296122239, + 796328358350, + 796361361479, + 796395827681, + 796428541880, + 796463792296, + 796497095472, + 796532161578, + 796565694732, + 796597149171, + 796628622878, + 796662103284, + 796695536983, + 796728391750, + 796761520036, + 796793849306, + 796830932955, + 796862118048, + 796895052087, + 796929384224, + 796963362676, + 796996703795, + 797028966068, + 797064681996, + 797101999253, + 797131916135, + 797166655140, + 797197702773, + 797231147959, + 797265541842, + 797298138403, + 797330615743, + 797365202561, + 797396104366, + 797429492507, + 797464638839, + 797497332226, + 797530507990, + 797563015536, + 797597041611, + 797629296557, + 797664779096, + 797697926527, + 797730370765, + 797764963286, + 797797463294, + 797830083496, + 797863979106, + 797898099235, + 797932652799, + 797965870970, + 797996587150, + 798030151209, + 798063353329, + 798096036284, + 798130835595, + 798163598138, + 798197909373, + 798230639355, + 798264443605, + 798296201029, + 798330555038, + 798364317366, + 798397227101, + 798430875206, + 798462948440, + 798496469084, + 798530001899, + 798563071455, + 798595423781, + 798629318765, + 798665476210, + 798697995255, + 798730403622, + 798762988694, + 798797639996, + 798830457269, + 798866391975, + 798897920436, + 798930723388, + 798966880286, + 798997782166, + 799032854056, + 799063502157, + 799097910738, + 799131431870, + 799166416504, + 799198139603, + 799233028071, + 799265497956, + 799298720841, + 799331859070, + 799364692050, + 799400024856, + 799431864029, + 799465377030, + 799498004065, + 799530706611, + 799562332509, + 799595775022, + 799629337501, + 799662991554, + 799696394757, + 799733978267, + 799764702757, + 799797672818, + 799831732681, + 799864249060, + 799898455620, + 799931544023, + 799966127003, + 799999305024, + 800031637103, + 800066008614, + 800097274062, + 800129340310, + 800165132847, + 800199620491, + 800231702216, + 800264401957, + 800297853691, + 800332709574, + 800365106174, + 800397596192, + 800432011294, + 800465620246, + 800498498639, + 800531183598, + 800563690688, + 800597185609, + 800631849410, + 800662737485, + 800696738233, + 800729357289, + 800766410405, + 800797484674, + 800831715197, + 800864709963, + 800896913941, + 800932309638, + 800967590176, + 800998100427, + 801032618670, + 801064613271, + 801096972395, + 801131412051, + 801164580004, + 801197010405, + 801232740669, + 801264123681, + 801296965058, + 801329423582, + 801363203340, + 801398367595, + 801432257459, + 801463959867, + 801496400375, + 801530196170, + 801564447235, + 801597088529, + 801631650760, + 801663979011, + 801696766914, + 801730415589, + 801764524207, + 801797320126, + 801830183933, + 801864965650, + 801897580407, + 801930614062, + 801966493725, + 801998191292, + 802032066484, + 802064115086, + 802096128570, + 802130020061, + 802165808028, + 802198957299, + 802231944968, + 802266136685, + 802298812267, + 802331761467, + 802365246490, + 802399155616, + 802431655897, + 802463738468, + 802497053264, + 802529861089, + 802562446376, + 802598863823, + 802632008279, + 802665274395, + 802696889134, + 802730490863, + 802763994726, + 802798748852, + 802834757608, + 802866576490, + 802897468279, + 802931712061, + 802965129227, + 802997643887, + 803033501713, + 803065978379, + 803097859908, + 803130918785, + 803165991775, + 803199388363, + 803232271586, + 803264528122, + 803297842528, + 803332374295 + ], + "dropped_frames": 0, + "timestamp_backjumps": 0, + "sync_note": "\u6bcf\u884c\u6570\u636e\u6765\u81ea\u540c\u4e00 frameset\uff1bD405 \u5f69\u8272\u53d6\u81ea\u5de6\u6210\u50cf\u5668\uff0c\u4e0e\u6df1\u5ea6\u540c\u4e00\u6b21\u66dd\u5149\uff0c\u786c\u4ef6\u7ea7\u540c\u6b65" +} \ No newline at end of file diff --git a/docs/20260915_171525/intrinsics.json b/docs/20260915_171525/intrinsics.json new file mode 100644 index 0000000..753fe18 --- /dev/null +++ b/docs/20260915_171525/intrinsics.json @@ -0,0 +1,1789 @@ +{ + "fx": 437.34423828125, + "fy": 436.91009521484375, + "cx": 427.2064208984375, + "cy": 246.0908660888672, + "depth_scale_m": 9.999999747378752e-05, + "aligned_to": "color", + "frames": 352, + "width": 848, + "height": 480, + "fps_nominal": 30, + "fps_actual": 29.97, + "save_depth": true, + "time_source": "device hardware clock, global_time disabled", + "timestamp_domain": "timestamp_domain.hardware_clock", + "timestamps_ms": [ + 511179.702, + 511213.04600000003, + 511246.39, + 511279.734, + 511313.07800000004, + 511346.422, + 511379.766, + 511413.11, + 511446.454, + 511479.798, + 511513.142, + 511546.48600000003, + 511579.83, + 511613.174, + 511646.517, + 511679.86100000003, + 511713.205, + 511746.549, + 511779.893, + 511813.237, + 511846.581, + 511879.925, + 511913.26900000003, + 511946.613, + 511979.957, + 512013.30100000004, + 512046.645, + 512079.988, + 512113.332, + 512146.67600000004, + 512180.02, + 512213.364, + 512246.708, + 512280.052, + 512313.396, + 512346.74, + 512380.08400000003, + 512413.428, + 512446.772, + 512480.11600000004, + 512513.45900000003, + 512546.803, + 512580.147, + 512613.49100000004, + 512646.835, + 512680.179, + 512713.523, + 512746.867, + 512780.211, + 512813.555, + 512846.89900000003, + 512880.243, + 512913.587, + 512946.931, + 512980.27400000003, + 513013.618, + 513046.962, + 513080.306, + 513113.65, + 513146.994, + 513180.338, + 513213.68200000003, + 513247.026, + 513280.37, + 513313.71400000004, + 513347.058, + 513380.402, + 513413.745, + 513447.08900000004, + 513480.433, + 513513.777, + 513547.121, + 513580.465, + 513613.809, + 513647.153, + 513680.49700000003, + 513713.841, + 513747.185, + 513780.52900000004, + 513813.873, + 513847.216, + 513880.56, + 513913.90400000004, + 513947.248, + 513980.592, + 514013.936, + 514047.28, + 514080.624, + 514113.968, + 514147.31200000003, + 514180.656, + 514214.0, + 514247.344, + 514280.688, + 514314.031, + 514347.375, + 514380.719, + 514414.063, + 514447.407, + 514480.751, + 514514.09500000003, + 514547.439, + 514580.783, + 514614.12700000004, + 514647.471, + 514680.815, + 514714.159, + 514747.50200000004, + 514780.846, + 514814.19, + 514847.534, + 514880.878, + 514914.222, + 514947.566, + 514980.91000000003, + 515014.254, + 515047.598, + 515080.94200000004, + 515114.286, + 515147.63, + 515180.973, + 515214.31700000004, + 515247.661, + 515281.005, + 515314.349, + 515347.693, + 515381.037, + 515414.381, + 515447.72500000003, + 515481.069, + 515514.413, + 515547.757, + 515581.101, + 515614.445, + 515647.788, + 515681.132, + 515714.476, + 515747.82, + 515781.164, + 515814.50800000003, + 515847.852, + 515881.196, + 515914.54000000004, + 515947.884, + 515981.228, + 516014.572, + 516047.916, + 516081.259, + 516114.603, + 516147.947, + 516181.291, + 516214.635, + 516247.979, + 516281.32300000003, + 516314.667, + 516348.011, + 516381.35500000004, + 516414.699, + 516448.043, + 516481.387, + 516514.73000000004, + 516548.074, + 516581.418, + 516614.762, + 516648.106, + 516681.45, + 516714.794, + 516748.13800000004, + 516781.482, + 516814.826, + 516848.17, + 516881.514, + 516914.858, + 516948.202, + 516981.545, + 517014.889, + 517048.233, + 517081.577, + 517114.92100000003, + 517148.265, + 517181.609, + 517214.95300000004, + 517248.297, + 517281.641, + 517314.985, + 517348.329, + 517381.673, + 517415.016, + 517448.36, + 517481.704, + 517515.048, + 517548.392, + 517581.73600000003, + 517615.08, + 517648.424, + 517681.768, + 517715.112, + 517748.456, + 517781.8, + 517815.14400000003, + 517848.487, + 517881.831, + 517915.175, + 517948.51900000003, + 517981.863, + 518015.207, + 518048.55100000004, + 518081.895, + 518115.239, + 518148.583, + 518181.927, + 518215.271, + 518248.615, + 518281.95900000003, + 518315.302, + 518348.646, + 518381.99, + 518415.33400000003, + 518448.678, + 518482.022, + 518515.36600000004, + 518548.71, + 518582.054, + 518615.398, + 518648.742, + 518682.086, + 518715.43, + 518748.773, + 518782.117, + 518815.461, + 518848.805, + 518882.14900000003, + 518915.493, + 518948.837, + 518982.181, + 519015.525, + 519048.869, + 519082.213, + 519115.55700000003, + 519148.901, + 519182.244, + 519215.588, + 519248.93200000003, + 519282.276, + 519315.62, + 519348.96400000004, + 519382.308, + 519415.652, + 519448.996, + 519482.34, + 519515.684, + 519549.028, + 519582.37200000003, + 519615.716, + 519649.059, + 519682.403, + 519715.74700000003, + 519749.091, + 519782.435, + 519815.77900000004, + 519849.123, + 519882.467, + 519915.811, + 519949.155, + 519982.499, + 520015.843, + 520049.18700000003, + 520082.53, + 520115.874, + 520149.218, + 520182.56200000003, + 520215.906, + 520249.25, + 520282.594, + 520315.938, + 520349.282, + 520382.626, + 520415.97000000003, + 520449.314, + 520482.658, + 520516.001, + 520549.34500000003, + 520582.689, + 520616.033, + 520649.37700000004, + 520682.721, + 520716.065, + 520749.409, + 520782.753, + 520816.097, + 520849.441, + 520882.78500000003, + 520916.129, + 520949.473, + 520982.816, + 521016.16000000003, + 521049.504, + 521082.848, + 521116.19200000004, + 521149.536, + 521182.88, + 521216.224, + 521249.568, + 521282.912, + 521316.256, + 521349.60000000003, + 521382.944, + 521416.287, + 521449.631, + 521482.97500000003, + 521516.319, + 521549.663, + 521583.007, + 521616.351, + 521649.695, + 521683.039, + 521716.38300000003, + 521749.727, + 521783.071, + 521816.41500000004, + 521849.75800000003, + 521883.102, + 521916.446, + 521949.79000000004, + 521983.134, + 522016.478, + 522049.822, + 522083.166, + 522116.51, + 522149.854, + 522183.19800000003, + 522216.542, + 522249.886, + 522283.23000000004, + 522316.57300000003, + 522349.917, + 522383.261, + 522416.60500000004, + 522449.949, + 522483.293, + 522516.637, + 522549.981, + 522583.325, + 522616.669, + 522650.01300000004, + 522683.357, + 522716.701, + 522750.044, + 522783.38800000004, + 522816.732, + 522850.076, + 522883.42 + ], + "depth_timestamps_ms": [ + 511179.398, + 511212.742, + 511246.086, + 511279.43, + 511312.77400000003, + 511346.118, + 511379.462, + 511412.806, + 511446.15, + 511479.494, + 511512.838, + 511546.18200000003, + 511579.526, + 511612.869, + 511646.213, + 511679.55700000003, + 511712.901, + 511746.245, + 511779.58900000004, + 511812.933, + 511846.277, + 511879.621, + 511912.965, + 511946.309, + 511979.653, + 512012.99700000003, + 512046.34, + 512079.684, + 512113.028, + 512146.37200000003, + 512179.716, + 512213.06, + 512246.40400000004, + 512279.748, + 512313.092, + 512346.436, + 512379.78, + 512413.124, + 512446.468, + 512479.81200000003, + 512513.155, + 512546.499, + 512579.843, + 512613.18700000003, + 512646.531, + 512679.875, + 512713.219, + 512746.563, + 512779.907, + 512813.251, + 512846.59500000003, + 512879.939, + 512913.283, + 512946.626, + 512979.97000000003, + 513013.314, + 513046.658, + 513080.00200000004, + 513113.346, + 513146.69, + 513180.034, + 513213.378, + 513246.722, + 513280.066, + 513313.41000000003, + 513346.754, + 513380.097, + 513413.441, + 513446.78500000003, + 513480.129, + 513513.473, + 513546.81700000004, + 513580.161, + 513613.505, + 513646.849, + 513680.193, + 513713.537, + 513746.881, + 513780.22500000003, + 513813.569, + 513846.912, + 513880.256, + 513913.60000000003, + 513946.944, + 513980.288, + 514013.632, + 514046.976, + 514080.32, + 514113.664, + 514147.00800000003, + 514180.352, + 514213.696, + 514247.04000000004, + 514280.38300000003, + 514313.727, + 514347.071, + 514380.41500000004, + 514413.759, + 514447.103, + 514480.447, + 514513.791, + 514547.135, + 514580.479, + 514613.82300000003, + 514647.167, + 514680.511, + 514713.854, + 514747.19800000003, + 514780.542, + 514813.886, + 514847.23000000004, + 514880.574, + 514913.918, + 514947.262, + 514980.606, + 515013.95, + 515047.294, + 515080.63800000004, + 515113.982, + 515147.326, + 515180.669, + 515214.01300000004, + 515247.357, + 515280.701, + 515314.045, + 515347.389, + 515380.733, + 515414.077, + 515447.42100000003, + 515480.765, + 515514.109, + 515547.45300000004, + 515580.797, + 515614.14, + 515647.484, + 515680.82800000004, + 515714.172, + 515747.516, + 515780.86, + 515814.204, + 515847.548, + 515880.892, + 515914.23600000003, + 515947.58, + 515980.924, + 516014.268, + 516047.61100000003, + 516080.955, + 516114.299, + 516147.643, + 516180.987, + 516214.331, + 516247.675, + 516281.01900000003, + 516314.363, + 516347.707, + 516381.05100000004, + 516414.395, + 516447.739, + 516481.083, + 516514.42600000004, + 516547.77, + 516581.114, + 516614.458, + 516647.802, + 516681.146, + 516714.49, + 516747.83400000003, + 516781.178, + 516814.522, + 516847.86600000004, + 516881.21, + 516914.554, + 516947.897, + 516981.24100000004, + 517014.585, + 517047.929, + 517081.273, + 517114.617, + 517147.961, + 517181.305, + 517214.64900000003, + 517247.993, + 517281.337, + 517314.681, + 517348.025, + 517381.368, + 517414.712, + 517448.056, + 517481.4, + 517514.744, + 517548.088, + 517581.43200000003, + 517614.776, + 517648.12, + 517681.46400000004, + 517714.808, + 517748.152, + 517781.496, + 517814.84, + 517848.183, + 517881.527, + 517914.871, + 517948.215, + 517981.559, + 518014.903, + 518048.24700000003, + 518081.591, + 518114.935, + 518148.27900000004, + 518181.623, + 518214.967, + 518248.311, + 518281.65400000004, + 518314.998, + 518348.342, + 518381.686, + 518415.03, + 518448.374, + 518481.718, + 518515.06200000003, + 518548.406, + 518581.75, + 518615.094, + 518648.438, + 518681.782, + 518715.125, + 518748.469, + 518781.813, + 518815.157, + 518848.501, + 518881.84500000003, + 518915.189, + 518948.533, + 518981.87700000004, + 519015.221, + 519048.565, + 519081.909, + 519115.253, + 519148.597, + 519181.94, + 519215.284, + 519248.628, + 519281.972, + 519315.316, + 519348.66000000003, + 519382.004, + 519415.348, + 519448.69200000004, + 519482.036, + 519515.38, + 519548.724, + 519582.068, + 519615.411, + 519648.755, + 519682.099, + 519715.443, + 519748.787, + 519782.131, + 519815.47500000003, + 519848.819, + 519882.163, + 519915.507, + 519948.851, + 519982.195, + 520015.539, + 520048.882, + 520082.226, + 520115.57, + 520148.914, + 520182.25800000003, + 520215.602, + 520248.946, + 520282.29000000004, + 520315.634, + 520348.978, + 520382.322, + 520415.666, + 520449.01, + 520482.354, + 520515.697, + 520549.041, + 520582.385, + 520615.729, + 520649.07300000003, + 520682.417, + 520715.761, + 520749.10500000004, + 520782.449, + 520815.793, + 520849.137, + 520882.481, + 520915.825, + 520949.168, + 520982.512, + 521015.856, + 521049.2, + 521082.544, + 521115.88800000004, + 521149.232, + 521182.576, + 521215.92, + 521249.264, + 521282.608, + 521315.952, + 521349.29600000003, + 521382.639, + 521415.983, + 521449.327, + 521482.67100000003, + 521516.015, + 521549.359, + 521582.70300000004, + 521616.047, + 521649.391, + 521682.735, + 521716.079, + 521749.423, + 521782.767, + 521816.11100000003, + 521849.454, + 521882.798, + 521916.142, + 521949.48600000003, + 521982.83, + 522016.174, + 522049.518, + 522082.862, + 522116.206, + 522149.55, + 522182.89400000003, + 522216.238, + 522249.582, + 522282.925, + 522316.26900000003, + 522349.613, + 522382.957, + 522416.30100000004, + 522449.645, + 522482.989, + 522516.333, + 522549.677, + 522583.021, + 522616.365, + 522649.70900000003, + 522683.053, + 522716.396, + 522749.74, + 522783.08400000003, + 522816.428, + 522849.772, + 522883.11600000004 + ], + "color_frame_numbers": [ + 6178, + 6179, + 6180, + 6181, + 6182, + 6183, + 6184, + 6185, + 6186, + 6187, + 6188, + 6189, + 6190, + 6191, + 6192, + 6193, + 6194, + 6195, + 6196, + 6197, + 6198, + 6199, + 6200, + 6201, + 6202, + 6203, + 6204, + 6205, + 6206, + 6207, + 6208, + 6209, + 6210, + 6211, + 6212, + 6213, + 6214, + 6215, + 6216, + 6217, + 6218, + 6219, + 6220, + 6221, + 6222, + 6223, + 6224, + 6225, + 6226, + 6227, + 6228, + 6229, + 6230, + 6231, + 6232, + 6233, + 6234, + 6235, + 6236, + 6237, + 6238, + 6239, + 6240, + 6241, + 6242, + 6243, + 6244, + 6245, + 6246, + 6247, + 6248, + 6249, + 6250, + 6251, + 6252, + 6253, + 6254, + 6255, + 6256, + 6257, + 6258, + 6259, + 6260, + 6261, + 6262, + 6263, + 6264, + 6265, + 6266, + 6267, + 6268, + 6269, + 6270, + 6271, + 6272, + 6273, + 6274, + 6275, + 6276, + 6277, + 6278, + 6279, + 6280, + 6281, + 6282, + 6283, + 6284, + 6285, + 6286, + 6287, + 6288, + 6289, + 6290, + 6291, + 6292, + 6293, + 6294, + 6295, + 6296, + 6297, + 6298, + 6299, + 6300, + 6301, + 6302, + 6303, + 6304, + 6305, + 6306, + 6307, + 6308, + 6309, + 6310, + 6311, + 6312, + 6313, + 6314, + 6315, + 6316, + 6317, + 6318, + 6319, + 6320, + 6321, + 6322, + 6323, + 6324, + 6325, + 6326, + 6327, + 6328, + 6329, + 6330, + 6331, + 6332, + 6333, + 6334, + 6335, + 6336, + 6337, + 6338, + 6339, + 6340, + 6341, + 6342, + 6343, + 6344, + 6345, + 6346, + 6347, + 6348, + 6349, + 6350, + 6351, + 6352, + 6353, + 6354, + 6355, + 6356, + 6357, + 6358, + 6359, + 6360, + 6361, + 6362, + 6363, + 6364, + 6365, + 6366, + 6367, + 6368, + 6369, + 6370, + 6371, + 6372, + 6373, + 6374, + 6375, + 6376, + 6377, + 6378, + 6379, + 6380, + 6381, + 6382, + 6383, + 6384, + 6385, + 6386, + 6387, + 6388, + 6389, + 6390, + 6391, + 6392, + 6393, + 6394, + 6395, + 6396, + 6397, + 6398, + 6399, + 6400, + 6401, + 6402, + 6403, + 6404, + 6405, + 6406, + 6407, + 6408, + 6409, + 6410, + 6411, + 6412, + 6413, + 6414, + 6415, + 6416, + 6417, + 6418, + 6419, + 6420, + 6421, + 6422, + 6423, + 6424, + 6425, + 6426, + 6427, + 6428, + 6429, + 6430, + 6431, + 6432, + 6433, + 6434, + 6435, + 6436, + 6437, + 6438, + 6439, + 6440, + 6441, + 6442, + 6443, + 6444, + 6445, + 6446, + 6447, + 6448, + 6449, + 6450, + 6451, + 6452, + 6453, + 6454, + 6455, + 6456, + 6457, + 6458, + 6459, + 6460, + 6461, + 6462, + 6463, + 6464, + 6465, + 6466, + 6467, + 6468, + 6469, + 6470, + 6471, + 6472, + 6473, + 6474, + 6475, + 6476, + 6477, + 6478, + 6479, + 6480, + 6481, + 6482, + 6483, + 6484, + 6485, + 6486, + 6487, + 6488, + 6489, + 6490, + 6491, + 6492, + 6493, + 6494, + 6495, + 6496, + 6497, + 6498, + 6499, + 6500, + 6501, + 6502, + 6503, + 6504, + 6505, + 6506, + 6507, + 6508, + 6509, + 6510, + 6511, + 6512, + 6513, + 6514, + 6515, + 6516, + 6517, + 6518, + 6519, + 6520, + 6521, + 6522, + 6523, + 6524, + 6525, + 6526, + 6527, + 6528, + 6529 + ], + "depth_frame_numbers": [ + 6192, + 6193, + 6194, + 6195, + 6196, + 6197, + 6198, + 6199, + 6200, + 6201, + 6202, + 6203, + 6204, + 6205, + 6206, + 6207, + 6208, + 6209, + 6210, + 6211, + 6212, + 6213, + 6214, + 6215, + 6216, + 6217, + 6218, + 6219, + 6220, + 6221, + 6222, + 6223, + 6224, + 6225, + 6226, + 6227, + 6228, + 6229, + 6230, + 6231, + 6232, + 6233, + 6234, + 6235, + 6236, + 6237, + 6238, + 6239, + 6240, + 6241, + 6242, + 6243, + 6244, + 6245, + 6246, + 6247, + 6248, + 6249, + 6250, + 6251, + 6252, + 6253, + 6254, + 6255, + 6256, + 6257, + 6258, + 6259, + 6260, + 6261, + 6262, + 6263, + 6264, + 6265, + 6266, + 6267, + 6268, + 6269, + 6270, + 6271, + 6272, + 6273, + 6274, + 6275, + 6276, + 6277, + 6278, + 6279, + 6280, + 6281, + 6282, + 6283, + 6284, + 6285, + 6286, + 6287, + 6288, + 6289, + 6290, + 6291, + 6292, + 6293, + 6294, + 6295, + 6296, + 6297, + 6298, + 6299, + 6300, + 6301, + 6302, + 6303, + 6304, + 6305, + 6306, + 6307, + 6308, + 6309, + 6310, + 6311, + 6312, + 6313, + 6314, + 6315, + 6316, + 6317, + 6318, + 6319, + 6320, + 6321, + 6322, + 6323, + 6324, + 6325, + 6326, + 6327, + 6328, + 6329, + 6330, + 6331, + 6332, + 6333, + 6334, + 6335, + 6336, + 6337, + 6338, + 6339, + 6340, + 6341, + 6342, + 6343, + 6344, + 6345, + 6346, + 6347, + 6348, + 6349, + 6350, + 6351, + 6352, + 6353, + 6354, + 6355, + 6356, + 6357, + 6358, + 6359, + 6360, + 6361, + 6362, + 6363, + 6364, + 6365, + 6366, + 6367, + 6368, + 6369, + 6370, + 6371, + 6372, + 6373, + 6374, + 6375, + 6376, + 6377, + 6378, + 6379, + 6380, + 6381, + 6382, + 6383, + 6384, + 6385, + 6386, + 6387, + 6388, + 6389, + 6390, + 6391, + 6392, + 6393, + 6394, + 6395, + 6396, + 6397, + 6398, + 6399, + 6400, + 6401, + 6402, + 6403, + 6404, + 6405, + 6406, + 6407, + 6408, + 6409, + 6410, + 6411, + 6412, + 6413, + 6414, + 6415, + 6416, + 6417, + 6418, + 6419, + 6420, + 6421, + 6422, + 6423, + 6424, + 6425, + 6426, + 6427, + 6428, + 6429, + 6430, + 6431, + 6432, + 6433, + 6434, + 6435, + 6436, + 6437, + 6438, + 6439, + 6440, + 6441, + 6442, + 6443, + 6444, + 6445, + 6446, + 6447, + 6448, + 6449, + 6450, + 6451, + 6452, + 6453, + 6454, + 6455, + 6456, + 6457, + 6458, + 6459, + 6460, + 6461, + 6462, + 6463, + 6464, + 6465, + 6466, + 6467, + 6468, + 6469, + 6470, + 6471, + 6472, + 6473, + 6474, + 6475, + 6476, + 6477, + 6478, + 6479, + 6480, + 6481, + 6482, + 6483, + 6484, + 6485, + 6486, + 6487, + 6488, + 6489, + 6490, + 6491, + 6492, + 6493, + 6494, + 6495, + 6496, + 6497, + 6498, + 6499, + 6500, + 6501, + 6502, + 6503, + 6504, + 6505, + 6506, + 6507, + 6508, + 6509, + 6510, + 6511, + 6512, + 6513, + 6514, + 6515, + 6516, + 6517, + 6518, + 6519, + 6520, + 6521, + 6522, + 6523, + 6524, + 6525, + 6526, + 6527, + 6528, + 6529, + 6530, + 6531, + 6532, + 6533, + 6534, + 6535, + 6536, + 6537, + 6538, + 6539, + 6540, + 6541, + 6542, + 6543 + ], + "host_arrival_monotonic_ns": [ + 791630160462, + 791662625688, + 791697053890, + 791730869230, + 791763895729, + 791796555406, + 791830521225, + 791862881403, + 791897824571, + 791929241674, + 791963682450, + 791996707457, + 792029222226, + 792062988574, + 792094975938, + 792131033968, + 792162627702, + 792195778833, + 792227934844, + 792261165117, + 792295959511, + 792327331981, + 792362175626, + 792395061453, + 792429581072, + 792462485560, + 792496156022, + 792530332787, + 792562442368, + 792596949440, + 792628721725, + 792663729927, + 792695923458, + 792729364153, + 792762453392, + 792799796231, + 792849068761, + 792869790966, + 792897532782, + 792928295419, + 792965354228, + 792996352584, + 793027776506, + 793063557568, + 793095284822, + 793130649862, + 793162471659, + 793196599146, + 793229558748, + 793263009971, + 793296433271, + 793328107917, + 793362161792, + 793397643937, + 793429381633, + 793462432244, + 793496952622, + 793527726907, + 793564266842, + 793595809004, + 793628377153, + 793660807320, + 793694445569, + 793728463812, + 793762095521, + 793803464198, + 793829943390, + 793863227607, + 793895757731, + 793928186026, + 793962655249, + 793994804170, + 794028386026, + 794061792766, + 794094995990, + 794128698456, + 794161992398, + 794196112997, + 794230678084, + 794264006566, + 794298032626, + 794331020383, + 794363994246, + 794397289572, + 794431273454, + 794464646377, + 794496822431, + 794529246199, + 794562103051, + 794596403089, + 794632399879, + 794662224818, + 794697013061, + 794729044474, + 794762654780, + 794797565383, + 794829951578, + 794864813428, + 794896425203, + 794932450860, + 794965430889, + 794997553682, + 795030635088, + 795063963156, + 795096905714, + 795129064673, + 795161383836, + 795196967135, + 795230515114, + 795264160349, + 795296408395, + 795329402387, + 795362717780, + 795397541668, + 795430012007, + 795463844538, + 795496921604, + 795532979607, + 795564286363, + 795598019781, + 795631532165, + 795664974405, + 795697614200, + 795729730368, + 795763803304, + 795797759796, + 795831234134, + 795863144081, + 795899871848, + 795930476260, + 795966606854, + 795997690608, + 796030718515, + 796064040830, + 796096840892, + 796130363954, + 796162599328, + 796195542681, + 796229497810, + 796263675800, + 796296122239, + 796328358350, + 796361361479, + 796395827681, + 796428541880, + 796463792296, + 796497095472, + 796532161578, + 796565694732, + 796597149171, + 796628622878, + 796662103284, + 796695536983, + 796728391750, + 796761520036, + 796793849306, + 796830932955, + 796862118048, + 796895052087, + 796929384224, + 796963362676, + 796996703795, + 797028966068, + 797064681996, + 797101999253, + 797131916135, + 797166655140, + 797197702773, + 797231147959, + 797265541842, + 797298138403, + 797330615743, + 797365202561, + 797396104366, + 797429492507, + 797464638839, + 797497332226, + 797530507990, + 797563015536, + 797597041611, + 797629296557, + 797664779096, + 797697926527, + 797730370765, + 797764963286, + 797797463294, + 797830083496, + 797863979106, + 797898099235, + 797932652799, + 797965870970, + 797996587150, + 798030151209, + 798063353329, + 798096036284, + 798130835595, + 798163598138, + 798197909373, + 798230639355, + 798264443605, + 798296201029, + 798330555038, + 798364317366, + 798397227101, + 798430875206, + 798462948440, + 798496469084, + 798530001899, + 798563071455, + 798595423781, + 798629318765, + 798665476210, + 798697995255, + 798730403622, + 798762988694, + 798797639996, + 798830457269, + 798866391975, + 798897920436, + 798930723388, + 798966880286, + 798997782166, + 799032854056, + 799063502157, + 799097910738, + 799131431870, + 799166416504, + 799198139603, + 799233028071, + 799265497956, + 799298720841, + 799331859070, + 799364692050, + 799400024856, + 799431864029, + 799465377030, + 799498004065, + 799530706611, + 799562332509, + 799595775022, + 799629337501, + 799662991554, + 799696394757, + 799733978267, + 799764702757, + 799797672818, + 799831732681, + 799864249060, + 799898455620, + 799931544023, + 799966127003, + 799999305024, + 800031637103, + 800066008614, + 800097274062, + 800129340310, + 800165132847, + 800199620491, + 800231702216, + 800264401957, + 800297853691, + 800332709574, + 800365106174, + 800397596192, + 800432011294, + 800465620246, + 800498498639, + 800531183598, + 800563690688, + 800597185609, + 800631849410, + 800662737485, + 800696738233, + 800729357289, + 800766410405, + 800797484674, + 800831715197, + 800864709963, + 800896913941, + 800932309638, + 800967590176, + 800998100427, + 801032618670, + 801064613271, + 801096972395, + 801131412051, + 801164580004, + 801197010405, + 801232740669, + 801264123681, + 801296965058, + 801329423582, + 801363203340, + 801398367595, + 801432257459, + 801463959867, + 801496400375, + 801530196170, + 801564447235, + 801597088529, + 801631650760, + 801663979011, + 801696766914, + 801730415589, + 801764524207, + 801797320126, + 801830183933, + 801864965650, + 801897580407, + 801930614062, + 801966493725, + 801998191292, + 802032066484, + 802064115086, + 802096128570, + 802130020061, + 802165808028, + 802198957299, + 802231944968, + 802266136685, + 802298812267, + 802331761467, + 802365246490, + 802399155616, + 802431655897, + 802463738468, + 802497053264, + 802529861089, + 802562446376, + 802598863823, + 802632008279, + 802665274395, + 802696889134, + 802730490863, + 802763994726, + 802798748852, + 802834757608, + 802866576490, + 802897468279, + 802931712061, + 802965129227, + 802997643887, + 803033501713, + 803065978379, + 803097859908, + 803130918785, + 803165991775, + 803199388363, + 803232271586, + 803264528122, + 803297842528, + 803332374295 + ], + "dropped_frames": 0, + "timestamp_backjumps": 0, + "sync_note": "\u6bcf\u884c\u6570\u636e\u6765\u81ea\u540c\u4e00 frameset\uff1bD405 \u5f69\u8272\u53d6\u81ea\u5de6\u6210\u50cf\u5668\uff0c\u4e0e\u6df1\u5ea6\u540c\u4e00\u6b21\u66dd\u5149\uff0c\u786c\u4ef6\u7ea7\u540c\u6b65" +} \ No newline at end of file diff --git a/docs/20260916_104026/intrinsics.json b/docs/20260916_104026/intrinsics.json new file mode 100644 index 0000000..1af3421 --- /dev/null +++ b/docs/20260916_104026/intrinsics.json @@ -0,0 +1,4494 @@ +{ + "fx": 437.34423828125, + "fy": 436.91009521484375, + "cx": 427.2064208984375, + "cy": 246.0908660888672, + "depth_scale_m": 9.999999747378752e-05, + "aligned_to": "color", + "frames": 893, + "width": 848, + "height": 480, + "fps_nominal": 30, + "fps_actual": 30.0, + "save_depth": true, + "time_source": "device hardware clock, global_time disabled", + "timestamp_domain": "timestamp_domain.hardware_clock", + "timestamps_ms": [ + 246981.283, + 247014.62600000002, + 247047.97, + 247081.314, + 247114.658, + 247148.002, + 247181.34600000002, + 247214.69, + 247248.034, + 247281.378, + 247314.722, + 247348.066, + 247381.41, + 247414.75400000002, + 247448.097, + 247481.441, + 247514.785, + 247548.12900000002, + 247581.473, + 247614.817, + 247648.161, + 247681.505, + 247714.84900000002, + 247748.193, + 247781.537, + 247814.881, + 247848.225, + 247881.568, + 247914.912, + 247948.256, + 247981.6, + 248014.94400000002, + 248048.288, + 248081.632, + 248114.976, + 248148.32, + 248181.66400000002, + 248215.008, + 248248.352, + 248281.696, + 248315.04, + 248348.383, + 248381.727, + 248415.071, + 248448.415, + 248481.759, + 248515.103, + 248548.44700000001, + 248581.791, + 248615.135, + 248648.479, + 248681.823, + 248715.16700000002, + 248748.511, + 248781.854, + 248815.198, + 248848.54200000002, + 248881.886, + 248915.23, + 248948.574, + 248981.918, + 249015.26200000002, + 249048.606, + 249081.95, + 249115.294, + 249148.638, + 249181.98200000002, + 249215.325, + 249248.669, + 249282.013, + 249315.35700000002, + 249348.701, + 249382.045, + 249415.389, + 249448.733, + 249482.07700000002, + 249515.421, + 249548.765, + 249582.109, + 249615.453, + 249648.797, + 249682.14, + 249715.484, + 249748.828, + 249782.172, + 249815.516, + 249848.86000000002, + 249882.204, + 249915.548, + 249948.892, + 249982.236, + 250015.58000000002, + 250048.924, + 250082.268, + 250115.611, + 250148.95500000002, + 250182.299, + 250215.643, + 250248.987, + 250282.331, + 250315.67500000002, + 250349.019, + 250382.363, + 250415.707, + 250449.051, + 250482.39500000002, + 250515.739, + 250549.082, + 250582.426, + 250615.77000000002, + 250649.114, + 250682.458, + 250715.802, + 250749.146, + 250782.49, + 250815.834, + 250849.178, + 250882.522, + 250915.866, + 250949.21, + 250982.554, + 251015.897, + 251049.241, + 251082.585, + 251115.929, + 251149.27300000002, + 251182.617, + 251215.961, + 251249.305, + 251282.649, + 251315.99300000002, + 251349.337, + 251382.681, + 251416.025, + 251449.36800000002, + 251482.712, + 251516.056, + 251549.4, + 251582.744, + 251616.08800000002, + 251649.432, + 251682.776, + 251716.12, + 251749.464, + 251782.80800000002, + 251816.152, + 251849.496, + 251882.839, + 251916.18300000002, + 251949.527, + 251982.871, + 252016.215, + 252049.559, + 252082.903, + 252116.247, + 252149.59100000001, + 252182.935, + 252216.279, + 252249.623, + 252282.967, + 252316.31100000002, + 252349.654, + 252382.998, + 252416.342, + 252449.68600000002, + 252483.03, + 252516.374, + 252549.718, + 252583.062, + 252616.40600000002, + 252649.75, + 252683.094, + 252716.438, + 252749.782, + 252783.125, + 252816.469, + 252849.813, + 252883.157, + 252916.50100000002, + 252949.845, + 252983.189, + 253016.533, + 253049.877, + 253083.22100000002, + 253116.565, + 253149.909, + 253183.253, + 253216.59600000002, + 253249.94, + 253283.284, + 253316.628, + 253349.972, + 253383.316, + 253416.66, + 253450.00400000002, + 253483.348, + 253516.692, + 253550.036, + 253583.38, + 253616.72400000002, + 253650.068, + 253683.411, + 253716.755, + 253750.09900000002, + 253783.443, + 253816.787, + 253850.131, + 253883.475, + 253916.81900000002, + 253950.163, + 253983.507, + 254016.851, + 254050.195, + 254083.53900000002, + 254116.882, + 254150.226, + 254183.57, + 254216.91400000002, + 254250.258, + 254283.602, + 254316.946, + 254350.29, + 254383.634, + 254416.978, + 254450.32200000001, + 254483.666, + 254517.01, + 254550.353, + 254583.69700000001, + 254617.041, + 254650.385, + 254683.729, + 254717.073, + 254750.41700000002, + 254783.761, + 254817.105, + 254850.449, + 254883.793, + 254917.13700000002, + 254950.481, + 254983.825, + 255017.168, + 255050.51200000002, + 255083.856, + 255117.2, + 255150.544, + 255183.888, + 255217.23200000002, + 255250.576, + 255283.92, + 255317.264, + 255350.608, + 255383.95200000002, + 255417.296, + 255450.639, + 255483.983, + 255517.32700000002, + 255550.671, + 255584.015, + 255617.359, + 255650.703, + 255684.047, + 255717.391, + 255750.73500000002, + 255784.079, + 255817.423, + 255850.767, + 255884.11000000002, + 255917.454, + 255950.798, + 255984.142, + 256017.486, + 256050.83000000002, + 256084.174, + 256117.518, + 256150.862, + 256184.206, + 256217.55000000002, + 256250.894, + 256284.238, + 256317.582, + 256350.92500000002, + 256384.269, + 256417.613, + 256450.957, + 256484.301, + 256517.64500000002, + 256550.989, + 256584.333, + 256617.677, + 256651.021, + 256684.36500000002, + 256717.709, + 256751.053, + 256784.396, + 256817.74000000002, + 256851.084, + 256884.428, + 256917.772, + 256951.116, + 256984.46, + 257017.804, + 257051.14800000002, + 257084.492, + 257117.836, + 257151.18, + 257184.524, + 257217.867, + 257251.211, + 257284.555, + 257317.899, + 257351.24300000002, + 257384.587, + 257417.931, + 257451.275, + 257484.619, + 257517.96300000002, + 257551.307, + 257584.651, + 257617.995, + 257651.339, + 257684.682, + 257718.026, + 257751.37, + 257784.714, + 257818.05800000002, + 257851.402, + 257884.746, + 257918.09, + 257951.434, + 257984.778, + 258018.122, + 258051.46600000001, + 258084.81, + 258118.153, + 258151.497, + 258184.84100000001, + 258218.185, + 258251.529, + 258284.873, + 258318.217, + 258351.56100000002, + 258384.905, + 258418.249, + 258451.593, + 258484.937, + 258518.28100000002, + 258551.624, + 258584.968, + 258618.312, + 258651.65600000002, + 258685.0, + 258718.344, + 258751.688, + 258785.032, + 258818.37600000002, + 258851.72, + 258885.064, + 258918.408, + 258951.752, + 258985.09600000002, + 259018.439, + 259051.783, + 259085.127, + 259118.47100000002, + 259151.815, + 259185.159, + 259218.503, + 259251.847, + 259285.191, + 259318.535, + 259351.87900000002, + 259385.223, + 259418.567, + 259451.91, + 259485.25400000002, + 259518.598, + 259551.942, + 259585.286, + 259618.63, + 259651.97400000002, + 259685.318, + 259718.662, + 259752.006, + 259785.35, + 259818.69400000002, + 259852.038, + 259885.381, + 259918.725, + 259952.06900000002, + 259985.413, + 260018.757, + 260052.101, + 260085.445, + 260118.78900000002, + 260152.133, + 260185.477, + 260218.821, + 260252.165, + 260285.509, + 260318.853, + 260352.196, + 260385.54, + 260418.884, + 260452.228, + 260485.57200000001, + 260518.916, + 260552.26, + 260585.604, + 260618.948, + 260652.29200000002, + 260685.636, + 260718.98, + 260752.324, + 260785.66700000002, + 260819.011, + 260852.355, + 260885.699, + 260919.043, + 260952.38700000002, + 260985.731, + 261019.075, + 261052.419, + 261085.763, + 261119.10700000002, + 261152.451, + 261185.795, + 261219.138, + 261252.48200000002, + 261285.826, + 261319.17, + 261352.514, + 261385.858, + 261419.20200000002, + 261452.546, + 261485.89, + 261519.234, + 261552.578, + 261585.922, + 261619.266, + 261652.61000000002, + 261685.953, + 261719.297, + 261752.641, + 261785.98500000002, + 261819.329, + 261852.673, + 261886.017, + 261919.361, + 261952.70500000002, + 261986.049, + 262019.393, + 262052.737, + 262086.081, + 262119.424, + 262152.768, + 262186.112, + 262219.456, + 262252.8, + 262286.14400000003, + 262319.488, + 262352.832, + 262386.176, + 262419.52, + 262452.864, + 262486.208, + 262519.552, + 262552.895, + 262586.239, + 262619.583, + 262652.927, + 262686.271, + 262719.615, + 262752.95900000003, + 262786.303, + 262819.647, + 262852.991, + 262886.335, + 262919.679, + 262953.023, + 262986.367, + 263019.71, + 263053.054, + 263086.398, + 263119.742, + 263153.086, + 263186.43, + 263219.77400000003, + 263253.118, + 263286.462, + 263319.806, + 263353.15, + 263386.494, + 263419.838, + 263453.181, + 263486.525, + 263519.869, + 263553.213, + 263586.55700000003, + 263619.901, + 263653.245, + 263686.589, + 263719.933, + 263753.277, + 263786.621, + 263819.965, + 263853.309, + 263886.652, + 263919.996, + 263953.34, + 263986.684, + 264020.028, + 264053.37200000003, + 264086.716, + 264120.06, + 264153.404, + 264186.748, + 264220.092, + 264253.436, + 264286.78, + 264320.124, + 264353.467, + 264386.811, + 264420.155, + 264453.499, + 264486.843, + 264520.18700000003, + 264553.531, + 264586.875, + 264620.219, + 264653.563, + 264686.907, + 264720.251, + 264753.59500000003, + 264786.938, + 264820.282, + 264853.626, + 264886.97000000003, + 264920.314, + 264953.658, + 264987.002, + 265020.346, + 265053.69, + 265087.034, + 265120.378, + 265153.722, + 265187.066, + 265220.409, + 265253.753, + 265287.097, + 265320.441, + 265353.78500000003, + 265387.129, + 265420.473, + 265453.817, + 265487.161, + 265520.505, + 265553.849, + 265587.193, + 265620.537, + 265653.881, + 265687.224, + 265720.568, + 265753.912, + 265787.256, + 265820.6, + 265853.944, + 265887.288, + 265920.632, + 265953.976, + 265987.32, + 266020.664, + 266054.00800000003, + 266087.352, + 266120.695, + 266154.039, + 266187.38300000003, + 266220.727, + 266254.071, + 266287.415, + 266320.759, + 266354.103, + 266387.447, + 266420.791, + 266454.135, + 266487.479, + 266520.82300000003, + 266554.166, + 266587.51, + 266620.854, + 266654.19800000003, + 266687.542, + 266720.886, + 266754.23, + 266787.574, + 266820.918, + 266854.262, + 266887.606, + 266920.95, + 266954.294, + 266987.638, + 267020.981, + 267054.325, + 267087.669, + 267121.013, + 267154.357, + 267187.701, + 267221.045, + 267254.389, + 267287.733, + 267321.077, + 267354.42100000003, + 267387.765, + 267421.109, + 267454.452, + 267487.79600000003, + 267521.14, + 267554.484, + 267587.828, + 267621.172, + 267654.516, + 267687.86, + 267721.204, + 267754.548, + 267787.892, + 267821.23600000003, + 267854.58, + 267887.923, + 267921.267, + 267954.61100000003, + 267987.955, + 268021.299, + 268054.643, + 268087.987, + 268121.331, + 268154.675, + 268188.01900000003, + 268221.363, + 268254.707, + 268288.051, + 268321.395, + 268354.738, + 268388.082, + 268421.426, + 268454.77, + 268488.114, + 268521.458, + 268554.802, + 268588.146, + 268621.49, + 268654.83400000003, + 268688.178, + 268721.522, + 268754.866, + 268788.20900000003, + 268821.553, + 268854.897, + 268888.241, + 268921.585, + 268954.929, + 268988.273, + 269021.617, + 269054.961, + 269088.305, + 269121.64900000003, + 269154.993, + 269188.337, + 269221.68, + 269255.02400000003, + 269288.368, + 269321.712, + 269355.056, + 269388.4, + 269421.744, + 269455.088, + 269488.43200000003, + 269521.776, + 269555.12, + 269588.464, + 269621.808, + 269655.152, + 269688.495, + 269721.839, + 269755.183, + 269788.527, + 269821.871, + 269855.215, + 269888.559, + 269921.903, + 269955.24700000003, + 269988.591, + 270021.935, + 270055.279, + 270088.623, + 270121.966, + 270155.31, + 270188.654, + 270221.998, + 270255.342, + 270288.686, + 270322.03, + 270355.374, + 270388.718, + 270422.06200000003, + 270455.406, + 270488.75, + 270522.094, + 270555.43700000003, + 270588.781, + 270622.125, + 270655.469, + 270688.813, + 270722.157, + 270755.501, + 270788.84500000003, + 270822.189, + 270855.533, + 270888.877, + 270922.221, + 270955.565, + 270988.909, + 271022.252, + 271055.596, + 271088.94, + 271122.284, + 271155.628, + 271188.972, + 271222.316, + 271255.66000000003, + 271289.004, + 271322.348, + 271355.692, + 271389.036, + 271422.38, + 271455.723, + 271489.067, + 271522.411, + 271555.755, + 271589.099, + 271622.443, + 271655.787, + 271689.131, + 271722.475, + 271755.819, + 271789.163, + 271822.507, + 271855.851, + 271889.194, + 271922.538, + 271955.882, + 271989.226, + 272022.57, + 272055.914, + 272089.25800000003, + 272122.602, + 272155.946, + 272189.29, + 272222.634, + 272255.978, + 272289.322, + 272322.666, + 272356.009, + 272389.353, + 272422.697, + 272456.041, + 272489.385, + 272522.729, + 272556.07300000003, + 272589.417, + 272622.761, + 272656.105, + 272689.449, + 272722.793, + 272756.137, + 272789.48, + 272822.824, + 272856.168, + 272889.512, + 272922.856, + 272956.2, + 272989.544, + 273022.888, + 273056.232, + 273089.576, + 273122.92, + 273156.264, + 273189.608, + 273222.951, + 273256.295, + 273289.639, + 273322.983, + 273356.327, + 273389.67100000003, + 273423.015, + 273456.359, + 273489.703, + 273523.047, + 273556.391, + 273589.735, + 273623.079, + 273656.423, + 273689.766, + 273723.11, + 273756.454, + 273789.798, + 273823.142, + 273856.48600000003, + 273889.83, + 273923.174, + 273956.518, + 273989.862, + 274023.206, + 274056.55, + 274089.89400000003, + 274123.237, + 274156.581, + 274189.925, + 274223.26900000003, + 274256.613, + 274289.957, + 274323.301, + 274356.645, + 274389.989, + 274423.333, + 274456.677, + 274490.021, + 274523.365, + 274556.708, + 274590.052, + 274623.396, + 274656.74, + 274690.08400000003, + 274723.428, + 274756.772, + 274790.116, + 274823.46, + 274856.804, + 274890.148, + 274923.492, + 274956.836, + 274990.18, + 275023.523, + 275056.867, + 275090.211, + 275123.555, + 275156.89900000003, + 275190.243, + 275223.587, + 275256.931, + 275290.275, + 275323.619, + 275356.963, + 275390.30700000003, + 275423.651, + 275456.994, + 275490.338, + 275523.68200000003, + 275557.026, + 275590.37, + 275623.714, + 275657.058, + 275690.402, + 275723.746, + 275757.09, + 275790.434, + 275823.778, + 275857.12200000003, + 275890.465, + 275923.809, + 275957.153, + 275990.49700000003, + 276023.841, + 276057.185, + 276090.529, + 276123.873, + 276157.217, + 276190.561, + 276223.905, + 276257.249, + 276290.593, + 276323.93700000003, + 276357.28, + 276390.624, + 276423.968, + 276457.31200000003, + 276490.656, + 276524.0, + 276557.344, + 276590.688, + 276624.032, + 276657.376, + 276690.72000000003, + 276724.064 + ], + "depth_timestamps_ms": [ + 246980.978, + 247014.32200000001, + 247047.666, + 247081.01, + 247114.354, + 247147.698, + 247181.04200000002, + 247214.386, + 247247.73, + 247281.074, + 247314.418, + 247347.76200000002, + 247381.106, + 247414.449, + 247447.793, + 247481.13700000002, + 247514.481, + 247547.825, + 247581.169, + 247614.513, + 247647.85700000002, + 247681.201, + 247714.545, + 247747.889, + 247781.233, + 247814.57700000002, + 247847.921, + 247881.264, + 247914.608, + 247947.95200000002, + 247981.296, + 248014.64, + 248047.984, + 248081.328, + 248114.672, + 248148.016, + 248181.36000000002, + 248214.704, + 248248.048, + 248281.392, + 248314.73500000002, + 248348.079, + 248381.423, + 248414.767, + 248448.111, + 248481.45500000002, + 248514.799, + 248548.143, + 248581.487, + 248614.831, + 248648.17500000002, + 248681.519, + 248714.863, + 248748.206, + 248781.55000000002, + 248814.894, + 248848.238, + 248881.582, + 248914.926, + 248948.27000000002, + 248981.614, + 249014.958, + 249048.302, + 249081.646, + 249114.99, + 249148.334, + 249181.678, + 249215.021, + 249248.365, + 249281.709, + 249315.053, + 249348.397, + 249381.741, + 249415.085, + 249448.429, + 249481.77300000002, + 249515.117, + 249548.461, + 249581.805, + 249615.149, + 249648.492, + 249681.836, + 249715.18, + 249748.524, + 249781.86800000002, + 249815.212, + 249848.556, + 249881.9, + 249915.244, + 249948.58800000002, + 249981.932, + 250015.276, + 250048.62, + 250081.96300000002, + 250115.307, + 250148.651, + 250181.995, + 250215.339, + 250248.68300000002, + 250282.027, + 250315.371, + 250348.715, + 250382.059, + 250415.403, + 250448.747, + 250482.09100000001, + 250515.435, + 250548.778, + 250582.122, + 250615.46600000001, + 250648.81, + 250682.154, + 250715.498, + 250748.842, + 250782.18600000002, + 250815.53, + 250848.874, + 250882.218, + 250915.562, + 250948.90600000002, + 250982.249, + 251015.593, + 251048.937, + 251082.28100000002, + 251115.625, + 251148.969, + 251182.313, + 251215.657, + 251249.00100000002, + 251282.345, + 251315.689, + 251349.033, + 251382.377, + 251415.72, + 251449.064, + 251482.408, + 251515.752, + 251549.09600000002, + 251582.44, + 251615.784, + 251649.128, + 251682.472, + 251715.816, + 251749.16, + 251782.50400000002, + 251815.848, + 251849.192, + 251882.535, + 251915.87900000002, + 251949.223, + 251982.567, + 252015.911, + 252049.255, + 252082.59900000002, + 252115.943, + 252149.287, + 252182.631, + 252215.975, + 252249.31900000002, + 252282.663, + 252316.006, + 252349.35, + 252382.69400000002, + 252416.038, + 252449.382, + 252482.726, + 252516.07, + 252549.41400000002, + 252582.758, + 252616.102, + 252649.446, + 252682.79, + 252716.134, + 252749.477, + 252782.821, + 252816.165, + 252849.509, + 252882.853, + 252916.19700000001, + 252949.541, + 252982.885, + 253016.229, + 253049.573, + 253082.91700000002, + 253116.261, + 253149.605, + 253182.949, + 253216.29200000002, + 253249.636, + 253282.98, + 253316.324, + 253349.668, + 253383.01200000002, + 253416.356, + 253449.7, + 253483.044, + 253516.388, + 253549.73200000002, + 253583.076, + 253616.42, + 253649.763, + 253683.10700000002, + 253716.451, + 253749.795, + 253783.139, + 253816.483, + 253849.82700000002, + 253883.171, + 253916.515, + 253949.859, + 253983.203, + 254016.547, + 254049.891, + 254083.234, + 254116.578, + 254149.922, + 254183.266, + 254216.61000000002, + 254249.954, + 254283.298, + 254316.642, + 254349.986, + 254383.33000000002, + 254416.674, + 254450.018, + 254483.362, + 254516.706, + 254550.049, + 254583.393, + 254616.737, + 254650.081, + 254683.42500000002, + 254716.769, + 254750.113, + 254783.457, + 254816.801, + 254850.14500000002, + 254883.489, + 254916.833, + 254950.177, + 254983.52000000002, + 255016.864, + 255050.208, + 255083.552, + 255116.896, + 255150.24000000002, + 255183.584, + 255216.928, + 255250.272, + 255283.616, + 255316.96, + 255350.304, + 255383.64800000002, + 255416.991, + 255450.335, + 255483.679, + 255517.02300000002, + 255550.367, + 255583.711, + 255617.055, + 255650.399, + 255683.74300000002, + 255717.087, + 255750.431, + 255783.775, + 255817.119, + 255850.46300000002, + 255883.806, + 255917.15, + 255950.494, + 255983.83800000002, + 256017.182, + 256050.526, + 256083.87, + 256117.214, + 256150.55800000002, + 256183.902, + 256217.246, + 256250.59, + 256283.934, + 256317.277, + 256350.621, + 256383.965, + 256417.309, + 256450.653, + 256483.997, + 256517.34100000001, + 256550.685, + 256584.029, + 256617.373, + 256650.717, + 256684.06100000002, + 256717.405, + 256750.748, + 256784.092, + 256817.43600000002, + 256850.78, + 256884.124, + 256917.468, + 256950.812, + 256984.15600000002, + 257017.5, + 257050.844, + 257084.188, + 257117.532, + 257150.87600000002, + 257184.22, + 257217.563, + 257250.907, + 257284.25100000002, + 257317.595, + 257350.939, + 257384.283, + 257417.627, + 257450.97100000002, + 257484.315, + 257517.659, + 257551.003, + 257584.347, + 257617.691, + 257651.034, + 257684.378, + 257717.722, + 257751.066, + 257784.41, + 257817.75400000002, + 257851.098, + 257884.442, + 257917.786, + 257951.13, + 257984.47400000002, + 258017.818, + 258051.162, + 258084.505, + 258117.84900000002, + 258151.193, + 258184.537, + 258217.881, + 258251.225, + 258284.56900000002, + 258317.913, + 258351.257, + 258384.601, + 258417.945, + 258451.28900000002, + 258484.633, + 258517.977, + 258551.32, + 258584.66400000002, + 258618.008, + 258651.352, + 258684.696, + 258718.04, + 258751.384, + 258784.728, + 258818.07200000001, + 258851.416, + 258884.76, + 258918.104, + 258951.448, + 258984.791, + 259018.135, + 259051.479, + 259084.823, + 259118.16700000002, + 259151.511, + 259184.855, + 259218.199, + 259251.543, + 259284.88700000002, + 259318.231, + 259351.575, + 259384.919, + 259418.26200000002, + 259451.606, + 259484.95, + 259518.294, + 259551.638, + 259584.98200000002, + 259618.326, + 259651.67, + 259685.014, + 259718.358, + 259751.70200000002, + 259785.046, + 259818.39, + 259851.734, + 259885.07700000002, + 259918.421, + 259951.765, + 259985.109, + 260018.453, + 260051.797, + 260085.141, + 260118.48500000002, + 260151.829, + 260185.173, + 260218.517, + 260251.861, + 260285.20500000002, + 260318.548, + 260351.892, + 260385.236, + 260418.58000000002, + 260451.924, + 260485.268, + 260518.612, + 260551.956, + 260585.30000000002, + 260618.644, + 260651.988, + 260685.332, + 260718.676, + 260752.019, + 260785.363, + 260818.707, + 260852.051, + 260885.39500000002, + 260918.739, + 260952.083, + 260985.427, + 261018.771, + 261052.11500000002, + 261085.459, + 261118.803, + 261152.147, + 261185.491, + 261218.834, + 261252.178, + 261285.522, + 261318.866, + 261352.21, + 261385.554, + 261418.89800000002, + 261452.242, + 261485.586, + 261518.93, + 261552.274, + 261585.61800000002, + 261618.962, + 261652.305, + 261685.649, + 261718.99300000002, + 261752.337, + 261785.681, + 261819.025, + 261852.369, + 261885.71300000002, + 261919.057, + 261952.401, + 261985.745, + 262019.089, + 262052.43300000002, + 262085.776, + 262119.12, + 262152.464, + 262185.808, + 262219.152, + 262252.496, + 262285.84, + 262319.184, + 262352.528, + 262385.87200000003, + 262419.216, + 262452.56, + 262485.904, + 262519.248, + 262552.591, + 262585.935, + 262619.279, + 262652.623, + 262685.967, + 262719.311, + 262752.655, + 262785.999, + 262819.343, + 262852.68700000003, + 262886.031, + 262919.375, + 262952.719, + 262986.06200000003, + 263019.406, + 263052.75, + 263086.094, + 263119.438, + 263152.782, + 263186.126, + 263219.47000000003, + 263252.814, + 263286.158, + 263319.502, + 263352.846, + 263386.19, + 263419.533, + 263452.877, + 263486.221, + 263519.565, + 263552.909, + 263586.253, + 263619.597, + 263652.941, + 263686.28500000003, + 263719.629, + 263752.973, + 263786.317, + 263819.661, + 263853.005, + 263886.348, + 263919.692, + 263953.036, + 263986.38, + 264019.724, + 264053.068, + 264086.412, + 264119.756, + 264153.1, + 264186.444, + 264219.788, + 264253.132, + 264286.476, + 264319.819, + 264353.163, + 264386.507, + 264419.851, + 264453.195, + 264486.539, + 264519.88300000003, + 264553.227, + 264586.571, + 264619.915, + 264653.259, + 264686.603, + 264719.947, + 264753.29, + 264786.634, + 264819.978, + 264853.322, + 264886.666, + 264920.01, + 264953.354, + 264986.69800000003, + 265020.042, + 265053.386, + 265086.73, + 265120.074, + 265153.418, + 265186.762, + 265220.105, + 265253.449, + 265286.793, + 265320.137, + 265353.481, + 265386.825, + 265420.169, + 265453.513, + 265486.857, + 265520.201, + 265553.545, + 265586.889, + 265620.233, + 265653.576, + 265686.92, + 265720.264, + 265753.608, + 265786.952, + 265820.29600000003, + 265853.64, + 265886.984, + 265920.328, + 265953.672, + 265987.016, + 266020.36, + 266053.704, + 266087.047, + 266120.391, + 266153.735, + 266187.079, + 266220.423, + 266253.767, + 266287.11100000003, + 266320.455, + 266353.799, + 266387.143, + 266420.487, + 266453.831, + 266487.175, + 266520.51900000003, + 266553.862, + 266587.206, + 266620.55, + 266653.89400000003, + 266687.238, + 266720.582, + 266753.926, + 266787.27, + 266820.614, + 266853.958, + 266887.302, + 266920.646, + 266953.99, + 266987.333, + 267020.677, + 267054.021, + 267087.365, + 267120.70900000003, + 267154.053, + 267187.397, + 267220.741, + 267254.085, + 267287.429, + 267320.773, + 267354.117, + 267387.461, + 267420.804, + 267454.148, + 267487.492, + 267520.836, + 267554.18, + 267587.52400000003, + 267620.868, + 267654.212, + 267687.556, + 267720.9, + 267754.244, + 267787.588, + 267820.93200000003, + 267854.276, + 267887.619, + 267920.963, + 267954.30700000003, + 267987.651, + 268020.995, + 268054.339, + 268087.683, + 268121.027, + 268154.371, + 268187.715, + 268221.059, + 268254.403, + 268287.74700000003, + 268321.09, + 268354.434, + 268387.778, + 268421.12200000003, + 268454.466, + 268487.81, + 268521.154, + 268554.498, + 268587.842, + 268621.186, + 268654.53, + 268687.874, + 268721.218, + 268754.561, + 268787.905, + 268821.249, + 268854.593, + 268887.93700000003, + 268921.281, + 268954.625, + 268987.969, + 269021.313, + 269054.657, + 269088.001, + 269121.34500000003, + 269154.689, + 269188.033, + 269221.376, + 269254.72000000003, + 269288.064, + 269321.408, + 269354.752, + 269388.096, + 269421.44, + 269454.784, + 269488.128, + 269521.472, + 269554.816, + 269588.16000000003, + 269621.504, + 269654.847, + 269688.191, + 269721.53500000003, + 269754.879, + 269788.223, + 269821.567, + 269854.911, + 269888.255, + 269921.599, + 269954.943, + 269988.287, + 270021.631, + 270054.975, + 270088.318, + 270121.662, + 270155.006, + 270188.35, + 270221.694, + 270255.038, + 270288.382, + 270321.726, + 270355.07, + 270388.414, + 270421.75800000003, + 270455.102, + 270488.446, + 270521.79, + 270555.13300000003, + 270588.477, + 270621.821, + 270655.165, + 270688.509, + 270721.853, + 270755.197, + 270788.541, + 270821.885, + 270855.229, + 270888.57300000003, + 270921.917, + 270955.261, + 270988.604, + 271021.94800000003, + 271055.292, + 271088.636, + 271121.98, + 271155.324, + 271188.668, + 271222.012, + 271255.356, + 271288.7, + 271322.044, + 271355.388, + 271388.732, + 271422.075, + 271455.419, + 271488.763, + 271522.107, + 271555.451, + 271588.795, + 271622.139, + 271655.483, + 271688.827, + 271722.17100000003, + 271755.515, + 271788.859, + 271822.203, + 271855.547, + 271888.89, + 271922.234, + 271955.578, + 271988.922, + 272022.266, + 272055.61, + 272088.954, + 272122.298, + 272155.642, + 272188.98600000003, + 272222.33, + 272255.674, + 272289.018, + 272322.36100000003, + 272355.705, + 272389.049, + 272422.393, + 272455.737, + 272489.081, + 272522.425, + 272555.76900000003, + 272589.113, + 272622.457, + 272655.801, + 272689.145, + 272722.489, + 272755.832, + 272789.176, + 272822.52, + 272855.864, + 272889.208, + 272922.552, + 272955.896, + 272989.24, + 273022.58400000003, + 273055.928, + 273089.272, + 273122.616, + 273155.96, + 273189.304, + 273222.647, + 273255.991, + 273289.335, + 273322.679, + 273356.023, + 273389.367, + 273422.711, + 273456.055, + 273489.39900000003, + 273522.743, + 273556.087, + 273589.431, + 273622.775, + 273656.118, + 273689.462, + 273722.806, + 273756.15, + 273789.494, + 273822.838, + 273856.18200000003, + 273889.526, + 273922.87, + 273956.214, + 273989.558, + 274022.902, + 274056.246, + 274089.589, + 274122.933, + 274156.277, + 274189.621, + 274222.965, + 274256.309, + 274289.653, + 274322.99700000003, + 274356.341, + 274389.685, + 274423.029, + 274456.373, + 274489.717, + 274523.061, + 274556.404, + 274589.748, + 274623.092, + 274656.436, + 274689.78, + 274723.124, + 274756.468, + 274789.81200000003, + 274823.156, + 274856.5, + 274889.844, + 274923.188, + 274956.532, + 274989.875, + 275023.219, + 275056.563, + 275089.907, + 275123.251, + 275156.59500000003, + 275189.939, + 275223.283, + 275256.627, + 275289.971, + 275323.315, + 275356.659, + 275390.003, + 275423.346, + 275456.69, + 275490.034, + 275523.378, + 275556.722, + 275590.066, + 275623.41000000003, + 275656.754, + 275690.098, + 275723.442, + 275756.786, + 275790.13, + 275823.474, + 275856.818, + 275890.161, + 275923.505, + 275956.849, + 275990.193, + 276023.537, + 276056.881, + 276090.225, + 276123.569, + 276156.913, + 276190.257, + 276223.601, + 276256.945, + 276290.289, + 276323.632, + 276356.976, + 276390.32, + 276423.664, + 276457.00800000003, + 276490.352, + 276523.696, + 276557.04, + 276590.384, + 276623.728, + 276657.072, + 276690.416, + 276723.76 + ], + "color_frame_numbers": [ + 3778, + 3779, + 3780, + 3781, + 3782, + 3783, + 3784, + 3785, + 3786, + 3787, + 3788, + 3789, + 3790, + 3791, + 3792, + 3793, + 3794, + 3795, + 3796, + 3797, + 3798, + 3799, + 3800, + 3801, + 3802, + 3803, + 3804, + 3805, + 3806, + 3807, + 3808, + 3809, + 3810, + 3811, + 3812, + 3813, + 3814, + 3815, + 3816, + 3817, + 3818, + 3819, + 3820, + 3821, + 3822, + 3823, + 3824, + 3825, + 3826, + 3827, + 3828, + 3829, + 3830, + 3831, + 3832, + 3833, + 3834, + 3835, + 3836, + 3837, + 3838, + 3839, + 3840, + 3841, + 3842, + 3843, + 3844, + 3845, + 3846, + 3847, + 3848, + 3849, + 3850, + 3851, + 3852, + 3853, + 3854, + 3855, + 3856, + 3857, + 3858, + 3859, + 3860, + 3861, + 3862, + 3863, + 3864, + 3865, + 3866, + 3867, + 3868, + 3869, + 3870, + 3871, + 3872, + 3873, + 3874, + 3875, + 3876, + 3877, + 3878, + 3879, + 3880, + 3881, + 3882, + 3883, + 3884, + 3885, + 3886, + 3887, + 3888, + 3889, + 3890, + 3891, + 3892, + 3893, + 3894, + 3895, + 3896, + 3897, + 3898, + 3899, + 3900, + 3901, + 3902, + 3903, + 3904, + 3905, + 3906, + 3907, + 3908, + 3909, + 3910, + 3911, + 3912, + 3913, + 3914, + 3915, + 3916, + 3917, + 3918, + 3919, + 3920, + 3921, + 3922, + 3923, + 3924, + 3925, + 3926, + 3927, + 3928, + 3929, + 3930, + 3931, + 3932, + 3933, + 3934, + 3935, + 3936, + 3937, + 3938, + 3939, + 3940, + 3941, + 3942, + 3943, + 3944, + 3945, + 3946, + 3947, + 3948, + 3949, + 3950, + 3951, + 3952, + 3953, + 3954, + 3955, + 3956, + 3957, + 3958, + 3959, + 3960, + 3961, + 3962, + 3963, + 3964, + 3965, + 3966, + 3967, + 3968, + 3969, + 3970, + 3971, + 3972, + 3973, + 3974, + 3975, + 3976, + 3977, + 3978, + 3979, + 3980, + 3981, + 3982, + 3983, + 3984, + 3985, + 3986, + 3987, + 3988, + 3989, + 3990, + 3991, + 3992, + 3993, + 3994, + 3995, + 3996, + 3997, + 3998, + 3999, + 4000, + 4001, + 4002, + 4003, + 4004, + 4005, + 4006, + 4007, + 4008, + 4009, + 4010, + 4011, + 4012, + 4013, + 4014, + 4015, + 4016, + 4017, + 4018, + 4019, + 4020, + 4021, + 4022, + 4023, + 4024, + 4025, + 4026, + 4027, + 4028, + 4029, + 4030, + 4031, + 4032, + 4033, + 4034, + 4035, + 4036, + 4037, + 4038, + 4039, + 4040, + 4041, + 4042, + 4043, + 4044, + 4045, + 4046, + 4047, + 4048, + 4049, + 4050, + 4051, + 4052, + 4053, + 4054, + 4055, + 4056, + 4057, + 4058, + 4059, + 4060, + 4061, + 4062, + 4063, + 4064, + 4065, + 4066, + 4067, + 4068, + 4069, + 4070, + 4071, + 4072, + 4073, + 4074, + 4075, + 4076, + 4077, + 4078, + 4079, + 4080, + 4081, + 4082, + 4083, + 4084, + 4085, + 4086, + 4087, + 4088, + 4089, + 4090, + 4091, + 4092, + 4093, + 4094, + 4095, + 4096, + 4097, + 4098, + 4099, + 4100, + 4101, + 4102, + 4103, + 4104, + 4105, + 4106, + 4107, + 4108, + 4109, + 4110, + 4111, + 4112, + 4113, + 4114, + 4115, + 4116, + 4117, + 4118, + 4119, + 4120, + 4121, + 4122, + 4123, + 4124, + 4125, + 4126, + 4127, + 4128, + 4129, + 4130, + 4131, + 4132, + 4133, + 4134, + 4135, + 4136, + 4137, + 4138, + 4139, + 4140, + 4141, + 4142, + 4143, + 4144, + 4145, + 4146, + 4147, + 4148, + 4149, + 4150, + 4151, + 4152, + 4153, + 4154, + 4155, + 4156, + 4157, + 4158, + 4159, + 4160, + 4161, + 4162, + 4163, + 4164, + 4165, + 4166, + 4167, + 4168, + 4169, + 4170, + 4171, + 4172, + 4173, + 4174, + 4175, + 4176, + 4177, + 4178, + 4179, + 4180, + 4181, + 4182, + 4183, + 4184, + 4185, + 4186, + 4187, + 4188, + 4189, + 4190, + 4191, + 4192, + 4193, + 4194, + 4195, + 4196, + 4197, + 4198, + 4199, + 4200, + 4201, + 4202, + 4203, + 4204, + 4205, + 4206, + 4207, + 4208, + 4209, + 4210, + 4211, + 4212, + 4213, + 4214, + 4215, + 4216, + 4217, + 4218, + 4219, + 4220, + 4221, + 4222, + 4223, + 4224, + 4225, + 4226, + 4227, + 4228, + 4229, + 4230, + 4231, + 4232, + 4233, + 4234, + 4235, + 4236, + 4237, + 4238, + 4239, + 4240, + 4241, + 4242, + 4243, + 4244, + 4245, + 4246, + 4247, + 4248, + 4249, + 4250, + 4251, + 4252, + 4253, + 4254, + 4255, + 4256, + 4257, + 4258, + 4259, + 4260, + 4261, + 4262, + 4263, + 4264, + 4265, + 4266, + 4267, + 4268, + 4269, + 4270, + 4271, + 4272, + 4273, + 4274, + 4275, + 4276, + 4277, + 4278, + 4279, + 4280, + 4281, + 4282, + 4283, + 4284, + 4285, + 4286, + 4287, + 4288, + 4289, + 4290, + 4291, + 4292, + 4293, + 4294, + 4295, + 4296, + 4297, + 4298, + 4299, + 4300, + 4301, + 4302, + 4303, + 4304, + 4305, + 4306, + 4307, + 4308, + 4309, + 4310, + 4311, + 4312, + 4313, + 4314, + 4315, + 4316, + 4317, + 4318, + 4319, + 4320, + 4321, + 4322, + 4323, + 4324, + 4325, + 4326, + 4327, + 4328, + 4329, + 4330, + 4331, + 4332, + 4333, + 4334, + 4335, + 4336, + 4337, + 4338, + 4339, + 4340, + 4341, + 4342, + 4343, + 4344, + 4345, + 4346, + 4347, + 4348, + 4349, + 4350, + 4351, + 4352, + 4353, + 4354, + 4355, + 4356, + 4357, + 4358, + 4359, + 4360, + 4361, + 4362, + 4363, + 4364, + 4365, + 4366, + 4367, + 4368, + 4369, + 4370, + 4371, + 4372, + 4373, + 4374, + 4375, + 4376, + 4377, + 4378, + 4379, + 4380, + 4381, + 4382, + 4383, + 4384, + 4385, + 4386, + 4387, + 4388, + 4389, + 4390, + 4391, + 4392, + 4393, + 4394, + 4395, + 4396, + 4397, + 4398, + 4399, + 4400, + 4401, + 4402, + 4403, + 4404, + 4405, + 4406, + 4407, + 4408, + 4409, + 4410, + 4411, + 4412, + 4413, + 4414, + 4415, + 4416, + 4417, + 4418, + 4419, + 4420, + 4421, + 4422, + 4423, + 4424, + 4425, + 4426, + 4427, + 4428, + 4429, + 4430, + 4431, + 4432, + 4433, + 4434, + 4435, + 4436, + 4437, + 4438, + 4439, + 4440, + 4441, + 4442, + 4443, + 4444, + 4445, + 4446, + 4447, + 4448, + 4449, + 4450, + 4451, + 4452, + 4453, + 4454, + 4455, + 4456, + 4457, + 4458, + 4459, + 4460, + 4461, + 4462, + 4463, + 4464, + 4465, + 4466, + 4467, + 4468, + 4469, + 4470, + 4471, + 4472, + 4473, + 4474, + 4475, + 4476, + 4477, + 4478, + 4479, + 4480, + 4481, + 4482, + 4483, + 4484, + 4485, + 4486, + 4487, + 4488, + 4489, + 4490, + 4491, + 4492, + 4493, + 4494, + 4495, + 4496, + 4497, + 4498, + 4499, + 4500, + 4501, + 4502, + 4503, + 4504, + 4505, + 4506, + 4507, + 4508, + 4509, + 4510, + 4511, + 4512, + 4513, + 4514, + 4515, + 4516, + 4517, + 4518, + 4519, + 4520, + 4521, + 4522, + 4523, + 4524, + 4525, + 4526, + 4527, + 4528, + 4529, + 4530, + 4531, + 4532, + 4533, + 4534, + 4535, + 4536, + 4537, + 4538, + 4539, + 4540, + 4541, + 4542, + 4543, + 4544, + 4545, + 4546, + 4547, + 4548, + 4549, + 4550, + 4551, + 4552, + 4553, + 4554, + 4555, + 4556, + 4557, + 4558, + 4559, + 4560, + 4561, + 4562, + 4563, + 4564, + 4565, + 4566, + 4567, + 4568, + 4569, + 4570, + 4571, + 4572, + 4573, + 4574, + 4575, + 4576, + 4577, + 4578, + 4579, + 4580, + 4581, + 4582, + 4583, + 4584, + 4585, + 4586, + 4587, + 4588, + 4589, + 4590, + 4591, + 4592, + 4593, + 4594, + 4595, + 4596, + 4597, + 4598, + 4599, + 4600, + 4601, + 4602, + 4603, + 4604, + 4605, + 4606, + 4607, + 4608, + 4609, + 4610, + 4611, + 4612, + 4613, + 4614, + 4615, + 4616, + 4617, + 4618, + 4619, + 4620, + 4621, + 4622, + 4623, + 4624, + 4625, + 4626, + 4627, + 4628, + 4629, + 4630, + 4631, + 4632, + 4633, + 4634, + 4635, + 4636, + 4637, + 4638, + 4639, + 4640, + 4641, + 4642, + 4643, + 4644, + 4645, + 4646, + 4647, + 4648, + 4649, + 4650, + 4651, + 4652, + 4653, + 4654, + 4655, + 4656, + 4657, + 4658, + 4659, + 4660, + 4661, + 4662, + 4663, + 4664, + 4665, + 4666, + 4667, + 4668, + 4669, + 4670 + ], + "depth_frame_numbers": [ + 3792, + 3793, + 3794, + 3795, + 3796, + 3797, + 3798, + 3799, + 3800, + 3801, + 3802, + 3803, + 3804, + 3805, + 3806, + 3807, + 3808, + 3809, + 3810, + 3811, + 3812, + 3813, + 3814, + 3815, + 3816, + 3817, + 3818, + 3819, + 3820, + 3821, + 3822, + 3823, + 3824, + 3825, + 3826, + 3827, + 3828, + 3829, + 3830, + 3831, + 3832, + 3833, + 3834, + 3835, + 3836, + 3837, + 3838, + 3839, + 3840, + 3841, + 3842, + 3843, + 3844, + 3845, + 3846, + 3847, + 3848, + 3849, + 3850, + 3851, + 3852, + 3853, + 3854, + 3855, + 3856, + 3857, + 3858, + 3859, + 3860, + 3861, + 3862, + 3863, + 3864, + 3865, + 3866, + 3867, + 3868, + 3869, + 3870, + 3871, + 3872, + 3873, + 3874, + 3875, + 3876, + 3877, + 3878, + 3879, + 3880, + 3881, + 3882, + 3883, + 3884, + 3885, + 3886, + 3887, + 3888, + 3889, + 3890, + 3891, + 3892, + 3893, + 3894, + 3895, + 3896, + 3897, + 3898, + 3899, + 3900, + 3901, + 3902, + 3903, + 3904, + 3905, + 3906, + 3907, + 3908, + 3909, + 3910, + 3911, + 3912, + 3913, + 3914, + 3915, + 3916, + 3917, + 3918, + 3919, + 3920, + 3921, + 3922, + 3923, + 3924, + 3925, + 3926, + 3927, + 3928, + 3929, + 3930, + 3931, + 3932, + 3933, + 3934, + 3935, + 3936, + 3937, + 3938, + 3939, + 3940, + 3941, + 3942, + 3943, + 3944, + 3945, + 3946, + 3947, + 3948, + 3949, + 3950, + 3951, + 3952, + 3953, + 3954, + 3955, + 3956, + 3957, + 3958, + 3959, + 3960, + 3961, + 3962, + 3963, + 3964, + 3965, + 3966, + 3967, + 3968, + 3969, + 3970, + 3971, + 3972, + 3973, + 3974, + 3975, + 3976, + 3977, + 3978, + 3979, + 3980, + 3981, + 3982, + 3983, + 3984, + 3985, + 3986, + 3987, + 3988, + 3989, + 3990, + 3991, + 3992, + 3993, + 3994, + 3995, + 3996, + 3997, + 3998, + 3999, + 4000, + 4001, + 4002, + 4003, + 4004, + 4005, + 4006, + 4007, + 4008, + 4009, + 4010, + 4011, + 4012, + 4013, + 4014, + 4015, + 4016, + 4017, + 4018, + 4019, + 4020, + 4021, + 4022, + 4023, + 4024, + 4025, + 4026, + 4027, + 4028, + 4029, + 4030, + 4031, + 4032, + 4033, + 4034, + 4035, + 4036, + 4037, + 4038, + 4039, + 4040, + 4041, + 4042, + 4043, + 4044, + 4045, + 4046, + 4047, + 4048, + 4049, + 4050, + 4051, + 4052, + 4053, + 4054, + 4055, + 4056, + 4057, + 4058, + 4059, + 4060, + 4061, + 4062, + 4063, + 4064, + 4065, + 4066, + 4067, + 4068, + 4069, + 4070, + 4071, + 4072, + 4073, + 4074, + 4075, + 4076, + 4077, + 4078, + 4079, + 4080, + 4081, + 4082, + 4083, + 4084, + 4085, + 4086, + 4087, + 4088, + 4089, + 4090, + 4091, + 4092, + 4093, + 4094, + 4095, + 4096, + 4097, + 4098, + 4099, + 4100, + 4101, + 4102, + 4103, + 4104, + 4105, + 4106, + 4107, + 4108, + 4109, + 4110, + 4111, + 4112, + 4113, + 4114, + 4115, + 4116, + 4117, + 4118, + 4119, + 4120, + 4121, + 4122, + 4123, + 4124, + 4125, + 4126, + 4127, + 4128, + 4129, + 4130, + 4131, + 4132, + 4133, + 4134, + 4135, + 4136, + 4137, + 4138, + 4139, + 4140, + 4141, + 4142, + 4143, + 4144, + 4145, + 4146, + 4147, + 4148, + 4149, + 4150, + 4151, + 4152, + 4153, + 4154, + 4155, + 4156, + 4157, + 4158, + 4159, + 4160, + 4161, + 4162, + 4163, + 4164, + 4165, + 4166, + 4167, + 4168, + 4169, + 4170, + 4171, + 4172, + 4173, + 4174, + 4175, + 4176, + 4177, + 4178, + 4179, + 4180, + 4181, + 4182, + 4183, + 4184, + 4185, + 4186, + 4187, + 4188, + 4189, + 4190, + 4191, + 4192, + 4193, + 4194, + 4195, + 4196, + 4197, + 4198, + 4199, + 4200, + 4201, + 4202, + 4203, + 4204, + 4205, + 4206, + 4207, + 4208, + 4209, + 4210, + 4211, + 4212, + 4213, + 4214, + 4215, + 4216, + 4217, + 4218, + 4219, + 4220, + 4221, + 4222, + 4223, + 4224, + 4225, + 4226, + 4227, + 4228, + 4229, + 4230, + 4231, + 4232, + 4233, + 4234, + 4235, + 4236, + 4237, + 4238, + 4239, + 4240, + 4241, + 4242, + 4243, + 4244, + 4245, + 4246, + 4247, + 4248, + 4249, + 4250, + 4251, + 4252, + 4253, + 4254, + 4255, + 4256, + 4257, + 4258, + 4259, + 4260, + 4261, + 4262, + 4263, + 4264, + 4265, + 4266, + 4267, + 4268, + 4269, + 4270, + 4271, + 4272, + 4273, + 4274, + 4275, + 4276, + 4277, + 4278, + 4279, + 4280, + 4281, + 4282, + 4283, + 4284, + 4285, + 4286, + 4287, + 4288, + 4289, + 4290, + 4291, + 4292, + 4293, + 4294, + 4295, + 4296, + 4297, + 4298, + 4299, + 4300, + 4301, + 4302, + 4303, + 4304, + 4305, + 4306, + 4307, + 4308, + 4309, + 4310, + 4311, + 4312, + 4313, + 4314, + 4315, + 4316, + 4317, + 4318, + 4319, + 4320, + 4321, + 4322, + 4323, + 4324, + 4325, + 4326, + 4327, + 4328, + 4329, + 4330, + 4331, + 4332, + 4333, + 4334, + 4335, + 4336, + 4337, + 4338, + 4339, + 4340, + 4341, + 4342, + 4343, + 4344, + 4345, + 4346, + 4347, + 4348, + 4349, + 4350, + 4351, + 4352, + 4353, + 4354, + 4355, + 4356, + 4357, + 4358, + 4359, + 4360, + 4361, + 4362, + 4363, + 4364, + 4365, + 4366, + 4367, + 4368, + 4369, + 4370, + 4371, + 4372, + 4373, + 4374, + 4375, + 4376, + 4377, + 4378, + 4379, + 4380, + 4381, + 4382, + 4383, + 4384, + 4385, + 4386, + 4387, + 4388, + 4389, + 4390, + 4391, + 4392, + 4393, + 4394, + 4395, + 4396, + 4397, + 4398, + 4399, + 4400, + 4401, + 4402, + 4403, + 4404, + 4405, + 4406, + 4407, + 4408, + 4409, + 4410, + 4411, + 4412, + 4413, + 4414, + 4415, + 4416, + 4417, + 4418, + 4419, + 4420, + 4421, + 4422, + 4423, + 4424, + 4425, + 4426, + 4427, + 4428, + 4429, + 4430, + 4431, + 4432, + 4433, + 4434, + 4435, + 4436, + 4437, + 4438, + 4439, + 4440, + 4441, + 4442, + 4443, + 4444, + 4445, + 4446, + 4447, + 4448, + 4449, + 4450, + 4451, + 4452, + 4453, + 4454, + 4455, + 4456, + 4457, + 4458, + 4459, + 4460, + 4461, + 4462, + 4463, + 4464, + 4465, + 4466, + 4467, + 4468, + 4469, + 4470, + 4471, + 4472, + 4473, + 4474, + 4475, + 4476, + 4477, + 4478, + 4479, + 4480, + 4481, + 4482, + 4483, + 4484, + 4485, + 4486, + 4487, + 4488, + 4489, + 4490, + 4491, + 4492, + 4493, + 4494, + 4495, + 4496, + 4497, + 4498, + 4499, + 4500, + 4501, + 4502, + 4503, + 4504, + 4505, + 4506, + 4507, + 4508, + 4509, + 4510, + 4511, + 4512, + 4513, + 4514, + 4515, + 4516, + 4517, + 4518, + 4519, + 4520, + 4521, + 4522, + 4523, + 4524, + 4525, + 4526, + 4527, + 4528, + 4529, + 4530, + 4531, + 4532, + 4533, + 4534, + 4535, + 4536, + 4537, + 4538, + 4539, + 4540, + 4541, + 4542, + 4543, + 4544, + 4545, + 4546, + 4547, + 4548, + 4549, + 4550, + 4551, + 4552, + 4553, + 4554, + 4555, + 4556, + 4557, + 4558, + 4559, + 4560, + 4561, + 4562, + 4563, + 4564, + 4565, + 4566, + 4567, + 4568, + 4569, + 4570, + 4571, + 4572, + 4573, + 4574, + 4575, + 4576, + 4577, + 4578, + 4579, + 4580, + 4581, + 4582, + 4583, + 4584, + 4585, + 4586, + 4587, + 4588, + 4589, + 4590, + 4591, + 4592, + 4593, + 4594, + 4595, + 4596, + 4597, + 4598, + 4599, + 4600, + 4601, + 4602, + 4603, + 4604, + 4605, + 4606, + 4607, + 4608, + 4609, + 4610, + 4611, + 4612, + 4613, + 4614, + 4615, + 4616, + 4617, + 4618, + 4619, + 4620, + 4621, + 4622, + 4623, + 4624, + 4625, + 4626, + 4627, + 4628, + 4629, + 4630, + 4631, + 4632, + 4633, + 4634, + 4635, + 4636, + 4637, + 4638, + 4639, + 4640, + 4641, + 4642, + 4643, + 4644, + 4645, + 4646, + 4647, + 4648, + 4649, + 4650, + 4651, + 4652, + 4653, + 4654, + 4655, + 4656, + 4657, + 4658, + 4659, + 4660, + 4661, + 4662, + 4663, + 4664, + 4665, + 4666, + 4667, + 4668, + 4669, + 4670, + 4671, + 4672, + 4673, + 4674, + 4675, + 4676, + 4677, + 4678, + 4679, + 4680, + 4681, + 4682, + 4683, + 4684 + ], + "host_arrival_monotonic_ns": [ + 7659383515910, + 7659418016692, + 7659450569759, + 7659484596049, + 7659518090244, + 7659551997947, + 7659584999998, + 7659619076337, + 7659651114837, + 7659685365681, + 7659718910456, + 7659750594619, + 7659783390607, + 7659815837981, + 7659849281967, + 7659883629443, + 7659915570343, + 7659949419693, + 7659983537048, + 7660018299659, + 7660050190262, + 7660082490297, + 7660116588924, + 7660149545157, + 7660183961919, + 7660216941125, + 7660251462855, + 7660283824027, + 7660316609057, + 7660350382235, + 7660382934868, + 7660415929939, + 7660448545813, + 7660482118203, + 7660516276773, + 7660548286270, + 7660581239935, + 7660619578721, + 7660651441634, + 7660683167477, + 7660716310546, + 7660749380411, + 7660784976881, + 7660817156469, + 7660849620695, + 7660882828902, + 7660915348878, + 7660951584384, + 7660984667341, + 7661016976780, + 7661049949158, + 7661084990914, + 7661117480194, + 7661149657187, + 7661182445147, + 7661216430568, + 7661252620668, + 7661284490393, + 7661334856660, + 7661355646574, + 7661386523660, + 7661422992258, + 7661450934432, + 7661483662162, + 7661516221483, + 7661551057947, + 7661583480067, + 7661615867857, + 7661649204445, + 7661685509456, + 7661717796925, + 7661749604199, + 7661782761241, + 7661816478289, + 7661851958865, + 7661884193835, + 7661917575482, + 7661950611897, + 7661985333012, + 7662018757022, + 7662051650660, + 7662083972978, + 7662116124694, + 7662150757191, + 7662184279167, + 7662217485830, + 7662250986489, + 7662283111670, + 7662317078868, + 7662350339263, + 7662383669017, + 7662418176006, + 7662449559100, + 7662485480018, + 7662517266109, + 7662550177919, + 7662585344507, + 7662617733634, + 7662649711355, + 7662682268760, + 7662716732791, + 7662750185225, + 7662783482182, + 7662816548677, + 7662849665825, + 7662884718619, + 7662917216882, + 7662949802306, + 7662983238446, + 7663016994394, + 7663050937773, + 7663084532914, + 7663116420054, + 7663149226292, + 7663184645187, + 7663218221524, + 7663251187590, + 7663282650087, + 7663316034190, + 7663349357028, + 7663382569242, + 7663416230083, + 7663449227629, + 7663483651422, + 7663518775127, + 7663551340586, + 7663583500418, + 7663616373808, + 7663650419691, + 7663684487382, + 7663719813316, + 7663751114696, + 7663783682343, + 7663816929334, + 7663849486197, + 7663883315052, + 7663918277525, + 7663951959453, + 7663984786670, + 7664016855934, + 7664049121745, + 7664084506221, + 7664120557616, + 7664151813105, + 7664184054378, + 7664216861794, + 7664252895577, + 7664284855333, + 7664316814500, + 7664350620921, + 7664383779514, + 7664418187516, + 7664450911857, + 7664483523448, + 7664517012042, + 7664550583013, + 7664583212462, + 7664617935722, + 7664651358899, + 7664684244018, + 7664721682380, + 7664752241747, + 7664783977767, + 7664817052043, + 7664849760463, + 7664883563089, + 7664916532211, + 7664951164930, + 7664985971289, + 7665020641685, + 7665053234181, + 7665084584955, + 7665117571494, + 7665151148639, + 7665185446779, + 7665219561538, + 7665251710862, + 7665283943873, + 7665318038521, + 7665351765318, + 7665384077224, + 7665418308594, + 7665450906488, + 7665484203488, + 7665516353362, + 7665553185429, + 7665585365174, + 7665618795162, + 7665652236210, + 7665684481864, + 7665718642098, + 7665752269572, + 7665785316876, + 7665817725520, + 7665850795572, + 7665884891058, + 7665918345807, + 7665953019118, + 7665985870104, + 7666018689511, + 7666053732505, + 7666085781267, + 7666118191650, + 7666150663560, + 7666183978054, + 7666217807001, + 7666251930594, + 7666285141202, + 7666320408654, + 7666352673403, + 7666386672636, + 7666418708764, + 7666450658301, + 7666483652958, + 7666517865011, + 7666551560336, + 7666586478294, + 7666619231234, + 7666653093494, + 7666686550458, + 7666719079082, + 7666751971451, + 7666784282722, + 7666818108207, + 7666853827554, + 7666885458935, + 7666918483300, + 7666952111973, + 7666985099307, + 7667020387665, + 7667052206826, + 7667084470528, + 7667118973624, + 7667152705516, + 7667184654409, + 7667217381589, + 7667252171000, + 7667286568627, + 7667318498884, + 7667350926358, + 7667384520020, + 7667420941000, + 7667452674907, + 7667484904551, + 7667517035326, + 7667550163729, + 7667584563410, + 7667618362896, + 7667651379166, + 7667684193637, + 7667718519117, + 7667753036870, + 7667785680621, + 7667818050469, + 7667850643325, + 7667884498987, + 7667917253919, + 7667950845483, + 7667985523979, + 7668021774333, + 7668053312422, + 7668085276014, + 7668118563309, + 7668150678158, + 7668183845059, + 7668220814239, + 7668252885534, + 7668284530790, + 7668317715893, + 7668350906181, + 7668385466824, + 7668419689363, + 7668451997243, + 7668486706719, + 7668520071018, + 7668552371799, + 7668585549129, + 7668619257691, + 7668653413659, + 7668685151631, + 7668718194216, + 7668752061953, + 7668786882480, + 7668819605512, + 7668852163758, + 7668884861850, + 7668917495531, + 7668952698359, + 7668986877327, + 7669018873532, + 7669051712288, + 7669084823737, + 7669121139946, + 7669152698058, + 7669185471478, + 7669218968268, + 7669253449514, + 7669285439652, + 7669318186378, + 7669351554056, + 7669386213205, + 7669420317403, + 7669452635058, + 7669484999196, + 7669519265221, + 7669553208105, + 7669586537223, + 7669619627130, + 7669653032658, + 7669685421593, + 7669718618258, + 7669753101379, + 7669786120437, + 7669819015824, + 7669854225123, + 7669886358287, + 7669918233116, + 7669952066347, + 7669984500097, + 7670020080167, + 7670053325191, + 7670087218419, + 7670118736389, + 7670153218283, + 7670186441602, + 7670219511347, + 7670253051424, + 7670285264424, + 7670320358316, + 7670354031908, + 7670385984350, + 7670419539602, + 7670451457603, + 7670489616576, + 7670519035346, + 7670551944549, + 7670586144151, + 7670621438268, + 7670652762184, + 7670687252266, + 7670720111084, + 7670755652784, + 7670786814549, + 7670819551967, + 7670853923881, + 7670885224352, + 7670920424461, + 7670952764468, + 7670985658113, + 7671019740996, + 7671052435875, + 7671086888663, + 7671119457116, + 7671151501409, + 7671184310277, + 7671220654673, + 7671256780433, + 7671286485250, + 7671320293559, + 7671352992853, + 7671386974071, + 7671420331890, + 7671452563958, + 7671486932236, + 7671519861862, + 7671552343407, + 7671585669848, + 7671618562370, + 7671651814738, + 7671687232894, + 7671719288420, + 7671751789527, + 7671789423500, + 7671825131360, + 7671853777688, + 7671886207006, + 7671918640173, + 7671953239302, + 7671987780573, + 7672019787327, + 7672052774576, + 7672085689635, + 7672121792930, + 7672153739044, + 7672185907648, + 7672220654797, + 7672254402760, + 7672286895889, + 7672319397172, + 7672352784818, + 7672386024111, + 7672420557848, + 7672453543813, + 7672486714507, + 7672522851199, + 7672553326375, + 7672587848227, + 7672619947508, + 7672653571759, + 7672685362444, + 7672718609047, + 7672751649978, + 7672785924178, + 7672818080547, + 7672851823282, + 7672888046040, + 7672919903527, + 7672951937722, + 7672984808978, + 7673018640486, + 7673053274065, + 7673086677917, + 7673119400022, + 7673152752674, + 7673188518152, + 7673220685075, + 7673253727733, + 7673285622988, + 7673318598518, + 7673352524481, + 7673386299743, + 7673419832184, + 7673452833833, + 7673486147393, + 7673519541117, + 7673551674883, + 7673585566423, + 7673618698587, + 7673652433942, + 7673686944271, + 7673719747531, + 7673753011051, + 7673789109690, + 7673820377497, + 7673851688514, + 7673885479499, + 7673918665746, + 7673953267494, + 7673987009674, + 7674020150816, + 7674053210406, + 7674087959159, + 7674120207826, + 7674153325760, + 7674186167824, + 7674218966156, + 7674254437778, + 7674287465546, + 7674319372089, + 7674352268541, + 7674386117170, + 7674420129411, + 7674452536032, + 7674485802747, + 7674517529858, + 7674553891708, + 7674587341101, + 7674618951642, + 7674651669865, + 7674685626602, + 7674720908198, + 7674755241649, + 7674785951237, + 7674819420271, + 7674853456393, + 7674887689075, + 7674920440172, + 7674953563478, + 7674987473063, + 7675021815755, + 7675053689026, + 7675086016572, + 7675120952700, + 7675155236646, + 7675190531244, + 7675222427509, + 7675254103507, + 7675287313884, + 7675320100617, + 7675352084456, + 7675386613257, + 7675419908836, + 7675454204799, + 7675487506803, + 7675519829967, + 7675552615440, + 7675587670822, + 7675620076528, + 7675653405943, + 7675686066768, + 7675719011780, + 7675753379401, + 7675787575488, + 7675820300783, + 7675853964368, + 7675888206831, + 7675920670246, + 7675953092492, + 7675986332190, + 7676020015934, + 7676055403333, + 7676088334249, + 7676120557951, + 7676152598612, + 7676188392306, + 7676222376833, + 7676254270399, + 7676288237978, + 7676338543027, + 7676358582894, + 7676389109800, + 7676421144555, + 7676453840281, + 7676487291576, + 7676520754364, + 7676555617626, + 7676586948439, + 7676620171731, + 7676653729323, + 7676688547053, + 7676720429001, + 7676753927347, + 7676786888665, + 7676820639093, + 7676853903185, + 7676886363306, + 7676919723857, + 7676954236506, + 7676988414279, + 7677021154668, + 7677054348266, + 7677087499426, + 7677121706967, + 7677154527921, + 7677186281887, + 7677222834107, + 7677255172257, + 7677288913312, + 7677321595789, + 7677352963410, + 7677386405682, + 7677421794672, + 7677455097167, + 7677488255358, + 7677519871329, + 7677552893800, + 7677586918666, + 7677621213454, + 7677652857195, + 7677685535743, + 7677720537832, + 7677755393749, + 7677787418659, + 7677820813910, + 7677853780955, + 7677888806346, + 7677920175427, + 7677956835872, + 7677986993108, + 7678020837939, + 7678056939932, + 7678088031915, + 7678120753023, + 7678153129735, + 7678188899622, + 7678221491092, + 7678253927843, + 7678286671098, + 7678320697823, + 7678355863417, + 7678388145610, + 7678420285173, + 7678455425315, + 7678489584347, + 7678521055396, + 7678553812474, + 7678586846591, + 7678620571256, + 7678655204116, + 7678686866403, + 7678720513579, + 7678753644213, + 7678788608259, + 7678821154184, + 7678854137165, + 7678886879887, + 7678919576030, + 7678956252722, + 7678990515552, + 7679020997097, + 7679054383708, + 7679088799655, + 7679119791317, + 7679153104860, + 7679185660206, + 7679219576627, + 7679257831877, + 7679288193247, + 7679319595143, + 7679352915589, + 7679386134918, + 7679420341406, + 7679453911188, + 7679487813146, + 7679522938374, + 7679555380472, + 7679587854960, + 7679619760869, + 7679652928738, + 7679686878743, + 7679720942440, + 7679753761428, + 7679787285529, + 7679820215159, + 7679855290456, + 7679887833455, + 7679919831898, + 7679953053192, + 7679986227685, + 7680021435785, + 7680057799524, + 7680089424840, + 7680121550395, + 7680157726919, + 7680188820674, + 7680221951806, + 7680255253456, + 7680286085008, + 7680322122760, + 7680355471714, + 7680387683984, + 7680420483823, + 7680453682880, + 7680487264621, + 7680520531910, + 7680557710931, + 7680589914726, + 7680622687551, + 7680655319755, + 7680688513399, + 7680720942397, + 7680755922353, + 7680788148137, + 7680821703543, + 7680854132824, + 7680886935460, + 7680920836876, + 7680954277398, + 7680987028914, + 7681020496513, + 7681053544562, + 7681088356145, + 7681122512719, + 7681155069954, + 7681187661074, + 7681223912214, + 7681256114794, + 7681287722314, + 7681322183936, + 7681354203846, + 7681388533356, + 7681421737879, + 7681454576416, + 7681487739954, + 7681521194530, + 7681554728162, + 7681588415821, + 7681620728170, + 7681654846134, + 7681689051981, + 7681722266519, + 7681754323371, + 7681787640532, + 7681823119224, + 7681855469154, + 7681888050330, + 7681921147183, + 7681956532456, + 7681990892820, + 7682022181370, + 7682055508788, + 7682088137525, + 7682122134987, + 7682155886352, + 7682188835593, + 7682221794575, + 7682257730244, + 7682289243501, + 7682322092555, + 7682355103787, + 7682387593377, + 7682422934565, + 7682456609998, + 7682488583098, + 7682521186066, + 7682557953493, + 7682589727987, + 7682622282611, + 7682656362400, + 7682688614780, + 7682722174697, + 7682755595525, + 7682788029097, + 7682821892838, + 7682857041664, + 7682888457415, + 7682922053839, + 7682955004369, + 7682988485715, + 7683023139488, + 7683056452774, + 7683089376375, + 7683122069005, + 7683157185210, + 7683189319281, + 7683222322270, + 7683255075552, + 7683287727396, + 7683323162429, + 7683356577183, + 7683390193055, + 7683423613940, + 7683457004092, + 7683488616569, + 7683520228577, + 7683554604453, + 7683587198867, + 7683624371017, + 7683656957920, + 7683688278424, + 7683721534239, + 7683755696257, + 7683790913143, + 7683822109298, + 7683855042368, + 7683888361119, + 7683924031717, + 7683957163394, + 7683993008670, + 7684023098744, + 7684056189436, + 7684089472763, + 7684122488090, + 7684154732592, + 7684187898622, + 7684222173475, + 7684255620952, + 7684288362217, + 7684321422022, + 7684354756290, + 7684389753960, + 7684422856143, + 7684455957423, + 7684489606360, + 7684521858884, + 7684554810587, + 7684590542080, + 7684622965849, + 7684654781951, + 7684687821962, + 7684722481513, + 7684755780321, + 7684788129170, + 7684820747899, + 7684853685057, + 7684889035262, + 7684921439048, + 7684955077225, + 7684987870409, + 7685024012975, + 7685057587603, + 7685089220028, + 7685121504719, + 7685157504937, + 7685188157100, + 7685221732931, + 7685255927416, + 7685288180419, + 7685322478848, + 7685355322512, + 7685388297478, + 7685423191272, + 7685456980382, + 7685489184158, + 7685525017279, + 7685555649630, + 7685588690334, + 7685622335838, + 7685655355484, + 7685687505549, + 7685722043200, + 7685756456857, + 7685788733004, + 7685821092753, + 7685854390494, + 7685888228329, + 7685922935633, + 7685956841699, + 7685988777538, + 7686024265070, + 7686057332047, + 7686088958761, + 7686122676541, + 7686156378433, + 7686188217363, + 7686223787320, + 7686256173356, + 7686288669880, + 7686341360695, + 7686365518585, + 7686390992275, + 7686422803412, + 7686455645751, + 7686488610188, + 7686522165589, + 7686555458440, + 7686588621465, + 7686621077236, + 7686655368636, + 7686689961093, + 7686722584793, + 7686755308563, + 7686787999213, + 7686823786381, + 7686855689908, + 7686890417472, + 7686922150637, + 7686958069796, + 7686989359119, + 7687021522329, + 7687055206214, + 7687088283674, + 7687121607909, + 7687155019385, + 7687188981031, + 7687222952405, + 7687256678101, + 7687289450354, + 7687320722731, + 7687353919705, + 7687388039240, + 7687422263109, + 7687456690517, + 7687490088968, + 7687521502672, + 7687556807649, + 7687588551454, + 7687621319580, + 7687654745742, + 7687687683173, + 7687721919751, + 7687757238476, + 7687789783978, + 7687821761605, + 7687856044230, + 7687889493495, + 7687922281928, + 7687954873892, + 7687988714187, + 7688023543489, + 7688057507317, + 7688089706894, + 7688122671061, + 7688156615764, + 7688190838460, + 7688225904359, + 7688256857292, + 7688288449870, + 7688324326021, + 7688356747892, + 7688389574614, + 7688422981474, + 7688455598892, + 7688488935762, + 7688522516679, + 7688555649094, + 7688588305808, + 7688623345927, + 7688656948929, + 7688689477188, + 7688722674994, + 7688756207185, + 7688788998037, + 7688822459252, + 7688856068171, + 7688888815901, + 7688923291965, + 7688956640943, + 7688988510760, + 7689022217056, + 7689055726101, + 7689089376828, + 7689122610888 + ], + "dropped_frames": 0, + "timestamp_backjumps": 0, + "sync_note": "\u6bcf\u884c\u6570\u636e\u6765\u81ea\u540c\u4e00 frameset\uff1bD405 \u5f69\u8272\u53d6\u81ea\u5de6\u6210\u50cf\u5668\uff0c\u4e0e\u6df1\u5ea6\u540c\u4e00\u6b21\u66dd\u5149\uff0c\u786c\u4ef6\u7ea7\u540c\u6b65" +} \ No newline at end of file diff --git a/docs/FOUNDATIONPOSE_SETUP.md b/docs/FOUNDATIONPOSE_SETUP.md new file mode 100644 index 0000000..3f6d378 --- /dev/null +++ b/docs/FOUNDATIONPOSE_SETUP.md @@ -0,0 +1,38 @@ +# FoundationPose local setup + +Validated 2026-09-16 for model-based RGB-D inference using an existing STL. + +- Source: `third_party/FoundationPose`, upstream commit `a1b694b83e633c2cb6115b9063d940a687759392`. +- Interpreter: existing `.venv/bin/python`; reused Torch 2.7.1+cu128 and PyTorch3D. +- CUDA toolkit: existing `.cuda/usr/local/cuda-12.8`; GPU architecture 8.6. +- Installed upstream Python requirements, pybind11, ninja, and locally built nvdiffrast 0.4.0. +- Compiled `third_party/FoundationPose/mycpp/build/mycpp.cpython-311-x86_64-linux-gnu.so`. +- Weights and mirror provenance: `third_party/FoundationPose/weights/download_manifest.json`. + +## Run the verified smoke check + +From the HandFlow root, in a shell with NVIDIA GPU access: + +```bash +source scripts/foundationpose_env.sh +python scripts/verify_foundationpose_setup.py +``` + +The environment script clears ROS-related Python/library overrides and selects the existing project environment/toolkit. It does not install Torch or CUDA. + +The check exercises actual CUDA rasterization, loads both pretrained networks and executes their forward passes, and initializes FoundationPose with `docs/上半.stl`, including compiled C++ rotation clustering. Results: `output/foundationpose_setup/runtime_verification.json`; installation/build logs are in the same directory. + +This does not validate pose accuracy or contact constraints on the recorded video. Optional BundleSDF `mycuda`/NeRF extensions are not required for the existing-STL path and were not built. + +## Rebuild compiled modules + +```bash +source scripts/foundationpose_env.sh +python -m pip install --no-build-isolation --no-deps ./third_party/nvdiffrast +cmake -S third_party/FoundationPose/mycpp -B third_party/FoundationPose/mycpp/build \ + -DCMAKE_BUILD_TYPE=Release \ + -Dpybind11_DIR="$FP_ROOT/.venv/lib/python3.11/site-packages/pybind11/share/cmake/pybind11" \ + -DPYTHON_EXECUTABLE="$FP_ROOT/.venv/bin/python" \ + -DPython_EXECUTABLE="$FP_ROOT/.venv/bin/python" +cmake --build third_party/FoundationPose/mycpp/build -j2 +``` diff --git a/docs/INTEGRATED_PROJECT.md b/docs/INTEGRATED_PROJECT.md index 600c24d..3486da9 100644 --- a/docs/INTEGRATED_PROJECT.md +++ b/docs/INTEGRATED_PROJECT.md @@ -1,5 +1,7 @@ # Dyn-HaMR → dex-retargeting → SPIDER 集成说明 +部署前先阅读 [整体配置与权重下载指南](SETUP_AND_WEIGHTS.md),环境模板为 `configs/project_env.example.sh`。 + ## 组件和职责 | 层 | 上游/实现 | 输入与输出 | diff --git a/docs/PIPELINE_LATEST.md b/docs/PIPELINE_LATEST.md new file mode 100644 index 0000000..db7f578 --- /dev/null +++ b/docs/PIPELINE_LATEST.md @@ -0,0 +1,273 @@ +# 最新流程(2026-09-17):RGB-D 双手视频 → Dyn-HaMR → L20 重定向 → FoundationPose → 参考修复 → SPIDER 物理 + +本文是 `docs/20260915_171525`(D405 头戴 RGB-D,352 帧,双手 + 红/蓝两个盒子)这条数据上**实际跑通并交付**的链路,按执行顺序给出每一步的脚本、输入、输出、关键参数和验证文件。旧的单手 / 瓶子链路见 [INTEGRATED_PROJECT.md](INTEGRATED_PROJECT.md),此处不再重复。RL 跟踪训练线不在本文范围。 + +**当前状态** + +- 运动学参考已交付:`output/final_smooth_20260915/`。手-盒 / 手-桌穿透 0(含 2.5 mm 间隙、400 Hz 插值 4681 步),精确 CAD 探针侵入 0,腕位置二阶差分 0.4 mm,手指 0.35°。 +- SPIDER 物理滚动:`output/final_spider_run_20260915/`。4680 步全部有限、穿透 0、CAD 探针 0;**但手不真正合拢(实际接触步 20/4680),红盒抬起与蓝盒翻转都未复现**。这是当前主问题。 +- 所有"接触"都是几何假设(人手指尖到 CAD 表面最近点),没有触觉真值;蓝盒第 177 帧后被遮挡,位姿只是保持最后可靠值。 + +## 0. 总览 + +```text +docs//{color.mp4, depth/*.png, intrinsics.json} docs/上半.stl(红盒) docs/下半.stl(蓝盒) + │ + [0] RGB-D 预处理 + 静态背景相机里程计 prepare_rgbd_20260915.py → output//rgbd_camera.npz + │ + [1] Dyn-HaMR 双手重建 run_dynhamr_rgbd_20260915.py → output/_dynhamr/optimization/ + 时序修正(SO(3) 局部均值 + 高斯) correct_dynhamr_rgbd.py → .../corrected/prior/*_world_results.npz + 导出人手 21 关节 export_dynhamr_bimanual_rgbd.py → .../human_joints_{left,right}.npz + │ + [2] L20 重定向(左右手各跑一次) retarget_l20_video.py → .../l20_{left,right}_stable/motion.npz + │ + [3] FoundationPose 物体 6D 逐帧跟踪 track_yesterday_boxes.py → output/foundationpose_spider_20260915/foundationpose_objects.npz + 碰撞资产(CoACD) decompose_yesterday_boxes.py / build_box_collision_v2.py / build_palm_collision_v2.py + │ + [4] 手物配准 ①(世界系 + 尺度 + 深度) register_hands_rgbd.py → .../registered/ + SPIDER 任务初版 prepare_yesterday_spider.py → output/registered_spider_20260915/ + 物体参考物理自洽 smooth_object_reference.py → output/final_spider_20260915/foundationpose_objects.npz + 手物配准 ②(指根对齐) register_hands_rgbd.py HF_ALIGN=mcp → .../aligned/ + SPIDER 任务(最终) prepare_yesterday_spider.py → output/final_spider_20260915/ + │ + [5] 碰撞修复 + 桌面约束 fix_yesterday_hand_collision.py → output/final_collision_fix_20260915/ + 验证 / 审计 / 渲染 verify_fix_interpolation.py · audit_collision_fix.py · render_collision_fix.py + 分开平滑 + 逐帧联合投影 smooth_project_reference.py → output/final_smooth_20260915/ ← 最终运动学参考 + │ + [6] SPIDER 物理滚动 configure_spider_task.py → run_yesterday_spider.py → output/final_spider_run_20260915/ + 独立审计 / 重放 / 渲染 / 交互回放 audit_collision_physics.py · check_spider_dynamics_replay.py · audit_spider_dynamic_surfaces.py · render_collision_fix.py --physics · play_spider_result.py +``` + +### 解释器 + +| 解释器 | 用在哪 | +|---|---| +| `.venv/bin/python`(执行 `source scripts/foundationpose_env.sh` 后就是 `python`) | FoundationPose、RGB-D 预处理、Dyn-HaMR 启动/修正/导出、配准、prepare / fix / verify / audit / smooth / render、交互回放 | +| `third_party/Dyn-HaMR/.dynhamr/bin/python` | Dyn-HaMR `run_opt.py`(由 `run_dynhamr_rgbd_20260915.py` 内部调用,不用手动进) | +| `.dex/bin/python` | dex-retargeting 0.5.0:`retarget_l20_video.py` | +| `.spider/bin/python` | SPIDER / MuJoCo Warp:`run_yesterday_spider.py`、`audit_collision_physics.py`、`check_spider_dynamics_replay.py` | + +离屏渲染脚本默认 `MUJOCO_GL=osmesa`;交互回放脚本自动切 `glfw`。所有脚本从项目根目录执行。环境安装见 [SETUP_AND_WEIGHTS.md](SETUP_AND_WEIGHTS.md)、[FOUNDATIONPOSE_SETUP.md](FOUNDATIONPOSE_SETUP.md)。 + +### 输入数据 + +- `docs/20260915_171525/color.mp4`(848×480,标称 30 fps,真实硬件时间跨度 11.70 s)、`depth/`(352 张对齐深度 PNG)、`intrinsics.json`(fx fy cx cy、帧数、时间戳)。RGB / 深度逐帧时间差 < 0.31 ms。 +- `docs/上半.stl` = 红盒,`docs/下半.stl` = 蓝盒。米制,18.16×16.60×4.69 cm。**红盒网格不封闭**(一条 T 形边),碰撞资产阶段在副本上修复,原文件不改。 +- L20 模型:`third_party/l20_assets/L20/{LEFT,RIGHT}/linkerhand_g20_{left,right}.urdf` 与网格。左右手各用自己的 URDF,不是镜像。 + +## 1. 各阶段 + +### [0] RGB-D 预处理与相机里程计 + +```bash +source scripts/foundationpose_env.sh +python scripts/prepare_rgbd_20260915.py +``` + +- 输出 `output/20260915_171525/rgbd_camera.npz`:`c2w [352,4,4]`、`intrinsics`、`time`、`valid`、`method`;另有 `rgbd_preflight.json`、`object_assets/`。 +- 方法:静态背景 RGB-D 里程计,屏蔽皮肤、红蓝物体和画面底部;无闭环、无外部定位真值。 +- **相机是头戴的**,背景帧间移动约 17 px,里程计不能省略;后面所有"世界系"都指这个里程计世界(首帧相机系)。 + +### [1] Dyn-HaMR 双手重建 + +```bash +python scripts/run_dynhamr_rgbd_20260915.py # 内部调用 .dynhamr 解释器 +python scripts/correct_dynhamr_rgbd.py +python scripts/export_dynhamr_bimanual_rgbd.py +python scripts/render_corrected_dynhamr_rgbd.py # 可选:多视角回放视频 +``` + +- `run_dynhamr_rgbd_20260915.py` 把 `rgbd_camera.npz` 写成 ViPE 文件格式(`rgbd_cameras/{pose,intrinsics}/`),然后跑上游 `run_opt.py`:`data=video_vipe is_static=False model.opt_scale=False run_opt=True run_prior=True run_vis=True +data.vipe_pipeline=dynhamr_cameras`,Hugging Face / Torch 缓存离线(`.hf_cache`、`.torch_cache`),EGL 渲染。原始结果在 `output/20260915_171525_dynhamr/optimization/`,逐帧检测在 `dataset/dynhamr/track_preds//{000,001}/`。 +- `correct_dynhamr_rgbd.py` → `corrected/prior/20260915_171525_000000_world_results.npz`、`comparison_joints.npz`、`validation.json`。腕与手指旋转做合法旋转空间局部加权平均(σ 1.2 帧),平移高斯 σ 1.5 帧(用未来帧,非实时滤波)。腕位置二阶差分 RMS 左 5.38→0.65、右 6.04→0.67 mm。**这一步的 RGB-D 深度门槛把所有帧都拒了(`depth_accepted_frames = 0`),深度修正实际在 [4] 完成。** +- `export_dynhamr_bimanual_rgbd.py` → `human_joints_{left,right}.npz`:`joints [352,21,3]`(腕相对、世界朝向)、`wrist_world`、`time`、`detection_valid`;并把 `rgbd_camera.npz`、`object_assets/` 复制进 `_dynhamr` 目录供后续使用。 +- 诊断(只读):`diagnose_dynhamr_rgbd.py`(`diagnosis/report.md`)、`diagnose_yesterday_hand_depth.py`(留出像素深度残差,`output/hand_depth_diagnosis_20260915/`)。 + +### [2] L20 重定向 + +```bash +MUJOCO_GL=osmesa .dex/bin/python scripts/retarget_l20_video.py \ + --input output/20260915_171525_dynhamr/human_joints_right.npz \ + --output-dir output/20260915_171525_dynhamr/l20_right_stable --side right +MUJOCO_GL=osmesa .dex/bin/python scripts/retarget_l20_video.py \ + --input output/20260915_171525_dynhamr/human_joints_left.npz \ + --output-dir output/20260915_171525_dynhamr/l20_left_stable --side left +``` + +- dex-retargeting 0.5.0 `PositionOptimizer`,16 个独立关节,其余按原 URDF **线性 mimic** 精确展开(`l20_consistent_optimizer.py` 修正了 0.5.0 目标函数与梯度不一致)。离线平滑 `--sigma 2.5` 帧。 +- 输出:`motion.npz`(腕位姿 + 关节角)、`trajectory.csv`、`l20_{side}.xml` / `l20_moving.xml`(MuJoCo)、`l20_dex.urdf`、`dex_config.json`、`retarget_validation.json`(指尖拟合误差、最大关节步长、限位越界、mimic 误差、Pinocchio 与 MuJoCo FK 一致性)。日志 `retarget_{side}.log`。 +- 手指重定向只用关节**方向**,所以 [4] 的相似变换配准不需要重跑重定向(`register_hands_rgbd.py` 只改写 `motion.npz` 里的腕位置)。 + +### [3] FoundationPose 物体跟踪与碰撞资产 + +```bash +source scripts/foundationpose_env.sh +python scripts/track_yesterday_boxes.py # → output/foundationpose_spider_20260915/foundationpose_objects.npz, foundationpose_overlay.mp4 +python scripts/decompose_yesterday_boxes.py # → output/foundationpose_spider_20260915/collision/{upper,lower}_NNN.obj(CoACD ≤20 块,初版,prepare 用) +python scripts/build_box_collision_v2.py # → output/collision_fix_20260915/collision_v2/(修 T 形边后的封闭 CAD *_closed.ply + 不限块数 CoACD + manifest.json,fix/smooth 用) +python scripts/build_palm_collision_v2.py # → output/collision_fix_20260915/hand_collision/(L20 手部网格 CoACD + manifest.json) +``` + +- 逐帧模型法 RGB-D 位姿,两个 STL 各自跟踪;`T_world = c2w @ T_cam`,c2w 来自 [0]。 +- 接受判据(非外部真值):可见颜色像素 > 500、投影覆盖 > 65%、重叠像素深度中位残差 < 20 mm。红盒 307/352 帧接受,未接受帧插值;蓝盒 177/352,第 177 帧后遮挡,**保持最后可靠位姿并停止作为接触目标来源**。 +- 实测噪声:静止红盒逐帧 2nd-diff 5.3 mm、0.56°;遮挡时静止物体会漂 2 cm。这些在 [4] 的物体参考自洽里处理。 +- 环境验证:`python scripts/verify_foundationpose_setup.py`,说明见 [FOUNDATIONPOSE_SETUP.md](FOUNDATIONPOSE_SETUP.md)。 + +### [4] 手物配准、物体参考自洽、SPIDER 任务构建 + +按下面顺序,5 条命令: + +```bash +source scripts/foundationpose_env.sh +# ① 配准(默认:腕对齐,原始里程计相机)→ output/20260915_171525_dynhamr/registered/ +python scripts/register_hands_rgbd.py +# ② 任务初版(供物体自洽读取手 / 桌面 / 接触目标) +HF_SPIDER_OUT=$PWD/output/registered_spider_20260915 \ +HF_HAND_BASE=$PWD/output/20260915_171525_dynhamr/registered \ +python scripts/prepare_yesterday_spider.py +# ③ 物体参考物理自洽(输入/输出路径写死:registered_spider_20260915 + registered/ → final_spider_20260915/) +python scripts/smooth_object_reference.py +# ④ 配准第二遍:指根对齐 + 平滑后的相机 → output/20260915_171525_dynhamr/aligned/ +HF_REG_OUT=$PWD/output/20260915_171525_dynhamr/aligned \ +HF_CAMERA_FILE=$PWD/output/final_spider_20260915/rgbd_camera_smooth.npz \ +HF_ALIGN=mcp python scripts/register_hands_rgbd.py +# ⑤ 最终任务(同目录里已有 ③ 写好的自洽物体参考) +HF_SPIDER_OUT=$PWD/output/final_spider_20260915 \ +HF_HAND_BASE=$PWD/output/20260915_171525_dynhamr/aligned \ +python scripts/prepare_yesterday_spider.py +``` + +**① `register_hands_rgbd.py`**(三项刚体/相似修正,不改手指关节;`registration.json`) + +1. 世界系:Dyn-HaMR world 与里程计 world 差常量平移 (−0.048, −0.023, −0.115) m,模 12.6 cm,逐帧标准差 < 4 mm。旧流程当同一个 world 用,手物错位 12.6 cm,这是此前腕修正量 8–10 cm 的主因。 +2. 尺度:MANO 手在错的深度上与图像轮廓重合(渲染/皮肤面积比 ≈ 1.0),说明真手比 MANO 大;绕相机中心常量缩放,右 1.105、左 1.047,轮廓不变、深度对齐。 +3. 残余深度:沿腕射线的逐帧留出像素残差,σ 2 帧平滑。留出残差右 42→9.4 mm、左 20→10 mm。 + 环境变量:`HF_ALIGN`(`wrist` | `mcp`)、`HF_CAMERA_FILE`、`HF_REG_OUT`。 + +**② / ⑤ `prepare_yesterday_spider.py`**:由深度拟合桌面(z-up 是桌面法线,不是 IMU 重力);逐指尖选最近可观测物体生成可切换接触目标(进入 25 mm / 释放 35 mm);有界关节/腕平移热启动。输出 `datasets/processed/current/l20/bimanual/boxes/{scene_act.xml, task_info.json, 0/trajectory_kinematic_act.npz}`、`reference_video_rate.npz`、`config.json`、`model_{side}/l20_{side}.xml`、`object_reference.npz`、`preparation.json`。`collision/`、`foundationpose_objects.npz` 缺省时符号链接到 `foundationpose_spider_20260915`。环境变量:`HF_SPIDER_OUT`、`HF_HAND_BASE`。 + +**③ `smooth_object_reference.py`**:物体只允许两种状态。静止段(0.5 s 窗内位移 < 6 mm、旋转 < 2°、无手接触)冻结在"深度共识"位姿,深度驱动分裂(连续 ≥ 9 帧比原观测差 3 mm 判定真动过),贴桌吸附只在深度残差不变差时接受(深度拟合的桌面只准到 ~1 cm);握持段(≥ 2 指尖进入 25 mm)在腕系 σ 4 帧平滑后随手刚性运动,比纯平滑差 > 1 mm 则退回;自由段 σ 2;里程计 c2w σ 2。红盒 2nd-diff 4.7→1.5 mm,蓝盒 8.8→2.6 mm,深度残差不劣于原始。输出 `foundationpose_objects.npz`(同 schema)、`rgbd_camera_smooth.npz`、`object_segments.npz`、`object_smoothing.json`。 + +**④ `HF_ALIGN=mcp`**:L20 比缩放后的人手大 35–60%,差异几乎全在手掌(腕到指根 148 vs 90 mm)。改为 L20 指根中心对齐人手指根,腕自动后退 58–61 mm;人手指尖(接触目标)不变。L20 base 系约定已数值验证(`landmark_world = wrist_R @ local + wrist_pos`,误差 0)。 + +### [5] 碰撞修复 + 桌面约束 + 平滑投影 → 最终运动学参考 + +```bash +source scripts/foundationpose_env.sh +export HF_FIX_OLD=$PWD/output/final_spider_20260915 \ + HF_FIX_OUT=$PWD/output/final_collision_fix_20260915 \ + HF_WRIST_WEIGHT=200 HF_MARGIN=0.0025 HF_TABLE=1 \ + SPIDER_TASK_OUT=$PWD/output/final_collision_fix_20260915 +python scripts/fix_yesterday_hand_collision.py # 修复 +python scripts/verify_fix_interpolation.py --video --buffer # 视频率状态检查 + 投影 +python scripts/verify_fix_interpolation.py --buffer # 400 Hz 插值 4681 步检查 + 投影(物体固定) +python scripts/audit_collision_fix.py # 独立审计 → validation.json +python scripts/render_collision_fix.py # hand_collision_before_after.mp4 + +export HF_SMOOTH_OUT=$PWD/output/final_smooth_20260915 +python scripts/smooth_project_reference.py # 平滑 + 联合投影 +export HF_FIX_OUT=$HF_SMOOTH_OUT SPIDER_TASK_OUT=$HF_SMOOTH_OUT +python scripts/verify_fix_interpolation.py --buffer +python scripts/audit_collision_fix.py +python scripts/render_collision_fix.py +``` + +- `fix_yesterday_hand_collision.py`:解析接触雅可比 + 精确 mimic,逐帧最小改动求解,目标 = 接触目标 + 腕先验(`HF_WRIST_WEIGHT`,默认 25,本链 200)+ 关节限位;间隙放进求解器(`HF_MARGIN`,默认 0.5 mm,本链 2.5 mm)而不是事后投影(事后投影曾在 201–203 帧产生 88 mm 跳变);`HF_TABLE=1` 把桌面变成硬约束(floor conaffinity 3)。碰撞体来自 `collision_v2/` + `hand_collision/`。 +- `verify_fix_interpolation.py` / `audit_collision_fix.py`:分别检查视频率与 400 Hz 插值状态、全连杆碰撞、限位、连续性和**精确闭合 CAD 采样探针**(CoACD 近似之外的独立检查)。都吃 `HF_FIX_OLD / HF_FIX_OUT / HF_TABLE`。 +- `smooth_project_reference.py`:腕位置高斯 σ 2 帧、腕旋转 SO(3) 局部加权均值 σ 2、16 个独立手指关节 Savitzky–Golay(窗 11、二阶)后精确 mimic 展开、物体不动;再逐帧最小改动投影:线性化接触距离 ≥ 2.5 mm(盒 + 桌)、关节限位、相对上一帧步长上限(8 mm / 7° / 8.6°)。环境变量 `HF_SMOOTH_OUT HF_SIGMA_POS HF_SIGMA_ROT HF_SG_WINDOW HF_CLEARANCE`。 +- 交付目录 `output/final_smooth_20260915/`(README 在内):`reference_video_rate.npz`、`datasets/.../0/trajectory_kinematic_act.npz`(400 Hz)、`validation.json`、`smoothing_report.json`、`hand_collision_before_after.mp4`。交互回放:`python scripts/play_spider_result.py --directory output/final_smooth_20260915 --mode reference`(空格暂停,← → 逐帧)。 + +| 指标(右 / 左) | 修复后(final_collision_fix) | 平滑投影后(final_smooth) | +|---|---:|---:| +| 腕位置二阶差分 RMS | 3.05 · 3.15 mm | 0.41 · 0.37 mm | +| 手指二阶差分 RMS | 1.49° · 1.33° | 0.38° · 0.32° | +| 右腕修正中位 / 118–235 帧段 | 11 / 45 mm | — | +| 接触间隙中位 | 7.1 · 5.7 mm | 8.0 · 7.8 mm | +| 最大穿透(盒 + 桌,含 400 Hz 插值) | 0 | 0 | +| 精确 CAD 探针最大侵入(红 / 蓝) | — | 0.00 / 0.00 mm | + +### [6] SPIDER 物理滚动 + +```bash +export SPIDER_SRC=$PWD/output/final_smooth_20260915 \ + SPIDER_TASK_OUT=$PWD/output/final_spider_run_20260915 \ + HF_FIX_OLD=$PWD/output/final_spider_20260915 +source scripts/foundationpose_env.sh +python scripts/configure_spider_task.py # 复制任务 + 施加物理参数 → physics_parameters.json +.spider/bin/python scripts/run_yesterday_spider.py --pilot # 冒烟:80 步、32 样本、2 轮 +.spider/bin/python scripts/run_yesterday_spider.py # 全程:64 样本 × 3 轮,horizon 0.4 s,ctrl_dt 0.1 s,4680 步 +.spider/bin/python scripts/audit_collision_physics.py # CPU 独立碰撞逐步检查 → physics_validation.json, physics_motion.npz +.spider/bin/python scripts/check_spider_dynamics_replay.py # 保存控制量在全新 GPU 世界 + CPU 重放 → control_replay.json +python scripts/audit_spider_dynamic_surfaces.py # 原始手部可视网格 vs 精确封闭 CAD 探针 → dynamic_surface_validation.json +python scripts/render_collision_fix.py --physics # spider_collision_rollout.mp4(原视频 / 运动学参考 / 物理结果) +python scripts/play_spider_result.py --directory output/final_spider_run_20260915 --mode physics # 交互回放 +``` + +- `configure_spider_task.py`:从 `SPIDER_SRC` 复制 `scene_act.xml`、`task_info.json`、`trajectory_kinematic_act.npz`、`reference_video_rate.npz`、`config.json`(只改 `dataset_dir`),并写入经 GPU/CPU 短段重放选出的物理设置(仿真假设,非硬件标定):求解 80 / 线搜索 50;所有 geom `solimp .9 .95 .001 .5 2`、`solref .02 1`;手/物/桌 `margin 4 mm gap 2 mm`(2 mm 提前激活);腕平移 kp 500 kv 30 ±20 N,腕旋转 kp 15 kv 2 ±2 Nm,手指 kp 4 kv 0.15 ±0.1 Nm(`HF_FINGER_KP / HF_FINGER_KV / HF_FINGER_TORQUE` 可改);物体执行器零增益(真实滚动无物体辅助);桌面与手碰撞。`SPIDER_CONFIG_OVERRIDES='{"contact_rew_scale": 8, "num_samples": 128}'` 可覆盖 SPIDER config。 +- `run_yesterday_spider.py`:调用上游 `third_party/spider/examples/run_mjwp.py`(MuJoCo Warp 采样优化),本地补丁:双手含被动关节的奖励权重划分、跨物体换握时关闭"单手固定物体"启发式、非有限状态即停。物体质量 0.12 kg、摩擦 0.8 是假设。上游 SPIDER 的其余本地修改(辅助弹簧力公式、接触奖励权重、重采样漏控制量、GPU 接触参数)随 `third_party/spider` 源码快照保留。 +- 结果(`final_spider_run_20260915/README.md`): + +| 指标 | 值 | +|---|---:| +| 物理步 / 全部有限 | 4680 / 是 | +| 最大手物穿透(CPU 独立检测)/ CAD 探针 | 0 / 0 mm | +| 实际接触步数(< 1 mm) | 20 / 4680 | +| 红盒观测段位置误差均值 / 旋转误差 | 93 mm / 8.1°(0–5 s 跟得住,6 s 参考抬起后物理盒留桌上) | +| 蓝盒观测段位置误差均值 / 旋转误差 | 39 mm / 123°(1–2 s 参考翻转,物理盒不翻) | +| 保存控制量独立 GPU 重放最大状态差 | 0.456 | + +结论:参考干净之后,失败模式变清楚,**手不真正合拢握住物体**,位置伺服跟参考就是"悬着"。参考间隙 0.5 mm + `contact_rew_scale 8` + 128 样本 × 5 轮的变体(`output/final_spider_run_v2_20260915/`)**无实质变化**(接触步 20→5),说明不是采样量问题,是参考/奖励只要求指尖到表面、不要求夹紧。 + +## 2. 已知问题与候选方向(运动学 / SPIDER 线) + +- 接触语义:L20 比人手大,接触目标取人手指尖最近点,右手 118–235 帧腕仍要抬 45 mm 才能不进盒。候选:用 L20 指尖自身最近点做目标、按手长比例外推、或该段降接触权重。 +- 夹紧:参考里给 on-接触帧手指多屈几度(目标进入物体一点);手指力矩上限 0.1→0.3 Nm / kp 4→8;或奖励加法向力项。这些都改变仿真假设,需要决定。 +- 右手指尖到红盒表面配准后仍约 4 cm(遮挡下 MANO 手指姿态误差),接触目标本身有此量级不确定性。 +- 蓝盒 177 帧后无观测;红蓝相对装配不是已验证事实。 +- 未验证真机;未做力/摩擦标定。 + +## 3. 换一段新视频要改什么 + +脚本里序列名和输入路径大多写死为 `20260915_171525`(或 `yesterday` 命名),环境变量只覆盖**输出**目录。新视频需要: + +1. 放到 `docs//{color.mp4, depth/, intrinsics.json}`,物体 CAD 放 `docs/`。 +2. 改这些脚本里的常量:`prepare_rgbd_20260915.py`(`SRC/OUT`)、`run_dynhamr_rgbd_20260915.py`(`SEQ`)、`correct_dynhamr_rgbd.py` / `export_dynhamr_bimanual_rgbd.py`(`BASE/SRC`)、`track_yesterday_boxes.py`(`SRC/OUT`、STL 名)、`decompose_yesterday_boxes.py`(STL 名)、`smooth_object_reference.py`(`SRC/OLD/HAND/OUT`)、`register_hands_rgbd.py`(`BASE`)。 +3. `prepare_yesterday_spider.py` 里颜色→CAD 映射(红 = `上半`/upper,蓝 = `下半`/lower)和物体质量 / 摩擦假设。 +4. 之后 [4]–[6] 只靠环境变量指路径,不用改代码。 + +## 4. 脚本索引 + +| 阶段 | 脚本 | 解释器 | 输入 → 输出 | 环境变量 / 参数 | +|---|---|---|---|---| +| 0 | `prepare_rgbd_20260915.py` | .venv | docs/ → output//rgbd_camera.npz | — | +| 1 | `run_dynhamr_rgbd_20260915.py` | .venv(内部 .dynhamr) | color.mp4 + rgbd_camera → _dynhamr/optimization | — | +| 1 | `correct_dynhamr_rgbd.py` | .venv | optimization → corrected/prior/*_world_results.npz | — | +| 1 | `export_dynhamr_bimanual_rgbd.py` | .venv | corrected → human_joints_{left,right}.npz | — | +| 1 | `render_corrected_dynhamr_rgbd.py`、`diagnose_dynhamr_rgbd.py`、`diagnose_yesterday_hand_depth.py` | .venv | 可视化 / 诊断(只读) | — | +| 2 | `retarget_l20_video.py` | .dex | human_joints_{side}.npz → l20_{side}_stable/ | `--input --output-dir --side --sigma` | +| 3 | `track_yesterday_boxes.py` | .venv (foundationpose_env) | RGB-D + STL → foundationpose_objects.npz | — | +| 3 | `decompose_yesterday_boxes.py`、`build_box_collision_v2.py`、`build_palm_collision_v2.py` | .venv | STL / L20 网格 → CoACD 凸块 | — | +| 4 | `register_hands_rgbd.py` | .venv | human_joints + rgbd_camera → registered/ 或 aligned/ | `HF_ALIGN HF_CAMERA_FILE HF_REG_OUT` | +| 4 | `prepare_yesterday_spider.py` | .venv | 手 + 物体 → SPIDER 任务目录 | `HF_SPIDER_OUT HF_HAND_BASE` | +| 4 | `smooth_object_reference.py` | .venv | registered_spider + registered → final_spider(物体参考) | — | +| 5 | `fix_yesterday_hand_collision.py` | .venv | 任务目录 → 无穿透参考 | `HF_FIX_OLD HF_FIX_OUT HF_WRIST_WEIGHT HF_MARGIN HF_TABLE`;`--pilot` | +| 5 | `verify_fix_interpolation.py` | .venv | 视频率 / 400 Hz 检查投影 | `HF_FIX_OLD HF_FIX_OUT HF_TABLE`;`--video --buffer` | +| 5 | `audit_collision_fix.py` | .venv | 独立审计 → validation.json | `HF_FIX_OLD HF_FIX_OUT HF_TABLE` | +| 5 | `smooth_project_reference.py` | .venv | 修复参考 → 平滑投影参考 | `HF_FIX_OLD HF_FIX_OUT HF_SMOOTH_OUT HF_TABLE HF_SIGMA_POS HF_SIGMA_ROT HF_SG_WINDOW HF_CLEARANCE` | +| 5/6 | `render_collision_fix.py` | .venv | 三栏对比视频 | `SPIDER_TASK_OUT HF_FIX_OLD`;`--physics` | +| 6 | `configure_spider_task.py` | .venv | 参考目录 → SPIDER 任务 + 物理参数 | `SPIDER_SRC SPIDER_TASK_OUT HF_FINGER_KP/KV/TORQUE SPIDER_CONFIG_OVERRIDES` | +| 6 | `run_yesterday_spider.py` | .spider | 任务目录 → trajectory_mjwp_act.npz | `SPIDER_TASK_OUT`;`--pilot --steps` | +| 6 | `audit_collision_physics.py`、`check_spider_dynamics_replay.py` | .spider | 物理轨迹审计 / 控制量重放 | `SPIDER_TASK_OUT` | +| 6 | `audit_spider_dynamic_surfaces.py` | .venv | 精确 CAD 探针 | `SPIDER_TASK_OUT` | +| 6 | `play_spider_result.py` | .venv | 交互回放 | `--directory --mode {reference,physics} --check` | + +## 5. 历史 / 对照目录(不要误用) + +| 目录 | 说明 | +|---|---| +| `output/20260915_171525/` | HandFlow 分支的双手初版(镜像左手 + ICP 物体位姿),已被 Dyn-HaMR + FoundationPose 链路取代 | +| `output/foundationpose_spider_20260915/` | 首次全链路(配准前)。手物错位 12.6 cm,Spider 红盒误差 61 mm | +| `output/collision_fix_20260915/` | 配准前的碰撞修复,腕修正中位 82 / 100 mm;其 `collision_v2/`、`hand_collision/` 资产仍被后续目录符号链接 | +| `output/registered_collision_fix_*_20260915/` | 配准后三个修复变体,推荐 `w200m25`(腕先验 200 + 内置 2.5 mm 间隙) | +| `output/desktop_smooth_fix_20260915/` | `smooth_table_constrained.py` 只抬腕出桌面,手盒穿透回到 24.5 mm,**失败对照** | +| `output/final_smooth_c05_20260915/`、`output/final_spider_run_v2_20260915/` | 间隙 0.5 mm + 接触奖励 8 + 128 样本变体,无实质变化 | +| `output/spider_dynamics_fix_20260915/` | Codex 的 SPIDER 动力学修复轮(物理参数从这里选出) | diff --git a/docs/SETUP_AND_WEIGHTS.md b/docs/SETUP_AND_WEIGHTS.md new file mode 100644 index 0000000..d7b6df0 --- /dev/null +++ b/docs/SETUP_AND_WEIGHTS.md @@ -0,0 +1,205 @@ +# 安装依赖、环境与权重(2026-09-17) + +适用:源码快照 `hand-motion-pipeline-20260917`(由 `scripts/package_source_release.py` 生成)。链路为 **RGB-D → Dyn-HaMR → L20 重定向 → FoundationPose → 参考修复 → SPIDER**,执行顺序与每步命令见 [PIPELINE_LATEST.md](PIPELINE_LATEST.md)。本文按本机(2026-09-17)实际环境核对;**没有在干净机器上从零复现安装**,版本表是观测值,不是跨平台 lock。 + +## 1. 机器与系统依赖 + +| 项 | 本机观测 | +|---|---| +| OS / GPU / 驱动 | Ubuntu 22.04.5,RTX 3080 Laptop 16 GB,NVIDIA 驱动 595.91 | +| CUDA toolkit(nvcc) | 12.8,解包在项目内 `.cuda/usr/local/cuda-12.8`(用户级,不需要 sudo;`scripts/foundationpose_env.sh` 把 `CUDA_HOME` 指向它)。任何能提供 nvcc 12.8 的安装方式都可以,Torch 是 cu128 版所以 toolkit 主版本要对上 | +| apt 包 | `build-essential cmake ninja-build ffmpeg libeigen3-dev libosmesa6-dev libegl1-mesa-dev libgl1-mesa-dev libglfw3-dev` | +| 渲染 | 离屏 `MUJOCO_GL=osmesa`(默认);交互回放 `glfw`;Dyn-HaMR 可视化用 EGL | + +`scripts/foundationpose_env.sh` 会清空 ROS 注入的 `PYTHONPATH / LD_LIBRARY_PATH`,把 `.venv`、nvcc、编译缓存(`.cache/`)指到项目内。**所有 `.venv` 脚本都先 `source` 它**。 + +## 2. 环境角色(4 个解释器,不要混装) + +| 角色 | 路径约定 | Python | Torch | 关键依赖(观测版本) | 谁用 | +|---|---|---|---|---|---| +| 主环境 | `.venv` | 3.11.15 | 2.7.1+cu128 / torchvision 0.22.1 | pytorch3d 0.7.9、nvdiffrast 0.4.0、mujoco 3.5.0、mujoco-warp 3.5.0、warp-lang 1.14.0、open3d 0.19.0、trimesh 4.12.2、coacd 1.0.14、kornia 0.8.3、opencv 5.0 | FoundationPose、RGB-D 预处理/里程计、Dyn-HaMR 启动与导出、配准、prepare / fix / verify / audit / smooth / render、回放([PIPELINE_LATEST.md](PIPELINE_LATEST.md) 里除重定向和 SPIDER 外全部);HandFlow / ViPE 分支也在这里 | +| Dyn-HaMR | `third_party/Dyn-HaMR/.dynhamr` | 3.10.12 | 2.7.0+cu128 / torchvision 0.22.0 | ultralytics 8.1.34、hydra-core 1.3.6、kornia 0.8.2、smplx 0.1.28 | `run_opt.py`(由 `scripts/run_dynhamr_rgbd_20260915.py` 内部调用) | +| 重定向 | `.dex` | 3.11.15 | 2.7.1+cu128 | dex-retargeting 0.5.0、pin 4.1.0、mujoco 3.5.0、h5py 3.16.0 | `retarget_l20_video.py`、HDF5 导出 | +| SPIDER | `.spider` | 3.11.15(兼容安装;上游 pyproject 要求 ≥ 3.12) | 2.7.1+cu128 | mujoco 3.7.0、mujoco-warp 3.7.0.1、warp-lang 1.12.1、open3d 0.19.0、loguru | `run_yesterday_spider.py`、`audit_collision_physics.py`、`check_spider_dynamics_replay.py` | + +完整观测表:[requirements/runtime_observed.json](../requirements/runtime_observed.json)。`.dex` 的 MuJoCo 3.5 与 SPIDER 的 3.7 不能装进同一个环境。 + +## 3. 安装顺序 + +### 3.1 主环境 `.venv`(FoundationPose + 全部流程脚本) + +```bash +python3.11 -m venv .venv +.venv/bin/python -m pip install torch==2.7.1 torchvision==0.22.1 --index-url https://download.pytorch.org/whl/cu128 +.venv/bin/python -m pip install -r requirements/pipeline_venv.txt +source scripts/foundationpose_env.sh # 之后 python == .venv/bin/python,CUDA_HOME 指向 nvcc 12.8 +python -m pip install --no-build-isolation "git+https://github.com/facebookresearch/pytorch3d.git@stable" # 观测 0.7.9 +python -m pip install --no-build-isolation --no-deps ./third_party/nvdiffrast # 快照 0.4.0,需 nvcc +# FoundationPose C++ 旋转聚类模块(pybind11),产物 third_party/FoundationPose/mycpp/build/mycpp.cpython-311-*.so +cmake -S third_party/FoundationPose/mycpp -B third_party/FoundationPose/mycpp/build -DCMAKE_BUILD_TYPE=Release \ + -Dpybind11_DIR="$FP_ROOT/.venv/lib/python3.11/site-packages/pybind11/share/cmake/pybind11" \ + -DPYTHON_EXECUTABLE="$FP_ROOT/.venv/bin/python" -DPython_EXECUTABLE="$FP_ROOT/.venv/bin/python" +cmake --build third_party/FoundationPose/mycpp/build -j2 +python scripts/verify_foundationpose_setup.py # GPU 光栅化 + 两个网络前向 + STL 估计器初始化 → output/foundationpose_setup/runtime_verification.json +``` + +- `requirements/pipeline_venv.txt` 是本机观测的 pin 列表(不含 torch / pytorch3d / nvdiffrast,这三者按上面单独装)。 +- FoundationPose 上游 `requirements.txt` 建议 cu124;本机用 cu128 + 上述版本跑通,不要再按上游装一遍 torch。 +- BundleSDF 的 `mycuda` / NeRF 扩展**不需要**(只跑模型法 STL 路径),未编译。 +- HandFlow / ViPE 单目分支也用这个环境,但本链路不需要它们(见第 4.4 节)。详细说明:[FOUNDATIONPOSE_SETUP.md](FOUNDATIONPOSE_SETUP.md)。 + +### 3.2 Dyn-HaMR `.dynhamr` + +```bash +python3.10 -m venv third_party/Dyn-HaMR/.dynhamr +third_party/Dyn-HaMR/.dynhamr/bin/python -m pip install torch==2.7.0 torchvision==0.22.0 --index-url https://download.pytorch.org/whl/cu128 +# 其余依赖按 third_party/Dyn-HaMR/scripts/install_pip.sh 逐条执行(跳过它里面的 torch 行;该脚本钉的是旧版 torch) +``` + +Dyn-HaMR 自带 `third-party/hamer`(含 ViTPose)与 `third-party/Hand-BMC-pytorch-main`,都在快照里;运行时 `PYTHONPATH` 由 `scripts/run_dynhamr_rgbd_20260915.py` 设置,不需要手动 export。本快照对上游的修改(`run_opt.py`、`vis/tools.py`、`vis/viewer.py` 等 6 处 + 新增 `HMP/windowed.py`)已包含在源码里,见 `UPSTREAM_SOURCES.json` 的 `local_changes`。 + +### 3.3 重定向 `.dex` + +```bash +python3.11 -m venv .dex +.dex/bin/python -m pip install torch==2.7.1 torchvision==0.22.1 --index-url https://download.pytorch.org/whl/cu128 +.dex/bin/python -m pip install -r requirements/retarget.txt # dex-retargeting 0.5.0、mujoco 3.5.0、h5py 等 +``` + +### 3.4 SPIDER `.spider` + +上游 `third_party/spider/pyproject.toml` 要求 Python ≥ 3.12。干净机器按上游装: + +```bash +python3.12 -m venv .spider +.spider/bin/python -m pip install torch==2.7.1 torchvision==0.22.1 --index-url https://download.pytorch.org/whl/cu128 +.spider/bin/python -m pip install -e third_party/spider # 解析 pyproject 全部直接依赖(mujoco 3.7、mujoco-warp、warp-lang 1.12 等) +env PYTHONPATH= LD_LIBRARY_PATH= .spider/bin/python scripts/verify_spider_install.py # GPU 两世界 60 步物理冒烟 +``` + +本机是 Python 3.11 兼容安装(`--ignore-requires-python --no-deps` 装 spider 本体,再装解析出的依赖),记录在 `requirements/spider_resolved.txt`(完整解析结果,含本机共享环境里已有的包)和 `requirements/spider_installed_local.txt`(本机实际新增的 51 个包)。这是历史记录,不是安装脚本。本快照对上游 SPIDER 的本地补丁(`examples/run_mjwp.py`、`spider/optimizers/sampling.py`、`spider/simulators/mjwp.py`:辅助弹簧力公式、接触奖励权重、重采样漏控制量、GPU 接触参数)已在源码里;`-e` 安装即生效。 + +## 4. 权重与外部资源 + +### 4.0 从代码托管平台克隆(权重已随仓库,Git LFS) + +仓库 `https://gitea.robotquan.com/liyang/hand-motion-pipeline` 通过 Git LFS 带了下表中"本链路需要"的全部权重、L20 模型与录制数据(MANO 除外)。服务器单次请求上限 50 MB,超过的 10 个文件以 48 MiB 分片 `*.part-NNN` 存放,清单在 `configs/large_files.json`。克隆后先拼回: + +```bash +git lfs install +git clone https://gitea.robotquan.com/liyang/hand-motion-pipeline.git && cd hand-motion-pipeline +git lfs pull # 约 6.3 GB +python3 scripts/large_files.py assemble # 拼回 hamer.ckpt / UniDepth / HandFlow / HMP / FoundationPose / detector,逐个 sha256 校验 +python3 scripts/large_files.py assemble --clean # 可选:拼回后删除分片省空间 +``` + +`third_party/Dyn-HaMR/_DATA/` 里指向 HaMeR 检查点、MANO、检测器的是相对符号链接,克隆后自动生效。之后只差 MANO(4.2 节)。 + +源码包只含代码、配置、文档和两个小 CAD(`docs/上半.stl`、`docs/下半.stl`)。下表其余项都要另行准备;哈希与大小见包内 `EXTERNAL_ASSETS.json`。 + +| 资源 | 大小 | 放置位置 | 来源 | 本链路需要 | +|---|---|---|---|---| +| Dyn-HaMR 数据包 `_DATA/`(`hamer_ckpts/`、`data/mano/`、`hmp_model/`、`BMC/`) | 787 MB | `third_party/Dyn-HaMR/_DATA/` | 上游 `scripts/prepare.sh`(Google Drive)+ HaMeR `fetch_demo_data.sh` | 是 | +| `MANO_RIGHT.pkl` | — | `third_party/Dyn-HaMR/_DATA/data/mano/` | [MANO 官网](https://mano.is.tue.mpg.de/) 注册下载,不在任何包里 | 是(双手处理按 Dyn-HaMR 代码检查左手资源) | +| WiLoR 手检测器 `detector.pt` | 52 MB | `weights/detector.pt` + `third_party/Dyn-HaMR/third-party/hamer/pretrained_models/detector.pt` | [WiLoR HF Space](https://huggingface.co/spaces/rolpotamias/WiLoR/tree/main/pretrained_models) | 是 | +| FoundationPose 权重:`2023-10-28-18-33-37/`(refiner,68 MB)、`2024-01-11-20-02-45/`(scorer,190 MB),各含 `config.yml + model_best.pth` | 247 MB | `third_party/FoundationPose/weights/` | `scripts/download_foundationpose_weights.py`(清单 `configs/foundationpose_weights_manifest.json`,SHA-256 校验) | 是 | +| L20 灵巧手模型:`L20/{LEFT,RIGHT}/linkerhand_g20_{left,right}.urdf` + `meshes/` + 标定 JSON | 36 MB | `third_party/l20_assets/` | 模型提供方交付;左右手各自的 URDF,不是镜像 | 是 | +| 录制数据 `color.mp4` + `depth/*.png` + `intrinsics.json` | 116 MB | `docs/20260915_171525/` | 自采(D405 头戴) | 是 | +| 盒子 CAD `上半.stl`(红)、`下半.stl`(蓝) | 104 KB | `docs/` | **已随源码包** | 是 | +| HandFlow 权重(`handflow_denoiser.pt`、`normalization_stats.npz`、UniDepth) | 2 GB | `weights/` | [mxxu00/HandFlow](https://huggingface.co/mxxu00/HandFlow)、HF `lpiccinelli/unidepth-v2-vitl14` | 否(HandFlow 单目分支) | +| HaMeR 独立副本 `_DATA/` | 2.6 GB | `third_party/hamer/_DATA/` | HaMeR `fetch_demo_data.sh` | 否(HandFlow 分支) | +| ViPE 模型缓存(SAM、DeAOT、GroundingDINO、UniDepth、DROID、GeoCalib、Depth-Anything) | ~4 GB | `.torch_cache/`、`.hf_cache/` | 见 4.4 | 否(单目无深度视频才需要) | +| 瓶子参数化模型 | 156 KB | `third_party/bottle_model_parametric/` | 项目附件 | 否(旧单手链路) | + +### 4.1 Dyn-HaMR 数据包、MANO、检测器 + +```bash +cd third_party/Dyn-HaMR && bash scripts/prepare.sh && cd ../.. # 需要 gdown;下载后核对 _DATA 内四个子目录都非空 +# MANO:官网下载后放 third_party/Dyn-HaMR/_DATA/data/mano/MANO_RIGHT.pkl +mkdir -p weights third_party/Dyn-HaMR/third-party/hamer/pretrained_models +curl -fL --retry 3 -C - 'https://huggingface.co/spaces/rolpotamias/WiLoR/resolve/main/pretrained_models/detector.pt' -o weights/detector.pt +cp weights/detector.pt third_party/Dyn-HaMR/third-party/hamer/pretrained_models/detector.pt +``` + +`prepare.sh` 还会拉旧的 DROID 资源,可按脚本逐项取舍。不能用"脚本退出码 0"代替 checkpoint 存在性检查。 + +### 4.2 FoundationPose 权重 + +```bash +source scripts/foundationpose_env.sh +python scripts/download_foundationpose_weights.py +``` + +官方 Google Drive 在制作快照时配额超限,清单里的 URL 是社区 Hugging Face 镜像 `gpue/foundationpose-weights`,按镜像 LFS SHA-256 校验,**未与官方文件独立比对**。有官方链接时替换清单里的 `url` 即可,哈希不变。 + +### 4.3 L20 模型与录制数据 + +由提供方 / 自己放到上表路径。`third_party/l20_assets` 与 `docs/20260915_171525` 的逐文件 SHA-256 在 `EXTERNAL_ASSETS.json`,可用来核对搬运是否完整。 + +### 4.4 可选:HandFlow 与 ViPE 分支 + +只在跑单目 RGB 视频(没有深度、没有里程计)时需要。本链路的相机位姿来自 `prepare_rgbd_20260915.py` 的 RGB-D 里程计,只借用 ViPE 的**文件格式**喂给 Dyn-HaMR,不运行 ViPE。 + +- HandFlow:`hf download mxxu00/HandFlow handflow_denoiser.pt normalization_stats.npz --local-dir weights` +- ViPE 权重(本快照 `configs/pipeline/default.yaml` 所需):SAM ViT-B(`.torch_cache/hub/sam/`)、DeAOT(`.torch_cache/hub/aot/`)、GroundingDINO(HF `ShilongLiu/GroundingDINO`)、BERT(HF `google-bert/bert-base-uncased`)、UniDepth-L(HF `lpiccinelli/unidepth-v2-vitl14`)、DROID(`.torch_cache/hub/droid_slam/droid.pth`)、GeoCalib、Depth-Anything 系列。做法:设置 `configs/project_env.example.sh` 的缓存变量,联网跑一段短视频让它按需下载,全部就绪后再开 `HF_HUB_OFFLINE=1`。搬缓存要连 `snapshots/blobs/refs` 一起搬。 + +### 4.5 下载慢或中断 + +```bash +curl -L --range 0-1048575 --max-time 15 -o /dev/null -w 'HTTP=%{http_code} speed=%{speed_download} bytes/s\n' '' +``` + +Hugging Face 可换 `hf-mirror.com` 测速(第三方镜像)。支持 Range 时用 `curl -C -` 或 `aria2c -c -x 16 -s 16 --min-split-size=10M` 续传。拿到 HTML 错误页时文件名再对也不算成功。 + +## 5. 路径与环境变量 + +```bash +source configs/project_env.example.sh # TORCH_HOME / HF_HOME / WARP_CACHE_PATH / HAMER_CKPT / MANO_ROOT / DETECTOR_CKPT / MUJOCO_GL +source scripts/foundationpose_env.sh # .venv + nvcc + 清 ROS 干扰;跑 .venv 脚本前必 source +``` + +流程脚本自己的环境变量(`HF_*`、`SPIDER_*`)只改**输出**目录,序列名与输入路径写死在脚本里;换视频要改哪些常量见 [PIPELINE_LATEST.md §3](PIPELINE_LATEST.md)。 + +## 6. 启动前检查 + +```bash +source scripts/foundationpose_env.sh +python -c "import torch, pytorch3d, nvdiffrast.torch, mujoco, mujoco_warp, warp, trimesh, coacd, open3d, kornia, cv2; print(torch.cuda.is_available(), torch.__version__, mujoco.__version__)" +python scripts/verify_foundationpose_setup.py +for f in 2023-10-28-18-33-37 2024-01-11-20-02-45; do test -s third_party/FoundationPose/weights/$f/model_best.pth || echo "missing FoundationPose $f"; done +ls third_party/Dyn-HaMR/_DATA/hamer_ckpts third_party/Dyn-HaMR/_DATA/data/mano third_party/Dyn-HaMR/_DATA/hmp_model third_party/Dyn-HaMR/_DATA/BMC +test -s third_party/Dyn-HaMR/_DATA/data/mano/MANO_RIGHT.pkl +test -s third_party/Dyn-HaMR/third-party/hamer/pretrained_models/detector.pt +third_party/Dyn-HaMR/.dynhamr/bin/python -c "import torch, smplx, ultralytics; print(torch.__version__)" +.dex/bin/python -c "import dex_retargeting, pinocchio, mujoco, h5py; print('retarget ok', mujoco.__version__)" +.spider/bin/python -c "import torch, mujoco, mujoco_warp, warp, spider; print(torch.cuda.is_available(), mujoco.__version__)" +env PYTHONPATH= LD_LIBRARY_PATH= .spider/bin/python scripts/verify_spider_install.py +test -s third_party/l20_assets/L20/RIGHT/linkerhand_g20_right.urdf && test -s third_party/l20_assets/L20/LEFT/linkerhand_g20_left.urdf +test -s docs/20260915_171525/intrinsics.json && test -s docs/上半.stl && test -s docs/下半.stl +``` + +导入通过只是第一层;随后按 [PIPELINE_LATEST.md](PIPELINE_LATEST.md) 用 `--pilot` / 短段先跑一遍。 + +## 7. 源码包与上传 + +```bash +python3 scripts/package_source_release.py --output dist/hand-motion-pipeline-20260917 +``` + +产物 `dist/hand-motion-pipeline-20260917/` 与同名 `.tar.gz`。包内:`scripts/ docs/ configs/ requirements/ model/ utils/ preprocessing/ visualization/`、根文件,以及 6 个上游快照 `third_party/{hamer, vipe, Dyn-HaMR, spider, FoundationPose, nvdiffrast}`(工作树内容,含本地补丁,各自 LICENSE 保留)。清单:`UPSTREAM_SOURCES.json`(上游 commit / origin / 本地改动)、`EXTERNAL_ASSETS.json`(未打包资源哈希)、`EXCLUDED_FILES.json`、`PORTABILITY_REPORT.json`(写死的本机路径与序列名)、`RELEASE_VALIDATION.json`、`SHA256SUMS`。排除:权重、`_DATA`、缓存、venv、`output/`、编译产物、录制数据、`.git`。 + +```bash +tar xzf dist/hand-motion-pipeline-20260917.tar.gz && cd hand-motion-pipeline-20260917 +git init -b main && git add . && git commit -m "Import RGB-D hand-object pipeline: Dyn-HaMR, L20 retargeting, FoundationPose, SPIDER" +git remote add origin <仓库地址> && git push -u origin main +``` + +包里的 `.gitignore` 已把权重、缓存、录制数据、`output/` 挡在仓库外;大文件走平台附件或独立存储,按第 4 节路径恢复。 + +## 8. 已知限制 + +- 没有干净机器的端到端安装复现记录;版本表是本机观测。 +- SPIDER 在本机是 Python 3.11 兼容安装,上游声明 ≥ 3.12。 +- FoundationPose 权重来自社区镜像,未与官方文件比对。 +- 脚本序列名写死为 `20260915_171525`;`PORTABILITY_REPORT.json` 列出全部此类行。 +- 交付的是运动学参考与 SPIDER 物理审计结果;**未验证真机**,抓取未复现(见 PIPELINE_LATEST.md 现状)。 diff --git a/docs/上半.stl b/docs/上半.stl new file mode 100644 index 0000000..5c1df76 --- /dev/null +++ b/docs/上半.stl @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:d0565299f32944770e867a3c5714ed0e14468764efbd19a475a5bcae581c3ff5 +size 69684 diff --git a/docs/下半.stl b/docs/下半.stl new file mode 100644 index 0000000..e30a481 --- /dev/null +++ b/docs/下半.stl @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:ec647027c0f3a0827eb7cf6fc343f4d8adc53f9ecb7df662c8fc5035ed30ad1c +size 28984 diff --git a/requirements/pipeline_venv.txt b/requirements/pipeline_venv.txt new file mode 100644 index 0000000..2d907a7 --- /dev/null +++ b/requirements/pipeline_venv.txt @@ -0,0 +1,47 @@ +# Main environment (.venv) for the RGB-D pipeline: FoundationPose, RGB-D preprocessing, hand/object registration, +# collision fix / audits / rendering, Dyn-HaMR launcher and playback. Observed versions on the reference machine +# (Python 3.11.15, Ubuntu 22.04, CUDA 12.8) — pins, not a cross-platform lock. +# +# Install order: +# 1. torch==2.7.1 torchvision==0.22.1 --index-url https://download.pytorch.org/whl/cu128 (must match driver/CUDA) +# 2. pip install -r requirements/pipeline_venv.txt +# 3. pip install --no-build-isolation git+https://github.com/facebookresearch/pytorch3d.git@stable (needs nvcc; observed 0.7.9) +# 4. pip install --no-build-isolation --no-deps ./third_party/nvdiffrast (needs nvcc; snapshot 0.4.0) +# 5. build FoundationPose mycpp (see docs/SETUP_AND_WEIGHTS.md) +numpy==2.4.4 +scipy==1.17.1 +opencv-python==5.0.0.93 +trimesh==4.12.2 +coacd==1.0.14 +open3d==0.19.0 +kornia==0.8.3 +omegaconf==2.3.1 +hydra-core==1.3.2 +imageio==2.37.3 +imageio-ffmpeg==0.6.0 +pillow==12.2.0 +matplotlib==3.11.0 +pandas==3.0.5 +scikit-learn==1.9.1 +joblib==1.6.0 +h5py==3.16.0 +PyYAML==6.0.3 +ruamel.yaml==0.19.1 +transformations==2026.1.18 +pyrender==0.1.45 +PyOpenGL==3.1.0 +PyOpenGL-accelerate==3.1.10 +psutil==7.2.2 +tqdm==4.68.3 +requests==2.34.2 +mujoco==3.5.0 +mujoco-warp==3.5.0 +warp-lang==1.14.0 +glfw==2.10.0 +smplx==0.1.28 +einops==0.8.2 +timm==1.0.29 +huggingface-hub==0.28.1 +safetensors==0.8.0 +ninja==1.13.2 +pybind11==3.1.0 diff --git a/requirements/runtime_observed.json b/requirements/runtime_observed.json new file mode 100644 index 0000000..5a3c1c6 --- /dev/null +++ b/requirements/runtime_observed.json @@ -0,0 +1,66 @@ +{ + "handflow": { + "python": "3.11.15", + "torch": "2.7.1+cu128", + "torchvision": "0.22.1+cu128", + "numpy": "2.4.4", + "scipy": "1.17.1", + "mujoco": "3.5.0", + "mujoco-warp": "3.5.0", + "warp-lang": "1.14.0", + "pytorch3d": "0.7.9", + "ultralytics": "8.4.147" + }, + "dynhamr": { + "python": "3.10.12", + "torch": "2.7.0+cu128", + "torchvision": "0.22.0+cu128", + "numpy": "2.2.6", + "scipy": "1.15.3", + "ultralytics": "8.1.34" + }, + "retarget": { + "python": "3.11.15", + "torch": "2.7.1+cu128", + "torchvision": "0.22.1+cu128", + "numpy": "2.4.6", + "scipy": "1.17.1", + "mujoco": "3.5.0", + "mujoco-warp": "3.5.0", + "warp-lang": "1.14.0", + "dex-retargeting": "0.5.0", + "pytorch3d": "0.7.9", + "ultralytics": "8.4.147" + }, + "spider": { + "python": "3.11.15", + "torch": "2.7.1+cu128", + "torchvision": "0.22.1+cu128", + "numpy": "2.4.4", + "scipy": "1.17.1", + "mujoco": "3.7.0", + "mujoco-warp": "3.7.0.1", + "warp-lang": "1.12.1", + "ultralytics": "8.4.147" + }, + "pipeline_venv": { + "python": "3.11.15", + "torch": "2.7.1+cu128", + "torchvision": "0.22.1+cu128", + "pytorch3d": "0.7.9", + "nvdiffrast": "0.4.0 (third_party/nvdiffrast @ 253ac4f)", + "mujoco": "3.5.0", + "mujoco-warp": "3.5.0", + "warp-lang": "1.14.0", + "numpy": "2.4.4", + "scipy": "1.17.1", + "opencv-python": "5.0.0.93", + "trimesh": "4.12.2", + "coacd": "1.0.14", + "open3d": "0.19.0", + "kornia": "0.8.3", + "cuda_toolkit": "12.8 (project-local .cuda/usr/local/cuda-12.8)", + "foundationpose": "third_party/FoundationPose @ a1b694b, mycpp built", + "note": "same interpreter as the 'handflow' role; used by all scripts/ in docs/PIPELINE_LATEST.md except retarget (.dex) and SPIDER (.spider)" + } +} diff --git a/requirements/spider_installed_local.txt b/requirements/spider_installed_local.txt new file mode 100644 index 0000000..5f96705 --- /dev/null +++ b/requirements/spider_installed_local.txt @@ -0,0 +1,51 @@ +blinker==1.9.0 +comm==0.2.3 +configargparse==1.7.7 +daqp==0.9.1 +dash==4.4.1 +debugpy==1.8.21 +fastjsonschema==2.22.2 +flask==3.1.3 +google-auth==2.58.0 +google-auth-oauthlib==1.4.1 +gspread==6.2.1 +ipdb==0.13.13 +ipykernel==7.3.0 +ipywidgets==8.1.9 +itsdangerous==2.2.0 +janus==2.0.0 +joblib==1.6.0 +jsonschema==4.26.0 +jsonschema-specifications==2025.9.1 +jupyter-client==8.10.0 +jupyter-core==5.9.1 +jupyterlab-widgets==3.0.17 +loguru==0.7.3 +loop-rate-limiters==1.2.0 +mink==1.1.0 +mujoco==3.7.0 +mujoco-warp==3.7.0.1 +narwhals==2.26.0 +nbformat==5.11.1 +nest-asyncio==1.6.0 +nest-asyncio2==1.7.2 +oauthlib==3.3.1 +open3d==0.19.0 +plotly==7.0.0 +pyarrow==25.0.1 +pyasn1==0.6.4 +pyasn1-modules==0.4.2 +pymeshlab==2023.12.post3 +pyquaternion==0.9.9 +pyzmq==27.2.0 +qpsolvers==4.13.0 +referencing==0.37.0 +requests-oauthlib==2.0.0 +rerun-sdk==0.26.2 +retrying==1.4.2 +rpds-py==2026.6.3 +scikit-learn==1.9.1 +threadpoolctl==3.6.0 +tornado==6.5.8 +warp-lang==1.12.1 +widgetsnbextension==4.0.16 diff --git a/requirements/spider_resolved.txt b/requirements/spider_resolved.txt new file mode 100644 index 0000000..c723b2e --- /dev/null +++ b/requirements/spider_resolved.txt @@ -0,0 +1,557 @@ +# This file was autogenerated by uv via the following command: +# uv pip compile /tmp/spider_no_torch.txt --constraint /tmp/spider_existing_constraints.txt --python .spider/bin/python -o /tmp/spider_resolved.txt +absl-py==2.5.0 + # via + # -c /tmp/spider_existing_constraints.txt + # mujoco + # mujoco-warp +addict==2.4.0 + # via + # -c /tmp/spider_existing_constraints.txt + # open3d +annotated-types==0.7.0 + # via + # -c /tmp/spider_existing_constraints.txt + # pydantic +antlr4-python3-runtime==4.9.3 + # via + # -c /tmp/spider_existing_constraints.txt + # hydra-core + # omegaconf +anyio==4.15.1 + # via + # -c /tmp/spider_existing_constraints.txt + # httpx +asttokens==3.0.1 + # via + # -c /tmp/spider_existing_constraints.txt + # stack-data +attrs==26.1.0 + # via + # -c /tmp/spider_existing_constraints.txt + # jsonschema + # referencing + # rerun-sdk +blinker==1.9.0 + # via flask +certifi==2026.6.17 + # via + # -c /tmp/spider_existing_constraints.txt + # httpcore + # httpx + # requests +cffi==2.0.0 + # via + # -c /tmp/spider_existing_constraints.txt + # cryptography +charset-normalizer==3.4.7 + # via + # -c /tmp/spider_existing_constraints.txt + # requests +click==8.4.2 + # via + # -c /tmp/spider_existing_constraints.txt + # flask + # huggingface-hub +cloudpickle==3.1.2 + # via + # -c /tmp/spider_existing_constraints.txt + # joblib +comm==0.2.3 + # via + # dash + # ipykernel + # ipywidgets +configargparse==1.7.7 + # via open3d +contourpy==1.3.3 + # via + # -c /tmp/spider_existing_constraints.txt + # matplotlib +cryptography==49.0.0 + # via + # -c /tmp/spider_existing_constraints.txt + # google-auth +cycler==0.12.1 + # via + # -c /tmp/spider_existing_constraints.txt + # matplotlib +daqp==0.9.1 + # via qpsolvers +dash==4.4.1 + # via open3d +debugpy==1.8.21 + # via ipykernel +decorator==5.3.1 + # via + # -c /tmp/spider_existing_constraints.txt + # ipdb + # ipython +docstring-parser==0.18.0 + # via + # -c /tmp/spider_existing_constraints.txt + # tyro +etils==1.14.0 + # via + # -c /tmp/spider_existing_constraints.txt + # mujoco + # mujoco-warp +executing==2.2.1 + # via + # -c /tmp/spider_existing_constraints.txt + # stack-data +fastjsonschema==2.22.2 + # via nbformat +filelock==3.29.0 + # via + # -c /tmp/spider_existing_constraints.txt + # huggingface-hub +flask==3.1.3 + # via + # dash + # open3d +fonttools==4.63.0 + # via + # -c /tmp/spider_existing_constraints.txt + # matplotlib +fsspec==2026.4.0 + # via + # -c /tmp/spider_existing_constraints.txt + # etils + # huggingface-hub +glfw==2.10.0 + # via + # -c /tmp/spider_existing_constraints.txt + # mujoco +google-auth==2.58.0 + # via + # -r /tmp/spider_no_torch.txt + # google-auth-oauthlib + # gspread +google-auth-oauthlib==1.4.1 + # via gspread +gspread==6.2.1 + # via -r /tmp/spider_no_torch.txt +h11==0.16.0 + # via + # -c /tmp/spider_existing_constraints.txt + # httpcore +hf-xet==1.6.0 + # via + # -c /tmp/spider_existing_constraints.txt + # huggingface-hub +httpcore==1.0.9 + # via + # -c /tmp/spider_existing_constraints.txt + # httpx +httpx==0.28.1 + # via + # -c /tmp/spider_existing_constraints.txt + # huggingface-hub +huggingface-hub==1.31.0 + # via + # -c /tmp/spider_existing_constraints.txt + # -r /tmp/spider_no_torch.txt +hydra-core==1.3.2 + # via + # -c /tmp/spider_existing_constraints.txt + # -r /tmp/spider_no_torch.txt +idna==3.18 + # via + # -c /tmp/spider_existing_constraints.txt + # anyio + # httpx + # requests +imageio==2.37.3 + # via + # -c /tmp/spider_existing_constraints.txt + # -r /tmp/spider_no_torch.txt + # viser +imageio-ffmpeg==0.6.0 + # via + # -c /tmp/spider_existing_constraints.txt + # imageio +importlib-metadata==9.0.0 + # via + # -c /tmp/spider_existing_constraints.txt + # dash +ipdb==0.13.13 + # via -r /tmp/spider_no_torch.txt +ipykernel==7.3.0 + # via -r /tmp/spider_no_torch.txt +ipython==9.15.0 + # via + # -c /tmp/spider_existing_constraints.txt + # ipdb + # ipykernel + # ipywidgets + # mediapy +ipython-pygments-lexers==1.1.1 + # via + # -c /tmp/spider_existing_constraints.txt + # ipython +ipywidgets==8.1.9 + # via open3d +itsdangerous==2.2.0 + # via flask +janus==2.0.0 + # via dash +jedi==0.20.0 + # via + # -c /tmp/spider_existing_constraints.txt + # ipython +jinja2==3.1.6 + # via + # -c /tmp/spider_existing_constraints.txt + # flask +joblib==1.6.0 + # via scikit-learn +jsonschema==4.26.0 + # via nbformat +jsonschema-specifications==2025.9.1 + # via jsonschema +jupyter-client==8.10.0 + # via ipykernel +jupyter-core==5.9.1 + # via + # ipykernel + # jupyter-client + # nbformat +jupyterlab-widgets==3.0.17 + # via ipywidgets +kiwisolver==1.5.0 + # via + # -c /tmp/spider_existing_constraints.txt + # matplotlib +loguru==0.7.3 + # via -r /tmp/spider_no_torch.txt +loop-rate-limiters==1.2.0 + # via -r /tmp/spider_no_torch.txt +markdown-it-py==4.2.0 + # via + # -c /tmp/spider_existing_constraints.txt + # rich +markupsafe==3.0.3 + # via + # -c /tmp/spider_existing_constraints.txt + # flask + # jinja2 + # werkzeug +matplotlib==3.11.0 + # via + # -c /tmp/spider_existing_constraints.txt + # -r /tmp/spider_no_torch.txt + # mediapy + # open3d +matplotlib-inline==0.2.2 + # via + # -c /tmp/spider_existing_constraints.txt + # ipykernel + # ipython +mdurl==0.1.2 + # via + # -c /tmp/spider_existing_constraints.txt + # markdown-it-py +mediapy==1.2.7 + # via + # -c /tmp/spider_existing_constraints.txt + # -r /tmp/spider_no_torch.txt +mink==1.1.0 + # via -r /tmp/spider_no_torch.txt +msgspec==0.21.1 + # via + # -c /tmp/spider_existing_constraints.txt + # viser +mujoco==3.7.0 + # via + # -r /tmp/spider_no_torch.txt + # mink + # mujoco-warp +mujoco-warp==3.7.0.1 + # via -r /tmp/spider_no_torch.txt +narwhals==2.26.0 + # via + # plotly + # scikit-learn +nbformat==5.11.1 + # via open3d +nest-asyncio==1.6.0 + # via dash +nest-asyncio2==1.7.2 + # via ipykernel +numpy==2.4.4 + # via + # -c /tmp/spider_existing_constraints.txt + # -r /tmp/spider_no_torch.txt + # contourpy + # imageio + # matplotlib + # mediapy + # mujoco + # mujoco-warp + # open3d + # opencv-python + # pandas + # pymeshlab + # pyquaternion + # qpsolvers + # rerun-sdk + # scikit-learn + # scipy + # trimesh + # viser + # warp-lang +oauthlib==3.3.1 + # via requests-oauthlib +omegaconf==2.3.1 + # via + # -c /tmp/spider_existing_constraints.txt + # hydra-core +open3d==0.19.0 + # via -r /tmp/spider_no_torch.txt +opencv-python==5.0.0.93 + # via + # -c /tmp/spider_existing_constraints.txt + # -r /tmp/spider_no_torch.txt +packaging==26.2 + # via + # -c /tmp/spider_existing_constraints.txt + # huggingface-hub + # hydra-core + # ipykernel + # matplotlib + # plotly +pandas==3.0.5 + # via + # -c /tmp/spider_existing_constraints.txt + # open3d +parso==0.8.7 + # via + # -c /tmp/spider_existing_constraints.txt + # jedi +pexpect==4.9.0 + # via + # -c /tmp/spider_existing_constraints.txt + # ipython +pillow==12.2.0 + # via + # -c /tmp/spider_existing_constraints.txt + # imageio + # matplotlib + # mediapy + # open3d + # rerun-sdk +platformdirs==4.10.0 + # via + # -c /tmp/spider_existing_constraints.txt + # jupyter-core +plotly==7.0.0 + # via dash +prompt-toolkit==3.0.52 + # via + # -c /tmp/spider_existing_constraints.txt + # ipython +psutil==7.2.2 + # via + # -c /tmp/spider_existing_constraints.txt + # imageio + # ipykernel + # ipython +ptyprocess==0.7.0 + # via + # -c /tmp/spider_existing_constraints.txt + # pexpect +pure-eval==0.2.3 + # via + # -c /tmp/spider_existing_constraints.txt + # stack-data +pyarrow==25.0.1 + # via rerun-sdk +pyasn1==0.6.4 + # via pyasn1-modules +pyasn1-modules==0.4.2 + # via google-auth +pycparser==3.0 + # via + # -c /tmp/spider_existing_constraints.txt + # cffi +pydantic==2.13.4 + # via + # -c /tmp/spider_existing_constraints.txt + # dash +pydantic-core==2.46.4 + # via + # -c /tmp/spider_existing_constraints.txt + # pydantic +pygments==2.20.0 + # via + # -c /tmp/spider_existing_constraints.txt + # ipython + # ipython-pygments-lexers + # rich +pymeshlab==2023.12.post3 + # via -r /tmp/spider_no_torch.txt +pyopengl==3.1.0 + # via + # -c /tmp/spider_existing_constraints.txt + # mujoco +pyparsing==3.3.2 + # via + # -c /tmp/spider_existing_constraints.txt + # matplotlib +pyquaternion==0.9.9 + # via open3d +python-dateutil==2.9.0.post0 + # via + # -c /tmp/spider_existing_constraints.txt + # jupyter-client + # matplotlib + # pandas +pyyaml==6.0.3 + # via + # -c /tmp/spider_existing_constraints.txt + # huggingface-hub + # omegaconf + # open3d +pyzmq==27.2.0 + # via + # ipykernel + # jupyter-client +qpsolvers==4.13.0 + # via mink +referencing==0.37.0 + # via + # jsonschema + # jsonschema-specifications +requests==2.34.2 + # via + # -c /tmp/spider_existing_constraints.txt + # dash + # requests-oauthlib + # viser +requests-oauthlib==2.0.0 + # via google-auth-oauthlib +rerun-sdk==0.26.2 + # via -r /tmp/spider_no_torch.txt +retrying==1.4.2 + # via dash +rich==14.3.4 + # via + # -c /tmp/spider_existing_constraints.txt + # viser +rpds-py==2026.6.3 + # via + # jsonschema + # referencing +scikit-learn==1.9.1 + # via open3d +scipy==1.17.1 + # via + # -c /tmp/spider_existing_constraints.txt + # qpsolvers + # scikit-learn +setuptools==79.0.1 + # via + # -c /tmp/spider_existing_constraints.txt + # dash +six==1.17.0 + # via + # -c /tmp/spider_existing_constraints.txt + # python-dateutil +stack-data==0.6.3 + # via + # -c /tmp/spider_existing_constraints.txt + # ipython +threadpoolctl==3.6.0 + # via scikit-learn +tornado==6.5.8 + # via + # ipykernel + # jupyter-client +tqdm==4.68.3 + # via + # -c /tmp/spider_existing_constraints.txt + # huggingface-hub + # open3d + # viser +traitlets==5.15.1 + # via + # -c /tmp/spider_existing_constraints.txt + # ipykernel + # ipython + # ipywidgets + # jupyter-client + # jupyter-core + # matplotlib-inline + # nbformat +trimesh==4.12.2 + # via + # -c /tmp/spider_existing_constraints.txt + # viser +typeguard==4.5.2 + # via + # -c /tmp/spider_existing_constraints.txt + # tyro +typing-extensions==4.16.0 + # via + # -c /tmp/spider_existing_constraints.txt + # anyio + # dash + # etils + # huggingface-hub + # ipython + # jupyter-client + # mink + # pydantic + # pydantic-core + # referencing + # rerun-sdk + # typeguard + # typing-inspection + # tyro + # viser +typing-inspection==0.4.2 + # via + # -c /tmp/spider_existing_constraints.txt + # pydantic +tyro==1.0.15 + # via + # -c /tmp/spider_existing_constraints.txt + # -r /tmp/spider_no_torch.txt +urllib3==2.7.0 + # via + # -c /tmp/spider_existing_constraints.txt + # requests +viser==1.0.30 + # via + # -c /tmp/spider_existing_constraints.txt + # -r /tmp/spider_no_torch.txt +warp-lang==1.12.1 + # via + # -r /tmp/spider_no_torch.txt + # mujoco-warp +wcwidth==0.8.2 + # via + # -c /tmp/spider_existing_constraints.txt + # prompt-toolkit +websockets==16.0 + # via + # -c /tmp/spider_existing_constraints.txt + # viser +werkzeug==3.1.8 + # via + # -c /tmp/spider_existing_constraints.txt + # dash + # flask + # open3d +widgetsnbextension==4.0.16 + # via ipywidgets +zipp==4.1.0 + # via + # -c /tmp/spider_existing_constraints.txt + # etils + # importlib-metadata +zstandard==0.25.0 + # via + # -c /tmp/spider_existing_constraints.txt + # viser diff --git a/scripts/align_collision_floor.py b/scripts/align_collision_floor.py new file mode 100644 index 0000000..5dc7628 --- /dev/null +++ b/scripts/align_collision_floor.py @@ -0,0 +1,8 @@ +"""Remove initial CAD/table interpenetration before unassisted physics; record the offset.""" +from pathlib import Path +import numpy as np,mujoco,xml.etree.ElementTree as E,json +O=Path(__file__).resolve().parents[1]/'output/collision_fix_20260915';T=O/'datasets/processed/current/l20/bimanual/boxes';p=T/'scene_act.xml';m=mujoco.MjModel.from_xml_path(str(p));d=mujoco.MjData(m);d.qpos[:]=np.load(T/'0/trajectory_kinematic_act.npz')['qpos'][0];mujoco.mj_fwdPosition(m,d);z=[] +for g in range(m.ngeom): + if m.geom_contype[g]!=2:continue + mid=m.geom_dataid[g];v=m.mesh_vert[m.mesh_vertadr[mid]:m.mesh_vertadr[mid]+m.mesh_vertnum[mid]];world=v@d.geom_xmat[g].reshape(3,3).T+d.geom_xpos[g];z.append(world[:,2].min()) +height=float(min(z)-.0005);tree=E.parse(p);floor=tree.getroot().find("worldbody/geom[@name='right_floor']");old=float(floor.get('pos').split()[2]);floor.set('pos',f'0 0 {height}');tree.write(p);report={'old_height_m':old,'new_height_m':height,'offset_m':height-old,'reason':'No object/table initial overlap. Simulation alignment correction, not a new measured plane; object poses unchanged.'};(O/'floor_alignment.json').write_text(json.dumps(report,indent=2));print(report) diff --git a/scripts/audit_collision_fix.py b/scripts/audit_collision_fix.py new file mode 100644 index 0000000..07449bc --- /dev/null +++ b/scripts/audit_collision_fix.py @@ -0,0 +1,42 @@ +"""Independent before/after full-link collision, limits, continuity, and CAD surface probes.""" +import os +os.environ['OPENBLAS_NUM_THREADS']='1';os.environ['OMP_NUM_THREADS']='1' +from pathlib import Path +import json,numpy as np,mujoco,trimesh,open3d as o3d +from scipy.spatial.transform import Rotation as R +from red_box_distance import signed_distance +ROOT=Path(__file__).resolve().parents[1];O=Path(os.environ.get('HF_FIX_OUT',ROOT/'output/collision_fix_20260915'));OLD=Path(os.environ.get('HF_FIX_OLD',ROOT/'output/foundationpose_spider_20260915'));T=O/'datasets/processed/current/l20/bimanual/boxes';m=mujoco.MjModel.from_xml_path(str(T/'scene_act.xml'));d=mujoco.MjData(m);ref=np.load(O/'reference_video_rate.npz');before=np.load(OLD/'reference_video_rate.npz');q=ref['qpos'];sites=[m.site(s+'_hand_'+f+'_track').id for s in ['right','left'] for f in ['thumb','index','middle','ring','pinky']] +def audit(rows): + pen=[];gap=[];byhand=[[],[]];lim=[];mimic=[] + for f,row in enumerate(rows): + d.qpos[:]=row;mujoco.mj_fwdPosition(m,d);ds=[[],[]] + for c in d.contact: + g0,g1=map(int,c.geom) + if {int(m.geom_contype[g0]),int(m.geom_contype[g1])} not in ([{1,2},{1,4}] if os.environ.get('HF_TABLE','0')=='1' else [{1,2}]):continue + h=g0 if m.geom_contype[g0]==1 else g1;side=0 if m.geom(h).name.startswith('right_') else 1;ds[side].append(c.dist) + for j in range(2):byhand[j].append(max(0,-min(ds[j]))*1000 if ds[j] else 0.) + pen.append(max(byhand[0][-1],byhand[1][-1]));on=ref['contact'][f].astype(bool);gap.extend(np.linalg.norm(d.site_xpos[sites][on]-ref['contact_pos'][f][on],axis=1)*1000) + ids=np.flatnonzero(m.jnt_limited);v=row[m.jnt_qposadr[ids]];lim.append(max(np.maximum(m.jnt_range[ids,0]-v,0).max(),np.maximum(v-m.jnt_range[ids,1],0).max())) + for e in range(m.neq): + j1,j2=m.eq_obj1id[e],m.eq_obj2id[e];poly=m.eq_data[e,:5];x=row[m.jnt_qposadr[j2]];mimic.append(abs(row[m.jnt_qposadr[j1]]-sum(poly[k]*x**k for k in range(5)))) + result=dict(frames=len(rows),all_finite=bool(np.isfinite(rows).all()),max_hand_object_penetration_mm=float(max(pen)),median_frame_max_penetration_mm=float(np.median(pen)),frames_over_0_5mm=int(np.sum(np.array(pen)>.5)),inferred_contact_gap_mean_mm=float(np.mean(gap)),inferred_contact_gap_p95_mm=float(np.percentile(gap,95)),max_joint_limit_violation_rad=float(max(lim)),max_mimic_error_rad=float(max(mimic))) + for i,(side,st) in enumerate([('right',0),('left',27)]): + result[side]={'max_penetration_mm':float(max(byhand[i])),'max_wrist_step_mm':float(np.linalg.norm(np.diff(rows[:,st:st+3],axis=0),axis=1).max()*1000),'max_wrist_rotation_step_deg':float((R.from_euler('XYZ',rows[:-1,st+3:st+6]).inv()*R.from_euler('XYZ',rows[1:,st+3:st+6])).magnitude().max()*180/np.pi)} + return result,np.array(pen) +report={};report['before_same_new_collision_model'],bp=audit(before['qpos']);report['corrected_reference'],ap=audit(q) +report['object_qpos_unchanged']=bool(np.array_equal(q[:,-12:],before['qpos'][:,-12:]));report['wrist_correction_mm']={s:{'median':float(np.median(np.linalg.norm(q[:,j:j+3]-before['qpos'][:,j:j+3],axis=1))*1000),'max':float(np.max(np.linalg.norm(q[:,j:j+3]-before['qpos'][:,j:j+3],axis=1))*1000)} for s,j in [('right',0),('left',27)]} +# Sample the original visible link meshes, independently of the CoACD collision mesh. +probes=[] +for g in range(m.ngeom): + n=m.geom(g).name or '' + if '_visual_' not in n or not n.startswith(('right_','left_')):continue + mid=m.geom_dataid[g];v=m.mesh_vert[m.mesh_vertadr[mid]:m.mesh_vertadr[mid]+m.mesh_vertnum[mid]];faces=m.mesh_face[m.mesh_faceadr[mid]:m.mesh_faceadr[mid]+m.mesh_facenum[mid]];cent=v[faces].mean(1);pts=np.r_[v[np.linspace(0,len(v)-1,min(64,len(v)),dtype=int)],cent[np.linspace(0,len(cent)-1,min(64,len(cent)),dtype=int)]];probes.append((g,pts)) +report['cad_probe_scope']=f'{sum(len(p) for g,p in probes)} deterministic original-link surface samples per frame, all 352 frames; sampled test, not exhaustive mesh intersection.' +for name,side in [('upper','right'),('lower','left')]: + mesh=trimesh.load(O/'collision_v2'/f'{name}_closed.ply');assert mesh.is_watertight;sc=o3d.t.geometry.RaycastingScene();sc.add_triangles(o3d.t.geometry.TriangleMesh.from_legacy(o3d.geometry.TriangleMesh(o3d.utility.Vector3dVector(mesh.vertices),o3d.utility.Vector3iVector(mesh.faces))));obj=m.body(side+'_object').id;values=[] + for row in q: + d.qpos[:]=row;mujoco.mj_kinematics(m,d);points=np.concatenate([v@d.geom_xmat[g].reshape(3,3).T+d.geom_xpos[g] for g,v in probes]);local=(points-d.xpos[obj])@d.xmat[obj].reshape(3,3);dist=signed_distance(sc,mesh,local);values.append(max(0,-dist.min())*1000) + report[name+'_exact_CAD_surface_probes']={'max_inside_mm':float(max(values)),'frames_over_1mm':int(np.sum(np.array(values)>1))};print(name,'probe',report[name+'_exact_CAD_surface_probes'],flush=True) +if (T/'0/trajectory_mjwp_act.npz').exists() and np.load(T/'0/trajectory_mjwp_act.npz')['time'].max()>=ref['time'][-1]-.05: + a=np.load(T/'0/trajectory_mjwp_act.npz');rows=a['qpos'].reshape(-1,66);times=a['time'].ravel();ix=np.argmin(abs(times[:,None]-ref['time'][None,:]),axis=0);report['spider_physics'],_=audit(rows[ix]);report['spider_physics']['duration_s']=float(times[-1]);report['spider_physics']['steps']=len(rows) +report['interpolation']=json.loads((O/'interpolation_validation.json').read_text());(O/'validation.json').write_text(json.dumps(report,indent=2));np.savez_compressed(O/'collision_comparison.npz',before=before['qpos'],after=q,time=ref['time'],before_penetration_mm=bp,after_penetration_mm=ap);print(json.dumps(report,indent=2)) diff --git a/scripts/audit_collision_physics.py b/scripts/audit_collision_physics.py new file mode 100644 index 0000000..ad78a93 --- /dev/null +++ b/scripts/audit_collision_physics.py @@ -0,0 +1,16 @@ +"""Inspect every executed SPIDER state, using CPU collision detection independently.""" +import os +os.environ['OPENBLAS_NUM_THREADS']='1';os.environ['OMP_NUM_THREADS']='1' +from pathlib import Path +import numpy as np,mujoco,json +from scipy.spatial.transform import Rotation as R +ROOT=Path(__file__).resolve().parents[1];O=Path(os.environ.get('SPIDER_TASK_OUT',str(ROOT/'output/collision_fix_20260915')));T=O/'datasets/processed/current/l20/bimanual/boxes';m=mujoco.MjModel.from_xml_path(str(T/'scene_act.xml'));d=mujoco.MjData(m);dr=mujoco.MjData(m);a=np.load(T/'0/trajectory_mjwp_act.npz');ref=np.load(T/'0/trajectory_kinematic_act.npz');qref=ref['qpos'];fp=np.load(ROOT/'output/foundationpose_spider_20260915/foundationpose_objects.npz');q=a['qpos'].reshape(-1,66);times=a['time'].ravel();assert np.isfinite(q).all();pen=[];counts=[];activecounts=[];errors=[[],[]];angles=[[],[]] +for f,row in enumerate(q): + d.qpos[:]=row;mujoco.mj_fwdPosition(m,d);i=min(int(round(times[f]/.0025)),len(qref)-1);dr.qpos[:]=qref[i];mujoco.mj_kinematics(m,dr);cs=[c for c in d.contact if {int(m.geom_contype[c.geom[0]]),int(m.geom_contype[c.geom[1]])}=={1,2}];pen.append(max([max(0,-c.dist) for c in cs],default=0)*1000);counts.append(sum(c.dist<.001 for c in cs));activecounts.append(sum(c.dist1)),'steps_with_hand_object_contact':int(np.sum(np.array(counts)>0)),'steps_with_active_hand_object_constraint':int(np.sum(np.array(activecounts)>0)),'contact_count_definition':'within 1 mm; active constraints separately include configured contact activation distance','max_joint_limit_violation_rad':limits,'zero_object_actuator_gains':bool(not m.actuator_gainprm[-12:].any() and not m.actuator_biasprm[-12:].any())} +videoindex=np.minimum(np.round(times*30).astype(int),351) +for k,n in enumerate(['upper','lower']): + mask=fp[n+'_valid'][videoindex];ee=np.array(errors[k]);aa=np.array(angles[k]);report[n]={'observed_centroid_error_mean_mm':float(ee[mask].mean()),'observed_centroid_error_max_mm':float(ee[mask].max()),'observed_rotation_error_mean_deg':float(aa[mask].mean())} +report['grasp_tracking_20mm_passed']=all(report[n]['observed_centroid_error_max_mm']<20 for n in ['upper','lower']);(O/'physics_validation.json').write_text(json.dumps(report,indent=2));np.savez_compressed(O/'physics_motion.npz',qpos=q,ctrl=a['ctrl'].reshape(-1,56),time=times,max_penetration_mm=pen,contact_counts=counts,active_contact_counts=activecounts,object_centroid_error_mm=np.array(errors).T,object_rotation_error_deg=np.array(angles).T);print(json.dumps(report,indent=2)) diff --git a/scripts/audit_red_depth_metrics.py b/scripts/audit_red_depth_metrics.py new file mode 100644 index 0000000..ec86fa8 --- /dev/null +++ b/scripts/audit_red_depth_metrics.py @@ -0,0 +1,24 @@ +"""Independent distance sign audit and paired metric recomputation from saved geometry.""" +from pathlib import Path +import numpy as np,trimesh,open3d as o3d,json +from red_box_distance import signed_distance +ROOT=Path(__file__).resolve().parents[1];out=ROOT/'output/depth_ablation_red_20260916';a=np.load(out/'foreground_left_depth_comparison.npz');m=trimesh.load(out/'red_box_closed.ply',process=False) +scene=o3d.t.geometry.RaycastingScene();scene.add_triangles(o3d.t.geometry.TriangleMesh.from_legacy(o3d.geometry.TriangleMesh(o3d.utility.Vector3dVector(m.vertices),o3d.utility.Vector3iVector(m.faces)))) +rows=json.loads((out/'foreground_left_frame_metrics.json').read_text());report=json.loads((out/'comparison_metrics.json').read_text()) +for i in np.flatnonzero(a['paired']): + T=a['object_pose'][i];v=a['verts_before'][i];j=a['joints_before'][i];groups=[np.argsort(np.linalg.norm(v-j[k],axis=1))[:20] for k in [4,8,12,16,20]] + for label in ['before','after']: + local=(a['verts_'+label][i]-T[:3,3])@T[:3,:3];sd=signed_distance(scene,m,local) + gap=[float(np.min(np.abs(sd[g]))*1000) for g in groups] + rows[i][label]={'penetrating_vertex_fraction':float((sd<-.001).mean()),'max_vertex_penetration_mm':float(max(0,-sd.min())*1000),'nearest_fingertip_surface_gap_mm':min(gap),'fingertip_surface_gaps_mm':gap} + if i%60==0:print('audited',i,flush=True) +s=report['hands']['foreground_left'] +for label in ['before','after']: + for key in ['penetrating_vertex_fraction','max_vertex_penetration_mm','nearest_fingertip_surface_gap_mm']: + x=np.array([rows[i][label][key] for i in np.flatnonzero(a['paired'])]);s[label][key]={'median':float(np.median(x)),'p95':float(np.percentile(x,95)),'mean':float(np.mean(x))} +report['distance_sign']='Unsigned nearest surface distance with solid-angle winding number; outside bounding box explicitly exterior. Ray parity produced false inside classifications for this CAD and was replaced.' +# Geometry and data consistency invariants. +d=a['translation_camera'];assert np.max(np.abs(a['verts_after']-a['verts_before']-d[:,None,:]))<1e-6 +assert np.max(np.abs(a['joints_after']-a['joints_before']-d[:,None,:]))<1e-6 +assert np.isfinite(a['verts_after']).all();assert len(a['verts_before'])==315 +(out/'foreground_left_frame_metrics.json').write_text(json.dumps(rows,indent=2));(out/'comparison_metrics.json').write_text(json.dumps(report,indent=2));print(json.dumps(s,indent=2)) diff --git a/scripts/audit_smooth_table_result.py b/scripts/audit_smooth_table_result.py new file mode 100644 index 0000000..69984d9 --- /dev/null +++ b/scripts/audit_smooth_table_result.py @@ -0,0 +1,34 @@ +"""Audit filtered/projected reference for table and hand-object penetration.""" +from pathlib import Path +import json +import numpy as np +import mujoco + +ROOT = Path(__file__).resolve().parents[1] +O = ROOT / "output/desktop_smooth_fix_20260915" +z = np.load(O / "reference_video_rate.npz") +m = mujoco.MjModel.from_xml_path(str(O / "scene_act.xml")); d = mujoco.MjData(m) +floor = next(i for i in range(m.ngeom) if "_floor" in (m.geom(i).name or "")) +floor_z = float(m.geom_pos[floor][2]) +result = {"frames": len(z["qpos"]), "floor_z_m": floor_z, "max_table_penetration_mm": 0.0, + "max_hand_object_penetration_mm": 0.0, "frames_table_over_0_5mm": 0, + "frames_hand_object_over_0_5mm": 0} +for row in z["qpos"]: + d.qpos[:] = row; mujoco.mj_forward(m, d) + floor_min = {"right": 1.0, "left": 1.0} + for g in range(m.ngeom): + n = m.geom(g).name or "" + if not n.startswith(("right_", "left_")) or m.geom_dataid[g] < 0: + continue + side = n.split("_")[0]; mid = int(m.geom_dataid[g]); a = int(m.mesh_vertadr[mid]); b = a + int(m.mesh_vertnum[mid]) + v = m.mesh_vert[a:b: max(1, (b-a)//128)] + world = v @ d.geom_xmat[g].reshape(3,3).T + d.geom_xpos[g] + floor_min[side] = min(floor_min[side], float(world[:,2].min()-floor_z)) + tp = max(0.0, -min(floor_min.values())) * 1000 + hp = 0.0 + for c in d.contact: + a,b = int(c.geom[0]), int(c.geom[1]); ta,tb = int(m.geom_contype[a]),int(m.geom_contype[b]) + if {ta,tb} == {1,2}: hp = max(hp, max(0.0,-float(c.dist))*1000) + result["max_table_penetration_mm"] = max(result["max_table_penetration_mm"],tp); result["max_hand_object_penetration_mm"] = max(result["max_hand_object_penetration_mm"],hp) + result["frames_table_over_0_5mm"] += tp > .5; result["frames_hand_object_over_0_5mm"] += hp > .5 +(O/"smoothing_table_validation.json").write_text(json.dumps(result,indent=2)); print(json.dumps(result,indent=2)) diff --git a/scripts/audit_spider_dynamic_surfaces.py b/scripts/audit_spider_dynamic_surfaces.py new file mode 100644 index 0000000..54e5912 --- /dev/null +++ b/scripts/audit_spider_dynamic_surfaces.py @@ -0,0 +1,18 @@ +"""Check original visual hand surfaces against exact closed CAD in dynamic rollout.""" +import os +os.environ['OPENBLAS_NUM_THREADS']='1';os.environ['OMP_NUM_THREADS']='1' +from pathlib import Path +import json,numpy as np,mujoco,trimesh,open3d as o3d +from red_box_distance import signed_distance +ROOT=Path(__file__).resolve().parents[1];O=Path(os.environ.get('SPIDER_TASK_OUT',str(ROOT/'output/spider_dynamics_fix_20260915')));m=mujoco.MjModel.from_xml_path(str(O/'datasets/processed/current/l20/bimanual/boxes/scene_act.xml'));d=mujoco.MjData(m);a=np.load(O/'physics_motion.npz');t=np.load(O/'reference_video_rate.npz')['time'];ix=np.argmin(abs(a['time'][:,None]-t[None,:]),axis=0);q=a['qpos'][ix];probes=[] +for g in range(m.ngeom): + n=m.geom(g).name or '' + if '_visual_' not in n or not n.startswith(('right_','left_')):continue + mid=m.geom_dataid[g];v=m.mesh_vert[m.mesh_vertadr[mid]:m.mesh_vertadr[mid]+m.mesh_vertnum[mid]];faces=m.mesh_face[m.mesh_faceadr[mid]:m.mesh_faceadr[mid]+m.mesh_facenum[mid]];cent=v[faces].mean(1);pts=np.r_[v[np.linspace(0,len(v)-1,min(64,len(v)),dtype=int)],cent[np.linspace(0,len(cent)-1,min(64,len(cent)),dtype=int)]];probes.append((g,pts)) +report={'sampled_frames':len(q),'hand_surface_samples_per_frame':sum(len(p) for g,p in probes),'scope':'Original visible hand meshes vs closed CAD; deterministic surface sampling, not exhaustive intersection or continuous-time proof.'} +for name,side in [('upper','right'),('lower','left')]: + mesh=trimesh.load(ROOT/'output/collision_fix_20260915/collision_v2'/f'{name}_closed.ply');assert mesh.is_watertight;sc=o3d.t.geometry.RaycastingScene();sc.add_triangles(o3d.t.geometry.TriangleMesh.from_legacy(o3d.geometry.TriangleMesh(o3d.utility.Vector3dVector(mesh.vertices),o3d.utility.Vector3iVector(mesh.faces))));obj=m.body(side+'_object').id;values=[] + for row in q: + d.qpos[:]=row;mujoco.mj_kinematics(m,d);points=np.concatenate([v@d.geom_xmat[g].reshape(3,3).T+d.geom_xpos[g] for g,v in probes]);local=(points-d.xpos[obj])@d.xmat[obj].reshape(3,3);dist=signed_distance(sc,mesh,local);values.append(max(0,-dist.min())*1000) + report[name]={'max_inside_mm':float(max(values)),'frames_over_1mm':int(np.sum(np.array(values)>1))};print(name,report[name],flush=True) +(O/'dynamic_surface_validation.json').write_text(json.dumps(report,indent=2)) diff --git a/scripts/audit_yesterday_geometry.py b/scripts/audit_yesterday_geometry.py new file mode 100644 index 0000000..c3e8b06 --- /dev/null +++ b/scripts/audit_yesterday_geometry.py @@ -0,0 +1,24 @@ +"""Independent full-rate collision audit and exact closed-red-mesh landmark probes.""" +import os +os.environ.setdefault('MUJOCO_GL','osmesa') +from pathlib import Path +import json,numpy as np,mujoco,trimesh,open3d as o3d +from red_box_distance import signed_distance +ROOT=Path(__file__).resolve().parents[1];O=ROOT/'output/foundationpose_spider_20260915';T=O/'datasets/processed/current/l20/bimanual/boxes' +m=mujoco.MjModel.from_xml_path(str(T/'scene_act.xml'));d=mujoco.MjData(m);r=np.load(O/'comparison_motion.npz');q=r['spider_full_qpos'];pens=[];counts=[] +for row in q: + d.qpos[:]=row;mujoco.mj_fwdPosition(m,d) + selected=[c for c in d.contact if {int(m.geom_contype[c.geom[0]]),int(m.geom_contype[c.geom[1]])}=={1,2}] + pens.append(max([max(0.,-c.dist) for c in selected],default=0)*1000);counts.append(len(selected)) +mesh=trimesh.load(ROOT/'output/depth_ablation_red_20260916/red_box_closed.ply');assert mesh.is_watertight +sc=o3d.t.geometry.RaycastingScene();sc.add_triangles(o3d.t.geometry.TriangleMesh.from_legacy(o3d.geometry.TriangleMesh(o3d.utility.Vector3dVector(mesh.vertices),o3d.utility.Vector3iVector(mesh.faces)))) +ids=[i for i in range(m.nsite) if 'landmark_' in m.site(i).name or '_hand_' in m.site(i).name];assert len(ids)==42 +obj=m.body('right_object').id;report={} +for key in ['original','reference','spider']: + points=[] + for row in r[key]: + d.qpos[:]=row;mujoco.mj_kinematics(m,d);points.append((d.site_xpos[ids]-d.xpos[obj])@d.xmat[obj].reshape(3,3)) + dist=signed_distance(sc,mesh,np.array(points).reshape(-1,3)).reshape(-1,len(ids));report[key]={'frames_with_landmark_inside_red_over_1mm':int((dist.min(1)<-.001).sum()),'max_red_landmark_inside_depth_mm':float(max(0.,-dist.min())*1000)} +report['full_rate_spider_collision']={'steps':len(q),'max_hand_object_penetration_mm':float(max(pens)),'median_step_max_penetration_mm':float(np.median(pens)),'hand_object_contact_steps':int((np.array(counts)>0).sum())} +report['scope']='42 kinematic landmarks are point probes, not complete hand mesh intersection. Exact red sign uses winding on topology-repaired watertight CAD; full-rate contacts use CoACD approximation.' +(O/'geometry_audit.json').write_text(json.dumps(report,indent=2));np.savez_compressed(O/'full_rate_contacts.npz',time=r['spider_full_time'],max_penetration_mm=pens,contact_counts=counts);print(json.dumps(report,indent=2)) diff --git a/scripts/build_box_collision_v2.py b/scripts/build_box_collision_v2.py new file mode 100644 index 0000000..24fc134 --- /dev/null +++ b/scripts/build_box_collision_v2.py @@ -0,0 +1,24 @@ +"""Repair CAD T junctions without geometry changes, then decompose without a 20-hull cap.""" +from pathlib import Path +import numpy as np,trimesh,json,coacd +ROOT=Path(__file__).resolve().parents[1];O=ROOT/'output/collision_fix_20260915';C=O/'collision_v2';C.mkdir(parents=True,exist_ok=True) +report={};coacd.set_log_level('warn') +for name,file in [('upper','上半.stl'),('lower','下半.stl')]: + old=trimesh.load(ROOT/'docs'/file);m=old.copy();splits=0 + for iteration in range(100): + count=np.bincount(m.edges_unique_inverse);edges=m.edges_unique[count==1];changed=False;v=m.vertices + for a,b in edges: + ab=v[b]-v[a];t=(v-v[a])@ab/(ab@ab);distance=np.linalg.norm(v-(v[a]+t[:,None]*ab),axis=1);inside=np.flatnonzero((t>1e-7)&(t<1-1e-7)&(distance<1e-8)) + if len(inside)==0:continue + faces=m.faces.tolist();hit=[i for i,f in enumerate(faces) if a in f and b in f] + if len(hit)!=1:continue + face=faces.pop(hit[0]);k=next(k for k in range(3) if face[k] in [a,b] and face[(k+1)%3] in [a,b]);a,b,third=face[k],face[(k+1)%3],face[(k+2)%3];inside=sorted(inside,key=lambda x:np.linalg.norm(v[x]-v[a]));chain=[a,*inside,b];faces.extend([[int(x),int(y),int(third)] for x,y in zip(chain[:-1],chain[1:])]);m=trimesh.Trimesh(v.copy(),faces,process=False);splits+=1;changed=True;break + if not changed:break + assert np.array_equal(old.vertices,m.vertices) and abs(old.area-m.area)<1e-8 and abs(old.volume-m.volume)<1e-8 + print(name,'closed',m.is_watertight,'splits',splits,'boundary',int(np.sum(np.bincount(m.edges_unique_inverse)==1)),flush=True) + assert m.is_watertight and m.is_winding_consistent + m.export(C/(name+'_closed.ply')) + parts=coacd.run_coacd(coacd.Mesh(m.vertices,m.faces),threshold=.03,max_convex_hull=96,preprocess_mode='off',resolution=1500,mcts_nodes=10,mcts_iterations=60,mcts_max_depth=3,merge=True,seed=0) + for i,(v,f) in enumerate(parts):trimesh.Trimesh(v,f,process=False).export(C/f'{name}_{i:03d}.obj') + report[name]={'parts':len(parts),'watertight':bool(m.is_watertight),'t_junction_splits':splits,'surface_area_change':float(m.area-old.area),'volume_change':float(m.volume-old.volume),'cad_volume':float(m.volume),'convex_sum_volume':float(sum(trimesh.Trimesh(v,f,process=False).volume for v,f in parts)),'normalized_concavity_threshold':.03,'part_cap':96} + (C/'manifest.json').write_text(json.dumps(report,indent=2));print(name,report[name],flush=True) diff --git a/scripts/build_palm_collision_v2.py b/scripts/build_palm_collision_v2.py new file mode 100644 index 0000000..78b3d9b --- /dev/null +++ b/scripts/build_palm_collision_v2.py @@ -0,0 +1,11 @@ +from pathlib import Path +import xml.etree.ElementTree as E,trimesh,coacd,json +R=Path(__file__).resolve().parents[1];O=R/'output/collision_fix_20260915/hand_collision';O.mkdir(exist_ok=True) +r=E.parse(R/'output/foundationpose_spider_20260915/datasets/processed/current/l20/bimanual/boxes/scene_act.xml').getroot();report={};coacd.set_log_level('warn') +for x in r.findall('asset/mesh'): + n=x.get('name') + if not any(n==s+'_'+part+'_mesh_0' for s in ['right','left'] for part in ['hand_base_link','thumb_metacarpals']):continue + m=trimesh.load(x.get('file'));parts=coacd.run_coacd(coacd.Mesh(m.vertices,m.faces),threshold=.015,max_convex_hull=16,preprocess_mode='auto',preprocess_resolution=60,resolution=1000,mcts_nodes=10,mcts_iterations=40,mcts_max_depth=3,merge=True,seed=0) + for i,(v,f) in enumerate(parts):trimesh.Trimesh(v,f,process=False).export(O/f'{n}_{i:03d}.obj') + report[n]={'parts':len(parts),'original_hull_volume':float(m.convex_hull.volume),'parts_volume_sum':float(sum(trimesh.Trimesh(v,f,process=False).volume for v,f in parts)),'nonwatertight_source':not m.is_watertight} + (O/'manifest.json').write_text(json.dumps(report,indent=2));print(n,report[n],flush=True) diff --git a/scripts/check_spider_dynamics_replay.py b/scripts/check_spider_dynamics_replay.py new file mode 100644 index 0000000..e2ed694 --- /dev/null +++ b/scripts/check_spider_dynamics_replay.py @@ -0,0 +1,19 @@ +"""Compare saved controls in fresh GPU worlds and CPU, without optimization.""" +import os,json +from pathlib import Path +ROOT=Path(__file__).resolve().parents[1] +os.environ['WARP_CACHE_PATH']=str(ROOT/'.spider_cache/warp');os.environ['OMP_NUM_THREADS']='4' +import numpy as np,mujoco,mujoco_warp as mw,warp as wp +O=Path(os.environ.get('SPIDER_TASK_OUT',str(ROOT/'output/spider_dynamics_fix_20260915')));T=O/'datasets/processed/current/l20/bimanual/boxes' +a=np.load(T/'0/trajectory_mjwp_act.npz');r=np.load(T/'0/trajectory_kinematic_act.npz');rq,rv,rc=(r[k] for k in ['qpos','qvel','ctrl']);u=a['ctrl'].reshape(-1,56);expected=a['qpos'].reshape(-1,66) +m=mujoco.MjModel.from_xml_path(str(T/'scene_act.xml'));settings=json.loads((O/'physics_parameters.json').read_text());m.opt.iterations=settings['solver_iterations'];m.opt.ls_iterations=settings['ls_iterations'];m.opt.timestep=.0025;m.opt.integrator=mujoco.mjtIntegrator.mjINT_IMPLICITFAST;m.opt.o_solref[:]=[.02,1];m.opt.o_solimp[:]=[0,.95,.03,.5,2] +assert not m.actuator_gainprm[-12:].any() and not m.actuator_biasprm[-12:].any() +d=mujoco.MjData(m);d.qpos[:]=rq[0].astype(np.float32);d.qvel[:]=rv[0].astype(np.float32);d.ctrl[:]=rc[0].astype(np.float32);mujoco.mj_step(m,d) +wp.init();gpu=[];cpu=[] +with wp.ScopedDevice('cuda:0'): + wm=mw.put_model(m);wd=mw.put_data(m,d,nworld=64,nconmax=1024,njmax=3072) + with wp.ScopedCapture() as capture:mw.step(wm,wd) + for i,ctrl in enumerate(u): + wd.ctrl.assign(np.tile(ctrl.astype(np.float32),(64,1)));wp.capture_launch(capture.graph);wp.synchronize();gpu.append(wd.qpos.numpy()[0].copy());d.ctrl[:]=ctrl;mujoco.mj_step(m,d);cpu.append(d.qpos.copy()) +gpu=np.array(gpu);cpu=np.array(cpu);diff=np.abs(gpu-expected);reports={'steps':len(u),'all_finite':bool(np.isfinite(gpu).all() and np.isfinite(cpu).all()),'zero_object_assistance':True,'gpu_first_step_max_qpos_difference':float(diff[0].max()),'gpu_max_qpos_difference':float(diff.max()),'cpu_gpu_max_qpos_difference':float(np.abs(cpu-gpu).max()),'note':'qpos differences mix meters and radians; 64 fresh duplicate worlds, same model/initial state/controls'} +np.savez_compressed(O/'control_replay.npz',gpu=gpu,cpu=cpu,expected=expected,ctrl=u,time=a['time'].ravel());(O/'control_replay.json').write_text(json.dumps(reports,indent=2));print(json.dumps(reports,indent=2)) diff --git a/scripts/check_spider_state_restore.py b/scripts/check_spider_state_restore.py new file mode 100644 index 0000000..ff9c8ae --- /dev/null +++ b/scripts/check_spider_state_restore.py @@ -0,0 +1,19 @@ +"""Test whether speculative rollout state can affect the next executed step.""" +import os,sys,json +from pathlib import Path +ROOT=Path(__file__).resolve().parents[1];O=ROOT/'output/spider_dynamics_fix_20260915';T=O/'datasets/processed/current/l20/bimanual/boxes' +os.environ['WARP_CACHE_PATH']=str(ROOT/'.spider_cache/warp');sys.path.insert(0,str(ROOT/'third_party/spider')) +import numpy as np,torch,mujoco,warp as wp +from spider.config import Config +from spider.simulators import mjwp as s +c=Config(device='cuda:0',num_samples=64,nconmax_per_env=1024,njmax_per_env=3072,sim_dt=.0025,embodiment_type='bimanual');c.model_path=str(T/'scene_act.xml');c.object_actuator_ids=list(range(44,56));c.npair=0 +setup=s.setup_mj_model +def model(c): + m=setup(c);m.opt.iterations=80;return m +s.setup_mj_model=model +r=np.load(T/'0/trajectory_kinematic_act.npz');ref=tuple(torch.tensor(r[k],device='cuda:0',dtype=torch.float32) for k in ['qpos','qvel','ctrl','contact','contact_pos']);env=s.setup_env(c,ref);u=ref[2][0];state=s.save_state(env);s.step_env(c,env,u);wp.synchronize();expected=env.data_wp.qpos.numpy().copy();s.load_state(env,state) +for _ in range(80):s.step_env(c,env,u) +s.load_state(env,state);s.step_env(c,env,u);wp.synchronize();after=env.data_wp.qpos.numpy().copy();report={'state_only_max_qpos_difference':float(np.max(abs(expected-after)))} +s.load_state(env,state);params=s.save_env_params(c,env);s.load_env_params(c,env,{'kp':np.ones(12,dtype=np.float32)*10,'kd':np.ones(12,dtype=np.float32)}) +for _ in range(80):s.step_env(c,env,u) +s.load_state(env,state);s.load_env_params(c,env,params);s.step_env(c,env,u);wp.synchronize();after=env.data_wp.qpos.numpy().copy();report['state_and_parameters_max_qpos_difference']=float(np.max(abs(expected-after)));(O/'state_restore_check.json').write_text(json.dumps(report,indent=2));print(report) diff --git a/scripts/compare_red_hand_depth.py b/scripts/compare_red_hand_depth.py new file mode 100644 index 0000000..801b863 --- /dev/null +++ b/scripts/compare_red_hand_depth.py @@ -0,0 +1,75 @@ +"""Paired depth-only translation ablation; fixed hand articulation and object trajectory.""" +import os +os.environ.setdefault('OMP_NUM_THREADS','4') +import sys,json +from pathlib import Path +ROOT=Path(__file__).resolve().parents[1];sys.path.insert(0,str(ROOT));sys.path.insert(0,str(ROOT/'third_party/FoundationPose')) +import numpy as np,cv2,torch,trimesh,open3d as o3d,nvdiffrast.torch as dr +from Utils import nvdiffrast_render,make_mesh_tensors +from utils.mano_utils import MANOForwardKinematics +from red_box_distance import signed_distance +OUT=ROOT/'output/depth_ablation_red_20260916';SRC=ROOT/'docs/20260916_104026' +torch.set_num_threads(4) +meta=json.loads((SRC/'intrinsics.json').read_text());K=np.array([[meta['fx'],0,meta['cx']],[0,meta['fy'],meta['cy']],[0,0,1.]]) +obj=np.load(ROOT/'output/foundationpose_red_20260916/poses.npz');P=obj['T_camera_from_object'][:315];objvalid=obj['accepted'][:315] +mesh=trimesh.load(OUT/'red_box_closed.ply',process=False);assert mesh.is_watertight +scene=o3d.t.geometry.RaycastingScene();scene.add_triangles(o3d.t.geometry.TriangleMesh.from_legacy(o3d.geometry.TriangleMesh(o3d.utility.Vector3dVector(mesh.vertices),o3d.utility.Vector3iVector(mesh.faces)))) +ctx=dr.RasterizeCudaContext();eye=torch.eye(4,device='cuda')[None] +fk=MANOForwardKinematics(str(ROOT/'third_party/hamer/_DATA/data/mano'),torch.device('cpu')) +yy,xx=np.indices((480,848));train=((xx//8+yy//8)%2)==0 +all_data={};report={'design':'Same RGB-only HandFlow output, fixed STL/object pose, articulation/shape unchanged. Single robust camera-ray translation using depth. No added smoothing or interpolation. Fit and evaluation use disjoint 8x8 image checkerboard cells.','limits':['Visible skin selected with a color heuristic; correspondence and occlusion errors remain.','Held-out cells belong to the same depth sensor and frame, not independent ground truth.','Vertex signed distances do not detect every triangle-triangle collision.','Contact labels and force ground truth unavailable; nearest fingertip gap is not proof of contact.'],'hands':{}} +def render(v,faces): + mt={'pos':torch.as_tensor(v,device='cuda',dtype=torch.float32),'faces':torch.as_tensor(faces,device='cuda',dtype=torch.int32),'vnormals':torch.zeros((len(v),3),device='cuda'),'vertex_color':torch.ones((len(v),3),device='cuda')} + with torch.inference_mode():_,d,_=nvdiffrast_render(K=K,H=480,W=848,ob_in_cams=eye,glctx=ctx,mesh_tensors=mt) + return d[0].cpu().numpy() +def signed(v,T): + local=(v-T[:3,3])@T[:3,:3] + return signed_distance(scene,mesh,local) +def stat(x): + x=np.asarray(x);x=x[np.isfinite(x)];return {'median':float(np.median(x)),'p95':float(np.percentile(x,95)),'mean':float(np.mean(x))} if len(x) else None +for side in ['foreground_left']: + src=np.load(OUT/'left_mirrored'/'handflow_results.npz');v=src['verts_cam'];n=len(v);assert n==315 + pose=torch.tensor(src['pose']);betas=torch.tensor(src['betas']);trans=torch.tensor(src['trans']); + if betas.ndim==1:betas=betas[None].expand(n,-1) + elif len(betas)==1:betas=betas.expand(n,-1) + j=fk.joints(pose,betas,trans,['right']*n).numpy()/1000 + reproduced=fk.verts(pose,betas,trans,['right']*n).numpy()/1000;assert np.max(np.abs(reproduced-v))<1e-5 + # Restore mirrored-right geometry to physical left; reverse face winding under reflection. + v=v.copy();v[:,:,0]*=-1;j[:,:,0]*=-1 + faces=np.ascontiguousarray(src['faces'][:,[0,2,1]]) + corrected=v.copy();jc=j.copy();deltas=np.zeros((n,3));supported=np.zeros(n,bool);rows=[];cap=cv2.VideoCapture(str(SRC/'color.mp4')) + for t in range(n): + ok,b=cap.read();assert ok;z=cv2.imread(str(SRC/'depth'/f'{t:06d}.png'),-1).astype('float32')*meta['depth_scale_m'] + r={'frame':t,'detected':bool(src['pred_valid'][t]),'object_accepted':bool(objvalid[t]),'depth_supported':False} + if src['pred_valid'][t]: + d0=render(v[t],faces);skin=cv2.inRange(cv2.cvtColor(b,cv2.COLOR_BGR2YCrCb),np.array([0,133,77]),np.array([255,173,127]))>0 + hsv=cv2.cvtColor(b,cv2.COLOR_BGR2HSV);red=((hsv[:,:,0]<10)|(hsv[:,:,0]>170))&(hsv[:,:,1]>115) + skin &= ~red + interior=cv2.erode((d0>.1).astype('uint8'),np.ones((5,5),np.uint8))>0 + mask=skin&interior&(z>.1)&(z<.85) + fit=mask&train;errors=z[fit]-d0[fit];delta=float(np.median(errors)) if len(errors) else 0.;mad=float(np.median(np.abs(errors-delta))) if len(errors) else 1. + accept=len(errors)>=100 and mad<.025 and abs(delta)<.25 and j[t,0,2]>.1 + if accept: + deltas[t]=j[t,0]/j[t,0,2]*delta;corrected[t]+=deltas[t];jc[t]+=deltas[t];supported[t]=True + d1=render(corrected[t],faces) if accept else d0 + test=mask&(~train)&(d1>.1) + r.update(depth_supported=bool(accept),fit_pixels=int(fit.sum()),fit_mad_mm=mad*1000,shift_z_mm=float(deltas[t,2]*1000),heldout_pixels=int(test.sum()),depth_error_before_mm=float(np.median(np.abs(d0[test]-z[test]))*1000) if test.any() else None,depth_error_after_mm=float(np.median(np.abs(d1[test]-z[test]))*1000) if test.any() else None) + if objvalid[t]: + # Use nearest surface vertices to each MANO fingertip; preserve exactly the same vertices in A/B. + tips=[4,8,12,16,20];groups=[np.argsort(np.linalg.norm(v[t]-j[t,k],axis=1))[:20] for k in tips] + for label,verts in [('before',v[t]),('after',corrected[t])]: + sd=signed(verts,P[t]);gap=[float(np.min(np.abs(sd[g]))*1000) for g in groups] + r[label]={'penetrating_vertex_fraction':float((sd<-.001).mean()),'max_vertex_penetration_mm':float(max(0,-sd.min())*1000),'nearest_fingertip_surface_gap_mm':min(gap),'fingertip_surface_gaps_mm':gap} + rows.append(r) + if t%60==0:print(side,t,flush=True) + cap.release();paired=np.array([r['depth_supported'] and r['object_accepted'] and 'before' in r for r in rows]);held=[r for r in rows if r['depth_supported'] and r.get('heldout_pixels',0)>=100] + summary={'detected_frames':int(src['pred_valid'].sum()),'depth_supported_frames':int(supported.sum()),'paired_object_frames':int(paired.sum()),'heldout_depth_frames':len(held),'translation_norm_mm':stat(np.linalg.norm(deltas[supported],axis=1)*1000),'before':{},'after':{}} + triple=paired[:-2]&paired[1:-1]&paired[2:] + for label,joints in [('before',j),('after',jc)]: + summary[label]['heldout_depth_error_mm']=stat([r['depth_error_'+label+'_mm'] for r in held]) + for key in ['penetrating_vertex_fraction','max_vertex_penetration_mm','nearest_fingertip_surface_gap_mm']:summary[label][key]=stat([rows[t][label][key] for t in np.flatnonzero(paired)]) + second=np.diff(joints[:,0],n=2,axis=0)[triple];summary[label]['wrist_second_difference_rms_mm']=float(np.sqrt(np.mean(np.sum(second**2,axis=-1)))*1000) if len(second) else None + summary['jitter_valid_triples']=int(triple.sum());report['hands'][side]=summary + np.savez_compressed(OUT/f'{side}_depth_comparison.npz',verts_before=v,verts_after=corrected,joints_before=j,joints_after=jc,faces=faces,translation_camera=deltas,depth_supported=supported,detected=src['pred_valid'],paired=paired,K=K,object_pose=P,object_accepted=objvalid) + (OUT/f'{side}_frame_metrics.json').write_text(json.dumps(rows,indent=2)) +(OUT/'comparison_metrics.json').write_text(json.dumps(report,indent=2));print(json.dumps(report,indent=2)) diff --git a/scripts/configure_collision_physics.py b/scripts/configure_collision_physics.py new file mode 100644 index 0000000..c79364d --- /dev/null +++ b/scripts/configure_collision_physics.py @@ -0,0 +1,14 @@ +"""Apply the selected bounded-drive contact model after geometric preparation.""" +from pathlib import Path +import json,xml.etree.ElementTree as E +O=Path(__file__).resolve().parents[1]/'output/collision_fix_20260915';p=O/'datasets/processed/current/l20/bimanual/boxes/scene_act.xml';tree=E.parse(p);r=tree.getroot();r.find('option').set('iterations','80');r.find('option').set('ls_iterations','50') +for g in r.findall('.//geom'): + if g.get('contype') in ['1','2']:g.set('solimp','.999 .9999 .0001 .5 2');g.set('solref','.005 1') +for a in r.findall('actuator/position'): + n=a.get('name') + if '_object_' in n:continue + if '_hand_pos_' in n:kp,kv,limit=500,30,20 + elif '_hand_rot_' in n:kp,kv,limit=15,2,2 + else:kp,kv,limit=4,.15,.1 + a.attrib.update(kp=str(kp),kv=str(kv),forcerange=f'{-limit} {limit}') +tree.write(p);(O/'physics_parameters.json').write_text(json.dumps({'solver_iterations':80,'ls_iterations':50,'wrist_position_kp':500,'wrist_position_kv':30,'wrist_force_cap_N':20,'wrist_rotation_kp':15,'wrist_rotation_kv':2,'wrist_torque_cap_Nm':2,'finger_kp':4,'finger_kv':.15,'finger_torque_cap_Nm':.1,'contact_solimp':[.999,.9999,.0001,.5,2],'contact_solref':[.005,1],'note':'Simulation drive assumptions, not measured hardware ratings. Chosen after full-clip CPU control-replay comparison.'},indent=2)) diff --git a/scripts/configure_spider_stable_contact.py b/scripts/configure_spider_stable_contact.py new file mode 100644 index 0000000..30fdc5d --- /dev/null +++ b/scripts/configure_spider_stable_contact.py @@ -0,0 +1,9 @@ +"""Apply contact settings selected by the matched CPU/GPU short replay probe.""" +from pathlib import Path +import json,xml.etree.ElementTree as ET +O=Path(__file__).resolve().parents[1]/'output/spider_dynamics_fix_20260915';p=O/'datasets/processed/current/l20/bimanual/boxes/scene_act.xml';tree=ET.parse(p) +for g in tree.findall('.//geom'): + g.set('solimp','.9 .95 .001 .5 2');g.set('solref','.02 1') + if g.get('contype') in ['1','2']:g.set('margin','.004');g.set('gap','.002') +tree.write(p) +s=json.loads((O/'physics_parameters.json').read_text());s.update(contact_solimp=[.9,.95,.001,.5,2],contact_solref=[.02,1],contact_activation_distance_m=.002,note='Contact regularization and 2 mm activation distance selected from GPU/CPU short replay. Simulation parameters, not measured hardware.');(O/'physics_parameters.json').write_text(json.dumps(s,indent=2)) diff --git a/scripts/configure_spider_task.py b/scripts/configure_spider_task.py new file mode 100644 index 0000000..bdc20b1 --- /dev/null +++ b/scripts/configure_spider_task.py @@ -0,0 +1,29 @@ +"""Build a SPIDER task directory from a kinematic reference directory and apply the validated physics settings. +Env: SPIDER_SRC (reference dir with datasets/.../boxes, reference_video_rate.npz, config.json), SPIDER_TASK_OUT (target). +Settings follow output/spider_dynamics_fix_20260915/physics_parameters.json (simulation assumptions, not hardware).""" +import os, json, shutil, xml.etree.ElementTree as E +from pathlib import Path +ROOT = Path(__file__).resolve().parents[1]; SRC = Path(os.environ['SPIDER_SRC']); OUT = Path(os.environ['SPIDER_TASK_OUT']) +T = OUT / 'datasets/processed/current/l20/bimanual/boxes'; (T / '0').mkdir(parents=True, exist_ok=True) +for rel in ['datasets/processed/current/l20/bimanual/boxes/scene_act.xml', 'datasets/processed/current/l20/bimanual/boxes/task_info.json', 'datasets/processed/current/l20/bimanual/boxes/0/trajectory_kinematic_act.npz', 'reference_video_rate.npz']: + shutil.copy2(SRC / rel, OUT / rel) +cfg = json.loads((SRC / 'config.json').read_text()); cfg['dataset_dir'] = str(OUT / 'datasets'); cfg.update(json.loads(os.environ.get('SPIDER_CONFIG_OVERRIDES', '{}'))); (OUT / 'config.json').write_text(json.dumps(cfg, indent=2)) +for name in ['collision_v2', 'hand_collision']: + if not (OUT / name).exists(): os.symlink(os.path.relpath(ROOT / 'output/collision_fix_20260915' / name, OUT), OUT / name) +p = T / 'scene_act.xml'; tree = E.parse(p); r = tree.getroot() +r.find('option').set('iterations', '80'); r.find('option').set('ls_iterations', '50') +for g in r.findall('.//geom'): + g.set('solimp', '.9 .95 .001 .5 2'); g.set('solref', '.02 1') + if g.get('contype') in ['1', '2', '4']: g.set('margin', '.004'); g.set('gap', '.002') +for a in r.findall('actuator/position'): + n = a.get('name') + if '_object_' in n: continue + if '_hand_pos_' in n: kp, kv, limit = 500, 30, 20 + elif '_hand_rot_' in n: kp, kv, limit = 15, 2, 2 + else: kp, kv, limit = float(os.environ.get('HF_FINGER_KP', '4')), float(os.environ.get('HF_FINGER_KV', '.15')), float(os.environ.get('HF_FINGER_TORQUE', '.1')) + a.attrib.update(kp=str(kp), kv=str(kv), forcerange=f'{-limit} {limit}') +tree.write(p) +settings = {'solver_iterations': 80, 'ls_iterations': 50, 'wrist_position_kp': 500, 'wrist_position_kv': 30, 'wrist_force_cap_N': 20, 'wrist_rotation_kp': 15, 'wrist_rotation_kv': 2, 'wrist_torque_cap_Nm': 2, 'finger_kp': float(os.environ.get('HF_FINGER_KP', '4')), 'finger_kv': float(os.environ.get('HF_FINGER_KV', '.15')), 'finger_torque_cap_Nm': float(os.environ.get('HF_FINGER_TORQUE', '.1')), + 'contact_solimp': [.9, .95, .001, .5, 2], 'contact_solref': [.02, 1], 'contact_activation_distance_m': .002, 'table_collides_with_hands': True, + 'note': 'Same contact regularization and 2 mm activation as spider_dynamics_fix_20260915 (selected by GPU/CPU short replay); table now a hand collider. Simulation parameters, not measured hardware.', 'source_reference': str(SRC)} +(OUT / 'physics_parameters.json').write_text(json.dumps(settings, indent=2)); print('configured', OUT) diff --git a/scripts/constrain_rgbd_hand.py b/scripts/constrain_rgbd_hand.py new file mode 100644 index 0000000..5894475 --- /dev/null +++ b/scripts/constrain_rgbd_hand.py @@ -0,0 +1,70 @@ +"""Use observed skin-surface depth to correct camera-space hand translation. + +This is a robust translation fit, not depth-derived skeleton ground truth. +""" +from pathlib import Path +import json +import argparse +import numpy as np +import cv2 +from scipy.ndimage import gaussian_filter1d +from scipy.spatial.transform import Rotation + +ROOT=Path(__file__).resolve().parents[1];BASE=ROOT/'output/20260915_171525' +SRC=ROOT/'docs/20260915_171525' +parser=argparse.ArgumentParser();parser.add_argument('--left',action='store_true');args=parser.parse_args() +folder='handflow_left' if args.left else 'handflow' +suffix='_left' if args.left else '' +h=np.load(BASE/folder/'handflow_results.npz');j=dict(np.load(BASE/folder/'human_joints.npz')) +meta=json.loads((SRC/'intrinsics.json').read_text());camera=np.load(BASE/'rgbd_camera.npz') +K=np.array([meta[k] for k in ['fx','fy','cx','cy']]);N=len(h['pose']) +c2w=camera['c2w'];old_camera=h['c2w'] +wrist_cam=np.einsum('tji,tj->ti',old_camera[:,:3,:3],j['wrist_world']-old_camera[:,:3,3]) +shifts=[];records=[];valid=[];cap=cv2.VideoCapture(str(SRC/'color.mp4')) +for t,verts in enumerate(h['verts_cam']): + ok,color=cap.read();assert ok + dep=cv2.imread(str(SRC/'depth'/f'{t:06d}.png'),-1).astype(float)*meta['depth_scale_m'] + ycc=cv2.cvtColor(color,cv2.COLOR_BGR2YCrCb) + skin=cv2.inRange(ycc,np.array([0,133,77]),np.array([255,173,127]))>0 + uv=np.rint(verts[:,:2]/verts[:,2,None]*K[:2]+K[2:]).astype(int) + keep=(verts[:,2]>.05)&(uv[:,0]>=2)&(uv[:,0]=2)&(uv[:,1].1)&(patch<.85)] + if len(values)<5 or np.ptp(values)>.04:continue + errors.append(float(np.median(values)-verts[index,2])) + dz=float(np.median(errors)) if errors else 0. + mad=float(np.median(np.abs(np.asarray(errors)-dz))) if errors else 1. + accepted=bool(len(errors)>=25 and mad<.025 and abs(dz)<.25 and h['pred_valid'][t]) + shifts.append(dz);valid.append(accepted) + records.append(dict(frame=t,samples=len(errors),median_depth_residual_m=dz,mad_m=mad,accepted=accepted)) +cap.release();valid=np.asarray(valid);idx=np.flatnonzero(valid) +assert len(idx)>=N*.5, f'Insufficient depth support: {len(idx)}/{N}' +filled=np.interp(np.arange(N),idx,np.asarray(shifts)[idx]);smooth=gaussian_filter1d(filled,2) +delta=wrist_cam/wrist_cam[:,2,None]*smooth[:,None] +corrected=wrist_cam+delta +world=np.einsum('tij,tj->ti',c2w[:,:3,:3],corrected)+c2w[:,:3,3] +root_world=Rotation.from_matrix(c2w[:,:3,:3]@Rotation.from_rotvec(h['pose'][:,:3]).as_matrix()).as_rotvec() +j.update(wrist_world=world,root_orient=root_world,time=camera['time'], + before_depth_wrist_camera=wrist_cam,depth_translation_camera=delta, + depth_valid=valid,depth_correction_interpolated=~valid, + detection_valid=j['detection_valid']&valid&camera['valid'], + source=str((BASE/folder/'handflow_results.npz').resolve()), + metric_scale_provenance='D405 depth_scale from recording metadata; skin-surface translation fit, model size unchanged; RGBD static-background odometry world. Not calibrated motion ground truth.') +np.savez_compressed(BASE/f'human_joints_rgbd{suffix}.npz',**j) +np.savez_compressed(BASE/f'depth_hand_fit{suffix}.npz',raw_depth_shift=shifts,applied_depth_shift=smooth, + accepted=valid,wrist_camera=corrected,wrist_world=world,c2w=c2w, + corrected_verts_camera=h['verts_cam']+delta[:,None,:]) +report=dict(frames=N,depth_supported_frames=int(valid.sum()),rejected_depth_frames=np.flatnonzero(~valid).tolist(), + median_abs_raw_depth_residual_m=float(np.median(np.abs(np.asarray(shifts)[valid]))), + median_abs_remaining_sample_residual_m=float(np.median(np.abs(np.asarray(shifts)[valid]-smooth[valid]))), + applied_depth_shift_range_m=[float(smooth.min()),float(smooth.max())],frames_detail=records, + boundary='Translation-only fit to selected visible skin depth; self-occlusion and skin-mask errors remain. No finger-joint depth truth. Rejected depth corrections interpolated and marked invalid.') +(BASE/f'depth_fit_validation{suffix}.json').write_text(json.dumps(report,indent=2));print(json.dumps({k:v for k,v in report.items() if k!='frames_detail'},indent=2)) diff --git a/scripts/correct_dynhamr_rgbd.py b/scripts/correct_dynhamr_rgbd.py new file mode 100644 index 0000000..396ea2c --- /dev/null +++ b/scripts/correct_dynhamr_rgbd.py @@ -0,0 +1,80 @@ +"""Offline, rotation-safe smoothing and conservative visible-surface RGB-D fit.""" +import sys,json +from pathlib import Path +import numpy as np +import torch,cv2 +from scipy.spatial.transform import Rotation +from scipy.ndimage import gaussian_filter1d +ROOT=Path(__file__).resolve().parents[1] +sys.path.insert(0,str(ROOT/'third_party/Dyn-HaMR/dyn-hamr')) +from body_model import MANO,run_mano +BASE=ROOT/'output/20260915_171525_dynhamr'; OUT=BASE/'corrected'; OUT.mkdir(exist_ok=True) +SRC=ROOT/'docs/20260915_171525'; meta=json.loads((SRC/'intrinsics.json').read_text()) +a=dict(np.load(BASE/'optimization/prior/20260915_171525_000000_world_results.npz')); N=352 +torch.set_num_threads(4) +model=MANO(model_path=str(ROOT/'third_party/Dyn-HaMR/_DATA/data/mano'),batch_size=704,pose2rot=True) +def geometry(d): + with torch.no_grad(): + o=run_mano(model,*[torch.tensor(d[k]).float() for k in ['trans','root_orient','pose_body','is_right','betas']]) + return o['joints'].numpy(),o['vertices'].numpy() +def smooth_rotation(v,sigma=1.2): + shape=v.shape; v=v.reshape(N,-1,3); out=np.empty_like(v) + for j in range(v.shape[1]): + r=Rotation.from_rotvec(v[:,j]) + for t in range(N): + ix=np.arange(max(0,t-4),min(N,t+5)); w=np.exp(-.5*((ix-t)/sigma)**2) + out[t,j]=r[ix].mean(weights=w).as_rotvec() + return out.reshape(shape) +j0,v0=geometry(a); d={k:v.copy() for k,v in a.items()} +for b in range(2): + d['root_orient'][b]=smooth_rotation(a['root_orient'][b]); d['pose_body'][b]=smooth_rotation(a['pose_body'][b]) +# Correct translations in world coordinates; left MANO translation has reflected X. +R=a['cam_R'][0]; ct=a['cam_t'][0]; K=np.array([meta[k] for k in ['fx','fy','cx','cy']]) +vc=np.einsum('tij,btvj->btvi',R,v0)+ct[None,:,None,:] +wc=np.einsum('tij,btj->bti',R,j0[:,:,0])+ct[None,:,:] +raw=np.zeros((2,N)); valid=np.zeros((2,N),bool); counts=np.zeros((2,N),int) +cap=cv2.VideoCapture(str(SRC/'color.mp4')) +for t in range(N): + ok,c=cap.read(); assert ok + dep=cv2.imread(str(SRC/'depth'/f'{t:06d}.png'),-1)*meta['depth_scale_m'] + skin=cv2.inRange(cv2.cvtColor(c,cv2.COLOR_BGR2YCrCb),np.array([0,133,77]),np.array([255,173,127]))>0 + for b in range(2): + v=vc[b,t]; uv=np.rint(v[:,:2]/v[:,2,None]*K[:2]+K[2:]).astype(int); closest={} + for i in np.flatnonzero((v[:,2]>.1)&(uv[:,0]>=2)&(uv[:,0]<846)&(uv[:,1]>=2)&(uv[:,1]<478)): + x,y=uv[i]; key=(x//4,y//4) + if key not in closest or v[i,2].1)&(patch<.85)] + if len(z)>=5 and np.ptp(z)<.02:errors.append(np.median(z)-v[i,2]) + counts[b,t]=len(errors) + if errors: + dz=np.median(errors); mad=np.median(np.abs(np.array(errors)-dz));raw[b,t]=dz + valid[b,t]=len(errors)>=25 and mad<.01 and abs(dz)<.025 +cap.release() +# Only directly supported neighborhoods receive a depth update. No long-gap extrapolation. +shifts=np.zeros_like(raw) +for b in range(2): + numerator=gaussian_filter1d(raw[b]*valid[b],2); support=gaussian_filter1d(valid[b].astype(float),2) + shifts[b]=np.where(support>.6,numerator/np.maximum(support,1e-6),0) + shifts[b]=gaussian_filter1d(shifts[b],1) + delta_c=wc[b]/wc[b,:,2,None]*shifts[b,:,None] + delta_w=np.einsum('tji,tj->ti',R,delta_c) + delta_w[:,0]*=2*a['is_right'][b]-1 + d['trans'][b]=gaussian_filter1d(a['trans'][b]+delta_w,1.5,axis=0) +j1,v1=geometry(d) +report={'frames':N,'method':'Rotation weighted means on SO(3), sigma 1.2 frames; translation Gaussian sigma 1.5 frames; conservative skin-depth residual <=25mm and MAD<10mm, support-weighted local correction','offline_uses_future_frames':True,'depth_fit_is_surface_estimate_not_ground_truth':True,'hands':{}} +for b in range(2): + side='right' if a['is_right'][b,0]>.5 else 'left' + def rms2(x):return float(np.sqrt(np.mean(np.sum(np.diff(x,n=2,axis=0)**2,axis=-1)))*1000) + deviation=np.linalg.norm(j1[b,:,0]-j0[b,:,0],axis=-1)*1000 + angles=(Rotation.from_rotvec(a['root_orient'][b]).inv()*Rotation.from_rotvec(d['root_orient'][b])).magnitude()*180/np.pi + report['hands'][side]={'depth_accepted_frames':int(valid[b].sum()),'depth_shift_max_mm':float(abs(shifts[b]).max()*1000),'wrist_second_difference_rms_mm_before':rms2(j0[b,:,0]),'wrist_second_difference_rms_mm_after':rms2(j1[b,:,0]),'wrist_change_max_mm':float(deviation.max()),'wrist_change_p95_mm':float(np.percentile(deviation,95)),'root_change_max_deg':float(angles.max())} + assert np.isfinite(j1[b]).all() and deviation.max()<40 and angles.max()<15 +phase=OUT/'prior';phase.mkdir(exist_ok=True) +np.savez_compressed(phase/'20260915_171525_000000_world_results.npz',**d) +np.savez_compressed(OUT/'depth_fit.npz',raw_shift=raw,accepted=valid,applied_shift=shifts,sample_count=counts) +np.savez_compressed(OUT/'comparison_joints.npz',before=j0,after=j1) +(OUT/'validation.json').write_text(json.dumps(report,indent=2));print(json.dumps(report,indent=2)) diff --git a/scripts/decompose_yesterday_boxes.py b/scripts/decompose_yesterday_boxes.py new file mode 100644 index 0000000..f4e1714 --- /dev/null +++ b/scripts/decompose_yesterday_boxes.py @@ -0,0 +1,13 @@ +from pathlib import Path +import numpy as np,trimesh,coacd,json +ROOT=Path(__file__).resolve().parents[1];out=ROOT/'output/foundationpose_spider_20260915/collision';out.mkdir(exist_ok=True) +coacd.set_log_level('warn');report={} +for name,file in [('upper','上半.stl'),('lower','下半.stl')]: + m=trimesh.load(ROOT/'docs'/file) + if name=='upper':m=trimesh.load(ROOT/'output/depth_ablation_red_20260916/red_box_closed.ply',process=False) + print(name,'watertight',m.is_watertight,flush=True) + parts=coacd.run_coacd(coacd.Mesh(m.vertices,m.faces),threshold=.03,max_convex_hull=20,preprocess_mode='auto',preprocess_resolution=60,resolution=1500,mcts_nodes=10,mcts_iterations=60,mcts_max_depth=3,merge=True,seed=0) + for i,(v,f) in enumerate(parts):trimesh.Trimesh(v,f,process=False).export(out/f'{name}_{i:03d}.obj') + report[name]={'parts':len(parts),'mesh_volume_m3':m.volume,'hull_volume_sum_m3':sum(trimesh.Trimesh(v,f,process=False).volume for v,f in parts),'approximation':'CoACD convex decomposition, threshold .03; not exact CAD collision'} + print(name,report[name],flush=True) +(out/'manifest.json').write_text(json.dumps(report,indent=2)) diff --git a/scripts/diagnose_dynhamr_rgbd.py b/scripts/diagnose_dynhamr_rgbd.py new file mode 100644 index 0000000..426ba54 --- /dev/null +++ b/scripts/diagnose_dynhamr_rgbd.py @@ -0,0 +1,58 @@ +"""Read-only trajectory diagnosis; frame differences are not ground-truth errors.""" +import json +import sys +from pathlib import Path +import numpy as np +import torch +from scipy.spatial.transform import Rotation +ROOT = Path(__file__).resolve().parents[1] +sys.path.insert(0, str(ROOT/'third_party/Dyn-HaMR/dyn-hamr')) +from body_model import MANO, run_mano +from vis.tools import smooth_results +torch.set_num_threads(4) +BASE = ROOT/'output/20260915_171525_dynhamr' +OUT = BASE/'diagnosis' +OUT.mkdir(exist_ok=True) +def stats(x): + x=np.asarray(x) + return dict(rms=float(np.sqrt(np.mean(x*x))), p95=float(np.percentile(x,95)), maximum=float(np.max(x))) +def positions(x): + d=np.linalg.norm(np.diff(x,axis=0),axis=-1)*1000 + dd=np.linalg.norm(np.diff(x,n=2,axis=0),axis=-1)*1000 + return dict(step_mm=stats(d), second_difference_mm=stats(dd), top_step_frames=(np.argsort(d)[-8:][::-1]+1).tolist()) +def rotations(x): + r=Rotation.from_rotvec(x.reshape(-1,3)).as_matrix().reshape(x.shape[:-1]+(3,3)) + inc=r[1:]@np.swapaxes(r[:-1],-1,-2) + a=Rotation.from_matrix(inc.reshape(-1,3,3)).magnitude()*180/np.pi + return stats(a) +model=MANO(model_path=str(ROOT/'third_party/Dyn-HaMR/_DATA/data/mano'),batch_size=704,pose2rot=True) +report={} +saved={} +for stage in ['smooth_fit','prior']: + path=sorted((BASE/'optimization'/stage).glob('*world_results.npz'))[-1] + arr=np.load(path) + for filtered in [False,True]: + d={k:torch.from_numpy(arr[k].copy()).float() for k in ['trans','root_orient','pose_body','is_right','betas']} + if filtered: + d['root_orient'],d['pose_body'],d['betas'],d['trans']=smooth_results(d['root_orient'],d['pose_body'],d['betas'],d['is_right'],d['trans']) + with torch.no_grad(): + j=run_mano(model,d['trans'],d['root_orient'],d['pose_body'],d['is_right'],d['betas'])['joints'].numpy() + key=stage+('_render_filtered' if filtered else '_raw') + saved[key]=j + report[key]={} + for b in range(2): + side='right' if arr['is_right'][b,0]>.5 else 'left' + report[key][side]=dict(wrist_world=positions(j[b,:,0]),root_rotation_step_deg=rotations(d['root_orient'][b].numpy()),finger_rotation_step_deg=rotations(d['pose_body'][b].numpy().reshape(352,15,3))) + report[key]['relative_wrist']=positions(j[1,:,0]-j[0,:,0]) + if stage=='prior': + R=arr['cam_R'][0]; t=arr['cam_t'][0] + for b in range(2): + side='right' if arr['is_right'][b,0]>.5 else 'left' + report[key][side]['wrist_camera']=positions(np.einsum('tij,tj->ti',R,j[b,:,0])+t) +cam=np.load(ROOT/'output/20260915_171525/rgbd_camera.npz') +report['camera_position']=positions(cam['c2w'][:,:3,3]) +report['camera_rotation_step_deg']=rotations(Rotation.from_matrix(cam['c2w'][:,:3,:3]).as_rotvec()) +report['caveat']='Frame differences include true motion; not a causal decomposition or accuracy metric. Render filtering replicated for static views; source camera render excludes translation filtering.' +np.savez_compressed(OUT/'joints_diagnostic.npz',**saved,time=cam['time']) +(OUT/'metrics.json').write_text(json.dumps(report,indent=2)) +print(json.dumps(report,indent=2)) diff --git a/scripts/diagnose_yesterday_hand_depth.py b/scripts/diagnose_yesterday_hand_depth.py new file mode 100644 index 0000000..7908dc5 --- /dev/null +++ b/scripts/diagnose_yesterday_hand_depth.py @@ -0,0 +1,129 @@ +"""Held-out pixel depth residual of yesterday's (20260915_171525) Dyn-HaMR hand trajectory. + +Same method as scripts/compare_red_hand_depth.py: render MANO hand depth into the D405 +camera, select visible skin pixels, fit a single camera-ray translation on one half of an +8x8 checkerboard, evaluate the residual on the other half. No smoothing, no articulation change. +""" +import os, sys, json +os.environ.setdefault('OMP_NUM_THREADS', '4') +from pathlib import Path +ROOT = Path(__file__).resolve().parents[1] +sys.path.insert(0, str(ROOT / 'third_party/Dyn-HaMR/dyn-hamr')) +sys.path.insert(0, str(ROOT / 'third_party/FoundationPose')) +import numpy as np, cv2, torch, nvdiffrast.torch as dr +from body_model import MANO, run_mano +from Utils import nvdiffrast_render + +SRC = ROOT / 'docs/20260915_171525' +BASE = ROOT / 'output/20260915_171525_dynhamr' +PRIOR = BASE / 'corrected/prior/20260915_171525_000000_world_results.npz' +OUT = ROOT / 'output/hand_depth_diagnosis_20260915'; OUT.mkdir(exist_ok=True) +torch.set_num_threads(4) +meta = json.loads((SRC / 'intrinsics.json').read_text()) +K = np.array([[meta['fx'], 0, meta['cx']], [0, meta['fy'], meta['cy']], [0, 0, 1.]]) +H, W = 480, 848 +N = 352 + +p = dict(np.load(PRIOR)) +model = MANO(model_path=str(ROOT / 'third_party/Dyn-HaMR/_DATA/data/mano'), batch_size=2 * N, pose2rot=True) +with torch.no_grad(): + o = run_mano(model, *[torch.tensor(p[k]).float() for k in ['trans', 'root_orient', 'pose_body', 'is_right', 'betas']]) +J = o['joints'].numpy(); V = o['vertices'].numpy() # (2,N,J,3), (2,N,778,3) world, metres +faces = None +for attr in ['faces', 'faces_tensor']: + f = getattr(model, attr, None) + if f is None: f = getattr(getattr(model, 'bm', None), attr, None) + if f is not None: + faces = np.asarray(f.cpu() if torch.is_tensor(f) else f).astype(np.int32); break +assert faces is not None and faces.max() < V.shape[2], 'MANO faces not found' +R = p['cam_R'][0]; ct = p['cam_t'][0] +Vc = np.einsum('tij,btvj->btvi', R, V) + ct[None, :, None, :] +Jc = np.einsum('tij,btkj->btki', R, J) + ct[None, :, None, :] +sides = ['right' if p['is_right'][b, 0] > .5 else 'left' for b in range(2)] + +ctx = dr.RasterizeCudaContext(); eye = torch.eye(4, device='cuda')[None] +ft = torch.as_tensor(faces, device='cuda', dtype=torch.int32) +def render(v): + mt = {'pos': torch.as_tensor(v, device='cuda', dtype=torch.float32), 'faces': ft, + 'vnormals': torch.zeros((len(v), 3), device='cuda'), 'vertex_color': torch.ones((len(v), 3), device='cuda')} + with torch.inference_mode(): + _, d, _ = nvdiffrast_render(K=K, H=H, W=W, ob_in_cams=eye, glctx=ctx, mesh_tensors=mt) + return d[0].cpu().numpy() + +yy, xx = np.indices((H, W)); train = ((xx // 8 + yy // 8) % 2) == 0 +def stat(x): + x = np.asarray([v for v in x if v is not None], float); x = x[np.isfinite(x)] + return {'n': int(len(x)), 'median': float(np.median(x)), 'p5': float(np.percentile(x, 5)), 'p95': float(np.percentile(x, 95)), 'mean': float(np.mean(x))} if len(x) else None + +rows = {s: [] for s in sides}; deltas = np.zeros((2, N, 3)); supported = np.zeros((2, N), bool) +cap = cv2.VideoCapture(str(SRC / 'color.mp4')); keep = {0, 60, 120, 175, 240, 300, 351} +for t in range(N): + ok, bgr = cap.read(); assert ok + z = cv2.imread(str(SRC / 'depth' / f'{t:06d}.png'), -1).astype('float32') * meta['depth_scale_m'] + ycc = cv2.cvtColor(bgr, cv2.COLOR_BGR2YCrCb); hsv = cv2.cvtColor(bgr, cv2.COLOR_BGR2HSV) + skin = cv2.inRange(ycc, np.array([0, 133, 77]), np.array([255, 173, 127])) > 0 + red = ((hsv[:, :, 0] < 10) | (hsv[:, :, 0] > 170)) & (hsv[:, :, 1] > 115) + blue = (hsv[:, :, 0] > 95) & (hsv[:, :, 0] < 130) & (hsv[:, :, 1] > 115) + skin &= ~red & ~blue + d0 = [render(Vc[b, t]) for b in range(2)] + vis = [] + for b in range(2): + other = d0[1 - b] + occluded = (other > .1) & (other < d0[b]) + interior = cv2.erode((d0[b] > .1).astype('uint8'), np.ones((5, 5), np.uint8)) > 0 + mask = skin & interior & (z > .1) & (z < .85) & ~occluded + fit = mask & train; err = z[fit] - d0[b][fit] + delta = float(np.median(err)) if len(err) else 0.; mad = float(np.median(np.abs(err - delta))) if len(err) else 1. + wz = float(Jc[b, t, 0, 2]) + accept = len(err) >= 100 and mad < .025 and abs(delta) < .25 and wz > .1 + r = {'frame': t, 'side': sides[b], 'wrist_z_m': wz, 'mask_pixels': int(mask.sum()), 'fit_pixels': int(fit.sum()), + 'fit_mad_mm': mad * 1000, 'fit_shift_z_mm': delta * 1000, 'depth_supported': bool(accept), + 'passes_yesterday_gate': bool(len(err) >= 25 and mad < .010 and abs(delta) < .025)} + test0 = mask & ~train & (d0[b] > .1) + r['heldout_pixels'] = int(test0.sum()) + r['signed_residual_before_mm'] = float(np.median(z[test0] - d0[b][test0]) * 1000) if test0.any() else None + r['abs_residual_before_mm'] = float(np.median(np.abs(z[test0] - d0[b][test0])) * 1000) if test0.any() else None + d1 = d0[b] + if accept: + deltas[b, t] = Jc[b, t, 0] / Jc[b, t, 0, 2] * delta; supported[b, t] = True + d1 = render(Vc[b, t] + deltas[b, t]) + test1 = mask & ~train & (d1 > .1) + r['abs_residual_after_mm'] = float(np.median(np.abs(z[test1] - d1[test1])) * 1000) if test1.any() else None + r['translation_norm_mm'] = float(np.linalg.norm(deltas[b, t]) * 1000) + else: + r['abs_residual_after_mm'] = None; r['translation_norm_mm'] = None + rows[sides[b]].append(r); vis.append((d0[b], d1)) + if t in keep: + img = bgr.copy() + for b, col in enumerate([(255, 128, 0), (0, 200, 255)]): + for d, thick in [(vis[b][0], 1), (vis[b][1], 2)]: + cnts, _ = cv2.findContours((d > .1).astype('uint8'), cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE) + cv2.drawContours(img, cnts, -1, col, thick) + cv2.putText(img, f'frame {t} thin=before thick=after orange={sides[0]} yellow={sides[1]}', (8, 20), cv2.FONT_HERSHEY_SIMPLEX, .5, (255, 255, 255), 1) + cv2.imwrite(str(OUT / f'overlay_{t:04d}.jpg'), img) + if t % 50 == 0: print('frame', t, flush=True) +cap.release() + +summary = {'method': 'Same as depth_ablation_red_20260916: rendered MANO depth vs D405 depth on visible skin pixels; single camera-ray translation fit on checkerboard half, held-out on other half. Hands rendered jointly, mutual occlusion excluded.', + 'input': str(PRIOR.relative_to(ROOT)), 'frames': N, 'hands': {}} +seg = [(0, 117), (118, 235), (236, 351)] +for b, s in enumerate(sides): + rs = rows[s]; sup = [r for r in rs if r['depth_supported']] + summary['hands'][s] = { + 'depth_supported_frames': len(sup), + 'frames_passing_yesterday_gate': int(sum(r['passes_yesterday_gate'] for r in rs)), + 'fit_pixels': stat([r['fit_pixels'] for r in rs]), + 'fit_mad_mm': stat([r['fit_mad_mm'] for r in rs]), + 'wrist_z_m': stat([r['wrist_z_m'] for r in rs]), + 'heldout_abs_residual_before_mm': stat([r['abs_residual_before_mm'] for r in rs]), + 'heldout_signed_residual_before_mm': stat([r['signed_residual_before_mm'] for r in rs]), + 'heldout_abs_residual_after_mm': stat([r['abs_residual_after_mm'] for r in sup]), + 'fit_shift_z_mm': stat([r['fit_shift_z_mm'] for r in sup]), + 'translation_norm_mm': stat([r['translation_norm_mm'] for r in sup]), + 'frames_shift_over_25mm': int(sum(abs(r['fit_shift_z_mm']) > 25 for r in sup)), + 'shift_z_by_segment_mm': {f'{a}-{c}': stat([r['fit_shift_z_mm'] for r in sup if a <= r['frame'] <= c]) for a, c in seg}, + } +np.savez_compressed(OUT / 'hand_depth_fit.npz', translation_camera=deltas, depth_supported=supported, sides=np.array(sides), K=K) +(OUT / 'frame_metrics.json').write_text(json.dumps(rows, indent=1)) +(OUT / 'summary.json').write_text(json.dumps(summary, indent=2)) +print(json.dumps(summary, indent=2, ensure_ascii=False)) diff --git a/scripts/download_foundationpose_weights.py b/scripts/download_foundationpose_weights.py new file mode 100644 index 0000000..33b22f1 --- /dev/null +++ b/scripts/download_foundationpose_weights.py @@ -0,0 +1,33 @@ +"""Download and verify the FoundationPose checkpoints listed in configs/foundationpose_weights_manifest.json. + +Files go to third_party/FoundationPose/weights//{config.yml, model_best.pth}; resumable (Range requests), SHA-256 checked. +The manifest URLs point at a community Hugging Face mirror (official Google Drive quota was exceeded when the snapshot was made); +replace `url` entries with the official links if you have them. Usage: .venv/bin/python scripts/download_foundationpose_weights.py +""" +import hashlib, json, time +from pathlib import Path +import requests + +ROOT = Path(__file__).resolve().parents[1] +manifest = json.loads((ROOT / 'configs/foundationpose_weights_manifest.json').read_text()) +dest = ROOT / 'third_party/FoundationPose/weights' +for item in manifest['files']: + path = dest / item['path']; path.parent.mkdir(parents=True, exist_ok=True); tmp = path.with_suffix(path.suffix + '.part') + if path.exists() and path.stat().st_size == item['size'] and hashlib.sha256(path.read_bytes()).hexdigest() == item['sha256']: + print('ok ', item['path']); continue + for attempt in range(5): + try: + offset = tmp.stat().st_size if tmp.exists() else 0 + print(f'download {item["path"]} from byte {offset}', flush=True) + with requests.get(item['url'], headers={'Range': f'bytes={offset}-{item["size"] - 1}'}, stream=True, timeout=(20, 60)) as r: + r.raise_for_status() + if offset and r.status_code != 206: offset = 0 + with tmp.open('ab' if offset else 'wb') as f: + for chunk in r.iter_content(1024 * 1024): f.write(chunk) + if tmp.stat().st_size == item['size']: tmp.replace(path); break + except requests.RequestException as e: + print('retry:', type(e).__name__, flush=True); time.sleep(3) + assert path.exists() and path.stat().st_size == item['size'], f'incomplete: {path}' + digest = hashlib.sha256(path.read_bytes()).hexdigest(); assert digest == item['sha256'], f'sha256 mismatch: {path}' + print('verified', item['path'], flush=True) +(dest / 'download_manifest.json').write_text(json.dumps(manifest, indent=2)); print('all FoundationPose weights present in', dest) diff --git a/scripts/evaluate_yesterday_spider.py b/scripts/evaluate_yesterday_spider.py new file mode 100644 index 0000000..ba7b16b --- /dev/null +++ b/scripts/evaluate_yesterday_spider.py @@ -0,0 +1,56 @@ +"""Audit saved SPIDER states; render source, kinematic reference, and physical rollout.""" +import os +os.environ.setdefault('MUJOCO_GL','osmesa') +from pathlib import Path +import json,re,sys +import numpy as np,mujoco,cv2,imageio.v2 as imageio +from scipy.spatial.transform import Rotation as R +ROOT=Path(__file__).resolve().parents[1];OUT=ROOT/'output/foundationpose_spider_20260915';TASK=OUT/'datasets/processed/current/l20/bimanual/boxes';TRIAL=TASK/'0' +r=np.load(TRIAL/'trajectory_kinematic_act.npz');a=np.load(TRIAL/'trajectory_mjwp_act.npz');q=a['qpos'].reshape(-1,66);ctrl=a['ctrl'].reshape(-1,56);times=a['time'].ravel();ref=np.load(OUT/'reference_video_rate.npz');fp=np.load(OUT/'foundationpose_objects.npz') +assert np.isfinite(q).all(),'Nonfinite SPIDER result; cannot render as valid motion' +m=mujoco.MjModel.from_xml_path(str(TASK/'scene_act.xml'));d=mujoco.MjData(m);dr=mujoco.MjData(m);assert not m.actuator_gainprm[-12:].any() and not m.actuator_biasprm[-12:].any() +# Evaluate at video times, preserving actual step timestamps rather than nominal array indices. +ix=np.argmin(abs(times[:,None]-ref['time'][None,:]),axis=0);qv=q[ix];bv=np.load(OUT/'reference_before_contact_fit.npz')['qpos'];sid=[m.site(f'{s}_hand_{f}_track').id for s in ['right','left'] for f in ['thumb','index','middle','ring','pinky']] +def metrics(rows): + pen=[];contacts=[];gaps=[];err=[[],[]];centererr=[[],[]];ang=[[],[]];lim=[];mim=[] + for f,row in enumerate(rows): + d.qpos[:]=row;mujoco.mj_forward(m,d);dr.qpos[:]=ref['qpos'][f];mujoco.mj_kinematics(m,dr) + selected=[c for c in d.contact if {int(m.geom_contype[c.geom[0]]),int(m.geom_contype[c.geom[1]])}=={1,2}] + pen.append(max([max(0.,-c.dist) for c in selected],default=0)*1000);contacts.append(len(selected)) + mask=ref['contact'][f].astype(bool);ds=np.linalg.norm(d.site_xpos[sid]-ref['contact_pos'][f],axis=1);gaps.extend(ds[mask]*1000) + for k,s in enumerate(['right','left']): + oid=m.body(s+'_object').id;cs=m.site(s+'_object_track').id;centererr[k].append(np.linalg.norm(d.site_xpos[cs]-dr.site_xpos[cs])*1000);err[k].append(np.linalg.norm(d.xpos[oid]-dr.xpos[oid])*1000);ang[k].append(R.from_matrix(d.xmat[oid].reshape(3,3)@dr.xmat[oid].reshape(3,3).T).magnitude()*180/np.pi) + inds=np.flatnonzero(m.jnt_limited);v=row[m.jnt_qposadr[inds]];lim.append(max(np.maximum(m.jnt_range[inds,0]-v,0).max(),np.maximum(v-m.jnt_range[inds,1],0).max())) + for j in range(m.neq): + j1,j2=m.eq_obj1id[j],m.eq_obj2id[j];coef=m.eq_data[j,:5];expected=sum(coef[k]*row[m.jnt_qposadr[j2]]**k for k in range(5));mim.append(abs(row[m.jnt_qposadr[j1]]-expected)) + result=dict(finite=bool(np.isfinite(rows).all()),sampled_video_frames=len(rows),hand_object_contact_frames=int(np.sum(np.array(contacts)>0)),max_hand_object_penetration_mm=float(np.max(pen)),median_frame_max_penetration_mm=float(np.median(pen)),inferred_contact_target_gap_mean_mm=float(np.mean(gaps)) if gaps else None,max_joint_limit_violation_rad=float(max(lim)),max_mimic_residual_rad=float(max(mim))) + for k,name in enumerate(['upper','lower']): + mask=fp[name+'_valid'];e=np.array(err[k]);aa=np.array(ang[k]);result[name]=dict(observed_frames=int(mask.sum()),observed_position_error_mean_mm=float(e[mask].mean()),observed_position_error_max_mm=float(e[mask].max()),observed_rotation_error_mean_deg=float(aa[mask].mean()),all_frames_position_error_mean_mm=float(e.mean()),observed_centroid_error_mean_mm=float(np.array(centererr[k])[mask].mean()),observed_centroid_error_max_mm=float(np.array(centererr[k])[mask].max())) + return result,np.array(pen),np.array(contacts) +report={};report['original_hands_new_objects'],_,_=metrics(bv);report['kinematic_contact_warmstart'],_,_=metrics(ref['qpos']);report['spider'],pen,counts=metrics(qv) +if (OUT/'independent_gpu_replay.npz').exists(): + replay=np.load(OUT/'independent_gpu_replay.npz') + for key in replay.files: + rows=replay[key][ix] + report['independent_gpu_'+key]=metrics(rows)[0] if np.isfinite(rows).all() else {'finite':False} + report['independent_replay_integrity']=json.loads((OUT/'independent_gpu_replay.json').read_text()) +report.update(full_steps=len(q),duration_s=float(times[-1]),zero_object_assistance=True,validity_note='Video-depth consistency is not external pose ground truth; lower reference after176 is held and excluded as a new target; either hand may contact red.',collision_note='Penetration measured using approximate convex-decomposed collision meshes, not exact CAD signed distance.',contact_note='Masks are geometric hypotheses, not measured contact labels.',all_states_finite=bool(np.isfinite(q).all())) +log=(OUT/'spider_full.log').read_text();report['completion_marker']='SPIDER_RUN_COMPLETE' in log;report['nonfinite_candidate_warning_count']=len(re.findall('NaNs or infs in rews',log)) +report['tracking_20mm_gate_passed']=all(report['spider'][n]['observed_position_error_max_mm']<20 for n in ['upper','lower']) +(OUT/'validation.json').write_text(json.dumps(report,indent=2));np.savez_compressed(OUT/'comparison_motion.npz',original=bv,reference=ref['qpos'],spider=qv,time=ref['time'],spider_full_qpos=q,spider_full_ctrl=ctrl,spider_full_time=times) +np.savetxt(OUT/'spider_trajectory.csv',np.c_[times,q,ctrl],delimiter=',',header=','.join(['time_s']+[m.joint(i).name for i in range(m.njnt)]+['ctrl_'+m.actuator(i).name for i in range(m.nu)]),comments='') +if '--skip-render' in sys.argv: + print(json.dumps(report,indent=2));sys.exit(0) +m.vis.headlight.ambient[:]=.7;m.vis.quality.offsamples=1;renderer=mujoco.Renderer(m,height=360,width=640);opt=mujoco.MjvOption();opt.sitegroup[:]=0;opt.geomgroup[3:]=0;cid=m.camera('source_camera').id +cap=cv2.VideoCapture(str(ROOT/'docs/20260915_171525/color.mp4'));writer=imageio.get_writer(OUT/'original_foundationpose_spider.mp4',fps=30,codec='libx264',quality=8,macro_block_size=1) +for f in range(len(qv)): + ok,img=cap.read();assert ok;panels=[cv2.cvtColor(cv2.resize(img,(640,360)),cv2.COLOR_BGR2RGB)] + cam=ref['camera_world_from_cv'][f];m.cam_pos[cid]=cam[:3,3];quat=R.from_matrix(cam[:3,:3]@np.diag([1,-1,-1])).as_quat();m.cam_quat[cid]=quat[[3,0,1,2]] + for row in [ref['qpos'][f],qv[f]]: + d.qpos[:]=row;mujoco.mj_forward(m,d);renderer.update_scene(d,camera='source_camera',scene_option=opt);panels.append(renderer.render().copy()) + panel=np.concatenate(panels,1);cv2.rectangle(panel,(0,0),(1920,45),(20,25,30),-1) + for j,label in enumerate(['Recorded RGB-D','FoundationPose + contact IK reference','Actual SPIDER: free objects, zero assistance']):cv2.putText(panel,label,(j*640+8,18),cv2.FONT_HERSHEY_SIMPLEX,.47,(255,255,255),1) + label=f'frame {f}/351 | red observed={bool(fp["upper_valid"][f])}, blue observed={bool(fp["lower_valid"][f])} | SPIDER penetration={pen[f]:.1f}mm, contacts={counts[f]}' + cv2.putText(panel,label,(8,38),cv2.FONT_HERSHEY_SIMPLEX,.45,(255,255,255),1);writer.append_data(panel) + if f in [0,80,160,240,351]:cv2.imwrite(str(OUT/f'comparison_{f:04d}.jpg'),cv2.cvtColor(panel,cv2.COLOR_RGB2BGR)) +writer.close();renderer.close();cap.release();print(json.dumps(report,indent=2),flush=True) diff --git a/scripts/export_camera_delivery.py b/scripts/export_camera_delivery.py new file mode 100644 index 0000000..8af7525 --- /dev/null +++ b/scripts/export_camera_delivery.py @@ -0,0 +1,214 @@ +"""Package the 1333-frame replay cameras, source cameras and frame mapping.""" +import os +os.environ.setdefault('MUJOCO_GL', 'osmesa') +import csv +import hashlib +import json +from pathlib import Path +import shutil +import subprocess +import tarfile + +import cv2 +import mujoco +import numpy as np +from scipy.spatial.transform import Rotation + +ROOT = Path(__file__).resolve().parents[1] +OUT = ROOT/'output/camera_delivery_1333' + + +def save_json(path, value): + path.write_text(json.dumps(value, ensure_ascii=False, indent=2)+'\n') + + +def pose(R, t): + result = np.broadcast_to(np.eye(4), (*np.asarray(R).shape[:-2],4,4)).copy() + result[..., :3,:3] = R + result[..., :3,3] = t + return result + + +def probe(path): + return json.loads(subprocess.check_output([ + 'ffprobe','-v','error','-select_streams','v:0','-show_streams', + '-show_frames','-show_entries', + 'stream=width,height,avg_frame_rate,time_base,nb_frames,start_time,duration:frame=best_effort_timestamp,best_effort_timestamp_time', + '-of','json',str(path)])) + + +def main(): + OUT.mkdir(exist_ok=True) + replay = ROOT/'output/l20_full_replay' + original = Path('/home/timessage/下载/2047635068.mp4') + final = np.load(replay/'motion.npz') + temporal = np.load(ROOT/'output/l20_2047635068/jitter_audit/temporal/motion.npz') + reconstruction = np.load(ROOT/'output/results_2047635068/world_results.npz') + vipe_k = np.load(ROOT/'output/vipe_2047635068/intrinsics/2047635068.npz') + vipe_pose = np.load(ROOT/'output/vipe_2047635068/pose/2047635068.npz') + N = len(final['qpos']) + assert N == 1333 and np.array_equal(vipe_k['inds'],np.arange(N)) + assert np.array_equal(vipe_pose['inds'],np.arange(N)) + video_meta = {name:probe(path) for name,path in { + 'original':original,'full_replay':replay/'full_replay.mp4', + 'comparison':replay/'original_vs_mujoco.mp4'}.items()} + for value in video_meta.values(): + assert len(value['frames']) == N + timestamps = {name:np.array([float(x['best_effort_timestamp_time']) for x in value['frames']]) + for name,value in video_meta.items()} + with (OUT/'frame_mapping.csv').open('w') as f: + writer=csv.writer(f) + writer.writerow(['hdf5_frame_0based','source_video_frame_0based','source_extracted_jpg', + 'source_pts_ticks','source_pts_s','hdf5_time_s','full_replay_pts_s','comparison_pts_s']) + for i in range(N): + writer.writerow([i,i,f'{i+1:06d}.jpg',video_meta['original']['frames'][i]['best_effort_timestamp'], + timestamps['original'][i],final['time'][i],timestamps['full_replay'][i],timestamps['comparison'][i]]) + # Reconstruct the fixed world transforms from the actual build script. + source_R = Rotation.from_quat(temporal['wrist_quat_wxyz'][:,[1,2,3,0]]) + bottle_R = source_R * Rotation.from_euler('x',-np.pi/2) + bottle_p = temporal['wrist_pos'] + source_R.apply(np.tile([.12,-.065,.15],(N,1))) + align = bottle_R[600].inv() + shift = np.array([0,0,.3])-align.apply(bottle_p[600]) + before_floor = align.apply(temporal['wrist_pos'])+shift+final['correction'][:,16:19] + residual = final['wrist_pos']-before_floor + floor_shift = residual.mean(axis=0) + assert np.max(np.abs(residual-floor_shift)) < 1e-9 + assert np.max(np.abs(floor_shift[:2])) < 1e-9 + final_from_temporal = pose(align.as_matrix(),shift+floor_shift) + temporal_from_dyn = pose(temporal['scene_rotation'],temporal['scene_translation']) + final_from_dyn = final_from_temporal @ temporal_from_dyn + track = int(np.load(ROOT/'output/dex_2047635068/human_joints.npz')['source_track']) + camera_from_dyn = pose(reconstruction['cam_R'][track],reconstruction['cam_t'][track]) + camera_from_final = camera_from_dyn @ np.linalg.inv(final_from_dyn) + final_from_camera = np.linalg.inv(camera_from_final) + # world_results cam_t already contains world_scale; do not scale twice. + K4 = reconstruction['intrins'].astype(float) + K = np.array([[K4[0],0,K4[2]],[0,K4[1],K4[3]],[0,0,1.]]) + np.savez_compressed(OUT/'source_camera.npz', + frame_index=np.arange(N), time=final['time'], intrinsics_fx_fy_cx_cy=vipe_k['data'], + K=K, camera_from_final_world=camera_from_final, + final_world_from_camera=final_from_camera, + camera_from_dynhamr_world=camera_from_dyn, + final_world_from_dynhamr_world=final_from_dyn, + vipe_world_from_camera_raw=vipe_pose['data']) + with (OUT/'source_intrinsics.csv').open('w') as f: + w=csv.writer(f);w.writerow(['frame_0based','fx','fy','cx','cy','width','height']) + for i,row in enumerate(vipe_k['data']):w.writerow([i,*row,1280,720]) + with (OUT/'source_extrinsics.csv').open('w') as f: + w=csv.writer(f);w.writerow(['frame_0based']+[f'camera_from_final_world_{i}{j}' for i in range(4) for j in range(4)]) + for i,T in enumerate(camera_from_final):w.writerow([i,*T.ravel()]) + source_config = dict(width=1280,height=720,fps=30,K=K.tolist(), + model='estimated pinhole',calibration='ViPE/Dyn-HaMR estimate, not measured calibration', + distortion_coefficients=None,undistortion='No explicit lens undistortion found in this inference path; capture-device processing unknown.', + preprocessing='Input video and extracted JPG are 1280x720; no full-image crop/resize observed in those inputs. Network-internal crops/resizes do not redefine this full-image K.', + camera_axes='OpenCV: +X right, +Y down, +Z forward; pixel coordinates u right, v down', + world_axes='Final scene.xml right-handed display world, Z display up; physical gravity not calibrated', + transform_convention='column vectors: p_camera = camera_from_final_world @ p_final_world; inverse is final_world_from_camera', + world_scale_already_applied=float(reconstruction['world_scale'].item()), + intrinsics_max_variation=float(np.ptp(vipe_k['data'],axis=0).max()), + warning='Contact registration changes hand motion per frame. This camera transform preserves the fixed world change, not inverse contact fitting. Exact original-video hand overlap is not guaranteed.') + save_json(OUT/'source_camera.json',source_config) + save_json(OUT/'world_transforms.json',dict(final_world_from_dynhamr_world=final_from_dyn.tolist(), + temporal_world_from_dynhamr_world=temporal_from_dyn.tolist(), + final_world_from_temporal_world=final_from_temporal.tolist(), + recovered_floor_translation=floor_shift.tolist(), + per_frame_contact_correction_included_in_camera=False, + derivation='scripts/build_l20_full_replay.py: alignment using bottle orientation at frame 600, shift to [0,0,.3], then floor clearance. Floor shift recovered from saved wrist minus fixed transform minus saved contact correction, verified constant across all frames.')) + # Recover actual mono renderer cameras, including the moving close-up lookat. + m=mujoco.MjModel.from_xml_path(str(replay/'scene.xml'));d=mujoco.MjData(m) + option=mujoco.MjvOption();option.sitegroup[:]=0;option.geomgroup[3:]=0 + scene=mujoco.MjvScene(m,maxgeom=10000) + cameras=[] + for distance,azimuth,elevation in [(1.05,135,-23),(.62,35,-12)]: + c=mujoco.MjvCamera();c.lookat[:]=final['camera_center'];c.distance=distance;c.azimuth=azimuth;c.elevation=elevation;cameras.append(c) + arrays={name:[] for name in ['wide_world_from_camera_cv','close_world_from_camera_cv','close_lookat','wide_gl_pos','close_gl_pos']} + for row in final['qpos']: + d.qpos[:]=row;mujoco.mj_forward(m,d) + cameras[1].lookat[:]=(d.xpos[m.body('hand_base_link').id]+d.xpos[m.body('right_object').id])/2 + cameras[1].lookat[2]+=.06 + arrays['close_lookat'].append(cameras[1].lookat.copy()) + for name,c in zip(['wide','close'],cameras): + mujoco.mjv_updateScene(m,d,option,None,c,mujoco.mjtCatBit.mjCAT_ALL,scene) + gl=mujoco.mjv_averageCamera(scene.camera[0],scene.camera[1]) + forward=gl.forward.astype(float);forward/=np.linalg.norm(forward) + right=np.cross(forward,gl.up);right/=np.linalg.norm(right) + up=np.cross(right,forward) + arrays[name+'_world_from_camera_cv'].append(pose(np.column_stack([right,-up,forward]),gl.pos)) + arrays[name+'_gl_pos'].append(gl.pos.copy()) + arrays={k:np.asarray(v) for k,v in arrays.items()} + fy=720/(2*np.tan(np.deg2rad(float(m.vis.global_.fovy))/2)) + render_K=np.array([[fy,0,320],[0,fy,360],[0,0,1.]]) + np.savez_compressed(OUT/'render_cameras.npz',frame_index=np.arange(N),time=final['time'], + K_panel=render_K,**arrays, + wide_camera_from_world_cv=np.linalg.inv(arrays['wide_world_from_camera_cv']), + close_camera_from_world_cv=np.linalg.inv(arrays['close_world_from_camera_cv'])) + save_json(OUT/'render_camera_config.json',dict(mujoco_version=mujoco.__version__, + panel_width=640,panel_height=720,full_video_width=1280,full_video_height=720, + vertical_fov_degrees=float(m.vis.global_.fovy),K_panel=render_K.tolist(), + pixel_convention='K uses continuous image-edge coordinates, pixel centers at (i+0.5,j+0.5); subtract 0.5 from cx/cy for integer pixel-center indexing.', + wide=dict(lookat=final['camera_center'].tolist(),distance=1.05,azimuth=135,elevation=-23,panel_x=0), + close=dict(lookat='(hand_base_link.xpos + right_object.xpos)/2 + [0,0,.06], per frame',distance=.62,azimuth=35,elevation=-12,panel_x=640), + stereo='monoscopic, average of MjvScene left/right GL cameras', + option=dict(sitegroup='all zero',geomgroup_3_and_above='zero'), + banners=dict(top_y=[0,64],bottom_y=[690,720]), + comparison_video=dict(width=1600,height=720, + original_panel='source scaled 1280x720 -> 960x540, offset [0,90]', + render_panel='full_replay crop x=640,y=64,w=640,h=626, then pad y=64 and place x=960', + source_image_to_comparison=[[.75,0,0],[0,.75,90],[0,0,1]], + close_panel_to_comparison=[[1,0,960],[0,1,0],[0,0,1]]))) + # Check time correspondence, coordinate algebra and a rendered frame. + sample_frames=[0,160,600,1332] + pixel_errors=[] + cap=cv2.VideoCapture(str(original)) + for i in sample_frames: + cap.set(cv2.CAP_PROP_POS_FRAMES,i);ok,im=cap.read();assert ok + jpg=cv2.imread(str(ROOT/f'third_party/Dyn-HaMR/test/images/2047635068/{i+1:06d}.jpg')) + assert jpg.shape==im.shape + pixel_errors.append(float(np.mean(np.abs(im.astype(float)-jpg.astype(float))))) + cap.release() + renderer=mujoco.Renderer(m,height=720,width=640) + render_errors=[] + cap=cv2.VideoCapture(str(replay/'full_replay.mp4')) + for i in [0,600,1332]: + d.qpos[:]=final['qpos'][i];mujoco.mj_forward(m,d) + cameras[1].lookat[:]=arrays['close_lookat'][i] + panels=[] + for c in cameras: + renderer.update_scene(d,camera=c,scene_option=option);panels.append(renderer.render().copy()) + generated=np.concatenate(panels,axis=1) + cap.set(cv2.CAP_PROP_POS_FRAMES,i);ok,video=cap.read();assert ok + video=cv2.cvtColor(video,cv2.COLOR_BGR2RGB) + error=float(np.mean(np.abs(generated[65:689].astype(float)-video[65:689].astype(float)))) + render_errors.append(error) + if i==0:cv2.imwrite(str(OUT/'render_check_frame_0000.png'),cv2.cvtColor(generated,cv2.COLOR_RGB2BGR)) + renderer.close();cap.release() + assert max(render_errors)<8,render_errors + assert max(pixel_errors)<8,pixel_errors + points=np.column_stack([np.linspace(0,.1,N),np.ones(N)*.2,np.ones(N),np.ones(N)]) + lhs=np.einsum('tij,tj->ti',camera_from_dyn,points) + rhs=np.einsum('tij,tj->ti',camera_from_final,points@final_from_dyn.T) + validation=dict(frames=N,source_to_hdf5_time_max_error_s=float(np.max(np.abs(timestamps['original']-final['time']))), + world_transform_roundtrip_max_error=float(np.max(np.abs(lhs-rhs))), + floor_shift_consistency_max_error_m=float(np.max(np.abs(residual-floor_shift))), + source_jpg_sample_frames=sample_frames,source_jpg_mean_absolute_pixel_error=pixel_errors, + rerender_checked_frames=[0,600,1332],rerender_mean_absolute_pixel_error=render_errors, + cameras_finite=bool(np.isfinite(camera_from_final).all()), + calibration_is_measured=False,source_distortion_calibration_available=False) + assert validation['source_to_hdf5_time_max_error_s']<1e-6 + assert validation['world_transform_roundtrip_max_error']<1e-8 + save_json(OUT/'validation.json',validation) + save_json(OUT/'video_metadata.json',{k:v['streams'][0] for k,v in video_meta.items()}) + target=OUT/'replay';target.mkdir(exist_ok=True) + for name in ['scene.xml','motion.npz','full_replay.mp4','original_vs_mujoco.mp4']: + shutil.copy2(replay/name,target/name) + shutil.copytree(replay/'assets',target/'assets',dirs_exist_ok=True) + shutil.copy2(original,OUT/'original_2047635068.mp4') + shutil.copy2(ROOT/'scripts/play_l20_full.py',OUT/'play_l20_full.py') + shutil.copy2(replay/'make_comparison.sh',OUT/'make_comparison_original.sh') + shutil.copy2(ROOT/'scripts/build_l20_full_replay.py',OUT/'build_l20_full_replay_reference.py') + shutil.copy2(ROOT/'output/hdf5_delivery/right_1333/demonstrations_right.hdf5',OUT/'demonstrations_right.hdf5') + print(json.dumps(validation,indent=2)) + + +if __name__ == '__main__': + main() diff --git a/scripts/export_dynhamr_bimanual_rgbd.py b/scripts/export_dynhamr_bimanual_rgbd.py new file mode 100644 index 0000000..408c3ad --- /dev/null +++ b/scripts/export_dynhamr_bimanual_rgbd.py @@ -0,0 +1,18 @@ +"""Export corrected Dyn-HaMR world-oriented, wrist-relative joints for both L20 hands.""" +from pathlib import Path +import json,shutil,numpy as np +ROOT=Path(__file__).resolve().parents[1]; BASE=ROOT/'output/20260915_171525_dynhamr' +source=BASE/'corrected/prior/20260915_171525_000000_world_results.npz' +a=np.load(source); joints=np.load(BASE/'corrected/comparison_joints.npz')['after']; camera=np.load(ROOT/'output/20260915_171525/rgbd_camera.npz') +assert joints.shape==(2,352,21,3) and np.isfinite(joints).all() +for b in range(2): + side='right' if a['is_right'][b,0]>.5 else 'left' + assert np.all(a['is_right'][b]==a['is_right'][b,0]) + track=BASE/f'dataset/dynhamr/track_preds/20260915_171525/{b:03d}' + valid=np.array([(track/f'{t+1:06d}_keypoints.json').is_file() for t in range(352)]) + np.savez_compressed(BASE/f'human_joints_{side}.npz',joints=joints[b]-joints[b,:,:1],wrist_world=joints[b,:,0],root_orient=np.zeros((352,3)),fps=30.,time=camera['time'],detection_valid=valid,source=str(source),side=side,coordinate_note='World-oriented wrist-relative joints from exact Dyn-HaMR run_mano, including left reflection; root rotation identity because joint directions already carry world orientation.',valid_semantics='Upstream keypoint file available; not per-joint confidence or depth acceptance.',depth_correction_applied=False) + print(side,'frames',len(joints[b]),'track available',valid.sum()) +for name in ['rgbd_camera.npz','object_poses.npz','object_fit_validation.json']: + shutil.copy2(ROOT/'output/20260915_171525'/name,BASE/name) +shutil.copytree(ROOT/'output/20260915_171525/object_assets',BASE/'object_assets',dirs_exist_ok=True) +(BASE/'object_assets/colors.json').write_text(json.dumps({'upper':{'source':'上半.stl','rgba':[.8,.08,.05,1]},'lower':{'source':'下半.stl','rgba':[.05,.15,.65,1]},'note':'STL geometry is colorless; colors stored in replay scene.xml material/rgba.'},ensure_ascii=False,indent=2)) diff --git a/scripts/fit_rgbd_objects.py b/scripts/fit_rgbd_objects.py new file mode 100644 index 0000000..c7427e8 --- /dev/null +++ b/scripts/fit_rgbd_objects.py @@ -0,0 +1,93 @@ +"""Estimate two colored object poses from RGB-D; partial-surface ICP, not pose truth.""" +import os +os.environ.setdefault('OMP_NUM_THREADS','4') +from pathlib import Path +import json +import cv2 +import numpy as np +import open3d as o3d +from scipy.spatial.transform import Rotation,Slerp +from scipy.ndimage import gaussian_filter1d + +ROOT=Path(__file__).resolve().parents[1];SRC=ROOT/'docs/20260915_171525';BASE=ROOT/'output/20260915_171525' +meta=json.loads((SRC/'intrinsics.json').read_text());cam=np.load(BASE/'rgbd_camera.npz');N=len(cam['time']) +o3d.utility.random.seed(7) +models={} +for name in ['upper','lower']: + mesh=o3d.io.read_triangle_mesh(str(BASE/'object_assets'/f'{name}.stl')) + cloud=mesh.sample_points_uniformly(6000);cloud=cloud.voxel_down_sample(.003) + models[name]=(cloud,np.asarray(mesh.vertices).mean(0),mesh.get_axis_aligned_bounding_box().get_center()) +cap=cv2.VideoCapture(str(SRC/'color.mp4'));clouds={n:[] for n in models};centers={n:[] for n in models} +for t in range(N): + ok,im=cap.read();assert ok + hsv=cv2.cvtColor(im,cv2.COLOR_BGR2HSV);z=cv2.imread(str(SRC/'depth'/f'{t:06d}.png'),-1)*meta['depth_scale_m'] + for name in models: + hue=hsv[:,:,0];mask=((hue<12)|(hue>170)) if name=='upper' else ((hue>90)&(hue<135)) + mask=(mask&(hsv[:,:,1]>100)&(hsv[:,:,2]>35)&(z>.12)&(z<.85)).astype(np.uint8) + # Restrict to the tabletop work area, then largest coherent colored component. + mask[:80]=0 + count,labels,stats,_=cv2.connectedComponentsWithStats(mask,8) + if count<=1 or stats[1:,cv2.CC_STAT_AREA].max()<300: + clouds[name].append(None);centers[name].append(None);continue + selected=1+np.argmax(stats[1:,cv2.CC_STAT_AREA]);mask=(labels==selected).astype(np.uint8) + contours,_=cv2.findContours(mask,cv2.RETR_EXTERNAL,cv2.CHAIN_APPROX_SIMPLE) + if name=='lower' and contours: + hull=cv2.convexHull(max(contours,key=cv2.contourArea));cv2.fillConvexPoly(mask,hull,1) + ycc=cv2.cvtColor(im,cv2.COLOR_BGR2YCrCb) + mask[cv2.inRange(ycc,np.array([0,133,77]),np.array([255,173,127]))>0]=0 + mask[(z<=.12)|(z>=.85)]=0 + mask=cv2.erode(mask,np.ones((3,3),np.uint8))>0 + v,u=np.nonzero(mask);zz=z[v,u];points=np.c_[(u-meta['cx'])/meta['fx']*zz,(v-meta['cy'])/meta['fy']*zz,zz] + points=points@cam['c2w'][t,:3,:3].T+cam['c2w'][t,:3,3] + pc=o3d.geometry.PointCloud(o3d.utility.Vector3dVector(points)).voxel_down_sample(.004) + clouds[name].append(pc);centers[name].append(np.median(np.asarray(pc.points),axis=0)) +cap.release() +arrays={};report={} +for name,(model,_,model_center) in models.items(): + poses=[];good=[];records=[];previous=None;previous_center=None + frames=np.unique(np.r_[np.arange(0,N,3),N-1]) + for t in frames: + target=clouds[name][t] + if target is None: + poses.append(np.eye(4) if previous is None else previous.copy());good.append(False);records.append(dict(frame=int(t),accepted=False));continue + points=np.asarray(target.points);center=centers[name][t] + seeds=[] + if previous is not None: + seed=previous.copy();seed[:3,3]+=center-previous_center;seeds.append(seed) + # Update observed face normal continuously, preserving the previous + # in-plane direction rather than reselecting a symmetric CAD yaw. + plane,inliers=target.segment_plane(.004,3,80) + normal=np.asarray(plane[:3]);normal/=np.linalg.norm(normal) + if normal@previous[:3,1]<0:normal=-normal + x=previous[:3,0]-normal*(normal@previous[:3,0]);x/=np.linalg.norm(x) + rr=np.column_stack([x,normal,np.cross(x,normal)]) + normal_seed=np.eye(4);normal_seed[:3,:3]=rr;normal_seed[:3,3]=center-rr@model_center;seeds.append(normal_seed) + if previous is None: + _,axes=np.linalg.eigh(np.cov(points.T));normal=axes[:,0];x=axes[:,2];zaxis=np.cross(x,normal) + for sign in [1,-1]: + basis=np.column_stack([x,normal*sign,zaxis*sign]) + for yaw in [0,np.pi/2,np.pi,3*np.pi/2]: + R=basis@Rotation.from_euler('y',yaw).as_matrix();T=np.eye(4);T[:3,:3]=R;T[:3,3]=center-R@model_center;seeds.append(T) + candidates=[] + for seed in seeds: + fit=o3d.pipelines.registration.registration_icp(target,model,.04,np.linalg.inv(seed),o3d.pipelines.registration.TransformationEstimationPointToPoint(),o3d.pipelines.registration.ICPConvergenceCriteria(max_iteration=35)) + candidates.append((fit.inlier_rmse+(.1*(1-fit.fitness)),fit)) + _,fit=min(candidates,key=lambda x:x[0]);T=np.linalg.inv(fit.transformation) + rotation_step=0. if previous is None else float(Rotation.from_matrix(previous[:3,:3].T@T[:3,:3]).magnitude()) + accepted=bool(fit.fitness>.8 and fit.inlier_rmse<.015 and np.isfinite(T).all() and rotation_step<.7) + if accepted:previous=T;previous_center=center + poses.append(T if accepted or previous is None else previous.copy());good.append(accepted) + records.append(dict(frame=int(t),accepted=accepted,fitness=float(fit.fitness),rmse_m=float(fit.inlier_rmse),rotation_step_rad=rotation_step)) + if t%60==0:print(name,t,'rmse',fit.inlier_rmse,'fitness',fit.fitness,flush=True) + poses=np.asarray(poses);good=np.asarray(good);ix=np.flatnonzero(good) + assert len(ix)>2,(name,len(ix)) + knots=frames[ix];ts=np.clip(np.arange(N),knots[0],knots[-1]);p=np.stack([np.interp(ts,knots,poses[ix,k,3]) for k in range(3)],axis=1) + rot=Slerp(knots,Rotation.from_matrix(poses[ix,:3,:3]))(ts) + arrays[name+'_position_world']=gaussian_filter1d(p,1,axis=0) + arrays[name+'_quaternion_xyzw']=rot.as_quat() + arrays[name+'_keyframes']=frames;arrays[name+'_keyframe_valid']=good + # Interpolation is explicit; accepted ICP is geometric fit, not unique pose identification. + report[name]=dict(keyframes=len(frames),accepted_keyframes=int(good.sum()),median_rmse_m=float(np.median([r['rmse_m'] for r in records if r['accepted']])),frames=records) +np.savez_compressed(BASE/'object_poses.npz',time=cam['time'],**arrays) +report['boundary']='Red=upper, blue=lower assumed from recording. Fixed meter-scale STL, partial colored visible-surface ICP; pose symmetry and occlusion ambiguity unresolved. Interpolated between keyframes; not contact labels or ground truth.' +(BASE/'object_fit_validation.json').write_text(json.dumps(report,indent=2));print(json.dumps({n:{k:v for k,v in r.items() if k!='frames'} for n,r in report.items() if isinstance(r,dict)},indent=2)) diff --git a/scripts/fix_yesterday_hand_collision.py b/scripts/fix_yesterday_hand_collision.py new file mode 100644 index 0000000..71a2864 --- /dev/null +++ b/scripts/fix_yesterday_hand_collision.py @@ -0,0 +1,118 @@ +"""Collision-constrained L20 retargeting: analytic contact Jacobians, exact mimic, verified steps.""" +import os +os.environ.setdefault('MUJOCO_GL','osmesa') +os.environ['OPENBLAS_NUM_THREADS']='1';os.environ['OMP_NUM_THREADS']='1' +from pathlib import Path +import json,copy,argparse,xml.etree.ElementTree as E +import numpy as np,mujoco,trimesh +from scipy.optimize import minimize +from scipy.spatial.transform import Rotation as Rotation +import l20_model_source as source +R=Path(__file__).resolve().parents[1];OLD=Path(os.environ.get('HF_FIX_OLD',R/'output/foundationpose_spider_20260915'));OUT=Path(os.environ.get('HF_FIX_OUT',R/'output/collision_fix_20260915'));T=OUT/'datasets/processed/current/l20/bimanual/boxes';(T/'0').mkdir(parents=True,exist_ok=True) +p=argparse.ArgumentParser();p.add_argument('--legacy-geoms',action='store_true');p.add_argument('--pilot',action='store_true');args=p.parse_args() +root=E.parse(OLD/'datasets/processed/current/l20/bimanual/boxes/scene_act.xml').getroot();assets=root.find('asset') +if not args.legacy_geoms: + manifest=json.loads((OUT/'collision_v2/manifest.json').read_text());handmanifest=json.loads((OUT/'hand_collision/manifest.json').read_text());assert len(manifest)==2 and len(handmanifest)==4 + for name,side in [('upper','right'),('lower','left')]: + body=root.find(f"worldbody/body[@name='{side}_object']") + for g in list(body): + if g.tag=='geom' and '_collision_' in g.get('name',''):body.remove(g) + for a in list(assets): + if a.get('name','').startswith(name+'_collision_'):assets.remove(a) + for i,path in enumerate(sorted((OUT/'collision_v2').glob(name+'_*.obj'))): + part=trimesh.load(path) + if abs(part.volume)<1e-12 or np.linalg.matrix_rank(part.vertices-part.vertices.mean(0),tol=1e-9)<3:continue + n=f'{name}_collision_{i}';E.SubElement(assets,'mesh',name=n,file=str(path),maxhullvert='128');E.SubElement(body,'geom',name=n,type='mesh',mesh=n,contype='2',conaffinity='7',friction='.8 .005 .001',condim='3',solref='.006 1',solimp='.95 .99 .001',mass='0',group='3') + for body in root.findall('.//body'): + for g in list(body.findall('geom')): + mesh=g.get('mesh') + if mesh not in handmanifest:continue + g.set('contype','0');g.set('conaffinity','0') + for i,path in enumerate(sorted((OUT/'hand_collision').glob(mesh+'_*.obj'))): + n=f'{mesh}_part_{i}';E.SubElement(assets,'mesh',name=n,file=str(path),maxhullvert='128');attrs=dict(g.attrib);attrs.update(name=n,mesh=n,contype='1',conaffinity='2',group='3',mass='0',solref='.006 1',solimp='.95 .99 .001');E.SubElement(body,'geom',**attrs) +TABLE=os.environ.get('HF_TABLE','0')=='1';PAIRS=[{1,2},{1,4}] if TABLE else [{1,2}] +if TABLE: + for fl in root.findall(".//geom[@name='right_floor']"):fl.set('conaffinity','3') # hands (contype 1) now collide with the table +# Keep contacts within 4 mm available to the solver, without early physical forces. +for g in root.findall('.//geom'): + if int(g.get('contype','0')) in ([1,2,4] if TABLE else [1,2]):g.attrib.update(margin='.004',gap='.004',solref='.005 1',solimp='.95 .99 .001') +scene=T/('scene_legacy.xml' if args.legacy_geoms else 'scene_act.xml');E.ElementTree(root).write(scene) +m=mujoco.MjModel.from_xml_path(str(scene));d=mujoco.MjData(m);source.REPO_ROOT=R/'third_party/l20_assets' +oldref=np.load(OLD/'reference_video_rate.npz');original=np.load(OLD/'reference_before_contact_fit.npz')['qpos'];q=oldref['qpos'].copy();cp=oldref['contact_pos'];contact=oldref['contact'];N=len(q);meta={} +for side in ['right','left']: + k=source.HandKinematics(OLD/('model_'+side)/('l20_'+side+'.xml'),side);wn=[side+'_hand_'+s for s in ['pos_x','pos_y','pos_z','rot_x','rot_y','rot_z']];wa=np.array([m.jnt_qposadr[m.joint(n).id] for n in wn]);ja=np.array([m.jnt_qposadr[m.joint(side+'_'+n).id] for n in k.joint_names]);B=np.zeros((m.nv,22));B[wa,:6]=np.eye(6);B[ja,6:]=k.expansion + meta[side]=(k,wa,ja,B,[m.site(side+'_hand_'+f+'_track').id for f in source.FINGERS]) +# All hand/object geometry is used, rather than only tip points. +def collision_rows(side,B,derivatives=True): + rows=[];dist=[] + for c in d.contact: + g0,g1=map(int,c.geom);types={int(m.geom_contype[g0]),int(m.geom_contype[g1])} + if types not in PAIRS:continue + hg=g0 if m.geom_contype[g0]==1 else g1 + if not m.geom(hg).name.startswith(side+'_'):continue + dist.append(float(c.dist)) + if derivatives: + jac=np.zeros((3,m.nv));mujoco.mj_jac(m,d,jac,None,c.pos,int(m.geom_bodyid[hg]));normal=c.frame[:3]*(1 if hg==g1 else -1);rows.append(normal@jac@B) + return np.asarray(rows).reshape(-1,22),np.array(dist) +def put(x,base,k,wa,ja): + d.qpos[:]=base;d.qpos[wa]=x[:6];d.qpos[ja]=k.expand(x[6:]);mujoco.mj_fwdPosition(m,d) +frames=[0,36,46,80,160,240,351] if args.pilot else range(N);report=[];previous={};margin=float(os.environ.get('HF_MARGIN','.0005')) +for f in frames: + for si,side in enumerate(['right','left']): + k,wa,ja,B,sites=meta[side];base=q[f].copy();xorig=np.r_[original[f,wa],original[f,ja][k.independent_indices]];x=np.r_[q[f,wa],q[f,ja][k.independent_indices]] + low=np.r_[xorig[:3]-.15,xorig[3:6]-.65,k.lower];high=np.r_[xorig[:3]+.15,xorig[3:6]+.65,k.upper];x=np.clip(x,low,high) + target=cp[f,si*5:si*5+5];on=contact[f,si*5:si*5+5].astype(bool);prior=xorig.copy() + if side in previous and not args.pilot: + prior=np.clip(xorig+previous[side]*(1. if on.any() else .95),low,high);x=prior.copy() + prev=np.r_[q[f-1,wa],q[f-1,ja][k.independent_indices]] + temporal=np.r_[[.012]*3,[.18]*3,[.25]*16];low=np.maximum(low,prev-temporal);high=np.minimum(high,prev+temporal) + # Transport the previous hand with its currently contacted object to stay on the same side. + if on.any(): + d.qpos[:]=base;mujoco.mj_kinematics(m,d);center=target[on].mean(0);choices=[54] if f>=177 else [54,60] + start=min(choices,key=lambda j:np.linalg.norm(center-d.site_xpos[m.site(('right' if j==54 else 'left')+'_object_track').id])) + old_obj=q[f-1,start:start+6];new_obj=q[f,start:start+6];delta=Rotation.from_euler('XYZ',new_obj[3:])*Rotation.from_euler('XYZ',old_obj[3:]).inv() + x[:3]=delta.apply(prev[:3]-old_obj[:3])+new_obj[:3] + e=(delta*Rotation.from_euler('XYZ',prev[3:6])).as_euler('XYZ');candidates=np.array([e,[e[0]+np.pi,np.pi-e[1],e[2]+np.pi]]);candidates+=2*np.pi*np.round((prev[3:6]-candidates)/(2*np.pi));x[3:6]=candidates[np.argmin(np.linalg.norm(candidates-prev[3:6],axis=1))];x[6:]=prev[6:] + x=np.clip(x,low,high) + put(x,base,k,wa,ja);_,dist=collision_rows(side,B,False);initial=max(0,-dist.min()) if len(dist) else 0 + # Exit deep or contradictory overlap first; no object pose changes or mesh shrinking. + if initial>.004 and (f==0 or args.pilot): + seeds=[x.copy()];directions=np.array([[i,j,z] for i in [-1,0,1] for j in [-1,0,1] for z in [-1,0,1] if (i,j,z)!=(0,0,0)],float);directions/=np.linalg.norm(directions,axis=1)[:,None] + for radius in [.025,.05,.08,.12]: + for direction in directions: + seed=x.copy();seed[:3]=np.clip(x[:3]+direction*radius,low[:3],high[:3]);seeds.append(seed) + best=None + for seed in seeds: + put(seed,base,k,wa,ja);_,ds=collision_rows(side,B,False);pen=max(0,-ds.min()) if len(ds) else 0;gap=np.linalg.norm(d.site_xpos[sites][on]-target[on],axis=1).mean() if on.any() else 0 + score=500*pen+np.linalg.norm(seed[:3]-xorig[:3])+gap*.3 + if best is None or score.0001 and tp.part-000, .part-001, ... (each chunk is a Git LFS object) together with +configs/large_files.json (original path, size, sha256, chunk count). + + python3 scripts/large_files.py assemble # after clone: rebuild every original file, verify sha256 + python3 scripts/large_files.py split ... # maintainer: chunk new large files and update the manifest + python3 scripts/large_files.py check # verify assembled files against the manifest + +Chunks are left in place after assembly; delete them with `assemble --clean` if you need the space. +""" +import argparse, hashlib, json, sys +from pathlib import Path + +ROOT = Path(__file__).resolve().parents[1] +MANIFEST = ROOT / 'configs/large_files.json' +CHUNK = 48 * 1024 * 1024 + + +def sha256(path): + h = hashlib.sha256() + with path.open('rb') as f: + for block in iter(lambda: f.read(1 << 24), b''): + h.update(block) + return h.hexdigest() + + +def load(): + return json.loads(MANIFEST.read_text()) if MANIFEST.exists() else {'chunk_bytes': CHUNK, 'files': []} + + +def split(paths): + m = load(); known = {e['path'] for e in m['files']} + for p in paths: + src = (ROOT / p).resolve(); rel = src.relative_to(ROOT).as_posix() + if rel in known: + print('already in manifest:', rel); continue + n = 0 + with src.open('rb') as f: + while True: + block = f.read(CHUNK) + if not block: break + (src.parent / f'{src.name}.part-{n:03d}').write_bytes(block); n += 1 + m['files'].append({'path': rel, 'bytes': src.stat().st_size, 'sha256': sha256(src), 'parts': n}) + print(f'split {rel}: {n} parts'); src.unlink() + MANIFEST.write_text(json.dumps(m, indent=2) + '\n') + + +def assemble(clean=False): + m = load(); bad = 0 + for e in m['files']: + dst = ROOT / e['path']; parts = [dst.parent / f'{dst.name}.part-{i:03d}' for i in range(e['parts'])] + if dst.exists() and dst.stat().st_size == e['bytes'] and sha256(dst) == e['sha256']: + print('ok ', e['path']) + else: + missing = [p.name for p in parts if not p.exists() or p.stat().st_size < 200] + if missing: + print('MISSING chunks (run `git lfs pull` first):', e['path'], missing[:3], '...'); bad += 1; continue + with dst.open('wb') as out: + for p in parts: out.write(p.read_bytes()) + if sha256(dst) != e['sha256']: + print('SHA256 MISMATCH', e['path']); bad += 1; continue + print('assembled', e['path'], f"({e['bytes'] / 1e6:.0f} MB, {e['parts']} parts)") + if clean: + for p in parts: p.unlink(missing_ok=True) + if bad: sys.exit(f'{bad} file(s) could not be assembled') + + +def check(): + for e in load()['files']: + dst = ROOT / e['path'] + ok = dst.exists() and dst.stat().st_size == e['bytes'] and sha256(dst) == e['sha256'] + print('ok ' if ok else 'BAD ', e['path']) + + +if __name__ == '__main__': + ap = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter) + sub = ap.add_subparsers(dest='cmd', required=True) + s = sub.add_parser('split'); s.add_argument('paths', nargs='+') + a = sub.add_parser('assemble'); a.add_argument('--clean', action='store_true') + sub.add_parser('check') + args = ap.parse_args() + if args.cmd == 'split': split(args.paths) + elif args.cmd == 'assemble': assemble(args.clean) + else: check() diff --git a/scripts/package_source_release.py b/scripts/package_source_release.py index 80d9489..d15bb71 100644 --- a/scripts/package_source_release.py +++ b/scripts/package_source_release.py @@ -39,9 +39,12 @@ def main(): reason = 'symlink: restore dependency explicitly' elif any(p in EXCLUDE_PARTS for p in relative.parts): reason = 'runtime/data/cache directory' + elif relative.parts[0] == 'third_party' and ('build' in relative.parts or '.cache' in relative.parts or src.suffix == '.part' or any(x.endswith('.egg-info') for x in relative.parts)): + reason = 'build artifact or partial download' elif relative.name == '.gitmodules' or (relative.parts[0] == 'third_party' and relative.name in {'.gitignore','.gitattributes'}): reason = 'vendored snapshot: omit nested Git filters and metadata' - elif src.suffix not in EXTENSIONS and src.name not in SPECIAL and not src.name.startswith(('LICENSE','COPYING','NOTICE')): + elif src.suffix not in EXTENSIONS and src.name not in SPECIAL and not src.name.startswith(('LICENSE','COPYING','NOTICE')) \ + and not (relative.parts[0] == 'docs' and src.suffix.lower() == '.stl' and src.stat().st_size < 1024*1024): reason = 'non-source file; restore separately when needed' elif src.stat().st_size > 5*1024*1024: reason = 'file exceeds 5 MiB source limit' @@ -49,7 +52,7 @@ def main(): excluded.append({'path':relative.as_posix(),'reason':reason}) return raw = src.read_bytes() - if b'\0' in raw: + if b'\0' in raw and src.suffix.lower() != '.stl': excluded.append({'path':relative.as_posix(),'reason':'binary contents'}) return target = dest/relative @@ -57,7 +60,7 @@ def main(): shutil.copy2(src,target) if src.suffix in {'.py','.sh','.yaml','.yml','.json','.toml'}: for i,line in enumerate(raw.decode('utf-8',errors='replace').splitlines(),1): - if re.search(r'/home/|/tmp/|2047635068|15886123',line): + if re.search(r'/home/|/tmp/|2047635068|15886123|20260915_171525|20260916_104026',line): portability.append({'path':relative.as_posix(),'line':i, 'kind':'machine path or fixed example identifier'}) @@ -97,7 +100,7 @@ def main(): copy_file(src,src.relative_to(ROOT)) for name in ['README.md','LICENSE','.gitignore','requirements.txt','setup_env.sh','setup_vipe_env.sh']: copy_file(ROOT/name,Path(name)) - for name in ['hamer','vipe','Dyn-HaMR','spider']: + for name in ['hamer','vipe','Dyn-HaMR','spider','FoundationPose','nvdiffrast']: snapshot(ROOT/'third_party'/name,Path('third_party')/name) repos.insert(0,{'path':'.','commit':git(ROOT,'rev-parse','HEAD'), 'origin':git(ROOT,'remote','get-url','origin'), @@ -107,10 +110,13 @@ def main(): '# Source snapshot: runtime exclusions anchored to repository root.\n' '/output/\n/results/\n/logs/\n/weights/\n/dist/\n/.venv/\n/venv/\n' '/.dex/\n/.spider/\n/.cuda/\n/.hf_cache/\n/.torch_cache/\n/.spider_cache/\n' - '/.uv_cache_spider/\n/.env\n__pycache__/\n*.pyc\n') + '/.uv_cache_spider/\n/.env\n__pycache__/\n*.pyc\n' + '/third_party/FoundationPose/weights/\n/third_party/*/build/\n/.cache/\n*.part\n' + '/docs/20260915_171525/\n/docs/20260916_104026/\n') assets=[] - for name in ['l20_assets','bottle_model_parametric','dex-assets']: - folder=ROOT/'third_party'/name + for rel in ['third_party/l20_assets','third_party/bottle_model_parametric','third_party/dex-assets', + 'third_party/FoundationPose/weights','docs/20260915_171525']: + folder=ROOT/rel files=[{'path':p.relative_to(ROOT).as_posix(),'bytes':p.stat().st_size, 'sha256':hashlib.sha256(p.read_bytes()).hexdigest()} for p in sorted(folder.rglob('*')) if p.is_file() and not p.is_symlink()] diff --git a/scripts/play_compare.py b/scripts/play_compare.py new file mode 100644 index 0000000..7727770 --- /dev/null +++ b/scripts/play_compare.py @@ -0,0 +1,84 @@ +"""Side-by-side interactive playback: the same scene duplicated in one MuJoCo world. +Left copy = trajectory A (e.g. kinematic reference), right copy = trajectory B (e.g. physics rollout), time-synchronised.""" +import argparse, os, time, json, copy +from pathlib import Path +import xml.etree.ElementTree as E +p = argparse.ArgumentParser(description=__doc__) +p.add_argument('--scene', type=Path, required=True); p.add_argument('--a', type=Path, required=True); p.add_argument('--b', type=Path, required=True) +p.add_argument('--label-a', default='A'); p.add_argument('--label-b', default='B'); p.add_argument('--offset', type=float, default=.7); p.add_argument('--check', action='store_true'); p.add_argument('--out', type=Path) +a = p.parse_args(); os.environ['MUJOCO_GL'] = 'osmesa' if a.check else 'glfw' +import numpy as np, mujoco +root = E.parse(a.scene).getroot(); world = root.find('worldbody') +NAMED = ['name', 'joint', 'joint1', 'joint2', 'site', 'body', 'target', 'geom1', 'geom2', 'body1', 'body2'] +def prefix_tree(el, pre): + for e in el.iter(): + for k in NAMED: + v = e.get(k) + if v and k != 'mesh' and k != 'material' and k != 'class': e.set(k, pre + v) +originals = list(world) +copyB = E.Element('body', name='B_root', pos=f'0 {a.offset} 0') +for e in originals: + if e.tag == 'camera' or e.tag == 'light': continue + c = copy.deepcopy(e); prefix_tree(c, 'B_'); copyB.append(c) +for e in originals: + if e.tag not in ('camera', 'light'): prefix_tree(e, 'A_') +world.append(copyB) +for sec in ['actuator', 'equality', 'contact', 'sensor', 'tendon']: + s = root.find(sec) + if s is None: continue + items = list(s) + for e in items: + c = copy.deepcopy(e); prefix_tree(c, 'B_'); s.append(c); prefix_tree(e, 'A_') +out = a.out or (a.a.parent / 'compare_scene.xml'); E.ElementTree(root).write(out) +m = mujoco.MjModel.from_xml_path(str(out)); d = mujoco.MjData(m) +def load(path): + z = np.load(path); return z['qpos'], z['time'] +qa, ta = load(a.a); qb, tb = load(a.b); n = qa.shape[1]; assert qb.shape[1] == n and m.nq == 2 * n, (m.nq, n) +# qpos layout: joints of A come first (original order), then B (same order) -> verify via joint names +ja = [i for i in range(m.njnt) if m.joint(i).name.startswith('A_')]; jb = [i for i in range(m.njnt) if m.joint(i).name.startswith('B_')] +adrA = np.concatenate([np.arange(m.jnt_qposadr[i], m.jnt_qposadr[i] + (7 if m.jnt_type[i] == 0 else 4 if m.jnt_type[i] == 1 else 1)) for i in ja]); adrB = np.concatenate([np.arange(m.jnt_qposadr[i], m.jnt_qposadr[i] + (7 if m.jnt_type[i] == 0 else 4 if m.jnt_type[i] == 1 else 1)) for i in jb]) +assert len(adrA) == n and len(adrB) == n +tmax = float(min(ta[-1], tb[-1])) +def frame(tq, q, t): i = int(np.clip(np.searchsorted(tq, t, side='right') - 1, 0, len(tq) - 1)); return q[i] +def set_time(t): + d.qpos[adrA] = frame(ta, qa, t); d.qpos[adrB] = frame(tb, qb, t); d.time = t; mujoco.mj_forward(m, d) +if a.check: + for t in [0, tmax / 2, tmax]: set_time(t) + print(json.dumps({'nq': m.nq, 'duration_s': tmax, 'finite': bool(np.isfinite(d.qpos).all()), 'scene': str(out)})); raise SystemExit +import mujoco.viewer +state = {'paused': False, 'time': 0., 'speed': 1., 'reset_camera': False} +def key(k): + if k == 32: state['paused'] = not state['paused'] + elif k in [82, 114]: state['time'] = 0. + elif k in [262, 263]: state['time'] = float(np.clip(state['time'] + (1 if k == 262 else -1) / 30, 0, tmax)); state['paused'] = True + elif k == 265: state['speed'] = min(4., state['speed'] * 2) + elif k == 264: state['speed'] = max(.125, state['speed'] / 2) + elif k in [67, 99]: state['reset_camera'] = True +m.vis.headlight.ambient[:] = .7; set_time(0) +centers = [np.mean([d.site_xpos[m.site(pre + s + '_object_track').id] for s in ['right', 'left']], axis=0) for pre in ['A_', 'B_']]; center = np.mean(centers, axis=0) +cid = m.camera('A_source_camera').id if any(m.camera(i).name == 'A_source_camera' for i in range(m.ncam)) else 0; offset = d.cam_xpos[cid] - centers[0] +camera = {'distance': float(np.linalg.norm(offset)) * 1.6, 'azimuth': float(np.degrees(np.arctan2(offset[1], offset[0]))), 'elevation': float(-np.degrees(np.arctan2(offset[2], np.linalg.norm(offset[:2]))))} +status = out.parent / 'compare_playback_status.json' +print(f'LEFT (y=0): {a.label_a} | RIGHT (y=+{a.offset}): {a.label_b} | SPACE pause | R restart | LEFT/RIGHT frame step | UP/DOWN speed | C camera', flush=True) +with mujoco.viewer.launch_passive(m, d, key_callback=key) as viewer: + def reset_camera(): + viewer.cam.type = mujoco.mjtCamera.mjCAMERA_FREE; viewer.cam.lookat[:] = center + for k, x in camera.items(): setattr(viewer.cam, k, x) + with viewer.lock(): + reset_camera(); viewer.opt.sitegroup[:] = 0; viewer.opt.geomgroup[3:] = 0 + # colour markers: green sphere above A, red above B + for i, (c, rgba) in enumerate([(centers[0], (0, .8, 0, 1)), (centers[1], (.9, .1, .1, 1))]): + g = viewer.user_scn.geoms[i]; mujoco.mjv_initGeom(g, mujoco.mjtGeom.mjGEOM_SPHERE, np.array([.02, 0, 0]), np.array(c) + np.array([0, 0, .35]), np.eye(3).ravel(), np.array(rgba, dtype=np.float32)) + viewer.user_scn.ngeom = 2 + last = time.perf_counter(); last_status = 0 + while viewer.is_running(): + now = time.perf_counter(); el = now - last; last = now + if not state['paused']: state['time'] = (state['time'] + el * state['speed']) % tmax + with viewer.lock(): + if state['reset_camera']: reset_camera(); state['reset_camera'] = False + set_time(state['time']) + viewer.sync() + if now - last_status > 1: + status.write_text(json.dumps({'running': True, 'pid': os.getpid(), 'left': a.label_a, 'right': a.label_b, 'time_s': state['time'], 'speed': state['speed'], 'paused': state['paused'], 'updated_unix': time.time()}, indent=2)); last_status = now + time.sleep(.008) +status.write_text(json.dumps({'running': False, 'pid': os.getpid(), 'updated_unix': time.time()})) diff --git a/scripts/play_spider_result.py b/scripts/play_spider_result.py new file mode 100644 index 0000000..6b13654 --- /dev/null +++ b/scripts/play_spider_result.py @@ -0,0 +1,60 @@ +"""Interactive wall-clock playback of saved SPIDER states (no re-simulation).""" +import argparse,os,time,json +from pathlib import Path +p=argparse.ArgumentParser(description=__doc__) +p.add_argument('--directory',type=Path,default=Path(__file__).resolve().parents[1]/'output/spider_dynamics_fix_20260915') +p.add_argument('--mode',choices=['physics','reference'],default='physics') +p.add_argument('--check',action='store_true');a=p.parse_args() +os.environ['MUJOCO_GL']='osmesa' if a.check else 'glfw' +import numpy as np,mujoco +root=a.directory.resolve();task=root/'datasets/processed/current/l20/bimanual/boxes' +m=mujoco.MjModel.from_xml_path(str(task/'scene_act.xml'));d=mujoco.MjData(m) +source=root/('reference_video_rate.npz' if a.mode=='reference' else 'physics_motion.npz') +mode='collision-corrected kinematic reference (middle video panel)' if a.mode=='reference' else 'saved physical state playback, no re-simulation' +with np.load(source) as z: + q=z['qpos'];t=z['time'] + if a.mode=='physics':u=z['ctrl'] +if a.mode=='reference': + v=np.zeros((len(q),m.nv));u=q[:,m.jnt_qposadr[m.actuator_trnid[:,0]]] +else: + with np.load(task/'0/trajectory_mjwp_act.npz') as z:v=z['qvel'].reshape(len(q),-1) +assert q.shape==(len(t),m.nq) and np.isfinite(q).all() and np.all(np.diff(t)>0) +if a.check: + for i in [0,len(q)//2,len(q)-1]: + d.qpos[:]=q[i];d.qvel[:]=v[i];d.ctrl[:]=u[i];mujoco.mj_forward(m,d) + print(json.dumps({'steps':len(q),'duration_s':float(t[-1]),'nq':m.nq,'finite':True,'mode':mode}));raise SystemExit +import mujoco.viewer +state={'paused':False,'time':0.,'speed':1.,'reset_camera':False} +def key(k): + if k==32:state['paused']=not state['paused'] + elif k in [82,114]:state['time']=0. + elif k in [262,263]:state['time']=float(np.clip(state['time']+(1 if k==262 else -1)/30,0,t[-1]));state['paused']=True + elif k==265:state['speed']=min(4.,state['speed']*2) + elif k==264:state['speed']=max(.125,state['speed']/2) + elif k in [67,99]:state['reset_camera']=True +m.vis.headlight.ambient[:]=.7 +d.qpos[:]=q[0];mujoco.mj_forward(m,d) +center=np.mean([d.site_xpos[m.site(s+'_object_track').id] for s in ['right','left']],axis=0) +cid=m.camera('source_camera').id;offset=d.cam_xpos[cid]-center +camera={'distance':float(np.linalg.norm(offset)),'azimuth':float(np.degrees(np.arctan2(offset[1],offset[0]))),'elevation':float(-np.degrees(np.arctan2(offset[2],np.linalg.norm(offset[:2]))))} +status=root/'interactive_playback_status.json' +print(mode+' | SPACE pause | R restart | LEFT/RIGHT video-frame step | UP/DOWN speed | C camera reset | mouse orbit/zoom',flush=True) +with mujoco.viewer.launch_passive(m,d,key_callback=key) as viewer: + def reset_camera(): + viewer.cam.type=mujoco.mjtCamera.mjCAMERA_FREE;viewer.cam.lookat[:]=center + for k,x in camera.items():setattr(viewer.cam,k,x) + with viewer.lock(): + reset_camera();viewer.opt.sitegroup[:]=0;viewer.opt.geomgroup[3:]=0 + last=time.perf_counter();last_status=0 + while viewer.is_running(): + now=time.perf_counter();elapsed=now-last;last=now + if not state['paused']:state['time']=(state['time']+elapsed*state['speed'])%float(t[-1]) + i=int(np.clip(np.searchsorted(t,state['time'],side='right')-1,0,len(t)-1)) + with viewer.lock(): + if state['reset_camera']:reset_camera();state['reset_camera']=False + d.qpos[:]=q[i];d.qvel[:]=v[i];d.ctrl[:]=u[i];d.time=float(t[i]);mujoco.mj_forward(m,d) + viewer.sync() + if now-last_status>1: + status.write_text(json.dumps({'running':True,'pid':os.getpid(),'mode':mode,'source':str(source),'step':i,'time_s':float(t[i]),'speed':state['speed'],'paused':state['paused'],'updated_unix':time.time()},indent=2));last_status=now + time.sleep(.008) +status.write_text(json.dumps({'running':False,'pid':os.getpid(),'updated_unix':time.time()})) diff --git a/scripts/prepare_rgbd_20260915.py b/scripts/prepare_rgbd_20260915.py new file mode 100644 index 0000000..1f8c295 --- /dev/null +++ b/scripts/prepare_rgbd_20260915.py @@ -0,0 +1,53 @@ +"""Audit RGB-D input, preserve object meshes and estimate static-background camera motion.""" +import os +os.environ.setdefault('OMP_NUM_THREADS','4') +from pathlib import Path +import json +import shutil +import cv2 +import numpy as np +import open3d as o3d + +ROOT=Path(__file__).resolve().parents[1] +SRC=ROOT/'docs/20260915_171525';OUT=ROOT/'output/20260915_171525';OUT.mkdir(exist_ok=True) +meta=json.loads((SRC/'intrinsics.json').read_text()) +times=np.asarray(meta['timestamps_ms'])/1000;times-=times[0] +assert len(times)==meta['frames'] and np.all(np.diff(times)>0) +assert len(list((SRC/'depth').glob('*.png')))==len(times) +assets=OUT/'object_assets';assets.mkdir(exist_ok=True) +meshes=[] +for source,name in [('上半.stl','upper.stl'),('下半.stl','lower.stl')]: + shutil.copy2(ROOT/'docs'/source,assets/name) + mesh=o3d.io.read_triangle_mesh(str(assets/name));v=np.asarray(mesh.vertices) + meshes.append(dict(file=name,vertices=len(v),triangles=len(mesh.triangles),bounds=[v.min(0).tolist(),v.max(0).tolist()],extent_m=np.ptp(v,axis=0).tolist())) +(assets/'manifest.json').write_text(json.dumps(dict(units='assumed meters from numeric dimensions; compare observed depth before claiming registration',parts=meshes,preserve_shared_coordinates=True),indent=2)) +cap=cv2.VideoCapture(str(SRC/'color.mp4'));K=o3d.camera.PinholeCameraIntrinsic(424,240,meta['fx']/2,meta['fy']/2,(meta['cx']+.5)/2-.5,(meta['cy']+.5)/2-.5) +option=o3d.pipelines.odometry.OdometryOption();option.depth_max=1.5;option.depth_min=.10;option.depth_diff_max=.025 +option.iteration_number_per_pyramid_level=o3d.utility.IntVector([15,8,5]) +c2w=[np.eye(4)];valid=[True];rows=[];previous=None +for i in range(len(times)): + ok,frame=cap.read();assert ok + depth=cv2.imread(str(SRC/'depth'/f'{i:06d}.png'),-1);assert depth.dtype==np.uint16 and depth.shape==frame.shape[:2] + color=cv2.resize(frame,(424,240),interpolation=cv2.INTER_AREA) + z=cv2.resize(depth,(424,240),interpolation=cv2.INTER_NEAREST).astype(np.float32)*meta['depth_scale_m'] + hsv=cv2.cvtColor(color,cv2.COLOR_BGR2HSV);ycc=cv2.cvtColor(color,cv2.COLOR_BGR2YCrCb) + skin=cv2.inRange(ycc,np.array([0,133,77]),np.array([255,173,127]))>0 + colored=(hsv[:,:,1]>85)&(((hsv[:,:,0]<14)|(hsv[:,:,0]>165))|((hsv[:,:,0]>90)&(hsv[:,:,0]<135))) + dynamic=skin|colored;dynamic[160:]=True + dynamic=cv2.dilate(dynamic.astype(np.uint8),np.ones((9,9),np.uint8))>0 + z[dynamic]=0 + rgbd=o3d.geometry.RGBDImage.create_from_color_and_depth(o3d.geometry.Image(cv2.cvtColor(color,cv2.COLOR_BGR2RGB)),o3d.geometry.Image(z),depth_scale=1.,depth_trunc=1.5,convert_rgb_to_intensity=True) + if previous is not None: + success,T,info=o3d.pipelines.odometry.compute_rgbd_odometry(previous,rgbd,K,np.eye(4),o3d.pipelines.odometry.RGBDOdometryJacobianFromHybridTerm(),option) + angle=np.arccos(np.clip((np.trace(T[:3,:3])-1)/2,-1,1)) + accepted=bool(success and np.isfinite(T).all() and np.linalg.norm(T[:3,3])<.06 and angle<.12) + valid.append(accepted) + c2w.append(c2w[-1]@np.linalg.inv(T) if accepted else c2w[-1].copy()) + rows.append(dict(frame=i,accepted=accepted,translation_step_m=float(np.linalg.norm(T[:3,3])),rotation_step_rad=float(angle))) + previous=rgbd + if i%50==0:print('RGBD camera',i,len(times),flush=True) +cap.release() +np.savez_compressed(OUT/'rgbd_camera.npz',c2w=np.asarray(c2w),intrinsics=np.array([meta[k] for k in ['fx','fy','cx','cy']]),time=times,valid=np.asarray(valid),method='Open3D adjacent RGBD odometry; mask skin/red/blue/bottom third; first camera world; no loop closure') +report=dict(frames=len(times),time_span_s=float(times[-1]),color_depth_timestamp_max_difference_ms=float(np.max(np.abs(np.array(meta['timestamps_ms'])-meta['depth_timestamps_ms']))),camera_valid_steps=int(np.sum(valid)),camera_rejected_frames=np.flatnonzero(~np.asarray(valid)).tolist(),steps=rows,boundary='Odometry estimates, not external tracking truth; drift possible; failed steps held and explicitly invalid') +(OUT/'rgbd_preflight.json').write_text(json.dumps(report,indent=2)) +print(json.dumps({k:v for k,v in report.items() if k!='steps'},indent=2)) diff --git a/scripts/prepare_yesterday_spider.py b/scripts/prepare_yesterday_spider.py new file mode 100644 index 0000000..8c158ce --- /dev/null +++ b/scripts/prepare_yesterday_spider.py @@ -0,0 +1,129 @@ +"""Build the two-hand/two-object SPIDER task from recorded, calibrated tracks.""" +import os +os.environ.setdefault('MUJOCO_GL','osmesa') +from pathlib import Path +import copy,json,xml.etree.ElementTree as E +import numpy as np,cv2,mujoco,trimesh,open3d as o3d +from scipy.spatial.transform import Rotation as R,Slerp +from scipy.optimize import least_squares +import l20_model_source as source +ROOT=Path(__file__).resolve().parents[1];OUT=Path(os.environ.get('HF_SPIDER_OUT',ROOT/'output/foundationpose_spider_20260915'));BASE=Path(os.environ.get('HF_HAND_BASE',ROOT/'output/20260915_171525_dynhamr')) +TASK=OUT/'datasets/processed/current/l20/bimanual/boxes';TRIAL=TASK/'0';TRIAL.mkdir(parents=True,exist_ok=True) +source.REPO_ROOT=ROOT/'third_party/l20_assets';fingers=source.FINGERS;fp=np.load(OUT/'foundationpose_objects.npz');N=len(fp['time']);t=np.arange(N)/30 +# Fit the visible table, excluding saturated foreground; z-up is table normal, not IMU gravity. +cap=cv2.VideoCapture(str(ROOT/'docs/20260915_171525/color.mp4'));ok,img=cap.read();cap.release();hsv=cv2.cvtColor(img,cv2.COLOR_BGR2HSV) +dep=cv2.imread(str(ROOT/'docs/20260915_171525/depth/000000.png'),-1)*.0001;yy,xx=np.indices(dep.shape);K=fp['K'] +mask=(xx>100)&(xx<780)&(yy>200)&(yy<460)&(hsv[:,:,1]<50)&(dep>.2)&(dep<1) +pts=np.stack([(xx-K[0,2])*dep/K[0,0],(yy-K[1,2])*dep/K[1,1],dep],-1)[mask][::3] +pc=o3d.geometry.PointCloud(o3d.utility.Vector3dVector(pts));plane,ii=pc.segment_plane(.004,3,1000);normal=np.array(plane[:3]);offset=plane[3] +if normal[2]>0:normal=-normal;offset=-offset +S=R.align_vectors([[0,0,1]],[normal])[0].as_matrix();shift=np.array([0.,0.,offset]);world_from_source=np.eye(4);world_from_source[:3,:3]=S;world_from_source[:3,3]=shift +# Interpolate only upper rejected intervals. Lower occlusion gets an explicit, separate hypothesis. +poses={};provenance={} +for name in ['upper','lower']: + raw=fp[name+'_T_world'].copy();valid=fp[name+'_valid'].astype(bool);idx=np.flatnonzero(valid) + out=raw.copy();clip=np.clip(t,t[idx[0]],t[idx[-1]]) + out[:,:3,3]=np.stack([np.interp(t,t[idx],raw[idx,k,3]) for k in range(3)],1) + out[:,:3,:3]=Slerp(t[idx],R.from_matrix(raw[idx,:3,:3]))(clip).as_matrix() + if name=='lower': + # Hold last visible world pose after frame176. No fabricated assembled pose. + provenance[name]='Observed through frame176; held last valid world pose during occlusion, not a measurement.' + else:provenance[name]='Rejected frames interpolated between accepted poses; endpoints held.' + poses[name]=world_from_source[None]@out +np.savez_compressed(OUT/'object_reference.npz',**{k:v for k,v in poses.items()},source_world_to_sim=world_from_source,time=t,upper_observed=fp['upper_observed_raw'] if 'upper_observed_raw' in fp else fp['upper_valid'],lower_observed=fp['lower_observed_raw'] if 'lower_observed_raw' in fp else fp['lower_valid']) +root=None;handmeta={} +for side in ['right','left']: + build=OUT/('model_'+side);path=source.build_model(side,build);rr=E.parse(path).getroot() + for mesh in rr.findall('asset/mesh'):mesh.set('file',str((build/mesh.get('file')).resolve()));mesh.set('maxhullvert','32') + # Prefix all named entities and their references before combining. + for e in rr.iter(): + for a in ['name','mesh','material','joint1','joint2']: + if e.get(a):e.set(a,side+'_'+e.get(a)) + hand=rr.find('worldbody/body');wn=[] + for i in range(6): + n=f'{side}_hand_'+('pos_'+'xyz'[i] if i<3 else 'rot_'+'xyz'[i-3]);wn.append(n) + hand.insert(i,E.Element('joint',name=n,type='slide' if i<3 else 'hinge',axis=['1 0 0','0 1 0','0 0 1'][i%3],limited='false',damping='0',armature='.02')) + for g in hand.iter('geom'):g.attrib.update(contype='1',conaffinity='2',friction='.8 .005 .001',condim='3',solref='.015 1') + for j,f in enumerate(fingers):hand.find(f".//site[@name='{side}_landmark_{4+4*j:02d}']").set('name',f'{side}_hand_{f}_track') + kin=source.HandKinematics(path,side);handmeta[side]=(kin,wn) + if root is None: + root=rr;world=root.find('worldbody');assets=root.find('asset');act=E.SubElement(root,'actuator') + floor=world.find("geom[@name='right_floor']");floor.attrib.update(pos='0 0 0',size='2 2 .01',contype='4',conaffinity='2',rgba='.6 .62 .65 1') + else: + assets.extend(list(rr.find('asset')));world.append(hand);root.find('equality').extend(list(rr.find('equality'))) + for i,n in enumerate(wn):E.SubElement(act,'position',name=n+'_act',joint=n,kp='2000' if i<3 else '40',kv='60' if i<3 else '2',forcerange='-200 200' if i<3 else '-20 20') + for n,lo,hi in zip(kin.independent_joint_names,kin.lower,kin.upper):E.SubElement(act,'position',name=side+'_'+n+'_act',joint=side+'_'+n,kp='8',kv='.15',forcerange='-1 1',ctrlrange=source._fmt([lo,hi])) +root.find('option').attrib.update(timestep='.0025',gravity='0 0 -9.81',integrator='implicitfast',cone='elliptic') +root.find('default/joint').attrib.update(damping='.05',armature='.002') +for eq in root.findall('equality/joint'):eq.set('solref','.01 1') +meshes={};scenes={} +for side,name,stl,color in [('right','upper','上半.stl','.85 .08 .06 1'),('left','lower','下半.stl','.06 .18 .8 1')]: + mesh=trimesh.load(ROOT/'docs'/stl);meshes[name]=mesh;assets.append(E.Element('mesh',name=name+'_visual',file=str(ROOT/'docs'/stl))) + obj=E.SubElement(world,'body',name=side+'_object') + # Mass is an explicit simulation assumption, not a sensor measurement. + E.SubElement(obj,'inertial',pos=source._fmt(mesh.center_mass),mass='.12',diaginertia='.0004 .0006 .0004') + for i in range(6): + n=side+'_object_'+('pos_'+'xyz'[i] if i<3 else 'rot_'+'xyz'[i-3]);E.SubElement(obj,'joint',name=n,type='slide' if i<3 else 'hinge',axis=['1 0 0','0 1 0','0 0 1'][i%3],limited='false',damping='0.001',armature='.001');E.SubElement(act,'position',name=n,joint=n,kp='0',kv='0') + E.SubElement(obj,'geom',name=name+'_visual',type='mesh',mesh=name+'_visual',rgba=color,contype='0',conaffinity='0',mass='0',group='1') + for i,p in enumerate(sorted((OUT/'collision').glob(name+'_*.obj'))): + mn=f'{name}_collision_{i}';E.SubElement(assets,'mesh',name=mn,file=str(p),maxhullvert='32');E.SubElement(obj,'geom',name=mn,type='mesh',mesh=mn,contype='2',conaffinity='7',friction='.8 .005 .001',condim='3',solref='.015 1',mass='0',group='3') + E.SubElement(obj,'site',name=side+'_object_track',pos=source._fmt(mesh.centroid)) + sc=o3d.t.geometry.RaycastingScene();sc.add_triangles(o3d.t.geometry.TriangleMesh.from_legacy(o3d.geometry.TriangleMesh(o3d.utility.Vector3dVector(mesh.vertices),o3d.utility.Vector3iVector(mesh.faces))));scenes[name]=sc +E.SubElement(world,'camera',name='source_camera',fovy=str(np.degrees(2*np.arctan(480/(2*K[1,1]))))) +scene=TASK/'scene_act.xml';E.ElementTree(root).write(scene) +m=mujoco.MjModel.from_xml_path(str(scene));d=mujoco.MjData(m);q=np.zeros((N,m.nq));contact=np.zeros((N,10));cp=np.zeros((N,10,3));fit_report={};initialq=None +for si,(side,name) in enumerate([('right','upper'),('left','lower')]): + kin,wn=handmeta[side];motion=np.load(BASE/f'l20_{side}_stable/motion.npz');human=np.load(BASE/f'human_joints_{side}.npz') + wp=(motion['wrist_pos']-motion['scene_translation'])@motion['scene_rotation'];wr=np.einsum('ij,njk->nik',motion['scene_rotation'].T,R.from_quat(motion['wrist_quat_wxyz'][:,[1,2,3,0]]).as_matrix()) + wp=wp@S.T+shift;wr=np.einsum('ij,njk->nik',S,wr);angles=np.unwrap(R.from_matrix(wr).as_euler('XYZ'),axis=0) + wa=[m.jnt_qposadr[m.joint(n).id] for n in wn];ja=[m.jnt_qposadr[m.joint(side+'_'+n).id] for n in kin.joint_names] + full=motion['qpos'][:,[list(motion['joint_names']).index(n) for n in kin.joint_names]];a=full[:,kin.independent_indices];a=np.clip(a,kin.lower,kin.upper) + q[:,wa]=np.c_[wp,angles];q[:,ja]=kin.expand(a) + oa=[m.jnt_qposadr[m.joint(side+'_object_'+s).id] for s in ['pos_x','pos_y','pos_z','rot_x','rot_y','rot_z']];q[:,oa]=np.c_[poses[name][:,:3,3],np.unwrap(R.from_matrix(poses[name][:,:3,:3]).as_euler('XYZ'),axis=0)] + # Human fingertips yield geometric contact hypotheses, independently of retargeted hand error. + hp=(human['joints']+human['wrist_world'][:,None,:])@S.T+shift + candidates=[];surface=[] + for candidate in ['upper','lower']: + local=np.einsum('nji,nkj->nki',poses[candidate][:,:3,:3],hp[:,[4,8,12,16,20]]-poses[candidate][:,None,:3,3]) + nearest=scenes[candidate].compute_closest_points(o3d.core.Tensor(local.astype(np.float32).reshape(-1,3)))['points'].numpy().reshape(N,5,3) + distance=np.linalg.norm(nearest-local,axis=-1) + if candidate=='lower':distance[177:]=100. + candidates.append(distance);surface.append(np.einsum('nij,nkj->nki',poses[candidate][:,:3,:3],nearest)+poses[candidate][:,None,:3,3]) + choose=np.argmin(candidates,axis=0);dist=np.min(candidates,axis=0);on=np.zeros(5,dtype=bool) + for f in range(N): + on=np.where(on,dist[f]<.035,dist[f]<.025);contact[f,si*5:si*5+5]=on + for tip in range(5):cp[f,si*5+tip]=surface[choose[f,tip]][f,tip] + fit_report[side]={'contact_frames':int(np.any(contact[:,si*5:si*5+5],axis=1).sum()),'human_tip_distance_median_mm':float(np.median(dist)*1000)} +np.savez_compressed(OUT/'reference_before_contact_fit.npz',qpos=q,time=t) +# Bounded kinematic warm start; subsequent optimization is actual upstream SPIDER. +for si,(side,name) in enumerate([('right','upper'),('left','lower')]): + kin,wn=handmeta[side];wa=[m.jnt_qposadr[m.joint(n).id] for n in wn];ja=[m.jnt_qposadr[m.joint(side+'_'+n).id] for n in kin.joint_names];sid=[m.site(f'{side}_hand_{f}_track').id for f in fingers];errors=[] + handgeom=[g for g in range(m.ngeom) if m.geom_contype[g]==1 and m.geom(g).name.startswith(side+'_')] + for f in range(N): + active=contact[f,si*5:si*5+5].astype(bool) + old=q[f].copy();base=old[ja][kin.independent_indices];target=cp[f,si*5:si*5+5] + def residual(x): + d.qpos[:]=old;d.qpos[wa[:3]]=old[wa[:3]]+x[16:];d.qpos[ja]=kin.expand(x[:16]);mujoco.mj_fwdPosition(m,d) + penetrations=np.zeros(len(handgeom));lookup={g:k for k,g in enumerate(handgeom)} + for cc in d.contact: + g0,g1=map(int,cc.geom) + if g0 in lookup and m.geom_contype[g1]==2:penetrations[lookup[g0]]=max(penetrations[lookup[g0]],-cc.dist-.001) + if g1 in lookup and m.geom_contype[g0]==2:penetrations[lookup[g1]]=max(penetrations[lookup[g1]],-cc.dist-.001) + return np.r_[penetrations/.002,(d.site_xpos[sid][active]-target[active]).ravel()/.008,.15*(x[:16]-base),.3*x[16:]/.03] + fit=least_squares(residual,np.r_[base,np.zeros(3)],bounds=(np.r_[kin.lower,[-.05]*3],np.r_[kin.upper,[.05]*3]),max_nfev=30,diff_step=1e-4) + residual(fit.x);q[f]=d.qpos; + if active.any():errors.append(np.linalg.norm(d.site_xpos[sid][active]-target[active],axis=-1).mean()*1000) + if f%80==0:print('warmstart',side,f,flush=True) + fit_report[side]['warmstart_contact_gap_mean_mm']=float(np.mean(errors)) if errors else None + print('fit',side,fit_report[side],flush=True) +# Interpolate unwrapped scalar coordinates, with exact mimic preserved by linear interpolation. +dt=.0025;times=np.arange(int(np.ceil(t[-1]/.1)*.1/dt)+1)*dt +qi=np.stack([np.interp(times,t,q[:,j]) for j in range(m.nq)],1);ci=np.stack([np.interp(times,t,cp.reshape(N,-1)[:,j]) for j in range(30)],1).reshape(-1,10,3) +nearestidx=np.minimum(np.searchsorted(t,times),N-1);ct=contact[nearestidx];vel=np.gradient(qi,dt,axis=0);vel[0]=0 +ctrl=qi[:,m.jnt_qposadr[m.actuator_trnid[:,0]]];siteids=[m.site(f'{s}_hand_{f}_track').id for s in ['right','left'] for f in fingers] +np.savez_compressed(TRIAL/'trajectory_kinematic_act.npz',qpos=qi,qvel=vel,ctrl=ctrl,contact=ct,contact_pos=ci,time=times) +np.savez_compressed(OUT/'reference_video_rate.npz',qpos=q,contact=contact,contact_pos=cp,time=t,camera_world_from_cv=world_from_source[None]@fp['c2w']) +(TASK/'task_info.json').write_text(json.dumps({'ref_dt':dt,'contact_site_ids':siteids},indent=2)) +c=json.loads((ROOT/'output/spider_l20_contact/config.json').read_text());c.update(dataset_dir=str(OUT/'datasets'),task='boxes',embodiment_type='bimanual',object_action_dims=12,num_samples=96,max_num_iterations=3,horizon=.4,max_sim_steps=len(times)-1,nconmax_per_env=512,njmax_per_env=1536,contact_rew_scale=2.,pos_rew_scale=10.) +(OUT/'config.json').write_text(json.dumps(c,indent=2));report=dict(nq=m.nq,nv=m.nv,nu=m.nu,frames=N,duration_s=float(times[-1]),table_plane=np.asarray(plane).tolist(),table_inliers=len(ii),source_world_to_sim=world_from_source.tolist(),object_provenance=provenance,contact=fit_report,contact_rule='Human fingertip to CAD distance: engage25mm release35mm; nearest observed object per fingertip; lower excluded after176. Geometric hypothesis, no tactile labels.',mass_assumption_kg=.12,friction_assumption=.8,collision='20 CoACD convex parts per object; approximate concavity',warmstart='Bounded finger+translation least-squares with collision penalty; not physics and not SPIDER',object_joint_armature=.001,object_real_actuator_gains_zero=bool(not m.actuator_gainprm[-12:].any())) +(OUT/'preparation.json').write_text(json.dumps(report,indent=2));print(json.dumps(report,indent=2),flush=True) diff --git a/scripts/probe_collision_dynamics.py b/scripts/probe_collision_dynamics.py new file mode 100644 index 0000000..bf88342 --- /dev/null +++ b/scripts/probe_collision_dynamics.py @@ -0,0 +1,22 @@ +"""Small full-clip CPU replay to choose bounded drive and contact settings before GPU rerun.""" +import os +os.environ['OPENBLAS_NUM_THREADS']='1';os.environ['OMP_NUM_THREADS']='1' +from pathlib import Path +import json,time,numpy as np,mujoco +O=Path(__file__).resolve().parents[1]/'output/collision_fix_20260915';T=O/'datasets/processed/current/l20/bimanual/boxes';r=np.load(T/'0/trajectory_kinematic_act.npz');rq=r['qpos'];rv=r['qvel'];rc=r['ctrl'];a=np.load(O/'physics_attempt1/trajectory_mjwp_act.npz');ctrl=a['ctrl'].reshape(-1,56);allreports=[] +for name,drive,hard,sub in [('hard_only',1.,True,1),('bounded_drive',.1,True,1),('bounded_half_dt',.1,True,2),('softer_drive',.03,True,2)]: + m=mujoco.MjModel.from_xml_path(str(T/'scene_act.xml'));m.opt.iterations=80;m.opt.ls_iterations=50;m.opt.timestep=.0025/sub + if hard: + ids=np.flatnonzero(np.isin(m.geom_contype,[1,2]));m.geom_solimp[ids,:]=[.999,.9999,.0001,.5,2];m.geom_solref[ids,:]=[.005,1] + for i in range(m.nu-12): + n=m.actuator(i).name + if '_hand_pos_' in n:kp,kv,limit=500,30,200*drive + elif '_hand_rot_' in n:kp,kv,limit=15,2,20*drive + else:kp,kv,limit=4,.15,drive + m.actuator_gainprm[i,0]=kp;m.actuator_biasprm[i,1]=-kp;m.actuator_biasprm[i,2]=-kv;m.actuator_forcerange[i]=[-limit,limit] + d=mujoco.MjData(m);d.qpos[:]=rq[0];d.qvel[:]=rv[0];d.ctrl[:]=rc[0];mujoco.mj_step(m,d);pen=[];rows=[];start=time.monotonic() + for f,u in enumerate(ctrl): + d.ctrl[:]=u + for _ in range(sub):mujoco.mj_step(m,d) + mujoco.mj_kinematics(m,d);mujoco.mj_collision(m,d);cs=[c for c in d.contact if {int(m.geom_contype[c.geom[0]]),int(m.geom_contype[c.geom[1]])}=={1,2}];pen.append(max([max(0,-c.dist) for c in cs],default=0)*1000);rows.append(d.qpos.copy()) + report=dict(name=name,drive_scale=drive,hard=hard,substeps=sub,max_penetration_mm=float(max(pen)),steps_over_1mm=int(np.sum(np.array(pen)>1)),finite=bool(np.isfinite(rows).all()),wall_s=time.monotonic()-start);allreports.append(report);np.savez_compressed(O/(name+'_cpu_probe.npz'),qpos=rows,max_penetration_mm=pen);(O/'dynamics_probe.json').write_text(json.dumps(allreports,indent=2));print(report,flush=True) diff --git a/scripts/probe_spider_gpu_contact.py b/scripts/probe_spider_gpu_contact.py new file mode 100644 index 0000000..1dff57a --- /dev/null +++ b/scripts/probe_spider_gpu_contact.py @@ -0,0 +1,21 @@ +"""Isolate early CPU/GPU contact instability with identical saved controls.""" +import os,json +from pathlib import Path +ROOT=Path(__file__).resolve().parents[1];O=ROOT/'output/spider_dynamics_fix_20260915';T=O/'datasets/processed/current/l20/bimanual/boxes' +os.environ['WARP_CACHE_PATH']=str(ROOT/'.spider_cache/warp');os.environ['OMP_NUM_THREADS']='2' +import numpy as np,mujoco,mujoco_warp as mw,warp as wp +r=np.load(T/'0/trajectory_kinematic_act.npz');rq,rv,rc=(r[k] for k in ['qpos','qvel','ctrl']);u=np.load(O/'hard_contact_attempt/control_replay.npz')['ctrl'][:80];wp.init();reports=[] +variants=[('hard',.999,.9999,.0001,.005,2),('medium',.95,.99,.001,.005,2),('default',.9,.95,.001,.005,2),('softtime',.9,.95,.001,.01,2),('cg',.95,.99,.001,.005,1),('tau20',.9,.95,.001,.02,2),('tau40',.9,.95,.001,.04,2),('imp90',.8,.9,.001,.01,2),('imp80',.7,.8,.001,.01,2),('buffer_tau10',.9,.95,.001,.01,2),('buffer_tau20',.9,.95,.001,.02,2)] +for name,dmin,dmax,width,tau,solver in variants: + m=mujoco.MjModel.from_xml_path(str(O/'hard_contact_attempt/scene_act.xml'));m.opt.iterations=80;m.opt.ls_iterations=50;m.opt.solver=solver;m.opt.integrator=mujoco.mjtIntegrator.mjINT_IMPLICITFAST;m.geom_solimp[:]=[dmin,dmax,width,.5,2];m.geom_solref[:]=[tau,1] + if name.startswith('buffer'):m.geom_gap[m.geom_contype>0]=.002 + d=mujoco.MjData(m);d.qpos[:]=rq[0];d.qvel[:]=rv[0];d.ctrl[:]=rc[0];mujoco.mj_step(m,d);cpu=[];gpu=[] + with wp.ScopedDevice('cuda:0'): + wm=mw.put_model(m);wd=mw.put_data(m,d,nworld=2,nconmax=1024,njmax=3072) + with wp.ScopedCapture() as cap:mw.step(wm,wd) + for ctrl in u: + wd.ctrl.assign(np.tile(ctrl.astype(np.float32),(2,1)));wp.capture_launch(cap.graph);wp.synchronize();gpu.append(wd.qpos.numpy()[0].copy());d.ctrl[:]=ctrl;mujoco.mj_step(m,d);cpu.append(d.qpos.copy()) + gpu=np.array(gpu);cpu=np.array(cpu);pen=[];shift=[] + for q in gpu: + d.qpos[:]=q;mujoco.mj_fwdPosition(m,d);pen.append(max([max(0,-c.dist) for c in d.contact if {int(m.geom_contype[c.geom[0]]),int(m.geom_contype[c.geom[1]])}=={1,2}],default=0)*1000);shift.append([d.xipos[m.body(n).id].copy() for n in ['right_object','left_object']]) + shift=np.array(shift);report={'name':name,'max_cpu_gpu_qpos_difference':float(np.max(abs(gpu-cpu))),'max_penetration_mm':float(max(pen)),'max_object_com_displacement_mm':float(np.max(np.linalg.norm(shift-shift[0],axis=2))*1000),'finite':bool(np.isfinite(gpu).all())};reports.append(report);np.savez_compressed(O/('contact_probe_'+name+'.npz'),gpu=gpu,cpu=cpu);(O/'gpu_contact_probe.json').write_text(json.dumps(reports,indent=2));print(report,flush=True) diff --git a/scripts/red_box_distance.py b/scripts/red_box_distance.py new file mode 100644 index 0000000..3b30796 --- /dev/null +++ b/scripts/red_box_distance.py @@ -0,0 +1,18 @@ +"""Unsigned closest distance with solid-angle winding sign for a closed mesh. +Avoid ray-parity misclassification at collinear CAD tessellation edges. +""" +import numpy as np +import open3d as o3d + +def signed_distance(scene,mesh,points): + p=np.asarray(points,dtype=np.float64) + d=scene.compute_distance(o3d.core.Tensor(p.astype('float32'))).numpy() + ids=np.flatnonzero(((p>=mesh.bounds[0])&(p<=mesh.bounds[1])).all(1)) + triangles=np.asarray(mesh.triangles,dtype=np.float64) + for start in range(0,len(ids),32): + ix=ids[start:start+32];v=triangles[None]-p[ix,None,None,:];a,b,c=v[:,:,0],v[:,:,1],v[:,:,2];length=np.linalg.norm(v,axis=-1) + numerator=np.einsum('bij,bij->bi',a,np.cross(b,c)) + denominator=length.prod(-1)+(a*b).sum(-1)*length[:,:,2]+(b*c).sum(-1)*length[:,:,0]+(c*a).sum(-1)*length[:,:,1] + winding=np.arctan2(numerator,denominator).sum(-1)/(2*np.pi) + d[ix[np.abs(winding)>.5]]*=-1 + return d diff --git a/scripts/refit_rgbd_objects_outline.py b/scripts/refit_rgbd_objects_outline.py new file mode 100644 index 0000000..49e9a94 --- /dev/null +++ b/scripts/refit_rgbd_objects_outline.py @@ -0,0 +1,60 @@ +"""Refit CAD poses from observed outlines and robust depth planes; fixed CAD scale.""" +import os +os.environ.setdefault('OMP_NUM_THREADS','4') +from pathlib import Path +import json,cv2,numpy as np,open3d as o3d +from scipy.spatial.transform import Rotation,Slerp +ROOT=Path(__file__).resolve().parents[1];BASE=ROOT/'output/20260915_171525_dynhamr';OUT=BASE/'object_refit';OUT.mkdir(exist_ok=True) +meta=json.loads((ROOT/'docs/20260915_171525/intrinsics.json').read_text());cam=np.load(BASE/'rgbd_camera.npz');old=np.load(BASE/'object_poses.npz');N=352 +K=np.array([[meta['fx'],0,meta['cx']],[0,meta['fy'],meta['cy']],[0,0,1.]]) +meshes={n:np.asarray(o3d.io.read_triangle_mesh(str(BASE/'object_assets'/f'{n}.stl')).vertices) for n in ['upper','lower']} +poses={n:[] for n in meshes};records={n:[] for n in meshes};cap=cv2.VideoCapture(str(ROOT/'docs/20260915_171525/color.mp4')) +def project(v,T): + p=v@T[:3,:3].T+T[:3,3];return (p@K.T)[:,:2]/p[:,2,None] +def score(v,T,mask): + uv=project(v,T); hull=cv2.convexHull(np.rint(uv).astype(np.int32)); pred=np.zeros_like(mask);cv2.fillConvexPoly(pred,hull,1) + obs=np.zeros_like(mask);cs,_=cv2.findContours(mask,cv2.RETR_EXTERNAL,cv2.CHAIN_APPROX_SIMPLE);cv2.fillConvexPoly(obs,cv2.convexHull(max(cs,key=cv2.contourArea)),1) + return float(np.sum((pred>0)&(obs>0))/max(1,np.sum((pred>0)|(obs>0)))) +for t in range(N): + ok,im=cap.read();assert ok + hsv=cv2.cvtColor(im,cv2.COLOR_BGR2HSV);dep=cv2.imread(str(ROOT/'docs/20260915_171525/depth'/f'{t:06d}.png'),-1)*meta['depth_scale_m'] + for n,v in meshes.items(): + hue=hsv[:,:,0]; mask=(((hue<12)|(hue>170)) if n=='upper' else ((hue>90)&(hue<135)))&(hsv[:,:,1]>90)&(hsv[:,:,2]>35)&(dep>.12)&(dep<.85);mask[:80]=False + num,labels,stats,_=cv2.connectedComponentsWithStats(mask.astype('uint8'),8);label=1+np.argmax(stats[1:,4]);mask=(labels==label).astype('uint8') + cs,_=cv2.findContours(mask,cv2.RETR_EXTERNAL,cv2.CHAIN_APPROX_SIMPLE);hull=cv2.convexHull(max(cs,key=cv2.contourArea));poly=None + for eps in [.015,.025,.04,.06]: + q=cv2.approxPolyDP(hull,eps*cv2.arcLength(hull,True),True) + if len(q)==4:poly=q[:,0].astype(float);break + if poly is None:poly=cv2.boxPoints(cv2.minAreaRect(hull)).astype(float) + center=poly.mean(0);poly=poly[np.argsort(np.arctan2(poly[:,1]-center[1],poly[:,0]-center[0]))];poly=np.roll(poly,-np.argmin(poly.sum(1)),axis=0) + # Robust plane uses only colored measured pixels, never white insert/background hull fill. + yy,xx=np.nonzero(cv2.erode(mask,np.ones((3,3),np.uint8)));z=dep[yy,xx];pts=np.c_[(xx-meta['cx'])/meta['fx']*z,(yy-meta['cy'])/meta['fy']*z,z] + pc=o3d.geometry.PointCloud(o3d.utility.Vector3dVector(pts[::3]));plane,inliers=pc.segment_plane(.004,3,100);normal=np.array(plane[:3]);rays=np.c_[poly,np.ones(4)]@np.linalg.inv(K).T;corners=rays*(-plane[3]/(rays@normal))[:,None] + lo=v.min(0);hi=v.max(0) + if n=='upper':local=np.array([[lo[0],lo[1],hi[2]],[hi[0],lo[1],hi[2]],[hi[0],lo[1],lo[2]],[lo[0],lo[1],lo[2]]]) + else:local=np.array([[lo[0],-.0172,lo[2]],[hi[0],-.0172,lo[2]],[hi[0],-.0172,hi[2]],[lo[0],-.0172,hi[2]]]) + A=local-local.mean(0);B=corners-corners.mean(0);u,s,vt=np.linalg.svd(A.T@B);rr=vt.T@np.diag([1,1,np.linalg.det(vt.T@u.T)])@u.T + T=np.eye(4);T[:3,:3]=rr;T[:3,3]=corners.mean(0)-rr@local.mean(0) + before=np.eye(4);before[:3,:3]=Rotation.from_quat(old[n+'_quaternion_xyzw'][t]).as_matrix();before[:3,3]=old[n+'_position_world'][t];before=np.linalg.inv(cam['c2w'][t])@before + candidates=[T] + for shift in [0,2]: + success,rv,tv=cv2.solvePnP(local.astype(float),np.roll(poly,shift,axis=0).astype(float),K,None,flags=cv2.SOLVEPNP_ITERATIVE) + if success: + candidate=np.eye(4);candidate[:3,:3]=Rotation.from_rotvec(rv.ravel()).as_matrix();candidate[:3,3]=tv.ravel();candidates.append(candidate) + T=max(candidates,key=lambda candidate:score(v,candidate,mask)-2*abs((local@candidate[:3,:3].T+candidate[:3,3])[:,2].mean()-corners[:,2].mean())) + rr=T[:3,:3] + iou0=score(v,before,mask);iou1=score(v,T,mask);rmse=np.sqrt(np.mean(np.sum((local@rr.T+T[:3,3]-corners)**2,axis=1))) + accepted=bool(iou1>.65 and rmse<.06 and len(inliers)>len(pts[::3])*.5) + poses[n].append(cam['c2w'][t]@T);records[n].append(dict(frame=t,accepted=accepted,old_outline_iou=iou0,new_outline_iou=iou1,corner_fit_rmse_m=float(rmse))) + if t in [0,176,351]: + pic=im.copy() + for transform,color in [(before,(0,200,255)),(T,(0,255,0))]:cv2.polylines(pic,[cv2.convexHull(np.rint(project(v,transform)).astype('int32'))],True,color,2) + cv2.imwrite(str(OUT/f'{n}_{t:04d}_overlay.jpg'),pic) + if t%80==0:print('refit',t,flush=True) +cap.release();(OUT/'all_candidates.json').write_text(json.dumps(records));arrays={'time':cam['time']};summary={} +for n in meshes: + P=np.asarray(poses[n]);good=np.array([r['accepted'] for r in records[n]]);idx=np.flatnonzero(good);assert len(idx)>N*.5,(n,len(idx)) + ts=np.clip(np.arange(N),idx[0],idx[-1]);p=np.stack([np.interp(ts,idx,P[idx,k,3]) for k in range(3)],axis=1);rot=Slerp(idx,Rotation.from_matrix(P[idx,:3,:3]))(ts) + arrays.update({n+'_position_world':p,n+'_quaternion_xyzw':rot.as_quat(),n+'_keyframes':np.arange(N),n+'_keyframe_valid':good}) + summary[n]=dict(accepted=int(good.sum()),old_iou_median=float(np.median([r['old_outline_iou'] for r in records[n]])),new_iou_median=float(np.median([r['new_outline_iou'] for r in records[n]])),frames=records[n]) +np.savez_compressed(OUT/'object_poses.npz',**arrays);(OUT/'validation.json').write_text(json.dumps(summary,indent=2));print({n:{k:v for k,v in s.items() if k!='frames'} for n,s in summary.items()}) diff --git a/scripts/refit_rgbd_objects_tracked.py b/scripts/refit_rgbd_objects_tracked.py new file mode 100644 index 0000000..b7d5736 --- /dev/null +++ b/scripts/refit_rgbd_objects_tracked.py @@ -0,0 +1,94 @@ +"""Refit CAD poses from observed outlines and robust depth planes; fixed CAD scale.""" +import os +os.environ.setdefault('OMP_NUM_THREADS','4') +from pathlib import Path +import json,cv2,numpy as np,open3d as o3d +from scipy.spatial.transform import Rotation,Slerp +ROOT=Path(__file__).resolve().parents[1];BASE=ROOT/'output/20260915_171525_dynhamr';OUT=BASE/'object_refit_tracked';OUT.mkdir(exist_ok=True) +meta=json.loads((ROOT/'docs/20260915_171525/intrinsics.json').read_text());cam=np.load(BASE/'rgbd_camera.npz');old=np.load(BASE/'object_poses.npz');N=352 +K=np.array([[meta['fx'],0,meta['cx']],[0,meta['fy'],meta['cy']],[0,0,1.]]) +meshes={n:np.asarray(o3d.io.read_triangle_mesh(str(BASE/'object_assets'/f'{n}.stl')).vertices) for n in ['upper','lower']} +poses={n:[] for n in meshes};records={n:[] for n in meshes};cap=cv2.VideoCapture(str(ROOT/'docs/20260915_171525/color.mp4')) +def project(v,T): + p=v@T[:3,:3].T+T[:3,3];return (p@K.T)[:,:2]/p[:,2,None] +def score(v,T,mask): + uv=project(v,T); hull=cv2.convexHull(np.rint(uv).astype(np.int32)); pred=np.zeros_like(mask);cv2.fillConvexPoly(pred,hull,1) + obs=np.zeros_like(mask);cs,_=cv2.findContours(mask,cv2.RETR_EXTERNAL,cv2.CHAIN_APPROX_SIMPLE);cv2.fillConvexPoly(obs,cv2.convexHull(max(cs,key=cv2.contourArea)),1) + return float(np.sum((pred>0)&(obs>0))/max(1,np.sum((pred>0)|(obs>0)))) +polygons={'upper':np.array([[491,268],[601,275],[631,399],[488,379]],dtype=np.float32),'lower':np.array([[309,254],[436,264],[442,382],[291,375]],dtype=np.float32)} +previous_gray=None +face_choice={} +for t in range(N): + ok,im=cap.read();assert ok + gray=cv2.cvtColor(im,cv2.COLOR_BGR2GRAY) + if previous_gray is not None: + for n,poly in polygons.items(): + featuremask=np.zeros_like(gray);cv2.fillConvexPoly(featuremask,poly.astype('int32'),255) + points=cv2.goodFeaturesToTrack(previous_gray,300,.01,5,mask=featuremask) + if points is not None and len(points)>=8: + nxt,st,_=cv2.calcOpticalFlowPyrLK(previous_gray,gray,points,None,winSize=(21,21),maxLevel=3) + back,st2,_=cv2.calcOpticalFlowPyrLK(gray,previous_gray,nxt,None,winSize=(21,21),maxLevel=3) + good=(st.ravel()>0)&(st2.ravel()>0)&(np.linalg.norm(back-points,axis=(1,2))<1.) + if good.sum()>=8: + aff,inl=cv2.estimateAffinePartial2D(points[good],nxt[good],method=cv2.RANSAC,ransacReprojThreshold=2.) + H=None if aff is None else np.vstack([aff,[0,0,1]]) + if H is not None and inl.sum()>=8: + proposed=cv2.perspectiveTransform(poly[None],H)[0] + if np.max(np.linalg.norm(proposed-poly,axis=1))<30:polygons[n]=proposed + previous_gray=gray + hsv=cv2.cvtColor(im,cv2.COLOR_BGR2HSV);dep=cv2.imread(str(ROOT/'docs/20260915_171525/depth'/f'{t:06d}.png'),-1)*meta['depth_scale_m'] + for n,v in meshes.items(): + hue=hsv[:,:,0]; mask=(((hue<12)|(hue>170)) if n=='upper' else ((hue>90)&(hue<135)))&(hsv[:,:,1]>90)&(hsv[:,:,2]>35)&(dep>.12)&(dep<.85);mask[:80]=False + mask &= ~(cv2.inRange(cv2.cvtColor(im,cv2.COLOR_BGR2YCrCb),np.array([0,133,77]),np.array([255,173,127]))>0) + num,labels,stats,_=cv2.connectedComponentsWithStats(mask.astype('uint8'),8);label=1+np.argmax(stats[1:,4]);mask=(labels==label).astype('uint8') + cs,_=cv2.findContours(mask,cv2.RETR_EXTERNAL,cv2.CHAIN_APPROX_SIMPLE);hull=cv2.convexHull(max(cs,key=cv2.contourArea)) + if cv2.contourArea(hull)>6500 and mask.sum()/cv2.contourArea(hull)>.55: + for eps in [.015,.025,.04,.06]: + quad=cv2.approxPolyDP(hull,eps*cv2.arcLength(hull,True),True) + if len(quad)==4: + poly=quad[:,0].astype('float32');center=poly.mean(0);poly=poly[np.argsort(np.arctan2(poly[:,1]-center[1],poly[:,0]-center[0]))];poly=np.roll(poly,-np.argmin(poly.sum(1)),axis=0) + poly=min([np.roll(poly,k,axis=0) for k in range(4)],key=lambda q:np.sum((q-polygons[n])**2)) + polygons[n]=poly;break + poly=polygons[n].astype(float) + observed=np.zeros_like(mask);cv2.fillConvexPoly(observed,poly.astype('int32'),1) + mask=mask*observed + # Robust plane uses only colored measured pixels, never white insert/background hull fill. + yy,xx=np.nonzero(cv2.erode(mask,np.ones((3,3),np.uint8)));z=dep[yy,xx];pts=np.c_[(xx-meta['cx'])/meta['fx']*z,(yy-meta['cy'])/meta['fy']*z,z] + if len(pts)<30:pts=np.array([[0,0,.5],[.1,0,.5],[0,.1,.5]]) + pc=o3d.geometry.PointCloud(o3d.utility.Vector3dVector(pts[::max(1,len(pts)//1500)]));plane,inliers=pc.segment_plane(.004,3,100);normal=np.array(plane[:3]);rays=np.c_[poly,np.ones(4)]@np.linalg.inv(K).T;corners=rays*(-plane[3]/(rays@normal))[:,None] + lo=v.min(0);hi=v.max(0) + if n=='upper':local=np.array([[lo[0],lo[1],hi[2]],[hi[0],lo[1],hi[2]],[hi[0],lo[1],lo[2]],[lo[0],lo[1],lo[2]]]) + else:local=np.array([[lo[0],-.0172,lo[2]],[hi[0],-.0172,lo[2]],[hi[0],-.0172,hi[2]],[lo[0],-.0172,hi[2]]]) + A=local-local.mean(0);B=corners-corners.mean(0);u,s,vt=np.linalg.svd(A.T@B);rr=vt.T@np.diag([1,1,np.linalg.det(vt.T@u.T)])@u.T + T=np.eye(4);T[:3,:3]=rr;T[:3,3]=corners.mean(0)-rr@local.mean(0) + before=np.eye(4);before[:3,:3]=Rotation.from_quat(old[n+'_quaternion_xyzw'][t]).as_matrix();before[:3,3]=old[n+'_position_world'][t];before=np.linalg.inv(cam['c2w'][t])@before + candidates=[] + for face in [lo[1],hi[1]]: + for flip in [False,True]: + loc=local.copy();loc[:,1]=face + if flip:loc=loc[[3,2,1,0]] + success,rv,tv=cv2.solvePnP(loc.astype(float),poly.astype(float),K,None,flags=cv2.SOLVEPNP_ITERATIVE) + if success: + candidate=np.eye(4);candidate[:3,:3]=Rotation.from_rotvec(rv.ravel()).as_matrix();candidate[:3,3]=tv.ravel();candidates.append(candidate) + previous=None if not poses[n] else np.linalg.inv(cam['c2w'][t])@poses[n][-1] + if n not in face_choice:face_choice[n]=int(np.argmax([score(v,candidate,observed) for candidate in candidates])) + T=candidates[face_choice[n]] + rr=T[:3,:3] + iou0=score(v,before,observed);iou1=score(v,T,observed);rmse=np.sqrt(np.mean(np.sum((local@rr.T+T[:3,3]-corners)**2,axis=1))) + center_step=0 if previous is None else np.linalg.norm((T[:3,:3]@v.mean(0)+T[:3,3])-(previous[:3,:3]@v.mean(0)+previous[:3,3])) + angle_step=0 if previous is None else Rotation.from_matrix(previous[:3,:3].T@T[:3,:3]).magnitude() + accepted=bool(iou1>.60 and np.isfinite(T).all() and center_step<.04 and angle_step<.3) + # Keep candidate for continuity diagnostics; rejected poses are excluded at interpolation. + poses[n].append(cam['c2w'][t]@T);records[n].append(dict(frame=t,accepted=accepted,old_outline_iou=iou0,new_outline_iou=iou1,corner_fit_rmse_m=float(rmse))) + if t in [0,176,351]: + pic=im.copy() + for transform,color in [(before,(0,200,255)),(T,(0,255,0))]:cv2.polylines(pic,[cv2.convexHull(np.rint(project(v,transform)).astype('int32'))],True,color,2) + cv2.imwrite(str(OUT/f'{n}_{t:04d}_overlay.jpg'),pic) + if t%80==0:print('refit',t,flush=True) +cap.release();(OUT/'all_candidates.json').write_text(json.dumps(records));arrays={'time':cam['time']};summary={} +for n in meshes: + P=np.asarray(poses[n]);good=np.array([r['accepted'] for r in records[n]]);idx=np.flatnonzero(good);assert len(idx)>N*.5,(n,len(idx)) + ts=np.clip(np.arange(N),idx[0],idx[-1]);p=np.stack([np.interp(ts,idx,P[idx,k,3]) for k in range(3)],axis=1);rot=Slerp(idx,Rotation.from_matrix(P[idx,:3,:3]))(ts) + arrays.update({n+'_position_world':p,n+'_quaternion_xyzw':rot.as_quat(),n+'_keyframes':np.arange(N),n+'_keyframe_valid':good}) + summary[n]=dict(accepted=int(good.sum()),old_iou_median=float(np.median([r['old_outline_iou'] for r in records[n]])),new_iou_median=float(np.median([r['new_outline_iou'] for r in records[n]])),frames=records[n]) +np.savez_compressed(OUT/'object_poses.npz',**arrays);(OUT/'validation.json').write_text(json.dumps(summary,indent=2));print({n:{k:v for k,v in s.items() if k!='frames'} for n,s in summary.items()}) diff --git a/scripts/register_hands_rgbd.py b/scripts/register_hands_rgbd.py new file mode 100644 index 0000000..21e1901 --- /dev/null +++ b/scripts/register_hands_rgbd.py @@ -0,0 +1,122 @@ +"""Register yesterday's Dyn-HaMR hands to the RGB-D odometry world used by the object tracks. + +Three corrections, all rigid/similarity, no articulation change: +1. World frame: Dyn-HaMR world differs from the RGB-D odometry world (object tracks, table plane) + by a constant translation (~12.6 cm). Hands are re-expressed in the odometry world. +2. Scale: MANO silhouettes match the image at the wrong depth (right hand ~10% too close), + so a per-hand constant scale about the camera centre fixes depth while keeping the silhouette. +3. Residual per-frame depth shift along the wrist ray (smoothed), from the held-out depth fit. +Outputs go to output/20260915_171525_dynhamr/registered/ ; nothing upstream is overwritten. +""" +import os, sys, json, shutil +os.environ.setdefault('OMP_NUM_THREADS', '4') +from pathlib import Path +ROOT = Path(__file__).resolve().parents[1] +sys.path.insert(0, str(ROOT / 'third_party/Dyn-HaMR/dyn-hamr')); sys.path.insert(0, str(ROOT / 'third_party/FoundationPose')); sys.path.insert(0, str(ROOT / 'scripts')) +import numpy as np, cv2, torch, trimesh, open3d as o3d, nvdiffrast.torch as dr +from scipy.ndimage import gaussian_filter1d +from body_model import MANO, run_mano +from Utils import nvdiffrast_render +from red_box_distance import signed_distance + +SRC = ROOT / 'docs/20260915_171525'; BASE = ROOT / 'output/20260915_171525_dynhamr' +PRIOR = BASE / 'corrected/prior/20260915_171525_000000_world_results.npz' +DIAG = ROOT / 'output/hand_depth_diagnosis_20260915/frame_metrics.json' +FP = ROOT / 'output/foundationpose_spider_20260915/foundationpose_objects.npz' +OUT = Path(os.environ.get('HF_REG_OUT', BASE / 'registered')); OUT.mkdir(exist_ok=True) +CAMERA = Path(os.environ.get('HF_CAMERA_FILE', BASE / 'rgbd_camera.npz')); ALIGN = os.environ.get('HF_ALIGN', 'wrist') # 'wrist' | 'mcp' +N = 352; torch.set_num_threads(4) +meta = json.loads((SRC / 'intrinsics.json').read_text()); K = np.array([[meta['fx'], 0, meta['cx']], [0, meta['fy'], meta['cy']], [0, 0, 1.]]) + +p = dict(np.load(PRIOR)); cam = np.load(CAMERA); fp = np.load(FP); fm = json.load(open(DIAG)) +model = MANO(model_path=str(ROOT / 'third_party/Dyn-HaMR/_DATA/data/mano'), batch_size=2 * N, pose2rot=True) +with torch.no_grad(): + o = run_mano(model, *[torch.tensor(p[k]).float() for k in ['trans', 'root_orient', 'pose_body', 'is_right', 'betas']]) +J = o['joints'].numpy(); V = o['vertices'].numpy(); faces = np.asarray(model.faces if hasattr(model, 'faces') else model.bm.faces).astype(np.int32) +sides = ['right' if p['is_right'][b, 0] > .5 else 'left' for b in range(2)] +R = p['cam_R'][0]; tD = p['cam_t'][0]; c2w = cam['c2w'] +w2cR = np.transpose(c2w[:, :3, :3], (0, 2, 1)); tO = -np.einsum('tij,tj->ti', w2cR, c2w[:, :3, 3]) +rot_gap = float(np.abs(R - w2cR).max()); assert rot_gap < (1e-5 if ALIGN == 'wrist' and CAMERA.name == 'rgbd_camera.npz' else 0.2), 'camera rotation differs too much' +offset = np.einsum('tji,tj->ti', R, tD - tO); offset_mean = offset.mean(0) +if CAMERA.name == 'rgbd_camera.npz': assert offset.std(0).max() < .005 +Jc = np.einsum('tij,btkj->btki', R, J) + tD[None, :, None, :]; Vc = np.einsum('tij,btvj->btvi', R, V) + tD[None, :, None, :] + +ctx = dr.RasterizeCudaContext(); eye = torch.eye(4, device='cuda')[None]; ft = torch.as_tensor(faces, device='cuda', dtype=torch.int32) +def render(v): + mt = {'pos': torch.as_tensor(v, device='cuda', dtype=torch.float32), 'faces': ft, 'vnormals': torch.zeros((len(v), 3), device='cuda'), 'vertex_color': torch.ones((len(v), 3), device='cuda')} + with torch.inference_mode(): _, d, _ = nvdiffrast_render(K=K, H=480, W=848, ob_in_cams=eye, glctx=ctx, mesh_tensors=mt) + return d[0].cpu().numpy() +scenes = {} +for name in ['upper', 'lower']: + mesh = trimesh.load(ROOT / f'output/collision_fix_20260915/collision_v2/{name}_closed.ply', process=False); assert mesh.is_watertight + sc = o3d.t.geometry.RaycastingScene(); sc.add_triangles(o3d.t.geometry.TriangleMesh.from_legacy(o3d.geometry.TriangleMesh(o3d.utility.Vector3dVector(mesh.vertices), o3d.utility.Vector3iVector(mesh.faces)))); scenes[name] = (sc, mesh) +def tip_gap(pts_world, name, t): + T = fp[name + '_T_world'][t]; local = (pts_world - T[:3, 3]) @ T[:3, :3] + return float(np.min(np.abs(signed_distance(scenes[name][0], scenes[name][1], local))) * 1000) + +yy, xx = np.indices((480, 848)); train = ((xx // 8 + yy // 8) % 2) == 0 +report = {'camera_file': str(CAMERA), 'l20_alignment': ALIGN, 'offset_dynhamr_to_odometry_world_m': offset_mean.tolist(), 'offset_norm_mm': float(np.linalg.norm(offset_mean) * 1000), 'offset_std_mm': (offset.std(0) * 1000).tolist(), 'hands': {}} +Jw_new = np.zeros_like(J); Vc_new = np.zeros_like(Vc) +for b, side in enumerate(sides): + rows = {r['frame']: r for r in fm[side]} + wz = Jc[b, :, 0, 2]; dz = np.array([rows[t]['fit_shift_z_mm'] / 1000 if rows[t]['depth_supported'] else np.nan for t in range(N)]) + sup = np.isfinite(dz); k = float(np.median(1 + dz[sup] / wz[sup])) + resid = dz - (k - 1) * wz; resid = np.interp(np.arange(N), np.flatnonzero(sup), resid[sup]); resid = gaussian_filter1d(resid, 2) + ray = Jc[b, :, 0] / wz[:, None] + Jc_new = k * Jc[b] + (ray * resid[:, None])[:, None, :]; Vc_new[b] = k * Vc[b] + (ray * resid[:, None])[:, None, :] + Jw_new[b] = np.einsum('tji,tkj->tki', w2cR, Jc_new - tO[:, None, :]) # world of the chosen camera file + # independent check: odometry camera must reproduce the corrected camera coordinates + back = np.einsum('tij,tkj->tki', w2cR, Jw_new[b]) + tO[:, None, :]; assert np.abs(back - Jc_new).max() < 1e-5 + report['hands'][side] = {'scale_about_camera': k, 'depth_supported_frames': int(sup.sum()), 'residual_shift_mm': {'median_abs': float(np.median(np.abs(resid)) * 1000), 'max_abs': float(np.abs(resid).max() * 1000)}, + 'wrist_move_mm': {'median': float(np.median(np.linalg.norm(Jw_new[b, :, 0] - J[b, :, 0], axis=1)) * 1000), 'max': float(np.max(np.linalg.norm(Jw_new[b, :, 0] - J[b, :, 0], axis=1)) * 1000)}} +# held-out depth residual after correction (same masks/method as the diagnosis), plus tip gaps +cap = cv2.VideoCapture(str(SRC / 'color.mp4')); after = {s: [] for s in sides}; gaps = {s: {'before': [], 'after': []} for s in sides}; keep = {60, 175, 240, 300} +pair = {'right': 'upper', 'left': 'lower'} +for t in range(N): + ok, bgr = cap.read(); assert ok + z = cv2.imread(str(SRC / 'depth' / f'{t:06d}.png'), -1).astype('float32') * meta['depth_scale_m'] + hsv = cv2.cvtColor(bgr, cv2.COLOR_BGR2HSV); skin = cv2.inRange(cv2.cvtColor(bgr, cv2.COLOR_BGR2YCrCb), np.array([0, 133, 77]), np.array([255, 173, 127])) > 0 + skin &= ~(((hsv[:, :, 0] < 10) | (hsv[:, :, 0] > 170)) & (hsv[:, :, 1] > 115)); skin &= ~((hsv[:, :, 0] > 95) & (hsv[:, :, 0] < 130) & (hsv[:, :, 1] > 115)) + d1 = [render(Vc_new[b, t]) for b in range(2)] + for b, side in enumerate(sides): + other = d1[1 - b]; occ = (other > .1) & (other < d1[b]); interior = cv2.erode((d1[b] > .1).astype('uint8'), np.ones((5, 5), np.uint8)) > 0 + test = skin & interior & (z > .1) & (z < .85) & ~occ & ~train + if test.sum() >= 100: after[side].append(float(np.median(np.abs(z[test] - d1[b][test])) * 1000)) + name = pair[side]; lim = (t >= 150) if side == 'right' else (t < 177) + if fp[name + '_valid'][t] and lim: + tips = J[b, t, [4, 8, 12, 16, 20]]; gaps[side]['before'].append(tip_gap(tips, name, t)); gaps[side]['after'].append(tip_gap(Jw_new[b, t, [4, 8, 12, 16, 20]], name, t)) + if t in keep: + img = bgr.copy() + for b, col in enumerate([(255, 128, 0), (0, 200, 255)]): + cnts, _ = cv2.findContours((d1[b] > .1).astype('uint8'), cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE); cv2.drawContours(img, cnts, -1, col, 2) + cv2.imwrite(str(OUT / f'overlay_registered_{t:04d}.jpg'), img) + if t % 100 == 0: print('frame', t, flush=True) +cap.release() +def rms2(x): return float(np.sqrt(np.mean(np.sum(np.diff(x, n=2, axis=0) ** 2, axis=-1))) * 1000) +for b, side in enumerate(sides): + before = [r['abs_residual_before_mm'] for r in fm[side] if r['abs_residual_before_mm'] is not None] + report['hands'][side].update({'heldout_abs_residual_mm': {'before_median': float(np.median(before)), 'after_median': float(np.median(after[side])), 'after_p95': float(np.percentile(after[side], 95))}, + 'nearest_tip_to_box_mm': {'pipeline_before_median': float(np.median(gaps[side]['before'])), 'after_median': float(np.median(gaps[side]['after'])), 'frames': len(gaps[side]['after'])}, + 'wrist_second_difference_rms_mm': {'before': rms2(J[b, :, 0]), 'after': rms2(Jw_new[b, :, 0])}}) +# exports: human joints + L20 motion with wrist replaced; finger retargeting is direction-only so joints are unchanged +for b, side in enumerate(sides): + old = dict(np.load(BASE / f'human_joints_{side}.npz', allow_pickle=True)) + old.update(joints=Jw_new[b] - Jw_new[b, :, :1], wrist_world=Jw_new[b, :, 0], source=str(PRIOR), depth_correction_applied=True, + coordinate_note='RGB-D odometry world (first camera frame), same as FoundationPose object tracks. Constant world offset applied; per-hand scale about camera centre; smoothed residual depth shift along wrist ray. Wrist-relative joints scaled accordingly.', + scale_about_camera=np.float64(report['hands'][side]['scale_about_camera']), world_offset_m=offset_mean) + np.savez_compressed(OUT / f'human_joints_{side}.npz', **old) + src = BASE / f'l20_{side}_stable'; dst = OUT / f'l20_{side}_stable'; dst.mkdir(exist_ok=True) + for f in src.iterdir(): + if f.name != 'motion.npz': shutil.copy2(f, dst / f.name) + mo = dict(np.load(src / 'motion.npz', allow_pickle=True)); assert np.abs(mo['wrist_world'] - J[b, :, 0]).max() < 1e-4, 'old motion wrist differs from prior joints' + ww = Jw_new[b, :, 0].copy() + if ALIGN == 'mcp': + # place the L20 so its knuckle row (MCP centroid) coincides with the human's; the wrist retreats by the palm-length difference + v = mo['actual'][:, [5, 9, 13, 17]].mean(1) - mo['actual'][:, 0]; mcp_h = Jw_new[b][:, [5, 9, 13, 17]].mean(1) + ww = mcp_h - np.einsum('tij,tj->ti', mo['wrist_world_R'], v) + report['hands'][side]['mcp_alignment_wrist_retreat_mm'] = {'median': float(np.median(np.linalg.norm(ww - Jw_new[b, :, 0], axis=1)) * 1000), 'max': float(np.max(np.linalg.norm(ww - Jw_new[b, :, 0], axis=1)) * 1000)} + rot = mo['scene_rotation']; wp = (ww - ww[0]) @ rot.T + np.array([0, 0, .2]) + mo.update(wrist_world=ww, wrist_pos=wp, wrist_pos_unsmoothed=wp, scene_translation=np.array([0, 0, .2]) - rot @ ww[0], source=str(OUT / f'human_joints_{side}.npz'), l20_alignment=ALIGN) + np.savez_compressed(dst / 'motion.npz', **mo) + chk = (mo['wrist_pos'] - mo['scene_translation']) @ mo['scene_rotation']; assert np.abs(chk - ww).max() < 1e-6 +(OUT / 'registration.json').write_text(json.dumps(report, indent=2)); print(json.dumps(report, indent=2)) diff --git a/scripts/render_collision_fix.py b/scripts/render_collision_fix.py new file mode 100644 index 0000000..db9a5aa --- /dev/null +++ b/scripts/render_collision_fix.py @@ -0,0 +1,23 @@ +"""Source video versus old and collision-corrected reference; no physics claim.""" +import os +os.environ.setdefault('MUJOCO_GL','osmesa');os.environ['LP_NUM_THREADS']='4' +import sys +from pathlib import Path +import numpy as np,mujoco,cv2,imageio.v2 as imageio +from scipy.spatial.transform import Rotation as R +ROOT=Path(__file__).resolve().parents[1];O=Path(os.environ.get('SPIDER_TASK_OUT',str(ROOT/'output/collision_fix_20260915')));OLD=Path(os.environ.get('HF_FIX_OLD',str(ROOT/'output/foundationpose_spider_20260915')));ref=np.load(O/'reference_video_rate.npz');old=np.load(OLD/'reference_video_rate.npz');m=mujoco.MjModel.from_xml_path(str(O/'datasets/processed/current/l20/bimanual/boxes/scene_act.xml'));d=mujoco.MjData(m);m.vis.quality.offsamples=1;m.vis.headlight.ambient[:]=.7 +physics='--physics' in sys.argv +if physics: + actual=np.load(O/'physics_motion.npz');ref={k:ref[k][ref['time']<=actual['time'][-1]+1e-6] for k in ref.files};ix=np.argmin(abs(actual['time'][:,None]-ref['time'][None,:]),axis=0);old=ref;ref=dict(ref);ref['qpos']=actual['qpos'][ix] +renderer=mujoco.Renderer(m,height=270,width=480);opt=mujoco.MjvOption();opt.sitegroup[:]=0;opt.geomgroup[3:]=0;opt.flags[mujoco.mjtVisFlag.mjVIS_TRANSPARENT]=False;cid=m.camera('source_camera').id +cap=cv2.VideoCapture(str(ROOT/'docs/20260915_171525/color.mp4'));writer=imageio.get_writer(O/('spider_collision_rollout.mp4' if physics else 'hand_collision_before_after.mp4'),fps=30,codec='libx264',quality=8,macro_block_size=1) +for f in range(len(ref['qpos'])): + ok,img=cap.read();assert ok;panels=[cv2.cvtColor(cv2.resize(img,(480,270)),cv2.COLOR_BGR2RGB)];cam=ref['camera_world_from_cv'][f];m.cam_pos[cid]=cam[:3,3];quat=R.from_matrix(cam[:3,:3]@np.diag([1,-1,-1])).as_quat();m.cam_quat[cid]=quat[[3,0,1,2]] + for rows in [old['qpos'],ref['qpos']]: + d.qpos[:]=rows[f];mujoco.mj_kinematics(m,d);mujoco.mj_camlight(m,d);renderer.update_scene(d,camera='source_camera',scene_option=opt);renderer.scene.flags[mujoco.mjtRndFlag.mjRND_SHADOW]=False;panels.append(renderer.render().copy()) + panel=np.concatenate(panels,axis=1);cv2.rectangle(panel,(0,0),(1440,39),(20,25,30),-1) + for j,label in enumerate((['Recorded RGB-D: red + blue','Collision-corrected reference','SPIDER: free-object physical rollout'] if physics else ['Recorded RGB-D: red + blue','Before: old contact reference','After: collision-corrected reference'])):cv2.putText(panel,label,(j*480+8,16),cv2.FONT_HERSHEY_SIMPLEX,.44,(255,255,255),1) + cv2.putText(panel,f'frame {f}/{len(ref["qpos"])-1} | '+('SPIDER physical result - zero object assistance' if physics else 'KINEMATIC COMPARISON - not a free-object grasp rollout'),(8,33),cv2.FONT_HERSHEY_SIMPLEX,.42,(255,255,255),1);writer.append_data(panel) + if f in [0,46,160,240,351]:cv2.imwrite(str(O/(('physics' if physics else 'collision')+f'_comparison_{f:04d}.jpg')),cv2.cvtColor(panel,cv2.COLOR_RGB2BGR)) + if f%80==0:print('render',f,flush=True) +writer.close();cap.release();renderer.close();print('RENDER_COMPLETE',flush=True) diff --git a/scripts/render_corrected_dynhamr_rgbd.py b/scripts/render_corrected_dynhamr_rgbd.py new file mode 100644 index 0000000..af3c04c --- /dev/null +++ b/scripts/render_corrected_dynhamr_rgbd.py @@ -0,0 +1,15 @@ +import os,subprocess,sys,shutil +from pathlib import Path +ROOT=Path(__file__).resolve().parents[1]; dyn=ROOT/'third_party/Dyn-HaMR'; base=ROOT/'output/20260915_171525_dynhamr' +env=os.environ.copy();env.update(TORCH_HOME=str(ROOT/'.torch_cache'),HF_HOME=str(ROOT/'.hf_cache'),HF_HUB_OFFLINE='1',TRANSFORMERS_OFFLINE='1',PYOPENGL_PLATFORM='egl',LD_LIBRARY_PATH='',MPLCONFIGDIR='/tmp/matplotlib-dynhamr',HYDRA_FULL_ERROR='1') +env['DYNHAMR_HIDE_CAMERA_MARKERS']='1' +env['PATH']=str(dyn/'.dynhamr/bin')+':'+env['PATH'];env['PYTHONPATH']=':'.join(str(dyn/p) for p in ['dyn-hamr','third-party/hamer','third-party/hamer/third-party/ViTPose']) +target=base/('raw_comparison' if '--raw' in sys.argv else 'corrected') +if '--raw' in sys.argv: + (target/'prior').mkdir(parents=True,exist_ok=True) + shutil.copy2(base/'optimization/prior/20260915_171525_000000_world_results.npz',target/'prior/20260915_171525_000000_world_results.npz') +cmd=[str(dyn/'.dynhamr/bin/python'),'-u','run_opt.py','data=video_vipe','data.root='+str(base/'dataset'),'data.seq="20260915_171525"','data.vipe_dir='+str(base/'rgbd_cameras'),'data.frame_opts.fps=30','run_opt=False','run_vis=True','temporal_smooth=False','vis.phases=[prior]','hydra.run.dir='+str(base/'corrected')] +cmd[-1]='hydra.run.dir='+str(target) +cmd+=['vis.overwrite=True'] +with (target/'render.log').open('w') as f:r=subprocess.run(cmd,cwd=dyn/'dyn-hamr',env=env,stdout=f,stderr=subprocess.STDOUT) +(target/'render_exit.txt').write_text(str(r.returncode));raise SystemExit(r.returncode) diff --git a/scripts/render_foundationpose_red.py b/scripts/render_foundationpose_red.py new file mode 100644 index 0000000..8fef067 --- /dev/null +++ b/scripts/render_foundationpose_red.py @@ -0,0 +1,39 @@ +"""Render, audit and package the red-box tracking result without rerunning inference.""" +import sys,json,subprocess +from pathlib import Path +ROOT=Path(__file__).resolve().parents[1];sys.path.insert(0,str(ROOT/'third_party/FoundationPose')) +import cv2,numpy as np,torch,trimesh,nvdiffrast.torch as dr +from scipy.spatial.transform import Rotation +from Utils import make_mesh_tensors,nvdiffrast_render +out=ROOT/'output/foundationpose_red_20260916';src=ROOT/'docs/20260916_104026' +a=np.load(out/'poses.npz');P=a['T_camera_from_object'];metrics=json.loads((out/'frame_metrics.json').read_text());N=len(P) +mesh=trimesh.load(ROOT/'docs/上半.stl');mt=make_mesh_tensors(mesh);ctx=dr.RasterizeCudaContext();cap=cv2.VideoCapture(str(src/'color.mp4')) +writer=subprocess.Popen(['ffmpeg','-y','-v','error','-f','rawvideo','-pix_fmt','bgr24','-s','1696x480','-r','30','-i','-','-an','-c:v','libx264','-preset','fast','-crf','20','-pix_fmt','yuv420p','-movflags','+faststart',str(out/'comparison.mp4')],stdin=subprocess.PIPE) +finite=np.isfinite(P).all((1,2));centers=P[:,:3,:3]@mesh.bounds.mean(0)+P[:,:3,3];steps=[];angles=[] +for i in range(N): + ok,b=cap.read();assert ok;pic=b.copy() + if finite[i]: + with torch.inference_mode():_,d,_=nvdiffrast_render(K=a['K'],H=480,W=848,ob_in_cams=torch.as_tensor(P[i:i+1],device='cuda',dtype=torch.float32),glctx=ctx,mesh_tensors=mt) + m=d[0].cpu().numpy()>0 + pic[m]=(pic[m]*.65+np.array([0,220,0])*.35).astype('uint8') + contours,_=cv2.findContours(m.astype('uint8'),cv2.RETR_EXTERNAL,cv2.CHAIN_APPROX_SIMPLE);cv2.drawContours(pic,contours,-1,(0,255,0),1) + if i and finite[i-1]: + steps.append([i,float(np.linalg.norm(centers[i]-centers[i-1]))]);angles.append([i,float(np.rad2deg(Rotation.from_matrix(P[i-1,:3,:3].T@P[i,:3,:3]).magnitude()))]) + label=f'{i:04d} {i/30:.2f}s '+metrics[i]['status'] + cv2.rectangle(pic,(0,0),(848,45),(0,0,0),-1);cv2.putText(pic,label,(12,29),0,.65,(0,255,255),2) + cv2.putText(b,'Original RGB',(12,29),0,.7,(0,255,255),2) + writer.stdin.write(np.hstack([b,pic]).tobytes()) +writer.stdin.close();assert writer.wait()==0;cap.release() +# Independent checks: homogeneous matrices, SO(3), time synchronization, video decoding. +assert np.isfinite(P[finite]).all();assert np.allclose(P[finite,3,:],[0,0,0,1],atol=1e-5) +r=P[finite,:3,:3];ortho=float(np.max(np.abs(r.transpose(0,2,1)@r-np.eye(3))));det=float(np.max(np.abs(np.linalg.det(r)-1)));assert max(ortho,det)<1e-3 +v=cv2.VideoCapture(str(out/'comparison.mp4'));decoded=0 +while v.read()[0]:decoded+=1 +v.release();assert decoded==N +meta=json.loads((src/'intrinsics.json').read_text());deptherr=[x['depth_median_error_m'] for x in metrics if x.get('depth_median_error_m') is not None] +valid=a['accepted'];ranges=[] +for i in np.flatnonzero(~valid): + if ranges and i==ranges[-1][1]+1:ranges[-1][1]=int(i) + else:ranges.append([int(i),int(i)]) +report={'video_frames_decoded':decoded,'pose_matrices':int(finite.sum()),'quality_accepted':int(valid.sum()),'invalid_frame_ranges_inclusive':ranges,'rotation_orthogonality_max_error':ortho,'rotation_determinant_max_error':det,'color_depth_timestamp_difference_max_ms':float(np.max(np.abs(np.array(meta['timestamps_ms'])-meta['depth_timestamps_ms']))),'depth_residual_median_mm':float(np.median(deptherr)*1000),'largest_center_steps_m':sorted(steps,key=lambda x:x[1],reverse=True)[:8],'largest_rotation_steps_deg':sorted(angles,key=lambda x:x[1],reverse=True)[:8],'warning':'Residual measures consistency with input depth, not ground-truth pose error. Heuristic quality flags cannot resolve object symmetries or prove contact correctness.'} +(out/'validation.json').write_text(json.dumps(report,indent=2));print(json.dumps(report,indent=2)) diff --git a/scripts/render_object_alignment_overlay.py b/scripts/render_object_alignment_overlay.py new file mode 100644 index 0000000..1b0c390 --- /dev/null +++ b/scripts/render_object_alignment_overlay.py @@ -0,0 +1,15 @@ +from pathlib import Path +import cv2,json,numpy as np,trimesh +from scipy.spatial.transform import Rotation +B=Path(__file__).resolve().parents[1]/'output/20260915_171525_dynhamr';ROOT=B.parents[1];meta=json.loads((ROOT/'docs/20260915_171525/intrinsics.json').read_text());cam=np.load(B/'rgbd_camera.npz');old=np.load(B/'object_poses.npz');new=np.load(B/'object_aligned/object_poses.npz') +models={n:trimesh.load(B/'object_assets'/f'{n}.stl',force='mesh').vertices for n in ['upper','lower']} +c=cv2.VideoCapture(str(ROOT/'docs/20260915_171525/color.mp4'));out=B/'object_aligned';w=cv2.VideoWriter(str(out/'alignment_overlay.mp4'),cv2.VideoWriter_fourcc(*'mp4v'),30,(848,480)) +for t in range(352): + ok,im=c.read();assert ok + for data,color in [(old,(0,190,255)),(new,(0,255,0))]: + for n,v in models.items(): + p=Rotation.from_quat(data[n+'_quaternion_xyzw'][t]).apply(v)+data[n+'_position_world'][t];p=(p-cam['c2w'][t,:3,3])@cam['c2w'][t,:3,:3];uv=p[:,:2]/p[:,2,None]*[meta['fx'],meta['fy']]+[meta['cx'],meta['cy']] + cv2.polylines(im,[cv2.convexHull(np.rint(uv).astype('int32'))],True,color,2) + cv2.putText(im,'Orange: previous | Green: refitted',(10,25),0,.65,(255,255,255),2);w.write(im) + if t in [0,83,176,351]:cv2.imwrite(str(out/f'overlay_{t:04d}.jpg'),im) +c.release();w.release() diff --git a/scripts/render_red_depth_comparison.py b/scripts/render_red_depth_comparison.py new file mode 100644 index 0000000..7906dde --- /dev/null +++ b/scripts/render_red_depth_comparison.py @@ -0,0 +1,39 @@ +"""Camera overlay and fixed oblique 3D comparison; identical view for both conditions.""" +import sys,json,subprocess +from pathlib import Path +ROOT=Path(__file__).resolve().parents[1];sys.path.insert(0,str(ROOT/'third_party/FoundationPose')) +import cv2,numpy as np,torch,trimesh,nvdiffrast.torch as dr +from Utils import nvdiffrast_render,make_mesh_tensors +from scipy.spatial.transform import Rotation +out=ROOT/'output/depth_ablation_red_20260916';a=np.load(out/'foreground_left_depth_comparison.npz');N=len(a['verts_before']);K=a['K'];P=a['object_pose'];mesh=trimesh.load(out/'red_box_closed.ply');ctx=dr.RasterizeCudaContext();eye=torch.eye(4,device='cuda')[None] +center=np.median(np.einsum('tij,j->ti',P[:,:3,:3],mesh.bounds.mean(0))+P[:,:3,3],axis=0);R=Rotation.from_euler('y',55,degrees=True).as_matrix();virtualK=np.array([[700.,0,424],[0,700,240],[0,0,1.]]) +def render(v,f,color,k): + m=trimesh.Trimesh(vertices=v,faces=f,process=False);m.visual.vertex_colors=np.tile([*color,255],(len(v),1));mt=make_mesh_tensors(m) + with torch.inference_mode():c,d,_=nvdiffrast_render(K=k,H=480,W=848,ob_in_cams=eye,glctx=ctx,mesh_tensors=mt,use_light=True) + return (c[0].cpu().numpy()[:,:,::-1]*255).astype('uint8'),d[0].cpu().numpy() +def scene(b,hand,box,k): + colors=[];depth=[] + for v,f,c in [(box,mesh.faces,[60,185,95]),(hand,a['faces'],[80,145,235])]: + cc,d=render(v,f,c,k);colors.append(cc);depth.append(d) + masks=[d>0 for d in depth];front=masks[1]&((~masks[0])|(depth[1]1e-7)&(t<1-1e-7)&(dist<1e-9));assert len(inside)==2 +faces=m.faces.tolist();hit=[i for i,f in enumerate(faces) if a in f and b in f];assert len(hit)==1 +face=faces.pop(hit[0]);k=next(k for k in range(3) if face[k] in edge and face[(k+1)%3] in edge);a,b,third=face[k],face[(k+1)%3],face[(k+2)%3] +inside=sorted(inside,key=lambda q:np.linalg.norm(v[q]-v[a]));chain=[a,*inside,b];faces.extend([[int(x),int(y),int(third)] for x,y in zip(chain[:-1],chain[1:])]);m=trimesh.Trimesh(vertices=v.copy(),faces=faces,process=False) +assert m.is_watertight and m.is_winding_consistent and m.volume>0 +assert np.array_equal(m.vertices,old.vertices) and abs(m.area-old.area)<1e-10 and abs(m.volume-old.volume)<1e-10 +m.export(out/'red_box_closed.ply') +(out/'mesh_repair.json').write_text(json.dumps({'method':'Split original long boundary edge at two existing collinear vertices (T-junction); no added vertices or geometric hole filling','before_faces':len(old.faces),'after_faces':len(m.faces),'watertight':m.is_watertight,'winding_consistent':m.is_winding_consistent,'area_change_m2':m.area-old.area,'volume_change_m3':m.volume-old.volume},indent=2)) +print('Topology repaired with unchanged geometry',m.is_watertight) diff --git a/scripts/replay_yesterday_spider_gpu.py b/scripts/replay_yesterday_spider_gpu.py new file mode 100644 index 0000000..e9d704b --- /dev/null +++ b/scripts/replay_yesterday_spider_gpu.py @@ -0,0 +1,19 @@ +"""Independent no-optimizer GPU replay and reference-control baseline.""" +import os +from pathlib import Path +ROOT=Path(__file__).resolve().parents[1];OUT=ROOT/'output/foundationpose_spider_20260915';TASK=OUT/'datasets/processed/current/l20/bimanual/boxes' +os.environ['WARP_CACHE_PATH']=str(ROOT/'.spider_cache/warp');os.environ.setdefault('MUJOCO_GL','osmesa') +import numpy as np,json,mujoco,mujoco_warp as mw,warp as wp +r=np.load(TASK/'0/trajectory_kinematic_act.npz');a=np.load(TASK/'0/trajectory_mjwp_act.npz');ctrl=a['ctrl'].reshape(-1,56);expected=a['qpos'].reshape(-1,66) +m=mujoco.MjModel.from_xml_path(str(TASK/'scene_act.xml'));m.opt.iterations=20;m.opt.ls_iterations=50;m.opt.o_solref[:]=[.02,1];m.opt.o_solimp[:]=[0,.95,.03,.5,2] +assert not m.actuator_gainprm[-12:].any() and not m.actuator_biasprm[-12:].any() +d=mujoco.MjData(m);d.qpos[:]=r['qpos'][0];d.qvel[:]=r['qvel'][0];d.ctrl[:]=r['ctrl'][0];mujoco.mj_step(m,d);d.time=0 +wp.init();rows=[] +with wp.ScopedDevice('cuda:0'): + wm=mw.put_model(m);wd=mw.put_data(m,d,nworld=2,nconmax=512,njmax=1536) + with wp.ScopedCapture() as capture:mw.step(wm,wd) + for i,u in enumerate(ctrl): + wd.ctrl.assign(np.stack([r['ctrl'][i],u]).astype(np.float32));wp.capture_launch(capture.graph);wp.synchronize();rows.append(wd.qpos.numpy()) +rows=np.array(rows);np.savez_compressed(OUT/'independent_gpu_replay.npz',baseline=rows[:,0],optimized_controls=rows[:,1]) +report=dict(all_finite=bool(np.isfinite(rows).all()),max_difference_from_native_spider_qpos=float(np.nanmax(np.abs(rows[:,1]-expected))),zero_assistance=True,steps=len(rows),note='Fresh two-world GPU rollout; numerical contact divergence can differ from 96-world MPC replay.') +(OUT/'independent_gpu_replay.json').write_text(json.dumps(report,indent=2));print(json.dumps(report,indent=2),flush=True) diff --git a/scripts/report_collision_fix.py b/scripts/report_collision_fix.py new file mode 100644 index 0000000..de3dd61 --- /dev/null +++ b/scripts/report_collision_fix.py @@ -0,0 +1,12 @@ +from pathlib import Path +import json,numpy as np,mujoco,cv2 +R=Path(__file__).resolve().parents[1];O=R/'output/collision_fix_20260915';T=O/'datasets/processed/current/l20/bimanual/boxes';v=json.loads((O/'validation.json').read_text());phys=json.loads((O/'physics_validation.json').read_text()) if (O/'physics_validation.json').exists() else None +m=mujoco.MjModel.from_xml_path(str(T/'scene_act.xml'));ref=np.load(T/'0/trajectory_kinematic_act.npz');np.savetxt(O/'reference_trajectory.csv',np.c_[ref['time'],ref['qpos'],ref['ctrl']],delimiter=',',header=','.join(['time_s']+[m.joint(i).name for i in range(m.njnt)]+['ctrl_'+m.actuator(i).name for i in range(m.nu)]),comments='') +c=cv2.VideoCapture(str(O/'hand_collision_before_after.mp4'));n=0 +while c.read()[0]:n+=1 +c.release();assert n==352;(O/'video_integrity.json').write_text(json.dumps({'decoded_frames':n,'expected':352,'passed':True},indent=2)) +b=v['before_same_new_collision_model'];a=v['corrected_reference'];lines=['# 双手与红/蓝盒碰撞修复','', '**状态:参考轨迹的手物碰撞修复已通过;Spider 自由物体动态抓取验证仍失败,仍有短时穿透和明显物体偏离。**','', '输入仍为 `docs/20260915_171525`。FoundationPose 物体位姿及有效标记沿用上一轮。修复输出独立保存,旧结果未覆盖。','', '## 修复内容','', '- 两个 CAD 的 T 形网格连接修复:只拆分共线三角形边,顶点、表面积与体积不变;两个模型均闭合。','- 红/蓝盒碰撞由各 20 个凸块改为 94/96 个有效凸块;手掌与拇指根部分离视觉网格和分块碰撞网格。它们仍是碰撞近似。','- 逐连杆检测碰撞,用解析接触法向与运动学雅可比构造分离约束;保持关节限位及原 URDF 线性 mimic。','- 接触时沿物体运动延续上一帧手姿,加相邻帧运动限制,避免从盒子一侧突然跳到另一侧。','- 手腕及物体旋转使用四元数 SLERP 插值,再选择连续欧拉角分支供 Spider 使用。逐个检查并修正全部 400 Hz 插值状态。','- 增加 2.5 mm 碰撞安全间隙,覆盖凸分解与原始表面的部分误差。','- 物理接触参数从较软的时间常数改为 5 ms,阻抗 `.999 .9999 .0001`,接触求解迭代提高到 80 次;手腕平移最大推力 20 N、旋转最大力矩 2 Nm、手指最大力矩 0.1 Nm(仿真假设);初始桌面向下校正 9.84 mm,消除盒子初态与桌面的重叠。这是仿真对齐假设,不是新观测。','', '## 参考轨迹验证','', '以下前后对比使用同一套新碰撞几何,避免把更换模型本身算作轨迹改善。','', '| 指标 | 修复前 | 修复后 |','|---|---:|---:|',f'| 全 352 帧最大手物穿透/mm | {b["max_hand_object_penetration_mm"]:.2f} | {a["max_hand_object_penetration_mm"]:.2f} |',f'| 穿透超过 0.5 mm 的帧数 | {b["frames_over_0_5mm"]} | {a["frames_over_0_5mm"]} |',f'| 推断接触目标平均间隙/mm | {b["inferred_contact_gap_mean_mm"]:.2f} | {a["inferred_contact_gap_mean_mm"]:.2f} |', '',f'全部 4,681 个高频状态:最大手物穿透 **{v["interpolation"]["max_penetration_after_mm"]:.3f} mm**。状态均有限;物体坐标未更改。关节限位最大超出 {a["max_joint_limit_violation_rad"]:.3g} rad,mimic 最大残差 {a["max_mimic_error_rad"]:.3g} rad。','',f'独立使用原始手部视觉网格的 {v["cad_probe_scope"].split()[0]} 个表面采样点/帧,对闭合原始 CAD 检查:红盒最大侵入 **{v["upper_exact_CAD_surface_probes"]["max_inside_mm"]:.2f} mm**,蓝盒 **{v["lower_exact_CAD_surface_probes"]["max_inside_mm"]:.2f} mm**,超过 1 mm 的帧数均为零。采样检查不是完整网格相交证明。','', '## 代价与边界','', '- 本次修复范围是双手与两个盒子的碰撞,不包含手部自碰撞或手与桌面的完整验证。','- 原始手—物体配准偏差较大,修复有明显手腕位移;右/左手相对旧接触参考的修正中位数分别为 '+f'{v["wrist_correction_mm"]["right"]["median"]:.1f}/{v["wrist_correction_mm"]["left"]["median"]:.1f} mm。这不能当作已校准的人手绝对位姿。','- 指尖接触目标来自距离推断,平均间隙仍约 '+f'{a["inferred_contact_gap_mean_mm"]:.1f} mm。防穿透通过不等于稳定抓取通过。','- 蓝盒第 177 帧起被遮挡,仍保持最后可靠参考位姿;这部分没有新增观测。',''] +if phys: + lines+=['## SPIDER 实际物理滚动','',f'完整 {phys["steps"]} 步 / {phys["duration_s"]:.2f} 秒,全部有限值:{phys["all_finite"]};物体真实执行器增益为零:{phys["zero_object_actuator_gains"]}。', '',f'独立 CPU 碰撞检测检查每个保存的 GPU 状态:最大穿透 **{phys["max_penetration_mm"]:.2f} mm**,逐步最大穿透中位数 **{phys["median_step_max_penetration_mm"]:.2f} mm**,超过 1 mm 的步数 {phys["steps_over_1mm"]}。', '',f'物体中心最大误差(仅可观测段):红盒 {phys["upper"]["observed_centroid_error_max_mm"]:.1f} mm,蓝盒 {phys["lower"]["observed_centroid_error_max_mm"]:.1f} mm;20 mm 抓取跟踪门槛通过:**{phys["grasp_tracking_20mm_passed"]}**。','', f'实际发生手物接近/接触的物理步数 {phys["steps_with_hand_object_contact"]}/{phys["steps"]};最大物理关节限位超出 {phys["max_joint_limit_violation_rad"]:.4f} rad。碰撞变少也可能来自接触丢失,不能独立证明抓取成功。','参考轨迹修复和真实物理结果分开保存;没有通过强制写回物体位姿制造成功抓取。',''] +lines+=['## 文件','', '- `hand_collision_before_after.mp4`:原视频 / 修复前参考 / 修复后参考。视频明确标为运动学对比。','- `reference_video_rate.npz`、`reference_trajectory.csv`:修复后的手与物体参考及高频关节/控制数据。','- `validation.json`:同模型前后碰撞、关节限位、mimic、原 CAD 表面探针。','- `interpolation_validation.json`:全部高频状态检查。开启 buffer 时,`max_penetration_before_mm` 字段表示距所要求间隙的不足,见 `before_metric`;after 字段始终表示实际穿透。','- `spider_collision_rollout.mp4`:修复参考与实际 Spider 自由物体滚动对比。','- `physics_validation.json`、`physics_motion.npz`:实际 Spider 物理结果(失败验证,不能视为可用抓取控制)。','- `dynamics_probe.json`:同一组保存控制量的 CPU 参数对比;CPU 中小于 1 mm 的结果未在本次 GPU 优化中完整复现。','- `collision_v2/`、`hand_collision/`:新碰撞几何;`datasets/processed/current/l20/bimanual/boxes/scene_act.xml` 为完整场景。','', '## 复现','', '```bash','.venv/bin/python scripts/build_box_collision_v2.py','.venv/bin/python scripts/build_palm_collision_v2.py','.venv/bin/python scripts/fix_yesterday_hand_collision.py','.venv/bin/python scripts/verify_fix_interpolation.py --video --buffer','.venv/bin/python scripts/interpolate_collision_reference.py','.venv/bin/python scripts/verify_fix_interpolation.py --buffer','.venv/bin/python scripts/align_collision_floor.py','.venv/bin/python scripts/configure_collision_physics.py','SPIDER_TASK_OUT=$PWD/output/collision_fix_20260915 .spider/bin/python scripts/run_yesterday_spider.py','.venv/bin/python scripts/audit_collision_fix.py','.venv/bin/python scripts/audit_collision_physics.py','.venv/bin/python scripts/render_collision_fix.py','.venv/bin/python scripts/report_collision_fix.py','```','', '旧几何探索碎片在 `collision/` 中,仅供排查,不由新场景使用。'] +(O/'README.md').write_text('\n'.join(lines)+'\n');print('REPORT_COMPLETE',n) diff --git a/scripts/report_spider_dynamics_fix.py b/scripts/report_spider_dynamics_fix.py new file mode 100644 index 0000000..bc8fe5c --- /dev/null +++ b/scripts/report_spider_dynamics_fix.py @@ -0,0 +1,17 @@ +"""Evidence report for the corrected SPIDER free-object collision run.""" +from pathlib import Path +import json,hashlib,numpy as np,cv2,mujoco +ROOT=Path(__file__).resolve().parents[1];O=ROOT/'output/spider_dynamics_fix_20260915';p=O/'datasets/processed/current/l20/bimanual/boxes';v=json.loads((O/'physics_validation.json').read_text());old=json.loads((ROOT/'output/collision_fix_20260915/physics_validation.json').read_text());probe=json.loads((O/'gpu_contact_probe.json').read_text());replay=json.loads((O/'control_replay.json').read_text());reset=json.loads((O/'state_restore_check.json').read_text());surface=json.loads((O/'dynamic_surface_validation.json').read_text());a=np.load(O/'physics_motion.npz');m=mujoco.MjModel.from_xml_path(str(p/'scene_act.xml')) +qnames=[m.joint(j).name for j in range(m.njnt)];unames=[m.actuator(j).name for j in range(m.nu)];np.savetxt(O/'spider_trajectory.csv',np.c_[a['time'],a['qpos'],a['ctrl']],delimiter=',',header=','.join(['time_s']+qnames+['ctrl_'+n for n in unames]),comments='') +video=O/'spider_collision_rollout.mp4';cap=cv2.VideoCapture(str(video));fps=cap.get(cv2.CAP_PROP_FPS);n=0 +while True: + ok,frame=cap.read() + if not ok:break + n+=1 +cap.release();integrity={'decoded_frames':n,'fps':fps,'expected_frames':352,'passed':n==352 and abs(fps-30)<1e-6};(O/'video_integrity.json').write_text(json.dumps(integrity,indent=2));assert integrity['passed'] +status='碰撞小于 1 mm 门槛通过' if v['max_penetration_mm']<1 else '碰撞小于 1 mm 门槛未通过' +lines=['# Spider 动态碰撞继续修复','',f'状态:**{status};自由物体抓取跟踪 20 mm 门槛:{v["grasp_tracking_20mm_passed"]}。**','',f'完整 {v["steps"]} 步 / {v["duration_s"]:.2f} 秒,所有执行状态有限。输入 `docs/20260915_171525`,两只 L20、红盒和蓝盒。物体执行器在真实执行阶段保持零增益;没有把物体固定到手上。','', '## 完整动态结果','', '| 指标 | 上轮 | 本轮 |','|---|---:|---:|',f'| 最大手盒穿透/mm | {old["max_penetration_mm"]:.3f} | {v["max_penetration_mm"]:.3f} |',f'| 穿透超过 1 mm 的物理步数 | {old["steps_over_1mm"]} | {v["steps_over_1mm"]} |',f'| 红盒有效观测段质心平均误差/mm | {old["upper"]["observed_centroid_error_mean_mm"]:.1f} | {v["upper"]["observed_centroid_error_mean_mm"]:.1f} |',f'| 蓝盒有效观测段质心平均误差/mm | {old["lower"]["observed_centroid_error_mean_mm"]:.1f} | {v["lower"]["observed_centroid_error_mean_mm"]:.1f} |', '', '低穿透不代表抓取成功。物体位姿跟踪仍需单独通过验证;几何接触目标没有触觉真值,手指接触保持和抓取质量尚未解决。', '', '## 已修复的代码问题','', '- 虚拟辅助伺服力改为 `kp*ctrl - kp*q - kd*qvel`,并完整保存和恢复辅助增益。','- 候选轨迹重采样复制包含当前步的控制前缀,避免状态与控制历史不一致(当前任务默认关闭此功能,属于潜在路径修复)。','- 接触奖励实际乘入配置权重;双手状态按含被动关节的自由度划分。','- 新候选批次同步约束求解初值,避免遗留失败候选的 warmstart。','- 零物体辅助断言检查全部仿射偏置项。','', '## GPU 接触数值问题','', '旧的 `solimp=.999 .9999 .0001`、`solref=.005 1` 在当前 GPU 模型中导致蓝盒接触桌面后弹飞;同控制 CPU 重放未出现该早期爆炸。短段对照改为 `solimp=.9 .95 .001 .5 2`、`solref=.02 1`,并以 2 mm 提前量激活接触约束。碰撞网格没有扩大;检测使用实际几何距离。提前生效是数值缓冲,不能视为真实接触位置精度。','', '| 同一短段控制的参数对照 | 最大手盒穿透/mm | 最大物体质心位移/mm |','|---|---:|---:|'] +for x in probe: + lines.append(f'| {x["name"]} | {x["max_penetration_mm"]:.3f} | {x["max_object_com_displacement_mm"]:.2f} |') +lines+=['','## 验证与边界','', '- 5 项回归测试通过:伺服平衡/恢复、控制重采样因果性、双手自由度索引、奖励权重和批次 warmstart 同步。',f'- GPU 状态恢复专项:仅状态恢复差异 {reset["state_only_max_qpos_difference"]:.3g};状态和参数恢复差异 {reset["state_and_parameters_max_qpos_difference"]:.3g}。这些是下一步 qpos 混合米/弧度的最大差异。',f'- 独立重放 {replay["steps"]} 步,首步最大 qpos 差异 {replay["gpu_first_step_max_qpos_difference"]:.3g},全段最大差异 {replay["gpu_max_qpos_difference"]:.3g}(混合米/弧度)。长段仍不能视为严格复现的控制轨迹。','- 每个保存的物理状态由 CPU 独立碰撞检测审计;凸分解几何仍是近似,未验证连续两步之间或手部自碰撞。',f'- 原始手网格表面独立采样:每帧 {surface["hand_surface_samples_per_frame"]} 点,共 {surface["sampled_frames"]} 帧;红盒最大侵入 {surface["upper"]["max_inside_mm"]:.3f} mm,蓝盒 {surface["lower"]["max_inside_mm"]:.3f} mm。不是完整网格相交证明。',f'- 最大关节限位超出 {v["max_joint_limit_violation_rad"]:.4f} rad;该动态轨迹不应直接用于硬件。','- 物体质量 0.12 kg、摩擦和执行器力限制均为仿真假设。','- 蓝盒第 177 帧起缺少可靠 FoundationPose 观测,沿用保持最后有效位姿的参考,不代表测到了遮挡后的运动。','- FoundationPose 原始位姿、有效标记和运动学防穿透参考沿用上一轮;`reference_inherited_*` 仅为继承证据。','', '## 输出','', '- `spider_collision_rollout.mp4`:原视频 / 防穿透运动学参考 / 本轮真实自由物体动态结果,352 帧已完整解码。','- `physics_validation.json`、`physics_motion.npz`:全物理步穿透与跟踪审计。','- `spider_trajectory.csv`:关节轨迹及控制。','- `gpu_contact_probe.json`:匹配控制的参数对照。','- `control_replay.npz/json`、`state_restore_check.json`:重放及状态恢复检查。','- `spider_fixes.patch`:上游本地代码修复;`hard_contact_attempt/`、`stable_short/`:失败和短段对照存档。','', '## 复现','', '```bash','.spider/bin/python tests/test_spider_dynamics_contract.py','SPIDER_TASK_OUT="$PWD/output/spider_dynamics_fix_20260915" .spider/bin/python scripts/run_yesterday_spider.py','SPIDER_TASK_OUT="$PWD/output/spider_dynamics_fix_20260915" .venv/bin/python scripts/audit_collision_physics.py','.spider/bin/python scripts/check_spider_dynamics_replay.py','SPIDER_TASK_OUT="$PWD/output/spider_dynamics_fix_20260915" .venv/bin/python scripts/render_collision_fix.py --physics','.venv/bin/python scripts/report_spider_dynamics_fix.py','```',''] +(O/'README.md').write_text('\n'.join(lines));print(status);print(integrity) diff --git a/scripts/report_yesterday_spider.py b/scripts/report_yesterday_spider.py new file mode 100644 index 0000000..fab30f2 --- /dev/null +++ b/scripts/report_yesterday_spider.py @@ -0,0 +1,21 @@ +from pathlib import Path +import json,cv2,numpy as np +ROOT=Path(__file__).resolve().parents[1];O=ROOT/'output/foundationpose_spider_20260915' +v=json.loads((O/'validation.json').read_text());p=json.loads((O/'preparation.json').read_text());extra={} +for name in ['foundationpose_overlay.mp4','original_foundationpose_spider.mp4']: + c=cv2.VideoCapture(str(O/name));count=0 + while c.read()[0]:count+=1 + c.release();extra[name]={'decoded_frames':count,'expected_frames':352,'passed':count==352} +(O/'video_integrity.json').write_text(json.dumps(extra,indent=2)) +lines=['# 昨日双手抓取:FoundationPose + SPIDER','', '输入:`docs/20260915_171525`,352 帧。复用昨日 RGB-D 校正后的 Dyn-HaMR/L20 双手轨迹;没有重新做人手推理。','', '## 已完成的链路','', '1. FoundationPose 根据 RGB、对齐深度、内参与两个 STL 逐帧估计物体相机坐标位姿。','2. 使用昨日 RGB-D 相机里程计统一手、物体坐标;用初始深度拟合桌面方向。','3. 逐指尖选择最近可观测物体,生成可切换的接触目标;接触阈值 25 mm,释放阈值 35 mm。','4. 先做有界关节/手腕平移接触与碰撞初始化,再运行真正的 `third_party/spider/examples/run_mjwp.py`,使用 MuJoCo Warp 物理滚动优化。','5. 最终实际滚动物体执行器增益为零;物体不是逐帧强制写回参考轨迹。','', '## 观测边界','', '- 红盒 307/352 帧通过可见颜色覆盖与深度一致性筛选,未通过帧插值补全。','- 蓝盒 177/352 帧可用;177 帧起遮挡,参考位姿保持最后可用世界位姿,并停止将其作为新的接触目标来源。蓝盒遮挡段不是观测结果,后续红蓝相对装配也不是已验证事实。','- 有效判定不是外部真值:颜色像素 >500、投影覆盖 >65%、重叠像素深度中位残差 <20 mm。蓝盒遮挡边界另行显式屏蔽。','- 接触是几何假设,不是触觉标签。左右手均可切换到红盒接触。','- 物体质量各 0.12 kg、摩擦系数 0.8 是仿真假设;桌面方向来自深度,不是 IMU。','- 每个物体采用 20 个 CoACD 凸分块;孔洞和凹槽碰撞仍是近似。物体关节增加 0.001 armature 抑制欧拉角坐标翻转时的数值病态,非实测惯量。','', '## 本次实际结果','', '| 指标 | 昨日手 + 新物体位姿 | 接触/碰撞初始化 | SPIDER 物理结果 |','|---|---:|---:|---:|'] +for label,key in [('最大手物穿透/mm','max_hand_object_penetration_mm'),('逐帧最大穿透中位数/mm','median_frame_max_penetration_mm'),('推断接触目标平均间隙/mm','inferred_contact_target_gap_mean_mm'),('有关联手物碰撞的帧数','hand_object_contact_frames')]: + vals=[v[k][key] for k in ['original_hands_new_objects','kinematic_contact_warmstart','spider']];lines.append('| '+label+' | '+' | '.join(f'{x:.2f}' for x in vals)+' |') +for name,cn in [('upper','红盒'),('lower','蓝盒')]: + r=v['spider'][name];lines+=['',f'{cn}可观测帧的位置误差:平均 **{r["observed_position_error_mean_mm"]:.1f} mm**,最大 **{r["observed_position_error_max_mm"]:.1f} mm**;平均旋转误差 **{r["observed_rotation_error_mean_deg"]:.1f}°**。误差对照 FoundationPose 参考,并非外部真值。'] +lines+=['',f'物理软约束残差:视频抽样最大关节限位超出 {v["spider"]["max_joint_limit_violation_rad"]:.4f} rad,最大 mimic 偏差 {v["spider"]["max_mimic_residual_rad"]:.4f} rad。', '',f'完整物理轨迹:{v["full_steps"]} 步,{v["duration_s"]:.2f} 秒,全部有限值:{v["all_states_finite"]}。20 mm 最大物体位置跟踪门槛通过:**{v["tracking_20mm_gate_passed"]}**。', '', '穿透统计使用碰撞近似模型,按视频 30 Hz 抽样;不能当成全部 400 Hz 物理步的最大穿透,也不能代表 STL 精确有符号距离。', '', '## 文件','', '- `original_foundationpose_spider.mp4`:原视频 / FoundationPose+接触初始化参考 / 实际 SPIDER 物理结果。','- `foundationpose_overlay.mp4`:物体投影检查,逐帧显示有效标记。','- `foundationpose_objects.npz`:原始相机/世界位姿与有效标记。','- `object_reference.npz`:明确补全后的仿真参考与源世界到仿真变换。','- `comparison_motion.npz`:原手、新参考、Spider 视频帧轨迹及完整物理轨迹。','- `spider_trajectory.csv`:完整物理 qpos 与控制量,关节顺序见表头。','- `validation.json`、`preparation.json`、`video_integrity.json`:验证和假设。','- `independent_gpu_replay.npz/json`:保存控制量的独立 GPU 重放与未优化控制基线。','', '## 适配与复现','', '两个 L20 各有 27 状态自由度、22 个执行器通道。包装器修正上游使用 actuator 数量切分双手状态奖励的索引;原始 L20 线性 mimic 被保留。由于存在双手换物接触,关闭“每只手永远对应固定物体”的拇指位移启发式,保留物体参考虚拟辅助衰减及逐指接触奖励。实际滚动始终无物体辅助。', '', '```bash','source scripts/foundationpose_env.sh','.venv/bin/python scripts/track_yesterday_boxes.py','.venv/bin/python scripts/decompose_yesterday_boxes.py','.venv/bin/python scripts/prepare_yesterday_spider.py','.spider/bin/python scripts/run_yesterday_spider.py','.spider/bin/python scripts/replay_yesterday_spider_gpu.py','.venv/bin/python scripts/evaluate_yesterday_spider.py','.venv/bin/python scripts/audit_yesterday_geometry.py','.venv/bin/python scripts/report_yesterday_spider.py','```','', '运行需要 GPU;未提供硬件控制命令,也未验证真实机械手稳定抓取。'] +if not v['tracking_20mm_gate_passed']:lines.insert(2,'**状态:完整链路已跑通,但物理抓取跟踪未通过 20 mm 门槛;不能宣称已经修复稳定抓取。**\n') +if (O/'geometry_audit.json').exists(): + g=json.loads((O/'geometry_audit.json').read_text());z=g['full_rate_spider_collision'];lines+=['','## 独立几何检查','',f'全部 {z["steps"]} 个物理步的最大手物碰撞穿透:{z["max_hand_object_penetration_mm"]:.2f} mm(CoACD 近似几何)。','', '红盒精确闭合 CAD 的 42 个手部关键点探针结果见 `geometry_audit.json`。这是点探针检查,不能替代完整机械手网格相交检测。'] +if 'independent_replay_integrity' in v: + x=v['independent_replay_integrity'];lines+=['','## 保存控制量重放','',f'独立 GPU 重放全部有限值:{x["all_finite"]};相对原 Spider 轨迹最大 qpos 数值差:{x["max_difference_from_native_spider_qpos"]:.6g}(混合位置/角度单位)。独立重放未复现原 Spider 物体轨迹,存在明显接触动力学差异,不能作为已验证可重放的抓取控制。'] +(O/'README.md').write_text('\n'.join(lines)+'\n');print(json.dumps(extra,indent=2)) diff --git a/scripts/retarget_l20_video.py b/scripts/retarget_l20_video.py index 337adb0..cc52982 100644 --- a/scripts/retarget_l20_video.py +++ b/scripts/retarget_l20_video.py @@ -23,11 +23,13 @@ def main(): p.add_argument('--input', type=Path, required=True) p.add_argument('--output-dir', type=Path, required=True) p.add_argument('--sigma', type=float, default=2.5) + p.add_argument('--side', choices=['right','left'], default='right') a = p.parse_args() torch.set_num_threads(4) out = a.output_dir.resolve() - fixed, urdf = build_assets(out) - original = ET.parse(ASSET / 'linkerhand_g20_right.urdf').getroot() + fixed, urdf = build_assets(out, a.side) + asset = ASSET.parent/a.side.upper() + original = ET.parse(asset / f'linkerhand_g20_{a.side}.urdf').getroot() joints = [j for j in original.findall('joint') if j.get('type') != 'fixed'] active = [j.get('name') for j in joints if j.find('mimic') is None] limits = {j.get('name'):np.array([float(j.find('limit').get(k)) for k in ['lower','upper']]) for j in joints} @@ -109,7 +111,9 @@ def main(): quat = gaussian_filter1d(quat, a.sigma, axis=0) quat /= np.linalg.norm(quat, axis=1, keepdims=True) quat = quat[:, [3, 0, 1, 2]] - times = np.arange(count) / fps + times = np.asarray(human['time'], dtype=np.float64) if 'time' in human else np.arange(count) / fps + assert times.shape == (count,) and np.isfinite(times).all() and np.all(np.diff(times) > 0) + times = times - times[0] np.savez_compressed(out / 'motion.npz', qpos=qpos, qpos_raw=qraw, joint_names=robot.dof_joint_names, active_qpos=active_q, active_joint_names=active, targets=targets, actual=actual, wrist_pos=wrist, wrist_pos_unsmoothed=wrist_raw, wrist_quat_wxyz=quat, diff --git a/scripts/rl_track_reference.py b/scripts/rl_track_reference.py new file mode 100644 index 0000000..7c9366f --- /dev/null +++ b/scripts/rl_track_reference.py @@ -0,0 +1,194 @@ +"""PPO tracking policy: kinematic reference as PD targets + residual actions, MuJoCo Warp parallel physics. + +Reward (DexMachina/ManipTrans style): object pose tracking + landmark imitation + joint-space bc + contact-point term +- action-rate/magnitude penalties. Virtual object controller (VOC) with decaying gains assists early training. +Reference state initialization: episodes start at random demo frames. Eval: full demo from t=0, VOC off, deterministic. +""" +import os, sys, json, time, math, argparse +from pathlib import Path +ROOT = Path(__file__).resolve().parents[1] +os.environ['WARP_CACHE_PATH'] = str(ROOT / '.spider_cache/warp'); os.environ.setdefault('MUJOCO_GL', 'osmesa') +import numpy as np, torch, torch.nn as nn, mujoco, mujoco_warp as mw, warp as wp +p = argparse.ArgumentParser() +p.add_argument('--out', type=Path, default=ROOT / 'output/rl_track_20260915'); p.add_argument('--nworld', type=int, default=512); p.add_argument('--iters', type=int, default=1500) +p.add_argument('--horizon', type=int, default=24); p.add_argument('--decimation', type=int, default=8); p.add_argument('--eval-every', type=int, default=50); p.add_argument('--seed', type=int, default=0) +p.add_argument('--voc-kp', type=float, default=200.); p.add_argument('--resume', type=Path); p.add_argument('--smoke', action='store_true') +p.add_argument('--phase-control', action='store_true', help='extra action: reference playback rate in [0,1.5]'); p.add_argument('--wrist-pos-scale', type=float, default=.03); p.add_argument('--wrist-rot-scale', type=float, default=.15); p.add_argument('--finger-scale', type=float, default=.4); p.add_argument('--prog-weight', type=float, default=.2); p.add_argument('--rate-max', type=float, default=1.5); p.add_argument('--seg-start', type=int, default=-1, help='segment mode: episodes start in [seg-start, seg-start+seg-jitter] (400 Hz index)'); p.add_argument('--seg-end', type=int, default=-1); p.add_argument('--seg-jitter', type=int, default=80); p.add_argument('--prog-mode', choices=['bonus', 'scale'], default='bonus', help='bonus: + w*(rate-1); scale: tracking reward multiplied by rate') +p.add_argument('--voc-start', type=float, default=1.0); p.add_argument('--voc-zero-iters', type=int, default=0, help='iterations (from start) until assistance is forced to zero; 0 = 70%% of --iters'); p.add_argument('--contact-start-frac', type=float, default=0.0); p.add_argument('--vel-penalty', type=float, default=0.0) +A = p.parse_args(); A.out.mkdir(parents=True, exist_ok=True); torch.manual_seed(A.seed); np.random.seed(A.seed) +dev = torch.device('cuda:0'); wp.init() +# ---------------- reference ---------------- +R = dict(np.load(A.out / 'reference.npz')); N = len(R['qpos']); ref = {k: torch.as_tensor(R[k], device=dev, dtype=torch.float32) for k in ['qpos', 'qvel', 'ctrl', 'contact', 'obj_pos', 'obj_quat', 'tip_pos', 'landmark_pos', 'cp_local']} +cp_obj = torch.as_tensor(R['cp_obj'], device=dev, dtype=torch.long); OBJ = [int(x) for x in R['obj_body_ids']]; TIPS = [int(x) for x in R['tip_site_ids']]; LM = [int(x) for x in R['landmark_site_ids']]; WRIST = [int(x) for x in R['wrist_qpos_adr']] +m = mujoco.MjModel.from_xml_path(str(R['scene'])); phys = json.loads(Path(str(R['physics_parameters'])).read_text()); m.opt.iterations = phys['solver_iterations']; m.opt.ls_iterations = phys['ls_iterations'] +assert not m.actuator_gainprm[44:].any(); NQ, NU = m.nq, m.nu; HAND_Q = 54; NA_H = 44; NA = NA_H + (1 if A.phase_control else 0) +ctrl_lo = torch.as_tensor(m.actuator_ctrlrange[:NA_H, 0], device=dev); ctrl_hi = torch.as_tensor(m.actuator_ctrlrange[:NA_H, 1], device=dev); ctrl_lim = torch.as_tensor(m.actuator_ctrllimited[:NA_H].astype(bool), device=dev) +ACT_SCALE = torch.tensor(([A.wrist_pos_scale] * 3 + [A.wrist_rot_scale] * 3 + [A.finger_scale] * 16) * 2, device=dev) +d0 = mujoco.MjData(m); d0.qpos[:] = R['qpos'][0]; mujoco.mj_forward(m, d0) +with wp.ScopedDevice('cuda:0'): + wm = mw.put_model(m); wd = mw.put_data(m, d0, nworld=A.nworld, nconmax=1024, njmax=3072) + with wp.ScopedCapture() as cap: mw.step(wm, wd) + graph = cap.graph +T_ = lambda a: wp.to_torch(a) +qpos, qvel, ctrl, xfrc, site_xpos, xpos, xquat, cvel = T_(wd.qpos), T_(wd.qvel), T_(wd.ctrl), T_(wd.xfrc_applied), T_(wd.site_xpos), T_(wd.xpos), T_(wd.xquat), T_(wd.cvel) +EXTRA_RESET = [T_(getattr(wd, k)) for k in ['qacc_warmstart', 'qacc', 'qfrc_applied', 'act', 'act_dot', 'efc_force'] if hasattr(wd, k) and getattr(wd, k) is not None and getattr(wd, k).shape[0] == A.nworld] +NW = A.nworld; DEC = A.decimation; CTRL_STEPS = N // DEC +def kin(): + torch.cuda.synchronize(); mw.kinematics(wm, wd); wp.synchronize() +def sim_steps(n): + torch.cuda.synchronize() + for _ in range(n): wp.capture_launch(graph) + wp.synchronize() +# ---------------- helpers ---------------- +def quat_mul(a, b): + w1, x1, y1, z1 = a.unbind(-1); w2, x2, y2, z2 = b.unbind(-1) + return torch.stack([w1 * w2 - x1 * x2 - y1 * y2 - z1 * z2, w1 * x2 + x1 * w2 + y1 * z2 - z1 * y2, w1 * y2 - x1 * z2 + y1 * w2 + z1 * x2, w1 * z2 + x1 * y2 - y1 * x2 + z1 * w2], -1) +def quat_conj(q): return q * torch.tensor([1, -1, -1, -1], device=q.device) +def quat_rotate(q, v): + qv = torch.cat([torch.zeros_like(v[..., :1]), v], -1); return quat_mul(quat_mul(q, qv), quat_conj(q))[..., 1:] +def rotvec_err(q_ref, q): # rotation vector taking q to q_ref (world frame) + dq = quat_mul(q_ref, quat_conj(q)); dq = dq * torch.sign(dq[..., :1] + 1e-12); ang = 2 * torch.acos(dq[..., 0].clamp(-1, 1)); s = torch.sqrt((1 - dq[..., 0] ** 2).clamp_min(1e-9)); return dq[..., 1:] / s[..., None] * ang[..., None], ang +# ---------------- env state ---------------- +t_idx = torch.zeros(NW, dtype=torch.long, device=dev); t_pos = torch.zeros(NW, device=dev); max_len = torch.zeros(NW, dtype=torch.long, device=dev); ep_len = torch.zeros(NW, dtype=torch.long, device=dev); prev_a = torch.zeros(NW, NA, device=dev) +voc_gain = torch.full((NW,), float(A.voc_start), device=dev); voc_gain[:16] = 0. # first 16 worlds never get object assistance +OBJ_QADR = torch.arange(54, 66, device=dev); OBJ_VADR = OBJ_QADR - 0 # nv == nq here (all hinge/slide) +assert m.nv == m.nq +CONTACT_FRAMES = torch.nonzero(ref['contact'][:N - 2 * DEC * 60].sum(-1) > 0).squeeze(-1) +def reset(mask, start=None): + idx = torch.nonzero(mask).squeeze(-1) + if len(idx) == 0: return + if start is None and A.seg_start >= 0: + s = A.seg_start + torch.randint(0, max(1, A.seg_jitter), (len(idx),), device=dev) + elif start is None: + s = torch.randint(0, N - 2 * DEC * 60, (len(idx),), device=dev) + if A.contact_start_frac > 0 and len(CONTACT_FRAMES): + pick = torch.rand(len(idx), device=dev) < A.contact_start_frac; s = torch.where(pick, CONTACT_FRAMES[torch.randint(0, len(CONTACT_FRAMES), (len(idx),), device=dev)], s) + else: s = torch.full((len(idx),), int(start), device=dev, dtype=torch.long) + qpos[idx] = ref['qpos'][s]; qvel[idx] = ref['qvel'][s] * 0.; ctrl[idx] = ref['ctrl'][s]; xfrc[idx] = 0 + for arr in EXTRA_RESET: arr[idx] = 0 + qpos[idx] = ref['qpos'][s]; t_idx[idx] = s; t_pos[idx] = s.float(); max_len[idx] = (((A.seg_end if A.seg_end > 0 else N) - s).float() / DEC * 1.5).long() + 5; ep_len[idx] = 0; prev_a[idx] = 0 +def obj_state(): + op = xpos[:, OBJ]; oq = xquat[:, OBJ]; ov = qvel[:, OBJ_QADR].view(NW, 2, 6); return op, oq, ov[..., :3], ov[..., 3:] +def apply_voc(t): + op, oq, lv, av = obj_state(); pr = ref['obj_pos'][t]; qr = ref['obj_quat'][t]; g = voc_gain[:, None, None] + F = (A.voc_kp * (pr - op) - 10. * lv).clamp(-20, 20) * g; rv, _ = rotvec_err(qr, oq); Tq = (2. * rv - .1 * av).clamp(-1, 1) * g + xfrc[:, OBJ, :3] = F; xfrc[:, OBJ, 3:] = Tq +def observe(t): + hq = qpos[:, :HAND_Q]; hv = qvel[:, :HAND_Q]; hq_ref = ref['qpos'][t, :HAND_Q]; op, oq, lv, av = obj_state(); pr = ref['obj_pos'][t]; qr = ref['obj_quat'][t] + rv, ang = rotvec_err(qr, oq); t2 = (t + 20).clamp(max=N - 1); t3 = (t + 40).clamp(max=N - 1) + tips = site_xpos[:, TIPS]; rel = (tips[:, :, None, :] - op[:, None, :, :]).reshape(NW, -1) + obs = torch.cat([hq, hv * .05, hq - hq_ref, (op - pr).reshape(NW, -1), oq.reshape(NW, -1), rv.reshape(NW, -1), lv.reshape(NW, -1) * .2, av.reshape(NW, -1) * .05, (ref['obj_pos'][t2] - op).reshape(NW, -1), (ref['obj_pos'][t3] - op).reshape(NW, -1), rel, ref['contact'][t], (t.float() / N)[:, None], voc_gain[:, None], prev_a], -1) + return obs +def reward(t, a): + op, oq, lv, av = obj_state(); pr = ref['obj_pos'][t]; qr = ref['obj_quat'][t]; ep = (op - pr).norm(dim=-1); _, ang = rotvec_err(qr, oq) + r_obj = (torch.exp(-20 * ep) * torch.exp(-3 * ang)).mean(-1) + lm = site_xpos[:, LM]; e_lm = (lm - ref['landmark_pos'][t]).norm(dim=-1).mean(-1); r_imi = torch.exp(-20 * e_lm) + dq = qpos[:, :HAND_Q] - ref['qpos'][t, :HAND_Q]; w = torch.ones(HAND_Q, device=dev); w[WRIST[:3]] = 10; w[WRIST[6:9]] = 10; r_bc = torch.exp(-2 * (dq.abs() * w).mean(-1)) + con = ref['contact'][t] > .5; cpl = ref['cp_local'][t]; co = cp_obj[t]; oq_sel = torch.gather(oq, 1, co[..., None].expand(-1, -1, 4)); op_sel = torch.gather(op, 1, co[..., None].expand(-1, -1, 3)) + target = quat_rotate(oq_sel, cpl) + op_sel; dtip = (site_xpos[:, TIPS] - target).norm(dim=-1); rc = torch.exp(-50 * dtip); n_on = con.sum(-1); r_con = torch.where(n_on > 0, (rc * con).sum(-1) / n_on.clamp_min(1), torch.full_like(r_obj, .5)) + pen = .05 * ((a - prev_a[:, :NA_H]) ** 2).mean(-1) + .01 * (a ** 2).mean(-1) + if A.vel_penalty > 0: + tn = (t + DEC).clamp(max=N - 1); v_ref = (ref['obj_pos'][tn] - ref['obj_pos'][t]) / (DEC * float(m.opt.timestep)); dv = ((lv - v_ref) ** 2).sum(-1).clamp(max=4.).mean(-1); pen = pen + A.vel_penalty * dv + track = 1.0 * r_obj + .3 * r_imi + .1 * r_bc + .5 * r_con; r = track - pen + return r, dict(r_obj=r_obj, r_imi=r_imi, r_bc=r_bc, r_con=r_con, obj_err_cm=ep.mean(-1) * 100, obj_rot_deg=ang.mean(-1) * 57.3, lm_err_cm=e_lm * 100, track=track, pen=pen, rot_each=ang * 57.3, err_each=ep * 100) +FAIL_THR = torch.tensor(.15, device=dev) +def step(a): + t = t_idx; hand = ref['ctrl'][t, :NA_H] + a[:, :NA_H] * ACT_SCALE; hand = torch.where(ctrl_lim, hand.clamp(ctrl_lo, ctrl_hi), hand); ctrl[:, :NA_H] = hand; ctrl[:, NA_H:] = ref['ctrl'][t, NA_H:] + rate = (1. + .5 * a[:, NA_H]).clamp(0., A.rate_max) if A.phase_control else torch.ones(NW, device=dev) + apply_voc(t); sim_steps(DEC); t_pos.add_(DEC * rate); t_idx[:] = t_pos.round().long().clamp(max=N - 1); ep_len.add_(1); t = t_idx + r, info = reward(t, a[:, :NA_H]); op, oq, _, _ = obj_state(); ep = (op - ref['obj_pos'][t]).norm(dim=-1).max(-1).values + if A.phase_control: + if A.prog_mode == 'scale': r = info['track'] * rate - info['pen'] + else: r = r + A.prog_weight * (rate - 1.) + info['rate'] = rate + wrist_err = torch.stack([(qpos[:, WRIST[:3]] - ref['qpos'][t][:, WRIST[:3]]).norm(dim=-1), (qpos[:, WRIST[6:9]] - ref['qpos'][t][:, WRIST[6:9]]).norm(dim=-1)], -1).max(-1).values + seg_end = A.seg_end if A.seg_end > 0 else N - 1 + bad = ~torch.isfinite(qpos).all(-1) | (ep > FAIL_THR) | (wrist_err > .25) | (ep_len >= max_len); trunc = (t_pos >= seg_end) & ~bad + r = torch.where(bad, r - 1., r); done = bad | trunc; prev_a[:] = a; info['fail'] = bad.float(); info['trunc'] = trunc.float(); info['nan_worlds'] = (~torch.isfinite(qpos).all(-1)).float().sum().expand(1) + return r, done, trunc, info +# ---------------- PPO ---------------- +class RunningNorm: + def __init__(s, n): s.mean = torch.zeros(n, device=dev); s.var = torch.ones(n, device=dev); s.count = 1e-4 + def update(s, x): + x = x[torch.isfinite(x).all(-1)] + if len(x) < 2: return + bm, bv, bc = x.mean(0), x.var(0, unbiased=False), x.shape[0]; delta = bm - s.mean; tot = s.count + bc + s.mean = s.mean + delta * bc / tot; s.var = (s.var * s.count + bv * bc + delta ** 2 * s.count * bc / tot) / tot; s.count = tot + def __call__(s, x): return torch.nan_to_num((x - s.mean) / torch.sqrt(s.var + 1e-6), nan=0., posinf=10., neginf=-10.).clamp(-10, 10) + def state(s): return dict(mean=s.mean, var=s.var, count=s.count) +def mlp(i, o, h=(512, 256, 128)): + layers = []; last = i + for hh in h: layers += [nn.Linear(last, hh), nn.ELU()]; last = hh + return nn.Sequential(*layers, nn.Linear(last, o)) +reset(torch.ones(NW, dtype=torch.bool, device=dev)); kin(); OBS = observe(t_idx).shape[1] +actor, critic = mlp(OBS, NA).to(dev), mlp(OBS, 1).to(dev); log_std = nn.Parameter(torch.full((NA,), math.log(.5), device=dev)) +opt = torch.optim.Adam(list(actor.parameters()) + list(critic.parameters()) + [log_std], lr=3e-4); norm = RunningNorm(OBS); lr = 3e-4 +start_iter = 0 +if A.resume: + ck = torch.load(A.resume, map_location=dev); actor.load_state_dict(ck['actor']); critic.load_state_dict(ck['critic']); log_std.data[:] = ck['log_std']; norm.mean, norm.var, norm.count = ck['norm']['mean'], ck['norm']['var'], ck['norm']['count']; voc_gain[16:] = min(ck.get('voc', 1.), float(A.voc_start)); start_iter = ck.get('iter', 0); lr = ck.get('lr', lr) + for g_ in opt.param_groups: g_['lr'] = lr +GAMMA, LAM, CLIP, EPOCHS, MB = .99, .95, .2, 5, 4; H = A.horizon +from torch.utils.tensorboard import SummaryWriter +tb = SummaryWriter(str(A.out / 'tb')); csv = open(A.out / 'train_log.csv', 'a') +def evaluate(it): + res_all = {} + for mode in ['det', 'sto']: + res_all[mode] = evaluate_mode(it, mode) + res = res_all['det']; res.update({k + '_sto': v for k, v in res_all['sto'].items() if k not in ('iter',)}) + (A.out / 'eval_log.jsonl').open('a').write(json.dumps(res) + '\n'); print('EVAL', json.dumps(res), flush=True) + torch.save(dict(actor=actor.state_dict(), critic=critic.state_dict(), log_std=log_std.data, norm=norm.state(), voc=float(voc_gain[16:].mean()), iter=it, lr=lr, obs_dim=OBS), A.out / f'ckpt_iter{it:05d}.pt'); torch.save(dict(actor=actor.state_dict(), critic=critic.state_dict(), log_std=log_std.data, norm=norm.state(), voc=float(voc_gain[16:].mean()), iter=it, lr=lr, obs_dim=OBS), A.out / 'ckpt_latest.pt') + reset(torch.ones(NW, dtype=torch.bool, device=dev)); kin() +def evaluate_mode(it, mode): + reset(torch.ones(NW, dtype=torch.bool, device=dev), start=max(0, A.seg_start)); kin(); g_save = voc_gain.clone(); voc_gain[:] = 0; errs = []; rots = []; rot_each = []; err_each = []; traj = []; times = []; alive = torch.ones(NW, dtype=torch.bool, device=dev); seg_end = A.seg_end if A.seg_end > 0 else N - 1 + with torch.no_grad(): + for k in range(int(CTRL_STEPS * 1.5) + 5): + o = norm(observe(t_idx.clamp(max=N - 1))); mu = actor(o); a = (mu if mode == 'det' else torch.distributions.Normal(mu, log_std.exp()).sample()).clamp(-3, 3); r, done, trunc, info = step(a) + alive &= ~(info['fail'] > 0); errs.append(info['obj_err_cm']); rots.append(info['obj_rot_deg']); rot_each.append(info['rot_each']); err_each.append(info['err_each']); traj.append(qpos[0].clone()); times.append(float(k * DEC * m.opt.timestep)) + if t_pos[0] >= seg_end or (t_pos >= seg_end).all(): break + E = torch.stack(errs); Rr = torch.stack(rots); RE = torch.stack(rot_each); EE = torch.stack(err_each); voc_gain[:] = g_save; ok = torch.isfinite(E).all(0) & torch.isfinite(Rr).all(0); E = E[:, ok]; Rr = Rr[:, ok]; alive = alive[ok]; RE = RE[:, ok]; EE = EE[:, ok] + nlast = max(1, len(errs) // 10); final_rot = RE[-nlast:].mean(0); final_err = EE[-nlast:].mean(0) # per world, per object, last 10% of the rollout + res = dict(iter=it, finite_worlds=int(ok.sum()), final_rot_deg_red_median=float(final_rot[:, 0].median()), final_rot_deg_blue_median=float(final_rot[:, 1].median()), flip_success_blue_frac=float(((final_rot[:, 1] < 30) & (final_err[:, 1] < 10)).float().mean()), red_ok_frac=float(((final_rot[:, 0] < 30) & (final_err[:, 0] < 10)).float().mean()), phase_reached_frac=float((t_pos[0] - max(0, A.seg_start)) / (seg_end - max(0, A.seg_start))), eval_steps=len(errs), obj_err_cm_mean=float(E.mean()), obj_err_cm_max_over_time_mean=float(E.max(0).values.mean()), obj_rot_deg_mean=float(Rr.mean()), survived_frac=float(alive.float().mean()), success_5cm_frac=float((E.max(0).values < 5).float().mean()), success_10cm_frac=float((E.max(0).values < 10).float().mean())) + np.savez_compressed(A.out / (f'eval_iter{it:05d}.npz' if mode == 'det' else f'eval_iter{it:05d}_sto.npz'), qpos=torch.stack(traj).cpu().numpy(), time=np.array(times), metrics=json.dumps(res), final_rot_deg=final_rot.cpu().numpy(), final_err_cm=final_err.cpu().numpy()) + return res +robj_hist = [] +iters = 3 if A.smoke else A.iters +for it in range(start_iter, start_iter + iters): + t0 = time.perf_counter(); obs_b = torch.zeros(H, NW, OBS, device=dev); act_b = torch.zeros(H, NW, NA, device=dev); mu_b = torch.zeros(H, NW, NA, device=dev); std_old = log_std.exp().detach().clone(); logp_b = torch.zeros(H, NW, device=dev); rew_b = torch.zeros(H, NW, device=dev); done_b = torch.zeros(H, NW, device=dev); val_b = torch.zeros(H + 1, NW, device=dev); agg = {} + with torch.no_grad(): + for k in range(H): + o_raw = observe(t_idx.clamp(max=N - 1)); norm.update(o_raw); o = norm(o_raw); mu = actor(o); std = log_std.exp(); dist = torch.distributions.Normal(mu, std); a = dist.sample().clamp(-3, 3); logp = dist.log_prob(a).sum(-1); v = critic(o).squeeze(-1) + r, done, trunc, info = step(a); r = torch.nan_to_num(r, nan=-1., posinf=1., neginf=-1.) + # bootstrap truncated episodes with value of next state + if trunc.any(): + kin(); vn = critic(norm(observe(t_idx.clamp(max=N - 1)))).squeeze(-1); r = torch.where(trunc & ~(info['fail'] > 0), r + GAMMA * vn, r) + obs_b[k], act_b[k], logp_b[k], rew_b[k], done_b[k], val_b[k], mu_b[k] = o, a, logp, r, done.float(), v, mu + for kk, vv in info.items(): agg.setdefault(kk, []).append(vv.mean().item()) + reset(done); kin() + val_b[H] = critic(norm(observe(t_idx.clamp(max=N - 1)))).squeeze(-1) + adv = torch.zeros(H, NW, device=dev); last = torch.zeros(NW, device=dev) + for k in reversed(range(H)): + nd = 1. - done_b[k]; delta = rew_b[k] + GAMMA * val_b[k + 1] * nd - val_b[k]; last = delta + GAMMA * LAM * nd * last; adv[k] = last + ret = adv + val_b[:H]; adv_n = (adv - adv.mean()) / (adv.std() + 1e-8) + B = H * NW; ob, ab, lpb, rb, advb, vb, mub = obs_b.reshape(B, -1), act_b.reshape(B, -1), logp_b.reshape(B), ret.reshape(B), adv_n.reshape(B), val_b[:H].reshape(B), mu_b.reshape(B, -1); kls = [] + for ep in range(EPOCHS): + perm = torch.randperm(B, device=dev) + for mb in range(MB): + i = perm[mb * B // MB:(mb + 1) * B // MB]; mu_new = actor(ob[i]); dist = torch.distributions.Normal(mu_new, log_std.exp()); lp = dist.log_prob(ab[i]).sum(-1); ratio = torch.exp(lp - lpb[i]) + s1 = ratio * advb[i]; s2 = ratio.clamp(1 - CLIP, 1 + CLIP) * advb[i]; pl = -torch.min(s1, s2).mean(); vpred = critic(ob[i]).squeeze(-1); vcl = vb[i] + (vpred - vb[i]).clamp(-CLIP, CLIP); vl = torch.max((vpred - rb[i]) ** 2, (vcl - rb[i]) ** 2).mean() + loss = pl + .5 * vl; opt.zero_grad(); loss.backward(); nn.utils.clip_grad_norm_(list(actor.parameters()) + list(critic.parameters()) + [log_std], 1.); opt.step() + with torch.no_grad(): # analytic KL(old || new) between diagonal Gaussians (rsl_rl style) + s_new = log_std.exp(); kl = (torch.log(s_new / std_old) + (std_old ** 2 + (mub[i] - mu_new) ** 2) / (2 * s_new ** 2) - .5).sum(-1).mean().item(); kls.append(kl) + if kl > .02: lr = max(1e-5, lr / 1.5) + elif kl < .005: lr = min(1e-3, lr * 1.5) + for g_ in opt.param_groups: g_['lr'] = lr + # VOC curriculum: decay when object tracking is good; hard schedule guarantees zero by 70% of training + robj_hist.append(np.mean(agg['r_obj'])); gated = len(robj_hist) >= 10 and np.mean(robj_hist[-10:]) > .6 + zero_iters = A.voc_zero_iters if A.voc_zero_iters > 0 else int(.7 * A.iters); sched = max(0., A.voc_start * (1. - (it - start_iter) / zero_iters)); g = voc_gain[16:].mean().item() + if gated: g *= .97 + g = min(g, sched); g = 0. if g < .02 else g; voc_gain[16:] = g + fps = B * DEC / (time.perf_counter() - t0); row = dict(iter=it, fps=int(fps), rew=float(rew_b.mean()), ep_len=float(ep_len.float().mean()), voc=g, lr=lr, kl=float(np.mean(kls)), std=float(log_std.exp().mean()), **{k: float(np.mean(v)) for k, v in agg.items()}) + for k, v in row.items(): tb.add_scalar(k, v, it) + csv.write(json.dumps(row) + '\n'); csv.flush() + if it % 5 == 0 or A.smoke: print(' '.join(f'{k}={v:.3g}' if isinstance(v, float) else f'{k}={v}' for k, v in row.items()), flush=True) + if (it + 1) % A.eval_every == 0 or A.smoke and it == start_iter + iters - 1: evaluate(it + 1) +print('TRAIN_DONE', flush=True) diff --git a/scripts/run_dynhamr_rgbd_20260915.py b/scripts/run_dynhamr_rgbd_20260915.py new file mode 100644 index 0000000..dda0d8d --- /dev/null +++ b/scripts/run_dynhamr_rgbd_20260915.py @@ -0,0 +1,49 @@ +"""Run both-hand Dyn-HaMR with measured intrinsics and RGB-D odometry.""" +import json +import os +from pathlib import Path +import subprocess +import time +import numpy as np + +ROOT = Path(__file__).resolve().parents[1] +OUT = ROOT / 'output/20260915_171525_dynhamr' +OUT.mkdir(parents=True, exist_ok=True) +SEQ = '20260915_171525' +dataset = OUT / 'dataset' +(dataset / 'videos').mkdir(parents=True, exist_ok=True) +video = dataset / 'videos' / (SEQ + '.mp4') +if not video.exists(): + video.symlink_to(ROOT / 'docs' / SEQ / 'color.mp4') +camera = np.load(ROOT / 'output' / SEQ / 'rgbd_camera.npz') +assert camera['c2w'].shape == (352, 4, 4) +for kind, data in [('pose', camera['c2w']), ('intrinsics', np.tile(camera['intrinsics'], (352, 1)))]: + directory = OUT / 'rgbd_cameras' / kind + directory.mkdir(parents=True, exist_ok=True) + np.savez(directory / (SEQ + '.npz'), data=data.astype(np.float32), inds=np.arange(352)) +(OUT / 'camera_provenance.json').write_text(json.dumps(dict( + source=str(ROOT / 'output' / SEQ / 'rgbd_camera.npz'), + method=str(camera['method']), adapter='ViPE file schema only; cameras estimated from D405 RGB-D', + hand_reconstruction='Dyn-HaMR', hands=['left', 'right']), indent=2)) +dyn = ROOT / 'third_party/Dyn-HaMR' +env = os.environ.copy() +env.update(TORCH_HOME=str(ROOT / '.torch_cache'), HF_HOME=str(ROOT / '.hf_cache'), + HF_HUB_OFFLINE='1', TRANSFORMERS_OFFLINE='1', PYTHONUNBUFFERED='1', + MPLCONFIGDIR='/tmp/matplotlib-dynhamr', HYDRA_FULL_ERROR='1', + PYOPENGL_PLATFORM='egl', LD_LIBRARY_PATH='') +env['PATH'] = str(dyn / '.dynhamr/bin') + ':' + env['PATH'] +env['PYTHONPATH'] = ':'.join(str(dyn / p) for p in ['dyn-hamr', 'third-party/hamer', 'third-party/hamer/third-party/ViTPose']) +cmd = [str(dyn / '.dynhamr/bin/python'), '-u', 'run_opt.py', 'data=video_vipe', + 'data.root=' + str(dataset), 'data.seq="' + SEQ + '"', + 'data.vipe_dir=' + str(OUT / 'rgbd_cameras'), 'data.frame_opts.fps=30', + 'is_static=False', 'model.opt_scale=False', 'run_opt=True', 'run_vis=True', + 'run_prior=True', '+data.vipe_pipeline=dynhamr_cameras', + 'HMP.vid_path=' + str(video), 'hydra.run.dir=' + str(OUT / 'optimization')] +def status(stage, **kw): + (OUT / 'status.json').write_text(json.dumps(dict(stage=stage, + updated=time.strftime('%Y-%m-%d %H:%M:%S'), pid=os.getpid(), **kw), indent=2)) +status('RUNNING', command=cmd) +with (OUT / 'run.log').open('a') as log: + result = subprocess.run(cmd, cwd=dyn / 'dyn-hamr', env=env, stdout=log, stderr=subprocess.STDOUT) +status('RECONSTRUCTION_PROCESS_EXITED' if result.returncode == 0 else 'FAILED', returncode=result.returncode) +raise SystemExit(result.returncode) diff --git a/scripts/run_red_handflow_comparison.py b/scripts/run_red_handflow_comparison.py new file mode 100644 index 0000000..a7ddb75 --- /dev/null +++ b/scripts/run_red_handflow_comparison.py @@ -0,0 +1,15 @@ +"""Prepare RGB-only HandFlow baseline for both hands with fixed camera calibration.""" +import os,subprocess,json +from pathlib import Path +ROOT=Path(__file__).resolve().parents[1];OUT=ROOT/'output/depth_ablation_red_20260916';m=json.loads((ROOT/'docs/20260916_104026/intrinsics.json').read_text()) +env=os.environ.copy();env.update(PYTHONPATH=str(ROOT),LD_LIBRARY_PATH='',OMP_NUM_THREADS='4',MKL_NUM_THREADS='4',OPENBLAS_NUM_THREADS='4',TORCH_HOME=str(ROOT/'.torch_cache'),HF_HOME=str(ROOT/'.hf_cache'),HF_HUB_OFFLINE='1',TRANSFORMERS_OFFLINE='1',TORCH_FORCE_NO_WEIGHTS_ONLY_LOAD='1',PYTHONUNBUFFERED='1',HAMER_CKPT=str(ROOT/'third_party/hamer/_DATA/hamer_ckpts/checkpoints/hamer.ckpt'),DETECTOR_CKPT=str(ROOT/'weights/detector.pt'),MANO_ROOT=str(ROOT/'third_party/hamer/_DATA/data/mano'),HANDFLOW_NORMALIZATION_STATS=str(ROOT/'weights/normalization_stats.npz')) +for side in ['right','left']: + dst=OUT/side;dst.mkdir(exist_ok=True) + if (dst/'handflow_results.npz').exists():continue + cmd=[str(ROOT/'.venv/bin/python'),'-u','scripts/demo.py','--input',str(OUT/'input_315.mkv'),'--fm_ckpt',str(ROOT/'weights/handflow_denoiser.pt'),'--intrinsics',','.join(str(m[k]) for k in ['fx','fy','cx','cy']),'--fix_camera','--side',side,'--output_dir',str(dst),'--save_npz',str(dst/'handflow_results.npz')] + print('Running',side,flush=True) + for attempt in range(2): + if (dst/'handflow_results.npz').exists():break + with (dst/'inference.log').open('a') as f:subprocess.run(cmd,cwd=ROOT,env=env,stdout=f,stderr=subprocess.STDOUT,check=True) + assert (dst/'handflow_results.npz').exists(), f'Missing final results: {side}' +print('Both RGB-only baselines complete',flush=True) diff --git a/scripts/run_red_mirrored_handflow.py b/scripts/run_red_mirrored_handflow.py new file mode 100644 index 0000000..63b68fe --- /dev/null +++ b/scripts/run_red_mirrored_handflow.py @@ -0,0 +1,16 @@ +"""Prepare RGB-only HandFlow baseline for both hands with fixed camera calibration.""" +import os,subprocess,json +from pathlib import Path +ROOT=Path(__file__).resolve().parents[1];OUT=ROOT/'output/depth_ablation_red_20260916';m=json.loads((ROOT/'docs/20260916_104026/intrinsics.json').read_text()) +env=os.environ.copy();env.update(PYTHONPATH=str(ROOT),LD_LIBRARY_PATH='',OMP_NUM_THREADS='4',MKL_NUM_THREADS='4',OPENBLAS_NUM_THREADS='4',TORCH_HOME=str(ROOT/'.torch_cache'),HF_HOME=str(ROOT/'.hf_cache'),HF_HUB_OFFLINE='1',TRANSFORMERS_OFFLINE='1',TORCH_FORCE_NO_WEIGHTS_ONLY_LOAD='1',PYTHONUNBUFFERED='1',HAMER_CKPT=str(ROOT/'third_party/hamer/_DATA/hamer_ckpts/checkpoints/hamer.ckpt'),DETECTOR_CKPT=str(ROOT/'weights/detector.pt'),MANO_ROOT=str(ROOT/'third_party/hamer/_DATA/data/mano'),HANDFLOW_NORMALIZATION_STATS=str(ROOT/'weights/normalization_stats.npz')) +m['cx']=m['width']-1-m['cx'] +for side in ['left_mirrored']: + dst=OUT/side;dst.mkdir(exist_ok=True) + if (dst/'handflow_results.npz').exists():continue + cmd=[str(ROOT/'.venv/bin/python'),'-u','scripts/demo.py','--input',str(OUT/'input_315_mirrored.mkv'),'--fm_ckpt',str(ROOT/'weights/handflow_denoiser.pt'),'--intrinsics',','.join(str(m[k]) for k in ['fx','fy','cx','cy']),'--fix_camera','--side','right','--output_dir',str(dst),'--save_npz',str(dst/'handflow_results.npz')] + print('Running',side,flush=True) + for attempt in range(2): + if (dst/'handflow_results.npz').exists():break + with (dst/'inference.log').open('a') as f:subprocess.run(cmd,cwd=ROOT,env=env,stdout=f,stderr=subprocess.STDOUT,check=True) + assert (dst/'handflow_results.npz').exists(), f'Missing final results: {side}' +print('Mirrored-left RGB-only baseline complete',flush=True) diff --git a/scripts/run_rgbd_20260915.py b/scripts/run_rgbd_20260915.py new file mode 100644 index 0000000..b667373 --- /dev/null +++ b/scripts/run_rgbd_20260915.py @@ -0,0 +1,47 @@ +"""Resume the 20260915_171525 RGB-D right-hand reconstruction.""" +import os +from pathlib import Path +import subprocess +import json +import time +import argparse +import numpy as np + +parser=argparse.ArgumentParser() +parser.add_argument('--left',action='store_true') +args=parser.parse_args() + +ROOT=Path(__file__).resolve().parents[1] +SOURCE=ROOT/'docs/20260915_171525' +OUT=ROOT/('output/20260915_171525/handflow_left_mirrored' if args.left else 'output/20260915_171525/handflow') +OUT.mkdir(parents=True,exist_ok=True) +meta=json.loads((SOURCE/'intrinsics.json').read_text()) +env=os.environ.copy() +env.update(PYTHONPATH=str(ROOT),LD_LIBRARY_PATH='',OMP_NUM_THREADS='4',MKL_NUM_THREADS='4', + OPENBLAS_NUM_THREADS='4',TORCH_HOME=str(ROOT/'.torch_cache'),HF_HOME=str(ROOT/'.hf_cache'), + HF_HUB_OFFLINE='1',TRANSFORMERS_OFFLINE='1',TORCH_FORCE_NO_WEIGHTS_ONLY_LOAD='1', + PYTHONUNBUFFERED='1',VIPE_PYTHON=str(ROOT/'.venv/bin/python'), + HAMER_CKPT=str(ROOT/'third_party/hamer/_DATA/hamer_ckpts/checkpoints/hamer.ckpt'), + DETECTOR_CKPT=str(ROOT/'weights/detector.pt'), + MANO_ROOT=str(ROOT/'third_party/hamer/_DATA/data/mano'), + HANDFLOW_NORMALIZATION_STATS=str(ROOT/'weights/normalization_stats.npz')) +cmd=[str(ROOT/'.venv/bin/python'),'-u','scripts/demo.py','--input',str(SOURCE/'color.mp4'), + '--fm_ckpt',str(ROOT/'weights/handflow_denoiser.pt'),'--intrinsics', + ','.join(str(meta[k]) for k in ['fx','fy','cx','cy']),'--side','right', + '--rotation_smooth_window','5','--output_dir',str(OUT),'--save_npz',str(OUT/'handflow_results.npz')] +if args.left: + cmd[cmd.index('--input')+1]=str(OUT.parent/'left_mirrored.mkv') + intr=np.array([meta[k] for k in ['fx','fy','cx','cy']]);intr[2]=meta['width']-1-intr[2] + c=np.load(OUT.parent/'rgbd_camera.npz');reflection=np.diag([-1.,1.,1.,1.]) + np.savez(OUT/'camera_slam.npz',intrinsics=intr,c2w=reflection@c['c2w']@reflection) +status=OUT.parent/'status.json' +for attempt in range(2): + if (OUT/'handflow_results.npz').exists():break + status.write_text(json.dumps(dict(stage='RECONSTRUCTION_RUNNING',attempt=attempt,command=cmd,time=time.time()),indent=2)) + with (OUT/'inference.log').open('a') as log: + result=subprocess.run(cmd,cwd=ROOT,env=env,stdout=log,stderr=subprocess.STDOUT) + if result.returncode: + status.write_text(json.dumps(dict(stage='RECONSTRUCTION_FAILED',returncode=result.returncode,time=time.time()),indent=2)) + raise SystemExit(result.returncode) +if not (OUT/'handflow_results.npz').exists():raise RuntimeError('No final reconstruction arrays') +status.write_text(json.dumps(dict(stage='RECONSTRUCTION_DONE_PENDING_DEPTH_AND_RETARGET',time=time.time()),indent=2)) diff --git a/scripts/run_yesterday_spider.py b/scripts/run_yesterday_spider.py new file mode 100644 index 0000000..764de6f --- /dev/null +++ b/scripts/run_yesterday_spider.py @@ -0,0 +1,38 @@ +"""Actual upstream SPIDER, with L20 passive-joint reward indexing corrected locally.""" +import os,sys,json,importlib.util,argparse +from pathlib import Path +ROOT=Path(__file__).resolve().parents[1];OUT=Path(os.environ.get('SPIDER_TASK_OUT',str(ROOT/'output/foundationpose_spider_20260915'))) +os.environ['WARP_CACHE_PATH']=str(ROOT/'.spider_cache/warp');os.environ['TORCHINDUCTOR_CACHE_DIR']=str(ROOT/'.spider_cache/torchinductor');os.environ.setdefault('MUJOCO_GL','osmesa');os.environ['OMP_NUM_THREADS']='4' +sys.path.insert(0,str(ROOT/'third_party/spider')) +import torch +torch.set_num_threads(4) +from spider.config import Config +import spider.simulators.mjwp as sim +original=sim._weight_diff_qpos +def weights(config): + if config.embodiment_type!='bimanual':return original(config) + # 27 state DOFs per L20 (6 wrist+21 finger), versus 22 actuator channels. + half=(config.nv-12)//2;w=torch.full((config.nv,),config.joint_rew_scale,device=config.device) + for start in [0,half]:w[start:start+3]=config.base_pos_rew_scale;w[start+3:start+6]=config.base_rot_rew_scale + for start in [config.nv-12,config.nv-6]:w[start:start+3]=config.pos_rew_scale;w[start+3:start+6]=config.rot_rew_scale + return w +sim._weight_diff_qpos=weights +spec=importlib.util.spec_from_file_location('spider_original_runner',ROOT/'third_party/spider/examples/run_mjwp.py');module=importlib.util.module_from_spec(spec);sys.modules[spec.name]=module;spec.loader.exec_module(module) +# Cross-object regrasp invalidates upstream one-object-per-hand delta heuristic. +# Retain reference object assistance schedule and per-finger contact reward. +module.compute_contact_point_delta=lambda *args,**kwargs: None +get_qpos_original=module.get_qpos +def checked_qpos(config,env): + q=get_qpos_original(config,env) + if not torch.isfinite(q[0]).all():raise RuntimeError('Nonfinite executed state: stop instead of saving invalid trajectory') + return q +module.get_qpos=checked_qpos +p=argparse.ArgumentParser();p.add_argument('--pilot',action='store_true');p.add_argument('--steps',type=int);args=p.parse_args();c=json.loads((OUT/'config.json').read_text()) +if args.pilot:c.update(max_sim_steps=80,num_samples=32,max_num_iterations=2) +if args.steps:c['max_sim_steps']=args.steps +if (OUT/'physics_parameters.json').exists(): + settings=json.loads((OUT/'physics_parameters.json').read_text());setup_original=sim.setup_mj_model + def configured_model(config): + m=setup_original(config);m.opt.iterations=settings['solver_iterations'];m.opt.ls_iterations=settings['ls_iterations'];return m + sim.setup_mj_model=configured_model;module.setup_mj_model=configured_model +module.main(Config(**c));print('SPIDER_RUN_COMPLETE',flush=True) diff --git a/scripts/smooth_object_reference.py b/scripts/smooth_object_reference.py new file mode 100644 index 0000000..5237b71 --- /dev/null +++ b/scripts/smooth_object_reference.py @@ -0,0 +1,200 @@ +"""Physically-consistent object reference: rest segments frozen (and snapped to the table), +held segments rigidly bound to the holding hand, free segments lightly smoothed, camera odometry smoothed. + +Input : output/registered_spider_20260915/foundationpose_objects.npz (raw FoundationPose, raw odometry) + output/20260915_171525_dynhamr/registered/ (hands, raw-odometry world) +Output: output/final_spider_20260915/foundationpose_objects.npz (same schema; *_valid all True where a pose is + defined, *_observed_raw keeps the original acceptance flags), rgbd_camera_smooth.npz, object_segments.npz, + object_smoothing.json. Validation renders the CAD into the sensor depth before/after. +""" +import os, sys, json +os.environ.setdefault('OMP_NUM_THREADS', '4') +from pathlib import Path +ROOT = Path(__file__).resolve().parents[1] +sys.path.insert(0, str(ROOT / 'third_party/FoundationPose')); sys.path.insert(0, str(ROOT / 'scripts')) +import numpy as np, cv2, torch, trimesh, open3d as o3d, nvdiffrast.torch as dr +from scipy.ndimage import gaussian_filter1d +from scipy.spatial.transform import Rotation as R, Slerp +from Utils import nvdiffrast_render +from red_box_distance import signed_distance + +SRC = ROOT / 'docs/20260915_171525'; OLD = ROOT / 'output/registered_spider_20260915'; HAND = ROOT / 'output/20260915_171525_dynhamr/registered' +OUT = ROOT / 'output/final_spider_20260915'; OUT.mkdir(exist_ok=True) +meta = json.loads((SRC / 'intrinsics.json').read_text()); fp = dict(np.load(OLD / 'foundationpose_objects.npz')); N = 352 +K = fp['K']; c2w_raw = fp['c2w']; w2cR = np.transpose(c2w_raw[:, :3, :3], (0, 2, 1)); tO = -np.einsum('tij,tj->ti', w2cR, c2w_raw[:, :3, 3]) +SIG_CAM, SIG_FREE, SIG_HELD, BLEND = 2., 2., 4., 4 +REST_WIN, REST_POS, REST_ROT, REST_MIN = 15, .006, 2., 10 # 0.5 s window, 6 mm, 2 deg, min 10 frames +ENGAGE, RELEASE, MIN_TIPS = .025, .035, 2 + +def smooth_rot(rot, sigma, idx=None): + n = len(rot); out = [] + for t in range(n): + ix = np.arange(max(0, t - int(3 * sigma)), min(n, t + int(3 * sigma) + 1)); w = np.exp(-.5 * ((ix - t) / sigma) ** 2) + out.append(rot[ix].mean(weights=w)) + return R.concatenate(out) +def smooth_T(T, sigma): + out = T.copy(); out[:, :3, 3] = gaussian_filter1d(T[:, :3, 3], sigma, axis=0, mode='nearest'); out[:, :3, :3] = smooth_rot(R.from_matrix(T[:, :3, :3]), sigma).as_matrix(); return out +def interp_T(T, valid): + idx = np.flatnonzero(valid); t = np.arange(len(T)); out = T.copy(); clip = np.clip(t, idx[0], idx[-1]) + out[:, :3, 3] = np.stack([np.interp(t, idx, T[idx, k, 3]) for k in range(3)], 1); out[:, :3, :3] = Slerp(idx, R.from_matrix(T[idx, :3, :3]))(clip).as_matrix(); out[:, 3, :] = [0, 0, 0, 1]; return out +def inv(T): o = np.eye(4); o[:3, :3] = T[:3, :3].T; o[:3, 3] = -T[:3, :3].T @ T[:3, 3]; return o + +# --- camera: smoothed odometry --- +c2w_s = smooth_T(c2w_raw, SIG_CAM) +np.savez_compressed(OUT / 'rgbd_camera_smooth.npz', c2w=c2w_s, intrinsics=np.array([meta[k] for k in ['fx', 'fy', 'cx', 'cy']]), time=fp['time'], valid=fp['camera_valid'], method='registered_spider c2w, Gaussian sigma 2 frames on position and SO(3)') +# --- hands in camera frame (independent of odometry) --- +hands = {} +for side in ['right', 'left']: + hj = np.load(HAND / f'human_joints_{side}.npz', allow_pickle=True); mo = np.load(HAND / f'l20_{side}_stable/motion.npz', allow_pickle=True) + tips_w = hj['joints'][:, [4, 8, 12, 16, 20]] + hj['wrist_world'][:, None]; tips_c = np.einsum('tij,tkj->tki', w2cR, tips_w) + tO[:, None] + Th = np.tile(np.eye(4), (N, 1, 1)); Th[:, :3, :3] = np.einsum('tij,tjk->tik', w2cR, mo['wrist_world_R']); Th[:, :3, 3] = np.einsum('tij,tj->ti', w2cR, mo['wrist_world']) + tO + hands[side] = dict(tips_cam=tips_c, T_cam=Th) +# --- table plane (frame 0, camera-0 frame), as in prepare --- +cap = cv2.VideoCapture(str(SRC / 'color.mp4')); ok, img0 = cap.read(); cap.release(); hsv0 = cv2.cvtColor(img0, cv2.COLOR_BGR2HSV) +dep0 = cv2.imread(str(SRC / 'depth/000000.png'), -1) * meta['depth_scale_m']; yy, xx = np.indices(dep0.shape) +mask = (xx > 100) & (xx < 780) & (yy > 200) & (yy < 460) & (hsv0[:, :, 1] < 50) & (dep0 > .2) & (dep0 < 1) +pts = np.stack([(xx - K[0, 2]) * dep0 / K[0, 0], (yy - K[1, 2]) * dep0 / K[1, 1], dep0], -1)[mask][::3] +pc = o3d.geometry.PointCloud(o3d.utility.Vector3dVector(pts)); plane, _ = pc.segment_plane(.004, 3, 1000); n_cam = np.array(plane[:3]); off = plane[3] +if n_cam[2] > 0: n_cam, off = -n_cam, -off # normal points up (toward the camera side) +# --- meshes --- +scenes = {} +for name, stl in [('upper', '上半.stl'), ('lower', '下半.stl')]: + closed = trimesh.load(ROOT / f'output/collision_fix_20260915/collision_v2/{name}_closed.ply', process=False); assert closed.is_watertight + sc = o3d.t.geometry.RaycastingScene(); sc.add_triangles(o3d.t.geometry.TriangleMesh.from_legacy(o3d.geometry.TriangleMesh(o3d.utility.Vector3dVector(closed.vertices), o3d.utility.Vector3iVector(closed.faces)))) + scenes[name] = dict(scene=sc, closed=closed, render=trimesh.load(ROOT / 'docs' / stl)) +def tip_dist(name, T_cam, tips_cam): + local = (tips_cam - T_cam[:3, 3]) @ T_cam[:3, :3]; return np.abs(signed_distance(scenes[name]['scene'], scenes[name]['closed'], local)) + +ctx = dr.RasterizeCudaContext(); mt = {} +for name in scenes: + m_ = scenes[name]['render']; mt[name] = {'pos': torch.as_tensor(m_.vertices, device='cuda', dtype=torch.float32), 'faces': torch.as_tensor(m_.faces, device='cuda', dtype=torch.int32), 'vnormals': torch.zeros((len(m_.vertices), 3), device='cuda'), 'vertex_color': torch.ones((len(m_.vertices), 3), device='cuda')} +def render(name, T): + with torch.inference_mode(): _, d, _ = nvdiffrast_render(K=K, H=480, W=848, ob_in_cams=torch.as_tensor(T[None], device='cuda', dtype=torch.float32), glctx=ctx, mesh_tensors=mt[name]) + return d[0].cpu().numpy() +_cap = cv2.VideoCapture(str(SRC / 'color.mp4')); _frames = {} +def frame_data(f): + if f not in _frames: + _cap.set(cv2.CAP_PROP_POS_FRAMES, f); ok, b = _cap.read(); assert ok + z = cv2.imread(str(SRC / 'depth' / f'{f:06d}.png'), -1).astype('float32') * meta['depth_scale_m']; h = cv2.cvtColor(b, cv2.COLOR_BGR2HSV); _frames[f] = (z, h) + return _frames[f] +def depth_residual(name, f, T): + z, h = frame_data(f); hue = h[:, :, 0] + cm = (((hue < 12) | (hue > 170)) if name == 'upper' else ((hue > 95) & (hue < 135))) & (h[:, :, 1] > 90) & (h[:, :, 2] > 40) & (z > .1) & (z < 1.); cm[:200] = False + d = render(name, T); over = (d > 0) & cm + return float(np.median(np.abs(d[over] - z[over])) * 1000) if over.sum() > 200 else None +report = {'parameters': dict(camera_sigma=SIG_CAM, free_sigma=SIG_FREE, held_relative_sigma=SIG_HELD, blend_frames=BLEND, rest_window=REST_WIN, rest_pos_m=REST_POS, rest_rot_deg=REST_ROT, engage_m=ENGAGE, release_m=RELEASE, min_tips=MIN_TIPS), 'objects': {}} +out = dict(fp); labels = {} +for name in ['upper', 'lower']: + T_raw = fp[name + '_T_camera'].copy(); valid = fp[name + '_valid'].astype(bool).copy(); obs = valid.copy() + last = N if name == 'upper' else 177 # lower unobservable after 176: hold + T = interp_T(T_raw, valid); T[last:] = T[last - 1] + # holder per frame (camera frame, hysteresis per hand) + holder = np.full(N, '', dtype=object); engaged = {s: np.zeros(5, bool) for s in hands} + for t in range(N): + best, best_n = '', 0 + for s in hands: + dist = tip_dist(name, T[t], hands[s]['tips_cam'][t]); engaged[s] = np.where(engaged[s], dist < RELEASE, dist < ENGAGE) + n_on = int(engaged[s].sum()) + if n_on >= MIN_TIPS and (n_on > best_n or (n_on == best_n and s == holder[t - 1] if t else False)): best, best_n = s, n_on + holder[t] = best + # rest test on lightly pre-smoothed poses, over a 0.5 s window + pre = smooth_T(T, 2.); rest = np.zeros(N, bool); h = REST_WIN // 2 + for t in range(N): + a, b = max(0, t - h), min(N - 1, t + h); dp = np.linalg.norm(pre[b, :3, 3] - pre[a, :3, 3]); dr_ = (R.from_matrix(pre[a, :3, :3]).inv() * R.from_matrix(pre[b, :3, :3])).magnitude() * 180 / np.pi + rest[t] = dp < REST_POS and dr_ < REST_ROT + rest[last:] = True + lab = np.array(['free'] * N, dtype=object); lab[rest] = 'rest' + for t in range(N): + if not rest[t] and holder[t]: lab[t] = 'held:' + holder[t] + # drop short rest runs; then merge rest-(free with no hand)-rest, since a static object cannot move by itself + t = 0 + while t < N: + u = t + while u < N and lab[u] == lab[t]: u += 1 + if lab[t] == 'rest' and u - t < REST_MIN: lab[t:u] = 'free' + t = u + changed = True + while changed: + changed = False; t = 0 + while t < N: + u = t + while u < N and lab[u] == lab[t]: u += 1 + if lab[t] == 'free' and t > 0 and u < N and lab[t - 1] == 'rest' and lab[u] == 'rest' and not any(holder[t:u]): + lab[t:u] = 'rest'; changed = True + t = u + # per-segment poses + S = T.copy(); alt = T.copy(); segs = []; t = 0; depth_split = 0 + while t < N: + u = t + while u < N and lab[u] == lab[t]: u += 1 + kind = lab[t]; sl = slice(t, u) + if kind == 'rest': + # consensus pose: among the median and sampled observed poses, take the one the sensor depth agrees with best + fr = [f for f in range(t, u, 3) if obs[f]] + Tm = np.eye(4); Tm[:3, :3] = R.from_matrix(T[sl, :3, :3]).mean().as_matrix(); Tm[:3, 3] = np.median(T[sl, :3, 3], axis=0) + cands = [('median', Tm)] + [(f'obs{f}', T[f]) for f in range(t, u, max(1, (u - t) // 12)) if obs[f]] + def score(Tc): rs = [depth_residual(name, f, Tc) for f in fr]; rs = [x for x in rs if x is not None]; return float(np.median(rs)) if rs else 1e9 + scored = [(score(Tc), lab_, Tc) for lab_, Tc in cands]; best_score, best_lab, Tr = min(scored, key=lambda x: x[0]); Tr = Tr.copy() + cl = scenes[name]['closed']; hv = (cl.vertices @ Tr[:3, :3].T + Tr[:3, 3]) @ n_cam + off; gap = float(hv.min()) # signed height of lowest vertex above the table + snapped = False + if abs(gap) < .03: + Ts = Tr.copy(); Ts[:3, 3] -= n_cam * gap; snap_score = score(Ts) + if snap_score <= best_score + 1.0: Tr, snapped, best_score = Ts, True, snap_score + # split test: if the consensus pose disagrees with the sensor for a sustained run, the object actually moved + per = [(f, depth_residual(name, f, Tr), depth_residual(name, f, T[f])) for f in fr]; bad = [f for f, a, b in per if a is not None and b is not None and a > b + 3.0] + runs = []; + for f in bad: + if runs and f - runs[-1][-1] <= 3: runs[-1].append(f) + else: runs.append([f]) + long_runs = [r for r in runs if len(r) >= 3] + if long_runs and u - t >= 2 * REST_MIN and depth_split < 3: + cut = long_runs[0][0] if long_runs[0][0] - t >= REST_MIN else long_runs[0][-1] + 1 + if t + REST_MIN <= cut <= u - REST_MIN: + lab[cut:cut + 3] = 'free'; depth_split += 1; continue # re-segment from t with the new boundary + S[sl] = Tr; segs.append(dict(kind='rest', start=t, end=u - 1, table_gap_mm=gap * 1000, snapped=snapped, pose_source=best_lab, depth_residual_mm=best_score, median_pose_residual_mm=scored[0][0], split_runs=[(r[0], r[-1]) for r in long_runs])) + elif kind.startswith('held'): + s = kind.split(':')[1]; Th = hands[s]['T_cam'][sl]; rel = np.array([inv(Th[i]) @ T[t + i] for i in range(u - t)]) + rel_s = smooth_T(rel, SIG_HELD) if u - t > 2 else rel; S[sl] = np.array([Th[i] @ rel_s[i] for i in range(u - t)]) + alt[sl] = smooth_T(T[sl], SIG_FREE) if u - t > 2 else T[sl] + segs.append(dict(kind=kind, start=t, end=u - 1, relative_position_2nd_diff_rms_mm_before=float(np.sqrt((np.linalg.norm(np.diff(rel[:, :3, 3], n=2, axis=0), axis=1) ** 2).mean()) * 1000) if u - t > 2 else 0., after=float(np.sqrt((np.linalg.norm(np.diff(rel_s[:, :3, 3], n=2, axis=0), axis=1) ** 2).mean()) * 1000) if u - t > 2 else 0.)) + else: + S[sl] = smooth_T(T[sl], SIG_FREE) if u - t > 2 else T[sl]; segs.append(dict(kind='free', start=t, end=u - 1)) + t = u + # held segments: keep the hand-bound version only if the sensor depth agrees at least as well as plain smoothing + for sg in segs: + if not sg['kind'].startswith('held') or sg['end'] - sg['start'] < 3: continue + fr = [f for f in range(sg['start'], sg['end'] + 1, 3) if obs[f]] + rb = [depth_residual(name, f, S[f]) for f in fr]; ra = [depth_residual(name, f, alt[f]) for f in fr]; rb = [x for x in rb if x is not None]; ra = [x for x in ra if x is not None] + sg['depth_residual_bound_mm'] = float(np.median(rb)) if rb else None; sg['depth_residual_smoothed_mm'] = float(np.median(ra)) if ra else None + if rb and ra and np.median(rb) > np.median(ra) + 1.0: S[sg['start']:sg['end'] + 1] = alt[sg['start']:sg['end'] + 1]; sg['chosen'] = 'own-trajectory smoothing' + else: sg['chosen'] = 'hand-bound' + # blend across segment boundaries + B = S.copy() + for i in range(1, len(segs)): + b = segs[i]['start']; a0, a1 = max(0, b - BLEND), min(N, b + BLEND); rot = R.from_matrix(np.stack([S[a0, :3, :3], S[a1 - 1, :3, :3]])); sl = Slerp([a0, a1 - 1], rot) + for k in range(a0, a1): + w = (k - a0) / max(1, a1 - 1 - a0); B[k, :3, 3] = (1 - w) * S[a0, :3, 3] + w * S[a1 - 1, :3, 3]; B[k, :3, :3] = sl(k).as_matrix() + # keep observed frames close to observation: report deviation + dev = np.linalg.norm(B[obs, :3, 3] - T_raw[obs, :3, 3], axis=1) * 1000; rdev = (R.from_matrix(B[obs, :3, :3]).inv() * R.from_matrix(T_raw[obs, :3, :3])).magnitude() * 180 / np.pi + def jit(X): p = X[:, :3, 3]; rr = R.from_matrix(X[:, :3, :3]); return dict(pos_2nd_diff_rms_mm=float(np.sqrt((np.linalg.norm(np.diff(p, n=2, axis=0), axis=1) ** 2).mean()) * 1000), pos_step_p95_mm=float(np.percentile(np.linalg.norm(np.diff(p, axis=0), axis=1), 95) * 1000), rot_step_p95_deg=float(np.percentile((rr[:-1].inv() * rr[1:]).magnitude() * 180 / np.pi, 95))) + out[name + '_T_camera'] = B; out[name + '_T_world'] = c2w_s @ B; out[name + '_valid'] = np.r_[np.ones(last, bool), np.zeros(N - last, bool)]; out[name + '_observed_raw'] = obs + labels[name] = lab.astype(str) + report['objects'][name] = dict(segments=segs, frames_rest=int((lab == 'rest').sum()), frames_held=int(np.char.startswith(lab.astype(str), 'held').sum()), frames_free=int((lab == 'free').sum()), jitter_before=jit(T[:last]), jitter_after=jit(B[:last]), deviation_from_observed_mm=dict(median=float(np.median(dev)), p95=float(np.percentile(dev, 95)), max=float(dev.max())), deviation_rot_deg=dict(median=float(np.median(rdev)), p95=float(np.percentile(rdev, 95)))) +out['c2w'] = c2w_s +np.savez_compressed(OUT / 'foundationpose_objects.npz', **out); np.savez_compressed(OUT / 'object_segments.npz', **{k: v for k, v in labels.items()}) +# --- validation against the sensor depth (every 3rd frame) --- +val = {n: {'raw': [], 'smoothed': []} for n in scenes} +for t in range(0, N, 3): + for name in scenes: + if not fp[name + '_valid'][t]: continue + for key, T in [('raw', fp[name + '_T_camera'][t]), ('smoothed', out[name + '_T_camera'][t])]: + r = depth_residual(name, t, T) + if r is not None: val[name][key].append(r) +for name in scenes: report['objects'][name]['depth_residual_mm'] = {k: dict(median=float(np.median(v)), p95=float(np.percentile(v, 95)), frames=len(v)) for k, v in val[name].items()} +report['camera'] = dict(pos_2nd_diff_rms_mm_before=float(np.sqrt((np.linalg.norm(np.diff(c2w_raw[:, :3, 3], n=2, axis=0), axis=1) ** 2).mean()) * 1000), after=float(np.sqrt((np.linalg.norm(np.diff(c2w_s[:, :3, 3], n=2, axis=0), axis=1) ** 2).mean()) * 1000), max_change_mm=float(np.linalg.norm(c2w_s[:, :3, 3] - c2w_raw[:, :3, 3], axis=1).max() * 1000)) +report['table_plane_camera0'] = dict(normal=n_cam.tolist(), offset=float(off)) +(OUT / 'object_smoothing.json').write_text(json.dumps(report, indent=2, default=str)); print(json.dumps({k: v for k, v in report.items() if k != 'objects'}, indent=1)) +for name in scenes: + r = report['objects'][name]; print(name, 'rest/held/free', r['frames_rest'], r['frames_held'], r['frames_free'], '| jitter before', {k: round(v, 2) for k, v in r['jitter_before'].items()}, '| after', {k: round(v, 2) for k, v in r['jitter_after'].items()}) + print(' deviation from observed', {k: round(v, 1) for k, v in r['deviation_from_observed_mm'].items()}, '| depth residual', r['depth_residual_mm']) + print(' segments:', [(s['kind'], s['start'], s['end'], ('snap' if s.get('snapped') else '') + (f"gap{s['table_gap_mm']:.0f}" if 'table_gap_mm' in s else '') + (f" {s['chosen']} b{s['depth_residual_bound_mm']:.1f}/s{s['depth_residual_smoothed_mm']:.1f}" if s.get('chosen') and s.get('depth_residual_bound_mm') is not None else '')) for s in r['segments']]) diff --git a/scripts/smooth_project_reference.py b/scripts/smooth_project_reference.py new file mode 100644 index 0000000..d9e235f --- /dev/null +++ b/scripts/smooth_project_reference.py @@ -0,0 +1,114 @@ +"""Smooth the collision-corrected L20 reference, then re-project every frame to be collision-free (boxes + table). + +Smoothing is applied separately: wrist position (Gaussian), wrist rotation (SO(3) local weighted mean), +independent finger joints (Savitzky-Golay, then exact mimic expansion). Objects are untouched. +Projection: per frame, minimal weighted change s.t. linearised contact distances >= clearance and joint limits. +Env: HF_FIX_OUT (input fix dir), HF_FIX_OLD (prepare dir), HF_SMOOTH_OUT (output dir), HF_TABLE=1 to include the table, + HF_SIGMA_POS (frames, default 2), HF_SIGMA_ROT (default 2), HF_SG_WINDOW (default 9), HF_CLEARANCE (m, default .0025). +""" +import os, json, shutil +os.environ.setdefault('MUJOCO_GL', 'osmesa'); os.environ['OPENBLAS_NUM_THREADS'] = '1'; os.environ['OMP_NUM_THREADS'] = '1' +from pathlib import Path +import numpy as np, mujoco +from scipy.optimize import minimize +from scipy.ndimage import gaussian_filter1d +from scipy.signal import savgol_filter +from scipy.spatial.transform import Rotation as R +import l20_model_source as source +ROOT = Path(__file__).resolve().parents[1] +IN = Path(os.environ['HF_FIX_OUT']); OLD = Path(os.environ['HF_FIX_OLD']); OUT = Path(os.environ['HF_SMOOTH_OUT']); OUT.mkdir(parents=True, exist_ok=True) +TABLE = os.environ.get('HF_TABLE', '0') == '1'; PAIRS = [{1, 2}, {1, 4}] if TABLE else [{1, 2}] +SIG_POS = float(os.environ.get('HF_SIGMA_POS', '2')); SIG_ROT = float(os.environ.get('HF_SIGMA_ROT', '2')); SG = int(os.environ.get('HF_SG_WINDOW', '11')); CLEAR = float(os.environ.get('HF_CLEARANCE', '.0025')) +source.REPO_ROOT = ROOT / 'third_party/l20_assets' +TIN = IN / 'datasets/processed/current/l20/bimanual/boxes'; TOUT = OUT / 'datasets/processed/current/l20/bimanual/boxes'; (TOUT / '0').mkdir(parents=True, exist_ok=True) +shutil.copy2(TIN / 'scene_act.xml', TOUT / 'scene_act.xml'); shutil.copy2(TIN / 'task_info.json', TOUT / 'task_info.json') +cfg = json.loads((IN / 'config.json').read_text()); cfg['dataset_dir'] = str(OUT / 'datasets'); (OUT / 'config.json').write_text(json.dumps(cfg, indent=2)) +for name in ['collision_v2', 'hand_collision']: + if not (OUT / name).exists(): os.symlink(os.path.relpath(IN / name, OUT), OUT / name) +m = mujoco.MjModel.from_xml_path(str(TOUT / 'scene_act.xml')); d = mujoco.MjData(m) +ref = dict(np.load(IN / 'reference_video_rate.npz')); q0 = ref['qpos'].copy(); N = len(q0); contact = ref['contact']; cp = ref['contact_pos'] +meta = {} +for side in ['right', 'left']: + k = source.HandKinematics(OLD / f'model_{side}/l20_{side}.xml', side) + wa = np.array([m.jnt_qposadr[m.joint(f'{side}_hand_{s}').id] for s in ['pos_x', 'pos_y', 'pos_z', 'rot_x', 'rot_y', 'rot_z']]); ja = np.array([m.jnt_qposadr[m.joint(f'{side}_{n}').id] for n in k.joint_names]) + B = np.zeros((m.nv, 22)); B[wa, :6] = np.eye(6); B[ja, 6:] = k.expansion + meta[side] = dict(k=k, wa=wa, ja=ja, B=B, sites=[m.site(f'{side}_hand_{f}_track').id for f in source.FINGERS]) + +def smooth_rot_euler(e, sigma): + rot = R.from_euler('XYZ', e); out = [] + for t in range(len(e)): + ix = np.arange(max(0, t - int(3 * sigma)), min(len(e), t + int(3 * sigma) + 1)); w = np.exp(-.5 * ((ix - t) / sigma) ** 2); out.append(rot[ix].mean(weights=w)) + es = R.concatenate(out).as_euler('XYZ'); es += 2 * np.pi * np.round((e - es) / (2 * np.pi)) # stay on the original continuous branch + return es +def smooth(q): + qs = q.copy() + for side, mt in meta.items(): + k, wa, ja = mt['k'], mt['wa'], mt['ja'] + qs[:, wa[:3]] = gaussian_filter1d(q[:, wa[:3]], SIG_POS, axis=0, mode='nearest'); qs[:, wa[3:]] = smooth_rot_euler(q[:, wa[3:]], SIG_ROT) + ind = q[:, ja][:, k.independent_indices]; ind = savgol_filter(ind, SG, 2, axis=0, mode='nearest'); ind = np.clip(ind, k.lower, k.upper) + qs[:, ja] = np.array([k.expand(a) for a in ind]) + return qs +def contacts(side, B, derivatives=True): + rows, dist = [], [] + for c in d.contact: + g0, g1 = map(int, c.geom); types = {int(m.geom_contype[g0]), int(m.geom_contype[g1])} + if types not in PAIRS: continue + hg = g0 if m.geom_contype[g0] == 1 else g1 + if not m.geom(hg).name.startswith(side + '_'): continue + dist.append(float(c.dist)) + if derivatives: + jac = np.zeros((3, m.nv)); mujoco.mj_jac(m, d, jac, None, c.pos, int(m.geom_bodyid[hg])); normal = c.frame[:3] * (1 if hg == g1 else -1); rows.append(normal @ jac @ B) + return np.asarray(rows).reshape(-1, 22), np.array(dist) +def put(side, x, base): + mt = meta[side]; d.qpos[:] = base; d.qpos[mt['wa']] = x[:6]; d.qpos[mt['ja']] = mt['k'].expand(x[6:]); mujoco.mj_fwdPosition(m, d) +W = np.r_[[1.] * 3, [.05] * 3, [.01] * 16] # metres, radians: prefer moving fingers, then wrist rotation, then wrist position +STEP = np.r_[[.008] * 3, [.12] * 3, [.15] * 16] # per-frame bound relative to the previous projected frame: 8 mm, 7 deg, 8.6 deg +def solve(side, x, base, prev): + k, B = meta[side]['k'], meta[side]['B'] + for it in range(10): + put(side, x, base); A, ds = contacts(side, B) + if not len(ds) or ds.min() >= CLEAR - 2e-5: break + lo = np.r_[[-.03] * 3, [-.2] * 3, k.lower - x[6:]]; hi = np.r_[[.03] * 3, [.2] * 3, k.upper - x[6:]] + if prev is not None: lo = np.maximum(lo, prev - STEP - x); hi = np.minimum(hi, prev + STEP - x); lo = np.minimum(lo, 0); hi = np.maximum(hi, 0) + H = np.diag((W / np.r_[[.01] * 3, [.05] * 3, [.05] * 16]) ** 2) + fit = minimize(lambda z: .5 * z @ H @ z, np.zeros(22), jac=lambda z: H @ z, bounds=list(zip(lo, hi)), constraints=[dict(type='ineq', fun=lambda z: A @ z + ds - CLEAR - 3e-4, jac=lambda z: A)], method='SLSQP', options={'ftol': 1e-10, 'maxiter': 100}) + x = x + fit.x + put(side, x, base); _, ds = contacts(side, B, False); short = max(0., CLEAR - ds.min()) if len(ds) else 0. + return x, short +def project(q): + qp = q.copy(); moves = {s: [] for s in meta}; worst = 0.; fallback = 0; prev = {s: None for s in meta} + for f in range(N): + for side, mt in meta.items(): + k, wa, ja = mt['k'], mt['wa'], mt['ja']; base = qp[f].copy(); x0 = np.r_[base[wa], base[ja][k.independent_indices]] + x, short = solve(side, x0.copy(), base, prev[side]) + if short > 2e-4 and prev[side] is not None: # temporal bound made it infeasible: release the bound for this frame + x, short = solve(side, x0.copy(), base, None); fallback += 1 + worst = max(worst, short); put(side, x, base); qp[f] = d.qpos.copy(); moves[side].append(float(np.linalg.norm(x[:3] - x0[:3]) * 1000)); prev[side] = x + print('projection: frames with released temporal bound', fallback, flush=True) + return qp, moves, worst +def stats(q, label): + out = {} + for side, mt in meta.items(): + wa, ja, k = mt['wa'], mt['ja'], mt['k']; p = q[:, wa[:3]]; step = np.linalg.norm(np.diff(p, axis=0), axis=1) * 1000; acc = np.linalg.norm(np.diff(p, n=2, axis=0), axis=1) * 1000 + rot = R.from_euler('XYZ', q[:, wa[3:]]); rs = (rot[:-1].inv() * rot[1:]).magnitude() * 180 / np.pi; fj = q[:, ja][:, k.independent_indices]; fs = np.abs(np.diff(fj, axis=0)).max(1) * 180 / np.pi; fa = np.sqrt((np.diff(fj, n=2, axis=0) ** 2).mean(1)) * 180 / np.pi + gaps = []; pen = [] + for f in range(N): + d.qpos[:] = q[f]; mujoco.mj_fwdPosition(m, d); on = contact[f, (0 if side == 'right' else 5):(5 if side == 'right' else 10)].astype(bool) + if on.any(): gaps.extend(np.linalg.norm(d.site_xpos[mt['sites']][on] - cp[f][(0 if side == 'right' else 5):(5 if side == 'right' else 10)][on], axis=1) * 1000) + _, ds = contacts(side, mt['B'], False); pen.append(max(0., -ds.min()) * 1000 if len(ds) else 0.) + out[side] = dict(wrist_step_mm=dict(median=float(np.median(step)), p95=float(np.percentile(step, 95)), max=float(step.max())), wrist_2nd_diff_rms_mm=float(np.sqrt((acc ** 2).mean())), wrist_rot_step_deg=dict(p95=float(np.percentile(rs, 95)), max=float(rs.max())), finger_step_deg=dict(p95=float(np.percentile(fs, 95)), max=float(fs.max())), finger_2nd_diff_rms_deg=float(fa.mean()), contact_gap_mm=dict(median=float(np.median(gaps)), p95=float(np.percentile(gaps, 95))) if gaps else None, max_penetration_mm=float(max(pen)), frames_penetrating_over_0_5mm=int(np.sum(np.array(pen) > .5))) + print(label, json.dumps(out), flush=True); return out +report = {'parameters': dict(sigma_pos=SIG_POS, sigma_rot=SIG_ROT, sg_window=SG, clearance_m=CLEAR, table=TABLE), 'input': stats(q0, 'input')} +q1 = smooth(q0); report['after_smoothing_only'] = stats(q1, 'smoothed') +q2, moves, worst = project(q1); report['after_projection'] = stats(q2, 'projected'); report['projection_move_mm'] = {s: dict(median=float(np.median(v)), max=float(np.max(v))) for s, v in moves.items()}; report['worst_clearance_shortfall_mm'] = worst * 1000 +q3 = smooth(q2); q3[:, :] = q3; qs = q3.copy() +# second, lighter pass: blend half-way toward the re-smoothed solution, then project again +q3 = .5 * (q2 + q3); q4, moves2, worst2 = project(q3); report['after_second_pass'] = stats(q4, 'pass2'); report['projection_move_pass2_mm'] = {s: dict(median=float(np.median(v)), max=float(np.max(v))) for s, v in moves2.items()} +assert np.array_equal(q4[:, 54:], q0[:, 54:]) and np.isfinite(q4).all() +np.savez_compressed(OUT / 'reference_video_rate.npz', qpos=q4, contact=contact, contact_pos=cp, time=ref['time'], camera_world_from_cv=ref['camera_world_from_cv']) +np.savez_compressed(OUT / 'smoothing_comparison.npz', before=q0, after=q4, time=ref['time']) +times = np.load(IN / 'datasets/processed/current/l20/bimanual/boxes/0/trajectory_kinematic_act.npz')['time']; t = ref['time'] +qi = np.stack([np.interp(times, t, q4[:, i]) for i in range(m.nq)], 1); vel = np.gradient(qi, .0025, axis=0); vel[0] = 0; ctrl = qi[:, m.jnt_qposadr[m.actuator_trnid[:, 0]]] +idx = np.minimum(np.searchsorted(t, times), N - 1); ci = np.stack([np.interp(times, t, cp.reshape(N, -1)[:, j]) for j in range(30)], 1).reshape(-1, 10, 3) +np.savez_compressed(TOUT / '0/trajectory_kinematic_act.npz', qpos=qi, qvel=vel, ctrl=ctrl, contact=contact[idx], contact_pos=ci, time=times) +(OUT / 'smoothing_report.json').write_text(json.dumps(report, indent=2)); print('SMOOTH_PROJECT_DONE', flush=True) diff --git a/scripts/smooth_refitted_objects.py b/scripts/smooth_refitted_objects.py new file mode 100644 index 0000000..ec52d79 --- /dev/null +++ b/scripts/smooth_refitted_objects.py @@ -0,0 +1,35 @@ +from pathlib import Path +import json,numpy as np +from scipy.ndimage import gaussian_filter1d +from scipy.spatial.transform import Rotation +ROOT=Path(__file__).resolve().parents[1];B=ROOT/'output/20260915_171525_dynhamr';a=dict(np.load(B/'object_refit_tracked/object_poses.npz'));report={} +for n in ['upper','lower']: + center=np.array([0, .00415 if n=='upper' else -.03335, .2185 if n=='upper' else .21975]) + r=Rotation.from_quat(a[n+'_quaternion_xyzw']);c=r.apply(np.tile(center,(352,1)))+a[n+'_position_world'];cs=gaussian_filter1d(c,2,axis=0) + rs=[] + for t in range(352): + ix=np.arange(max(0,t-6),min(352,t+7));rs.append(r[ix].mean(weights=np.exp(-.5*((ix-t)/2)**2)).as_quat()) + rr=Rotation.from_quat(rs) + # Resolve visible-face ambiguity using the source video: flat red face and circular blue feature. + rr=rr*Rotation.from_matrix(np.diag([-1.,-1.,1.])) + a[n+'_position_world']=cs-rr.apply(np.tile(center,(352,1)));a[n+'_quaternion_xyzw']=rr.as_quat() + angles=np.rad2deg((rr[:-1].inv()*rr[1:]).magnitude());steps=np.linalg.norm(np.diff(cs,axis=0),axis=1)*1000 + report[n]={'max_center_step_mm':float(steps.max()),'max_rotation_step_deg':float(angles.max()),'center_smoothing_max_change_mm':float(np.linalg.norm(cs-c,axis=1).max()*1000)} + assert np.isfinite(cs).all() and angles.max()<20 and steps.max()<30 +# Video inspection: lower part becomes occluded during placement (frames 180-214), +# and the common CAD assembly is visible from frame 215. This is an explicit assumption. +from scipy.spatial.transform import Slerp +hold_p=a['lower_position_world'][176].copy();hold_q=a['lower_quaternion_xyzw'][176].copy() +a['lower_position_world'][177:190]=hold_p;a['lower_quaternion_xyzw'][177:190]=hold_q +for t in range(190,215): + alpha=(t-189)/26 + a['lower_position_world'][t]=(1-alpha)*hold_p+alpha*a['upper_position_world'][t] + a['lower_quaternion_xyzw'][t]=Slerp([0,1],Rotation.from_quat([hold_q,a['upper_quaternion_xyzw'][t]]))([alpha]).as_quat()[0] +a['lower_position_world'][215:]=a['upper_position_world'][215:] +a['lower_quaternion_xyzw'][215:]=a['upper_quaternion_xyzw'][215:] +a['lower_keyframe_valid'][177:]=False +a['lower_assembly_assumed']=np.arange(352)>=215 +a['lower_occluded_transition_assumed']=(np.arange(352)>=177)&(np.arange(352)<215) +report['assembly_assumption']={'from_frame':215,'transition_frames':[177,214],'basis':'Visual inspection and shared CAD assembly coordinates; not independent measured lower poses.'} +np.savez_compressed(B/'object_aligned/object_poses.npz',**a) +(B/'object_aligned/smoothing_validation.json').write_text(json.dumps(report,indent=2));print(report) diff --git a/scripts/smooth_table_constrained.py b/scripts/smooth_table_constrained.py new file mode 100644 index 0000000..366cf77 --- /dev/null +++ b/scripts/smooth_table_constrained.py @@ -0,0 +1,96 @@ +"""Smooth the corrected reference and project hand poses above the table.""" +from pathlib import Path +import json, shutil +import numpy as np +import mujoco +from scipy.signal import savgol_filter + +ROOT = Path(__file__).resolve().parents[1] +SRC = ROOT / "output/spider_dynamics_fix_20260915" +OUT = ROOT / "output/desktop_smooth_fix_20260915" +OUT.mkdir(exist_ok=True) +shutil.copy2(SRC / "reference_video_rate.npz", OUT / "reference_before_smoothing.npz") +shutil.copy2(SRC / "datasets/processed/current/l20/bimanual/boxes/scene_act.xml", OUT / "scene_act.xml") + +z = np.load(SRC / "reference_video_rate.npz") +arrays = {k: z[k].copy() for k in z.files} +q = arrays["qpos"] +original = q.copy() +m = mujoco.MjModel.from_xml_path(str(OUT / "scene_act.xml")) +d = mujoco.MjData(m) +# Filter each hand component independently. The object six-DOF states are kept +# exactly as estimated; only hand states are regularized. +window, poly = (9, 2) +for sl in (slice(0, 27), slice(27, 54)): + q[:, sl] = savgol_filter(q[:, sl], window_length=window, polyorder=poly, axis=0, mode="interp") + +# Recompute the exact five linear-mimic joints from the MuJoCo equality rows. +for e in range(m.neq): + j1, j2 = m.eq_obj1id[e], m.eq_obj2id[e] + a1, a2 = m.jnt_qposadr[j1], m.jnt_qposadr[j2] + p = m.eq_data[e, :5] + q[:, a1] = sum(p[k] * q[:, a2] ** k for k in range(5)) + +floor_ids = [i for i in range(m.ngeom) if "_floor" in (m.geom(i).name or "")] +for i in floor_ids: + m.geom_conaffinity[i] = 3 +hand_geom = { + "right": [i for i in range(m.ngeom) if m.geom_contype[i] == 1 and (m.geom(i).name or "").startswith("right_")], + "left": [i for i in range(m.ngeom) if m.geom_contype[i] == 1 and (m.geom(i).name or "").startswith("left_")], +} +hand_vertices = {} +for side in ("right", "left"): + hand_vertices[side] = [] + for i in hand_geom[side]: + mid = int(m.geom_dataid[i]) + if mid < 0: + continue + # Collision meshes are convex approximations; use the original visual + # mesh vertices to detect the visible hand crossing the table. + start = int(m.mesh_vertadr[mid]); end = start + int(m.mesh_vertnum[mid]) + vertices = m.mesh_vert[start:end].copy() + hand_vertices[side].append((i, vertices[::max(1, len(vertices)//128)])) + +def min_floor_dist(side): + floor_z = float(d.geom_xpos[floor_ids[0]][2]) + vals = [] + for i, vertices in hand_vertices[side]: + world = vertices @ d.geom_xmat[i].reshape(3, 3).T + d.geom_xpos[i] + vals.append(float(world[:, 2].min() - floor_z)) + return min(vals) if vals else 1.0 + +projected = [] +for f in range(len(q)): + d.qpos[:] = q[f] + mujoco.mj_forward(m, d) + for side, sl in (("right", slice(0, 27)), ("left", slice(27, 54))): + # Binary search the smallest wrist-z lift giving 0.5 mm clearance. + base = q[f].copy() + d.qpos[:] = base; mujoco.mj_forward(m, d) + if min_floor_dist(side) < 0.0005: + lo, hi = 0.0, 0.15 + for _ in range(24): + mid = (lo + hi) / 2 + trial = base.copy(); trial[sl.start + 2] += mid + d.qpos[:] = trial; mujoco.mj_forward(m, d) + if min_floor_dist(side) >= 0.0005: hi = mid + else: lo = mid + q[f, sl.start + 2] += hi + projected.append((f, side, hi)) + +arrays["qpos"] = q +dt = float(np.median(np.diff(arrays["time"]))) +arrays["qvel"] = np.gradient(q, dt, axis=0); arrays["qvel"][0] = 0 +arrays["ctrl"] = q[:, m.jnt_qposadr[m.actuator_trnid[:, 0]]] +np.savez_compressed(OUT / "reference_video_rate.npz", **arrays) +report = { + "frames": len(q), "filter": {"type": "Savitzky-Golay", "window": window, "polyorder": poly}, + "projected_hand_table_contacts": len(projected), + "max_table_lift_mm": float(max((x[2] for x in projected), default=0) * 1000), + "max_hand_change_mm": float(max(np.linalg.norm(q[:, :3]-original[:, :3], axis=1).max(), np.linalg.norm(q[:, 27:30]-original[:, 27:30], axis=1).max()) * 1000), + "object_qpos_unchanged": bool(np.array_equal(q[:, -12:], original[:, -12:])), + "all_finite": bool(np.isfinite(q).all()), + "note": "Table projection uses the simulation floor plane and hand collision geoms; object pose remains fixed." +} +(OUT / "smoothing_table_report.json").write_text(json.dumps(report, indent=2)) +print(json.dumps(report, indent=2)) diff --git a/scripts/stabilize_l20_temporal.py b/scripts/stabilize_l20_temporal.py index d5c6aca..0500d4a 100644 --- a/scripts/stabilize_l20_temporal.py +++ b/scripts/stabilize_l20_temporal.py @@ -18,6 +18,7 @@ def main(): p = argparse.ArgumentParser() p.add_argument('--source', type=Path, default=ROOT/'output/l20_15886123') p.add_argument('--output', type=Path, default=ROOT/'output/l20_15886123_stable') + p.add_argument('--side', choices=['right','left'], default='right') a = p.parse_args() a.output.mkdir(parents=True, exist_ok=True) with np.load(a.source/'motion.npz') as f: @@ -28,7 +29,7 @@ def main(): eye = sparse.eye(n,format='csc') names, active = list(m['joint_names']), list(m['active_joint_names']) ai = [names.index(k) for k in active] - original = ET.parse(ROOT/'third_party/l20_assets/L20/RIGHT/linkerhand_g20_right.urdf').getroot() + original = ET.parse(ROOT/f'third_party/l20_assets/L20/{a.side.upper()}/linkerhand_g20_{a.side}.urdf').getroot() limits = {j.get('name'):np.array([float(j.find('limit').get(k)) for k in ['lower','upper']]) for j in original.findall('joint') if j.get('type')!='fixed'} low, high = np.array([limits[k] for k in active]).T.copy() mimic = {} @@ -99,9 +100,9 @@ def main(): m['before_stabilization_'+k] = v m['stabilization_method'] = 'whole-sequence data fidelity + second/third difference regularization; component distortion budgets' m['stabilization_source'] = str(a.source.resolve()) - for filename in ['l20_moving.xml','l20_right.xml','l20_dex.urdf','dex_config.json']: + for filename in ['l20_moving.xml',f'l20_{a.side}.xml','l20_dex.urdf','dex_config.json']: shutil.copy2(a.source/filename,a.output/filename) - model = mujoco.MjModel.from_xml_path(str(a.output/'l20_right.xml')) + model = mujoco.MjModel.from_xml_path(str(a.output/f'l20_{a.side}.xml')) data = mujoco.MjData(model) addr = [model.jnt_qposadr[model.joint(k).id] for k in names] sites = [model.site(f'landmark_{i:02d}').id for i in range(21)] diff --git a/scripts/track_red_box_foundationpose.py b/scripts/track_red_box_foundationpose.py new file mode 100644 index 0000000..1d8b2d5 --- /dev/null +++ b/scripts/track_red_box_foundationpose.py @@ -0,0 +1,76 @@ +"""Offline RGB-D red-box tracking, with explicit partial-view and quality flags.""" +import os +os.environ.setdefault('OMP_NUM_THREADS','4') +import sys,json,time,argparse,logging +from pathlib import Path +import cv2,numpy as np +ROOT=Path(__file__).resolve().parents[1] +sys.path.insert(0,str(ROOT/'third_party/FoundationPose')) +from estimater import FoundationPose,ScorePredictor,PoseRefinePredictor +from Utils import nvdiffrast_render,make_mesh_tensors +import torch,trimesh,nvdiffrast.torch as dr +from scipy.spatial.transform import Rotation + +def redmask(b,z): + h=cv2.cvtColor(b,cv2.COLOR_BGR2HSV) + m=(((h[:,:,0]<10)|(h[:,:,0]>170))&(h[:,:,1]>115)&(h[:,:,2]>65)&(z>.1)&(z<.85)).astype('uint8') + m[:150]=0 + n,l,st,c=cv2.connectedComponentsWithStats(m,8) + if n<2:return m*0 + j=1+np.argmax(st[1:,4]);return (l==j).astype('uint8') + +def main(): + a=argparse.ArgumentParser();a.add_argument('--end',type=int,default=893);a.add_argument('--init',type=int,default=30);a.add_argument('--output',default='output/foundationpose_red_20260916');args=a.parse_args() + out=ROOT/args.output;out.mkdir(parents=True,exist_ok=True);(out/'snapshots').mkdir(exist_ok=True) + logging.basicConfig(level=logging.WARNING) + src=ROOT/'docs/20260916_104026';meta=json.loads((src/'intrinsics.json').read_text());K=np.array([[meta['fx'],0,meta['cx']],[0,meta['fy'],meta['cy']],[0,0,1.]]) + cap=cv2.VideoCapture(str(src/'color.mp4'));frames=[] + while True: + ok,b=cap.read() + if not ok:break + frames.append(b) + cap.release();N=min(len(frames),args.end);assert len(frames)==meta['frames'] + mesh=trimesh.load(ROOT/'docs/上半.stl');mesh.visual.vertex_colors=np.tile([200,35,35,255],(len(mesh.vertices),1)) + ctx=dr.RasterizeCudaContext();est=FoundationPose(model_pts=mesh.vertices,model_normals=mesh.vertex_normals,mesh=mesh,glctx=ctx,debug=0,debug_dir=str(out/'debug')) + original_tensors=make_mesh_tensors(mesh);H,W=frames[0].shape[:2] + poses=np.full((N,4,4),np.nan);records=[None]*N;init_state=None + sequence=list(range(args.init,N))+list(range(args.init-1,-1,-1));start=time.time() + for i in sequence: + b=frames[i];z=cv2.imread(str(src/'depth'/f'{i:06d}.png'),-1).astype(np.float32)*meta['depth_scale_m'];z[(z<.1)|(z>.85)]=0;m=redmask(b,z);yy,xx=np.nonzero(m) + row={'frame':i,'red_pixels':len(xx),'status':'not_observable','accepted':False};pic=b.copy() + # After the red box is put aside it is severely cropped; do not invent its full pose. + observable=i<=335 # Manually reviewed red manipulation interval for this recording. + if i==args.init-1:est.pose_last=init_state.clone() + if observable: + rgb=cv2.cvtColor(b,cv2.COLOR_BGR2RGB) + if i==args.init: + cv2.imwrite(str(out/'initial_mask.png'),m*255);cv2.imwrite(str(out/'initial_frame.png'),b) + pose=est.register(K=K,rgb=rgb,depth=z,ob_mask=m,iteration=5);init_state=est.pose_last.clone() + else:pose=est.track_one(rgb=rgb,depth=z,K=K,iteration=2) + poses[i]=pose + with torch.inference_mode(): + _,rd,_=nvdiffrast_render(K=K,H=H,W=W,ob_in_cams=torch.as_tensor(pose[None],device='cuda'),glctx=ctx,mesh_tensors=original_tensors) + rd=rd[0].cpu().numpy();render=rd>0;overlap=render&(m>0)&(z>0) + coverage=float((render&(m>0)).sum()/max(1,m.sum()));err=float(np.median(np.abs(rd[overlap]-z[overlap]))) if overlap.any() else None + partial_left=bool(i>=315 or (m[:,:3]>0).any()) + accepted=coverage>.65 and err is not None and err<.02 and not partial_left + row.update(status='partial_view' if partial_left else ('tracked' if accepted else 'low_confidence'),accepted=accepted,red_coverage=coverage,depth_median_error_m=err) + # Show all predicted surfaces, including those behind the hand, for honest overlay QA. + pic[render]=(pic[render]*.65+np.array([0,220,0])*.35).astype('uint8') + contours,_=cv2.findContours(render.astype('uint8'),cv2.RETR_EXTERNAL,cv2.CHAIN_APPROX_SIMPLE);cv2.drawContours(pic,contours,-1,(0,255,0),1) + records[i]=row + cv2.putText(pic,f'{i:04d} {row["status"]}',(15,30),0,.7,(0,255,255),2) + if i%30==0 or i in [args.init,89,119,149,179,209,239,269,299,329]:cv2.imwrite(str(out/'snapshots'/f'{i:06d}.jpg'),pic) + if i%30==0:print(json.dumps({**row,'elapsed_s':round(time.time()-start,1)}),flush=True);np.save(out/'poses_checkpoint.npy',poses) + np.savez_compressed(out/'poses.npz',T_camera_from_object=poses,time_s=(np.array(meta['timestamps_ms'][:N])-meta['timestamps_ms'][0])/1000,accepted=np.array([r['accepted'] for r in records]),frame_index=np.arange(N),K=K) + (out/'frame_metrics.json').write_text(json.dumps(records,indent=2)) + # Export original STL origin, and its geometric bounding-box center, separately. + rows=[] + for i,T in enumerate(poses): + q=Rotation.from_matrix(T[:3,:3]).as_quat() if np.isfinite(T).all() else np.full(4,np.nan) + center=T[:3,:3]@mesh.bounds.mean(0)+T[:3,3] + rows.append([i,(meta['timestamps_ms'][i]-meta['timestamps_ms'][0])/1000,*T[:3,3],*q,*center,int(records[i]['accepted'])]) + np.savetxt(out/'poses.csv',rows,delimiter=',',header='frame,time_s,origin_x_m,origin_y_m,origin_z_m,qx,qy,qz,qw,center_x_m,center_y_m,center_z_m,accepted',comments='') + summary={'frames':N,'estimated':int(np.isfinite(poses).all((1,2)).sum()),'accepted_by_heuristics':sum(r['accepted'] for r in records),'init_frame':args.init,'offline_backward_frames':args.init,'coordinate':'original STL object coordinates to fixed OpenCV camera: x right, y down, z forward; meters assumed and checked by depth residual','manual_reviewed_tracking_interval_inclusive':[0,335],'mesh':'docs/上半.stl','mesh_extents_m':mesh.extents.tolist(),'boundary':'Predicted poses, not ground truth. Color/depth quality gates are heuristics; no contact or dynamics validation. Unobservable frames are NaN, not held.','elapsed_s':time.time()-start} + (out/'summary.json').write_text(json.dumps(summary,indent=2));print(json.dumps(summary),flush=True) +if __name__=='__main__':main() diff --git a/scripts/track_yesterday_boxes.py b/scripts/track_yesterday_boxes.py new file mode 100644 index 0000000..5d024a9 --- /dev/null +++ b/scripts/track_yesterday_boxes.py @@ -0,0 +1,52 @@ +"""FoundationPose RGB-D tracking for both CAD parts, preserving camera/world transforms.""" +import os +os.environ.setdefault('OMP_NUM_THREADS','4') +import sys,json,time,subprocess +from pathlib import Path +ROOT=Path(__file__).resolve().parents[1];sys.path.insert(0,str(ROOT/'third_party/FoundationPose')) +import cv2,numpy as np,torch,trimesh,nvdiffrast.torch as dr +from estimater import FoundationPose,ScorePredictor,PoseRefinePredictor +from Utils import nvdiffrast_render,make_mesh_tensors +OUT=ROOT/'output/foundationpose_spider_20260915';SRC=ROOT/'docs/20260915_171525';m=json.loads((SRC/'intrinsics.json').read_text());N=m['frames'];K=np.array([[m['fx'],0,m['cx']],[0,m['fy'],m['cy']],[0,0,1.]]) +cam=np.load(ROOT/'output/20260915_171525_dynhamr/rgbd_camera.npz');c=cv2.VideoCapture(str(SRC/'color.mp4'));frames=[] +while True: + ok,b=c.read() + if not ok:break + frames.append(b) +assert len(frames)==N +ctx=dr.RasterizeCudaContext();score=ScorePredictor();refine=PoseRefinePredictor();arrays={'K':K,'time':np.array(m['timestamps_ms'])/1000-m['timestamps_ms'][0]/1000,'c2w':cam['c2w'],'camera_valid':cam['valid']};meshes={};allmetrics={} +for name,file,init,color in [('upper','上半.stl',0,[210,35,35,255]),('lower','下半.stl',80,[35,65,210,255])]: + mesh=trimesh.load(ROOT/'docs'/file);mesh.visual.vertex_colors=np.tile(color,(len(mesh.vertices),1));meshes[name]=mesh + est=FoundationPose(model_pts=mesh.vertices,model_normals=mesh.vertex_normals,mesh=mesh,scorer=score,refiner=refine,glctx=ctx,debug=0,debug_dir=str(OUT/name/'debug'));mt=make_mesh_tensors(mesh);P=np.full((N,4,4),np.nan);valid=np.zeros(N,bool);metrics=[None]*N;start_state=None + for i in list(range(init,N))+list(range(init-1,-1,-1)): + b=frames[i];z=cv2.imread(str(SRC/'depth'/f'{i:06d}.png'),-1).astype('float32')*m['depth_scale_m'];z[(z<.1)|(z>1.)]=0 + h=cv2.cvtColor(b,cv2.COLOR_BGR2HSV);hue=h[:,:,0];mask=((((hue<12)|(hue>170)) if name=='upper' else ((hue>95)&(hue<135)))&(h[:,:,1]>90)&(h[:,:,2]>40)&(z>.1)&(z<1.)).astype('uint8');mask[:200]=0 + num,lab,st,_=cv2.connectedComponentsWithStats(mask,8) + if num>1:mask=(lab==1+np.argmax(st[1:,4])).astype('uint8') + if i==init-1:est.pose_last=start_state.clone() + rgb=cv2.cvtColor(b,cv2.COLOR_BGR2RGB) + if i==init: + cv2.imwrite(str(OUT/name/'initial_mask.png'),mask*255);p=est.register(K=K,rgb=rgb,depth=z,ob_mask=mask,iteration=5);start_state=est.pose_last.clone() + else:p=est.track_one(rgb=rgb,depth=z,K=K,iteration=2) + P[i]=p + with torch.inference_mode():_,depth,_=nvdiffrast_render(K=K,H=480,W=848,ob_in_cams=torch.as_tensor(p[None],device='cuda'),glctx=ctx,mesh_tensors=mt) + depth=depth[0].cpu().numpy();over=(depth>0)&(mask>0)&(z>0);coverage=float(over.sum()/max(1,mask.sum()));err=float(np.median(np.abs(depth[over]-z[over]))) if over.any() else 1. + valid[i]=mask.sum()>500 and coverage>.65 and err<.02 + # The lower part becomes occluded during assembly; independent lower tracking is not reliable there. + if name=='lower' and i>=177:valid[i]=False + metrics[i]={'frame':i,'valid':bool(valid[i]),'visible_color_pixels':int(mask.sum()),'coverage':coverage,'depth_residual_m':err} + if i%60==0:print(name,metrics[i],flush=True) + arrays[name+'_T_camera']=P;arrays[name+'_T_world']=cam['c2w']@P;arrays[name+'_valid']=valid;allmetrics[name]=metrics +np.savez_compressed(OUT/'foundationpose_objects.npz',**arrays);(OUT/'object_metrics.json').write_text(json.dumps(allmetrics,indent=2)) +writer=subprocess.Popen(['ffmpeg','-y','-v','error','-f','rawvideo','-pix_fmt','bgr24','-s','848x480','-r','30','-i','-','-an','-c:v','libx264','-crf','20','-pix_fmt','yuv420p',str(OUT/'foundationpose_overlay.mp4')],stdin=subprocess.PIPE) +for i,b in enumerate(frames): + pic=b.copy() + for name,col in [('upper',[0,220,0]),('lower',[220,180,0])]: + with torch.inference_mode():_,d,_=nvdiffrast_render(K=K,H=480,W=848,ob_in_cams=torch.tensor(arrays[name+'_T_camera'][i:i+1],device='cuda',dtype=torch.float32),glctx=ctx,mesh_tensors=make_mesh_tensors(meshes[name])) + mask=d[0].cpu().numpy()>0 + if arrays[name+'_valid'][i]:pic[mask]=(pic[mask]*.65+np.array(col)*.35).astype('uint8') + cv2.putText(pic,f'{i:03d} red:{arrays["upper_valid"][i]} blue:{arrays["lower_valid"][i]}',(8,25),0,.55,(0,255,255),2) + writer.stdin.write(pic.tobytes()) + if i in [0,40,80,120,160,200,240,280,351]:cv2.imwrite(str(OUT/f'objects_{i:04d}.jpg'),pic) +writer.stdin.close();assert writer.wait()==0 +print('FOUNDATIONPOSE_DONE', {n:int(arrays[n+'_valid'].sum()) for n in meshes},flush=True) diff --git a/scripts/unmirror_left_rgbd.py b/scripts/unmirror_left_rgbd.py new file mode 100644 index 0000000..e73dfa5 --- /dev/null +++ b/scripts/unmirror_left_rgbd.py @@ -0,0 +1,20 @@ +"""Restore mirrored-right inference to left-hand geometry in the original world.""" +from pathlib import Path +import numpy as np +from scipy.spatial.transform import Rotation +ROOT=Path(__file__).resolve().parents[1];BASE=ROOT/'output/20260915_171525' +src=BASE/'handflow_left_mirrored';out=BASE/'handflow_left';out.mkdir(exist_ok=True) +M=np.diag([-1.,1.,1.]);M4=np.diag([-1.,1.,1.,1.]) +h=dict(np.load(src/'handflow_results.npz'));j=dict(np.load(src/'human_joints.npz')) +for k in ['verts_cam','verts_world','trans']:h[k]=h[k]@M +h['c2w']=M4@h['c2w']@M4 +h['pose']=h['pose'].copy();h['pose'][:,:3]=Rotation.from_matrix(M@Rotation.from_rotvec(h['pose'][:,:3]).as_matrix()@M).as_rotvec() +h['intrinsics']=h['intrinsics'].copy();h['intrinsics'][2]=848-1-h['intrinsics'][2] +h['side']='left';h['source']=str(ROOT/'docs/20260915_171525/color.mp4') +h['convention_note']='Canonical right prediction from mirrored video, geometry reflected into actual left camera/world; body-pose fields remain canonical and are not native MANO-left parameters.' +for k in ['joints','joints_raw','joints_world_raw','wrist_world','wrist_world_raw']: + if k in j:j[k]=j[k]@M +j['root_orient']=Rotation.from_matrix(M@Rotation.from_rotvec(j['root_orient']).as_matrix()@M).as_rotvec() +j['source']=str(out/'handflow_results.npz');j['side']='left' +np.savez_compressed(out/'handflow_results.npz',**h);np.savez_compressed(out/'human_joints.npz',**j) +print('Unmirrored left geometry',len(j['joints'])) diff --git a/scripts/verify_fix_interpolation.py b/scripts/verify_fix_interpolation.py new file mode 100644 index 0000000..4b6561f --- /dev/null +++ b/scripts/verify_fix_interpolation.py @@ -0,0 +1,45 @@ +"""Check and project all 400 Hz interpolated hand/object states, keeping objects fixed.""" +import os +os.environ['OPENBLAS_NUM_THREADS']='1';os.environ['OMP_NUM_THREADS']='1' +from pathlib import Path +import json,sys,numpy as np,mujoco +from scipy.optimize import minimize +import l20_model_source as source +R=Path(__file__).resolve().parents[1];O=Path(os.environ.get('HF_FIX_OUT',R/'output/collision_fix_20260915'));T=O/'datasets/processed/current/l20/bimanual/boxes';OLD=Path(os.environ.get('HF_FIX_OLD',R/'output/foundationpose_spider_20260915'));source.REPO_ROOT=R/'third_party/l20_assets';m=mujoco.MjModel.from_xml_path(str(T/'scene_act.xml'));d=mujoco.MjData(m);video_mode='--video' in sys.argv;clearance=.0025 if '--buffer' in sys.argv else 0.;r=np.load(O/'reference_video_rate.npz' if video_mode else T/'0/trajectory_kinematic_act.npz');arrays={k:r[k] for k in r.files};q=arrays['qpos'].copy();original=q.copy();meta={} +for side in ['right','left']: + k=source.HandKinematics(OLD/f'model_{side}/l20_{side}.xml',side);wa=np.array([m.jnt_qposadr[m.joint(side+'_hand_'+s).id] for s in ['pos_x','pos_y','pos_z','rot_x','rot_y','rot_z']]);ja=np.array([m.jnt_qposadr[m.joint(side+'_'+n).id] for n in k.joint_names]);B=np.zeros((m.nv,22));B[wa,:6]=np.eye(6);B[ja,6:]=k.expansion;meta[side]=(k,wa,ja,B) +def contacts(side=None,B=None): + ds=[];rows=[] + for c in d.contact: + g0,g1=map(int,c.geom) + if {int(m.geom_contype[g0]),int(m.geom_contype[g1])} not in ([{1,2},{1,4}] if os.environ.get('HF_TABLE','0')=='1' else [{1,2}]):continue + hg=g0 if m.geom_contype[g0]==1 else g1 + if side is not None and not m.geom(hg).name.startswith(side+'_'):continue + ds.append(float(c.dist)) + if B is not None: + J=np.zeros((3,m.nv));mujoco.mj_jac(m,d,J,None,c.pos,int(m.geom_bodyid[hg]));rows.append((c.frame[:3]*(1 if hg==g1 else -1))@J@B) + return np.asarray(rows).reshape(-1,22),np.asarray(ds) +pre=[];post=[];changed=[] +for f in range(len(q)): + d.qpos[:]=q[f];mujoco.mj_fwdPosition(m,d);_,ds=contacts();pen=max(0,clearance-ds.min()) if len(ds) else 0;pre.append(pen) + if pen>.00005: + for side,(k,wa,ja,B) in meta.items(): + for it in range(15): + A,ds=contacts(side,B) + if not len(ds) or ds.min()>clearance-.00002:break + x=np.r_[d.qpos[wa],d.qpos[ja][k.independent_indices]];scale=np.r_[[.01]*3,[.1]*3,[.1]*16];lo=np.maximum(-scale,np.r_[[-np.inf]*6,k.lower]-x);hi=np.minimum(scale,np.r_[[np.inf]*6,k.upper]-x);H=np.diag(1/scale**2) + fit=minimize(lambda z:.5*z@H@z,np.zeros(22),jac=lambda z:H@z,bounds=list(zip(lo,hi)),constraints=[dict(type='ineq',fun=lambda z:A@z+ds-clearance-.0003,jac=lambda z:A)],method='SLSQP',options={'ftol':1e-9,'maxiter':80}) + old=d.qpos.copy();best=None + for alpha in [1.,.5,.25]: + y=x+alpha*fit.x;d.qpos[:]=old;d.qpos[wa]=y[:6];d.qpos[ja]=k.expand(y[6:]);mujoco.mj_fwdPosition(m,d);_,test=contacts(side);p=max(0,clearance-test.min()) if len(test) else 0 + if best is None or p0).sum()>0 +score=ScorePredictor();refine=PoseRefinePredictor() +outputs={} +with torch.inference_mode(): + for name,predictor in [('score',score),('refine',refine)]: + h,w=predictor.cfg.input_resize + a=torch.zeros((1,predictor.cfg.c_in,h,w),device='cuda') + out=predictor.model(a,a,L=1) if name=='score' else predictor.model(a,a) + assert all(torch.isfinite(v).all() for v in out.values()) + outputs[name]={k:list(v.shape) for k,v in out.items()} +mesh=trimesh.load(ROOT/'docs/上半.stl') +est=FoundationPose(model_pts=mesh.vertices,model_normals=mesh.vertex_normals,mesh=mesh,scorer=score,refiner=refine,glctx=ctx,debug=0,debug_dir=str(ROOT/'output/foundationpose_setup/smoke_debug')) +torch.cuda.synchronize() +report={'status':'passed','gpu':torch.cuda.get_device_name(),'torch':torch.__version__,'cuda':torch.version.cuda,'mycpp':Utils.mycpp.__file__,'rasterized_pixels':int((rast[...,3]>0).sum()),'network_outputs':outputs,'red_box_rotation_hypotheses':list(est.rot_grid.shape),'scope':'GPU rendering, network forward passes and STL estimator initialization; no recorded-video pose accuracy validation','optional_bundlesdf_mycuda':'not built; not required for model-based FoundationPose'} +(ROOT/'output/foundationpose_setup/runtime_verification.json').write_text(json.dumps(report,indent=2)) +print(json.dumps(report,indent=2)) diff --git a/scripts/verify_rgbd_bimanual.py b/scripts/verify_rgbd_bimanual.py new file mode 100644 index 0000000..76f4686 --- /dev/null +++ b/scripts/verify_rgbd_bimanual.py @@ -0,0 +1,69 @@ +"""Independently check both URDF constraints and export common-world trajectories.""" +from pathlib import Path +import json,xml.etree.ElementTree as E,argparse +import cv2,mujoco,numpy as np +from scipy.spatial.transform import Rotation +ROOT=Path(__file__).resolve().parents[1] +parser=argparse.ArgumentParser();parser.add_argument('--base',type=Path,default=ROOT/'output/20260915_171525');args=parser.parse_args() +BASE=args.base.resolve();OUT=BASE/'replay' +motion=np.load(OUT/'motion.npz');q=motion['qpos'];model=mujoco.MjModel.from_xml_path(str(OUT/'scene.xml'));data=mujoco.MjData(model) +bundle=dict(time=motion['time'],fps=motion['fps'],camera_world_from_cv=motion['camera_world_from_cv']) +reports={};samples={side:[] for side in ['right','left']} +for side in ['right','left']: + prefix='' if side=='right' else 'left_';hand=np.load(BASE/f'l20_{side}_stable/motion.npz') + names=hand['joint_names'].tolist();addrs=[model.jnt_qposadr[model.joint(prefix+n).id] for n in names] + angles=q[:,addrs];assert np.allclose(angles,hand['qpos'],atol=1e-12) + root=E.parse(ROOT/f'third_party/l20_assets/L20/{side.upper()}/linkerhand_g20_{side}.urdf').getroot() + violation=0.;mimic=0.;fk=0.;positions=[];quats=[] + Rshared=np.load(BASE/'l20_right_stable/motion.npz')['scene_rotation'];tshared=np.load(BASE/'l20_right_stable/motion.npz')['scene_translation'] + for j in root.findall('joint'): + if j.get('type')=='fixed':continue + col=angles[:,names.index(j.get('name'))];lim=j.find('limit') + violation=max(violation,float(np.maximum(float(lim.get('lower'))-col,0).max()),float(np.maximum(col-float(lim.get('upper')),0).max())) + mi=j.find('mimic') + if mi is not None: + mimic=max(mimic,float(np.abs(col-float(mi.get('multiplier','1'))*angles[:,names.index(mi.get('joint'))]-float(mi.get('offset','0'))).max())) + for i,row in enumerate(q): + data.qpos[:]=row;mujoco.mj_forward(model,data) + pos=data.xpos[model.body(prefix+'hand_base_link').id].copy();quat=data.xquat[model.body(prefix+'hand_base_link').id].copy() + positions.append(pos);quats.append(quat) + localR=Rotation.from_quat(hand['wrist_quat_wxyz'][i,[1,2,3,0]]).as_matrix() + expectedR=Rshared@hand['scene_rotation'].T@localR + expectedp=Rshared@hand['scene_rotation'].T@(hand['wrist_pos'][i]-hand['scene_translation'])+tshared + expected=hand['actual'][i]@expectedR.T+expectedp + actual=np.array([data.site_xpos[model.site(prefix+f'landmark_{k:02d}').id] for k in range(21)]) + fk=max(fk,float(np.abs(actual-expected).max())) + positions=np.asarray(positions);quats=np.asarray(quats) + for i in range(1,len(quats)): + if quats[i]@quats[i-1]<0:quats[i]*=-1 + bundle.update({side+'_joint_names':np.asarray(names),side+'_joint_position':angles, + side+'_wrist_position':positions,side+'_wrist_quaternion_wxyz':quats, + side+'_valid':hand['detection_valid']}) + np.savetxt(OUT/f'{side}_hand_trajectory.csv',np.c_[motion['time'],positions,quats,angles],delimiter=',', + header=','.join(['time_s','wrist_x','wrist_y','wrist_z','qw','qx','qy','qz']+names),comments='') + assert violation<1e-8 and mimic<1e-10 and fk<1e-5 + reports[side]=dict(frames=len(q),valid_frames=int(hand['detection_valid'].sum()),urdf_limit_violation_rad=violation,mimic_error_rad=mimic,independent_world_fk_max_error_m=fk) +for name in ['upper','lower']: + addr=model.jnt_qposadr[model.joint(name+'_free').id] + bundle[name+'_position']=q[:,addr:addr+3];bundle[name+'_quaternion_wxyz']=q[:,addr+3:addr+7] + fit=np.load(BASE/'object_poses.npz') + observed=np.zeros(len(q),dtype=bool) + observed[fit[name+'_keyframes'][fit[name+'_keyframe_valid']]]=True + bundle[name+'_accepted_fit_keyframe']=observed + bundle[name+'_interpolated_or_held']=~observed + for suffix in ['assembly_assumed','occluded_transition_assumed']: + key=name+'_'+suffix + if key in fit:bundle[key]=fit[key] +bundle['world_convention']='Common right-hand replay scene transform applied to first-camera RGBD odometry world; display axes, gravity not measured.' +bundle['valid_semantics']='Per-hand source validity; see human_joints source metadata. Object poses are partial-surface estimates; not contact labels.' +assert np.isfinite(q).all() and np.all(np.diff(motion['time'])>0) +np.savez_compressed(OUT/'demonstrations_bimanual.npz',**bundle) +cap=cv2.VideoCapture(str(OUT/'original_vs_l20_bimanual.mp4'));n=0 +while True: + ok,im=cap.read() + if not ok:break + assert im.shape==(360,1280,3);n+=1 +cap.release();assert n==len(q) +reports.update(video_decoded_frames=n,time_last_s=float(motion['time'][-1]),finite=True, + scope='Dual-hand kinematic retargeting with separate RGBD CAD pose estimates; no contact or dynamics success validation.') +(OUT/'validation.json').write_text(json.dumps(reports,indent=2));print(json.dumps(reports,indent=2)) diff --git a/third_party/Dyn-HaMR/_DATA/BMC b/third_party/Dyn-HaMR/_DATA/BMC new file mode 120000 index 0000000..6905a90 --- /dev/null +++ b/third_party/Dyn-HaMR/_DATA/BMC @@ -0,0 +1 @@ +../dyn-hamr/optim/BMC \ No newline at end of file diff --git a/third_party/Dyn-HaMR/_DATA/data/mano b/third_party/Dyn-HaMR/_DATA/data/mano new file mode 120000 index 0000000..c4710a9 --- /dev/null +++ b/third_party/Dyn-HaMR/_DATA/data/mano @@ -0,0 +1 @@ +../../../hamer/_DATA/data/mano \ No newline at end of file diff --git a/third_party/Dyn-HaMR/_DATA/data/mano_mean_params.npz b/third_party/Dyn-HaMR/_DATA/data/mano_mean_params.npz new file mode 120000 index 0000000..5b33bae --- /dev/null +++ b/third_party/Dyn-HaMR/_DATA/data/mano_mean_params.npz @@ -0,0 +1 @@ +../../../hamer/_DATA/data/mano_mean_params.npz \ No newline at end of file diff --git a/third_party/Dyn-HaMR/_DATA/hamer_ckpts/checkpoints b/third_party/Dyn-HaMR/_DATA/hamer_ckpts/checkpoints new file mode 120000 index 0000000..c697efc --- /dev/null +++ b/third_party/Dyn-HaMR/_DATA/hamer_ckpts/checkpoints @@ -0,0 +1 @@ +../../../hamer/_DATA/hamer_ckpts/checkpoints \ No newline at end of file diff --git a/third_party/Dyn-HaMR/_DATA/hamer_ckpts/dataset_config.yaml b/third_party/Dyn-HaMR/_DATA/hamer_ckpts/dataset_config.yaml new file mode 100644 index 0000000..77b6725 --- /dev/null +++ b/third_party/Dyn-HaMR/_DATA/hamer_ckpts/dataset_config.yaml @@ -0,0 +1,42 @@ +COCOW-TRAIN: + TYPE: ImageDataset + URLS: hamer_training_data/dataset_tars/cocow-train/{000000..000036}.tar + epoch_size: 78666 +DEX-TRAIN: + TYPE: ImageDataset + URLS: hamer_training_data/dataset_tars/dex-train/{000000..000406}.tar + epoch_size: 406888 +FREIHAND-MOCAP: + DATASET_FILE: hamer_training_data/freihand_mocap.npz +FREIHAND-TRAIN: + TYPE: ImageDataset + URLS: hamer_training_data/dataset_tars/freihand-train/{000000..000130}.tar + epoch_size: 130240 +H2O3D-TRAIN: + TYPE: ImageDataset + URLS: hamer_training_data/dataset_tars/h2o3d-train/{000000..000060}.tar + epoch_size: 121996 +HALPE-TRAIN: + TYPE: ImageDataset + URLS: hamer_training_data/dataset_tars/halpe-train/{000000..000022}.tar + epoch_size: 34289 +HO3D-TRAIN: + TYPE: ImageDataset + URLS: hamer_training_data/dataset_tars/ho3d-train/{000000..000083}.tar + epoch_size: 83325 +INTERHAND26M-TRAIN: + TYPE: ImageDataset + URLS: hamer_training_data/dataset_tars/interhand26m-train/{000000..001056}.tar + epoch_size: 1424632 +MPIINZSL-TRAIN: + TYPE: ImageDataset + URLS: hamer_training_data/dataset_tars/mpiinzsl-train/{000000..000015}.tar + epoch_size: 15184 +MTC-TRAIN: + TYPE: ImageDataset + URLS: hamer_training_data/dataset_tars/mtc-train/{000000..000306}.tar + epoch_size: 363947 +RHD-TRAIN: + TYPE: ImageDataset + URLS: hamer_training_data/dataset_tars/rhd-train/{000000..000041}.tar + epoch_size: 61705 diff --git a/third_party/Dyn-HaMR/_DATA/hamer_ckpts/model_config.yaml b/third_party/Dyn-HaMR/_DATA/hamer_ckpts/model_config.yaml new file mode 100644 index 0000000..035c068 --- /dev/null +++ b/third_party/Dyn-HaMR/_DATA/hamer_ckpts/model_config.yaml @@ -0,0 +1,111 @@ +task_name: train +tags: +- dev +train: true +test: false +ckpt_path: null +seed: null +DATASETS: + TRAIN: + FREIHAND-TRAIN: + WEIGHT: 0.25 + INTERHAND26M-TRAIN: + WEIGHT: 0.25 + MTC-TRAIN: + WEIGHT: 0.1 + RHD-TRAIN: + WEIGHT: 0.05 + COCOW-TRAIN: + WEIGHT: 0.1 + HALPE-TRAIN: + WEIGHT: 0.05 + MPIINZSL-TRAIN: + WEIGHT: 0.05 + HO3D-TRAIN: + WEIGHT: 0.05 + H2O3D-TRAIN: + WEIGHT: 0.05 + DEX-TRAIN: + WEIGHT: 0.05 + VAL: + FREIHAND-TRAIN: + WEIGHT: 1.0 + MOCAP: FREIHAND-MOCAP + BETAS_REG: true + CONFIG: + SCALE_FACTOR: 0.3 + ROT_FACTOR: 30 + TRANS_FACTOR: 0.02 + COLOR_SCALE: 0.2 + ROT_AUG_RATE: 0.6 + TRANS_AUG_RATE: 0.5 + DO_FLIP: false + FLIP_AUG_RATE: 0.0 + EXTREME_CROP_AUG_RATE: 0.0 + EXTREME_CROP_AUG_LEVEL: 1 +extras: + ignore_warnings: false + enforce_tags: true + print_config: true +exp_name: hamer +MANO: + DATA_DIR: _DATA/data/ + MODEL_PATH: data/mano + GENDER: neutral + NUM_HAND_JOINTS: 15 + MEAN_PARAMS: data/mano_mean_params.npz + CREATE_BODY_POSE: false +EXTRA: + FOCAL_LENGTH: 5000 + NUM_LOG_IMAGES: 4 + NUM_LOG_SAMPLES_PER_IMAGE: 8 + PELVIS_IND: 0 +GENERAL: + TOTAL_STEPS: 1000000 + LOG_STEPS: 1000 + VAL_STEPS: 1000 + CHECKPOINT_STEPS: 10000 + CHECKPOINT_SAVE_TOP_K: 1 + NUM_WORKERS: 8 + PREFETCH_FACTOR: 2 +TRAIN: + LR: 1.0e-05 + WEIGHT_DECAY: 0.0001 + BATCH_SIZE: 32 + LOSS_REDUCTION: mean + NUM_TRAIN_SAMPLES: 2 + NUM_TEST_SAMPLES: 64 + POSE_2D_NOISE_RATIO: 0.01 + SMPL_PARAM_NOISE_RATIO: 0.005 +MODEL: + IMAGE_SIZE: 256 + IMAGE_MEAN: + - 0.485 + - 0.456 + - 0.406 + IMAGE_STD: + - 0.229 + - 0.224 + - 0.225 + BACKBONE: + TYPE: vit + PRETRAINED_WEIGHTS: hamer_training_data/vitpose_backbone.pth + MANO_HEAD: + TYPE: transformer_decoder + IN_CHANNELS: 2048 + TRANSFORMER_DECODER: + depth: 6 + heads: 8 + mlp_dim: 1024 + dim_head: 64 + dropout: 0.0 + emb_dropout: 0.0 + norm: layer + context_dim: 1280 +LOSS_WEIGHTS: + KEYPOINTS_3D: 0.05 + KEYPOINTS_2D: 0.01 + GLOBAL_ORIENT: 0.001 + HAND_POSE: 0.001 + BETAS: 0.0005 + ADVERSARIAL: 0.0005 diff --git a/third_party/Dyn-HaMR/_DATA/hmp_model/mean-neutral-128-30fps.pt b/third_party/Dyn-HaMR/_DATA/hmp_model/mean-neutral-128-30fps.pt new file mode 100644 index 0000000..39d5283 --- /dev/null +++ b/third_party/Dyn-HaMR/_DATA/hmp_model/mean-neutral-128-30fps.pt @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:f895ea3e9913dcf0daeab06c770d44bdcd1e780a887f2ef8046157688fd0f431 +size 5834 diff --git a/third_party/Dyn-HaMR/_DATA/hmp_model/results/model/global_encoder.pth b/third_party/Dyn-HaMR/_DATA/hmp_model/results/model/global_encoder.pth new file mode 100644 index 0000000..d30cec4 --- /dev/null +++ b/third_party/Dyn-HaMR/_DATA/hmp_model/results/model/global_encoder.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:53ea55a05bad9816e087d9296b4aa40cc5133139417156e3faffdc3ff9957f54 +size 16944962 diff --git a/third_party/Dyn-HaMR/_DATA/hmp_model/results/model/local_encoder.pth.part-000 b/third_party/Dyn-HaMR/_DATA/hmp_model/results/model/local_encoder.pth.part-000 new file mode 100644 index 0000000..47e989a --- /dev/null +++ b/third_party/Dyn-HaMR/_DATA/hmp_model/results/model/local_encoder.pth.part-000 @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:f7c24671031d4a4210d826d025aad4fcedd96e32e9884699b583fb4bc5e2f486 +size 50331648 diff --git a/third_party/Dyn-HaMR/_DATA/hmp_model/results/model/local_encoder.pth.part-001 b/third_party/Dyn-HaMR/_DATA/hmp_model/results/model/local_encoder.pth.part-001 new file mode 100644 index 0000000..ac855cb --- /dev/null +++ b/third_party/Dyn-HaMR/_DATA/hmp_model/results/model/local_encoder.pth.part-001 @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:66e2c7c9e0086736f786a29bd09c554a56c76929f8399905f40c0dd1ba8e7f5e +size 50331648 diff --git a/third_party/Dyn-HaMR/_DATA/hmp_model/results/model/local_encoder.pth.part-002 b/third_party/Dyn-HaMR/_DATA/hmp_model/results/model/local_encoder.pth.part-002 new file mode 100644 index 0000000..f07d4f4 --- /dev/null +++ b/third_party/Dyn-HaMR/_DATA/hmp_model/results/model/local_encoder.pth.part-002 @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:a80f2af2b87889f05bd0da82d8030e3941377c4de9470316b0cd87ad83f25e1d +size 50331648 diff --git a/third_party/Dyn-HaMR/_DATA/hmp_model/results/model/local_encoder.pth.part-003 b/third_party/Dyn-HaMR/_DATA/hmp_model/results/model/local_encoder.pth.part-003 new file mode 100644 index 0000000..8319ff0 --- /dev/null +++ b/third_party/Dyn-HaMR/_DATA/hmp_model/results/model/local_encoder.pth.part-003 @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:2ce86e188c448068f646941e6087598bce62cf5e708aa1b6d7b1f5c9871d8bed +size 10195744 diff --git a/third_party/Dyn-HaMR/_DATA/hmp_model/results/model/loss.pkl b/third_party/Dyn-HaMR/_DATA/hmp_model/results/model/loss.pkl new file mode 100644 index 0000000..c71b38e --- /dev/null +++ b/third_party/Dyn-HaMR/_DATA/hmp_model/results/model/loss.pkl @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:85d6b88d2aa847462c6f73797a29e66c12c77797583b31f174a5f8ee9d817e28 +size 11964704 diff --git a/third_party/Dyn-HaMR/_DATA/hmp_model/results/model/nemf.pth.part-000 b/third_party/Dyn-HaMR/_DATA/hmp_model/results/model/nemf.pth.part-000 new file mode 100644 index 0000000..f68a40a --- /dev/null +++ b/third_party/Dyn-HaMR/_DATA/hmp_model/results/model/nemf.pth.part-000 @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:024d2a563638073e9558fcbe310efff4eb98b54f634a057251df5a185cb64642 +size 50331648 diff --git a/third_party/Dyn-HaMR/_DATA/hmp_model/results/model/nemf.pth.part-001 b/third_party/Dyn-HaMR/_DATA/hmp_model/results/model/nemf.pth.part-001 new file mode 100644 index 0000000..50579ef --- /dev/null +++ b/third_party/Dyn-HaMR/_DATA/hmp_model/results/model/nemf.pth.part-001 @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:01130530a2ddd32bba952bc1f05dcc211067109ba70db20d0b160abdfc71dc08 +size 42289252 diff --git a/third_party/Dyn-HaMR/_DATA/hmp_model/results/model/optimizer.pth.part-000 b/third_party/Dyn-HaMR/_DATA/hmp_model/results/model/optimizer.pth.part-000 new file mode 100644 index 0000000..b297325 --- /dev/null +++ b/third_party/Dyn-HaMR/_DATA/hmp_model/results/model/optimizer.pth.part-000 @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:a760b0f7f579e5810d50301a389f14ce203a55633ddbd1d6c4bd38669a88c5f8 +size 50331648 diff --git a/third_party/Dyn-HaMR/_DATA/hmp_model/results/model/optimizer.pth.part-001 b/third_party/Dyn-HaMR/_DATA/hmp_model/results/model/optimizer.pth.part-001 new file mode 100644 index 0000000..3a4693d --- /dev/null +++ b/third_party/Dyn-HaMR/_DATA/hmp_model/results/model/optimizer.pth.part-001 @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:825cc19bce9e45c06a8989180bc7eea8893241a3fb988c38e7b0e07ac8fab0e6 +size 50331648 diff --git a/third_party/Dyn-HaMR/_DATA/hmp_model/results/model/optimizer.pth.part-002 b/third_party/Dyn-HaMR/_DATA/hmp_model/results/model/optimizer.pth.part-002 new file mode 100644 index 0000000..dc6fbc8 --- /dev/null +++ b/third_party/Dyn-HaMR/_DATA/hmp_model/results/model/optimizer.pth.part-002 @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:00a6e5c7680a36beb14f05963a0d0037b91ce47a712862f89e81a0240bf5d9d2 +size 50331648 diff --git a/third_party/Dyn-HaMR/_DATA/hmp_model/results/model/optimizer.pth.part-003 b/third_party/Dyn-HaMR/_DATA/hmp_model/results/model/optimizer.pth.part-003 new file mode 100644 index 0000000..21aac3d --- /dev/null +++ b/third_party/Dyn-HaMR/_DATA/hmp_model/results/model/optimizer.pth.part-003 @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:81e385ac56e3d35029f696ab8bb4b6c32bc76d6456c4a24fbc4bfd5a5fdcd4b8 +size 50331648 diff --git a/third_party/Dyn-HaMR/_DATA/hmp_model/results/model/optimizer.pth.part-004 b/third_party/Dyn-HaMR/_DATA/hmp_model/results/model/optimizer.pth.part-004 new file mode 100644 index 0000000..7405d60 --- /dev/null +++ b/third_party/Dyn-HaMR/_DATA/hmp_model/results/model/optimizer.pth.part-004 @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:7abc4f1ab62e72ce3adb0dfcb8400f0be69f29197c376b24f223add673e5a85a +size 50331648 diff --git a/third_party/Dyn-HaMR/_DATA/hmp_model/results/model/optimizer.pth.part-005 b/third_party/Dyn-HaMR/_DATA/hmp_model/results/model/optimizer.pth.part-005 new file mode 100644 index 0000000..b9e1afe --- /dev/null +++ b/third_party/Dyn-HaMR/_DATA/hmp_model/results/model/optimizer.pth.part-005 @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:fc08da53eae0cc5de0487f8847be3265d838b23a0d3d5a93662b20223b345359 +size 50331648 diff --git a/third_party/Dyn-HaMR/_DATA/hmp_model/results/model/optimizer.pth.part-006 b/third_party/Dyn-HaMR/_DATA/hmp_model/results/model/optimizer.pth.part-006 new file mode 100644 index 0000000..b9ec698 --- /dev/null +++ b/third_party/Dyn-HaMR/_DATA/hmp_model/results/model/optimizer.pth.part-006 @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:e68b1eaf62bc2462056c671c44ef32b4d6afe73fe15c8af9ddca45dd2fcff7b6 +size 50331648 diff --git a/third_party/Dyn-HaMR/_DATA/hmp_model/results/model/optimizer.pth.part-007 b/third_party/Dyn-HaMR/_DATA/hmp_model/results/model/optimizer.pth.part-007 new file mode 100644 index 0000000..3c1c7f5 --- /dev/null +++ b/third_party/Dyn-HaMR/_DATA/hmp_model/results/model/optimizer.pth.part-007 @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:19469f7e7d4920d360ced1f459e245815efab4e7acb5fd43c3bf9a3ef71821ff +size 50331648 diff --git a/third_party/Dyn-HaMR/_DATA/hmp_model/results/model/optimizer.pth.part-008 b/third_party/Dyn-HaMR/_DATA/hmp_model/results/model/optimizer.pth.part-008 new file mode 100644 index 0000000..de5dde7 --- /dev/null +++ b/third_party/Dyn-HaMR/_DATA/hmp_model/results/model/optimizer.pth.part-008 @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:a86fbdd60b08f12e369954c155d062c5afa12cc6dca11b9d5459aa7aa55a42ea +size 50331648 diff --git a/third_party/Dyn-HaMR/_DATA/hmp_model/results/model/optimizer.pth.part-009 b/third_party/Dyn-HaMR/_DATA/hmp_model/results/model/optimizer.pth.part-009 new file mode 100644 index 0000000..27ad475 --- /dev/null +++ b/third_party/Dyn-HaMR/_DATA/hmp_model/results/model/optimizer.pth.part-009 @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:c84826384b6d32333800e6e1248e8d142c4babb3c4bc9d817e7e43a91b5a7acd +size 50331648 diff --git a/third_party/Dyn-HaMR/_DATA/hmp_model/results/model/optimizer.pth.part-010 b/third_party/Dyn-HaMR/_DATA/hmp_model/results/model/optimizer.pth.part-010 new file mode 100644 index 0000000..757e5c6 --- /dev/null +++ b/third_party/Dyn-HaMR/_DATA/hmp_model/results/model/optimizer.pth.part-010 @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:e76007b500a5373752591782044a0e5385c136f4bb550d7d3580728d5344f53f +size 38213818 diff --git a/third_party/Dyn-HaMR/_DATA/hmp_model/std-neutral-128-30fps.pt b/third_party/Dyn-HaMR/_DATA/hmp_model/std-neutral-128-30fps.pt new file mode 100644 index 0000000..5f87d66 --- /dev/null +++ b/third_party/Dyn-HaMR/_DATA/hmp_model/std-neutral-128-30fps.pt @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:e1da9a50afbc1015e9fc846edbd4457a46e3594d85265020ed20ba61546a126a +size 5756 diff --git a/third_party/Dyn-HaMR/dyn-hamr/optim/BMC/CONVEX_HULLS.npy b/third_party/Dyn-HaMR/dyn-hamr/optim/BMC/CONVEX_HULLS.npy new file mode 100644 index 0000000..77e43a7 --- /dev/null +++ b/third_party/Dyn-HaMR/dyn-hamr/optim/BMC/CONVEX_HULLS.npy @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:013177b2fdbdbbb834d5f3daab5e291f66c67193e2f52efed5247c8865edfec3 +size 4897 diff --git a/third_party/Dyn-HaMR/dyn-hamr/optim/BMC/PHI_max.npy b/third_party/Dyn-HaMR/dyn-hamr/optim/BMC/PHI_max.npy new file mode 100644 index 0000000..43b16d0 --- /dev/null +++ b/third_party/Dyn-HaMR/dyn-hamr/optim/BMC/PHI_max.npy @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:9b9e95091c47381fa872b93b28d28c093dbcc1576cc617eb29f23ce53648da99 +size 160 diff --git a/third_party/Dyn-HaMR/dyn-hamr/optim/BMC/PHI_min.npy b/third_party/Dyn-HaMR/dyn-hamr/optim/BMC/PHI_min.npy new file mode 100644 index 0000000..e178be1 --- /dev/null +++ b/third_party/Dyn-HaMR/dyn-hamr/optim/BMC/PHI_min.npy @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:ce403440680ecfdfc23ee7dae42fb6a0fdfac1919053458d476e589bab63710c +size 160 diff --git a/third_party/Dyn-HaMR/dyn-hamr/optim/BMC/bone_len_max.npy b/third_party/Dyn-HaMR/dyn-hamr/optim/BMC/bone_len_max.npy new file mode 100644 index 0000000..9cd0dae --- /dev/null +++ b/third_party/Dyn-HaMR/dyn-hamr/optim/BMC/bone_len_max.npy @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:26f9ea48a090100fb2f70f688d3fad46dacfbbf365e5f8b6a483b6a971a7fa67 +size 208 diff --git a/third_party/Dyn-HaMR/dyn-hamr/optim/BMC/bone_len_min.npy b/third_party/Dyn-HaMR/dyn-hamr/optim/BMC/bone_len_min.npy new file mode 100644 index 0000000..4b56e93 --- /dev/null +++ b/third_party/Dyn-HaMR/dyn-hamr/optim/BMC/bone_len_min.npy @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:f91a46e2df3f805095aa7b20d5294bc30b9d83d76e4f5a4e7fd1f1ed3bc692fa +size 208 diff --git a/third_party/Dyn-HaMR/dyn-hamr/optim/BMC/curvatures_max.npy b/third_party/Dyn-HaMR/dyn-hamr/optim/BMC/curvatures_max.npy new file mode 100644 index 0000000..39069bd --- /dev/null +++ b/third_party/Dyn-HaMR/dyn-hamr/optim/BMC/curvatures_max.npy @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:e576ffc657cd50ca0b1a5292bd4e44639b960a68e4eb8bd758cb502620c48880 +size 160 diff --git a/third_party/Dyn-HaMR/dyn-hamr/optim/BMC/curvatures_min.npy b/third_party/Dyn-HaMR/dyn-hamr/optim/BMC/curvatures_min.npy new file mode 100644 index 0000000..9363b96 --- /dev/null +++ b/third_party/Dyn-HaMR/dyn-hamr/optim/BMC/curvatures_min.npy @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:31cc5766a93f4b018959e4b18cc93c309b3d0d8627bd5cf10f8d3cd429549b37 +size 160 diff --git a/third_party/Dyn-HaMR/dyn-hamr/optim/BMC/joint_angles.npy.part-000 b/third_party/Dyn-HaMR/dyn-hamr/optim/BMC/joint_angles.npy.part-000 new file mode 100644 index 0000000..4cf473c --- /dev/null +++ b/third_party/Dyn-HaMR/dyn-hamr/optim/BMC/joint_angles.npy.part-000 @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:1f6cae4c998871477b35f1c5f671943f1063d247c5dc2668fd90e71894bb29bb +size 50331648 diff --git a/third_party/Dyn-HaMR/dyn-hamr/optim/BMC/joint_angles.npy.part-001 b/third_party/Dyn-HaMR/dyn-hamr/optim/BMC/joint_angles.npy.part-001 new file mode 100644 index 0000000..b952ebf --- /dev/null +++ b/third_party/Dyn-HaMR/dyn-hamr/optim/BMC/joint_angles.npy.part-001 @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:c34b254b128f5123b4a99369c4a43e93811124c968207ffd7276c0ac03432986 +size 39784880 diff --git a/third_party/Dyn-HaMR/dyn-hamr/vis/tools.py b/third_party/Dyn-HaMR/dyn-hamr/vis/tools.py index 648726f..029eeb8 100644 --- a/third_party/Dyn-HaMR/dyn-hamr/vis/tools.py +++ b/third_party/Dyn-HaMR/dyn-hamr/vis/tools.py @@ -1037,6 +1037,11 @@ def batch_rodrigues(param): def smooth_global_rot_matrix(pred_rots, OE_filter): rot_mat = batch_rodrigues(pred_rots[None]).squeeze(0) smoothed_rot_mat = OE_filter.process(rot_mat) + # Elementwise filtering leaves SO(3); project back before axis-angle conversion. + u, _, vh = torch.linalg.svd(smoothed_rot_mat) + correction = torch.eye(3, dtype=u.dtype, device=u.device) + correction[-1, -1] = torch.det(u @ vh) + smoothed_rot_mat = u @ correction @ vh smoothed_rot = rotation_matrix_to_angle_axis(smoothed_rot_mat.reshape(1,3,3)).reshape(-1) return smoothed_rot @@ -1106,4 +1111,4 @@ class OneEuroFilter: cutoff = self.mincutoff + self.beta * torch.abs(edx) if print_inter: print(self.compute_alpha(cutoff)) - return self.x_filter.process(x, self.compute_alpha(cutoff)) \ No newline at end of file + return self.x_filter.process(x, self.compute_alpha(cutoff)) diff --git a/third_party/Dyn-HaMR/dyn-hamr/vis/viewer.py b/third_party/Dyn-HaMR/dyn-hamr/vis/viewer.py index ce75a06..19f49e5 100644 --- a/third_party/Dyn-HaMR/dyn-hamr/vis/viewer.py +++ b/third_party/Dyn-HaMR/dyn-hamr/vis/viewer.py @@ -177,6 +177,8 @@ class AnimationBase(object): """ Add a single camera marker node that we move around when we animate """ + if os.environ.get("DYNHAMR_HIDE_CAMERA_MARKERS") == "1": + return print("ADDING CAMERA MARKERS") if self.cam_marker_node is None: cam_marker = make_camera_marker(up="y") diff --git a/third_party/Dyn-HaMR/third-party/hamer/pretrained_models/detector.pt b/third_party/Dyn-HaMR/third-party/hamer/pretrained_models/detector.pt new file mode 120000 index 0000000..c0a1589 --- /dev/null +++ b/third_party/Dyn-HaMR/third-party/hamer/pretrained_models/detector.pt @@ -0,0 +1 @@ +../../../../../weights/detector.pt \ No newline at end of file diff --git a/third_party/FoundationPose/LICENSE b/third_party/FoundationPose/LICENSE new file mode 100644 index 0000000..6704932 --- /dev/null +++ b/third_party/FoundationPose/LICENSE @@ -0,0 +1,94 @@ +Copyright (c) 2022-Present, NVIDIA Corporation & affiliates. All rights reserved. + + +======================================================================= + +1. Definitions + +"Licensor" means any person or entity that distributes its Work. + +"Software" means the original work of authorship made available under +this License. + +"Work" means the Software and any additions to or derivative works of +the Software that are made available under this License. + +The terms "reproduce," "reproduction," "derivative works," and +"distribution" have the meaning as provided under U.S. copyright law; +provided, however, that for the purposes of this License, derivative +works shall not include works that remain separable from, or merely +link (or bind by name) to the interfaces of, the Work. + +Works, including the Software, are "made available" under this License +by including in or with the Work either (a) a copyright notice +referencing the applicability of this License to the Work, or (b) a +copy of this License. + +2. License Grants + + 2.1 Copyright Grant. Subject to the terms and conditions of this + License, each Licensor grants to you a perpetual, worldwide, + non-exclusive, royalty-free, copyright license to reproduce, + prepare derivative works of, publicly display, publicly perform, + sublicense and distribute its Work and any resulting derivative + works in any form. + +3. Limitations + + 3.1 Redistribution. You may reproduce or distribute the Work only + if (a) you do so under this License, (b) you include a complete + copy of this License with your distribution, and (c) you retain + without modification any copyright, patent, trademark, or + attribution notices that are present in the Work. + + 3.2 Derivative Works. You may specify that additional or different + terms apply to the use, reproduction, and distribution of your + derivative works of the Work ("Your Terms") only if (a) Your Terms + provide that the use limitation in Section 3.3 applies to your + derivative works, and (b) you identify the specific derivative + works that are subject to Your Terms. Notwithstanding Your Terms, + this License (including the redistribution requirements in Section + 3.1) will continue to apply to the Work itself. + + 3.3 Use Limitation. The Work and any derivative works thereof only + may be used or intended for use non-commercially. Notwithstanding + the foregoing, NVIDIA and its affiliates may use the Work and any + derivative works commercially. As used herein, "non-commercially" + means for research or evaluation purposes only. + + 3.4 Patent Claims. If you bring or threaten to bring a patent claim + against any Licensor (including any claim, cross-claim or + counterclaim in a lawsuit) to enforce any patents that you allege + are infringed by any Work, then your rights under this License from + such Licensor (including the grant in Section 2.1) will terminate + immediately. + + 3.5 Trademarks. This License does not grant any rights to use any + Licensor�s or its affiliates� names, logos, or trademarks, except + as necessary to reproduce the notices described in this License. + + 3.6 Termination. If you violate any term of this License, then your + rights under this License (including the grant in Section 2.1) will + terminate immediately. + +4. Disclaimer of Warranty. + +THE WORK IS PROVIDED "AS IS" WITHOUT WARRANTIES OR CONDITIONS OF ANY +KIND, EITHER EXPRESS OR IMPLIED, INCLUDING WARRANTIES OR CONDITIONS OF +MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, TITLE OR +NON-INFRINGEMENT. YOU BEAR THE RISK OF UNDERTAKING ANY ACTIVITIES UNDER +THIS LICENSE. + +5. Limitation of Liability. + +EXCEPT AS PROHIBITED BY APPLICABLE LAW, IN NO EVENT AND UNDER NO LEGAL +THEORY, WHETHER IN TORT (INCLUDING NEGLIGENCE), CONTRACT, OR OTHERWISE +SHALL ANY LICENSOR BE LIABLE TO YOU FOR DAMAGES, INCLUDING ANY DIRECT, +INDIRECT, SPECIAL, INCIDENTAL, OR CONSEQUENTIAL DAMAGES ARISING OUT OF +OR RELATED TO THIS LICENSE, THE USE OR INABILITY TO USE THE WORK +(INCLUDING BUT NOT LIMITED TO LOSS OF GOODWILL, BUSINESS INTERRUPTION, +LOST PROFITS OR DATA, COMPUTER FAILURE OR MALFUNCTION, OR ANY OTHER +COMMERCIAL DAMAGES OR LOSSES), EVEN IF THE LICENSOR HAS BEEN ADVISED OF +THE POSSIBILITY OF SUCH DAMAGES. + +======================================================================= diff --git a/third_party/FoundationPose/Utils.py b/third_party/FoundationPose/Utils.py new file mode 100644 index 0000000..283028f --- /dev/null +++ b/third_party/FoundationPose/Utils.py @@ -0,0 +1,1022 @@ +# Copyright (c) 2023, NVIDIA CORPORATION. All rights reserved. +# +# NVIDIA CORPORATION and its licensors retain all intellectual property +# and proprietary rights in and to this software, related documentation +# and any modifications thereto. Any use, reproduction, disclosure or +# distribution of this software and related documentation without an express +# license agreement from NVIDIA CORPORATION is strictly prohibited. + + +import os, sys, time,torch,pickle,trimesh,itertools,pdb,zipfile,datetime,imageio,gzip,logging,joblib,importlib,uuid,signal,multiprocessing,psutil,subprocess,tarfile,scipy,argparse +from pytorch3d.transforms import so3_log_map,so3_exp_map,se3_exp_map,se3_log_map,matrix_to_axis_angle,matrix_to_euler_angles,euler_angles_to_matrix, rotation_6d_to_matrix +from pytorch3d.renderer import FoVPerspectiveCameras, PerspectiveCameras, look_at_view_transform, look_at_rotation, RasterizationSettings, MeshRenderer, MeshRasterizer, BlendParams, SoftSilhouetteShader, HardPhongShader, PointLights, TexturesVertex +from pytorch3d.renderer.mesh.rasterize_meshes import barycentric_coordinates +from pytorch3d.renderer.mesh.shader import SoftDepthShader, HardFlatShader +from pytorch3d.renderer.mesh.textures import Textures +from pytorch3d.structures import Meshes +from scipy.interpolate import griddata +import nvdiffrast.torch as dr +import torch.nn.functional as F +import torchvision +import torch.nn as nn +from functools import partial +import pandas as pd +import open3d as o3d +from uuid import uuid4 +import cv2 +from PIL import Image +import numpy as np +from collections import defaultdict +import multiprocessing as mp +import matplotlib.pyplot as plt +import math,glob,re,copy +from transformations import * +from scipy.spatial import cKDTree +from collections import OrderedDict +import ruamel.yaml +yaml = ruamel.yaml.YAML() +code_dir = os.path.dirname(os.path.realpath(__file__)) +sys.path.append(code_dir) +try: + import kornia +except Exception: + kornia = None +try: + _mycpp_build = os.path.join(code_dir, 'mycpp', 'build') + if os.path.isdir(_mycpp_build): + sys.path.insert(0, _mycpp_build) + import mycpp +except Exception: + mycpp = None +try: + from bundlesdf.mycuda import common +except: + common = None +try: + import warp as wp + wp.init() +except: + wp = None +enable_timer = 0 + +def NestDict(): + return defaultdict(NestDict) + +to8b = lambda x : (255*np.clip(x,0,1)).astype(np.uint8) + +BAD_DEPTH = 99 +BAD_COLOR = 0 + +glcam_in_cvcam = np.array([[1,0,0,0], + [0,-1,0,0], + [0,0,-1,0], + [0,0,0,1]]).astype(float) + +COLOR_MAP=np.array([[0, 0, 0], #Ignore + [128,0,0], #Background + [0,128,0], #Wall + [128,128,0], #Floor + [0,0,128], #Ceiling + [128,0,128], #Table + [0,128,128], #Chair + [128,128,128], #Window + [64,0,0], #Door + [192,0,0], #Monitor + [64, 128, 0], # 11th + [192, 0, 128], + [64, 128, 128], + [192, 128, 128], + [0, 64, 0], + [128, 64, 0], + [0, 192, 0], + [128, 192, 0], + ]) + + +def set_logging_format(level=logging.INFO): + importlib.reload(logging) + FORMAT = '[%(funcName)s()] %(message)s' + logging.basicConfig(level=level, format=FORMAT) + +set_logging_format() + + + + +def make_mesh_tensors(mesh, device='cuda', max_tex_size=None): + mesh_tensors = {} + if isinstance(mesh.visual, trimesh.visual.texture.TextureVisuals): + img = np.array(mesh.visual.material.image.convert('RGB')) + img = img[...,:3] + if max_tex_size is not None: + max_size = max(img.shape[0], img.shape[1]) + if max_size>max_tex_size: + scale = 1/max_size * max_tex_size + img = cv2.resize(img, fx=scale, fy=scale, dsize=None) + mesh_tensors['tex'] = torch.as_tensor(img, device=device, dtype=torch.float)[None]/255.0 + mesh_tensors['uv_idx'] = torch.as_tensor(mesh.faces, device=device, dtype=torch.int) + uv = torch.as_tensor(mesh.visual.uv, device=device, dtype=torch.float) + uv[:,1] = 1 - uv[:,1] + mesh_tensors['uv'] = uv + else: + if mesh.visual.vertex_colors is None: + logging.info(f"WARN: mesh doesn't have vertex_colors, assigning a pure color") + mesh.visual.vertex_colors = np.tile(np.array([128,128,128]).reshape(1,3), (len(mesh.vertices), 1)) + mesh_tensors['vertex_color'] = torch.as_tensor(mesh.visual.vertex_colors[...,:3], device=device, dtype=torch.float)/255.0 + + mesh_tensors.update({ + 'pos': torch.tensor(mesh.vertices, device=device, dtype=torch.float), + 'faces': torch.tensor(mesh.faces, device=device, dtype=torch.int), + 'vnormals': torch.tensor(mesh.vertex_normals, device=device, dtype=torch.float), + }) + return mesh_tensors + + +def nvdiffrast_render(K=None, H=None, W=None, ob_in_cams=None, glctx=None, context='cuda', get_normal=False, mesh_tensors=None, mesh=None, projection_mat=None, bbox2d=None, output_size=None, use_light=False, light_color=None, light_dir=np.array([0,0,1]), light_pos=np.array([0,0,0]), w_ambient=0.8, w_diffuse=0.5, extra={}): + '''Just plain rendering, not support any gradient + @K: (3,3) np array + @ob_in_cams: (N,4,4) torch tensor, openCV camera + @projection_mat: np array (4,4) + @output_size: (height, width) + @bbox2d: (N,4) (umin,vmin,umax,vmax) if only roi need to render. + @light_dir: in cam space + @light_pos: in cam space + ''' + if glctx is None: + if context == 'gl': + glctx = dr.RasterizeGLContext() + elif context=='cuda': + glctx = dr.RasterizeCudaContext() + else: + raise NotImplementedError + logging.info("created context") + + if mesh_tensors is None: + mesh_tensors = make_mesh_tensors(mesh) + pos = mesh_tensors['pos'] + vnormals = mesh_tensors['vnormals'] + pos_idx = mesh_tensors['faces'] + has_tex = 'tex' in mesh_tensors + + ob_in_glcams = torch.tensor(glcam_in_cvcam, device='cuda', dtype=torch.float)[None]@ob_in_cams + if projection_mat is None: + projection_mat = projection_matrix_from_intrinsics(K, height=H, width=W, znear=0.001, zfar=100) + projection_mat = torch.as_tensor(projection_mat.reshape(-1,4,4), device='cuda', dtype=torch.float) + mtx = projection_mat@ob_in_glcams + + if output_size is None: + output_size = np.asarray([H,W]) + + pts_cam = transform_pts(pos, ob_in_cams) + pos_homo = to_homo_torch(pos) + pos_clip = (mtx[:,None]@pos_homo[None,...,None])[...,0] + if bbox2d is not None: + l = bbox2d[:,0] + t = H-bbox2d[:,1] + r = bbox2d[:,2] + b = H-bbox2d[:,3] + tf = torch.eye(4, dtype=torch.float, device='cuda').reshape(1,4,4).expand(len(ob_in_cams),4,4).contiguous() + tf[:,0,0] = W/(r-l) + tf[:,1,1] = H/(t-b) + tf[:,3,0] = (W-r-l)/(r-l) + tf[:,3,1] = (H-t-b)/(t-b) + pos_clip = pos_clip@tf + rast_out, _ = dr.rasterize(glctx, pos_clip, pos_idx, resolution=np.asarray(output_size)) + xyz_map, _ = dr.interpolate(pts_cam, rast_out, pos_idx) + depth = xyz_map[...,2] + if has_tex: + texc, _ = dr.interpolate(mesh_tensors['uv'], rast_out, mesh_tensors['uv_idx']) + color = dr.texture(mesh_tensors['tex'], texc, filter_mode='linear') + else: + color, _ = dr.interpolate(mesh_tensors['vertex_color'], rast_out, pos_idx) + + if use_light: + get_normal = True + if get_normal: + vnormals_cam = transform_dirs(vnormals, ob_in_cams) + normal_map, _ = dr.interpolate(vnormals_cam, rast_out, pos_idx) + normal_map = F.normalize(normal_map, dim=-1) + normal_map = torch.flip(normal_map, dims=[1]) + else: + normal_map = None + + if use_light: + if light_dir is not None: + light_dir_neg = -torch.as_tensor(light_dir, dtype=torch.float, device='cuda') + else: + light_dir_neg = torch.as_tensor(light_pos, dtype=torch.float, device='cuda').reshape(1,1,3) - pts_cam + diffuse_intensity = (F.normalize(vnormals_cam, dim=-1) * F.normalize(light_dir_neg, dim=-1)).sum(dim=-1).clip(0, 1)[...,None] + diffuse_intensity_map, _ = dr.interpolate(diffuse_intensity, rast_out, pos_idx) # (N_pose, H, W, 1) + if light_color is None: + light_color = color + else: + light_color = torch.as_tensor(light_color, device='cuda', dtype=torch.float) + color = color*w_ambient + diffuse_intensity_map*light_color*w_diffuse + + color = color.clip(0,1) + color = color * torch.clamp(rast_out[..., -1:], 0, 1) # Mask out background using alpha + color = torch.flip(color, dims=[1]) # Flip Y coordinates + depth = torch.flip(depth, dims=[1]) + extra['xyz_map'] = torch.flip(xyz_map, dims=[1]) + return color, depth, normal_map + + +def set_seed(random_seed): + import torch,random + np.random.seed(random_seed) + random.seed(random_seed) + torch.manual_seed(random_seed) + torch.cuda.manual_seed_all(random_seed) + torch.backends.cudnn.deterministic = True + torch.backends.cudnn.benchmark = False + + +def add_err(pred,gt,model_pts,symetry_tfs=np.eye(4)[None]): + """ + Average Distance of Model Points for objects with no indistinguishable views + - by Hinterstoisser et al. (ACCV 2012). + """ + pred_pts = transform_pts(model_pts, pred) + gt_pts = transform_pts(model_pts, gt) + e = np.linalg.norm(pred_pts - gt_pts, axis=-1).mean() + return e + +def adds_err(pred,gt,model_pts): + """ + @pred: 4x4 mat + @gt: + @model: (N,3) + """ + pred_pts = transform_pts(model_pts, pred) + gt_pts = transform_pts(model_pts, gt) + nn_index = cKDTree(pred_pts) + nn_dists, _ = nn_index.query(gt_pts, k=1, workers=-1) + e = nn_dists.mean() + return e + +def compute_auc_sklearn(errs, max_val=0.1, step=0.001): + from sklearn import metrics + errs = np.sort(np.array(errs)) + X = np.arange(0, max_val+step, step) + Y = np.ones(len(X)) + for i,x in enumerate(X): + y = (errs<=x).sum()/len(errs) + Y[i] = y + if y>=1: + break + auc = metrics.auc(X, Y) / (max_val*1) + return auc + + + +def normalizeRotation(pose): + '''Assume no shear case + ''' + new_pose = pose.copy() + scales = np.linalg.norm(pose[:3,:3],axis=0) + new_pose[:3,:3] /= scales.reshape(1,3) + return new_pose + + + +def toOpen3dCloud(points,colors=None,normals=None): + cloud = o3d.geometry.PointCloud() + cloud.points = o3d.utility.Vector3dVector(points.astype(np.float64)) + if colors is not None: + if colors.max()>1: + colors = colors/255.0 + cloud.colors = o3d.utility.Vector3dVector(colors.astype(np.float64)) + if normals is not None: + cloud.normals = o3d.utility.Vector3dVector(normals.astype(np.float64)) + return cloud + + + +def make_grid_image(imgs, nrow, padding=5, pad_value=255): + ''' + @imgs: (B,H,W,C) np array + @nrow: num of images per row + ''' + grid = torchvision.utils.make_grid(torch.as_tensor(np.asarray(imgs)).permute(0,3,1,2), nrow=nrow, padding=padding, pad_value=pad_value) + grid = grid.permute(1,2,0).contiguous().data.cpu().numpy().astype(np.uint8) + return grid + + +if wp is not None: + @wp.kernel(enable_backward=False) + def bilateral_filter_depth_kernel(depth:wp.array(dtype=float, ndim=2), out:wp.array(dtype=float, ndim=2), radius:int, zfar:float, sigmaD:float, sigmaR:float): + h,w = wp.tid() + H = depth.shape[0] + W = depth.shape[1] + if w>=W or h>=H: + return + out[h,w] = 0.0 + mean_depth = float(0.0) + num_valid = int(0) + for u in range(w-radius, w+radius+1): + if u<0 or u>=W: + continue + for v in range(h-radius, h+radius+1): + if v<0 or v>=H: + continue + cur_depth = depth[v,u] + if cur_depth>=0.001 and cur_depth=W: + continue + for v in range(h-radius, h+radius+1): + if v<0 or v>=H: + continue + cur_depth = depth[v,u] + if cur_depth>=0.001 and cur_depth0 and num_valid>0: + out[h,w] = sum/sum_weight + + def bilateral_filter_depth(depth, radius=2, zfar=100, sigmaD=2, sigmaR=100000, device='cuda'): + if isinstance(depth, np.ndarray): + depth_wp = wp.array(depth, dtype=float, device=device) + else: + depth_wp = wp.from_torch(depth) + out_wp = wp.zeros(depth.shape, dtype=float, device=device) + wp.launch(kernel=bilateral_filter_depth_kernel, device=device, dim=[depth.shape[0], depth.shape[1]], inputs=[depth_wp, out_wp, radius, zfar, sigmaD, sigmaR]) + depth_out = wp.to_torch(out_wp) + + if isinstance(depth, np.ndarray): + depth_out = depth_out.data.cpu().numpy() + return depth_out + + + @wp.kernel(enable_backward=False) + def erode_depth_kernel(depth:wp.array(dtype=float, ndim=2), out:wp.array(dtype=float, ndim=2), radius:int, depth_diff_thres:float, ratio_thres:float, zfar:float): + h,w = wp.tid() + H = depth.shape[0] + W = depth.shape[1] + if w>=W or h>=H: + return + d_ori = depth[h,w] + if d_ori<0.001 or d_ori>=zfar: + out[h,w] = 0.0 + bad_cnt = float(0) + total = float(0) + for u in range(w-radius, w+radius+1): + if u<0 or u>=W: + continue + for v in range(h-radius, h+radius+1): + if v<0 or v>=H: + continue + cur_depth = depth[v,u] + total += 1.0 + if cur_depth<0.001 or cur_depth>=zfar or abs(cur_depth-d_ori)>depth_diff_thres: + bad_cnt += 1.0 + if bad_cnt/total>ratio_thres: + out[h,w] = 0.0 + else: + out[h,w] = d_ori + + + def erode_depth(depth, radius=2, depth_diff_thres=0.001, ratio_thres=0.8, zfar=100, device='cuda'): + depth_wp = wp.from_torch(torch.as_tensor(depth, dtype=torch.float, device=device)) + out_wp = wp.zeros(depth.shape, dtype=float, device=device) + wp.launch(kernel=erode_depth_kernel, device=device, dim=[depth.shape[0], depth.shape[1]], inputs=[depth_wp, out_wp, radius, depth_diff_thres, ratio_thres, zfar],) + depth_out = wp.to_torch(out_wp) + + if isinstance(depth, np.ndarray): + depth_out = depth_out.data.cpu().numpy() + return depth_out + + + +def depth2xyzmap(depth, K, uvs=None): + invalid_mask = (depth<0.001) + H,W = depth.shape[:2] + if uvs is None: + vs,us = np.meshgrid(np.arange(0,H),np.arange(0,W), sparse=False, indexing='ij') + vs = vs.reshape(-1) + us = us.reshape(-1) + else: + uvs = uvs.round().astype(int) + us = uvs[:,0] + vs = uvs[:,1] + zs = depth[vs,us] + xs = (us-K[0,2])*zs/K[0,0] + ys = (vs-K[1,2])*zs/K[1,1] + pts = np.stack((xs.reshape(-1),ys.reshape(-1),zs.reshape(-1)), 1) #(N,3) + xyz_map = np.zeros((H,W,3), dtype=np.float32) + xyz_map[vs,us] = pts + xyz_map[invalid_mask] = 0 + return xyz_map + + +def depth2xyzmap_batch(depths, Ks, zfar): + ''' + @depths: torch tensor (B,H,W) + @Ks: torch tensor (B,3,3) + ''' + bs = depths.shape[0] + invalid_mask = (depths<0.001) | (depths>zfar) + H,W = depths.shape[-2:] + vs,us = torch.meshgrid(torch.arange(0,H),torch.arange(0,W), indexing='ij') + vs = vs.reshape(-1).float().cuda()[None].expand(bs,-1) + us = us.reshape(-1).float().cuda()[None].expand(bs,-1) + zs = depths.reshape(bs,-1) + Ks = Ks[:,None].expand(bs,zs.shape[-1],3,3) + xs = (us-Ks[...,0,2])*zs/Ks[...,0,0] #(B,N) + ys = (vs-Ks[...,1,2])*zs/Ks[...,1,1] + pts = torch.stack([xs,ys,zs], dim=-1) #(B,N,3) + xyz_maps = pts.reshape(bs,H,W,3) + xyz_maps[invalid_mask] = 0 + return xyz_maps + + + +def rle_to_mask(rle: dict) -> np.ndarray: + """Compute a binary mask from an uncompressed RLE.""" + h, w = rle["size"] + mask = np.empty(h * w, dtype=bool) + idx = 0 + parity = False + for count in rle["counts"]: + mask[idx : idx + count] = parity + idx += count + parity ^= True + mask = mask.reshape(w, h) + return mask.transpose() # Put in C order + + +def depth_to_vis(depth, zmin=None, zmax=None, mode='rgb', inverse=True): + if zmin is None: + zmin = depth.min() + if zmax is None: + zmax = depth.max() + + if inverse: + invalid = depth<0.001 + vis = zmin/(depth+1e-8) + vis[invalid] = 0 + else: + depth = depth.clip(zmin, zmax) + invalid = (depth==zmin) | (depth==zmax) + vis = (depth-zmin)/(zmax-zmin) + vis[invalid] = 1 + + if mode=='gray': + vis = (vis*255).clip(0, 255).astype(np.uint8) + elif mode=='rgb': + vis = cv2.applyColorMap((vis*255).astype(np.uint8), cv2.COLORMAP_JET)[...,::-1] + else: + raise RuntimeError + + return vis + + + +def sample_views_icosphere(n_views, subdivisions=None, radius=1): + if subdivisions is not None: + mesh = trimesh.creation.icosphere(subdivisions=subdivisions, radius=radius) + else: + subdivision = 1 + while 1: + mesh = trimesh.creation.icosphere(subdivisions=subdivision, radius=radius) + if mesh.vertices.shape[0]>=n_views: + break + subdivision += 1 + cam_in_obs = np.tile(np.eye(4)[None], (len(mesh.vertices),1,1)) + cam_in_obs[:,:3,3] = mesh.vertices + up = np.array([0,0,1]) + z_axis = -cam_in_obs[:,:3,3] #(N,3) + z_axis /= np.linalg.norm(z_axis, axis=-1).reshape(-1,1) + x_axis = np.cross(up.reshape(1,3), z_axis) + invalid = (x_axis==0).all(axis=-1) + x_axis[invalid] = [1,0,0] + x_axis /= np.linalg.norm(x_axis, axis=-1).reshape(-1,1) + y_axis = np.cross(z_axis, x_axis) + y_axis /= np.linalg.norm(y_axis, axis=-1).reshape(-1,1) + cam_in_obs[:,:3,0] = x_axis + cam_in_obs[:,:3,1] = y_axis + cam_in_obs[:,:3,2] = z_axis + return cam_in_obs + + + +def to_homo(pts): + ''' + @pts: (N,3 or 2) will homogeneliaze the last dimension + ''' + assert len(pts.shape)==2, f'pts.shape: {pts.shape}' + homo = np.concatenate((pts, np.ones((pts.shape[0],1))),axis=-1) + return homo + + +def to_homo_torch(pts): + ''' + @pts: shape can be (...,N,3 or 2) or (N,3) will homogeneliaze the last dimension + ''' + ones = torch.ones((*pts.shape[:-1],1), dtype=torch.float, device=pts.device) + homo = torch.cat((pts, ones),dim=-1) + return homo + + +def transform_pts(pts,tf): + """Transform 2d or 3d points + @pts: (...,N_pts,3) + @tf: (...,4,4) + """ + if len(tf.shape)>=3 and tf.shape[-3]!=pts.shape[-2]: + tf = tf[...,None,:,:] + return (tf[...,:-1,:-1]@pts[...,None] + tf[...,:-1,-1:])[...,0] + + +def transform_dirs(dirs,tf): + """ + @dirs: (...,3) + @tf: (...,4,4) + """ + if len(tf.shape)>=3 and tf.shape[-3]!=dirs.shape[-2]: + tf = tf[...,None,:,:] + return (tf[...,:3,:3]@dirs[...,None])[...,0] + + + +def random_direction(): + '''https://stackoverflow.com/questions/33976911/generate-a-random-sample-of-points-distributed-on-the-surface-of-a-unit-sphere + ''' + vec = np.random.randn(3).reshape(3) + vec /= np.linalg.norm(vec) + return vec + + + +def compute_mesh_diameter(model_pts=None, mesh=None, n_sample=1000): + from sklearn.decomposition import TruncatedSVD + if mesh is not None: + u, s, vh = scipy.linalg.svd(mesh.vertices, full_matrices=False) + pts = u@s + diameter = np.linalg.norm(pts.max(axis=0)-pts.min(axis=0)) + return float(diameter) + + if n_sample is None: + pts = model_pts + else: + ids = np.random.choice(len(model_pts), size=min(n_sample, len(model_pts)), replace=False) + pts = model_pts[ids] + dists = np.linalg.norm(pts[None]-pts[:,None], axis=-1) + diameter = dists.max() + return diameter + + +def compute_crop_window_tf_batch(pts=None, H=None, W=None, poses=None, K=None, crop_ratio=1.2, out_size=None, rgb=None, uvs=None, method='min_box', mesh_diameter=None): + '''Project the points and find the cropping transform + @pts: (N,3) + @poses: (B,4,4) tensor + @min_box: min_box/min_circle + @scale: scale to apply to the tightly enclosing roi + ''' + def compute_tf_batch(left, right, top, bottom): + B = len(left) + left = left.round() + right = right.round() + top = top.round() + bottom = bottom.round() + + tf = torch.eye(3)[None].expand(B,-1,-1).contiguous() + tf[:,0,2] = -left + tf[:,1,2] = -top + new_tf = torch.eye(3)[None].expand(B,-1,-1).contiguous() + new_tf[:,0,0] = out_size[0]/(right-left) + new_tf[:,1,1] = out_size[1]/(bottom-top) + tf = new_tf@tf + return tf + + B = len(poses) + torch.set_default_tensor_type('torch.cuda.FloatTensor') + if method=='box_3d': + radius = mesh_diameter*crop_ratio/2 + offsets = torch.tensor([0,0,0, + radius,0,0, + -radius,0,0, + 0,radius,0, + 0,-radius,0]).reshape(-1,3) + pts = poses[:,:3,3].reshape(-1,1,3)+offsets.reshape(1,-1,3) + K = torch.as_tensor(K) + projected = (K@pts.reshape(-1,3).T).T + uvs = projected[:,:2]/projected[:,2:3] + uvs = uvs.reshape(B, -1, 2) + center = uvs[:,0] #(B,2) + radius = torch.abs(uvs-center.reshape(-1,1,2)).reshape(B,-1).max(axis=-1)[0].reshape(-1) #(B) + left = center[:,0]-radius + right = center[:,0]+radius + top = center[:,1]-radius + bottom = center[:,1]+radius + tfs = compute_tf_batch(left, right, top, bottom) + return tfs + + else: + raise RuntimeError + + return tf + + + +def cv_draw_text(img,text,uv_top_left,color=(255, 255, 255),fontScale=0.5,thickness=1,fontFace=cv2.FONT_HERSHEY_SIMPLEX,outline_color=None,line_spacing=1.5): + H,W = img.shape[:2] + uv_top_left = np.array(uv_top_left, dtype=float) + assert uv_top_left.shape == (2,) + + for line in text.splitlines(): + (w, h), _ = cv2.getTextSize(text=line,fontFace=fontFace,fontScale=fontScale,thickness=thickness,) + uv_bottom_left_i = uv_top_left + [0, h] + + ############# Ensure inside image + while uv_bottom_left_i[0]<0: + uv_bottom_left_i[0] += 1 + while uv_bottom_left_i[0]+w>=W: + uv_bottom_left_i[0] -= 1 + while uv_bottom_left_i[1]>=H: + uv_bottom_left_i[1] -= 1 + while uv_bottom_left_i[1]-h<0: + uv_bottom_left_i[1] += 1 + + org = tuple(uv_bottom_left_i.astype(int)) + + if outline_color is not None: + cv2.putText(img,text=line,org=org,fontFace=fontFace,fontScale=fontScale,color=outline_color,thickness=thickness,lineType=cv2.LINE_AA,) + cv2.putText(img,text=line,org=org,fontFace=fontFace,fontScale=fontScale,color=color,thickness=thickness,lineType=cv2.LINE_AA,) + uv_top_left[1] = uv_bottom_left_i[1]-h+h*line_spacing + return img + + +def trimesh_add_pure_colored_texture(mesh, color=np.array([255,255,255]), resolution=5): + tex_img = np.tile(color.reshape(1,1,3), (resolution, resolution, 1)).astype(np.uint8) + mesh = mesh.unwrap() + mesh.visual = trimesh.visual.texture.TextureVisuals(uv=mesh.visual.uv,image=Image.fromarray(tex_img)) + return mesh + + + + +def project_3d_to_2d(pt,K,ob_in_cam): + pt = pt.reshape(4,1) + projected = K @ ((ob_in_cam@pt)[:3,:]) + projected = projected.reshape(-1) + projected = projected/projected[2] + return projected.reshape(-1)[:2].round().astype(int) + + +def draw_xyz_axis(color, ob_in_cam, scale=0.1, K=np.eye(3), thickness=3, transparency=0,is_input_rgb=False): + ''' + @color: BGR + ''' + if is_input_rgb: + color = cv2.cvtColor(color,cv2.COLOR_RGB2BGR) + xx = np.array([1,0,0,1]).astype(float) + yy = np.array([0,1,0,1]).astype(float) + zz = np.array([0,0,1,1]).astype(float) + xx[:3] = xx[:3]*scale + yy[:3] = yy[:3]*scale + zz[:3] = zz[:3]*scale + origin = tuple(project_3d_to_2d(np.array([0,0,0,1]), K, ob_in_cam)) + xx = tuple(project_3d_to_2d(xx, K, ob_in_cam)) + yy = tuple(project_3d_to_2d(yy, K, ob_in_cam)) + zz = tuple(project_3d_to_2d(zz, K, ob_in_cam)) + line_type = cv2.LINE_AA + arrow_len = 0 + tmp = color.copy() + tmp1 = tmp.copy() + tmp1 = cv2.arrowedLine(tmp1, origin, xx, color=(0,0,255), thickness=thickness,line_type=line_type, tipLength=arrow_len) + mask = np.linalg.norm(tmp1-tmp, axis=-1)>0 + tmp[mask] = tmp[mask]*transparency + tmp1[mask]*(1-transparency) + tmp1 = tmp.copy() + tmp1 = cv2.arrowedLine(tmp1, origin, yy, color=(0,255,0), thickness=thickness,line_type=line_type, tipLength=arrow_len) + mask = np.linalg.norm(tmp1-tmp, axis=-1)>0 + tmp[mask] = tmp[mask]*transparency + tmp1[mask]*(1-transparency) + tmp1 = tmp.copy() + tmp1 = cv2.arrowedLine(tmp1, origin, zz, color=(255,0,0), thickness=thickness,line_type=line_type, tipLength=arrow_len) + mask = np.linalg.norm(tmp1-tmp, axis=-1)>0 + tmp[mask] = tmp[mask]*transparency + tmp1[mask]*(1-transparency) + tmp = tmp.astype(np.uint8) + if is_input_rgb: + tmp = cv2.cvtColor(tmp,cv2.COLOR_BGR2RGB) + + return tmp + + +def draw_posed_3d_box(K, img, ob_in_cam, bbox, line_color=(0,255,0), linewidth=2): + '''Revised from 6pack dataset/inference_dataset_nocs.py::projection + @bbox: (2,3) min/max + @line_color: RGB + ''' + min_xyz = bbox.min(axis=0) + xmin, ymin, zmin = min_xyz + max_xyz = bbox.max(axis=0) + xmax, ymax, zmax = max_xyz + + def draw_line3d(start,end,img): + pts = np.stack((start,end),axis=0).reshape(-1,3) + pts = (ob_in_cam@to_homo(pts).T).T[:,:3] #(2,3) + projected = (K@pts.T).T + uv = np.round(projected[:,:2]/projected[:,2].reshape(-1,1)).astype(int) #(2,2) + img = cv2.line(img, uv[0].tolist(), uv[1].tolist(), color=line_color, thickness=linewidth, lineType=cv2.LINE_AA) + return img + + for y in [ymin,ymax]: + for z in [zmin,zmax]: + start = np.array([xmin,y,z]) + end = start+np.array([xmax-xmin,0,0]) + img = draw_line3d(start,end,img) + + for x in [xmin,xmax]: + for z in [zmin,zmax]: + start = np.array([x,ymin,z]) + end = start+np.array([0,ymax-ymin,0]) + img = draw_line3d(start,end,img) + + for x in [xmin,xmax]: + for y in [ymin,ymax]: + start = np.array([x,y,zmin]) + end = start+np.array([0,0,zmax-zmin]) + img = draw_line3d(start,end,img) + + return img + + +def projection_matrix_from_intrinsics(K, height, width, znear, zfar, window_coords='y_down'): + """Conversion of Hartley-Zisserman intrinsic matrix to OpenGL proj. matrix. + + Ref: + 1) https://strawlab.org/2011/11/05/augmented-reality-with-OpenGL + 2) https://github.com/strawlab/opengl-hz/blob/master/src/calib_test_utils.py + + :param K: 3x3 ndarray with the intrinsic camera matrix. + :param x0 The X coordinate of the camera image origin (typically 0). + :param y0: The Y coordinate of the camera image origin (typically 0). + :param w: Image width. + :param h: Image height. + :param nc: Near clipping plane. + :param fc: Far clipping plane. + :param window_coords: 'y_up' or 'y_down'. + :return: 4x4 ndarray with the OpenGL projection matrix. + """ + x0 = 0 + y0 = 0 + w = width + h = height + nc = znear + fc = zfar + + depth = float(fc - nc) + q = -(fc + nc) / depth + qn = -2 * (fc * nc) / depth + + # Draw our images upside down, so that all the pixel-based coordinate + # systems are the same. + if window_coords == 'y_up': + proj = np.array([ + [2 * K[0, 0] / w, -2 * K[0, 1] / w, (-2 * K[0, 2] + w + 2 * x0) / w, 0], + [0, -2 * K[1, 1] / h, (-2 * K[1, 2] + h + 2 * y0) / h, 0], + [0, 0, q, qn], # Sets near and far planes (glPerspective). + [0, 0, -1, 0] + ]) + + # Draw the images upright and modify the projection matrix so that OpenGL + # will generate window coords that compensate for the flipped image coords. + elif window_coords == 'y_down': + proj = np.array([ + [2 * K[0, 0] / w, -2 * K[0, 1] / w, (-2 * K[0, 2] + w + 2 * x0) / w, 0], + [0, 2 * K[1, 1] / h, (2 * K[1, 2] - h + 2 * y0) / h, 0], + [0, 0, q, qn], # Sets near and far planes (glPerspective). + [0, 0, -1, 0] + ]) + else: + raise NotImplementedError + + return proj + + + +def symmetry_tfs_from_info(info, rot_angle_discrete=5): + symmetry_tfs = [np.eye(4)] + if 'symmetries_discrete' in info: + tfs = np.array(info['symmetries_discrete']).reshape(-1,4,4) + tfs[...,:3,3] *= 0.001 + symmetry_tfs = [np.eye(4)] + symmetry_tfs += list(tfs) + if 'symmetries_continuous' in info: + axis = np.array(info['symmetries_continuous'][0]['axis']).reshape(3) + offset = info['symmetries_continuous'][0]['offset'] + rxs = [0] + rys = [0] + rzs = [0] + if axis[0]>0: + rxs = np.arange(0,360,rot_angle_discrete)/180.0*np.pi + elif axis[1]>0: + rys = np.arange(0,360,rot_angle_discrete)/180.0*np.pi + elif axis[2]>0: + rzs = np.arange(0,360,rot_angle_discrete)/180.0*np.pi + for rx in rxs: + for ry in rys: + for rz in rzs: + tf = euler_matrix(rx, ry, rz) + tf[:3,3] = offset + symmetry_tfs.append(tf) + if len(symmetry_tfs)==0: + symmetry_tfs = [np.eye(4)] + symmetry_tfs = np.array(symmetry_tfs) + return symmetry_tfs + + + +def pose_to_egocentric_delta_pose(A_in_cam, B_in_cam): + '''Used for Pose Refinement. Given the object's two poses in camera, convert them to relative poses in camera's egocentric view + @A_in_cam: (B,4,4) torch tensor + ''' + trans_delta = B_in_cam[:,:3,3] - A_in_cam[:,:3,3] + rot_mat_delta = B_in_cam[:,:3,:3]@A_in_cam[:,:3,:3].permute(0,2,1) + return trans_delta, rot_mat_delta + + + +def egocentric_delta_pose_to_pose(A_in_cam, trans_delta, rot_mat_delta): + '''Used for Pose Refinement. Given the object's two poses in camera, convert them to relative poses in camera's egocentric view + @A_in_cam: (B,4,4) torch tensor + ''' + B_in_cam = torch.eye(4, dtype=torch.float, device=A_in_cam.device)[None].expand(len(A_in_cam),-1,-1).contiguous() + B_in_cam[:,:3,3] = A_in_cam[:,:3,3]+trans_delta + B_in_cam[:,:3,:3] = rot_mat_delta@A_in_cam[:,:3,:3] + return B_in_cam + + +def sdg_load_bounding_box(file_path: str): + """Load bounding boxes. + Args: + file_path: Path of the bounding box. + + Returns: + A dictionary of the bounding boxes. + """ + bbox_dict = {} + bbox_array = np.load(file_path) + for id, x_min, y_min, x_max, y_max, occlusion_ratio in zip( + bbox_array["semanticId"], + bbox_array["x_min"], + bbox_array["y_min"], + bbox_array["x_max"], + bbox_array["y_max"], + bbox_array["occlusionRatio"], + ): + bbox_dict[id] = { + "x_min": x_min, + "y_min": y_min, + "x_max": x_max, + "y_max": y_max, + "occlusion_ratio": occlusion_ratio, + } + return bbox_dict + + +def texture_map_interpolation(tex_image_numpy): + all_channels = [] + mask = np.all(tex_image_numpy == 0, axis=2) + x = np.arange(0, tex_image_numpy.shape[1]) + y = np.arange(0, tex_image_numpy.shape[0]) + xx, yy = np.meshgrid(x, y) + for each_channel in range(tex_image_numpy.shape[2]): + curr_channel = tex_image_numpy[:,:,each_channel] + x1 = xx[~mask] + y1 = yy[~mask] + newarr = curr_channel[~mask] + GD1 = griddata((x1, y1), newarr.ravel(), (xx, yy), method='nearest') + all_channels.append(GD1[:,:,np.newaxis].round().astype(np.uint8)) + final_image = np.concatenate(all_channels, axis =-1) + return final_image + + + +class OctreeManager: + def __init__(self,pts=None,max_level=None,octree=None): + import kaolin + if octree is None: + pts_quantized = kaolin.ops.spc.quantize_points(pts.contiguous(), level=max_level) + self.octree = kaolin.ops.spc.unbatched_points_to_octree(pts_quantized, max_level, sorted=False) + else: + self.octree = octree + lengths = torch.tensor([len(self.octree)], dtype=torch.int32).cpu() + self.max_level, self.pyramids, self.exsum = kaolin.ops.spc.scan_octrees(self.octree,lengths) + self.finest_vox_size = 2.0/(2**self.max_level) + self.n_level = self.max_level+1 + self.vox_point_all_levels = kaolin.ops.spc.generate_points(self.octree, self.pyramids, self.exsum) + self.point_hierarchy_dual, self.pyramid_dual = kaolin.ops.spc.unbatched_make_dual(self.vox_point_all_levels, self.pyramids[0]) + self.trinkets, self.pointers_to_parent = kaolin.ops.spc.unbatched_make_trinkets(self.vox_point_all_levels, self.pyramids[0], self.point_hierarchy_dual, self.pyramid_dual) + self.n_vox = len(self.vox_point_all_levels) + self.n_corners = len(self.point_hierarchy_dual) + + for level in range(self.n_level): + vox_pts = self.get_level_quantized_points(level) + corner_pts = self.get_level_corner_quantized_points(level) + logging.info(f'level:{level}, vox_pts:{vox_pts.shape}, corner_pts:{corner_pts.shape}') + + def get_level_corner_quantized_points(self,level): + start = self.pyramid_dual[...,1,level] + num = self.pyramid_dual[...,0,level] + return self.point_hierarchy_dual[start:start+num] + + def get_level_quantized_points(self,level): + start = self.pyramids[...,1,level] + num = self.pyramids[...,0,level] + return self.vox_point_all_levels[start:start+num] + + + def get_center_ids(self,x,level): + '''Get ids with 0 starting from current level's first point + ''' + import kaolin + pidx = kaolin.ops.spc.unbatched_query(self.octree, self.exsum, x.float(), level, with_parents=False) + return pidx + + + def get_vox_size_at_level(self, level): + return 2.0/(2**level) + + + def draw(self,level, method='point'): + import kaolin + logging.info(f"level:{level}") + vox_size = self.get_vox_size_at_level(level) + + if method=='point': + corner_coords = self.get_level_corner_quantized_points(level) + pts = corner_coords*vox_size - 1 + mesh = trimesh.points.PointCloud(pts.data.cpu().numpy().reshape(-1,3)) + return mesh + + + def ray_trace(self,rays_o,rays_d,level,debug=False): + """Octree is in normalized [-1,1] world coordinate frame + 'rays_o': ray origin in normalized world coordinate system + 'rays_d': (N,3) unit length ray direction in normalized world coordinate system + 'octree': spc + @voxel_size: in the scale of [-1,1] space + Return: + ray_depths_in_out: traveling times, NOT the Z value; invalid will be zeros + """ + from mycuda import common + import kaolin + + ray_index, rays_pid, depth_in_out = kaolin.render.spc.unbatched_raytrace(self.octree,self.vox_point_all_levels,self.pyramids[0],self.exsum,rays_o,rays_d,level=level,return_depth=True,with_exit=True) + if ray_index.size()[0] == 0: + pdb.set_trace() + print("[WARNING] batch has 0 intersections!!") + ray_depths_in_out = torch.zeros((rays_o.shape[0],1,2)) + rays_pid = -torch.ones_like(rays_o[:, :1]) + rays_near = torch.zeros_like(rays_o[:, :1]) + rays_far = torch.zeros_like(rays_o[:, :1]) + return rays_near, rays_far, rays_pid, ray_depths_in_out + + intersected_ray_ids,counts = torch.unique_consecutive(ray_index,return_counts=True) + max_intersections = counts.max().item() + start_poss = torch.cat([torch.tensor([0], device=counts.device),torch.cumsum(counts[:-1],dim=0)],dim=0) + + ray_depths_in_out = common.postprocessOctreeRayTracing(ray_index.long().contiguous(),depth_in_out.contiguous(),intersected_ray_ids.long().contiguous(),start_poss.long().contiguous(), max_intersections, rays_o.shape[0]) + + rays_far = ray_depths_in_out[:,:,1].max(dim=-1)[0].reshape(-1,1) + rays_near = ray_depths_in_out[:,0,0].reshape(-1,1) + + return rays_near, rays_far, rays_pid, ray_depths_in_out + + +def make_yaml_dumpable(D): + if isinstance(D, np.ndarray): + return D.tolist() + for d in D: + if isinstance(D[d], dict) or isinstance(D[d], OrderedDict) or isinstance(D[d], defaultdict): + D[d] = dict(D[d]) + D[d] = make_yaml_dumpable(D[d]) + continue + if isinstance(D[d], np.ndarray): + D[d] = D[d].tolist() + continue + if np.issubdtype(type(D[d]), int): + D[d] = int(D[d]) + continue + if np.issubdtype(type(D[d]), float): + D[d] = float(D[d]) + continue + if np.issubdtype(type(D[d]), str): + D[d] = str(D[d]) + continue + if isinstance(D[d], list): + for i in range(len(D[d])): + D[d][i] = make_yaml_dumpable(D[d][i]) + continue + return dict(D) diff --git a/third_party/FoundationPose/build_all.sh b/third_party/FoundationPose/build_all.sh new file mode 100644 index 0000000..155fb45 --- /dev/null +++ b/third_party/FoundationPose/build_all.sh @@ -0,0 +1,8 @@ +DIR=$(pwd) + +cd $DIR/mycpp/ && mkdir -p build && cd build && cmake .. -DPYTHON_EXECUTABLE=$(which python) && make -j11 +cd /kaolin && rm -rf build *egg* && pip install -e . +# Optional: BundleSDF CUDA ops (required for model-free NeRF path only) +# cd $DIR/bundlesdf/mycuda && rm -rf build *egg* && pip install -e . + +cd "${DIR}" diff --git a/third_party/FoundationPose/build_all_conda.sh b/third_party/FoundationPose/build_all_conda.sh new file mode 100755 index 0000000..b1b59fb --- /dev/null +++ b/third_party/FoundationPose/build_all_conda.sh @@ -0,0 +1,23 @@ +#!/usr/bin/env bash +# Build native extensions for a local conda environment. +# BundleSDF CUDA ops (bundlesdf/mycuda) are omitted; model-free NeRF path needs them separately. +set -euo pipefail + +PROJ_ROOT="$(cd -- "$(dirname "${BASH_SOURCE[0]}")" &>/dev/null && pwd)" + +: "${CONDA_PREFIX:?Set CONDA_PREFIX by activating your conda env first.}" + +export CMAKE_PREFIX_PATH="${CONDA_PREFIX}:${CMAKE_PREFIX_PATH:-}" + +cd "${PROJ_ROOT}/mycpp" +rm -rf build +mkdir -p build +cd build +cmake .. \ + -DCMAKE_BUILD_TYPE=Release \ + -DCMAKE_PREFIX_PATH="${CMAKE_PREFIX_PATH}" \ + -DPython3_ROOT_DIR="${CONDA_PREFIX}" \ + -DPYBIND11_PYTHON_EXECUTABLE="${CONDA_PREFIX}/bin/python" +cmake --build . -j"$(nproc)" + +cd "${PROJ_ROOT}" diff --git a/third_party/FoundationPose/bundlesdf/config_linemod.yml b/third_party/FoundationPose/bundlesdf/config_linemod.yml new file mode 100644 index 0000000..d2cf07f --- /dev/null +++ b/third_party/FoundationPose/bundlesdf/config_linemod.yml @@ -0,0 +1,105 @@ +notes: '' +n_step: 1000 +netdepth: 8 +netwidth: 256 +netdepth_fine: 8 +netwidth_fine: 256 +N_rand: 2048 # Batch number of rays +first_frame_ray_in_batch: 0 +lrate: 0.01 +lrate_pose: 0.01 +pose_optimize_start: 0 +decay_rate: 0.1 +chunk: 99999999999 +netchunk: 6553600 +no_batching: 0 +amp: true + +N_samples: 128 #number of coarse samples per ray +N_samples_around_depth: 128 +N_importance: 0 +N_importance_iter: 1 +perturb: 1 +use_viewdirs: 1 +i_embed: 1 #set 1 for hashed embedding, 0 for default positional encoding, 2 for spherical; 3 for octree grid +i_embed_views: 2 #set 1 for hashed embedding, 0 for default positional encoding, 2 for spherical +multires: 8 #log2 of max freq for positional encoding (3D location) +multires_views: 3 #log2 of max freq for positional encoding (2D direction) +feature_grid_dim: 2 +raw_noise_std: 0 +white_bkgd: 0 +gradient_max_norm: 0.1 +gradient_pose_max_norm: 0.1 + +# logging/saving options +i_print: 500 +i_img: 500 +i_weights: 500 +i_mesh: 500 +i_nerf_normals: 500 +i_save_ray: 500 +i_pose: 500 +save_octree_clouds: True + +finest_res: 256 +base_res: 32 +num_levels: 16 +log2_hashmap_size: 22 +datadir: /mnt/9a72c439-d0a7-45e8-8d20-d7a235d02763/github/HashNeRF-pytorch/data/bundlesdf_bundlesdf_e03000196 +n_train_image: 300 +use_octree: 1 +first_frame_weight: 1 +denoise_depth_use_octree_cloud: true +octree_embed_base_voxel_size: 0.02 +octree_smallest_voxel_size: 0.02 # This determines the smallest feature vox size +octree_raytracing_voxel_size: 0.02 +octree_dilate_size: 0.02 # meters +down_scale_ratio: 1 +bounding_box: [[-1,-1,-1], [1,1,1]] +farthest_pose_sampling: 0 # Sampling train images. This replace uniform skip +use_mask: 1 +dilate_mask_size: 0 +rays_valid_depth_only: true +near: 0.1 +far: 2 +rgb_weight: 1 +depth_weight: 0 +trunc: 0.01 #length of the truncation region in meters +trunc_start: 0.01 +sdf_lambda: 5 +neg_trunc_ratio: 1 # -trunc distance ratio compared to +trunc +trunc_decay_type: '' +sdf_loss_type: l2 +fs_weight: 1000 +empty_weight: 1 +fs_rgb_weight: 0 +trunc_weight: 6000 +sparse_loss_weight: 0 +tv_loss_weight: 0 +frame_features: 2 #number of channels of the learnable per-frame features +optimize_poses: 1 #optimize a pose refinement for the initial poses +pose_reg_weight: 0 +point_cloud_loss_weight: 0 +point_cloud_loss_normal_weight: 0 +eikonal_weight: 0 +normal_loss_weight: 0 +feature_reg_weight: 0.1 +deformation_reg_weight: 0 +use_deformation_field: 0 #use a deformation field to account for inaccuracies in intrinsic parameters +max_pix_shift: 0.02 # Pixel shift ratio relative to image size +share_coarse_fine: 1 +mode: sdf +fs_sdf: 1 # Uncertain free space +crop: 0 +mesh_resolution: 0.003 +max_trans: 0.02 # meters +max_rot: 10 # deg + +continual: True + +######### dbscan +dbscan_eps: 0.01 +dbscan_eps_min_samples: 1 + +####### bundlenerf +sync_max_delay: 0 # 0 for strict sync \ No newline at end of file diff --git a/third_party/FoundationPose/bundlesdf/config_ycbv.yml b/third_party/FoundationPose/bundlesdf/config_ycbv.yml new file mode 100644 index 0000000..620c654 --- /dev/null +++ b/third_party/FoundationPose/bundlesdf/config_ycbv.yml @@ -0,0 +1,105 @@ +notes: '' +n_step: 1000 +netdepth: 8 +netwidth: 256 +netdepth_fine: 8 +netwidth_fine: 256 +N_rand: 2048 # Batch number of rays +first_frame_ray_in_batch: 0 +lrate: 0.01 +lrate_pose: 0.01 +pose_optimize_start: 0 +decay_rate: 0.1 +chunk: 99999999999 +netchunk: 6553600 +no_batching: 0 +amp: true + +N_samples: 128 #number of coarse samples per ray +N_samples_around_depth: 128 +N_importance: 0 +N_importance_iter: 1 +perturb: 1 +use_viewdirs: 1 +i_embed: 1 #set 1 for hashed embedding, 0 for default positional encoding, 2 for spherical; 3 for octree grid +i_embed_views: 2 #set 1 for hashed embedding, 0 for default positional encoding, 2 for spherical +multires: 8 #log2 of max freq for positional encoding (3D location) +multires_views: 3 #log2 of max freq for positional encoding (2D direction) +feature_grid_dim: 2 +raw_noise_std: 0 +white_bkgd: 0 +gradient_max_norm: 0.1 +gradient_pose_max_norm: 0.1 + +# logging/saving options +i_print: 500 +i_img: 500 +i_weights: 500 +i_mesh: 500 +i_nerf_normals: 500 +i_save_ray: 500 +i_pose: 500 +save_octree_clouds: True + +finest_res: 512 +base_res: 32 +num_levels: 16 +log2_hashmap_size: 22 +datadir: /mnt/9a72c439-d0a7-45e8-8d20-d7a235d02763/github/HashNeRF-pytorch/data/bundlesdf_bundlesdf_e03000196 +n_train_image: 300 +use_octree: 1 +first_frame_weight: 1 +denoise_depth_use_octree_cloud: true +octree_embed_base_voxel_size: 0.02 +octree_smallest_voxel_size: 0.02 # This determines the smallest feature vox size +octree_raytracing_voxel_size: 0.02 +octree_dilate_size: 0.02 # meters +down_scale_ratio: 1 +bounding_box: [[-1,-1,-1], [1,1,1]] +farthest_pose_sampling: 0 # Sampling train images. This replace uniform skip +use_mask: 1 +dilate_mask_size: 0 +rays_valid_depth_only: true +near: 0.1 +far: 2 +rgb_weight: 100 +depth_weight: 0 +trunc: 0.01 #length of the truncation region in meters +trunc_start: 0.01 +sdf_lambda: 5 +neg_trunc_ratio: 1 # -trunc distance ratio compared to +trunc +trunc_decay_type: '' +sdf_loss_type: l2 +fs_weight: 100 +empty_weight: 1 +fs_rgb_weight: 0 +trunc_weight: 6000 +sparse_loss_weight: 0 +tv_loss_weight: 0 +frame_features: 2 #number of channels of the learnable per-frame features +optimize_poses: 1 #optimize a pose refinement for the initial poses +pose_reg_weight: 0 +point_cloud_loss_weight: 0 +point_cloud_loss_normal_weight: 0 +eikonal_weight: 0 +normal_loss_weight: 0 +feature_reg_weight: 0.1 +deformation_reg_weight: 0 +use_deformation_field: 0 #use a deformation field to account for inaccuracies in intrinsic parameters +max_pix_shift: 0.02 # Pixel shift ratio relative to image size +share_coarse_fine: 1 +mode: sdf +fs_sdf: 1 # Uncertain free space +crop: 0 +mesh_resolution: 0.003 +max_trans: 0.02 # meters +max_rot: 10 # deg + +continual: True + +######### dbscan +dbscan_eps: 0.01 +dbscan_eps_min_samples: 1 + +####### bundlenerf +sync_max_delay: 0 # 0 for strict sync \ No newline at end of file diff --git a/third_party/FoundationPose/bundlesdf/mycuda/__init__.py b/third_party/FoundationPose/bundlesdf/mycuda/__init__.py new file mode 100644 index 0000000..e69de29 diff --git a/third_party/FoundationPose/bundlesdf/mycuda/bindings.cpp b/third_party/FoundationPose/bundlesdf/mycuda/bindings.cpp new file mode 100644 index 0000000..605ffb4 --- /dev/null +++ b/third_party/FoundationPose/bundlesdf/mycuda/bindings.cpp @@ -0,0 +1,20 @@ +/* + * Copyright (c) 2023, NVIDIA CORPORATION. All rights reserved. + * + * NVIDIA CORPORATION and its licensors retain all intellectual property + * and proprietary rights in and to this software, related documentation + * and any modifications thereto. Any use, reproduction, disclosure or + * distribution of this software and related documentation without an express + * license agreement from NVIDIA CORPORATION is strictly prohibited. + */ + + +#include +#include "common.h" + +PYBIND11_MODULE(TORCH_EXTENSION_NAME, m) { + m.def("sampleRaysUniformOccupiedVoxels", &sampleRaysUniformOccupiedVoxels); + m.def("postprocessOctreeRayTracing", &postprocessOctreeRayTracing); + m.def("rayColorToTextureImageCUDA", &rayColorToTextureImageCUDA); + +} \ No newline at end of file diff --git a/third_party/FoundationPose/bundlesdf/mycuda/common.cu b/third_party/FoundationPose/bundlesdf/mycuda/common.cu new file mode 100644 index 0000000..97974d2 --- /dev/null +++ b/third_party/FoundationPose/bundlesdf/mycuda/common.cu @@ -0,0 +1,313 @@ +/* + * Copyright (c) 2023, NVIDIA CORPORATION. All rights reserved. + * + * NVIDIA CORPORATION and its licensors retain all intellectual property + * and proprietary rights in and to this software, related documentation + * and any modifications thereto. Any use, reproduction, disclosure or + * distribution of this software and related documentation without an express + * license agreement from NVIDIA CORPORATION is strictly prohibited. + */ + + + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include "common.h" +#include "Eigen/Dense" + + + + + +/** + * @brief + * + * @tparam scalar_t + * @param z_sampled + * @param z_in_out + * @param z_vals + * @return __global__ + */ +template +__global__ void sample_rays_uniform_occupied_voxels_kernel(const torch::PackedTensorAccessor32 z_sampled, const torch::PackedTensorAccessor32 z_in_out, torch::PackedTensorAccessor32 z_vals) +{ + const int i_ray = blockIdx.x * blockDim.x + threadIdx.x; + const int i_sample = blockIdx.y * blockDim.y + threadIdx.y; + if (i_ray>=z_sampled.size(0)) return; + if (i_sample>=z_sampled.size(1)) return; + + int i_box = 0; + float z_remain = z_sampled[i_ray][i_sample]; + auto z_in_out_cur_ray = z_in_out[i_ray]; + const float eps = 1e-4; + const int max_n_box = z_in_out.size(1); + + if (z_in_out_cur_ray[0][0]==0) return; + + while (1) + { + if (i_box>=max_n_box) + { + if (z_remain<=eps) + { + z_vals[i_ray][i_sample] = z_in_out_cur_ray[max_n_box-1][1]; + } + else + { + printf("ERROR sample_rays_uniform_occupied_voxels_kernel: z_remain=%f, i_ray=%d, i_sample=%d, i_box=%d, z_in_out_cur_ray=(%f,%f)\n",z_remain,i_ray,i_sample,i_box,z_in_out_cur_ray[i_box][0],z_in_out_cur_ray[i_box][1]); + for (int i=0;i=1) + { + z_vals[i_ray][i_sample] = z_in_out_cur_ray[i_box-1][1]; + return; + } + else + { + printf("ERROR sample_rays_uniform_occupied_voxels_kernel: z_remain=%f, i_ray=%d, i_sample=%d, i_box=%d, z_in_out_cur_ray=(%f,%f)\n",z_remain,i_ray,i_sample,i_box,z_in_out_cur_ray[i_box][0],z_in_out_cur_ray[i_box][1]); + for (int i=0;i<<<{divCeil(N_rays,threadx),divCeil(N_samples,thready)}, {threadx,thready}>>>(z_sampled.packed_accessor32(),z_in_out.packed_accessor32(),z_vals.packed_accessor32()); + })); + + return z_vals; +} + +template +__global__ void postprocessOctreeRayTracingKernel(const torch::PackedTensorAccessor32 ray_index, const torch::PackedTensorAccessor32 depth_in_out, const torch::PackedTensorAccessor32 unique_intersect_ray_ids, const torch::PackedTensorAccessor32 start_poss, torch::PackedTensorAccessor32 depths_in_out_padded) +{ + const int unique_id_pos = blockIdx.x * blockDim.x + threadIdx.x; + if (unique_id_pos>=unique_intersect_ray_ids.size(0)) return; + const int i_ray = unique_intersect_ray_ids[unique_id_pos]; + + int i_intersect = 0; + auto cur_depths_in_out_padded = depths_in_out_padded[i_ray]; + for (int i=start_poss[unique_id_pos];idepth_in_out[i][1]) continue; + if (abs(depth_in_out[i][1]-depth_in_out[i][0])<1e-4) continue; + + cur_depths_in_out_padded[i_intersect][0] = depth_in_out[i][0]; + cur_depths_in_out_padded[i_intersect][1] = depth_in_out[i][1]; + + i_intersect++; + + } +} + +at::Tensor postprocessOctreeRayTracing(const at::Tensor ray_index, const at::Tensor depth_in_out, const at::Tensor unique_intersect_ray_ids, const at::Tensor start_poss, const int max_intersections, const int N_rays) +{ + CHECK_INPUT(ray_index); + CHECK_INPUT(depth_in_out); + CHECK_INPUT(start_poss); + + const int n_unique_ids = unique_intersect_ray_ids.sizes()[0]; + at::Tensor depths_in_out_padded = at::zeros({N_rays,max_intersections,2}, torch::TensorOptions().dtype(torch::kFloat32).device(torch::kCUDA, 0).requires_grad(false)); + dim3 threads = {256}; + dim3 blocks = {divCeil(n_unique_ids,threads.x)}; + AT_DISPATCH_FLOATING_TYPES(depth_in_out.type(), "postprocessOctreeRayTracingKernel", ([&] + { + postprocessOctreeRayTracingKernel<<>>(ray_index.packed_accessor32(), depth_in_out.packed_accessor32(), unique_intersect_ray_ids.packed_accessor32(), start_poss.packed_accessor32(), depths_in_out_padded.packed_accessor32()); + })); + + return depths_in_out_padded; +} + + +at::Tensor calculateBarycentricCoordinate3D(const at::Tensor &triangle, const at::Tensor &p) +{ + auto vector_A = triangle[1]-triangle[2]; + auto vector_B = triangle[1]-triangle[0]; + auto normal = vector_A.cross(vector_B); + auto areaABC = (normal * at::cross(triangle[1]-triangle[0], triangle[2]-triangle[0])).sum(); + auto areaPBC = (normal * at::cross(triangle[1]-p, triangle[2]-p)).sum(); + auto areaPCA = (normal * at::cross(triangle[2]-p, triangle[0]-p)).sum(); + at::Tensor ws = at::zeros({3}).to(torch::kFloat32); + ws[0] = areaPBC / areaABC; + ws[1] = areaPCA / areaABC; + ws[2] = 1-ws[0]-ws[1]; + return ws; +} + + +__device__ Eigen::Vector3f calculateBarycentricCoordinate3DKernel(const Eigen::Matrix &triangle, const Eigen::Vector3f &p) +{ + Eigen::Vector3f vector_A = triangle.row(1)-triangle.row(2); + Eigen::Vector3f vector_B = triangle.row(1)-triangle.row(0); + Eigen::Vector3f normal = vector_A.cross(vector_B); + float areaABC = (normal.array() * (triangle.row(1)-triangle.row(0)).cross(triangle.row(2)-triangle.row(0)).transpose().array()).sum(); + float areaPBC = (normal.array() * (triangle.row(1).transpose()-p).cross(triangle.row(2).transpose()-p).array()).sum(); + float areaPCA = (normal.array() * (triangle.row(2).transpose()-p).cross(triangle.row(0).transpose()-p).array()).sum(); + Eigen::Vector3f w; + w[0] = areaPBC / areaABC; + w[1] = areaPCA / areaABC; + w[2] = 1-w[0]-w[1]; + return w; + +} + + +__device__ void calculateBarycentricCoordinate2DKernel(const Eigen::Matrix &triangle, const Eigen::Vector2f &p, Eigen::Vector3f &w) +{ + Eigen::Vector2f CA = triangle.row(0)-triangle.row(2); + Eigen::Vector2f AC = -CA; + Eigen::Vector2f CP = p-triangle.row(2).transpose(); + Eigen::Vector2f AB = triangle.row(1)-triangle.row(0); + Eigen::Vector2f AP = p-triangle.row(0).transpose(); + Eigen::Matrix2f numerator, denominator; + numerator << CA, CP; + denominator< +__global__ void rayColorToTextureImageKernel(const torch::PackedTensorAccessor32 F, const torch::PackedTensorAccessor32 V, const torch::PackedTensorAccessor32 hit_locations, const torch::PackedTensorAccessor32 hit_face_ids, const torch::PackedTensorAccessor32 uvs_tex, torch::PackedTensorAccessor32 uvs) +{ + const int i_hit = blockIdx.x*blockDim.x + threadIdx.x; + if (i_hit>=hit_locations.size(0)) return; + + auto face = F[hit_face_ids[i_hit]]; + Eigen::Matrix3f tri_v = Eigen::Matrix3f::Zero(); + for (int r=0;r<3;r++) + { + for (int c=0;c<3;c++) + { + tri_v(r,c) = V[face[r]][c]; + } + } + auto location = hit_locations[i_hit]; + Eigen::Vector3f p(location[0], location[1], location[2]); + auto w = calculateBarycentricCoordinate3DKernel(tri_v, p); + + Eigen::Matrix tri_uvs; + for (int i=0;i<3;i++) + { + for (int j=0;j<2;j++) + { + tri_uvs(i,j) = uvs_tex[face[i]][j]; + } + } + + Eigen::Vector2f cur_uv = tri_uvs.transpose() * w; + uvs[i_hit][0] = cur_uv(0); + uvs[i_hit][1] = cur_uv(1); +} + + + +void rayColorToTextureImageCUDA(const at::Tensor &F, const at::Tensor &V, const at::Tensor &hit_locations, const at::Tensor &hit_face_ids, const at::Tensor &uvs_tex, at::Tensor &uvs) +{ + CHECK_CONTIGUOUS(F); + CHECK_CONTIGUOUS(V); + CHECK_CONTIGUOUS(hit_locations); + CHECK_CONTIGUOUS(hit_face_ids); + CHECK_CONTIGUOUS(uvs_tex); + + dim3 threads = {512}; + dim3 blocks = {divCeil(int(hit_locations.sizes()[0]),threads.x)}; + + AT_DISPATCH_FLOATING_TYPES(V.type(), "rayColorToTextureImageKernel", ([&] + { + rayColorToTextureImageKernel<<>>(F.packed_accessor32(), V.packed_accessor32(), hit_locations.packed_accessor32(), hit_face_ids.packed_accessor32(), uvs_tex.packed_accessor32(), uvs.packed_accessor32()); + })); +} + + + +__device__ bool isLineIntersectLine(Eigen::Vector2f l1p1, Eigen::Vector2f l1p2, Eigen::Vector2f l2p1, Eigen::Vector2f l2p2) +{ + float q = (l1p1(1) - l2p1(1)) * (l2p2(0) - l2p1(0)) - (l1p1(0) - l2p1(0)) * (l2p2(1) - l2p1(1)); + float d = (l1p2(0) - l1p1(0)) * (l2p2(1) - l2p1(1)) - (l1p2(1) - l1p1(1)) * (l2p2(0) - l2p1(0)); + + if ( d == 0 ) + { + return false; + } + + float r = q / d; + + q = (l1p1(1) - l2p1(1)) * (l1p2(0) - l1p1(0)) - (l1p1(0) - l2p1(0)) * (l1p2(1) - l1p1(1)); + float s = q / d; + + if( r < 0 || r > 1 || s < 0 || s > 1 ) + { + return false; + } + + return true; +} + +__device__ bool isLineIntersectSquare(const Eigen::Vector2f &la, const Eigen::Vector2f &lb, const Eigen::Vector2f &p0, const Eigen::Vector2f &p1, const Eigen::Vector2f &p2, const Eigen::Vector2f &p3) +{ + return isLineIntersectLine(la, lb, p0, p1) || isLineIntersectLine(la, lb, p1, p2) || isLineIntersectLine(la, lb, p2, p3) || isLineIntersectLine(la, lb, p3, p0); +} + + +__device__ bool isPixelInsideTriangle(const Eigen::Matrix &triangle, Eigen::Vector2f p, Eigen::Vector3f &w) +{ + calculateBarycentricCoordinate2DKernel(triangle, p, w); + for (int j=0;j<3;j++) + { + if (w(j)<0) return false; + } + return true; +} diff --git a/third_party/FoundationPose/bundlesdf/mycuda/common.h b/third_party/FoundationPose/bundlesdf/mycuda/common.h new file mode 100644 index 0000000..77d98cb --- /dev/null +++ b/third_party/FoundationPose/bundlesdf/mycuda/common.h @@ -0,0 +1,30 @@ +/* + * Copyright (c) 2023, NVIDIA CORPORATION. All rights reserved. + * + * NVIDIA CORPORATION and its licensors retain all intellectual property + * and proprietary rights in and to this software, related documentation + * and any modifications thereto. Any use, reproduction, disclosure or + * distribution of this software and related documentation without an express + * license agreement from NVIDIA CORPORATION is strictly prohibited. + */ + + +#pragma once + +#include +#include + +#define CHECK_CUDA(x) AT_ASSERTM(x.is_cuda(), #x " must be a CUDA tensor") +#define CHECK_CONTIGUOUS(x) AT_ASSERTM(x.is_contiguous(), #x " must be contiguous") +#define CHECK_INPUT(x) CHECK_CUDA(x); CHECK_CONTIGUOUS(x) + + +inline int divCeil(int a, int b) +{ + return (a+b-1)/b; +}; + + +at::Tensor sampleRaysUniformOccupiedVoxels(const at::Tensor z_in_out, const at::Tensor z_sampled, at::Tensor z_vals); +at::Tensor postprocessOctreeRayTracing(const at::Tensor ray_index, const at::Tensor depth_in_out, const at::Tensor unique_intersect_ray_ids, const at::Tensor start_poss, const int max_intersections, const int N_rays); +void rayColorToTextureImageCUDA(const at::Tensor &F, const at::Tensor &V, const at::Tensor &hit_locations, const at::Tensor &hit_face_ids, const at::Tensor &uvs_tex, at::Tensor &uvs); \ No newline at end of file diff --git a/third_party/FoundationPose/bundlesdf/mycuda/setup.py b/third_party/FoundationPose/bundlesdf/mycuda/setup.py new file mode 100644 index 0000000..ad00334 --- /dev/null +++ b/third_party/FoundationPose/bundlesdf/mycuda/setup.py @@ -0,0 +1,41 @@ +# Copyright (c) 2023, NVIDIA CORPORATION. All rights reserved. +# +# NVIDIA CORPORATION and its licensors retain all intellectual property +# and proprietary rights in and to this software, related documentation +# and any modifications thereto. Any use, reproduction, disclosure or +# distribution of this software and related documentation without an express +# license agreement from NVIDIA CORPORATION is strictly prohibited. + + +from setuptools import setup +import os,sys +from torch.utils.cpp_extension import BuildExtension, CUDAExtension +from torch.utils.cpp_extension import load + +code_dir = os.path.dirname(os.path.realpath(__file__)) + + +nvcc_flags = ['-Xcompiler', '-O3', '-std=c++14', '-U__CUDA_NO_HALF_OPERATORS__', '-U__CUDA_NO_HALF_CONVERSIONS__', '-U__CUDA_NO_HALF2_OPERATORS__'] +c_flags = ['-O3', '-std=c++14'] + +setup( + name='common', + extra_cflags=c_flags, + extra_cuda_cflags=nvcc_flags, + ext_modules=[ + CUDAExtension('common', [ + 'bindings.cpp', + 'common.cu', + ],extra_compile_args={'gcc': c_flags, 'nvcc': nvcc_flags}), + CUDAExtension('gridencoder', [ + f"{code_dir}/torch_ngp_grid_encoder/gridencoder.cu", + f"{code_dir}/torch_ngp_grid_encoder/bindings.cpp", + ],extra_compile_args={'gcc': c_flags, 'nvcc': nvcc_flags}), + ], + include_dirs=[ + "/usr/local/include/eigen3", + "/usr/include/eigen3", + ], + cmdclass={ + 'build_ext': BuildExtension +}) diff --git a/third_party/FoundationPose/bundlesdf/mycuda/torch_ngp_grid_encoder/LICENSE b/third_party/FoundationPose/bundlesdf/mycuda/torch_ngp_grid_encoder/LICENSE new file mode 100644 index 0000000..56f524a --- /dev/null +++ b/third_party/FoundationPose/bundlesdf/mycuda/torch_ngp_grid_encoder/LICENSE @@ -0,0 +1,21 @@ +MIT License + +Copyright (c) 2022 hawkey + +Permission is hereby granted, free of charge, to any person obtaining a copy +of this software and associated documentation files (the "Software"), to deal +in the Software without restriction, including without limitation the rights +to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +copies of the Software, and to permit persons to whom the Software is +furnished to do so, subject to the following conditions: + +The above copyright notice and this permission notice shall be included in all +copies or substantial portions of the Software. + +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +SOFTWARE. \ No newline at end of file diff --git a/third_party/FoundationPose/bundlesdf/mycuda/torch_ngp_grid_encoder/bindings.cpp b/third_party/FoundationPose/bundlesdf/mycuda/torch_ngp_grid_encoder/bindings.cpp new file mode 100644 index 0000000..afa6f64 --- /dev/null +++ b/third_party/FoundationPose/bundlesdf/mycuda/torch_ngp_grid_encoder/bindings.cpp @@ -0,0 +1,8 @@ +#include + +#include "gridencoder.h" + +PYBIND11_MODULE(TORCH_EXTENSION_NAME, m) { + m.def("grid_encode_forward", &grid_encode_forward, "grid_encode_forward (CUDA)"); + m.def("grid_encode_backward", &grid_encode_backward, "grid_encode_backward (CUDA)"); +} \ No newline at end of file diff --git a/third_party/FoundationPose/bundlesdf/mycuda/torch_ngp_grid_encoder/grid.py b/third_party/FoundationPose/bundlesdf/mycuda/torch_ngp_grid_encoder/grid.py new file mode 100644 index 0000000..6573ce4 --- /dev/null +++ b/third_party/FoundationPose/bundlesdf/mycuda/torch_ngp_grid_encoder/grid.py @@ -0,0 +1,158 @@ +import numpy as np +import os,sys,pdb +code_dir = os.path.dirname(os.path.realpath(__file__)) +sys.path.append(code_dir) +import torch +import torch.nn as nn +from torch.autograd import Function +from torch.autograd.function import once_differentiable +from torch.cuda.amp import custom_bwd, custom_fwd +from setuptools import setup +from torch.utils.cpp_extension import BuildExtension, CUDAExtension +from torch.utils.cpp_extension import load +import os,sys +import gridencoder + + +_gridtype_to_id = { + 'hash': 0, + 'tiled': 1, +} + +class _grid_encode(Function): + @staticmethod + @custom_fwd + def forward(ctx, inputs, embeddings, offsets, per_level_scale, base_resolution, calc_grad_inputs=False, gridtype=0, align_corners=False): + # inputs: [B, D], float in [0, 1] + # embeddings: [sO, C], float + # offsets: [L + 1], int + # RETURN: [B, F], float + + inputs = inputs.contiguous() + + B, D = inputs.shape + L = offsets.shape[0] - 1 + C = embeddings.shape[1] + S = np.log2(per_level_scale) + H = base_resolution + + if torch.is_autocast_enabled() and C % 2 == 0: + embeddings = embeddings.to(torch.half) + + outputs = torch.empty(L, B, C, device=inputs.device, dtype=embeddings.dtype) + + if calc_grad_inputs: + dy_dx = torch.empty(B, L * D * C, device=inputs.device, dtype=embeddings.dtype) + else: + dy_dx = torch.empty(1, device=inputs.device, dtype=embeddings.dtype) # placeholder... TODO: a better way? + + gridencoder.grid_encode_forward(inputs, embeddings, offsets, outputs, B, D, C, L, S, H, calc_grad_inputs, dy_dx, gridtype, align_corners) + + outputs = outputs.permute(1, 0, 2).reshape(B, L * C) + + ctx.save_for_backward(inputs, embeddings, offsets, dy_dx) + ctx.dims = [B, D, C, L, S, H, gridtype] + ctx.calc_grad_inputs = calc_grad_inputs + ctx.align_corners = align_corners + + return outputs + + @staticmethod + @custom_bwd + def backward(ctx, grad): + + inputs, embeddings, offsets, dy_dx = ctx.saved_tensors + B, D, C, L, S, H, gridtype = ctx.dims + calc_grad_inputs = ctx.calc_grad_inputs + align_corners = ctx.align_corners + + # grad: [B, L * C] --> [L, B, C] + grad = grad.view(B, L, C).permute(1, 0, 2).contiguous() + + grad_embeddings = torch.zeros_like(embeddings) + + if calc_grad_inputs: + grad_inputs = torch.zeros_like(inputs, dtype=embeddings.dtype) + else: + grad_inputs = torch.zeros(1, device=inputs.device, dtype=embeddings.dtype) + + gridencoder.grid_encode_backward(grad, inputs, embeddings, offsets, grad_embeddings, B, D, C, L, S, H, calc_grad_inputs, dy_dx, grad_inputs, gridtype, align_corners) + + if calc_grad_inputs: + grad_inputs = grad_inputs.to(inputs.dtype) + return grad_inputs, grad_embeddings, None, None, None, None, None, None + else: + return None, grad_embeddings, None, None, None, None, None, None + + +grid_encode = _grid_encode.apply + + +# https://github.com/ashawkey/torch-ngp/blob/main/gridencoder/grid.py +class GridEncoder(nn.Module): + def __init__(self, input_dim=3, n_levels=16, level_dim=2, base_resolution=16, log2_hashmap_size=19, desired_resolution=None, gridtype='hash', align_corners=False): + super().__init__() + + per_level_scale = np.exp2(np.log2(desired_resolution / base_resolution) / (n_levels - 1)) + + self.input_dim = input_dim # coord dims, 2 or 3 + self.n_levels = n_levels # num levels, each level multiply resolution by 2 + self.level_dim = level_dim # encode channels per level + self.per_level_scale = per_level_scale # multiply resolution by this scale at each level. + self.log2_hashmap_size = log2_hashmap_size + self.base_resolution = base_resolution + self.out_dim = n_levels * level_dim + self.gridtype = gridtype + self.gridtype_id = _gridtype_to_id[gridtype] # "tiled" or "hash" + self.align_corners = align_corners + + # allocate parameters + offsets = [] + offset = 0 + self.max_params = 2 ** log2_hashmap_size + for i in range(n_levels): + resolution = int(np.ceil(base_resolution * per_level_scale ** i)) + params_in_level = min(self.max_params, (resolution if align_corners else resolution + 1) ** input_dim) # limit max number + params_in_level = int(np.ceil(params_in_level / 8) * 8) # make divisible + print(f"level {i}, resolution: {resolution}") + offsets.append(offset) + offset += params_in_level + offsets.append(offset) + offsets = torch.from_numpy(np.array(offsets, dtype=np.int32)) + self.register_buffer('offsets', offsets) + + self.n_params = offsets[-1] * level_dim + + # parameters + self.embeddings = nn.Parameter(torch.empty(offset, level_dim)) + # self.embeddings = nn.Embedding(offset, level_dim, sparse=True) + + self.reset_parameters() + + def reset_parameters(self): + std = 1e-4 + self.embeddings.data.uniform_(-std, std) + # self.embeddings.weight.data.uniform_(-std, std) + + + def __repr__(self): + return f"GridEncoder: input_dim={self.input_dim} n_levels={self.n_levels} level_dim={self.level_dim} resolution={self.base_resolution} -> {int(round(self.base_resolution * self.per_level_scale ** (self.n_levels - 1)))} per_level_scale={self.per_level_scale:.4f} params={tuple(self.embeddings.shape)} gridtype={self.gridtype} align_corners={self.align_corners}" + + + def forward(self, inputs, bound=1): + # inputs: [..., input_dim], normalized real world positions in [-bound, bound] + # return: [..., num_levels * level_dim] + + inputs = (inputs + bound) / (2 * bound) # map to [0, 1] + + #print('inputs', inputs.shape, inputs.dtype, inputs.min().item(), inputs.max().item()) + + prefix_shape = list(inputs.shape[:-1]) + inputs = inputs.view(-1, self.input_dim) + + outputs = grid_encode(inputs, self.embeddings, self.offsets, self.per_level_scale, self.base_resolution, inputs.requires_grad, self.gridtype_id, self.align_corners) + outputs = outputs.view(prefix_shape + [self.out_dim]) + + #print('outputs', outputs.shape, outputs.dtype, outputs.min().item(), outputs.max().item()) + + return outputs \ No newline at end of file diff --git a/third_party/FoundationPose/bundlesdf/mycuda/torch_ngp_grid_encoder/gridencoder.cu b/third_party/FoundationPose/bundlesdf/mycuda/torch_ngp_grid_encoder/gridencoder.cu new file mode 100644 index 0000000..6bb45c7 --- /dev/null +++ b/third_party/FoundationPose/bundlesdf/mycuda/torch_ngp_grid_encoder/gridencoder.cu @@ -0,0 +1,500 @@ +#include +#include +#include + +#include +#include + +#include +#include + +#include +#include + + +#define CHECK_CUDA(x) TORCH_CHECK(x.device().is_cuda(), #x " must be a CUDA tensor") +#define CHECK_CONTIGUOUS(x) TORCH_CHECK(x.is_contiguous(), #x " must be a contiguous tensor") +#define CHECK_IS_INT(x) TORCH_CHECK(x.scalar_type() == at::ScalarType::Int, #x " must be an int tensor") +#define CHECK_IS_FLOATING(x) TORCH_CHECK(x.scalar_type() == at::ScalarType::Float || x.scalar_type() == at::ScalarType::Half || x.scalar_type() == at::ScalarType::Double, #x " must be a floating tensor") + + +// just for compatability of half precision in AT_DISPATCH_FLOATING_TYPES_AND_HALF... +static inline __device__ at::Half atomicAdd(at::Half *address, at::Half val) { + // requires CUDA >= 10 and ARCH >= 70 + // this is very slow compared to float or __half2, and never used. + //return atomicAdd(reinterpret_cast<__half*>(address), val); +} + + +template +static inline __host__ __device__ T div_round_up(T val, T divisor) { + return (val + divisor - 1) / divisor; +} + + +template +__device__ uint32_t fast_hash(const uint32_t pos_grid[D]) { + static_assert(D <= 7, "fast_hash can only hash up to 7 dimensions."); + + // While 1 is technically not a good prime for hashing (or a prime at all), it helps memory coherence + // and is sufficient for our use case of obtaining a uniformly colliding index from high-dimensional + // coordinates. + constexpr uint32_t primes[7] = { 1, 2654435761, 805459861, 3674653429, 2097192037, 1434869437, 2165219737 }; + + uint32_t result = 0; + #pragma unroll + for (uint32_t i = 0; i < D; ++i) { + result ^= pos_grid[i] * primes[i]; + } + + return result; +} + + +template +__device__ uint32_t get_grid_index(const uint32_t gridtype, const bool align_corners, const uint32_t ch, const uint32_t hashmap_size, const uint32_t resolution, const uint32_t pos_grid[D]) { + uint32_t stride = 1; + uint32_t index = 0; + + #pragma unroll + for (uint32_t d = 0; d < D && stride <= hashmap_size; d++) { + index += pos_grid[d] * stride; + stride *= align_corners ? resolution: (resolution + 1); + } + + // NOTE: for NeRF, the hash is in fact not necessary. Check https://github.com/NVlabs/instant-ngp/issues/97. + // gridtype: 0 == hash, 1 == tiled + if (gridtype == 0 && stride > hashmap_size) { + index = fast_hash(pos_grid); + } + + return (index % hashmap_size) * C + ch; +} + + +/** + * @brief + * + * @tparam scalar_t + * @tparam D : point dimension usually 3D + * @tparam C : embedding dim 1/2/4 + * @param inputs + * @param grid + * @param offsets + * @param outputs + * @param B + * @param L + * @param S + * @param H + * @param calc_grad_inputs + * @param dy_dx + * @param gridtype + * @param align_corners + * @return __global__ + */ +template +__global__ void kernel_grid( + const float * __restrict__ inputs, + const scalar_t * __restrict__ grid, + const int * __restrict__ offsets, + scalar_t * __restrict__ outputs, + const uint32_t B, const uint32_t L, const float S, const uint32_t H, + const bool calc_grad_inputs, + scalar_t * __restrict__ dy_dx, + const uint32_t gridtype, + const bool align_corners +) { + const uint32_t b = blockIdx.x * blockDim.x + threadIdx.x; // batch id + + if (b >= B) return; + + const uint32_t level = blockIdx.y; + + // locate + grid += (uint32_t)offsets[level] * C; + inputs += b * D; + outputs += level * B * C + b * C; + + // check input range (should be in [0, 1]) + bool flag_oob = false; + #pragma unroll + for (uint32_t d = 0; d < D; d++) { + if (inputs[d] < 0 || inputs[d] > 1) { + flag_oob = true; + } + } + // if input out of bound, just set output to 0 + if (flag_oob) { + #pragma unroll + for (uint32_t ch = 0; ch < C; ch++) { + outputs[ch] = 0; + } + if (calc_grad_inputs) { + dy_dx += b * D * L * C + level * D * C; // B L D C + #pragma unroll + for (uint32_t d = 0; d < D; d++) { + #pragma unroll + for (uint32_t ch = 0; ch < C; ch++) { + dy_dx[d * C + ch] = 0; + } + } + } + return; + } + + const uint32_t hashmap_size = offsets[level + 1] - offsets[level]; + const float scale = exp2f(level * S) * H - 1.0f; + const uint32_t resolution = (uint32_t)ceil(scale) + 1; + + // calculate coordinate + float pos[D]; + uint32_t pos_grid[D]; + + #pragma unroll + for (uint32_t d = 0; d < D; d++) { + pos[d] = inputs[d] * scale + (align_corners ? 0.0f : 0.5f); + pos_grid[d] = floorf(pos[d]); + pos[d] -= (float)pos_grid[d]; + } + + //printf("[b=%d, l=%d] pos=(%f, %f)+(%d, %d)\n", b, level, pos[0], pos[1], pos_grid[0], pos_grid[1]); + + // interpolate + scalar_t results[C] = {0}; // temp results in register + + #pragma unroll + for (uint32_t idx = 0; idx < (1 << D); idx++) { + float w = 1; + uint32_t pos_grid_local[D]; + + #pragma unroll + for (uint32_t d = 0; d < D; d++) { + if ((idx & (1 << d)) == 0) { + w *= 1 - pos[d]; + pos_grid_local[d] = pos_grid[d]; + } else { + w *= pos[d]; + pos_grid_local[d] = pos_grid[d] + 1; + } + } + + uint32_t index = get_grid_index(gridtype, align_corners, 0, hashmap_size, resolution, pos_grid_local); // Hashing + + // writing to register (fast) + #pragma unroll + for (uint32_t ch = 0; ch < C; ch++) { + results[ch] += w * grid[index + ch]; + } + + //printf("[b=%d, l=%d] int %d, idx %d, w %f, val %f\n", b, level, idx, index, w, grid[index]); + } + + // writing to global memory (slow) + #pragma unroll + for (uint32_t ch = 0; ch < C; ch++) { + outputs[ch] = results[ch]; + } + + // prepare dy_dx for calc_grad_inputs + // differentiable (soft) indexing: https://discuss.pytorch.org/t/differentiable-indexing/17647/9 + if (calc_grad_inputs) { + + dy_dx += b * D * L * C + level * D * C; // B L D C + + #pragma unroll + for (uint32_t gd = 0; gd < D; gd++) { + + scalar_t results_grad[C] = {0}; + + #pragma unroll + for (uint32_t idx = 0; idx < (1 << (D - 1)); idx++) { + float w = scale; + uint32_t pos_grid_local[D]; + + #pragma unroll + for (uint32_t nd = 0; nd < D - 1; nd++) { + const uint32_t d = (nd >= gd) ? (nd + 1) : nd; + + if ((idx & (1 << nd)) == 0) { + w *= 1 - pos[d]; + pos_grid_local[d] = pos_grid[d]; + } else { + w *= pos[d]; + pos_grid_local[d] = pos_grid[d] + 1; + } + } + + pos_grid_local[gd] = pos_grid[gd]; + uint32_t index_left = get_grid_index(gridtype, align_corners, 0, hashmap_size, resolution, pos_grid_local); + pos_grid_local[gd] = pos_grid[gd] + 1; + uint32_t index_right = get_grid_index(gridtype, align_corners, 0, hashmap_size, resolution, pos_grid_local); + + #pragma unroll + for (uint32_t ch = 0; ch < C; ch++) { + results_grad[ch] += w * (grid[index_right + ch] - grid[index_left + ch]); + } + } + + #pragma unroll + for (uint32_t ch = 0; ch < C; ch++) { + dy_dx[gd * C + ch] = results_grad[ch]; + } + } + } +} + + +template +__global__ void kernel_grid_backward( + const scalar_t * __restrict__ grad, + const float * __restrict__ inputs, + const scalar_t * __restrict__ grid, + const int * __restrict__ offsets, + scalar_t * __restrict__ grad_grid, + const uint32_t B, const uint32_t L, const float S, const uint32_t H, + const uint32_t gridtype, + const bool align_corners +) { + const uint32_t b = (blockIdx.x * blockDim.x + threadIdx.x) * N_C / C; + if (b >= B) return; + + const uint32_t level = blockIdx.y; + const uint32_t ch = (blockIdx.x * blockDim.x + threadIdx.x) * N_C - b * C; + + // locate + grad_grid += offsets[level] * C; + inputs += b * D; + grad += level * B * C + b * C + ch; // L, B, C + + const uint32_t hashmap_size = offsets[level + 1] - offsets[level]; + const float scale = exp2f(level * S) * H - 1.0f; + const uint32_t resolution = (uint32_t)ceil(scale) + 1; + + // check input range (should be in [0, 1]) + #pragma unroll + for (uint32_t d = 0; d < D; d++) { + if (inputs[d] < 0 || inputs[d] > 1) { + return; // grad is init as 0, so we simply return. + } + } + + // calculate coordinate + float pos[D]; + uint32_t pos_grid[D]; + + #pragma unroll + for (uint32_t d = 0; d < D; d++) { + pos[d] = inputs[d] * scale + (align_corners ? 0.0f : 0.5f); + pos_grid[d] = floorf(pos[d]); + pos[d] -= (float)pos_grid[d]; + } + + scalar_t grad_cur[N_C] = {0}; // fetch to register + #pragma unroll + for (uint32_t c = 0; c < N_C; c++) { + grad_cur[c] = grad[c]; + } + + // interpolate + #pragma unroll + for (uint32_t idx = 0; idx < (1 << D); idx++) { + float w = 1; + uint32_t pos_grid_local[D]; + + #pragma unroll + for (uint32_t d = 0; d < D; d++) { + if ((idx & (1 << d)) == 0) { + w *= 1 - pos[d]; + pos_grid_local[d] = pos_grid[d]; + } else { + w *= pos[d]; + pos_grid_local[d] = pos_grid[d] + 1; + } + } + + uint32_t index = get_grid_index(gridtype, align_corners, ch, hashmap_size, resolution, pos_grid_local); + + // atomicAdd for __half is slow (especially for large values), so we use __half2 if N_C % 2 == 0 + // TODO: use float which is better than __half, if N_C % 2 != 0 + if (std::is_same::value && N_C % 2 == 0) { + #pragma unroll + for (uint32_t c = 0; c < N_C; c += 2) { + // process two __half at once (by interpreting as a __half2) + __half2 v = {(__half)(w * grad_cur[c]), (__half)(w * grad_cur[c + 1])}; + atomicAdd((__half2*)&grad_grid[index + c], v); + } + // float, or __half when N_C % 2 != 0 (which means C == 1) + } else { + #pragma unroll + for (uint32_t c = 0; c < N_C; c++) { + atomicAdd(&grad_grid[index + c], w * grad_cur[c]); + } + } + } +} + + +template +__global__ void kernel_input_backward( + const scalar_t * __restrict__ grad, + const scalar_t * __restrict__ dy_dx, + scalar_t * __restrict__ grad_inputs, + uint32_t B, uint32_t L +) { + const uint32_t t = threadIdx.x + blockIdx.x * blockDim.x; + if (t >= B * D) return; + + const uint32_t b = t / D; + const uint32_t d = t - b * D; + + dy_dx += b * L * D * C; + + scalar_t result = 0; + + # pragma unroll + for (int l = 0; l < L; l++) { + # pragma unroll + for (int ch = 0; ch < C; ch++) { + result += grad[l * B * C + b * C + ch] * dy_dx[l * D * C + d * C + ch]; + } + } + + grad_inputs[t] = result; +} + + +template +void kernel_grid_wrapper(const float *inputs, const scalar_t *embeddings, const int *offsets, scalar_t *outputs, const uint32_t B, const uint32_t C, const uint32_t L, const float S, const uint32_t H, const bool calc_grad_inputs, scalar_t *dy_dx, const uint32_t gridtype, const bool align_corners) { + static constexpr uint32_t N_THREAD = 512; + const dim3 blocks_hashgrid = { div_round_up(B, N_THREAD), L, 1 }; + switch (C) { + case 1: kernel_grid<<>>(inputs, embeddings, offsets, outputs, B, L, S, H, calc_grad_inputs, dy_dx, gridtype, align_corners); break; + case 2: kernel_grid<<>>(inputs, embeddings, offsets, outputs, B, L, S, H, calc_grad_inputs, dy_dx, gridtype, align_corners); break; + case 4: kernel_grid<<>>(inputs, embeddings, offsets, outputs, B, L, S, H, calc_grad_inputs, dy_dx, gridtype, align_corners); break; + case 8: kernel_grid<<>>(inputs, embeddings, offsets, outputs, B, L, S, H, calc_grad_inputs, dy_dx, gridtype, align_corners); break; + default: throw std::runtime_error{"GridEncoding: C must be 1, 2, 4, or 8."}; + } +} + +// inputs: [B, D], float, in [0, 1] +// embeddings: [sO, C], float +// offsets: [L + 1], uint32_t +// outputs: [L, B, C], float (L first, so only one level of hashmap needs to fit into cache at a time.) +// H: base resolution +// dy_dx: [B, L * D * C] +template +void grid_encode_forward_cuda(const float *inputs, const scalar_t *embeddings, const int *offsets, scalar_t *outputs, const uint32_t B, const uint32_t D, const uint32_t C, const uint32_t L, const float S, const uint32_t H, const bool calc_grad_inputs, scalar_t *dy_dx, const uint32_t gridtype, const bool align_corners) { + switch (D) { + case 1: kernel_grid_wrapper(inputs, embeddings, offsets, outputs, B, C, L, S, H, calc_grad_inputs, dy_dx, gridtype, align_corners); break; + case 2: kernel_grid_wrapper(inputs, embeddings, offsets, outputs, B, C, L, S, H, calc_grad_inputs, dy_dx, gridtype, align_corners); break; + case 3: kernel_grid_wrapper(inputs, embeddings, offsets, outputs, B, C, L, S, H, calc_grad_inputs, dy_dx, gridtype, align_corners); break; + case 4: kernel_grid_wrapper(inputs, embeddings, offsets, outputs, B, C, L, S, H, calc_grad_inputs, dy_dx, gridtype, align_corners); break; + case 5: kernel_grid_wrapper(inputs, embeddings, offsets, outputs, B, C, L, S, H, calc_grad_inputs, dy_dx, gridtype, align_corners); break; + default: throw std::runtime_error{"GridEncoding: D must be 1, 2, 3, 4, or 5."}; + } + +} + +template +void kernel_grid_backward_wrapper(const scalar_t *grad, const float *inputs, const scalar_t *embeddings, const int *offsets, scalar_t *grad_embeddings, const uint32_t B, const uint32_t C, const uint32_t L, const float S, const uint32_t H, const bool calc_grad_inputs, scalar_t *dy_dx, scalar_t *grad_inputs, const uint32_t gridtype, const bool align_corners) { + static constexpr uint32_t N_THREAD = 256; + const uint32_t N_C = std::min(2u, C); // n_features_per_thread + const dim3 blocks_hashgrid = { div_round_up(B * C / N_C, N_THREAD), L, 1 }; + switch (C) { + case 1: + kernel_grid_backward<<>>(grad, inputs, embeddings, offsets, grad_embeddings, B, L, S, H, gridtype, align_corners); + if (calc_grad_inputs) kernel_input_backward<<>>(grad, dy_dx, grad_inputs, B, L); + break; + case 2: + kernel_grid_backward<<>>(grad, inputs, embeddings, offsets, grad_embeddings, B, L, S, H, gridtype, align_corners); + if (calc_grad_inputs) kernel_input_backward<<>>(grad, dy_dx, grad_inputs, B, L); + break; + case 4: + kernel_grid_backward<<>>(grad, inputs, embeddings, offsets, grad_embeddings, B, L, S, H, gridtype, align_corners); + if (calc_grad_inputs) kernel_input_backward<<>>(grad, dy_dx, grad_inputs, B, L); + break; + case 8: + kernel_grid_backward<<>>(grad, inputs, embeddings, offsets, grad_embeddings, B, L, S, H, gridtype, align_corners); + if (calc_grad_inputs) kernel_input_backward<<>>(grad, dy_dx, grad_inputs, B, L); + break; + default: throw std::runtime_error{"GridEncoding: C must be 1, 2, 4, or 8."}; + } +} + + +// grad: [L, B, C], float +// inputs: [B, D], float, in [0, 1] +// embeddings: [sO, C], float +// offsets: [L + 1], uint32_t +// grad_embeddings: [sO, C] +// H: base resolution +template +void grid_encode_backward_cuda(const scalar_t *grad, const float *inputs, const scalar_t *embeddings, const int *offsets, scalar_t *grad_embeddings, const uint32_t B, const uint32_t D, const uint32_t C, const uint32_t L, const float S, const uint32_t H, const bool calc_grad_inputs, scalar_t *dy_dx, scalar_t *grad_inputs, const uint32_t gridtype, const bool align_corners) { + switch (D) { + case 1: kernel_grid_backward_wrapper(grad, inputs, embeddings, offsets, grad_embeddings, B, C, L, S, H, calc_grad_inputs, dy_dx, grad_inputs, gridtype, align_corners); break; + case 2: kernel_grid_backward_wrapper(grad, inputs, embeddings, offsets, grad_embeddings, B, C, L, S, H, calc_grad_inputs, dy_dx, grad_inputs, gridtype, align_corners); break; + case 3: kernel_grid_backward_wrapper(grad, inputs, embeddings, offsets, grad_embeddings, B, C, L, S, H, calc_grad_inputs, dy_dx, grad_inputs, gridtype, align_corners); break; + case 4: kernel_grid_backward_wrapper(grad, inputs, embeddings, offsets, grad_embeddings, B, C, L, S, H, calc_grad_inputs, dy_dx, grad_inputs, gridtype, align_corners); break; + case 5: kernel_grid_backward_wrapper(grad, inputs, embeddings, offsets, grad_embeddings, B, C, L, S, H, calc_grad_inputs, dy_dx, grad_inputs, gridtype, align_corners); break; + default: throw std::runtime_error{"GridEncoding: D must be 1, 2, 3, 4, or 5."}; + } +} + + + +void grid_encode_forward(const at::Tensor inputs, const at::Tensor embeddings, const at::Tensor offsets, at::Tensor outputs, const uint32_t B, const uint32_t D, const uint32_t C, const uint32_t L, const float S, const uint32_t H, const bool calc_grad_inputs, at::Tensor dy_dx, const uint32_t gridtype, const bool align_corners) { + CHECK_CUDA(inputs); + CHECK_CUDA(embeddings); + CHECK_CUDA(offsets); + CHECK_CUDA(outputs); + CHECK_CUDA(dy_dx); + + CHECK_CONTIGUOUS(inputs); + CHECK_CONTIGUOUS(embeddings); + CHECK_CONTIGUOUS(offsets); + CHECK_CONTIGUOUS(outputs); + CHECK_CONTIGUOUS(dy_dx); + + CHECK_IS_FLOATING(inputs); + CHECK_IS_FLOATING(embeddings); + CHECK_IS_INT(offsets); + CHECK_IS_FLOATING(outputs); + CHECK_IS_FLOATING(dy_dx); + + AT_DISPATCH_FLOATING_TYPES_AND_HALF( + embeddings.scalar_type(), "grid_encode_forward", ([&] { + grid_encode_forward_cuda(inputs.data_ptr(), embeddings.data_ptr(), offsets.data_ptr(), outputs.data_ptr(), B, D, C, L, S, H, calc_grad_inputs, dy_dx.data_ptr(), gridtype, align_corners); + })); +} + +void grid_encode_backward(const at::Tensor grad, const at::Tensor inputs, const at::Tensor embeddings, const at::Tensor offsets, at::Tensor grad_embeddings, const uint32_t B, const uint32_t D, const uint32_t C, const uint32_t L, const float S, const uint32_t H, const bool calc_grad_inputs, const at::Tensor dy_dx, at::Tensor grad_inputs, const uint32_t gridtype, const bool align_corners) { + CHECK_CUDA(grad); + CHECK_CUDA(inputs); + CHECK_CUDA(embeddings); + CHECK_CUDA(offsets); + CHECK_CUDA(grad_embeddings); + CHECK_CUDA(dy_dx); + CHECK_CUDA(grad_inputs); + + CHECK_CONTIGUOUS(grad); + CHECK_CONTIGUOUS(inputs); + CHECK_CONTIGUOUS(embeddings); + CHECK_CONTIGUOUS(offsets); + CHECK_CONTIGUOUS(grad_embeddings); + CHECK_CONTIGUOUS(dy_dx); + CHECK_CONTIGUOUS(grad_inputs); + + CHECK_IS_FLOATING(grad); + CHECK_IS_FLOATING(inputs); + CHECK_IS_FLOATING(embeddings); + CHECK_IS_INT(offsets); + CHECK_IS_FLOATING(grad_embeddings); + CHECK_IS_FLOATING(dy_dx); + CHECK_IS_FLOATING(grad_inputs); + + AT_DISPATCH_FLOATING_TYPES_AND_HALF( + grad.scalar_type(), "grid_encode_backward", ([&] { + grid_encode_backward_cuda(grad.data_ptr(), inputs.data_ptr(), embeddings.data_ptr(), offsets.data_ptr(), grad_embeddings.data_ptr(), B, D, C, L, S, H, calc_grad_inputs, dy_dx.data_ptr(), grad_inputs.data_ptr(), gridtype, align_corners); + })); + +} diff --git a/third_party/FoundationPose/bundlesdf/mycuda/torch_ngp_grid_encoder/gridencoder.h b/third_party/FoundationPose/bundlesdf/mycuda/torch_ngp_grid_encoder/gridencoder.h new file mode 100644 index 0000000..b093e78 --- /dev/null +++ b/third_party/FoundationPose/bundlesdf/mycuda/torch_ngp_grid_encoder/gridencoder.h @@ -0,0 +1,15 @@ +#ifndef _HASH_ENCODE_H +#define _HASH_ENCODE_H + +#include +#include + +// inputs: [B, D], float, in [0, 1] +// embeddings: [sO, C], float +// offsets: [L + 1], uint32_t +// outputs: [B, L * C], float +// H: base resolution +void grid_encode_forward(const at::Tensor inputs, const at::Tensor embeddings, const at::Tensor offsets, at::Tensor outputs, const uint32_t B, const uint32_t D, const uint32_t C, const uint32_t L, const float S, const uint32_t H, const bool calc_grad_inputs, at::Tensor dy_dx, const uint32_t gridtype, const bool align_corners); +void grid_encode_backward(const at::Tensor grad, const at::Tensor inputs, const at::Tensor embeddings, const at::Tensor offsets, at::Tensor grad_embeddings, const uint32_t B, const uint32_t D, const uint32_t C, const uint32_t L, const float S, const uint32_t H, const bool calc_grad_inputs, const at::Tensor dy_dx, at::Tensor grad_inputs, const uint32_t gridtype, const bool align_corners); + +#endif \ No newline at end of file diff --git a/third_party/FoundationPose/bundlesdf/nerf_helpers.py b/third_party/FoundationPose/bundlesdf/nerf_helpers.py new file mode 100644 index 0000000..0b8e0f2 --- /dev/null +++ b/third_party/FoundationPose/bundlesdf/nerf_helpers.py @@ -0,0 +1,475 @@ +# Copyright (c) 2023, NVIDIA CORPORATION. All rights reserved. +# +# NVIDIA CORPORATION and its licensors retain all intellectual property +# and proprietary rights in and to this software, related documentation +# and any modifications thereto. Any use, reproduction, disclosure or +# distribution of this software and related documentation without an express +# license agreement from NVIDIA CORPORATION is strictly prohibited. + + +import torch,pdb,os,sys +import torch.nn as nn +import torch.nn.functional as F +import numpy as np +code_dir = os.path.dirname(os.path.realpath(__file__)) +sys.path.append(f'{code_dir}/../') +from Utils import * +from pytorch3d.transforms import so3_log_map,so3_exp_map,se3_exp_map + +img2mse = lambda x, y : torch.mean((x - y) ** 2) +img2mae = lambda x, y: (torch.abs(x - y)).mean() +mse2psnr = lambda x : -10. * torch.log(x) / torch.log(torch.Tensor([10.])) +to8b = lambda x : (255*np.clip(x,0,1)).astype(np.uint8) + + +class FeatureArray(nn.Module): + """ + Per-frame corrective latent code. + """ + + def __init__(self, num_frames, num_channels): + super().__init__() + + self.num_frames = num_frames + self.num_channels = num_channels + + self.data = nn.parameter.Parameter(torch.normal(0,1,size=[num_frames, num_channels]).float(), requires_grad=True) + self.register_parameter('data',self.data) + + + def __call__(self, ids): + return self.data[ids] + + +class PoseArray(nn.Module): + def __init__(self, num_frames,max_trans,max_rot): + super().__init__() + self.num_frames = num_frames + self.max_trans = max_trans + self.max_rot = max_rot + self.data = nn.parameter.Parameter(torch.zeros([num_frames, 6]).float(), requires_grad=True) + self.register_parameter('data',self.data) + + + def get_matrices(self,ids): + if not torch.is_tensor(ids): + ids = torch.tensor(ids).long() + theta = torch.tanh(self.data) + trans = theta[:,:3] * self.max_trans + rot = theta[:,3:6] * self.max_rot/180.0*np.pi + Ts_data = se3_exp_map(torch.cat((trans,rot),dim=-1)).permute(0,2,1) + Ts = torch.eye(4, device=self.data.device).reshape(1,4,4).repeat(len(ids),1,1) + mask = ids!=0 + Ts[mask] = Ts_data[ids[mask]] + return Ts + + + +class SHEncoder(nn.Module): + '''Spherical encoding + ''' + def __init__(self, input_dim=3, degree=4): + + super().__init__() + + self.input_dim = input_dim + self.degree = degree + + assert self.input_dim == 3 + assert self.degree >= 1 and self.degree <= 5 + + self.out_dim = degree ** 2 + + self.C0 = 0.28209479177387814 + self.C1 = 0.4886025119029199 + self.C2 = [ + 1.0925484305920792, + -1.0925484305920792, + 0.31539156525252005, + -1.0925484305920792, + 0.5462742152960396 + ] + self.C3 = [ + -0.5900435899266435, + 2.890611442640554, + -0.4570457994644658, + 0.3731763325901154, + -0.4570457994644658, + 1.445305721320277, + -0.5900435899266435 + ] + self.C4 = [ + 2.5033429417967046, + -1.7701307697799304, + 0.9461746957575601, + -0.6690465435572892, + 0.10578554691520431, + -0.6690465435572892, + 0.47308734787878004, + -1.7701307697799304, + 0.6258357354491761 + ] + + def forward(self, input, **kwargs): + + result = torch.empty((*input.shape[:-1], self.out_dim), dtype=input.dtype, device=input.device) + x, y, z = input.unbind(-1) + + result[..., 0] = self.C0 + if self.degree > 1: + result[..., 1] = -self.C1 * y + result[..., 2] = self.C1 * z + result[..., 3] = -self.C1 * x + if self.degree > 2: + xx, yy, zz = x * x, y * y, z * z + xy, yz, xz = x * y, y * z, x * z + result[..., 4] = self.C2[0] * xy + result[..., 5] = self.C2[1] * yz + result[..., 6] = self.C2[2] * (2.0 * zz - xx - yy) + #result[..., 6] = self.C2[2] * (3.0 * zz - 1) # xx + yy + zz == 1, but this will lead to different backward gradients, interesting... + result[..., 7] = self.C2[3] * xz + result[..., 8] = self.C2[4] * (xx - yy) + if self.degree > 3: + result[..., 9] = self.C3[0] * y * (3 * xx - yy) + result[..., 10] = self.C3[1] * xy * z + result[..., 11] = self.C3[2] * y * (4 * zz - xx - yy) + result[..., 12] = self.C3[3] * z * (2 * zz - 3 * xx - 3 * yy) + result[..., 13] = self.C3[4] * x * (4 * zz - xx - yy) + result[..., 14] = self.C3[5] * z * (xx - yy) + result[..., 15] = self.C3[6] * x * (xx - 3 * yy) + if self.degree > 4: + result[..., 16] = self.C4[0] * xy * (xx - yy) + result[..., 17] = self.C4[1] * yz * (3 * xx - yy) + result[..., 18] = self.C4[2] * xy * (7 * zz - 1) + result[..., 19] = self.C4[3] * yz * (7 * zz - 3) + result[..., 20] = self.C4[4] * (zz * (35 * zz - 30) + 3) + result[..., 21] = self.C4[5] * xz * (7 * zz - 3) + result[..., 22] = self.C4[6] * (xx - yy) * (7 * zz - 1) + result[..., 23] = self.C4[7] * xz * (xx - 3 * yy) + result[..., 24] = self.C4[8] * (xx * (xx - 3 * yy) - yy * (3 * xx - yy)) + + return result + + +class Embedder(nn.Module): + def __init__(self, **kwargs): + super().__init__() + self.kwargs = kwargs + self.create_embedding_fn() + + def create_embedding_fn(self): + embed_fns = [] + d = self.kwargs['input_dims'] + out_dim = 0 + if self.kwargs['include_input']: + embed_fns.append(lambda x : x) + out_dim += d + + max_freq = self.kwargs['max_freq_log2'] + N_freqs = self.kwargs['num_freqs'] + + if self.kwargs['log_sampling']: + freq_bands = 2.**torch.linspace(0., max_freq, steps=N_freqs) + else: + freq_bands = torch.linspace(2.**0., 2.**max_freq, steps=N_freqs) + + for freq in freq_bands: + for p_fn in self.kwargs['periodic_fns']: + embed_fns.append(lambda x, p_fn=p_fn, freq=freq : p_fn(x * freq)) + out_dim += d + + self.embed_fns = embed_fns + self.out_dim = out_dim + + def forward(self, inputs): + return torch.cat([fn(inputs) for fn in self.embed_fns], -1) + + +def get_embedder(multires, cfg, i=0, octree_m=None): + if i == -1: + return nn.Identity(), 3 + elif i==0: + embed_kwargs = { + 'include_input' : True, + 'input_dims' : 3, + 'max_freq_log2' : multires-1, + 'num_freqs' : multires, + 'log_sampling' : True, + 'periodic_fns' : [torch.sin, torch.cos], + } + + embed = Embedder(**embed_kwargs) + out_dim = embed.out_dim + elif i==1: + from mycuda.torch_ngp_grid_encoder.grid import GridEncoder + embed = GridEncoder(input_dim=3, n_levels=cfg['num_levels'], log2_hashmap_size=cfg['log2_hashmap_size'], desired_resolution=cfg['finest_res'], base_resolution=cfg['base_res'], level_dim=cfg['feature_grid_dim']) + print(embed) + out_dim = embed.out_dim + elif i==2: + embed = SHEncoder(degree=cfg['multires_views']) + out_dim = embed.out_dim + return embed, out_dim + + + +def mesh_to_real_world(mesh,pose_offset,translation,sc_factor): + ''' + @pose_offset: optimized delta pose of the first frame. Usually it's identity + ''' + mesh.vertices = mesh.vertices/sc_factor - np.array(translation).reshape(1,3) + mesh.apply_transform(pose_offset) + return mesh + + +def get_optimized_poses_in_real_world(poses_normalized, pose_array, sc_factor, translation): + ''' + @poses_normalized: np array, cam_in_ob (opengl convention), normalized to [-1,1] and centered + @pose_array: PoseArray, delta poses + Return: + cam_in_ob, real-world unit, opencv convention + ''' + original_poses = poses_normalized.copy() + original_poses[:, :3, 3] /= sc_factor # To true world scale + original_poses[:, :3, 3] -= translation + + # Apply pose transformation + tf = pose_array.get_matrices(np.arange(len(poses_normalized))).reshape(-1,4,4).data.cpu().numpy() + optimized_poses = tf@poses_normalized + + optimized_poses = np.array(optimized_poses).astype(np.float32) + optimized_poses[:, :3, 3] /= sc_factor + optimized_poses[:, :3, 3] -= translation + + original_init_ob_in_cam = optimized_poses[0].copy() + offset = np.linalg.inv(original_init_ob_in_cam)@original_poses[0] + for i in range(len(optimized_poses)): + new_ob_in_cam = optimized_poses[i]@offset + optimized_poses[i] = new_ob_in_cam + optimized_poses[i] = optimized_poses[i]@glcam_in_cvcam + + return optimized_poses,offset + +def preprocess_data(rgbs,depths,masks,normal_maps,poses,sc_factor,translation): + ''' + @rgbs: np array (N,H,W,3) + @depths: (N,H,W) + @masks: (N,H,W) + @normal_maps: (N,H,W,3) + @poses: (N,4,4) + ''' + depths[depths<0.001] = BAD_DEPTH + if masks is not None: + rgbs[masks==0] = BAD_COLOR + depths[masks==0] = BAD_DEPTH + if normal_maps is not None: + normal_maps[...,[1,2]] *= -1 # To OpenGL + normal_maps[masks==0] = 0 + masks = masks[...,None] + + rgbs = (rgbs / 255.0).astype(np.float32) + depths *= sc_factor + depths = depths[...,None] + poses[:, :3, 3] += translation + poses[:, :3, 3] *= sc_factor + return rgbs,depths,masks,normal_maps,poses + + +class NeRFSmall(nn.Module): + def __init__(self,num_layers=3,hidden_dim=64,geo_feat_dim=15,num_layers_color=4,hidden_dim_color=64,input_ch=3, input_ch_views=3): + super(NeRFSmall, self).__init__() + + self.input_ch = input_ch + self.input_ch_views = input_ch_views + + # sigma network + self.num_layers = num_layers + self.hidden_dim = hidden_dim + self.geo_feat_dim = geo_feat_dim + + sigma_net = [] + for l in range(num_layers): + if l == 0: + in_dim = self.input_ch + else: + in_dim = hidden_dim + + if l == num_layers - 1: + out_dim = 1 + self.geo_feat_dim # 1 sigma + 15 SH features for color + else: + out_dim = hidden_dim + + sigma_net.append(nn.Linear(in_dim, out_dim, bias=True)) + if l!=num_layers-1: + sigma_net.append(nn.ReLU(inplace=True)) + + self.sigma_net = nn.Sequential(*sigma_net) + torch.nn.init.constant_(self.sigma_net[-1].bias, 0.1) # Encourage last layer predict positive SDF + + # color network + self.num_layers_color = num_layers_color + self.hidden_dim_color = hidden_dim_color + + color_net = [] + for l in range(num_layers_color): + if l == 0: + in_dim = self.input_ch_views + self.geo_feat_dim + else: + in_dim = hidden_dim + + if l == num_layers_color - 1: + out_dim = 3 # 3 rgb + else: + out_dim = hidden_dim + + color_net.append(nn.Linear(in_dim, out_dim, bias=True)) + if l!=num_layers_color-1: + color_net.append(nn.ReLU(inplace=True)) + + self.color_net = nn.Sequential(*color_net) + + def forward_sdf(self,x): + ''' + @x: embedded positions + ''' + h = self.sigma_net(x) + sigma, geo_feat = h[..., 0], h[..., 1:] + return sigma + + + def forward(self, x): + x = x.float() + input_pts, input_views = torch.split(x, [self.input_ch, self.input_ch_views], dim=-1) + + # sigma + h = input_pts + h = self.sigma_net(h) + + sigma, geo_feat = h[..., 0], h[..., 1:] + + # color + h = torch.cat([input_views, geo_feat], dim=-1) + color = self.color_net(h) + + outputs = torch.cat([color, sigma.unsqueeze(dim=-1)], -1) + + return outputs + + +def sample_pdf(bins, weights, N_samples, det=False): + weights = weights + 1e-5 # prevent nans + pdf = weights / torch.sum(weights, -1, keepdim=True) + cdf = torch.cumsum(pdf, -1) + cdf = torch.cat([torch.zeros_like(cdf[...,:1]), cdf], -1) # (batch, len(bins)) + + if det: + u = torch.linspace(0., 1., steps=N_samples) + u = u.expand(list(cdf.shape[:-1]) + [N_samples]) + else: + u = torch.rand(list(cdf.shape[:-1]) + [N_samples]) + + u = u.contiguous() + inds = torch.searchsorted(cdf, u, right=True) + below = torch.max(torch.zeros_like(inds-1), inds-1) + above = torch.min((cdf.shape[-1]-1) * torch.ones_like(inds), inds) + inds_g = torch.stack([below, above], -1) # (batch, N_samples, 2) + + matched_shape = [inds_g.shape[0], inds_g.shape[1], cdf.shape[-1]] + cdf_g = torch.gather(cdf.unsqueeze(1).expand(matched_shape), 2, inds_g) + bins_g = torch.gather(bins.unsqueeze(1).expand(matched_shape), 2, inds_g) + + denom = (cdf_g[...,1]-cdf_g[...,0]) + denom = torch.where(denom<1e-5, torch.ones_like(denom), denom) + t = (u-cdf_g[...,0])/denom + samples = bins_g[...,0] + t * (bins_g[...,1]-bins_g[...,0]) + + return samples + + + +def get_camera_rays_np(H, W, K): + """Get ray origins, directions from a pinhole camera.""" + i, j = np.meshgrid(np.arange(W, dtype=np.float32), + np.arange(H, dtype=np.float32), indexing='xy') + dirs = np.stack([(i - K[0,2])/K[0,0], -(j - K[1,2])/K[1,1], -np.ones_like(i)], axis=-1) + return dirs + + + +def get_masks(z_vals, target_d, truncation, cfg, dir_norm=None): + valid_depth_mask = (target_d>=cfg['near']*cfg['sc_factor']) & (target_d<=cfg['far']*cfg['sc_factor']) + front_mask = (z_vals < target_d - truncation) + back_mask = (z_vals > target_d + truncation*cfg['neg_trunc_ratio']) + + sdf_mask = (1.0 - front_mask.float()) * (1.0 - back_mask.float()) * valid_depth_mask + + num_fs_samples = front_mask.sum() + num_sdf_samples = sdf_mask.sum() + num_samples = num_sdf_samples + num_fs_samples + fs_weight = 0.5 + sdf_weight = 1.0 - fs_weight + return front_mask.bool(), sdf_mask.bool(), fs_weight, sdf_weight + + +def get_sdf_loss(z_vals, target_d, predicted_sdf, truncation, cfg, return_mask=False, sample_weights=None, rays_d=None): + dir_norm = rays_d.norm(dim=-1,keepdim=True) + front_mask, sdf_mask, fs_weight, sdf_weight = get_masks(z_vals, target_d, truncation, cfg, dir_norm=dir_norm) + front_mask = front_mask.bool() + + mask = (target_d>cfg['far']*cfg['sc_factor']) & (predicted_sdf tymax) | (tymin > tmax)] = 0 + tmin[tymin>tmin] = tymin[tymin>tmin] + tmax[tymax tzmax) | (tzmin > tmax)] = 0 + tmin[tzmin>tmin] = tzmin[tzmin>tmin] #(N) + tmax[tzmax=0 + near = (dirs_unit*tmin.reshape(-1,1))[:,2] + far = (dirs_unit*tmax.reshape(-1,1))[:,2] + good_rays = rays[ishit] + near = near[ishit] + far = far[ishit] + near = np.abs(near) + far = np.abs(far) + good_rays = np.concatenate((good_rays,near.reshape(-1,1),far.reshape(-1,1)), axis=-1) #(N,8+2) + + return good_rays + +@torch.no_grad() +def sample_rays_uniform(N_samples,near,far,lindisp=False,perturb=True): + ''' + @near: (N_ray,1) + ''' + N_ray = near.shape[0] + t_vals = torch.linspace(0., 1., steps=N_samples, device=near.device).reshape(1,-1) + if not lindisp: + z_vals = near * (1.-t_vals) + far * (t_vals) + else: + z_vals = 1./(1./near * (1.-t_vals) + 1./far * (t_vals)) #(N_ray,N_sample) + + if perturb > 0.: + mids = .5 * (z_vals[...,1:] + z_vals[...,:-1]) + upper = torch.cat([mids, z_vals[...,-1:]], -1) + lower = torch.cat([z_vals[...,:1], mids], -1) + t_rand = torch.rand(z_vals.shape, device=far.device) + z_vals = lower + (upper - lower) * t_rand + z_vals = torch.clip(z_vals,near,far) + + return z_vals.reshape(N_ray,N_samples) + + +class DataLoader: + def __init__(self,rays,batch_size): + self.rays = rays + self.batch_size = batch_size + self.pos = 0 + self.ids = torch.randperm(len(self.rays)) + + def __next__(self): + if self.pos+self.batch_size0) & (rays[:,self.ray_depth_slice]<=self.cfg['far']*self.cfg['sc_factor']) + rays_dir = rays[mask][:,self.ray_dir_slice] + rays_depth = rays[mask][:,self.ray_depth_slice] + pts3d = rays_dir*rays_depth.reshape(-1,1) + frame_ids = rays[mask][:,self.ray_frame_id_slice].astype(int) + pts3d_w = (self.poses[frame_ids]@to_homo(pts3d)[...,None])[:,:3,0] + logging.info(f"Denoising rays based on octree cloud") + + kdtree = cKDTree(self.build_octree_pts) + dists,indices = kdtree.query(pts3d_w,k=1,workers=-1) + bad_mask = dists>0.02*self.cfg['sc_factor'] + bad_ids = np.arange(len(rays))[mask][bad_mask] + rays[bad_ids,self.ray_depth_slice] = BAD_DEPTH*self.cfg['sc_factor'] + rays[bad_ids, self.ray_type_slice] = 1 + rays = rays[rays[:,self.ray_type_slice]==0] + logging.info(f"bad_mask#={bad_mask.sum()}") + + rays = torch.tensor(rays, dtype=torch.float).cuda() + + self.rays = rays + print("rays",rays.shape) + self.data_loader = DataLoader(rays=self.rays, batch_size=self.cfg['N_rand']) + + + def create_nerf(self,device=torch.device("cuda")): + """Instantiate NeRF's MLP model. + """ + models = {} + embed_fn, input_ch = get_embedder(self.cfg['multires'], self.cfg, i=self.cfg['i_embed'], octree_m=self.octree_m) + embed_fn = embed_fn.to(device) + models['embed_fn'] = embed_fn + + input_ch_views = 0 + embeddirs_fn = None + if self.cfg['use_viewdirs']: + embeddirs_fn, input_ch_views = get_embedder(self.cfg['multires_views'], self.cfg, i=self.cfg['i_embed_views'], octree_m=self.octree_m) + models['embeddirs_fn'] = embeddirs_fn + + output_ch = 4 + skips = [4] + + model = NeRFSmall(num_layers=2,hidden_dim=64,geo_feat_dim=15,num_layers_color=3,hidden_dim_color=64,input_ch=input_ch, input_ch_views=input_ch_views+self.cfg['frame_features']).to(device) + model = model.to(device) + models['model'] = model + + model_fine = None + if self.cfg['N_importance'] > 0: + if not self.cfg['share_coarse_fine']: + model_fine = NeRFSmall(num_layers=2,hidden_dim=64,geo_feat_dim=15,num_layers_color=3,hidden_dim_color=64,input_ch=input_ch, input_ch_views=input_ch_views).to(device) + models['model_fine'] = model_fine + + # Create feature array + num_training_frames = len(self.images) + feature_array = None + if self.cfg['frame_features'] > 0: + feature_array = FeatureArray(num_training_frames, self.cfg['frame_features']).to(device) + models['feature_array'] = feature_array + # Create pose array + pose_array = None + if self.cfg['optimize_poses']: + pose_array = PoseArray(num_training_frames,max_trans=self.cfg['max_trans']*self.cfg['sc_factor'],max_rot=self.cfg['max_rot']).to(device) + models['pose_array'] = pose_array + self.models = models + + + + def make_frame_rays(self,frame_id): + mask = self.masks[frame_id,...,0].copy() + rays = get_camera_rays_np(self.H, self.W, self.K) # [self.H, self.W, 3] We create rays frame-by-frame to save memory + rays = np.concatenate([rays, self.images[frame_id]], -1) # [H, W, 6] + rays = np.concatenate([rays, self.depths[frame_id]], -1) # [H, W, 7] + rays = np.concatenate([rays, self.masks[frame_id]>0], -1) # [H, W, 8] + if self.normal_maps is not None: + rays = np.concatenate([rays, self.normal_maps[frame_id]], -1) # [H, W, 11] + rays = np.concatenate([rays, frame_id*np.ones(self.depths[frame_id].shape)], -1) # [H, W, 12] + ray_types = np.zeros((self.H,self.W,1)) # 0 is good; 1 is invalid depth (uncertain) + invalid_depth = ((self.depths[frame_id,...,0]self.cfg['far']*self.cfg['sc_factor'])) & (mask>0) + ray_types[invalid_depth] = 1 + rays = np.concatenate((rays,ray_types), axis=-1) + self.ray_dir_slice = [0,1,2] + self.ray_rgb_slice = [3,4,5] + self.ray_depth_slice = 6 + self.ray_mask_slice = 7 + if self.normal_maps is not None: + self.ray_normal_slice = [8,9,10] + self.ray_frame_id_slice = 11 + self.ray_type_slice = 12 + else: + self.ray_frame_id_slice = 8 + self.ray_type_slice = 9 + + n = rays.shape[-1] + + ########## Option2: dilate + down_scale_ratio = int(self.cfg['down_scale_ratio']) + if frame_id==0: #!NOTE first frame ob mask is assumed perfect + kernel = np.ones((100, 100), np.uint8) + mask = cv2.dilate(mask.astype(np.uint8), kernel, iterations=1) + if self.occ_masks is not None: + mask[self.occ_masks[frame_id]>0] = 0 + else: + dilate = 60//down_scale_ratio + kernel = np.ones((dilate, dilate), np.uint8) + mask = cv2.dilate(mask.astype(np.uint8), kernel, iterations=1) + if self.occ_masks is not None: + mask[self.occ_masks[frame_id]>0] = 0 + + + if self.cfg['rays_valid_depth_only']: + mask[invalid_depth] = 0 + + vs,us = np.where(mask>0) + cur_rays = rays[vs,us].reshape(-1,n) + cur_rays = cur_rays[cur_rays[:,self.ray_type_slice]==0] + cur_rays = compute_near_far_and_filter_rays(self.poses[frame_id],cur_rays,self.cfg) + if self.normal_maps is not None: + self.ray_near_slice = 13 + self.ray_far_slice = 14 + else: + self.ray_near_slice = 10 + self.ray_far_slice = 11 + + if self.cfg['use_octree']: + rays_o_world = (self.poses[frame_id]@to_homo(np.zeros((len(cur_rays),3))).T).T[:,:3] + rays_o_world = torch.from_numpy(rays_o_world).cuda().float() + rays_unit_d_cam = cur_rays[:,:3]/np.linalg.norm(cur_rays[:,:3],axis=-1).reshape(-1,1) + rays_d_world = (self.poses[frame_id][:3,:3]@rays_unit_d_cam.T).T + rays_d_world = torch.from_numpy(rays_d_world).cuda().float() + + vox_size = self.cfg['octree_raytracing_voxel_size']*self.cfg['sc_factor'] + level = int(np.floor(np.log2(2.0/vox_size))) + near,far,_,ray_depths_in_out = self.octree_m.ray_trace(rays_o_world,rays_d_world,level=level) + near = near.cpu().numpy() + valid = (near>0).reshape(-1) + cur_rays = cur_rays[valid] + + return cur_rays + + + def build_octree(self): + if self.cfg['save_octree_clouds']: + dir = f"{self.cfg['save_dir']}/build_octree_cloud.ply" + pcd = toOpen3dCloud(self.build_octree_pts) + o3d.io.write_point_cloud(dir,pcd) + if self._run is not None: + self._run.add_artifact(dir) + pts = torch.tensor(self.build_octree_pts).cuda().float() # Must be within [-1,1] + octree_smallest_voxel_size = self.cfg['octree_smallest_voxel_size']*self.cfg['sc_factor'] + finest_n_voxels = 2.0/octree_smallest_voxel_size + max_level = int(np.round(np.log2(finest_n_voxels))) + octree_smallest_voxel_size = 2.0/(2**max_level) + + #################### Dilate + dilate_radius = int(np.round(self.cfg['octree_dilate_size']/octree_smallest_voxel_size)) + dilate_radius = max(1, dilate_radius) + logging.info(f"Octree voxel dilate_radius:{dilate_radius}") + shifts = [] + for dx in [-1,0,1]: + for dy in [-1,0,1]: + for dz in [-1,0,1]: + shifts.append([dx,dy,dz]) + shifts = torch.tensor(shifts).cuda().long() # (27,3) + coords = torch.floor((pts+1)/octree_smallest_voxel_size).long() #(N,3) + dilated_coords = coords.detach().clone() + for iter in range(dilate_radius): + dilated_coords = (dilated_coords[None].expand(shifts.shape[0],-1,-1) + shifts[:,None]).reshape(-1,3) + dilated_coords = torch.unique(dilated_coords,dim=0) + pts = (dilated_coords+0.5) * octree_smallest_voxel_size - 1 + pts = torch.clip(pts,-1,1) + + if self.cfg['save_octree_clouds']: + pcd = toOpen3dCloud(pts.data.cpu().numpy()) + dir = f"{self.cfg['save_dir']}/build_octree_cloud_dilated.ply" + o3d.io.write_point_cloud(dir,pcd) + if self._run is not None: + self._run.add_artifact(dir) + #################### + + assert pts.min()>=-1 and pts.max()<=1 + self.octree_m = OctreeManager(pts, max_level) + + if self.cfg['save_octree_clouds']: + dir = f"{self.cfg['save_dir']}/octree_boxes_max_level.ply" + m = self.octree_m.draw(level=max_level,method='point') + m.export(dir) + if self._run is not None: + self._run.add_artifact(dir) + vox_size = self.cfg['octree_raytracing_voxel_size']*self.cfg['sc_factor'] + level = int(np.round(np.log2(2.0/vox_size))) + if self.cfg['save_octree_clouds']: + dir = f"{self.cfg['save_dir']}/octree_boxes_ray_tracing_level.ply" + m = self.octree_m.draw(level=level,method='point') + m.export(dir) + if self._run is not None: + self._run.add_artifact(dir) + + + def create_optimizer(self): + params = [] + for k in self.models: + if self.models[k] is not None and k!='pose_array': + params += list(self.models[k].parameters()) + + param_groups = [{'name':'basic', 'params':params, 'lr':self.cfg['lrate']}] + if self.models['pose_array'] is not None: + param_groups.append({'name':'pose_array', 'params':self.models['pose_array'].parameters(), 'lr':self.cfg['lrate_pose']}) + + self.optimizer = torch.optim.Adam(param_groups, betas=(0.9, 0.999),weight_decay=0,eps=1e-15) + + self.param_groups_init = copy.deepcopy(self.optimizer.param_groups) + + + def save_weights(self,out_file,models): + data = { + 'global_step': self.global_step, + 'model': models['model'].state_dict(), + 'optimizer': self.optimizer.state_dict(), + } + if 'model_fine' in models and models['model_fine'] is not None: + data['model_fine'] = models['model_fine'].state_dict() + if models['embed_fn'] is not None: + data['embed_fn'] = models['embed_fn'].state_dict() + if models['embeddirs_fn'] is not None: + data['embeddirs_fn'] = models['embeddirs_fn'].state_dict() + if self.cfg['optimize_poses']>0: + data['pose_array'] = models['pose_array'].state_dict() + if self.cfg['frame_features'] > 0: + data['feature_array'] = models['feature_array'].state_dict() + if self.octree_m is not None: + data['octree'] = self.octree_m.octree + dir = out_file + torch.save(data,dir) + print('Saved checkpoints at', dir) + if self._run is not None: + self._run.add_artifact(dir) + dir1 = copy.deepcopy(dir) + dir = f'{os.path.dirname(out_file)}/model_latest.pth' + if dir1!=dir: + os.system(f'cp {dir1} {dir}') + if self._run is not None: + self._run.add_artifact(dir) + + + def schedule_lr(self): + for i,param_group in enumerate(self.optimizer.param_groups): + init_lr = self.param_groups_init[i]['lr'] + new_lrate = init_lr * (self.cfg['decay_rate'] ** (float(self.global_step) / self.N_iters)) + param_group['lr'] = new_lrate + + + def render_images(self,img_i,cur_rays=None): + if cur_rays is None: + frame_ids = self.rays[:, self.ray_frame_id_slice].cuda() + cur_rays = self.rays[frame_ids==img_i].cuda() + gt_depth = cur_rays[:,self.ray_depth_slice] + gt_rgb = cur_rays[:,self.ray_rgb_slice].cpu() + ray_type = cur_rays[:,self.ray_type_slice].data.cpu().numpy() + + ori_chunk = self.cfg['chunk'] + self.cfg['chunk'] = copy.deepcopy(self.cfg['N_rand']) + with torch.no_grad(): + rgb, extras = self.render(rays=cur_rays, lindisp=False,perturb=False,raw_noise_std=0, depth=gt_depth) + self.cfg['chunk'] = ori_chunk + + sdf = extras['raw'][...,-1] + z_vals = extras['z_vals'] + signs = sdf[:, 1:] * sdf[:, :-1] + empty_rays = (signs>0).all(dim=-1) + mask = signs<0 + inds = torch.argmax(mask.float(), axis=1) + inds = inds[..., None] + depth = torch.gather(z_vals,dim=1,index=inds) + depth[empty_rays] = self.cfg['far']*self.cfg['sc_factor'] + depth = depth[..., None].data.cpu().numpy() + + rgb = rgb.data.cpu().numpy() + + rgb_full = np.zeros((self.H,self.W,3),dtype=float) + depth_full = np.zeros((self.H,self.W),dtype=float) + ray_mask_full = np.zeros((self.H,self.W,3),dtype=np.uint8) + X = cur_rays[:,self.ray_dir_slice].data.cpu().numpy() + X[:,[1,2]] = -X[:,[1,2]] + projected = (self.K@X.T).T + uvs = projected/projected[:,2].reshape(-1,1) + uvs = uvs.round().astype(int) + uvs_good = uvs[ray_type==0] + ray_mask_full[uvs_good[:,1],uvs_good[:,0]] = [255,0,0] + uvs_uncertain = uvs[ray_type==1] + ray_mask_full[uvs_uncertain[:,1],uvs_uncertain[:,0]] = [0,255,0] + rgb_full[uvs[:,1],uvs[:,0]] = rgb.reshape(-1,3) + depth_full[uvs[:,1],uvs[:,0]] = depth.reshape(-1) + gt_rgb_full = np.zeros((self.H,self.W,3),dtype=float) + gt_rgb_full[uvs[:,1],uvs[:,0]] = gt_rgb.reshape(-1,3).data.cpu().numpy() + gt_depth_full = np.zeros((self.H,self.W),dtype=float) + gt_depth_full[uvs[:,1],uvs[:,0]] = gt_depth.reshape(-1).data.cpu().numpy() + + return rgb_full, depth_full, ray_mask_full, gt_rgb_full, gt_depth_full, extras + + + def get_gradients(self): + if self.models['pose_array'] is not None: + max_pose_grad = torch.abs(self.models['pose_array'].data.grad).max() + max_embed_grad = 0 + for embed in self.models['embed_fn'].embeddings: + max_embed_grad = max(max_embed_grad,torch.abs(embed.weight.grad).max()) + if self.models['feature_array'] is not None: + max_feature_grad = torch.abs(self.models['feature_array'].data.grad).max() + return max_pose_grad, max_embed_grad, max_feature_grad + + + def get_truncation(self): + '''Annearl truncation over training + ''' + if self.cfg['trunc_decay_type']=='linear': + truncation = self.cfg['trunc_start'] - (self.cfg['trunc_start']-self.cfg['trunc']) * float(self.global_step)/self.cfg['n_step'] + elif self.cfg['trunc_decay_type']=='exp': + lamb = np.log(self.cfg['trunc']/self.cfg['trunc_start']) / (self.cfg['n_step']/4) + truncation = self.cfg['trunc_start']*np.exp(self.global_step*lamb) + truncation = max(truncation,self.cfg['trunc']) + else: + truncation = self.cfg['trunc'] + + truncation *= self.cfg['sc_factor'] + return truncation + + + def train_loop(self,batch): + target_s = batch[:, self.ray_rgb_slice] # Color (N,3) + target_d = batch[:, self.ray_depth_slice] # Normalized scale (N) + + target_mask = batch[:,self.ray_mask_slice].bool().reshape(-1) + frame_ids = batch[:,self.ray_frame_id_slice] + + rgb, extras = self.render(rays=batch, depth=target_d,lindisp=False,perturb=True,raw_noise_std=self.cfg['raw_noise_std'], get_normals=False) + + valid_samples = extras['valid_samples'] #(N_ray,N_samples) + z_vals = extras['z_vals'] # [N_rand, N_samples + N_importance] + sdf = extras['raw'][..., -1] + + N_rays,N_samples = sdf.shape[:2] + valid_rays = (valid_samples>0).any(dim=-1).bool().reshape(N_rays) & (batch[:,self.ray_type_slice]==0) + + ray_type = batch[:,self.ray_type_slice].reshape(-1) + ray_weights = torch.ones((N_rays), device=rgb.device, dtype=torch.float32) + ray_weights[(frame_ids==0).view(-1)] = self.cfg['first_frame_weight'] + ray_weights = ray_weights*valid_rays.view(-1) + sample_weights = ray_weights.view(N_rays,1).expand(-1,N_samples) * valid_samples + img_loss = (((rgb-target_s)**2 * ray_weights.view(-1,1))).mean() + rgb_loss = self.cfg['rgb_weight'] * img_loss + loss = rgb_loss + + rgb0_loss = torch.tensor(0) + if 'rgb0' in extras: + img_loss0 = (((extras['rgb0']-target_s)**2 * ray_weights.view(-1,1))).mean() + rgb0_loss = img_loss0*self.cfg['rgb_weight'] + loss += rgb0_loss + + depth_loss = torch.tensor(0) + depth_loss0 = torch.tensor(0) + if self.cfg['depth_weight']>0: + signs = sdf[:, 1:] * sdf[:, :-1] + mask = signs<0 + inds = torch.argmax(mask.float(), axis=1) + inds = inds[..., None] + z_min = torch.gather(z_vals,dim=1,index=inds) + weights = ray_weights * (depth<=self.cfg['far']*self.cfg['sc_factor']) * (mask.any(dim=-1)) + depth_loss = ((z_min*weights-depth.view(-1,1)*weights)**2).mean() * self.cfg['depth_weight'] + loss = loss+depth_loss + + truncation = self.get_truncation() + sample_weights[ray_type==1] = 0 + fs_loss, sdf_loss,empty_loss, front_mask,sdf_mask = get_sdf_loss(z_vals, target_d.reshape(-1,1).expand(-1,N_samples), sdf, truncation, self.cfg,return_mask=True, sample_weights=sample_weights, rays_d=batch[:,self.ray_dir_slice]) + fs_loss = fs_loss*self.cfg['fs_weight'] + empty_loss = empty_loss*self.cfg['empty_weight'] + sdf_loss = sdf_loss*self.cfg['trunc_weight'] + loss = loss + fs_loss + sdf_loss + empty_loss + + fs_rgb_loss = torch.tensor(0) + if self.cfg['fs_rgb_weight']>0: + fs_rgb_loss = ((((torch.sigmoid(extras['raw'][...,:3])-1)*front_mask[...,None])**2) * sample_weights[...,None]).mean() + loss += fs_rgb_loss*self.cfg['fs_rgb_weight'] + + eikonal_loss = torch.tensor(0) + if self.cfg['eikonal_weight']>0: + nerf_normals = extras['normals'] + eikonal_loss = ((torch.norm(nerf_normals[sdf<1], dim=-1)-1)**2).mean() * self.cfg['eikonal_weight'] + loss += eikonal_loss + + point_cloud_loss = torch.tensor(0) + point_cloud_normal_loss = torch.tensor(0) + + + reg_features = torch.tensor(0) + if self.models['feature_array'] is not None: + reg_features = self.cfg['feature_reg_weight'] * (self.models['feature_array'].data**2).mean() + loss += reg_features + + if self.models['pose_array'] is not None: + pose_array = self.models['pose_array'] + pose_reg = self.cfg['pose_reg_weight']*pose_array.data[1:].norm() + loss += pose_reg + + variation_loss = torch.tensor(0) + + self.optimizer.zero_grad() + self.amp_scaler.scale(loss).backward() + + self.amp_scaler.step(self.optimizer) + self.amp_scaler.update() + if self.global_step%10==0 and self.global_step>0: + self.schedule_lr() + + if self.global_step%self.cfg['i_weights']==0 and self.global_step>0: + self.save_weights(out_file=os.path.join(self.cfg['save_dir'], f'model_latest.pth'), models=self.models) + + if self.global_step % self.cfg['i_img'] == 0 and self.global_step>0: + ids = torch.unique(self.rays[:, self.ray_frame_id_slice]).data.cpu().numpy().astype(int).tolist() + ids.sort() + last = ids[-1] + ids = ids[::max(1,len(ids)//5)] + if last not in ids: + ids.append(last) + canvas = [] + for frame_idx in ids: + rgb, depth, ray_mask, gt_rgb, gt_depth, _ = self.render_images(frame_idx) + mask_vis = (rgb*255*0.2 + ray_mask*0.8).astype(np.uint8) + mask_vis = np.clip(mask_vis,0,255) + rgb = np.concatenate((rgb,gt_rgb),axis=1) + far = self.cfg['far']*self.cfg['sc_factor'] + gt_depth = np.clip(gt_depth, self.cfg['near']*self.cfg['sc_factor'], far) + depth_vis = np.concatenate((to8b(depth / far), to8b(gt_depth / far)), axis=1) + depth_vis = np.tile(depth_vis[...,None],(1,1,3)) + row = np.concatenate((to8b(rgb),depth_vis,mask_vis),axis=1) + canvas.append(row) + canvas = np.concatenate(canvas,axis=0).astype(np.uint8) + dir = f"{self.cfg['save_dir']}/image_step_{self.global_step:07d}.png" + imageio.imwrite(dir,canvas) + if self._run is not None: + self._run.add_artifact(dir) + + + if self.global_step%self.cfg['i_print']==0: + msg = f"Iter: {self.global_step}, valid_samples: {valid_samples.sum()}/{torch.numel(valid_samples)}, valid_rays: {valid_rays.sum()}/{torch.numel(valid_rays)}, " + metrics = { + 'loss':loss.item(), + 'rgb_loss':rgb_loss.item(), + 'rgb0_loss':rgb0_loss.item(), + 'fs_rgb_loss': fs_rgb_loss.item(), + 'depth_loss':depth_loss.item(), + 'depth_loss0':depth_loss0.item(), + 'fs_loss':fs_loss.item(), + 'point_cloud_loss': point_cloud_loss.item(), + 'point_cloud_normal_loss':point_cloud_normal_loss.item(), + 'sdf_loss':sdf_loss.item(), + 'eikonal_loss': eikonal_loss.item(), + "variation_loss": variation_loss.item(), + 'truncation(meter)': self.get_truncation()/self.cfg['sc_factor'], + } + if self.models['pose_array'] is not None: + metrics['pose_reg'] = pose_reg.item() + if 'feature_array' in self.models: + metrics['reg_features'] = reg_features.item() + for k in metrics.keys(): + msg += f"{k}: {metrics[k]:.7f}, " + msg += "\n" + logging.info(msg) + + if self._run is not None: + for k in metrics.keys(): + self._run.log_scalar(k,metrics[k],self.global_step) + + if self.global_step % self.cfg['i_mesh'] == 0 and self.global_step > 0: + with torch.no_grad(): + model = self.models['model_fine'] if self.models['model_fine'] is not None else self.models['model'] + mesh = self.extract_mesh(isolevel=0, voxel_size=self.cfg['mesh_resolution']) + self.mesh = copy.deepcopy(mesh) + if mesh is not None: + dir = os.path.join(self.cfg['save_dir'], f'step_{self.global_step:07d}_mesh_normalized_space.obj') + mesh.export(dir) + if self._run is not None: + self._run.add_artifact(dir) + dir = os.path.join(self.cfg['save_dir'], f'step_{self.global_step:07d}_mesh_real_world.obj') + if self.models['pose_array'] is not None: + _,offset = get_optimized_poses_in_real_world(self.poses,self.models['pose_array'],translation=self.cfg['translation'],sc_factor=self.cfg['sc_factor']) + else: + offset = np.eye(4) + mesh = mesh_to_real_world(mesh,offset,translation=self.cfg['translation'],sc_factor=self.cfg['sc_factor']) + mesh.export(dir) + if self._run is not None: + self._run.add_artifact(dir) + + if self.global_step % self.cfg['i_pose'] == 0 and self.global_step > 0: + if self.models['pose_array'] is not None: + optimized_poses,offset = get_optimized_poses_in_real_world(self.poses,self.models['pose_array'],translation=self.cfg['translation'],sc_factor=self.cfg['sc_factor']) + else: + optimized_poses = self.poses + dir = os.path.join(self.cfg['save_dir'], f'step_{self.global_step:07d}_optimized_poses.txt') + np.savetxt(dir,optimized_poses.reshape(-1,4)) + if self._run is not None: + self._run.add_artifact(dir) + + + def train(self): + set_seed(0) + + for iter in range(self.N_iters): + if iter%(self.N_iters//10)==0: + logging.info(f'train progress {iter}/{self.N_iters}') + batch = next(self.data_loader) + self.train_loop(batch.cuda()) + self.global_step += 1 + + + + @torch.no_grad() + def sample_rays_uniform_occupied_voxels(self,rays_d,depths_in_out,lindisp=False,perturb=False, depths=None, N_samples=None): + '''We first connect the discontinuous boxes for each ray and treat it as uniform sample, then we disconnect into correct boxes + @rays_d: (N_ray,3) + @depths_in_out: Padded tensor each has (N_ray,N_intersect,2) tensor, the time travel of each ray + ''' + N_rays = rays_d.shape[0] + N_intersect = depths_in_out.shape[1] + dirs = rays_d/rays_d.norm(dim=-1,keepdim=True) + + ########### Convert the time to Z + z_in_out = depths_in_out.cuda()*torch.abs(dirs[...,2]).reshape(N_rays,1,1).cuda() + + if depths is not None: + depths = depths.reshape(-1,1) + trunc = self.get_truncation() + valid = (depths>=self.cfg['near']*self.cfg['sc_factor']) & (depths<=self.cfg['far']*self.cfg['sc_factor']).expand(-1,N_intersect) + valid = valid & (z_in_out>0).all(dim=-1) #(N_ray, N_intersect) + z_in_out[valid] = torch.clip(z_in_out[valid], + min=torch.zeros_like(z_in_out[valid]), + max=torch.ones_like(z_in_out[valid])*(depths.reshape(-1,1,1).expand(-1,N_intersect,2)[valid]+trunc)) + + + depths_lens = z_in_out[:,:,1]-z_in_out[:,:,0] #(N_ray,N_intersect) + z_vals_continous = sample_rays_uniform(N_samples,torch.zeros((N_rays,1),device=z_in_out.device).reshape(-1,1),depths_lens.sum(dim=-1).reshape(-1,1),lindisp=lindisp,perturb=perturb) #(N_ray,N_sample) + + ############# Option2 mycuda extension + N_samples = z_vals_continous.shape[1] + z_vals = torch.zeros((N_rays,N_samples), dtype=torch.float, device=rays_d.device) + z_vals = common.sampleRaysUniformOccupiedVoxels(z_in_out.contiguous(),z_vals_continous.contiguous(), z_vals) + z_vals = z_vals.float().to(rays_d.device) #(N_ray,N_sample) + + return z_vals,z_vals_continous + + + def render_rays(self,ray_batch,retraw=True,lindisp=False,perturb=False,raw_noise_std=0.,depth=None, get_normals=False, tf=None): + """Volumetric rendering. + Args: + ray_batch: array of shape [batch_size, ...]. All information necessary + for sampling along a ray, including: ray origin, ray direction, min + dist, max dist, and unit-magnitude viewing direction, frame_ids. + model: function. Model for predicting RGB and density at each point + in space. + N_samples: int. Number of different times to sample along each ray. + retraw: bool. If True, include model's raw, unprocessed predictions. + lindisp: bool. If True, sample linearly in inverse depth rather than in depth. + perturb: float, 0 or 1. If non-zero, each ray is sampled at stratified + random points in time. + N_importance: int. Number of additional times to sample along each ray. + These samples are only passed to model_fine. + model_fine: "fine" network with same spec as model. + raw_noise_std: ... + verbose: bool. If True, print more debugging info. + @depth: depth (from depth image) values of the ray (N_ray,1) + @tf: (N_ray,4,4) glcam in ob, normalized space + Returns: + rgb_map: [num_rays, 3]. Estimated RGB color of a ray. Comes from fine model. + disp_map: [num_rays]. Disparity map. 1 / depth. + acc_map: [num_rays]. Accumulated opacity along each ray. Comes from fine model. + raw: [num_rays, num_samples, 4]. Raw predictions from model. + rgb0: See rgb_map. Output for coarse model. + disp0: See disp_map. Output for coarse model. + acc0: See acc_map. Output for coarse model. + z_std: [num_rays]. Standard deviation of distances along ray for each + sample. + """ + N_rays = ray_batch.shape[0] + rays_d = ray_batch[:,self.ray_dir_slice] + rays_o = torch.zeros_like(rays_d) + viewdirs = rays_d/rays_d.norm(dim=-1,keepdim=True) + + frame_ids = ray_batch[:,self.ray_frame_id_slice].long() + + tf = self.c2w_array[frame_ids] + if self.models['pose_array'] is not None: + tf = self.models['pose_array'].get_matrices(frame_ids)@tf + + rays_o_w = transform_pts(rays_o,tf) + viewdirs_w = (tf[:,:3,:3]@viewdirs[:,None].permute(0,2,1))[:,:3,0] + voxel_size = self.cfg['octree_raytracing_voxel_size']*self.cfg['sc_factor'] + level = int(np.floor(np.log2(2.0/voxel_size))) + near,far,_,depths_in_out = self.octree_m.ray_trace(rays_o_w,viewdirs_w,level=level,debug=0) + z_vals,_ = self.sample_rays_uniform_occupied_voxels(rays_d=viewdirs,depths_in_out=depths_in_out,lindisp=lindisp,perturb=perturb, depths=depth, N_samples=self.cfg['N_samples']) + + if self.cfg['N_samples_around_depth']>0 and depth is not None: #!NOTE only fine when depths are all valid + valid_depth_mask = (depth>=self.cfg['near']*self.cfg['sc_factor']) & (depth<=self.cfg['far']*self.cfg['sc_factor']) + valid_depth_mask = valid_depth_mask.reshape(-1) + trunc = self.get_truncation() + near_depth = depth[valid_depth_mask]-trunc + far_depth = depth[valid_depth_mask]+trunc*self.cfg['neg_trunc_ratio'] + z_vals_around_depth = torch.zeros((N_rays,self.cfg['N_samples_around_depth']), device=ray_batch.device).float() + # if torch.sum(inside_mask)>0: + z_vals_around_depth[valid_depth_mask] = sample_rays_uniform(self.cfg['N_samples_around_depth'],near_depth.reshape(-1,1),far_depth.reshape(-1,1),lindisp=lindisp,perturb=perturb) + invalid_depth_mask = valid_depth_mask==0 + + if invalid_depth_mask.any() and self.cfg['use_octree']: + z_vals_invalid,_ = self.sample_rays_uniform_occupied_voxels(rays_d=viewdirs[invalid_depth_mask],depths_in_out=depths_in_out[invalid_depth_mask],lindisp=lindisp,perturb=perturb, depths=None, N_samples=self.cfg['N_samples_around_depth']) + z_vals_around_depth[invalid_depth_mask] = z_vals_invalid + else: + z_vals_around_depth[invalid_depth_mask] = sample_rays_uniform(self.cfg['N_samples_around_depth'],near[invalid_depth_mask].reshape(-1,1),far[invalid_depth_mask].reshape(-1,1),lindisp=lindisp,perturb=perturb) + + z_vals = torch.cat((z_vals,z_vals_around_depth), dim=-1) + valid_samples = torch.ones(z_vals.shape, dtype=torch.bool, device=ray_batch.device) # During pose update if ray out of box, it becomes invalid + + pts = rays_o[...,None,:] + rays_d[...,None,:] * z_vals[...,:,None] # [N_rays, N_samples, 3] + + deformation = None + raw,normals,valid_samples = self.run_network(pts, viewdirs, frame_ids, tf=tf, valid_samples=valid_samples, get_normals=get_normals) # [N_rays, N_samples, 4] + + rgb_map, weights = self.raw2outputs(raw, z_vals, rays_d, raw_noise_std=raw_noise_std, valid_samples=valid_samples, depth=depth) + + if self.cfg['N_importance'] > 0: + rgb_map_0 = rgb_map + + for iter in range(self.cfg['N_importance_iter']): + z_vals_mid = .5 * (z_vals[...,1:] + z_vals[...,:-1]) + z_samples = sample_pdf(z_vals_mid, weights[...,1:-1], self.cfg['N_importance'], det=(perturb==0.)) + z_samples = z_samples.detach() + valid_samples_importance = torch.ones(z_samples.shape, dtype=torch.bool).to(z_vals.device) + valid_samples_importance[torch.all(valid_samples==0, dim=-1).reshape(-1)] = 0 + + if self.models['model_fine'] is not None and self.models['model_fine']!=self.models['model']: + z_vals, _ = torch.sort(torch.cat([z_vals, z_samples], -1), -1) + pts = rays_o[...,None,:] + rays_d[...,None,:] * z_vals[...,:,None] # [N_rays, N_samples + self.cfg['N_importance'], 3] + raw, normals,valid_samples = self.run_network(pts, viewdirs, frame_ids, tf=tf, valid_samples=valid_samples, get_normals=False) + else: + pts = rays_o[..., None, :] + rays_d[..., None, :] * z_samples[..., :, None] # [N_rays, N_samples + self.cfg['N_importance'], 3] + raw_fine,valid_samples_importance = self.run_network(pts, viewdirs, frame_ids, tf=tf, valid_samples=valid_samples_importance, get_normals=False) + z_vals = torch.cat([z_vals, z_samples], -1) #(N_ray, N_sample) + indices = torch.argsort(z_vals, dim=-1) + z_vals = torch.gather(z_vals,dim=1,index=indices) + raw = torch.gather(torch.cat([raw, raw_fine], dim=1), dim=1, index=indices[...,None].expand(-1,-1,raw.shape[-1])) + valid_samples = torch.cat((valid_samples,valid_samples_importance), dim=-1) + + rgb_map, weights = self.raw2outputs(raw, z_vals, rays_d, raw_noise_std=raw_noise_std,valid_samples=valid_samples) + + ret = {'rgb_map' : rgb_map, 'valid_samples':valid_samples, 'weights':weights, 'z_vals':z_vals} + + if retraw: + ret['raw'] = raw + + if normals is not None: + ret['normals'] = normals + + if deformation is not None: + ret['deformation'] = deformation + + if self.cfg['N_importance'] > 0: + ret['rgb0'] = rgb_map_0 + + return ret + + + def raw2outputs(self, raw, z_vals, rays_d, raw_noise_std=0, valid_samples=None, depth=None): + """Transforms model's predictions to semantically meaningful values. + Args: + raw: [num_rays, num_samples along ray, 4]. Prediction from model. + z_vals: [num_rays, num_samples along ray]. Integration time. + rays_d: [num_rays, 3]. Direction of each ray. + Returns: + rgb_map: [num_rays, 3]. Estimated RGB color of a ray. + disp_map: [num_rays]. Disparity map. Inverse of depth map. + acc_map: [num_rays]. Sum of weights along each ray. + weights: [num_rays, num_samples]. Weights assigned to each sampled color. + depth_map: [num_rays]. Estimated distance to object. + """ + truncation = self.get_truncation() + if depth is not None: + depth = depth.view(-1,1) + + if valid_samples is None: + valid_samples = torch.ones(z_vals.shape, dtype=torch.bool).to(z_vals.device) + + def sdf2weights(sdf): + sdf_from_depth = (depth.view(-1,1)-z_vals)/truncation + weights = torch.sigmoid(sdf_from_depth*self.cfg['sdf_lambda']) * torch.sigmoid(-sdf_from_depth*self.cfg['sdf_lambda']) # This not work well + + invalid = (depth>self.cfg['far']*self.cfg['sc_factor']).reshape(-1) + mask = (z_vals-depth<=truncation*self.cfg['neg_trunc_ratio']) & (z_vals-depth>=-truncation) + weights[~invalid] = weights[~invalid] * mask[~invalid] + weights[invalid] = 0 + + return weights / (weights.sum(dim=-1,keepdim=True) + 1e-10) + + rgb = torch.sigmoid(raw[...,:3]) # [N_rays, N_samples, 3] + weights = sdf2weights(raw[..., 3]) + + weights[valid_samples==0] = 0 + rgb_map = torch.sum(weights[...,None] * rgb, -2) # [N_rays, 3] + + return rgb_map, weights + + + def render(self, rays, depth=None,lindisp=False,perturb=False,raw_noise_std=0.0, get_normals=False): + """Render rays + Args: + H: int. Height of image in pixels. + W: int. Width of image in pixels. + K: float. Focal length of pinhole camera. + chunk: int. Maximum number of rays to process simultaneously. Used to + control maximum memory usage. Does not affect final results. + rays: array of shape [batch_size, 6]. Ray origin and direction for + each example in batch. + ndc: bool. If True, represent ray origin, direction in NDC coordinates. Only true for llff data + near: float or array of shape [batch_size]. Nearest distance for a ray. + far: float or array of shape [batch_size]. Farthest distance for a ray. + @depth: depth values (N_ray,1) + Returns: + rgb_map: [batch_size, 3]. Predicted RGB values for rays. + disp_map: [batch_size]. Disparity map. Inverse of depth. + acc_map: [batch_size]. Accumulated opacity (alpha) along a ray. + extras: dict with everything. + """ + all_ret = self.batchify_rays(rays,depth=depth,lindisp=lindisp,perturb=perturb,raw_noise_std=raw_noise_std, get_normals=get_normals) + + k_extract = ['rgb_map'] + ret_list = [all_ret[k] for k in k_extract] + ret_dict = {k : all_ret[k] for k in all_ret if k not in k_extract} + return ret_list + [ret_dict] + + + + def batchify_rays(self,rays_flat, depth=None,lindisp=False,perturb=False,raw_noise_std=0.0, get_normals=False): + """Render rays in smaller minibatches to avoid OOM. + """ + all_ret = {} + chunk = self.cfg['chunk'] + for i in range(0, rays_flat.shape[0], chunk): + if depth is not None: + cur_depth = depth[i:i+chunk] + else: + cur_depth = None + ret = self.render_rays(rays_flat[i:i+chunk],depth=cur_depth,lindisp=lindisp,perturb=perturb,raw_noise_std=raw_noise_std, get_normals=get_normals) + for k in ret: + if ret[k] is None: + continue + if k not in all_ret: + all_ret[k] = [] + all_ret[k].append(ret[k]) + + all_ret = {k : torch.cat(all_ret[k], 0) for k in all_ret} + return all_ret + + + def run_network(self, inputs, viewdirs, frame_ids=None, tf=None, latent_code=None, valid_samples=None, get_normals=False): + """Prepares inputs and applies network 'fn'. + @inputs: (N_ray,N_sample,3) sampled points on rays in GL camera's frame + @viewdirs: (N_ray,3) unit length vector in camera frame, z-axis backward + @frame_ids: (N_ray) + @tf: (N_ray,4,4) + @latent_code: (N_ray, D) + """ + N_ray,N_sample = inputs.shape[:2] + + if valid_samples is None: + valid_samples = torch.ones((N_ray,N_sample), dtype=torch.bool, device=inputs.device) + + inputs_flat = torch.reshape(inputs, [-1, inputs.shape[-1]]) + + tf_flat = tf[:,None].expand(-1,N_sample,-1,-1).reshape(-1,4,4) + inputs_flat = transform_pts(inputs_flat, tf_flat) + + valid_samples = valid_samples.bool() & (torch.abs(inputs_flat)<=1).all(dim=-1).view(N_ray,N_sample).bool() + + embedded = torch.zeros((inputs_flat.shape[0],self.models['embed_fn'].out_dim), device=inputs_flat.device) + if valid_samples is None: + valid_samples = torch.ones((N_ray,N_sample), dtype=torch.bool, device=inputs_flat.device) + + if get_normals: + if inputs_flat.requires_grad==False: + inputs_flat.requires_grad = True + + with torch.cuda.amp.autocast(enabled=self.cfg['amp']): + if self.cfg['i_embed'] in [3]: + embedded[valid_samples.reshape(-1)], valid_samples_embed = self.models['embed_fn'](inputs_flat[valid_samples.reshape(-1)]) + valid_samples = valid_samples.reshape(-1) + prev_valid_ids = valid_samples.nonzero().reshape(-1) + bad_ids = prev_valid_ids[valid_samples_embed==0] + new_valid_ids = torch.ones((N_ray*N_sample),device=inputs.device).bool() + new_valid_ids[bad_ids] = 0 + valid_samples = valid_samples & new_valid_ids + valid_samples = valid_samples.reshape(N_ray,N_sample).bool() + else: + embedded[valid_samples.reshape(-1)] = self.models['embed_fn'](inputs_flat[valid_samples.reshape(-1)]).to(embedded.dtype) + embedded = embedded.float() + + # Add latent code + if self.models['feature_array'] is not None: + if latent_code is None: + frame_features = self.models['feature_array'](frame_ids) + D = frame_features.shape[-1] + frame_features = frame_features[:,None].expand(-1,N_sample,-1).reshape(-1,D) + else: + D = latent_code.shape[-1] + frame_features = latent_code[:,None].expand(N_ray,N_sample,latent_code.shape[-1]).reshape(-1,D) + embedded = torch.cat([embedded, frame_features], -1) + + # Add view directions + if self.models['embeddirs_fn'] is not None: + input_dirs = (tf[..., :3, :3]@viewdirs[...,None])[...,0] #(N_ray,3) + embedded_dirs = self.models['embeddirs_fn'](input_dirs) + tmp = embedded_dirs.shape[1:] + embedded_dirs_flat = embedded_dirs[:,None].expand(-1,N_sample,*tmp).reshape(-1,*tmp) + embedded = torch.cat([embedded, embedded_dirs_flat], -1) + + outputs_flat = [] + with torch.cuda.amp.autocast(enabled=self.cfg['amp']): + chunk = self.cfg['netchunk'] + for i in range(0,embedded.shape[0],chunk): + out = self.models['model'](embedded[i:i+chunk]) + outputs_flat.append(out) + outputs_flat = torch.cat(outputs_flat, dim=0).float() + outputs = torch.reshape(outputs_flat, list(inputs.shape[:-1]) + [outputs_flat.shape[-1]]).float() + + normals = None + if get_normals: + sdf = outputs[...,-1] + d_output = torch.zeros(sdf.shape, device=sdf.device) + normals = torch.autograd.grad(outputs=sdf,inputs=inputs_flat,grad_outputs=d_output,create_graph=False,retain_graph=True,only_inputs=True,allow_unused=True)[0] + normals = normals.reshape(N_ray,N_sample,3) + + return outputs,normals,valid_samples + + + def run_network_density(self, inputs, get_normals=False): + """Directly query the network w/o pose transformations or deformations (inputs are already in normalized [-1,1]); Particularly used for mesh extraction + @inputs: (N,3) sampled points on rays in GL camera's frame + """ + inputs_flat = torch.reshape(inputs, [-1, inputs.shape[-1]]) + + inputs_flat = torch.clip(inputs_flat,-1,1) + valid_samples = torch.ones((len(inputs_flat)),device=inputs.device).bool() + + if not inputs_flat.requires_grad: + inputs_flat.requires_grad = True + + with torch.cuda.amp.autocast(enabled=self.cfg['amp']): + if self.cfg['i_embed'] in [3]: + embedded, valid_samples_embed = self.models['embed_fn'](inputs_flat) + valid_samples = valid_samples.reshape(-1) + prev_valid_ids = valid_samples.nonzero().reshape(-1) + bad_ids = prev_valid_ids[valid_samples_embed==0] + new_valid_ids = torch.ones((len(inputs_flat)),device=inputs.device).bool() + new_valid_ids[bad_ids] = 0 + valid_samples = valid_samples & new_valid_ids + else: + embedded = self.models['embed_fn'](inputs_flat) + embedded = embedded.float() + input_ch = embedded.shape[-1] + + outputs_flat = [] + with torch.cuda.amp.autocast(enabled=self.cfg['amp']): + chunk = self.cfg['netchunk'] + for i in range(0,embedded.shape[0],chunk): + alpha = self.models['model'].forward_sdf(embedded[i:i+chunk]) #(N,1) + outputs_flat.append(alpha.reshape(-1,1)) + outputs_flat = torch.cat(outputs_flat,dim=0).float() + outputs = torch.reshape(outputs_flat, list(inputs.shape[:-1]) + [outputs_flat.shape[-1]]) + + if get_normals: + d_output = torch.ones_like(outputs, requires_grad=False, device=outputs.device) + normal = torch.autograd.grad(outputs=outputs,inputs=inputs_flat,grad_outputs=d_output,create_graph=False,retain_graph=True,only_inputs=True,allow_unused=True)[0] + outputs = torch.cat((outputs, normal), dim=-1) + + return outputs,valid_samples + + + @torch.no_grad() + def extract_mesh(self, level=None, voxel_size=0.003, isolevel=0.0, return_sigma=False): + voxel_size *= self.cfg['sc_factor'] # in "network space" + + bounds = np.array(self.cfg['bounding_box']).reshape(2,3) + x_min, x_max = bounds[0,0], bounds[1,0] + y_min, y_max = bounds[0,1], bounds[1,1] + z_min, z_max = bounds[0,2], bounds[1,2] + tx = np.arange(x_min+0.5*voxel_size, x_max, voxel_size) + ty = np.arange(y_min+0.5*voxel_size, y_max, voxel_size) + tz = np.arange(z_min+0.5*voxel_size, z_max, voxel_size) + N = len(tx) + query_pts = torch.tensor(np.stack(np.meshgrid(tx, ty, tz, indexing='ij'), -1).astype(np.float32).reshape(-1,3)).float().cuda() + + if self.octree_m is not None: + vox_size = self.cfg['octree_raytracing_voxel_size']*self.cfg['sc_factor'] + level = int(np.floor(np.log2(2.0/vox_size))) + center_ids = self.octree_m.get_center_ids(query_pts, level) + valid = center_ids>=0 + else: + valid = torch.ones(len(query_pts), dtype=bool).cuda() + + logging.info(f'query_pts:{query_pts.shape}, valid:{valid.sum()}') + flat = query_pts[valid] + + sigma = [] + chunk = self.cfg['netchunk'] + for i in range(0,flat.shape[0],chunk): + inputs = flat[i:i+chunk] + with torch.no_grad(): + outputs,valid_samples = self.run_network_density(inputs=inputs) + sigma.append(outputs) + sigma = torch.cat(sigma, dim=0) + sigma_ = torch.ones((N**3)).float().cuda() + sigma_[valid] = sigma.reshape(-1) + sigma = sigma_.reshape(N,N,N).data.cpu().numpy() + + logging.info('Running Marching Cubes') + from skimage import measure + try: + vertices, triangles, normals, values = measure.marching_cubes(sigma, isolevel) + except Exception as e: + logging.info(f"ERROR Marching Cubes {e}") + return None + + logging.info(f'done V:{vertices.shape}, F:{triangles.shape}') + + voxel_size_ndc = np.array([tx[-1] - tx[0], ty[-1] - ty[0], tz[-1] - tz[0]]) / np.array([[tx.shape[0] - 1, ty.shape[0] - 1, tz.shape[0] - 1]]) + offset = np.array([tx[0], ty[0], tz[0]]) + vertices[:, :3] = voxel_size_ndc.reshape(1,3) * vertices[:, :3] + offset.reshape(1,3) + + mesh = trimesh.Trimesh(vertices, triangles, process=False) + + if return_sigma: + return mesh,sigma,query_pts + + return mesh + + + def mesh_texture_from_train_images(self, mesh, rgbs_raw, tex_res=1024): + ''' + @rgbs_raw: raw complete image that was trained on, no black holes + @mesh: in normalized space + ''' + assert len(self.images)==len(rgbs_raw) + + frame_ids = torch.arange(len(self.images)).long().cuda() + tf = self.c2w_array[frame_ids] + if self.models['pose_array'] is not None: + tf = self.models['pose_array'].get_matrices(frame_ids)@tf + tf = tf.data.cpu().numpy() + from offscreen_renderer import ModelRendererOffscreen + + tex_image = torch.zeros((tex_res,tex_res,3)).cuda().float() + weight_tex_image = torch.zeros(tex_image.shape[:-1]).cuda().float() + mesh.merge_vertices() + mesh.remove_duplicate_faces() + mesh = mesh.unwrap() + H,W = tex_image.shape[:2] + uvs_tex = (mesh.visual.uv*np.array([W-1,H-1]).reshape(1,2)) #(n_V,2) + + renderer = ModelRendererOffscreen(cam_K=self.K, H=self.H, W=self.W, zfar=self.cfg['far']*self.cfg['sc_factor']) + + vertices_cuda = torch.from_numpy(mesh.vertices).float().cuda() + faces_cuda = torch.from_numpy(mesh.faces).long().cuda() + face_vertices = torch.zeros((len(faces_cuda),3,3)) + for i in range(3): + face_vertices[:,i] = vertices_cuda[faces_cuda[:,i]] + + all_tri_list= {key: [] for key in range(mesh.triangles.shape[0])} + for i in range(len(rgbs_raw)): + cvcam_in_ob = tf[i]@np.linalg.inv(glcam_in_cvcam) + _, render_depth = renderer.render(mesh=mesh, ob_in_cvcam=np.linalg.inv(cvcam_in_ob)) + xyz_map = depth2xyzmap(render_depth, self.K) + mask = self.masks[i].reshape(self.H,self.W).astype(bool) + valid = (render_depth.reshape(self.H,self.W)>=0.1*self.cfg['sc_factor']) & (mask) + pts = xyz_map[valid].reshape(-1,3) + pts = transform_pts(pts, cvcam_in_ob) + ray_colors = rgbs_raw[i][valid].reshape(-1,3) + locations, distance, index_tri = trimesh.proximity.closest_point(mesh, pts) + normals = mesh.face_normals[index_tri] + for ind_tri, each_tri in enumerate(index_tri): + rays_o = np.zeros(3) + pts = transform_pts(rays_o, cvcam_in_ob) #transform to world space + rays_d = locations[ind_tri]-pts + rays_d /= np.linalg.norm(rays_d) + dot_product = np.dot(-rays_d, normals[ind_tri]) + angle_radians = np.arccos(dot_product) + angle_degrees = np.degrees(angle_radians) + all_tri_list[each_tri].extend([[i,angle_degrees]]) + + _CHOOSE_TOP_N = 4 + all_triangles_dict={} + for k,v in all_tri_list.items(): + if(v): + v.sort(key=lambda x: x[1]) + tep = [i[0] for i in v[:_CHOOSE_TOP_N]] + all_triangles_dict[k]=set(list(tep)) + + + all_tri_visited= {key: 0 for key in range(mesh.triangles.shape[0])} + + logging.info(f"Texture: Texture map computation") + for i in range(len(rgbs_raw)): + print(f'project train_images {i}/{len(rgbs_raw)}') + + cvcam_in_ob = tf[i]@np.linalg.inv(glcam_in_cvcam) + _, render_depth = renderer.render(mesh=mesh, ob_in_cvcam=np.linalg.inv(cvcam_in_ob)) + xyz_map = depth2xyzmap(render_depth, self.K) + mask = self.masks[i].reshape(self.H,self.W).astype(bool) + valid = (render_depth.reshape(self.H,self.W)>=0.1*self.cfg['sc_factor']) & (mask) + pts = xyz_map[valid].reshape(-1,3) + pts = transform_pts(pts, cvcam_in_ob) + ray_colors = rgbs_raw[i][valid].reshape(-1,3) + locations, distance, index_tri = trimesh.proximity.closest_point(mesh, pts) + normals = mesh.face_normals[index_tri] + rays_o = np.zeros((len(normals),3)) + rays_o = transform_pts(rays_o,cvcam_in_ob) + rays_d = locations-rays_o + rays_d /= np.linalg.norm(rays_d,axis=-1).reshape(-1,1) + dots = (normals*(-rays_d)).sum(axis=-1) + ray_weights = np.ones((len(rays_o))) + bool_weights=torch.zeros(len(locations), dtype=torch.bool, device='cuda') + count = 0 + for jj, trtind__ in enumerate(index_tri): + if(i in all_triangles_dict[trtind__] ): + bool_weights[jj]=1 + all_tri_visited[trtind__]=1 + count +=1 + + uvs = torch.zeros((len(locations),2)).cuda().float() + common.rayColorToTextureImageCUDA(torch.from_numpy(mesh.faces).cuda().long(), torch.from_numpy(mesh.vertices).cuda().float(), torch.from_numpy(locations).cuda().float(), torch.from_numpy(index_tri).cuda().long(), torch.from_numpy(uvs_tex).cuda().float(), uvs) + uvs = torch.round(uvs).long() + uvs_flat = uvs[:,1]*(W-1) + uvs[:,0] + uvs_flat_unique, inverse_ids, cnts = torch.unique(uvs_flat, return_counts=True, return_inverse=True) + perm = torch.arange(inverse_ids.size(0)).cuda() + inverse_ids, perm = inverse_ids.flip([0]), perm.flip([0]) + unique_ids = inverse_ids.new_empty(uvs_flat_unique.size(0)).scatter_(0, inverse_ids, perm) + uvs_unique = torch.stack((uvs_flat_unique%(W-1), uvs_flat_unique//(W-1)), dim=-1).reshape(-1,2) + cur_weights= bool_weights[unique_ids].cuda().float() + tex_image[uvs_unique[:,1],uvs_unique[:,0]] += torch.from_numpy(ray_colors).cuda().float()[unique_ids]*cur_weights.reshape(-1,1) + weight_tex_image[uvs_unique[:,1], uvs_unique[:,0]] += cur_weights + + tex_image = tex_image/weight_tex_image[...,None] + tex_image = tex_image.data.cpu().numpy() + tex_image = np.clip(tex_image,0,255).astype(np.uint8) + tex_image = tex_image[::-1].copy() + new_texture = texture_map_interpolation(tex_image) + + mesh.visual = trimesh.visual.texture.TextureVisuals(uv=mesh.visual.uv,image=Image.fromarray(new_texture)) + return mesh diff --git a/third_party/FoundationPose/bundlesdf/run_nerf.py b/third_party/FoundationPose/bundlesdf/run_nerf.py new file mode 100644 index 0000000..5cfea7d --- /dev/null +++ b/third_party/FoundationPose/bundlesdf/run_nerf.py @@ -0,0 +1,115 @@ +# Copyright (c) 2023, NVIDIA CORPORATION. All rights reserved. +# +# NVIDIA CORPORATION and its licensors retain all intellectual property +# and proprietary rights in and to this software, related documentation +# and any modifications thereto. Any use, reproduction, disclosure or +# distribution of this software and related documentation without an express +# license agreement from NVIDIA CORPORATION is strictly prohibited. + + +from nerf_runner import * +code_dir = os.path.dirname(os.path.realpath(__file__)) +sys.path.append(f'{code_dir}/../') +from datareader import * +from bundlesdf.tool import * +import yaml,argparse + + +def run_neural_object_field(cfg, K, rgbs, depths, masks, cam_in_obs, debug=0, save_dir='/home/bowen/debug/foundationpose_bundlesdf'): + rgbs = np.asarray(rgbs) + depths = np.asarray(depths) + masks = np.asarray(masks) + cam_in_obs = np.asarray(cam_in_obs) + glcam_in_obs = cam_in_obs@glcam_in_cvcam + + cfg['save_dir'] = save_dir + os.makedirs(save_dir, exist_ok=True) + + for i,rgb in enumerate(rgbs): + imageio.imwrite(f'{save_dir}/rgb_{i:07d}.png', rgb) + + sc_factor,translation,pcd_real_scale, pcd_normalized = compute_scene_bounds(None,glcam_in_obs, K, use_mask=True,base_dir=save_dir,rgbs=rgbs,depths=depths,masks=masks, eps=cfg['dbscan_eps'], min_samples=cfg['dbscan_eps_min_samples']) + cfg['sc_factor'] = sc_factor + cfg['translation'] = translation + + o3d.io.write_point_cloud(f'{save_dir}/pcd_normalized.ply', pcd_normalized) + + rgbs_, depths_, masks_, normal_maps,poses = preprocess_data(rgbs, depths, masks,normal_maps=None,poses=glcam_in_obs,sc_factor=cfg['sc_factor'],translation=cfg['translation']) + + nerf = NerfRunner(cfg, rgbs_, depths_, masks_, normal_maps=None, poses=poses, K=K, occ_masks=None, build_octree_pcd=pcd_normalized) + nerf.train() + + mesh = nerf.extract_mesh(isolevel=0,voxel_size=cfg['mesh_resolution']) + mesh = nerf.mesh_texture_from_train_images(mesh, rgbs_raw=rgbs, tex_res=1028) + optimized_cvcam_in_obs,offset = get_optimized_poses_in_real_world(poses,nerf.models['pose_array'],cfg['sc_factor'],cfg['translation']) + mesh = mesh_to_real_world(mesh, pose_offset=offset, translation=nerf.cfg['translation'], sc_factor=nerf.cfg['sc_factor']) + return mesh + + +def run_one_ob(base_dir, cfg, use_refined_mask=False): + save_dir = f'{base_dir}/nerf' + os.system(f'rm -rf {save_dir} && mkdir -p {save_dir}') + with open(f'{base_dir}/select_frames.yml','r') as ff: + info = yaml.safe_load(ff) + rgbs = [] + depths = [] + masks = [] + cam_in_obs = [] + color_files = sorted(glob.glob(f'{base_dir}/rgb/*.png')) + K = np.loadtxt(f'{base_dir}/K.txt') + for i,color_file in enumerate(color_files): + rgb = imageio.imread(color_file) + depth = cv2.imread(color_file.replace('rgb','depth_enhanced'), -1)/1e3 + if use_refined_mask: + mask = cv2.imread(color_file.replace('rgb','mask_refined'), -1) + else: + mask = cv2.imread(color_file.replace('rgb','mask'), -1) + cam_in_ob = np.loadtxt(color_file.replace('rgb','cam_in_ob').replace('.png','.txt')).reshape(4,4) + rgbs.append(rgb) + depths.append(depth) + masks.append(mask) + cam_in_obs.append(cam_in_ob) + + mesh = run_neural_object_field(cfg, K, rgbs, depths, masks, cam_in_obs, save_dir=save_dir, debug=0) + return mesh + + +def run_ycbv(): + ob_ids = np.arange(1,22) + code_dir = os.path.dirname(os.path.realpath(__file__)) + with open(f'{code_dir}/config_ycbv.yml','r') as ff: + cfg = yaml.safe_load(ff) + + for ob_id in ob_ids: + base_dir = f'{args.ref_view_dir}/ob_{ob_id:07d}' + mesh = run_one_ob(base_dir=base_dir, cfg=cfg) + out_file = f'{base_dir}/model/model.obj' + os.makedirs(os.path.dirname(out_file), exist_ok=True) + mesh.export(out_file) + + +def run_linemod(): + ob_ids = np.setdiff1d(np.arange(1,16), np.array([7,3])).tolist() + code_dir = os.path.dirname(os.path.realpath(__file__)) + with open(f'{code_dir}/config_linemod.yml','r') as ff: + cfg = yaml.safe_load(ff) + for ob_id in ob_ids: + base_dir = f'{args.ref_view_dir}/ob_{ob_id:07d}' + mesh = run_one_ob(base_dir=base_dir, cfg=cfg, use_refined_mask=True) + out_file = f'{base_dir}/model/model.obj' + os.makedirs(os.path.dirname(out_file), exist_ok=True) + mesh.export(out_file) + logging.info(f"saved to {out_file}") + + +if __name__=="__main__": + parser = argparse.ArgumentParser() + code_dir = os.path.dirname(os.path.realpath(__file__)) + parser.add_argument('--ref_view_dir', type=str, default=f'/mnt/9a72c439-d0a7-45e8-8d20-d7a235d02763/DATASET/YCB_Video/bowen_addon/ref_views_16') + parser.add_argument('--dataset', type=str, default=f'ycbv', help='one of [ycbv/linemod]') + args = parser.parse_args() + + if args.dataset=='ycbv': + run_ycbv() + else: + run_linemod() diff --git a/third_party/FoundationPose/bundlesdf/tool.py b/third_party/FoundationPose/bundlesdf/tool.py new file mode 100644 index 0000000..d52bb9d --- /dev/null +++ b/third_party/FoundationPose/bundlesdf/tool.py @@ -0,0 +1,132 @@ +# Copyright (c) 2023, NVIDIA CORPORATION. All rights reserved. +# +# NVIDIA CORPORATION and its licensors retain all intellectual property +# and proprietary rights in and to this software, related documentation +# and any modifications thereto. Any use, reproduction, disclosure or +# distribution of this software and related documentation without an express +# license agreement from NVIDIA CORPORATION is strictly prohibited. + + +import joblib,json,gzip,pickle +from sklearn.cluster import DBSCAN +import shutil,re,imageio,pdb,os,sys +from Utils import * +import pandas as pd + + +def find_biggest_cluster(pts, eps=0.06, min_samples=1): + dbscan = DBSCAN(eps=eps,min_samples=min_samples,n_jobs=-1) + dbscan.fit(pts) + ids, cnts = np.unique(dbscan.labels_, return_counts=True) + best_id = ids[cnts.argsort()[-1]] + keep_mask = dbscan.labels_==best_id + pts_cluster = pts[keep_mask] + return pts_cluster, keep_mask + + +def compute_translation_scales(pts,max_dim=2,cluster=True, eps=0.06, min_samples=1): + if cluster: + pts, keep_mask = find_biggest_cluster(pts, eps, min_samples) + else: + keep_mask = np.ones((len(pts)), dtype=bool) + max_xyz = pts.max(axis=0) + min_xyz = pts.min(axis=0) + center = (max_xyz+min_xyz)/2 + sc_factor = max_dim/(max_xyz-min_xyz).max() #Normalize to [-1,1] + sc_factor *= 0.9 + translation_cvcam = -center + return translation_cvcam, sc_factor, keep_mask + + +def compute_scene_bounds_worker(color_file,K,glcam_in_world,use_mask,rgb=None,depth=None,mask=None): + if rgb is None: + depth_file = color_file.replace('images','depth_filtered') + mask_file = color_file.replace('images','masks') + rgb = np.array(Image.open(color_file))[...,:3] + depth = cv2.imread(depth_file,-1)/1e3 + xyz_map = depth2xyzmap(depth,K) + valid = depth>=0.1 + if use_mask: + if mask is None: + mask = cv2.imread(mask_file,-1) + valid = valid & (mask>0) + pts = xyz_map[valid].reshape(-1,3) + if len(pts)==0: + return None + colors = rgb[valid].reshape(-1,3) + pcd = toOpen3dCloud(pts,colors) + pcd = pcd.voxel_down_sample(0.01) + pcd, ind = pcd.remove_statistical_outlier(nb_neighbors=30,std_ratio=2.0) + cam_in_world = glcam_in_world@glcam_in_cvcam + pcd.transform(cam_in_world) + return np.asarray(pcd.points).copy(), np.asarray(pcd.colors).copy() + + +def compute_scene_bounds(color_files,glcam_in_worlds,K,use_mask=True,base_dir=None,rgbs=None,depths=None,masks=None,cluster=True, translation_cvcam=None, sc_factor=None, eps=0.06, min_samples=1): + assert color_files is None or rgbs is None + + if base_dir is None: + base_dir = os.path.dirname(color_files[0])+'/../' + + args = [] + if rgbs is not None: + for i in range(len(rgbs)): + args.append((None,K,glcam_in_worlds[i],use_mask,rgbs[i],depths[i],masks[i])) + else: + for i in range(len(color_files)): + args.append((color_files[i],K,glcam_in_worlds[i],use_mask)) + + logging.info(f"compute_scene_bounds_worker start") + ret = joblib.Parallel(n_jobs=10, prefer="threads")(joblib.delayed(compute_scene_bounds_worker)(*arg) for arg in args) + logging.info(f"compute_scene_bounds_worker done") + + pcd_all = None + for r in ret: + if r is None: + continue + if pcd_all is None: + pcd_all = toOpen3dCloud(r[0],r[1]) + else: + pcd_all += toOpen3dCloud(r[0],r[1]) + pcd = pcd_all.voxel_down_sample(eps/5) + + logging.info(f"merge pcd") + + o3d.io.write_point_cloud(f'{base_dir}/naive_fusion.ply',pcd) + pts = np.asarray(pcd.points).copy() + + def make_tf(translation_cvcam, sc_factor): + tf = np.eye(4) + tf[:3,3] = translation_cvcam + tf1 = np.eye(4) + tf1[:3,:3] *= sc_factor + tf = tf1@tf + return tf + + if translation_cvcam is None: + translation_cvcam, sc_factor, keep_mask = compute_translation_scales(pts, cluster=cluster, eps=eps, min_samples=min_samples) + tf = make_tf(translation_cvcam, sc_factor) + else: + tf = make_tf(translation_cvcam, sc_factor) + tmp = copy.deepcopy(pcd) + tmp.transform(tf) + tmp_pts = np.asarray(tmp.points) + keep_mask = (np.abs(tmp_pts)<1).all(axis=-1) + + logging.info(f"compute_translation_scales done") + + pcd = toOpen3dCloud(pts[keep_mask],np.asarray(pcd.colors)[keep_mask]) + o3d.io.write_point_cloud(f"{base_dir}/naive_fusion_biggest_cluster.ply",pcd) + pcd_real_scale = copy.deepcopy(pcd) + print(f'translation_cvcam={translation_cvcam}, sc_factor={sc_factor}') + with open(f'{base_dir}/normalization.yml','w') as ff: + tmp = { + 'translation_cvcam':translation_cvcam.tolist(), + 'sc_factor':float(sc_factor), + } + yaml.dump(tmp,ff) + + pcd.transform(tf) + return sc_factor, translation_cvcam, pcd_real_scale, pcd + + diff --git a/third_party/FoundationPose/check_env.py b/third_party/FoundationPose/check_env.py new file mode 100755 index 0000000..231c021 --- /dev/null +++ b/third_party/FoundationPose/check_env.py @@ -0,0 +1,122 @@ +#!/usr/bin/env python3 +"""Quick sanity checks for a local FoundationPose install (no inference).""" + +from __future__ import annotations + +import argparse +import os +import sys +from pathlib import Path + + +def main() -> int: + parser = argparse.ArgumentParser(description=__doc__) + parser.add_argument( + "--demo-data", + action="store_true", + help="Also check default paths used by run_demo.py (demo_data/mustard0).", + ) + args = parser.parse_args() + + repo_root = Path(__file__).resolve().parent + os.chdir(repo_root) + if str(repo_root) not in sys.path: + sys.path.insert(0, str(repo_root)) + + errors: list[str] = [] + warnings: list[str] = [] + + print(f"Repository: {repo_root}") + print(f"Python: {sys.version.split()[0]}") + + # --- PyTorch / CUDA --- + try: + import torch + + print(f"PyTorch: {torch.__version__}") + print(f"CUDA available: {torch.cuda.is_available()}") + if torch.cuda.is_available(): + print(f"CUDA device: {torch.cuda.get_device_name(0)}") + except Exception as e: + errors.append(f"torch: {e}") + + # --- Core imports (same chain as run_demo / estimater) --- + modules = [ + "pytorch3d", + "nvdiffrast.torch", + "trimesh", + "open3d", + "cv2", + "warp", + ] + for mod in modules: + try: + __import__(mod) + print(f"import {mod}: ok") + except Exception as e: + errors.append(f"import {mod}: {e}") + + # --- Native extension --- + build_dir = repo_root / "mycpp" / "build" + so_files = list(build_dir.glob("mycpp*.so")) + if not so_files: + warnings.append(f"No mycpp extension found under {build_dir} (run bash build_all_conda.sh).") + else: + print(f"mycpp extension: {so_files[0].name}") + + try: + sys.path.insert(0, str(build_dir)) + import mycpp + + print(f"import mycpp: ok ({mycpp})") + except Exception as e: + errors.append(f"import mycpp: {e}") + + try: + import estimater + + print("import estimater: ok") + except Exception as e: + errors.append(f"import estimater: {e}") + + # --- Weights (hard-coded run names in learning/training/predict_*.py) --- + weight_sets = [ + ("scorer", "2024-01-11-20-02-45"), + ("refiner", "2023-10-28-18-33-37"), + ] + weights_root = repo_root / "weights" + for label, stamp in weight_sets: + d = weights_root / stamp + ckpt = d / "model_best.pth" + cfg = d / "config.yml" + missing = [p.name for p in (ckpt, cfg) if not p.is_file()] + if missing: + warnings.append( + f"Weights ({label}, {stamp}): missing {', '.join(missing)} — download per readme Data prepare." + ) + else: + print(f"Weights ({label}, {stamp}): ok") + + # --- Optional demo bundle --- + if args.demo_data: + demo_scene = repo_root / "demo_data" / "mustard0" + demo_mesh = demo_scene / "mesh" / "textured_simple.obj" + if not demo_mesh.is_file(): + warnings.append(f"run_demo demo scene: missing {demo_mesh}") + else: + print(f"run_demo demo_data: ok ({demo_mesh.relative_to(repo_root)})") + + print() + for w in warnings: + print(f"WARNING: {w}") + for e in errors: + print(f"ERROR: {e}") + + # Non-zero only when imports or core checks fail; missing weights/data still exit 0. + if errors: + return 1 + return 0 + + +if __name__ == "__main__": + raise SystemExit(main()) diff --git a/third_party/FoundationPose/datareader.py b/third_party/FoundationPose/datareader.py new file mode 100644 index 0000000..b7a6606 --- /dev/null +++ b/third_party/FoundationPose/datareader.py @@ -0,0 +1,613 @@ +# Copyright (c) 2023, NVIDIA CORPORATION. All rights reserved. +# +# NVIDIA CORPORATION and its licensors retain all intellectual property +# and proprietary rights in and to this software, related documentation +# and any modifications thereto. Any use, reproduction, disclosure or +# distribution of this software and related documentation without an express +# license agreement from NVIDIA CORPORATION is strictly prohibited. + + +from Utils import * +import json,os,sys + + +BOP_LIST = ['lmo','tless','ycbv','hb','tudl','icbin','itodd'] +BOP_DIR = os.getenv('BOP_DIR') + +def get_bop_reader(video_dir, zfar=np.inf): + if 'ycbv' in video_dir or 'YCB' in video_dir: + return YcbVideoReader(video_dir, zfar=zfar) + if 'lmo' in video_dir or 'LINEMOD-O' in video_dir: + return LinemodOcclusionReader(video_dir, zfar=zfar) + if 'tless' in video_dir or 'TLESS' in video_dir: + return TlessReader(video_dir, zfar=zfar) + if 'hb' in video_dir: + return HomebrewedReader(video_dir, zfar=zfar) + if 'tudl' in video_dir: + return TudlReader(video_dir, zfar=zfar) + if 'icbin' in video_dir: + return IcbinReader(video_dir, zfar=zfar) + if 'itodd' in video_dir: + return ItoddReader(video_dir, zfar=zfar) + else: + raise RuntimeError + + +def get_bop_video_dirs(dataset): + if dataset=='ycbv': + video_dirs = sorted(glob.glob(f'{BOP_DIR}/ycbv/test/*')) + elif dataset=='lmo': + video_dirs = sorted(glob.glob(f'{BOP_DIR}/lmo/lmo_test_bop19/test/*')) + elif dataset=='tless': + video_dirs = sorted(glob.glob(f'{BOP_DIR}/tless/tless_test_primesense_bop19/test_primesense/*')) + elif dataset=='hb': + video_dirs = sorted(glob.glob(f'{BOP_DIR}/hb/hb_test_primesense_bop19/test_primesense/*')) + elif dataset=='tudl': + video_dirs = sorted(glob.glob(f'{BOP_DIR}/tudl/tudl_test_bop19/test/*')) + elif dataset=='icbin': + video_dirs = sorted(glob.glob(f'{BOP_DIR}/icbin/icbin_test_bop19/test/*')) + elif dataset=='itodd': + video_dirs = sorted(glob.glob(f'{BOP_DIR}/itodd/itodd_test_bop19/test/*')) + else: + raise RuntimeError + return video_dirs + + + +class YcbineoatReader: + def __init__(self,video_dir, downscale=1, shorter_side=None, zfar=np.inf): + self.video_dir = video_dir + self.downscale = downscale + self.zfar = zfar + self.color_files = sorted(glob.glob(f"{self.video_dir}/rgb/*.png")) + self.K = np.loadtxt(f'{video_dir}/cam_K.txt').reshape(3,3) + self.id_strs = [] + for color_file in self.color_files: + id_str = os.path.basename(color_file).replace('.png','') + self.id_strs.append(id_str) + self.H,self.W = cv2.imread(self.color_files[0]).shape[:2] + + if shorter_side is not None: + self.downscale = shorter_side/min(self.H, self.W) + + self.H = int(self.H*self.downscale) + self.W = int(self.W*self.downscale) + self.K[:2] *= self.downscale + + self.gt_pose_files = sorted(glob.glob(f'{self.video_dir}/annotated_poses/*')) + + self.videoname_to_object = { + 'bleach0': "021_bleach_cleanser", + 'bleach_hard_00_03_chaitanya': "021_bleach_cleanser", + 'cracker_box_reorient': '003_cracker_box', + 'cracker_box_yalehand0': '003_cracker_box', + 'mustard0': '006_mustard_bottle', + 'mustard_easy_00_02': '006_mustard_bottle', + 'sugar_box1': '004_sugar_box', + 'sugar_box_yalehand0': '004_sugar_box', + 'tomato_soup_can_yalehand0': '005_tomato_soup_can', + } + + + def get_video_name(self): + return self.video_dir.split('/')[-1] + + def __len__(self): + return len(self.color_files) + + def get_gt_pose(self,i): + try: + pose = np.loadtxt(self.gt_pose_files[i]).reshape(4,4) + return pose + except: + logging.info("GT pose not found, return None") + return None + + + def get_color(self,i): + color = imageio.imread(self.color_files[i])[...,:3] + color = cv2.resize(color, (self.W,self.H), interpolation=cv2.INTER_NEAREST) + return color + + def get_mask(self,i): + mask = cv2.imread(self.color_files[i].replace('rgb','masks'),-1) + if len(mask.shape)==3: + for c in range(3): + if mask[...,c].sum()>0: + mask = mask[...,c] + break + mask = cv2.resize(mask, (self.W,self.H), interpolation=cv2.INTER_NEAREST).astype(bool).astype(np.uint8) + return mask + + def get_depth(self,i): + depth = cv2.imread(self.color_files[i].replace('rgb','depth'),-1)/1e3 + depth = cv2.resize(depth, (self.W,self.H), interpolation=cv2.INTER_NEAREST) + depth[(depth<0.001) | (depth>=self.zfar)] = 0 + return depth + + + def get_xyz_map(self,i): + depth = self.get_depth(i) + xyz_map = depth2xyzmap(depth, self.K) + return xyz_map + + def get_occ_mask(self,i): + hand_mask_file = self.color_files[i].replace('rgb','masks_hand') + occ_mask = np.zeros((self.H,self.W), dtype=bool) + if os.path.exists(hand_mask_file): + occ_mask = occ_mask | (cv2.imread(hand_mask_file,-1)>0) + + right_hand_mask_file = self.color_files[i].replace('rgb','masks_hand_right') + if os.path.exists(right_hand_mask_file): + occ_mask = occ_mask | (cv2.imread(right_hand_mask_file,-1)>0) + + occ_mask = cv2.resize(occ_mask, (self.W,self.H), interpolation=cv2.INTER_NEAREST) + + return occ_mask.astype(np.uint8) + + def get_gt_mesh(self): + ob_name = self.videoname_to_object[self.get_video_name()] + YCB_VIDEO_DIR = os.getenv('YCB_VIDEO_DIR') + mesh = trimesh.load(f'{YCB_VIDEO_DIR}/models/{ob_name}/textured_simple.obj') + return mesh + + +class BopBaseReader: + def __init__(self, base_dir, zfar=np.inf, resize=1): + self.base_dir = base_dir + self.resize = resize + self.dataset_name = None + self.color_files = sorted(glob.glob(f"{self.base_dir}/rgb/*")) + if len(self.color_files)==0: + self.color_files = sorted(glob.glob(f"{self.base_dir}/gray/*")) + self.zfar = zfar + + self.K_table = {} + with open(f'{self.base_dir}/scene_camera.json','r') as ff: + info = json.load(ff) + for k in info: + self.K_table[f'{int(k):06d}'] = np.array(info[k]['cam_K']).reshape(3,3) + self.bop_depth_scale = info[k]['depth_scale'] + + if os.path.exists(f'{self.base_dir}/scene_gt.json'): + with open(f'{self.base_dir}/scene_gt.json','r') as ff: + self.scene_gt = json.load(ff) + self.scene_gt = copy.deepcopy(self.scene_gt) # Release file handle to be pickle-able by joblib + assert len(self.scene_gt)==len(self.color_files) + else: + self.scene_gt = None + + self.make_id_strs() + + + def make_scene_ob_ids_dict(self): + with open(f'{BOP_DIR}/{self.dataset_name}/test_targets_bop19.json','r') as ff: + self.scene_ob_ids_dict = {} + data = json.load(ff) + for d in data: + if d['scene_id']==self.get_video_id(): + id_str = f"{d['im_id']:06d}" + if id_str not in self.scene_ob_ids_dict: + self.scene_ob_ids_dict[id_str] = [] + self.scene_ob_ids_dict[id_str] += [d['obj_id']]*d['inst_count'] + + + def get_K(self, i_frame): + K = self.K_table[self.id_strs[i_frame]] + if self.resize!=1: + K[:2,:2] *= self.resize + return K + + + def get_video_dir(self): + video_id = int(self.base_dir.rstrip('/').split('/')[-1]) + return video_id + + def make_id_strs(self): + self.id_strs = [] + for i in range(len(self.color_files)): + name = os.path.basename(self.color_files[i]).split('.')[0] + self.id_strs.append(name) + + + def get_instance_ids_in_image(self, i_frame:int): + ob_ids = [] + if self.scene_gt is not None: + name = int(os.path.basename(self.color_files[i_frame]).split('.')[0]) + for k in self.scene_gt[str(name)]: + ob_ids.append(k['obj_id']) + elif self.scene_ob_ids_dict is not None: + return np.array(self.scene_ob_ids_dict[self.id_strs[i_frame]]) + else: + mask_dir = os.path.dirname(self.color_files[0]).replace('rgb','mask_visib') + id_str = self.id_strs[i_frame] + mask_files = sorted(glob.glob(f'{mask_dir}/{id_str}_*.png')) + ob_ids = [] + for mask_file in mask_files: + ob_id = int(os.path.basename(mask_file).split('.')[0].split('_')[1]) + ob_ids.append(ob_id) + ob_ids = np.asarray(ob_ids) + return ob_ids + + + def get_gt_mesh_file(self, ob_id): + raise RuntimeError("You should override this") + + + def get_color(self,i): + color = imageio.imread(self.color_files[i]) + if len(color.shape)==2: + color = np.tile(color[...,None], (1,1,3)) # Gray to RGB + if self.resize!=1: + color = cv2.resize(color, fx=self.resize, fy=self.resize, dsize=None) + return color + + + def get_depth(self,i, filled=False): + if filled: + depth_file = self.color_files[i].replace('rgb','depth_filled') + depth_file = f'{os.path.dirname(depth_file)}/0{os.path.basename(depth_file)}' + depth = cv2.imread(depth_file,-1)/1e3 + else: + depth_file = self.color_files[i].replace('rgb','depth').replace('gray','depth') + depth = cv2.imread(depth_file,-1)*1e-3*self.bop_depth_scale + if self.resize!=1: + depth = cv2.resize(depth, fx=self.resize, fy=self.resize, dsize=None, interpolation=cv2.INTER_NEAREST) + depth[depth<0.001] = 0 + depth[depth>self.zfar] = 0 + return depth + + def get_xyz_map(self,i): + depth = self.get_depth(i) + xyz_map = depth2xyzmap(depth, self.get_K(i)) + return xyz_map + + + def get_mask(self, i_frame:int, ob_id:int, type='mask_visib'): + ''' + @type: mask_visib (only visible part) / mask (projected mask from whole model) + ''' + pos = 0 + name = int(os.path.basename(self.color_files[i_frame]).split('.')[0]) + if self.scene_gt is not None: + for k in self.scene_gt[str(name)]: + if k['obj_id']==ob_id: + break + pos += 1 + mask_file = f'{self.base_dir}/{type}/{name:06d}_{pos:06d}.png' + if not os.path.exists(mask_file): + logging.info(f'{mask_file} not found') + return None + else: + # mask_dir = os.path.dirname(self.color_files[0]).replace('rgb',type) + # mask_file = f'{mask_dir}/{self.id_strs[i_frame]}_{ob_id:06d}.png' + raise RuntimeError + mask = cv2.imread(mask_file, -1) + if self.resize!=1: + mask = cv2.resize(mask, fx=self.resize, fy=self.resize, dsize=None, interpolation=cv2.INTER_NEAREST) + return mask>0 + + + def get_gt_mesh(self, ob_id:int): + mesh_file = self.get_gt_mesh_file(ob_id) + mesh = trimesh.load(mesh_file) + mesh.vertices *= 1e-3 + return mesh + + + def get_model_diameter(self, ob_id): + dir = os.path.dirname(self.get_gt_mesh_file(self.ob_ids[0])) + info_file = f'{dir}/models_info.json' + with open(info_file,'r') as ff: + info = json.load(ff) + return info[str(ob_id)]['diameter']/1e3 + + + + def get_gt_poses(self, i_frame, ob_id): + gt_poses = [] + name = int(self.id_strs[i_frame]) + for i_k, k in enumerate(self.scene_gt[str(name)]): + if k['obj_id']==ob_id: + cur = np.eye(4) + cur[:3,:3] = np.array(k['cam_R_m2c']).reshape(3,3) + cur[:3,3] = np.array(k['cam_t_m2c'])/1e3 + gt_poses.append(cur) + return np.asarray(gt_poses).reshape(-1,4,4) + + + def get_gt_pose(self, i_frame:int, ob_id, mask=None, use_my_correction=False): + ob_in_cam = np.eye(4) + best_iou = -np.inf + best_gt_mask = None + name = int(self.id_strs[i_frame]) + for i_k, k in enumerate(self.scene_gt[str(name)]): + if k['obj_id']==ob_id: + cur = np.eye(4) + cur[:3,:3] = np.array(k['cam_R_m2c']).reshape(3,3) + cur[:3,3] = np.array(k['cam_t_m2c'])/1e3 + if mask is not None: # When multi-instance exists, use mask to determine which one + gt_mask = cv2.imread(f'{self.base_dir}/mask_visib/{self.id_strs[i_frame]}_{i_k:06d}.png', -1).astype(bool) + intersect = (gt_mask*mask).astype(bool) + union = (gt_mask+mask).astype(bool) + iou = float(intersect.sum())/union.sum() + if iou>best_iou: + best_iou = iou + best_gt_mask = gt_mask + ob_in_cam = cur + else: + ob_in_cam = cur + break + + + if use_my_correction: + if 'ycb' in self.base_dir.lower() and 'train_real' in self.color_files[i_frame]: + video_id = self.get_video_id() + if ob_id==1: + if video_id in [12,13,14,17,24]: + ob_in_cam = ob_in_cam@self.symmetry_tfs[ob_id][1] + return ob_in_cam + + + def load_symmetry_tfs(self): + dir = os.path.dirname(self.get_gt_mesh_file(self.ob_ids[0])) + info_file = f'{dir}/models_info.json' + with open(info_file,'r') as ff: + info = json.load(ff) + self.symmetry_tfs = {} + self.symmetry_info_table = {} + for ob_id in self.ob_ids: + self.symmetry_info_table[ob_id] = info[str(ob_id)] + self.symmetry_tfs[ob_id] = symmetry_tfs_from_info(info[str(ob_id)], rot_angle_discrete=5) + self.geometry_symmetry_info_table = copy.deepcopy(self.symmetry_info_table) + + + def get_video_id(self): + return int(self.base_dir.split('/')[-1]) + + +class LinemodOcclusionReader(BopBaseReader): + def __init__(self,base_dir='/mnt/9a72c439-d0a7-45e8-8d20-d7a235d02763/DATASET/LINEMOD-O/lmo_test_all/test/000002', zfar=np.inf): + super().__init__(base_dir, zfar=zfar) + self.dataset_name = 'lmo' + self.K = list(self.K_table.values())[0] + self.ob_ids = [1,5,6,8,9,10,11,12] + self.ob_id_to_names = { + 1: 'ape', + 2: 'benchvise', + 3: 'bowl', + 4: 'camera', + 5: 'water_pour', + 6: 'cat', + 7: 'cup', + 8: 'driller', + 9: 'duck', + 10: 'eggbox', + 11: 'glue', + 12: 'holepuncher', + 13: 'iron', + 14: 'lamp', + 15: 'phone', + } + self.load_symmetry_tfs() + + def get_gt_mesh_file(self, ob_id): + mesh_dir = f'{BOP_DIR}/{self.dataset_name}/models/obj_{ob_id:06d}.ply' + return mesh_dir + + + +class LinemodReader(LinemodOcclusionReader): + def __init__(self, base_dir='/mnt/9a72c439-d0a7-45e8-8d20-d7a235d02763/DATASET/LINEMOD/lm_test_all/test/000001', zfar=np.inf, split=None): + super().__init__(base_dir, zfar=zfar) + self.dataset_name = 'lm' + if split is not None: # train/test + with open(f'/mnt/9a72c439-d0a7-45e8-8d20-d7a235d02763/DATASET/LINEMOD/Linemod_preprocessed/data/{self.get_video_id():02d}/{split}.txt','r') as ff: + lines = ff.read().splitlines() + self.color_files = [] + for line in lines: + id = int(line) + self.color_files.append(f'{self.base_dir}/rgb/{id:06d}.png') + self.make_id_strs() + + self.ob_ids = np.setdiff1d(np.arange(1,16), np.array([7,3])).tolist() # Exclude bowl and mug + self.load_symmetry_tfs() + + + def get_gt_mesh_file(self, ob_id): + root = self.base_dir + while 1: + if os.path.exists(f'{root}/lm_models'): + mesh_dir = f'{root}/lm_models/models/obj_{ob_id:06d}.ply' + break + else: + root = os.path.abspath(f'{root}/../') + return mesh_dir + + + def get_reconstructed_mesh(self, ob_id, ref_view_dir): + mesh = trimesh.load(os.path.abspath(f'{ref_view_dir}/ob_{ob_id:07d}/model/model.obj')) + return mesh + + +class YcbVideoReader(BopBaseReader): + def __init__(self, base_dir, zfar=np.inf): + super().__init__(base_dir, zfar=zfar) + self.dataset_name = 'ycbv' + self.K = list(self.K_table.values())[0] + + self.make_id_strs() + + self.ob_ids = np.arange(1,22).astype(int).tolist() + YCB_VIDEO_DIR = os.getenv('YCB_VIDEO_DIR') + names = sorted(os.listdir(f'{YCB_VIDEO_DIR}/models/')) + self.ob_id_to_names = {} + self.name_to_ob_id = {} + for i,ob_id in enumerate(self.ob_ids): + self.ob_id_to_names[ob_id] = names[i] + self.name_to_ob_id[names[i]] = ob_id + + if 'BOP' not in self.base_dir: + with open(f'{self.base_dir}/../../keyframe.txt','r') as ff: + self.keyframe_lines = ff.read().splitlines() + + self.load_symmetry_tfs() + for ob_id in self.ob_ids: + if ob_id in [1,4,6,18]: # Cylinder + self.geometry_symmetry_info_table[ob_id] = { + 'symmetries_continuous': [ + {'axis':[0,0,1], 'offset':[0,0,0]}, + ], + 'symmetries_discrete': euler_matrix(0, np.pi, 0).reshape(1,4,4).tolist(), + } + elif ob_id in [13]: + self.geometry_symmetry_info_table[ob_id] = { + 'symmetries_continuous': [ + {'axis':[0,0,1], 'offset':[0,0,0]}, + ], + } + elif ob_id in [2,3,9,21]: # Rectangle box + tfs = [] + for rz in [0, np.pi]: + for rx in [0,np.pi]: + for ry in [0,np.pi]: + tfs.append(euler_matrix(rx, ry, rz)) + self.geometry_symmetry_info_table[ob_id] = { + 'symmetries_discrete': np.asarray(tfs).reshape(-1,4,4).tolist(), + } + else: + pass + + def get_gt_mesh_file(self, ob_id): + if 'BOP' in self.base_dir: + mesh_file = os.path.abspath(f'{self.base_dir}/../../ycbv_models/models/obj_{ob_id:06d}.ply') + else: + mesh_file = f'{self.base_dir}/../../ycbv_models/models/obj_{ob_id:06d}.ply' + return mesh_file + + + def get_gt_mesh(self, ob_id:int, get_posecnn_version=False): + if get_posecnn_version: + YCB_VIDEO_DIR = os.getenv('YCB_VIDEO_DIR') + mesh = trimesh.load(f'{YCB_VIDEO_DIR}/models/{self.ob_id_to_names[ob_id]}/textured_simple.obj') + return mesh + mesh_file = self.get_gt_mesh_file(ob_id) + mesh = trimesh.load(mesh_file, process=False) + mesh.vertices *= 1e-3 + tex_file = mesh_file.replace('.ply','.png') + if os.path.exists(tex_file): + from PIL import Image + im = Image.open(tex_file) + uv = mesh.visual.uv + material = trimesh.visual.texture.SimpleMaterial(image=im) + color_visuals = trimesh.visual.TextureVisuals(uv=uv, image=im, material=material) + mesh.visual = color_visuals + return mesh + + + def get_reconstructed_mesh(self, ob_id, ref_view_dir): + mesh = trimesh.load(os.path.abspath(f'{ref_view_dir}/ob_{ob_id:07d}/model/model.obj')) + return mesh + + + def get_transform_reconstructed_to_gt_model(self, ob_id): + out = np.eye(4) + return out + + + def get_visible_cloud(self, ob_id): + file = os.path.abspath(f'{self.base_dir}/../../models/{self.ob_id_to_names[ob_id]}/visible_cloud.ply') + pcd = o3d.io.read_point_cloud(file) + return pcd + + + def is_keyframe(self, i): + color_file = self.color_files[i] + video_id = self.get_video_id() + frame_id = int(os.path.basename(color_file).split('.')[0]) + key = f'{video_id:04d}/{frame_id:06d}' + return (key in self.keyframe_lines) + + + +class TlessReader(BopBaseReader): + def __init__(self, base_dir, zfar=np.inf): + super().__init__(base_dir, zfar=zfar) + self.dataset_name = 'tless' + + self.ob_ids = np.arange(1,31).astype(int).tolist() + self.load_symmetry_tfs() + + + def get_gt_mesh_file(self, ob_id): + mesh_file = f'{self.base_dir}/../../../models_cad/obj_{ob_id:06d}.ply' + return mesh_file + + + def get_gt_mesh(self, ob_id): + mesh = trimesh.load(self.get_gt_mesh_file(ob_id)) + mesh.vertices *= 1e-3 + mesh = trimesh_add_pure_colored_texture(mesh, color=np.ones((3))*200) + return mesh + + +class HomebrewedReader(BopBaseReader): + def __init__(self, base_dir, zfar=np.inf): + super().__init__(base_dir, zfar=zfar) + self.dataset_name = 'hb' + self.ob_ids = np.arange(1,34).astype(int).tolist() + self.load_symmetry_tfs() + self.make_scene_ob_ids_dict() + + + def get_gt_mesh_file(self, ob_id): + mesh_file = f'{self.base_dir}/../../../hb_models/models/obj_{ob_id:06d}.ply' + return mesh_file + + + def get_gt_pose(self, i_frame:int, ob_id, use_my_correction=False): + logging.info("WARN HomeBrewed doesn't have GT pose") + return np.eye(4) + + + +class ItoddReader(BopBaseReader): + def __init__(self, base_dir, zfar=np.inf): + super().__init__(base_dir, zfar=zfar) + self.dataset_name = 'itodd' + self.make_id_strs() + + self.ob_ids = np.arange(1,29).astype(int).tolist() + self.load_symmetry_tfs() + self.make_scene_ob_ids_dict() + + + def get_gt_mesh_file(self, ob_id): + mesh_file = f'{self.base_dir}/../../../itodd_models/models/obj_{ob_id:06d}.ply' + return mesh_file + + +class IcbinReader(BopBaseReader): + def __init__(self, base_dir, zfar=np.inf): + super().__init__(base_dir, zfar=zfar) + self.dataset_name = 'icbin' + self.ob_ids = np.arange(1,3).astype(int).tolist() + self.load_symmetry_tfs() + + def get_gt_mesh_file(self, ob_id): + mesh_file = f'{self.base_dir}/../../../icbin_models/models/obj_{ob_id:06d}.ply' + return mesh_file + + +class TudlReader(BopBaseReader): + def __init__(self, base_dir, zfar=np.inf): + super().__init__(base_dir, zfar=zfar) + self.dataset_name = 'tudl' + self.ob_ids = np.arange(1,4).astype(int).tolist() + self.load_symmetry_tfs() + + def get_gt_mesh_file(self, ob_id): + mesh_file = f'{self.base_dir}/../../../tudl_models/models/obj_{ob_id:06d}.ply' + return mesh_file + + diff --git a/third_party/FoundationPose/docker/run_container.sh b/third_party/FoundationPose/docker/run_container.sh new file mode 100644 index 0000000..8503571 --- /dev/null +++ b/third_party/FoundationPose/docker/run_container.sh @@ -0,0 +1,3 @@ +docker rm -f foundationpose +DIR=$(pwd)/../ +xhost + && docker run --gpus all --env NVIDIA_DISABLE_REQUIRE=1 -it --network=host --name foundationpose --cap-add=SYS_PTRACE --security-opt seccomp=unconfined -v $DIR:$DIR -v /home:/home -v /mnt:/mnt -v /tmp/.X11-unix:/tmp/.X11-unix -v /tmp:/tmp --ipc=host -e DISPLAY=${DISPLAY} -e GIT_INDEX_FILE foundationpose:latest bash -c "cd $DIR && bash" diff --git a/third_party/FoundationPose/environment.yml b/third_party/FoundationPose/environment.yml new file mode 100644 index 0000000..79b3f4a --- /dev/null +++ b/third_party/FoundationPose/environment.yml @@ -0,0 +1,17 @@ +# Conda-only dependencies (compilers, CMake, Eigen, Boost, pybind11). +# After creating the env, install PyTorch and GPU extensions as described in readme.md +# (PyTorch index URL may need to match your CUDA driver), then: +# pip install -r requirements.txt +# bash build_all_conda.sh +name: foundationpose +channels: + - conda-forge +dependencies: + - python=3.11 + - pip + - cmake + - ninja + - eigen + - boost-cpp + - pybind11 + - cxx-compiler diff --git a/third_party/FoundationPose/estimater.py b/third_party/FoundationPose/estimater.py new file mode 100644 index 0000000..e147d56 --- /dev/null +++ b/third_party/FoundationPose/estimater.py @@ -0,0 +1,270 @@ +# Copyright (c) 2023, NVIDIA CORPORATION. All rights reserved. +# +# NVIDIA CORPORATION and its licensors retain all intellectual property +# and proprietary rights in and to this software, related documentation +# and any modifications thereto. Any use, reproduction, disclosure or +# distribution of this software and related documentation without an express +# license agreement from NVIDIA CORPORATION is strictly prohibited. + + +from Utils import * +from datareader import * +import itertools +from learning.training.predict_score import * +from learning.training.predict_pose_refine import * +import yaml + + +class FoundationPose: + def __init__(self, model_pts, model_normals, symmetry_tfs=None, mesh=None, scorer:ScorePredictor=None, refiner:PoseRefinePredictor=None, glctx=None, debug=0, debug_dir='/home/bowen/debug/novel_pose_debug/'): + self.gt_pose = None + self.ignore_normal_flip = True + self.debug = debug + self.debug_dir = debug_dir + os.makedirs(debug_dir, exist_ok=True) + + self.reset_object(model_pts, model_normals, symmetry_tfs=symmetry_tfs, mesh=mesh) + self.make_rotation_grid(min_n_views=40, inplane_step=60) + + self.glctx = glctx + + if scorer is not None: + self.scorer = scorer + else: + self.scorer = ScorePredictor() + + if refiner is not None: + self.refiner = refiner + else: + self.refiner = PoseRefinePredictor() + + self.pose_last = None # Used for tracking; per the centered mesh + + + def reset_object(self, model_pts, model_normals, symmetry_tfs=None, mesh=None): + max_xyz = mesh.vertices.max(axis=0) + min_xyz = mesh.vertices.min(axis=0) + self.model_center = (min_xyz+max_xyz)/2 + if mesh is not None: + self.mesh_ori = mesh.copy() + mesh = mesh.copy() + mesh.vertices = mesh.vertices - self.model_center.reshape(1,3) + + model_pts = mesh.vertices + self.diameter = compute_mesh_diameter(model_pts=mesh.vertices, n_sample=10000) + self.vox_size = max(self.diameter/20.0, 0.003) + logging.info(f'self.diameter:{self.diameter}, vox_size:{self.vox_size}') + self.dist_bin = self.vox_size/2 + self.angle_bin = 20 # Deg + pcd = toOpen3dCloud(model_pts, normals=model_normals) + pcd = pcd.voxel_down_sample(self.vox_size) + self.max_xyz = np.asarray(pcd.points).max(axis=0) + self.min_xyz = np.asarray(pcd.points).min(axis=0) + self.pts = torch.tensor(np.asarray(pcd.points), dtype=torch.float32, device='cuda') + self.normals = F.normalize(torch.tensor(np.asarray(pcd.normals), dtype=torch.float32, device='cuda'), dim=-1) + logging.info(f'self.pts:{self.pts.shape}') + self.mesh_path = None + self.mesh = mesh + if self.mesh is not None: + self.mesh_path = f'/tmp/{uuid.uuid4()}.obj' + self.mesh.export(self.mesh_path) + self.mesh_tensors = make_mesh_tensors(self.mesh) + + if symmetry_tfs is None: + self.symmetry_tfs = torch.eye(4).float().cuda()[None] + else: + self.symmetry_tfs = torch.as_tensor(symmetry_tfs, device='cuda', dtype=torch.float) + + logging.info("reset done") + + + + def get_tf_to_centered_mesh(self): + tf_to_center = torch.eye(4, dtype=torch.float, device='cuda') + tf_to_center[:3,3] = -torch.as_tensor(self.model_center, device='cuda', dtype=torch.float) + return tf_to_center + + + def to_device(self, s='cuda:0'): + for k in self.__dict__: + self.__dict__[k] = self.__dict__[k] + if torch.is_tensor(self.__dict__[k]) or isinstance(self.__dict__[k], nn.Module): + logging.info(f"Moving {k} to device {s}") + self.__dict__[k] = self.__dict__[k].to(s) + for k in self.mesh_tensors: + logging.info(f"Moving {k} to device {s}") + self.mesh_tensors[k] = self.mesh_tensors[k].to(s) + if self.refiner is not None: + self.refiner.model.to(s) + if self.scorer is not None: + self.scorer.model.to(s) + if self.glctx is not None: + self.glctx = dr.RasterizeCudaContext(s) + + + + def make_rotation_grid(self, min_n_views=40, inplane_step=60): + cam_in_obs = sample_views_icosphere(n_views=min_n_views) + logging.info(f'cam_in_obs:{cam_in_obs.shape}') + rot_grid = [] + for i in range(len(cam_in_obs)): + for inplane_rot in np.deg2rad(np.arange(0, 360, inplane_step)): + cam_in_ob = cam_in_obs[i] + R_inplane = euler_matrix(0,0,inplane_rot) + cam_in_ob = cam_in_ob@R_inplane + ob_in_cam = np.linalg.inv(cam_in_ob) + rot_grid.append(ob_in_cam) + + rot_grid = np.asarray(rot_grid) + logging.info(f"rot_grid:{rot_grid.shape}") + rot_grid = mycpp.cluster_poses(30, 99999, rot_grid, self.symmetry_tfs.data.cpu().numpy()) + rot_grid = np.asarray(rot_grid) + logging.info(f"after cluster, rot_grid:{rot_grid.shape}") + self.rot_grid = torch.as_tensor(rot_grid, device='cuda', dtype=torch.float) + logging.info(f"self.rot_grid: {self.rot_grid.shape}") + + + def generate_random_pose_hypo(self, K, rgb, depth, mask, scene_pts=None): + ''' + @scene_pts: torch tensor (N,3) + ''' + ob_in_cams = self.rot_grid.clone() + center = self.guess_translation(depth=depth, mask=mask, K=K) + ob_in_cams[:,:3,3] = torch.tensor(center, device='cuda', dtype=torch.float).reshape(1,3) + return ob_in_cams + + + def guess_translation(self, depth, mask, K): + vs,us = np.where(mask>0) + if len(us)==0: + logging.info(f'mask is all zero') + return np.zeros((3)) + uc = (us.min()+us.max())/2.0 + vc = (vs.min()+vs.max())/2.0 + valid = mask.astype(bool) & (depth>=0.001) + if not valid.any(): + logging.info(f"valid is empty") + return np.zeros((3)) + + zc = np.median(depth[valid]) + center = (np.linalg.inv(K)@np.asarray([uc,vc,1]).reshape(3,1))*zc + + if self.debug>=2: + pcd = toOpen3dCloud(center.reshape(1,3)) + o3d.io.write_point_cloud(f'{self.debug_dir}/init_center.ply', pcd) + + return center.reshape(3) + + + def register(self, K, rgb, depth, ob_mask, ob_id=None, glctx=None, iteration=5): + '''Copmute pose from given pts to self.pcd + @pts: (N,3) np array, downsampled scene points + ''' + set_seed(0) + logging.info('Welcome') + + if self.glctx is None: + if glctx is None: + self.glctx = dr.RasterizeCudaContext() + # self.glctx = dr.RasterizeGLContext() + else: + self.glctx = glctx + + depth = erode_depth(depth, radius=2, device='cuda') + depth = bilateral_filter_depth(depth, radius=2, device='cuda') + + if self.debug>=2: + xyz_map = depth2xyzmap(depth, K) + valid = xyz_map[...,2]>=0.001 + pcd = toOpen3dCloud(xyz_map[valid], rgb[valid]) + o3d.io.write_point_cloud(f'{self.debug_dir}/scene_raw.ply',pcd) + cv2.imwrite(f'{self.debug_dir}/ob_mask.png', (ob_mask*255.0).clip(0,255)) + + normal_map = None + valid = (depth>=0.001) & (ob_mask>0) + if valid.sum()<4: + logging.info(f'valid too small, return') + pose = np.eye(4) + pose[:3,3] = self.guess_translation(depth=depth, mask=ob_mask, K=K) + return pose + + if self.debug>=2: + imageio.imwrite(f'{self.debug_dir}/color.png', rgb) + cv2.imwrite(f'{self.debug_dir}/depth.png', (depth*1000).astype(np.uint16)) + valid = xyz_map[...,2]>=0.001 + pcd = toOpen3dCloud(xyz_map[valid], rgb[valid]) + o3d.io.write_point_cloud(f'{self.debug_dir}/scene_complete.ply',pcd) + + self.H, self.W = depth.shape[:2] + self.K = K + self.ob_id = ob_id + self.ob_mask = ob_mask + + poses = self.generate_random_pose_hypo(K=K, rgb=rgb, depth=depth, mask=ob_mask, scene_pts=None) + poses = poses.data.cpu().numpy() + logging.info(f'poses:{poses.shape}') + center = self.guess_translation(depth=depth, mask=ob_mask, K=K) + + poses = torch.as_tensor(poses, device='cuda', dtype=torch.float) + poses[:,:3,3] = torch.as_tensor(center.reshape(1,3), device='cuda') + + add_errs = self.compute_add_err_to_gt_pose(poses) + logging.info(f"after viewpoint, add_errs min:{add_errs.min()}") + + xyz_map = depth2xyzmap(depth, K) + poses, vis = self.refiner.predict(mesh=self.mesh, mesh_tensors=self.mesh_tensors, rgb=rgb, depth=depth, K=K, ob_in_cams=poses.data.cpu().numpy(), normal_map=normal_map, xyz_map=xyz_map, glctx=self.glctx, mesh_diameter=self.diameter, iteration=iteration, get_vis=self.debug>=2) + if vis is not None: + imageio.imwrite(f'{self.debug_dir}/vis_refiner.png', vis) + + scores, vis = self.scorer.predict(mesh=self.mesh, rgb=rgb, depth=depth, K=K, ob_in_cams=poses.data.cpu().numpy(), normal_map=normal_map, mesh_tensors=self.mesh_tensors, glctx=self.glctx, mesh_diameter=self.diameter, get_vis=self.debug>=2) + if vis is not None: + imageio.imwrite(f'{self.debug_dir}/vis_score.png', vis) + + add_errs = self.compute_add_err_to_gt_pose(poses) + logging.info(f"final, add_errs min:{add_errs.min()}") + + ids = torch.as_tensor(scores).argsort(descending=True) + logging.info(f'sort ids:{ids}') + scores = scores[ids] + poses = poses[ids] + + logging.info(f'sorted scores:{scores}') + + best_pose = poses[0]@self.get_tf_to_centered_mesh() + self.pose_last = poses[0] + self.best_id = ids[0] + + self.poses = poses + self.scores = scores + + return best_pose.data.cpu().numpy() + + + def compute_add_err_to_gt_pose(self, poses): + ''' + @poses: wrt. the centered mesh + ''' + return -torch.ones(len(poses), device='cuda', dtype=torch.float) + + + def track_one(self, rgb, depth, K, iteration, extra={}): + if self.pose_last is None: + logging.info("Please init pose by register first") + raise RuntimeError + logging.info("Welcome") + + depth = torch.as_tensor(depth, device='cuda', dtype=torch.float) + depth = erode_depth(depth, radius=2, device='cuda') + depth = bilateral_filter_depth(depth, radius=2, device='cuda') + logging.info("depth processing done") + + xyz_map = depth2xyzmap_batch(depth[None], torch.as_tensor(K, dtype=torch.float, device='cuda')[None], zfar=np.inf)[0] + + pose, vis = self.refiner.predict(mesh=self.mesh, mesh_tensors=self.mesh_tensors, rgb=rgb, depth=depth, K=K, ob_in_cams=self.pose_last.reshape(1,4,4).data.cpu().numpy(), normal_map=None, xyz_map=xyz_map, mesh_diameter=self.diameter, glctx=self.glctx, iteration=iteration, get_vis=self.debug>=2) + logging.info("pose done") + if self.debug>=2: + extra['vis'] = vis + self.pose_last = pose + return (pose@self.get_tf_to_centered_mesh()).data.cpu().numpy().reshape(4,4) + + diff --git a/third_party/FoundationPose/learning/datasets/h5_dataset.py b/third_party/FoundationPose/learning/datasets/h5_dataset.py new file mode 100644 index 0000000..3bb51a5 --- /dev/null +++ b/third_party/FoundationPose/learning/datasets/h5_dataset.py @@ -0,0 +1,220 @@ +# Copyright (c) 2023, NVIDIA CORPORATION. All rights reserved. +# +# NVIDIA CORPORATION and its licensors retain all intellectual property +# and proprietary rights in and to this software, related documentation +# and any modifications thereto. Any use, reproduction, disclosure or +# distribution of this software and related documentation without an express +# license agreement from NVIDIA CORPORATION is strictly prohibited. + + + +import os,sys,h5py,bisect,io,json +code_dir = os.path.dirname(os.path.realpath(__file__)) +sys.path.append(f'{code_dir}/../../../../') +from Utils import * +from learning.datasets.pose_dataset import * + + + + +class PairH5Dataset(torch.utils.data.Dataset): + def __init__(self, cfg, h5_file, mode='train', max_num_key=None, cache_data=None): + self.cfg = cfg + self.h5_file = h5_file + self.mode = mode + + logging.info(f"self.h5_file:{self.h5_file}") + self.n_perturb = None + self.H_ori = None + self.W_ori = None + self.cache_data = cache_data + + if self.mode=='test': + pass + else: + self.object_keys = [] + key_file = h5_file.replace('.h5','_keys.pkl') + if os.path.exists(key_file): + with open(key_file, 'rb') as ff: + self.object_keys = pickle.load(ff) + logging.info(f'object_keys loaded#:{len(self.object_keys)} from {key_file}') + if max_num_key is not None: + self.object_keys = self.object_keys[:max_num_key] + else: + with h5py.File(h5_file, 'r', libver='latest') as hf: + for k in hf: + self.object_keys.append(k) + if max_num_key is not None and len(self.object_keys)>=max_num_key: + logging.info("break due to max_num_key") + break + + logging.info(f'self.object_keys#:{len(self.object_keys)}, max_num_key:{max_num_key}') + + with h5py.File(h5_file, 'r', libver='latest') as hf: + group = hf[self.object_keys[0]] + cnt = 0 + for k_perturb in group: + if 'i_perturb' in k_perturb: + cnt += 1 + if 'crop_ratio' in group[k_perturb]: + self.cfg['crop_ratio'] = float(group[k_perturb]['crop_ratio'][()]) + if self.H_ori is None: + if 'H_ori' in group[k_perturb]: + self.H_ori = int(group[k_perturb]['H_ori'][()]) + self.W_ori = int(group[k_perturb]['W_ori'][()]) + else: + self.H_ori = 540 + self.W_ori = 720 + self.n_perturb = cnt + logging.info(f'self.n_perturb:{self.n_perturb}') + + + def __len__(self): + if self.mode=='test': + return 1 + return len(self.object_keys) + + + + def transform_depth_to_xyzmap(self, batch:BatchPoseData, H_ori, W_ori, bound=1): + bs = len(batch.rgbAs) + H,W = batch.rgbAs.shape[-2:] + mesh_radius = batch.mesh_diameters.cuda()/2 + tf_to_crops = batch.tf_to_crops.cuda() + crop_to_oris = batch.tf_to_crops.inverse().cuda() #(B,3,3) + batch.poseA = batch.poseA.cuda() + batch.Ks = batch.Ks.cuda() + + if batch.xyz_mapAs is None: + depthAs_ori = kornia.geometry.transform.warp_perspective(batch.depthAs.cuda().expand(bs,-1,-1,-1), crop_to_oris, dsize=(H_ori, W_ori), mode='nearest', align_corners=False) + batch.xyz_mapAs = depth2xyzmap_batch(depthAs_ori[:,0], batch.Ks, zfar=np.inf).permute(0,3,1,2) #(B,3,H,W) + batch.xyz_mapAs = kornia.geometry.transform.warp_perspective(batch.xyz_mapAs, tf_to_crops, dsize=(H,W), mode='nearest', align_corners=False) + batch.xyz_mapAs = batch.xyz_mapAs.cuda() + if self.cfg['normalize_xyz']: + invalid = batch.xyz_mapAs[:,2:3]<0.001 + batch.xyz_mapAs = batch.xyz_mapAs-batch.poseA[:,:3,3].reshape(bs,3,1,1) + if self.cfg['normalize_xyz']: + batch.xyz_mapAs *= 1/mesh_radius.reshape(bs,1,1,1) + invalid = invalid.expand(bs,3,-1,-1) | (torch.abs(batch.xyz_mapAs)>=2) + batch.xyz_mapAs[invalid.expand(bs,3,-1,-1)] = 0 + + if batch.xyz_mapBs is None: + depthBs_ori = kornia.geometry.transform.warp_perspective(batch.depthBs.cuda().expand(bs,-1,-1,-1), crop_to_oris, dsize=(H_ori, W_ori), mode='nearest', align_corners=False) + batch.xyz_mapBs = depth2xyzmap_batch(depthBs_ori[:,0], batch.Ks, zfar=np.inf).permute(0,3,1,2) #(B,3,H,W) + batch.xyz_mapBs = kornia.geometry.transform.warp_perspective(batch.xyz_mapBs, tf_to_crops, dsize=(H,W), mode='nearest', align_corners=False) + batch.xyz_mapBs = batch.xyz_mapBs.cuda() + if self.cfg['normalize_xyz']: + invalid = batch.xyz_mapBs[:,2:3]<0.001 + batch.xyz_mapBs = batch.xyz_mapBs-batch.poseA[:,:3,3].reshape(bs,3,1,1) + if self.cfg['normalize_xyz']: + batch.xyz_mapBs *= 1/mesh_radius.reshape(bs,1,1,1) + invalid = invalid.expand(bs,3,-1,-1) | (torch.abs(batch.xyz_mapBs)>=2) + batch.xyz_mapBs[invalid.expand(bs,3,-1,-1)] = 0 + + return batch + + + + def transform_batch(self, batch:BatchPoseData, H_ori, W_ori, bound=1): + '''Transform the batch before feeding to the network + !NOTE the H_ori, W_ori could be different at test time from the training data, and needs to be set + ''' + bs = len(batch.rgbAs) + batch.rgbAs = batch.rgbAs.cuda().float()/255.0 + batch.rgbBs = batch.rgbBs.cuda().float()/255.0 + + batch = self.transform_depth_to_xyzmap(batch, H_ori, W_ori, bound=bound) + return batch + + + + +class TripletH5Dataset(PairH5Dataset): + def __init__(self, cfg, h5_file, mode, max_num_key=None, cache_data=None): + super().__init__(cfg, h5_file, mode, max_num_key, cache_data=cache_data) + + + def transform_depth_to_xyzmap(self, batch:BatchPoseData, H_ori, W_ori, bound=1): + bs = len(batch.rgbAs) + H,W = batch.rgbAs.shape[-2:] + mesh_radius = batch.mesh_diameters.cuda()/2 + tf_to_crops = batch.tf_to_crops.cuda() + crop_to_oris = batch.tf_to_crops.inverse().cuda() #(B,3,3) + batch.poseA = batch.poseA.cuda() + batch.Ks = batch.Ks.cuda() + + if batch.xyz_mapAs is None: + depthAs_ori = kornia.geometry.transform.warp_perspective(batch.depthAs.cuda().expand(bs,-1,-1,-1), crop_to_oris, dsize=(H_ori, W_ori), mode='nearest', align_corners=False) + batch.xyz_mapAs = depth2xyzmap_batch(depthAs_ori[:,0], batch.Ks, zfar=np.inf).permute(0,3,1,2) #(B,3,H,W) + batch.xyz_mapAs = kornia.geometry.transform.warp_perspective(batch.xyz_mapAs, tf_to_crops, dsize=(H,W), mode='nearest', align_corners=False) + batch.xyz_mapAs = batch.xyz_mapAs.cuda() + invalid = batch.xyz_mapAs[:,2:3]<0.1 + batch.xyz_mapAs = (batch.xyz_mapAs-batch.poseA[:,:3,3].reshape(bs,3,1,1)) + if self.cfg['normalize_xyz']: + batch.xyz_mapAs *= 1/mesh_radius.reshape(bs,1,1,1) + invalid = invalid.expand(bs,3,-1,-1) | (torch.abs(batch.xyz_mapAs)>=2) + batch.xyz_mapAs[invalid.expand(bs,3,-1,-1)] = 0 + + if batch.xyz_mapBs is None: + depthBs_ori = kornia.geometry.transform.warp_perspective(batch.depthBs.cuda().expand(bs,-1,-1,-1), crop_to_oris, dsize=(H_ori, W_ori), mode='nearest', align_corners=False) + batch.xyz_mapBs = depth2xyzmap_batch(depthBs_ori[:,0], batch.Ks, zfar=np.inf).permute(0,3,1,2) #(B,3,H,W) + batch.xyz_mapBs = kornia.geometry.transform.warp_perspective(batch.xyz_mapBs, tf_to_crops, dsize=(H,W), mode='nearest', align_corners=False) + batch.xyz_mapBs = batch.xyz_mapBs.cuda() + invalid = batch.xyz_mapBs[:,2:3]<0.1 + batch.xyz_mapBs = (batch.xyz_mapBs-batch.poseA[:,:3,3].reshape(bs,3,1,1)) + if self.cfg['normalize_xyz']: + batch.xyz_mapBs *= 1/mesh_radius.reshape(bs,1,1,1) + invalid = invalid.expand(bs,3,-1,-1) | (torch.abs(batch.xyz_mapBs)>=2) + batch.xyz_mapBs[invalid.expand(bs,3,-1,-1)] = 0 + + return batch + + + def transform_batch(self, batch:BatchPoseData, H_ori, W_ori, bound=1): + bs = len(batch.rgbAs) + batch.rgbAs = batch.rgbAs.cuda().float()/255.0 + batch.rgbBs = batch.rgbBs.cuda().float()/255.0 + + batch = self.transform_depth_to_xyzmap(batch, H_ori, W_ori, bound=bound) + return batch + + + +class ScoreMultiPairH5Dataset(TripletH5Dataset): + def __init__(self, cfg, h5_file, mode, max_num_key=None, cache_data=None): + super().__init__(cfg, h5_file, mode, max_num_key, cache_data=cache_data) + if mode in ['train', 'val']: + self.cfg['train_num_pair'] = self.n_perturb + + +class PoseRefinePairH5Dataset(PairH5Dataset): + def __init__(self, cfg, h5_file, mode='train', max_num_key=None, cache_data=None): + super().__init__(cfg=cfg, h5_file=h5_file, mode=mode, max_num_key=max_num_key, cache_data=cache_data) + + if mode!='test': + with h5py.File(h5_file, 'r', libver='latest') as hf: + group = hf[self.object_keys[0]] + for key_perturb in group: + depthA = imageio.imread(group[key_perturb]['depthA'][()]) + depthB = imageio.imread(group[key_perturb]['depthB'][()]) + self.cfg['n_view'] = min(self.cfg['n_view'], depthA.shape[1]//depthB.shape[1]) + logging.info(f'n_view:{self.cfg["n_view"]}') + self.trans_normalizer = group[key_perturb]['trans_normalizer'][()] + if isinstance(self.trans_normalizer, np.ndarray): + self.trans_normalizer = self.trans_normalizer.tolist() + self.rot_normalizer = group[key_perturb]['rot_normalizer'][()]/180.0*np.pi + logging.info(f'self.trans_normalizer:{self.trans_normalizer}, self.rot_normalizer:{self.rot_normalizer}') + break + + + def transform_batch(self, batch:BatchPoseData, H_ori, W_ori, bound=1): + '''Transform the batch before feeding to the network + !NOTE the H_ori, W_ori could be different at test time from the training data, and needs to be set + ''' + bs = len(batch.rgbAs) + batch.rgbAs = batch.rgbAs.cuda().float()/255.0 + batch.rgbBs = batch.rgbBs.cuda().float()/255.0 + + batch = self.transform_depth_to_xyzmap(batch, H_ori, W_ori, bound=bound) + return batch + diff --git a/third_party/FoundationPose/learning/datasets/pose_dataset.py b/third_party/FoundationPose/learning/datasets/pose_dataset.py new file mode 100644 index 0000000..8e76ee4 --- /dev/null +++ b/third_party/FoundationPose/learning/datasets/pose_dataset.py @@ -0,0 +1,135 @@ +# Copyright (c) 2023, NVIDIA CORPORATION. All rights reserved. +# +# NVIDIA CORPORATION and its licensors retain all intellectual property +# and proprietary rights in and to this software, related documentation +# and any modifications thereto. Any use, reproduction, disclosure or +# distribution of this software and related documentation without an express +# license agreement from NVIDIA CORPORATION is strictly prohibited. + + +import os,sys +from dataclasses import dataclass +from typing import Iterator, List, Optional, Set, Union +import numpy as np +import torch +code_dir = os.path.dirname(os.path.realpath(__file__)) +sys.path.append(f'{code_dir}/../../../../') +from Utils import * + + +@dataclass +class PoseData: + """ + rgb: (h, w, 3) uint8 + depth: (bsz, h, w) float32 + bbox: (4, ) int + K: (3, 3) float32 + """ + rgb: np.ndarray = None + bbox: np.ndarray = None + K: np.ndarray = None + depth: Optional[np.ndarray] = None + object_data = None + mesh_diameter: float = None + rgbA: np.ndarray = None + rgbB: np.ndarray = None + depthA: np.ndarray = None + depthB: np.ndarray = None + maskA = None + maskB = None + poseA: np.ndarray = None #(4,4) + target: float = None + + def __init__(self, rgbA=None, rgbB=None, depthA=None, depthB=None, maskA=None, maskB=None, normalA=None, normalB=None, xyz_mapA=None, xyz_mapB=None, poseA=None, poseB=None, K=None, target=None, mesh_diameter=None, tf_to_crop=None, crop_mask=None, model_pts=None, label=None, model_scale=None): + self.rgbA = rgbA #(H,W,3) or (H,W*n_view,3) when multiview + self.rgbB = rgbB + self.depthA = depthA + self.depthB = depthB + self.poseA = poseA + self.poseB = poseB + self.maskA = maskA + self.maskB = maskB + self.crop_mask = crop_mask + self.normalA = normalA + self.normalB = normalB + self.xyz_mapA = xyz_mapA + self.xyz_mapB = xyz_mapB + self.target = target + self.K = K + self.mesh_diameter = mesh_diameter + self.tf_to_crop = tf_to_crop + self.model_pts = model_pts + self.label = label + self.model_scale = model_scale + + +@dataclass +class BatchPoseData: + """ + rgbs: (bsz, 3, h, w) torch tensor uint8 + depths: (bsz, h, w) float32 + bboxes: (bsz, 4) int + K: (bsz, 3, 3) float32 + """ + + rgbs: torch.Tensor = None + object_datas = None + bboxes: torch.Tensor = None + K: torch.Tensor = None + depths: Optional[torch.Tensor] = None + rgbAs = None + rgbBs = None + depthAs = None + depthBs = None + normalAs = None + normalBs = None + poseA = None #(B,4,4) + poseB = None + targets = None # Score targets, torch tensor (B) + + def __init__(self, rgbAs=None, rgbBs=None, depthAs=None, depthBs=None, normalAs=None, normalBs=None, maskAs=None, maskBs=None, poseA=None, poseB=None, xyz_mapAs=None, xyz_mapBs=None, tf_to_crops=None, Ks=None, crop_masks=None, model_pts=None, mesh_diameters=None, labels=None): + self.rgbAs = rgbAs + self.rgbBs = rgbBs + self.depthAs = depthAs + self.depthBs = depthBs + self.normalAs = normalAs + self.normalBs = normalBs + self.poseA = poseA + self.poseB = poseB + self.maskAs = maskAs + self.maskBs = maskBs + self.xyz_mapAs = xyz_mapAs + self.xyz_mapBs = xyz_mapBs + self.tf_to_crops = tf_to_crops + self.crop_masks = crop_masks + self.Ks = Ks + self.model_pts = model_pts + self.mesh_diameters = mesh_diameters + self.labels = labels + + + def pin_memory(self) -> "BatchPoseData": + for k in self.__dict__: + if self.__dict__[k] is not None: + try: + self.__dict__[k] = self.__dict__[k].pin_memory() + except Exception as e: + pass + return self + + def cuda(self): + for k in self.__dict__: + if self.__dict__[k] is not None: + try: + self.__dict__[k] = self.__dict__[k].cuda() + except: + pass + return self + + def select_by_indices(self, ids): + out = BatchPoseData() + for k in self.__dict__: + if self.__dict__[k] is not None: + out.__dict__[k] = self.__dict__[k][ids.to(self.__dict__[k].device)] + return out + diff --git a/third_party/FoundationPose/learning/models/network_modules.py b/third_party/FoundationPose/learning/models/network_modules.py new file mode 100644 index 0000000..a0e4246 --- /dev/null +++ b/third_party/FoundationPose/learning/models/network_modules.py @@ -0,0 +1,138 @@ +# Copyright (c) 2023, NVIDIA CORPORATION. All rights reserved. +# +# NVIDIA CORPORATION and its licensors retain all intellectual property +# and proprietary rights in and to this software, related documentation +# and any modifications thereto. Any use, reproduction, disclosure or +# distribution of this software and related documentation without an express +# license agreement from NVIDIA CORPORATION is strictly prohibited. + + +import os,sys,copy,math,tqdm +import numpy as np +dir_path = os.path.dirname(os.path.realpath(__file__)) +sys.path.append(dir_path) +import torch.nn.functional as F +import torch +import torch.nn as nn +import time +import cv2 +sys.path.append(f'{dir_path}/../../../../') +from Utils import * + + + +class ConvBN(nn.Module): + def __init__(self, C_in, C_out, kernel_size=3, stride=1, groups=1, bias=True,dilation=1,): + super().__init__() + padding = (kernel_size - 1) // 2 + self.net = nn.Sequential( + nn.Conv2d(C_in, C_out, kernel_size, stride, padding, groups=groups, bias=bias,dilation=dilation), + nn.BatchNorm2d(C_out), + ) + + def forward(self, x): + return self.net(x) + + +class ConvBNReLU(nn.Module): + def __init__(self, C_in, C_out, kernel_size=3, stride=1, groups=1, bias=True,dilation=1, norm_layer=nn.BatchNorm2d): + super().__init__() + padding = (kernel_size - 1) // 2 + layers = [ + nn.Conv2d(C_in, C_out, kernel_size, stride, padding, groups=groups, bias=bias,dilation=dilation), + ] + if norm_layer is not None: + layers.append(norm_layer(C_out)) + layers.append(nn.ReLU(inplace=True)) + self.net = nn.Sequential(*layers) + + def forward(self, x): + return self.net(x) + + +class ConvPadding(nn.Module): + def __init__(self,C_in, C_out, kernel_size=3, stride=1, groups=1, bias=True,dilation=1): + super(ConvPadding, self).__init__() + padding = (kernel_size - 1) // 2 + self.conv = nn.Conv2d(C_in, C_out, kernel_size, stride, padding, groups=groups, bias=bias,dilation=dilation) + + def forward(self,x): + return self.conv(x) + + +def conv3x3(in_planes, out_planes, stride=1, groups=1, dilation=1, bias=False): + """3x3 convolution with padding""" + return nn.Conv2d(in_planes, out_planes, kernel_size=3, stride=stride, + padding=dilation, groups=groups, bias=bias, dilation=dilation) + + +def conv1x1(in_planes, out_planes, stride=1, bias=False): + """1x1 convolution""" + return nn.Conv2d(in_planes, out_planes, kernel_size=1, stride=stride, bias=bias) + +class ResnetBasicBlock(nn.Module): + __constants__ = ['downsample'] + + def __init__(self, inplanes, planes, stride=1, downsample=None, groups=1, base_width=64, dilation=1, norm_layer=nn.BatchNorm2d, bias=False): + super().__init__() + self.norm_layer = norm_layer + if groups != 1 or base_width != 64: + raise ValueError('BasicBlock only supports groups=1 and base_width=64') + if dilation > 1: + raise NotImplementedError("Dilation > 1 not supported in BasicBlock") + # Both self.conv1 and self.downsample layers downsample the input when stride != 1 + self.conv1 = conv3x3(inplanes, planes, stride,bias=bias) + if self.norm_layer is not None: + self.bn1 = norm_layer(planes) + self.relu = nn.ReLU(inplace=True) + self.conv2 = conv3x3(planes, planes,bias=bias) + if self.norm_layer is not None: + self.bn2 = norm_layer(planes) + self.downsample = downsample + self.stride = stride + + def forward(self, x): + identity = x + + out = self.conv1(x) + if self.norm_layer is not None: + out = self.bn1(out) + out = self.relu(out) + + out = self.conv2(out) + if self.norm_layer is not None: + out = self.bn2(out) + + if self.downsample is not None: + identity = self.downsample(x) + out += identity + out = self.relu(out) + + return out + + + +class PositionalEmbedding(nn.Module): + def __init__(self, d_model, max_len=512): + super().__init__() + + # Compute the positional encodings once in log space. + pe = torch.zeros(max_len, d_model).float() + pe.require_grad = False + + position = torch.arange(0, max_len).float().unsqueeze(1) #(N,1) + div_term = (torch.arange(0, d_model, 2).float() * -(math.log(10000.0) / d_model)).exp()[None] + + pe[:, 0::2] = torch.sin(position * div_term) #(N, d_model/2) + pe[:, 1::2] = torch.cos(position * div_term) + + pe = pe.unsqueeze(0) + self.register_buffer('pe', pe) #(1, max_len, D) + + + def forward(self, x): + ''' + @x: (B,N,D) + ''' + return x + self.pe[:, :x.size(1)] + diff --git a/third_party/FoundationPose/learning/models/refine_network.py b/third_party/FoundationPose/learning/models/refine_network.py new file mode 100644 index 0000000..c2e399e --- /dev/null +++ b/third_party/FoundationPose/learning/models/refine_network.py @@ -0,0 +1,93 @@ +# Copyright (c) 2023, NVIDIA CORPORATION. All rights reserved. +# +# NVIDIA CORPORATION and its licensors retain all intellectual property +# and proprietary rights in and to this software, related documentation +# and any modifications thereto. Any use, reproduction, disclosure or +# distribution of this software and related documentation without an express +# license agreement from NVIDIA CORPORATION is strictly prohibited. + + +import os,sys +import numpy as np +code_dir = os.path.dirname(os.path.realpath(__file__)) +sys.path.append(code_dir) +sys.path.append(f'{code_dir}/../../../../') +from Utils import * +import torch.nn.functional as F +import torch +import torch.nn as nn +import cv2 +from functools import partial +from network_modules import * +from Utils import * + + + +class RefineNet(nn.Module): + def __init__(self, cfg=None, c_in=4, n_view=1): + super().__init__() + self.cfg = cfg + if self.cfg.use_BN: + norm_layer = nn.BatchNorm2d + norm_layer1d = nn.BatchNorm1d + else: + norm_layer = None + norm_layer1d = None + + self.encodeA = nn.Sequential( + ConvBNReLU(C_in=c_in,C_out=64,kernel_size=7,stride=2, norm_layer=norm_layer), + ConvBNReLU(C_in=64,C_out=128,kernel_size=3,stride=2, norm_layer=norm_layer), + ResnetBasicBlock(128,128,bias=True, norm_layer=norm_layer), + ResnetBasicBlock(128,128,bias=True, norm_layer=norm_layer), + ) + + self.encodeAB = nn.Sequential( + ResnetBasicBlock(256,256,bias=True, norm_layer=norm_layer), + ResnetBasicBlock(256,256,bias=True, norm_layer=norm_layer), + ConvBNReLU(256,512,kernel_size=3,stride=2, norm_layer=norm_layer), + ResnetBasicBlock(512,512,bias=True, norm_layer=norm_layer), + ResnetBasicBlock(512,512,bias=True, norm_layer=norm_layer), + ) + + embed_dim = 512 + num_heads = 4 + self.pos_embed = PositionalEmbedding(d_model=embed_dim, max_len=400) + + self.trans_head = nn.Sequential( + nn.TransformerEncoderLayer(d_model=embed_dim, nhead=num_heads, dim_feedforward=512, batch_first=True), + nn.Linear(512, 3), + ) + + if self.cfg['rot_rep']=='axis_angle': + rot_out_dim = 3 + elif self.cfg['rot_rep']=='6d': + rot_out_dim = 6 + else: + raise RuntimeError + self.rot_head = nn.Sequential( + nn.TransformerEncoderLayer(d_model=embed_dim, nhead=num_heads, dim_feedforward=512, batch_first=True), + nn.Linear(512, rot_out_dim), + ) + + + def forward(self, A, B): + """ + @A: (B,C,H,W) + """ + bs = len(A) + output = {} + + x = torch.cat([A,B], dim=0) + x = self.encodeA(x) + a = x[:bs] + b = x[bs:] + + ab = torch.cat((a,b),1).contiguous() + ab = self.encodeAB(ab) #(B,C,H,W) + + ab = self.pos_embed(ab.reshape(bs, ab.shape[1], -1).permute(0,2,1)) + + output['trans'] = self.trans_head(ab).mean(dim=1) + output['rot'] = self.rot_head(ab).mean(dim=1) + + return output diff --git a/third_party/FoundationPose/learning/models/score_network.py b/third_party/FoundationPose/learning/models/score_network.py new file mode 100644 index 0000000..fbf362e --- /dev/null +++ b/third_party/FoundationPose/learning/models/score_network.py @@ -0,0 +1,90 @@ +# Copyright (c) 2023, NVIDIA CORPORATION. All rights reserved. +# +# NVIDIA CORPORATION and its licensors retain all intellectual property +# and proprietary rights in and to this software, related documentation +# and any modifications thereto. Any use, reproduction, disclosure or +# distribution of this software and related documentation without an express +# license agreement from NVIDIA CORPORATION is strictly prohibited. + + +import os,sys +import numpy as np +code_dir = os.path.dirname(os.path.realpath(__file__)) +sys.path.append(code_dir) +sys.path.append(f'{code_dir}/../../../../') +from Utils import * +from functools import partial +import torch.nn.functional as F +import torch +import torch.nn as nn +import cv2 +from network_modules import * +from Utils import * + + + + +class ScoreNetMultiPair(nn.Module): + def __init__(self, cfg=None, c_in=4): + super().__init__() + self.cfg = cfg + if self.cfg.use_BN: + norm_layer = nn.BatchNorm2d + else: + norm_layer = None + + self.encoderA = nn.Sequential( + ConvBNReLU(C_in=c_in,C_out=64,kernel_size=7,stride=2, norm_layer=norm_layer), + ConvBNReLU(C_in=64,C_out=128,kernel_size=3,stride=2, norm_layer=norm_layer), + ResnetBasicBlock(128,128,bias=True, norm_layer=norm_layer), + ResnetBasicBlock(128,128,bias=True, norm_layer=norm_layer), + ) + + self.encoderAB = nn.Sequential( + ResnetBasicBlock(256,256,bias=True, norm_layer=norm_layer), + ResnetBasicBlock(256,256,bias=True, norm_layer=norm_layer), + ConvBNReLU(256,512,kernel_size=3,stride=2, norm_layer=norm_layer), + ResnetBasicBlock(512,512,bias=True, norm_layer=norm_layer), + ResnetBasicBlock(512,512,bias=True, norm_layer=norm_layer), + ) + + embed_dim = 512 + num_heads = 4 + self.att = nn.MultiheadAttention(embed_dim=embed_dim, num_heads=num_heads, bias=True, batch_first=True) + self.att_cross = nn.MultiheadAttention(embed_dim=embed_dim, num_heads=num_heads, bias=True, batch_first=True) + + self.pos_embed = PositionalEmbedding(d_model=embed_dim, max_len=400) + self.linear = nn.Linear(embed_dim, 1) + + + def extract_feat(self, A, B): + """ + @A: (B*L,C,H,W) L is num of pairs + """ + bs = A.shape[0] # B*L + + x = torch.cat([A,B], dim=0) + x = self.encoderA(x) + a = x[:bs] + b = x[bs:] + ab = torch.cat((a,b), dim=1) + ab = self.encoderAB(ab) + ab = self.pos_embed(ab.reshape(bs, ab.shape[1], -1).permute(0,2,1)) + ab, _ = self.att(ab, ab, ab) + return ab.mean(dim=1).reshape(bs,-1) + + + def forward(self, A, B, L): + """ + @A: (B*L,C,H,W) L is num of pairs + @L: num of pairs + """ + output = {} + bs = A.shape[0]//L + feats = self.extract_feat(A, B) #(B*L, C) + x = feats.reshape(bs,L,-1) + x, _ = self.att_cross(x, x, x) + + output['score_logit'] = self.linear(x).reshape(bs,L) # (B,L) + + return output diff --git a/third_party/FoundationPose/learning/training/predict_pose_refine.py b/third_party/FoundationPose/learning/training/predict_pose_refine.py new file mode 100644 index 0000000..fabb888 --- /dev/null +++ b/third_party/FoundationPose/learning/training/predict_pose_refine.py @@ -0,0 +1,296 @@ +# Copyright (c) 2023, NVIDIA CORPORATION. All rights reserved. +# +# NVIDIA CORPORATION and its licensors retain all intellectual property +# and proprietary rights in and to this software, related documentation +# and any modifications thereto. Any use, reproduction, disclosure or +# distribution of this software and related documentation without an express +# license agreement from NVIDIA CORPORATION is strictly prohibited. + + +import functools +import os,sys,kornia +import time +code_dir = os.path.dirname(os.path.realpath(__file__)) +sys.path.append(f'{code_dir}/../../') +import numpy as np +import torch +from omegaconf import OmegaConf +from learning.models.refine_network import RefineNet +from learning.datasets.h5_dataset import * +from Utils import * +from datareader import * + + + +@torch.inference_mode() +def make_crop_data_batch(render_size, ob_in_cams, mesh, rgb, depth, K, crop_ratio, xyz_map, normal_map=None, mesh_diameter=None, cfg=None, glctx=None, mesh_tensors=None, dataset:PoseRefinePairH5Dataset=None): + logging.info("Welcome make_crop_data_batch") + H,W = depth.shape[:2] + args = [] + method = 'box_3d' + tf_to_crops = compute_crop_window_tf_batch(pts=mesh.vertices, H=H, W=W, poses=ob_in_cams, K=K, crop_ratio=crop_ratio, out_size=(render_size[1], render_size[0]), method=method, mesh_diameter=mesh_diameter) + + logging.info("make tf_to_crops done") + + B = len(ob_in_cams) + poseA = torch.as_tensor(ob_in_cams, dtype=torch.float, device='cuda') + + bs = 512 + rgb_rs = [] + depth_rs = [] + normal_rs = [] + xyz_map_rs = [] + + bbox2d_crop = torch.as_tensor(np.array([0, 0, cfg['input_resize'][0]-1, cfg['input_resize'][1]-1]).reshape(2,2), device='cuda', dtype=torch.float) + bbox2d_ori = transform_pts(bbox2d_crop, tf_to_crops.inverse()).reshape(-1,4) + + for b in range(0,len(poseA),bs): + extra = {} + rgb_r, depth_r, normal_r = nvdiffrast_render(K=K, H=H, W=W, ob_in_cams=poseA[b:b+bs], context='cuda', get_normal=cfg['use_normal'], glctx=glctx, mesh_tensors=mesh_tensors, output_size=cfg['input_resize'], bbox2d=bbox2d_ori[b:b+bs], use_light=True, extra=extra) + rgb_rs.append(rgb_r) + depth_rs.append(depth_r[...,None]) + normal_rs.append(normal_r) + xyz_map_rs.append(extra['xyz_map']) + rgb_rs = torch.cat(rgb_rs, dim=0).permute(0,3,1,2) * 255 + depth_rs = torch.cat(depth_rs, dim=0).permute(0,3,1,2) #(B,1,H,W) + xyz_map_rs = torch.cat(xyz_map_rs, dim=0).permute(0,3,1,2) #(B,3,H,W) + Ks = torch.as_tensor(K, device='cuda', dtype=torch.float).reshape(1,3,3) + if cfg['use_normal']: + normal_rs = torch.cat(normal_rs, dim=0).permute(0,3,1,2) #(B,3,H,W) + + logging.info("render done") + + rgbBs = kornia.geometry.transform.warp_perspective(torch.as_tensor(rgb, dtype=torch.float, device='cuda').permute(2,0,1)[None].expand(B,-1,-1,-1), tf_to_crops, dsize=render_size, mode='bilinear', align_corners=False) + if rgb_rs.shape[-2:]!=cfg['input_resize']: + rgbAs = kornia.geometry.transform.warp_perspective(rgb_rs, tf_to_crops, dsize=render_size, mode='bilinear', align_corners=False) + else: + rgbAs = rgb_rs + if xyz_map_rs.shape[-2:]!=cfg['input_resize']: + xyz_mapAs = kornia.geometry.transform.warp_perspective(xyz_map_rs, tf_to_crops, dsize=render_size, mode='nearest', align_corners=False) + else: + xyz_mapAs = xyz_map_rs + xyz_mapBs = kornia.geometry.transform.warp_perspective(torch.as_tensor(xyz_map, device='cuda', dtype=torch.float).permute(2,0,1)[None].expand(B,-1,-1,-1), tf_to_crops, dsize=render_size, mode='nearest', align_corners=False) #(B,3,H,W) + + if cfg['use_normal']: + normalAs = kornia.geometry.transform.warp_perspective(normal_rs, tf_to_crops, dsize=render_size, mode='nearest', align_corners=False) + normalBs = kornia.geometry.transform.warp_perspective(torch.as_tensor(normal_map, dtype=torch.float, device='cuda').permute(2,0,1)[None].expand(B,-1,-1,-1), tf_to_crops, dsize=render_size, mode='nearest', align_corners=False) + else: + normalAs = None + normalBs = None + + logging.info("warp done") + + mesh_diameters = torch.ones((len(rgbAs)), dtype=torch.float, device='cuda')*mesh_diameter + pose_data = BatchPoseData(rgbAs=rgbAs, rgbBs=rgbBs, depthAs=None, depthBs=None, normalAs=normalAs, normalBs=normalBs, poseA=poseA, poseB=None, xyz_mapAs=xyz_mapAs, xyz_mapBs=xyz_mapBs, tf_to_crops=tf_to_crops, Ks=Ks, mesh_diameters=mesh_diameters) + pose_data = dataset.transform_batch(batch=pose_data, H_ori=H, W_ori=W, bound=1) + + logging.info("pose batch data done") + + return pose_data + + + +class PoseRefinePredictor: + def __init__(self,): + logging.info("welcome") + self.amp = True + self.run_name = "2023-10-28-18-33-37" + model_name = 'model_best.pth' + code_dir = os.path.dirname(os.path.realpath(__file__)) + ckpt_dir = f'{code_dir}/../../weights/{self.run_name}/{model_name}' + + self.cfg = OmegaConf.load(f'{code_dir}/../../weights/{self.run_name}/config.yml') + + self.cfg['ckpt_dir'] = ckpt_dir + self.cfg['enable_amp'] = True + + ########## Defaults, to be backward compatible + if 'use_normal' not in self.cfg: + self.cfg['use_normal'] = False + if 'use_mask' not in self.cfg: + self.cfg['use_mask'] = False + if 'use_BN' not in self.cfg: + self.cfg['use_BN'] = False + if 'c_in' not in self.cfg: + self.cfg['c_in'] = 4 + if 'crop_ratio' not in self.cfg or self.cfg['crop_ratio'] is None: + self.cfg['crop_ratio'] = 1.2 + if 'n_view' not in self.cfg: + self.cfg['n_view'] = 1 + if 'trans_rep' not in self.cfg: + self.cfg['trans_rep'] = 'tracknet' + if 'rot_rep' not in self.cfg: + self.cfg['rot_rep'] = 'axis_angle' + if 'zfar' not in self.cfg: + self.cfg['zfar'] = 3 + if 'normalize_xyz' not in self.cfg: + self.cfg['normalize_xyz'] = False + if isinstance(self.cfg['zfar'], str) and 'inf' in self.cfg['zfar'].lower(): + self.cfg['zfar'] = np.inf + if 'normal_uint8' not in self.cfg: + self.cfg['normal_uint8'] = False + logging.info(f"self.cfg: \n {OmegaConf.to_yaml(self.cfg)}") + + self.dataset = PoseRefinePairH5Dataset(cfg=self.cfg, h5_file='', mode='test') + self.model = RefineNet(cfg=self.cfg, c_in=self.cfg['c_in']).cuda() + + logging.info(f"Using pretrained model from {ckpt_dir}") + ckpt = torch.load(ckpt_dir) + if 'model' in ckpt: + ckpt = ckpt['model'] + self.model.load_state_dict(ckpt) + + self.model.cuda().eval() + logging.info("init done") + self.last_trans_update = None + self.last_rot_update = None + + + @torch.inference_mode() + def predict(self, rgb, depth, K, ob_in_cams, xyz_map, normal_map=None, get_vis=False, mesh=None, mesh_tensors=None, glctx=None, mesh_diameter=None, iteration=5): + ''' + @rgb: np array (H,W,3) + @ob_in_cams: np array (N,4,4) + ''' + torch.set_default_tensor_type('torch.cuda.FloatTensor') + logging.info(f'ob_in_cams:{ob_in_cams.shape}') + tf_to_center = np.eye(4) + ob_centered_in_cams = ob_in_cams + mesh_centered = mesh + + logging.info(f'self.cfg.use_normal:{self.cfg.use_normal}') + if not self.cfg.use_normal: + normal_map = None + + crop_ratio = self.cfg['crop_ratio'] + logging.info(f"trans_normalizer:{self.cfg['trans_normalizer']}, rot_normalizer:{self.cfg['rot_normalizer']}") + bs = 1024 + + B_in_cams = torch.as_tensor(ob_centered_in_cams, device='cuda', dtype=torch.float) + + + if mesh_tensors is None: + mesh_tensors = make_mesh_tensors(mesh_centered) + + rgb_tensor = torch.as_tensor(rgb, device='cuda', dtype=torch.float) + depth_tensor = torch.as_tensor(depth, device='cuda', dtype=torch.float) + xyz_map_tensor = torch.as_tensor(xyz_map, device='cuda', dtype=torch.float) + trans_normalizer = self.cfg['trans_normalizer'] + if not isinstance(trans_normalizer, float): + trans_normalizer = torch.as_tensor(list(trans_normalizer), device='cuda', dtype=torch.float).reshape(1,3) + + for _ in range(iteration): + logging.info("making cropped data") + pose_data = make_crop_data_batch(self.cfg.input_resize, B_in_cams, mesh_centered, rgb_tensor, depth_tensor, K, crop_ratio=crop_ratio, normal_map=normal_map, xyz_map=xyz_map_tensor, cfg=self.cfg, glctx=glctx, mesh_tensors=mesh_tensors, dataset=self.dataset, mesh_diameter=mesh_diameter) + B_in_cams = [] + for b in range(0, pose_data.rgbAs.shape[0], bs): + A = torch.cat([pose_data.rgbAs[b:b+bs].cuda(), pose_data.xyz_mapAs[b:b+bs].cuda()], dim=1).float() + B = torch.cat([pose_data.rgbBs[b:b+bs].cuda(), pose_data.xyz_mapBs[b:b+bs].cuda()], dim=1).float() + logging.info("forward start") + with torch.cuda.amp.autocast(enabled=self.amp): + output = self.model(A,B) + for k in output: + output[k] = output[k].float() + logging.info("forward done") + if self.cfg['trans_rep']=='tracknet': + if not self.cfg['normalize_xyz']: + trans_delta = torch.tanh(output["trans"])*trans_normalizer + else: + trans_delta = output["trans"] + + elif self.cfg['trans_rep']=='deepim': + def project_and_transform_to_crop(centers): + uvs = (pose_data.Ks[b:b+bs]@centers.reshape(-1,3,1)).reshape(-1,3) + uvs = uvs/uvs[:,2:3] + uvs = (pose_data.tf_to_crops[b:b+bs]@uvs.reshape(-1,3,1)).reshape(-1,3) + return uvs[:,:2] + + rot_delta = output["rot"] + z_pred = output['trans'][:,2]*pose_data.poseA[b:b+bs][...,2,3] + uvA_crop = project_and_transform_to_crop(pose_data.poseA[b:b+bs][...,:3,3]) + uv_pred_crop = uvA_crop + output['trans'][:,:2]*self.cfg['input_resize'][0] + uv_pred = transform_pts(uv_pred_crop, pose_data.tf_to_crops[b:b+bs].inverse().cuda()) + center_pred = torch.cat([uv_pred, torch.ones((len(rot_delta),1), dtype=torch.float, device='cuda')], dim=-1) + center_pred = (pose_data.Ks[b:b+bs].inverse().cuda()@center_pred.reshape(len(rot_delta),3,1)).reshape(len(rot_delta),3) * z_pred.reshape(len(rot_delta),1) + trans_delta = center_pred-pose_data.poseA[b:b+bs][...,:3,3] + + else: + trans_delta = output["trans"] + + if self.cfg['rot_rep']=='axis_angle': + rot_mat_delta = torch.tanh(output["rot"])*self.cfg['rot_normalizer'] + rot_mat_delta = so3_exp_map(rot_mat_delta).permute(0,2,1) + elif self.cfg['rot_rep']=='6d': + rot_mat_delta = rotation_6d_to_matrix(output['rot']).permute(0,2,1) + else: + raise RuntimeError + + if self.cfg['normalize_xyz']: + trans_delta *= (mesh_diameter/2) + + B_in_cam = egocentric_delta_pose_to_pose(pose_data.poseA[b:b+bs], trans_delta=trans_delta, rot_mat_delta=rot_mat_delta) + B_in_cams.append(B_in_cam) + + B_in_cams = torch.cat(B_in_cams, dim=0).reshape(len(ob_in_cams),4,4) + + B_in_cams_out = B_in_cams@torch.tensor(tf_to_center[None], device='cuda', dtype=torch.float) + torch.cuda.empty_cache() + self.last_trans_update = trans_delta + self.last_rot_update = rot_mat_delta + + if get_vis: + logging.info("get_vis...") + canvas = [] + padding = 2 + pose_data = make_crop_data_batch(self.cfg.input_resize, torch.as_tensor(ob_centered_in_cams), mesh_centered, rgb, depth, K, crop_ratio=crop_ratio, normal_map=normal_map, xyz_map=xyz_map_tensor, cfg=self.cfg, glctx=glctx, mesh_tensors=mesh_tensors, dataset=self.dataset, mesh_diameter=mesh_diameter) + for id in range(0, len(B_in_cams)): + rgbA_vis = (pose_data.rgbAs[id]*255).permute(1,2,0).data.cpu().numpy() + rgbB_vis = (pose_data.rgbBs[id]*255).permute(1,2,0).data.cpu().numpy() + row = [rgbA_vis, rgbB_vis] + H,W = rgbA_vis.shape[:2] + if pose_data.depthAs is not None: + depthA = pose_data.depthAs[id].data.cpu().numpy().reshape(H,W) + depthB = pose_data.depthBs[id].data.cpu().numpy().reshape(H,W) + elif pose_data.xyz_mapAs is not None: + depthA = pose_data.xyz_mapAs[id][2].data.cpu().numpy().reshape(H,W) + depthB = pose_data.xyz_mapBs[id][2].data.cpu().numpy().reshape(H,W) + zmin = min(depthA.min(), depthB.min()) + zmax = max(depthA.max(), depthB.max()) + depthA_vis = depth_to_vis(depthA, zmin=zmin, zmax=zmax, inverse=False) + depthB_vis = depth_to_vis(depthB, zmin=zmin, zmax=zmax, inverse=False) + row += [depthA_vis, depthB_vis] + if pose_data.normalAs is not None: + pass + row = make_grid_image(row, nrow=len(row), padding=padding, pad_value=255) + row = cv_draw_text(row, text=f'id:{id}', uv_top_left=(10,10), color=(0,255,0), fontScale=0.5) + canvas.append(row) + canvas = make_grid_image(canvas, nrow=1, padding=padding, pad_value=255) + + pose_data = make_crop_data_batch(self.cfg.input_resize, B_in_cams, mesh_centered, rgb, depth, K, crop_ratio=crop_ratio, normal_map=normal_map, xyz_map=xyz_map_tensor, cfg=self.cfg, glctx=glctx, mesh_tensors=mesh_tensors, dataset=self.dataset, mesh_diameter=mesh_diameter) + canvas_refined = [] + for id in range(0, len(B_in_cams)): + rgbA_vis = (pose_data.rgbAs[id]*255).permute(1,2,0).data.cpu().numpy() + rgbB_vis = (pose_data.rgbBs[id]*255).permute(1,2,0).data.cpu().numpy() + row = [rgbA_vis, rgbB_vis] + H,W = rgbA_vis.shape[:2] + if pose_data.depthAs is not None: + depthA = pose_data.depthAs[id].data.cpu().numpy().reshape(H,W) + depthB = pose_data.depthBs[id].data.cpu().numpy().reshape(H,W) + elif pose_data.xyz_mapAs is not None: + depthA = pose_data.xyz_mapAs[id][2].data.cpu().numpy().reshape(H,W) + depthB = pose_data.xyz_mapBs[id][2].data.cpu().numpy().reshape(H,W) + zmin = min(depthA.min(), depthB.min()) + zmax = max(depthA.max(), depthB.max()) + depthA_vis = depth_to_vis(depthA, zmin=zmin, zmax=zmax, inverse=False) + depthB_vis = depth_to_vis(depthB, zmin=zmin, zmax=zmax, inverse=False) + row += [depthA_vis, depthB_vis] + row = make_grid_image(row, nrow=len(row), padding=padding, pad_value=255) + canvas_refined.append(row) + + canvas_refined = make_grid_image(canvas_refined, nrow=1, padding=padding, pad_value=255) + canvas = make_grid_image([canvas, canvas_refined], nrow=2, padding=padding, pad_value=255) + torch.cuda.empty_cache() + return B_in_cams_out, canvas + + return B_in_cams_out, None + diff --git a/third_party/FoundationPose/learning/training/predict_score.py b/third_party/FoundationPose/learning/training/predict_score.py new file mode 100644 index 0000000..6e15f62 --- /dev/null +++ b/third_party/FoundationPose/learning/training/predict_score.py @@ -0,0 +1,227 @@ +# Copyright (c) 2023, NVIDIA CORPORATION. All rights reserved. +# +# NVIDIA CORPORATION and its licensors retain all intellectual property +# and proprietary rights in and to this software, related documentation +# and any modifications thereto. Any use, reproduction, disclosure or +# distribution of this software and related documentation without an express +# license agreement from NVIDIA CORPORATION is strictly prohibited. + + +import functools +import os,sys,kornia +import time +import numpy as np +import torch +import torch.distributed as dist +from omegaconf import OmegaConf +from tqdm import tqdm +code_dir = os.path.dirname(os.path.realpath(__file__)) +sys.path.append(f'{code_dir}/../../../') +from learning.datasets.h5_dataset import * +from learning.models.score_network import * +from learning.datasets.pose_dataset import * +from Utils import * +from datareader import * + + +def vis_batch_data_scores(pose_data, ids, scores, pad_margin=5): + assert len(scores)==len(ids) + canvas = [] + for id in ids: + rgbA_vis = (pose_data.rgbAs[id]*255).permute(1,2,0).data.cpu().numpy() + rgbB_vis = (pose_data.rgbBs[id]*255).permute(1,2,0).data.cpu().numpy() + H,W = rgbA_vis.shape[:2] + zmin = pose_data.depthAs[id].data.cpu().numpy().reshape(H,W).min() + zmax = pose_data.depthAs[id].data.cpu().numpy().reshape(H,W).max() + depthA_vis = depth_to_vis(pose_data.depthAs[id].data.cpu().numpy().reshape(H,W), zmin=zmin, zmax=zmax, inverse=False) + depthB_vis = depth_to_vis(pose_data.depthBs[id].data.cpu().numpy().reshape(H,W), zmin=zmin, zmax=zmax, inverse=False) + if pose_data.normalAs is not None: + pass + pad = np.ones((rgbA_vis.shape[0],pad_margin,3))*255 + if pose_data.normalAs is not None: + pass + else: + row = np.concatenate([rgbA_vis, pad, depthA_vis, pad, rgbB_vis, pad, depthB_vis], axis=1) + s = 100/row.shape[0] + row = cv2.resize(row, fx=s, fy=s, dsize=None) + row = cv_draw_text(row, text=f'id:{id}, score:{scores[id]:.3f}', uv_top_left=(10,10), color=(0,255,0), fontScale=0.5) + canvas.append(row) + pad = np.ones((pad_margin, row.shape[1], 3))*255 + canvas.append(pad) + canvas = np.concatenate(canvas, axis=0).astype(np.uint8) + return canvas + + + +@torch.no_grad() +def make_crop_data_batch(render_size, ob_in_cams, mesh, rgb, depth, K, crop_ratio, normal_map=None, mesh_diameter=None, glctx=None, mesh_tensors=None, dataset:TripletH5Dataset=None, cfg=None): + logging.info("Welcome make_crop_data_batch") + H,W = depth.shape[:2] + + args = [] + method = 'box_3d' + tf_to_crops = compute_crop_window_tf_batch(pts=mesh.vertices, H=H, W=W, poses=ob_in_cams, K=K, crop_ratio=crop_ratio, out_size=(render_size[1], render_size[0]), method=method, mesh_diameter=mesh_diameter) + logging.info("make tf_to_crops done") + + B = len(ob_in_cams) + poseAs = torch.as_tensor(ob_in_cams, dtype=torch.float, device='cuda') + + bs = 512 + rgb_rs = [] + depth_rs = [] + xyz_map_rs = [] + + bbox2d_crop = torch.as_tensor(np.array([0, 0, cfg['input_resize'][0]-1, cfg['input_resize'][1]-1]).reshape(2,2), device='cuda', dtype=torch.float) + bbox2d_ori = transform_pts(bbox2d_crop, tf_to_crops.inverse()[:,None]).reshape(-1,4) + + for b in range(0,len(ob_in_cams),bs): + extra = {} + rgb_r, depth_r, normal_r = nvdiffrast_render(K=K, H=H, W=W, ob_in_cams=poseAs[b:b+bs], context='cuda', get_normal=cfg['use_normal'], glctx=glctx, mesh_tensors=mesh_tensors, output_size=cfg['input_resize'], bbox2d=bbox2d_ori[b:b+bs], use_light=True, extra=extra) + rgb_rs.append(rgb_r) + depth_rs.append(depth_r[...,None]) + xyz_map_rs.append(extra['xyz_map']) + + rgb_rs = torch.cat(rgb_rs, dim=0).permute(0,3,1,2) * 255 + depth_rs = torch.cat(depth_rs, dim=0).permute(0,3,1,2) + xyz_map_rs = torch.cat(xyz_map_rs, dim=0).permute(0,3,1,2) #(B,3,H,W) + logging.info("render done") + + rgbBs = kornia.geometry.transform.warp_perspective(torch.as_tensor(rgb, dtype=torch.float, device='cuda').permute(2,0,1)[None].expand(B,-1,-1,-1), tf_to_crops, dsize=render_size, mode='bilinear', align_corners=False) + depthBs = kornia.geometry.transform.warp_perspective(torch.as_tensor(depth, dtype=torch.float, device='cuda')[None,None].expand(B,-1,-1,-1), tf_to_crops, dsize=render_size, mode='nearest', align_corners=False) + if rgb_rs.shape[-2:]!=cfg['input_resize']: + rgbAs = kornia.geometry.transform.warp_perspective(rgb_rs, tf_to_crops, dsize=render_size, mode='bilinear', align_corners=False) + depthAs = kornia.geometry.transform.warp_perspective(depth_rs, tf_to_crops, dsize=render_size, mode='nearest', align_corners=False) + else: + rgbAs = rgb_rs + depthAs = depth_rs + + if xyz_map_rs.shape[-2:]!=cfg['input_resize']: + xyz_mapAs = kornia.geometry.transform.warp_perspective(xyz_map_rs, tf_to_crops, dsize=render_size, mode='nearest', align_corners=False) + else: + xyz_mapAs = xyz_map_rs + + normalAs = None + normalBs = None + + Ks = torch.as_tensor(K, dtype=torch.float).reshape(1,3,3).expand(len(rgbAs),3,3) + mesh_diameters = torch.ones((len(rgbAs)), dtype=torch.float, device='cuda')*mesh_diameter + + pose_data = BatchPoseData(rgbAs=rgbAs, rgbBs=rgbBs, depthAs=depthAs, depthBs=depthBs, normalAs=normalAs, normalBs=normalBs, poseA=poseAs, xyz_mapAs=xyz_mapAs, tf_to_crops=tf_to_crops, Ks=Ks, mesh_diameters=mesh_diameters) + pose_data = dataset.transform_batch(pose_data, H_ori=H, W_ori=W, bound=1) + + logging.info("pose batch data done") + + return pose_data + + +class ScorePredictor: + def __init__(self, amp=True): + self.amp = amp + self.run_name = "2024-01-11-20-02-45" + + model_name = 'model_best.pth' + code_dir = os.path.dirname(os.path.realpath(__file__)) + ckpt_dir = f'{code_dir}/../../weights/{self.run_name}/{model_name}' + + self.cfg = OmegaConf.load(f'{code_dir}/../../weights/{self.run_name}/config.yml') + + self.cfg['ckpt_dir'] = ckpt_dir + self.cfg['enable_amp'] = True + + ########## Defaults, to be backward compatible + if 'use_normal' not in self.cfg: + self.cfg['use_normal'] = False + if 'use_BN' not in self.cfg: + self.cfg['use_BN'] = False + if 'zfar' not in self.cfg: + self.cfg['zfar'] = np.inf + if 'c_in' not in self.cfg: + self.cfg['c_in'] = 4 + if 'normalize_xyz' not in self.cfg: + self.cfg['normalize_xyz'] = False + if 'crop_ratio' not in self.cfg or self.cfg['crop_ratio'] is None: + self.cfg['crop_ratio'] = 1.2 + + logging.info(f"self.cfg: \n {OmegaConf.to_yaml(self.cfg)}") + + self.dataset = ScoreMultiPairH5Dataset(cfg=self.cfg, mode='test', h5_file=None, max_num_key=1) + self.model = ScoreNetMultiPair(cfg=self.cfg, c_in=self.cfg['c_in']).cuda() + + logging.info(f"Using pretrained model from {ckpt_dir}") + ckpt = torch.load(ckpt_dir) + if 'model' in ckpt: + ckpt = ckpt['model'] + self.model.load_state_dict(ckpt) + + self.model.cuda().eval() + logging.info("init done") + + + @torch.inference_mode() + def predict(self, rgb, depth, K, ob_in_cams, normal_map=None, get_vis=False, mesh=None, mesh_tensors=None, glctx=None, mesh_diameter=None): + ''' + @rgb: np array (H,W,3) + ''' + logging.info(f"ob_in_cams:{ob_in_cams.shape}") + ob_in_cams = torch.as_tensor(ob_in_cams, dtype=torch.float, device='cuda') + + logging.info(f'self.cfg.use_normal:{self.cfg.use_normal}') + if not self.cfg.use_normal: + normal_map = None + + logging.info("making cropped data") + + if mesh_tensors is None: + mesh_tensors = make_mesh_tensors(mesh) + + rgb = torch.as_tensor(rgb, device='cuda', dtype=torch.float) + depth = torch.as_tensor(depth, device='cuda', dtype=torch.float) + + pose_data = make_crop_data_batch(self.cfg.input_resize, ob_in_cams, mesh, rgb, depth, K, crop_ratio=self.cfg['crop_ratio'], glctx=glctx, mesh_tensors=mesh_tensors, dataset=self.dataset, cfg=self.cfg, mesh_diameter=mesh_diameter) + + def find_best_among_pairs(pose_data:BatchPoseData): + logging.info(f'pose_data.rgbAs.shape[0]: {pose_data.rgbAs.shape[0]}') + ids = [] + scores = [] + bs = pose_data.rgbAs.shape[0] + for b in range(0, pose_data.rgbAs.shape[0], bs): + A = torch.cat([pose_data.rgbAs[b:b+bs].cuda(), pose_data.xyz_mapAs[b:b+bs].cuda()], dim=1).float() + B = torch.cat([pose_data.rgbBs[b:b+bs].cuda(), pose_data.xyz_mapBs[b:b+bs].cuda()], dim=1).float() + if pose_data.normalAs is not None: + A = torch.cat([A, pose_data.normalAs.cuda().float()], dim=1) + B = torch.cat([B, pose_data.normalBs.cuda().float()], dim=1) + with torch.cuda.amp.autocast(enabled=self.amp): + output = self.model(A, B, L=len(A)) + scores_cur = output["score_logit"].float().reshape(-1) + ids.append(scores_cur.argmax()+b) + scores.append(scores_cur) + ids = torch.stack(ids, dim=0).reshape(-1) + scores = torch.cat(scores, dim=0).reshape(-1) + return ids, scores + + pose_data_iter = pose_data + global_ids = torch.arange(len(ob_in_cams), device='cuda', dtype=torch.long) + scores_global = torch.zeros((len(ob_in_cams)), dtype=torch.float, device='cuda') + + while 1: + ids, scores = find_best_among_pairs(pose_data_iter) + if len(ids)==1: + scores_global[global_ids] = scores + 100 + break + global_ids = global_ids[ids] + pose_data_iter = pose_data.select_by_indices(global_ids) + + scores = scores_global + + logging.info(f'forward done') + torch.cuda.empty_cache() + + if get_vis: + logging.info("get_vis...") + canvas = [] + ids = scores.argsort(descending=True) + canvas = vis_batch_data_scores(pose_data, ids=ids, scores=scores) + return scores, canvas + + return scores, None + diff --git a/third_party/FoundationPose/learning/training/training_config.py b/third_party/FoundationPose/learning/training/training_config.py new file mode 100644 index 0000000..9d689c6 --- /dev/null +++ b/third_party/FoundationPose/learning/training/training_config.py @@ -0,0 +1,102 @@ +# Copyright (c) 2023, NVIDIA CORPORATION. All rights reserved. +# +# NVIDIA CORPORATION and its licensors retain all intellectual property +# and proprietary rights in and to this software, related documentation +# and any modifications thereto. Any use, reproduction, disclosure or +# distribution of this software and related documentation without an express +# license agreement from NVIDIA CORPORATION is strictly prohibited. + + +import os,sys +from dataclasses import dataclass, field +from typing import List, Optional, Tuple,Union +import numpy as np +import omegaconf +import torch + + +@dataclass +class TrainingConfig(omegaconf.dictconfig.DictConfig): + input_resize: tuple = (160, 160) + normalize_xyz:Optional[bool] = True + use_mask:Optional[bool] = False + crop_ratio:Optional[float] = None + split_objects_across_gpus: bool = True + max_num_key: Optional[int] = None + use_normal:bool = False + n_view:int = 1 + zfar:float = np.inf + c_in:int = 6 + train_num_pair:Optional[int] = None + make_pair_online:Optional[bool] = False + render_backend:Optional[str] = 'nvdiffrast' + + # Run management + run_id: Optional[str] = None + exp_name:Optional[str] = None + resume_run_id: Optional[str] = None + save_dir: Optional[str] = None + batch_size: int = 64 + epoch_size: int = 115200 + val_size: int = 1280 + n_epochs: int = 25 + save_epoch_interval: int = 100 + n_dataloader_workers: int = 20 + n_rendering_workers: int = 1 + gradient_max_norm:float = np.inf + max_step_per_epoch: Optional[int] = 25000 + + # Network + use_BN:bool = True + loss_type:Optional[str] = 'pairwise_valid' + + # Optimizer + optimizer: str = "adam" + weight_decay: float = 0.0 + clip_grad_norm: float = np.inf + lr: float = 0.0001 + warmup_step: int = -1 # -1 means disable + n_epochs_warmup: int = 1 + + # Visualization + vis_interval: Optional[int] = 1000 + + debug: Optional[bool] = None + + + +@dataclass +class TrainRefinerConfig: + # Datasets + input_resize: tuple = (160, 160) #(W,H) + crop_ratio:Optional[float] = None + max_num_key: Optional[int] = None + use_normal:bool = False + use_mask:Optional[bool] = False + normal_uint8:bool = False + normalize_xyz:Optional[bool] = True + trans_normalizer:Optional[list] = None + rot_normalizer:Optional[float] = None + c_in:int = 6 + n_view:int = 1 + zfar:float = np.inf + trans_rep:str = 'tracknet' # tracknet/deepim + rot_rep:Optional[str] = 'axis_angle' # 6d/axis_angle + save_dir: Optional[str] = None + + # Run management + run_id: Optional[str] = None + exp_name:Optional[str] = None + batch_size: int = 64 + use_BN:bool = True + optimizer: str = "adam" + weight_decay: float = 0.0 + clip_grad_norm: float = np.inf + lr: float = 0.0001 + warmup_step: int = -1 + loss_type:str = 'l2' # l1/l2/add + + vis_interval: Optional[int] = 1000 + debug: Optional[bool] = None + + diff --git a/third_party/FoundationPose/mycpp/CMakeLists.txt b/third_party/FoundationPose/mycpp/CMakeLists.txt new file mode 100644 index 0000000..7c8a9a5 --- /dev/null +++ b/third_party/FoundationPose/mycpp/CMakeLists.txt @@ -0,0 +1,25 @@ +cmake_minimum_required(VERSION 3.15) +project(mycpp) + + +set(CMAKE_BUILD_TYPE Release) +set(CMAKE_EXPORT_COMPILE_COMMANDS ON) +set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -std=c++14 -fopenmp -g3 -O3") + + +find_package(Boost REQUIRED COMPONENTS system program_options) +find_package(OpenMP REQUIRED) +find_package(Eigen3 REQUIRED) +find_package(pybind11 REQUIRED) + +include_directories( + include + ${BLAS_INCLUDE_DIR} +) + +file(GLOB MY_SRC ${PROJECT_SOURCE_DIR}/src/*.cpp) + +set(PYBIND11_CPP_STANDARD -std=c++14) + +pybind11_add_module(mycpp src/app/pybind_api.cpp ${MY_SRC}) +target_link_libraries(mycpp PRIVATE ${Boost_LIBRARIES} ${OpenMP_CXX_FLAGS} Eigen3::Eigen) \ No newline at end of file diff --git a/third_party/FoundationPose/mycpp/include/Utils.h b/third_party/FoundationPose/mycpp/include/Utils.h new file mode 100644 index 0000000..41703f5 --- /dev/null +++ b/third_party/FoundationPose/mycpp/include/Utils.h @@ -0,0 +1,46 @@ +/*Copyright (c) 2023, NVIDIA CORPORATION. All rights reserved. +# +# NVIDIA CORPORATION and its licensors retain all intellectual property +# and proprietary rights in and to this software, related documentation +# and any modifications thereto. Any use, reproduction, disclosure or +# distribution of this software and related documentation without an express +# license agreement from NVIDIA CORPORATION is strictly prohibited. +*/ + + +#pragma once + +// STL +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + + +using vectorMatrix4f = std::vector>; + + +namespace Utils +{ +float rotationGeodesicDistance(const Eigen::Matrix3f &R1, const Eigen::Matrix3f &R2); + +} // namespace Utils diff --git a/third_party/FoundationPose/mycpp/src/Utils.cpp b/third_party/FoundationPose/mycpp/src/Utils.cpp new file mode 100644 index 0000000..c1455d5 --- /dev/null +++ b/third_party/FoundationPose/mycpp/src/Utils.cpp @@ -0,0 +1,29 @@ +/* +# Copyright (c) 2023, NVIDIA CORPORATION. All rights reserved. +# +# NVIDIA CORPORATION and its licensors retain all intellectual property +# and proprietary rights in and to this software, related documentation +# and any modifications thereto. Any use, reproduction, disclosure or +# distribution of this software and related documentation without an express +# license agreement from NVIDIA CORPORATION is strictly prohibited. +*/ + + +#include "Utils.h" + + + +namespace Utils +{ + + +// Difference angle in radian +float rotationGeodesicDistance(const Eigen::Matrix3f &R1, const Eigen::Matrix3f &R2) +{ + float cos = ((R1 * R2.transpose()).trace()-1) / 2.0; + cos = std::max(std::min(cos, 1.0f), -1.0f); + return std::acos(cos); +} + + +} // namespace Utils diff --git a/third_party/FoundationPose/mycpp/src/app/pybind_api.cpp b/third_party/FoundationPose/mycpp/src/app/pybind_api.cpp new file mode 100644 index 0000000..6a415c3 --- /dev/null +++ b/third_party/FoundationPose/mycpp/src/app/pybind_api.cpp @@ -0,0 +1,77 @@ +/* +# Copyright (c) 2023, NVIDIA CORPORATION. All rights reserved. +# +# NVIDIA CORPORATION and its licensors retain all intellectual property +# and proprietary rights in and to this software, related documentation +# and any modifications thereto. Any use, reproduction, disclosure or +# distribution of this software and related documentation without an express +# license agreement from NVIDIA CORPORATION is strictly prohibited. +*/ + + +#include "Utils.h" +#include +#include +#include +#include + +namespace py = pybind11; + + + +//@angle_diff: unit is degree +//@dist_diff: unit is meter +vectorMatrix4f cluster_poses(float angle_diff, float dist_diff, const vectorMatrix4f &poses_in, const vectorMatrix4f &symmetry_tfs) +{ + printf("num original candidates = %d\n",poses_in.size()); + vectorMatrix4f poses_out; + poses_out.push_back(poses_in[0]); + + const float radian_thres = angle_diff/180.0*M_PI; + + for (int i=1;i=dist_diff) + { + continue; + } + + for (const auto &tf: symmetry_tfs) + { + Eigen::Matrix4f cur_pose_tmp = cur_pose*tf; + float rot_diff = Utils::rotationGeodesicDistance(cur_pose_tmp.block(0,0,3,3), cluster.block(0,0,3,3)); + if (rot_diff < radian_thres) + { + isnew = false; + break; + } + } + + if (!isnew) break; + } + + if (isnew) + { + poses_out.push_back(poses_in[i]); + } + } + + printf("num of pose after clustering: %d\n",poses_out.size()); + return poses_out; +} + + + + + +PYBIND11_MODULE(mycpp, m) +{ + m.def("cluster_poses", &cluster_poses, py::call_guard()); +} \ No newline at end of file diff --git a/third_party/FoundationPose/offscreen_renderer.py b/third_party/FoundationPose/offscreen_renderer.py new file mode 100644 index 0000000..54e2faf --- /dev/null +++ b/third_party/FoundationPose/offscreen_renderer.py @@ -0,0 +1,79 @@ +# Copyright (c) 2023, NVIDIA CORPORATION. All rights reserved. +# +# NVIDIA CORPORATION and its licensors retain all intellectual property +# and proprietary rights in and to this software, related documentation +# and any modifications thereto. Any use, reproduction, disclosure or +# distribution of this software and related documentation without an express +# license agreement from NVIDIA CORPORATION is strictly prohibited. + + +import os,sys,time,socket +code_path = os.path.dirname(os.path.realpath(__file__)) +sys.path.append(code_path) +import open3d as o3d +import numpy as np +from PIL import Image +import cv2,imageio +import time +import trimesh +import pyrender +from transformations import * +import numpy as np +from PIL import Image +import cv2 +import time +import argparse,pickle +from Utils import * + + +cvcam_in_glcam = np.array([[1,0,0,0], + [0,-1,0,0], + [0,0,-1,0], + [0,0,0,1]]) + +class ModelRendererOffscreen: + def __init__(self, cam_K, H,W, zfar=100): + ''' + @window_sizes: H,W + ''' + self.K = cam_K + self.scene = pyrender.Scene(ambient_light=[1., 1., 1.],bg_color=[0,0,0]) + self.camera = pyrender.IntrinsicsCamera(fx=cam_K[0,0],fy=cam_K[1,1],cx=cam_K[0,2],cy=cam_K[1,2],znear=0.001,zfar=zfar) + self.cam_node = self.scene.add(self.camera, pose=np.eye(4), name='cam') + self.mesh_nodes = [] + + self.H = H + self.W = W + self.r = pyrender.OffscreenRenderer(self.W, self.H) + + + def set_cam_pose(self, cam_pose): + self.cam_node.matrix = cam_pose + + def add_mesh(self, mesh): + mesh = pyrender.Mesh.from_trimesh(mesh, smooth=False) + mesh_node = self.scene.add(mesh,pose=np.eye(4), name='ob') # Object pose parent is cam + self.mesh_nodes.append(mesh_node) + + + def add_point_light(self, intensity=3): + light = pyrender.DirectionalLight(color=[1.0, 1.0, 1.0], intensity=intensity) + self.scene.add(light, pose=np.eye(4)) # Same as camera position + + + def clear_mesh_nodes(self): + for n in self.mesh_nodes: + self.scene.remove_node(n) + + + def render(self,mesh=None,ob_in_cvcam=None, get_normal=False): + if mesh is not None: + mesh = mesh.copy() + mesh.apply_transform(cvcam_in_glcam@ob_in_cvcam) + mesh = pyrender.Mesh.from_trimesh(mesh, smooth=False) + mesh_node = self.scene.add(mesh, pose=np.eye(4), name='ob') # Object pose parent is cam + color, depth = self.r.render(self.scene) # depth: float + if mesh is not None: + self.scene.remove_node(mesh_node) + + return color, depth diff --git a/third_party/FoundationPose/readme.md b/third_party/FoundationPose/readme.md new file mode 100644 index 0000000..54f1edf --- /dev/null +++ b/third_party/FoundationPose/readme.md @@ -0,0 +1,226 @@ +# FoundationPose: Unified 6D Pose Estimation and Tracking of Novel Objects +[[Paper]](https://arxiv.org/abs/2312.08344) [[Website]](https://nvlabs.github.io/FoundationPose/) + +This is the official implementation of our paper to be appeared in CVPR 2024 (Highlight) + +Contributors: Bowen Wen, Wei Yang, Jan Kautz, Stan Birchfield + +We present FoundationPose, a unified foundation model for 6D object pose estimation and tracking, supporting both model-based and model-free setups. Our approach can be instantly applied at test-time to a novel object without fine-tuning, as long as its CAD model is given, or a small number of reference images are captured. We bridge the gap between these two setups with a neural implicit representation that allows for effective novel view synthesis, keeping the downstream pose estimation modules invariant under the same unified framework. Strong generalizability is achieved via large-scale synthetic training, aided by a large language model (LLM), a novel transformer-based architecture, and contrastive learning formulation. Extensive evaluation on multiple public datasets involving challenging scenarios and objects indicate our unified approach outperforms existing methods specialized for each task by a large margin. In addition, it even achieves comparable results to instance-level methods despite the reduced assumptions. + + + + +**🤖 For ROS version, please check [Isaac ROS Pose Estimation](https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_pose_estimation), which enjoys TRT fast inference and C++ speed up.** + +\ +**🥇 No. 1 on the world-wide [BOP leaderboard](https://bop.felk.cvut.cz/leaderboards/pose-estimation-unseen-bop23/core-datasets/) (as of 2024/03) for model-based novel object pose estimation.** + + +## Demos + +Robotic Applications: + +https://github.com/NVlabs/FoundationPose/assets/23078192/aa341004-5a15-4293-b3da-000471fd74ed + + +AR Applications: + +https://github.com/NVlabs/FoundationPose/assets/23078192/80e96855-a73c-4bee-bcef-7cba92df55ca + + +Results on YCB-Video dataset: + +https://github.com/NVlabs/FoundationPose/assets/23078192/9b5bedde-755b-44ed-a973-45ec85a10bbe + + + +# Bibtex +```bibtex +@InProceedings{foundationposewen2024, +author = {Bowen Wen, Wei Yang, Jan Kautz, Stan Birchfield}, +title = {{FoundationPose}: Unified 6D Pose Estimation and Tracking of Novel Objects}, +booktitle = {CVPR}, +year = {2024}, +} +``` + +If you find the model-free setup useful, please also consider cite: + +```bibtex +@InProceedings{bundlesdfwen2023, +author = {Bowen Wen and Jonathan Tremblay and Valts Blukis and Stephen Tyree and Thomas M\"{u}ller and Alex Evans and Dieter Fox and Jan Kautz and Stan Birchfield}, +title = {{BundleSDF}: {N}eural 6-{DoF} Tracking and {3D} Reconstruction of Unknown Objects}, +booktitle = {CVPR}, +year = {2023}, +} +``` + +# Data prepare + + +1) Download all network weights from [here](https://drive.google.com/drive/folders/1DFezOAD0oD1BblsXVxqDsl8fj0qzB82i?usp=sharing) and put them under the folder `weights/`. For the refiner, you will need `2023-10-28-18-33-37`. For scorer, you will need `2024-01-11-20-02-45`. + +1) [Download demo data](https://drive.google.com/drive/folders/1pRyFmxYXmAnpku7nGRioZaKrVJtIsroP?usp=sharing) and extract them under the folder `demo_data/` + +1) [Optional] Download our large-scale training data: ["FoundationPose Dataset"](https://drive.google.com/drive/folders/1s4pB6p4ApfWMiMjmTXOFco8dHbNXikp-?usp=sharing) + +1) [Optional] Download our preprocessed reference views [here](https://drive.google.com/drive/folders/1PXXCOJqHXwQTbwPwPbGDN9_vLVe0XpFS?usp=sharing) in order to run model-free few-shot version. + +# Env setup option 1: docker (recommended) + ``` + cd docker/ + docker pull wenbowen123/foundationpose && docker tag wenbowen123/foundationpose foundationpose # Or to build from scratch: docker build --network host -t foundationpose . + bash docker/run_container.sh + ``` + + +If it's the first time you launch the container, you need to build extensions. Run this command *inside* the Docker container. +``` +bash build_all.sh +``` + +Later you can execute into the container without re-build. +``` +docker exec -it foundationpose bash +``` + +For more recent GPU such as 4090, refer to [this](https://github.com/NVlabs/FoundationPose/issues/27). +In short, do the following: +``` +docker pull shingarey/foundationpose_custom_cuda121:latest +``` +Then modify the bash script to use this image instead of `foundationpose:latest`. + + +# Env setup option 2: conda (local) + +1) **Create the environment** (C++ build deps + Python; all on `conda-forge`): + +```bash +conda env create -f environment.yml +conda activate foundationpose +``` + +2) **Install PyTorch** with a CUDA build that matches your machine. The [PyTorch “Get Started”](https://pytorch.org/get-started/locally/) page lists the right `--index-url` (for example `cu124` works on most current NVIDIA drivers). Example: + +```bash +python -m pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu124 +``` + +3) **Install PyTorch3D and NVDiffRast** (compile from source; needs the CUDA toolkit for `nvcc`. Point `CUDA_HOME` at your install, e.g. `/usr/local/cuda-12.8` or `/usr/local/cuda`): + +```bash +export CUDA_HOME=/usr/local/cuda # or e.g. /usr/local/cuda-12.8 +export PATH="$CUDA_HOME/bin:$PATH" +python -m pip install --no-build-isolation "git+https://github.com/facebookresearch/pytorch3d.git" +python -m pip install --no-build-isolation "git+https://github.com/NVlabs/nvdiffrast.git" +``` + +`--no-build-isolation` is required so the build can see the `torch` you already installed. + +4) **Install remaining Python dependencies** and **build the `mycpp` extension** (BundleSDF’s `mycuda` step is optional; skip it unless you need the model-free / NeRF path): + +```bash +python -m pip install -r requirements.txt +bash build_all_conda.sh +``` + +5) **Optional — Kaolin** (only for the model-free setup; version must match your PyTorch/CUDA—see the [Kaolin install docs](https://kaolin.readthedocs.io/en/latest/notes/installation.html)). + + +# Run model-based demo +The paths have been set in argparse by default. If you need to change the scene, you can pass the args accordingly. By running on the demo data, you should be able to see the robot manipulating the mustard bottle. Pose estimation is conducted on the first frame, then it automatically switches to tracking mode for the rest of the video. The resulting visualizations will be saved to the `debug_dir` specified in the argparse. (Note the first time running could be slower due to online compilation) +``` +python run_demo.py +``` + + + + + +Feel free to try on other objects (**no need to retrain**) such as driller, by changing the paths in argparse. + + + + +# Run on public datasets (LINEMOD, YCB-Video) + +For this you first need to download LINEMOD dataset and YCB-Video dataset. + +To run model-based version on these two datasets respectively, set the paths based on where you download. The results will be saved to `debug` folder +``` +python run_linemod.py --linemod_dir /mnt/9a72c439-d0a7-45e8-8d20-d7a235d02763/DATASET/LINEMOD --use_reconstructed_mesh 0 + +python run_ycb_video.py --ycbv_dir /mnt/9a72c439-d0a7-45e8-8d20-d7a235d02763/DATASET/YCB_Video --use_reconstructed_mesh 0 +``` + +To run model-free few-shot version. You first need to train Neural Object Field. `ref_view_dir` is based on where you download in the above "Data prepare" section. Set the `dataset` flag to your interested dataset. +``` +python bundlesdf/run_nerf.py --ref_view_dir /mnt/9a72c439-d0a7-45e8-8d20-d7a235d02763/DATASET/YCB_Video/bowen_addon/ref_views_16 --dataset ycbv +``` + +Then run the similar command as the model-based version with some small modifications. Here we are using YCB-Video as example: +``` +python run_ycb_video.py --ycbv_dir /mnt/9a72c439-d0a7-45e8-8d20-d7a235d02763/DATASET/YCB_Video --use_reconstructed_mesh 1 --ref_view_dir /mnt/9a72c439-d0a7-45e8-8d20-d7a235d02763/DATASET/YCB_Video/bowen_addon/ref_views_16 +``` + +# Troubleshooting + + +- For more recent GPU such as 4090, refer to [this](https://github.com/NVlabs/FoundationPose/issues/27). + +- For setting up on Windows, refer to [this](https://github.com/NVlabs/FoundationPose/issues/148). + +- If you are getting unreasonable results, check [this](https://github.com/NVlabs/FoundationPose/issues/44#issuecomment-2048141043) and [this](https://github.com/030422Lee/FoundationPose_manual) + +# Training data download +Our training data include scenes using 3D assets from GSO and Objaverse, rendered with high quality photo-realism and large domain randomization. Each data point includes **RGB, depth, object pose, camera pose, instance segmentation, 2D bounding box**. [[Google Drive]](https://drive.google.com/drive/folders/1s4pB6p4ApfWMiMjmTXOFco8dHbNXikp-?usp=sharing). + + + +- To parse the camera params including extrinsics and intrinsics + ``` + glcam_in_cvcam = np.array([[1,0,0,0], + [0,-1,0,0], + [0,0,-1,0], + [0,0,0,1]]).astype(float) + W, H = camera_params["renderProductResolution"] + with open(f'{base_dir}/camera_params/camera_params_000000.json','r') as ff: + camera_params = json.load(ff) + world_in_glcam = np.array(camera_params['cameraViewTransform']).reshape(4,4).T + cam_in_world = np.linalg.inv(world_in_glcam)@glcam_in_cvcam + world_in_cam = np.linalg.inv(cam_in_world) + focal_length = camera_params["cameraFocalLength"] + horiz_aperture = camera_params["cameraAperture"][0] + vert_aperture = H / W * horiz_aperture + focal_y = H * focal_length / vert_aperture + focal_x = W * focal_length / horiz_aperture + center_y = H * 0.5 + center_x = W * 0.5 + + fx, fy, cx, cy = focal_x, focal_y, center_x, center_y + K = np.eye(3) + K[0,0] = fx + K[1,1] = fy + K[0,2] = cx + K[1,2] = cy + ``` + + + +# Notes +Due to the legal restrictions of Stable-Diffusion that is trained on LAION dataset, we are not able to release the diffusion-based texture augmented data, nor the pretrained weights using it. We thus release the version without training on diffusion-augmented data. Slight performance degradation is expected. + +# Acknowledgement + +We would like to thank Jeff Smith for helping with the code release; NVIDIA Isaac Sim and Omniverse team for the support on synthetic data generation; Tianshi Cao for the valuable discussions. Finally, we are also grateful for the positive feebacks and constructive suggestions brought up by reviewers and AC at CVPR. + + + + +# License +The code and data are released under the NVIDIA Source Code License. Copyright © 2024, NVIDIA Corporation. All rights reserved. + + +# Contact +For questions, please contact [Bowen Wen](https://wenbowen123.github.io/). diff --git a/third_party/FoundationPose/requirements.txt b/third_party/FoundationPose/requirements.txt new file mode 100644 index 0000000..4759280 --- /dev/null +++ b/third_party/FoundationPose/requirements.txt @@ -0,0 +1,43 @@ +# Install PyTorch first (CUDA build must match your driver; cu124 works on most recent NVIDIA GPUs): +# pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu124 +# +# Then GPU extensions (need nvcc; set CUDA_HOME to your toolkit, e.g. /usr/local/cuda): +# pip install --no-build-isolation git+https://github.com/facebookresearch/pytorch3d.git +# pip install --no-build-isolation git+https://github.com/NVlabs/nvdiffrast.git +# +# Then this file: +# pip install -r requirements.txt + +# Core numerics & IO +numpy>=2 +scipy +scikit-learn +h5py +joblib + +# Config / serialization +PyYAML>=6 +ruamel.yaml>=0.18 + +# Vision & 3D +opencv-python +imageio +open3d +trimesh +transformations +matplotlib +pandas +Pillow +pyrender +pyOpenGL>=3.1 +pyOpenGL_accelerate>=3.1 + +# Inference stack (torch installed separately) +kornia>=0.7 +omegaconf>=2.3 +psutil +tqdm +warp-lang + +# Optional: faster / utility (import guarded in code) +scikit-image diff --git a/third_party/FoundationPose/run_demo.py b/third_party/FoundationPose/run_demo.py new file mode 100644 index 0000000..fc542b4 --- /dev/null +++ b/third_party/FoundationPose/run_demo.py @@ -0,0 +1,79 @@ +# Copyright (c) 2023, NVIDIA CORPORATION. All rights reserved. +# +# NVIDIA CORPORATION and its licensors retain all intellectual property +# and proprietary rights in and to this software, related documentation +# and any modifications thereto. Any use, reproduction, disclosure or +# distribution of this software and related documentation without an express +# license agreement from NVIDIA CORPORATION is strictly prohibited. + + +from estimater import * +from datareader import * +import argparse + + +if __name__=='__main__': + parser = argparse.ArgumentParser() + code_dir = os.path.dirname(os.path.realpath(__file__)) + parser.add_argument('--mesh_file', type=str, default=f'{code_dir}/demo_data/mustard0/mesh/textured_simple.obj') + parser.add_argument('--test_scene_dir', type=str, default=f'{code_dir}/demo_data/mustard0') + parser.add_argument('--est_refine_iter', type=int, default=5) + parser.add_argument('--track_refine_iter', type=int, default=2) + parser.add_argument('--debug', type=int, default=1) + parser.add_argument('--debug_dir', type=str, default=f'{code_dir}/debug') + args = parser.parse_args() + + set_logging_format() + set_seed(0) + + mesh = trimesh.load(args.mesh_file) + + debug = args.debug + debug_dir = args.debug_dir + os.system(f'rm -rf {debug_dir}/* && mkdir -p {debug_dir}/track_vis {debug_dir}/ob_in_cam') + + to_origin, extents = trimesh.bounds.oriented_bounds(mesh) + bbox = np.stack([-extents/2, extents/2], axis=0).reshape(2,3) + + scorer = ScorePredictor() + refiner = PoseRefinePredictor() + glctx = dr.RasterizeCudaContext() + est = FoundationPose(model_pts=mesh.vertices, model_normals=mesh.vertex_normals, mesh=mesh, scorer=scorer, refiner=refiner, debug_dir=debug_dir, debug=debug, glctx=glctx) + logging.info("estimator initialization done") + + reader = YcbineoatReader(video_dir=args.test_scene_dir, shorter_side=None, zfar=np.inf) + + for i in range(len(reader.color_files)): + logging.info(f'i:{i}') + color = reader.get_color(i) + depth = reader.get_depth(i) + if i==0: + mask = reader.get_mask(0).astype(bool) + pose = est.register(K=reader.K, rgb=color, depth=depth, ob_mask=mask, iteration=args.est_refine_iter) + + if debug>=3: + m = mesh.copy() + m.apply_transform(pose) + m.export(f'{debug_dir}/model_tf.obj') + xyz_map = depth2xyzmap(depth, reader.K) + valid = depth>=0.001 + pcd = toOpen3dCloud(xyz_map[valid], color[valid]) + o3d.io.write_point_cloud(f'{debug_dir}/scene_complete.ply', pcd) + else: + pose = est.track_one(rgb=color, depth=depth, K=reader.K, iteration=args.track_refine_iter) + + os.makedirs(f'{debug_dir}/ob_in_cam', exist_ok=True) + np.savetxt(f'{debug_dir}/ob_in_cam/{reader.id_strs[i]}.txt', pose.reshape(4,4)) + + if debug>=1: + center_pose = pose@np.linalg.inv(to_origin) + vis = draw_posed_3d_box(reader.K, img=color, ob_in_cam=center_pose, bbox=bbox) + vis = draw_xyz_axis(color, ob_in_cam=center_pose, scale=0.1, K=reader.K, thickness=3, transparency=0, is_input_rgb=True) + cv2.imshow('1', vis[...,::-1]) + cv2.waitKey(1) + + + if debug>=2: + os.makedirs(f'{debug_dir}/track_vis', exist_ok=True) + imageio.imwrite(f'{debug_dir}/track_vis/{reader.id_strs[i]}.png', vis) + diff --git a/third_party/FoundationPose/run_linemod.py b/third_party/FoundationPose/run_linemod.py new file mode 100644 index 0000000..b927bc6 --- /dev/null +++ b/third_party/FoundationPose/run_linemod.py @@ -0,0 +1,149 @@ +# Copyright (c) 2023, NVIDIA CORPORATION. All rights reserved. +# +# NVIDIA CORPORATION and its licensors retain all intellectual property +# and proprietary rights in and to this software, related documentation +# and any modifications thereto. Any use, reproduction, disclosure or +# distribution of this software and related documentation without an express +# license agreement from NVIDIA CORPORATION is strictly prohibited. + + +from Utils import * +import json,uuid,joblib,os,sys +import scipy.spatial as spatial +from multiprocessing import Pool +import multiprocessing +from functools import partial +from itertools import repeat +import itertools +from datareader import * +from estimater import * +code_dir = os.path.dirname(os.path.realpath(__file__)) +sys.path.append(f'{code_dir}/mycpp/build') +import yaml + + + +def get_mask(reader, i_frame, ob_id, detect_type): + if detect_type=='box': + mask = reader.get_mask(i_frame, ob_id) + H,W = mask.shape[:2] + vs,us = np.where(mask>0) + umin = us.min() + umax = us.max() + vmin = vs.min() + vmax = vs.max() + valid = np.zeros((H,W), dtype=bool) + valid[vmin:vmax,umin:umax] = 1 + elif detect_type=='mask': + mask = reader.get_mask(i_frame, ob_id) + if mask is None: + return None + valid = mask>0 + elif detect_type=='detected': + mask = cv2.imread(reader.color_files[i_frame].replace('rgb','mask_cosypose'), -1) + valid = mask==ob_id + else: + raise RuntimeError + return valid + + + +def run_pose_estimation_worker(reader, i_frames, est:FoundationPose=None, debug=0, ob_id=None, device='cuda:0'): + torch.cuda.set_device(device) + est.to_device(device) + est.glctx = dr.RasterizeCudaContext(device=device) + + result = NestDict() + + for i, i_frame in enumerate(i_frames): + logging.info(f"{i}/{len(i_frames)}, i_frame:{i_frame}, ob_id:{ob_id}") + video_id = reader.get_video_id() + color = reader.get_color(i_frame) + depth = reader.get_depth(i_frame) + id_str = reader.id_strs[i_frame] + H,W = color.shape[:2] + + debug_dir =est.debug_dir + + ob_mask = get_mask(reader, i_frame, ob_id, detect_type=detect_type) + if ob_mask is None: + logging.info("ob_mask not found, skip") + result[video_id][id_str][ob_id] = np.eye(4) + return result + + est.gt_pose = reader.get_gt_pose(i_frame, ob_id) + + pose = est.register(K=reader.K, rgb=color, depth=depth, ob_mask=ob_mask, ob_id=ob_id) + logging.info(f"pose:\n{pose}") + + if debug>=3: + m = est.mesh_ori.copy() + tmp = m.copy() + tmp.apply_transform(pose) + tmp.export(f'{debug_dir}/model_tf.obj') + + result[video_id][id_str][ob_id] = pose + + return result + + +def run_pose_estimation(): + wp.force_load(device='cuda') + reader_tmp = LinemodReader(f'{opt.linemod_dir}/lm_test_all/test/000002', split=None) + + debug = opt.debug + use_reconstructed_mesh = opt.use_reconstructed_mesh + debug_dir = opt.debug_dir + + res = NestDict() + glctx = dr.RasterizeCudaContext() + mesh_tmp = trimesh.primitives.Box(extents=np.ones((3)), transform=np.eye(4)).to_mesh() + est = FoundationPose(model_pts=mesh_tmp.vertices.copy(), model_normals=mesh_tmp.vertex_normals.copy(), symmetry_tfs=None, mesh=mesh_tmp, scorer=None, refiner=None, glctx=glctx, debug_dir=debug_dir, debug=debug) + + for ob_id in reader_tmp.ob_ids: + ob_id = int(ob_id) + if use_reconstructed_mesh: + mesh = reader_tmp.get_reconstructed_mesh(ob_id, ref_view_dir=opt.ref_view_dir) + else: + mesh = reader_tmp.get_gt_mesh(ob_id) + symmetry_tfs = reader_tmp.symmetry_tfs[ob_id] + + args = [] + + video_dir = f'{opt.linemod_dir}/lm_test_all/test/{ob_id:06d}' + reader = LinemodReader(video_dir, split=None) + video_id = reader.get_video_id() + est.reset_object(model_pts=mesh.vertices.copy(), model_normals=mesh.vertex_normals.copy(), symmetry_tfs=symmetry_tfs, mesh=mesh) + + for i in range(len(reader.color_files)): + args.append((reader, [i], est, debug, ob_id, "cuda:0")) + + outs = [] + for arg in args: + out = run_pose_estimation_worker(*arg) + outs.append(out) + + for out in outs: + for video_id in out: + for id_str in out[video_id]: + for ob_id in out[video_id][id_str]: + res[video_id][id_str][ob_id] = out[video_id][id_str][ob_id] + + with open(f'{opt.debug_dir}/linemod_res.yml','w') as ff: + yaml.safe_dump(make_yaml_dumpable(res), ff) + + +if __name__=='__main__': + parser = argparse.ArgumentParser() + code_dir = os.path.dirname(os.path.realpath(__file__)) + parser.add_argument('--linemod_dir', type=str, default="/mnt/9a72c439-d0a7-45e8-8d20-d7a235d02763/DATASET/LINEMOD", help="linemod root dir") + parser.add_argument('--use_reconstructed_mesh', type=int, default=0) + parser.add_argument('--ref_view_dir', type=str, default="/mnt/9a72c439-d0a7-45e8-8d20-d7a235d02763/DATASET/YCB_Video/bowen_addon/ref_views_16") + parser.add_argument('--debug', type=int, default=0) + parser.add_argument('--debug_dir', type=str, default=f'{code_dir}/debug') + opt = parser.parse_args() + set_seed(0) + + detect_type = 'mask' # mask / box / detected + + run_pose_estimation() diff --git a/third_party/FoundationPose/run_ycb_video.py b/third_party/FoundationPose/run_ycb_video.py new file mode 100644 index 0000000..5a5968b --- /dev/null +++ b/third_party/FoundationPose/run_ycb_video.py @@ -0,0 +1,149 @@ +# Copyright (c) 2023, NVIDIA CORPORATION. All rights reserved. +# +# NVIDIA CORPORATION and its licensors retain all intellectual property +# and proprietary rights in and to this software, related documentation +# and any modifications thereto. Any use, reproduction, disclosure or +# distribution of this software and related documentation without an express +# license agreement from NVIDIA CORPORATION is strictly prohibited. + + +from Utils import * +import json,uuid,joblib,os,sys,argparse +from datareader import * +from estimater import * +code_dir = os.path.dirname(os.path.realpath(__file__)) +sys.path.append(f'{code_dir}/mycpp/build') +import yaml + + +def get_mask(reader, i_frame, ob_id, detect_type): + if detect_type=='box': + mask = reader.get_mask(i_frame, ob_id) + H,W = mask.shape[:2] + vs,us = np.where(mask>0) + umin = us.min() + umax = us.max() + vmin = vs.min() + vmax = vs.max() + valid = np.zeros((H,W), dtype=bool) + valid[vmin:vmax,umin:umax] = 1 + elif detect_type=='mask': + mask = reader.get_mask(i_frame, ob_id, type='mask_visib') + valid = mask>0 + elif detect_type=='cnos': #https://github.com/nv-nguyen/cnos + mask = cv2.imread(reader.color_files[i_frame].replace('rgb','mask_cnos'), -1) + valid = mask==ob_id + else: + raise RuntimeError + + return valid + + + +def run_pose_estimation_worker(reader, i_frames, est:FoundationPose, debug=False, ob_id=None, device:int=0): + result = NestDict() + torch.cuda.set_device(device) + est.to_device(f'cuda:{device}') + est.glctx = dr.RasterizeCudaContext(device) + debug_dir = est.debug_dir + + for i in range(len(i_frames)): + i_frame = i_frames[i] + id_str = reader.id_strs[i_frame] + logging.info(f"{i}/{len(i_frames)}, video:{reader.get_video_id()}, id_str:{id_str}") + color = reader.get_color(i_frame) + depth = reader.get_depth(i_frame) + + H,W = color.shape[:2] + scene_ob_ids = reader.get_instance_ids_in_image(i_frame) + video_id = reader.get_video_id() + + logging.info(f"video:{reader.get_video_id()}, id_str:{id_str}, ob_id:{ob_id}") + if ob_id not in scene_ob_ids: + logging.info(f'skip {ob_id} as it does not exist in this scene') + continue + ob_mask = get_mask(reader, i_frame, ob_id, detect_type=detect_type) + + est.gt_pose = reader.get_gt_pose(i_frame, ob_id) + pose = est.register(K=reader.K, rgb=color, depth=depth, ob_mask=ob_mask, ob_id=ob_id, iteration=5) + logging.info(f"pose:\n{pose}") + + + if debug>=3: + tmp = est.mesh_ori.copy() + tmp.apply_transform(pose) + tmp.export(f'{debug_dir}/model_tf.obj') + + result[video_id][id_str][ob_id] = pose + + return result + + +def run_pose_estimation(): + wp.force_load(device='cuda') + video_dirs = sorted(glob.glob(f'{opt.ycbv_dir}/test/*')) + res = NestDict() + + debug = opt.debug + use_reconstructed_mesh = opt.use_reconstructed_mesh + debug_dir = opt.debug_dir + + reader_tmp = YcbVideoReader(video_dirs[0]) + glctx = dr.RasterizeCudaContext() + mesh_tmp = trimesh.primitives.Box(extents=np.ones((3)), transform=np.eye(4)) + est = FoundationPose(model_pts=mesh_tmp.vertices.copy(), model_normals=mesh_tmp.vertex_normals.copy(), symmetry_tfs=None, mesh=mesh_tmp, scorer=None, refiner=None, glctx=glctx, debug_dir=debug_dir, debug=debug) + + ob_ids = reader_tmp.ob_ids + + for ob_id in ob_ids: + if use_reconstructed_mesh: + mesh = reader_tmp.get_reconstructed_mesh(ob_id, ref_view_dir=opt.ref_view_dir) + else: + mesh = reader_tmp.get_gt_mesh(ob_id) + symmetry_tfs = reader_tmp.symmetry_tfs[ob_id] + + args = [] + for video_dir in video_dirs: + reader = YcbVideoReader(video_dir, zfar=1.5) + scene_ob_ids = reader.get_instance_ids_in_image(0) + if ob_id not in scene_ob_ids: + continue + video_id = reader.get_video_id() + + for i in range(len(reader.color_files)): + if not reader.is_keyframe(i): + continue + args.append((reader, [i], est, debug, ob_id, 0)) + + est.reset_object(model_pts=mesh.vertices.copy(), model_normals=mesh.vertex_normals.copy(), symmetry_tfs=symmetry_tfs, mesh=mesh) + outs = [] + for arg in args: + out = run_pose_estimation_worker(*arg) + outs.append(out) + + for out in outs: + for video_id in out: + for id_str in out[video_id]: + res[video_id][id_str][ob_id] = out[video_id][id_str][ob_id] + + with open(f'{opt.debug_dir}/ycbv_res.yml','w') as ff: + yaml.safe_dump(make_yaml_dumpable(res), ff) + + + +if __name__=='__main__': + parser = argparse.ArgumentParser() + code_dir = os.path.dirname(os.path.realpath(__file__)) + parser.add_argument('--ycbv_dir', type=str, default="/mnt/9a72c439-d0a7-45e8-8d20-d7a235d02763/DATASET/YCB_Video", help="data dir") + parser.add_argument('--use_reconstructed_mesh', type=int, default=0) + parser.add_argument('--ref_view_dir', type=str, default="/mnt/9a72c439-d0a7-45e8-8d20-d7a235d02763/DATASET/YCB_Video/bowen_addon/ref_views_16") + parser.add_argument('--debug', type=int, default=0) + parser.add_argument('--debug_dir', type=str, default=f'{code_dir}/debug') + opt = parser.parse_args() + os.environ["YCB_VIDEO_DIR"] = opt.ycbv_dir + + set_seed(0) + + detect_type = 'mask' # mask / box / detected + + run_pose_estimation() diff --git a/third_party/FoundationPose/weights/2023-10-28-18-33-37/config.yml b/third_party/FoundationPose/weights/2023-10-28-18-33-37/config.yml new file mode 100644 index 0000000..d962a1d --- /dev/null +++ b/third_party/FoundationPose/weights/2023-10-28-18-33-37/config.yml @@ -0,0 +1,39 @@ +lr: 0.0001 +c_in: 6 +zfar: 'Infinity' +debug: null +w_rot: 0.1 +n_view: 1 +run_id: null +use_BN: true +rot_rep: axis_angle +ckpt_dir: null +exp_name: 2023-10-28-18-33-37 +save_dir: /tmp/2023-10-28-18-33-37/ +loss_type: l2 +optimizer: adam +trans_rep: tracknet +batch_size: 64 +crop_ratio: 1.2 +use_normal: false +BN_momentum: 0.1 +max_num_key: null +warmup_step: -1 +input_resize: +- 160 +- 160 +max_step_val: 1000 +normal_uint8: false +vis_interval: 1000 +weight_decay: 0 +n_max_objects: null +normalize_xyz: true +clip_grad_norm: 'Infinity' +rot_normalizer: 0.3490658503988659 +trans_normalizer: +- 0.019999999552965164 +- 0.019999999552965164 +- 0.05000000074505806 +max_step_per_epoch: 25000 +val_epoch_interval: 10 +n_dataloader_workers: 60 diff --git a/third_party/FoundationPose/weights/2023-10-28-18-33-37/model_best.pth.part-000 b/third_party/FoundationPose/weights/2023-10-28-18-33-37/model_best.pth.part-000 new file mode 100644 index 0000000..3fef31f --- /dev/null +++ b/third_party/FoundationPose/weights/2023-10-28-18-33-37/model_best.pth.part-000 @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:d1187f2cc55ae8f934bbac8926f9e8f26ba59f418e7d41541c510458c4e2c6d7 +size 50331648 diff --git a/third_party/FoundationPose/weights/2023-10-28-18-33-37/model_best.pth.part-001 b/third_party/FoundationPose/weights/2023-10-28-18-33-37/model_best.pth.part-001 new file mode 100644 index 0000000..d1b41cf --- /dev/null +++ b/third_party/FoundationPose/weights/2023-10-28-18-33-37/model_best.pth.part-001 @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:dd58bf0caffb6a934b5d934a149630ba34d6f409c48618510b37ee9b1d292f55 +size 17888461 diff --git a/third_party/FoundationPose/weights/2024-01-11-20-02-45/config.yml b/third_party/FoundationPose/weights/2024-01-11-20-02-45/config.yml new file mode 100644 index 0000000..69cf5c0 --- /dev/null +++ b/third_party/FoundationPose/weights/2024-01-11-20-02-45/config.yml @@ -0,0 +1,41 @@ +lr: 0.0001 +c_in: 6 +zfar: 'Infinity' +debug: null +n_view: 1 +run_id: 3wy8qqex +use_BN: true +exp_name: 2024-01-11-20-02-45 +n_epochs: 62 +save_dir: /home/bowenw/debug/2024-01-11-20-02-45/ +use_mask: false +loss_type: pairwise_valid +optimizer: adam +batch_size: 64 +crop_ratio: 1.1 +enable_amp: true +use_normal: false +max_num_key: null +warmup_step: -1 +input_resize: +- 160 +- 160 +max_step_val: 1000 +vis_interval: 1000 +weight_decay: 0 +normalize_xyz: true +resume_run_id: null +clip_grad_norm: 'Infinity' +lr_epoch_decay: 500 +render_backend: nvdiffrast +train_num_pair: 5 +lr_decay_epochs: +- 50 +n_epochs_warmup: 1 +make_pair_online: false +gradient_max_norm: 'Infinity' +max_step_per_epoch: 10000 +n_rendering_workers: 1 +save_epoch_interval: 100 +n_dataloader_workers: 100 +split_objects_across_gpus: true diff --git a/third_party/FoundationPose/weights/2024-01-11-20-02-45/model_best.pth.part-000 b/third_party/FoundationPose/weights/2024-01-11-20-02-45/model_best.pth.part-000 new file mode 100644 index 0000000..82759bf --- /dev/null +++ b/third_party/FoundationPose/weights/2024-01-11-20-02-45/model_best.pth.part-000 @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:41e83db84fae9dbaa7920a908de728613c55387d1207dc20fbf1132dcf2153fc +size 50331648 diff --git a/third_party/FoundationPose/weights/2024-01-11-20-02-45/model_best.pth.part-001 b/third_party/FoundationPose/weights/2024-01-11-20-02-45/model_best.pth.part-001 new file mode 100644 index 0000000..0e06aa4 --- /dev/null +++ b/third_party/FoundationPose/weights/2024-01-11-20-02-45/model_best.pth.part-001 @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:57ae9d0e6697500c1e864df2b76d99bfbb821ac84caa99a7da9f9c8d2a3c0c21 +size 50331648 diff --git a/third_party/FoundationPose/weights/2024-01-11-20-02-45/model_best.pth.part-002 b/third_party/FoundationPose/weights/2024-01-11-20-02-45/model_best.pth.part-002 new file mode 100644 index 0000000..68baa2b --- /dev/null +++ b/third_party/FoundationPose/weights/2024-01-11-20-02-45/model_best.pth.part-002 @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:f4dc6beac31aa5705f1cdd3583ae83423e8012b7fcd7b5cb435e79233088d640 +size 50331648 diff --git a/third_party/FoundationPose/weights/2024-01-11-20-02-45/model_best.pth.part-003 b/third_party/FoundationPose/weights/2024-01-11-20-02-45/model_best.pth.part-003 new file mode 100644 index 0000000..403e9c8 --- /dev/null +++ b/third_party/FoundationPose/weights/2024-01-11-20-02-45/model_best.pth.part-003 @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:54bd0d36f9a4ca662790b4d6ee1d001b333559324918bef9b61624b120ef96ae +size 39234445 diff --git a/third_party/FoundationPose/weights/download_manifest.json b/third_party/FoundationPose/weights/download_manifest.json new file mode 100644 index 0000000..a226022 --- /dev/null +++ b/third_party/FoundationPose/weights/download_manifest.json @@ -0,0 +1,30 @@ +{ + "source": "community mirror gpue/foundationpose-weights; official Google Drive quota exceeded", + "verification": "size and mirror LFS SHA256; not independently compared to official files", + "files": [ + { + "path": "2023-10-28-18-33-37/config.yml", + "size": 708, + "sha256": "28a6ba94a33230ee5fc3c51939486281578b0972542bd9e38ca6123e75605686", + "url": "https://hf-mirror.com/gpue/foundationpose-weights/resolve/main/2023-10-28-18-33-37/config.yml" + }, + { + "path": "2023-10-28-18-33-37/model_best.pth", + "size": 68220109, + "sha256": "774700586ddc435d408fc01c9809c43e151232936369dfbea0f0f964ba471d60", + "url": "https://hf-mirror.com/gpue/foundationpose-weights/resolve/main/2023-10-28-18-33-37/model_best.pth" + }, + { + "path": "2024-01-11-20-02-45/config.yml", + "size": 778, + "sha256": "a79db4de3b95885dd5ae86833b37b8698a75dad81e87d1086cd50b2fcd8dda3f", + "url": "https://hf-mirror.com/gpue/foundationpose-weights/resolve/main/2024-01-11-20-02-45/config.yml" + }, + { + "path": "2024-01-11-20-02-45/model_best.pth", + "size": 190229389, + "sha256": "81924d384bf5c26c646ee4783104982ae3d1e049c181c36641b6a7aeae494c26", + "url": "https://hf-mirror.com/gpue/foundationpose-weights/resolve/main/2024-01-11-20-02-45/model_best.pth" + } + ] +} \ No newline at end of file diff --git a/third_party/bottle_model_parametric/README.md b/third_party/bottle_model_parametric/README.md new file mode 100644 index 0000000..cc84d3a --- /dev/null +++ b/third_party/bottle_model_parametric/README.md @@ -0,0 +1,16 @@ +# 按尺寸重新建立的理想化瓶子 + +高度 **157.856 mm**,最大直径 **54.110 mm**。这两个尺寸沿用用户提供的瓶子资产;不再使用其异常收腰的半径轮廓。 + +圆柱瓶身高 103 mm,锥形瓶肩至 139 mm,瓶颈直径 28 mm,顶端总高 157.856 mm。瓶肩和瓶颈比例是设计选择,没有宣称还原实物细节。顶底封闭,无内壁、瓶底凹槽、螺纹或独立瓶盖。 + +- `bottle.xml`:可直接加载的 MuJoCo 自由瓶子;视觉网格与三段碰撞几何使用同一轮廓。 +- `assets/bottle.obj`、`bottle.stl`、`bottle.glb`:网格格式,均以米为单位。 +- `preview.png`:预览;`validation.json`:尺寸、封闭性及加载检查。 +- `profile.npz`:新设计的高度/半径控制点。 + +碰撞为圆柱瓶身、凸锥台瓶肩、圆柱瓶颈,去掉旧版十段忽大忽小的圆柱。质量暂沿用旧 XML 的 0.2308 kg,惯量按均匀实心模型重算;质量、材料及质量分布未实测。原始三目资产未修改。 + +重建命令:`env PYTHONPATH= LD_LIBRARY_PATH= .dex/bin/python scripts/build_parametric_bottle.py`。 + +新资产已接入 `output/l20_2047635068/bottle_grasp_v2/contact_scene.xml` 并通过加载检查。该场景是后续抓取优化入口,尚未运行新版物理抓取验证。 diff --git a/third_party/bottle_model_parametric/assets/bottle.obj b/third_party/bottle_model_parametric/assets/bottle.obj new file mode 100644 index 0000000..b90da32 --- /dev/null +++ b/third_party/bottle_model_parametric/assets/bottle.obj @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:e4d48245adfed5ea0a72ee3dc247b68c6d419cf1ccf5651cc8f7576fe2f440f2 +size 32331 diff --git a/third_party/bottle_model_parametric/assets/shoulder.obj b/third_party/bottle_model_parametric/assets/shoulder.obj new file mode 100644 index 0000000..06de99c --- /dev/null +++ b/third_party/bottle_model_parametric/assets/shoulder.obj @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:d27b809f8510e37fe20c14c50f43a6977e4b6c1c62edbbaf258f8df76b7a3c92 +size 15949 diff --git a/third_party/bottle_model_parametric/bottle.glb b/third_party/bottle_model_parametric/bottle.glb new file mode 100644 index 0000000000000000000000000000000000000000..0245038613625446b3e69ec47c79c38504215121 GIT binary patch literal 19252 zcmZ{q2Xs}{+C`&cLq$=sV1p}mgd~s6a^JT1*Hdw5Q0fSX^IN=hP{i}dsie! zM6qD+f{MNOhGO@v`DJZ?#(VybvEDb|T!(wl-TRy^Z;Ur~!oZ$u78De;@2meA8v3qbs1oBW51aIya)WwxTXqK0P<3qNcX8dSb3T zSKOvB*E%;=f3#*qU3HDjxVpMYwdHNwj;*X4H+f{6QPov#t18D=j2_pvuBNi8qIO(v zrao!ZsEXR!>Y9dc(KVB*Css_X8#r~6ZlR>8xTJNiE;u)!d)NKC9;ELWIeE;Oikd-{ z6-UQSj;fwKQTNiWutRa{T-AtUbdx1T?R1k>mAWr(xb^hh|9)`kKOa13pYHAW*r|0D z{l|=nAEw{F`nMFd)(6D5loYj3UsBknu((5ENl9U0VOd#eyW*n4(z4e7k1Q(bP*ze} zSXkV?eaDVv9orTEb9;OK>!0{{t4m|`S-aQ_ZOe!%t&^6$x~ihCX6i8ZpyHUinh{#h z>A6WY)vRvC=v;YS&EyI#_av=+WnJYI|M3xZ+KQ2r>-ewJa|iSv(0gF-{{3?0MY^%9yP9V!swa`R=p@z{on2`Hom-&r>J3x z8+M-6->+ig*t&80_OkXRrT=#UJMD{N=i`q4d$(5q;Mf45(oxZWO zsI*K!u7&>Ds#iflx8D8qLKGCNp&TF7FlX8E&YWOoat%(eP{E02CLOMkGTY3g!>z5% zF*E6K>nJCgnRK{ym6Od(I^257DP|@eZhhrcGm{S2SUJtiq{D5XoNi{);WkvxFf-|J z8!2a+nRK`&%2{S69d2XgY%`M%w~2C&nMsFhs?0Sr>2RAW=bD*xxXqOF%uG7m=F0hI zCLOMsa)FsihucEA(9ER6ZK+&jX42udQZ6<#>2O;smzbG!xaP{GW+okO8|5-HlMc77 za=DpFhucoM!px+@ZLeHuX42tyP_8mF>2NKSc?!&=YoWAMt~N90DY%?6-^`@L?WkO1 zX42tWDc72rbhw?A>#+|J7NW+okO7v%;slMc75a-*3^huclL$;_m~?XKKxX42vI zP;N0Z>2R%;Tg^;5+@8v9W+ol3jdHu0Nr!8z++k+Y;R=;I%}hF6k#d)rNrx*|?lv>& zaP5?P%uG65d*xm;lMYv++-GLe;YyYJ%}hF62ju}XlMYv=EHE?aa2=Hg%}hF6xw6pA zq{DSm>dj0#Twck{Ogda=GsRe8kBT&&=_ zDUX_&bhz%y5;Kzy*F$;C%%sEZqdabA(&2h4Pnel>xP6r;%}hF6FJ-BjNr&sLJY{Cm z;rb{~o0)XDzRELZCLL}+!&<#X42vMD=(OtbhrbQ7tKsM+=0qV zW+ol(AmwE2QOUH_c2s+z{n0Gm{Q?xU#~`q{AJdylrOE;f5;jn3;6AVamH^CLM0L@}2@S z>4qyKl=scd_Y~Yn@V3X42ut zDW92{bht|8b2F0;cck)#nMsEmuY74{(%~j3UzwS7xGLpqGm{QCQTfKqq{CG!-2Sv> zf18B>2T*Lo0yq&xVcJGGm{Q?uCl3_NryX6+04wO!=0~eZf4TqE>N18 znRK`dl`YInI^0FdmS!d$?qX#tGm{Q?iL$ksNr$^sX>MlH;Vx6QF*E6Kmn++vnRK`- zl?kZ&mGm{QCPibLh(&4UFTAG=3xcN%X%%sCzqwHvA(&4UETA7)2 zxa*Xi%uG7m^~%m>CLQhuWfwD(4tJxntC>lMyGhy2%%sEJtn6-P(&27V_AoQ)aJMS0 z%}hGnZOWczCLQi}rHz?Mhr2^*Yi82n?o2Qmcz0FKI+#^a?Gm{SYsM5{Mq{A&yx|^AF zxW|+pW+ol(ab+JflMeTU($mbO!#%0&Yi82nmMXo>Ogh|CN^dih4)?Uu$IPU|J)`tB zGwE>8D*Kt4bhzh~{mo1|-1AC5Gm{SYg3{m2q{F?a9AIYB;a*Y>G&AXNFDnO`nRK{U zlmTWY9qv_SpqWXBdrdjm%%sD;t_(6W>2S-GL(EJ%+;ZhmGm{SYhH{vhNr!t=8Ej_K z;oee)n3;6A70TgeCLQi=eDZ|Z7I^6rp2s4uo_kl9f z%%sD8sEjf*>2Mz@qs>e@+{a3VnMsHHL>Xgd(&0W;#+sRQxX+YvW+ol(bEVSEq{Dro z9BF3K;l5PHo0)XDuapU9CLQi;rOM2t!+oPnG&AXN-zwE+CLQiOWs;dmhx=YR%FLw0 z{h-vCnRK`xm0B~C4!2UNGc)OMKPi*VOgh}p$`mt`4!24<+RUWG{h}OWX42t)Ri>Jm zbhzJ?X=WxJ?ssLnnMsHHLz!V_(&7G8W}2CFxWANR%}hGn-^y`jCLL~;QebA%G1pLz zH#2A1t*Lo}nMsFROF7ZZq{B5*W}BIGxV4ozW+okO9pxl5lMc78agbZD!KpHc`$o zGwEO%%sC@qg-ZY(&4sME;lpjaN8+Yn3;6A?UgIdOgh{S%2j42 z9j=8k&&;I5wN$P)GwE}L{wNkD%GwE&;9$+%C!u zW+okOSLH@ClMc6=a+8@!hudAb+03NF?V;RaX42tWE4P}Nbhtg0+ssTlTpQ(fGm{S2 zR=LB>q{9^|cbb`WxFY2)Gm{QitlVv8(&5@E_n4VEonMsH1q|}?4bhx~dnVEFB&dNh(CLOMevdGM& z!|kO!Y-ZBo_Er{~nRK|W$|Ghb9j=@5sF_KJ>#i&@GwE9j?Fff|*H&J3x8S%%sB|sJvum(%}wLUN$r7a08TA%uG7mK;>04lMZ*V@|u}R zha054Zf4Tq4pEkwnRK{AmE~q89qusY4KtGtH&}Vo%%sB&QQk5$>2QZDE6hwf+!4y# zW+okOsPc}PNrxMzylZCC;f5>knVEFB5z6~!CLM01@`0I2ha07QXlBykMk^ngnRK`c zm#LT3_ja5E1GwE>Sl+VmeI$WjlxtU3aJ5u?=%%sDOSH3he>2MR2ugpw3 zT$S>*nMsG6sC;8)(&4I=Z_P|P+$7~YGm{Q?l=8iqNr$UZelRoXaJ9;hW+ol3PFZPY z(%~j6Kbe_yxGBodW+ol(Xl0d|NryW|`Nhnn!%bCwH8bgO)0E%LOgh|j<##ia4mU&j z!_1_^%~bw0GwE>0Du0=obhzV`zs*cKT!Avn%%m$=GqgZcj6dF9|CK)y9(RKMT5{3p zKX517Hn;_Up^h;c=(hHpk_M6Gk;c@5LZ!SlN$DMEAOpXqZyTE=6IXXPn?_O0dU@VHy;_mrc<<8HHWBS(kF z-EQAjjt-Bz!@f|C4v)LjzDSM^kGsphSdI>lyW75<9338ak9~VNIy~-P`w}@iJnlaG zQaL(2?tc3Ya&&my1NLQdba>nX`;KySc-({b<#Kd*+(P?Ka&&lHy?tJe4v)+1JIm4G zaSz#dk)y-o7TND5M~BBfY`?c09UiyXzN;J^9`}fSH#s^y?os>ha&&my68j!oj z_WQ`u;c<`K_mrc<7=H%F*F*FWDa?M~BC~Y(GGb z4v%}qexMv39`~yK!E$tX+-vrOBt*&ixLhsQ0qKTM7ek9)&@upAv8 z_on?2IXXP>IZkL)Yt=T337CJ+*kHha&&my*Y*?T=BVx1S+LhsXV4KU0nlkNea9SUEa8?l1e} zs|NDc-%~ERTzn#bPPxGN~=SLLWUJmEyuE;<5^q!sIL-KFs8#g;4I6wT2 z{LGcN1~>id^?c9DWx@Tod^LaBt?{p^g2lJJnD20Kv-sSZOP(l4!d+XEp>VC3x zKf2E>-KXw9OZTtem8IXM-r5UkGdMxxOCu=LGxngZ;h1zMqWieV@U; z|6u>FVE^7=uS2lcC)n#2?DY)xItP3GgS{`o-mhTqW3cx(*!v#r=ONh7NwA-vU_V#E ze%^xp90vRO4EA#y?B_Yy&w230LmrsV9~#zFdIo!)gT4O2-j`tS zSFran*!vsoeGm5Y5bWn9*w0U}pQ~U$Z^3>JgZ+F4`?(EfEm>36$Ip3iI{)qC{D=L% zq|b@-AMf`oyx+TEzmLIwPlNsb2K&7Z_WK^}b0FB~L$J?{V4o+!K4*e`{sgDk20a613P`}~K~`46Y_e>J~wwd?EQ zbpD5L`1ecrbpGSh`46Y_A5P~#oX&qZo&Rt;|KW80!|D8o)A-Qyn z4u6kIpU>Z~g8kkF`+W@ddm8NbH`woWu;2G!p98@@AA)^u1p7P*_Bj*m^Cvi+|NNG8 z{=@0~htv5Fr}H09=Rcgze>k20a613tbpFHX{D;%|5BuDXXVvFUG&0|Q-a4J|Kd%~|Z}fL@@QhhQ^VQ!D4PJWO zkbIjx<^X7{GGcFB2>WcyS)6Tl1;rhY-^L<)97JO}=KKaeME)U*dQIGu0Hk9iFvhO<~rU zwT25D&1}gY*e^_8*Dp-h>vQVU=jik6)934Z>(lq@e(KZx=sxSyed_+})BWpr)u-R3 z-&>!4uhyYHt%KI5KCO?|ElcaB^~}JfA(_H~PIu*Ymyv`#lQw`xWf>F4*s5u;0^QzrVqLuY>)*2m2fd_W2O( zb0gU2NwCkEV4i*Ufcxj2U=O@soL9lTL+k;bcZ)rMdFR*zn0JvqfO$vR1DJQ0J%D+q z*#nq&ojr)_8_wb2I2&|E;5jqc1DLafJ%Bl5*aJA7|7%>=GWl}Y*9ZG^I^q5K!JIwp z0p9l$?E4J%{Wo0i^PlVed&7Gjg1tV$Ubj5gbM~+Yu-7@*>mTfW3HE*kdmn?nzro)3 zU_TGReolh@`~>^C3ik6B?B_7p&u6fo+hEQf)|C7Aa~_<|fBQK9VZSfwbK?BR`~3>< z_b%A)W3b=TV86e?ey@Z5z6bjp2=@69>~kZSXPG_V9(>LO`}_$`=Ren^^B+#D4Rhtv7LnqRou^}*@<$EWij zPUkK=O@_DRj{A8U_Xb!>HOz@()kbjd5&w+`HxTMKb+2gUmxc`oX&qZo&T`k zyL3PNeJt4TX|UhlV87SFe&2(A4g~vr2==)V?DHhp=S;BApWt- +