liyang/hand-motion-pipeline
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Initial snapshot: hand motion pipeline (Dyn-HaMR + dex-retargeting + SPIDER)

Browse Source 
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
liyang
2026-09-15 14:13:49 +08:00
commit 7e4ef6f98b
7002 changed files with 1452249 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
third_party/hamer/third-party/ViTPose/docs/en/tasks/2d_animal_keypoint.md
Vendored
+448
View File
@@ -0,0 +1,448 @@
# 2D Animal Keypoint Dataset
It is recommended to symlink the dataset root to `$MMPOSE/data`.
If your folder structure is different, you may need to change the corresponding paths in config files.
MMPose supported datasets:
- [Animal-Pose](#animal-pose) \[ [Homepage](https://sites.google.com/view/animal-pose/) \]
- [AP-10K](#ap-10k) \[ [Homepage](https://github.com/AlexTheBad/AP-10K/) \]
- [Horse-10](#horse-10) \[ [Homepage](http://www.mackenziemathislab.org/horse10) \]
- [MacaquePose](#macaquepose) \[ [Homepage](http://www.pri.kyoto-u.ac.jp/datasets/macaquepose/index.html) \]
- [Vinegar Fly](#vinegar-fly) \[ [Homepage](https://github.com/jgraving/DeepPoseKit-Data) \]
- [Desert Locust](#desert-locust) \[ [Homepage](https://github.com/jgraving/DeepPoseKit-Data) \]
- [Grévy’s Zebra](#grvys-zebra) \[ [Homepage](https://github.com/jgraving/DeepPoseKit-Data) \]
- [ATRW](#atrw) \[ [Homepage](https://cvwc2019.github.io/challenge.html) \]
## Animal-Pose
<!-- [DATASET] -->
<details>
<summary align="right"><a href="http://openaccess.thecvf.com/content_ICCV_2019/html/Cao_Cross-Domain_Adaptation_for_Animal_Pose_Estimation_ICCV_2019_paper.html">Animal-Pose (ICCV'2019)</a></summary>
```bibtex
@InProceedings{Cao_2019_ICCV,
author = {Cao, Jinkun and Tang, Hongyang and Fang, Hao-Shu and Shen, Xiaoyong and Lu, Cewu and Tai, Yu-Wing},
title = {Cross-Domain Adaptation for Animal Pose Estimation},
booktitle = {The IEEE International Conference on Computer Vision (ICCV)},
month = {October},
year = {2019}
}
```
</details>
For [Animal-Pose](https://sites.google.com/view/animal-pose/) dataset, we prepare the dataset as follows:
1. Download the images of [PASCAL2011](http://www.google.com/url?q=http%3A%2F%2Fhost.robots.ox.ac.uk%2Fpascal%2FVOC%2Fvoc2011%2Findex.html&sa=D&sntz=1&usg=AFQjCNGmiJGkhSSWtShDe7NwRPyyyBUYSQ), especially the five categories (dog, cat, sheep, cow, horse), which we use as trainval dataset.
1. Download the [test-set](https://drive.google.com/drive/folders/1DwhQobZlGntOXxdm7vQsE4bqbFmN3b9y?usp=sharing) images with raw annotations (1000 images, 5 categories).
1. We have pre-processed the annotations to make it compatible with MMPose. Please download the annotation files from [annotations](https://download.openmmlab.com/mmpose/datasets/animalpose_annotations.tar). If you would like to generate the annotations by yourself, please check our dataset parsing [codes](/tools/dataset/parse_animalpose_dataset.py).
Extract them under {MMPose}/data, and make them look like this:
```text
mmpose
├── mmpose
├── docs
├── tests
├── tools
├── configs
`── data
│── animalpose
│
│-- VOC2011
│ │-- Annotations
│ │-- ImageSets
│ │-- JPEGImages
│ │-- SegmentationClass
│ │-- SegmentationObject
│
│-- animalpose_image_part2
│ │-- cat
│ │-- cow
│ │-- dog
│ │-- horse
│ │-- sheep
│
│-- annotations
│ │-- animalpose_train.json
│ |-- animalpose_val.json
│ |-- animalpose_trainval.json
│ │-- animalpose_test.json
│
│-- PASCAL2011_animal_annotation
│ │-- cat
│ │ |-- 2007_000528_1.xml
│ │ |-- 2007_000549_1.xml
│ │ │-- ...
│ │-- cow
│ │-- dog
│ │-- horse
│ │-- sheep
│
│-- annimalpose_anno2
│ │-- cat
│ │ |-- ca1.xml
│ │ |-- ca2.xml
│ │ │-- ...
│ │-- cow
│ │-- dog
│ │-- horse
│ │-- sheep
```
The official dataset does not provide the official train/val/test set split.
We choose the images from PascalVOC for train & val. In total, we have 3608 images and 5117 annotations for train+val, where
2798 images with 4000 annotations are used for training, and 810 images with 1117 annotations are used for validation.
Those images from other sources (1000 images with 1000 annotations) are used for testing.
## AP-10K
<!-- [DATASET] -->
<details>
<summary align="right"><a href="https://arxiv.org/abs/2108.12617">AP-10K (NeurIPS'2021)</a></summary>
```bibtex
@misc{yu2021ap10k,
title={AP-10K: A Benchmark for Animal Pose Estimation in the Wild},
author={Hang Yu and Yufei Xu and Jing Zhang and Wei Zhao and Ziyu Guan and Dacheng Tao},
year={2021},
eprint={2108.12617},
archivePrefix={arXiv},
primaryClass={cs.CV}
}
```
</details>
For [AP-10K](https://github.com/AlexTheBad/AP-10K/) dataset, images and annotations can be downloaded from [download](https://drive.google.com/file/d/1-FNNGcdtAQRehYYkGY1y4wzFNg4iWNad/view?usp=sharing).
Note, this data and annotation data is for non-commercial use only.
Extract them under {MMPose}/data, and make them look like this:
```text
mmpose
├── mmpose
├── docs
├── tests
├── tools
├── configs
`── data
│── ap10k
│-- annotations
│ │-- ap10k-train-split1.json
│ |-- ap10k-train-split2.json
│ |-- ap10k-train-split3.json
│ │-- ap10k-val-split1.json
│ |-- ap10k-val-split2.json
│ |-- ap10k-val-split3.json
│ |-- ap10k-test-split1.json
│ |-- ap10k-test-split2.json
│ |-- ap10k-test-split3.json
│-- data
│ │-- 000000000001.jpg
│ │-- 000000000002.jpg
│ │-- ...
```
The annotation files in 'annotation' folder contains 50 labeled animal species. There are total 10,015 labeled images with 13,028 instances in the AP-10K dataset. We randonly split them into train, val, and test set following the ratio of 7:1:2.
## Horse-10
<!-- [DATASET] -->
<details>
<summary align="right"><a href="https://openaccess.thecvf.com/content/WACV2021/html/Mathis_Pretraining_Boosts_Out-of-Domain_Robustness_for_Pose_Estimation_WACV_2021_paper.html">Horse-10 (WACV'2021)</a></summary>
```bibtex
@inproceedings{mathis2021pretraining,
title={Pretraining boosts out-of-domain robustness for pose estimation},
author={Mathis, Alexander and Biasi, Thomas and Schneider, Steffen and Yuksekgonul, Mert and Rogers, Byron and Bethge, Matthias and Mathis, Mackenzie W},
booktitle={Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision},
pages={1859--1868},
year={2021}
}
```
</details>
For [Horse-10](http://www.mackenziemathislab.org/horse10) dataset, images can be downloaded from [download](http://www.mackenziemathislab.org/horse10).
Please download the annotation files from [horse10_annotations](https://download.openmmlab.com/mmpose/datasets/horse10_annotations.tar). Note, this data and annotation data is for non-commercial use only, per the authors (see http://horse10.deeplabcut.org for more information).
Extract them under {MMPose}/data, and make them look like this:
```text
mmpose
├── mmpose
├── docs
├── tests
├── tools
├── configs
`── data
│── horse10
│-- annotations
│ │-- horse10-train-split1.json
│ |-- horse10-train-split2.json
│ |-- horse10-train-split3.json
│ │-- horse10-test-split1.json
│ |-- horse10-test-split2.json
│ |-- horse10-test-split3.json
│-- labeled-data
│ │-- BrownHorseinShadow
│ │-- BrownHorseintoshadow
│ │-- ...
```
## MacaquePose
<!-- [DATASET] -->
<details>
<summary align="right"><a href="https://www.ncbi.nlm.nih.gov/pmc/articles/pmc7874091/">MacaquePose (bioRxiv'2020)</a></summary>
```bibtex
@article{labuguen2020macaquepose,
title={MacaquePose: A novel ‘in the wild’macaque monkey pose dataset for markerless motion capture},
author={Labuguen, Rollyn and Matsumoto, Jumpei and Negrete, Salvador and Nishimaru, Hiroshi and Nishijo, Hisao and Takada, Masahiko and Go, Yasuhiro and Inoue, Ken-ichi and Shibata, Tomohiro},
journal={bioRxiv},
year={2020},
publisher={Cold Spring Harbor Laboratory}
}
```
</details>
For [MacaquePose](http://www.pri.kyoto-u.ac.jp/datasets/macaquepose/index.html) dataset, images can be downloaded from [download](http://www.pri.kyoto-u.ac.jp/datasets/macaquepose/index.html).
Please download the annotation files from [macaque_annotations](https://download.openmmlab.com/mmpose/datasets/macaque_annotations.tar).
Extract them under {MMPose}/data, and make them look like this:
```text
mmpose
├── mmpose
├── docs
├── tests
├── tools
├── configs
`── data
│── macaque
│-- annotations
│ │-- macaque_train.json
│ |-- macaque_test.json
│-- images
│ │-- 01418849d54b3005.jpg
│ │-- 0142d1d1a6904a70.jpg
│ │-- 01ef2c4c260321b7.jpg
│ │-- 020a1c75c8c85238.jpg
│ │-- 020b1506eef2557d.jpg
│ │-- ...
```
Since the official dataset does not provide the test set, we randomly select 12500 images for training, and the rest for evaluation (see [code](/tools/dataset/parse_macaquepose_dataset.py)).
## Vinegar Fly
<!-- [DATASET] -->
<details>
<summary align="right"><a href="https://www.nature.com/articles/s41592-018-0234-5">Vinegar Fly (Nature Methods'2019)</a></summary>
```bibtex
@article{pereira2019fast,
title={Fast animal pose estimation using deep neural networks},
author={Pereira, Talmo D and Aldarondo, Diego E and Willmore, Lindsay and Kislin, Mikhail and Wang, Samuel S-H and Murthy, Mala and Shaevitz, Joshua W},
journal={Nature methods},
volume={16},
number={1},
pages={117--125},
year={2019},
publisher={Nature Publishing Group}
}
```
</details>
For [Vinegar Fly](https://github.com/jgraving/DeepPoseKit-Data) dataset, images can be downloaded from [vinegar_fly_images](https://download.openmmlab.com/mmpose/datasets/vinegar_fly_images.tar).
Please download the annotation files from [vinegar_fly_annotations](https://download.openmmlab.com/mmpose/datasets/vinegar_fly_annotations.tar).
Extract them under {MMPose}/data, and make them look like this:
```text
mmpose
├── mmpose
├── docs
├── tests
├── tools
├── configs
`── data
│── fly
│-- annotations
│ │-- fly_train.json
│ |-- fly_test.json
│-- images
│ │-- 0.jpg
│ │-- 1.jpg
│ │-- 2.jpg
│ │-- 3.jpg
│ │-- ...
```
Since the official dataset does not provide the test set, we randomly select 90\% images for training, and the rest (10\%) for evaluation (see [code](/tools/dataset/parse_deepposekit_dataset.py)).
## Desert Locust
<!-- [DATASET] -->
<details>
<summary align="right"><a href="https://elifesciences.org/articles/47994">Desert Locust (Elife'2019)</a></summary>
```bibtex
@article{graving2019deepposekit,
title={DeepPoseKit, a software toolkit for fast and robust animal pose estimation using deep learning},
author={Graving, Jacob M and Chae, Daniel and Naik, Hemal and Li, Liang and Koger, Benjamin and Costelloe, Blair R and Couzin, Iain D},
journal={Elife},
volume={8},
pages={e47994},
year={2019},
publisher={eLife Sciences Publications Limited}
}
```
</details>
For [Desert Locust](https://github.com/jgraving/DeepPoseKit-Data) dataset, images can be downloaded from [locust_images](https://download.openmmlab.com/mmpose/datasets/locust_images.tar).
Please download the annotation files from [locust_annotations](https://download.openmmlab.com/mmpose/datasets/locust_annotations.tar).
Extract them under {MMPose}/data, and make them look like this:
```text
mmpose
├── mmpose
├── docs
├── tests
├── tools
├── configs
`── data
│── locust
│-- annotations
│ │-- locust_train.json
│ |-- locust_test.json
│-- images
│ │-- 0.jpg
│ │-- 1.jpg
│ │-- 2.jpg
│ │-- 3.jpg
│ │-- ...
```
Since the official dataset does not provide the test set, we randomly select 90\% images for training, and the rest (10\%) for evaluation (see [code](/tools/dataset/parse_deepposekit_dataset.py)).
## Grévy’s Zebra
<!-- [DATASET] -->
<details>
<summary align="right"><a href="https://elifesciences.org/articles/47994">Grévy’s Zebra (Elife'2019)</a></summary>
```bibtex
@article{graving2019deepposekit,
title={DeepPoseKit, a software toolkit for fast and robust animal pose estimation using deep learning},
author={Graving, Jacob M and Chae, Daniel and Naik, Hemal and Li, Liang and Koger, Benjamin and Costelloe, Blair R and Couzin, Iain D},
journal={Elife},
volume={8},
pages={e47994},
year={2019},
publisher={eLife Sciences Publications Limited}
}
```
</details>
For [Grévy’s Zebra](https://github.com/jgraving/DeepPoseKit-Data) dataset, images can be downloaded from [zebra_images](https://download.openmmlab.com/mmpose/datasets/zebra_images.tar).
Please download the annotation files from [zebra_annotations](https://download.openmmlab.com/mmpose/datasets/zebra_annotations.tar).
Extract them under {MMPose}/data, and make them look like this:
```text
mmpose
├── mmpose
├── docs
├── tests
├── tools
├── configs
`── data
│── zebra
│-- annotations
│ │-- zebra_train.json
│ |-- zebra_test.json
│-- images
│ │-- 0.jpg
│ │-- 1.jpg
│ │-- 2.jpg
│ │-- 3.jpg
│ │-- ...
```
Since the official dataset does not provide the test set, we randomly select 90\% images for training, and the rest (10\%) for evaluation (see [code](/tools/dataset/parse_deepposekit_dataset.py)).
## ATRW
<!-- [DATASET] -->
<details>
<summary align="right"><a href="https://arxiv.org/abs/1906.05586">ATRW (ACM MM'2020)</a></summary>
```bibtex
@inproceedings{li2020atrw,
title={ATRW: A Benchmark for Amur Tiger Re-identification in the Wild},
author={Li, Shuyuan and Li, Jianguo and Tang, Hanlin and Qian, Rui and Lin, Weiyao},
booktitle={Proceedings of the 28th ACM International Conference on Multimedia},
pages={2590--2598},
year={2020}
}
```
</details>
ATRW captures images of the Amur tiger (also known as Siberian tiger, Northeast-China tiger) in the wild.
For [ATRW](https://cvwc2019.github.io/challenge.html) dataset, please download images from
[Pose_train](https://lilablobssc.blob.core.windows.net/cvwc2019/train/atrw_pose_train.tar.gz),
[Pose_val](https://lilablobssc.blob.core.windows.net/cvwc2019/train/atrw_pose_val.tar.gz), and
[Pose_test](https://lilablobssc.blob.core.windows.net/cvwc2019/test/atrw_pose_test.tar.gz).
Note that in the ATRW official annotation files, the key "file_name" is written as "filename". To make it compatible with
other coco-type json files, we have modified this key.
Please download the modified annotation files from [atrw_annotations](https://download.openmmlab.com/mmpose/datasets/atrw_annotations.tar).
Extract them under {MMPose}/data, and make them look like this:
```text
mmpose
├── mmpose
├── docs
├── tests
├── tools
├── configs
`── data
│── atrw
│-- annotations
│ │-- keypoint_train.json
│ │-- keypoint_val.json
│ │-- keypoint_trainval.json
│-- images
│ │-- train
│ │ │-- 000002.jpg
│ │ │-- 000003.jpg
│ │ │-- ...
│ │-- val
│ │ │-- 000001.jpg
│ │ │-- 000013.jpg
│ │ │-- ...
│ │-- test
│ │ │-- 000000.jpg
│ │ │-- 000004.jpg
│ │ │-- ...
```
third_party/hamer/third-party/ViTPose/docs/en/tasks/2d_body_keypoint.md
Vendored
+500
View File
@@ -0,0 +1,500 @@
# 2D Body Keypoint Datasets
It is recommended to symlink the dataset root to `$MMPOSE/data`.
If your folder structure is different, you may need to change the corresponding paths in config files.
MMPose supported datasets:
- Images
- [COCO](#coco) \[ [Homepage](http://cocodataset.org/) \]
- [MPII](#mpii) \[ [Homepage](http://human-pose.mpi-inf.mpg.de/) \]
- [MPII-TRB](#mpii-trb) \[ [Homepage](https://github.com/kennymckormick/Triplet-Representation-of-human-Body) \]
- [AI Challenger](#aic) \[ [Homepage](https://github.com/AIChallenger/AI_Challenger_2017) \]
- [CrowdPose](#crowdpose) \[ [Homepage](https://github.com/Jeff-sjtu/CrowdPose) \]
- [OCHuman](#ochuman) \[ [Homepage](https://github.com/liruilong940607/OCHumanApi) \]
- [MHP](#mhp) \[ [Homepage](https://lv-mhp.github.io/dataset) \]
- Videos
- [PoseTrack18](#posetrack18) \[ [Homepage](https://posetrack.net/users/download.php) \]
- [sub-JHMDB](#sub-jhmdb-dataset) \[ [Homepage](http://jhmdb.is.tue.mpg.de/dataset) \]
## COCO
<!-- [DATASET] -->
<details>
<summary align="right"><a href="https://link.springer.com/chapter/10.1007/978-3-319-10602-1_48">COCO (ECCV'2014)</a></summary>
```bibtex
@inproceedings{lin2014microsoft,
title={Microsoft coco: Common objects in context},
author={Lin, Tsung-Yi and Maire, Michael and Belongie, Serge and Hays, James and Perona, Pietro and Ramanan, Deva and Doll{\'a}r, Piotr and Zitnick, C Lawrence},
booktitle={European conference on computer vision},
pages={740--755},
year={2014},
organization={Springer}
}
```
</details>
For [COCO](http://cocodataset.org/) data, please download from [COCO download](http://cocodataset.org/#download), 2017 Train/Val is needed for COCO keypoints training and validation.
[HRNet-Human-Pose-Estimation](https://github.com/HRNet/HRNet-Human-Pose-Estimation) provides person detection result of COCO val2017 to reproduce our multi-person pose estimation results.
Please download from [OneDrive](https://1drv.ms/f/s!AhIXJn_J-blWzzDXoz5BeFl8sWM-) or [GoogleDrive](https://drive.google.com/drive/folders/1fRUDNUDxe9fjqcRZ2bnF_TKMlO0nB_dk?usp=sharing).
Optionally, to evaluate on COCO'2017 test-dev, please download the [image-info](https://download.openmmlab.com/mmpose/datasets/person_keypoints_test-dev-2017.json).
Download and extract them under $MMPOSE/data, and make them look like this:
```text
mmpose
├── mmpose
├── docs
├── tests
├── tools
├── configs
`── data
│── coco
│-- annotations
│ │-- person_keypoints_train2017.json
│ |-- person_keypoints_val2017.json
│ |-- person_keypoints_test-dev-2017.json
|-- person_detection_results
| |-- COCO_val2017_detections_AP_H_56_person.json
| |-- COCO_test-dev2017_detections_AP_H_609_person.json
│-- train2017
│ │-- 000000000009.jpg
│ │-- 000000000025.jpg
│ │-- 000000000030.jpg
│ │-- ...
`-- val2017
│-- 000000000139.jpg
│-- 000000000285.jpg
│-- 000000000632.jpg
│-- ...
```
## MPII
<!-- [DATASET] -->
<details>
<summary align="right"><a href="http://openaccess.thecvf.com/content_cvpr_2014/html/Andriluka_2D_Human_Pose_2014_CVPR_paper.html">MPII (CVPR'2014)</a></summary>
```bibtex
@inproceedings{andriluka14cvpr,
author = {Mykhaylo Andriluka and Leonid Pishchulin and Peter Gehler and Schiele, Bernt},
title = {2D Human Pose Estimation: New Benchmark and State of the Art Analysis},
booktitle = {IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
year = {2014},
month = {June}
}
```
</details>
For [MPII](http://human-pose.mpi-inf.mpg.de/) data, please download from [MPII Human Pose Dataset](http://human-pose.mpi-inf.mpg.de/).
We have converted the original annotation files into json format, please download them from [mpii_annotations](https://download.openmmlab.com/mmpose/datasets/mpii_annotations.tar).
Extract them under {MMPose}/data, and make them look like this:
```text
mmpose
├── mmpose
├── docs
├── tests
├── tools
├── configs
`── data
│── mpii
|── annotations
| |── mpii_gt_val.mat
| |── mpii_test.json
| |── mpii_train.json
| |── mpii_trainval.json
| `── mpii_val.json
`── images
|── 000001163.jpg
|── 000003072.jpg
```
During training and inference, the prediction result will be saved as '.mat' format by default. We also provide a tool to convert this '.mat' to more readable '.json' format.
```shell
python tools/dataset/mat2json ${PRED_MAT_FILE} ${GT_JSON_FILE} ${OUTPUT_PRED_JSON_FILE}
```
For example,
```shell
python tools/dataset/mat2json work_dirs/res50_mpii_256x256/pred.mat data/mpii/annotations/mpii_val.json pred.json
```
## MPII-TRB
<!-- [DATASET] -->
<details>
<summary align="right"><a href="http://openaccess.thecvf.com/content_ICCV_2019/html/Duan_TRB_A_Novel_Triplet_Representation_for_Understanding_2D_Human_Body_ICCV_2019_paper.html">MPII-TRB (ICCV'2019)</a></summary>
```bibtex
@inproceedings{duan2019trb,
title={TRB: A Novel Triplet Representation for Understanding 2D Human Body},
author={Duan, Haodong and Lin, Kwan-Yee and Jin, Sheng and Liu, Wentao and Qian, Chen and Ouyang, Wanli},
booktitle={Proceedings of the IEEE International Conference on Computer Vision},
pages={9479--9488},
year={2019}
}
```
</details>
For [MPII-TRB](https://github.com/kennymckormick/Triplet-Representation-of-human-Body) data, please download from [MPII Human Pose Dataset](http://human-pose.mpi-inf.mpg.de/).
Please download the annotation files from [mpii_trb_annotations](https://download.openmmlab.com/mmpose/datasets/mpii_trb_annotations.tar).
Extract them under {MMPose}/data, and make them look like this:
```text
mmpose
├── mmpose
├── docs
├── tests
├── tools
├── configs
`── data
│── mpii
|── annotations
| |── mpii_trb_train.json
| |── mpii_trb_val.json
`── images
|── 000001163.jpg
|── 000003072.jpg
```
## AIC
<!-- [DATASET] -->
<details>
<summary align="right"><a href="https://arxiv.org/abs/1711.06475">AI Challenger (ArXiv'2017)</a></summary>
```bibtex
@article{wu2017ai,
title={Ai challenger: A large-scale dataset for going deeper in image understanding},
author={Wu, Jiahong and Zheng, He and Zhao, Bo and Li, Yixin and Yan, Baoming and Liang, Rui and Wang, Wenjia and Zhou, Shipei and Lin, Guosen and Fu, Yanwei and others},
journal={arXiv preprint arXiv:1711.06475},
year={2017}
}
```
</details>
For [AIC](https://github.com/AIChallenger/AI_Challenger_2017) data, please download from [AI Challenger 2017](https://github.com/AIChallenger/AI_Challenger_2017), 2017 Train/Val is needed for keypoints training and validation.
Please download the annotation files from [aic_annotations](https://download.openmmlab.com/mmpose/datasets/aic_annotations.tar).
Download and extract them under $MMPOSE/data, and make them look like this:
```text
mmpose
├── mmpose
├── docs
├── tests
├── tools
├── configs
`── data
│── aic
│-- annotations
│ │-- aic_train.json
│ |-- aic_val.json
│-- ai_challenger_keypoint_train_20170902
│ │-- keypoint_train_images_20170902
│ │ │-- 0000252aea98840a550dac9a78c476ecb9f47ffa.jpg
│ │ │-- 000050f770985ac9653198495ef9b5c82435d49c.jpg
│ │ │-- ...
`-- ai_challenger_keypoint_validation_20170911
│-- keypoint_validation_images_20170911
│-- 0002605c53fb92109a3f2de4fc3ce06425c3b61f.jpg
│-- 0003b55a2c991223e6d8b4b820045bd49507bf6d.jpg
│-- ...
```
## CrowdPose
<!-- [DATASET] -->
<details>
<summary align="right"><a href="http://openaccess.thecvf.com/content_CVPR_2019/html/Li_CrowdPose_Efficient_Crowded_Scenes_Pose_Estimation_and_a_New_Benchmark_CVPR_2019_paper.html">CrowdPose (CVPR'2019)</a></summary>
```bibtex
@article{li2018crowdpose,
title={CrowdPose: Efficient Crowded Scenes Pose Estimation and A New Benchmark},
author={Li, Jiefeng and Wang, Can and Zhu, Hao and Mao, Yihuan and Fang, Hao-Shu and Lu, Cewu},
journal={arXiv preprint arXiv:1812.00324},
year={2018}
}
```
</details>
For [CrowdPose](https://github.com/Jeff-sjtu/CrowdPose) data, please download from [CrowdPose](https://github.com/Jeff-sjtu/CrowdPose).
Please download the annotation files and human detection results from [crowdpose_annotations](https://download.openmmlab.com/mmpose/datasets/crowdpose_annotations.tar).
For top-down approaches, we follow [CrowdPose](https://arxiv.org/abs/1812.00324) to use the [pre-trained weights](https://pjreddie.com/media/files/yolov3.weights) of [YOLOv3](https://github.com/eriklindernoren/PyTorch-YOLOv3) to generate the detected human bounding boxes.
For model training, we follow [HigherHRNet](https://github.com/HRNet/HigherHRNet-Human-Pose-Estimation) to train models on CrowdPose train/val dataset, and evaluate models on CrowdPose test dataset.
Download and extract them under $MMPOSE/data, and make them look like this:
```text
mmpose
├── mmpose
├── docs
├── tests
├── tools
├── configs
`── data
│── crowdpose
│-- annotations
│ │-- mmpose_crowdpose_train.json
│ │-- mmpose_crowdpose_val.json
│ │-- mmpose_crowdpose_trainval.json
│ │-- mmpose_crowdpose_test.json
│ │-- det_for_crowd_test_0.1_0.5.json
│-- images
│-- 100000.jpg
│-- 100001.jpg
│-- 100002.jpg
│-- ...
```
## OCHuman
<!-- [DATASET] -->
<details>
<summary align="right"><a href="http://openaccess.thecvf.com/content_CVPR_2019/html/Zhang_Pose2Seg_Detection_Free_Human_Instance_Segmentation_CVPR_2019_paper.html">OCHuman (CVPR'2019)</a></summary>
```bibtex
@inproceedings{zhang2019pose2seg,
title={Pose2seg: Detection free human instance segmentation},
author={Zhang, Song-Hai and Li, Ruilong and Dong, Xin and Rosin, Paul and Cai, Zixi and Han, Xi and Yang, Dingcheng and Huang, Haozhi and Hu, Shi-Min},
booktitle={Proceedings of the IEEE conference on computer vision and pattern recognition},
pages={889--898},
year={2019}
}
```
</details>
For [OCHuman](https://github.com/liruilong940607/OCHumanApi) data, please download the images and annotations from [OCHuman](https://github.com/liruilong940607/OCHumanApi),
Move them under $MMPOSE/data, and make them look like this:
```text
mmpose
├── mmpose
├── docs
├── tests
├── tools
├── configs
`── data
│── ochuman
│-- annotations
│ │-- ochuman_coco_format_val_range_0.00_1.00.json
│ |-- ochuman_coco_format_test_range_0.00_1.00.json
|-- images
│-- 000001.jpg
│-- 000002.jpg
│-- 000003.jpg
│-- ...
```
## MHP
<!-- [DATASET] -->
<details>
<summary align="right"><a href="https://dl.acm.org/doi/abs/10.1145/3240508.3240509">MHP (ACM MM'2018)</a></summary>
```bibtex
@inproceedings{zhao2018understanding,
title={Understanding humans in crowded scenes: Deep nested adversarial learning and a new benchmark for multi-human parsing},
author={Zhao, Jian and Li, Jianshu and Cheng, Yu and Sim, Terence and Yan, Shuicheng and Feng, Jiashi},
booktitle={Proceedings of the 26th ACM international conference on Multimedia},
pages={792--800},
year={2018}
}
```
</details>
For [MHP](https://lv-mhp.github.io/dataset) data, please download from [MHP](https://lv-mhp.github.io/dataset).
Please download the annotation files from [mhp_annotations](https://download.openmmlab.com/mmpose/datasets/mhp_annotations.tar.gz).
Please download and extract them under $MMPOSE/data, and make them look like this:
```text
mmpose
├── mmpose
├── docs
├── tests
├── tools
├── configs
`── data
│── mhp
│-- annotations
│ │-- mhp_train.json
│ │-- mhp_val.json
│
`-- train
│ │-- images
│ │ │-- 1004.jpg
│ │ │-- 10050.jpg
│ │ │-- ...
│
`-- val
│ │-- images
│ │ │-- 10059.jpg
│ │ │-- 10068.jpg
│ │ │-- ...
│
`-- test
│ │-- images
│ │ │-- 1005.jpg
│ │ │-- 10052.jpg
│ │ │-- ...~~~~
```
## PoseTrack18
<!-- [DATASET] -->
<details>
<summary align="right"><a href="http://openaccess.thecvf.com/content_cvpr_2018/html/Andriluka_PoseTrack_A_Benchmark_CVPR_2018_paper.html">PoseTrack18 (CVPR'2018)</a></summary>
```bibtex
@inproceedings{andriluka2018posetrack,
title={Posetrack: A benchmark for human pose estimation and tracking},
author={Andriluka, Mykhaylo and Iqbal, Umar and Insafutdinov, Eldar and Pishchulin, Leonid and Milan, Anton and Gall, Juergen and Schiele, Bernt},
booktitle={Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition},
pages={5167--5176},
year={2018}
}
```
</details>
For [PoseTrack18](https://posetrack.net/users/download.php) data, please download from [PoseTrack18](https://posetrack.net/users/download.php).
Please download the annotation files from [posetrack18_annotations](https://download.openmmlab.com/mmpose/datasets/posetrack18_annotations.tar).
We have merged the video-wise separated official annotation files into two json files (posetrack18_train & posetrack18_val.json). We also generate the [mask files](https://download.openmmlab.com/mmpose/datasets/posetrack18_mask.tar) to speed up training.
For top-down approaches, we use [MMDetection](https://github.com/open-mmlab/mmdetection) pre-trained [Cascade R-CNN](https://download.openmmlab.com/mmdetection/v2.0/cascade_rcnn/cascade_rcnn_x101_64x4d_fpn_20e_coco/cascade_rcnn_x101_64x4d_fpn_20e_coco_20200509_224357-051557b1.pth) (X-101-64x4d-FPN) to generate the detected human bounding boxes.
Please download and extract them under $MMPOSE/data, and make them look like this:
```text
mmpose
├── mmpose
├── docs
├── tests
├── tools
├── configs
`── data
│── posetrack18
│-- annotations
│ │-- posetrack18_train.json
│ │-- posetrack18_val.json
│ │-- posetrack18_val_human_detections.json
│ │-- train
│ │ │-- 000001_bonn_train.json
│ │ │-- 000002_bonn_train.json
│ │ │-- ...
│ │-- val
│ │ │-- 000342_mpii_test.json
│ │ │-- 000522_mpii_test.json
│ │ │-- ...
│ `-- test
│ │-- 000001_mpiinew_test.json
│ │-- 000002_mpiinew_test.json
│ │-- ...
│
`-- images
│ │-- train
│ │ │-- 000001_bonn_train
│ │ │ │-- 000000.jpg
│ │ │ │-- 000001.jpg
│ │ │ │-- ...
│ │ │-- ...
│ │-- val
│ │ │-- 000342_mpii_test
│ │ │ │-- 000000.jpg
│ │ │ │-- 000001.jpg
│ │ │ │-- ...
│ │ │-- ...
│ `-- test
│ │-- 000001_mpiinew_test
│ │ │-- 000000.jpg
│ │ │-- 000001.jpg
│ │ │-- ...
│ │-- ...
`-- mask
│-- train
│ │-- 000002_bonn_train
│ │ │-- 000000.jpg
│ │ │-- 000001.jpg
│ │ │-- ...
│ │-- ...
`-- val
│-- 000522_mpii_test
│ │-- 000000.jpg
│ │-- 000001.jpg
│ │-- ...
│-- ...
```
The official evaluation tool for PoseTrack should be installed from GitHub.
```shell
pip install git+https://github.com/svenkreiss/poseval.git
```
## sub-JHMDB dataset
<!-- [DATASET] -->
<details>
<summary align="right"><a href="https://link.springer.com/chapter/10.1007/978-3-030-58580-8_27">RSN (ECCV'2020)</a></summary>
```bibtex
@misc{cai2020learning,
title={Learning Delicate Local Representations for Multi-Person Pose Estimation},
author={Yuanhao Cai and Zhicheng Wang and Zhengxiong Luo and Binyi Yin and Angang Du and Haoqian Wang and Xinyu Zhou and Erjin Zhou and Xiangyu Zhang and Jian Sun},
year={2020},
eprint={2003.04030},
archivePrefix={arXiv},
primaryClass={cs.CV}
}
```
</details>
For [sub-JHMDB](http://jhmdb.is.tue.mpg.de/dataset) data, please download the [images](<(http://files.is.tue.mpg.de/jhmdb/Rename_Images.tar.gz)>) from [JHMDB](http://jhmdb.is.tue.mpg.de/dataset),
Please download the annotation files from [jhmdb_annotations](https://download.openmmlab.com/mmpose/datasets/jhmdb_annotations.tar).
Move them under $MMPOSE/data, and make them look like this:
```text
mmpose
├── mmpose
├── docs
├── tests
├── tools
├── configs
`── data
│── jhmdb
│-- annotations
│ │-- Sub1_train.json
│ |-- Sub1_test.json
│ │-- Sub2_train.json
│ |-- Sub2_test.json
│ │-- Sub3_train.json
│ |-- Sub3_test.json
|-- Rename_Images
│-- brush_hair
│ │--April_09_brush_hair_u_nm_np1_ba_goo_0
| │ │--00001.png
| │ │--00002.png
│-- catch
│-- ...
```
third_party/hamer/third-party/ViTPose/docs/en/tasks/2d_face_keypoint.md
Vendored
+306
View File
@@ -0,0 +1,306 @@
# 2D Face Keypoint Datasets
It is recommended to symlink the dataset root to `$MMPOSE/data`.
If your folder structure is different, you may need to change the corresponding paths in config files.
MMPose supported datasets:
- [300W](#300w-dataset) \[ [Homepage](https://ibug.doc.ic.ac.uk/resources/300-W/) \]
- [WFLW](#wflw-dataset) \[ [Homepage](https://wywu.github.io/projects/LAB/WFLW.html) \]
- [AFLW](#aflw-dataset) \[ [Homepage](https://www.tugraz.at/institute/icg/research/team-bischof/lrs/downloads/aflw/) \]
- [COFW](#cofw-dataset) \[ [Homepage](http://www.vision.caltech.edu/xpburgos/ICCV13/) \]
- [COCO-WholeBody-Face](#coco-wholebody-face) \[ [Homepage](https://github.com/jin-s13/COCO-WholeBody/) \]
## 300W Dataset
<!-- [DATASET] -->
<details>
<summary align="right"><a href="https://www.sciencedirect.com/science/article/pii/S0262885616000147">300W (IMAVIS'2016)</a></summary>
```bibtex
@article{sagonas2016300,
title={300 faces in-the-wild challenge: Database and results},
author={Sagonas, Christos and Antonakos, Epameinondas and Tzimiropoulos, Georgios and Zafeiriou, Stefanos and Pantic, Maja},
journal={Image and vision computing},
volume={47},
pages={3--18},
year={2016},
publisher={Elsevier}
}
```
</details>
For 300W data, please download images from [300W Dataset](https://ibug.doc.ic.ac.uk/resources/300-W/).
Please download the annotation files from [300w_annotations](https://download.openmmlab.com/mmpose/datasets/300w_annotations.tar).
Extract them under {MMPose}/data, and make them look like this:
```text
mmpose
├── mmpose
├── docs
├── tests
├── tools
├── configs
`── data
│── 300w
|── annotations
| |── face_landmarks_300w_train.json
| |── face_landmarks_300w_valid.json
| |── face_landmarks_300w_valid_common.json
| |── face_landmarks_300w_valid_challenge.json
| |── face_landmarks_300w_test.json
`── images
|── afw
| |── 1051618982_1.jpg
| |── 111076519_1.jpg
| ...
|── helen
| |── trainset
| | |── 100032540_1.jpg
| | |── 100040721_1.jpg
| | ...
| |── testset
| | |── 296814969_3.jpg
| | |── 2968560214_1.jpg
| | ...
|── ibug
| |── image_003_1.jpg
| |── image_004_1.jpg
| ...
|── lfpw
| |── trainset
| | |── image_0001.png
| | |── image_0002.png
| | ...
| |── testset
| | |── image_0001.png
| | |── image_0002.png
| | ...
`── Test
|── 01_Indoor
| |── indoor_001.png
| |── indoor_002.png
| ...
`── 02_Outdoor
|── outdoor_001.png
|── outdoor_002.png
...
```
## WFLW Dataset
<!-- [DATASET] -->
<details>
<summary align="right"><a href="http://openaccess.thecvf.com/content_cvpr_2018/html/Wu_Look_at_Boundary_CVPR_2018_paper.html">WFLW (CVPR'2018)</a></summary>
```bibtex
@inproceedings{wu2018look,
title={Look at boundary: A boundary-aware face alignment algorithm},
author={Wu, Wayne and Qian, Chen and Yang, Shuo and Wang, Quan and Cai, Yici and Zhou, Qiang},
booktitle={Proceedings of the IEEE conference on computer vision and pattern recognition},
pages={2129--2138},
year={2018}
}
```
</details>
For WFLW data, please download images from [WFLW Dataset](https://wywu.github.io/projects/LAB/WFLW.html).
Please download the annotation files from [wflw_annotations](https://download.openmmlab.com/mmpose/datasets/wflw_annotations.tar).
Extract them under {MMPose}/data, and make them look like this:
```text
mmpose
├── mmpose
├── docs
├── tests
├── tools
├── configs
`── data
│── wflw
|── annotations
| |── face_landmarks_wflw_train.json
| |── face_landmarks_wflw_test.json
| |── face_landmarks_wflw_test_blur.json
| |── face_landmarks_wflw_test_occlusion.json
| |── face_landmarks_wflw_test_expression.json
| |── face_landmarks_wflw_test_largepose.json
| |── face_landmarks_wflw_test_illumination.json
| |── face_landmarks_wflw_test_makeup.json
|
`── images
|── 0--Parade
| |── 0_Parade_marchingband_1_1015.jpg
| |── 0_Parade_marchingband_1_1031.jpg
| ...
|── 1--Handshaking
| |── 1_Handshaking_Handshaking_1_105.jpg
| |── 1_Handshaking_Handshaking_1_107.jpg
| ...
...
```
## AFLW Dataset
<!-- [DATASET] -->
<details>
<summary align="right"><a href="https://ieeexplore.ieee.org/abstract/document/6130513/">AFLW (ICCVW'2011)</a></summary>
```bibtex
@inproceedings{koestinger2011annotated,
title={Annotated facial landmarks in the wild: A large-scale, real-world database for facial landmark localization},
author={Koestinger, Martin and Wohlhart, Paul and Roth, Peter M and Bischof, Horst},
booktitle={2011 IEEE international conference on computer vision workshops (ICCV workshops)},
pages={2144--2151},
year={2011},
organization={IEEE}
}
```
</details>
For AFLW data, please download images from [AFLW Dataset](https://www.tugraz.at/institute/icg/research/team-bischof/lrs/downloads/aflw/).
Please download the annotation files from [aflw_annotations](https://download.openmmlab.com/mmpose/datasets/aflw_annotations.tar).
Extract them under {MMPose}/data, and make them look like this:
```text
mmpose
├── mmpose
├── docs
├── tests
├── tools
├── configs
`── data
│── aflw
|── annotations
| |── face_landmarks_aflw_train.json
| |── face_landmarks_aflw_test_frontal.json
| |── face_landmarks_aflw_test.json
`── images
|── flickr
|── 0
| |── image00002.jpg
| |── image00013.jpg
| ...
|── 2
| |── image00004.jpg
| |── image00006.jpg
| ...
`── 3
|── image00032.jpg
|── image00035.jpg
...
```
## COFW Dataset
<!-- [DATASET] -->
<details>
<summary align="right"><a href="http://openaccess.thecvf.com/content_iccv_2013/html/Burgos-Artizzu_Robust_Face_Landmark_2013_ICCV_paper.html">COFW (ICCV'2013)</a></summary>
```bibtex
@inproceedings{burgos2013robust,
title={Robust face landmark estimation under occlusion},
author={Burgos-Artizzu, Xavier P and Perona, Pietro and Doll{\'a}r, Piotr},
booktitle={Proceedings of the IEEE international conference on computer vision},
pages={1513--1520},
year={2013}
}
```
</details>
For COFW data, please download from [COFW Dataset (Color Images)](http://www.vision.caltech.edu/xpburgos/ICCV13/Data/COFW_color.zip).
Move `COFW_train_color.mat` and `COFW_test_color.mat` to `data/cofw/` and make them look like:
```text
mmpose
├── mmpose
├── docs
├── tests
├── tools
├── configs
`── data
│── cofw
|── COFW_train_color.mat
|── COFW_test_color.mat
```
Run the following script under `{MMPose}/data`
`python tools/dataset/parse_cofw_dataset.py`
And you will get
```text
mmpose
├── mmpose
├── docs
├── tests
├── tools
├── configs
`── data
│── cofw
|── COFW_train_color.mat
|── COFW_test_color.mat
|── annotations
| |── cofw_train.json
| |── cofw_test.json
|── images
|── 000001.jpg
|── 000002.jpg
```
## COCO-WholeBody (Face)
[DATASET]
```bibtex
@inproceedings{jin2020whole,
title={Whole-Body Human Pose Estimation in the Wild},
author={Jin, Sheng and Xu, Lumin and Xu, Jin and Wang, Can and Liu, Wentao and Qian, Chen and Ouyang, Wanli and Luo, Ping},
booktitle={Proceedings of the European Conference on Computer Vision (ECCV)},
year={2020}
}
```
For [COCO-WholeBody](https://github.com/jin-s13/COCO-WholeBody/) dataset, images can be downloaded from [COCO download](http://cocodataset.org/#download), 2017 Train/Val is needed for COCO keypoints training and validation.
Download COCO-WholeBody annotations for COCO-WholeBody annotations for [Train](https://drive.google.com/file/d/1thErEToRbmM9uLNi1JXXfOsaS5VK2FXf/view?usp=sharing) / [Validation](https://drive.google.com/file/d/1N6VgwKnj8DeyGXCvp1eYgNbRmw6jdfrb/view?usp=sharing) (Google Drive).
Download person detection result of COCO val2017 from [OneDrive](https://1drv.ms/f/s!AhIXJn_J-blWzzDXoz5BeFl8sWM-) or [GoogleDrive](https://drive.google.com/drive/folders/1fRUDNUDxe9fjqcRZ2bnF_TKMlO0nB_dk?usp=sharing).
Download and extract them under $MMPOSE/data, and make them look like this:
```text
mmpose
├── mmpose
├── docs
├── tests
├── tools
├── configs
`── data
│── coco
│-- annotations
│ │-- coco_wholebody_train_v1.0.json
│ |-- coco_wholebody_val_v1.0.json
|-- person_detection_results
| |-- COCO_val2017_detections_AP_H_56_person.json
│-- train2017
│ │-- 000000000009.jpg
│ │-- 000000000025.jpg
│ │-- 000000000030.jpg
│ │-- ...
`-- val2017
│-- 000000000139.jpg
│-- 000000000285.jpg
│-- 000000000632.jpg
│-- ...
```
Please also install the latest version of [Extended COCO API](https://github.com/jin-s13/xtcocoapi) to support COCO-WholeBody evaluation:
`pip install xtcocotools`
third_party/hamer/third-party/ViTPose/docs/en/tasks/2d_fashion_landmark.md
Vendored
+76
View File
@@ -0,0 +1,76 @@
# 2D Fashion Landmark Dataset
It is recommended to symlink the dataset root to `$MMPOSE/data`.
If your folder structure is different, you may need to change the corresponding paths in config files.
MMPose supported datasets:
- [DeepFashion](#deepfashion) \[ [Homepage](http://mmlab.ie.cuhk.edu.hk/projects/DeepFashion/LandmarkDetection.html) \]
## DeepFashion (Fashion Landmark Detection, FLD)
<!-- [DATASET] -->
<details>
<summary align="right"><a href="http://openaccess.thecvf.com/content_cvpr_2016/html/Liu_DeepFashion_Powering_Robust_CVPR_2016_paper.html">DeepFashion (CVPR'2016)</a></summary>
```bibtex
@inproceedings{liuLQWTcvpr16DeepFashion,
author = {Liu, Ziwei and Luo, Ping and Qiu, Shi and Wang, Xiaogang and Tang, Xiaoou},
title = {DeepFashion: Powering Robust Clothes Recognition and Retrieval with Rich Annotations},
booktitle = {Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
month = {June},
year = {2016}
}
```
</details>
<!-- [DATASET] -->
<details>
<summary align="right"><a href="https://link.springer.com/chapter/10.1007/978-3-319-46475-6_15">DeepFashion (ECCV'2016)</a></summary>
```bibtex
@inproceedings{liuYLWTeccv16FashionLandmark,
author = {Liu, Ziwei and Yan, Sijie and Luo, Ping and Wang, Xiaogang and Tang, Xiaoou},
title = {Fashion Landmark Detection in the Wild},
booktitle = {European Conference on Computer Vision (ECCV)},
month = {October},
year = {2016}
}
```
</details>
For [DeepFashion](http://mmlab.ie.cuhk.edu.hk/projects/DeepFashion/LandmarkDetection.html) dataset, images can be downloaded from [download](http://mmlab.ie.cuhk.edu.hk/projects/DeepFashion/LandmarkDetection.html).
Please download the annotation files from [fld_annotations](https://download.openmmlab.com/mmpose/datasets/fld_annotations.tar).
Extract them under {MMPose}/data, and make them look like this:
```text
mmpose
├── mmpose
├── docs
├── tests
├── tools
├── configs
`── data
│── fld
│-- annotations
│ │-- fld_upper_train.json
│ |-- fld_upper_val.json
│ |-- fld_upper_test.json
│ │-- fld_lower_train.json
│ |-- fld_lower_val.json
│ |-- fld_lower_test.json
│ │-- fld_full_train.json
│ |-- fld_full_val.json
│ |-- fld_full_test.json
│-- img
│ │-- img_00000001.jpg
│ │-- img_00000002.jpg
│ │-- img_00000003.jpg
│ │-- img_00000004.jpg
│ │-- img_00000005.jpg
│ │-- ...
```
third_party/hamer/third-party/ViTPose/docs/en/tasks/2d_hand_keypoint.md
Vendored
+319
View File
@@ -0,0 +1,319 @@
# 2D Hand Keypoint Datasets
It is recommended to symlink the dataset root to `$MMPOSE/data`.
If your folder structure is different, you may need to change the corresponding paths in config files.
MMPose supported datasets:
- [OneHand10K](#onehand10k) \[ [Homepage](https://www.yangangwang.com/papers/WANG-MCC-2018-10.html) \]
- [FreiHand](#freihand-dataset) \[ [Homepage](https://lmb.informatik.uni-freiburg.de/projects/freihand/) \]
- [CMU Panoptic HandDB](#cmu-panoptic-handdb) \[ [Homepage](http://domedb.perception.cs.cmu.edu/handdb.html) \]
- [InterHand2.6M](#interhand26m) \[ [Homepage](https://mks0601.github.io/InterHand2.6M/) \]
- [RHD](#rhd-dataset) \[ [Homepage](https://lmb.informatik.uni-freiburg.de/resources/datasets/RenderedHandposeDataset.en.html) \]
- [COCO-WholeBody-Hand](#coco-wholebody-hand) \[ [Homepage](https://github.com/jin-s13/COCO-WholeBody/) \]
## OneHand10K
<!-- [DATASET] -->
<details>
<summary align="right"><a href="https://ieeexplore.ieee.org/abstract/document/8529221/">OneHand10K (TCSVT'2019)</a></summary>
```bibtex
@article{wang2018mask,
title={Mask-pose cascaded cnn for 2d hand pose estimation from single color image},
author={Wang, Yangang and Peng, Cong and Liu, Yebin},
journal={IEEE Transactions on Circuits and Systems for Video Technology},
volume={29},
number={11},
pages={3258--3268},
year={2018},
publisher={IEEE}
}
```
</details>
For [OneHand10K](https://www.yangangwang.com/papers/WANG-MCC-2018-10.html) data, please download from [OneHand10K Dataset](https://www.yangangwang.com/papers/WANG-MCC-2018-10.html).
Please download the annotation files from [onehand10k_annotations](https://download.openmmlab.com/mmpose/datasets/onehand10k_annotations.tar).
Extract them under {MMPose}/data, and make them look like this:
```text
mmpose
├── mmpose
├── docs
├── tests
├── tools
├── configs
`── data
│── onehand10k
|── annotations
| |── onehand10k_train.json
| |── onehand10k_test.json
`── Train
| |── source
| |── 0.jpg
| |── 1.jpg
| ...
`── Test
|── source
|── 0.jpg
|── 1.jpg
```
## FreiHAND Dataset
<!-- [DATASET] -->
<details>
<summary align="right"><a href="http://openaccess.thecvf.com/content_ICCV_2019/html/Zimmermann_FreiHAND_A_Dataset_for_Markerless_Capture_of_Hand_Pose_and_ICCV_2019_paper.html">FreiHand (ICCV'2019)</a></summary>
```bibtex
@inproceedings{zimmermann2019freihand,
title={Freihand: A dataset for markerless capture of hand pose and shape from single rgb images},
author={Zimmermann, Christian and Ceylan, Duygu and Yang, Jimei and Russell, Bryan and Argus, Max and Brox, Thomas},
booktitle={Proceedings of the IEEE International Conference on Computer Vision},
pages={813--822},
year={2019}
}
```
</details>
For [FreiHAND](https://lmb.informatik.uni-freiburg.de/projects/freihand/) data, please download from [FreiHand Dataset](https://lmb.informatik.uni-freiburg.de/resources/datasets/FreihandDataset.en.html).
Since the official dataset does not provide validation set, we randomly split the training data into 8:1:1 for train/val/test.
Please download the annotation files from [freihand_annotations](https://download.openmmlab.com/mmpose/datasets/frei_annotations.tar).
Extract them under {MMPose}/data, and make them look like this:
```text
mmpose
├── mmpose
├── docs
├── tests
├── tools
├── configs
`── data
│── freihand
|── annotations
| |── freihand_train.json
| |── freihand_val.json
| |── freihand_test.json
`── training
|── rgb
| |── 00000000.jpg
| |── 00000001.jpg
| ...
|── mask
|── 00000000.jpg
|── 00000001.jpg
...
```
## CMU Panoptic HandDB
<!-- [DATASET] -->
<details>
<summary align="right"><a href="http://openaccess.thecvf.com/content_cvpr_2017/html/Simon_Hand_Keypoint_Detection_CVPR_2017_paper.html">CMU Panoptic HandDB (CVPR'2017)</a></summary>
```bibtex
@inproceedings{simon2017hand,
title={Hand keypoint detection in single images using multiview bootstrapping},
author={Simon, Tomas and Joo, Hanbyul and Matthews, Iain and Sheikh, Yaser},
booktitle={Proceedings of the IEEE conference on Computer Vision and Pattern Recognition},
pages={1145--1153},
year={2017}
}
```
</details>
For [CMU Panoptic HandDB](http://domedb.perception.cs.cmu.edu/handdb.html), please download from [CMU Panoptic HandDB](http://domedb.perception.cs.cmu.edu/handdb.html).
Following [Simon et al](https://arxiv.org/abs/1704.07809), panoptic images (hand143_panopticdb) and MPII & NZSL training sets (manual_train) are used for training, while MPII & NZSL test set (manual_test) for testing.
Please download the annotation files from [panoptic_annotations](https://download.openmmlab.com/mmpose/datasets/panoptic_annotations.tar).
Extract them under {MMPose}/data, and make them look like this:
```text
mmpose
├── mmpose
├── docs
├── tests
├── tools
├── configs
`── data
│── panoptic
|── annotations
| |── panoptic_train.json
| |── panoptic_test.json
|
`── hand143_panopticdb
| |── imgs
| | |── 00000000.jpg
| | |── 00000001.jpg
| | ...
|
`── hand_labels
|── manual_train
| |── 000015774_01_l.jpg
| |── 000015774_01_r.jpg
| ...
|
`── manual_test
|── 000648952_02_l.jpg
|── 000835470_01_l.jpg
...
```
## InterHand2.6M
<!-- [DATASET] -->
<details>
<summary align="right"><a href="https://link.springer.com/content/pdf/10.1007/978-3-030-58565-5_33.pdf">InterHand2.6M (ECCV'2020)</a></summary>
```bibtex
@InProceedings{Moon_2020_ECCV_InterHand2.6M,
author = {Moon, Gyeongsik and Yu, Shoou-I and Wen, He and Shiratori, Takaaki and Lee, Kyoung Mu},
title = {InterHand2.6M: A Dataset and Baseline for 3D Interacting Hand Pose Estimation from a Single RGB Image},
booktitle = {European Conference on Computer Vision (ECCV)},
year = {2020}
}
```
</details>
For [InterHand2.6M](https://mks0601.github.io/InterHand2.6M/), please download from [InterHand2.6M](https://mks0601.github.io/InterHand2.6M/).
Please download the annotation files from [annotations](https://drive.google.com/drive/folders/1pWXhdfaka-J0fSAze0MsajN0VpZ8e8tO).
Extract them under {MMPose}/data, and make them look like this:
```text
mmpose
├── mmpose
├── docs
├── tests
├── tools
├── configs
`── data
│── interhand2.6m
|── annotations
| |── all
| |── human_annot
| |── machine_annot
| |── skeleton.txt
| |── subject.txt
|
`── images
| |── train
| | |-- Capture0 ~ Capture26
| |── val
| | |-- Capture0
| |── test
| | |-- Capture0 ~ Capture7
```
## RHD Dataset
<!-- [DATASET] -->
<details>
<summary align="right"><a href="https://lmb.informatik.uni-freiburg.de/projects/hand3d/">RHD (ICCV'2017)</a></summary>
```bibtex
@TechReport{zb2017hand,
author={Christian Zimmermann and Thomas Brox},
title={Learning to Estimate 3D Hand Pose from Single RGB Images},
institution={arXiv:1705.01389},
year={2017},
note="https://arxiv.org/abs/1705.01389",
url="https://lmb.informatik.uni-freiburg.de/projects/hand3d/"
}
```
</details>
For [RHD Dataset](https://lmb.informatik.uni-freiburg.de/resources/datasets/RenderedHandposeDataset.en.html), please download from [RHD Dataset](https://lmb.informatik.uni-freiburg.de/resources/datasets/RenderedHandposeDataset.en.html).
Please download the annotation files from [rhd_annotations](https://download.openmmlab.com/mmpose/datasets/rhd_annotations.zip).
Extract them under {MMPose}/data, and make them look like this:
```text
mmpose
├── mmpose
├── docs
├── tests
├── tools
├── configs
`── data
│── rhd
|── annotations
| |── rhd_train.json
| |── rhd_test.json
`── training
| |── color
| | |── 00000.jpg
| | |── 00001.jpg
| |── depth
| | |── 00000.jpg
| | |── 00001.jpg
| |── mask
| | |── 00000.jpg
| | |── 00001.jpg
`── evaluation
| |── color
| | |── 00000.jpg
| | |── 00001.jpg
| |── depth
| | |── 00000.jpg
| | |── 00001.jpg
| |── mask
| | |── 00000.jpg
| | |── 00001.jpg
```
## COCO-WholeBody (Hand)
[DATASET]
```bibtex
@inproceedings{jin2020whole,
title={Whole-Body Human Pose Estimation in the Wild},
author={Jin, Sheng and Xu, Lumin and Xu, Jin and Wang, Can and Liu, Wentao and Qian, Chen and Ouyang, Wanli and Luo, Ping},
booktitle={Proceedings of the European Conference on Computer Vision (ECCV)},
year={2020}
}
```
For [COCO-WholeBody](https://github.com/jin-s13/COCO-WholeBody/) dataset, images can be downloaded from [COCO download](http://cocodataset.org/#download), 2017 Train/Val is needed for COCO keypoints training and validation.
Download COCO-WholeBody annotations for COCO-WholeBody annotations for [Train](https://drive.google.com/file/d/1thErEToRbmM9uLNi1JXXfOsaS5VK2FXf/view?usp=sharing) / [Validation](https://drive.google.com/file/d/1N6VgwKnj8DeyGXCvp1eYgNbRmw6jdfrb/view?usp=sharing) (Google Drive).
Download person detection result of COCO val2017 from [OneDrive](https://1drv.ms/f/s!AhIXJn_J-blWzzDXoz5BeFl8sWM-) or [GoogleDrive](https://drive.google.com/drive/folders/1fRUDNUDxe9fjqcRZ2bnF_TKMlO0nB_dk?usp=sharing).
Download and extract them under $MMPOSE/data, and make them look like this:
```text
mmpose
├── mmpose
├── docs
├── tests
├── tools
├── configs
`── data
│── coco
│-- annotations
│ │-- coco_wholebody_train_v1.0.json
│ |-- coco_wholebody_val_v1.0.json
|-- person_detection_results
| |-- COCO_val2017_detections_AP_H_56_person.json
│-- train2017
│ │-- 000000000009.jpg
│ │-- 000000000025.jpg
│ │-- 000000000030.jpg
│ │-- ...
`-- val2017
│-- 000000000139.jpg
│-- 000000000285.jpg
│-- 000000000632.jpg
│-- ...
```
Please also install the latest version of [Extended COCO API](https://github.com/jin-s13/xtcocoapi) to support COCO-WholeBody evaluation:
`pip install xtcocotools`
third_party/hamer/third-party/ViTPose/docs/en/tasks/2d_wholebody_keypoint.md
Vendored
+125
View File
@@ -0,0 +1,125 @@
# 2D Wholebody Keypoint Datasets
It is recommended to symlink the dataset root to `$MMPOSE/data`.
If your folder structure is different, you may need to change the corresponding paths in config files.
MMPose supported datasets:
- [COCO-WholeBody](#coco-wholebody) \[ [Homepage](https://github.com/jin-s13/COCO-WholeBody/) \]
- [Halpe](#halpe) \[ [Homepage](https://github.com/Fang-Haoshu/Halpe-FullBody/) \]
## COCO-WholeBody
<!-- [DATASET] -->
<details>
<summary align="right"><a href="https://link.springer.com/chapter/10.1007/978-3-030-58545-7_12">COCO-WholeBody (ECCV'2020)</a></summary>
```bibtex
@inproceedings{jin2020whole,
title={Whole-Body Human Pose Estimation in the Wild},
author={Jin, Sheng and Xu, Lumin and Xu, Jin and Wang, Can and Liu, Wentao and Qian, Chen and Ouyang, Wanli and Luo, Ping},
booktitle={Proceedings of the European Conference on Computer Vision (ECCV)},
year={2020}
}
```
</details>
For [COCO-WholeBody](https://github.com/jin-s13/COCO-WholeBody/) dataset, images can be downloaded from [COCO download](http://cocodataset.org/#download), 2017 Train/Val is needed for COCO keypoints training and validation.
Download COCO-WholeBody annotations for COCO-WholeBody annotations for [Train](https://drive.google.com/file/d/1thErEToRbmM9uLNi1JXXfOsaS5VK2FXf/view?usp=sharing) / [Validation](https://drive.google.com/file/d/1N6VgwKnj8DeyGXCvp1eYgNbRmw6jdfrb/view?usp=sharing) (Google Drive).
Download person detection result of COCO val2017 from [OneDrive](https://1drv.ms/f/s!AhIXJn_J-blWzzDXoz5BeFl8sWM-) or [GoogleDrive](https://drive.google.com/drive/folders/1fRUDNUDxe9fjqcRZ2bnF_TKMlO0nB_dk?usp=sharing).
Download and extract them under $MMPOSE/data, and make them look like this:
```text
mmpose
├── mmpose
├── docs
├── tests
├── tools
├── configs
`── data
│── coco
│-- annotations
│ │-- coco_wholebody_train_v1.0.json
│ |-- coco_wholebody_val_v1.0.json
|-- person_detection_results
| |-- COCO_val2017_detections_AP_H_56_person.json
│-- train2017
│ │-- 000000000009.jpg
│ │-- 000000000025.jpg
│ │-- 000000000030.jpg
│ │-- ...
`-- val2017
│-- 000000000139.jpg
│-- 000000000285.jpg
│-- 000000000632.jpg
│-- ...
```
Please also install the latest version of [Extended COCO API](https://github.com/jin-s13/xtcocoapi) (version>=1.5) to support COCO-WholeBody evaluation:
`pip install xtcocotools`
## Halpe
<!-- [DATASET] -->
<details>
<summary align="right"><a href="https://arxiv.org/abs/2004.00945">Halpe (CVPR'2020)</a></summary>
```bibtex
@inproceedings{li2020pastanet,
title={PaStaNet: Toward Human Activity Knowledge Engine},
author={Li, Yong-Lu and Xu, Liang and Liu, Xinpeng and Huang, Xijie and Xu, Yue and Wang, Shiyi and Fang, Hao-Shu and Ma, Ze and Chen, Mingyang and Lu, Cewu},
booktitle={CVPR},
year={2020}
}
```
</details>
For [Halpe](https://github.com/Fang-Haoshu/Halpe-FullBody/) dataset, please download images and annotations from [Halpe download](https://github.com/Fang-Haoshu/Halpe-FullBody).
The images of the training set are from [HICO-Det](https://drive.google.com/open?id=1QZcJmGVlF9f4h-XLWe9Gkmnmj2z1gSnk) and those of the validation set are from [COCO](http://images.cocodataset.org/zips/val2017.zip).
Download person detection result of COCO val2017 from [OneDrive](https://1drv.ms/f/s!AhIXJn_J-blWzzDXoz5BeFl8sWM-) or [GoogleDrive](https://drive.google.com/drive/folders/1fRUDNUDxe9fjqcRZ2bnF_TKMlO0nB_dk?usp=sharing).
Download and extract them under $MMPOSE/data, and make them look like this:
```text
mmpose
├── mmpose
├── docs
├── tests
├── tools
├── configs
`── data
│── halpe
│-- annotations
│ │-- halpe_train_v1.json
│ |-- halpe_val_v1.json
|-- person_detection_results
| |-- COCO_val2017_detections_AP_H_56_person.json
│-- hico_20160224_det
│ │-- anno_bbox.mat
│ │-- anno.mat
│ │-- README
│ │-- images
│ │ │-- train2015
│ │ │ │-- HICO_train2015_00000001.jpg
│ │ │ │-- HICO_train2015_00000002.jpg
│ │ │ │-- HICO_train2015_00000003.jpg
│ │ │ │-- ...
│ │ │-- test2015
│ │-- tools
│ │-- ...
`-- val2017
│-- 000000000139.jpg
│-- 000000000285.jpg
│-- 000000000632.jpg
│-- ...
```
Please also install the latest version of [Extended COCO API](https://github.com/jin-s13/xtcocoapi) (version>=1.5) to support Halpe evaluation:
`pip install xtcocotools`
third_party/hamer/third-party/ViTPose/docs/en/tasks/3d_body_keypoint.md
Vendored
+120
View File
@@ -0,0 +1,120 @@
# 3D Body Keypoint Datasets
It is recommended to symlink the dataset root to `$MMPOSE/data`.
If your folder structure is different, you may need to change the corresponding paths in config files.
MMPose supported datasets:
- [Human3.6M](#human36m) \[ [Homepage](http://vision.imar.ro/human3.6m/description.php) \]
- [CMU Panoptic](#cmu-panoptic) \[ [Homepage](http://domedb.perception.cs.cmu.edu/) \]
## Human3.6M
<!-- [DATASET] -->
<details>
<summary align="right"><a href="https://ieeexplore.ieee.org/abstract/document/6682899/">Human3.6M (TPAMI'2014)</a></summary>
```bibtex
@article{h36m_pami,
author = {Ionescu, Catalin and Papava, Dragos and Olaru, Vlad and Sminchisescu, Cristian},
title = {Human3.6M: Large Scale Datasets and Predictive Methods for 3D Human Sensing in Natural Environments},
journal = {IEEE Transactions on Pattern Analysis and Machine Intelligence},
publisher = {IEEE Computer Society},
volume = {36},
number = {7},
pages = {1325-1339},
month = {jul},
year = {2014}
}
```
</details>
For [Human3.6M](http://vision.imar.ro/human3.6m/description.php), please download from the official website and run the [preprocessing script](/tools/dataset/preprocess_h36m.py), which will extract camera parameters and pose annotations at full framerate (50 FPS) and downsampled framerate (10 FPS). The processed data should have the following structure:
```text
mmpose
├── mmpose
├── docs
├── tests
├── tools
├── configs
`── data
├── h36m
├── annotation_body3d
| ├── cameras.pkl
| ├── fps50
| | ├── h36m_test.npz
| | ├── h36m_train.npz
| | ├── joint2d_rel_stats.pkl
| | ├── joint2d_stats.pkl
| | ├── joint3d_rel_stats.pkl
| | `── joint3d_stats.pkl
| `── fps10
| ├── h36m_test.npz
| ├── h36m_train.npz
| ├── joint2d_rel_stats.pkl
| ├── joint2d_stats.pkl
| ├── joint3d_rel_stats.pkl
| `── joint3d_stats.pkl
`── images
├── S1
| ├── S1_Directions_1.54138969
| | ├── S1_Directions_1.54138969_00001.jpg
| | ├── S1_Directions_1.54138969_00002.jpg
| | ├── ...
| ├── ...
├── S5
├── S6
├── S7
├── S8
├── S9
`── S11
```
Please note that Human3.6M dataset is also used in the [3D_body_mesh](/docs/en/tasks/3d_body_mesh.md) task, where different schemes for data preprocessing and organizing are adopted.
## CMU Panoptic
<details>
<summary align="right"><a href="https://openaccess.thecvf.com/content_iccv_2015/html/Joo_Panoptic_Studio_A_ICCV_2015_paper.html">CMU Panoptic (ICCV'2015)</a></summary>
```bibtex
@Article = {joo_iccv_2015,
author = {Hanbyul Joo, Hao Liu, Lei Tan, Lin Gui, Bart Nabbe, Iain Matthews, Takeo Kanade, Shohei Nobuhara, and Yaser Sheikh},
title = {Panoptic Studio: A Massively Multiview System for Social Motion Capture},
booktitle = {ICCV},
year = {2015}
}
```
</details>
Please follow [voxelpose-pytorch](https://github.com/microsoft/voxelpose-pytorch) to prepare this dataset.
1. Download the dataset by following the instructions in [panoptic-toolbox](https://github.com/CMU-Perceptual-Computing-Lab/panoptic-toolbox) and extract them under `$MMPOSE/data/panoptic`.
2. Only download those sequences that are needed. You can also just download a subset of camera views by specifying the number of views (HD_Video_Number) and changing the camera order in `./scripts/getData.sh`. The used sequences and camera views can be found in [VoxelPose](https://arxiv.org/abs/2004.06239). Note that the sequence "160906_band3" might not be available due to errors on the server of CMU Panoptic.
3. Note that we only use HD videos, calibration data, and 3D Body Keypoint in the codes. You can comment out other irrelevant codes such as downloading 3D Face data in `./scripts/getData.sh`.
The directory tree should be like this:
```text
mmpose
├── mmpose
├── docs
├── tests
├── tools
├── configs
`── data
├── panoptic
├── 16060224_haggling1
| | ├── hdImgs
| | ├── hdvideos
| | ├── hdPose3d_stage1_coco19
| | ├── calibration_160224_haggling1.json
├── 160226_haggling1
├── ...
```
third_party/hamer/third-party/ViTPose/docs/en/tasks/3d_body_mesh.md
Vendored
+342
View File
@@ -0,0 +1,342 @@
# 3D Body Mesh Recovery Datasets
It is recommended to symlink the dataset root to `$MMPOSE/data`.
If your folder structure is different, you may need to change the corresponding paths in config files.
To achieve high-quality human mesh estimation, we use multiple datasets for training.
The following items should be prepared for human mesh training:
<!-- TOC -->
- [3D Body Mesh Recovery Datasets](#3d-body-mesh-recovery-datasets)
- [Notes](#notes)
- [Annotation Files for Human Mesh Estimation](#annotation-files-for-human-mesh-estimation)
- [SMPL Model](#smpl-model)
- [COCO](#coco)
- [Human3.6M](#human36m)
- [MPI-INF-3DHP](#mpi-inf-3dhp)
- [LSP](#lsp)
- [LSPET](#lspet)
- [CMU MoShed Data](#cmu-moshed-data)
<!-- TOC -->
## Notes
### Annotation Files for Human Mesh Estimation
For human mesh estimation, we use multiple datasets for training.
The annotation of different datasets are preprocessed to the same format. Please
follow the [preprocess procedure](https://github.com/nkolot/SPIN/tree/master/datasets/preprocess)
of SPIN to generate the annotation files or download the processed files from
[here](https://download.openmmlab.com/mmpose/datasets/mesh_annotation_files.zip),
and make it look like this:
```text
mmpose
├── mmpose
├── docs
├── tests
├── tools
├── configs
`── data
│── mesh_annotation_files
├── coco_2014_train.npz
├── h36m_valid_protocol1.npz
├── h36m_valid_protocol2.npz
├── hr-lspet_train.npz
├── lsp_dataset_original_train.npz
├── mpi_inf_3dhp_train.npz
└── mpii_train.npz
```
### SMPL Model
```bibtex
@article{loper2015smpl,
title={SMPL: A skinned multi-person linear model},
author={Loper, Matthew and Mahmood, Naureen and Romero, Javier and Pons-Moll, Gerard and Black, Michael J},
journal={ACM transactions on graphics (TOG)},
volume={34},
number={6},
pages={1--16},
year={2015},
publisher={ACM New York, NY, USA}
}
```
For human mesh estimation, SMPL model is used to generate the human mesh.
Please download the [gender neutral SMPL model](http://smplify.is.tue.mpg.de/),
[joints regressor](https://download.openmmlab.com/mmpose/datasets/joints_regressor_cmr.npy)
and [mean parameters](https://download.openmmlab.com/mmpose/datasets/smpl_mean_params.npz)
under `$MMPOSE/models/smpl`, and make it look like this:
```text
mmpose
├── mmpose
├── ...
├── models
│── smpl
├── joints_regressor_cmr.npy
├── smpl_mean_params.npz
└── SMPL_NEUTRAL.pkl
```
## COCO
<!-- [DATASET] -->
<details>
<summary align="right"><a href="https://link.springer.com/chapter/10.1007/978-3-319-10602-1_48">COCO (ECCV'2014)</a></summary>
```bibtex
@inproceedings{lin2014microsoft,
title={Microsoft coco: Common objects in context},
author={Lin, Tsung-Yi and Maire, Michael and Belongie, Serge and Hays, James and Perona, Pietro and Ramanan, Deva and Doll{\'a}r, Piotr and Zitnick, C Lawrence},
booktitle={European conference on computer vision},
pages={740--755},
year={2014},
organization={Springer}
}
```
</details>
For [COCO](http://cocodataset.org/) data, please download from [COCO download](http://cocodataset.org/#download). COCO'2014 Train is needed for human mesh estimation training.
Download and extract them under $MMPOSE/data, and make them look like this:
```text
mmpose
├── mmpose
├── docs
├── tests
├── tools
├── configs
`── data
│── coco
│-- train2014
│ ├── COCO_train2014_000000000009.jpg
│ ├── COCO_train2014_000000000025.jpg
│ ├── COCO_train2014_000000000030.jpg
| │-- ...
```
## Human3.6M
<!-- [DATASET] -->
<details>
<summary align="right"><a href="https://ieeexplore.ieee.org/abstract/document/6682899/">Human3.6M (TPAMI'2014)</a></summary>
```bibtex
@article{h36m_pami,
author = {Ionescu, Catalin and Papava, Dragos and Olaru, Vlad and Sminchisescu, Cristian},
title = {Human3.6M: Large Scale Datasets and Predictive Methods for 3D Human Sensing in Natural Environments},
journal = {IEEE Transactions on Pattern Analysis and Machine Intelligence},
publisher = {IEEE Computer Society},
volume = {36},
number = {7},
pages = {1325-1339},
month = {jul},
year = {2014}
}
```
</details>
For [Human3.6M](http://vision.imar.ro/human3.6m/description.php), we use the MoShed data provided in [HMR](https://github.com/akanazawa/hmr) for training.
However, due to license limitations, we are not allowed to redistribute the MoShed data.
For the evaluation on Human3.6M dataset, please follow the
[preprocess procedure](https://github.com/nkolot/SPIN/tree/master/datasets/preprocess)
of SPIN to extract test images from
[Human3.6M](http://vision.imar.ro/human3.6m/description.php) original videos,
and make it look like this:
```text
mmpose
├── mmpose
├── docs
├── tests
├── tools
├── configs
`── data
│── Human3.6M
├── images
   ├── S11_Directions_1.54138969_000001.jpg
   ├── S11_Directions_1.54138969_000006.jpg
   ├── S11_Directions_1.54138969_000011.jpg
   ├── ...
```
The download of Human3.6M dataset is quite difficult, you can also download the
[zip file](https://drive.google.com/file/d/1WnRJD9FS3NUf7MllwgLRJJC-JgYFr8oi/view?usp=sharing)
of the test images. However, due to the license limitations, we are not allowed to
redistribute the images either. So the users need to download the original video and
extract the images by themselves.
## MPI-INF-3DHP
<!-- [DATASET] -->
```bibtex
@inproceedings{mono-3dhp2017,
author = {Mehta, Dushyant and Rhodin, Helge and Casas, Dan and Fua, Pascal and Sotnychenko, Oleksandr and Xu, Weipeng and Theobalt, Christian},
title = {Monocular 3D Human Pose Estimation In The Wild Using Improved CNN Supervision},
booktitle = {3D Vision (3DV), 2017 Fifth International Conference on},
url = {http://gvv.mpi-inf.mpg.de/3dhp_dataset},
year = {2017},
organization={IEEE},
doi={10.1109/3dv.2017.00064},
}
```
For [MPI-INF-3DHP](http://gvv.mpi-inf.mpg.de/3dhp-dataset/), please follow the
[preprocess procedure](https://github.com/nkolot/SPIN/tree/master/datasets/preprocess)
of SPIN to sample images, and make them like this:
```text
mmpose
├── mmpose
├── docs
├── tests
├── tools
├── configs
`── data
├── mpi_inf_3dhp_test_set
│   ├── TS1
│   ├── TS2
│   ├── TS3
│   ├── TS4
│   ├── TS5
│   └── TS6
├── S1
│   ├── Seq1
│   └── Seq2
├── S2
│   ├── Seq1
│   └── Seq2
├── S3
│   ├── Seq1
│   └── Seq2
├── S4
│   ├── Seq1
│   └── Seq2
├── S5
│   ├── Seq1
│   └── Seq2
├── S6
│   ├── Seq1
│   └── Seq2
├── S7
│   ├── Seq1
│   └── Seq2
└── S8
├── Seq1
└── Seq2
```
## LSP
<!-- [DATASET] -->
```bibtex
@inproceedings{johnson2010clustered,
title={Clustered Pose and Nonlinear Appearance Models for Human Pose Estimation.},
author={Johnson, Sam and Everingham, Mark},
booktitle={bmvc},
volume={2},
number={4},
pages={5},
year={2010},
organization={Citeseer}
}
```
For [LSP](https://sam.johnson.io/research/lsp.html), please download the high resolution version
[LSP dataset original](http://sam.johnson.io/research/lsp_dataset_original.zip).
Extract them under `$MMPOSE/data`, and make them look like this:
```text
mmpose
├── mmpose
├── docs
├── tests
├── tools
├── configs
`── data
│── lsp_dataset_original
├── images
   ├── im0001.jpg
   ├── im0002.jpg
   └── ...
```
## LSPET
<!-- [DATASET] -->
```bibtex
@inproceedings{johnson2011learning,
title={Learning effective human pose estimation from inaccurate annotation},
author={Johnson, Sam and Everingham, Mark},
booktitle={CVPR 2011},
pages={1465--1472},
year={2011},
organization={IEEE}
}
```
For [LSPET](https://sam.johnson.io/research/lspet.html), please download its high resolution form
[HR-LSPET](http://datasets.d2.mpi-inf.mpg.de/hr-lspet/hr-lspet.zip).
Extract them under `$MMPOSE/data`, and make them look like this:
```text
mmpose
├── mmpose
├── docs
├── tests
├── tools
├── configs
`── data
│── lspet_dataset
├── images
│   ├── im00001.jpg
│   ├── im00002.jpg
│   ├── im00003.jpg
│   └── ...
└── joints.mat
```
## CMU MoShed Data
<!-- [DATASET] -->
```bibtex
@inproceedings{kanazawa2018end,
title={End-to-end recovery of human shape and pose},
author={Kanazawa, Angjoo and Black, Michael J and Jacobs, David W and Malik, Jitendra},
booktitle={Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition},
pages={7122--7131},
year={2018}
}
```
Real-world SMPL parameters are used for the adversarial training in human mesh estimation.
The MoShed data provided in [HMR](https://github.com/akanazawa/hmr) is included in this
[zip file](https://download.openmmlab.com/mmpose/datasets/mesh_annotation_files.zip).
Please download and extract it under `$MMPOSE/data`, and make it look like this:
```text
mmpose
├── mmpose
├── docs
├── tests
├── tools
├── configs
`── data
│── mesh_annotation_files
├── CMU_mosh.npz
└── ...
```
third_party/hamer/third-party/ViTPose/docs/en/tasks/3d_hand_keypoint.md
Vendored
+55
View File
@@ -0,0 +1,55 @@
# 3D Hand Keypoint Datasets
It is recommended to symlink the dataset root to `$MMPOSE/data`.
If your folder structure is different, you may need to change the corresponding paths in config files.
MMPose supported datasets:
- [InterHand2.6M](#interhand26m) \[ [Homepage](https://mks0601.github.io/InterHand2.6M/) \]
## InterHand2.6M
<!-- [DATASET] -->
<details>
<summary align="right"><a href="https://link.springer.com/content/pdf/10.1007/978-3-030-58565-5_33.pdf">InterHand2.6M (ECCV'2020)</a></summary>
```bibtex
@InProceedings{Moon_2020_ECCV_InterHand2.6M,
author = {Moon, Gyeongsik and Yu, Shoou-I and Wen, He and Shiratori, Takaaki and Lee, Kyoung Mu},
title = {InterHand2.6M: A Dataset and Baseline for 3D Interacting Hand Pose Estimation from a Single RGB Image},
booktitle = {European Conference on Computer Vision (ECCV)},
year = {2020}
}
```
</details>
For [InterHand2.6M](https://mks0601.github.io/InterHand2.6M/), please download from [InterHand2.6M](https://mks0601.github.io/InterHand2.6M/).
Please download the annotation files from [annotations](https://drive.google.com/drive/folders/1pWXhdfaka-J0fSAze0MsajN0VpZ8e8tO).
Extract them under {MMPose}/data, and make them look like this:
```text
mmpose
├── mmpose
├── docs
├── tests
├── tools
├── configs
`── data
│── interhand2.6m
|── annotations
| |── all
| |── human_annot
| |── machine_annot
| |── skeleton.txt
| |── subject.txt
|
`── images
| |── train
| | |-- Capture0 ~ Capture26
| |── val
| | |-- Capture0
| |── test
| | |-- Capture0 ~ Capture7
```