liyang/hand-motion-pipeline
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Initial snapshot: hand motion pipeline (Dyn-HaMR + dex-retargeting + SPIDER)

Browse Source 
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
liyang
2026-09-15 14:13:49 +08:00
commit 7e4ef6f98b
7002 changed files with 1452249 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
third_party/hamer/third-party/ViTPose/tools/webcam/configs/background/README.md
Vendored
+73
View File
@@ -0,0 +1,73 @@
# Matting Effects
We can apply background matting to the videos.
## Instruction
### Get started
Launch the demo from the mmpose root directory:
```shell
python tools/webcam/run_webcam.py --config tools/webcam/configs/background/background.py
```
### Hotkeys
| Hotkey | Function |
| -- | -- |
| b | Toggle the background matting effect on/off. |
| h | Show help information. |
| m | Show the monitoring information. |
| q | Exit. |
Note that the demo will automatically save the output video into a file `record.mp4`.
### Configuration
- **Choose a detection model**
Users can choose detection models from the [MMDetection Model Zoo](https://mmdetection.readthedocs.io/en/v2.20.0/model_zoo.html). Just set the `model_config` and `model_checkpoint` in the detector node accordingly, and the model will be automatically downloaded and loaded.
Note that in order to perform background matting, the model should be able to produce segmentation masks.
```python
# 'DetectorNode':
# This node performs object detection from the frame image using an
# MMDetection model.
dict(
type='DetectorNode',
name='Detector',
model_config='demo/mmdetection_cfg/mask_rcnn_r50_fpn_2x_coco.py',
model_checkpoint='https://download.openmmlab.com/'
'mmdetection/v2.0/mask_rcnn/mask_rcnn_r50_fpn_2x_coco/'
'mask_rcnn_r50_fpn_2x_coco_bbox_mAP-0.392'
'__segm_mAP-0.354_20200505_003907-3e542a40.pth',
input_buffer='_input_', # `_input_` is a runner-reserved buffer
output_buffer='det_result'),
```
- **Run the demo without GPU**
If you don't have GPU and CUDA in your device, the demo can run with only CPU by setting `device='cpu'` in all model nodes. For example:
```python
dict(
type='DetectorNode',
name='Detector',
model_config='demo/mmdetection_cfg/mask_rcnn_r50_fpn_2x_coco.py',
model_checkpoint='https://download.openmmlab.com/'
'mmdetection/v2.0/mask_rcnn/mask_rcnn_r50_fpn_2x_coco/'
'mask_rcnn_r50_fpn_2x_coco_bbox_mAP-0.392'
'__segm_mAP-0.354_20200505_003907-3e542a40.pth',
device='cpu',
input_buffer='_input_', # `_input_` is a runner-reserved buffer
output_buffer='det_result'),
```
- **Debug webcam and display**
You can launch the webcam runner with a debug config:
```shell
python tools/webcam/run_webcam.py --config tools/webcam/configs/examples/test_camera.py
```
third_party/hamer/third-party/ViTPose/tools/webcam/configs/background/background.py
Vendored
+93
View File
@@ -0,0 +1,93 @@
# Copyright (c) OpenMMLab. All rights reserved.
runner = dict(
# Basic configurations of the runner
name='Matting Effects',
camera_id=0,
camera_fps=10,
synchronous=False,
# Define nodes.
# The configuration of a node usually includes:
# 1. 'type': Node class name
# 2. 'name': Node name
# 3. I/O buffers (e.g. 'input_buffer', 'output_buffer'): specify the
# input and output buffer names. This may depend on the node class.
# 4. 'enable_key': assign a hot-key to toggle enable/disable this node.
# This may depend on the node class.
# 5. Other class-specific arguments
nodes=[
# 'DetectorNode':
# This node performs object detection from the frame image using an
# MMDetection model.
dict(
type='DetectorNode',
name='Detector',
model_config='demo/mmdetection_cfg/mask_rcnn_r50_fpn_2x_coco.py',
model_checkpoint='https://download.openmmlab.com/'
'mmdetection/v2.0/mask_rcnn/mask_rcnn_r50_fpn_2x_coco/'
'mask_rcnn_r50_fpn_2x_coco_bbox_mAP-0.392'
'__segm_mAP-0.354_20200505_003907-3e542a40.pth',
input_buffer='_input_', # `_input_` is a runner-reserved buffer
output_buffer='det_result'),
# 'TopDownPoseEstimatorNode':
# This node performs keypoint detection from the frame image using an
# MMPose top-down model. Detection results is needed.
dict(
type='TopDownPoseEstimatorNode',
name='Human Pose Estimator',
model_config='configs/wholebody/2d_kpt_sview_rgb_img/'
'topdown_heatmap/coco-wholebody/'
'vipnas_mbv3_coco_wholebody_256x192_dark.py',
model_checkpoint='https://openmmlab-share.oss-cn-hangz'
'hou.aliyuncs.com/mmpose/top_down/vipnas/vipnas_mbv3_co'
'co_wholebody_256x192_dark-e2158108_20211205.pth',
cls_names=['person'],
input_buffer='det_result',
output_buffer='human_pose'),
# 'ModelResultBindingNode':
# This node binds the latest model inference result with the current
# frame. (This means the frame image and inference result may be
# asynchronous).
dict(
type='ModelResultBindingNode',
name='ResultBinder',
frame_buffer='_frame_', # `_frame_` is a runner-reserved buffer
result_buffer='human_pose',
output_buffer='frame'),
# 'MattingNode':
# This node draw the matting visualization result in the frame image.
# mask results is needed.
dict(
type='BackgroundNode',
name='Visualizer',
enable_key='b',
enable=True,
frame_buffer='frame',
output_buffer='vis_bg',
cls_names=['person']),
# 'NoticeBoardNode':
# This node show a notice board with given content, e.g. help
# information.
dict(
type='NoticeBoardNode',
name='Helper',
enable_key='h',
frame_buffer='vis_bg',
output_buffer='vis',
content_lines=[
'This is a demo for background changing effects. Have fun!',
'', 'Hot-keys:', '"b": Change background',
'"h": Show help information',
'"m": Show diagnostic information', '"q": Exit'
],
),
# 'MonitorNode':
# This node show diagnostic information in the frame image. It can
# be used for debugging or monitoring system resource status.
dict(
type='MonitorNode',
name='Monitor',
enable_key='m',
enable=False,
frame_buffer='vis',
output_buffer='_display_') # `_frame_` is a runner-reserved buffer
])
third_party/hamer/third-party/ViTPose/tools/webcam/configs/examples/README.md
Vendored
+110
View File
@@ -0,0 +1,110 @@
# Pose Estimation Demo
This demo performs human bounding box and keypoint detection, and visualizes results.
<div align="center">
<img src="https://user-images.githubusercontent.com/15977946/148911749-57b263c0-1075-4a65-af53-b51fc815da68.gif" width="600px" alt><br>
</div>
## Instruction
### Get started
Launch the demo from the mmpose root directory:
```shell
python tools/webcam/run_webcam.py --config tools/webcam/configs/examples/pose_estimation.py
```
### Hotkeys
| Hotkey | Function |
| -- | -- |
| v | Toggle the pose visualization on/off. |
| h | Show help information. |
| m | Show the monitoring information. |
| q | Exit. |
Note that the demo will automatically save the output video into a file `record.mp4`.
### Configuration
- **Choose a detection model**
Users can choose detection models from the [MMDetection Model Zoo](https://mmdetection.readthedocs.io/en/v2.20.0/model_zoo.html). Just set the `model_config` and `model_checkpoint` in the detector node accordingly, and the model will be automatically downloaded and loaded.
```python
# 'DetectorNode':
# This node performs object detection from the frame image using an
# MMDetection model.
dict(
type='DetectorNode',
name='Detector',
model_config='demo/mmdetection_cfg/'
'ssdlite_mobilenetv2_scratch_600e_coco.py',
model_checkpoint='https://download.openmmlab.com'
'/mmdetection/v2.0/ssd/'
'ssdlite_mobilenetv2_scratch_600e_coco/ssdlite_mobilenetv2_'
'scratch_600e_coco_20210629_110627-974d9307.pth',
input_buffer='_input_',
output_buffer='det_result')
```
- **Choose a or more pose models**
In this demo we use two [top-down](https://github.com/open-mmlab/mmpose/tree/master/configs/body/2d_kpt_sview_rgb_img/topdown_heatmap) pose estimation models for humans and animals respectively. Users can choose models from the [MMPose Model Zoo](https://mmpose.readthedocs.io/en/latest/modelzoo.html). To apply different pose models on different instance types, you can add multiple pose estimator nodes with `cls_names` set accordingly.
```python
# 'TopDownPoseEstimatorNode':
# This node performs keypoint detection from the frame image using an
# MMPose top-down model. Detection results is needed.
dict(
type='TopDownPoseEstimatorNode',
name='Human Pose Estimator',
model_config='configs/wholebody/2d_kpt_sview_rgb_img/'
'topdown_heatmap/coco-wholebody/'
'vipnas_mbv3_coco_wholebody_256x192_dark.py',
model_checkpoint='https://openmmlab-share.oss-cn-hangz'
'hou.aliyuncs.com/mmpose/top_down/vipnas/vipnas_mbv3_co'
'co_wholebody_256x192_dark-e2158108_20211205.pth',
cls_names=['person'],
input_buffer='det_result',
output_buffer='human_pose'),
dict(
type='TopDownPoseEstimatorNode',
name='Animal Pose Estimator',
model_config='configs/animal/2d_kpt_sview_rgb_img/topdown_heatmap'
'/animalpose/hrnet_w32_animalpose_256x256.py',
model_checkpoint='https://download.openmmlab.com/mmpose/animal/'
'hrnet/hrnet_w32_animalpose_256x256-1aa7f075_20210426.pth',
cls_names=['cat', 'dog', 'horse', 'sheep', 'cow'],
input_buffer='human_pose',
output_buffer='animal_pose')
```
- **Run the demo without GPU**
If you don't have GPU and CUDA in your device, the demo can run with only CPU by setting `device='cpu'` in all model nodes. For example:
```python
dict(
type='DetectorNode',
name='Detector',
model_config='demo/mmdetection_cfg/'
'ssdlite_mobilenetv2_scratch_600e_coco.py',
model_checkpoint='https://download.openmmlab.com'
'/mmdetection/v2.0/ssd/'
'ssdlite_mobilenetv2_scratch_600e_coco/ssdlite_mobilenetv2_'
'scratch_600e_coco_20210629_110627-974d9307.pth',
device='cpu',
input_buffer='_input_',
output_buffer='det_result')
```
- **Debug webcam and display**
You can lanch the webcam runner with a debug config:
```shell
python tools/webcam/run_webcam.py --config tools/webcam/configs/examples/test_camera.py
```
third_party/hamer/third-party/ViTPose/tools/webcam/configs/examples/pose_estimation.py
Vendored
+115
View File
@@ -0,0 +1,115 @@
# Copyright (c) OpenMMLab. All rights reserved.
runner = dict(
# Basic configurations of the runner
name='Pose Estimation',
camera_id=0,
camera_fps=20,
synchronous=False,
# Define nodes.
# The configuration of a node usually includes:
# 1. 'type': Node class name
# 2. 'name': Node name
# 3. I/O buffers (e.g. 'input_buffer', 'output_buffer'): specify the
# input and output buffer names. This may depend on the node class.
# 4. 'enable_key': assign a hot-key to toggle enable/disable this node.
# This may depend on the node class.
# 5. Other class-specific arguments
nodes=[
# 'DetectorNode':
# This node performs object detection from the frame image using an
# MMDetection model.
dict(
type='DetectorNode',
name='Detector',
model_config='demo/mmdetection_cfg/'
'ssdlite_mobilenetv2_scratch_600e_coco.py',
model_checkpoint='https://download.openmmlab.com'
'/mmdetection/v2.0/ssd/'
'ssdlite_mobilenetv2_scratch_600e_coco/ssdlite_mobilenetv2_'
'scratch_600e_coco_20210629_110627-974d9307.pth',
input_buffer='_input_', # `_input_` is a runner-reserved buffer
output_buffer='det_result'),
# 'TopDownPoseEstimatorNode':
# This node performs keypoint detection from the frame image using an
# MMPose top-down model. Detection results is needed.
dict(
type='TopDownPoseEstimatorNode',
name='Human Pose Estimator',
model_config='configs/wholebody/2d_kpt_sview_rgb_img/'
'topdown_heatmap/coco-wholebody/'
'vipnas_mbv3_coco_wholebody_256x192_dark.py',
model_checkpoint='https://download.openmmlab.com/mmpose/top_down/'
'vipnas/vipnas_mbv3_coco_wholebody_256x192_dark'
'-e2158108_20211205.pth',
cls_names=['person'],
input_buffer='det_result',
output_buffer='human_pose'),
dict(
type='TopDownPoseEstimatorNode',
name='Animal Pose Estimator',
model_config='configs/animal/2d_kpt_sview_rgb_img/topdown_heatmap'
'/animalpose/hrnet_w32_animalpose_256x256.py',
model_checkpoint='https://download.openmmlab.com/mmpose/animal/'
'hrnet/hrnet_w32_animalpose_256x256-1aa7f075_20210426.pth',
cls_names=['cat', 'dog', 'horse', 'sheep', 'cow'],
input_buffer='human_pose',
output_buffer='animal_pose'),
# 'ModelResultBindingNode':
# This node binds the latest model inference result with the current
# frame. (This means the frame image and inference result may be
# asynchronous).
dict(
type='ModelResultBindingNode',
name='ResultBinder',
frame_buffer='_frame_', # `_frame_` is a runner-reserved buffer
result_buffer='animal_pose',
output_buffer='frame'),
# 'PoseVisualizerNode':
# This node draw the pose visualization result in the frame image.
# Pose results is needed.
dict(
type='PoseVisualizerNode',
name='Visualizer',
enable_key='v',
frame_buffer='frame',
output_buffer='vis'),
# 'NoticeBoardNode':
# This node show a notice board with given content, e.g. help
# information.
dict(
type='NoticeBoardNode',
name='Helper',
enable_key='h',
enable=True,
frame_buffer='vis',
output_buffer='vis_notice',
content_lines=[
'This is a demo for pose visualization and simple image '
'effects. Have fun!', '', 'Hot-keys:',
'"v": Pose estimation result visualization',
'"s": Sunglasses effect B-)', '"b": Bug-eye effect 0_0',
'"h": Show help information',
'"m": Show diagnostic information', '"q": Exit'
],
),
# 'MonitorNode':
# This node show diagnostic information in the frame image. It can
# be used for debugging or monitoring system resource status.
dict(
type='MonitorNode',
name='Monitor',
enable_key='m',
enable=False,
frame_buffer='vis_notice',
output_buffer='display'),
# 'RecorderNode':
# This node save the output video into a file.
dict(
type='RecorderNode',
name='Recorder',
out_video_file='record.mp4',
frame_buffer='display',
output_buffer='_display_'
# `_display_` is a runner-reserved buffer
)
])
third_party/hamer/third-party/ViTPose/tools/webcam/configs/examples/test_camera.py
Vendored
+19
View File
@@ -0,0 +1,19 @@
# Copyright (c) OpenMMLab. All rights reserved.
runner = dict(
name='Debug CamRunner',
camera_id=0,
camera_fps=20,
nodes=[
dict(
type='MonitorNode',
name='Monitor',
enable_key='m',
frame_buffer='_frame_',
output_buffer='display'),
dict(
type='RecorderNode',
name='Recorder',
out_video_file='webcam_output.mp4',
frame_buffer='display',
output_buffer='_display_')
])
third_party/hamer/third-party/ViTPose/tools/webcam/configs/eyes/README.md
Vendored
+31
View File
@@ -0,0 +1,31 @@
# Sunglasses and Bug-eye Effects
We can apply fun effects on videos with pose estimation results, like adding sunglasses on the face, or make the eyes look bigger.
<div align="center">
<img src="https://user-images.githubusercontent.com/15977946/124059525-ce20c580-da5d-11eb-8e4a-2d96cd31fe9f.gif" width="600px" alt><br>
</div>
## Instruction
### Get started
Launch the demo from the mmpose root directory:
```shell
python tools/webcam/run_webcam.py --config tools/webcam/configs/examples/pose_estimation.py
```
### Hotkeys
| Hotkey | Function |
| -- | -- |
| s | Toggle the sunglasses effect on/off. |
| b | Toggle the bug-eye effect on/off. |
| h | Show help information. |
| m | Show the monitoring information. |
| q | Exit. |
### Configuration
See the [README](/tools/webcam/configs/examples/README.md#configuration) of pose estimation demo for model configurations.
third_party/hamer/third-party/ViTPose/tools/webcam/configs/eyes/eyes.py
Vendored
+114
View File
@@ -0,0 +1,114 @@
# Copyright (c) OpenMMLab. All rights reserved.
runner = dict(
# Basic configurations of the runner
name='Eye Effects',
camera_id=0,
camera_fps=20,
synchronous=False,
# Define nodes.
# The configuration of a node usually includes:
# 1. 'type': Node class name
# 2. 'name': Node name
# 3. I/O buffers (e.g. 'input_buffer', 'output_buffer'): specify the
# input and output buffer names. This may depend on the node class.
# 4. 'enable_key': assign a hot-key to toggle enable/disable this node.
# This may depend on the node class.
# 5. Other class-specific arguments
nodes=[
# 'DetectorNode':
# This node performs object detection from the frame image using an
# MMDetection model.
dict(
type='DetectorNode',
name='Detector',
model_config='demo/mmdetection_cfg/'
'ssdlite_mobilenetv2_scratch_600e_coco.py',
model_checkpoint='https://download.openmmlab.com'
'/mmdetection/v2.0/ssd/'
'ssdlite_mobilenetv2_scratch_600e_coco/ssdlite_mobilenetv2_'
'scratch_600e_coco_20210629_110627-974d9307.pth',
input_buffer='_input_', # `_input_` is a runner-reserved buffer
output_buffer='det_result'),
# 'TopDownPoseEstimatorNode':
# This node performs keypoint detection from the frame image using an
# MMPose top-down model. Detection results is needed.
dict(
type='TopDownPoseEstimatorNode',
name='Human Pose Estimator',
model_config='configs/wholebody/2d_kpt_sview_rgb_img/'
'topdown_heatmap/coco-wholebody/'
'vipnas_mbv3_coco_wholebody_256x192_dark.py',
model_checkpoint='https://openmmlab-share.oss-cn-hangz'
'hou.aliyuncs.com/mmpose/top_down/vipnas/vipnas_mbv3_co'
'co_wholebody_256x192_dark-e2158108_20211205.pth',
cls_names=['person'],
input_buffer='det_result',
output_buffer='human_pose'),
dict(
type='TopDownPoseEstimatorNode',
name='Animal Pose Estimator',
model_config='configs/animal/2d_kpt_sview_rgb_img/topdown_heatmap'
'/animalpose/hrnet_w32_animalpose_256x256.py',
model_checkpoint='https://download.openmmlab.com/mmpose/animal/'
'hrnet/hrnet_w32_animalpose_256x256-1aa7f075_20210426.pth',
cls_names=['cat', 'dog', 'horse', 'sheep', 'cow'],
input_buffer='human_pose',
output_buffer='animal_pose'),
# 'ModelResultBindingNode':
# This node binds the latest model inference result with the current
# frame. (This means the frame image and inference result may be
# asynchronous).
dict(
type='ModelResultBindingNode',
name='ResultBinder',
frame_buffer='_frame_', # `_frame_` is a runner-reserved buffer
result_buffer='animal_pose',
output_buffer='frame'),
# 'SunglassesNode':
# This node draw the sunglasses effect in the frame image.
# Pose results is needed.
dict(
type='SunglassesNode',
name='Visualizer',
enable_key='s',
enable=True,
frame_buffer='frame',
output_buffer='vis_sunglasses'),
# 'BugEyeNode':
# This node draw the bug-eye effetc in the frame image.
# Pose results is needed.
dict(
type='BugEyeNode',
name='Visualizer',
enable_key='b',
enable=False,
frame_buffer='vis_sunglasses',
output_buffer='vis_bugeye'),
# 'NoticeBoardNode':
# This node show a notice board with given content, e.g. help
# information.
dict(
type='NoticeBoardNode',
name='Helper',
enable_key='h',
frame_buffer='vis_bugeye',
output_buffer='vis',
content_lines=[
'This is a demo for pose visualization and simple image '
'effects. Have fun!', '', 'Hot-keys:',
'"s": Sunglasses effect B-)', '"b": Bug-eye effect 0_0',
'"h": Show help information',
'"m": Show diagnostic information', '"q": Exit'
],
),
# 'MonitorNode':
# This node show diagnostic information in the frame image. It can
# be used for debugging or monitoring system resource status.
dict(
type='MonitorNode',
name='Monitor',
enable_key='m',
enable=False,
frame_buffer='vis',
output_buffer='_display_') # `_frame_` is a runner-reserved buffer
])
third_party/hamer/third-party/ViTPose/tools/webcam/configs/face_swap/README.md
Vendored
+31
View File
@@ -0,0 +1,31 @@
# Sunglasses and Bug-eye Effects
Look! Where is my face?:eyes: And whose face is it?:laughing:
<div align="center">
<img src="https://user-images.githubusercontent.com/15977946/149705000-8818d55c-9264-409f-b995-77718cc6c822.gif" width="600px" alt><br>
</div>
## Instruction
### Get started
Launch the demo from the mmpose root directory:
```shell
python tools/webcam/run_webcam.py --config tools/webcam/configs/face_swap/face_swap.py
```
### Hotkeys
| Hotkey | Function |
| -- | -- |
| s | Switch between modes <ul><li>Shuffle: Randomly shuffle all faces</li><li>Clone: Choose one face and clone it for everyone</li><li>None: Nothing happens and everyone is safe :)</li></ul>|
| v | Toggle the pose visualization on/off. |
| h | Show help information. |
| m | Show diagnostic information. |
| q | Exit. |
### Configuration
See the [README](/tools/webcam/configs/examples/README.md#configuration) of pose estimation demo for model configurations.
third_party/hamer/third-party/ViTPose/tools/webcam/configs/face_swap/face_swap.py
Vendored
+79
View File
@@ -0,0 +1,79 @@
# Copyright (c) OpenMMLab. All rights reserved.
runner = dict(
name='FaceSwap',
camera_id=0,
camera_fps=20,
synchronous=False,
nodes=[
dict(
type='DetectorNode',
name='Detector',
model_config='demo/mmdetection_cfg/'
'ssdlite_mobilenetv2_scratch_600e_coco.py',
model_checkpoint='https://download.openmmlab.com'
'/mmdetection/v2.0/ssd/'
'ssdlite_mobilenetv2_scratch_600e_coco/ssdlite_mobilenetv2_'
'scratch_600e_coco_20210629_110627-974d9307.pth',
device='cpu',
input_buffer='_input_', # `_input_` is a runner-reserved buffer
output_buffer='det_result'),
dict(
type='TopDownPoseEstimatorNode',
name='TopDown Pose Estimator',
model_config='configs/wholebody/2d_kpt_sview_rgb_img/'
'topdown_heatmap/coco-wholebody/'
'vipnas_res50_coco_wholebody_256x192_dark.py',
model_checkpoint='https://openmmlab-share.oss-cn-hangzhou'
'.aliyuncs.com/mmpose/top_down/vipnas/'
'vipnas_res50_wholebody_256x192_dark-67c0ce35_20211112.pth',
device='cpu',
cls_names=['person'],
input_buffer='det_result',
output_buffer='pose_result'),
dict(
type='ModelResultBindingNode',
name='ResultBinder',
frame_buffer='_frame_', # `_frame_` is a runner-reserved buffer
result_buffer='pose_result',
output_buffer='frame'),
dict(
type='FaceSwapNode',
name='FaceSwapper',
mode_key='s',
frame_buffer='frame',
output_buffer='face_swap'),
dict(
type='PoseVisualizerNode',
name='Visualizer',
enable_key='v',
frame_buffer='face_swap',
output_buffer='vis_pose'),
dict(
type='NoticeBoardNode',
name='Help Information',
enable_key='h',
content_lines=[
'Swap your faces! ',
'Hot-keys:',
'"v": Toggle the pose visualization on/off.',
'"s": Switch between modes: Shuffle, Clone and None',
'"h": Show help information',
'"m": Show diagnostic information',
'"q": Exit',
],
frame_buffer='vis_pose',
output_buffer='vis_notice'),
dict(
type='MonitorNode',
name='Monitor',
enable_key='m',
enable=False,
frame_buffer='vis_notice',
output_buffer='display'),
dict(
type='RecorderNode',
name='Recorder',
out_video_file='faceswap_output.mp4',
frame_buffer='display',
output_buffer='_display_')
])
third_party/hamer/third-party/ViTPose/tools/webcam/configs/meow_dwen_dwen/README.md
Vendored
+44
View File
@@ -0,0 +1,44 @@
# Meow Dwen Dwen
Do you know [Bing DwenDwen (冰墩墩)](https://en.wikipedia.org/wiki/Bing_Dwen_Dwen_and_Shuey_Rhon_Rhon), the mascot of 2022 Beijing Olympic Games?
<div align="center">
<img src="https://user-images.githubusercontent.com/15977946/152742707-c0c51844-e1d0-42d0-9a12-e369002e082f.jpg" width="224px" alt><br>
</div>
Now you can dress your cat up in this costume and TA-DA! Be prepared for super cute **Meow Dwen Dwen**.
<div align="center">
<img src="https://user-images.githubusercontent.com/15977946/152942243-a17194a2-0fd1-4467-993c-634f6d7966d8.gif" width="300px" alt><br>
</div>
You are a dog fan? Hold on, here comes Woof Dwen Dwen.
<div align="center">
<img src="https://user-images.githubusercontent.com/15977946/152942747-84240692-3944-48a5-b60b-e60bd0a4339c.gif" width="300px" alt><br>
</div>
## Instruction
### Get started
Launch the demo from the mmpose root directory:
```shell
python tools/webcam/run_webcam.py --config tools/webcam/configs/meow_dwen_dwen/meow_dwen_dwen.py
```
### Hotkeys
| Hotkey | Function |
| -- | -- |
| s | Change the background. |
| h | Show help information. |
| m | Show diagnostic information. |
| q | Exit. |
### Configuration
- **Use video input**
As you can see in the config, we set `camera_id` as the path of the input image. You can also set it as a video file path (or url), or a webcam ID number (e.g. `camera_id=0`), to capture the dynamic face from the video input.
third_party/hamer/third-party/ViTPose/tools/webcam/configs/meow_dwen_dwen/meow_dwen_dwen.py
Vendored
+92
View File
@@ -0,0 +1,92 @@
# Copyright (c) OpenMMLab. All rights reserved.
runner = dict(
# Basic configurations of the runner
name='Little fans of 2022 Beijing Winter Olympics',
# Cat image
camera_id='https://user-images.githubusercontent.com/'
'15977946/152932036-b5554cf8-24cf-40d6-a358-35a106013f11.jpeg',
# Dog image
# camera_id='https://user-images.githubusercontent.com/'
# '15977946/152932051-cd280b35-8066-45a0-8f52-657c8631aaba.jpg',
camera_fps=20,
nodes=[
dict(
type='DetectorNode',
name='Detector',
model_config='demo/mmdetection_cfg/'
'ssdlite_mobilenetv2_scratch_600e_coco.py',
model_checkpoint='https://download.openmmlab.com'
'/mmdetection/v2.0/ssd/'
'ssdlite_mobilenetv2_scratch_600e_coco/ssdlite_mobilenetv2_'
'scratch_600e_coco_20210629_110627-974d9307.pth',
input_buffer='_input_', # `_input_` is a runner-reserved buffer
output_buffer='det_result'),
dict(
type='TopDownPoseEstimatorNode',
name='Animal Pose Estimator',
model_config='configs/animal/2d_kpt_sview_rgb_img/topdown_heatmap'
'/ap10k/hrnet_w32_ap10k_256x256.py',
model_checkpoint='https://download.openmmlab.com/mmpose/animal/'
'hrnet/hrnet_w32_ap10k_256x256-18aac840_20211029.pth',
cls_names=['cat', 'dog'],
input_buffer='det_result',
output_buffer='animal_pose'),
dict(
type='TopDownPoseEstimatorNode',
name='TopDown Pose Estimator',
model_config='configs/wholebody/2d_kpt_sview_rgb_img/'
'topdown_heatmap/coco-wholebody/'
'vipnas_res50_coco_wholebody_256x192_dark.py',
model_checkpoint='https://openmmlab-share.oss-cn-hangzhou'
'.aliyuncs.com/mmpose/top_down/vipnas/'
'vipnas_res50_wholebody_256x192_dark-67c0ce35_20211112.pth',
device='cpu',
cls_names=['person'],
input_buffer='animal_pose',
output_buffer='human_pose'),
dict(
type='ModelResultBindingNode',
name='ResultBinder',
frame_buffer='_frame_', # `_frame_` is a runner-reserved buffer
result_buffer='human_pose',
output_buffer='frame'),
dict(
type='XDwenDwenNode',
name='XDwenDwen',
mode_key='s',
resource_file='tools/webcam/configs/meow_dwen_dwen/'
'resource-info.json',
out_shape=(480, 480),
frame_buffer='frame',
output_buffer='vis'),
dict(
type='NoticeBoardNode',
name='Helper',
enable_key='h',
enable=False,
frame_buffer='vis',
output_buffer='vis_notice',
content_lines=[
'Let your pet put on a costume of Bing-Dwen-Dwen, '
'the mascot of 2022 Beijing Winter Olympics. Have fun!', '',
'Hot-keys:', '"s": Change the background',
'"h": Show help information',
'"m": Show diagnostic information', '"q": Exit'
],
),
dict(
type='MonitorNode',
name='Monitor',
enable_key='m',
enable=False,
frame_buffer='vis_notice',
output_buffer='display'),
dict(
type='RecorderNode',
name='Recorder',
out_video_file='record.mp4',
frame_buffer='display',
output_buffer='_display_'
# `_display_` is a runner-reserved buffer
)
])
third_party/hamer/third-party/ViTPose/tools/webcam/configs/meow_dwen_dwen/resource-info.json
Vendored
+26
View File
@@ -0,0 +1,26 @@
[
{
"id": 1,
"result": "{\"width\":690,\"height\":713,\"valid\":true,\"rotate\":0,\"step_1\":{\"toolName\":\"pointTool\",\"result\":[{\"x\":374.86387434554973,\"y\":262.8020942408377,\"attribute\":\"\",\"valid\":true,\"id\":\"8SK9cVyu\",\"sourceID\":\"\",\"textAttribute\":\"\",\"order\":2},{\"x\":492.8261780104712,\"y\":285.2,\"attribute\":\"\",\"valid\":true,\"id\":\"qDk54WsI\",\"sourceID\":\"\",\"textAttribute\":\"\",\"order\":1},{\"x\":430.11204188481673,\"y\":318.0502617801047,\"attribute\":\"\",\"valid\":true,\"id\":\"4H80L7lL\",\"sourceID\":\"\",\"textAttribute\":\"\",\"order\":3}]},\"step_2\":{\"dataSourceStep\":0,\"toolName\":\"polygonTool\",\"result\":[{\"id\":\"pwUsrf9u\",\"sourceID\":\"\",\"valid\":true,\"textAttribute\":\"\",\"pointList\":[{\"x\":423.3926701570681,\"y\":191.87539267015708},{\"x\":488.3465968586388,\"y\":209.04712041884818},{\"x\":535.3821989528797,\"y\":248.6167539267016},{\"x\":549.5675392670157,\"y\":306.8513089005236},{\"x\":537.6219895287959,\"y\":349.407329842932},{\"x\":510.74450261780106,\"y\":381.51099476439794},{\"x\":480.1340314136126,\"y\":394.9497382198953},{\"x\":411.4471204188482,\"y\":390.47015706806286},{\"x\":355.45235602094243,\"y\":373.29842931937173},{\"x\":306.17696335078534,\"y\":327.00942408376966},{\"x\":294.97801047120424,\"y\":284.45340314136126},{\"x\":306.9235602094241,\"y\":245.6303664921466},{\"x\":333.8010471204189,\"y\":217.25968586387435},{\"x\":370.3842931937173,\"y\":196.35497382198955}],\"attribute\":\"\",\"order\":1}]}}",
"url": "https://user-images.githubusercontent.com/15977946/152742677-35fe8a01-bd06-4a12-a02e-949e7d71f28a.jpg",
"fileName": "bing_dwen_dwen1.jpg"
},
{
"id": 2,
"result": "{\"width\":690,\"height\":659,\"valid\":true,\"rotate\":0,\"step_1\":{\"dataSourceStep\":0,\"toolName\":\"pointTool\",\"result\":[{\"x\":293.2460732984293,\"y\":242.89842931937173,\"attribute\":\"\",\"valid\":true,\"id\":\"KgPs39bY\",\"sourceID\":\"\",\"textAttribute\":\"\",\"order\":1},{\"x\":170.41675392670155,\"y\":270.50052356020944,\"attribute\":\"\",\"valid\":true,\"id\":\"XwHyoBFU\",\"sourceID\":\"\",\"textAttribute\":\"\",\"order\":2},{\"x\":224.24083769633506,\"y\":308.45340314136126,\"attribute\":\"\",\"valid\":true,\"id\":\"Qfs4YfuB\",\"sourceID\":\"\",\"textAttribute\":\"\",\"order\":3}]},\"step_2\":{\"dataSourceStep\":0,\"toolName\":\"polygonTool\",\"result\":[{\"id\":\"ts5jlJxb\",\"sourceID\":\"\",\"valid\":true,\"textAttribute\":\"\",\"pointList\":[{\"x\":178.69738219895285,\"y\":184.93403141361256},{\"x\":204.91937172774865,\"y\":172.5130890052356},{\"x\":252.5329842931937,\"y\":169.0628272251309},{\"x\":295.3162303664921,\"y\":175.27329842931937},{\"x\":333.95916230366487,\"y\":195.2848167539267},{\"x\":360.18115183246067,\"y\":220.1267015706806},{\"x\":376.0523560209424,\"y\":262.909947643979},{\"x\":373.98219895287957,\"y\":296.0324607329843},{\"x\":344.99999999999994,\"y\":335.365445026178},{\"x\":322.22827225130885,\"y\":355.37696335078533},{\"x\":272.544502617801,\"y\":378.1486910994764},{\"x\":221.48062827225127,\"y\":386.42931937172773},{\"x\":187.6680628272251,\"y\":385.7392670157068},{\"x\":158.68586387434553,\"y\":369.1780104712042},{\"x\":137.98429319371724,\"y\":337.43560209424083},{\"x\":127.63350785340312,\"y\":295.34240837696336},{\"x\":131.0837696335078,\"y\":242.89842931937173},{\"x\":147.64502617801045,\"y\":208.3958115183246}],\"attribute\":\"\",\"order\":1}]}}",
"url": "https://user-images.githubusercontent.com/15977946/152742707-c0c51844-e1d0-42d0-9a12-e369002e082f.jpg",
"fileName": "bing_dwen_dwen2.jpg"
},
{
"id": 3,
"result": "{\"width\":690,\"height\":811,\"valid\":true,\"rotate\":0,\"step_1\":{\"dataSourceStep\":0,\"toolName\":\"pointTool\",\"result\":[{\"x\":361.13507853403144,\"y\":300.62198952879584,\"attribute\":\"\",\"valid\":true,\"id\":\"uAtbXtf2\",\"sourceID\":\"\",\"textAttribute\":\"\",\"order\":1},{\"x\":242.24502617801048,\"y\":317.60628272251313,\"attribute\":\"\",\"valid\":true,\"id\":\"iLtceHMA\",\"sourceID\":\"\",\"textAttribute\":\"\",\"order\":2},{\"x\":302.5392670157068,\"y\":356.67015706806285,\"attribute\":\"\",\"valid\":true,\"id\":\"n9MTlJ6A\",\"sourceID\":\"\",\"textAttribute\":\"\",\"order\":3}]},\"step_2\":{\"dataSourceStep\":0,\"toolName\":\"polygonTool\",\"result\":[{\"id\":\"5sTLU5wF\",\"sourceID\":\"\",\"valid\":true,\"textAttribute\":\"\",\"pointList\":[{\"x\":227.80837696335078,\"y\":247.12146596858642},{\"x\":248.18952879581153,\"y\":235.23246073298432},{\"x\":291.4994764397906,\"y\":225.04188481675394},{\"x\":351.7937172774869,\"y\":229.28795811518327},{\"x\":393.40523560209425,\"y\":245.42303664921468},{\"x\":424.8261780104712,\"y\":272.59790575916236},{\"x\":443.5089005235602,\"y\":298.07434554973827},{\"x\":436.7151832460733,\"y\":345.6303664921466},{\"x\":406.1434554973822,\"y\":382.9958115183247},{\"x\":355.1905759162304,\"y\":408.4722513089006},{\"x\":313.57905759162304,\"y\":419.5120418848168},{\"x\":262.6261780104712,\"y\":417.81361256544506},{\"x\":224.41151832460733,\"y\":399.9801047120419},{\"x\":201.48272251308902,\"y\":364.3130890052356},{\"x\":194.68900523560208,\"y\":315.0586387434555},{\"x\":202.33193717277487,\"y\":272.59790575916236}],\"attribute\":\"\",\"order\":1}]}}",
"url": "https://user-images.githubusercontent.com/15977946/152742728-99392ecf-8f5c-46cf-b5c4-fe7fb6b39976.jpg",
"fileName": "bing_dwen_dwen3.jpg"
},
{
"id": 4,
"result": "{\"width\":690,\"height\":690,\"valid\":true,\"rotate\":0,\"step_1\":{\"dataSourceStep\":0,\"toolName\":\"pointTool\",\"result\":[{\"x\":365.9528795811519,\"y\":464.5759162303665,\"attribute\":\"\",\"valid\":true,\"id\":\"IKprTuHS\",\"sourceID\":\"\",\"textAttribute\":\"\",\"order\":1},{\"x\":470.71727748691103,\"y\":445.06806282722516,\"attribute\":\"\",\"valid\":true,\"id\":\"Z90CWkEI\",\"sourceID\":\"\",\"textAttribute\":\"\",\"order\":2},{\"x\":410.74869109947645,\"y\":395.2146596858639,\"attribute\":\"\",\"valid\":true,\"id\":\"UWRstKZk\",\"sourceID\":\"\",\"textAttribute\":\"\",\"order\":3}]},\"step_2\":{\"dataSourceStep\":0,\"toolName\":\"polygonTool\",\"result\":[{\"id\":\"C30Pc9Ww\",\"sourceID\":\"\",\"valid\":true,\"textAttribute\":\"\",\"pointList\":[{\"x\":412.91623036649213,\"y\":325.85340314136124},{\"x\":468.5497382198953,\"y\":335.9685863874345},{\"x\":501.78534031413614,\"y\":369.2041884816754},{\"x\":514.0680628272252,\"y\":415.44502617801044},{\"x\":504.67539267015707,\"y\":472.5235602094241},{\"x\":484.44502617801044,\"y\":497.0890052356021},{\"x\":443.26178010471205,\"y\":512.9842931937172},{\"x\":389.7958115183246,\"y\":518.7643979057591},{\"x\":336.32984293193715,\"y\":504.31413612565444},{\"x\":302.3717277486911,\"y\":462.40837696335075},{\"x\":298.0366492146597,\"y\":416.89005235602093},{\"x\":318.26701570680626,\"y\":372.0942408376963},{\"x\":363.0628272251309,\"y\":341.0261780104712}],\"attribute\":\"\",\"order\":1}]}}",
"url": "https://user-images.githubusercontent.com/15977946/152742755-9dc75f89-4156-4103-9c6d-f35f1f409d11.jpg",
"fileName": "bing_dwen_dwen4.jpg"
}
]
third_party/hamer/third-party/ViTPose/tools/webcam/configs/newyear/README.md
Vendored
+31
View File
@@ -0,0 +1,31 @@
# New Year Hat and Firecracker Effects
This demo provides new year effects with pose estimation results, like adding hat on the head and firecracker in the hands.
<div align="center">
<img src="https://user-images.githubusercontent.com/28900607/149774252-8bff7738-2a54-4480-a31c-0a574e7159cb.gif" width="600px" alt><br>
</div>
## Instruction
### Get started
Launch the demo from the mmpose root directory:
```shell
python tools/webcam/run_webcam.py --config tools/webcam/configs/newyear/new_year.py
```
### Hotkeys
| Hotkey | Function |
| -- | -- |
| t | Toggle the hat effect on/off. |
| f | Toggle the firecracker effect on/off. |
| h | Show help information. |
| m | Show the monitoring information. |
| q | Exit. |
### Configuration
See the [README](/tools/webcam/configs/examples/README.md#configuration) of pose estimation demo for model configurations.
third_party/hamer/third-party/ViTPose/tools/webcam/configs/newyear/new_year.py
Vendored
+122
View File
@@ -0,0 +1,122 @@
# Copyright (c) OpenMMLab. All rights reserved.
runner = dict(
# Basic configurations of the runner
name='Pose Estimation',
camera_id=0,
camera_fps=20,
synchronous=False,
# Define nodes.
# The configuration of a node usually includes:
# 1. 'type': Node class name
# 2. 'name': Node name
# 3. I/O buffers (e.g. 'input_buffer', 'output_buffer'): specify the
# input and output buffer names. This may depend on the node class.
# 4. 'enable_key': assign a hot-key to toggle enable/disable this node.
# This may depend on the node class.
# 5. Other class-specific arguments
nodes=[
# 'DetectorNode':
# This node performs object detection from the frame image using an
# MMDetection model.
dict(
type='DetectorNode',
name='Detector',
model_config='demo/mmdetection_cfg/'
'ssdlite_mobilenetv2_scratch_600e_coco.py',
model_checkpoint='https://download.openmmlab.com'
'/mmdetection/v2.0/ssd/'
'ssdlite_mobilenetv2_scratch_600e_coco/ssdlite_mobilenetv2_'
'scratch_600e_coco_20210629_110627-974d9307.pth',
input_buffer='_input_', # `_input_` is a runner-reserved buffer
output_buffer='det_result'),
# 'TopDownPoseEstimatorNode':
# This node performs keypoint detection from the frame image using an
# MMPose top-down model. Detection results is needed.
dict(
type='TopDownPoseEstimatorNode',
name='Human Pose Estimator',
model_config='configs/wholebody/2d_kpt_sview_rgb_img/'
'topdown_heatmap/coco-wholebody/'
'vipnas_mbv3_coco_wholebody_256x192_dark.py',
model_checkpoint='https://openmmlab-share.oss-cn-hangz'
'hou.aliyuncs.com/mmpose/top_down/vipnas/vipnas_mbv3_co'
'co_wholebody_256x192_dark-e2158108_20211205.pth',
cls_names=['person'],
input_buffer='det_result',
output_buffer='human_pose'),
dict(
type='TopDownPoseEstimatorNode',
name='Animal Pose Estimator',
model_config='configs/animal/2d_kpt_sview_rgb_img/topdown_heatmap'
'/animalpose/hrnet_w32_animalpose_256x256.py',
model_checkpoint='https://download.openmmlab.com/mmpose/animal/'
'hrnet/hrnet_w32_animalpose_256x256-1aa7f075_20210426.pth',
cls_names=['cat', 'dog', 'horse', 'sheep', 'cow'],
input_buffer='human_pose',
output_buffer='animal_pose'),
# 'ModelResultBindingNode':
# This node binds the latest model inference result with the current
# frame. (This means the frame image and inference result may be
# asynchronous).
dict(
type='ModelResultBindingNode',
name='ResultBinder',
frame_buffer='_frame_', # `_frame_` is a runner-reserved buffer
result_buffer='animal_pose',
output_buffer='frame'),
# 'HatNode':
# This node draw the hat effect in the frame image.
# Pose results is needed.
dict(
type='HatNode',
name='Visualizer',
enable_key='t',
frame_buffer='frame',
output_buffer='vis_hat'),
# 'FirecrackerNode':
# This node draw the firecracker effect in the frame image.
# Pose results is needed.
dict(
type='FirecrackerNode',
name='Visualizer',
enable_key='f',
frame_buffer='vis_hat',
output_buffer='vis_firecracker'),
# 'NoticeBoardNode':
# This node show a notice board with given content, e.g. help
# information.
dict(
type='NoticeBoardNode',
name='Helper',
enable_key='h',
enable=True,
frame_buffer='vis_firecracker',
output_buffer='vis_notice',
content_lines=[
'This is a demo for pose visualization and simple image '
'effects. Have fun!', '', 'Hot-keys:', '"t": Hat effect',
'"f": Firecracker effect', '"h": Show help information',
'"m": Show diagnostic information', '"q": Exit'
],
),
# 'MonitorNode':
# This node show diagnostic information in the frame image. It can
# be used for debugging or monitoring system resource status.
dict(
type='MonitorNode',
name='Monitor',
enable_key='m',
enable=False,
frame_buffer='vis_notice',
output_buffer='display'),
# 'RecorderNode':
# This node save the output video into a file.
dict(
type='RecorderNode',
name='Recorder',
out_video_file='record.mp4',
frame_buffer='display',
output_buffer='_display_'
# `_display_` is a runner-reserved buffer
)
])
third_party/hamer/third-party/ViTPose/tools/webcam/configs/supersaiyan/README.md
Vendored
+96
View File
@@ -0,0 +1,96 @@
# Super Saiyan Effects
We can apply fun effects on videos with pose estimation results, like Super Saiyan transformation.
https://user-images.githubusercontent.com/11788150/150138076-2192079f-068a-4d43-bf27-2f1fd708cabc.mp4
## Instruction
### Get started
Launch the demo from the mmpose root directory:
```shell
python tools/webcam/run_webcam.py --config tools/webcam/configs/supersaiyan/saiyan.py
```
### Hotkeys
| Hotkey | Function |
| -- | -- |
| s | Toggle the Super Saiyan effect on/off. |
| h | Show help information. |
| m | Show the monitoring information. |
| q | Exit. |
Note that the demo will automatically save the output video into a file `record.mp4`.
### Configuration
- **Choose a detection model**
Users can choose detection models from the [MMDetection Model Zoo](https://mmdetection.readthedocs.io/en/v2.20.0/model_zoo.html). Just set the `model_config` and `model_checkpoint` in the detector node accordingly, and the model will be automatically downloaded and loaded.
```python
# 'DetectorNode':
# This node performs object detection from the frame image using an
# MMDetection model.
dict(
type='DetectorNode',
name='Detector',
model_config='demo/mmdetection_cfg/mask_rcnn_r50_fpn_2x_coco.py',
model_checkpoint='https://download.openmmlab.com/'
'mmdetection/v2.0/mask_rcnn/mask_rcnn_r50_fpn_2x_coco/'
'mask_rcnn_r50_fpn_2x_coco_bbox_mAP-0.392'
'__segm_mAP-0.354_20200505_003907-3e542a40.pth',
input_buffer='_input_', # `_input_` is a runner-reserved buffer
output_buffer='det_result'),
```
- **Choose a or more pose models**
In this demo we use two [top-down](https://github.com/open-mmlab/mmpose/tree/master/configs/body/2d_kpt_sview_rgb_img/topdown_heatmap) pose estimation models for humans and animals respectively. Users can choose models from the [MMPose Model Zoo](https://mmpose.readthedocs.io/en/latest/modelzoo.html). To apply different pose models on different instance types, you can add multiple pose estimator nodes with `cls_names` set accordingly.
```python
# 'TopDownPoseEstimatorNode':
# This node performs keypoint detection from the frame image using an
# MMPose top-down model. Detection results is needed.
dict(
type='TopDownPoseEstimatorNode',
name='Human Pose Estimator',
model_config='configs/wholebody/2d_kpt_sview_rgb_img/'
'topdown_heatmap/coco-wholebody/'
'vipnas_mbv3_coco_wholebody_256x192_dark.py',
model_checkpoint='https://openmmlab-share.oss-cn-hangz'
'hou.aliyuncs.com/mmpose/top_down/vipnas/vipnas_mbv3_co'
'co_wholebody_256x192_dark-e2158108_20211205.pth',
cls_names=['person'],
input_buffer='det_result',
output_buffer='human_pose')
```
- **Run the demo without GPU**
If you don't have GPU and CUDA in your device, the demo can run with only CPU by setting `device='cpu'` in all model nodes. For example:
```python
dict(
type='DetectorNode',
name='Detector',
model_config='demo/mmdetection_cfg/mask_rcnn_r50_fpn_2x_coco.py',
model_checkpoint='https://download.openmmlab.com/'
'mmdetection/v2.0/mask_rcnn/mask_rcnn_r50_fpn_2x_coco/'
'mask_rcnn_r50_fpn_2x_coco_bbox_mAP-0.392'
'__segm_mAP-0.354_20200505_003907-3e542a40.pth',
device='cpu',
input_buffer='_input_', # `_input_` is a runner-reserved buffer
output_buffer='det_result'),
```
- **Debug webcam and display**
You can launch the webcam runner with a debug config:
```shell
python tools/webcam/run_webcam.py --config tools/webcam/configs/examples/test_camera.py
```
third_party/hamer/third-party/ViTPose/tools/webcam/configs/supersaiyan/saiyan.py
Vendored
+93
View File
@@ -0,0 +1,93 @@
# Copyright (c) OpenMMLab. All rights reserved.
runner = dict(
# Basic configurations of the runner
name='Super Saiyan Effects',
camera_id=0,
camera_fps=30,
synchronous=False,
# Define nodes.
# The configuration of a node usually includes:
# 1. 'type': Node class name
# 2. 'name': Node name
# 3. I/O buffers (e.g. 'input_buffer', 'output_buffer'): specify the
# input and output buffer names. This may depend on the node class.
# 4. 'enable_key': assign a hot-key to toggle enable/disable this node.
# This may depend on the node class.
# 5. Other class-specific arguments
nodes=[
# 'DetectorNode':
# This node performs object detection from the frame image using an
# MMDetection model.
dict(
type='DetectorNode',
name='Detector',
model_config='demo/mmdetection_cfg/mask_rcnn_r50_fpn_2x_coco.py',
model_checkpoint='https://download.openmmlab.com/'
'mmdetection/v2.0/mask_rcnn/mask_rcnn_r50_fpn_2x_coco/'
'mask_rcnn_r50_fpn_2x_coco_bbox_mAP-0.392'
'__segm_mAP-0.354_20200505_003907-3e542a40.pth',
input_buffer='_input_', # `_input_` is a runner-reserved buffer
output_buffer='det_result'),
# 'TopDownPoseEstimatorNode':
# This node performs keypoint detection from the frame image using an
# MMPose top-down model. Detection results is needed.
dict(
type='TopDownPoseEstimatorNode',
name='Human Pose Estimator',
model_config='configs/wholebody/2d_kpt_sview_rgb_img/'
'topdown_heatmap/coco-wholebody/'
'vipnas_mbv3_coco_wholebody_256x192_dark.py',
model_checkpoint='https://openmmlab-share.oss-cn-hangz'
'hou.aliyuncs.com/mmpose/top_down/vipnas/vipnas_mbv3_co'
'co_wholebody_256x192_dark-e2158108_20211205.pth',
cls_names=['person'],
input_buffer='det_result',
output_buffer='human_pose'),
# 'ModelResultBindingNode':
# This node binds the latest model inference result with the current
# frame. (This means the frame image and inference result may be
# asynchronous).
dict(
type='ModelResultBindingNode',
name='ResultBinder',
frame_buffer='_frame_', # `_frame_` is a runner-reserved buffer
result_buffer='human_pose',
output_buffer='frame'),
# 'SaiyanNode':
# This node draw the Super Saiyan effect in the frame image.
# Pose results is needed.
dict(
type='SaiyanNode',
name='Visualizer',
enable_key='s',
cls_names=['person'],
enable=True,
frame_buffer='frame',
output_buffer='vis_saiyan'),
# 'NoticeBoardNode':
# This node show a notice board with given content, e.g. help
# information.
dict(
type='NoticeBoardNode',
name='Helper',
enable_key='h',
frame_buffer='vis_saiyan',
output_buffer='vis',
content_lines=[
'This is a demo for super saiyan effects. Have fun!', '',
'Hot-keys:', '"s": Saiyan effect',
'"h": Show help information',
'"m": Show diagnostic information', '"q": Exit'
],
),
# 'MonitorNode':
# This node show diagnostic information in the frame image. It can
# be used for debugging or monitoring system resource status.
dict(
type='MonitorNode',
name='Monitor',
enable_key='m',
enable=False,
frame_buffer='vis',
output_buffer='_display_') # `_frame_` is a runner-reserved buffer
])
third_party/hamer/third-party/ViTPose/tools/webcam/configs/valentinemagic/README.md
Vendored
+35
View File
@@ -0,0 +1,35 @@
# Valentine Magic
Do you want to show your **love** to your beloved one, especially on Valentine's Day? Express it with your pose using MMPose right away and see the Valentine Magic!
Try to pose a hand heart gesture, and see what will happen?
Prefer a blow kiss? Here comes your flying heart~
<div align="center">
<img src="https://user-images.githubusercontent.com/87690686/153837208-8975eda1-17d5-4ee7-8c83-9b535e5ee9d9.gif" width="300px" alt><br>
</div>
## Instruction
### Get started
Launch the demo from the mmpose root directory:
```shell
python tools/webcam/run_webcam.py --config tools/webcam/configs/valentinemagic/valentinemagic.py
```
### Hotkeys
| Hotkey | Function |
| -- | -- |
| l | Toggle the Valentine Magic effect on/off. |
| v | Toggle the pose visualization on/off. |
| h | Show help information. |
| m | Show diagnostic information. |
| q | Exit. |
### Configuration
See the [README](/tools/webcam/configs/examples/README.md#configuration) of pose estimation demo for model configurations.
third_party/hamer/third-party/ViTPose/tools/webcam/configs/valentinemagic/valentinemagic.py
Vendored
+118
View File
@@ -0,0 +1,118 @@
# Copyright (c) OpenMMLab. All rights reserved.
runner = dict(
# Basic configurations of the runner
name='Human Pose and Effects',
camera_id=0,
camera_fps=30,
# Define nodes.
#
# The configuration of a node usually includes:
# 1. 'type': Node class name
# 2. 'name': Node name
# 3. I/O buffers (e.g. 'input_buffer', 'output_buffer'): specify the
# input and output buffer names. This may depend on the node class.
# 4. 'enable_key': assign a hot-key to toggle enable/disable this node.
# This may depend on the node class.
# 5. Other class-specific arguments
nodes=[
# 'DetectorNode':
# This node performs object detection from the frame image using an
# MMDetection model.
dict(
type='DetectorNode',
name='Detector',
model_config='demo/mmdetection_cfg/'
'ssdlite_mobilenetv2_scratch_600e_coco.py',
model_checkpoint='https://download.openmmlab.com'
'/mmdetection/v2.0/ssd/'
'ssdlite_mobilenetv2_scratch_600e_coco/ssdlite_mobilenetv2_'
'scratch_600e_coco_20210629_110627-974d9307.pth',
input_buffer='_input_', # `_input_` is a runner-reserved buffer
output_buffer='det_result'),
# 'TopDownPoseEstimatorNode':
# This node performs keypoint detection from the frame image using an
# MMPose top-down model. Detection results is needed.
dict(
type='TopDownPoseEstimatorNode',
name='Human Pose Estimator',
model_config='configs/wholebody/2d_kpt_sview_rgb_img/'
'topdown_heatmap/coco-wholebody/'
'vipnas_mbv3_coco_wholebody_256x192_dark.py',
model_checkpoint='https://download.openmmlab.com/mmpose/top_down/'
'vipnas/vipnas_mbv3_coco_wholebody_256x192_dark'
'-e2158108_20211205.pth',
cls_names=['person'],
input_buffer='det_result',
output_buffer='pose_result'),
# 'ModelResultBindingNode':
# This node binds the latest model inference result with the current
# frame. (This means the frame image and inference result may be
# asynchronous).
dict(
type='ModelResultBindingNode',
name='ResultBinder',
frame_buffer='_frame_', # `_frame_` is a runner-reserved buffer
result_buffer='pose_result',
output_buffer='frame'),
# 'PoseVisualizerNode':
# This node draw the pose visualization result in the frame image.
# Pose results is needed.
dict(
type='PoseVisualizerNode',
name='Visualizer',
enable_key='v',
enable=False,
frame_buffer='frame',
output_buffer='vis'),
# 'ValentineMagicNode':
# This node draw heart in the image.
# It can launch dynamically expanding heart from the middle of
# hands if the persons pose a "hand heart" gesture or blow a kiss.
# Only there are two persons in the image can trigger this effect.
# Pose results is needed.
dict(
type='ValentineMagicNode',
name='Visualizer',
enable_key='l',
frame_buffer='vis',
output_buffer='vis_heart',
),
# 'NoticeBoardNode':
# This node show a notice board with given content, e.g. help
# information.
dict(
type='NoticeBoardNode',
name='Helper',
enable_key='h',
enable=False,
frame_buffer='vis_heart',
output_buffer='vis_notice',
content_lines=[
'This is a demo for pose visualization and simple image '
'effects. Have fun!', '', 'Hot-keys:',
'"h": Show help information', '"l": LoveHeart Effect',
'"v": PoseVisualizer', '"m": Show diagnostic information',
'"q": Exit'
],
),
# 'MonitorNode':
# This node show diagnostic information in the frame image. It can
# be used for debugging or monitoring system resource status.
dict(
type='MonitorNode',
name='Monitor',
enable_key='m',
enable=False,
frame_buffer='vis_notice',
output_buffer='display'), # `_frame_` is a runner-reserved buffer
# 'RecorderNode':
# This node record the frames into a local file. It can save the
# visualiztion results. Uncommit the following lines to turn it on.
dict(
type='RecorderNode',
name='Recorder',
out_video_file='record.mp4',
frame_buffer='display',
output_buffer='_display_')
])