Skip to content

Commit db4cef5

Browse files
author
1996scarlet
committed
update readme
1 parent 3f636b7 commit db4cef5

File tree

1 file changed

+40
-30
lines changed

1 file changed

+40
-30
lines changed

Readme.md

Lines changed: 40 additions & 30 deletions
Original file line numberDiff line numberDiff line change
@@ -1,11 +1,12 @@
11
# OpenVtuber-虚拟爱抖露共享计划
22

3-
## Kizuna-Ai MMD demo : face capture via single RGB camera
3+
Kizuna-Ai MMD demo : face capture via single RGB camera
44

55
<p align="center"><img src="docs/images/one.gif" /></p>
66
<p align="center"><img src="docs/images/two.gif" /></p>
77

88
## Installation
9+
910
### Requirements
1011

1112
* Python 3.5+
@@ -22,56 +23,65 @@ While not required, for optimal performance(especially for the detector) it is h
2223
* `python3.7 ./PythonClient/vtuber_usb_camera.py --gpu -1`
2324

2425
## 人脸检测 (Face Detection)
25-
* [RetinaFace: Single-stage Dense Face Localisation in the Wild](https://arxiv.org/abs/1905.00641)
26-
* [RetinaFace (mxnet version)](https://github.com/deepinsight/insightface/tree/master/RetinaFace)
27-
28-
RetinaFace is a practical single-stage [SOTA](http://shuoyang1213.me/WIDERFACE/WiderFace_Results.html) face detector which is initially described in [arXiv technical report](https://arxiv.org/abs/1905.00641)
2926

30-
![demoimg1](https://github.com/deepinsight/insightface/blob/master/resources/11513D05.jpg)
27+
[RetinaFace: Single-stage Dense Face Localisation in the Wild](https://openaccess.thecvf.com/content_CVPR_2020/html/Deng_RetinaFace_Single-Shot_Multi-Level_Face_Localisation_in_the_Wild_CVPR_2020_paper.html) of **CVPR 2020**, is a practical single-stage [SOTA](http://shuoyang1213.me/WIDERFACE/WiderFace_Results.html) face detector. It is highly recommended to read the official repo [RetinaFace (mxnet version)](https://github.com/deepinsight/insightface/tree/master/RetinaFace).
3128

32-
![demoimg2](https://github.com/deepinsight/insightface/blob/master/resources/widerfacevaltest.png)
29+
However, since the detection target of the face capture system is in the middle-close range, there is no need for complex pyramid scaling. We designed and published [Faster RetinaFace](https://github.com/1996scarlet/faster-mobile-retinaface) to trade off between speed and accuracy, which can reach 500~1000 fps on normal laptops.
3330

34-
## 头部姿态估计(Head Pose Estimation)
35-
* [head-pose-estimation](https://github.com/lincolnhard/head-pose-estimation)
31+
| Plan | Inference | Postprocess | Throughput Capacity (FPS)
32+
| --------|-----|--------|---------
33+
| 9750HQ+1660TI | 0.9ms | 1.5ms | 500~1000
34+
| Jetson-Nano | 4.6ms | 11.4ms | 80~200
3635

3736
## 特征点检测(Facial Landmarks Tracking)
37+
3838
The 2D pre-trained model is from the [deep-face-alignment](https://github.com/deepinx/deep-face-alignment) repository.
39+
3940
* Algorithm from [TPAMI 2019](https://arxiv.org/pdf/1808.04803.pdf)
4041
* Training set is based on i-bug 300-W datasets. It's annotation is shown below:<br><br>
4142
![ibug](https://cloud.githubusercontent.com/assets/16308037/24229391/1910e9cc-0fb4-11e7-987b-0fecce2c829e.JPG)
4243

43-
## 注视估计(Gaze Estimation)
44+
## 头部姿态估计(Head Pose Estimation
4445

45-
- [Laser Eye : Gaze Estimation via Deep Neural Networks](https://github.com/1996scarlet/Laser-Eye)
46+
* [head-pose-estimation](https://github.com/lincolnhard/head-pose-estimation)
4647

47-
## MMD Loader
48+
## 注视估计(Gaze Estimation)
4849

49-
- [Three.js Webgl Loader](https://threejs.org/examples/?q=MMD#webgl_loader_mmd)
50+
* [Laser Eye : Gaze Estimation via Deep Neural Networks](https://github.com/1996scarlet/Laser-Eye)
5051

51-
## Live2D
52+
## MMD Loader
5253

53-
- [插件版本](https://github.com/EYHN/hexo-helper-live2d)
54-
- [打包版本](https://github.com/galnetwen/Live2D)
54+
We apply [Three.js Webgl Loader](https://threejs.org/examples/?q=MMD#webgl_loader_mmd) to render MMD model on web pages.
5555

56-
## Thanks
56+
## Special Thanks
5757

58-
- [threejs.org](https://threejs.org/)
59-
- [kizunaai.com](http://kizunaai.com/)
58+
* [threejs.org](https://threejs.org/)
59+
* [kizunaai.com](http://kizunaai.com/)
6060

6161
## Citation
6262

63-
```
63+
``` bibtex
64+
@misc{sun2020backbone,
65+
title={A Backbone Replaceable Fine-tuning Network for Stable Face Alignment},
66+
author={Xu Sun and Yingjie Guo and Shihong Xia},
67+
year={2020},
68+
eprint={2010.09501},
69+
archivePrefix={arXiv},
70+
primaryClass={cs.CV}
71+
}
72+
6473
@article{Bulat2018Hierarchical,
65-
title={Hierarchical binary CNNs for landmark localization with limited resources},
66-
author={Bulat, Adrian and Tzimiropoulos, Yorgos},
67-
journal={IEEE Transactions on Pattern Analysis & Machine Intelligence},
68-
year={2018},
74+
title={Hierarchical binary CNNs for landmark localization with limited resources},
75+
author={Bulat, Adrian and Tzimiropoulos, Yorgos},
76+
journal={IEEE Transactions on Pattern Analysis & Machine Intelligence},
77+
year={2018},
6978
}
70-
71-
@inproceedings{deng2019retinaface,
72-
title={RetinaFace: Single-stage Dense Face Localisation in the Wild},
73-
author={Deng, Jiankang and Guo, Jia and Yuxiang, Zhou and Jinke Yu and Irene Kotsia and Zafeiriou, Stefanos},
74-
booktitle={arxiv},
75-
year={2019}
79+
80+
@InProceedings{Deng_2020_CVPR,
81+
author = {Deng, Jiankang and Guo, Jia and Ververas, Evangelos and Kotsia, Irene and Zafeiriou, Stefanos},
82+
title = {RetinaFace: Single-Shot Multi-Level Face Localisation in the Wild},
83+
booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
84+
month = {June},
85+
year = {2020}
7686
}
7787
```

0 commit comments

Comments
 (0)