# 3D Face

## Surveys & Doctoral Thesis

- Face Image Analysis using a Multiple Features Fitting Strategy(2005, Basel)

- 3D Face Modelling for 2D+3D Face Recognition(2007, Surrey)

- Image Based 3D Face Reconstruction: A Survey(IJIG2009, Georgios Stylianou,

Andreas Lanitis, EUC, CUT)

`early 3D facial acquisition approaches`

- animation reconstruction of deformable surfaces(2010, Hao Li, ETHz)

- Inverse Rendering of Faces with a 3D Morphable Model(2012, Oswald Aldrian, York)

- Digital Geometry Processing Theory and Applications(2012, Kun Zhou, Zhengjiang,

中文)

- ***State of the Art on Monocular 3D Face Reconstruction, Tracking, and

Applications***

**State of the Art on 3D Reconstruction with RGB-D Cameras**(EG2018, MZ, CT, MPI,
Stanford, TUM, Disney, Technicolor, UEN)
[[talks]](http://web.stanford.edu/~zollhoef/papers/EG18_FaceSTAR/page.html)

## Papers & Codes

### Reconstruction&3D Alignment&Correspondences

#### 1998 - 2015

- A Morphable Model For The Synthesis Of 3D Faces

(SIGGRAPH1998, [V Blanz](https://scholar.google.com.hk/citations?
user=jYCidWgAAAAJ&hl=zh-CN&oi=sra), [T
Vetter](https://scholar.google.com.hk/citations?user=HKLgZpYAAAAJ&hl=zh-
CN&oi=sra) , MPI)

`3dmm,analysis-by-synthesis(cascaded, coarse to fine, using texture

information), mid-detail `

- Efficient, Robust and Accurate Fitting of a 3D Morphable Model

(ICCV2003, S Romdhani, [T Vetter](https://scholar.google.com.hk/citations?

user=HKLgZpYAAAAJ&hl=zh-CN&oi=sra) , Basel)

`3dmm, fitting Algorithm needs: Efficient, Robust, Accurate, Automatic. Mid-

detail`

- Estimating 3D Shape and Texture Using Pixel Intensity, Edges, Specular

Highlights, Texture Constraints and a Prior

(CVPR2005, S Romdhani, [T Vetter](https://scholar.google.com.hk/citations?

user=HKLgZpYAAAAJ&hl=zh-CN&oi=sra) , Basel)
 `3dmm, multiple features`

- A 3D Face Model for Pose and Illumination Invariant Face Recognition

(AVSS2009, Paysan, P., Knothe, R., Amberg, B., Romdhani, S., & Vetter, T. ,
Basel) [[data](BFM)](https://faces.cs.unibas.ch/bfm/)

- 3D Face Reconstruction from a Single Image Using a Single Reference Face Shape

(TPAMI2011, [I
Kemelmacher-Shlizerman](https://scholar.google.com.hk/citations?
user=P97vI1EAAAAJ&hl=zh-CN&oi=sra), Basri R, UW)

`template, sfs, texture information, mid-detail`

- Face Reconstruction in the Wild

(ICCV2011, Kemelmacher-Shlizerman I, Seitz S M , UW)

`collection, sparse correspondence, warp template, low-rank

approximation(photometric stereo, for expression normalization), mid-detail `

- A FACS Valid 3D Dynamic Action Unit Database with Applications to 3D Dynamic

Morphable Facial Modeling

(ICCV2011, Cosker D, Krumhuber E, Hilton A. , UofSurrey)

`aam, expression`

- Viewing Real-World Faces in 3D

(ICCV2013, [T Hassner](https://scholar.google.com.hk/citations?
user=ehe5pyIAAAAJ&hl=zh-CN&oi=sra), Open U Israel)

`template, sparse correspondence, pose adjustment, depth optimization(SIFT) `

- Improving 3D Face Details based on Normal Map of Hetero-source Images

(CVPRW2014, Yang, C., Chen, J., Su, N., & Su, G. , Tsinghua University)

- Total Moving Face Reconstruction

(LNCS2014, Suwajanakorn S, Kemelmacher-Shlizerman I, Seitz S M. , Washington)

`video(collections), template, average shape, pose estimation, 3d

flow(correspondence), refinement, high-detail `

- FaceWarehouse: a 3D Facial Expression Database for Visual Computing

(VCG2014, Cao, C., Weng, Y., Zhou, S., Tong, Y., & Zhou, K., Zhejiang) [[data]]()

- Intrinsic Face Image Decomposition with Human Face Priors

(ECCV2014, Li C, Zhou K, Lin S , Zhejiang)

- Fitting 3D Morphable Models using Local Features

(ICIP2015, Huber, P., Feng, Z. H., Christmas, W., Kittler, J., & Ratsch, M,
Surrey)

`sparse correspondence, 3dmm, regression`

- What Makes Tom Hanks Look Like Tom Hanks

(ICCV2015, Suwajanakorn S, Seitz S M, Kemelmacher-Shlizerman I. , Washington)

`collection, template, average model, 3D flow, correspondence, deformation

vector, TPS, expression sililarity weighted, high-frequency details, Laplacian
pyramid `

- Unconstrained Realtime Facial Performance Capture

(CVPR2015, Hsieh, P. L., Ma, C., Yu, J., & Li, H. , USC)

`video,image collections, occlusion, segmentation, landmarks `

- Unconstrained 3D Face Reconstruction

(CVPR2015, Roth, J., Tong, Y., & Liu, X, MSU)

`collection, sparse correspondence(landmarks), template, photometric stereo(SVD),

matrix completion, `

- Pose-Invariant 3D Face Alignment

(ICCV2015, Jourabloo, A., & Liu, X., MSU)

`alignment, dense correspondence, visibility, cascaded regressor, 3DPDM. `

- Discriminative 3D Morphable Model Fitting

(CVPR2015)

#### 2016

**#CVPR**

* Large-pose Face Alignment via CNN-based Dense 3D Model Fitting

(CVPR2016, Jourabloo, A., & Liu, X., MSU)

`alignment, 3dmm `

* Automated 3D Face Reconstruction from Multiple Images using Quality Measures

(CVPR2016, Piotraschke, M., & Blanz, V , Siegen)

* A Robust Multilinear Model Learning Framework for 3D Faces

(CVPR2016, Bolkart, T., & Wuhrer, S., Saarland)

* Face Alignment Across Large Poses: A 3D Solution

(CVPR2016, Zhu, X., Lei, Z., Liu, X., Shi, H., & Li, S. Z. , MSU, CASIA)
* Adaptive 3D Face Reconstruction from Unconstrained Photo Collections

(CVPR2016, Roth, J., Tong, Y., & Liu, X, MSU)

`landmarks, 3dmm, coarse-to-fine,photometric stereo, time: 7 minutes `

* Augmented Blendshapes for Real-time Simultaneous 3D Head Modeling and Facial

Motion Capture

(CVPR2016, Thomas, D., & Taniguchi, R. I. , Kyushu University)

* A 3D Morphable Model learnt from 10,000 faces

(CVPR2016, Booth, J., Roussos, A., Zafeiriou, S., Ponniah, A., & Dunaway, D.,
ICL)

**#ECCV**

* Joint Face Alignment and 3D Face Reconstruction

(ECCV2016, Liu, F., Zeng, D., Zhao, Q., & Liu, X, MSU, Sichuan U)

`alignment, landmarks`

* Real-Time Facial Segmentation and Performance Capture from RGB Input

(ECCV2016, Saito, S., Li, T., & Li, H. , USC)

`occlusions, tracking`

**#Others**

- 3D Face Reconstruction by Learning from Synthetic Data

(3DV2016, Richardson, E., Sela, M., & Kimmel, R, IIT)

`3dmm, cnn, regress 3dmm parameters, sfs`

- Face Reconstruction on Mobile Devices Using a Height Map Shape Model and Fast
Regularization

(3DV2016, Maninchedda, F., Häne, C., Oswald, M. R., & Pollefeys, M., ETH)

- A Multiresolution 3D Morphable Face Model and Fitting Framework [[code]]

(https://github.com/patrikhuber/eos)

(IJCV2016, Huber, P., Hu, G., Tena, R., Mortazavian, P., Koppen, P., Christmas,
W. J., ... & Kittler, J. ，Surrey)

- Rapid Photorealistic Blendshape Modeling from RGB-D Sensors

(2016, USC)

- 3D Face Reconstruction with Region Based Best Fit Blending Using Mobile Phone for
Virtual Reality Based Social Media

(2016, Anbarjafari, G., Haamer, R. E., Lusi, I., Tikk, T., & Valgma, L. , Turkey)

`landmarks, uv texture, region`

#### 2017

**#CVPR**

* 3D Face Morphable Models “In-the-Wild”

(CVPR2017, Booth, J., Antonakos, E., Ploumpis, S., Trigeorgis, G., Panagakis, Y.,
& Zafeiriou, S., ICL)

`3dmm, register, UV`

* Face Normals “in-the-wild” using Fully Convolutional Networks

(CVPR2017, Trigeorgis, G., Snape, P., Kokkinos, I., & Zafeiriou, S. , ICL)

* Regressing Robust and Discriminative 3D Morphable Models with a very Deep Neural
Network

(CVPR2017, Tran, A. T., Hassner, T., Masi, I., & Medioni, G. , USC)

* Fast 3D Reconstruction of Faces with Glasses

(CVPR2017, Maninchedda, F., Oswald, M. R., & Pollefeys, M., ETH)

* DenseReg: Fully Convolutional Dense Shape Regression In-the-Wild

(CVPR2017, Güler, R. A., Trigeorgis, G., Antonakos, E., Snape, P., Zafeiriou, S.,
& Kokkinos, I. , ICL)

`dense correspondence, uv`

* Learning Detailed Face Reconstruction from a Single Image

(CVPR2017, Richardson, E., Sela, M., Or-El, R., & Kimmel, R. , Washington)

* End-to-end 3D face reconstruction with deep neural networks

(CVPR2017, Dou, P., Shah, S. K., & Kakadiaris, I. A., UofHouston)

`3dmm, dl, directly learn 3dmm parameters`

* A Generative Model for Depth-based Robust 3D Facial Pose Tracking

(CVPR2017, Cai, L. S. J., Pavlovic, T. J. C. V., & Ngan, K. N. , CUHK)

`occlusions`

**#ICCV**

* 3D Morphable Models as Spatial Transformer Networks

(ICCV2017, Bas, A., Huber, P., Smith, W. A., Awais, M., & Kittler, J. , York,
Surrey )

`dl, cnn, uv texture, landmarks, stn, 3dmm`

* Faster Than Real-time Facial Alignment: A 3D Spatial Transformer Network Approach
in Unconstrained Poses

(ICCV2017, Bhagavatula, C., Zhu, C., Luu, K., & Savvides, M., CMU)

* Pose-Invariant Face Alignment with a Single CNN

(ICCV2017, Jourabloo, A., Ye, M., Liu, X., & Ren, L. , MSU)

`alignment, 3dmm, dl, cnn`

* Large Pose 3D Face Reconstruction from a Single Image via Direct Volumetric CNN
Regression [code](https://github.com/AaronJackson/vrn)

(ICCV2017, Jackson, A. S., Bulat, A., Argyriou, V., & Tzimiropoulos, G. ,
Nottingham)

`end-to-end, 3dmm, dl, cnn, landmarks, voxel `

* Unrestricted Facial Geometry Reconstruction Using Image-to-Image Translation

(ICCV2017, Sela, M., Richardson, E., & Kimmel, R. , IIT)

* MoFA: Model-based Deep Convolutional Face Autoencoder for Unsupervised Monocular

Reconstruction

(ICCV2017, Tewari, A., Zollhöfer, M., Kim, H., Garrido, P., Bernard, F., Pérez,
P., & Theobalt, C. , MPI)

* Dense Face Alignment [code](http://cvlab.cse.msu.edu/project-pifa.html)

(ICCVW2017, Liu, Y., Jourabloo, A., Ren, W., & Liu, X. , MSU)

* Realtime Dynamic 3D Facial Reconstruction for Monocular Video In-the-Wild

(ICCVW2017)

* Learning Dense Facial Correspondences in Unconstrained Images

(ICCV2017, Yu, R., Saito, S., Li, H., Ceylan, D., & Li, H.  , USC)

`dense correspondence`

**#others**

* Large Scale 3D Morphable Models [[data](LSFM)](https://faces.cs.unibas.ch/bfm/)

(IJCV2017, Booth, J., Roussos, A., Ponniah, A., Dunaway, D., & Zafeiriou, S. ,
ICL)

`for alignment, template, cnn, dl, sparse correspondence, landmarks, tps warping
`

* What does 2D geometric information really tell us about 3D face shape? (2017,
Bas, A., & Smith, W. A )

`shape from landmarks, shape from contours`

* Pix2Face: Direct 3D Face Model Estimation

 (2017)

`dense correspondence, 3dmm`

* 3D Face Reconstruction with Geometry Details from a Single Image

(TIP2017, Jiang, L., Zhang, J., Deng, B., Li, H., & Liu, L. , USC, USTC)

`coarse-to-fine, landmarks, corrective deformatio, sfs `

#### 2018

**#CVPR**

* Unsupervised Training for 3D Morphable Model Regression

[code](https://github.com/google/tf_mesh_renderer)

(CVPR2018, Genova, K., Cole, F., Maschinot, A., Sarna, A., Vlasic, D., & Freeman,
W. T , Google)

- 4DFAB: A Large Scale 4D Database for Facial Expression Analysis and Biometric
Applications [data]( )

(CVPR2018, Cheng, S., Kotsia, I., Pantic, M., & Zafeiriou, S., ICL)

- Sparse Photometric 3D Face Reconstruction Guided by Morphable Models

(CVPR2018, Cao, X., Chen, Z., Chen, A., Chen, X., Li, S., & Yu, J.  ,
shanghaitech)

`5 input images, 3dmm, shadow processing, light calibration, photometric stereo,

denoising `

- Disentangling Features in 3D Face Shapes for Joint Face Reconstruction and

Recognition

(CVPR2018, Liu, F., Zhu, R., Zeng, D., Zhao, Q., & Liu, X. , MSU, Sichuan)

- Mesoscopic Facial Geometry Inference Using Deep Neural Networks

(CVPR2018, Hao Li, USC)

`high-detail, dl, scan, uv texture, displacement `

- Self-supervised Multi-level Face Model Learning for Monocular Reconstruction at

over 250 Hz

(CVPR2018, Tewari, A., Zollhöfer, M., Garrido, P., Bernard, F., Kim, H., Pérez,
P., & Theobalt, C., MPI)

- SfSNet : Learning Shape, Reflectance and Illuminance of Faces in the Wild

(CVPR2018, Sengupta, S., Kanazawa, A., Castillo, C. D., & Jacobs, D.  , Maryland,
UCB )

- Probabilistic Joint Face-Skull Modelling for Facial Reconstruction

 (CVPR2018, Madsen, D., Lüthi, M., Schneider, A., & Vetter, T. , Basel)

- Alive Caricature from 2D to 3D

(CVPR2018, Wu, Q., Zhang, J., Lai, Y. K., Zheng, J., & Cai, J, USTC)

- Nonlinear 3D Face Morphable Model

(CVPR2018, Tran, L., & Liu, X. , MSU)

- InverseFaceNet: Deep Monocular Inverse Face Rendering

(CVPR2018, Kim, H., Zollhöfer, M., Tewari, A., Thies, J., Richardt, C., &
Theobalt, C. , MPI)

`Self-Supervised Bootstrapping`

- Extreme 3D Face Reconstruction: Looking Past Occlusions

(CVPR2018, Tran, A. T., Hassner, T., Masi, I., Paz, E., Nirkin, Y., & Medioni, G.
, USC)

- Total Capture: A 3D Deformation Model for Tracking Faces, Hands, and Bodies

(CVPR2018, best student paper, Joo, H., Simon, T., & Sheikh, Y. , CMU)

- Modeling Facial Geometry using Compositional VAEs

(CVPR2018, Bagautdinov, T., Wu, C., Saragih, J., Fua, P., & Sheikh, Y., EPEL, FRL
)

**#ECCV**

* 3D Face Reconstruction from Light Field Images: A Model-free Approach

(ECCV2018, Feng, M., Gilani, S. Z., Wang, Y., & Mian, A., Western Australia,
Hunan)

`epopolar plane images`

* Generating 3D Faces using Convolutional Mesh Autoencoders

[[code]](https://github.com/anuragranj/coma)

(ECCV2018, Ranjan, A., Bolkart, T., Sanyal, S., & Black, M. J., MPI)

* Joint 3D Face Reconstruction and Dense Alignment with Position Map Regression
Network [[code]](https://github.com/YadiraF/PRNet)

(ECCV2018, Feng, Y., Wu, F., Shao, X., Wang, Y., & Zhou, X., SJTU)

**#others**

* Morphable Face Models - An Open Framework

(FG2018, Gerig, T., Morel-Forster, A., Blumer, C., Egger, B., Luthi, M.,
Schönborn, S., & Vetter, T. , Basel)

* CNN-based Real-time Dense Face Reconstruction with Inverse-rendered Photo-

realistic Face Images [[data&code]](https://github.com/Juyong/3DFace)

(TPAMI2018, Yudong Guo, Juyong Zhang, Jianfei Cai, Boyi Jiang, Jianmin Zheng,
USTC)

* Multilinear Autoencoder for 3D Face Model Learning(WACV 2018, Universite

Grenoble Alpes (LJK), France)

`3d scan to registered mesh. dl. height map `

* On Face Segmentation, Face Swapping, and Face Perception(AFGR,2018, HT)

* Evaluation of Dense 3D Reconstruction from 2D Face Images in the Wild(FG2018)

[data]( )

**#arxiv**

* Joint Face Alignment and 3D Face Reconstruction with Application to Face

Recognition(2017, Feng Liu, Xiaoming Liu)

* Convolutional Point-set Representation: A Convolutional Bridge Between a Densely

Annotated Image and 3D Face Alignment(20180317)

* Unsupervised Depth Estimation, 3D Face Rotation and Replacement(20180325)

### Production-level Reconstruction

> more in computer graphics

- High-Quality Single-Shot Capture of Facial Geometry(TOG2010, ETHZ, Disney)

`cg, high-detail,stereo system, calibration, surface refinement, normal

direction, mesoscopic `

- Multiview Face Capture using Polarized Spherical Gradient Illumination(TOG2011)

`image collecitons`

- High-Quality Passive Facial Performance Capture using Anchor Frames(SIGGRAPH2011,

ETHZ, Disney)

`cg, stereo,anchor frame, tracking, mesh progration, physical movement, motion

estimation, refinement `

- Lightweight binocular facial perfor- mance capture under uncontrolled

lighting(TOG2012, MPI)

`cg, high-detail, stereo, template,flow,data term, geometry term, smoothness

term, mesh tracking, motion refinement, shape refinement, sfs `

- Reconstructing Detailed Dynamic Face Geometry from Monocular Video(TOG2013, MPI)

`cg, dynamic, high-detail, blend model, sparse correspondence, dense

correspondence(appearance matching, LBP), pose estimation , shape refinement, sfs
`
- 3D Shape Regression for Real-time Facial Animation(TOG2013, ZJU)

- Real-Time High-Fidelity Facial Performance Capture (TOG2015, ZJU)

`cg, landmarks, optical flow, train a regressor to learn detail `

- Dynamic 3D Avatar Creation from Hand-held Video Input(TOG2015, EPEL)

`cg, dynamic, mobile, high-detail, avatar, 3dmm,sparse correspondence, eye mesh,

tracking, refinement, sfs, detail map `

- Reconstruction of Personalized 3D Face Rigs from Monocular Video(TOG2016, MPI)

`parametric shape prior, coarse-scale reconstruction, fine-scale(sfs), coase-

>medium->fine, 3dmm, corrective `

- Production-Level Facial Performance Capture Using Deep Convolutional Neural

Networks（ASCA2017, USC)

- Multi-View Stereo on Consistent Face Topology(EG2017, USC)

`cg, high-detail, landmarks, template, pose estimation, refinement`

- Avatar Digitization From a Single Image For Real-Time Rendering(SIGGRAPH Asia

2017, USC)

`cg, avatar, segmentation, head, hair, 3DMM, landmarks, texture completion `

- Learning a model of facial shape and expression from 4D scans(TOG2017, USC, MPI)

- DeepSketch2Face: A Deep Learning Based Sketching System for 3D Face and

Caricature Modeling(SIGGRAPH2017)

- High-Fidelity Facial Reflectance and Geometry Inference From an Unconstrained

Image(SIGGRAPH2018, USC)

### Texture

> 3D-aid texture generation/ UV texture completion

> Keys: GAN

- Face Synthesis from Facial Identity Features(CVPR2017, google)

`3dmm, dl, landmarks`

- Photorealistic Facial Texture Inference Using Deep Neural Networks(CVPR2017, Hao

Li, USC)

`texture completion`

- UV-GAN: Adversarial Facial UV Map Completion for Pose-invariant Face

Recognition(CVPR2018, SZ, ICL)

`gan, 3dmm, uv texture completion`

- Multi-Attribute Robust Component Analysis for Facial UV Maps(2017, SZ, ICL)

- Realistic Dynamic Facial Textures from a Single Image using GANs(CVPR2017, Hao
Li, USC, DeepMind)

- Semi-supervised Adversarial Learning to Generate Photorealistic Face Images of

New Identities from 3D Morphable Model(2018)

- Side Information for Face Completion: a Robust PCA Approach(20180120, SZ, ICL)

### Transfer&Reenactment(Applications)

- Face Transfer with Multilinear Models (SIGGRAPH2005)

`Cartesian product(ID x EX x VI)`

- Online Modeling For Realtime Facial Animation(TOG2013)

`rgbd, blendshape, corrective field `

- Displaced Dynamic Expression Regression for Real-time Facial Tracking and

Animation(SIGGRAPH2014)

- Real-time Expression Transfer for Facial Reenactment(SIGGRAPH AISA 2015)

- Face2Face: Real-time Face Capture and Reenactment of RGB Videos(CVPR2016)

`capture, transfer, 3dmm, landmarks, texture, expression, mouth retrieval `

- Synthesizing Obama: Learning Lip Sync from Audio(SIGGRAPH2017)

- Deep Video Portrait(SIGGRAPH2018)

- HeadOn: Real-time Reenactment of Human Portrait Videos(SIGGRAPH2018)

### 3D-aid 2D face recognition

- Tom-vs-Pete Classifiers and Identity-Preserving Alignment for Face

Verification(ECCV2012, Columbia University)

- Face Recognition from a Single Training Image under Arbitrary Unknown Lighting
Using Spherical Harmonics(PAMI2006)

- 3D-aided face recognition robust to expression and pose variations (CVPR2014)

- Effective 3D based Frontalization for Unconstrained Face Recognition(ICPR2016,

MICC, Florence)

- Effective Face Frontalization in Unconstrained Images(CVPR2015, TH, Israel)

- Do We Really Need to Collect Millions of Faces for Effective Face

Recognition(ECCV2016, TH, USC, Israel)

- High-Fidelity Pose and Expression Normalization for Face Recognition in the

Wild(CVPR2015)

- When 3D-Aided 2D Face Recognition Meets Deep Learning: An extended UR2D for Pose-
Invariant Face Recognition(2017)

- Towards Large-Pose Face Frontalization in the Wild

- Fully Automatic Pose-Invariant Face Recognition via 3D Pose Normalization

(ICCV2011, Cambridge, MA, USA)

### 3D face recognition

- Face Identification across Different Poses and Illuminations with a 3D Morphable

Model(Automatic Face and Gesture Recognition2002, VB&TV)

- Preliminary Face Recognition Grand Challenge Results(2006)

- expression Invariant 3D Face Recognition with a Morphable Model(FG2008, TV,

Basel)

- Bosphorus Database for 3D Face Analysis(2008)[data]()

- Robust Learning from Normals for 3D face recognition(ECCV2012, SZ, ICL)

- Static and dynamic 3D facial expression recognition: A comprehensive

survey(IVC2012, SZ, LijunYin)

- Deep 3D Face Identification(2017, USC)

- Robust Face Recognition with Deeply Normalized Depth Images (2018)

`depth image(front&neural)`

- Learning from Millions of 3D Scans for Large-scale 3D Face Recognition(CVPR2018,

Western Australia)