Label Distribution-Based Facial Attractiveness Computation by Deep Residual Learning

Yang Yu Fan, Shu Liu, Bo Li, Zhe Guo, Ashok K Samal, Jun Wan, Stan Z. Li

Research output: Contribution to journalArticle

3 Citations (Scopus)

Abstract

Two key challenges lie in the facial attractiveness computation research: the lack of discriminative face representations, and the scarcity of sufficient and complete training data. Motivated by the recent promising work in face recognition using deep neural networks to learn effective features, the first challenge is expected to be addressed from a deep learning point of view. A very deep residual network is utilized to enable automatic learning of hierarchical aesthetics representation. The inspiration to deal with the second challenge comes from the natural representation of the training data, whereby each training face can be associated with a label (score) distribution given by human raters rather than a single label (average score). This paper, therefore, recasts facial attractiveness computation as a label distribution learning problem. Integrating these two ideas, an end-to-end attractiveness learning framework is established. We also perform feature-level fusion by incorporating the low-level geometric features to further improve the computational performance. Extensive experiments are conducted on a standard benchmark, the SCUT-FBP dataset, where our approach shows significant advantages over other state-of-the-art work.

Original languageEnglish (US)
JournalIEEE Transactions on Multimedia
DOIs
StateAccepted/In press - Dec 5 2017

Fingerprint

Labels
Face recognition
Fusion reactions
Experiments
Deep neural networks
Deep learning

Keywords

  • Computational modeling
  • Computer architecture
  • deep residual network
  • Face
  • Facial attractiveness computation
  • Feature extraction
  • feature fusion
  • Image color analysis
  • label distribution
  • SCUT-FBP
  • Training data

ASJC Scopus subject areas

  • Signal Processing
  • Media Technology
  • Computer Science Applications
  • Electrical and Electronic Engineering

Cite this

Label Distribution-Based Facial Attractiveness Computation by Deep Residual Learning. / Fan, Yang Yu; Liu, Shu; Li, Bo; Guo, Zhe; Samal, Ashok K; Wan, Jun; Li, Stan Z.

In: IEEE Transactions on Multimedia, 05.12.2017.

Research output: Contribution to journalArticle

@article{be7d95ebcb804e15895fc8c7c3f9311d,
title = "Label Distribution-Based Facial Attractiveness Computation by Deep Residual Learning",
abstract = "Two key challenges lie in the facial attractiveness computation research: the lack of discriminative face representations, and the scarcity of sufficient and complete training data. Motivated by the recent promising work in face recognition using deep neural networks to learn effective features, the first challenge is expected to be addressed from a deep learning point of view. A very deep residual network is utilized to enable automatic learning of hierarchical aesthetics representation. The inspiration to deal with the second challenge comes from the natural representation of the training data, whereby each training face can be associated with a label (score) distribution given by human raters rather than a single label (average score). This paper, therefore, recasts facial attractiveness computation as a label distribution learning problem. Integrating these two ideas, an end-to-end attractiveness learning framework is established. We also perform feature-level fusion by incorporating the low-level geometric features to further improve the computational performance. Extensive experiments are conducted on a standard benchmark, the SCUT-FBP dataset, where our approach shows significant advantages over other state-of-the-art work.",
keywords = "Computational modeling, Computer architecture, deep residual network, Face, Facial attractiveness computation, Feature extraction, feature fusion, Image color analysis, label distribution, SCUT-FBP, Training data",
author = "Fan, {Yang Yu} and Shu Liu and Bo Li and Zhe Guo and Samal, {Ashok K} and Jun Wan and Li, {Stan Z.}",
year = "2017",
month = "12",
day = "5",
doi = "10.1109/TMM.2017.2780762",
language = "English (US)",
journal = "IEEE Transactions on Multimedia",
issn = "1520-9210",
publisher = "Institute of Electrical and Electronics Engineers Inc.",

}

TY - JOUR

T1 - Label Distribution-Based Facial Attractiveness Computation by Deep Residual Learning

AU - Fan, Yang Yu

AU - Liu, Shu

AU - Li, Bo

AU - Guo, Zhe

AU - Samal, Ashok K

AU - Wan, Jun

AU - Li, Stan Z.

PY - 2017/12/5

Y1 - 2017/12/5

N2 - Two key challenges lie in the facial attractiveness computation research: the lack of discriminative face representations, and the scarcity of sufficient and complete training data. Motivated by the recent promising work in face recognition using deep neural networks to learn effective features, the first challenge is expected to be addressed from a deep learning point of view. A very deep residual network is utilized to enable automatic learning of hierarchical aesthetics representation. The inspiration to deal with the second challenge comes from the natural representation of the training data, whereby each training face can be associated with a label (score) distribution given by human raters rather than a single label (average score). This paper, therefore, recasts facial attractiveness computation as a label distribution learning problem. Integrating these two ideas, an end-to-end attractiveness learning framework is established. We also perform feature-level fusion by incorporating the low-level geometric features to further improve the computational performance. Extensive experiments are conducted on a standard benchmark, the SCUT-FBP dataset, where our approach shows significant advantages over other state-of-the-art work.

AB - Two key challenges lie in the facial attractiveness computation research: the lack of discriminative face representations, and the scarcity of sufficient and complete training data. Motivated by the recent promising work in face recognition using deep neural networks to learn effective features, the first challenge is expected to be addressed from a deep learning point of view. A very deep residual network is utilized to enable automatic learning of hierarchical aesthetics representation. The inspiration to deal with the second challenge comes from the natural representation of the training data, whereby each training face can be associated with a label (score) distribution given by human raters rather than a single label (average score). This paper, therefore, recasts facial attractiveness computation as a label distribution learning problem. Integrating these two ideas, an end-to-end attractiveness learning framework is established. We also perform feature-level fusion by incorporating the low-level geometric features to further improve the computational performance. Extensive experiments are conducted on a standard benchmark, the SCUT-FBP dataset, where our approach shows significant advantages over other state-of-the-art work.

KW - Computational modeling

KW - Computer architecture

KW - deep residual network

KW - Face

KW - Facial attractiveness computation

KW - Feature extraction

KW - feature fusion

KW - Image color analysis

KW - label distribution

KW - SCUT-FBP

KW - Training data

UR - http://www.scopus.com/inward/record.url?scp=85038352518&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85038352518&partnerID=8YFLogxK

U2 - 10.1109/TMM.2017.2780762

DO - 10.1109/TMM.2017.2780762

M3 - Article

JO - IEEE Transactions on Multimedia

JF - IEEE Transactions on Multimedia

SN - 1520-9210

ER -