Criminal tendency detection from facial images and the gender bias effect

Mahdi Hashemi, Margeret Hall

Research output: Contribution to journalArticle

Abstract

Explosive performance and memory space growth in computing machines, along with recent specialization of deep learning models have radically boosted the role of images in semantic pattern recognition. In the same way that a textual post on social media reveals individual characteristics of its author, facial images may manifest some personality traits. This work is the first milestone in our attempt to infer personality traits from facial images. With this ultimate goal in mind, here we explore a new level of image understanding, inferring criminal tendency from facial images via deep learning. In particular, two deep learning models, including a standard feedforward neural network (SNN) and a convolutional neural network (CNN) are applied to discriminate criminal and non-criminal facial images. Confusion matrix and training and test accuracies are reported for both models, using tenfold cross-validation on a set of 10,000 facial images. The CNN was more consistent than the SNN in learning to reach its best test accuracy, which was 8% higher than the SNN’s test accuracy. Next, to explore the classifier’s hypothetical bias due to gender, we controlled for gender by applying only male facial images. No meaningful discrepancies in classification accuracies or learning consistencies were observed, suggesting little to no gender bias in the classifier. Finally, dissecting and visualizing convolutional layers in CNN showed that the shape of the face, eyebrows, top of the eye, pupils, nostrils, and lips are taken advantage of by CNN in order to classify the two sets of images.

Original languageEnglish (US)
Article number2
JournalJournal of Big Data
Volume7
Issue number1
DOIs
StatePublished - Dec 1 2020

Fingerprint

Neural networks
Feedforward neural networks
Classifiers
Image understanding
Pattern recognition
Semantics
Data storage equipment
Gender bias
Deep learning

Keywords

  • Convolutional neural network
  • Deep learning
  • Facial images
  • Image classification
  • Machine learning
  • Personality traits

ASJC Scopus subject areas

  • Information Systems
  • Hardware and Architecture
  • Computer Networks and Communications
  • Information Systems and Management

Cite this

Criminal tendency detection from facial images and the gender bias effect. / Hashemi, Mahdi; Hall, Margeret.

In: Journal of Big Data, Vol. 7, No. 1, 2, 01.12.2020.

Research output: Contribution to journalArticle

@article{92e2b5fc5c544e5bbb3b82806eb7702b,
title = "Criminal tendency detection from facial images and the gender bias effect",
abstract = "Explosive performance and memory space growth in computing machines, along with recent specialization of deep learning models have radically boosted the role of images in semantic pattern recognition. In the same way that a textual post on social media reveals individual characteristics of its author, facial images may manifest some personality traits. This work is the first milestone in our attempt to infer personality traits from facial images. With this ultimate goal in mind, here we explore a new level of image understanding, inferring criminal tendency from facial images via deep learning. In particular, two deep learning models, including a standard feedforward neural network (SNN) and a convolutional neural network (CNN) are applied to discriminate criminal and non-criminal facial images. Confusion matrix and training and test accuracies are reported for both models, using tenfold cross-validation on a set of 10,000 facial images. The CNN was more consistent than the SNN in learning to reach its best test accuracy, which was 8{\%} higher than the SNN’s test accuracy. Next, to explore the classifier’s hypothetical bias due to gender, we controlled for gender by applying only male facial images. No meaningful discrepancies in classification accuracies or learning consistencies were observed, suggesting little to no gender bias in the classifier. Finally, dissecting and visualizing convolutional layers in CNN showed that the shape of the face, eyebrows, top of the eye, pupils, nostrils, and lips are taken advantage of by CNN in order to classify the two sets of images.",
keywords = "Convolutional neural network, Deep learning, Facial images, Image classification, Machine learning, Personality traits",
author = "Mahdi Hashemi and Margeret Hall",
year = "2020",
month = "12",
day = "1",
doi = "10.1186/s40537-019-0282-4",
language = "English (US)",
volume = "7",
journal = "Journal of Big Data",
issn = "2196-1115",
publisher = "Springer Open",
number = "1",

}

TY - JOUR

T1 - Criminal tendency detection from facial images and the gender bias effect

AU - Hashemi, Mahdi

AU - Hall, Margeret

PY - 2020/12/1

Y1 - 2020/12/1

N2 - Explosive performance and memory space growth in computing machines, along with recent specialization of deep learning models have radically boosted the role of images in semantic pattern recognition. In the same way that a textual post on social media reveals individual characteristics of its author, facial images may manifest some personality traits. This work is the first milestone in our attempt to infer personality traits from facial images. With this ultimate goal in mind, here we explore a new level of image understanding, inferring criminal tendency from facial images via deep learning. In particular, two deep learning models, including a standard feedforward neural network (SNN) and a convolutional neural network (CNN) are applied to discriminate criminal and non-criminal facial images. Confusion matrix and training and test accuracies are reported for both models, using tenfold cross-validation on a set of 10,000 facial images. The CNN was more consistent than the SNN in learning to reach its best test accuracy, which was 8% higher than the SNN’s test accuracy. Next, to explore the classifier’s hypothetical bias due to gender, we controlled for gender by applying only male facial images. No meaningful discrepancies in classification accuracies or learning consistencies were observed, suggesting little to no gender bias in the classifier. Finally, dissecting and visualizing convolutional layers in CNN showed that the shape of the face, eyebrows, top of the eye, pupils, nostrils, and lips are taken advantage of by CNN in order to classify the two sets of images.

AB - Explosive performance and memory space growth in computing machines, along with recent specialization of deep learning models have radically boosted the role of images in semantic pattern recognition. In the same way that a textual post on social media reveals individual characteristics of its author, facial images may manifest some personality traits. This work is the first milestone in our attempt to infer personality traits from facial images. With this ultimate goal in mind, here we explore a new level of image understanding, inferring criminal tendency from facial images via deep learning. In particular, two deep learning models, including a standard feedforward neural network (SNN) and a convolutional neural network (CNN) are applied to discriminate criminal and non-criminal facial images. Confusion matrix and training and test accuracies are reported for both models, using tenfold cross-validation on a set of 10,000 facial images. The CNN was more consistent than the SNN in learning to reach its best test accuracy, which was 8% higher than the SNN’s test accuracy. Next, to explore the classifier’s hypothetical bias due to gender, we controlled for gender by applying only male facial images. No meaningful discrepancies in classification accuracies or learning consistencies were observed, suggesting little to no gender bias in the classifier. Finally, dissecting and visualizing convolutional layers in CNN showed that the shape of the face, eyebrows, top of the eye, pupils, nostrils, and lips are taken advantage of by CNN in order to classify the two sets of images.

KW - Convolutional neural network

KW - Deep learning

KW - Facial images

KW - Image classification

KW - Machine learning

KW - Personality traits

UR - http://www.scopus.com/inward/record.url?scp=85077534594&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85077534594&partnerID=8YFLogxK

U2 - 10.1186/s40537-019-0282-4

DO - 10.1186/s40537-019-0282-4

M3 - Article

AN - SCOPUS:85077534594

VL - 7

JO - Journal of Big Data

JF - Journal of Big Data

SN - 2196-1115

IS - 1

M1 - 2

ER -