Detecting and classifying online dark visual propaganda

M. Hashemi, Margeret Hall

Research output: Contribution to journalArticle

4 Citations (Scopus)

Abstract

The staggering increase in the amount of information on the World Wide Web (referred to as Web) has made Web page classification essential to retrieve useful information while filtering out unwanted, futile, or harmful contents. This massive information-sharing platform is occasionally abused for propagating extreme and radical ideologies and posing threats to national security and citizens. Detecting the so called dark material has gained more impetus following the recent outbreak of extremist groups and radical ideologies across the Web. The goal of this project, being the first of its own, is to surveil online social networks (OSN) and Web for real-time detection of visual propaganda by violent extremist organizations (VEOs). This is valuable not only for flagging and removing such content from OSN and Web, but also to provide military insight and narrative context inside VEOs. Visual propaganda by VEOs are not only detected, but also further classified based on the type of VEO and focus or intent of the image into hard propaganda, soft propaganda, symbolic propaganda, landscape, and organizational communications. Over 1.2 million images were automatically collected from suspicious OSN accounts and Web pages over a course of four years. Out of which, 120,000 images were manually classified to provide the training data for a convolutional neural network. An overall generalization accuracy of 97.02% and F1 of 97.89% were achieved for a binary classification or mere detection of visual VEO propaganda and an overall generalization accuracy of 86.08% and F1¯ of 85.76% for an eight-way classification based on the intent of the image.

Original languageEnglish (US)
Pages (from-to)95-105
Number of pages11
JournalImage and Vision Computing
Volume89
DOIs
StatePublished - Sep 2019

Fingerprint

World Wide Web
Websites
Information filtering
National security
Neural networks
Communication

Keywords

  • Convolutional neural networks
  • Dark Web
  • Deep learning
  • Image classification
  • Machine learning
  • Violent extremist organizations

ASJC Scopus subject areas

  • Signal Processing
  • Computer Vision and Pattern Recognition

Cite this

Detecting and classifying online dark visual propaganda. / Hashemi, M.; Hall, Margeret.

In: Image and Vision Computing, Vol. 89, 09.2019, p. 95-105.

Research output: Contribution to journalArticle

@article{0c3c812bab3147c18822a31dcafd8b69,
title = "Detecting and classifying online dark visual propaganda",
abstract = "The staggering increase in the amount of information on the World Wide Web (referred to as Web) has made Web page classification essential to retrieve useful information while filtering out unwanted, futile, or harmful contents. This massive information-sharing platform is occasionally abused for propagating extreme and radical ideologies and posing threats to national security and citizens. Detecting the so called dark material has gained more impetus following the recent outbreak of extremist groups and radical ideologies across the Web. The goal of this project, being the first of its own, is to surveil online social networks (OSN) and Web for real-time detection of visual propaganda by violent extremist organizations (VEOs). This is valuable not only for flagging and removing such content from OSN and Web, but also to provide military insight and narrative context inside VEOs. Visual propaganda by VEOs are not only detected, but also further classified based on the type of VEO and focus or intent of the image into hard propaganda, soft propaganda, symbolic propaganda, landscape, and organizational communications. Over 1.2 million images were automatically collected from suspicious OSN accounts and Web pages over a course of four years. Out of which, 120,000 images were manually classified to provide the training data for a convolutional neural network. An overall generalization accuracy of 97.02{\%} and F1 of 97.89{\%} were achieved for a binary classification or mere detection of visual VEO propaganda and an overall generalization accuracy of 86.08{\%} and F1¯ of 85.76{\%} for an eight-way classification based on the intent of the image.",
keywords = "Convolutional neural networks, Dark Web, Deep learning, Image classification, Machine learning, Violent extremist organizations",
author = "M. Hashemi and Margeret Hall",
year = "2019",
month = "9",
doi = "10.1016/j.imavis.2019.06.001",
language = "English (US)",
volume = "89",
pages = "95--105",
journal = "Image and Vision Computing",
issn = "0262-8856",
publisher = "Elsevier Limited",

}

TY - JOUR

T1 - Detecting and classifying online dark visual propaganda

AU - Hashemi, M.

AU - Hall, Margeret

PY - 2019/9

Y1 - 2019/9

N2 - The staggering increase in the amount of information on the World Wide Web (referred to as Web) has made Web page classification essential to retrieve useful information while filtering out unwanted, futile, or harmful contents. This massive information-sharing platform is occasionally abused for propagating extreme and radical ideologies and posing threats to national security and citizens. Detecting the so called dark material has gained more impetus following the recent outbreak of extremist groups and radical ideologies across the Web. The goal of this project, being the first of its own, is to surveil online social networks (OSN) and Web for real-time detection of visual propaganda by violent extremist organizations (VEOs). This is valuable not only for flagging and removing such content from OSN and Web, but also to provide military insight and narrative context inside VEOs. Visual propaganda by VEOs are not only detected, but also further classified based on the type of VEO and focus or intent of the image into hard propaganda, soft propaganda, symbolic propaganda, landscape, and organizational communications. Over 1.2 million images were automatically collected from suspicious OSN accounts and Web pages over a course of four years. Out of which, 120,000 images were manually classified to provide the training data for a convolutional neural network. An overall generalization accuracy of 97.02% and F1 of 97.89% were achieved for a binary classification or mere detection of visual VEO propaganda and an overall generalization accuracy of 86.08% and F1¯ of 85.76% for an eight-way classification based on the intent of the image.

AB - The staggering increase in the amount of information on the World Wide Web (referred to as Web) has made Web page classification essential to retrieve useful information while filtering out unwanted, futile, or harmful contents. This massive information-sharing platform is occasionally abused for propagating extreme and radical ideologies and posing threats to national security and citizens. Detecting the so called dark material has gained more impetus following the recent outbreak of extremist groups and radical ideologies across the Web. The goal of this project, being the first of its own, is to surveil online social networks (OSN) and Web for real-time detection of visual propaganda by violent extremist organizations (VEOs). This is valuable not only for flagging and removing such content from OSN and Web, but also to provide military insight and narrative context inside VEOs. Visual propaganda by VEOs are not only detected, but also further classified based on the type of VEO and focus or intent of the image into hard propaganda, soft propaganda, symbolic propaganda, landscape, and organizational communications. Over 1.2 million images were automatically collected from suspicious OSN accounts and Web pages over a course of four years. Out of which, 120,000 images were manually classified to provide the training data for a convolutional neural network. An overall generalization accuracy of 97.02% and F1 of 97.89% were achieved for a binary classification or mere detection of visual VEO propaganda and an overall generalization accuracy of 86.08% and F1¯ of 85.76% for an eight-way classification based on the intent of the image.

KW - Convolutional neural networks

KW - Dark Web

KW - Deep learning

KW - Image classification

KW - Machine learning

KW - Violent extremist organizations

UR - http://www.scopus.com/inward/record.url?scp=85069840826&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85069840826&partnerID=8YFLogxK

U2 - 10.1016/j.imavis.2019.06.001

DO - 10.1016/j.imavis.2019.06.001

M3 - Article

AN - SCOPUS:85069840826

VL - 89

SP - 95

EP - 105

JO - Image and Vision Computing

JF - Image and Vision Computing

SN - 0262-8856

ER -