Event prediction with learning algorithms - A study of events surrounding the Egyptian revolution of 2011 on the basis of micro blog data

Benedikt Boecking, Margeret Hall, Jeff Schneider

Research output: Contribution to journalReview article

14 Citations (Scopus)

Abstract

We aim to predict activities of political nature influencing or reflecting societal-scale behavior and beliefs by applying learning algorithms to Twitter data. This study focuses on capturing domestic events in Egypt from November 2009 to November 2013. To this extent we study underlying communication patterns by evaluating content and metadata of 1.3 million tweets through computationally supported classification, without targeting specific keywords or users from the Twitter stream. Support Vector Machine (SVM) and Support Distribution Machine (SDM) classification algorithms are applied to detect and predict societal-scale unrest. Latent Dirichlet Allocation (LDA) is used to create content-based input patterns for the SVM while the SDM is used to classify sets of features created from meta-data. The experiments reveal that user centric approaches based on meta-data outperform methods employing content-based input despite the use of well established natural language processing algorithms. The results show that distributions over user centric meta information provide an important signal when detecting and predicting events. Applying this approach can assist policymakers and stakeholders in their efforts toward proactive community management.

Original languageEnglish (US)
Pages (from-to)159-184
Number of pages26
JournalPolicy and Internet
Volume7
Issue number2
DOIs
StatePublished - Jun 1 2015

Fingerprint

Blogging
Blogs
Metadata
weblog
Learning algorithms
Learning
Support vector machines
event
twitter
Natural Language Processing
learning
Egypt
Politics
communication pattern
Communication
stakeholder
Processing
experiment
language
Experiments

Keywords

  • Arab Spring
  • Twitter data
  • classification
  • event prediction
  • machine learning
  • modeling
  • politics

ASJC Scopus subject areas

  • Health(social science)
  • Public Administration
  • Health Policy
  • Computer Science Applications

Cite this

Event prediction with learning algorithms - A study of events surrounding the Egyptian revolution of 2011 on the basis of micro blog data. / Boecking, Benedikt; Hall, Margeret; Schneider, Jeff.

In: Policy and Internet, Vol. 7, No. 2, 01.06.2015, p. 159-184.

Research output: Contribution to journalReview article

@article{c7ed30a181074cc0927e0f022da63aa1,
title = "Event prediction with learning algorithms - A study of events surrounding the Egyptian revolution of 2011 on the basis of micro blog data",
abstract = "We aim to predict activities of political nature influencing or reflecting societal-scale behavior and beliefs by applying learning algorithms to Twitter data. This study focuses on capturing domestic events in Egypt from November 2009 to November 2013. To this extent we study underlying communication patterns by evaluating content and metadata of 1.3 million tweets through computationally supported classification, without targeting specific keywords or users from the Twitter stream. Support Vector Machine (SVM) and Support Distribution Machine (SDM) classification algorithms are applied to detect and predict societal-scale unrest. Latent Dirichlet Allocation (LDA) is used to create content-based input patterns for the SVM while the SDM is used to classify sets of features created from meta-data. The experiments reveal that user centric approaches based on meta-data outperform methods employing content-based input despite the use of well established natural language processing algorithms. The results show that distributions over user centric meta information provide an important signal when detecting and predicting events. Applying this approach can assist policymakers and stakeholders in their efforts toward proactive community management.",
keywords = "Arab Spring, Twitter data, classification, event prediction, machine learning, modeling, politics",
author = "Benedikt Boecking and Margeret Hall and Jeff Schneider",
year = "2015",
month = "6",
day = "1",
doi = "10.1002/poi3.89",
language = "English (US)",
volume = "7",
pages = "159--184",
journal = "Policy and Internet",
issn = "1944-2866",
publisher = "John Wiley and Sons Ltd",
number = "2",

}

TY - JOUR

T1 - Event prediction with learning algorithms - A study of events surrounding the Egyptian revolution of 2011 on the basis of micro blog data

AU - Boecking, Benedikt

AU - Hall, Margeret

AU - Schneider, Jeff

PY - 2015/6/1

Y1 - 2015/6/1

N2 - We aim to predict activities of political nature influencing or reflecting societal-scale behavior and beliefs by applying learning algorithms to Twitter data. This study focuses on capturing domestic events in Egypt from November 2009 to November 2013. To this extent we study underlying communication patterns by evaluating content and metadata of 1.3 million tweets through computationally supported classification, without targeting specific keywords or users from the Twitter stream. Support Vector Machine (SVM) and Support Distribution Machine (SDM) classification algorithms are applied to detect and predict societal-scale unrest. Latent Dirichlet Allocation (LDA) is used to create content-based input patterns for the SVM while the SDM is used to classify sets of features created from meta-data. The experiments reveal that user centric approaches based on meta-data outperform methods employing content-based input despite the use of well established natural language processing algorithms. The results show that distributions over user centric meta information provide an important signal when detecting and predicting events. Applying this approach can assist policymakers and stakeholders in their efforts toward proactive community management.

AB - We aim to predict activities of political nature influencing or reflecting societal-scale behavior and beliefs by applying learning algorithms to Twitter data. This study focuses on capturing domestic events in Egypt from November 2009 to November 2013. To this extent we study underlying communication patterns by evaluating content and metadata of 1.3 million tweets through computationally supported classification, without targeting specific keywords or users from the Twitter stream. Support Vector Machine (SVM) and Support Distribution Machine (SDM) classification algorithms are applied to detect and predict societal-scale unrest. Latent Dirichlet Allocation (LDA) is used to create content-based input patterns for the SVM while the SDM is used to classify sets of features created from meta-data. The experiments reveal that user centric approaches based on meta-data outperform methods employing content-based input despite the use of well established natural language processing algorithms. The results show that distributions over user centric meta information provide an important signal when detecting and predicting events. Applying this approach can assist policymakers and stakeholders in their efforts toward proactive community management.

KW - Arab Spring

KW - Twitter data

KW - classification

KW - event prediction

KW - machine learning

KW - modeling

KW - politics

UR - http://www.scopus.com/inward/record.url?scp=84930476731&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84930476731&partnerID=8YFLogxK

U2 - 10.1002/poi3.89

DO - 10.1002/poi3.89

M3 - Review article

AN - SCOPUS:84930476731

VL - 7

SP - 159

EP - 184

JO - Policy and Internet

JF - Policy and Internet

SN - 1944-2866

IS - 2

ER -