Event prediction with learning algorithms - A study of events surrounding the Egyptian revolution of 2011 on the basis of micro blog data

Benedikt Boecking, Margeret Hall, Jeff Schneider

Research output: Contribution to journalReview article

18 Scopus citations

Abstract

We aim to predict activities of political nature influencing or reflecting societal-scale behavior and beliefs by applying learning algorithms to Twitter data. This study focuses on capturing domestic events in Egypt from November 2009 to November 2013. To this extent we study underlying communication patterns by evaluating content and metadata of 1.3 million tweets through computationally supported classification, without targeting specific keywords or users from the Twitter stream. Support Vector Machine (SVM) and Support Distribution Machine (SDM) classification algorithms are applied to detect and predict societal-scale unrest. Latent Dirichlet Allocation (LDA) is used to create content-based input patterns for the SVM while the SDM is used to classify sets of features created from meta-data. The experiments reveal that user centric approaches based on meta-data outperform methods employing content-based input despite the use of well established natural language processing algorithms. The results show that distributions over user centric meta information provide an important signal when detecting and predicting events. Applying this approach can assist policymakers and stakeholders in their efforts toward proactive community management.

Original languageEnglish (US)
Pages (from-to)159-184
Number of pages26
JournalPolicy and Internet
Volume7
Issue number2
DOIs
Publication statusPublished - Jun 1 2015

    Fingerprint

Keywords

  • Arab Spring
  • Twitter data
  • classification
  • event prediction
  • machine learning
  • modeling
  • politics

ASJC Scopus subject areas

  • Health(social science)
  • Public Administration
  • Health Policy
  • Computer Science Applications

Cite this