Topic Detection and Compressed Classification in Twitter

Dimitrios Milioris (1, 2, 3, 4) , Philippe Jacquet (4, 2, 3)

1 X - École polytechnique
2 LINCS - Laboratory of Information, Network and Communication Sciences
3 Alcatel-Lucent Bell Labs France [Nozay]
4 EVA - Wireless Networking for Evolving & Adaptive Applications

In this paper we introduce a novel information propagation method in Twitter, while maintaining a low computational complexity. It exploits the power of Compressive Sensing in conjunction with a Kalman filter to update the states of a dynamical system. The proposed method first employs Joint Complexity, which is defined as the cardinality of a set of all distinct factors of a given string represented by suffix trees, to perform topic detection. Then based on the nature of the data, we apply the theory of Compressive Sensing to perform topic classification by recovering an indicator vector, while reducing significantly the amount of information from the tweets. We exploit datasets in various languages collected by using the Twitter streaming API and achieve better classification accuracy when compared with state-of-the-art methods.

Domaines: Traitement du texte et du document; Statistiques [math.ST]; Théorie de l'information [cs.IT]; Traitement du signal et de l'image [eess.SP]

Dimitrios Milioris, Philippe Jacquet. Topic Detection and Compressed Classification in Twitter. IEEE European Signal Processing Conference, Aug 2015, Nice, France. ⟨hal-01154837⟩