Arcomem training Topic Analysis Models advanced

Topic Analysis in ARCOMEM

Yahoo Research Barcelona

What is Probabilistic Topic Modelling?

Exploring and retrieving meaningful information from large collections of textual documents is a challenging task

Probabilistic topic models are a suite of algorithms (a framework) that aim to discover and annotate large archives of documents

with thematic information.

They do not require any prior annotations or labeling of the documents.

Topics emerge from the statistical analysis of the original texts

Probabilistic Topic ModelTopic models are based upon the idea that documents are mixtures

of topics, where a topic is a probability distribution over a fixed vocabulary.

A topic model is a generative model for documents: it specifies a simple probabilistic procedure by which documents can be generated.

The idea is to study the co-occurrence of words, assuming that words that tend to co-occur frequently, express, or belong to, the

same semantic concept.

Example: A document (d) can be represented by the following mixture of topics

Biology PhysicsMathemati

cs

0,6 0,3 0,1In the topic “Biology” words such as “Dna, genetic, evolution” have high probability

Intuition behind topic modelling

Documents exhibit multiple topics

Each topic is individually interpretable, providing a probability distribution over words that picks out a coherent cluster of correlated terms

Evolution BiologyGeneticsStatistical Analysis

Generative process

We only observe the documents

Our goal is to infer the underlying topic structure

What are the topics?

How are the documents divided according to those topics?

Topic 1: ?Topic 1: ?

Topic 2: ?Topic 2: ?

time seriesnonlinearmathematicsgeometricdynamics

Ecologistpopulationspeciesnaturalnature human

Text Modeling

A word is the basic unit of discrete data, defined to be an item from a vocabulary indexed by {1, . . . , V }.

A document is a sequence of N words denoted by w = (w1,w2,... ,wN), where wn is the nth word in the sequence.

A corpus is a collection of M documents denoted by D = {w1,w2,... ,wM}.

Bag-of-words assumption: the only information relevant to the model is the number of times words are produced. We don’t

consider word-order!!!!

Latent Dirichlet Allocation

The challenge is to identify, for each campaign, significant and important topics that are relevant to the two user cases, broadcasting

and parliament libraries.

Topic analysis provides semantic useful categories which allow end-users to search and browse content archives.

Try out on SARA: Trending topics

Try out on SARA: Statistical Topic Models