[gensim:1689] Negative perplexity ?

[Dave Robinson]

I'm just getting my feet wet with the variational methods for LDA so I
apologize if this is an obvious question.

While I appreciate the concept in a philosophical sense, what does negative
perplexity for an LDA model imply? I get a very large negative value for
LdaModel.bound(corpus=ModelCorpus) . Looking at the Hoffman,Blie,Bach paper
(Eq 16) leads me to believe that this is 'difficult' to observe.

Or is the exponent from Eq 16 what is being presented?

Cheers,
=Dave

[Radim Řehůřek]

Hi Dave,

Post by Dave Robinson
While I appreciate the concept in a philosophical sense, what does negative
perplexity for an LDA model imply?  I get a very large negative value for
LdaModel.bound(corpus=ModelCorpus) . Looking at the Hoffman,Blie,Bach paper
(Eq 16) leads me to believe that this is 'difficult' to observe.
Or is the exponent from Eq 16 what is being presented?

exactly, the probabilities are infinitesimal, so all computations
happen in log space.

Best,
Radim

Post by Dave Robinson
Cheers,
=Dave

[Han Xu]

Hi Radim,

Thank you for your kind answer. I'm just wondering if this means the
smaller this score given by bound(), the better the language model is
likely to be?

Thanks and Regards,
Han

Post by Radim Řehůřek
Hi Dave,

Post by Dave Robinson
While I appreciate the concept in a philosophical sense, what does

negative

Post by Dave Robinson
perplexity for an LDA model imply? I get a very large negative value

for

Post by Dave Robinson
LdaModel.bound(corpus=ModelCorpus) . Looking at the Hoffman,Blie,Bach

paper

Post by Dave Robinson
(Eq 16) leads me to believe that this is 'difficult' to observe.
Or is the exponent from Eq 16 what is being presented?

exactly, the probabilities are infinitesimal, so all computations
happen in log space.
Best,
Radim

Post by Dave Robinson
Cheers,
=Dave