[gensim:11738] pyLDAvis error: Not all rows (distributions) in topic_term_dists sum to 1.

[Sankaranarayanan Subbayya]

lda_model = gensim.models.ldamodel.LdaModel(corpus=corpus,
id2word=dictionary,
num_topics=10,
random_state=100,
update_every=1,
chunksize=10,
passes=10,
alpha='auto',
per_word_topics=True)

lda_model runs fine

Perplexity: -65.42412684458839

Coherence Score: 0.26230824386689766


# Visualize the topics
pyLDAvis.enable_notebook()
vis = pyLDAvis.gensim.prepare(lda_model, corpus, id2word)
vis


How to debug this error message: Not all rows (distributions) in topic_term_dists sum to 1



ValidationError Traceback (most recent call last)<ipython-input-75-3af4665cd12f> in <module>() 1 # Visualize the topics 2 pyLDAvis.enable_notebook()----> 3 vis = pyLDAvis.gensim.prepare(lda_model, corpus, id2word) 4 vis
/anaconda3/lib/python3.6/site-packages/pyLDAvis/gensim.py in prepare(topic_model, corpus, dictionary, doc_topic_dist, **kwargs) 117 """ 118 opts = fp.merge(_extract_data(topic_model, corpus, dictionary, doc_topic_dist), kwargs)--> 119 return vis_prepare(**opts)
/anaconda3/lib/python3.6/site-packages/pyLDAvis/_prepare.py in prepare(topic_term_dists, doc_topic_dists, doc_lengths, vocab, term_frequency, R, lambda_step, mds, n_jobs, plot_opts, sort_topics) 372 doc_lengths = _series_with_name(doc_lengths, 'doc_length') 373 vocab = _series_with_name(vocab, 'vocab')--> 374 _input_validate(topic_term_dists, doc_topic_dists, doc_lengths, vocab, term_frequency) 375 R = min(R, len(vocab)) 376
/anaconda3/lib/python3.6/site-packages/pyLDAvis/_prepare.py in _input_validate(*args) 63 res = _input_check(*args) 64 if res:---> 65 raise ValidationError('\n' + '\n'.join([' * ' + s for s in res])) 66 67
ValidationError:
* Not all rows (distributions) in topic_term_dists sum to 1.


Any help would be very much appreciated.


Thanks,


Sankar