[gensim:11806] How to load a aligned word2vec model?

[e***@gmail.com]

Hi, I have a time series dataset. therefore, for each time period I trained
a word2vec model and realigned the models.

However, when I try to load the aligned word2vec models as follows I get
the below mentioned error.

#Load model
model = word2vec.Word2Vec.load('1970')

*train_words_pow += wv.vocab[wv.index2word[word_index]].count**power*
*KeyError: 'ctrx'*

Is there a way to resolve this error? :)

[Andrey Kutuzov]

Hi,

Are you sure the word 'ctrx' is present in your '1970' model?

Post by e***@gmail.com
Hi, I have a time series dataset. therefore, for each time period I
trained a word2vec model and realigned the models.
However, when I try to load the aligned word2vec models as follows I get
the below mentioned error.
#Load model
model = word2vec.Word2Vec.load('1970')
*train_words_pow += wv.vocab[wv.index2word[word_index]].count**power*
*KeyError: 'ctrx'*
*
*
Is there a way to resolve this error? :)
--
You received this message because you are subscribed to the Google Groups "Gensim" group.
To unsubscribe from this group and stop receiving emails from it, send
For more options, visit https://groups.google.com/d/optout.

[e***@gmail.com]

Hi Andrey. I am performing wor2vec alignment using the gensim code
in https://gist.github.com/quadrismegistus/09a93e219a6ffc4f216fb85235535faf

Therefore, the vocabulary may be altered. Is there any way to resolve this
issue?

[Andrey Kutuzov]

Hi,

It depends on what exactly you are trying to achieve.
But the Procrustes alignment by definition works only with the words
present in _both_ models you are aligning. Thus, the aligned model M2
vocabulary contains only the words M2 shares with the original model M1
(intersection of their vocabularies).

Post by e***@gmail.com
Hi Andrey. I am performing wor2vec alignment using the gensim code
in https://gist.github.com/quadrismegistus/09a93e219a6ffc4f216fb85235535faf
Therefore, the vocabulary may be altered. Is there any way to resolve
this issue?
--
You received this message because you are subscribed to the Google Groups "Gensim" group.
To unsubscribe from this group and stop receiving emails from it, send
For more options, visit https://groups.google.com/d/optout.

[e***@gmail.com]

Hi Andrey, my code is as follows.

#Load two models
word2vec_model_1980 = Word2Vec.load('1980')
word2vec_model_1970 = Word2Vec.load('1970')

#Perform Alignment
aligned_1970 = smart_procrustes_align_gensim(word2vec_model_1980,
word2vec_model_1970, words=None)

#Save aligned model for later use
aligned_1970.save('1970')

Later I reponed the aligned model in a different python program as
mentioned in the question. I got the error while performing that.

Am I doing anything wrong while aligning the models?

[Andrey Kutuzov]

It would be much easier to answer your question if you showed this
'different Python program'.
Without it, there is no way to guess what is happening (for example,
where the `word_index` variable comes from).

Post by e***@gmail.com
Hi Andrey, my code is as follows.
#Load two models
word2vec_model_1980 = Word2Vec.load('1980')
word2vec_model_1970 = Word2Vec.load('1970')
#Perform Alignment
aligned_1970 = smart_procrustes_align_gensim(word2vec_model_1980,
word2vec_model_1970, words=None)
#Save aligned model for later use
aligned_1970.save('1970')
Later I reponed the aligned model in a different python program as
mentioned in the question. I got the error while performing that.
Am I doing anything wrong while aligning the models?
--
You received this message because you are subscribed to the Google Groups "Gensim" group.
To unsubscribe from this group and stop receiving emails from it, send
For more options, visit https://groups.google.com/d/optout.

[e***@gmail.com]

Hi Andrey,

My other python programme is as follows (These are only lines I have).

######Input Data
word2vec_model_name_1 = "model_1970"

logging.basicConfig(format='%(asctime)s : %(levelname)s : %(message)s',
level=logging.INFO)

######Open word2vec model
model_1 = word2vec.Word2Vec.load(word2vec_model_name_1)

Here is the log I received. Any thoughts on how to resolve this issue? :)

2018-11-30 14:23:43,897 : INFO : loading Word2Vec object from model_19702018-11-30 14:23:43,961 : INFO : loading wv recursively from model_1970.wv.* with mmap=None2018-11-30 14:23:43,965 : INFO : loading vectors from model_1970.wv.vectors.npy with mmap=None2018-11-30 14:23:44,005 : INFO : setting ignored attribute vectors_norm to None2018-11-30 14:23:44,009 : INFO : loading vocabulary recursively from model_1970.vocabulary.* with mmap=None2018-11-30 14:23:44,009 : INFO : loading trainables recursively from model_1970.trainables.* with mmap=None2018-11-30 14:23:44,009 : INFO : loading syn1neg from model_1970.trainables.syn1neg.npy with mmap=None2018-11-30 14:23:44,053 : INFO : setting ignored attribute cum_table to None2018-11-30 14:23:44,053 : INFO : loaded model_1970Reloaded modules: __mp_main__Traceback (most recent call last):

File "<ipython-input-3-3b9230dacba9>", line 1, in <module>
runfile('C:/Users/Emi/Desktop/code/word2vec_distance.py', wdir='C:/Users/Emi/Desktop/code')

File "C:\Users\Emi\Anaconda3\lib\site-packages\spyder_kernels\customize\spydercustomize.py", line 668, in runfile
execfile(filename, namespace)

File "C:\Users\Emi\Anaconda3\lib\site-packages\spyder_kernels\customize\spydercustomize.py", line 108, in execfile
exec(compile(f.read(), filename, 'exec'), namespace)

File "C:/Users/Emi/Desktop/code/word2vec_distance.py", line 26, in <module>
model_1 = word2vec.Word2Vec.load(word2vec_model_name_1)

File "C:\Users\Emi\Anaconda3\lib\site-packages\gensim\models\word2vec.py", line 975, in load
return super(Word2Vec, cls).load(*args, **kwargs)

File "C:\Users\Emi\Anaconda3\lib\site-packages\gensim\models\base_any2vec.py", line 631, in load
model.vocabulary.make_cum_table(model.wv) # rebuild cum_table from vocabulary

File "C:\Users\Emi\Anaconda3\lib\site-packages\gensim\models\word2vec.py", line 1383, in make_cum_table
train_words_pow += wv.vocab[wv.index2word[word_index]].count**power
KeyError: 'cmnx'

[Andrey Kutuzov]

Hi,

The log you provided tells that the model is loaded successfully
('2018-11-30 14:23:44,053 : INFO : loaded model_1970').

What happens after that is a result of some other code that you seem to
run. Can you provide the full content of your word2vec_distance.py script?

P.S. I now tried to align two models using the same code and then load
the aligned model into the latest Gensim. No problems with that.

Post by e***@gmail.com
Hi Andrey,
My other python programme is as follows (These are only lines I have).
######Input Data
word2vec_model_name_1 = "model_1970"
logging.basicConfig(format='%(asctime)s : %(levelname)s : %(message)s',
level=logging.INFO)
######Open word2vec model
model_1 = word2vec.Word2Vec.load(word2vec_model_name_1)
Here is the log I received. Any thoughts on how to resolve this issue? :)
|2018-11-3014:23:43,897:INFO :loading Word2Vecobject frommodel_1970
2018-11-3014:23:43,961:INFO :loading wv recursively
frommodel_1970.wv.*withmmap=None2018-11-3014:23:43,965:INFO :loading
vectors frommodel_1970.wv.vectors.npy
withmmap=None2018-11-3014:23:44,005:INFO :setting ignored attribute
vectors_norm to None2018-11-3014:23:44,009:INFO :loading vocabulary
recursively
frommodel_1970.vocabulary.*withmmap=None2018-11-3014:23:44,009:INFO
:loading trainables recursively
frommodel_1970.trainables.*withmmap=None2018-11-3014:23:44,009:INFO
:loading syn1neg frommodel_1970.trainables.syn1neg.npy
withmmap=None2018-11-3014:23:44,053:INFO :setting ignored attribute
cum_table to None2018-11-3014:23:44,053:INFO :loaded model_1970
Reloadedmodules:__mp_main__ Traceback(most recent call
last):File"<ipython-input-3-3b9230dacba9>",line
1,in<module>runfile('C:/Users/Emi/Desktop/code/word2vec_distance.py',wdir='C:/Users/Emi/Desktop/code')File"C:\Users\Emi\Anaconda3\lib\site-packages\spyder_kernels\customize\spydercustomize.py",line
668,inrunfile
execfile(filename,namespace)File"C:\Users\Emi\Anaconda3\lib\site-packages\spyder_kernels\customize\spydercustomize.py",line
108,inexecfile
exec(compile(f.read(),filename,'exec'),namespace)File"C:/Users/Emi/Desktop/code/word2vec_distance.py",line
26,in<module>model_1
=word2vec.Word2Vec.load(word2vec_model_name_1)File"C:\Users\Emi\Anaconda3\lib\site-packages\gensim\models\word2vec.py",line
975,inload
returnsuper(Word2Vec,cls).load(*args,**kwargs)File"C:\Users\Emi\Anaconda3\lib\site-packages\gensim\models\base_any2vec.py",line
631,inload model.vocabulary.make_cum_table(model.wv)# rebuild cum_table
from
vocabularyFile"C:\Users\Emi\Anaconda3\lib\site-packages\gensim\models\word2vec.py",line
1383,inmake_cum_table train_words_pow
+=wv.vocab[wv.index2word[word_index]].count**power KeyError:'cmnx'|
--
You received this message because you are subscribed to the Google Groups "Gensim" group.
To unsubscribe from this group and stop receiving emails from it, send
For more options, visit https://groups.google.com/d/optout.

[Gordon Mohr]

That superficially-confirmatory log line comes from the superclass's
`load()`, before the full load has finished – this error is happening when
the subclass tries to complete the necessary reconstruction of of the
cumulative-distribution-table, used for negative-sampling.

So the error is coming from gensim's `load()`, not user code. It suggests
to me that some part of the 'alignment' may have left the internal
vocabulary-related structures inconsistent, perhaps when working with a
mixture of older and newer code, or models.

But, that `make_cum_table()` line hasn't been on line 631 since gensim
3.3.0. So a first thing to try would be using the latest gensim – to
generate the models and/or align them.

(Maybe also, ***@gmail.com is using the linked GIST code exactly,
while Andrey's successful test has applied the update he suggests in a
comment there?)

- Gordon

Post by Andrey Kutuzov
Hi,
The log you provided tells that the model is loaded successfully
('2018-11-30 14:23:44,053 : INFO : loaded model_1970').
What happens after that is a result of some other code that you seem to
run. Can you provide the full content of your word2vec_distance.py script?
P.S. I now tried to align two models using the same code and then load
the aligned model into the latest Gensim. No problems with that.

Post by e***@gmail.com
Hi Andrey,
My other python programme is as follows (These are only lines I have).
######Input Data
word2vec_model_name_1 = "model_1970"
logging.basicConfig(format='%(asctime)s : %(levelname)s : %(message)s',
level=logging.INFO)
######Open word2vec model
model_1 = word2vec.Word2Vec.load(word2vec_model_name_1)
Here is the log I received. Any thoughts on how to resolve this issue?

:)

Post by e***@gmail.com
|2018-11-3014:23:43,897:INFO :loading Word2Vecobject frommodel_1970
2018-11-3014:23:43,961:INFO :loading wv recursively
frommodel_1970.wv.*withmmap=None2018-11-3014:23:43,965:INFO :loading
vectors frommodel_1970.wv.vectors.npy
withmmap=None2018-11-3014:23:44,005:INFO :setting ignored attribute
vectors_norm to None2018-11-3014:23:44,009:INFO :loading vocabulary
recursively
frommodel_1970.vocabulary.*withmmap=None2018-11-3014:23:44,009:INFO
:loading trainables recursively
frommodel_1970.trainables.*withmmap=None2018-11-3014:23:44,009:INFO
:loading syn1neg frommodel_1970.trainables.syn1neg.npy
withmmap=None2018-11-3014:23:44,053:INFO :setting ignored attribute
cum_table to None2018-11-3014:23:44,053:INFO :loaded model_1970
Reloadedmodules:__mp_main__ Traceback(most recent call
last):File"<ipython-input-3-3b9230dacba9>",line

1,in<module>runfile('C:/Users/Emi/Desktop/code/word2vec_distance.py',wdir='C:/Users/Emi/Desktop/code')File"C:\Users\Emi\Anaconda3\lib\site-packages\spyder_kernels\customize\spydercustomize.py",line

Post by e***@gmail.com
668,inrunfile

execfile(filename,namespace)File"C:\Users\Emi\Anaconda3\lib\site-packages\spyder_kernels\customize\spydercustomize.py",line

Post by e***@gmail.com
108,inexecfile

exec(compile(f.read(),filename,'exec'),namespace)File"C:/Users/Emi/Desktop/code/word2vec_distance.py",line

Post by e***@gmail.com
26,in<module>model_1

=word2vec.Word2Vec.load(word2vec_model_name_1)File"C:\Users\Emi\Anaconda3\lib\site-packages\gensim\models\word2vec.py",line

Post by e***@gmail.com
975,inload

returnsuper(Word2Vec,cls).load(*args,**kwargs)File"C:\Users\Emi\Anaconda3\lib\site-packages\gensim\models\base_any2vec.py",line

Post by e***@gmail.com
631,inload model.vocabulary.make_cum_table(model.wv)# rebuild cum_table
from

vocabularyFile"C:\Users\Emi\Anaconda3\lib\site-packages\gensim\models\word2vec.py",line

Post by e***@gmail.com
1383,inmake_cum_table train_words_pow
+=wv.vocab[wv.index2word[word_index]].count**power KeyError:'cmnx'|
--
You received this message because you are subscribed to the Google
Groups "Gensim" group.
To unsubscribe from this group and stop receiving emails from it, send
For more options, visit https://groups.google.com/d/optout.

--
Solve et coagula!
Andrey

[Gordon Mohr]

That superficially-confirmatory log line comes from the superclass's
`load()`, before the full load has finished – this error is happening when
the subclass tries to complete the necessary reconstruction of of the
cumulative-distribution-table, used for negative-sampling.

So the error is coming from gensim's `load()`, not user code. It suggests
to me that some part of the 'alignment' may have left the internal
vocabulary-related structures inconsistent, perhaps when working with a
mixture of older and newer code, or models.

But, that `make_cum_table()` line hasn't been on line 631 since gensim
3.3.0. So a first thing to try would be using the latest gensim – to
generate the models and/or align them.

(Maybe also, ***@gmail.com is using the linked GIST code exactly,
while Andrey's successful test has applied the update he suggests in a
comment there?)

- Gordon

Post by Andrey Kutuzov
Hi,
The log you provided tells that the model is loaded successfully
('2018-11-30 14:23:44,053 : INFO : loaded model_1970').
What happens after that is a result of some other code that you seem to
run. Can you provide the full content of your word2vec_distance.py script?
P.S. I now tried to align two models using the same code and then load
the aligned model into the latest Gensim. No problems with that.

Post by e***@gmail.com
Hi Andrey,
My other python programme is as follows (These are only lines I have).
######Input Data
word2vec_model_name_1 = "model_1970"
logging.basicConfig(format='%(asctime)s : %(levelname)s : %(message)s',
level=logging.INFO)
######Open word2vec model
model_1 = word2vec.Word2Vec.load(word2vec_model_name_1)
Here is the log I received. Any thoughts on how to resolve this issue?

:)

Post by e***@gmail.com
|2018-11-3014:23:43,897:INFO :loading Word2Vecobject frommodel_1970
2018-11-3014:23:43,961:INFO :loading wv recursively
frommodel_1970.wv.*withmmap=None2018-11-3014:23:43,965:INFO :loading
vectors frommodel_1970.wv.vectors.npy
withmmap=None2018-11-3014:23:44,005:INFO :setting ignored attribute
vectors_norm to None2018-11-3014:23:44,009:INFO :loading vocabulary
recursively
frommodel_1970.vocabulary.*withmmap=None2018-11-3014:23:44,009:INFO
:loading trainables recursively
frommodel_1970.trainables.*withmmap=None2018-11-3014:23:44,009:INFO
:loading syn1neg frommodel_1970.trainables.syn1neg.npy
withmmap=None2018-11-3014:23:44,053:INFO :setting ignored attribute
cum_table to None2018-11-3014:23:44,053:INFO :loaded model_1970
Reloadedmodules:__mp_main__ Traceback(most recent call
last):File"<ipython-input-3-3b9230dacba9>",line

1,in<module>runfile('C:/Users/Emi/Desktop/code/word2vec_distance.py',wdir='C:/Users/Emi/Desktop/code')File"C:\Users\Emi\Anaconda3\lib\site-packages\spyder_kernels\customize\spydercustomize.py",line

Post by e***@gmail.com
668,inrunfile

execfile(filename,namespace)File"C:\Users\Emi\Anaconda3\lib\site-packages\spyder_kernels\customize\spydercustomize.py",line

Post by e***@gmail.com
108,inexecfile

exec(compile(f.read(),filename,'exec'),namespace)File"C:/Users/Emi/Desktop/code/word2vec_distance.py",line

Post by e***@gmail.com
26,in<module>model_1

=word2vec.Word2Vec.load(word2vec_model_name_1)File"C:\Users\Emi\Anaconda3\lib\site-packages\gensim\models\word2vec.py",line

Post by e***@gmail.com
975,inload

returnsuper(Word2Vec,cls).load(*args,**kwargs)File"C:\Users\Emi\Anaconda3\lib\site-packages\gensim\models\base_any2vec.py",line

Post by e***@gmail.com
631,inload model.vocabulary.make_cum_table(model.wv)# rebuild cum_table
from

vocabularyFile"C:\Users\Emi\Anaconda3\lib\site-packages\gensim\models\word2vec.py",line

Post by e***@gmail.com
1383,inmake_cum_table train_words_pow
+=wv.vocab[wv.index2word[word_index]].count**power KeyError:'cmnx'|
--
You received this message because you are subscribed to the Google
Groups "Gensim" group.
To unsubscribe from this group and stop receiving emails from it, send
For more options, visit https://groups.google.com/d/optout.

--
Solve et coagula!
Andrey

[Andrey Kutuzov]

Yes, thanks.
Trying the latest Gensim makes sense indeed.

Post by Gordon Mohr
That superficially-confirmatory log line comes from the superclass's
`load()`, before the full load has finished – this error is happening
when the subclass tries to complete the necessary reconstruction of of
the cumulative-distribution-table, used for negative-sampling. 
So the error is coming from gensim's `load()`, not user code. It
suggests to me that some part of the 'alignment' may have left the
internal vocabulary-related structures inconsistent, perhaps when
working with a mixture of older and newer code, or models. 
But, that `make_cum_table()` line hasn't been on line 631 since gensim
3.3.0. So a first thing to try would be using the latest gensim – to
generate the models and/or align them.
while Andrey's successful test has applied the update he suggests in a
comment there?)
- Gordon
Hi,
The log you provided tells that the model is loaded successfully
('2018-11-30 14:23:44,053 : INFO : loaded model_1970').
What happens after that is a result of some other code that you seem to
run. Can you provide the full content of your word2vec_distance.py script?
P.S. I now tried to align two models using the same code and then load
the aligned model into the latest Gensim. No problems with that.

Post by e***@gmail.com
Hi Andrey,
My other python programme is as follows (These are only lines I

have).

Post by e***@gmail.com
######Input Data
word2vec_model_name_1 = "model_1970"

%(message)s',

Post by e***@gmail.com
level=logging.INFO)
######Open word2vec model
model_1 = word2vec.Word2Vec.load(word2vec_model_name_1)
Here is the log I received. Any thoughts on how to resolve this

issue? :)

Post by e***@gmail.com
|2018-11-3014:23:43,897:INFO :loading Word2Vecobject frommodel_1970
2018-11-3014:23:43,961:INFO :loading wv recursively
frommodel_1970.wv.*withmmap=None2018-11-3014:23:43,965:INFO :loading
vectors frommodel_1970.wv.vectors.npy
withmmap=None2018-11-3014:23:44,005:INFO :setting ignored attribute
vectors_norm to None2018-11-3014:23:44,009:INFO :loading vocabulary
recursively
frommodel_1970.vocabulary.*withmmap=None2018-11-3014:23:44,009:INFO
:loading trainables recursively
frommodel_1970.trainables.*withmmap=None2018-11-3014:23:44,009:INFO
:loading syn1neg frommodel_1970.trainables.syn1neg.npy
withmmap=None2018-11-3014:23:44,053:INFO :setting ignored attribute
cum_table to None2018-11-3014:23:44,053:INFO :loaded model_1970
Reloadedmodules:__mp_main__ Traceback(most recent call
last):File"<ipython-input-3-3b9230dacba9>",line

1,in<module>runfile('C:/Users/Emi/Desktop/code/word2vec_distance.py',wdir='C:/Users/Emi/Desktop/code')File"C:\Users\Emi\Anaconda3\lib\site-packages\spyder_kernels\customize\spydercustomize.py",line

Post by e***@gmail.com
668,inrunfile

execfile(filename,namespace)File"C:\Users\Emi\Anaconda3\lib\site-packages\spyder_kernels\customize\spydercustomize.py",line

Post by e***@gmail.com
108,inexecfile

exec(compile(f.read(),filename,'exec'),namespace)File"C:/Users/Emi/Desktop/code/word2vec_distance.py",line

Post by e***@gmail.com
26,in<module>model_1

=word2vec.Word2Vec.load(word2vec_model_name_1)File"C:\Users\Emi\Anaconda3\lib\site-packages\gensim\models\word2vec.py",line

Post by e***@gmail.com
975,inload

returnsuper(Word2Vec,cls).load(*args,**kwargs)File"C:\Users\Emi\Anaconda3\lib\site-packages\gensim\models\base_any2vec.py",line

Post by e***@gmail.com
631,inload model.vocabulary.make_cum_table(model.wv)# rebuild

cum_table

Post by e***@gmail.com
from

vocabularyFile"C:\Users\Emi\Anaconda3\lib\site-packages\gensim\models\word2vec.py",line

Post by e***@gmail.com
1383,inmake_cum_table train_words_pow
+=wv.vocab[wv.index2word[word_index]].count**power KeyError:'cmnx'|
--
You received this message because you are subscribed to the Google
Groups "Gensim" group.
To unsubscribe from this group and stop receiving emails from it,

send

Post by e***@gmail.com
For more options, visit https://groups.google.com/d/optout

<https://groups.google.com/d/optout>.
--
Solve et coagula!
Andrey
--
You received this message because you are subscribed to the Google Groups "Gensim" group.
To unsubscribe from this group and stop receiving emails from it, send
For more options, visit https://groups.google.com/d/optout.

[e***@gmail.com]

Hi Andrey, It is really great to hear that it works with the latest Gensim.
Could you please kindly let me know what are the changes you did? It would
be really great if you could share me the code. Thanks a lot in advance :)

[Andrey Kutuzov]

Hi ***@gmail.com,

The changes you have to make are the ones Gordon mentioned earlier: you
should replace calls for '.index2word', '.vocab', '.syn0' and
'.syn0norm' with '.wv.index2word', '.wv.vocab', '.wv.syn0' and
'.wv.syn0norm' respectively. Then the alignment works perfectly.

This is because much has changed in Gensim since this gist was published.

Post by e***@gmail.com
Hi Andrey, It is really great to hear that it works with the latest
Gensim. Could you please kindly let me know what are the changes you
did? It would be really great if you could share me the code. Thanks a
lot in advance :)
--
You received this message because you are subscribed to the Google Groups "Gensim" group.
To unsubscribe from this group and stop receiving emails from it, send
For more options, visit https://groups.google.com/d/optout.

[e***@gmail.com]

Dear Andrey, Thanks a lot. I made these changes and it worked. Thanks again
for helping to resolve my issue :)

Just wondering if you can resolve my current issue, which I posted in the
group.
Link to the
question: https://groups.google.com/forum/#!topic/gensim/DoX9vs3qMBk

Looking forward to hearing from you.