zoukankan      html  css  js  c++  java
  • 关于word2vec词向量读取

    import warnings
    import logging
    import os.path
    import sys
    import multiprocessing
    
    import gensim
    from gensim.models import Word2Vec
    from gensim.models.word2vec import LineSentence
    # 忽略警告
    warnings.filterwarnings(action='ignore', category=UserWarning, module='gensim')
     
    if __name__ == '__main__':
        
        program = os.path.basename(sys.argv[0]) # 读取当前文件的文件名
        logger = logging.getLogger(program)
        logging.basicConfig(format='%(asctime)s: %(levelname)s: %(message)s',level=logging.INFO)
        logger.info("running %s" % ' '.join(sys.argv))
     
        # inp为输入语料, outp1为输出模型, outp2为vector格式的模型
        inp = 'D:/csv/test/test1.txt'
        out_model = 'D:/csv/test/test.model'
        out_vector = 'D:/csv/test/test.vector'
     
        # 训练skip-gram模型
        model = Word2Vec(LineSentence(inp), size=50, window=5, min_count=5,
                         workers=multiprocessing.cpu_count())
     
        # 保存模型
        model.save(out_model)
        # 保存词向量
        model.wv.save_word2vec_format(out_vector, binary=False)

    会得到三个文件:.model,.model.syn0.npy,.model.syn1neg.npy,读取就可以:

    from gensim.models.deprecated.word2vec import Word2Vec
    model = Word2Vec.load('D:/csv/test/text.model') # 3个文件放在一起:.model,.model.syn0.npy,.model.syn1neg.npy
    print("read model successful")

    另外一种就是获得无后缀的文件,目测这个文件包含了这三个文件的。

    import warnings
    import gensim
    from gensim.models import Word2Vec
    from gensim.models.word2vec import LineSentence
    # 忽略警告
    warnings.filterwarnings(action='ignore', category=UserWarning, module='gensim')

    filePath = "D:/csv/test/test1.txt" sentences = gensim.models.word2vec.LineSentence(filePath) model = gensim.models.Word2Vec(sentences, size=100, window=5, min_count=1, workers=4) # 训练模型 model.save('D:/csv/test/test1')

    读取时:

    import gensim    
    
    inp = 'D:/csv/test/text'  # 读取词向量
    model = gensim.models.Word2Vec.load(inp)
    print("read model successful")

    感谢这里这里

  • 相关阅读:
    捡到一本<C++ Reference>
    题目1008:最短路径问题
    题目1014:排名
    题目1080:进制转换
    题目1081:递推数列
    题目1086:最小花费
    题目1076:N的阶乘
    题目1035:找出直系亲属
    在Mac上搭建Jenkins环境
    获取鼠标点击UGUI,先对于特定物体的相对坐标
  • 原文地址:https://www.cnblogs.com/wzwi/p/11045695.html
Copyright © 2011-2022 走看看