zoukankan      html  css  js  c++  java
  • 使用余弦定理计算两篇文章的相似性

    使用余弦定理计算两篇文章的相似性:(方法论,细致易懂版)

    http://blog.csdn.net/dearwind153/article/details/52316151

    python 实现(代码):

    http://outofmemory.cn/code-snippet/35172/match-text-release

    (结巴分词下载及安装:http://www.cnblogs.com/kaituorensheng/p/3595879.html)

    java 实现(代码+方法描述):

    https://my.oschina.net/leejun2005/blog/116291

    (以上是我参考的资料)

    -----------------------------------------------------------------------------------------------------------------------------------------------

    我用的是Python实现的,需安装结巴分词的Python包

    代码如下:

    #!/usr/bin/env python
    # -*- coding: utf-8 -*
    
    import re
    from math import sqrt
    #You have to install the python lib
    import jieba
    
    def file_reader(filename,filename2):
        file_words = {}
        ignore_list = [u'的',u'了',u'和',u'呢',u'啊',u'哦',u'恩',u'嗯',u'吧'];
        accepted_chars = re.compile("[\u4E00-\u9FA5]+")
    
        file_object = open(filename)
    
        try:
            all_the_text = file_object.read()
            seg_list = jieba.cut(all_the_text, cut_all=True)
            #print "/ ".join(seg_list)
            for s in seg_list:
                if accepted_chars.match(s) and s not in ignore_list:
                    if s not in file_words.keys():
                        file_words[s] = [1,0]
                    else:
                        file_words[s][0] += 1
        finally:
            file_object.close()
    
        file_object2 = open(filename2)
    
        try:
            all_the_text = file_object2.read()
            seg_list = jieba.cut(all_the_text, cut_all=True)
            for s in seg_list:
                if accepted_chars.match(s) and s not in ignore_list:
                    if s not in file_words.keys():
                        file_words[s] = [0,1]
                    else:
                        file_words[s][1] += 1
        finally:
            file_object2.close()
    
        sum_2 = 0
        sum_file1 = 0
        sum_file2 = 0
        for word in file_words.values():
            sum_2 += word[0]*word[1]
            sum_file1 += word[0]**2
            sum_file2 += word[1]**2
    
        rate = sum_2/(sqrt(sum_file1*sum_file2))
        print('rate: ')
        print(rate)
    
    
    file_reader('thefile.txt','thefile2.txt')
    #该片段来自于http://outofmemory.cn
    
      
  • 相关阅读:
    安装jupyter
    git 查看分支图
    Docker原生健康检查使用
    压力测试指标判定
    Docker限制日志
    docker link 过时不再用了?那容器互联、服务发现怎么办?(2017年文章,建议使用docker network自定义网络)
    Nginx配置TCP服务负载均衡
    【转让】看看有你喜欢的书籍嘛?--都是我翻过的。
    《SOD框架企业级应用数据架构实战》新书简介和预定
    一年之计在于春,2015开篇:PDF.NET SOD Ver 5.1完全开源
  • 原文地址:https://www.cnblogs.com/xitingxie/p/6059929.html
Copyright © 2011-2022 走看看