zoukankan      html  css  js  c++  java
  • Python 英文词频统计

    1. 词频统计预处理
    2. 下载一首英文的歌词或文章
    3. 将所有,.?!’:等分隔符全部替换为空格
    4. 将所有大写转换为小写
    5. 生成单词列表
    6. 生成词频统计
    7. 排序
    8. 排除语法型词汇,代词、冠词、连词
    9. 输出词频最大TOP10
    article ='''
    Big data analytics and business analytics
    by Duan, Lian; Xiong, Ye
    Over the past few decades, with the development of automatic identification, data capture and storage technologies, 
    people generate data much faster and collect data much bigger than ever before in business, science, engineering, education and other areas. 
    Big data has emerged as an important area of study for both practitioners and researchers. 
    It has huge impacts on data-related problems. 
    In this paper, we identify the key issues related to big data analytics and then investigate its applications specifically related to business problems.
    '''
    
    split = article.split()
    print(split)
    
    #使用空格替换标点符号
    article = article.replace(",","").replace(".","").replace(":","").replace(";","").replace("?","")
    
    
    #大写字母转换成小写字母
    exchange = article.lower();
    print(exchange)
    
    #生成单词列表
    list = exchange.split()
    print(list)
    
    #生成词频统计
    dic = {}
    for i in list:
        count = list.count(i)
        dic[i] = count
    print(dic)
    
    #排除特定单词
    word = {'and','the','with','in','by','its','for','of','an','to'}
    for i in word:
        del(dic[i])
    print(dic)
    
    #排序
    dic1= sorted(dic.items(),key=lambda d:d[1],reverse= True)
    print(dic1)
    
    #输出词频最大的前十位单词
    for i in range(10):
        print(dic1[i])

  • 相关阅读:
    汉诺塔IX
    N!
    卡片游戏
    vuejs 2—bind
    vuejs 1—基础
    困难的串 Kryptn Factor Uva129
    JavaScript13—JSON
    JavaScript练习—二级菜单
    JavaScript12—tools.js
    JavaScript11—定时器
  • 原文地址:https://www.cnblogs.com/zxc109525/p/8651841.html
Copyright © 2011-2022 走看看