第一次个人编程作业

第一次个人编程作业

我的github

计算模块接口的设计与实现过程

具体算法流程图如下

模块介绍

基本思想：余弦相似度算法 参考博客

one_hot用于onehot编码

    def one_hot(word_dict, keywords):  
         cut_code = [word_dict[word] for word in keywords]
        cut_code = [0]*len(word_dict)
        for word in keywords:
            cut_code[word_dict[word]] += 1
        return cut_code

def extract_keyword用于提取关键词

    def extract_keyword(content):  
        re_exp = re.compile(r'(<style>.*?</style>)|(<[^>]+>)', re.S)
        content = re_exp.sub(' ', content)
        content = html.unescape(content)
        seg = [i for i in jieba.cut(content, cut_all=True) if i != '']
        # 提取关键词
        keywords = jieba.analyse.extract_tags("|".join(seg), topK=200, withWeight=False)
        return keywords

计算模块接口部分的性能改进

消耗最大的部分

如图所示，main.py消耗最大

性能分析图

计算模块部分单元测试展示

测试结果：

基本都在0.8左右，上下浮动，较为符合预期。

部分测试代码：

if __name__ == '__main__':
    with open('F:/qq/sim_0.8/orig.txt', 'r', encoding="UTF-8") as x1, open('F:/qq/sim_0.8/orig_0.8_add.txt', 'r',
                                                                          encoding="UTF-8") as y1:
        content_x1 = x1.read()
        content_y1 = y1.read()
        similarity = CosineSimilarity(content_x1, content_y1)
        similarity = similarity.main()
        print('相似度: %.2f%%
' % (similarity * 100))
    with open('F:/qq/sim_0.8/orig.txt', 'r', encoding="UTF-8") as x2, open('F:/qq/sim_0.8/orig_0.8_del.txt', 'r',
                                                                          encoding="UTF-8") as y2:
        content_x2 = x2.read()
        content_y2 = y2.read()
        similarity = CosineSimilarity(content_x2, content_y2)
        similarity = similarity.main()
        print('相似度: %.2f%%
' % (similarity * 100))
    with open('F:/qq/sim_0.8/orig.txt', 'r', encoding="UTF-8") as x3, open('F:/qq/sim_0.8/orig_0.8_dis_1.txt', 'r',
                                                                          encoding="UTF-8") as y3:
        content_x3 = x3.read()
        content_y3 = y3.read()
        similarity = CosineSimilarity(content_x3, content_y3)
        similarity = similarity.main()
        print('相似度: %.2f%%
' % (similarity * 100))
    with open('F:/qq/sim_0.8/orig.txt', 'r', encoding="UTF-8") as x4, open('F:/qq/sim_0.8/orig_0.8_dis_3.txt', 'r',
                                                                               encoding="UTF-8") as y4:
        content_x4 = x4.read()
        content_y4 = y4.read()
        similarity = CosineSimilarity(content_x4, content_y4)
        similarity = similarity.main()
        print('相似度: %.2f%%
' % (similarity * 100))

    with open('F:/qq/sim_0.8/orig.txt', 'r', encoding="UTF-8") as x6, open('F:/qq/sim_0.8/orig_0.8_dis_7.txt', 'r',
                                                                          encoding="UTF-8") as y6:
        content_x6 = x6.read()
        content_y6 = y6.read()
        similarity = CosineSimilarity(content_x6, content_y6)
        similarity = similarity.main()
        print('相似度: %.2f%%
' % (similarity * 100))
    with open('F:/qq/sim_0.8/orig.txt', 'r', encoding="UTF-8") as x7, open('F:/qq/sim_0.8/orig_0.8_dis_10.txt', 'r',
                                                                          encoding="UTF-8") as y7:
        content_x7 = x7.read()
        content_y7 = y7.read()
        similarity = CosineSimilarity(content_x7, content_y7)
        similarity = similarity.main()
        print('相似度: %.2f%%
' % (similarity * 100))
    with open('F:/qq/sim_0.8/orig.txt', 'r', encoding="UTF-8") as x8, open('F:/qq/sim_0.8/orig_0.8_dis_15.txt', 'r',
                                                                          encoding="UTF-8") as y8:
        content_x8 = x8.read()
        content_y8 = y8.read()
        similarity = CosineSimilarity(content_x8, content_y8)
        similarity = similarity.main()
        print('相似度: %.2f%%
' % (similarity * 100))
    with open('F:/qq/sim_0.8/orig.txt', 'r', encoding="UTF-8") as x9, open('F:/qq/sim_0.8/orig_0.8_mix.txt', 'r',
                                                                          encoding="UTF-8") as y9:
        content_x9 = x9.read()
        content_y9 = y9.read()
        similarity = CosineSimilarity(content_x9, content_y9)
        similarity = similarity.main()
        print('相似度: %.2f%%
' % (similarity * 100))
    with open('F:/qq/sim_0.8/orig.txt', 'r', encoding="UTF-8") as x0, open('F:/qq/sim_0.8/orig_0.8_rep.txt', 'r',
                                                                          encoding="UTF-8") as y0:
        content_x0 = x0.read()
        content_y0 = y0.read()
        similarity = CosineSimilarity(content_x0, content_y0)
        similarity = similarity.main()
        print('相似度: %.2f%%
' % (similarity * 100))

计算模块部分异常处理说明

设计空白对比文档和完全一致的文档

  空白文档的结果：

没有异常。

完全一致文档的结果：

没有异常。
时间有限，暂时没有发现模块异常。

PSP表格如下

PSP2.1	Personal Software Process Stages	预估耗时（分钟）	实际耗时（分钟）
Planning	计划	30	40
Estimate	估计这个任务需要多少时间	20	20
Development	开发	480	300
Analysis	需求分析 (包括学习新技术)	300	200
Design Spec	生成设计文档	60	30
Design Review	设计复审	30	20
Coding Standard	代码规范 (为目前的开发制定合适的规范)	30	30
Design	具体设计	60	60
Coding	具体编码	300	200
Code Review	代码复审	30	30
Test	测试（自我测试，修改代码，提交修改）	120	90
Reporting	报告	60	80
Test Repor	测试报告	30	20
Size Measurement	计算工作量	30	15
Postmortem & Process Improvement Plan	事后总结, 并提出过程改进计划	40	35
Total	合计	1620	1140

小总结

  第一次做这种作业，没有经验，难度有点高。只能在网上论坛上找找别人的东西，参考了很多才完成作业。自己还是有很多不足，希望以后再接再厉。