zoukankan      html  css  js  c++  java

  • VIPS: a VIsion based Page Segmentation Algorithm

    VIPS: a VIsion based Page Segmentation Algorithm

    VIPS: a VIsion based Page Segmentation Algorithm

    Introduction

    The VIsion-based Page Segmentation (VIPS) algorithm aims to extract the semantic structure of a web page based on its visual presentation. Such semantic structure is a tree structure; each node in the tree corresponds to a block. Each node will be assigned a value (Degree of Coherence) to indicate how coherent of the content in the block based on visual perception, the bigger is the DoC value, the more coherent is the block. The VIPS algo-rithm makes full use of page layout structure. It first extracts all the suitable blocks from the html DOM tree, and then it finds the separators between these blocks. Here, separators denote the hori-zontal or vertical lines in a web page that visually cross with no blocks. Based on these separators, the semantic tree of the web page is constructed. Thus, a web page can be represented as a set of blocks (leaf nodes of the semantic tree). Compared with DOM based methods, the segments obtained by VIPS are much more semantically aggregated. Noisy information, such as navigation, advertisement, and decoration can be easily removed because they are often placed in certain positions of a page. Contents with different topics are distinguished as separate blocks.

     

    Paper List

    Original Paper

    Applications using VIPS

    If you find the VIPS algoirthm useful, we appreciate it very much if you can cite our following works:

    @Inproceedings{CHWM04
    author = "Deng Cai and Xiaofei He and Ji-Rong Wen and Wei-Ying Ma",
    title = "Block-level link analysis",
    booktitle = "Proceedings of the 27th annual international ACM SIGIR conference on Research and development in information retrieval (SIGIR'04)",
    pages = {440--447},
    year = "2004"}

    @Inproceedings{CYWM04
    author = "Deng Cai and Shipeng Yu and Ji-Rong Wen and Wei-Ying Ma",
    title = "Block-based web search",
    booktitle = "Proceedings of the 27th annual international ACM SIGIR conference on Research and development in information retrieval (SIGIR'04)",
    pages = {456--463},
    year = "2004"}

    @Inproceedings{YCWM03,
    author = "Shipeng Yu and Deng Cai and Ji-Rong Wen and Wei-Ying Ma",
    title = "Improving Pseudo-Relevance Feedback in Web Information Retrieval Using Web Page Segmentation",
    booktitle = "Twelfth International World Wide Web Conference (WWW2003)",
    year = "2003"}

    @Inproceedings{CYWM03,
    author = "Deng Cai and Shipeng Yu and Ji-Rong Wen and Wei-Ying Ma",
    title = "Extracting Content Structure for Web Pages based on Visual Representation",
    booktitle = "Fifth Asia Pacific Web Conference (APWeb2003)",
    year = "2003"}

     

    Demo

    Copyright Notice: All these programs can only be used for research.

    VIPS dll (The VIPS DLL is always under development. All versions are downloadable here.)

    • VIPS dll (pageanalyzer.dll) (release date: 03/26/2008. One bug fixed. Thanks Ankur Gupta for pointing out the bug.)

       

    • VIPS dll (pageanalyzer.dll) (release date: 01/16/2006. Some people requested for the HTML source code output, I added it. Also I changed some interfaces so you need to rebuild your program if you want to use this new dll. Meanwhile, please download the newest demo.)

       

    • VIPS Demo (release date: 01/16/2006) (You should download VIPS dll and register it first! This demo can only work on the new VIPS dll)

       

    • VIPS dll (pageanalyzer.dll) (release date: 03/20/2005, some bugs fixed)

       

    • VIPS dll (pageanalyzer.dll) (release date: 08/20/2004)

       

    • VIPS Demo (release date: 08/20/2004) (You should download VIPS dll and register it first!)

    How to use VIPS dll.

    • You should familiar with how to host a webbrowser(Internet Explorer) in your program. Some articles in MSDN are very useful.

       

    • A more powerful example of using VIPS dll in VS2003 (release date: 01/25/2006)
      (This example provides source code on how to process batch job using VIPS dll. The framework of this example is based on MFCbrowser, which is a demo project in MSDN. You only need to focus on the MFCbrowserView.cpp and MFCbrowserView.h. I added some comments and hopefully these two files are self explained. Email me if you still have any questions.)

       

    • A example of using VIPS dll in VC6.0 (release date: 08/20/2004) (You should download VIPS dll and register it first!)

    Notice: we are currently working to enhance the VIPS algorithm, any suggestions or problems can be send to dengcai2 AT cs DOT uiuc DOT edu.
  • 相关阅读:
    CSS中一个冒号和两个冒号有什么区别
    伪类元素实现可伸缩时间轴
    Gulp实现css、js、图片的压缩以及css、js文件的MD5命名
    for 循环进化史
    细谈sass和less中的变量及其作用域
    Vue2.0源码阅读笔记--双向绑定实现原理
    你所不知道的setTimeout
    前端COOKIE与SESSION的区别
    js移动端向左滑动出现删除按钮
    推荐几款屏幕录制工具（可录制GIF）
  • 原文地址：https://www.cnblogs.com/lexus/p/3600470.html
Copyright © 2011-2022 走看看