java动态爬虫jsoup以及正则表达式的运用

1.jsoup是java的HTML解析器，可直接解析某个URL地址，HTML文本内容。http://jsoup.org/官网

2.解析URL地址

 Document doc = Jsoup
                    .connect(url)
                    .userAgent(
                            "Mozilla/5.0 (Windows NT 6.1; WOW64; rv:37.0) Gecko/20100101 Firefox/37.0)") // 设置User-Agent
                    .timeout(5000) // 设置连接超时时间
                    .get();

Elements elements = doc.getElementsByClass("desc");
Elements subelements = elements.get(0).getElementsByTag("li");
 Elements dayElements = eachDayElement.getElementsByTag("tr");
 Elements firstSubElements = firstElement.getElementsByTag("td");
String text = elements.get(0).text();
private static String regEx_publishDate = "由中央气象台\s*(\d+):(\d+)\s*发布的";
    private static Pattern pattern_publishDate = Pattern
            .compile(regEx_publishDate);
Matcher matcher = pattern_publishDate.matcher(text);
if (matcher.find()) {
            int hour = Integer.parseInt(matcher.group(1));
            int minute = Integer.parseInt(matcher.group(2));}

3.要有jsoup的jar包

4. s 匹配任意的空白符 S匹配任意不是空白符的字符  d匹配数字 +重复一次或更多次  * 重复零次或更多次

demo:

(\d{4})-(\d{2})-(\d{2})\s+(\d{2}):(\d{2})发布
(\S+过敏\S+)：\s+(\S+)\s+(\S+)
\s+(感冒\S+)：\s+(\S+)\s+(\S+)
\s*(\S+)\s*
首要污染物：\s*(\S+)\s*"

正则表达式语法：

https://msdn.microsoft.com/zh-cn/library/ae5bf541%28v=vs.80%29.aspx