zoukankan      html  css  js  c++  java
  • Jsoup获取部分页面数据失败 org.jsoup.UnsupportedMimeTypeException: Unhandled content type. Must be text/*, application/xml, or application/xhtml+xml.

    1 现象 
    用Jsoup在获取一些网站的数据时,起初获取很顺利,但是在访问抖音的数据是Jsoup报错,应该是请求头里面的请求类型(ContextType)不符合要求。

    请求代码如下:

        private static void testOuGuanMatch() throws IOException{
            Document doc = Jsoup.connect("我的URL").userAgent("Mozilla/5.0 (Windows; U; Windows NT 5.1; zh-CN; rv:1.9.2.15)").timeout(5000).get();
            System.out.println(doc);
        }

    能看到我这里设置了请求代理和相应时间。

    报错信息如下:

    org.jsoup.UnsupportedMimeTypeException: Unhandled content type. Must be text/*, application/xml, or application/xhtml+xml. Mimetype=application/javascript, URL=....
        at org.jsoup.helper.HttpConnection$Response.execute(HttpConnection.java:472)
        at org.jsoup.helper.HttpConnection$Response.execute(HttpConnection.java:424)
        at org.jsoup.helper.HttpConnection.execute(HttpConnection.java:178)
        at org.jsoup.helper.HttpConnection.get(HttpConnection.java:167)
        at calendarSpider.SpiderTest.testOuGuanMatch(SpiderTest.java:174)
        at calendarSpider.SpiderTest.main(SpiderTest.java:39)

    2 方案
    添加ignoreContentType(true),修改后代码:

        private static void testOuGuanMatch() throws IOException{
            Document doc = Jsoup.connect("我的URL").ignoreContentType(true).userAgent("Mozilla/5.0 (Windows; U; Windows NT 5.1; zh-CN; rv:1.9.2.15)").timeout(5000).get();
            System.out.println(doc);
        }

    那这里的ignoreContentType(true)看词就知道忽略ContextType的检查

    转摘于:https://www.cnblogs.com/parryyang/p/5587929.html

  • 相关阅读:
    【Aizu
    【Aizu
    【OpenJ_Bailian
    leetcode-225-Implement Stack using Queues
    leetcode-219-Contains Duplicate II(使用set来判断长度为k+1的闭区间中有没有重复元素)
    leetcode-217-Contains Duplicate(使用排序来判断整个数组有没有重复元素)
    leetcode-840-Magic Squares In Grid
    leetcode-198-House Robber(动态规划)
    leetcode-191-Number of 1 Bits
    leetcode-189-Rotate Array
  • 原文地址:https://www.cnblogs.com/fanbi/p/15006480.html
Copyright © 2011-2022 走看看