scrapy_redis 相关: 查看保存的数据

0.参考资料

https://redis.io/topics/data-types-intro　　An introduction to Redis data types and abstractions

http://redisdoc.com/　　Redis 命令参考

1.scrapy_redis

2.redis-cli 查看数据

2.1 匹配数据库内所有 key

redis-cli

127.0.0.1:6379> KEYS *
1) "mycrawler_redis:dupefilter"
2) "mycrawler_redis:requests"
6) "mycrawler_redis:items"

2.2 List（列表）

127.0.0.1:6379> type mycrawler_redis:items
list
127.0.0.1:6379> llen mycrawler_redis:items
(integer) 701
127.0.0.1:6379> LRANGE mycrawler_redis:items 0 1
1) "{"text": "\u201cA woman is like a tea bag; you never know how strong it is until it's in hot water.\u201d", "crawled": "2018-02-21 03:38:17", "spider": "mycrawler_redis", "author": "Eleanor Roosevelt"}"
2) "{"text": "\u201cThe world as we have created it is a process of our thinking. It cannot be changed without changing our thinking.\u201d", "crawled": "2018-02-21 03:38:17", "spider": "mycrawler_redis", "author": "Albert Einstein"}"
127.0.0.1:6379> LRANGE mycrawler_redis:items -2 -1
1) "{"text": "\u201cThe opposite of love is not hate, it's indifference. The opposite of art is not ugliness, it's indifference. The opposite of faith is not heresy, it's indifference. And the opposite of life is not death, it's indifference.\u201d", "crawled": "2018-02-21 03:43:34", "spider": "mycrawler_redis", "author": "Elie Wiesel"}"
2) "{"text": "\u201cIt is not a lack of love, but a lack of friendship that makes unhappy marriages.\u201d", "crawled": "2018-02-21 03:43:34", "spider": "mycrawler_redis", "author": "Friedrich Nietzsche"}"

 

2.3 Set（集合）

PS: size是容量，但cardinality是「基数」，是集合论中的术语

127.0.0.1:6379> type mycrawler_redis:dupefilter
set
127.0.0.1:6379> SCARD mycrawler_redis:dupefilter
(integer) 18603
127.0.0.1:6379> SRANDMEMBER mycrawler_redis:dupefilter
"5faa874e145528c84d636d5a95959583301e18f2"
127.0.0.1:6379> SRANDMEMBER mycrawler_redis:dupefilter
"68f9f6842efcd0392236b953ba6cf5c4616d4c91"

2.4 SortedSet（有序集合）

20180726 更新: 也可通过 ZCARD key命令返回有序集 key 的基数。

127.0.0.1:6379> type mycrawler_redis:requests
zset
127.0.0.1:6379> ZLEXCOUNT mycrawler_redis:requests - +
(integer) 18199
127.0.0.1:6379> ZRANGE mycrawler_redis:requests 0 1 WITHSCORES
1) "x80x02}qx01(Ux04bodyqx02Ux00U _encodingqx03Ux05utf-8qx04Uacookiesqx05}qx06Ux04metaqa}q(Ux05depthq Kx02U link_textq clxml.etree _ElementStringResult qx0bUx0cspiritualityqx0cx85x81q }qx0e(Ua_parentqx0fNUx0cis_attributeqx10x89Uattrnameqx11NUais_textqx12x89Uais_tailqx13x89ubUx04ruleqx14Kx00uUaheadersqx15}qx16UaRefererqx17]qx18U https://www.goodreads.com/quotesqx19asUx03urlqx1aX1x00x00x00https://www.goodreads.com/quotes/tag/spiritualityUx0bdont_filterqx1bx89Upriorityqx1cKx00Ucallbackqx1dUx14_response_downloadedqx1eUx05flagsqx1f]q Ux06methodq!Ux03GETq"Uaerrbackq#Nu."
2) "0"
3) "x80x02}qx01(Ux04bodyqx02Ux00U _encodingqx03Ux05utf-8qx04Uacookiesqx05}qx06Ux04metaqa}q(Ux05depthq Kx02U link_textq clxml.etree _ElementStringResult qx0bU Choice Awardsqx0cx85x81q }qx0e(Ua_parentqx0fNUx0cis_attributeqx10x89Uattrnameqx11NUais_textqx12x89Uais_tailqx13x89ubUx04ruleqx14Kx00uUaheadersqx15}qx16UaRefererqx17]qx18U https://www.goodreads.com/quotesqx19asUx03urlqx1aX&x00x00x00https://www.goodreads.com/choiceawardsUx0bdont_filterqx1bx89Upriorityqx1cKx00Ucallbackqx1dUx14_response_downloadedqx1eUx05flagsqx1f]q Ux06methodq!Ux03GETq"Uaerrbackq#Nu."
4) "0"
127.0.0.1:6379> ZRANGE mycrawler_redis:requests -2 -1 WITHSCORES
1) "x80x02}qx01(Ux04bodyqx02Ux00U _encodingqx03Ux05utf-8qx04Uacookiesqx05}qx06Ux04metaqa}q(U link_textq Xx00x00x00x00Ux04ruleq Kx00Ux10download_timeoutqx0bG@fx80x00x00x00x00x00Ux05depthqx0cKx02Ux0bretry_timesq Kx01U download_slotqx0eUx0fwww.youtube.comqx0fuUaheadersqx10}qx11(Ux0fAccept-Languageqx12]qx13Ux02enqx14aUaRefererqx15]qx16Ux17https://scrapinghub.comqx17aUx0fAccept-Encodingqx18]qx19Ux0cgzip,deflateqx1aaUx06Acceptqx1b]qx1cU?text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8qx1daU User-Agentqx1e]qx1fU7scrapy-redis (+https://github.com/rolando/scrapy-redis)q auUx03urlq!X#x00x00x00https://www.youtube.com/scrapinghubUx0bdont_filterq"x88Upriorityq#JxffxffxffxffUcallbackq$Ux14_response_downloadedq%Ux05flagsq&]q'Ux06methodq(Ux03GETq)Uaerrbackq*Nu."
2) "1"
3) "x80x02}qx01(Ux04bodyqx02Ux00U _encodingqx03Ux05utf-8qx04Uacookiesqx05}qx06Ux04metaqa}q(U link_textq Xx00x00x00x00Ux04ruleq Kx00Ux10download_timeoutqx0bG@fx80x00x00x00x00x00Ux05depthqx0cKx02Ux0bretry_timesq Kx01U download_slotqx0eUx10www.facebook.comqx0fuUaheadersqx10}qx11(Ux0fAccept-Languageqx12]qx13Ux02enqx14aUaRefererqx15]qx16Ux17https://scrapinghub.comqx17aUx0fAccept-Encodingqx18]qx19Ux0cgzip,deflateqx1aaUx06Acceptqx1b]qx1cU?text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8qx1daU User-Agentqx1e]qx1fU7scrapy-redis (+https://github.com/rolando/scrapy-redis)q auUx03urlq!X%x00x00x00https://www.facebook.com/ScrapingHub/Ux0bdont_filterq"x88Upriorityq#JxffxffxffxffUcallbackq$Ux14_response_downloadedq%Ux05flagsq&]q'Ux06methodq(Ux03GETq)Uaerrbackq*Nu."
4) "1"