Agumaster 将爬虫取股票名称代号子系统分出来成agumaster_crawler, 两系统通过RabbitMq连接

agumaster_crawler系统负责启动爬虫取得数据,之后便往队列中推送.

agumaster_crawler系统中pom.xml关于RabbitMq的依赖是:

        <!-- RabbitMq -->
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-amqp</artifactId>
        </dependency>

agumaster_crawler系统中application.properties文件里对于RabbitMq的设置是:

spring.rabbitmq.host=localhost
spring.rabbitmq.port=5672
spring.rabbitmq.username=guest
spring.rabbitmq.password=guest

之后,就可以把Sender类写出来:

package com.heyang.agumasterCrawler;

import org.springframework.amqp.core.AmqpTemplate;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Component;

@Component
public class Sender {
    @Autowired
    private AmqpTemplate mqTlt;
    
    public void send(String msg) {
        this.mqTlt.convertAndSend("stockQueue",msg);
    }
}

具体使用Sender类的JUnit测试函数:

@SpringBootTest
class AgumasterCrawlerApplicationTests {
    @Autowired
    private Sender sender;
    
    @Test
    void contextLoads() throws Exception {
        BaseCrawler crawler=new FenghuangCrawler();
        List<Stock> stockList=crawler.getStockList();
        ObjectMapper mapper = new ObjectMapper();
        
        for(Stock s:stockList) {
            String str=mapper.writeValueAsString(s);
            this.sender.send(str);
        }
    }
}

发送给完毕后,RabbitMq队列的情况:

三千八百零一支股票都送到了.

而原有Agumaster系统中,也要添加RabbitMq的依赖,

        <!-- RabbitMq -->
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-amqp</artifactId>
        </dependency>

这个和上面的是一样的.

之后就可以写接收类了:

package com.ufo.hy.agumaster.mq;

import org.springframework.amqp.rabbit.annotation.RabbitHandler;
import org.springframework.amqp.rabbit.annotation.RabbitListener;
import org.springframework.stereotype.Component;

/**
 * Used to receive stock code/names
 * @author Heyang
 *
 */
@Component
@RabbitListener(queues="stockQueue")
public class Receiver {
    @RabbitHandler
    public void QueueReceive(String receivedMsg) {
        System.out.println(receivedMsg);
    }
}

这个类在工程Agumaster启动后便会去队列里取得消息回来,下面是部分它取得的消息:

{"id":2790,"code":"600538","name":"国发股份","utime":null,"src":null,"ctime":null}
{"id":2791,"code":"002367","name":"康力电梯","utime":null,"src":null,"ctime":null}
{"id":2792,"code":"600410","name":"华胜天成","utime":null,"src":null,"ctime":null}
{"id":2793,"code":"601007","name":"金陵饭店","utime":null,"src":null,"ctime":null}
{"id":2794,"code":"603955","name":"大千生态","utime":null,"src":null,"ctime":null}
{"id":2795,"code":"300227","name":"光韵达","utime":null,"src":null,"ctime":null}
{"id":2796,"code":"603195","name":"公牛集团","utime":null,"src":null,"ctime":null}
{"id":2797,"code":"000726","name":"鲁 泰Ａ","utime":null,"src":null,"ctime":null}
{"id":2798,"code":"002013","name":"中航机电","utime":null,"src":null,"ctime":null}
{"id":2799,"code":"002868","name":"绿康生化","utime":null,"src":null,"ctime":null}
{"id":2800,"code":"002558","name":"巨人网络","utime":null,"src":null,"ctime":null}
{"id":2801,"code":"002391","name":"长青股份","utime":null,"src":null,"ctime":null}
{"id":2802,"code":"300010","name":"立思辰","utime":null,"src":null,"ctime":null}
{"id":2803,"code":"000902","name":"新洋丰","utime":null,"src":null,"ctime":null}
{"id":2804,"code":"601965","name":"中国汽研","utime":null,"src":null,"ctime":null}
{"id":2805,"code":"300171","name":"东富龙","utime":null,"src":null,"ctime":null}
{"id":2806,"code":"300406","name":"九强生物","utime":null,"src":null,"ctime":null}
{"id":2807,"code":"600857","name":"宁波中百","utime":null,"src":null,"ctime":null}
{"id":2808,"code":"002463","name":"沪电股份","utime":null,"src":null,"ctime":null}
{"id":2809,"code":"002560","name":"通达股份","utime":null,"src":null,"ctime":null}
....

这样做,就用消息系统完成了系统的部分解耦.

--2020年5月9日--