在做一个数据分析的一个项目 有大量的数据分析处理,逻辑复杂,导致响应速度很慢。3分钟。。。。。。这怎么可能容忍话不多说,多线程搞起(这里就不说多线程优缺点,只是记录一下下次使用方便)
一开始我使用的 三个线程代码如下
latch = new CountDownLatch(3); long startTime = System.currentTimeMillis(); List<DetailDataDto> demandInputDtoPageInfo = listDetailDataByDate(conditionVo); class HeadIn implements Runnable { @Override public void run() { demandInputDtoPageInfo.parallelStream().forEach(s -> { s.setCheckTime(basicInfoMapper.getCreateTime(s.getRequirementNO())); }); latch.countDown(); } } class AllIn implements Runnable { @Override public void run() { demandInputDtoPageInfo.parallelStream().forEach(s -> { s.setEndTime(demandInputMapper.getEndTime(s.getRequirementNO())); }); latch.countDown(); } } class HeadNotIn implements Runnable { @Override public void run() { demandInputDtoPageInfo.parallelStream().forEach(s -> { s.setDeployTime(basicInfoMapper.getDeployTime(s.getRequirementNO())); }); latch.countDown(); } } Thread myThread1 = new Thread(new HeadIn()); myThread1.start(); Thread myThread2 = new Thread(new AllIn()); myThread2.start(); Thread myThread3 = new Thread(new HeadNotIn()); myThread3.start(); latch.await(); //等待线程结束后执行最后逻辑
这样有些优化 快来一分钟 但是还不能忍受 没事接着来 百度各种搜,完了 忘了原地址了 抱歉(…………)
于是就有下面的东西
class Task implements Callable<List<DetailDataDto>> { private List<DetailDataDto> list; private int start; private int end; public Task(List<DetailDataDto> list, int start, int end) { this.list = list; this.start = start; this.end = end; } @Override public List<DetailDataDto> call() throws Exception { Object obj = null; List<DetailDataDto> retList = new ArrayList<DetailDataDto>(); for (int i = start; i < end; i++) { obj = list.get(i); Thread myThread1 = new Thread(new HeadIn()); myThread1.start(); Thread myThread2 = new Thread(new AllIn()); myThread2.start(); Thread myThread3 = new Thread(new HeadNotIn()); myThread3.start(); latch.await(); // 我把三个多线程在这执行 //这个可以填写自己的逻辑代码 } //返回处理结果 return retList; } } List<DetailDataDto> list = new ArrayList<DetailDataDto>(10000); int index = 0; ExecutorService ex = Executors.newFixedThreadPool(5); int dealSize = 2000; List<Future<List<DetailDataDto>>> futures = new ArrayList<>(5); //分配 for (int i = 0; i <= 5; i++, index += dealSize) { int start = index; if (start >= list.size()) { break; } int end = start + dealSize; end = end > list.size() ? list.size() : end; futures.add(ex.submit(new Task(list, start, end))); } try { //处理 List<Object> result = new ArrayList<>(); for (Future<List<DetailDataDto>> future : futures) { //合并操作 result.addAll(future.get()); } } catch (Exception e) { e.printStackTrace(); }
结果是很明显的 3秒 恩。。。这个还是可以接受,其他在硬件上优化 就是多线程分批跑数据 最后合并 在处理的过程。 记录的有些粗糙 不巧看到这篇贴子大神 勿喷。多谢