在GPU上的测试(4个Tesla P100-SXM2-16GB)
1.1万行测试:
CREATE TABLE flights_2008_new AS (SELECT * FROM flights_2008_10k);
select count(*) from flights_2008_new;
返回:
EXPR$0
10000
drop table flights_2008_test;
2.20万行测试:
CREATE TABLE flights_2008_new AS (SELECT * FROM flights_2008_200k);
select count(*) from flights_2008_new;
返回:
EXPR$0
200000
drop table flights_2008_new;
3.700万行测试:
CREATE TABLE flights_2008_new AS (SELECT * FROM flights_2008_7M);
报错,GPU内存溢出:
E0627 18:37:56.306819 3148 MapDHandler.cpp:2384] Exception: Query couldn't keep the entire working set of columns in GPU memory
Exception: Query couldn't keep the entire working set of columns in GPU memory
在CPU上的测试(ppc64le 128核):
1.1万行测试:
CREATE TABLE flights_2008_new AS (SELECT * FROM flights_2008_10k);
select count(*) from flights_2008_new;
返回:
EXPR$0
10000
drop table flights_2008_test;
2.20万行测试:
CREATE TABLE flights_2008_new AS (SELECT * FROM flights_2008_200k);
select count(*) from flights_2008_new;
返回:
EXPR$0
200000
drop table flights_2008_new;
3.700万行测试:
CREATE TABLE flights_2008_new AS (SELECT * FROM flights_2008_7M);
select count(*) from flights_2008_new;
返回:
EXPR$0
7009728
4.5600万行测试:
CREATE TABLE flights_2008_new AS (SELECT * FROM flights_2008_56M);
(前期运行平稳,后来,我一个128核CPU,占用情况达百分之一万二千多,然后服务器自动freezen了,这个方法先不试了)。
drop table flights_2008_new;
5.1亿行测试:
CREATE TABLE flights_2008_new AS (SELECT * FROM flights_2008_100M);
报错:Thrift: Wed Jun 28 09:43:13 2017 TSocket::write_partial() send() <Host: localhost Port: 9091>Broken pipe
Thrift: Wed Jun 28 09:43:13 2017 TSocket::open() connect() <Host: localhost Port: 9091>Connection refused
Thrift: Wed Jun 28 09:43:13 2017 TSocket::open() connect() <Host: localhost Port: 9091>Connection refused
Thrift error: connect() failed: Connection refused
第二次尝试:
Thrift: Wed Jun 28 10:02:52 2017 TSocket::open() connect() <Host: localhost Port: 9091>Connection refused
Thrift: Wed Jun 28 10:02:52 2017 TSocket::open() connect() <Host: localhost Port: 9091>Connection refused
Thrift error: connect() failed: Connection refused