1、要求
内嵌TPCC测试脚本,支持PG、oracle、mysql等数据库的压力测试,使用前需要安装jdk,原来支持linux和windows上测试,自2.3.5版本以来不再支持windows上的压力测试,所以若想在Windows上测试,需要用2.3.5版本。
2、描述
每个仓库负责十个区域的供货,每个区域 3000 个客户服务,每个仓库维护 100000 种商品的库存纪录,每个仓库至多有10个终端
BenchmarkSQL数据量:
每个仓库数据量大约为76823.04KB即约为76M
所以 Warehouse=14时,数据量大约为1G
Warehouse=140时,数据量大约为10G
想要加载500G左右的数据,warehouse大约为5000即可,除了表数据还有索引
记一次加载时间:
128G内存、机械硬盘、7000warehouse
19:20开始加载数据,大约在第三天17:00左右加载完数据(近46h),然后开始建索引,直到第四天
测试模拟5中事务处理,新订单、支付操作、订单状态查询、发货、库存状态查询;TPC-C基准测试针对一种模拟订单录入与销售环境测量每分钟商业事务(tpmC)吞吐量。
3、简用
(1)修改prop.pg
driver=org.postgresql.Driver
conn=jdbc:postgresql://localhost:5866/benchdb
user=bench
password=bench
warehouses=10
terminals=1
//To run specified transactions per terminal- runMins must equal zero
runTxnsPerTerminal=0
//To run for specified minutes- runTxnsPerTerminal must equal zero
runMins=5
//Number of total transactions per minute
limitTxnsPerMin=300
(2)创建测试表
[highgo@localhost run]$ ./runSQL.sh props.pg sqlTableCreates ------步骤一
DROP SCHEMA IF EXISTS benchmarksql CASCADE;
CREATE SCHEMA benchmarksql;
create table benchmarksql.warehouse (
w_id integer not null,
w_ytd decimal(12,2),
w_tax decimal(4,4),
w_name varchar(10),
w_street_1 varchar(20),
w_street_2 varchar(20),
w_city varchar(20),
w_state char(2),
w_zip char(9)
);
create table benchmarksql.district (
d_w_id integer not null,
d_id integer not null,
d_ytd decimal(12,2),
d_tax decimal(4,4),
d_next_o_id integer,
d_name varchar(10),
d_street_1 varchar(20),
d_street_2 varchar(20),
d_city varchar(20),
d_state char(2),
d_zip char(9)
);
create table benchmarksql.customer (
c_w_id integer not null,
c_d_id integer not null,
c_id integer not null,
c_discount decimal(4,4),
c_credit char(2),
c_last varchar(16),
c_first varchar(16),
c_credit_lim decimal(12,2),
c_balance decimal(12,2),
c_ytd_payment float,
c_payment_cnt integer,
c_delivery_cnt integer,
c_street_1 varchar(20),
c_street_2 varchar(20),
c_city varchar(20),
c_state char(2),
c_zip char(9),
c_phone char(16),
c_since timestamp,
c_middle char(2),
c_data varchar(500)
);
create sequence hist_id_seq;
create table benchmarksql.history (
hist_id integer not null default nextval('hist_id_seq') primary key,
h_c_id integer,
h_c_d_id integer,
h_c_w_id integer,
h_d_id integer,
h_w_id integer,
h_date timestamp,
h_amount decimal(6,2),
h_data varchar(24)
);
create table benchmarksql.oorder (
o_w_id integer not null,
o_d_id integer not null,
o_id integer not null,
o_c_id integer,
o_carrier_id integer,
o_ol_cnt decimal(2,0),
o_all_local decimal(1,0),
o_entry_d timestamp
);
create table benchmarksql.new_order (
no_w_id integer not null,
no_d_id integer not null,
no_o_id integer not null
);
create table benchmarksql.order_line (
ol_w_id integer not null,
ol_d_id integer not null,
ol_o_id integer not null,
ol_number integer not null,
ol_i_id integer not null,
ol_delivery_d timestamp,
ol_amount decimal(6,2),
ol_supply_w_id integer,
ol_quantity decimal(2,0),
ol_dist_info char(24)
);
create table benchmarksql.stock (
s_w_id integer not null,
s_i_id integer not null,
s_quantity decimal(4,0),
s_ytd decimal(8,2),
s_order_cnt integer,
s_remote_cnt integer,
s_data varchar(50),
s_dist_01 char(24),
s_dist_02 char(24),
s_dist_03 char(24),
s_dist_04 char(24),
s_dist_05 char(24),
s_dist_06 char(24),
s_dist_07 char(24),
s_dist_08 char(24),
s_dist_09 char(24),
s_dist_10 char(24)
);
create table benchmarksql.item (
i_id integer not null,
i_name varchar(24),
i_price decimal(5,2),
i_data varchar(50),
i_im_id integer
);
(3)导入数据
①直接导入
[highgo@localhost run]$ ./runLoader.sh props.pg numWarehouses 10 ------步骤二
Starting BenchmarkSQL LoadData
----------------- Initialization -------------------
numWarehouses
10
driver=org.postgresql.Driver
conn=jdbc:postgresql://localhost:5866/benchdb
user=bench
password=******
------------- LoadData StartTime = Wed Aug 10 18:58:49 PDT 2016-------------
Start Whse Load for 10 Whses @ Wed Aug 10 18:58:49 PDT 2016 ...
Elasped Time(ms): 0.029
End Whse Load @ Wed Aug 10 18:58:49 PDT 2016
Start Item Load for 100000 Items @ Wed Aug 10 18:58:49 PDT 2016 ...
Elasped Time(ms): 0.379 Writing record 10000 of 100000
Elasped Time(ms): 0.856 Writing record 20000 of 100000
Elasped Time(ms): 0.729 Writing record 30000 of 100000
Elasped Time(ms): 0.52 Writing record 40000 of 100000
Elasped Time(ms): 0.576 Writing record 50000 of 100000
Elasped Time(ms): 0.281 Writing record 60000 of 100000
Elasped Time(ms): 0.36 Writing record 70000 of 100000
Elasped Time(ms): 0.269 Writing record 80000 of 100000
Elasped Time(ms): 0.251 Writing record 90000 of 100000
……
……
②可以将数据导出到csv文件中,以后可从这些文件将数据更快的多次导入数据库
[highgo@localhost run]$ ./runLoader.sh props.pg numWarehouses 10 fileLocation /home/highgo/Desktop/BMcsv/ (注意,目录最后BMcsv后/一定存在,不然会在Desktop目录下以BMcsv为文件名生成文件)
Starting BenchmarkSQL LoadData
----------------- Initialization -------------------
numWarehouses
10
/home/highgo/Desktop/BMcsv
driver=org.postgresql.Driver
conn=jdbc:postgresql://localhost:5866/benchdb
user=bench
password=******
------------- LoadData StartTime = Wed Aug 10 19:16:22 PDT 2016-------------
Start Whse Load for 10 Whses @ Wed Aug 10 19:16:22 PDT 2016 ...
Elasped Time(ms): 0.179
End Whse Load @ Wed Aug 10 19:16:22 PDT 2016
Start Item Load for 100000 Items @ Wed Aug 10 19:16:22 PDT 2016 ...
Elasped Time(ms): 0.2 Writing record 10000 of 100000
……
……
10仓库的数据量,导出到文件大小
[highgo@localhost BMcsv]$ ll -h
total 696M
-rw-rw-r–. 1 highgo highgo 19M Aug 10 22:55 cust-hist.csv
-rw-rw-r–. 1 highgo highgo 162M Aug 10 22:55 customer.csv
-rw-rw-r–. 1 highgo highgo 8.8K Aug 10 22:54 district.csv
-rw-rw-r–. 1 highgo highgo 7.0M Aug 10 22:54 item.csv
-rw-rw-r–. 1 highgo highgo 809K Aug 10 22:55 new-order.csv
-rw-rw-r–. 1 highgo highgo 13M Aug 10 22:55 order.csv
-rw-rw-r–. 1 highgo highgo 212M Aug 10 22:55 order-line.csv
-rw-rw-r–. 1 highgo highgo 283M Aug 10 22:54 stock.csv
-rw-rw-r–. 1 highgo highgo 829 Aug 10 22:54 warehouse.csv
将数据加载到数据库,首先修改sqlTableCopies中文件位置为导出目录,如:
copy benchmarksql.warehouse
(w_id, w_ytd, w_tax, w_name, w_street_1, w_street_2, w_city, w_state, w_zip)
from '/home/highgo/Desktop/BMcsv/warehouse.csv' WITH CSV;
[highgo@localhost run]$ ./runSQL.sh props.pg sqlTableCopies
copy benchmarksql.warehouse
(w_id, w_ytd, w_tax, w_name, w_street_1, w_street_2, w_city, w_state, w_zip)
from '/home/highgo/Desktop/BMcsv/warehouse.csv' WITH CSV;
copy benchmarksql.item
(i_id, i_name, i_price, i_data, i_im_id)
from '/home/highgo/Desktop/BMcsv/item.csv' WITH CSV;
copy benchmarksql.stock
……
……
(4)创建索引
[highgo@localhost run]$ ./runSQL.sh props.pg sqlIndexCreates ------步骤三
alter table benchmarksql.warehouse add constraint pk_warehouse
primary key (w_id);
alter table benchmarksql.district add constraint pk_district
primary key (d_w_id, d_id);
alter table benchmarksql.customer add constraint pk_customer
primary key (c_w_id, c_d_id, c_id);
……
……
(5)执行压测(仓库数为10,数据量大约为696M)
[highgo@localhost run]$ ./runBenchmark.sh props.pg ------步骤四
2016-08-10 23:16:43,365 INFO - Term-00,
2016-08-10 23:16:43,383 INFO - Term-00, +-------------------------------------------------------------+
2016-08-10 23:16:43,383 INFO - Term-00, BenchmarkSQL v4.1
2016-08-10 23:16:43,383 INFO - Term-00, +-------------------------------------------------------------+
2016-08-10 23:16:43,383 INFO - Term-00, (c) 2003, Raul Barbosa
2016-08-10 23:16:43,383 INFO - Term-00, (c) 2004-2014, Denis Lussier
2016-08-10 23:16:43,384 INFO - Term-00, +-------------------------------------------------------------+
2016-08-10 23:16:43,384 INFO - Term-00,
2016-08-10 23:16:43,384 INFO - Term-00, driver=org.postgresql.Driver
2016-08-10 23:16:43,384 INFO - Term-00, conn=jdbc:postgresql://localhost:5866/benchdb
2016-08-10 23:16:43,384 INFO - Term-00, user=bench
2016-08-10 23:16:43,385 INFO - Term-00,
2016-08-10 23:16:43,385 INFO - Term-00, warehouses=10
2016-08-10 23:16:43,385 INFO - Term-00, terminals=1
2016-08-10 23:16:43,385 INFO - Term-00, runMins=5
2016-08-10 23:16:43,386 INFO - Term-00, limitTxnsPerMin=300
2016-08-10 23:16:43,386 INFO - Term-00,
2016-08-10 23:16:43,386 INFO - Term-00, newOrderWeight=45
2016-08-10 23:16:43,386 INFO - Term-00, paymentWeight=43
2016-08-10 23:16:43,387 INFO - Term-00, orderStatusWeight=4
2016-08-10 23:16:43,387 INFO - Term-00, deliveryWeight=4
2016-08-10 23:16:43,387 INFO - Term-00, stockLevelWeight=4
2016-08-10 23:16:43,387 INFO - Term-00,
Term-00, Running Average tpmTOTAL: 339.46 Current tpmTOTAL: 12 Memory Usag
……
……
Term-00, Running Average tpmTOTAL: 281.66 Current tpmTOTAL: 7740 Memory UsTerm-00, Running Average tpmTOTAL: 281.43 Current tpmTOTAL: 7740 Memory UsTerm-00, Running Average tpmTOTAL: 281.67 Current tpmTOTAL: 7740 Memory UsTerm-00, Running Average tpmTOTAL: 281.67 Current tpmTOTAL: 7740 Memory UsTerm-00, Running Average tpmTOTAL: 281.71 Current tpmTOTAL: 7740 Memory Usage2016-08-10 23:21:44,425 INFO - Term-00, Current tpmTOTAL: 9672 Memory Usage: 3MB / 15MB
2016-08-10 23:21:44,426 INFO - Term-00,
2016-08-10 23:21:44,426 INFO - Term-00, Measured tpmC (NewOrders) = 124.11
2016-08-10 23:21:44,426 INFO - Term-00, Measured tpmTOTAL = 284.0
2016-08-10 23:21:44,426 INFO - Term-00, Session Start = 2016-08-10 23:16:44
2016-08-10 23:21:44,426 INFO - Term-00, Session End = 2016-08-10 23:21:44
2016-08-10 23:21:44,426 INFO - Term-00, Transaction Count = 1420