TPC-DS性能测试及使用方法

一、安装TPC-DS工具

1、下载工具
 下载地址：http://www.tpc.org/tpc_documents_current_versions/download_programs/tools-download-request.asp?bm_type=TPC-DS&bm_vers=2.6.0&mode=CURRENT-ONLY
注意：必须输入邮箱，他会发下载地址到邮箱中，点击下载地址即可下载。
2、解压编译工具
   1）解压
    执行命令：unzip 944eb36c-5624-45ea-bece-646814a75b63-tpc-ds-tool.zip
   2）进入tools目录编译
    执行命令：make

  以上TPC-DS测试环境就在准备好了

---

二、创建TPC-DS测试需要用到的表

1、准备好创建表语句
TPC-DS已经提前准备好了创建表相关的SQL文件
文件位于:tools目录下，具体文件如下：
tpcds.sql 创建25张表的sql语句
tpcds_ri.sql 创建表与表之间关系的sql语句
tpcds_source.sql
2、利用数据库连接工具(如：navicat preminer)创建相应的数据库和表
    将准备好的建表语句复制到工具中创建表

---

三、生成数据并将数据导入到相应的表中

1、生成数据
到tools目录下执行如下命令：
-dir 生成数据存放目录
-scale 生成数据大小
1）命令1:./dsdgen -DIR /part2/tpcds/v2.6.0/datas/ -SCALE 1 （scale表示产生1G测试数据）
2）命令2：./dsdgen -DIR /part2/tpcds/v2.6.0/datas/ -SCALE 10 -parallel 4 -child 1 （并行产生1g数据）
2、对生成的数据进行处理（不处理无法导入到表中）—每一行多了一个“|”
在/part2/tpcds/v2.6.0/datas/目录下创建目录handled
命令：mkdir handled
在/part2/tpcds/v2.6.0/datas/目录下执行如下命令：

for i in `ls *.dat`
    do
     name="handled/$i"
     echo $name
     `touch $name`
     `chmod 777 $name`
     sed 's/|$//' $i >> $name;
    done

3、将数据加载到表中

copy call_center from '/part2/tpcds/v2.6.0/datas/handled/call_center.dat' with delimiter as '|' NULL '';
    copy catalog_page from '/part2/tpcds/v2.6.0/datas/handled/catalog_page.dat' with delimiter as '|' NULL '';
    copy catalog_returns from '/part2/tpcds/v2.6.0/datas/handled/catalog_returns.dat' with delimiter as '|' NULL '';
    copy catalog_sales from '/part2/tpcds/v2.6.0/datas/handled/catalog_sales.dat' with delimiter as '|' NULL '';
    copy customer from '/part2/tpcds/v2.6.0/datas/handled/customer.dat' with delimiter as '|' NULL '';
    copy customer_address from '/part2/tpcds/v2.6.0/datas/handled/customer_address.dat' with delimiter as '|' NULL '';
    copy customer_demographics from '/part2/tpcds/v2.6.0/datas/handled/customer_demographics.dat' with delimiter as '|' NULL '';
    copy date_dim from '/part2/tpcds/v2.6.0/datas/handled/date_dim.dat' with delimiter as '|' NULL '';
    copy dbgen_version from '/part2/tpcds/v2.6.0/datas/handled/dbgen_version.dat' with delimiter as '|' NULL '';
    copy household_demographics from '/part2/tpcds/v2.6.0/datas/handled/household_demographics.dat' with delimiter as '|' NULL '';
    copy income_band from '/part2/tpcds/v2.6.0/datas/handled/income_band.dat' with delimiter as '|' NULL '';
    copy inventory from '/part2/tpcds/v2.6.0/datas/handled/inventory.dat' with delimiter as '|' NULL '';
    copy item from '/part2/tpcds/v2.6.0/datas/handled/item.dat' with delimiter as '|' NULL '';
    copy promotion from '/part2/tpcds/v2.6.0/datas/handled/promotion.dat' with delimiter as '|' NULL '';
    copy reason from '/part2/tpcds/v2.6.0/datas/handled/reason.dat' with delimiter as '|' NULL '';
    copy ship_mode from '/part2/tpcds/v2.6.0/datas/handled/ship_mode.dat' with delimiter as '|' NULL '';
    copy store from '/part2/tpcds/v2.6.0/datas/handled/store.dat' with delimiter as '|' NULL '';
    copy store_returns from '/part2/tpcds/v2.6.0/datas/handled/store_returns.dat' with delimiter as '|' NULL '';
    copy store_sales from '/part2/tpcds/v2.6.0/datas/handled/store_sales.dat' with delimiter as '|' NULL '';
    copy time_dim from '/part2/tpcds/v2.6.0/datas/handled/time_dim.dat' with delimiter as '|' NULL '';
    copy warehouse from '/part2/tpcds/v2.6.0/datas/handled/warehouse.dat' with delimiter as '|' NULL '';
    copy web_page from '/part2/tpcds/v2.6.0/datas/handled/web_page.dat' with delimiter as '|' NULL '';
    copy web_returns from '/part2/tpcds/v2.6.0/datas/handled/web_returns.dat' with delimiter as '|' NULL '';
    copy web_sales from '/part2/tpcds/v2.6.0/datas/handled/web_sales.dat' with delimiter as '|' NULL '';
    copy web_site from '/part2/tpcds/v2.6.0/datas/handled/web_site.dat' with delimiter as '|' NULL '';

---

四、生成99个查询语句

1、修改query_template下query1-99模板，在行尾加define _END = “”，否则执行生成命令会出错;
   1)、编写shell脚本update_query.sh，并添加如下内容：

#!/bin/bash

COUNTER=1
while [ $COUNTER -lt 100 ]
do
echo $COUNTER
echo "define _END = \"\";">>query$COUNTER.tpl
COUNTER=`expr $COUNTER + 1`
done    

   2)、将update_query.sh复制到query_template目录下，并执行脚本
    ./update_query.sh

2、到/part2/tpc-ds/v2.6.0/tools目录下执行如下命令：
     ./dsqgen -output_dir /part2/tpcds/querydata/ -input ../query_templates/templates.lst -scale 1 -dialect oracle -directory ../query_templates

注意：生成的语句某些数据库语法可能不支持，如：PostgreSQL

---

使用说明持续更新中…….