6.2不一致的数据转换

最新推荐文章于 2024-09-28 04:10:04 发布

keykongbai

最新推荐文章于 2024-09-28 04:10:04 发布

阅读量216

点赞数

分类专栏： ETL 文章标签：大数据 etl 数据库

本文链接：https://blog.csdn.net/qq_56941108/article/details/127835954

版权

ETL 专栏收录该内容

11 篇文章 1 订阅

订阅专栏

一.数据准备

通过Kettle工具将对数据表company进行不一致数据的转换操作，即通过与供货商提供的标准价格表进行比较，得出不一致数据，从而进行修改，最终输出到数据表company中。

suppilers表：

create table `suppliers` (
	`id` INT,
	`brand` varchar (150),
	`model` varchar (150),
	`unitPrice` varchar (150)
); 
insert into `suppliers` (`id`, `brand`, `model`, `unitPrice`) values('1','华为','华为nova 5 Pro（8GB/128GB/全网通）','2999');
insert into `suppliers` (`id`, `brand`, `model`, `unitPrice`) values('2','三星','三星GALAXY Note 10+（12GB/256GB/全网通/5G版）','7999');
insert into `suppliers` (`id`, `brand`, `model`, `unitPrice`) values('3','OPPO','OPPO Reno2（8GB/128GB/全网通）','2999');
insert into `suppliers` (`id`, `brand`, `model`, `unitPrice`) values('4','华为','华为Mate30（8GB/128GB/全网通/5G版/玻璃版）','4999');
insert into `suppliers` (`id`, `brand`, `model`, `unitPrice`) values('5','OPPO','OPPO Reno 10倍变焦版（8GB/256GB/全网通）','4299');
insert into `suppliers` (`id`, `brand`, `model`, `unitPrice`) values('6','vivo','vivo NEX 3（8GB/256GB/全网通/5G版）','5698');
insert into `suppliers` (`id`, `brand`, `model`, `unitPrice`) values('7','vivo','vivo NEX 3（8GB/128GB/全网通）','4998');
insert into `suppliers` (`id`, `brand`, `model`, `unitPrice`) values('8','vivo','vivo iQOO Pro（8GB/128GB/5G全网通）','3798');
insert into `suppliers` (`id`, `brand`, `model`, `unitPrice`) values('9','小米','小米9（8GB/256GB/全网通）','2999');
insert into `suppliers` (`id`, `brand`, `model`, `unitPrice`) values('10','华为','华为Mate 20 X（8GB/256GB/全网通/5G版）','6199');
insert into `suppliers` (`id`, `brand`, `model`, `unitPrice`) values('11','OPPO','OPPO R17（8GB RAM/全网通）','2499');
insert into `suppliers` (`id`, `brand`, `model`, `unitPrice`) values('12','苹果','苹果iPhone 11 Pro Max（6GB/64GB/全网通）','9599');
insert into `suppliers` (`id`, `brand`, `model`, `unitPrice`) values('13','三星','三星Galaxy S10+（8GB RAM/陶瓷版/全网通）','7499');
insert into `suppliers` (`id`, `brand`, `model`, `unitPrice`) values('14','苹果','苹果iPhone 11（4GB/128GB/全网通）','5999');
insert into `suppliers` (`id`, `brand`, `model`, `unitPrice`) values('15','苹果','苹果iPhone 11 Pro（6GB/256GB/全网通）','9999');
insert into `suppliers` (`id`, `brand`, `model`, `unitPrice`) values('16','华为','华为P30 Pro（8GB/128GB/全网通）','4988');
insert into `suppliers` (`id`, `brand`, `model`, `unitPrice`) values('17','苹果','苹果iPhone 11（4GB/256GB/全网通）','6799');
insert into `suppliers` (`id`, `brand`, `model`, `unitPrice`) values('18','小米','小米9 Pro（8GB/256GB/全网通/5G版）','3799');
insert into `suppliers` (`id`, `brand`, `model`, `unitPrice`) values('19','三星','三星GALAXY Note 10（8GB/256GB/全网通）','6599');
insert into `suppliers` (`id`, `brand`, `model`, `unitPrice`) values('20','苹果','苹果iPhone 11（4GB/64GB/全网通）','5499');
insert into `suppliers` (`id`, `brand`, `model`, `unitPrice`) values('21','vivo','vivo X27 Pro（8GB RAM/全网通）','3598');
insert into `suppliers` (`id`, `brand`, `model`, `unitPrice`) values('22','苹果','苹果iPhone 11 Pro（6GB/64GB/全网通）','8699');
insert into `suppliers` (`id`, `brand`, `model`, `unitPrice`) values('23','华为','华为Mate30 Pro（8GB/128GB/全网通）','5799');
insert into `suppliers` (`id`, `brand`, `model`, `unitPrice`) values('24','小米','小米MIX 3（6GB RAM/全网通）','2299');
insert into `suppliers` (`id`, `brand`, `model`, `unitPrice`) values('25','华为','华为Mate30 Pro（8GB/256GB/全网通/5G版/玻璃版）','6899');

二.建立转换添加控件

三.配置控件

1.配置“表输入”控件

双击进入配置页面，建立数据库连接，并输入SQL语句，如下图：

SQL语句：

SELECT brand,model FROM company c2 
WHERE c2.id 
NOT IN 
(SELECT c1.id FROM suppliers s1 
JOIN company c1 ON s1.brand = c1.brand 
AND s1.model = c1.model 
AND s1.unitPrice=c1.unitPrice);

2.配置“表输入2”控件

双击进入配置页面，具体操作如下：

在SQL框中编写SQL语句，查询出不一致数据在数据表suppliers中的全部信息；在“从步骤插入数据”的下拉框选择“表输入”步骤；勾选“执行每一行”处的复选框，用于将表输入控件流中的数据放入到SQL语句对应的参数位置，通过条件查询与数据表 suppliers中的数据进行匹配，具体配置如图所示。

3.配置“表输出3”控件

双击进入配置页面，配置如下图：

SQL语句：

SELECT id,brand,model,unitPrice FROM company c2 
WHERE c2.id NOT IN 
(SELECT c1.id FROM suppliers s1 
JOIN company c1 ON s1.brand = c1.brand 
AND s1.model = c1.model 
AND s1.unitPrice=c1.unitPrice);