大数据平台运维之Sqoop

在master和slaver节点安装Sqoop Clients，完成后，在master节点查看Sqoop的版本信息，将操作命令和输出结果以文本形式提交到答题框中。

[root@master ~]# sqoop version

Warning: /usr/hdp/2.4.3.0-227/accumulo does not exist!Accumulo imports will fail.

Please set $ACCUMULO_HOME to the root of your Accumuloinstallation.

17/05/07 06:56:25 INFO sqoop.Sqoop: Running Sqoopversion: 1.4.6.2.4.3.0-227

Sqoop 1.4.6.2.4.3.0-227

git commit id d296ad374bd38a1c594ef0f5a2d565d71e798aa6

Compiled by jenkins on Sat Sep 10 00:58:52 UTC 2016

 

38.使用Sqoop工具列出master节点中MySQL中所有数据库，将操作命令和输出结果以文本形式提交到答题框中。

[root@master ~]# sqoop list-databases --connectjdbc:mysql://localhost --username root --password bigdata            

Warning: /usr/hdp/2.4.3.0-227/accumulo does not exist!Accumulo imports will fail.

Please set $ACCUMULO_HOME to the root of your Accumuloinstallation.

17/05/07 07:02:34 INFO sqoop.Sqoop: Running Sqoopversion: 1.4.6.2.4.3.0-227

17/05/07 07:02:34 WARN tool.BaseSqoopTool: Settingyour password on the command-line is insecure. Consider using -P instead.

17/05/07 07:02:34 INFO manager.MySQLManager: Preparingto use a MySQL streaming resultset.

information_schema

ambari

mysql

performance_schema

 

39.使用Sqoop工具列出master节点中MySQL中ambari数据库中所有的数据表，将操作命令和输出结果以文本形式提交到答题框中。

[root@master ~]# sqoop list-tables --connectjdbc:mysql://localhost/ambari --username root --password bigdata 

Warning: /usr/hdp/2.4.3.0-227/accumulo does not exist!Accumulo imports will fail.

Please set $ACCUMULO_HOME to the root of your Accumuloinstallation.

17/05/07 07:07:01 INFO sqoop.Sqoop: Running Sqoopversion: 1.4.6.2.4.3.0-227

17/05/07 07:07:01 WARN tool.BaseSqoopTool: Settingyour password on the command-line is insecure. Consider using -P instead.

17/05/07 07:07:02 INFO manager.MySQLManager: Preparingto use a MySQL streaming resultset.

ClusterHostMapping

QRTZ_BLOB_TRIGGERS

QRTZ_CALENDARS

QRTZ_CRON_TRIGGERS

QRTZ_FIRED_TRIGGERS

QRTZ_JOB_DETAILS

QRTZ_LOCKS

QRTZ_PAUSED_TRIGGER_GRPS

QRTZ_SCHEDULER_STATE

QRTZ_SIMPLE_TRIGGERS

QRTZ_SIMPROP_TRIGGERS

QRTZ_TRIGGERS

adminpermission

adminprincipal

adminprincipaltype

adminprivilege

adminresource

adminresourcetype

alert_current

alert_definition

alert_group

alert_group_target

alert_grouping

alert_history

alert_notice

alert_target

alert_target_states

ambari_sequences

artifact

blueprint

blueprint_configuration

clusterEvent

cluster_version

clusterconfig

clusterconfigmapping

clusters

clusterservices

clusterstate

confgroupclusterconfigmapping

configgroup

configgrouphostmapping

execution_command

groups

hdfsEvent

host_role_command

host_version

hostcomponentdesiredstate

hostcomponentstate

hostconfigmapping

hostgroup

hostgroup_component

hostgroup_configuration

hosts

hoststate

job

kerberos_descriptor

kerberos_principal

kerberos_principal_host

key_value_store

mapreduceEvent

members

metainfo

repo_version

request

requestoperationlevel

requestresourcefilter

requestschedule

requestschedulebatchrequest

role_success_criteria

servicecomponentdesiredstate

serviceconfig

serviceconfighosts

serviceconfigmapping

servicedesiredstate

stack

stage

task

taskAttempt

topology_host_info

topology_host_request

topology_host_task

topology_hostgroup

topology_logical_request

topology_logical_task

topology_request

upgrade

upgrade_group

upgrade_item

users

viewentity

viewinstance

viewinstancedata

viewinstanceproperty

viewmain

viewparameter

viewresource

widget

widget_layout

widget_layout_user_widget

workflow

 

40.在MySQL中创建名为xiandian的数据库，在xiandian数据库中创建xd_phy_course数据表，其数据表结构如表1所示。使用Hive工具来创建数据表xd_phy_course，将phy_course_xd.txt导入到该表中，其中xd_phy_course表的数据结构如表2所示。使用Sqoop工具将hive数据仓库中的xd_phy_course表导出到master节点的MySQL中xiandain 数据库的xd_phy_course表。将以上操作命令和输出结果以文本形式提交到答题框中。

 

[root@master ~]# mysql -uroot -pbigdata

Welcome to the MariaDB monitor.  Commands end with ; or \g.

Your MariaDB connection id is 37

Server version: 5.5.44-MariaDB MariaDB Server

 

Copyright (c) 2000, 2015, Oracle, MariaDB CorporationAb and others.

 

Type 'help;' or '\h' for help. Type '\c' to clear thecurrent input statement.

 

MariaDB [(none)]> create database xiandian;

Query OK, 1 row affected (0.00 sec)

 

MariaDB [(none)]> use xiandian;

Database changed

MariaDB [xiandian]> create table xd_phy_course(stnamevarchar(20),stID int(1),class varchar(20),opt_cour varchar(20));

Query OK, 0 rows affected (0.20 sec)

 

hive> create table xd_phy_course3 (stnamestring,stID int,class string,opt_cour string) row format delimited fieldsterminated by '\t' lines terminated by '\n';

OK

Time taken: 3.136 seconds