sqoop之hive数据导入mysql

最新推荐文章于 2024-09-12 20:31:11 发布
最新推荐文章于 2024-09-12 20:31:11 发布
阅读量584 收藏 4
点赞数 1
分类专栏： # Sqoop 大数据 文章标签： sqoop hive mysql
版权声明：本文为博主原创文章，遵循 CC 4.0 BY-SA 版权协议，转载请附上原文出处链接和本声明。
本文链接：https://blog.csdn.net/hyj_king/article/details/107024526
版权

1、查询hive表中的student数据及信息

hive> select * from student;
OK
001	xiaohong	002
002	xiaolan	002
001	xiaohong	003
002	xiaolan	003
Time taken: 0.114 seconds, Fetched: 4 row(s)
hive> desc formatted student;
OK
# col_name            	data_type           	comment             
	 	 
id                  	string              	                    
name                	string              	                    
	 	 
# Partition Information	 	 
# col_name            	data_type           	comment             
	 	 
classroom           	string              	                    
	 	 
# Detailed Table Information	 	 
Database:           	default             	 
Owner:              	root                	 
CreateTime:         	Thu Mar 19 14:00:19 CST 2020	 
LastAccessTime:     	UNKNOWN             	 
Retention:          	0                   	 
Location:           	hdfs://master:9000/user/hive/warehouse/student	 
Table Type:         	MANAGED_TABLE       	 
Table Parameters:	 	 
	transient_lastDdlTime	1584597619          
	 	 
# Storage Information	 	 
SerDe Library:      	org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe	 
InputFormat:        	org.apache.hadoop.mapred.TextInputFormat	 
OutputFormat:       	org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat	 
Compressed:         	No                  	 
Num Buckets:        	-1                  	 
Bucket Columns:     	[]                  	 
Sort Columns:       	[]                  	 
Storage Desc Params:	 	 
	field.delim         	,                   
	serialization.format	,                   
Time taken: 0.065 seconds, Fetched: 32 row(s)

从上面的信息可知，student是以classroom为分区进行存储的；数据存储位置为/user/hive/warehouse/student；

2、查看hive中student表数据的存储位置

[root@master ~]# hdfs dfs -ls /user/hive/warehouse/student/
Found 2 items
drwxr-xr-x   - root supergroup          0 2020-03-19 14:04 /user/hive/warehouse/student/classroom=002
drwxr-xr-x   - root supergroup          0 2020-03-19 14:10 /user/hive/warehouse/student/classroom=003
[root@master ~]# hdfs dfs -ls /user/hive/warehouse/student/classroom=002
Found 1 items
-rwxr-xr-x   1 root supergroup         25 2020-03-19 14:04 /user/hive/warehouse/student/classroom=002/stu.txt
[root@master ~]# hdfs dfs -cat /user/hive/warehouse/student/classroom=002/stu.txt
001,xiaohong
002,xiaolan

由此可知根据逗号分隔数据；

3、先在mysql中创建字段和类型相同的表

mysql> create table student(id varchar(255),name varchar(255));
Query OK, 0 rows affected (0.02 sec)

mysql> select * from student;
Empty set (0.00 sec)

mysql>

4、sqoop导入数据

[root@master bin]# ./sqoop export --connect "jdbc:mysql://192.168.230.21:3306/mysql" --username root --password 123456 --table student  --fields-terminated-by "," --export-dir "/user/hive/warehouse/student/classroom=002";
Warning: /opt/softWare/sqoop/sqoop-1.4.7/../hcatalog does not exist! HCatalog jobs will fail.
Please set $HCAT_HOME to the root of your HCatalog installation.
Warning: /opt/softWare/sqoop/sqoop-1.4.7/../accumulo does not exist! Accumulo imports will fail.
Please set $ACCUMULO_HOME to the root of your Accumulo installation.
Warning: /opt/softWare/sqoop/sqoop-1.4.7/../zookeeper does not exist! Accumulo imports will fail.
Please set $ZOOKEEPER_HOME to the root of your Zookeeper installation.
20/06/29 17:27:57 INFO sqoop.Sqoop: Running Sqoop version: 1.4.7
20/06/29 17:27:58 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure. Consider using -P instead.
20/06/29 17:27:58 INFO manager.MySQLManager: Preparing to use a MySQL streaming resultset.
20/06/29 17:27:58 INFO tool.CodeGenTool: Beginning code generation
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/opt/softWare/hadoop/hadoop-2.7.3/share/hadoop/common/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/opt/softWare/hbase/hbase-1.2.6/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/opt/softWare/hive/apache-hive-2.1.1-bin/lib/log4j-slf4j-impl-2.4.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
20/06/29 17:27:58 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM `student` AS t LIMIT 1
20/06/29 17:27:58 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM `student` AS t LIMIT 1
20/06/29 17:27:58 INFO orm.CompilationManager: HADOOP_MAPRED_HOME is /opt/softWare/hadoop/hadoop-2.7.3
Note: /tmp/sqoop-root/compile/ee585e722d7b5a438d67d53c369ed36e/student.java uses or overrides a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
20/06/29 17:28:00 INFO orm.CompilationManager: Writing jar file: /tmp/sqoop-root/compile/ee585e722d7b5a438d67d53c369ed36e/student.jar
20/06/29 17:28:00 INFO mapreduce.ExportJobBase: Beginning export of student
20/06/29 17:28:00 INFO Configuration.deprecation: mapred.jar is deprecated. Instead, use mapreduce.job.jar
20/06/29 17:28:01 INFO Configuration.deprecation: mapred.reduce.tasks.speculative.execution is deprecated. Instead, use mapreduce.reduce.speculative
20/06/29 17:28:01 INFO Configuration.deprecation: mapred.map.tasks.speculative.execution is deprecated. Instead, use mapreduce.map.speculative
20/06/29 17:28:01 INFO Configuration.deprecation: mapred.map.tasks is deprecated. Instead, use mapreduce.job.maps
20/06/29 17:28:01 INFO client.RMProxy: Connecting to ResourceManager at master/192.168.230.21:8032
20/06/29 17:28:06 INFO input.FileInputFormat: Total input paths to process : 1
20/06/29 17:28:06 INFO input.FileInputFormat: Total input paths to process : 1
20/06/29 17:28:06 INFO mapreduce.JobSubmitter: number of splits:4
20/06/29 17:28:06 INFO Configuration.deprecation: mapred.map.tasks.speculative.execution is deprecated. Instead, use mapreduce.map.speculative
20/06/29 17:28:07 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1593394152340_0027
20/06/29 17:28:07 INFO impl.YarnClientImpl: Submitted application application_1593394152340_0027
20/06/29 17:28:07 INFO mapreduce.Job: The url to track the job: http://master:8088/proxy/application_1593394152340_0027/
20/06/29 17:28:07 INFO mapreduce.Job: Running job: job_1593394152340_0027
20/06/29 17:28:16 INFO mapreduce.Job: Job job_1593394152340_0027 running in uber mode : false
20/06/29 17:28:16 INFO mapreduce.Job:  map 0% reduce 0%
20/06/29 17:28:37 INFO mapreduce.Job:  map 50% reduce 0%
20/06/29 17:28:38 INFO mapreduce.Job:  map 100% reduce 0%
20/06/29 17:28:38 INFO mapreduce.Job: Job job_1593394152340_0027 completed successfully
20/06/29 17:28:38 INFO mapreduce.Job: Counters: 30
	File System Counters
		FILE: Number of bytes read=0
		FILE: Number of bytes written=551488
		FILE: Number of read operations=0
		FILE: Number of large read operations=0
		FILE: Number of write operations=0
		HDFS: Number of bytes read=764
		HDFS: Number of bytes written=0
		HDFS: Number of read operations=19
		HDFS: Number of large read operations=0
		HDFS: Number of write operations=0
	Job Counters 
		Launched map tasks=4
		Data-local map tasks=4
		Total time spent by all maps in occupied slots (ms)=71778
		Total time spent by all reduces in occupied slots (ms)=0
		Total time spent by all map tasks (ms)=71778
		Total vcore-milliseconds taken by all map tasks=71778
		Total megabyte-milliseconds taken by all map tasks=73500672
	Map-Reduce Framework
		Map input records=2
		Map output records=2
		Input split bytes=681
		Spilled Records=0
		Failed Shuffles=0
		Merged Map outputs=0
		GC time elapsed (ms)=799
		CPU time spent (ms)=3130
		Physical memory (bytes) snapshot=445382656
		Virtual memory (bytes) snapshot=8325513216
		Total committed heap usage (bytes)=170655744
	File Input Format Counters 
		Bytes Read=0
	File Output Format Counters 
		Bytes Written=0
20/06/29 17:28:38 INFO mapreduce.ExportJobBase: Transferred 764 bytes in 36.6371 seconds (20.8532 bytes/sec)
20/06/29 17:28:38 INFO mapreduce.ExportJobBase: Exported 2 records.
[root@master bin]# ./sqoop export --connect "jdbc:mysql://192.168.230.21:3306/mysql" --username root --password 123456 --table student  --fields-terminated-by "," --export-dir "/user/hive/warehouse/student/classroom=003";
Warning: /opt/softWare/sqoop/sqoop-1.4.7/../hcatalog does not exist! HCatalog jobs will fail.
Please set $HCAT_HOME to the root of your HCatalog installation.
Warning: /opt/softWare/sqoop/sqoop-1.4.7/../accumulo does not exist! Accumulo imports will fail.
Please set $ACCUMULO_HOME to the root of your Accumulo installation.
Warning: /opt/softWare/sqoop/sqoop-1.4.7/../zookeeper does not exist! Accumulo imports will fail.
Please set $ZOOKEEPER_HOME to the root of your Zookeeper installation.
20/06/29 17:29:21 INFO sqoop.Sqoop: Running Sqoop version: 1.4.7
20/06/29 17:29:21 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure. Consider using -P instead.
20/06/29 17:29:22 INFO manager.MySQLManager: Preparing to use a MySQL streaming resultset.
20/06/29 17:29:22 INFO tool.CodeGenTool: Beginning code generation
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/opt/softWare/hadoop/hadoop-2.7.3/share/hadoop/common/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/opt/softWare/hbase/hbase-1.2.6/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/opt/softWare/hive/apache-hive-2.1.1-bin/lib/log4j-slf4j-impl-2.4.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
20/06/29 17:29:22 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM `student` AS t LIMIT 1
20/06/29 17:29:22 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM `student` AS t LIMIT 1
20/06/29 17:29:22 INFO orm.CompilationManager: HADOOP_MAPRED_HOME is /opt/softWare/hadoop/hadoop-2.7.3
Note: /tmp/sqoop-root/compile/b4bafb65b47b2cded3caa3989b6e4d00/student.java uses or overrides a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
20/06/29 17:29:24 INFO orm.CompilationManager: Writing jar file: /tmp/sqoop-root/compile/b4bafb65b47b2cded3caa3989b6e4d00/student.jar
20/06/29 17:29:24 INFO mapreduce.ExportJobBase: Beginning export of student
20/06/29 17:29:24 INFO Configuration.deprecation: mapred.jar is deprecated. Instead, use mapreduce.job.jar
20/06/29 17:29:25 INFO Configuration.deprecation: mapred.reduce.tasks.speculative.execution is deprecated. Instead, use mapreduce.reduce.speculative
20/06/29 17:29:25 INFO Configuration.deprecation: mapred.map.tasks.speculative.execution is deprecated. Instead, use mapreduce.map.speculative
20/06/29 17:29:25 INFO Configuration.deprecation: mapred.map.tasks is deprecated. Instead, use mapreduce.job.maps
20/06/29 17:29:25 INFO client.RMProxy: Connecting to ResourceManager at master/192.168.230.21:8032
20/06/29 17:29:31 INFO input.FileInputFormat: Total input paths to process : 1
20/06/29 17:29:31 INFO input.FileInputFormat: Total input paths to process : 1
20/06/29 17:29:31 INFO mapreduce.JobSubmitter: number of splits:4
20/06/29 17:29:31 INFO Configuration.deprecation: mapred.map.tasks.speculative.execution is deprecated. Instead, use mapreduce.map.speculative
20/06/29 17:29:32 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1593394152340_0028
20/06/29 17:29:32 INFO impl.YarnClientImpl: Submitted application application_1593394152340_0028
20/06/29 17:29:32 INFO mapreduce.Job: The url to track the job: http://master:8088/proxy/application_1593394152340_0028/
20/06/29 17:29:32 INFO mapreduce.Job: Running job: job_1593394152340_0028
20/06/29 17:29:41 INFO mapreduce.Job: Job job_1593394152340_0028 running in uber mode : false
20/06/29 17:29:41 INFO mapreduce.Job:  map 0% reduce 0%
20/06/29 17:29:52 INFO mapreduce.Job:  map 50% reduce 0%
20/06/29 17:29:54 INFO mapreduce.Job:  map 75% reduce 0%
20/06/29 17:29:55 INFO mapreduce.Job:  map 100% reduce 0%
20/06/29 17:29:55 INFO mapreduce.Job: Job job_1593394152340_0028 completed successfully
20/06/29 17:29:56 INFO mapreduce.Job: Counters: 31
	File System Counters
		FILE: Number of bytes read=0
		FILE: Number of bytes written=551488
		FILE: Number of read operations=0
		FILE: Number of large read operations=0
		FILE: Number of write operations=0
		HDFS: Number of bytes read=764
		HDFS: Number of bytes written=0
		HDFS: Number of read operations=19
		HDFS: Number of large read operations=0
		HDFS: Number of write operations=0
	Job Counters 
		Launched map tasks=4
		Data-local map tasks=2
		Rack-local map tasks=2
		Total time spent by all maps in occupied slots (ms)=35902
		Total time spent by all reduces in occupied slots (ms)=0
		Total time spent by all map tasks (ms)=35902
		Total vcore-milliseconds taken by all map tasks=35902
		Total megabyte-milliseconds taken by all map tasks=36763648
	Map-Reduce Framework
		Map input records=2
		Map output records=2
		Input split bytes=681
		Spilled Records=0
		Failed Shuffles=0
		Merged Map outputs=0
		GC time elapsed (ms)=500
		CPU time spent (ms)=2940
		Physical memory (bytes) snapshot=448909312
		Virtual memory (bytes) snapshot=8319729664
		Total committed heap usage (bytes)=166592512
	File Input Format Counters 
		Bytes Read=0
	File Output Format Counters 
		Bytes Written=0
20/06/29 17:29:56 INFO mapreduce.ExportJobBase: Transferred 764 bytes in 31.064 seconds (24.5944 bytes/sec)
20/06/29 17:29:56 INFO mapreduce.ExportJobBase: Exported 2 records.

5、查看mysql中student表数据结果

mysql> select * from student;
+------+----------+
| id   | name     |
+------+----------+
| 001  | xiaohong |
| 002  | xiaolan  |
| 001  | xiaohong |
| 002  | xiaolan  |
+------+----------+
4 rows in set (0.00 sec)

mysql>

 

郝少
关注
专栏目录
使用Sqoop实现Mysql到Hive的数据导入
QYHuiiQ
05-11 6248
使用sqoop将mysql数据导入至hive的本质，其实是数据先导入至hdfs，然后会有一个脚本去执行将hdfs的数据load至hive中。所以我们将数据导入到hive时，其实hdfs中也会有这个数据。 使用sqoop将数据导入至hive时，需要用到一个hive-common-xxx.jar下面的一个类，所以我们需要先将这个jar包上传至sqoop的lib目录下： 添加jar [root@hadoop01 ~]# cp $HIVE_HOME/lib/hive-common-2.3.9.jar $SQO
使用Sqoop实现HIve分析结果数据入mysql库
lrf2454224026的博客
09-11 1141
1.将hive作业中的结果，使用sqoop 导入mysql数据库。        创建stock表并导入数据                        创建stock_result表来存放stock的查询结果              自定义jar包并创建相应的函数（这一步走可以省略，可以使用hive自带的函数）                     将查询结果导入stoc...
Sqoop将hive处理的数据导出到MySQL
weixin_62369128的博客
05-11 4240
sqoop将hive数据导出到mysql
从hive导出数据到mysql（超详细）
最新发布
m0_58419490的博客
09-12 378
用kettle把hive中的数据导出到mysql
sqoop将hive中数据导入mysql（实例讲解）
Saint
07-02 4022
假设这样一个场景： 数据已经导入hive（hive实例讲解看这里），现在要把hive中数据库导入出到（mysql）关系型数据库中，这里使用的工具是sqoop(sqoop详细讲解看这里)。 该怎么处理呢？ 首先，我们需要的是在mysql中建立对应的表，数据字段类型与hive数据表中要导出的数据类型保持一致； 然后，我们需要在假如你不是需要原样输出而是需要经过计算统计的话，就需要写在h...
Hive中使用sql的注意事项
weixin_30572613的博客
09-13 353
一、别名的使用 1 定义别名：columnA as X 不需要使用单引号 2 使用别名：不与where同时使用 花式报错-->有说hive不支持where后使用别名 二、GROUP BY select m.makeName ,SUM(c) from (select t.makeId as id ,count(o...
大数据教程（13.6）sqoop使用教程
weixin_34162695的博客
03-18 580
2019独角兽企业重金招聘Python工程师标准>>> ...
sqoop将hive数据导入mysql报错
XIAOMO__的博客
01-29 1992
sqoop export \ --connect 'jdbc:mysql://192.168.1.135:3306/safe_manager?useUnicode=true&characterEncoding=utf-8' \ --username root \ --password Free-Wi11 \ --table bigscreen_line1 \ --input-null-string '\\N' --input-null-non-string '\\N' \ --export...
sqoop从mysql中导入数据到parquet格式的hive中
07-07
sqoop导入数据到hive
hive源数据通过sqoop数据集成工具导入到mysql报：ERROR tool.ExportTool: Error during export
hellowangxiansheng的博客
04-07 626
除了hive表结构之外，我们再检查一下hive的数据，在hdfs/user/hive/warehouse/co中，可以在linux中hadoop fs -cat /user/hive/warehouse/co查看，为了方便我这里选择在windows中下载查看。在检查后我发现两个表结构是完全一致的，也有可能是hive的分隔符弄错了（–input-fields-terminated-by ‘,’），检查后也没有问题，hive中确实使用建表的’,’作为分隔符，那为什么会报错嘞？desc [表名];
在使用Sqoop将数据从Hive导入MySQL过程中，出现一下错误
大JAVA解决方案
05-03 3995
ERROR manager.SqlManager: Error executing statement: java.sql.SQLException: Access denied for user '2018年04月08日 17:39:22阅读数：35在使用Sqoop将数据从Hive导入MySQL过程中，出现一下错误Warning: /usr/local/sqoop/../hcatalog doe...
sqoop hive mysql 拒接连接
weixin_39712314的博客
12-13 661
2111 JobHistoryServer [hadoop@namenode1 ~]$ sqoop export --connect jdbc:mysql://localhost:3306/dbtaobao --username root --table user_log --export-dir '/usr/local/hive/warehouse/dbtaobao.db/inner_user_...
sqoop数据采集
枣泥馅的博客
07-03 1480
什么是sqoop sqoop的全称是SQL-To-Hadoop，它是一个关系型数据库和Hadoop之间数据交换的工具，它从sql导入到hadoop的过程叫做import，从hadoop导出sql的过程叫做export。import和export都是基于mapreduce的 安装Sqoop sqoop的安装非常节点，只需要把它的包解压后，再配置一下环境变量就可以了 如果采集Oracle中的数据，需要把Oracle安装包中lib的jar放入sqoop的lib目录中我的虚拟机中oracle的jdbc包ojdbc1
SQL注入—我是如何一步步攻破一家互联网公司的
wetest_tencent的博客
04-15 1179
本文由腾讯WeTest团队提供，更多资讯可直接戳链接查看：http://wetest.qq.com/lab/ 微信号：TencentWeTest最近在研究Web安全相关的知识，特别是SQL注入类的相关知识。接触了一些与SQL注入相关的工具。周末在家闲着无聊，想把平时学的东东结合起来攻击一下身边某个小伙伴去的公司，看看能不能得逞。不试不知道，一试还真TM得逞了，内心有些小激动，特在此写博文一篇，...
使用Sqoop将Hive表数据导入到mysql
热门推荐
jun_life的专栏
06-14 2万+
通过Sqoop将Hive表数据导入到Mysql通常有两种情况。 第一种是将hive上某张表的全部数据导入到mysql对应的表中。 第二种是将hive上某张表中的部分数据导入到mysql对应的表中。 两种方式的区别在于第二种情况需要指定要导入数据的列名称。两种情况的导入方式分别如下： 1.全部导入 Sqoop export --connect  jdbc:mysql://127.0.0.1
SQL Programming（3）：Executing SQL Statements
u012501350的专栏
02-25 509
QSqlQuery 类为执行SQL语句和浏览查询结果上提供了交互接口。     QSqlQueryModel类和QSqlTableModel在访问数据库上提供了更高水平的接口，这两个类将在下一部分进行说明。如果你不熟悉SQL，那么你可以直接查看下一部分（Using the SQL Model Class）。 执行一条语句 要执行一条SQL语句，你可以创建一个QSqlQuery
linux查看sqoop目录,Sqoop安装和简单使用
weixin_39639568的博客
04-30 373
下载sqoop因为官方并不建议在生产环境中使用sqoop2，即1.99.7，所以这里我用的是sqoop1,即1.4.7点击下载：http://mirror.bit.edu.cn/apache/sqoop/解压和上传我用的xshell辅助工具，先在本地解压改了名称之后上传到Linux上的sqoop.png上传linuxlinux下.png也可上传压缩包，执行解压缩，执行命令：cd /home/...
Sqoop向hdfs上传文件时报错
qq_42714108的博客
07-15 600
INFO client.RMProxy: Connecting to ResourceManager at master/192.168.49.200:8032 INFO ipc.Client: Retrying connect to server: master/192.168.49.200:8032. Already tried 0 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=100, sleepTime
linux查看sqoop目录,Linux安装部署Sqoop
weixin_42356844的博客
04-30 195
下载sqoopwget http://apache-mirror.8birdsvideo.com/sqoop/1.4.7/sqoop-1.4.7.tar.gz解压重命名[root@hadoop001 software]# tar -xzvf sqoop-1.4.7.tar.gz[root@hadoop001 software]# mv sqoop-1.4.7/ sqoopMySQL JDBC驱动包...
sqoop将hive数据导入mysql
03-16
sqoop是一种用于在Hadoop和关系型数据库之间传输数据的工具。要将Hive数据导入MySQL，可以使用以下步骤： 1. 确保已经安装了sqoop和MySQL驱动程序。 2. 在Hive中创建一个表，并将其导出为一个文件。例如，可以使用以下命令将Hive表导出为一个CSV文件： ``` hive -e 'SELECT * FROM my_table' | sed 's/[\t]/,/g' > my_table.csv ``` 3. 使用sqoop将CSV文件导入MySQL。例如，可以使用以下命令将CSV文件导入MySQL中的一个表： ``` sqoop import --connect jdbc:mysql://localhost/my_database --username my_username --password my_password --table my_table --fields-terminated-by ',' --lines-terminated-by '\n' --input-null-string '\\N' --input-null-non-string '\\N' --delete-target-dir --target-dir /tmp/my_table --bindir /tmp/my_table/bin --outdir /tmp/my_table/out --input-fields-terminated-by ',' ``` 其中，--connect指定MySQL连接字符串，--username和--password指定MySQL用户名和密码，--table指定要导入的MySQL表名，--fields-terminated-by指定CSV文件中字段之间的分隔符，--lines-terminated-by指定CSV文件中行之间的分隔符，--input-null-string和--input-null-non-string指定CSV文件中的空值表示方式，--delete-target-dir指定在导入之前删除目标目录，--target-dir指定HDFS中的目标目录，--bindir、--outdir和--input-fields-terminated-by指定生成的Java类的位置和属性分隔符。 4. 确认数据已经成功导入MySQL中的表中。
评论 1
添加红包

请填写红包祝福语或标题

个

红包个数最小为10个

元

红包金额最低5元

当前余额3.43元 前往充值 >
需支付：10.00元
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包

打赏作者

郝少

你的鼓励将是我创作的最大动力

¥1 ¥2 ¥4 ¥6 ¥10 ¥20
扫码支付：¥1
获取中
扫码支付

您的余额不足，请更换扫码支付或充值

打赏作者

实付元
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明：

1.余额是钱包充值的虚拟货币，按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载，可以购买VIP、付费专栏及课程。

余额充值