大数据学习——配置环境

 

ubuntu虚拟机安装https://blog.csdn.net/Andrewniu/article/details/88743846

jvm+hadoop配置https://blog.csdn.net/kh896424665/article/details/78765175

jvm+hadoop+hbase配置https://blog.csdn.net/wuruiaoxue/article/details/54833621

mysql安装https://blog.csdn.net/weixin_42209572/article/details/98983741

hive安装https://www.cnblogs.com/xiaolan-Lin/p/11358901.html

hive中conf/hive-env.sh

# Set HADOOP_HOME to point to a specific hadoop install directory
HADOOP_HOME=/home/hadoop/bigdata/hadoop

# Hive Configuration Directory can be controlled by:
export HIVE_CONF_DIR=/home/hadoop/bigdata/hive/conf

# Folder containing extra libraries required for hive compilation/execution can be controlled by:
export HIVE_AUX_JARS_PATH=/home/hadoop/bigdata/hive/lib
"hive-env.sh" 54L, 2426C 

hive中conf/hive-site.sh

<configuration>
    <!-- Hive产生的元数据在hdfs存放位置-->
<property>
    <name>hive.metastore.warehouse.dir</name>
    <value>/user/hive/warehouse</value>
</property>
    <!--- 使用本地服务连接Hive,默认为true-->
<property>
    <name>hive.metastore.local</name>
    <value>true</value>
</property>

    <!-- 数据库连接JDBC的URL地址-->
<property>
    <name>javax.jdo.option.ConnectionURL</name>
    <value>jdbc:mysql://localhost:3306/hive?useSSL=false</value>
</property>
    <!-- 数据库连接driver，即MySQL驱动-->
<property>
    <name>javax.jdo.option.ConnectionDriverName</name>
    <value>com.mysql.cj.jdbc.Driver</value>
</property>
    <!-- MySQL数据库用户名-->
<property>
    <name>javax.jdo.option.ConnectionUserName</name>
    <value>root</value>
</property>
    <!-- MySQL数据库密码-->
<property>
    <name>javax.jdo.option.ConnectionPassword</name>
    <value> </value>
 </property>
<property>
    <name>hive.metastore.schema.verification</name>
    <value>false</value>
 </property>
<property>
    <name>datanucleus.schema.autoCreateAll</name>
    <value>true</value>
 </property>
</configuration>

目前在官网下载低于jdk1.8的java jdk和mysql jar包的时候需要登陆，借用别人账号

账号:  13707259624@163.com
密码：Java123456

安装zookeeper https://blog.csdn.net/fengchen0123456789/article/details/82220522

                         https://blog.csdn.net/u010889616/article/details/80171307

                         https://www.cnblogs.com/biehongli/p/7650570.html

安装Hbase https://blog.csdn.net/June_Xixi/article/details/88880981

安装sqoop https://www.jianshu.com/p/44ffe378c69a

安装kafka https://blog.csdn.net/u010889616/article/details/80641922

                 https://www.cnblogs.com/justuntil/p/8033792.html

安装flume