Bigdata-Hbase+Geomesa+Geoserver部署并发布地图服务

安装java

1、卸载预装的java：
[root@geomesa geomesa_soft]# rpm -qa|grep java
tzdata-java-2017b-1.el7.noarch
java-1.8.0-openjdk-1.8.0.131-11.b12.el7.x86_64
java-1.7.0-openjdk-1.7.0.141-2.6.10.5.el7.x86_64
javapackages-tools-3.4.1-11.el7.noarch
java-1.8.0-openjdk-headless-1.8.0.131-11.b12.el7.x86_64
python-javapackages-3.4.1-11.el7.noarch
java-1.7.0-openjdk-headless-1.7.0.141-2.6.10.5.el7.x86_64
[root@geomesa geomesa_soft]# rpm -e --nodeps rpm -qa | grep java
[root@geomesa geomesa_soft]# java -version
bash: java: command not found…
2、安装jdk：
[root@geomesa geomesa_soft]# tar -zxvf jdk-8u192-linux-x64.tar.gz -C /home/soft

3、配置环境变量
[root@geomesa geomesa_soft]# vi /etc/profile
export JAVA_HOME=/home/soft/jdk1.8.0_192
export JRE_HOME=$JAV{A}_{H}OME/jreexportCLASSPATH=.:${JAVA_HOME}/lib:$JR{E}_{H}OME/libexportPATH=${JAVA_HOME}/bin:$PATH
[root@geomesa geomesa_soft]# source /etc/profile

验证：
[root@geomesa geomesa_soft]# java -version
java version “1.8.0_192”
Java™ SE Runtime Environment (build 1.8.0_192-b12)
Java HotSpot™ 64-Bit Server VM (build 25.192-b12, mixed mode)

安装hadoop

1、创建用户hadoop
[root@geomesa geomesa_soft]# useradd hadoop
[root@geomesa geomesa_soft]# passwd hadoop 密码hadoop
加入root组：
[root@geomesa geomesa_soft]# usermod -g root hadoop
[root@geomesa geomesa_soft]# id hadoop
uid=1001(hadoop) gid=0(root) groups=0(root)

2、解压文件
[root@geomesa geomesa_soft]# tar -zxvf hadoop-2.8.4.tar.gz -C /home/hadoop
[root@geomesa geomesa_soft]# cd /home/hadoop/
[root@geomesa hadoop]# ls
hadoop-2.8.4
[root@geomesa hadoop]# mv hadoop-2.8.4 hadoop2.8

3、环境变量：
[root@geomesa hadoop]# vi /etc/profile
export JAVA_HOME=/home/soft/jdk1.8.0_192
export JRE_HOME=$JAV{A}_{H}OME/jreexportCLASSPATH=.:${JAVA_HOME}/lib:$JR{E}_{H}OME/libexportHADOO{P}_{H}OME=/home/hadoop/hadoop2.8exportHADOO{P}_{C}OMMO{N}_{L}I{B}_{N}ATIV{E}_{D}IR=$HADOOP_HOME/lib/native
export HADOOP_OPTS="-Djava.library.path=$HADOO{P}_{H}OME/lib"exportPATH=.:${JAVA_HOME}/bin:$HADOO{P}_{H}OME/bin:$PATH
[root@geomesa hadoop]# source /etc/profile

4、创建文件夹
这个创建文件夹是因为root下有更好的安全性
mkdir /root/hadoop
mkdir /root/hadoop/tmp
mkdir /root/hadoop/var
mkdir /root/hadoop/dfs
mkdir /root/hadoop/dfs/name
mkdir /root/hadoop/dfs/data

5、修改hadoop配置文件
[root@geomesa hadoop]# cd /home/hadoop/hadoop2.8/etc/hadoop/

[root@geomesa hadoop]# vim core-site.xml


hadoop.tmp.dir
/root/hadoop/tmp
Abase for other temporary directories.


fs.default.name
hdfs://geomesa:9000

[root@geomesa hadoop]# vim hadoop-env.sh
修改：
export JAVA_HOME=/home/soft/jdk1.8.0_192

[root@geomesa hadoop]# vim hdfs-site.xml


dfs.name.dir
/root/hadoop/dfs/name
Path on the local filesystem where theNameNode stores the namespace and transactions logs persistently.


dfs.data.dir
/root/hadoop/dfs/data
Comma separated list of paths on the localfilesystem of a DataNode where it should store its blocks.


dfs.replication
2


dfs.permissions
false
need not permissions

[root@geomesa hadoop]# cp mapred-site.xml.template mapred-site.xml
[root@geomesa hadoop]# vim mapred-site.xml


mapred.job.tracker
geomesa:9001


mapred.local.dir
/root/hadoop/var


mapreduce.framework.name
yarn

6、启动hadoop
[root@geomesa hadoop]# cd /home/hadoop/hadoop2.8/bin/
[root@geomesa bin]# ./hadoop namenode -format
[root@geomesa bin]# cd /home/hadoop/hadoop2.8/sbin
[root@geomesa sbin]# start-dfs.sh
[root@geomesa sbin]# start-yarn.sh
[root@geomesa sbin]# jps
21744 NameNode
21873 DataNode
22210 ResourceManager
22610 Jps
22499 NodeManager
22037 SecondaryNameNode

安装hbase

1、解压
[root@geomesa geomesa_soft]# mkdir /home/hbase
[root@geomesa geomesa_soft]# tar -zxvf hbase-1.3.3-bin.tar.gz -C /home/hbase

2、环境变量
[root@geomesa geomesa_soft]# vi /etc/profile
export HBASE_HOME=/home/hbase/hbase-1.3.3
export PATH=.:$JAV{A}_{H}OME/bin:${HADOOP_HOME}/bin:$HBAS{E}_{H}OME/bin:$PATH
[root@geomesa geomesa_soft]# source /etc/profile

3、新建文件夹
mkdir /root/hbase
mkdir /root/hbase/tmp
mkdir /root/hbase/pids

4、修改hbase配置文件
[root@geomesa geomesa_soft]# cd /home/hbase/hbase-1.3.3/conf/
[root@geomesa conf]# vi hbase-env.sh
添加：
export JAVA_HOME=/home/soft/jdk1.8.0_192
export HADOOP_HOME=/home/hadoop/hadoop2.8
export HBASE_HOME=/home/hbase/hbase-1.3.3
export HBASE_CLASSPATH=/home/hadoop/hadoop2.8/etc/hadoop
export HBASE_PID_DIR=/root/hbase/pids
export HBASE_MANAGES_ZK=false

[root@geomesa conf]# vi hbase-site.xml

hbase.rootdir hdfs://geomesa:9000/hbase The directory shared byregion servers. hbase.zookeeper.property.clientPort 2181 Property from ZooKeeper'sconfig zoo.cfg. The port at which the clients will connect. zookeeper.session.timeout 120000 hbase.zookeeper.quorum geomesa hbase.tmp.dir /root/hbase/tmp hbase.cluster.distributed false

5、启动hbase
[root@geomesa conf]# cd /home/hbase/hbase-1.3.3/bin/
[root@geomesa bin]# ./start-hbase.sh

访问hbase16010端口可见

安装geomesa

1、安装geomesa
[root@geomesa bin]# cd /root/geomesa_soft/
[root@geomesa geomesa_soft]# tar -zxvf geomesa-hbase_2.11-2.2.1-bin.tar.gz -C /usr/local/

2、配置
[root@geomesa ~]# vi /etc/profile
[root@geomesa ~]# source /etc/profile
添加和修改：
export GEOMESA_HBASE_HOME=/usr/local/geomesa-hbase_2.11-2.2.1/
export PATH=.:$JAV{A}_{H}OME/bin:${HADOOP_HOME}/bin:$HBAS{E}_{H}OME/bin:${GEOMESA_HBASE_HOME}/bin:$PATH

[root@geomesa ~]# cp /usr/local/geomesa-hbase_2.11-2.2.1/dist/hbase/geomesa-hbase-distributed-runtime_2.11-2.2.1.jar /home/hbase/hbase-1.3.3/lib/

添加geomesa环境变量：
[root@geomesa ~]# vi /usr/local/geomesa-hbase_2.11-2.2.1/conf/geomesa-env.sh
export HADOOP_HOME=/home/hadoop/hadoop2.8
export HBASE_HOME=/home/hbase/hbase-1.3.3
export GEOMESA_HBASE_HOME=/usr/local/geomesa-hbase_2.11-2.2.1/
export PATH="$PATH:${GEOMESA_HBASE_HOME}/bin"

3、安装形状文件支持的依赖项
[root@geomesa ~]# cd /usr/local/geomesa-hbase_2.11-2.2.1/
[root@geomesa geomesa-hbase_2.11-2.2.1]# bin/install-jai.sh
[root@geomesa geomesa-hbase_2.11-2.2.1]# bin/install-jline.sh

4、注册Coprocessors
hbase-site.xml

增加：

hbase.coprocessor.user.region.classes
org.locationtech.geomesa.hbase.coprocessor.GeoMesaCoprocessor

5、使用官方提供的测试示例进行测试
[root@geomesa ~]# git clone https://github.com/geomesa/geomesa-tutorials.git
[root@geomesa ~]# cd geomesa-tutorials/
[root@geomesa geomesa-tutorials]# mvn clean install -pl geomesa-tutorials-hbase/geomesa-tutorials-hbase-quickstart -am
。。。。。。。
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO]
[INFO] GeoMesa Tutorials and Examples … SUCCESS [2:09.422s]
[INFO] GeoMesa Tutorials - Common … SUCCESS [12:44.115s]
[INFO] GeoMesa Tutorials - HBase … SUCCESS [15:31.734s]
[INFO] GeoMesa Tutorials - HBase - Quick Start … SUCCESS [48.311s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 31:14.648s
[INFO] Finished at: Fri May 31 20:18:26 CST 2019
[INFO] Final Memory: 98M/732M
[INFO] ------------------------------------------------------------------------

[root@geomesa geomesa-tutorials]# java -cp geomesa-tutorials-hbase/geomesa-tutorials-hbase-quickstart/target/geomesa-tutorials-hbase-quickstart-2.4.0-SNAPSHOT.jar org.geomesa.example.hbase.HBaseQuickStart --hbase.zookeepers master --hbase.catalog geomesa_hbase

注意：
在这一步遇到geomesa java.lang.RuntimeException: Error running quickstart问题，重启hbase、zookeeper解决

使用hbase shell进去查看：
hbase(main):001:0> list
TABLE
geomesa_hbase
geomesa_hbase_gdelt_2dquickstart_attr_EventCode_geom_dtg_v8
geomesa_hbase_gdelt_2dquickstart_id_v4
geomesa_hbase_gdelt_2dquickstart_z2_geom_v5
geomesa_hbase_gdelt_2dquickstart_z3_geom_dtg_v6
5 row(s) in 0.3430 seconds

=> [“geomesa_hbase”, “geomesa_hbase_gdelt_2dquickstart_attr_EventCode_geom_dtg_v8”, “geomesa_hbase_gdelt_2dquickstart_id_v4”, “geomesa_hbase_gdelt_2dquickstart_z2_geom_v5”, “geomesa_hbase_gdelt_2dquickstart_z3_geom_dtg_v6”]

安装tomcat和部署geoserver

一、安装tomcat
1、解压缩
[root@geomesa geomesa_soft]# mkdir /usr/local/tomcat
[root@geomesa geomesa_soft]# cp apache-tomcat-9.0.16.tar.gz /usr/local/tomcat/
[root@geomesa geomesa_soft]# tar -xzvf /usr/local/tomcat/apache-tomcat-9.0.16.tar.gz -C /usr/local/tomcat
[root@geomesa geomesa_soft]# rm /usr/local/tomcat/apache-tomcat-9.0.16.tar.gz
rm: remove regular file ‘/usr/local/tomcat/apache-tomcat-9.0.16.tar.gz’? y
[root@geomesa geomesa_soft]#

2、环境变量
[root@geomesa geomesa_soft]# vi /etc/profile
export CATALINA_HOME=/usr/local/tomcat/apache-tomcat-9.0.16
export CATALINA_BASE=/usr/local/tomcat/apache-tomcat-9.0.16
export PATH=.:$JAV{A}_{H}OME/bin:${HADOOP_HOME}/bin:$HBAS{E}_{H}OME/bin:${GEOMESA_HBASE_HOME}/bin:$M{2}_{H}OME/bin:${CATALINA_BASE}/bin:$PATH
[root@geomesa geomesa_soft]# source /etc/profile

3、创建setenv.sh脚本文件和修改catalina.sh文件
[root@geomesa tomcat]# vi apache-tomcat-9.0.16/bin/setenv.sh
CATALINA_PID="$CATALINA_BASE/tomcat.pid"

[root@geomesa tomcat]# vi apache-tomcat-9.0.16/bin/catalina.sh
#OS specific support. $va{r}_{m}us{t}_{b}esettoeithertrueorfalse.前添加：JAV{A}_{H}OME=/home/soft/jdk1.8.0_{1}92JR{E}_{H}OME=${JAVA_HOME}/jre
4、启动tomcat
[root@geomesa tomcat]# $CATALINA_BASE/bin/startup.sh
Using CATALINA_BASE: /usr/local/tomcat/apache-tomcat-9.0.16
Using CATALINA_HOME: /usr/local/tomcat/apache-tomcat-9.0.16
Using CATALINA_TMPDIR: /usr/local/tomcat/apache-tomcat-9.0.16/temp
Using JRE_HOME: /home/soft/jdk1.8.0_192/jre
Using CLASSPATH: /usr/local/tomcat/apache-tomcat-9.0.16/bin/bootstrap.jar:/usr/local/tomcat/apache-tomcat-9.0.16/bin/tomcat-juli.jar
Using CATALINA_PID: /usr/local/tomcat/apache-tomcat-9.0.16/tomcat.pid
Tomcat started.

之后可将tomcat配置为服务。

二、部署geoserver
[root@geomesa geomesa_soft]# yum install -y unzip
[root@geomesa geomesa_soft]# cd
[root@geomesa ~]# mkdir geoserver
[root@geomesa ~]# unzip /root/geomesa_soft/geoserver-2.14.2-war.zip -d geoserver
[root@geomesa ~]# cp geoserver/geoserver.war $CATALINA_BASE/webapps/

重启tomcat服务
[root@geomesa bin]# $CATALINA_BASE/bin/shutdown.sh
[root@geomesa bin]# $CATALINA_BASE/bin/startup.sh

此时访问localhost:8080/geoserver可以访问，使用默认的用户名 admin 和密码 geoserver 登录主界面

三、整合geoserver
将/usr/local/geomesa-hbase_2.11-2.2.1/dist/gs-plugins/目录下geomesa-hbase-gs-plugin_2.11-2.2.1-install.tar.gz解压到$CATALINA_BASE/webapps/geoserver/WEB-INF/lib ：
[root@geomesa gs-plugins]# tar -zxvf geomesa-hbase-gs-plugin_2.11-2.2.1-install.tar.gz -C $CATALINA_BASE/webapps/geoserver/WEB-INF/lib

另需的相关jar包，到/home/hbase/hbase-1.3.3/lib和hadoop目录找：
[root@geomesa jar]# ls
commons-cli-1.2.jar hbase-client-1.3.3.jar
commons-configuration-1.6.jar hbase-common-1.3.3.jar
commons-io-2.4.jar hbase-protocol-1.3.3.jar
hadoop-annotations-2.5.1.jar hbase-server-1.3.3.jar
hadoop-auth-2.5.1.jar htrace-core-3.1.0-incubating.jar
hadoop-common-2.5.1.jar metrics-core-2.2.0.jar
hadoop-mapreduce-client-core-2.5.1.jar netty-all-4.0.50.Final.jar
hadoop-yarn-api-2.5.1.jar zookeeper-3.4.6.jar
hadoop-yarn-common-2.5.1.jar

这些包在https://mvnrepository.com/可以下载

将这些jar包拷贝至$CATALINA_BASE/webapps/geoserver/WEB-INF/lib

导入数据

[root@geomesa bhutan-latest-free.shp]# geomesa-hbase ingest --catalog gis_osm_buildings_a_free_1 --feature-name gis_osm_buildings_a_free_1 --input-format shp “/root/geomesa_soft/bhutan-latest-free.shp/gis_osm_buildings_a_free_1.shp”

之后可打开主机名：8080/geoserver新建存储并发布地图服务。