StreamSets 3.15 Full RPM 安装教程

StreamSets Data Collector 3.15 Full RPM EL7安装

因为工作需要，尝试在本地虚拟机安装StreamSets Data Collector。StreamSets的种种功能就不赘述，直接进入正题。

下面是详细安装过程：

根据官方文档，以下是安装的前提条件，请确保你的环境符合如下需求。

https://streamsets.com/documentation/datacollector/latest/help/datacollector/UserGuide/Installation/FullInstall_ServiceStart.html#concept_e45_3dr_bx

Installation Requirements

Install Data Collector on a machine that meets the following minimum requirements. To run pipelines in cluster execution mode, each node in the cluster must meet the minimum requirements.

Component	Minimum Requirement
Operating system	Use one of the following operating systems and versions: Mac OS X CentOS 6.x or 7.x Oracle Linux 6.x or 7.x Red Hat Enterprise Linux 6.x or 7.x Ubuntu 14.04 LTS or 16.04 LTS
Cores	2
RAM	1 GB
Disk space	6 GB
File descriptors	32768
Java	Oracle Java 8 or OpenJDK 8
Browser	Use the latest version of one of the following browsers: Chrome Firefox Safari

我本人是虚拟机啥啥没有，一切从头来。

1. 安装centos7 in VMware14 PRO

从官网下载了centos7镜像，然后参照网上的虚拟机安装教程以及上面的安装需求，一步步完成。值得说的就是安装带有GUI的，然后就是空间分配要合理。因为这个StreamSets Full RPM有5个多G。

记得设置本地和虚拟机的共享文件目录，后面有用到。

2.配置File descriptorsulimit -n数量值为32768

可以用下面的命令查看

ulimit -n

在/etc/security/limits.conf中设置的，加入下面两行，完了之后reboot，你就可以再次运行命令查看。

*    soft    nofile 8192
*    hard    nofile 8192

3.配置java环境

无论是OpenJDK 8 还是 Oracle Java 8都可以。Centos7安装好了自带OpenJDK8，只是没有配置JAVA_HOME之类。具体办法请网上搜索。

4.安装StreamSets Data Collector 3.15 Full RPM EL7

就是这么简单，上面三部弄好，你就可以去官网下载RPM包，然后放入共享目录里。我们就可以安装官网的教程开始安装。

我下载的文件名： streamsets-datacollector-3.15.0-el7-activation-all-rpms.tar

官网链接：https://streamsets.com/products/dataops-platform/data-collector/download/

下载页面里，Choose Download Format 选择Full RPM，再点击Full RPM EL7你就可以下载了。文件很大，时间很长。你可以边下载边装虚拟机。

下面是官网的安装步骤，我完全安装它的走。链接如下。重点从下面的第三步开始。给他们标注了浪漫的粉~

https://streamsets.com/documentation/datacollector/latest/help/datacollector/UserGuide/Installation/FullInstall_ServiceStart.html#concept_e45_3dr_bx

Full Installation and Launch (Service Start)

To install the full Data Collector as a service, you can download the Data Collector RPM package or the Data Collector tarball from the StreamSets website.

You can run Data Collector as a service on all supported Linux operating systems.

Installing from the RPM Package

You can install the Data Collector RPM package and start it as a service on CentOS, Oracle Linux, or Red Hat Enterprise Linux.

When you install from the RPM package, Data Collector uses the default directories and runs as the default system user and group.

The default system user and group are named sdc. If an sdc user and an sdc group do not exist on the machine, the installation creates the user and group for you and assigns them the next available user ID and group ID.

Tip: To use specific IDs for the sdc user and group, create the user and group before installation and specify the IDs that you want to use. For example, if you’re installing Data Collector on multiple machines, you might want to create the system user and group before installation to ensure that the user ID and group ID are consistent across the machines.

Installing the full Data Collector as a service requires root privileges.

1. Access the Data Collector RPM package from one of the following locations:下载地址

StreamSets Support portal if you have a StreamSets enterprise license.

StreamSets website if you do not have an enterprise license.

2. Download the RPM package for your operating system:下载适合你系统的包。

For CentOS 6, Oracle Linux 6, or Red Hat Enterprise Linux 6, download the RPM EL6 package.

For CentOS 7, Oracle Linux 7, or Red Hat Enterprise Linux 7, download the RPM EL7 package.

3. Use the following command to extract the file to the desired location。解压缩下载的tar包。

解压可以直接在VMware与本机的sharefolder解压。Centos7的共享目录一会显示一会没有，折腾了很久= =。希望大家有好运气，不能挂载的请自行搜索。

tar xf streamsets-datacollector-<version>-<operating_system>-all-rpms.tar

For example, to extract version 3.15.0 on CentOS 7, use the following command:

tar xf streamsets-datacollector-3.15.0-el7-all-rpms.tar

4. Use the following command to install the full Data Collector RPM package。用下面的命令，安装解压之后的rpm。

我安装的时候直接进入上一步解压之后的目录里运行如下命令，不需要copy到虚拟机的目录。

yum localinstall streamsets*.rpm

安装时请确保网络连通。我碰到过下面这个问题，我安装时是用的NAT网络，参照下面的链接的方法一解决了，需要重启。

小白解决CENTOS7错误:Cannot find a valid baseurl for repo: base/7

5. To start Data Collector as a service, use the required command for your operating system:启动服务，非常简单，照做就行。

For CentOS 6, Oracle Linux 6, or Red Hat Enterprise Linux 6, use:

service sdc start

For CentOS 7, Oracle Linux 7, or Red Hat Enterprise Linux 7, use:

systemctl start sdc

To access the Data Collector UI, enter the following URL in the address bar of your browser:直接用命令hostname获取你的主机名称然后替换下面的地址，去浏览器打开UI。

http://<hostname>:18630/

 

最后，浏览器打开之后，默认用户admin，密码admin登陆。然后会需要提供一些个人信息去激活。邮箱地址可以是个人的，不一定用公司邮箱，之后它会发送一个active key到你的邮箱。把active key复制到页面的框框里，你就可以开始试试，做demo啦。

 

以上就是我的安装经历，自己做个记录，也希望能帮到大家。