How to install CKAN on Linux CentOS 7

Installing CKAN from source — CKAN 2.9.5 documentation

US Third party data catalog: http://datacatalog.pwcinternal.com/catalog/

Install CKAN 2.9 on CentOS 7: https://github.com/ckan/ckan/wiki/How-to-Install-CKAN-2.9-on-CentOS-7

Step 1: Check the Linux CentOS version:

[root@XXXXXXX]# pwd
/etc/yum.repos.d
[root@XXXXXXX]# cp bak/CentOS-Base.repo  CentOS-Base.repo
[root@XXXXXXX]#

Adding the ‘exclude=postgresql*’ to the CentOS-Base.repo


[root@XXXXXXX]# pwd
/etc/yum.repos.d
[root@XXXXXXX]# cat CentOS-Base.repo
# CentOS-Base.repo
#
# The mirror system uses the connecting IP address of the client and the
# update status of each mirror to pick mirrors that are updated to and
# geographically close to the client.  You should use this for CentOS updates
# unless you are manually picking other mirrors.
#
# If the mirrorlist= does not work for you, as a fall back you can try the
# remarked out baseurl= line instead.
#
#

[base]
name=CentOS-$releasever - Base
mirrorlist=http://mirrorlist.centos.org/?release=$releasever&arch=$basearch&repo=os&infra=$infra
#baseurl=http://mirror.centos.org/centos/$releasever/os/$basearch/
gpgcheck=1
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-7

exclude=postgresql*

#released updates
[updates]
name=CentOS-$releasever - Updates
mirrorlist=http://mirrorlist.centos.org/?release=$releasever&arch=$basearch&repo=updates&infra=$infra
#baseurl=http://mirror.centos.org/centos/$releasever/updates/$basearch/
gpgcheck=1
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-7

exclude=postgresql*

#additional packages that may be useful
[extras]
name=CentOS-$releasever - Extras
mirrorlist=http://mirrorlist.centos.org/?release=$releasever&arch=$basearch&repo=extras&infra=$infra
#baseurl=http://mirror.centos.org/centos/$releasever/extras/$basearch/
gpgcheck=1
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-7

#additional packages that extend functionality of existing packages
[centosplus]
name=CentOS-$releasever - Plus
mirrorlist=http://mirrorlist.centos.org/?release=$releasever&arch=$basearch&repo=centosplus&infra=$infra
#baseurl=http://mirror.centos.org/centos/$releasever/centosplus/$basearch/
gpgcheck=1
enabled=0
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-7

[root@XXXXXXX]#

Step 2: Check the PostgreSQL
The PostgreSQL repository includes information for all available PostgreSQL releases. You can see all available packages and versions using the following command:

yum list postgresql*

Our Linux has already installed the PostgreSQL.

See the database in Postgresql

[root@XXXXXXX]# su - postgres
Last login: Mon Jan 17 09:52:47 HKT 2022 on pts/3
-bash-4.2$ psql -l
                                  List of databases
   Name    |  Owner   | Encoding |   Collate   |    Ctype    |   Access privileges
-----------+----------+----------+-------------+-------------+-----------------------
 postgres  | postgres | UTF8     | en_US.UTF-8 | en_US.UTF-8 | =Tc/postgres         +
           |          |          |             |             | postgres=CTc/postgres+
           |          |          |             |             | layer1=CTc/postgres
 template0 | postgres | UTF8     | en_US.UTF-8 | en_US.UTF-8 | =c/postgres          +
           |          |          |             |             | postgres=CTc/postgres
 template1 | postgres | UTF8     | en_US.UTF-8 | en_US.UTF-8 | =c/postgres          +
           |          |          |             |             | postgres=CTc/postgres
(3 rows)


Create a new PostgreSQL user called ckan_default, and enter a password for the user when prompted. You’ll need this password later:  (ckan_default/enjoy2022)
createuser -S -D -R -P ckan_default



-bash-4.2$ createuser -S -D -R -P ckan_default
Enter password for new role:


Create a new PostgreSQL database, called ckan_default, owned by the database user you just created:

-bash-4.2$ createdb -O ckan_default ckan_default -E utf-8
-bash-4.2$

Step 3: Install CKAN

First, create a CKAN User. The ckan user is created with a shell of /sbin/nologin and a home directory of /usr/lib/ckan to mirror what is shown in the CKAN Deployment documentation.

[root@cnshaappulv921 ~]# useradd -m -s /sbin/nologin -d /usr/lib/ckan -c "CKAN User" ckan


Open the newly created directory up for read access so that the content will eventually be able to be served out via httpd.
chmod 755 /usr/lib/ckan
Switch to the ckan user.


[root@cnshaappulv921 ~]# su -s /bin/bash - ckan
[ckan@cnshaappulv921 ~]$

•	Install an isolated Python environment, called default, to host CKAN from.
mkdir -p /usr/lib/ckan/default
chown `whoami` /usr/lib/ckan/default
python3 -m venv /usr/lib/ckan/default
. /usr/lib/ckan/default/bin/activate



[ckan@cnshaappulv921 ~]$ mkdir -p /usr/lib/ckan/default
[ckan@cnshaappulv921 ~]$ whoami
ckan
[ckan@cnshaappulv921 ~]$ chown ckan /usr/lib/ckan/default
[ckan@cnshaappulv921 ~]$ python3.7 -m venv /usr/lib/ckan/default
[ckan@cnshaappulv921 ~]$ . /usr/lib/ckan/default/bin/activate
(default) [ckan@cnshaappulv921 ~]$

Step 4: Install the recommended setuptools version and up-to-date pip:

(default) [ckan@cnshaappulv921 ~]$ pip3.7 install setuptools==44.1.0
Collecting setuptools==44.1.0
Downloading setuptools-44.1.0-py2.py3-none-any.whl (583 kB)
|████████████████████████████████| 583 kB 1.6 MB/s
Installing collected packages: setuptools
Attempting uninstall: setuptools
Found existing installation: setuptools 47.1.0
Uninstalling setuptools-47.1.0:
Successfully uninstalled setuptools-47.1.0
Successfully installed setuptools-44.1.0
WARNING: You are using pip version 20.1.1; however, version 21.3.1 is available.
You should consider upgrading via the ‘/usr/lib/ckan/default/bin/python3.7 -m pip install --upgrade pip’ command.
(default) [ckan@cnshaappulv921 ~]$

(default) [ckan@cnshaappulv921 ~]$ pip3.7 install --upgrade pip
Collecting pip
Downloading pip-21.3.1-py3-none-any.whl (1.7 MB)
|████████████████████████████████| 1.7 MB 1.6 MB/s
Installing collected packages: pip
Attempting uninstall: pip
Found existing installation: pip 20.1.1
Uninstalling pip-20.1.1:
Successfully uninstalled pip-20.1.1
Successfully installed pip-21.3.1
(default) [ckan@cnshaappulv921 ~]$

Step 5: Install the CKAN source code into your virtualenv.

(default) [ckan@cnshaappulv921 ~]$ pip3.7 install -e 'git+https://github.com/ckan/ckan.git@ckan-2.9.0#egg=ckan[requirements]'
Obtaining ckan[requirements] from git+https://github.com/ckan/ckan.git@ckan-2.9.0#egg=ckan[requirements]
  Cloning https://github.com/ckan/ckan.git (to revision ckan-2.9.0) to ./default/src/ckan
  Running command git clone -q https://github.com/ckan/ckan.git /usr/lib/ckan/default/src/ckan
  Running command git checkout -q ba0120dc0c798bbc9b6d8e9ad83db01a197ea179
  Resolved https://github.com/ckan/ckan.git to commit ba0120dc0c798bbc9b6d8e9ad83db01a197ea179
  Installing build dependencies ... done
  Checking if build backend supports build_editable ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Collecting pathtools==0.1.2
  Downloading pathtools-0.1.2.tar.gz (11 kB)
  Preparing metadata (setup.py) ... error
  ERROR: Command errored out with exit status 1:
   command: /usr/lib/ckan/default/bin/python3.7 -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-g2lwtw1e/pathtools_7b86338768fb4b3b952515a38e5c36f7/setup.py'"'"'; __file__='"'"'/tmp/pip-install-g2lwtw1e/pathtools_7b86338768fb4b3b952515a38e5c36f7/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base /tmp/pip-pip-egg-info-gjn38arx
       cwd: /tmp/pip-install-g2lwtw1e/pathtools_7b86338768fb4b3b952515a38e5c36f7/
  Complete output (9 lines):
  running egg_info
  creating /tmp/pip-pip-egg-info-gjn38arx/pathtools.egg-info
  writing /tmp/pip-pip-egg-info-gjn38arx/pathtools.egg-info/PKG-INFO
  writing dependency_links to /tmp/pip-pip-egg-info-gjn38arx/pathtools.egg-info/dependency_links.txt
  writing top-level names to /tmp/pip-pip-egg-info-gjn38arx/pathtools.egg-info/top_level.txt
  writing manifest file '/tmp/pip-pip-egg-info-gjn38arx/pathtools.egg-info/SOURCES.txt'
  /tmp/pip-install-g2lwtw1e/pathtools_7b86338768fb4b3b952515a38e5c36f7/setup.py:25: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses
    import imp
  error: [Errno 13] Permission denied: '/usr/local/lib/python3.7/lib2to3/Grammar3.7.11.final.0.pickle'
  ----------------------------------------
WARNING: Discarding https://files.pythonhosted.org/packages/e7/7f/470d6fcdf23f9f3518f6b0b76be9df16dcc8630ad409947f8be2eb0ed13a/pathtools-0.1.2.tar.gz#sha256=7c35c5421a39bb82e58018febd90e3b6e5db34c5443aaaf742b3f33d4655f1c0 (from https://pypi.org/simple/pathtools/). Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.
ERROR: Could not find a version that satisfies the requirement pathtools==0.1.2; extra == "requirements" (from ckan[requirements]) (from versions: 0.1.0, 0.1.1, 0.1.2)
ERROR: No matching distribution found for pathtools==0.1.2; extra == "requirements"
(default) [ckan@cnshaappulv921 ~]$




[root@cnshaappulv921 ~]# chmod 777 /usr/local/lib/python3.7/lib2to3/Grammar3.7.11.final.0.pickle
[root@cnshaappulv921 ~]#
[root@cnshaappulv921 ~]# su -s /bin/bash - ckan
Last login: Mon Jan 17 13:45:48 HKT 2022 on pts/3
[ckan@cnshaappulv921 ~]$ . /usr/lib/ckan/default/bin/activate
(default) [ckan@cnshaappulv921 ~]$



(default) [ckan@cnshaappulv921 ~]$ pwd
/usr/lib/ckan
(default) [ckan@cnshaappulv921 ~]$
(default) [ckan@cnshaappulv921 ~]$ pip3.7 install -e 'git+https://github.com/ckan/ckan.git@ckan-2.9.0#egg=ckan'
Obtaining ckan from git+https://github.com/ckan/ckan.git@ckan-2.9.0#egg=ckan
  Updating ./default/src/ckan clone (to revision ckan-2.9.0)
  Running command git fetch -q
  Running command git reset --hard -q ba0120dc0c798bbc9b6d8e9ad83db01a197ea179
  Installing build dependencies ... done
  Checking if build backend supports build_editable ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Installing collected packages: ckan
  Running setup.py develop for ckan
Successfully installed ckan-2.9.0
(default) [ckan@cnshaappulv921 ~]$

Step 6: Create a directory to contain the site’s config files

[root@cnshaappulv921 ~]# sudo mkdir -p /etc/ckan/default
[root@cnshaappulv921 ~]# sudo chown -R ckan /etc/ckan/
[root@cnshaappulv921 ~]#

Edit the ckan.ini file in a text editor, changing the following options:

sqlalchemy.url

This should refer to the database and password that you created above.

Step 7: Setup Apache SOLR

[root@cnshaappulv921 opt]# pwd
/opt
[root@cnshaappulv921 opt]# wget https://archive.apache.org/dist/lucene/solr/6.5.1/solr-6.5.1.tgz
--2022-01-17 14:25:27--  https://archive.apache.org/dist/lucene/solr/6.5.1/solr-6.5.1.tgz
Resolving archive.apache.org (archive.apache.org)... 138.201.131.134, 2a01:4f8:172:2ec5::2
Connecting to archive.apache.org (archive.apache.org)|138.201.131.134|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 148699036 (142M) [application/x-gzip]
Saving to: ‘solr-6.5.1.tgz’

100%[================================================================================================================>] 148,699,036  763KB/s   in 3m 1s

2022-01-17 14:28:29 (802 KB/s) - ‘solr-6.5.1.tgz’ saved [148699036/148699036]

[root@cnshaappulv921 opt]#


[root@cnshaappulv921 opt]# tar xzf solr-6.5.1.tgz solr-6.5.1/bin/install_solr_service.sh --strip-components=2
[root@cnshaappulv921 opt]# ls -l
total 145232
-rwxr-xr-x 1 root     root         12601 Apr 21  2017 install_solr_service.sh
drwxr-xr-x 7 root     root           173 Dec 17 17:20 managesoft
drwxr-xr-x 4 root     root            40 Jan 12 07:41 microsoft
-rw------- 1 root     root     148699036 Apr 21  2017 solr-6.5.1.tgz
drwxr-xr-x 9 splunker splunker       231 Dec 20 13:58 splunkforwarder
[root@cnshaappulv921 opt]# bash ./install_solr_service.sh solr-6.5.1.tgz
Solr requires java, please install or set JAVA_HOME properly
[root@cnshaappulv921 opt]#



https://www.devopsroles.com/install-jdk-on-centos-step-by-step/



(default) [ckan@cnshaappulv921 default]$ whereis ckan
ckan: /usr/lib/ckan /etc/ckan /usr/lib/ckan/default/bin/ckan
(default) [ckan@cnshaappulv921 default]$ 

 

--------------------大数据小团队--------------------
蒙昭良：《Oracle DBA高效入门指南》作者
2013-2019: 摩根大通 – 美国哥伦布
2021-目前 : 普华永道 – 北京
大数据小团队论坛

蒙昭良1021×1189 148 KB

--------------------大数据小团队--------------------