初识ClickHouse

ClickHouse 官网

1. ClickHouse 介绍

  • ClickHouse 是一个采用列式存储,用于联机分析(OLAP)的数据库管理系统(DBMS)。
  • 优点:
    • 列式存储
    • 实时的数据更新
    • 支持近似计算
    • 支持数据复制和数据完整性
  • 缺点:
    • 没有完整的事务支持
    • 缺少高频率、低延迟的修改或删除已存在数据的能力,仅能用于批量删除或修改数据

2. 部署 ClickHouse

部署文档(官方)

  • 本次部署采用的是单点部署
2.1 检查机器环境是否可以部署 ClickHouse
[hadoop@bigdata ~]$ grep -q sse4_2 /proc/cpuinfo && echo "SSE 4.2 supported" || echo "SSE 4.2 not supported"
SSE 4.2 supported
2.2 部署 ClickHouse
[hadoop@bigdata ~]$ sudo yum install yum-utils
[hadoop@bigdata ~]$ sudo rpm --import https://repo.clickhouse.tech/CLICKHOUSE-KEY.GPG
[hadoop@bigdata ~]$ sudo yum-config-manager --add-repo https://repo.clickhouse.tech/rpm/stable/x86_64
[hadoop@bigdata ~]$ sudo yum install clickhouse-server clickhouse-client
2.3 启动 ClickHouse
[hadoop@bigdata ~]$ sudo systemctl start clickhouse-server
[hadoop@bigdata ~]$ sudo systemctl status clickhouse-server
● clickhouse-server.service - ClickHouse Server (analytic DBMS for big data)
   Loaded: loaded (/etc/systemd/system/clickhouse-server.service; enabled; vendor preset: disabled)
   Active: active (running) since 日 2021-02-14 14:08:45 CST; 6s ago
 Main PID: 10498 (clckhouse-watch)
   CGroup: /system.slice/clickhouse-server.service
           ├─10498 clickhouse-watchdog --config=/etc/clickhouse-server/config.xml --pid-file=/run/clickhouse-server/clickhouse-server.pid
           └─10499 /usr/bin/clickhouse-server --config=/etc/clickhouse-server/config.xml --pid-file=/run/clickhouse-server/clickhouse-server.pid

2月 14 14:08:45 bigdata systemd[1]: Started ClickHouse Server (analytic DBMS for big data).
2月 14 14:08:45 bigdata clickhouse-server[10498]: Processing configuration file '/etc/clickhouse-server/config.xml'.
2月 14 14:08:45 bigdata clickhouse-server[10498]: Logging trace to /var/log/clickhouse-server/clickhouse-server.log
2月 14 14:08:45 bigdata clickhouse-server[10498]: Logging errors to /var/log/clickhouse-server/clickhouse-server.err.log
2月 14 14:08:45 bigdata clickhouse-server[10498]: Processing configuration file '/etc/clickhouse-server/users.xml'.
2月 14 14:08:45 bigdata clickhouse-server[10498]: Saved preprocessed configuration to '/var/lib/clickhouse/preprocessed_configs/users.xml'.
2月 14 14:08:47 bigdata clickhouse-server[10498]: Processing configuration file '/etc/clickhouse-server/config.xml'.
2月 14 14:08:47 bigdata clickhouse-server[10498]: Saved preprocessed configuration to '/var/lib/clickhouse/preprocessed_configs/config.xml'.
  • 进入到 ClickHouse 客户端
[hadoop@bigdata ~]$ clickhouse-client

3. 修改 ClickHouse 配置文件

  • 修改配置文件:/etc/clickhouse-server/config.xml
  • ClickHouse JDBC 端口号
<tcp_port>9000</tcp_port>
  • ClickHouse 修改监听的 HostName
 <listen_host>::</listen_host>
  • 重启 ClickHouse
[hadoop@bigdata ~]$ sudo systemctl start clickhouse-server
[hadoop@bigdata ~]$ clickhouse-client --host=bigdata --port=9000
bigdata :) show databases;

SHOW DATABASES

Query id: 478c40aa-9367-4322-a66a-61c61f906138

┌─name────┐
│ default │
│ system  │
└─────────┘

2 rows in set. Elapsed: 0.004 sec. 
  • 客户端执行的 SQL 语句如果需要换行操作
[hadoop@bigdata ~]$ clickhouse-client --host=bigdata --port=9000 -m
bigdata :) show
:-] databases;

SHOW DATABASES

Query id: 66526349-6668-4e89-b53c-dd70b5aeb99f

┌─name────┐
│ default │
│ system  │
└─────────┘

2 rows in set. Elapsed: 0.006 sec. 
  • 修改 users.xml
  • 增加一个用户
[root@bigdata clickhouse-server]# pwd
/etc/clickhouse-server
[root@bigdata clickhouse-server]# vim users.xml 
<bigdata>
	<password>123456</password>
	<networks>
		<ip>::/0</ip>
	</networks>
	<profile>default</profile>
	<quota>default</quota>
</bigdata>
<$username>
	<password>$password</password>
	<networks>
		<ip>::/0</ip>
	</networks>
	<profile>default</profile>
	<quota>default</quota>
</$username>
  • 重启服务并登录
[hadoop@bigdata ~]$ sudo systemctl start clickhouse-server
[hadoop@bigdata ~]$ clickhouse-client --host=bigdata --port=9000 -m --user=bigdata --password=123456
ClickHouse client version 21.2.2.8 (official build).
Connecting to bigdata:9000 as user bigdata.
Connected to ClickHouse server version 21.2.2 revision 54447.

bigdata :) show databases;

SHOW DATABASES

Query id: aad8f902-2908-447f-8683-8df1c58ee47c

┌─name────┐
│ default │
│ system  │
└─────────┘

3 rows in set. Elapsed: 0.004 sec. 

4. ClickHouse 默认重要的路径

  • 配置文件
/etc/clickhouse-server
  • 数据路径
  • 一个 database 一个文件夹
/var/lib/clickhouse/data
  • 元数据路径
/var/lib/clickhouse/metadata
  • 命令行路径
/usr/bin/clickhouse-server
/usr/bin/clickhouse-client
/usr/bin/clickhouse-benchmark

5. ClickHouse 基本操作

5.1 创建一个数据库
CREATE DATABASE [IF NOT EXISTS] db_name [ON CLUSTER cluster] [ENGINE = engine(...)]
bigdata :) create database bigdata;

CREATE DATABASE bigdata

Query id: aebd4d34-7af4-4c15-92d3-7b777476fd08

Ok.

0 rows in set. Elapsed: 0.005 sec. 
bigdata :) use bigdata;

USE bigdata

Query id: 31543b31-ddf2-4fa4-a617-6c8a21303afe

Ok.

0 rows in set. Elapsed: 0.002 sec. 
bigdata :) select currentDatabase() ;

SELECT currentDatabase()

Query id: 2cd2d043-1966-43a9-9e2c-2f5e02d50803

┌─currentDatabase()─┐
│ bigdata           │
└───────────────────┘

1 rows in set. Elapsed: 0.004 sec. 
5.2 创建一张表
CREATE TABLE [IF NOT EXISTS] [db.]table_name [ON CLUSTER cluster]
(
    name1 [type1] [NULL|NOT NULL] [DEFAULT|MATERIALIZED|ALIAS expr1] [compression_codec] [TTL expr1],
    name2 [type2] [NULL|NOT NULL] [DEFAULT|MATERIALIZED|ALIAS expr2] [compression_codec] [TTL expr2],
    ...
) ENGINE = engine
bigdata :) CREATE TABLE test
:-] (
:-]     id Int32,
:-]     name String
:-] ) ENGINE = Memory;

CREATE TABLE test
(
    `id` Int32,
    `name` String
)
ENGINE = Memory

Query id: 7adb3381-57f1-4c6b-847a-f82590888cc9

Ok.

0 rows in set. Elapsed: 0.020 sec. 
5.3 插入一笔数据
bigdata :) insert into test values(1,'hadoop'),(2,'spark');

INSERT INTO test VALUES

Query id: 50160821-b508-42b7-a344-05087d6932a9

Ok.

2 rows in set. Elapsed: 0.005 sec.
5.4 查询数据
bigdata :) select * from test;

SELECT *
FROM test

Query id: 51dba55f-39df-4c66-832a-20ecb3117c51

┌─id─┬─name───┐
│  1 │ hadoop │
│  2 │ spark  │
└────┴────────┘

2 rows in set. Elapsed: 0.005 sec. 
5.5 修改数据
bigdata :) ALTER TABLE test UPDATE name = 'spark1' where id = 2; 

ALTER TABLE test
    UPDATE name = 'spark1' WHERE id = 2

Query id: 3ede61c5-72b9-4af1-b75b-33882da9288e

Ok.

0 rows in set. Elapsed: 0.003 sec. 

bigdata :) select * from test;

SELECT *
FROM test

Query id: d9045117-ec85-4146-82c8-6b11109adf69

┌─id─┬─name───┐
│  1 │ hadoop │
│  2 │ spark1 │
└────┴────────┘

2 rows in set. Elapsed: 0.005 sec. 
5.6 删除数据
bigdata :) ALTER TABLE test delete where id = 2;

ALTER TABLE test
    DELETE WHERE id = 2

Query id: 9cecf262-aa88-4ba2-81b8-5db27046e1d2

Ok.

0 rows in set. Elapsed: 0.005 sec. 

bigdata :) select * from test;

SELECT *
FROM test

Query id: 931e9a24-3f86-48ec-9df0-e8f4c7b3d603

┌─id─┬─name───┐
│  1 │ hadoop │
└────┴────────┘

1 rows in set. Elapsed: 0.006 sec. 

6. ClickHouse 常用数据类型

  • ClickHouse 数据类型中区分大小写
  • 查看本版本中有哪些数据类型:
bigdata :) select * from system.data_type_families;

SELECT *
FROM system.data_type_families

Query id: e5b07c0c-b7a5-42c5-9040-c69a04bf7e67

┌─name────────────────────────────┬─case_insensitive─┬─alias_to────┐
│ Polygon                         │                0 │             │
│ Ring                            │                0 │             │
│ MultiPolygon                    │                0 │             │
│ IPv6                            │                0 │             │
│ IntervalSecond                  │                0 │             │
│ IPv4                            │                0 │             │
│ UInt32                          │                0 │             │
│ IntervalYear                    │                0 │             │
│ IntervalQuarter                 │                0 │             │
│ IntervalMonth                   │                0 │             │
│ Int64                           │                0 │             │
│ IntervalDay                     │                0 │             │
│ IntervalHour                    │                0 │             │
│ UInt256                         │                0 │             │
│ Int16                           │                0 │             │
│ LowCardinality                  │                0 │             │
│ AggregateFunction               │                0 │             │
│ Nothing                         │                0 │             │
│ Decimal256                      │                1 │             │
│ Tuple                           │                0 │             │
│ Array                           │                0 │             │
│ Enum16                          │                0 │             │
│ IntervalMinute                  │                0 │             │
│ FixedString                     │                0 │             │
│ String                          │                0 │             │
│ DateTime                        │                1 │             │
│ Map                             │                0 │             │
│ UUID                            │                0 │             │
│ Decimal64                       │                1 │             │
│ Nullable                        │                0 │             │
│ Enum                            │                0 │             │
│ Int32                           │                0 │             │
│ UInt8                           │                0 │             │
│ Date                            │                1 │             │
│ Decimal32                       │                1 │             │
│ Point                           │                0 │             │
│ Float64                         │                0 │             │
│ DateTime64                      │                1 │             │
│ Int128                          │                0 │             │
│ Decimal128                      │                1 │             │
│ Int8                            │                0 │             │
│ SimpleAggregateFunction         │                0 │             │
│ Nested                          │                0 │             │
│ Decimal                         │                1 │             │
│ Int256                          │                0 │             │
│ IntervalWeek                    │                0 │             │
│ UInt64                          │                0 │             │
│ Enum8                           │                0 │             │
│ DateTime32                      │                1 │             │
│ UInt16                          │                0 │             │
│ Float32                         │                0 │             │
│ INET6                           │                1 │ IPv6        │
│ INET4                           │                1 │ IPv4        │
│ BINARY                          │                1 │ FixedString │
│ NATIONAL CHAR VARYING           │                1 │ String      │
│ BINARY VARYING                  │                1 │ String      │
│ NCHAR LARGE OBJECT              │                1 │ String      │
│ NATIONAL CHARACTER VARYING      │                1 │ String      │
│ NATIONAL CHARACTER LARGE OBJECT │                1 │ String      │
│ NATIONAL CHARACTER              │                1 │ String      │
│ NATIONAL CHAR                   │                1 │ String      │
│ CHARACTER VARYING               │                1 │ String      │
│ LONGBLOB                        │                1 │ String      │
│ MEDIUMTEXT                      │                1 │ String      │
│ TEXT                            │                1 │ String      │
│ TINYBLOB                        │                1 │ String      │
│ VARCHAR2                        │                1 │ String      │
│ CHARACTER LARGE OBJECT          │                1 │ String      │
│ DOUBLE PRECISION                │                1 │ Float64     │
│ LONGTEXT                        │                1 │ String      │
│ NVARCHAR                        │                1 │ String      │
│ INT1 UNSIGNED                   │                1 │ UInt8       │
│ VARCHAR                         │                1 │ String      │
│ CHAR VARYING                    │                1 │ String      │
│ MEDIUMBLOB                      │                1 │ String      │
│ NCHAR                           │                1 │ String      │
│ CHAR                            │                1 │ String      │
│ SMALLINT UNSIGNED               │                1 │ UInt16      │
│ TIMESTAMP                       │                1 │ DateTime    │
│ FIXED                           │                1 │ Decimal     │
│ TINYTEXT                        │                1 │ String      │
│ NUMERIC                         │                1 │ Decimal     │
│ DEC                             │                1 │ Decimal     │
│ TINYINT UNSIGNED                │                1 │ UInt8       │
│ INTEGER UNSIGNED                │                1 │ UInt32      │
│ INT UNSIGNED                    │                1 │ UInt32      │
│ CLOB                            │                1 │ String      │
│ MEDIUMINT UNSIGNED              │                1 │ UInt32      │
│ BOOL                            │                1 │ Int8        │
│ SMALLINT                        │                1 │ Int16       │
│ INTEGER SIGNED                  │                1 │ Int32       │
│ NCHAR VARYING                   │                1 │ String      │
│ INT SIGNED                      │                1 │ Int32       │
│ TINYINT SIGNED                  │                1 │ Int8        │
│ BIGINT SIGNED                   │                1 │ Int64       │
│ BINARY LARGE OBJECT             │                1 │ String      │
│ SMALLINT SIGNED                 │                1 │ Int16       │
│ MEDIUMINT                       │                1 │ Int32       │
│ INTEGER                         │                1 │ Int32       │
│ INT1 SIGNED                     │                1 │ Int8        │
│ BIGINT UNSIGNED                 │                1 │ UInt64      │
│ BYTEA                           │                1 │ String      │
│ INT                             │                1 │ Int32       │
│ SINGLE                          │                1 │ Float32     │
│ FLOAT                           │                1 │ Float32     │
│ MEDIUMINT SIGNED                │                1 │ Int32       │
│ BOOLEAN                         │                1 │ Int8        │
│ DOUBLE                          │                1 │ Float64     │
│ INT1                            │                1 │ Int8        │
│ CHAR LARGE OBJECT               │                1 │ String      │
│ TINYINT                         │                1 │ Int8        │
│ BIGINT                          │                1 │ Int64       │
│ CHARACTER                       │                1 │ String      │
│ BYTE                            │                1 │ Int8        │
│ BLOB                            │                1 │ String      │
│ REAL                            │                1 │ Float32     │
└─────────────────────────────────┴──────────────────┴─────────────┘

116 rows in set. Elapsed: 0.006 sec.
6.1 Int 类型
  • Int 类型有以下这些:
  • UInt8, UInt16, UInt32, UInt64, UInt256, Int8, Int16, Int32, Int64, Int128, Int256
Int Ranges 
Int8 — [-128 : 127]
Int16 — [-32768 : 32767]
Int32 — [-2147483648 : 2147483647]
Int64 — [-9223372036854775808 : 9223372036854775807]
Int128 — [-170141183460469231731687303715884105728 : 170141183460469231731687303715884105727]
Int256 — [-57896044618658097711785492504343953926634992332820282019728792003956564819968 : 57896044618658097711785492504343953926634992332820282019728792003956564819967]

Uint Ranges 
UInt8 — [0 : 255]
UInt16 — [0 : 65535]
UInt32 — [0 : 4294967295]
UInt64 — [0 : 18446744073709551615]
UInt256 — [0 : 115792089237316195423570985008687907853269984665640564039457584007913129639935]
6.2 Float 类型
  • Float32 — float
  • Float64 — double
  • NaN and Inf
    • Inf : 无穷
    • NaN : 不是一个数字
bigdata :) SELECT 0.5 / 0
:-] ;

SELECT 0.5 / 0

Query id: 0af32c1e-100c-40e1-8f1c-469390d545be

┌─divide(0.5, 0)─┐
│            inf │
└────────────────┘

1 rows in set. Elapsed: 0.004 sec. 

bigdata :) SELECT -0.5 / 0;

SELECT -0.5 / 0

Query id: 60f4d58c-c969-4453-bd3b-7531ef309861

┌─divide(-0.5, 0)─┐
│            -inf │
└─────────────────┘

1 rows in set. Elapsed: 0.006 sec. 
bigdata :) SELECT 0 / 0
:-] ;

SELECT 0 / 0

Query id: b9275b88-d23e-44b7-80df-4d2d78e7ccbb

┌─divide(0, 0)─┐
│          nan │
└──────────────┘

1 rows in set. Elapsed: 0.004 sec. 
6.3 Decimal 类型
  • Decimal(P, S), Decimal32(S), Decimal64(S), Decimal128(S), Decimal256(S)
    • P - precision. Valid range: [ 1 : 76 ]. 代表数字可以有多少个小数位数(包括分数)。
    • S - scale. Valid range: [ 0 : P ]. 小数部分可以有多少个小数位数。
  • 加减乘除运算:
    • add, subtract: S = max(S1, S2):加、减
    • multuply: S = S1 + S2:乘法
    • divide: S = S1:除法
6.4 Boolean 类型
There is no separate type for boolean values. Use UInt8 type, restricted to the values 0 or 1.
  • 可以使用 UInt8 中 0、1来表示
6.5 String 类型
Strings of an arbitrary length. The length is not limited. The value can contain an arbitrary set of bytes, including null bytes.
The String type replaces the types VARCHAR, BLOB, CLOB, and others from other DBMSs.
6.6 Fixedstring 类型
  • 固定长度的 String 类型
  • FixedString(N)
bigdata :) CREATE TABLE test_fixedstring
:-] (
:-]     id Int32,
:-]     name FixedString(5)
:-] ) ENGINE = Memory;

CREATE TABLE test_fixedstring
(
    `id` Int32,
    `name` FixedString(5)
)
ENGINE = Memory

Query id: a7394848-4af2-4cb5-acd6-230188b035b0

Ok.

0 rows in set. Elapsed: 0.008 sec. 

bigdata :) insert into test_fixedstring values(1,'spark');

INSERT INTO test_fixedstring VALUES

Query id: 3811b8f1-d68c-4457-bc57-b45be5c719b6

Ok.

1 rows in set. Elapsed: 0.005 sec. 

bigdata :) insert into test_fixedstring values(2,'hadoop');

INSERT INTO test_fixedstring VALUES

Query id: 8aeb2b97-b44f-40ab-bdee-ff6106a75606


Exception on client:
Code: 131. DB::Exception: String too long for type FixedString(5): while executing 'FUNCTION CAST(assumeNotNull(_dummy_0) :: 2, 'FixedString(5)' :: 1) -> cast(assumeNotNull(_dummy_0), 'FixedString(5)') FixedString(5) : 4': data for INSERT was parsed from query

Connecting to database bigdata at bigdata:9000 as user default.
Connected to ClickHouse server version 21.2.2 revision 54447.
6.7 UUID 类型
A universally unique identifier (UUID) is a 16-byte number used to identify records
  • 一般是配合 generateUUIDv4 函数一起使用
INSERT INTO t_uuid SELECT generateUUIDv4(), 'Example 1'
bigdata :) CREATE TABLE test_uuid
:-] (
:-]     id UUID,
:-]     name String
:-] ) ENGINE = Memory;

CREATE TABLE test_uuid
(
    `id` UUID,
    `name` String
)
ENGINE = Memory

Query id: d3fde3cd-609f-497c-b662-4f75d252b076

Ok.

0 rows in set. Elapsed: 0.004 sec. 

bigdata :) insert into test_uuid(name) values('spark'),('hadoop');

INSERT INTO test_uuid (name) VALUES

Query id: a513ba06-55a5-4dfd-accf-acfa03636969

Ok.

2 rows in set. Elapsed: 0.005 sec. 

bigdata :) select * from test_uuid;

SELECT *
FROM test_uuid

Query id: 69a9a816-1308-41aa-8d1d-64759557795b

┌───────────────────────────────────id─┬─name───┐
│ 00000000-0000-0000-0000-000000000000 │ spark  │
│ 00000000-0000-0000-0000-000000000000 │ hadoop │
└──────────────────────────────────────┴────────┘

2 rows in set. Elapsed: 0.007 sec. 
6.8 Date 类型
  • Date : 精确到日期
  • Datetime :精确到秒,可以自己选择时区:DateTime([timezone])
  • Datetime64 :自定义精确到哪里,最多可以精确到亚秒:DateTime64(precision, [timezone]):Tick size (precision): 10-precision seconds
CREATE TABLE dt
(
    `timestamp` Date,
    `event_id` UInt8
)
ENGINE = TinyLog;
INSERT INTO dt Values (1546300800, 1), ('2019-01-01', 2);
SELECT * FROM dt;
┌──timestamp─┬─event_id─┐
│ 2019-01-01 │        1 │
│ 2019-01-01 │        2 │
└────────────┴──────────┘
CREATE TABLE dt
(
    `timestamp` DateTime('Europe/Moscow'),
    `event_id` UInt8
)
ENGINE = TinyLog;
INSERT INTO dt Values (1546300800, 1), ('2019-01-01 00:00:00', 2);
SELECT * FROM dt;
┌───────────timestamp─┬─event_id─┐
│ 2019-01-01 03:00:00 │        1 │
│ 2019-01-01 00:00:00 │        2 │
└─────────────────────┴──────────┘
CREATE TABLE dt
(
    `timestamp` DateTime64(3, 'Europe/Moscow'),
    `event_id` UInt8
)
ENGINE = TinyLog
INSERT INTO dt Values (1546300800000, 1), ('2019-01-01 00:00:00', 2)
SELECT * FROM dt
┌───────────────timestamp─┬─event_id─┐
│ 2019-01-01 03:00:00.000 │        1 │
│ 2019-01-01 00:00:00.000 │        2 │
└─────────────────────────┴──────────┘
  • 0
    点赞
  • 2
    收藏
    觉得还不错? 一键收藏
  • 0
    评论

“相关推荐”对你有帮助么?

  • 非常没帮助
  • 没帮助
  • 一般
  • 有帮助
  • 非常有帮助
提交
评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值