Clickhouse Research Reports

FilterPostgresql(default) /msPostgresql(Index+view) /msClickHouse (default) /msClickHouse +table, index /msClickHouse Cluster /ms
Taxonomy (default)220171921343150
Taxonomy (ngrams)574513621791122
Taxonomy Score (default)6564214605800431
Taxonomy node (ngrams)138736743395307

Taxonomy (default)

PostgresqlClickHouseClickHouse Cluster
default22017351164
table index21903347172
table index+view20784438279
table index+view, index1921❌ (view not support add index)❌ (view not support add index)
table index + unnest table, index397

Taxonomy (ngrams)

PostgresqlClickHouseClickHouse Cluster
default5745238112
table index4200249119
table index+view3847254135
table index+view, index136
table index + unnest table, index101

Taxonomy node ngram

PostgresqlClickHouseClickHouse Cluster
default1387440
table index337519
table index+view### blocking
table index+view, index367
table index + unnest table, index

Taxonomy score default

PostgresqlClickHouseClickHouse Cluster
default65642812473
table index875489
table index+view609701007
table index+view, index14605
table index + unnest table, index

default

EXPLAIN
SELECT
    col0 AS diffcol0,
    count(*) AS count
FROM stream_616d2986bd17682e66aedbb3 AS tab0,
(
    SELECT
        tab0._id,
        arrayJoin(`f_7d1_ff35_t_labels$textual`) AS col0
    FROM stream_616d2986bd17682e66aedbb3 AS tab0
) AS tab1
WHERE tab1._id = tab0._id
GROUP BY diffcol0
ORDER BY count DESC NULLS LAST
LIMIT 0, 50

Query id: edb8e6a4-d562-4513-bcc0-4cc07013b454

┌─explain──────────────────────────────────────────────────────────────────────────────────────────────────┐
│ Expression (Projection)                                                                                  │
│   Limit (preliminary LIMIT (without OFFSET))                                                             │
│     Sorting (Sorting for ORDER BY)                                                                       │
│       Expression (Before ORDER BY)                                                                       │
│         Aggregating                                                                                      │
│           Expression (Before GROUP BY)                                                                   │
│             Filter (WHERE)                                                                               │
│               Join (JOIN)                                                                                │
│                 Expression (Before JOIN)                                                                 │
│                   SettingQuotaAndLimits (Set limits and quota after reading from storage)                │
│                     ReadFromMergeTree                                                                    │
│                 Expression ((Joined actions + (Rename joined columns + (Projection + Before ORDER BY)))) │
│                   SettingQuotaAndLimits (Set limits and quota after reading from storage)                │
│                     ReadFromMergeTree                                                                    │
└──────────────────────────────────────────────────────────────────────────────────────────────────────────┘

14 rows in set. Elapsed: 0.008 sec. 
50 rows in set. Elapsed: 0.351 sec. Processed 1.79 million rows, 171.82 MB (5.11 million rows/s., 489.56 MB/s.)

docs: https://clickhouse.com/docs

Clickhouse: 4核16G

Postgres:

Clickhouse Cluster: 4Core 16G * 3

create ngrams table, index

create table stream_ngrams_index (_id Int32, vw_unnested Text, INDEX f_n61_af35_ngrams$textual_string vw_unnested  TYPE bloom_filter GRANULARITY 3) ENGINE=MergeTree order by vw_unnested

taxonomy (default) Use arrayJoin function:

PGSQL: 22017.586 ms → 1921.623 ms

ClickHouse: 343 ms

ClickHouse: Cluster: 150ms

EXPLAIN SELECT
    col0 AS diffcol0,
    count(*) AS count
FROM stream_616d2986bd17682e66aedbb3 AS tab0,
(
    SELECT
        tab0._id,
        arrayJoin(`f_7d1_ff35_t_labels$textual`) AS col0
    FROM stream_616d2986bd17682e66aedbb3 AS tab0
) AS tab1
WHERE tab1._id = tab0._id
GROUP BY diffcol0
ORDER BY count DESC NULLS LAST
LIMIT 0, 50


┌─explain─────────────────────────────────────────────────────┐
│ (Expression)                                                │
│ ExpressionTransform                                         │
│   (Limit)                                                   │
│   Limit                                                     │
│     (Sorting)                                               │
│     MergeSortingTransform                                   │
│       LimitsCheckingTransform                               │
│         PartialSortingTransform                             │
│           (Expression)                                      │
│           ExpressionTransform                               │
│             (Aggregating)                                   │
│             Resize 41                                    │
│               AggregatingTransform × 4                      │
│                 StrictResize 44                          │
│                   (Expression)                              │
│                   ExpressionTransform × 4                   │
│                     (Filter)                                │
│                     FilterTransform × 4                     │
│                       (Join)                                │
│                       JoiningTransform × 4 21            │
│                         Resize 14                        │
│                           FillingRightJoinSide              │
│                             Resize 41                    │
│                               (Expression)                  │
│                               ExpressionTransform × 4       │
│                                 (SettingQuotaAndLimits)     │
│                                   (ReadFromMergeTree)       │
│                                   MergeTreeThread × 4 01 │
│                               (Expression)                  │
│                               ExpressionTransform × 4       │
│                                 (SettingQuotaAndLimits)     │
│                                   (ReadFromMergeTree)       │
│                                   MergeTreeThread × 4 01 │
└─────────────────────────────────────────────────────────────┘
Explain Time:33 rows in set. Elapsed: 0.005 sec.
Execution Time: 50 rows in set. Elapsed: 0.343 sec. Processed 1.79 million rows, 171.82 MB (5.22 million rows/s., 500.44 MB/s.)
Cluster Execution: 50 rows in set. Elapsed: 0.150 sec. Processed 1.79 million rows, 171.82 MB (11.93 million rows/s., 1.14 GB/s.)
EXPLAIN
SELECT
    col0 AS diffcol0,
    count(*) AS count
FROM stream_617fc79a355b7f54684a9c8b AS tab0,
(
    SELECT
        tab0._id,
        arrayJoin(`f_jq2_20d7_t_labels$textual`) AS col0
    FROM stream_617fc79a355b7f54684a9c8b AS tab0
) AS tab1
WHERE tab1._id = tab0._id
GROUP BY diffcol0
ORDER BY count DESC NULLS LAST
LIMIT 0, 50

Query id: ac9a0578-cf17-4e5c-8e44-34320d454380

┌─explain──────────────────────────────────────────────────────────────────────────────────────────────────┐
│ Expression (Projection)                                                                                  │
│   Limit (preliminary LIMIT (without OFFSET))                                                             │
│     Sorting (Sorting for ORDER BY)                                                                       │
│       Expression (Before ORDER BY)                                                                       │
│         Aggregating                                                                                      │
│           Expression (Before GROUP BY)                                                                   │
│             Filter (WHERE)                                                                               │
│               Join (JOIN)                                                                                │
│                 Expression (Before JOIN)                                                                 │
│                   SettingQuotaAndLimits (Set limits and quota after reading from storage)                │
│                     ReadFromMergeTree                                                                    │
│                 Expression ((Joined actions + (Rename joined columns + (Projection + Before ORDER BY)))) │
│                   SettingQuotaAndLimits (Set limits and quota after reading from storage)                │
│                     ReadFromMergeTree                                                                    │
└──────────────────────────────────────────────────────────────────────────────────────────────────────────┘

Explain Time: 14 rows in set. Elapsed: 0.003 sec.
Execution Time: 50 rows in set. Elapsed: 0.585 sec. Processed 5.33 million rows, 244.40 MB (9.10 million rows/s., 417.47 MB/s.)
Cluster Execution Time: 50 rows in set. Elapsed: 0.287 sec. Processed 5.33 million rows, 244.40 MB (18.58 million rows/s., 851.80 MB/s.)

taxonomy (ngrams) Use arrayJoin function

EXPLAIN
SELECT
    col0 AS diffcol0,
    count('*') AS count
FROM stream_616d2986bd17682e66aedbb3 AS tab0,
(
    SELECT
        tab0._id,
        arrayJoin(`f_7d1_ff35_t_labels$textual`) AS col0
    FROM stream_616d2986bd17682e66aedbb3 AS tab0
) AS tab1
WHERE hasAny(`f_n61_af35_ngrams$textual`, ['日期_新鲜', '值得_信赖']) AND (tab1._id = tab0._id)
GROUP BY diffcol0
ORDER BY diffcol0 ASC NULLS FIRST
LIMIT 0, 200

Query id: 56501396-937e-478d-a8e2-09635a2104dc

┌─explain──────────────────────────────────────────────────────────────────────────────────────────────────┐
│ Expression (Projection)                                                                                  │
│   Limit (preliminary LIMIT (without OFFSET))                                                             │
│     Sorting (Sorting for ORDER BY)                                                                       │
│       Expression (Before ORDER BY)                                                                       │
│         Aggregating                                                                                      │
│           Expression (Before GROUP BY)                                                                   │
│             Filter (WHERE)                                                                               │
│               Join (JOIN)                                                                                │
│                 Expression (Before JOIN)                                                                 │
│                   SettingQuotaAndLimits (Set limits and quota after reading from storage)                │
│                     ReadFromMergeTree                                                                    │
│                 Expression ((Joined actions + (Rename joined columns + (Projection + Before ORDER BY)))) │
│                   SettingQuotaAndLimits (Set limits and quota after reading from storage)                │
│                     ReadFromMergeTree                                                                    │
└──────────────────────────────────────────────────────────────────────────────────────────────────────────┘

EXPLAIN Time: 14 rows in set. Elapsed: 0.004 sec. 
Execution Time: 125 rows in set. Elapsed: 0.217 sec. Processed 1.79 million rows, 204.71 MB (8.25 million rows/s., 941.35 MB/s.)
Cluster Execution Time: 50 rows in set. Elapsed: 0.122 sec. Processed 1.79 million rows, 204.71 MB (14.74 million rows/s., 1.68 GB/s.)

taxonomy (ngrams) Use arrayJoin function + table, index

EXPLAIN
SELECT
    col0 AS diffcol0,
    count('*') AS count
FROM stream_616d2986bd17682e66aedbb3_ngrams_index_1 AS tab0,
(
    SELECT
        tab0._id,
        arrayJoin(`f_7d1_ff35_t_labels$textual`) AS col0
    FROM stream_616d2986bd17682e66aedbb3_ngrams_index_1 AS tab0
) AS tab1
WHERE (tab1._id IN (
    SELECT _id
    FROM stream_ngrams_index
    WHERE vw_unnested IN ['日期_新鲜', '值得_信赖']
)) AND (tab1._id = tab0._id)
GROUP BY diffcol0
ORDER BY diffcol0 ASC NULLS FIRST
LIMIT 0, 200

Query id: e54603a8-2370-417e-b038-d728b5051eea

┌─explain──────────────────────────────────────────────────────────────────────────────────────────────────┐
│ Expression (Projection)                                                                                  │
│   Limit (preliminary LIMIT (without OFFSET))                                                             │
│     Sorting (Sorting for ORDER BY)                                                                       │
│       Expression (Before ORDER BY)                                                                       │
│         Aggregating                                                                                      │
│           Expression (Before GROUP BY)                                                                   │
│             Filter (WHERE)                                                                               │
│               Join (JOIN)                                                                                │
│                 Expression (Before JOIN)                                                                 │
│                   SettingQuotaAndLimits (Set limits and quota after reading from storage)                │
│                     ReadFromMergeTree                                                                    │
│                 Expression ((Joined actions + (Rename joined columns + (Projection + Before ORDER BY)))) │
│                   SettingQuotaAndLimits (Set limits and quota after reading from storage)                │
│                     ReadFromMergeTree                                                                    │
└──────────────────────────────────────────────────────────────────────────────────────────────────────────┘

14 rows in set. Elapsed: 0.011 sec. 
125 rows in set. Elapsed: 0.091 sec. Processed 1.79 million rows, 82.19 MB (19.61 million rows/s., 898.72 MB/s.)

taxonomy (ngrams) Use arrayJoin function+ View

EXPLAIN
SELECT
    col0 AS diffcol0,
    count('*') AS count
FROM stream_616d2986bd17682e66aedbb3 AS tab0,
(
    SELECT
        vw_view._id,
        vw_view.vw_unnested AS col0
    FROM `vw_s_dbb3_f_7d1_ff35_t_labels$textual_` AS vw_view
) AS tab1
WHERE hasAny(`f_n61_af35_ngrams$textual`, ['日期_新鲜', '值得_信赖']) AND (tab1._id = tab0._id)
GROUP BY diffcol0
ORDER BY diffcol0 ASC NULLS FIRST
LIMIT 0, 200

Query id: 5bf23233-02a3-4be5-b233-912badedf908

┌─explain────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┐
│ Expression (Projection)                                                                                                                                                │
│   Limit (preliminary LIMIT (without OFFSET))                                                                                                                           │
│     Sorting (Sorting for ORDER BY)                                                                                                                                     │
│       Expression (Before ORDER BY)                                                                                                                                     │
│         Aggregating                                                                                                                                                    │
│           Expression (Before GROUP BY)                                                                                                                                 │
│             Filter (WHERE)                                                                                                                                             │
│               Join (JOIN)                                                                                                                                              │
│                 Expression (Before JOIN)                                                                                                                               │
│                   SettingQuotaAndLimits (Set limits and quota after reading from storage)                                                                              │
│                     ReadFromMergeTree                                                                                                                                  │
│                 Expression ((Joined actions + (Rename joined columns + (Projection + Before ORDER BY))))                                                               │
│                   SettingQuotaAndLimits (Set limits and quota after reading from storage)                                                                              │
│                     Expression ((Convert VIEW subquery result to VIEW table structure + (Materialize constants after VIEW subquery + (Projection + Before ORDER BY)))) │
│                       SettingQuotaAndLimits (Set limits and quota after reading from storage)                                                                          │
│                         ReadFromMergeTree                                                                                                                              │
└────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┘

16 rows in set. Elapsed: 0.010 sec. 
125 rows in set. Elapsed: 0.227 sec. Processed 1.79 million rows, 204.71 MB (7.91 million rows/s., 903.07 MB/s.)

taxonomy score default

EXPLAIN
SELECT
    col0 AS diffcol0,
    colnest1 AS diffcolnest1,
    count('*') AS col2
FROM stream_616d2986bd17682e66aedbb3 AS tab0,
(
    SELECT
        tab0._id,
        arrayJoin(`f_7d1_ff35_t_labels$textual`) AS col0
    FROM stream_616d2986bd17682e66aedbb3 AS tab0
) AS tab2,
(
    SELECT
        tab0._id,
        arrayJoin(`f_8d1_ff35_t_scores$numerical`) AS colnest1
    FROM stream_616d2986bd17682e66aedbb3 AS tab0
) AS tab3
WHERE (tab2._id = tab0._id) AND (tab3._id = tab0._id)
GROUP BY
    diffcol0,
    diffcolnest1
ORDER BY diffcol0 ASC NULLS FIRST
LIMIT 0, 200

Query id: 76bacd84-e295-492c-a9e1-390b98767dc3

┌─explain────────────────────────────────────────────────────────────────────────────────────────────────────────┐
│ Expression (Projection)                                                                                        │
│   Limit (preliminary LIMIT (without OFFSET))                                                                   │
│     Sorting (Sorting for ORDER BY)                                                                             │
│       Expression (Before ORDER BY)                                                                             │
│         Aggregating                                                                                            │
│           Expression (Before GROUP BY)                                                                         │
│             Filter (WHERE)                                                                                     │
│               Join (JOIN)                                                                                      │
│                 Expression ((Before JOIN + (Projection + Before ORDER BY)))                                    │
│                   Filter (WHERE)                                                                               │
│                     Join (JOIN)                                                                                │
│                       Expression (Before JOIN)                                                                 │
│                         SettingQuotaAndLimits (Set limits and quota after reading from storage)                │
│                           ReadFromMergeTree                                                                    │
│                       Expression ((Joined actions + (Rename joined columns + (Projection + Before ORDER BY)))) │
│                         SettingQuotaAndLimits (Set limits and quota after reading from storage)                │
│                           ReadFromMergeTree                                                                    │
│                 Expression ((Joined actions + (Rename joined columns + (Projection + Before ORDER BY))))       │
│                   SettingQuotaAndLimits (Set limits and quota after reading from storage)                      │
│                     ReadFromMergeTree                                                                          │
└────────────────────────────────────────────────────────────────────────────────────────────────────────────────┘

Explain Time: 20 rows in set. Elapsed: 0.008 sec. 
Execution Time:  200 rows in set. Elapsed: 0.800 sec. Processed 2.69 million rows, 190.64 MB (3.36 million rows/s., 238.25 MB/s.)
Cluster Execution Time: 200 rows in set. Elapsed: 0.431 sec. Processed 2.09 million rows, 178.10 MB (4.85 million rows/s., 413.09 MB/s.)

taxonomy node (ngrams) Use arrayJoin function

EXPLAIN
SELECT count(*)
FROM stream_616d2986bd17682e66aedbb3 AS tab0
WHERE hasAny(`f_n61_af35_ngrams$textual`, ['日期_新鲜', '值得_信赖']) AND (_id IN (
    SELECT al._id
    FROM
    (
        SELECT
            unwhe._id,
            arrayJoin(`f_7d1_ff35_t_labels$textual`) AS oo,
            `f_7d1_ff35_t_labels$textual` AS af
        FROM stream_616d2986bd17682e66aedbb3 AS unwhe
    ) AS al
    WHERE ((oo[1]) = '消费洞察话题') AND (_id = al._id) AND hasAll(arrayFlatten(af), ['消费洞察话题'])
))

Query id: fd6721e8-4cdd-47b1-bdcc-b05e874c949b

┌─explain─────────────────────────────────────────────────────────────────────────┐
│ Expression ((Projection + Before ORDER BY))                                     │
│   Aggregating                                                                   │
│     Expression (Before GROUP BY)                                                │
│       Filter (WHERE)                                                            │
│         SettingQuotaAndLimits (Set limits and quota after reading from storage) │
│           ReadFromMergeTree                                                     │
└─────────────────────────────────────────────────────────────────────────────────┘

Explain Time: 6 rows in set. Elapsed: 0.390 sec.
Execution Time: 1 rows in set. Elapsed: 0.433 sec. Processed 896.55 thousand rows, 36.47 MB (2.05 million rows/s., 83.46 MB/s.)
Cluster Execution Time: 1 rows in set. Elapsed: 0.307 sec. Processed 896.55 thousand rows, 36.47 MB (2.92 million rows/s., 118.84 MB/s.)

taxonomy node (ngrams) Use arrayJoin function + table , index

EXPLAIN
SELECT count(*)
FROM stream_616d2986bd17682e66aedbb3_ngrams_index_1 AS tab0
WHERE _id IN (
    SELECT al._id
    FROM
    (
        SELECT
            unwhe._id,
            arrayJoin(`f_7d1_ff35_t_labels$textual`) AS oo,
            `f_7d1_ff35_t_labels$textual` AS af
        FROM stream_616d2986bd17682e66aedbb3_ngrams_index_1 AS unwhe
    ) AS al
    WHERE (_id IN (
        SELECT _id
        FROM stream_ngrams_index
        WHERE vw_unnested IN ['日期_新鲜', '值得_信赖']
    )) AND ((oo[1]) = '消费洞察话题') AND (_id = al._id) AND hasAll(arrayFlatten(af), ['消费洞察话题'])
)

Query id: 77bc34ab-d1e3-4ee9-9956-0410e88dbfa8

┌─explain─────────────────────────────────────────────────────────────────────────┐
│ Expression ((Projection + Before ORDER BY))                                     │
│   Aggregating                                                                   │
│     Expression (Before GROUP BY)                                                │
│       Filter (WHERE)                                                            │
│         SettingQuotaAndLimits (Set limits and quota after reading from storage) │
│           ReadFromMergeTree                                                     │
└─────────────────────────────────────────────────────────────────────────────────┘

6 rows in set. Elapsed: 0.088 sec. 
1 rows in set. Elapsed: 0.095 sec. Processed 896.55 thousand rows, 3.59 MB (9.46 million rows/s., 37.83 MB/s.)
  • 0
    点赞
  • 0
    收藏
    觉得还不错? 一键收藏
  • 0
    评论
智慧校园整体解决方案是响应国家教育信息化政策,结合教育改革和技术创新的产物。该方案以物联网、大数据、人工智能和移动互联技术为基础,旨在打造一个安全、高效、互动且环保的教育环境。方案强调从数字化校园向智慧校园的转变,通过自动数据采集、智能分析和按需服务,实现校园业务的智能化管理。 方案的总体设计原则包括应用至上、分层设计和互联互通,确保系统能够满足不同用户角色的需求,并实现数据和资源的整合与共享。框架设计涵盖了校园安全、管理、教学、环境等多个方面,构建了一个全面的校园应用生态系统。这包括智慧安全系统、校园身份识别、智能排课及选课系统、智慧学习系统、精品录播教室方案等,以支持个性化学习和教学评估。 建设内容突出了智慧安全和智慧管理的重要性。智慧安全管理通过分布式录播系统和紧急预案一键启动功能,增强校园安全预警和事件响应能力。智慧管理系统则利用物联网技术,实现人员和设备的智能管理,提高校园运营效率。 智慧教学部分,方案提供了智慧学习系统和精品录播教室方案,支持专业级学习硬件和智能化网络管理,促进个性化学习和教学资源的高效利用。同时,教学质量评估中心和资源应用平台的建设,旨在提升教学评估的科学性和教育资源的共享性。 智慧环境建设则侧重于基于物联网的设备管理,通过智慧教室管理系统实现教室环境的智能控制和能效管理,打造绿色、节能的校园环境。电子班牌和校园信息发布系统的建设,将作为智慧校园的核心和入口,提供教务、一卡通、图书馆等系统的集成信息。 总体而言,智慧校园整体解决方案通过集成先进技术,不仅提升了校园的信息化水平,而且优化了教学和管理流程,为学生、教师和家长提供了更加便捷、个性化的教育体验。
评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值