BigDataSmallTeam-CSDN博客

原创 Power BI connects the Azure Data Lake Gen2 Error: We couldn‘t retrieve the data model for report

Power BI connects the Azure Data Lake Gen2 Error: We couldn't retrieve the data model for reportError in below: Feedback Type:Frown (Error)Timestamp:2022-07-09T01:01:36.8926049ZLocal Time:2022-07-09T09:01:36.8926049+08:00Session ID:a7903c6f-db48-4aef-8

2022-07-09 11:07:28 823 1

原创 How to install CKAN (Data Portal) on Ubuntu 20.04 in Aliyun

Comprehensive Knowledge Archive Network (CKAN) is an open source software for open data portal, CMS that can help us manage and publish data sets. CKAN is used by central and local governments, research institutions, and other organizations to publish data

2022-06-21 20:02:53 419

原创 How to install Open Source paperless-ng in Beijing area on Aliyun

root@iZ2ze2ql9nozv8wqdc69vyZ:~# history 1 whoami 2 sudo apt update 3 sudo apt install docker.io 4 sudo systemctl enable --now docker 5 sudo usermod -aG docker paperless 6 sudo adduser paperless 7 sudo usermod -aG docker p

2022-06-13 23:38:18 190

原创 How to install Paperless with docker on Ubuntu 20.04 Aliyun

Update the repo to get latest versionsInstall the latest versionSet Docker to start on startupGive your user permissions to docker, replacing with your usernameTest it has installed correctly by getting the docker versionInstalling Paperles

2022-06-13 23:19:17 250

原创 How to install Apache Superset Docker Image on Linux CentOS7 that without internet

Step 1: Login Linux and check the CentOS version[root@bigdatasmallteam ~]# whoamiroot[root@bigdatasmallteam ~]# free total used free shared buff/cache availableMem: 16247588 1266648 11102236 27416

2022-03-16 20:50:05 3719

原创最流行的开源数据湖 Apache Iceberg

最流行的开源数据湖 Apache Iceberg--------------------大数据小团队--------------------蒙昭良：《Oracle DBA高效入门指南》作者2013-2019: 摩根大通 – 美国哥伦布2021-目前 : 普华永道 – 北京大数据小团队论坛--------------------大数据小团队--------------------...

2022-03-09 18:06:56 2287

原创 2022年Dremio 数据湖引擎的最新组件和开源数据湖Apache Iceberg

2022年Dremio 数据湖引擎的最新组件和开源数据湖Apache Iceberg--------------------大数据小团队--------------------蒙昭良：《Oracle DBA高效入门指南》作者2013-2019: 摩根大通 – 美国哥伦布2021-目前 : 普华永道 – 北京大数据小团队论坛--------------------大数据小团队---------...

2022-03-09 18:00:46 2452

原创 Dremio data lake engine architecture - 数据湖引擎架构

Dremio data lake engine architecture - 数据湖引擎架构101189×620 151 KB--------------------大数据小团队--------------------蒙昭良：《Oracle DBA高效入门指南》作者2013-2019: 摩根大通 – 美国哥伦布2021-目前 : 普华永道 – 北

2022-03-09 17:53:49 2150

原创下一代的数据湖引擎 Dremio

大数据的出现，出现了Hadoop分布式大数据系统；但Hadoop因为太过笨重，没有成功的处理大数据的解决方案。最终MapReduce的计算能力被Spark代替；同时储存的复杂性被亚马逊的S3来代替。前几年出现的存储和计算分离大数据的云服务平台Snowflake 横空出世，把Hadoop远远抛在脑后。Snowflake的数据库和数仓结合虽然节省了成本，单数还有很多各种各样的数据很难整合在数仓中。现在的数据湖引起Dremio 不仅仅可以处理不同的大数据格式，并且可以处理传统关系型数据库格式，更重要的是使用了

2022-03-07 21:49:39 593

原创 How to install CKAN on Linux CentOS 7

Installing CKAN from source — CKAN 2.9.5 documentationUS Third party data catalog:http://datacatalog.pwcinternal.com/catalog/Install CKAN 2.9 on CentOS 7:https://github.com/ckan/ckan/wiki/How-to-Install-CKAN-2.9-on-CentOS-7Step 1: Check the Linux C..

2022-01-27 20:05:03 450 1

原创 Start Airflow and airflow schedule, create airflow user in Postgresql

Start Airflow and airflow schedule, create airflow user in PostgresqlWe will first createairflow_dband a user withairflow_userandairflow_pass. Below SQL commands can help us achieve the same:Create Airflow User in PostgreSQL:root@SecondaryData...

2022-01-27 19:47:43 2056 1

原创 How to install Airflow on Linus Ubuntu 20.04 (Aliyun ECS)

Install Airflow in Ubuntu 20.04 (Aliyun ECS）Step 1: Check pip3 on Linuxroot@SecondaryDataUse:~# pwd/rootroot@SecondaryDataUse:~# pip3 --versionpip 20.0.2 from /usr/lib/python3/dist-packages/pip (python 3.8)Step 2: Install airflow using pip3

2022-01-27 19:31:54 2477

原创 Start and Stop mysql in Linux, and create database, user in Mysql database

How to change the root’s password in MySQLroot@SecondaryDataUse:~/nocodb/docker-compose/mysql# pwd/root/nocodb/docker-compose/mysqlroot@SecondaryDataUse:~/nocodb/docker-compose/mysql# sudo mysqlWelcome to the MySQL monitor. Commands end with ; or

2022-01-27 19:07:38 2030

原创 What is NocoDB, NocoDB is an open source platform that turns any database into a smart spreadsheet

NocoDB is an open source#NoCodeplatform that turns any database into a smart spreadsheetPrerequisites Must haves node.js >= 12/Docker MySql/Postgres/SQLserver/ SQLite Database Nice to haves Existing schemas can help to create AP...

2022-01-27 19:03:29 1635

原创 How to install Apache Superset on Linux CentOS 7.9

Linux: Root user updates the user’s password command:[root@iZ2zehb3zobkh31kn9cuo2Z ~]# whoamiroot[root@iZ2zehb3zobkh31kn9cuo2Z ~]# **passwd superset_user**Changing password for user superset_user.New password:Retype new password:passwd: all authen

2022-01-24 22:37:28 3716

原创 How to install Docker on Linux CentOS 7.9 (Aliyun)

Step 1: Update the system in CentOS 7[root@iZ2zehb3zobkh31kn9cuo2Z ~]# [root@iZ2zehb3zobkh31kn9cuo2Z ~]# sudo yum check-updateStep 2: download the latest version of Docker, and install it[root@iZ2zehb3zobkh31kn9cuo2Z ~]# curl -fsSL https://get.d

2022-01-24 20:51:31 350

原创查询CentOS 7 Linux的具体版本

阿里云ECS 服务器，2CPU 4GB 内存 40GB 硬盘 CentOS 7.9 系统，每月163 云2022年1月24日：星期一：CentOS 7 Linux 的具体版本：[root@iZ2zehb3zobkh31kn9cuo2Z ~]# uname -aLinux iZ2zehb3zobkh31kn9cuo2Z 3.10.0-1160.49.1.el7.x86_64#1SMP Tue Nov 30 15:51:32 UTC 2021 x86_64 x86_64 x8..

2022-01-24 19:50:40 898

原创 dbt(data build tool) is an open source data transformation, and using Jinji templating

dbt(data build tool) is an open source data transformation, and using Jinji templatingdbt(data build tool) is an open source data transformation, an...

2022-01-23 22:42:48 1822

原创 What if we put dbt(data build tool) at the center of our architecture

What if we put dbt(data build tool) at the center of our architectureUse Cases:Business Analytics:

2022-01-23 22:28:14 85

原创 What‘s dbt (Data Build Tool), open source for the transformation (T of ELT)

dbt (Data Build Tool) is a transformation workflow that lets teams quickly and collaboratively deploy analytics code following software engineering best practices like modularity, portability, CI/CD, and documentation. Now anyone who knows SQL can build pr

2022-01-23 22:16:04 1559

原创 What‘s Airbyte - Open Data Integration

Airbyte is an open-source data integration engine that helps you consolidate your data in your data warehouses, lakes and databases.Open Data Integration(ELT - Extract load transform)

2022-01-23 15:37:23 103

原创 What‘s the data governance

From :What's the data governance - Data Management - Big Data Small TeamData governance is a measure of a company’s control over its dataThe typical data flow is the following :You collect data from various sources from your busin.

2022-01-23 15:10:33 94

原创 CKAN is a “Data Management System“ and open source

From:CKAN is a “Data Management System" and open source - CKAN - Big Data Small TeamCKANis stand for ’ComprehensiveKnowledgeArchiveNetwork’.This open source CKAN is used by by national and regional government organisations throughout the Europe...

2022-01-23 14:47:22 1509

原创 How to add the Plugins on Discouse

I like the discourse plugin, these plugin like wordpress plugin, we can add some plugins when we need them.But I don’t like the way how to add the plugins. In Discourse, we have to start the application when we add the plugins. It means our website(Disco

2022-01-23 14:13:03 178

原创 Discourse 解决问题非英语标题的问题”Title seems unclear, one or more words isvery long”

Discourse 解决问题非英语标题的问题"Title seems unclear, one or more words isvery long"Discourse会自动检查标题，而且因为是国外的开发人员，大多数网站习惯使用的是英语，如果是非英语，比如汉字，一般会提示错误，不能发布成功。系统默认是设置 Title 是最大30个英语的词语。如下所示：Solution:用管理员把默认的30个字符改成300个即可（或者更长）...

2022-01-23 13:42:09 215

原创 Discourse论坛的mailgun电子邮件服务可以使用outlook和163的电子邮箱

在昨天 2022年1月22日，Discourse 不能发电子邮件到hotmail。Google了一下，大概是hotmail 阻挡了mailgun 服务器的电子邮件。刚才测试，如果Discourse使用了outlook的电子邮件，是可以发送电子邮件到outlook 的。同时，如果是国内163的电子邮件的用户，接收用户也是没有任何问题。我们使用mailgun，和Discourse 官方论坛推荐的原因，主要是可以每月可以免费1万封电子邮件。...

2022-01-23 13:08:12 1092

原创 Hotmail users do not receive account verification email in Discourse forum

Step 1: Change to the other email(example： gmail) from hotmail in the app.xmlroot@iZj6c746e7oevje00s2gumZ:~# cd /var/discourse/root@iZj6c746e7oevje00s2gumZ:/var/discourse# cd containers/root@iZj6c746e7oevje00s2gumZ:/var/discourse/containers# pwd/var.

2022-01-22 14:15:52 661

原创 Install Discourse on Linux Ubuntu20.04 in Hongkong (Aliyun)

Install Discourse on Linux Ubuntu20.04 in Hongkong (Aliyun)root@iZj6c746e7oevje00s2gumZ:~# uname -aLinux iZj6c746e7oevje00s2gumZ 5.4.0-92-generic #103-Ubuntu SMP Fri Nov 26 16:13:00 UTC 2021 x86_64 x86_64 x86_64 GNU

2022-01-22 13:09:21 6711

原创在阿里云上Create Discourse forum 论坛 2 -- 不能访问Git导致失败

Blow error when install the Discourse:root@iZ2ze472xetukni84zf7l3Z:/var/discourse# ./discourse-doctorDISCOURSE DOCTOR Sat 22 Jan 2022 08:37:26 AM CSTOS: Linux iZ2ze472xetukni84zf7l3Z 5.4.0-92-generic #103-Ubuntu SMP Fri Nov 26 16:13:00 UTC 2021 x86_6.

2022-01-22 09:29:50 1389

BigDataSmallTeam 大数据小团队--蒙昭良的博客