openmetadata本地编译环境搭建

龍尐

已于 2024-08-06 16:35:06 修改

阅读量907

点赞数 14

分类专栏： openmetadata 文章标签： openmetadata

于 2024-08-06 16:34:52 首次发布

本文链接：https://blog.csdn.net/qq_37279279/article/details/140959579

版权

openmetadata 专栏收录该内容

4 篇文章 0 订阅

订阅专栏

openmetadata本地编译环境搭建

本地环境：

Docker 20 or higher
Java JDK 17
Antlr 4.9.2 - sudo make install_antlr_cli
JQ - brew install jq (osx) apt-get install jq (Ubuntu)
Maven 3.5.x or higher - (with Java JDK 11)
Python 3.7, 3.8 or 3.9
Node >=16.0.0 & Node <= 18.0.0
Yarn ^1.22.0
Rpm (可选项，仅用于使用 maven 运行 RPM 配置文件)

查看本地依赖组件版本
```
make prerequisites
```

一.源码下载

git clone git@github.com:open-metadata/OpenMetadata.git

二.目录结构

外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传

**ingestion： **Python模块，用于metadata获取，profile、quality任务获取
**openmetadata-service：**Java后端服务
**openmetadata-ui：**前端服务
**openmetadata-oidc：**单点登录集成模块
**openmetadata-dist：**编译后的服务jar包生成模块
**conf：**后端服务运行参数配置目录
**docker：**服务编译docker-compose目录

三.编译运行

1.预编译

git clone http://jn-gitlab.hollicube.com/group-analysis/OpenMetadata.git
cd openmetadata
python3 -m venv env
source env/bin/activate
pip install pre-commit
make install_dev
make install_test precommit_instal

2.antlr4安装

sudo make install_antlr_cli

3. python环境安装

pip install -r requirments.txt

requirments.txt

adal==1.2.7
adlfs==2023.8.0
aenum==3.1.11
aiobotocore==2.5.2
aiofiles==23.1.0
aiohttp==3.8.4
aioitertools==0.11.0
aiosignal==1.3.1
alembic==1.10.4
amqp==5.1.1
aniso8601==9.0.1
antlr4-python3-runtime==4.9.2
anyio==3.7.1
apache-airflow==2.6.3
apache-airflow-providers-docker==3.7.1
apache-airflow-providers-http==4.4.2
apispec==5.2.2
appdirs==1.4.4
argcomplete==3.1.1
asgiref==3.7.2
asn1crypto==1.5.1
async-timeout==4.0.2
asynch==0.2.2
attrs==23.1.0
Authlib==1.2.1
avro==1.11.3
azure-batch==14.0.0
azure-common==1.1.28
azure-core==1.27.1
azure-cosmos==4.4.0
azure-datalake-store==0.0.53
azure-identity==1.13.0
azure-keyvault-secrets==4.7.0
azure-kusto-data==0.0.45
azure-mgmt-containerinstance==1.5.0
azure-mgmt-core==1.4.0
azure-mgmt-datafactory==1.1.0
azure-mgmt-datalake-nspkg==3.0.1
azure-mgmt-datalake-store==0.5.0
azure-mgmt-nspkg==3.0.2
azure-mgmt-resource==23.0.1
azure-nspkg==3.0.2
azure-servicebus==7.11.0
azure-storage-blob==12.16.0
azure-storage-common==2.1.0
azure-storage-file==2.1.0
azure-storage-file-datalake==12.11.0
azure-synapse-spark==0.7.0
Babel==2.12.1
backoff==2.2.1
bcrypt==4.0.1
beautifulsoup4==4.12.2
billiard==4.1.0
bitarray==2.8.2
black==23.1a1
bleach==6.1.0
blinker==1.6.2
blis==0.7.11
boto3==1.26.161
botocore==1.29.161
cached-property==1.5.2
cachelib==0.9.0
cachetools==5.3.1
catalogue==2.0.10
cattrs==23.1.2
celery==5.3.1
certifi==2023.5.7
cffi==1.15.1
chardet==4.0.0
charset-normalizer==3.1.0
ciso8601==2.3.0
click==8.1.4
click-didyoumean==0.3.0
click-plugins==1.1.1
click-repl==0.3.0
clickclick==20.10.2
clickhouse-cityhash==1.0.2.4
clickhouse-driver==0.2.6
clickhouse-sqlalchemy==0.2.4
cloudpickle==2.2.1
collate-sqllineage==1.1.5
colorama==0.4.6
coloredlogs==14.0
colorlog==4.8.0
confection==0.1.3
ConfigUpdater==3.1.1
confluent-kafka==2.1.1
connexion==2.14.2
couchbase==4.1.8
cron-descriptor==1.4.0
croniter==1.3.15
cryptography==40.0.2
cx-Oracle==8.3.0
cymem==2.0.8
dagster==1.5.3
dagster-graphql==1.5.3
dagster-pipes==1.5.3
dask==2023.6.1
databricks-cli==0.18.0
databricks-sdk==0.11.0
databricks-sql-connector==2.9.3
datamodel-code-generator==0.22.1
db-dtypes==1.1.1
dbt-artifacts-parser==0.4.2
decorator==5.1.1
delta-spark==2.3.0
Deprecated==1.2.14
diff_cover==8.0.0
dill==0.3.1.1
distlib==0.3.6
distributed==2023.6.1
dnspython==2.3.0
docker==6.1.3
docstring-parser==0.15
docutils==0.20.1
ecdsa==0.18.0
elastic-transport==8.4.1
elasticsearch==7.13.1
elasticsearch-dbapi==0.2.10
elasticsearch-dsl==7.4.1
elasticsearch8==8.9.0
email-validator==1.3.1
entrypoints==0.4
et-xmlfile==1.1.0
eventlet==0.33.3
exceptiongroup==1.1.2
fastavro==1.8.4
filelock==3.12.2
Flask==2.2.5
Flask-Admin==1.6.0
Flask-AppBuilder==4.3.1
Flask-Babel==2.0.0
Flask-Caching==2.0.2
Flask-JWT-Extended==4.5.2
Flask-Limiter==3.3.1
Flask-Login==0.6.2
Flask-Session==0.5.0
Flask-SQLAlchemy==2.5.1
Flask-WTF==1.1.1
flatdict==4.0.1
flower==2.0.0
frozenlist==1.3.3
fsspec==2022.11.0
future==0.18.3
gcloud-aio-auth==4.2.3
gcloud-aio-bigquery==6.3.0
gcloud-aio-storage==8.2.0
gcsfs==2022.11.0
genson==1.2.2
GeoAlchemy2==0.14.1
gevent==22.10.2
gitdb==4.0.10
GitPython==3.1.38
giturlparse==0.12.0
google==3.0.0
google-ads==21.2.0
google-api-core==2.11.0
google-api-python-client==2.92.0
google-auth==2.21.0
google-auth-httplib2==0.1.0
google-auth-oauthlib==1.0.0
google-cloud==0.34.0
google-cloud-aiplatform==1.27.1
google-cloud-appengine-logging==1.3.1
google-cloud-audit-log==0.2.5
google-cloud-automl==2.11.2
google-cloud-bigquery==3.11.3
google-cloud-bigquery-datatransfer==3.11.2
google-cloud-bigquery-storage==2.22.0
google-cloud-bigtable==2.19.0
google-cloud-build==3.17.1
google-cloud-compute==1.12.1
google-cloud-container==2.26.0
google-cloud-core==2.3.3
google-cloud-datacatalog==3.13.1
google-cloud-dataflow-client==0.8.4
google-cloud-dataform==0.5.2
google-cloud-dataplex==1.5.1
google-cloud-dataproc==5.4.2
google-cloud-dataproc-metastore==1.12.0
google-cloud-dlp==3.12.2
google-cloud-kms==2.18.0
google-cloud-language==2.10.1
google-cloud-logging==3.5.0
google-cloud-memcache==1.7.2
google-cloud-monitoring==2.15.1
google-cloud-orchestration-airflow==1.9.1
google-cloud-os-login==2.9.1
google-cloud-pubsub==2.17.1
google-cloud-redis==2.13.1
google-cloud-resource-manager==1.10.2
google-cloud-secret-manager==2.16.2
#google-cloud-spanner==3.36.0
google-cloud-speech==2.21.0
google-cloud-storage==1.43.0
google-cloud-tasks==2.13.2
google-cloud-texttospeech==2.14.1
google-cloud-translate==3.11.2
google-cloud-videointelligence==2.11.3
google-cloud-vision==3.4.4
google-cloud-workflows==1.10.2
google-crc32c==1.5.0
google-re2==1.0
google-resumable-media==2.5.0
googleapis-common-protos==1.59.1
gql==3.4.1
graphene==3.3
graphql-core==3.2.3
graphql-relay==3.2.0
graphviz==0.20.1
greenlet==2.0.2
grpc-google-iam-v1==0.12.6
grpcio==1.59.0
grpcio-gcp==0.2.2
grpcio-health-checking==1.59.0
grpcio-status==1.56.0
grpcio-tools==1.59.0
gunicorn==20.1.0
h11==0.14.0
hdbcli==2.18.24
httpcore==0.16.3
httplib2==0.22.0
httpx==0.23.3
humanfriendly==10.0
humanize==4.7.0
hvac==1.1.1
idna==2.10
importlib-metadata==6.7.0
importlib-resources==5.12.0
impyla==0.18.0
inflect==5.6.2
inflection==0.5.1
iniconfig==2.0.0
isodate==0.6.1
isort==5.12.0
itsdangerous==2.1.2
Jinja2==3.1.2
jmespath==0.10.0
joblib==1.3.2
json-merge-patch==0.2
jsonpatch==1.32
jsonpath-ng==1.5.3
jsonpointer==2.4
jsonschema==4.17.3
jsonschema-spec==0.1.6
jsonschema-specifications==2023.6.1
kerberos==1.3.1
kombu==5.3.1
kubernetes==23.6.0
kubernetes-asyncio==24.2.3
langcodes==3.3.0
lazy-object-proxy==1.9.0
ldap3==2.9.1
leb128==1.0.5
limits==3.5.0
linkify-it-py==2.0.2
lkml==1.3.1
locket==1.0.0
lockfile==0.12.2
looker-sdk==23.10.0
lxml==4.9.3
lz4==4.3.2
Mako==1.2.4
Markdown==3.4.3
markdown-it-py==3.0.0
MarkupSafe==2.1.3
marshmallow==3.19.0
marshmallow-enum==1.5.1
marshmallow-oneofschema==3.0.1
marshmallow-sqlalchemy==0.26.1
mdit-py-plugins==0.4.0
mdurl==0.1.2
memory-profiler==0.61.0
mlflow-skinny==2.7.1
msal==1.22.0
msal-extensions==1.0.0
msgpack==1.0.5
msrest==0.7.1
msrestazure==0.6.4
multidict==6.0.4
murmurhash==1.0.10
mypy-boto3-appflow==1.28.0
mypy-boto3-rds==1.28.0
mypy-boto3-redshift-data==1.28.0
mypy-boto3-s3==1.28.0
mypy-extensions==1.0.0
mysqlclient==2.1.1
neo4j==5.3.0
networkx==3.1
numpy==1.24.4
oauthlib==3.2.2
okta==2.9.3
openapi-schema-validator==0.4.4
openapi-spec-validator==0.5.7
openmetadata-ingestion==1.2.0.0.dev0
openmetadata-managed-apis==1.2.0.0.dev0
openpyxl==3.1.2
oracledb==1.4.2
ordered-set==4.1.0
oscrypto==1.3.0
packaging==21.3
pandas==1.3.5
pandas-gbq==0.19.2
paramiko==3.2.0
partd==1.4.0
pathable==0.4.3
pathspec==0.9.0
pathy==0.10.2
pendulum==2.1.2
phonenumbers==8.13.23
pinotdb==0.5.0
platformdirs==3.8.1
pluggy==1.2.0
ply==3.11
portalocker==2.7.0
prance==23.6.21.0
preshed==3.0.9
presidio-analyzer==2.2.32
presto-types-parser==0.0.2
prison==0.2.1
prometheus-client==0.17.0
prompt-toolkit==3.0.39
proto-plus==1.22.3
protobuf==4.23.4
psutil==5.9.5
#psycopg2==2.9.9
pure-sasl==0.6.2
py4j==0.10.9.5
pyarrow==10.0.1
pyasn1==0.4.8
pyasn1-modules==0.2.8
PyAthena==3.0.8
pycparser==2.21
pycryptodomex==3.18.0
pydantic==1.10.11
pydash==7.0.6
pydata-google-auth==1.8.0
pydomo==0.3.0.10
pydruid==0.6.5
Pygments==2.15.1
pyhcl==0.4.4
PyHive==0.7.0
PyJWT==2.7.0
pymongo==4.5.0
pymssql==2.2.9
PyMySQL==1.1.0
PyNaCl==1.5.0
pyodbc==4.0.39
pyOpenSSL==23.2.0
pyparsing==3.1.0
pyrsistent==0.19.3
PySnooper==1.2.0
pyspark==3.3.3
pytest==7.4.2
python-daemon==3.0.1
python-dateutil==2.8.2
python-dotenv==1.0.0
python-http-client==3.3.7
python-jose==3.3.0
python-ldap==3.4.3
python-nvd3==0.15.0
python-on-whales==0.55.0
python-slugify==8.0.1
python-snappy==0.6.1
python-tds==1.13.0
pytz==2023.3
pytzdata==2020.1
PyYAML==6.0
redis==4.6.0
redshift-connector==2.0.912
referencing==0.29.1
regex==2023.10.3
requests==2.31.0
requests-aws4auth==1.2.3
requests-file==1.5.1
requests-oauthlib==1.3.1
requests-toolbelt==0.10.1
rfc3339-validator==0.1.4
rfc3986==1.5.0
rich==13.4.2
rich-argparse==1.2.0
rpds-py==0.8.8
rsa==4.9
ruamel.yaml==0.17.35
ruamel.yaml.clib==0.2.8
s3fs==0.4.2
s3transfer==0.6.1
sasl==0.3.1
scikit-learn==1.3.1
scipy==1.11.3
scramp==1.4.4
sendgrid==6.10.0
setproctitle==1.3.2
Shapely==1.8.5.post1
simple-salesforce==1.11.4
six==1.16.0
slack-sdk==3.21.3
smart-open==6.4.0
smmap==5.0.1
sniffio==1.3.0
snowflake-connector-python==3.0.4
snowflake-sqlalchemy==1.4.7
sortedcontainers==2.4.0
soupsieve==2.4.1
spacy==3.5.0
spacy-legacy==3.0.12
spacy-loggers==1.0.5
SQLAlchemy==1.4.49
sqlalchemy-bigquery==1.6.1
sqlalchemy-databricks==0.2.0
sqlalchemy-hana==0.5.0
SQLAlchemy-JSONField==1.0.1.post0
sqlalchemy-pgspider==0.1.0
sqlalchemy-pytds==0.3.5
sqlalchemy-redshift==0.8.12
SQLAlchemy-Utils==0.41.1
sqlalchemy-vertica==0.0.5
sqlfluff==2.1.4
sqlparse==0.4.3
srsly==2.4.8
sshtunnel==0.4.0
starkbank-ecdsa==2.2.0
starlette==0.31.1
statsd==4.0.1
tableau-api-lib==0.1.50
tabulate==0.9.0
tblib==2.0.0
tenacity==8.2.2
termcolor==2.3.0
text-unidecode==1.3
thinc==8.1.12
threadpoolctl==3.2.0
thrift==0.16.0
thrift-sasl==0.4.3
tldextract==5.0.0
toml==0.10.2
tomli==2.0.1
toolz==0.12.0
toposort==1.10
tornado==6.3.2
tqdm==4.66.1
trino==0.327.0
typeguard==4.0.0
typer==0.7.0
typing-compat==0.1.0
typing-inspect==0.9.0
typing_extensions==4.5.0
tzdata==2023.3
tzlocal==5.1
uc-micro-py==1.0.2
unicodecsv==0.14.1
universal-pathlib==0.1.3
uritemplate==4.1.1
urllib3==1.26.16
vertica-python==1.3.5
vine==5.0.0
virtualenv==20.23.1
wasabi==1.1.2
watchdog==3.0.0
watchtower==2.0.1
wcwidth==0.2.6
webencodings==0.5.1
websocket-client==1.6.1
Werkzeug==2.2.3
wrapt==1.15.0
WTForms==3.0.1
xmltodict==0.13.0
yarl==1.9.2
zict==3.0.0
zipp==3.15.0
zope.event==5.0
zope.interface==6.0
zstd==1.5.5.1

python 源码编译（ingestion模块构建）
```
make generate
```

5. 前端源码构建

# 安装依赖，编译json生成ts文件
make yarn_install_cache
# 启动本地项目
make yarn_start_dev_ui

6. 服务后端配置文件修改（conf/openmetadata.yaml）

#  Copyright 2021 Collate
#  Licensed under the Apache License, Version 2.0 (the "License");
#  you may not use this file except in compliance with the License.
#  You may obtain a copy of the License at
#  http://www.apache.org/licenses/LICENSE-2.0
#  Unless required by applicable law or agreed to in writing, software
#  distributed under the License is distributed on an "AS IS" BASIS,
#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
#  See the License for the specific language governing permissions and
#  limitations under the License.

clusterName: ${OPENMETADATA_CLUSTER_NAME:-openmetadata}

swagger:
resourcePackage: org.openmetadata.service.resources


server:
rootPath: '/api/*'
applicationConnectors:
 - type: http
   port: ${SERVER_PORT:-8585}
adminConnectors:
 - type: http
   port: ${SERVER_ADMIN_PORT:-8586}

logging:
level: ${LOG_LEVEL:-INFO}
loggers:
 org.openmetadata.service.util.OpenMetadataSetup:
   level: INFO
   appenders:
     - type: console
       logFormat: "%msg%n"
       timeZone: UTC
     - type: file
       logFormat: "%level [%d{ISO8601,UTC}] [%t] %logger{5} - %msg%n"
       layout:
         type: json
       currentLogFilename: ./logs/openmetadata-operations.log
       archivedLogFilenamePattern: ./logs/openmetadata-operations-%d{yyyy-MM-dd}-%i.log.gz
       archivedFileCount: 7
       timeZone: UTC
       maxFileSize: 50MB
appenders:
 - type: console
   threshold: TRACE
   logFormat: "%level [%d{ISO8601,UTC}] [%t] %logger{5} - %msg%n"
   timeZone: UTC
 - type: file
   layout:
     type: json
     appendLineSeparator: true
   filterFactories:
     - type: audit-exclude-filter-factory
   threshold: TRACE
   logFormat: "%level [%d{ISO8601,UTC}] [%t] %logger{5} - %msg%n"
   currentLogFilename: ./logs/openmetadata.log
   archivedLogFilenamePattern: ./logs/openmetadata-%d{yyyy-MM-dd}-%i.log.gz
   archivedFileCount: 7
   timeZone: UTC
   maxFileSize: 50MB
 - type: file
   layout:
     type: json
     appendLineSeparator: true
   filterFactories:
     - type: audit-only-filter-factory
   threshold: TRACE
   logFormat: "%level [%d{ISO8601,UTC}] [%t] %logger{5} - %msg%n"
   currentLogFilename: ./logs/audit.log
   archivedLogFilenamePattern: ./logs/audit-%d{yyyy-MM-dd}-%i.log.gz
   archivedFileCount: 25
   timeZone: UTC
   maxFileSize: 50MB

database:
# the name of the JDBC driver, mysql in our case
driverClass: ${DB_DRIVER_CLASS:-org.postgresql.Driver}
#${DB_DRIVER_CLASS:-com.mysql.cj.jdbc.Driver}
# the username and password
user: ${DB_USER:-postgres}
password: ${DB_USER_PASSWORD:-PostDb#4erT}
url: jdbc:postgresql://20.5.2.134:15432/openmetadata_db?allowPublicKeyRetrieval=true&useSSL=false&serverTimezone=UTC
#  user: ${DB_USER:-openmetadata_user}
#  password: ${DB_USER_PASSWORD:-openmetadata_password}
#  url: jdbc:postgresql://127.0.0.1:5432/openmetadata_db?allowPublicKeyRetrieval=true&useSSL=false&serverTimezone=UTC
#jdbc:${DB_SCHEME:-mysql}://${DB_HOST:-localhost}:${DB_PORT:-3306}/${OM_DATABASE:-openmetadata_db}?${DB_PARAMS:-allowPublicKeyRetrieval=true&useSSL=false&serverTimezone=UTC}

migrationConfiguration:
flywayPath: "./bootstrap/sql/migrations/flyway"
nativePath: "./bootstrap/sql/migrations/native"
extensionPath: ""

# Authorizer Configuration
authorizerConfiguration:
className: ${AUTHORIZER_CLASS_NAME:-org.openmetadata.service.security.DefaultAuthorizer}
containerRequestFilter: ${AUTHORIZER_REQUEST_FILTER:-org.openmetadata.service.security.JwtFilter}
#adminPrincipals: ${AUTHORIZER_ADMIN_PRINCIPALS:-[admin]}
adminPrincipals:
 - "admin-user"
 - "admin"
 - "root"
allowedEmailRegistrationDomains: ${AUTHORIZER_ALLOWED_REGISTRATION_DOMAIN:-["all"]}
principalDomain: ${AUTHORIZER_PRINCIPAL_DOMAIN:-"hollysys.com"}
enforcePrincipalDomain: ${AUTHORIZER_ENFORCE_PRINCIPAL_DOMAIN:-false}
enableSecureSocketConnection : ${AUTHORIZER_ENABLE_SECURE_SOCKET:-false}

authenticationConfiguration:
provider: ${AUTHENTICATION_PROVIDER:-basic}
# This is used by auth provider provide response as either id_token or code
responseType: ${AUTHENTICATION_RESPONSE_TYPE:-id_token}
# This will only be valid when provider type specified is customOidc
providerName: ${CUSTOM_OIDC_AUTHENTICATION_PROVIDER_NAME:-""}
publicKeyUrls: ${AUTHENTICATION_PUBLIC_KEYS:-[http://localhost:8585/api/v1/system/config/jwks]}
authority: ${AUTHENTICATION_AUTHORITY:-https://accounts.google.com}
clientId: ${AUTHENTICATION_CLIENT_ID:-""}
callbackUrl: ${AUTHENTICATION_CALLBACK_URL:-""}
jwtPrincipalClaims: ${AUTHENTICATION_JWT_PRINCIPAL_CLAIMS:-[email,preferred_username,sub]}
enableSelfSignup : ${AUTHENTICATION_ENABLE_SELF_SIGNUP:-true}
#  provider: "custom-oidc"
#  providerName: "IDAAS"
#  responseType: "code" #${AUTHENTICATION_RESPONSE_TYPE:-id_token}
#  publicKeyUrls:
#    #    - "http://localhost:8081/realms/data-sec/protocol/openid-connect/certs"
#    - "http://localhost:8111/oauth2/keys"
#    - "http://localhost:8585/api/v1/system/config/jwks" # Update with your Domain and Make sure this "/api/v1/config/jwks" is always configured to enable JWT tokens
#  #  authority: "http://localhost:8081/realms/data-sec"
#  authority: "http://localhost:8111"
#  #  clientId: "open-metadata"
#  clientId: "openmetadata"
#  #  callbackUrl: "http://localhost:8585/callback"
#  callbackUrl: "http://localhost:8111/callback"
#  jwtPrincipalClaims: [email,preferred_username,sub] #${AUTHENTICATION_JWT_PRINCIPAL_CLAIMS:-[email,preferred_username,sub]}
#  enableSelfSignup : ${AUTHENTICATION_ENABLE_SELF_SIGNUP:-true}
samlConfiguration:
 debugMode: ${SAML_DEBUG_MODE:-false}
 idp:
   entityId: ${SAML_IDP_ENTITY_ID:-""}
   ssoLoginUrl: ${SAML_IDP_SSO_LOGIN_URL:-""}
   idpX509Certificate: ${SAML_IDP_CERTIFICATE:-""}
   authorityUrl: ${SAML_AUTHORITY_URL:-"http://localhost:8585/api/v1/saml/login"}
   nameId: ${SAML_IDP_NAME_ID:-"urn:oasis:names:tc:SAML:2.0:nameid-format:emailAddress"}
 sp:
   entityId: ${SAML_SP_ENTITY_ID:-"http://localhost:8585/api/v1/saml/metadata"}
   acs: ${SAML_SP_ACS:-"http://localhost:8585/api/v1/saml/acs"}
   spX509Certificate: ${SAML_SP_CERTIFICATE:-""}
   callback: ${SAML_SP_CALLBACK:-"http://localhost:8585/saml/callback"}
 security:
   strictMode: ${SAML_STRICT_MODE:-false}
   tokenValidity: ${SAML_SP_TOKEN_VALIDITY:-"3600"}
   sendEncryptedNameId: ${SAML_SEND_ENCRYPTED_NAME_ID:-false}
   sendSignedAuthRequest: ${SAML_SEND_SIGNED_AUTH_REQUEST:-false}
   signSpMetadata: ${SAML_SIGNED_SP_METADATA:-false}
   wantMessagesSigned: ${SAML_WANT_MESSAGE_SIGNED:-false}
   wantAssertionsSigned: ${SAML_WANT_ASSERTION_SIGNED:-false}
   wantAssertionEncrypted: ${SAML_WANT_ASSERTION_ENCRYPTED:-false}
   wantNameIdEncrypted: ${SAML_WANT_NAME_ID_ENCRYPTED:-false}
   keyStoreFilePath: ${SAML_KEYSTORE_FILE_PATH:-""}
   keyStoreAlias: ${SAML_KEYSTORE_ALIAS:-""}
   keyStorePassword: ${SAML_KEYSTORE_PASSWORD:-""}
ldapConfiguration:
 host: ${AUTHENTICATION_LDAP_HOST:-}
 port: ${AUTHENTICATION_LDAP_PORT:-}
 dnAdminPrincipal: ${AUTHENTICATION_LOOKUP_ADMIN_DN:-""}
 dnAdminPassword: ${AUTHENTICATION_LOOKUP_ADMIN_PWD:-""}
 userBaseDN: ${AUTHENTICATION_USER_LOOKUP_BASEDN:-""}
 groupBaseDN: ${AUTHENTICATION_GROUP_LOOKUP_BASEDN:-""}
 roleAdminName: ${AUTHENTICATION_USER_ROLE_ADMIN_NAME:-}
 allAttributeName: ${AUTHENTICATION_USER_ALL_ATTR:-}
 mailAttributeName: ${AUTHENTICATION_USER_MAIL_ATTR:-}
 usernameAttributeName: ${AUTHENTICATION_USER_NAME_ATTR:-}
 groupAttributeName: ${AUTHENTICATION_USER_GROUP_ATTR:-}
 groupAttributeValue: ${AUTHENTICATION_USER_GROUP_ATTR_VALUE:-}
 groupMemberAttributeName: ${AUTHENTICATION_USER_GROUP_MEMBER_ATTR:-}
 #the mapping of roles to LDAP groups
 authRolesMapping: ${AUTH_ROLES_MAPPING:-""}
 authReassignRoles: ${AUTH_REASSIGN_ROLES:-[]}
 #optional
 maxPoolSize: ${AUTHENTICATION_LDAP_POOL_SIZE:-3}
 sslEnabled: ${AUTHENTICATION_LDAP_SSL_ENABLED:-}
 truststoreConfigType: ${AUTHENTICATION_LDAP_TRUSTSTORE_TYPE:-TrustAll}
 trustStoreConfig:
   customTrustManagerConfig:
     trustStoreFilePath: ${AUTHENTICATION_LDAP_TRUSTSTORE_PATH:-}
     trustStoreFilePassword: ${AUTHENTICATION_LDAP_KEYSTORE_PASSWORD:-}
     trustStoreFileFormat: ${AUTHENTICATION_LDAP_SSL_KEY_FORMAT:-}
     verifyHostname: ${AUTHENTICATION_LDAP_SSL_VERIFY_CERT_HOST:-}
     examineValidityDates: ${AUTHENTICATION_LDAP_EXAMINE_VALIDITY_DATES:-}
   hostNameConfig:
     allowWildCards: ${AUTHENTICATION_LDAP_ALLOW_WILDCARDS:-}
     acceptableHostNames: ${AUTHENTICATION_LDAP_ALLOWED_HOSTNAMES:-[]}
   jvmDefaultConfig:
     verifyHostname: ${AUTHENTICATION_LDAP_SSL_VERIFY_CERT_HOST:-}
   trustAllConfig:
     examineValidityDates: ${AUTHENTICATION_LDAP_EXAMINE_VALIDITY_DATES:-true}

jwtTokenConfiguration:
rsapublicKeyFilePath: ${RSA_PUBLIC_KEY_FILE_PATH:-"./conf/public_key.der"}
rsaprivateKeyFilePath: ${RSA_PRIVATE_KEY_FILE_PATH:-"./conf/private_key.der"}
jwtissuer: ${JWT_ISSUER:-"http://iam.jn-sit.ieccloud.hollicube.com/"}
keyId: ${JWT_KEY_ID:-"Gb389a-9f76-gdjs-a92j-0242bk94356"}

elasticsearch:
searchType: ${SEARCH_TYPE:- "elasticsearch"}
host: ${ELASTICSEARCH_HOST:-holli-om-es-svc.jn-sit.ieccloud.hollicube.com}
port: ${ELASTICSEARCH_PORT:-80}
#  host: ${ELASTICSEARCH_HOST:-localhost}
#  port: ${ELASTICSEARCH_PORT:-9200}
scheme: ${ELASTICSEARCH_SCHEME:-http}
username: ${ELASTICSEARCH_USER:-""}
password: ${ELASTICSEARCH_PASSWORD:-""}
clusterAlias: ${ELASTICSEARCH_CLUSTER_ALIAS:-"openmetadata_local"}
truststorePath: ${ELASTICSEARCH_TRUST_STORE_PATH:-""}
truststorePassword: ${ELASTICSEARCH_TRUST_STORE_PASSWORD:-""}
connectionTimeoutSecs: ${ELASTICSEARCH_CONNECTION_TIMEOUT_SECS:-5}
socketTimeoutSecs: ${ELASTICSEARCH_SOCKET_TIMEOUT_SECS:-60}
keepAliveTimeoutSecs: ${ELASTICSEARCH_KEEP_ALIVE_TIMEOUT_SECS:-600}
batchSize: ${ELASTICSEARCH_BATCH_SIZE:-10}
searchIndexMappingLanguage : ${ELASTICSEARCH_INDEX_MAPPING_LANG:-EN}
searchIndexFactoryClassName : org.openmetadata.service.search.SearchIndexFactory

eventMonitoringConfiguration:
eventMonitor: ${EVENT_MONITOR:-prometheus}  # Possible values are "prometheus", "cloudwatch"
batchSize: ${EVENT_MONITOR_BATCH_SIZE:-10}
pathPattern: ${EVENT_MONITOR_PATH_PATTERN:-["/api/v1/tables/*", "/api/v1/health-check"]}
latency: ${EVENT_MONITOR_LATENCY:-[]} # For value p99=0.99, p90=0.90, p50=0.50 etc.
# it will use the default auth provider for AWS services if parameters are not set
# parameters:
#   region: ${OM_MONITOR_REGION:-""}
#   accessKeyId: ${OM_MONITOR_ACCESS_KEY_ID:-""}
#   secretAccessKey: ${OM_MONITOR_ACCESS_KEY:-""}

eventHandlerConfiguration:
eventHandlerClassNames:
 - "org.openmetadata.service.events.AuditEventHandler"
 - "org.openmetadata.service.events.ChangeEventHandler"
 - "org.openmetadata.service.events.WebAnalyticEventHandler"

pipelineServiceClientConfiguration:
enabled: ${PIPELINE_SERVICE_CLIENT_ENABLED:-true}
# If we don't need this, set "org.openmetadata.service.clients.pipeline.noop.NoopClient"
className: ${PIPELINE_SERVICE_CLIENT_CLASS_NAME:-"org.openmetadata.service.clients.pipeline.airflow.AirflowRESTClient"}
apiEndpoint: ${PIPELINE_SERVICE_CLIENT_ENDPOINT:-http://host.docker.internal:8080}
metadataApiEndpoint: ${SERVER_HOST_API_URL:-http://host.docker.internal:8585/api}
ingestionIpInfoEnabled: ${PIPELINE_SERVICE_IP_INFO_ENABLED:-false}
hostIp: ${PIPELINE_SERVICE_CLIENT_HOST_IP:-""}
healthCheckInterval: ${PIPELINE_SERVICE_CLIENT_HEALTH_CHECK_INTERVAL:-300}
# This SSL information is about the OpenMetadata server.
# It will be picked up from the pipelineServiceClient to use/ignore SSL when connecting to the OpenMetadata server.
verifySSL: ${PIPELINE_SERVICE_CLIENT_VERIFY_SSL:-"no-ssl"} # Possible values are "no-ssl", "ignore", "validate"
sslConfig:
 certificatePath: ${PIPELINE_SERVICE_CLIENT_SSL_CERT_PATH:-""} # Local path for the Pipeline Service Client

# Secrets Manager Loader: specify to the Ingestion Framework how to load the SM credentials from its env
# Supported: noop, airflow, env
secretsManagerLoader: ${PIPELINE_SERVICE_CLIENT_SECRETS_MANAGER_LOADER:-"noop"}

# Default required parameters for Airflow as Pipeline Service Client
parameters:
 username: ${AIRFLOW_USERNAME:-admin}
 password: ${AIRFLOW_PASSWORD:-admin}
 timeout: ${AIRFLOW_TIMEOUT:-10}
 # If we need to use SSL to reach Airflow
 truststorePath: ${AIRFLOW_TRUST_STORE_PATH:-""}
 truststorePassword: ${AIRFLOW_TRUST_STORE_PASSWORD:-""}

# no_encryption_at_rest is the default value, and it does what it says. Please read the manual on how
# to secure your instance of OpenMetadata with TLS and encryption at rest.
fernetConfiguration:
fernetKey: ${FERNET_KEY:-jJ/9sz0g0OHxsfxOoSfdFdmk3ysNmPRnH3TUAbz3IHA=}

secretsManagerConfiguration:
secretsManager: ${SECRET_MANAGER:-db} # Possible values are "db", "managed-aws", "managed-aws-ssm"
prefix: ${SECRET_MANAGER_PREFIX:-""} # Define the secret key ID as /<prefix>/<clusterName>/<key>
tags: ${SECRET_MANAGER_TAGS:-[]} # Add tags to the created resource. Format is `[key1:value1,key2:value2,...]`
# it will use the default auth provider for the secrets' manager service if parameters are not set
parameters:
 ## For AWS
 region: ${OM_SM_REGION:-""}
 accessKeyId: ${OM_SM_ACCESS_KEY_ID:-""}
 secretAccessKey: ${OM_SM_ACCESS_KEY:-""}
 ## For Azure Key Vault
 clientId: ${OM_SM_CLIENT_ID:-""}
 clientSecret: ${OM_SM_CLIENT_SECRET:-""}
 tenantId: ${OM_SM_TENANT_ID:-""}
 vaultName: ${OM_SM_VAULT_NAME:-""}

health:
delayedShutdownHandlerEnabled: true
shutdownWaitPeriod: 1s
healthChecks:
 - name: OpenMetadataServerHealthCheck
   critical: true
   schedule:
     checkInterval: 2500ms
     downtimeInterval: 10s
     failureAttempts: 2
     successAttempts: 1

email:
emailingEntity: ${OM_EMAIL_ENTITY:-"OpenMetadata"}
supportUrl: ${OM_SUPPORT_URL:-"https://slack.open-metadata.org"}
enableSmtpServer : ${AUTHORIZER_ENABLE_SMTP:-false}
openMetadataUrl: ${OPENMETADATA_SERVER_URL:-"http://localhost:8585"}
senderMail: ${OPENMETADATA_SMTP_SENDER_MAIL:-""}
serverEndpoint: ${SMTP_SERVER_ENDPOINT:-""}
serverPort: ${SMTP_SERVER_PORT:-""}
username: ${SMTP_SERVER_USERNAME:-""}
password: ${SMTP_SERVER_PWD:-""}
transportationStrategy: ${SMTP_SERVER_STRATEGY:-"SMTP_TLS"}

web:
uriPath: ${WEB_CONF_URI_PATH:-"/api"}
hsts:
 enabled: ${WEB_CONF_HSTS_ENABLED:-false}
 maxAge: ${WEB_CONF_HSTS_MAX_AGE:-"365 days"}
 includeSubDomains: ${WEB_CONF_HSTS_INCLUDE_SUBDOMAINS:-"true"}
 preload: ${WEB_CONF_HSTS_PRELOAD:-"true"}
frame-options:
 enabled: ${WEB_CONF_FRAME_OPTION_ENABLED:-false}
 option: ${WEB_CONF_FRAME_OPTION:-"SAMEORIGIN"}
 origin: ${WEB_CONF_FRAME_ORIGIN:-""}
content-type-options:
 enabled: ${WEB_CONF_CONTENT_TYPE_OPTIONS_ENABLED:-false}
xss-protection:
 enabled: ${WEB_CONF_XSS_PROTECTION_ENABLED:-false}
 on: ${WEB_CONF_XSS_PROTECTION_ON:-true}
 block: ${WEB_CONF_XSS_PROTECTION_BLOCK:-true}
csp:
 enabled: ${WEB_CONF_XSS_CSP_ENABLED:-false}
 policy: ${WEB_CONF_XSS_CSP_POLICY:-"default-src 'self'"}
 reportOnlyPolicy: ${WEB_CONF_XSS_CSP_REPORT_ONLY_POLICY:-""}
referrer-policy:
 enabled: ${WEB_CONF_REFERRER_POLICY_ENABLED:-false}
 option: ${WEB_CONF_REFERRER_POLICY_OPTION:-"SAME_ORIGIN"}
permission-policy:
 enabled: ${WEB_CONF_PERMISSION_POLICY_ENABLED:-false}
 option: ${WEB_CONF_PERMISSION_POLICY_OPTION:-""}

dataQualityConfiguration:
severityIncidentClassifier: ${DATA_QUALITY_SEVERITY_INCIDENT_CLASSIFIER:-"org.openmetadata.service.util.incidentSeverityClassifier.LogisticRegressionIncidentSeverityClassifier"}

7. es容器版启动

docker compose up -f docker/docker-compose-es/docker-compose-es.yml

8. Ingestion 容器版启动

docker compose up -f docker/docker-compose-ingestion/docker-compose-ingestion.yml

9. java源码编译及后端服务启动

maven编译

mvn clean install -DskipTests=true

数据库初始化

./bootstrap/bootstrap_storage.sh drop-create-all

（一）编译源码jar启动

1.服务启动

cd openmetadata-dist/target/

tar -zxvf openmetadata-1.3.1.tar.gz

cd openmetadata-1.3.1/

sh bin/openmetadata-server-start.sh conf/openmetadata.yaml

外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传

2.服务界面

http://localhost:8585/

账号密码：admin/admin

外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传

（二）前后端分别启动：

1.后端启动

org.openmetadata.service.OpenMetadataApplication

server ./conf/openmetadata.yaml

外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传

后端日志输出：

外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传

2.前端服务启动启动

# 安装依赖，编译json生成ts文件
make yarn_install_cache
# 启动本地项目
make yarn_start_dev_ui

源码编译控制台日志：

外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传

服务启动日志输出：

外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传

3. 服务界面

http://localhost:3000/

账号密码：admin/admin

外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传

龍尐

关注

14
点赞
踩
8

收藏

觉得还不错? 一键收藏
0
评论
openmetadata本地编译环境搭建

Docker 20 or higherJava JDK 17Antlr 4.9.2 - JQ - (osx) (Ubuntu)Maven 3.5.x or higher - (with Java JDK 11)Python 3.7, 3.8 or 3.9Node >=16.0.0 & Node
复制链接

扫一扫