skywalking服务端_SkyWalking 后端服务存储配置

描述

采集的数据需要存储，提供给相关人员查看系统调用链路信息，因此需要配置存储项。

一般在本地尝鲜可以使用默认的h2数据库。可以很快的部署SkyWalking服务，体验相关功能。

开发环境可以使用MySQL数据库。

测试和生产环境建议使用elasticsearch。

配置

在application.yml里面，配置storage模块。

具体配置项如下：

selector：存储选择器

elasticsearch：elasticsearch6.X以下配置项

nameSpace：命名空间，如：SkyWalking

clusterNodes：集群节点，如：localhost:9200

protocol：协议，如：HTTP

trustStorePath：SSL配置路径

trustStorePass：SSL配置，7.0.0之后可以在secretsManagementFile里面管理

user：用户名，7.0.0之后可以在secretsManagementFile里面管理

password：密码，7.0.0之后可以在secretsManagementFile里面管理

secretsManagementFile：安全管理文件

enablePackedDownsampling：如果开启，则采样索引(小时和天精度)将向下合并到分钟索引中，es7.0.0的一个增强功能，降低50%的索引数(移除了天和小时的索引需求)，但是性能和之前的几乎一致。对于es6.x版本的影响只是保持小时和天的索引，用户需要手动移除他们。

dayStep：es6.x版本，分钟/小时/月粒度的指标和追踪索引被整合到天里面，ES每天创建一个新的索引。

indexShardsNumber：索引分片数

indexReplicasNumber：索引备份数

recordDataTTL：记录有效时长

otherMetricsDataTTL：其他指标有效时长(分钟/小时/天)

monthMetricsDataTTL：月度指标有效时长

bulkActions：批量操作数

flushInterval：刷盘间隔时间

concurrentRequests：并发请求数

resultWindowMaxSize：结果窗口最大值

metadataQueryMaxSize：元数据查询最大值

segmentQueryMaxSize：追踪段最大查询数

profileTaskQueryMaxSize：性能剖析最大查询数

advanced：

elasticsearch7：elasticsearch7.X以下配置项

nameSpace：命名空间

clusterNodes：集群节点

protocol：协议

trustStorePath：SSL文件路径

trustStorePass：SSL PASS项，7.0.0之后可以在secretsManagementFile里面管理

enablePackedDownsampling： 如果开启，则采样索引(小时和天精度)将向下合并到分钟索引中，es7.0.0的一个增强功能，降低50%的索引数(移除了天和小时的索引需求)，但是性能和之前的几乎一致。

dayStep：

user：用户名，7.0.0之后可以在secretsManagementFile里面管理

password：密码，7.0.0之后可以在secretsManagementFile里面管理

secretsManagementFile：安全管理文件

indexShardsNumber：分片数

indexReplicasNumber：副本数

recordDataTTL：记录有效时长

otherMetricsDataTTL：其他指标有效时长(分钟/小时/天)

monthMetricsDataTTL：月度指标有效时长

bulkActions：批量操作数

flushInterval：刷盘间隔时间

concurrentRequests：并发请求数

resultWindowMaxSize：结果窗口最大值

metadataQueryMaxSize：元数据查询最大值

segmentQueryMaxSize：追踪段最大查询数

profileTaskQueryMaxSize：性能剖析最大查询数

advanced：

h2：h2把内存数据库

driver：驱动

url：数据库URL

user：用户名

metadataQueryMaxSize：查询最大值

mysql：MySQL数据库

properties：

jdbcUrl：连接URL

dataSource.user：用户名

dataSource.password：密码

dataSource.cachePrepStmts：预加载缓存

dataSource.prepStmtCacheSize：预加载缓存大小

dataSource.prepStmtCacheSqlLimit：预加载SQL限制

dataSource.useServerPrepStmts：开启服务端预加载机制

metadataQueryMaxSize：最大查询数

influxdb：

metabaseType：数据库类型(目前可选择：h2, mysql)

h2Props：

dataSourceClassName：数据库驱动

dataSource.url：数据库URL

dataSource.user：用户名

dataSource.password：密码

mysqlProps：

jdbcUrl：连接URL

dataSource.user：用户名

dataSource.password：密码

dataSource.cachePrepStmts：预加载缓存

dataSource.prepStmtCacheSize：预加载缓存大小

dataSource.prepStmtCacheSqlLimit：预加载SQL限制

dataSource.useServerPrepStmts：开启服务端预加载机制

metadataQueryMaxSize：最大查询数

url：InfluxDB连接URL

user：InfluxDB用户名

password：InfluxDB密码

database：InfluxDB数据库

actions：InfluxDB收集活动数

duration：InfluxDB等待时长

fetchTaskLogMaxSize：请求中提取任务日志的最大数量

示例

具体配置在 application.yml 的 storage 模块 。 可以看出，配置项都是可以使用运行时参数替换的，也就是说，这些配置都可以在运行时指定参数。 一般除了连接项，如IP，端口，用户名，密码这样的必须指定的配置项，其他配置建议使用默认值。

storage:

selector: ${SW_STORAGE:h2}

elasticsearch:

nameSpace: ${SW_NAMESPACE:"skywalking"}

clusterNodes: ${SW_STORAGE_ES_CLUSTER_NODES:172.18.20.107:9220}

protocol: ${SW_STORAGE_ES_HTTP_PROTOCOL:"http"}

trustStorePath: ${SW_SW_STORAGE_ES_SSL_JKS_PATH:"../es_keystore.jks"}

trustStorePass: ${SW_SW_STORAGE_ES_SSL_JKS_PASS:""}

user: ${SW_ES_USER:""}

password: ${SW_ES_PASSWORD:""}

secretsManagementFile: ${SW_ES_SECRETS_MANAGEMENT_FILE:""} # Secrets management file in the properties format includes the username, password, which are managed by 3rd party tool.

enablePackedDownsampling: ${SW_STORAGE_ENABLE_PACKED_DOWNSAMPLING:true} # Hour and Day metrics will be merged into minute index.

dayStep: ${SW_STORAGE_DAY_STEP:1} # Represent the number of days in the one minute/hour/day index.

indexShardsNumber: ${SW_STORAGE_ES_INDEX_SHARDS_NUMBER:2}

indexReplicasNumber: ${SW_STORAGE_ES_INDEX_REPLICAS_NUMBER:0}

# Those data TTL settings will override the same settings in core module.

recordDataTTL: ${SW_STORAGE_ES_RECORD_DATA_TTL:7} # Unit is day

otherMetricsDataTTL: ${SW_STORAGE_ES_OTHER_METRIC_DATA_TTL:45} # Unit is day

monthMetricsDataTTL: ${SW_STORAGE_ES_MONTH_METRIC_DATA_TTL:18} # Unit is month

# Batch process setting, refer to https://www.elastic.co/guide/en/elasticsearch/client/java-api/5.5/java-docs-bulk-processor.html

bulkActions: ${SW_STORAGE_ES_BULK_ACTIONS:1000} # Execute the bulk every 1000 requests

flushInterval: ${SW_STORAGE_ES_FLUSH_INTERVAL:10} # flush the bulk every 10 seconds whatever the number of requests

concurrentRequests: ${SW_STORAGE_ES_CONCURRENT_REQUESTS:2} # the number of concurrent requests

resultWindowMaxSize: ${SW_STORAGE_ES_QUERY_MAX_WINDOW_SIZE:10000}

metadataQueryMaxSize: ${SW_STORAGE_ES_QUERY_MAX_SIZE:5000}

segmentQueryMaxSize: ${SW_STORAGE_ES_QUERY_SEGMENT_SIZE:200}

profileTaskQueryMaxSize: ${SW_STORAGE_ES_QUERY_PROFILE_TASK_SIZE:200}

advanced: ${SW_STORAGE_ES_ADVANCED:""}

elasticsearch7:

nameSpace: ${SW_NAMESPACE:""}

clusterNodes: ${SW_STORAGE_ES_CLUSTER_NODES:localhost:9200}

protocol: ${SW_STORAGE_ES_HTTP_PROTOCOL:"http"}

trustStorePath: ${SW_SW_STORAGE_ES_SSL_JKS_PATH:"../es_keystore.jks"}

trustStorePass: ${SW_SW_STORAGE_ES_SSL_JKS_PASS:""}

enablePackedDownsampling: ${SW_STORAGE_ENABLE_PACKED_DOWNSAMPLING:true} # Hour and Day metrics will be merged into minute index.

dayStep: ${SW_STORAGE_DAY_STEP:1} # Represent the number of days in the one minute/hour/day index.

user: ${SW_ES_USER:""}

password: ${SW_ES_PASSWORD:""}

secretsManagementFile: ${SW_ES_SECRETS_MANAGEMENT_FILE:""} # Secrets management file in the properties format includes the username, password, which are managed by 3rd party tool.

indexShardsNumber: ${SW_STORAGE_ES_INDEX_SHARDS_NUMBER:2}

indexReplicasNumber: ${SW_STORAGE_ES_INDEX_REPLICAS_NUMBER:0}

# Those data TTL settings will override the same settings in core module.

recordDataTTL: ${SW_STORAGE_ES_RECORD_DATA_TTL:7} # Unit is day

otherMetricsDataTTL: ${SW_STORAGE_ES_OTHER_METRIC_DATA_TTL:45} # Unit is day

monthMetricsDataTTL: ${SW_STORAGE_ES_MONTH_METRIC_DATA_TTL:18} # Unit is month

# Batch process setting, refer to https://www.elastic.co/guide/en/elasticsearch/client/java-api/5.5/java-docs-bulk-processor.html

bulkActions: ${SW_STORAGE_ES_BULK_ACTIONS:1000} # Execute the bulk every 1000 requests

flushInterval: ${SW_STORAGE_ES_FLUSH_INTERVAL:10} # flush the bulk every 10 seconds whatever the number of requests

concurrentRequests: ${SW_STORAGE_ES_CONCURRENT_REQUESTS:2} # the number of concurrent requests

resultWindowMaxSize: ${SW_STORAGE_ES_QUERY_MAX_WINDOW_SIZE:10000}

metadataQueryMaxSize: ${SW_STORAGE_ES_QUERY_MAX_SIZE:5000}

segmentQueryMaxSize: ${SW_STORAGE_ES_QUERY_SEGMENT_SIZE:200}

profileTaskQueryMaxSize: ${SW_STORAGE_ES_QUERY_PROFILE_TASK_SIZE:200}

advanced: ${SW_STORAGE_ES_ADVANCED:""}

h2:

driver: ${SW_STORAGE_H2_DRIVER:org.h2.jdbcx.JdbcDataSource}

url: ${SW_STORAGE_H2_URL:jdbc:h2:mem:skywalking-oap-db}

user: ${SW_STORAGE_H2_USER:sa}

metadataQueryMaxSize: ${SW_STORAGE_H2_QUERY_MAX_SIZE:5000}

mysql:

properties:

jdbcUrl: ${SW_JDBC_URL:"jdbc:mysql://localhost:3306/swtest"}

dataSource.user: ${SW_DATA_SOURCE_USER:root}

dataSource.password: ${SW_DATA_SOURCE_PASSWORD:root@1234}

dataSource.cachePrepStmts: ${SW_DATA_SOURCE_CACHE_PREP_STMTS:true}

dataSource.prepStmtCacheSize: ${SW_DATA_SOURCE_PREP_STMT_CACHE_SQL_SIZE:250}

dataSource.prepStmtCacheSqlLimit: ${SW_DATA_SOURCE_PREP_STMT_CACHE_SQL_LIMIT:2048}

dataSource.useServerPrepStmts: ${SW_DATA_SOURCE_USE_SERVER_PREP_STMTS:true}

metadataQueryMaxSize: ${SW_STORAGE_MYSQL_QUERY_MAX_SIZE:5000}

influxdb:

# Metadata storage provider configuration

metabaseType: ${SW_STORAGE_METABASE_TYPE:H2} # There are 2 options as Metabase provider, H2 or MySQL.

h2Props:

dataSourceClassName: ${SW_STORAGE_METABASE_DRIVER:org.h2.jdbcx.JdbcDataSource}

dataSource.url: ${SW_STORAGE_METABASE_URL:jdbc:h2:mem:skywalking-oap-db}

dataSource.user: ${SW_STORAGE_METABASE_USER:sa}

dataSource.password: ${SW_STORAGE_METABASE_PASSWORD:}

mysqlProps:

jdbcUrl: ${SW_STORAGE_METABASE_URL:"jdbc:mysql://localhost:3306/swtest"}

dataSource.user: ${SW_STORAGE_METABASE_USER:root}

dataSource.password: ${SW_STORAGE_METABASE_PASSWORD:root@1234}

dataSource.cachePrepStmts: ${SW_STORAGE_METABASE_CACHE_PREP_STMTS:true}

dataSource.prepStmtCacheSize: ${SW_STORAGE_METABASE_PREP_STMT_CACHE_SQL_SIZE:250}

dataSource.prepStmtCacheSqlLimit: ${SW_STORAGE_METABASE_PREP_STMT_CACHE_SQL_LIMIT:2048}

dataSource.useServerPrepStmts: ${SW_STORAGE_METABASE_USE_SERVER_PREP_STMTS:true}

metadataQueryMaxSize: ${SW_STORAGE_METABASE_QUERY_MAX_SIZE:5000}

# InfluxDB configuration

url: ${SW_STORAGE_INFLUXDB_URL:http://localhost:8086}

user: ${SW_STORAGE_INFLUXDB_USER:root}

password: ${SW_STORAGE_INFLUXDB_PASSWORD:}

database: ${SW_STORAGE_INFLUXDB_DATABASE:skywalking}

actions: ${SW_STORAGE_INFLUXDB_ACTIONS:1000} # the number of actions to collect

duration: ${SW_STORAGE_INFLUXDB_DURATION:1000} # the time to wait at most (milliseconds)

fetchTaskLogMaxSize: ${SW_STORAGE_INFLUXDB_FETCH_TASK_LOG_MAX_SIZE:5000} # the max number of fetch task log in a request