fastapi集成elastic-apm,实现性能监控
本文适用于 Starlette/FastAPI
软件版本背景:
elastic 8.9
apm 6.x
官方原文如下:
Starlette/FastAPI Support
Incorporating Elastic APM into your Starlette project only requires a few easy steps.
Installation
Install the Elastic APM agent using pip:
$ pip install elastic-apm
or add elastic-apm
to your project’s requirements.txt
file.
Setup
To set up the agent, you need to initialize it with appropriate settings.
The settings are configured either via environment variables, or as initialization arguments.
You can find a list of all available settings in the Configuration page.
To initialize the agent for your application using environment variables, add the ElasticAPM middleware to your Starlette application:
from starlette.applications import Starlette
from elasticapm.contrib.starlette import ElasticAPM
app = Starlette()
app.add_middleware(ElasticAPM)
Warning | If you are using any BaseHTTPMiddleware middleware, you must add them before the ElasticAPM middleware. This is because BaseHTTPMiddleware breaks contextvar propagation, as noted here. |
---|
To configure the agent using initialization arguments:
from starlette.applications import Starlette
from elasticapm.contrib.starlette import make_apm_client, ElasticAPM
apm = make_apm_client({
'SERVICE_NAME': '<SERVICE-NAME>',
'SECRET_TOKEN': '<SECRET-TOKEN>',
})
app = Starlette()
app.add_middleware(ElasticAPM, client=apm)
FastAPI
Because FastAPI supports Starlette middleware, using the agent with FastAPI is almost exactly the same as with Starlette:
from fastapi import FastAPI
from elasticapm.contrib.starlette ElasticAPM
app = FastAPI()
app.add_middleware(ElasticAPM)
Usage
Once you have configured the agent, it will automatically track transactions and capture uncaught exceptions within starlette.
Capture an arbitrary exception by calling capture_exception
:
try:
1 / 0
except ZeroDivisionError:
apm.client.capture_exception()
Log a generic message with capture_message
:
apm.client.capture_message('hello, world!')
Performance metrics
If you’ve followed the instructions above, the agent has installed our instrumentation middleware which will process all requests through your app. This will measure response times, as well as detailed performance data for all supported technologies.
Note | Due to the fact that asyncio drivers are usually separate from their synchronous counterparts, specific instrumentation is needed for all drivers. The support for asynchronous drivers is currently quite limited. |
---|
Ignoring specific routes
You can use the TRANSACTIONS_IGNORE_PATTERNS
configuration option to ignore specific routes. The list given should be a list of regular expressions which are matched against the transaction name:
apm = make_apm_client({
# ...
'TRANSACTIONS_IGNORE_PATTERNS': ['^GET /secret', '/extra_secret']
# ...
})
This would ignore any requests using the GET /secret
route and any requests containing /extra_secret
.
Supported Starlette and Python versions
A list of supported Starlette and Python versions can be found on our Supported Technologies page.
Note | Elastic APM only supports asyncio when using Python 3.7+ |
---|
具体使用示例
在加入了以上的代码配置后,按照正常启动应用的步骤就可以在kibana的apm界面看到对应的服务监控了。
关于配置文件,可以参考 elastic-apm包下的文件
venv/lib/python3.7/site-packages/elasticapm/conf/init.py 中的 class Config,其中全部大写的这些字段就是配置项
class Config(_ConfigBase):
service_name = _ConfigValue(
"SERVICE_NAME",
validators=[RegexValidator("^[a-zA-Z0-9 _-]+$")],
default="unknown-python-service",
required=True,
)
service_node_name = _ConfigValue("SERVICE_NODE_NAME")
environment = _ConfigValue("ENVIRONMENT")
transport_json_serializer = _ConfigValue("TRANSPORT_JSON_SERIALIZER")
secret_token = _ConfigValue("SECRET_TOKEN")
api_key = _ConfigValue("API_KEY")
debug = _BoolConfigValue("DEBUG", default=False)
server_url = _ConfigValue("SERVER_URL", default="http://127.0.0.1:8200", required=True)
server_cert = _ConfigValue("SERVER_CERT", validators=[FileIsReadableValidator()])
server_ca_cert_file = _ConfigValue("SERVER_CA_CERT_FILE", validators=[FileIsReadableValidator()])
verify_server_cert = _BoolConfigValue("VERIFY_SERVER_CERT", default=True)
use_certifi = _BoolConfigValue("USE_CERTIFI", default=True)
include_paths = _ListConfigValue("INCLUDE_PATHS")
exclude_paths = _ListConfigValue("EXCLUDE_PATHS", default=compat.get_default_library_patters())
filter_exception_types = _ListConfigValue("FILTER_EXCEPTION_TYPES")
server_timeout = _ConfigValue(
"SERVER_TIMEOUT",
type=float,
validators=[
UnitValidator(r"^((?:-)?\d+)(ms|s|m)?$", r"\d+(ms|s|m)", {"ms": 0.001, "s": 1, "m": 60, None: 1000})
],
default=5,
)
hostname = _ConfigValue("HOSTNAME", default=socket.gethostname())
auto_log_stacks = _BoolConfigValue("AUTO_LOG_STACKS", default=True)
transport_class = _ConfigValue("TRANSPORT_CLASS", default="elasticapm.transport.http.Transport", required=True)
processors = _ListConfigValue(
"PROCESSORS",
default=[
"elasticapm.processors.sanitize_stacktrace_locals",
"elasticapm.processors.sanitize_http_request_cookies",
"elasticapm.processors.sanitize_http_response_cookies",
"elasticapm.processors.sanitize_http_headers",
"elasticapm.processors.sanitize_http_wsgi_env",
"elasticapm.processors.sanitize_http_request_body",
],
)
sanitize_field_names = _ListConfigValue(
"SANITIZE_FIELD_NAMES", type=starmatch_to_regex, default=BASE_SANITIZE_FIELD_NAMES
)
metrics_sets = _ListConfigValue(
"METRICS_SETS",
default=[
"elasticapm.metrics.sets.cpu.CPUMetricSet",
],
)
metrics_interval = _DurationConfigValue(
"METRICS_INTERVAL",
validators=[ExcludeRangeValidator(0.001, 0.999, "{range_start} - {range_end} s")],
default=timedelta(seconds=30),
)
breakdown_metrics = _BoolConfigValue("BREAKDOWN_METRICS", default=True)
prometheus_metrics = _class Config(_ConfigBase):
service_name = _ConfigValue(
"SERVICE_NAME",
validators=[RegexValidator("^[a-zA-Z0-9 _-]+$")],
default="unknown-python-service",
required=True,
)
service_node_name = _ConfigValue("SERVICE_NODE_NAME")
environment = _ConfigValue("ENVIRONMENT")
transport_json_serializer = _ConfigValue("TRANSPORT_JSON_SERIALIZER")
secret_token = _ConfigValue("SECRET_TOKEN")
api_key = _ConfigValue("API_KEY")
debug = _BoolConfigValue("DEBUG", default=False)
server_url = _ConfigValue("SERVER_URL", default="http://127.0.0.1:8200", required=True)
server_cert = _ConfigValue("SERVER_CERT", validators=[FileIsReadableValidator()])
server_ca_cert_file = _ConfigValue("SERVER_CA_CERT_FILE", validators=[FileIsReadableValidator()])
verify_server_cert = _BoolConfigValue("VERIFY_SERVER_CERT", default=True)
use_certifi = _BoolConfigValue("USE_CERTIFI", default=True)
include_paths = _ListConfigValue("INCLUDE_PATHS")
exclude_paths = _ListConfigValue("EXCLUDE_PATHS", default=compat.get_default_library_patters())
filter_exception_types = _ListConfigValue("FILTER_EXCEPTION_TYPES")
server_timeout = _ConfigValue(
"SERVER_TIMEOUT",
type=float,
validators=[
UnitValidator(r"^((?:-)?\d+)(ms|s|m)?$", r"\d+(ms|s|m)", {"ms": 0.001, "s": 1, "m": 60, None: 1000})
],
default=5,
)
hostname = _ConfigValue("HOSTNAME", default=socket.gethostname())
auto_log_stacks = _BoolConfigValue("AUTO_LOG_STACKS", default=True)
transport_class = _ConfigValue("TRANSPORT_CLASS", default="elasticapm.transport.http.Transport", required=True)
processors = _ListConfigValue(
"PROCESSORS",
default=[
"elasticapm.processors.sanitize_stacktrace_locals",
"elasticapm.processors.sanitize_http_request_cookies",
"elasticapm.processors.sanitize_http_response_cookies",
"elasticapm.processors.sanitize_http_headers",
"elasticapm.processooolConfigValue("PROMETHEUS_METRICS", default=False)
prometheus_metrics_prefix = _ConfigValue("PROMETHEUS_METRICS_PREFIX", default="prometheus.metrics.")
disable_metrics = _ListConfigValue("DISABLE_METRICS", type=starmatch_to_regex, default=[])
central_config = _BoolConfigValue("CENTRAL_CONFIG", default=True)
api_request_size = _ConfigValue("API_REQUEST_SIZE", type=int, validators=[size_validator], default=768 * 1024)
api_request_time = _DurationConfigValue("API_REQUEST_TIME", default=timedelta(seconds=10))
transaction_sample_rate = _ConfigValue(
"TRANSACTION_SAMPLE_RATE", type=float, validators=[PrecisionValidator(4, 0.0001)], default=1.0
)
transaction_max_spans = _ConfigValue("TRANSACTION_MAX_SPANS", type=int, default=500)
stack_trace_limit = _ConfigValue("STACK_TRACE_LIMIT", type=int, default=50)
span_frames_min_duration = _DurationConfigValue(
"SPAN_FRAMES_MIN_DURATION", default=timedelta(seconds=0.005), unitless_factor=0.001
)
span_stack_trace_min_duration = _DurationConfigValue(
"SPAN_STACK_TRACE_MIN_DURATION", default=timedelta(seconds=0.005), unitless_factor=0.001
)
span_compression_enabled = _BoolConfigValue("SPAN_COMPRESSION_ENABLED", default=True)
span_compression_exact_match_max_duration = _DurationConfigValue(
"SPAN_COMPRESSION_EXACT_MATCH_MAX_DURATION",
default=timedelta(seconds=0.05),
)
span_compression_same_kind_max_duration = _DurationConfigValue(
"SPAN_COMPRESSION_SAME_KIND_MAX_DURATION",
default=timedelta(seconds=0),
)
exit_span_min_duration = _DurationConfigValue(
"EXIT_SPAN_MIN_DURATION",
default=timedelta(seconds=0),
)
collect_local_variables = _ConfigValue("COLLECT_LOCAL_VARIABLES", default="errors")
source_lines_error_app_frames = _ConfigValue("SOURCE_LINES_ERROR_APP_FRAMES", type=int, default=5)
source_lines_error_library_frames = _ConfigValue("SOURCE_LINES_ERROR_LIBRARY_FRAMES", type=int, default=5)
source_lines_span_app_frames = _ConfigValue("SOURCE_LINES_SPAN_APP_FRAMES", type=int, default=0)
source_lines_span_library_frames = _ConfigValue("SOURCE_LINES_SPAN_LIBRARY_FRAMES", type=int, default=0)
local_var_max_length = _ConfigValue("LOCAL_VAR_MAX_LENGTH", type=int, default=200)
local_var_list_max_length = _ConfigValue("LOCAL_VAR_LIST_MAX_LENGTH", type=int, default=10)
local_var_dict_max_length = _ConfigValue("LOCAL_VAR_DICT_MAX_LENGTH", type=int, default=10)
capture_body = _ConfigValue(
"CAPTURE_BODY",
default="off",
validators=[lambda val, _: {"errors": "error", "transactions": "transaction"}.get(val, val)],
)
async_mode = _BoolConfigValue("ASYNC_MODE", default=True)
instrument_django_middleware = _BoolConfigValue("INSTRUMENT_DJANGO_MIDDLEWARE", default=True)
autoinsert_django_middleware = _BoolConfigValue("AUTOINSERT_DJANGO_MIDDLEWARE", default=True)
transactions_ignore_patterns = _ListConfigValue("TRANSACTIONS_IGNORE_PATTERNS", default=[])
transaction_ignore_urls = _ListConfigValue("TRANSACTION_IGNORE_URLS", type=starmatch_to_regex, default=[])
ignore_message_queues = _ListConfigValue("IGNORE_MESSAGE_QUEUES", type=starmatch_to_regex, default=[])
service_version = _ConfigValue("SERVICE_VERSION")
framework_name = _ConfigValue("FRAMEWORK_NAME")
framework_version = _ConfigValue("FRAMEWORK_VERSION")
global_labels = _DictConfigValue("GLOBAL_LABELS")
disable_send = _BoolConfigValue("DISABLE_SEND", default=False)
enabled = _BoolConfigValue("ENABLED", default=True)
recording = _BoolConfigValue("RECORDING", default=True)
instrument = _BoolConfigValue("INSTRUMENT", default=True)
enable_distributed_tracing = _BoolConfigValue("ENABLE_DISTRIBUTED_TRACING", default=True)
capture_headers = _BoolConfigValue("CAPTURE_HEADERS", default=True)
django_transaction_name_from_route = _BoolConfigValue("DJANGO_TRANSACTION_NAME_FROM_ROUTE", default=False)
disable_log_record_factory = _BoolConfigValue("DISABLE_LOG_RECORD_FACTORY", default=False)
use_elastic_traceparent_header = _BoolConfigValue("USE_ELASTIC_TRACEPARENT_HEADER", default=True)
use_elastic_excepthook = _BoolConfigValue("USE_ELASTIC_EXCEPTHOOK", default=False)
cloud_provider = _ConfigValue("CLOUD_PROVIDER", default=True)
log_level = _ConfigValue(
"LOG_LEVEL",
validators=[EnumerationValidator(["trace", "debug", "info", "warning", "warn", "error", "critical", "off"])],
callbacks=[_log_level_callback],
)
log_file = _ConfigValue("LOG_FILE", default="")
log_file_size = _ConfigValue("LOG_FILE_SIZE", validators=[size_validator], type=int, default=50 * 1024 * 1024)
log_ecs_reformatting = _ConfigValue(
"LOG_ECS_REFORMATTING",
validators=[EnumerationValidator(["off", "override"])],
callbacks=[_log_ecs_reformatting_callback],
default="off",
)
trace_continuation_strategy = _ConfigValue(
"TRACE_CONTINUATION_STRATEGY",
validators=[
EnumerationValidator(
[
TRACE_CONTINUATION_STRATEGY.CONTINUE,
TRACE_CONTINUATION_STRATEGY.RESTART,
TRACE_CONTINUATION_STRATEGY.RESTART_EXTERNAL,
]
)
],
default=TRACE_CONTINUATION_STRATEGY.CONTINUE,
)
include_process_args = _BoolConfigValue("INCLUDE_PROCESS_ARGS", default=False)
官网的配置项介绍
https://www.elastic.co/guide/en/apm/agent/python/6.x/configuration.html
各语言框架集成elastic-apm官网介绍
https://www.elastic.co/guide/en/apm/guide/current/_step_3_install_apm_agents.html