日志级别改成调试
TL;DR: Collect debug information in an in-memory buffer that is printed to the output when an error occurs (and quietly discarded in all other cases).
TL; DR:在发生错误时将调试信息收集到内存中的缓冲区中,该缓冲区将打印到输出中(在所有其他情况下会被静默丢弃)。
I may have come up with a useful idea. Or something that is not all that useful. Or even something that already exists (and I am an ignorant fool for not knowing about it). If the latter — my apologies.
我可能想出了一个有用的主意。 或不是所有有用的东西。 甚至是已经存在的东西(我是一个无知的傻瓜)。 如果是后者,我表示歉意。
We had a production issue this morning. Actually, it started yesterday afternoon and we only learned about it this morning. Not good. And while we good resolve the issue relatively easily — a developer remembered the change introduced into the runtime platform that was the likely and as it turned out the actual source of the the problem, we were not happy about the information available to us to perform the analysis. Our system produces some logging at runtime in the production environment, but clearly not an enormously lot of detail. In fact, as the volume handled by the system has increased over the past weeks, we have been increasing the log level (from INFO to WARN) to stem the wave of logging. The log files gave us but the barest of clues.
今天早上我们遇到了生产问题。 实际上,它是从昨天下午开始的,而我们直到今天早上才知道。 不好。 而且,尽管我们可以相对轻松地解决问题-开发人员想起了可能在运行时平台中引入的更改,但事实证明是问题的真正根源,但我们对提供给我们的执行信息并不满意。分析。 我们的系统在生产环境中的运行时生成一些日志记录,但是显然没有太多细节。 实际上,随着过去几周系统处理量的增加,我们一直在提高日志级别(从INFO到WARN)以阻止日志记录浪潮。 日志文件给了我们一些线索。
Ironically, we have just had a team discussion about logging. To define the guidelines that our logging should abide by. What labelling system for our log messages will we use — to identify the source and meaning of the messages, even in an automated way. And to bring consistency in the log message output format to make automated collection and analysis of the log messages easier with the Elastic Stack, Splunk, Azure Application Monitoring and Oracle Cloud Logging.
具有讽刺意味的是,我们刚刚进行了有关日志记录的小组讨论。 定义我们的日志记录应遵守的准则。 我们将使用哪种日志消息标记系统-甚至以自动化方式识别消息的来源和含义。 并通过使用弹性堆栈,Splunk,Azure应用程序监视和Oracle云日志记录,使日志消息输出格式具有一致性,从而使日志消息的自动收集和分析更加容易。
We had a serious debate about the log levels we wanted to group our logging into and the necessity to be able at run time to manipulate the log level threshold to for example start analysing in detail in the production environment for specific components by lowering the logging threshold to INFO for example.
我们对要分组登录的日志级别进行了认真的辩论,还涉及在运行时能够操纵日志级别阈值的必要性,例如,通过降低日志记录阈值,开始在生产环境中针对特定组件进行详细分析以INFO为例。
However, if the production log level threshold is WARN — then when the incident occurs, there will not be any INFO or DEBUG logging written in the log files. Changing the logging threshold after the incident is ‘de put dempen als het kalf al verdronken is’ if you will pardon me for this Dutch expression. It is too late.
但是,如果生产日志级别阈值为WARN-则在发生事件时,日志文件中将不会写入任何INFO或DEBUG日志。 如果您愿意以这种荷兰语表达原谅我,请在事件发生后更改日志记录阈值为“ de dempen als het kalf al verdronken”。 为时已晚。
Ideally, so goes my thinking, you can get the logging at debug level from everything immediately preceding a problematic exception in our code. We do not have to set the logging threshold on DEBUG and wade through mountains of irrelevant details but when the *** has hit the fan, the little pile of debug level logging is written to the output — as if in the period just prior to the issue someone with insider knowledge had turned on debug level logging.
理想情况下,我的想法是这样,您可以从代码中有问题的异常之前的所有内容获取调试级别的日志记录。 我们不必在DEBUG上设置日志记录阈值,也不必花大量时间查看不相关的细节,但是当***遇到麻烦时,便会在输出中写入一小堆调试级别的日志记录-就像在之前有内部专家知识的问题已打开调试级别日志记录。
Sounds nice, does it not? But how to implement this?
听起来不错,不是吗? 但是如何实现呢?
In my current project, our components are Serverless Functions — little Java and Node applications. It really would not be hard at all to add the following mechanism in these applications:
在我当前的项目中,我们的组件是无服务器功能-小Java和Node应用程序。 在这些应用程序中添加以下机制确实一点也不困难:
- when entering a new context — method/function, iteration in a loop — create a debug log message buffer (an array) 输入新的上下文(方法/函数,循环迭代)时,创建调试日志消息缓冲区(数组)
- push messages to the buffer that would be relevant in case of an error 将消息推送到缓冲区,以防发生错误
- in case of a run time exception (at least the ones that are unexpected): 如果发生运行时异常(至少是意外异常):
- write the contents of the buffer to the logging output — regardless of the current log level threshold, at ERROR level 将缓冲区的内容写入日志输出—无论当前日志级别阈值如何,均处于ERROR级别
- handle the exception 处理异常
- if no exception occurred, the local buffer is removed as soon as the current context is returned from 如果没有异常发生,则从当前上下文返回后立即删除本地缓冲区
Note that we could pass the debug buffer to called functions and use the debug buffer object in way that is very similar to the call stack. Instead of the names of the methods invoked throughout the call history, it contains the log messages. And with each successful closing of a branch in the program execution, all logging associated with that branch can be discarded. And at each level in the program execution, all previous debug logging from higher up levels is available. This probably makes sense if this debug logging needs to be retained across asynchronous steps.
注意,我们可以将调试缓冲区传递给被调用的函数,并以与调用堆栈非常相似的方式使用调试缓冲区对象。 它包含日志消息,而不是整个调用历史记录中调用的方法的名称。 随着程序执行中分支的每次成功关闭,与该分支关联的所有日志记录都将被丢弃。 并且在程序执行的每个级别,都可以使用更高级别的所有以前的调试日志记录。 如果此调试日志记录需要跨异步步骤保留,那么这可能很有意义。
In Node (JS) because of the asynchronous and single thread nature of processing, it may be necessary to define the debug buffer object in the Async Local Storage object that was just introduced in Node v13/14/v12.7. Using lower level async-hooks APIs, you can implement this in earlier versions as well. In Java, when multiple threads are running in parallel, Thread local storage may be the best place for the semi-global debug buffer object.
在Node(JS)中,由于处理的异步和单线程性质,可能有必要在Node v13 / 14 / v12.7中刚刚引入的Async Local Storage对象中定义调试缓冲区对象。 使用较低级别的异步钩子API,您也可以在较早版本中实现此功能。 在Java中,当多个线程并行运行时,线程本地存储可能是半全局调试缓冲区对象的最佳位置。
A simple example in Node (warning: for same of simplicity, I have not yet used Async Local Storage which means that this example does not work for processing of multiple requests at the same time). In this example, functions calculator and calculateSums prepare their local log buffer, write messages to it and produce their contents in case of an exception. Here is the code that processes a series of calculations by dynamically evaluating strings:
Node中的一个简单示例( 警告:为简单起见,我尚未使用异步本地存储,这意味着该示例不适用于同时处理多个请求)。 在此示例中,函数计算器和calculateSums准备其本地日志缓冲区,向其中写入消息并在发生异常的情况下产生其内容。 这是通过动态评估字符串来处理一系列计算的代码:
And here is the outcome triggered by the clearly incorrect sum “hgah”:
这是明显不正确的总和“ hgah”触发的结果:
![Image for post](https://i-blog.csdnimg.cn/blog_migrate/ea3a86450e443df9f1127482676ee245.png)
Note that the first call to function calculateSums does not show up in the logging — because no INFO level logs were produced from that function. However, because in the second call to the function, an exception occurred, we get all the debug level output from the function, also from before the actual moment of the exception. I am surprised by the way that 2/0 did not result in an exception.
注意,对函数calculateSums的第一次调用不会出现在日志记录中,因为该函数未生成任何INFO级别的日志。 但是,因为在第二次调用该函数时发生了异常,所以我们也从该异常的实际时刻之前获得了该函数的所有调试级别输出。 我对2/0没有导致异常的方式感到惊讶。
This demonstrates my point: under normal circumstances, no logging would be produced by this little application — apart from the console.info reports with the results (if the logging level allows for INFO). When an exception occurs, then the debug from the current context and the calling context is available, for instant analysis.
这表明了我的观点:在正常情况下,除了console.info报告结果(如果日志记录级别允许使用INFO)之外,这个小应用程序不会生成任何日志记录。 当发生异常时,可以从当前上下文和调用上下文进行调试,以进行即时分析。
Does this make sense? How could this best be applied?
这有意义吗? 如何最好地应用?
Another implementation: send logging to a more central object (an append and pop store) with a “savepoint” identifier. After successfully completing a function, release the logging by removing (everything logged after) the savepoint. In case of problems, print all logging for the current context’s savepoint. In code, that would look like:
另一个实现:将日志记录发送到带有“保存点”标识符的更中心的对象(追加和弹出存储)。 成功完成功能后,通过删除保存点(此后记录的所有内容)来释放日志记录。 如果出现问题,请为当前上下文的保存点打印所有日志记录。 在代码中,它看起来像:
资源资源 (Resources)
Exploring Async Local Storage in Node v14 — https://blog.kuzzle.io/nodejs-14-asynclocalstorage-asynchronous-calls
探索节点v14中的异步本地存储-https: //blog.kuzzle.io/nodejs-14-asynclocalstorage-asynchronous-calls
Node Documentarion on AsyncLocalStorage — https://nodejs.org/api/async_hooks.html#async_hooks_class_asynclocalstorage
AsyncLocalStorage上的Node Documentarion — https://nodejs.org/api/async_hooks.html#async_hooks_class_asynclocalstorage
Medium Article on Thread Local (emulation) in NodeJS — https://medium.com/@tabu_craig/nodejs-and-thread-local-storage-eb2c1a24881
关于NodeJS中的线程本地(仿真)的中型文章— https://medium.com/@tabu_craig/nodejs-and-thread-local-storage-eb2c1a24881
日志级别改成调试