riptrace调试微服务和容器应用程序

介绍 (Introduction)

Debugging microservices is an inherently complex topic due to the interactions between multiple disparate applications. Yes, decoupling should always be kept at the forefront when developing microservices, but more often than not, a highly decoupled system is an unattainable utopian dream. The microservices will undoubtedly rely on each other, and when one fails, it can have a cascading effect on the others. In addition to this, coming up with isolated unit tests will often only account for happy path scenarios, without taking into consideration the chaotic path that orchestrating multiple microservices can bring about. What if there was a way to introduce distributed breakpoint-style debugging across multiple microservices at the same time, with the capability of tracking local variables, timing, and thread execution? Enter RipTrace.

由于多个不同应用程序之间的交互，调试微服务是一个固有的复杂主题。 是的，在开发微服务时，应该始终将去耦放在首位，但是通常，高度去耦的系统是无法实现的乌托邦梦想。 微服务无疑会相互依赖，当一个微服务失败时，它可能对其他微服务产生级联效应。 除此之外，提出隔离的单元测试通常只会考虑到令人满意的路径方案，而没有考虑协调多个微服务可能带来的混乱路径。 如果有一种方法可以同时在多个微服务之间引入分布式断点式调试，并且具有跟踪局部变量，定时和线程执行的功能，那该怎么办？ 输入RipTrace。

专注于围棋 (A Focus on Go)

Due to the prevalence of its use in distributed systems (Kubernetes, Docker, Moby, Etcd) and ease of deployment (single compiled binary) for microservices, Go was chosen as the target for RipTrace. In addition to the above, the magnificent Delve debuger (https://github.com/go-delve/delve) is a perfect resource to tap for building a distributed breakpoint debugging system. RipTrace combines Go’s concurrency patterns, the robustness of the Delve debugger, and the NATS messaging system (https://nats.io/) into a distributed debugging system with the goal of making Go-based microservices easier to troubleshoot.

由于它在分布式系统(Kubernetes，Docker，Moby，Etcd)中的普遍使用，以及微服务的易于部署(单编译二进制)，因此Go被选为RipTrace的目标。 除上述内容外，宏伟的Delve调试器( https://github.com/go-delve/delve )是开发分布式断点调试系统的理想资源。 RipTrace将Go的并发模式，Delve调试器的健壮性和NATS消息传递系统( https://nats.io/ )组合到一个分布式调试系统中，旨在使基于Go的微服务更易于故障排除。

RipTrace体系结构 (RipTrace Architecture)

Below is a diagram that presents the overall proposed architecture of RipTrace.

下图显示了RipTrace总体建议的体系结构。

On the left side of the diagram is the RipTrace agent. The RipTrace agent leverages Delve and its capability of attaching to processes using ptrace to debug existing Golang applications. It is important to note that the container must have PTRACE capabilities, therefore this should only be used in environments where the containers can be trusted.

图的左侧是RipTrace代理。 RipTrace代理利用Delve及其使用ptrace附加到进程的功能来调试现有的Golang应用程序。 重要的是要注意容器必须具有PTRACE功能，因此仅应在可以信任容器的环境中使用。

All commands on the agent are executed using NATS topics. By using NATS topics, this means the RipTrace agent only needs to be able to speak out to the NATS server, and the server does not need to have access into the RipTrace agent. Theoretically speaking, this means that the NATS server and RipTrace server can exist outside of the network(s) that the agents are operating in. Below are an example of some of the request/response and topic patterns that are seen within RipTrace:

使用NATS主题执行代理上的所有命令。 通过使用NATS主题，这意味着RipTrace代理仅需要能够对NATS服务器讲话，并且该服务器不需要访问RipTrace代理。 从理论上讲，这意味着NATS服务器和RipTrace服务器可以存在于代理运行所在的网络之外。以下是一些在RipTrace中看到的请求/响应和主题模式的示例：

• <host>.profile.get: Allows the RipTrace server to request an agent to return the host’s profile, or all currently executing Go processes that can be attached to, as well as some various host information about the system.

  <host> .profile.get：允许RipTrace服务器请求代理返回主机的配置文件，或可以附加到当前所有正在执行的Go进程，以及有关系统的一些主机信息。

• <host>.debugger.attach: The server will provide the agent with a process ID (acquired during the profile.get request) and the agent will attempt to attach a debugger to the provided process. Once a debugger is attached, this will create a “TRACE_PUBLISHER” that will publish all output from the debugger as tracepoints are hit.

  <host> .debugger.attach：服务器将为代理提供进程ID(在profile.get请求期间获取)，并且代理将尝试将调试器附加到所提供的进程。 连接调试器后，将创建一个“ TRACE_PUBLISHER”，当命中跟踪点时，它将发布调试器的所有输出。

• <host>.debugger.createTracepoint: A message on this topic will notify the agent to inject a tracepoint at the specified location, identified by filename and line just like a typical debugger. Each time a tracepoint is hit, it will emit messages on the TRACE_PUBLISHER that can then be picked up by the RipTrace server.

  <host> .debugger.createTracepoint：关于此主题的消息将通知代理在指定位置注入跟踪点，该跟踪点由文件名和行标识，就像典型的调试器一样。 每次命中一个跟踪点时，它将在TRACE_PUBLISHER上发出消息，然后可由RipTrace服务器拾取。

• <host>.debugger.trace: This topic is where messages emitted by the TRACE_PUBLISHER can be found.

  <host> .debugger.trace：在本主题中可以找到TRACE_PUBLISHER发出的消息。

By using a messaging system like NATS, it is very easy to scale the system up to have many agents, and be able to coordinate and orchestrate many debugging instances at the same time. In addition to this, each trace message is tagged with a coordinated timestamp (provided by the server) so that strict ordering is guaranteed. How is this powerful? For example, say that tracepoints are injected into multiple microservices and a request is run. It is then possible to see the order in which lines were hit, the mutation of local variables along the way, and if any error handling lines were hit. This essentially provides a unified breakpoint debugger across multiple systems that are running at the same time.

通过使用像NATS这样的消息传递系统，可以很容易地将系统扩展到具有许多代理的位置，并且能够同时协调和协调许多调试实例。 除此之外，每条跟踪消息都用协调的时间戳(由服务器提供)标记，以确保严格的排序。 这个有多强大？ 例如，假设将跟踪点注入到多个微服务中并运行了一个请求。 这样就可以看到行的顺序，沿途的局部变量的突变以及是否有错误处理行。 这实质上是跨多个同时运行的系统提供统一的断点调试器。

RipTrace服务器 (RipTrace Server)

The RipTrace server coordinates and orchestrates the “fleet” of active debugging agents by injecting tracepoint, attach, and profile request messages into the NATS topics, while also attaching to the TRACE_PUBLISHER topics to monitor for trace messages being emitted by the debuggers. In addition this, the RipTrace server will also provide a frontend interface very similar to an IDE where lines can be clicked on to inject the breakpoints. The server will synchronize with Git repositories to reflect the code currently being executed in the microservices. This step of source code synchronization is very important to ensure accurate breakpoints and filenames.

RipTrace服务器通过将跟踪点，附加和概要文件请求消息注入到NATS主题中来协调和协调活动的调试代理的“队列”，同时还附加到TRACE_PUBLISHER主题以监视调试器发出的跟踪消息。 除此之外，RipTrace服务器还将提供一个非常类似于IDE的前端接口，在该接口上，可以单击行以插入断点。 服务器将与Git存储库同步，以反映微服务中当前正在执行的代码。 源代码同步的这一步骤对于确保准确的断点和文件名非常重要。

现状和目标 (Current Status and Goals)

RipTrace is at the very early stages of design and development. Leveraging Delve’s internal resources was highly experimental, but is currently in a working state. Currently, the agent will respond to the above topics and trace the attached process correctly, without affecting the runtime of the attached process.

RipTrace处于设计和开发的早期阶段。 利用Delve的内部资源是高度实验性的，但是目前处于工作状态。 当前，代理将响应以上主题并正确跟踪附加过程，而不会影响附加过程的运行时间。

Long term, the goals of the RipTrace project are as follows:

从长远来看，RipTrace项目的目标如下：

• Ability to coordinate n microservices and agents to debug large distributed systems

  能够协调n个微服务和代理来调试大型分布式系统

• A method to inject reproducible tests (in the form of configuration files) that can be run on a set of services at the same time
  一种注入可重现的测试(以配置文件的形式)的方法，该测试可以同时在一组服务上运行
• Provide an easy to use frontend that presents an easy-to-use and familiar IDE-like experience that is seamless with other developer tools
  提供易于使用的前端，呈现与其他开发人员工具无缝集成的易于使用和熟悉的类似IDE的体验

意见和反馈 (Feedback and Ideas)

The project is located here https://github.com/csthompson/riptrace, but keep in mind the project is in its VERY VERY early stages. I just want to start getting feedback from other developers to see if the idea is viable or worthwhile. Feel free to drop a comment below!

该项目位于https://github.com/csthompson/riptrace ，但请记住，该项目处于非常非常早期的阶段。 我只是想开始从其他开发人员那里获得反馈，以了解该想法是否可行或值得。 随时在下面发表评论！

翻译自: https://medium.com/swlh/riptrace-debugging-microservice-and-container-applications-fea1644f0dda