nameko微服务_Nameko用于微服务

nameko微服务

In December some of the tech guys at onefinestay invited me over to London to do some general improvements on their nameko library. This collaboration came together because nameko was pretty similar to how I generally like to build certain infrastructure and I had some experience with very similar systems.

在12月份某一在高科技家伙onefinestay邀请我到伦敦做他们的一些普遍改善菇库。 之所以合作，是因为nameko与我通常喜欢构建某些基础结构的方式非常相似，并且我在非常相似的系统上也有一些经验。

So now that some of those improvements hit the release version of nameko I figured it might be a good idea to give some feedback on why I like this sort of architecture.

因此，既然这些改进中的一些达到了nameko的发行版本，我认为对我为什么喜欢这种架构提供一些反馈可能是个好主意。

解放思想 (Freeing your Mind)

Right now if you want to build a web service in Python there are many tools you can pick from, but most of them live in a very specific part of your stack. The most common tool is a web framework and that will typically provide you with whatever glue is necessary to connect your own code to an incoming HTTP request that comes from your client.

现在，如果要使用Python构建Web服务，可以选择许多工具，但是大多数工具都位于堆栈的特定部分。 最常见的工具是一个Web框架，它通常会为您提供将您自己的代码连接到来自客户端的传入HTTP请求所必需的任何粘合剂。

However that’s not all you need in an application. For instance very often you have periodic tasks that you need to execute and in that case, your framework is often not just not helping, it’s also in your way. For instance because you might have built your code with the assumption that it has access to the HTTP request object. If you now want to run it from a cronjob that request object is unavailable.

但是，这不是您在应用程序中所需的全部。 例如，您经常有需要执行的定期任务，在这种情况下，您的框架通常不仅无济于事，而且也很麻烦。 例如，因为您可能在假设代码可以访问HTTP请求对象的情况下构建了代码。 如果现在要从cronjob运行它，则请求对象不可用。

In addition to crons there is often also the wish to execute something as the result of the request of a client, but without blocking that request. For instance imagine there is an admin panel in which you can trigger some very expensive data conversion task. What you actually want is for the current request to finish but the conversion task to keep on working in the background until your data set is converted.

除客户端之外，通常还希望根据客户端的请求执行某些操作，但又不阻止该请求。 例如，假设有一个管理面板，您可以在其中触发一些非常昂贵的数据转换任务。 您真正想要的是完成当前请求，但转换任务将继续在后台运行，直到转换数据集为止。

There are obviously many existing solutions for that. Celery comes to mind. However they are typically very separated from the rest of the stack.

显然有许多现有的解决方案。 芹菜浮现在脑海。 但是，它们通常与堆栈的其余部分非常分开。

Having a system which treats all of this processes the same frees up your mind. This is what makes microservices interesting. Away with having HTTP request handlers that have no direct relationship with message queue worker tasks or cronjobs. Instead you can have a coherent system where any component can talk through well defined points with other parts of the system.

拥有一个将所有这些流程都一视同仁的系统，可以解放您的思想。 这就是使微服务变得有趣的原因。 远离具有与消息队列工作器任务或cronjobs没有直接关系的HTTP请求处理程序。 相反，您可以拥有一个一致的系统，其中任何组件都可以通过定义明确的点与系统的其他部分进行通信。

This is especially useful in Python where traditionally our support for parallel execution has been between very bad to abysmal.

这在Python中特别有用，在Python中，传统上我们对并行执行的支持介于非常差到非常糟糕之间。

输入Nameko (Enter Nameko)

Nameko is an implementation of this idea. It’s very similar in architecture to how we structure code at Fireteam. It’s based on distributing work between processes through AMQP. It’s not just AMQP though. Nameko abstracts away from that and allows you to write your own transports, while staying true to the AMQP patterns.

Nameko是此想法的实现。 这在架构上与我们在Fireteam中构造代码的方式非常相似。 它基于通过AMQP在流程之间分配工作。 虽然不只是AMQP。 Nameko对此进行了抽象，使您可以编写自己的传输方式，同时忠于AMQP模式。

Nameko does a handful of things and you can build very complex systems with it. The idea is that you build individual services which can emit events to which other services can subscribe to or they can directly invoke each other via RPC. All communication between the services happens through AMQP. You don’t need to manually deal with any connectivity of those.

Nameko做很多事情，您可以用它构建非常复杂的系统。 这个想法是，您构建单个服务，这些服务可以发出其他服务可以订阅的事件，或者它们可以通过RPC直接相互调用。 服务之间的所有通信都是通过AMQP进行的。 您无需手动处理这些连接。

In addition to message exchange, services also use a lifecycle management to find useful resources through dependency injection. That sounds like a mouthful but is actually very simple. Because services are classes, you can add special attributes to them which will be resolved at runtime. The lifetime of the value resolved can be customized. For instance it becomes possible to attach a property to the class which can provide access to a database connection. The lifetime of that database connection can be automatically managed.

除了消息交换之外，服务还使用生命周期管理通过依赖项注入来找到有用的资源。 听起来像是满嘴，但实际上非常简单。 因为服务是类，所以可以向它们添加特殊的属性，这些属性将在运行时解析。 可以自定义解析值的生命周期。 例如，可以将属性附加到可以提供对数据库连接访问权限的类。 该数据库连接的生存期可以自动进行管理。

So how does that look in practice? Something like this:

那么实际情况如何？ 像这样：

from from nameko.rpc nameko.rpc import import rpc

rpc

class class HelloWorldServiceHelloWorldService (( objectobject ):
    ):
    name name = = 'helloworld'

    'helloworld'

    @rpc
    @rpc
    def def hellohello (( selfself , , namename ):
        ):
        return return "Hello, {}!""Hello, {}!" .. formatformat (( namename )
)

This defines a basic service that provides one method that can be invoked via RPC. Either another service can do that, or any other process that runs nameko can also invoke that, for as long as they connect to the same AMQP server. To experiment with this service, Nameko provides a shell helper that launches an interactive Python shell with an n object that provides RPC access:

这定义了一种基本服务，该服务提供了一种可以通过RPC调用的方法。 只要它们连接到同一AMQP服务器，要么其他服务可以执行此操作，否则运行nameko的任何其他进程也可以调用该操作。 为了试验该服务，Nameko提供了一个shell帮助器，该帮助器使用提供RPC访问的n对象启动一个交互式Python shell：

If the AMQP server is running, rpc.helloworld.hello contacts the helloworld service and resolves the hello method on it. Upon calling this method a message will be dispatched via the AMQP broker and be picked up by a nameko process. The shell will then block and wait for the result to come back.

如果AMQP服务器正在运行，则rpc.helloworld.hello将联系helloworld服务并解析其上的hello方法。 调用此方法后，将通过AMQP代理分发消息，并由nameko进程提取消息。 然后，shell将阻塞并等待结果返回。

A more useful example is what happens when services want to collaborate on some activity. For instance it’s quite common that one service wants to respond to the changes another service performs to update it’s own state. This can be achieved through events:

一个更有用的示例是，当服务要在某些活动上进行协作时会发生什么。 例如，一种服务想要响应另一种服务执行以更新其自身状态的更改是很常见的。 这可以通过以下事件来实现：

from from nameko.events nameko.events import import EventDispatcherEventDispatcher , , event_handler
event_handler
from from nameko.rpc nameko.rpc import import rpc

rpc

class class ServiceAServiceA (( objectobject ):
    ):
    name name = = 'servicea'
    'servicea'
    dispatch dispatch = = EventDispatcherEventDispatcher ()

    ()

    @rpc
    @rpc
    def def emit_an_eventemit_an_event (( selfself ):
        ):
        selfself .. dispatchdispatch (( 'my_event_type''my_event_type' , , 'payload''payload' )


)


class class ServiceBServiceB (( objectobject ):
    ):
    name name = = 'serviceb'

    'serviceb'

    @event_handler@event_handler (( 'servicea''servicea' , , 'my_event_type''my_event_type' )
    )
    def def handle_an_eventhandle_an_event (( selfself , , payloadpayload ):
        ):
        print print 'service b received''service b received' , , payload
payload

The default behavior is that one service instance of each service type will pick up the event. However nameko can also route an event to every single instance of every single service. This is useful for in-process cache invalidation for instance.

默认行为是每种服务类型的一个服务实例将接收事件。 但是，nameko还可以将事件路由到每个服务的每个实例。 例如，这对于进程内缓存无效很有用。

网络 (The Web)

Nameko is not just good for internal communication however. It uses Werkzeug to provide a bridge to the outside world. This allows you to accept an HTTP request and to ingest a task into your service world:

但是，Nameko不仅适合内部交流。 它使用Werkzeug为通往外界的桥梁。 这使您可以接受HTTP请求并将任务吸收到您的服务世界中：

The endpoint function can itself invoke other parts of the system via RPC or other methods.

端点函数本身可以通过RPC或其他方法调用系统的其他部分。

This functionality generally also extends into the websocket world, even though that part is currently quite experimental. It for instance is possible to listen to events and forward them into websocket connections.

该功能通常也扩展到websocket领域，即使该部分目前还处于试验阶段。 例如，可以侦听事件并将其转发到websocket连接。

依赖注入 (Dependency Injection)

One of the really neat design concepts in Nameko is the use of dependency injection to find resources. A good example is the SQLAlchemy bridge which attaches a SQLAlchemy database session to a service through dependency injection. The descriptor itself will hook into the lifecycle management to automatically manage the database resources:

Nameko真正精巧的设计概念之一是使用依赖项注入来查找资源。 一个很好的例子是SQLAlchemy桥，该桥通过依赖项注入将SQLAlchemy数据库会话附加到服务。 描述符本身将挂接到生命周期管理中，以自动管理数据库资源：

from from nameko_sqlalchemy nameko_sqlalchemy import import Session

Session

import import sqlalchemy sqlalchemy as as sa
sa
from from sqlalchemy.ext.declarative sqlalchemy.ext.declarative import import declarative_base

declarative_base

Base Base = = declarative_basedeclarative_base ()

()

class class UserUser (( BaseBase ):
    ):
    __tablename__ __tablename__ = = 'users'
    'users'
    id id = = sasa .. ColumnColumn (( sasa .. IntegerInteger , , primary_keyprimary_key == TrueTrue )
    )
    username username = = sasa .. ColumnColumn (( sasa .. StringString )


)


class class MyServiceMyService (( objectobject ):
    ):
    name name = = 'myservice'
    'myservice'
    session session = = SessionSession (( BaseBase )

    )

    @rpc
    @rpc
    def def get_usernameget_username (( selfself , , user_iduser_id ):
        ):
        user user = = selfself .. sessionsession .. queryquery (( UserUser )) .. getget (( user_iduser_id )
        )
        if if user user is is not not NoneNone :
            :
            return return useruser .. username
username

The implementation of the Session dependency provider itself is ridiculously simple. The whole functionality could be implemented like this:

会话依赖关系提供程序本身的实现非常简单。 整个功能可以这样实现：

The actual implementation is only a tiny bit more complicated, and that is basically just a bit of extra code to support different database URLs for different services and declarative bases. Overall the concept is the same however. When the dependency is needed, a connection to the database is established and when the worker shuts down, the session is closed.

实际的实现只是稍微复杂一点，并且基本上只是一些额外的代码，以支持用于不同服务和声明性基础的不同数据库URL。 总体而言，概念是相同的。 需要依赖项时，将建立与数据库的连接，当工作程序关闭时，会话将关闭。

并发与并行 (Concurrency and Parallelism)

What makes nameko interesting is that scales out really well through the use of AMQP and eventlet. First of all, when nameko starts a service container it uses eventlet to patch up the Python interpreter to support green concurrency. This allows a service container to become quite concurrent to do multiple things at once. This is very useful when a service waits on another service as threads in Python are a very disappointing story. As this however largely eliminates the possibility of true parallelism it becomes necessary to start multiple instances of services to scale up. Thanks to the use of AMQP however, this becomes a very transparent process. For as long as services do not need to store local state, it becomes very trivial to run as many of those service containers as necessary.

使nameko有趣的是，通过使用AMQP和eventlet可以很好地进行扩展。 首先，当nameko启动服务容器时，它使用eventlet修补Python解释器以支持绿色并发。 这使服务容器变得非常并发，可以一次执行多项操作。 当服务正在等待另一个服务时，这非常有用，因为Python中的线程是一个非常令人失望的故事。 但是，由于这在很大程度上消除了真正的并行性的可能性，因此有必要启动多个服务实例以进行扩展。 但是，由于使用了AMQP，这成为一个非常透明的过程。 只要服务不需要存储本地状态，就可以根据需要运行尽可能多的服务容器。

我的承担 (My Take On It)

Nameko as it stands has all the right principles for building a platform out of small services and it’s probably the best Open Source solution for this problem in the Python world so far.

目前的Nameko具有使用小型服务构建平台的所有正确原则，并且它可能是迄今为止Python世界中解决此问题的最佳开源解决方案。

It’s a bit disappointing that Python’s async story is so diverging between different Python versions and frameworks, but eventlet and gevent are by far the cleanest and most practical implementations, so for most intents and purposes the eventlet base in nameko is probably the best you can currently get for async IO. Fear not though, Nameko 2.0 now also runs on Python3.

令人失望的是，Python的异步故事在不同的Python版本和框架之间存在分歧，但是eventlet和gevent到目前为止是最干净，最实用的实现，因此，对于大多数意图和目的，nameko中的eventlet可能是目前最好的。获取异步IO。 不用担心，Nameko 2.0现在也可以在Python3上运行。

If you haven’t tried this sort of service setup yet, you might want to give Nameko a try.

如果您还没有尝试过这种服务设置，则可以尝试一下Nameko。

翻译自: https://www.pybloggers.com/2015/04/nameko-for-microservices/

nameko微服务