linux应用程序开源架构_Deepstream：用于构建实时应用程序的开源服务器

linux应用程序开源架构

It’s Open Source Week at SitePoint! All week we’re publishing articles focused on everything Open Source, Free Software and Community, so keep checking the OSW tag for the latest updates.

这是SitePoint的开源周！ 我们整周都在发布有关开源，免费软件和社区的文章，因此请继续检查OSW标签以获取最新更新。

Elton, the deepstream mascot

深水吉祥物埃尔顿

Realtime apps are getting really popular, but they’re also hard to build. Wolfram Hempel introduces deepstream, an open-source server he co-founded to make data-sync, request-response and publish-subscribe a whole lot easier.

实时应用正变得非常流行，但也很难构建。 Wolfram Hempel引入了Deepstream ，这是他与他人共同创建的开源服务器，它使数据同步，请求响应和发布-订阅变得更加容易。

实时应用的兴起 (The Rise of Realtime Apps)

Realtime is eating the world! Or at least it’s taking bigger and bigger bites. Whether it’s collaborative editing in Google Docs, chatting via Facebook messenger, financial trading on the move, IoT controls, live dashboards or multiplayer gaming — users are increasingly expecting to see changes happen as they happen.

实时正在吞噬世界！ 或至少它正在越来越大的咬人。 无论是在Google文档中进行协作式编辑，通过Facebook Messenger进行聊天，移动中的金融交易，物联网控件，实时仪表板还是多人游戏，用户都越来越期望看到变化的发生。

Even traditionally static sites like social networks or forums are starting to abandon the refresh button and instead stream updates directly into your feed.

甚至传统的静态网站(例如社交网络或论坛)也开始放弃刷新按钮，而是将更新直接流式传输到您的Feed中。

But as popular as realtime apps are, they’re also hard to build. While it’s possible for smaller projects or POCs to introduce realtime features just by adding a pinch of Socket.io, large-scale use cases require a fundamentally different architecture. Concepts like concurrent connections, failover, streaming data-consistency, persistence, encryption and permissioning all have to be woven into the fabric that powers this new generation of apps.

但是，与实时应用程序一样，它们也很难构建。 尽管对于较小的项目或POC而言 ，仅通过添加少量Socket.io即可引入实时功能是可能的，但大规模的用例需要根本不同的体系结构。 诸如并发连接，故障转移，流数据一致性，持久性，加密和许可之类的概念都必须编织到为新一代应用程序提供支持的结构中。

One industry where I learned this only too well is investment banking. The servers that power the myriad of flashing screens on the world’s trading floors are monolithic beasts, complicated and breathtakingly expensive. But they are fast. Very fast. And they’ve got something else right: they use a concept called “data-sync”.

我学得很好的一个行业是投资银行。 在全球交易大厅中为无数闪烁屏幕提供动力的服务器都是整体的野兽，它们既复杂又昂贵。 但是他们很快。 非常快。 他们有其他正确的选择：他们使用了一种称为“数据同步”的概念。

实时概念 (Realtime Concepts)

If you’ve already built a realtime app, chances are you’ve used a pattern called “publish-subscribe” or “pub-sub” for short: subscribers listen for events on a channel and others publish these events. It’s an efficient mechanism for many-to-many communication that’s supported by a wide range of technologies and services such as Socket.io or SocketCluster on the open source side, or Pusher, PubNub or Ably in the PaaS space.

如果您已经构建了一个实时应用程序，那么您可能会使用一种称为“发布-订阅”或“发布-订阅”的模式：订户收听频道上的事件，而其他人则发布这些事件。 这是一个有效的机制，多到很多，用一个广泛的技术和服务，如支持的通信Socket.io或SocketCluster在开源方面，或推 ， PubNub或干练的PaaS的空间。

But there’s one crucial thing that pub-sub can’t provide: state. Pretty much every app has some state — data that needs to be created, read, updated and deleted, but pub-sub only delivers one-off messages that vanish immediately. A common workaround is the use of events as update notifications which in turn prompt the client to retrieve the latest state via traditional request-response. But that’s complicated, prone to inconsistencies and, most importantly, slow.

但是pub-sub无法提供的一项关键功能是：状态。 几乎每个应用程序都具有某种状态-需要创建，读取，更新和删除数据，但是pub-sub仅提供一次性消息，这些消息会立即消失。 常见的解决方法是将事件用作更新通知，这又提示客户端通过传统的请求-响应来检索最新状态。 但这很复杂，容易出现不一致，而且最重要的是，它很慢。

This has led to the increasing move towards data-sync, an approach that combines statefulness with realtime updates. Data is persistent as well as kept in sync between connected clients and backend processes.

这导致了越来越多的数据同步化，这种方法将状态与实时更新结合在一起。 数据是持久的，并且在连接的客户端和后端进程之间保持同步。

Technologies that support this are much rarer. On the open-source side there used to be the now discontinued horizon.io; in the PaaS space there’s Google’s Firebase.

支持此功能的技术很少见。 在开源方面，曾经是现已停产的horizo​​n.io ； 在PaaS空间中，有Google的Firebase 。

To fill this gap, we’ve started deepstream.io. Our aim was to create an open-source server with the same performance and versatility that financial trading or multiplayer gaming systems deliver, but in an open and extendable way that makes it easy to use for any kind of app.

为了填补这一空白，我们已经启动deepstream.io 。 我们的目标是创建一种具有与金融交易或多人游戏系统相同的性能和多功能性的开源服务器，但其开放性和可扩展性使其可以轻松用于任何类型的应用程序。

Deepstream.io (Deepstream.io)

Deepstream is a new type of server that handles realtime data at scale. Its installed similar to a database or an HTTP server. End users and backend services connect to it via lightweight SDKs that come in a range of different programming languages, such as JS/Node, Java/Android, or Swift/ObjC.

Deepstream是一种新型服务器，可大规模处理实时数据。 它的安装类似于数据库或HTTP服务器。 最终用户和后端服务通过轻量级SDK与其连接，这些轻量级SDK包含一系列不同的编程语言，例如JS / Node，Java / Android或Swift / ObjC。

It provides data-sync as well as pub-sub and classic request-response and caters for a wide range of functional requirements such as failover, permissioning, encryption, consistency and conflict resolution.

它提供数据同步，发布订阅和经典请求响应，并满足各种功能需求，例如故障转移，许可，加密，一致性和冲突解决。

Deepstream is designed to thrive in open-source ecosystems, and comes with a range of connectors for popular databases, caches or message busses.

Deepstream旨在在开源生态系统中蓬勃发展，并提供了一系列用于流行数据库，缓存或消息总线的连接器。

But most importantly: it’s scalable, reliable and very fast.

但最重要的是：它可扩展，可靠且非常快。

使用深度流 (Using Deepstream)

All of this sounds good and well — but how does it actually work? Let’s touch on the main points quickly.

所有这些听起来都很好，但是实际上如何工作？ 让我们快速介绍要点。

安装 (Installation)

Deepstream comes as Mac and Windows executable, yum and apt package or Docker image, all of which can be found on the install page.

Deepstream是Mac和Windows可执行文件， yum和apt软件包或Docker映像，所有这些都可以在安装页面上找到。

组态 (Configuration)

Every aspect of the server can be configured in a file called config.yml, located in either /etc/deepstream/conf/ on Linux or in the conf directory on Windows or Mac

可以在名为config.yml的文件中配置服务器的各个方面，该文件位于Linux上的/etc/deepstream/conf/或Windows或Mac上的conf目录中。

启动服务器 (Starting the server)

The server is started either by running deepstream start on the command line or by double clicking the executable.

通过在命令行上运行deepstream start或双击可执行文件来deepstream start服务器。

获取客户端SDK (Getting a client SDK)

Connecting to deepstream requires an SDK for the given programming language. For browsers and Node, for example, this can be installed via npm install deepstream.io-client-js.

连接到Deepstream需要使用给定编程语言的SDK。 例如，对于浏览器和Node，可以通过npm install deepstream.io-client-js进行安装。

连接到服务器 (Connecting to the server)

The simplest way to connect is by calling var client = deepstream('localhost:6020').login(). There’s a lot more to add to this line, such as passing client options, authentication parameters or waiting for a callback after login, but let’s just leave it at that.

连接的最简单方法是调用var client = deepstream('localhost:6020').login() 。 在此行中还有很多要添加的内容，例如传递客户端选项，身份验证参数或在登录后等待回调，但我们只需要保留它即可。

使用数据同步 (Using data-sync)

Deepstream’s data-sync uses a concept called “records” — JSON documents that can be manipulated and observed and are synced across all connected clients as well as persisted on the backend. This sounds more complicated than it is. Records are identified by a unique name and created or loaded on the fly:

Deepstream的数据同步使用一种称为“记录”的概念— JSON文档可以进行操作和观察，并且可以在所有连接的客户端之间进行同步，也可以保留在后端。 这听起来比实际要复杂。 记录由唯一名称标识，并即时创建或加载：

pizzaGuy = ds.record.getRecord( 'driver/14' )

The value of a record can be set like so:

记录的值可以这样设置：

pizzaGuy.set({
  name: 'John Doe',
  position: { x: 4234, y: 2454 },
  speed: 22
})

… or partially, like so:

……或部分地像这样：

pizzaGuy.set( 'position.x', 4244 )

Similarly, other clients can subscribe to changes to the entire record:

同样，其他客户端可以订阅对整个记录的更改：

pizzaGuy.subscribe(( data )=>{
  //...
})

… or to a path within it:

…或其中的路径：

pizzaGuy.subscribe( 'position', updateMapMarker )

使用事件 (Using events)

Events are deepstream’s pub-sub mechanism. They provide ephemeral many-to-many messaging. Every client can subscribe to an event:

事件是Deepstream的pub-sub机制。 它们提供短暂的多对多消息传递。 每个客户都可以订阅一个事件：

ds.event.subscribe( 'something-happened', data => {})

… or emit it:

…或发出它：

ds.event.emit( 'something-happened', { size: 'big' })

使用RPC (Using RPCs)

Remote procedure calls are deepstream’s mechanism for request-response communication. Deepstream routes requests between provider and requestor, manages failover, retrying and data-serialisation.

远程过程调用是Deepstream的请求-响应通信机制。 深度流在提供者和请求者之间路由请求，管理故障转移，重试和数据序列化。

Processes can register themselves as RPC providers:

进程可以将自己注册为RPC提供程序：

ds.rpc.provide( 'add-two', input, ( response ) => {
  response.send( input + 2 )
})

… and request them:

…并要求他们：

ds.rpc.make( 'add-two', 5, ( err, result ) => { /* result = 7 */})

身份验证和许可 (Authentication and permissioning)

Deepstream offers a range of different strategies to authenticate incoming connections, such as via config files or http-webhooks. Every incoming request is authenticated using a realtime permission language called Valve.

Deepstream提供了一系列不同的策略来验证传入的连接，例如通过配置文件或http-webhooks 。 每个传入的请求都使用称为Valve的实时许可语言进行身份验证。

record:
  #an auctioned item
  auction/item/$sellerId/$itemId:

    #everyone can see the item and its price
    read: true

    #only users with canBid flag in their authData can bid
    #and bids can only be higher than the current price
    write: "user.data.canBid && data.price > oldData.price"

    #only the seller can delete the item
    delete: "user.id == $sellerId"

添加连接器 (Adding connectors)

It’s easy to add databases such as Mongo, Rethink or Postgres, caches like Redis, Memcached or Hazelcast, or Messaging Systems such as RabbitMQ or Kafka to deepstream using connectors. All connectors are installed via the commandline — for example, by running the following:

使用连接器将Mongo ， Rethink或Postgres之类的数据库， Redis ， Memcached或Hazelcast之类的缓存或RabbitMQ或Kafka之类的消息系统添加到使用连接器的深流中很容易。 所有连接器都是通过命令行安装的，例如，通过运行以下命令：

deepstream install cache redis

放在一起 (Putting It All Together)

To summarize, deepstream is a universal, scalable and performant realtime server that’s usable as a backend for use cases ranging from CRUD applications to demanding messaging apps, realtime dashboard or even multiplayer games. It’s robust, secure and provides all the features necessary to run large-scale realtime apps in production.

总而言之，Deepstream是一种通用，可扩展且高性能的实时服务器，可用作从CRUD应用程序到苛刻的消息传递应用程序，实时仪表板甚至多人游戏等用例的后端。 它强大，安全，并提供在生产中运行大型实时应用程序所需的所有功能。

翻译自: https://www.sitepoint.com/deepstream-an-open-source-server-for-building-realtime-apps/

linux应用程序开源架构