Thrift双向异步远程调用(外文翻译)学习第一步

本文翻译了一篇关于使用Thrift实现双向异步远程调用的文章。通过创建客户端对象,服务端可以在接收到客户端连接后发送消息。文章介绍了服务端创建处理器工厂和客户端的MessageReceiver线程,确保在保持连接时可以读取和处理异步消息。这种技术避免了轮询和双连接带来的问题,适用于需要双向通信的场景。
摘要由CSDN通过智能技术生成
最近在钻研thrift协议.
经过浅薄的学习后发现thrift在设计时是针对应答的模式,这样的方式使得服务端显得很被动,因为只有在客户端发送过来请求时服务端才能返回一条消息到客户端.经过在网上查询资料得知的确如此,网上目前最常见的方法是轮询和建立双连接的方式,这两种方法都属于"笨办法",不过也能实现双向通信.同时这两种方法存在各种各样的缺陷,比如防火墙穿透问题.这自然无法让人信服.后来在geogle浏览器的帮助下,找到一篇来自外国友人的文章,他在使用thrift实现了双向通信后将自己的宝贵经验分享出来供大家参考,因此我决定把这篇文章翻译出来,以使得各位能够参考下,本人英语水平特别差,但这不是重点,最重要的是能让大家都了解了解,如果有不对的地方希望大家能指正,本人不胜感激.
原文地址:http://joelpm.com/2009/04/03/thrift-bidirectional-async-rpc.html,内有原作者的源码分享
下面是原文翻译:
Thrift:双向异步远程调用
一个读者发送了一封thrift用户邮件,想知道thrift RPC(远程过程调用)是否可能实现从server端发送信息到client端.回复表明,通过轮询和双客户端双服务端的形式可以实现该目的.第二种方法要求客户端同时打开一个服务端,也就是说也需要一个端口,并且需要解决防火墙问题.为了完成类似的目的,我曾经使用过一些技术,读者的反映使我认识到,把这个技术写成描述并且举出对应的代码示例是有价值的.

(A reader posted to the thrift-user mailing list wondering if it was possible for a Thrift RPC server to send messages to the client. The responses indicated that this could be accomplished by polling the server for updates or hosting another Thrift server in the client that could receive RPCs from the server (requires opening another port on the client and handling firewall issues). I responded with a technique I'd used for accomplishing something similar and the responses made me think maybe this would be worth writing up and posting some example code.)

下面是我的发往邮件列表的email,其中描述了我的详细工作:
 我认为我之前所做的工作与你正在尝试的工作是类似的,和使用双客户端双服务端来实现从客户端接收服务端发来的数据一样的效果.
 当你的RPC被标识为异步,服务端不再主动返回一个回应,客户端也不再去读这个回应.因此如果所有的RPC的客户端全部为异步时,你必须有效释放socket的另一半连接.这意味着你可以用它来接收从服务端发来的异步消息.唯一需要注意的一点就是,你必须开启一个新的线程来读取和分派发来的异步RPC呼叫.
 在一个典型的Thrift RPC系统中你需要在服务端创建一个MyService.Processor并且在客户端创建一个MyService.Client.而为了达成双向异步通信的目的,你需要多做一步,在服务端为每一个连接到此的客户端创建一个MyService.Client,然后在每一个客户端都创建一个MyService.Processor.(假设你已经创建了如上所述的MyService结构并且有很多可选择的消息,另一个选择就是把客户端和服务端分开来定义)带有双客户端的现有连接如下所示:


(Here's my email to the mailing list that describes what I'm doing:

I think I've done something similar to what you're trying to do, and as long as you can commit to using only async messages it's possible to pull it off without having to start a server on the client to accept RPCs from the server.

When your RPC is marked as async the server doesn't send a response and the client doesn't try to read one. So, if all your RPC calls from the client to the server are async you have effectively freed up the inbound half of the socket connection. That means that you can use it for receiving async messages from the server - the only catch is that you have to start a new thread to read and dispatch the incoming async RPC calls.

In a typical Thrift RPC system you'd create a MyService.Processor on your server and a MyService.Client on your client. To do bidirectional async message sending you'll need to go a step further and create a MyService.Client on your server for each client that connects (this can be accomplished by providing your own TProcessorFactory) and then on each client you create a MyService.Processor. (This assumes that you've gone with a generic MyService definition like you described above that has a bunch of optional messages, another option would be to define separate service definitions for the client and server.) With two clients connected the objects in existence would look something like this:)



 Server:
  MyService.Processor mainProcessor -处理远程异步调用
   MyService.Client clientA - 用来发送异步消息到ClientA
   MyService.Client clientB - 用来发送异步消息到ClientB
 ClientA:
  MyService.Client - 用来向Server端发送消息
   MyService.Processor clientProcessor - 用来加工(通过一个新的线程)发送来的RPC

 ClientB:
   MyService.Client - 用来向Server端发送消息
   MyService.Processor clientProcessor - 用来加工(通过一个新的线程)发送来的RPC

但愿上面的示意能够解释清楚这个概念.如果你需要一个代码示例,我可以尝试写点新的东西出来(将使用Java语言).值得一提的是,你不需要创建双连接,因此你可以绕开防火墙等障碍.我已经将这种方法用于生产并且没有出过问题.当你在客户端使用新的线程去运行Processor,你基本上需要中断(阻塞)读取操作,等待从server端发来的数据.这样做的好处就是当server端关闭时你可以马上获得通知而不是必须等待直到你接收到一个消息时才发现TCP连接被重置了.

(Hopefully that explains the concept. If you need example code I can try and pull something together (it will be in Java). The nice thing about this method is that you don't have to establish two connections, so you can get around the firewall issues others have mentioned. I've been using this method on a service in production and haven't had any problems. When you have a separate thread in your client running a Processor you're basically blocking on a read, waiting for a message from the server. The benefit of this is that you're notified immediately when the server shuts down instead of having to wait until you send a message and then finding out that the TCP connection was reset.)

下面举一个例子,在一个已连接的客户端和服务端之间发送数据.类似于聊天应用.
(Here's an example app that sends messages between clients connected to a server. It's similar to a chat app.)


1.  Thrift Definition(Thrift 定义)

首先,定义你要用到的Thrift对象类和service.该例子中的类与service很简单.
(First, define your Thrift objects and the service. Our object and service are extremely simple:)

---------------------------------------------------------
namespace java com.joelpm.bidiMessages.generated

 
struct Message {
   1: string clientName,
   2: string message
}
 
 
service MessageService {
   oneway void sendMessage(Message msg),
}

-----------------------------------------------------------
在这个例子中定义了一个特别简单的service,客户端和服务端都将使用这个相同的service定义.如果需要的话,我们也可以创建一个ClientMessageService和一个ServerMessageService.
(In this case the service is generic enough that both the client and server will use the same service definition. We could also create a ClientMessageService and a ServerMesageService if we needed different functionality.)

2. Server(服务端定义)

在服务端,当一个客户端连接被接收到后,需要创建一个MessageService.Client用来回复一个消息到客户端.我们可以通过创建我们自己的TProcessorFactory并且使用getProcessor()方法作为获得在客户端与服务端之间传送消息的机会:
(On the server side, when a client connection is accepted we want to create a MessageService.Client object that we'll use to send messages back to the client. We can accomplish this by creating our own TProcessorFactory and using the getProcessor method as an opportunity to get access to the transport being used between the client and server:)

-------------------------------------------------------------------------------
final MessageDistributor messageDistributor = new MessageDistributor(); //消息分发器
 
 
     new Thread(messageDistributor).start(); //启动新的线程
     
     TProcessorFactory processorFactory = new TProcessorFactory(null) {
       @Override
       public TProcessor getProcessor(TTransport trans) {
          messageDistributor.addClient(new MessageServiceClient(trans));
          return new MessageService.Processor(messageDistributor);
        }
     };
 
 
     TServerTransport serverTransport = new TServerSocket(port);
     TServer server = new TThreadPoolServer(processorFactory, serverTransport);
     LOGGER.info("Server started");
     server.serve(); //开启服务器
------------------------------------------------------------------------------------
如上述,我们可以为每一个新创建的processor使用相同的MessageDistributor(消息分发器).在创建和返回processor之前,我们需要创建一个新的客户端并且把它添加到消息分发器产生的客户端列表中.服务端十分简单,你可以阅读源码来了解消息分发器如何使用客户端来回复消息

(As you can see above, we're using the same MessageDistributor for each new processor that we create. Before we create and return the processor we create a new client and add it to the list of clients that the MessageDistributor is aware of. The server is pretty simple and you can take a look at the code to see how the MessageDistributor uses the clients to send messages back.)

3. Client(客户端定义)
在客户端的定义略微复杂一些,因为我们需要创建新的线程去读取发送过来的消息(服务端通过TThreadPoolServer来处理),下面是读取消息的类:
(On the client side things are a little more complex because we have to create a separate thread to read incoming messages (this is handled by the TThreadPoolServer on the server side). Here's the class that reads incoming messages:)
--------------------------------------------------------------------------------------------
public class MessageReceiver extends ConnectionRequiredRunnable {
   private final MessageService.Processor processor;
   private final TProtocol protocol;
   
   public MessageReceiver(
        TProtocol protocol,
        MessageService.Iface messageService,
        ConnectionStatusMonitor connectionMonitor) {
      super(connectionMonitor, "Message Receiver");
      this.protocol = protocol;
      this.processor = new MessageService.Processor(messageService);
   }
 
 
   @Override
   public void run() {
      connectWait();
      while (true) {
         try {
           while (processor.process(protocol, protocol) == true) { }
         } catch (TException e) {
           disconnected();
         }
       }
   }
 }
---------------------------------------------------------------------------------------------+
它继承自一个叫做ConnectionRequiredRunnable的通用类,这个类提供了一些实用的方法来处理服务端断开和重新连接,但是总体来说还是非常简单的因为这个类事实上是用来处理发送来的消息的.我们也可以创建一个MessageService.Client,但是我们把它封装在一个新的线程中并使用一个阻塞式队列使得系统中的其他部分想要发送一个消息时变得非常快捷.或者至少在提交消息的时候非常快速.
(It extends a utility class called ConnectionRequiredRunnable that provides utility methods for handling server disconnects and reconnects, but on the whole it's pretty simple because we pass in a separate class that actually handles the incoming messages. We also create a MessageService.Client, but we wrap it in a separate thread and use a blocking queue so that other components in the system wanting to send a message can do so very quickly - or at least, have the message handed off for delivery extremely quickly.)

下面是负责处理消息发送的类:
(Here's the class that handles our message sending:)
---------------------------------------------------------------------------------+
public class MessageSender extends ConnectionRequiredRunnable {
   private final MessageService.Client client;
   private final BlockingQueue<Message> msgSendQueue;
   
   public MessageSender(TProtocol protocol,ConnectionStatusMonitor connectionMonitor) {
      super(connectionMonitor, "Message Sender");
      this.client = new MessageService.Client(protocol);
      this.msgSendQueue = new LinkedBlockingQueue<Message>();
   }
   
   public void send(Message msg) {
      msgSendQueue.add(msg);
   }
   
   @Override
   public void run() {
      connectWait();
      while (true) {
         try {
          Message msg = msgSendQueue.take();
          try {
             client.sendMessage(msg);
          } catch (TException e) {
            // The message isn't lost, but it could end up being sent out of
            // order - not ideal.
             msgSendQueue.add(msg);
           disconnected();
         }
       } catch (InterruptedException e) {
         // Thread will be interrupted if connection is lost, we should wait
         // for reconnection if that happens.
         connectWait();
       }
     }
   }
 }
-------------------------------------------------------------------------------------+

这个类继承自ConnectionRequiredRunnable类,无法在建立连接之前发送消息.下面是与服务端建立连接的客户端的main方法:
(This class also extends ConnectionRequiredRunnable since it can't send messages without a connection. Here's the main method of the Client that establishes the connection to the server:)
-------------------------------------------------------------------------------------+
 this.transport = new TSocket(server, port);
      this.protocol = new TBinaryProtocol(transport);
    
      this.connectionMonitor = new ConnectionStatusMonitor(transport);
    
      this.sender = new MessageSender(protocol, connectionMonitor);
      this.receiver = new MessageReceiver(protocol, messageHandler, connectionMonitor);

      new Thread(sender).start();
      new Thread(receiver).start();
    
      this.connectionMonitor.tryOpen();

-------------------------------------------------------------------------------------+

这个方法实际上是非常简单的因为所有的不同的片段都拆分成不同的类.一个叫做ConnectionStatusMonitor的类被用来开启连接并在连接建立时提醒MessageSender和MessageReceiver.与此同时它们将开始发动和接收消息.如果服务端因异常而关闭,所有的进程都将停止并等待直到一个连接被重新建立(由ConnectionStatusMonitor负责).下面是从服务端发出的样本:
(It actually looks pretty simple since all the different pieces are organized in separate classes. The ConnectionStatusMonitor class is responsible for opening the actual connection and notifying the MessageSender and MessageReceiver when the connection has been established, at which point they'll start sending and receiving messages. If the server dies both of those processes will stop and wait until a connection has been re-established (a task the ConnectionStatusMonitor is responsible for). Here's sample output from the server:)
-----------------------------------------------------------------------------------------------------------------+
1 2009-04-03 16:28:44,029  INFO main com.joelpm.bidiMessages.server.Server:43 - Server started
2 2009-04-03 16:28:45,814  INFO pool-1-thread-1 com.joelpm.bidiMessages.server.MessageDistributor:36 - Added client at 127.0.0.1
3 2009-04-03 16:28:45,822  INFO pool-1-thread-1 com.joelpm.bidiMessages.server.MessageDistributor:66 - Adding message to queue:
4 Message(clientName:client1, message:Hello there!)
5 2009-04-03 16:28:45,823  INFO pool-1-thread-1 com.joelpm.bidiMessages.server.MessageDistributor:66 - Adding message to queue:
6 Message(clientName:client1, message:Message 0)
7 2009-04-03 16:28:46,807  INFO pool-1-thread-1 com.joelpm.bidiMessages.server.MessageDistributor:66 - Adding message to queue:
8 Message(clientName:client1, message:Message 1)
9 2009-04-03 16:28:46,864  INFO pool-1-thread-2 com.joelpm.bidiMessages.server.MessageDistributor:36 - Added client at 127.0.0.1
10 2009-04-03 16:28:46,895  INFO pool-1-thread-2 com.joelpm.bidiMessages.server.MessageDistributor:66 - Adding message to queue:
11 Message(clientName:client2, message:Hello there!)
12 2009-04-03 16:28:46,897  INFO pool-1-thread-2 com.joelpm.bidiMessages.server.MessageDistributor:66 - Adding message to queue:
13 Message(clientName:client2, message:Message 0)
14 2009-04-03 16:28:47,805  INFO pool-1-thread-1 com.joelpm.bidiMessages.server.MessageDistributor:66 - Adding message to queue:
15 Message(clientName:client1, message:Message 2)
16 2009-04-03 16:28:47,885  INFO pool-1-thread-2 com.joelpm.bidiMessages.server.MessageDistributor:66 - Adding message to queue:
17 Message(clientName:client2, message:Message 1)
18 2009-04-03 16:28:48,806  INFO pool-1-thread-1 com.joelpm.bidiMessages.server.MessageDistributor:66 - Adding message to queue:
19 Message(clientName:client1, message:Message 3)
20 2009-04-03 16:28:48,885  INFO pool-1-thread-2 com.joelpm.bidiMessages.server.MessageDistributor:66 - Adding message to queue:
21 Message(clientName:client2, message:Message 2)
22 2009-04-03 16:28:49,806  INFO pool-1-thread-1 com.joelpm.bidiMessages.server.MessageDistributor:66 - Adding message to queue:
23 Message(clientName:client1, message:Message 4)
24 2009-04-03 16:28:49,885  INFO pool-1-thread-2 com.joelpm.bidiMessages.server.MessageDistributor:66 - Adding message to queue:
25 Message(clientName:client2, message:Message 3)
26 ^C2009-04-03 16:28:50,807  INFO pool-1-thread-1 com.joelpm.bidiMessages.server.MessageDistributor:66 - Adding message to queue:
27 Message(clientName:client1, message:Message 5)

---------------------------------------------------------------------------------------------------------+
还有client发送的数据样本:
(And here's output from client1:)
----------------------------------------------------------------------------------------------------------+

$ java -jar Client/target/BidiMessages.Client-0.9-jar-with-dependencies.jar client1 localhost 10101
2 2009-04-03 16:28:45,792  INFO Thread-3 com.joelpm.bidiMessages.client.ConnectionRequiredRunnable:42 - Message Receiver waiting for connection to be established.
3 2009-04-03 16:28:45,792  INFO Thread-2 com.joelpm.bidiMessages.client.ConnectionRequiredRunnable:42 - Message Sender waiting for connection to be established.
4 2009-04-03 16:28:45,803  INFO Thread-2 com.joelpm.bidiMessages.client.ConnectionRequiredRunnable:48 - Message Sender notified of connection, resuming execution
5 2009-04-03 16:28:45,806  INFO Thread-3 com.joelpm.bidiMessages.client.ConnectionRequiredRunnable:48 - Message Receiver notified of connection, resuming execution
6 Got msg: Message(clientName:client1, message:Hello there!)
7 Got msg: Message(clientName:client1, message:Message 0)
8 Got msg: Message(clientName:client1, message:Message 1)
9 Got msg: Message(clientName:client2, message:Hello there!)
10 Got msg: Message(clientName:client2, message:Message 0)
11 Got msg: Message(clientName:client1, message:Message 2)
12 Got msg: Message(clientName:client2, message:Message 1)
13 Got msg: Message(clientName:client1, message:Message 3)
14 Got msg: Message(clientName:client2, message:Message 2)
15 Got msg: Message(clientName:client1, message:Message 4)
16 Got msg: Message(clientName:client2, message:Message 3)
17 Got msg: Message(clientName:client1, message:Message 5)
18 2009-04-03 16:28:51,146  INFO Thread-3 com.joelpm.bidiMessages.client.ConnectionRequiredRunnable:30 - Message Receiver detected a disconnect from the server.
19 2009-04-03 16:28:51,148  INFO Thread-3 com.joelpm.bidiMessages.client.ConnectionRequiredRunnable:42 - Message Receiver waiting for connection to be established.
20 2009-04-03 16:28:51,149  INFO Thread-2 com.joelpm.bidiMessages.client.ConnectionRequiredRunnable:42 - Message Sender waiting for connection to be established.

---------------------------------------------------------------------------------------------------------+
然后是client2发送的数据样本:
(And here's from client2:)

1 $java -jar Client/target/BidiMessages.Client-0.9-jar-with-dependencies.jar client2 localhost 10101
2 2009-04-03 16:28:46,851  INFO Thread-2 com.joelpm.bidiMessages.client.ConnectionRequiredRunnable:42 - Message Sender waiting for connection to be established.
3 2009-04-03 16:28:46,854  INFO Thread-3 com.joelpm.bidiMessages.client.ConnectionRequiredRunnable:42 - Message Receiver waiting for connection to be established.
4 2009-04-03 16:28:46,867  INFO Thread-2 com.joelpm.bidiMessages.client.ConnectionRequiredRunnable:48 - Message Sender notified of connection, resuming execution
5 2009-04-03 16:28:46,879  INFO Thread-3 com.joelpm.bidiMessages.client.ConnectionRequiredRunnable:48 - Message Receiver notified of connection, resuming execution
6 Got msg: Message(clientName:client2, message:Hello there!)
7 Got msg: Message(clientName:client2, message:Message 0)
8 Got msg: Message(clientName:client1, message:Message 2)
9 Got msg: Message(clientName:client2, message:Message 1)
10 Got msg: Message(clientName:client1, message:Message 3)
11 Got msg: Message(clientName:client2, message:Message 2)
12 Got msg: Message(clientName:client1, message:Message 4)
13 Got msg: Message(clientName:client2, message:Message 3)
14 Got msg: Message(clientName:client1, message:Message 5)
15 2009-04-03 16:28:51,146  INFO Thread-3 com.joelpm.bidiMessages.client.ConnectionRequiredRunnable:30 - Message Receiver detected a disconnect from the server.
16 2009-04-03 16:28:51,147  INFO Thread-3 com.joelpm.bidiMessages.client.ConnectionRequiredRunnable:42 - Message Receiver waiting for connection to be established.
17 2009-04-03 16:28:51,147  INFO Thread-2 com.joelpm.bidiMessages.client.ConnectionRequiredRunnable:42 - Message Sender waiting for connection to be established.

---------------------------------------------------------------------------------------------------------+
可以看出当服务端终止后,client1和client2都暂停了.重启服务端后客户端将重新连接并再一次开始发送数据.
(You can see that client1 and client2 paused when the server was terminated. Had the server restarted the clients would have reconnected and begun sending messages again.)

翻译完了,作者很用心,把实现原理和测试程序都发出来了,而且分享了自己的源码,虽然已经是09年的文章,但的确让人觉得作者非常了不起.







评论 1
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值