gRPC 服务端流式RPC基于python的应用
应用场景: python服务端 java客户端 通过grpc进行通信
java客户端
-
在与Sources同级的文件夹下新建proto文件夹,在proto文件夹中新建 msg.proto
-
msg.proto内容如下
// @1 使用proto3语法
syntax = "proto3";
// @2 生成多个类(一个类方便管理)
option java_multiple_files = false;
// @3 生成java类所在包
option java_package = "com.example.spbclient.proto";
// @4 生成外层类类名
option java_outer_classname = "MsgProto";
// @6 .proto包名(逻辑包名)
package msg;
// @7 定义服务,用于描述要生成的API接口,类似于Java的业务逻辑接口类
service MsgService {
// imgIdentify 方法名 ImgRequest 传入参数 ImgResponse 返回响应
//注意:这里是returns 不是return
// Unary RPC(单项RPC)
rpc GetMsg (MsgRequest) returns (MsgResponse) {}
// A server-to-client streaming RPC.(服务端流式RPC)
rpc GetMsgByServerStream(MsgRequest) returns (stream MsgResponse) {}
// A client-to-server streaming RPC. (客户端流式RPC)
rpc GetMsgByClientStream(stream MsgRequest) returns (MsgResponse) {}
// A Bidirectional streaming RPC.(双向流式RPC)
rpc GetMsgByBidirectionalStream(stream MsgRequest) returns (stream MsgResponse) {}
}
//定义请求数据结构
// string 数据类型
// calName 参数名称
// 1 序号、索引值(表示第一个参数,防止传参顺序错乱),一旦开始就不能改变
// int32 int
// int64 long
// 不可以使用 19000-19999 保留字
// TODO 大小写问题
message MsgRequest {
string name = 1;
}
message MsgResponse {
string msg = 1;
}
-
pom.xml文件增加依赖和插件
<?xml version="1.0" encoding="UTF-8"?> <project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd"> <modelVersion>4.0.0</modelVersion> <parent> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-parent</artifactId> <version>3.1.1</version> <relativePath/> <!-- lookup parent from repository --> </parent> <groupId>com.example</groupId> <artifactId>grpc-demo</artifactId> <version>0.0.1-SNAPSHOT</version> <name>grpc-demo</name> <description>grpc-demo</description> <properties> <java.version>17</java.version> <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding> <grpc.version>1.56.0</grpc.version><!-- CURRENT_GRPC_VERSION --> <protobuf.version>3.22.3</protobuf.version> <protoc.version>3.22.3</protoc.version> </properties> <dependencyManagement> <dependencies> <dependency> <groupId>io.grpc</groupId> <artifactId>grpc-bom</artifactId> <version>${grpc.version}</version> <type>pom</type> <scope>import</scope> </dependency> </dependencies> </dependencyManagement> <dependencies> <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-thymeleaf</artifactId> </dependency> <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-web</artifactId> </dependency> <dependency> <groupId>org.projectlombok</groupId> <artifactId>lombok</artifactId> <optional>true</optional> </dependency> <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-test</artifactId> <scope>test</scope> </dependency> <!--grpc所需的依赖--> <dependency> <groupId>io.grpc</groupId> <artifactId>grpc-netty-shaded</artifactId> <scope>runtime</scope> </dependency> <dependency> <groupId>io.grpc</groupId> <artifactId>grpc-protobuf</artifactId> </dependency> <dependency> <groupId>io.grpc</groupId> <artifactId>grpc-services</artifactId> </dependency> <dependency> <groupId>io.grpc</groupId> <artifactId>grpc-stub</artifactId> </dependency> <dependency> <groupId>com.google.protobuf</groupId> <artifactId>protobuf-java-util</artifactId> <version>${protobuf.version}</version> </dependency> <dependency> <groupId>com.google.code.gson</groupId> <artifactId>gson</artifactId> <version>2.10.1</version> <!-- prevent downgrade via protobuf-java-util --> </dependency> <dependency> <groupId>com.google.guava</groupId> <artifactId>guava</artifactId> <version>31.1-jre</version> <!-- prevent downgrade of version in protobuf-java-util --> </dependency> <dependency> <groupId>org.apache.tomcat</groupId> <artifactId>annotations-api</artifactId> <version>6.0.53</version> <scope>provided</scope> <!-- not needed at runtime --> </dependency> <dependency> <groupId>io.grpc</groupId> <artifactId>grpc-testing</artifactId> <scope>test</scope> </dependency> <dependency> <groupId>junit</groupId> <artifactId>junit</artifactId> <version>4.13.2</version> <scope>test</scope> </dependency> <!-- <dependency>--> <!-- <groupId>org.mockito</groupId>--> <!-- <artifactId>mockito-core</artifactId>--> <!-- <version>3.4.0</version>--> <!-- <scope>test</scope>--> <!-- </dependency>--> <dependency> <groupId>net.devh</groupId> <artifactId>grpc-client-spring-boot-starter</artifactId> <version>2.12.0.RELEASE</version> </dependency> </dependencies> <build> <extensions> <extension> <groupId>kr.motd.maven</groupId> <artifactId>os-maven-plugin</artifactId> <version>1.7.1</version> </extension> </extensions> <plugins> <plugin> <groupId>org.xolstice.maven.plugins</groupId> <artifactId>protobuf-maven-plugin</artifactId> <version>0.6.1</version> <configuration> <protocArtifact>com.google.protobuf:protoc:${protoc.version}:exe:${os.detected.classifier} </protocArtifact> <pluginId>grpc-java</pluginId> <pluginArtifact>io.grpc:protoc-gen-grpc-java:${grpc.version}:exe:${os.detected.classifier} </pluginArtifact> </configuration> <executions> <execution> <goals> <goal>compile</goal> <goal>compile-custom</goal> </goals> </execution> </executions> </plugin> <plugin> <groupId>org.apache.maven.plugins</groupId> <artifactId>maven-enforcer-plugin</artifactId> <version>1.4.1</version> <executions> <execution> <id>enforce</id> <goals> <goal>enforce</goal> </goals> <configuration> <rules> <requireUpperBoundDeps/> </rules> </configuration> </execution> </executions> </plugin> <plugin> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-maven-plugin</artifactId> <configuration> <excludes> <exclude> <groupId>org.projectlombok</groupId> <artifactId>lombok</artifactId> </exclude> </excludes> </configuration> </plugin> </plugins> </build> </project>
这里grpc1.56.0版本用到的mockito-core和springboot里引用的mockito-core发生了版本冲突,直接使用springboot引用的版本,目前没发现任何问题
-
直接编译maven项目,或者使用protobuf插件都可以生成grpc的代码
-
执行完成可以在target/generated-sources中看到生成的文件,主要grpc-java和java都需要是源文件类型,不是的话自己设置一下
-
java作为客户端的代码
package com.example.grpc.demo.client; import com.example.spbclient.proto.MsgProto; import com.example.spbclient.proto.MsgServiceGrpc; import io.grpc.Channel; import io.grpc.ManagedChannel; import io.grpc.ManagedChannelBuilder; import io.grpc.StatusRuntimeException; import io.grpc.stub.StreamObserver; import lombok.SneakyThrows; import lombok.extern.slf4j.Slf4j; import java.util.Iterator; import java.util.concurrent.TimeUnit; @Slf4j public class MsgClient { private final MsgServiceGrpc.MsgServiceBlockingStub msgServiceBlockingStub; private final MsgServiceGrpc.MsgServiceStub msgServiceStub; public MsgClient(Channel channel) { msgServiceBlockingStub = MsgServiceGrpc.newBlockingStub(channel); msgServiceStub = MsgServiceGrpc.newStub(channel); } /** * 给服务端发送请求 * * @param name */ public void getMsg(String name) { log.info("给服务端发送消息: " + name + "..."); MsgProto.MsgRequest request = MsgProto.MsgRequest.newBuilder().setName(name).build(); MsgProto.MsgResponse response = msgServiceBlockingStub.getMsg(request); log.info("收到服务端返回的消息: " + response.getMsg()); } /** * 给服务端发送请求 服务端流式RPC * * @param name */ public void getMsgByServerStream(String name) { log.info("给服务端发送消息: " + name + "..."); MsgProto.MsgRequest request = MsgProto.MsgRequest.newBuilder().setName(name).build(); Iterator<MsgProto.MsgResponse> msgResponseIterator; try { msgResponseIterator = msgServiceBlockingStub.getMsgByServerStream(request); for (int i = 1; msgResponseIterator.hasNext(); i++) { MsgProto.MsgResponse msgResponse = msgResponseIterator.next(); log.info("第{}次收到服务端返回消息:{}", i, msgResponse.getMsg()); } } catch (StatusRuntimeException e) { log.warn("RPC failed: {}", e.getStatus()); } } /** * 给服务端发送请求 客户端流式RPC * * @param name */ @SneakyThrows public void getMsgByClientStream(String name) { System.out.println("-----------------------------"); System.out.println("流式请求-响应,getMsgByClientStream"); StreamObserver<MsgProto.MsgResponse> responseObserver = new StreamObserver<MsgProto.MsgResponse>() { @Override public void onNext(MsgProto.MsgResponse msgResponse) { log.warn("getMsgByClientStream onNext:{}", msgResponse.getMsg()); } @Override public void onError(Throwable t) { log.error("getMsgByClientStream onError: {}", t.toString()); } @Override public void onCompleted() { log.warn("getMsgByClientStream onCompleted"); } }; StreamObserver<MsgProto.MsgRequest> requestObserver = msgServiceStub.getMsgByClientStream(responseObserver); try { //循环发送100次消息 for (int i = 0; i < 5; i++) { MsgProto.MsgRequest request = MsgProto.MsgRequest.newBuilder().setName(name + i).build(); log.info(String.format("客户端发送%s", request.getName())); requestObserver.onNext(request); // Sleep for a bit before sending the next one. Thread.sleep(1000); } } catch (RuntimeException e) { // Cancel RPC requestObserver.onError(e); throw e; } requestObserver.onCompleted(); try { Thread.sleep(50000); } catch (InterruptedException e) { e.printStackTrace(); } log.info("客户端流式请求发送完毕"); } /** * 双向流 * * @param name */ public void getMsgByBidirectionalStream(String name) { System.out.println("-----------------------------"); System.out.println("流式请求-流式响应,调用BiTalk"); StreamObserver<MsgProto.MsgRequest> requestStreamObserver = msgServiceStub.getMsgByBidirectionalStream(new StreamObserver<MsgProto.MsgResponse>() { @Override public void onNext(MsgProto.MsgResponse response) { System.out.println("收到服务端返回的数据" + response.getMsg()); } @Override public void onError(Throwable throwable) { System.out.println(throwable.getMessage()); } @Override public void onCompleted() { System.out.println("onComplated"); } }); for (int i = 0; i < 10; i++) { requestStreamObserver.onNext(MsgProto.MsgRequest.newBuilder().setName(name + i).build()); try { Thread.sleep(1000); } catch (InterruptedException e) { e.printStackTrace(); } } try { Thread.sleep(50000); } catch (InterruptedException e) { e.printStackTrace(); } } /** * 测试 * * @param args * @throws InterruptedException */ public static void main(String[] args) throws InterruptedException { String user = "Tom"; String target = "localhost:50051"; ManagedChannel channel = ManagedChannelBuilder.forTarget(target) .usePlaintext() .build(); try { MsgClient client = new MsgClient(channel); // // client.getMsg(user); client.getMsgByServerStream(user); // client.getMsgByClientStream(user); // client.getMsgByBidirectionalStream(user); } finally { channel.shutdownNow().awaitTermination(5, TimeUnit.SECONDS); } } }
python服务端
-
新建python项目,项目中新建prtot包,把之前写好的msg.proto拷贝到包路径下,然后在该包路径下执行一下python命令,生成代码
pip install grpcio pip install grpcio-tools googleapis-common-protos python -m grpc_tools.protoc -I . --python_out=. --grpc_python_out=. msg.proto
-
执行完成后我们可以看到proto文件夹下多了两个文件,msg_pb2.py和msg_pb2_grpc.py。
-
在proto包下的init.py文件中添加如下代码(否则无法引入proto包下的文件)
import sys # 将__init__.py文件所在目录加入Python搜索目录中,否则会提示找不到myModule1 sys.path.append(__file__[:-12])
-
在项目目录下新建main_server.py文件
import grpc import proto.msg_pb2 as msg_pb2 import proto.msg_pb2_grpc as msg_pb2_grpc # 并发 from concurrent import futures import time # Unary RPC(单项RPC) _ONE_DAY_IN_SECONDS = 60 * 60 * 24 # service 实现GetMsg方法 MsgServicer继承MsgServiceServicer class MsgServicer(msg_pb2_grpc.MsgServiceServicer): def GetMsg(self, request, context): print("收到客户端请求: %s" % request.name) return msg_pb2.MsgResponse(msg='Hello, %s!' % request.name) def GetMsgByServerStream(self, request, context): print("流式服务器收到客户端请求: %s" % request.name) while True: yield msg_pb2.MsgResponse(msg='Hello, %s!' % request.name) time.sleep(1) def serve(): server = grpc.server(futures.ThreadPoolExecutor(max_workers=10)) msg_pb2_grpc.add_MsgServiceServicer_to_server(MsgServicer(), server) server.add_insecure_port('[::]:50051') server.start() try: while True: time.sleep(_ONE_DAY_IN_SECONDS) except KeyboardInterrupt: server.stop(0) if __name__ == '__main__': serve()
-
启动python服务端,再启动java客户端即可看到 流式服务端的效果
总结:grpc流式服务端,实际就是客户端发送个消息,表示你可以给我发消息了,然后服务端就开始一直给客户端返回消息。