protobuf.js如何读取二进制数据

How to read binary data in the browser or under node.js?

When reading/writing binary data in the browser or under node.js, it is mandatory to understand that just reading it (as a string) is not enough. When doing this, the data will probably become corrupted when it is converted between character sets and protobuf.js will not be able to decode it or will return random stuff. What’s actually required here is to properly handle the data as binary.
To make it easier for you to get this right, here is some insight on the topic:
• XMLHttpRequest: responseType=”arraybuffer”
• WebSockets: binaryType=”arraybuffer”
• node.js: Buffer
Browser XMLHttpRequest snippet
Simple case, when you do request with response encoded only with protobuf bytes

var xhr = new XMLHttpRequest();
xhr.open(
    /* method */ "GET",
    /* file */ "/path/to/encodedSomeMessageData.bin",
    /* async */ true
);
xhr.responseType = "arraybuffer";
xhr.onload = function(evt) {
    var msg = SomeMessage.decode(xhr.response);
    alert(JSON.stringify(msg, null, 4)); // Correctly decoded
}
xhr.send(null);

Multipart case, when you make request to endpoint that produce response encoded in “multipart” format, where different part of message could be encoded in different types, for example one part is encoded with XML and second part is encoded with protobuf. (For example, you can meet this case when you make requests against MTOM2 endpoint):

var xhr = new XMLHttpRequest();
xhr.open(
    /* method */ "GET",
    /* file */ "/path/to/encodedSomeMessageData.mtom2",
    /* async */ true
);
xhr.responseType = "arraybuffer";
xhr.onload = function(evt) {
    // reading whole body
    var bodyEncodedInString = String.fromCharCode.apply(String, new Uint8Array(xhr.response));
    console.log(bodyEncodedInString);
    /*
        for example it would be something like that in console:
        `RESPONSE_BODY:
            --protobuf
            [...someprotobytes]
            --protobuf--
        `
    */
var protoStart = bodyEncodedInString.indexOf('--protobuf');
   var protoEnd = bodyEncodedInString.indexOf('--protobuf--');
/*
        "Offset start" and "offset end" are required for line endings. Don't forget, that on various operating systems there is different line endings, and it is not so uncommon when your backend is running on Windows and generates "\r\n" in place of line endings.
    */
   var offsetStart = 2;
   var offsetEnd = 2;
var bufferSliceWhereProtobufBytesIs = xhr.response.slice(protoStart + offsetStart, protoEnd - offsetEnd);
var msg = SomeMessage.decode(bufferSliceWhereProtobufBytesIs);
   alert(JSON.stringify(msg, null, 4)); // Correctly decoded
}
xhr.send(null);

NOTE: In multipart case you need to use xhr.responseType = “arraybuffer”; and slice buffers via offsets because browsers, when read binary encoded parts, creates some corrupted text from it, and you can’t convert it to buffer and get correct ArrayBuffer for protobuf, so you need to slice original buffer (i did not found any solution, correct me if you find, please. Tested in Chrome, with native Typed arrays and Array Buffers).
Depending on the actual browser you are using, this may differ.
node.js: Properly using Buffers

var http = require("http");
http.get("http://some/url/to/binary.pb", function(res) {
    var data = []; // List of Buffer objects
    res.on("data", function(chunk) {
        data.push(chunk); // Append Buffer object
    });
    res.on("end", function() {
        data = Buffer.concat(data); // Make one large Buffer of it
        var myMessage = MyMessage.decode(data);
        ...
    });
    ...
});
...
protobuf中,可以使用bytes类型来存放二进制文件数据protobuf是一种用于序列化结构化数据的开源库,它可以将结构化数据转换为二进制格式,以便在网络传输或存储时使用。 在protobuf中,bytes类型是一种特殊的数据类型,用于存储任意二进制数据。它可以用来表示图片、音频、视频等二进制文件的内容。在.proto文件中定义消息类型时,可以使用bytes类型来声明一个字段,例如: message MyMessage { bytes file_data = 1; } 在上面的例子中,MyMessage消息类型包含一个名为file_data的字段,它的类型是bytes。这个字段可以用来存储二进制文件的内容。 当使用protobuf编码器将结构化数据编码为二进制格式时,可以将二进制文件的内容赋值给bytes类型的字段。例如,在C++中使用protobuf库编码时,可以使用以下代码: MyMessage message; message.set_file_data(file_content, file_size); 在上面的代码中,file_content是一个指向二进制文件内容的指针,file_size是文件的大小。通过调用set_file_data函数,将文件内容赋值给file_data字段。 当使用protobuf解码器将二进制数据解码为结构化数据时,可以通过访问bytes类型的字段来获取二进制文件的内容。例如,在C++中使用protobuf库解码时,可以使用以下代码: const std::string& file_data = message.file_data(); // 使用file_data进行后续处理 在上面的代码中,通过访问file_data字段,可以获取存储在其中的二进制文件内容,并进行后续处理。 需要注意的是,protobuf并不会对二进制文件的内容进行任何处理或解析,它只是将二进制数据存储在bytes类型的字段中。因此,在使用protobuf存储二进制文件时,需要确保正确地读取和写入文件内容。
评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值