Questions tagged [protobuf]

Protocol Buffers are a serialization format with an interface description language developed by Google. Source: en.wikipedia.org/wiki/Protobuf

Filter by
Sorted by
Tagged with
19
votes
2answers
11k views

Protobuf design patterns

I am evaluating Google Protocol Buffers for a Java based service (but am expecting language agnostic patterns). I have two questions: The first is a broad general question: What patterns are we ...
13
votes
1answer
7k views

Why Protobuf 3 made all fields on the messages optional?

The syntax 3 of protobuf made all the fields optional dropping the keywords required and optional from previous proto2 syntax. Reading some comments from developers it seems that it was done for ...
5
votes
1answer
3k views

Is it rational to convert protobuf into json to send it to a web server?

Protobuf is a good thing, it allows c++ devs to not bother with class serialization/deserialization, is fast and .proto format is very nice. It also allows automatic data validation. But the messages ...
3
votes
2answers
544 views

Reverse engineering an ETL pipeline

I have a one-way large ETL pipeline in Scala where I start with protobuf schema A and end up with protobuf schema B. I want to create a many-to-many mapping between the fields of schema A and schema B,...
2
votes
1answer
62 views

Protobuf: compile specification once, use generated code everywhere?

Once written, a protobuf specification can be compiled using protoc to a variety of implementations in different languages (e.g. python, c++, go). That's great. But to compile that generated code, you ...
1
vote
1answer
255 views

Why protobuf is said to be in binary format although we write it in text format similar to json

JSON { "hello": "world" } Protobuf message HelloWorld { required string name = 1; } Why do we say that protobuf uses binary format during network exchange and json don't, even though network ...
1
vote
2answers
99 views

Options for having model, parsers and serializers for a given data-format generated in multiple languages?

I am member of the Apache PLC4X (incubating) project. Here we are currently implementing multiple industry PLC protocols. While we initially focussed on creating Java versions of these, we are ...
1
vote
0answers
26 views

Can I add new function for protobuf generated struct

I am using protobuf definitions to define message types. Structure looks like message1 { } message2 { bytes msg1 = 1 } So message2 has a field which is a byte array representation of message1. I ...
1
vote
2answers
751 views

Arbitrary protobuf message as byte-array over Websocket - how to determine actual message type upfront

Protobuf is nice and dandy, but it was not made with self-description in mind. Now this is totally okay if you're using a well-defined protocol and want to replace e.g. SOAP-messages or just want to ...
0
votes
2answers
596 views

Consuming external APIs within the microservice architecture

I'm writing a small microservice-based application. One of the services is assigned the task of querying an external API and processing the JSON response (filtered list of houses). Since I'm using ...
0
votes
2answers
97 views

What are the “scalar fields” and “composite fields” in JAVA?

Need to understand the below definition, from the protobuf tutorial: mergeFrom(Message other): (builder only) merges the contents of other into this message, overwriting singular scalar fields, ...
0
votes
1answer
20 views

How to handle adding new field / partial data update for a gRPC API?

Assuming we have following message that will be used to update data and it just got updated to version 2. message HelloRequest { string name = 1; // version 1 bool is_valid = 2; // version 2 } ...
0
votes
1answer
662 views

JSON or protocol buffer to replace the custom TCP message

Originally, We have two applications communication with TCP/IP, and both of them are implemented by C++. The messages between them are custom message type. Now the client program will be changed to ...
0
votes
0answers
140 views

Is it possible to use Protocol Buffers natively for message passing between a Go web service and an Android app?

I want to connect an Android app to a Go web service (I have a small social network in mind) and would like to pass data to and from the web service using Protocol Buffers, since it recently seems to ...