14

The syntax 3 of protobuf made all the fields optional dropping the keywords required and optional from previous proto2 syntax. Reading some comments from developers it seems that it was done for enhancing forward/backward binary compatibility.

But for me, that could be enforced by just versioning the package names, say com.example.messages.v1 and then let clients to implement deserializers they understand. At the same time it removes some contracts stated as a type that are useful from a software engineering point of view. For instance if I have

message Location {
   double latitude = 1;
   double longitude = 2;
}

In proto3 it is possible to create a half backed but perfectly valid Location by not providing one of the required fields.

Isn't that a big drawback when creating a schema based serialization format for exchanging data between clients? Isn't worse to move extra validation code to each client checking that all required fields have valid values?

13

proto3 makes a number of changes aimed (as I understand it) at making it far more usable in cross-platform scenarios. Explicit tracking of "assigned" vs "not assigned but reporting the default value" can be very hard to implement on some of the target platforms, and can also be confusing to use. As such, proto3 adopts a much simpler approach:

  • the implicit default value is the natural zero value (numbers / enums), false (booleans) or the empty string (strings)
  • only the implicit default is allowed; no other default values are permitted
  • if a field has that default value, it is not serialized; it doesn't matter whether it was assigned explicitly to zero / false / empty-string vs never assigned
  • because of this, there is no concept of "required", since "explicitly assigned a zero value" and "never assigned a value" look identical

In proto3 it is possible to create a half backed but perfectly valid Location by not providing one of the required fields.

The other value is: zero. The fact that you didn't explicitly assign it to zero is moot. Whether or not this is desirable is up to you, but it makes sense to me and is how a lot of "initialize a new object / struct" works on a wide range of platforms.

Isn't worse to move extra validation code to each client checking that all required fields have valid values?

There's nothing to validate! The layout is exactly what it would be if the value zero was explicitly assigned. If that is legal, it is legal. If it is illegal (because zero doesn't make sense to you), it is illegal; but it would be illegal whether it was explicit or implicit. The amount of validation involved doesn't change.

Isn't that a big drawback when creating a schema based serialization format for exchanging data between clients?

Not usually, no... especially since the schema version is explicit. If you want to use proto2: use proto2. Nothing changes automatically.

  • I wonder what "only natural zero as default" buys them, since protobuf isn't really usable without some kind of schema. Ensuring consistent defaults doesn't sound much harder than ensuring consistent schemas in the first place. – CodesInChaos Jun 8 '17 at 15:26
  • 1
    @CodesInChaos that's interesting, because I've been doing schema-less protobuf for ... well, forever :) in protobuf-net, it is expected that most users will be code-first and schema-less. And interestingly (for me, at least) most of the "keep it simple, stupid" default strategies that protobuf-net uses are exactly the same as what proto3 elected to use. I'm not going to claim that this validates my random partisan decisions, but... it totally does :) – Marc Gravell Jun 8 '17 at 15:52
  • I consider a C# class a kind of schema and you can certainly specify default values via attributes. Contrast this with for example msgpack or json, which can create a meaningful datastructure without any kind of schema. – CodesInChaos Jun 8 '17 at 15:54
  • @CodesInChaos agreed and noted - and yes, protobuf-net will observe and respect any such declarations – Marc Gravell Jun 8 '17 at 15:55

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.