0

I love gRPC, but I find every step of the protobuf process rather frustrating (particularly in Python). Even though they are structurally similar to data structures composed of lists and dicts, you can't really handle them that way.

  • You can't assign a list to a repeated field or a dict to a message (also means you can't use iterators/comprehensions)
  • You can't delete field attributes either (but you can del msg.lst[:])
  • Unpopulated fields are not "falsey": z.HasField('foo') will be false but not not z.foo is True.

I could go on. Basically, it feels like the folks that wrote the Python implementation of protobuf either don't actually know how to write idiomatic python, or want to actively discourage devs from using protobufs as datastructures within a service and only use them at API boundaries. Which is really unfortunate since they are extremely handy for enforcing types and structures (I aggressively use mypy/hints/autocomplete, **kwargs infuriates me). I've heard (haven't been able to find a source) that you aren't actually supposed to use protos dynamically this because of performance reasons, but I've also heard that Google does precisely this.

That leaves me with a few options:

  • Deal with un-ergonomic protobuf interfaces, take the possible performance hit
  • Use dicts and lists and parse them with json_format at ser/de boundaries (files, gRPC, etc) and lose static type checking
  • Parse at boundaries, but use stricter typed data objects. But now I have to maintain these data structures in lock step with the .proto files (I want to write a custom protoc generator but I don't have the time right now).

What's the best approach, given that I like to write rather structured python?

Improve this question
3
  • 1
    I can't speak for python, but I often walk between languages, and bring idioms along for the ride. I think worry less about being idiomatic, and worry more about whoever else is working on this code base. Will this be familiar to them? Are they happy to trial this? Do you need to mark the code with a banner describing the idioms you are using? Your chief worry is that this will make the code harder to work with, that depends a lot on who is working with that code.
    – Kain0_0
    Commented Apr 9, 2020 at 0:43
  • Great philosophy and 100% agree with going with the codebase flow, but I am pretty much pioneering gRPC techniques at my company so I have a lot of leeway to set the idioms here. I'm all about making the code work hard, not the coder, which points me toward eventually writing that custom generator. Because often there are neat things I want to bolt on to objects, and protobuf makes that nigh impossible. (e.g. a bounding_box type with .slice which lets you slice an array)
    – DeusXMachina
    Commented Apr 9, 2020 at 1:47
  • Oh, this actually isn't that complicated at all! stackoverflow.com/questions/28958135/…
    – DeusXMachina
    Commented Apr 9, 2020 at 2:47

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.