If the User Management Service is scaling, leading to multiple instances running at the same time, how can it be ensured that the logic doesn't add the same user twice?
There are a couple of different answers here.
A good starting point would be Marc de Graauw's Nobody Needs Reliable Messaging. The basic idea is that the multiple messages have the same business semantics, and therefore consumers can reject duplicates themselves.
That in turn means pushing backwards through your protocol; if we have two copies of an HTTP Request, possibly separated in time, that attempt to create the "same" user, then two instances of the User Management Service handling those requests should end up trying to add semantically equivalent events to the store.
With that property in place, any given consumer can use the semantics of the message to eliminate duplicates.
Event store implementations can help with Conditional Publish. For example, Event Store achieves this by supporting an expected version parameter in the write command.
In a race between competing producers, the two writers will be competing to write to the same expected version of the stream; one producer will succeed, and the second will fail -- the second producer then knows that its locally cached representation of the stream is out of date. It can then refresh its cache, and try processing the message again.
In other words, writes to the event collections are achieved by compare and swap of the tail reference, rather than by "append".
As far as I can tell, in 2017 Kafka doesn't support conditional publish. The exactly once delivery feature in 0.11 doesn't appear to handle that case.
Multiple processes writing to the same event partition may not be what you want. Reasoning about the behavior of a single authority is so much easier. Instead of having multiple instances of the User Management Service sharing the authority to write to a single stream, you might be better served by creating multiple streams, each with a single authority (essentially, each distinct stream has its own leader election).
I don't think I understood the last part 100%, though; would that look somewhat like this: i.imgur.com/b0C2xNV.png?
Yes, in the sense that each user service has its own topic (book of record) for output events. But also, you want to be sure that all events related to a specific entity in your model get written to the same topic. So there would be a piece of logic responsible for ensuring that each command gets handled by the authoritative instance of the user service for that entity.