I'm working on a Kafka consumer in C# (using the Confluent C# lib).. and practically all of the documentation/examples online leverage console apps to do the consuming.
So, this is fine for logic validation and general playing about but clearly not suitable for a production workload. Is there a recommended deployment model for running Kafka consumers especially in Azure environment?
Technically, i guess it needs to run as a background service.. that connects to, and then acknowledges items on each topic.. i'm leaning towards Azure Service Fabric as i need VNet integration too, although my experience with it in the past is that it's quite a lot of "cruft" for workloads that aren't really that heavy..
Another option i figured would maybe be Docker/AKS, but again.. this feels way too heavy for what this consumer is doing.. basically reading messages, and routing them through to Azure Event Hubs..
Thoughts on alternatives? Bear in mind that i need to be able to control the VNet connectivity as the Kafka cluster(s) are in the 10.xxx.xxx.xxx range.. Ty!