Go (Golang) efficient logic processing in a chat system

Question

In order to build a messaging app, I have followed this example : https://github.com/gorilla/websocket/tree/master/examples/chat

This consists of a Hub, running in a single goroutine in the program, which binds together the Clients (the intermediary between the socket and the Hub).

In this example, each Client (connected user/websocket) runs 2 goroutines : one for reading messages, one for writing. So we end up with a single goroutine for the Hub, and many goroutines for all the connected users in the app (2 per each user)

But this is for an app which all it does is broadcast all the messages to all the connected users. Since my goal is to build a messaging app (similar in functionality to whatsapp, messenger and the likes, where a user has private conversations with a friend(s)), I need to adjust this design accordingly.

So, I've split this into two layers :

1. Connection layer - Hub + Client. This layer is responsible for keeping the connections with the clients (plus everything that it entails - reading/writing messages, encoding/decoding, etc.)
2. Business Logic layer - the "core" / the logic itself. Meaning - communicating with the DBs, be it an SQL or Cache like Redis, in order to insert messages, finding participants in each conversation, etc.

Challenges :

Since the Logic layer has more resource-consuming operations (reading and writing to DBs, finding out which users belong in the conversation so it would know to tell the hub which users in its Clients mapping to send the message to), I'm having hard time to decide where that should lie and how the two layers should interact/communicate with each other?

Because the Hub is central to the app and has 1 goroutines to handle all traffic, I'm assuming it cannot just call "processMessage" in the Service layer, and wait for its response. Otherwise we can potentially reach a state where we have many pending messages and users will start receiving messages with great delay. (Please do correct me if I'm wrong here)

Ideas so far + problems :

So what I thought of doing, is to have another Service goroutine for the app, just like the Hub, which the Hub will run once it's initialized, and then the Hub could send a message for it to process using a channel between them. Then, once the Service is done, it sends back to the Hub's channel something like "hey dear Hub, send this message I gave you, to users: 1, 2, 3".

But here I think I have another problem - since Hub and Service are in different packages, it means I need to have a cyclic dependency, because Hub knows about Service (it forwards a received message for it to process), and Service knows about the Hub (it tells the Hub what message to send to which users after it's done processing the received message from hub).

So it seems that each option has either a performance issue or an architectural/design issue. Or both. Since I'm not an expert in Go, that would be really helpful to learn how to handle this kind of case both efficiently(performance-wise) and correctly (design-wise).

EDIT :

Following Jory Geerts's fantastic answer using one goroutine for the service layer, I have tried to pull together a solution that runs service/logic goroutine for each client instead of one goroutine for the app. But I'm not sure about this solution, and I'm also not sure how to terminate the service gorourtine when a client disconnects.

this is a half-baked code, just to show what I had in mind :

type Service struct {
    ('processing' chan of type messaging.Incoming will come as a parameter in run())

    delivery chan messaging.Outgoing // reference from Hub
    repo *sqlx.DB // pool of postgres connections.
    cache redis.Pool // pool of redis connections.
} 

type Hub struct {
    clients map[*Client]bool
    delivery chan messaging.Outgoing
}

type Client struct {
    hub *Hub
    send chan []byte
    conn *websocket.Conn
    processing chan messaging.Incoming 
} 

and the code itself :

main.go :

func main() {
    ...
    ...

    hub := newHub()
    go hub.run()

    service := logic.NewService(delivery : hub.delivery, repo: postgresPool, cache: redisPool)

    http.HandleFunc("/ws", func(w http.ResponseWriter, r *http.Request) {
        serveWs(hub, service, w, r)
    })
}

func serveWs(hub, service, w, r) {
    processing := make(chan messaging.Incoming)

    go service.run(processing)

    client := &Client{hub, processing, conn, send: make(chan []byte, 256)} 
    go client.readPump()
}

client.go (package connection):

func readPump() {
    ...
    defer func() {
        c.conn.Close()
        close(c.processing)
    }()

    for {
       msg := c.socket.ReadMessage()
       c.processing <- msg
    }
}

service.go (package logic) :

func (s *Service) run(processing <-chan messaging.Incoming) {
    for {
        select {
        case incomingMessage := <-processing:
            result := processMessage(incomingMessage)
            s.delivery <- result
    }
}

hub.go (package connection) :

func (h *Hub) run() {
    for {
        select {
        case result := <-h.delivery:
            clients[result.userId].send <- result.msg
        }
    }
}

Answer 1

Lets start with your specific concern about a cyclic dependency. In Go, you can easily solve this by defining interfaces in the dependent packages had having those interfaces only expose types from outside their package.

That could look something like this:

package messaging

type Incoming struct {}

type Outgoing struct{}

type ProcessingResult struct {}

type DeliveryResult struct {}

package hub

type service interface {
  Process(m messaging.Incoming) messaging.ProcessingResult
}

func New() *hub {}

type hub struct {}

func (hub) Deliver(m messaging.Outgoing) messaging.DeliveryResult {}

package service

type hub interface {
  Deliver(m messaging.Outgoing) messaging.DeliveryResult
}

func New() *service {}

type service struct {}

func (service) Process(m messaging.Incoming) messaging.ProcessingResult {}

So now, both service and hub can use each other, without knowing each other. The only problem that is left is constructing them with a reference to one another. One option for that would be to not have a New() function that sets up a ready-to-use instance, but give service a method that receives hub (say, func (service) ProcessFor(h hub)) - or the other way around, or both.

Another option (which is a better match Go, in my opinion) would be for them to not know each other at all. Instead you create two channels in your main(). One that hub writes to and service reads from and one the other way around. In addition, each time your push a thing to a channel for which you need to know if it was handled successfully, you also pass along a single-use channel to receive the result on.

That could look something like this:

package main

func main () {
    processing := make(chan messaging.Incoming)
    delivery := make(chan messaging.Outgoing)

    go hub.New(processing, delivery)
    go service.New(processing, delivery)
}

package hub

func New(processing chan<- messaging.Incoming, delivering <-chan messaging.Outgoing) {

    // whenever the hub gets a message from somewhere
    msg := new(messaging.Incoming)
    processing <- *msg
    result := <- msg.Result

    // delivering messages
    for {
        msg := <-delivering
        // do your delivery
        msg.Result <- messaging.DeliveryResult{}
    }
}


package service

func New(processing <-chan messaging.Incoming, delivering chan<- messaging.Outgoing) {

    // whenever the service needs a message to be delivered
    msg := new(messaging.Outgoing)
    delivering <- *msg
    result := <- msg.Result

    // processing things the hub needs to have processed
    for {
        msg := <-processing
        // do your delivery
        msg.Result <- messaging.ProcessingResult{}
    }
}

This approach gives you a number of benefits. For one, things are more loosely coupled, which is always a good thing. In addition, it becomes easier to scale things. If the service is to slow, just start two. If the hub is to slow, just shard it (ok, that one is a little bit more complicated).

A downside of this is that reasoning about your code becomes a bit more complex and unit tests become a little bit more complicated.

For higher throughput, a next step can be to let both hub and service start an ad-hoc inline goroutine around the async communication - that way, other messages won't have to wait. This is, again, more complicated though.