I’ve recently been considering an idea for a new project, and I’ve slowly began to code it in my spare time, when I’m not developing Instahero. The basic idea is a bot that will hang out in companies’ channels, giving them useful commands, logging, and lots of other functionality.

There are various open-source solutions which one can use to do the same thing, but I think there’s also need for a hosted solution where you can just register, enter your room details and get a bot in it in one minute. As a developer, I know my life would be much easier if we had deployment commands, conversions, various searches, bug report notifications, etc available right in the channel where our team spends the day. I’m sure other developers feel that way too, so I started developing Instabot.

Designing the architecture

Starting out, I had to think about how the whole thing would have to be architected. The obvious idea one would start with is to make each bot a separate thread, receive commands from the channel (in this case IRC, but Instabot supports more endpoints and it’s trivial to add more, as I’ll explain), process them and send the result to the channel. This is pretty straightforward, you just spawn a thread whenever a new room is requested, and all state is in the thread.

A slightly better approach is to use a coroutine framework, such as gevent, because threads are too heavyweight for an application which might need to scale to thousands of rooms, with each room simply sitting idle, waiting for network I/O. Coroutines allow you to make microthreads, which are very cheap, and to switch between them more efficiently.

This monolithic approach, however, has certain very significant drawbacks: You will have to stop and restart the bots every time you need to create a new command or make the slightest update, and it’s not very good for the product if the customer sees their bot quit and rejoin (potentially losing messages) every time you make minor updates to your code.

Separating the frontend

The reasonable way to fix this problem would be to separate frontend and backend. After all, you don’t really need the command processing code in the same thread as the actual IRC client, the client is only there to ferry commands back and forth from the channel to the backend for processing. Therefore, a good way to approach it is to have the frontend be a minimal IRC/XMPP/Campfire/HipChat/etc client that will receive messages from the channel, send them to the backend for processing, receive the reply and send it to the channel.

This simplifies our architecture significantly. We no longer need to keep state in the frontend, all it needs to know is the channel, or maybe the channel id, which is specific to the application. A unique id for each channel works well, and is a very simple piece of data the frontend can easily store. Another advantage of a frontend this simple is that you will almost never need to upgrade it (since it doesn’t do very much), and thus it can just run (approximately) forever.

We also need a message queue to ferry messages back and forth, so one might want to use RabbitMQ for this, but my needs are simple, for now, so I’ve decided to go with redis. A simple list will work well, as it has all the components we need to make a message queue:

  • You can push to the front.
  • You can pop from the back (so you have a normal FIFO queue).
  • The coroutine can block forever on popping, consuming no resources, until a message comes in.

I wrote the initial frontend in Python, and it worked well, but I wanted to give Go a shot, both because I heard it had built-in concurrency features and to learn it, so I started rewriting the frontend in Go.

Go frontend specifics

As I said before, Go supports a version of coroutines, which it calls goroutines, which is pretty much standard functions that run concurrently. It also has channels as a first-class method of communication, and they can be used to send or receive messages between functions or goroutines.

With these tools in our arsenal, the architecture of the frontend is pretty straightforward:

  • There is a map of (id -> channel) for the workers, so we can retrieve the messaging channel of a worker by its id.
  • There is an incoming message dispatcher function that receives messages from redis, decodes them, looks at the id of the recipient goroutine and sends it to said goroutine, which forwards it to the channel.
  • There is an outgoing message dispatcher function that receives messages from goroutines and pushes them to the redis queue.

The frontend consists of basically just these three things.

The map

The (id -> channel) map is really very simple:

var workerChannels = make(map[string]chan IncomingMessage)

When a user requests a new bot for their room, the backend sends the room details, along with the id, to the frontend, a goroutine is launched and registered on this channel map. From then on, the incoming message dispatcher can send messages to it by looking its id up in this map.

The incoming message dispatcher

The incoming message dispatcher blocks on the controlling redis queue, until a message is pushed to it. It then pops the message and dispatches it to the appropriate goroutine.

func IncomingMessageDispatcher() {
    client := redis.New("", 1, "")
    for {
        output, _ := client.Blpop([]string{"control"}, 0)
        json.Unmarshal(output.Elems[1].Elem.Bytes(), &message)
        workerChannels[message.Id] <- message
    }
}

There’s nothing really very interesting about this either, the messages are in JSON and are decoded and sent to the goroutine via its channel, as described above. When the goroutine receives the message, it reads it and performs one of various actions, such as joining a channel, quitting, sending messages, and anything else the backend instructed it to do.

The outgoing message dispatcher

The outgoing message dispatcher is another very simple goroutine:

func OutgoingMessageDispatcher() {
    client := redis.New("", 1, "")
    for {
        message := <-outgoingChannel
        b, _ := json.Marshal(*message)
        client.Rpush("messages", b)
    }
}

It loops forever, waiting for messages from the outgoing channel. When it receives a message from a client goroutine (i.e. from the chat room), it encodes it to JSON and pushes it to the appropriate redis queue. The backend then receives the message and evaluates it to see what the user requested.

Wrapping up

This very simple frontend is roughly all that is required for such a service. Due to Go’s concurrency primitives and the way it handles threads, we will probably only ever need this to run in one process and ferry messages back and forth between the channel and the message queue.

As you can see, this approach has many advantages compared to the monolithic architecture above. Restarting/deploying code doesn’t touch the frontend, so the users are never adversely affected. You also don’t need any complex logic there at all, making it very future-proof, easy to reason about and reasonably foolproof.

As always, let me know if you have any suggestions or questions in the comments!