Articles Go HTTP Servers for Hackers Courses CloudCasts Chipper CI

VIII. Multiple Listeners

So far the proxy only listens for incoming requests on port 80. That's dumb.

It would be way more useful if we could listen for both http:// and https:// (TLS) connections, or even listen on custom ports.

To do that, we need to create multiple listeners, where each listener is a network socket to use to listen for requests (such as 0.0.0.0:443).

We'll run srv.Serve(listener) once for each listener defined.

Expanding on Listeners

Previously we started an HTTP server by just passing it a string address. This address is converted to a net.Listener and used to listen for connections:

srv := &http.Server{Addr: ":80", Handler: r.proxy}

We want to be able to listen on multiple addresses of our choosing.

To accomplish this, we can build a Listener concept into the code.

Let's start by defining a new type: type Listener struct. This will contain things we need to listen and serve http:// or https:// connections.

We can create a new file listener.go for this - Here's the updated project layout:

.
├── go.mod
├── go.sum
├── main.go
└── reverseproxy
    └── listener.go
    └── reverseproxy.go

File listener.go contains a new Listener struct and some methods to make it convenient to use:

package reverseproxy

import "net"

type Listener struct {
    Addr    string
    TLSCert string
    TLSKey  string
}

// Make creates a net.Listen object to be used
// when starting an http server
func (l *Listener) Make() (net.Listener, error) {
    return net.Listen("tcp", l.Addr)
}

// ServeTLS tells us if we should be serving TLS
// connections instead of unsecured connections
func (l *Listener) ServesTLS() bool {
    return len(l.TLSCert) > 0 && len(l.TLSKey) > 0
}

The Listener struct contains an Address compatible with net.Listen(), and optional fields for a TLS certificate/key.

Method Make() generates a net.Listener for us (or an error, if our address Addr is invalid).

Method ServesTLS() returns a boolean - essentially just saying this listener is meant to be used for TLS certificates if the certificate and key fields are used.

Now that we've abstracted the concept of a Listener, we can use it!

Implementing Multiple Listeners

We need to update our ReverseProxy object to make use of multiple listeners.

Let's see what that looks like. Here are the changes within reverseproxy.go:

type ReverseProxy struct {
    listeners []Listener
    proxy     *httputil.ReverseProxy
    servers   []*http.Server
    targets   []*Target
}

// AddListener adds a listener for non-TLS connections on the given address
func (r *ReverseProxy) AddListener(address string) {
    l := Listener{
        Addr: address,
    }

    r.listeners = append(r.listeners, l)
}

// AddListenerTLS adds a listener for TLS connections on the given address
func (r *ReverseProxy) AddListenerTLS(address, tlsCert, tlsKey string) {
    l := Listener{
        Addr:    address,
        TLSCert: tlsCert,
        TLSKey:  tlsKey,
    }

    r.listeners = append(r.listeners, l)
}

// Start will listen on configured listeners
func (r *ReverseProxy) Start() error {
    r.proxy = &httputil.ReverseProxy{
        Director: r.Director(),
    }

    for _, l := range r.listeners {
        listener, err := l.Make()
        if err != nil {
            // todo: Close any listeners that
            //       were created successfully
            //       before one returned error
            return err
        }

        srv := &http.Server{Handler: r.proxy}

        r.servers = append(r.servers, srv)

        // TODO: Handle unexpected errors from our servers
        if l.ServesTLS() {
            go func() {
                if err := srv.ServeTLS(listener, l.TLSCert, l.TLSKey); err != nil && !errors.Is(err, http.ErrServerClosed) {
                    log.Println(err)
                }
            }()
        } else {
            go func() {
                if err := srv.Serve(listener); err != nil && !errors.Is(err, http.ErrServerClosed) {
                    log.Println(err)
                }
            }()
        }
    }

    return nil
}

Let's review what's going on!

Our ReverseProxy struct gets 2 new properties

  1. One is a slice of Listener objects.
  2. The other is a slice of *http.Server objects.

Each listener needs its own HTTP server to accept and handle connections, so we need one of each per network socket we're listening on.

A network socket is a network interface (address) + port combo being bound to in order to listen for connections.

We add two methods (which is a bit more consistent with the stdlib's HTTP methods) - one for regular connections, and one for TLS connections. These just add to our slice of listeners.

The Start() method used to run this:

srv := &http.Server{Addr: ":80", Handler: r.proxy}
srv.ListenAndServe()

But now we have multiple listener objects! We'll need a server for each listener, and we'll use the Serve / ServeTLS methods instead of ListenAndServe.

Additionally, ListenAndServe() / Serve() / ServeTLS() are all blocking. That means these all need to run in a goroutine, so they can run concurrently.

For each listener, we create a http.Server, passing it our http.ReverseProxy handler. Each server listens on our given address defined by our Listener objects. TLS listeners get the TLS treatment.

We also handle (which is to say, "log") errors here, where as before we just ignored them.

Here's a fun fact: the Serve methods always return an error. If the error is ErrServerClosed, it just means the server was shutdown properly and will no longer accept new connections. All other errors are "true" errors.

Our Start() method now just returns an error if an unexpected error is returned. However, if the Serve methods return an unexpected error, we just log them.

I didn't go through the hoops of using an error channel to handle those errors (yet?).

Running the Server

We can now update our main.go file to try to run this.

package main

import (
    "github.com/fideloper/someproxy/reverseproxy"
    "github.com/gorilla/mux"
)

func main() {
    proxy := &reverseproxy.ReverseProxy{}

    // Let's assume we have 2 backends to proxy to
    // localhost:8000 (for our fun API requests)
    // and localhost:8001 (for everything else)

    // Match requests to "fid.dev/api" and "fid.dev/api/*"
    r := mux.NewRouter()
    r.Host("fid.dev").PathPrefix("/api")
    proxy.AddTarget("http://localhost:8000", r)

    // Catch-all for all other requests
    proxy.AddTarget("http://localhost:8001", nil)

    // Listen for http://
    proxy.AddListener(":80")

    // Listen for https://
    proxy.AddListenerTLS(":443", "keys/fid.dev.pem", "keys/fid.dev-key.pem")

    if err := proxy.Start(); err != nil {
        log.Fatal(err)
    }
}

Wait! TLS keys! I used mkcert to generate some TLS keys (and a local CA to authenticate them) for me. The domain I used is fid.dev. I edited my /etc/hosts file so fid.dev pointed to 127.0.0.1.

I should probably have just used something like fid.local or the more grim fid.localhost. Oh well!

In any case, mkcert Just Worked™ for me as described in its readme (on MacOS Monteray). I placed the keys it generated in a new directory named keys.

We gotta fix a thing

OK, if you try to run this....it will just exit immediately.

Remember how we ran our servers in goroutines? Yeah, there's nothing blocking anymore, so the program just ends (and goroutines get shut down).

We need a way to keep our program running. To do that, we can listen (and wait) for an interrupt signal (SIGINT), which is roughly Windows/Mac/Linux compatible.

This gives us the ability to block (keep the servers running) until we hit ctrl+c. Based on some random Stack Overflow answer, os.Interrupt is the only signal that will work on Windows.

With some additions, main.go now looks like this:

package main

import (
    "github.com/fideloper/someproxy/reverseproxy"
    "github.com/gorilla/mux"
)

func main() {
    proxy := &reverseproxy.ReverseProxy{}

    // Let's assume we have 2 backends to proxy to
    // localhost:8000 (for our fun API requests)
    // and localhost:8001 (for everything else)

    // Match requests to "localhost/api" and "localhost/api/*"
    r := mux.NewRouter()
    r.Host("localhost").PathPrefix("/api")
    proxy.AddTarget("http://localhost:8000", r)

    // Catch-all for all other requests
    proxy.AddTarget("http://localhost:8001", nil)

    // Listen for http://
    proxy.AddListener(":80")

    // Listen for https://
    proxy.AddListenerTLS(":443", "keys/fid.dev.pem", "keys/fid.dev-key.pem")

    if err := proxy.Start(); err != nil {
        log.Fatal(err)
    }

    // Shutdown when we receive Ctrl+c (interrupt)
    c := make(chan os.Signal, 1)
    
    // We'll accept graceful shutdowns when quit via SIGINT (Ctrl+C)
    // SIGKILL, SIGQUIT or SIGTERM (Ctrl+/) will not be caught.
    signal.Notify(c, os.Interrupt)

    // Block until we receive our signal.
    <-c
}

Now our program will run indefinitely until we send it SIGINT (interrupt).

When we hit ctrl+c, the server will shut down. It will also cut off any connections! We'll next see how to do graceful shutdowns.