I spent a week staring at the Caddy and Traefik code bases trying to understand how they were put together. While certainly a bunch of things started making sense (who spends a week staring at a code base?!), there were specific questions I couldn't find answers to.

Eventually I got to an understanding where I could ask a coherent question on Caddy's community forum about Caddy.

Matt (creator of Caddy) was generous enough to give me an extremely solid answer, pointing out certain parts of the stdlib http module that did what I was asking about. It dawned on me that my grasp on how Golang handles HTTP was more tenuous than I thought.

So I set about seeing what I could understand! And I wrote it down. So,here's some ċ̷͍͔o̷̯͓̓̒n̸̨͍̫͒͛t̷̛̫̞̃̀̓̄ë̴͇̜͈̗͉́͗̽ń̸̞̮͈͎t̴̝͉̹̤̖͌͗.

The simplest lil' HTTP server

Golang handles HTTP natively. There's a lot that goes into that - http/1, http/2, h2c, TLS, websockets, trailer headers, upgrading connections - the list goes on!

But we're not there yet. Let's just make a very simple web server:

package main

import (
    "fmt"
    "net/http"
)

func main() {
    http.HandleFunc("/", func(writer http.ResponseWriter, r *http.Request) {
        // Send a 200 response. This is technically
        // superfluous as it implicitly sends a 200
        // when we write any response to the writer
        writer.WriteHeader(200)

        // Here we write "HELLO" to the ResponseWriter
        fmt.Fprint(writer, "HELLO")
    })

    http.ListenAndServe(":80", nil)
}

We used function http.HandleFunc to pass a URI to match, and a "handler function". The handler function takes a http.ResponseWriter which you write response data to, and a http.Request object with the request data.

By default you need to handle GET vs POST (etc) yourself by reading r.Method. Fancy libraries such as gorilla/mux help you there.

The writer.WriteHeader method just takes an HTTP status code. If we omitted it, a 200 response status would have been sent when we wrote anything to the writer, like our "HELLO" string.

The WriteHeader() method is just sending the HTTP response header, e.g. HTTP/1.1 200.

Here's something more interesting: The function passed to HandleFunc actually handles all requests. What we're passing in for a URI is technically a "URI pattern". Patterns ending in a trailing slash / are "rooted subtrees", which is Golang's insufferable way of saying that it'll match the given URI and anything after it.

The URI pattern "/" will handle any URI not otherwise matched. We don't have any other URI patterns defined, so it's a catch-all route!

But why does this work?

I set a catch-all handler, and then just told http to ListenAndServe on some port.

But those are two top-level functions defined in the stdlib http module. There's no obvious connection between those 2 actions! Shouldn't I have had to pass that handler to the server somehow?

// These two top-level functions from http
// don't appear to be "connected" in any
// way at first.
http.HandleFunc("/", func(writer http.ResponseWriter, r *http.Request) { 
    // snip
});

// In fact, we pass nil to the param
// that normally would take a handler
http.ListenAndServe(":80", nil)

It turns out that the http stdlib module has default objects and uses them if we don't define those objects ourself.

1. The http.HandleFunc() function adds a handler to a http.DefaultServerMux object, which is conveniently predefined
2. Deep in http/server.go, a serverHandler{} struct is created. It has a ServeHTTP method on it. This method checks if a handler (the mux) is defined on the Server object. If not, it uses the http.DefaultServerMux

So the simple lil' HTTP server works because of syntactic sugar. There are default objects the http module uses unless you explicitly define them.

Explicitly defining things

Let's take this exact same setup but make it more complicated. For science!

package main

import (
    "fmt"
    "net"
    "net/http"
)

func main() {
    mux := http.NewServeMux()
    mux.HandleFunc("/", func(writer http.ResponseWriter, r *http.Request) {
        writer.WriteHeader(200)
        fmt.Fprint(writer, "HELLO")
    })

    srv := &http.Server{
        Handler: mux,
    }

    ln, err := net.Listen("tcp", ":80")
    if err != nil {
        panic(err)
    }

    srv.Serve(ln)
}

We have more going on here, but with the exact same result. We took the syntactic sugar and made it less sweet.

Previously, the http module added the route and handler function to http.DefaultServerMux.

Here we create a Mux ourselves, and then register our route against it. The HandleFunc method is exactly the same, but one is "global" to the http module and one is on ServeMux objects.

// this:
http.HandleFunc("/", func(writer http.ResponseWriter, r *http.Request) {
    // snip
})

// versus this:
mux := http.NewServeMux()
mux.HandleFunc("/", func(writer http.ResponseWriter, r *http.Request) {
    // snip
})

A Mux is a "HTTP request multiplexor" and is responsible for matching incoming requests against a list of registered routes. It sends requests to the correct handler.

After that, we create an instance of http.Server, passing it the Mux as its Handler.

srv := &http.Server{
    Handler: mux,
}

That's curious, though.

If the ServeMux is a Mux, and we pass handler functions to that Mux, why is the ServeMux object referred to as a Handler within the http.Server object?

Interestingly, ServeMux is actually an http.Handler, meaning it "satisfies" interface http.Handler - it has a ServeHTTP() method on it:

// ServeMux has a method ServeHTTP()!
type Handler interface {
    ServeHTTP(ResponseWriter, *Request)
}

So http.Server's just wants any http.Handler. It doesn't actually need to be a ServeMux, but we usually use one to route specific requests to the code of our choice.

Here's another tidbit: ServeHTTP(http.ResponseWriter, *http.Request) has an equivalant signature to the handler function we passed to mux.HandleFunc(). Suspicious! Let's table that for a hot second, but keep it in mind.

// Our handler:
func(writer http.ResponseWriter, r *http.Request) {
    // snip
}

// Is basically an unnamed `ServeHTTP` method:
type Handler interface {
    ServeHTTP(ResponseWriter, *Request)
}

The ServeHTTP() method will come up a lot, the http module leans on the http.Handler interface pretty hard.

Finally, we create a network listener, and pass that to the server. The server will take the listener and Accept() new connections/data on the defined network socket for HTTP connections (port 80 on all networks, in this case).

So, all of this work boils down to a less-sweet way of doing exactly what our most basic web server did.

Along the way, we learned about the ServeMux, creating http.Server instances, and noticing that the mux is actually a Handler.

We also saw that we can create a network listener ourselves and pass it to our server.

I've hinted that Handlers are sort of interesting. Let's get into that next.