An Introduction to Webprogramming in Clojure - Ring and Middleware

Ring is where web-development with Clojure begins. It is the underlying technology on which many other frameworks rely on. This article features an introduction to Ring. I assume that the reader has a basic understanding of the HTTP-protocol, as well as some knowledge in programming Clojure and working with Leiningen.

Ring in its Ecosystem

Ring is a library with capabilities roughly equivalent to Rack in Ruby or WSGI in Python. Both provide a reasonably pleasant HTTP interface. They also abstract the access to various web-servers for their respective platform.

Ring is also a Clojure wrapper around the Java HTTP Servlet API. The class HttpServlet as well as the interfaces HttpServletRequest, and HttpServletResponse are relevant. Ring doesn't provide an own implementation of the Servlet API. We can use any of the available open-source or comercial implementations. The Jetty servlet container stands out; mainly because it is very easy to use from the context of a development or testing environment¹.

On the other hand, the underlying specification of Ring is at least as important then the actual implementation. It embodies a clever way how the pieces of a webserver can be composed on the basis of higher order functions. We will see how some of those fit together in the following sections.

Ring Quickstart

We create a new project of the name "quickstart" by invoking lein new quickstart. We edit the file project.clj such that it includes all required dependencies as follows:

(defproject quickstart "1.0.0-SNAPSHOT"
                :description "FIXME: write description"
                :dependencies [[org.clojure/clojure "1.3.0"]
                               [ring "1.1.5"]])

We can now fetch and install the dependencies with lein deps². Now, let us edit src/quickstart/core.clj such that it reads as the following:

(ns quickstart.core)
              
              (defn handler [request]
                {:status 200
                 :headers {"Content-Type" "text/plain"}
                 :body "Hello World!"})

We open a REPL with lein REPL³, we load and refer the adapter with (use 'ring.adapter.jetty) as well as the code from our quickstart project (use 'quickstart.core), and finally start the server with (run-jetty handler {:port 8383}). We will seed the body as defined previously when we open http://localhost:8383 in a browser.

The Handler and a closer look at Requests and Responses

The request is passed on to the handler as a clojure hash which contains keywords and simple textual information⁴. Let us rewrite our handler, such that it returns a textual description of the request.

(defn handler [request]
                { :status 200
                  :headers {"Content-Type" "text/plain"}
                  :body (str "Request:\n\n"
                             (with-out-str (clojure.pprint/pprint request)))})

Dynamic Reloading

We restart the REPL and procede as before. However, we use (run-jetty #'handler {:port 8383 :join? false}) to start the server this time. This enables us to change the code "on the fly", and afterwards reload it with (use :reload 'quickstart.core). The changes will be reflected immediately, without need to restart the server or even ending the REPL! Other variants to achieve the same behavior are discussed in ring's wiki Interactive Development.

Request Parameters

We can now see the full and rather lengthy request. The information most relevant for forming the response is the uri, the query-string and the request-method. We can easily influence the first two, e.g. by typing http://localhost:8383/blahblah?somekey=somevalue into the address bar of the browser and hitting enter. Try to find the relevant information in the output.

Unfortunately, we still can't inspect the response headers, which can be quite important during development.

Sending Requests and Receiving Responses with clj-http

We will use the clj-http library to send and receive requests for developing purposes from the REPL. Alternative tools can be browser-embedded, like the chrome development tool or firebug together with the JQuery library⁵; or possibly a command line tool like curl⁶. The first is the most portable and dependt variant, so let us add it to the project dependencies⁷ :dependencies [ [clj-http "0.5.3"], ....

Now, let us descend into the project's folder and start a second REPL. We, load the http-client with (require '[clj-http.client :as client]), it is available as client afterwards. We can now send a request with the get function, and display the response too:

user=> (pprint (client/get "http://localhost:8383/foo/"))
              {:trace-redirects ["http://localhost:8383/foo/"],
               :request-time 37,
               :status 200,
               :headers
               {"content type" "text/plain",
                "connection" "close",
                "server" "Jetty(7.6.1.v20120215)"},
               :body
               "Request:\n\n{:ssl-client-cert nil,\n :remote-addr \"127.0.0.1\",\n :scheme :http,\n :request-method :get,\n :query-string nil,\n :content-type nil,\n :uri \"/foo/\",\n :server-name \"localhost\",\n :headers\n {\"accept-encoding\" \"gzip, deflate\",\n  \"connection\" \"close\",\n  \"user-agent\" \"Apache-HttpClient/4.2.1 (java 1.5)\",\n  \"content-length\" \"0\",\n  \"host\" \"localhost:8383\"},\n :content-length 0,\n :server-port 8383,\n :character-encoding nil,\n :body #<HttpInput org.eclipse.jetty.server.HttpInput@48268a>}\n"}

The response contains the status code 200, a few headers, and the body as it is prepared by our handler. The latter isn't printed out nicely this way. But this can be improved with (println (:body (client/get "http://localhost:8383"))).

Inference

Let us try to request with the HEAD verb : (pprint (client/head "http://localhost:8383")). The response is very similar. The pronounced difference to GET is that the body is missing. This is in accordance with the specification. However, it might be a surprise since we didn't take any action for this to happen when we defined the handler. Suppressing the body is handled upstream. It already happens in the Java servlet container and not even in Ring itself⁸. The key observation is that modification of the response can happen after we created it and passed it along.

Dispatching

Next, we request with the DELETE verb: (pprint (client/delete "http://localhost:8383/foo")). We retrieve essentially the same respond as with GET, and in particular a 2xx respond code, meaning that the request has been successfully processed. This is certainly not what should happen since no resource "foo" has, or is going to bee erased. Let us fix this.

(ns quickstart.core)
              
              (defn respond-to-get [uri & more]
                { :status 200
                  :headers {"Content-Type" "text/plain"}
                  :body (str "Hello, you requested " uri)})
              
              (defn respond-to-other []
                { :status 405 })
              
              (defn handler [request]
                (condp = (:request-method request)
                       :get (respond-to-get (:uri request) (:query-string request))
                       :head (respond-to-get (:uri request) (:query-string request))
                       (respond-to-other)))

The above code dispatches on the http verb, either to the function respond-to-get or respond-to-other respectively. We send GET or HEAD requests to this function (see the previous section on why we can procede as such with HEAD) and all others to respond-to-other.

The function respond-to-other will send the status code 405, meaning that requesting this method is not allowed. The function respond-to-get is just a slight variation of our previous response.

Dispatching on the verb is most likely second step after dispatching on the uri, and possibly on the request-query too. We will discuss how to dissect the request-query string into a more accessible structure by the use of middleware in the following section.

Ring Middleware

One composable way to add or modify the information of and response is to wrap the handler by an higher order function. Such a function takes the (original) handler as its first argument, followed by possible further arguments. It returns a handler function.

Adding a Custom Header

An example that adds an additional header follows below:

(defn add-app-version-header [handler]
                (fn [request]
                    (let [resp (handler request)
                          headers (:headers resp)]
                      (assoc resp :headers
                             (assoc headers "X-APP-INFO" "MyTerifficApp Version 0.0.1-Alpha")))))

We add this function to quickstart/core. We need to restart the REPL this time. We are going to wrap the new function around the existing handler and therefore start the server with (run-jetty (add-app-version-header handler) {:port 8383 :join? false}). The additional header is included in response if requested via get or head as indicated previously:

user=> (pprint (client/get "http://localhost:8383/foo/"))
              {:trace-redirects ["http://localhost:8383/foo/"],
               :status 200,
               :headers
               {"x-app-info" "MyTerifficApp Version 0.0.1-Alpha",
                "content-type" "text/plain;charset=ISO-8859-1",
                "connection" "close",
                "server" "Jetty(7.6.1.v20120215)"},
               :body "Hello, you requested /foo/"}

Dissecting the Request-Query

A middleware wrapper can perform two things:

1. It can modify the response before it is given back up to the servlet, and
2. it can modify the request before it is given down to the handler.

We look at the second mechanism in this section. The request-query is available as as simple string in the request headers, which is an inconvenient format to work with. The middleware wrap-params will dissect the query and put the result into a hash structure. Here is how we are going to use it:

(ns quickstart.core
                (:use  [ring.middleware.content-type]
                       [ring.middleware.params]))
              
              (defn handler [request]
                { :status 200
                  :body (str "Request:\n\n"
                             (with-out-str (clojure.pprint/pprint request)))})
              
              (defn add-app-version-header [handler]
                "This Function adds a very meaningful header to the response."
                (fn [request]
                    (let [resp (handler request)
                          headers (:headers resp)]
                      (assoc resp :headers
                             (assoc headers "X-APP-INFO" "MyTerifficApp Version 0.0.1-Alpha")))))
              
              (def app
                (-> handler
                    (add-app-version-header)
                    (wrap-params)
                    (wrap-content-type "text/plain")))

We did not only use wrap-params but also kept our own wrapper and added the wrap-content-type middleware for demonstration. The following shows the response after we have restarted the webserver with app given as the handler.

user=> (println (:body (client/get "http://localhost:8383/foobaz?somekey=somevalue")))
              Request:
              
              {:ssl-client-cert nil,
               :remote-addr "127.0.0.1",
               :scheme :http,
               :query-params {"somekey" "somevalue"},
               :form-params {},
               :request-method :get,
               :query-string "somekey=somevalue",
               :content-type nil,
               :uri "/foobaz",
               :server-name "localhost",
               :params {"somekey" "somevalue"},
               :headers
               {"accept-encoding" "gzip, deflate",
                "connection" "close",
                "user-agent" "Apache-HttpClient/4.1.2 (java 1.5)",
                "content-length" "0",
                "host" "localhost:8383"},
               :content-length 0,
               :server-port 8383,
               :character-encoding nil,
               :body #<HttpInput org.eclipse.jetty.server.HttpInput@22de9e>}

Note the presence of the :query-params and :params keys. More details are given in the middleware params documentation.

Where do go from Here

References to the comprehensive documentation of ring have been given throughout the article. The book Clojure Programming contains a very good overview to web-programming with clojure and ring in particular.