DevTalk.net

ActiveMesa's software development blog. MathSharp, R2P, X2C and more!

A Simple Web Server in F#

with one comment

Intro

Every once in a while we all suffer from the Not Invented Here syndrome. So do I. This time, despite of being able to use either one of available web servers or previously written ASP.NET MVC-like web serving framework, I decided to write my own.

My requirements are that the server should serve JSON, provide meaningful URLs, be easy to use (anything that you write is by design easy to use for you), it should be hostable in a simple console app, should not use reflection as ASP.NET MVC does, and it should be written in F#. Because “not-written-in-F#” is a fundamental flaw of most existing systems I know of.

Design

Since this project is going to be a quick and dirty prototype of a future system – let’s use HttpListener as a serving mechanism. It is pretty straightforward and easy to use. Ok. What’s next?

Any kind of ‘web serving’ starts with a URL. Those who tried Django already know how URL handling is performed there. For others, here are the basics. There is a single dictionary (I mean something like System.Collections.Generic.IDictionary here) that maps a URL to the function that should handle requests to that URL. I’m oversimplifying but that’s enough for our purposes. The request handling function should get something in and return back our JSON as a string, which then will be wrapped in a HTTP response and sent back to the user agent. The only question now is what should be passed to the handler function. If we were writing a real web server, it would require us to pass the whole response context into the function, but we want some REST here, therefore the requested URL would be enough. In Django, they use regular expression match groups to obtain arguments for handler functions from the requested URL. Why not do so as well? Let’s create a simple URL mapping for clarity.

[("^/hello/([a-zA-Z]+)", helloNameHandler);]

The above means that when a URL matching the pattern is requested, we should take the URL substring containing a person’s name and pass it to the handler function helloNameHandler. Let’s not forget about the URL query string. We’ll parse it as well and pass to the handling function as a list of (string * string) pairs.

Implementation

Enough said, lets start implementing. First of all we’ll describe all the types already discussed:

type RequestData =
  // first list is url substrings, second list is query string pairs
  | RequestData of string list * (string * string) list
 
type ResponseData =
  | Json of string
 
type HandlerFunction = (RequestData -> ResponseData)
type DispatchEntry = (string * HandlerFunction)
type DispatchEntryEx = (Regex * HandlerFunction)

Since we’re using regular expressions for our patterns we might have some speed gain if they are compiled before usage, hence the DispatchEntryEx type.

let makeDispatcher (pattern, handler) =
  let fullPattern = sprintf "%s/?.*" pattern
  let rx = Regex(fullPattern, RegexOptions.Compiled)
  (rx, handler)
let prepareDispatchers (table:DispatchEntry list) : DispatchEntryEx list =
  List.map makeDispatcher table
module Query =
  let decode : string -> string =
    HttpUtility.UrlDecode
  
  let processParams (s:string) =
    if String.IsNullOrEmpty(s) then
      []
    else
      let s' = s.TrimStart([|'?'|])
      s'.Split([|'&'|])
      |> Array.map (fun pair -> pair.Split([|'='|]))
      |> Array.map (fun  [|h;t|] -> (h, t))
      |> Array.toList
  
  let processPath (rx:Regex) (s:string) =
    rx.Match(s).Groups.Cast<Group>()
    |> Seq.skip(1) // skipping the overall group
    |> Seq.map (fun g -> g.Value)
    |> Seq.toList

Nothing interesting here: we are creating a dispatch table with compiled regular expressions and two functions to provide parameters to our handler functions. Now we are ready to implement a dispatch function, but first we should define its type.

The HttpListener class provides a GetContext() method of type HttpListenerContext that in turn has the Request and Response properties. Processing a request means reading from context.Request and writing something back into context.Response instance. That said, our dispatch function should have the following type: DispatchEntryEx list -> HttpListenerRequest -> HttpListenerResponse -> unit. It does not return any value, because the actual result is written directly to the response as a side effect (We could purify the approach if we wanted to, but not this time).

let dispatch (ds:DispatchEntryEx list) request response =
  let (!!) = Query.decode
  let absPath = !! request.Url.AbsolutePath
  let queryString = !! request.Url.Query
  
  printfn "(II) '%s' requested..." <| !! request.RawUrl
  
  let (rx, handler) =
    ds |> Seq.find (fun (r, _) -> r.IsMatch(request.Url.AbsolutePath))
  
  let path = Query.processPath rx request.Url.AbsolutePath
  let query = Query.processParams queryString
  
  let resultBytes =
    match handler (RequestData (path, query)) with
    | ResponseData.Json json ->
      let callback = Query.get "callback" "callback" query
      json |> Encoding.UTF8.GetBytes
  
  response.ContentLength64 <- int64 resultBytes.Length
  response.OutputStream.Write(resultBytes, 0, resultBytes.Length)

Yet again, nothing complex. The function does exactly what it supposed to. First it searches for the matching handler in dispatch table, then parses the request URL path and query string, after that applies handler function and finally writes everything back to response stream.

The last thing we should implement before we could test our server, is server initialization function.

let start (prefix:string) (table:DispatchEntry list) =
  let listener = new HttpListener()
  listener.Prefixes.Add(prefix)
  listener.Start()
  
  let tableEx = prepareDispatchers table
  
  let requestProcessor =
    let processContext (ctx:HttpListenerContext) = async {
      dispatch tableEx ctx.Request ctx.Response
      ctx.Response.Close()
    }
    async {
      while true do
      try
        let ctx = listener.GetContext()
        processContext ctx |> Async.Start
      with
      | ex -> printfn "(EE): %s" ex.Message
    }
  
  requestProcessor
  |> Async.StartAsTask
  |> ignore
  
  {new IDisposable with
  member this.Dispose () =
  listener.Close()
  }

Please note the usage of async and Async.Start with the processContext function. Without it the whole server will be able to only process requests one after another. HttpListener.GetContext blocks execution until the request is received, right after that we start a new processing thread with Async.Start, which returns immediately and the while loop continues to the next call of GetContext leaving context processing to a separate thread.

Another thing to note is the return type of the start function. It returns IDisposable instance which will stop the listener when disposed, so we don’t have to expose the fact that we use HttpListener to the user still having some control over listeners lifetime.

Some improvements

The code above seems to work (at least it should), but then we realize that JSON is not the only thing we’d like to be able to serve. We would also need some binary data such as images. To support this we have to modify the return type of our handler function. Thanks to discriminated unions, it won’t require too many changes in the code.

There are actually only two places in the whole code that are subjected to change in order to support serving images. First – the ResponseData type:

type ResponseData =
| Binary of byte[]
| Json of string

And second, the dispatch function:

let dispatch (ds:DispatchEntryEx list) request response =
//...
let resultBytes =
match handler (RequestData (path, query)) with
| ResponseData.Json json -> json |> Encoding.UTF8.GetBytes
| ResponseData.Binary bytes -> bytes
// ...

That’s all! We simply added support for a new case to all places where ResponseData is used. No need to change already developed handler functions.

Possible further improvements

Lets imagine that once this “web server” evolves into something more complex and we would need to support “areas” of site, where each area will process URLs separately so that dispatch table will become unbearably long and uncomfortable to use. F# discriminated unions can help us here as well, simply change the HandlerFunction into something like type UrlHandler = | HandlerFunction of (...) | SiteArea of (...) etc. Django also has this feature but here we don’t use dynamic types.

It is also possible to use some templating libraries (or view engines) to render responses from handler functions. This will require just another case in ResponseData type and a slight change dispatch function.

Conclusion

In this article we developed a dead simple yet working web serving solution that can be used to prototype URL base REST services to be hosted in your application. It has not been properly tested and surely has limitations, but the development process showed that it is possible and pretty easy to implement a web serving solution in a .NET environment without using reflection or dynamic typing.

Written by Maxim Moiseev

January 17th, 2011 at 8:02 pm

Posted in FSharp