Google
As a follow up to my last post on middleware and monads I'll be going over how the Warden class, as implemented inside Vagrant, uses concepts derived from some of the basic monad combinators to achieve a rescuable middleware stack. As before, this article assumes you have some knowledge of the Haskell language and middleware as defined in the Rack specification/Pep 333. Also, if you've never heard of Vagrant you can check out the documentation and some press.

Quick Recap

For a full treatment of the similarities between function composition and Rack middleware please see the previous article. Function composition and the method call chain created when defining a middleware stack are strikingly similar. Each middleware is called from the previous with the environment object as the only argument. Borrowing from previous examples
app = Rack::Builder.new do
  use Rack::Upcase
  use Rack::Reverse

  run lambda { |env| [200, { 'Content-Type' => 'text/html' }, 'Hello World'] }
end
Which, once its made its way down the call "request" chain, takes the psuedo-code-form:
Rack::Reverse#call <- Rack::Upcase#call <- Proc#call
So, aside from the exposure to external state, and being informed about the subsequent middleware during the initialization it actually looks very much like function composition in haskell
reverse . upcase . \env -> (200, [("Content-Type", "text/html")], "Hello World!")
It boils down to function chaining with a common argument, namely env. However, if a guarantee on error handling is important, PEP 333 doesn't have much to offer. In contrast, taking function composition and making use of a different combinator in place of the (.) composition operator provides an opportunity for error handling abstraction. From the previous article
reverse <=< upcase <=< head <=< \env -> Right (200, [("Content-Type", "text/html")], "")
By lifting the functions into the Either monad and using the monadic composition operator <=<, error condition handling is abstracted outside of the functions operating on the env. The Warden class inside Vagrant was implemented to provide similar functionality by inserting itself into the Vagrant middleware call stack.

Warden

Vagrant, being a command line application unlike Rack, requires different things of its middleware. Particularly when it comes to error handling. For example, when running `vagrant up` it duplicates a base virtual machine. If for some reason there's an issue during this process its best not to leave 500mb+ virtual disk files lying around. Prior to the introduction of the middleware Warden, each middleware would call its own cleanup method and either return back up the call chain when a problem was encountered or mark the env with an error. This meant that each middleware had to contain its own logic for error checking the environment and for dealing with those errors appropriately.
class MiddlewareExample
  def initialize(app, env)
    @next_app = app
  end

  def call(env)
    @next_app.call(env)

    # was the environment tagged with an error by
    # middleware farther down the stack?
    if env.error?
      fixit
    end
  end

  def fixit
    #cleanup
  end
end
To improve on this situation the Warden class was introduced to act, in a similar fashion to the <=< operator, as a middle man inserted by the Builder class. The only requirement (one met before the Warden was created thanks to Mitchell's diligence with well factored code) being that each middleware wishing to perform some cleanup action must implement a separate cleanup method, recover, which the Warden would call when necessary.
class MiddlewareExample
  def initialize(app, env)
    @next_app = app
  end

  def call(env)
    @next_app.call(env)
  end

  def recover
    #cleanup
  end
end
If the example was a pre-Warden Vagrant middleware, the @next_app instance variable would have been an instance of the next middleware. With Warden its actually an instance of the Warden class. To use the illustration from earlier:
Rack::Reverse#call <- Rack::Upcase#call <- Proc#call
Which becomes:
Warden#call <- Rack::Reverse#call <- Warden#call <- Rack::Upcase#call <- Warden#call <- Proc#call
Swapping the monadic composition operator (<=<) into the pseudo-code call chain for the Warden instance makes the inspiration a bit clearer.
<=< Rack::Reverse#call <=< Rack::Upcase#call <=< Proc#call

Implementation

To create the modified middleware stack the Builder class first sends all the middleware classes, lambdas, and procs into the Warden initializer to be finalized (instantiated or wrapped in the case of a lambda/proc):
def initialize(actions, env)
  @stack = []
  @actions = actions.map { |m| finalize_action(m, env) }
end
Next the Builder class calls the Warden instance just as it would normally call the first middleware and the Warden instance tracks which middleware have been run:
def call(env)
  return if @actions.empty?

  begin
    # Call the next middleware in the sequence, appending to the stack
    # of "recoverable" middlewares in case something goes wrong!
    raise Errors::VagrantInterrupt.new if env.interrupted?
    @stack.unshift(@actions.shift).first.call(env)
    raise Errors::VagrantInterrupt.new if env.interrupted?
  rescue

    # Something went horribly wrong. Start the rescue chain then
    # reraise the exception to properly kick us out of limbo here.
    begin_rescue(env)
    raise
  end
end
Lets break this down a little so its clear how this method does the middleware management in the midst of the interupt handling code.
  @stack.unshift(@actions.shift).first.call(env)
@actions.shift pulls the first middleware off the front of the uncalled array and @stack.unshift pushes it onto the front of the rescuable middleware array. Calling first on the result of unshift, the new mutated list, returns the current action which is subsequently called.
  raise Errors::VagrantInterrupt.new if env.interrupted?
The interrupt checks happen on both sides of the middleware management code to handle interrupts before and after calls to the middleware. Any other exceptions raised during the course of execution by the middleware are obviously caught in the same manner and when they are the begin_rescue method is invoked. Here the Warden will begin walking back up the stack of already executed middleware, starting with the most recent, calling the recover method on those that define it:
def begin_rescue(env)
  @stack.each do |act|
    act.recover(env) if act.respond_to?(:recover)
  end
end
While the Warden has added some complexity to the traditional middleware stack the call method on the Warden instance provides a central place to catch exceptions and and begin the rescue process.

But where are the Monads?

The implementation of the Warden class is distinctly imperative. Still, the basic premise of inserting something, be it a combinator or non-middleware instance, between otherwise normal method/function calls remains intact. Its an important addition of functionality without intruding on the original purpose of each middleware. Better still, and much like the Either monad, the Warden "combinator" has imbued the call stack with the chance to recover from a bad state while defining a simple way for the middleware to participate in that process. In fact this one change in Vagrant resulted in several hundred lines of code removal by refocusing each call method definition back towards its original purpose.

Learning from Haskell

Anyone who follows Haskell can see that its fortunes are rising in "the real world". Still, you might be forgiven for thinking its goals too lofty and learning curve too steep for taking the time to learn it. For my part the Warden implementation in Vagrant has been a clear example of what can be gained from spending time learning Haskell. The analogs between function composition and middleware were evident when talking with Mitchell about his middleware implementation in Vagrant. Viewed from that perspective it was natural to look for other techniques and abstractions guided by function composition and combinators that might provide interesting benefits where middleware "composition" was concerned. My goal for the next article will be to build a middleware implementation in Haskell that reflects the same rescue-able behavior. Then I hope to compare its implementation to existing Rack-like software such as Hack and Wai.

Published

18 Oct 2010

Tags