Scratching the Itch

A couple of weeks ago I found an article by Ryan Day about implementing the Nasdaq ITCH protocol in python. I was really excited about this because I had always thought the barriers to entry for writing a program relating to the stock market would be very high. Turns out writing an application for stock markets is not the barrier, it’s getting the live data needed to be able to make money with it.

I wanted to use RabbitMQ because I’ve been hearing a lot of great things about it. It has a really nice web interface you can use to check on the status of your queues and see how many messages are passing through. Also, since I had used MassTransit in the past with MSMQ, and MassTransit now supports RabbitMQ, I figured it’d be a breeze to setup.

RabbitMQ was very easy to get up and running, along with the plugin for the web interface. It turns out I don’t remember as much as I thought I did about MassTransit, but Joshua Arnold was able to help me get setup pretty quick and I had messages flowing through RabbitMQ in what seemed like no time at all.

The message rate is very slow… I realized that I’m only able to publish about 3,000 messages per second to the message queue. After taking MassTransit out of the mix and publishing directly to RabbitMQ, that number raises to about 10,000 messages per second. If I don’t publish any messages at all and only parse the data from the file, I can parse about 450,000 messages per second. So, RabbitMQ and MassTransit are causing a bottleneck.

Consuming messages is even slower. I got MassTransit to pull messages off the queue at a top speed of about 1,200 messages per second. Way too slow to keep up with a trading day, where I think I would need to be able to handle at least 20,000 messages per second in order to keep up with bursts of traffic.

After doing a little digging, I found a nice comparison of some different queues. It sounds to me like ZeroMQ would be a perfect fit for this project. If I have some free time I’m going to give it a shot and see if it performs as well as they say.

Take a look at the code and let me know what you think: http://github.com/RexMorgan/Itch

FubuMVC: Redirect and Transfer

Note: As with everything in an open source project (and especially one which is pre-1.0), things may change. I’ll attempt to keep this article up to date or link to updated articles if things change with the usage of FubuContinuation.

Sometimes instead of simply rendering a view from your action, I need to redirect the user to another URL, or call another action.

In this post I’m going to explain how FubuMVC allows you to perform these actions. In a multi-tenant application I’m working on, I handle all default requests to many domains with a single action in my FubuMVC application. I want to choose which behavior chain to run, depending on the URL that the user has browsed to.

An example of this, just to make sure I’m explaining this properly, is Tumblr. Browsing to tumblr.com should display the home page for Tumblr (sign up form, information about why Tumblr is great, etc). However, browsing to {username}.tumblr.com should display the posts by that particular user. (I honestly don’t know if this is handled by a single piece of code on Tumblr, but I’d like to mimic this, with a single piece of code, in my project.)

To start, I’m going to create a FrontLoaderEnpoint that all default requests will go through.

In my FrontLoaderEndpoint, I need to check what URL the user is coming from. In order to do this, I have a property called HTTP_HOST on my input model. FubuMVC will automatically fill in the information from the ServerVariables collection. (I hate to just leave it as this “just works”, but model binding will be the topic of another blog post.)

Now I’m taking the HTTP_HOST from the input model and attempting to look up the user that the domain belongs to.

If there is a user that is using the domain, then I’m going to go ahead and transfer the request to the endpoint that handles the user specific domains. I tell FubuMVC that I want to transfer the user by returning FubuContinuation.TransferTo. The TransferTo method takes in the input model of the action that you want to call.

If there’s not a user found for the domain, I’m going to redirect the user to the home page (which allows new users to sign up, etc). To redirect the user I return FubuContinuation.RedirectTo and, just like the TransferTo method, pass in the input model of the action I want to redirect the user to.

How does it work? 

The way this works is RedirectTo and TransferTo are both static methods on the FubuContinuation class, which return a FubuContinuation object. In the constructor of the FubuContinuation object, it takes in the ContinuationType (Redirect, Transfer, or NextBehavior) and an action that should be called on the IContinuationDirector.

The IContinuationDirector is what actually does all of the work, by calling RedirectToUrl on the IOutputWriter if you’re doing a redirect (just like I did in Step 2 of FubuMVC: Authentication.), or building a partial from the IPartialFactory and invoking it, if you’re doing a transfer. The action to call is resolved using the input model that you provided when you called FubuContinuation.RedirectTo / FubuContinuation.TransferTo. Since we’re using the thunderdome pattern, each of our actions are uniquely defined by the type of the input model.

Internally, FubuMVC has a built-in convention which gets run. This convention checks for any actions that have a FubuContinuation as an output type. If it finds one, it adds an internal behavior called ContinuationHandler directly after the action call.

When the ContinuationHandler gets called, it pulls the FubuContinuation that was set in my action call from the IFubuRequest and calls Process on it, passing into it, “this” (the ContinuationHandler being called.) The Process method calls the action that was passed into the FubuContinuation constructor and passes int he ContinuationHandler that’s being run. (Note: ContinuationHandler implements IContinuationDirector.)

Bonus!

If you want to redirect the user to a URL of an outside site, just pass the URL directly into FubuContinuation.RedirectTo. The ContinuationHandler has a mechanism in place to redirect directly to a URL if it’s provided as a string.

FubuMVC: One Model In, One Model Out

A pattern that’s regularly used with FubuMVC is the one model in, one model out pattern. This is also known as OMIOMO and, my personal favorite, the “thunderdome” pattern. It took me a little while to understand why this pattern is in place and seems to be a gotcha when I come back to FubuMVC after an extended period of time. I’m going to write about why I use this pattern, to document it for both others, and my future self.

Origins

From the articles I can find online, the thunderdome pattern was created as a way to increase testability of your actions. By only allowing a single model in, it’s very easy to setup the context of the test. It also removes the need to rely on the HttpContext inside the actions. In FubuMVC, if information is needed from the HttpContext, a property can be added to the input model and it’s the responsibility of model binding to set that property to the corresponding value in the HttpContext. (Model Binding will be the topic of another post.)

Benefits

Once you decide to use the thunderdome pattern, it’s not a big step to make each of the actions take a unique input model. If each behavior chain has a unique input model, then each behavior chain can be identified by its input type. This is a great way, in my opinion, to take advantage of static typing and make it work for you.

In FubuMVC, this is the most common way to identify behavior chains. Since each behavior chain is associated to a route, it’s the most common way to build URLs, also.

Other Patterns

It’s important to note that the thunderdome pattern is not the only way to use FubuMVC. All of the combinations of One/Zero model in One/Zero model out patterns are supported. I can see using a one model in, zero model out for use of services where you only care about the HTTP Response code. (If you want to do this with the thunderdome pattern, you can just return an HttpStatusCode from your action. There’s a built in convention in FubuMVC to handle this return type.)

I don’t use either of the patterns which do not have an input model. If I have an action that doesn’t have any input, then I typically have it accept an empty “marker” input model, anyway. This gives me the ability to uniquely identify it if I ever need to create a link to it, or for any other reason.

FubuMVC: Authorization

When it comes to authorization, FubuMVC is very powerful in letting you setup extremely customizable rules for deciding who has access to what. Luckily, Joshua Arnold was able to help me get these rules setup and I’d like to document the process for everyone else.

In this post, I’ll be going over how to write a custom authorization rule to keep users from being able to edit another user’s blog post in a simple blogging application. I’ll do this by plugging into the authorization facilities built into FubuMVC.

Overview

  1. Come up with a convention to identify which action calls need authorization, and how to access the information we need from the input models
  2. Do the heavy lifting by writing an authorization policy to check and see if the user is authorized to do what they’re trying to do
  3. Teach FubuMVC about the convention
  4. Tell FubuMVC to use the convention

Step 1: Come up with a convention

In order to figure out which routes/action calls need authorization, I’m going to implement an interface on my input models. This will not only mark them so I can look them up later, but also provide me with a generic way of accessing the properties that I’m going to be running my authorization code against. In this case, all I need from the models are the post id, so that’s all that’s required on the interface. It’s also worth noting here that I have two input models. I use EditPostRequestModel for the http get action call (to get the post in order to display it for the user to edit), and the EditPostInputModel for the http post action call (which handles the saving of the post after it’s been edited by the user.)

Step 2: Do the heavy lifting

Now that I’ve marked which actions need authorization, I’m going to write an authorization policy so I can specify how that authorization needs to be done. In this case I want to verify that the post being requested belongs to the user that is requesting it. This is a simple example, but you can do any sort of complex checking here that you wanted (e.g. If it were possible that your users could be editors and edit other user’s posts, that check would go here.)

For my simple case, we can just check that the user property on the requested post, is the same user as the user that’s currently logged in. The CurrentUser dependency is what I use to get the currently logged in user. Use whatever your convention is to do that here, instead.

Step 3: Teach FubuMVC about the convention

So, I have my action calls setup that need authorization, and I’ve created a way to do that authorization, the next step is to tie the two together and tell FubuMVC how to apply my authorization policy.

FubuMVC has a DSL to do this, which can be accessed through the Authorization property on the behavior chain. The way this works is inside FubuMVC, after it has applied all of your conventions, it will check to see if there have been any policies added to the Authorization property on the behavior chain. If it finds that there has been, it will wrap the chain with an internal behavior called AuthorizationBehavior.

The way the AuthorizationBehavior works is by calling the authorization policies for the behavior and checking the return value of them. If any of the policies return AuthorizationRight.Deny, then the chain is stopped and IAuthorizationFailureHandler is called on to handle the erred request.

It’s very important that FubuMVC sets this convention up after it sets up all of the custom conventions. This ensures that the AuthenticationBehavior gets run before anything else in the behavior chain.

What I want to do is add a policy to the Authorization property for all action calls which have an input type that implements my IPostModel interface.

Since I made the policy an open generic, I’ll close that generic with the type that the action call is using. This is what allows the authorization policy to get the model out of the IFubuRequest (since things are keyed by type), which means I can check values specific to that particular request.

Step 4: Tell FubuMVC to use the convention

Now that everything is all setup and I’ve defined how my convention works, I just have to tell FubuMVC to use the convention. To do this, in my FubuRegistry, I just add the following line.

Now when I start up the application and browse to /post/edit/:id, I get a 403 forbidden message if the id is for a post that isn’t mine. If the post is mine, I’m allowed to pass right through to the rest of the chain.

Bonus!

What if I don’t want to return a 403 forbidden message? I want to do something unique and special!

Remember when I said that the AuthorizationBehavior will call on IAuthorizationFailureHandler if any of the policies go wrong in order to handle the authorization error? Well, all you have to do is implement your own IAuthorizationFailureHandler and tell FubuMVC to replace the default implementation with your custom one, in your FubuRegistry.

Now when I attempt to browse to /post/edit/:id, if I don’t own the post, I get redirected to the login screen.

FubuMVC: Authentication

If you’re using FubuMVC and your site requires users to login, you’ll probably want to use the built in authentication facilities that FubuMVC provides. In this post, I’ll attempt to explain how this works.

I’m going to write an authentication convention to block access to certain actions from unauthenticated users.

Overview

  1. Come up with a convention to determine which action calls require the user to be authenticated
  2. Write a behavior to redirect a user if they’re not authenticated
  3. Teach FubuMVC about the convention
  4. Tell FubuMVC to use the convention
  5. Let FubuMVC know when someone has logged in or out

Step 1: Come up with a convention

For this example, my convention is going to be any action call marked with a [Secured] attribute, will require an authenticated user in order to execute. As you’ll see, this can be adapted to match whatever convention you want to use.

To start, I’ll just create a simple marker attribute and throw it on the action calls that I want to be secured.

Since we’ve marked the action calls that we want to secure, we’ll be able to pick them out later when we want to teach FubuMVC about our convention.

Step 2: Write a behavior

In order to do the actual work of checking to see if the user is logged in, and redirecting them if they’re not, I’ll need to create a behavior. If you’re unfamiliar with behaviors, I’d highly recommend reading these articles about them.

The behavior will depend on the ISecurityContext provided by FubuMVC, so I can check if the user has been authenticated. If the user has not been authenticated, I’ll just redirect them to the login page.

It’s important to note that, by default, the ISecurityContext is a wrapper around the current HttpContext. This means that FubuMVC is really just using standard ASP.NET forms authentication.

When FubuMVC calls the behavior, it will look at the result coming from the performInvoke() method. If the method returns DoNext.Continue, then the next behavior in the chain will be called. However, if it returns DoNext.Stop, then it will not call the next behavior in the chain and execution of the request stops. (It actually doesn’t just stop, if you’ve been through any behaviors that wrap the authentication behavior, you will begin to start calling the afterInsideBehavior method on these.

Step 3: Teach FubuMVC about the convention

So I’ve setup our action calls to use the convention and the behavior to kick unauthorized people out. Now I just need to teach FubuMVC what that convention is, and how it should be implemented. In order to do that, I’ll need to create an implementation of the IConfigurationAction interface (these are typically called conventions.) My convention will modify the graph and wrap all action calls that have the Secured attribute with our behavior.

In the convention, I have access to all of the Actions that FubuMVC knows about. I’m able to filter those actions to only get those which implement the SecuredAttribute, and then wrap those with the behavior I just created.

This is where I could change it up and use another convention if I wanted to. Instead of filtering the actions based on attributes, I can check for anything. If I wanted to filter for all actions with an input type that starts with “Add” or “Edit”, that’s possible by using this convention.

Step 4: Tell FubuMVC to use the convention

In order to tell FubuMVC to apply the convention, I need to call this method from the FubuRegistry.

Now, if I run my project and attempt to browse to an action that has been locked down, I get redirected to the login page, which is exactly what I wanted it to do.

Step 5: Let FubuMVC know when someone has logged in or out

In order to tell FubuMVC that a user has been logged in, I need to depend on IAuthenticationContext in the action that logs users in. Once I’ve verified the user’s credentials are correct, I can call the ThisUserHasBeenAuthenticated method on IAuthenticationContext to let FubuMVC know.

When I want to log a user out, call SignOut on the IAuthenticationContext interface.

Gotcha!

As I stated earlier, the default authentication in FubuMVC is really just a wrapper for forms authentication in ASP.NET. So, in order to use the ISecurityContext, you’ll need to turn on forms authentication in your web.config, otherwise the ISecurityContext will always say that the user is authenticated.