Tuesday, June 28, 2011

ETags and optimistic concurrency control with WCF Web Apis

In my previous post I wrote about how to take advantage of ETags to implement HTTP cache validation with WcfWebApis, but this is not the only use of ETags, you can also use them to implement optimistic concurrency control over HTTP. The following workflow illustrates this feature:

  1. The client wants to get a resource and for that it sends a GET request and the server answers with the resource representation and its ETag.
  2. The client decides to update the resource and for that it sends a PUT request with the modified resource representation along with the ETag from the previous step.
  3. The server checks the incoming ETag against the current resource representation ETag from its repository (i.e: Cache, DB).
    • If they match, the server carries on with the resource update and returns a 200 HTTP status if it succeeds.
    • If they don’t match, it’s because someone else has updated the resource in the meantime. Therefore the server invalidates the request and returns a 402 Precondition Failed status. (Concurrency error)

Now replace “GET” by “SELECT” and “PUT” by “UPDATE… WHERE” and this is exactly the same optimistic concurrency mechanism that you can usually find implemented in relational databases.

Enough of theory, we can implement step 3 from the workflow with a DelegatingChannel. The channel takes advantage of the ETag cache that I used in my previous post.

public class EntityTagConcurrencyChannel : DelegatingChannel
    private readonly ETagCache _eTagCache = ETagCacheProvider.Instance;
    public EntityTagConcurrencyChannel(HttpMessageChannel innerChannel)
        : base(innerChannel)
    protected override Task<HttpResponseMessage> SendAsync(HttpRequestMessage request, 
                                                  CancellationToken cancellationToken)
        if (request.Method == HttpMethod.Put && 
            var etag = request.Headers.IfNoneMatch.First().ToString();
            if (etag != "*")
                var cached = _eTagCache.Get(request.RequestUri.ToString());
                return etag != cached
                  ? Task.Factory.StartNew(() => CreatePreconditionFailedResponse(),
                  : base.SendAsync(request, cancellationToken);
        return base.SendAsync(request, cancellationToken);
    private static HttpResponseMessage CreatePreconditionFailedResponse()
        return new HttpResponseMessage(HttpStatusCode.PreconditionFailed, 
As per the HTTP specification, the special wildcard “*” value makes the server not to run the validation.

Saturday, June 25, 2011

ETags with WCF Web APIs Preview 4

Jose Romaniello wrote a very nice blog post about to handle HTTP cache validation with ETags and the WCF Web APIs. For the ETags server implementation there are two different aspects to be considered: the code that checks if a resource has been modified based on the incoming request ETag value (this is handled by a DelegatingChannel in Jose’s implementation), and the part where the resource ETag is added to the response so the client can use this value to make further requests (In Jose’s implementation this is handled on case by case basis in every resource handler). I was interested to see if the second part could be handled in a generic and automated way by using some conventions, so the custom logic in the resource handler could be spared. So let’s get started!

First of all, let’s define the resource.

public class ProductResource
    public int Id { get; set; }
    public string Name { get; set; }
    public float Price { get; set; }
    public string ETag { get; set; }

As in Jose’s post, for the ETag value we can use the DB version field for example. The resource handler goes as follows.

public class ProductResourceHandler
    private readonly ProductRepository _productRepository = new ProductRepository();
    [WebGet(UriTemplate = "{id}")]
    public HttpResponseMessage<ProductResource> Get(int id)
        var product = _productRepository.Get(id);
        return product == null ? new HttpResponseMessage<ProductResource>(HttpStatusCode.NotFound) 
                               : new HttpResponseMessage<ProductResource>(product);
    [WebInvoke(UriTemplate = "{id}", Method = "PUT")]
    public ProductResource Put(int id, ProductResource product)
        return product;
    [WebInvoke(UriTemplate = "{id}", Method = "DELETE")]
    public HttpResponseMessage Delete(int id)
        var product = _productRepository.Get(id);
        if (product == null)
            return new HttpResponseMessage<ProductResource>(HttpStatusCode.NotFound);
        return new HttpResponseMessage(HttpStatusCode.OK, "Deleted");
The EntityTagResponseHandlerChannel handles the ETag injection to the responses. On top of that, when a PUT or DELETE request succeeds, the channel removes the ETag associated to the incoming url from the cache. This ensures that further GET request won’t get a stale version of the resource.

   1: public class EntityTagResponseHandlerChannel : DelegatingChannel
   2: {
   3:     private readonly ETagCache _eTagCache = ETagCacheProvider.Instance;
   5:     public EntityTagResponseHandlerChannel(HttpMessageChannel innerChannel)
   6:         : base(innerChannel)
   7:     {
   8:     }
  10:     protected override Task<HttpResponseMessage> SendAsync(HttpRequestMessage request, 
  11:                                                             CancellationToken cancellationToken)
  12:     {
  13:         return base.SendAsync(request, cancellationToken).ContinueWith(task =>
  14:         {
  15:             var response = task.Result;
  16:             if (response.StatusCode != HttpStatusCode.OK) return response;
  17:             var requestMethod = response.RequestMessage.Method;
  18:             if (requestMethod == HttpMethod.Put || requestMethod == HttpMethod.Delete)
  19:             {
  20:                 _eTagCache.Remove(response.RequestMessage.RequestUri.ToString());
  21:             }
  22:             else if (requestMethod == HttpMethod.Get || requestMethod == HttpMethod.Head)
  23:             {
  24:                 if (SetETag(response))
  25:                 {
  26:                     _eTagCache.Add(response.RequestMessage.RequestUri.ToString(), 
  27:                                     response.Headers.ETag.ToString());
  28:                 }
  29:             }
  31:             return response;
  32:         });
  33:     }
  35:     private static bool SetETag(HttpResponseMessage response)
  36:     {
  37:         if (response.Headers.ETag != null || response.Content == null)
  38:             return false;
  39:         dynamic content = response.Content;
  40:         response.Headers.ETag = ETagProvider.GetETag(content.ReadAs());
  41:         return response.Headers.ETag != null;
  42:     }
  43: }
Note that we check if the ETag has been previously been set (line 37) meaning that custom ETag handling logic in the resource handler overrides the Channel logic.

The tricky part here is to get the ETag value from the resource associated to the response. The object contained in the response variable is a generic HttpResponseMessage<TResource> that contains the original resource. The problem is that, as the DelegatingChannels are non-generic objects, they can only reference the non-generic versions of the response entities (HttpResponseMessage, ObjectContent base classes) which don’t have access to the original resource object.  Since casting to the generic version is out of the question (this Channel should handle any resource type request), we take advantage of a dynamic variable to execute the generic ReadAs method that gives us the original resource entity (line 40).

Finally the ETagProvider retrieves the ETag from the resource object if exists.

   1: static class ETagProvider
   2: {
   3:     private static readonly ConcurrentDictionary<Type, PropertyInfo> ETagPropertyInfoByType = 
   4:         new ConcurrentDictionary<Type, PropertyInfo>();
   5:     private const string ETagPropertyName = "ETag";
   7:     internal static EntityTagHeaderValue GetETag(object resource)
   8:     {
   9:         if (resource == null) return null;
  10:         var resourceType = resource.GetType();
  11:         PropertyInfo propertyInfo;
  12:         if (!ETagPropertyInfoByType.TryGetValue(resourceType, out propertyInfo))
  13:         {
  14:             propertyInfo = resourceType.GetProperty(ETagPropertyName);
  15:             ETagPropertyInfoByType.AddOrUpdate(resourceType, propertyInfo, (t, p) => p);
  16:         }
  17:         return GetETag(resource, propertyInfo);
  18:     }
  20:     private static EntityTagHeaderValue GetETag(object resource, PropertyInfo propertyInfo)
  21:     {
  22:         return propertyInfo == null ? null
  23:                                     : new EntityTagHeaderValue(String.Format(CultureInfo.InvariantCulture,
  24:                                                                 "\"{0}\"",
  25:                                                                 propertyInfo.GetValue(resource, null)));
  26:     }
  27: }

The ETag property is retrieved by using Reflection and a dictionary of PropertyInfos by type is used to improve performance.

In closing

HTTP cache expiration could be implemented in a similar way by using for example a .NET attribute defining the resource MaxAge, then a DelegatingChannel would grab this value and add it to the associated response.

Nevertheless, bear in mind that this is by no means production-ready code! My main purpose with this post was to show how the Wcf Web Apis infrastructure can be leveraged to implement HTTP protocol features such as cache validation.

Friday, June 10, 2011

AppFabric Cache: Setting MaxBufferSize in a SQL Sever-based cluster configuration

The AppFabric Cache limits by default the size of individual items to 8MB. If you've gone over that limit, you have probably got this error already:

ErrorCode<ERRCA0016>:SubStatus<ES0001>:The connection was terminated, possibly due to server or network problems or serialized Object size is greater than MaxBufferSize on server. Result of the request is unknown.

To increase the limit, you can override the default MaxBufferSize value in the cluster XML configuration file.

      <transportProperties maxBufferSize="10000000" />

But what happens if you're using SQL Server as the AppFabric Cache cluster configuration store? If you take a look at the Config DB, you'll quickly realize that the data stored over there looks nothing like the original xml. So how can we change this setting?

PowerShell to the rescue!  Use the command below to extract the configuration file that represents your cluster configuration. The command works regardless of the underlying configuration stored being used.

Export-CacheClusterConfig -File c:\config.xml 

Do the change in the file, stop the cluster, import the ammended config file and start the cluster again.

Import-CacheClusterConfig -File c:\config.xml -Force 

Hope this helps!

Saturday, April 16, 2011

Tunnelling PUT and DELETE with WCF Web APIs Preview 4

PUT and DELETE are first class citizen HTTP methods in REST resource implementations but unfortunately not all clients support these methods and in fact the XHTML4 specification defines GET and POST as the only valid Form methods. Browsers can use Javascript and the XmlHttpRequest to get around this limitation, but other issues such as the fact that some firewalls block PUTs and DELETE requests, recommend having an alternative strategy to convey these methods semantics in the cleanest possible way.

Existing REST frameworks implement a tunnelling technique that solves this issue: clients make POST request and provide the “real” method somewhere in the request. For example, Ruby On Rails uses a hidden form field called _method and users of the Google GData API store the real method in the X-HTTP-Method-Override Custom HTTP header.

Implementing PUT and DELETE tunnelling is a very easy task thanks to the new extensibility point: Message Channels. Message Channels can influence what Resource Method is to be executed by the framework and that’s precisely what we’re going to do:

public class HttpMethodTunnelChannel : DelegatingChannel
    public HttpMethodTunnelChannel(HttpMessageChannel innerChannel)
        : base(innerChannel)
    protected override Task<HttpResponseMessage> SendAsync(HttpRequestMessage request, 
                                                           CancellationToken cancellationToken)
        return base.SendAsync(request, cancellationToken);
SetOverrideMethodIfAny is a custom extension method of HttpResponseMessage.

public static class HttpRequestMessageExtensions
    public static void SetOverrideMethodIfAny(this HttpRequestMessage request)
        var method = request.GetOverrideMethod();
        if (method != null) request.Method = method;
    public static HttpMethod GetOverrideMethod(this HttpRequestMessage request)
        var method = HttpUtility.ParseQueryString(request.RequestUri.Query)["_method"];
        if (String.IsNullOrEmpty(method))
            method = request.Headers
                .Where(h => h.Key == "X-HTTP-Method-Override")
                .SelectMany(h => h.Value)
        return MapMethod(method);
    private static HttpMethod MapMethod(string method)
        if (String.IsNullOrEmpty(method)) return null;
        if (String.Compare(method, HttpMethod.Put.Method, true) == 0)
            return HttpMethod.Put;
        if (String.Compare(method, HttpMethod.Delete.Method, true) == 0)
            return HttpMethod.Delete;
        if (String.Compare(method, HttpMethod.Options.Method, true) == 0)
            return HttpMethod.Options;
        if (String.Compare(method, HttpMethod.Head.Method, true) == 0)
            return HttpMethod.Head;
        return null;
In the sample, the query string _method takes precedence over the custom HTTP Header. Bear in mind that the code above not only tunnels PUT and DELETE requests, clients could actually do weird things such as tunnelling GETs over POST or the non-recommended practice of tunnelling unsafe operations (i.e. DELETE) over safe requests (i.e.: GET) which could end up with Google deleting your data when it crawls your site!

UPDATE: You can find a more complete implementation here. The idea is that safe methods should not become unsafe when tunnelled and the other way around.

Finally, let’s use the new fluent API to bind our message channel.

public class Global : System.Web.HttpApplication
    protected void Application_Start(object sender, EventArgs e)
        var config = HttpHostConfiguration.Create().
        RouteTable.Routes.MapServiceRoute<ContactResource>("Contact", config);

And that’s it! As you can see, I’ve just scratched the surface of Message Channels and you may have already realized their potential for implementing cross cutting concerns such as logging, exception handling or security for example.