Streaming files with httpclient and multiple controllers

In a recent project I was faced with a requirement which stated that all access to databases, file system and what not, had to go via trusted endpoints. It’s not uncommon but you do hit some roadblocks. Since the application was file system intensive I had to look for a way to stream files across machine and web application boundaries. Simplified, the application looked like this:

If you want to stream files in this situation you don’t want to load the entire file in memory in the web api part, then sent it to the public facing web app which in turn loads the complete file in memory and then gives it to the client browser. To increase performance of the system the file should be able to get streamed from where its stored to the browser of the user.

The API controller in the private API part is pretty straightforward. Lookup the file and pass it along using a custom HttpActionResult: FileResult.

[RoutePrefix("api/files")]
[Authorize]
public class FilesController : ApiController
{       
    [Route("{fileId:long}")]
    [HttpGet]
    public IHttpActionResult Get(long fileId)
    {
        FileMetaData metadata = LoadFileMetaData(fileId);
        if (File.Exists(metadata.Location))
        {
            return new FileResult(File.Open(metadata.Location, FileMode.Open, FileAccess.Read), metadata.ContentType, metadata.FileName);
        }
        else
        {
            return NotFound();
        }   
    }
}

Creating a custom HttpActionResult is pretty straightforward, if you look around online you’ll find plenty of examples. This is the one I ended up with myself. What’s important in this case, apart from loading the file, is populating the mimetype and setting the contentdisposition header.

class FileResult : IHttpActionResult
{
    private readonly string _filePath;
    private readonly string _contentType;
    private readonly string _filename;
    private readonly Stream _stream;
 
    public FileResult(string filePath, string contentType = null, string filename = null)
    {
        if (filePath == null) throw new ArgumentNullException("filePath");
 
        _filePath = filePath;
        _contentType = contentType;
        _filename = filename;
    }
 
    public FileResult(Stream stream, string contentType = null, string filename = null)
    {
        if (stream == null) throw new ArgumentNullException("stream");
 
        _stream = stream;
        _contentType = contentType;
        _filename = filename;
    }
 
    public Task<HttpResponseMessage> ExecuteAsync(CancellationToken cancellationToken)
    {
        var response = new HttpResponseMessage(HttpStatusCode.OK)
        {
            Content = new StreamContent(_stream ?? File.OpenRead(_filePath))
        };
 
        var contentType = _contentType ?? MimeMapping.GetMimeMapping(Path.GetExtension(_filePath));
        response.Content.Headers.ContentType = new MediaTypeHeaderValue(contentType);
        if (!string.IsNullOrWhiteSpace(_filename))
        {
            response.Content.Headers.ContentDisposition = new ContentDispositionHeaderValue("attachment")
            {
                FileName = _filename
            };
        }
 
        return Task.FromResult(response);
    }
}

Then comes the tricky part: streaming the file directly to the user from the public facing web application. In this case the public application was an MVC site. I’m using HttpClient to call the private API and indicate I want my code to process the headers immediately after they are received. This allows me to see if the file was found or not. If we don’t have a 401 error code I check if I might have another error, if not I asynchronously load the response stream and pass it along the standard FileActionResult. An essential part is the Response.BufferOutput part. By default it’s set to true and will force the web app to load the file completely in memory before giving it to the client.

[Route("file/{fileId:long}")]
[HttpGet]
public virtual async Task<ActionResult> File(long fileId)
{
    var httpClient = new HttpClient(new HttpClientHandler() { UseDefaultCredentials = true });
 
    var response = await httpClient.GetAsync(ConfigurationManager.AppSettings["App.PrivateApi"] + $"/{fileId}", HttpCompletionOption.ResponseHeadersRead);
    if (response.StatusCode == System.Net.HttpStatusCode.NotFound)
    {
        return HttpNotFound();
    }
    else
    {
        response.EnsureSuccessStatusCode();
        var fileStream = await response.Content.ReadAsStreamAsync();
        Response.BufferOutput = false;
        return File(fileStream, response.Content.Headers?.ContentType?.MediaType, response.Content.Headers?.ContentDisposition?.FileName);
    }
}

Model binding with Headers in ASP.NET WebAPI

While writing the previous blog post I noticed that Outlook sends an additional header “IfModifiedSince” when updating its subscription of the iCal feed. It would be nice to support this additional parameter in the API to retrieve appointments. Instead of always returning the entire list of appointments, an additional filter will be used to limit the appointments to those that were changed since our last synchronization.

ifModifiedSinceWhile we could just read the header value inside of our controller action, it would be much nicer if our action would receive it as a parameter. The extension points we need in this case fall under the model binding category and while it shares the same idea and goals with ASP.NET MVC, there are some differences between ASP.NET MVC and WebAPI. There is a great MSDN article which covers most of the things we need.

The two concepts we need to grasp are model binders and value providers. Value providers are an abstraction over, well, values. For instance there’s a query string value provider that reads the query string and will allow those values to be used as parameters in your actions or by model binders. Model binders actually do something with values, they will use value providers to retrieve i.e. a first name and last name value and create a more complex instance.

So in this case we need to read a value from a header inside our HttpRequestMessage. Let’s implement a value provider for our IfModifiedSince header by implementing the IValueProvider interface.

public class IfModifiedValuesProvider
    : IValueProvider
{
    private HttpRequestMessage _request;
    private const string header = "IfModifiedSince";
 
    public IfModifiedValuesProvider(HttpRequestMessage requestMessage)
    {
        _request = requestMessage;
    }
 
    public bool ContainsPrefix(string prefix)
    {
        bool found = false;
        if (string.Equals(header, prefix, StringComparison.OrdinalIgnoreCase))
        {
            found = _request.Headers.Any(x => x.Key == prefix);    
        }
        return found;            
    }
 
    public ValueProviderResult GetValue(string key)
    {
        var headerValue = _request.Headers.IfModifiedSince;
        ValueProviderResult result = null;
        if (headerValue.HasValue)
        {
            result = new ValueProviderResult(headerValue, headerValue.ToString(), CultureInfo.InvariantCulture);
        }
        return result;
    }
}

The two methods we need to implement are ContainsPrefix and GetValue. The ContainsPrefix is of no importance in this case, GetValue is where the magic happens. In this method we read the the value from the header and, if it exists, return a ValueProviderResult populated with the current values. ValueProviders are always accompanied by ValueProviderFactories. It’s the responsibility of the factory to create and setup the value provider. In this case we want to supply our value provider with a reference to the current HttpRequestMessage.

public class IfModifiedValuesProviderFactory 
    : ValueProviderFactory
{
    public override IValueProvider GetValueProvider(HttpActionContext actionContext)
    {
        return new IfModifiedValuesProvider(actionContext.Request);
    }
}

Inheriting from the abstract base ValueProviderFactory allows us to override the GetValueProvider method where we initialize our value provider. We now have enough infrastructure to go back to our AppointmentController.

public IEnumerable<AppointmentModel> Get([ValueProvider(typeof(IfModifiedValuesProviderFactory))]DateTimeOffset? ifModifiedSince = null)
{
    IEnumerable<AppointmentModel> models = null;
    using (var context = new AppointmentsEntities())
    {               
        IQueryable appointments = context.Appointments;
        if (ifModifiedSince.HasValue) 
        {
            appointments = appointments.Where(x => x.LastModifiedDate >= ifModifiedSince.Value);
        }
        models = MapAppointents(appointments);
    }
    return models;
}

By decorating the ifModifiedSince parameter with the ValueProvider attribute it will be populated with the result of the GetValue call. Which does resolve our issue, but it would be even better if users of our API would be able to pass the ifModifiedSince date by using the header or supply it via a parameter in the query string. There are several ways to make this happen.

One approach would be to use the ValueProvider attribute again, chaining along every value provider we want to use.

public IEnumerable<AppointmentModel> Get(
    [ValueProvider(typeof(IfModifiedValuesProviderFactory), typeof(QueryStringValueProviderFactory ))]
    DateTimeOffset? ifModifiedSince = null)
{
    // omitted
}

Adding the QueryStringValueProviderFactory to the list of value providers will help us, but every time we want to add another source of our ifModifiedSince parameter we will have to add it here.

A better approach is to remove the attribute on the parameter entirely and add our value provider to the configuration of our WebAPI.

public static void Register(HttpConfiguration config)
{
    config.Services.Add(typeof(ValueProviderFactory), new IfModifiedValuesProviderFactory());
}

If we now run the application and use a query string to supply the value for our action, we will see that the date is passed along to our controller. Unfortunately if a client application uses the header, our custom value provider is not invoked at all. What’s missing?

Well it turns out that when you declare actions on your controller, by default only data that’s present in the route data dictionary or the query string will be passed to the controller. It’s like putting [FromUri] on your parameters. If we want have our own value provider come into play we have to use the [ModelBinder] attribute as well.

public IEnumerable<AppointmentModel> Get([ModelBinder]DateTimeOffset? ifModifiedSince = null)
{
   // omitted
}

Now we’re telling WebAPI to use the model binding infrastructure. The default model binder will use all the registered value providers to create a match. Since we’ve registered our IfModifiedValuesProviderFactory in the WebAPI configuration, it will be automatically picked up. If a user of our API uses a query string to pass along the ifModifiedSince value, that will keep working as well. If we add a CookieValueProvider in the future, we will only have to implement the value provider and add it to the configuration of our application. We will not have to inspect every method to see where we should add them explicitly. Best of both worlds. There’s a nice poster of the lifecycle of an HttpRequestMessage on MSDN which includes an illustration on how model binding works.

Exposing iCal data in WebAPI

With ASP.NET Web API it’s now easier than ever to create lightweight HTTP services in .NET. Out of the box the ApiControllers you implement can read json, xml and form encoded values from the HTTP request but also write xml and json to the HTTP response.

HTTP has the concept of content negotiation. This means that when a client requests a resource, it can tell the server that it wants the result in a specific format.

Below you can see an HTTP request that requests json:

And in the HTTP response the data is formatted accordingly:

If the client requests the response to be formatted as xml:

Then the result will be returned as xml:

This mechanism can be extended to support different kind of formatters to read from or write to the body of a request or response.

Let’s say we want to support an additional format that can write appointments in iCal format. To create a custom formatter you inherit from BufferedMediaTypeFormatter or MediaTypeFormatter. For this example I chose the first one.

The code is pretty straightforward and represents a very simple implementation of the iCal standard. The only WebAPI specific code can be found in the constructor.There we add the mapping for the headers we want the formatter to be invoked for. After we add the formatter to the configuration object, it will be invoked automatically whenever a client says it accepts “text/iCal”.

The current setup works fine in Fiddler, or when you use a custom client (JavaScript or HttpClient). But for a true end-to-end sample I want to use Outlook to connect to my appointment service.

Unfortunately Outlook does not send an accept header with text/iCal when it requests resources from an internet calendar. So we need to work around this problem.

Here another extensibility point of ASP.NET Web API comes into play: MessageHandlers.

MessageHandlers allow you to plug into the request processing pipeline on its lowest level. You can inspect the request and response message and make changes. In this case we can inspect the user agent that is added to the request when Outlook contacts our service. When we find a match, we will add an additional header to the incoming request.

We also add this message handler to the configuration object.

We now have everything in place to add an internet calendar in Outlook and view the appointments in our WebAPI.

  1. Open Outlook
  2. Go to Calendar
  3. In the ribbon, click on “Open Calendar” and then “From Internet”
  4. Fill in the url of the AppointmentService in WebAPI i.e. http://localhost:58250/api/appointments
  5. Click Ok.

You now have one AppointmentController serving json, xml and iCal! The complete source can be downloaded here.