• Part 1 – DataContext Per Request
• Part 2 – AutoCommit and the RoutePreParser
• Part 3 – DataLoadOptions per Request

Code for this series is available here.

One common problem when using an ORM is the issue of “select n+1″. For example, take the following Linq to Sql query:

var context = new BlogDataContext();
var posts = context.Posts;
 
foreach(var post in posts) {
   foreach(var comment in post.Comments) {
     var commenter = comment.Commenter.Name;
     Console.WriteLine("User {0} commented on post {1}", commenter, post.Title);
   }
}

This might look fine on the surface, but by default Linq to Sql will lazily-load all associations. This means that for each post loaded from the database Linq to Sql will issue a query to load the comments, and for each comment Linq to Sql will issue another query to load the commenter. So if you have 10 posts, and each post has 5 comments then you’ll end up making 61 queries to the database (1 for the posts, 1 for each post to get the comments, and 1 for each comment to get the user)

This can be mitigated by using DataLoadOptions to eagerly load all of the data in a single query:

var context = new BlogDataContext();
 
var options = new DataLoadOptions();
options.LoadWith<Post>(p => p.Comments);
options.LoadWith<Comment>(c => c.Commenter);
context.LoadOptions = options;
 
var posts = context.Posts;
foreach(var post in posts) {
 //...
}

The Problem

However, once a query has been executed, the LoadOptions property of a DataContext is frozen – you cannot then specify any additional eager loading paths.*

If you’re using a DataContext per request as per my previous post then this can be a problem. For example, imagine you have the following controller action:

public class PostsController : Controller {
  private BlogDataContext context;
 
  public PostsController(BlogDataContext context) {
     this.context = context; //injected via our IoC container
  }
 
  public ActionResult Show(int id) {
    var options = new DataLoadOptions();
    options.LoadWith<Post>(p => p.Comments);
    options.LoadWith<Comment>(c => c.Commenter);
 
    context.LoadOptions = options;
 
    var post = context.Posts.Single(p => p.Id == id);
    return View(post);
  }
}

…then this will work as expected. But now imagine your action is decorated by an ActionFilter:

[LoadCurrentUser]
public ActionResult Show(int id) { 
  //...
}

…and let’s assume that our LoadCurrentUser filter attempts to load the details of the current user from the database and store them in ViewData:

public class LoadCurrentUserAttribute : ActionFilterAttribute {
	public override void OnActionExecuting(ActionExecutingContext filterContext) {
		var context = ObjectFactory.GetInstance<BlogDataContext>();
		var identity = filterContext.HttpContext.User.Identity;
 
		if (identity.IsAuthenticated) {
			var currentUser = context.Users.Single(x => x.UserName == identity.Name);
			filterContext.Controller.ViewData["CurrentUser"] = currentUser;
		}
	}
}

Now when the controller is executed we will get the following error:

Setting load options is not allowed after results have been returned from a query.

This happens because we’re trying to set the LoadOptions for the Post after the LoadCurrentUser filter already executed a query.

The Solution

The approach I use to work around this problem is to use a custom AuthorizationFilter in conjunction with an eager loading specification.

First, we define an interface for eager loading specifications:

public interface IEagerLoadingSpecification {
  void Build(DataLoadOptions options);
}

Next, we can write some classes that implement IEagerLoadingSpecification that define particular specifications for eager loading. Using the example above, we have PostWithComments and CommentWithCommenter:

public class PostWithComments : IEagerLoadingSpecification {
  public void Build(DataLoadOptions options) {
    options.LoadWith<Post>(p => p.Comments);
  }
}
 
public class CommentWithCommenter : IEagerLoadingSpecification  {
  public void Build(DataLoadOptions options) {
     options.LoadWith<Comment>(c => c.Commenter);
  }
}

Next, we write an EagerlyLoadAttribute which implements IAuthorizationFilter and takes an array of Types in its constructor:

public class EagerlyLoadAttribute : FilterAttribute, IAuthorizationFilter {
  private Type[] types;
 
  public EagerlyLoadAttribute(params Type[] types) {
    this.types = types;
  }
 
  public void OnAuthorization(AuthorizationContext filterContext) {
    var loadOptions = new DataLoadOptions();
    var context = ObjectFactory.GetInstance<BlogDataContext>();
 
    foreach(var type in types) {
       if(! typeof(IEagerLoadingSpecification).IsAssignableFrom(type)) {
          throw new InvalidOperationException(string.Format("Type {0} does not implement IEagerLoadingSpecification", type));
       }
 
      var spec = (IEagerLoadingSpecification)Activator.CreateInstance(type);
      spec.Build(loadOptions);
    }
 
   context.LoadOptions = loadOptions;
  }
}

So what does this filter do? Firstly, because it implements IAuthorizationFilter this means it will be invoked before any action filters that decorate your controller action. When OnAuthorization is invoked, it loops over each of the Types that we’ve passed to its constructor, instantiates them and casts them to IEagerLoadingSpecification. Next, a single DataLoadOptions instance is passed to each of the specifications in turn so they can build up the required eager loading paths.

The end result is that you can now define your actions like this:

  [EagerlyLoad(typeof(PostWithComments), typeof(CommentWithCommenter))]
  [LoadCurrentUser]
  public ActionResult Show(int id) {
    var post = context.Posts.Single(p => p.Id == id);
    return View(post);
  }

Because the EagerlyLoadAttribute is an IAuthorizationFilter, it will be invoked before anything else meaning that by the time both our LoadCurrentUser filter and the Show action are invoked the DataLoadOptions have already been set on our DataContext.

* Note that other ORMs like NHibernate, LLBLGen and the Entity Framework don’t have this problem because they allow eager loading paths to be specified at the query level.

• Part 1 – DataContext Per Request
• Part 2 – AutoCommit and the RoutePreParser
• Part 3 – DataLoadOptions per Request

Code for this series is available here.

In my previoust post, I demonstrated how you could scope a Linq to Sql DataContext to a single HTTP Request by using StructureMap to manage the lifetime of the DataContext instance. This often works well, but has a couple of gotchas. Let’s take a look at an example:

public class PostController : Controller {
	private readonly BlogDataContext context;
 
	public PostController(BlogDataContext context) {
		this.context = context;
	}
 
	public ActionResult Create() {
		return View();
	}
 
	[AcceptVerbs(HttpVerbs.Post), AutoCommit]
	public ActionResult Create(Post post) {
		context.Posts.InsertOnSubmit(post);
		return RedirectToAction("Show", new{ id = post.Id });
	}
 
	public ActionResult Show(int id) {
		var post = context.Posts.SingleOrDefault(x => x.Id == id);
		if (post == null) {
			throw new HttpException(404, "The post could not be found.");
		}
 
		return View(post);
	}
}

In this example, we have a Post controller for our fictional Blog database. The Create action renders a view where the user can create a new Post and then sends an HTTP POST to the other overload to the Create action (decorated with AcceptVerbs(HttpVerbs.Post)).

This action is decorated with the AutoCommit attribute from my previous post so that SubmitChanges will automatically be called on our DataContext. We then redirect to the “Show” action passing the Id of the newly created post in the route data.

The Show action loads the Post from the database with the corresponding Id and displays it to the user. If a post with the specified Id could not be found, it will throw a 404 exception.

There is a bug here that may not be immediately obvious. Assume that the Post’s Id property is generated by an auto-incrementing Identity field in the database. If you were to run this application and create a new Post, the Show action will always throw 404 even though the Post has been successfully created.

This happens because of when SubmitChanges is called. Our AutoCommit filter is invoked after the action has finished executing this means that at the time we call RedirectToAction(“Show”, new{ id = post.Id }) the new post has not yet been saved, so its Id will be 0.

Linq to Sql will not update the Id property until after SubmitChanges is called, which is too late in the process. The user will end up being redirected to Post/Show/0 instead of using the Id of the newly created post.

The simple workaround is to call SubmitChanges directly inside the action:

[AcceptVerbs(HttpVerbs.Post)]
public ActionResult Create(Post post) {
	context.Posts.InsertOnSubmit(post);
 
	if (ModelState.IsValid) {
		using (var transaction = new TransactionScope()) {
			context.SubmitChanges();
			transaction.Complete();
		}
	}
 
	return RedirectToAction("Show", new { id = post.Id });
}

…but this then makes our AutoCommit attribute useless and also makes the controller action significantly more complex.

However, there is another approach we could use which still allows us to use the AutoCommit attribute. To understand this, we first need to look at the order in which things are happening when our action is invoked:

1. The IoC container creates our DataContext
2. The controller is instantiated and the DataContext is passed to its constructor
3. The Create action is invoked with a Post instance created by MVC’s DefaultModelBinder
4. The post is attached to the DataContext by calling InsertOnSubmit
5. A RedirectToRouteResult is created by calling “RedirectToAction” with a dictionary of route values
6. The AutoCommit attribute calls SubmitChanges to our DataContext
7. The Post is written to the database
8. ExecuteResult on RedirectToRouteResult is invoked
9. The Redirect URL is generated by the RouteCollection
10. The user’s browser is redirected to this URL

Instead of passing the Post Id (which will be 0) to RedirectToAction, we could pass the entire post instance. After the post has been saved we can then *replace* the post instance in the RouteValueDictionary with the post’s Id before the URL is generated. This can be done by intercepting the RouteValueDictionary just before the URL is generated.

Introducing the RoutePreParser

The first thing we need is a way to identify how an object (in this case, our Post) should be converted to a route value. To do this, we can create an interface, IUrlRoutable:

public interface IUrlRoutable {
	object GetRouteParameter();
}

…and we can implement this interface in our Post class:

public partial class Post : IUrlRoutable {
	public object GetRouteParameter() {
		return Id;
	}
}

Next we can create a “fake” route. This route never generates a URL or handles a request – it is merely used to intercept the RouteValueDictionary before a URL is generated:

public class RoutePreParser : RouteBase {
	public override RouteData GetRouteData(HttpContextBase httpContext) {
		return null;
	}
 
	public override VirtualPathData GetVirtualPath(RequestContext requestContext, RouteValueDictionary values) {
		var query = from pair in values
				where pair.Value != null
				let routable = pair.Value as IUrlRoutable
				where routable != null
				select new { pair.Key, Value = routable.GetRouteParameter() };
 
		foreach (var pair in query.ToList()) {
			values[pair.Key] = pair.Value;
		}
 
		return null;
	}
}

Here we loop through each key-value pair in the RouteValueDictionary. If the value implements IUrlRoutable then we call GetRouteParameter on that object and replace the original value in the RouteValueDictionary with the result of this method.

Next, we have to add the fake route to the RouteCollection in Application_Start. Note that this must be the first route added so that it gets a chance to intercept the RouteValueDictionary.

public static void RegisterRoutes(RouteCollection routes) {
	//Add our fake route first
	routes.Add(new RoutePreParser());
 
	routes.IgnoreRoute("{resource}.axd/{*pathInfo}");
 
	routes.MapRoute(
		"Default", // Route name
		"{controller}/{action}/{id}", // URL with parameters
		new { controller = "Home", action = "Index", id = "" } // Parameter defaults
	);
}

Now, in our controller action we change the Create action to store the Post instance in the route values:

[AcceptVerbs(HttpVerbs.Post), AutoCommit]
public ActionResult Create(Post post) {
	context.Posts.InsertOnSubmit(post);
	return RedirectToAction("Show", new{ id = post });
}

…and everything now works as expected.

To recap, this is the new chain of events:

1. The IoC container creates our DataContext
2. The controller is instantiated and the DataContext is passed to its constructor
3. The Create action is invoked with a Post instance created by MVC’s DefaultModelBinder
4. The post is attached to the DataContext by calling InsertOnSubmit
5. A RedirectToRouteResult is created by calling “RedirectToAction” with a dictionary of route values. (The “Id” parameter is our new Post instance.)
6. The AutoCommit attribute calls SubmitChanges to our DataContext
7. The Post is written to the database
8. The Post instance stored in the RouteValueDictionary automatically has its Id property updated
9. ExecuteResult on RedirectToRouteResult is invoked
10. Our RoutePreParser inspects the RouteValueDictionary.
11. GetRouteParameter on Post is invoked, returning the now-populated Post Id
12. The RoutePreParser removes the Post instance from the RouteValueDictionary
13. The RoutePreParser inserts the Post Id as the “id” in the RouteValueDictionary
14. The (now correct) redirect URL is generated by the RouteCollection
15. The user’s browser is redirected to this URL

• Part 1 – DataContext Per Request
• Part 2 – AutoCommit and the RoutePreParser
• Part 3 – DataLoadOptions per Request

Code for this series is available here.

When using an ORM tool within a web application, it’s often common to scope a unit of work to the lifetime of an HTTP Request. If you’re using Linq to Sql and ASP.NET MVC, you can achieve this by using an Inversion of Control container in conjunction with an ActionFilter.

For this example, I’m going to be using the StructureMap IoC container alongside a fictional “Blog” database.

Firstly, you’ll need to configure StructureMap by calling ObjectFactory.Configure inside your Global.asax passing in a custom Registry instance:

protected void Application_Start() {
	RegisterRoutes(RouteTable.Routes);
 
	ObjectFactory.Configure(cfg => {
		cfg.AddRegistry(new MyRegistry());
	});
}

The code for MyRegistry looks like this:

public class MyRegistry : Registry {
	public MyRegistry() {
		For<BlogDataContext>()
			.HttpContextScoped()
			.Use(c => new BlogDataContext());
 
		Scan(scan => {
			scan.AddAllTypesOf<Controller>();
		});
	}
}

Here I’m telling StructureMap to create one instance of my BlogDataContext per HTTP Request as well as registering each Controller instance with the container.

Next, we need to tell MVC to use StructureMap to instantiate our controllers. This can be done by creating a custom ControllerFactory:

public class StructureMapControllerFactory : DefaultControllerFactory {
	protected override IController GetControllerInstance(RequestContext requestContext, Type controllerType) {
		return (IController) ObjectFactory.GetInstance(controllerType);
	}
}

We can then replace the DefaultControllerFactory with the StructureMapControllerFactory in our Application_Start:

protected void Application_Start() {
	RegisterRoutes(RouteTable.Routes);
 
	ObjectFactory.Configure(cfg => {
		cfg.AddRegistry(new MyRegistry());
	});
 
	ControllerBuilder.Current.SetControllerFactory(
		new StructureMapControllerFactory());
}

Now, if we create a PostController (for creating and editing Posts in our fictional blog), we can now take a BlogDataContext in the constructor (for better testability, you’d probably want to hide the DataContext behind an interface):

public class PostController : Controller {
	private readonly BlogDataContext context;
 
	public PostController(BlogDataContext context) {
		this.context = context;
	}
 
	public ActionResult Edit(int id) {
		var post = context.Posts.Single(x => x.Id == id);
		return View(post);
	}
}

Now, visiting http://mysite/Post/Edit/1 would return a view to display a post with the Id of 1 (assuming the appropriate Post exists in the database).

Automatically Submitting Changes

We can take this a stage further by adding an ActionFilter to our application that will automatically call SubmitChanges on our DataContext at the end of the HTTP Request:

public class AutoCommitAttribute : ActionFilterAttribute {
	public override void OnActionExecuted(ActionExecutedContext filterContext) {
		if (filterContext.Controller.ViewData.ModelState.IsValid) {
			var currentDataContext = ObjectFactory.GetInstance<BlogDataContext>();
 
			using (var transaction = new TransactionScope()) {
				currentDataContext.SubmitChanges();
				transaction.Complete();
			}
		}
	}
}

Here we call into StructureMap to retrieve our current DataContext and commit any changes back to the database inside a transaction. Note that this will only happen if the ModelState is valid (ie there are no validation errors).

Another thing to keep in mind that using an IoC container as a Service Locator (as we’re doing in this filter) is generally considered bad practice. There are ways around this but these are outside the scope of this post.

Now, whenever we make a change to one of our entities the change will automatically be committed to the database provided the action is decorated with the AutoCommit attribute:

[AcceptVerbs(HttpVerbs.Post), AutoCommit]
public ActionResult Create(Post post) {
	context.Posts.InsertOnSubmit(post);
	return RedirectToAction("Index");
}

The main addition to V2.6 is support for a full Linq provider, so writing queries in LLBLGen just became a lot easier. V2.6 also includes a new Prefetching API which I wrote during the 2.6 beta period, which Frans then included in the product.

For example, to eagerly load a Customer -> Orders relationship prior to v2.6, you’d have to do something like this:

EntityCollection<CustomerEntity> customers = new EntityCollection<CustomerEntity>(new CustomerEntityFactory());
 
PrefetchPath2 prefetcher = new PrefetchPath2(EntityType.Customer);
prefetcher.Add(CustomerEntity.PrefetchPathOrders);
 
using(DataAccessAdapter adapter = new DataAccessAdapter()) {
	adapter.FetchEntityCollection(customers, null, prefetcher);
}

Now, with the lambda prefetching API you can do this:

using(var adapter = new DataAccessAdapter()) {
	var linq = new LinqMetaData(adapter);
 
	var customers = (from c in linq.Customers select c)
				.WithPath(path => path.Prefetch(c => c.Orders));
 
}

Likewise, if you wanted to filter the prefetched orders (eg, only return orders costing more than £10) and prefetch each order’s OrderDetail, you’d need to do something like this:

EntityCollection<CustomerEntity> customers = new EntityCollection<CustomerEntity>(new CustomerEntityFactory());
 
PrefetchPath2 prefetcher = new PrefetchPath2(EntityType.Customer);
prefetcher.Add(CustomerEntity.PrefetchPathOrders, 0, new PredicateExpression(OrderFields.TotalCost > 10))
	.SubPath.Add(OrderEntity.PrefetchPathOrderDetail);
 
using(DataAccessAdapter adapter = new DataAccessAdapter()) {
	adapter.FetchEntityCollection(customers, null, prefetcher);
}

…while now you can do this:

using(var adapter = new DataAccessAdapter()) {
	var linq = new LinqMetaData(adapter);
 
	var customers = (from c in linq.Customers select c)
				.WithPath(path =>
					path.Prefetch<OrderEntity>(c => c.Orders)
						.FilterOn(o => o.TotalCost > 10)
						.SubPath(orderPath => orderPath.Prefetch(o => o.OrderDetail))
				);
 
}

Lambdas rock

LLBLGen’s querying API is very powerful, but also somewhat complex. For example, to retrieve a list of orders from all customers in the Northwind database for customers in the UK, you would write something like this:

EntityCollection<OrderEntity> orders = new EntityCollection<OrderEntity>(new OrderEntityFactory());
RelationPredicateBucket bucket = new RelationPredicateBucket(CustomerFields.Country == "UK")
bucket.Relations.Add(OrderEntity.Relations.CustomerEntityUsingCustomerId)
using(DataAccessAdapter adapter = new DataAccessAdapter())
{
	adapter.FetchEntityCollection(orders, bucket)
}

Linq support maintains the type safety, whilst also allowing this query to be expressed in a more SQL-like fashion (which I personally find to be more intuitive):

using(DataAccessAdapter adapter = new DataAccessAdapter())
{
	var meta = new LinqMetaData(adapter);
	var query = from o in meta.Order
			where o.Customer.Country == "UK"
			select o;
 
	var results = query.ToList();
}

This got me thinking…now that there are several ORMs that have linq support, wouldn’t it be nice if there was a consistent method for performing linq-based queries, independent of the underlying ORM implementation. And thus the linq-repository was born.

The majority of the work is done by the BaseRepository class which acts as a wrapper around the underlying linq provider. There’s also an IRepository interface which the BaseRepository implements:

public interface IRepository<T> : IQueryable<T> where T : class
{
	void Save(T toSave);
	void Save(T toSave, bool isNew);
	void Delete(T toDelete);
}
 
public abstract class BaseRepository<T> : IRepository<T> where T : class
{
	private IQueryable<T> source;
 
	protected IQueryable<T> Source
	{
		get { return source; }
		set { source = value; }
	}
 
	protected BaseRepository(IQueryable<T> source)
	{
		this.source = source;
	}
 
	IEnumerator<T> IEnumerable<T>.GetEnumerator()
	{
		return source.GetEnumerator();
	}
 
	public IEnumerator GetEnumerator()
	{
		return source.GetEnumerator();
	}
 
	public Expression Expression
	{
		get { return source.Expression; }
	}
 
	public Type ElementType
	{
		get { return source.ElementType; }
	}
 
	public IQueryProvider Provider
	{
		get { return source.Provider; }
	}
 
	public abstract void Save(T toSave);
	public abstract void Delete(T toDelete);
	public abstract void Save(T toSave, bool isNew);

Note that the constructor for BaseRepository takes an IQueryable representing the underlying linq provider. Now, the LLBLGen-specific subclass:

public class LLBLGenRepository<T> : BaseRepository<T> where T : EntityBase2, IEntity2, new()
{
	public IDataAccessAdapter Adapter { get; private set; }
 
	public LLBLGenRepository(IDataAccessAdapter adapter, IElementCreator2 elementCreator)
			: base(CreateQuery(adapter, elementCreator))
	{
		this.Adapter = adapter;
	}
 
	private static IQueryable CreateQuery(IDataAccessAdapter adapter, IElementCreator2 elementCreator)
	{
		return new DataSource2(adapter, elementCreator, functionMappings, null);
	}
 
	public override void Save(T toSave)
	{
		if (toSave == null)
			throw new ArgumentNullException("toSave");
 
		Adapter.SaveEntity(toSave);
	}
 
	public override void Save(T toSave, bool isNew)
	{
		if (toSave == null)
			throw new ArgumentNullException("toSave");
 
		if (isNew) toSave.IsNew = true;
 
		Adapter.SaveEntity(toSave);
	}
 
	public override void Delete(T toDelete)
	{
		Adapter.DeleteEntity(entity);
	}

Note that the constructor for the LLBLGenRepository takes instances of an IDataAccessAdapter and an IElementCreator (the two objects necessary for running linq-queries against LLBLGen) and creates a DataSource2 object (the LLBLGen query provider) which is then passed to the BaseRepository’s constructor.

So, I can now write code like this:

IRepository<OrderEntity> orders = new LLBLGenRepository<OrderEntity>(new DataAccessAdapter(), new ElementCreator());
 
var query = from o in orders
		where o.Customer.Country == "UK"
		select o;

To remove the calls to new DataAccessAdapter() and new ElementCreator(), I moved the responsibility for instantiating the repository over to an IoC container:

//in my application startup routine:
IoC.Initialise(new WindsorContainer());
IoC.Container.AddComponent<IDataAccessAdapter, DataAccessAdapter>();
IoC.Container.AddComponent<IElementCreator2, ElementCreator>();
IoC.Container.AddComponent("Repository", typeof(IRepository<>), typeof(LLBLGenRepository<>));

The IoC static class is simply a wrapper for the Windsor container.

Now I can instantiate repositories like this:

IRepository<OrderEntity> orders = IoC.Resolve<IRepository<OrderEntity>>();

I don’t really like this, so I wrap it in a static gateway:

public static class Repository
{
	public static IRepository<T&t; For<T>() where T : class, new()
	{
		return IoC.Resolve<IRepository<T>>();
	}
}

…and now I can write queries like this:

var query = from o in Repository.For<OrderEntity>
		where o.Customer.Country == "UK"
		select o;

Alternatively, now that the repository is registered with Windsor, I can inject the repository directly into my ASPNET MVC Controllers:

public class OrdersController : ConventionController
{
	private IRepository<OrderEntity> ordersRepository;
 
	public OrdersController(IRepository<OrderEntity> repository)
	{
		this.ordersRepository = repository;
	}
 
	public void OrdersFromCustomersInTheUk()
	{
		ViewData["orders"] = from o in ordersRepository
						where o.Customer.Country == "UK"
						select o;
	}
}