An explanation of Linq query providers

Linq query providers appear all over the place. Some say "Linq to Everything" to refer to all flavors of Linq that are out there: Linq to Objects, XML, SQL, DataSets, SharePoint, Amazon, Flickr, and so on. These are generally referred to as Linq query providers. However, a query provider is something different. Here’s how.

A Linq query provider is any API that implements the query pattern. So, next question: what is a query pattern? A query pattern is a piece of code that will make a reasonably normal query expression compile (and usually do something). You can look back at the Linq quiz question of Bart de Smet to see that this can be achieved in another way.

To understand how this works, remember that the compiler expands a query expression like this:

var query = from p in Process.GetProcesses()

            where p.ProcessName.Contains("System")

            orderby p.Modules.Count

            select p;

into explicit dot notation like so:

var query = Process.GetProcesses()

            .Where(p => p.ProcessName.Contains("System"))

            .OrderBy(p => p.Modules.Count)

            .Select(p => p);

If you can make the compiler happy to compile the latter, you have created an implementation of the query pattern and hence a Linq query provider by means of your class(es) that are used, i.e. the source (Process.GetProcesses in the sample above) and the return types of the subsequent query operators.

Microsoft’s implementation in the System.Core assembly ships with two query providers, just two.

One implementation uses an in-memory evaluation using yield operators. It is a large set of extension methods that extend the IEnumerable<T> implementing types. The extension methods are all of the common Linq query operators, amongst which the Where, OrderBy and Select that you saw in the second code fragment. These methods are defined in the System.Linq.Enumerable static class. This query provider could be called the ‘IEnumerable provider’, or captured as ‘Linq to Objects’. Both Linq to XML and Linq to DataSets (two Linq flavors that ship with .NET Framework 3.5) are both using the IEnumerable provider and will execute in-memory.

The second query provider uses the extension methods like Linq to Objects, except that the extension methods extend the IQueryable<T> interface. The internal implementation of the extension methods do not use iterators, but built an expression tree of the entire query expression instead.

IQueryable<T> and related interface are defined as follows:

public interface IQueryable<T> : IEnumerable<T>, IQueryable, IEnumerable




public interface IQueryable : IEnumerable


  Type ElementType { get; }

  Expression Expression { get; }

  IQueryProvider Provider { get; }



public interface IQueryProvider


  IQueryable CreateQuery(Expression expression);

  IQueryable<TElement> CreateQuery<TElement>(Expression expression);

  object Execute(Expression expression);

  TResult Execute<TResult>(Expression expression);


This query provider could be called the IQueryable provider. Most Linq to implementations that you have heard of Linq to LDAP, SharePoint, SQL, NHibernate and others are implementations that lift on the back of IQueryable. How are they implemented? The answer is ‘by implementing a Queryable provider’. In most cases such an implementation will translate the expression tree to another query language (e.g. an LDAP, WMI or CAML) or some API (e.g. Amazon Web services or Flickr REST API). The isolation of the part that does the translation is done by a provider model design pattern.

As you can see the IQueryable interface holds the property for the provider. Someone who wants to create a Linq query provider based on translating a expression tree of the Linq query expression can do so by creating an implementation of the IQueryProvider interface. The name of the interface is a little confusing at first. It is not a way to create a whole new query provider (i.e. implementation of the query pattern) per se, just a provider for the IQueryable provider model.

Matt Warren, one of the brains behind ObjectSpaces and Linq to SQL has a series on creating your own provider for IQueryable. Quite aptly it is named: "LINQ: Building an IQueryable Provider". Go read it if you want to know more on creating your own Linq to Something.

Hope this helped. Comments are welcome if you have questions.

This entry was posted in Uncategorized. Bookmark the permalink.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s