AssignAsync Extension Method for ADO.NET Data Services

ADO.NET Data Services is a rapidly evolving set of tools that provides data access to remote clients through a set of REST-based services.  The Data Services Client Library for .NET performs the magic of translating your Linq queries to URLs and passing them to the data service back-end, as well as retrieving results and hydrating objects in the client to represent them.

After running into a number of problems with the current CTP of RIA Services (see my article), I decided to fall back on Data Services to provide data access in my newest project.  Data Services has the advantage of allowing you to write fairly normal Linq queries against Entity Framework entity sets, and entity data models can reside in a dedicated data model assembly (instead of requiring them to be part of the web project).

One of the differences that remain when using Data Services in Silverlight—as opposed to accessing an Entity Framework ObjectContext directly—is that Silverlight doesn’t allow asynchronous calls.  So code like this, which would force a synchronous call (with FirstOrDefault), will fail in Silverlight:

var result = (from p in context.Properties
              where p.Required
              select p).FirstOrDefault();

This forces us to adopt some new patterns for data access.  This isn’t a bad thing, however.  And it’s an inevitable transition we’re making to asynchronous, concurrent program logic.

Here’s a typical example of querying data with Data Services in Silverlight:

var RequiredProperties = from p in context.Properties
                         where p.Required
                         select p;

var dsq = RequiredProperties as DataServiceQuery<Node>;
dsq.BeginExecute(ar =>
    {
        var result = dsq.EndExecute(ar);
        // do something with the the result
    }, null);

When using a lambda statement for brevity, the syntax isn’t too bad, but the pattern gets a little more involved when you include error handling logic.  If EndExecute fails, you’ll need the ability to perform some compensating action.

So what I’ve done to keep my client code simple is to define an extension method called AssignAsync that encapsulates this whole pattern.

public static class DataServicesExtensions
{
    public static void AssignAsync<T>(this IEnumerable<T> expression,
        Action<IEnumerable<T>> Assignment,
        Action<Exception> Fail)
    {
        var dsq = expression as DataServiceQuery<T>;
        dsq.BeginExecute(ar =>
            {
                IEnumerable<T> result = null;
                try
                {
                    result = dsq.EndExecute(ar) as IEnumerable<T>;
                }
                catch (Exception ex)
                {
                    Fail(ex);
                    return;
                }
                Assignment(result);
            }, null);
    }
}

This enables me to write the following code:

var RequiredProperties = from p in context.Properties
                         where p.Required
                         select p;

RequiredProperties.AssignAsync(result => properties = result,
    ex => Debug.WriteLine(ex));

In other words: if the query succeeds, assign the result to the properties collection; if it fails, send the exception object to Debug output.  Either action can be used to send signals to other parts of your application that will respond appropriately.  Instead of Debug.WriteLine, you might add the exception object to some collection that triggers an error dialog to appear and your logging framework to record the event.  Instead of assigning the result to a simple collection, you could convert it to an ObservableCollection and assign it to an ItemsControl in WPF or Silverlight.  Anything is possible.

As I explore Data Services further, I will be looking for ways to share query and other model-centric logic between Silverlight and non-Silverlight clients.  I suspect that the same asynchronous patterns can be used in non-Silverlight projects as well, and that those projects will benefit from this query style.

Problems with RIA Services (Feedback for July 2009 CTP)

RIA Services (new home page) is a collection of tools and libraries for making Rich Internet Applications, especially line of business applications, easier to develop.  Brad Abrams did a great presentation of RIA Services at MIX 2009 that touches on querying, validation, authentication, and how to share logic between the server and client sides.  Brad also has a huge series of articles (26 as I write this) on using Silverlight and RIA Services to build a realistic application.

I love the concept of RIA Services.  Brad and his team have done a fantastic job of identifying the critical issues for LOB systems and have the right idea to simplify those common data access tasks through the whole pipeline from database to UI controls, using libraries, Visual Studio tooling, or whatever it takes to get the job done.

So before I lay down some heavy criticism of RIA Services, take into consideration that it’s still a CTP and that my scenario pushes the boundaries of what was likely conceived of for this product, at least for such an early stage.

Shared Data Model with WPF & Silverlight Clients

The cause of so much of my grief with RIA Services has been my need to share a data model, and access to a shared database, across WPF as well as Silverlight client applications.  Within the constraints of this situation, I keep running into problem after problem while trying to use RIA Services productively.

The intuitive thing to do is: define a single data model project that compiles to a single assembly, and then reference that in my Silverlight and non-Silverlight projects.  This would be a 100% full-fidelity shared data model.  As long as the code I wrote was a subset of both Silverlight and normal .NET Frameworks (an intersection), we could share identical types and write complex validation and model manipulation logic, all without having to constrain ourselves to work within the limitations of a convoluted code generation scheme.  Back when I wrote Compact Framework applications, I did this with great success despite the platform gap, and I didn’t have anything like RIA Services to help.

Incompatible Assemblies

Part of the problem arises because Silverlight assemblies are incompatible with non-Silverlight assemblies.  A lot of what RIA Services is doing is trying to find a way around this limitation: picking up attributes and code files from one project and inserting that code into the Silverlight project with a build action.  This Visual Studio “magic” has been criticized for its weakness in dealing with multiple-solution systems where Visual Studio can’t update the client because it’s not loaded, and I’ve heard there’s work being done to address this, but for my current needs, this magic aspect of it isn’t a problem.  The specifics of how it works, however, are.

Different Data Access APIs

Accessing entities requires a different API in Silverlight via RiaContextBase versus ObjectContext elsewhere.  Complex logic in the model (for validation and other actions against the model) requires access to other entities and therefore access to the current object context, but the context APIs for Silverlight and WPF are very different.  Part of this has to do with Silverlight’s inability to make synchronous calls to the server.

In significantly large systems that I build, I use validation logic such as “this entity is valid if it’s pointing to an entity of a different type that contains a PropertyX value of Y”.  One of my tables stores a tree of data, so I have methods for loading entire subtrees and ensuring that no circular references exist.  For these kinds of tasks, I need access to the data context in basic validation methods.  When I delete nodes from a tree, I need to delete child nodes, so update logic is part of the model that needs to be the same in every client.  I don’t want to define that multiple times for multiple clients.  I like to program very DRY.  In other words, I find myself in need of a shared model.

RIA Services doesn’t provide anything like type equivalence for a shared model, however.  Data model classes in Silverlight inherit from Entity, but EntityObject in WPF.  In the RIA Services domain context, we RaiseDataMemberChanged, but in a normal EF object context, we need to ReportPropertyChanged.  In WPF, I can call MyEntity.Load(MergeOption.PreserveChanges), but in Silverlight there’s no Load method on the entity and no MergeOption enum.  In WPF I can query against context.SomeEntitySet, but in Silverlight you would query against context.GetSomeEntitySetQuery() and then execute the query with another method call.

This chasm of disparity makes all but the simplest shared model logic impractical and frustrating.  The code generation technique, though good in principle, keeps getting in the way.  For example, I have both parameterless and parameterized constructors in my entity classes.  This works great in my WPF client, but when this code is synchronized to my Silverlight client, I get an error because the Silverlight-side entity class is generated in two parts: in the hidden partial class, a parameterless constructor is generated which calls partial method OnCreated; and in the visible partial class, the constructor method I defined on the server is dumped into another file, so I have duplicate constructors.  If I remove the parameterless constructor from the server side, I get an error because my entity class requires a parameterless constructor (and defining a non-default constructor effectively removes the default one from the resulting type unless it’s explicitly defined).  I thought I could define the partial method OnCreated and put my construction logic in there, but the partial method is only defined on the client side.  That means sharing construction logic consists of copying and pasting the OnCreated method across the various clients—far from an ideal solution.

Entity Data Model Required to be in Web Project

Another strategy I attempted was to define the .edmx file and my partial class extensions in a class library, and then reference that from the web project.  I could define the LinqToEntitiesDomainService<MyDataContext>, but sharing entity class code (by generating code in the Silverlight project) isn’t possible unless the .edmx file and partial class extensions are defined in the web project itself.  This would mean that my WPF client would have to reference a web project for data access, which by itself seems wrong.  (Or making a copy of the data model, which is worse.)  It would be better for the WPF client to talk to the same domain service as the Silverlight client, but RIA Services doesn’t give you an option to link that web project to a non-Silverlight project, so again I ran into a brick wall.

So Don’t Do That

The kind of advice I’m getting for this is, “so don’t do that”.  In other words, don’t write complex validation logic in the model or otherwise try to access the data context; don’t write parameterized constructors; don’t aim for 100% type fidelity across all endpoints of a system; don’t try to share data models with Silverlight and non-Silverlight projects, etc.  But I see the potential for RIA Services, so I have to push for these things unless I hear really convincing arguments against them (or compelling alternatives).

Conclusion

The fact that there are different data contexts and data item definitions within those contexts imposes a burden on the developer to use different techniques for each environment, and creates challenges for centralizing data model logic and reusing equivalent logic across different kinds of clients.  My gut feeling is that RIA Services in its current form has some fundamental design flaws that will need to be addressed, taking into consideration systems with a mix of Silverlight, WPF, and other clients, before it becomes a truly robust data access platform.

Filtering with Metadata in the Managed Extensibility Framework

The Managed Extensibility Framework (MEF) is the new extensibility framework from Microsoft.  Pioneered by Glenn Block in the patterns & practices group, and leveraged by the behemoth Visual Studio 2010, it has a striking resemblance to my own Inversion of Control (IoC) and Dependency Injection (DI) framework—which led to me to have a couple great conversations about IoC with Glenn at Tech Ed 2008 and then again at PDC 2008.

But MEF isn’t really written to be your IoC.  Instead, the IoC engine and DI aspects are implementation details, allowing you to do really no more than “MEF things together”.  The core concept of MEF is to provide very simple and powerful application composability.  Not in the user interface composition sense—for that, see Prism for WPF and Silverlight (explained in MSDN Magazine, September 2008)—but for virtually all other dynamic component assembly needs, MEF is your best friend.

The two things I like most about MEF is its simplicity as its lack of presumption on how it will be used.  Compose collections of strings, single method delegates, or implementations of complex services.  All you’re doing is importing and exporting things, with little code required to wire things up.

MEF is currently in its seventh preview release, so expect beta-like quality.  My own experience with it has been very positive, but there are a number of shortcomings in the API.  This article is about a few of them and what can be done to add some much-needed functionality.

System.AddIn vs. MEF

There’s been some confusion with Microsoft coopetition among products with similar aims, and extensibility and composition are no exception.  The AddIn API (team blog) serves a similar purpose as MEF.  (See this two-part MSDN article on System.AddIn: first and second.)  The primary differentiator, from my understanding, is that the AddIn API is a bit more robust and a lot more complicated, and supports such things as isolating extensions in separate AppDomains.

With Visual Studio siding with MEF, it’s personally hard for me to imagine using the AddIn API.  If MEF is flexible and robust enough for Visual Studio, is it really likely to fall short for my own much smaller software systems?  Krzysztof Cwalina suggests they are complementary approaches, but I find that hard to swallow.  Why would I want to use two different extensibility frameworks instead of one coherent API?  If anything, I imagine that the lessons learned from the AddIn API will eventually migrate to MEF.

Daniel Moth notes that with the AddIn API, “there are many design decisions to make and quite a few subtleties in implementing those decisions in particular when it comes to discovering addins, version resiliency, isolation from the host etc.”  A customer of mine using the AddIn API was using a Visual Studio plug-in to manage pipelines, and things were a real mess.  There were a bunch of assemblies, a lot of generated code, and not much clarity or confidence that it was all really necessary.

MEF: Import & ImportMany

In MEF, the Import attribute allows you to inject a value that is exported somewhere else using the Export attribute—typically from another assembly.  There is also an ImportMany attribute which is useful when you expect several exports that use the same contract.  By defining an IEnumerable<T> field or property and decorating it with the ImportMany attribute, all matching exports will be added to an enumerable type.

[ImportMany]
public IEnumerable<IVehicle> Vehicles;

What if you want to filter the exported vehicle types by some kind of metadata, though?  Let’s take a look at the IVehicle contract and some concrete classes that implement the contract.

public interface IVehicle { }

[Export(typeof(IVehicle))]
[ExportMetadata("Speed", "Slow")]
public class ToyotaPrius : IVehicle
{
    public ToyotaPrius() { }
}

[Export(typeof(IVehicle))]
[ExportMetadata("Speed", "Fast")]
public class LamborghiniDiablo : IVehicle
{
    public LamborghiniDiablo() { }
}

The object model isn’t very interesting, but that’s not the point.  What is interesting is that MEF allows us to supply metadata corresponding to our exports.  In this case, my contrived example has defined a metadata variable of “Speed”, with two possible values: “Fast” and “Slow”.  The variable name must be a string, but its value can be any value; that is, any value that’s supported from within an attribute, which means string literals and constants, type objects, and the like.

Filtering Imports on Metadata

What if you want to ImportMany for all exports that have a particular metadata value?  Unfortunately, there are no such options in the ImportMany attribute class.

In my scenario, I’ve defined a static factory class called VehicleFactory, which at some imaginary point in the future will be responsible for building a city full of trafic.

public static class TrafficFactory
{
    // type initialization fails without a static constructor
    static TrafficFactory() { }

    public static IEnumerable<IVehicle> SlowVehicles =
        App.Container.GetExportedValues<IVehicle>(metadata => metadata.ContainsKeyWithValue("Speed", "Slow"));

    public static IEnumerable<IVehicle> FastVehicles =
        App.Container.GetExportedValues<IVehicle>(metadata => metadata.ContainsKeyWithValue("Speed", "Fast"));

    public static IDictionary<object, IVehicle> AllVehicles =
        App.Container.GetKeyedExportedValues<IVehicle>("Speed");
}

This is what I want to do, but there is no overload of GetExportedValues that supplies a metadata-dependent predicate function.  Adding one is easy, though.  While we’re at it, we’ll also add the ContainsKeyWithValue which I borrow from The Code Junky article also on MEF container filtering.

public static class IDictionaryExtensions
{
    public static bool ContainsKeyWithValue<KeyType, KeyValue>(
        this IDictionary<KeyType, ValueType> Dictionary,
        KeyType Key, ValueType Value)
    {
        return (Dictionary.ContainsKey(Key) && Dictionary[Key].Equals(Value));
    }
}

public static class MEFExtensions
{
    public static IEnumerable<T> GetExportedValues<T>(this CompositionContainer Container,
        Func<IDictionary<string, object>, bool> Predicate)
    {
        var result = new List<T>();

        foreach (var PartDef in Container.Catalog.Parts)
        {
            foreach (var ExportDef in PartDef.ExportDefinitions)
            {
                if (ExportDef.ContractName == typeof(T).FullName)
                {
                    if (Predicate(ExportDef.Metadata))
                        result.Add((T)PartDef.CreatePart().GetExportedValue(ExportDef));
                }
            }
        }

        return result;
    }
}

Now we can test this logic by wiring up MEF and then accessing the two filtered collections of cars, which will each contain a single IVehicle instance.

class App
{
    [Export]
    public CompositionContainer Container;

    static void Main(string[] args)
    {
        AssemblyCatalog catalog = new AssemblyCatalog(Assembly.GetExecutingAssembly());
        Container = new CompositionContainer(catalog);
        Container.ComposeParts();

        var FastCars = TrafficFactory.FastVehicles;
        var SlowCars = TrafficFactory.SlowVehicles;
    }
}

Viola!  We have metadata-based filtering.

You’ll also noticed that I added an Export attribute to the Container itself.  By doing this, you can Import the container into any module that gets dynamically loaded.  It’s not used in this article, but getting to the container from a module is otherwise impossible without some kind of work-around.  (Thanks for pointing out the problem, Damon.)

Using Metadata to Assign Dictionary Keys

Let’s take this one step further.  Let’s say you want to import many instances of MEF exported values into a Dictionary, using one of the metadata properties as the key.  This is how I’d like it to work:

public static IDictionary<object, IVehicle> AllVehicles =
    App.Container.GetKeyedExportedValues<IVehicle>("Speed");

Again, the current MEF Preview doesn’t support this, but another extension method is all we need.  We’ll add two, so that one version gives us all exported values and the other allows us to filter that selection based on other metadata.

public static IDictionary<object, T> GetKeyedExportedValues<T>(this CompositionContainer Container,
    string MetadataKey, Func<IDictionary<string, object>, bool> Predicate)
{
    var result = new Dictionary<object, T>();

    foreach (var PartDef in Container.Catalog.Parts)
    {
        foreach (var ExportDef in PartDef.ExportDefinitions)
        {
            if (ExportDef.ContractName == typeof(T).FullName)
            {
                if (Predicate(ExportDef.Metadata))
                    result.Add(ExportDef.Metadata[MetadataKey],
                        (T)PartDef.CreatePart().GetExportedValue(ExportDef));
            }
        }
    }

    return result;
}

public static IDictionary<object, T> GetKeyedExportedValues<T>(this CompositionContainer Container,
    string MetadataKey)
{
    return GetKeyedExportedValues<T>(Container, MetadataKey, metadata => true);
}

Add an assignment to TrafficFactory.AllVehicles in the App.Main method and see for yourself that it works.

If you’re using metadata values as Dictionary keys, it’s probably important for you not to mess them up.  I recommend using enum values for both metadata property names as well as valid values if it’s possible to enumerate them, and string const values otherwise.

Now go forth and start using MEF!

Better Tool Support for .NET

Productivity Enhancing Tools

Visual Studio has come a long way since its debut in 2002.  With the imminent release of 2010, we’ll see a desperately-needed overhauling of the archaic COM extensibility mechanisms (to support the Managed Package Framework, as well as MEF, the Managed Extensibility Framework) and a redesign of the user interface in WPF that I’ve been pushing for and predicted as inevitable quite some time ago.

For many alpha geeks, the Visual Studio environment has been extended with excellent third-party, productivity-enhancing tools such as CodeRush and Resharper.  I personally feel that the Visual Studio IDE team has been slacking in this area, providing only very weak support for refactorings, code navigation, and better Intellisense.  While I understand their desire to avoid stepping on partners’ toes, this is one area I think makes sense for them to be deeply invested in.  In fact, I think a new charter for a Developer Productivity Team is warranted (or an expansion of their team if it already exists).

It’s unfortunately a minority of .NET developers who know about and use these third-party tools, and the .NET community as a whole would without a doubt be significantly more productive if these tools were installed in the IDE from day one.  It would also help to overcome resistance from development departments in larger organizations that are wary of third-party plug-ins, due perhaps to the unstable nature of many of them.  Microsoft should consider purchasing one or both of them, or paying a licensing fee to include them in every copy of Visual Studio.  Doing so, in my opinion, would make them heroes in the eyes of the overwhelming majority of .NET developers around the world.

It’s not that I mind paying a few hundred dollars for these tools.  Far from it!  The tools pay for themselves very quickly in time saved.  The point is to make them ubiquitous: to make high-productivity coding a standard of .NET development instead of a nice add-on that is only sometimes accepted.

Consider just from the perspective of watching speakers at conferences coding up samples.  How many of them don’t use such a tool in their demonstration simply because they don’t want to confuse their audience with an unfamiliar development interface?  How many more demonstrations could they be completing in the limited time they have available if they felt more comfortable using these tools in front of the masses?  You know you pay good money to attend these conferences.  Wouldn’t you like to cover significantly more ground while you’re there?  This is only likely to happen when the tool’s delivery vehicle is Visual Studio itself.  Damon Payne makes a similar case for the inclusion of the Managed Extensibility Framework in .NET Framework 4.0: build it into the core and people will accept it.

The Gorillas in the Room

CodeRush and Resharper have both received recent mention in the Hanselminutes podcast (episode 196 with Mark Miller) and in the Deep Fried Bytes podcast (episode 35 with Corey Haines).  If you haven’t heard of CodeRush, I recommend watching these videos on their use.

• CodeRush Express (which is free)
• CodeRush with Refactor!
• DXCore (the library you can use to build your own add-ins, and on which CodeRush and Refactor! are built)

For secondary information on CodeRush, DXCore, and the principles with which they were designed, I recommend these episodes of DotNetRocks:

• Tool Development
• Discoverability
• The Science of Good UI

I don’t mean to be so biased toward CodeRush, but this is the tool I’m personally familiar with, has a broader range of functionality, and it seems to get the majority of press coverage.  However, those who do talk about Resharper do speak highly of it, so I recommend you check out both of them to see which one works best for you.  But above all: go check them out!

Refactor – Rename

Refactoring code is something we should all be doing constantly to avoid the accumulation of technical debt as software projects and the requirements on which they are based evolve.  There are many refactorings in Visual Studio for C#, and many more in third-party tools for several languages, but I’m going to focus here on what I consider to be the most important refactoring of them all: Rename.

Why is Rename so important?  Because it’s so commonly used, and it has such far-reaching effects.  It is frequently the case that we give poor names to identifiers before we clearly understand their role in the “finished” system, and even more frequent that an item’s role changes as the software evolves.  Failure to rename items to accurately reflect their current purpose is a recipe for code rot and greater code maintenance costs, developer confusion, and therefore buggy logic (with its associated support costs).

When I rename an identifier with a refactoring tool, all of the references to that identifier are also updated.  There might be hundreds of references.  In the days before refactoring tools, one would accomplish this with Find-and-Replace, but this is dangerous.  Even with options like “match case” and “match whole word”, it’s easy to rename the wrong identifiers, rename pieces of string literals, and so on; and if you forget to set these options, it’s worse.  You can go through each change individually, but that can take a very long time with hundreds of potential updates and is a far cry from a truly intelligent update.

Ultimately, the intelligence of the Rename refactoring provides safety and confidence for making far-reaching changes, encouraging more aggressive refactoring practices on a more regular basis.

Abolishing Magic Strings

I am intensely passionate about any tool or coding practice that encourages refactoring and better code hygiene.  One example of such a coding practice is the use of lambda expressions to select identifiers instead of using evil “magical strings”.  From my article on dynamically sorting Linq queries, the use of “magic strings” would force me to write something like this to dynamically sort a Linq query:

Customers = Customers.Order("LastName").Order("FirstName", SortDirection.Descending);

The problem here is that “LastName” and “FirstName” are oblivious to the Rename refactoring.  Using the refactoring tool might give me a false sense of security in thinking that all of my references to those two fields have been renamed, leading me to The Pit of Despair.  Instead, I can define a function and use it like the following:

public static IOrderedEnumerable<T> Order<T>(this IEnumerable<T> Source,
    Expression<Func<T, object>> Selector, SortDirection SortDirection)
{
    return Order(Source, (Selector.Body as MemberExpression).Member.Name, SortDirection);
}

Customers = Customers.Order(c => c.LastName).Order(c => c.FirstName, SortDirection.Descending);

This requires a little understanding of the structure of expressions to implement, but the benefit is huge: I can now use the refactoring tool with much greater confidence that I’m not introducing subtle reference bugs into my code.  For such a simple example, the benefit is dubious, but multiply this by hundreds or thousands of magic string references, and the effort involved in refactoring quickly becomes overwhelming.

Coding in this style is most valuable when it’s a solution-wide convention.  So long as you have code that strays from this design philosophy, you’ll find yourself grumbling and reaching for the inefficient and inelegant Find-and-Replace tool.  The only time it really becomes an issue, then, is when accessing libraries that you have no control over, such as the Linq-to-Entities and the Entity Framework, which makes extensive use of magic strings.  In the case of EF, this is mitigated somewhat by your ability to regenerate the code it uses.  In other libraries, it may be possible to write extension methods like the Order method shown above.

It’s my earnest hope that library and framework authors such as the .NET Framework team will seriously consider alternatives to, and an abolition of, “magic strings” and other coding practices that frustrate otherwise-powerful refactoring tools.

Refactoring Across Languages

A tool is only as valuable as it is practical.  The Rename refactoring is more valuable when coding practices don’t frustrate it, as explained above.  Another barrier to the practical use of this tool is the prevalence of multiple languages within and across projects in a Visual Studio solution.  The definition of a project as a single-language container is dubious when you consider that a C# or VB.NET project may also contain HTML, ASP.NET, XAML, or configuration XML markup.  These are all languages with their own parsers and other language services.

So what happens when identifiers are shared across languages and a Rename refactoring is executed?  It depends on the languages involved, unfortunately.

When refactoring a C# class in Visual Studio, the XAML’s x:Class value is also updated.  What we’re seeing here is cross-language refactoring, but unfortunately it only works in one direction.  There is no refactor command to update the x:Class value from the XAML editor, so manually changing it causes my C# class to become sadly out of sync.  Furthermore, this seems to be XAML specific.  If I refactor the name of an .aspx.cs class, the Inherits attribute of the Page directive in the .aspx file doesn’t update.

How frequent do you think it is that someone would want to change a code-behind file for an ASP.NET page, and yet would not want to change the Inherits attribute?  Probably not very common (okay, probably NEVER).  This is a matter of having sensible defaults.  When you change an identifier name in this way, the development environment does not respond in a sensible way by default, forcing the developer to do extra work and waste time.  This is a failure in UI design for the same reason that Intellisense has been such a resounding success: Intellisense anticipates our needs and works with us; the failure to keep identifiers in sync by default is diametrically opposed to this intelligence.  This represents a fragmented and inconsistent design for an IDE to possess, thus my hope that it will be addressed in the near future.

The problem should be recognized as systemic, however, and addressed in a generalized way.  Making individual improvements in the relationships between pairs of languages has been almost adequate, but I think it would behoove us to take a step back and take a look at the future family of languages supported by the IDE, and the circumstances that will quickly be upon us with Microsoft’s Oslo platform, which enables developers to more easily build tool-supported languages (especially DSLs, Domain Specific Languages). 

Even without Oslo, we have seen a proliferation of languages: IronRuby, IronPython, F#, and the list goes on.  A refactoring tool that is hard-coded for specific languages will be unable to keep pace with the growing family of .NET and markup languages, and certainly unable to deal with the demands of every DSL that emerges in the next few years.  If instead we had a way to identify our code identifiers to the refactoring tool, and indicate how they should be bound to identifiers in other languages in other files, or even other projects or solutions, the tools would be able to make some intelligent decisions without understanding each language ahead of time.  Each language’s language service could supply this information.  For more information on Microsoft Oslo and its relationship to a world of many languages, see my article on Why Oslo Is Important.

Without this cross-language identifier binding feature, we’ll remain in refactoring hell.  I offered a feature suggestion to the Oslo team regarding this multi-master synchronization of a model across languages that was rejected, much to my dismay.  I’m not sure if the Oslo team is the right group to address this, or if it’s more appropriate for the Visual Studio IDE team, so I’m not willing to give up on this yet.

A Default of Refactor-Rename

The next idea I’d like to propose here is that the Rename refactoring is, in fact, a sensible default behavior.  In other words, when I edit an identifier in my code, I more often than not want all of the references to that identifier to change as well.  This is based on my experience in invoking the refactoring explicitly countless times, compared to the relatively few times I want to “break away” that identifier from all the code that references.

Think about it: if you have 150 references to variable Foo, and you change Foo to FooBar, you’re going to have 150 broken references.  Are you going to create a new Foo variable to replace them?  That workflow doesn’t make any sense.  Why not just start editing the identifier and have the references update themselves implicitly?  If you want to be aware of the change, it would be trivial for the IDE to indicate the number of references that were updated behind the scenes.  Then, if for some reason you really did want to break the references, you could explicitly launch a refactoring tool to “break references”, allowing you to edit that identifier definition separately.

The challenge that comes to mind with this default behavior concerns code that spans across solutions that aren’t loaded into the IDE at the same time.  In principle, this could be dealt with by logging the refactoring somewhere accessible to all solutions involved, in a location they can all access and which gets checked into source control.  The next time the other solutions are loaded, the log is loaded and the identifiers are renamed as specified.

Language Property Paths

If you’ve done much development with Silverlight or WPF, you’ve probably run into the PropertyPath class when using data binding or animation.  PropertyPath objects represent a traversal path to a property such as “Company.CompanyName.Text”.  The travesty is that they’re always “magic strings”.

My argument is that the property path is such an important construct that it deserves to be an core part of language syntax instead of just a type in some UI-platform-specific library.  I created a data binding library for Windows Forms for which I created my own property path syntax and type, and there are countless non-UI scenarios in which this construct would also be incredibly useful.

The advantage of having a language like C# understand property path syntax is that you avoid a whole class of problems that developers have used “magic strings” to solve.  The compiler can then make intelligent decisions about the correctness of paths, and errors can be identified very early in the cycle.

Imagine being able to pass property paths to methods or return then from functions as first-class citizens.  Instead of writing this:

Binding NameTextBinding = new Binding("Name") { Source = customer1; }

… we could write something like this, have access to the Rename refactoring, and even get Intellisense support when hitting the dot (.) operator:

Binding NameTextBinding = new Binding(@Customer.Name) { Source = customer1; }

In this code example, I use the fictitious @ operator to inform the compiler that I’m specifying a property path and not trying to reference a static property called Name on the Customer class.

With property paths in the language, we could solve our dynamic Linq sort problem cleanly, without using lambda expressions to hack around the problem:

Customers = Customers.Order(@Customer.LastName).Order(@Customer.FirstName, SortDirection.Descending);

That looks and feels right to me.  How about you?

Summary

There are many factors of developer productivity, and I’ve established refactoring as one of them.  In this article I discussed tooling and coding practices that support or frustrate refactoring.  We took a deep look into the most important refactoring we have at our disposal, Rename, and examined how to get the greatest value out of it in terms of personal habits, as well as long-term tooling vision and language innovation.  I proposed including property paths in language syntax due to its general usefulness and its ability to solve a whole class of problems that have traditionally been solved using problematic “magic strings”.

It gives me hope to see the growing popularity of Fluent Interfaces and the use of lambda expressions to provide coding conventions that can be verified by the compiler, and a growing community of bloggers (such as here and here) writing about the abolition of “magic strings” in their code.  We can only hope that Microsoft program managers, architects, and developers on the Visual Studio and .NET Framework teams are listening.

Strongly-Typed, Dynamic Linq Order Operator

A Community Solution

I love social technologies like Stack Overflow, where people can collaborate loosely to share knowledge and help get things done.  Stack Overflow does on a large scale what developer blogs like mine have been doing on a smaller scale: creating a community around the sharing of ideas and methods.

Every once in a while, I get some great feedback that includes a fix for one of my bugs, a performance tweak I didn’t realize was possible, or an extension to some library I left unfinished.

This morning I ran into a question about my dynamic Linq sort, solved and answered by “Ch00k”, allowing one to get compile-time checking of identifier names.  Well done!

(It’s too bad Stack Overflow doesn’t promote the use of real names for professional developers to better market themselves with skill and reputation.)

My original article (with source code) is here.  All I added to the library was this:

public static IOrderedEnumerable<T> Order<T>(this IEnumerable<T> Source,
    Expression<Func<T, object>> Selector, SortDirection SortDirection)
{
    return Order(Source, (Selector.Body as MemberExpression).Member.Name, SortDirection);
}

To test it, I used this code:

IEnumerable<Customer> Customers = new Customer[] { new Customer("Dan", "Vanderboom"), new Customer("Steve", "Vanderboom"),
    new Customer("Tracey", "Vanderboom"), new Customer("Sarah", "Barkelew") };

Customers = Customers.Order(c => c.LastName, SortDirection.Ascending);
Customers = Customers.Order(c => c.FirstName, SortDirection.Descending);

foreach (var cust in Customers)
{
    Console.WriteLine(cust.FirstName + " " + cust.LastName);
}

Now I can refactor these data model classes with a tool and all my dynamic sorting queries will stay in sync!

Multicasting with Silverlight 3 Local Messaging

[This article and the sample solution included were written with Silverlight 3 Beta.]

The very first thing I did to experiment with Silverlight 3’s new local messaging feature was to create an application with a listener name of “Everyone”, pop up multiple instances of the application, and try sending a message to all of the instances.  I got a nasty HRESULT E_FAIL exception message upon firing up the second instance.  I closed the application and restarted, only to find I got the same error message on the first instance as well (until I rebooted).

The problem was that a listener must have a unique name and I was violating that rule.  There are no groups, and multicasting to multiple receivers in the same group isn’t supported out of the box.  Because I didn’t dispose of the object, that name was never released.  This seems like a design flaw; when a Silverlight application instance ends, the Silverlight runtime should be able to detect that and release this name resource.

When I heard about this local message passing ability, my first thought was that it would create a neat opportunity, especially in out-of-browser applications, for multiple-window applications.  This would be great for those of us who use multiple monitors, as we could then slide panels around where we wanted them, taking full advantage of our workspace.

My sample application, which you can download here, consists of a TextBox, a Submit button that sends the content of that TextBox to all the other instances, and a TextBlock that displays important events.  The first time the application runs, it identifies itself as the master window.  All subsequent application runs identify themselves as child windows.  Here’s a screenshot of this application running out-of-browser:

To accomplish this, the master window will need to have a well known name.  I chose MyApp/Master to identify both the application and the window name.  Each of the child windows require a unique name, so I chose the format MyApp/{guid}.  Once an instance realizes there’s already a master window, it gives itself a child window GUID name and then registers that name with the master window.  When a child instance exits, it unregisters itself with the master instance.  And finally, when the master window exits, it informs all of the child windows (so they can shutdown, most likely).

I defined several static members in the App class itself, so they would be visible across pages, and also because I wanted to hook into Application_Exit and needed access from there.

public const string MasterWindowName = "MyApp/Master";
public static string WindowName;
public static Guid WindowID;
public static List<Guid> ChildWindows;

Hooking into the LocalMessageReceiver’s MessageReceived event, I looked for specific keywords in a protocol I quickly cooked up, and in most cases extracted a parameter by parsing the message string.  These commands are NewWindow, CloseWindow, MasterWindowClosed, and UpdateText.

void MessageReceiver_MessageReceived(object sender, MessageReceivedEventArgs e)
{
    if (e.Message.StartsWith("NewWindow:"))
    {
        if (App.ChildWindows == null)
            App.ChildWindows = new List<Guid>();

        Guid NewWindowID = new Guid(e.Message.Substring("NewWindow:".Length));

        App.ChildWindows.Add(NewWindowID);

        Log("New window detected, id = " + NewWindowID.ToString());

        return;
    }

    if (e.Message.StartsWith("CloseWindow:"))
    {
        var id = new Guid(e.Message.Substring("CloseWindow:".Length));

        if (App.WindowName == App.MasterWindowName)
            App.ChildWindows.Remove(id);

        Log("Closing window, id = " + id.ToString());

        return;
    }

    if (e.Message == "MasterWindowClosed" && App.WindowName != App.MasterWindowName)
    {
        Log("Master Window Closed");
        return;
    }

    if (e.Message.StartsWith("UpdateText"))
    {
        var text = e.Message.Substring("UpdateText:".Length);

        txtName.Text = text;

        // if this is the master window, then distribute to all child windows
        if (App.WindowName == "MyApp/Master")
            UpdateTextMulticast(text);

        return;
    }
}

As you can see, most of this is simple housekeeping code to track which application instances or windows are currently connected to the master window.  The UpdateText command calls the UpdateTextMulticast method, which looks like this:

private void UpdateTextMulticast(string Text)
{
    foreach (var id in App.ChildWindows)
    {
        var MessageSender = new LocalMessageSender("MyApp/" + id.ToString());
        MessageSender.SendAsync("UpdateText:" + txtName.Text);
    }
}

If the window is a child window, the Submit button sends a message to the master window; the master window, when clicking on Submit, calls the UpdateTextMulticast method.

private void btnSubmit_Click(object sender, RoutedEventArgs e)
{
    if (App.WindowName == "MyApp/Master")
    {
        UpdateTextMulticast(txtName.Text);
    }
    else
    {
        var MessageSender = new LocalMessageSender("MyApp/Master");
        MessageSender.SendAsync("UpdateText:" + txtName.Text);
    }
}

Finally, this is how a window alerts other windows that it’s closing (in App.xaml.cs):

private void Application_Exit(object sender, EventArgs e)
{
    if (WindowName != MasterWindowName)
    {
        var MessageSender = new LocalMessageSender(MasterWindowName);
        MessageSender.SendAsync("CloseWindow:" + WindowID);
    }
    else if (ChildWindows != null)
    {
        foreach (var id in ChildWindows)
        {
            var MessageSender = new LocalMessageSender("MyApp/" + id.ToString());
            MessageSender.SendAsync("MasterWindowClosed");
        }
    }
}

That’s about all there is to it.  Admittedly, having the same Silverlight application act as both master and child window might not be the best arrangement, and it certainly adds a little to the complexity of the code, but the sky is the limit as far as how the new local messaging feature of Silverlight 3 could be used.

What I’d really like to see is some kind of WCF customization that could define WCF services and host them specifically for consumption across this local messaging channel.  Doing so would eliminate the need for defining and parsing a protocol as I’ve done in this example, as WCF could handle the serialization and service method invocation.

Advanced Customization of a Silverlight ListBox

[This article and its solution are based on Silverlight 3 Beta and Blend 3 Beta.]

The more I work with Silverlight, the more impressed I am.  Though I do keep running into frustrating situations, I haven’t encountered nearly as many dead ends as I did writing Windows Forms applications.  But where I used to run into dead ends, I now find myself lost in a labyrinth of deeply-composed control hierarchies, dichotomized content controls, and numerous interrelated control and data templates.

But ultimately I can find a way to do what I set out to do.  That’s huge.  If the learning curve is treacherously steep and my solution to a problem is tricky and twisted, I can reassure myself of more fluent UI design in the future due to increased understanding.  The difficulty of solving these problems is due to complexity of the UI design itself, the immaturity of Silverlight and its APIs, and my own inexperience working with it.

You can download the finished solution here.

The Goal

When I set out to customize a list control, I didn’t start with the tools of Silverlight.  I sketched out a design that assumed anything would be possible, and decided to figure out later how to implement exactly that (in behavior and layout, not final visuals).  The mock-up below is similar to what I came up with, simplified to include only those elements I’m going to illustrate in this article.

The first thing you’ll notice is that the data template renders differently based on the data for that item.  I found a Code Project article by Anil Gupta on doing the same kind of thing.  This turns out to be the easy part.  (The space around and between items isn’t intended to be rendered as such, and was added only to emphasize the separate identities of the item templates.)

I also wanted each item in the list to be expandable to display more information and to host interactive controls like sliders (shown in the example) to manipulate the underlying data.  In noticing that this button and behavior, as well as the border, are common elements to each of the item templates, I decided that what I was looking at involved a ContentControl.  This new control would contain these common elements and a ContentPresenter which would be filled by the specific item type template (one for airplane, one for truck, one for boat).  That way, I could build a whole bunch of new templates for new item types and I wouldn’t have to worry about placing the button correctly or wiring up its behavior each time.

Though the illustration might suggest that the only difference among the templates is background color, I wanted to be able to completely differentiate them.  The only thing that would be standard would be a collapsed height of 32 to give a nice consistent vertical layout (and for this example, a standard expanded height of 64).  Inside, the controls and their layout could follow any design.

Unspecified at first, one of my presumptions was that the width of each item would fill all of the available space, which is the width of the ListBox minus borders, padding, and the vertical ScrollBar.  This would prove to be the most difficult challenge, and I found some discussion in the forums, but ultimately I would find my own way to solve it.

Finally, I wanted to do as much as possible in Blend.  XAML is fine for setting complex bindings and wiring up other things, but for drawing of graphics (editing templates), I wanted to leverage Blend as much as possible.

The Solution

First we need a data model, to know the shape of items that we’re binding to in our ListBox.  I used a simple example of a Vehicle base class and three derived types.  Elsewhere, I instantiate several of each type of vehicle and add them to the ListBox’s ItemsSource collection.

public abstract class Vehicle
{
    public string Manufacturer { get; set; }
    public string Model { get; set; }
    public double Price { get; set; }
}

public class Truck : Vehicle
{
    public bool HasFourWheelDrive { get; set; }
}

public class Boat : Vehicle
{
    public double HullWidth { get; set; }
}

public class Airplane : Vehicle
{
    public int MaxAltitude { get; set; }
}

I then created three UserControls, one for each vehicle type, and called them AirplaneTemplate, BoatTemplate, and TruckTemplate.  I gave each of them a DesignHeight of 64 to represent their expanded state, let their Width be Auto, and set HorizontalAlignment to Stretch.  I set the Height of each of the two Grid rows to 32, to ensure they wouldn’t stretch as the ContentControl hosting this content collapsed.

Selecting a Template Based on Item Data

There’s no way that I know of to write an expression in XAML that will bind to a different data template based on item data.  I also know of no way to write code behind a data template.  To get around these limitations, I created a data template called VehicleListDataTemplate that contains a single VehicleItemTemplate custom control which I could write code behind.  This control is a ContentControl, so it’s capable of drawing its own content as well drawing content passed into it.  The content that it supplies itself consists of the common UI elements: the border and the button to toggle the expansion or collapse of the item.

This control is mocked up like so, showing both collapsed and expanded states:

The control’s ContentPresenter, set with its Content property, would occupy the same space, although the button would be placed on top to ensure it was clickable.

This was my first custom Silverlight control (other than UserControls), so several things were new to me.  For one, defining a default control template in generic.xaml and writing a separate class file for behavior.  This is what the default template looks like:

<Style TargetType="local:VehicleItemTemplate">
    <Setter Property="Template">
        <Setter.Value>
            <ControlTemplate TargetType="local:VehicleItemTemplate">

                <Grid x:Name="Core" Background="{TemplateBinding Background}"
                      d:DesignHeight="32" Height="{TemplateBinding Height}"
                      d:DesignWidth="312" Width="Auto"
                      VerticalAlignment="Stretch" HorizontalAlignment="Stretch">

                    <Border VerticalAlignment="Stretch" CornerRadius="5,5,5,5"
                            BorderBrush="{TemplateBinding BorderBrush}" BorderThickness="2" />

                    <ContentPresenter
                        VerticalAlignment="{TemplateBinding VerticalContentAlignment}"
                        HorizontalAlignment="{TemplateBinding HorizontalContentAlignment}"
                        Content="{TemplateBinding Content}"
                        ContentTemplate="{TemplateBinding ContentTemplate}" Margin="0"/>

                    <Button x:Name="Expander" VerticalAlignment="Top" HorizontalAlignment="Right"
                            Margin="0,4,4,0" Width="28" Height="24"
                            BorderBrush="{TemplateBinding BorderBrush}"/>
                </Grid>

            </ControlTemplate>
        </Setter.Value>
    </Setter>
</Style>

And here it’s referenced by the data template:

<DataTemplate x:Key="VehicleListDataTemplate">
    <local:VehicleItemTemplate VerticalAlignment="Top" HorizontalAlignment="Left"
        Background="#0014145D" Margin="0,0,0,0" BorderBrush="#FF5063A5"
        d:DesignHeight="32" Height="32" d:DesignWidth="430"
        VerticalContentAlignment="Stretch" HorizontalContentAlignment="Stretch" />
</DataTemplate>

So far that’s not too bad.  We have a data template, which refers to VehicleItemTemplate (a ContentControl) that gives our common appearance and hosts a specific vehicle UserControl depending on the item data in question.  I count three layers so far, but unfortunately that isn’t enough.

Let’s take a look at how we set the content:

private void VehicleItemTemplate_Loaded(object sender, RoutedEventArgs e)
{
    var vehicle = DataContext as Vehicle;

    // vehicle will be null when this is executed in the designer
    if (vehicle == null)
        return;

    if (vehicle is Airplane)
        Content = new AirplaneTemplate();
    else if (vehicle is Truck)
        Content = new TruckTemplate();
    else if (vehicle is Boat)
        Content = new BoatTemplate();
}

Pretty simple: the DataContext is our item data object, we can inspect the type to figure out which one it is, and create a new vehicle UserControl of the appropriate matching type to set the Content.

To make it expand and collapse, we first need to get a reference to the button in our template, which is based on the parts I defined for the control.

[TemplatePart(Name = "Core", Type = typeof(FrameworkElement))]
[TemplatePart(Name = "Expander", Type = typeof(ButtonBase))]
public class VehicleItemTemplate : ContentControl

In the template, the Expander part must be some control that inherits from ButtonBase, and which therefore implements a Click event.  When the template is applied to the control at runtime, OnApplyTemplate is run, so we hook into that event there:

public override void OnApplyTemplate()
{
    base.OnApplyTemplate();
    ToggleButton = GetTemplateChild("Expander") as ButtonBase;
    ToggleButton.Click += new RoutedEventHandler(btnToggleSize_Click);
}

private void btnToggleSize_Click(object sender, RoutedEventArgs e)
{
    Duration duration = new Duration(TimeSpan.FromSeconds(0.2));

    Storyboard sb = new Storyboard();
    sb.Duration = duration;

    DoubleAnimation ani1 = new DoubleAnimation();
    ani1.Duration = duration;
    ani1.To = Height == 32 ? 64 : 32;
    Storyboard.SetTarget(ani1, this);
    Storyboard.SetTargetProperty(ani1, new PropertyPath("FrameworkElement.Height"));

    DoubleAnimation ani2 = new DoubleAnimation();
    ani2.Duration = duration;
    ani2.To = Height == 32 ? 64 : 32;
    Storyboard.SetTarget(ani2, Content as Control);
    Storyboard.SetTargetProperty(ani2, new PropertyPath("FrameworkElement.Height"));

    sb.Children.Add(ani1);
    sb.Children.Add(ani2);

    sb.Begin();
}

Now we have an animation that will smoothly expand or collapse our item and its content, and because I use the To property, we avoid jumping from one state to another.  Instead, if we click to expand and then click to collapse again, it will animate from its current position to the desired position.

Setting the Correct Width of ListBox Items

The biggest problem I had was in setting the correct width.  With all Widths set to Auto, each item in the list will take up only as much space as it needs.  They can be shorter or longer than the ListBox’s width, and each item could be rendered a different width (depending on the template).

The first thing I tried was to set the VehicleItemTemplate’s Width to the ActualWidth of the ListBox.  I didn’t have enough items in my list to see the vertical ScrollBar appear, but even without it, the borders of my item templates were being clipped by the right side of the ListBox, and I could see a gap of several pixels to the left as well as above and below each item.

With the default rendering of Silverlight being that nothing is drawn (border widths are zero, brushes are null, etc.), I find it odd that the ListBox assumes I want padding where I haven’t specified any.  After all, if I wanted this, couldn’t I add a Margin to my data template?

I removed the ListBox border, and finally added a ListBoxItem manually to the ListBox in Blend.  Right-clicking on that ListBoxItem, I edited a copy of the control template, shown in the screenshot below:

This turns out to be different from the data template defined earlier.  This ListBoxItem template is itself a ContentControl, and its content is my VehicleListTemplate… (which is another ContentControl that hosts the specific vehicle UserControls…).  See how confusing this can get?  I feel like Alice in Wonderland sometimes, seeing how far the Silverlight hole really goes.  I also wonder why there doesn’t appear to be any way to edit this template without manually creating a ListBoxItem, when it clearly matters even when you’re defining a data template.

It’s also in this ListBoxItem template that you can render visual decorators to indicate various visual states: Normal, MouseOver, Pressed, and so on.  You might expect to handle that in your data template, but that doesn’t seem to be the case.

Anyway, within that ListBoxItem template was this ContentPresenter.

<ContentPresenter x:Name="contentPresenter"
                  HorizontalAlignment="{TemplateBinding HorizontalContentAlignment}"
                  Margin="{TemplateBinding Padding}" Content="{TemplateBinding Content}"
                  ContentTemplate="{TemplateBinding ContentTemplate}"/>

The third line shows a Margin being bound to the Padding property.  I removed this Margin altogether, and the gratuitous extra space around my items disappeared, making me happy.

Once you have this custom ListBoxItem template, you need to do two things:

1. Delete the ListBoxItem you manually added in Blend.  Otherwise you’ll get an error when trying to set the ListBox’s ItemsSource property.
2. Set the ItemContainerStyle property of the ListBox to point to this new template.  Note that this is different from the ItemTemplate property which sets the data template.

The ListBox XAML should now look something like this:

<ListBox x:Name="VehicleList" HorizontalAlignment="Stretch"
         Margin="20,20,20,20" Width="Auto"
         BorderThickness="2,2,2,2" BorderBrush="#FF99A712"
         ItemTemplate="{StaticResource VehicleListDataTemplate}"
         ItemContainerStyle="{StaticResource ListBoxItemStyle}">
    <ListBox.Background>
        <LinearGradientBrush EndPoint="0.5,1" StartPoint="0.5,0">
            <GradientStop Color="#FF03021E"/>
            <GradientStop Color="#FF191651" Offset="1"/>
        </LinearGradientBrush>
    </ListBox.Background>
</ListBox>

Now we’re at a point where the item container itself isn’t adding any extra space, so if we go without borders or a vertical ScrollBar, everything fits just right… until the vertical ScrollBar appears.  This is close, but not good enough.  How can we take care of the space taken up by the vertical ScrollBar?

While I was digging through the ListBox template, I noticed that the ScrollViewer control had a property called ViewportWidth, and with some debugging saw that it was not quite as wide as the total ListBox width.  If only I had a reference to the ScrollViewer from within my VehicleItemTemplate ContentControl!  I looked for a while but found nothing.  You can call GetTemplateChild from within a control’s class but not from the outside because it’s a protected method.

I decided to cheat.  I created a new ListBox class that exposed the ScrollViewer as a property.  I felt it was safe to do so because ScrollViewer is a TemplatePart defined in the ListBox’s parts and states contract.

public class MyListBox : ListBox
{
    public MyListBox() : base() { }

    public ScrollViewer ScrollViewer
    {
        get { return GetTemplateChild("ScrollViewer") as ScrollViewer; }
    }
}

I also needed to be able to reference the MyListBox object from each VehicleItemTemplate, so I created a DependencyProperty to store that:

// store a reference to the MyListBox that contains this item
public static readonly DependencyProperty ParentListBoxProperty = DependencyProperty.Register(
    "ParentListBox", typeof(MyListBox), typeof(VehicleItemTemplate), new PropertyMetadata(null));

public MyListBox ParentListBox
{
    get { return GetValue(ParentListBoxProperty) as MyListBox; }
    set { SetValue(ParentListBoxProperty, value); }
}

Next, I set this new ParentListBox property from within the data template I defined earlier (using Element binding, which is new to Silverlight 3):

<DataTemplate x:Key="VehicleListDataTemplate">
    <local:VehicleItemTemplate VerticalAlignment="Top" HorizontalAlignment="Left"
        Background="#0014145D" Margin="0,0,0,0" BorderBrush="#FF5063A5"
        d:DesignHeight="32" Height="32" d:DesignWidth="430"
        VerticalContentAlignment="Stretch" HorizontalContentAlignment="Stretch"
        ParentListBox="{Binding ElementName=VehicleList, Mode=OneWay}"/>
</DataTemplate>

Finally, I replaced the ListBox I was using with MyListBox, and in the VehicleItemTemplate_Loaded method, I added the following data binding in code:

// set up data binding:
// ViewportWidth of the ListBox's ScrollViewer tells us how much space we have available
//(ListBox.ActualWidth - borders - scrollbar)
WidthBinding = new Binding();
WidthBinding.Source = ParentListBox.ScrollViewer;
WidthBinding.Path = new PropertyPath("ViewportWidth");
SetBinding(WidthProperty, WidthBinding);

When there are more items than will fit in the ListBox, the vertical ScrollBar appears, and the width of all the item templates shrinks to accomodate it.  When the ListBox itself widens or shrinks, it adjusts.  This seems to produce the perfect fit for items.  If you download the sample source code, you’ll notice I set the page to auto size, so when you resize the browser, the ListBox will grow and shrink along with it, and you can easily test it.  Also, if you have the ListBox almost full and you expand one of the items with the expander button, you’ll see it adjust then as well.

Here is the final product:

 

After all of that, we can finally rest assured that we’ll never see a horizontal ScrollBar in our ListBox.

Conclusion

There are several ListBox templates we didn’t take full advantage of: the ItemsPanel for the layout of items, the various embedded templates for parts such as ScrollBars, and the different states of the ListBoxItem template.  However, customization of these templates has been covered fairly well by other articles.

Being somewhat new to Silverlight, I’m curious to see how others would have accomplished the same things.  Is there an easier way to do some of this?  Are there some Silverlight API calls I could have used to reference the ListBox’s ScrollViewer, for example?

I spent many hours working out these details.  I hope I can spare some of you the trouble I encountered.  Happy Silverlight developing!

Windows Azure: Blobs and Blocks

I’ve been busy building a new cloud-based service for the past few weeks, using Windows Azure on the back end and Silverlight for the client.  One of the requirements of my service is to allow users to upload files to a highly scalable Internet storage system.  I’m experimenting with Azure’s blob storage for this, and I have a need to upload these blobs (Binary Large OBjects) in separate blocks.  There are two reasons I can tell why you’d want to do this:

1. Although blobs can be as large as 2 GB in the current technical preview, the largest blob you can put in one operation is 4 MB.  If your file is larger, you have to store separate blocks, and then put a block list to assemble them together and commit them as a blob.
2. If you want different users to upload different portions of a file, each user will have to upload individual blocks, and you’ll have to put the block list when all blocks are present.  This is something like a reverse BitTorrent or other P2P protocol.

My service needs to deal with separate blocks for the second reason, though the first is likely to be much more common.

Although there’s a good deal of information about blocks and blobs in the REST API for Azure Storage Services, piecing together code to make REST calls with all the appropriate headers (including authentication signatures) isn’t very fun.  Where is the .NET library to make it easy?

There is one, in fact.  If you’ve downloaded and installed the Azure SDK (Jan 2009), you’ll find a samples.zip file that needs to be unzipped, and the solutions built within it.  Particularly, you’ll need the StorageClient solution.  In it, you’ll find that you can save and load blobs (as well as use queues and table storage), but there’s nothing in the API that suggests it supports putting individual blocks, let alone putting block lists to combine all of those blocks into a blob.  The raw state of this API is unfortunate, but the Azure platform is in an early tech preview stage, so we can expect vast improvements in the future.

Until then, however, I dug into it and discovered that there actually was code to put blocks and commit block lists, but it wasn’t exposed in the API (in BlobContainerRest.PutLargeBlobImpl).  Rather, it was called only when the blob you try to put was over the 4 MB limit.  Taking this code and hacking it a bit, I extended the StorageClient library to provide this needed functionality.

First, add these abstract method definitions to the BlobContainer class (in BlobStorage.cs):

public abstract bool PutBlobBlockList(BlobProperties blobProperties,
    IEnumerable<string> BlockIDs, bool overwrite, string eTag);

public abstract bool PutBlobBlock(BlobProperties blobProperties, string BlockID,
    Stream stream, long BlockSize, bool overwrite, string eTag);

Next, you’ll need to add the implementations to the BlobContainerRest class (in RestBlobStorage.cs):

public override bool PutBlobBlock(BlobProperties blobProperties, string BlockID,
    Stream stream, long BlockSize, bool overwrite, string eTag)
{
    NameValueCollection nvc = new NameValueCollection();
    nvc.Add(QueryParams.QueryParamComp, CompConstants.Block);
    nvc.Add(QueryParams.QueryParamBlockId,
        Convert.ToBase64String(Encoding.Unicode.GetBytes(BlockID)));
    return UploadData(blobProperties, stream, BlockSize, overwrite, eTag, nvc);
}

public override bool PutBlobBlockList(BlobProperties blobProperties,
    IEnumerable<string> BlockIDs, bool overwrite, string eTag)
{
    bool retval = false;

    using (MemoryStream buffer = new MemoryStream())
    {
        XmlTextWriter writer = new XmlTextWriter(buffer, Encoding.UTF8);
        writer.WriteStartDocument();
        writer.WriteStartElement(XmlElementNames.BlockList);
        foreach (string id in BlockIDs)
        {
            writer.WriteElementString(XmlElementNames.Block,
                Convert.ToBase64String(Encoding.Unicode.GetBytes(id)));
        }
        writer.WriteEndElement();
        writer.WriteEndDocument();
        writer.Flush();
        buffer.Position = 0; //Rewind

        NameValueCollection nvc = new NameValueCollection();
        nvc.Add(QueryParams.QueryParamComp, CompConstants.BlockList);

        retval = UploadData(blobProperties, buffer, buffer.Length, overwrite, eTag, nvc);
    }

    return retval;
}

In order to test this, I added two buttons to an ASP.NET page, one to upload the blocks and put the block list, and a second to read the blob back to verify the write operations worked:

protected void btnUploadBlobBlocks_Click(object sender, EventArgs e)
{
    var account = new StorageAccountInfo(new Uri("http://127.0.0.1:10000/"), null, "devstoreaccount1",
        "Eby8vdM02xNOcqFlqUwJPLlmEtlCDXJ1OUzFT50uSRZ6IFsuFq2UVErCz4I6tq/K1SZFPTOtr/KBHBeksoGMGw==");
    var storage = BlobStorage.Create(account);
    var container = storage.GetBlobContainer("testfiles");

    if (!container.DoesContainerExist())
        container.CreateContainer();

    var properties = new BlobProperties("TestBlob");

    // put block 0

    var ms = new MemoryStream();
    using (StreamWriter sw = new StreamWriter(ms))
    {
        sw.Write("This is block 0.");
        sw.Flush();
        ms.Position = 0;

        var PutBlock0Success = container.PutBlobBlock(properties, "block 0", ms, ms.Length, true, null);
    }

    // put block 1

    ms = new MemoryStream();
    using (StreamWriter sw = new StreamWriter(ms))
    {
        sw.WriteLine("... and this is block 1.");
        sw.Flush();
        ms.Position = 0;

        var PutBlock1Success = container.PutBlobBlock(properties, "block 1", ms, ms.Length, true, null);
    }

    // put block list

    List<string> BlockIDs = new List<string>();
    BlockIDs.Add("block 0");
    BlockIDs.Add("block 1");

    var PutBlockListSuccess = container.PutBlobBlockList(properties, BlockIDs, true, null);
}

protected void btnTestReadBlob_Click(object sender, EventArgs e)
{
    var account = new StorageAccountInfo(new Uri("http://127.0.0.1:10000/"), null, "devstoreaccount1",
        "Eby8vdM02xNOcqFlqUwJPLlmEtlCDXJ1OUzFT50uSRZ6IFsuFq2UVErCz4I6tq/K1SZFPTOtr/KBHBeksoGMGw==");
    var storage = BlobStorage.Create(account);
    var container = storage.GetBlobContainer("testfiles");

    MemoryStream ms = new MemoryStream();
    BlobContents contents = new BlobContents(ms);
    container.GetBlob("TestBlob", contents, false);
    ms.Position = 0;

    using (var sr = new StreamReader(ms))
    {
        string x = sr.ReadToEnd();
        sr.Close();
    }
}

It’s nothing fancy, but if you put a breakpoint on the last sr.Close command, you’ll see that the value of x contains both blocks of data, equal to “This is block 0…. and this is block 1.”

A KeyedList Implementation: Syntax Subtleties for an Intuitive API

[This article is a follow-up on the theme of data structures started in my article on a Non-Binary Tree Data Structure.] 

Clear API Design

When designing object models, there are often times when a Dictionary is the best choice for rapid lookup and access of items.  However, when attempting to make those object models as intuitive and simple as possible, sometimes the fact that a dictionary is being used is an implementation detail and needn’t be exposed externally.

Take this model, for example:

class Database
{
    public IEnumerable<Table> Tables;

    public Database()
    {
        _Tables = new List<Table>();
    }
}

class Table
{
    public string TableName;

    public Table(string TableName)
    {
        this.TableName = TableName;
    }
}

This is nice and simple.  I can loop through the Tables collection like this:

Database db = new Database();

foreach (var table in db.Tables)
{
    Console.WriteLine(table.TableName);
}

So it's too bad that the consumer end of this has to change when we use a dictionary.  By making a change to our Database type...

class Database
{
    public Dictionary<string, Table> Tables;

    public Database()
    {
        Tables = new Dictionary<string, Table>();
    }
}

... we now have to specify the "Values" collection to get an IEnumerable<T> of the correct type.

foreach (var table in db.Tables.Values)
{
    Console.WriteLine(table.TableName);
}

The consumers of our API may not care about iterating through key-value pairs, but now they have to remember to use this Values property or face the wrath of red squiggles and compiler errors when they forget.  Of course, there’s a pattern we could use to hide this dictionary inner goo from the outside.

class Database
{
    public IEnumerable<Table> Tables { get { return _Tables.Values; } }
    private Dictionary<string, Table> _Tables { get; set; }

    public Database()
    {
        _Tables = new Dictionary<string, Table>();
    }
}

Now from the outside, we can foreach over db.Tables, but inside we can use the Dictionary for fast access to elements by key.

The Need for a Dictionary-List Hybrid

This is an either-or approach: that is, it assumes that the API consumer is better off with an IEnumerable collection and won’t have any need for keyed access to data (or even adding data, in this case).  How can we have the best of both words, with the ability to write this kind of code?

Database db = new Database();

foreach (var table in db.Tables)
{
    Console.WriteLine(table.TableName);
}

Table t = db.Tables["Customers"];

This is a hybrid of a Dictionary and a List.  (Don’t confuse this with a HybridDictionary, which is purely a dictionary with runtime-adapting storage strategies.)  It provides an IEnumerable<T> enumerator (instead of IEnumerable<K, T>), as well as an indexer for convenient lookup by key.

There’s another aspect of working with dictionaries that has always bugged me:

db.Tables.Add("Vendors", new Table("Vendors"));

This is repetitive, plus it says the same thing twice.  What if I misspell my key in one of these two places?  What I’d really like is to tell my collection which property of the Table class to use, and have it fill in the key for me.  How can I do that?  Well, I know I can select a property value concisely (in a compiler-checked and refactoring-friendly way) with a lambda expression.  So perhaps I can supply that expression in the collection’s constructor.  I decided to call my new collection KeyedList<K, T>, which inherits from Dictionary so I don’t have to do all the heavy lifting.  Here’s how construction looks:

Tables = new KeyedList<string, Table>(t => t.TableName);

Now I can add Table objects to my collection, and the collection will use my lambda expression to fill in the key for me.

Tables.Add(new Table("Vendors"));

How does this work, exactly?  Here's a first cut at our KeyedList class:

public class KeyedList<K, T> : Dictionary<K, T>, IEnumerable<T>
{
    private Func<T, K> LinkedValue;

    public KeyedList()
    {
        LinkedValue = null;
    }

    public KeyedList(Func<T, K> LinkedValue)
    {
        this.LinkedValue = LinkedValue;
    }

    public void Add(T item)
    {
        if (LinkedValue == null)
            throw new InvalidOperationException("Can't call KeyedList<K, T>.Add(T) " +
                "unless a LinkedValue function has been assigned");

        Add(LinkedValue(item), item);
    }

    public new IEnumerator<T> GetEnumerator()
    {
        foreach (var item in Values)
        {
            yield return item;
        }
        yield break;
    }
}

This is still pretty simple, but I can think of one thing that it’s missing (aside from a more complete IList<T> implementation).  With a collection class like this, with tightly-integrated knowledge about the relationship between the key property of an item and the key in the Dictionary, what happens when we change that key property in the item?  Suddenly it doesn’t match the dictionary key, and we have to remember to update this in an explicit separate step in our code whenever this happens.  It seems that this is a great opportunity to forget something and introduce a bug into our code.  How could our KeyedCollection class track and update this for us?

Unfortunately, there’s no perfect solution.  “Data binding” in .NET is weak in my opinion, and requires implementation of INotifyPropertyChanged in our classes to participate; and when it does so, we only get notification of the property name that changed (supplied as a string), and have no idea what the old value was unless we store that somewhere ourselves.  Automatically injecting all classes with data binding code isn’t practical, of course, even using AOP (since many BCL classes, for example, reside in signed assemblies).  Hopefully a future CLR will be able to perform some tricks, such as intelligently and dynamically modifing those classes, for which other class’s data binding code specify interest, so we can have effortless and universal data binding.

Now back to reality.  I want to mention that although my code typically works just as well in Compact Framework as it does in Full Framework, I’m going off the reservation here.  I’m going to be using expression trees, which are not supported in Compact Framework at all.

Expressions

The Expression class (in System.Linq.Expressions) is really neat.  With it, you can wrap a delegate type to create an expression tree, which you can explore and modify, and at some point even compile into a function which you can invoke.  The best part is that lambda expressions can be assigned to Expression types in the same way that they can be assigned to normal delegates.

Func<int> func = () => 5;
Expression<Func<int>> expr = () => 5;

The first line defines a function that returns an int, and a function is supplied as a lambda that returns the constant 5.  The second line defines an expression tree of a function that returns an int.  This extra level of indirection allows us to take a step back and look at the structure of the function itself in a precompiled state.  The structure is a tree, which can be arbitrarily complex.  You can think of this as a way of modeling the expression in a data structure.  While func can be executed immediately, expr requires that we compile it by calling the Compile method (which generates IL for the method and returns Func<int>).

int FuncResult1 = func.Invoke();
int FuncResult2 = func();

int ExprResult1 = expr.Compile().Invoke();
int ExprResult2 = expr.Compile()();

The first two lines are equivalent, as are the last two.  I just wanted to point out here, with the two ways of calling the functions, how they are in fact the same, even though the last line looks funky.

Synchronizing Item & Dictionary Keys

So why do we need expressions?  Because we need to know the name of the property we’ve supplied in our KeyedList constructor.  You can’t extract that information out of a function (supplied as a lambda expression or otherwise).  But expressions contain all the metadata we need.  Note that for this synchronization to work, it requires that the items in our collection implement INotifyPropertyChanged.

class Table : INotifyPropertyChanged
{
    public event PropertyChangedEventHandler PropertyChanged;

    private string _TableName;
    public string TableName
    {
        get { return _TableName; }
        set
        {
            _TableName = value;

            if (PropertyChanged != null)
                PropertyChanged(this, new PropertyChangedEventArgs("TableName"));
        }
    }

    public Table(string TableName)
    {
        this.TableName = TableName;
    }
}

This is tedious work, and though there are some patterns and code snippets I use to ease the burden a little, it’s still a lot of work to go through to implement such a primitive ability as data binding.

In order to get at the expression metadata, we’ll have to update our constructor to ask for an expression:

public KeyedList(Expression<Func<T, K>> LinkedValueExpression)
{
    this.LinkedValueExpression = LinkedValueExpression;
}

We’ll also need to define a field to store this, and a property will help to compile it for us.

private Func<T, K> LinkedValue;

private Expression<Func<T, K>> _LinkedValueExpression;
public Expression<Func<T, K>> LinkedValueExpression
{
    get { return _LinkedValueExpression; }
    set
    {
        _LinkedValueExpression = value;
        LinkedValue = (value == null) ? null : _LinkedValueExpression.Compile();
    }
}

Now that the groundwork has been set, we can hook into the PropertyChanged event if it’s implemented, which we do by shadowing the Add method.

public new void Add(K key, T item)
{
    base.Add(key, item);

    if (typeof(INotifyPropertyChanged).IsAssignableFrom(typeof(T)))
        (this[key] as INotifyPropertyChanged).PropertyChanged += new PropertyChangedEventHandler(KeyList_PropertyChanged);
}

One caveat about this approach: our shadowing method Add will unfortunately not be called if accessed through a variable of the base class.  That is, if you assign a KeyedList object to a Dictionary object, and call Add from that Dictionary variable, the Dictionary.Add method will be called and not KeyedList.Add, so synchronization of keys will not work properly in that case.  It’s extremely rare that you’d do such a thing, but I want to point it out regardless.  As inheritor of a base class, I would prefer the derived class be in fuller control of these behaviors, but we work with what we have.  I’ll actually take advantage of this later on in a helper method.

Finally, the tricky part.  We need to examine our lambda’s expression tree and extract the property or field name from it.  We’ll compare that to the property name reported to us as changed.  The comparison is actually done between two MemberInfo variables, which is possible because reflection ensures that only one MemberInfo object will exist for each member.  The MemberExpression object, which iniherits from Expression, possesses a Member property, and the other we get from typeof(T).GetMember.  Here’s what that looks like:

private void KeyList_PropertyChanged(object sender, PropertyChangedEventArgs e)
{
    var lambda = LinkedValueExpression as LambdaExpression;
    if (lambda == null)
        return;

    var expr = lambda.Body as MemberExpression;
    if (expr == null)
        return;

    MemberInfo[] members = typeof(T).GetMember(e.PropertyName, MemberTypes.Property | MemberTypes.Field, BindingFlags.Instance | BindingFlags.Public);
    if (members.Length == 0)
        throw new ApplicationException("Field or property " + e.PropertyName + " not found in type " + typeof(T).FullName);

    MemberInfo mi = members[0];
    if (mi == expr.Member)
    {
        // we don't know what the old key was, so we have to find the object in the dictionary
        // then remove it and re-add it
        foreach (var kvp in KeyValuePairs)
        {
            if ((typeof(T).IsValueType && kvp.Value.Equals(sender)) || (kvp.Value as object) == (sender as object))
            {
                T item = this[kvp.Key];
                Remove(kvp.Key);
                Add(item);
                return;
            }
        }
    }
}

This code makes an important assumption; namely, that a lambda expression will be used, which will contain a single field or property access.  It does not support composite or calculated keys, such as (t.SchemaName + “.” + t.TableName), though it’s possible.  I’m currently working on a method that recursively explores an Expression tree and checks for member access anywhere in the tree, to support scenarios like this.  For now, and for the purpose of this article, we’ll stick to the simple case of single member access.

I found that having access to the list of KeyValuePairs was actually useful in my code, and to keep the PropertyChanged handler concise, I added a new KeyValuePairs property to expose the base Dictionary’s enumerator, which you can find in the complete listing of the KeyedList class toward the end of this article.  I now have two iterators; and the way I’ve flipped it around, the default iterator of the base class has become a secondary, named iterator of KeyedList.

Here is a test program to demonstrate the functionality and flexibility of the KeyedList class.

class Program
{
    static void Main(string[] args)
    {
        // use a lambda expression to select the member of Table to use as the dictionary key
        var Tables = new KeyedList<string, Table>(t => t.TableName);

        // add a Table object to our KeyList like an old-fashioned Dictionary
        Tables.Add("Customers", new Table("Customers"));

        // add a Table object to our KeyList without explicitly specifying a key
        Tables.Add(new Table("Vendors"));

        // prove that a change in an item's key property updates the corresponding dictionary key
        Table Vendors = Tables["Vendors"];
        Vendors.TableName = "VENDORS";
        Console.WriteLine("Is 'Vendors' a valid dictionary key? " + Tables.ContainsKey("Vendors"));
        Console.WriteLine("Is 'VENDORS' a valid dictionary key? " + Tables.ContainsKey("VENDORS"));

        // prove that the IEnumerable<T> iterator works
        // note that we don't have to loop through Tables.Values
        foreach (var table in Tables)
        {
            Console.WriteLine(table.TableName);
        }

        Console.ReadKey();
    }
}

Complete Source for KeyedList

For your convenience, here is the complete listing of KeyedList.

public class KeyedList<K, T> : Dictionary<K, T>, IEnumerable<T>
{
    private Func<T, K> LinkedValue;

    private Expression<Func<T, K>> _LinkedValueExpression;
    public Expression<Func<T, K>> LinkedValueExpression
    {
        get { return _LinkedValueExpression; }
        set
        {
            _LinkedValueExpression = value;
            LinkedValue = (value == null) ? null : _LinkedValueExpression.Compile();
        }
    }

    public KeyedList()
    {
        LinkedValueExpression = null;
    }

    public KeyedList(Expression<Func<T, K>> LinkedValueExpression)
    {
        this.LinkedValueExpression = LinkedValueExpression;
    }

    public void Add(T item)
    {
        if (LinkedValue == null)
            throw new InvalidOperationException("Can't call KeyedList<K, T>.Add(T) " +
                "unless a LinkedValue function has been assigned");

        Add(LinkedValue(item), item);
    }

    public new void Add(K key, T item)
    {
        base.Add(key, item);

        if (typeof(INotifyPropertyChanged).IsAssignableFrom(typeof(T)))
            (this[key] as INotifyPropertyChanged).PropertyChanged += new PropertyChangedEventHandler(KeyList_PropertyChanged);
    }

    private void KeyList_PropertyChanged(object sender, PropertyChangedEventArgs e)
    {
        var lambda = LinkedValueExpression as LambdaExpression;
        if (lambda == null)
            return;

        var expr = lambda.Body as MemberExpression;
        if (expr == null)
            return;

        MemberInfo[] members = typeof(T).GetMember(e.PropertyName,
            MemberTypes.Property | MemberTypes.Field,
            BindingFlags.Instance | BindingFlags.Public);
        if (members.Length == 0)
            throw new ApplicationException("Field or property " + e.PropertyName + " not found in type " + typeof(T).FullName);

        MemberInfo mi = members[0];
        if (mi == expr.Member)
        {
            // we don't know what the old key was, so we have to find the object in the dictionary
            // then remove it and re-add it
            foreach (var kvp in KeyValuePairs)
            {
                if ((typeof(T).IsValueType && kvp.Value.Equals(sender))
                    || (!typeof(T).IsValueType && (kvp.Value as object) == (sender as object)))
                {
                    T item = this[kvp.Key];
                    Remove(kvp.Key);
                    Add(item);
                    return;
                }
            }
        }
    }

    public new IEnumerator<T> GetEnumerator()
    {
        foreach (var item in Values)
        {
            yield return item;
        }
        yield break;
    }

    public IEnumerable<KeyValuePair<K, T>> KeyValuePairs
    {
        get
        {
            // because GetEnumerator is shadowed (to provide the more intuitive IEnumerable<T>),
            // and foreach'ing over "base" isn't allowed,
            // we use a Dictionary variable pointing to "this"
            // so we can use its IEnumerable<KeyValuePair<K, T>>
            Dictionary<K, T> dict = this;
            foreach (var kvp in dict)
            {
                yield return kvp;
            }
            yield break;
        }
    }
}

Conclusion

As with the non-binary Tree data structure I created in this article, I prefer to work with more intelligent object containers that establish tighter integration between the container itself and the elements they contain.  I believe this reduces mental friction, both for author and consumer of components (or the author and consumer roles when they are the same person), and allows a single generic data structure to be used where custom collection classes are normally defined.  Additionally, by using data structures that expose more flexible surface areas, we can often reap the benefits of having powerful lookup features without locking ourselves out of the simple List facades that serve so well in our APIs.

The Achilles Heel of this solution is a weak and un-guaranteed data binding infrastructure combined with lack of support for Expressions in Compact Framework.  In other words, items have to play along, and it’s not platform universal.

Clearly, more work needs to be done.  Event handlers need to be unhooked when items are removed, optimizations could be done to speed up key synchronization, and more complex expressions could be supported without too much trouble.  Ultimately what I’d like to see is a core collection class with the ability to add and access multiple indexes (such as with a database), instead of presuming that a Dictionary with its solitary key is all we can or should use.  The hashed key of a Dictionary seems like a great adornment to an existing collection class, rather than a hard-coded stand-alone structure.  But I think this is a good start toward addressing some of the fundamental shortcomings of these existing approaches, and hopefully demonstrating the value of more intelligent collection and container classes.

Other Implementations

I was at a loss for a while as to what to call this collection class; I considered DictionaryList (and ListDictionary), as well as HashedList, before arriving at KeyedList.  To my amazement, I found several other implementations of the same kind of data structure with the same name, so it must be a good name.  The implementations here and here are more complete than mine, but neither auto-assign keys with a key-selection function or update dictionary keys using data binding, which ultimately is what I’m emphasizing here.  Hint: it wouldn’t be tough to combine what I have here with either of them.

‘In’ Operator for .NET Framework

As a C#/.NET developer, one thing that I miss about Pascal is the in operator.  For example, instead of this:

IF (Letter = ‘A’) OR (Letter = ‘E’) OR (Letter = ‘I’) OR (Letter = ‘O’) OR (Letter = ‘U’) THEN …

… you can write this:

IF (Letter in ['A', 'E', 'I', 'O', 'U']) THEN …

It’s hard to argue that code like this is succinct, elegant, and easy to read and write.  It’s a common enough scenario to check for membership in a set that it warrants support in a language, although from inquiries I’ve made of the Microsoft C# compiler team, the syntax presents some parsing complexity and therefore implementation difficulty.

In C#, we could always do this:

char c = 'E';
if (Array.IndexOf(new char[] { 'A', 'E', 'I', 'O' }, c) != -1) // ...

But this is a lot of work (relative to the task): calling a static member on a type, instantiating an array with the new operator, comparing the result to -1 (which is a detail of our approach rather than the essense of our intention), and obscuring the value we want tested (c, the primary actor in the conditional operation) in the middle of a busy syntax.

The IEnumerable<T>.Contains<T> extension method provides an alternative:

if ((new char[] { 'A', 'E', 'I', 'O' }).Contains(c)) // ...

Using the Contains extension method is better, as it more directly expresses our intention, but we’re still creating the array explicitly and using symbols on the keyboard that slow us down–if we had a whole bunch of these to write all at once, it would create some micro-irritation.  As with the Array.IndexOf example, we’re also reversing the flow of thought from the original Pascal example, specifying a set and then checking for membership, instead of starting with a member and testing if it belongs to a set.  The end result is the same, but I personally like Pascal’s direction and flow.

It’s pretty difficult to match the simplicity of Pascal’s in operator.  But in the spirit of stubbornness and mild obsession, I came up with this extension method to reverse the direction of Contains:

public static bool In<T>(this T obj, params T[] values)
{
    for (int i = 0; i < values.Length; i++)
    {
        if (Object.Equals(obj, values[i]))
            return true;
    }
    return false;
}

With this generic extension method defined on Object, I can now write strongly-typed IN-operator-style conditional expressions like so:

if (c.In('A', 'E', 'I', 'O', 'U')) // ...

if (column.DataType.In("nchar", "nvarchar", "ntext")) // ...

I could have implemented the method more simply, like this:

return values.Contains(obj);

… but that involves another method call, and I’d rather not introduce any additional overhead for a commonly-used, global operator if it’s this easy not to.

One obligatory cautionary note: I don’t recommend adding too many extension methods on Object because of the potential for cluttering Intellisense (which currently doesn’t have an option for filtering out extension methods), but this is one of those general-purpose operators that I’d like to always have available; thus the exception.