Visual Studio 2008 – Project Reference Oddness

I can’t count the number of hours I’ve spent over the past few years fighting with references.  In the old days of C, Pascal, and other languages, you told the compiler to look here for a source or header file, and I’ll be damned if that’s not where it looked.  But in working with Visual Studio and the .NET Framework, those days are forever gone.

Project Reference Paths – Hints and Clues

The picture is a great deal more complicated with things like “hint paths” and directory searching algorithms.  The name hint path should be evidence enough of the absurdity of the situation.  We’re not telling you where something is.  Instead, we’re going to give you some hints and clues, and it will be a big fun mystery to figure it out.

To make things worse, these project reference paths are not stored in the Visual Studio project file, which gets checked into source control and shared with other team members.  Instead, they’re stored in a separate “user settings” file, which is more specific to the user, really, since it is also specific to the directory it’s in.  So every time a developer rebuilds their machine, or checks out a project to a different machine, or even branches a project in source control, they have to set up these project reference paths all over again.  I work on some fairly large solutions and use a healthy number of third party libraries, so setting up these paths can take 30–45 minutes or more.  I’ve been so annoyed by this that I’ve considered writing a Visual Studio extension that would allow you to store these paths in the project file itself, or failing that, some external file that would be checked in.  (Checking in the user settings file is undesirable, as there are many settings such as open windows and breakpoints that you would not want to share across a team.)

I’m sure there is a Big List of Good Reasons somewhere that describes why all of these convolutions were deemed necessary (such as allowing developers to check out dependent assemblies into different relative paths), but ultimately I believe this complexity is a weakness: it creates fragility.  Simplicity is wonderful for many reasons, and while sometimes we really do need to make things more complicated, we need to be careful to grow in complexity lest we also grow in chaos.

[What’s the difference?  Chaos is disorganized complexity, or dynamic instability.  Organized complexity is the result of structures in equilibrium.]

If a more complicated scenario is supported, then some product and tool maturity really begs to be included to provide discoverability, to make it obvious what is going on and why.  For a good discussion on discoverability, check out this .NET Rocks podcast where Mark Miller explains:

http://www.dotnetrocks.com/default.aspx?showNum=185

Or better yet, download a trial of CodeRush and open the discoverability window, which is an excellent example of a great user interface that makes a complicated set of functionality simple and obvious to use.  The designers of the Visual Studio IDE could learn a lot from these folks.

Reference Flavors & Properties

Visual Studio supports project references in addition to references to specific binaries.  When we have a static DLL, such as a third party library, we reference that DLL specifically.  Project references are nice because we can reference one of our own projects that hasn’t been built yet, and therefore has no DLL.  When we build project A that depends on project B, it will force project B to build as well.

In each reference, we can also specify whether to Copy Local (copy files locally), and this can be a point of confusion.  Local to what?  Here we’re talking about copying the assemblies that project B produces into project A’s output folder.  Why would we want to do this?  I’m not entirely certain, though I could make a guess that it’s easier to cascade these copy operations from one project to the next, to the next, and so on, until they wind up at their final location.  But considering how the performance of builds is tied so closely to disk access, as thousands of files often need to be processed, it begs the question of whether the locations of these files couldn’t just be tracked in memory, and only output in their final destinations at the very end of the build.  Furthermore, I ran across MSDN documentation a few months ago that recommended not changing the default Copy Local behavior, as it will likely cause problems.  No explanation was offered, as if to suggest “yes we know it’s complicated, so much that we’re not going to tell you why you should just leave it alone”.  Or such was my interpretation.

Specific Version is Too Specific

Another consideration to make is whether to target the Specific Version of the assembly or not.  This is the point of my most recent painful experience with references.  Typically, when I add a new reference to a third party library, Specific Version defaults to true.  When the default works, I leave it alone.

Yesterday I installed a new version of a third party library.  I maintain my own directory structure of third party libraries, copying them from their default install directories to organize them better, and so they stick around if I need to run their uninstaller (when supporting tools go bad).  I also check this directory into source control so they can all be easily shared across the development team.

The first thing I did was to update my reference paths.  Everyone on the team has to, which is a big repeated waste of time.  Then I noticed that my references were splatted.  That is, there were little yellow yield signs indicating that references were missing or otherwise confused.  It didn’t take long to figure out that the reference was pointing to version 7.3.3 with Specific Version set to true, and the new assemblies were version 7.3.4.  It had the name of the assembly, and the path to point right to it, but because I had originally told it to point to that specific version, that’s what it did.  That part makes sense.

I changed the Specific Version setting to false, on all of the references (separately) across all projects (another annoyance), and the splat signs went away.  I rebuilt the solution and it took about 20 minutes, compared to the minute or two it normally takes.  Not acceptable!  What was going on?

Upon looking at the build output window, and turning verbosity to Detailed, I noticed many lines that said “Considering …”.  In the process of looking for these new assemblies, it was considering just about every path it was aware of that had anything to do with development, Visual Studio, libraries, etc., to find these assemblies.  Why?  After all, I had given it valid reference hint paths.  I told it exactly where those assemblies were stored.  The names of the assemblies hadn’t changed.  I verified everything three, four, five times.  Yet it was ignoring all of that because of some set of complicated search rules that I am ignorant of, defined and buried deep in the bowels of MSBuild.  It told me where it was looking, but didn’t tell me why, so all of that output seems useless.

Where is this documented, you ask?  I don’t know.  I’ve asked on the forums and have gotten a bunch of “I don’t knows” for answers.  If you’re an MSBuild guru, please drop me a note and help me out.  It will be greatly appreciated.

I decided to turn Specific Version back on, thinking that it would see the new assembly the reference was pointing to, but doing so gave me the splat icons again.  Did the assemblies disappear off the disk?  No.  But for some reason, those references remember the version number of the old assemblies that I originally added the references to.  I expected that it would update the version, and cause the Specific Version references to point to the new 7.3.4 DLLs.  This didn’t happen, to my frustration.  I had to remove all of those references (keeping track of which ones were needed in which projects), and then re-add them all from scratch.  Specific Version was set to true by default, and when I compiled, MSBuild didn’t spend half the day searching for them.  Half a day later, after another half day of meetings, I was able to test the 10 lines of code I had written earlier.

It’s issues like this that you have to consider when someone suggests, “we should just upgrade our control library, it will be real quick”.  You never know.

Deployment Output – Where are we going, exactly?

I work primarily in Compact Framework for mobile devices, so part of the develop-test cycle involves deploying after building.  Just as there are complications in the .NET Framework’s build system, deployment can also be tricky.  If your solutions contain multiple projects, you need to be careful what platform you’re targeting, you have to be careful what version of the framework you’re targeting, etc.

When building and deploying, we noticed that all of our add-in DLLs weren’t being deployed.  We got no build or deployment errors in the output window.  Everything was successful, it said.  How could this be?

Apparently, you have to be really careful about where you’re deploying to.  In our system, we have an entry point EXE project and a bunch of add-in DLL projects.  The entry point application deploys into one directory, and add-ins get deployed into a subfolder called AddIns.  This has been working since Visual Studio 2005.  Now that we’ve switched to VS2008, have updated third party libraries, and have made some other changes, something went wrong.  At some point, our deployment folders changed on us. 

Counting on the output window for troubleshooting information once again, I noticed that there isn’t a lot of information regarding deployments.  Builds can fill thousands of lines very quickly, but deployment output just tells you what files it’s deploying—that is, where they’re coming from.  Unfortunately, it doesn’t tell you where it’s deploying them to.  What piece of information could be more critical to report during deployment?!  I wanted to know where my files were supposedly going, but no luck.

After several hours of scrutinizing our settings, I noticed that some of the add-in projects had a root deployment path of %CSIDL_PROGRAM_FILES% (\Program Files), where we were used to deploying, and others had a path of %CSIDL_PROGRAMS% (\Windows\Start Menu\Programs).  With the properties window being set as narrow as it is typically, this fact was obscured, but it makes a world of difference.  Looking in that new location, I found the files, and after updating the project properties, the problem was solved.

A Plea for Simplicity or Discoverability

Dependencies of any type are difficult to track.  Assembly dependencies can certainly get complicated, and that complexity can spiral out of control (into chaos) if we don’t have the right tools to understand and manage them.  When we add new ways of configuring, searching, upgrading, and using dependencies, we’re going to need much better tool support.  Above all, there should be some way to override all of the complications and tell our build system, “forget all that other crap and JUST GO HERE to find this”.

As far as we’ve come technologically with Visual Studio 2008, .NET Framework 3.5, C# 3.0, and all of the other goodness that I use and love (and we have come so far!), we still have a long way to go when it comes to developer experience.  We spend more time these days talking about improving user experience and interaction design in the applications we build for our clients, but the Visual Studio shell hasn’t shown any significant strides forward for many years.  I notice slight updates in linear gradients and the rendering of window borders and tabs, but there are no sweeping, dramatic changes.  There are many ways of making great improvements while still maintaining familiarity in usability, and in future posts, I’ll be talking about many of the ideas that I have to demonstrate.

This was continued with a discussion on deployment for mobile applications here, and a solution to some of these issues has been proposed in this article.

Tags: MSBuild, Compact+Framework, Visual+Studio, User+Interaction+Design, Interaction+Design, Discoverability, Reference+Paths, Project+Reference