Critical Development

Language design, framework development, UI design, robotics and more.

Visual Studio 2008 – Project Reference Oddness

Posted by Dan Vanderboom on December 11, 2007

I can’t count the number of hours I’ve spent over the past few years fighting with references.  In the old days of C, Pascal, and other languages, you told the compiler to look here for a source or header file, and I’ll be damned if that’s not where it looked.  But in working with Visual Studio and the .NET Framework, those days are forever gone.

Project Reference Paths – Hints and Clues

The picture is a great deal more complicated with things like “hint paths” and directory searching algorithms.  The name hint path should be evidence enough of the absurdity of the situation.  We’re not telling you where something is.  Instead, we’re going to give you some hints and clues, and it will be a big fun mystery to figure it out.

To make things worse, these project reference paths are not stored in the Visual Studio project file, which gets checked into source control and shared with other team members.  Instead, they’re stored in a separate “user settings” file, which is more specific to the user, really, since it is also specific to the directory it’s in.  So every time a developer rebuilds their machine, or checks out a project to a different machine, or even branches a project in source control, they have to set up these project reference paths all over again.  I work on some fairly large solutions and use a healthy number of third party libraries, so setting up these paths can take 30–45 minutes or more.  I’ve been so annoyed by this that I’ve considered writing a Visual Studio extension that would allow you to store these paths in the project file itself, or failing that, some external file that would be checked in.  (Checking in the user settings file is undesirable, as there are many settings such as open windows and breakpoints that you would not want to share across a team.)

I’m sure there is a Big List of Good Reasons somewhere that describes why all of these convolutions were deemed necessary (such as allowing developers to check out dependent assemblies into different relative paths), but ultimately I believe this complexity is a weakness: it creates fragility.  Simplicity is wonderful for many reasons, and while sometimes we really do need to make things more complicated, we need to be careful to grow in complexity lest we also grow in chaos.

[What’s the difference?  Chaos is disorganized complexity, or dynamic instability.  Organized complexity is the result of structures in equilibrium.]

If a more complicated scenario is supported, then some product and tool maturity really begs to be included to provide discoverability, to make it obvious what is going on and why.  For a good discussion on discoverability, check out this .NET Rocks podcast where Mark Miller explains:

http://www.dotnetrocks.com/default.aspx?showNum=185

Or better yet, download a trial of CodeRush and open the discoverability window, which is an excellent example of a great user interface that makes a complicated set of functionality simple and obvious to use.  The designers of the Visual Studio IDE could learn a lot from these folks.

Reference Flavors & Properties

Visual Studio supports project references in addition to references to specific binaries.  When we have a static DLL, such as a third party library, we reference that DLL specifically.  Project references are nice because we can reference one of our own projects that hasn’t been built yet, and therefore has no DLL.  When we build project A that depends on project B, it will force project B to build as well.

In each reference, we can also specify whether to Copy Local (copy files locally), and this can be a point of confusion.  Local to what?  Here we’re talking about copying the assemblies that project B produces into project A’s output folder.  Why would we want to do this?  I’m not entirely certain, though I could make a guess that it’s easier to cascade these copy operations from one project to the next, to the next, and so on, until they wind up at their final location.  But considering how the performance of builds is tied so closely to disk access, as thousands of files often need to be processed, it begs the question of whether the locations of these files couldn’t just be tracked in memory, and only output in their final destinations at the very end of the build.  Furthermore, I ran across MSDN documentation a few months ago that recommended not changing the default Copy Local behavior, as it will likely cause problems.  No explanation was offered, as if to suggest “yes we know it’s complicated, so much that we’re not going to tell you why you should just leave it alone”.  Or such was my interpretation.

Specific Version is Too Specific

Another consideration to make is whether to target the Specific Version of the assembly or not.  This is the point of my most recent painful experience with references.  Typically, when I add a new reference to a third party library, Specific Version defaults to true.  When the default works, I leave it alone.

Yesterday I installed a new version of a third party library.  I maintain my own directory structure of third party libraries, copying them from their default install directories to organize them better, and so they stick around if I need to run their uninstaller (when supporting tools go bad).  I also check this directory into source control so they can all be easily shared across the development team.

The first thing I did was to update my reference paths.  Everyone on the team has to, which is a big repeated waste of time.  Then I noticed that my references were splatted.  That is, there were little yellow yield signs indicating that references were missing or otherwise confused.  It didn’t take long to figure out that the reference was pointing to version 7.3.3 with Specific Version set to true, and the new assemblies were version 7.3.4.  It had the name of the assembly, and the path to point right to it, but because I had originally told it to point to that specific version, that’s what it did.  That part makes sense.

I changed the Specific Version setting to false, on all of the references (separately) across all projects (another annoyance), and the splat signs went away.  I rebuilt the solution and it took about 20 minutes, compared to the minute or two it normally takes.  Not acceptable!  What was going on?

Upon looking at the build output window, and turning verbosity to Detailed, I noticed many lines that said “Considering …”.  In the process of looking for these new assemblies, it was considering just about every path it was aware of that had anything to do with development, Visual Studio, libraries, etc., to find these assemblies.  Why?  After all, I had given it valid reference hint paths.  I told it exactly where those assemblies were stored.  The names of the assemblies hadn’t changed.  I verified everything three, four, five times.  Yet it was ignoring all of that because of some set of complicated search rules that I am ignorant of, defined and buried deep in the bowels of MSBuild.  It told me where it was looking, but didn’t tell me why, so all of that output seems useless.

Where is this documented, you ask?  I don’t know.  I’ve asked on the forums and have gotten a bunch of “I don’t knows” for answers.  If you’re an MSBuild guru, please drop me a note and help me out.  It will be greatly appreciated.

I decided to turn Specific Version back on, thinking that it would see the new assembly the reference was pointing to, but doing so gave me the splat icons again.  Did the assemblies disappear off the disk?  No.  But for some reason, those references remember the version number of the old assemblies that I originally added the references to.  I expected that it would update the version, and cause the Specific Version references to point to the new 7.3.4 DLLs.  This didn’t happen, to my frustration.  I had to remove all of those references (keeping track of which ones were needed in which projects), and then re-add them all from scratch.  Specific Version was set to true by default, and when I compiled, MSBuild didn’t spend half the day searching for them.  Half a day later, after another half day of meetings, I was able to test the 10 lines of code I had written earlier.

It’s issues like this that you have to consider when someone suggests, “we should just upgrade our control library, it will be real quick”.  You never know.

Deployment Output – Where are we going, exactly?

I work primarily in Compact Framework for mobile devices, so part of the develop-test cycle involves deploying after building.  Just as there are complications in the .NET Framework’s build system, deployment can also be tricky.  If your solutions contain multiple projects, you need to be careful what platform you’re targeting, you have to be careful what version of the framework you’re targeting, etc.

When building and deploying, we noticed that all of our add-in DLLs weren’t being deployed.  We got no build or deployment errors in the output window.  Everything was successful, it said.  How could this be?

Apparently, you have to be really careful about where you’re deploying to.  In our system, we have an entry point EXE project and a bunch of add-in DLL projects.  The entry point application deploys into one directory, and add-ins get deployed into a subfolder called AddIns.  This has been working since Visual Studio 2005.  Now that we’ve switched to VS2008, have updated third party libraries, and have made some other changes, something went wrong.  At some point, our deployment folders changed on us. 

Counting on the output window for troubleshooting information once again, I noticed that there isn’t a lot of information regarding deployments.  Builds can fill thousands of lines very quickly, but deployment output just tells you what files it’s deploying—that is, where they’re coming from.  Unfortunately, it doesn’t tell you where it’s deploying them to.  What piece of information could be more critical to report during deployment?!  I wanted to know where my files were supposedly going, but no luck.

After several hours of scrutinizing our settings, I noticed that some of the add-in projects had a root deployment path of %CSIDL_PROGRAM_FILES% (\Program Files), where we were used to deploying, and others had a path of %CSIDL_PROGRAMS% (\Windows\Start Menu\Programs).  With the properties window being set as narrow as it is typically, this fact was obscured, but it makes a world of difference.  Looking in that new location, I found the files, and after updating the project properties, the problem was solved.

A Plea for Simplicity or Discoverability

Dependencies of any type are difficult to track.  Assembly dependencies can certainly get complicated, and that complexity can spiral out of control (into chaos) if we don’t have the right tools to understand and manage them.  When we add new ways of configuring, searching, upgrading, and using dependencies, we’re going to need much better tool support.  Above all, there should be some way to override all of the complications and tell our build system, “forget all that other crap and JUST GO HERE to find this”.

As far as we’ve come technologically with Visual Studio 2008, .NET Framework 3.5, C# 3.0, and all of the other goodness that I use and love (and we have come so far!), we still have a long way to go when it comes to developer experience.  We spend more time these days talking about improving user experience and interaction design in the applications we build for our clients, but the Visual Studio shell hasn’t shown any significant strides forward for many years.  I notice slight updates in linear gradients and the rendering of window borders and tabs, but there are no sweeping, dramatic changes.  There are many ways of making great improvements while still maintaining familiarity in usability, and in future posts, I’ll be talking about many of the ideas that I have to demonstrate.

This was continued with a discussion on deployment for mobile applications here, and a solution to some of these issues has been proposed in this article.

About these ads

10 Responses to “Visual Studio 2008 – Project Reference Oddness”

  1. lordabdul said

    In order to change the version of your references, instead of removing them and re-adding them, you could have unloaded the projects (right-click > unload in VS2005/2008), and then edited it (right-click > edit ProjectName.csproj).
    There, you could replace the 7.2.3 by 7.2.4 or whatever.
    You could also have done search-and-replace in all your csprojs.
    You can even start customizing your MSBuild project, and use a centralized $(MyReferenceVersion) property, defined in an included .targets file, so that when you update the version of your reference library, you need only change it once.

  2. Dan Vanderboom said

    The project file doesn’t always contain the paths. For example, I have references like this in my .csproj file:

    Reference Include=”Whatever.Compact, Version=7.3.5.0, Culture=neutral, processorArchitecture=MSIL”

    It’s the .csproj.user files that define the reference paths that determine where to look. And yes, I can search through and edit the user files a bit faster than manipulating them in Visual Studio.

    If your paths ARE in your project file and they’re checked into a version control system, you need to check them out first. If you have a lot of projects in multiple solutions where these libraries are referenced (as I do), this can be a slow process. When you try checking out a project in Solution Explorer, it wants to check out all the files as well. You could instead go to Source Control Explorer to check out only the project files. So you’re using one tool to check stuff out, another tool to search and replace the text files, a third tool to verify it worked, etc.

    I was mainly just ranting about the lack of support for an easy way to do this within Visual Studio. At the end of the day, we have to find a way to make it work, and do whatever it takes.

  3. borbird said

    Indeed we are transitioning from VS2003 to VS2008 and dealing with … reference difficulties. I am not sure why adding any reference defaults to being version specific, and then requiring the second step to make it not version specific. We many SLNs for our product line and we are constantly making dll references (in essence, referencing between SLNs) so the versions of the DLLs are constantly changing. Often times developers are not remembering to set the reference to be not version specific. This causes our build machines to yak during our daily builds. We have a app we wrote that checks out the csprojs and removes the text in the reference line after the dll name: “DLL REFERENCE, Version=1.0.0.0, Culture=neutral, processorArchitecture=MSIL” changed to “DLL REFERENCE”. That cleaned up errors for us…but it is quite a pain.

    Anyone have any better thoughts on this?

  4. Dan Vanderboom said

    This comment has been moved to a new blog post here.

  5. maxi326 said

    Mr Dan,
    How do I assign a reference folder path on a project/solution bases?
    I have really very little experience on reference setup and are about to go crazy
    where each time I have to struggle reference location with my fellow developers.
    Thank you very much.

    • Mark Harmon said

      Learning msbuild script would be a good place to start. References are setup as PropertyGroups in a csproj file. You can use $() variables(aka properties) along with a Condition statement to setup project/solution dependent references.

  6. Tim said

    I have installed a new version of a component… everything seems fine but when you debug or run compiled it can’t find assembly of that component and it specifies a really old version….

    Where in the solution in VS2008 do I look to fix that? The References point correctly to the latest version…. whereas the error is in a particular .cs file but searching that component by name in there does not show version numbers…..

    This is driving me nuts…

  7. Could it be fixed for VS.NET 2010?

    I’ve tried to reproduce…
    I’ve removed all default .net references, having only 1 custom reference analog to this:

    c:\dir\dir\2.0.0.0\name_of_dll.dll

    When I look into msbuild detailed output, I do not see ‘considered’ for this dll.
    What I do see is ‘considered ….’ for other dll’s.

    The ‘considered…’ is for the references of name_of_dll, sometimes called the n-tier dependencies.
    It is described in bigger detail here: http://sstjean.blogspot.com/2006_09_01_archive.html

    So in the end I don’t understand why everybody is saying it is mandadory to have SpecificVersion=True.
    It seems to work without it…

    Cheers

  8. Reference said

    Reference…

    […]Visual Studio 2008 – Project Reference Oddness « Critical Development[…]…

  9. Meridith said

    Appreciating the dedication you put into your blog and in depth information you offer.
    It’s great to come across a blog every once
    in a while that isn’t the same unwanted rehashed material.
    Wonderful read! I’ve bookmarked your site and I’m adding your RSS feeds to my Google account.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

 
Follow

Get every new post delivered to your Inbox.

%d bloggers like this: