The Archetype Language (Part 9)

Overview

This is part of a continuing series of articles about a new .NET language under development called Archetype.  Archetype is a C-style (curly brace) functional, object-oriented (class-based), metaprogramming-capable language with features and syntax borrowed from many languages, as well as some new constructs.  A major design goal is to succinctly and elegantly implement common patterns that normally require a lot of boilerplate code which can be difficult, error-prone, or just plain onerous to write.

You can follow the news and progress on the Archetype compiler on twitter @archetypelang.

Links to the individual articles:

Part 1 – Properties and fields, function syntax, the me keyword

Part 2 – Start function, named and anonymous delegates, delegate duck typing, bindable properties, composite bindings, binding expressions, namespace imports, string concatenation

Part 3 – Exception handling, local variable definition, namespace imports, aliases, iteration (loop, fork-join, while, unless), calling functions and delegates asynchronously, messages

Part 4 – Conditional selection (if), pattern matching, regular expression literals, agents, classes and traits

Part 5 – Type extensions, custom control structures

Part 6 – If expressions, enumerations, nullable types, tuples, streams, list comprehensions, subrange types, type constraint expressions

Part 7 – Semantic density, operator overloading, custom operators

Part 8 – Constructors, declarative Archetype: the initializer body

Part 9 – Params & fluent syntax, safe navigation operator, null coalescing operators

Conceptual articles about language design and development tools:

Language Design: Complexity, Extensibility, and Intention

Reimagining the IDE

Better Tool Support for .NET

Params & Fluent Syntax

C# has a parameter modifier called params that allows you to supply additional function arguments to populate a single array parameter.

void Display(params string[] Names)
{
    // …
}

Without the params modifier, we’d have to call it like this:

Display(new string[] { "Dan", "Josa", "Sarah" });

Because params is declared, we can do this instead:

Display("Dan", "Josa", "Sarah");

If there’s one thing you can take away from Archetype’s design, it’s that syntactic sugar is everything.  After examining my own procedural animation library (Animate.NET) to see how it could be used best in Archetype, I came to the conclusion that these params parameters can be substantial.  When they are, they create syntactic unpleasantries, especially when nested structures are involved.

Consider the following C# example.

var anim =

Animate.Wait(0.2.seconds(),

RedChip.MoveBy(0.4.seconds(), -40, 0),

RedChip.FadeIn(0.2.seconds()),

 

BlackChip.MoveBy(0.4.seconds(), 0, 40),

BlackChip.FadeOut(0.4.seconds())

)

.WhenComplete(a =>

{

MainStage.Children.Remove(RedChip);

MainStage.Children.Remove(BlackChip);

})

.Begin();

First, a quick explanation of the code.  Animate is a static class, and the Wait function returns an object called GroupAnimation that inherits from Animation.  After a 0.2 second wait, the following params list of Animation objects will execute.  RedChip and BlackChip are FrameworkElements (Silverlight/WPF objects), and animation commands such as MoveBy and FadeOut are extension methods on FrameworkElement.  Each of these animation commands returns an Animation-derived object.  The seconds() extension method on int and float types convert to TimeSpan objects.

The ultimate goal of this first Wait section of code is to define a set of animations—nested sets are possible, which form a tree of animations.  These trees can get more complicated than this, but we’ll keep the example simple for now.

Now for the criticism.  Look at the matching parentheses of the Wait function.  The normal TimeSpan parameter is listed as an equal along with the Animation parameter list, and what is being used as a complex, nested structure is holding up the closing parenthesis and dragging it down to the end of the entire list.  If only there were a cleaner way of treating this nested structure like constructor initializers (see Part 8).  These correspond, in terms of visual layout, to the attributes and the child elements of an XML node.

What else is wrong with this picture?  The .WhenComplete and .Begin functions are being invoked on the result of the previous expression.  It’s characteristic of fluent-style APIs to define functions (or extension methods) that operate on the result of the previous operation so they can be strung together into sentence-like patterns.  The dot before both WhenComplete and Begin look odd when appearing on lines by themselves, and the lambda expression would be better promoted to a proper code block.

Finally, it’s unfortunate that in declaring a new local variable, we have to indent the whole animation block this way.  Here’s what the same code looks like in Archetype:

Animate.Wait (0.2 seconds) -> anim

{

RedChip.MoveBy(0.4 seconds, -40, 0),

RedChip.FadeIn(0.2 seconds),

 

BlackChip.MoveBy(0.4 seconds, 0, 40),

BlackChip.FadeOut(0.4 seconds)

}

WhenComplete (a)

{

MainStage.Children.Remove(RedChip),

MainStage.Children.Remove(BlackChip)

}

Begin();

This is more like it.  Notice the declarative assignment (declaration + assignment) with –> anim on the first line, and the way the parentheses can be closed after the TimeSpan object (see Part 7 on custom operators for an explanation of the syntax “0.2 seconds”).  There’s no more need to indent the whole structure to make it line up nicely in an assignment.  The following initializer code block (in curly braces) supplies Animation object values to the params parameter in the Wait function, and the WhenComplete and Begin functions don’t require a leading dot to operate on the previous expression (Intellisense would reflect these options).

The Archetype code is much cleaner.  It’s easier to see where groups of constructs begin and end, enabling fluent-style APIs with arbitrarily-complicated nested structures to be easily constructed.  Let’s take a look at one more example with a more deeply nested structure:

 

Animate.Group –> anim

{

RedChip.MoveBy(0.4 seconds, -40, 0),

RedChip.FadeIn(0.2 seconds),

 

BlackChip.MoveBy(0.4 seconds, 0, 40),

BlackChip.FadeOut(0.4 seconds),

 

Animate.Wait (0.4 seconds)

{

Animate.CrossFade(1.5 seconds, RedChip, BlackChip),

BlackChip.MoveTo(0.2 seconds, 20, 150)

}

}

Here, a GroupAnimation is defined that contains, as one of its child Animations, another GroupAnimation (created with the Wait function).  The animation isn’t started in this case, so anim.Begin() can be called later, or anim could be composed into a larger animation somewhere.  A peek at the function headers for Group and Wait functions should make the ease and power of this design clear.

static Animate object

{

// a stream is the only parameter

Group GroupAnimation (Animations Animation* params)

{

}

 

// a stream is the last parameter, so [ list ] syntax can still be used

Wait GroupAnimation (WaitTime TimeSpan, Animations Animation* params)

{

}

}

Because the class is static, individual members are assumed to be static as well.

 

The easiest way to support this would be to allow this initializer block to be used with a params parameter that’s declared last.

         

Null Coalescing Operators

The null coalescing operator in C# allows you to compare a value to null, and to supply a default value to use in its place.  This is handy in scenarios like this:

var location = (cust.Address.City ?? "Unknown") + ", " + (cust.Address.State ?? "Unknown");

 

Chris Eargle makes a good point in his article suggesting a “null coalescing assignment operator” when making assignments such as:

cust.Address.City = cust.Address.City ?? "Unknown";

 

There should be a way to eliminate this redundancy.  By combining null coalescing with assignment, we can do this:

cust.Address.City ??= "Unknown";

Groovy’s Elvis Operator serves a similar role, but operates on a value of false in addition to null.

Safe Navigation Operator

There are many situations where we find ourselves needing to check the value of a deeply nested member, but if we access it directly without first checking whether each part of the path is null, we get a NullReferenceException.

var city = cust.Address.City;

 

If either cust or Address are null, an exception will be thrown.  To get around this problem, we have to do something like this in C#:

string city = null;

 

if (cust != null && cust.Address != null)

city = cust.Address.City;

 

The && operator is short-circuiting, which means that if the first boolean expression evaluates to false, the rest of the expression—which would produce a NullReferenceException—never gets executed.  As tedious as this is, without short circuiting operators, our error-prevention code would be even longer.

Jeff Handley wrote a clever safe navigation operator of sorts for C#, using an extension method called _ that takes a delegate (supplied as a lambda).  You can find that code here.  In his code, he does return a null value when the path short circuits.  As you can see, however, the limitations of C# cause this simple example to get confusing quickly, which you can see if we make City a non-primitive object as well:

var city = cust._(c => c.Address._(a => a.City.Name));

Groovy implements a Safe Navigation Operator in the language itself, which is cleaner:
 

var city = cust?.Address.City;

 

This is equivalent to the more verbose code above.  Archetype takes a similar approach:

var city = cust..Address.City;

Because of the .. operator in this member access expression, the type of the city variable is Option<string> (more on Option types).  If the path leading up to City is invalid (because Address is null), the value of city will be None.  This works the same as Nullable<T>, except that None means “doesn’t have a value; not even null”.

I like to think of None as the “mu constant”.  What is mu?  It’s the Japanese word that variously means “not”, “doesn’t exist”, etc., and is illustrated by the well-known Zen Buddhist koan:

A monk asked Zhaozhou Congshen, a Chinese Zen master (known as Jōshū in Japanese), "Has a dog Buddha-nature or not?" Zhaozhou answered, "Wú" (in Japanese, Mu)

—The Gateless Gate, koan 1, translation by Robert Aitken

Yasutani Haku’un of the Sanbo Kyodan maintained that "the koan is not about whether a dog does or does not have a Buddha-nature because everything is Buddha-nature, and either a positive or negative answer is absurd because there is no particular thing called Buddha-nature.

In other words, Mu has often been used to mean “I disagree with the presuppositions of the question.”

There are a few basic patterns around options, nullable objects, and safe navigation that occur frequently, so I’ll outline them here with examples:

// if Address is null, this evaluates to false

if (cust..Address.City == "Milwaukee")

WorkHarder();

 

// if City is None because Address is null, set to "Address Missing"; otherwise, get the city text

var city = cust..Address.City ?! "Address Missing";

 

// if City is Some<string> and City == null, set to empty string

var city = cust..Address.City ?? string.Empty;

 

// if Address is null (City is None), set to "Address Not Found";

// but if City == null, set to empty string

var city = cust..Address.City ?! "Address Not Found" ?? string.Empty;

 

// if Address points to an object, leave it alone; otherwise, create a new object

cust.Address ??= new Address(City="Milwaukee");

 

// an assertion

cust..Address ?! new Exception("Address missing");

 

// set the city if possible, throw a specific exception if not

var city = cust..Address.City ?! new Exception("Address missing");

 

Summary

By now it should be obvious that Archetype aims to liberate the developer from the constraints and inefficiencies of ordinary programming languages.  It is designed with modern practices in mind such as fluent-style development and declarative object graph construction.

This article wraps up the material started in Part 8 on declarative programming in Archetype.  In addition, I introduced the safe navigation and null coelescing operators.  These are simple but powerful language elements for cleanly and succinctly specifying common idioms that come up in daily coding.

The Archetype Language (Part 7)

Overview

This is part of a continuing series of articles about a new .NET language under development called Archetype.  Archetype is a C-style (curly brace) functional, object-oriented (class-based), metaprogramming-capable language with features and syntax borrowed from many languages, as well as some new constructs.  A major design goal is to succinctly and elegantly implement common patterns that normally require a lot of boilerplate code which can be difficult, error-prone, or just plain onerous to write.

You can follow the news and progress on the Archetype compiler on twitter @archetypelang.

Links to the individual articles:

Part 1 – Properties and fields, function syntax, the me keyword

Part 2 – Start function, named and anonymous delegates, delegate duck typing, bindable properties, composite bindings, binding expressions, namespace imports, string concatenation

Part 3 – Exception handling, local variable definition, namespace imports, aliases, iteration (loop, fork-join, while, unless), calling functions and delegates asynchronously, messages

Part 4 – Conditional selection (if), pattern matching, regular expression literals, agents, classes and traits

Part 5 – Type extensions, custom control structures

Part 6 – If expressions, enumerations, nullable types, tuples, streams, list comprehensions, subrange types, type constraint expressions

Part 7 – Semantic density, operator overloading, custom operators

Part 8 – Constructors, declarative Archetype: the initializer body

Part 9 – Params & fluent syntax, safe navigation operator, null coalescing operators

Conceptual articles about language design and development tools:

Language Design: Complexity, Extensibility, and Intention

Reimagining the IDE

Better Tool Support for .NET

Semantic Density

As an avid reader growing up, I noticed that my knowledge and understanding of a topic grew more easily the faster I read.  Instead of going through a chapter every day or two, which puts weeks or months between the front and back covers, I devoured 200-300 pages in a night, getting through the largest books in a couple days.  And in reading multiple books on a subject back-to-back, it was easier to find relationships and tie together concepts for things that were still so fresh in my memory.

In my study of linguistics, I learned that legends like Noam Chomsky could learn hundreds of langauges; the previous librarian at the Vatican could read 97.  Bodmer’s excellent book The Loom of Language attempts to teach 10 languages at once, and it seems that the more languages you learn, the easier it is to pick up others.

What these examples have in common is semantic density.  It might seem from what I’ve said that this would be like drinking from a firehose which only the most gifted could endure, but I would argue that intensely-focused learning puts our minds in a highly alert and receptive condition.  In such a state, being able to draw more connections between statements and ideas, we are better able to comprehend the whole in a holistic, intuitive way.

Code Example

Semantic density is important in code, too.  With a pattern like INotifyPropertyChanged, formatted as I have it below, it’s 12 lines of code, 13 if you separate your fields and properties with a blank line, but in the ballpark of a dozen lines of code.  (This is additional explanation for a feature described in part 2 of this series.)

1:   string _Display;
2:   public string Display
3:   {
4:       get { return _Display; }
5:       set
6:       {
7:           _Display = value;
8: 
9:           if (PropertyChanged != null)
10:              PropertyChanged(this, new PropertyChangedEventArgs("Display"));
11:      }
12:  }

Does the ability to inform external code of changes seem like it should take a dozen lines to get there?  This can be somewhat compressed by defining a SetProperty method:

1:   string _Display;
2:   public string Display
3:   {
4:       get { return _Display; }
5:       set { SetProperty("Display", ref _Display, value); }
6:   }

This chops the line count in half, bringing it down to six lines–seven if you include a space above or below to separate it from other members.  On my monitor, that means I can see about eight property definitions at a time.  Now that’s usually enough, but I’ve written a few custom controls  that have upwards of 30 properties.  For a new pair of eyes, getting the gist of that class is going to involve a lot of scrolling, never seeing more than a small slice at a time of a much large picture.  The frame of time I mentioned earlier in regard to studying a subject is analogous here to the frame of space.  Seeing 20% of a class at any time lends itself to faster grokking than seeing a 2% sliver at a time.  Our minds, marvelous as they are, do have limits.  Lowering semantic density, such as by spreading meaning over large distances or time spans, makes us work harder to accomplish the same task, trying to put all the pieces together, and the differences are often dramatic.

Just as nouns can be modified by adjectives in natural languages, types in Archetype support user-defined type modifiers.  By defining a new type modifier called bindable to encapsulate the INotifyPropertyChanged pattern, we can collapse the above example into a single line:

1:   Age bindable int;

 

I have no problem stacking these properties right on top of one another.  Although expressed in a highly dense form, it’s actually easier to understand at a glance in these three simple tokens than in the half-dozen lines above, which beg for interpretation to assemble their meaning.  Even if we have 30 of these, they’d all fit on one screen, and the purpose of the class as a whole is quickly gathered.

 

One thing I noticed in developing the animation library Animate.NET is how much code I saved: not having to worry about the details of storyboard creation, key frames, and so on.  It allows you to get right to the point of stating your intention.  Often a library like this is enough, but once in a while language extensibility is a much better approach; and when it is, not having the option can be painful and time consuming.

 

Custom Operators & Operator Overloading

As in most languages, Archetype supports two forms of syntax for operations: functions and operators.  Functions are invoked by including a pair of parentheses after their name that contain any arguments to pass in, whereas operators appear adjacent to or between sub-expressions. 

In C#, some operators are available for overloading.  Archetype supports these operator overloads by using the same names for them.  This allows Archetype to use operators defined in C# and to expose supported operators to C# consumers.

However, Archetype goes one step further and allows you to define custom operators.  There are three basic kinds of custom operator:

1. unary prefix
2. unary suffix
3. binary

If we wanted an easy way to duplicate strings in C#, we might define an extension method called Dup, but in Archetype we also have this option:

// "ABC" dup 3 == "ABCABCABC"

binary dup string (left string, right int)

return string.Repeat(left, right)

The expression parser sees “ABC”, identifies it as a string value, and then looks at the next token.  If the dot operator were found (.), it would look for a member of string or an extension member on string, but because the next token isn’t the dot operator, it looks up the token in the operator table.  An operator called dup is defined with a left string argument and a right int argument, matching the expression.  If the operator were more complicated, it would have a curly-brace code block, but because it’s a single return statement, that’s optional.

Archetype operators aren’t limited to letters, though.  We can also use symbols (but not numbers) in our operator names.  Here’s a “long forward arrow” (compiled with name DashDashGreaterThan) that allows us to write a single function parameter before the function name itself:

// "Hey" –> Console.WriteLine;

binary<T> –> void (left T, right Action<T>)

right(left);

Note that the generic type parameter is attached to the binary keyword.  I arrived at this placement through much experimentation.  Names like –><T> are difficult to read and can be trickier to parse.

There is a special binary operator called adjacent which you can think of as an “invisible operator” capable of inserting an operation between two sub-expressions.  In the following example, two adjacent strings are interpreted as a concatenation of the two.

// "123" "45" == "12345"

binary adjacent string (left string, right string)

return left + right;

With custom operators, what was originally part of the language can now be defined in a library instead.  This greatly simplifies the language.  Just as methods can be shadowed to override them, so too will some ability be needed in Archetype to block or override operators that would otherwise be imported along with a namespace.

The next operator we’ll look at is the unary suffix.  The example consists of units of time: minutes and seconds.

// 12 minutes == TimeSpan.FromMinutes(12)

unary suffix minutes TimeSpan (short, int)

return TimeSpan.FromMinutes((int)value);

 

// 3 seconds == TimeSpan.FromSeconds(3)

unary suffix seconds TimeSpan (short, int)

return TimeSpan.FromSeconds((int)value);

With support for extension properties, we could have also said 12.minutes or 3.seconds, which is already better than C#’s 12.minutes() and 3.seconds(), but by defining these tokens as unary suffix operators, we can eliminate even the dot operator and make it that much more fluent and natural to type (without losing any syntactic precision).  Notice how a list of types is provided instead of a parameter list.  Unary operators by definition have only a single argument, but we often want them to operate on several different types.

Here’s a floating point operator for seconds.

// 2.5 seconds == (2 seconds).Add(0.5 * 1000 milliseconds)

// WholeNumber and Fraction are extension properties on float, double, and decimal

unary suffix seconds TimeSpan (float, double, decimal)

return value.WholeNumber seconds + value.Fraction * 1000 milliseconds;

We can use the adjacent operator on TimeSpans the same that we did for strings above.

// 4 minutes 10 seconds == (4 minutes).Add(10 seconds)

binary adjacent TimeSpan (left TimeSpan, right TimeSpan)

return left.Add(right);

Now let’s combine the use of a few of these operators into a single example.

alias min = minutes;

alias s = seconds;

var later = DateTime.Now + 2 min 15 s;

// Schedule is an extension method on DateTime

later.Schedule

{

// schedule this to run later

}

We can also define our DateTime without assigning its value to a variable.

(DateTime.Now + 10 seconds).Schedule

{

}

 

(DateTime.Now + 10 seconds).Schedule (Repeat=10 seconds)

{

}

Repeat is an optional parameter of Schedule, defaulting to TimeSpan.Zero, meaning “don’t repeat”.

Additional Notes

When more than one operator is valid in a given position, the most specific operator (in terms of its parameter types) is used.  If there’s any ambiguity or overlap remaining, a compiler error is issued.

Unary operators will take precedence over binary operators, but it hasn’t been determined yet what precedence either one will actually have in relation to all of the other operators, or whether this will be specified in the operator definition.

Because of this design for custom operators, I’ve been able to remove things from the language itself and include them as operator definitions in a library.

Summary

This article provided some deeper explanation into previously covered material and introduced the syntax for Archetype’s very powerful custom operator declaration syntax.  The next article will cover some special operators built into Archetype, and property path syntax which is something I came up with a while back to safely reference identifiers that would be impervious to both refactoring and obfuscation.

I’m curious to read your feedback on custom operators in particular, so keep the great comments coming!

Language Design: Complexity, Extensibility, and Intention

Introduction

The object-oriented approach to software is great, and that greatness draws from the power of extensibility.  That we can create our own types, our own abstractions, has opened up worlds of possibilities.  System design is largely focused on this element of development: observing and repeating object-oriented patterns, analyzing their qualities, and adding to our mental toolbox the ones that serve us best.  We also focus on collecting libraries and controls because they encapsulate the patterns we need.

This article explores computer languages as a human-machine interface, the purpose and efficacy of languages, complexity of syntactic structure, and the connection between human and computer languages.  The Archetype project is an on-going effort to incorporate these ideas into language design.  In the same way that some furniture is designed ergonomically, Archetype is an attempt to design a powerful programming language with an ergonomic focus; in other words, with the human element always in mind.

Programming Language as Human-Machine Interface

A programming language is the interface between the human mind and executable code.  The point isn’t to turn human programmers into pure mathematical or machine thinkers, but to leverage the talent that people are born with to manipulate abstract symbols in language.  There is an elite class of computer language experts who have trained themselves to think in terms of purely functional approaches, low-level assembly instructions, or regular, monotonous expression structures—and this is necessary for researchers pushing themselves to understand ever more—but for the every day developer, a more practical approach is required.

Archetype is a series of experiments to build the perfect bridge between the human mind and synthetic computation.  As such, it is based as much as possible on a small core of extensible syntax and maintains a uniformity of expression within each facet of syntax that the human mind can easily keep separate.  At the same time, it honors syntactic variety and is being designed to shift us closer to a balance where all of the elements, blocks, clauses and operation types in a language can be extended or modified equally.  These represent the two most important design tenets of Archetype: the intuitive, natural connection to the human mind, and the maximization of its expressive power.

These forces often seem at odds with each other—at first glance seemingly impossible to resolve—and yet experience has shown that the languages we use are limited in ways we’re often surprised by, indicating that processes such as analogical extension are at work in our minds but not fully leveraged by those languages.

Syntactic Complexity & Extensibility

Most of a programming language’s syntax is highly static, and just a few areas (such as types, members, and sometimes operators) can be extended.  Lisp is the most famous example of a highly extensible language with support for macros which allow the developer to manipulate code as if it were data, and to extend the language to encode data in the form of state machines.  The highly regular, parenthesized syntax is very simple to parse and therefore to extend… so long as you don’t deviate from the parenthesized form.  Therefore Lisp gets away with powerful extensibility at the cost of artificially limiting its structural syntax.

In Lisp we write (+ 4 5) to add two numbers, or (foo 1 2) to call a function with two parameters.  Very uniform.  In C we write 4 + 5 because the infix operator is what we grew up seeing in school, and we vary the syntax for calling the function foo(1, 2) to provide visual cues to the viewer’s brain that the function is qualitatively something different from a basic math operation, and that its name is somehow different from its parameters.

Think about syntax features as visual manifestations of the abstract logical concepts that provide the foundation for all algorithmic expression.  A rich set of fundamental operations can be obscured by a monotony of syntax or confused by a poorly chosen syntactic style.  Archetype involves a lot of research in finding the best features across many existing languages, and exploring the limits, benefits, problems, and other details of each feature and syntactic representation of it.

Syntactic complexity provides greater flexibility, and wider channels with which to convey intent.  This is why people color code file folders and add graphic icons to public signage.  More cues enable faster recognition.  It’s possible to push complexity too far, of course, but we often underestimate what our minds are capable of when augmented by a system of external cues which is carefully designed and supported by good tools.

Imagine if your natural spoken language followed such simple and regular rules as Lisp: although everyone would learn to read and write easily, conversation would be monotonous.  Extend this to semantics, for example with a constructed spoken language like Lojban which is logically pure and provably unambiguous, and it becomes obvious that our human minds aren’t well suited to communicating this way.

Now consider a language like C with its 15 levels of operator precedence which were designed to match programmers’ expectations (although the authors admitted to getting some of this “wrong”, which further proves the point).  This language has given rise to very popular derivatives (C++, C#, Java) and are all easily learned, despite their syntactic complexity.

Natural languages and old world cities have grown with civilization organically, creating winding roads and wonderful linguistic variation.  These complicated structures have been etched into our collective unconscious, stirring within us and giving rise to awareness, thought, and creativity.  Although computers are excellent at processing regular, predictable patterns, it’s the complex interplay of external forces and inner voices that we’re most comfortable with.

Risk, Challenge & Opportunity

There are always trade-offs.  By focusing almost all extensibility in one or two small parts of a language, semantic analysis and code improvement optimizations are easier to develop and faster to execute.  Making other syntactical constructs extensible, if one isn’t careful, can create complexity that quickly spirals out of control, resulting in unverifiable, unpredictable and unsafe logic.

The way this is being managed in Archetype so far isn’t to allow any piece of the syntax tree to be modified, but rather to design regions of syntax with extensibility points built-in.  Outputting C# code as an intermediary (for now) lays a lot of burden on the C# compiler to ensure safety.  It’s also possible to mitigate more computationally expensive semantic analysis and code generation by taking advantage of both multicore and cloud-based processing.  What helps keep things in check is that potential extensibility points are being considered in the context of specific code scenarios and desired outcomes, based on over 25 years of real-world experience, not a disconnected sense of language purity or design ideals.

Creating a language that caters to the irregular texture of thought, while supporting a system of extensions that are both useful and safe, is not a trivial undertaking, but at the same time holds the greatest potential.  The more that computers can accommodate people instead of forcing people to make the effort to cater to machines, the better.  At least to the extent that it enables us to specify our designs unambiguously, which is somewhat unnatural for the human mind and will always require some training.

Summary

So much of the code we write is driven by a set of rituals that, while they achieve their purpose, often beg to be abstracted further away.  Even when good object models exist, they often require intricate or tedious participation to apply (see INotifyPropertyChanged).  Having the ability to incorporate the most common and solid of those patterns into language syntax (or extensions which appear to modify the language) is the ultimate mechanism for abstraction, and goes furthest in minimizing development effort.  By obviating the need to write convoluted yet routine boilerplate code, Archetype aims to filter out the noise and bring one’s intent more clearly into focus.

Animate.NET: Fluent Animation Library for Silverlight & WPF

Overview

The basic idea—in Silverlight and WPF—that an animation is just a change in some DependencyProperty over time is simple and powerful.  However, at that level of detail, the API for defining and managing complex animations involves writing a ton of code.  There are code-less animations, of course, such as those created in the Visual State Manager, but when you want to perform really dynamic animations, state-based animations can become impractical or outright impossible.

In response to this, I’ve published my fluent-style code-based animation library for Silverlight and WPF on CodePlex at http://animatedotnet.codeplex.com.  This is an API for making code-based animations intuitive and simple without having to write dozens or even hundreds of lines of code to create and configure storyboards, keyframes, perform repetitive math to calculate alignment, rotation, and other low-level details that distract one from the original purpose of the animation.  In one example, I counted over 120 lines of standard storyboard code, and with the abstractions and fluent API I’ve come up with, reduced that down to half a dozen lines of beautiful, pure intent.  As a result, it’s much more readable and faster to write.

I was initially inspired by Nigel Sampson from his blog article on building a Silverlight animation framework.  The code on his site was a good first step in creating higher-level abstractions, going above DoubleAnimation to define PositionAnimation and RotationAnimation, and I decided to build on top of that, adding other abstractions as well as a fluent-style API in the form of extension methods that hide even those classes.

Concepts

All Animate.NET animations derive from the Animation class which tracks the UI element being modified, the duration of animation, whether it has completed, and fires an event when the animation completes.  It manages building and executing the Storyboard object so you don’t have to.

Subclasses of Animation currently include OpacityAnimation, PositionAnimation, RotationAnimation, SizeAnimation, TransformAnimation, and GroupAnimation.  TransformAnimation is the parent class of RotateAnimation, and in the future ScaleAnimation and TranslateAnimation may also be included.

GroupAnimation is special because it allows you to combine multiple animations.  These groups can be nested and each group can include a wait time before starting (to stagger animations).

The Animate static class includes all of the extension methods that make up the fluent API, and the intention is for this to be the master class for building complex group animations.  Most of these methods come in pairs: you can RotateTo a specific angle or RotateBy relative to your current angle; MoveTo a specific location or MoveBy relative to your current position, etc.

Here’s the list so far:

• Group and Wait
• Fade, FadeIn, FadeOut, and CrossFade
• RotateTo and RotateBy
• ResizeTo and ResizeBy
• MoveTo and MoveBy

Examples

Animate.NET can best be understood and appreciated with examples.

Basic Animations

Let’s say you want to resize an element to a new size.  Normally you’d need a storyboard and two DoubleAnimations: one for x and another for y, and for each you’d need to set several properties.  With Animate.NET, you can define and execute your animation beginning with a reference to the element you want to animate:

var rect = new Rectangle()
{
    Height = 250,
    Width = 350,
    Fill = new SolidColorBrush(Colors.Blue)
};
MainStage.Children.Add(rect);
rect.SetPosition(50, 50);

rect.ResizeTo(150, 150, 1.5.seconds()).Begin();

Only a single line of code, the last one, is needed to resize the rect element.

Note the call to Begin.  Without this, the ResizeTo (and all other fluent API calls) will return an object that derives from Animation but will not run.  We can, if needed, obtain a reference to the animation and begin the animation separately, like this:

var anim = rect.ResizeTo(150, 150, 1.5.seconds());
anim.Begin();

This allows us to compose animations into groups and manipulate animations after they’ve started, and is very similar to how LINQ queries are composed and later executed.

You’ll also notice the use of several other extension methods:

• SetPosition – sets Left and Top currently.  In future versions, you’ll be able to define a registration point for positioning that may be located elsewhere, such as the center of the element.
• seconds() – along with milliseconds, minutes, etc., allows you to specify a TimeSpan object more fluently.  I saw this in some Ruby code and loved it.  If only the C# team would implement extension properties, it would look even cleaner (eliminate the need for parentheses).
• Center() and GetCenter() – centers an element immediately, and gets a Point object representing the center of the object respectively.  Not used in these examples, but worth mentioning.

Group Animations

Next I’ll show an example of a group animation using the Animate class’s Group method:

Animate.Group(
    rect.RotateBy(rect.GetCenter(), -90, 1.seconds()),
    rect.FadeOut(1.seconds())
    ).Begin();

This group animation contains two child animations: one to rotate the rectangle 90 degrees counterclockwise, and the other to fade the rectangle out (make it completely transparent).  The method takes a params array, so you can include as many animations as you like.

Because the animations listed are peers in the group, they begin running at the same time.  Often you will want to stagger animations, however.  You can accomplish this with the Wait method, which is the Group method in disguise (it simply includes an additional TimeSpan parameter).

Animate.Group(
    rect.RotateBy(rect.GetCenter(), -90, 1.5.seconds()),
    rect.FadeIn(0.5.seconds()),
    Animate.Wait(1.seconds(),
        rect.FadeOut(0.5.seconds())
        )
    ).Begin();

This animation rotates the rectangle for 1.5 seconds.  During the first 0.5 seconds, it fades in; during the last 0.5 seconds, it fades out.  Only one element, rect, was used in this example, but any number of UI elements can participate.

Animations can be nested and staggered to arbitrary complexity.  Because all animations derive from the Animation class, you can write properties or methods to encapsulate group animations, and assemble them programmatically before executing them.  Because all the ceremony of storyboards and keyframes is abstracted away, it’s very easy to see what’s happening in this code in terms of the end result.

Method Chaining

One of the benefits of a fluent API is the ability to chain together methods that modify a primary object.  For example, the Animation class defines a WhenComplete method that can be used to respond to the completion of an animation.  In the samples project on CodePlex, I create new UI objects at the beginning of each animation, and remove them afterward:

rect.ResizeTo(150, 150, 1.5.seconds())
    .WhenComplete(a =>
    {
        Thread.Sleep(2000);
        MainStage.Children.Remove(rect);
    })
    .Begin();

I pause for a couple seconds after displaying the final result before removing that object from its container.

Extension methods will be used more in the future for this library.  Uses will include modifying the animation to apply easing functions, responding to collision detection (by stopping or reversing), and so on.  This might end up looking something like:

rect.ResizeTo(150, 150, 1.5.seconds())
    .Ease(EasingFunction.Cubic(0.5))
    .StopIf(a => Animate.Collision(GetCollisionObjects()))
    .Begin();

Feedback and Future Direction

I’m releasing this as a very early experiment, and I’m interested in your feedback on the library and its API.

What kind of functionality would you like to see added?  Do the method names and syntax feel right?  What major, common animation scenarios have I omitted?  What other kinds of samples would you like to see?

Download the library and samples and give it a try!

Filtering with Metadata in the Managed Extensibility Framework

The Managed Extensibility Framework (MEF) is the new extensibility framework from Microsoft.  Pioneered by Glenn Block in the patterns & practices group, and leveraged by the behemoth Visual Studio 2010, it has a striking resemblance to my own Inversion of Control (IoC) and Dependency Injection (DI) framework—which led to me to have a couple great conversations about IoC with Glenn at Tech Ed 2008 and then again at PDC 2008.

But MEF isn’t really written to be your IoC.  Instead, the IoC engine and DI aspects are implementation details, allowing you to do really no more than “MEF things together”.  The core concept of MEF is to provide very simple and powerful application composability.  Not in the user interface composition sense—for that, see Prism for WPF and Silverlight (explained in MSDN Magazine, September 2008)—but for virtually all other dynamic component assembly needs, MEF is your best friend.

The two things I like most about MEF is its simplicity as its lack of presumption on how it will be used.  Compose collections of strings, single method delegates, or implementations of complex services.  All you’re doing is importing and exporting things, with little code required to wire things up.

MEF is currently in its seventh preview release, so expect beta-like quality.  My own experience with it has been very positive, but there are a number of shortcomings in the API.  This article is about a few of them and what can be done to add some much-needed functionality.

System.AddIn vs. MEF

There’s been some confusion with Microsoft coopetition among products with similar aims, and extensibility and composition are no exception.  The AddIn API (team blog) serves a similar purpose as MEF.  (See this two-part MSDN article on System.AddIn: first and second.)  The primary differentiator, from my understanding, is that the AddIn API is a bit more robust and a lot more complicated, and supports such things as isolating extensions in separate AppDomains.

With Visual Studio siding with MEF, it’s personally hard for me to imagine using the AddIn API.  If MEF is flexible and robust enough for Visual Studio, is it really likely to fall short for my own much smaller software systems?  Krzysztof Cwalina suggests they are complementary approaches, but I find that hard to swallow.  Why would I want to use two different extensibility frameworks instead of one coherent API?  If anything, I imagine that the lessons learned from the AddIn API will eventually migrate to MEF.

Daniel Moth notes that with the AddIn API, “there are many design decisions to make and quite a few subtleties in implementing those decisions in particular when it comes to discovering addins, version resiliency, isolation from the host etc.”  A customer of mine using the AddIn API was using a Visual Studio plug-in to manage pipelines, and things were a real mess.  There were a bunch of assemblies, a lot of generated code, and not much clarity or confidence that it was all really necessary.

MEF: Import & ImportMany

In MEF, the Import attribute allows you to inject a value that is exported somewhere else using the Export attribute—typically from another assembly.  There is also an ImportMany attribute which is useful when you expect several exports that use the same contract.  By defining an IEnumerable<T> field or property and decorating it with the ImportMany attribute, all matching exports will be added to an enumerable type.

[ImportMany]
public IEnumerable<IVehicle> Vehicles;

What if you want to filter the exported vehicle types by some kind of metadata, though?  Let’s take a look at the IVehicle contract and some concrete classes that implement the contract.

public interface IVehicle { }

[Export(typeof(IVehicle))]
[ExportMetadata("Speed", "Slow")]
public class ToyotaPrius : IVehicle
{
    public ToyotaPrius() { }
}

[Export(typeof(IVehicle))]
[ExportMetadata("Speed", "Fast")]
public class LamborghiniDiablo : IVehicle
{
    public LamborghiniDiablo() { }
}

The object model isn’t very interesting, but that’s not the point.  What is interesting is that MEF allows us to supply metadata corresponding to our exports.  In this case, my contrived example has defined a metadata variable of “Speed”, with two possible values: “Fast” and “Slow”.  The variable name must be a string, but its value can be any value; that is, any value that’s supported from within an attribute, which means string literals and constants, type objects, and the like.

Filtering Imports on Metadata

What if you want to ImportMany for all exports that have a particular metadata value?  Unfortunately, there are no such options in the ImportMany attribute class.

In my scenario, I’ve defined a static factory class called VehicleFactory, which at some imaginary point in the future will be responsible for building a city full of trafic.

public static class TrafficFactory
{
    // type initialization fails without a static constructor
    static TrafficFactory() { }

    public static IEnumerable<IVehicle> SlowVehicles =
        App.Container.GetExportedValues<IVehicle>(metadata => metadata.ContainsKeyWithValue("Speed", "Slow"));

    public static IEnumerable<IVehicle> FastVehicles =
        App.Container.GetExportedValues<IVehicle>(metadata => metadata.ContainsKeyWithValue("Speed", "Fast"));

    public static IDictionary<object, IVehicle> AllVehicles =
        App.Container.GetKeyedExportedValues<IVehicle>("Speed");
}

This is what I want to do, but there is no overload of GetExportedValues that supplies a metadata-dependent predicate function.  Adding one is easy, though.  While we’re at it, we’ll also add the ContainsKeyWithValue which I borrow from The Code Junky article also on MEF container filtering.

public static class IDictionaryExtensions
{
    public static bool ContainsKeyWithValue<KeyType, KeyValue>(
        this IDictionary<KeyType, ValueType> Dictionary,
        KeyType Key, ValueType Value)
    {
        return (Dictionary.ContainsKey(Key) && Dictionary[Key].Equals(Value));
    }
}

public static class MEFExtensions
{
    public static IEnumerable<T> GetExportedValues<T>(this CompositionContainer Container,
        Func<IDictionary<string, object>, bool> Predicate)
    {
        var result = new List<T>();

        foreach (var PartDef in Container.Catalog.Parts)
        {
            foreach (var ExportDef in PartDef.ExportDefinitions)
            {
                if (ExportDef.ContractName == typeof(T).FullName)
                {
                    if (Predicate(ExportDef.Metadata))
                        result.Add((T)PartDef.CreatePart().GetExportedValue(ExportDef));
                }
            }
        }

        return result;
    }
}

Now we can test this logic by wiring up MEF and then accessing the two filtered collections of cars, which will each contain a single IVehicle instance.

class App
{
    [Export]
    public CompositionContainer Container;

    static void Main(string[] args)
    {
        AssemblyCatalog catalog = new AssemblyCatalog(Assembly.GetExecutingAssembly());
        Container = new CompositionContainer(catalog);
        Container.ComposeParts();

        var FastCars = TrafficFactory.FastVehicles;
        var SlowCars = TrafficFactory.SlowVehicles;
    }
}

Viola!  We have metadata-based filtering.

You’ll also noticed that I added an Export attribute to the Container itself.  By doing this, you can Import the container into any module that gets dynamically loaded.  It’s not used in this article, but getting to the container from a module is otherwise impossible without some kind of work-around.  (Thanks for pointing out the problem, Damon.)

Using Metadata to Assign Dictionary Keys

Let’s take this one step further.  Let’s say you want to import many instances of MEF exported values into a Dictionary, using one of the metadata properties as the key.  This is how I’d like it to work:

public static IDictionary<object, IVehicle> AllVehicles =
    App.Container.GetKeyedExportedValues<IVehicle>("Speed");

Again, the current MEF Preview doesn’t support this, but another extension method is all we need.  We’ll add two, so that one version gives us all exported values and the other allows us to filter that selection based on other metadata.

public static IDictionary<object, T> GetKeyedExportedValues<T>(this CompositionContainer Container,
    string MetadataKey, Func<IDictionary<string, object>, bool> Predicate)
{
    var result = new Dictionary<object, T>();

    foreach (var PartDef in Container.Catalog.Parts)
    {
        foreach (var ExportDef in PartDef.ExportDefinitions)
        {
            if (ExportDef.ContractName == typeof(T).FullName)
            {
                if (Predicate(ExportDef.Metadata))
                    result.Add(ExportDef.Metadata[MetadataKey], 
                        (T)PartDef.CreatePart().GetExportedValue(ExportDef));
            }
        }
    }

    return result;
}

public static IDictionary<object, T> GetKeyedExportedValues<T>(this CompositionContainer Container,
    string MetadataKey)
{
    return GetKeyedExportedValues<T>(Container, MetadataKey, metadata => true);
}

Add an assignment to TrafficFactory.AllVehicles in the App.Main method and see for yourself that it works.

If you’re using metadata values as Dictionary keys, it’s probably important for you not to mess them up.  I recommend using enum values for both metadata property names as well as valid values if it’s possible to enumerate them, and string const values otherwise.

Now go forth and start using MEF!