Phidgets Robotics Programming in C#

Download the Source

Bridgeware is the word you need to know for rapidly prototyping electronics and robotics.  These components bridge programs running on computers with electronic components such as sensors and motors.  There’s a good video about it by Trossen Robotics on You Tube.  I bought my components from Trossen Robotics online and I recommend them (they have a number of how-to videos).

Day 1

I got my Phidgets components in the mail, and went right to work assembling my pan-tilt camera mount, controlled with two servos from a controller board, and connected to my laptop from there by a USB cable.  A tiny allen wrench was included, but not a phillips screwdriver, which you will also need.  You can see the components assembled and connected in the picture below.

The black thing in the back right corner is just a USB hub.  This is connected to (on the right) an 8/8/8 controller (with lots of room for more components to plug in), and then to the mini joystick.  The other USB connector plugs into a 4-servo controller, and then to the dual servos in the pan-tilt assembly in the back left of the picture.

On the software side, using the Phidgets .NET API was very easy.  Just instantiate a Servo object, provide an event handler to run when a device is attached, and from there I was able to set each servo position or read the position back.  The only confusing part was realizing that the servo controller board is represented by the Servo class, and the individual servos plugged into that controller are represented by the ServoServo class.  (Was this API written in Bora Bora, perhaps?)  I would have named them ServoController and Servo, respectively, but I got over it and moved on.

What you see visualized in the test application (see screenshot below) is a coordinate graph custom control that displays the position of both servos.  When I hooked up the mini joystick, I made the mistake of mapping the joystick position to the graph, but then realized that my graph would be useless unless I was controlling everything with the joystick.  I wanted to script out servo movements and replay the sequences and still have their position represented in the coordinate graph control, so I made that update (and ever since have been unable to calibrate the system to center itself).

Hooking up the hardware took only 15 minutes, and most of the application involving the Phidgets API took an hour or two at the most, but I spent the rest of the day creating the custom graph control and getting it to translate coordinate systems correctly.  The joystick maps values from 0 to 1000 along each axis, the servos have a servo position range of -22 to 232 on one axis and something close to that on the other, and the graph control was 150 pixels wide.  I made it general enough to work with any coordinate ranges on both axes.

First Impressions

I have to say that it’s really cool to see physical objects interacting with software that you write yourself.  (It reminds me of fun I had with a hydraulic robotic arm I programmed in high school using an Apple 2c and low-level calls to the parallel port, but this is way more powerful).  The bridgeware components are easy to connect, and the APIs are a breeze to use.  Building the intelligence between inputs and outputs, however, is the really fun and challenging part.  Even though this initial experiement was simple, I can already tell that coordinating a much more complicated set of inputs and outputs will require careful planning and the use of tools such as Microsoft Robotics Studio, which include the Concurrency & Coordination Runtime and Decentralized Software Services Protocol.

Now that I’ve gotten my feet wet and have some confidence that I can build, connect, and interface with these components (and have a box full of other goodies like 3-axis accelerometers, light and temperature sensors, sliders, buttons, and switches), I have a bunch of ideas for at least one cool summer project that I hope to announce in the next month or two.

VS Macro to Locate Solution Explorer Item – Corrected

A few weeks ago, I posted an article about a Visual Studio macro (with source code) to locate and select an item in Solution Explorer corresponding to the active document.  I discovered that Solution Explorer loads its UIHierarchy items on demand, so items only become available in thet tree structure if their parent node has been expanded.

The key to solving this was a single line of code:

solution.FindProjectItem(DTE.ActiveDocument.FullName).ExpandView()

This forces Solution Explorer to load the necessary items and also expands the tree so that the item becomes visible.  After that, selecting the item becomes possible.

I’ve updated the original article to correct the source code, and from what I can tell from more extensive testing, it seems to work all the time now.

Tree Data Structure Source Code Posted

[Updated 8/14/2014] The source code for this library can be found here at GitHub. Also check out my blog post announcing it.

For those of you who have been waiting for the source code to the Tree<T> article I wrote, I’m happy to announce that it’s finally available.  I had originally intended to supply some non-trivial examples of its use, but with my super busy schedule at work and otherwise, I’ve had to reduce the scope just to get it out there.  One of my examples is large enough to warrant one or more large articles, so I also didn’t want to just toss them in there without sufficient explanation as to how it all works.

While I work on getting those ready, check out Damon Payne’s article on using a non-binary tree for modeling dependencies among concurrently-running tasks with the Task Parallel Library.  This is a great use of the tree data structure.  It would be interesting to see how that would work for tasks with cross-branch dependencies.  Does the tree become a graph?  Would iteration become a garbage-collection-like traversal?  How would it need to respond to insertion of tasks or modification or dependencies during execution?

My non-binary tree article, which has been updated with a short section just before the conclusion, can be found here.  At the very top of the article is a link to the source code in the form of a Visual Studio 2008 solution.  For those of you with VS2005, you should be able to easily extract the code and create a VS2005 project out of it.  I specifically targeted .NET 2.0 instead of 3.5, and originally tested it against Compact Framework.

I’ll also be doing further development of this library, so if you have any cool feature requests, let me know.

Project: Code-Named “SQL Mobile Bridge”

This is not the final name.  But it will be a useful product.  With as much as I’ve been working with rapidly-evolving mobile database schemas lately, I expect to save from 30 minutes to an hour a day in my frequent build-deploy-test cycles.  The lack of a good tool for mobile device database queries causes me a lot of grief.  I know Visual Studio 2008 has something built-in to connect to mobile devices over ActiveSync, but let’s face it: ActiveSync has been a real pain in the arse, and more often fails than works (my next blog will detail some of those errors).  I can only connect to one device at a time, and I lose that connection frequently (meanwhile, SOTI Pocket Controller continues to work and communicate effectively).  Plus I have a window constantly bugging me to create an ActiveSync association.

I work on enteprise systems using sometimes hundreds of Windows Mobile devices on a network.  So I don’t want to create an association on each one of those, and getting ActiveSync to work over wireless requires an association, as far as I know.

Pocket Controller or other screen-sharing tools can be used to view the mobile device, and run Query Analyzer in QVGA from the desktop, but my queries get big and ugly, and even the normal-looking ones don’t fit very well on such as small screen.  Plus Query Analyzer on PDAs is very sparse, with few of the features that most of us have grown accustomed to in our tools.  Is Pocket Query Analyzer where you want to be doing some hardcore query building or troubleshooting?

So what would a convenient, time-saving, full-featured mobile database query tool look like?  How could it save us time?  First, all of the basics would have to be there.  Loading and saving query files, syntax color coding, executing queries, and displaying the response in a familiar “Query Analyzer”/”Management Studio” UI design.  I want to highlight a few lines of SQL and press F5 to run it, and I expect others have that instinct as well.  I also want to be able to view connected devices, and to use several tabs for queries, and to know exactly which device and database the active query window corresponds to.  No hunting and searching for this information.  It should also have the ability to easily write new providers for different database engines (or different versions of them).

Second, integration.  It should integrate into my development environments, Visual Studio 2005 & 2008.  It should also give an integrated list of databases, working with normal SQL Server databases as well as mobile servers.  If we have a nice extensible tool for querying our data, why limit it to Windows Mobile databases?

Sometimes it’s the little details, the micro-behaviors and features, the nuances of the API and data model, that defines the style and usefulness of a product.  I’ve been paying a lot of attention to these little gestures, features, and semantics, and I’m aiming for a very smooth experience.  I’m curious to know what happens when we remove all the unnecessary friction in our development workflows (when our brains are free to define solutions as fast as we can envision them).

Third, an appreciation for and focus on performance.  Instead of waiting for the entire result to return before marshalling the data back to the client, why not stream it across as it’s read? — several rows at a time.  Users could get nearly instantaneous feedback to their queries, even if the query takes a while to come fully across the wire.  Binary serialization should be used for best performance, and is on the roadmap, but that’s coming after v1.0, after I decide to build vs. buy that piece.

Finally, a highly-extensible architecture that creates the opportunity for additional functionality (and therefore product longevity).  The most exciting part of this project is probably not the query tool itself, but the Device Explorer window, the auto-discovering composite assets it visualizes, and the ability to remotely fetch asset objects and execute commands on them.

The Device identifies (broadcasts) itself and can be interrogated for its assets, which are hierarchically composed to represent what is visualized as an asset tree.  One Device might have some Database assets and some Folder and File assets.  The Database assets will contain a collection of Table assets, which will contain DatabaseRow and DatabaseColumn assets, etc.  In this way, the whole inventory of objects on the device that can be interrogated, discovered, and manipulated in a standard way that makes inherent sense to the human brain.  RegistryEntry, VideoCamera, whatever you want a handle to.

This involves writing “wrapper” classes (facades or proxies) for each kind of asset, along with the code to manipualte it locally.  Because the asset classes are proxies or pointers to the actual thing, and because they inherit from a base class that handles serialization, persistence, data binding, etc., they automatically support being remoted across the network, from any node to any other node.  Asset objects are retrieved in a lazy-load fashion: when a client interrogates the device, it actually interrogates the Device object.  From there it can request child assets, which may fetch them from the remote device at that time, or use its locally-cached copies.  If a client already knows about a remote asset, it can connect to and manipulate it directly (as long as the remote device is online).

With a remoting framework that makes shuffling objects around natural, much less message parsing and interpretation code needs to be written.  Normal validation and replication collision logic can be written in the same classes that define the persistent schema.

So what about services?  Where are the protocols defined?  Assets and Services have an orthagonal relationship, so I think that Services should still exist as Service classes, but each service could provide a set of extension methods to extend the Asset classes.  That way, if you add a reference to ServiceX, you will have the ability to access a member Asset.ServiceXMember (like Device.Databases, which would call a method in MobileQueryService).  If this works out the way I expect, this will be my first real use of extension methods.  (I have ideas to extend string and other simple classes for parsing, etc., of course, but not as an extension to something else I own the code for.)  In the linguistic way that I’m using to visualize this: Services = Verbs, Assets = Nouns.  Extension methods are the sticky tape between Nouns and Verbs.

public static AssetCollection<Database> Databases(this Device device) { }

With an ability to effortlessly and remotely drill into the assets in a mobile device (or any computer, for that matter), and the ability to manipulate them through a simple object model, I expect to be a significantly productive platform on which to build.  Commands executed against those assets could be scripted for automatic software updates, they could be queued for guaranteed delivery, or they could be supplemented with new commands in plug-in modules that aid in debugging, diagnostics, runtime statistics gathering, monitoring, synchronizing the device time with a server, capturing video or images, delivering software updates, etc.

And if the collection of assets can grow, so can UI components such as context menu items, document windows, and so on, extending and adding to the usefulness of the Device Explorer window.  By defining UI components as UserControls and defining my own Command invocation mechanism, they can be hosted in Visual Studio or used outside of that with just a few adjustments.

More details to come.

Tags: Software+Architecture, Software+Design, Object+Oriented+Design, SQL+Server+Mobile, Development+Tools, Compact+Framework, Problem+Modeling

Visual Studio Add-In: Reference Sentinel

My article on Project Reference Oddness has been the most popular so far.  I like seeing the blog statistics, and I’m constantly observing the search terms people use to get to my blog.  I guess I shouldn’t have been surprised by the number of people having problems with references and searching for answers online.  It’s a pretty messed up system.

I’ve been contemplating writing an add-in for Visual Studio that would track and manage references, set some rules, and enforce them. I’d like to be able to specify that I use v3.1 of Library X across all projects in a solution. I’d then like to be able to make a change in one place (like a Reference Explorer) and have it update all projects in the solution. If one of the references changes outside of this system (for whatever reason), it should act as a sentinel, detecting the change and changing it back to what you defined in the Reference Explorer.  (It should also notify you that this has happened to help troubleshoot the root cause.)

It could also set reference paths.  By saving this information as an extension to the solution file, it could be checked into source control and shared with the rest of your development team.  This should eliminate the warnings and errors you get when you accidentally reference different versions of the same assembly.  It would also reduce the pain of creating a new branch or pulling code down to a new machine and having to spend a bunch of time setting the reference paths over and over again.

Am I missing anything?  What other reference pains could be alleviated?

Tags: Visual+Studio, Project+Reference, Reference+Path

Multithreaded Design: Dedicated Task Threads or Bucket Brigade Strategy?

A few days ago, I took a dive into building a new software product.  It’s aim is mobile developers, such as myself, whose applications access databases on those mobile devices.  After six weeks of development, v1.0 will be released (by the end of January, 2008).  More details on that product to come later.

One of my goals is that its user interface should be very responsive.  When commands take a while to complete, they’ll need to execute on some other thread.  It’s a .NET Framework application, and in that environment, only one thread can update the user interface, so it’s a rare resource that needs to be protected from doing too much work that doesn’t fit that primary purpose.

I’ve written a fair amount of multithreaded code in the past.  Working with RFID hardware before the “Gen 2” standard, on pre-release Symbol (now Motorola) devices, I figured out how to integrate with the RFID radio at a low level, and only by using several threads was I able to obtain acceptable performance.  After wrestling with the issues like race conditions, deadlocks, and other complexities, I have a healthy respect for the amount of planning that’s required to make it work correctly.

My new product consists of three major layers: a user interface layer, a service layer, and a network communication layer.  The Command pattern will be used to publish and service commands.  Actions in the user interface will enqueue command objects to the service layer, where they’ll either be processed immediately or passed to another node in the network for processing remotely.

A simplification of this scenario is illustrated in the sequence diagram below.  Each color represents a separate thread.

The red user interface thread does minimal work.  When it calls the DataServiceProxy to request data, a command object is enqueued and the thread is allowed to return immediately to the user interface.  The blue line represents one thread in a larger pool of worker threads that grows or shrinks depending on the command queue size, and is dedicated to processing these commands.  In this example, the DataServiceProxy makes a call to the TCPConnection object and returns back to the pool.  When a message does arrive, this happens on a thread owned by the TCPConnection object.  Because we don’t want this thread to “get lost” and therefore be unavailable to process more TCPConnection logic, we enqueue a response object and return control immediately.  Finally, the DataServiceProxy object, using one of its threads, fires a MessageReceived event, which in turn calls Invoke so that it can update the user interface on the only thread that is capable of doing so.

I like using Sequence diagrams like this to plan multithreaded work.  It clears up issues that you wouldn’t normally be able to visualize clearly.  The first (and worst) mistake would be to allow the UI thread to cross all the way through to the TCPConnection object, making some blocking call while the network connection times out, throws an exception, handles the exception, etc.  Meanwhile, the user is waiting until they can interact with the screen again, waiting for the hourglass to go away so they can get on with their work.  By assigning specific roles to threads, and creating mechanisms by which threads can be kept within specific boundaries, their behavior is easier to predict and manage.

I don’t know a lot about how ThreadPools behave, but I’m going to be looking into that in detail over the next few weeks, and if I come across anything particularly noteworthy, I’ll write about it.  My instinct is to continue to use this pattern of thread role assignment, especially for my new product, but there’s another pattern I’ve been thinking about that may also work well.  A friend of mine at Bucketworks mentioned to me a while back about the efficiency of bucket brigades, and about how intensely they’ve been studied and found to create self-organizing and automatically-optimizing lines of workers.

Each Thread Is A Worker

The way I’ve been modeling thread work, I treat each thread like a worker: the UI worker hands off a job description to a DataServiceProxy worker, who then does something with it and passes it off to a TCPConnection worker, who in turn does some work with it and hands it off (over a network in that case).  So what happens when the scenario becomes more complicated than the simple sequence diagram above?  What if I have four or five hand-offs?  Is the assignment of specific threads to specific roles really the best use of those threads?  The UI thread is stuck, but the rest of them could technically move around.  Perhaps they’re devoted to one task at a time, but when a task is completed with nothing in that queue, could the worker thread be moved to perform another task?  So I started thinking about how a bucket brigade strategy may help.

Threads aren’t exactly like people workers, so I have to be careful about the analogy.  It’s probably no big deal processing-wise for threads to be sleeping if they’re not used, but each thread does consume some memory, and there must be some kind of overhead to deallocate and reallocate their thread-local resources.  Would it be better to share a single thread pool among a connected linkage of worker objects, or for each service object to allocate its own thread pool?

Will this work?  Bucket brigades tend to work best when the density of work is high and the amount of time to complete a task is relatively small and constant.  If a task takes too long to complete, the line can more easily become unbalanced.  I’m going to be thinking more about this possibility, and come up with some objective measurements that I can use to compare the approaches and determine when this would be appropriate in a thread-management strategy (if at all).

Bucket Brigades In Business

If you want some background on bucket brigades and their use in manufacturing and assembly lines, check out this paper by John Bartholdi and Donald Eisenstein.  They explain how expensive time-motion studies are done to balance assembly lines in large organizations to maximize throughput, and how this needs to be adjusted or redone when the length or configuration of work changes, or when worker productivity varies.  With bucket brigades, however, there’s less need for planning because the strategy “spontaneously generates the optimal division of work”.  Pretty neat stuff.

The list of companies using bucket brigades is impressive: McGraw-Hill, Subway (yes, the sandwich company), Readers Digest, Ford Customer Service Division, The Gap, Walgreen’s, Radio Shack, etc.

If anyone’s using this kind of strategy to manage work in software systems, I’d love to hear about it.

Tags: Bucket+Brigade, Multithreading

Panatracker – Mobile Inventory/Asset Tracking System

For the past three and a half years, I’ve been developing software for Panatrack to track inventory, assets, time and attendance, and more, using primarily Symbol PDAs with integrated barcode and RFID scanners.  This software system is called Panatracker, and it has modules that integrate into DynamicsGP (formerly called Great Plains) as well modules that can run independent of a back-end system.  Here are a few screen shots of the splash and menu screens:

Pretty, isn’t it?  One of the reasons we consistently win when going up against our competitors is the nice looking, easy to use interface.  We take advantage of the color touch screen, making sure everything is easy to manipulate with a simple touch, without having to pull out the stylus (which is annoying and not very practical in warehouse and retail environments).  The menus have big, easy to push buttons (even when wearing gloves, while working in a freezer, for example), and all workflows are streamlined for extreme efficiency and an intuitive feel.  Compare this to old monochrome telnet applications that are all text and provide poor formatting, navigation, and connectivity options.  It’s displayed in the picture above running on a Symbol MC50, but we also deploy to MC70, MC9090 (which looks like a gun with its trigger handle), and occassionally other devices.

This has been a labor of love, and it shows: in breadth of functionality, ease of use, modularity, upgradability, extensibility, and much more.  It’s been designed with solid object-oriented principles and is positioned to remain a significant and relevant product for the long run.  It’s built upon a highly reusable framework; new transactions and supporting user interface screens can be plugged into its shell easily.  It operates not only in a connected environment (over a wireless network), but also supports occassionally-connected workflows, using data synchronization so that it can be used when disconnected from the server.  Data is collected out in the field, and then synchronized over a cellular network in a secure manner once a connection is available again.

There is a complete administrative website as well that provides access to reports, centralized configuration and management of all handheld devices, and more.  Here’s a teaser screen shot of that (shrunk down a bit to fit better on the blog page):

It is growing so rapidly now (in functionality as well as sales) and is always exciting to work on.  I remember looking at an empty shell, a single menu item, and now we have to use multiple menu levels, grouping our workflows into several modules.  Pretty soon we’ll support so many different supply chain workflows that we’ll run out of menu space!

The retail RFID module of Panatracker was even shown on TV in the news.  It’s about 3:15 into the video clip.

Before Gen 2 of RFID came around, before Symbol’s MC8069R hardware want into mass production, we got our hands on one and I figured out how to integrate into it, in a realistically performant way (using several threads and some clever tricks) when companies many times our size couldn’t figure it out and even came to us for help.  We got several contracts with large companies that I don’t think I can name publicly, but one of them was a major big box retailer (hint: their/our app was in the news).

But ultimately, RFID is a difficult technology to justify.  Once you’ve waded through the hype, you realize that the engineering is more difficult to justify and satisfy that one would optimistically expect.  Radio waves are easily reflected and distorted by metals, liquids, special surfaces, and magnetic sources.  You can confirm failed writes to tags, but you can’t always confirm a successful write, nor can you easily determine what tag or tags were written to.  If many tags are in the vicinity, they are difficult to isolate and verify which tags you’re reading.  High-density environments, we call them.

One project I was involved with was based upon a Symbol Apex device (Apex may be a code name and not a model name), which contains a proximity sensor, an accelerometer, and an RFID radio (plus a wireless network radio) running Windows Mobile.  To minimize the damage done to product moved with clamp trucks in warehouses (millions of dollars per year), we would need to detect when the clamp truck slowed down and approached product, activate the RFID scanner, determine the number and stacking configuration of products, look up in a database the correct amount of pressure to apply, and then activate the clamp device to close with the correct pressure.  The largest obstacle and challenge was the chosen device and lack of leeway and budget in using additional sensory devices, and therefore the ability to realistically determine the stacking configuration.  How do you count RFID tags when there are likely to be many others within reading range?  Can you come up with a scheme for raising and lowering the radio power settings depending on the mode the device is in (what function it is trying to accomplish)?  How accurately can you model the physical geometry to isolate what you’re aiming at from a fast moving vehicle with a 400–600 MHz processor that is already overburdened with tasks?

Mobile devices are a challenge not just because of their space-constraint-caused limitations, but because they are mobile and therefore involved more in dynamic situations and environments.  A computer that sits stationary on a desk isn’t very interesting, but there are some cell phones that really get around and are starting to be used in some interesting ways.  The trend is growing exponentially.  Mobility is the seed of digital ubiquity, and we all know deep down that’s where we’re headed.  But that’s just futurist speak.  The point is that we’re at the beginning of a very exciting thing.  The technology seems clumsy and immature to me right now, but I have no doubt that we’ll see large strides forward in the near future.  Google’s Android open-source operating system and development platform looks fantastic, and their marketing is brilliant: to give away millions of dollars in programming contests to build applications for it.  I have a feeling this is going to grow a huge community and many good products, and that’s coming from a real .NET-C#-Microsoft fan.

Tags: Compact+Framework, Windows+Mobile, DynamicsGP, Great+Plains, Panatrack, Panatracker, Symbol, Handheld+Computers, Mobile+Devices, Great+Plains, DynamicsGP, Warehouse+Management+System, WMS, RFID, Mobile+RFID