关于移动应用UI部分管理的一些思考

我认为移动应用程序相对于桌面程序的主要特点是：强交互。

这样就引出一个问题：用什么样的设计可以使程序的 UI，Animation 等交互部分有序、受控？

我最近在思考这个问题，目前认为解决这类问题有两个思路：

1、基于游戏的技术，比如场景管理。对于游戏开发的领域不是很了解，这个暂且放一放。

2、基于状态机的UI管理框架。

在想到第二个思路时，用 Google 搜索了下 “GUI Architecture State Machine”，

从结果中看到很多有用的论文与博客，有点忧伤，有点高兴。

忧伤与高兴都是因为：自己不是第一个吃螃蟹的人。

下面摘取一些资料给自己还有大家做参考。

=========================================================================

ref：http://lassala.net/2008/02/05/state-machines-and-gui-interaction-part-i/

ref：http://lassala.net/2008/02/19/state-machines-and-gui-interaction-part-ii/

State Machines and GUI interaction – Part I

I’ve been doing some research recently regarding State Machines and GUI (Graphical User Interface) interaction. The motivation for this is that GUI interaction is something that keeps coming back at me one way or another, and it’s always something deemed painful by the developers working in the team.

For instance, let’s take the "enable/disabled" kind of logic: depending on different things, controls on the screen may be enabled or disabled (it could be related to a business rule, or to a security rule, or both). I’ve worked on a project years ago that any developer would shiver just because of the bare thought of having to fix a bug or touch code in a form that had a very heavy enable/disable kind of logic. It was probably one of the most fragile pieces of the application.

With UI’s getting more and more complex, I’ve been doing some research on how I could address this scenario in a way that’s flexible and reliable. I’m not sure I have found the silver bullet yet, but I think I’m getting there.

With this post I’ll start a series of posts where I’ll document my findings, and hopefully get feedback from various people.

A quick simple example

The following excellent book has some good chapters on state machine, state diagrams, and state design pattern:

Agile Principles, Patterns, and Practices in C# (Robert C. Martin Series) by Robert C. Martin, Micah Martin Read more about this title…

The book has a very quick but good example for a state machine in the GUI interaction realm. I decided to sit down and write some code that implements the example mentioned in the book. I’m going to quote the book’s explanation of the scenario:

"Imagine that you want to allow your users to draw rectangles on the screen. The gestures they use are as follows. A user clicks the rectangle icon in the pallet window, positions the mouse in the canvas window at one corner of the rectangle, presses the mouse button, and drags the mouse toward the desired second corner. As the user drags, an animated image of the potential rectangle appears on the screen. The user manipulates the rectangle to the desired shape by continuing to hold the mouse button down while dragging the mouse. When the rectangle is right, the user releases the mouse button. The program then stops the animation and draws a fixed rectangle on the screen.

Of course, the user can abort this at any time by clicking a different pallet icon. If the user drags the mouse out of the canvas window, the animation disappears. If the mouse returns to the canvas window, the animation reappears.

Finally, having finished drawing a rectangle, the user can draw another one simply by clicking and dragging again in the canvas window. There is no need to click the rectangle icon in the pallet."

After that explanation, there’s a state diagram, which I’ve roughly reproduced in Visio just to help me out here:

A State Machine toolkit and code generator

Somewhere in the book, it explains state machines, the state pattern, and a state machine code generator that the author wrote which is available for download.

The book explains state machines, and it also points out to a code generator that the author wrote that produces code to implement concrete state machines. The core of the explanation can be found in this article. I definitely recommend getting the book, though, since it also offers the code in C#, as well as unit tests that help understanding the design. The SMC (Finite State Machine Compiler) can be downloaded here.

Instead of using the generator mentioned by the book, though, I’ve decided to use the .NET State Machine Toolkit I found on the Code Project website. There are three excellent articles supporting the toolkit, and the articles not only explain state machines, but they also explain the design of the toolkit, how to use it to implement one simple and one slightly more advanced state machine, and how its state machine code generator works and has been implemented. I definitely recommend the reader to check out the tree parts of that article:

A .NET State Machine Toolkit – Part I
A .NET State Machine Toolkit – Part II
A .NET State Machine Toolkit – Part III

The final product of the sample application

The final product of the sample application I built here looks just like a simple form that works as a canvas where the user can draw a rectangle on it. The application changes the mouse pointer as the user draws the rectangle, and also changes it when the user has the left-button pressed but is moving it outside the boundaries of the form. I’ve also implemented a "fancier" version of the drawing algorithm so that it draws multiple rectangles and allows the user to create some simple effects. The fancier version also displays the coordinates of the new rectangle as the user draws it.

 

 

 

 

 

 

 

 

Yeah, I know, the application does not look all that exciting, but keep in mind that that’s not the point of this exercise. The idea is that if I have the correct design in place, a developer with good GDI+ plus skills could produce a little component that would take care of rendering some real fancy rectangles. Also, we should be able to take the state machine and implement it, say, in a WPF application.

The Implementation

The quick and dirty implementation of such application would basically override a few methods on the form to capture mouse clicks, do the rendering, etc. If I went that route, what would the code look like when I wanted to provide different implementations of the rendering logic? The code would certainly get very complicated and hard to maintain. Also, what if I wanted to create a WPF version of the application? I’d probably have to rewrite everything from scratch, since the rendering can be accomplished in better ways in WPF than in WinForms.

With all of that in mind I started the implementation of my little application by using the State Machine Maker (from the State Machine Toolkit), to produce a state machine based on the state diagram (showed earlier on in this post):

Here’s a State Transition Table with all the details for our state machine (this is just a simplification of the state diagram presented earlier):

Current State	Event	New State	Action (transition)
WaitingForClick	MouseDown	Dragging	RecordFirstPoint BeginAnimation
	Abort		StopAnimation
Dragging	MouseMove	Dragging	AnimateRectangle
	MouseLeave	OutOfCanvas	PauseAnimation
	MouseUp	WaitingForClick	StopAnimation DrawRectangle
	Abort		StopAnimation
OutOfCanvas	MouseEnter	Dragging	ResumeAnimation
	Abort		StopAnimation

The next step was to subclass the generated state machine (that way, if I need to regenerate the machine I don’t lose my changes). I named the subclass WinFormsRectangleToolStateMachine (from here on I’ll refer to this class as "the state machine". 

With the state machine ready, I could go ahead and start putting some code inside the "action" methods (such as DrawRectangle, AnimateRectangle, etc.). For instance, the AnimateRectangle method could look like this:

protected override void AnimateRectangle(object[] args)
{
    this.m_SecondPoint = (Point)args[0];
    Graphics g = (Graphics)args[1];
    g.Clear(Color.White);
    this.DrawRectangleCore(g);
}

Just for the records, args contains both the "second point" selected by the user (bottom-right of the rectangle), as well as the Graphics object to draw the rectangle on.

However, what if I wanted to support different algorithms for this "rectangle tool" of mine? Maybe I’d like to have a ways to draw "simple" rectangles, and/or "fancy" rectangles. In order to support these variances, I’d have to stick all the code in my action methods, and branch them into switch blocks or something along those lines.

That said, if I wanted to support a dozen different ways to render rectangles, my class would look pretty nasty and hard to maintain. Not only that, but what if I also needed to allow other people to write the rendering mechanism? I don’t like the idea of having to give them my source code so that they could add a little bit more mess to it.

Put some Controllers into the mix…

I decided to let my "RectangleToolStateMachine" only handle the change of states, but not put any implementation for the actions right into it. Instead, the machine should call out to a "RectangleToolController", which is the object responsible for actually implementing the actions. With that in mind, I’ve defined the following interface:

public interface IRectangleToolController
{
    void RecordFirstPoint(object[] args);
    void BeginAnimation(object[] args);
    void StopAnimation(object[] args);
    void PauseAnimation(object[] args);
    void ResumeAnimation(object[] args);
    void DrawRectangle(object[] args);
    void AnimateRectangle(object[] args);
}

Note that the IRectangleToolController interface declares methods that map to the actions/transitions specified by our state machine.

Next, I’ve changed the constructor for my WinFormsRectangleToolStateMachine so that it takes an instance of an IRectangleToolController:

And then I changed the implementation of my action methods on the state machine so that they delegate the actual actions to the controller (as opposed to implementing the actions itself):

I’ve then created a WinFormsOption1RectangleToolController class, which implements the IRectangleToolController interface, and filled in the methods with the code that handlers the rendering of simple rectangles.

Now when it comes to using my state machine and controller, it’s all a matter of instantiating the machine and pass a controller into it:

Triggering the transitions

The only part that’s left is to trigger the transitions on the state machine. Since my machine is used by a simple form (which I called "Canvas"), all I’ve had to do was to hook up to mouse events and trigger the appropriate transitions.

Every transition on my machine expects the same parameters: the Graphics object where we draw on, and the location where the mouse has been either clicked or released. I created a simple method that gathers those two parameters, send them to the state machine, and tell the machine to execute the transition:

private void SendEventToMachine(int eventId, Point mouseLocation)
{
    Graphics g = this.CreateGraphics();
    object[] args = new object[] { mouseLocation, g };

    m_RectangleToolSM.Send(eventId, args);
    m_RectangleToolSM.Execute();
}

And finally, my handlers for the mouse events on the form look pretty much like this:

In order words, the handlers just send the appropriate events to the state machine. There’s no code in the form whatsoever that deals with rendering rectangle; that responsibility is delegate to the controller associated with the machine (whose only responsibility is to handle the transitions).

Implementing other controllers

Just to try out my design, I created a "FancyWinFormsOption1RectangleToolController" class, which renders much fancier rectangles, and it also displays coordinates on the canvas as to where the rectangle is being rendered (wow! well, ok, I was just trying out the design here, so it’s not like I’ve put a lot of effort into coming up with better ways to render rectangles…).

The fact is that all I had to do was to:

• Create the new controller class, making it inherit from my IRectangleToolController interface;
• Implement how the controller handles rendering rectangles and the other methods exposed by the interface;
• Let my Canvas form know about my new controller.

Nothing else had to be changed. I could certainly have made it even better, by going with a plug-in architecture, where new controllers could be made available to the application just by registering them somehow (but that was beyond the scope of this exercise).

==============================================================

State Machines and GUI interaction – Part II

In this installment I’m going to cover a simple scenario that I’m always running across: a simple data-entry form, and the synchronization of its buttons used to load, create, or save data. The sample form I’ll cover here is pretty simple, and it looks like this:

That’s nothing more than a simple form to maintain Customer’s data. I’m just capturing the customer’s name, because the amount of data gathered here isn’t relevant to the exercise. The main thing I’m looking for here is the synchronization of the Load, New, and Save buttons. For instance, when the user clicks New, I want all the buttons to become disabled. As soon as the user makes any change to the data on the form, the Save button then becomes enabled. Then, when the user clicks on Save, both Load and New become enabled, whereas Save becomes disabled. Of course, there’s a lot more buttons we’d normally have in such form (such as Delete, Search, etc.), but that’s beyond the scope here.

The quick implementation would be to just go on the form and add some event handlers for the buttons, in which we’d enable/disable other controls accordingly. However, eventually we may want to move those buttons somewhere else (maybe put them a user control, as opposed to directly sitting on the form, or maybe put them in a toolbar or Ribbon control…). Also, we may want other ways to access the functionality (for instance, we may want to press Ctrl+N for New, and Control+S for Save…). The problem is that if all the logic is in the form, we’d have to put too much code in there so to handle buttons being somewhere, shortcuts, etc.

I decided to look at the form as it having different "states": for instance, a "viewing" state, which is the case when the user is viewing some data, and a "dirty" state, which is the case when the user has made changes to the data but hasn’t saved it yet.

Enter the State Machine

The following State Transition Table lists the simple states I’ve identified for my simple form:

Current State	Event	New State	Action (transition)
Initial	New	New
	Load	Viewing
New	DataChanged	Dirty	TurnOnDirtyIndicator
Dirty	Save	Viewing	TurnOffDirtyIndicator
Viewing	New	New
	DataChanged	Dirty	TurnOnDirtyIndicator

Below is a State Diagram to present another view of the states:

Using the State Machine Toolkit (discussed in Part I), I’ve created a state machine to handle those states. I’ve named the machine DataInterfaceInteractionBase (the idea being that the machine handles the interactions in a "data" user interface…). My default concrete implementation of this machine is called DataInterfaceInteraction. fThe diagram below shows what these classes look like:

The DataInterfaceInteraction class implements the abstract methods (TurnOnDirtyIndicator and TurnOffDirtyIndicator), as well as it overrides a few of the virtual methods (EntryDirty, EntryInitial, EntryNew, EntryViewing). The methods delegate execution to an IDataInterfaceInteractionController, like so:

The diagram below shows the relationship between the state machine and the controller:

Notice the following aspects of the IDataInterfaceInteractionController:

• It has methods that match the ones on the state machine (such as EntryDirty, EntryNew, TurnOnDirtyIndicator, etc.);
• The SetHost method is designed to take in a reference to the state machine that the controller handles;
• The TurnOnDirtyIndicator and TurnOffDirtyIndicator methods raise the TurningOnDirtyIndicator and TurningOffDirtyIndicator events, respectively;
• The RegisterInterfaceController method takes in "data interface controllers" and register them with the Interaction controller. Any number of interface controllers can be registered.

The constructor for that class looks like this:

It takes in an interaction controller, as well as an array of data interface controllers, and then uses those references to register things accordingly.

This is what the IDataInterfaceController looks like:

 

The properties essentially dictate whether Load, New, and Save features should be enabled or not. The SetInteractionStateMachine method takes an instance of the the interaction state machine that this controller is associated with.

The DataInterfaceInteractionController class

Next, I’ve implemented my default DataInterfaceInteractionController class, which we attach to the interaction state machine. Let’s look at the diagram to see how things are related thus far:

Notice the class has a DataInterfaceControllers collection, which is a Collection of IDataInterfaceController implementers. Nothing too fancy there. The RegisterInterfaceController method is used for adding controllers to that collection:

More importantly, the methods such as EntryViewing and EntryDirty (which get called when the state machine is entering those states), iterate through the data interface controllers and set the LoadEnabled, NewEnabled, and SaveEnabled properties accordingly:

Notice, for instance, that when the user is "viewing" some data, both Load and New are enabled, while Save is disabled. When the user then makes changes to the data, causing a transition to the Dirty state, both Load and New become disabled, whereas Save becomes enabled. In the real world we’d probably have more complex implementations so to decide whether or not some features are enable or disabled (for instance, we may have security restrictions depending on the user, or any other sort of business rule).

At this point we’re only determining which "features" should be enabled or disabled depending on the current state we’re in. That means we’re only programming the behavior here. We don’t have any code at this level that is enabling or disabling buttons, keyboard shortcuts, or anything like that; we leave that up to the implementes of IDataIntefaceController, which will take a look at next.

Hooking up the CustomerEditForm to the Interaction state machine

The CustomerEditForm has an Interactions property, which stores a reference to the DataInterfaceInteractionBase class (which is the abstract baseclass for the interactions state machine):

Setting up the Data Interface Interaction Controller

Next, I’ve created a GetInteractionController method, which instantiates the interaction controller, and sets up event handlers for its TurningOffDirtyIndicator and TurningOnDirtyIndicator events:

The event handlers are just simple anonymous methods. All we’re doing there is to display a little asterisk (*) after the title in the form’s title bar whenever the data is  "dirty":

Sending events to the State Machine

It’s necessary to send events to the state machine, so that it can transition to the appropriate states. We’ll first use the buttons located on the customer edit form to send those events:

Since I’ve based my state machine on a "passive" machine, anytime I send events to the machine (by calling the Send method and passing in the event id), I also have to call the Execute method. I created a simple helper method around these calls:

Next, I’ve hooked up event handlers for the Load, New, and Save buttons. Those handlers are very simple, since all they do is to send events to the state machine. I’d also handle any event that would warn me about the data being dirty (such as TextChanged events in this form…):

The first Data Interface Controller

Remember that the Data Interface Controller is the object that enables and disables the features (new, load, save…) accessed by the interaction state machine. We can have as many data interface controllers as we need.

The first implementer for the IDataInterfaceInteractionController interface is the customer edit form itself, since it hosts the buttons that send events to the state machine. The implementation of the interface is as simple as this (one property for each feature):

Nothing to exciting, right? Just some interaction with the Enabled property on the specific buttons.

I’ve then created an InitializeInteractions method, which I call from the form’s constructor. The method looks like this:

A few considerations:

• I call the GetInteractionController to get an instance of the Interfaction Controller that’s to be associated with the interaction state machine;
• I handle the TransitionCompleted event on the state machine (the event handler, shown below, just listens to transition completion on the state machine, and updates the status bar on the customer edit form accordingly);
• The state machine is then instantiated, with two parameters passed to the constructor:
  • a reference to the interaction controller;
  • an array of IDataInterfaceController objects (which at this point only has a reference to the form itself, since it does implement that interface).

This is the handler for the state machine’s TransitionCompleted event:

At this point, the form is operable. Now I need a few other interface controllers.

Triggering the actions with some fancier buttons

Let’s say that instead of having Load, New, Save buttons placed directly on the form, we wanted to create a fancy user control containing those buttons. Or maybe the buttons were sitting up at a toolbar or ribbon control. It doesn’t matter. The point is that now the form wouldn’t own the buttons anymore. I’ve went with the special user control for this example:

The user control itself (I’ve named it SimpleDataButtons) is a very simple one: it just has the few buttons sitting on it, and some properties were set just to make it look "fancy" (yeah, right…). The class implements IDataInterfaceController, and the implementation is pretty simple:

The SetInteractionsStateMachine takes in a reference to the state machine that this controller sends events to, and the properties (NewEnabled, LoadEnabled…) simply interact with the Enabled property on the controls.

The event handlers for the buttons look pretty much the same as we had on the customer edit form; it just sends events to the state machine:

Now all that’s left to do is to drop the SimpleDataButtons user control on the customer edit form (I’ve named the field simpleDataButtons), and add it to the array of IDataInterfaceControllers:

That’s it. Now when I run the form, I get both set of buttons, and they both work interchangeably (of course, in a real app, I’d have either one or the other…).

Adding shortcuts to trigger the events

Another interface controller that I wanted to add is one that handles keyboard shortcuts. Essentially, I want to access the New, Load, and Save functionality by using Ctrl+N, Ctrl+L, and Ctrl+S, respectively. With the current architecture, this is pretty easy. The "trickiest" part is probably figuring out how to actually handle the keyboard shortcuts. In fact, not too long ago I’ve posted a little design and implementation to do just that (make sure to check out that post!).

For the scope of this post, all you have to know is that I’ve created a DataInterfaceKeyboardHandler (which handles trapping the keystrokes and executing some action depending on the shortcut mapping), and this class implements the IDataInterfaceController interface. Besides receiving a reference to the state machine that the controller sends events to, the implementation of the NewEnabled, LoadEnabled, and SaveEnabled properties essentially does the following:

• The getter returns true or false, depending on whether or not the specific shortcut is registered with in the list of shortcuts;
• The setter, registers or un-registers the specific shortcut, depending on whether a true or false is passed to it.

Again, all the shortcuts do is to send the appropriate events to the state machine. Once again, make sure to check out this post to understand how this DataInterfaceKeyboardHandler class handlers registering, un-registering, and executing keyboard shortcuts.

At this point, it’s only a matter of adding an instance of the keyboard handler to the customer edit form, and add it to the array of IDataInterfaceControllers:

If I run the application now, I can either use the buttons or the shortcuts to access the functionality.

Some improvements

One quick improvement to this design and implementation would certainly be a more generic way to register interface controllers with the data entry forms, but I’ll leave that as an exercise to the reader.

Another improvement would be to add functionality to the State Machine Toolkit’s code generator so that, besides generating the state machine, it could also generate the interfaces for the both the interaction and data interface controllers, as well as a default implementation for the state machine (delegating actions to the interaction controller).

I’m going to also, definitely, clean up names so to make the intention of things clearer, and clean up the design as much as possible.

I also want to play with Windows Workflow Foundation (WF), since it has the option to create state machines. Ideally, it’d be good if I could swap what state machine engines I’m using, without having to change anything else in the application.

Summing up

Bringing this thing to the point where I have it right now has been really fun. It allowed me to play with and exercise a lot of cool things, such as state machines, delegates, anonymous types, commands, shortcuts, interfaces, controllers, etc.

I’ll keep working on this thing as I move some of this knowledge into some existing applications, and will certainly post anything else I may that could be interesting for somebody else (or may just as reminder to myself…).

Downloading the source code

Since Microsoft has recently released the MSDN Code Gallery, I’m posting the source code for these posts over there. You can get to it here: GUIInteractions.