Live Distributed Objects

Implementing Objects in C#

Object Libraries
Creating and Deploying a Skeleton of a Live Object Library
Objects
Creating a Simple User Interface Object Backed by a Communication Channel
Creating a Simple Object with a Custom Interface and with Custom Communication Channel Types
Creating a Simple Mash-Up Object That Contains Other Objects and Uses Reflection

Object Libraries

Creating and Deploying a Skeleton of a Live Object Library

Before you read the tutorial, you may watch this short video that illustrates the sequence of actions we will walk you through.



As mentioned before, live objects and types used in their definitions can be defined by the user in .NET code. This is achieved by annotating ordinary class and interface definitions with special attributes, which are parsed by the live objects runtime, and deploying the compiled code in a dedicated folder. When during parsing an XML description of an object, a reference to an object or type is found that is not a type known to the runtime, the runtime looks for a library that defines this object or type and tries to dynamically load the code into the process to make the object definition or type available to the object loader. Below, we will walk through the process of defining and deploying a simple custom object library.

Start by launching Visual Studio and creating a class library project in C# (you can perfectly well use any other .NET language, such as VB, C++, J# etc., but since C# is, in some sense, the "model" .NET language, in this tutorial we will use examples in C#).

screenshot_036.gif

screenshot_037.gif

We will be using custom attributes and types defined by the runtime, so we need to add a reference to the part of the runtime where those are stored. Open the "References" tab and add a reference to the library "liveobjects_1.dll". You don't need to, and in fact should not add references to any other libraries (class names in other DLLs are randomly obfuscated, referencing any of them will make your code unusable). You will find the "liveobjects_1.dll" file in the "\bin" folder relative to the root of the distribution. If you haven't changed the default installation folder, the full path to the live objects library that you need to import will be "C:\liveobjects\bin\liveobjects_1.dll".

screenshot_038.gif

screenshot_039.gif

Now, we need to assign a unique namespace identifier to the library. All types and objects defined in this DLL will be placed in the specified namespace. Open the source file "AssemblyInfo.cs" hidden in the "Properties" folder in your project, and add a new attribute of type "QS.Fx.Reflection.Library", with a single string argument, as shown below.

screenshot_040.gif

The argument is a namespace identifier and version number, separated by a back-apostrophe "`". For a component to share with others, we will provide a way of generating a registering a namespace identifier through our website. For simple testing, you can generate the namespace identifier with the "Create GUID" tool in the "Tools" menu in Visual Studio. You need to use the "registry" format, and remove any characters besides the letters and digits from the identifier created this way. The sequence of actions you need to take to generate a GUID is shown below.

Select "Tools"...

screenshot_058.gif

Select "Create GUID"...

screenshot_059.gif

Click on "New GUID", and then on "Copy".

screenshot_060.gif

Paste the GUID into the editor and remove all ", ", and "-" characters from it. Then, append a back-apostrophe and a version number 1 to the GUID string, and pass the resulting string as an argument to the QS.Fx.Reflection.Library attribute constructor, as shown below.

[assembly: QS.Fx.Reflection.Library("76A5C94847B44F88933AE426369FAF92`1")]
Note that the back apostrophe character "`" we used (grave accent, Unicode 0060) is not the same as the regular single apostrophe "'". The "`" character is most likely located in the upper left corner of your keyboard, next to the character for number "1" . Don't get confused!

Now, we shall create a simple UI object. Create a new custom user control, as shown below.

screenshot_041.gif

Once the controls gets added to your project, switch to the code view.

screenshot_042.gif

Now, we will annotate the custom control code as an implementation of a live object. We do this by applying a custom attribute "QS.Fx.Reflection.ComponentClass". The first argument is the identifier and a version of the new component. Let's pick identifier 1 and version 1. The second and third arguments to the attribute are a short name and a longer description of the object that will be displayed e.g. in the object designer. The annotated class should look something like the following.

screenshot_043.gif

Now, we need to specify what type of object we are implementing. We do this by inheriting from an interface that defines an object type. In this example, we will use a predefined interface, defined by the built-in class "QS.Fx.Object.Classes.IUI", representing objects that have a 2-dimensional Windows Forms style UI and doesn't provide any programmatic API. Later on in the tutorial, we will demonstrate how to implement new types of objects. For now, inherit from the built-in type mentioned above. Also, make the class "sealed": inheritance is not permitted for live objects definitions anyway, and making the class sealed will allow for more compiler optimizations, so you should do this whenever appropriate.

screenshot_044.gif

Inheriting from "QS.Fx.Object.Classes.IUI" forces you to implement a property defined in this interface, you can automatically generate stub code by positioning the cursor at the beginning of the interface name, and choosing the option to "explicitly" implement the interface, as shown below.

screenshot_045.gif

You should see a stub similar to the one below generated in your source file.

screenshot_046.gif

As you may remember from, the way of interacting with live objects is to connect to their endpoints. The interface exposed by a live object is therefore simply a list of endpoints. The "UI" class defines a single endpoint, and the code generated above should return the endpoint that others will connect to. Endpoints are constructed by the endpoint factory class "QS.Fx.Endpoints.Internal.Create". To create an endpoint of the type matching the interface, we use the "ExportedUI" method. The construction should happen in the object's constructor, the created endpoint reference should be saved in a private field, and returned by the property getter, as shown below.

screenshot_047.gif

The factory method "ExportedUI" that creates the endpoint takes a single argument. This is a user interface endpoint, and we need to provide the UI that implements the user interface. Since the class Foo we created is the UI, it simply passes a reference to itself.

Right now, the object does nothing at all, but its definition is technically complete. Before we add more functionality, let's try to deploy and execute it. In general, we will use automated tools for this purpose, and we will discuss the necessary setup in another part of the tutorial. For simple debugging, it is best and easiest to deploy things manually, so this is what we'll do in this tutorial.

The runtime stores libraries in a subfolder "\libaries" of the root installation path. The folder contains a subfolder for each of the namespaces, the name of the subfolder being the namespace identifier. In the namespace folder, we have a separate subfolder for each version of the namespace. In the subfolder of a given version, there is a single file named "metadata.xml", which, as the name suggests, contains some metadata related to the namespace, and a subfolder named "data", which contains the actual binaries.

The distribution already comes with an example custom library of components deployed, so before deploying our custom library, let's look at the example. The library implements components in namespace "1A3160A2B3C64B589A9754161FD6B6EF", and the current version is "1", hence the relevant files can be found in "C:\liveobjects\libraries\1A3160A2B3C64B589A9754161FD6B6EF\1". The folder structure is as on the pictures below.

screenshot_048.gif

screenshot_049.gif

The metadata file is an XML file that has a single root element "<Library>". The element has an XML attribute "id" that identifies the namespace and the version of the namespace this library implements. It may seem redundant to include this information here, considering that it is already present in the folder structure, but this helps catch bugs or silly mistakes during library deployment. The library element also contains one or more "<Include>" elements with a "filename" attribute that points to specific binary files relative to the "data" subfolder. We assume that the contents "data" subfolder can be anything, some of the files stored there may be libraries unrelated to live objects, and some may be custom data or configuration settings, therefore we need to explicitly point to the files that the live objects runtime should load.

<Library id="1A3160A2B3C64B589A9754161FD6B6EF`1">
<Include filename="examples_1.dll" />
</Library>
To deploy our custom library manually, we need to create a directory structure and a metadata file such as those above, and copy the binaries over. The resulting directory structure should look like on the following pictures.

screenshot_050.gif

screenshot_051.gif

Note that we do not copy over any of the live objects DLLs, even "liveobjects_1.dll", to the "data" folder. When loading the library, the live objects runtime will hook things up as needed using its own DLLs, as installed on your machine in the "\bin" folder, and placing here any live objects DLLs could only cause problems.

The contents of the "metadata.xml" file should look like the following:

<Library id="76A5C94847B44F88933AE426369FAF92`1">
<Include filename="SomeStuff.dll" />
</Library>
In practice, copying things over manually is tedious, hence during debugging, you may want to do so in post-build steps. To do so, first include file "metadata.xml" with the content above in your project, create an empty file named "postbuild.bat" and also add it to your project.

A word of caution: don't create the batch file from under Visual Studio, because there appears to be a bug in Visual Studio causing text files created in it to start with a garbage Unicode string that isn't visible to a human eye, but will fool the shell and make your script unusable. You may then see error messages such as one below.

'set' is not recognized as an internal or external command
If you run into this situation, the only cure is to edit the file using some hexadecimal editor and surgically remove the offending characters at the beginning of the file. To avoid running into this annoying bug, create a text file in notepad, or from the Windows Explorer context menu, and include it in the project. To do this, first click on the "Show All Files" button below to display the full set of files in the project folder, including those that aren't part of the project (that's the button circled in red in the picture below).

screenshot_056.gif

Now, select the newly created batch file, and click on the "Include In Project" command in the context menu to add the existing file to your project.

screenshot_057.gif

Now that all files are in place, edit the "post-build" step of your project, so that the post-build script is invoked after a successful build. For your convenience, the command to invoke the build script that you need to type into the post-build step dialog is pasted below as a string you can conveniently copy.

"$(ProjectDir)postbuild.bat" "$(ProjectDir)" "$(TargetDir)"
The text typed into the post-build script dialog is shown on the picture below. To get to this dialog, right-click on a project in the tree view, select "Properties", and navigate to the "Build Events" tab in it.

screenshot_052.gif

If you need to customize this script, click "Edit Post-build...", and click on the "Macros" button. This will display a list of macros that encapsulate things like a path to your solution folder, the produced DLL or executable etc., which you can use in your script to make it portable. Note that in the script, we do not separate the name of the batch file from the macro with a backslash. If we did, the string would resolve to a path with a double-backslash, because the "$(ProjectDir)" macro already contains a backslash in it. Unfortunately, the shell isn't smart enough to ignore the redundant backslash, and it will fail to execute the script with a path formatted this way.

Now, in the "postbuild.bat" file, type in the following logic to automatically deploy your compiled code in the "libraries" folder after each successful build.

set namespace=76A5C94847B44F88933AE426369FAF92 
set version=1 
set root=C:\liveobjects 
if not exist %root%\libraries\%namespace% mkdir %root%\libraries\%namespace% 
if not exist %root%\libraries\%namespace%\%version% mkdir %root%\libraries\%namespace%\%version% 
del /f /s /q %root%\libraries\%namespace%\%version%\*.* 
mkdir %root%\libraries\%namespace%\%version%\data 
xcopy /y %2SomeStuff.dll %root%\libraries\%namespace%\%version%\data\ 
xcopy /y %2SomeStuff.pdb %root%\libraries\%namespace%\%version%\data\ 
xcopy /y %1metadata.xml %root%\libraries\%namespace%\%version%\

If you rebuild the project now, you should see the files being automatically deployed. If you're getting error messages, most likely they are related to spaces in file or folder names, and you might need to put some parts of the script in quotation marks.

To test the newly created object, create a new file with the ".liveobject" extension e.g. in notepad, and put the following contents into the file.

<?xml version="1.0" encoding="utf-8" ?>
<Root xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema">
<Object xsi:type="ReferenceObject" id="76A5C94847B44F88933AE426369FAF92`1:1`1" />
</Root>
screenshot_062.gif

screenshot_063.gif

screenshot_064.gif

The simple live object reference we created tells the runtime to locate version 1 or higher of namespace 76A5C94847B44F88933AE426369FAF92, load the component library, and find version 1 or higher of object 1 in it. After running the file, you should see your user interface object being displayed, as shown below.

screenshot_053.gif


Objects

Creating a Simple User Interface Object Backed by a Communication Channel

Before you read the tutorial, you may watch this short video that illustrates the sequence of actions we will walk you through.



In this section, we will extend the skeleton library created earlier, and implement an object that uses a communication channel to persist its data. Recall that the goal of our technology is to enable building applications that consist entirely of live objects. Hence, the communication channel will also be a live object. Our new UI object should work with any communication channel, and it should not make any assumptions about the underlying channel implementation.

The easiest way to achieve this is to make our UI object a template that will take a channel object as an argument. We have already seen examples of such parameterized objects in the earlier parts of the tutorial, when discussing the internal structure of XML files and examples in the object designer.

To add a parameter that is a reference to another object, we simply add a parameter annotated with .NET attribute "QS.Fx.Reflection.Parameter" to the constructor. The attribute has two arguments, the first being the name of the parameter, and the second being the category. On object reference is not a type, but an actual value, hence we apply category "Value". The .NET data type of the references is a template type QS.Fx.Object.Reference<ObjectClass>, where ObjectClass is a .NET class that serves as an alias for a class of live objects. We won't discuss the concept of the alias here, the reader may find more information in the paper on live objects. For the purposes of this tutorial, we will just resort to examples, hopefully after a few of those the idea will become intuitively clear.

A correct constructor declaration for an object with a single parameter that is a reference to another object looks as follows.

screenshot_054.gif

For the purposes of verification, the complete source file is shown below.

screenshot_065.gif

You can actually recompile the project again, and after dragging the ".liveobject" file into the designer, you will see that the presence of a parameter has been recognized, as shown below. Since we have not actually assigned anything to the parameter (the XML file does not have any "<Parameter>" elements), the parameter is assumed to be null, i.e. a null reference is used.

screenshot_055.gif

Although the designer accepted the object, the object actually won't run. The designer is generally more forgiving, and fixes some problems for you. If you drag your object from the designer out onto the desktop, and reopen it, you will see that the designer has added some missing code to the XML file, explicitly passing a null reference to the parameter. The runtime requires of a well-formed ".liveobject" file to have all parameters always assigned, even if the parameters are null. The designer also decorated the definition with some attributes, such as name or description, extracted from your C# code.

<?xml version="1.0" encoding="utf-16"?>
<Root xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema">
<Object xsi:type="ReferenceObject" id="76A5C94847B44F88933AE426369FAF92`1:1`1">
<Attribute id="9F4C608A9A6D44FFAD8A2FDC662E185B" value="Foo" />
<Attribute id="7EC27D89CBD410DADE60463E4D46985" value="This is my custom object Foo." />
<Attribute id="9F64B7ABB95245F28596BFFD1F549558" value="1" />
<ObjectClass id="0`1:5211621A78F4CFBB993FA43186A5403`0" />
<Parameter id="channel">
<Value xsi:type="ReferenceObject" />
</Parameter>
</Object>
</Root>
After this fix, the object is usable, but again doesn't do anything. Furthermore, the type of the parameter being just any object reference won't do. Right now, we can pas any type of object as a value of the parameter "channel", for example a 3D airplane object, and as shown below, the object will type check correctly, and in fact it will run as well. The newly created UI object will receive a correct reference, it simply won't be able to do much with it.

screenshot_067.gif

To restrict the type of the parameter, we need to point to a more specific object class. Instead of the predefined .NET class "QS.Fx.Object.Classes.IObject", which served as an alias for a generic class of objects that all other object classes are a subtype of, we will use a .NET class "QS.Fx.Object.Classes.ICheckpointedCommunicationChannel", which serves as an alias for a class of objects that provide a single communication channel endpoint with state transfer support. The type of the parameter "channel" of the constructor needs to change as shown below.

screenshot_066.gif

The above definition deserves a comment. The class of channel objects we use here is actually a template, parameterized by the type of values that will be sent in messages, and the type of values contained in the checkpoints. In order to use this class here, we need to assign values to both of those parameters. Here, we use the predefined class "QS.Fx.Value.Classes.IText", which represents a simple string. The reason why we don't just use System.String is because the particular implementation of a multicast channel we use here requires value classed that support the type of serialization used by live objects. We will discuss the serialization-related topics in more detail later on.

If you recompile and redeploy the library with the change highlighted above, passing an airplane to the "channel" parameter will no longer type check, as shown below. Note also how the type of the parameter has changed (compare this with the preceding screenshot). Previously, "channel" was supposed to be a value of type "ObjectReference", where the class of the referenced object was simply the generic "Object" class. Now, as you see in the picture below, the class of the referenced object is based on the "Checkpointed communication channel" template, parameterized with the "Text" value class. Hence, the parameter requires channels that can carry textual messages and transfer textual state.

screenshot_068.gif

For quick initial testing of the new object, we can use the "LocalCommunicationChannel" object that provides empty initial state and simply bounces all messages back. We actually went through the setup process in an earlier part of the tutorial, so instead of repeating it all here, here's the desired outcome. You need to drag and drop the "LocalCommunicationChannel" and value type "Text" so that you end up with a correct live object reference that looks like below.

screenshot_069.gif

Now, select the object in the tree view and drag it onto the desktop.

screenshot_070.gif

screenshot_071.gif

Again, you can run the object. It still doesn't do anything, but now it does get initialized with a correct reference to a working communication channel.

Now it's time to use the channel. Go back to the code, and type in code that will instantiate a channel proxy from the channel reference by invoking the getter of the "Object" property of the reference, as shown below. First, you create a new private field that will keep a reference to the proxy, like this.

screenshot_074.gif

Now, you dereference the channel reference in the constructor to create the proxy.

screenshot_073.gif

The relevant piece of the code that changes now looks like this.

screenshot_072.gif

Now that we have the proxy, we need to ask it to provide its communication endpoint, and connect to it. In order to connect to the channel's endpoint, we will have to have our own matching endpoint, so we need to create it here.

Go to the designer window, find the channel object and its endpoint, browse down to the "EndpointClass" element, and choose "Generate Code for the Matching Endpoint" from the context menu. This is going to place in Clipboard a piece of C# code representing the alias type for the endpoint we need to generate to connect to the channel, as shown below.

screenshot_075.gif

Now, paste from clipboard into your code, to create a private field with this type, as shown below. The part being pasted is selected.

screenshot_076.gif

Then, write code in the constructor that will construct this endpoint. To get the constructor method, simply produce a copy of the text above, replacing type name "QS.Fx.Endpoint.Internal.IDualInterface" with method name "QS.Fx.Endpoint.Internal.Create.DualInterface", while leaving the template parameters as they are, so that you get the following code in the constructor.

screenshot_077.gif

Note we have passed a single argument to method "Create". To construct and endpoint from template "dual interface", we need to pass as argument some object that implements interface "QS.Fx.Interface.Classes.ICheckpointedCommunicationChannelClient". This interface provides handlers for events that might arrive through the endpoint. The easiest thing to do at this point is to let the class you're defining implement those event handlers, so we pass "this", and extend the class definition to implement the interface with incoming event handlers, as shown below.

screenshot_078.gif

To make the code compile again, we just need to implement the event handlers. Implement them explicitly, as we explained earlier, by positioning the cursor at the beginning of the type name and choosing option to "implement explicitly" from the menu that pops up. The complete code with generated stubs should look like the following.

screenshot_079.gif

You can verify that the code now compiles and runs. While still nothing is happening, a proxy of the embedded channel objects is being instantiated, and an internal endpoint is being created. So, let's now connect the endpoint we created to the internal channel object and add some event handlers.

To establish a connection between objects, we invoke the "Connect" method of an endpoint. The method returns an object of type "QS.Fx.Endpoint.IConnection", which represents a reference to the connection, and controls the connection's lifetime. We store the connection reference in a private variable. The variable we need to declare is shown below.

screenshot_080.gif

The code we need to add at the end of the constructor to establish the connection is shown below.

screenshot_081.gif

Now, let's look at the event handlers. As a client of the checkpointed communication channel, we need to implement three incoming vents. Event "Initialize" is the first one we will receive, and carries a checkpointed state object that we can use to initialize this proxy. Event "Receive" carries messages multicast on the channel, which represent updates. The checkpointed state is synchronized with the updates that follow, and represent the version of the replicated state exactly before the next update that follows it, hence if we use the checkpoint to initialize our proxy, and then process updates received from the channel one after another, the proxy will stay synchronized with the other proxies. Finally, event "Checkpoint" is a request to generate a checkpoint of the current state. This is used to initialize other proxies that have just recently joined.

First, to visualize the processing that is going to take place, let's extend the UI with a text box that will display the object's state, and a button that will generate updates. By double-clicking on "Foo.cs" in the tree view (or choosing "Designer" from the "View" menu), we shift to the design view of the UI object. Add a button and a multi-line text box, as shown below.

screenshot_082.gif

Now, in the body of the "Initialize" method, let us add code that will replace the contents of the text box with the textual data received from the channel, as shown below. Note that we must handle the case when the checkpoint is simply a null object. We do that by initializing the text box with an empty string.

screenshot_083.gif

Similarly, to generate checkpoints that will be used to initialize other proxies, we do the reverse, i.e. we produce a checkpoint using the textual data from the text box. To create a serializable object of type "QS.Fx.Value.Classes.IText", we may use class "QS.Fx.Value.UnicodeText", as shown below.

screenshot_084.gif

Finally, we process an update by appending the received string to the contents of the current window.

screenshot_085.gif

We have handled all input events, but if we run the object, nothing would happen because nobody is generating updates. So, switch to the design view, and double-click on the button to associate the click even with code and automatically generate an appropriate stub. You should see code similar to the one below being added to your source file.

screenshot_086.gif

In the body of the code, create some random string, and publish the string on the channel by calling method "Send" available through the "Interface" property of the channel endpoint, as shown below.

screenshot_087.gif

Now, if you rebuild and start your object, you will see messages being posted after button is clicked, as shown below.

screenshot_088.gif

Note that we do not simply update the UI in response to events. Because the object is distributed, any changes to its state must be synchronized between all proxies running it. Whenever we need to make an update, we publish a request on the communication channel. Only after the request has been ordered relative to other requests and delivered by the communication channel to all proxies of the object can the proxies act upon the request. This ensures that proxies are always synchronized.

Although the object now correctly handles all events and runs with the simple channel that bounces messages back, it won't run in a real multicast scenario, for the updates to the UI controls are done from the context of the networking threads, not from the context of the UI thread. This is a limitation of Windows Forms that you will frequently run into when coding any UI components. You can read about the underlying problem in the MSDN documentation, in the description accompanying the class "System.Windows.Forms.Control.BeginInvoke".

A quick and dirty solution to the problem would be to simply use "BeginInvoke" to process the incoming message, such as below.

screenshot_089.gif

While this might work for extremely simple objects, it is not really a correct solution for a number of reasons, such as e.g. that it may result in updates being applied out of order. A much better technique is to place the received updates on a queue, and use BeginUpdate to schedule a callback that will process those updates asynchronously.

First, we need to define the structure that will represent on the queue, and declare a private variable that will represent the queue, as shown below.

screenshot_090.gif

Now, we modify the code of the "Initialize" and "Receive" handlers to simply place an appropriate event on the queue, and schedule asynchronous processing through BeginInvoke. To avoid redundant work, we don't schedule a callback if the queue is not empty, for this means that a callback has already been schedule. In the callback, we will simply process data in bulk. The updates bodies of the two methods are shown below.

screenshot_091.gif

The body of the method scheduling the callback, invoked from both methods shown above, looks like the following.

screenshot_092.gif

Finally, the body of the callback that processes the updates would look as shown below.

screenshot_093.gif

You can check that the project compiles again, and the object works just as before. This time, however, we are ready to replace the underlying mock communication channel with one that works across multiple processes. To keep things simple, we will reuse here one of the communication channels pre-configured in the distribution. Launch the object designer and drag file "shared_text_1.liveobject" from the "examples" folder into the designer window, then locate object passed as parameter "channel" to the root UI object, as shown below.

screenshot_094.gif

Now, select the repository object representing the channel, and drag it out of the designer and onto the desktop, as shown below.

screenshot_095.gif

screenshot_096.gif

You may check if the object was exported correctly by dragging its ".liveobject" file from the desktop back to the designer. You should see something like this.

screenshot_097.gif

Now, drag the original ".liveobject" file referencing your new UI component into the designer, and drag the channel you just exported onto the "channel" parameter. The object should type check correctly, and you should see a structure similar to the one shown below.

screenshot_098.gif

Now, select the root object in the tree view and drag it out onto the desktop. This produces a new ".liveobject" file with the updated reference. Then, make sure that the "Live Distributed Objects" service that controls multicast channels is running. If it is not, start it now by "Services" tool in "Administrative Tools" in "Start Menu", or by typing in the command "net start "QuickSilver Live Objects"" in command prompt. FInally, launch your object again, by double-clicking the updated ".liveobject" you just created on the desktop by dragging the updated object definition from the designer. Launch the object at least twice, so you have multiple clients accessing it. Whenever you click the button in any of the clients, updates should be multicast and applied consistently everywhere, with the as shown below.

screenshot_099.gif

For reference, compare your code with the complete solution of the module implemented in this section included with this documentation.


Creating a Simple Object with a Custom Interface and with Custom Communication Channel Types

The example object in the preceding section used predefined type "Text" for both update messages and state that is transferred between replicas. While there are performance advantages to using built-in types, most objects will require custom data structures. Our platform allows the user to define and use new types of messages, by defining a new .NET class and annotating it with a "ValueClass" attribute, much in the way we annotated a custom component in the preceding example. In this section, we will illustrate this with a distributed banking example.

An example type that will represent a financial transaction is shown below.

screenshot_100.gif

Note the use of "XmlAttribute" in this example (defined in namespace "System.Xml.Serialization"). This attribute marks the fields of this class as serializable. Annotated this way, the "Transaction" class can be serialized using the built-in .NET serializer. In order to be able to send messages across the network, our platform requires that these messages are serializable using either one of the techniques built into .NET, binary serialization or XML serialization, or using QuickSilver's custom internal serialization scheme. The choice of the technique will depend on the communication channel used. In this example, we will use XML serialization because it is much easier to work with than the QuickSilver's internal scheme, and it is much easier to debug than the built-in .NET binary serialization (the latter has the highly annoying property that recompiling your code breaks binary compatibility). The built-in XML serialization is slow and has high overhead, but the performance of our example object isn't critical. For uses such as multimedia streams, using live object built-in types or QuickSilver's serialization scheme is recommended (these will be described in another part of the tutorial).

We will assume here that you are familiar with XML serialization. If not, you might want to visit http://msdn2.microsoft.com/en-us/library/182eeyhh.aspx.

In our banking example, the "Transaction" class will represent transactions, i.e. operations made against the bank, and in the implementation of our bank object, which again will rely on a multicast stream to carry state and updates between bank replicas, the type "Transaction" defined above will be assigned to the "MessageClass" parameter. The type of checkpoints representing the bank's state at a point in time (the value of the parameter "CheckpointClass") is defined as a separate class "Accounts", containing a list of account statuses stored in yet another custom type "Account", as shown below. Note that each of these types is separately annotated with the "ValueClass" attribute, each has a different identifier, and each contains a default no-parameter constructor and annotations required for correct XML serialization.

screenshot_101.gif

screenshot_102.gif

Now, if you compile and deploy your library as explained in the preceding examples, and expand the "value classes" node in the right-hand pane in the object designer, you can see your new value classes listed, as shown below.

screenshot_104.gif

With the value classes in place, we can now create a communication channel that will be used by our new object. Now, unlike in the previous example, we can't directly use any of the examples channels that have been preconfigured in the distribution, not just because none of them is of the right type, but also because the built-in communication system supports only channels that carry messages serializable using QuickSilver's built-in serialization scheme. In this example, we want to use messages serialized as XML, we need a "wrapper" channel that will take care of the translation between XML-serializable messages generated by our custom object, and messages in the format that our communication system can support.

We can accomplish this with the built-in object "XmlChannel". In the object designer window, navigate to this object in the "objects" section in the right-hand pane, and drag it onto the design sheet. Then, drag your newly created value class "Transaction" from the "value classes" section onto the "MessageClass" parameter, and drag the newly created value class "Accounts" onto the "CheckpointClass" parameter. The object should now type-check correctly, and you should see a result similar to the one below.

screenshot_105.gif

We're not ready just yet. The XML channel object does only translation to and from XML, but doesn't actually perform any network communication. For the latter, it needs a communication channel that will be used internally to carry messages serialized to XML. It expects a generic type of a channel, in which both message and checkpoint type are of the base type "ISerializable".

Right now, we don't have a channel of the required type, so we're going to create one. To illustrate the process, we're not going to rely on any fancy tool, we'll actually do it manually. We're going to make things somewhat simpler by using one of the example channels as a template. To get started, fire object "channels.liveobject" from the "examples" folder, and drag any of the channels onto the "underlyingchannel" parameter of the "Xml Channel" object. Note that "channels.liveobject" will fail if the "Live Distributed Objects" service is not running. You should see the result similar to one below.

screenshot_106.gif

The object doesn't type-check correctly because the type of the underlying channel does not match. To fix it, find the "ISerializable" value class in the right-hand pane, and drag in onto the parameters of the underlying channel "Text_1". This time, the entire object should type-check correctly. You can now grab the entire object and drag it onto the desktop.

Although the channel object typechecks correctly, it is not usable, because it has a wrong identifier: the channel with the identifier we used already exists, and has a different type. If you tried to use this channel in any context, the runtime would throw and exception. At the very least, we need to change the channel identifier. At the time of writing this tutorial, the object designer didn't support renaming objects just yet, so we will do it by hand. If you open the newly generated ".liveobject" reference in an XML editor, you will see in it a description of the underlying channel that starts out as follows.

screenshot_107.gif

To fix the definition, replace the value of the "id" attribute of the the top "ReferenceObject" element with an unused channel identifier (consult the list of channels displayed by the "channels" object if not sure). While at it, you might also update the channel name and description encoded in custom attributes in the two lines right below it.

screenshot_108.gif

Reload the object in the designer to make sure that you didn't make a typo while editing the XML, you should see the updated channel identifier, name and description, as shown below.

screenshot_109.gif

By now, we have a complete, well-formed definition of a channel object that can be used in our banking system. However, it won't work because the communication system doesn't know about the underlying channel "9999999999999999999", we need to configure the channel on the channel server.

All communication substrates built into the live objects platform maintain the list of channels and channel settings in configuration files in the "channels" subfolder relative to the root installation folder of the distribution. Channel definitions are stored only on a selected "controller" node, which managed the network. For each channel, there is a separate folder named after the channel identifier, containing a single file "metadata.xml" with the channel's configuration settings. To keep things simple, all communication substrates use the same files, so once configured, channels can be easily reused between all the different built-in configuration platforms. In addition to the metadata, depending on the communication substrate used, the channel folder might contain additional data, such as checkpoints saved on the server etc. The simple default communication substrate used in this tutorial only needs the metadata file. The file is formatted in XML, with a single top "<Metadata>" element that contains elements "<ID>", "<Name>", and "<Comment>" with the channel's identifier, optional name and optional description, and elements"<MessageClass>" and "<CheckpointClass>" storing the type definitions of messages and checkpoints exchanged on this channel. An example channel configuration file is shown below.

screenshot_103.gif

To configure a new channel on the server, we simply create a new folder, with a new "metadata.xml" describing the channel. Since in this tutorial, we are only going to communicate within a single machine, and the channel server resides on the same machine on which we're doing the development (this is by far the most convenient scenario for debugging simple objects), we're going to create the folder on the same machine on which we're running the objects. For the same reason, each new installation of live objects automatically deploys a number of example channels on each machine. Although the ultimate goal is to share objects across the network, for simple debugging and demonstration purposes, one will often use a single machine, hence the preconfigured channel configuration files created locally by the setup. In real deployments, only one machine would be used as a channel "controller" that manages communication for an entire network. In those scenarios, only the channel definitions on the channel controller would be used. The definitions deployed on all other machines would be ignored, and could just as well be deleted.

For the ease of debugging our example, we will include the configuration files for a single channel in the solution, and add a post-build step that automatically deploys these definitions on the local machine. We start off by adding a channel folder and the new metadata file to the project, as shown below.

screenshot_110.gif

Note the 128-bit values of "MessageClass" and "CheckpointClass", we copy those values over from our XML channel definition (they represent the built-in generic "ISerializable" type mentioned earlier).

Now, we extend the post-build step with the additional copy command to locally deploy the channel definition:

screenshot_111.gif

You should now be able to recompile the project, and see the channel definition deployed in the "channels" folder. If you start the "QuickSilver Live Objects" service and fire up "channels.liveobject" from the "examples" folder, you should see your new channel listed, as shown below. Note that if the service is already running, it may not pick up the new channel definition immediately (the service is scanning the folder periodically, but it does so only every once in a while), and you may need to restart it to make sure the channel is immediately visible to the clients.

screenshot_112.gif

Note: if you are using an older version of the system, the service may have a bug that will cause it not to notice that you deployed new channels until you restart it.

As an exercise, you may actually drag this channel into the "underlyingchannel" parameter of your Xml channel object, to see that it is correctly served by the communication system, and has a correctly configured type. The object should type-check correctly, and you should also see your updated object name and description retrieved from the channel metadata in the "metadata.xml" file, as shown below.

screenshot_113.gif

Now, we're going to define the bank object that uses this channel to coordinate its replicas. Rather than bundling "business logic" with the UI code, as in the previous example, we're going to separate the two, and implement the "business logic" as a separate object that is passed to the UI as a parameter.

We start off by defining a new object type to represent the bank. The object will expose one bi-directional endpoint. We define a class of such objects by creating a new .NET interface derived from the generic object class "QS.Fx.Object.Classes.IObject", annotating it with "QS.Fx.Reflection.ObjectClass" attribute, and defining a number of properties that return endpoints of the desired types, each decorated with an "QS.Fx.Reflection.Endpoint" attribute, as shown below.

screenshot_114.gif

The above definition states that from now on, the .NET interface "IBank" defined in this class library will represent a live object class with identifier "1", version "1" and a user-friendly name "Bank", and that live objects of this class have a single endpoint named "Bank", which is a bi-directional event channel, with the types of events determined by the two .NET interfaces "IOutgoing" and "IIncoming".

The definitions of .NET interface "IIncoming" looks as follows. We declare five types of message exchanges named "Ready", "Account", "Transfer", "Deposit", and "Withdraw". The first will be used by the bank client to test whether the bank replica is fully initialized and ready to respond to requests. The second is used to retrieve the list of bank accounts. The last three are used to initiate transactions.

screenshot_115.gif

In the live objects nomenclature, this interface would define both input and output events, the input events corresponding to each of the method calls, and the output events corresponding to the messages carrying the results. In the most "orthodox" implementation of the live objects model, each interface would only describe one set of events, and communication would be purely asynchronous. In this prototype, we opted for a solution that developers are comfortable with, and allow interfaces to contain declarations of methods that synchronously return values to the caller. The implicit "rule of conduct" we adopt is that all event handlers should terminate promptly, and ideally, never block, lest the system might block. In future versions of the platform, we will add mechanisms to help enforce this by preempting long-running objects, hosting different subsets of objects in different threads or different application domains.

The definition of the outgoing events interface "IOutgoing" follows a similar pattern. We define two events, "Ready" to notify the user of the bank object that the bank replica is fully initialized and ready to accept requests, and another to alert of changes to account balances.

screenshot_116.gif

Having defined the bank's object class, we now define the actual object. The structure of the code should look as below. As usual, the class is annotated with the "QS.Fx.Reflection.ComponentClass" attribute to make it recognizable to the live objects runtime. Since this is not a UI object any more, we don't derive from the "QS.Fx.Object.Classes.IUI" interface. Instead, we derive from the newly created "IBank" interface. Since this object will have two endpoints, one to expose to other objects and one to use internally to communicate with an embedded channel object, we also implement two event handler interfaces, "IIncoming" for the external endpoint that is to be exposed, and "ICheckpointedCommunicationChannelClient" for the internal endpoint that is to be connected to the channel. The class takes the channel reference as a parameter to the constructor, much in the same way as it was in the previous example. We see a plumbing code in the constructor that by now should have a familiar structure. Finally, in addition to the two endpoints and the internal endpoint connection, we have one data field "myaccounts" to maintain the bank's state, and one boolean field "ready" to indicate whether the bank's state has been initialized.

screenshot_117.gif

The implementation of the "IBank" interface is straightforward. As with other interfaces that are object class aliases, we simply return the requested endpoints.

screenshot_118.gif

The implementation of the bank's incoming events is also straightforward: we either return the locally cached information, or use the embedded channel to multicast a transaction to be performed against the bank's replicated state.

screenshot_119.gif

Finally, we implement the event handlers for channel communications. In the initialization event, we move data from the checkpoint to the internal data structure "myaccounts", mark the bank replica as ready to process requests, and if there is some other object connected to the bank replica, we notify the object that the bank is ready by sending event "Ready" through the bank endpoint.

screenshot_120.gif

When asked to produce a checkpoint, we use the same code as in the bank interface above.

screenshot_121.gif

The implementation of the update event is a bit more complex, but that's just because we chose to cram more complex logic into a single message: the bank performs deposit, withdrawal, or transfer of funds depending on whether the "from" or "to" fields are null or empty, and creates new accounts as needed. We also iterate over all accounts that changed and issue an outgoing event through the bank interface to alert the connected object to the change.

screenshot_122.gif

With the bank logic embedded in the bank object, we can now write a host of different frontends to perform different bank-related tasks. We will implement two frontends, the first will be a report that just shows the list of accounts and balances, and the second will have an interface through which one can issue transactions. By now, the structure of the code should be self-explanatory, so we will just print the actual code. The overall code structure of the report object looks as below.

screenshot_123.gif

In response to any notifications from the bank, we simply refresh the entire view.

screenshot_124.gif

The actual code that does the refresh simply retrieves all accounts from the bank object and populates a list view.

screenshot_125.gif

This is it. The code of the second object is even simpler. The overall structure is as shown below.

screenshot_126.gif

The event handlers do nothing and simply ignore the incoming events, and pressing a button results in a new ooutgoing event passed to the bank object.

screenshot_127.gif

To test the project, strart up atleast one copy of the control and one copy of the report objects. To create a new account, leave the "from" field empty, put some name in the "to" field, and some amount in the "amount" field. This will create a new account with the amount requested. Every time you transfer money into an acocunt that doesn't exist, one is created. The source of account can be either another account, or if none is specified, money is created out of the thin air.

For reference, compare your code with a complete solution of the example implemented in this section.

screenshot_128.gif

This concludes our example. Of course, the example is rather naive, and we wouldn't implement a real banking service this way. If nothing else, we wouldn't fully replicate and transfer bank state across all clients, and instead leave it to a core set of replicas, perhaps one per site. We would not want to retrieve the full list of accounts in each report, and we wouldn't want to issue notifications for all account changes. We would also need some level security, and a fault-tolerance component that ensures there is always at least a few working replicas maintaining the bank's state (in this example, all the accounts are erased at the second the last bank object replica disappears, so we would need to make sure that a core set of such replicas is always running). A more sophisticated implementation would take these factors into account. We leave it up to the user to come up with the solutions (if you do, and would like to add some of your ideas to this tutorial, please let us know).


Creating a Simple Mash-Up Object That Contains Other Objects and Uses Reflection

The third example will illustrate the process of loading other objects and reflection mechanisms that can be used to inspect their types. We will create a simple shared object container that allows the user to drag and drop visual objects inside. Example usage is shown below. Here, two shared text objects have been dragged inside, and are displayed one after another.

screenshot_129.gif

As before, we start by defining the types of messages and state checkpoints. The distributed state in our object will consist of a list of serialized XML object definitions, and so the checkpoint will be simply a list of strings. The checkpoint would look as shown below.

screenshot_130.gif

The contents of the update class would in general depend on the type of update being applied, so one of the fields of the update message will be the operation code. In this example, we will implement only one operation, adding new objects to the mash-up. As before, we will simply include the serialized XML definition of the object being added in the update message.

screenshot_131.gif

As before, we will be using the "XmlChannel" wrapper to carry objects, so we need another binary channel, as shown below.

screenshot_132.gif

The header of the new object class will need to include the usual annotation with the "ComponentClass" attribute. We will be coding a UI object, so we will inherit from the "QS.Fx.Object.Classes.IUI" interface to inform the runtime that this is a UI object. Finally, we will be using an embedded communication channel, and this class will implement all the event handlers, so we inherit from channel client event handler interface "QS.Fx.Interface.Classes.ICheckpointedCommunicationChannelClient" parameterized by the types of checkpoints and messages we will exchange.

screenshot_133.gif

The constructor will have two parameters. Apart from the communication channel, we will also need some object that will help us de-serialize the XML definitions, and produce usable object references. This will allow us to dynamically instantiate proxies to the embedded objects.

screenshot_134.gif

In the body of the constructor, we simply create all the necessary endpoints, one to the embedded channel, and another to the embedded loader object, and we establish connections.

screenshot_135.gif

To support the code above, we will need a few extra fields shown below.

screenshot_136.gif

As it is with any type of object, we need to return on demand endpoints that other objects can use, so we have the usual property getter that returns the external endpoint we created in the constructor.

screenshot_140.gif

Now, we move on the the running state that we will maintain. First, we will keep a list of the serialized XML definitions of all embedded objects. Keeping them serialized is convenient when we need to provide a checkpoint for other objects.

screenshot_137.gif

Returning a checkpoint is straightforward, we simply wrap the list of serialized definitions in the checkpoint class.

screenshot_141.gif

Initialization will look similar, but we will need a bit of extra code. When we initialize the local replica, we not only need to store the object definitions, but we also need to actually load and display the embedded objects. Since this involves UI code, we need to do so asynchronously. To this purpose, aside from storing the definitions on the list, we also store new definitions on a queue. These queued definitions will need to be "processed" later.

screenshot_138.gif

Initialization code would then look as below. Note the call to the "LoadEmbeddedObject" method. This method will asynchronously process the queued definitions.

screenshot_142.gif

Processing an update looks very similar, we just store a single definitions rather than multiple definitions at once.

screenshot_143.gif

The asynchronous processing method "LoadEmbeddedObject" switches to the UI context, and then simply processes the queue, calling another helper method on each of the XML strings one by one.

screenshot_144.gif

The second helper method looks as follows. This one requires a bit more explanation. First, we use the embedded loader to translate a serialized XML specification into an object reference. Next, we access the "ObjectClass" field of the reference to inspect the class of the referenced object. We use the "IsSubtypeOf" method to test whether the object is a UI object. We pass as an argument a reference to the class of UI objects that we saved earlier (the relevant code is also given below). If the test is successful, we cast the generic object reference to a UI object reference. Having obtained a UI reference, we can dereference it to produce a proxy to the actual UI object. Subsequently, we create an endpoint that we will use to connect to this proxy, we dynamically establish the connection, and store the connection reference on the list of connections (if we didn't store it, it would be garbage collected and disconnected).

screenshot_145.gif

To support this code, we need another local variable to store the connections. We will also need to get a reference to the class of UI objects that the above code relies upon. We do that by calling generic method "ObjectClass" of "QS.Fx.Reflection.Library.Create", passing the alias type of the class of UI objects as a generic parameter.

screenshot_139.gif

By now, we have all we need to be able to display and exchange the replicated state, but we don't have any code that would modify it. We make the container accept drag and drop events (you need to set the "AllowDrop" property on the visual elements), and implement the "DragEnter" and "DragDrop" methods (you need to create associations from the UI, consult the MSDN documentation or look at the zipped example if you don't know how). In this example, we will only accept objects dragged as text, so the "DragEnter" method will look as follows.

screenshot_146.gif

It is straightforward to modify it to also accept drag and drop of files, and we leave this as an exercise for you. Now, the code that handles the actual drop operation should look familiar. We simply access the dragged text, wrap it as an update, and push along the channel.

screenshot_147.gif

That's it. Our object is complete. To test it, we just need a ".liveobject" reference file. The process of creating it is similar to what we did in the previous examples, so we won't go over it step by step. You can compare your code with the complete definitions shown below.

First, the visual representation of the object reference.

screenshot_152.gif

And here's the left-pane tree view listing showing all relevant types and parameter values.

screenshot_153.gif

The structure of the underlying XML file looks as below.

screenshot_148.gif

The "channel" parameter passed as an argument to your object looks like below.

screenshot_149.gif

The "underlyingchannel" parameter of the XmlChanel object is given below.

screenshot_150.gif

Finally, here's how the "loader" parameter passed to your object would look.

screenshot_151.gif

For reference, compare your code with a complete solution of the example implemented in this section.


Last edited Oct 7, 2010 at 7:17 PM by mikeryan, version 9

Comments

No comments yet.