Domain-Specific Languages

Boo

Half the work I’m currently doing here at Video Stream Networks is to design a DSL runtime engine as the agent which drives all the flow of data through our services. In fact, our current implementation is not in heart a true DSL as it donesn’t complete move you away from more mundane tasks like parsing data or having to solve some library/infrastructure matters (we are always trying to improve this point tho).

That was a response to our needs to provide the customers (and us)  a tool to deliver custom solutions, as often our customers are in need of an specific workflow which always requiered modifications to our “traditional” software. Building a server-side scripting system allows this and also lets thridparty enterprises join the game and develop custom tools that operate on top of our solution, further improving integration.

Our scripting language of choice is Boo, a dynamically typed programming language for the CLR, which has some interesting extensibility capabilities. By using macros, some code behind and time, one could develop some kind of DSL that may help the user concentrate on solving domain problems instead of wasting time solving technical problems related to the architecture of choice.

The source of inspiration was DSLs in Boo: Domain-Specific Languages in .Net, a not so structured book which guides the reader to each step involved on implementing a DSL based in Boo and RhinoDSL (his own dsl scripts factory library), it’s worth reading tho.

Our implementation has various key features:

  • Persitence through a well defined persistence points. Flow can be later resumed.
  • Wait for external events.
  • Not tied to any specific domain object.
  • Not tied to any specific domain object.
  • Parallel execution of scripts.
  • Service Oriented Architecture.

The script flow is moved from step to step (stages) which are the main code block within an script. These stages also define where the flow is persisted and subsequent runs won’t enter already executed stages.

The input of the script is open. It is defined by the script itself and its clients, which are responsible of feeding the correct data. To improve data handling we share a set of libraries to clients and server, where the input is defined for both parties.

Our runtime is capable of handling concurrent execution of scripts using the same (shared) underlying servicies (like persistence, tracking or any service that we want to publish to the script side).

We call this scripts “workflows”, being the script runtime one service of the new SOA platform we have built to deliver a product that can be easily modified to our customer needs.

Advertisements

Rest back-end and UIs

We have been spending the last four months moving our Win32 stand-alone applications to a more service oriented architecture. This new back-end  can be scaled to handle a wide variety of  TV Broadcasting enterprises. It has been a major movement from us, as many domain logic that was previously hardcoded into the desktop applications is now implemented into this back-end and shared among all clients. The logic is customizable per client basis through an scripting system.

We have targeted two desktop operating systems, MacOSX 10.6 and WindowsXP+ (in this order). These led to a new problem: developing a front-end that would offer the most user-friendly experience to the end user. Cocoa and WPF were the best choices and Monobjc provided a nice bridge to handle the Cocoa interface through Net/Mono. Silverlight was the third choice to those interfaces that could be also used from the web. The RESTful back-end allows us to easily develop and deploy any user interface while maintaining full compatibility within the installation with other clients.

Compared to AJAX and HTML5, Silverlight was the most cost-effective technology for us as we are used to write .Net code and we could use a large .net library we have already developed. So we have software (desktop applications for video editing, video review and cataloging) that share a common back-end and offer a native interface through Cocoa and WPF and some Web applications using Silverlight that also target MacOSX and Win32 clients but are more suited for the Web.

This huge project has been fun and having the chance to take part in almost any part of the new architecture (I have designed and implemented  part of our back-end and also being in charge of evaluating and teaching  Cocoa/Monobjc).

Monobjc and MonoMac

A few days ago Miguel de Icaza annocuded the availability of MonoMac, a new set of Binding to bring the Cocoa world to Mono.

There are others binding around, being Monobjc the most advanced one and more feature-rich from my point of view. The MonoMac approach is more dotNet-user friendly as it has .Net naming convention. This introduces a few requirements that I feel like a drawback more than an improvement. This naming convention needs a new and consistent documentation that allows the user to under the framework which will make the life harder if you are also reusing objective-c code, you cannot translate the code easily since you need a conversion step and You cannot either easily use the vast amount of cocoa documentation and examples found in the net, as the Cocoa api and MonoMac api are not named equally.

The other problem so far is the lack of examples and documentation in MonoMac, something that Monobjc project has solved from the early stages.

A new binding won’t do any good to the community, the user swill be spread around this two projects, lowering the final quality of both. We will continue with monobjc and we are looking forward to the new upcoming changes.

SynchronizationContext for Cocoa/Monobjc

Our current project does a heavy use of SynchronizationContext to be sure that all asynchronous notifications are executed in the proper thread. For our UI on Windows this means the main WPF’s thread which is provided by the framework. The problem raised when we started writing our Cocoa UI since we wanted to use the SynchronizationContext pattern we had been using on other platforms.

Monobjc provides a ISynchronizeInvoke interface implementation in NSView and is in fact quite straightforward to use but building your own SynchronizationCotext subclass for Cocoa
around the Monobjc bridge is also not that hard. The idea I had was to encapsulate the .Net callback and object state around an object (SyncObject) which inherits from NSObject.

Then we register our Objective-C callback into the main’s thread runloop by calling performSelectorOnMainThread:withObject:waitUntilDone: The ObjC callback is intercepted by the bridge which is then ran on the Mono side. There we can safely call the original .Net callback which is executed in the main thread.

The SyncObject can be (optionally) registered in a dictionary so that the Mono’s GC doesn’t have the chance to clean the object before the selector is executed. We unregister the SyncObject once the callback has been executed. This step is in fact unnecessary as the Monobjc bridge caches the object until a Release is intercepted from the ObjC side, that reference will keep the GC away from us. I have left the code commented just in case the implementation changes (which seems quite rare to happen…).

The idea can also be extended to any thread with a bit more work. If you extend this I would like to hear from you.

using System;
using System.Collections.Generic;
using System.Text;
using System.Threading;
using Monobjc.Cocoa;
using Monobjc;

namespace vsn.MacOSXComponents.Threading
{
    [ObjectiveCClass]
    public class SyncObject : NSObject
    {
        private SendOrPostCallback m_Callback;
        private Object m_State;

        public delegate void DeregisterDelegate(SyncObject sync_object);
#if USE_SYNCOBJS_DICTIONARY
        private DeregisterDelegate m_DeregisterDelegate;
#endif

public static readonly Class SyncObjectClass = Class.GetClassFromType(typeof(SyncObject));

        ///
        /// Initializes a new instance of the  class.
        ///
        public SyncObject() { }

        ///
        /// Initializes a new instance of the  class.
        ///
        ///

        public SyncObject(IntPtr nativePointer)
            : base(nativePointer) { }

        public override void  Dispose()
{
// This is here for testing purpouses
//Console.Write("Disposing SyncObject");
 	 base.Dispose();
}

        ///
        /// initializes the synchronization object with the given call back and object state
        ///
        /// callback
        /// state
        /// if true, the current thread waits until the operation is performed
        ///

        public void Initialize(SendOrPostCallback d, Object state, bool wait, DeregisterDelegate dereg_delegate)
        {
            m_Callback = d;
            m_State = state;
#if USE_SYNCOBJS_DICTIONARY
            m_DeregisterDelegate = dereg_delegate;
#endif

            // Call the NSObject's performSelector method from csharp code
        PerformSelectorOnMainThreadWithObjectWaitUntilDone(ObjectiveCRuntime.Selector("bridgedCallback:"), this, wait);
        }

        ///
        /// This is our callback which is invoked by the main thread's runloop
        ///
        [ObjectiveCMessage("bridgedCallback:")]
        public void BridgedCallback(NSObject param)
        {
            // Invoke the true callback
            m_Callback(m_State);         

#if USE_SYNCOBJS_DICTIONARY
    // Un register this object from our cache
            if (m_DeregisterDelegate != null)
                m_DeregisterDelegate(this);
#endif

// This region is unsafe as we are not protected against the GC. Monobjc
// has a cached reference to this object so shouldn't be a problem but
// could be a problem if the implementation changes.
// Consider using a CER
// GC.Collect(); // Testing

// Mark this object for deletion by an AutoreleasePool
this.Autorelease();
        }
    }

    ///
    /// Cocoa's SynchronizationContext implementation
    ///
    /// Only Post and Send methods are implemented
    ///
    public class CocoaRunLoopSynchronizationContext : SynchronizationContext
    {
        private Dictionary m_SyncObjectsDictionary;
        private readonly object m_lock = null;

        ///
        /// Default constructor
        ///
        public CocoaRunLoopSynchronizationContext()
            : base()
        {
            m_SyncObjectsDictionary = new Dictionary();
            m_lock = new object();
        }

        public override void Post(SendOrPostCallback d, Object state)
        {
    SyncObject sync = new SyncObject();
#if USE_SYNCOBJS_DICTIONARY
            lock(m_lock)
               m_SyncObjectsDictionary.Add(sync.GetHashCode(), sync);
#endif
            sync.Initialize(d, state, false, DeregisterSyncObject);

        }

        public override void Send(SendOrPostCallback d, Object state)
        {
            SyncObject sync = new SyncObject();
#if USE_SYNCOBJS_DICTIONARY
            lock (m_lock)
                m_SyncObjectsDictionary.Add(sync.GetHashCode(), sync);
#endif
            sync.Initialize(d, state, true, DeregisterSyncObject);
        }

        ///
        /// Called by SyncObject to remove itself from the dictionary so that no reference is held
        ///
        ///
target object to unregister
        private void DeregisterSyncObject(SyncObject sync_object)
        {
#if USE_SYNCOBJS_DICTIONARY
    lock (m_lock)
m_SyncObjectsDictionary.Remove(sync_object.GetHashCode());
#endif
        }
    }
}

Ping

I have been quite busy lately. Paris, Madrid and other personal stuff, but I’m still around.

I have been actively developing under Mono this month with not bad results so far. I have mixed feelings, it’s scaring when some internal behavior differs from the .Net implementation but it’s also amazing when you manage to host the Mono JIT inside a Cocoa dll which is loaded from a 100% Apple application.

I wish we had more documentation on how to mix Cocoa/Mono/Carbon/WPF in the ways we are doing at the office. I’m starting to think seriously on writing a book (which is something I have proposed to a colleague several times).

Workflows Foundation .Net 4

I have been making a prototype to evaluate the new incarnation of the Windows Workflow Foundation (WWF 4 beta 1), which will be part of the .Net 4 release. I’m glad of the new changes so far, asynchronous operations are now much easier, and this also applies to activities authoring too. I’m not so happy with the designer, it’s still a bit slow, although it is not an issue for me right now.

Workflow isolation has also been left behind. Running untrusted code can completely break the engine wich will also stop the other workflows running in parallel along with the faulted one. This is a big issue when running unmanaged code, where memory access violations or any other severe exception will crash the process. Writing the correct infrastructure to handle this situation is not trivial.

Many business problems are easily modeled using workflows, specially now that asynchronous transactions have been improved in this beta release. And WWF supports many (if not all) of the common workflow patterns. State machine workflows seems to be hard to implement with this release but I haven’t look into this yet.

You can find more info at the Windows Workflow Foundation for .Net 4 developer center: http://msdn.microsoft.com/en-us/netframework/aa663328.aspx

More on workflow patterns here: http://www.workflowpatterns.com/patterns/index.php