Open code and you

doom

Moving some of my projects to the wild, I mean to GitHub, has made me realize what others already knew, much of the software we use everyday is open source or has some roots there. Be honest, peeking to someone else’s software is great. As to my own experience I started learning programming as a child reading the source code of the Doom Editing Utilities (and that probably shifted and twisted my understanding of computers for my entire life). This concepts of seeing others work and learning from it is a hidden fact that affects our life in for example every gadget we use (read Android/iOS). But not only to learn but to understand what a piece of software does, its source code is the ultimate documentation. If you are using a third party library, ask for its source code and peek at it when needed. At Video Stream Networks we made a similar approach, we had our domain logic code written in C++ every time we had to tune it for every customer’s workflow it made us feel miserable. After we moved our domain logic into Casper, a Domain Specific Language for the media and broadcast world, our customer can not only peek at what our app does but understand and fine tune it themselves.

Thanks to GitHub and the Open Source Community, ShadingZen Engine is slowly maturing. The GitHub repo received a pull request with some great changes. The project now has Maven build, the structure is more clear with all the examples and documentation centralized in just one repository and the eclipse requirement has been removed (This last point allowed me to try out IntellIJ IDEA which so far I find as a really good java IDE).

Yes, open source level editors as well a robust designed and extensible engine was behind the success of DOOM, the very same that made some of us programmers.

Domain-Specific Languages

Boo

Half the work I’m currently doing here at Video Stream Networks is to design a DSL runtime engine as the agent which drives all the flow of data through our services. In fact, our current implementation is not in heart a true DSL as it donesn’t complete move you away from more mundane tasks like parsing data or having to solve some library/infrastructure matters (we are always trying to improve this point tho).

That was a response to our needs to provide the customers (and us)  a tool to deliver custom solutions, as often our customers are in need of an specific workflow which always requiered modifications to our “traditional” software. Building a server-side scripting system allows this and also lets thridparty enterprises join the game and develop custom tools that operate on top of our solution, further improving integration.

Our scripting language of choice is Boo, a dynamically typed programming language for the CLR, which has some interesting extensibility capabilities. By using macros, some code behind and time, one could develop some kind of DSL that may help the user concentrate on solving domain problems instead of wasting time solving technical problems related to the architecture of choice.

The source of inspiration was DSLs in Boo: Domain-Specific Languages in .Net, a not so structured book which guides the reader to each step involved on implementing a DSL based in Boo and RhinoDSL (his own dsl scripts factory library), it’s worth reading tho.

Our implementation has various key features:

  • Persitence through a well defined persistence points. Flow can be later resumed.
  • Wait for external events.
  • Not tied to any specific domain object.
  • Not tied to any specific domain object.
  • Parallel execution of scripts.
  • Service Oriented Architecture.

The script flow is moved from step to step (stages) which are the main code block within an script. These stages also define where the flow is persisted and subsequent runs won’t enter already executed stages.

The input of the script is open. It is defined by the script itself and its clients, which are responsible of feeding the correct data. To improve data handling we share a set of libraries to clients and server, where the input is defined for both parties.

Our runtime is capable of handling concurrent execution of scripts using the same (shared) underlying servicies (like persistence, tracking or any service that we want to publish to the script side).

We call this scripts “workflows”, being the script runtime one service of the new SOA platform we have built to deliver a product that can be easily modified to our customer needs.