Migrating Higher Order Programming to the Web

My application, HOPE (Higher Order Programming Environment) is, in its simplest definition, a semantic publisher-subscriber application builder.  I’ve been wanting to move it to the web, but this is fraught with security concerns, hence my investigations into Docker technology and using a language like Python for the programming of “receptors” — the autonomous computational units that process semantic data.  Another stumbling block is my lack of experience with HTML5 canvas and graphics rendering.  Regardless though, there was no reason not to put together a proof of concept.

Here’s a quick walkthrough — no, this code is not yet publicly available.

Step 1: create a few receptors.

We’ll do some computations based on inputting a birth date.

Receptor #1: computing the age of a person in terms of years and days:


Notice the class name computeAge.  We’ll talk about this later.

Receptor #2: compute the number of days to the birth day:


Note the class name daysToBirthday.

Receptor #3: Get the interesting people born on the same month and day:


Again, note the class name personsOfInterest.

Step 2: Add the Receptors to the Surface Membrane


Step 3: Inject a semantic JSON object

Run the “membrane” and inject:

{"birthday": {"year": 1962, "month": 8, "day": 19}}


And here’s the result:


The full output being:

 "age": {
 "years": 37,
 "days": 204
 "daysToBirthday": 161
 "personsOfInterest": [
 "1724 Samuel Hood, 1st Viscount Hood, British admiral in the American Revolutionary War and the French Revolutionary Wars, born in Butleigh, England (d. 1816)",
 "1745 John Jay, American statesman, 1st US Chief Justice, born in New York City",
 "1863 Edvard Munch, Norwegian painter and print maker (The Scream), born in Ådalsbruk, Løten, Norway (d. 1944)",
 "1915 Frank Sinatra, American singer (Strangers in the Night, My Way) and actor (From Here to Eternity) known as 'old blue eyes', born in Hoboken, New Jersey (d. 1998)",
 "1932 robert pettit, American NBA star (St Louis Bombers/1959 MVP), born in Baton Rouge, Louisiana"

What’s Going On?

Simply put, a preprocessor creates a mapping between semantic types and receptors (Python classes):

receptorMap = 
  'birthday':[computeAge(), personsOfInterest()],

and the semantic processor engine routes the JSON semantic types to the Python class receptor’s process method.  When you inject “birthday”, it routes that type’s data to the computeAge and personOfInterest receptors.  The output of computeAge, which is of type “age”, is routed to the daysToBirthday receptor.

In this manner, one can create a library of small computational units (receptors) and build interesting “computational stories” by mixing and matching the desired computations.  Creating the receptors in Python makes this approach perfectly suited for running in Docker containers.  My ultimate vision is that people would start publishing interesting receptors in the open source community.

There’s still much more to go, but even as such, it’s a fun prototype to play with!  Some of the interesting problems that come out of this is, how do we let the end user create a visual interface (a UI, in other words) that facilitates both intuitive input of the semantic data as well as displaying real time output of the semantic computations.  Just the kind of challenging stuff I like!

FlowSharpCode, continued…


A simple example, but the “problem” is that the three Drakon shapes (begin loop, output, and end loop) each still have individual C# code-behind in each shape.  For example, the begin loop has the code-behind:

 var n in Enumerable.Range(1, 10)

My original idea was that the Drakon shape description should not define the language-specific syntax, instead that should be implemented by the developer in the code-behind.

In practice (and I’ve written a complex application in FlowSharpCode, so I know) it becomes unwieldy to deal with one-liner code behind, the result of which is that I tend not to use Drakon shapes, but that results in nothing better than a meaningless box with some code in it.

I’m also reluctant to put the code in the shape label (though this is supported) as again we’re now dealing with language specific syntax.

I’m also reluctant to create a meta-language for Drakon shapes, for example, something that could interpret:

n = 1..10

into C#, Python, whatever.  What if the developer wants to write:

Count from 1 to 10

So, what I’m considering is letting the developer create the Domain Specific Language (DSL) so that they can expressively communicate the semantics of a Drakon shape and also provide the rules for how the semantics is parsed, ideally in an intermediate language (IL), for example, something that expresses a for loop, a method call, whatever.

The advantage to this is that the developer can create whatever DSL they like to work in, the IL glues it together into the concrete language.

Two things happen then:

  1. The DSL is interchangeable.  Any IL can be super-composed into your DSL choice.
  2. The IL is language independent, so it can be de-composed into language specific syntax.

Item #2 of course imposes some significant limitations — what if a language doesn’t support classes, or interfaces, or yield operator, or whatever?  I’m not particularly too concerned about that as a language-independent DSL/IL is more of a curiosity piece, as it becomes rapidly untenable when your code starts calling language-framework-platform dependencies.

However, I’d love to hear my readers thoughts on this DSL/IL concept I’m considering.



Code Iterations – A Mentoring Example


One of the pleasures in life is mentoring another developer, particularly when the other developer is smart and motivated to learn.  This article, on populating a tree from a collection of paths, was the result of some weekend prep work and is a good case study on refactoring.  By going through the process myself and documenting it, I was able to present the problem in general terms, and the person I was mentoring did the heavy mental lifting with only occasional guidance on my part.  This worked because I was prepared — had I not done this prep work, I would have taken away from my mentee’s experience to  actually solving the problem himself.

Article here.

IDX – Integrated Developer Experience


Move aside IDE’s (Integrated Developer Environment) – it’s time for the new kid on the block, the Integrated Developer Experience!

Ok, so the acronym is already taken (Internet Data Exchange, Indonesia Stock Exchange, and probably others) but I’m co-opting it for how to talk about FlowSharpCode.  I’m actually surprised “Integrated Developer Experience” isn’t used somewhere already.  Maybe my google-fu is not up to snuff right now.

FlowSharpCode Gets DRAKON Shapes


I’ve added some select DRAKON shapes for creating flowcharts.  The Python code in the lower right editor is generated from the flowchart, and the output from the run is shown on the left.

PyLint is also now integrated into FlowSharpCode’s PythonCompilerService.  This really improves the development process as many syntactical errors are detected before even running the code.

Also, the code generator creates an execution tree which independent of the language syntax, which means that support for other languages is easily added.  Now granted, the code itself in each of the DRAKON shapes is Python code, but I have some ideas of how to make that code agnostic as well.

The Juice

the juice.png

I was recently asked (paraphrasing) what parts of the work of software engineering do I find “juicy” so I came up with this diagram. Any software engineering task involves both developers (if only me), customers (might be a client), and the processes of design and implementation.

The “external” blue lines are where the customer potentially interacts with the developers, the output of the design, and the implementation phase (you can probably imagine how Agile fits in this.) The “internal” red lines are where the developers interact with the each other and the design and implementation phases.

From a certain perspective, the left side represents the “process” and the right side represents the “results.” Process and results should be balanced – developers may discover they require training in new skills, teams adjust based on where the team is in the process, etc. The process creates results which the developer and customer team review.

The process – results flow iterates with each result. The earlier results are produced, the better for everyone because this is where “education” occurs, for example, the developers learn more about the customer’s requirements, the customer may refine their requirements (or change them!) Both the developers and the customers learn things during iterations which in turn create adjustments in the process.

The list of items within the boxes is basically just all the stuff that I find “juicy” – the more of those items that get checked off for a project, the more excited I typically find myself regarding working on the project.