Higher Order Programming on the Web is Alive

tademo1

A hard to read screenshot of HOPE (aka Semantic Computing) is now working on the web under the code name “Temporal Agency” (I love that name!), using:

  • Docker to isolate user projects and sessions
  • C# (Yay!)
  • DotNet Core 2.0 (Yay!)

Features:

  • IDE-style programming.
  • Syntax highlighting editor (courtesy of Ace).
  • Layout of results declared in XML.

This is very snazzy, in my humble opinion.  I have a couple demos working:

  • Weather forecast, tying in to UndergroundWeather, with summary (as shown above) and full forecast layouts.
  • Address verification using the USPS API’s.

More to come!

HOPE Semantic Rendering of Microservice Data on the Browser

In a previous blog post, I described the high level architecture of microservices
running in a Docker container.  This blog post describes (again at a high level) the behind-the-scenes implementation of how a we render the output of a microservice back to the client’s browser. We’ll use the Weather Underground API for acquiring our current weather conditions.

First, let’s start with a semantic address:

public class ST_Address : ISemanticType
{
  public string Address1 { get; set; }
  public string Address2 { get; set; }
  public ST_City City { get; set; }
  public ST_State State { get; set; }
  public ST_Zip Zip { get; set; }

  public ST_Address()
  {
    City = new ST_City();
    State = new ST_State();
    Zip = new ST_Zip();
}

Note that there are other semantics involved here (ST_City, ST_State, and ST_Zip) which we won’t go into here.  We’ll populate the ST_Address with the two pieces that Weather Underground requires, city and state, and then publish this type to the docker container:

http://192.168.99.100:5001/publishSemanticType?
  semanticTypeName=ST_Address&instanceJson=
  {
    "City":
      {"City":"Hudson"},
    "State":
      {"State":"NY"}
  }

The microservice, in HOPE terminology, is a “receptor” that is interested in
the type ST_Address.  As implemented in C# (fragment):

public class WeatherForecast : IReceptor
{
  public void Process(ISemanticProcessor proc, IMembrane membrane, ST_Address address)
  {
    string json = Http.Get("http://api.wunderground.com/api/[yourapikey]/forecast/q/" + 
      address.State.State + "/" + 
      address.City.City + 
      ".json");

    var data = WeatherUnderground.FromJson(json);

    var statForecast = data.Forecast.Simpleforecast.Forecastday.Select(d =>
      new
        {
          Month = d.Date.Monthname,
          Day = d.Date.Day,
          Weekday = d.Date.Weekday,
          High = d.High.Fahrenheit,
          Low = d.Low.Fahrenheit,
          Conditions = d.Conditions,
          IconUrl = d.IconUrl,
          MaxWind = d.Maxwind.Mph,
        }).ToList();

    // ...
    // Building the day and night detail forecast is not shown.
    // ...

    Response result = new Response()
    {
      HopeResultWrapper = new WeatherResult()
      {
        StatForecast = statForecast,
        DayForecast = daySummaryForecast,
        NightForecast = nightSummaryForecast,
      }
    };

    // Call back to the host with the result.
    RestCall.Post("http://192.168.0.2/hope/result", result);
  }
}

The response is associated with an XML file that determines the rendering.
Here’s a fragment showing just the first to rows (the day and the icon) of
rendering definition:

hope-5

HOPE renders this into HTML and sends the HTML over an HTML5 websocket to the browser where the page defines this tag:

hope-6

and on receiving the message from host, executes this Javascript:

 ws.onmessage = function (wsevt)
 {
   var msg = wsevt.data;
   $("#appContent").empty();
   $("#appContent").append(msg);
 };

The result is rendered on the browser like this:

hope-2

By changing the rendering to a VerticalGrid we instead get:

hope-3

By changing the rendering XML definition again (not shown, it’s quite long) we can show the results from all three collections — stat, day, and night — to render a more detailed forecast.

hope-4

Of course, in real life, you’ll want to render the results of many microservices.  We’ll look at how that works soon!

HOPE for the Web

hope-1

The above diagram is a high level view of what I’ve implemented with .NET Core 2.0 and Docker.  The idea is to take the work I’ve done with semantic types and allow users to create public or private repos of semantic types and microservices that operate on those types whenever they are “published” in the user’s environment.

With .NET Core 2.0, all the core pieces that I need (reflection, NewtonsoftJson, HttpListener, to name a few) are all fully usable with C# 7.  And while I’ll eventually return to supporting Python, it is a pleasure work with this concept in a strongly-typed language, as that’s a really key component to semantic processing (yes, of course it can be done in a dynamically-typed language with Python as well, as I’ve demonstrated in previous posts and an article.)

And the beauty of doing this with .NET Core is that it can run in a Linux container, which is so much faster than using Docker for Windows!

I’ll be posting more about the implementation details and writing some articles on Code Project on the subject.

Migrating Higher Order Programming to the Web

My application, HOPE (Higher Order Programming Environment) is, in its simplest definition, a semantic publisher-subscriber application builder.  I’ve been wanting to move it to the web, but this is fraught with security concerns, hence my investigations into Docker technology and using a language like Python for the programming of “receptors” — the autonomous computational units that process semantic data.  Another stumbling block is my lack of experience with HTML5 canvas and graphics rendering.  Regardless though, there was no reason not to put together a proof of concept.

Here’s a quick walkthrough — no, this code is not yet publicly available.

Step 1: create a few receptors.

We’ll do some computations based on inputting a birth date.

Receptor #1: computing the age of a person in terms of years and days:

r1.png

Notice the class name computeAge.  We’ll talk about this later.

Receptor #2: compute the number of days to the birth day:

r2.png

Note the class name daysToBirthday.

Receptor #3: Get the interesting people born on the same month and day:

r3.png

Again, note the class name personsOfInterest.

Step 2: Add the Receptors to the Surface Membrane

r4.png

Step 3: Inject a semantic JSON object

Run the “membrane” and inject:

{"birthday": {"year": 1962, "month": 8, "day": 19}}

r5.png

And here’s the result:

r6.png

The full output being:

[
 {
 "age": {
 "years": 37,
 "days": 204
 }
 },
 {
 "daysToBirthday": 161
 },
 {
 "personsOfInterest": [
 "1724 Samuel Hood, 1st Viscount Hood, British admiral in the American Revolutionary War and the French Revolutionary Wars, born in Butleigh, England (d. 1816)",
 "1745 John Jay, American statesman, 1st US Chief Justice, born in New York City",
 "1863 Edvard Munch, Norwegian painter and print maker (The Scream), born in Ådalsbruk, Løten, Norway (d. 1944)",
 "1915 Frank Sinatra, American singer (Strangers in the Night, My Way) and actor (From Here to Eternity) known as 'old blue eyes', born in Hoboken, New Jersey (d. 1998)",
 "1932 robert pettit, American NBA star (St Louis Bombers/1959 MVP), born in Baton Rouge, Louisiana"
 ]
 }
]

What’s Going On?

Simply put, a preprocessor creates a mapping between semantic types and receptors (Python classes):

receptorMap = 
{
  'birthday':[computeAge(), personsOfInterest()],
  'age':[daysToBirthday()]
}

and the semantic processor engine routes the JSON semantic types to the Python class receptor’s process method.  When you inject “birthday”, it routes that type’s data to the computeAge and personOfInterest receptors.  The output of computeAge, which is of type “age”, is routed to the daysToBirthday receptor.

In this manner, one can create a library of small computational units (receptors) and build interesting “computational stories” by mixing and matching the desired computations.  Creating the receptors in Python makes this approach perfectly suited for running in Docker containers.  My ultimate vision is that people would start publishing interesting receptors in the open source community.

There’s still much more to go, but even as such, it’s a fun prototype to play with!  Some of the interesting problems that come out of this is, how do we let the end user create a visual interface (a UI, in other words) that facilitates both intuitive input of the semantic data as well as displaying real time output of the semantic computations.  Just the kind of challenging stuff I like!

Distributed Semantic Computing

distcomp1

I’ve written an article on Distributed Semantic Computing:

Read the full article on Code Project: http://www.codeproject.com/Articles/1012650/Distributed-Semantic-Computing

Excerpts:

In this article, I will demonstrate distributed semantic computing using a Type-First Development (TFD) approach, a term first coined by Tomas Petricek in his blog entry “Why type-first development matters.”

In this article, I’ve re-written the HOPE engine to utilize “type declarative programming.”  This is a style of programming that relies heavily on generics to declaratively describe what should be done, not how.  It is the other side of the TFD coin — in addition to developing types first, we also implement processes that operate on generic types, particularly those that implement specific interfaces.  Similar to how events, delegates, callbacks, and so forth are used for an inversion of control with regards to program behavior, “type declarative programming” is an inversion of control for instantiating objects.  Unlike HOPE, where types are declared in XML and compiled at runtime, here we use types implemented in the code itself.  Because of .NET’s rich reflection and assembly loading capabilities, the difference is irrelevant to the overall goals of HOPE, but the difference to the developer is significant, especially with regards to the safety that a typed language gives you at runtime and the ease of programming (Intellisense and compile-time checking) in a typed language during development.

Type First Development (coined by Tomas Petricek) is applicable to imperative languages as well as functional languages.  We can use the C#’s type system to create rich types and declaratively establish the relationship between types and the methods that process those types.  We can also create containers (membranes) to create computational islands and control the flow of type instances between computational islands.  By using a semantic processor, the membranes, types and receptors that are declared in a “semantic system” becomes a expressive computational unit.  Specialized receptors, such as the distributed receptors illustrated in this article, demonstrate how easy it is to create a distributed semantic computing system.

Semantic Database: Concept, Architecture and Implementation

I’ve posted an article on Code Project describing a concept for a semantic database.  It’s in two parts, first discussing the concepts and looking briefly at the Semantic Web, RDF, and OWL, as well as relational, graph and NoSQL database architectures.

The second part is dedicated to the architecture and implementation of a semantic database with emphasis on unit testing.

This is Part 1 of 2 — Part II will look at using the semantic database in a real life HOPE application.