Migrating Higher Order Programming to the Web

My application, HOPE (Higher Order Programming Environment) is, in its simplest definition, a semantic publisher-subscriber application builder.  I’ve been wanting to move it to the web, but this is fraught with security concerns, hence my investigations into Docker technology and using a language like Python for the programming of “receptors” — the autonomous computational units that process semantic data.  Another stumbling block is my lack of experience with HTML5 canvas and graphics rendering.  Regardless though, there was no reason not to put together a proof of concept.

Here’s a quick walkthrough — no, this code is not yet publicly available.

Step 1: create a few receptors.

We’ll do some computations based on inputting a birth date.

Receptor #1: computing the age of a person in terms of years and days:

r1.png

Notice the class name computeAge.  We’ll talk about this later.

Receptor #2: compute the number of days to the birth day:

r2.png

Note the class name daysToBirthday.

Receptor #3: Get the interesting people born on the same month and day:

r3.png

Again, note the class name personsOfInterest.

Step 2: Add the Receptors to the Surface Membrane

r4.png

Step 3: Inject a semantic JSON object

Run the “membrane” and inject:

{"birthday": {"year": 1962, "month": 8, "day": 19}}

r5.png

And here’s the result:

r6.png

The full output being:

[
 {
 "age": {
 "years": 37,
 "days": 204
 }
 },
 {
 "daysToBirthday": 161
 },
 {
 "personsOfInterest": [
 "1724 Samuel Hood, 1st Viscount Hood, British admiral in the American Revolutionary War and the French Revolutionary Wars, born in Butleigh, England (d. 1816)",
 "1745 John Jay, American statesman, 1st US Chief Justice, born in New York City",
 "1863 Edvard Munch, Norwegian painter and print maker (The Scream), born in Ådalsbruk, Løten, Norway (d. 1944)",
 "1915 Frank Sinatra, American singer (Strangers in the Night, My Way) and actor (From Here to Eternity) known as 'old blue eyes', born in Hoboken, New Jersey (d. 1998)",
 "1932 robert pettit, American NBA star (St Louis Bombers/1959 MVP), born in Baton Rouge, Louisiana"
 ]
 }
]

What’s Going On?

Simply put, a preprocessor creates a mapping between semantic types and receptors (Python classes):

receptorMap = 
{
  'birthday':[computeAge(), personsOfInterest()],
  'age':[daysToBirthday()]
}

and the semantic processor engine routes the JSON semantic types to the Python class receptor’s process method.  When you inject “birthday”, it routes that type’s data to the computeAge and personOfInterest receptors.  The output of computeAge, which is of type “age”, is routed to the daysToBirthday receptor.

In this manner, one can create a library of small computational units (receptors) and build interesting “computational stories” by mixing and matching the desired computations.  Creating the receptors in Python makes this approach perfectly suited for running in Docker containers.  My ultimate vision is that people would start publishing interesting receptors in the open source community.

There’s still much more to go, but even as such, it’s a fun prototype to play with!  Some of the interesting problems that come out of this is, how do we let the end user create a visual interface (a UI, in other words) that facilitates both intuitive input of the semantic data as well as displaying real time output of the semantic computations.  Just the kind of challenging stuff I like!

FiddleDock – Creating a Dockerized Python Fiddle Web App

fiddledock.png

 

Using C#, a simple web server, and Docker, I’ve been putting together an example of one approach for how to create a “Fiddle” web app to run Python (or other script languages) in a Docker container.

The source code can be found on GitHub, but beware that at the time of this writing, the implementation isn’t complete (only “Run on Docker” and “Run on Host” buttons do anything) and I’m in the midst of writing the article on how I put this together (also in that repo – watch the progress as I write the article and finish the web app!)

Who was Born on Your Birthday?

cupcake2.jpg

Six lines (not including imports) of Python code to scrape the website onthisday.com for the “persons of interest.”  Impressive!

Package dependencies:

pip install lxml
pip install cssselect
pip install requests

The Python code:

import requests
from lxml import html
from lxml.cssselect import CSSSelector
from lxml import etree
 
page = requests.get("http://www.onthisday.com/birthdays/august/19")
tree = html.fromstring(page.content)
sel = CSSSelector('.section--person-of-interest')
pois = sel(tree)
 
for poi in pois:
 poi.xpath("div/div/div[1]/p")[0].text_content()

The results (for my birthday):

'1871 Orville Wright, aviator (Wright Brothers), born in Dayton, Ohio (d. 1912)'
'1878 Manuel Luis Quezon y Molina, Second President of the Philippines (1935-42), born in Baler, Aurora, Philippines (d. 1944)'
'1919 Malcolm Forbes, American publisher of Forbes Magazine, born in Brooklyn, New York (d. 1990)'
'1946 Bill Clinton [William Jefferson], 42nd US President (Democrat, 1993-2001), born in Hope, Arkansas'
'1967 Satya Nadella, Indian-American businessman (CEO of Microsoft), born in Hyderabad'

 

FlowSharpCode Gets DRAKON Shapes

drakon1.png

I’ve added some select DRAKON shapes for creating flowcharts.  The Python code in the lower right editor is generated from the flowchart, and the output from the run is shown on the left.

PyLint is also now integrated into FlowSharpCode’s PythonCompilerService.  This really improves the development process as many syntactical errors are detected before even running the code.

Also, the code generator creates an execution tree which independent of the language syntax, which means that support for other languages is easily added.  Now granted, the code itself in each of the DRAKON shapes is Python code, but I have some ideas of how to make that code agnostic as well.

Spinning 3D Box with BeagleBoneBlack and L3G4200D Gyro

bbbgyro.png

Got a fun little project working this weekend.  I wired up an L3G4200D gyroscope module to a BeagleBoneBlack, wrote the Python code to read the sensor data and send it over using RabbitMQ to a C# app on Windows displaying a 3D box.  Now when I rotate the BeagleBoneBlack, the cube mirrors my movements!

Full article will be posted on Code Project in the next week.

 

The Dangers of Duck-Typed Languages

ducktyped.jpg

Try these examples yourself in repl.it

First, is the ambiguity of what something is.  For example, consider this Python example:

> a=[] 

We have no idea what “a” is an array of.  Now, many people will say, it doesn’t matter, you’re not supposed to know, you just operate on the array.  There is a certain point to that, which however can lead to trouble.

Let’s try this:

> a=[1, 2, 3]

Ah, so you think we have an array of integers?  Think again:

> a.append('4')
> a
[1, 2, 3, '4']

Whoa!  That’s an array of mixed types.  In some ways that’s cool, in other ways, that’s dangerous.  Let’s say we want to add one to each element in the array, and we trust that the programmer that created / modified the array knows that it is supposed to be an array of int’s.  But how would they know?  Someone else can come along and not realize that they’re appending the array with a string.  So now we come along, expecting a happy array of int’s, and do this:

> [x+1 for x in a]
TypeError: cannot concatenate 'str' and 'int' objects

Oops – we get a runtime error!

What happens in Ruby:

> a=[1, 2, 3, '4']
[1, 2, 3, "4"]
> a.map {|x| x+1}
no implicit conversion of Fixnum into String

What happens in Javascript:

> a=[1, 2, 3, '4']
[1, 2, 3, '4']
> a.map(function(x) {return x+1})
[2, 3, 4, '41']

Holy Cow, Batman!  In Javascript, the string element is concatenated!

What does this mean?

It means that, among other things, the programmer must be defensive against, not necessarily the errors (sorry, I meant “usage”) of other programmers, but certainly the lack of strong typing in the language.  Consider these “solutions”:

Python:

> [int(x)+1 for x in a]
[2, 3, 4, 5]

Ruby:

> a.map {|x| x.to_i + 1}
[2, 3, 4, 5]

Javascript:

> a.map(function(x) {return parseInt(x)+1})
[ 2, 3, 4, 5 ]

Of course, if you have a floating point number in the array, it’ll be converted to a integer, possibly an unintended side-effect.

Another “stronger” option is to create a class specifically for integer arrays:

Python:

class IntArray(object):
  def __init__(self, arry = []):
    self._verifyElementsAreInts(arry)
    self.arry = arry

  # support appending to array.
  def __add__(self, n):
    self._verify(n)
    self.arry.append(n)
    return self

  # support removing element from array.
  def __sub__(self, n):
    self._verify(n)
    self.arry.remove(n)
    return self

  def _verifyElementsAreInts(self, arry):
    for e in arry:
      self._verify(e)

  def _verify(self, e):
    if (not isinstance(e, int)):
      raise Exception("Array must contain only integers.")


# good array
a = IntArray([1, 2, 3])
a += 4
print(a.arry)
a -= 4
print(a.arry)

try:
  a += '4'
except Exception as e:
  print(str(e))

# bad array
try:
  IntArray([1, 2, 3, '4'])
except Exception as e:
  print(str(e))

With the results:

[1, 2, 3, 4]
[1, 2, 3]
Array must contain only integers.
Array must contain only integers.

What this accomplishes is:

  1. Creating a type checking system that a strongly typed language does for you at compile-time
  2. Inflicting a specific way for programmers to add and remove items from the array (what about inserting at a specific point?)
  3. Actually doesn’t prevent the programmer from manipulating arry directly.
  4. Javascript? It doesn’t have classes, unless you are using ECMAScript 6, in which case, classes are syntactical sugar over JavaScript’s existing prototype-based inheritance.sed inheritance.

The worst part about a duck-typed language is that the “mistake” can be made but not discovered until the program executes the code that expects certain types.  Would you use a duck-typed language as the programming language for, say, a Mars reconnaissance orbiter?  It’ll be fun (and costly) to discover an error in the type when the code executes that fires up the thrusters to do the orbital insertion!

Which is why developers who promote duck-typed languages also strongly promote unit testing.  Unit testing, particularly in duck-typed languages, is the “fix” for making sure you haven’t screwed up the type.

And of course the irony of it all is that underlying, the interpreter still knows the type.

It’s just that you don’t.