Kent PhD proposal

I’m delighted to have accepted an offer to study at the University of Kent for a full-time PhD this September. I spent January doing most of the work for this which is why I didn’t publish any posts that month. But now I’ve finally got round to the task of announcing it here.

Back in September, on Steve Krouse’s Future of Programming Slack channel, somebody posted a link to Tomas Petricek’s blog post Programming As Interaction, so thank you to whoever did that for starting me off on this path.

1. Introduction

Software as tools and products

Computer software, seen as a medium, can behave in any way we want within the physical constraints of the device it’s on. In theory, people are already carrying around universal simulators that they can tailor to their own desires. In practice, this is infeasible. Most software appears “fixed” or unchangeable, and must be accepted as-is.

If a user wants to adapt some spreadsheet for their own context, any ordinary spreadsheet application will do. But if they are dissatisfied with this application itself, then to do anything about it they must become familiar with its language, development, and architecture, before then forking and maintaining their own version — assuming it’s open-source in the first place.

Even if such an immense time investment succeeds, the same problem is simply pushed one step further. Although their text editor and compiler let them adapt the spreadsheet program to their wishes, if they wish to adapt these tools, they must go through the entire process again for each one.

The overarching metaphor is of “making” software “products” or “artefacts”, modifying them with software “tools”. Such do not come with their own mechanisms for adaptation, but rely instead on bringing in separate artefacts. The text editor, compiler, operating system and language runtime are all themselves software “products” previously “built” in such a way.

Nevertheless, all of this is software, made using itself. It must be asked whether the stark boundary between tool and product is really an unavoidable property of the material itself, or just a consequence of a particular methodology.

The whole is different

Curiously, when a programmer succeeds in hacking some desired change to their system, they do so using the very same system. All their work is accomplished by the same physical interface that they also “use” for everyday non-programming.

They certainly don’t need to buy a separate “development device” to program their own tools, nor a “development-device development device” to address the shortcomings of that! At the “whole device” level, it isn’t necessary to have “something special and separate” for development-as-opposed-to-use[1]. For the PC, the creation and adaptation of software is just another one of its uses.

Unlike the real-world supply chain, any user of computer software can easily download their own personal copies of “everything that is needed”. Yet, in a world where separate tools produce separate products, making use of this demands ever-growing expertise with more and more of the entire software economy.

An individual has about as much chance of understanding that as they would have with the real-life economy, at the intimate level of detail required.

If we are so inclined to have software toolchains emulate the real world, where modifying a stage requires familiarity with its preceding stage — then the PC is surely a counterexample. Its adaptation doesn’t translate to yet more stages on a pipeline. Could we give this property recursively to the software within it?

([1] The story is less clear for smartphones, tablets and embedded systems, but this only applies one level deep. True, embedded computer system software is developed outside the target device, and then deployed onto it as a final step. But such software is developed on a self-programmable machine, just pushed one step away: the developer’s computer.)

Software as something that must be interacted with

The actually-experienced reality of computing consists of humans interacting with a physical interface to a virtual world. This informs a model of software not as mathematical entities or lines of code, but as a dynamical system having time evolution in tight feedback with its user. The “current state” is defined by physical output capabilities[2], with the “independent variables” (besides time) defined by input capabilities[3].

Currently, achieving time-evolution of such overwhelmingly visual systems is accomplished through “coding”: a reductionist paradigm of serialising all non-textual entities in some way to text, to be suitable for automation via text. Colours, curves and polygons are worked with as lists of co-ordinates and quantities, themselves reduced to lists of digits — whether functioning as a count, length, angle, or something else. Thousands of words are routinely used to describe pictures.

The benefit of reductionism is the unifying of disparate concepts into a common representation. The benefit of this reductionism is to the implementors of basic tools:

• Monospaced glyphs in a single typeface are straightforward to lay out
• Character sets are well-defined, unlike the set of possible filled shapes
• Insert/delete within a character list is trivial compared to the space of “general graphical interactions”

But the costs, as if obeying a conservation law, are pushed to the user. Much of the precious human cognitive budget is diverted, from the actual problem being solved, to tasks including:

• Mental visualisation, simulation (while facing the screen of a universal simulator)
• Transcribing and decoding things that are not text
• Ensuring nested structures are correctly escaped, mentally decoding them back
• Locating, fixing, avoiding syntax errors
• ASCII art, indentation management

With so much existing text-based infrastructure, it is tempting to automatically write off anything else as hopelessly impractical. Nevertheless, for too many programming tasks, text is decidedly not “the right tool for the job”.

([2] Commonly: sounds and coloured shapes.)
([3] Commonly: keyboard keys, mouse or touchscreen.)

In a nutshell

The dominant way of working with software is as a set of production lines, all blueprinted in monospaced ASCII. This is rather a lot of one-size-fits-all assumptions.

It would be good to have an alternative that invites context-sensitive variation. Users would adapt their own copies of software artefacts to fit themselves, not the other way around. Correspondingly, designers of artefacts would not be dumped with the responsibility of anticipating every possible use case. Ideally, software would be some sort of “universal clay”, molded as desired.

While this is a vague end result, there is a clearer step towards it: the reduction of the huge gap between “development” and “use” of any given software tool. This has already been done “in the large”, in the form of the PC; it is natural to expect that this can be extended all the way down.

2. Related work

My interests here follow two strands: meta-circularity, and alternatives to plain text. I have already made some practical investigations in both of these areas.

Meta-circularity

Meta-circular languages claim to be implemented “in themselves”. This means that there is considerable freedom to alter syntax and semantics without the usual “stepping up the toolchain ladder”, but instead in the same source code or runtime environment being used.

The paper [OROM] demonstrates a minimal object system called “Id”, of 3 object types and 5 methods, which can be used to extend itself — even as far as arbitrary single/multiple inheritance, mixin, or traits semantics. It forms part of the authors’ wider work on “[COLAs]”, which draws heavily from two famously meta-circular languages: LISP and Smalltalk. They propose a fusion of the “even more late-bound Smalltalk” data model described in [OROM], and an “even more late-bound LISP” described in [OECM].

A term used frequently in the COLA work, from the field of compilers, is “bootstrapping”. This is the gradual building-up of a system, from some small starting point (the substrate), using itself. For example, we can use machine code to write the first assembler. Then, automatically-translated assembly can implement the first high-level compiler. The next language ecosystem is then born from the previous, and so on. This need not start from machine code — “self-hosting” compilers start from whatever is available.

There are other, fully-fledged Smalltalk derivatives: Squeak, Lively Kernel, Pharo, Fizzygum, and the Io language.

Of course, my interest is in meta-circularity as an intuitive alternative to the “toolchain ladder”. Stephen Kell’s work, on the boundaries between software artefacts, could provide a valuable counterpoint to investing too much in this view of the problem. One could argue that a meta-circular system defines its own little world, in competition with the host OS and ultimately aiming to replace it. This picture is rather different to one in which different meta-circular ecosystems can all co-exist and use each other’s features.

[OAP] and [Entangle] resemble the COLA work in aims and language (“open” / “closed”), if not in method. A [critique] of the latter is available, providing insights on the concepts of “substrate” and “open” systems. Its depiction of use / development as a relationship between “scope of change” and “difficulty” is relevant for the proposed Task 1.

Alternative substrates to character lists

As mentioned, the standard interface of automation is language, specifically its rendering as byte-lists. This is the case regardless of what exactly is being automated: whether transformations of other byte-lists, or movements of a 3D skeletal model. While it may have made sense in the day of the Model 33 Teletype, it can feel hopelessly anachronistic developing this way for modern use cases in which graphics, layout and direct manipulation are ubiquitous.

Ivan Sutherland’s Sketchpad was an early exploration of human-computer interaction via geometry instead of language. It is notable for the originality of the concept in 1962, the emphasis on constraint solving, and its facilities for graphical abstraction.

The Geometer’s Sketchpad is a “programmable geometry” prototype amongst other relevant projects in the book Watch What I Do. The Briar drawing program adds constraints on top of direct manipulation. Palimpsest is a non-textual environment for programming layer-based graphical compositions.

Bret Victor has presented video demonstrations and interactive prototypes of programmatically animating shapes primarily via shapes.

Infra presents an extensible structure that does away with constant parsing and serialisation. Worth mentioning is its Patch computation model (Section 6). Patch nodes give Infra structures “homoiconicity” in that evolutions of state (code) are represented in the same medium as the state itself. Infra could be a better substrate than bitstrings.

The Subtext project contains many years of serious research at doing better than character lists for specifying computation. One early innovation was the use of 2D “schematic tables” to simplify complex conditional logic.

My own work

BootstrapLab began as a table-based Web prototype of the Id object system. It let me work with a live analogue of the authors’ C code, without constantly typing commands to read and write the state of objects. It was a success at helping me understand their proposed “initial version” of a meta-circular system. It was, however, not clear how to scope the task of implementing the system “in itself”, because of the rich HTML substrate and GUI.

Due to this experience of “meta-circularity via text” outside of the terminal, I explored some avenues of text-free meta-circularity. Progress was more ad-hoc and still without guiding theory, so it was necessary to make some independent steps into an interaction-first model of computation, resembling the approach in [Entangle].

Other

My online collaborator Dan Cook has been developing ideas and prototypes in these areas. His explanation of the “clay” idea can be found here.

Finally, the phrasing “production of software by means of software” seems an apt way of describing this whole topic. It is a nod to the work of the Italian economist Piero Sraffa. Despite sharing an “input-output” recipe model, the connection might not be anything deep, since the core of Sraffa’s work is quantitative, and it isn’t clear how to quantify things like “amount of C compiler”.

Abbreviated references

[OROM] Open, Reusable Object Models
[COLAs] Combined Object Lambda Architectures
[OECM] Open, Extensible Composition Models
[OAP] The Open Authorial Principle
[Entangle] An Anatomy of Interaction: Co-Occurrences and Entanglements
[critique] Critique of Co-Occurrences and Entanglements

3. Proposed Work

Task 1

Evidently, there are many projects branching from LISP and Smalltalk that preserve the “meta-circular” spirit. In this respect, they are all doing the same thing. They imported an existing “bootstrapping” concept from compiler research. Software development as a whole “stands on its own shoulders”, as well as individual PC hacking.

Clearly these are all related, and there is plenty of informal demonstration of these principles. Since proponents of meta-circularity argue it to be so powerful, one would think there would be some systematic treatment of what exactly it means to implement something “in itself”, and how this differs from not doing so. However, there doesn’t seem to be any.

Such theory is critically needed. My own investigations suffered greatly from this lack of conceptual clarity and precise terminology. Doing successful practical research means first coming up with at least a way to talk about “producing software by means of software”. To start with, the term “meta-circularity” suggests “meta-linearity” for the conventional X-makes-Y-makes-Z, production- line model.

Also needing clarification is the difference between “use-as-intended” and “unan- ticipated modification”. The [critique] suggests that the issue may be of difficulty fitting scope, instead of somehow making everything the same[4].

Task 1: find, or develop, a theoretical framework in which existing meta-circular and meta-linear systems can be analysed and compared.

([4] For example, correcting a spelling error in an Excel menu does seem to have smaller scope than specifying relationships among cells. But unlike the latter, it’s not an anticipated use scenario, so its difficulty is in the same ballpark as other “code changes”.)

Task 2

Given so many similar projects in Section 2, this research does not seek to simply add another. Instead, it aims to explore an obvious gap:

• There are meta-circular textual systems. These are valuable, but risk stunting meta-circularity research by casting it as a property of languages rather than software.
• There are non-textual programming systems. However, it’s safe to assume that they were implemented via text[5]. Thus, as none seem to be meta- circular, they cannot escape text as their ultimate means of adaptation.

To fill this gap is to achieve software “bootstrapping” in a way not dependent on text: a minimal graphical system built up via keyboard, screen and mouse, in the same way that the “Id” system can be built up via typewriter and teletype printout. Armed with coherent guiding theory from Task 1, this could be a good opportunity to create a response to the aforementioned “textual reductionism”.

Programming nowadays could just as easily operate on a “graphical” reductionism – no less absurd — that treats text as shapes instead of shapes as text. This is a better fit for the computing of today: text glyphs are shapes just like any other. They must now be placed, sized and rasterised instead of “printed”.

Task 2: bootstrap a minimal Turing-complete graphical system in which visible shapes and interaction are central; implement text in terms of these.

By providing an example of a bootstrapped system that doesn’t conflate “software” with “language”, I hope to find out what is so special about meta-circularity. This should shed light on why the “programmability” of the computer as a whole, using itself, differs so greatly from the “programmability” of its subcomponents. I hope that a better understanding of this will contribute towards a future of genuinely open and ownable software.

([5] Needless to say, if there already exists a non-textual system not ultimately dependent on text, Task 2 may not be necessary.)