In “Levels of Organization of General Intelligence: The Modality Level”, Eliezer Yudkowsky mentions the idea of a “code” sensory modality, as part of a path which would lead to Artificial General Intelligence (to be fair to Eliezer, later remarks have mentioned that his latest insights have made this document obsolete, and I do not know if he still believes that). In that document, Eliezer makes concrete the challenge as: “What features does this modality need to extract to perceive the recursive algorithm for the Fibonacci sequence and the iterative algorithm for the Fibonacci sequence as ‘the same piece of code’?”

That is an interesting question: how does one design a sensory modality for code? My thinking started out fairly metaphorical. The vision system automatically gives us a sense of a moving object, but the object looks “the same” no matter what speed it moves.

For the “object” analogy, I would have “invariants”, or “assertions”. The sense would “perceive” that at line 152, the expression “(m+n)%2==0” is true (in words: n and m are even or odd together). In the Fibonacci example, the sense would “perceive” that, at the line just before the return statement, “return_value == fibonacci(input-1)+fibonacci(input-2)”. Obviously, the same sensory perceptions would lead to identifying the same “object”.

For the “movement” analogy, I would have the code try to prove performance guarantees: “From line 134-140, the code runs with at most 5*input operations”. Similarly, I would like space usage to be accounted for.

Implementing those senses is not easy — but not impossible. It works on a model of a theorem prover: it tries to prove theorems about the code, and then the question becomes a question of trying to guess at likely theorems and finding a proof for them. For cases where there is a dependency on a global variable, in many cases there is no “pure” theorem that can be proven, although theorems of the form “if this global variable does not change during the lifetime of the function…”. We would also need to prevent “trivial theorems” — those should be done away with by demanding that the orems which would be true for many different programs are not “interesting” (at the very least, being true for a function with no code, or for a line-range with no code, is not interesting).

If we had this sensory modality implemented as a program, and we already had some AI, we could obviously give this program to the AI to make it better at writing programs. However, even if we do not have an AI, this would be useful. For most editors, it would not be hard to convince them to run in a way that continuously saved the working file to disk. If we did that, we could run our “sensory modality” program, and continuously display its output to the side of the relevant lines in the program. That would be a big boon, both for understanding existing code and in trying to modify it, if we could immediately “see” the structure.


One Response to

  1. Joshua Fox says:


    This is nice. I wouldn’t have thought quite so far ahead.

    But at the basic level, it seems that a code sensory modality would “sense” the abstract-syntax-tree structure of code, as well as the various code paths possible, whether loops, conditionals, or potential exceptions.

    It might even have a higher-level semantic understanding and sense more and less important code paths; and which variables are “meant to be” nullable, for example.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: