Warning: These are just a few notes quickly thrown together, to be clarified and expanded.
Our current "state of the art" in the area is represented by the test pypy.jit.test.test_hint_timeshift.test_arith_plus_minus(). It is a really tiny interpreter which gets turned into a compiler. Here is its complete source:
def ll_plus_minus(encoded_insn, nb_insn, x, y): acc = x pc = 0 while pc < nb_insn: op = (encoded_insn >> (pc*4)) & 0xF op = hint(op, concrete=True) # <----- if op == 0xA: acc += y elif op == 0x5: acc -= y pc += 1 return acc
This interpreter goes via several transformations which you can follow in Pygame:
py.test test_hint_timeshift.py -k test_arith_plus_minus --view
What this does is turning (the graph of) the above interpreter into (the graph of) a compiler. This compiler takes a user program as input -- i.e. an encoded_insn and nb_insn -- and produces as output a new graph, which is the compiled version of the user program. The new output graph is called the residual graph. It takes x and y as input.
This generated compiler is not "just-in-time" in any sense. It is a regular compiler, for now.
Let's follow how the interpreter is turned into a compiler. First, the source of the interpreter is turned into low-level graphs in the usual way (it is actually already a low-level function). This low-level graph goes then through a pass called the "hint-annotator", whose goal is to give colors -- red, green and blue -- to each variable in the graph. The color represents the "time" at which the value of a variable is expected to be known, when the interpreter works as a compiler. In the above example, variables like pc and encoded_insn need to be known to the compiler -- otherwise, it wouldn't even know which program it must compile. These variables need to be green. Variables like x and acc, on the other hand, are expected to appear in the residual graph; they need to be red.
The color of each variable is derived based on the hint (see the <----- line in the source): the hint() forces op to be a green variable, along with any previous variable that is essential to compute the value of op. The hint-annotator computes dependencies and back-propagates "greenness" from hint() calls.
The hint-annotator is implemented on top of the normal annotator; it's in hintannotator.py, hintmodel.py, hintbookkeeper.py, hintcontainer.py and hintvlist.py. The latter two files are concerned about the blue variables, which are variable that contain pointers to structures of a mixed kind: structures which are themselves -- as containers -- known to the compiler, i.e. green, but whose fields may not all be known to the compiler. There is no blue variable in the code above, but the stack of a stack-based interpreter is an example: the "shape" of the stack is known to the compiler when compiling any bytecode position, but the actual run-time values in the stack are not. The hint-annotator can now handle many cases of blue structures and arrays. For low-level structures and arrays that actually correspond to RPython lists, hintvlist.py recognize the RPython-level operations and handles them directly -- this avoids problems with low-level details like over-allocation, which causes several identical RPython lists to look different when represented as low-level structs and arrays.
Once the graph has been colored, enters the "timeshifter". This tool -- loosely based on the normal RTyper -- transforms the colored low-level graphs into the graphs of the compiler. Broadly speaking, this is done by transforming operations annotated with red variables into operations that will generate the original operation. Indeed, red variables are the variables whose run-time content is unknown to the compiler. So for example, if the arguments of an int_add have been annotated as red, it means that the real value of these variables will not be known to the compiler; when the compiler actually runs, all it can do is generate a new int_add operation into the residual graph.
In the example above, only acc += y and acc -= y are annotated with red arguments. After hint-rtyping, the ll_plus_minus() graph -- which is now the graph of a compiler -- is mostly unchanged except for these two operations, which are replaced by a few operations which call helpers; when the graph of the now-compiler is running, these helpers will produce new int_add and int_sub operations.
XXX the following is not the way it works currently, but rather a proposal for how it might work -- although it's all open to changes. It is a mix of how Psyco and the Flow Object Space work.
Unlike ll_plus_minus() above, any realistic interpreter needs to handle two complications:
Keep in mind that this is all about how the interpreter is transformed to become a compiler. Unless explicitly specified, I don't speak about the interpreted user program here.
As a reminder, this is handled in the Flow Space as follows:
In Psyco:
The "tree of EggBlocks" approach doesn't work too well in general. For example, it unrolls loops infinitely if they are not loops in the bytecode but loops in the implementation of a single opcode (we had this problem working on the annotator in Vilnius).
The current preliminary work on the timeshifter turns the interpreter into a compiler that saves its state at _all_ join-points permanently. This makes sure that loops are not unexpectedly unrolled, and that the code that follows an if/else is not duplicated (as it would be in the tree-of-EggBlocks approach). It is also inefficient and perfect to explode the memory usage.
I think we could try to target the following model -- which also has the advantage that simple calls in the interpreter are still simple calls in the compiler, as in Psyco:
The motivation to do point 2. differently than in Psyco is that it is both more powerful (no extra unrolling/duplication of code) and closer to what we have already now: the bookkeeping code inserted by the timeshifter in the compiler's graphs. In Psyco it would have been a mess to write that bookkeeping code everywhere by hand, not to mention changing it to experiment with other ideas.
An idea to consider: red variables in the compiler could come with a concrete value attached too, which represents a real execution-time value. The compiler would perform concrete operations on it in addition to generating residual operations. In other words, the compiler would also perform directly some interpretation as it goes along. In this way, we can avoid some of the recompilation by using this attached value e.g. as the first switch case in the red-to-green promotions, or as a hint about which outcome of a run-time condition is more likely.