This file contains various notes about the design of the compiler.
OUTLINE
The main job of the compiler is to translate Mercury into C, although it
can also translate (subsets of) Mercury to some other languages (Goedel,
the bytecode of a debugger currently under development, and in the future
the Aditi Relational Language).
The top-level of the compiler is in the file mercury_compile.m.
The basic design is that compilation is broken into the following
stages:
- parsing (source files -> HLDS)
- semantic analysis and error checking (HLDS -> annotated HLDS)
- high-level transformations (annotated HLDS -> annotated HLDS)
- code generation (annotated HLDS -> LLDS)
- low-level optimizations (LLDS -> LLDS)
- output C code (LLDS -> C)
Note that in reality the separation is not quite as simple as that.
Although parsing is listed as step 1 and semantic analysis is listed
as step 2, the last stage of parsing actually includes some semantic checks.
And although optimization is listed as steps 3 and 5, it also occurs in
steps 2, 4, and 6. For example, elimination of assignments to dead
variables is done in mode analysis; middle-recursion optimization and
the use of static constants for ground terms is done in code
generation; and a few low-level optimizations are done in llds_out.m
as we are spitting out the C code.
DETAILED DESIGN
(well, more detailed than the OUTLINE anyway ;-)
The action is co-ordinated from mercury_compile.m.
0. Option handling
The command-line options are defined in the module options.m.
mercury_compile.m calls library/getopt.m, passing the predicates
defined in options.m as arguments, to parse them. It then invokes
handle_options.m to postprocess the option set. The results are
stored in the io__state, using the type globals defined in globals.m.
1. Parsing
The result at this stage is the High Level Data Structure,
which is defined in four files:
- hlds_data.m defines the parts of the HLDS concerned with
function symbols, types, insts, modes and determinisms;
- hlds_goal.m defines the part of the HLDS concerned with the
structure of goals, including the annotations on goals;
- hlds_pred.m defines the part of the HLDS concerning
predicates and procedures;
- hlds_module.m defines the top-level parts of the HLDS,
including the type module_info.
The module hlds_out.m contains predicates to dump the HLDS to a file.
The module goal_util.m contains predicates for renaming variables
in an HLDS goal.
2. Semantic analysis and error checking
- implicit quantification
-
quantification.m handles implicit quantification and computes
the set of non-local variables for each sub-goal
- type checking
-
- typecheck.m handles type checking, overloading resolution &
module name resolution, and almost fully qualifies all predicate
and functor names. It sets the map(var, type) field in the
pred_info. However, typecheck.m doesn't figure out the pred_id
for function calls or calls to overloaded predicates; that can't
be done in a single pass of typechecking, and so it is done
later on in modes.m. When it has finished, typecheck.m calls
clause_to_proc.m to make duplicate copies of the clauses for
each different mode of a predicate; all later stages work on
procedures, not predicates.
- type_util.m contains utility predicates dealing with types
that are used in a variety of different places within the compiler
- mode analysis
-
- indexing and determinism analysis
-
- switch_detection.m transforms into switches those disjunctions
in which several disjuncts test the same variable against different
function symbols.
- cse_detection.m looks for disjunctions in which each disjunct tests
the same variable against the same function symbols, and hoists any
such unifications out of the disjunction.
If cse_detection.m modifies the code,
it will re-run mode analysis and switch detection.
- det_analysis.m annotates each goal with its determinism;
it inserts cuts in the form of "some" goals wherever the determinisms
and delta instantiations of the goals involved make it necessary.
Any errors found during determinism analysis are reported by
det_report.m.
Det_util.m contains utility predicates used in several modules.
- checking of unique modes (unique_modes.m)
-
unique_modes.m checks that non-backtrackable unique modes were
not used in a context which might require backtracking.
Note that what unique_modes.m does is quite similar to
what modes.m does, and unique_modes calls lots of predicates
defined in modes.m to do it.
- simplification (simplify.m)
-
simplify.m finds and exploits opportunities for simplifying the
internal form of the program, both to optimize the code and to
massage the code into a form the code generator will accept.
It also warns the programmer about any constructs that are so simple
that they should not have been included in the program in the first
place.
simplify.m calls common.m which looks for (a) construction unifications
that construct a term that is the same as one that already exists,
or (b) repeated calls to a predicate with the same inputs, and replaces
them with assignment unifications.
simplify.m also attempts to partially evaluate calls to builtin
procedures if the inputs are all constants (see const_prop.m).
3. High-level transformations
The first two passes of this stage are code simplifications.
To improve efficiency, the above two passes are actually combined into
one - polymorphism.m calls calls lambda__transform_lambda directly.
Most of the remaining HLDS-to-HLDS transformations are optimizations:
- specialization of higher-order predicates where the value of the
higher-order arguments are known (higher_order.m)
- inlining (i.e. unfolding) of simple procedures (inlining.m)
- constraint propagation (constraint.m)
Not yet working.
- issue warnings about unused arguments from predicates, and create
specialized versions without them (unused_args.m); type_infos are
often unused
- elimination of dead procedures (dead_proc_elim.m). Inlining, higher-order
specialization and the elimination of unused args can make procedures dead
even the user doesn't, and automatically constructed unification and
comparison predicates are often dead as well.
- elimination of useless assignments, assignments that merely introduce
another name for an already existing variable (excess.m).
- reducing the number of variables that have to be saved across procedure calls
(saved_vars.m). We do this by putting the code that generates the value of
a variable just before the use of that variable, duplicating the variable
and the code that produces it if necessary, provided the cost of doing so
is smaller than the cost of saving and restoring the variable would be.
The module transform.m contains stuff that is supposed to be useful
for high-level optimizations (but which is not yet used).
Eventually we plan to make Mercury the programming language of the Aditi
deductive database system. When this happens, we will need to be able to
apply the magic set transformation, which is defined for predicates
whose definitions are disjunctive normal form. The module dnf.m translates
definitions into DNF, introducing auxiliary predicates as necessary.
4. Code generation
- pre-passes to annotate the HLDS
-
Before code generation there are a few more passes which
annotate the HLDS with information used for code generation:
- choosing registers for procedure arguments (arg_info.m)
-
Currently uses one of two simple algorithms, but
we may add other algorithms later.
- annotation of goals with liveness information (liveness.m)
-
This records the birth and death of each variable
in the HLDS goal_info.
- allocation of stack slots
-
This is done by live_vars.m, which works
out which variables need to be saved on the
stack when, and then uses graph_colour.m to determine
a good allocation of variables to stack slots.
- migration of builtins following branched structures
-
This transformation, which is performed by
follow_code.m, improves the results of follow_vars.
- allocating the follow vars (follow_vars.m)
-
Traverses backwards over the HLDS, annotating some
goals with information about what locations variables
will be needed in next. This allows us to generate
more efficient code by putting variables in the right
spot directly. This module is not called from
mercury_compile.m; it is called from store_alloc.m.
- allocating the store map (store_alloc.m)
-
Annotates each branched goal with variable location
information so that we can generate correct code
by putting variables in the same spot at the end
of each branch.
- code generation
-
For code generation itself, the main module is code_gen.m.
It handles conjunctions and negations, but calls sub-modules
to do most of the other work:
- ite_gen.m (if-then-elses)
- call_gen.m (predicate calls and also calls to
out-of-line unification procedures)
- disj_gen.m (disjunctions)
- unify_gen.m (unifications)
- switch_gen.m (switches), which has sub-modules
- dense_switch.m
- lookup_switch.m
- string_switch.m
- tag_switch.m
- pragma_c_gen.m (embedded C code)
It also calls middle_rec.m to do middle recursion optimization.
The code generation modules make use of
- code_info.m
-
The main data structure for the code generator
- code_exprn.m
-
This defines the exprn_info type, which is
a sub-component of the code_info data structure
which holds the information about
the contents of registers and
the values/locations of variables.
- exprn_aux.m
-
Various preds which use exprn_info
- code_util.m
-
Some miscellaneous preds used for code generation
- code_aux.m
-
Some miscellaneous preds which, unlike those in
code_util, use code_info
- continuation_info.m
-
For accurate garbage collection, collects
information about each live value after calls,
and saves information about procedures.
The result of code generation is the Low Level Data Structure (llds.m).
The code is generated as a tree of code fragments which is then
flattened (tree.m).
5. Low-level optimization
The various LLDS-to-LLDS optimizations are invoked from optimize.m.
They are:
- optimization of jumps to jumps (jumpopt.m)
- elimination of duplicate code sequences (dupelim.m)
- optimization of stack frame allocation/deallocation (frameopt.m)
- filling branch delay slots (delay_slot.m)
- dead code and dead label removal (labelopt.m)
- peephole optimization (peephole.m)
- value numbering
This is done by value_number.m, which has the following sub-modules:
- vn_block.m
-
Traverse an extended basic block, building up tables showing
the actions that must be taken, and the current and desired
contents of locations.
- vn_cost.m
-
Computes the cost of instruction sequences.
Value numbering should never replace an instruction
sequence with a more expensive sequence. Unfortunately,
computing costs accurately is very difficult.
- vn_debug.m
-
Predicates to dump data structures used in value
numbering.
- vn_filter.m
-
Module to eliminate useless temporaries introduced by
value numbering. Not generating them in the first place
would be better, but would be quite difficult.
- vn_flush.m
-
Given the tables built up by vn_block and a list of nodes
computed by vn_order, generate code to assign the required
values to each temporary and live location in what is
hopefully the fastest and most compact way.
- vn_order.m
-
Given tables built up by vn_block showing the actions that
must be taken, and the current and desired contents of
locations, find out which shared subexpressions should
have temporaries allocated to them and in what order these
temporaries and the live locations should be assigned to.
This module uses the module atsort.m to perform an approximate
topological sort on the nodes of the location dependency
graph it operations on (since the graph may have cycles,
a precise topological sort may not exist).
- vn_table.m
-
Abstract data type showing the current and desired
contents of locations.
- vn_temploc.m
-
Abstract data type to keep track of the availability
of registers and temporaries.
- vn_type.m
-
This module defines the types used by the other
modules of the value numbering optimization.
- vn_util.m
-
Utility predicates.
- vn_verify.m
-
Sanity checks to make sure that (a) the optimized code
computes the same values as the original code, and (b)
the optimized code does not dereference tagged pointers
until the tag is known. (Violations of (b) usually cause
unaligned accesses, which cause bus errors on many machines.)
Several of these modules (and also frameopt, above) use livemap.m,
which finds the set of locations live at each label.
Depending on which optimization flags are enabled,
optimize.m may invoke many of these passes multiple times.
Some of the low-level optimization passes use opt_util.m, which
contains miscellaneous predicates for LLDS-to-LLDS optimization.
6. Output C code
- base_type_info.m generates the base_type_info structures that list the
unification, index and compare predicates associated with each declared
type constructor. These are added to the LLDS.
- base_type_layout.m generates the base_type_layout structures that give
information on how to interpret values of a given type. It also
creates base_type_functors structures that give information on
the functors of a given type. The base_type_layout and base_type_functors
structures of each declared type constructor are added to the LLDS.
- llds_common.m extracts static terms from the main body of the LLDS, and
puts them at the front. If a static term originally appeared several times,
it will now appear as a single static term with multiple references to it.
- Final generation of C code is done in llds_out.m.
BYTECODE
The Mercury compiler can translate Mercury programs into bytecode for
interpretation by the Mercury debugger currently under development.
The generation of bytecode happens after semantic checks have been
completed.
- bytecode.m defines the internal representation of bytecodes, and contains
the predicates to emit them in two forms. The raw bytecode form is emitted
into <filename>.bytecode for interpretation, while a human-readable form
is emitted into <filename>.bytedebug for visual inspection.
- bytecode_gen.m contains the predicates that translate HLDS into bytecode.
MISCELLANEOUS
- det_util:
-
This module contains utility predicates needed by the parts
of the semantic analyzer and optimizer concerned with
determinism.
- special_pred.m, unify_proc.m:
-
These modules contain stuff for handling the special
compiler-generated predicates which are generated for
each type: unify/2, compare/3, index/1 (used in the
implementation of compare/3), and also type_to_term/2
and term_to_type/2 (but those last two are disabled
at the moment).
- dependency_graph.m:
-
This contains predicates to compute the call graph for a
module, and to print it out to a file.
(The call graph file is used by the profiler.)
The call graph may eventually also be used by det_analysis.m,
inlining.m, and other parts of the compiler which could benefit
from traversing the predicates in a module in a bottom-up or
top-down fashion with respect to the call graph.
- passes_aux.m
-
Contains code to write progress messages, and higher-order
code to traverse all the predicates defined in the current
module and do something with each one.
- opt_debug.m:
-
Utility routines for debugging the LLDS-to-LLDS optimizations.
CURRENTLY USELESS
The following modules do not serve any function at the moment.
Some of them are obsolete; other are work-in-progress.
(For some of them its hard to say which!)
- lco.m:
-
This finds predicates whose implementations would benefit
from last call optimization modulo constructor application.
It does not apply the optimization and will not until the
mode system is capable of expressing definite aliasing.
- mercury_to_goedel.m:
-
This converts from item_list to Goedel source code.
It works for simple programs, but doesn't handle
various Mercury constructs such as lambda expressions,
higher-order predicates, and functor overloading.
- mercury_to_c.m:
-
The very incomplete beginnings of an alternate
code generator. When finished, it will convert HLDS
to high-level C code (without going via LLDS).
Last update was $Date: 1997/09/23 16:48:43 $ by $Author: fjh $@cs.mu.oz.au.