For Yacas developers


A crash course in Yacas maintenance for developers

This document intends to give a concise description of the way Yacas is maintained. There are a few parts to maintenance to take into account:

http://www.xs4all.nl/~apinkus/backups/


The autoconf/automake system

The short story is as follows. You probably do not need to bother about this unless you introduce a new file. However, if you add a new file, it probably should be added to the Makefile.am file in the same directory. In many cases, it should be clear from the Makefile.am file where your new file should be added. For instance, new Yacas script files go into the huge list in scripts/Makefile.am that is assigned to the SCRIPTFILES variable. Similarly, test scripts should go in the list in tests/Makefile.am that is assigned to the TESTFILES variable. Note that you should probably also run the cvs add command, as explained in the section on CVS below. If you remove a file, then you should go through the inverse procedure.

The addition of new files to the Makefile.am ensures that it will be added to the tarball yacas-*.tar.gz which is uploaded to the backup repository. This has the nice side effect that you can have local files which don't automatically get added to the distribution (by not adding them to the Makefile.am file). Additionally, files which are not listed in Makefile.am may not be built and/or installed automatically. To make sure that the tar.gz distribution is complete, you can run the command
make distcheck
This may take a little while, as it needs to rebuild and test the whole system from the tar.gz tarball.

If you want to do more complicated things (like adding files which are not Yacas script or test files, or files which should be compiled or installed only conditionally), or if you are simply curious, you can read more in the chapter entitled "The Yacas build system".


Maintaining Yacas through a cvs repository

CVS provides an efficient way for developers to work together, automatically merging changes various developers make, and at the same tile is a back up system (uploading your changes to another computer from which you can easily obtain it at a later time). After a little effort setting it up it becomes very easy to use. This section describes the few commands needed for keeping your version and the version in the Yacas repository up to date.


How does cvs work?

CVS has a copy of the files in the repository somewhere in a directory on some system, possibly your computer. Then there is such a thing as a cvs server which you can talk to to synchronize your version of the source code with the version on the server.

CVS uses a diff-like scheme for merging differences: it looks at two text files, determines the different lines, and merges accordingly. It discovers the changes you made by looking at the version you checked out last and the version you have now, to discover which lines changed (it maintains an automatic version number for each file).

If the version of a file on your system and the version in the cvs repository has a line that has been changed by both you and some one else, the cvs repository will obviously not know what to do with that, and it will signal a 'collision' which you will have to solve by hand (don't worry, this rarely happens). More on that later.

In commands to be described in this document are in short:


Checking out an initial version of Yacas

There are two ways to check out a version of Yacas: as anonymous user and as maintainer. Anonymous users don't need to log in, but also have no right to commit changes. Maintainers first need to get an account (at sourceforge), and their account needs to be enabled so they are allowed by the maintainer to make changes. A maintainer needs to log in with every command. To be able to log in, you need ssh1 installed (ssh2 will not work). You can find this at http://www.ssh.org/download.html.

To check out Yacas as anonymous user, type:

cvs -d:pserver:anonymous@cvs.yacas.
  sourceforge.net:/cvsroot/yacas login
cvs -z3 -d:pserver:anonymous@cvs.yacas.
  sourceforge.net:/cvsroot/yacas co yacas

To check out as a maintainer, type:

export CVS_RSH=ssh1

This will tell CVS to use ssh1 for communication. Then, in order to download the yacas source tree, type

cvs -d:ext:loginname@cvs.yacas.sourceforge.
  net:/cvsroot/yacas co yacas
where loginname is your name on the sourceforge system. This creates a directory yacas/ with the full most recent distribution. You need to enter your password there, but other than that, that's it!

Those lines typed above are long and obscure, but it is also the last time you need to type them. From now on, if you want to do anything with cvs, just go into the yacas/ directory you just checked out, and type the cvs command without the -d:... flag. This flag just tells cvs where to find the repository. But future cvs commands will know where to find them, which is why you don't need that flag.


Use case scenario 1 : getting the latest version of Yacas

You haven't looked at Yacas for a while (shame on you!) and want to check out the latest version. Just type

cvs update -d

on the command line in the yacas directory, and that should essentially download the latest version for you in that directory (just the changes). The -d option here states that you are also interested in new directories that were added to the repository. Oddly enough, cvs will only get you changed and added files, not added directories, by default.

A command

cvs -q update -d

will print messages only about changed files.


Use case scenario 2 : you made changes to Yacas

You got the latest version, but saw this huge, glaring omission in Yacas, and start hacking away to add it yourself. After a while, after playing with the code you wrote, and if you think you are finished with it, you decide you like to add it to the cvs repository.

First, you should test the new Yacas system:
make test
If there are any failed tests, you need to fix them.

Now you can start entering your changes to the CVS. If you created some new files, you need to tell CVS to add them to the source tree:

cvs add [list of file names of ascii text files]

This adds ascii text files. If you added binary files (GIF images in the documentation directory, or something like that), you can add it to the CVS with

cvs add -kb [list of file names of binary files]

Note that, when adding files to the CVS, you should normally also add them to the Yacas tar.gz distribution. This is done by adding the file name to the EXTRA_DIST variable in the file Makefile.am in the directory where you were adding the file.

In case files need to be removed, there are two options:

There seems to be no easy way to rename or move files; you would have to remove them at their old location and add them at a new location.

Now, when finished with that, you might want to 'commit' all changes with

cvs commit

If the commit succeeds, an email is sent out to the maintainers, who can then scan the diff files for changes, to see if they agree with the changes, and perhaps fix mistakes made (if any).

If there is a collision, the commit fails (it will tell you so). This might happen because someone else also edited the same place in a file and their changes cannot be automatically merged with yours. In case of a collision, you need to invoke cvs update twice. The cvs update outputs a list of file names with a character in front of them. The important ones are the files with a 'C' before them. They have a collision. You can go into the file, and see the collision, which the cvs system conveniently marks as:

<<<<<<
old version
===========
new version
>>>>>>

You can edit the file by merging the two versions by hand. This happens very rarely, but it can happen. Use cvs commit afterwards to commit.

The commit and update commands can be performed in specific directories, and on specific files, if necessary, by stating them on the command line. Or you can go into a sub directory and do a cvs commit or cvs update there, if you are confident that is the only place that changed or whose changes you are interested in.

That is basically it, a quick crash course cvs. It is actually very convenient in that usually all that is needed is a cvs commit to fix small bugs. You type that in, and your version gets merged with the changes others made, and they get your changes, and you backed up your changes at the same time (all with that little command!).

You can find more information about cvs at http://cvsbook.red-bean.com/ .


Preparing and maintaining Yacas documentation


Introduction

Yacas documentation in HTML and PS/PDF formats is generated by Yacas scripts from Yacas source files. Prior to version 1.0.48, all documentation had to be written directly in the Yacas language. However, it was very cumbersome to write those source files in the Yacas language. The scripts txt2yacasdoc.pl, book2TeX.sh, book2ys.sh, ytxt2tex were created to help maintain the documentation in an easy-to-read form.

The "source" form of all documentation is maintained in a special plain text format. The format is such that it is clearly readable without any processing and is easy to edit. To compile the documents, the system processes the plain text docs with a script to prepare Yacas-language files and then runs other scripts to produce the final documentation in HTML and other formats.

The source text must be formatted in a certain fashion to delimit sections, code examples, and so on, but the format is easy enough to enter in a plain text editor. Text is marked up mostly by TAB characters, spaces, and asterisks "*" at the beginning of a line. The format is easy enough so that, for example, the text of the GNU GPL (the file COPYING) can be used as a documentation source file without changes. The script txt2yacasdoc.pl converts this markup into Yacas code for further processing.


Organization of the Yacas documentation

All documentation source files are kept in the subdirectory manmake. During compilation, Yacas language files as well as HTML, LaTeX and PS/PDF files are automatically generated in the manmake subdirectory. Contributors should only need to edit *.txt files in manmake.

Currently, the Yacas documentation consists of four "core" books (the introductory tutorial, the programming tutorial, the user's reference manual, and the programmer's reference manual), and three "extra" books (algorithms, Lisp programming, essays).

There is also a documentation book describing the Emacs interface to Yacas (the "Yacas Notebook" mode). This book is not available as online help but is installed separately in the TeXinfo or PostScript formats.

All Yacas documentation books are distributed under the GNU Free Documentation License (FDL). If you add a new documentation book to Yacas, please include the file FDL.chapt.

The documentation books are meant to be stand-alone texts, except the two "reference manual" books which are meant to be used together because they share a common hyperlinked table of contents. The Yacas Help() command will show a reference article from either of the two reference books.

Stand-alone books are free-form, but reference books must be written with a certain template that allows online hyperlinking. The file manmake/dummies is an example template for a reference manual section.

Books are divided into "chapters", "sections" and "subsections". The reference manuals contain descriptions of Yacas commands and functions, and each function is given in a separate (unnumbered) section which is marked by a special *CMD label (see below).

At the beginning of each book there must be a book title and a short book description (labeled *BLURB). The short description appears in the printed version as the subtitle on the title page. It also goes into the HTML top-level book index as the book description.

At the beginning of each chapter there may be a "chapter introduction" labeled *INTRO which is also a short description of the contents of that chapter. It may be one paragraph only. The "chapter intro" feature is only used in the HTML reference manual because it is the text that appears at the very top of a reference manual section. As you can see in the HTML version of reference docs, each chapter contains a list of all functions described in it. This list goes right after the first paragraph of "chapter intro". In the printed (PS/PDF) documentation, the *INTRO label is ignored and the "chapter intro" paragraph is shown exactly like any other text paragraph.

Each printed book contains a table of contents at the beginning and an index at the end. The index should help a reader to quickly find a particular concept. For example, all documented Yacas commands are automatically entered into the index in the reference manual. Additional index entries should be inserted by hand using the *A label (see below).


Translating the documentation

The Yacas system is under active development and its documentation is constantly updated. An effort to translate the Yacas documentation from English into other languages has been started.

Translators should try to prepare translated documentation in the same plain text format as the original documentation. Any problems with conversion to HTML and PS/PDF formats should be easy to solve (at least for European languages).

The most important documentation books to translate are the reference manual and the tutorials. There is substantial documentation aimed at developers, for instance the algorithms book or the essays book. It is probably not as important to translate such books, since Yacas developers have to speak English to communicate.


Formatting of source text files

Formatting of source text files uses TAB symbols; if your editor does not support them and converts them to spaces, you should convert the results back to contain real TAB symbols using the standard Unix unexpand utility or a custom perl script.

You may want to examine the source of this file (manmake/ YacasDocs.chapt.txt) to see how various features of the markup are used. Currently the following markup is implemented:

*	Item
*	Another item
This will produce:

Note that the text of an item continues until the next itemized line is given or until end of paragraph (does not have to be all in one line).

Nesting of enumerated or itemized environments is not supported, except for fringe cases of nesting just one itemized list at the very end of an enumerated list or vice versa.

The enumerated environment is currently only implemented in LaTeX docs; HTML docs render them as itemized.

Note: Currently, only the HTML documentation is hyperlinked, while the printed PS/PDF documentation contains only text.

There is a special feature for displayed equations: Any punctuation immediately following the second pair of dollar signs will be displayed on the same line. (This is used to get around the limitation of mathematical expressions that cannot end with a comma or a period or with another punctuation mark.) For example, the formula "$$x^2/2$$," will produce

x^2/2,

with the comma on the same line. A formula such as "$x+1;$" will generate an error; the semicolon should be moved out of the dollar signs.

As special exceptions, one can enter the symbols " TeX" and " LaTeX" as if they are Yacas expressions, i.e. "$TeX$" produces " TeX". One can also create a capitalized form of the name Yacas as "{Yacas}".

Please note: Mathematical expressions must be valid Yacas expressions, with no unbalanced parentheses, no undefined infix operators, no hanging periods and so on, or else the Yacas script that formats the docs will fail! (This limits the scope of mathematical formulae but is hopefully not critical.)

Also, please avoid putting equations into documentation as plain text. Expressions such as a>0 are not correctly typeset by TeX if included into the plain text without the dollar signs: "a>0". (The HTML documentation is currently not affected.)

Currently, when creating online HTML documentation, all mathematics is kept in Yacas notation and set in boldface font. (This may change in the future.) Of course, LaTeX typesets the documentation with correct mathematical symbols.

Another feature of the LaTeX exporter is that it will first try to represent all functions and infix operators according to their mathematical meaning, and if no such meaning is defined in Yacas, then it will show them exactly as they are written in Yacas. For infix operators to work, they have to be declared in the standard library, or else an error will occur when processing the manual.

For example, the Yacas operators = and == are represented in LaTeX by an equals sign (=), the operator := becomes "identically equal" ( a:=b), and the cosmetic operators <> and <=> become a<>b and a<=>b. But you cannot use an infix operator such as ":=*" because it is not defined in the standard library. A Yacas function which is not defined in the standard library, for example "func(x)", will appear just like that: func(x).

*INCLUDE ../essays/howto.chapt

Note that the included document is the file howto.chapt, not howto.chapt.txt, because it must be a Yacas-language file, not a .txt file. (The IncludeFile() call, an alias to Load(), will be used to execute the specified file.)

For example, the text
*FOOT This is an example footnote
generates the footnote

This is an example footnote.

For example,
*EVAL "Yacas version: " : Version()
will insert the string ` Yacas version: 1.0.57 ' into the manual. Note the spaces around the version string---these additional spaces are currently unavoidable.

If the absence of spaces is critically important, you can create the required text in a Yacas expression.


Formatting conventions for the reference manual

The formatting explained in the previous section is enough to create most of the user guide and tutorial documentation. The script txt2yacasdoc.pl also implements some additional markup features to help create the reference manual.

A typical reference manual subsection documenting a certain function may look like this in plain text:

*CMD PrintList --- print list with padding
*STD
*CALL
   {PrintList}(list)
   {PrintList}(list, padding);

*PARMS

{list} -- a list to be printed

{padding} -- (optional) a string

*DESC
Prints {list} and inserts the {padding} ...

*E.G.

	In> PrintList({a,b,{c, d}}, " .. ")
	Out> " a ..  b .. { c ..  d}";

*SEE Write, WriteString
Compare this with the reference manual section on the function PrintList to see how this plain text markup is rendered in the finished documentation.

Notes:

*CMD Sin, Cos, Tan --- Trigonometric ...

In addition to the above labels, there are the following tags:

Usage of the *A label currently does not directly affect the appearance of the docs. In the HTML docs, it inserts the insivible anchor tags <a></a>. In the printed LaTeX docs, the *A label adds an index entry. The *CMD tag generates all necessary HTML anchors and index entries for commands in the reference manual. So only non-command index entries need to be manually entered using *A.

The *INTRO and *BLURB tags only work for one paragraph. There must be no empty line between *INTRO/*BLURB and that paragraph. Also, there must be no empty line between the "blurb" and the book title (for technical reasons). There must be one and only one "blurb" paragraph in a "book" and no more than one "chapter intro" paragraph per chapter.

This markup should be sufficient for creating reference documentation in plain text.


Indexing the documentation books

It is not difficult to automatically generate an alphabetically sorted index for the books. An "index entry" is a piece of text that does not appear in the book where it is entered, but instead is printed in the alphabetical list at the end of the text with the relevant page number.

Currently the following facilities are provided for indexing:

After LaTeX generates a "raw" index file *.idx, the makeindex utility is used to post-process and sort the index into the .ind file. If you do not have makeindex on your system, the book indices will not be generated.

Note that makeindex is not always friendly to special (non-alphanumeric) characters. For example, it uses the symbol ! to separate index topics, which may conflict with Yacas commands. In other words, document index must be tested and sometimes debugged.

In the HTML docs, the index is currently not generated on a separate page, although HTML anchors are inserted in the text. The reference manual uses the HTML anchors to provide online help through the ? command.

An index entry may be a "topic" with "subtopics", which usually appears in book indices like this:
gnus, 51
   tame, 51
   wild, 52-341
This effect can be achieved with the ! topic separator:
*A gnus
*A gnus!tame
*A gnus!wild
This is a special feature of makeindex.

Currently, it is possible to include a command or an equation into an index entry, for example,
*A {InterestingCommand}
*A calculation of $Sqrt(x)$
But this may sometimes conflict with the topic separator.


Summary of mark-up labels

Mark-up labels must appear as first characters on a line. The following mark-up labels are currently defined:

Labels with an argument on the same line (affect only the current line):

Labels that affect the rest of the line and the subsequent paragraph:

Special labels for the reference manual that accept several arguments on the same line:

Special labels without arguments that generate headings for the reference manual:

Other special markup:


Summary of special markup syntax

Special syntax entities include:


Debugging the manual

Sometimes the manual compilation make or make texdocs will break after you edit the plaintext manual sources. This can happen for one of these reasons:

In case of a math syntax error, the documentation exporter cannot print the paragraph where the error occurred, but it usually prints the preceding paragraph. Currently, the easiest way to locate the error is to generate the .tex output and look at it, e.g.:
make ref.book.tex; less ref.book.tex
The last line in the .tex file must be \end{document}. If it is not, then the last portion of the text you see in the .tex file is the text directly before the paragraph where the error occurred. Most probably, there is a malformatted math formula in the next paragraph of your plaintext source.

If the last line is \end{document} but latex does not finish, you will have to run latex by hand, e.g.
latex ref.book.tex
and look at the error message(s) it prints.


Using the script txt2yacasdoc.pl

The script txt2yacasdoc.pl is used to transform plain text markup into the Yacas language. The script acts as a stream filter:

perl txt2yacasdoc.pl < file.txt > file.chapt

In this example, file.txt contains some formatted plain text (source text) and the resulting file file.chapt will be produced in Yacas-language documentation format.

There is a single option for txt2yacasdoc:

perl txt2yacasdoc.pl -debug < file.txt \
  > file.chapt
This option is to be used for debugging, i.e. when the resulting file does not compile in Yacas. The effect of this option is to introduce more breaks between text strings in the generated file, so that the Text() function is called more often. It is then easier to locate the source of the problem in the Yacas-language file (Yacas will tell you the last line in the Yacas-language file at which a syntax error occurred). This option is largely obsolete because the Text() function is called frequently enough by default. See below for hints about finding syntax errors in documentation when the manual does not compile.


book2TeX: preparing typeset documentation

The script book2TeX.sh prepares a TeX file out of a Yacas-language documentation book. Usage is similar to book2txt.sh, except that only one file is processed at a time and the file must be a "book", not just a "chapter". For example:
book2TeX.sh intro.book intro.book.tex
will create a LaTeX-formatted version of the introductory tutorial. The LaTeX file can be processed with standard tools, for example
latex intro.book.tex
dvips -o intro.book.ps intro.book dvi
will prepare a Postscript version, while
pdflatex intro.book.tex
will prepare a PDF version.

To generate printed docs, it is necessary to run latex (at least) three times in a row. This is because at first latex does not know how much space will be taken by the table of contents and the index, so the page numbers are all off by a few pages. Only on the second run latex generates correct page numbers for the TOC (the .aux file) and for the index (the .idx file). After this the index file has to be processed by the makeindex routine to sort it, and the third latex run is needed to actually insert the correct TOC and the processed index into the final document.

The shell commands in book2txt.sh execute the following Yacas commands:
Use("book2TeX.ys");
ToFile("file.chapt.tex")
  Load("file.chapt");
This requires that the Yacas script book2TeX.ys be available in the current directory. The shell script book2TeX.sh assumes that book2TeX.ys is stored in the same directory as book2TeX.sh and that the Yacas executable is available in the directory ../src/. Alternatively, the command line of the Yacas executable can be specified by the -run option. For example, the Makefile runs book2TeX.sh like this:
book2TeX.sh -run "yacas-dir/src/yacas --rootdir
  yacas-dir/scripts" file.book file.book.tex
Note that the entire Yacas command line is given in quotes.

Some concerns with the printed documentation
Not all features of the Yacas documentation-generating scripts are compatible with TeX typesetting. To prevent errors, documentation source should avoid certain things. In general, it is a good idea to check the typeset appearance of documentation, since it helps detect errors.

For example, the symbols %, { }, < >, #, \, _ and & are special to TeX. They should not normally be used in plain text; it is okay to use them in "typewriter" text (within braces {}) or code samples -- but not in section or chapter heads, because it makes it difficult to export to TeX correctly. TeX commands may be entered but will not be correctly rendered in HTML online documentation.

Sometimes fixed-font text will hang over the right edge of the printed page. A workaround is to break the fixed-font text into shorter fragments or to rephrase the text.

Another concern is that code examples (TAB-indented blocks) are typeset in a fixed-width font and may not fit into the width of the page. To avoid this, the lines in the code examples should not be longer than about 50 characters.

The current implementation uses a "report" style which allows chapters, sections, subsections and includes an automatically generated table of contents and an index. The standard 10 point font and two-column format are used to save space (and trees).

The script txt2yacasdoc.pl attempts to convert double quotes "" into proper English quotation marks "". However, this automatic conversion sometimes fails and produces wrongly-directed quotes. One such case is if the quotes are on the same line as a TAB character (e.g. in an itemized environment). This problem can be circumvented by putting the quoted words on a different line.

Some complicated mathematical expressions may not correctly render in TeX. This is because Yacas uses its library function TeXForm() to transform Yacas expressions to TeX. Mathematical expressions are entered in the plain text documentation source using Yacas syntax, then transformed to a special non-evaluating call TeXMath() in the Yacas-language documentation, which formats into HTML using a Write() call or into TeX using a TeXForm() call, as necessary. Testing should be performed on documentation before releasing it. The most stringent limitation is that the expression between dollar signs should evaluate in Yacas (preferably to itself) and not cause syntax errors. In case of doubt, check that the expression evaluates without errors and then try to use TeXForm on that expression and see if that evaluates without errors as well. For example, expressions such as x=+1 will cause a syntax error and this will break the compilation of the manual (both HTML and TeX).


book2ys: extracting Yacas code from books ("literate programming")

book2ys.sh is a shell script that extracts Yacas code examples from a documentation chapter into a separate file. All other text is omitted.

Usage is similar to book2TeX.sh. For example, the benchmarking test code wester.yts can be automatically extracted from the corresponding essay chapter by the command
sh ../manmake/book2ys.sh wester-1994.chapt \
  wester.yts
After this command, the file wester.yts is created. Note that wester-1994.chapt is in Yacas language and is itself a generated file.

To prepare a documentation chapter in such a way that code extraction is possible, one needs to make sure that all code examples in the chapter taken together will become a correct sequence of Yacas expressions when cut out and written sequentially into a file. So, for instance, semicolons at the end of each statement are required. The script book2ys will not export example Yacas session code with "In>" and "Out>" prompts but it will export all other example code.

The example code with "In>" and "Out>" prompts will become comments in the exported Yacas file. It is possible to suppress this comment generation by the -strip option to the book2ys.sh script.

If the output file name is not given, the name will be the same as the Yacas book source name but with the .ys extension.

See the source file wester-1994.chapt.txt to get a feeling of how the source documentation is formatted to allow completely automatic code extraction. Note that the printed documentation may be in twocolumn format, and therefore it is necessary to split lines that are too long.

Using the script book2ys.sh, one can write documentation and code together, a la "literate programming". The main idea of literate programming is that the program code and the documentation should be written as one large book explaining to humans how the program is organized and how it works. From this book, a printable copy is generated and all code is automatically extracted for compilation.

Literate programming may require that code be split between many source files, and yet it may be convenient to keep all descriptions in one book. The special document formatting label *YSFILE can be used to redirect output to different Yacas source files.

By default, the output goes to the file specified on the book2ys.sh command line. This default can be restored by the directive "*YSFILE -" at any time. Otherwise, all code output will be printed to the file specified after the *YSFILE label.

Note that the multiple file support is somewhat restrictive:

Here is an example of using multiple files. Note how documentation text is interspersed with TAB-indented code.

Text that does not appear in the code.
    // code into default file
    // more code
*YSFILE script1.ys
This will not appear in the file.
    x:=1;
    // some code for the file script1.ys
*YSFILE script2.ys
    // some code for the file script2.ys
End of the example, need to reset *YSFILE.
*YSFILE -
After processing this file with book2ys.sh, one should get three files with Yacas code.


ys2book: extracting documentation from Yacas code ("literate programming")

The standard view of literate programming is that one prepares a book readable by humans, and all code is automatically extracted from the book. The focus in this approach is on writing explanations on how the code works.

The converse approach is to write primarily code and embed some documentation as comments in the code files. This approach is implemented by the script ys2book.pl.

This script takes a Yacas script file and extracts Yacas comments from it into another file. Usage may look like this:

perl ys2book.pl < file.ys > file.chapt.txt
perl ys2book.pl -strip < file.ys > file.chapt.txt
Here file.ys is the source file and file.chapt.txt is the output file.

Not all comments may be desirable as documentation. Formatting of comments is implemented as follows:

All exported text is printed to the output file as is, without any additional reformatting. The only change to the text is stripping of initial spaces.

Any leading spaces after the beginning of the comment sign are removed. For example,
/*      text */
will be exported as just "text" without the leading spaces. In a multiline comment, such as
/*    start
      of comment */
the leading spaces in the first line will be stripped. However, the leading spaces (and TABs) in other lines of the multiline comment block will be preserved.

Empty lines can be introduced into the documentation either as part of a multiline comment block, or as a standalone empty comments such as
///
////////

With these features it is easy to prepare the embedded documentation in the Yacas plaintext documentation format. This format requires space- and TAB-based formatting, which is mostly preserved by the script ys2book.pl.


ytxt2tex: Conversion of plain text documentation to LaTeX

An auxiliary script ytxt2tex converts plain text documentation to LaTeX. The script ytxt2tex can be used outside of the Yacas source tree to convert individual documents to LaTeX. This is useful if you would like to produce TeX documents and if you find the plain text format of the Yacas documentation more comfortable. Therefore, ytxt2tex is a kind of a special-purpose TeX preprocessor designed for producing Yacas documentation.

This is a standalone script; it is installed by default into /usr/local/bin and requires the Yacas executable also on the path, as well as the script files book2TeX.* and txt2yacasdoc.pl in the /manmake subdirectory of the Yacas installation tree /usr/local/share/yacas/. The script also requires perl and the Unix shell sh.

Limitations of this script are:

These limitations may be easily overcome by editing the resulting TeX file (but you need to know at least some TeX to do that).

The general usage pattern is
ytxt2tex [-o outputfile] file1.txt [file2.txt] ...
All source files must have extension ".txt". The command-line option -o specifies the name of the output TeX file. If the -o option is not given, the output file will be file1.tex (i.e. the name of the first .txt file with the .tex extension). If several .txt files are given, the first one must *INCLUDE all others.

To illustrate the usage of the script ytxt2tex, consider two examples.

The first example is just one plaintext file example1.txt. This file will have to be a "book" in itself, i.e. it will have to include a book title indented by four TAB symbols. For example:

*REM file: example1.txt
                Example Document Title
*REM this is a section title:
		Numbers and letters
*REM here are some index entries:
*A numbers
*REM simple index entries like this are OK
*A letters

Numbers and letters are very important, etc.

This file example1.txt can be converted to a LaTeX file example1.tex by the following simple command:
ytxt2tex example1.txt
If the resulting file should be named something other than example1.tex, say output1.tex, then the command is
ytxt2tex -o output1.tex example1.txt

The second example is a longer "book" consisting of several plaintext files. One of these files is a "master file" and it should include all other files using the *INCLUDE label. The *INCLUDE label should contain file names without the .txt extension.

Suppose we have prepared the files book1.txt, chapter1.txt, and chapter2.txt containing the preamble text and two chapters. For example:

*REM this is the file "book1.txt"
*BLURB or, The Multitudinous Attempts
to Avoid Counterproductivity
                 Relationships of Entities
*INCLUDE chapter1
*INCLUDE chapter2

The chapter files might be:

*REM this is the file "chapter1.txt"
        Entities and Epiphenomena
The history of the ambiguous question of
epiphenomenological discourse can be
traced to the pre-postmodern period...

*REM this is the file "chapter2.txt"
        Substrates and Superficiality
In the preceding chapter, we have thoroughly
investigated the metaphilosophical aspects of
the trans-homocentric considerations...

The command to create the final LaTeX file book1.tex is
ytxt2tex book1.txt chapter1.txt chapter2.txt
The "master file" book1.txt that includes all other text files must be given first. The -o option can be used if the final LaTeX file should be named something else than book1.tex. For example,
ytxt2tex -o MyBook.tex book1.txt chapter*.txt

By default, both table of contents and the index are generated. The commands to create a PostScript file out of the LaTeX file might be:
latex MyBook.tex
latex MyBook.tex
makeindex MyBook.idx -o MyBook.ind
latex MyBook.tex

Note that the resulting LaTeX file needs to be processed three times if the table of contents or index are to be used. Without a table of contents and index, it is enough to process the file with LaTeX twice.


book2txt: Conversion of existing documentation to plain text

(Note: as of version 1.0.49, all Yacas documentation is converted to plaintext format. This section is left for reference only.)

Currently, most but not all of the Yacas documentation markup functionality is implemented in the simple plaintext filter; also, documentation includes some extra HTML files. However, almost all of the reasonable markup needed to write documentation is present. Therefore it is possible to maintain most of the documentation in the plain text format described above. To convert existing Yacas documentation back to the plain text format, a script book2txt.ys/book2txt.sh can be used.

By using a command such as
book2txt.sh file.chapt
one can create a source text file file.chapt.txt corresponding to the Yacas documentation file file.chapt. For example:

12:51pm scriabin> book2txt.sh intro.book
[editvi.ys] [gnuplot.ys] 
True;
Out> True;
Quitting...
File 'intro.book.txt' was created.
12:51pm scriabin>

In the above example, the shell commands in book2txt.sh executed the following Yacas commands,
Use("book2txt.ys");
ToFile("file.chapt.txt")
  Load("file.chapt");
This requires that the Yacas script book2txt.ys be available in the current directory. The shell script book2txt.sh assumes that book2txt.ys is stored in the same directory as book2txt.sh.

Of course, it is possible that some features of Yacas documentation were not implemented in the script and in that case the resulting file must be edited by hand. But the purpose of the book2txt script is exactly this: to make a plain text source file to be edited and maintained.

Several files can be converted at once, for example:
book2txt.sh f1.chapt f2.chapt file3.book
Each file is processed by an independent Yacas session. Any errors of processing are printed on the screen.


The Yacas build system


Introduction

This chapter describes the build system of Yacas. So here you will find what happens when you give the configure or the make command, and how to change this. It will concentrate on Unix systems; other architectures are briefly covered in the final section.

As the Yacas build system is built on the GNU autotools suite, which contains both automake and autoconf, we will start with a short description of this package. Then we will turn to the various components of Yacas: the program itself, the script files, the documentation, the test suite, and so on.

As explained in the INSTALL file, building Yacas requires the following steps.

Both configure and make accept many options. Some of them are explained below.


The GNU autotools suite

The GNU autotools suite is a collection of applications to streamline the build system of other programs, like Yacas. Its two main goals are to present a consistent build procedure to users, and to assist developers in tackling portability problems.

The autotools suite consists of a number of utilities. These are developed separately, but are designed to be used together. They are

The users do not need to run automake and autoconf (here, "users" refers to the people who do not want to make any changes to Yacas and includes those who just want to compile Yacas from a tar.gz source archive). They do need the libtool script. But the libtool package is included in the Yacas distribution, so the users do not need to install libtool themselves.

Developers do need to install autoconf and automake on their systems. But they usually do not need to run these tools directly, as the Makefiles contain the necessary commands. When the Makefiles are not present, which occurs for instance when installing afresh from the CVS repository, the makemake script in the root of the Yacas source tree can (and probably should) be used to invoke automake and autoconf in the right order and with the correct flags.

In the following three sections, these utilities are briefly explained. In all cases, the reader is referred to the documentation included in the autotools suite for more information. Another useful source of information is GNU Autoconf, Automake, and Libtool by Gary V. Vaughan, Ben Elliston, Tom Tromey and Ian Lance, which is published by New Riders. An online version is available from http://sources.redhat.com/autobook .


The automake tool

Automake is a tool to generate standard-compliant Makefiles. More precisely, automake uses the information in Makefile.am to produce a Makefile.in file, which will be turned into a Makefile by the configure script generated by the autoconf utility.

The Makefile.am file contains the definition of certain macros that are used by automake. The rest of the Makefile.am file is copied verbatim to the generated Makefile.in file.

The most important macros which are used by automake are the so-called primaries. These list the files that make up the Yacas package. For instance, in the src directory, the file Makefile.am contains the following line
 bin_PROGRAMS = yacas
This is an example of the PROGRAMS primary, and says that the directory contains a program called yacas. Hence it will be built if the make all command is executed, it will be installed at make install, etc. Other useful primaries are SCRIPTS for executable scripts, HEADERS for header files, LIBRARIES for static libraries, LTLIBRARIES for libtool libraries, and DATA for all files which are just copied verbatim at installation time (this includes Yacas scripts).

The bin prefix in the example above says that yacas should be installed in the binary directory, as determined by the configure script. By default, this is the directory /usr/local/bin. There are also prefixes for the other directories, as well as some prefixes with different meanings: the noinst prefix says that the specified file need not be installed, and the check prefix says that the file is only needed when testing.

There are also so-called associated variables. The same Makefile.am contains the following variables associated to the Yacas executable:
yacas_SOURCES = yacasmain.cpp commandline.cpp \
      unixcommandline.cpp stdcommandline.cpp
yacas_LDADD = libyacas.a libyacasplatform.a \
      @NUMBERS_LIB@ @NUMBERS_LDFLAGS@ 
These lines tell that the executable is built from four source files (yacasmain.cpp, commandline.cpp, unixcommandline.cpp and stdcommandline.cpp) and two static libraries (libyacas.a and libyacasplatform.a). The @NUMBERS_LIB@ and @NUMBERS_LDFLAGS@ symbols are defined when the configure script is run, as explained in the next section. They contain the names of additional libraries to link in.

From the information contained in these lines, automake can construct the necessary commands to go in the final Makefile. This Makefile does not only support building, testing, and installing the package, but also rolling the tar-ball for release (use make dist for this, as explained in the section "Targets for make" below). In the above example, automake can figure out that yacasmain.cpp should be included in the distribution.

Unfortunately not everything is supported that well. For instance, Yacas comes with its own documentation system, which is of course not supported by automake. So we need to tell automake how to handle these files. To specify which files should be included in the distribution, the EXTRA_DIST variable can be used. The developer should list all files to be included in the distribution that automake does not know about here. If we want to run some commands at build or installation time, we can specify them by Makefile rules in the traditional ways. Just write the rules in Makefile.am and they will be copied verbatim in the generated Makefile. Please keep in mind that the rules should work on a wide variety of platforms in order to retain portability. In particular,

We currently assume automake version 1.4 or later. Note that version 1.5 breaks backward compatibility and should be avoided. Version 1.6 contains some useful additions, like the nobase prefix and the possibility to define new prefixes, so at a certain point we may require version 1.6.

For more information about automake, the reader is referred to the documentation that comes with the package.


The autoconf tool

Autoconf is a tool to generate portable shell scripts that users can run to configure the package (in our case Yacas) for their system. It reads the file configure.in and produces the configure script. The latter script can be run by the user to prepare for building Yacas.

The configure.in file consists of standard shell code, interspersed with special macros defined by the autoconf package. These can be recognized by the AC_ prefix.

As the configure.in file only rarely needs to be changed, we will only describe the autoconf tool by one example. As explained in the previous section, "The automake tool" , the symbols @NUMBERS_LIB@ and @NUMBERS_LDFLAGS@ are used in the Makefile.in to link a library with basic numerical routines into the Yacas executable. This gives the user the option to choose between two libraries: the GNU multi-precision arithmetic (GNU MP) library and a library provided by the Yacas team.

This effect is achieved by the following fragment of configure.in (for clarity, a simplified version is presented).

AC_ARG_WITH(numlib, [  --with-numlib=LIB ... ],  \
            with_numlib=$withval,                \
            with_numlib="native")
case $with_numlib in
  native) 
    NUMBERS_LIB="libyacasnumbers.la"
    NUMBERS_LDFLAGS="-lm"
  gmp)
    AC_CHECK_LIB(gmp, __gmpz_init, \
                 have_gmp=yes, have_gmp=no)
    if test "$have_gmp" = "no" ; then
      AC_MSG_ERROR([GNU MP library not found])
    fi
    NUMBERS_LIB="libgmpnumbers.la"
    NUMBERS_LDFLAGS="-lgmp -lm"
esac
AC_SUBST(NUMBERS_LIB)
AC_SUBST(NUMBERS_LDFLAGS)

The first line tells the configure script to accept an extra option, --with-numlib=LIB. The shell variable with_numlib is set to the value LIB given by the user, or to native if the user does not specify this option on the command line.

If the shell variable with_numlib has the value native (which means that the user has either given the --with-numlib=native option to configure, or not used the option at all), then the NUMBERS_LIB and NUMBERS_LDFLAGS shell variables are set to libyacasnumbers.la and -lm respectively.

If on the other hand the --with-numlib=gmp option is passed to the configure script, then first it is checked that the GNU MP library is available -- if this is not the case, the configuration terminates with an error. Then the NUMBERS_LIB and NUMBERS_LDFLAGS shell variables are set to suitable values.

The last two lines say that the value of the NUMBERS_LIB and NUMBERS_LDFLAGS shell variables should be substituted for the @NUMBER_LIB@ and @NUMBER_LDFLAGS@ symbols respectively, in the Makefile.in.

This ends the brief description of autoconf. For more information, the reader is referred to the documentation that comes with the package.

We currently assume autoconf version 2.13 or later.


The libtool utility

The libtool utility takes care of all the peculiarities of creating, linking and loading shared and static libraries across a great number of platforms, providing a uniform command line interface to the Yacas developer. The inner workings of libtool are fairly complicated, but fortunately Yacas developers very rarely need to concern themselves with it as libtool is tightly integrated with automake and autoconf. In fact, it should suffice to replace any LIBRARIES primary in Makefile.am with an LTLIBRARIES primary to indicate that libtool libraries should be used, and to keep in mind that the correct suffices are used: .lo for object files and .la for any libraries.

For the people that want to know everything, we will now explain the idea of libtool in a nutshell by a simple example: suppose that we want to build and install a library from the source file hello.c. The following commands do the trick, on any system:

libtool gcc -c hello.c
libtool gcc -rpath /usr/local/lib -o \
  libhello.la hello.lo
libtool cp libtrim.la /usr/local/lib

The first command builds a libtool object with the name hello.lo. This file encapsulates the hello.o object file. Subsequently, the libtool library with the name libhello.la is built, which encapsulates both the static and the shared library. The last command installs both the static and the shared library in the /usr/local/lib directory. Note that the GNU compiler gcc need not be installed on the system; libtool will call the native compiler with the correct switches if necessary.

The libtool distribution includes libltdl, the LibTool Dynamic Loading library. This library is used to support Yacas plugins.


The configure script

The configure script is run by the user to prepare the Yacas package for the build and installation process. It examines the user's system and the options passed to the script by the user, and generates suitable Makefiles. Furthermore, it generates yacas.spec which is used to build a package in Red Hat's .rpm format, and the C header file config.h.

A nice feature is the possibility to build in a different directory than the source directory by simply running configure from that directory. This not only prevents the source directory from being cluttered up by object files and so on, but it also enables the user to build for different architectures in different directories or to have the source in a read-only directory. For example, suppose that the Yacas source is installed under /mnt/cdrom/src/yacas and that you want to build Yacas under /tmp/build-yacas. This is achieved by the following commands
mkdir /tmp/build-yacas
cd /tmp/build-yacas
/mnt/cdrom/src/yacas/configure
make

A list of options accepted by the configure script can be retrieved by invoking it with the help option
./configure --help
The most important is the prefix option, which influences where everything will be installed. The default is /usr/local, meaning that for instance the yacas executable is installed as /usr/local/bin/yacas. To change this in /usr/bin/yacas, invoke the script as follows
./configure --prefix=/usr
Other options to configure enable the user to fine-tune the location where the various files should be installed.

We will not describe the common configure options which are shared by all packages, but will restrict ourselves to the options exclusively used by Yacas.

The following three options pertain to the extensive documentation that comes with Yacas. By default, only HTML documentation is generated.

Then there are three options describing where to install various parts of the Yacas package.

Furthermore, the opposites of the above options (eg. disable-server) are also recognized.


Targets for make

One of the advantages of using the autotool suite to generate Makefiles, is that the resulting Makefiles support a variety of targets. Here is a partial list of them.


The Yacas executable

The main executable is called yacas and resides in the src directory.

Conceptually, the build process consists of the following parts:

At installation time, not only the yacas executable is installed but also the static libraries and the header files. This is also to enable developers to embed Yacas in their own programs.


The Yacas script files

The Yacas script files with extension ys can be found in the scripts/ directory and its "repository" subdirectories.

All script files and .def files are listed in Makefile.am. This definition is used to generate the packages.ys file, which contains a list of all .def files. The corefunctions.ys file is also automatically generated. This is done by running the gencorefunctions program, which resides in the src/ directory.

At installation time, all the script files and .def files that are listed in Makefile.am are copied. During this installation, the directory structure should be preserved.

The check target checks that all the script files and .def files listed in the Makefile.am are indeed present, and vice versa, that all files present in the scripts/ hierarchy are indeed listed.


The documentation

The documentation system for Yacas is explained in Essays on Yacas, Chapter 5, Section 2 .

The documentation is generated from plain text source files with the txt extension in the manmake directory. The source files are converted to Yacas code by the txt2yacasdoc.pl script. The result is a set of Yacas source files containing the text of the documentation. These Yacas source files are not well suited for human reading and must be further processed to obtain the books in HTML and PS/PDF formats.

To generate the books in HTML format, the Yacas source files are processed by the Yacas interpreter using the scripts manualmaker and indexmaker. The script indexmaker creates the file books.html with the top-level index to all HTML documentation books. The script manualmaker creates the HTML files with the book text (both the framed and the non-framed versions are written out). It is also possible to generate more than one HTML book at once. The two reference manuals are generated together to produce the full hyperlinked index to all commands.

To generate the books in PostScript and PDF format, the book2TeX.sh script is used. This script converts the Yacas source files to TeX files. They can then be processed by the standard TeX tools.

By default, only the HTML-formatted documents are generated. To change this, use the options disable-html-doc, enable-ps-doc and enable-pdf-doc in the configure script. The with-html-dir and with-ps-dir options can be used to tell where the documentation should be installed.

At the moment, the Makefile.am for the documentation is rather messy. For this reason, we do not describe it in detail. Instead, we just point out some special features:


The test suite

The Yacas distribution contains a (hopefully comprehensive) test suite, which can be used to check whether Yacas is (still) functioning correctly. The command make check tests the latest compiled version of Yacas. To test the executable that is installed on your system, use the command make installcheck.

The yts files in the tests directory contain the tests for different subsystems of Yacas. For instance, the file arithmetic.yts contains tests for the basic arithmetic functions; the first test is to check that 3+2 evaluates to 5. All the test files are listed in the TESTFILES variable. This variable can be overridden. For example, the command
make TESTFILES='comments.yts complex.yts' clean
only runs the test scripts comments.yts and complex.yts.

The shell script test-yacas does the actual testing. It runs the scripts listed in the TESTFILES variable through the Yacas executable. The tests that take a long time are at the end of the list, so that these tests are performed last. The output of the tests is collected in the file testresult.txt. The test is considered to be failed, if the exit code is nonzero (which happens for instance if Yacas crashes) or if either of the strings ****** (six stars) or Error appear in the output.

A special case is the last test, which goes through the problems put forward by Michael Wester (see the section M. Wester's CAS benchmark and Yacas ). The commands for this tests are not in a yts file in the tests directory, but are extracted from the documentation (see the immediately preceding section ).


The library archive

The library archive is a file, commonly called scripts.dat, which contains all the scripts in compressed form. If Yacas is started with the --archive flag, it uses the contents of this file. This is useful for binary releases, as one needs only two files: the Yacas executable and the library archive.

Support for the library archive resides in the src/ directory. It contains the code for the compressor program. This program generates the library archive.

The archive is only built if the enable-archive option is passed to the configure script. In that case, the compressor program is built and run. At installation time, the generated archive is installed in the library directory (/usr/local/lib by default). There is also a test to check that the generated archive in fact works.


Plugins

The directory plugins contains the files which are necessary to build plugins, which can be dynamically loaded into Yacas. See Essays on Yacas, Chapter 3, Section 3 for details.

Plugins are almost the same as shared libraries, except that they are not linked when the Yacas executable is started, but at the user's request while Yacas is being run. The steps for building plugins are as follows. First we need to run Yacas on the .stub file. Then the generated C++ file is compiled, together with the files which contain the actual code for the functionality to be provided by the plugin. Finally, the resulting object files are linked. We need to add the linker flags -module -avoid-version -no-undefined in order to get a plugin. If the plugin uses any functions from other libraries, these libraries should be specified with the -L option. The plugin is installed in pkglib (which defaults to /usr/local/lib/yacas) when the user gives the make install command.

The following plugins are included with Yacas:


Proteus

The directory proteus contains an experimental implementation of a graphical interface to Yacas.

The executable is called proteusworksheet. It is only built if the build-proteus option was given to the configure script, and if the fltk development libraries are present. If this is the case, the executable is built by linking three libraries from the src directory with the object files in the proteus directory.

The install target installs the executable and some auxiliary data files that Proteus needs to function.


Other components

This section lists the components of the Yacas distributions that have not yet been described.

The build system does not perform any actions for the following components of Yacas.


Non-Unix architectures

Until now, we have only talked about the support for building Yacas on Unix-like systems. Here are pointers to descriptions of the support for other architectures. Some architectures may sometimes be trailing behind.