The install-tests script installs the L8TF testing regime into a source directory, SUBJECT. This creates a subdirectory, ALLTESTS, with the files necessary for running tests, and an autoconf hook file for configure.ac to include so that the eventual configure script run will create within the build directory a test run directory with suitable make targets.

The two most common invocations will be

1. to modify a developer environment for SUBJECT to include tests:

     cd SUBJECT
     ./path/to/L8TF/install-tests
   

   or

2. to add tests to DISTRIB, a source distribution under construction. In this case, scenario data and additional sources need to be copied in so as not to require that L8TF be available on the target host, hence the need for an extra option:

     cd DISTRIB
     ./path/to/L8TF/install-tests --distcopy
   

Subsequent runs of install-tests without arguments on the same subject directory update the test files as needed if the L8TF sources have changed.

• General options
  • --(no-)verbose(=level) (-v)
• Informational modes
  • --help (-?, -h)
  • --version (-V)
  • --about
  • --args
  • --dry-run (-n)
• Directory path options
  • --subject=path
  • --l8tf-root=path
  • --alltests=path
  • --distcopy(=path) vs. --no-distcopy
  • --run=path
• Subject-specific issues
  • --subject-test-ac=path
  • --subject-marker=filename
  • --specs=path
  • --unit-sources=path
  • --unit-prefix=string
• Final actions
  • --autoconf vs. --no-autoconf
  • --configure-tests vs. --no-configure-tests
• Update options
  • --clobber vs. --no-clobber
  • --diff(=flags) vs. --no-diff
  • --recreate vs. --no-recreate
• Initialization files
  • --no-init
  • --site-init vs. --no-site-init
  • --subject-init vs. --no-subject-init
  • --prev-init vs. --no-prev-init
  • --user-init(=path) vs. --no-user-init

Options that are nearly always applicable:

shows additional information, as appropriate. Higher level numbers may give you more. Or not, depending.

The default level is 0 (same as --no-verbose). -v or --verbose gives you level 1. Use --verbose=LEVEL for any higher level.

This option is ignored in startup files.

These all run to display some information without actually doing anything:

shows help text and exits.

This option is ignored in startup files.

shows version information and exits.

This option is ignored in startup files.

shows the "about" box and exits. This is equivalent to --version --verbose=MAXIMUM, with biographies of the developers and everything (well okay, maybe not that much).

This option is ignored in startup files.

displays all options currently in effect and then exits immediately (unless --dry-run is also given), showing how the various unmentioned options are being resolved/defaulted.

This option is ignored in startup files.

does as much validation of the given command line as possible, including checking that it is possible to create necessary new directories and files, without actually making changes (aside from directory access/modification times, which will inevitably get bumped). This can be combined with --args.

This option is ignored in startup files.

These specify the various directories that install-test creates or needs to reference.

path to SUBJECT, the top-level/root directory of the subject source tree, either a repository root or a distribution — whether under construction or already unpacked somewhere, — this being the source directory in need of a test suite or updates thereto.

This defaults to the current directory, i.e., one is normally expected to have cd'ed there before running this script, but this allows running install-tests from some other directory.

Providing an explicit --subject=SUBJECT effectively first does cd SUBJECT, then adjusts as needed any provided --l8tf-root — the only other option for which the current directory can be relevant, — and then continues from there.

Since install-tests does rather a lot of rm -rf and mv -f and thus can potentially do significant damage if run in the wrong directory, there are precautions:

• SUBJECT must have a configure.ac file at the top level, which, in turn
  • must have an AC_CONFIG_SRCDIR directive that
    • names a marker file that actually exists (the configure script would fail otherwise), and
    • the marker file name, see --subject-marker, must be the one we are expecting;
  • must reference the .ac file that install-tests has been specifed to create; see --subject-test-ac,

install-tests will abort if any of these conditions fails.

This option is ignored in startup files.

path to L8TF, the top-level/root testing-sources directory — whether a repository or an unpacked stand-alone distribution thereof — that this installation/update is to be done from. Relative paths are taken as being from the current directory.

By default, L8TF is inferred from the execution path of install-tests itself. That is, since this script is supposed to be located at the top-level of some viable L8TF instance, the simple fact of it having been found and successfully executed ought to validate that directory as indeed being a proper L8TF repository/source directory.

Explicit use of this option should thus only be necessary if

• inferring L8TF from the execution path does not work for some reason (e.g., stupid symbolic link games, or the script is somehow being run with $0 obfuscated), or

• you want to use this install-tests script, but have it pretend to be living in some other testing-sources directory, at which point we have no choice but to assume you know what you are doing, even if this is not likely to end well.

install-tests checks whether particular files that this version of the script depends on exist in the purported L8TF tree and, to protect against unexpected $0-obfuscation or typos in the --l8tf-root argument, aborts if they are not found.

This option is ignored in startup files.

path to ALLTESTS, the directory where (pre-target-platform-configure versions of) test suite shell scripts, the test run directory template, and the runall script for testing multiple configurations at once are to be installed.

This pathname is absolute or relative to SUBJECT. It is created if it does not yet exist.

Default value is tests.

Having ALLTESTS be located ouside of the SUBJECT tree could work, but could also cause confusion since its content is intended to be specific to a given subject source tree.

install-tests will abort if the specified directory already exists and does not contain the marker file indicating it was previously created by install-tests on this subjest.

An existing directory with the requisite marker file will, by default, have its files overwritten each time install-tests runs (meaning if you edit files in this directory you should not expect your edits to survive. To change how a given test works, edit its source (in L8TF) and let the makefiles do their thing.)

But see also --recreate and --no-clobber for alternative behaviors.

whether to install/update, within SUBJECT or ALLTESTS, copies of those testing source auxiliary and scenario data files needed for running tests, optionally specifying where to do so.

This option is for preparing a distribution to be unpacked on a host where the L8TF repository is not normally expected to be available, so we create a skeletal copy of L8TF.

--distcopy by itself indicates that the default location within ALLTESTS is good enough.

To specify a location explicitly, use --distcopy=path, with pathname relative to SUBJECT.

Warning 

Specifying a location that is within ALLTESTS but different from the default is not recommended since you may inadvertantly overwrite something that matters, and even if you are sure that will not be the case for the current version of L8TF, future versions may change the layout of ALLTESTS.

The default, --no-distcopy, indicates that no such copies are needed and that test scripts will reference L8TF itself directly, which is likely what you want for a development environment.

When appearing in a startup file, --distcopy= will only set the default location, leaving the default behavior of not creating copies in place.

specifies the test run directory, i.e., the subdirectory of the build directory where the test suites will be run, where individual test-run logs and error-output files will show up (see GNU autotest docs for everything this gets used for).

This path must be relative and strictly downward, i.e., the test run directory must have the root build directory as an ancestor (due to VPATH builds, the build directory can be anywhere). There is also unlikely to be any point to giving it multiple components (i.e., making it a grandchild or deeper).

The default test run directory name is t.

In order that this framework can actually be used on a particular subject tree, there are a number of subject-specific matters that need to be specified, having to do with

• content of SUBJECT/configure.ac (which we use to recognize a subject directory and which needs to include a hook file that install-tests can write),

• unit test source files, which will inherently be subject-specific, need to be findable and distinguishable from the regular source files,

• configuration nomenclature, i.e., how we translate between lists of configure arguments and the configuration names to be used by runall

• definitions for the test schedule keywords (smoke, etc...)

  (this list will likely get expanded)

Currently, these all default to what they need to be for the LambdaMOO C server source, though the --subject-init file provides a means for subject authors to have their own defaults.

name for a .ac file that is m4_[s]include()ed by SUBJECT/configure.ac, preferably immediately before the final AC_OUTPUT directive, in order to define how to build and populate the test run directory

This defaults to testing.ac

install-tests writes this file but will abort if the file already exists and does not contain the magic string indicating it was written by a previous run of install-tests.

See also --no-clobber.

install-tests will also abort if SUBJECT/configure.ac does not appear to reference this file at all (an autoconf --trace is performed).

Name of marker file expected to be present in the subject directory as named by the AC_CONFIG_SRCDIR declaration in SUBJECT/configure.ac.

install-tests will abort if the marker filename declared is not the one we were expecting.

The default here is Minimal.db which identifies the LambdaMOO C language server source.

path to where the subject specification files can be found. The default is @R/ssmoo.

This directory is expected to include

cfname.m4	the configuration naming scheme
schedules.sh	available schedules for runall

The pathname must begin with either @R or @S, according as you want the remaining path to be relative to the L8TF root or SUBJECT. In the former case this directory will be included in any --distcopy.

path to the directory where the unit test sources can be found. This defaults to the --specs directory and likewise must begin with either @R or @S, interpreted in the same way.

The distinctive prefix, defaulting to uT-, that is shared by all unit test executables.

(Requiring this makes it easier to write the make rules for unit tests.)

Once the various files have been created, there are two optional final actions that may be taken.

whether or not to run autoconf to regenerate SUBJECT/configure. The default is to do this, and this happens after all other file creation/updating.

Use --no-autoconf to skip this step, though there is hardly ever any reason to.

(Note that this has nothing to do with the ALLTESTS/configure script which is always created/updated by install-tests. See --configure-tests for that)

whether or not to run ALLTESTS/configure.

The configure script in ALLTESTS, the one that chooses testing tools and creates/updates the test run directory templates, normally does not need to run very often. Typically this only matters if available system tools or test tool requirements have changed (e.g., an alternative netcat becomes available after an OS update or a new test requires a completely new tool).

This is the only script install-tests invokes that depends on the target environment; the ability to have install-tests run this script is mainly a convenience.

Both SUBJECT/configure and ALLTESTS/runall already ensure that ALLTESTS/configure has been run at all, but there is no way to know whether it needs to be run again without actually running it. However, invoking it on every individual test suite run may be rather expensive.

Running it on install-test updates is infrequent enough to be the Right Thing, at least if this is a development host.

Running it on a distribution under construction is very much not the Right Thing (configure scripts exist to determine the characteristics of the target/build host, so packaging the results of a configure run on the development host makes no sense.)

Therefore, the default behavior is to run ALLTESTS/configure if and only if this is a --no-distcopy run. If that is not what you want, that's what this option is for.

Also, you can always manually cd ALLTESTS; ./configure whenever you like.

Re-running install-tests updates the files it created the first time. These options control how updates work.

The default behavior (--clobber) is that files highly likely to have been created by prior runs of install-tests will be silently overwritten as needed. Here, "highly likely" refers to

• files under ALLTESTS if that directory has the expected marker, or
• the --subject-test-ac file and the --prev-init startup file if they contain the requisite magic strings.

If you consider this to be insufficiently paranoid, --no-clobber will, if there exist files needing to be changed, cause install-tests to abort and list those files, at which point you can decide what you care about, then remove or otherwise shuffle files manually as you see fit, or just reissue the command without --no-clobber

(or with --clobber, if you made --no-clobber the default by putting it in your user startup file, see --user-init=)

Note that --no-clobber will not prevent new files from being created.

See also --dry-run.

whether to show differences in files being updated and, if so, which flags to use.

Tip 

The --diff=FLAGS argument needs to be read as a single word but then the FLAGS portion will get path-expanded and whitespace-separated. Make sure you understand shell quoting.

The default behavior of install-tests in the case of ALLTESTS or the directory created by --distcopy, where either or both of those directories previously existed, is to update in place those files it knows about and ignore everything else. Theoretically, this should suffice since test scripts should not be referencing any other files.

But there are remotely possible situations where simply having unexpected files exist (e.g., obsolete files left behind by prior versions of L8TF or random files that you yourself added for reasons of your own) causes unexpected behavior.

With --recreate, any previously created ALLTESTS directory (i.e., that has the requisite marker file) is entirely removed and a new one is built from scratch.

--recreate overrides --no-clobber for files within ALLTESTS, but the latter still applies to other files that can be modified (i.e., specifying both soulc still make sense in some situations).

This is about the files that install-tests reads at startup.

The content of each of these files is a sequence of command line arguments for install-tests, but newlines can be arbitrarily added to make the file more readable by putting each option on its own line. Also, a # appearing either at the beginning of a line or preceded by whitespace is treated as the start of a comment and everything from there to the end of the line is ignored.

1. L8TF/.install-tests.site is for package maintainers, if, say, an OS distribution has rules about where user personal preference files should go.

2. SUBJECT/.install-tests.conf is for subject authors to configure/customize aspects of how tests are installed on this particular subject.

3. SUBJECT/.install-tests.prev records the "sticky" arguments used for the most recent successful installation of tests on this subject (which then become the default for updates)

4. the user preferences file, ~/.install-tests.rc usually, but this can be changed by --user-init= in one of the previous files.

Certain options behave differently when they appear in a startup file.

--subject, --l8tf-root and any options that change the mode install-tests runs in (e.g., --help, --version, or --dry-run) are ignored in startup files.

--distcopy= and --user-init= in a startup file will only specify a default name for the directory or file to use/create; whether the corresponding action happens depends on the command line only.

The following options customize how initialization works:

disables all reading of startup files. This is equivalent to specifying all of --no-site-init, --no-subject-init, --no-prev-init, and --no-user-init

Any of the corresponding non-negated options can appear later in the command line to restore reading of particular startup files as desired.

This option is ignored in startup files.

whether to read L8TF/.install-tests.site on startup.

This option is ignored in startup files.

whether to read SUBJECT/.install-tests.conf on startup.

This option is ignored in startup files.

whether to read SUBJECT/.install-tests.prev on startup.

This option is ignored in startup files.

whether to read the user preferences file on startup.

To explicitly specify the location of this file, use --user-init=path, where the pathname is relative to the user's home directory.

When appearing in a startup file, --user-init= will only set the default location; any --no-user-init on the command line will take precedence.

The default location is ~/.install-tests.rc unless this is modified elsewhere (e.g., in L8TF/.install-tests.site).