# Installing GCC: Testing

Before you install GCC, we encourage you to run the testsuites and to compare your results with results from a similar configuration that have been submitted to the gcc-testresults mailing list. This step is optional and may require you to download additional software, but it can give you confidence in your new GCC installation or point out problems before you install and start using your new GCC.

First, you must have downloaded the testsuites. These are part of the full distribution, but if you downloaded the "core" compiler plus any front ends, you must download the testsuites separately.

Second, you must have the testing tools installed. This includes a current version of DejaGnu; dejagnu 1.3 is not sufficient. It also includes Tcl and Expect; the DejaGnu site has links to these.

Now you may need specific preparations:

- The following environment variables may need to be set appropriately, as in the following example (which assumes that DejaGnu has been installed under `/usr/local`):

                     TCL_LIBRARY = /usr/local/share/tcl8.0
                     DEJAGNULIBS = /usr/local/share/dejagnu

  On systems such as Cygwin, these paths are required to be actual paths, not mounts or links; presumably this is due to some lack of portability in the DejaGnu code.

  If the directories where `runtest` and `expect` were installed are in the `PATH`, it should not be necessary to set these environment variables.

Finally, you can run the testsuite (which may take a long time):

              cd objdir; make -k check

The testing process will try to test as many components in the GCC distribution as possible, including the C, C++, Objective-C and Fortran compilers as well as the C++ and Java runtime libraries.

While running the testsuite, DejaGnu might emit messages resembling `WARNING: Couldn't find the global config file.` or `WARNING: Couldn't find tool init file`. These messages are harmless and do not affect the validity of the tests.

### How can I run the test suite on selected tests?

As a first possibility to cut down the number of tests that are run it is possible to use `make check-gcc` or `make check-g++` in the `gcc` subdirectory of the object directory. To further cut down the tests the following is possible:

             make check-gcc RUNTESTFLAGS="execute.exp other-options"

This will run all `gcc` execute tests in the testsuite.

             make check-g++ RUNTESTFLAGS="old-deja.exp=9805* other-options"

This will run the `g++` "old-deja" tests in the testsuite where the filename matches `9805*`.

The `*.exp` files are located in the testsuite directories of the GCC source, the most important ones being `compile.exp`, `execute.exp`, `dg.exp` and `old-deja.exp`. To get a list of the possible `*.exp` files, pipe the output of `make check` into a file and look at the `Running ... .exp` lines.

To run only the tests for a library, run `make check` from the the library's testsuite in a subdirectory of the object directory: `libstdc++-v3/testsuite` or `libcgj/testsuite`.

### Additional testing for Java Class Libraries

The Mauve Project provides a suite of tests for the Java Class Libraries. This suite can be run as part of libgcj testing by specifying the location of the Mauve tree when invoking `make`, as in `make MAUVEDIR=~/mauve check`.

### How to interpret test results

After the testsuite has run you'll find various `*.sum` and `*.log` files in the testsuite subdirectories. The `*.log` files contain a detailed log of the compiler invocations and the corresponding results, the `*.sum` files summarize the results. These summaries list all the tests that have been run with a corresponding status code:

- PASS: the test passed as expected
- XPASS: the test unexpectedly passed
- FAIL: the test unexpectedly failed
- XFAIL: the test failed as expected
- UNSUPPORTED: the test is not supported on this platform
- ERROR: the testsuite detected an error
- WARNING: the testsuite detected a possible problem

It is normal for some tests to report unexpected failures. At the current time our testing harness does not allow fine grained control over whether or not a test is expected to fail. We expect to fix this problem in future releases.

### Submitting test results

If you want to report the results to the GCC project, use the `contrib/test_summary` shell script. Start it in the objdir with

             srcdir/contrib/test_summary -p your_commentary.txt \
                 -m gcc-testresults@gcc.gnu.org |sh

This script uses the `Mail` program to send the results, so make sure it is in your `PATH`. The file `your_commentary.txt` is prepended to the testsuite summary and should contain any special remarks you have on your results or your build environment. Please do not edit the testsuite result block or the subject line, as these messages are automatically parsed and presented at the GCC testresults web page. Here you can also gather information on how specific tests behave on different platforms and compare them with your results. A few failing testcases are possible even on released versions and you should look here first if you think your results are unreasonable.

Return to the GCC Installation page