Compiler test cases or how to test a compiler [closed]

Answer 1

There was an earlier question related to this for C, but it comes down to a carefully written compiler test suite.

As to when compilers get the code wrong, I've hit that often enough in my professional career, thanks. It's happened less and less over time, but I found a bug in MS C++ compilers targeting CLI this week.

Answer 2

There are several compiler test suites out there. We've had some luck using the Plum Hall test suite for a C compiler. It consists of a large set of C code specifically written to test against the language standard. It verifies that the compiler can handle the language syntax and semantics.

Answer 3

The Eiffel compiler is open source and has an extensive library of test cases and internal design contracts.

http://dev.eiffel.com

Answer 4

GCC has a quite large test suite (https://gcc.gnu.org/onlinedocs/gccint/Testsuites.html#Testsuites). It is available on the SCM : https://github.com/gcc-mirror/gcc/tree/master/gcc/testsuite

Answer 5

For the idea to compile a big open source project:

You could take a project that itself has a test suite. Then you compile the project and its test suite and see if the tests pass. To validate these results you compile project and test suite with an other compiler, and run the tests again.

Answer 6

The general practice is to create a large set of small programs that each demonstrate one aspects of the compiler. These will include both program that compile and ones that shouldn't. General the ASM coming out the back end is not checked but rather the program is run and it's output checked. As for how to make sure there are no bugs in the test cases: make them small, as in 5-10 lines each.

These test suites can be very large as in hundreds to thousands of tests (for example: an out of date test suite for the D programming language) and generally include one or more test cases for every bug ever reported.