Maven build with AnnotationProcessor that parses files in src/main/java and generates sources to generated-test-sources/test-annotations

Question

I'm using maven 3.0.3 and Java7.

I've got an AnnotationProcessor that is supposed to parse annotated java-files in src/main/java (not src/test/java) and generate Helper-Classes for JUnit-Tests. These Helper-Classes are supposed to be stored in target/generated-test-sources/test-annotations because they use libraries that are only available in test-scope. (Note: everything works fine as long as this dependency isn't in test-scope, but the build fails as soon as it is. It's definitely only needed in test-scope / during unit-tests and compilation of test-classes.)

I tried several configurations without any luck:

1. I configured the maven-compiler-plugin to use the AnnotationProcessor during compile:compile. The generated HelperClass would be stored in generated-sources/annotations. Not in generated-test-sources/test-annotations as desired. The result was, that Test-Scoped dependencies would not have been used. Build failed due to Compilation Error "cannot find symbol". Fail

2. I used the above configuration and redefined the generatedSourcesDirectory:

   <generatedSourcesDirectory>
      ${project.build.directory}/generated-test-sources/test-annotations
   </generatedSourcesDirectory>
   

   The generated class would be stored in generated-test-sources/test-annotations as expected, but the build still failed because it tried to compile that file as above and missed the test-scoped dependencies. Fail

3. I tried to use the above configuration and excluded **/generated-test-sources/test-annotations/**/*.java in order to prevent the compiler from compiling in this phase:

   <excludes>
      <exclude>**/generated-test-sources/test-annotations/**/*.java</exclude>
   </excludes>
   

   no luck. Same Compiler-Error as above. Fail

4. I configured the maven-compiler-plugin to use the AnnotationProcessor during test-compile:testCompile. The HelperClass might theoretically have been generated in generated-test-sources/test-annotations, but the AnnotationProcessor wouldn't stumble upon the annotated class that is located in src/main/java, not in src/test/java, which is AFAIK the compilation-scope during test-compile:testCompile. So the Annotated class would not be found, the HelperClass would not be generated and could therefore not be stored in generated-test-sources. Fail

5. Tried to run that during compile:testCompile and test-compile:compile, which lead in both cases to classes of src/main/java not yet being compiled - thus compiler error. Fail

What I really would like to do is:

1. Configure the compiler to use the AnnotationProcessor during compile:compile to generate my HelperClass to ${project.build.directory}/generated-test-sources/test-annotations but don't let maven compile it
2. Then compile the HelperClass during test-compile:testCompile.

I'm failing to do so. I'm not sure if I am missing important maven basics (concept) here, or if there's a problem with the exclusion configuration, or whatever it is.

Any help is very appreciated. Thanks in advance.

Answer 1

I performed several tests with includes, excludes, testExcludes and so on (which I find rather badly documented). It seems that maven simply ignores these configuration settings. Can't really believe it, but though it showed that includes and excludes have been configured correctly and these configurations where used (mvn -X clean install), it still compiled excluded files or didn't compile included files. (BTW: I tested it from CLI as well, as I found out that some IDEs still ignore those settings.) Any solutions found out there, like adding resource directories, including, excluding, defining generatedTestSourcesDirectory during test-compile:testCompile to match the generatedSourcesDirectory of compile:compile simply didn't work. Whatever.

I found a way to solve the problem as follows:

1. let the first compile-step (compile:compile) only use Annotation-Processors, but don't compile generated classes: <proc>only</proc>. Define the generatedSourcesDirectory to be where test-sources should be generated to: ${project.build.directory}/generated-test-sources/test-annotations.
2. use build-helper-maven-plugin to add test-source-directory in the correct phase and goal. The implicit test-compile:testCompile would then be executed with the generated classes sourcepath added to the normal sourcepath.

Below is my configuration. Note that I need to generate other stuff before my actual problem occurs and that has to be generated into generated-sources/annotations, so there's one compile-step more than needed for the solution.

So, this is it:

<build>
    <plugins>
        <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-compiler-plugin</artifactId>
            <version>3.0</version>
            <configuration>
                <source>1.7</source>
                <target>1.7</target>
                <encoding>${project.build.sourceEncoding}</encoding>
            </configuration>
            <executions>
                <execution>
                    <!-- normal compile and generation of other classes to standard location (implicit, you shouldn't need that) -->
                    <id>default-compile</id>
                    <goals>
                        <goal>compile</goal>
                    </goals>
                </execution>
                <execution>
                    <!-- Generates Test-Helper-Class into ${project.build.directory}/generated-test-sources/test-annotations WITHOUT compiling it -->
                    <id>compile-TestHelperClass</id>
                    <goals>
                        <goal>compile</goal>
                    </goals>
                    <configuration>
                        <annotationProcessors>
                            <annotationProcessor>org.my.UnitTestGenerationProcessor</annotationProcessor>
                        </annotationProcessors>
                        <generatedSourcesDirectory>${project.build.directory}/generated-test-sources/test-annotations</generatedSourcesDirectory>
                        <!-- generated class depends on test-scope libs, so don't compile now: proc:only DISABLES compilation of generated classes-->
                        <proc>only</proc>
                    </configuration>
                </execution>
                <!-- implicit test-compile:testCompile -->
            </executions>
        </plugin>
        <plugin>
            <!-- adds source-dir during generate-test-sources:add-test-source 
                 so that the path to our generated class is now known to the 
                 compiler during test-compile:testCompile -->
            <groupId>org.codehaus.mojo</groupId>
            <artifactId>build-helper-maven-plugin</artifactId>
            <version>1.8</version>
            <executions>
                <execution>
                    <id>add-test-source</id>
                    <phase>generate-test-sources</phase>
                    <goals>
                        <goal>add-test-source</goal>
                    </goals>
                    <configuration>
                        <sources>
                            <source>${project.build.directory}/generated-test-sources/test-annotations</source>
                        </sources>
                    </configuration>
                </execution> 
            </executions>
        </plugin>
    </plugins>
</build>

Answer 2

To be perfectly honest, maven-compiler-plugin generally sucks. I use a third party plugin that seems to always work correctly for me. I just use <proc>none</proc> for maven-compiler-plugin, then:

     <plugin>
        <groupId>org.bsc.maven</groupId>
        <artifactId>maven-processor-plugin</artifactId>
        <executions>
          <execution>
            <id>annogen</id>
            <phase>generate-sources</phase>
            <goals>
              <goal>process</goal>
            </goals>
            <inherited>false</inherited>
            <configuration>
              <processors>
                <processor>
                  xapi.dev.processor.AnnotationMirrorProcessor
                </processor>
              </processors>
              <outputDirectory>${project.build.directory}/generated-sources/annotations</outputDirectory>
              <appendSourceArtifacts>true</appendSourceArtifacts>
              <additionalSourceDirectories>
                <additionalSourceDirectory>
                  ${project.basedir}/../api/src/main/java
                </additionalSourceDirectory>
              </additionalSourceDirectories>
            </configuration>
          </execution>
        </executions>
      </plugin>

Answer 3

You may want to check out this blog post: http://deors.wordpress.com/2011/10/08/annotation-processors/

It looks like you need to set the following in pom.xml

build>plugins>plugin>configuration>compilerArgument>-proc:none

I'm not sure if this applies to your particular situation, but I was working with annotations yesterday, and thought this may help you.

It looks like this argument will let you compile without any annotation processors. Then you can compile with them at a later point in the life cycle.

Answer 4

I have a similar problem, and tried the suggested solution (two executions of maven-compiler-plugin, the first time with proc:none and the second time with proc:only) but ran into the following issues:

1) Running mvn clean install : both executions (proc:none and proc:only) will compile the source code and generate class files, before running the annotation processor:

[INFO] --- maven-compiler-plugin:3.1:compile (default-compile) @ foo ---
[INFO] Changes detected - recompiling the module!
[INFO] Compiling 6 source files to <mumble>\foo\target\classes
[INFO] 
[INFO] --- maven-compiler-plugin:3.1:compile (process-annotations) @ foo ---
[INFO] Changes detected - recompiling the module!
[INFO] Compiling 6 source files to <mumble>\foo\target\classes
[MyAnnotationProcessor] processing ...

2) Running mvn install after a previous install: both executions (proc:none and proc:only) determine that the class files are up to date and do nothing, and the annotation processor is not run:

[INFO] --- maven-compiler-plugin:3.1:compile (default-compile) @ foo ---
[INFO] Nothing to compile - all classes are up to date
[INFO] 
[INFO] --- maven-compiler-plugin:3.1:compile (process-annotations) @ foo ---
[INFO] Nothing to compile - all classes are up to date

So in the first case, the java files are compiled twice (and one of my modules has thousands of files, and this affects build time significantly). And in the second case, nothing happens at all because the class files already exist. This is using version 3.1 and 3.0 of maven-compiler-plugin.

For maven-compiler-plugin version 2.5.1, the second execution (proc:only) would always determine that class files were up to date and not run the annotation processor, whether I was running mvn clean install or mvn install, so the second execution was pointless.

The plugin is configured as follows:

<plugin>
  <groupId>org.apache.maven.plugins</groupId>
  <artifactId>maven-compiler-plugin</artifactId>
  <version>3.1</version>
  <configuration>
    <source>1.7</source>
    <target>1.7</target>
  </configuration>
  <executions>
    <execution>
      <id>default-compile</id>
      <configuration>
        <proc>none</proc>
      </configuration>
    </execution>
    <execution>
      <id>process-annotations</id>
      <goals><goal>compile</goal></goals>
      <configuration>
        <proc>only</proc>
        <annotationProcessors>
          <annotationProcessor>MyAnnotationProcessor</annotationProcessor>
        </annotationProcessors>
      </configuration>
    </execution>
  </plugin>

So proc:only option seems to compile the source code anyway in version 3.0 and 3.1 (making it the same as proc:both, which affects performance) or take no action in version 2.5.1 (making it the same as proc:none, which affects functionality).

Not sure how to get around this problem ...

Edit #1

The verbose output includes the following:

[INFO] --- maven-compiler-plugin:3.1:compile (process-annotations) @ foo ---
[DEBUG] Configuring mojo 'org.apache.maven.plugins:maven-compiler-plugin:3.1:compile' with basic configurator -->
[DEBUG]   (f) annotationProcessors = [MyAnnotationProcessor]
[DEBUG]   (f) basedir = <mumble>\foo
[DEBUG]   (f) buildDirectory = <mumble>\foo\target
[DEBUG]   (f) classpathElements = [<mumble>\foo\target\classes, ...]
[DEBUG]   (f) compileSourceRoots = [<mumble>\foo\src\main\java]
[DEBUG]   (f) compilerId = javac
[DEBUG]   (f) debug = true
[DEBUG]   (f) failOnError = true
[DEBUG]   (f) forceJavacCompilerUse = false
[DEBUG]   (f) fork = false
[DEBUG]   (f) generatedSourcesDirectory = <mumble>\foo\target\generated-sources\annotations
[DEBUG]   (f) mojoExecution = org.apache.maven.plugins:maven-compiler-plugin:3.1:compile {execution: process-annotations}
[DEBUG]   (f) optimize = false
[DEBUG]   (f) outputDirectory = <mumble>\foo\target\classes
[DEBUG]   (f) proc = only
[DEBUG]   (f) projectArtifact = foo:jar:1.0-SNAPSHOT
[DEBUG]   (f) showDeprecation = false
[DEBUG]   (f) showWarnings = false
[DEBUG]   (f) skipMultiThreadWarning = false
[DEBUG]   (f) source = 1.7
[DEBUG]   (f) staleMillis = 0
[DEBUG]   (f) target = 1.7
[DEBUG]   (f) useIncrementalCompilation = true
[DEBUG]   (f) verbose = true
[DEBUG]   (f) mavenSession = org.apache.maven.execution.MavenSession@6fb65730
[DEBUG]   (f) session = org.apache.maven.execution.MavenSession@6fb65730
[DEBUG] -- end configuration --
[DEBUG] Using compiler 'javac'.
[DEBUG] Source directories: [<mumble>\foo\src\main\java]
[DEBUG] Classpath: [<mumble>\foo\target\classes ... ]
[DEBUG] Output directory: <mumble>\foo\target\classes
[DEBUG] CompilerReuseStrategy: reuseCreated
[DEBUG] useIncrementalCompilation enabled
[INFO] Changes detected - recompiling the module!
[DEBUG] Classpath:
[DEBUG]  <mumble>\foo\target\classes ...
[DEBUG] Source roots:
[DEBUG]  <mumble>\foo\src\main\java
[DEBUG] Command line options:
[DEBUG] -d <mumble>\foo\target\classes
    -classpath <mumble>\foo\target\classes;...;
    -sourcepath <mumble>\foo\src\main\java;
    -s <mumble>\foo\target\generated-sources\annotations
    -proc:only
    -processor MyAnnotationProcessor
    -g -verbose -nowarn -target 1.7 -source 1.7
[DEBUG] incrementalBuildHelper#beforeRebuildExecution
[INFO] Compiling 6 source files to <mumble>\foo\target\classes
[parsing started RegularFileObject[<mumble>\foo\src\main\java\Foo.java]]
[parsing completed 0ms]
...
[search path for source files: <mumble>\foo\src\main\java]
[MyAnnotationProcessor] processing ...

Even though there's a proc:only in the compiler options there's still a [parsing started RegularFileObject] entry ... maybe it's just logging that but actually loads the class file?