Custom plugin not getting detected in EAR with log4j2 API

Question

I am migrating an EAR application from log4j to log4j2. I had classes extending appenders, filters, layouts in different jars of EAR and now, I have converted those to plugins. This means I have custom plugins in more than one jar (assume 3 jars).

I am not using packages attribute in log4j2.xml and am initializing the logging system by using Dlog4j.configurationFile JVM argument pointing to log4j2.xml location in META-INF of EAR.

Adding the below plugin in all the three jar projects did not work .

<plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-compiler-plugin</artifactId>
            <executions>
                <execution>
                    <id>log4j-plugin-processor</id>
                    <goals>
                        <goal>compile</goal>
                    </goals>
                    <phase>process-classes</phase>
                    <configuration>
                        <proc>only</proc>
                        <annotationProcessors>
                            <annotationProcessor>org.apache.logging.log4j.core.config.plugins.processor.PluginProcessor</annotationProcessor>
                        </annotationProcessors>
                    </configuration>
                </execution>
            </executions>
        </plugin>

Pattern layout: In the below pattern layout, e is the custom pattern where a custom pattern converter plugin is written to convert this string.

<Pattern>%d %-5p [%c{1}] [EventId: %e] [%t] %m%n</Pattern>

Custom converter plugin for the above pattern layout (in jar1):

jar1 has Log4J2Plugins.dat file under META-INF in org.apache.. folder.

@Plugin(name = "EventPatternConverter", category = "Converter")
@ConverterKeys({"e"})

public class EventPatternConverter extends LogEventPatternConverter {

protected EventPatternConverter(String name, String style) {
    super(name, style);
}

public static EventPatternConverter newInstance(String[] options) {
  return new EventPatternConverter("e", "e");
}

@Override
public void format(LogEvent event, StringBuilder toAppendTo) {      
    String eventId= "";
    // Append empty string (OR) value        
    toAppendTo.append(eventId);
  }
}

But, I am getting the below error

ERROR Unrecognized format specifier [e]

Even, none of the custom plugins are identified as I am getting invalid element for rest of the custom plugins which are all available in jar2, jar3 and they all have Log4J2Plugins.dat file.

ERROR File contains an invalid element or attribute "TestFilter"

I am using log4j-api-2.4.jar, log4j-core-2.4.jar, log4j-jcl-2.4.jar, log4j-web-2.4.1.jar, commons-logging-1.1.1.jar jars in the EAR.

I have defined a custom pattern converter plugin and expecting this converter gets applied to all pattern layout including default pattern layout defined using <patternlayout>. Is this right ?

If yes, the please help if anyone faced this issue and guide me if I am wrong in defining the custom plugin as they are all not getting detected from jars in EAR.

Answer 1

When compiling a custom Plugin, the Log4J pom.xml defines a plugin that automatically generates cache data in the file META-INF/org/apache/logging/log4j/core/config/plugins/Log4j2Plugins.dat You can see this under your target/classes in a Maven project.

The log4j-core-2.x.x.jar also contains a Log4j2Plugins.dat defining its cache data.

The problem is a single JAR is created when testing an EAR using ShrinkWrap and normally the log4j-core-2.x.x.jar Log4j2Plugins.dat is added to the test JAR as it would most likely be first in the class path.

This means your custom plugin cache is missing.

The solution using ShrinkWrap is to create a new Log4j2Plugins.dat merging any required custom plugin cache files with the cores and then adding that to the JAR.

The following function achieves that...

private static void mergeLog4J2Log4j2PluginsFile(JavaArchive ja, Class... uniqueJARClasses) {
  // @Author: Johnathan Ingram <jingram@rogueware.org>
  // Log4J2 uses /META-INF/org/apache/logging/log4j/core/config/plugins/Log4j2Plugins.dat within a JAR to define custom plugins
  //        This is automatically generated by the plugin defined in the log4j-core-2.x.x pom.xml when compiling your custom plugin
  //        The problem with shrinkwrap is that the JAR is not preserved and only a single 
  //        /META-INF/org/apache/logging/log4j/core/config/plugins/Log4j2Plugins.dat
  //        file can exist as JAR files cannot be added to a JAR file as a library.
  //        This is normally the default contained in log4j-core-2.x.x.jar which does not expose any custom plugins
  //        To rectify, both the core and the custom plugin JAR file Log4j2Plugins.dat need to be merged into a single Log4j2Plugins.dat
  try {
     // List of a unique class in each JAR containing a Log4j2Plugins.dat requiring merging
     Vector<URL> datUrls = new Vector<URL>();
     for (Class klass : uniqueJARClasses) {
        // Find  the JAR the class belongs to
        URL classLoc = klass.getProtectionDomain().getCodeSource().getLocation();
        URL resourceURL = classLoc.toString().endsWith(".jar")
                ? new URL("jar:" + URLDecoder.decode(classLoc.toString(), "UTF-8") + "!/META-INF/org/apache/logging/log4j/core/config/plugins/Log4j2Plugins.dat")
                : new URL(URLDecoder.decode(classLoc.toString(), "UTF-8") + "/META-INF/org/apache/logging/log4j/core/config/plugins/Log4j2Plugins.dat");
        datUrls.add(resourceURL);
     }

     // Use the Log4J2 PluginCache to build a merged Log4j2Plugins.dat
     File mergedDatFile = new File("target/Log4j2Plugins.dat");
     try (FileOutputStream fo = new FileOutputStream(mergedDatFile)) {
        org.apache.logging.log4j.core.config.plugins.processor.PluginCache pc = new org.apache.logging.log4j.core.config.plugins.processor.PluginCache();
        pc.loadCacheFiles(datUrls.elements());
        pc.writeCache(fo);
     }

     // Replace the default Log4j2Plugins.dat if present
     ja.delete("/META-INF/org/apache/logging/log4j/core/config/plugins/Log4j2Plugins.dat");
     ja.addAsManifestResource(mergedDatFile, "org/apache/logging/log4j/core/config/plugins/Log4j2Plugins.dat");

  } catch (Exception ex) {
     ex.printStackTrace(System.err);
  }
}

To run using your example:

JavaArchive ja = ShrinkWrap.create(JavaArchive.class, "my-test.jar");
...
mergeLog4J2Log4j2PluginsFile(ja, org.apache.logging.log4j.core.config.plugins.processor.PluginCache.class, EventPatternConverter.class);