aws-lambda-java-log4j2 1.6.3 regression breaks logging

[brandondahler]

Problem

Log4j2 logging no longer works after upgrading to com.amazonaws:aws-lambda-java-log4j2:1.6.3.

Log4j prints the error ERROR Error processing element Lambda ([Appenders: null]): CLASS_NOT_FOUND to stderr (which is not logged by lambda).

Reproduction Steps

1. Create a new maven project as described below
2. Execute the code via mvn package exec:java -Dexec.mainClass="bug.reproduction.App"

Reproduction Files

./pom.xml

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
  <modelVersion>4.0.0</modelVersion>

  <groupId>bug.reproduction</groupId>
  <artifactId>aws-lambda-java-log4j2</artifactId>
  <version>1.0-SNAPSHOT</version>

  <name>aws-lambda-java-log4j2</name>

  <properties>
    <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
    <maven.compiler.release>17</maven.compiler.release>
  </properties>

  <dependencies>
    <dependency>
      <groupId>com.amazonaws</groupId>
      <artifactId>aws-lambda-java-log4j2</artifactId>
      <version>1.6.3</version> <!-- 1.6.2 works -->
    </dependency>
    <dependency>
      <groupId>org.apache.logging.log4j</groupId>
      <artifactId>log4j-core</artifactId>
      <version>2.25.4</version>
    </dependency>
    <dependency>
      <groupId>org.apache.logging.log4j</groupId>
      <artifactId>log4j-api</artifactId>
      <version>2.25.4</version>
    </dependency>
    <dependency>
      <groupId>org.apache.logging.log4j</groupId>
      <artifactId>log4j-layout-template-json</artifactId>
      <version>2.25.4</version>
    </dependency>
  </dependencies>

  <build>
    <pluginManagement>
      <plugins>
        <plugin>
          <artifactId>maven-clean-plugin</artifactId>
          <version>3.4.0</version>
        </plugin>
        <plugin>
          <artifactId>maven-resources-plugin</artifactId>
          <version>3.3.1</version>
        </plugin>
        <plugin>
          <artifactId>maven-compiler-plugin</artifactId>
          <version>3.13.0</version>
        </plugin>
        <plugin>
          <artifactId>maven-jar-plugin</artifactId>
          <version>3.4.2</version>
          <configuration>
            <archive>
              <manifest>
                <mainClass>bug.reproduction.App</mainClass>
              </manifest>
            </archive>
          </configuration>
        </plugin>
      </plugins>
    </pluginManagement>
  </build>
</project>

./src/main/java/bug/reproduction/App.java

package bug.reproduction;

import org.apache.logging.log4j.LogManager;

/**
 * Hello world!
 */
public class App {
    public static void main(String[] args) {
        System.out.println("Should print test below");
        LogManager.getLogger("foo").info("test");
        System.out.println("Should have print test above");
    }
}

./src/main/resources/log4j2.xml

<?xml version="1.0" encoding="UTF-8"?>
<Configuration>
    <Appenders>
        <Lambda name="Console">
            <PatternLayout pattern="%d{yyyy-MM-dd HH:mm:ss} %-5p %c{1} - %m%n"/>
        </Lambda>
    </Appenders>
    <Loggers>
        <Root level="INFO">
            <AppenderRef ref="Console"/>
        </Root>
    </Loggers>
</Configuration>

Expected Output

[INFO] Scanning for projects...
[INFO]
[INFO] --------------< bug.reproduction:aws-lambda-java-log4j2 >---------------
[INFO] Building aws-lambda-java-log4j2 1.0-SNAPSHOT
[INFO]   from pom.xml
[INFO] --------------------------------[ jar ]---------------------------------
[INFO]
[INFO] --- resources:3.3.1:resources (default-resources) @ aws-lambda-java-log4j2 ---
[INFO] Copying 1 resource from src/main/resources to target/classes
[INFO]
[INFO] --- compiler:3.13.0:compile (default-compile) @ aws-lambda-java-log4j2 ---
[INFO] Nothing to compile - all classes are up to date.
[INFO]
[INFO] --- resources:3.3.1:testResources (default-testResources) @ aws-lambda-java-log4j2 ---
[INFO] skip non existing resourceDirectory /Users/bnd/reproduction/aws-lambda-java-log4j2/src/test/resources
[INFO]
[INFO] --- compiler:3.13.0:testCompile (default-testCompile) @ aws-lambda-java-log4j2 ---
[INFO] No sources to compile
[INFO]
[INFO] --- surefire:3.5.4:test (default-test) @ aws-lambda-java-log4j2 ---
[INFO] No tests to run.
[INFO]
[INFO] --- jar:3.4.2:jar (default-jar) @ aws-lambda-java-log4j2 ---
[INFO]
[INFO] --- exec:3.6.3:java (default-cli) @ aws-lambda-java-log4j2 ---
Should print test below
2026-05-19 12:18:10 INFO  foo - test
Should have print test above
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  0.776 s
[INFO] Finished at: 2026-05-19T12:18:10-04:00
[INFO] ------------------------------------------------------------------------

To reproduce expected output, update line 21 in ./pom.xml to <version>1.6.2</version>

Actual Output

[INFO] Scanning for projects...
[INFO]
[INFO] --------------< bug.reproduction:aws-lambda-java-log4j2 >---------------
[INFO] Building aws-lambda-java-log4j2 1.0-SNAPSHOT
[INFO]   from pom.xml
[INFO] --------------------------------[ jar ]---------------------------------
[INFO]
[INFO] --- resources:3.3.1:resources (default-resources) @ aws-lambda-java-log4j2 ---
[INFO] Copying 1 resource from src/main/resources to target/classes
[INFO]
[INFO] --- compiler:3.13.0:compile (default-compile) @ aws-lambda-java-log4j2 ---
[INFO] Nothing to compile - all classes are up to date.
[INFO]
[INFO] --- resources:3.3.1:testResources (default-testResources) @ aws-lambda-java-log4j2 ---
[INFO] skip non existing resourceDirectory /Users/bnd/reproduction/aws-lambda-java-log4j2/src/test/resources
[INFO]
[INFO] --- compiler:3.13.0:testCompile (default-testCompile) @ aws-lambda-java-log4j2 ---
[INFO] No sources to compile
[INFO]
[INFO] --- surefire:3.5.4:test (default-test) @ aws-lambda-java-log4j2 ---
[INFO] No tests to run.
[INFO]
[INFO] --- jar:3.4.2:jar (default-jar) @ aws-lambda-java-log4j2 ---
[INFO] Building jar: /Users/bnd/reproduction/aws-lambda-java-log4j2/target/aws-lambda-java-log4j2-1.0-SNAPSHOT.jar
[INFO]
[INFO] --- exec:3.6.3:java (default-cli) @ aws-lambda-java-log4j2 ---
Should print test below
2026-05-19T16:19:47.710360Z bug.reproduction.App.main() ERROR Error processing element Lambda ([Appenders: null]): CLASS_NOT_FOUND
2026-05-19T16:19:47.720281Z bug.reproduction.App.main() ERROR Unable to locate appender "Console" for logger config "root"
Should have print test above
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  1.038 s
[INFO] Finished at: 2026-05-19T12:19:47-04:00
[INFO] ------------------------------------------------------------------------

Note that the two bug.reproduction.App.main() ERROR lines are written to stderr, as evidenced by running it while redirecting stderr to /dev/null via mvn package exec:java -Dexec.mainClass="bug.reproduction.App" 2> /dev/null

[darklight3it]

Found it was a deployment issue. We are going to redeploy a new version with the correct change.

[darklight3it]

Checking the artifact is not including the dat file

mvn dependency:copy -Dartifact=com.amazonaws:aws-lambda-java-log4j2:1.6.3:jar -DoutputDirectory=/tmp/check -q && \
jar tf /tmp/check/aws-lambda-java-log4j2-1.6.3.jar | grep -i "\.dat" || echo "(none)" && \
mvn dependency:copy -Dartifact=com.amazonaws:aws-lambda-java-log4j2:1.6.2:jar -DoutputDirectory=/tmp/check -q && \
jar tf /tmp/check/aws-lambda-java-log4j2-1.6.2.jar | grep -i "\.dat" || echo "(none)"

(none)
META-INF/org/apache/logging/log4j/core/config/plugins/Log4j2Plugins.dat

DAT files are necessary to register Lambda

[darklight3it]

Confirmed that maven built 1.6.3 with a newer version of java. In new version of jdk annotation processor are [disabled by default] (https://maven.apache.org/plugins/maven-compiler-plugin-4.x/examples/annotation-processor.html),

Tested the 1.6.4 pre-release jar. Log4j2Plugins.dat and native-image/reflect-config.json are back. Jar contents are identical to 1.6.2. (except for the updat in log4j that was intended to be released), the check I run:

$ jar tf aws-lambda-java-log4j2-1.6.4.jar | grep -i "\.dat"
META-INF/org/apache/logging/log4j/core/config/plugins/Log4j2Plugins.dat

$ diff <(jar tf aws-lambda-java-log4j2-1.6.2.jar | sort) <(jar tf aws-lambda-java-log4j2-1.6.4.jar | sort)
(no differences)

Also run repro and it is working

$ mvn package exec:java -Dexec.mainClass="bug.reproduction.App"
Should print test below
2026-05-19 21:23:16 INFO  foo - test
Should have print test above

Thanks @brandondahler for the heads up ❤️.