Log4j 2.x XSD Is Not Fully Descriptive

In the blog post, JAXB and Log4j XML Configuration Files, I discussed "nuances and subtleties associated with using JAXB to work with [Log4j 1.x and Log4j 2.x] XML configuration files via Java classes." In this post, I look at another challenge associated with generation of Log4j 2.x configuration XML via JAXB objects generated from the Log4j 2.x XML Schema file Log4j-config.xsd: it doesn't fully specify the Log4j 2.x components' configuration characteristics.

When working with Log4j 2.x XML configuration, one of the first distinctions that is important to make is which "flavor" of XML is to be used ("concise" or "strict"). The concise format may be easier because names of XML elements correspond to the Log4j 2 components they represent, but only the strict format is supported by the XSD. The implication here is that any XML marshaled from JAXB objects generated from the Log4j 2.x XSD will necessarily be of "strict" format rather than of "concise" format.

Unfortunately, the XSD (Log4j-config.xsd) currently provided with the Log4j 2.x distribution is not sufficient for generating the full "strict" XML configuration supported by Log4j 2. I demonstrate this here with discussion of the XSD-defined complex type, "AppenderType", because it's one of the most extreme cases of a supported element lacking specification of its potential attributes in the XSD. The code listing below shows AppenderType's definition in the Log4j-config.xsd as of Log4j 2.6.2.

AppenderType Defined in Log4j 2.6.2's Log4j-config.xsd

<xs:complexType name="AppenderType">
   <xs:sequence>
      <xs:element name="Layout" type="LayoutType" minOccurs="0"/>
      <xs:choice minOccurs="0" maxOccurs="1">
         <xs:element name="Filters" type="FiltersType"/>
         <xs:element name="Filter" type="FilterType"/>
      </xs:choice>
   </xs:sequence>
   <xs:attribute name="type" type="xs:string" use="required"/>
   <xs:attribute name="name" type="xs:string" use="required"/>
   <xs:attribute name="fileName" type="xs:string" use="optional"/>
</xs:complexType>

The excerpt of the XSD just shown tells us that an appender described in XSD-compliant XML will only be able to have one or more of the three attributes (type, name, and fileName). The "type" attribute is used to identify the type of appender it is (such as "File", "RollingFile", "Console", "Socket", and "Syslog"). The problem is that each "type" of appender has different properties and characteristics that ideally would be described by attributes on this AppenderType.

The Log4j 2.x documentation on Appenders lists characteristics of the different types of appenders. For example, this page indicates that the ConsoleAppender has seven parameters: filter, layout, follow, direct, name, ignoreExceptions, and target. The name is one of the attributes supported by the general AppenderType complex type, while filter and layout are supported via nested elements in that AppenderType. The other four parameters that are available for a ConsoleAppender, however, have no mechanism prescribed in the XSD to define them.

Without even considering custom Log4j 2.x appenders, the built-in Log4j 2.x appenders don't share the same attributes and characteristics and most of them have more characteristics than the three attributes and two nested elements of the AppenderType specify. I discussed the seven parameters of a Console Appender previously and other examples include the RollingFileAppender with its twelve parameters (append, bufferedIO, bufferSize, filter, fileName, filePattern, immediateFlush, layout, name, policy, strategy, ignoreExceptions) the JDBCAppender with its seven parameters (name, ignoreExceptions, filter, bufferSize, connectionSource, tableName, columnConfigs), and the JMSAppender with its thirteen parameters (factoryBindingName, factoryName, filter, layout, name, password, providerURL, destinationBindingName, securityPrincipalName, securityCredentials, ignoreExceptions, urlPkgPrefixes, userName).

To describe each parameter available for a given appender type in the XSD would require the ability in XML Schema to write that a particular set of available attributes depends on the setting of the AppenderType's type attribute. Unfortunately, XML Schema doesn't readily support this type of conditional specification in which the available attributes of a given complex type are different based on one of the complex type's other attributes.

Because of the limitations of the schema language, a person wanting to use JAXB to generate objects with full support for all of the provided appenders would need to change the XSD. One approach would be to change the XSD so that an AppenderType had all possible attributes of any of the built-in appenders available as optional attributes for the element. The most obvious downside of this is that that XSD would then allow any appender type to have any attribute even when the attribute did not apply to a particular appender type. However, this approach would allow for the JAXB-generated objects to marshal out all XML attributes for a given appender type. The next code snippet illustrates how this might be started. Some of the additional attributes different appenders need are specified here, but even this longer list doesn't contain all of the possible appender attributes needed to support attributes of all possible built-in appender types.

Some Appender Attributes Added to AppenderType

<xs:complexType name="AppenderType">
   <xs:sequence>
      <xs:element name="Layout" type="LayoutType" minOccurs="0"/>
      <xs:choice minOccurs="0" maxOccurs="1">
         <xs:element name="Filters" type="FiltersType"/>
         <xs:element name="Filter" type="FilterType"/>
      </xs:choice>
   </xs:sequence>
   <xs:attribute name="type" type="xs:string" use="required"/>
   <xs:attribute name="name" type="xs:string" use="required"/>
   <xs:attribute name="fileName" type="xs:string" use="optional"/>
   <!-- Attributes specified below here are not in Log4j 2.x Log4j-config.xsd -->
   <xs:attribute name="target" type="xs:string" use="optional"/>
   <xs:attribute name="follow" type="xs:string" use="optional"/>
   <xs:attribute name="append" type="xs:string" use="optional"/>
   <xs:attribute name="filePattern" type="xs:string" use="optional"/>
   <xs:attribute name="host" type="xs:string" use="optional"/>
   <xs:attribute name="port" type="xs:string" use="optional"/>
   <xs:attribute name="protocol" type="xs:string" use="optional"/>
   <xs:attribute name="connectTimeoutMillis" type="xs:integer" use="optional"/>
   <xs:attribute name="reconnectionDelayMillis" type="xs:string" use="optional"/>
   <xs:attribute name="facility" type="xs:string" use="optional"/>
   <xs:attribute name="id" type="xs:string" use="optional"/>
   <xs:attribute name="enterpriseNumber" type="xs:integer" use="optional"/>
   <xs:attribute name="useMdc" type="xs:boolean" use="optional"/>
   <xs:attribute name="mdcId" type="xs:string" use="optional"/>
   <xs:attribute name="mdcPrefix" type="xs:string" use="optional"/>
   <xs:attribute name="eventPrefix" type="xs:string" use="optional"/>
   <xs:attribute name="newLine" type="xs:boolean" use="optional"/>
   <xs:attribute name="newLineEscape" type="xs:string" use="optional"/>
</xs:complexType>

A second approach to changing the Log4j 2.x XSD to support all built-in appenders fully would be to change the XSD design from having a single AppenderType whose specific type was specified by the type attribute to having many different complex types each representing the different built-in appender types. With this approach, all the attributes for any given appender and only the attributes associated with that given appender could be enforced by the XSD. This approach of having an element type per appender is similar to how the "concise" XML format works, but there is no XSD support for that currently.

Note that I have intentionally focused on the built-in appender types here because that's what a static XSD could be expected to reasonably, adequately, and completely support. Aside: this could be supported by specifying arbitrary name/value pairs for attributes as is done for filters or with parameters, but these also lead to the ability to specify extra and even nonsense attributes without any ability for the schema to catch those. A third approach that would support custom types would be to not use a static XSD for describing the grammar, but instead use a generated XSD. One could hand-write such an XSD based upon the descriptions of Log4j 2.x components in the documentation, but a better approach might be to take advantage of the @PluginFactory, @PluginElement, and @PluginAttribute annotations used in the Log4j 2.x source code. The two code listings that follow are from the Apache Log4j 2.6.2 code base and demonstrate how these annotations describe what would be the elements and attributes of given types.

ConsoleAppender.createAppender() Signature

@PluginFactory
public static ConsoleAppender createAppender(
   @PluginElement("Layout") Layout extends Serializable> layout,
   @PluginElement("Filter") final Filter filter,
   @PluginAttribute(value = "target", defaultString = "SYSTEM_OUT") final String targetStr,
   @PluginAttribute("name") final String name,
   @PluginAttribute(value = "follow", defaultBoolean = false) final String follow,
   @PluginAttribute(value = "ignoreExceptions", defaultBoolean = true) final String ignore)

SysLogAppender.createAppender() Signature

@PluginFactory
public static SyslogAppender createAppender(
   // @formatter:off
   @PluginAttribute("host") final String host,
   @PluginAttribute(value = "port", defaultInt = 0) final int port,
   @PluginAttribute("protocol") final String protocolStr,
   @PluginElement("SSL") final SslConfiguration sslConfig,
   @PluginAttribute(value = "connectTimeoutMillis", defaultInt = 0) final int connectTimeoutMillis,
   @PluginAliases("reconnectionDelay") // deprecated
   @PluginAttribute(value = "reconnectionDelayMillis", defaultInt = 0) final int reconnectionDelayMillis,
   @PluginAttribute(value = "immediateFail", defaultBoolean = true) final boolean immediateFail,
   @PluginAttribute("name") final String name,
   @PluginAttribute(value = "immediateFlush", defaultBoolean = true) final boolean immediateFlush,
   @PluginAttribute(value = "ignoreExceptions", defaultBoolean = true) final boolean ignoreExceptions,
   @PluginAttribute(value = "facility", defaultString = "LOCAL0") final Facility facility,
   @PluginAttribute("id") final String id,
   @PluginAttribute(value = "enterpriseNumber", defaultInt = Rfc5424Layout.DEFAULT_ENTERPRISE_NUMBER) final int enterpriseNumber,
   @PluginAttribute(value = "includeMdc", defaultBoolean = true) final boolean includeMdc,
   @PluginAttribute("mdcId") final String mdcId,
   @PluginAttribute("mdcPrefix") final String mdcPrefix,
   @PluginAttribute("eventPrefix") final String eventPrefix,
   @PluginAttribute(value = "newLine", defaultBoolean = false) final boolean newLine,
   @PluginAttribute("newLineEscape") final String escapeNL,
   @PluginAttribute("appName") final String appName,
   @PluginAttribute("messageId") final String msgId,
   @PluginAttribute("mdcExcludes") final String excludes,
   @PluginAttribute("mdcIncludes") final String includes,
   @PluginAttribute("mdcRequired") final String required,
   @PluginAttribute("format") final String format,
   @PluginElement("Filter") final Filter filter,
   @PluginConfiguration final Configuration config,
   @PluginAttribute(value = "charset", defaultString = "UTF-8") final Charset charsetName,
   @PluginAttribute("exceptionPattern") final String exceptionPattern,
   @PluginElement("LoggerFields") final LoggerFields[] loggerFields, @PluginAttribute(value = "advertise", defaultBoolean = false) final boolean advertise)

This approach requires several steps because one would need to dynamically generate the XSD using knowledge of the main components of the Log4j 2.x architecture in conjunction with annotations processing and then use JAXB to generate the Java classes capable of marshaling the comprehensive Log4j 2.x XML.

Another option to be considered is to use "concise" XML or another form of Log4j 2.x configuration (such as JSON or properties files) and not use the XSD to generate JAXB objects for marshaling Log4j 2.x configuration. It's worth noting that XML configuration files used for Log4j 2.x with the "strict" format obviously do not need to validate against the Log4j-config.xsd or else the "strict" form of XML would not be able to fully specify the Log4j2 configuration. The implication of this is that the remaining value of even having the XSD is either for our own tools or scripts to use it to validate our XML configuration prior to using it with Log4j 2.x or for use in marshaling/unmarshaling Log4j 2.x XML with JAXB.

Conclusion

The Log4j-config.xsd provided with the Log4j2 distribution is not sufficient for validating all Log4j 2.x constructs in "strict" XML configuration and is likewise insufficient for generating JAXB objects to use to marshal Log4j2 strict XML. Developers wishing to use XSD for validation or JAXB class generation would need to manually change the XSD or generate one from the Log4j2 source code.

Additional References

These references were linked to inline in the post above, but are listed here for emphasis.

• In Log4j2, how do I associate an XML Schema with log4j2.xml?
• [LOG4J2-170] Schema validation aginst Log4J-V2.0.xsd fails
• [LOG4J2-411] Support of XSD/DTD linked to the configuration file
• logging-log4j-user mailing list: "Log4j2 Appender attributes with strict xml config"
  • Generate XSD from annotations?
  • Make Schema more like concise format?
  • KeyValuePairs on Appenders
  • Support Parameters on AppenderType