Sorting out Log4J 2.0’s strict schemas

Posted on May 8, 2018 by Rick Jelliffe

Log4J version 2.0 has two dialects of its XML configuration language: concise and strict.

Concise mode is fairly well document and freeform. It uses the reflection API and plugins, so that if there is a plugin available, you can just call its name directly.  So there is no schema: the elements and attributes depend on whatever the plugin designer selected.  And it means that Apache can offer YAML, JSON and property files as alternatives without much maintenance effort.

But, of course, a config file can get complicated quite quickly. So being able to validate the XML file is a pretty good thing: unless you have a debugger and the log4j code, it becomes essential. So they provide a strict mode with an XSD schema.

But the trouble is that the Log4J documentation is pretty lousy for the strict schema. And the actual Schema they provide is not really complete.  And that is a problem because if you have a schema complex enough to need to validate, you need to the schema to be pretty much complete.

Since this is Schematron.com, let me add the unnecessary and gratuitous comment that they would not have had this problem, to the same extent, if they had gone with Schematron rather than XSD 1.0.  And, by the same token, XSD 1.1 has some improvements that would help.  For example, in their property lists, they have standard properties where the value of the name attibute will determine the possible values of the type attribute.  Easy for Schematron, and possible with XSD 1.0 assertions and with RELAX NG too.  Just not in DTDs or XSD 1.0.

So what gives with the strict mode?

You can see the basic operation in the Apache source code in XMLConfiguration.java

  • Initialize:
    • Line 95 reads the config file into an XML DOM
    • Line 113 reads the top level attributes into a property map
    • Line 152 validates the document if you are in strict mode, and you supplied the schema to use, both in the config file.
  • Setup:
    • Line 258 is a method setup() that calls constructHierarchy() where…
    • Line 288 recursively iterates through all the nodes of the DOM and populates a home-made DOM-like data structure, Node.
    • Line 232 is the function to supply the name of the Node: if it is strict mode, it looks in an attribute “type”, otherwise it uses the element name, using
    • Line 338  element.getTagName().   This is the key: it only uses the qname (the element name as tagged) so in fact it does not matter whether your instance has a namespace or not, from the POV of executing it. Validating it requires it to be in the namespace  xmlns="http://logging.apache.org/log4j/2.0/config", but log4j doesn’t care, as long as you don’t mess up the name using prefixes.

This explains why many examples  of the “strict” syntax on the WWW fail to have a namespace declaration: they didn’t actually validate them, they just got something that somehow ran.

So the problems with the strict mode are three IMHO:

  1. The XSD is incomplete, so you cannot be guaranteed that the Log4j features you need are available. However, if you are prepared to reverse engineer the Java code and enhance the XSD,  it should work.
  2. The Apache documentation is partial and confusing.  Even looking at the schema is no indication of how do anything: I am trying to figure out how to do RollingFileAppender   and it is not remotely clear
  3. A lot of of the documentation on the WWW is wrong. Some things apply to Log4J. Somethings apply to earlier versions of the schema (they added a few things later). Some things they give examples which we know (because of no namespace) will not really validate.

The most recent version of the schema is at the GitHub site.  I have made a little updated version of it, which at the moment just has a few minor updated top-level attributes.

(I am intrigued by the possibility of treat the strict mode as extensible, by allowing xs:any wildcard elements in no namespace in most positions, with skipped validation   <xs:any namespace="##local" processContents="skip" />.  The instance could just, in theory, the use  any log4j elements (with xmlns=””) and the validation would not fail but just ignore those quasi-foreign elements. This would only work if log4j plugins don’t care about an unexpected xmlns attribute on the config elements. If I look at this further, I’ll write it up.  Until then, I suppose that we just use the validation as far as it goes, but then if ultimately  we need elements that are not present in the XSD then at that stage we forego validating the configuration file.)

<?xml version="1.0" encoding="UTF-8"?>
<!--
    This information is as at 7 May 2018.
    
    This is an enhanced version of the Apache Log4j strict 
    configuration XML Schema from 
       master/log4j-core/src/main/resources/Log4j-config.xsd
    Compared to the documentation of earlier versions, 
    users may find it useful that that the current Apache 
    XSD schema now has elements for 
       /Configuration/Appenders/Console  
       /Configuration/Appenders/Console/PatternLayout 
       /Configuration/CustomLevels/CustomLevel
       /Configuration/CustomLevel
       /Configuration/Properties/Filter/KeyValuePair
       /Configuration/Properties/Filters/Filter/KeyValuePair
       
    with the following changes:
    
    1) Top-level attributes added, following Rathai, 
    and checked with the code
    master/log4j-core/src/main/java/org/apache/
         logging/log4j/core/config/xml/XmlConfiguration.java
     Where Rathai is the schema suggested here
     http://mail-archives.apache.org/mod_mbox/logging-log4j-user/
    201310.mbox/%3c399C55D178537F4FB31E4F8F6A8AE1ECCD8B8D@mscas.materna.com%3e 
    2) Top-level attributes are made optional
 -->
<!--
 Licensed to the Apache Software Foundation (ASF) under one or more
 contributor license agreements.  See the NOTICE file distributed with
 this work for additional information regarding copyright ownership.
 The ASF licenses this file to You under the Apache License, Version 2.0
 (the "License"); you may not use this file except in compliance with
 the License.  You may obtain a copy of the License at
      http://www.apache.org/licenses/LICENSE-2.0
 Unless required by applicable law or agreed to in writing, software
 distributed under the License is distributed on an "AS IS" BASIS,
 WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 See the License for the specific language governing permissions and
 limitations under the License.
--> 
<xs:schema xmlns:xs="http://www.w3.org/2001/XMLSchema" xmlns="http://logging.apache.org/log4j/2.0/config" targetNamespace="http://logging.apache.org/log4j/2.0/config" elementFormDefault="qualified" attributeFormDefault="unqualified">
    <xs:element name="Configuration" type="ConfigurationType"/>
    <xs:complexType name="ConfigurationType">
        <xs:sequence>
            <xs:choice minOccurs="0" maxOccurs="1">
                <xs:element name="CustomLevels" type="CustomLevelsType"/>
                <xs:element name="CustomLevel" type="CustomLevelType"/>
            </xs:choice>
            <xs:element name="Properties" type="PropertiesType" minOccurs="0"/>
            <xs:choice minOccurs="0" maxOccurs="1">
                <xs:element name="Filters" type="FiltersType"/>
                <xs:element name="Filter" type="FilterType"/>
            </xs:choice>
            <xs:element name="ThresholdFilter" type="ThresholdFilterType" minOccurs="0"/>
            <xs:element name="Appenders" type="AppendersType"/>
            <xs:element name="Loggers" type="LoggersType"/>
        </xs:sequence>
        <xs:attribute name="advertiser" type="xs:string" use="optional"/>
        <xs:attribute name="dest" type="xs:string" use="optional"/>
        <xs:attribute name="monitorInterval" type="xs:int" use="optional"/>
        <xs:attribute name="name" type="xs:string"/>
        <xs:attribute name="packages" type="xs:string" use="optional"/>
        <!-- packages is a comma separated list -->
        <xs:attribute name="schema" type="xs:string" use="optional"/>
        <xs:attribute name="shutdownHook" type="xs:boolean" use="optional"/>
        <xs:attribute name="shutdownTimeout" type="xs:string" use="optional"/>
        <xs:attribute name="status" type="xs:string" use="optional"/>
        <xs:attribute name="strict" type="xs:string" use="optional"/>
        <xs:attribute name="verbose" type="xs:boolean" use="optional"/>
    </xs:complexType>
    <xs:complexType name="PropertiesType">
        <xs:sequence>
            <xs:element name="Property" type="PropertyType" minOccurs="0" maxOccurs="unbounded"/>
        </xs:sequence>
    </xs:complexType>
    <xs:complexType name="AppenderType">
        <xs:sequence>
            <xs:element name="Layout" type="LayoutType" minOccurs="0"/>
            <xs:choice minOccurs="0" maxOccurs="1">
                <xs:element name="Filters" type="FiltersType"/>
                <xs:element name="Filter" type="FilterType"/>
            </xs:choice>
        </xs:sequence>
        <xs:attribute name="type" type="xs:string" use="required"/>
        <xs:attribute name="name" type="xs:string" use="required"/>
        <xs:attribute name="fileName" type="xs:string" use="optional"/>
    </xs:complexType>
    <xs:complexType name="RootType">
        <xs:sequence>
            <xs:element name="AppenderRef" type="AppenderRefType" minOccurs="1" maxOccurs="unbounded"/>
        </xs:sequence>
        <xs:attribute name="level" type="xs:string"/>
    </xs:complexType>
    <xs:complexType name="PropertyType">
        <xs:simpleContent>
            <xs:extension base="xs:string">
                <xs:attribute name="name" type="xs:string"/>
            </xs:extension>
        </xs:simpleContent>
    </xs:complexType>
    <xs:complexType name="KeyValuePairType">
        <xs:simpleContent>
            <xs:extension base="xs:string">
                <xs:attribute name="key" type="xs:string"/>
                <xs:attribute name="value" type="xs:string"/>
            </xs:extension>
        </xs:simpleContent>
    </xs:complexType>
    <xs:complexType name="AppendersType">
        <xs:choice minOccurs="0" maxOccurs="unbounded">
            <xs:element name="Appender" type="AppenderType"/>
            <xs:element name="Console" type="ConsoleType"/>
        </xs:choice>
    </xs:complexType>
    <xs:complexType name="ConsoleType">
        <xs:sequence>
            <xs:element name="PatternLayout" type="PatternLayoutType" minOccurs="1" maxOccurs="unbounded"/>
        </xs:sequence>
        <xs:attribute type="xs:string" name="name" use="required"/>
        <xs:attribute type="xs:string" name="target" use="required"/>
        <xs:attribute type="xs:string" name="follow" use="optional"/>
    </xs:complexType>
    <xs:complexType name="PatternLayoutType">
        <xs:simpleContent>
            <xs:extension base="xs:string">
                <xs:attribute type="xs:string" name="pattern" use="required"/>
            </xs:extension>
        </xs:simpleContent>
    </xs:complexType>
    <xs:complexType name="AppenderRefType">
        <xs:simpleContent>
            <xs:extension base="xs:string">
                <xs:attribute name="ref" type="xs:string" use="required"/>
            </xs:extension>
        </xs:simpleContent>
    </xs:complexType>
    <xs:complexType name="LoggerType">
        <xs:sequence>
            <xs:choice minOccurs="0" maxOccurs="1">
                <xs:element name="Filters" type="FiltersType"/>
                <xs:element name="Filter" type="FilterType"/>
            </xs:choice>
            <xs:element name="AppenderRef" type="AppenderRefType" minOccurs="0" maxOccurs="unbounded"/>
        </xs:sequence>
        <xs:attribute name="name" type="xs:string" use="required"/>
        <xs:attribute name="level" type="xs:string" use="optional"/>
        <xs:attribute name="additivity" type="xs:string" use="optional"/>
    </xs:complexType>
    <xs:complexType name="FilterType" mixed="true">
        <xs:sequence>
            <xs:element name="KeyValuePair" type="KeyValuePairType" minOccurs="0"/>
        </xs:sequence>
        <xs:attribute name="type" type="xs:string" use="required"/>
        <xs:attribute name="level" type="xs:string" use="optional"/>
        <xs:attribute name="marker" type="xs:string" use="optional"/>
        <xs:attribute name="onMatch" type="xs:string" use="optional"/>
        <xs:attribute name="onMismatch" type="xs:string" use="optional"/>
    </xs:complexType>
    <xs:complexType name="FiltersType">
        <xs:sequence>
            <xs:element name="Filter" type="FilterType" minOccurs="0" maxOccurs="unbounded"/>
        </xs:sequence>
    </xs:complexType>
    <xs:complexType name="CustomLevelType">
        <xs:attribute name="name" type="xs:string" use="required"/>
        <xs:attribute name="intLevel" type="xs:string" use="required"/>
    </xs:complexType>
    <xs:complexType name="CustomLevelsType">
        <xs:sequence>
            <xs:element name="CustomLevel" type="CustomLevelType" minOccurs="0" maxOccurs="unbounded"/>
        </xs:sequence>
    </xs:complexType>
    <xs:complexType name="LoggersType" mixed="true">
        <xs:sequence>
            <xs:element name="Logger" type="LoggerType" minOccurs="0" maxOccurs="unbounded"/>
            <xs:element name="Root" type="RootType" minOccurs="1" maxOccurs="1"/>
        </xs:sequence>
    </xs:complexType>
    <xs:complexType name="LayoutType" mixed="true">
        <xs:sequence>
            <xs:element name="Pattern" type="xs:string" minOccurs="0"/>
        </xs:sequence>
        <xs:attribute name="type" type="xs:string" use="required"/>
        <xs:attribute name="pattern" type="xs:string" use="optional"/>
    </xs:complexType>
    <xs:complexType name="ThresholdFilterType">
        <xs:attribute name="level" type="xs:string" use="optional"/>
        <xs:attribute name="onMatch" type="xs:string" use="optional"/>
        <xs:attribute name="onMismatch" type="xs:string" use="optional"/>
    </xs:complexType>
</xs:schema>