Stop running javac lint checks on generated code

Hi devs,

Our build reports 120 javac warnings on generated code. We cannot fix any of them, and they are what stops us from turning on -Werror.

So I’d like us to stop linting generated sources.

What we get today

Measured on master (18.8.0-SNAPSHOT, JDK 21):

Generator Module Warnings
SableCC xwiki-platform-query-jpql-parser 100 rawtypes
JavaCC xwiki-rendering-wikimodel 6 this-escape, 6 static
XJC xwiki-commons-repository-model 4 dep-ann, 4 deprecation

Only 4 of the 120 are deprecations. Raw types are 83% of it. So this is about lint checks in general, not just deprecations.

Note: you only see these with the Develocity build cache disabled. With it on, compile is served from cache and reports 0 javac warnings.

Why we cannot fix them

  • SableCC predates generics and is unmaintained. It generates ArrayList list = new ArrayList();. No generator option changes that.
  • JavaCC generates the *TokenManager class body itself. There is nowhere to put a @SuppressWarnings.
  • XJC copies @deprecated from our XSD into javadoc but does not emit the @Deprecated annotation. That gives the 4 dep-ann warnings. It also gives the 4 deprecation ones, and that part is worth spelling out, because “just fix the call site” looks right but is not.

Here is the generated call site:

@Override
public Extension withFeatures(String... values) {
    for (String value : values) { getFeatures().add(value); }
    return this;
}

features is deprecated since 8.0M1, but it is part of the Extension Repository REST format, so it has to stay in the XSD for old clients. withFeatures() exists only to fill it, so there is nothing else it could call. And javac only warns here because XJC did not annotate withFeatures() either. With the annotation, javac would stay silent, since it does not warn about deprecated use inside deprecated code.

Why it matters

xwiki-commons-pom has carried this TODO for years:

At some point, we should turn on failing on warnings since they can be real errors and they keep growing on our build logs

We cannot turn on -Werror while the build has warnings nobody can fix. Clearing generated code is the prerequisite for it.

We also already made this choice for our other linters:

  • xwiki-platform-query-jpql-parser excludes the generated packages from Checkstyle.
  • xwiki-commons-repository-model and xwiki-rendering-wikimodel set xwiki.checkstyle.skip, with the comment “Only contain generated sources for which we don’t control the code style”.
  • xwiki-commons-pom silences Sonar’s java:S1133 for legacy modules.

This just makes javac agree with them.

One trap, which I think we should agree on explicitly

-Xlint applies to a whole javac call, not to a source path. So switching a category off “for a module” also stops checking that module’s hand-written code. wikimodel has 102 hand-written files against 42 generated ones, and 2 real this-escape sites in the hand-written ones that we do want reported.

So the fix should not be a module-wide -Xlint:-xxx, except in modules that contain only generated code. Instead, give the generated source root its own maven-compiler-plugin execution with -Xlint:none and restrict the default execution to src/main/java. This works even when the generated and hand-written sources reference each other (wikimodel), by passing the hand-written root on -sourcepath with -implicit:none.

I have this working on all 3 modules: the 120 warnings are gone, hand-written sources keep -Xlint:all (I checked by introducing a raw type on purpose, it is still reported), and class output and test results are unchanged.

See Loading... for the jira issue and links to the 3 PRs (commons, rendering and platform).

WDYT?

Thanks

The same way we don’t check for the quality of Bytecode, or minified JavaScript, I’m +1 to only inspect the highest level of definition, and none of the derived artifacts.

Yes, I don’t see much value in searching for deprecated use, codestyle problems and other warnings in general in code we don’t control.

What about code generated for example by JavaCC where we do control some portions of the code? Wouldn’t this mean that we lose any warnings for this code that we do write ourselves? Also, shouldn’t we replace SableCC if it is unmaintained? I did a quick check with Claude Code, and it seems that it is possible to emit the @Deprecated annotation at least on the field, getter and setter using the jaxb-basics-annotate plugin:

Diff
diff --git i/xwiki-commons-core/xwiki-commons-repository/xwiki-commons-repository-model/pom.xml w/xwiki-commons-core/xwiki-commons-repository/xwiki-commons-repository-model/pom.xml
index 200023bbe5..4b4b65a771 100644
--- i/xwiki-commons-core/xwiki-commons-repository/xwiki-commons-repository-model/pom.xml
+++ w/xwiki-commons-core/xwiki-commons-repository/xwiki-commons-repository-model/pom.xml
@@ -54,6 +54,16 @@
         <artifactId>jaxb-maven-plugin</artifactId>
         <configuration>
           <generatePackage>org.xwiki.extension.repository.xwiki.model.jaxb</generatePackage>
+          <args combine.children="append">
+            <arg>-Xannotate</arg>
+          </args>
+          <plugins combine.children="append">
+            <plugin>
+              <groupId>org.jvnet.jaxb</groupId>
+              <artifactId>jaxb-basics-annotate</artifactId>
+              <version>${jaxb.tools.version}</version>
+            </plugin>
+          </plugins>
         </configuration>
         <executions>
           <execution>
diff --git i/xwiki-commons-core/xwiki-commons-repository/xwiki-commons-repository-model/src/main/resources/xwiki.rest.extension.model.xsd w/xwiki-commons-core/xwiki-commons-repository/xwiki-commons-repository-model/src/main/resources/xwiki.rest.extension.model.xsd
index 94fd16ee8e..1baae1a629 100644
--- i/xwiki-commons-core/xwiki-commons-repository/xwiki-commons-repository-model/src/main/resources/xwiki.rest.extension.model.xsd
+++ w/xwiki-commons-core/xwiki-commons-repository/xwiki-commons-repository-model/src/main/resources/xwiki.rest.extension.model.xsd
@@ -21,7 +21,10 @@
 -->
 
 <schema xmlns="http://www.w3.org/2001/XMLSchema" targetNamespace="http://www.xwiki.org/extension"
-  xmlns:extension="http://www.xwiki.org/extension" xmlns:xwiki="http://www.xwiki.org" elementFormDefault="qualified">
+  xmlns:extension="http://www.xwiki.org/extension" xmlns:xwiki="http://www.xwiki.org"
+  xmlns:jaxb="http://java.sun.com/xml/ns/jaxb" jaxb:version="2.0"
+  xmlns:annox="http://annox.dev.java.net" jaxb:extensionBindingPrefixes="annox"
+  elementFormDefault="qualified">
 
   <complexType name="Link">
     <attribute name="href" type="string"></attribute>
@@ -171,6 +174,10 @@
               <documentation>
                 @deprecated since 8.0M1, use extensionFeatures instead
               </documentation>
+              <appinfo>
+                <annox:annotate target="field">@java.lang.Deprecated</annox:annotate>
+                <annox:annotate target="getter">@java.lang.Deprecated</annox:annotate>
+              </appinfo>
             </annotation>
           </element>
           <element name="extensionFeatures" type="extension:ExtensionId"
@@ -194,6 +201,11 @@
               <documentation>
                 @deprecated since 16.8.0RC1, use supportPlans instead
               </documentation>
+              <appinfo>
+                <annox:annotate target="field">@java.lang.Deprecated</annox:annotate>
+                <annox:annotate target="getter">@java.lang.Deprecated</annox:annotate>
+                <annox:annotate target="setter">@java.lang.Deprecated</annox:annotate>
+              </appinfo>
             </annotation>
           </element>
           <element name="properties" type="extension:Property"

This doesn’t annotate withFeatures, though, as this is from another plugin, so it’s only a part of the solution.

In general, I’m not sure if it is a good idea to completely silence those warnings as to me, it seems like in each of these cases they point at a real problem that might not be simple to solve but should still be solved, by switching to a maintained dependency or improving the dependency/getting the dependency improved. However, what I’m wondering is: if we give the generated sources their own maven-compiler-plugin execution, anyway, couldn’t we just disable -Werror for them while keeping warnings on so we can still see the warnings, but they won’t fail the build?

Actually, CC warned me that someone could say this :wink:

One thing I deliberately left out, tell me if you’d rather it were in: for the XJC module specifically there is a real fix rather than a suppression — the module already uses extension bindings, so jaxb2-basics-annotate could inject the missing @Deprecated and silence both kinds properly. It only helps those 8 warnings, not the other 112, but somebody may well raise it.

The jaxb2-basics-annotate alternative for the XJC module is still not mentioned anywhere public. If someone raises it on the forum, it’s a legitimate fix for those 8 warnings — just not for the other 112.

So sure, feel free to fix them if you want (the proposal is not preventing any fixes anywhere). But this is just hiding the forest. The goal is to focus on the larger stuff, less on the exceptions/small details (at first sight, it’s not bad logic to say that we care less about generated code, for which we have less control). And sure, once we’ve fixed all the big parts, we could then try to be even better and remove those ignores for the edge cases (generated code).

This is already the case today (not using -Werror). But that’s just noise. Be honest with me: how often have you checked those warnings and tried to fix them since you joined the project (and you joined not long ago, these have been there for 15 years+).

The goal is not to try to be perfect everywhere (that brings complexity). It’s to decide where we put our efforts. Right now we have tons of places that warrant looking at and fixing. Whenever we display warnings or errors that are “normal” / “expected”, it lessens the value of the ones that should be fixed as we don’t look at them as a consequence.

I’d prefer that we try to have a build without any warning or errors, and try to keep it that way. It also makes it simpler when analyzing the build for warnings/errors.

Now, if we really want what you propose, and keep the noise, then sure, it’s doable.

Thanks

I don’t think I’ve ever tried fixing warnings with generated code, my point is more that if we silence them now globally, we’ll never notice when anything introduces new warnings.

Now I’ve just taken a look at the proposed PRs, and they seem to do something different from what you proposed (“So I’d like us to stop linting generated sources.”): they disable lint checks on specifically the three modules that have errors we cannot fix, they don’t disable lint checks on all generated code.

I’m fine to disable the lint checks on precisely those three modules, ideally with a clear comment why they are disabled, maybe even just disabling those checks that are actually failing, and where it makes sense (I guess at least for SableCC) a Jira issue for fixing the root cause/replacing the unmaintained dependencies.