Hi devs,
In the context of improving support for dynamic clustering with XWiki (you dynamically create and remove XWiki instances which are based on identical images when you need them, for example in something like Kubernetes) we identified with @MichaelHamann a need to allow providing differences between cluster members (stuff like the cluster member id, the jgoups configuration when it’s different, which node is in charge of scheduler jobs, etc.) dynamically (typically through environment variable in the case of Kubernetes).
Since we need that for some configuration properties, we might as well provide this for all configuration properties, especially since we are not even yet 100% sure for which properties we are going to need that.
So here are some more precise proposals.
- Allow overwriting
xwiki.propertiesandxwiki.cfgConfigurationSourceproperties using Java system properties
In terms of key syntax, I propose to use -D<configurationsourcehint>:<propertyname>=<value>.
And for the parsing the value (if the ConfigurationSource is asked for anything else than a String) use the xwiki-properties framework (i.e. the same things as the macros) as this is the closest we have as a standard for stuff like this in the XWiki ecosystem.
So that gives something like:
-Dxwikiproperties:environment.permanentDirectory=/some/alternate/path
-Dxwikicfg:xwiki.authentication.cookiedomains=xwiki.org,myxwiki.org
- Allow overwriting
xwiki.propertiesandxwiki.cfgConfigurationSourceproperties using Java system properties
It seems passing Java properties is a bit of a pain in Kubernetes, so I propose to also support environment variables.
Of course, supporting environment variables have its own challenges: to work with any system, we are basically limited to [a-zA-Z_]+[a-zA-Z0-9_]*. I’m sure you noticed that this does not include the . character, which is used in pretty much all properties in XWiki. So I had to come up with some kind of encoding, having (IMO) the following criteria:
- has to be easy to encode for a human
- we don’t really need to decode it, we just need to avoid potential conflicts
So the idea would be to follow the same syntax as described in the Java system properties section above, but with the following encoding:
.,:and-(don’t hesitate if you think of something else that might be common in a configuration file key) are converted to__remains_- any other forbidden character is converted to
_<UTF8 code>
So the Java system properties example would become:
export xwikiproperties_environment_permanentDirectory=/some/alternate/path
export xwikicfg_xwiki_authentication_cookiedomains=xwiki.org,myxwiki.org
Yeah, I know, definitely not the safest encoding out there. But it’s super easy to “encode” for a human in most cases, and I really don’t expect any conflict in practice for the target use case.
- Notes
I’m not really planning to add support for this in more than xwiki.properties and xwiki.cfg configuration sources for now (it does not make that much sense to me for wiki based configuration sources), but I still prefer to have a key syntax which makes easy to target any configuration source (and I’m planning to provide tooling to easily add the same support any other ConfigurationSource) if needed.
Yes, xwiki.properties (through Apache Commons Configuration) already support both system properties and environment variables: you can “prepare” the file by setting stuff like ${sys:user.home} and ${env:JAVA_HOME} as property values. But I feel we need something more generic and which require less setup.
So WDYT ?