Disclaimer: the tooling described below was built with Claude Code, on my prompting, and this proposal was drafted with it too.
As you know, xwiki-dev-llm is our public Claude Code plugin, holding the XWiki conventions, knowledge base and skills we share for working on xwiki/* and xwiki-contrib/* repositories. I’ve been working on two more skills, both about security:
xwiki-security-pass: a structured security audit of the XWiki source (platform and extensions), run against a disposable local instance: plan the scopes to cover, look for issues, prove each candidate with a reproduction, check whether it’s already known, and draft the report.
xwiki-security-review: a security review of a single change — a pull request, a commit, or the fix of a security issue (including checking that the fix is complete: other places with the same problem, other branches).
They rely on a knowledge base describing how XWiki’s security-relevant layers actually behave (author model, right checks, rendering and escaping, HTTP layer, etc.).
Note that all this is still a draft: I’ll keep working on it and testing it to tune it and make sure it works well, so please don’t use it just yet. What I’d like to agree on now is where it should live.
Why a separate, private repository
I hesitated to put these in xwiki-dev-llm, which is public. The generic secure-coding rules belong in the open, since they help every contributor write safer code: they are on our Security Developer Guide, and I’ve just proposed adding more of them in Adding more best practices to the Security Developer Guide. But the audit tooling and its knowledge base are mostly useful to someone looking for vulnerabilities in XWiki, and publishing them would make an attacker’s job easier.
So I’m proposing to keep them in a separate plugin, xwiki-security, in a private repository: https://github.com/xwiki/xwiki-dev-security-llm (I’ve created it so that you can see what it would look like).
Access: the members of the security team on GitHub, i.e. the committers handling security issues. For now, only the org owners can see it.
It never holds a vulnerability. Issues found with these skills follow our Security Policy like any other: a restricted JIRA issue, nothing public until disclosure. The skills only draft; a human decides what gets reported.
Content moves to the public side when it can. When something in the knowledge base is a rule that contributors should apply and nothing undisclosed depends on it staying private, it’s proposed for the Security Developer Guide (and then xwiki-dev-llm).
It builds on the public xwiki-dev-llm plugin rather than duplicating it.
If committers don’t agree with having this repository, I’ll remove it.
This looks like something that could be public. I suppose you’re worried that someone could abuse this by making a change that introduces a security issue and then asking Claude what “other places with the same problem” exist?
+1 to have the private xwiki-dev-security-llm repo. We’ll have to decide case by case which skills go there and which go in the public xwiki-dev-llm.
Note that the security team is more than the committers, but it’s good to use the security team as it also contains contributors of extensions.
Honestly I’m mixed about having it private, but I agree it’s probably better to avoid having malicious usage if it simplifies a lot vulnerability discovery. Even if there’s no real confidential stuff in it AFAIU.
Right, thanks. The access is the GitHub security team, i.e. the same people who can see the restricted security issues in JIRA, including extension contributors, not only committers.
I’ve re-checked the split between what’s public and what’s in the private repo:
What remains private is detailed knowledge of how XWiki’s security-relevant layers behave, the audit procedure itself (how to plan a pass, sweep a vulnerability class across the code base, prove and dedup findings), and some examples of past issues. None of this is a secret on its own, but together they make finding vulnerabilities a lot easier.
I also need to correct something I wrote in my first post: “It never holds a vulnerability”. Descriptions of vulnerabilities, including ones not yet disclosed, are useful as examples: a fix often leaves similar issues elsewhere, and the examples help find them. Since the people who can read the repository are the same people who can read the restricted JIRA issues, I don’t see a reason to exclude them. What doesn’t change: any issue found with these skills is always reported first as a restricted JIRA issue, following our Security Policy, and JIRA stays the reference for whether an issue is fixed or disclosed. I’ve updated the repository accordingly.
In the future, we can review the private knowledge bit by bit to decide what should move to the public side, as I did with the Security Developer Guide proposal above. It’s probably best to do it carefully though: publishing a rule also tells anyone what code to look for that breaks it, so we should check the code base against a rule (and fix what doesn’t follow it) before making it public.