Hello all,
Important disclaimer, please read first: the research, the measurements, the spike and the drafting of this proposal were all done by Claude Code, on my prompting. I have little hands-on knowledge of the modern JavaScript testing ecosystem myself, so I am in a poor position to judge whether the claims and the strategy below are actually sound. Everything here needs to be verified by our front-end experts before it is treated as a decision — the tool comparisons, the numbers, the conclusions about what is and isn’t feasible, and the migration plan. I am posting it because I think it is a useful starting point for a discussion we have been deferring since 2022, not because I am confident it is right. Please push back hard.
I’d like to propose that we officially adopt Vitest as the JavaScript/TypeScript unit test library for XWiki Standard, and that we retire the jasmine-maven-plugin.
This is narrow on purpose. It is about unit tests for front-end code in xwiki-platform. It is not about functional/E2E tests (Docker + Selenium on the Java side, Playwright on the Cristal side), and it is not about React component testing, which was already settled in Choice of the testing framework for React (BlockNote).
Why now
Three things have been sitting unresolved:
- XWIKI-19316 “Get rid of jasmine-maven-plugin” has been Open since January 2022. The dev.xwiki.org Testing page still documents
mvn jasmine:bddas the way to write a JS unit test, while warning that the plugin is “a bit dead”. - Standardizing our use of node in maven projects (Feb 2024) established pnpm + Vite + the
webjar-nodepackaging platform-wide. It mentions Vitest twice — as a motivation (“easier definition of front-end tests (e.g., Vitest)”) and as a build step (“call vitest”) — but explicitly says “Testing is not present in the PoC”. The test library was deferred and never proposed. - Unit Test Library choice (Oct 2023) did select Vitest, but it is scoped to Cristal (the design page lives under
Proposal/Cristal/Technologies/Testing/) and framed around testing Vue 3 components. Its adoption inxwiki-platformhappened de facto, never by an explicit decision for XWiki Standard. I believe that is precisely why XWIKI-19316 was never closed: everyone knew the answer, nobody had the mandate to write it down.
So this proposal is mostly about formalising what we already do, plus deciding what to do with what’s left over.
Where we actually stand today
Measured on master (18.9.0-SNAPSHOT), excluding node_modules/ and target/:
package.json files in the pnpm workspace |
78 |
…of which declare vitest |
68 |
Vitest test files (*.test.ts / .js / .tsx, *.spec.*) |
~93 |
package.json referencing Jest, Karma or Mocha |
0 |
| Remaining Jasmine spec files | 12, in 2 modules |
Legacy non-minified .js under src/main/** outside src/main/node |
~168 |
Maven already runs these tests: the webjar-node packaging (defined in xwiki-commons-tool-webjar-node-handlers) binds frontend-maven-plugin:pnpm run test:unit to the Maven test phase, exactly as compile runs pnpm run build and verify runs pnpm run lint. Ten platform modules use that packaging today, plus the whole xwiki-platform-node tree.
The 12 leftover Jasmine specs are in xwiki-platform-ckeditor-plugins (7) and xwiki-platform-web-war (5). An important detail: both jasmine-maven-plugin executions sit in an integration-tests-chrome profile, with the comment “we cannot run the tests on CI (Chrome is not installed)”. So none of those 12 specs have run on CI for years. Whatever we do with them, we are not losing coverage we currently have.
The alternatives
I want to be fair here: several of these are genuinely good tools, and the argument for Vitest is not that the others are bad. It is that one of them fits our existing build with no impedance mismatch.
Jest — a genuinely good alternative
The long-time default, huge ecosystem, excellent docs, great mocking and snapshot support. If we had no bundler, this would be a real contest.
Against it, for us: ESM support is still behind a flag and awkward, so we would need Babel or ts-jest to transform our TypeScript — a second transform pipeline, configured separately from the Vite one we already use to produce the actual webjars. That means the code under test is not transformed the same way as the code we ship. Vitest, by contrast, reuses the module’s existing vite.config.ts. That single point is, in my view, the decisive one.
Jest’s popularity is also clearly rolling over: State of JS 2025 shows Vitest and Playwright as the two biggest gainers in the testing category (+14pp usage each), with Jest’s satisfaction trending down.
Node.js built-in test runner (node:test) — also genuinely good
Zero dependencies, ships with the runtime, fast, and much more capable than when it landed. For a pure Node library I would consider it seriously.
Against it, for us: no DOM environment (we would wire up jsdom ourselves), no transform pipeline (so TypeScript and .vue need our own loader plumbing), no browser mode, and a much thinner mocking/snapshot story. We would be assembling by hand what Vitest gives us configured.
Web Test Runner (@web/test-runner) — genuinely good for the browser-first case
Standards-focused, runs tests in real browsers, well engineered. If our priority were “everything must run in a real browser”, this would be a serious candidate and I would not argue strongly against it.
Against it, for us: smaller ecosystem and momentum, and no native Vite integration — which again means a second build description. Vitest 4 now covers the real-browser case in-family (see below), which removes most of the reason to reach for it.
Mocha (+ Chai + Sinon)
Mature and very flexible — and, to correct something stated in the 2023 comparison, Mocha and Jasmine do have perfectly good mocking stories (Sinon, and Jasmine’s own spies). The real objection is assembly: runner, assertion library, mocking library, TS transform and DOM environment are five separate decisions, none of which is aligned with our Vite build.
Jasmine — the status quo
Worth separating two things. Jasmine the library is not dead and is still maintained. jasmine-maven-plugin is — its last stable release is 2.2, from September 2016, and the 3.0 beta is only available through JitPack. Likewise Karma, the runner it leans on, has been deprecated since 2023 (no new features, no general bug fixes); Angular, by far the largest Jasmine/Karma user base, shipped experimental Vitest in v20 and makes Vitest the default runner in v21.
There is no plausible future in which we keep the current setup. Even if we chose Jasmine the library, we would have to replace the way we run it.
AVA / uvu / tape
Minimal Node-oriented runners, no DOM story, and uvu is effectively unmaintained. Not credible for us.
Playwright component testing
Not a unit test framework, and not a competitor here — it is the component/browser-level layer, already adopted for React/BlockNote. Mentioned so the boundary is explicit.
What I propose
P1 — Vitest is the unit test library for JavaScript and TypeScript in XWiki Standard. It already is, in 68 of 78 packages; this makes it official and lets us close XWIKI-19316.
P2 — jsdom is the default test environment, configured per module by merging the module’s own Vite config, as we already do:
export default mergeConfig(viteConfig, defineConfig({ test: { environment: "jsdom" } }));
P3 — For AMD / RequireJS-based and globals-based webjar code, use @xwiki/platform-test-requirejs. Its mockRequireJS() installs fake define / require / requirejs globals backed by an in-memory registry. It is already used by seven packages, and it is the piece that makes non-Vue, non-ESM legacy code testable at all.
P4 — Retire jasmine-maven-plugin from xwiki-platform-web-war and xwiki-platform-ckeditor-plugins, and with it the integration-tests-chrome profiles.
P5 — Do not convert the 7 CKEditor 4 specs. CKEditor 4 is being replaced by BlockNote, the specs have not run on CI in years, and converting them would be effort spent on code with a known end date. I propose we delete them along with the plugin, or let them go when CK4 goes — but explicitly, not by accident.
P6 — Convert the web-war specs that can be converted, into xwiki-platform-web-webjar/src/main/node, which already declares vitest and wires "test:unit": "vitest --run --passWithNoTests" and currently contains zero tests. See the caveats below — this is the part I cannot yet promise in full.
P7 — Conventions: the module’s npm script is test:unit (that is what the webjar-node lifecycle invokes); test files live next to the code they test in a __tests__ directory, named *.test.ts; import describe / it / expect explicitly from vitest rather than enabling globals: true.
P8 — No enforced JS coverage threshold for now. We have none today (no @vitest/coverage-* anywhere, no coverage config), in contrast with the JaCoCo ratios we enforce on the Java side. I would rather adopt the framework first and discuss a coverage policy as a follow-up than block this on it.
Evidence gathered, including what did not work
The hard cases were spiked rather than assumed, because “Vitest is modern” is not an argument that survives contact with a 15-year-old prototype.js codebase.
CKEditor 4 under Vitest + jsdom: works. xwiki-table.js was ported verbatim. The real 766 KB minified ckeditor.js loads into the jsdom window, the globals-based plugin evaluates in, and all 2 tests / 17 assertions pass in ~1.3s. The general lesson matters more than the specific spec: a legacy globals-based script — neither AMD nor ESM — can be unit-tested under Vitest + jsdom by evaluating it into the window, which is the pattern we need for the ~168 remaining legacy files.
The legacy web-war xwiki.js under jsdom: only partly. Loading prototype.js 1.7.3, jQuery 3.7.1 and the built entityReference.js all work. The legacy xwiki.js then fails in two stages: first ReferenceError: require is not defined (solved — that is exactly what mockRequireJS() is for), and then TypeError: element.attachEvent is not a function, thrown from inside prototype.js’s event layer as xwiki.js registers its top-level document.observe('xwiki:dom:…') handlers. The same failure occurs under happy-dom. It has not been root-caused.
There is a trap worth flagging: the existing spec’s 8 assertions still pass after that failure, because the URL builders are assigned before the document.observe calls that blow up. A naive port looks green while testing a half-loaded file.
Encouragingly, the migration path already exists in-repo. entityReference has already moved from legacy webjar JS to xwiki-platform-web-webjar/src/main/node/src/entityReference.ts, and ends with what I think should be our documented interop pattern:
globalThis.XWiki = Object.assign(globalThis.XWiki ?? {}, api);
if (typeof define === "function" && define.amd) {
define("xwiki-entityReference", [], () => globalThis.XWiki);
}
A TypeScript/ESM source that publishes both a global and an AMD module — testable with Vitest, consumable by the legacy page.
What remains to be verified
I would rather propose with the gaps visible than oversell:
- The prototype.js / jsdom incompatibility. Whether
element.attachEventis a small shim away, or whether prototype.js-era DOM code fundamentally needs a real browser. This decides how much of web-war we can actually convert. Of the 5 specs,jQueryNoConflictandxwiki.jslook feasible under jsdom;eventsBridge(a prototype↔jQuery event bridge),suggestandexporterare DOM-heavy and will hit this head-on. - Consequently, whether we need a real-browser escape hatch, and which one. Vitest 4.0 (22 Oct 2025) made Browser Mode stable with Playwright/WebdriverIO providers, which is the in-family option. But topic 17120 concluded the opposite for React — “if we need a puppet browser no matter what, it is a better choice to directly use Playwright without introducing the additional indirection layer that is Vitest” — and that conclusion predates Vitest 4.0. I do not want to reopen it casually. My inclination is: leave 17120 standing for component-level tests, and decide the legacy-DOM case on the evidence from point 1. Input from @ClementEXWiki and @mleduc especially welcome.
- The web-war “test the minified output” property. The current Jasmine setup points
jsSrcDirat the build output, so it incidentally verifies that minification did not break anything. Vitest tests sources. I propose we consciously drop that property rather than contort the setup — but it should be a decision, not an oversight. - Scope for contrib extensions. Topic 14066 (post 11) noted that pnpm cannot resolve webjar dependencies from outside the repo, so this setup does not extend to contrib without publishing npm packages. I propose this proposal covers
xwiki-platformonly, and contrib stays out of scope for now. (xwiki-commonsandxwiki-renderingcontain no JavaScript at all, so they are moot.)
Two smaller wrinkles worth recording: js/xwiki/xwiki.js is a Velocity template (its ${request.contextPath} placeholders are asserted literally by the existing spec), so whatever we do here tests a templated file; and the build pins node.version to v24.18.0 in xwiki-commons/pom.xml while jsdom 29 refuses to load on Node < 22.12, which contributors running tests outside Maven need to know.
Plan if accepted
- Close XWIKI-19316 by removing
jasmine-maven-pluginand theintegration-tests-chromeprofiles from both modules. - Resolve open question 1 above, then convert whichever web-war specs are convertible into
xwiki-platform-web-webjar. - Update dev.xwiki.org Testing, which still documents
mvn jasmine:bdd,src/test/javascript, anOPENSSL_CONF=""workaround and “tests that do not rely on the DOM”. The “Choosing a type of test to write” section says “a Javascript unit test (using Jasmine for example)” and needs the same fix. - Document the conventions from P2/P3/P7 and the
entityReferenceinterop pattern, so the next person writing a test for legacy webjar code has something to copy.
WDYT?