Making it easy to parse wiki markup in documents, including inside macros and xobjects

Development
1

Hi devs,

What triggered this topic is my need in the docapp contrib extension to perform checks on the XDOM and include checking inside wiki markup located inside macros that contain wiki markup.

This is what I had to write:

...
        // Check for violations inside the document's main content.
        XDOM xdom = document.getXDOM();
        checkXDOM(xdom, violations);

        // Also check inside macros that contain wiki markup.
        List<MacroBlock> macroBlocks =
            xdom.getBlocks(new ClassBlockMatcher(MacroBlock.class), Block.Axes.DESCENDANT);
        for (MacroBlock macroBlock : macroBlocks) {
            try {
                ContentDescriptor contentDescriptor = this.macroManager.getMacro(new MacroId(macroBlock.getId()))
                    .getDescriptor().getContentDescriptor();
                if (contentDescriptor != null && Block.LIST_BLOCK_TYPE.equals(contentDescriptor.getType())) {
                    TransformationContext context = new TransformationContext(xdom, document.getSyntax());
                    MacroTransformationContext macroContext = new MacroTransformationContext(context);
                    XDOM macroXDOM = this.contentParser.parse(macroBlock.getContent(), macroContext, false, false);
                    checkXDOM(macroXDOM, violations);
                }
            } catch (MacroLookupException e) {
                this.logger.warn("Failed to look up macro [{}]. Ignoring Image Macro check inside it. "
                    + "Root error cause: [{}]", macroBlock.getId(), ExceptionUtils.getRootCauseMessage(e));
            } catch (MacroExecutionException e) {
                this.logger.warn("Failed to parse the content of macro [{}]. Ignoring Image Macro check inside it. "
                    + "Root error cause: [{}]", macroBlock.getId(), ExceptionUtils.getRootCauseMessage(e));
            }
        }
...

Then @MichaelHamann mentioned the more generic need of doing that but also including wiki markup found inside xproperties that can contain wiki markup.

Let’s brainstorm about what APIs we could add for these use cases.

API 1 Idea: Inside BlockMatcher class

Add a BlockMatcher that returns Blocks for all MacroBlock that have a macro ContentDescript of type List<Block>.

API 2 Idea: Inside Block class

Add a new Block.Axes enum to mean descendant, including macros which accept wiki markup, so that Block.getBlocks() would return the blocks.

API 3 Idea: somewhere in platform. Maybe in refactoring-api?

Introduce some API taking as input a DocumentReference or a Document/XWikiDocument and returning all XDOMs (from the content, from all xproperties containing wiki markup).

Something List<List<Block>> DocumentBlockFinder#getBlocks(BlockMatcher, Axes) where a List is returned for each XDOM. Now, this won’t be enough since the calling code will need to have more info like the name of the xproperties, so we’ll probably need to introduce some new object containg both an Entity Reference + the List<Block>.

So something like List<DocumentMatch> DocumentBlockFinder#getBlocks(BlockMatcher, Axes).

More ideas?

In my opinion we might need all these APIs, at the different levels.

WDYT?

Thanks

1 Like
2

Thank you a lot Vincent Massol for your brief solution. ChatGPT sent me here. I wonder if there is a more convenient solution to this problem at the moment. For our institute’s migration from Confluence to XWiki, I am writing confluence adapter macros. This includes a proprietary macro “Table Transformator” (or similar) that we bought for Confluence.

My Script Component class saves any named table-excerpt macro contents of a saved page as XObjects on the same page and the table-joiner macro on any other page collects such XObjects and joins them with a flexible full outer join. No SQL needed.

My problem is, even a table-excerpt is directly saved inside a table-joiner-macro (instead of just the table-excerpt-include references), it won’t be saved as an XObject as it is not found.

3

The current solution is most of the time some version of recursive parsing based on the macro properties, like you can find in https://snippets.xwiki.org/xwiki/bin/view/Extension/Replace%20macros%20using%20Macro%20Converters%20from%20XDOM/ or https://snippets.xwiki.org/xwiki/bin/view/Extension/RemoveMacrosFromXDOM/ - this should be adaptable for your situation.

1 Like