Brainstorming: Usability criteria for auditing macros in the WYSIWYG editor

Development
, , ,
1

Hello!

Context

As part of the WYSIWYG usability of XWiki macros proposal, I’m going to audit the macros of XWiki Standard for how easy they are to insert and configure from the WYSIWYG editor. The main goal is macro fluidity, meaning that using a macro in the editor should not break the editing flow (or slow it as little as possible for complex macros ^^’ ).

To keep this audit somewhat consistent from one macro to the next, and make sure we cover everything, I drafted a list of usability criteria most macros would be checked against. Before starting the audit itself, I’d like to gather your feedback on these criteria.

When checking those criteria, I’ll open tickets on Jira for any violation that makes sense (I’ll still use my good sense to avoid reporting unnecessary things – I might discover some issues with the criteria when starting the audit itself).

Proposal

Macros already categorized as deprecated or internal are not audited.
Each macro is classified as one of two types:

  • Wrapper macro: most of its output is content the user types, and the macro decorates or structures it. For example: warning, info, box or expand.
  • Generator macro: most of its output comes from its parameters. For example table of contents, livedata or document tree.

Some criteria only apply to one of the two types. The criteria are grouped by macro use steps: Discovery, Learnability, Insertion, Configuration and Editing. The content of a macro is for the most part treated as a parameter like any other. Macro IDs set in bold are ideas that have already been discussed with a small group of users and devs in private before, I expect those to be less subject to discussion because it’s not something I thought off just by myself :slight_smile:

ID Name Description Applies to
D1 wording The macro name uses the user’s words for what it produces (a warning, a diagram), not technical jargon. All
D2 summary The macro description is one sentence that says what the macro adds to the page. All
D3 category The macro is in the category a user would expect it in. :warning: This is especially important for deprecated and internal macros, which are hidden from insertion only when they declare the Deprecated or Internal category. All
D4 quick-action The Quick Actions menu finds the macro with the words a user would type, including common synonyms like “note” or “callout” for info or warning. All
L1 consistency The macro behaves like the other macros of the same kind, for the mandatory content, the insertion process and its default parameter values. All
L2 doc-parameters The documentation of the macro parameters is up to date. All
L3 doc-example The documentation of the macro contains at least one example for its most common use. All
I1 pattern There is no macro-specific insertion behavior. The macro follows a common behavior, whether inserted from the Quick Actions menu or from the Insert Macro dialog. All
I2 zero-config Inserting with only default values gives a sensible result, not an error box. Generator macros may require configuration before giving a sensible result. Wrapper (All?)
I3 instant Inserting from the Quick Actions menu does not open the Edit Macro dialog. The macro appears in the page right away. Wrapper
I4 cursor After insertion, the cursor is in the macro content, so the user can keep typing without an extra click. Wrapper
I5 focus Inserting opens the Edit Macro dialog with the essential parameters first and focus on the first one. Generator
C1 clarity Each parameter name and description is self-explanatory, and uses user terms. All
C2 mandatory A parameter is mandatory only when the macro’s behavior is ambiguous without it. All
C3 defaults The default value of each optional parameter is the value used in most cases, and the Edit Macro dialog displays it clearly. All
C4 input Each parameter uses an appropriate input, such as a picker, a dropdown or a checkbox. Pickers find what the user looks for. All
C5 groups Parameters used together are in the same group, and groups appear in a logical order. All
C6 order Within each group, and among the parameters outside any group, parameters are ordered by how often they are used. All
C7 alternatives When several parameters are alternative ways to set the same feature, the Edit Macro dialog presents them as a unit, and requires one of them when the macro needs it. All
C8 advanced Rarely used parameters are marked as advanced. All
C9 technical Only parameters that are meant to be edited are shown. All
E1 wysiwyg The macro renders in the WYSIWYG editor the same way as in view mode. All
E2 errors An invalid parameter value produces a message that says what is wrong and how to fix it, not a stack trace. All
E3 nesting The macro behaves consistently when it is placed inside a list, a table or the content of another macro. When the macro has content, that content accepts lists, tables and other macros like the rest of the page. All
E4 empty Empty content still renders in the editor and stays editable, with no zero-width area that cannot receive the cursor. Wrapper
E5 inline The content can be edited as rich text in the editor, not as wiki syntax in a text area of the Edit Macro dialog. Wrapper

This is a simplified view. The full version on the design wiki also gives, for each criterion, a measure explaining how to check it, and an existing Jira issue as an example when we have one.

Open questions

  • Does the split between wrapper and generator macros make sense to you? Are you okay using these names in discussions on the topic?
  • Is there a criterion missing, or one you would drop?

In a week from now, I’ll set a process and templates for myself for auditing all the XS macros, it’d be great if the list of criteria gets somewhat stable by then.
Any feedback is welcome, the earlier the better :slight_smile:

@lucaa @surli tagging you because I figured you especially want to check this topic out and you have a great idea of the usability issues we currently have with macros :wink:

Thank you!
Lucas C.

2

Very good idea to establish a criteria grid. This should allow asking CC to review all macros based on this grid. :grin:

I haven’t checked the criteria yet (will need my computer to see on a large screen).

Thx

3

Thanks for starting this.

We tried applying it to our macros (roughly 75), and it works quite well. I would only broaden the Generator definition a bit: generated output may depend not only on parameters, but also on the current document/context, referenced content, queries, permissions, or external data. As a result, it can be perfectly normal for the same macro invocation to produce different output for different users. This is worth keeping in mind when auditing such macros. Example: the same Live Data macro invocation may expose 10 matching entries to one user and 5,000 to another, depending on the data the respective user is allowed to access.)

A few additional criteria came to mind, and I would slightly refine E1:

ID Name Criterion Applies to
D5 search-id Macro search matches the stable macro ID as well as the localized macro name, so that a macro can be found regardless of the current UI language using the identifier shown in wiki syntax, documentation or error messages. Example: In a German UI, searching for display does not find the Display macro; it can only be found by searching for its localized name, Inhalte. All
E1 wysiwyg The editor representation preserves the semantic meaning of the macro. It does not need to reproduce the exact visual or interactive behavior of View when a dedicated edit representation improves usability. Examples: XWiki’s Dashboard macro intentionally uses a dedicated edit-mode representation instead of reproducing the exact View behavior. Conversely, Live Data in the standalone WYSIWYG editor remains stuck in its loading state in 17.10 (XWIKI-24854). A meaningful semantic edit representation would be more useful than trying to reproduce the original View behavior in the editor. All
E6 unknown-macro An unresolved macro remains understandable and repairable in the editor: the error is at least as informative as in View, and the existing macro ID, parameters and body remain inspectable without data loss. Example: For an unavailable jira macro, View explains that the macro is not registered, suggests checking the spelling, and even lists a compatible extension. In the WYSIWYG editor, the same macro is reduced to a generic “failed to retrieve macro descriptor” error, while the existing macro ID, parameters and body are no longer accessible for repair. All

I personally use Wiki edit mode quite often because a 1:1-like WYSIWYG representation can make authoring harder on macro-heavy pages. If most of the page is generated or included content and only a small part is actually editable, the editable sentence can be difficult to locate. This is another reason why I would prefer semantic fidelity over visual fidelity in E1.

If I can help with the analysis, let me know.

4

Thank you very much for you help on the idea!

It makes sense, thanks for confirming this classification actually works :slight_smile: New definition proposed:

Generator macro: most of its output comes from its parameters and the context around it. For example table of contents, Live Data or document tree.

Note that I don’t plan to put this into our documentation yet, if it becomes useful outside of this fluidity analysis context, we can propose adding it to the xwiki org glossary (once it’s implemented, see XWiki.org Documentation Reorganization (batch 5)).

  • D5: I’d just wrap this up in D4, a localized name is very close to the “synonym” idea we have in D4, but it’s probably even more important. I thought the macro ID was already searchable by default. If it’s not, it’s a bug with Quick actions itself and not something we’d need to check for every macro.
  • E1: I agree it makes more sense. It’s difficult to always say what edit representation improves usability, what’s true for your use case might not be for everyone. I think I’ll just assume that if there’s already a semantic replacement, that’s not a break of the rule but I’ll keep a note on the analysis sheet to mention it. The most complete (but complex) implementation would be to have both semantic and exact representation available for every generator macro. Or we could just agree that every generator macro isn’t doing WYSIWYG but using a semantic replacement in the editor (but AFAIK this would need a proposal). I’ll keep at recording the results for now. I’ll update the rule.
  • E6: IMO this is a problem from the macro system itself, not something we need to check for each macro. The invalid values part is covered by E2 already. Loading... is close to what you describe for missing macros. For now I won’t update the criteria table with this.

:+1: Okay I understand, do you want to create the proposal about semantic/visual fidelity for generator macros right now or would you rather wait for actual numbers from the analysis to support the proposal?

Updated the design page

5

I don’t understand this one. With BlockNote we can implement custom blocks that are mapped to certain macro calls, allowing us to provide a dedicated editing experience for a given macro if it makes sense. Take for instance the code macro. BlockNote supports a code block which allows editing the code in-place and also provides a dropdown to select the code language. This is certainly easier to use than the default Macro edit modal. I don’t see why we would prevent this.

What about the mandatory parameters a macro can have? I find it strange to allow the user to insert the macro right away and then present them a rendering error saying that some parameter is mandatory. You can “complain” the user hasn’t provided a parameter if you didn’t even asked for it.

I wouldn’t enforce this. For some macros it makes sense to have a dedicated editing experience that is different than what you get in view mode, like the code macro example I mentioned.

We can have contextual macros that behave differently depending on where they are inserted. I don’t think we can forbid this.

We can’t enforce this either because some macros may accept only some particular content. I don’t see a problem with limitting the editing capabilies inside a macro to match the macro needs (e.g. disable table support while editing the macro content in-place, if the macro doesn’t support tables inside its content for some reason).

Thanks,
Marius

6

Yes, of course. I was wondering whether it might still be useful to have a separate set of platform-level checks for functionality that all macros depend on, such as the Quick Actions search workflow or the handling and rendering of unknown macros.

So not as criteria to verify for every individual macro, but rather as cross-cutting checks of the editor and macro infrastructure itself.

I’d rather wait for some results from the analysis first. I don’t yet know the project’s direction well enough to judge how such a proposal should best be framed. With some concrete findings, it might also be easier to decide whether semantic fidelity, visual fidelity, or a combination of both would make the most sense as a general guideline.

7

One things I did not see mentioned, but maybe you don’t see it as part of this work: macros can indicate if they support inline context or not and sometimes a macro declare it support inline, but it actually does not really work (fail, or produce standalone content in an inline context). Or the other way around (a macro that does not have any technical reason to be restricted to standalone context only does not officially support inline context).

8

What I mean here is that every macro inline editing experience should be aligned. E.g. the code macro dropdown should be in the same place as other macro inline dropdowns. I don’t think we hit this criteria much with CKEditor since inline editing is fairly limited and dialog edition is very standardized, but it’s always safer to keep a check in the list.

This is only for wrapper macros. I’d expect wrapper macros to have only a few parameters and no mandatory (right now sometimes the content itself is mandatory but it’s inline editable and we might want to fix this anyways).

We talked about E1 with CycleSEC above, we discussed rendering a semantically significant replacement for some macros in some cases. For the code macro, I think it’d still pass the criteria in my eyes if:

  • The “language” dropdown doesn’t stay shown, it only appears when hovering or selecting the macro.
  • The code block is highlighted the same way it would have been in view mode.

This would mean that the macro would look 90% like the view mode which I’d count as a pass.
If we add a grid with the language dropdown and do not include syntax highlighting in the editor, then IMO it’s too far away from what’s viewed and it could confuse some users. It’s very subjective judgement.
For generative macros, as discussed above, I propose to be loose in the evaluation of this criteria and accept semantic replacements (with a note in the report).

But maybe this is too complex/subjective and we just drop it from the list. :confused:

My wording here was off, changing it to use “predictably”. Note that this is just a guideline for a checklist, if macros have a good reason for changing behaviour from one context to the other I wouldn’t have treated it as a violation. But in a lot of cases it would be an indicator that we need two distinct macros or an additional parameter. E.g. using a “low width” version in some context is understandable, but if the low width looks too far away from the full width version (e.g. different color theme) I’d consider this a problem. If the low width is radically different, then I think it’d make sense to add a boolean parameter to trigger it whenever the user wants it rather than rely on an arbitrary limit.

For the sake of keeping a clear history, I’m only updating the design page, not the abstract at the start of this topic.

If there’s a specific limitation on the content, shouldn’t this parameter use a picker that’s more appropriate. From what I understand, content should always be a rich text field that accepts pretty much anything. If a macro only accepts a list as content, then we should not use the standard “content” and introduce a parameter list-content with its own rules, picker and validation instead. WDYT?
TLDR. content is a standard and macros should not add complex logic above it, because it wouldn’t look like a standard to users, just a field that sometimes accepts rich text and sometimes not.

Thanks for your feedback @mflorea !

9

The header message for this topic and title were probably not clear enough. We definitely still have some generic macro edition bugs, related to our system. What I want to find with this analysis is inconsistencies in between macros, so that we can make sure every single one is up to par :slight_smile:

Eventually I’ll update https://design.xwiki.org/xwiki/bin/view/Proposal/WYSIWYGusabilityofXWikimacros/Cross-macro%20findings/#HGenericmacroissues to contain the reported generic issues, but I don’t think it makes much sense setting up a checklist to find those, discovery of them relies much more on experimentation and user feedback.

10

When reading your comment, my first question was “how does a macro indicate that it supports inline?” If it’s only with an error message when used inline, then we definitely have a new macro editor improvement to tackle :smiley:
I created Loading... and Loading... to track this idea.

Good idea, added a new criterion to the list at https://design.xwiki.org/xwiki/bin/view/Proposal/WYSIWYGusabilityofXWikimacros/Usability%20criteria/#H5.Editing :+1: (kept it close to E3 since it’s very similar)

11

The use case I had in mind is not when a macro supports only very limited type of content but when it doesn’t support some specific type of content. It’s definitely smoother for the end user to edit the macro content in-place, with some feature disabled, than to use the macro modal. Moreover, on our side, implementing a custom picker is probably more complex than disabling a specific feature when editing the macro content inplace. Imagine a “heading” macro like:

{{heading level="2"}}
My Heading
{{/heading}}

The user will want to edit the macro content in-place, but we don’t want to allow them to insert block-level content. Moreover, the user will surely be happy to change the heading level in-place (with a dropdown that might look and be placed differently than for the code macro) than to open the macro modal. This is of course a hypothetical example, but I think we can have macros like this.

Thanks,
Marius

12

There is a metadata for it:

  • Java macro : implements #supportsInlineMode()
  • Wiki macro: there is a supportsInlineMode property
13

It’s never shown to the end user and quite difficult to find without indirect clues (e.g. the error box when trying to use a macro inline). I think there’s some improvement we could make here :slight_smile:

14

Not sure which user you are talking about here. It’s a choice made by the author of the macro and on this side of things it’s pretty visible.

If what you mean is that the WYSIWYG does not take into account the context at all and show you all macros even if some of them cannot really be used (because you are inline and are trying to use a standalone only macro for example) then indeed that’s not great, but it’s a bit different from what I had in mind (which was only about making sure that macros have the right metadata, without really thinking of how this shows up in the UI).

15

Sorry for the misunderstanding, I think we agree on a lot but there’s so many moving parts I’m getting confused.

Thanks, I understand the use case better now. I agree that editing in place with some features disabled is smoother than the macro modal or a custom picker. I’m not suggesting we send users back to the modal :slight_smile:

My point is about which part of the macro definition holds that content. The macro descriptor has a dedicated place for the content. I think it’s better for overall UX to keep it to a single meaning for users: rich text that accepts the same things as the rest of the page. When a macro needs restricted content, I’d rather it declares a named parameter for it, with its own type and rules.

That wouldn’t mean giving up in-place editing. CKEditor can already edit a parameter of type wiki content in place: wiki macros do it with {{wikimacroparameter}}, and Java macros can do it by giving the parameter a List<Block> display type and parsing it with the non-generated-content metadata, the same way they handle their content. The editor’s feature restrictions are keyed by the type of the editable, not by whether it’s the content or a non standard parameter, so disabling a feature costs the same either way.

For your heading example, it would look like:

{{heading level="2" text="My Heading"/}}

text is edited in place as inline-only rich text, and level can be changed in place too.
This is a bit less readable in XWiki syntax.

So I’d reword the second part of E3 as: “The macro “content” accepts the same content as the rest of the page. When a macro needs restricted content, it uses a dedicated parameter, which can still be edited in place.” Custom wiki text fields can look just like “content” inplace, but when the user opens the edit modal to understand better why it’s not working as expected they get:

  • A unique parameter name that could explain inherently some limitations.
  • A parameter description that explicitely states what’s allowed or forbidden in this parameter.

From my understanding, it’s a good way to make it clear for the user when “anything’s allowed” and “there’s some restriction”, rather than have them guess and remember it on a macro by macro basis. For existing macros that restrict their “content” parameter, the audit would report it.

This criteria list is there to give the audit a wide coverage, so that we find as many UX problems as possible. It’s not a commitment on which fixes we’ll apply or when we’ll apply them. Each finding will still be discussed on its own, and some fixes might not be worth it, for example moving the restricted content of an existing macro to a parameter, since that changes its wiki syntax. AFAIU, this work highlights the UX issue so that we can avoid creating it again in new macros.

Are you okay with this change?

16

Thanks @CharpentierLucas !

+1 for the names. Out of curiosity, where do you place the code macro that have typed content but a generated (highlighting) output?

For I3, it is not clear to me what happens when a macro has mandatory parameters? Is it excluded from quick actions?
It also feels in contradiction with I5 since displaying the modal shouldn’t be conditioned by how a macro is inserted.

For E1 I would rephrase to mention

  • Never show an empty area
  • Never show a technical error (or only as a secondary debug helper)
  • If the actual rendering cannot be displayed in edit mode, provide information detailing which macro it is and what it will display.

For C4 I would extend it to user assistance more generally:

  • The user must never have to type by hand something that can be suggested (e.g., a document reference through a picker)
  • But, always make it possible to do so (e.g., pasting a document reference as a string from the clipboard).

Some other criteria we could consider:

  • Early validation: Always raise an error for inconsistencies and invalid values that can be caught before submitting (E2 is still needed for users not using WYSIWYG).
  • Rights: A variant is to not propose to users macros that they can’t use because of their rights (and by extension, hide properties a given user can’t use because of their rights)
  • Dependencies: A field whose value or relevance depends on another field is hidden or disabled until that field is filled.
17
  • The code macro has a content parameter.
  • A code macro without content is just an empty block, it still makes sense.
  • When I look at the code macro, the main information is the content itself, only rarely the highlighting itself. The macro just makes it nicer to look at.

That’s the two hints that would make me classify it as a wrapper.

I3 should apply only on wrapper macros and yes, I think that wrapper macros shouldn’t have mandatory parameters. This way the user can just use it fluidly and avoid breaking their chain of thoughts by needing to fill up a form. The solution would be to change the macro so that it could have a proper state without any mandatory information filled up.

It might not be clear from this, but my idea with I5 is that whenever the user lands on the macro editor, the form is ready to fill up. There’s still ways to land on the macro editor at creation with a generative macro (which in a lot of cases have mandatory fields, there’s no great ways to edit them inplace).
I’ll reword the criterion to When inserting a modal through the Edit Macro dialog....

Note that I3 is only on wrapper macros, I5 is only on generative macros so there’s no contradiction. That’s one of the reasons why we need to differentiate those categories, I don’t think we can propose the same UI for every single macro.

IMO it’s covered separately in: empty (E4 in the topic header), errors (E2), the new formulation of E1 discussed with CycleSEC above: The macro renders in the WYSIWYG editor the same way as in view mode, or with a dedicated edit representation that keeps the meaning of the macro.

From what I understand, you want each input to support both “suggestion” and “typing” selection modes right?
I’ll update C4 in the design doc to: Each parameter uses an appropriate input, such as a picker or a checkbox. Pickers support both suggestion and typing for selection. (note that in the case of a checkbox for a boolean, typing/suggestion is definitely overkill, I’ll let the auditor make the distinction based on good sense)

It’d be great for every macro, but AFAIK we barely don’t have any validation implemented in any picker right now. From my understanding it’s an improvement we need to make on standard pickers and not something inconsistent from one macro to another. I’m 50/50 in including it in the scope of the analysis, the conclusion will probably be “every single macro parameter needs validation, except for those 5.”

That’s definitely a problem at the level of the macro system itself and not something relevant to check for every macro. see Loading...

AFAIU we’re still missing the descriptor for this, so it’s an issue for all macros and cheching this on every parameter wouldn’t bring much.

18

Okay, on this point we were not talking about the same thing :+1:

I still agree on the check on inline edition, thanks!

19

For the three quotes above, I still think they are real usability issue. I’m not sure if we want to dismiss them because they are difficult to tackle.
Also depends, how you plan to use the criteria, but even if we don’t use them for existing macros, having the would avoid introducing the same problems to newly developed ones.

20

The code macro is a good counter example for this. It’s a wrapper macro (as you mentioned) but it doesn’t accept content as the rest of the page. Its content can be large, spanning multiple lines, so it doesn’t make sense to save it in a macro parameter. I’m pretty sure other macros can be in the same case, so I wouldn’t limit the usage of the macro content (for wrapper macros) based on whether that content supports rich text (wiki syntax) or not. I think we need to relax the criteria. The way I see it:

  • wrapper macros should use the macro content field for their main data (unless that data is always small), independent of how the macro content is edited in WYSIWYG edit mode
  • in edit mode, the content of wrapper macros should be editable in-place, as much as possible, whether that means restricting the editing feature or not when the caret is inside it
  • we can decide case by case, for each wrapper macro, what is the best way to edit its content in-place; I don’t see a problem with allowing wrapper macros to provide a dedicated in-place editing experience that is tailored for their specific needs / data. E.g. the way you edit the code macro, the info message or the container / columns macro in-place can be different.

Thanks,
Marius