Brainstorming: Interactivity of annotations

Hi!
When implementing the design in Loading... , I came across a couple problems and I’m not sure about the solution. I’ll present the problems shortly, and the solutions I thought would be best so far. It would be great to get feedback right now so that we agree on the way to implement it :slight_smile:

Context

Right now, annotations are split into multiple HTML nodes. Each text node in the selected text is wrapped into an annotation node. This node provides the “highlighted” style, and the interactivity is done through the post-it button that appears at the end of the annotation:

The goal is to get rid of the post-it button and do the interaction through the highlighted text itself.
I went with the postulate that the annotations must be mark HTML nodes (semantically it fits the best IMO).

Problem 1

How to make the annotation keyboard interactable?
We can just add the click listener that was on the icon to the different highlight pieces. But:

  1. The mark nodes are not in the tab index by default. We need extra markup to make it accessible using a keyboard.
  2. Even when added to the tab-index, the Space key does not trigger the click event like it would for a button naturally.

My solution to this is to nest a button inside each mark. This also makes it easier to provide a clear text alternative that describes the button action.

But there’s still a problem. In cases where the mark is split in a lot of parts (like the screenshot above), it would create that many buttons and crowd the tab index for little use (all the buttons do the same thing – open the annotation bubble). I decided to only add a button on the first mark of each annotation. The negatives left with this solution:

  • buttons will not wrap across lines. This means that if the first text node is long, showing the annotation will deface the content. I can’t figure out the style combination to avoid this… (assuming it even exists)

  • assistive techs will not see the whole annotated text content easily from this button, but IMO it’s a small enough problem to stop complexifying the system here.

Here is what the DOM would look like:

Do you see a better solution or an easy way to tackle the first drawback? The best solution to keep the style would be to mimic a button but we’d lose all the native features…

Problem 2

How to highlight the whole annotation at once?
A small part of the design is to change the color of an annotation on hover.
With the current DOM and the one showed to solve problem 1, it’s easy to get the link between parts of the same annotation in javascript, but in CSS it’s difficult/hackish at best.

I searched through a few solutions:

  • Change the way we generate the mark nodes to only have one. => In some case or another, what I tried would break. The way we do it right now is the safest and looks more sturdy than any other way I could think of.
  • Replace our custom system with the CSS Custom Highlight API - Web APIs | MDN . Unfortunately it looks like this does not work well with interactivity. This would be a lot of time spent on moving things, and I fear that there’ll be something missing in this system for it to answer our use cases. It looks quite heavy to setup next to what’s existing just to get a hover effect.

The solution I figured would be best:

  • set a hover/focus listener on each mark that adds a class on all of its siblings. The CSS can just check if the class is here to display the “hover” state, instead of using the standard :hover pseudo class. The drawbacks of this solution is that it’s more custom code so more bugs and computation cost.

This solution is definitely not the best and I feel like I’m missing something. Do you agree with using it or should I search for a different one?

Note that the design notes made it clear that “highlighting on hover” could be dropped if it was too complex to do. I’d like to avoid dropping it, it looks way nicer with it :slight_smile:


Thank you in advance for your help!
I’ll resume implementing a proposal for this ticket in a few days if there’s no ongoing discussion.

Lucas C.

Why not use a link (anchor) instead of a button on the first marked node?

Not siblings, because the marked nodes can be in different subtrees, but yes, the only solution I see is to use JavaScript to find all parts of the same annotation (based on the annotation ID). I don’t think it’s that complex or hard to maintain.

Good idea! Thank you for the help. Action links can still convey somewhat correct semantics (not perfect but it’ll do) without breaking the inline text :slight_smile:

It’s not that complex, but it’s still an additional moving part in the system. Thank you for confirming I’m not missing something obvious here :slight_smile:

1 Like

Sorry, I missed this discussion and now that I’ve been asked to review the PR to implement this at XWIKI-22696: Annotation marker is not keyboard accessible by Sereza7 · Pull Request #4960 · xwiki/xwiki-platform · GitHub I’ve noticed that the proposed solution here would completely break interacting with links inside the selected content unless I missed something. For this reason, I would prefer to keep an extra icon/button to open to the annotation popup. Maybe we could make it bigger and inside the regular text flow? This could move the text, but it might be better than other solutions. What do you think?

Can another option be something like what we have in Live Data when clicking on a link on an editable value?
In case of ambiguity (e.g., the user can edit the cell or follow the link), a popover is displayed on hover or on touch, presenting all the available actions (here, follow the link or see the annotation).

Unfortunately annotations can go on pretty much any text, so any custom interaction could be inside there. The solution I think we’ll go with is:

  • Click on the marked up text: do both. In most cases, opening the annotation bubble won’t have much of an impact on the other action.
  • Keyboard click on a button added at the end of the annotation that’s only visible when focused with the keyboard: only open the annotations.

This way, users who know the feature a bit can do both.
I’m not sure how easy it would be to consistently spot “this element is interactive” in javascript. getEventListener is obviously enough for most solutions, but what if a parent element is listened to and in the callback there’s a filter to check that the event is triggered on this child…