Realtime: Auto-save original document vs. a draft

Hi everyone,

Realtime editing requires auto-save. When multiple users are editing at the same time, the responsibility of saving the content is not well-defined:

  • either each user will leave it to the others to save, so no one will save, loosing content
  • or every one will try to save, often at the same time, leading to merge conflicts

For this reason the save needs to be synchronized between the editing users and performed automatically. This is independent of the edit mode (Wiki vs. WYSIWYG) or the editor used (CKEditor or something else). But what should be saved?

There seems to be two main approaches:

  1. Update directly the edited document (Notion)
    • :green_circle: weā€™re already doing this so no big change is needed
    • :green_circle: if no one is currently editing then what you get in edit mode is what you saw in view mode before starting to edit (this is especially important for the in-place edit mode)
    • :green_circle: the history is complete: you can see the intermediate auto-saved revisions and thus determine more precisely when a change was done and by whom
    • :green_circle: if the edited content has scripts that rely on the current page reference they will work as expected
    • :red_circle: we have to deal with merge conflicts when the edited document is saved outside the realtime session (e.g. if thereā€™s a realtime WYSIWYG and realtime Wiki session at the same time)
    • :red_circle: we canā€™t group the revisions auto-saved from realtime because we could have revisions created outside the realtime session, interleaved
    • :red_circle: discarding the changes done during realtime edit could revert as well changes done (at the same time) outside the realtime session
    • :red_circle: The auto-saved content can be in an unfinished state that prevents it from being properly rendered. This is less of a problem for the WYSIWYG editor, but itā€™s a big problem for the realtime Wiki editor, where partial wiki syntax can be auto-saved that breaks the rendering of the document for those that look at the page in view mode.
    • :red_circle: the users accessing the page in view mode will see partial content (even if the partial content is properly rendered)
  2. Create a draft that can be later merged with the original document (Confluence)
    • :green_circle: no merge conflicts to deal with (from saves done outside the realtime session), because the draft is owned by the realtime session
    • :green_circle: A change summary can be specified when merging (like when we merge a pull request by squash), leading to a more clean history on the original document. The user can also see at this point all the changes made during the realtime edit (the diff), reducing the risk of accidental / unwanted changes, especially if we remove the Cancel button from the realtime edit.
    • :green_circle: no need to group or identify the auto-saved revisions on the original document history
    • :green_circle: the draft can be easily discarded at any moment without affecting the original document
    • :green_circle: the user looking at the page in view mode is seeing a final / polished version of the content
    • :red_circle: more complex to implement
    • :red_circle: The draft has to be persisted beyond the life of the realtime session. Starting a new realtime editing session should resume the edit of the previous draft. This can cause confusion if the content of the draft is very different from what the user had seen in view mode before editing.
    • :red_circle: the draft can diverge from the original document significantly, making it more difficult to merge (if the original document is updated during the realtime session)
    • :red_circle: we need a different draft for WYSIWYG and Wiki edit modes (per preferred editor)
    • :red_circle: the history of the original document may be incomplete if we chose to merge the draft by squashing its revisions
    • :red_circle: weā€™d have to persist the script level of the realtime session on the draft document so that when realtime edit is resumed the script level is not reset (until after the draft is merged)
    • :red_circle: scripts that rely on the current document reference might not work as expected (the draft is not the same as the original document)
    • :red_circle: overlaps in functionality with Change Request application

Do you see any other pros and cons I missed? Iā€™m not sure whatā€™s best long term for XWiki. WDYT? I guess this needs to be discussed also in the context of Cristal, ping @mleduc .

Note that I started a design page on how to improve the UI/UX around realtime editing https://design.xwiki.org/xwiki/bin/view/Proposal/RealtimeEditingSession . This topic is just a part of it.

Thanks,
Marius

Hi Marius,

personally I think the draft option is a better approach on the long run.

I donā€™t see that as a cons but almost as pro: it seems to me that weā€™re more and more seeing this need of being able to have draft of wiki pages that are able to mimic all features of the original pages, even if theyā€™re not technically stored at the same place. I think Workflow Application and Book Version might have same needs.

We can probably mitigate that a bit, by informing users when thereā€™s an ongoing realtime session to propose them to join it instead of performing a single edit (maybe the mechanism already exists) and probably also by proposing in the realtime session to perform the merge right away when a save is detected on the original page.

Honestly Iā€™m a bit puzzled about that one: is there a true need for allowing realtime editing in wiki mode? I mean, I see the need of switching to source mode from the WYSIWYG editor from time to time, but then the realtime session would be resumed as soon as the user switch back to WYSIWYG no?

By incomplete I guess you mean it doesnā€™t contain every single save of the draft: for me itā€™s almost more a feature than a bug. What could be done for mitigating this, maybe, is to propose to have milestones that could be configured or manually decided by the user. Now I wouldnā€™t go there at the beginning: Iā€™m not sure thereā€™s a real need.

I think Cristal and XWiki are pretty much aligned on this topic.

Another option, that probably does not exist in an actual product is the following.

  • autosave is always on
  • at some point users can decide to tag a save
  • the most recent tag is the one presented to users by default, but user or page configuration can force to view to ā€œliveā€ version

That way

  • users are able to collaborate freely without disrupting the reading experience
  • tagged versions are intentionnaly currated, but minor intermediate changes are not lost

Iā€™m mentionning this idea but, Iā€™m sure it has significant impact on various related aspect (e.g., large history etc).

But, this is somewhat in contradiction with the ā€œwikiā€ approach where content is public by default. Adding intermediate steps before making knowledge public can be hurtful. For instance, users with stalled draft documents and nobody available to make them public.

+1, Iā€™m interested in motivational usecases for this point.

In the specific case of realtime, for me, this concept of stalled draft wouldnā€™t exist: either you save your session and you perform the merge of the draft, or you juste cancel and delete the draft.
I was mentionning the need of having this draft document more on the technical aspect of XWiki.

We have an extension that enables realtime editing for the Wiki editor, https://extensions.xwiki.org/xwiki/bin/view/Extension/Realtime%20Wiki%20Editor/ . Its code is even in xwiki-platform. Users can install and use it. But realtime WYSIWYG and realtime Wiki canā€™t share the same session. I mean, you canā€™t have in the same session a user that edits the content with the WYSIWYG editor and another one that uses the Wiki editor, mainly because when you edit directly the wiki syntax you can have partial wiki syntax that is pushed to the other users (like opening a macro call {{hm) that easily messes up the editing in the WYSIWYG editor.

So as long as we have some kind of support for realtime Wiki editor, we will have to:

  • either use a separate draft, but I agree itā€™s not nice
  • or have a preferred editor for the realtime session, set when the realtime session is created, that cannot change for the life of the realtime session, meaning that users joining the session will have to use the editor preferred by the realtime session, not the one specified in their user preferences. Basically, the user that starts the realtime session implicitly sets the preferred editor for the session

Thanks,
Marius

I called them ā€œMilestone revisionsā€ in my design page https://design.xwiki.org/xwiki/bin/view/Proposal/RealtimeEditingSession#HMilestonerevisions , but you went a bit further, suggesting to also change the view mode to display the latest tagged / milestone revision, which is an interesting idea. The problem I see is that it requires coordination between the editing users to decide when to create a tagged revision, which is hard to achieve without a video call or chat.

Note that in Confluence there is a single draft per document. Itā€™s not per user. Whoever edits the document will continue to edit the draft from where it was left by the previous realtime editing session (if thereā€™s no realtime editing session in progress already). Iā€™m not sure itā€™s a good idea to have drafts per user.

No, it canā€™t work like that. The draft needs to leave beyond the realtime session because:

  • the realtime session is closed when all the users are disconnected (no user left in the session)
  • users can be disconnected due to network issues, not necessarily because they want to leave the editing session

So when the users reconnect they should be able to resume the draft from where it was left. Of course, the users can also choose to discard the current draft and start fresh.

I think if a draft is saved in the live page, it is no longer a draft. Option 1 redefines WYSIWYG to WYSIWYH, changing ā€˜Gā€™ (Get) to ā€˜Hā€™ (Have). :wink: I think this makes realtime editing a bit too much ā€˜real-time.ā€™

For that reason, realtime editing is currently deactivated in all our customersā€™ and our own environments, even though it would be great to use it."

Even with a single draft per page. Itā€™s still an issue if the admins are not really available to make the single draft public after an external contributor worked on it.
Mentioning it as Iā€™m currently experiencing this issue on personal projects, where new volunteers are willing to do some work, but nobody is available to grant rights, or move things from draft to published. But it might be a corner case :slight_smile:

The draft is not merged automatically. The users decide when the draft is complete, at which point they can document / explain the changes by merging with a detailed version summary. This means adding a new revision to the original document, with all the changes done in the realtime session (from the draft).

So if realtime editing would save the changes in a draft that you control / decide when it is merged with the original document, it would work for you? I mean, you would be willing to use realtime editing in that case right?

Thanks,
Marius

I agree with this. Itā€™s one of the downside of using drafts. But if we go this way, I guess we could support both:

  • edit directly the original document, but youā€™re editing alone, without auto-save
  • edit a draft based on the original document, but you can collaborate with others (with auto-save)

So users could choose.

Thanks,
Marius

Actually, a last option could be to allow anybody with edit rights to promote a draft to a final version. I think that would address my concerns.

Exactly the same needs as for realtime WYSIWYG :grinning:, but for users preferring to use the wiki editor. For me your question is the same as asking whether thereā€™s a need for the wiki edit mode. What did I miss?

Thx

I strongly prefer that we develop a proper draft mechanism that is used both in realtime and when editing alone (by auto-saving to a draft). This draft mechanism could also replace temporary attachments by a permanent solution that survives session invalidation and restarts of XWiki. The draft could be shared or user-specific for non-realtime use cases, this could also be a configuration. If both realtime and non-realtime sessions exist with a shared draft at the same time, every automatic save would perform a merge of the draft I guess which sounds a bit problematic. Maybe we could use the existing page locking mechanism to prevent this situation?

Regarding storage, I wonder if we could use the existing revisions store to save a draft revision. A problem of this idea is that this might be expensive on big pages, in particular with many objects, so ideally we would develop a storage mechanism that kind of saves a ā€œdiffā€ with only the changed parts, and not, e.g., all XObjects including those that werenā€™t modified.

I think if there is a (shared) draft of a document, every user with edit right should see a message above the document that there is a draft of this document with the option to either view or edit that draft. [edit] And I guess a direct link to a diff between current version and draft would also be good.

1 Like

Yes, of course. It is a quite useful feature if you work in a team, even remotely. However, it is essential to leave the page unchanged until you really accept to save.

(As far as I know, this is not only the case for Confluence but also for SharePoint.)

1 Like