For full visibility and disclosure, here’s the project we submitted and that is sponsored by the NGI Search & Discovery call.
XWiki Federation
Abstract
Can you explain the whole project and its expected outcome(s).
Please be short and to the point in your answers; focus primarily on the what and how, not so much on the why. Add longer descriptions as attachments (see below). If English isn’t your first language, don’t worry - our reviewers don’t care about spelling errors, only about great ideas. We apologise for the inconvenience of having to submit in English. On the up side, you can be as technical as you need to be (but you don’t have to). Do stay concrete. Use plain text in your reply only, if you need any HTML to make your point please include this as attachment.
So far, XWiki has been focusing on providing the best collaboration experience and features to its users. It’s now the right time for XWiki to start collaborating externally and be part of the larger federation of collaboration and social software (a.k.a. fediverse). The proposal is to achieve this collaboration by embracing the W3C ActivityPub specification (https://www.w3.org/TR/activitypub/) and becoming compliant with it. More specifically to implement at least the server to server part of the specifications and be able to both view activity and content happening in external services (such as NextCloud, Discourse, Mastodon, PeerTube, Friendica, Plume, etc) inside XWiki itself and to make XWiki’s activity and content available from these other services too. Similary, we would also like to consider XWiki as an external service and work on the XWiki federation itself. This means considering the various XWiki instances around the world and be able to share activity and content between these instances.
Note: We use AP below this point to mean ActivityPub.
General information about AP:
Use Cases & Integration Points
-
Groundwork:
- Integrate into XWiki’s profiles by exposing each profile as an AP URL returning data as JSON.
- Integrate into XWiki’s authentication/permissions system so that only allowed external users are able to perform AP queries against XWiki
- Implement discovery of user profiles, using WebFinger. The idea is to convert a user alias (in the form of name@domain) into an AP URL.
- Integrate and test with other services. Since the integration between services requires some discussion and collaboration (the specifications leaves some points open), focus on integrating and testing with a few services to start with: XWiki instances, NextCloud, Discourse. The choice of NextCloud is because we think it makes business sense to get closer to them and because of the maturity of their implementation. The choice of Discourse is because we use it as the forum for the XWiki open source project at https://forum.xwiki.org and an integration with Discourse would thus be interesting.
-
UC1: Personal Following/Followers
- Integrate into XWiki’s existing follower/following feature. Also allow each XWiki user to follow the activity of external users from external services. This means modifying the XWiki profile UI to be able to enter the external user handles.
- Contribute the activity done by users you follow in your XWiki Activity Stream.
- Support this use case for external XWiki instances + NextCloud + Discourse (provided their implementation is good and ready to be used)
-
UC2: Global Following & Content viewing
- Add the ability for an XWiki admin to enter users to follow at the level of the whole XWiki and contribute their activity to the general Activity Stream of XWiki.
- Be able to not only follow the activity but also view the related content from inside XWiki (created content, modified content, etc).
- Support this use case for external XWiki instances + NextCloud + Discourse (provided their implementation is good and ready to be used)
- Performance of content viewing. For performance reasons, the first time some content is fetched remotely, it would be stored locally on the XWiki server so that subsequent queries to view content would serve it locally until the content is modified again remotely.
-
UC3: Expose XWiki Content
- Expose XWiki pages as AP endpoints so that they can be viewed from external services (or other XWiki instances).
-
UC4: Messaging
- Integrate with XWiki messaging feature so that messages can be posted to external users and not just internal users as it’s currently the case.
For all use cases:
- Support integration with external XWiki instances. Ideally NextCloud and Discourse would also be able to display XWiki info in their UI but it depends on their willingness to do so and we can’t commit on this.
Future use cases (not planned for this first grant. Candidates for future work):
- UC5: Integration in XWiki’s notifications
- Display external events and content inside XWiki’s notifications system.
- UC6: Federated search
- Integrate into XWiki’s search feature and return results from XWiki nodes and other software connected via ActivityPub.
- UC7: External modifications
- Be able to also edit XWiki content externally and not just view it. It’s more complex and requires handling conflicts and merge.
- UC8: Cryptpad integration
- Integrate and implement AP also in Cryptpad so that XWiki and CryptPad (and the rest of the world) can share activity and content.
- UC9: Blog Integration
- Integration of AP in the XWiki Blog Extension to list blog posts from followed users in the blog view. When creating new blog posts, also publish them with AP so that they can be viewed in other platforms, such as WordPress. Add extension points inside XWiki for implementing this cleanly.
- UC10: More Integrations
- Increase the number of external services XWiki integrates with by testing and adjusting the XWiki AP implementation depending on how these services have implemented the AP specification.
- UC11: Advanced messaging
- Support of message threads and support the sharing of partial conversations.
Advantages
- Increase search and discovery on the internet by increasing collaboration on content and activity from the growing list of services implementing AP. XWiki’s content/activity become available to all services implement AP and the content/activity from these external services become available from within XWiki.
- Implement the XWiki federation by allowing decentralization for XWiki and the ability to achieve synchronization of different instances of XWiki using AP.
- Make XWiki a nicer web citizen by integrating with other services. It also provides more integrations of services for XWiki and thus makes it simpler for users to cross software boundaries and not be locked to a single software.
- Generally speaking, promote decentralization and federation in the open source at large.
It should also be noted that while there are a number of solutions already implementing at least partly ActivityPub, there is no wiki engine currently implementing it (the closest to the wiki approach might be Dokieli, but it has only basic wiki features). This would make XWiki a pioneer and possibly inspire others. The recent Wikipedia outage shows that a new generation of solutions is needed for enhanced resilience. Decentralized wikis can be a good solution.
Past Experience
Have you been involved with projects or organisations relevant to this project before? And if so, can you tell us a bit about your contributions?
XWiki SAS (the main company sponsoring the development of the XWiki open source software and of Cryptpad) has participated in several research projects in the past that are are relevant to this domain:
The outcomes of the project will be published as an XWiki Extension and thus installable and usable immediately (through XWiki’s Extension Manager) by the 7000+ current active installations of XWiki over the world.
Comparison
Compare your own project with existing or historical efforts.
Compared to the experiments done in the “Past Experience” section above we believe that AP is more interesting as it’s a standard specification that applies to all content/activity software. Once we implement the specifications in XWiki, this makes XWiki interoperable with all other software implementing AP too (In practice this won’t be completely true since the specifications leaves some areas underspecified and open but it makes partially true and the remaining work to make it work with a specific software is marginal). If AP catches on and more and more software/services implement it, it would be a game changer for interoperability over the internet and a win for open standards and for XWiki.
Challenges
What are significant technical challenges you expect to solve during the project, if any?
The main challenge we foresee are differences in the various actor’s AP implementations. This can happen for 2 reasons: the AP specification is underspecified in some areas and thus leaves it open for interpretation (such as authentication) and it’s also extensible and thus if we want to benefit from the extension inside XWiki we would need to discuss and interact strongly with the actors to understand how they extended the AP formats.
Ecosystem
Describe the ecosystem of the project, and how you will engage with relevant actors and promote the outcomes?
We propose to interact and integrate with at least 2 other actors (NextCloud and Discourse) to make sure XWiki’s AP implementation and these actor’s AP implementation work well together.
We will also promote search and discovery and AP virally through the fact that XWiki is open source and well known and used over the world but also through XWiki SAS’s regular technical conferences attended every year around XWiki’s technologies (XWiki SAS does between 10 to 20 events annually at various conferences).
Attachments
Attachments: add any additional information about the project that may help us to gain more insight into the proposed effort, for instance a more detailed task description, a justification of costs or relevant endorsements. Attachments should only contain background information, please make sure that the proposal without attachments is self-contained and concise. Don’t waste too much time on this. Really.