AI in XWiki brainstorming

This new content could be automatically generated, in a natural language, by a very powerful IA.

Any news on this subject? There are a lot of advances made in the last 4 1/2 years

https://cloud.google.com/natural-language

yes, this is why I’ve added new posts on this thread a few days ago. I don’t know anyone who has implemented any AI-based feature on XWiki so far though. FYI, I’ve proposed an AI topic for the 2023 GSOC at XWiki @ Google Summer of Code 2023 - XWiki and Michael has proposed one too. If you’re willing to help out, propose some topic and mentor some GSOC student let us know.

While there is nothing necessarily wrong with brainstorming, I guess my concern with this thread’s approach to integrating AI into XWiki is not evaluating the motivation behind the addition of AI. You have to ask yourself the following: Is there an actual problem that exists in XWiki today where AI is the optimal solution, or is the motivation just to wedge a buzzword into XWiki? If a brainstorming session has to last for years in order for everyone to convince themselves that AI should be a part of XWiki, then this may not be the right way to approach this situation. Instead, it seems like a better approach would be to use the XWiki Jira tracker to identify issues that the users want solved, and then ask if AI makes sense as the most effective and elegant solution to the Jira issues. Otherwise, you merely have an AI “solution” that’s forever in search of a problem.

Not trying to disrupt the conversation here, but I just thought I would point out that most users don’t necessarily care about how a product works under the hood but instead care about how the product’s features can help them accomplish their goals.

Interesting idea but I don’t think that works re jira for one reason: users mostly report issues for bugs and small improvements. They rarely create issues for big ideas or feature needs. Those are usually created in forum posts and don’t get put often into jira issues.

The reason nothing has been implemented so far in this thread is for 2 reasons IMO:

  • AI is not easy to implement and requires knowledge. There’s nobody with such knowledge in the core dev team and in the contributors. What’s interesting nowadays is that there are more AI libraries/frameworks that can make it easier to implement some AI stuff without mastering the details.
  • No users or companies have been willing to sponsor any AI development in XWiki so far.

I still find it interesting to discuss AI ideas in XWiki. First, it reminds everyone that AI exists and what it could do for XWiki. That’s interesting for anyone willing to sponsor a feature. It’s also interesting for developers to start researching AI topics. The proof is that thanks to this thread, we’ve now proposed 2 AI projects for the upcoming GSOC. I hope we’ll find good enough students to implement them!

Now, to conclude, I agree with you that the best is to find real life use cases from user’s feedback (which some of the items proposed in this thread do, but not all for sure).

Definitely, this is a thread mostly for developers, even if it can sparkle interest for some users/companies who could be interested to sponsor implementing some of the ideas. Note that the idea of hiding AI goes against what Notion AI is proposing for example since they’re marketing AI directly : Notion AI

Thanks for your point of view!

While this is true for bug reports, any improvement request (or group of related improvement requests) would be an opportunity to evaluate a potential application for AI. So those opportunities may be something good to look for.

You’re welcome. I have seen software projects completely derailed by goals like “let’s add AI” or “let’s add blockchain” when the users don’t necessarily care and when a less complicated solution may suffice. So while there may in fact be a good use for AI somewhere within XWiki, I’m just trying to encourage maintaining the proper perspective when it comes to adding new technologies to a project like this.

Sure, do you have examples to provide? (i.e. sets of existing jira issues where you can see that implement some machine learnings/AI would solve the need nicely?).

For example users may say something like the search doesn't return results if I don't prefix a part of a word with *. Ofc we could imagine some AI that would understanding what the user means even with a partial word. However, a simpler solution would be to implement google-like search and that doesn’t require any AI. Similarly, users may say: I’d like that search results use synonyms. That’s also doable using solr and synonyms but doesn’t necessarily means using AI. Ofc using a language-recognition AI (such as chatgpt) and training it on a user’s wiki content could achieve amazing results but it’s far from trivial (especially as the AI needs to be trained on the user wiki content and the search needs to work locally without outside internet connexions). These are ideas 12 and 25 above BTW :wink:

Idea 42: When saving a page, propose an auto-correction and rewrite of the content. See for example chatgpt (asking it to fix some text for grammar and corrections) or DeepL Write: AI-powered writing companion

Hello,

In the last weeks, I have been following the evolution of Chat-GPT and also the Open Source GPT Technologies to try to understand how important they are for the XWiki software. I believe that we need to look into this a bit deeper and see if there are already valuable things we can do for example using Chat-GPT itself in a similar way that Notion is using it. However I believe we would also need to look at if it’s possible to provide a pure “Open Source” solution for this (It seems a possible major problem if a lot of data from tools such as XWiki or other is flowing to the same central place), and also more advance ones that involve learning of the data of the wiki:

Therefore XWiki SAS is going to have an intern to explore these technologies in May-August. There is also a project on the Summer of Code. In order to prepare for this I’ve created a design page on xwiki.org where I have:

If there are any persons in the community that are interested in this subject and would like to be involved, do not hesitate to tell us. We could open a public matrix chat to open the discussion more. Also if you believe there are relevant examples or technologies, do not hesitate to update the wiki pages.

In any case, the goal will be to publish the results in the open.
Ludovic

1 Like

Maybe add GPT4All to the list, see https://twitter.com/andriy_mulyar/status/1640836003194630144 ?

Done. I’m testing it.
I’ve also added the posts from NextCloud about what they announced:

NextCloud has launched AI

https://nextcloud.com/blog/hub-4-pioneers-ethical-ai-integration-for-a-more-productive-and-collaborative-future/

Also includes the “Ethical AI Rating” in order to tell users which AI features are based on components that are Open (both the code, the data sets or the end weight of the model), and whether it’s possible to self-host them. This is important because AI systems have risks, such as privacy and security of data, discrimination and bias or energy usage. In order to solve these risks, Open access will be key. This concept is very interesting and it would make a lot of sense to join them and do something similar and ideally the same.

More info at: Nextcloud Ethical AI Rating - Nextcloud

Idea 43: Generate transcripts for video embeds. It can be really time consuming to generate transcripts and current AI solutions generate “correct” results (better than nothing :zipper_mouth_face:).

1 Like

Given the current rate of advancement in the field I think it’s not too crazy to go a step further with the generalization. Touching upon ideas proposed earlier along with some possible but not fully tested approaches:

Idea 44: XWiki personified

Enabling it to:

  • Talk (XWiki as a platform and/or the knowledge within)
    – Fine tuning one of the many LLMs available today (or tomorrow) on the available documentation, code, forum discussions etc. to imbue it with understanding of what XWiki is, what it can do and what users usually struggle with (PEFT/LoRA)
    – Maybe provide a way for further fine tuning on the knowledge within
    – Using vector databases for fast similarity checks and/or giving it access to solr (check out stuff like faiss
  • See - images and maybe even auto-taken screenshots of any rendered page ( see SAM )
  • Do (tasks)
    – Something akin to AutoGPT/BabyAGI/AgentGPT
    – Using frameworks like LangChain (or others) to provide it with access to the XWiki API as a tool ( with user validation prior to executing a proposed set of actions )
  • Speak (optional - by tts)
    – Providing a hands free way of interacting with the platform
    – Things like bark or uberduck
  • Generate (content - based on the context of the wiki combined with prompts form the user)
  • Research (additional information gathering from external sources)
  • Auto-repare (to a certain extent)
    – Given access to logs and the information about the general setup + maybe a bit of clever prompt engineering for guidance, it may be able to diagnose itself, propose solutions and execute them (again with the use of something like AutoGPT)
  • Code
  • Give it contextual or selectable personalities

The main idea being abstracting most of the complexity by creating a natural language interface between the user and the platform, truncating the learning curve needed to use and manage an instance.

1 Like

Idea 45: Implement an equivalent of GitHub copilot but when editing a wiki page and typing scripting content.