Improving onboarding of developers

Development
#1

Hi everyone,

I’d like to discuss this topic again (yes we talked about it in the past and worked on some actions to improve it, especially when we did the Google CodeIn). It would be great to see if we can improve that further.

@gcoquard since you joined recently, you have this fresh in your mind. It would be great if you had any idea about the things we could do to make it simpler for newcomers to be productive in xwiki dev. Do you have any ideas?

Some ideas/questions to bootstrap the discussion:

  • Start on a contrib project instead of the core
  • Apply PRs faster, even when they’re not perfectly correct and have the newcomer fix non-logic issues later (accepting to increase the technical debt temporarily - hopefully). By non-logic, I mean for ex: code style, missing tests, missing refactorings, not perfect code location, etc. How do we implement this in practice? (if we want it).
  • Assign a mentor and have that mentor talk a few minutes every day to discuss progress and issues.
  • Any missing doc that would help a lot if it were there?
  • Any code architecture/design or patterns that are too complex when they could simpler? which ones?
  • Make it simpler to setup one’s own dev environment? @gcoquard Was that a problem for you? Did it take long? What could we provide?
  • Do more refactoring and take more time to improve hard to use/understand APIs. Which ones?
  • Improve deployment speed of code changes to test it. In JS you just need to edit the source code and refresh your browser and you can see the changes immediately. I think being able to mvn install an extension and then in XS to “reload the extension” would already help a lot. WDYT?

I’m looking for tangible actions that we could implement.

Let’s start the discussion!

Thanks

#2

One question I’m asking myself is whether we have too much scaffolding code required whenever we need to add logic in XWiki, or not. Any thoughts on this?

One reason I’m asking this is because some developers like to code stuff that do things and not code the scaffolding part which is less interesting to them.

#3

Some pain points that comes immediately to my mind on the subject:

  1. Velocity. Yeah we all know about it, but IMO that’s the first problem for bringing up new devs, and that’s the first thing that makes difficult to contribute, even for a simple contrib app.
  2. Huge code base and not so easy to find doc about it: it takes time to understand where you should look at when you want to fix something. I think it’s partly because of velocity that is not helping code navigation (+ you need to understand to look on the templates located either in web or skin module), but also because we might miss some mapping between our Maven modules and our extension doc. We should probably be more exhaustive in this mapping.
  3. Old frontend technologies: also something we know about, but also a difficulty IMO, it’s easy to contribute backend, but for front you have to rely on old tech and our documentation is a bit partial on that part I think. Developing a simple app that performs REST requests to write stuff and refresh a DOM element is not so easy and we might miss a tutorial for that for example.

So for 3 we’re slowly improving things, I think 2 could be easy to address (but takes time), and I don’t have a simple solution for 1.

#4

Note that we have Contributions pain points (Community.ContributingPainPoints) - XWiki even if it’s not been updated in a while.

#5

Hi,

First of all, XWiki is an interesting project to work in as a developer. It allows to explore a lot of things.

  1. But the first thing I struggled with was to find the good informations to help me go through some tasks/problems. Using XWiki internal search kept me looking for things when the good keywords combination on Google gave the expected results much faster as @vmassol showed me multiple times. Thus, it is also a consequence of the scattering of these informations. The key informations are difficult to find and as a matter of fact Contributing Documentation shows the multiple places where you may check for informations. So, maybe unify and centralize the documentations to make finding the right informations faster.

  2. Also, XWiki, as @surli said, is a huge code base, the type I didn’t already experienced before, XWiki being my first experience in terms of big project contribution. Understand the tricky things on which relied the code I was editing was a bit painful. That said, my Java knowledge and understanding of key Java concepts used in XWiki was too low, the needs were higher than what I was expecting and I kind of overestimated my capacities to adapt to these needs. In the same row, Velocity didn’t ease the things. In that field, having a mentor is a good way to address part of these problems and working closely with an experienced one in the first moment of contribution is surely time consuming, but also it ensures to remove doubts and wrong thoughts/ideas about how to do things and in my opinion will benefit in shortening the onboarding process.

  3. Keep being strict and rigorous on PR, I think it’s the right thing to do. However writing tests is maybe to move out of the roadmap time for newcomer. It’s on the tests that I spent too much of my time and the multitude of way to instantiate XWiki bring a lot of source of issues in setting up the right test environment. Also, good to know, working on XWiki testing (integration tests) needs knowledges about Docker.
    Also, we discussed a lot where to move/put code and even after we had decided something we’re discussing again this. This is not bad itself, but in my opinion, we need to establish a more specific policy about project, modules and APIs organization in order to avoid discussing again and again.

  4. Regarding the build and the different way to run XWiki, it has to be more clear regarding Maven, which profiles should be used and how the JAR replacement has to be done for example. Then, regarding Docker, there’s also multiple ways to initiate an XWiki instance and the whole thing ends being a bit confusing to me. By the way, your last point would really help a lot, for sure.

  5. Also, having to work directly on security related code of a big project, using a programming language I used only a few times during my studies was not the greatest idea I had this year and intricacy of this work was too much from this point of view so I can’t really say if patterns were too complex or just necessary for what we used them.

#6

Could you provide an example of what we should do precisely in your opinion. We already try to centralize documentation so I’d like to see some tangible ideas for improving this further.

I’m not sure I understand the scattering point WRT https://dev.xwiki.org/xwiki/bin/view/Community/Contributing#HDocumentation. For me we provide links to different topics. I don’t think a single page with snippets + user doc + admin doc + dev doc for all features of XWiki would be a great idea. Yes it would be centralized, but it would a major pain to read for someone who is a user or an admin or a developer, don’t you think?

On this topic, what we’re trying to do (but slowly, it’s a lot of work) is to stop dividing the doc by user type but by topics. This means that each extension page on e.x.o is supposed to contain user doc + admin doc + dev doc. Is this going in a good direction or not? But snippets or FAQ would remain separate as they are cross-topics.

Why is that? Velocity is the simplest possible language with a 4-5 keywords, and that’s all. How did it bring complexity in your opinion, I’m curious. Basically all it does is call Java APIs.

How come you spent so much time on testing, especially with the help of #xwiki chat channel?

Tests are not optional so I don’t think I’d agree to not require them. Also unit tests are not just for testing, they’re helping for designing the code and they act as a user for APIs. Telling new developers that testing is optional is a very bad signal IMO. They’re not optional. They’re even more important than the code IMO because adding a feature is nice but adding a not working feature (or a feature that will stop working when code is changed) is really bad as it frustrates XWiki users and the overall perceived quality of the product. Said differently, I feel it’s better to be slower but with more guarantee that the code works. Now I appreciate the frustration that can results for it, and that coding takes longer overall (initially) but I feel that’s what it takes to be a real developer. Otherwise it’s called hacking :wink: Also my experience is that if tests are not written right with the code, they’re never written and if they are it takes x2 time to write them since by the time either you forgot your code and need to read it again and understand it again, or someone else who didn’t write the code will write the tests and it’s much harder since that person doesn’t know the code.

So I’d prefer to devise ways to make testing simpler than saying it’s optional and can be postponed.

Why is that? AFAIK you don’t need any docker knowledge since we encapsulate docker with our test framework. Could you explain what you mean?

Funnily, this is exactly what we’re doing (discussing specific policy about project, modules and APIs organization in order to avoid discussing again and again) and why it’s taking time :wink:

Now I agree that we should do it in 2 steps as much as possible when backward compat is not broken. When backward compat will be broken with code moves then it’s another matter but it could still be done in 2 steps with the @Unstable annotation:

  • step 1: commit the changes
  • step 2: do the refactoring (make sure it’s planned and not skipped though ;))

WDYM? What is missing from https://dev.xwiki.org/xwiki/bin/view/Community/Building/#HBuildingwithMaven ? Where did you struggle?

WDYM? There’s a single way AFAIK.

Thanks a lot for your feedback Guillaume! I’d like to act on what you said, which is why I’m asking for more details. It’s one thing to state generic issues, but we need precise points to improve and suggestions to improve them or this whole thread will remain just some ideal wish and it won’t lead to anything.

Keep it coming!