New Filter Streams Extension: GitHub Importer

Development
, ,
21

Thank you!
Yes I will do what is requested :slight_smile:

22

Hi @vmassol I am going to start the project. What I know so far:

  1. Request a repo under XWiki-Contrib.
  2. Request for JIRA project.
  3. How should we name (?) :
    groupId = org.xwiki.contrib
    artifactId = ghimp / githubimp / githubwiki / ghwiki
    contrib = githubwiki / github-wiki / ghwiki / githubimporter / githubimp / ghimp
    (I saw this Contrib Naming but filter streams application extensions are using simple naming like: mediawiki, dokuwiki etc.)
  4. Manual builds first, CI / Snapshot builds will start later when requested by mentor.

After:
Creation of Contrib repo, JIRA project:

  1. Start the project on local machine.
  2. Follow JIRA issues, create the extension through the issues.
  3. Development is tracked through JIRA.

After:
Basic functional requirement completes:

  1. Create README.md on the repo.
  2. Create extension page on XWiki.
  3. Update documentation on the extension page.

After:
All required use cases complete:

  1. Prepare the release.
  2. Update documentation with Release Notes.

Please correct me and let me know when I can start the work.
Thanks!

23

I’m proposing:

  • GitHub repo name: application-github-importer
  • group id: org.xwiki.contrib
  • artifact ids: github-importer-* or application-github-importer-*

Note: https://contrib.xwiki.org/xwiki/bin/view/Main/#HChoosingthename is about the repository name, not the artifact ids (there are several artifacts that will be produced, at least 6).

We should put it directly under the CI. It takes 3 lines to do so and about 15 seconds total so we should do it! :wink: That allows to see when the build is broken, it should always pass at all time.

You haven’t started already? :slight_smile: You should do it ASAP. You’re responsible for the development so if no work is done it’s your responsibility! You’ll be judged on the result. That’s very important to understand.

Yes sounds ok except that it’s missing the most important IMO:

  • Start by creating all JIRA issues you can imagine for the project
  • Define roadmaps of 15 days, this means releasing a new version of the github importer every 15 days. 1 month would be too long since the GSOC is only about 2-3 months and we need more releases. So please propose release dates from now.
  • For the first release date, please list all the JIRAs that you can implement from the full list of jiras that you’ll have created.

Please try to provide this roadmap in the coming 1-2 days.

I’m now creating the github project + jira project.

Thanks!

24

Done:

I’ve also invited you to xwiki-contrib on github so that you can push directly.

You can start pushing code, have fun! But first the list of JIRAs + the roadmap :slight_smile:
-Vincent

25

Thanks @vmassol for creating the repository and JIRA.

Thanks a lot!

Actually, I did start working locally.

Ok, got it.

I have created a several issues on the JIRA project.

Sure, I will provide you a roadmap in a day or two.

Thanks!

26

Thanks. The jira issues seem to be too technical and fine-grained. We need end user use cases not technical ones. For example Loading... is really a technical detail. The issue should be about “Ability to import GitHub Pages from a Git repository” and not “Append .wiki automatically” which doesn’t mean much and is a technical detail.

Also “GitHub Importer Wiki Input Filter” for Loading... is not good. The issue should be “Ability to import GitHub pages from github.com”. Etc…

Could you review and fix the issues? Thx

27

Done. Thanks for the corrections.
I was following the naming by looking at MediaWiki and DokuWiki.

I see why it’s better to have them closer to user use cases. So I’ve fixed the names and descriptions now. Let me know if not good.
Thanks.

28

So, the first Roadmap of 15 days:

Roadmap 1:

  • Issues to be resolved:
  • Requirements to be implemented:
    • 1

I am mainly focusing on the primary use case: i.e. The user will be able to go to Filter Streams Converter and choose github+wiki instance input and XWiki instance output and get the GitHub Wiki into XWiki. So the basic use case will be completed.
I will let you know when to put it under CI (soon this coming week). First I was thinking to put it under CI after first release.

29

IMO this is too ambitious. A first basic version of GHIMPORT-1 is already good.

I don’t understand this. The JIRA issues are the use cases.

So it’s missing the target date. I propose a target date of June 22nd for a released v1.0.

WDYT?

30

Note that I prefer that you are under-optimistic than fail to deliver. You’re setting expectations here! Now if you do more then it’ll be awesome. What’s important is quality work. No shortcuts. Everything documented. Good and clean code, following xwiki best practices. Tests.

31

Seems good.

I mean the requirements posted here in the OP, seems like mixed again with use cases :slight_smile:

Oh I see, I was actually keeping “It works” phrase in mind. I have updated to GHIMPORT-1 only. I wonder about tests, just letting you know I will first test manually, then create the tests. I want to see if my code works or not and creating tests takes time. I would ask a lot of your help on creating tests.
Edit 1: I see DokuWiki (similar GSoC project) is doing Integration Tests.

32

It takes time to learn it, but it’s much faster than manually actually. And actually it doesn’t take that much time, you can have a working functional test setup under 15mn when you know how to do it (it won’t do much but it’ll allow you to automatically package a minimal xwiki version with your extensions deployed in it which you can then test manually or automatically with a few lines).

1 Like
33

I have no idea about the quality of the DokuWiki extension. It’s not supported by the xwiki dev team. You can have a look at the LaTeX extension one for ex.

34

And ofc check the doc at https://dev.xwiki.org/xwiki/bin/view/Community/Testing/DockerTesting/

35

Thanks, let me see.

36

Hi @vmassol, I have commited some additions on the extension repository, ignore the testing code as they have not been properly created.
Now I am going to add the extension under CI.

37

@haxsen Could you make sure that you always set the “'fix versions” in JIRA for the upcoming release?You need to set the “fix versions” in JIRA for the issues planned for 1.0,

1 Like
38

Yes, sure :slight_smile:
(I thought we only do it for next versions and not first release, but now I know)
Thanks.

39

Thanks I’ve set the date for 1.0 release to 22nd of June, see Error - XWiki.org JIRA

Don’t hesitate to do more than GHIMPORT-1 if you can.

Let me know also when there’s something ready to build/test.

Thanks!

1 Like
40

Update: I am trying to create tests using docker but unable to test the examples shown at DockerTesting. I am missing something so I would like to have some help.
What I have done so far:

  1. Installed Docker (took a long time due to many issues appearing one after another).
  2. Installed Kitematic for quicker UI usage.
  3. Downloaded and ran these containers:
    • jenkins
    • openjdk
  4. Set up docker in IntelliJ and successfully connect it.

The introductory talk in french on YT shows an example test done live but the main problem is, I do not have the run button in IntelliJ like the video shows (which means my setup is probably incomplete).
Edit 1: Ok I found XWiki container on docker, downloading it.