New Filter Streams Extension: GitHub Importer

Development
, ,
#23

I’m proposing:

  • GitHub repo name: application-github-importer
  • group id: org.xwiki.contrib
  • artifact ids: github-importer-* or application-github-importer-*

Note: https://contrib.xwiki.org/xwiki/bin/view/Main/#HChoosingthename is about the repository name, not the artifact ids (there are several artifacts that will be produced, at least 6).

We should put it directly under the CI. It takes 3 lines to do so and about 15 seconds total so we should do it! :wink: That allows to see when the build is broken, it should always pass at all time.

You haven’t started already? :slight_smile: You should do it ASAP. You’re responsible for the development so if no work is done it’s your responsibility! You’ll be judged on the result. That’s very important to understand.

Yes sounds ok except that it’s missing the most important IMO:

  • Start by creating all JIRA issues you can imagine for the project
  • Define roadmaps of 15 days, this means releasing a new version of the github importer every 15 days. 1 month would be too long since the GSOC is only about 2-3 months and we need more releases. So please propose release dates from now.
  • For the first release date, please list all the JIRAs that you can implement from the full list of jiras that you’ll have created.

Please try to provide this roadmap in the coming 1-2 days.

I’m now creating the github project + jira project.

Thanks!

#24

Done:

I’ve also invited you to xwiki-contrib on github so that you can push directly.

You can start pushing code, have fun! But first the list of JIRAs + the roadmap :slight_smile:
-Vincent

#25

Thanks @vmassol for creating the repository and JIRA.

Thanks a lot!

Actually, I did start working locally.

Ok, got it.

I have created a several issues on the JIRA project.

Sure, I will provide you a roadmap in a day or two.

Thanks!

#26

Thanks. The jira issues seem to be too technical and fine-grained. We need end user use cases not technical ones. For example https://jira.xwiki.org/browse/GHIMPORT-2 is really a technical detail. The issue should be about “Ability to import GitHub Pages from a Git repository” and not “Append .wiki automatically” which doesn’t mean much and is a technical detail.

Also “GitHub Importer Wiki Input Filter” for https://jira.xwiki.org/browse/GHIMPORT-1 is not good. The issue should be “Ability to import GitHub pages from github.com”. Etc…

Could you review and fix the issues? Thx

#27

Done. Thanks for the corrections.
I was following the naming by looking at MediaWiki and DokuWiki.

I see why it’s better to have them closer to user use cases. So I’ve fixed the names and descriptions now. Let me know if not good.
Thanks.

#28

So, the first Roadmap of 15 days:

Roadmap 1:

  • Issues to be resolved:
  • Requirements to be implemented:
    • 1

I am mainly focusing on the primary use case: i.e. The user will be able to go to Filter Streams Converter and choose github+wiki instance input and XWiki instance output and get the GitHub Wiki into XWiki. So the basic use case will be completed.
I will let you know when to put it under CI (soon this coming week). First I was thinking to put it under CI after first release.

#29

IMO this is too ambitious. A first basic version of GHIMPORT-1 is already good.

I don’t understand this. The JIRA issues are the use cases.

So it’s missing the target date. I propose a target date of June 22nd for a released v1.0.

WDYT?

#30

Note that I prefer that you are under-optimistic than fail to deliver. You’re setting expectations here! Now if you do more then it’ll be awesome. What’s important is quality work. No shortcuts. Everything documented. Good and clean code, following xwiki best practices. Tests.

#31

Seems good.

I mean the requirements posted here in the OP, seems like mixed again with use cases :slight_smile:

Oh I see, I was actually keeping “It works” phrase in mind. I have updated to GHIMPORT-1 only. I wonder about tests, just letting you know I will first test manually, then create the tests. I want to see if my code works or not and creating tests takes time. I would ask a lot of your help on creating tests.
Edit 1: I see DokuWiki (similar GSoC project) is doing Integration Tests.

#32

It takes time to learn it, but it’s much faster than manually actually. And actually it doesn’t take that much time, you can have a working functional test setup under 15mn when you know how to do it (it won’t do much but it’ll allow you to automatically package a minimal xwiki version with your extensions deployed in it which you can then test manually or automatically with a few lines).

1 Like
#33

I have no idea about the quality of the DokuWiki extension. It’s not supported by the xwiki dev team. You can have a look at the LaTeX extension one for ex.

#34

And ofc check the doc at https://dev.xwiki.org/xwiki/bin/view/Community/Testing/DockerTesting/

#35

Thanks, let me see.

#36

Hi @vmassol, I have commited some additions on the extension repository, ignore the testing code as they have not been properly created.
Now I am going to add the extension under CI.

#37

@haxsen Could you make sure that you always set the “'fix versions” in JIRA for the upcoming release?You need to set the “fix versions” in JIRA for the issues planned for 1.0,

1 Like
#38

Yes, sure :slight_smile:
(I thought we only do it for next versions and not first release, but now I know)
Thanks.

#39

Thanks I’ve set the date for 1.0 release to 22nd of June, see https://jira.xwiki.org/projects/GHIMPORT/versions/18554

Don’t hesitate to do more than GHIMPORT-1 if you can.

Let me know also when there’s something ready to build/test.

Thanks!

1 Like
#40

Update: I am trying to create tests using docker but unable to test the examples shown at DockerTesting. I am missing something so I would like to have some help.
What I have done so far:

  1. Installed Docker (took a long time due to many issues appearing one after another).
  2. Installed Kitematic for quicker UI usage.
  3. Downloaded and ran these containers:
    • jenkins
    • openjdk
  4. Set up docker in IntelliJ and successfully connect it.

The introductory talk in french on YT shows an example test done live but the main problem is, I do not have the run button in IntelliJ like the video shows (which means my setup is probably incomplete).
Edit 1: Ok I found XWiki container on docker, downloading it.

#41

@haxsen So your steps 2 to 4 are not needed.

Same you don’t need this:

All you need is to follow https://dev.xwiki.org/xwiki/bin/view/Community/Testing/DockerTesting and as I’ve told you already to mimick what’s done in the LaTeX extension’s functional docker tests. Did you try that? If so what’s the problem you encountered?

Thanks

#42

Hi @vmassol, the LaTeX extension’s tests are failing on my end by doing mvn clean install or mvn test. What I did:

  1. git clone https://github.com/xwiki-contrib/latex
  2. mvn test or mvn clean install

So I am following xwiki-platform-mail which has no test failures on my end and as mentioned on the DockerTesting page in Full examples.
About my previous post: I was asking how to run the tests, what do I need to do with docker after installing locally and how to have the Run Test button like you are having on the youtube video. I know I am lacking some configurations that’s why I asked since I have no idea what I am missing and the page https://dev.xwiki.org/xwiki/bin/view/Community/Testing/DockerTesting doesn’t mention about it. There’s also a lot of information on that page which mixes up a lot of things and creates confusion. (Edit1) And there’s also no FLV video files in the target directory when I do mvn test on xwiki-platform-mail.
So today, I was learning the xwiki-platform-mail tests and I was running the tests by mvn test but it would be better if I can test inside IntelliJ IDEA. (Edit2) It’s not even recognizing what @UITest is.
Let me know if I am missing something or am unclear about my post.