Re: [GNUnet-developers] Discussion, and Help Wanted: Moving to Gitlab for Git, CI, and Issues

[ng0]

Devan C. dvn transcribed 6.5K bytes:
> Hello my fellow GNUnetians,
> 
> As some of you know, I have been pushing for and working on getting us
> to CI/CD system based on Gitlab CI. This is pretty much ready to start
> using and building out the pipelines for now.
> 
> In a previous thread[0] there was a decision to give Gitlab a try, as
> more than just our CI system, and to migrate our repos from Gitolite to
> our own Gitlab instance.
> There was less agreement regarding the idea to move from MantisBT to
> Gitlab's issue tracker, but in the end we decided to try importing all
> of the >2k Mantis tickets into Gitlab.[1]
> 
> This email is a status update on the overall progress of moving to
> Gitlab.

Cool, and thanks for your work on this :)
 
> Current Gitlab status:
> 
> - Gitlab is running and accessible at `https://gitlab.gnunet.org`

So here's a problem I see with this as it is right now:
I'm a git admin. Before I give people a certain kind of access, be
it for one repo only, a range of repos or the group 'gnunet', I
have a sort of checklist. Can I digitally verify to some extent that the
key sent to me matches the person? Do we have a CAA signature? etc.
Now I see already one name as 'Owner' in the gnunet group who, to
my knowledge, has never signed anything. Correct me if I'm wrong
about ic.rbow.
We can only trust each other.
Since we have this CAA in place, we need more than trust, we need
some guidelines when someone is added to which permission level
in gitlab.
Previously the communication about what happened, which steps
were followed and that there is a new committer, were betwee
1 or 2 people involved in administration. Now potentially everyone
can do this, which is either bad or good, so at the very least
we need to communicate about new rights given.
 
> - Registration is open. There are no guarantees on uptime, or even
>   data retention (though I don't expect data to disappear).
> 
> - wldhx has kindly offered two "Gitlab Runners" for running CI jobs.
>   These will be added as shared runners, to be used by any projects on
>   the instance. This may be changed later to only be shared by projects
>   under the GNUnet namespace.
> 
>  **TODO**
> 
> - Setup email. Used for registration, password resets,
>   notifcations, and interaction (eg. issue threads).
> 
> - Currently run using containers with docker-compose. Will switch to
> using systemd services to with the containers, removing docker-compose.
> 
> - Daily remote backups. Perhaps 'firefly.gnunet.org' could be the backup
>   site, hmm?
> 
> - Change configured hostname (in Gitlab) to 'gitlab.gnunet.org'.
> 
> - Setup redirect from 'git.gnunet.org' -> 'gitlab.gnunet.org'
> ----
> 
> Current [MantisBT -> Gitlab Issues] status:
> 
> Exporting:
> - Mantis can export to CSV and "Excell" XML
>   - These do not contain comments (bugnotes). It looks like there might
>     be a possibility to enable them via a configuration option[2]. Not
>     sure who all has admin access, whom I could coordinate with. Maybe
>     easier if I can get admin rights? Grothoff, what do you think?
> 
> Importing:
> - I have found only a couple of scripts[3,4] for this. They are both out
>   of date, for old versions of both softwares. I have tried both to no
>   avail. [4] is the most promising; It's not so old.
>   I would really appreciate any help working on this.
> 
> I suppose this means that we will continue to use Mantis, and disable
> issues in Gitlab for now. Any protests or ideas?
> ----
> 
> Migrating from Gitolite:
> 
> For those whom are not aware, we currently use gitolite for all of the
> lovely repos in our collection. We will need to copy all of the repos to
> Gitlab, as well as duplicate permissions, and setup githooks.
> 
>  1. Create namespaces/groups on our Gitlab
>  
>  2. Clone repos. This can be done via the web interface "Import" option
>  when creating a new repo, or the new remote can be added, and then
>  pushed. The little script found here can help with getting all the
>  repos from Gitolite[5]
>  
>  3. Setup redirects. eg.  https://gnunet.org/git ->
>  https://gitlab.gnunet.org
>  
>  4. Manually replicate permissions. Will need a Gitolite admin's help
>  on this.
>  
>  5. Setup githooks. We have quite a few githooks setup, so we will 
>  need to recreate those.
> 
> After all of that is done, I think we should be ready to switch over
> to Gitlab for at least the git management and CI/CD.
> ----
> 
> That brings us to the final update: The CI System...
> 
> - We have a couple of small runners (thanks wldhx).
> 
> - We have some very basic '.gitlab-ci.yml'[6] files, defining jobs.
>   - I will begin expanding these in the coming weeks.
> 
> **TODO**
> 
> - As we build out a matrix of pipelines, we will need more resources.
> 'firefly.gnunet.org' is a logical option. In the past I've seen it
> utilized heavily by experiments. As long as we are okay with dedicating
> some CPU and RAM to runners, then I will begin setting them up.
>  
> - Setup Gitlab Container Registry [7] for storing our CI artifacts.
> 
> - Expand our '.gitlab-ci.yml' files to include e2e tests, builds for
>   multiple architecures, and continuous delivery of packages for various
>   package managers.
> ----
> 
> Wow, so that's a lot of text. A lot of people have been asking me about
> the status of Gitlab, and if and how they can help with CI. I hope this
> gives people a thorough update, and answers. I really believe Gitlab can
> be a useful software suite, despite its shortcomings. My hopes are that
> it will help increase the feedback loop between development and testing,
> as well as make it easier and more welcoming for new contributors. 
> 
> 
> Be well, and Happy Hacking!
> - Devan
> 
> 
> [0] https://lists.gnu.org/archive/html/gnunet-developers/2019-01/msg00071.html
> [1] https://lists.gnu.org/archive/html/gnunet-developers/2019-01/msg00082.html
> [2] 
> http://www.mantisbt.org/docs/master/en-US/Admin_Guide/html-desktop/#admin.config.display
> [3] https://gitlab.kitware.com/utils/mantis-to-gitlab 
> [4] https://github.com/timwiel/mantis2gitlab 
> [5] 
> https://tutorials.technology/tutorials/89-show-all-repositories-of-a-git-server.html
> [6] https://docs.gitlab.com/ce/ci/quick_start/
> [7] https://docs.gitlab.com/ce/user/project/container_registry.html



> _______________________________________________
> GNUnet-developers mailing list
> address@hidden
> https://lists.gnu.org/mailman/listinfo/gnunet-developers

signature.asc
Description: PGP signature