Fwd: Re: About Trac to GitLab migration

Sajith Sasidharan sajith at hcoop.net
Thu Jul 23 16:26:13 UTC 2020


Daniel Stone migrated freedesktop.org's stuff to a self-hosted GitLab.  Below
are Daniel's thoughts.


-------- Forwarded Message --------
From: Daniel Stone
To: Sajith Sasidharan
Cc: Ben Gamari
Subject: Re: About Trac to GitLab migration
Date: Thu, 23 Jul 2020 12:34:08 +0100

Hi all,

On Thu, 23 Jul 2020 at 03:53, Sajith Sasidharan <sajith at hcoop.net> wrote:
> On Wed, 2020-07-22 at 19:26 -0400, Ben Gamari wrote:
> > Anyways, I hope this helps! I would be happy to chat if you have any
> > specific questions.
> 
> Thank you very much!  What you wrote is very helpful.  The GHC blog post did
> not turn up in my search results.  Glad that I asked: I got more this way.
> :)

Nice to meet you Sajith! Unfortunately I don't have anything
intelligent to say about Trac, but I can definitely help with non-Trac
migration and hosting things.

> We have been talking to GitLab about their open source program.  I don't
> believe we can (or should!) spare the resources for self-hosting GitLab.
> 
> It sounds like we won't be able to preserve issue numbers if we're to move
> straight to gitlab.com, but I don't know how important of an issue this
> would
> be for us.  Tahoe-LAFS wiki is rather modest in comparison to GHC wiki, so
> the
> manual intervention also must be modest.

GitLab issue numbers are per-project rather than global, so if you're
splitting Trac issues between different projects, then I imagine you'd
lose the 1:1 correspondence.

Rather than using GitLab's native import, we reused and adapted
GNOME's bztogl to migrate from Bugzilla:
https://gitlab.gnome.org/World/bztogl +
https://gitlab.freedesktop.org/freedesktop/bztogl - maybe that's of
some help to you, maybe not. One difficult issue is preserving users.
Since we run our own instance, we could make sure that everyone had
the right local username, and that user accounts were preserved in the
migration. If you're hosting on gitlab.com, that option is not open to
you, so that's one thing to think about.

> I actually have some more questions:
> 
> Have you been able to update your patched GitLab installation to a
> "standard"
> GitHub release later?

freedesktop.org runs an unpatched GitLab codebase, though from
time-to-time we do monkeypatch some important hotfixes in. We just use
the Helm charts to run our instance in Kubernetes, so we can deploy
whatever we can get inside a Docker container. We haven't had any
issues with stability of database/on-disk formats to speak of.

> What is the setup like for macOS and Windows CI runners?

For macOS I have no idea, since getting hosted macOS instances is
amazingly painful - thankfully we don't really have to care about it.
For Windows, we do run a combination of native (execute in PowerShell
on the host, with all the downsides Ben mentioned) and also
Docker-on-Windows runners. We only use the native executor to build
Docker containers (since Windows can't do Docker-in-Docker, and better
tools like buildah/kaniko/etc don't run there), and everything else
runs in the Docker executor, cf:
https://gitlab.freedesktop.org/mesa/mesa/-/blob/master/.gitlab-ci.yml#L366
(core job definitions throughout this file)
https://gitlab.freedesktop.org/mesa/mesa/-/tree/master/.gitlab-ci/windows
(container-build job runs mesa_container.ps1 -> Dockerfile ->
mesa_deps.ps1; in-container builds just run mesa_build.ps1)

Happy to speak more about that if it helps, though I should note that
high on my list is stealing the ghc script to clean up after Windows
docker instances. ;)

We run those on Windows Server Datacentre edition instances; if you're
not using DC, then you're restricted to Hyper-V containers rather than
process-isolation, so it's a lot slower. But DC is also quite
expensive, so ...

> I am guessing that you must have considered making a Pandoc reader for Trac
> wiki markup before writing a custom tool -- is that a viable approach?

We just ignored the GitLab wikis because they are frankly bad. The
history on them is really poor, and I also don't like that you have to
have access to the project to be able to do anything with the wiki.
Instead of using the wikis, we encourage all our projects to keep
their documentation in GitLab Pages projects: that way people can fork
the repository and send MRs against the documentation, so docs can be
reviewed, approved, and shipped the same way as code. That requires
using a static site generator like Jekyll, Hugo, Sphinx, or whatever.
There are a lot of templates around there for using pretty much every
SSG you can think of in CI with Pages.

My impression of the GitLab wiki is that they wrote it very quickly as
a proof of concept to match GitHub feature-for-feature, and at that
point they were abandoned. The discoverability, searchability, and
usability of them is really awful.

> May I share our communication with tahoe-dev mailing list?  It is a low
> traffic public list (currently a little broken too, perhaps), with a public
> list archive.

Happy for anything I say to be made public.

I can't really think of any kind of 'gotchas' or anything you should
particularly know. Apart from that you should definitely get familiar
with both the CI pipeline and the GitLab API. Both of them are really
powerful, and we've now found that almost our entire workflows are
scripted and automated through a combination of those two.

Cheers,
Daniel




More information about the tahoe-dev mailing list