Re: branching stable/2.22?

[Jean Abou Samra]

Hi,

> Le 24 août 2020 à 08:30, Jonas Hahnfeld <hahnjo@hahnjo.de> a écrit :
> 
> Am Sonntag, den 23.08.2020, 23:44 +0200 schrieb Jean Abou Samra:
>> Hi,
>> 
>> (Sorry about the strange reply style.)
>> 
>>> On Sun, Aug 23, 2020 at 1:58 PM Jonas Hahnfeld <hah...@hahnjo.de> wrote:
>>> I'd like to ask what it would take in principle to branch stable/2.22
>>> and what others think about this.
>>> 
>>> Personally I'm convinced that creating the branch and only picking bug
>>> fixes from master is the only guaranteed way to stabilize. Now you
>>> might say that there were only few unstable releases (AFAICT there was
>>> 2.19.65 before branching stable/2.20). However, I think there are
>>> already quite some user-facing changes and also the switch to Python 3.
>>> With Python 2.x being EOL since January, it's only a matter of time
>>> until Linux distributions and BSDs want to drop that, and it would be
>>> unfortunate if that put a (temporary) end to providing LilyPond for
>>> their users. If we had release candidates or even a stable version
>>> until then, it would definitely help.
>> 
>> Maybe we could try to release 2.20.1 with Python 3?
> 
> That would mean porting 50+ commits which sounds like a bad idea and
> even gets worse because of the reformatting in master. The latter
> implies that any bug fix made in master will result in merge conflicts.
> Plus I don't understand the proposal if you're at the same time saying
> that the scripts are fragile. By that logic, why would we backport such
> extensive changes and claim they're stable?

Right, I was oblique: the scripts are fragile at present, so branching 
release/2.22 now is no good in my opinion, but hopefully we can stabilize them 
faster than we stabilize LilyPond as a whole, and have that in 2.20.1 or 2.20.2.

Can you explain why porting 50 commits from master to 2.20 is a bad idea? If we 
port all Python related-commits (including the reformatting), there won't be 
any merge conflicts, or am I being dense somehow?

I do understand that having LilyPond 2.20.0 support exclusively Python 2 but 
2.20.1 be Python 3 only feels weird. However, I value the interest of the 
average user more than that of packagers. Neither Python 3 nor Guile 2 is a 
breaking change from the user's point of view. If having LilyPond 2.20.1 (or 
2.20.2) support Python 3 is not feasible, I think we should just let 
distributions drop LilyPond (see below). I'm not happy about it, but this is, 
in my opinion, preferable to making a stable release, in a hurry, that will 
contain more bugs and few user-facing changes.

>>> The same can of course happen with Guile, but that situation has been
>>> going on for years. Furthermore, it's at least possible to compile and
>>> use current master with Guile 2.x, even if slower. In my understanding,
>>> things are getting better but properly integrating byte compilation is
>>> still a good amount of work that nobody could give a deadline on.
>>> 
>>> WDYT?
>> 
>> [Han-Wen] I think that both Python 3 support and usable (if imperfect) GUILE 
>> 2 support is a strong argument for releasing a new stable version soon.
>> 
>> Why Guile 2? If it's still imperfect and slower, we don't want to make it 
>> the default in the binaries at lilypond.org, do we? So how will the 
>> situation be different from 2.20? Sorry, I must be missing something obvious 
>> here... I don't understand you.
> 
> At least in my book, it's not about changing the default but at least
> making it possible for distributions to compile with Guile 2.x instead
> of just throwing LilyPond out.

Does this mean that we'll receive bug reports that won't be reproducible by 
others because they'll actually be related to Guile 2? In my opinion, the 
distributions just throwing out LilyPond is better.

Additionally, the sh installers are recommended by the official website over 
distribution package managers.

>> At any rate, Python 3 support is great, but the scripts are fragile at the 
>> moment. This is clear from the tracker and the bug-lilypond list. Our Python 
>> scripts still need fixes in the way we distribute them, plus 
>> encoding-related issues (I'm planning on tackling the latter point in a 
>> short period at the end of August, but who knows what that will reveal).
> 
> Could you please point to concrete issues? The distribution of scripts
> hasn't changed, so it basically still works the same it did for years
> (decades?).

I'm writing this with a browser that is too old to view gitlab.com. There is at 
least
https://www.mail-archive.com/bug-lilypond@gnu.org/msg43376.html
which looks serious at first sight.

>> [Han-Wen] I've been working on the build system (obviously), with in the 
>> back of my mind having a build that is no longer recursive, but that work 
>> could be paused for a bit while we prepare for releasing a 2.22. Is there a 
>> list of problems in the current master that have to be resolved?
>> 
>> Problems are basically popping every week (e.g., info installation, 
>> translation tools, etc.). You're fixing them every week, which is really 
>> great, but before creating a release branch that is devoted to stability, I 
>> think we need a couple months to see what new problems appear, don't we?
>> 
>> We have unstable releases to publish new features and get testing. In my 
>> opinion, stable releases should really focus on stability, there's no need 
>> to rush because of Python 3 and Guile 2.
> 
> I don't think we'll get testing from distributions until we declare a
> way to stabilize.

We're speaking from different points of view here: in my book, our main source 
of testing is our development and beta releases brought to users through 
installers. I mean that LilyPond 2.22 should introduce full support for Guile 2 
with byte compilation, probably dropping Guile 1 support too, and we get Guile 
2 testing from those who try out the betas.

>> At least four areas are currently under flux: Python scripts, the build 
>> system, Guile 2 support, and fonts (Owen's project), and I don't see that 
>> master is coming any close to stability. I think we are better with focusing 
>> on these areas as long as they still require substantial attention, so as to 
>> get a stable and nice LilyPond 2.22 in a timely manner. You derecursify the 
>> build, David completes his stack of rather extensive purportive changes, 
>> Owen merges the GSoC branch, etc., and only then it'll be time to care about 
>> LilyPond 2.22.
> 
> And that's my point: If we just sit and wait, the next stable release
> might happen after all major distributions threw LilyPond out - simply
> because it cannot be built in a world of Python 3.
>> Am Sonntag, den 23.08.2020, 14:59 +0200 schrieb David Kastrup:
>>>> I'd like to ask what it would take in principle to branch stable/2.22
>>>> and what others think about this.
>>> I don't see that this is a good point of time.
>>> There has been an influx of badly tested changes to the build system and
>>> directory setup and the web pages that has to stabilise and result in a
>>> workable version of LilyPond. I don't see the point in branching a
>>> "stable" branch if there is so much in a destabilised state: you'd have
>>> to cherry-pick loads of stuff from the unstable branch as it comes in.
>> 
>> [Jonas] I fully agree 
>> 
>> ... and so do I (for what my opinion's worth, really) ...
>> 
>> [Jonas] and I should have been more clear that I don't expect the branch to 
>> happen in the next week. The point was to find out what it would take 
>> because just waiting for some unspoken condition to become true is not 
>> exactly going to happen without some effort.
>> 
>> What about scheduling the release?
>> 
>> While I do know that "Grass doesn't grow faster when you pull on it.", I 
>> would definitely like having a defined point in time where the stable 
>> release is to happen, so that everyone can focus on bug fixes before it 
>> happens. Sure, we aren't going to get agreement in a second about the date 
>> (even if not more precise than a month), but to me, having this talk now is 
>> preferable so as to give LilyPond development a tempo. To say it with other 
>> words, we've got a score to play; arguing about the tempo is better than 
>> starting the piece with different tempi.
>> 
>> As sort of a shot in the dark, how about planning the 2.22 release for May 
>> 2021, for example?
> 
> Do you mean branching stable/2.22 or releasing 2.22.0 in May 2021? In
> my understanding, the past process includes the release of beta
> versions from the branch. That makes it close to impossible to predict
> the final date of the stable version, and that's not the purpose of
> this thread.

I mean releasing 2.22.0 in May 2021. This is not about predictions, but 
objectives. I think that instead of planning each small step on the fly with no 
idea how the future looks like, we should settle an expected date for the 
release and plan backwards accordingly. Sure, there could be critical bugs that 
delay the actual release, but setting the expectation enables more resources to 
focus on the release by the time it is due to happen. In my opinion, this is 
the way we can avoid things like the 2.14 release that is documented in the CG.

So, an expected release in May 2021 would mean branching release/2.20 around 
January (?) and beta releases at a monthly cadence until the release is out.

I'm curious about what others think of that. In fact it looks like you already 
proposed something along these lines:
https://www.mail-archive.com/lilypond-devel@gnu.org/msg72997.html
But David was reluctant for reasons that sound sensible. David, what would be 
your opinion today?

Best,
Jean