bug#62762: 'make' often errors with "Org version mismatch" after pulling a new version of the code

[Stefan Monnier]

Hi Max,

> `load-prefer-newer' is a kludge as well.

I have to point out at this stage that I feel your answers aren't
helping address the very concrete short term question at hand.
Instead you're pointing to well known problems left and right that only
bite in very specific corner cases and with no really clear solution.

The problems `org-assert-version` is trying to fix happen a lot more
often than those corner cases and are much easier to handle.

> E.g. Python people migrated from comparison of timestamps to
> inscribing of .py file hash into byte compiled .pyc files.

Good for them.  But that's unrelated to he problem at hand (and would
only make a difference in cases that are a lot more hypothetical than
the ones we're considering here).

E.g. the mixed-version problems that `org-assert-version` tries to
detect can happen without any `.elc` file in sight at all.

Could we focus on the problem at hand, please?

> By the way, `load-prefer-newer' make things even worse [...] (.el
> files not changed, so they are older)

No, if ".el files not changed, so they are older", then
`load-prefer-newer` has no effect at all, so it can't make things worse.

>>> `org-assert-version' is just the most apparent manifestation.
>> AFAIK `org-assert-version` tries to solve a different problem.
> Of course `org-assert-version' was introduced to solve another
> problem, but it highlighted issues with dependency handling during
> incremental builds.

Yes, ELisp has lot of other problems.  But can we move those discussions
to other bug reports, otherwise I'm afraid we'll never get anywhere.

>> Indeed, we recently introduced `org--inhibit-version-check`
>> specifically so as not to use `org-assert-version` for Emacs's broken
>> incremental builds (i.e. we decided we preferred that brokenness
>> there).
> `org--inhibit-version-check' is even uglier kludge that partially
> deactivated `org-assert-version'

Maybe so, but its addition is still the embodiment of people's judgment
that for `git pull; make` in Emacs's sources, `org-assert-version` is
worse than the problems it tries to address.

> and just hides the issue with dependencies for incremental builds.

No, it doesn't.

>>> Perhaps a similar trick may be done in elisp by advicing e.g. `load'.
> Another idea: while single file is compiled per emacs invocation
> `load-history' may be compared before and after byte compilation.

It's not really "another idea" (unless you meant `advicing` in a very
narrow sense, but since we don't like Emacs to use advice within itself,
it would be a non-starter).
It's just one of the many ways to implement that same idea.

>> I can already warn them that an important problem will be the
>> presence of cyclic dependencies.
> In the C and C++ world the solution for cyclic dependencies is forward
> declarations. Some kind of such approach I see in Org as
> well. lisp/org/ol.el and lisp/org/org-element.el are mutually
> dependent. org-element.el requires 'ol, while the latter just declares
> functions from 'org-element.
>
> When dependency files are generated, it is possible to create a report on
> cyclic dependency and to disentangle them.

As part of my janitorial work, I regularly have to try and disentangle
circular dependencies.  In my experience the hardest part is to convince
the upstream maintainers to accept the churn, even though it brings no
material improvement (and forces them to get used to the new code
organization).

So, yes, technical changes could help, but it would only go so far.

[ I'm stopping here, I have to go.  ]


        Stefan