Re: Native compilation: the bird-eye view

[Andrea Corallo]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

>>> I don't have a clear understanding of the problem this is trying to solve.
>>> Could you summarize it?
>>
>> Sure,
>>
>> we need a rule to go from the source file to the eln filename.
>>
>> /xxx/foo.el -> foo-something1.eln
>> /yyy/foo.el -> foo-something2.eln
>>
>> Given we store all eln "flatted" in a single directory indeed we need to
>> have "something1" different from "something2".
> [...]
>> I then came-up with the current idea of using the source content as
>> input of the hash, this should be robust.
>
> Yes, the use of the content-hash should solve this problem "once and
> forall".  Maybe the performance cost will force us to reevaluate this
> decision, but for now I'm happy with it, so consider it a "solved problem".

Agree, I think as a last caveat we should add to this hash also
parameters like comp-speed to have them effective when recompiling.

>> At the moment we are using 2 separate hashes to make the cleaning (easy)
>> possible, this is a separate problem that's correct.  The idea is that
>> with the two hashes when recompiling is easy to just remove the old file
>> as you can identify using the first hash the old one.
>
> I must be missing something: in which way does the filename-hash help here?
> Oh, you mean we could look for <same-name-hash>-*.eln and remove them?

Yep

> What I was thinking of doing was: when we generate a new .elc file, just
> before saving the file over the old .elc file, we compute the hash of the
> old .elc file and erase any matching .eln file.

Remeber ATM the input to the ashing is the .el content and not the .elc,
this motivated by:

  - For the future we may not have to necessarily produce the .elc in
    certain condition (AoT compilation during the build).

  - The .elc may not capture all the input to the compilation that have
    an effect on the native code generation (I don't have a working
    example for this is just a suspect).

>> Additionally to that, at any point in time, if you have two file
>> sharing the first hash you know that all but the most recent can
>> be removed.
>
> Then we could move the content-hash from the .eln filename to within the
> .eln file itself.

Yes we can do that, the API would be just more complex than the ls
<same-name-hash>-*.eln one.

> Note that for the "system-wide" .eln files it can be trickier because we
> may want to generate them once and then use them with Emacs installed at
> difference locations, so we'd then want to try and avoid using the
> absolute file name (and instead use file names that are relative to the
> root of the Emacs install or something).

I think in this case we could generalize what we have in place now to
allow reusing the same eln for local build and installed.

We are currently replacing in the filename if matches
PATH_DUMPLOADSEARCH or PATH_LOADSEARCH with '//' before feeding the hash
algorithm.

>> I think this may be problematic as we could have two different Emacsen
>> with two different load-path that are sharing the eln-cache folders.
>> This may be a bit of patological case tho.
>
> No, it's definitely not pathological.  It's pretty common for packages to
> modify the `load-path` upon first use, so your `load-path` can change
> long after startup.
>
> Not to mention situations where you have several different configs
> (e.g. I always have 2 Emacs sessions, one dedicated to Gnus and
> a "general" one and they share *some* of the config but not all, so
> they don't have the same `load-path`).

Thanks

  Andrea

-- 
akrl@sdf.org