Re: GNU Lightning needs a new maintainer

[Paulo César Pereira de Andrade]

Em seg., 9 de mai. de 2022 às 20:22, Paul Cercueil
<paul@crapouillou.net> escreveu:
>
> Hi Paulo, great to hear from you.
>
> Le lun., mai 9 2022 at 13:53:33 -0300, Paulo César Pereira de Andrade
> <paulo.cesar.pereira.de.andrade@gmail.com> a écrit :
> > ]Em qua., 13 de abr. de 2022 às 15:43, Paul Cercueil
> > <paul@crapouillou.net> escreveu:
> >>
> >>  Hi,
> >
> >   Hi,
> >
> >>  We are a few people trying to contribute to GNU Lightning, but the
> >>  maintainer (Paulo) seems to be MIA. His last email on the mailing
> >> list
> >>  was on May of last year. Since then we couldn't get a patch in.
> >
> >   Sorry. I understand the situation. I missed most emails, and only
> > look at this email address every few weeks or months. I will attempt
> > to reply next to at least a few from archives.
> >
> >   My job and my real life are absorbing most of my time, and very few
> > time is left for volunteer efforts.
>
> That's understandable. The problem is that you are the only one who can
> accept patches into the repository... Maybe we could have a
> co-maintainer?

  I cannot guarantee I will become very active in some projects, or
personal projects. I wish I could...
  Could also have a github or other place mirror, that could work as a
more official fork, or would need to sign the proper papers about copyright
issues and become a gnu maintainer. And become comaintainer at
https://savannah.gnu.org/projects/lightning/

> >>  I think it would make sense to find a new maintainer to unlock the
> >>  situation.
> >>
> >>  The second point I would like to talk about: I do wish this project
> >> was
> >>  on github/gitlab, one part for the convenience (managing PRs and
> >>  integrated bugzilla), and the other part is the need of CI. GNU
> >>  Lightning supports 10+ architectures in various configurations, it
> >> is
> >>  just impossible to test properly every patch that we make. Automated
> >>  builds would be a big help.
> >>
> >>  Cheers,
> >>  -Paul
> >
> > Thanks!
> > Paulo
> >
> >
> > From more recent to older:
> >
> > [PATCH] Add conditional move opcodes:
> > I absolutely do not dislike it. But this might be an optimization in
> > the
> > wrong place. Either lightning should understand and create it, but
> > need
> > a back end specific optimizer, or this is just the start of making it
> > far
> > more complex, and would need a fallback for arches that do not have
> > it, and it might be complex, would be like jit_b{o,x}{add,sub}* but
> > instead
> > of a conditional jump, a conditional move.
> > There are a few optimizations, and this likely could be done in the
> > generic optimizer, if it knows the target extra features, or, it
> > could be
> > done by a backend specific optimization pass. But really, it would be
> > better to think on how make lightning simpler, not more complicated.
>
> Some time ago I did update the patch, the latest can be found here:
> https://github.com/pcercuei/gnu_lightning/commit/6f101bf8eccef737d60bf7e6ba85558db49e7908
>
> I added conditional opcodes for ARM and Aarch64, and a fallback
> implementation for PowerPC. Adding fallback implementations for the
> rest of the architectures shouldn't be too hard, the hard part is
> creating a qemu image and testing for every arch...
>
> If I understand you correctly, the optimizer would detect this pattern:
>
> branch = jit_beqi(r2, 0);
> jit_movr(r0, r1);
> jit_patch(branch);
>
> And replace it with a conditional opcode for the current arch?
>
> I guess that works for me, it keeps the API simple, but I'm not sure I
> can implement that. The inner Lightning (outside *-cpu.c) is scary and
> I wouldn't know where to start.

  This would be an extra pass, like jit_thread_jumps, that has optimizations
named redundant_jump, reverse_jump and shortcut_jump, and other basic/safe
optimizations.
  But in this case, it would be a pass defined in the architecture specific
backend code. Just an extra list traversal after doing basic arch independent
optimizations and/or redundancy removal.
  There is already also minimal code generation time optimizations,
like using shifts for divisions by power or two or related.

> As for making Lightning more complex... there are a few opcodes that I
> think are really missing, for instance endian-swap opcodes (htonr/ntohr
> only do something on little-endian), and maybe a "count leading zeros"
> opcode. Also a jit_dead() similar to jit_live(). For the rest I'm
> pretty happy with what Lightning provides.

  It would be really good, but as long as we do not keep changing api/abi,
if could split in some layers of code, so that one could even fully bypass
the linked list, and go straight and blindly generating code, like original
lightning, and have an upper level like the current one. A very minimal
direct generation would be desirable for certain languages, or, even a
different approach, instead fully parsing a long list, dynamically parse
state very fast, but still keep register clobbering information state, or,
have static temp registers... This would have a lot of redundancy, but
time executing redundant code likely would be faster than preprocessing
to generate more sane machine code.

> > [Patches for configure Script and Doc]
> > Sorry about these. I do not use Ubuntu but will add the patches and
> > possibly patch it to not break at least in the environments I use.
> >
> > [[PATCH 0/6] Misc patches to pass the testsuite and remove warnings]
> > Can work on some of these, but really, -fno-strict-aliasing should be
> > default if doing any optimized build, or, you might end with a hidden
> > bug
> > very difficult to find. Better to just stick with
> > -fno-strict-aliasing.
> >
> > [Issue when using jit_label() before a jump or branch]
> > It appears it was already discussed in the thread. Unless I
> > misunderstood
> > the issue, the generated code is correct. It is a non conditional
> > jump to
> > a non conditional jump, so, it is ok to take a short circuit.
> >
> > [[PATCH 1/6] riscv: Fix movi/relatives loading incorrect value]
> > The patch is incorrect. The sign extension is expected. The "magic" is
> > the use of addiw. Note that constants are 64 bit, so, you might be
> > doing something wrong.
> >
> > [[PATCH 2/6] arm: Force RES0 bits in TST/A1 to be 0]
> > Need to check it on actual hardware. For the past 10+ years I did only
> > test arm variants on qemu images.
> >
> > [[PATCH 3/6] mips: Use DEXT when in 64-bit mode instead of EXT]
> > I did never test on mips 64. Note that apparently will just amend a
> > patch you already did send. But I can add it.
> >
> > [[PATCH 6/6] Include math.h in check/lightning.c]
> > This is straight and should not have any side effects.
> >
> > [[PATCH 5/6] misc: Add bfd_arch to arm/aarch64/mips]
> > This appears to dependent on the binutils version. As long as it works
> > with older binutils (from my test vms) it can be added, otherwise,
> > will
> > need a binutils version check.
>
> The binutils version check is broken, it always detects wrong here.
>
> By the way, I have this alternative patch that greatly simplifies the
> BFD init code:
> https://github.com/pcercuei/gnu_lightning/commit/08cfdf131fa5cd772cba48e88f68dc00a7fd0a3d

  Ok, I will experiment with this also.

> > [[PATCH 4/6] ppc: Add endianness check to _truncr_d_i]
> > Thanks, this appears to be correct, and somehow missed when doing
> > the ppc64le port. Need to review again, probably I missed a make check
> > in some of the last steps.

  BTW, I remember something about this... either I did not add the patch
in the last moment, or it might not be required, but I remember testing
this and getting it right... But it was only a few days with access to a
ppc64le system.

> > [Atomic operations]
> > If patches are provided, sure it would be welcome. My main usage of
> > lightning was fine with calls to pthread_* as it was only jit for a
> > custom
> > language. The main problem comes with fallbacks for arches that do
> > not support it, so, only some minimal set likely could be added.
>
> Fallbacks mean using real locking, which means calls to C, is that
> something we really want?
>
> Cheers,
> -Paul
>
> > From here I see my comments... but will review further the archives.

Thanks!
Paulo