Simplifying the Hexagon frontend

[Alessandro Di Federico]

Hi, this e-mail is intended to bootstrap a public discussion on how to
improve the Hexagon frontend implementation. At rev.ng, Niccolò and I,
developed an Hexagon frontend, and we're (finally!) joining forces with
the QuIC guys to merge our efforts (did you see our talk [1]?).

The status is as follows:

* QuIC has its own fully working implementation that has been submitted
  for review.
* We're working to integrate in their implementation our mechanism to
  automatically generate code to generate tiny code. But this will take
  some more work.

In the following, some initial considerations on how the latest
patchset could be simplified.

Here you can find a graph I've put together of the build process:

    https://rev.ng/downloads/qemu-hexagon/temporary/graph.svg
    https://rev.ng/downloads/qemu-hexagon/temporary/graph.dot

Colors indicate language.
Oval nodes are generated.
Rectangles are hand-written.

Taylor, I think some simplifications can be made to the process in order
to ease the review process.

* As far as I understand, from he "Source of Truth" set of files
  (`alu.idef`, `encode_pp.def`...), through `gen_semantics`, you
  generate `semantics_generated.pyinc`, which is then included by
  `do_qemu.py` script, which does the real job.

  I would suggest to keep `gen_semantics` and all its inputs
  out-of-tree. It increases complexity in a non-negligible way, while
  bringing a reduced benefit in terms of automation.

  I'd suggest replace `gen_semantics`'s output
  (`semantics_generated.pyinc`) with a human readable JSON file that
  could be manipulated by hand and is then parsed by `do_qemu.py`. I
  think JSON is more appropriate than generating executable python code
  that is then imported.

* I suggest to switch to the decoding approach developed by Richard.
  That would simplify the build process and reduce the code that has to
  be reviewed.
  I'm not 100% of the effort required to do this, maybe Richard can
  weigh on this.

* The current implementation can generate a helper function for each
  Hexagon instruction and, for a subset of instructions, it has an
  "override" mechanism to directly generate tiny code instructions
  corresponding to the semantics of the original instruction (i.e.,
  without using helpers).

  This override mechanism is implemented with the `fWRAP` macros. They
  have benefits, but they are quite convoluted. We should strive to
  minimize the number of macros and alternative macro implementations
  to what's strictly necessary in order to generate as much code as we
  can from the "Source of Truth", but no more than that.

  As a simpler override mechanism, we could use weak functions. But I
  think that, for simplicity, we should try to get in tree a simpler
  version of the frontend that relies exclusively on helper functions.
  It won't have optimal performances, but it will be fully functional.

  Later on, once our work for automatically generating functions
  generating tiny code is mature enough, we can extend the existing
  implementation with an appropriate override system.

In the meantime, we're setting up a Dockerfile based on Debian 10
providing a minimal C toolchain that we can use to automate testing.

Feedback is more than welcome.

-- 
Alessandro Di Federico
rev.ng

[1] https://www.youtube.com/watch?v=3EpnTYBOXCI