Re: [PATCH v3 0/8] qapi-go: add generator for Golang interfaces

[Victor Toso]

Hi,

On Mon, Jan 13, 2025 at 01:52:25PM +0100, Markus Armbruster wrote:
> Victor Toso <victortoso@redhat.com> writes:
> 
> > This patch series intent is to introduce a generator that produces a Go
> > module for Go applications to interact over QMP with QEMU.
> >
> > The initial Goal is to have a Go module that works as intended and can
> > be improved upon. I'd consider initial releases to be alpha while we
> > work with utilities tools and libraries on top of this.
> >
> > The generated code should reside in a separated Git repository, similar
> > to python-qemu-qmp.
> >
> > Applications should be able to consume this under qemu.org
> > namespace (e.g: import "qemu.org/go/qemu"), see Daniel's suggestion:
> > https://lists.gnu.org/archive/html/qemu-devel/2023-09/msg07024.html
> >
> > This is the third iteration:
> > v2: https://lists.gnu.org/archive/html/qemu-devel/2023-10/msg04785.html
> >
> > I've pushed this series in my gitlab fork:
> > https://gitlab.com/victortoso/qapi-go/
> >
> > The fork contains some tests, including tests that were generated from
> > QAPI's own examples from another generator created for testing, if you
> > are interested in it:
> > https://lists.gnu.org/archive/html/qemu-devel/2023-09/msg04946.html
> >
> > I've generated the qapi-go module over each commit of this series, see:
> > https://gitlab.com/victortoso/qapi-go/-/commits/qapi-golang-v3-by-patch
> >
> > I've also generated the qapi-go module over QEMU tags: v9.1.0, v9.2.0:
> > https://gitlab.com/victortoso/qapi-go/-/commits/qapi-golang-v3-by-tags
> >
> > --
> >
> > Sorry that its been awhile between v2 and v3, I had to prioritize other
> > things. I hope to get this back on track in 2025.
> >
> > Cheers,
> > Victor
> >
> > * Changes:
> >
> > On generated go:
> >  - the output should be formatted as gofmt/goimports tools (Daniel)
> >
> >  - Included QAPI's documentation too (Daniel), see:
> >    https://lists.gnu.org/archive/html/qemu-devel/2024-11/msg01621.html
> >     
> >  - Commands and Events should Marshal directly (Andrea)
> >
> > On python script:
> >  - rebased: now uses QAPISchemaBranches, QAPISchemaAlternatives
> >
> >  - use textwrap as much as possible (Andrea)
> >
> >  - lots of changes to make the output like gofmt does
> >
> > Victor Toso (8):
> >   qapi: golang: Generate enum type
> >   qapi: golang: Generate alternate types
> >   qapi: golang: Generate struct types
> >   qapi: golang: structs: Address nullable members
> >   qapi: golang: Generate union type
> >   qapi: golang: Generate event type
> >   qapi: golang: Generate command type
> >   docs: add notes on Golang code generator
> >
> >  docs/devel/index-build.rst          |    1 +
> >  docs/devel/qapi-golang-code-gen.rst |  548 +++++++++
> >  scripts/qapi/golang.py              | 1645 +++++++++++++++++++++++++++
> >  scripts/qapi/main.py                |    3 +
> >  4 files changed, 2197 insertions(+)
> >  create mode 100644 docs/devel/qapi-golang-code-gen.rst
> >  create mode 100644 scripts/qapi/golang.py
> 
> This is series adds a backend that slots in cleanly, i.e. without any
> changes to the core.  That makes it as low-risk to merge as it gets.
> 
> I'd like an Acked-by for the generated Go from someone familiar the kind
> of software that could use it.
> 
> The Go backend is a single golang.py, which generates:
> 
> * Types:
>   alternates.go enums.go structs.go unions.go
> 
> * Commands:
>   commands.go
> 
> * Events:
>   events.go
> 
> It doesn't generate visitors or introspection code.
> 
> Correct?

Correct. I've actually thought about following what we did with
libvirt-go-module which appends _generated to the files that are
generated, meaning that we would also have files that are without
_generated in the name, with custoam/manual code related to that
file.

Either way, that does not change the Go module (does not break
API/ABI) so we can customize it if needed at a later time.
 
> The existing C backend generates code for
> 
> * Types (types.py):
>   qapi-builtin-types.[ch]
>   qapi-types.[ch] qapi-types-*.[ch]
> 
> * Visitors (visit.py):
> 
>   qapi-builtin-visit.h
>   qapi-visit.[ch] qapi-visit-*.[ch]
> 
> * Commands (commands.py):
> 
>   qapi-init-commands.h
>   qapi-commands.[ch] qapi-commands-*.[ch]
> 
> * Events (events.py):
> 
>   qapi-emit-events.h
>   qapi-events.[ch] qapi-events-*.[ch]
> 
> * Introspection (introspect.py):
> 
>   qapi-introspect.h
> 
> The -* files are all one pair of files per module (the things pulled in
> with include directives), if any.  We do this to avoid "touch the QAPI
> schema, recompile the world."
> 
> The generated Go is monolithic.  No "recompile the world" problem with
> Go?

From my experience, Go compiler only builds when the .go file
changes. If the QAPI changes and the generator outputs different
.go files, it'll recompile at the project using it.
 
> golang.py is somewhat big.  Whether splitting it up along the lines of
> the C backend would improve things I can't say.  No need to worry about
> that now.

Yes, I'm not super experienced with python. I'm all ears to any
suggestions on how to organize the source better. I'm happy to do
it as a follow-up.

Cheers,
Victor

signature.asc
Description: PGP signature