[Qemu-commits] [qemu/qemu] d08ac8: qapi: Fix regression with '-netdev help'

[GitHub]

  Branch: refs/heads/master
  Home:   https://github.com/qemu/qemu
  Commit: d08ac81a459258ce20b3184fa9325c6c1350ac9e
      
https://github.com/qemu/qemu/commit/d08ac81a459258ce20b3184fa9325c6c1350ac9e
  Author: Eric Blake <address@hidden>
  Date:   2015-10-15 (Thu, 15 Oct 2015)

  Changed paths:
    M scripts/qapi-visit.py

  Log Message:
  -----------
  qapi: Fix regression with '-netdev help'

Commit e36c714e causes 'qemu -netdev help' to dump core, because the
call to visit_end_union() is no longer conditional on whether *obj was
allocated.

Reported by Marc-André Lureau <address@hidden>
Signed-off-by: Eric Blake <address@hidden>
Message-Id: <address@hidden>
[Commit message tweaked to say 'help' instead of '?']
Signed-off-by: Markus Armbruster <address@hidden>


  Commit: 25a0d9c977c2f5db914b0a1619759fd77d97b016
      
https://github.com/qemu/qemu/commit/25a0d9c977c2f5db914b0a1619759fd77d97b016
  Author: Eric Blake <address@hidden>
  Date:   2015-10-15 (Thu, 15 Oct 2015)

  Changed paths:
    M scripts/qapi-introspect.py
    M scripts/qapi-types.py
    M scripts/qapi-visit.py
    M scripts/qapi.py

  Log Message:
  -----------
  qapi: Use predicate callback to determine visit filtering

Previously, qapi-types and qapi-visit filtered out implicit
objects during visit_object_type() by using 'info' (works since
implicit objects do not [yet] have associated info); meanwhile
qapi-introspect filtered out all schema types on the first pass
by returning a python type from visit_begin(), which was then
used at a distance in QAPISchema.visit() to do the filtering.

Rather than keeping these ad hoc approaches, add a new visitor
callback visit_needed() which returns False to skip a given
entity, and which defaults to True unless overridden.  Use the
new mechanism to simplify all three filtering visitors.

No change to the generated code.

Suggested-by: Markus Armbruster <address@hidden>
Signed-off-by: Eric Blake <address@hidden>
Message-Id: <address@hidden>
Signed-off-by: Markus Armbruster <address@hidden>


  Commit: 7618b91ff80ec42b84b29be24d8ef53ddb377110
      
https://github.com/qemu/qemu/commit/7618b91ff80ec42b84b29be24d8ef53ddb377110
  Author: Eric Blake <address@hidden>
  Date:   2015-10-15 (Thu, 15 Oct 2015)

  Changed paths:
    M scripts/qapi.py

  Log Message:
  -----------
  qapi: Prepare for errors during check()

The next few patches will start migrating error checking from
ad hoc parse methods into the QAPISchema*.check() methods.  But
for an error message to display, we first have to fix the
overall 'try' to catch those errors.  We also want to enable a
few more assertions, such as making sure every attempt to
raise a semantic error is passed a valid location info, or that
various preconditions hold.

The general approach for moving error checking will then be to
relax an assertion into an if that raises an exception if the
condition does not hold, and removing the counterpart ad hoc
check done during the parse phase.

Signed-off-by: Eric Blake <address@hidden>
Message-Id: <address@hidden>
Signed-off-by: Markus Armbruster <address@hidden>


  Commit: baabb84c5b27115b979d864cb2af4d63b823983f
      
https://github.com/qemu/qemu/commit/baabb84c5b27115b979d864cb2af4d63b823983f
  Author: Eric Blake <address@hidden>
  Date:   2015-10-15 (Thu, 15 Oct 2015)

  Changed paths:
    M tests/Makefile
    R tests/qapi-schema/alternate-good.err
    R tests/qapi-schema/alternate-good.exit
    R tests/qapi-schema/alternate-good.json
    R tests/qapi-schema/alternate-good.out

  Log Message:
  -----------
  qapi: Drop redundant alternate-good test

The alternate-good.json test was already covered by
qapi-schema-test.json.  As future commits will be tweaking
how alternates are laid out, removing the duplicate test now
reduces churn.

Signed-off-by: Eric Blake <address@hidden>
Message-Id: <address@hidden>
Signed-off-by: Markus Armbruster <address@hidden>


  Commit: 625b251c69d983959efaeadeee12405b1a66d331
      
https://github.com/qemu/qemu/commit/625b251c69d983959efaeadeee12405b1a66d331
  Author: Eric Blake <address@hidden>
  Date:   2015-10-15 (Thu, 15 Oct 2015)

  Changed paths:
    M tests/Makefile
    R tests/qapi-schema/enum-empty.err
    R tests/qapi-schema/enum-empty.exit
    R tests/qapi-schema/enum-empty.json
    R tests/qapi-schema/enum-empty.out
    M tests/qapi-schema/qapi-schema-test.json
    M tests/qapi-schema/qapi-schema-test.out

  Log Message:
  -----------
  qapi: Move empty-enum to compile-time test

Rather than just asserting that we can parse an empty enum,
let's also make sure we can compile it, by including it in
qapi-schema-test.

Signed-off-by: Eric Blake <address@hidden>
Message-Id: <address@hidden>
Signed-off-by: Markus Armbruster <address@hidden>


  Commit: cae95eae6270c1ea28a9ba8eee75c441b1015f68
      
https://github.com/qemu/qemu/commit/cae95eae6270c1ea28a9ba8eee75c441b1015f68
  Author: Eric Blake <address@hidden>
  Date:   2015-10-15 (Thu, 15 Oct 2015)

  Changed paths:
    M scripts/qapi.py
    M tests/Makefile
    M tests/qapi-schema/qapi-schema-test.json
    M tests/qapi-schema/qapi-schema-test.out
    R tests/qapi-schema/returns-int.err
    R tests/qapi-schema/returns-int.exit
    R tests/qapi-schema/returns-int.json
    R tests/qapi-schema/returns-int.out
    M tests/test-qmp-commands.c

  Log Message:
  -----------
  qapi: Drop redundant returns-int test

qapi-schema-test was already testing that we could have a
command returning int, but burned a command name in the whitelist.
Merge the redundant positive test returns-int, and pick a name
that reduces the whitelist size.

Signed-off-by: Eric Blake <address@hidden>
Message-Id: <address@hidden>
Signed-off-by: Markus Armbruster <address@hidden>


  Commit: 70478cef837e86b1cff08073153081ce480e0e6c
      
https://github.com/qemu/qemu/commit/70478cef837e86b1cff08073153081ce480e0e6c
  Author: Eric Blake <address@hidden>
  Date:   2015-10-15 (Thu, 15 Oct 2015)

  Changed paths:
    M tests/Makefile
    R tests/qapi-schema/flat-union-reverse-define.err
    R tests/qapi-schema/flat-union-reverse-define.exit
    R tests/qapi-schema/flat-union-reverse-define.json
    R tests/qapi-schema/flat-union-reverse-define.out
    M tests/qapi-schema/qapi-schema-test.json

  Log Message:
  -----------
  qapi: Drop redundant flat-union-reverse-define test

As of commit 8c3f8e77, we test compilation of forward references
for a struct base type (UserDefOne), flat union base type
(UserDefUnionBase), and flat union branch type
(UserDefFlatUnion2). The only remaining forward reference being
tested for parsing in flat-union-reverse-define was a forward
enum declaration.  Once we make sure that always compiles,
the smaller parse-only test is redundant and can be deleted.

Signed-off-by: Eric Blake <address@hidden>
Message-Id: <address@hidden>
Signed-off-by: Markus Armbruster <address@hidden>


  Commit: 849ab13c1657b51b89693282ddd42ca1f6255354
      
https://github.com/qemu/qemu/commit/849ab13c1657b51b89693282ddd42ca1f6255354
  Author: Eric Blake <address@hidden>
  Date:   2015-10-15 (Thu, 15 Oct 2015)

  Changed paths:
    M tests/Makefile
    R tests/qapi-schema/args-member-array.err
    R tests/qapi-schema/args-member-array.exit
    R tests/qapi-schema/args-member-array.json
    R tests/qapi-schema/args-member-array.out

  Log Message:
  -----------
  qapi: Drop redundant args-member-array test

qapi-schema-test already ensures that we can correctly compile
an array of enums (__org.qemu_x-command), an array of builtins
(UserDefNativeListUnion), and an array of structs (again
__org.qemu_x-command).  That means args-member-array is not
adding any additional parse-only test coverage, so drop it.

Signed-off-by: Eric Blake <address@hidden>
Message-Id: <address@hidden>
Signed-off-by: Markus Armbruster <address@hidden>


  Commit: 49823c4b4304a3e4aa5d67e089946b12d6a52d64
      
https://github.com/qemu/qemu/commit/49823c4b4304a3e4aa5d67e089946b12d6a52d64
  Author: Eric Blake <address@hidden>
  Date:   2015-10-15 (Thu, 15 Oct 2015)

  Changed paths:
    M scripts/qapi-types.py
    M scripts/qapi-visit.py
    M scripts/qapi.py

  Log Message:
  -----------
  qapi: Don't use info as witness of implicit object type

A future patch will enable error reporting from the various
QAPISchema*.check() methods.  But to report an error related
to an implicit type, we'll need to associate a location with
the type (the same location as the top-level entity that is
causing the creation of the implicit type), and once we do
that, keying off of whether foo.info exists is no longer a
viable way to determine if foo is an implicit type.

Instead, add an is_implicit() method to QAPISchemaEntity, and use it.
It can be overridden later for ObjectType and EnumType, when implicit
instances of those classes gain info.

Signed-off-by: Eric Blake <address@hidden>
Message-Id: <address@hidden>
Signed-off-by: Markus Armbruster <address@hidden>


  Commit: 9f08c8ec73878122ad4b061ed334f0437afaaa32
      
https://github.com/qemu/qemu/commit/9f08c8ec73878122ad4b061ed334f0437afaaa32
  Author: Eric Blake <address@hidden>
  Date:   2015-10-15 (Thu, 15 Oct 2015)

  Changed paths:
    M qapi-schema.json
    M scripts/qapi.py
    M tests/qapi-schema/qapi-schema-test.json
    M tests/qapi-schema/qapi-schema-test.out

  Log Message:
  -----------
  qapi: Lazy creation of array types

Commit ac88219a had several TODO markers about whether we needed
to automatically create the corresponding array type alongside
any other type.  It turns out that most of the time, we don't!

There are a few exceptions: 1) We have a few situations where we
use an array type in internal code but do not expose that type
through QMP; fix it by declaring a dummy type that forces the
generator to see that we want to use the array type.

2) The builtin arrays (such as intList for QAPI ['int']) must
always be generated, because of the way our QAPI_TYPES_BUILTIN
compile guard works: we have situations (at the very least
tests/test-qmp-output-visitor.c) that include both top-level
"qapi-types.h" (via "error.h") and a secondary
"test-qapi-types.h". If we were to only emit the builtin types
when used locally, then the first .h file would not include all
types, but the second .h does not declare anything at all because
the first .h set QAPI_TYPES_BUILTIN, and we would end up with
compilation error due to things like unknown type 'int8List'.

Actually, we may need to revisit how we do type guards, and
change from a single QAPI_TYPES_BUILTIN over to a different
usage pattern that does one #ifdef per qapi type - right now,
the only types that are declared multiple times between two qapi
.json files for inclusion by a single .c file happen to be the
builtin arrays.  But now that we have QAPI 'include' statements,
it is logical to assume that we will soon reach a point where
we want to reuse non-builtin types (yes, I'm thinking about what
it will take to add introspection to QGA, where we will want to
reuse the SchemaInfo type and friends).  One #ifdef per type
will help ensure that generating the same qapi type into more
than one qapi-types.h won't cause collisions when both are
included in the same .c file; but we also have to solve how to
avoid creating duplicate qapi-types.c entry points.  So that
is a problem left for another day.

Generated code for qapi-types and qapi-visit is drastically
reduced; less than a third of the arrays that were blindly
created were actually needed (a quick grep shows we dropped
from 219 to 69 *List types), and the .o files lost more than
30% of their bulk.  [For best results, diff the generated
files with 'git diff --patience --no-index pre post'.]

Interestingly, the introspection output is unchanged - this is
because we already cull all types that are not indirectly
reachable from a command or event, so introspection was already
using only a subset of array types.  The subset of types
introspected is now a much larger percentage of the overall set
of array types emitted in qapi-types.h (since the larger set
shrunk), but still not 100% (evidence that the array types
emitted for our new Dummy structs, and the new struct itself,
don't affect QMP).

Signed-off-by: Eric Blake <address@hidden>
Message-Id: <address@hidden>
[Moved array info tracking to a later patch]
Signed-off-by: Markus Armbruster <address@hidden>


  Commit: 46292ba75c515baf733df18644052b2ce9492728
      
https://github.com/qemu/qemu/commit/46292ba75c515baf733df18644052b2ce9492728
  Author: Eric Blake <address@hidden>
  Date:   2015-10-15 (Thu, 15 Oct 2015)

  Changed paths:
    M scripts/qapi.py

  Log Message:
  -----------
  qapi: Create simple union type member earlier

For simple unions, we were creating the implicit 'type' tag
member during the QAPISchemaObjectTypeVariants constructor.
This is different from every other implicit QAPISchemaEntity
object, which get created by QAPISchema methods.  Hoist the
creation to the caller (renaming _make_tag_enum() to
_make_implicit_tag()), and pass the entity rather than the
string name, so that we have the nice property that no
entities are created as a side effect within a different
entity.  A later patch will then have an easier time of
associating location info with each entity creation.

No change to generated code.

Signed-off-by: Eric Blake <address@hidden>
Message-Id: <address@hidden>
Signed-off-by: Markus Armbruster <address@hidden>


  Commit: 99df5289d8c7ebf373c3570d8fba3f3a73360281
      
https://github.com/qemu/qemu/commit/99df5289d8c7ebf373c3570d8fba3f3a73360281
  Author: Eric Blake <address@hidden>
  Date:   2015-10-15 (Thu, 15 Oct 2015)

  Changed paths:
    M scripts/qapi.py

  Log Message:
  -----------
  qapi: Track location that created an implicit type

A future patch will move some error checking from the parser
to the various QAPISchema*.check() methods, which run only
after parsing completes.  It will thus be possible to create
a python instance representing an implicit QAPI type that
parses fine but will fail validation during check().  Since
all errors have to have an associated 'info' location, we
need a location to be associated with those implicit types.
The intuitive info to use is the location of the enclosing
entity that caused the creation of the implicit type.

Note that we do not anticipate builtin types being used in
an error message (as they are not part of the user's QAPI
input, the user can't cause a semantic error in their
behavior), so we exempt those types from requiring info, by
setting a flag to track the completion of _def_predefineds(),
and tracking that flag in _def_entity().

No change to the generated code.

Signed-off-by: Eric Blake <address@hidden>
Message-Id: <address@hidden>
[Missing QAPISchemaArrayType.is_implicit() supplied]
Signed-off-by: Markus Armbruster <address@hidden>


  Commit: 61f7901bb8a7f2f2503b5b025c4c33dbeac9cf5b
      
https://github.com/qemu/qemu/commit/61f7901bb8a7f2f2503b5b025c4c33dbeac9cf5b
  Author: Peter Maydell <address@hidden>
  Date:   2015-10-16 (Fri, 16 Oct 2015)

  Changed paths:
    M qapi-schema.json
    M scripts/qapi-introspect.py
    M scripts/qapi-types.py
    M scripts/qapi-visit.py
    M scripts/qapi.py
    M tests/Makefile
    R tests/qapi-schema/alternate-good.err
    R tests/qapi-schema/alternate-good.exit
    R tests/qapi-schema/alternate-good.json
    R tests/qapi-schema/alternate-good.out
    R tests/qapi-schema/args-member-array.err
    R tests/qapi-schema/args-member-array.exit
    R tests/qapi-schema/args-member-array.json
    R tests/qapi-schema/args-member-array.out
    R tests/qapi-schema/enum-empty.err
    R tests/qapi-schema/enum-empty.exit
    R tests/qapi-schema/enum-empty.json
    R tests/qapi-schema/enum-empty.out
    R tests/qapi-schema/flat-union-reverse-define.err
    R tests/qapi-schema/flat-union-reverse-define.exit
    R tests/qapi-schema/flat-union-reverse-define.json
    R tests/qapi-schema/flat-union-reverse-define.out
    M tests/qapi-schema/qapi-schema-test.json
    M tests/qapi-schema/qapi-schema-test.out
    R tests/qapi-schema/returns-int.err
    R tests/qapi-schema/returns-int.exit
    R tests/qapi-schema/returns-int.json
    R tests/qapi-schema/returns-int.out
    M tests/test-qmp-commands.c

  Log Message:
  -----------
  Merge remote-tracking branch 'remotes/armbru/tags/pull-qapi-2015-10-15' into 
staging

QAPI patches

# gpg: Signature made Thu 15 Oct 2015 07:40:46 BST using RSA key ID EB918653
# gpg: Good signature from "Markus Armbruster <address@hidden>"
# gpg:                 aka "Markus Armbruster <address@hidden>"

* remotes/armbru/tags/pull-qapi-2015-10-15:
  qapi: Track location that created an implicit type
  qapi: Create simple union type member earlier
  qapi: Lazy creation of array types
  qapi: Don't use info as witness of implicit object type
  qapi: Drop redundant args-member-array test
  qapi: Drop redundant flat-union-reverse-define test
  qapi: Drop redundant returns-int test
  qapi: Move empty-enum to compile-time test
  qapi: Drop redundant alternate-good test
  qapi: Prepare for errors during check()
  qapi: Use predicate callback to determine visit filtering
  qapi: Fix regression with '-netdev help'

Signed-off-by: Peter Maydell <address@hidden>


Compare: https://github.com/qemu/qemu/compare/e95bdb4341c3...61f7901bb8a7