[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[Qemu-commits] [qemu/qemu] 72e9e5: docs/interop/qmp-spec: How to force k
|
From: |
GitHub |
|
Subject: |
[Qemu-commits] [qemu/qemu] 72e9e5: docs/interop/qmp-spec: How to force known good par... |
|
Date: |
Sat, 25 Aug 2018 02:58:31 -0700 |
Branch: refs/heads/master
Home: https://github.com/qemu/qemu
Commit: 72e9e569d03c826641eada68d291971d165460bc
https://github.com/qemu/qemu/commit/72e9e569d03c826641eada68d291971d165460bc
Author: Markus Armbruster <address@hidden>
Date: 2018-08-24 (Fri, 24 Aug 2018)
Changed paths:
M docs/interop/qmp-spec.txt
Log Message:
-----------
docs/interop/qmp-spec: How to force known good parser state
Section "QGA Synchronization" specifies that sending "a raw 0xFF
sentinel byte" makes the server "reset its state and discard all
pending data prior to the sentinel." What actually happens there is a
lexical error, which will produce one or more error responses.
Moreover, it's not specific to QGA.
Create new section "Forcing the JSON parser into known-good state" to
document the technique properly. Rewrite section "QGA
Synchronization" to document just the other direction, i.e. command
guest-sync-delimited.
Section "Protocol Specification" mentions "synchronization bytes
(documented below)". Delete that.
While there, fix it not to claim '"Server" is QEMU itself', but
'"Server" is either QEMU or the QEMU Guest Agent'.
Signed-off-by: Markus Armbruster <address@hidden>
Reviewed-by: Eric Blake <address@hidden>
Message-Id: <address@hidden>
Commit: 956a104a6ccbc7c5599b84e05d9c438ca85623f8
https://github.com/qemu/qemu/commit/956a104a6ccbc7c5599b84e05d9c438ca85623f8
Author: Markus Armbruster <address@hidden>
Date: 2018-08-24 (Fri, 24 Aug 2018)
Changed paths:
M tests/check-qjson.c
Log Message:
-----------
check-qjson: Cover multiple JSON objects in same string
qobject_from_json() & friends misbehave when the JSON text has more
than one JSON value. Add test coverage to demonstrate the bugs.
Signed-off-by: Markus Armbruster <address@hidden>
Reviewed-by: Eric Blake <address@hidden>
Message-Id: <address@hidden>
Commit: a3694181e3049db7354613c8bb86bff5b0f70333
https://github.com/qemu/qemu/commit/a3694181e3049db7354613c8bb86bff5b0f70333
Author: Markus Armbruster <address@hidden>
Date: 2018-08-24 (Fri, 24 Aug 2018)
Changed paths:
M tests/check-qjson.c
Log Message:
-----------
check-qjson: Cover blank and lexically erroneous input
qobject_from_json() can return null without setting an error on
lexical errors. I call that a bug. Add test coverage to demonstrate
it.
Signed-off-by: Markus Armbruster <address@hidden>
Reviewed-by: Eric Blake <address@hidden>
Message-Id: <address@hidden>
Commit: 5365490879199fbaa7fb4b3acf32e0624108e4d1
https://github.com/qemu/qemu/commit/5365490879199fbaa7fb4b3acf32e0624108e4d1
Author: Markus Armbruster <address@hidden>
Date: 2018-08-24 (Fri, 24 Aug 2018)
Changed paths:
M tests/check-qjson.c
Log Message:
-----------
check-qjson: Cover whitespace more thoroughly
Signed-off-by: Markus Armbruster <address@hidden>
Reviewed-by: Eric Blake <address@hidden>
Message-Id: <address@hidden>
Commit: d93bb9d5c34c1b4ed97bf824e6459b5516db94f6
https://github.com/qemu/qemu/commit/d93bb9d5c34c1b4ed97bf824e6459b5516db94f6
Author: Markus Armbruster <address@hidden>
Date: 2018-08-24 (Fri, 24 Aug 2018)
Changed paths:
M MAINTAINERS
M tests/Makefile.include
A tests/qmp-cmd-test.c
M tests/qmp-test.c
Log Message:
-----------
qmp-cmd-test: Split off qmp-test
qmp-test is for QMP protocol tests. Commit e4a426e75ef added generic,
basic tests of query commands to it. Move them to their own test
program qmp-cmd-test, to keep qmp-test focused on the protocol.
Signed-off-by: Markus Armbruster <address@hidden>
Reviewed-by: Eric Blake <address@hidden>
Message-Id: <address@hidden>
Commit: aed877c53b5b8760b3fdb3605a212188c60f2eb3
https://github.com/qemu/qemu/commit/aed877c53b5b8760b3fdb3605a212188c60f2eb3
Author: Markus Armbruster <address@hidden>
Date: 2018-08-24 (Fri, 24 Aug 2018)
Changed paths:
M tests/libqtest.c
M tests/libqtest.h
M tests/qmp-test.c
Log Message:
-----------
qmp-test: Cover syntax and lexical errors
qmp-test neglects to cover QMP input that isn't valid JSON. libqtest
doesn't let us send such input. Add qtest_qmp_send_raw() for this
purpose, and put it to use in qmp-test.
Signed-off-by: Markus Armbruster <address@hidden>
Reviewed-by: Eric Blake <address@hidden>
Message-Id: <address@hidden>
[Commit message typo fixed]
Commit: e2f64a688b70e2e6882b534f9a6a68ef589c261d
https://github.com/qemu/qemu/commit/e2f64a688b70e2e6882b534f9a6a68ef589c261d
Author: Markus Armbruster <address@hidden>
Date: 2018-08-24 (Fri, 24 Aug 2018)
Changed paths:
M tests/libqtest.c
M tests/libqtest.h
M tests/test-qga.c
Log Message:
-----------
test-qga: Clean up how we test QGA synchronization
To permit recovering from arbitrary JSON parse errors, the JSON parser
resets itself on lexical errors. We recommend sending a 0xff byte for
that purpose, and test-qga covers this usage since commit 5229564b832.
That commit had to add an ugly hack to qmp_fd_vsend() to make capable
of sending this byte (it's designed to send only valid JSON).
The previous commit added a way to send arbitrary text. Put that to
use for this purpose, and drop the hack from qmp_fd_vsend().
Signed-off-by: Markus Armbruster <address@hidden>
Reviewed-by: Eric Blake <address@hidden>
Message-Id: <address@hidden>
Commit: 4e1df9b73480420dbd5b2d22a42038079bb6e265
https://github.com/qemu/qemu/commit/4e1df9b73480420dbd5b2d22a42038079bb6e265
Author: Markus Armbruster <address@hidden>
Date: 2018-08-24 (Fri, 24 Aug 2018)
Changed paths:
M tests/check-qjson.c
Log Message:
-----------
check-qjson: Cover escaped characters more thoroughly, part 1
escaped_string() first tests double quoted strings, then repeats a few
tests with single quotes. Repeat all of them: store the strings to
test without quotes, and wrap them in either kind of quote for
testing.
Signed-off-by: Markus Armbruster <address@hidden>
Reviewed-by: Eric Blake <address@hidden>
Message-Id: <address@hidden>
Commit: f3cfdd3a30a4bd0158d255daeabde027b76da83f
https://github.com/qemu/qemu/commit/f3cfdd3a30a4bd0158d255daeabde027b76da83f
Author: Markus Armbruster <address@hidden>
Date: 2018-08-24 (Fri, 24 Aug 2018)
Changed paths:
M tests/check-qjson.c
Log Message:
-----------
check-qjson: Streamline escaped_string()'s test strings
Merge a few closely related test strings, and drop a few redundant
ones.
Signed-off-by: Markus Armbruster <address@hidden>
Reviewed-by: Eric Blake <address@hidden>
Message-Id: <address@hidden>
Commit: e0fe2a978e9a8c0a712afa5cfd5bc38e389ae30f
https://github.com/qemu/qemu/commit/e0fe2a978e9a8c0a712afa5cfd5bc38e389ae30f
Author: Markus Armbruster <address@hidden>
Date: 2018-08-24 (Fri, 24 Aug 2018)
Changed paths:
M tests/check-qjson.c
Log Message:
-----------
check-qjson: Cover escaped characters more thoroughly, part 2
Cover escaped single quote, surrogates, invalid escapes, and
noncharacters. This demonstrates that valid surrogate pairs are
misinterpreted, and invalid surrogates and noncharacters aren't
rejected.
Signed-off-by: Markus Armbruster <address@hidden>
Reviewed-by: Eric Blake <address@hidden>
Message-Id: <address@hidden>
Commit: 069946f402de10f544e198b0b5c016e2a70e8dd4
https://github.com/qemu/qemu/commit/069946f402de10f544e198b0b5c016e2a70e8dd4
Author: Markus Armbruster <address@hidden>
Date: 2018-08-24 (Fri, 24 Aug 2018)
Changed paths:
M tests/check-qjson.c
Log Message:
-----------
check-qjson: Consolidate partly redundant string tests
simple_string() and single_quote_string() have become redundant with
escaped_string(), except for embedded single and double quotes.
Replace them by a test that covers just that.
Signed-off-by: Markus Armbruster <address@hidden>
Reviewed-by: Eric Blake <address@hidden>
Message-Id: <address@hidden>
Commit: 6ad8444f6aadb8b43c78583cab09e8839c79305f
https://github.com/qemu/qemu/commit/6ad8444f6aadb8b43c78583cab09e8839c79305f
Author: Markus Armbruster <address@hidden>
Date: 2018-08-24 (Fri, 24 Aug 2018)
Changed paths:
M tests/check-qjson.c
Log Message:
-----------
check-qjson: Cover UTF-8 in single quoted strings
utf8_string() tests only double quoted strings. Cover single quoted
strings, too: store the strings to test without quotes, then wrap them
in either kind of quote.
Signed-off-by: Markus Armbruster <address@hidden>
Reviewed-by: Eric Blake <address@hidden>
Message-Id: <address@hidden>
Commit: 32846e93047899bdde498d8c6a14d291fe6fa4e2
https://github.com/qemu/qemu/commit/32846e93047899bdde498d8c6a14d291fe6fa4e2
Author: Markus Armbruster <address@hidden>
Date: 2018-08-24 (Fri, 24 Aug 2018)
Changed paths:
M tests/check-qjson.c
Log Message:
-----------
check-qjson: Simplify utf8_string()
The previous commit made utf8_string()'s test_cases[].utf8_in
superfluous: we can use .json_in instead. Except for the case testing
U+0000. \x00 doesn't work in C strings, so it tests \\u0000 instead.
But testing \\uXXXX is escaped_string()'s job. It's covered there.
Test U+0001 here, and drop .utf8_in.
Signed-off-by: Markus Armbruster <address@hidden>
Reviewed-by: Eric Blake <address@hidden>
Message-Id: <address@hidden>
Commit: 5f454e662e710300b4c5414b2d5fd109ee18682b
https://github.com/qemu/qemu/commit/5f454e662e710300b4c5414b2d5fd109ee18682b
Author: Markus Armbruster <address@hidden>
Date: 2018-08-24 (Fri, 24 Aug 2018)
Changed paths:
M tests/check-qjson.c
Log Message:
-----------
check-qjson: Fix utf8_string() to test all invalid sequences
Some of utf8_string()'s test_cases[] contain multiple invalid
sequences. Testing that qobject_from_json() fails only tests we
reject at least one invalid sequence. That's incomplete.
Additionally test each non-space sequence in isolation.
This demonstrates that the JSON parser accepts invalid sequences
starting with \xC2..\xF4. Add a FIXME comment.
Signed-off-by: Markus Armbruster <address@hidden>
Reviewed-by: Eric Blake <address@hidden>
Message-Id: <address@hidden>
Commit: 6bc93a3401e32441c190c1f53e3967d226d4eb7c
https://github.com/qemu/qemu/commit/6bc93a3401e32441c190c1f53e3967d226d4eb7c
Author: Markus Armbruster <address@hidden>
Date: 2018-08-24 (Fri, 24 Aug 2018)
Changed paths:
M tests/check-qjson.c
M tests/qmp-test.c
Log Message:
-----------
check-qjson qmp-test: Cover control characters more thoroughly
RFC 8259 "The JavaScript Object Notation (JSON) Data Interchange
Format" requires control characters in strings to be escaped.
Demonstrate the JSON parser accepts U+0001 .. U+001F unescaped.
Signed-off-by: Markus Armbruster <address@hidden>
Reviewed-by: Eric Blake <address@hidden>
Message-Id: <address@hidden>
Commit: 2e933f5701c57cc857044fbd818e272059811e48
https://github.com/qemu/qemu/commit/2e933f5701c57cc857044fbd818e272059811e48
Author: Markus Armbruster <address@hidden>
Date: 2018-08-24 (Fri, 24 Aug 2018)
Changed paths:
M tests/check-qjson.c
Log Message:
-----------
check-qjson: Cover interpolation more thoroughly
Signed-off-by: Markus Armbruster <address@hidden>
Reviewed-by: Eric Blake <address@hidden>
Message-Id: <address@hidden>
Commit: a2ec6be72b80770b063cf08c95c78f0d36705355
https://github.com/qemu/qemu/commit/a2ec6be72b80770b063cf08c95c78f0d36705355
Author: Markus Armbruster <address@hidden>
Date: 2018-08-24 (Fri, 24 Aug 2018)
Changed paths:
M qobject/json-lexer.c
Log Message:
-----------
json: Fix lexer to include the bad character in JSON_ERROR token
json_lexer[] maps (lexer state, input character) to the new lexer
state. The input character is consumed unless the new state is
terminal and the input character doesn't belong to this token,
i.e. the state transition uses look-ahead. When this is the case,
input character '\0' would result in the same state transition.
TERMINAL_NEEDED_LOOKAHEAD() exploits this.
Except this is wrong for transitions to IN_ERROR. There, the
offending input character is in fact consumed: case IN_ERROR returns.
It isn't added to the JSON_ERROR token, though.
Fix that by making TERMINAL_NEEDED_LOOKAHEAD() return false for
transitions to IN_ERROR.
There's a slight complication. json_lexer_flush() passes input
character '\0' to flush an incomplete token. If this results in
JSON_ERROR, we'd now add the '\0' to the token. Suppress that.
Signed-off-by: Markus Armbruster <address@hidden>
Reviewed-by: Eric Blake <address@hidden>
Message-Id: <address@hidden>
Commit: 340db1ed82f8ced40a3e778c08963005369e2926
https://github.com/qemu/qemu/commit/340db1ed82f8ced40a3e778c08963005369e2926
Author: Markus Armbruster <address@hidden>
Date: 2018-08-24 (Fri, 24 Aug 2018)
Changed paths:
M qobject/json-lexer.c
M tests/check-qjson.c
M tests/qmp-test.c
Log Message:
-----------
json: Reject unescaped control characters
Fix the lexer to reject unescaped control characters in JSON strings,
in accordance with RFC 8259 "The JavaScript Object Notation (JSON)
Data Interchange Format".
Bonus: we now recover more nicely from unclosed strings. E.g.
{"one: 1}\n{"two": 2}
now recovers cleanly after the newline, where before the lexer
remained confused until the next unpaired double quote or lexical
error.
Signed-off-by: Markus Armbruster <address@hidden>
Reviewed-by: Eric Blake <address@hidden>
Message-Id: <address@hidden>
Commit: eddc0a7f0ad84edd0f8dd27d4a70a305ccd7bc5f
https://github.com/qemu/qemu/commit/eddc0a7f0ad84edd0f8dd27d4a70a305ccd7bc5f
Author: Markus Armbruster <address@hidden>
Date: 2018-08-24 (Fri, 24 Aug 2018)
Changed paths:
M qobject/json-lexer.c
Log Message:
-----------
json: Revamp lexer documentation
Signed-off-by: Markus Armbruster <address@hidden>
Reviewed-by: Eric Blake <address@hidden>
Message-Id: <address@hidden>
Commit: 00ea57fadcb899b7e21079dacb47b6fb8af5b2fa
https://github.com/qemu/qemu/commit/00ea57fadcb899b7e21079dacb47b6fb8af5b2fa
Author: Markus Armbruster <address@hidden>
Date: 2018-08-24 (Fri, 24 Aug 2018)
Changed paths:
M qobject/json-parser.c
Log Message:
-----------
json: Tighten and simplify qstring_from_escaped_str()'s loop
Simplify loop control, and assert that the string ends with the
appropriate quote (the lexer ensures it does).
Signed-off-by: Markus Armbruster <address@hidden>
Reviewed-by: Eric Blake <address@hidden>
Message-Id: <address@hidden>
Commit: a89d3104a29c400dfed4b675d6385a17223f9e0f
https://github.com/qemu/qemu/commit/a89d3104a29c400dfed4b675d6385a17223f9e0f
Author: Markus Armbruster <address@hidden>
Date: 2018-08-24 (Fri, 24 Aug 2018)
Changed paths:
M tests/check-qjson.c
Log Message:
-----------
check-qjson: Document we expect invalid UTF-8 to be rejected
The JSON parser rejects some invalid sequences, but accepts others
without correcting the problem.
We should either reject all invalid sequences, or minimize overlong
sequences and replace all other invalid sequences by a suitable
replacement character. A common choice for replacement is U+FFFD.
I'm going to implement the former. Update the comments in
utf8_string() to expect this.
Signed-off-by: Markus Armbruster <address@hidden>
Reviewed-by: Eric Blake <address@hidden>
Message-Id: <address@hidden>
Commit: e59f39d40397645477b959255aedfa17a7c9c779
https://github.com/qemu/qemu/commit/e59f39d40397645477b959255aedfa17a7c9c779
Author: Markus Armbruster <address@hidden>
Date: 2018-08-24 (Fri, 24 Aug 2018)
Changed paths:
M include/qemu/unicode.h
M qobject/json-parser.c
M tests/check-qjson.c
M util/unicode.c
Log Message:
-----------
json: Reject invalid UTF-8 sequences
We reject bytes that can't occur in valid UTF-8 (\xC0..\xC1,
\xF5..\xFF in the lexer. That's insufficient; there's plenty of
invalid UTF-8 not containing these bytes, as demonstrated by
check-qjson:
* Malformed sequences
- Unexpected continuation bytes
- Missing continuation bytes after start bytes other than
\xC0..\xC1, \xF5..\xFD.
* Overlong sequences with start bytes other than \xC0..\xC1,
\xF5..\xFD.
* Invalid code points
Fixing this in the lexer would be bothersome. Fixing it in the parser
is straightforward, so do that.
Signed-off-by: Markus Armbruster <address@hidden>
Reviewed-by: Eric Blake <address@hidden>
Message-Id: <address@hidden>
Commit: 574bf16ff1f836a442d8de1853eb248c218a591d
https://github.com/qemu/qemu/commit/574bf16ff1f836a442d8de1853eb248c218a591d
Author: Markus Armbruster <address@hidden>
Date: 2018-08-24 (Fri, 24 Aug 2018)
Changed paths:
M qobject/json-parser.c
Log Message:
-----------
json: Report first rather than last parse error
Quiz time! When a parser reports multiple errors, but the user gets
to see just one, which one is (on average) the least useful one?
Yes, you're right, it's the last one! You're clearly familiar with
compilers.
Which one does QEMU report?
Right again, the last one! You're clearly familiar with QEMU.
Reproducer: feeding
{"abc\xC2ijk": 1}\n
to QMP produces
{"error": {"class": "GenericError", "desc": "JSON parse error, key is not a
string in object"}}
Report the first error instead. The reproducer now produces
{"error": {"class": "GenericError", "desc": "JSON parse error, invalid
UTF-8 sequence in string"}}
Signed-off-by: Markus Armbruster <address@hidden>
Reviewed-by: Eric Blake <address@hidden>
Message-Id: <address@hidden>
Commit: de930f45cb56ccf7535cbacee3f3686d416f5283
https://github.com/qemu/qemu/commit/de930f45cb56ccf7535cbacee3f3686d416f5283
Author: Markus Armbruster <address@hidden>
Date: 2018-08-24 (Fri, 24 Aug 2018)
Changed paths:
M qobject/json-lexer.c
Log Message:
-----------
json: Leave rejecting invalid UTF-8 to parser
Both the lexer and the parser (attempt to) validate UTF-8 in JSON
strings.
The lexer rejects bytes that can't occur in valid UTF-8: \xC0..\xC1,
\xF5..\xFF. This rejects some, but not all invalid UTF-8. It also
rejects ASCII control characters \x00..\x1F, in accordance with RFC
8259 (see recent commit "json: Reject unescaped control characters").
When the lexer rejects, it ends the token right after the first bad
byte. Good when the bad byte is a newline. Not so good when it's
something like an overlong sequence in the middle of a string. For
instance, input
{"abc\xC0\xAFijk": 1}\n
produces the tokens
JSON_LCURLY {
JSON_ERROR "abc\xC0
JSON_ERROR \xAF
JSON_KEYWORD ijk
JSON_ERROR ": 1}\n
The parser then reports four errors
Invalid JSON syntax
Invalid JSON syntax
JSON parse error, invalid keyword 'ijk'
Invalid JSON syntax
before it recovers at the newline.
The commit before previous made the parser reject invalid UTF-8
sequences. Since then, anything the lexer rejects, the parser would
reject as well. Thus, the lexer's rejecting is unnecessary for
correctness, and harmful for error reporting.
However, we want to keep rejecting ASCII control characters in the
lexer, because that produces the behavior we want for unclosed
strings.
We also need to keep rejecting \xFF in the lexer, because we
documented that as a way to reset the JSON parser
(docs/interop/qmp-spec.txt section 2.6 QGA Synchronization), which
means we can't change how we recover from this error now. I wish we
hadn't done that.
I think we should treat \xFE the same as \xFF.
Change the lexer to accept \xC0..\xC1 and \xF5..\xFD. It now rejects
only \x00..\x1F and \xFE..\xFF. Error reporting for invalid UTF-8 in
strings is much improved, except for \xFE and \xFF. For the example
above, the lexer now produces
JSON_LCURLY {
JSON_STRING "abc\xC0\xAFijk"
JSON_COLON :
JSON_INTEGER 1
JSON_RCURLY
and the parser reports just
JSON parse error, invalid UTF-8 sequence in string
Signed-off-by: Markus Armbruster <address@hidden>
Reviewed-by: Eric Blake <address@hidden>
Message-Id: <address@hidden>
Commit: 4b1c0cd7c7f9f9cf2e46c0a9c9cd88b2cba3decd
https://github.com/qemu/qemu/commit/4b1c0cd7c7f9f9cf2e46c0a9c9cd88b2cba3decd
Author: Markus Armbruster <address@hidden>
Date: 2018-08-24 (Fri, 24 Aug 2018)
Changed paths:
M qobject/json-lexer.c
M qobject/json-parser.c
M tests/check-qjson.c
Log Message:
-----------
json: Accept overlong \xC0\x80 as U+0000 ("modified UTF-8")
Since the JSON grammer doesn't accept U+0000 anywhere, this merely
exchanges one kind of parse error for another. It's purely for
consistency with qobject_to_json(), which accepts \xC0\x80 (see commit
e2ec3f97680).
Signed-off-by: Markus Armbruster <address@hidden>
Reviewed-by: Eric Blake <address@hidden>
Message-Id: <address@hidden>
Commit: b2da4a4d7537567b44db60b7b79cd14f64e48f2f
https://github.com/qemu/qemu/commit/b2da4a4d7537567b44db60b7b79cd14f64e48f2f
Author: Markus Armbruster <address@hidden>
Date: 2018-08-24 (Fri, 24 Aug 2018)
Changed paths:
M qobject/json-lexer.c
M qobject/json-parser.c
Log Message:
-----------
json: Leave rejecting invalid escape sequences to parser
Both lexer and parser reject invalid escape sequences in strings. The
parser's check is useless.
The lexer ends the token right after the first non-well-formed byte.
This tends to lead to suboptimal error reporting. For instance, input
{"address@hidden": 1}
produces the tokens
JSON_LCURLY {
JSON_ERROR "abc\@
JSON_KEYWORD ijk
JSON_ERROR ": 1}\n
The parser then reports three errors
Invalid JSON syntax
JSON parse error, invalid keyword 'ijk'
Invalid JSON syntax
before it recovers at the newline.
Drop the lexer's escape sequence checking, and make it accept the same
characters after backslash it accepts elsewhere in strings. It now
produces
JSON_LCURLY {
JSON_STRING "address@hidden"
JSON_COLON :
JSON_INTEGER 1
JSON_RCURLY
and the parser reports just
JSON parse error, invalid escape sequence in string
While there, fix parse_string()'s inaccurate function comment.
Signed-off-by: Markus Armbruster <address@hidden>
Reviewed-by: Eric Blake <address@hidden>
Message-Id: <address@hidden>
Commit: de6decfe8e2939464fc27ac2ab4ef87689329fec
https://github.com/qemu/qemu/commit/de6decfe8e2939464fc27ac2ab4ef87689329fec
Author: Markus Armbruster <address@hidden>
Date: 2018-08-24 (Fri, 24 Aug 2018)
Changed paths:
M qobject/json-parser.c
Log Message:
-----------
json: Simplify parse_string()
Signed-off-by: Markus Armbruster <address@hidden>
Reviewed-by: Eric Blake <address@hidden>
Message-Id: <address@hidden>
Commit: 46a628b1398ae6a58d6847223736431225c4c0cc
https://github.com/qemu/qemu/commit/46a628b1398ae6a58d6847223736431225c4c0cc
Author: Markus Armbruster <address@hidden>
Date: 2018-08-24 (Fri, 24 Aug 2018)
Changed paths:
M qobject/json-parser.c
M tests/check-qjson.c
Log Message:
-----------
json: Reject invalid \uXXXX, fix \u0000
The JSON parser translates invalid \uXXXX to garbage instead of
rejecting it, and swallows \u0000.
Fix by using mod_utf8_encode() instead of flawed wchar_to_utf8().
Valid surrogate pairs are now differently broken: they're rejected
instead of translated to garbage. The next commit will fix them.
Signed-off-by: Markus Armbruster <address@hidden>
Reviewed-by: Eric Blake <address@hidden>
Message-Id: <address@hidden>
Commit: dc45a07c3628b82817a96fcb7df3d211d901af5d
https://github.com/qemu/qemu/commit/dc45a07c3628b82817a96fcb7df3d211d901af5d
Author: Markus Armbruster <address@hidden>
Date: 2018-08-24 (Fri, 24 Aug 2018)
Changed paths:
M qobject/json-parser.c
M tests/check-qjson.c
Log Message:
-----------
json: Fix \uXXXX for surrogate pairs
The JSON parser treats each half of a surrogate pair as unpaired
surrogate. Fix it to recognize surrogate pairs.
Signed-off-by: Markus Armbruster <address@hidden>
Reviewed-by: Eric Blake <address@hidden>
Message-Id: <address@hidden>
Commit: c473c379e1069079542c51f7063d44c2692abe6b
https://github.com/qemu/qemu/commit/c473c379e1069079542c51f7063d44c2692abe6b
Author: Markus Armbruster <address@hidden>
Date: 2018-08-24 (Fri, 24 Aug 2018)
Changed paths:
M tests/check-qjson.c
Log Message:
-----------
check-qjson: Fix and enable utf8_string()'s disabled part
Signed-off-by: Markus Armbruster <address@hidden>
Reviewed-by: Eric Blake <address@hidden>
Message-Id: <address@hidden>
Commit: 7c1e1d5481fe8d2e757469179f9ccd14e8838ed1
https://github.com/qemu/qemu/commit/7c1e1d5481fe8d2e757469179f9ccd14e8838ed1
Author: Marc-André Lureau <address@hidden>
Date: 2018-08-24 (Fri, 24 Aug 2018)
Changed paths:
M include/qapi/qmp/json-lexer.h
M include/qapi/qmp/json-streamer.h
M qobject/json-lexer.c
M qobject/json-streamer.c
Log Message:
-----------
json: remove useless return value from lexer/parser
The lexer always returns 0 when char feeding. Furthermore, none of the
caller care about the return value.
Signed-off-by: Marc-André Lureau <address@hidden>
Message-Id: <address@hidden>
Reviewed-by: Markus Armbruster <address@hidden>
Reviewed-by: Thomas Huth <address@hidden>
Signed-off-by: Markus Armbruster <address@hidden>
Message-Id: <address@hidden>
Commit: e8b19d7d7300366a1dd85273512657bbeab564ab
https://github.com/qemu/qemu/commit/e8b19d7d7300366a1dd85273512657bbeab564ab
Author: Marc-André Lureau <address@hidden>
Date: 2018-08-24 (Fri, 24 Aug 2018)
Changed paths:
M qobject/json-parser.c
Log Message:
-----------
json-parser: simplify and avoid JSONParserContext allocation
parser_context_new/free() are only used from json_parser_parse(). We
can fold the code there and avoid an allocation altogether.
Signed-off-by: Marc-André Lureau <address@hidden>
Message-Id: <address@hidden>
Reviewed-by: Markus Armbruster <address@hidden>
Message-Id: <address@hidden>
Commit: 037f2440888a22bd00ea0d8e37a1b7ed7d2bba88
https://github.com/qemu/qemu/commit/037f2440888a22bd00ea0d8e37a1b7ed7d2bba88
Author: Markus Armbruster <address@hidden>
Date: 2018-08-24 (Fri, 24 Aug 2018)
Changed paths:
M include/qapi/qmp/json-lexer.h
M include/qapi/qmp/json-streamer.h
M qobject/json-lexer.c
M qobject/json-streamer.c
Log Message:
-----------
json: Have lexer call streamer directly
json_lexer_init() takes the function to process a token as an
argument. It's always json_message_process_token(). Makes the code
harder to understand for no actual gain. Drop the indirection.
Signed-off-by: Markus Armbruster <address@hidden>
Reviewed-by: Eric Blake <address@hidden>
Message-Id: <address@hidden>
Commit: 62815d85aed71eff7b6c3a524705180fb04f5d30
https://github.com/qemu/qemu/commit/62815d85aed71eff7b6c3a524705180fb04f5d30
Author: Markus Armbruster <address@hidden>
Date: 2018-08-24 (Fri, 24 Aug 2018)
Changed paths:
M include/qapi/qmp/json-parser.h
M include/qapi/qmp/json-streamer.h
M monitor.c
M qapi/qmp-dispatch.c
M qga/main.c
M qobject/json-parser.c
M qobject/json-streamer.c
M qobject/qjson.c
M tests/check-qjson.c
M tests/libqtest.c
Log Message:
-----------
json: Redesign the callback to consume JSON values
The classical way to structure parser and lexer is to have the client
call the parser to get an abstract syntax tree, the parser call the
lexer to get the next token, and the lexer call some function to get
input characters.
Another way to structure them would be to have the client feed
characters to the lexer, the lexer feed tokens to the parser, and the
parser feed abstract syntax trees to some callback provided by the
client. This way is more easily integrated into an event loop that
dispatches input characters as they arrive.
Our JSON parser is kind of between the two. The lexer feeds tokens to
a "streamer" instead of a real parser. The streamer accumulates
tokens until it got the sequence of tokens that comprise a single JSON
value (it counts curly braces and square brackets to decide). It
feeds those token sequences to a callback provided by the client. The
callback passes each token sequence to the parser, and gets back an
abstract syntax tree.
I figure it was done that way to make a straightforward recursive
descent parser possible. "Get next token" becomes "pop the first
token off the token sequence". Drawback: we need to store a complete
token sequence. Each token eats 13 + input characters + malloc
overhead bytes.
Observations:
1. This is not the only way to use recursive descent. If we replaced
"get next token" by a coroutine yield, we could do without a
streamer.
2. The lexer reports errors by passing a JSON_ERROR token to the
streamer. This communicates the offending input characters and
their location, but no more.
3. The streamer reports errors by passing a null token sequence to the
callback. The (already poor) lexical error information is thrown
away.
4. Having the callback receive a token sequence duplicates the code to
convert token sequence to abstract syntax tree in every callback.
5. Known bug: the streamer silently drops incomplete token sequences.
This commit rectifies 4. by lifting the call of the parser from the
callbacks into the streamer. Later commits will address 3. and 5.
The lifting removes a bug from qjson.c's parse_json(): it passed a
pointer to a non-null Error * in certain cases, as demonstrated by
check-qjson.c.
json_parser_parse() is now unused. It's a stupid wrapper around
json_parser_parse_err(). Drop it, and rename json_parser_parse_err()
to json_parser_parse().
Signed-off-by: Markus Armbruster <address@hidden>
Reviewed-by: Eric Blake <address@hidden>
Message-Id: <address@hidden>
Commit: ff281a272f67a07d8ea29ca0350c4a3e0d3de73c
https://github.com/qemu/qemu/commit/ff281a272f67a07d8ea29ca0350c4a3e0d3de73c
Author: Markus Armbruster <address@hidden>
Date: 2018-08-24 (Fri, 24 Aug 2018)
Changed paths:
M qobject/json-parser.c
M qobject/json-streamer.c
Log Message:
-----------
json: Don't pass null @tokens to json_parser_parse()
json_parser_parse() normally returns the QObject on success. Except
it returns null when its @tokens argument is null.
Its only caller json_message_process_token() passes null @tokens when
emitting a lexical error. The call is a rather opaque way to say json
= NULL then.
Simplify matters by lifting the assignment to json out of the emit
path: initialize json to null, set it to the value of
json_parser_parse() when there's no lexical error. Drop the special
case from json_parser_parse().
Signed-off-by: Markus Armbruster <address@hidden>
Reviewed-by: Eric Blake <address@hidden>
Message-Id: <address@hidden>
Commit: 269e57ae28599b07368134a64452674009491593
https://github.com/qemu/qemu/commit/269e57ae28599b07368134a64452674009491593
Author: Markus Armbruster <address@hidden>
Date: 2018-08-24 (Fri, 24 Aug 2018)
Changed paths:
M qobject/json-streamer.c
Log Message:
-----------
json: Don't create JSON_ERROR tokens that won't be used
Signed-off-by: Markus Armbruster <address@hidden>
Reviewed-by: Eric Blake <address@hidden>
Message-Id: <address@hidden>
Commit: 61030280ca2d67bd63cb068250aee55849cd38ca
https://github.com/qemu/qemu/commit/61030280ca2d67bd63cb068250aee55849cd38ca
Author: Markus Armbruster <address@hidden>
Date: 2018-08-24 (Fri, 24 Aug 2018)
Changed paths:
M include/qapi/qmp/json-lexer.h
M qobject/json-lexer.c
M qobject/json-parser.c
Log Message:
-----------
json: Rename token JSON_ESCAPE & friends to JSON_INTERP
The JSON parser optionally supports interpolation. The code calls it
"escape". Awkward, because it uses the same term for escape sequences
within strings. The latter usage is consistent with RFC 8259 "The
JavaScript Object Notation (JSON) Data Interchange Format" and ISO C.
Call the former "interpolation" instead.
Signed-off-by: Markus Armbruster <address@hidden>
Reviewed-by: Eric Blake <address@hidden>
Message-Id: <address@hidden>
Commit: 2cbd15aa6f4d4694376dd0d231d56e572ac870c1
https://github.com/qemu/qemu/commit/2cbd15aa6f4d4694376dd0d231d56e572ac870c1
Author: Markus Armbruster <address@hidden>
Date: 2018-08-24 (Fri, 24 Aug 2018)
Changed paths:
M include/qapi/qmp/json-lexer.h
M qobject/json-lexer.c
M qobject/json-parser.c
M qobject/json-streamer.c
M tests/qmp-test.c
Log Message:
-----------
json: Treat unwanted interpolation as lexical error
The JSON parser optionally supports interpolation. The lexer
recognizes interpolation tokens unconditionally. The parser rejects
them when interpolation is disabled, in parse_interpolation().
However, it neglects to set an error then, which can make
json_parser_parse() fail without setting an error.
Move the check for unwanted interpolation from the parser's
parse_interpolation() into the lexer's finite state machine. When
interpolation is disabled, '%' is now handled like any other
unexpected character.
The next commit will improve how such lexical errors are handled.
Signed-off-by: Markus Armbruster <address@hidden>
Reviewed-by: Eric Blake <address@hidden>
Message-Id: <address@hidden>
Commit: 84a56f38b23440cb3127eaffe4e495826a29f18c
https://github.com/qemu/qemu/commit/84a56f38b23440cb3127eaffe4e495826a29f18c
Author: Markus Armbruster <address@hidden>
Date: 2018-08-24 (Fri, 24 Aug 2018)
Changed paths:
M include/qapi/qmp/qerror.h
M monitor.c
M qga/main.c
M qobject/json-lexer.c
M qobject/json-streamer.c
M tests/check-qjson.c
M tests/libqtest.c
Log Message:
-----------
json: Pass lexical errors and limit violations to callback
The callback to consume JSON values takes QObject *json, Error *err.
If both are null, the callback is supposed to make up an error by
itself. This sucks.
qjson.c's consume_json() neglects to do so, which makes
qobject_from_json() null instead of failing. I consider that a bug.
The culprit is json_message_process_token(): it passes two null
pointers when it runs into a lexical error or a limit violation. Fix
it to pass a proper Error object then. Update the callbacks:
* monitor.c's handle_qmp_command(): the code to make up an error is
now dead, drop it.
* qga/main.c's process_event(): lumps the "both null" case together
with the "not a JSON object" case. The former is now gone. The
error message "Invalid JSON syntax" is misleading for the latter.
Improve it to "Input must be a JSON object".
* qobject/qjson.c's consume_json(): no update; check-qjson
demonstrates qobject_from_json() now sets an error on lexical
errors, but still doesn't on some other errors.
* tests/libqtest.c's qmp_response(): the Error object is now reliable,
so use it to improve the error message.
Signed-off-by: Markus Armbruster <address@hidden>
Reviewed-by: Eric Blake <address@hidden>
Message-Id: <address@hidden>
Commit: f7617d45d4652ae10d38bd0c917d7488d155cccb
https://github.com/qemu/qemu/commit/f7617d45d4652ae10d38bd0c917d7488d155cccb
Author: Markus Armbruster <address@hidden>
Date: 2018-08-24 (Fri, 24 Aug 2018)
Changed paths:
M qobject/json-lexer.c
M qobject/json-parser.c
M tests/check-qjson.c
Log Message:
-----------
json: Leave rejecting invalid interpolation to parser
Both lexer and parser reject invalid interpolation specifications.
The parser's check is useless.
The lexer ends the token right after the first bad character. This
tends to lead to suboptimal error reporting. For instance, input
[ %04d ]
produces the tokens
JSON_LSQUARE [
JSON_ERROR %0
JSON_INTEGER 4
JSON_KEYWORD d
JSON_RSQUARE ]
The parser then yields an error, an object and two more errors:
error: Invalid JSON syntax
object: 4
error: JSON parse error, invalid keyword
error: JSON parse error, expecting value
Dumb down the lexer to accept [A-Za-z0-9]*. The parser's check is now
used. Emit a proper error there.
The lexer now produces
JSON_LSQUARE [
JSON_INTERP %04d
JSON_RSQUARE ]
and the parser reports just
JSON parse error, invalid interpolation '%04d'
Signed-off-by: Markus Armbruster <address@hidden>
Reviewed-by: Eric Blake <address@hidden>
Message-Id: <address@hidden>
Commit: 53a0d616fecab09870411573afc58fd24ffb8648
https://github.com/qemu/qemu/commit/53a0d616fecab09870411573afc58fd24ffb8648
Author: Markus Armbruster <address@hidden>
Date: 2018-08-24 (Fri, 24 Aug 2018)
Changed paths:
M qobject/json-parser.c
M tests/check-qjson.c
Log Message:
-----------
json: Replace %I64d, %I64u by %PRId64, %PRIu64
Support for %I64d got added in commit 2c0d4b36e7f "json: fix PRId64 on
Win32". We had to hard-code I64d because we used the lexer's finite
state machine to check interpolations. No more, so clean this up.
Additional conversion specifications would be easy enough to implement
when needed.
Signed-off-by: Markus Armbruster <address@hidden>
Reviewed-by: Eric Blake <address@hidden>
Message-Id: <address@hidden>
Commit: 4d40066142278a7a9cfbde1ae481e0e94a4d24c3
https://github.com/qemu/qemu/commit/4d40066142278a7a9cfbde1ae481e0e94a4d24c3
Author: Markus Armbruster <address@hidden>
Date: 2018-08-24 (Fri, 24 Aug 2018)
Changed paths:
M qobject/json-lexer.c
Log Message:
-----------
json: Improve names of lexer states related to numbers
Signed-off-by: Markus Armbruster <address@hidden>
Reviewed-by: Eric Blake <address@hidden>
Message-Id: <address@hidden>
Commit: 2a4794ba146d6560bd77ca840ff6908f81d585f4
https://github.com/qemu/qemu/commit/2a4794ba146d6560bd77ca840ff6908f81d585f4
Author: Markus Armbruster <address@hidden>
Date: 2018-08-24 (Fri, 24 Aug 2018)
Changed paths:
M qobject/qjson.c
M tests/check-qjson.c
Log Message:
-----------
qjson: Fix qobject_from_json() & friends for multiple values
qobject_from_json() & friends use the consume_json() callback to
receive either a value or an error from the parser.
When they are fed a string that contains more than either one JSON
value or one JSON syntax error, consume_json() gets called multiple
times.
When the last call receives a value, qobject_from_json() returns that
value. Any other values are leaked.
When any call receives an error, qobject_from_json() sets the first
error received. Any other errors are thrown away.
When values follow errors, qobject_from_json() returns both a value
and sets an error. That's bad. Impact:
* block.c's parse_json_protocol() ignores and leaks the value. It's
used to to parse pseudo-filenames starting with "json:". The
pseudo-filenames can come from the user or from image meta-data such
as a QCOW2 image's backing file name.
* vl.c's parse_display_qapi() ignores and leaks the error. It's used
to parse the argument of command line option -display.
* vl.c's main() case QEMU_OPTION_blockdev ignores the error and leaves
it in @err. main() will then pass a pointer to a non-null Error *
to net_init_clients(), which is forbidden. It can lead to assertion
failure or other misbehavior.
* check-qjson.c's multiple_values() demonstrates the badness.
* The other callers are not affected since they only pass strings with
exactly one JSON value or, in the case of negative tests, one
error.
The impact on the _nofail() functions is relatively harmless. They
abort when any call receives an error. Else they return the last
value, and leak the others, if any.
Fix consume_json() as follows. On the first call, save value and
error as before. On subsequent calls, if any, don't save them. If
the first call saved a value, the next call, if any, replaces the
value by an "Expecting at most one JSON value" error. Take care not
to leak values or errors that aren't saved.
Signed-off-by: Markus Armbruster <address@hidden>
Reviewed-by: Eric Blake <address@hidden>
Message-Id: <address@hidden>
Commit: e06d008ac833ac220b920f4d8858e6f62f7a59f9
https://github.com/qemu/qemu/commit/e06d008ac833ac220b920f4d8858e6f62f7a59f9
Author: Markus Armbruster <address@hidden>
Date: 2018-08-24 (Fri, 24 Aug 2018)
Changed paths:
M qobject/json-parser.c
Log Message:
-----------
json: Fix latent parser aborts at end of input
json-parser.c carefully reports end of input like this:
token = parser_context_pop_token(ctxt);
if (token == NULL) {
parse_error(ctxt, NULL, "premature EOI");
goto out;
}
Except parser_context_pop_token() can't return null, it fails its
assertion instead. Same for parser_context_peek_token(). Broken in
commit 65c0f1e9558, and faithfully preserved in commit 95385fe9ace.
Only a latent bug, because the streamer throws away any input that
could trigger it.
Drop the assertions, so we can fix the streamer in the next commit.
Signed-off-by: Markus Armbruster <address@hidden>
Reviewed-by: Eric Blake <address@hidden>
Message-Id: <address@hidden>
Commit: f9277915ee7b2654f5347c4c261c8a0651fdd561
https://github.com/qemu/qemu/commit/f9277915ee7b2654f5347c4c261c8a0651fdd561
Author: Markus Armbruster <address@hidden>
Date: 2018-08-24 (Fri, 24 Aug 2018)
Changed paths:
M include/qapi/qmp/json-lexer.h
M qobject/json-lexer.c
M qobject/json-streamer.c
M tests/check-qjson.c
Log Message:
-----------
json: Fix streamer not to ignore trailing unterminated structures
json_message_process_token() accumulates tokens until it got the
sequence of tokens that comprise a single JSON value (it counts curly
braces and square brackets to decide). It feeds those token sequences
to json_parser_parse(). If a non-empty sequence of tokens remains at
the end of the parse, it's silently ignored. check-qjson.c cases
unterminated_array(), unterminated_array_comma(), unterminated_dict(),
unterminated_dict_comma() demonstrate this bug.
Fix as follows. Introduce a JSON_END_OF_INPUT token. When the
streamer receives it, it feeds the accumulated tokens to
json_parser_parse().
Signed-off-by: Markus Armbruster <address@hidden>
Reviewed-by: Eric Blake <address@hidden>
Message-Id: <address@hidden>
Commit: 5d50113cf675ec96337ac6eaf81d83fbf69273bc
https://github.com/qemu/qemu/commit/5d50113cf675ec96337ac6eaf81d83fbf69273bc
Author: Markus Armbruster <address@hidden>
Date: 2018-08-24 (Fri, 24 Aug 2018)
Changed paths:
M qobject/json-parser.c
Log Message:
-----------
json: Assert json_parser_parse() consumes all tokens on success
Signed-off-by: Markus Armbruster <address@hidden>
Reviewed-by: Eric Blake <address@hidden>
Message-Id: <address@hidden>
Commit: dd98e8481992741a6b5ec0bdfcee05c1c8f602d6
https://github.com/qemu/qemu/commit/dd98e8481992741a6b5ec0bdfcee05c1c8f602d6
Author: Markus Armbruster <address@hidden>
Date: 2018-08-24 (Fri, 24 Aug 2018)
Changed paths:
M block.c
M qapi/qobject-input-visitor.c
M qobject/qjson.c
M tests/check-qjson.c
Log Message:
-----------
qjson: Have qobject_from_json() & friends reject empty and blank
The last case where qobject_from_json() & friends return null without
setting an error is empty or blank input. Callers:
* block.c's parse_json_protocol() reports "Could not parse the JSON
options". It's marked as a work-around, because it also covered
actual bugs, but they got fixed in the previous few commits.
* qobject_input_visitor_new_str() reports "JSON parse error". Also
marked as work-around. The recent fixes have made this unreachable,
because it currently gets called only for input starting with '{'.
* check-qjson.c's empty_input() and blank_input() demonstrate the
behavior.
* The other callers are not affected since they only pass input with
exactly one JSON value or, in the case of negative tests, one error.
Fail with "Expecting a JSON value" instead of returning null, and
simplify callers.
Signed-off-by: Markus Armbruster <address@hidden>
Reviewed-by: Eric Blake <address@hidden>
Message-Id: <address@hidden>
Commit: da09cfbf9dcd07c48fe95bdfb2968305de9b9690
https://github.com/qemu/qemu/commit/da09cfbf9dcd07c48fe95bdfb2968305de9b9690
Author: Markus Armbruster <address@hidden>
Date: 2018-08-24 (Fri, 24 Aug 2018)
Changed paths:
M qobject/json-streamer.c
Log Message:
-----------
json: Enforce token count and size limits more tightly
Token count and size limits exist to guard against excessive heap
usage. We check them only after we created the token on the heap.
That's assigning a cowboy to the barn to lasso the horse after it has
bolted. Close the barn door instead: check before we create the
token.
Signed-off-by: Markus Armbruster <address@hidden>
Reviewed-by: Eric Blake <address@hidden>
Message-Id: <address@hidden>
Commit: 8d3265b3d00db1071d1d3bf8433b4818088fdeb5
https://github.com/qemu/qemu/commit/8d3265b3d00db1071d1d3bf8433b4818088fdeb5
Author: Markus Armbruster <address@hidden>
Date: 2018-08-24 (Fri, 24 Aug 2018)
Changed paths:
M qobject/json-streamer.c
Log Message:
-----------
json: Streamline json_message_process_token()
Signed-off-by: Markus Armbruster <address@hidden>
Reviewed-by: Eric Blake <address@hidden>
Message-Id: <address@hidden>
Commit: a2731e08ee8633fcdc2af944b8f8f315678f7669
https://github.com/qemu/qemu/commit/a2731e08ee8633fcdc2af944b8f8f315678f7669
Author: Markus Armbruster <address@hidden>
Date: 2018-08-24 (Fri, 24 Aug 2018)
Changed paths:
M include/qapi/qmp/json-streamer.h
M qobject/json-parser.c
M qobject/json-streamer.c
Log Message:
-----------
json: Unbox tokens queue in JSONMessageParser
Signed-off-by: Markus Armbruster <address@hidden>
Reviewed-by: Eric Blake <address@hidden>
Message-Id: <address@hidden>
Commit: abe7c2067c21a89c6fd4cf2d8ba2fa37160d1d55
https://github.com/qemu/qemu/commit/abe7c2067c21a89c6fd4cf2d8ba2fa37160d1d55
Author: Markus Armbruster <address@hidden>
Date: 2018-08-24 (Fri, 24 Aug 2018)
Changed paths:
M include/qapi/qmp/json-parser.h
M include/qapi/qmp/json-streamer.h
M qobject/json-parser.c
M qobject/json-streamer.c
Log Message:
-----------
json: Make JSONToken opaque outside json-parser.c
Signed-off-by: Markus Armbruster <address@hidden>
Reviewed-by: Eric Blake <address@hidden>
Message-Id: <address@hidden>
Commit: 812ce33eadfc419795a5d1c7fe8f487fdfe2188c
https://github.com/qemu/qemu/commit/812ce33eadfc419795a5d1c7fe8f487fdfe2188c
Author: Markus Armbruster <address@hidden>
Date: 2018-08-24 (Fri, 24 Aug 2018)
Changed paths:
M include/qapi/qmp/json-parser.h
M qobject/json-lexer.c
M qobject/json-streamer.c
M qobject/qbool.c
M qobject/qlist.c
M qobject/qnull.c
M qobject/qnum.c
M qobject/qobject.c
M qobject/qstring.c
Log Message:
-----------
qobject: Drop superfluous includes of qemu-common.h
Signed-off-by: Markus Armbruster <address@hidden>
Reviewed-by: Eric Blake <address@hidden>
Message-Id: <address@hidden>
Commit: 86cdf9ec8dec2763702cc52fa412d108a5dc9608
https://github.com/qemu/qemu/commit/86cdf9ec8dec2763702cc52fa412d108a5dc9608
Author: Markus Armbruster <address@hidden>
Date: 2018-08-24 (Fri, 24 Aug 2018)
Changed paths:
R include/qapi/qmp/json-lexer.h
M include/qapi/qmp/json-parser.h
R include/qapi/qmp/json-streamer.h
M monitor.c
M qga/main.c
M qobject/json-lexer.c
A qobject/json-parser-int.h
M qobject/json-parser.c
M qobject/json-streamer.c
M qobject/qjson.c
M tests/libqtest.c
Log Message:
-----------
json: Clean up headers
The JSON parser has three public headers, json-lexer.h, json-parser.h,
json-streamer.h. They all contain stuff that is of no interest
outside qobject/json-*.c.
Collect the public interface in include/qapi/qmp/json-parser.h, and
everything else in qobject/json-parser-int.h.
Signed-off-by: Markus Armbruster <address@hidden>
Reviewed-by: Eric Blake <address@hidden>
Message-Id: <address@hidden>
Commit: 83273e84d9646bb902e1debfdcb68c8afd38614f
https://github.com/qemu/qemu/commit/83273e84d9646bb902e1debfdcb68c8afd38614f
Author: Markus Armbruster <address@hidden>
Date: 2018-08-24 (Fri, 24 Aug 2018)
Changed paths:
M tests/drive_del-test.c
Log Message:
-----------
tests/drive_del-test: Fix harmless JSON interpolation bug
test_after_failed_device_add() does this:
response = qmp("{'execute': 'device_add',"
" 'arguments': {"
" 'driver': 'virtio-blk-%s',"
" 'drive': 'drive0'"
"}}", qvirtio_get_dev_type());
Wrong. An interpolation specification must be a JSON token, it
doesn't work within JSON string tokens. The code above doesn't use
the value of qvirtio_get_dev_type(), and sends arguments
{"driver": "virtio-blk-%s", "drive": "drive0"}}
The command fails because there is no driver named "virtio-blk-%".
Harmless, since the test wants the command to fail. Screwed up in
commit 2f84a92ec63.
Fix the obvious way. The command now fails because the drive is
empty, like it did before commit 2f84a92ec63.
Signed-off-by: Markus Armbruster <address@hidden>
Reviewed-by: Eric Blake <address@hidden>
Message-Id: <address@hidden>
Commit: ada74c3ba1b4f51e4462e186c251eaa974015bb8
https://github.com/qemu/qemu/commit/ada74c3ba1b4f51e4462e186c251eaa974015bb8
Author: Markus Armbruster <address@hidden>
Date: 2018-08-24 (Fri, 24 Aug 2018)
Changed paths:
M qobject/json-parser.c
Log Message:
-----------
json: Keep interpolation state in JSONParserContext
The recursive descent parser passes along a pointer to
JSONParserContext. It additionally passes a pointer to interpolation
state (a va_alist *) as needed to reach its consumer
parse_interpolation().
Stuffing the latter pointer into JSONParserContext saves us the
trouble of passing it along, so do that.
Signed-off-by: Markus Armbruster <address@hidden>
Reviewed-by: Eric Blake <address@hidden>
Message-Id: <address@hidden>
Commit: 16a485992112be1c8b47b58b0124357db9037093
https://github.com/qemu/qemu/commit/16a485992112be1c8b47b58b0124357db9037093
Author: Markus Armbruster <address@hidden>
Date: 2018-08-24 (Fri, 24 Aug 2018)
Changed paths:
M qobject/json-parser.c
M tests/check-qjson.c
Log Message:
-----------
json: Improve safety of qobject_from_jsonf_nofail() & friends
The JSON parser optionally supports interpolation. This is used to
build QObjects by parsing string templates. The templates are C
literals, so parse errors (such as invalid interpolation
specifications) are actually programming errors. Consequently, the
functions providing parsing with interpolation
(qobject_from_jsonf_nofail(), qobject_from_vjsonf_nofail(),
qdict_from_jsonf_nofail(), qdict_from_vjsonf_nofail()) pass
&error_abort to the parser.
However, there's another, more dangerous kind of programming error:
since we use va_arg() to get the value to interpolate, behavior is
undefined when the variable argument isn't consistent with the
interpolation specification.
The same problem exists with printf()-like functions, and the solution
is to have the compiler check consistency. This is what
GCC_FMT_ATTR() is about.
To enable this type checking for interpolation as well, we carefully
chose our interpolation specifications to match printf conversion
specifications, and decorate functions parsing templates with
GCC_FMT_ATTR().
Note that this only protects against undefined behavior due to type
errors. It can't protect against use of invalid interpolation
specifications that happen to be valid printf conversion
specifications.
However, there's still a gaping hole in the type checking: GCC
recognizes '%' as start of printf conversion specification anywhere in
the template, but the parser recognizes it only outside JSON strings.
For instance, if someone were to pass a "{ '%s': %d }" template, GCC
would require a char * and an int argument, but the parser would
va_arg() only an int argument, resulting in undefined behavior.
Avoid undefined behavior by catching the programming error at run
time: have the parser recognize and reject '%' in JSON strings.
Signed-off-by: Markus Armbruster <address@hidden>
Reviewed-by: Eric Blake <address@hidden>
Message-Id: <address@hidden>
Commit: 8bca4613e6cddd948895b8db3def05950463495b
https://github.com/qemu/qemu/commit/8bca4613e6cddd948895b8db3def05950463495b
Author: Markus Armbruster <address@hidden>
Date: 2018-08-24 (Fri, 24 Aug 2018)
Changed paths:
M qobject/json-parser.c
M tests/check-qjson.c
Log Message:
-----------
json: Support %% in JSON strings when interpolating
The previous commit makes JSON strings containing '%' awkward to
express in templates: you'd have to mask the '%' with an Unicode
escape \u0025. No template currently contains such JSON strings.
Support the printf conversion specification %% in JSON strings as a
convenience anyway, because it's trivially easy to do.
Signed-off-by: Markus Armbruster <address@hidden>
Reviewed-by: Eric Blake <address@hidden>
Message-Id: <address@hidden>
Commit: 37aded92c27d0e56cd27f1c29494fc9f8c873cdd
https://github.com/qemu/qemu/commit/37aded92c27d0e56cd27f1c29494fc9f8c873cdd
Author: Markus Armbruster <address@hidden>
Date: 2018-08-24 (Fri, 24 Aug 2018)
Changed paths:
M include/qapi/qmp/qnum.h
M qapi/introspect.json
M qobject/json-parser.c
Log Message:
-----------
json: Update references to RFC 7159 to RFC 8259
RFC 8259 (December 2017) obsoletes RFC 7159 (March 2014).
Signed-off-by: Markus Armbruster <address@hidden>
Message-Id: <address@hidden>
Reviewed-by: Eric Blake <address@hidden>
Commit: cc9821fa9ac43728a7ece0a5e42e0147e6aadbf4
https://github.com/qemu/qemu/commit/cc9821fa9ac43728a7ece0a5e42e0147e6aadbf4
Author: Peter Maydell <address@hidden>
Date: 2018-08-25 (Sat, 25 Aug 2018)
Changed paths:
M MAINTAINERS
M block.c
M docs/interop/qmp-spec.txt
R include/qapi/qmp/json-lexer.h
M include/qapi/qmp/json-parser.h
R include/qapi/qmp/json-streamer.h
M include/qapi/qmp/qerror.h
M include/qapi/qmp/qnum.h
M include/qemu/unicode.h
M monitor.c
M qapi/introspect.json
M qapi/qmp-dispatch.c
M qapi/qobject-input-visitor.c
M qga/main.c
M qobject/json-lexer.c
A qobject/json-parser-int.h
M qobject/json-parser.c
M qobject/json-streamer.c
M qobject/qbool.c
M qobject/qjson.c
M qobject/qlist.c
M qobject/qnull.c
M qobject/qnum.c
M qobject/qobject.c
M qobject/qstring.c
M tests/Makefile.include
M tests/check-qjson.c
M tests/drive_del-test.c
M tests/libqtest.c
M tests/libqtest.h
A tests/qmp-cmd-test.c
M tests/qmp-test.c
M tests/test-qga.c
M util/unicode.c
Log Message:
-----------
Merge remote-tracking branch 'remotes/armbru/tags/pull-qobject-2018-08-24'
into staging
QObject patches for 2018-08-24
# gpg: Signature made Fri 24 Aug 2018 20:28:53 BST
# gpg: using RSA key 3870B400EB918653
# gpg: Good signature from "Markus Armbruster <address@hidden>"
# gpg: aka "Markus Armbruster <address@hidden>"
# Primary key fingerprint: 354B C8B3 D7EB 2A6B 6867 4E5F 3870 B400 EB91 8653
* remotes/armbru/tags/pull-qobject-2018-08-24: (58 commits)
json: Update references to RFC 7159 to RFC 8259
json: Support %% in JSON strings when interpolating
json: Improve safety of qobject_from_jsonf_nofail() & friends
json: Keep interpolation state in JSONParserContext
tests/drive_del-test: Fix harmless JSON interpolation bug
json: Clean up headers
qobject: Drop superfluous includes of qemu-common.h
json: Make JSONToken opaque outside json-parser.c
json: Unbox tokens queue in JSONMessageParser
json: Streamline json_message_process_token()
json: Enforce token count and size limits more tightly
qjson: Have qobject_from_json() & friends reject empty and blank
json: Assert json_parser_parse() consumes all tokens on success
json: Fix streamer not to ignore trailing unterminated structures
json: Fix latent parser aborts at end of input
qjson: Fix qobject_from_json() & friends for multiple values
json: Improve names of lexer states related to numbers
json: Replace %I64d, %I64u by %PRId64, %PRIu64
json: Leave rejecting invalid interpolation to parser
json: Pass lexical errors and limit violations to callback
...
Signed-off-by: Peter Maydell <address@hidden>
Compare: https://github.com/qemu/qemu/compare/e2e6fa67931f...cc9821fa9ac4
**NOTE:** This service has been marked for deprecation:
https://developer.github.com/changes/2018-04-25-github-services-deprecation/
Functionality will be removed from GitHub.com on January 31st, 2019.
| [Prev in Thread] |
Current Thread |
[Next in Thread] |
- [Qemu-commits] [qemu/qemu] 72e9e5: docs/interop/qmp-spec: How to force known good par...,
GitHub <=