Re: [Qemu-ppc] [Qemu-devel] [RFC PATCH] tests/device-introspect: Test devices with all machines, not only with "none"

[Thomas Huth]

On 26.04.2018 13:45, Markus Armbruster wrote:
> Thomas Huth <address@hidden> writes:
[...]
>> @@ -260,6 +263,26 @@ static void test_abstract_interfaces(void)
>>      qtest_end();
>>  }
>>  
>> +static void add_machine_test_case(const char *mname)
>> +{
>> +    char *path, *args;
>> +
>> +    /* Ignore blacklisted machines */
>> +    if (g_str_equal("xenfv", mname) || g_str_equal("xenpv", mname)) {
>> +        return;
>> +    }
>> +
>> +    path = g_strdup_printf("device/introspect/concrete-defaults-%s", mname);
>> +    args = g_strdup_printf("-machine %s", mname);
>> +    qtest_add_data_func(path, args, test_device_intro_concrete);
> 
> This runs test_device_intro_concrete() with "-machine M" for all machine
> types M, in SPEED=slow mode.
> 
>> +    g_free(path);
>> +
>> +    path = g_strdup_printf("device/introspect/concrete-nodefaults-%s", 
>> mname);
>> +    args = g_strdup_printf("-nodefaults -machine %s", mname);
>> +    qtest_add_data_func(path, args, test_device_intro_concrete);
> 
> This runs test_device_intro_concrete() with "-nodefaults -machine M" for
> all machine types M, in SPEED=slow mode.
> 
> Has "without -nodefaults" exposed additional bugs?

After testing this with all machines, I had to discover that
"-nodefaults" does not work so easily: A lot of the embedded machines
(especially the ARM machines) simply refuse to work with "-nodefaults"
and exit immediately instead. E.g.:

$ arm-softmmu/qemu-system-arm -nodefaults -nographic -M n810,accel=qtest
qemu-system-arm: missing SecureDigital device

So we'd either need a rather big black list for the machines that do not
work, or simply drop the "-nodefaults" tests from this patch.

> Please mention "with and without -nodefaults" in the commit message.
> 
> I'd try "with -nodefaults" before "without", because "with" is the
> simpler test case.

For most boards, it seems rather to be the more "difficult" setting
since most boards are only tested without "-nodefaults" obviously.

>> +    g_free(path);
>> +}
>> +
>>  int main(int argc, char **argv)
>>  {
>>      g_test_init(&argc, &argv, NULL);
>> @@ -268,8 +291,12 @@ int main(int argc, char **argv)
>>      qtest_add_func("device/introspect/list-fields", test_qom_list_fields);
>>      qtest_add_func("device/introspect/none", test_device_intro_none);
>>      qtest_add_func("device/introspect/abstract", 
>> test_device_intro_abstract);
>> -    qtest_add_func("device/introspect/concrete", 
>> test_device_intro_concrete);
>>      qtest_add_func("device/introspect/abstract-interfaces", 
>> test_abstract_interfaces);
>> +    qtest_add_data_func("device/introspect/concrete", g_strdup(common_args),
>> +                        test_device_intro_concrete);
> 
> This runs test_device_intro_concrete() with "-nodefaults -machine
> none".  Duplicate in SPEED=slow mode?

Yes, it's a duplicate, we should skip that in SPEED=slow mode.

 Thomas