Re: [Qemu-ppc] [PATCH v2 6/6] .travis.yml: Split enterprise vs. hobbyist acceptance test job

[Philippe Mathieu-Daudé]

On 9/16/19 11:46 AM, Alex Bennée wrote:
> 
> Philippe Mathieu-Daudé <address@hidden> writes:
> 
>> On 9/16/19 10:43 AM, Alex Bennée wrote:
>>> Philippe Mathieu-Daudé <address@hidden> writes:
>>>
>>>> Signed-off-by: Philippe Mathieu-Daudé <address@hidden>
>>>> ---
>>>>  .travis.yml | 18 ++++++++++++++++--
>>>>  1 file changed, 16 insertions(+), 2 deletions(-)
>>>>
>>>> diff --git a/.travis.yml b/.travis.yml
>>>> index 69a37f7387..753276eb33 100644
>>>> --- a/.travis.yml
>>>> +++ b/.travis.yml
>>>> @@ -265,9 +265,23 @@ matrix:
>>>>          - "3.6"
>>>>
>>>>
>>>> -    # Acceptance (Functional) tests
>>>> +    # Acceptance (Functional) tests [enterprise]
>>>>      - env:
>>>> -        - CONFIG="--python=/usr/bin/python3 
>>>> --target-list=x86_64-softmmu,mips-softmmu,mips64el-softmmu,aarch64-softmmu,arm-softmmu,s390x-softmmu,alpha-softmmu,ppc-softmmu,ppc64-softmmu,m68k-softmmu"
>>>> +        - CONFIG="--python=/usr/bin/python3
>>>> --target-list=x86_64-softmmu,mips64el-softmmu,aarch64-softmmu,s390x-softmmu,ppc64-softmmu"
>>>
>>> You could use:
>>>
>>>   --target-list=${MAIN_SOFTMMU_TARGETS}
>>>
>>> and
>>>
>>>   --disable-user --target-list-exclude=${MAIN_SOFTMMU_TARGETS}
>>
>> I like the idea, but this variable is slighly different:
>>
>> MAIN_SOFTMMU_TARGETS="aarch64-softmmu,arm-softmmu,i386-softmmu,mips-softmmu,mips64-softmmu,ppc64-softmmu,riscv64-softmmu,s390x-softmmu,x86_64-softmmu"
>>
>> Are you suggesting we use MAIN_SOFTMMU_TARGETS for 'Enterprise'
>> targets?
> 
> Broadly - I'm note sure how we want to make the distinction in QEMU. We
> have targets which are actively maintained and represent current
> architectures that people still ship software to run on. The other
> targets are machines that are either esoteric designs that exist in
> particular niches or of academic interest and the old architecture kept
> alive out of a sense of nostalgia.
> 
>>
>> As long as mips64-softmmu is present, I think we can remove mips-softmmu
>> from MAIN_TARGETS. Maybe i386-softmmu can be removed too.
> 
> Well the MIPS targets are actively maintained although for 32 bit you
> don't see many distros for them these days. There might be an argument
> for demoting i386-softmmu as I'm guessing there isn't much 32 bit kvm in
> the enterprise right now (although It would be unsurprising to be told
> otherwise).
> 
>>
>> That would give the following hobbyist list:
>>
>> i386-softmmu,mips-softmmu,alpha-softmmu,ppc-softmmu,m68k-softmmu
> 
> I'm not sure about the hobbyist moniker as pretty much any of these TCG
> targets is more actively maintained than the x86 which is mostly all
> about the HW virtualisation these days. Trouble is I can't think of
> another name that doesn't sound condescending:
> 
>   HISTORICAL_SOFTMMU_TARGETS
>   LEGACY_SOFTMMU_TARGETS
>   FASTERTHANNATIVE_SOFTMMU_TARGETS
>   ARCHIVISTS_SOFTMMU_TARGETS
> 
> ? As ever naming things is hard....

:) I'm not trying to classify tests, I simply want users/maintainers to
not be reluctant/scared to add tests because "CI takes too long".

Also, I find it practical to have small jobs, what I do when testing a
series when I know it only affects a set of jobs, I push and directly
click "cancel all jobs" on the Travis web, then I just click on "restart
this job" on the set I'm interested in.
This way I can quickly figure out the series is OK or rework it.

>> (I plan to add hppa-softmmu there too).
>>
>>>> +        - TEST_CMD="make check-acceptance"
>>>> +      after_failure:
>>>> +        - cat tests/results/latest/job.log
>>>> +      addons:
>>>> +        apt:
>>>> +          packages:
>>>> +            - python3-pil
>>>> +            - python3-pip
>>>> +            - python3.5-venv
>>>> +
>>>> +
>>>> +    # Acceptance (Functional) tests [hobbyist]
>>>> +    - env:
>>>> +        - CONFIG="--python=/usr/bin/python3 
>>>> --target-list=mips-softmmu,arm-softmmu,alpha-softmmu,ppc-softmmu,m68k-softmmu"
>>>>          - TEST_CMD="make check-acceptance"
>>>>        after_failure:
>>>>          - cat tests/results/latest/job.log
>>>
>>>
>>> --
>>> Alex Bennée
>>>
> 
> 
> --
> Alex Bennée
>