Re: [PATCH 1/2] tests: Add 'fetch-acceptance' rule

[Wainer dos Santos Moschetta]

Hello,

On 7/24/20 4:35 AM, Philippe Mathieu-Daudé wrote:
> Add a rule to fetch acceptance test assets.
>
> This is particularly useful in a CI context, when a single job
> can fetch and save the cache so other jobs reuse it directly.
>
> It is also useful to measure the time spent downloading the
> assets versus the time spent running the tests.
>
> Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
> ---
>   tests/Makefile.include | 9 +++++++++
>   1 file changed, 9 insertions(+)
>
> diff --git a/tests/Makefile.include b/tests/Makefile.include
> index c7e4646ded..238974d8da 100644
> --- a/tests/Makefile.include
> +++ b/tests/Makefile.include
> @@ -944,6 +944,15 @@ get-vm-image-fedora-31-%: check-venv
>   # download all vm images, according to defined targets
>   get-vm-images: check-venv $(patsubst %,get-vm-image-fedora-31-%, 
> $(FEDORA_31_DOWNLOAD))
>   
> +# fetch acceptance test assets
> +fetch-acceptance: check-venv

This new target misses an entry on check-help.

> +       $(call quiet-command, \
> +            $(TESTS_VENV_DIR)/bin/python -m avocado \
> +            $(if $(V),--show=$(AVOCADO_SHOW)) \
> +            assets fetch \

Perhaps pass '--ignore-errors' so that intermittent network failurse 
won't disturb the execution (the test will have a second chance to 
download the asset later when it executes).

> +            $(wildcard tests/acceptance/*.py), \
> +            "AVOCADO", "tests/acceptance")

nit: print "Downloading acceptance tests assets" (similar to 
get-vm-image-fedora-32-% target).

Talking about get-vm-images...that target is pre-req of 
check-acceptance, which makes me think that fetch-acceptance should be 
either (for the sake of consistency.) The downside is that - as a 
developer running it on my machine - `avocado assets fetch` will attempt 
to download artifacts even for those tests which I'm not going to run 
anyway. Any opinion?

Regards,

Wainer

> +
>   check-acceptance: check-venv $(TESTS_RESULTS_DIR) get-vm-images
>         $(call quiet-command, \
>               $(TESTS_VENV_DIR)/bin/python -m avocado \