Re: [PATCH v3 2/5] tests/acceptance/boot_linux_console: remove Armbian 19.11.3 bionic test for orangepi-pc machine

[Philippe Mathieu-Daudé]

On 3/22/21 6:12 PM, Willian Rampazzo wrote:
> Hi Philippe,
> 
> On Mon, Mar 22, 2021 at 1:59 PM Philippe Mathieu-Daudé <f4bug@amsat.org> 
> wrote:
>>
>> On Mon, Mar 22, 2021 at 5:54 PM Philippe Mathieu-Daudé <f4bug@amsat.org> 
>> wrote:
>>>
>>> Hi Willian,
>>>
>>> On 3/8/21 9:44 PM, Willian Rampazzo wrote:
>>>> On Mon, Mar 8, 2021 at 5:41 PM Willian Rampazzo <wrampazz@redhat.com> 
>>>> wrote:
>>>>>
>>
>>>>>>>>> -    def do_test_arm_orangepi_uboot_armbian(self, image_path):
>>>>>>>>> +    @skipUnless(os.getenv('ARMBIAN_ARTIFACTS_CACHED'),
>>>>>>>>> +                'Test artifacts fetched from unreliable 
>>>>>>>>> apt.armbian.com')
>>>>>>>>> +    @skipUnless(os.getenv('AVOCADO_ALLOW_LARGE_STORAGE'), 'storage 
>>>>>>>>> limited')
>>>>>>>>> +    def test_arm_orangepi_bionic_20_08(self):
>>>>>>>>> +        """
>>>>>>>>> +        :avocado: tags=arch:arm
>>>>>>>>> +        :avocado: tags=machine:orangepi-pc
>>>>>>>>> +        :avocado: tags=device:sd
>>>>>>>>> +        """
>>>>>>>>> +
>>>>>>>>> +        # This test download a 275 MiB compressed image and expand it
>>>>>>>>> +        # to 1036 MiB, but the underlying filesystem is 1552 MiB...
>>>>>>>>> +        # As we expand it to 2 GiB we are safe.
>>>>>>>>> +
>>>>>>>>> +        image_url = ('https://dl.armbian.com/orangepipc/archive/'
>>>>>>>>> +                     
>>>>>>>>> 'Armbian_20.08.1_Orangepipc_bionic_current_5.8.5.img.xz')
>>>>>>>>
>>>>>>>> The Armbian 20 is not available on this link anymore. I can browse 
>>>>>>>> just 21.
>>>>>>>
>>>>>>> Cat-and-mouse game *sigh*.
>>>
>>> 2021-03-22 17:18:10,701 download         L0067 INFO | Fetching
>>> https://archive.armbian.com/orangepipc/archive/Armbian_20.08.1_Orangepipc_bionic_current_5.8.5.img.xz
>>> -> /home/phil/avocado/da
>>> ta/cache/by_location/f2eb27a12b81ce15e93f340fabbced2136af1caa/Armbian_20.08.1_Orangepipc_bionic_current_5.8.5.img.xz.nfwdzivg
>>> 2021-03-22 17:18:11,263 asset            L0136 INFO | Temporary asset
>>> file unavailable due to failed download attempt.
>>> 2021-03-22 17:18:11,263 asset            L0368 ERROR| URLError: <urlopen
>>> error [Errno 113] No route to host>
>>>
>>> It might be a temporary problem, but looking long term the
>>> current setup doesn't scale IMHO.
>>
>> Also I just noticed I *do* have the image cached, even twice:
>>
>> $ find /home/phil/avocado/data/cache/by_location/ -name
>> Armbian_20.08.1_Orangepipc_bionic_current_5.8.5.img.xz -ls
>>   131661 280768 -rw-rw-r--   1 phil     phil     287501560 Oct 23
>> 12:45 
>> /home/phil/avocado/data/cache/by_location/cac379e6a4480624b000dfde1b5d935454254ce2/Armbian_20.08.1_Orangepipc_bionic_current_5.8.5.img.xz
>>   131761 280776 -rw-rw-r--   1 phil     phil     287501560 Feb 20
>> 00:39 
>> /home/phil/avocado/data/cache/by_location/aac619d9e7ea3726ebc86c159fddfb9d3f6274a5/Armbian_20.08.1_Orangepipc_bionic_current_5.8.5.img.xz
>>
>> But the hash is different.
>>
> 
> You have 2 images because you fetched them from different locations.
> When you fetch by location, the assets go to the directory
> representing the hash of the location, This allows, for example, the
> fetch of multiple initrd or vmlinuz files from different locations,
> representing different files, but with the same name. Also, this was
> one of your requests some time ago :)

My request because we were fetching different artifacts for different
tests but all files had the same filename, i.e.:

url1://path1/to/vmlinux
url2://path2/to/vmlinux
url3://yet/another/path/to/vmlinux

I was obviously short sighted.

> What Beraldo worked on in the last week was a way to remove "unused"
> files. Your cache is a good example of files that are not necessary
> anymore because now you are using a different location to fetch them.

OK, I'll watch closely how you decide a cached file is "unused"...

> 
>> Cleber, why not use the blob hash as a cache key? It is supposed to be
>> constant...
>>
> 
> You can do that by setting the test to download the file by name, and
> not by location.

How so? Isn't it what I just described earlier we wanted to avoid?

> 
>> Cc'ing Beraldo because I guess remember he worked on improving Avocado cache.
>>
>>>>>> Just to clarify here: in this patch I wanted to only make the change to 
>>>>>> remove the test for the 19.11.3 image.
>>>>>> And in the 3rd patch (tests/acceptance/boot_linux_console: change URL 
>>>>>> for test_arm_orangepi_bionic_20_08)
>>>>>> do the update for to correct the 20.08 link. So each patch fixes one 
>>>>>> problem at a time.
>>
> 
>