Re: [PULL 00/41] testing updates (vm, gitlab, misc build fixes)

[Alex Bennée]

Peter Maydell <peter.maydell@linaro.org> writes:

> On Tue, 7 Jul 2020 at 08:09, Alex Bennée <alex.bennee@linaro.org> wrote:
>>
>> There will be some docker failures until the official repository has
>> seeded but local builds should continue to work.
>>
>> ----
>>
>> The following changes since commit eb6490f544388dd24c0d054a96dd304bc7284450:
>>
>>   Merge remote-tracking branch 
>> 'remotes/pmaydell/tags/pull-target-arm-20200703' into staging (2020-07-04 
>> 16:08:41 +0100)
>>
>> are available in the Git repository at:
>>
>>   https://github.com/stsquad/qemu.git tags/pull-testing-and-misc-070720-1
>>
>> for you to fetch changes up to 6a726e8ca0286e3ed69945abd447099f6f6a903c:
>>
>>   tests/qht-bench: Adjust threshold computation (2020-07-07 07:57:41 +0100)
>>
>> ----------------------------------------------------------------
>> Testing and build updates:
>>
>>   - tests/vm support for aarch64 VMs
>>   - tests/tcg better cross-compiler detection
>>   - update docker tooling to support registries
>>   - gitlab build docker images and store in registry
>>   - gitlab use docker images for builds
>>   - a number of skipIf updates to support move
>>   - linux-user MAP_FIXED_NOREPLACE fix
>>   - qht-bench compiler tweaks
>>   - configure fix for secret keyring
>>   - tsan fiber annotation clean-up
>
> freebsd failed:
>
<snip>
>
> con recv:  set console=comconso
>
> Failed to prepare guest environment
> Traceback (most recent call last):
>   File "/home/peter.maydell/qemu-freebsd/tests/vm/basevm.py", line 628, in 
> main
>     return vm.build_image(args.image)
>   File "/home/peter.maydell/qemu-freebsd/tests/vm/freebsd", line 163,
> in build_image
>     self.console_boot_serial()
>   File "/home/peter.maydell/qemu-freebsd/tests/vm/freebsd", line 76,
> in console_boot_serial
>     self.console_wait_send("OK", "boot\n")
>   File "/home/peter.maydell/qemu-freebsd/tests/vm/basevm.py", line
> 400, in console_wait_send
>     self.console_wait(wait)
>   File "/home/peter.maydell/qemu-freebsd/tests/vm/basevm.py", line
> 340, in console_wait
>     chars = vm.console_socket.recv(1)
>   File 
> "/home/peter.maydell/qemu-freebsd/tests/vm/../../python/qemu/console_socket.py",
> line 96, in recv
>     raise socket.timeout
> socket.timeout

Hmm works for me, but I'm guessing you have V=1?

> NetBSD failed:
> con recv: postfix: rebuilding /etc/mail/aliases (missing /etc/mail/aliases.db)
> con recv: Starting inetd.
> con recv: Starting cron.
> con recv: Thu Jul  9 10:40:07 UTC 2020
> con recv: NetBSD/amd64 (localhost) (constty)
> con recv: login:
> con send: qemu<enter>
> con recv:  Jul  9 10:40:09 localhost getty[756]: /dev/ttyE2: Device
> not configured
> con recv: Jul  9 10:40:09 localhost getty[703]: /dev/ttyE3: Device not
> configured
> con recv: Jul  9 10:40:09 localhost getty[753]: /dev/ttyE1: Device not
> configured
> con recv: qemu
> con recv: Password:
> con send: qemupass<enter>
> con recv: Login incorrect or refused on this terminal.
> console: *** read timeout ***
> console: waiting for: 'localhost$'
> console: line buffer:
>
> con recv: login:
>
> Failed to prepare guest environment
> Traceback (most recent call last):
>   File "/home/peter.maydell/qemu-netbsd/tests/vm/basevm.py", line 628, in main
>     return vm.build_image(args.image)
>   File "/home/peter.maydell/qemu-netbsd/tests/vm/netbsd", line 174, in
> build_image
>     self._config["guest_pass"])
>   File "/home/peter.maydell/qemu-netbsd/tests/vm/basevm.py", line 408,
> in console_ssh_init
>     self.console_wait_send(prompt,      "mkdir .ssh\n")
>   File "/home/peter.maydell/qemu-netbsd/tests/vm/basevm.py", line 400,
> in console_wait_send
>     self.console_wait(wait)
>   File "/home/peter.maydell/qemu-netbsd/tests/vm/basevm.py", line 340,
> in console_wait
>     chars = vm.console_socket.recv(1)
>   File 
> "/home/peter.maydell/qemu-netbsd/tests/vm/../../python/qemu/console_socket.py",
> line 96, in recv
>     raise socket.timeout
> socket.timeout

OK that I can replicate. So it looks like our console bypass has an
issue. I'll dig into it.

> arm-linux-static check-tcg failed:
>   TEST    pauth-4 on aarch64
>   TEST    semihosting on aarch64
>   SKIPPED semiconsole on aarch64 because MANUAL ONLY
>   TEST    basic gdbstub support
> /tmp/tmp6ftb0v61qemu-gdbstub/gdbstub.socket: No such device or address.
> SKIPPING (not connected)
>   BUILD   TCG tests for aarch64_be-linux-user
>   BUILD   aarch64_be-linux-user guest-tests SKIPPED
>   RUN     TCG tests for aarch64_be-linux-user
>   RUN     tests for aarch64_be-linux-user SKIPPED
> make: Entering directory
> '/home/petmay01/linaro/qemu-for-merges/build/all-linux-static'
>   BUILD   debian10
> Error response from daemon: manifest for
> registry.gitlab.com/qemu-project/qemu/qemu/debian10:latest not found
> make: Leaving directory
> '/home/petmay01/linaro/qemu-for-merges/build/all-linux-static'
> make: Entering directory
> '/home/petmay01/linaro/qemu-for-merges/build/all-linux-static'
>   BUILD   debian-alpha-cross
> Error response from daemon: manifest for
> registry.gitlab.com/qemu-project/qemu/qemu/debian-alpha-cross:latest
> not found
> registry.gitlab.com/qemu-project/qemu/qemu/debian10:latest not found
> Traceback (most recent call last):
>   File "/home/petmay01/linaro/qemu-for-merges/tests/docker/docker.py",
> line 701, in <module>
>     sys.exit(main())
>   File "/home/petmay01/linaro/qemu-for-merges/tests/docker/docker.py",
> line 697, in main
>     return args.cmdobj.run(args, argv)
>   File "/home/petmay01/linaro/qemu-for-merges/tests/docker/docker.py",
> line 494, in run
>     extra_files_cksum=cksum)
>   File "/home/petmay01/linaro/qemu-for-merges/tests/docker/docker.py",
> line 347, in build_image
>     quiet=quiet)
>   File "/home/petmay01/linaro/qemu-for-merges/tests/docker/docker.py",
> line 244, in _do_check
>     return subprocess.check_call(self._command + cmd, **kwargs)
>   File "/usr/lib/python3.6/subprocess.py", line 311, in check_call
>     raise CalledProcessError(retcode, cmd)
> subprocess.CalledProcessError: Command '['docker', 'build', '-t',
> 'qemu/debian-alpha-cross', '-f',
> '/tmp/docker_buildsstz1jcy/tmp_7c70xjh.docker', '--build-arg',
> 'BUILDKIT_INLINE_CACHE=1', '--cache-from',
> 'registry.gitlab.com/qemu-project/qemu/qemu/debian-alpha-cross',
> '/tmp/docker_buildsstz1jcy']' returned non-zero exit status 1.
> /home/petmay01/linaro/qemu-for-merges/tests/docker/Makefile.include:58:
> recipe for target 'docker-image-debian-alpha-cross' failed
> make: *** [docker-image-debian-alpha-cross] Error 1

This is it tripping up on missing registry (as I switched the registry
from my previous patch posts to point to the "official" gitlab). You can
work around this for now by passing:

  make check-tcg REGISTRY=registry.gitlab.com/stsquad/qemu

I also just posted:

  Subject: [PATCH  v1 11/13] tests/docker: fall back more gracefully when pull 
fails
  Date: Thu,  9 Jul 2020 15:13:25 +0100
  Message-Id: <20200709141327.14631-12-alex.bennee@linaro.org>

Maybe I should just squash it into:

  tests/docker: add --registry support to tooling

?

> Also a compile failure on s390x, but since this isn't related
> to changes you made afaict I wonder if it's the result of
> a change in the build environment:
> /home/ubuntu/qemu/block/ssh.c: In function ‘check_host_key_knownhosts’:
> /home/ubuntu/qemu/block/ssh.c:281:28: error: storage size of ‘state’ isn’t 
> known
>      enum ssh_known_hosts_e state;
>                             ^~~~~
> /home/ubuntu/qemu/block/ssh.c:289:13: error: implicit declaration of
> function ‘ssh_session_is_known_server’ [-Werror=implicit-funct
> ion-declaration]
>      state = ssh_session_is_known_server(s->session);
>              ^~~~~~~~~~~~~~~~~~~~~~~~~~~
> [and other errors]

As Philippe said I don't think this was me.

-- 
Alex Bennée