Re: [PATCH] docker: move tests from python2 to python3

[John Snow]

On 9/23/19 10:50 AM, Cleber Rosa wrote:
> On Fri, Sep 20, 2019 at 04:00:49PM -0400, John Snow wrote:
>> As part of the push to drop python2 support, replace any explicit python2
>> dependencies with python3 versions.
>>
>> For centos, python2 still exists as an implicit dependency, but by adding
>> python3 we will be able to build even if the configure script begins to
>> require python 3.5+.
>>
>> Tested with centos7, fedora, ubuntu, ubuntu1804, and debian 9 (amd64).
>> Tested under a custom configure script that requires Python 3.5+.
>>
>> Signed-off-by: John Snow <address@hidden>
>> ---
>>  tests/docker/dockerfiles/centos7.docker             | 2 +-
>>  tests/docker/dockerfiles/debian-ports.docker        | 2 +-
>>  tests/docker/dockerfiles/debian-sid.docker          | 2 +-
>>  tests/docker/dockerfiles/debian-xtensa-cross.docker | 2 +-
>>  tests/docker/dockerfiles/debian10.docker            | 2 +-
>>  tests/docker/dockerfiles/debian8.docker             | 3 +--
>>  tests/docker/dockerfiles/debian9.docker             | 2 +-
>>  tests/docker/dockerfiles/ubuntu.docker              | 2 +-
>>  tests/docker/dockerfiles/ubuntu1804.docker          | 2 +-
>>  9 files changed, 9 insertions(+), 10 deletions(-)
>>
>> diff --git a/tests/docker/dockerfiles/centos7.docker 
>> b/tests/docker/dockerfiles/centos7.docker
>> index e0b9d7dbe9..953637065c 100644
>> --- a/tests/docker/dockerfiles/centos7.docker
>> +++ b/tests/docker/dockerfiles/centos7.docker
>> @@ -25,6 +25,7 @@ ENV PACKAGES \
>>      nettle-devel \
>>      perl-Test-Harness \
>>      pixman-devel \
>> +    python3 \
> 
> Good, this gives us Python 3.6 (currently seeing package version
> 3.6.8-10.el7).
> 
>>      SDL-devel \
>>      spice-glib-devel \
>>      spice-server-devel \
>> @@ -34,4 +35,3 @@ ENV PACKAGES \
>>      zlib-devel
>>  RUN yum install -y $PACKAGES
>>  RUN rpm -q $PACKAGES | sort > /packages.txt
>> -
>> diff --git a/tests/docker/dockerfiles/debian-ports.docker 
>> b/tests/docker/dockerfiles/debian-ports.docker
>> index 61bc3f2993..f1ea8d3bd1 100644
>> --- a/tests/docker/dockerfiles/debian-ports.docker
>> +++ b/tests/docker/dockerfiles/debian-ports.docker
>> @@ -31,6 +31,6 @@ RUN apt-get update && \
>>          git \
>>          pkg-config \
>>          psmisc \
>> -        python \
>> +        python3 \
> 
> On my testing, Python 3.7.4-4 gets installed here at this time.
> 
> I wonder if this "unstable" distro can/will give us Python 3.8 in the
> near future and the consequences of that...  assuming that's a feature
> for now.
> 

I think it's effectively the point of using debian-sid, I'm not sure how
debian-ports differs.

I've encouraged Philippe to submit a testable target that uses the
debian-sid base so we can keep the partial image -- otherwise I want to
delete it as we can't build against it currently with the tools as they
exist in-tree.

>>          texinfo \
>>          $(apt-get -s build-dep qemu | egrep ^Inst | fgrep '[all]' | cut -d\ 
>>  -f2)
>> diff --git a/tests/docker/dockerfiles/debian-sid.docker 
>> b/tests/docker/dockerfiles/debian-sid.docker
>> index 2a1bcc33b2..dcb8e83715 100644
>> --- a/tests/docker/dockerfiles/debian-sid.docker
>> +++ b/tests/docker/dockerfiles/debian-sid.docker
>> @@ -31,5 +31,5 @@ RUN apt update && \
>>          git \
>>          pkg-config \
>>          psmisc \
>> -        python \
>> +        python3 \
> 
> Python 3.7.4-2 gets installed here.
> 
>>          texinfo || { echo "Failed to build - see debian-sid.docker notes"; 
>> exit 1; }
>> diff --git a/tests/docker/dockerfiles/debian-xtensa-cross.docker 
>> b/tests/docker/dockerfiles/debian-xtensa-cross.docker
>> index b9c2e2e531..e6f93f65ee 100644
>> --- a/tests/docker/dockerfiles/debian-xtensa-cross.docker
>> +++ b/tests/docker/dockerfiles/debian-xtensa-cross.docker
>> @@ -18,7 +18,7 @@ RUN apt-get update && \
>>          flex \
>>          gettext \
>>          git \
>> -        python-minimal
>> +        python3-minimal
> 
> I'm getting Python 3.5.3-1+deb9u1 here, so it LGTM.
> 

Oh, that's actually a bit of a problem. I tested on 3.5+, but I think
some people want 3.6+.

I don't know much about Debian, but we either need to guarantee 3.6+ or
backtrack our plans to 3.5+.

>>  
>>  ENV CPU_LIST csp dc232b dc233c
>>  ENV TOOLCHAIN_RELEASE 2018.02
>> diff --git a/tests/docker/dockerfiles/debian10.docker 
>> b/tests/docker/dockerfiles/debian10.docker
>> index 30a78813f2..dad498b52e 100644
>> --- a/tests/docker/dockerfiles/debian10.docker
>> +++ b/tests/docker/dockerfiles/debian10.docker
>> @@ -26,7 +26,7 @@ RUN apt update && \
>>          git \
>>          pkg-config \
>>          psmisc \
>> -        python \
>> +        python3 \
> 
> Python 3.7.3-2 gets installed here on my testing.
> 
>>          python3-sphinx \
>>          texinfo \
>>          $(apt-get -s build-dep qemu | egrep ^Inst | fgrep '[all]' | cut -d\ 
>>  -f2)
>> diff --git a/tests/docker/dockerfiles/debian8.docker 
>> b/tests/docker/dockerfiles/debian8.docker
>> index 1212a85c35..be67023826 100644
>> --- a/tests/docker/dockerfiles/debian8.docker
>> +++ b/tests/docker/dockerfiles/debian8.docker
>> @@ -30,5 +30,4 @@ RUN apt update && \
>>          git \
>>          gnupg \
>>          pkg-config \
>> -        python-minimal
>> -
>> +        python3-minimal
> 
> This results in the Python package version 3.4.2-1+deb8u7 being
> installed, so it's *NOT* enough for the 3.5+ requirements.
> 
> I suppose it'd be time to retire those distros instead of updating
> them?
> 

Yup, we're dropping it, as you've seen in the other series.

>> diff --git a/tests/docker/dockerfiles/debian9.docker 
>> b/tests/docker/dockerfiles/debian9.docker
>> index b36f1d4ed8..8cbd742bb5 100644
>> --- a/tests/docker/dockerfiles/debian9.docker
>> +++ b/tests/docker/dockerfiles/debian9.docker
>> @@ -26,7 +26,7 @@ RUN apt update && \
>>          git \
>>          pkg-config \
>>          psmisc \
>> -        python \
>> +        python3 \
> 
> Python 3.5.3-1+deb9u1 here, meeting the requirement.
> 

Tch, okay.

>>          python3-sphinx \
>>          texinfo \
>>          $(apt-get -s build-dep qemu | egrep ^Inst | fgrep '[all]' | cut -d\ 
>>  -f2)
>> diff --git a/tests/docker/dockerfiles/ubuntu.docker 
>> b/tests/docker/dockerfiles/ubuntu.docker
>> index a4f601395c..f486492224 100644
>> --- a/tests/docker/dockerfiles/ubuntu.docker
>> +++ b/tests/docker/dockerfiles/ubuntu.docker
>> @@ -60,7 +60,7 @@ ENV PACKAGES flex bison \
>>      libvte-2.91-dev \
>>      libxen-dev \
>>      make \
>> -    python-yaml \
>> +    python3-yaml \
> 
> Python 3.7.3-2ubuntu0.1 gets installed here, meeting the requirement.
> 
>>      python3-sphinx \
>>      sparse \
>>      texinfo \
>> diff --git a/tests/docker/dockerfiles/ubuntu1804.docker 
>> b/tests/docker/dockerfiles/ubuntu1804.docker
>> index 883f9bcf31..3cc4f492c4 100644
>> --- a/tests/docker/dockerfiles/ubuntu1804.docker
>> +++ b/tests/docker/dockerfiles/ubuntu1804.docker
>> @@ -47,7 +47,7 @@ ENV PACKAGES flex bison \
>>      libvte-2.91-dev \
>>      libxen-dev \
>>      make \
>> -    python-yaml \
>> +    python3-yaml \
> 
> Not related to this patch, but the lack of an explicit python3 package
> line gets to me. :)
> 
> Anyway, this ends up pulling Python 3.6.7-1~18.04, so LGTM.
> 
>>      python3-sphinx \
>>      sparse \
>>      texinfo \
>> -- 
>> 2.21.0
>>
>>
> 
> The only issue I see here is what action to take with regards to
> debian8.docker.
> 
> - Cleber.
> 

Thanks for the review!