qemu-commits
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[Qemu-commits] [qemu/qemu] 6d592c: target/ppc: Fix TCG leak with the evm


From: Peter Maydell
Subject: [Qemu-commits] [qemu/qemu] 6d592c: target/ppc: Fix TCG leak with the evmwsmiaa instru...
Date: Mon, 24 Aug 2020 04:15:32 -0700

  Branch: refs/heads/master
  Home:   https://github.com/qemu/qemu
  Commit: 6d592c557e2d7131585ea7f6a3f214aba71e8776
      
https://github.com/qemu/qemu/commit/6d592c557e2d7131585ea7f6a3f214aba71e8776
  Author: Matthieu Bucchianeri <matthieu.bucchianeri@leostella.com>
  Date:   2020-08-12 (Wed, 12 Aug 2020)

  Changed paths:
    M target/ppc/translate/spe-impl.inc.c

  Log Message:
  -----------
  target/ppc: Fix TCG leak with the evmwsmiaa instruction

Fix double-call to tcg_temp_new_i64(), where a temp is allocated both at
declaration time and further down the implementation of gen_evmwsmiaa().

Note that gen_evmwsmia() and gen_evmwsmiaa() are still not implemented
correctly, as they invoke gen_evmwsmi() which may return early, but the
return is not propagated. This will be fixed in my patch for bug #1888918.

Signed-off-by: Matthieu Bucchianeri <matthieu.bucchianeri@leostella.com>
Message-Id: <20200727172114.31415-1-matthieu.bucchianeri@leostella.com>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>


  Commit: ca7a2fdaa19ebbe91598d13fedbced43d47fff99
      
https://github.com/qemu/qemu/commit/ca7a2fdaa19ebbe91598d13fedbced43d47fff99
  Author: Lijun Pan <ljp@linux.ibm.com>
  Date:   2020-08-12 (Wed, 12 Aug 2020)

  Changed paths:
    M target/ppc/cpu.h

  Log Message:
  -----------
  target/ppc: Introduce Power ISA 3.1 flag

This flag will be used for Power10 instructions.

Signed-off-by: Lijun Pan <ljp@linux.ibm.com>
Message-Id: <20200701234344.91843-2-ljp@linux.ibm.com>
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>


  Commit: 9495edb08d9d7cd5674a237d95dbabfa7b355340
      
https://github.com/qemu/qemu/commit/9495edb08d9d7cd5674a237d95dbabfa7b355340
  Author: Lijun Pan <ljp@linux.ibm.com>
  Date:   2020-08-12 (Wed, 12 Aug 2020)

  Changed paths:
    M target/ppc/cpu.h
    M target/ppc/translate_init.inc.c

  Log Message:
  -----------
  target/ppc: Enable Power ISA 3.1

This patch enables the Power ISA 3.1 in QEMU.

Signed-off-by: Lijun Pan <ljp@linux.ibm.com>
Message-Id: <20200701234344.91843-3-ljp@linux.ibm.com>
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>


  Commit: 9d69cfa2faa7782ec91b9e42de3abb0a442afca8
      
https://github.com/qemu/qemu/commit/9d69cfa2faa7782ec91b9e42de3abb0a442afca8
  Author: Lijun Pan <ljp@linux.ibm.com>
  Date:   2020-08-12 (Wed, 12 Aug 2020)

  Changed paths:
    M target/ppc/translate.c

  Log Message:
  -----------
  target/ppc: add byte-reverse br[dwh] instructions

POWER ISA 3.1 introduces following byte-reverse instructions:
brd: Byte-Reverse Doubleword X-form
brw: Byte-Reverse Word X-form
brh: Byte-Reverse Halfword X-form

Signed-off-by: Lijun Pan <ljp@linux.ibm.com>
Message-Id: <20200701234344.91843-4-ljp@linux.ibm.com>
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>


  Commit: a285ffa680d14cdd7a88e3d551024e6f66684cba
      
https://github.com/qemu/qemu/commit/a285ffa680d14cdd7a88e3d551024e6f66684cba
  Author: Lijun Pan <ljp@linux.ibm.com>
  Date:   2020-08-12 (Wed, 12 Aug 2020)

  Changed paths:
    M target/ppc/helper.h
    M target/ppc/int_helper.c
    M target/ppc/translate/vmx-impl.inc.c

  Log Message:
  -----------
  target/ppc: convert vmuluwm to tcg_gen_gvec_mul

Convert the original implementation of vmuluwm to the more generic
tcg_gen_gvec_mul.

Signed-off-by: Lijun Pan <ljp@linux.ibm.com>
Message-Id: <20200701234344.91843-5-ljp@linux.ibm.com>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>


  Commit: adcced87842d01229993605321761a41869b9128
      
https://github.com/qemu/qemu/commit/adcced87842d01229993605321761a41869b9128
  Author: Lijun Pan <ljp@linux.ibm.com>
  Date:   2020-08-12 (Wed, 12 Aug 2020)

  Changed paths:
    M target/ppc/translate/vmx-impl.inc.c
    M target/ppc/translate/vmx-ops.inc.c

  Log Message:
  -----------
  target/ppc: add vmulld instruction

vmulld: Vector Multiply Low Doubleword.

Signed-off-by: Lijun Pan <ljp@linux.ibm.com>
Message-Id: <20200701234344.91843-6-ljp@linux.ibm.com>
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>


  Commit: 7abf97975088595682f52c7cfb030b45ea38d8c3
      
https://github.com/qemu/qemu/commit/7abf97975088595682f52c7cfb030b45ea38d8c3
  Author: Anton Blanchard <anton@ozlabs.org>
  Date:   2020-08-12 (Wed, 12 Aug 2020)

  Changed paths:
    M hw/ppc/spapr.c

  Log Message:
  -----------
  ppc/spapr: Fix 32 bit logical memory block size assumptions

When testing large LMB sizes (eg 4GB), I found a couple of places
that assume they are 32bit in size.

Signed-off-by: Anton Blanchard <anton@ozlabs.org>
Message-Id: <20200715004228.1262681-1-anton@ozlabs.org>
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>


  Commit: d9c5b5fa86ce37125dcc64c6274a1bc48cf11903
      
https://github.com/qemu/qemu/commit/d9c5b5fa86ce37125dcc64c6274a1bc48cf11903
  Author: Greg Kurz <groug@kaod.org>
  Date:   2020-08-12 (Wed, 12 Aug 2020)

  Changed paths:
    M hw/ppc/spapr_caps.c

  Log Message:
  -----------
  spapr: Use error_append_hint() in spapr_caps.c

We have a dedicated error API for hints. Use it instead of embedding
the hint in the error message, as recommanded in the "qapi/error.h"
header file.

While here, have cap_fwnmi_apply(), which already uses
error_append_hint(), to call ERRP_GUARD() as well.

Signed-off-by: Greg Kurz <groug@kaod.org>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Laurent Vivier <lvivier@redhat.com>
Message-Id: <159594297421.8262.14314530897345809924.stgit@bahia.lan>
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>


  Commit: 19d55e2031f2473a65ffc11aff5b1059e7c4173b
      
https://github.com/qemu/qemu/commit/19d55e2031f2473a65ffc11aff5b1059e7c4173b
  Author: Greg Kurz <groug@kaod.org>
  Date:   2020-08-12 (Wed, 12 Aug 2020)

  Changed paths:
    M hw/ppc/spapr_caps.c

  Log Message:
  -----------
  spapr: Forbid nested KVM-HV in pre-power9 compat mode

Nested KVM HV only works if the kernel is using the radix MMU mode, ie.
the CPU is POWER9 and it is not running in some pre-power9 compat mode.
Otherwise, the KVM HV module fails to load in the guest with -ENODEV.
It might be painful for a user to discover this late that nested cannot
work with their setup. Erroring out at machine init instead seems to be
the best we can do.

Signed-off-by: Greg Kurz <groug@kaod.org>
Reviewed-by: Laurent Vivier <lvivier@redhat.com>
Message-Id: <159491948127.188975.9621435875869177751.stgit@bahia.lan>
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>


  Commit: 4b160fad4f6179fa7f8385f87f124cfce1f809d3
      
https://github.com/qemu/qemu/commit/4b160fad4f6179fa7f8385f87f124cfce1f809d3
  Author: Gustavo Romero <gromero@linux.ibm.com>
  Date:   2020-08-12 (Wed, 12 Aug 2020)

  Changed paths:
    M hw/intc/xive.c

  Log Message:
  -----------
  ppc/xive: Fix some typos in comments

Fix some typos in comments about code modeling coalescing points in the
XIVE routing engine (IVRE).

Signed-off-by: Gustavo Romero <gromero@linux.ibm.com>
Message-Id: <1595461434-27725-1-git-send-email-gromero@linux.ibm.com>
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>


  Commit: de55d3b3815430c27b2e8fe36f85d8e3f2026c95
      
https://github.com/qemu/qemu/commit/de55d3b3815430c27b2e8fe36f85d8e3f2026c95
  Author: Lijun Pan <ljp@linux.ibm.com>
  Date:   2020-08-12 (Wed, 12 Aug 2020)

  Changed paths:
    M include/elf.h

  Log Message:
  -----------
  Update PowerPC AT_HWCAP2 definition

Add PPC2_FEATURE2_ARCH_3_10 to the PowerPC AT_HWCAP2 definitions.

Signed-off-by: Lijun Pan <ljp@linux.ibm.com>
Message-Id: <20200724045845.89976-2-ljp@linux.ibm.com>
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>


  Commit: 73ebe95e8e593567dce9477cb4adf86560c7d377
      
https://github.com/qemu/qemu/commit/73ebe95e8e593567dce9477cb4adf86560c7d377
  Author: Lijun Pan <ljp@linux.ibm.com>
  Date:   2020-08-12 (Wed, 12 Aug 2020)

  Changed paths:
    M tcg/ppc/tcg-target.h
    M tcg/ppc/tcg-target.inc.c

  Log Message:
  -----------
  target/ppc: add vmulld to INDEX_op_mul_vec case

Group vmuluwm and vmulld. Make vmulld-specific
changes since it belongs to new ISA 3.1.

Signed-off-by: Lijun Pan <ljp@linux.ibm.com>
Message-Id: <20200724045845.89976-3-ljp@linux.ibm.com>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>


  Commit: f3e0d864abb4de0a324cf7e77cb242b57ccb45c3
      
https://github.com/qemu/qemu/commit/f3e0d864abb4de0a324cf7e77cb242b57ccb45c3
  Author: Lijun Pan <ljp@linux.ibm.com>
  Date:   2020-08-12 (Wed, 12 Aug 2020)

  Changed paths:
    M target/ppc/helper.h
    M target/ppc/int_helper.c
    M target/ppc/translate/vmx-impl.inc.c
    M target/ppc/translate/vmx-ops.inc.c

  Log Message:
  -----------
  target/ppc: add vmulh{su}w instructions

vmulhsw: Vector Multiply High Signed Word
vmulhuw: Vector Multiply High Unsigned Word

Signed-off-by: Lijun Pan <ljp@linux.ibm.com>
Message-Id: <20200724045845.89976-4-ljp@linux.ibm.com>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>


  Commit: c4b8b49d68856ffacacfd792b0ab0d4aa0982e8d
      
https://github.com/qemu/qemu/commit/c4b8b49d68856ffacacfd792b0ab0d4aa0982e8d
  Author: Lijun Pan <ljp@linux.ibm.com>
  Date:   2020-08-12 (Wed, 12 Aug 2020)

  Changed paths:
    M target/ppc/helper.h
    M target/ppc/int_helper.c
    M target/ppc/translate/vmx-impl.inc.c
    M target/ppc/translate/vmx-ops.inc.c

  Log Message:
  -----------
  target/ppc: add vmulh{su}d instructions

vmulhsd: Vector Multiply High Signed Doubleword
vmulhud: Vector Multiply High Unsigned Doubleword

Signed-off-by: Lijun Pan <ljp@linux.ibm.com>
Message-Id: <20200724045845.89976-5-ljp@linux.ibm.com>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>


  Commit: 8dcdb535d7cc4ba6270bb756e12e1d323254ed4e
      
https://github.com/qemu/qemu/commit/8dcdb535d7cc4ba6270bb756e12e1d323254ed4e
  Author: Matthieu Bucchianeri <matthieu.bucchianeri@leostella.com>
  Date:   2020-08-12 (Wed, 12 Aug 2020)

  Changed paths:
    M target/ppc/translate/spe-impl.inc.c

  Log Message:
  -----------
  target/ppc: Fix SPE unavailable exception triggering

When emulating certain floating point instructions or vector instructions on
PowerPC machines, QEMU did not properly generate the SPE/Embedded Floating-
Point Unavailable interrupt. See the buglink further below for references to
the relevant NXP documentation.

This patch fixes the behavior of some evfs* instructions that were
incorrectly emitting the interrupt.

More importantly, this patch fixes the behavior of several efd* and ev*
instructions that were not generating the interrupt. Triggering the
interrupt for these instructions fixes lazy FPU/vector context switching on
some operating systems like Linux.

Without this patch, the result of some double-precision arithmetic could be
corrupted due to the lack of proper saving and restoring of the upper
32-bit part of the general-purpose registers.

Buglink: https://bugs.launchpad.net/qemu/+bug/1888918
Buglink: https://bugs.launchpad.net/qemu/+bug/1611394
Signed-off-by: Matthieu Bucchianeri <matthieu.bucchianeri@leostella.com>
Message-Id: <20200727175553.32276-1-matthieu.bucchianeri@leostella.com>
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>


  Commit: 61f5e1a34daebc0d92073bfc8159bee4855a1397
      
https://github.com/qemu/qemu/commit/61f5e1a34daebc0d92073bfc8159bee4855a1397
  Author: Daniel Henrique Barboza <danielhb413@gmail.com>
  Date:   2020-08-12 (Wed, 12 Aug 2020)

  Changed paths:
    M docs/specs/index.rst
    A docs/specs/ppc-spapr-numa.rst

  Log Message:
  -----------
  docs: adding NUMA documentation for pseries

This patch adds a new documentation file, ppc-spapr-numa.rst,
informing what developers and user can expect of the NUMA distance
support for the pseries machine, up to QEMU 5.1.

In the (hopefully soon) future, when we rework the NUMA mechanics
of the pseries machine to at least attempt to contemplate user
choice, this doc will be extended to inform about the new
support.

Signed-off-by: Daniel Henrique Barboza <danielhb413@gmail.com>
Message-Id: <20200803133440.825276-1-danielhb413@gmail.com>
Tested-by: Greg Kurz <groug@kaod.org>
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>


  Commit: 8d14523b1c98e7e64844a997d07dffcc32c83bef
      
https://github.com/qemu/qemu/commit/8d14523b1c98e7e64844a997d07dffcc32c83bef
  Author: Cédric Le Goater <clg@kaod.org>
  Date:   2020-08-12 (Wed, 12 Aug 2020)

  Changed paths:
    M docs/specs/ppc-spapr-xive.rst

  Log Message:
  -----------
  docs: Update POWER9 XIVE support for nested guests

It is not yet supported.

Signed-off-by: Cédric Le Goater <clg@kaod.org>
Message-Id: <20200804131639.407049-1-clg@kaod.org>
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>


  Commit: c55bcb1f47071a134a4b96b4137cccca831ac5cf
      
https://github.com/qemu/qemu/commit/c55bcb1f47071a134a4b96b4137cccca831ac5cf
  Author: Greg Kurz <groug@kaod.org>
  Date:   2020-08-12 (Wed, 12 Aug 2020)

  Changed paths:
    M docs/specs/ppc-spapr-xive.rst
    M hw/ppc/spapr_irq.c

  Log Message:
  -----------
  spapr: Clarify error and documentation for broken KVM XICS

When starting an L2 KVM guest with `ic-mode=dual,kernel-irqchip=on`,
QEMU fails with:

KVM is too old to support ic-mode=dual,kernel-irqchip=on

This error message was introduced to detect older KVM versions that
didn't allow destruction and re-creation of the XICS KVM device that
we do at reboot. But it is actually the same issue that we get with
nested guests : when running under pseries, KVM currently provides
a genuine XICS device (not the XICS-on-XIVE device that we get
under powernv) which doesn't support destruction/re-creation.

This will eventually be fixed in KVM but in the meantime, update
the error message and documentation to mention the nested case.
While here, mention that in "No XIVE support in KVM" section that
this can also happen with "guest OSes supporting XIVE" since
we check this at init time before starting the guest.

Reported-by: Satheesh Rajendran <sathnaga@linux.vnet.ibm.com>
Buglink: https://bugs.launchpad.net/qemu/+bug/1890290
Signed-off-by: Greg Kurz <groug@kaod.org>
Message-Id: <159664243614.622889.18307368735989783528.stgit@bahia.lan>
Reviewed-by: Cédric Le Goater <clg@kaod.org>
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>


  Commit: 82f086b5e7ec0adff5a3972f74c446325b4fef9a
      
https://github.com/qemu/qemu/commit/82f086b5e7ec0adff5a3972f74c446325b4fef9a
  Author: Greg Kurz <groug@kaod.org>
  Date:   2020-08-12 (Wed, 12 Aug 2020)

  Changed paths:
    M hw/intc/spapr_xive_kvm.c

  Log Message:
  -----------
  spapr/xive: Fix xive->fd if kvm_create_device() fails

If the creation of the KVM XIVE device fails for some reasons, the
negative errno ends up in xive->fd, but the rest of the code assumes
that xive->fd either contains an open fd, ie. positive value, or -1.

This doesn't cause any misbehavior except kvmppc_xive_disconnect()
that will try to close(xive->fd) during rollback and likely be
rewarded with an EBADF.

Only set xive->fd with a open fd.

Signed-off-by: Greg Kurz <groug@kaod.org>
Message-Id: <159673296585.766512.15404407281299745442.stgit@bahia.lan>
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>


  Commit: e781139539f3d73ac00f2aea17f5a154b10e4302
      
https://github.com/qemu/qemu/commit/e781139539f3d73ac00f2aea17f5a154b10e4302
  Author: Greg Kurz <groug@kaod.org>
  Date:   2020-08-12 (Wed, 12 Aug 2020)

  Changed paths:
    M hw/intc/spapr_xive_kvm.c

  Log Message:
  -----------
  spapr/xive: Simplify kvmppc_xive_disconnect()

Since this function begins with:

    /* The KVM XIVE device is not in use */
    if (!xive || xive->fd == -1) {
        return;
    }

we obviously don't need to check xive->fd again.

Signed-off-by: Greg Kurz <groug@kaod.org>
Message-Id: <159673297296.766512.14780055521619233656.stgit@bahia.lan>
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>


  Commit: a72c71b77d73f9888cbe8da8be09d7097ebca64f
      
https://github.com/qemu/qemu/commit/a72c71b77d73f9888cbe8da8be09d7097ebca64f
  Author: Gustavo Romero <gromero@linux.ibm.com>
  Date:   2020-08-13 (Thu, 13 Aug 2020)

  Changed paths:
    M target/ppc/translate_init.inc.c

  Log Message:
  -----------
  target/ppc: Integrate icount to purr, vtb, and tbu40

Currently if option '-icount auto' is passed to the QEMU TCG to enable
counting instructions the VM crashes with the following error report when
Linux runs on it:

qemu-system-ppc64: Bad icount read

This happens because read/write access to the SPRs PURR, VTB, and TBU40
is not integrated to the icount framework.

This commit fixes that issue by making the read/write access of these
SPRs aware of icount framework, adding the proper gen_io_start() calls
before calling the helpers to load/store these SPRs in TCG and ensuring
that the associated TBs end immediately after, accordingly to what's in
docs/devel/tcg-icount.rst.

Signed-off-by: Gustavo Romero <gromero@linux.ibm.com>
Message-Id: <20200811153235.4527-1-gromero@linux.ibm.com>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>


  Commit: cf36e5b3760df0a1fdd38970294cf7b0968fcc5c
      
https://github.com/qemu/qemu/commit/cf36e5b3760df0a1fdd38970294cf7b0968fcc5c
  Author: Greg Kurz <groug@kaod.org>
  Date:   2020-08-13 (Thu, 13 Aug 2020)

  Changed paths:
    M hw/intc/spapr_xive_kvm.c
    M hw/intc/xive.c
    M include/hw/ppc/xive.h

  Log Message:
  -----------
  ppc/xive: Rework setup of XiveSource::esb_mmio

Depending on whether XIVE is emultated or backed with a KVM XIVE device,
the ESB MMIOs of a XIVE source point to an I/O memory region or a mapped
memory region.

This is currently handled by checking kvm_irqchip_in_kernel() returns
false in xive_source_realize(). This is a bit awkward as we usually
need to do extra things when we're using the in-kernel backend, not
less. But most important, we can do better: turn the existing "xive.esb"
memory region into a plain container, introduce an "xive.esb-emulated"
I/O subregion and rename the existing "xive.esb" subregion in the KVM
code to "xive.esb-kvm". Since "xive.esb-kvm" is added with overlap
and a higher priority, it prevails over "xive.esb-emulated" (ie.
a guest using KVM XIVE will interact with "xive.esb-kvm" instead of
the default "xive.esb-emulated" region.

While here, consolidate the computation of the MMIO region size in
a common helper.

Suggested-by: Cédric Le Goater <clg@kaod.org>
Signed-off-by: Greg Kurz <groug@kaod.org>
Message-Id: <159679992680.876294.7520540158586170894.stgit@bahia.lan>
Reviewed-by: Cédric Le Goater <clg@kaod.org>
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>


  Commit: e519cdd9bc02990bddd395656bdaec821c94c8fe
      
https://github.com/qemu/qemu/commit/e519cdd9bc02990bddd395656bdaec821c94c8fe
  Author: Greg Kurz <groug@kaod.org>
  Date:   2020-08-13 (Thu, 13 Aug 2020)

  Changed paths:
    M hw/intc/spapr_xive.c
    M hw/intc/xive.c
    M include/hw/ppc/xive.h

  Log Message:
  -----------
  ppc/xive: Introduce dedicated kvm_irqchip_in_kernel() wrappers

Calls to the KVM XIVE device are guarded by kvm_irqchip_in_kernel(). This
ensures that QEMU won't try to use the device if KVM is disabled or if
an in-kernel irqchip isn't required.

When using ic-mode=dual with the pseries machine, we have two possible
interrupt controllers: XIVE and XICS. The kvm_irqchip_in_kernel() helper
will return true as soon as any of the KVM device is created. It might
lure QEMU to think that the other one is also around, while it is not.
This is exactly what happens with ic-mode=dual at machine init when
claiming IRQ numbers, which must be done on all possible IRQ backends,
eg. RTAS event sources or the PHB0 LSI table : only the KVM XICS device
is active but we end up calling kvmppc_xive_source_reset_one() anyway,
which fails. This doesn't cause any trouble because of another bug :
kvmppc_xive_source_reset_one() lacks an error_setg() and callers don't
see the failure.

Most of the other kvmppc_xive_* functions have similar xive->fd
checks to filter out the case when KVM XIVE isn't active. It
might look safer to have idempotent functions but it doesn't
really help to understand what's going on when debugging.

Since we already have all the kvm_irqchip_in_kernel() in place,
also have the callers to check xive->fd as well before calling
KVM XIVE specific code. This is straight-forward for the spapr
specific XIVE code. Some more care is needed for the platform
agnostic XIVE code since it cannot access xive->fd directly.
Introduce new in_kernel() methods in some base XIVE classes
for this purpose and implement them only in spapr.

In all cases, we still need to call kvm_irqchip_in_kernel() so that
compilers can optimize the kvmppc_xive_* calls away when CONFIG_KVM
isn't defined, thus avoiding the need for stubs.

Signed-off-by: Greg Kurz <groug@kaod.org>
Message-Id: <159679993438.876294.7285654331498605426.stgit@bahia.lan>
Reviewed-by: Cédric Le Goater <clg@kaod.org>
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>


  Commit: a4907119348d928e8b72e2d3b7566e50f272fa09
      
https://github.com/qemu/qemu/commit/a4907119348d928e8b72e2d3b7566e50f272fa09
  Author: Greg Kurz <groug@kaod.org>
  Date:   2020-08-13 (Thu, 13 Aug 2020)

  Changed paths:
    M hw/intc/spapr_xive_kvm.c

  Log Message:
  -----------
  spapr/xive: Convert KVM device fd checks to assert()

All callers guard these functions with an xive_in_kernel() helper. Make
it clear that they are only to be called when the KVM XIVE device exists.

Note that the check on xive is dropped in kvmppc_xive_disconnect(). It
really cannot be NULL since it comes from set_active_intc() which only
passes pointers to allocated objects.

Signed-off-by: Greg Kurz <groug@kaod.org>
Reviewed-by: David Gibson <david@gibson.dropbear.id.au>
Message-Id: <159679994169.876294.11026653581505077112.stgit@bahia.lan>
Reviewed-by: Cédric Le Goater <clg@kaod.org>
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>


  Commit: 4a6891b838e4914d25ab97a2a03f946e3c085a8f
      
https://github.com/qemu/qemu/commit/4a6891b838e4914d25ab97a2a03f946e3c085a8f
  Author: Greg Kurz <groug@kaod.org>
  Date:   2020-08-13 (Thu, 13 Aug 2020)

  Changed paths:
    M hw/ppc/spapr_pci.c

  Log Message:
  -----------
  spapr: Simplify error handling in spapr_phb_realize()

The spapr_phb_realize() function has a local_err variable which
is used to:

1) check failures of spapr_irq_findone() and spapr_irq_claim()

2) prepend extra information to the error message

Recent work from Markus Armbruster highlighted we get better
code when testing the return value of a function, rather than
setting up all the local_err boiler plate. For similar reasons,
it is now preferred to use ERRP_GUARD() and error_prepend()
rather than error_propagate_prepend().

Since spapr_irq_findone() and spapr_irq_claim() return negative
values in case of failure, do both changes.

This is just cleanup, no functional impact.

Signed-off-by: Greg Kurz <groug@kaod.org>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: David Gibson <david@gibson.dropbear.id.au>
Message-Id: <159707843851.1489912.6108405733810934642.stgit@bahia.lan>
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>


  Commit: 3885ca66881f1d2568e169dcbf793fd493146d14
      
https://github.com/qemu/qemu/commit/3885ca66881f1d2568e169dcbf793fd493146d14
  Author: Greg Kurz <groug@kaod.org>
  Date:   2020-08-13 (Thu, 13 Aug 2020)

  Changed paths:
    M hw/intc/spapr_xive_kvm.c
    M include/hw/ppc/xive.h

  Log Message:
  -----------
  spapr/xive: Rework error handling of kvmppc_xive_cpu_connect()

Use error_setg_errno() instead of error_setg(strerror()). While here,
use -ret instead of errno since kvm_vcpu_enable_cap() returns a negative
errno on failure.

Use ERRP_GUARD() to ensure that errp can be passed to error_append_hint(),
and get rid of the local_err boilerplate.

Propagate the return value so that callers may use it as well to check
failures.

Signed-off-by: Greg Kurz <groug@kaod.org>
Message-Id: <159707844549.1489912.4862921680328017645.stgit@bahia.lan>
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>


  Commit: 46407a2531da4ff206c1aefe8c3f6d8ad53f2de4
      
https://github.com/qemu/qemu/commit/46407a2531da4ff206c1aefe8c3f6d8ad53f2de4
  Author: Greg Kurz <groug@kaod.org>
  Date:   2020-08-13 (Thu, 13 Aug 2020)

  Changed paths:
    M hw/intc/spapr_xive_kvm.c

  Log Message:
  -----------
  spapr/xive: Rework error handling of kvmppc_xive_source_reset()

Since kvmppc_xive_source_reset_one() has a return value, convert
kvmppc_xive_source_reset() to use it for error checking. This
allows to get rid of the local_err boiler plate.

Propagate the return value so that callers may use it as well to check
failures.

Signed-off-by: Greg Kurz <groug@kaod.org>
Message-Id: <159707845245.1489912.9151822670764690034.stgit@bahia.lan>
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>


  Commit: b14adb4a27c80a255fb35451d7cb2bc70743e7f4
      
https://github.com/qemu/qemu/commit/b14adb4a27c80a255fb35451d7cb2bc70743e7f4
  Author: Greg Kurz <groug@kaod.org>
  Date:   2020-08-13 (Thu, 13 Aug 2020)

  Changed paths:
    M hw/intc/spapr_xive_kvm.c

  Log Message:
  -----------
  spapr/xive: Rework error handling of kvmppc_xive_mmap()

Callers currently check failures of kvmppc_xive_mmap() through the
@errp argument, which isn't a recommanded practice. It is preferred
to use a return value when possible.

Since NULL isn't an invalid address in theory, it seems better to
return MAP_FAILED and to teach callers to handle it.

Signed-off-by: Greg Kurz <groug@kaod.org>
Message-Id: <159707845972.1489912.719896767746375765.stgit@bahia.lan>
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>


  Commit: 5fa36b7ffbcb2056249929a7b1ee4e30c07dc67c
      
https://github.com/qemu/qemu/commit/5fa36b7ffbcb2056249929a7b1ee4e30c07dc67c
  Author: Greg Kurz <groug@kaod.org>
  Date:   2020-08-13 (Thu, 13 Aug 2020)

  Changed paths:
    M hw/intc/spapr_xive_kvm.c
    M include/hw/ppc/xive.h

  Log Message:
  -----------
  spapr/xive: Rework error handling of kvmppc_xive_cpu_[gs]et_state()

kvm_set_one_reg() returns a negative errno on failure, use that instead
of errno. Also propagate it to callers so they can use it to check
for failures and hopefully get rid of their local_err boilerplate.

Signed-off-by: Greg Kurz <groug@kaod.org>
Message-Id: <159707846665.1489912.14267225652103441921.stgit@bahia.lan>
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>


  Commit: f9a548edf2a54f59c37032dee3763f532e968fee
      
https://github.com/qemu/qemu/commit/f9a548edf2a54f59c37032dee3763f532e968fee
  Author: Greg Kurz <groug@kaod.org>
  Date:   2020-08-13 (Thu, 13 Aug 2020)

  Changed paths:
    M hw/intc/spapr_xive_kvm.c
    M include/hw/ppc/spapr_xive.h

  Log Message:
  -----------
  spapr/xive: Rework error handling of kvmppc_xive_[gs]et_queue_config()

Since kvm_device_access() returns a negative errno on failure, convert
kvmppc_xive_get_queue_config() and kvmppc_xive_set_queue_config() to
use it for error checking. This allows to get rid of the local_err
boilerplate.

Propagate the return value so that callers may use it as well to check
failures.

Signed-off-by: Greg Kurz <groug@kaod.org>
Message-Id: <159707847357.1489912.2032291280645236480.stgit@bahia.lan>
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>


  Commit: d53482a73bba983f521d6b9652e6f68e856ab794
      
https://github.com/qemu/qemu/commit/d53482a73bba983f521d6b9652e6f68e856ab794
  Author: Greg Kurz <groug@kaod.org>
  Date:   2020-08-13 (Thu, 13 Aug 2020)

  Changed paths:
    M hw/intc/spapr_xive_kvm.c

  Log Message:
  -----------
  spapr/xive: Rework error handling in kvmppc_xive_get_queues()

Since kvmppc_xive_get_queue_config() has a return value, convert
kvmppc_xive_get_queues() to use it for error checking. This allows
to get rid of the local_err boiler plate.

Propagate the return value so that callers may use it as well to check
failures.

Signed-off-by: Greg Kurz <groug@kaod.org>
Message-Id: <159707848069.1489912.14879208798696134531.stgit@bahia.lan>
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>


  Commit: d55daadcb801f309556fdbab00b2653d20e26603
      
https://github.com/qemu/qemu/commit/d55daadcb801f309556fdbab00b2653d20e26603
  Author: Greg Kurz <groug@kaod.org>
  Date:   2020-08-13 (Thu, 13 Aug 2020)

  Changed paths:
    M hw/intc/spapr_xive_kvm.c
    M include/hw/ppc/spapr_xive.h

  Log Message:
  -----------
  spapr/xive: Rework error handling of kvmppc_xive_set_source_config()

Since kvm_device_access() returns a negative errno on failure, convert
kvmppc_xive_set_source_config() to use it for error checking. This allows
to get rid of the local_err boilerplate.

Propagate the return value so that callers may use it as well to check
failures.

Signed-off-by: Greg Kurz <groug@kaod.org>
Message-Id: <159707848764.1489912.17078842252160674523.stgit@bahia.lan>
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>


  Commit: 42a92d925d03ec49dfdefb43c15b46c3ca55f9e4
      
https://github.com/qemu/qemu/commit/42a92d925d03ec49dfdefb43c15b46c3ca55f9e4
  Author: Greg Kurz <groug@kaod.org>
  Date:   2020-08-13 (Thu, 13 Aug 2020)

  Changed paths:
    M hw/intc/spapr_xive_kvm.c

  Log Message:
  -----------
  spapr/kvm: Fix error handling in kvmppc_xive_pre_save()

Now that kvmppc_xive_get_queues() returns a negative errno on failure, check
with that because it is preferred to local_err. And most of all, propagate
it because vmstate expects negative errnos.

Signed-off-by: Greg Kurz <groug@kaod.org>
Message-Id: <159707849455.1489912.6034461176847728064.stgit@bahia.lan>
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>


  Commit: a845a54cbe0f197b833921b2af13fee88bb8240d
      
https://github.com/qemu/qemu/commit/a845a54cbe0f197b833921b2af13fee88bb8240d
  Author: Greg Kurz <groug@kaod.org>
  Date:   2020-08-13 (Thu, 13 Aug 2020)

  Changed paths:
    M hw/intc/spapr_xive_kvm.c

  Log Message:
  -----------
  spapr/xive: Fix error handling in kvmppc_xive_post_load()

Now that all these functions return a negative errno on failure, check
that because it is preferred to local_err. And most of all, propagate it
because vmstate expects negative errnos.

Signed-off-by: Greg Kurz <groug@kaod.org>
Message-Id: <159707850148.1489912.18355118622296682631.stgit@bahia.lan>
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>


  Commit: 2a8100cb61959e7a934049f2bd3a49a0f84a066b
      
https://github.com/qemu/qemu/commit/2a8100cb61959e7a934049f2bd3a49a0f84a066b
  Author: Greg Kurz <groug@kaod.org>
  Date:   2020-08-13 (Thu, 13 Aug 2020)

  Changed paths:
    M hw/intc/xive.c

  Log Message:
  -----------
  ppc/xive: Fix error handling in vmstate_xive_tctx_*() callbacks

Now that kvmppc_xive_cpu_get_state() and kvmppc_xive_cpu_set_state()
return negative errnos on failures, use that instead local_err because
it is the recommended practice. Also return that instead of -1 since
vmstate expects negative errnos.

Signed-off-by: Greg Kurz <groug@kaod.org>
Message-Id: <159707850840.1489912.14912810818646455474.stgit@bahia.lan>
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>


  Commit: 6cdc0e20631faf781047b515215208660393c9a9
      
https://github.com/qemu/qemu/commit/6cdc0e20631faf781047b515215208660393c9a9
  Author: Greg Kurz <groug@kaod.org>
  Date:   2020-08-13 (Thu, 13 Aug 2020)

  Changed paths:
    M hw/intc/spapr_xive_kvm.c

  Log Message:
  -----------
  spapr/xive: Simplify error handling in kvmppc_xive_connect()

Now that all these functions return a negative errno on failure, check
that and get rid of the local_err boilerplate.

Signed-off-by: Greg Kurz <groug@kaod.org>
Message-Id: <159707851537.1489912.1030839306195472651.stgit@bahia.lan>
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>


  Commit: 61203f2b356f8aa7b6cfd3f792cd1f0cc7bdf99b
      
https://github.com/qemu/qemu/commit/61203f2b356f8aa7b6cfd3f792cd1f0cc7bdf99b
  Author: Greg Kurz <groug@kaod.org>
  Date:   2020-08-13 (Thu, 13 Aug 2020)

  Changed paths:
    M hw/intc/xive.c

  Log Message:
  -----------
  ppc/xive: Simplify error handling in xive_tctx_realize()

Now that kvmppc_xive_cpu_connect() returns a negative errno on failure,
use that and get rid of the local_err boilerplate.

Signed-off-by: Greg Kurz <groug@kaod.org>
Message-Id: <159707852234.1489912.16410314514265848075.stgit@bahia.lan>
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>


  Commit: 1118b6b72719ff83f2be1efc11d7248cf225074c
      
https://github.com/qemu/qemu/commit/1118b6b72719ff83f2be1efc11d7248cf225074c
  Author: Greg Kurz <groug@kaod.org>
  Date:   2020-08-13 (Thu, 13 Aug 2020)

  Changed paths:
    M hw/intc/spapr_xive_kvm.c
    M include/hw/ppc/xive.h

  Log Message:
  -----------
  spapr/xive: Simplify error handling of kvmppc_xive_cpu_synchronize_state()

Now that kvmppc_xive_cpu_get_state() returns negative on error, use that
and get rid of the temporary Error object and error_propagate().

Signed-off-by: Greg Kurz <groug@kaod.org>
Message-Id: <159707852916.1489912.8376334685349668124.stgit@bahia.lan>
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>


  Commit: 37035df51eaabb8d26b71da75b88a1c6727de8fa
      
https://github.com/qemu/qemu/commit/37035df51eaabb8d26b71da75b88a1c6727de8fa
  Author: Greg Kurz <groug@kaod.org>
  Date:   2020-08-14 (Fri, 14 Aug 2020)

  Changed paths:
    M hw/nvram/chrp_nvram.c
    M hw/nvram/mac_nvram.c
    M hw/nvram/spapr_nvram.c
    M hw/sparc/sun4m.c
    M hw/sparc64/sun4u.c
    M include/hw/nvram/chrp_nvram.h

  Log Message:
  -----------
  nvram: Exit QEMU if NVRAM cannot contain all -prom-env data

Since commit 61f20b9dc5b7 ("spapr_nvram: Pre-initialize the NVRAM to
support the -prom-env parameter"), pseries machines can pre-initialize
the "system" partition in the NVRAM with the data passed to all -prom-env
parameters on the QEMU command line.

In this case it is assumed that all the data fits in 64 KiB, but the user
can easily pass more and crash QEMU:

$ qemu-system-ppc64 -M pseries $(for ((x=0;x<128;x++)); do \
  echo -n " -prom-env " ; printf "%0.sx" {1..1024}; \
  done) # this requires ~128 Kib
malloc(): corrupted top size
Aborted (core dumped)

This happens because we don't check if all the prom-env data fits in
the NVRAM and chrp_nvram_set_var() happily memcpy() it passed the
buffer.

This crash affects basically all ppc/ppc64 machine types that use -prom-env:
- pseries (all versions)
- g3beige
- mac99

and also sparc/sparc64 machine types:
- LX
- SPARCClassic
- SPARCbook
- SS-10
- SS-20
- SS-4
- SS-5
- SS-600MP
- Voyager
- sun4u
- sun4v

Add a max_len argument to chrp_nvram_create_system_partition() so that
it can check the available size before writing to memory.

Since NVRAM is populated at machine init, it seems reasonable to consider
this error as fatal. So, instead of reporting an error when we detect that
the NVRAM is too small and adapt all machine types to handle it, we simply
exit QEMU in all cases. This is still better than crashing. If someone
wants another behavior, I guess this can be reworked later.

Tested with:

$ yes q | \
  (for arch in ppc ppc64 sparc sparc64; do \
       echo == $arch ==; \
       qemu=${arch}-softmmu/qemu-system-$arch; \
       for mach in $($qemu -M help | awk '! /^Supported/ { print $1 }'); do \
           echo $mach; \
           $qemu -M $mach -monitor stdio -nodefaults -nographic \
           $(for ((x=0;x<128;x++)); do \
                 echo -n " -prom-env " ; printf "%0.sx" {1..1024}; \
             done) >/dev/null; \
        done; echo; \
   done)

Without the patch, affected machine types cause QEMU to report some
memory corruption and crash:

malloc(): corrupted top size

free(): invalid size

*** stack smashing detected ***: terminated

With the patch, QEMU prints the following message and exits:

NVRAM is too small. Try to pass less data to -prom-env

It seems that the conditions for the crash have always existed, but it
affects pseries, the machine type I care for, since commit 61f20b9dc5b7
only.

Fixes: 61f20b9dc5b7 ("spapr_nvram: Pre-initialize the NVRAM to support the 
-prom-env parameter")
RHBZ: https://bugzilla.redhat.com/show_bug.cgi?id=1867739
Reported-by: John Snow <jsnow@redhat.com>
Reviewed-by: Laurent Vivier <laurent@vivier.eu>
Signed-off-by: Greg Kurz <groug@kaod.org>
Message-Id: <159736033937.350502.12402444542194031035.stgit@bahia.lan>
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>


  Commit: 3110f0ee19ccdb50adff3dfa1321039f69efddcd
      
https://github.com/qemu/qemu/commit/3110f0ee19ccdb50adff3dfa1321039f69efddcd
  Author: Greg Kurz <groug@kaod.org>
  Date:   2020-08-14 (Fri, 14 Aug 2020)

  Changed paths:
    M hw/intc/spapr_xive.c
    M hw/intc/spapr_xive_kvm.c

  Log Message:
  -----------
  spapr/xive: Use xive_source_esb_len()

static inline size_t xive_source_esb_len(XiveSource *xsrc)
{
    return (1ull << xsrc->esb_shift) * xsrc->nr_irqs;
}

Signed-off-by: Greg Kurz <groug@kaod.org>
Message-Id: <159733969034.320580.6571451425779179477.stgit@bahia.lan>
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>


  Commit: dd8014e4e904e895435aae9f11a686f072762782
      
https://github.com/qemu/qemu/commit/dd8014e4e904e895435aae9f11a686f072762782
  Author: Peter Maydell <peter.maydell@linaro.org>
  Date:   2020-08-24 (Mon, 24 Aug 2020)

  Changed paths:
    M docs/specs/index.rst
    A docs/specs/ppc-spapr-numa.rst
    M docs/specs/ppc-spapr-xive.rst
    M hw/intc/spapr_xive.c
    M hw/intc/spapr_xive_kvm.c
    M hw/intc/xive.c
    M hw/nvram/chrp_nvram.c
    M hw/nvram/mac_nvram.c
    M hw/nvram/spapr_nvram.c
    M hw/ppc/spapr.c
    M hw/ppc/spapr_caps.c
    M hw/ppc/spapr_irq.c
    M hw/ppc/spapr_pci.c
    M hw/sparc/sun4m.c
    M hw/sparc64/sun4u.c
    M include/elf.h
    M include/hw/nvram/chrp_nvram.h
    M include/hw/ppc/spapr_xive.h
    M include/hw/ppc/xive.h
    M target/ppc/cpu.h
    M target/ppc/helper.h
    M target/ppc/int_helper.c
    M target/ppc/translate.c
    M target/ppc/translate/spe-impl.c.inc
    M target/ppc/translate/vmx-impl.c.inc
    M target/ppc/translate/vmx-ops.c.inc
    M target/ppc/translate_init.c.inc
    M tcg/ppc/tcg-target.c.inc
    M tcg/ppc/tcg-target.h

  Log Message:
  -----------
  Merge remote-tracking branch 'remotes/dgibson/tags/ppc-for-5.2-20200818' into 
staging

ppc patch queue 2020-08-18

Here's my first pull request for qemu-5.2, which has quite a few
accumulated things.  Highlights are:

 * Preliminary support for POWER10 (Power ISA 3.1) instruction emulation
 * Add documentation on the (very confusing) pseries NUMA configuration
 * Fix some bugs handling edge cases with XICS, XIVE and kernel_irqchip
 * Fix icount for a number of POWER registers
 * Many cleanups to error handling in XIVE code
 * Validate size of -prom-env data

# gpg: Signature made Tue 18 Aug 2020 05:18:36 BST
# gpg:                using RSA key 75F46586AE61A66CC44E87DC6C38CACA20D9B392
# gpg: Good signature from "David Gibson <david@gibson.dropbear.id.au>" [full]
# gpg:                 aka "David Gibson (Red Hat) <dgibson@redhat.com>" [full]
# gpg:                 aka "David Gibson (ozlabs.org) <dgibson@ozlabs.org>" 
[full]
# gpg:                 aka "David Gibson (kernel.org) <dwg@kernel.org>" 
[unknown]
# Primary key fingerprint: 75F4 6586 AE61 A66C C44E  87DC 6C38 CACA 20D9 B392

* remotes/dgibson/tags/ppc-for-5.2-20200818: (40 commits)
  spapr/xive: Use xive_source_esb_len()
  nvram: Exit QEMU if NVRAM cannot contain all -prom-env data
  spapr/xive: Simplify error handling of kvmppc_xive_cpu_synchronize_state()
  ppc/xive: Simplify error handling in xive_tctx_realize()
  spapr/xive: Simplify error handling in kvmppc_xive_connect()
  ppc/xive: Fix error handling in vmstate_xive_tctx_*() callbacks
  spapr/xive: Fix error handling in kvmppc_xive_post_load()
  spapr/kvm: Fix error handling in kvmppc_xive_pre_save()
  spapr/xive: Rework error handling of kvmppc_xive_set_source_config()
  spapr/xive: Rework error handling in kvmppc_xive_get_queues()
  spapr/xive: Rework error handling of kvmppc_xive_[gs]et_queue_config()
  spapr/xive: Rework error handling of kvmppc_xive_cpu_[gs]et_state()
  spapr/xive: Rework error handling of kvmppc_xive_mmap()
  spapr/xive: Rework error handling of kvmppc_xive_source_reset()
  spapr/xive: Rework error handling of kvmppc_xive_cpu_connect()
  spapr: Simplify error handling in spapr_phb_realize()
  spapr/xive: Convert KVM device fd checks to assert()
  ppc/xive: Introduce dedicated kvm_irqchip_in_kernel() wrappers
  ppc/xive: Rework setup of XiveSource::esb_mmio
  target/ppc: Integrate icount to purr, vtb, and tbu40
  ...

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>


Compare: https://github.com/qemu/qemu/compare/8367a77c4d3f...dd8014e4e904



reply via email to

[Prev in Thread] Current Thread [Next in Thread]