Re: Emulating Solaris 10 on SPARC64 sun4u

[jasper.lowell]

Good idea.

The ESCC device looks like it's written as a sysbus device. I think the
Ultra 5 has no devices on the root sabre bus. The serial controller is
behind the ebus (essentially isa). I'm guessing I would need to write a
wrapper device around the memory io functions so that it can be used
under the ebus - judging from the serial-isa implementation.

I think it might be possible to leave the ESCC as a sysbus device but
I'm not familiar enough with OpenBIOS to expose the right information
to Solaris and reason about what's happening confidentally. I don't
expect writing a wrapper for isa to be difficult so I'll give that a
go. Do you think it would be fine to just choose an arbitrary ioport
address for the device?

Thanks,
Jasper Lowell.


On Fri, 2020-05-08 at 15:45 +0200, Artyom Tarasenko wrote:
> On Fri, May 8, 2020 at 10:51 AM Peter Tribble <
> address@hidden> wrote:
> > I see the same behaviour as reported here when booting current
> > SPARC illumos
> > (illumos is the ongoing fork of OpenSolaris) under qemu - looks
> > like it's booted
> > up fine, but I can't type anything on the console.
> 
> There is one more option which can be relatively easily tested:
> you can add two more ports (or replace the existing ttya/ttyb) to the
> qemu's ultra5 model: the escc (aka zs) ports.
> They definitely work under Solaris (Ultra-1, Ultra-2, E3000,
> E3500...), I've seen it.
> OpenBIOS already has a driver for them which is used for sun4m (and
> some ppc) targets. So no new devices have to be implemented.
> If your and Jasper's theory proofs to be correct, we can think of
> replacing ttya/ttyb su with zs.
> I guess the other sun4u operating systems do support zs as well.
> (This
> has to be tested though)
> 
> 
> > While I'm an illumos developer, and maintain it on SPARC, I'm
> > unfamiliar with
> > most of the internals. We'll try and have a poke around, though.
> > 
> > (The aim would be to use qemu to allow illumos developers to test
> > changes on SPARC
> > without requiring them to have SPARC hardware, hence my interest.)
> 
> I think I managed booting Tribblix under the QEMU niagara target. It
> has less devices than sun4u, but the console was working.
> 
> Regards,
> Artyom
> 
> > On Fri, May 8, 2020 at 3:40 AM <address@hidden> wrote:
> > > There are two different drivers for the 16550A in OpenSolaris.
> > > 
> > > There is a generic driver in /usr/src/uts/common/io/asy.c. This
> > > driver
> > > clearly states in comments that it is assigning the device to
> > > tty[a-d].
> > > It's really obvious to me that there is support in this driver
> > > for
> > > using the device for a tty.
> > > 
> > > There is another driver at /usr/src/uts/sun4/io/su_driver.c.
> > > Judging
> > > from the name this is specific for SuperIO. This driver is more
> > > than
> > > 1000 lines shorter than the other driver and contains a note at
> > > the top
> > > of the file that it is "modified as sparc keyboard/mouse driver."
> > > I
> > > don't have much experience with Solaris internals but I can't see
> > > any
> > > obvious signs that this can be used as a tty. I'd also question
> > > why
> > > there are two drivers if they have the same purpose/capability.
> > > If the
> > > su_driver exists to only add support for the keyboard/mouse I'm
> > > not
> > > sure why it would be shorter rather than longer. It would be
> > > helpful if
> > > someone with Solaris experience, that knows what they're doing (I
> > > do
> > > not), can take a look at this driver and give a better opinion.
> > > 
> > > When emulating Sun4u, the su driver is used. If you misconfigure
> > > the
> > > serial device in QEMU, you can trigger errors that are unique to
> > > the
> > > driver. Also, Solaris attaches the device as su0.
> > > 
> > > Solaris can access the registers, identify/attach the device, but
> > > it
> > > just never attempts to use it. Interrupts are explicitly disabled
> > > and
> > > the device is never polled. I don't think asyopen in su_driver.c
> > > is
> > > ever called. I've thoroughly stepped through every register
> > > access for
> > > the 16550A and nothing seems out of the ordinary to me. Something
> > > I've
> > > considered is that OpenBIOS is not setting properties for the
> > > device in
> > > a way that Solaris expects but I'm not familiar enough with Sun's
> > > OpenBOOT to make progress here. I've read through the PROM
> > > functions
> > > that Solaris calls when choosing what device to use as a console
> > > and
> > > nothing seemed inconsistent with what OpenBIOS does.
> > > Unfortunately,
> > > OpenBIOS seems to crash or hang when using the kernel debugger
> > > module,
> > > so dynamic analysis using breakpoints isn't an option.
> > > 
> > > I don't have any concrete explanations for the broken console but
> > > rather than do nothing I figured I'd see what happens by
> > > emulating the
> > > SAB 82532.
> > > 
> > > > At least Fujitsu Siemens PRIMEPOWER250 had a tty attached to
> > > > the
> > > > 16550A UART. I think there were more such machines. I don't
> > > > expect
> > > > there is anything in the Solaris kernel which would prevent any
> > > > serial
> > > > device known to it to be used as a tty.
> > > 
> > > Do you happen to know if this was over SuperIO? If this machine
> > > was
> > > using the same su_driver for a serial console it would rule out
> > > emulating the hardware-faithful SAB 82532 as being a possible
> > > improvement. I can't find anything online about someone using
> > > SuperIO
> > > for a ttya on an Ultra 5 running Solaris.
> > > 
> > > > Well, theoretically yes, but practically there is just one
> > > > baudrate
> > > > which can be specified in the OBP. I think it's perfectly safe
> > > > to use
> > > > max(rxrate,txrate), or min(rxrate,txrate), whatever you prefer.
> > > 
> > > I'll do this while looking at whether or not the different chip
> > > fixes
> > > the console problems on Sun4u. I'd personally like to avoid
> > > introducing
> > > a device to QEMU that makes assumptions about how the guest may
> > > configure the chip.
> > > 
> > > Thanks,
> > > Jasper Lowell.
> > > 
> > > 
> > > On Thu, 2020-05-07 at 17:02 +0200, Artyom Tarasenko wrote:
> > > > On Thu, May 7, 2020 at 4:29 PM <address@hidden> wrote:
> > > > > Just thought I'd chime in with an update.
> > > > > 
> > > > > We are currently emulating a 16550A UART. The guest sees this
> > > > > as
> > > > > the SU
> > > > > device, referring to the SuperIO port (a pair of 16550A
> > > > > UARTs). On
> > > > > the
> > > > > Ultra 5, the machine that Sun4u is modelled against, SuperIO
> > > > > was
> > > > > used
> > > > > for the keyboard and mouse. The Ultra 5 also had a DB25 (ttya
> > > > > default)
> > > > > and a DB9 (ttyb default) with a SAB82532 ESSC2.
> > > > > 
> > > > > Using tracing, I've looked through how the 16550A UART is
> > > > > touched
> > > > > and
> > > > > it looks like Solaris 10 has no issues identifying the
> > > > > device. I've
> > > > > matched register accesses with driver code in OpenSolaris and
> > > > > I'm
> > > > > pretty sure the device is attached successfully. Also, if you
> > > > > boot
> > > > > Solaris 10 with debugging output, you can see that the device
> > > > > gets
> > > > > labelled as su0. The only time Solaris is capable of writing
> > > > > to the
> > > > > console is when OpenBIOS is used as a proxy.
> > > > > 
> > > > > Rather than Solaris deciding against using SuperIO as a tty,
> > > > > I
> > > > > don't
> > > > > think there was ever any support for doing so (at least on
> > > > > SPARC
> > > > > machines). This could be an explanation for why the system
> > > > > appears
> > > > > to
> > > > > be trucking along just fine despite a seemingly frozen
> > > > > console -
> > > > > there
> > > > > is no console. I don't think the frozen console is the fault
> > > > > of
> > > > > broken
> > > > > interrupt routing as the 16550A UART is never programmed to
> > > > > generate
> > > > > them.
> > > > 
> > > > At least Fujitsu Siemens PRIMEPOWER250 had a tty attached to
> > > > the
> > > > 16550A UART. I think there were more such machines. I don't
> > > > expect
> > > > there is anything in the Solaris kernel which would prevent any
> > > > serial
> > > > device known to it to be used as a tty.
> > > > 
> > > > > I've started work on emulating the SAB 82532 ESSC2 but it's
> > > > > unfortunately way more complex than than the 16550A. For
> > > > > instance,
> > > > > it's
> > > > > possible to configure different baudrates for receiving and
> > > > > transmitting. QEMU's chardev interface doesn't appear to
> > > > > handle
> > > > > that.
> > > > > QEMUSerialSetParams has a single speed value that is passed
> > > > > to
> > > > > cfsetispeed and cfsetospeed. The chip also has support for
> > > > > stick
> > > > > parity
> > > > > , which aren't valid options right now either. If I'm wrong
> > > > > on
> > > > > either
> > > > > of those points please correct me. Unless there is an
> > > > > alternative,
> > > > > changes to the interface may need to be made if adding this
> > > > > device
> > > > > is
> > > > > to be considered.
> > > > 
> > > > Well, theoretically yes, but practically there is just one
> > > > baudrate
> > > > which can be specified in the OBP. I think it's perfectly safe
> > > > to use
> > > > max(rxrate,txrate), or min(rxrate,txrate), whatever you prefer.
> > > > 
> > > > Regards,
> > > > Artyom
> > > > > On Sun, 2020-02-09 at 11:26 +0000, Mark Cave-Ayland wrote:
> > > > > > On 05/02/2020 06:31, address@hidden wrote:
> > > > > > 
> > > > > > > I'm currently working towards emulating Solaris 10 on
> > > > > > > sun4u.
> > > > > > > 
> > > > > > > 
> > > > > > > 
> > > > > > > The Solaris 10 ISO image I am attempting to boot is the
> > > > > > > one
> > > > > > > from
> > > > > > > the Oracle
> > > > > > > 
> > > > > > > download page at
> > > > > > > https://www.oracle.com/solaris/solaris10/downloads/solaris10-get-jsp-downloads.html.
> > > > > > > 
> > > > > > > Image: sol-10-u11-ga-sparc-dvd.iso
> > > > > > > 
> > > > > > > MD5:   53e8b066f7f250ce2fd2cef063f8072b
> > > > > > > 
> > > > > > > 
> > > > > > > 
> > > > > > > I am using QEMU commit
> > > > > > > 7bd9d0a9e26c7a3c67c0f174f0009ba19969b158.
> > > > > > > 
> > > > > > > 
> > > > > > > 
> > > > > > > The command I am using to run QEMU is:
> > > > > > > 
> > > > > > > ./qemu/sparc64-softmmu/qemu-system-sparc64 -bios
> > > > > > > ./openbios/obj-sparc64/openbios-builtin.elf -cdrom
> > > > > > > ./iso/solaris/sol-10-u11-ga-sparc-dvd.iso -boot d
> > > > > > > -nographic -m
> > > > > > > 3G
> > > > > > > 
> > > > > > > 
> > > > > > > 
> > > > > > > ```
> > > > > > > 
> > > > > > > CPUs: 1 x SUNW,UltraSPARC-IIi
> > > > > > > 
> > > > > > > UUID: 00000000-0000-0000-0000-000000000000
> > > > > > > 
> > > > > > > Welcome to OpenBIOS v1.1 built on Feb 5 2020 19:15
> > > > > > > 
> > > > > > >   Type 'help' for detailed information
> > > > > > > 
> > > > > > > Trying cdrom:f...
> > > > > > > 
> > > > > > > Not a bootable ELF image
> > > > > > > 
> > > > > > > Not a bootable a.out image
> > > > > > > 
> > > > > > > 
> > > > > > > 
> > > > > > > Loading FCode image...
> > > > > > > 
> > > > > > > Loaded 7420 bytes
> > > > > > > 
> > > > > > > entry point is 0x4000
> > > > > > > 
> > > > > > > Evaluating FCode...
> > > > > > > 
> > > > > > > Evaluating FCode...
> > > > > > > 
> > > > > > > Ignoring failed claim for va 1000000 memsz af6d6!
> > > > > > > 
> > > > > > > Ignoring failed claim for va 1402000 memsz 4dcc8!
> > > > > > > 
> > > > > > > Ignoring failed claim for va 1800000 memsz 510c8!
> > > > > > > 
> > > > > > > SunOS Release 5.10 Version Generic_147147-26 64-bit
> > > > > > > 
> > > > > > > Copyright (c) 1983, 2013, Oracle and/or its affiliates.
> > > > > > > All
> > > > > > > rights
> > > > > > > reserved.
> > > > > > > 
> > > > > > > could not find debugger-vocabulary-
> > > > > > > hook>threads:interpret:
> > > > > > > exception -13 caught
> > > > > > > 
> > > > > > > interpret \ Copyright (c) 1995-1999 by Sun Microsystems,
> > > > > > > Inc.
> > > > > > > 
> > > > > > > \ All rights reserved.
> > > > > > > 
> > > > > > > \
> > > > > > > 
> > > > > > > \ ident "@(#)data64.fth  1.3     00/07/17 SMI"
> > > > > > > 
> > > > > > > 
> > > > > > > 
> > > > > > > hex
> > > > > > > 
> > > > > > > 
> > > > > > > 
> > > > > > > only forth also definitions
> > > > > > > 
> > > > > > > vocabulary kdbg-words
> > > > > > > 
> > > > > > > also kdbg-words definitions
> > > > > > > 
> > > > > > > 
> > > > > > > 
> > > > > > > defer p@
> > > > > > > 
> > > > > > > defer p!
> > > > > > > 
> > > > > > > ['] x@ is p@
> > > > > > > 
> > > > > > > ['] x! is p!
> > > > > > > 
> > > > > > > 
> > > > > > > 
> > > > > > > 8 constant ptrsize
> > > > > > > 
> > > > > > > 
> > > > > > > 
> > > > > > > d# 32 constant nbitsminor
> > > > > > > 
> > > > > > > h# ffffffff constant maxmin
> > > > > > > 
> > > > > > > \
> > > > > > > 
> > > > > > > \ Copyright 2008 Sun Microsystems, Inc.  All rights
> > > > > > > reserved.
> > > > > > > 
> > > > > > > \ Use is subject to license terms.
> > > > > > > 
> > > > > > > \
> > > > > > > 
> > > > > > > 
> > > > > > > 
> > > > > > > \ #pragma ident  "@(#)kdbg.fth    1.20    08/06/06 SMI"
> > > > > > > 
> > > > > > > 
> > > > > > > 
> > > > > > > h# 7ff constant v9bias
> > > > > > > 
> > > > > > > h# unix-tte:interpret: exception -13 caught
> > > > > > > 
> > > > > > > interpret ' unix-tte is va>tte-data failed with error
> > > > > > > ffffffffffffffed
> > > > > > > 
> > > > > > > WARNING: consconfig: cannot find driver for screen device
> > > > > > > /pci@1fe,
> > > > > > > 0/pci@1,1/QEMU,VGA@2
> > > > > > > 
> > > > > > > Configuring devices.
> > > > > > > 
> > > > > > > WARNING: Interrupt not seen after set_features
> > > > > > > 
> > > > > > > Using RPC Bootparams for network configuration
> > > > > > > information.
> > > > > > > 
> > > > > > > Attempting to configure interface hme0...
> > > > > > > 
> > > > > > > WARNING: Power off requested from power button or SC,
> > > > > > > powering
> > > > > > > down
> > > > > > > the system!
> > > > > > > 
> > > > > > > Skipped interface hme0
> > > > > > > 
> > > > > > > svc:/system/filesystem/local:default: WARNING:
> > > > > > > /usr/sbin/zfs
> > > > > > > mount
> > > > > > > -a failed: one or
> > > > > > > more file systems failed to mount
> > > > > > > 
> > > > > > > Serial console, reverting to text install
> > > > > > > 
> > > > > > > Beginning system identification...
> > > > > > > 
> > > > > > > Searching for configuration file(s)...
> > > > > > > 
> > > > > > > Search complete.
> > > > > > > 
> > > > > > > Discovering additional network configuration...
> > > > > > > 
> > > > > > > ```
> > > > > > > 
> > > > > > > 
> > > > > > > 
> > > > > > > The installation menu is shown after but the console is
> > > > > > > unresponsive.
> > > > > > > 
> > > > > > > 
> > > > > > > 
> > > > > > > After some debugging, it looks like the QEMU front-end is
> > > > > > > correctly
> > > > > > > filling
> > > > > > > 
> > > > > > > the serial receive buffer with characters, and then
> > > > > > > starts
> > > > > > > dropping
> > > > > > > them once
> > > > > > > 
> > > > > > > the number of characters in the buffer reach the
> > > > > > > interrupt
> > > > > > > level.
> > > > > > > The interrupt
> > > > > > > 
> > > > > > > level happens to be 1 when booting Solaris 10. This looks
> > > > > > > like
> > > > > > > normal behaviour
> > > > > > > 
> > > > > > > to me.
> > > > > > > 
> > > > > > > 
> > > > > > > 
> > > > > > > I started looking at why the serial receive buffer might
> > > > > > > not be
> > > > > > > consumed and
> > > > > > > 
> > > > > > > considered that interrupts might not be being raised
> > > > > > > correctly.
> > > > > > > I
> > > > > > > ran with
> > > > > > > 
> > > > > > > tracing and there were no interrupts for IRQ 0x2b like
> > > > > > > there
> > > > > > > are
> > > > > > > when using
> > > > > > > 
> > > > > > > OpenBSD. When inspecting the registers of the serial
> > > > > > > device it
> > > > > > > looks like the
> > > > > > > 
> > > > > > > Interrupt Enable Register is set to zero.
> > > > > > > 
> > > > > > > 
> > > > > > > 
> > > > > > > If Solaris 10 was using the device is polling mode, it
> > > > > > > should
> > > > > > > be
> > > > > > > reading the RBR
> > > > > > > 
> > > > > > > or at least the LSR. When tracing serial_ioport_read and
> > > > > > > serial_ioport_write,
> > > > > > > 
> > > > > > > once the menu is hit, I don't see any read or writes to
> > > > > > > the
> > > > > > > serial
> > > > > > > device
> > > > > > > 
> > > > > > > registers despite me trying to send characters and use
> > > > > > > the
> > > > > > > menu.
> > > > > > > 
> > > > > > > 
> > > > > > > 
> > > > > > > The driver Solaris 10 is using for the device appears to
> > > > > > > be
> > > > > > > similar/same as
> > > > > > > 
> > > > > > > /usr/src/uts/sun4/io/su_driver.c in the OpenSolaris code
> > > > > > > found
> > > > > > > at
> > > > > > > https://github.com/nxmirrors/onnv.
> > > > > > > 
> > > > > > > 
> > > > > > > 
> > > > > > > ```
> > > > > > > 
> > > > > > > asy->asy_hwtype = ASY16550AF;
> > > > > > > 
> > > > > > > OUTB(FIFOR, 0x00); /* clear fifo register */
> > > > > > > 
> > > > > > > asy->asy_trig_level = 0x00; /* sets the fifo Threshold to
> > > > > > > 1 */
> > > > > > > 
> > > > > > > 
> > > > > > > 
> > > > > > > /* set/Enable FIFO */
> > > > > > > 
> > > > > > > OUTB(FIFOR, FIFO_ON | FIFODMA | FIFOTXFLSH | FIFORXFLSH |
> > > > > > > 
> > > > > > > (asy->asy_trig_level & 0xff));
> > > > > > > 
> > > > > > > 
> > > > > > > 
> > > > > > > if ((INB(ISR) & 0xc0) == 0xc0)
> > > > > > > 
> > > > > > >     asy->asy_use_fifo = FIFO_ON; /* QEMU REACHES HERE. */
> > > > > > > 
> > > > > > > else {
> > > > > > > 
> > > > > > >     asy->asy_hwtype = ASY8250;
> > > > > > > 
> > > > > > >     OUTB(FIFOR, 0x00); /* NO FIFOs */
> > > > > > > 
> > > > > > >     asy->asy_trig_level = 0;
> > > > > > > 
> > > > > > > }
> > > > > > > 
> > > > > > > ```
> > > > > > > 
> > > > > > > 
> > > > > > > 
> > > > > > > From what I can tell when tracing serial_ioport_write and
> > > > > > > serial_ioport_read,
> > > > > > > 
> > > > > > > Solaris 10 correctly identifies the serial device and
> > > > > > > successfully
> > > > > > > attaches it.
> > > > > > > 
> > > > > > > In the asyattach function (OpenSolaris driver),
> > > > > > > interrupts are
> > > > > > > disabled by zeroing the
> > > > > > > 
> > > > > > > Interrupt Enable Register. From what I'm reading in
> > > > > > > OpenSolaris
> > > > > > > source code, interrupts
> > > > > > > 
> > > > > > > are reenabled when the device is "opened". This seems
> > > > > > > like
> > > > > > > consistent and
> > > > > > > 
> > > > > > > correct behaviour though I'm not sure why the device is
> > > > > > > not
> > > > > > > being
> > > > > > > opened to be
> > > > > > > 
> > > > > > > used by the serial console.
> > > > > > > 
> > > > > > > 
> > > > > > > 
> > > > > > > Is this an issue anyone else has tried to debug?
> > > > > > > 
> > > > > > > Are there any leads that I can follow up on for why the
> > > > > > > serial
> > > > > > > console becomes
> > > > > > > unresponsive
> > > > > > > 
> > > > > > > on Solaris 10?
> > > > > > 
> > > > > > It has been a while since I've looked at booting Solaris >=
> > > > > > 10
> > > > > > but
> > > > > > certainly the
> > > > > > messages above about set_features and the frozen console
> > > > > > suggest
> > > > > > that
> > > > > > something is
> > > > > > going amiss with interrupt routing, although since Linux
> > > > > > and
> > > > > > NetBSD
> > > > > > were fine the
> > > > > > last time I ran my OpenBIOS release tests then Solaris must
> > > > > > be
> > > > > > doing
> > > > > > something
> > > > > > different here.
> > > > > > 
> > > > > > Note that the serial interrupts are routed from the ebus
> > > > > > into
> > > > > > sabre
> > > > > > so the first
> > > > > > thing to check would be that the end-to-end routing from
> > > > > > device
> > > > > > to
> > > > > > CPU looks correct
> > > > > > when using Solaris.
> > > > > > 
> > > > > > 
> > > > > > ATB,
> > > > > > 
> > > > > > Mark.
> > 
> > 
> > --
> > -Peter Tribble
> > http://www.petertribble.co.uk/ - http://ptribble.blogspot.com/
> 
>