Re: [lmi] Problems on corporate server--live stream

[Vadim Zeitlin]

On Tue, 13 Oct 2020 16:08:18 +0000 Greg Chicares <gchicares@sbcglobal.net> 
wrote:

GC> Vadim--Feel free to jump in if you feel like it, or to ignore this if
GC> you don't.

 I'm not sure if I'm going to be of much help, to be honest, as I don't
know all these shell scripts, but let me try asking some questions just in
case they allow you to see things from a different angle -- which could, or
not, be useful.

GC> I'm just using this email to record the problems I'm seeing
GC> and the fixes I'm trying. In addition to the 'nemo' issues previously
GC> discussed, I'm seeing a failure in schroot's script that is supposed
GC> to unmount a chroot's dynamic mounts, and I've removed an installation
GC> of 'wine' on the host.
GC> 
GC> /opt/lmi/src/lmi[0]$git pull
GC> /opt/lmi/src/lmi[0]$pushd /tmp
GC> /tmp /opt/lmi/src/lmi ~
GC> /tmp[0]$/opt/lmi/src/lmi/gwc/redhat_rebuild.sh
GC> Last-modified header missing -- time-stamps turned off.
GC> 2020-10-12 19:50:05 
URL:https://raw.githubusercontent.com/let-me-illustrate/lmi/master/lmi_setup_00.sh
 [6116/6116] -> "lmi_setup_00.sh" [1]
GC> 20201012T235103Z ./lmi_setup_02.sh: Removed old chroot.
GC> 20201012T235118Z ./lmi_setup_05r.sh: Reconfigured redhat chroot.
GC> 20201012T235124Z ./lmi_setup_07r.sh: Installed EPEL.
GC> 20201012T235125Z ./lmi_setup_10r.sh: Installed 'debootstrap' [redhat].
GC> 20201012T235531Z ./lmi_setup_11.sh: Ran 'debootstrap'.
GC> 20201013T003102Z ./lmi_setup_20.sh: Installed debian packages.
GC> 20201013T003104Z ./lmi_setup_21.sh: Updated debian chroot.
GC> 20201013T003109Z ./lmi_setup_24.sh: Configured users.
GC> 20201013T003109Z ./lmi_setup_25.sh: Configured {zsh,vim,git} for user 
'root'.
GC> 20201013T003110Z ./lmi_setup_25.sh: Configured {zsh,vim,git} for user 
'[REDACTED]'.
GC> 20201013T003111Z ./lmi_setup_25.sh: Configured {zsh,vim,git} for user 
'[REDACTED2]'.
GC> E: You are required to change your password immediately (password aged)
GC> 20201013T003111Z ./lmi_setup_29.sh: Created lmi directories.
GC> 20201013T003112Z ./lmi_setup_30.sh: Copied optional files.
GC> 20201013T003137Z ./lmi_setup_40.sh: Configured 'wine' for user '[REDACTED]'.

 Why is there no line "Configured 'wine' for user 'root'" here? The loop
seems to be exactly the same as the one used for lmi_setup_25.sh above,
which outputs 3 lines, but this one somehow only outputs 2. Do you
understand why?

GC> 20201013T003156Z ./lmi_setup_40.sh: Configured 'wine' for user 
'[REDACTED2]'.
GC> E: You are required to change your password immediately (password aged)

 Do you know where do these errors come from? They're apparently being
given from lmi_setup_{29,40}.sh (unless the output is not in order due to
the use of "| tee /dev/tty"?), but I don't see anything that could explain
them in these scripts and schroot commands used to run them don't seem to
differ from the commands used for the other scripts, which don't result in
errors.

GC> Anyway, it ended there with return code 1. Instead, it was supposed to
GC> continue and run 'lmi_setup_43.sh' next. Examining the logs:
GC> 
GC> 20201013T011805Z ./lmi_setup_42.sh: Installed lmi for '[REDACTED]'.
GC> E: 10mount: umount: 
/run/schroot/mount/lmi_bullseye_3-9fc3415f-d9ef-4c67-be6c-0cb6ad47e2cb: target 
is busy.
GC> E: 10mount:         (In some cases useful info about processes that use
GC> E: 10mount:          the device is found by lsof(8) or fuser(1))
GC> E: 10mount: rmdir: failed to remove 
'/var/run/schroot/mount/lmi_bullseye_3-9fc3415f-d9ef-4c67-be6c-0cb6ad47e2cb': 
Device or resource busy
GC> E: lmi_bullseye_3-9fc3415f-d9ef-4c67-be6c-0cb6ad47e2cb: Chroot setup 
failed: stage=setup-stop

 This one seems to be coming from schroot command in lmi_setup_01.sh, but I
again have trouble seeing the difference between it and the previous
command that should have used the same --user option value in the loop
calling lmi_setup_40.sh just above

GC> https://wiki.debian.org/Schroot
GC> | to retrive the PID you can take a piece of the name of the directory,
GC> | says "d2c072e7" and look for in the /proc filesystem:
GC> |
GC> | ~$ grep -r d2c072e7 /proc/*/mountinfo
GC> 
GC> I tried that, with '0cb6ad', but it was no help--way too many PIDs are 
shown.

 This doesn't seem normal, is it? I.e. why should there be many processes
using this chroot? In any case, what are they?

GC> These commands:
GC>   lsof  /var/run/schroot/mount/lmi_bullseye_3-<Tab>
GC>   fuser /var/run/schroot/mount/lmi_bullseye_3-<Tab>
GC> return nothing,

 This is really strange. If you take one of the many PIDs found above and
use "lsof $PID" on it, don't you see this directory in the output?

 And, just to be clear, did you run these commands as root? Because you
definitely need to.

GC> So I used 'umount -l', the lazy-but-forcible option, and then
GC>   $mount | grep cb6ad
GC> showed nothing.

 To be honest I've never used "umount --lazy", but its man page seems to
say that it should really be followed by a reboot, so maybe you should do
this to avoid some [even more] mysterious problems in the future?

GC> Skipping 'lmi_setup_43.sh' (set up proprietary product repository),
GC> I tried running 'lmi_setup_44.sh' (nychthemeral test) directly:
GC> 
GC> $schroot --chroot=chroot:lmi_bullseye_3 --user=`whoami` --directory=tmp 
./lmi_setup_44.sh 2>1 |less -S
GC> 
GC> 0022
GC> 0002
GC> LMI_TRIPLET = "x86_64-w64-mingw32"
GC>   Production system built--ready to start GUI test in another session.
GC> wine: a wine server seems to be running, but I cannot connect to it.
GC>    You probably need to kill that process (it might be pid 29580).
GC> make[2]: *** [/opt/lmi/src/lmi/workhorse.make:1331: cli_selftest] Error 1
GC> make[1]: *** [/opt/lmi/src/lmi/workhorse.make:1316: cli_tests] Error 2
GC> make: *** [GNUmakefile:241: /opt/lmi/gcc_x86_64-w64-mingw32/build/ship] 
Error 2
GC> make: Target 'cgi_tests' not remade because of errors.
GC> make: Target 'cli_tests' not remade because of errors.
GC> 20201013T124435Z ./lmi_setup_44.sh: Ran nychthemeral test for '[REDACTED]'.
GC> 20201013T124435Z ./lmi_setup_44.sh: Ran nychthemeral test for '[REDACTED]'.
GC> 
GC> Searching for that exact error message:
GC>   
https://www.google.com/search?q=%22wine:+a+wine+server+seems+to+be+running,+but+I+cannot+connect+to+it.%22&filter=0
GC> finds one unanswered question at winehq.org, an apparent copy
GC> of the same at some domain whose name looks like leetspeak,
GC> and several Cyrillic instances of a "Lenux Game Tread", mostly
GC> at some Hong Kong domain, which seem to suggest using 'proton'
GC> instead of 'wine'. That's not a path I want to follow.

 Searching for "wine server" finds many more results, not all of them
relevant (although drowning our sorrows in a bottle starts seeming more and
more appealing, the further I read this message...), but the gist of the
ones which are seems to be that wine server is always supposed to be
stopped by wine itself when it exits. So I wonder if you still had some
wine processes running at this time?

 Also, have you tried looking at the PID 29580? Is it stuck in the kernel
on some non-interruptible IO call or something?

GC> Repeating the above command yet again, it seems close to succeeding.
GC> I see no errors obviously attributable to the OS or to 'wine'.
GC> Somehow, this file exists:
GC>   /opt/lmi/bin/data/configurable_settings.xml
GC> instead of the intended one:
GC>   /opt/lmi/data/configurable_settings.xml
GC> but at least that appears to be a problem under my control.

 I have absolutely no idea how could the file end up in a wrong directory,
but the idea of rebooting really doesn't seem so bad by now. Something
seems to be seriously wrong here.

 Again, sorry for the lack of help and good luck!
VZ

pgpuZM3uaADED.pgp
Description: PGP signature