Re: [lmi] Where in /etc are customizations preserved?

[Greg Chicares]

On 2020-08-17 21:35, Vadim Zeitlin wrote:
> On Mon, 17 Aug 2020 20:28:13 +0000 Greg Chicares <gchicares@sbcglobal.net> 
> wrote:
> 
> GC> If you really enjoy puzzles, explain this before reading further:
> GC> 
> GC> /opt/lmi/src/lmi[0]$make $coefficiency unit_tests 2>&1 | tee >(grep 
> '\*\*\*') >(grep '????') >(grep '!!!!' --count | xargs printf '%d tests 
> succeeded\n') >../log
> GC> tee: /proc/self/fd/17: No such file or directory
> GC> tee: /proc/self/fd/18: No such file or directory
> GC> tee: /proc/self/fd/19: No such file or directory
> GC> 0 tests succeeded
> 
>  My guess is that /proc wasn't mounted. Do I win?

Yes, you certainly have won. Me, I guessed that they had restricted
my capabilities to something less than ksh88. I went digging through
the process-substitution documentation in the zsh manual (and did
learn about its '=()' substitution, but that wouldn't help here).
I resolved to ask you before trying to rework it to use named pipes,
but first I decided to go poking around in /proc, which turned out
to be empty; so I looked at the mount table, and was enlightened.

> GC> SPOILER COMING BELOW
> GC> 
> GC> Vadim--Our redhat server just got an upgrade from 7.6 to 7.8,
> GC> and some files I had saved in /etc were removed.
> 
>  I'm not an expert in RedHat, but I'm almost sure that this is _not_ a
> normal consequence of an upgrade

That would seem well-nigh unimaginable.

> and that these files must have been
> explicitly removed by the system administrator for some reason. Perhaps
> they didn't really upgrade the system at all but just re-imaged it?

No, they left /srv and /opt untouched, as well as the directories
in /etc/schroot that are created by schroot installation. Looking
through root's command history, I see:

unalias -a; e1='161803399999999'; e2='314159265358979'; b='[[:blank:]]';

apparently because pi can be calculated in double precision but
phi cannot, and

echo __QUALYS\\_EOC__7__;umask 077 && umask;echo __QUALYS\\_EOC__8__

which means they must be using this:

https://qualysguard.qualys.com/qwebhelp/fo_portal/module_pc/policies/special_data_point_status_codes.htm

So it's like those source-code testers that says stuff like

    SEVERE DANGER!!!
    1  char *dir, *file, pname[MAXPATHLEN];
    2  if(strlcpy(pname, dir, sizeof(pname)) >= sizeof(pname))
    3    goto toolong;
    [2] strlcpy() is deprecated! use only strcpy_s!
    [3] goto is dangerous! never use it! or else!!!
    PROGRAM FAILED SECURITY CHECK!!!

whose authors provide fantastically detailed analysis, along
with a GUI with a slider marked "lax - strong - strictest"...
and someone who knows nothing about programming figures that
pushing the slider all the way to "strictest" accomplishes
some useful purpose.

And then people respond with
  #define branch goto
  #define strcpy_s strlcpy
and the net outcome is only the imposition of silly overhead.

> GC> Is there any standard guidance to inform me where I can place files
> GC> that need to be preserved?
> 
>  If they just overwrite /etc, I'm afraid there is not much that you can do.
> I use etckeeper (https://etckeeper.branchable.com/) religiously on all my
> Linux systems, but even it wouldn't have helped in this case -- unless you
> maintained a copy of /etc/.git somewhere outside of it.

Maybe that's not such a bad idea.

> GC> Accordingly, I had created a directory "relative to /etc/schroot":
> GC>   mkdir -p /etc/schroot/lmi_profile
> GC> and populated it, e.g.:
> GC>   cat >/etc/schroot/lmi_profile/fstab <<EOF
> GC>   ...
> GC>   touch    /etc/schroot/lmi_profile/copyfiles
> GC>   touch    /etc/schroot/lmi_profile/nssdatabases
> GC> but right after the upgrade those files vanished,
> 
>  Are these files created by the lmi setup scripts? If not, I'd still
> recommend storing them in a Git repository somewhere, to at least have a
> backup in case this happens again.

Yes, they're created by 'lmi_setup_11.sh'. Maybe we'll need a
standalone 'resurrect.sh' script to restore them.

> GC> Am I missing some general guideline that would have kept us
> GC> safe from this?
> 
>  Again, I don't really know what happened, but I strongly suspect they've
> just recreated all system directories. Have your other modifications, e.g.
> lmi group creation, survived this update?

Interesting question. User 'nemo' is still in /etc/passwd,
and 'lmi' is still in /etc/group, with the right members,
so whatever was mangled was very specific. Inside the chroot,
it looks like the users I added with
  --password="$(openssl passwd -1 --salt '' expired)"
have retained their passwords, but those are obviously
uncrackable because I used 'openssl'.

> GC> I wouldn't assume that the upgrade involved only the RHEL
> GC> equivalent of "apt-get dist-upgrade". I have a feeling they
> GC> probably ran some script obtained from some vendor. I tried
> GC> looking into the recent activity with 'etckeeper', which was
> GC> left installed...but its git repository was liquidated (root
> GC> cannot find it at all).
> 
>  Ah, so you did have it too... but no backup, I guess? I also run "git push
> --mirror" to some remote repository on the machines that I really care
> about, but in this case perhaps you could just keep a mirror of it on the
> same machine, but in your home directory, to at least be able to restore
> the files from it easily.

You do that manually? I haven't tried this yet, but it
looks like it would do the same thing automatically,
unless you've disabled daily autocommits:

  git remote add origin file:///home/greg/
  sed -i /etc/etckeeper/etckeeper.conf 
-e's/PUSH_REMOTE=""/PUSH_REMOTE="origin"/'

or maybe add an /etc/etckeeper/commit.d/65_push_mirror script
if you want to customize it.