[lmi] linux-perf in chroot via .deb abuse [Was: perf]

[Greg Chicares]

On 2020-09-28 21:10, Greg Chicares wrote:
> On 2020-09-28 13:41, Vadim Zeitlin wrote:
>> On Mon, 28 Sep 2020 12:02:19 +0000 Greg Chicares <gchicares@sbcglobal.net> 
>> wrote:
[...]
>> GC> My next step will be to compile lmi on the host. I haven't
>> GC> installed gcc there yet, but 'buster' has gcc-8.3.0-1, which
>> GC> should be good enough for now. For std::filesystem, we'll need
>> GC> gcc-10, which is in 'bullseye', and I'm sure it can't be too
>> GC> hard to install that in 'buster'.
>> 
>>  I'm afraid it's going to be difficult, i.e. installing the new package
>> will require upgrading so much that you will basically end up with a half
>> Buster/half Bullseye system, and mostly with Bullseye, I think.
> 
> I'm so glad you said that. You probably saved me days of agony.
> My 'buster' chroots are chimerical:
>   $ cat /etc/os-release
>   PRETTY_NAME="Debian GNU/Linux bullseye/sid"
> and I don't care because I can easily regenerate them; but
> I want to keep my host system stable.
> 
> What would be better?
Vadim--Do you think this approach is worse than trying to compile
'perf' in a chroot with the host's kernel version? It just seemed
easier to try this way first.

Whatever abusive approach is taken in a chroot shouldn't harm the
host, so let's transplant the host's 'perf' .deb files into the
chroot and run 'perf' there.

* in chroot:

[Build GNU/Linux native lmi-CLI: nothing new here.]

$export LMI_TRIPLET=x86_64-pc-linux-gnu
$make clean
GUI build not yet supported.
rm --force --recursive /opt/lmi/gcc_x86_64-pc-linux-gnu/build/ship
$LD_LIBRARY_PATH=: make $coefficiency cli_tests 2>&1 |less -S
$lmipath=/opt/lmi/gcc_x86_64-pc-linux-gnu/build/ship; export 
LD_LIBRARY_PATH=$lmipath; $lmipath/lmi_cli_shared --accept 
--data_path=/opt/lmi/data --selftest
Test speed:
  naic, no solve      : 1.921e-02 s mean;      18645 us least of  53 runs
  naic, specamt solve : 3.334e-02 s mean;      32641 us least of  30 runs
  naic, ee prem solve : 3.056e-02 s mean;      29947 us least of  33 runs
  finra, no solve     : 7.555e-03 s mean;       7260 us least of 100 runs
  finra, specamt solve: 2.047e-02 s mean;      19986 us least of  49 runs
  finra, ee prem solve: 1.905e-02 s mean;      18642 us least of  53 runs

[That's what we want to run under 'perf'.]

$mkdir /tmp/perfdir

* on host:

[Guesswork based on 'ldd' command immediately below.]

#cp -a /var/cache/apt/archives/linux-perf-4.19_4.19.146-1_amd64.deb 
/srv/chroot/lmi_bullseye_3/tmp 
#cp -a /var/cache/apt/archives/libopencsd0_0.10.1-1_amd64.deb 
/srv/chroot/lmi_bullseye_3/tmp                                                  
                                          
#cp -a /var/cache/apt/archives/libperl5.28_5.28.1-6+deb10u1_amd64.deb 
/srv/chroot/lmi_bullseye_3/tmp                                                  
                                  
#cp -a /var/cache/apt/archives/libpython3.7-minimal_3.7.3-2+deb10u2_amd64.deb 
/srv/chroot/lmi_bullseye_3/tmp 
#cp -a /var/cache/apt/archives/libbabeltrace1_1.5.6-2+deb10u1_amd64.deb 
/srv/chroot/lmi_bullseye_3/tmp                                                  
                                
#cp -a /var/cache/apt/archives/libpython3.7_3.7.3-2+deb10u2_amd64.deb 
/srv/chroot/lmi_bullseye_3/tmp/perfdir 

* in chroot:

$cd /tmp/perfdir
$sudo dpkg-deb -x linux-perf-4.19_4.19.146-1_amd64.deb .
$ldd ./usr/bin/perf_4.19 2>&1 |grep 'not found'
        libopencsd_c_api.so.0 => not found
        libopencsd.so.0 => not found
        libperl.so.5.28 => not found
        libpython3.7m.so.1.0 => not found
        libbabeltrace-ctf.so.1 => not found

[It's not too hard to guess what .deb files provide those dependencies.]

$for z in *.deb ; do sudo dpkg-deb -x "$z" . ; done
$LD_LIBRARY_PATH=./usr/lib/x86_64-linux-gnu/ ldd ./usr/bin/perf_4.19 |grep 'not 
found'

$sudo LD_LIBRARY_PATH=./usr/lib/x86_64-linux-gnu/ ./usr/bin/perf_4.19 record -- 
sh -c 'lmipath=/opt/lmi/gcc_x86_64-pc-linux-gnu/build/ship; export 
LD_LIBRARY_PATH=$lmipath; $lmipath/lmi_cli_shared --accept 
--data_path=/opt/lmi/data --selftest'
Test speed:
  naic, no solve      : 1.944e-02 s mean;      19051 us least of  52 runs
  naic, specamt solve : 3.367e-02 s mean;      33265 us least of  30 runs
  naic, ee prem solve : 3.095e-02 s mean;      30543 us least of  33 runs
  finra, no solve     : 7.699e-03 s mean;       7380 us least of 100 runs
  finra, specamt solve: 2.077e-02 s mean;      20367 us least of  49 runs
  finra, ee prem solve: 1.943e-02 s mean;      19060 us least of  52 runs
[ perf record: Woken up 4 times to write data ]
Failed to read max cpus, using default of 4096
[ perf record: Captured and wrote 0.952 MB perf.data (24593 samples) ]
$perf report
zsh: command not found: perf

[Of course we'll want to avoid using root.]

$sudo LD_LIBRARY_PATH=./usr/lib/x86_64-linux-gnu/ ./usr/bin/perf_4.19 report

Samples: 24K of event 'cycles:ppp', Event count (approx.): 19181550079
Overhead  Command         Shared Object        Symbol
   8.60%  lmi_cli_shared  libm-2.31.so         [.] expm1f64x
   6.24%  lmi_cli_shared  liblmi.so            [.] 
AccountValue::DecrementAVProportionally
   4.83%  lmi_cli_shared  liblmi.so            [.] Irc7702A::DetermineLowestBft
   4.46%  lmi_cli_shared  liblmi.so            [.] AccountValue::TxSetDeathBft
   2.84%  lmi_cli_shared  liblmi.so            [.] AccountValue::SurrChg
...
Cannot load tips.txt file, please install perf!

[Try the msw build--running wine as root just this once.
Donning my N95 mask and surgical gloves...]

$sudo LD_LIBRARY_PATH=./usr/lib/x86_64-linux-gnu/ ./usr/bin/perf_4.19 record -- 
sh -c 'wine /opt/lmi/bin/lmi_cli_shared --accept --data_path=/opt/lmi/data 
--selftest'                
[sudo] password for greg: 
wine: created the configuration directory '/root/.wine'
No protocol specified
...
0025:err:winediag:nodrv_CreateWindow Make sure that your X server is running 
and that $DISPLAY is set correctly.
Could not find Wine Gecko. HTML rendering will be disabled.
wine: configuration in L"/root/.wine" has been updated.

[/root/.wine: I'll destroy this chroot tonight]

Test speed:
  naic, no solve      : 7.047e-02 s mean;      69535 us least of  15 runs
  naic, specamt solve : 1.150e-01 s mean;     114303 us least of   9 runs
  naic, ee prem solve : 1.068e-01 s mean;     105734 us least of  10 runs
  finra, no solve     : 3.597e-02 s mean;      35363 us least of  28 runs
  finra, specamt solve: 7.554e-02 s mean;      74638 us least of  14 runs
  finra, ee prem solve: 7.068e-02 s mean;      69837 us least of  15 runs
[ perf record: Woken up 8 times to write data ]
sh: 1: /perf-read-vdso32: not found
Failed to read max cpus, using default of 4096
[ perf record: Captured and wrote 2.787 MB perf.data (65474 samples) ]
$sudo LD_LIBRARY_PATH=./usr/lib/x86_64-linux-gnu/ ./usr/bin/perf_4.19 report
Samples: 65K of event 'cycles:ppp', Event count (approx.): 34322852185
Overhead  Command         Shared Object             Symbol
   2.35%  lmi_cli_shared  perf-31985.map            [.] 0x000000000091b5aa
   1.20%  wineserver64    [kernel.kallsyms]         [k] do_syscall_64
   1.16%  lmi_cli_shared  perf-31985.map            [.] 0x000000000091b5b0
   1.07%  lmi_cli_shared  perf-31985.map            [.] 0x000000000091b5b2
   1.05%  lmi_cli_shared  ntdll.dll.so              [.] RtlSetLastWin32Error

[Not wonderful. But optimizing the native build helps the msw build.]

* Comments

The '.deb' approach feels unclean, but it does seem to work.
If it's worth cleaning up, there must be a way to deduce which
.deb files are required.

Running 'perf' as root is a security issue that can presumably
be solved by the technique here:
  https://www.kernel.org/doc/html/latest/admin-guide/perf-security.html

Messages like:
  Failed to read max cpus, using default of 4096
  Cannot load tips.txt file, please install perf!
imply that some part of perf is still missing.

This x86_64-pc-linux-gnu hotspot:
   8.60%  lmi_cli_shared  libm-2.31.so         [.] expm1f64x
looks up the symbol 'expm1f64x', so we needn't run 'addr2line'
to decode its address, as was necessary with a different approach
recently tried.