Re: How can I change the Makeconf file of R to not use hardcoded, wrong include paths when compiling a package with Rcpp

[zimoun]

Hi Moritz,

Hum? complicated issue... so it is more questions than concrete help. Sorry.

Currently, there is no easy path to achieve your goal, if I understand
correctly.
But that is an interesting question for improving the tools. :-)


On Tue, 14 Jan 2020 at 02:51, Moritz Lell <address@hidden> wrote:

> to reproduce a scientific project, I need a setup with an old R version
> (3.4.3), together with other, newer libraries. I created a profile, the
> manifest and package definitions are below. However, when loading the
> profile I cannot compile R packages with this version:

How many packages are required by your scientific project?
Only R?


> The last lines of the output of `install.packages("Rcpp_1.0.3.tar.gz",
> repos = NULL)` (I have to manually download/use the miniCRAN package
> because of SSL errors while downloading via R but that is another question)

I am not sure that mixing packages installed by Guix and other by R
itself do help to clean an unreproducible project, IMHO.
To me, packaging all your needs with Guix seems better. You should end
up with your own package definitions living in a Git repo (channel)
and a manifest file describing how to rebuild everything.

However, the issue is: you use an old R version with recent packages.
Not usual. ;-)

The naive solution is to re-write the dependency graph by replacing
the current r-minimal@3.6.2 with the old one r-minimal@3.4.3. But it
does not work. Sadly! The implicit inputs cannot be re-written on the
fly with the '--with-inputs' option.


Now, I list some steps that you have already done just in case others
want to reproduce and/or help. :-)

(Note the Git commit commands are launched where the Git repo was
cloned. And the Guix command are launched in /tmp/t.)


1. Find the version 3.4.3 in Guix:

     git log --oneline | grep r-minimal

--8<---------------cut here---------------start------------->8---
d22546cf80 gnu: r-minimal: Update to 3.5.0.
3146aab7a8 gnu: r-minimal: Add tcl/tk support.
c43ea99b39 gnu: r-minimal: Update to 3.4.4.
a768e41523 gnu: r-minimal, r: Update to 3.4.4.
9a7b578463 gnu: r-minimal: Add bash to inputs.
0fa4702c04 gnu: r-minimal: Respect SOURCE_DATE_EPOCH in srcfile.
cbe1314a7e gnu: Update r-minimal to 3.4.3.
1344deb937 gnu: r-minimal: Update to 3.4.2.
ebbb6301a3 gnu: r-minimal: Update to 3.4.1.
174fbd5f3b gnu: r-minimal: Remove timestamp from man page.
bd3a184613 gnu: r-minimal: Do not compress serialized files.
a8cd352304 gnu: r-minimal: Work around failure to embed reference to "which".
269504a797 gnu: python-pbr-minimal: Update to 3.0.1
a71d769d1e gnu: r-minimal: Update to 3.4.0.
60c9190e21 gnu: r-minimal: Fix remaining reproducibility problems.
2d7c4ae3ee gnu: r: Rename to r-minimal.
--8<---------------cut here---------------end--------------->8---


Well, we need any commit before the "Update to 3.4.4." (a768e41523),
so for example the parent:

  git log --pretty=%P -n 1 a768e41523

is the commit: 38ef437b0c08d8568d6cb1cb0ff1f5c9a66336dd.


2. Try the time machine with the commit
38ef437b0c08d8568d6cb1cb0ff1f5c9a66336dd.
Possible resources consuming... everything will be compiled as it was
in Guix at the time (March 2018) so some substitutes are not available
any more.

   guix time-machine --commit=38ef437b0c08d8568d6cb1cb0ff1f5c9a66336dd \
      -- environment --container --ad-hoc r-minimal -- R --version


3. In the meantime, let compile the version 3.4.3 with the current
Guix (commit 2aeca24).

  git show 38ef437b0c08d8568d6cb1cb0ff1f5c9a66336dd:gnu/packages/statistics.scm
\
           > /tmp/t/old-statistics.scm
  # edit the module name
  guix build -L . r-minimal@3.4.3

and the output is: /gnu/store/3wagrrl9gzb5p1an9i5xbm9kp3km4zim-r-minimal-3.4.3.


Compared to your output
gnu/store/d64gfblqc5b9gmwj3rbl8nx3zwbf973b-r-minimal-3.4.3, one
difference is the tzdata package which is different, I guess.


4. Speaking about reproducibility, all the toolchain matters. :-)

  guix time-machine --commit=14c272d5ce --  build -L . r-minimal@3.4.3

outputs /gnu/store/n0py7ixdqkim1lx9iis6xlr2mn7l897f-r-minimal-3.4.3.


So for example compare:

    guix time-machine --commit=2aeca24 \
      -- environment --ad-hoc gcc-toolchain -- gcc --version

    guix time-machine --commit=14c272d5ce \
      -- environment --ad-hoc gcc-toolchain -- gcc --version

And let aside the grafting system. ;-)


To be concrete, compare

  # GCC 8.3.0
  guix time-machine --commit=14c272d5ce --  build -L . r-minimal@3.4.3

# -> /gnu/store/n0py7ixdqkim1lx9iis6xlr2mn7l897f-r-minimal-3.4.3

  guix time-machine --commit=14c272d5ce \
    --  environment --container -L . --ad-hoc r-minimal@3.4.3 -- R --version


  # GCC 9.2.0
  guix time-machine --commit=2aeca24 --  build -L . r-minimal@3.4.3

# -> /gnu/store/3wagrrl9gzb5p1an9i5xbm9kp3km4zim-r-minimal-3.4.3

  guix time-machine --commit=2aeca24 \
    --  environment --container -L . --ad-hoc r-minimal@3.4.3 -- R --version


Well, my point is just to be clear about what needs to be fixed to end
up with something reproducible.



> The problem lies in the file
> /gnu/store/d64gfblqc5b9gmwj3rbl8nx3zwbf973b-r-minimal-3.4.3/etc/Makeconf

You mean:
/gnu/store/d64gfblqc5b9gmwj3rbl8nx3zwbf973b-r-minimal-3.4.3/lib/R/etc/Makeconf
?


> CPPFLAGS = -I/usr/local/include

Hum? need more investigations... I have no idea...


> CPPFLAGS is set to the wrong, hardcoded value, but for example FLIBS is
> set correctly. How can I install R such that CPPFLAGS refers to the
> /lib/include directory of my profile or doesn't refer to anything?

It appears to me odd that it is hard coded...


> I can define the PKG_CXXFLAGS environment variable that can be used to
> insert additional header locations for R compilation, but unfortunately
> they are inserted after the R default locations. But I can use it if the
> R default is set to nothing. I've also tested setting CPLUS_INCLUDE_PATH
> and PKG_CPPFLAGS, to no avail.

Instead, I propose to overwite the r-rcpp package by adding an
explicit reference to r-minimal and so to be able to use the option
'--with-input'.

Something like:

--8<---------------cut here---------------start------------->8---
(define-module (explicit-pkgs)
  #:use-module ((guix licenses) #:prefix license:)
  #:use-module (guix packages)
  #:use-module ((gnu packages cran) #:select (r-rcpp))
  #:use-module (old-statistics))        ;need the symbol r-minimal
;;; #:use-module (gnu packages statistics) ;or here


(define-public r-rcpp-explicit
  (package
    (inherit r-rcpp)
    (version "mine")
    (inputs
     `(("r-minimal" ,r-minimal)))))
--8<---------------cut here---------------end--------------->8---

Then,

    guix environment --container \
       -L . --with-input=r-minimal=r-minimal@3.4.3 \
      --ad-hoc r-minimal r-rcpp@mine \
      -- R

should spawn a R prompt using the version 3.4.3 with Rcpp inside.

But it is not working properly:

--8<---------------cut here---------------start------------->8---
> library(Rcpp)
Error in readRDS(pfile) :
  cannot read workspace version 3 written by R 3.6.2; need R 3.5.0 or newer
--8<---------------cut here---------------end--------------->8---

I do not know why... need more investigations. :-)


I hope the strategy makes sense.


Well,
 a) I do not know if it the good strategy; it is what I am thinking of;
 b) something easier is maybe possible;
 c) today, there is a discussion on IRC to see how it should be
possible to add "something" for changing the R version (e.g., see how
Python packages do with 2 vs 3).

What do you think?


All the best,
simon