Re: [PATCH] Project out of sources compilation

[Ergus]

Hi Dmitry:

On Sun, Mar 31, 2024 at 05:41:24AM +0300, Dmitry Gutov wrote:
> Hi Ergus,
>
> On 27/03/2024 18:38, Ergus wrote:
>
> > Here I attach a very simple patch to allow project out of sources
> > compilation and project build detection.
> >
> > So far I already implemented a very primitive backend [1] for project.el
> > using these features to detect and compile autotools and cmake projects;
> > and, at some point I will propose it for elpa.
> >
> > But I need to work a bit more on it; and, of course, the functionalities
> > in the attached patch need to go to vanilla this way or some other
> > equivalent.
>
> I originally suggested a variable that stores a relative file name. 
> But I see that you prefer the fetching of this file name to be more 
> dynamic.
Yes, that's actually the point. In out of sources compilation there is a
possibility that there will be more than one subdirectory to build, in
that case the user may be asked at least once which one to choose. This
is also valid to detect the `compile-commands.json' which may have
different version in each of the build-dir candidates.

https://github.com/Ergus/project-multi-mode/blob/c679d73a26b162e4158f6451027ca94e33faca0b/project-multi-mode.el#L109

> Why don't we just make it a function variable, at least at first?
Because I wanted somehow to follow the project.el schema of using a
generic that the backend could optionally define transparently.

And make that definition independent from the user config variable, so,
the user defined variable takes preference over the generic search
performed by the backend which if not defined automatically goes to
project root. Somehow adding a custom variable I think interferes with
the idea that the backend provides the functionality... But I may be
wrong.

> Call it 'project-compile-dir`, which would be settable to a string 
> (constant value, to be customized through dir-locals), or a function 
> that either accepts the current project value, or is simply called 
> "inside project root" (with default-directory bound to that value).
I will try this again, but I had some issues before.

> Alternatively, if you see a need to add additional methods 
> ('project-compile-command'?), it could become a new hook and a new 
> facility (e.g. project-build-functions) which could be structures 
> similarly to project.el, i.e. the hook returns a "project builder" 
> value, and the value has some methods that it can defined/override. 
> Then different build tool backends could be mixed with different 
> project backends more easily.
I would prefer avoid more abstraction layers in the api. We actually
need some `project-compile-command' like function, but structuring it
with generics and more layer is IMHO adding too much complexity for a
public api... Isn't there a simpler alternative?

> So far I only see one additional method, though, and the constructed 
> compile-command value is relatively simple. Curious to see whether 
> this will get more complex with additional targets.
I added one method and with that I already support autotools and cmake.

> As far as the redefinition of 'project-name' goes, perhaps we'll allow 
> project-vc-name to be a function as well? Or decide on some other way 
> to assign a build tool the name to a project.
In project-name I actually only changed the docstring, I didn't change
anything else.

> > Best,
> > Ergus
> >
> > [1] 
> > https://github.com/Ergus/project-multi-mode/blob/master/project-multi-mode.el
> >
> >
> >
> > On Tue, Mar 19, 2024 at 07:36:35PM +0100, Ergus wrote:
> > > Hi Dmitry:
> > >
> > > Here I attach a very simple approach to add support for out of sources
> > > compilation in project.el
> > >
> > > This is just a POC, but to have some starting point to discuss.
> > >
> > > Best,
> > > Ergus
> > >
> > >
> > >
> > > On Sun, Mar 17, 2024 at 06:47:43PM +0100, Ergus wrote:
> > > > On Sun, Mar 17, 2024 at 12:36:44PM +0100, Augusto Stoffel wrote:
> > > > > On Sat, 16 Mar 2024 at 14:12, Ergus wrote:
> > > > >
> > > > > > 5. Project local variables (a man can dream, a man can dream)
> > > > > >
> > > > > > There are some situations where we want to have variables 
> > > > > > shared among a
> > > > > > project. (i.e some output directory, logging option when executing,
> > > > > > flags, environment variables).
> > > > > >
> > > > > > At the moment these options work partially by using directory
> > > > > > variables. If we have the concept of a "project", maybe it 
> > > > > > is logical to
> > > > > > have some sort of project scope concept, specially for 
> > > > > > projects sharing
> > > > > > a common root.
> > > > >
> > > > > For Emacs variables, why you think dir-local variables only works
> > > > > “partially”?
> > > > >
> > > > > As to environment variables, they are generally assumed to be global in
> > > > > Emacs and it goes a bit against the grain to make them buffer-local /
> > > > > project dependent.  Nonetheless, there are packages for that (one is
> > > > > buffer-env, which I wrote) and they work fine (at least for me).
> > > > They work for one directory, but when the project includes multiple
> > > > roots (project-external-roots) the variables are somehow missing just
> > > > when jumping with xref to a definition.
> > > >
> > > > However, we can ignore this as I mentioned before.
> > > >
> > > > > > For example vs-code adds a subdirectory with project 
> > > > > > variables that the
> > > > > > user (but also any plugin) can refer to in the project's scope.
> > > > >
> > > > > Is this meant to store editor configuration or environment 
> > > > > variables?  I
> > > > > guess turning VSCode configuration variables into Emacs alternatives
> > > > > would be tricky.
> > > >
> > > > No please. No interaction-importing from other editors will be
> > > > maintainable in the long path.
> > > >
> > > > > But if this is about env vars and the directory
> > > > > organization is sensible, then one could configure buffer-env to
> > > > > interoperate.
> > > >
> > > > This is it. The idea is to extend project.el in order to bring more
> > > > project-oriented features.
> > > >
> > > > Lets say:
> > > >
> > > > The user (or project.el backend) could specify the `build` directory and
> > > > the environment that needs to be set to execute the program from the
> > > > build or bin directory.
> > > >
> > > > So with a simple modification of dir-locals.el the user may be capable
> > > > to run M-x compile (or project-compile), M-x gud-gdb... or being more
> > > > ambitious M-x project-generate (to call autogen.sh and ./configure or
> > > > cmake depending of the project-backend)
> > > >
> > > > For sure we wont cover the 100% of the projects, but wuth automake and
> > > > autotools we will be very close (and in the future someone could add
> > > > other backends for it)
> >
> > > commit cfa10dfeac01ae25a7acd6857fae0b377b625779
> > > Author: Jimmy Aguilar Mena <kratsbinovish@gmail.com>
> > > Date:   Tue Mar 19 17:57:25 2024 +0100
> > >
> > >    Add basic project-build-dir implementation
> > >
> > >    This is just a dummy prototype to discuss about alternatives.
> > >
> > > diff --git a/lisp/progmodes/project.el b/lisp/progmodes/project.el
> > > index ac18aceadcf..0ca4ca85566 100644
> > > --- a/lisp/progmodes/project.el
> > > +++ b/lisp/progmodes/project.el
> > > @@ -51,6 +51,9 @@
> > > ;; files inside the root must not be considered a part of it).  It
> > > ;; should be consistent with `project-files'.
> > > ;;
> > > +;; `project-build-dir' can be overridden if the project backend has some
> > > +;; extra information about the project build directory.
> > > +;;
> > > ;; This list can change in future versions.
> > > ;;
> > > ;; Transient project:
> > > @@ -208,6 +211,12 @@ project-prompter
> > >   :group 'project
> > >   :version "30.1")
> > >
> > > +(defcustom project-build-dir nil
> > > +  "Build directory for current project."
> > > +  :type 'directory
> > > +  :safe t
> > > +  :version "30.1")
> > > +
> > > ;;;###autoload
> > > (defun project-current (&optional maybe-prompt directory)
> > >   "Return the project instance in DIRECTORY, defaulting to 
> > > `default-directory'.
> > > @@ -289,8 +298,30 @@ project-external-roots
> > > headers search path, load path, class path, and so on."
> > >   nil)
> > >
> > > +(cl-defgeneric project-build-dir (_project)
> > > +  "Return build directory of the current PROJECT.
> > > +
> > > +This function is intended to be defined by the backend when possible.
> > > +Otherwise this returns nil and the `project-compile' command will be
> > > +called in the project-root.
> > > +If the user defines the directory-local variable `project-build-dir' it
> > > +will have precedence over the result of this function."
> > > + nil)
> > > +
> > > +(defun project-get-build-dir (project)
> > > +  "Return build directory of the current PROJECT.
> > > +If the variable `project-build-dir' is defined, this function returns
> > > +it, otherwise it returns the project root.  If the defined path is
> > > +relative, this expands it relatively to the project's root"
> > > +  (let ((dir (or project-build-dir
> > > +                 (project-build-dir project)
> > > +                 (project-root project)))) ;; I assume 
> > > project-root is absolute
> > > +    (if (file-name-absolute-p dir)
> > > +        dir
> > > +      (expand-file-name dir (project-root project)))))
> > > +
> > > (cl-defgeneric project-name (project)
> > > -  "A human-readable name for the project.
> > > +  "A human-readable name for the PROJECT.
> > > Nominally unique, but not enforced."
> > >   (file-name-nondirectory (directory-file-name (project-root project))))
> > >
> > > @@ -1391,7 +1422,7 @@ project-compile
> > >   "Run `compile' in the project root."
> > >   (declare (interactive-only compile))
> > >   (interactive)
> > > -  (let ((default-directory (project-root (project-current t)))
> > > +  (let ((default-directory (project-get-build-dir (project-current t)))
> > >         (compilation-buffer-name-function
> > >          (or project-compilation-buffer-name-function
> > >              compilation-buffer-name-function)))