Re: project.el semantics

[Dmitry Gutov]

On 11/11/2015 07:22 PM, John Wiegley wrote:

> > It's supposed to be a generic replacement for the top-level EDE ede-project
> > class, more or less.
>
> Except I don't know what that is. :(

ede-project is defined in lisp/cedet/ede/base.el. I think its defclass 
declaration is all that you need to know for this discussion.

It's the top parent of any EDE project. If someone wants to write a 
command acting on the current project, and they want it to work on any 
kind of EDE project, they would have to use the info that ede-project 
provides.

> > Those are outside. Does "external to the project" sound better?
>
> Ah, I think so, yes. "Ancillary roots" might be even better.

I meant "external to the project" instead of "outside the project", of 
course. In the project-library-roots docstring. But you seem to mean a 
rename, right?

Do you really think "project-ancillary-roots" is better than 
"project-library-roots"? The word "ancillary" doesn't mean anything (or 
it can mean basically anything). "Library" at least evokes some 
associations.

> > Since you both don't understand the few sentences that seem clear to me, I'm
> > apparently an utter failure of a technical writer. Which is not terribly
> > surprising, considering it's my second language.
>
> I don't mean to criticize your writing. It's hard to speak at the level of
> precise specification.

Yet by now I've spent a gargantuan amount of time writing these emails, 
as well as comments and docstrings in project.el. If reading them 
doesn't make things clearer to other people, maybe I should take up 
another, more productive hobby.

I honestly expected an "I see what you mean", or at least some targeted 
questions about the sentences I've already written. Not "I still don't 
understand what X and Y means".

> > Not "within the project", but related to the project. Does the term
> > "library" sound familiar?
>
> OK, this is starting to make more sense now. So, you're saying that *within* a
> project there are both distinguished directories, and file subsets with common
> meaning;

A project consists of directories. Most often, it consists of just one 
directory, but there are odd configurations where it's spread over 
several directories (Stephen gave some examples). It's not hard to 
support that case, so it's "project roots" now, instead of "project root".

We've not reached the stage where we want to group different kinds of 
files within the project somehow, yet. But we have project-ignores, so 
you might say it splits files into two groups (ignored and not).

> and *outside* the project (say, in "/usr/include"), that are likewise
> distinguished directories and file subsets with common meaning.

Yes. And `project-library-roots-function' docstring mentions "headers 
search path, load path, class path".

> What we may want to do is avoid the concept "root" entirely, and talk instead
> about general "filesets": That is, every "project" would have 0 or more
> associated filesets, and a way to identify which project (perhaps multiple!)
> that a given buffer is associated with.

That suggestion seems orthogonal. Does "project library filesets" sound 
more clear to you than "project library roots" or "project library root 
directories"? Does the difference between "project library filesets" and 
"project filesets" become more apparent?

The names ending with "-roots" mean the elements of the returned lists 
are treated in a recursive fashion (everything within each of project 
roots, including subdirectories, belongs to the project).

> A fileset could be defined as:
>
>      Everything within a directory
>      Everything within a directory tree
>      Everything within a directory or tree matching a predicate
>
> In the most general case, a fileset is determined by whatever function the
> user provides in .dir-locals.el.

What function? Each user will have to write a function now?

I'm happy to discuss patches along these lines, but please note that 
you're proposing to drastically increase the API's complexity, and 
you're not solving any of what I see as the two immediate problems (and 
the "feature freeze" is the day after tomorrow):

- The disagreement around what "library roots" means (or library 
filesets, etc). And whether we should have that in the API.

- Dealing with "library roots" for projects with multiple source 
languages inside, in an automatic fashion, without a combinatoric 
explosion of project backends. This is the first FIXME in project.el 
talks about. Would you like to comment on it?

> Filesets in turn have an extensible set of attributes:
>
>      Is the file part of the project, or external to it?
>      Is it under version control?
>      Is it a build product?
>      Should it be searched, tagged, presented in various contexts, etc.?
>        (This itself might be an extensible sublist)

I'd expect to see a justification for each of these features.

"Under version control" - this question is answered by VC. Why would we 
have that in the project API?

"build product" - why not simply ignore those using project-ignores?

"Should it be searched, tagged, presented in various contexts" - that 
description sounds like a pony.

> This way we can talk in general terms, and not about concrete roots or
> directories. In the most abstract case, the notion of "project" should be
> entirely definable by the user, and may look nothing like what we presently
> have in mind.

The more flexible the API allows every project to me, the more trouble, 
in general, the consumers of the API are going to have with it.

Example one:

You want to have "filesets" instead of simply directories. We'll have a 
new data structure to decide on, and the API consumers will have figure 
out what filesets are, and how to work with them. Directories are 
trivial in comparison.

Two:

You want filesets to be allowed to include only immediate directory 
contents, or the contents of the whole directory tree.

That means that any consumer will have to handle both of these cases. 
E.g. xref-collect-matches will have to use two different code paths to 
handle both recursive and non-recursive traversals.

As we've discussed with Stephen previously, a "directory to traverse 
non-recursively" is equivalent to "directory to traverse recursively, 
but ignoring all its subdirectories", and the second part can be 
implemented via project-ignores (that lists all subdirectories of the 
directory in question). That results in just one code path in all consumers.

Stephen still wanted to support both recursive and non-recursive 
traversals explicitly, so if you both insist on that, I guess I will 
give in.

Three:

Allowing "Everything within a directory or tree matching a predicate" 
means, I believe, that searching through the fileset won't be possible 
to implement with just one find+grep call. If that involves calling the 
predicate at each directory level, that is.

> It's even possible that a buffer without an associated file, say
> a *Compilation* buffer, could momentarily be considered part of a project, and
> as a candidate for cross-buffer searching within that project.

And that feature would mean that one can't implement searching through 
the project with find+grep at all. Do you consider it valuable enough, 
to give up on that simplicity?