Re: project-find-file: switch to include non-tracked files

[Dmitry Gutov]

On 14.10.2021 09:37, Manuel Uberti wrote:
> On 14/10/21 02:47, Dmitry Gutov wrote:
> > Thanks for confirming.
> >
> > I've pushed the discussed change to master, to be released in Emacs 29 
> > (but of course you'll be able to install a updated version of 
> > project.el from ELPA much sooner).
>
> Thank you for the update. On Emacs 29 it works as expected on different 
> projects (including Emacs source directory)

Excellent.

> albeit one, which 
> unfortunately I can share much about because the customer owns the code.
>
> I can report the bug if you want, but I will not be able to share a much 
> detailed recipe to reproduce the error.

In general, but reports are better, but I don't want to give you the 
run-around. Since reported here, let's continue here.

> From 'emacs -Q' the only thing I 
> did is the following:
>
> (dolist (dir '(".clj-kondo" ".cpcache" ".gitlab" "node_modules"))
>    (add-to-list 'vc-directory-exclusion-list dir t))

As an aside, I can see what kind of problem this is solving, and this is 
a (meta) problem by itself. We need, like, ignores which are still 
honored when we don't use any other ignore instructions. Maybe a 
separate variable or something? AFAICS helm-ls-git doesn't have anything 
for this.

The above will continue to work until we try to move the listing logic 
to a backend-specific method. Then vc-directory-exclusion-list won't be 
referenced anymore. That's the main reason this feature is not in 
emacs-28 anyway.

Also note that 'find' gets a noticeable slowdown the more ignore entries 
we ask it to handle. Consider removing some entries from 
vc-directory-exclusion-list as well.

> Then C-u C-x p f in the customer's project and I got this:
>
> Debugger entered--Lisp error: (error "File listing failed: 
> ./shadow-cljs.edn\0./public/cs...")
>    error("File listing failed: %s" 
> "./shadow-cljs.edn\0./public/css/site.css\0./public/i...")
>    project--files-in-directory("~/7bridges/skuro/cockpit/" ("SCCS/" 
> "RCS/" "CVS/" "MCVS/" ".src/" ".svn/" ".git/" ".hg/" ".bzr/" "_MTN/" 
> "_darcs/" "{arch}/" ".clj-kondo/" ".cpcache/" ".gitlab/" "node_modules/"))
>    #f(compiled-function (dir) #<bytecode 
> 0x16d43be495cb8511>)("~/7bridges/skuro/cockpit/")
>    project-find-file-in(nil ("~/7bridges/skuro/cockpit/") (vc . 
> "~/7bridges/skuro/cockpit/") (4))
>    project-find-file((4))
>    funcall-interactively(project-find-file (4))
>    command-execute(project-find-file)
>
> Let me know if you want me to file a proper bug.

As a rough guess, perhaps some directories in this repo are not readable 
for you?

Try edebug-ing project--files-in-directory. When you reach the (zerop 
status) line, the status will be non-zero for some reason. And there you 
could examine the buffer contents (though they are likely to be quite 
large).

Another approach is to step through the method, then evaluate the 
constructed 'command' (perhaps call (message "%s" command) to make sure 
it's not truncated), then copy it to the terminal with appropriate 
escaping and see the full output and the error status returned. The 
output might include an explanation.

You might want to replace '-print0' with '-print' when running the 
command in the terminal, too, so that it's not just one long line.