bug#71789: 29.3; Handling of nested directories of ELPA packages

[Xiyue Deng]

Hi,

I'm opening a bug to follow up the discussion at [1].  Given an ELPA
package `elpafoo' that contains another directory `elpabar' that also
holds source files, e.g. like the structure below:

,----
| ~/.config/emacs/elpa/elpafoo/
| ~/.config/emacs/elpa/elpafoo/elpafoo.el
| ~/.config/emacs/elpa/elpafoo/elpabar
| ~/.config/emacs/elpa/elpafoo/elpabar/elpabar.el
`----

When installing through package.el, both elpafoo.el and
elpabar/elpabar.el will be byte-compiled, but only the root source path
`~/.config/emacs/elpa/elpafoo' is added to `load-path', without any of
the nested directories.

I wonder whether this is the intended behavior regarding `load-path'
handling.  If nested directories are added, it has the advantage to
allow more fine-grained source organization; as a downside, it may
unnecessarily add more unneeded stuff and processing overhead (as
Michael pointed out at [2]), and potentially break existing code,
e.g. auctex, which seems to vendor stuff in the `style/' directories but
not loaded so it's not causing any trouble yet.

So I wonder what is the stance of the Emacs maintainer on this subject,
or whether this is documented anywhere.  Also one more step: whether
there is a recommended layout for source code organization for ELPA
packages.

[1] https://lists.gnu.org/archive/html/emacs-devel/2024-05/msg00658.html
[2] https://lists.gnu.org/archive/html/emacs-devel/2024-05/msg01039.html



In GNU Emacs 29.3 (build 1, x86_64-pc-linux-gnu, GTK+ Version 3.24.38,
 cairo version 1.16.0) of 2024-05-20, modified by Debian built on sbuild
Windowing system distributor 'The X.Org Foundation', version 11.0.12101007
System Description: Debian GNU/Linux 12 (bookworm)

Configured using:
 'configure --build x86_64-linux-gnu --prefix=/usr
 --sharedstatedir=/var/lib --libexecdir=/usr/libexec
 --localstatedir=/var/lib --infodir=/usr/share/info
 --mandir=/usr/share/man --with-libsystemd --with-pop=yes
 
--enable-locallisppath=/etc/emacs:/usr/local/share/emacs/29.3/site-lisp:/usr/local/share/emacs/site-lisp:/usr/share/emacs/29.3/site-lisp:/usr/share/emacs/site-lisp
 --with-sound=alsa --without-gconf --with-mailutils
 --with-native-compilation --build x86_64-linux-gnu --prefix=/usr
 --sharedstatedir=/var/lib --libexecdir=/usr/libexec
 --localstatedir=/var/lib --infodir=/usr/share/info
 --mandir=/usr/share/man --with-libsystemd --with-pop=yes
 
--enable-locallisppath=/etc/emacs:/usr/local/share/emacs/29.3/site-lisp:/usr/local/share/emacs/site-lisp:/usr/share/emacs/29.3/site-lisp:/usr/share/emacs/site-lisp
 --with-sound=alsa --without-gconf --with-mailutils
 --with-native-compilation --with-cairo --with-x=yes
 --with-x-toolkit=gtk3 --with-toolkit-scroll-bars 'CFLAGS=-g -O2
 -ffile-prefix-map=/build/reproducible-path/emacs-29.3+1=. 
-fstack-protector-strong
 -Wformat -Werror=format-security -Wall' 'CPPFLAGS=-Wdate-time
 -D_FORTIFY_SOURCE=2' LDFLAGS=-Wl,-z,relro'

Configured features:
ACL CAIRO DBUS FREETYPE GIF GLIB GMP GNUTLS GPM GSETTINGS HARFBUZZ JPEG
JSON LCMS2 LIBOTF LIBSELINUX LIBSYSTEMD LIBXML2 M17N_FLT MODULES
NATIVE_COMP NOTIFY INOTIFY PDUMPER PNG RSVG SECCOMP SOUND SQLITE3
THREADS TIFF TOOLKIT_SCROLL_BARS TREE_SITTER WEBP X11 XDBE XIM XINPUT2
XPM GTK3 ZLIB

Important settings:
  value of $LANG: en_US.UTF-8
  value of $XMODIFIERS: @im=ibus
  locale-coding-system: utf-8-unix

Major mode: Lisp Interaction

Minor modes in effect:
  global-git-commit-mode: t
  magit-auto-revert-mode: t
  shell-dirtrack-mode: t
  windmove-mode: t
  rcirc-track-minor-mode: t
  server-mode: t
  subword-mode: t
  bug-reference-prog-mode: t
  whitespace-mode: t
  yas-minor-mode: t
  xclip-mode: t
  global-treesit-auto-mode: t
  treemacs-project-follow-mode: t
  treemacs-follow-mode: t
  treemacs-git-mode: t
  treemacs-fringe-indicator-mode: t
  corfu-terminal-mode: t
  corfu-popupinfo-mode: t
  corfu-echo-mode: t
  global-corfu-mode: t
  corfu-mode: t
  activities-tabs-mode: t
  activities-mode: t
  fido-vertical-mode: t
  icomplete-vertical-mode: t
  icomplete-mode: t
  fido-mode: t
  override-global-mode: t
  global-display-line-numbers-mode: t
  display-line-numbers-mode: t
  global-auto-revert-mode: t
  tooltip-mode: t
  global-eldoc-mode: t
  eldoc-mode: t
  show-paren-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  tool-bar-mode: t
  tab-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  column-number-mode: t
  line-number-mode: t
  transient-mark-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t

Load-path shadows:
/usr/share/emacs/site-lisp/elpa/debian-el-37.13/debian-autoloads hides 
/usr/share/emacs/site-lisp/elpa/gnuplot-0.8.1/debian-autoloads
/usr/share/emacs/site-lisp/elpa/magit-3.3.0/magit-section hides 
/usr/share/emacs/site-lisp/elpa/magit-section-3.3.0/magit-section

Features:
(shadow emacsbug git-rebase tabify ert-x python etags fileloop generator
java-ts-mode c-ts-common sh-script smie executable debian-copyright
debian-changelog-mode noutline outline shortdoc completion
debian-control-mode cl-print help-fns radix-tree tramp-cmds
gitignore-mode conf-mode tramp-container tramp-sh tramp-cache time-stamp
dockerfile-ts-mode yaml-ts-mode reporter debian-bts-control debian-bug
reveal mailalias goto-addr vterm tramp tramp-loaddefs trampver
tramp-integration files-x tramp-compat term ehelp vterm-module misearch
multi-isearch shr-color flow-fill matlab matlab-scan matlab-syntax
matlab-compat qp url-http url-gw url-auth url-queue url-cache mm-archive
sort gnus-cite mail-extr textsec uni-scripts idna-mapping ucs-normalize
uni-confusable textsec-check gnus-async gnus-bcklg gnus-ml magit-extras
eglot external-completion array jsonrpc ert ewoc debug backtrace
find-func xref pcase perl-mode face-remap magit-bookmark magit-submodule
magit-obsolete magit-blame magit-stash magit-reflog magit-bisect
magit-push magit-pull magit-fetch magit-clone magit-remote magit-commit
magit-sequence magit-notes magit-worktree magit-tag magit-merge
magit-branch magit-reset magit-files magit-refs magit-status magit
magit-repos magit-apply magit-wip magit-log which-func imenu magit-diff
smerge-mode diff git-commit log-edit add-log magit-core magit-autorevert
magit-margin magit-transient magit-process with-editor shell pcomplete
magit-mode transient format-spec magit-git magit-section magit-utils crm
dired-aux mule-util jka-compr gnus-topic cursor-sensor utf-7 nnfolder
gnus-demon nnml ezgnus gnus-delay gnus-draft gnus-agent gnus-srvr
gnus-score score-mode nnvirtual nntp gnus-cache gnus-msg gnus-art mm-uu
mml2015 mm-view mml-smime smime dig gnus-sum shr pixel-fill kinsoku
url-file svg dom nndraft nnmh gnus-group gnus-undo smtpmail gnus-start
gnus-dbus dbus xml gnus-cloud nnimap nnmail mail-source utf7 nnoo
gnus-spec gnus-int gnus-range message sendmail yank-media rfc822 mml
mml-sec mm-decode mm-bodies mm-encode mail-parse rfc2231 rfc2047 rfc2045
ietf-drums mailabbrev gmm-utils mailheader gnus-win gnus nnheader
gnus-util mail-utils range mm-util mail-prsvr windmove flyspell ispell
gnutls network-stream puny nsm epa-file epa derived epg rfc6068
epg-config rcirc parse-time iso8601 time-date term/xterm xterm comp
comp-cstr server cap-words superword subword vc-hg vc-git diff-mode
vc-bzr vc-src vc-sccs vc-svn vc-cvs vc-rcs log-view pcvs-util vc
vc-dispatcher bug-reference disp-table whitespace yasnippet-snippets
yasnippet cus-edit cus-start wid-edit init zenburn-theme xclip
treesit-auto treesit treemacs-project-follow-mode treemacs-follow-mode
treemacs-rendering treemacs-annotations treemacs-async treemacs-visuals
treemacs-fringe-indicator pulse color treemacs-workspaces treemacs-dom
treemacs-icons treemacs-themes treemacs-scope treemacs-core-utils
treemacs-logging treemacs-customization pfuture inline ht s hl-line dash
keychain-environment exec-path-from-shell corfu-terminal popon
corfu-popupinfo corfu-echo corfu compat activities-tabs activities
persist bookmark pp edmacro kmacro advice icomplete cus-load
flymake-proc flymake project compile text-property-search comint
ansi-osc ansi-color ring warnings icons thingatpt cl-extra help-mode
use-package use-package-ensure use-package-delight use-package-diminish
use-package-bind-key bind-key easy-mmode use-package-core
display-line-numbers autorevert filenotify
keychain-environment-autoloads treesit-auto-autoloads xclip-autoloads rx
info debian-el dired dired-loaddefs finder-inf package browse-url url
url-proxy url-privacy url-expand url-methods url-history url-cookie
generate-lisp-file url-domsuf url-util mailcap url-handlers url-parse
auth-source cl-seq eieio eieio-core cl-macs password-cache json subr-x
map byte-opt gv bytecomp byte-compile url-vars cl-loaddefs cl-lib rmc
iso-transl tooltip cconv eldoc paren electric uniquify ediff-hook
vc-hooks lisp-float-type elisp-mode mwheel term/x-win x-win
term/common-win x-dnd tool-bar dnd fontset image regexp-opt fringe
tabulated-list replace newcomment text-mode lisp-mode prog-mode register
page tab-bar menu-bar rfn-eshadow isearch easymenu timer select
scroll-bar mouse jit-lock font-lock syntax font-core term/tty-colors
frame minibuffer nadvice seq simple cl-generic indonesian philippine
cham georgian utf-8-lang misc-lang vietnamese tibetan thai tai-viet lao
korean japanese eucjp-ms cp51932 hebrew greek romanian slovak czech
european ethiopic indian cyrillic chinese composite emoji-zwj charscript
charprop case-table epa-hook jka-cmpr-hook help abbrev obarray oclosure
cl-preloaded button loaddefs theme-loaddefs faces cus-face macroexp
files window text-properties overlay sha1 md5 base64 format env
code-pages mule custom widget keymap hashtable-print-readable backquote
threads dbusbind inotify lcms2 dynamic-setting system-font-setting
font-render-setting cairo move-toolbar gtk x-toolkit xinput2 x multi-tty
make-network-process native-compile emacs)

Memory information:
((conses 16 4151836 235820)
 (symbols 48 38894 35)
 (strings 32 186143 40188)
 (string-bytes 1 6165556)
 (vectors 16 121009)
 (vector-slots 8 3140568 287319)
 (floats 8 1184 5120)
 (intervals 56 274782 3076)
 (buffers 984 145))