bug#66676: 29.1; Should some aspects of shr rendering be configurable

[Rahguzar]

The background of this email is that I recently spent some time trying
to make reading mathematics articles on Wikipedia using eww more
pleasant for myself. The current results of that effort are located at
https://codeberg.org/rahguzar/eww-hacks/src/branch/main/eww-hacks.el

To see the difference open e.g.
https://en.wikipedia.org/wiki/Cyclotomic_polynomial without and without
that file loaded.

So I wanted to ask if some of the changes to shr might be useful to have
in Emacs itself. The changes ordered from most likely to be useful to
least likely (in my opinion) are as follows:

1) Using `visual-line-mode` for line wrapping. I think this is more
natural for html and makes resizing windows work more nicely.

2) Not introducing line-breaks for every image. Wikipedia uses small
inline images quite extensively for mathematical notation and shr breaks
line at each of them which makes the text quite hard to read. This line
breaking is scattered at three different places:

   a) shr-tag-image which inserts a newline before every image.

   b) shr-insert which inserts newline after the image by checking if
   the point is after an image.

   c) shr-put-image which inserts a newline after an image if the image
   is large enough.

   This scattered insertion of newlines makes it quite difficult to
   advice shr to not add these newlines. I think c) is good behavior but
   a) and b) are not. However to preserve backward compatibility not
   doing a) and b) should be done conditioned on a new variable.

3) shr uses 0.2 and -0.2 as value of raise property for superscripts and
subscripts. I think it makes sense to make these values customizable.
For me 0.2 for superscripts was too low and 0.4 worked much better.

4) shr uses 100 as the value of :ascent when creating an image. For me
this makes the inline images appear off centered. I fiddled with it a
bit and found 60 to be a good value for me. I wonder if it makes sense
to makes the value of :ascent to be customizable? My instinct is no
since I think the correct value can probably be computed from image
height and the average height of default face.

Finally one point related to eww directly. I am using a package on melpa
that provides outline support for eww but it does it by insert special
characters at the beginning of headings and then setting
`outline-regexp` to find them. Since Emacs 29 introduces
`outline-search-function` I think it makes sense to set one for eww that
searches for headings using text-properties and this doesn't require any
extra marks in the buffer.

I also have a couple of questions of how to hack on eww to make it a bit
better for my personal use:

1) I think reading Wikipedia math articles is quite pleasant now except
for proofs (there are a couple of those in the article linked above).
They are coded as an html table and I think to make width calculations
work shr doesn't render images and just drops them after the table.
However this makes the proof unreadable. Since in this cases the table
has only cell, is there an easy way to treat that cases specially and
not try to render it as a table? (I have no experience with html
rednering).

2) Is there an easy way to make eww not reload the page after clicking
on a link that points to a location on the same page?

Thanks,
Rahguzar



In GNU Emacs 29.1 (build 1, x86_64-suse-linux-gnu, GTK+ Version 3.24.38,
 cairo version 1.18.0) of 2023-10-08 built on i02-ch2a
System Description: openSUSE Tumbleweed

Configured using:
 'configure --host=x86_64-suse-linux-gnu --build=x86_64-suse-linux-gnu
 --program-prefix= --disable-dependency-tracking --prefix=/usr
 --exec-prefix=/usr --bindir=/usr/bin --sbindir=/usr/sbin
 --sysconfdir=/etc --datadir=/usr/share --includedir=/usr/include
 --libdir=/usr/lib64 --libexecdir=/usr/libexec --localstatedir=/var
 --sharedstatedir=/var/lib --mandir=/usr/share/man
 --infodir=/usr/share/info --with-pgtk --with-native-compilation=aot
 --with-cairo --with-libotf --with-jpeg --with-tiff --with-gif
 --with-png --with-rsvg --with-xft --with-xml2 --with-dbus --with-sound
 --with-json --with-mailutils --with-gnutls --with-tree-sitter
 
--enable-locallisppath=/usr/share/emacs/29.1/site-lisp:/usr/share/emacs/site-lisp
 'CFLAGS=-O2 -Wall -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=3
 -fstack-protector-strong -funwind-tables -fasynchronous-unwind-tables
 -fstack-clash-protection -Werror=return-type -flto=auto'
 LDFLAGS=-Wl,-O2'

Configured features:
CAIRO DBUS FREETYPE GIF GLIB GMP GNUTLS GSETTINGS HARFBUZZ JPEG JSON
LIBOTF LIBSELINUX LIBSYSTEMD LIBXML2 MODULES NATIVE_COMP NOTIFY INOTIFY
PDUMPER PGTK PNG RSVG SECCOMP SOUND SQLITE3 THREADS TIFF
TOOLKIT_SCROLL_BARS TREE_SITTER XIM GTK3 ZLIB

Important settings:
  value of $LANG: en_US.UTF-8
  locale-coding-system: utf-8-unix

Major mode: mu4e:main

Minor modes in effect:
  pdf-occur-global-minor-mode: t
  global-evil-surround-mode: t
  evil-surround-mode: t
  eros-mode: t
  recentf-mode: t
  save-place-mode: t
  global-so-long-mode: t
  which-key-mode: t
  savehist-mode: t
  better-jumper-mode: t
  better-jumper-local-mode: t
  global-company-mode: t
  company-mode: t
  vertico-mode: t
  nerd-icons-completion-mode: t
  marginalia-mode: t
  evil-goggles-mode: t
  evil-escape-mode: t
  evil-snipe-override-mode: t
  evil-snipe-mode: t
  gcmh-mode: t
  winner-mode: t
  smartparens-global-mode: t
  ws-butler-global-mode: t
  undo-fu-session-global-mode: t
  undo-fu-mode: t
  satchel-mode: t
  mu4e-search-minor-mode: t
  global-hl-line-mode: t
  hl-line-mode: t
  mu4e-update-minor-mode: t
  mu4e-context-minor-mode: t
  dirvish-override-dired-mode: t
  server-mode: t
  pixel-scroll-precision-mode: t
  repeat-mode: t
  evil-mode: t
  evil-local-mode: t
  +popup-mode: t
  +modeline-global-mode: t
  +modeline-mode: t
  global-ligature-mode: t
  ligature-mode: t
  override-global-mode: t
  general-override-mode: t
  global-eldoc-mode: t
  show-paren-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  window-divider-mode: t
  buffer-read-only: t
  size-indication-mode: t
  line-number-mode: t
  transient-mark-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  abbrev-mode: t

Load-path shadows:
/home/azeem/.emacs.d/.local/straight/build-29.1/emacsql/emacsql-sqlite-builtin 
hides 
/home/azeem/.emacs.d/.local/straight/build-29.1/emacsql-sqlite-builtin/emacsql-sqlite-builtin
/home/azeem/.emacs.d/.local/straight/build-29.1/citar-embark/citar-embark hides 
/home/azeem/.emacs.d/.local/straight/build-29.1/citar/citar-embark
/home/azeem/.emacs.d/.local/straight/build-29.1/mu4e/mu4e-view hides 
/usr/share/emacs/site-lisp/mu4e/mu4e-view
/home/azeem/.emacs.d/.local/straight/build-29.1/mu4e/mu4e-search hides 
/usr/share/emacs/site-lisp/mu4e/mu4e-search
/home/azeem/.emacs.d/.local/straight/build-29.1/mu4e/mu4e-org hides 
/usr/share/emacs/site-lisp/mu4e/mu4e-org
/home/azeem/.emacs.d/.local/straight/build-29.1/mu4e/mu4e-contacts hides 
/usr/share/emacs/site-lisp/mu4e/mu4e-contacts
/home/azeem/.emacs.d/.local/straight/build-29.1/mu4e/mu4e-config hides 
/usr/share/emacs/site-lisp/mu4e/mu4e-config
/home/azeem/.emacs.d/.local/straight/build-29.1/mu4e/mu4e hides 
/usr/share/emacs/site-lisp/mu4e/mu4e
/home/azeem/.emacs.d/.local/straight/build-29.1/mu4e/mu4e-window hides 
/usr/share/emacs/site-lisp/mu4e/mu4e-window
/home/azeem/.emacs.d/.local/straight/build-29.1/mu4e/mu4e-vars hides 
/usr/share/emacs/site-lisp/mu4e/mu4e-vars
/home/azeem/.emacs.d/.local/straight/build-29.1/mu4e/mu4e-update hides 
/usr/share/emacs/site-lisp/mu4e/mu4e-update
/home/azeem/.emacs.d/.local/straight/build-29.1/mu4e/mu4e-speedbar hides 
/usr/share/emacs/site-lisp/mu4e/mu4e-speedbar
/home/azeem/.emacs.d/.local/straight/build-29.1/mu4e/mu4e-server hides 
/usr/share/emacs/site-lisp/mu4e/mu4e-server
/home/azeem/.emacs.d/.local/straight/build-29.1/mu4e/mu4e-query-items hides 
/usr/share/emacs/site-lisp/mu4e/mu4e-query-items
/home/azeem/.emacs.d/.local/straight/build-29.1/mu4e/mu4e-obsolete hides 
/usr/share/emacs/site-lisp/mu4e/mu4e-obsolete
/home/azeem/.emacs.d/.local/straight/build-29.1/mu4e/mu4e-notification hides 
/usr/share/emacs/site-lisp/mu4e/mu4e-notification
/home/azeem/.emacs.d/.local/straight/build-29.1/mu4e/mu4e-modeline hides 
/usr/share/emacs/site-lisp/mu4e/mu4e-modeline
/home/azeem/.emacs.d/.local/straight/build-29.1/mu4e/mu4e-message hides 
/usr/share/emacs/site-lisp/mu4e/mu4e-message
/home/azeem/.emacs.d/.local/straight/build-29.1/mu4e/mu4e-mark hides 
/usr/share/emacs/site-lisp/mu4e/mu4e-mark
/home/azeem/.emacs.d/.local/straight/build-29.1/mu4e/mu4e-main hides 
/usr/share/emacs/site-lisp/mu4e/mu4e-main
/home/azeem/.emacs.d/.local/straight/build-29.1/mu4e/mu4e-lists hides 
/usr/share/emacs/site-lisp/mu4e/mu4e-lists
/home/azeem/.emacs.d/.local/straight/build-29.1/mu4e/mu4e-icalendar hides 
/usr/share/emacs/site-lisp/mu4e/mu4e-icalendar
/home/azeem/.emacs.d/.local/straight/build-29.1/mu4e/mu4e-helpers hides 
/usr/share/emacs/site-lisp/mu4e/mu4e-helpers
/home/azeem/.emacs.d/.local/straight/build-29.1/mu4e/mu4e-headers hides 
/usr/share/emacs/site-lisp/mu4e/mu4e-headers
/home/azeem/.emacs.d/.local/straight/build-29.1/mu4e/mu4e-folders hides 
/usr/share/emacs/site-lisp/mu4e/mu4e-folders
/home/azeem/.emacs.d/.local/straight/build-29.1/mu4e/mu4e-draft hides 
/usr/share/emacs/site-lisp/mu4e/mu4e-draft
/home/azeem/.emacs.d/.local/straight/build-29.1/mu4e/mu4e-contrib hides 
/usr/share/emacs/site-lisp/mu4e/mu4e-contrib
/home/azeem/.emacs.d/.local/straight/build-29.1/mu4e/mu4e-context hides 
/usr/share/emacs/site-lisp/mu4e/mu4e-context
/home/azeem/.emacs.d/.local/straight/build-29.1/mu4e/mu4e-compose hides 
/usr/share/emacs/site-lisp/mu4e/mu4e-compose
/home/azeem/.emacs.d/.local/straight/build-29.1/mu4e/mu4e-bookmarks hides 
/usr/share/emacs/site-lisp/mu4e/mu4e-bookmarks
/home/azeem/.emacs.d/.local/straight/build-29.1/mu4e/mu4e-actions hides 
/usr/share/emacs/site-lisp/mu4e/mu4e-actions

Features:
(macrostep-c cmacexp evil-collection-macrostep macrostep
cc-mode-expansions smartparens-c cc-mode cc-fonts cc-guess cc-menus
cc-cmds cc-styles cc-align cc-engine org-eldoc cdlatex toc-org evil-org
org-indent ol-bibtex shadow emacsbug vertico-directory time
inspirehep-pdf companion-mode inspirehep url-queue
evil-collection-reftex reftex-toc reftex-cite reftex-ref reftex-parse
reftex reftex-loaddefs reftex-vars pdf-annot facemenu pdf-outline
pdf-links pdf-roll pdf-loader pdf-occur evil-collection-tablist tablist
tablist-filter semantic/wisent/comp semantic/wisent
semantic/wisent/wisent semantic/util-modes semantic/util semantic
semantic/tag semantic/lex semantic/fw mode-local cedet pdf-isearch
pdf-misc evil-collection-pdf pdf-history pdf-tools
evil-collection-package-menu doom-packages package url-handlers pdf-view
pdf-cache pdf-info tq pdf-util pdf-macs image-mode exif char-fold
guess-language flyspell link-hint ffap goto-addr avy company-ispell
company-yasnippet company-dabbrev delsel adaptive-wrap jit-spell ispell
face-remap evil-collection-proced proced gnus-icalendar org-capture
icalendar diary-lib diary-loaddefs smerge-mode diff gnus-async
gnus-bcklg gnus-ml evil-collection-indent use-package-lint consult-imenu
evil-collection-consult consult evil-collection-vc-git vc-git
evil-collection-diff-mode diff-mode vc-dispatcher files-x
evil-collection-xref xref find-dired jka-compr auto-minor-mode
disp-table whitespace bicycle hideshow evil-embrace evil-surround
embrace expand-region text-mode-expansions the-org-mode-expansions
org-element org-persist xdg org-id org-refile avl-tree generator
er-basic-expansions expand-region-core expand-region-custom eros
highlight-quoted rainbow-delimiters highlight-numbers parent-mode
display-line-numbers hl-todo evil-collection-elisp-mode elisp-mode
recentf tree-widget saveplace evil-collection-so-long so-long
evil-collection-helpful helpful cc-langs cc-vars cc-defs
evil-collection-imenu imenu trace evil-collection-edebug edebug
evil-collection-debug debug backtrace info-look evil-collection-info
info f f-shortdoc help-fns radix-tree evil-collection-elisp-refs
elisp-refs s sort gnus-cite smiley shr-color mm-archive mail-extr
textsec uni-scripts idna-mapping ucs-normalize uni-confusable
textsec-check qp nndraft nnmh nnfolder cursor-sensor vertico-repeat
epa-file network-stream nsm gnus-agent gnus-srvr gnus-score score-mode
nnvirtual gnus-msg nntp gnus-cache hide-mode-line evil-collection-help
descr-text evil-collection-which-key which-key savehist better-jumper
company-capf company evil-collection-vertico vertico orderless
nerd-icons-completion marginalia compat evil-goggles pulse color
evil-easymotion evil-escape evil-snipe autorevert filenotify gcmh winner
smartparens-config smartparens-text smartparens loadhist dash ws-butler
undo-fu-session undo-fu satchel ibuf-ext evil-collection-ibuffer ibuffer
ibuffer-loaddefs desktop frameset evil-collection-mu4e mu4e mu4e-org
evil-collection-org smartparens-org org ob ob-tangle ob-ref ob-lob
ob-table ob-exp org-macro org-src ob-comint org-pcomplete pcomplete
org-list org-footnote org-faces org-entities noutline outline
ob-emacs-lisp ob-core ob-eval org-cycle org-table ol org-fold
org-fold-core org-keys oc-natbib oc-csl bibtex oc-biblatex oc
org-loaddefs find-func org-version org-compat org-macs mu4e-notification
notifications mu4e-main mu4e-view gnus-art mm-uu mml2015 mm-view
mml-smime smime gnutls dig gnus-sum gnus-group gnus-undo gnus-start
gnus-dbus dbus gnus-cloud nnimap nnmail mail-source utf7 nnoo parse-time
iso8601 gnus-spec gnus-int gnus-range gnus-win evil-collection-gnus gnus
nnheader range evil-collection-calendar cal-menu calendar cal-loaddefs
mu4e-headers mu4e-compose mu4e-draft mu4e-actions smtpmail mu4e-search
mu4e-lists mu4e-bookmarks mu4e-mark mu4e-message shr pixel-fill kinsoku
url-file svg xml dom browse-url url url-proxy url-privacy url-expand
url-methods url-history url-cookie generate-lisp-file url-domsuf
url-util url-parse auth-source json map url-vars flow-fill mule-util
hl-line mu4e-contacts mu4e-update mu4e-folders mu4e-context
mu4e-query-items mu4e-server mu4e-modeline mu4e-vars mu4e-helpers
mu4e-config mu4e-window evil-collection-bookmark bookmark ido message
sendmail mailcap yank-media puny dirvish transient format-spec eieio
eieio-core evil-collection-dired dired dired-loaddefs rfc822 mml mml-sec
password-cache evil-collection-epa epa epg rfc6068 epg-config gnus-util
time-date mm-decode mm-bodies mm-encode mail-parse rfc2231 rfc2047
rfc2045 mm-util ietf-drums mail-prsvr mailabbrev mail-utils gmm-utils
mailheader mu4e-obsolete server dtrt-indent ef-spring-theme ef-themes
pixel-scroll cua-base repeat midnight html2text let-alist
evil-collection-flymake flymake-proc flymake project
evil-collection-compile evil-collection-custom cus-edit cus-load
wid-edit evil-collection-comint evil-collection annalist compile
text-property-search comint ansi-osc ansi-color ibuf-macs evil
evil-integration evil-maps evil-commands reveal evil-jumps
evil-command-window evil-types evil-search evil-macros evil-repeat
evil-states evil-core byte-opt advice evil-common thingatpt rect
evil-vars ring nerd-icons nerd-icons-faces nerd-icons-data
nerd-icons-data-mdicon nerd-icons-data-flicon nerd-icons-data-codicon
nerd-icons-data-devicon nerd-icons-data-sucicon nerd-icons-data-wicon
nerd-icons-data-faicon nerd-icons-data-powerline nerd-icons-data-octicon
nerd-icons-data-pomicon nerd-icons-data-ipsicon ligature derived edmacro
kmacro use-package-bind-key bind-key comp comp-cstr warnings icons rx
doom-editor doom-projects doom-ui easy-mmode doom-keybinds pp cl-extra
help-mode use-package-core bytecomp byte-compile general tex-site
doom-start doom-modules cl-seq doom doom-lib cl-macs cl-loaddefs cl-lib
pcase gv harfbuzz jansson dynamic-modules subr-x rmc iso-transl tooltip
cconv eldoc paren electric uniquify ediff-hook vc-hooks lisp-float-type
mwheel term/pgtk-win pgtk-win term/common-win pgtk-dnd tool-bar dnd
fontset image regexp-opt fringe tabulated-list replace newcomment
text-mode lisp-mode prog-mode register page tab-bar menu-bar rfn-eshadow
isearch easymenu timer select scroll-bar mouse jit-lock font-lock syntax
font-core term/tty-colors frame minibuffer nadvice seq simple cl-generic
indonesian philippine cham georgian utf-8-lang misc-lang vietnamese
tibetan thai tai-viet lao korean japanese eucjp-ms cp51932 hebrew greek
romanian slovak czech european ethiopic indian cyrillic chinese
composite emoji-zwj charscript charprop case-table epa-hook
jka-cmpr-hook help abbrev obarray oclosure cl-preloaded button loaddefs
theme-loaddefs faces cus-face macroexp files window text-properties
overlay sha1 md5 base64 format env code-pages mule custom widget keymap
hashtable-print-readable backquote threads dbusbind inotify
dynamic-setting system-font-setting font-render-setting cairo gtk pgtk
multi-tty make-network-process native-compile emacs)

Memory information:
((conses 16 1300349 189182)
 (symbols 48 66318 75)
 (strings 32 312986 21752)
 (string-bytes 1 9875702)
 (vectors 16 148300)
 (vector-slots 8 3659358 277788)
 (floats 8 1572 1454)
 (intervals 56 83807 1891)
 (buffers 984 46))