bug#911: 23.0.60; custom-set-faces causes "New Frame" to fail (max-specpdl-size)

[martin rudalics]

Ian Miller has patiently continued to debug this and we finally came to
the conclusion that the following sequence of events leads to the error.

Ian uses

  (custom-set-faces
   '(default ((t (... :background "DarkSlateGray" ...)))))

When creating a new frame `frame-set-background-mode' gets called which
eventually calls `face-spec-recalc'.  The latter has this loop

      (dolist (spec theme-faces)
        (face-spec-set-2 face frame (cadr spec))))

which calls `face-spec-set-2' two times for the default face background,
one time for #ffffff and one time for DarkSlateGray.  `face-spec-set-2'
calls `set-face-attribute', which calls
`internal-set-lisp-face-attribute' which calls Fmodify_frame_parameters
which calls `frame-set-background-mode' again.  Now

(>= (apply '+ (color-values "DarkSlateGrey"))
    (* (apply '+ (color-values "white")) .6))

evaluates to nil while

(>= (apply '+ (color-values "#ffffff"))
    (* (apply '+ (color-values "white")) .6))

evaluates to t.  This means that `bg-mode' never equals `old-bg-mode'
but continuously alternates between 'light and 'dark, so
`frame-set-background-mode' will call `face-spec-recalc' thus sending
Emacs into a nasty loop.

Note that the looping behavior is due to the second call in the dolist
(that for DarkSlateGrey).  The first call (for #ffffff) terminates.

I'm not sure how to fix this because that looping has no specific entry
point.  In fact, Ian told me

> Incidently, whilst testing I have found that the same bug does indeed
> manifest itself elsewhere. Normally I have the toolbar disabled in my
> .emacs, but after stripping everything out of .emacs for testing
> purposes, I noticed that I get the following errors whenever I hover
> over an icon in the toolbar:
>
>   Error while displaying tooltip: (error Variable binding depth exceeds
>   max-specpdl-size)
>   Read a directory, to operate on its files
>   Error while displaying tooltip: (error Variable binding depth exceeds
>   max-specpdl-size)
>   Read an existing file into an Emacs buffer

which seems due to the fact that x_create_tip_frame calls
`modify-frame-parameters' ...

Anyone out there with good ideas for a fix?

martin