grub-devel
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [PATCH v4 1/3] docs: Add grub command documentation checks


From: Alex Burmashev
Subject: Re: [PATCH v4 1/3] docs: Add grub command documentation checks
Date: Tue, 26 May 2020 13:27:40 +0200
User-agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:68.0) Gecko/20100101 Thunderbird/68.8.0

Reviewed-by: Alex Burmashev <address@hidden>

11-May-20 1:12 AM, Hans Ulrich Niedermann пишет:
> This adds the docs/check-commands.py script which checks whether
> grub commands implemented in the git source tree are documented
> in docs/grub.texi and vice versa.
>
> During a standard "make" command, BUILT_SOURCES makes sure that
> docs/undocumented-commands.texi will be created (if it does not
> exist yet) or updated (if it differs from what the script would
> generate). Otherwise, the existing file is left alone.
>
> With docs/undocumented-commands.texi being shipped in dist tarballs
> alongside grub.info, building GRUB from a dist tarball will still
> work without rebuilding grub.info page unless an actual change has
> happened which causes a different docs/undocumented-commands.texi
> to be generated.
>
> If you run "make grub.info" in the docs/ subdirectory, the
> BUILT_SOURCES trick to create or update undocumented-commands.texi
> will not work (this is an Automake limitation). So if you want
> to avoid the "all" make target, you need to explictly run
> "make update-undoc" first.
>
> The generated docs/undocumented-commands.texi file will document each
> otherwise undocumented command by describing that it is undocumented,
> and by adding that it is mentioned somewhere else in the info page if
> that happens to be the case.
>
> The build time requirements of the make target's build rule are
> already required by other parts of the GRUB buildsystem:
>
>   * $(CMP)
>     compare two files by content
>
>   * $(PYTHON)
>     check-commands.py has been written to run on Python 2 and 3
>     (tested with Python 2.7 and Python 3.7)
>
>   * texinfo tools
>     When building from a git source tree, texinfo is required
>     to rebuild docs/grub.info. When building from a dist tarball,
>     texinfo is not required. This is standard Automake behaviour
>     which we do not change.
>
>     However, when building from a dist tarball if you ever run
>     "make clean", then the texinfo toolchain will be required as
>     docs/undocumented-commands.texi will be rebuilt and thus
>     a rebuild of docs/grub.info will be triggered.
>
>     I do not expect distribution package builds from dist
>     tarball to use "make clean", so I do not expect this new
>     requirement for the texinfo tools to have much of an impact.
>
> The docs/check-commands.py script runs fast enough to not perceivably
> slow down a "make" on the grub source tree. The script basically reads
> and more or less greps through docs/grub.texi and all *.c source files,
> and that happens very quickly.
>
> Signed-off-by: Hans Ulrich Niedermann <address@hidden>
>
>  create mode 100644 docs/check-commands.py
>
> diff --git a/.gitignore b/.gitignore
> index aa180fa89..b7a853b90 100644
> --- a/.gitignore
> +++ b/.gitignore
> @@ -96,6 +96,7 @@ widthspec.bin
>  /docs/*.info-[0-9]*
>  /docs/stamp-1
>  /docs/stamp-vti
> +/docs/undocumented-commands.texi
>  /docs/version-dev.texi
>  /docs/version.texi
>  /ehci_test
> diff --git a/docs/Makefile.am b/docs/Makefile.am
> index 93eb39627..a8009e7a9 100644
> --- a/docs/Makefile.am
> +++ b/docs/Makefile.am
> @@ -2,8 +2,20 @@ AUTOMAKE_OPTIONS = subdir-objects
>  
>  # AM_MAKEINFOFLAGS = --no-split --no-validate
>  info_TEXINFOS = grub.texi grub-dev.texi
> -grub_TEXINFOS = fdl.texi
> -
> -EXTRA_DIST = font_char_metrics.png font_char_metrics.txt
> +grub_TEXINFOS = fdl.texi undocumented-commands.texi
>  
> +EXTRA_DIST = check-commands.py font_char_metrics.png font_char_metrics.txt
>  
> +# This has only been tested to work without the Automake
> +# info-in-builddir option.
> +CLEANFILES = undocumented-commands.texi
> +BUILT_SOURCES = update-undoc
> +update-undoc:
> +     $(PYTHON) $(srcdir)/check-commands.py > undocumented-commands.texi.tmp
> +     @if test -f $(srcdir)/undocumented-commands.texi && $(CMP) 
> undocumented-commands.texi.tmp $(srcdir)/undocumented-commands.texi; then \
> +             echo "Not updating undocumented-commands.texi: is up to date"; \
> +             rm -f undocumented-commands.texi.tmp; \
> +     else \
> +             echo "Updating undocumented-commands.texi"; \
> +             mv -f undocumented-commands.texi.tmp 
> $(srcdir)/undocumented-commands.texi; \
> +     fi
> diff --git a/docs/check-commands.py b/docs/check-commands.py
> new file mode 100644
> index 000000000..4174b954a
> --- /dev/null
> +++ b/docs/check-commands.py
> @@ -0,0 +1,205 @@
> +#!/usr/bin/env python
> +#
> +# check-commands.py - compare commands defined in grub.texi the *.c sources
> +#
> +# Usage:
> +#     check-commands.py > undocumented-commands.texi.new
> +#
> +# There are no command line parameters. The locations of grub.texi and
> +# the *.c source files are determined relative to the location of the
> +# script file. The only output happens on stdout, in texinfo format
> +# for inclusione into grub.texi via '@include'.
> +
> +from __future__ import print_function
> +
> +import os
> +import re
> +
> +import sys
> +
> +
> +class CommandChecker:
> +
> +    def __init__(self):
> +        srcdir, self.prog = os.path.split(__file__)
> +        self.top_srcdir = os.path.dirname(os.path.abspath(srcdir))
> +
> +    def read_texi_text_commands(self, texi_filename):
> +        texi_pathname = os.path.join(self.top_srcdir, 'docs', texi_filename)
> +        command_re = re.compile(r'@command\{([a-zA-Z0-9_-]+)\}')
> +        commands = set()
> +        with open(texi_pathname) as texi:
> +            for line in texi.readlines():
> +                for m in command_re.finditer(line):
> +                    commands.add(m[1])
> +        return commands
> +
> +    def read_texi_command_menu(self, texi_filename):
> +        texi_pathname = os.path.join(self.top_srcdir, 'docs', texi_filename)
> +        menuentry_re = re.compile(r'\* ([^:]+)::')
> +        commands_node_re = re.compile(r'^@node .+ commands$')
> +        commands = set()
> +
> +        # Avoid using enum for the state machine state (for compatibility)
> +        waiting_for_node = True
> +        waiting_for_menu = False
> +        collecting_commands = False
> +
> +        with open(texi_pathname) as texi:
> +            for line in texi.readlines():
> +                line = line.strip()
> +                if line.startswith('@comment'):
> +                    continue
> +                if waiting_for_node:
> +                    if commands_node_re.match(line):
> +                        waiting_for_node = False
> +                        waiting_for_menu = True
> +                        continue
> +                if waiting_for_menu:
> +                    if line == '@menu':
> +                        waiting_for_menu = False
> +                        collecting_commands = True
> +                        continue
> +                if collecting_commands:
> +                    if line == '@end menu':
> +                        collecting_commands = False
> +                        waiting_for_node = True
> +                        continue
> +                    m = menuentry_re.match(line)
> +                    commands.add(m.group(1))
> +        return commands
> +
> +    def read_src_commands(self):
> +        top = os.path.join(self.top_srcdir, 'grub-core')
> +        register_re = 
> re.compile(r'grub_register_(command|command_p1|extcmd)\s*\("([a-z0-9A-Z_\[]+)",')
> +        commands = set()
> +        for dirpath, dirnames, filenames in os.walk(top):
> +            for filename in filenames:
> +                filepath = os.path.join(dirpath, filename)
> +                filepath_ext = os.path.splitext(filepath)[1]
> +                if filepath_ext == '.c':
> +                    with open(filepath) as cfile:
> +                        for line in cfile.readlines():
> +                            for m in register_re.finditer(line):
> +                                commands.add(m.group(2))
> +        return commands
> +
> +
> +def write_undocumented_commands(undocumented_commands,
> +                                mentioned_commands):
> +    print("""\
> +@node Undocumented commands
> +@section The list of undocumented commands
> +""")
> +
> +    if not undocumented_commands:
> +        print("""
> +There appear to be no undocumented commands at this time.
> +""")
> +        return
> +
> +    print("""\
> +These commands are implemented in the grub software but still need to be
> +documented and sorted into categories.
> +""")
> +
> +    sorted_undocumented_commands = sorted(list(undocumented_commands))
> +
> +    # Fit the longest command into the format string for the menu entries
> +    maxlen_str = sorted(list(undocumented_commands),
> +                        key=lambda cmd: len(cmd),
> +                        reverse=True)[0]
> +    fmt = '* %%-%ds %%s' % (2+len(maxlen_str))
> +
> +    print("@menu")
> +    for cmd in sorted_undocumented_commands:
> +        if cmd in mentioned_commands:
> +            print(fmt % ("%s::" % cmd, "Undocumented command (but mentioned 
> somewhere)"))
> +        else:
> +            print(fmt % ("%s::" % cmd, "Undocumented command"))
> +    print("@end menu")
> +    print()
> +
> +    for cmd in sorted_undocumented_commands:
> +        print("@node %s" % cmd)
> +        print("@subsection %s" % cmd)
> +        print()
> +        print("The grub command @command{%s} has not been documented 
> properly yet." % cmd)
> +        if cmd in mentioned_commands:
> +            print("""
> +However, the @command{%s} command @emph{is} mentioned
> +somewhere within this info file, so you might still find some
> +interesting information there.""" % cmd)
> +        print()
> +
> +
> +def print_set(st):
> +    for n, item in enumerate(sorted(list(st)), start=1):
> +        print("@comment", "  %d." % n, item)
> +
> +
> +def main():
> +    cc = CommandChecker()
> +
> +    print("""\
> +@comment Automatically generated by the %s script.
> +@comment Do not modify this generated file.
> +@comment
> +@comment If you want to document some of the commands listed below, feel
> +@comment free to copy over some of the @nodes and @subsections below
> +@comment into the grub.texi file proper and re-generate this file.
> +@comment""" % cc.prog)
> +
> +    # This gives a few good hints about what commands are at least
> +    # mentioned somewhere which can be helpful for finding out more
> +    # about commands.
> +    texi_text_commands = cc.read_texi_text_commands('grub.texi')
> +    print("@comment", "Commands mentioned somewhere in the grub.texi text:")
> +    print_set(texi_text_commands)
> +
> +    texi_node_commands = cc.read_texi_command_menu('grub.texi')
> +    # print("@comment", "Commands documented in grub.texi node/subsection:")
> +    # print_set(texi_node_commands)
> +
> +    src_commands = cc.read_src_commands()
> +    # print("@comment", "Commands registered in grub source:")
> +    # print_set(src_commands)
> +
> +    node_without_src = texi_node_commands - src_commands
> +    if node_without_src:
> +        print("@comment",
> +              "Cmds documented in grub.texi node/subsection but not 
> registered in source:")
> +        print_set(node_without_src)
> +        print("@comment")
> +    else:
> +        print("""\
> +@comment There appear to be no commands documented in a grub.texi
> +@comment node/subsection but not registered in grub source.
> +@comment""")
> +
> +    src_without_node = src_commands - texi_node_commands
> +    if src_without_node:
> +        print("@comment",
> +              "Cmds registered in source but not documented in grub.texi 
> node/subsection:")
> +        print_set(src_without_node)
> +    else:
> +        print("""\
> +@comment All commands registered in the source appear to have been
> +@comment documented in a grub.texi node/subsection. Congratulations!
> +@comment""")
> +
> +    print()
> +
> +    write_undocumented_commands(src_without_node, texi_text_commands)
> +
> +    # Once grub.texi actually documents all commands, we can uncomment
> +    # this and actually fail if the set of implemented commands and
> +    # the set of documented commands differ in any way.
> +    # if ((len(node_without_src) > 0) or (len(src_without_node) > 0)):
> +    #     sys.exit(1)
> +
> +    sys.exit(0)
> +
> +
> +if __name__ == '__main__':
> +    main()
> diff --git a/docs/grub.texi b/docs/grub.texi
> index d6408d242..0865ffa17 100644
> --- a/docs/grub.texi
> +++ b/docs/grub.texi
> @@ -3764,6 +3764,7 @@ shell}.
>  * General commands::
>  * Command-line and menu entry commands::
>  * Networking commands::
> +* Undocumented commands::
>  @end menu
>  
>  
> @@ -5636,6 +5637,9 @@ is given, use default list of servers.
>  @end deffn
>  
>  
> +@include undocumented-commands.texi
> +
> +
>  @node Internationalisation
>  @chapter Internationalisation
>  



reply via email to

[Prev in Thread] Current Thread [Next in Thread]