[RFC] Combining of Exuberant Ctags and GNU GLOBAL

[Shigio YAMAGUCHI]

Hello all,
I will post a proposal to the Exuberant Ctags developer's mailing list.
The title is 'Combining of Exuberant Ctags and GNU GLOBAL'.
Before posting, I would like to hear about whether there is any problem
in it. If you noticed some, please let me know. Thank you in advance.

Here is the proposal.

---------------------------------------------------------------------------

Combining of Exuberant Ctags and GNU GLOBAL     by Shigio Yamaguchi

0. Introduction

GNU GLOBAL is a source code tagging system. The feature of it is an
ability to treat not only definitions but also references. Although
it can be used independently, it can also be used in combination
with Exuberant Ctags. This proposal is a thing about the method of
combining two tools.

1. Current Status

Now, you can replace the built-in parser of GNU GLOBAL by Exuberant Ctags.
With some setup, gtags invokes Exuberant Ctags with the --format=1 and
--filter option, and read tag records from it.

The concrete setup method is as follows.

        (Installation of GLOBAL)
        $ ./configure --with-exuberant-ctags=/usr/local/bin/ctags
        $ sudo make install

        (Execution of GLOBAL)
        $ export GTAGSCONF=/usr/local/share/gtags/gtags.conf
        $ export GTAGSLABEL=exuberant-ctags
        $ gtags                         # invokes Exuberant Ctags internally
        $ global -x func                # locate tags
        func                5 test.c           func() {
        $ _

That is, GLOBAL already supports 41 programming languages supported
by Exuberant Ctags.  But it cannot generate reference tags, because
Exuberant Ctags does not pick up them. My proposal is to bury this lack.

2. Adding new format to treat references.

How about adding --format=3 option to Exuberant Ctags?

Format3 adds a type string to the format1.

        <a type string>    <format1>
                        ^
                        |
                one or more blanks

        The meaning of the type strings:

                "D": definition
                "R": other than definition

Please assume the following source code.

        [test.c]
        +---------------------
1       |sub() {
2       |       ...
3       |       return 1;
4       |}
5       |func() {
6       |       strcpy(...);
7       |       if (sub()) {
8       |               ...
9       |       }
10      |}
        +---------------------

Now, Exuberant Ctags with the --format=1 option generates
the following output.

$ ctags --format=1 -x test.c
func                5 test.c           func() {
sub                 1 test.c           sub() {

Please care about that the following symbols are ignored.

'strcpy' in the 6th line
'sub' in the 7th line

When the --format=3 option is specified, Exuberant Ctags may
pick up reference tags.

$ ctags --format=3 -x test.c
D func                5 test.c           func() {
R strcpy              6 test.c                  strcpy(...);         <=== added
D sub                 1 test.c           sub() {
R sub                 7 test.c                  if (sub()) {         <=== added

Algorithm of the parser may be like follows.

        foreach of all tokens {
                if (symbol?)
                        if (definition?)
                                Put a tag record with "D".
                        else
                                Put a tag record with "R".      <= Added
                }
        }

Rewriting GLOBAL to accept above format is easy.
Then GLOBAL commands will work like follows.

$ gtags                                                         # make tag files
$ global -xd sub
sub                 1 test.c           sub() {                  # show 
definitions
$ global -xr sub
sub                 7 test.c                  if (sub()) {      # show 
references
$ global -xs strcpy
strcpy              6 test.c                  strcpy(...);      # show other 
symbols
$ _

        [-d] shows definitions
        -r   shows references (defined in other places)
        -s   shows other symbols (not defined anywhere)

Please note that the definitions, references and other symbols are
treated separately.  This means that Exuberant Ctags become a perfect
parser of GLOBAL.  Although it is difficult for Exuberant Ctags to
support reference tags independently, it is easy if combined with
GNU GLOBAL. It is worthy for the users of two tools, I believe.


3. For the parsers which don't support reference tags.

The treatment of the parsers which don't generate reference tags is easy.
What is necessary is just to add a "D" string unconditionally at the head
of the records when the --format=3 option is specified.

$ ctags --format=3 -x test.c
D func                5 test.c           func() {
D sub                 1 test.c           sub() {

Though such output don't have "R" records, it is no problem.


4. Is this useful?

Even if this mechanism is realized, a parser does not necessarily
increase automatically. But it will become an infrastructure for
those who try to write a new parser, I think.

What do you think?

-- End of proposal
---------------------------------------------------------------------------
--
Shigio YAMAGUCHI <address@hidden>
PGP fingerprint: D1CB 0B89 B346 4AB6 5663  C4B6 3CA5 BBB3 57BE DDA3