Re: [RFC] Suggestion of new tag format

[Shigio Yamaguchi]

Hi,
For the new tag format, I'm planning to change the next version
of GLOBAL as folows.

o Remove the -cc, -ccc, -cccc option of gtags(1).
  These options are undocumented and only for testing.
o Move to new tag format completely.
o No support for older format.
  Though we can continue the support for the older format, it seems
  not to be worthy. Instead, the following error messages are
  displayed in a wrong usage.
        [older global and new tag file]
        $ global -x main
        GTAGS seems new format. Please install the latest GLOBAL.
        [new global and older tag file]
        $ global -x main
        GTAGS seems older format. Please remake tag files.
  
What do you think?

> I have a plan to make new tag record format.
> 
> This change allows path name including blanks, and decrease
> disk space used. In this suggestion, any user visible change
> is not intended.
> 
> I have already made the first version of it.
> Since it takes time to assess the code, I would like to commit
> it to the CVS repository for the test like the -cc option.
> After that, you can make tag files using new format like follows:
> 
>       $ gtags -ccc                    <= three 'c' option
> 
> This code is written so that we can remove it easily.
> 
> What do you think?
> Is there any problem?
> 
> --------------------------------------------------------------------
> Suggestion of new tag format (I play, for GLOBAL-5.0)
> 
> Purpose
> =======
> 
> o Allows path name including blanks.
> o Decrease disk space used.
> o Make tag format simpler.
> o Realize stable performance.
> 
> Specification
> =============
> 
> STANDARD5 format (for GTAGS)
> 
>       <file id> <tag name> <line number> <line image>
> 
>               * Separator is single blank.
> 
>       [example]
>       +------------------------------------
>       |110 func 10 int func(int a)
>       |110 func 30 func(int a1, int a2)
> 
> COMPACT5 format (for GRTAGS, GSYMS)
> 
>       <file id> <tag name> <line number>,...
> 
>               * Separator is single blank.
> 
>       [example]
>       +------------------------------------
>       |110 func 10,30
> 
> Description
> ===========
> 
> o STANDARD5 format is applied to GTAGS, and COMPACT5 format is applied
>   to GRTAGS and GSYMS by default.
>   Because GRTAGS was growing too much, COMPACT5 format might be suitable.
>   (The -c (--compact) option might be removed.)
> o Separator is single blank.
>   This decrease disk space used a little, and make it easy to parse
>   tag record.
> o Use file id instead of path name.
>   This decreases disk space used, and allows blanks in path name
>   at least in tag files.
> o Put file id at the head of tag record.
>   We can access file id without string processing.
>   This is advantageous for deleting tag record when incremental updating.
> o Format version of tag file is incremented (4).
>   This enables us to keep upper compatibility of tag files.
>   (Whether to keep it, a further examination is necessary though.)
>   For the detail of 'format version', please see the comment in
>   libutil/gtagsop.c.
> 
> Test data
> =========
> 
> Here is a test data when applying to FreeBSD kernel source code.
> 
>       Format    Command line   Making time Updating time Total size
>                                  (sec)       (sec)         (bytes)
>       -------------------------------------------------------------
>       standard    gtags        1137        6676           131391488
>       compact     gtags -c      330          98            36569088
>       pathindex   gtags -cc     948        4254           109953024
>       new format  gtags -ccc    392         112            43040768
>       -------------------------------------------------------------
> 
> Patch
> =====
--
Shigio YAMAGUCHI <address@hidden> - Tama Communications Corporation
PGP fingerprint: D1CB 0B89 B346 4AB6 5663  C4B6 3CA5 BBB3 57BE DDA3