Re: ftbench update: make integrated

[Werner LEMBERG]

> I am sending this mail as a reminder of last updates that not is
> received feedback yet and current GSoC status.

Uh, oh, please don't send such e-mails to me privately!  You should
always address the e-mail list.

Note that I'm still in semester-end mode, which means a lot of work
for the university.  It should be over in two days :-) Additionally it
seems that Toshiya-san is very busy, too...

> In summary, the benchmark working by Make is in operation so
> far. (baseline, benchmark, clean-benchmark) It passes the parameters
> FTBENCH_FLAGS=xx and creates a benchmark file that one of its
> example is in the attachments.

Well, there is still a lot to do :-)

* I see, for example, `Roboto_subset.txt` in the generated HTML page.
  Why `.txt`?  Shouldn't the extension be rather `.ttf` (or whatever
  the actual test file is called)?

* I suggest to add another column that shows the difference in
  percent, for example, '+3.5%' or '-2.7%'.  This gives a better
  overview.

* What about adding a warning message at the top (in red) if both the
  baseline and the benchmark have the same commit ID?  This would help
  users avoid mistakes.

* I suggest to shorten the commit IDs similar to what gitlab does;
  7 or 8 characters are sufficient.

* Please link the commit IDs to gitlab.

* Given that all tests use the same bench parameters, commit ID,
  commit date, and branch names I suggest to move this information to
  the top, printing it only once.

* `make baseline` still doesn't work from a separate build directory.
  Doing

     cd ~/git/freetype    # the assumed git repository location
     git clean -fdx
     ./autogen.sh

     cd ~
     mkdir benchmark
     cd benchmark
     ~/git/freetype/configure
     make -j12
     make baseline

  results in

```
Building ftbench...
libtool: compile:  gcc -I~/git/freetype/include \
                       -lfreetype \
                       ~/git/freetype/src/tools/ftbench/ftbench.c \
                       -fPIC -DPIC \
                       -o /home/wl/benchmark/.libs/bench.o
libtool: compile:  gcc -I~/git/freetype/include \
                       -lfreetype \
                       ~/git/freetype/src/tools/ftbench/ftbench.c \
                       -o /home/wl/benchmark/bench.o \
                       >/dev/null 2>&1
Creating directory...
Creating baseline...
fatal: not a git repository (or any parent up to mount point /)
Stopping at filesystem boundary (GIT_DISCOVERY_ACROSS_FILESYSTEM not set).
fatal: not a git repository (or any parent up to mount point /)
Stopping at filesystem boundary (GIT_DISCOVERY_ACROSS_FILESYSTEM not set).
fatal: not a git repository (or any parent up to mount point /)
Stopping at filesystem boundary (GIT_DISCOVERY_ACROSS_FILESYSTEM not set).
fatal: not a git repository (or any parent up to mount point /)
Stopping at filesystem boundary (GIT_DISCOVERY_ACROSS_FILESYSTEM not set).
fatal: not a git repository (or any parent up to mount point /)
Stopping at filesystem boundary (GIT_DISCOVERY_ACROSS_FILESYSTEM not set).
fatal: not a git repository (or any parent up to mount point /)
Stopping at filesystem boundary (GIT_DISCOVERY_ACROSS_FILESYSTEM not set).
fatal: not a git repository (or any parent up to mount point /)
Stopping at filesystem boundary (GIT_DISCOVERY_ACROSS_FILESYSTEM not set).
fatal: not a git repository (or any parent up to mount point /)
Stopping at filesystem boundary (GIT_DISCOVERY_ACROSS_FILESYSTEM not set).
fatal: not a git repository (or any parent up to mount point /)
Stopping at filesystem boundary (GIT_DISCOVERY_ACROSS_FILESYSTEM not set).
fatal: not a git repository (or any parent up to mount point /)
Stopping at filesystem boundary (GIT_DISCOVERY_ACROSS_FILESYSTEM not set).
fatal: not a git repository (or any parent up to mount point /)
Stopping at filesystem boundary (GIT_DISCOVERY_ACROSS_FILESYSTEM not set).
fatal: not a git repository (or any parent up to mount point /)
Stopping at filesystem boundary (GIT_DISCOVERY_ACROSS_FILESYSTEM not set).
fatal: not a git repository (or any parent up to mount point /)
Stopping at filesystem boundary (GIT_DISCOVERY_ACROSS_FILESYSTEM not set).
fatal: not a git repository (or any parent up to mount point /)
Stopping at filesystem boundary (GIT_DISCOVERY_ACROSS_FILESYSTEM not set).
fatal: not a git repository (or any parent up to mount point /)
Stopping at filesystem boundary (GIT_DISCOVERY_ACROSS_FILESYSTEM not set).
Baseline created.
```

  It's also bad that it says 'Baseline created' for this situation
  since exactly nothing is created.  In other words, you have to abort
  properly if one or more commands fail.

  Note that the created binary for the just executed build is called
  `bench.o`, which is inappropriate.  It should be `bench` instead (at
  least on a GNU/Linux) box.  I also see that the bench program is
  compiled twice, which looks fishy.

> I need some advices to future plans.  do i need to do absolute same
> for meson and cmake? which one first, any other thing to keep in
> mind?..

Meson is more important than CMake, and yes, it should essentially do
the same as the `make baseline` and friends – the goal is to have a
means to create the HTML page in a similar way.  The exact route may
differ, though.


    Werner