bison-patches
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

bench: add support to randomize the order of execution


From: Akim Demaille
Subject: bench: add support to randomize the order of execution
Date: Sun, 10 May 2020 11:53:44 +0200

Extracted from ongoing work on changing the way the parser accepts the input 
(i.e., the termination condition).

commit 9c21da3363054cccf7618d0b3a2ec91f68e67662
Author: Akim Demaille <address@hidden>
Date:   Sun May 10 11:46:40 2020 +0200

    bench: add support to randomize the order of execution
    
    It's amazing how much the order matters.  To a point that many of
    these benches are meaningless.  For instance (some of the benches
    where run with `make -C benches/latest rand
    BENCHFLAGS=--benchmark_min_time=3`):
    
        compiler: g++ -std=c++11 -O2
          0. %define nofinal
          1.
        --------------------------------------------------
        Benchmark           Time           CPU Iterations
        --------------------------------------------------
        BM_y0            1543 ns       1541 ns     441660
        BM_y1            1521 ns       1520 ns     456535
        --------------------------------------------------
        BM_y0            1531 ns       1530 ns     440584
        BM_y1            1512 ns       1511 ns     457591
        --------------------------------------------------
        BM_y0            1539 ns       1538 ns    2749330
        BM_y1            1516 ns       1515 ns    2771500
        --------------------------------------------------
        BM_y0            1571 ns       1570 ns    2600782
        BM_y1            1542 ns       1541 ns    2708349
        --------------------------------------------------
        BM_y0            1530 ns       1529 ns    2670363
        BM_y1            1519 ns       1518 ns    2764096
    
        --------------------------------------------------
        Benchmark           Time           CPU Iterations
        --------------------------------------------------
        BM_y1            1529 ns       1528 ns     451937
        BM_y0            1508 ns       1507 ns     453944
        --------------------------------------------------
        BM_y1            1525 ns       1524 ns    2750684
        BM_y0            1516 ns       1515 ns    2794034
        --------------------------------------------------
        BM_y1            1526 ns       1525 ns    2749620
        BM_y0            1515 ns       1514 ns    2808112
        --------------------------------------------------
        BM_y1            1524 ns       1523 ns    4475844
        BM_y0            1502 ns       1501 ns    4611665
    
    * etc/bench.pl.in: here.

diff --git a/etc/bench.pl.in b/etc/bench.pl.in
index 669e013c..b7d44f41 100755
--- a/etc/bench.pl.in
+++ b/etc/bench.pl.in
@@ -949,6 +949,10 @@ EOF
 bench: main
 \t./main \$(BENCHFLAGS) | tee -a README.md
 
+.PHONY: rand
+rand:
+\t-rm main \$(OBJECTS)
+\tmake bench OBJECTS="\$\$(shuf -e \$(OBJECTS) | xargs)"
 EOF
 
   my @obj = ();
@@ -971,8 +975,9 @@ BENCHMARK_MAIN();
 EOF
 
   print $makefile <<EOF;
-main: @{obj}
-\t$compiler -o main $cflags main.cc @obj -lbenchmark
+OBJECTS = @{obj}
+main: \$(OBJECTS)
+\t$compiler -o main $cflags main.cc \$(OBJECTS) -lbenchmark
 EOF
 
   run "make";




reply via email to

[Prev in Thread] Current Thread [Next in Thread]