[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[PATCH v7 14/21] scripts/simplebench: support iops
From: |
Vladimir Sementsov-Ogievskiy |
Subject: |
[PATCH v7 14/21] scripts/simplebench: support iops |
Date: |
Wed, 21 Oct 2020 17:58:52 +0300 |
Support benchmarks returning not seconds but iops. We'll use it for
further new test.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
---
scripts/simplebench/simplebench.py | 38 ++++++++++++++++++++++--------
1 file changed, 28 insertions(+), 10 deletions(-)
diff --git a/scripts/simplebench/simplebench.py
b/scripts/simplebench/simplebench.py
index 2445932fc2..2251cd34ea 100644
--- a/scripts/simplebench/simplebench.py
+++ b/scripts/simplebench/simplebench.py
@@ -24,9 +24,12 @@ def bench_one(test_func, test_env, test_case, count=5,
initial_run=True):
test_func -- benchmarking function with prototype
test_func(env, case), which takes test_env and test_case
- arguments and returns {'seconds': int} (which is benchmark
- result) on success and {'error': str} on error. Returned
- dict may contain any other additional fields.
+ arguments and on success returns dict with 'seconds' or
+ 'iops' (or both) fields, specifying the benchmark result.
+ If both 'iops' and 'seconds' provided, the 'iops' is
+ considered the main, and 'seconds' is just an additional
+ info. On failure test_func should return {'error': str}.
+ Returned dict may contain any other additional fields.
test_env -- test environment - opaque first argument for test_func
test_case -- test case - opaque second argument for test_func
count -- how many times to call test_func, to calculate average
@@ -34,8 +37,9 @@ def bench_one(test_func, test_env, test_case, count=5,
initial_run=True):
Returns dict with the following fields:
'runs': list of test_func results
- 'average': average seconds per run (exists only if at least one run
- succeeded)
+ 'dimension': dimension of results, may be 'seconds' or 'iops'
+ 'average': average value (iops or seconds) per run (exists only if at
+ least one run succeeded)
'delta': maximum delta between test_func result and the average
(exists only if at least one run succeeded)
'n-failed': number of failed runs (exists only if at least one run
@@ -54,11 +58,19 @@ def bench_one(test_func, test_env, test_case, count=5,
initial_run=True):
result = {'runs': runs}
- succeeded = [r for r in runs if ('seconds' in r)]
+ succeeded = [r for r in runs if ('seconds' in r or 'iops' in r)]
if succeeded:
- avg = sum(r['seconds'] for r in succeeded) / len(succeeded)
+ if 'iops' in succeeded[0]:
+ assert all('iops' in r for r in succeeded)
+ dim = 'iops'
+ else:
+ assert all('seconds' in r for r in succeeded)
+ assert all('iops' not in r for r in succeeded)
+ dim = 'seconds'
+ avg = sum(r[dim] for r in succeeded) / len(succeeded)
+ result['dimension'] = dim
result['average'] = avg
- result['delta'] = max(abs(r['seconds'] - avg) for r in succeeded)
+ result['delta'] = max(abs(r[dim] - avg) for r in succeeded)
if len(succeeded) < count:
result['n-failed'] = count - len(succeeded)
@@ -118,11 +130,17 @@ def ascii(results):
"""Return ASCII representation of bench() returned dict."""
from tabulate import tabulate
+ dim = None
tab = [[""] + [c['id'] for c in results['envs']]]
for case in results['cases']:
row = [case['id']]
for env in results['envs']:
- row.append(ascii_one(results['tab'][case['id']][env['id']]))
+ res = results['tab'][case['id']][env['id']]
+ if dim is None:
+ dim = res['dimension']
+ else:
+ assert dim == res['dimension']
+ row.append(ascii_one(res))
tab.append(row)
- return tabulate(tab)
+ return f'All results are in {dim}\n\n' + tabulate(tab)
--
2.21.3
- [PATCH v7 00/21] preallocate filter, Vladimir Sementsov-Ogievskiy, 2020/10/21
- [PATCH v7 02/21] block/io.c: drop assertion on double waiting for request serialisation, Vladimir Sementsov-Ogievskiy, 2020/10/21
- [PATCH v7 01/21] block: simplify comment to BDRV_REQ_SERIALISING, Vladimir Sementsov-Ogievskiy, 2020/10/21
- [PATCH v7 03/21] block/io: split out bdrv_find_conflicting_request, Vladimir Sementsov-Ogievskiy, 2020/10/21
- [PATCH v7 04/21] block/io: bdrv_wait_serialising_requests_locked: drop extra bs arg, Vladimir Sementsov-Ogievskiy, 2020/10/21
- [PATCH v7 05/21] block: bdrv_mark_request_serialising: split non-waiting function, Vladimir Sementsov-Ogievskiy, 2020/10/21
- [PATCH v7 06/21] block: introduce BDRV_REQ_NO_WAIT flag, Vladimir Sementsov-Ogievskiy, 2020/10/21
- [PATCH v7 14/21] scripts/simplebench: support iops,
Vladimir Sementsov-Ogievskiy <=
- [PATCH v7 08/21] block: introduce preallocate filter, Vladimir Sementsov-Ogievskiy, 2020/10/21
- [PATCH v7 09/21] qemu-io: add preallocate mode parameter for truncate command, Vladimir Sementsov-Ogievskiy, 2020/10/21
- [PATCH v7 16/21] simplebench: rename ascii() to results_to_text(), Vladimir Sementsov-Ogievskiy, 2020/10/21
- [PATCH v7 07/21] block: bdrv_check_perm(): process children anyway, Vladimir Sementsov-Ogievskiy, 2020/10/21
- [PATCH v7 15/21] scripts/simplebench: use standard deviation for +- error, Vladimir Sementsov-Ogievskiy, 2020/10/21
- [PATCH v7 10/21] iotests: qemu_io_silent: support --image-opts, Vladimir Sementsov-Ogievskiy, 2020/10/21
- [PATCH v7 11/21] iotests.py: execute_setup_common(): add required_fmts argument, Vladimir Sementsov-Ogievskiy, 2020/10/21
- [PATCH v7 12/21] iotests: add 298 to test new preallocate filter driver, Vladimir Sementsov-Ogievskiy, 2020/10/21
- [PATCH v7 13/21] scripts/simplebench: fix grammar: s/successed/succeeded/, Vladimir Sementsov-Ogievskiy, 2020/10/21
- [PATCH v7 17/21] simplebench: move results_to_text() into separate file, Vladimir Sementsov-Ogievskiy, 2020/10/21