[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: [PATCH v2 2/2] scripts/performance: Add list_helpers.py script
|
From: |
Ahmed Karaman |
|
Subject: |
Re: [PATCH v2 2/2] scripts/performance: Add list_helpers.py script |
|
Date: |
Tue, 28 Jul 2020 18:55:28 +0200 |
On Tue, Jul 28, 2020 at 12:30 PM Aleksandar Markovic
<aleksandar.qemu.devel@gmail.com> wrote:
>
>
>
> On Thursday, July 16, 2020, Ahmed Karaman <ahmedkhaledkaraman@gmail.com>
> wrote:
>>
>> Python script that prints executed helpers of a QEMU invocation.
>>
>
> Hi, Ahmed.
>
> You outlined the envisioned user workflow regarding this script in your
> report. As I understand it, it generally goes like this:
>
> 1) The user first discovers helpers, and their performance data.
> 2) The user examines the callees of a particular helper of choice (usually,
> the most instruction-consuming helper).
> 3) The user perhaps further examines a callee of a particular callee of the
> particular helper.
> 4) The user continues this way until the conclusion can be drawn, or maximal
> depth is reached.
>
> The procedure might be time consuming since each step requires running an
> emulation of the test program.
>
> This makes me think that the faster and easier tool for the user (but, to
> some, not that great, extent, harder for you) would be improved
> list_helpers.py (and list_fn_calees.py) that provides list of all callees for
> all helpers, in the tree form (so, callees of callees, callees of callees of
> callees, etc.), rather than providing just a list of immediate callees, like
> it currently does.
>
> I think you can provide such functionality relatively easily using recursion.
> See, let's say:
>
> https://realpython.com/python-thinking-recursively/
>
> Perhaps you can have a switch (let's say, --tree <yes|no>) that specifies
> whether the script outputs just immediate callee list, or entire callee tree.
I have to say, this is a very nice suggestion. I will start working on it!
>
> Thanks,
> Aleksandar
>
>>
>> Syntax:
>> list_helpers.py [-h] -- \
>> <qemu executable> [<qemu executable options>] \
>> <target executable> [<target executable options>]
>>
>> [-h] - Print the script arguments help message.
>>
>> Example of usage:
>> list_helpers.py -- qemu-mips coulomb_double-mips -n10
>>
>> Example output:
>> Total number of instructions: 108,933,695
>>
>> Executed QEMU Helpers:
>>
>> No. Ins Percent Calls Ins/Call Helper Name Source File
>> --- ------- ------- ------ -------- -------------------- ---------------
>> 1 183,021 0.168% 1,305 140 helper_float_sub_d
>> <qemu>/target/mips/fpu_helper.c
>> 2 177,111 0.163% 770 230 helper_float_madd_d
>> <qemu>/target/mips/fpu_helper.c
>> 3 171,537 0.157% 1,014 169 helper_float_mul_d
>> <qemu>/target/mips/fpu_helper.c
>> 4 157,298 0.144% 2,443 64 helper_lookup_tb_ptr
>> <qemu>/accel/tcg/tcg-runtime.c
>> 5 138,123 0.127% 897 153 helper_float_add_d
>> <qemu>/target/mips/fpu_helper.c
>> 6 47,083 0.043% 207 227 helper_float_msub_d
>> <qemu>/target/mips/fpu_helper.c
>> 7 24,062 0.022% 487 49 helper_cmp_d_lt
>> <qemu>/target/mips/fpu_helper.c
>> 8 22,910 0.021% 150 152 helper_float_div_d
>> <qemu>/target/mips/fpu_helper.c
>> 9 15,497 0.014% 321 48 helper_cmp_d_eq
>> <qemu>/target/mips/fpu_helper.c
>> 10 9,100 0.008% 52 175 helper_float_trunc_w_d
>> <qemu>/target/mips/fpu_helper.c
>> 11 7,059 0.006% 10 705 helper_float_sqrt_d
>> <qemu>/target/mips/fpu_helper.c
>> 12 3,000 0.003% 40 75 helper_cmp_d_ule
>> <qemu>/target/mips/fpu_helper.c
>> 13 2,720 0.002% 20 136 helper_float_cvtd_w
>> <qemu>/target/mips/fpu_helper.c
>> 14 2,477 0.002% 27 91 helper_swl
>> <qemu>/target/mips/op_helper.c
>> 15 2,000 0.002% 40 50 helper_cmp_d_le
>> <qemu>/target/mips/fpu_helper.c
>> 16 1,800 0.002% 40 45 helper_cmp_d_un
>> <qemu>/target/mips/fpu_helper.c
>> 17 1,164 0.001% 12 97 helper_raise_exception_
>> <qemu>/target/mips/op_helper.c
>> 18 720 0.001% 10 72 helper_cmp_d_ult
>> <qemu>/target/mips/fpu_helper.c
>> 19 560 0.001% 140 4 helper_cfc1
>> <qemu>/target/mips/fpu_helper.c
>>
>> Signed-off-by: Ahmed Karaman <ahmedkhaledkaraman@gmail.com>
>> ---
>> scripts/performance/list_helpers.py | 207 ++++++++++++++++++++++++++++
>> 1 file changed, 207 insertions(+)
>> create mode 100755 scripts/performance/list_helpers.py
>>
>> diff --git a/scripts/performance/list_helpers.py
>> b/scripts/performance/list_helpers.py
>> new file mode 100755
>> index 0000000000..a97c7ed4fe
>> --- /dev/null
>> +++ b/scripts/performance/list_helpers.py
>> @@ -0,0 +1,207 @@
>> +#!/usr/bin/env python3
>> +
>> +# Print the executed helpers of a QEMU invocation.
>> +#
>> +# Syntax:
>> +# list_helpers.py [-h] -- \
>> +# <qemu executable> [<qemu executable options>] \
>> +# <target executable> [<target executable options>]
>> +#
>> +# [-h] - Print the script arguments help message.
>> +#
>> +# Example of usage:
>> +# list_helpers.py -- qemu-mips coulomb_double-mips
>> +#
>> +# This file is a part of the project "TCG Continuous Benchmarking".
>> +#
>> +# Copyright (C) 2020 Ahmed Karaman <ahmedkhaledkaraman@gmail.com>
>> +# Copyright (C) 2020 Aleksandar Markovic <aleksandar.qemu.devel@gmail.com>
>> +#
>> +# This program is free software: you can redistribute it and/or modify
>> +# it under the terms of the GNU General Public License as published by
>> +# the Free Software Foundation, either version 2 of the License, or
>> +# (at your option) any later version.
>> +#
>> +# This program is distributed in the hope that it will be useful,
>> +# but WITHOUT ANY WARRANTY; without even the implied warranty of
>> +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
>> +# GNU General Public License for more details.
>> +#
>> +# You should have received a copy of the GNU General Public License
>> +# along with this program. If not, see <https://www.gnu.org/licenses/>.
>> +
>> +import argparse
>> +import os
>> +import subprocess
>> +import sys
>> +import tempfile
>> +
>> +
>> +def find_JIT_line(callgrind_data):
>> + """
>> + Search for the line with the JIT call in the callgrind_annotate
>> + output when ran using --tre=calling.
>> + All the helpers should be listed after that line.
>> +
>> + Parameters:
>> + callgrind_data (list): callgrind_annotate output
>> +
>> + Returns:
>> + (int): Line number of JIT call
>> + """
>> + line = -1
>> + for i in range(len(callgrind_data)):
>> + split_line = callgrind_data[i].split()
>> + if len(split_line) > 2 and \
>> + split_line[1] == "*" and \
>> + split_line[-1] == "[???]":
>> + line = i
>> + break
>> + return line
>> +
>> +
>> +def get_helpers(JIT_line, callgrind_data):
>> + """
>> + Get all helpers data given the line number of the JIT call.
>> +
>> + Parameters:
>> + JIT_line (int): Line number of the JIT call
>> + callgrind_data (list): callgrind_annotate output
>> +
>> + Returns:
>> + (list):[[number_of_instructions(int), helper_name(str),
>> + number_of_calls(int), source_file(str)]]
>> + """
>> + helpers = []
>> + next_helper = JIT_line + 1
>> + while (callgrind_data[next_helper] != "\n"):
>> + split_line = callgrind_data[next_helper].split()
>> + number_of_instructions = int(split_line[0].replace(",", ""))
>> + source_file = split_line[2].split(":")[0]
>> + callee_name = split_line[2].split(":")[1]
>> + number_of_calls = int(split_line[3][1:-2])
>> + helpers.append([number_of_instructions, callee_name,
>> + number_of_calls, source_file])
>> + next_helper += 1
>> + return sorted(helpers, reverse=True)
>> +
>> +
>> +def main():
>> + # Parse the command line arguments
>> + parser = argparse.ArgumentParser(
>> + usage="list_helpers.py [-h] -- "
>> + "<qemu executable> [<qemu executable options>] "
>> + "<target executable> [<target executable options>]")
>> +
>> + parser.add_argument("command", type=str, nargs="+",
>> help=argparse.SUPPRESS)
>> +
>> + args = parser.parse_args()
>> +
>> + # Extract the needed variables from the args
>> + command = args.command
>> +
>> + # Insure that valgrind is installed
>> + check_valgrind = subprocess.run(
>> + ["which", "valgrind"], stdout=subprocess.DEVNULL)
>> + if check_valgrind.returncode:
>> + sys.exit("Please install valgrind before running the script.")
>> +
>> + # Save all intermediate files in a temporary directory
>> + with tempfile.TemporaryDirectory() as tmpdirname:
>> + # callgrind output file path
>> + data_path = os.path.join(tmpdirname, "callgrind.data")
>> + # callgrind_annotate output file path
>> + annotate_out_path = os.path.join(tmpdirname,
>> "callgrind_annotate.out")
>> +
>> + # Run callgrind
>> + callgrind = subprocess.run((["valgrind",
>> + "--tool=callgrind",
>> + "--callgrind-out-file=" + data_path]
>> + + command),
>> + stdout=subprocess.DEVNULL,
>> + stderr=subprocess.PIPE)
>> + if callgrind.returncode:
>> + sys.exit(callgrind.stderr.decode("utf-8"))
>> +
>> + # Save callgrind_annotate output
>> + with open(annotate_out_path, "w") as output:
>> + callgrind_annotate = subprocess.run(
>> + ["callgrind_annotate", data_path,
>> + "--threshold=100", "--tree=calling"],
>> + stdout=output,
>> + stderr=subprocess.PIPE)
>> + if callgrind_annotate.returncode:
>> + sys.exit(callgrind_annotate.stderr.decode("utf-8"))
>> +
>> + # Read the callgrind_annotate output to callgrind_data[]
>> + callgrind_data = []
>> + with open(annotate_out_path, "r") as data:
>> + callgrind_data = data.readlines()
>> +
>> + # Line number with the total number of instructions
>> + total_instructions_line_number = 20
>> + # Get the total number of instructions
>> + total_instructions_line_data = \
>> + callgrind_data[total_instructions_line_number]
>> + total_instructions = total_instructions_line_data.split()[0]
>> +
>> + print("Total number of instructions:
>> {}\n".format(total_instructions))
>> +
>> + # Remove commas and convert to int
>> + total_instructions = int(total_instructions.replace(",", ""))
>> +
>> + # Line number with the JIT call
>> + JIT_line = find_JIT_line(callgrind_data)
>> + if JIT_line == -1:
>> + sys.exit("Couldn't locate the JIT call ... Exiting")
>> +
>> + # Get helpers
>> + helpers = get_helpers(JIT_line, callgrind_data)
>> +
>> + print("Executed QEMU Helpers:\n")
>> +
>> + # Print table header
>> + print("{:>4} {:>15} {:>10} {:>15} {:>10} {:<25} {}".
>> + format(
>> + "No.",
>> + "Instructions",
>> + "Percentage",
>> + "Calls",
>> + "Ins/Call",
>> + "Helper Name",
>> + "Source File")
>> + )
>> +
>> + print("{:>4} {:>15} {:>10} {:>15} {:>10} {:<25} {}".
>> + format(
>> + "-" * 4,
>> + "-" * 15,
>> + "-" * 10,
>> + "-" * 15,
>> + "-" * 10,
>> + "-" * 25,
>> + "-" * 30)
>> + )
>> +
>> + for (index, callee) in enumerate(helpers, start=1):
>> + instructions = callee[0]
>> + percentage = (callee[0] / total_instructions) * 100
>> + calls = callee[2]
>> + instruction_per_call = int(callee[0] / callee[2])
>> + helper_name = callee[1]
>> + source_file = callee[3]
>> + # Print extracted data
>> + print("{:>4} {:>15} {:>9.3f}% {:>15} {:>10} {:<25} {}".
>> + format(
>> + index,
>> + format(instructions, ","),
>> + round(percentage, 3),
>> + format(calls, ","),
>> + format(instruction_per_call, ","),
>> + helper_name,
>> + source_file)
>> + )
>> +
>> +
>> +if __name__ == "__main__":
>> + main()
>> --
>> 2.17.1
>>
Best regards,
Ahmed Karaman