When running in the now default parallel mode this test has been
frequently failing, while when running exclusively, on a quiet system,
it passes.
Since its expectations were established when serial testing was the
norm, mark it as exclusive to get this kind of resunt:
root@x1:~# perf test 106
106: perf script task-analyzer tests : Ok
root@x1:~# set -o vi
root@x1:~# perf stat --null --repeat 10 perf test 106
106: perf script task-analyzer tests : Ok
106: perf script task-analyzer tests : Ok
106: perf script task-analyzer tests : Ok
106: perf script task-analyzer tests : Ok
106: perf script task-analyzer tests : Ok
106: perf script task-analyzer tests : Ok
106: perf script task-analyzer tests : Ok
106: perf script task-analyzer tests : Ok
106: perf script task-analyzer tests : Ok
106: perf script task-analyzer tests : Ok
Performance counter stats for 'perf test 106' (10 runs):
4.8872 +- 0.0179 seconds time elapsed ( +- 0.37% )
root@x1:~#
Cc: Aditya Gupta <adityag@linux.ibm.com>
Cc: Adrian Hunter <adrian.hunter@intel.com>
Cc: Alexander Shishkin <alexander.shishkin@linux.intel.com>
Cc: Athira Rajeev <atrajeev@linux.vnet.ibm.com>
Cc: Ian Rogers <irogers@google.com>
Cc: Ingo Molnar <mingo@redhat.com
Cc: James Clark <james.clark@linaro.org>
Cc: Jiri Olsa <jolsa@kernel.org>
Cc: Kan Liang <kan.liang@linux.intel.com>
Cc: Mark Rutland <mark.rutland@arm.com>
Cc: Namhyung Kim <namhyung@kernel.org
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Peter Zijlstra <peterz@infradead.org
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Now that printing metric-value and metric-unit is optional,
print_running_json() shouldn't add the comma in case it becomes
trailing.
Replace all manual JSON comma stuff with a json_out() function that uses
the existing os->first tracking and auto inserts a comma if it's needed.
Update the test to handle that two of the fields can be missing.
This fixes the following test failure on Cortex A57 where the branch
misses metric is missing a required event:
$ perf test -vvv "json output"
106: perf stat JSON output linter:
--- start ---
test child forked, pid 665682
Checking json output: no args Test failed for input:
{"counter-value" : "3112.000000", "unit" : "",
"event" : "armv8_pmuv3_1/branch-misses/",
"event-runtime" : 20699340, "pcnt-running" : 100.00, }
...
json.decoder.JSONDecodeError: Expecting property name enclosed in
double quotes: line 12 column 144 (char 2109)
---- end(-1) ----
106: perf stat JSON output linter : FAILED!
Fixes: e1cc918b6c ("perf stat: Drop metric-unit if unit is NULL")
Signed-off-by: James Clark <james.clark@linaro.org>
Tested-by: Ian Rogers <irogers@google.com>
Cc: Adrian Hunter <adrian.hunter@intel.com>
Cc: Alexander Shishkin <alexander.shishkin@linux.intel.com>
Cc: Ian Rogers <irogers@google.com>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: Jiri Olsa <jolsa@kernel.org>
Cc: Kan Liang <kan.liang@linux.intel.com>
Cc: Mark Rutland <mark.rutland@arm.com>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Tim Chen <tim.c.chen@linux.intel.com>
Cc: Yicong Yang <yangyicong@hisilicon.com>
Link: https://lore.kernel.org/r/20241112160048.951213-2-james.clark@linaro.org
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Currently, we only have 'perf trace' augmentation tests for enum
arguments. This patch adds tests for more general syscall arguments,
such as struct pointers, strings, and buffers.
These tests utilize the 'perf config' system to configure 'the perf trace'
output, as suggested by Arnaldo Carvalho de Melo <acme@kernel.org>.
Committer testing:
root@number:~# perf test "BTF general"
109: perf trace BTF general tests : Ok
root@number:~# perf test -v "BTF general"
109: perf trace BTF general tests : Ok
root@number:~# perf test -vv "BTF general"
109: perf trace BTF general tests:
--- start ---
test child forked, pid 1410451
Checking if vmlinux BTF exists
Testing perf trace's string augmentation
Testing perf trace's buffer augmentation
Testing perf trace's struct augmentation
---- end(0) ----
109: perf trace BTF general tests : Ok
root@number:~#
It still fails sometimes, for instance when tested with:
root@number:~# perf stat --null -r 10 perf test "BTF general"
109: perf trace BTF general tests : Ok
109: perf trace BTF general tests : Ok
109: perf trace BTF general tests : Ok
109: perf trace BTF general tests : Ok
109: perf trace BTF general tests : FAILED!
109: perf trace BTF general tests : Ok
109: perf trace BTF general tests : Ok
109: perf trace BTF general tests : FAILED!
109: perf trace BTF general tests : Ok
109: perf trace BTF general tests : Ok
Performance counter stats for 'perf test BTF general' (10 runs):
2.148 +- 0.293 seconds time elapsed ( +- 13.63% )
root@number:~#
But we can go on from here and fix things up with followup patches.
Suggested-by: Arnaldo Carvalho de Melo <acme@kernel.org>
Signed-off-by: Howard Chu <howardchu95@gmail.com>
Acked-by: Namhyung Kim <namhyung@kernel.org>
Tested-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Adrian Hunter <adrian.hunter@intel.com>
Cc: Alexander Shishkin <alexander.shishkin@linux.intel.com>
Cc: Ian Rogers <irogers@google.com>
Cc: Jiri Olsa <jolsa@kernel.org>
Cc: Kan Liang <kan.liang@linux.intel.com>
Cc: Mark Rutland <mark.rutland@arm.com>
Link: https://lore.kernel.org/r/20241215190712.787847-2-howardchu95@gmail.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Currently print_overall_results prints the number of fails in the
summary, example from base_probe tests in testsuite_probe:
## [ FAIL ] ## perf_probe :: test_invalid_options SUMMARY ::
11 failures found
test_invalid_options contains multiple tests and out of that 11 failed.
Sometimes it could happen that it is due to missing dependency in the
build or environment dependency.
Example, perf probe -L requires DWARF enabled. otherwise
it fails as below:
./perf probe -L
Error: switch `L' is not available because NO_DWARF=1
"-L" is tested as one of the option in:
for opt in '-a' '-d' '-L' '-V'; do
<<perf probe test>>
print_results $PERF_EXIT_CODE $CHECK_EXIT_CODE "missing argument
for $opt"
Here -a and -d doesn't require DWARF. Similarly there are few other
tests requiring DWARF.
To hint the user that missing DWARF could be one issue, update
print_overall_results to print a comment string along with summary
hinting the possible cause. Update test_invalid_options.sh and
test_line_semantics.sh to pass the info about DWARF requirement since
these tests failed when perf is built without DWARF.
Use the check for presence of DWARF with "perf check feature" and append
the hint message based on the result.
With the change:
## [ FAIL ] ## perf_probe :: test_invalid_options SUMMARY ::
11 failures found :: Some of the tests need DWARF to run
Reviewed-by: Kajol Jain <kjain@linux.ibm.com>
Signed-off-by: Athira Rajeev <atrajeev@linux.vnet.ibm.com>
Cc: Adrian Hunter <adrian.hunter@intel.com>
Cc: Disha Goel <disgoel@linux.vnet.ibm.com>
Cc: Ian Rogers <irogers@google.com>
Cc: Jiri Olsa <jolsa@kernel.org>
Cc: Madhavan Srinivasan <maddy@linux.ibm.com>
Cc: Michael Petlan <mpetlan@redhat.com>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Veronika Molnarova <vmolnaro@redhat.com>
Cc: linuxppc-dev@lists.ozlabs.org
Link: https://lore.kernel.org/r/20241206135254.35727-1-atrajeev@linux.vnet.ibm.com
[ Minor edits changing "dwarf" to "DWARF" as its an acronym ]
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
The 'perf stat' output on aarch64 machines with topdown events wasn't
counted for in the 'perf stat STD output linter' test case. Add the
topdown metric to the skip_metric list as it is done for topdown events
on other systems.
The Topdown events are also disabled on aarch64 KVM guests because the
value of caps/slots is set to 0 due to the part of the system register
being a stub.
This prevents the metric for the topdown events from being computed,
leaving the 'perf stat' topdown metric without any value at all.
Add the "TopdownL1" to the skip_metric list as well to handle this
possibility.
Before aarch64:
100: perf stat STD output linter:
--- start ---
test child forked, pid 403305
Checking STD output: no args Unknown event name in TopdownL1 # 4.3 percent of slots slots_lost_misspeculation_fraction
---- end(-1) ----
100: perf stat STD output linter : FAILED!
Before aarch64 KVM:
100: perf stat STD output linter:
--- start ---
test child forked, pid 404671
Checking STD output: no args Unknown event name in TopdownL1
---- end(-1) ----
100: perf stat STD output linter : FAILED!
After:
100: perf stat STD output linter:
--- start ---
test child forked, pid 404777
Checking STD output: no args [Success]
Checking STD output: system wide [Success]
Checking STD output: interval [Success]
Checking STD output: per thread [Success]
Checking STD output: per node [Success]
Checking STD output: system wide no aggregation [Success]
Checking STD output: per core [Success]
Checking STD output: per cache instance [Success]
Checking STD output: per cluster [Success]
Checking STD output: per die [Success]
Checking STD output: per socket [Success]
---- end(0) ----
100: perf stat STD output linter : Ok
Signed-off-by: Veronika Molnarova <vmolnaro@redhat.com>
Acked-by: Ian Rogers <irogers@google.com>
Cc: Adrian Hunter <adrian.hunter@intel.com>
Cc: Alexander Shishkin <alexander.shishkin@linux.intel.com>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: Jiri Olsa <jolsa@kernel.org>
Cc: Kan Liang <kan.liang@linux.intel.com>
Cc: Mark Rutland <mark.rutland@arm.com>
Cc: Michael Petlan <mpetlan@redhat.com>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Tim Chen <tim.c.chen@linux.intel.com>
Cc: Yicong Yang <yangyicong@hisilicon.com>
Link: https://lore.kernel.org/r/20241029144347.25651-1-vmolnaro@redhat.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
The temp directory is made and a known fake hwmon PMU created within
it. Prior to this fix the events were being incorrectly written to the
temp directory rather than the fake PMU directory. This didn't impact
the test as the directory fd matched the wrong location, but it
doesn't mirror what a hwmon PMU would actually look like.
Signed-off-by: Ian Rogers <irogers@google.com>
Tested-by: Arnaldo Carvalho de Melo <acme@kernel.org>
Link: https://lore.kernel.org/r/20241206042306.1055913-2-irogers@google.com
Signed-off-by: Namhyung Kim <namhyung@kernel.org>
The perf_cpu_map__merge() function has two arguments, 'orig' and
'other'. The function definition might cause confusion as it could give
the impression that the CPU maps in the two arguments are copied into a
new allocated structure, which is then returned as the result.
The purpose of the function is to merge the CPU map 'other' into the CPU
map 'orig'. This commit changes the 'orig' argument to a pointer to
pointer, so the new result will be updated into 'orig'.
The return value is changed to an int type, as an error number or 0 for
success.
Update callers and tests for the new function definition.
Reviewed-by: Adrian Hunter <adrian.hunter@intel.com>
Signed-off-by: Leo Yan <leo.yan@arm.com>
Tested-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Alexander Shishkin <alexander.shishkin@linux.intel.com>
Cc: Ian Rogers <irogers@google.com>
Cc: James Clark <james.clark@linaro.org>
Cc: Jiri Olsa <jolsa@kernel.org>
Cc: Kan Liang <kan.liang@linux.intel.com>
Cc: Mark Rutland <mark.rutland@arm.com>
Cc: Namhyung Kim <namhyung@kernel.org>
Link: https://lore.kernel.org/r/20241107125308.41226-2-leo.yan@arm.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
This signal handler loops over all tests on ctrl-C, but it's active
while the test list is being constructed. process.pid is 0, then -1,
then finally set to the child pid on fork. If the Ctrl-C is received
during this point a kill(-1, SIGINT) can be sent which affects all
processes.
Make sure the child has forked first before forwarding the signal. This
can be reproduced with ctrl-C immediately after launching perf test
which terminates the ssh connection.
Fixes: 553d5efeb3 ("perf test: Add a signal handler to kill forked child processes")
Signed-off-by: James Clark <james.clark@linaro.org>
Reviewed-by: Ian Rogers <irogers@google.com>
Link: https://lore.kernel.org/r/20241129151948.3199732-1-james.clark@linaro.org
Signed-off-by: Namhyung Kim <namhyung@kernel.org>
Incorrectly the hwmon with PMU name test didn't pass "true". Fix and
address issue with hwmon_pmu__config_terms needing to load events - a
load bearing assert fired. Also fix missing list deletion when putting
the hwmon test PMU and lower some debug warnings to make the hwmon PMU
less spammy in verbose mode.
Signed-off-by: Ian Rogers <irogers@google.com>
Tested-by: Namhyung Kim <namhyung@kernel.org>
Link: https://lore.kernel.org/r/20241121000955.536930-1-irogers@google.com
Signed-off-by: Namhyung Kim <namhyung@kernel.org>
On s390 the perf test case ftrace sometimes fails as follows:
# ./perf test ftrace
79: perf ftrace tests : FAILED!
#
The failure depends on the kernel .config file. Some configurations
always work fine, some do not. The ftrace profile test mostly fails,
because the ring buffer was not large enough, and some lines
(especially the interesting ones with nanosleep in it) where dropped.
To achieve success for all tested kernel configurations, enlarge
the buffer to store the traces completely without wrapping.
The default buffer size is too small for all kernel configurations.
Set the buffer size of for the ftrace profile test to 16 MB.
Output after:
# ./perf test ftrace
79: perf ftrace tests : Ok
#
Signed-off-by: Thomas Richter <tmricht@linux.ibm.com>
Acked-by: Namhyung Kim <namhyung@kernel.org>
Cc: agordeev@linux.ibm.com
Cc: gor@linux.ibm.com
Cc: hca@linux.ibm.com
Cc: sumanthk@linux.ibm.com
Link: https://lore.kernel.org/r/20241119064856.641446-1-tmricht@linux.ibm.com
Suggested-by: Sven Schnelle <svens@linux.ibm.com>
Suggested-by: Namhyung Kim <namhyung@kernel.org>
Signed-off-by: Namhyung Kim <namhyung@kernel.org>
The purpose of this test is to test for races in the exit of 'perf
trace' missing the last events, it was failing when the COMM wasn't
resolved either because we missed some PERF_RECORD_COMM or somehow
raced on getting it from procfs.
Add --no-comm to the 'perf trace' command line so that we get a
consistent, pid only output, which allows the test to achieve its goal.
This is the output from
'perf trace --no-comm -e syscalls:sys_enter_exit_group':
0.000 21953 syscalls:sys_enter_exit_group()
0.000 21955 syscalls:sys_enter_exit_group()
0.000 21957 syscalls:sys_enter_exit_group()
0.000 21959 syscalls:sys_enter_exit_group()
0.000 21961 syscalls:sys_enter_exit_group()
0.000 21963 syscalls:sys_enter_exit_group()
0.000 21965 syscalls:sys_enter_exit_group()
0.000 21967 syscalls:sys_enter_exit_group()
0.000 21969 syscalls:sys_enter_exit_group()
0.000 21971 syscalls:sys_enter_exit_group()
Now it passes:
root@number:~# perf test "trace exit race"
110: perf trace exit race : Ok
root@number:~#
root@number:~# perf test -v "trace exit race"
110: perf trace exit race : Ok
root@number:~#
If we artificially make it run just 9 times instead of the 10 it runs,
i.e. by manually doing:
trace_shutdown_race() {
for _ in $(seq 9); do
that 9 is $iter, 10 in the patch, we get:
root@number:~# vim ~acme/libexec/perf-core/tests/shell/trace_exit_race.sh
root@number:~# perf test -v "trace exit race"
--- start ---
test child forked, pid 24629
Missing output, expected 10 but only got 9
---- end(-1) ----
110: perf trace exit race : FAILED!
root@number:~#
I.e. 9 'perf trace' calls produced the expected output, the inverse grep
didn't show anything, so the patch provided by Howard for the previous
patch kicks in and shows a more informative message.
Tested-by: Howard Chu <howardchu95@gmail.com>
Cc: Adrian Hunter <adrian.hunter@intel.com>
Cc: Alexander Shishkin <alexander.shishkin@linux.intel.com>
Cc: Benjamin Peterson <benjamin@engflow.com>
Cc: Ian Rogers <irogers@google.com>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: Jiri Olsa <jolsa@kernel.org>
Cc: Kan Liang <kan.liang@linux.intel.com>
Cc: Mark Rutland <mark.rutland@arm.com>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Link: https://lore.kernel.org/lkml/ZzdknoHqrJbojb6P@x1
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
If it fails we need to check what was the reason, what were the lines
that didn't match the expected format, so:
root@number:~# perf test -v "trace exit race"
--- start ---
test child forked, pid 2028724
Lines not matching the expected regexp: ' +[0-9]+\.[0-9]+ +true/[0-9]+ syscalls:sys_enter_exit_group\(\)$':
0.000 :2028750/2028750 syscalls:sys_enter_exit_group()
---- end(-1) ----
110: perf trace exit race : FAILED!
root@number:~#
In this case we're not resolving the process COMM for some reason and
fallback to printing just the pid/tid, this will be fixed in a followup
patch.
Howard Chu spotted a problem with single code surrounding a regexp, that
made the test always fail, but since there were some failures when I
tested (COMM not being resolved in some of the results) the end inverse
grep would show some lines and thus didn't notice the single quote
problem.
He also provided a patch to test if less than the number of expected
matches took place but all of them with the expected output, in which
case the inverse grep wouldn't show anything, confusing the tester.
Reviewed-by: Howard Chu <howardchu95@gmail.com>
Cc: Adrian Hunter <adrian.hunter@intel.com>
Cc: Alexander Shishkin <alexander.shishkin@linux.intel.com>
Cc: Benjamin Peterson <benjamin@engflow.com>
Cc: Ian Rogers <irogers@google.com>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: Jiri Olsa <jolsa@kernel.org>
Cc: Kan Liang <kan.liang@linux.intel.com>
Cc: Mark Rutland <mark.rutland@arm.com>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Link: https://lore.kernel.org/lkml/ZzdknoHqrJbojb6P@x1
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
During the parallel testing, I've noticed some ftrace test failures. It
seems the regex pattern checks 100 msec of nanosleep with the error
range of 10 msec. But sometimes it's affected by other processes and
resulted in more time in the syscall.
The following output shows that it took more than 120 msec and failed.
Let's update the regex pattern so that it can allow more drifts.
perf ftrace profile test
# Total (us) Avg (us) Max (us) Count Function
121279.500 121279.500 121279.500 1 __x64_sys_clock_nanosleep
121278.400 121278.400 121278.400 1 common_nsleep
121277.800 121277.800 121277.800 1 hrtimer_nanosleep
121277.100 121277.100 121277.100 1 do_nanosleep
341760.289 56960.048 121273.400 6 schedule
176.200 25.171 31.616 7 scheduler_tick
0.923 0.923 0.923 1 native_smp_send_reschedule
345522.360 69104.472 345320.600 5 __x64_sys_execve
345486.585 69097.317 345312.700 5 do_execveat_common.isra.0
340730.300 340730.300 340730.300 1 bprm_execve
1.758 0.879 0.883 2 sched_mm_cid_before_execve
1.112 1.112 1.112 1 sched_mm_cid_after_execve
---- end(-1) ----
81: perf ftrace tests : FAILED!
Reviewed-by: Ian Rogers <irogers@google.com>
Link: https://lore.kernel.org/r/20241102231702.2262258-1-namhyung@kernel.org
Signed-off-by: Namhyung Kim <namhyung@kernel.org>
The cpu-list part of this testcase has proven itself to be unreliable.
Sometimes, we get "<not counted>" for system.slice when pinned to CPUs
0 and 1. In such case, the test fails.
Since we cannot simply guarantee that any system.slice load will run
on any arbitrary list of CPUs, except the whole set of all CPUs, let's
rather remove the cpu-list subtest.
Fixes: a84260e314 ("perf test stat_bpf_counters_cgrp: Enhance perf stat cgroup BPF counter test")
Signed-off-by: Michael Petlan <mpetlan@redhat.com>
Cc: vmolnaro@redhat.com
Cc: bpf@vger.kernel.org
Link: https://lore.kernel.org/r/20241101102812.576425-1-mpetlan@redhat.com
Signed-off-by: Namhyung Kim <namhyung@kernel.org>
