Skip to content

-Zthreads=N support implemented as an additional dimension 'Parallel' #2421

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
azhogin wants to merge 1 commit into
base: master
Choose a base branch
from
+713 −275
Open

-Zthreads=N support implemented as an additional dimension 'Parallel' #2421

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
188 changes: 102 additions & 86 deletions ci/check-profiling.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -32,9 +32,10 @@ cargo build -p collector --bin rustc-fake
# --profiles Check \
# --cargo $bindir/cargo \
# --include helloworld \
# --scenarios Full
#test -f results/perf-Test-helloworld-Check-Full
#grep -q "PERFILE" results/perf-Test-helloworld-Check-Full
# --scenarios Full \
# --parallels 1
#test -f results/perf-Test-helloworld-Check-Full-par1
#grep -q "PERFILE" results/perf-Test-helloworld-Check-Full-par1

# oprofile: untested... it's not used much, and might have the same problems
# that `perf` has due to virtualized hardware.
Expand All @@ -50,13 +51,14 @@ RUST_BACKTRACE=1 RUST_LOG=raw_cargo_messages=trace,collector=debug,rust_sysroot=
--profiles Check \
--cargo $bindir/cargo \
--include helloworld \
--scenarios Full
test -f results/cgout-Test-helloworld-Check-Full
grep -q "events: Ir" results/cgout-Test-helloworld-Check-Full
test -f results/cgann-Test-helloworld-Check-Full
grep -q "PROGRAM TOTALS" results/cgann-Test-helloworld-Check-Full
--scenarios Full \
--parallels 1
test -f results/cgout-Test-helloworld-Check-Full-par1
grep -q "events: Ir" results/cgout-Test-helloworld-Check-Full-par1
test -f results/cgann-Test-helloworld-Check-Full-par1
grep -q "PROGRAM TOTALS" results/cgann-Test-helloworld-Check-Full-par1
# Ensure that we also profile the memory allocator
grep -q "malloc" results/cgann-Test-helloworld-Check-Full
grep -q "malloc" results/cgann-Test-helloworld-Check-Full-par1

# Callgrind.
RUST_BACKTRACE=1 RUST_LOG=raw_cargo_messages=trace,collector=debug,rust_sysroot=debug \
Expand All @@ -66,11 +68,12 @@ RUST_BACKTRACE=1 RUST_LOG=raw_cargo_messages=trace,collector=debug,rust_sysroot=
--profiles Check \
--cargo $bindir/cargo \
--include helloworld \
--scenarios Full
test -f results/clgout-Test-helloworld-Check-Full
grep -q "creator: callgrind" results/clgout-Test-helloworld-Check-Full
test -f results/clgann-Test-helloworld-Check-Full
grep -q "Profile data file" results/clgann-Test-helloworld-Check-Full
--scenarios Full \
--parallels 1
test -f results/clgout-Test-helloworld-Check-Full-par1
grep -q "creator: callgrind" results/clgout-Test-helloworld-Check-Full-par1
test -f results/clgann-Test-helloworld-Check-Full-par1
grep -q "Profile data file" results/clgann-Test-helloworld-Check-Full-par1

# DHAT.
RUST_BACKTRACE=1 RUST_LOG=raw_cargo_messages=trace,collector=debug,rust_sysroot=debug \
Expand All @@ -80,9 +83,10 @@ RUST_BACKTRACE=1 RUST_LOG=raw_cargo_messages=trace,collector=debug,rust_sysroot=
--profiles Check \
--cargo $bindir/cargo \
--include helloworld \
--scenarios Full
test -f results/dhout-Test-helloworld-Check-Full
grep -q "dhatFileVersion" results/dhout-Test-helloworld-Check-Full
--scenarios Full \
--parallels 1
test -f results/dhout-Test-helloworld-Check-Full-par1
grep -q "dhatFileVersion" results/dhout-Test-helloworld-Check-Full-par1


# DHAT (copy mode).
Expand All @@ -95,9 +99,10 @@ RUST_BACKTRACE=1 RUST_LOG=raw_cargo_messages=trace,collector=debug,rust_sysroot=
--profiles Check \
--cargo $bindir/cargo \
--include helloworld \
--scenarios Full
test -f results/dhcopy-Test-helloworld-Check-Full
grep -q "dhatFileVersion" results/dhcopy-Test-helloworld-Check-Full
--scenarios Full \
--parallels 1
test -f results/dhcopy-Test-helloworld-Check-Full-par1
grep -q "dhatFileVersion" results/dhcopy-Test-helloworld-Check-Full-par1

# Massif.
RUST_BACKTRACE=1 RUST_LOG=raw_cargo_messages=trace,collector=debug,rust_sysroot=debug \
Expand All @@ -107,9 +112,10 @@ RUST_BACKTRACE=1 RUST_LOG=raw_cargo_messages=trace,collector=debug,rust_sysroot=
--profiles Check \
--cargo $bindir/cargo \
--include helloworld \
--scenarios Full
test -f results/msout-Test-helloworld-Check-Full
grep -q "snapshot=0" results/msout-Test-helloworld-Check-Full
--scenarios Full \
--parallels 1
test -f results/msout-Test-helloworld-Check-Full-par1
grep -q "snapshot=0" results/msout-Test-helloworld-Check-Full-par1

# Bytehound.
# This is currently broken in CI, commenting out to fix CI for this.
Expand All @@ -120,8 +126,9 @@ grep -q "snapshot=0" results/msout-Test-helloworld-Check-Full
# --profiles Check \
# --cargo $bindir/cargo \
# --include helloworld \
# --scenarios Full
# test -f results/bhout-Test-helloworld-Check-Full
# --scenarios Full \
# --parallels 1
# test -f results/bhout-Test-helloworld-Check-Full-par1

# eprintln. The output file is empty because a vanilla rustc doesn't print
# anything to stderr.
Expand All @@ -132,9 +139,10 @@ RUST_BACKTRACE=1 RUST_LOG=raw_cargo_messages=trace,collector=debug,rust_sysroot=
--profiles Check \
--cargo $bindir/cargo \
--include helloworld \
--scenarios Full
test -f results/eprintln-Test-helloworld-Check-Full
test ! -s results/eprintln-Test-helloworld-Check-Full
--scenarios Full \
--parallels 1
test -f results/eprintln-Test-helloworld-Check-Full-par1
test ! -s results/eprintln-Test-helloworld-Check-Full-par1

# llvm-lines. `Debug` not `Check` because it doesn't support `Check` profiles.
# Including both `helloworld` and `regex-automata-0.4.8` benchmarks, as they exercise the
Expand All @@ -147,11 +155,12 @@ RUST_BACKTRACE=1 RUST_LOG=raw_cargo_messages=trace,collector=debug,rust_sysroot=
--profiles Debug \
--cargo $bindir/cargo \
--include helloworld,regex-automata-0.4.8 \
--scenarios Full
test -f results/ll-Test-helloworld-Debug-Full
grep -q "Lines.*Copies.*Function name" results/ll-Test-helloworld-Debug-Full
test -f results/ll-Test-regex-automata-0.4.8-Debug-Full
grep -q "Lines.*Copies.*Function name" results/ll-Test-regex-automata-0.4.8-Debug-Full
--scenarios Full \
--parallels 1
test -f results/ll-Test-helloworld-Debug-Full-par1
grep -q "Lines.*Copies.*Function name" results/ll-Test-helloworld-Debug-Full-par1
test -f results/ll-Test-regex-automata-0.4.8-Debug-Full-par1
grep -q "Lines.*Copies.*Function name" results/ll-Test-regex-automata-0.4.8-Debug-Full-par1

# llvm-ir. `Debug` not `Check` because it works better that way.
RUST_BACKTRACE=1 RUST_LOG=raw_cargo_messages=trace,collector=debug,rust_sysroot=debug \
Expand All @@ -161,9 +170,10 @@ RUST_BACKTRACE=1 RUST_LOG=raw_cargo_messages=trace,collector=debug,rust_sysroot=
--profiles Debug \
--cargo $bindir/cargo \
--include helloworld \
--scenarios Full
test -f results/llir-Test-helloworld-Debug-Full
grep -q "; ModuleID" results/llir-Test-helloworld-Debug-Full
--scenarios Full \
--parallels 1
test -f results/llir-Test-helloworld-Debug-Full-par1
grep -q "; ModuleID" results/llir-Test-helloworld-Debug-Full-par1

# mono-items. `Debug` not `Check` because it works better that way.
RUST_BACKTRACE=1 RUST_LOG=raw_cargo_messages=trace,collector=debug,rust_sysroot=debug \
Expand All @@ -173,9 +183,10 @@ RUST_BACKTRACE=1 RUST_LOG=raw_cargo_messages=trace,collector=debug,rust_sysroot=
--profiles Debug \
--cargo $bindir/cargo \
--include helloworld \
--scenarios Full
test -f results/mono-items-Test-helloworld-Debug-Full/raw
grep -q "MONO_ITEM" results/mono-items-Test-helloworld-Debug-Full/raw
--scenarios Full \
--parallels 1
test -f results/mono-items-Test-helloworld-Debug-Full-par1/raw
grep -q "MONO_ITEM" results/mono-items-Test-helloworld-Debug-Full-par1/raw

# dep-graph. `IncrFull` not `Full` because it doesn't work with `Full`.
RUST_BACKTRACE=1 RUST_LOG=raw_cargo_messages=trace,collector=debug,rust_sysroot=debug \
Expand All @@ -185,9 +196,10 @@ RUST_BACKTRACE=1 RUST_LOG=raw_cargo_messages=trace,collector=debug,rust_sysroot=
--profiles Check \
--cargo $bindir/cargo \
--include helloworld \
--scenarios IncrFull
test -f results/dep-graph-Test-helloworld-Check-IncrFull.txt
grep -q "hir_owner" results/dep-graph-Test-helloworld-Check-IncrFull.txt
--scenarios IncrFull \
--parallels 1
test -f results/dep-graph-Test-helloworld-Check-IncrFull-par1.txt
grep -q "hir_owner" results/dep-graph-Test-helloworld-Check-IncrFull-par1.txt

#----------------------------------------------------------------------------
# Test option handling
Expand All @@ -200,23 +212,24 @@ RUST_BACKTRACE=1 RUST_LOG=raw_cargo_messages=trace,collector=debug,rust_sysroot=
profile_local eprintln $bindir/rustc \
--id Builds1 \
--cargo $bindir/cargo \
--include helloworld
test -f results/eprintln-Builds1-helloworld-Check-Full
test -f results/eprintln-Builds1-helloworld-Check-IncrFull
test -f results/eprintln-Builds1-helloworld-Check-IncrPatched0
test -f results/eprintln-Builds1-helloworld-Check-IncrUnchanged
test -f results/eprintln-Builds1-helloworld-Debug-Full
test -f results/eprintln-Builds1-helloworld-Debug-IncrFull
test -f results/eprintln-Builds1-helloworld-Debug-IncrPatched0
test -f results/eprintln-Builds1-helloworld-Debug-IncrUnchanged
test -f results/eprintln-Builds1-helloworld-Opt-Full
test -f results/eprintln-Builds1-helloworld-Opt-IncrFull
test -f results/eprintln-Builds1-helloworld-Opt-IncrPatched0
test -f results/eprintln-Builds1-helloworld-Opt-IncrUnchanged
test ! -e results/eprintln-Builds1-helloworld-Doc-Full
test ! -e results/eprintln-Builds1-helloworld-Doc-IncrFull
test ! -e results/eprintln-Builds1-helloworld-Doc-IncrPatched0
test ! -e results/eprintln-Builds1-helloworld-Doc-IncrUnchanged
--include helloworld \
--parallels 1
test -f results/eprintln-Builds1-helloworld-Check-Full-par1
test -f results/eprintln-Builds1-helloworld-Check-IncrFull-par1
test -f results/eprintln-Builds1-helloworld-Check-IncrPatched0-par1
test -f results/eprintln-Builds1-helloworld-Check-IncrUnchanged-par1
test -f results/eprintln-Builds1-helloworld-Debug-Full-par1
test -f results/eprintln-Builds1-helloworld-Debug-IncrFull-par1
test -f results/eprintln-Builds1-helloworld-Debug-IncrPatched0-par1
test -f results/eprintln-Builds1-helloworld-Debug-IncrUnchanged-par1
test -f results/eprintln-Builds1-helloworld-Opt-Full-par1
test -f results/eprintln-Builds1-helloworld-Opt-IncrFull-par1
test -f results/eprintln-Builds1-helloworld-Opt-IncrPatched0-par1
test -f results/eprintln-Builds1-helloworld-Opt-IncrUnchanged-par1
test ! -e results/eprintln-Builds1-helloworld-Doc-Full-par1
test ! -e results/eprintln-Builds1-helloworld-Doc-IncrFull-par1
test ! -e results/eprintln-Builds1-helloworld-Doc-IncrPatched0-par1
test ! -e results/eprintln-Builds1-helloworld-Doc-IncrUnchanged-par1

# With `--profiles Doc` specified, `Check`/`Debug`/`Opt` files must not be
# present, and `Doc` files must be present (but not for incremental runs).
Expand All @@ -226,23 +239,24 @@ RUST_BACKTRACE=1 RUST_LOG=raw_cargo_messages=trace,collector=debug,rust_sysroot=
--id Builds2 \
--profiles Doc \
--cargo $bindir/cargo \
--include helloworld
test ! -e results/eprintln-Builds2-helloworld-Check-Full
test ! -e results/eprintln-Builds2-helloworld-Check-IncrFull
test ! -e results/eprintln-Builds2-helloworld-Check-IncrUnchanged
test ! -e results/eprintln-Builds2-helloworld-Check-IncrPatched0
test ! -e results/eprintln-Builds2-helloworld-Debug-Full
test ! -e results/eprintln-Builds2-helloworld-Debug-IncrFull
test ! -e results/eprintln-Builds2-helloworld-Debug-IncrUnchanged
test ! -e results/eprintln-Builds2-helloworld-Debug-IncrPatched0
test ! -e results/eprintln-Builds2-helloworld-Opt-Full
test ! -e results/eprintln-Builds2-helloworld-Opt-IncrFull
test ! -e results/eprintln-Builds2-helloworld-Opt-IncrUnchanged
test ! -e results/eprintln-Builds2-helloworld-Opt-IncrPatched0
test -f results/eprintln-Builds2-helloworld-Doc-Full
test ! -f results/eprintln-Builds2-helloworld-Doc-IncrFull
test ! -f results/eprintln-Builds2-helloworld-Doc-IncrPatched0
test ! -f results/eprintln-Builds2-helloworld-Doc-IncrUnchanged
--include helloworld \
--parallels 1
test ! -e results/eprintln-Builds2-helloworld-Check-Full-par1
test ! -e results/eprintln-Builds2-helloworld-Check-IncrFull-par1
test ! -e results/eprintln-Builds2-helloworld-Check-IncrUnchanged-par1
test ! -e results/eprintln-Builds2-helloworld-Check-IncrPatched0-par1
test ! -e results/eprintln-Builds2-helloworld-Debug-Full-par1
test ! -e results/eprintln-Builds2-helloworld-Debug-IncrFull-par1
test ! -e results/eprintln-Builds2-helloworld-Debug-IncrUnchanged-par1
test ! -e results/eprintln-Builds2-helloworld-Debug-IncrPatched0-par1
test ! -e results/eprintln-Builds2-helloworld-Opt-Full-par1
test ! -e results/eprintln-Builds2-helloworld-Opt-IncrFull-par1
test ! -e results/eprintln-Builds2-helloworld-Opt-IncrUnchanged-par1
test ! -e results/eprintln-Builds2-helloworld-Opt-IncrPatched0-par1
test -f results/eprintln-Builds2-helloworld-Doc-Full-par1
test ! -f results/eprintln-Builds2-helloworld-Doc-IncrFull-par1
test ! -f results/eprintln-Builds2-helloworld-Doc-IncrPatched0-par1
test ! -f results/eprintln-Builds2-helloworld-Doc-IncrUnchanged-par1

# With `--scenarios IncrUnchanged` specified, `IncrFull` and `IncrUnchanged`
# files must be present.
Expand All @@ -253,11 +267,12 @@ RUST_BACKTRACE=1 RUST_LOG=raw_cargo_messages=trace,collector=debug,rust_sysroot=
--profiles Check \
--cargo $bindir/cargo \
--include helloworld \
--scenarios IncrUnchanged
test ! -e results/eprintln-Runs1-helloworld-Check-Full
test -f results/eprintln-Runs1-helloworld-Check-IncrFull
test -f results/eprintln-Runs1-helloworld-Check-IncrUnchanged
test ! -e results/eprintln-Runs1-helloworld-Check-IncrPatched0
--scenarios IncrUnchanged \
--parallels 1
test ! -e results/eprintln-Runs1-helloworld-Check-Full-par1
test -f results/eprintln-Runs1-helloworld-Check-IncrFull-par1
test -f results/eprintln-Runs1-helloworld-Check-IncrUnchanged-par1
test ! -e results/eprintln-Runs1-helloworld-Check-IncrPatched0-par1

# With `--scenarios IncrPatched` specified, `IncrFull` and `IncrPatched0` files
# must be present.
Expand All @@ -268,11 +283,12 @@ RUST_BACKTRACE=1 RUST_LOG=raw_cargo_messages=trace,collector=debug,rust_sysroot=
--profiles Check \
--cargo $bindir/cargo \
--include helloworld \
--scenarios IncrPatched
test ! -e results/eprintln-Runs2-helloworld-Check-Full
test -f results/eprintln-Runs2-helloworld-Check-IncrFull
test ! -e results/eprintln-Runs2-helloworld-Check-IncrUnchanged
test -f results/eprintln-Runs2-helloworld-Check-IncrPatched0
--scenarios IncrPatched \
--parallels 1
test ! -e results/eprintln-Runs2-helloworld-Check-Full-par1
test -f results/eprintln-Runs2-helloworld-Check-IncrFull-par1
test ! -e results/eprintln-Runs2-helloworld-Check-IncrUnchanged-par1
test -f results/eprintln-Runs2-helloworld-Check-IncrPatched0-par1

kill $PING_LOOP_PID
exit 0
3 changes: 3 additions & 0 deletions collector/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -160,6 +160,8 @@ The following options alter the behaviour of the `bench_local` subcommand.
`IncrUnchanged`, `IncrPatched`, and `All`. The default is `All`. Note that
`IncrFull` is always run if either of `IncrUnchanged` or `IncrPatched` are
run (even if not requested).
- `--parallels <PARALLELS>`: the parallel frontend options to be benchmarked.
Comma-separated list of thread numbers (arguments for -Zthreads=N). The default is `1,4`.
- `--backends <BACKENDS>`: the codegen backends to be benchmarked. The possible
choices are one or more (comma-separated) of `Llvm`, `Cranelift`. The default
is `Llvm`.
Expand Down Expand Up @@ -489,6 +491,7 @@ The following options alter the behaviour of the `profile_local` subcommand.
diff files will also be produced.
- `--rustdoc <RUSTDOC>` as for `bench_local`.
- `--scenarios <SCENARIOS>`: as for `bench_local`.
- `--parallels <PARALLELS>`: as for `bench_local`.
- `--backends <BACKENDS>`: as for `bench_local`.
- `--jobs <JOB-COUNT>`: execute `<JOB-COUNT>` benchmarks in parallel. This is only allowed for certain
profilers whose results are not affected by system noise (e.g. `callgrind` or `eprintln`).
Expand Down
Loading
Loading