Skip to content

don't merge test#718

Draft
0xdeafbeef wants to merge 1 commit intorust-lang:masterfrom
0xdeafbeef:0xdeafbeef/push-mqvvksuspult
Draft

don't merge test#718
0xdeafbeef wants to merge 1 commit intorust-lang:masterfrom
0xdeafbeef:0xdeafbeef/push-mqvvksuspult

Conversation

@0xdeafbeef
Copy link
Copy Markdown
Contributor

cold_path was stabilized so i wanted to test how will it affect codegen.
It's a mixed bag but numbers are curious.

insert_foldhash_serial  time:   [9.1591 µs 9.2024 µs 9.2473 µs]
                        change: [−5.6410% −5.0708% −4.5204%] (p = 0.00 < 0.05)
                        Performance has improved.

insert_std_serial       time:   [17.560 µs 17.595 µs 17.630 µs]
                        change: [−0.8661% −0.5510% −0.2327%] (p = 0.00 < 0.05)
                        Change within noise threshold.
Found 1 outliers among 100 measurements (1.00%)
  1 (1.00%) high mild

insert_foldhash_highbits
                        time:   [9.9706 µs 10.009 µs 10.047 µs]
                        change: [−1.4015% −0.8995% −0.4049%] (p = 0.00 < 0.05)
                        Change within noise threshold.
Found 2 outliers among 100 measurements (2.00%)
  2 (2.00%) high mild

insert_std_highbits     time:   [17.644 µs 17.697 µs 17.752 µs]
                        change: [+1.3172% +1.6272% +1.9324%] (p = 0.00 < 0.05)
                        Performance has regressed.

insert_foldhash_random  time:   [9.8794 µs 9.9281 µs 9.9756 µs]
                        change: [−0.7432% −0.2771% +0.1486%] (p = 0.23 > 0.05)
                        No change in performance detected.
Found 1 outliers among 100 measurements (1.00%)
  1 (1.00%) low mild

insert_std_random       time:   [17.487 µs 17.528 µs 17.572 µs]
                        change: [−2.0907% −1.7920% −1.4810%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 3 outliers among 100 measurements (3.00%)
  2 (2.00%) high mild
  1 (1.00%) high severe

grow_insert_foldhash_serial
                        time:   [21.616 µs 21.633 µs 21.657 µs]
                        change: [+27.009% +27.156% +27.298%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 7 outliers among 100 measurements (7.00%)
  2 (2.00%) high mild
  5 (5.00%) high severe

grow_insert_std_serial  time:   [36.439 µs 36.455 µs 36.472 µs]
                        change: [−3.6564% −3.5539% −3.4338%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 4 outliers among 100 measurements (4.00%)
  1 (1.00%) high mild
  3 (3.00%) high severe

grow_insert_foldhash_highbits
                        time:   [17.708 µs 17.717 µs 17.727 µs]
                        change: [+11.994% +12.106% +12.211%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 10 outliers among 100 measurements (10.00%)
  1 (1.00%) low mild
  5 (5.00%) high mild
  4 (4.00%) high severe

grow_insert_std_highbits
                        time:   [36.272 µs 36.320 µs 36.380 µs]
                        change: [−0.9309% −0.8155% −0.6816%] (p = 0.00 < 0.05)
                        Change within noise threshold.
Found 11 outliers among 100 measurements (11.00%)
  8 (8.00%) high mild
  3 (3.00%) high severe

grow_insert_foldhash_random
                        time:   [18.265 µs 18.287 µs 18.315 µs]
                        change: [−1.4631% −1.3663% −1.2715%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 7 outliers among 100 measurements (7.00%)
  4 (4.00%) high mild
  3 (3.00%) high severe

grow_insert_std_random  time:   [36.200 µs 36.230 µs 36.271 µs]
                        change: [−1.4314% −1.3330% −1.2280%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 14 outliers among 100 measurements (14.00%)
  4 (4.00%) high mild
  10 (10.00%) high severe

insert_erase_foldhash_serial
                        time:   [10.674 µs 10.730 µs 10.787 µs]
                        change: [+2.9141% +3.3974% +3.9207%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 2 outliers among 100 measurements (2.00%)
  1 (1.00%) high mild
  1 (1.00%) high severe

insert_erase_std_serial time:   [25.181 µs 25.247 µs 25.317 µs]
                        change: [+2.0929% +2.3283% +2.5743%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 4 outliers among 100 measurements (4.00%)
  1 (1.00%) low mild
  2 (2.00%) high mild
  1 (1.00%) high severe

insert_erase_foldhash_highbits
                        time:   [10.241 µs 10.277 µs 10.316 µs]
                        change: [+0.9029% +1.2903% +1.7036%] (p = 0.00 < 0.05)
                        Change within noise threshold.
Found 8 outliers among 100 measurements (8.00%)
  5 (5.00%) high mild
  3 (3.00%) high severe

insert_erase_std_highbits
                        time:   [25.415 µs 25.477 µs 25.545 µs]
                        change: [+1.8273% +2.0815% +2.3278%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 3 outliers among 100 measurements (3.00%)
  3 (3.00%) high mild

insert_erase_foldhash_random
                        time:   [10.661 µs 10.696 µs 10.733 µs]
                        change: [+1.2696% +1.5684% +1.8594%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 3 outliers among 100 measurements (3.00%)
  3 (3.00%) high mild

insert_erase_std_random time:   [25.809 µs 25.916 µs 26.029 µs]
                        change: [+5.0430% +5.4167% +5.7420%] (p = 0.00 < 0.05)
                        Performance has regressed.

lookup_foldhash_serial  time:   [2.1241 µs 2.1318 µs 2.1402 µs]
                        change: [+39.655% +40.094% +40.548%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 8 outliers among 100 measurements (8.00%)
  5 (5.00%) high mild
  3 (3.00%) high severe

lookup_std_serial       time:   [9.3064 µs 9.3163 µs 9.3269 µs]
                        change: [+7.2886% +7.4283% +7.5592%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 5 outliers among 100 measurements (5.00%)
  4 (4.00%) high mild
  1 (1.00%) high severe

lookup_foldhash_highbits
                        time:   [2.1214 µs 2.1249 µs 2.1287 µs]
                        change: [+34.747% +34.994% +35.244%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 5 outliers among 100 measurements (5.00%)
  4 (4.00%) high mild
  1 (1.00%) high severe

lookup_std_highbits     time:   [9.5547 µs 9.5665 µs 9.5806 µs]
                        change: [+8.2415% +8.4106% +8.5805%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 4 outliers among 100 measurements (4.00%)
  3 (3.00%) high mild
  1 (1.00%) high severe

lookup_foldhash_random  time:   [2.2481 µs 2.2526 µs 2.2575 µs]
                        change: [+29.224% +29.900% +30.461%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 3 outliers among 100 measurements (3.00%)
  2 (2.00%) high mild
  1 (1.00%) high severe

lookup_std_random       time:   [9.3170 µs 9.3259 µs 9.3354 µs]
                        change: [+8.2626% +8.5006% +8.7102%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 9 outliers among 100 measurements (9.00%)
  6 (6.00%) high mild
  3 (3.00%) high severe

lookup_fail_foldhash_serial
                        time:   [3.0486 µs 3.1287 µs 3.2142 µs]
                        change: [+59.179% +62.447% +65.783%] (p = 0.00 < 0.05)
                        Performance has regressed.

lookup_fail_std_serial  time:   [9.2911 µs 9.3026 µs 9.3161 µs]
                        change: [+3.0660% +3.2486% +3.4319%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 4 outliers among 100 measurements (4.00%)
  3 (3.00%) high mild
  1 (1.00%) high severe

lookup_fail_foldhash_highbits
                        time:   [2.2387 µs 2.2419 µs 2.2454 µs]
                        change: [+14.592% +14.756% +14.917%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 5 outliers among 100 measurements (5.00%)
  5 (5.00%) high mild

lookup_fail_std_highbits
                        time:   [9.4587 µs 9.4722 µs 9.4860 µs]
                        change: [+4.9900% +5.2027% +5.4241%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 3 outliers among 100 measurements (3.00%)
  2 (2.00%) high mild
  1 (1.00%) high severe

lookup_fail_foldhash_random
                        time:   [2.3134 µs 2.3248 µs 2.3374 µs]
                        change: [+10.489% +11.109% +11.710%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 3 outliers among 100 measurements (3.00%)
  3 (3.00%) high mild

lookup_fail_std_random  time:   [9.4884 µs 9.5023 µs 9.5170 µs]
                        change: [+6.0421% +6.2367% +6.4397%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 5 outliers among 100 measurements (5.00%)
  5 (5.00%) high mild

loadfactor_lookup_14500 time:   [1.2443 µs 1.2471 µs 1.2506 µs]
                        change: [+24.049% +24.306% +24.580%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 4 outliers among 100 measurements (4.00%)
  3 (3.00%) high mild
  1 (1.00%) high severe

loadfactor_lookup_16500 time:   [1.2474 µs 1.2501 µs 1.2530 µs]
                        change: [+24.103% +24.411% +24.722%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 5 outliers among 100 measurements (5.00%)
  3 (3.00%) high mild
  2 (2.00%) high severe

loadfactor_lookup_18500 time:   [1.2458 µs 1.2478 µs 1.2497 µs]
                        change: [+23.954% +24.182% +24.410%] (p = 0.00 < 0.05)
                        Performance has regressed.

loadfactor_lookup_20500 time:   [1.2408 µs 1.2426 µs 1.2446 µs]
                        change: [+22.984% +23.612% +23.994%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 4 outliers among 100 measurements (4.00%)
  4 (4.00%) high mild

loadfactor_lookup_22500 time:   [1.2418 µs 1.2444 µs 1.2472 µs]
                        change: [+23.263% +23.813% +24.235%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 6 outliers among 100 measurements (6.00%)
  5 (5.00%) high mild
  1 (1.00%) high severe

loadfactor_lookup_24500 time:   [1.2388 µs 1.2408 µs 1.2429 µs]
                        change: [+23.708% +24.122% +24.576%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 3 outliers among 100 measurements (3.00%)
  3 (3.00%) high severe

loadfactor_lookup_26500 time:   [1.2386 µs 1.2407 µs 1.2429 µs]
                        change: [+23.523% +23.935% +24.543%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 5 outliers among 100 measurements (5.00%)
  4 (4.00%) high mild
  1 (1.00%) high severe

loadfactor_lookup_28500 time:   [1.2364 µs 1.2384 µs 1.2407 µs]
                        change: [+23.494% +23.730% +23.953%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 6 outliers among 100 measurements (6.00%)
  5 (5.00%) high mild
  1 (1.00%) high severe

loadfactor_lookup_fail_14500
                        time:   [1.1322 µs 1.1333 µs 1.1346 µs]
                        change: [+11.279% +11.455% +11.644%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 8 outliers among 100 measurements (8.00%)
  3 (3.00%) high mild
  5 (5.00%) high severe

loadfactor_lookup_fail_16500
                        time:   [1.1926 µs 1.1941 µs 1.1958 µs]
                        change: [+10.273% +10.485% +10.684%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 5 outliers among 100 measurements (5.00%)
  5 (5.00%) high mild

loadfactor_lookup_fail_18500
                        time:   [1.2837 µs 1.2869 µs 1.2902 µs]
                        change: [+8.8021% +9.1120% +9.4318%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 1 outliers among 100 measurements (1.00%)
  1 (1.00%) high mild

loadfactor_lookup_fail_20500
                        time:   [1.4434 µs 1.4483 µs 1.4533 µs]
                        change: [+7.2597% +7.7930% +8.3177%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 3 outliers among 100 measurements (3.00%)
  1 (1.00%) low mild
  1 (1.00%) high mild
  1 (1.00%) high severe

loadfactor_lookup_fail_22500
                        time:   [1.7366 µs 1.7456 µs 1.7550 µs]
                        change: [+4.7127% +5.4345% +6.0816%] (p = 0.00 < 0.05)
                        Performance has regressed.

loadfactor_lookup_fail_24500
                        time:   [2.3189 µs 2.3356 µs 2.3528 µs]
                        change: [+2.4425% +3.1791% +3.8580%] (p = 0.00 < 0.05)
                        Performance has regressed.

loadfactor_lookup_fail_26500
                        time:   [3.4285 µs 3.4472 µs 3.4654 µs]
                        change: [+0.7122% +1.4641% +2.2330%] (p = 0.00 < 0.05)
                        Change within noise threshold.
Found 1 outliers among 100 measurements (1.00%)
  1 (1.00%) high mild

loadfactor_lookup_fail_28500
                        time:   [5.4096 µs 5.4396 µs 5.4693 µs]
                        change: [+1.8399% +2.3830% +2.9455%] (p = 0.00 < 0.05)
                        Performance has regressed.

iter_foldhash_serial    time:   [607.93 ns 608.71 ns 609.55 ns]
                        change: [+0.6971% +0.9701% +1.2546%] (p = 0.00 < 0.05)
                        Change within noise threshold.
Found 5 outliers among 100 measurements (5.00%)
  4 (4.00%) high mild
  1 (1.00%) high severe

iter_std_serial         time:   [600.83 ns 601.52 ns 602.35 ns]
                        change: [−0.4747% −0.3185% −0.1557%] (p = 0.00 < 0.05)
                        Change within noise threshold.
Found 7 outliers among 100 measurements (7.00%)
  3 (3.00%) high mild
  4 (4.00%) high severe

iter_foldhash_highbits  time:   [599.81 ns 600.64 ns 601.60 ns]
                        change: [−0.4785% −0.2904% −0.1226%] (p = 0.00 < 0.05)
                        Change within noise threshold.
Found 10 outliers among 100 measurements (10.00%)
  7 (7.00%) high mild
  3 (3.00%) high severe

iter_std_highbits       time:   [600.79 ns 601.47 ns 602.18 ns]
                        change: [−0.4172% −0.2375% −0.0759%] (p = 0.01 < 0.05)
                        Change within noise threshold.
Found 5 outliers among 100 measurements (5.00%)
  4 (4.00%) high mild
  1 (1.00%) high severe

iter_foldhash_random    time:   [600.28 ns 600.74 ns 601.24 ns]
                        change: [−0.9655% −0.6027% −0.2860%] (p = 0.00 < 0.05)
                        Change within noise threshold.
Found 7 outliers among 100 measurements (7.00%)
  6 (6.00%) high mild
  1 (1.00%) high severe

iter_std_random         time:   [600.68 ns 601.40 ns 602.27 ns]
                        change: [−0.8302% −0.5227% −0.2829%] (p = 0.00 < 0.05)
                        Change within noise threshold.
Found 6 outliers among 100 measurements (6.00%)
  4 (4.00%) high mild
  2 (2.00%) high severe

clone_small             time:   [23.531 ns 23.555 ns 23.585 ns]
                        change: [−0.6116% −0.4539% −0.2973%] (p = 0.00 < 0.05)
                        Change within noise threshold.
Found 6 outliers among 100 measurements (6.00%)
  3 (3.00%) high mild
  3 (3.00%) high severe

clone_from_small        time:   [19.243 ns 19.264 ns 19.286 ns]
                        change: [−0.9831% −0.6455% −0.3443%] (p = 0.00 < 0.05)
                        Change within noise threshold.
Found 4 outliers among 100 measurements (4.00%)
  2 (2.00%) high mild
  2 (2.00%) high severe

clone_large             time:   [2.8637 µs 2.8686 µs 2.8742 µs]
                        change: [−2.3617% −2.1898% −2.0214%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 2 outliers among 100 measurements (2.00%)
  2 (2.00%) high mild

clone_from_large        time:   [2.8471 µs 2.8503 µs 2.8536 µs]
                        change: [−2.0758% −1.9431% −1.8032%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 2 outliers among 100 measurements (2.00%)
  2 (2.00%) high mild

rehash_in_place         time:   [157.06 µs 157.22 µs 157.40 µs]
                        change: [−2.1071% −1.9411% −1.7673%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 8 outliers among 100 measurements (8.00%)
  4 (4.00%) high mild
  4 (4.00%) high severe

insert                  time:   [4.8042 µs 4.8431 µs 4.8822 µs]
                        change: [+4.3565% +5.1871% +5.9945%] (p = 0.00 < 0.05)
                        Performance has regressed.

insert_unique_unchecked time:   [4.2504 µs 4.2870 µs 4.3249 µs]
                        change: [+5.8937% +6.8338% +7.7357%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 3 outliers among 100 measurements (3.00%)
  1 (1.00%) low mild
  2 (2.00%) high mild

set_ops_bit_or          time:   [40.754 µs 40.785 µs 40.819 µs]
                        change: [−0.4683% −0.2388% −0.0720%] (p = 0.01 < 0.05)
                        Change within noise threshold.
Found 5 outliers among 100 measurements (5.00%)
  4 (4.00%) high mild
  1 (1.00%) high severe

set_ops_bit_and         time:   [5.0576 µs 5.0612 µs 5.0648 µs]
                        change: [−1.1921% −1.0864% −0.9885%] (p = 0.00 < 0.05)
                        Change within noise threshold.
Found 2 outliers among 100 measurements (2.00%)
  2 (2.00%) high mild

set_ops_bit_xor         time:   [43.020 µs 43.054 µs 43.094 µs]
                        change: [−0.1147% +0.1251% +0.3441%] (p = 0.31 > 0.05)
                        No change in performance detected.
Found 7 outliers among 100 measurements (7.00%)
  4 (4.00%) high mild
  3 (3.00%) high severe

set_ops_sub_large_small time:   [41.513 µs 41.541 µs 41.575 µs]
                        change: [−2.1491% −2.0164% −1.8823%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 8 outliers among 100 measurements (8.00%)
  3 (3.00%) high mild
  5 (5.00%) high severe

set_ops_sub_small_large time:   [433.99 ns 434.30 ns 434.63 ns]
                        change: [+1.9068% +2.0282% +2.1504%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 5 outliers among 100 measurements (5.00%)
  4 (4.00%) high mild
  1 (1.00%) high severe

set_ops_bit_or_assign   time:   [26.953 µs 26.990 µs 27.036 µs]
                        change: [−5.0082% −4.7859% −4.6031%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 8 outliers among 100 measurements (8.00%)
  3 (3.00%) high mild
  5 (5.00%) high severe

set_ops_bit_and_assign  time:   [3.2670 µs 3.2689 µs 3.2709 µs]
                        change: [−1.3059% −1.0240% −0.7830%] (p = 0.00 < 0.05)
                        Change within noise threshold.
Found 9 outliers among 100 measurements (9.00%)
  2 (2.00%) low mild
  2 (2.00%) high mild
  5 (5.00%) high severe

set_ops_bit_xor_assign  time:   [28.375 µs 28.397 µs 28.422 µs]
                        change: [−3.0853% −2.8322% −2.6490%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 9 outliers among 100 measurements (9.00%)
  1 (1.00%) low mild
  1 (1.00%) high mild
  7 (7.00%) high severe

set_ops_sub_assign_large_small
                        time:   [28.692 µs 28.711 µs 28.731 µs]
                        change: [−3.3632% −3.2586% −3.1511%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 8 outliers among 100 measurements (8.00%)
  7 (7.00%) high mild
  1 (1.00%) high severe

set_ops_sub_assign_small_large
                        time:   [3.2875 µs 3.2891 µs 3.2908 µs]
                        change: [−1.8675% −1.7637% −1.6631%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 10 outliers among 100 measurements (10.00%)
  8 (8.00%) high mild
  2 (2.00%) high severe

with_capacity_000000    time:   [4.0365 ns 4.0385 ns 4.0407 ns]
                        change: [−0.3624% −0.1100% +0.0689%] (p = 0.38 > 0.05)
                        No change in performance detected.
Found 11 outliers among 100 measurements (11.00%)
  9 (9.00%) high mild
  2 (2.00%) high severe

with_capacity_000001    time:   [19.104 ns 19.130 ns 19.162 ns]
                        change: [−0.1710% +0.0705% +0.3706%] (p = 0.60 > 0.05)
                        No change in performance detected.
Found 8 outliers among 100 measurements (8.00%)
  3 (3.00%) high mild
  5 (5.00%) high severe

with_capacity_000003    time:   [19.099 ns 19.110 ns 19.121 ns]
                        change: [−0.1487% −0.0352% +0.0624%] (p = 0.54 > 0.05)
                        No change in performance detected.
Found 4 outliers among 100 measurements (4.00%)
  3 (3.00%) high mild
  1 (1.00%) high severe

with_capacity_000007    time:   [19.096 ns 19.119 ns 19.146 ns]
                        change: [+0.1360% +0.2833% +0.4612%] (p = 0.00 < 0.05)
                        Change within noise threshold.
Found 8 outliers among 100 measurements (8.00%)
  4 (4.00%) high mild
  4 (4.00%) high severe

with_capacity_000008    time:   [19.084 ns 19.106 ns 19.134 ns]
                        change: [−0.3085% −0.1528% +0.0058%] (p = 0.05 > 0.05)
                        No change in performance detected.
Found 5 outliers among 100 measurements (5.00%)
  3 (3.00%) high mild
  2 (2.00%) high severe

with_capacity_000016    time:   [20.331 ns 20.346 ns 20.361 ns]
                        change: [−0.5819% −0.3701% −0.1902%] (p = 0.00 < 0.05)
                        Change within noise threshold.
Found 6 outliers among 100 measurements (6.00%)
  2 (2.00%) low mild
  4 (4.00%) high mild

with_capacity_000032    time:   [39.147 ns 39.166 ns 39.187 ns]
                        change: [+4.2695% +4.4303% +4.5816%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 7 outliers among 100 measurements (7.00%)
  3 (3.00%) high mild
  4 (4.00%) high severe

with_capacity_000064    time:   [38.468 ns 38.507 ns 38.551 ns]
                        change: [−0.3992% −0.1869% −0.0149%] (p = 0.05 > 0.05)
                        No change in performance detected.
Found 6 outliers among 100 measurements (6.00%)
  4 (4.00%) high mild
  2 (2.00%) high severe

with_capacity_000128    time:   [40.399 ns 40.426 ns 40.454 ns]
                        change: [−11.146% −11.005% −10.849%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 5 outliers among 100 measurements (5.00%)
  3 (3.00%) high mild
  2 (2.00%) high severe

with_capacity_000256    time:   [43.422 ns 43.477 ns 43.532 ns]
                        change: [−4.4254% −4.2262% −3.9593%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 2 outliers among 100 measurements (2.00%)
  1 (1.00%) high mild
  1 (1.00%) high severe

with_capacity_000512    time:   [48.084 ns 48.161 ns 48.252 ns]
                        change: [−1.6354% −1.4676% −1.3091%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 7 outliers among 100 measurements (7.00%)
  4 (4.00%) high mild
  3 (3.00%) high severe

with_capacity_001024    time:   [53.685 ns 53.753 ns 53.825 ns]
                        change: [−1.4291% −1.0362% −0.4631%] (p = 0.00 < 0.05)
                        Change within noise threshold.
Found 4 outliers among 100 measurements (4.00%)
  2 (2.00%) high mild
  2 (2.00%) high severe

with_capacity_004096    time:   [88.273 ns 88.318 ns 88.365 ns]
                        change: [−5.9034% −5.6532% −5.4559%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 4 outliers among 100 measurements (4.00%)
  3 (3.00%) high mild
  1 (1.00%) high severe

with_capacity_016384    time:   [265.92 ns 266.08 ns 266.29 ns]
                        change: [−4.0201% −3.9056% −3.8058%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 10 outliers among 100 measurements (10.00%)
  7 (7.00%) high mild
  3 (3.00%) high severe

with_capacity_065536    time:   [979.42 ns 979.91 ns 980.45 ns]
                        change: [−1.5926% −1.4679% −1.3552%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 9 outliers among 100 measurements (9.00%)
  5 (5.00%) high mild
  4 (4.00%) high severe

@clarfonthey
Copy link
Copy Markdown
Contributor

Kind of confused why the stable cold_path is having any effect at all, to be honest. I guess that the presence of #[rustc_intrinsic] and #[rustc_nounwind] over #[inline(always)] just have some subtle side effects?

@0xdeafbeef
Copy link
Copy Markdown
Contributor Author

Kind of confused why the stable cold_path is having any effect at all, to be honest. I guess that the presence of #[rustc_intrinsic] and #[rustc_nounwind] over #[inline(always)] just have some subtle side effects?

Interpreting codegen is hard :)
Nightly also gives interesting results:

insert_foldhash_serial  time:   [9.3616 µs 9.4021 µs 9.4428 µs]
                        change: [−3.7128% −3.1698% −2.6765%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 4 outliers among 100 measurements (4.00%)
  1 (1.00%) low mild
  3 (3.00%) high mild

insert_std_serial       time:   [17.688 µs 17.739 µs 17.790 µs]
                        change: [+0.1938% +0.5737% +0.9604%] (p = 0.00 < 0.05)
                        Change within noise threshold.
Found 2 outliers among 100 measurements (2.00%)
  1 (1.00%) high mild
  1 (1.00%) high severe

insert_foldhash_highbits
                        time:   [9.7489 µs 9.7972 µs 9.8507 µs]
                        change: [−3.5902% −3.0743% −2.5667%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 1 outliers among 100 measurements (1.00%)
  1 (1.00%) high mild

insert_std_highbits     time:   [17.771 µs 17.828 µs 17.883 µs]
                        change: [+2.2747% +2.6148% +2.9732%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 3 outliers among 100 measurements (3.00%)
  1 (1.00%) low mild
  2 (2.00%) high mild

insert_foldhash_random  time:   [9.8818 µs 9.9260 µs 9.9699 µs]
                        change: [−0.7033% −0.2473% +0.2209%] (p = 0.31 > 0.05)
                        No change in performance detected.

insert_std_random       time:   [17.621 µs 17.673 µs 17.729 µs]
                        change: [−1.4543% −1.1446% −0.8193%] (p = 0.00 < 0.05)
                        Change within noise threshold.
Found 1 outliers among 100 measurements (1.00%)
  1 (1.00%) high mild

grow_insert_foldhash_serial
                        time:   [17.252 µs 17.264 µs 17.278 µs]
                        change: [+1.3603% +1.4720% +1.5972%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 7 outliers among 100 measurements (7.00%)
  6 (6.00%) high mild
  1 (1.00%) high severe

grow_insert_std_serial  time:   [36.850 µs 36.879 µs 36.913 µs]
                        change: [−2.5527% −2.4088% −2.2413%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 12 outliers among 100 measurements (12.00%)
  8 (8.00%) high mild
  4 (4.00%) high severe

grow_insert_foldhash_highbits
                        time:   [21.265 µs 21.282 µs 21.302 µs]
                        change: [+34.506% +34.649% +34.808%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 7 outliers among 100 measurements (7.00%)
  5 (5.00%) high mild
  2 (2.00%) high severe

grow_insert_std_highbits
                        time:   [36.715 µs 36.749 µs 36.788 µs]
                        change: [+0.3022% +0.4215% +0.5561%] (p = 0.00 < 0.05)
                        Change within noise threshold.
Found 15 outliers among 100 measurements (15.00%)
  6 (6.00%) high mild
  9 (9.00%) high severe

grow_insert_foldhash_random
                        time:   [18.230 µs 18.240 µs 18.253 µs]
                        change: [−1.6220% −1.5002% −1.3583%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 11 outliers among 100 measurements (11.00%)
  3 (3.00%) high mild
  8 (8.00%) high severe

grow_insert_std_random  time:   [36.604 µs 36.631 µs 36.659 µs]
                        change: [−0.3725% −0.2758% −0.1806%] (p = 0.00 < 0.05)
                        Change within noise threshold.

insert_erase_foldhash_serial
                        time:   [10.526 µs 10.553 µs 10.582 µs]
                        change: [+1.4915% +1.8822% +2.2634%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 7 outliers among 100 measurements (7.00%)
  1 (1.00%) low mild
  5 (5.00%) high mild
  1 (1.00%) high severe

insert_erase_std_serial time:   [24.847 µs 24.880 µs 24.913 µs]
                        change: [+0.6857% +0.8621% +1.0469%] (p = 0.00 < 0.05)
                        Change within noise threshold.
Found 2 outliers among 100 measurements (2.00%)
  1 (1.00%) high mild
  1 (1.00%) high severe

insert_erase_foldhash_highbits
                        time:   [10.362 µs 10.417 µs 10.484 µs]
                        change: [+2.1183% +2.5928% +3.0694%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 6 outliers among 100 measurements (6.00%)
  2 (2.00%) high mild
  4 (4.00%) high severe

insert_erase_std_highbits
                        time:   [25.194 µs 25.253 µs 25.311 µs]
                        change: [+0.6918% +0.8920% +1.0972%] (p = 0.00 < 0.05)
                        Change within noise threshold.

insert_erase_foldhash_random
                        time:   [10.791 µs 10.822 µs 10.852 µs]
                        change: [+2.5724% +2.9464% +3.3845%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 2 outliers among 100 measurements (2.00%)
  1 (1.00%) high mild
  1 (1.00%) high severe

insert_erase_std_random time:   [25.060 µs 25.118 µs 25.177 µs]
                        change: [+1.8258% +2.1288% +2.3858%] (p = 0.00 < 0.05)
                        Performance has regressed.

lookup_foldhash_serial  time:   [2.1042 µs 2.1097 µs 2.1155 µs]
                        change: [+38.127% +38.398% +38.679%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 3 outliers among 100 measurements (3.00%)
  3 (3.00%) high mild

lookup_std_serial       time:   [9.2977 µs 9.3057 µs 9.3140 µs]
                        change: [+7.2655% +7.4173% +7.5587%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 4 outliers among 100 measurements (4.00%)
  3 (3.00%) high mild
  1 (1.00%) high severe

lookup_foldhash_highbits
                        time:   [2.1434 µs 2.1470 µs 2.1514 µs]
                        change: [+36.160% +36.428% +36.670%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 5 outliers among 100 measurements (5.00%)
  4 (4.00%) high mild
  1 (1.00%) high severe

lookup_std_highbits     time:   [9.5590 µs 9.5665 µs 9.5744 µs]
                        change: [+8.2988% +8.4949% +8.6974%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 7 outliers among 100 measurements (7.00%)
  5 (5.00%) high mild
  2 (2.00%) high severe

lookup_foldhash_random  time:   [2.2508 µs 2.2548 µs 2.2588 µs]
                        change: [+29.259% +29.921% +30.459%] (p = 0.00 < 0.05)
                        Performance has regressed.

lookup_std_random       time:   [9.3094 µs 9.3166 µs 9.3243 µs]
                        change: [+8.2076% +8.4429% +8.6707%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 8 outliers among 100 measurements (8.00%)
  7 (7.00%) high mild
  1 (1.00%) high severe

lookup_fail_foldhash_serial
                        time:   [2.2015 µs 2.2066 µs 2.2126 µs]
                        change: [+14.256% +14.716% +15.153%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 8 outliers among 100 measurements (8.00%)
  7 (7.00%) high mild
  1 (1.00%) high severe

lookup_fail_std_serial  time:   [9.3045 µs 9.3179 µs 9.3319 µs]
                        change: [+3.1504% +3.3515% +3.5393%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 5 outliers among 100 measurements (5.00%)
  4 (4.00%) high mild
  1 (1.00%) high severe

lookup_fail_foldhash_highbits
                        time:   [2.2534 µs 2.2556 µs 2.2581 µs]
                        change: [+15.363% +15.506% +15.642%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 8 outliers among 100 measurements (8.00%)
  7 (7.00%) high mild
  1 (1.00%) high severe

lookup_fail_std_highbits
                        time:   [9.4628 µs 9.4759 µs 9.4920 µs]
                        change: [+5.0793% +5.4117% +5.9661%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 10 outliers among 100 measurements (10.00%)
  5 (5.00%) high mild
  5 (5.00%) high severe

lookup_fail_foldhash_random
                        time:   [2.3144 µs 2.3259 µs 2.3386 µs]
                        change: [+10.315% +10.933% +11.477%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 5 outliers among 100 measurements (5.00%)
  3 (3.00%) high mild
  2 (2.00%) high severe

lookup_fail_std_random  time:   [9.4120 µs 9.4245 µs 9.4381 µs]
                        change: [+5.2946% +5.5321% +5.8137%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 8 outliers among 100 measurements (8.00%)
  4 (4.00%) high mild
  4 (4.00%) high severe

loadfactor_lookup_14500 time:   [1.2467 µs 1.2497 µs 1.2527 µs]
                        change: [+24.356% +24.614% +24.893%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 1 outliers among 100 measurements (1.00%)
  1 (1.00%) high mild

loadfactor_lookup_16500 time:   [1.2471 µs 1.2506 µs 1.2543 µs]
                        change: [+24.239% +24.582% +24.916%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 4 outliers among 100 measurements (4.00%)
  3 (3.00%) high mild
  1 (1.00%) high severe

loadfactor_lookup_18500 time:   [1.2485 µs 1.2515 µs 1.2546 µs]
                        change: [+24.207% +24.491% +24.758%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 2 outliers among 100 measurements (2.00%)
  2 (2.00%) high mild

loadfactor_lookup_20500 time:   [1.2458 µs 1.2483 µs 1.2509 µs]
                        change: [+23.627% +24.298% +24.846%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 4 outliers among 100 measurements (4.00%)
  2 (2.00%) high mild
  2 (2.00%) high severe

loadfactor_lookup_22500 time:   [1.2483 µs 1.2511 µs 1.2541 µs]
                        change: [+23.849% +24.411% +24.832%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 2 outliers among 100 measurements (2.00%)
  2 (2.00%) high mild

loadfactor_lookup_24500 time:   [1.2465 µs 1.2493 µs 1.2522 µs]
                        change: [+24.490% +24.865% +25.311%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 3 outliers among 100 measurements (3.00%)
  2 (2.00%) high mild
  1 (1.00%) high severe

loadfactor_lookup_26500 time:   [1.2468 µs 1.2504 µs 1.2541 µs]
                        change: [+24.402% +24.695% +25.016%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 1 outliers among 100 measurements (1.00%)
  1 (1.00%) high mild

loadfactor_lookup_28500 time:   [1.2478 µs 1.2502 µs 1.2529 µs]
                        change: [+24.528% +24.777% +25.043%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 4 outliers among 100 measurements (4.00%)
  4 (4.00%) high mild

loadfactor_lookup_fail_14500
                        time:   [1.1456 µs 1.1470 µs 1.1487 µs]
                        change: [+12.602% +12.762% +12.932%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 9 outliers among 100 measurements (9.00%)
  6 (6.00%) high mild
  3 (3.00%) high severe

loadfactor_lookup_fail_16500
                        time:   [1.2073 µs 1.2090 µs 1.2110 µs]
                        change: [+11.688% +11.987% +12.387%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 6 outliers among 100 measurements (6.00%)
  4 (4.00%) high mild
  2 (2.00%) high severe

loadfactor_lookup_fail_18500
                        time:   [1.2988 µs 1.3016 µs 1.3045 µs]
                        change: [+10.111% +10.430% +10.758%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 2 outliers among 100 measurements (2.00%)
  1 (1.00%) high mild
  1 (1.00%) high severe

loadfactor_lookup_fail_20500
                        time:   [1.4587 µs 1.4651 µs 1.4713 µs]
                        change: [+8.2032% +8.7088% +9.2174%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 2 outliers among 100 measurements (2.00%)
  2 (2.00%) high mild

loadfactor_lookup_fail_22500
                        time:   [1.7678 µs 1.7787 µs 1.7895 µs]
                        change: [+6.3207% +7.0800% +7.7707%] (p = 0.00 < 0.05)
                        Performance has regressed.

loadfactor_lookup_fail_24500
                        time:   [2.3537 µs 2.3680 µs 2.3827 µs]
                        change: [+3.8322% +4.6022% +5.3531%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 1 outliers among 100 measurements (1.00%)
  1 (1.00%) high mild

loadfactor_lookup_fail_26500
                        time:   [3.4438 µs 3.4648 µs 3.4860 µs]
                        change: [+1.4003% +2.2308% +2.9955%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 2 outliers among 100 measurements (2.00%)
  1 (1.00%) low mild
  1 (1.00%) high mild

loadfactor_lookup_fail_28500
                        time:   [5.4314 µs 5.4623 µs 5.4934 µs]
                        change: [+2.4117% +2.9776% +3.5721%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 1 outliers among 100 measurements (1.00%)
  1 (1.00%) low mild

iter_foldhash_serial    time:   [601.31 ns 602.27 ns 603.57 ns]
                        change: [−0.4093% −0.0386% +0.3901%] (p = 0.87 > 0.05)
                        No change in performance detected.
Found 8 outliers among 100 measurements (8.00%)
  3 (3.00%) high mild
  5 (5.00%) high severe

iter_std_serial         time:   [601.24 ns 602.15 ns 603.27 ns]
                        change: [−0.3953% −0.1983% +0.0260%] (p = 0.06 > 0.05)
                        No change in performance detected.
Found 8 outliers among 100 measurements (8.00%)
  3 (3.00%) high mild
  5 (5.00%) high severe

iter_foldhash_highbits  time:   [606.24 ns 606.81 ns 607.41 ns]
                        change: [+0.6338% +0.8481% +1.0619%] (p = 0.00 < 0.05)
                        Change within noise threshold.
Found 6 outliers among 100 measurements (6.00%)
  4 (4.00%) high mild
  2 (2.00%) high severe

iter_std_highbits       time:   [601.45 ns 602.22 ns 603.05 ns]
                        change: [−0.3806% −0.1998% −0.0283%] (p = 0.02 < 0.05)
                        Change within noise threshold.
Found 8 outliers among 100 measurements (8.00%)
  7 (7.00%) high mild
  1 (1.00%) high severe

iter_foldhash_random    time:   [601.36 ns 602.11 ns 602.98 ns]
                        change: [−0.8140% −0.3880% +0.0730%] (p = 0.09 > 0.05)
                        No change in performance detected.
Found 7 outliers among 100 measurements (7.00%)
  4 (4.00%) high mild
  3 (3.00%) high severe

iter_std_random         time:   [600.90 ns 601.54 ns 602.24 ns]
                        change: [−0.7123% −0.3573% −0.0293%] (p = 0.04 < 0.05)
                        Change within noise threshold.
Found 7 outliers among 100 measurements (7.00%)
  2 (2.00%) high mild
  5 (5.00%) high severe

clone_small             time:   [23.900 ns 23.945 ns 23.998 ns]
                        change: [+0.9300% +1.1083% +1.2753%] (p = 0.00 < 0.05)
                        Change within noise threshold.
Found 6 outliers among 100 measurements (6.00%)
  2 (2.00%) high mild
  4 (4.00%) high severe

clone_from_small        time:   [19.360 ns 19.377 ns 19.396 ns]
                        change: [−0.3846% −0.0625% +0.1853%] (p = 0.72 > 0.05)
                        No change in performance detected.
Found 5 outliers among 100 measurements (5.00%)
  3 (3.00%) high mild
  2 (2.00%) high severe

clone_large             time:   [2.9823 µs 2.9868 µs 2.9914 µs]
                        change: [+1.6778% +1.8672% +2.0453%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 6 outliers among 100 measurements (6.00%)
  4 (4.00%) low mild
  2 (2.00%) high mild

clone_from_large        time:   [2.9563 µs 2.9616 µs 2.9663 µs]
                        change: [+1.7423% +1.9260% +2.0993%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 11 outliers among 100 measurements (11.00%)
  2 (2.00%) low severe
  7 (7.00%) low mild
  2 (2.00%) high mild

rehash_in_place         time:   [150.48 µs 150.61 µs 150.76 µs]
                        change: [−6.3743% −6.2428% −6.1182%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 7 outliers among 100 measurements (7.00%)
  1 (1.00%) low severe
  2 (2.00%) low mild
  3 (3.00%) high mild
  1 (1.00%) high severe

insert                  time:   [4.5391 µs 4.5655 µs 4.5929 µs]
                        change: [−1.1842% −0.5400% +0.0950%] (p = 0.11 > 0.05)
                        No change in performance detected.
Found 1 outliers among 100 measurements (1.00%)
  1 (1.00%) high mild

insert_unique_unchecked time:   [4.0024 µs 4.0323 µs 4.0624 µs]
                        change: [−0.8363% −0.0406% +0.6802%] (p = 0.92 > 0.05)
                        No change in performance detected.

set_ops_bit_or          time:   [40.390 µs 40.425 µs 40.471 µs]
                        change: [−1.2905% −0.2354% +1.5966%] (p = 0.83 > 0.05)
                        No change in performance detected.
Found 10 outliers among 100 measurements (10.00%)
  5 (5.00%) high mild
  5 (5.00%) high severe

set_ops_bit_and         time:   [5.1079 µs 5.1107 µs 5.1139 µs]
                        change: [−0.0685% +0.0272% +0.1205%] (p = 0.57 > 0.05)
                        No change in performance detected.
Found 9 outliers among 100 measurements (9.00%)
  9 (9.00%) high mild

set_ops_bit_xor         time:   [41.462 µs 41.486 µs 41.511 µs]
                        change: [−3.6588% −3.4489% −3.2660%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 4 outliers among 100 measurements (4.00%)
  1 (1.00%) low mild
  2 (2.00%) high mild
  1 (1.00%) high severe

set_ops_sub_large_small time:   [40.857 µs 40.880 µs 40.905 µs]
                        change: [−3.6764% −3.5400% −3.4028%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 9 outliers among 100 measurements (9.00%)
  6 (6.00%) high mild
  3 (3.00%) high severe

set_ops_sub_small_large time:   [431.48 ns 431.91 ns 432.41 ns]
                        change: [+1.3184% +1.4470% +1.5747%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 7 outliers among 100 measurements (7.00%)
  5 (5.00%) high mild
  2 (2.00%) high severe

set_ops_bit_or_assign   time:   [26.889 µs 26.914 µs 26.947 µs]
                        change: [−5.2408% −5.0311% −4.8923%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 7 outliers among 100 measurements (7.00%)
  5 (5.00%) high mild
  2 (2.00%) high severe

set_ops_bit_and_assign  time:   [3.2844 µs 3.2875 µs 3.2912 µs]
                        change: [−0.8367% −0.5632% −0.3278%] (p = 0.00 < 0.05)
                        Change within noise threshold.
Found 12 outliers among 100 measurements (12.00%)
  2 (2.00%) low mild
  6 (6.00%) high mild
  4 (4.00%) high severe

set_ops_bit_xor_assign  time:   [28.317 µs 28.334 µs 28.353 µs]
                        change: [−3.2160% −2.9653% −2.7837%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 2 outliers among 100 measurements (2.00%)
  1 (1.00%) high mild
  1 (1.00%) high severe

set_ops_sub_assign_large_small
                        time:   [28.373 µs 28.402 µs 28.436 µs]
                        change: [−4.3927% −4.1060% −3.6506%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 6 outliers among 100 measurements (6.00%)
  1 (1.00%) high mild
  5 (5.00%) high severe

set_ops_sub_assign_small_large
                        time:   [3.3413 µs 3.3447 µs 3.3485 µs]
                        change: [−0.3092% −0.1911% −0.0675%] (p = 0.00 < 0.05)
                        Change within noise threshold.
Found 4 outliers among 100 measurements (4.00%)
  3 (3.00%) high mild
  1 (1.00%) high severe

with_capacity_000000    time:   [4.0375 ns 4.0402 ns 4.0431 ns]
                        change: [−0.3574% −0.0825% +0.1393%] (p = 0.57 > 0.05)
                        No change in performance detected.
Found 8 outliers among 100 measurements (8.00%)
  6 (6.00%) high mild
  2 (2.00%) high severe

with_capacity_000001    time:   [19.157 ns 19.172 ns 19.192 ns]
                        change: [−0.0008% +0.1777% +0.3283%] (p = 0.03 < 0.05)
                        Change within noise threshold.
Found 7 outliers among 100 measurements (7.00%)
  4 (4.00%) high mild
  3 (3.00%) high severe

with_capacity_000003    time:   [19.139 ns 19.154 ns 19.171 ns]
                        change: [+0.1395% +0.2618% +0.3684%] (p = 0.00 < 0.05)
                        Change within noise threshold.
Found 6 outliers among 100 measurements (6.00%)
  2 (2.00%) high mild
  4 (4.00%) high severe

with_capacity_000007    time:   [19.089 ns 19.099 ns 19.110 ns]
                        change: [+0.0841% +0.1902% +0.2877%] (p = 0.00 < 0.05)
                        Change within noise threshold.
Found 8 outliers among 100 measurements (8.00%)
  4 (4.00%) high mild
  4 (4.00%) high severe

with_capacity_000008    time:   [19.106 ns 19.121 ns 19.138 ns]
                        change: [−0.2049% −0.0713% +0.0794%] (p = 0.32 > 0.05)
                        No change in performance detected.
Found 7 outliers among 100 measurements (7.00%)
  4 (4.00%) high mild
  3 (3.00%) high severe

with_capacity_000016    time:   [20.299 ns 20.309 ns 20.321 ns]
                        change: [−0.7086% −0.4949% −0.3179%] (p = 0.00 < 0.05)
                        Change within noise threshold.
Found 7 outliers among 100 measurements (7.00%)
  4 (4.00%) high mild
  3 (3.00%) high severe

with_capacity_000032    time:   [39.763 ns 39.798 ns 39.836 ns]
                        change: [+5.9714% +6.1928% +6.4786%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 8 outliers among 100 measurements (8.00%)
  5 (5.00%) high mild
  3 (3.00%) high severe

with_capacity_000064    time:   [38.437 ns 38.465 ns 38.497 ns]
                        change: [−0.4675% −0.2263% +0.0063%] (p = 0.06 > 0.05)
                        No change in performance detected.
Found 8 outliers among 100 measurements (8.00%)
  5 (5.00%) high mild
  3 (3.00%) high severe

with_capacity_000128    time:   [46.741 ns 46.796 ns 46.859 ns]
                        change: [+2.8837% +3.0563% +3.2307%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 6 outliers among 100 measurements (6.00%)
  3 (3.00%) high mild
  3 (3.00%) high severe

with_capacity_000256    time:   [43.150 ns 43.180 ns 43.216 ns]
                        change: [−5.1326% −4.9750% −4.7705%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 12 outliers among 100 measurements (12.00%)
  3 (3.00%) low mild
  5 (5.00%) high mild
  4 (4.00%) high severe

with_capacity_000512    time:   [45.601 ns 45.651 ns 45.708 ns]
                        change: [−6.6640% −6.4838% −6.2823%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 10 outliers among 100 measurements (10.00%)
  7 (7.00%) high mild
  3 (3.00%) high severe

with_capacity_001024    time:   [51.181 ns 51.292 ns 51.401 ns]
                        change: [−5.9524% −5.7551% −5.5550%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 4 outliers among 100 measurements (4.00%)
  2 (2.00%) low mild
  2 (2.00%) high mild

with_capacity_004096    time:   [88.494 ns 88.578 ns 88.675 ns]
                        change: [−5.6829% −5.4353% −5.2509%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 10 outliers among 100 measurements (10.00%)
  7 (7.00%) high mild
  3 (3.00%) high severe

with_capacity_016384    time:   [264.92 ns 265.06 ns 265.22 ns]
                        change: [−4.3361% −4.2107% −4.0884%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 10 outliers among 100 measurements (10.00%)
  5 (5.00%) high mild
  5 (5.00%) high severe

with_capacity_065536    time:   [979.45 ns 980.05 ns 980.74 ns]
                        change: [−1.5847% −1.4467% −1.3031%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 9 outliers among 100 measurements (9.00%)
  4 (4.00%) high mild
  5 (5.00%) high severe

@clarfonthey
Copy link
Copy Markdown
Contributor

Honestly, while I think that we should just use the stable version when available (as a reminder to use it when MSRV updates), this kind of feels well within the range of signal noise. Would probably trust some cachegrind benchmarks or just peeking at the generated code more, since those are much less susceptible to background noise, even though criterion tries its best.

Also, using the internal cold_path means that if any performance improvements are made to that in general, hashbrown can perpetually reap those, whereas having a non-intrinsic cold_path could potentially differ. At least being an intrinsic means that the optimizer can notice it better, which is already an improvement.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants