Skip to content

Fixed flaky test#9218

Merged
kr-igor merged 2 commits intomasterfrom
kr-igor/fix-flaky-test
Jul 22, 2025
Merged

Fixed flaky test#9218
kr-igor merged 2 commits intomasterfrom
kr-igor/fix-flaky-test

Conversation

@kr-igor
Copy link
Contributor

@kr-igor kr-igor commented Jul 22, 2025 •
edited
Loading

What Does This Do

Fix for a flaky test

Details

Elements in the collection were not sorted properly. Now this test should work as expected.
The issue was introduced in this PR.

@kr-igor kr-igor marked this pull request as ready for review July 22, 2025 17:42
@kr-igor kr-igor requested a review from a team as a code owner July 22, 2025 17:42
@github-actions
Copy link
Contributor

github-actions bot commented Jul 22, 2025 •
edited
Loading

Hi! 👋 Thanks for your pull request! 🎉

To help us review it, please make sure to:

  • Add at least one type, and one component or instrumentation label to the pull request

If you need help, please check our contributing guidelines.

@kr-igor kr-igor added comp: testing Testing comp: data streams Data Streams Monitoring labels Jul 22, 2025
AlexeyKuznetsov-DD
AlexeyKuznetsov-DD approved these changes Jul 22, 2025
View reviewed changes
Copy link
Contributor

@AlexeyKuznetsov-DD AlexeyKuznetsov-DD left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, do not forget to add labels and proper PR description by template

@kr-igor kr-igor added the type: bug Bug report and fix label Jul 22, 2025
@bm1549 bm1549 mentioned this pull request Jul 22, 2025
Merged
@pr-commenter
Copy link

pr-commenter bot commented Jul 22, 2025 •
edited
Loading

Benchmarks

Startup

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master kr-igor/fix-flaky-test
git_commit_date 1753206619 1753215515
git_commit_sha 7b6259d 69efa01
release_version 1.51.1-SNAPSHOT~7b6259d446 1.51.1-SNAPSHOT~69efa01716
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1753217451 1753217451
ci_job_id 1042184538 1042184538
ci_pipeline_id 71377968 71377968
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-zfyrx7zua-project-304-concurrent-0-gzbj0d6i 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux Linux runner-zfyrx7zua-project-304-concurrent-0-gzbj0d6i 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
module Agent Agent
parent None None

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 47 metrics, 12 unstable metrics.

Startup time reports for petclinic 
gantt
    title petclinic - global startup overhead: candidate=1.51.1-SNAPSHOT~69efa01716, baseline=1.51.1-SNAPSHOT~7b6259d446

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.039 s) : 0, 1039345
Total [baseline] (10.69 s) : 0, 10689561
Agent [candidate] (1.04 s) : 0, 1039680
Total [candidate] (10.662 s) : 0, 10661526
section appsec
Agent [baseline] (1.223 s) : 0, 1222684
Total [baseline] (10.811 s) : 0, 10811109
Agent [candidate] (1.22 s) : 0, 1219539
Total [candidate] (10.756 s) : 0, 10756000
section iast
Agent [baseline] (1.173 s) : 0, 1173332
Total [baseline] (10.881 s) : 0, 10880865
Agent [candidate] (1.176 s) : 0, 1175671
Total [candidate] (10.905 s) : 0, 10905463
section profiling
Agent [baseline] (1.192 s) : 0, 1191762
Total [baseline] (10.899 s) : 0, 10899405
Agent [candidate] (1.185 s) : 0, 1184657
Total [candidate] (10.878 s) : 0, 10878138
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.039 s -
Agent appsec 1.223 s 183.339 ms (17.6%)
Agent iast 1.173 s 133.986 ms (12.9%)
Agent profiling 1.192 s 152.417 ms (14.7%)
Total tracing 10.69 s -
Total appsec 10.811 s 121.548 ms (1.1%)
Total iast 10.881 s 191.304 ms (1.8%)
Total profiling 10.899 s 209.844 ms (2.0%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.04 s -
Agent appsec 1.22 s 179.859 ms (17.3%)
Agent iast 1.176 s 135.991 ms (13.1%)
Agent profiling 1.185 s 144.977 ms (13.9%)
Total tracing 10.662 s -
Total appsec 10.756 s 94.474 ms (0.9%)
Total iast 10.905 s 243.937 ms (2.3%)
Total profiling 10.878 s 216.613 ms (2.0%) 
gantt
    title petclinic - break down per module: candidate=1.51.1-SNAPSHOT~69efa01716, baseline=1.51.1-SNAPSHOT~7b6259d446

    dateFormat X
    axisFormat %s
section tracing
crashtracking [baseline] (1.422 ms) : 0, 1422
crashtracking [candidate] (1.43 ms) : 0, 1430
BytebuddyAgent [baseline] (729.528 ms) : 0, 729528
BytebuddyAgent [candidate] (729.855 ms) : 0, 729855
GlobalTracer [baseline] (242.215 ms) : 0, 242215
GlobalTracer [candidate] (241.985 ms) : 0, 241985
AppSec [baseline] (30.372 ms) : 0, 30372
AppSec [candidate] (30.528 ms) : 0, 30528
Debugger [baseline] (5.982 ms) : 0, 5982
Debugger [candidate] (5.996 ms) : 0, 5996
Remote Config [baseline] (640.252 µs) : 0, 640
Remote Config [candidate] (642.719 µs) : 0, 643
Telemetry [baseline] (8.17 ms) : 0, 8170
Telemetry [candidate] (8.181 ms) : 0, 8181
section appsec
crashtracking [baseline] (1.428 ms) : 0, 1428
crashtracking [candidate] (1.431 ms) : 0, 1431
BytebuddyAgent [baseline] (754.776 ms) : 0, 754776
BytebuddyAgent [candidate] (752.956 ms) : 0, 752956
GlobalTracer [baseline] (235.975 ms) : 0, 235975
GlobalTracer [candidate] (235.061 ms) : 0, 235061
AppSec [baseline] (168.865 ms) : 0, 168865
AppSec [candidate] (170.081 ms) : 0, 170081
Debugger [baseline] (7.216 ms) : 0, 7216
Debugger [candidate] (6.389 ms) : 0, 6389
Remote Config [baseline] (599.452 µs) : 0, 599
Remote Config [candidate] (599.61 µs) : 0, 600
Telemetry [baseline] (8.85 ms) : 0, 8850
Telemetry [candidate] (8.074 ms) : 0, 8074
IAST [baseline] (23.621 ms) : 0, 23621
IAST [candidate] (23.62 ms) : 0, 23620
section iast
crashtracking [baseline] (1.425 ms) : 0, 1425
crashtracking [candidate] (1.445 ms) : 0, 1445
BytebuddyAgent [baseline] (846.909 ms) : 0, 846909
BytebuddyAgent [candidate] (848.898 ms) : 0, 848898
GlobalTracer [baseline] (232.247 ms) : 0, 232247
GlobalTracer [candidate] (231.636 ms) : 0, 231636
AppSec [baseline] (26.582 ms) : 0, 26582
AppSec [candidate] (29.114 ms) : 0, 29114
Debugger [baseline] (8.383 ms) : 0, 8383
Debugger [candidate] (6.706 ms) : 0, 6706
Remote Config [baseline] (589.623 µs) : 0, 590
Remote Config [candidate] (589.598 µs) : 0, 590
Telemetry [baseline] (7.895 ms) : 0, 7895
Telemetry [candidate] (7.985 ms) : 0, 7985
IAST [baseline] (28.298 ms) : 0, 28298
IAST [candidate] (28.186 ms) : 0, 28186
section profiling
ProfilingAgent [baseline] (105.765 ms) : 0, 105765
ProfilingAgent [candidate] (104.365 ms) : 0, 104365
crashtracking [baseline] (1.411 ms) : 0, 1411
crashtracking [candidate] (1.4 ms) : 0, 1400
BytebuddyAgent [baseline] (763.937 ms) : 0, 763937
BytebuddyAgent [candidate] (759.724 ms) : 0, 759724
GlobalTracer [baseline] (222.232 ms) : 0, 222232
GlobalTracer [candidate] (220.922 ms) : 0, 220922
AppSec [baseline] (30.751 ms) : 0, 30751
AppSec [candidate] (30.27 ms) : 0, 30270
Debugger [baseline] (6.338 ms) : 0, 6338
Debugger [candidate] (6.282 ms) : 0, 6282
Remote Config [baseline] (691.639 µs) : 0, 692
Remote Config [candidate] (666.293 µs) : 0, 666
Telemetry [baseline] (11.843 ms) : 0, 11843
Telemetry [candidate] (12.476 ms) : 0, 12476
Profiling [baseline] (105.789 ms) : 0, 105789
Profiling [candidate] (104.386 ms) : 0, 104386
Loading
Startup time reports for insecure-bank 
gantt
    title insecure-bank - global startup overhead: candidate=1.51.1-SNAPSHOT~69efa01716, baseline=1.51.1-SNAPSHOT~7b6259d446

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.04 s) : 0, 1040318
Total [baseline] (8.645 s) : 0, 8645309
Agent [candidate] (1.049 s) : 0, 1048660
Total [candidate] (8.619 s) : 0, 8618800
section iast
Agent [baseline] (1.173 s) : 0, 1172829
Total [baseline] (9.332 s) : 0, 9332105
Agent [candidate] (1.174 s) : 0, 1173597
Total [candidate] (9.272 s) : 0, 9271877
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.04 s -
Agent iast 1.173 s 132.51 ms (12.7%)
Total tracing 8.645 s -
Total iast 9.332 s 686.796 ms (7.9%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.049 s -
Agent iast 1.174 s 124.937 ms (11.9%)
Total tracing 8.619 s -
Total iast 9.272 s 653.077 ms (7.6%) 
gantt
    title insecure-bank - break down per module: candidate=1.51.1-SNAPSHOT~69efa01716, baseline=1.51.1-SNAPSHOT~7b6259d446

    dateFormat X
    axisFormat %s
section tracing
crashtracking [baseline] (1.446 ms) : 0, 1446
crashtracking [candidate] (1.44 ms) : 0, 1440
BytebuddyAgent [baseline] (731.107 ms) : 0, 731107
BytebuddyAgent [candidate] (735.029 ms) : 0, 735029
GlobalTracer [baseline] (241.568 ms) : 0, 241568
GlobalTracer [candidate] (244.444 ms) : 0, 244444
AppSec [baseline] (30.299 ms) : 0, 30299
AppSec [candidate] (30.842 ms) : 0, 30842
Debugger [baseline] (5.997 ms) : 0, 5997
Debugger [candidate] (6.066 ms) : 0, 6066
Remote Config [baseline] (638.038 µs) : 0, 638
Remote Config [candidate] (656.927 µs) : 0, 657
Telemetry [baseline] (8.195 ms) : 0, 8195
Telemetry [candidate] (9.08 ms) : 0, 9080
section iast
crashtracking [baseline] (1.429 ms) : 0, 1429
crashtracking [candidate] (1.427 ms) : 0, 1427
BytebuddyAgent [baseline] (846.52 ms) : 0, 846520
BytebuddyAgent [candidate] (847.396 ms) : 0, 847396
GlobalTracer [baseline] (231.195 ms) : 0, 231195
GlobalTracer [candidate] (231.525 ms) : 0, 231525
AppSec [baseline] (27.46 ms) : 0, 27460
AppSec [candidate] (28.117 ms) : 0, 28117
Debugger [baseline] (8.346 ms) : 0, 8346
Debugger [candidate] (5.781 ms) : 0, 5781
Remote Config [baseline] (583.554 µs) : 0, 584
Remote Config [candidate] (584.698 µs) : 0, 585
Telemetry [baseline] (8.815 ms) : 0, 8815
Telemetry [candidate] (7.901 ms) : 0, 7901
IAST [baseline] (27.504 ms) : 0, 27504
IAST [candidate] (29.919 ms) : 0, 29919
Loading

Load

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master kr-igor/fix-flaky-test
git_commit_date 1753206619 1753215515
git_commit_sha 7b6259d 69efa01
release_version 1.51.1-SNAPSHOT~7b6259d446 1.51.1-SNAPSHOT~69efa01716
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1753217197 1753217197
ci_job_id 1042184539 1042184539
ci_pipeline_id 71377968 71377968
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-zfyrx7zua-project-304-concurrent-0-bl6f9f1b 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux Linux runner-zfyrx7zua-project-304-concurrent-0-bl6f9f1b 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux

Summary

Found 3 performance improvements and 1 performance regressions! Performance is the same for 8 metrics, 12 unstable metrics.

scenario Δ mean http_req_duration Δ mean throughput candidate mean http_req_duration candidate mean throughput baseline mean http_req_duration baseline mean throughput
scenario:load:petclinic:appsec:high_load better
[-2.623ms; -1.695ms] or [-5.339%; -3.450%]		 unstable
[-2.635op/s; +11.310op/s] or [-2.766%; +11.873%]		 46.972ms 99.600op/s 49.131ms 95.263op/s
scenario:load:petclinic:profiling:high_load better
[-2.693ms; -1.703ms] or [-5.449%; -3.445%]		 unstable
[-2.945op/s; +11.745op/s] or [-3.110%; +12.402%]		 47.228ms 99.100op/s 49.426ms 94.700op/s
scenario:load:petclinic:tracing:high_load worse
[+1.126ms; +1.912ms] or [+2.610%; +4.434%]		 unstable
[-11.267op/s; +3.917op/s] or [-10.387%; +3.611%]		 44.639ms 104.800op/s 43.120ms 108.475op/s
scenario:load:petclinic:code_origins:high_load better
[-2.146ms; -1.303ms] or [-4.651%; -2.824%]		 unstable
[-3.403op/s; +11.203op/s] or [-3.356%; +11.048%]		 44.423ms 105.300op/s 46.148ms 101.400op/s
Request duration reports for petclinic 
gantt
    title petclinic - request duration [CI 0.99] : candidate=1.51.1-SNAPSHOT~69efa01716, baseline=1.51.1-SNAPSHOT~7b6259d446
    dateFormat X
    axisFormat %s
section baseline
no_agent (36.955 ms) : 36649, 37261
.   : milestone, 36955,
appsec (49.131 ms) : 48691, 49570
.   : milestone, 49131,
code_origins (46.148 ms) : 45750, 46547
.   : milestone, 46148,
iast (44.223 ms) : 43826, 44619
.   : milestone, 44223,
profiling (49.426 ms) : 48952, 49900
.   : milestone, 49426,
tracing (43.12 ms) : 42771, 43470
.   : milestone, 43120,
section candidate
no_agent (37.047 ms) : 36755, 37338
.   : milestone, 37047,
appsec (46.972 ms) : 46549, 47395
.   : milestone, 46972,
code_origins (44.423 ms) : 44038, 44808
.   : milestone, 44423,
iast (44.944 ms) : 44566, 45323
.   : milestone, 44944,
profiling (47.228 ms) : 46782, 47675
.   : milestone, 47228,
tracing (44.639 ms) : 44259, 45020
.   : milestone, 44639,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 36.955 ms [36.649 ms, 37.261 ms] -
appsec 49.131 ms [48.691 ms, 49.57 ms] 12.176 ms (32.9%)
code_origins 46.148 ms [45.75 ms, 46.547 ms] 9.193 ms (24.9%)
iast 44.223 ms [43.826 ms, 44.619 ms] 7.268 ms (19.7%)
profiling 49.426 ms [48.952 ms, 49.9 ms] 12.471 ms (33.7%)
tracing 43.12 ms [42.771 ms, 43.47 ms] 6.166 ms (16.7%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 37.047 ms [36.755 ms, 37.338 ms] -
appsec 46.972 ms [46.549 ms, 47.395 ms] 9.925 ms (26.8%)
code_origins 44.423 ms [44.038 ms, 44.808 ms] 7.376 ms (19.9%)
iast 44.944 ms [44.566 ms, 45.323 ms] 7.898 ms (21.3%)
profiling 47.228 ms [46.782 ms, 47.675 ms] 10.181 ms (27.5%)
tracing 44.639 ms [44.259 ms, 45.02 ms] 7.593 ms (20.5%)
Request duration reports for insecure-bank 
gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.51.1-SNAPSHOT~69efa01716, baseline=1.51.1-SNAPSHOT~7b6259d446
    dateFormat X
    axisFormat %s
section baseline
no_agent (4.336 ms) : 4275, 4396
.   : milestone, 4336,
iast (9.571 ms) : 9402, 9740
.   : milestone, 9571,
iast_FULL (13.921 ms) : 13646, 14197
.   : milestone, 13921,
iast_GLOBAL (10.001 ms) : 9816, 10187
.   : milestone, 10001,
profiling (9.019 ms) : 8876, 9161
.   : milestone, 9019,
tracing (7.424 ms) : 7312, 7535
.   : milestone, 7424,
section candidate
no_agent (4.375 ms) : 4322, 4428
.   : milestone, 4375,
iast (9.474 ms) : 9313, 9636
.   : milestone, 9474,
iast_FULL (13.813 ms) : 13539, 14086
.   : milestone, 13813,
iast_GLOBAL (9.999 ms) : 9825, 10172
.   : milestone, 9999,
profiling (9.316 ms) : 9174, 9459
.   : milestone, 9316,
tracing (7.372 ms) : 7271, 7473
.   : milestone, 7372,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 4.336 ms [4.275 ms, 4.396 ms] -
iast 9.571 ms [9.402 ms, 9.74 ms] 5.235 ms (120.7%)
iast_FULL 13.921 ms [13.646 ms, 14.197 ms] 9.586 ms (221.1%)
iast_GLOBAL 10.001 ms [9.816 ms, 10.187 ms] 5.666 ms (130.7%)
profiling 9.019 ms [8.876 ms, 9.161 ms] 4.683 ms (108.0%)
tracing 7.424 ms [7.312 ms, 7.535 ms] 3.088 ms (71.2%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 4.375 ms [4.322 ms, 4.428 ms] -
iast 9.474 ms [9.313 ms, 9.636 ms] 5.099 ms (116.6%)
iast_FULL 13.813 ms [13.539 ms, 14.086 ms] 9.437 ms (215.7%)
iast_GLOBAL 9.999 ms [9.825 ms, 10.172 ms] 5.624 ms (128.5%)
profiling 9.316 ms [9.174 ms, 9.459 ms] 4.941 ms (112.9%)
tracing 7.372 ms [7.271 ms, 7.473 ms] 2.997 ms (68.5%)

Dacapo

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master kr-igor/fix-flaky-test
git_commit_date 1753206619 1753215515
git_commit_sha 7b6259d 69efa01
release_version 1.51.1-SNAPSHOT~7b6259d446 1.51.1-SNAPSHOT~69efa01716
See matching parameters
Baseline Candidate
application biojava biojava
ci_job_date 1753217725 1753217725
ci_job_id 1042184540 1042184540
ci_pipeline_id 71377968 71377968
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-zfyrx7zua-project-304-concurrent-1-hs3nlbib 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux Linux runner-zfyrx7zua-project-304-concurrent-1-hs3nlbib 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 10 metrics, 2 unstable metrics.

Execution time for tomcat 
gantt
    title tomcat - execution time [CI 0.99] : candidate=1.51.1-SNAPSHOT~69efa01716, baseline=1.51.1-SNAPSHOT~7b6259d446
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.472 ms) : 1460, 1483
.   : milestone, 1472,
appsec (3.616 ms) : 3401, 3831
.   : milestone, 3616,
iast (2.187 ms) : 2125, 2250
.   : milestone, 2187,
iast_GLOBAL (2.232 ms) : 2169, 2295
.   : milestone, 2232,
profiling (2.053 ms) : 2001, 2105
.   : milestone, 2053,
tracing (2.001 ms) : 1952, 2049
.   : milestone, 2001,
section candidate
no_agent (1.476 ms) : 1464, 1488
.   : milestone, 1476,
appsec (3.669 ms) : 3454, 3885
.   : milestone, 3669,
iast (2.19 ms) : 2127, 2252
.   : milestone, 2190,
iast_GLOBAL (2.25 ms) : 2187, 2314
.   : milestone, 2250,
profiling (2.485 ms) : 2311, 2658
.   : milestone, 2485,
tracing (2.008 ms) : 1959, 2057
.   : milestone, 2008,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.472 ms [1.46 ms, 1.483 ms] -
appsec 3.616 ms [3.401 ms, 3.831 ms] 2.144 ms (145.7%)
iast 2.187 ms [2.125 ms, 2.25 ms] 715.569 µs (48.6%)
iast_GLOBAL 2.232 ms [2.169 ms, 2.295 ms] 760.056 µs (51.6%)
profiling 2.053 ms [2.001 ms, 2.105 ms] 581.849 µs (39.5%)
tracing 2.001 ms [1.952 ms, 2.049 ms] 529.247 µs (36.0%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.476 ms [1.464 ms, 1.488 ms] -
appsec 3.669 ms [3.454 ms, 3.885 ms] 2.193 ms (148.6%)
iast 2.19 ms [2.127 ms, 2.252 ms] 713.654 µs (48.3%)
iast_GLOBAL 2.25 ms [2.187 ms, 2.314 ms] 774.185 µs (52.5%)
profiling 2.485 ms [2.311 ms, 2.658 ms] 1.009 ms (68.3%)
tracing 2.008 ms [1.959 ms, 2.057 ms] 531.969 µs (36.0%)
Execution time for biojava 
gantt
    title biojava - execution time [CI 0.99] : candidate=1.51.1-SNAPSHOT~69efa01716, baseline=1.51.1-SNAPSHOT~7b6259d446
    dateFormat X
    axisFormat %s
section baseline
no_agent (15.538 s) : 15538000, 15538000
.   : milestone, 15538000,
appsec (14.937 s) : 14937000, 14937000
.   : milestone, 14937000,
iast (18.011 s) : 18011000, 18011000
.   : milestone, 18011000,
iast_GLOBAL (17.808 s) : 17808000, 17808000
.   : milestone, 17808000,
profiling (15.36 s) : 15360000, 15360000
.   : milestone, 15360000,
tracing (14.699 s) : 14699000, 14699000
.   : milestone, 14699000,
section candidate
no_agent (15.503 s) : 15503000, 15503000
.   : milestone, 15503000,
appsec (14.991 s) : 14991000, 14991000
.   : milestone, 14991000,
iast (18.875 s) : 18875000, 18875000
.   : milestone, 18875000,
iast_GLOBAL (18.006 s) : 18006000, 18006000
.   : milestone, 18006000,
profiling (15.573 s) : 15573000, 15573000
.   : milestone, 15573000,
tracing (14.927 s) : 14927000, 14927000
.   : milestone, 14927000,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 15.538 s [15.538 s, 15.538 s] -
appsec 14.937 s [14.937 s, 14.937 s] -601.0 ms (-3.9%)
iast 18.011 s [18.011 s, 18.011 s] 2.473 s (15.9%)
iast_GLOBAL 17.808 s [17.808 s, 17.808 s] 2.27 s (14.6%)
profiling 15.36 s [15.36 s, 15.36 s] -178.0 ms (-1.1%)
tracing 14.699 s [14.699 s, 14.699 s] -839.0 ms (-5.4%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 15.503 s [15.503 s, 15.503 s] -
appsec 14.991 s [14.991 s, 14.991 s] -512.0 ms (-3.3%)
iast 18.875 s [18.875 s, 18.875 s] 3.372 s (21.8%)
iast_GLOBAL 18.006 s [18.006 s, 18.006 s] 2.503 s (16.1%)
profiling 15.573 s [15.573 s, 15.573 s] 70.0 ms (0.5%)
tracing 14.927 s [14.927 s, 14.927 s] -576.0 ms (-3.7%)

@pr-commenter
Copy link

pr-commenter bot commented Jul 22, 2025 •
edited
Loading

Kafka / producer-benchmark

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master kr-igor/fix-flaky-test
git_commit_date 1753205644 1753215515
git_commit_sha f21ceea 69efa01
See matching parameters
Baseline Candidate
ci_job_date 1753216787 1753216787
ci_job_id 1042184543 1042184543
ci_pipeline_id 71377968 71377968
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
jdkVersion 11.0.25 11.0.25
jmhVersion 1.36 1.36
jvm /usr/lib/jvm/java-11-openjdk-amd64/bin/java /usr/lib/jvm/java-11-openjdk-amd64/bin/java
jvmArgs -Dfile.encoding=UTF-8 -Djava.io.tmpdir=/go/src/github.com/DataDog/apm-reliability/dd-trace-java/platform/src/producer-benchmark/build/tmp/jmh -Duser.country=US -Duser.language=en -Duser.variant -Dfile.encoding=UTF-8 -Djava.io.tmpdir=/go/src/github.com/DataDog/apm-reliability/dd-trace-java/platform/src/producer-benchmark/build/tmp/jmh -Duser.country=US -Duser.language=en -Duser.variant
vmName OpenJDK 64-Bit Server VM OpenJDK 64-Bit Server VM
vmVersion 11.0.25+9-post-Ubuntu-1ubuntu122.04 11.0.25+9-post-Ubuntu-1ubuntu122.04

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 3 metrics, 0 unstable metrics.

See unchanged results
scenario Δ mean throughput
scenario:not-instrumented/KafkaProduceBenchmark.benchProduce same
scenario:only-tracing-dsm-disabled-benchmarks/KafkaProduceBenchmark.benchProduce same
scenario:only-tracing-dsm-enabled-benchmarks/KafkaProduceBenchmark.benchProduce same

@AlexeyKuznetsov-DD AlexeyKuznetsov-DD added the tag: no release notes Changes to exclude from release notes label Jul 22, 2025
@pr-commenter
Copy link

pr-commenter bot commented Jul 22, 2025

Kafka / consumer-benchmark

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master kr-igor/fix-flaky-test
git_commit_date 1753205644 1753215515
git_commit_sha f21ceea 69efa01
See matching parameters
Baseline Candidate
ci_job_date 1753216787 1753216787
ci_job_id 1042184544 1042184544
ci_pipeline_id 71377968 71377968
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
jdkVersion 11.0.25 11.0.25
jmhVersion 1.36 1.36
jvm /usr/lib/jvm/java-11-openjdk-amd64/bin/java /usr/lib/jvm/java-11-openjdk-amd64/bin/java
jvmArgs -Dfile.encoding=UTF-8 -Djava.io.tmpdir=/go/src/github.com/DataDog/apm-reliability/dd-trace-java/platform/src/consumer-benchmark/build/tmp/jmh -Duser.country=US -Duser.language=en -Duser.variant -Dfile.encoding=UTF-8 -Djava.io.tmpdir=/go/src/github.com/DataDog/apm-reliability/dd-trace-java/platform/src/consumer-benchmark/build/tmp/jmh -Duser.country=US -Duser.language=en -Duser.variant
vmName OpenJDK 64-Bit Server VM OpenJDK 64-Bit Server VM
vmVersion 11.0.25+9-post-Ubuntu-1ubuntu122.04 11.0.25+9-post-Ubuntu-1ubuntu122.04

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 3 metrics, 0 unstable metrics.

See unchanged results
scenario Δ mean throughput
scenario:not-instrumented/KafkaConsumerBenchmark.benchConsume same
scenario:only-tracing-dsm-disabled-benchmarks/KafkaConsumerBenchmark.benchConsume same
scenario:only-tracing-dsm-enabled-benchmarks/KafkaConsumerBenchmark.benchConsume same

@kr-igor kr-igor merged commit 2469447 into master Jul 22, 2025
505 checks passed
@kr-igor kr-igor deleted the kr-igor/fix-flaky-test branch July 22, 2025 21:33
@github-actions github-actions bot added this to the 1.52.0 milestone Jul 22, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@AlexeyKuznetsov-DD AlexeyKuznetsov-DD AlexeyKuznetsov-DD approved these changes

Assignees

No one assigned

Labels

comp: data streams Data Streams Monitoring comp: testing Testing tag: no release notes Changes to exclude from release notes type: bug Bug report and fix

Projects

None yet

Milestone

1.52.0

Development

Successfully merging this pull request may close these issues.

2 participants

@kr-igor @AlexeyKuznetsov-DD