Add alignment checks to NativeFunc mark methods by jbachorik · Pull Request #348 · DataDog/java-profiler

jbachorik · 2026-01-29T17:39:25Z

What does this PR do?:

Fixes NativeFunc mark methods to prevent ASAN segfaults and adds CI workflow improvements for label-based test configuration selection.

Core fix - NativeFunc marking:

Adds alignment validation to all NativeFunc mark-related methods (isMarked, mark getter/setter)
Prevents segfaults when accessing marks on non-NativeFunc allocated strings
Uses the same is_aligned() check pattern as libIndex()
Invalid pointers now return safe defaults (false, 0, or no-op) instead of crashing
Fixes inverted logic in is_aligned() helper
Adds comprehensive tests for marking behavior

CI workflow improvements:

Adds label-based test configuration: test:release, test:asan, test:tsan
PRs always run debug config; labels add additional configurations
30-second debounce on label events allows adding multiple labels before run starts
Fixes pre-existing bugs in check-for-pr job (broken jq expression)
Removes dead gh-release job with invalid workflow syntax

Motivation:

ASAN tests revealed a segfault in NativeFunc::mark() at codeCache.h:77. The profiler was calling mark() on function names that weren't allocated via NativeFunc::create(), causing invalid memory access.

The CI changes enable easily running ASAN tests on PRs by adding the test:asan label, which was needed to properly validate this fix.

How to test the change?:

Run ASAN tests: ./gradlew :ddprof-test:testAsan
Or add test:asan label to the PR to trigger ASAN CI run

The fix prevents crashes when the profiler encounters native frames from libraries without proper NativeFunc metadata.

For Datadog employees:

If this PR touches code that signs or publishes builds or packages, or handles
credentials of any kind, I've requested a review from @DataDog/security-design-and-guidance.
This PR doesn't touch any of that.

Unsure? Have a question? Request a review!

🤖 Generated with Claude Code

dd-octo-sts · 2026-01-29T17:41:31Z

Scan-Build Report

User:	runner@runnervmkj6or
Working Directory:	/home/runner/work/java-profiler/java-profiler/ddprof-lib/src/test/make
Command Line:	make -j4 clean all
Clang Version:	Ubuntu clang version 18.1.3 (1ubuntu1)
Date:	Thu Jan 29 17:40:39 2026

Bug Summary

Unused code
Bug Type	Quantity	Display?
All Bugs	9
Dead assignment	2
Dead initialization	6
Dead nested assignment	1

Reports

Bug Group	Bug Type ▾	File	Function/Method	Line	Path Length
Unused code	Dead assignment	libraryPatcher_linux.cpp	patch_library_unlocked	93	1
Unused code	Dead assignment	stackWalker.cpp	walkVM	575	1
Unused code	Dead initialization	javaApi.cpp	Java_com_datadoghq_profiler_JavaProfiler_getStatus0	119	1
Unused code	Dead initialization	profiler.cpp	runInternal	1653	1
Unused code	Dead initialization	flightRecorder.cpp	cleanupUnreferencedMethods	658	1
Unused code	Dead initialization	flightRecorder.cpp	cleanupUnreferencedMethods	691	1
Unused code	Dead initialization	flightRecorder.cpp	resolveMethod	351	1
Unused code	Dead initialization	javaApi.cpp	Java_com_datadoghq_profiler_OTelContext_setProcessCtx0	475	1
Unused code	Dead nested assignment	vmStructs.cpp	checkNativeBinding	782	1

ddprof-lib/src/main/cpp/codeCache.h

dd-octo-sts · 2026-01-30T09:25:53Z

Scan-Build Report

User:	runner@runnervmkj6or
Working Directory:	/home/runner/work/java-profiler/java-profiler/ddprof-lib/src/test/make
Command Line:	make -j4 clean all
Clang Version:	Ubuntu clang version 18.1.3 (1ubuntu1)
Date:	Fri Jan 30 09:24:59 2026

Bug Summary

Unused code
Bug Type	Quantity	Display?
All Bugs	9
Dead assignment	2
Dead initialization	6
Dead nested assignment	1

Reports

Bug Group	Bug Type ▾	File	Function/Method	Line	Path Length
Unused code	Dead assignment	libraryPatcher_linux.cpp	patch_library_unlocked	93	1
Unused code	Dead assignment	stackWalker.cpp	walkVM	575	1
Unused code	Dead initialization	javaApi.cpp	Java_com_datadoghq_profiler_JavaProfiler_getStatus0	119	1
Unused code	Dead initialization	profiler.cpp	runInternal	1653	1
Unused code	Dead initialization	flightRecorder.cpp	cleanupUnreferencedMethods	658	1
Unused code	Dead initialization	flightRecorder.cpp	cleanupUnreferencedMethods	691	1
Unused code	Dead initialization	flightRecorder.cpp	resolveMethod	351	1
Unused code	Dead initialization	javaApi.cpp	Java_com_datadoghq_profiler_OTelContext_setProcessCtx0	475	1
Unused code	Dead nested assignment	vmStructs.cpp	checkNativeBinding	782	1

dd-octo-sts · 2026-01-30T09:30:03Z

Scan-Build Report

User:	runner@runnervmkj6or
Working Directory:	/home/runner/work/java-profiler/java-profiler/ddprof-lib/src/test/make
Command Line:	make -j4 clean all
Clang Version:	Ubuntu clang version 18.1.3 (1ubuntu1)
Date:	Fri Jan 30 09:29:10 2026

Bug Summary

Unused code
Bug Type	Quantity	Display?
All Bugs	9
Dead assignment	2
Dead initialization	6
Dead nested assignment	1

Reports

Bug Group	Bug Type ▾	File	Function/Method	Line	Path Length
Unused code	Dead assignment	libraryPatcher_linux.cpp	patch_library_unlocked	93	1
Unused code	Dead assignment	stackWalker.cpp	walkVM	575	1
Unused code	Dead initialization	javaApi.cpp	Java_com_datadoghq_profiler_JavaProfiler_getStatus0	119	1
Unused code	Dead initialization	profiler.cpp	runInternal	1653	1
Unused code	Dead initialization	flightRecorder.cpp	cleanupUnreferencedMethods	658	1
Unused code	Dead initialization	flightRecorder.cpp	cleanupUnreferencedMethods	691	1
Unused code	Dead initialization	flightRecorder.cpp	resolveMethod	351	1
Unused code	Dead initialization	javaApi.cpp	Java_com_datadoghq_profiler_OTelContext_setProcessCtx0	475	1
Unused code	Dead nested assignment	vmStructs.cpp	checkNativeBinding	782	1

dd-octo-sts · 2026-01-30T10:10:46Z

Scan-Build Report

User:	runner@runnervmkj6or
Working Directory:	/home/runner/work/java-profiler/java-profiler/ddprof-lib/src/test/make
Command Line:	make -j4 clean all
Clang Version:	Ubuntu clang version 18.1.3 (1ubuntu1)
Date:	Fri Jan 30 10:09:51 2026

Bug Summary

Unused code
Bug Type	Quantity	Display?
All Bugs	9
Dead assignment	2
Dead initialization	6
Dead nested assignment	1

Reports

Bug Group	Bug Type ▾	File	Function/Method	Line	Path Length
Unused code	Dead assignment	libraryPatcher_linux.cpp	patch_library_unlocked	93	1
Unused code	Dead assignment	stackWalker.cpp	walkVM	575	1
Unused code	Dead initialization	javaApi.cpp	Java_com_datadoghq_profiler_JavaProfiler_getStatus0	119	1
Unused code	Dead initialization	profiler.cpp	runInternal	1653	1
Unused code	Dead initialization	flightRecorder.cpp	cleanupUnreferencedMethods	658	1
Unused code	Dead initialization	flightRecorder.cpp	cleanupUnreferencedMethods	691	1
Unused code	Dead initialization	flightRecorder.cpp	resolveMethod	351	1
Unused code	Dead initialization	javaApi.cpp	Java_com_datadoghq_profiler_OTelContext_setProcessCtx0	475	1
Unused code	Dead nested assignment	vmStructs.cpp	checkNativeBinding	782	1

Prevents ASAN segfault when accessing mark on non-NativeFunc strings. All mark methods now validate pointer alignment like libIndex() does. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>

- isMarked() → is_marked() - mark() getter → read_mark() - mark() setter → set_mark() Improves API clarity by distinguishing read/write operations. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>

Critical fix: is_aligned() had inverted logic using ~(alignment-1) instead of (alignment-1). This meant alignment checks always failed for non-NULL pointers, completely breaking the safety mechanism. Also fixed indentation in read_mark() and set_mark() to match other methods (4-space indent). 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>

Adds two new test suites to prevent marking from being accidentally disabled: 1. utils_ut.cpp - Tests is_aligned() directly - Verifies aligned pointers return true - Verifies unaligned pointers return false - Critical test catches inverted logic bug 2. nativefunc_ut.cpp - Tests NativeFunc end-to-end - Verifies marks can be set and read back - Tests all mark types (THREAD_ENTRY, COMPILER_ENTRY, etc.) - Critical test: MarkingDisabledDetection fails if is_aligned() broken These tests would have caught the recent is_aligned() bug where marking was silently disabled for all non-NULL pointers. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>

Feature: PR labels test:release, test:asan, test:tsan trigger additional test configurations. Debug always runs. Fixes: - check-for-pr: broken jq expression and variable handling - Removed dead gh-release job with invalid workflow syntax 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

dd-octo-sts · 2026-01-30T10:35:35Z

Scan-Build Report

User:	runner@runnervmkj6or
Working Directory:	/home/runner/work/java-profiler/java-profiler/ddprof-lib/src/test/make
Command Line:	make -j4 clean all
Clang Version:	Ubuntu clang version 18.1.3 (1ubuntu1)
Date:	Fri Jan 30 10:34:36 2026

Bug Summary

Unused code
Bug Type	Quantity	Display?
All Bugs	9
Dead assignment	2
Dead initialization	6
Dead nested assignment	1

Reports

Bug Group	Bug Type ▾	File	Function/Method	Line	Path Length
Unused code	Dead assignment	libraryPatcher_linux.cpp	patch_library_unlocked	93	1
Unused code	Dead assignment	stackWalker.cpp	walkVM	575	1
Unused code	Dead initialization	javaApi.cpp	Java_com_datadoghq_profiler_JavaProfiler_getStatus0	119	1
Unused code	Dead initialization	profiler.cpp	runInternal	1653	1
Unused code	Dead initialization	flightRecorder.cpp	cleanupUnreferencedMethods	658	1
Unused code	Dead initialization	flightRecorder.cpp	cleanupUnreferencedMethods	691	1
Unused code	Dead initialization	flightRecorder.cpp	resolveMethod	351	1
Unused code	Dead initialization	javaApi.cpp	Java_com_datadoghq_profiler_OTelContext_setProcessCtx0	475	1
Unused code	Dead nested assignment	vmStructs.cpp	checkNativeBinding	782	1

Uses hidden HTML marker () to find existing comments instead of GITHUB_ACTOR which doesn't work with app tokens. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

Applied the recommended change

dd-octo-sts · 2026-01-30T10:41:20Z

Scan-Build Report

User:	runner@runnervmkj6or
Working Directory:	/home/runner/work/java-profiler/java-profiler/ddprof-lib/src/test/make
Command Line:	make -j4 clean all
Clang Version:	Ubuntu clang version 18.1.3 (1ubuntu1)
Date:	Fri Jan 30 11:23:32 2026

Bug Summary

Unused code
Bug Type	Quantity	Display?
All Bugs	9
Dead assignment	2
Dead initialization	6
Dead nested assignment	1

Reports

Bug Group	Bug Type ▾	File	Function/Method	Line	Path Length
Unused code	Dead assignment	libraryPatcher_linux.cpp	patch_library_unlocked	93	1
Unused code	Dead assignment	stackWalker.cpp	walkVM	575	1
Unused code	Dead initialization	javaApi.cpp	Java_com_datadoghq_profiler_JavaProfiler_getStatus0	119	1
Unused code	Dead initialization	profiler.cpp	runInternal	1653	1
Unused code	Dead initialization	flightRecorder.cpp	cleanupUnreferencedMethods	658	1
Unused code	Dead initialization	flightRecorder.cpp	cleanupUnreferencedMethods	691	1
Unused code	Dead initialization	flightRecorder.cpp	resolveMethod	351	1
Unused code	Dead initialization	javaApi.cpp	Java_com_datadoghq_profiler_OTelContext_setProcessCtx0	475	1
Unused code	Dead nested assignment	vmStructs.cpp	checkNativeBinding	782	1

jbachorik · 2026-01-30T10:43:51Z

@rkennke @zhengyu123 I did the renames to make the code clearer. Also, added some tests and fixed the alignment check bug.

The CI changes are not strictly related but they are fixing some annoyances I found when working on this. Since I really want to get this merged ASAP and create a release so we can consume it in dd-trace-java, I opted for including all those changes in this PR as separate commits.

rkennke

Looks good, but what about Zhengyu's suggestion?

ddprof-lib/src/main/cpp/codeCache.h

rkennke

Thanks! Looks good!

jbachorik added the AI label Jan 29, 2026

jbachorik marked this pull request as ready for review January 29, 2026 17:40

jbachorik requested a review from a team as a code owner January 29, 2026 17:40

jbachorik requested a review from zhengyu123 January 29, 2026 17:40

zhengyu123 previously requested changes Jan 29, 2026

View reviewed changes

ddprof-lib/src/main/cpp/codeCache.h Show resolved Hide resolved

rkennke reviewed Jan 30, 2026

View reviewed changes

ddprof-lib/src/main/cpp/codeCache.h Outdated Show resolved Hide resolved

jbachorik added the test:asan Run CI tests with AddressSanitizer configuration label Jan 30, 2026

jbachorik and others added 5 commits January 30, 2026 11:30

jbachorik force-pushed the jb/marking_fix branch from 3262aa5 to ae581cc Compare January 30, 2026 10:33

rkennke requested changes Jan 30, 2026

View reviewed changes

ddprof-lib/src/main/cpp/codeCache.h Show resolved Hide resolved

Delegate 'is_marked' to 'read_mark() != 0'

9349714

rkennke approved these changes Jan 30, 2026

View reviewed changes

jbachorik merged commit ec1b4ce into main Jan 30, 2026
133 of 136 checks passed

jbachorik deleted the jb/marking_fix branch January 30, 2026 11:40

github-actions bot added this to the 1.37.0 milestone Jan 30, 2026

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add alignment checks to NativeFunc mark methods#348

Add alignment checks to NativeFunc mark methods#348
jbachorik merged 7 commits intomainfrom
jb/marking_fix

jbachorik commented Jan 29, 2026 •

edited

Loading

Uh oh!

dd-octo-sts bot commented Jan 29, 2026

Uh oh!

Uh oh!

Uh oh!

dd-octo-sts bot commented Jan 30, 2026

Uh oh!

dd-octo-sts bot commented Jan 30, 2026

Uh oh!

dd-octo-sts bot commented Jan 30, 2026

Uh oh!

dd-octo-sts bot commented Jan 30, 2026

Uh oh!

dd-octo-sts bot commented Jan 30, 2026 •

edited

Loading

Uh oh!

jbachorik commented Jan 30, 2026

Uh oh!

rkennke left a comment

Uh oh!

Uh oh!

rkennke left a comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

jbachorik commented Jan 29, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

dd-octo-sts bot commented Jan 29, 2026

Scan-Build Report

Bug Summary

Reports

Uh oh!

Uh oh!

Uh oh!

dd-octo-sts bot commented Jan 30, 2026

Scan-Build Report

Bug Summary

Reports

Uh oh!

dd-octo-sts bot commented Jan 30, 2026

Scan-Build Report

Bug Summary

Reports

Uh oh!

dd-octo-sts bot commented Jan 30, 2026

Scan-Build Report

Bug Summary

Reports

Uh oh!

dd-octo-sts bot commented Jan 30, 2026

Scan-Build Report

Bug Summary

Reports

Uh oh!

dd-octo-sts bot commented Jan 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Scan-Build Report

Bug Summary

Reports

Uh oh!

jbachorik commented Jan 30, 2026

Uh oh!

rkennke left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

rkennke left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

jbachorik commented Jan 29, 2026 •

edited

Loading

dd-octo-sts bot commented Jan 30, 2026 •

edited

Loading