Skip to content

GH-47825: [C++] Fix the issue that bitmap ops overriding partial leading byte #47912

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 2 commits into
base: main
Choose a base branch
from
Open

GH-47825: [C++] Fix the issue that bitmap ops overriding partial leading byte #47912

wants to merge 2 commits into from
+1,549 −1,444

Conversation

@zanmato1984
Copy link
Contributor

@zanmato1984 zanmato1984 commented Oct 22, 2025 •
edited by github-actions bot
Loading

Rationale for this change

BitmapOp overrides partial leading byte for unaligned offset.

What changes are included in this PR?

Fix by properly handle the leading byte in the fast path of BitmapOp.

Are these changes tested?

Bitmap op UT included.

E2E test for if_else function execution incluced.

Are there any user-facing changes?

None.

This PR contains a "Critical Fix". (If the changes fix either (a) a security vulnerability, (b) a bug that caused incorrect or invalid data to be produced, or (c) a bug that causes a crash (even when the API contract is upheld), please provide explanation. If not, you can remove this.)

@zanmato1984 zanmato1984 requested a review from pitrou October 22, 2025 23:26
@zanmato1984 zanmato1984 added the Critical Fix Bugfixes for security vulnerabilities, crashes, or invalid data. label Oct 22, 2025
pitrou
pitrou reviewed Oct 23, 2025
View reviewed changes
cpp/src/arrow/util/bitmap_test.cc Outdated

namespace arrow::internal {

TEST(BitmapOpsTest, PartialLeadingByteSafety) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since CopyBitmap and others are tested in bit_util_test.cc, should we move those tests here?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, saw the bunch of cases in bit_util_test.cc are actually about bitmap_ops.cc. Plus the bit_util_test.cc are already over 2500 LOC. Will do.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Moved a bunch of existing "bitmap" related tests (1000+ lines) from bit_util_test.cc to the new bitmap_test.cc. Now the PR is big enough to review :)

cpp/src/arrow/util/CMakeLists.txt Outdated
SOURCES
align_util_test.cc
atfork_test.cc
bitmap_test.cc
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would rather add this to bit-utility-test than to the already overcrowded utility-test.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, sounds right. Didn't see bit-utility-test is actually a collection of several test files. Will do.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated.

@pitrou
Copy link
Member

pitrou commented Oct 23, 2025

@github-actions crossbow submit -g cpp

@github-actions github-actions bot added awaiting committer review Awaiting committer review and removed awaiting review Awaiting review labels Oct 23, 2025
@github-actions
Copy link

Revision: 106c633

Submitted crossbow builds: ursacomputing/crossbow @ actions-4b57902598

Task Status
example-cpp-minimal-build-static GitHub Actions
example-cpp-minimal-build-static-system-dependency GitHub Actions
example-cpp-tutorial GitHub Actions
test-build-cpp-fuzz GitHub Actions
test-conda-cpp GitHub Actions
test-conda-cpp-valgrind GitHub Actions
test-cuda-cpp-ubuntu-22.04-cuda-11.7.1 GitHub Actions
test-debian-12-cpp-amd64 GitHub Actions
test-debian-12-cpp-i386 GitHub Actions
test-fedora-42-cpp GitHub Actions
test-ubuntu-22.04-cpp GitHub Actions
test-ubuntu-22.04-cpp-20 GitHub Actions
test-ubuntu-22.04-cpp-bundled GitHub Actions
test-ubuntu-22.04-cpp-emscripten GitHub Actions
test-ubuntu-22.04-cpp-no-threading GitHub Actions
test-ubuntu-24.04-cpp GitHub Actions
test-ubuntu-24.04-cpp-bundled-offline GitHub Actions
test-ubuntu-24.04-cpp-gcc-13-bundled GitHub Actions
test-ubuntu-24.04-cpp-gcc-14 GitHub Actions
test-ubuntu-24.04-cpp-minimal-with-formats GitHub Actions
test-ubuntu-24.04-cpp-thread-sanitizer GitHub Actions

@pitrou
Copy link
Member

pitrou commented Oct 23, 2025

@zanmato1984 In addition to the comments above, can you also rebase from git main?

@zanmato1984
Copy link
Contributor Author

@zanmato1984 In addition to the comments above, can you also rebase from git main?

Thanks for looking. Rebased.

@zanmato1984
Copy link
Contributor Author

@github-actions crossbow submit -g cpp

@github-actions
Copy link

Revision: 37a41be

Submitted crossbow builds: ursacomputing/crossbow @ actions-cf05f6aa35

Task Status
example-cpp-minimal-build-static GitHub Actions
example-cpp-minimal-build-static-system-dependency GitHub Actions
example-cpp-tutorial GitHub Actions
test-build-cpp-fuzz GitHub Actions
test-conda-cpp GitHub Actions
test-conda-cpp-valgrind GitHub Actions
test-cuda-cpp-ubuntu-22.04-cuda-11.7.1 GitHub Actions
test-debian-12-cpp-amd64 GitHub Actions
test-debian-12-cpp-i386 GitHub Actions
test-fedora-42-cpp GitHub Actions
test-ubuntu-22.04-cpp GitHub Actions
test-ubuntu-22.04-cpp-20 GitHub Actions
test-ubuntu-22.04-cpp-bundled GitHub Actions
test-ubuntu-22.04-cpp-emscripten GitHub Actions
test-ubuntu-22.04-cpp-no-threading GitHub Actions
test-ubuntu-24.04-cpp GitHub Actions
test-ubuntu-24.04-cpp-bundled-offline GitHub Actions
test-ubuntu-24.04-cpp-gcc-13-bundled GitHub Actions
test-ubuntu-24.04-cpp-gcc-14 GitHub Actions
test-ubuntu-24.04-cpp-minimal-with-formats GitHub Actions
test-ubuntu-24.04-cpp-thread-sanitizer GitHub Actions

@zanmato1984
Copy link
Contributor Author

Would you take another look? @pitrou thanks.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@pitrou pitrou pitrou left review comments

Assignees

No one assigned

Labels

awaiting committer review Awaiting committer review backport-candidate Component: C++ Critical Fix Bugfixes for security vulnerabilities, crashes, or invalid data.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@zanmato1984 @pitrou