Skip to content

branch-4.1: [fix](paimon) Support Variant in native reader - #66503

Open
Gabriel39 wants to merge 7 commits into
apache:branch-4.1from
Gabriel39:fix/paimon-native-variant-4.1
Open

branch-4.1: [fix](paimon) Support Variant in native reader#66503
Gabriel39 wants to merge 7 commits into
apache:branch-4.1from
Gabriel39:fix/paimon-native-variant-4.1

Conversation

@Gabriel39

Copy link
Copy Markdown
Contributor

What's changed

  • Backport the projected shredded Variant exchange and TopN fixes from [fix](variant) Preserve projected shredded states across exchange #66446.
  • Let the Paimon native Parquet reader interpret schema-matched, unannotated Variant carriers without misclassifying ordinary structs.
  • Preserve native projection and predicate pruning for shredded Variant leaves, including mixed shredded and unshredded files.
  • Keep lazy Variant reconstruction counters alive after scanner profile destruction.
  • Add schema, reader, lifecycle, mixed-layout, native/JNI, subpath, null, array, and predicate coverage.

Tests

  • Focused ASAN unit test binary: 187 tests passed.
  • FE Release build and Checkstyle passed.
  • BE Release compilation passed.
  • test_paimon_catalog_variant regression suite passed with both JNI and native reads.

@Gabriel39
Gabriel39 requested a review from yiguolei as a code owner August 5, 2026 12:33
@hello-stephen

Copy link
Copy Markdown
Contributor

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@Gabriel39

Copy link
Copy Markdown
Contributor Author

run buildall

@Gabriel39

Copy link
Copy Markdown
Contributor Author

/review

@github-actions

github-actions Bot commented Aug 5, 2026

Copy link
Copy Markdown
Contributor

Codex automated review failed and did not complete.

Error: All Codex review accounts are usage-limited; earliest retry is 2026-08-08T03:32:00Z.
Workflow run: https://github.com/apache/doris/actions/runs/31006186020

Please trigger /review again after that time.

@Gabriel39

Copy link
Copy Markdown
Contributor Author

run buildall

@Gabriel39

Copy link
Copy Markdown
Contributor Author

/review

@github-actions

github-actions Bot commented Aug 5, 2026

Copy link
Copy Markdown
Contributor

Codex automated review failed and did not complete.

Error: All Codex review accounts are usage-limited; earliest retry is 2026-08-08T03:32:00Z.
Workflow run: https://github.com/apache/doris/actions/runs/31011912192

Please trigger /review again after that time.

@hello-stephen

Copy link
Copy Markdown
Contributor

BE Regression && UT Coverage Report

Increment line coverage 80.76% (684/847) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 73.77% (30484/41323)
Line Coverage 57.72% (335369/580978)
Region Coverage 54.59% (279011/511145)
Branch Coverage 55.54% (125003/225066)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants