-
Notifications
You must be signed in to change notification settings - Fork 5.5k
misc: Move join prefilter optimizatino after connector optimization #26738
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: master
Are you sure you want to change the base?
Conversation
Reviewer's guide (collapsed on small PRs)Reviewer's GuideReorders the JoinPrefilter optimizer in the planner pipeline so that it runs after the connector-specific iterative optimization phase instead of before, allowing it to apply to plans produced or transformed by connector optimizations. Flow diagram for updated optimizer pipeline around JoinPrefilterflowchart TD
subgraph OptimizerPipeline
direction TB
InlineSqlFunctions[InlineSqlFunctions]
IterativeConnector[IterativeOptimizer_connector_specific]
SimplifyEmpty[SimplifyPlanWithEmptyInput]
PruneUnref[PruneUnreferencedOutputs]
JoinPrefilterOpt[JoinPrefilter]
IterativePost[IterativeOptimizer_post_join_prefilter]
end
InlineSqlFunctions --> IterativeConnector
IterativeConnector --> SimplifyEmpty
SimplifyEmpty --> PruneUnref
PruneUnref --> JoinPrefilterOpt
JoinPrefilterOpt --> IterativePost
classDef connector fill:#e3f2fd,stroke:#1e88e5,stroke-width:1px;
classDef rule fill:#f1f8e9,stroke:#7cb342,stroke-width:1px;
class IterativeConnector connector;
class IterativePost connector;
class InlineSqlFunctions,SimplifyEmpty,PruneUnref,JoinPrefilterOpt rule;
File-Level Changes
Tips and commandsInteracting with Sourcery
Customizing Your ExperienceAccess your dashboard to:
Getting Help
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hey there - I've reviewed your changes - here's some feedback:
- Since optimizer ordering is subtle and easy to regress, consider adding a brief code comment near
JoinPrefilterinPlanOptimizersexplaining why it must run after connector optimizations and afterSimplifyPlanWithEmptyInput/PruneUnreferencedOutputs. - Double-check whether
JoinPrefilterdepends on or affects any of the earlier iterative optimizers in this pipeline; if so, it might be worth explicitly enforcing or documenting assumptions about which transformations must precede it.
Prompt for AI Agents
Please address the comments from this code review:
## Overall Comments
- Since optimizer ordering is subtle and easy to regress, consider adding a brief code comment near `JoinPrefilter` in `PlanOptimizers` explaining why it must run after connector optimizations and after `SimplifyPlanWithEmptyInput`/`PruneUnreferencedOutputs`.
- Double-check whether `JoinPrefilter` depends on or affects any of the earlier iterative optimizers in this pipeline; if so, it might be worth explicitly enforcing or documenting assumptions about which transformations must precede it.Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.
Description
As title, so that join prefilter can be used for plans which are changed after connector optimizations.
Motivation and Context
As in description
Impact
As in description
Test Plan
Existing unit tests
Contributor checklist
Release Notes
Please follow release notes guidelines and fill in the release notes below.