Skip to content

feat: add DROP pipe operator support#20864

Open
jayendra13 wants to merge 5 commits intoapache:mainfrom
jayendra13:pipe-drop-operator
Open

feat: add DROP pipe operator support#20864
jayendra13 wants to merge 5 commits intoapache:mainfrom
jayendra13:pipe-drop-operator

Conversation

@jayendra13
Copy link
Copy Markdown

@jayendra13 jayendra13 commented Mar 11, 2026

Which issue does this PR close?

Rationale for this change

The DROP pipe operator (|> DROP x, y) is part of the BigQuery pipe syntax tracked in #14660. It removes columns from the output, equivalent to SELECT * EXCEPT(x, y).

What changes are included in this PR?

  • Added PipeOperator::Drop match arm in query.rs that reuses existing SELECT * EXCEPT infrastructure (WildcardOptions with ExceptSelectItem)
  • Added SLT tests covering single column drop, multi-column drop, chaining with other pipe operators, and error case for non-existent columns
  • Added DROP to the pipe operator documentation in select.md

Are these changes tested?

Yes, four sqllogictest cases added in pipe_operator.slt:

  • Drop two columns
  • Drop single column
  • Chained with WHERE
  • Error on non-existent column

Are there any user-facing changes?

Yes, users can now use |> DROP column1, column2 in pipe operator queries (BigQuery dialect).

@github-actions github-actions Bot added documentation Improvements or additions to documentation sql SQL Planner sqllogictest SQL Logic Tests (.slt) labels Mar 11, 2026
@jayendra13 jayendra13 mentioned this pull request Mar 11, 2026
18 tasks
Copy link
Copy Markdown
Contributor

@simonvandel simonvandel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Very cool, thanks. The changes look good to me.

I'm not a committer, so someone else needs to do the final review.

@jayendra13
Copy link
Copy Markdown
Author

Very cool, thanks. The changes look good to me.

I'm not a committer, so someone else needs to do the final review.

@simonvandel, whom can I ask for the final review ? @alamb, does this look good? If so, I can pick either RENAME or SET. Thanks.

Copy link
Copy Markdown
Contributor

@Jefffrey Jefffrey left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for picking this up; just some minor comments on tests, and once it's merged up from main it should be good to go

}
PipeOperator::Drop { columns } => {
let Some((first, rest)) = columns.split_first() else {
return plan_err!("DROP requires at least one column");
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would be good to have a test case covering this

Copy link
Copy Markdown
Author

@jayendra13 jayendra13 Apr 22, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the https://github.com/google/googlesql/blob/master/docs/pipe-syntax.md#drop-pipe-operator makes column_name required (|> DROP column_name [, ...]), which would make this a parse error rather than a planner error, and it feels like it belongs in sqlparser instead. Could you confirm I'm reading the spec right? If so, I'll need to fix it in the upstream crate first.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not familiar with the spec, just basing it off the code I see here (if we need to ensure columns aren't empty here then it means its possible to be empty? Or its just safety?)

3.3

# Error: DROP non-existent column
statement error
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could we validate the specific error that comes here too?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation sql SQL Planner sqllogictest SQL Logic Tests (.slt)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants