-
Notifications
You must be signed in to change notification settings - Fork 1.8k
feat: deprecate AggregateUDF: :is_nullable #18934
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
Jefffrey
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Surprisingly fewer changes than I expected
7f99f29 to
810cae5
Compare
martin-g
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
- https://github.com/codetyri0n/datafusion/blob/810cae5c0471a4050e2a9b4ac8580eaf5f039906/datafusion/ffi/src/udaf/mod.rs#L347 is not annotated with
#[expect(deprecated)] - https://github.com/codetyri0n/datafusion/blob/810cae5c0471a4050e2a9b4ac8580eaf5f039906/datafusion/functions-aggregate/src/count.rs#L299 - should something be done for overrides like this one ?
| self.inner.window_function_display_name(params) | ||
| } | ||
|
|
||
| pub fn is_nullable(&self) -> bool { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Should this method be deprecated too ?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
yep thanks!
thanks for catching the mod.rs miss @martin-g , i had left the count.rs file untouched as i dont see the signature or the implementation getting altered much in the near future |
810cae5 to
2aeedc8
Compare
|
can we merge this? |
|
Sorry I haven't gotten around to re-reviewing this after the latest changes |
Jefffrey
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think the changes look good in achieving what we want for the issue.
Some notes:
- We should look at UDAFs that implement
is_nullableand ensure they implementreturn_fieldin line with the deprecation; I think this only occurs for Count- Unfortunately it seems Rust can't flag this for us (implementations of deprecated methods)
- I think this PR doesn't introduce behaviour changes so long as any downstream UDAFs that implement
is_nullableeither don't implementreturn_field(in which case the default behaviour still usesis_nullable), or if they do implementreturn_fieldhopefully it is consistent withis_nullableotherwise behaviour may change- That latter case seems like a bug in the downstream anyway, though not sure how to communicate this 🤔
One more thing is I wonder how we should handle this for FFI; for example we have is_nullable as part of the FFI API I believe:
datafusion/datafusion/ffi/src/udaf/mod.rs
Lines 80 to 81 in a1bb74b
| /// FFI equivalent to the `is_nullable` of a [`AggregateUDF`] | |
| pub is_nullable: bool, |
Maybe @timsaucer can weigh in regarding FFI?
Apart from that, it would be good if could get some other opinions on if we want to proceed with this deprecation; I know for the UDF is_nullable deprecation there was some concern, see discussions on #14094 and #17074. cc @alamb if you have time
- Maybe we should include a note in the upgrade guide to make this more visible since Rust can't flag the implementation of a deprecated trait method 🤔
Which issue does this PR close?
AggregateUDFImpl::is_nullablein favour ofreturn_field#18882Rationale for this change
Are these changes tested?
Are there any user-facing changes?