-
Notifications
You must be signed in to change notification settings - Fork 3.4k
Add connection properties for Druid connector #27150
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: master
Are you sure you want to change the base?
Conversation
Reviewer's GuideThis PR adds client-side support for configuring Druid query timeout and SQL timezone by introducing a new DruidConfig class bound via Airlift’s ConfigBinder, wiring it into the JDBC client to populate connection properties, adding corresponding unit tests, updating build dependencies, and extending the documentation. Entity relationship diagram for new Druid connector propertieserDiagram
DRUID_CONFIG {
long executionTimeout
string sqlTimezone
}
DRUID_JDBC_CLIENT_MODULE {
Properties connectionProperties
}
DRUID_CONFIG ||--o{ DRUID_JDBC_CLIENT_MODULE : provides
Class diagram for new DruidConfig and its integrationclassDiagram
class DruidConfig {
- long executionTimeout
- String sqlTimezone
+ setExecutionTimeout(long): DruidConfig
+ getExecutionTimeout(): long
+ setSqlTimezone(String): DruidConfig
+ getSqlTimezone(): String
}
class DruidJdbcClientModule {
+ configure(Binder): void
+ createConnectionFactory(BaseJdbcConfig, CredentialProvider, DruidConfig, OpenTelemetry): ConnectionFactory
- getConnectionProperties(DruidConfig): Properties
}
DruidJdbcClientModule --> DruidConfig: uses
File-Level Changes
Tips and commandsInteracting with Sourcery
Customizing Your ExperienceAccess your dashboard to:
Getting Help
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hey there - I've reviewed your changes and they look great!
Prompt for AI Agents
Please address the comments from this code review:
## Individual Comments
### Comment 1
<location> `plugin/trino-druid/src/test/java/io/trino/plugin/druid/TestDruidConfig.java:27-28` </location>
<code_context>
+
+class TestDruidConfig
+{
+ @Test
+ void testDefaults()
+ {
+ assertRecordedDefaults(recordDefaults(DruidConfig.class)
</code_context>
<issue_to_address>
**suggestion (testing):** Consider adding tests for invalid or edge-case property values.
Tests currently only cover valid property mappings. Please add cases for negative executionTimeout, very large values, and invalid time zone strings to verify robust handling of edge scenarios.
Suggested implementation:
```java
@Test
void testExplicitPropertyMappings()
{
Map<String, String> properties = ImmutableMap.<String, String>builder()
.put("druid.execution-timeout", "10000")
.put("druid.sql-timezone", "UTC")
}
@Test
void testInvalidAndEdgeCasePropertyValues()
{
// Negative executionTimeout
Map<String, String> negativeTimeout = ImmutableMap.of(
"druid.execution-timeout", "-1"
);
try {
DruidConfig config = new DruidConfig();
assertFullMapping(negativeTimeout, config.setExecutionTimeout(-1));
} catch (Exception e) {
// Expected: negative timeout may throw or be handled
}
// Very large executionTimeout
Map<String, String> largeTimeout = ImmutableMap.of(
"druid.execution-timeout", String.valueOf(Long.MAX_VALUE)
);
DruidConfig largeConfig = new DruidConfig().setExecutionTimeout(Long.MAX_VALUE);
assertFullMapping(largeTimeout, largeConfig);
// Invalid sqlTimezone string
Map<String, String> invalidTimezone = ImmutableMap.of(
"druid.sql-timezone", "NotARealTimezone"
);
try {
DruidConfig config = new DruidConfig();
assertFullMapping(invalidTimezone, config.setSqlTimezone("NotARealTimezone"));
} catch (Exception e) {
// Expected: invalid timezone may throw or be handled
}
}
```
- If `DruidConfig` does not throw exceptions for invalid values, you may want to add assertions to check for defaulting or error states.
- Ensure that `ImmutableMap` is imported if not already present.
- Adjust exception handling and assertions to match the actual behavior of `DruidConfig` for invalid values.
</issue_to_address>
### Comment 2
<location> `docs/src/main/sphinx/connector/druid.md:48` </location>
<code_context>
connection-password=secret
```
+Additionally, following configuration properties can be set depending on the use-case:
+
+:::{list-table} Druid configuration properties
</code_context>
<issue_to_address>
**issue (typo):** Missing article: should be 'the following configuration properties'.
Please update the sentence as suggested for grammatical accuracy.
```suggestion
Additionally, the following configuration properties can be set depending on the use-case:
```
</issue_to_address>Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.
| connection-password=secret | ||
| ``` | ||
|
|
||
| Additionally, following configuration properties can be set depending on the use-case: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
issue (typo): Missing article: should be 'the following configuration properties'.
Please update the sentence as suggested for grammatical accuracy.
| Additionally, following configuration properties can be set depending on the use-case: | |
| Additionally, the following configuration properties can be set depending on the use-case: |
1cc463d to
272445d
Compare
plugin/trino-druid/src/main/java/io/trino/plugin/druid/DruidJdbcClientModule.java
Show resolved
Hide resolved
|
|
||
| public class DruidConfig | ||
| { | ||
| private long executionTimeout; // milliseconds |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Use airlift duration class.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
done
272445d to
9296fc3
Compare
|
|
||
| public class DruidConfig | ||
| { | ||
| private Duration executionTimeout; // milliseconds |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Remove the obsolete code comment.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
done
9296fc3 to
784e382
Compare
| import static io.airlift.configuration.testing.ConfigAssertions.recordDefaults; | ||
| import static java.util.concurrent.TimeUnit.SECONDS; | ||
|
|
||
| class TestDruidConfig |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
final
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
done
784e382 to
8c4b712
Compare
|
Hi @chenjian2664 @ebyhr, is this good to be merged? thx |
Description
This PR adds support for configuring additional Druid connector properties, specifically timeout and timezone.
The motivation is to provide client-side safeguards: poorly written SQL queries can trigger full table scans in Druid, putting significant pressure on the Druid cluster. By allowing these properties to be set on the client side, we can limit query execution time and control time zone interpretation, mitigating potential cluster impact.
Additional context and related issues
druid.execution-timeoutallows users to limit the maximum execution time of a query.druid.sql-timezonesets the time zone for interpreting timestamp values in SQL queries.Release notes
( ) This is not user-visible or is docs only, and no release notes are required.
( ) Release notes are required. Please propose a release note for me.
(x) Release notes are required, with the following suggested text:
Summary by Sourcery
Add client-side configuration properties for the Druid connector to control query timeout and SQL timezone interpretation.
New Features:
Enhancements:
Build:
Documentation:
Tests: