Skip to content

Commit 21b6166

Browse files
pan3793dongjoon-hyun
authored andcommitted
[SPARK-54488][BUILD] Fix Connect JVM client leaks protobuf-java-util in shading rules
### What changes were proposed in this pull request? Fix Connect JVM client leaks `protobuf-java-util` in shading rules by setting `protobuf-java-util` to `compile` scope (was `provided`) ### Why are the changes needed? A bug fix, `spark-connect-client-jvm` jar should correctly shade all protobuf classes, same as the `spark-connect` server jar does. ### Does this PR introduce _any_ user-facing change? Yes, users who pull `spark-connect-client-jvm` as dependencies, will not see transitive `protobuf-java-util` dependency. ### How was this patch tested? ``` build/mvn clean install -DskipTests -pl sql/connect/client/jvm -am ``` ```patch [INFO] --- shade:3.6.0:shade (default) spark-connect-client-jvm_2.13 --- [INFO] Including org.apache.spark:spark-connect-common_2.13:jar:4.2.0-SNAPSHOT in the shaded jar. [INFO] Including io.grpc:grpc-netty:jar:1.76.0 in the shaded jar. [INFO] Including io.grpc:grpc-api:jar:1.76.0 in the shaded jar. [INFO] Including org.codehaus.mojo:animal-sniffer-annotations:jar:1.24 in the shaded jar. [INFO] Including io.grpc:grpc-core:jar:1.76.0 in the shaded jar. [INFO] Including com.google.android:annotations:jar:4.1.1.4 in the shaded jar. [INFO] Including io.grpc:grpc-context:jar:1.76.0 in the shaded jar. [INFO] Including io.perfmark:perfmark-api:jar:0.27.0 in the shaded jar. [INFO] Including io.grpc:grpc-util:jar:1.76.0 in the shaded jar. [INFO] Including io.grpc:grpc-protobuf:jar:1.76.0 in the shaded jar. [INFO] Including com.google.api.grpc:proto-google-common-protos:jar:2.59.2 in the shaded jar. [INFO] Including io.grpc:grpc-protobuf-lite:jar:1.76.0 in the shaded jar. [INFO] Including io.grpc:grpc-services:jar:1.76.0 in the shaded jar. [INFO] Including io.grpc:grpc-stub:jar:1.76.0 in the shaded jar. [INFO] Including io.grpc:grpc-inprocess:jar:1.76.0 in the shaded jar. [INFO] Including io.netty:netty-codec-http2:jar:4.2.7.Final in the shaded jar. [INFO] Including io.netty:netty-common:jar:4.2.7.Final in the shaded jar. [INFO] Including io.netty:netty-buffer:jar:4.2.7.Final in the shaded jar. [INFO] Including io.netty:netty-transport:jar:4.2.7.Final in the shaded jar. [INFO] Including io.netty:netty-resolver:jar:4.2.7.Final in the shaded jar. [INFO] Including io.netty:netty-codec-base:jar:4.2.7.Final in the shaded jar. [INFO] Including io.netty:netty-handler:jar:4.2.7.Final in the shaded jar. [INFO] Including io.netty:netty-codec-http:jar:4.2.7.Final in the shaded jar. [INFO] Including io.netty:netty-codec-compression:jar:4.2.7.Final in the shaded jar. [INFO] Including io.netty:netty-handler-proxy:jar:4.2.7.Final in the shaded jar. [INFO] Including io.netty:netty-codec-socks:jar:4.2.7.Final in the shaded jar. [INFO] Including io.netty:netty-transport-native-unix-common:jar:4.2.7.Final in the shaded jar. [INFO] Including org.apache.spark:spark-sql-api_2.13:jar:4.2.0-SNAPSHOT in the shaded jar. [INFO] Including org.apache.arrow:arrow-vector:jar:18.3.0 in the shaded jar. [INFO] Including org.apache.arrow:arrow-format:jar:18.3.0 in the shaded jar. [INFO] Including org.apache.arrow:arrow-memory-core:jar:18.3.0 in the shaded jar. [INFO] Including com.google.flatbuffers:flatbuffers-java:jar:25.2.10 in the shaded jar. [INFO] Including org.apache.arrow:arrow-memory-netty:jar:18.3.0 in the shaded jar. [INFO] Including org.apache.arrow:arrow-memory-netty-buffer-patch:jar:18.3.0 in the shaded jar. [INFO] Including com.google.protobuf:protobuf-java:jar:4.33.0 in the shaded jar. + [INFO] Including com.google.protobuf:protobuf-java-util:jar:4.33.0 in the shaded jar. [INFO] Including com.google.code.gson:gson:jar:2.11.0 in the shaded jar. [INFO] Including com.google.guava:guava:jar:33.4.8-jre in the shaded jar. [INFO] Including com.google.guava:listenablefuture:jar:9999.0-empty-to-avoid-conflict-with-guava in the shaded jar. [INFO] Including com.google.guava:failureaccess:jar:1.0.3 in the shaded jar. ``` ### Was this patch authored or co-authored using generative AI tooling? No. Closes #53191 from pan3793/SPARK-54488. Authored-by: Cheng Pan <chengpan@apache.org> Signed-off-by: Dongjoon Hyun <dongjoon@apache.org>
1 parent 5a546de commit 21b6166

File tree

1 file changed

+5
-0
lines changed

1 file changed

+5
-0
lines changed

sql/connect/client/jvm/pom.xml

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -74,6 +74,11 @@
7474
<artifactId>protobuf-java</artifactId>
7575
<scope>compile</scope>
7676
</dependency>
77+
<dependency>
78+
<groupId>com.google.protobuf</groupId>
79+
<artifactId>protobuf-java-util</artifactId>
80+
<scope>compile</scope>
81+
</dependency>
7782
<dependency>
7883
<groupId>com.google.guava</groupId>
7984
<artifactId>guava</artifactId>

0 commit comments

Comments
 (0)