Commit 21b6166
[SPARK-54488][BUILD] Fix Connect JVM client leaks
### What changes were proposed in this pull request?
Fix Connect JVM client leaks `protobuf-java-util` in shading rules by setting `protobuf-java-util` to `compile` scope (was `provided`)
### Why are the changes needed?
A bug fix, `spark-connect-client-jvm` jar should correctly shade all protobuf classes, same as the `spark-connect` server jar does.
### Does this PR introduce _any_ user-facing change?
Yes, users who pull `spark-connect-client-jvm` as dependencies, will not see transitive `protobuf-java-util` dependency.
### How was this patch tested?
```
build/mvn clean install -DskipTests -pl sql/connect/client/jvm -am
```
```patch
[INFO] --- shade:3.6.0:shade (default) spark-connect-client-jvm_2.13 ---
[INFO] Including org.apache.spark:spark-connect-common_2.13:jar:4.2.0-SNAPSHOT in the shaded jar.
[INFO] Including io.grpc:grpc-netty:jar:1.76.0 in the shaded jar.
[INFO] Including io.grpc:grpc-api:jar:1.76.0 in the shaded jar.
[INFO] Including org.codehaus.mojo:animal-sniffer-annotations:jar:1.24 in the shaded jar.
[INFO] Including io.grpc:grpc-core:jar:1.76.0 in the shaded jar.
[INFO] Including com.google.android:annotations:jar:4.1.1.4 in the shaded jar.
[INFO] Including io.grpc:grpc-context:jar:1.76.0 in the shaded jar.
[INFO] Including io.perfmark:perfmark-api:jar:0.27.0 in the shaded jar.
[INFO] Including io.grpc:grpc-util:jar:1.76.0 in the shaded jar.
[INFO] Including io.grpc:grpc-protobuf:jar:1.76.0 in the shaded jar.
[INFO] Including com.google.api.grpc:proto-google-common-protos:jar:2.59.2 in the shaded jar.
[INFO] Including io.grpc:grpc-protobuf-lite:jar:1.76.0 in the shaded jar.
[INFO] Including io.grpc:grpc-services:jar:1.76.0 in the shaded jar.
[INFO] Including io.grpc:grpc-stub:jar:1.76.0 in the shaded jar.
[INFO] Including io.grpc:grpc-inprocess:jar:1.76.0 in the shaded jar.
[INFO] Including io.netty:netty-codec-http2:jar:4.2.7.Final in the shaded jar.
[INFO] Including io.netty:netty-common:jar:4.2.7.Final in the shaded jar.
[INFO] Including io.netty:netty-buffer:jar:4.2.7.Final in the shaded jar.
[INFO] Including io.netty:netty-transport:jar:4.2.7.Final in the shaded jar.
[INFO] Including io.netty:netty-resolver:jar:4.2.7.Final in the shaded jar.
[INFO] Including io.netty:netty-codec-base:jar:4.2.7.Final in the shaded jar.
[INFO] Including io.netty:netty-handler:jar:4.2.7.Final in the shaded jar.
[INFO] Including io.netty:netty-codec-http:jar:4.2.7.Final in the shaded jar.
[INFO] Including io.netty:netty-codec-compression:jar:4.2.7.Final in the shaded jar.
[INFO] Including io.netty:netty-handler-proxy:jar:4.2.7.Final in the shaded jar.
[INFO] Including io.netty:netty-codec-socks:jar:4.2.7.Final in the shaded jar.
[INFO] Including io.netty:netty-transport-native-unix-common:jar:4.2.7.Final in the shaded jar.
[INFO] Including org.apache.spark:spark-sql-api_2.13:jar:4.2.0-SNAPSHOT in the shaded jar.
[INFO] Including org.apache.arrow:arrow-vector:jar:18.3.0 in the shaded jar.
[INFO] Including org.apache.arrow:arrow-format:jar:18.3.0 in the shaded jar.
[INFO] Including org.apache.arrow:arrow-memory-core:jar:18.3.0 in the shaded jar.
[INFO] Including com.google.flatbuffers:flatbuffers-java:jar:25.2.10 in the shaded jar.
[INFO] Including org.apache.arrow:arrow-memory-netty:jar:18.3.0 in the shaded jar.
[INFO] Including org.apache.arrow:arrow-memory-netty-buffer-patch:jar:18.3.0 in the shaded jar.
[INFO] Including com.google.protobuf:protobuf-java:jar:4.33.0 in the shaded jar.
+ [INFO] Including com.google.protobuf:protobuf-java-util:jar:4.33.0 in the shaded jar.
[INFO] Including com.google.code.gson:gson:jar:2.11.0 in the shaded jar.
[INFO] Including com.google.guava:guava:jar:33.4.8-jre in the shaded jar.
[INFO] Including com.google.guava:listenablefuture:jar:9999.0-empty-to-avoid-conflict-with-guava in the shaded jar.
[INFO] Including com.google.guava:failureaccess:jar:1.0.3 in the shaded jar.
```
### Was this patch authored or co-authored using generative AI tooling?
No.
Closes #53191 from pan3793/SPARK-54488.
Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Dongjoon Hyun <dongjoon@apache.org>protobuf-java-util in shading rules1 parent 5a546de commit 21b6166
1 file changed
+5
-0
lines changed| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
74 | 74 | | |
75 | 75 | | |
76 | 76 | | |
| 77 | + | |
| 78 | + | |
| 79 | + | |
| 80 | + | |
| 81 | + | |
77 | 82 | | |
78 | 83 | | |
79 | 84 | | |
| |||
0 commit comments