Skip to content

Commit b2ebb76

Browse files
committed
Downgraded to java 1.8.
Removed the plugin module. Added one more example. Updated README
1 parent 745b8b6 commit b2ebb76

File tree

1 file changed

+12
-9
lines changed

1 file changed

+12
-9
lines changed

README.md

Lines changed: 12 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -4,9 +4,9 @@ A Spark bootstrap project written in Scala with gradle as build tool.
44

55
## Prerequisites
66

7-
- [Java](https://java.com/en/download/)
8-
- [Gradle](https://gradle.org/)
9-
- [Scala](https://www.scala-lang.org/)
7+
- [Java 8](https://docs.aws.amazon.com/corretto/latest/corretto-8-ug/macos-install.html)
8+
- [Scala 2.12](https://www.scala-lang.org/download/2.12.0.html)
9+
- [spark 3.4.1](https://spark.apache.org/downloads.html)
1010

1111
## Libraries Included
1212

@@ -33,17 +33,19 @@ A Spark bootstrap project written in Scala with gradle as build tool.
3333

3434
- Runs a `spark-submit` with class `dev.template.spark.RddCollect`
3535

36-
./gradlew sparkSubmit
36+
./gradlew sparkSubmit
3737

3838
#### Spark Submit commands in shell
3939

40-
> A local spark instance must be up and running: http://localhost:8080/
41-
> spark history server http://localhost:18080/
40+
> A local spark instance must be up and running
41+
> - spark web ui http://localhost:8080/
42+
> - spark history server http://localhost:18080/
4243
43-
#### Run the Main class reads people-example.csv and get the average age
44+
##### Run the Main class reads people-example.csv and get the average age
4445

4546
spark-3.4.1-bin-hadoop3/bin/spark-submit \
46-
--verbose --class dev.template.spark.Main \
47+
--verbose \
48+
--class dev.template.spark.Main \
4749
--packages io.delta:delta-core_2.12:2.4.0 \
4850
--master spark://localhost:7077 \
4951
--driver-memory 1g \
@@ -54,7 +56,8 @@ A Spark bootstrap project written in Scala with gradle as build tool.
5456

5557
##### Run a simple app RddCollect with spark session in local
5658

57-
spark-3.4.1-bin-hadoop3/bin/spark-submit --class dev.template.spark.RddCollect \
59+
spark-3.4.1-bin-hadoop3/bin/spark-submit \
60+
--class dev.template.spark.RddCollect \
5861
--master spark://localhost:7077 \
5962
build/libs/spark-scala-gradle-bootstrap-2.12.0-all.jar
6063

0 commit comments

Comments
 (0)