@@ -4,9 +4,9 @@ A Spark bootstrap project written in Scala with gradle as build tool.
44
55## Prerequisites
66
7- - [ Java] ( https://java. com/en/download/ )
8- - [ Gradle ] ( https://gradle. org/ )
9- - [ Scala ] ( https://www.scala-lang .org/ )
7+ - [ Java 8 ] ( https://docs.aws.amazon. com/corretto/latest/corretto-8-ug/macos-install.html )
8+ - [ Scala 2.12 ] ( https://www.scala-lang. org/download/2.12.0.html )
9+ - [ spark 3.4.1 ] ( https://spark.apache .org/downloads.html )
1010
1111## Libraries Included
1212
@@ -33,17 +33,19 @@ A Spark bootstrap project written in Scala with gradle as build tool.
3333
3434- Runs a ` spark-submit ` with class ` dev.template.spark.RddCollect `
3535
36- ./gradlew sparkSubmit
36+ ./gradlew sparkSubmit
3737
3838#### Spark Submit commands in shell
3939
40- > A local spark instance must be up and running: http://localhost:8080/
41- > spark history server http://localhost:18080/
40+ > A local spark instance must be up and running
41+ > - spark web ui http://localhost:8080/
42+ > - spark history server http://localhost:18080/
4243
43- #### Run the Main class reads people-example.csv and get the average age
44+ ##### Run the Main class reads people-example.csv and get the average age
4445
4546 spark-3.4.1-bin-hadoop3/bin/spark-submit \
46- --verbose --class dev.template.spark.Main \
47+ --verbose \
48+ --class dev.template.spark.Main \
4749 --packages io.delta:delta-core_2.12:2.4.0 \
4850 --master spark://localhost:7077 \
4951 --driver-memory 1g \
@@ -54,7 +56,8 @@ A Spark bootstrap project written in Scala with gradle as build tool.
5456
5557##### Run a simple app RddCollect with spark session in local
5658
57- spark-3.4.1-bin-hadoop3/bin/spark-submit --class dev.template.spark.RddCollect \
59+ spark-3.4.1-bin-hadoop3/bin/spark-submit \
60+ --class dev.template.spark.RddCollect \
5861 --master spark://localhost:7077 \
5962 build/libs/spark-scala-gradle-bootstrap-2.12.0-all.jar
6063
0 commit comments