第1关:Spark Shell

在右边命令行,逐个输入以下几个命令就ok了!

spark-shell
val textFile = spark.read.textFile("/data/workspace/myshixun/phone.txt")
val zt_ws_records = textFile.filter(line =>(line.contains("张涛") && line.contains("王世昌")) || (line.contains("王世昌") && line.contains("张涛")))
zt_ws_records.write.text("/root/re")
:quit

第2关:spark-submit提交

#!/bin/bash
 
cp -r  Spark/SparkRDD/target/project.jar /root
cd /opt/spark/dist/bin
#********** Begin **********#
./spark-submit \
  --class Student \
  --master local \
  /root/project.jar
#********** End **********#

Logo

技术共进,成长同行——讯飞AI开发者社区

更多推荐