PS:
1.scala是开发spark平台的一种语言。
2.如果开发spark的话,用scala开发是非常好的,Python的话一般,用java的话就是效果不好。
---------------------------Scala的安装配置
PS:首先必须要有java环境
1.安装scala的2.10.5
PS:文件中的内容,object保存着所有的静态方法和变量
object HelloScala{ def main(args:Array[String]){ println("i love you scala") } }
PS:编写scala不能用这么笨的方法,需要专业的工具,安装插件如下图
https://blog.csdn.net/iamlihongwei/article/details/72783459?locationNum=10&fps=1
1. Scala基础
1.1. 声明变量
PS:无需指定类型,系统会自动识别
package cn.itcast.scala //变量名在前,类型在后
|
1.2. 常用类型
Scala和Java一样,有7种数值类型Byte、Char、Short、Int、Long、Float和Double(无包装类型)和一个Boolean类型
PS:没有包装类
1.1. 条件表达式
Scala的的条件表达式比较简洁,例如:
package cn.itcast.scala
|
----------------------------------------------------------------------------------------------------------------------------------
1.1. 块表达式
PS:就是复杂好看点
package cn.itcast.scala
|
PS:选择Scala->Scala,然后选择项目
PS:系统有两个setting,第一个setting仅对项目修改,第二个对整个软件修改。
PS:创建时有class是类,object是单例对象,Trait 相当于接口
------------------------------------------------------------------
PS: 1 to 10会自动赋值给res1,这个res是自动生成的
1.1. 循环
在scala中有for循环和while循环,用for循环比较多
for循环语法结构:for (i <- 表达式/数组/集合)
1.until包头不包尾 是为了打印数组角标
2.会根据不同结构类型动态的识别
package cn.itcast.scala //每次循环将区间中的一个值赋给i //每次迭代生成集合中的一个值
|
PS: _代表当前下的元素
PS:1 to 10还可以这样写
1.1. 调用方法和函数
Scala中的+ - * / %等操作符的作用与Java一样,位操作符 & | ^ >> <<也一样。只是有
一点特别的:这些操作符实际上是方法。例如:
a + b
是如下方法调用的简写:
a.+(b)
a 方法 b可以写成 a.方法(b)
------------------------------------------------------------------------------
1.1. 定义方法和函数---是不同的
1.1.1. 定义方法
PS:对集合进行相应的操作。
/** * Created by bee on 2018/3/24. */ object HelloScala { //定义一个方法 //方法m2参数要求是一个函数,函数的参数必须是两个Int类型 //返回值类型也是Int类型 def m1(f: (Int, Int) => Int) : Int = { f(2, 6) } //定义一个函数f1,参数是两个Int类型,返回值是一个Int类型 val f1 = (x: Int, y: Int) => x + y //再定义一个函数f2 val f2 = (m: Int, n: Int) => m * n //main方法 def main(args: Array[String]) { //调用m1方法,并传入f1函数 val r1 = m1(f1) println(r1) //调用m1方法,并传入f2函数 val r2 = m1(f2) println(r2) } }
--------------------------------------
PS:方法是func2,参数是int和double。返回的是元组(Double和Int),里面是函数体。
-----------------------------------------------------------这里的Unit相当于void
1. 数组、映射、元组、集合
1.1. 数组
1.1.1. 定长数组和变长数组
package cn.itcast.scala //将数组转换成数组缓冲,就可以看到原数组中的内容了 //定义一个长度为3的定长数组
//删除指定位置的元素,包括该位置
|
1.1.2. 遍历数组
1.增强for循环
2.好用的until会生成脚标,0 until 10 包含0不包含10
package cn.itcast.scala
|
1.1.1. 数组转换
yield关键字将原始的数组进行转换会产生一个新的数组,原始的数组不变
package cn.itcast.scala
|
------------------数组常用算法
------------------------------------
PS: 元组就是truple
PS:默认我们生成的序列是不可变的
package cn.itcast.collect //将一个元素添加到lst1的后面产生一个新的集合 |
可变的序列 import scala.collection.mutable._
package cn.itcast.collect
|
WordCount
PS:WordCountScala
val words =Array("hello huan huan","hello bee") //println(arr.sortWith(_>_).toBuffer) val result =words.flatMap(_.split(" ")).map((_,1))// Array((hello,1), (huan,1), (huan,1), (hello,1), (bee,1)) val result1= result.toList.groupBy(_._1)//“_”取每一个元素(hello,1), "._1"去下面的第一个元素 Map(bee -> List((bee,1)), huan -> List((huan,1), (huan,1)), hello -> List((hello,1), (hello,1)))
//val result1= result.toList.groupBy(_._2) 如果是2的话就是这样 ------ Map(1 -> List((hello,1), (huan,1), (huan,1), (hello,1), (bee,1)))
val result4 = result1.map(t=>(t._1,t._2.size))// t代表 huan -> List((huan,1), (huan,1))Map(bee -> 1, huan -> 2, hello -> 2)
val result5 = result4.toList.sortBy(_._2).reverse;// 转成List 然后排序 List((huan,3), (hello,2), (bee,1))
println(result4)// Map(bee -> 1, huan -> 2, hello -> 2)
----------------------------------
PS :
1. Scala par能够并行计算
-------------------------------------------------------
1.1. 类
1.1.1. 类的定义
//在Scala中,类并不用声明为public。 //Scala源文件中可以包含多个类,所有这些类都具有公有可见性。 //(相当与Java中用final修饰的变量) //用var修饰的变量既有getter又有setter
//类私有字段,只能在类的内部使用 } |
/** * Created by bee on 2018/3/25. */ class Person {//这里默认对象为空 val id="9527" var name = "huaan" //类私有字段,只能在类的内部使用
/* val 只有get 方法,*/
private var name1: String = "唐伯虎" //对象私有字段,访问权限更加严格的,Person类的方法只能访问到当前对象的字段 private[this] val pet = "小强" } //伴生对象 object Person{ def main (args: Array[String]){ val p = new Person println(p.id) println(p.name) println(p.name1)//只是在本类中能用,换个其他的就不行了 // println(p.pet)打印pet会报错 } }
PS:
1.该类上面第一行是包含构造函数的
/** *每个类都有主构造器,主构造器的参数直接放置类名后面,与类交织在一起 */ class Student(val name: String, val age: Int){ //主构造器会执行类定义中的所有语句 println("执行主构造器") try { println("读取文件") // throw new IOException("io exception") } catch { case e: NullPointerException => println("打印异常Exception : " + e) // case e: IOException => println("打印异常Exception : " + e) } finally { println("执行finally部分") } private var gender = "male" //用this关键字定义辅助构造器 def this(name: String, age: Int, gender: String){ //每个辅助构造器必须以主构造器或其他的辅助构造器的调用开始 this(name, age)//在执行之前必须调用这个方法 println("执行辅助构造器") this.gender = gender } } object Student{ def main(args: Array[String]) { val s = new Student("bee",15) } }
---------------------------------------------------------------------
//package cn.itcast.scala import scala.collection.mutable.ArrayBuffer /** * Created by ZX on 2015/11/14. */ object SingletonDemo { def main(args: Array[String]) { //单例对象,不需要new,用【类名.方法】调用对象中的方法 val session = SessionFactory.getSession() println(session) } } object SessionFactory{ //该部分相当于java中的静态块 var counts = 5 val sessions = new ArrayBuffer[Session]() while(counts > 0){ sessions += new Session counts -= 1 } //在object中的方法相当于java中的静态方法 def getSession(): Session ={ sessions.remove(0) } } class Session{ }
package cn.itcast.scala /** * Created by ZX on 2015/11/14. */ class Dog { val id = 1 private var name = "itcast" def printName(): Unit ={ //在Dog类中可以访问伴生对象Dog的私有属性 println(Dog.CONSTANT + name ) } } /** * 伴生对象 */ object Dog { //伴生对象中的私有属性 private val CONSTANT = "汪汪汪 : " def main(args: Array[String]) { val p = new Dog //访问私有的字段name p.name = "123" p.printName() } }
/** * Created by bee on 2018/3/26. */ class ApplyDemo { val name = "dong" } object ApplyDemo{ def apply(name:String):Unit={ println(name) println("123") } def apply():ApplyDemo={ println("456") new ApplyDemo } def main(args: Array[String]) { val a1 = ApplyDemo("xxx") println(a1)//返回为()空 val a2 = ApplyDemo() println(a2) //********************上面调用apply方法 val a3 = new ApplyDemo println(a3) } }
1.1. 继承
1.1.1. 扩展类
在Scala中扩展类的方式和Java一样都是使用extends关键字
1.1.2. 重写方法
在Scala中重写一个非抽象的方法必须使用override修饰符
1.1.3. 类型检查和转换
Scala |
Java |
obj.isInstanceOf[C] |
obj instanceof C |
obj.asInstanceOf[C] |
(C)obj |
classOf[C] |
C.class |
1.1.4. 超类的构造
package cn.itcast.scala
|
/** * Created by bee on 2018/3/26. */ trait Animal { //类似于接口 def run(): Unit ={ println("animal run") } def eat(): Unit ={ println("animal eat") } }
/** * Created by bee on 2018/3/26. */ abstract class Human { }
/** * Created by bee on 2018/3/26. */ class Chinese extends Human with Animal{ //继承人类,实现接口 override def eat(): Unit ={ println("chinese eat") } } object Chinese{ def main(args: Array[String]) { var c = new Chinese c.eat() c.run() } }
PS:和java基本上一致
1.1. 匹配字符串
package cn.itcast.cases import scala.util.Random object CaseDemo01 extends App{ val arr = Array("YoshizawaAkiho", "YuiHatano", "AoiSola") val name = arr(Random.nextInt(arr.length)) name match { case "YoshizawaAkiho" => println("吉泽老师...") case "YuiHatano" => println("波多老师...") case _ => println("真不知道你们在说什么...") } }
1.1. 匹配数据类型
package cn.itcast.cases import scala.util.Random class CaseDemo{ } object CaseDemo01 extends App{ //val v = if(x >= 5) 1 else if(x < 2) 2.0 else "hello" val arr = Array("hello", 1, 2.0, new CaseDemo) val v = arr(Random.nextInt(4)) println(v) v match { case x: Int => println("Int " + x) case y: Double if(y >= 0) => println("Double "+ y) case z: String => println("String " + z) case _ => throw new Exception("not match exception") } }
1.1. 匹配数组、元组
package cn.itcast.cases object CaseDemo01 extends App{ //数组匹配 val arr = Array(1, 3, 5) arr match { case Array(1, x, y) => println(x + " " + y) case Array(0) => println("only 0") case Array(0, _*) => println("0 ...") case _ => println("something else") } //列表匹配 val lst = List(3, -1) lst match { case 0 :: Nil => println("only 0") case x :: y :: Nil => println(s"x: $x y: $y") case 0 :: tail => println("0 ...") case _ => println("something else") } //元组匹配 val tup = (2, 3, 7) tup match { case (1, x, y) => println(s"1, $x , $y") case (_, z, 5) => println(z) case _ => println("else") } }
package cn.itcast.cases import scala.util.Random case class SubmitTask(id: String, name: String) case class HeartBeat(time: Long) case object CheckTimeOutTask object CaseDemo01 extends App{ val arr = Array(CheckTimeOutTask, HeartBeat(12333), SubmitTask("0001", "task-0001")) arr(Random.nextInt(arr.length)) match { case SubmitTask(id, name) => { println(s"$id, $name")//前面需要加上s, $id直接取id的值 } case HeartBeat(time) => { println(time) } case CheckTimeOutTask => { println("check") } } }
PS:存在就是Some 不存在就是None object CaseDemo01 { def main(args: Array[String]) { val map = Map("a" -> 1, "b" -> 2) val v = map.get("b") match { case Some(i) => i case None => 0 } println(v) //更好的方式 val v1 = map.getOrElse("c", 0) println(v1) } }
package cn.itcast.cases object CaseDemo01 { def func1: PartialFunction[String, Int] = {//第一个参数是输入,第二个参数是输出 case "one" => 1 case "two" => 2 case _ => -1 } def func2(num: String) : Int = num match { case "one" => 1 case "two" => 2 case _ => -1 } def main(args: Array[String]) { println(func1("one")) println(func2("one")) } }
输出:1 1
PS:
akka的实现是在scala的基础之上的
//package cn.itcast.actor //注意导包是scala.actors.Actor import scala.actors.Actor object MyActor1 extends Actor{ //重新act方法 def act(){ for(i <- 1 to 10){ println("actor-1 " + i) Thread.sleep(2000) } } } object MyActor2 extends Actor{ //重新act方法 def act(){ for(i <- 1 to 10){ println("actor-2 " + i) Thread.sleep(2000) } } } object CaseDemo01 extends App{ //启动Actor MyActor1.start() MyActor2.start() }
import scala.actors.Actor /** * Created by ZX on 2016/4/4. */ class MyActor extends Actor { override def act(): Unit = { while (true) { receive {//receive这里是一个偏函数 case "start" => { println("starting ...") Thread.sleep(5000) println("started") } case "stop" => { println("stopping ...") Thread.sleep(5000) println("stopped ...") } } } } } object CaseDemo01 { def main(args: Array[String]) { val actor = new MyActor actor.start() actor ! "start" //异步传输 actor ! "stop" println("消息发送完成!") } }
import scala.actors.Actor /** * Created by ZX on 2016/4/4. */ class YourActor extends Actor { override def act(): Unit = { loop { react { case "start" => { println("starting ...") Thread.sleep(5000) println("started") } case "stop" => { println("stopping ...") Thread.sleep(8000) println("stopped ...") } } } } } object YourActor { def main(args: Array[String]) { val actor = new YourActor actor.start() actor ! "start" actor ! "stop" println("消息发送完成!") } }
1.1. 第四个例子(结合case class发送消息)
import scala.actors.Actor class AppleActor extends Actor { def act(): Unit = { while (true) { receive { case "start" => println("starting ...") case SyncMsg(id, msg) => { println(id + ",sync " + msg) Thread.sleep(5000) sender ! ReplyMsg(3,"finished") } case AsyncMsg(id, msg) => { println(id + ",async " + msg) Thread.sleep(5000) } } } } } object AppleActor { def main(args: Array[String]) { val a = new AppleActor a.start() //异步消息 a ! AsyncMsg(1, "hello actor") println("异步消息发送完成") //同步消息 //val content = a.!?(1000, SyncMsg(2, "hello actor")) //println(content) val reply = a !! SyncMsg(2, "hello actor") println(reply.isSet) //println("123") val c = reply.apply() println(reply.isSet) println(c) } } case class SyncMsg(id : Int, msg: String) case class AsyncMsg(id : Int, msg: String) case class ReplyMsg(id : Int, msg: String)
PS:WordCount
有两个文件,多线程读取处理
import java.io.File import scala.actors.{Actor, Future} import scala.collection.mutable import scala.io.Source /** * Created by ZX on 2016/4/4. */ class Task extends Actor { override def act(): Unit = { loop { react { case SubmitTask(fileName) => { val lines = Source.fromFile(new File(fileName)).getLines().toList val result = lines.flatMap(_.split(" ")).map((_, 1)).groupBy(_._1).mapValues(_.length) //val result = lines.flatMap(_.split(" ")).map((_, 1)).groupBy(_._1).mapValues(_.foldLeft(0)(_ + _._2)) sender ! ResultTask(result) //把结果发给发送者 } case StopTask => {//就是一个信号,所有没有参数 exit() } } } } } object WorkCount { def main(args: Array[String]) { val files = Array("d://word.txt", "d://word.log") val replaySet = new mutable.HashSet[Future[Any]] val resultList = new mutable.ListBuffer[ResultTask] for(f <- files) { val t = new Task val replay = t.start() !! SubmitTask(f) //同步发送,返回一个reply回复 replaySet += replay } while(replaySet.size > 0){ val toCumpute = replaySet.filter(_.isSet) //是否能够执行计算 for(r <- toCumpute){ val result = r.apply() resultList += result.asInstanceOf[ResultTask] replaySet.remove(r) } Thread.sleep(100) } //list里面装了很多的map val finalResult = resultList.map(_.result).flatten.groupBy(_._1).mapValues(x => x.foldLeft(0)(_ + _._2)) println(finalResult) //Map(qq -> 1, bjj -> 1, 123 -> 1, xxx -> 1, hanhan -> 1, xyz -> 1, helo -> 2, bee -> 1, zz -> 1, wzq -> 1, tyou -> 1, hello -> 10, bb -> 1, thank -> 1) } } case class SubmitTask(fileName: String) case object StopTask case class ResultTask(result: Map[String, Int])
object FunDemo { def main(args: Array[String]) { def f2(x: Int) = x * 2 val f3 = (x: Int) => x * 3 val f4: (Int) => Int = { x => x * 4 } val f4a: (Int) => Int = _ * 4 val f5 = (_: Int) * 5 val list = List(1, 2, 3, 4, 5) var new_list: List[Int] = null //第一种:最直观的方式 (Int) => Int //new_list = list.map((x: Int) => x * 3) //第二种:由于map方法知道你会传入一个类型为(Int) => Int的函数,你可以简写 //new_list = list.map((x) => x * 3) //第三种:对于只有一个参数的函数,你可以省去参数外围的() //new_list = list.map(x => x * 3) //第四种:(终极方式)如果参数在=>右侧只出现一次,可以使用_ new_list = list.map(_ * 3) new_list.foreach(println(_)) var a = Array(1,2,3) a.map(_* 3) } }
package cn.itcast.impli import java.io.File import scala.io.Source //隐式的增强File类的方法 class RichFile(val from: File) { def read = Source.fromFile(from.getPath).mkString } object RichFile { //隐式转换方法 implicit def file2RichFile(from: File) = new RichFile(from) } object MainApp{ def main(args: Array[String]): Unit = { //导入隐式转换 import RichFile._ //import RichFile.file2RichFile println(new File("c://words.txt").read) } } package cn.itcast.scala import java.awt.GridLayout /** * Created by ZX on 2015/11/13. */ object ImplicitContext{ //implicit def girl2Ordered(g : Girl) = new Ordered[Girl]{ // override def compare(that: Girl): Int = if (g.faceValue > that.faceValue) 1 else -1 //} implicit object OrderingGirl extends Ordering[Girl]{ override def compare(x: Girl, y: Girl): Int = if (x.faceValue > y.faceValue) 1 else -1 } } class Girl(var name: String, var faceValue: Double){ override def toString: String = s"name : $name, faveValue : $faceValue" } //class MissRight[T <% Ordered[T]](f: T, s: T){ // def choose() = if(f > s) f else s //} //class MissRight[T](f: T, s: T){ // def choose()(implicit ord: T => Ordered[T]) = if (f > s) f else s //} class MissRight[T: Ordering](val f: T, val s: T){ def choose()(implicit ord: Ordering[T]) = if(ord.gt(f, s)) f else s } object MissRight { def main(args: Array[String]) { import ImplicitContext.OrderingGirl val g1 = new Girl("yuihatano", 99) val g2 = new Girl("jzmb", 98) val mr = new MissRight(g1, g2) val result = mr.choose() println(result) } }