fs2的多线程编程模式不但提供了无阻碍I/O(java nio)能力,更为并行运算提供了良好的编程工具。在进入并行运算讨论前我们先示范一下fs2 pipe2对象里的一些Stream合并功能。我们先设计两个帮助函数(helper)来跟踪运算及模拟运算环境:
def log[A](prompt: String): Pipe[Task,A,A] = _.evalMap {a =>
Task.delay { println(prompt + a); a}} //> log: [A](prompt: String)fs2.Pipe[fs2.Task,A,A]
Stream(1,2,3).through(log(">")).run.unsafeRun //> >1
//| >2
//| >3
implicit val strategy = Strategy.fromFixedDaemonPool(4)
//> strategy : fs2.Strategy = Strategy
implicit val scheduler = Scheduler.fromFixedDaemonPool(2)
//> scheduler : fs2.Scheduler = Scheduler(java.util.concurrent.ScheduledThreadPoolExecutor@16022d9d[Running, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 0])
def randomDelay[A](max: FiniteDuration): Pipe[Task, A, A] = _.evalMap { a => {
val delay: Task[Int] = Task.delay {
scala.util.Random.nextInt(max.toMillis.toInt)
}
delay.flatMap { d => Task.now(a).schedule(d.millis) }
}
} //> randomDelay: [A](max: scala.concurrent.duration.FiniteDuration)fs2.Pipe[fs2.Task,A,A]
Stream(1,2,3).through(randomDelay(1.second)).through(log("delayed>")).run.unsafeRun
//> delayed>1
//| delayed>2
//| delayed>3
Stream(1,2,3).through(log("befor delay>"))
.through(randomDelay(1.second))
.through(log("after delay>")).run.unsafeRun
//> befor delay>1
//| after delay>1
//| befor delay>2
//| after delay>2
//| befor delay>3
//| after delay>3
下面我们来看看pipe2对象里的合并函数interleave:
val sa = Stream(1,2,3).through(randomDelay(1.second)).through(log("A>"))
//> sa : fs2.Stream[fs2.Task,Int] = Segment(Emit(Chunk(1, 2, 3))).flatMap(<function1>).flatMap(<function1>)
val sb = Stream(1,2,3).through(randomDelay(1.second)).through(log("B>"))
//> sb : fs2.Stream[fs2.Task,Int] = Segment(Emit(Chunk(1, 2, 3))).flatMap(<function1>).flatMap(<function1>)
(sa interleave sb).through(log("AB")).run.unsafeRun
//> A>1
//| B>1
//| AB>1
//| AB>1
//| A>2
//| B>2
//| AB>2
//| AB>2
//| A>3
//| B>3
//| AB>3
//| AB>3
(sa merge sb).through(log("AB>")).run.unsafeRun //> B>1
//| AB>1
//| B>2
//| AB>2
//| B>3
//| AB>3
//| A>1
//| AB>1
//| A>2
//| AB>2
//| A>3
//| AB>3
(sa either sb).through(log("AB>")).run.unsafeRun //> A>1
//| AB>Left(1)
//| B>1
//| AB>Right(1)
//| A>2
//| AB>Left(2)
//| B>2
//| AB>Right(2)
//| B>3
//| AB>Right(3)
//| A>3
//| AB>Left(3)
val sc = Stream.range(1,10).through(randomDelay(1.second)).through(log("C>"))
//> sc : fs2.Stream[fs2.Task,Int] = Segment(Emit(Chunk(()))).flatMap(<function1>).flatMap(<function1>).flatMap(<function1>)
((sa merge sb) merge sc).through(log("ABC>")).run.unsafeRun
//> B>1
//| ABC>1
//| C>1
//| ABC>1
//| A>1
//| ABC>1
//| B>2
//| ABC>2
//| A>2
//| ABC>2
//| B>3
//| ABC>3
//| C>2
//| ABC>2
//| A>3
//| ABC>3
//| C>3
//| ABC>3
//| C>4
//| ABC>4
//| C>5
//| ABC>5
//| C>6
//| ABC>6
//| C>7
//| ABC>7
//| C>8
//| ABC>8
//| C>9
//| ABC>9
Stream[Task,Stream[Task,A]]
val streams:Stream[Task,Stream[Task,Int]] = Stream(sa,sb,sc)
//> streams : fs2.Stream[fs2.Task,fs2.Stream[fs2.Task,Int]] = Segment(Emit(Chunk(Segment(Emit(Chunk(1, 2, 3))).flatMap(<function1>).flatMap(<function1>),Segment(Emit(Chunk(1, 2, 3))).flatMap(<function1>).flatMap(<function1>), S
egment(Emit(Chunk(()))).flatMap(<function1>).flatMap(<function1>).flatMap(<function1>))))
def join[F[_],O](maxOpen: Int)(outer: Stream[F,Stream[F,O]])(implicit F: Async[F]): Stream[F,O] = {...}
val ms = concurrent.join(3)(streams) //> ms : fs2.Stream[fs2.Task,Int] = attemptEval(Task).flatMap(<function1>).flatMap(<function1>)
ms.through(log("ABC>")).run.unsafeRun //> C>1
//| ABC>1
//| A>1
//| ABC>1
//| C>2
//| ABC>2
//| B>1
//| ABC>1
//| C>3
//| ABC>3
//| A>2
//| ABC>2
//| B>2
//| ABC>2
//| C>4
//| ABC>4
//| A>3
//| ABC>3
//| B>3
//| ABC>3
//| C>5
//| ABC>5
//| C>6
//| ABC>6
//| C>7
//| ABC>7
//| C>8
//| ABC>8
//| C>9
//| ABC>9
val rangedStreams = Stream.range(0,5).map {id =>
Stream.range(1,5).through(randomDelay(1.second)).through(log((('A'+id).toChar).toString +">")) }
//> rangedStreams : fs2.Stream[Nothing,fs2.Stream[fs2.Task,Int]] = Segment(Emit(Chunk(()))).flatMap(<function1>).mapChunks(<function1>)
concurrent.join(3)(rangedStreams).run.unsafeRun //> B>1
//| A>1
//| C>1
//| B>2
//| C>2
//| A>2
//| B>3
//| C>3
//| C>4
//| D>1
//| A>3
//| A>4
//| B>4
//| E>1
//| E>2
//| E>3
//| D>2
//| D>3
//| E>4
//| D>4
当我们的程序需要与外界程序交互时,可能会以下面的几种形式进行:
1、产生副作用的运算是同步运行的。这种情况最容易处理,因为直接可以获取结果
2、产生副作用的运算是异步的:通过调用一次callback函数来提供运算结果
3、产生副作用的运算是异步的,但结果必须通过多次调用callback函数来分批提供
下面我们就一种一种情况来分析:
1、同步运算最容易处理:我们只需要把运算包嵌在Stream.eval里就行了:
def destroyUniverse: Unit = println("BOOOOM!!!") //> destroyUniverse: => Unit
val s = Stream.eval_(Task.delay(destroyUniverse)) ++ Stream("...move on")
//> s : fs2.Stream[fs2.Task,String] = append(attemptEval(Task).flatMap(<function1>).flatMap(<function1>), Segment(Emit(Chunk(()))).flatMap(<function1>))
s.runLog.unsafeRun //> BOOOOM!!!
//| res8: Vector[String] = Vector(...move on)
trait Async[F[_]] extends Effect[F] { self =>
/**
Create an `F[A]` from an asynchronous computation, which takes the form
of a function with which we can register a callback. This can be used
to translate from a callback-based API to a straightforward monadic
version.
*/
def async[A](register: (Either[Throwable,A] => Unit) => F[Unit]): F[A] =
bind(ref[A]) { ref =>
bind(register { e => runSet(ref)(e) }) { _ => get(ref) }}
...
我们用一个实际的例子来做示范,假设我们有一个callback函数readBytes:
trait Connection {
def readBytes(onSuccess: Array[Byte] => Unit, onFailure: Throwable => Unit): Unit
这个Connection就是一个交互界面(interface)。假设它是这样实现实例化的:
val conn = new Connection {
def readBytes(onSuccess: Array[Byte] => Unit, onFailure: Throwable => Unit): Unit = {
Thread.sleep(1000)
onSuccess(Array(1,2,3,4,5))
}
} //> conn : demo.ws.fs2Concurrent.connection = demo.ws.fs2Concurrent$$anonfun$main$1$$anon$1@4c40b76e
我们可以用async登记(register)这个callback函数,把它变成纯代码可组合的(monadic)组件Task[Array[Byte]]:
val bytes = T.async[Array[Byte]] { (cb: Either[Throwable,Array[Byte]] => Unit) => {
Task.delay { conn.readBytes (
ready => cb(Right(ready)),
fail => cb(Left(fail))
) }
}} //> bytes : fs2.Task[Array[Byte]] = Task
这样我们才能用Stream.eval来运算bytes:
Stream.eval(bytes).map(_.toList).runLog.unsafeRun //> res9: Vector[List[Byte]] = Vector(List(1, 2, 3, 4, 5))
import fs2.async
import fs2.util.Async
type Row = List[String]
// defined type alias Row
trait CSVHandle {
def withRows(cb: Either[Throwable,Row] => Unit): Unit
}
// defined trait CSVHandle
def rows[F[_]](h: CSVHandle)(implicit F: Async[F]): Stream[F,Row] =
for {
q <- Stream.eval(async.unboundedQueue[F,Either[Throwable,Row]])
_ <- Stream.suspend { h.withRows { e => F.unsafeRunAsync(q.enqueue1(e))(_ => ()) }; Stream.emit(()) }
row <- q.dequeue through pipe.rethrow
} yield row
// rows: [F[_]](h: CSVHandle)(implicit F: fs2.util.Async[F])fs2.Stream[F,Row]
/**
* Asynchronous queue interface. Operations are all nonblocking in their
* implementations, but may be 'semantically' blocking. For instance,
* a queue may have a bound on its size, in which case enqueuing may
* block until there is an offsetting dequeue.
*/
trait Queue[F[_],A] {
/**
* Enqueues one element in this `Queue`.
* If the queue is `full` this waits until queue is empty.
*
* This completes after `a` has been successfully enqueued to this `Queue`
*/
def enqueue1(a: A): F[Unit]
/** Repeatedly call `dequeue1` forever. */
def dequeue: Stream[F, A] = Stream.repeatEval(dequeue1)
/** Dequeue one `A` from this queue. Completes once one is ready. */
def dequeue1: F[A]
...
fs2提供了signal,queue,semaphore等数据类型。下面是一些使用示范:async.signal
Stream.eval(async.signalOf[Task,Int](0)).flatMap {s =>
val monitor: Stream[Task,Nothing] =
s.discrete.through(log("s updated>")).drain
val data: Stream[Task,Int] =
Stream.range(10,16).through(randomDelay(1.second))
val writer: Stream[Task,Unit] =
data.evalMap {d => s.set(d)}
monitor merge writer
}.run.unsafeRun //> s updated>0
//| s updated>10
//| s updated>11
//| s updated>12
//| s updated>13
//| s updated>14
//| s updated>15
Stream.eval(async.boundedQueue[Task,Int](5)).flatMap {q =>
val monitor: Stream[Task,Nothing] =
q.dequeue.through(log("dequeued>")).drain
val data: Stream[Task,Int] =
Stream.range(10,16).through(randomDelay(1.second))
val writer: Stream[Task,Unit] =
data.to(q.enqueue)
monitor mergeHaltBoth writer
}.run.unsafeRun //> dequeued>10
//| dequeued>11
//| dequeued>12
//| dequeued>13
//| dequeued>14
//| dequeued>15
time.awakeEvery[Task](1.second)
.through(log("time:"))
.take(5).run.unsafeRun //> time:1002983266 nanoseconds
//| time:2005972864 nanoseconds
//| time:3004831159 nanoseconds
//| time:4002104307 nanoseconds
//| time:5005091850 nanoseconds
val tick = time.awakeEvery[Task](1.second).through(log("time:"))
//> tick : fs2.Stream[fs2.Task,scala.concurrent.duration.FiniteDuration] = Segment(Emit(Chunk(()))).flatMap(<function1>).flatMap(<function1>).flatMap(<function1>)
tick.run.unsafeRunFor(5.seconds) //> time:1005685270 nanoseconds
//| time:2004331473 nanoseconds
//| time:3005046945 nanoseconds
//| time:4002795227 nanoseconds
//| time:5002807816 nanoseconds
//| java.util.concurrent.TimeoutException
val tick = time.awakeEvery[Task](1.second).through(log("time:"))
//> tick : fs2.Stream[fs2.Task,scala.concurrent.duration.FiniteDuration] = Seg
ment(Emit(Chunk(()))).flatMap(<function1>).flatMap(<function1>).flatMap(<function1>)
tick.interruptWhen(Stream.eval(Task.schedule(true,5.seconds)))
.run.unsafeRun //> time:1004963839 nanoseconds
//| time:2005325025 nanoseconds
//| time:3005238921 nanoseconds
//| time:4004240985 nanoseconds
//| time:5001334732 nanoseconds
//| time:6003586673 nanoseconds
//| time:7004728267 nanoseconds
//| time:8004333608 nanoseconds
//| time:9003907670 nanoseconds
//| time:10002624970 nanoseconds
(time.sleep[Task](5.seconds) ++ Stream.emit(true)).runLog.unsafeRun
//> res14: Vector[Boolean] = Vector(true)
tick.interruptWhen(time.sleep[Task](5.seconds) ++ Stream.emit(true))
.run.unsafeRun //> time:1002078506 nanoseconds
//| time:2005144318 nanoseconds
//| time:3004049135 nanoseconds
//| time:4002963861 nanoseconds
//| time:5000088103 nanoseconds