【GO】Fan-in, Fan-out

Generator aka Iterator

Generator Pattern is used to generate a sequence of values which is used to produce some output. This pattern is widely used to introduce parallelism into loops. This allows the consumer of the data produced by the generator to run in parallel when the generator function is busy computing the next value.

package main

import "fmt"

// Generator func which produces data which might be computationally expensive.
func fib(n int) chan int {
    c := make(chan int)
    go func() {
	for i, j:= 0, 1; i < n ; i, j = i+j,i {
		c <- i
	 }
        close(c)
    }()
    return c
}

func main() {
    // fib returns the fibonacci numbers lesser than 1000
    for i := range fib(1000) {
	    // Consumer which consumes the data produced by the generator, which further does some extra computations
		v := i*i
        fmt.Println(v)
    }
}

Generators in Go are implemented with goroutines. The fib function passes the Fibonacci number with the help of channels, which is then consumed in the loop to generate output. The generator and the consumer can work concurrently (maybe in parallel) as the logic involved in both are different.

Fan-in, Fan-out

Fan-in Fan-out is a way of Multiplexing and Demultiplexing in golang.

Fan-in refers to processing multiple input data and combining into a single entity.

Fan-out is the exact opposite, dividing the data into multiple smaller chunks, distributing the work amongst a group of workers to parallelize CPU use and I/O.

package faninout

import (
	"fmt"
)

func main() {
	randomNumbers := []int{13, 44, 56, 99, 9, 45, 67, 90, 78, 23}
	// generate the common channel with inputs
	inputChan := generatePipeline(randomNumbers)

	// Fan-out to 2 Go-routine
	c1 :=squareNumber(inputChan)
	c2 :=squareNumber(inputChan)

	// Fan-in the resulting squared numbers
	c := fanIn(c1, c2)
	sum := 0

	// Do the summation
	for i := 0; i < len(randomNumbers); i++ {
		sum += <-c
	}
	fmt.Printf("Total Sum of Squares: %d", sum)
}

func generatePipeline(numbers []int) <-chan int {
	out := make(chan int)
	go func() {
		for _, n := range numbers {
			out <- n
		}
		close(out)
	}()
	return out
}

func squareNumber(in <-chan int) <-chan int {
	out := make(chan int)
	go func() {
		for n := range in {
			out <- n * n
		}
		close(out)
	}()
	return out
}


func fanIn(input1, input2 <-chan int) <-chan int {
	c := make(chan int)
	go func() {
		for {
			select {
			case s := <-input1:  c <- s
			case s := <-input2:  c <- s
			}
		}
	}()
	return c
}

Each instance of the squareNumber function reads from the same input channel until that channel is closed. The fanin function can read from multiple inputs channels and proceed until all are closed by multiplexing the input channels onto a single channel that’s closed when all the inputs are closed.

传递channel的channel

One of the most important properties of Go is that a channel is a first-class value that can be allocated and passed around like any other. A common use of this property is to implement safe, parallel demultiplexing.

In the example in the previous section, handle was an idealized handler for a request but we didn’t define the type it was handling. If that type includes a channel on which to reply, each client can provide its own path for the answer. Here’s a schematic definition of type Request.

在上一节中，我们主goroutine通过channel将请求传递给工作goroutine。同样，我们也可以通过channel将处理结果返回给主goroutine。

type Request struct {
    args        []int
    f           func([]int) int
    resultChan  chan int
}

The client provides a function and its arguments, as well as a channel inside the request object on which to receive the answer.

func sum(a []int) (s int) {
    for _, v := range a {
        s += v
    }
    return
}

request := &Request{[]int{3, 4, 5}, sum, make(chan int)}

// Send request
clientRequests <- request

// Wait for response.
fmt.Printf("answer: %d\n", <-request.resultChan)

On the server side, the handler function is the only thing that changes.

func handle(queue chan *Request) {
    for req := range queue {
        req.resultChan <- req.f(req.args)
    }
}

There’s clearly a lot more to do to make it realistic, but this code is a framework for a rate-limited, parallel, non-blocking RPC system, and there’s not a mutex in sight.

Daisy-Chain/Pipeline

一种沿总线传输信号的方法，其中设备串联，而信号则从一台设备传向下一台设备。菊花链连接方法可根据设备在总线上的电气地位分配其优先级。