本文介绍了如何使用C#中的Monad设计模式简化并行任务处理，特别是当需要同时调用多个微服务时。通过创建自定义的`SelectMany`重载，可以构建一个链式操作，使得异步操作更加简洁和可靠，避免了使用`Task.WhenAll`可能导致的繁琐代码和潜在错误。
c#并行队列任务
No doubts that async/await pattern has significantly simplified working with asynchronous operations in C#. However, this simplification relates only to the situation when asynchronous operations are executed consequently. If we need to execute several asynchronous operations simultaneously (e.g. we need to call several micro-services) then we do not have many built-in capabilities and most probably Task.WhenAll will be used:
毫无疑问， async/await模式已大大简化了C＃中异步操作的工作。 但是，这种简化仅涉及随后执行异步操作的情况。 如果我们需要同时执行多个异步操作(例如，我们需要调用多个微服务)，则我们没有很多内置功能，很可能将使用Task.WhenAll ：
Task<SomeType1> someAsyncOp1 = SomeAsyncOperation1();
Task<SomeType2> someAsyncOp2 = SomeAsyncOperation2();
Task<SomeType3> someAsyncOp3 = SomeAsyncOperation3();
Task<SomeType4> someAsyncOp4 = SomeAsyncOperation4();
await Task.WhenAll(someAsyncOp1, someAsyncOp2, someAsyncOp4);
var result = new SomeContainer(
someAsyncOp1.Result,someAsyncOp2.Result,someAsyncOp3.Result, someAsyncOp4.Result);This is a working solution, but it is quite verbose and not very reliable (you can forget to add a new task to “WhenAll”). I would prefer something like that instead:
这是一个可行的解决方案，但是它很冗长且不够可靠(您可以忘记在“ WhenAll”中添加新任务)。 我更喜欢这样的东西：
var result = await
from r1 in SomeAsyncOperation1()
from r2 in SomeAsyncOperation2()
from r3 in SomeAsyncOperation3()
from r4 in SomeAsyncOperation4()
select new SomeContainer(r1, r2, r3, r4);Further I will tell you what is necessary for this construction to work...
此外，我将告诉您该构造起作用的必要条件...
First, we should remember that C# query syntax is just a syntax sugar over method call chains and C# preprocessor will convert the previous statement into a cain of SelectMany calls:
首先，我们应该记住，C＃查询语法只是方法调用链上的语法糖，C＃预处理程序会将先前的语句转换为SelectMany调用的SelectMany ：
SomeAsyncOperation1()/*Task<T1>*/
.SelectMany(
(r1/*T1*/) => SomeAsyncOperation2()/*Task<T2>*/,
(r1/*T1*/, r2/*T2*/) => new {r1, r2}/*Anon type 1*/)
.SelectMany(
(t/*Anon type 1*/) => SomeAsyncOperation3()/*Task<T3>*/,
(t/*Anon type 1*/, r3/*T3*/) => new {t, r3}/*Anon type 2*/)
.SelectMany(
(t/*Anon type 2*/) => SomeAsyncOperation4()/*Task<T4>*/,
(t/*Anon type 2*/, r4/*T4*/) => new SomeContainer(t.t.r1, t.t.r2, t.r3, r4));Скрытыйтекст (Скрытый текст)
By default C# Compiler will complain about this code since SelectMany extension method is defined only for IEnumerable interface, but nothing prevents us to create our own overloads of SelectMany to make the code compilable.
默认情况下，由于SelectMany扩展方法仅为IEnumerable接口定义，因此C＃编译器将抱怨此代码，但是没有什么可以阻止我们创建自己的SelectMany重载以使代码可编译。
Each SelectMany function in the chain has two arguments*:
链中的每个SelectMany函数都有两个参数*：
the first argument is a link to a function which returns a next asynchronous operation
第一个参数是指向函数的链接，该函数返回下一个异步操作
(t/*Anon type 1*/) => SomeAsyncOperation3(),/*Task<T2>*/
(t/*Anon type 1*/) => SomeAsyncOperation3(),/*Task<T2>*/
the second argument is a link to a function that combines results of previous asynchronous operations with a result of the operation returned by the function which is passed as the first argument.
第二个参数是指向函数的链接，该函数将先前异步操作的结果与作为第一个参数传递的函数返回的操作的结果进行组合。
(t/*Anon type 1*/, r3/*T3*/) => new {t, r3}/*Anon type 2*/)
(t/*Anon type 1*/, r3/*T3*/) => new {t, r3}/*Anon type 2*/)
We can call the first functions to get a list of tasks which will be used in Task.WhenAll and then call the second mapping functions to build a result.
我们可以调用第一个函数以获取将在Task.WhenAll使用的任务列表，然后调用第二个映射函数以构建结果。
To get the task list and the list of mapping functions our SelectMany overload needs to return some objects which will contain links to a task and a mapping function. In addition to that, SelectMany receives a links to a previous object as this argument (in case if SelectMany is an extension method) — we also need this link to build a linked list which will contain all required data.
为了获得任务列表和映射函数列表，我们的SelectMany重载需要返回一些对象，这些对象将包含任务和映射函数的链接。 除此之外， SelectMany还会收到一个指向先前对象的链接作为this参数(如果SelectMany是扩展方法)—我们还需要此链接来构建一个包含所有必需数据的链接列表。
static class TaskAllExtensions
{
public static ITaskAccumulator<TRes> SelectMany<TCur, TNext, TRes>(
this ITaskAccumulator<TCur> source,
Func<TCur, Task<TNext>> getNextTaskFunc,
Func<TCur, TNext, TRes> mapperFunc)
=>
new TaskAccumulator<TCur, TNext, TRes>(
prev: source,
currentTask: getNextTaskFunc(default(TCur)),
mapper: mapperFunc);
}
class TaskAccumulator<TPrev, TCur, TRes> : ITaskAccumulator<TRes>
{
public readonly ITaskAccumulator<TPrev> Prev;
public readonly Task<TCur> CurrentTask;
public readonly Func<TPrev, TCur, TRes> Mapper;
...
}
初始蓄能器 (The initial accumulator)
The initial accumulator differs from the subsequent ones since It cannot have a link to a previous accumulator, but it should have a link to the first task:
初始累加器与后续累加器有所不同，因为它无法链接到先前的累加器，但是应该具有到第一个任务的链接：
static class TaskAllExtensions
{
...
public static ITaskAccumulator<TRes> SelectMany<TCur, TNext, TRes>(
this Task<TCur> source,
Func<TCur, Task<TNext>> getNextTaskFunc,
Func<TCur, TNext, TRes> mapperFunc)
=>
new TaskAccumulatorInitial<TCur, TNext, TRes>(
task1: source,
task2: getNextTaskFunc(default(TCur)),
mapper: mapperFunc);
...
}
class TaskAccumulatorInitial<TPrev, TCur, TRes> : ITaskAccumulator<TRes>
{
public readonly Task<TPrev> Task1;
public readonly Task<TCur> Task2;
public readonly Func<TPrev, TCur, TRes> Mapper;
...
}Now we can get the result by adding these methods:
现在我们可以通过添加以下方法获得结果：
class TaskAccumulator<TPrev, TCur, TRes> : ITaskAccumulator<TRes>
{
public async Task<TRes> Result()
{
await Task.WhenAll(this.Tasks);
return this.ResultSync();
}
internal IEnumerable<Task> Tasks
=> new Task[] { this.CurrentTask }.Concat(this.Prev.Tasks);
internal TRes ResultSync()
=> this.Mapper(this.Prev.ResultSync(), this.CurrentTask.Result);
...
public readonly ITaskAccumulator<TPrev> Prev;
public readonly Task<TCur> CurrentTask;
public readonly Func<TPrev, TCur, TRes> Mapper;
}Tasksproperty returns all tasks from the entire linked list.Tasks属性返回整个链接列表中的所有任务。ResultSync()recursively applies mapper functions to the task results (all the tasks are supposed to be already resolved).ResultSync()递归将映射器函数应用于任务结果(假定所有任务都已解决)。Result()resolves all tasks (throughawait Task.WhenAll(Tasks)) and returns result ofResultSync()Result()解决所有任务(通过await Task.WhenAll(Tasks))，并返回ResultSync()结果
Also, we can add a simple extension method to make await work with ITaskAccumulator:
另外，我们可以添加一个简单的扩展方法来使ITaskAccumulator await ITaskAccumulator ：
static class TaskAllExtensions
{
...
public static TaskAwaiter<T> GetAwaiter<T>(this ITaskAccumulator<T> source)
=> source.Result().GetAwaiter();
}Now the code is working:
现在代码可以正常工作了：
var result = await
from r1 in SomeAsyncOperation1()
from r2 in SomeAsyncOperation2()
from r3 in SomeAsyncOperation3()
from r4 in SomeAsyncOperation4()
select new SomeContainer(r1, r2, r3, r4);However, there is an issue here — C# allows using an intermediate result as an argument for further operations. For example:
但是，这里存在一个问题-C＃允许使用中间结果作为进一步操作的参数。 例如：
from r2 in SomeAsyncOperation2()
from r3 in SomeAsyncOperation3(r2)Such code will lead to "Null Reference Exception" since r2 is not yet resolved at the moment when SomeAsyncOperation3 is called:
这样的代码将导致“空引用异常”，因为在调用SomeAsyncOperation3时尚未解析r2：
...
task2:getNextTaskFunc(default(TCur)),
...since all the tasks are run in parallel.
因为所有任务都是并行运行的。
Unfortunately, I do not see a solution for that problem in the current state of C# language, but we can mitigate it by dividing tasks in two groups:
不幸的是，在C＃语言的当前状态下，我看不到该问题的解决方案，但是我们可以通过将任务分为两组来减轻它：
- Tasks that are executed in parallel 并行执行的任务
- Tasks that are executed consequently (which can use all previous results). 因此执行的任务(可以使用以前的所有结果)。
To do that let's introduce the two simple wrappers over a task:
为此，让我们为一个任务介绍两个简单的包装器：
public struct ParallelTaskWrapper<T>
{
public readonly Task<T> Task;
internal ParallelTaskWrapper(Task<T> task) => this.Task = task;
}
public struct SequentialTaskWrapper<T>
{
public readonly Task<T> Task;
public SequentialTaskWrapper(Task<T> task) => this.Task = task;
}帮手 (Helpers)
public static ParallelTaskWrapper<T> AsParallel<T>(this Task<T> task)
{
return new ParallelTaskWrapper<T>(task);
}
public static SequentialTaskWrapper<T> AsSequential<T>(this Task<T> task)
{
return new SequentialTaskWrapper<T>(task);
}The only purpose of the tasks is to specify what '''SelectMany''' overloads should be used:
任务的唯一目的是指定应使用的'''SelectMany'''重载：
public static ITaskAccumulator<TRes> SelectMany<TCur, TNext, TRes>(
this ITaskAccumulator<TCur> source,
Func<TCur, ParallelTaskWrapper<TNext>> exec,
Func<TCur, TNext, TRes> mapper)
...and (it is a new overload):
和(这是一个新的重载)：
public static ITaskAccumulator<TRes> SelectMany<TCur, TNext, TRes>(
this ITaskAccumulator<TCur> source,
Func<TCur, SequentialTaskWrapper<TNext>> exec,
Func<TCur, TNext, TRes> mapper)
{
return new SingleTask<TRes>(BuildTask());
async Task<TRes> BuildTask()
{
var arg1 = await source.Result();
var arg2 = await exec(arg1).Task;
return mapper(arg1, arg2);
}
}单任务 (SingleTask)
internal class SingleTask<T> : ITaskAccumulator<T>
{
private readonly Task<T> _task;
private readonly Task[] _tasks;
public SingleTask(Task<T> task)
{
this._task = task;
this._tasks = new Task[] { task };
}
public Task<T> Result() => this._task;
public IEnumerable<Task> Tasks => this._tasks;
public T ResultSync() => this._task.Result;
}As you see all previous tasks are resolved trough var arg1/*Anon Type X*/ = await source.Result();, so they can be used to retrieve a next task and the code bellow will work properly:
如您所见，所有以前的任务都是通过var arg1/*Anon Type X*/ = await source.Result(); ，因此它们可用于检索下一个任务，并且下面的代码将正常工作：
var result = await
from r1 in SomeAsyncOperation1().AsParallel()
from r2 in SomeAsyncOperation2().AsParallel()
from r3 in SomeAsyncOperation3().AsParallel()
from r4 in SomeAsyncOperation4(r1, r2, r3).AsSequential()
from r5 in SomeAsyncOperation5().AsParallel()
select new SomeContainer(r1, r2, r3, r4, r5); 更新(摆脱Task.WhenAll ) (Update (Getting rid of Task.WhenAll))
We introduced the task accumulator to get a list of tasks be able to call Task.WhenAll over them. But do we really need it? Actually, we do not! The thing is that once we received a link to the task it is already started execution and all the task below are running in parallel (the code from the beginning):
我们引入了任务累加器，以获取能够调用Task.WhenAll的任务列表。 但是我们真的需要吗？ 实际上，我们不！ 关键是，一旦我们收到该任务的链接，它就已经开始执行，并且以下所有任务并行运行(从头开始的代码)：
Task<SomeType1> someAsyncOp1 = SomeAsyncOperation1();
Task<SomeType2> someAsyncOp2 = SomeAsyncOperation2();
Task<SomeType3> someAsyncOp3 = SomeAsyncOperation3();
Task<SomeType4> someAsyncOp4 = SomeAsyncOperation4();But instead of Task.WhenAll we can use several await-s:
但是我们可以使用几个await -s代替Task.WhenAll：
SomeType1 op1Result = await someAsyncOp1;
SomeType2 op2Result = await someAsyncOp2;
SomeType3 op3Result = await someAsyncOp3;
SomeType4 op4Result = await someAsyncOp4;await immediately returns a result if a task is already resolved or waits till an asynchronous operation is completed, so the code will take the same amount of time as if Task.WhenAll was used.
如果任务已经解决或等待异步操作完成，则await立即返回结果，因此代码将花费与使用Task.WhenAll相同的时间。
That fact allows us to significantly simplify the code and get rid of the task accumulator:
这个事实使我们可以大大简化代码并摆脱任务累加器：
static class TaskAllExtensions
{
public static ParallelTaskWrapper<TRes> SelectMany<TCur, TNext, TRes>(
this ParallelTaskWrapper<TCur> source,
Func<TCur, ParallelTaskWrapper<TNext>> exec,
Func<TCur, TNext, TRes> mapper)
{
async Task<TRes> GetResult()
{
var nextTask = exec(default(TCur));//<--Important!
return mapper(await source.Task, await nextTask);
}
return new ParallelTaskWrapper<TRes>(GetResult());
}
public static ParallelTaskWrapper<TRes> SelectMany<TCur, TNext, TRes>(
this ParallelTaskWrapper<TCur> source,
Func<TCur, SequentialTaskWrapper<TNext>> exec,
Func<TCur, TNext, TRes> mapper)
{
async Task<TRes> GetResult()
{
return mapper(await source, await exec(await source).Task);
}
return new ParallelTaskWrapper<TRes>(GetResult());
}
public static TaskAwaiter<T> GetAwaiter<T>(
this ParallelTaskWrapper<T> source)
=>
source.Task.GetAwaiter();
}That is it.
这就对了。
All the code can be found on GitHub
所有代码都可以在GitHub上找到
...
...
Developers who familiar with functional programing languages might notice that the approach described above resembles “Monad” design pattern. It is no surprise since C# query notation is a kind of equivalent of “do” notation in Haskell which, in turn, is a “syntax sugar” for working with monads. If you are not familiar what that design pattern yet then, I hope, this demonstration will encourage you to get familiar with monads and functional programming.
熟悉功能性编程语言的开发人员可能会注意到，上述方法类似于“ Monad”设计模式。 毫不奇怪，因为C＃查询符号在Haskell中与“ do”符号等效，而后者又是用于处理monad的“语法糖”。 我希望，如果您还不熟悉该设计模式，那将鼓励您熟悉monad和函数式编程。
c#并行队列任务
扫一扫
2385
被折叠的 条评论
为什么被折叠?
到【灌水乐园】发言
