Framework and Key Technologies of Human-machine Hybrid-augmented Intelligence System for Large-scale-CSDN博客

本文链接：https://blog.csdn.net/qq_42161009/article/details/144755890

Shixiong Fan1, Jianbo Guo1, Shicong Ma1, Lixin Li1, Guozheng Wang2, Haotian Xu1, Jin Yang3, Zening Zhao1

Show Author Information

Abstract

With integration of large-scale renewable energy, new controllable devices, and required reinforcement of power grids, modern power systems have typical characteristics such as uncertainty, vulnerability and openness, which makes operation and control of power grids face severe security challenges. Application of artificial intelligence (AI) technologies represented by machine learning in power grid regulation is limited by reliability, interpretability and generalization ability of complex modeling. Mode of hybrid-augmented intelligence (HAI) based on human-machine collaboration (HMC) is a pivotal direction for future development of AI technology in this field. Based on characteristics of applications in power grid regulation, this paper discusses system architecture and key technologies of human-machine hybrid-augmented intelligence (HHI) system for large-scale power grid dispatching and control (PGDC). First, theory and application scenarios of HHI are introduced and analyzed; then physical and functional architectures of HHI system and human-machine collaborative regulation process are proposed. Key technologies are discussed to achieve a thorough integration of human/machine intelligence. Finally, state-of-the-art and future development of HHI in power grid regulation are summarized, aiming to efficiently improve the intelligent level of power grid regulation in a human-machine interactive and collaborative way.

Keywords

Artificial intelligencehuman-machine collaborative controlhuman-machine hybrid intelligenceoptimization and evolutionpower grid dispatching and control

Ⅰ Introduction

As the largest artificial system, power system has characteristics of openness, nonlinearity and complexity due to access of controllable devices and continuous changes in scale and structure of the power grid. Power grid dispatching and control (PGDC) system, being the control center of power systems, plays an important role in ensuring a safe and stable operation of the system for reliable power supply. At present, the PGDC information system is highly automated, integrates data collection, decision-making and control, as a typical human-machine interactive automation system. Generally, power grid operators effectively achieve safe and stable operations of power grid through status information with auxiliary decision-making obtained via a human-machine interaction, which is largely dependent on experiences from human experts.

Rapid development of renewable energy integration and commission of large-scale long-distance transmission lines, result in a constantly expanding scale of the power grid with complex operational modes. Thus, power system operators face challenges in analysis and control, with a current insufficiently developed system. It still relies on empirical analysis of operators due to its lack of learning and adaptive capabilities when handling complex tasks. Furthermore, drastic reconfiguration of power systems leads to complex operational characteristics, where the decision of operators can be affected by factors including personnel decision-making capabilities, psychological and physical conditions. It is possible operational accidents caused by human errors happen under complicated working conditions such as power grid operational failures and extreme weather conditions. Data-driven based artificial intelligence (AI) technologies have strong perception, prediction and decision-making capabilities, which can be effectively integrated into the PGDC system to improve the intelligence level of power grid operation and control, and reduce working pressure of operators.

Nowadays, AI technologies are widely used in Go game, image recognition, audio recognition and other scenarios, which trigger an upsurge of AI technology applications in various sectors. Researchers in electric power field have conducted preliminary exploratory research on application of new generation of AI. Advanced AI technologies such as deep neural network learning and reinforcement learning are applied to power systems, which has achieved valuable outcomes [1], [2]. Reference [3] focused on the field of PGDC and analyzed adaptability of AI in typical regulation application scenarios. The Convolutional Neural Networks (CNN) and Recurrent Neural Networks (RNN), as mainstream deep learning algorithms, are used in power system transient stability assessment [4], [5], renewable energy and load forecasting [6], [7], power grid fault diagnosis and analysis [8], due to their strong feature extraction and mapping capabilities. In addition, reinforcement learning algorithms such as Q-learning, deep Q-network (DQN) and deep deterministic policy gradient (DDPG) are applied to power grid emergency control [9], load frequency control (LFC) [10], automatic generation control (AGC) [11] and automatic voltage control (AVC) [12], to solve problems of sequential decision-making. However, the above-mentioned references only aim at intelligence of certain functionalities in PGDC, categorized as dedicated end-to-end intelligence. Appropriate input data and models are selected based on human’s understanding of a certain task to realize the intelligence.

Introduction of AI technologies has improved the intelligence of the PGDC system to a certain extent, while it still has great limitations in handling dynamic and complex human-machine collaborative tasks. The PGDC system is a typical human-machine intelligence (HMI) system. The key issue it faces is how to integrate the intelligence of operators and machine to tackle challenges of uncertainty, vulnerability and openness through human-machine interaction and coordination. At present, human-machine hybrid-augmented intelligence (HHI) technology is applied in fields of online intelligent learning [13], medical and health care [14], and robot control [15]. In power system applications, reference [16] conducted preliminary research on enhanced intelligent analysis of power systems. It proposed a framework for enhanced intelligent analysis and investigated its technical difficulties, yet enhanced intelligence from human-machine collaborative mode is not considered. So far, there is no dedicated systematic research on HHI in the field of PGDC. Therefore, research on design of the HHI system framework for modern large-scale PGDC is indispensable. To realize this, it is necessary to fully exploit the respective potential advantages of both operators and control system to complete complex human-machine collaborative tasks.

To summarize, the main contributions of this paper are as follows.

1) The framework of HHI system is proposed for large-scale PGDC from physical and functional perspectives. It also establishes a data-knowledge collaborative interaction mechanism among power grid control system, power grid digital twin (PGDT) system, AI system, and system operator, aiming at solving issues of uncertainty, openness and vulnerability of large power grids faced by AI application in power grid regulation.

2) A human-machine cooperative dispatching and control method is proposed for real-time power grid regulation based on the above technical framework of HHI system. Furthermore, the key technologies involved with HAI are analyzed and discussed to achieve deep integration of human intelligence and machine intelligence., which provides a useful reference for subsequent large-scale application of HAI in the field of power grid regulation. To the best of the authors’ knowledge, this is the first paper to systematically analyze and design application of hybrid-augmented intelligence in power grid regulation. It is expected this work can unfold a new technical theme for application of AI in large-scale power grid regulation.

Ⅱ HHI Theory and Technology Development

Machine intelligence has the ability to perceive high-dimensional states and rapid decision-making by virtue of its super computing power, while human intelligence has the ability to summarize and logically deduce, especially has certain advantages in understanding the real-world environment, processing incomplete information, and complex temporal and spatial relationship reasoning. When machine intelligence is applied to areas with high requirements for safety and reliability, e.g. automatic driving, power systems, human supervision and interaction are required as an important link in the calculation loop of the intelligent system to avoid problems such as risky decision-making and out-of-control caused by limitations of machine intelligence. In 2017, the “New Generation Artificial Intelligence Development Plan” issued by the P.R.C. Government made a comprehensive deployment of a national AI development strategy, which placed the hybrid intelligence of HMC as one of the key directions of AI technology planning and deployment [17]. Academician Zheng Nanning et al. divides HAI into human-in-the-loop augmented intelligence with HMC and cognitive computing based augmented intelligence [18]. The fundamental framework and paradigm of human-machine collaborative HAI were proposed. It mainly analyzes confidence level of output results of the AI system to determine whether humans actively intervene to adjust system parameters, which can be used as a feedback loop to provide a reasonably correct solution. Reference [19] defines “human-machine hybrid intelligence” as a new form of intelligence, which is a cross-species and cross-domain next-generation scientific entity.

According to its stages of development, HMI systems can be classified into human-machine system, human-in-the-loop system, and human-out-of-the-loop system, reflecting different degrees of machine intelligence, and human-machine interaction and collaboration. HMC modes can be classified into human-dominated (machine-followed), human-machine cooperation, and machine-dominated (human-supervised). They represent the intelligent progressive extent of AI technologies in the system. A reasonable HMC mechanism ensures the HMI system can achieve the best efficiency and optimal decision-making results in solving complex tasks. In addition, it can achieve human-machine knowledge collaboration and optimization in an open evolving environment.

HHI technology is still in its infancy, and preliminary results have been achieved in application scenarios such as autonomous driving [20] and robot control [21]. An autopilot system is a highly comprehensive artificial intelligence system; hence it has become a hot application field of AI research in recent years. An autonomous driving system and driver constitute a typical human-in-the-loop hybrid-augmented intelligent system. They have strong complementary characteristics. References [22]–[25] analyzed the role and function of the driver in human-machine collaborative control from aspects of driver state monitoring, intention recognition and driver operational behavior modeling, also constructed corresponding models. Reference [26] defines a subset of the vehicle state space based on a description of the driver’s capabilities to maneuver the vehicle, and uses vehicle models and reachability analysis methods to achieve transition from automated to manual driving, with verification via an actual system. Robots are widely used in various fields of industry, interact and collaborate with humans to an expanded extent that form an HMI system. An exoskeleton robot obtains the wearer’s posture and related physical human-computer interaction information through sensors to understand wearers’ motion intention, and dynamically plans wearers’ personalized bionic motion trajectory, which further improves wearing comfort [27]. Reference [28] proposed a framework for collaborative assembly planning to realize a hierarchical human-machine collaborative control, which can not only generate nominal plans, but also react to and cope with unforeseen possible fault events by combining the capabilities of humans and robots in an optimal way.

In summary, HAI is to introduce human functions or human cognitive models into AI systems to establish a form of “enhanced hybrid intelligence”, which is a key development of human-machine system intelligence in the future.

Ⅲ Technical Framework of HHI System for Large-scale Power Grid Regulation

The PGDC system provides reliable technical support and guarantees in secure, stable, and economic operations of a power grid, as well as rapid recovery from faults [29]. The current system, as an extension of the eye, hand, and brain of the operator when performing regulation tasks, plays an auxiliary and augmented role in terms of perception, analysis, and decision-making for operators. Current HMC mode belongs to a primary level at which machines can enhance perception of operators. As the intelligence level of power grid control increases, the relationship between operators and control systems is not limited to reminders, warnings or mutual switching, but a complex dynamic interactive relationship of human-machine paralleled decision-making and control. Overall design goal of the PGDC system based on HAI is to promote interaction between operators and system at the data level to a bidirectional knowledge interaction, to realize HAI of human-in-the-loop. It can improve capabilities of situation awareness and operational decision-making of power grid and further reduce working intensity of operators.

This section consists of three parts: overall approach, physical deployment and technical architecture design of the HHI system for power grid regulation.

A Overall Approach

This paper divides power grid operation and control process into inner and outer control loops based on timescale and structure hierarchy of system control, namely automatic power grid protection control and human-machine joint decision-making respectively. The inner loop uses real-time operation information to realize automatic execution of protection and control of the power grid, which forms a fast closed loop with the power grid. In the above-mentioned calculation and execution process, participation of operators is limited. The inner loop largely relies on automation and intelligence of the system. The outer loop is an important link of using AI technologies where operators and control systems participate jointly. It mainly involves interaction and collaboration between humans and machines, which can achieve HHI. This paper combines requirements and characteristics of power grid regulation to build a technical framework of HHI system based on advanced AI technology, which fully exploits intelligence integration of operators and machines under the outer loop.

B The Deployment of HHI System for Power Grid Regulation

The physical architecture design of the hybrid-augmented intelligent system is shown in Fig. 1. It is comprised of AI intelligent system, PGDC system, PGDT system with inputs from operators. In order to meet requirements of power system security and protection, the system is deployed in different information security partitions according to its operational and functional characteristics.

View original image Download original image

Fig. 1. The physical architecture design of the hybrid-augmented intelligent system.

Currently, multi-source data such as grid model data, real-time operating data, and management data are collected to the dispatching and control cloud platform. Together with dispatching and control cloud infrastructure (CPU resources, storage resources, and network resources), it can provide an AI platform applied to the field of power grid regulation with data, computing power, algorithms and other services [30]. Thus, data-driven AI system and control cloud platform are deployed in the same information management area to facilitate interaction of data and information between systems and model learning and training. AI system includes hardware facilities, model algorithms, AI model bases, and knowledge bases. The management information area where the AI system is deployed is between production control area and Internet area shown in Fig. 1. For the AI system, the adopted open-source framework and external information can be used for safe cross-regional interaction with secure access area through the Internet. In addition, data interaction with the production control area needs to pass through a forward and backward physical isolation device, which can avoid potential safety hazards caused by open source information. The trained model base and constructed knowledge base in the AI system can be directly used for power grid status perception and decision-making, which needs to be mirrored and deployed in the production control area. The AI system utilizes a backward isolation device to update the model base and knowledge base from the management information area to the production control area, realizing management of the entire life cycle of AI models.

The PGDT system is a digital map of the power system [31], [32]. Related research has been carried out in recent years and is still under development [33], [34]. While how to build the PGDT system is beyond the scope of this paper. It can simulate the state of the power grid under different operating modes and generate data samples under abnormal operating modes to ensure balance of AI system training samples. Moreover, it can also be used as a virtual evolution environment for interacting with AI systems, which trains the AI model to generate optimal control strategies. The PGDT system, as an interactive environment where operators and AI systems can perform analysis and calculations, is not directly used for power grid control. Thus, the PGDT system and AI system are deployed in the same management information area, which facilitates interaction with the AI system and operators, realizing autonomous collaboration and optimization of hybrid HMI.

Operators use human-machine interfaces to interact and collaborate with the AI system, PGDC system and the PGDT system deployed in different network security zones, to enhance their ability to regulate the complex power grid.

C Technical Architecture Design of HHI System

To construct a HHI system for PGDC, it is required to clarify functions and interactive information of various modules that build the HMI system, which realizes bidirectional communication of data and information flows between the PGDC system, the PGDT system, the AI system, and operators. The technical framework of this system is shown in Fig. 2. This section introduces overall technical architecture design of HHI system for power grid regulation from aspects of system function composition, human-machine system interaction collaboration, and power grid regulation method based on HMC.

View original image Download original image

Fig. 2. The technical framework of the HHI system for PGDC.

1 The Function of the HHI System for PGDC

The HHI system is upgraded and superimposed on the architecture and function of the current PGDC system. The PGDT system and AI intelligent system are added to the intelligence system architecture. Specific function components of the system are as follows:

1) The PGDT system has the information of power grid topology, physical model, historical data, and real-time operational data. Based on the above information, various processes of the power grid at different time scales can be simulated and predicted, and fast calculation verification can also be achieved. In the HHI system, the PGDT system mainly undertakes offline and online analysis, calculation, and checking of power grid. Functions related to analysis, calculation, auxiliary decision-making in conventional power grid regulation can be integrated into the PGDT system.

The PGDT system can support analysis and calculation of power grid in multi-mode, including real-time mode, research mode, planning mode, training mode, and test mode. Power grid analysis and calculation in different modes are used to solve tasks in different scenarios in PGDC. In real-time mode, the PGDT system analyzes and calculates state of real-time power grid operation mode by using real-time data collection, offline data, to provide corresponding auxiliary decision-making, which can realize application functions of power grid steady-state analysis and online security and stability analysis. Research mode is for case analysis of power grid characteristics, power grid operational mode calculation, fault inversion, according to tasks assigned to PGDT system by operators, which can realize power grid operational mode calculation, generation of security control strategy and other application functions. Planning mode is used to predict the trend of renewable energy generation and load in future power grid, then use forecast information for research of power grid planning under different future power grid scenarios, to assist auxiliary decision-making calculation for existing safety hazards. It can realize application functions such as load forecasting, scheduling plan and security checking. In training mode, static and dynamic responses of the power system and regulatory environment identical to the actual control center can be simulated by the PGDT system, which can help operators be familiar with and grasp functions of the power system in a simulated dispatching environment. Autonomous training and anti-accident exercises can be carried out for the purpose of targeted training through knowledge level evaluation of operators, which can improve operators’ operational experience in different working conditions of the power grid. Test mode can support system operation and maintenance personnel to test and debug the network model and advanced application functions used in practical power grid.

2) AI intelligent system is the core subsystem of the HHI system, mainly comprised of AI platform and applications for power grid regulation perception and decision-making services. The bottom layer of the AI platform consists of hardware devices and grid data. Hardware devices include computing facilities (CPU, GPU, TPU, etc.), storage resources, and network devices. In terms of data structure, grid data includes structured, semi-structured and unstructured data, where structured data is stored in relational databases, and regulatory text data, pictures, videos and audios are semi-structured and unstructured data. Based on the above hardware devices and grid data, the AI platform can provide various functions, such as data preprocessing, model and algorithm, model training and evaluation. On this basis, the model database and knowledge database suitable for PGDC is constructed. Furthermore, in order to ensure accuracy and generalization ability of the AI model, an evaluation index is established to evaluate performance of the model to achieve updating and upgrading of the model. The AI platform provides basic support of data and algorithm for upper-level AI application of the PGDC.

The model base is a set of AI models built for power grid applications. The AI models with different model structures and weighted parameters are trained by the AI platform, which uses its computation hardware and regulatory training data samples through algorithms such as machine learning, deep learning, reinforcement learning. The above models are integrated into unified management of the model base, which provides model support for cross-information security partition model sharing and intelligent regulation applications.

The knowledge base is a collection of various kinds of knowledge constructed for regulation applications, such as power grid characteristics, causal knowledge, association knowledge. The knowledge can be used to construct knowledge subsets by first-order logical predicate, production rule, RDF (Resource Description Framework) triplet and a variety of different knowledge expressions, and then build expert systems and knowledge graph systems to apply to power grid regulation scenarios.

2 Human-machine System Interaction and Collaboration

Human-machine interaction and collaboration are the foundation to build HAI of power grid regulation, which involves human-machine and machine-machine interaction information and interaction mechanism. It can be realized that bidirectional fusion, cooperation, and joint evolution of human intelligence and machine intelligence in large-scale PGDC by establishing the mechanism of human-machine interaction and clarifying information of human-machine interaction. Content of human-machine and machine-machine interactions includes data and knowledge level of the interaction, scope of interaction includes system internal interaction and interaction with external environment. Detailed content and form of interactions are as follows.

The PGDC system uses a Supervisory Control and Data Acquisition (SCADA) system to collect data in real-time, such as steady-state, dynamic, or transient information of power grid operation, secondary equipment status information, and auxiliary information for detection. Real-time and historical data and model information of grid are provided to the digital PGDT system for offline and online computation and analysis. Data collection and advanced application information related to security control, AGC and AVC in PGDC system interact with operators through a human-machine display interface, which improves operators’ ability to perceive system status.

The PGDT system accepts task information provided by operators and AI system to perform tasks related to power grid analysis and calculations, such as unbalanced sample generation, thermal stability calculation, transient stability analysis, and other tasks. The PGDT system can adopt Robotic Process Automation (RPA) to automatically perform the afore-mentioned tasks. The process is to transform power grid data to knowledge. Results obtained after task execution will also be provided to operators and AI system in the form of data samples and knowledge to achieve human-machine knowledge collaboration and optimal evolution.

AI system, as an important part of HHI system, interacts with other systems with data and knowledge. The sources of these include internal and external data and knowledge related to applications. Internal data of the system is composed of structured/semi-structured data collected and recorded by the PGDC system and unstructured text data for existing power grid, such as stable operation regulations, dispatching operation rules, fault handling plans. External data is mainly non-grid information, such as weather information and social information from other systems and the Internet. Since the operational range of the power system is constrained within the range of typical operational modes, collected data cannot cover all operational modes and scenarios. Therefore, data-driven based AI system needs to tackle issues such as failures or small samples in abnormal scenarios through simulation data generated by the task of the PGDT system given by operators and AI systems. In this way, balance of power grid training data samples can be realized.

Model base and knowledge base of the AI system can be evaluated by established indicators (such as recall rate, accuracy, precision, etc.) or human-machine interaction to determine whether it needs to be updated and expanded. In addition, it can be combined with data obtained by results of a given human-machine task and external database and knowledge base, which can upgrade and strengthen the model and knowledge of the AI system to ensure its accuracy in the application process. Output of the AI system is mainly prediction results of the machine learning model and related knowledge established by the knowledge base, which are provided to operators and the execution section of PGDC system through human-machine interaction for power grid perception and decision-making process under the HMC.

Operator, as a service object and final value judge of the intelligent machine system, is arbiter and guardian of the security operation of the power grid. Interaction and cooperation between operators and machine intelligence run through state perception and decision-making process of the power grid. Operators obtain data and information related to power grid operation through a visual human-machine interaction interface of the PGDC system. These data and information are summarized and deduced into knowledge for power grid perception and decision-making through human brain induction and deduction. Physiological information and capability assessment of operators will be transmitted to the AI system as the basis for human-machine hybrid intelligent decision-making. In addition to traditional keyboard, mouse, and other input methods, interaction between operators and system can also be carried out through speech, gesture, eye movement. The above-mentioned advanced multi-channel human-machine interaction technology can realize state perception and intention understanding of operators, which improves human-machine cooperation ability in power grid regulation tasks and promotes level of human-machine integration.

3 Power Grid Regulation Method Based on HMC

This section constructs a human-machine collaborative regulation method for real-time power grid regulation based on the above technical framework of HHI system. Regulation tasks are reasonably allocated to operators and machines for execution, and perception and decision-making control of the power grid are realized through HAI. Fig. 3 shows the power grid regulation scheme based on HMC, which primarily includes task planning and decision execution process.

View original image Download original image

Fig. 3. Power grid regulation scheme flow chart based on HMC.

The process of regulation scheme based on HMC is divided into the following four steps:

Step 1: Current task is decomposed into multiple interconnected subtasks according to steps in task execution process. According to types and complexity of subtasks, decomposed subtasks are fairly assigned to operators and machines for joint execution. Based on modes of HMC, types of subtasks can be divided into subtasks executed by machine and HMC, respectively.

Step 2: For subtasks executed by machines, network models generated by training of algorithms such as machine learning are used to generate results which will be inputs to the next subtask. Results of machine execution tasks are used to evaluate practical performance of training model by establishing corresponding model evaluation indexes, which ensures accuracy of models in different power grid operation scenarios. When evaluation index results are higher than a specified threshold, results of the model output are input into the next subtask; instead, when evaluation index results are below the threshold, then it needs to decide whether model is retrained or human intervene is required, depending on urgency of the subtask.

Step 3: Subtask of human-machine cooperation is executed by adopting an appropriate human-machine cooperation mode, and human-machine hybrid decision obtained is input into the next subtask.

Step 4: Check whether current task is complete. If not, execute next subtask. The process will follow Step 2 or Step 3 according to types of subtasks. When it is completed, task processing flow based on HMC is finished.

Based on the above-mentioned regulation process, HHI of power grid regulation can be realized by HMC, which enhances accuracy of machine learning models and ensures significance of humans in the process of the HMC.

Ⅳ Key Technologies of Human-machine Hybrid Augmented Intelligence for PGDC

HHI in power grid regulation is inseparable from interaction and collaboration of two important carriers: operators and control systems. In this section, human-machine modeling and evaluation techniques for power grid regulation is first analyzed. Then other key technologies are detailed such as complex task allocation and planning, unbiased technology for regulation operating data, and human-machine collaborative intelligent decision-making and control.

A Human-machine Modeling and Evaluation Techniques for Power Grid Regulation

Modeling and evaluation of human-machine systems are important prerequisites to ensure a safe and efficient HMC. HMC needs to take into account respective characteristics of operators and machines, and further to realize hybrid intelligence enhancement of human-machine systems by fully exploiting intelligence of operators and machines.

Research on human intelligence measurement has been relatively well-developed [35], and its evaluation division method can be used as a reference for evaluation of machine intelligence. At present, research on modeling and quantitative evaluation of human-machine systems is focused on intelligent driving [36], [37]. By modeling and evaluating driver’s behavior and state (e.g. skill, fatigue and distraction), intelligent vehicles can realize collaborative control between system and driver. Intelligent closed-loop system of human-machine co-driving that integrates human-vehicle-environment is formed to meet driver’s subjective feelings, and also improve performance indicators of vehicle handling, safety, economy, and comfort.

The process of HMC in the PGDC does not fully consider operator factors and level of system intelligence. There is a lack of reasonable quantitative evaluation indexes of the HMI level, which needs to be defined. Human-machine modeling and evaluation of power grid regulation can be studied from aspects of constructing operator’s behavior model and capability evaluation model of human/machine. Fig. 4 shows block diagram of human-machine modeling and evaluation technology for power grid regulation.

View original image Download original image

Fig. 4. The block diagram of human-machine modeling and evaluation technology for power grid regulation.

The main task of human-machine modeling and evaluation in power grid regulation is to establish behavior and capability models of operators and machines, and conduct quantitative assessment, which can provide a basis for human-machine task allocation and hybrid decision-making. Because operator is an important participant in the HMC of power grid regulation, effective assessment of behavior and capability of operators can avoid limitations to a large extent, which arises from subjective factors and their ability level in the process of decision control.

Behavior modeling of operators focuses on differences in operating behavior and habits of different operators. It is necessary to build a quantitative model of operators’ behavior based on multi-source data such as historical operating behaviors, command language and eye movements to effectively characterize behaviors of operators.

Capability assessment modeling for operators is used to establish model and evaluate operators’ competency considering related factors such as emotional intelligence, skill level, current task-oriented risk factor, and tolerance of operators, which provides a reference for HMC in power grid regulation. Machine capability modeling focuses on evaluating system’s level of intelligence, ability to handle tasks, generalization capability, and transferability. It is used to measure performance of intelligent machines.

For application scenario of human-machine intelligent collaboration in PGDC, based on the above content, model of the behavior and capability for operators and model of capability for machines can accurately reflect characteristics of human and machines, which can further improve initiative and intelligence of machines in the HMC.

B Optimal Allocation and Planning of Complex Tasks

The main task of power grid regulation is to monitor real-time operation of a power system and determine system operational mode arrangement and security control measures according to current power grid status and combine with personal experience and cognitive level of operators, which ensures safe and stable operations of power system and reliable power supply. In the process of power grid operation, operators face different types of regulation tasks, which can be classified into simple and complex tasks. Simple tasks can be assigned to machines to complete independently, while complex tasks are usually decomposed into interconnected or parallel subtasks according to basic functional units and boundary conditions of human-machine division of labor, and then decomposed subtasks are assigned to operators and machines for execution.

Task allocation is a key step in the HHI system for power grid regulation. Quality of task allocation directly determines execution efficiency and cost of the system. In a complex regulation task environment, how to scientifically and rationally allocate each subtask to operator and intelligent machine, how to maximize human-machine hybrid intelligence under a mutual trust mechanism are critical challenges urgently needing to be addressed. In essence, human-machine task allocation is a mathematical optimization problem with multiple objectives. At present, research on task allocation of HMC is focused on a collaborative assembly operation of human and industrial robots [38]. Task allocation between human and machine mainly considers factors (such as operation comfort, resource utilization, task complexity) as the objective function. Optimization algorithms (such as integer mixed programming [39] and genetic algorithm [40]) are used to achieve a task allocation scheme of human-robot collaboration [41], which realizes a reasonable allocation of human-machine tasks and further improves efficiency of production system and product quality. However, there is limited literature and research on optimization of human-machine task allocation in the field of power grid regulation. Reference [42] considers human factors influencing accuracy of dispatching operation to ensure workload balance across shifts, but machine intelligence is not involved. The process of human-machine task collaboration in power grid regulation is dominated by operators, followed by machines for execution, which lacks optimization in allocation of human-machine regulation task. In order to allow full initiative and intelligence of machines and further reduce work intensity of operators, it is necessary to establish a mathematical model for assignment of HMC in power grid regulation. Fig. 5 shows the flow chart of power grid regulation task allocation.

View original image Download original image

Fig. 5. The flow chart of power grid regulation task allocation.

High operational safety requirement of the power grid proves urgent need of intelligence in human-machine system, for power grid regulation task allocation. Optimal allocation of human-machine tasks can be achieved through construction and quantitative evaluation analysis of human-machine capability model and behavior model of operators, which are mentioned in the previous section. Assuming the set of regulation tasks is 𝑇={𝑇𝑎𝑠𝑘1,𝑇𝑎𝑠𝑘2,⋯,𝑇𝑎𝑠𝑘𝑛}T={Task1,Task2,⋯,Taskn}, decision variables for task allocation 𝑇𝑎𝑠𝑘𝐴𝑖TaskiA is described as:

(1)

𝑇𝑎𝑠𝑘𝐴𝑖={10𝑇𝑎𝑠𝑘𝐴𝑖 is assigned to machine𝑇𝑎𝑠𝑘𝐴𝑖 is assigned to human-machineTaskiA={1TaskiA is assigned to machine0TaskiA is assigned to human-machine

Capability vector of operator to solve a task is defined as 𝐻={𝐻1,𝐻2,⋯,𝐻𝑛}H={H1,H2,⋯,Hn}, where element 𝐻𝑖Hi denotes capability of operator to solve task 𝑖i, such as risk factor, tolerance, etc. Capability vector of the intelligent machine to solve a task is defined as 𝑀={𝑀1,𝑀2,⋯,𝑀𝑛}M={M1,M2,⋯,Mn}, where element 𝑀𝑖Mi denotes capability of machine to solve task 𝑖i, such as level of intelligence, ability to handle task, etc. Mathematical model of HMC can be described by

(2)

min𝑓(𝑥)=∑𝑗=1𝑚𝛼𝑗∑𝑖=1𝑛(𝑔𝑗(𝑀𝑖)𝑇𝑎𝑠𝑘𝐴𝑖+𝑔″𝑗(𝐻𝑖,𝑀𝑖)(1−𝑇𝑎𝑠𝑘𝐴𝑖))s.t.𝜆⎯⎯≤𝐻𝑖≤𝜆⎯⎯⎯,∀𝑖∈𝑇𝛿⎯⎯≤𝑀𝑖≤𝛿⎯⎯⎯,∀𝑖∈𝑇0≤𝛼𝑗≤1,∀𝑖∈𝑇∑𝑗=1𝑚𝛼𝑗=1,∀𝑖∈𝑇⋯minf(x)=∑j=1mαj∑i=1n(gj(Mi)TaskiA+gj″(Hi,Mi)(1−TaskiA))s.t.λ_≤Hi≤λ¯,∀i∈Tδ_≤Mi≤δ¯,∀i∈T0≤αj≤1,∀i∈T∑j=1mαj=1,∀i∈T⋯

where 𝑓(𝑥)f(x) is objective function, 𝑔𝑗(𝑥)gj(x) and 𝑔″𝑗(𝑥)gj″(x) are sub-objective functions related to capability of machines and human-machine, 𝑇T is set of regulation tasks, 𝛼𝑗αj is weight values of sub-objectives, 𝜆⎯⎯λ_, 𝜆⎯⎯⎯λ¯ and 𝛿⎯⎯δ_, 𝛿⎯⎯⎯δ¯ are thresholds of capability value for humans and machines to perform subtasks, 𝑚m is number of multiple objectives, 𝑛n is number of subtasks in 𝑇T.

Objective function of the above model can be set according to key factors of human-machine task allocation, such as minimum degree of participation of operator, minimum human-machine system cost, or minimum risk factor. Based on objective function and constraints of above task allocation model, an intelligent optimization algorithm is used to achieve a reasonable share of duties between humans and machines for power grid regulation tasks.

C Scenario-driven Data Unbiased Technology for Large-scale Power Grid Regulation

Dispatching operational region of a large-scale power grid is generally in the neighborhood of typical operation mode. There exists a large amount of steady-state operational data in power grids, while it faces shortage of grid extreme scene and fault samples [43]. Essence of data-driven AI technology is to extract features from a large amount of data and make a prediction through continuous training and learning of the model [44]. Features and quality of data will directly affect performance of AI applications. Therefore, in view of complex, open, and various operating scenarios of large power grids, application of AI technologies requires various operational scenario data as inputs to ensure accuracy and generalization ability of model [45].

Power grid operational data are unbalanced datasets. Therefore, a model generated by unbalanced dataset training is difficult to accurately predict instances of these few-shot sample categories, while accurate identification of few-shot type samples plays a critical role in safe operation of the power grid. Fig. 5 shows a schematic diagram of organization and generation of power grid dataset samples.

These samples mainly consist of real-time grid operational data, simulated data, and data generated based on machine learning, as shown in Fig. 6. Power grid operational data, as historical data, can be equalized by two common data processing methods: undersampling and oversampling. Undersampling processing method is to ensure balance of datasets by removing part of large class samples, with commonly used methods – (Random undersampling, Tomek-link [46], ENN (Edited Nearest Neighbor) [47], etc.). Differently, oversampling processing method increases small class samples, and a representative algorithm is SMOTE (Synthetic Minority Oversampling), which generates similar data by a fixed rule algorithm and cannot accurately express state information of the power grid. The above sampling techniques are suitable for scenarios with trivial sample imbalance.

View original image Download original image

Fig. 6. The organization and generation of power grid dataset samples.

Compared with other fields such as transportation and autonomous driving, the power system has an accurate digital twin simulation system. At present, offline analysis and decision-making of the power grid are based on simulation results. Therefore, increase of small class samples can be combined with experiences of operators and Monte Carlo simulation methods to generate required samples through simulations. Simulation data samples can be used as a useful supplement to meet data requirements for model training.

With development of AI technologies, sample generation based on deep learning can also be used to expand grid data, such as recently popular generative adversarial networks (GAN) and its series [48]. GAN-series can effectively learn distribution of samples and generate new samples by alternately training generator and discriminator, which has been applied in spatial load prediction [49], renewable energy scenario generation [50], [51], and power system dynamic security assessment [52].

It takes a certain amount of time to accumulate real-time operation data of the power grid, and there is a lack of historical data of typical extreme scenarios, which cannot ensure application performance of data-driven AI models. Thus, it is necessary to ensure training samples are balanced by above-mentioned methods of data sampling and processing based on HMC, which can further improve generalization ability of training model.

D Intelligent Decision-making and Control Technology Based on HMC

Existing research on collaborative decision-making and control of human-machine systems focused on fields of autonomous driving [53], [54] and industrial robotics [55], [56]. Reference [57] and [58] use SVM and reinforcement learning intelligent algorithms respectively, and realize human-machine collaborative decision-making and control by setting their confidence indicators. However, it is rarely applied in the field of power grid regulation. The key link of human-machine decision-making and control is to realize distribution and collaboration of decision-making authority under HMC mode by considering operator-grid-environment state, to improve overall performance of human-machine collaborative system.

Based on security requirements of power grid regulation and degree of HMC coupling, pattern of human-machine collaborative intelligent decision-making can be divided into three modes: human-machine decision-making authority switching, human-machine auxiliary decision-making and control, and human-machine joint decision-making and control, as shown in Fig. 7. Selection of the above three decision-making and control modes is determined according to current state of the power grid and level and capability of HMI. In Fig. 7(a), decision-making authority is actively or passively switched between operator and machine. When a low probability event or emergency control occurs, operator will take over the system proactively, decision-making authority is assigned to operator. When internal functions of the control system has an error or a task is too complex beyond the scope of system functions, control system will temporarily issue a request to complete passive switch of operator. In addition, when machine is able to handle a task independently or operator is in a state of fatigue, machine can take over decision-making authority either actively or passively. Fig. 7(b) shows human-machine auxiliary decision-making and control, where machine refines operator’s experience into knowledge by mining and analyzing operator’s behavior, which can further optimize decision-making target. It can automatically provide auxiliary decisions for power grid operation, guide and help operator proactively, quickly, comprehensively, and accurately control current status and development trend of power grid. Fig. 7(c) shows joint human-machine decision-making and control, where operator and machine collaborate to control power grid. For power grid regulation task, machine uses machine learning or other optimization algorithms to obtain desired control strategy and establish a prediction model of operator’s control strategy. According to desired control strategy and prediction model, hybrid decision-making under HMC is obtained to ensure joint control effect of human-machine is close to desired control target, which can realize intervention correction and control of operator’s actions, and improve efficiency of collaboration while reducing adverse effects of unknown random human behavior.

View original image Download original image

Fig. 7. Patterns of human-machine collaborative intelligent decision-making for power grid regulation. (a) Human-machine decision-making authority switching. (b) Human-computer auxiliary decision-making and control. (c) Human-machine hybrid decision-making and control.

At present, it is difficult to make a correct decision separately by operators or machines during execution of complex tasks in power grid regulation. Decision-making and control are still performed in a “human-dominated, machine-followed” manner. With continuous development of machine intelligence, intelligent decision-making and control can be realized in the above-mentioned different human-machine collaborative modes by using algorithms of evaluation, optimization, and machine learning, which further enhance safety control ability of complex power grid regulation.

Ⅴ Conclusion

HHI is an important development in application of AI technologies in power grid regulation. It is a promising intelligent solution to address issues related to uncertainties, fragility and openness in power grid. This paper analyzes theory and application scenarios of HHI, establishes overall physical architecture of HHI system for power grid regulation. Based on this, the technical framework is investigated in terms of system function composition, human-machine interaction and collaboration, with an in-depth analysis of respective functional composition of the PGDT system-AI system-PGDC system-operators, as well as interaction and cooperation between them. Furthermore, key technologies are discussed, such as human-machine modeling and evaluation, optimal allocation of complex tasks, unbiased data organization, and human-machine coordinated decision-making and control involved in HMC of power grid regulation.

The proposed PGDC system with HHI is still in the initial stage of development. Application of HHI is faced with the problem of human-machine autonomous boundary definition and human-machine interaction collaboration due to variety and complexity of control tasks. With continuous improvement of machine intelligence technology, above problems will also continue to change. Future development of PGDC system with HHI is envisaged to focus on the following aspects:

1) Human-machine system modeling: It is required to establish an HMC mode and a mathematical model of HHI suitable for power grid regulation, and form basic theory and method of HAI based on human-machine interaction and collaboration paradigm.

2) Human-machine interaction and collaboration: This is required to upgrade data between human and machine to the level of knowledge interaction to realize collaboration and update of human-machine knowledge. The method of human-machine bidirectional learning and joint evolution can be used to achieve hybrid intelligent convergence cooperation and continuous optimization.

3) Implementation and application of the PGDT system: A PGDT system based on mechanism and data fusion modeling is required to strengthen interaction of data and knowledge between the PGDT system, operators and AI systems, applying exploration mechanism of reinforcement/active learning to achieve human-machine bidirectional knowledge enhancement.

Funding

This work was supported by the National Key R&D Program of China (2018AAA0101500).

References

[1]

D. X. Zhang, X. Q. Han, and C. Y. Deng, “Review on the research and practice of deep learning and reinforcement learning in smart grids,” CSEE Journal of Power and Energy Systems, vol. 4, no. 3, pp. 362–370, Sep. 2018.