Going Deeper in Spiking Neural Networks:VGG and Residual Architectures笔记

摘要

论文提出了一种将ANN网络转换成SNN的算法,并证明了该算法在复杂视觉识别问题(如cifar-10和ImageNet)中的有效性。


介绍

将ANN直接转换成SNN时,在SNN上的性能远低于ANN,这是因为应为SNN中拥有处理时序信息的能力,这点与ANN有很大的区别


相关工作


主要贡献

1、针对ANN转SNN,提出几个ANN结构的约束。
2、提出ANN转SNN的normalization方法
3、提出一种可以用在SNN的残差结构块


准备工作

1、用频率编码的方式产生脉冲(泊松脉冲),论文中time-step为2500。
2、在ANN采用ReLU函数,在SNN使用IF模型
3、明确在ANN转SNN的关键在于调整SNN的阈值与突触权重。太高的阈值需要更长的时间才能产生一个脉冲,阈值太小就难以区分输入的差距。

ANN结构约束
1、不设置bias,因为bias在SNN中会增大阈值参数调整的难度,由于没有bias,所以BN也用不了,所以在ANN用drop out来替代BN。
2、使用average pool


深度卷积SNN体系结构:VGG

之前就有人提出“Data-Based Normalization”,具有实现方式有两种:“weight-normalization”和“threshold-normalization”,他们最终的目的是一致的,只是实现过程不同。<

  • 1
    点赞
  • 4
    收藏
    觉得还不错? 一键收藏
  • 0
    评论
This current book provides new research on artificial neural networks (ANNs). Topics discussed include the application of ANNs in chemistry and chemical engineering fields; the application of ANNs in the prediction of biodiesel fuel properties from fatty acid constituents; the use of ANNs for solar radiation estimation; the use of in silico methods to design and evaluate skin UV filters; a practical model based on the multilayer perceptron neural network (MLP) approach to predict the milling tool flank wear in a regular cut, as well as entry cut and exit cut, of a milling tool; parameter extraction of small-signal and noise models of microwave transistors based on ANNs; and the application of ANNs to deep-learning and predictive analysis in semantic TCM telemedicine systems. Chapter 1 - Today, the main effort is focused on the optimization of different processes in order to reduce and provide the optimal consumption of available and limited resources. Conventional methods such as one-variable-at-a-time approach optimize one factor at a time instead of all simultaneously. Unlike this method, artificial neural networks provide analysis of the impact of all process parameters simultaneously on the chosen responses. The architecture of each network consists of at least three layers depending on the nature of process which to be analyzed. The optimal conditions obtained after application of artificial neural networks are significantly improved compared with those obtained using conventional methods. Therefore artificial neural networks are quite common method in modeling and optimization of various processes without the full knowledge about them. For example, one study tried to optimize consumption of electricity in electric arc furnace that is known as one of the most energy-intensive processes in industry. Chemical content of scrap to be loaded and melted in the furnace was selected as the input variable while the specific electricity consumption was the output variable. Other studies modeled the extraction and adsorption processes. Many process parameters, such as extraction time, nature of solvent, solid to liquid ratio, extraction temperature, degree of disintegration of plant materials, etc. have impact on the extraction of bioactive compounds from plant materials. These parameters are commonly used as input variables, while the yields of bioactive compounds are used as output during construction of artificial neural network. During the adsorption, the amount of adsorbent and adsorbate, adsorption time, pH of medium are commonly used as the input variables, while the amount of adsorbate after treatment is selected as output variable. Based on the literature review, it can be concluded that the application of artificial neural networks will surely have an important role in the modeling and optimization of chemical processes in the future. viii Gayle Cain Chapter 2 - Problems in chemistry and chemical engineering are composed of complex systems. Various chemical processes in chemistry and chemical engineering can be described by different mathematical functions as, for example, linear, quadratic, exponential, hyperbolic et al. There are many of calculated and experimental descriptors/molecular properties to describe the chemical behavior of the substances. It is also possible that many variables can influence the desired response. Usually, chemometrics is widely used as a valuable tool to deal chemical data, and to solve complex problems. In this context, Artificial Neural Networks (ANN) is a chemometric tool that may provide accurate results for complex and non-linear problems that demand high computational costs. The main advantages of ANN techniques include learning and generalization ability of data, fault tolerance and inherent contextual information processing in addition to fast computation capacity. Due to the popularization, there is a substantial interest in ANN techniques, in special in their applications in various fields. The following types of applications are considered: data reduction using neural networks, overlapped signal resolution, experimental design and surface response, modeling, pattern recognition, and multivariate regression. Chapter 3 - Energy consumption in buildings and indoor thermal comfort nowadays issues in engineering applications. A deep analysis of these problems generally requires many resources. Many studies were carried out in order to improve the methodology available for the evaluation of the energy consumption or indoor thermal conditions; interesting solutions with a very good feedback found in the Literature are the Artificial Neural Networks (ANNs). The peculiarity of ANNs is the opportunity of simulating and resolving complex problems thanks to their architecture, which allows to identify the combination of the involved parameters even when they are in a large amount. The Artificial Neural Networks (ANNs) are very common in engineering applications for simulating the energy performance of buildings, for predicting a particular parameter, or for evaluating the indoor thermal conditions in specific environments. However, many different Artificial Neural Networks are available and each of them should be applied in a specific field. This chapter examines and describes the ANNs generally used in the engineering field. Studies of ANNs applied in topics such as energy consumption in buildings, gas emissions evaluation, indoor and outdoor thermal conditions calculation, renewable energy sources investigation, and lighting and acoustics applications are reported. After a brief description of the main characteristics of ANNs, which allows to focus on the main peculiarity and characteristics of this kind of algorithms, some applications shown in the Literature and applied to engineering issues are described. In the first part of the chapter an analysis of the main parameters which influence the ANN implementation in the examined papers was carried out, then some applications of ANN in energy and buildings field found in the Literature are described. In particular, the main studies were described considering five different clusters: in the first group the ANN applications to buildings and traditional energy plants are showed, in the second one the ANN implementation for the thermal and energy performance evaluation of renewable energy sources are reported. In the third and forth clusters the applications found in the Literature for the indoor thermal parameters investigation and outdoor thermal conditions calculation are described, while in the last one other topics investigated using ANN models such as lighting and acoustics issues are considered. Preface ix Chapter 4 - Biodiesel is generally accepted as an alternative fuel to fossil-derived diesel and has been produced from numerous oil-based biological sources. Determination of fuel properties of biodiesel has mainly being experimental which in most cases is expensive, time consuming and strenuous. These fuel properties are strongly linked to fatty acid (FA) composition of the oil used in biodiesel production. This paper presents the application of artificial neural network (ANN) in predicting selected biodiesel fuel properties (cetane number (CN), flash point (FP), kinematic viscosity (KV) and density) from the FA compositions of the oils contained in raw materials employed in biodiesel production. ANNs are nonlinear computer algorithms which are widely and successfully applied in many fields of study in simulating complex problems. Palmitic, stearic, oleic, linoleic and linolenic acids were observed to be the principal FAs in oils gathered from 58 feedstocks sourced from in literature. FAs outside the five prominent FAs were embedded into them based on their levels of saturation and unsaturation, and were used as inputs in training the networks. Neural network toolbox in MATLAB® (2013b) was employed in this study. Data of FAs and fuel properties were used in training CN, KV, FP and density networks based on back propagation algorithm. Levenberg–Marquardt algorithm, logsig (hidden layer) and purelin (out layer) were used as training algorithm and transfer functions, respectively. Different architectures (5-6-4 (CN and FP); 5-7-4 (KV); 6-5-4 (density)) were employed in training the networks due to variation in the number of neurons in both the input (temperature as additional parameter) and hidden layers. In this study, the networks achieved high accuracy for the prediction of CN, KV, FP and density with correlation coefficients of 0.962, 0.943, 0.987 and 0.985, respectively. This result indicates good agreement between the predicted results and the experimental values, and those of previous studies found in literature. Errors associated with the prediction performance of the networks were estimated using statistical methods and were found to be within satisfactory range of accuracy. Finally, this study shows that the networks via ANN modelling can be alternative methods in predicting CN, KV, FP and density from FA compositions outside the intricate and time-consuming standard test methods. Chapter 5 - The objective of this paper is show how ANN methods can be used for solar radiation estimation at short time-scale (5-min): firstly an ANN method was applied for estimating horizontal solar irradiation from other meteorological parameters more easily and frequently measured over the World and a second ANN model was developed for transforming horizontal solar irradiation into tilted irradiation. Only one thousand continental stations around the world measures solar radiation and often with a poor quality. The authors showed that 5-min solar irradiations can be estimated from more available, more readily measurable and cheaper data using Artificial Neural Networks (ANN). 7 meteorological parameters and 3 calculated parameters are used as inputs, thus 1023 combinations of inputs data are possible; the best combinations of inputs are pursued. The best ANN models have a good adequacy mainly with sunshine duration in the input set. The 6 and 10 inputs models have a relative root means square error (nRMSE) equal to 19.35% and 18.65% which is very good for such a time-step. Solar collector are rarely in horizontal position; However, solar radiation is always measured in a horizontal plane; converting measured horizontal global solar irradiance in tilted ones is a difficult task, particularly for a small time-step and for not-averaged data. Conventional methods (statistical, correlation, ...) are not always efficient with time-step less than one hour; thus, the authors want to know if an Artificial Neural Network (ANN) is able to realize this conversion with a good accuracy when applied to 5-min solar radiation data. x Gayle Cain nRMSE is around 8% for the optimal configuration, which corresponds to a very good accuracy for such a short time-step. These two successive studies show the applicability of ANN methods for the estimation of solar radiation; estimating solar radiation is particularly difficult because the sky diffuse component of solar radiation is anisotropic and the relations between parameters are rarely linear. Chapter 6 - Excessive exposure to sunlight is the major cause of progressive skin photo aging, sunburn and skin cancers. The UVB component of sunlight directly damages cellular DNA and leads to the formation of squamous cell carcinomas, while the UVA component of sunlight penetrates deeper into the skin causing DNA damage through generation of reactive oxygen species (ROS). UV filters are the active ingredients in sunscreen products, which protect skin from the dangerous effects of UV light by absorbing, reflecting, or diffusing UV radiation. In order to maintain effective UV protection, sunscreen filters should remain on the skin surface, accumulate in the stratum corneum, forming an effective barrier against UV radiation without transdermally penetrating into the systemic circulation. Further skin penetration significantly reduces their efficacy and may also cause phototoxic and photoallergic skin reactions. However, chemicals in contact with the skin have the potential to be absorbed into the skin and enter the systemic circulation, with several studies reporting that a number of organic filters significantly penetrate the skin. For assessment of dermal absorption, in vitro and in vivo methods are used, although in vitro tests are preferred for ethical reasons and feasibility. Therefore, it would be useful if the skin penetration of a sunscreen filter can be predicted from its chemical structure alone. Computational and QSAR based methods can be quite useful for development of skin permeability models and have been used to relate physicochemical parameters to dermal permeability to predict dermal penetration and absorption of chemicals. Skin penetration or partitioning like sorption processes are generally driven by hydrophobic effects, which are expected to correlate with molecular size and lipophilicity, together with the various intermolecular interactions, which occur between the permeant and the skin. Hence, this study aimed to develop a QSAR using a heterogeneous data set based on published skin penetration data and then to use this established model to predict the skin penetration of UV sunscreen filter molecules. In order to overcome the limitations associated with linear modelling, artificial neural networks (ANNs) were used to build the QSAR model. Sensitivity analysis was also incorporated into the modelling process in order to establish the molecular requirements for the ideal sunscreen filter. The developed model provides insight into the molecular structural requirements that are important for an effective UV sunscreen filter, particularly in relation to dermal absorption. Producing sunscreens with limited dermal absorption of actives is a challenge for the cosmetic industry so the developed QSAR model should prove useful in developing more effective and safer sunscreen actives. Chapter 7 - Milling cutters are important cutting tools used in milling machines to perform milling operations, which are prone to wear and subsequent failure. In this research work, a practical model based on the multilayer perceptron neural network (MLP) approach to predict the milling tool flank wear in a regular cut, as well as entry cut and exit cut, of a milling tool is proposed. Indeed, a MLP–based model was successfully used here to predict the milling tool flank wear (output variable) as a function of the following input variables: the time duration of experiment, depth of cut, feed, type of material, etc. Regression with optimal hyperparameters was performed and a correlation coefficient equal to 0.92 was obtained. To Preface xi accomplish the objective of this study, the experimental dataset represents experiments from runs on a milling machine under various operating conditions. Data sampled by three different types of sensors (acoustic emission sensor, vibration sensor and current sensor) were acquired at several positions. The MLP–based model’s goodness of fit to experimental data confirmed the good performance of this model. Finally, conclusions of this work are exposed. Chapter 8 - Microwave transistors are among the key components of circuits used in modern communication systems. In computer aided design of these circuits it is necessary to use their accurate and reliable models in order to represent them properly. There are a plenty of models developed, but still the models based on a transistor equivalent circuit representation are the most widely used and preferred by the circuit designers. The parameters of equivalent circuit models are extracted from a set of measured characteristics of a transistor to be modeled. For certain models there are analytical approaches for model parameter extraction. However, optimizations in circuit simulators are dominantly applied. Optimizations take a certain amount of time, which is especially important when repeated iterations are needed to determine the model parameters under different transistor working conditions. Artificial neural networks have appeared to be a very convenient tool to develop efficient extraction procedures of device model parameters. In this chapter a comprehensive study of the developed neural network based extraction approaches is given, considering transistor small-signal and noise models. A short introduction on the microwave transistor models and frequently used extraction procedures is given at the beginning, followed by a description of the multilayered neural networks and procedures of their training and validation. The main part of the Chapter refers to several extraction approaches based on neural networks, starting from the development of the extraction procedure, through their validation and up to the final application. The advantages and possible limitations are discussed. Appropriate numerical results are included to illustrate and verify the presented procedures. Chapter 9 - The study aims to establish a deep learning and predictive model in the semantic TCM telemedicine system using Artificial Neural Network Microsoft Azure Machine Learning. In Chinese Medicine diagnosis, four examination methods: Questioning/history taking, inspection, auscultation (listening) and olfaction (smelling), and palpation. Deep learning is an appropriate technique for the clinical decision support. The result is promising. Next step includs studying the herb-herb interaction. And when a model has been validated, it is easy to publish this as a web service with an auto-documented REST API, to be consumed by apps, and in future we deploy as SaaS and Integrative Medicine Model and using the Microsoft Azure and NVidia the state-of-the-art GPU Visualization Infrastructure and GPU Compute Infrastructure.

“相关推荐”对你有帮助么?

  • 非常没帮助
  • 没帮助
  • 一般
  • 有帮助
  • 非常有帮助
提交
评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值