(XJTLU)Methodology to MAV Auto-Polination

Methodology to MAV Auto-Pollination
(XJTLU,Jiawen Li,student ID:2039079)

Preprocessing of image data

The micro air vehicles(MAV) could precisely ‘feel’ the surrounding environment, but they also need to precept the situation to judge whether the flower needs to pollinate. In several articles, for formatting real-time data, they usually clip the image into a single square from the circle.
Figure 1. The visual field of MAVS & partial design

This preprocessing approach would eliminate some information on the edge(roughly 21.5%), triggering relatively inaccurate results. The easiest way to solve this issue is image padding. However, it might influence prediction distribution—nevertheless, the single zero-padding leads to space bias, which is not solvable to single normalization. The model needs another generative model to simulate corner data for padding. The best choice is Generative Adversarial Network(GAN). The training data would come from a big image combined with an enormous drone camera image.

However, the input still has some data missing. When the vehicle scans the overall flower’s area like a convolution kernel, the engineer has to realize those images could be regarded as continuous sequence data. Neither Residual Network(ResNet) or Gated Recurrent Unit(GRU) could handle this sort of input, not even the Temporal convolutional network(TCN). TCN perform well on dealing continuous sequence of images.But TCN need a long time for training[1] and then could not respond to the outer environment in time, which creates a time delay. Imagine when drones once detected the flower needed to pollinate, but the MAV had already passed over.

Figure 2. The training time of numerous classical models[2]

Principal Component Analysis (PCA) is necessary for simplified data to reduce the responding delay. PCA could simplify primary data with nominal size. In mathematics, it squeezes data to a single place(line, plane or multidimensional space) in a separate direction(it can persevere the original data distribution). By the way, the main body of the model would be MLP. After reducing the data dimension, the data complexity would rapidly decrease within major information.

Design the model

Please turn to the net structure for this application. The input data’s shape that we had already talked about is N1+1(N1from PCA, 1 for another embedding feature). This feature is the cosine similarity between the first two times action. (if the model is trained in 1st or second epoch, this feature would be zero). It illustrates the similarity of current behaviour to the one that MAV was used to have. Additionally, it is necessary to simplify the sequence model to the MLP.

As everyone knows, there are still have many other more complex, delicate ways to calculate similarity. The main reason for choosing cosine similarity is the simple form of the relation. Only could a concise relation learned by MLP easily.Furthermore, the output data should be designed as follows:
Output=

The sigma is a variable for judging whether the MAV need to pollinate or not(0 or 1), and mu to represents the velocity in each dimension, which are also in range 0 to 1(because 1 of any variable means the maximum speed that vehicle could reach, just for normalization of result). Therefore, the total output is a vector create for real-time action.

Next step is to define the number of unit in the net, a large network could caused a long training time or over-fitting, and a small network lack of presentational capability to solve the question(that bring under-fitting). There has a common formulas for define the size and the number of hidden layer.

Ni is the number of input
No is the number of output
Ns is the number of sample
α is a constant in range 2-10

In this application, we can get the result is S/( (N+5)*α). If we have 10000 samples and assume that each sample of the image would be a squeeze to 64 features, there should have at least 15 hidden units. Although this equation is only suited for a 1-hidden layer, it could be improved for multi-layers situations. Treat the previous hidden layer’s number as input of the current one, and then calculate it within the formula. Why is this empirical formula still work? Analogy to transfer learning, the model could reuse its first layers of weight to retrain and get new results for a new question. It is like treating those layers of the result as inputs when freezing those weights. 3 layers could represent time and more complex information, this network would composed by 5 layers. In the end, the loss function would adopt the log loss rather than the mean square error.

Construct the model

We had already discussed, the model need a sequence of continuous image on flower field. Due to the shortage of this sort of dataset on website, I could only make program in advance. The data processioning part would be complete on python (it is convenience to utilize PCA). Moreover, I used newff() and those tools to construct a fundamental network.This graph illustrate the fitting situation of the stimulation.
.

Figure 3. The fitting situation for the action vector
As we can see, the model does not perform well in that way. One of the possibilities is that the samples haven is enough information to support the network. The data that we get could only classify a flower need to be pollinated or not. It cannot decide the 3-d action on just several images. From another perspective, making another algorithm for the searching path is needless.If the data is not enough, the drone could scan the whole field and pollinate once the flower needs it that is more practical compared to use our model.

Get earlier local optimum

In major time, the environment is not as simple as we think. Several ‘abnormal’ data will cause the model shifting, thus the final result is not even the relatively best. (so we could not just waiting for the final result) The model need to set a scale for scanning whether the maximum of reward_meanloss (rewardloss) in the range is enhanced, and get the earlier result.(For instance, the figure below is the reward of 600 steps, hence the step we train next time would adjacent about 400~500 steps, the graph of loss is 10 steps per unit)

Figure 4. The loss with training steps(10 steps per unit)

Changing the learning rate and hidden layers is a common approach if the model still needs a further extension. However, be careful about the value. A tiny learning rate could induce decreasing expression of the net. Moreover, an oversize hidden layer would bring under-fitting. The bottom of the baseline model is hard to compile, so endless try at the real samples is necessary.

All in all, the model has a non-negligible disadvantage. This algorithm cannot recognize what the best direction to pollinate is. Additionally, MAV can only realize pollinate requirements based on the blossom whereas the database. When the model encounters a flower that does not bloom has a need, it creates an error.
The entire process have shown below:

Figure 5. The overall structure of MAV model design

In the next part, we will evaluate our system to a more specific degree via some criteria, such as the frequency of fully pollinating one flower.

Reference List
[1]M. Carreras, G. Deriu, L. Raffo, L. Benini, and P. Meloni, “Optimizing Temporal Convolutional Network Inference on FPGA-Based Accelerators,” IEEE Journal on Emerging and Selected Topics in Circuits and Systems, vol. 10, no. 3, pp. 348–361, Sep. 2020, doi: 10.1109/jetcas.2020.3014503.

[2]P. Lara-Bentez,Distribution of training and inference time results for all architectures.[Research Gate]. Available: https://www.researchgate.net/figure/Distribution-of-training-and-inference-time-results-for-all-architectures_fig4_347133536