analysis of image dataset checking result (image segmentation experiment)

Why we should ckeck data set before doing experiments on it

1. Although dataset from different organization is protected by copyright, maybe part of data in it is from open source dataset. So we could not guarantee that there is not the same data in different dataset from different organization until we check and prove that.
2. Extra
So what is a pair of same data? In the case of medical image, are the images the same or not from different time point of the same person? So why shouldn't we use the machine learning method to analysis and check it?
condition: 
if we have trained a 3d end-to-end segmentation network, so in fact we have extracted point wise fore-back ground classification features.

Using SPP( Spatital Pyramid Pooling) to gain position relativated features

a breif introduction of SPP feature selected method:
a layer of SPP is like this:

concating five one-dimension features which is reshape from five semantic level features maps on VNet, we get the final features which has 36208 dimensions.

with SPP feature selected method, which has a well robust on feature map transform, we have the posibility to meature the distance of two samples, or analysis the distrubution of the certain dataset.

using dimension redusing method to analysis data distribution

there is totally 221 CT scans((131+70 CodaLab)+(20 sliver07)) joined in this experiment.
Using PCA to reform linear correlation features and reduce dimension.
after this operation, 220 dimensions is left. Eigenvalues(first 20) and using the first 3 Principal Companent visualized on Matlab is shown below:

evaluation:
knowing the distribution of samples is helpful for generate or discreminate an unknow samples. So maybe we can use it to generate new samples with it, and then we cound have a stronger CNN network and get a more accurate segmentation result.

using distance matrix to find the similar data

1. We calculate the distance matrix (221x221).
2. reshape it to one dimension and sort it (221x(221-1)/2) in the order of ascending.
3.the result is below (first 50):

conclusion:
After reading these volumes, I know there is some CT scans similar to each other. They are from the same person but scan in a different time (1-33).