天池：零样本目标识别

最新推荐文章于 2024-06-20 23:51:02 发布

CVAIDL

最新推荐文章于 2024-06-20 23:51:02 发布

阅读量844

点赞数

天池：零样本目标识别

转载： https://blog.csdn.net/nima1994/article/details/82420637

2018之江杯全球人工智能大赛-零样本图像目标识别

简单数据分析

这里写图片描述

jupyter：github地址

数据预处理

将label_list和class_wordembeddings合并，处理后结果如标签\t特征。
sample_processing.py

<span style="color:#000000"><code class="language-python"><span style="color:#000088 !important">import</span> pandas <span style="color:#000088 !important">as</span> pd

dir = <span style="color:#009900 !important">"d:/ZSL_ImageGame/DatasetA_train_20180813/"</span>


<span style="color:#000088 !important">def</span> <span style="color:#009900 !important">words_embed</span><span style="color:#4f4f4f !important">()</span>:
    <span style="color:#009900 !important">"""
    处理label_list和class_wordembeddings,文本特征
    :return: data/word_embeddings.csv
    """</span>
    train_labels = pd.read_csv(dir + <span style="color:#009900 !important">"label_list.txt"</span>, sep=<span style="color:#009900 !important">'\t'</span>, header=<span style="color:#000088 !important">None</span>)
    print(train_labels.head())

    train_words = pd.read_csv(dir +<span style="color:#009900 !important">"class_wordembeddings.txt"</span>, sep=<span style="color:#009900 !important">' '</span>, header=<span style="color:#000088 !important">None</span>)
    print(train_words.head())

    res = pd.merge(train_labels, train_words, left_on=<span style="color:#006666 !important">1</span>, right_on=<span style="color:#006666 !important">0</span>)

    res = res.drop([<span style="color:#006666 !important">1</span>, <span style="color:#009900 !important">'1_x'</span>, <span style="color:#009900 !important">'0_y'</span>], axis=<span style="color:#006666 !important">1</span>)
    print(res.head())
    res.to_csv(<span style="color:#009900 !important">'output/word_embeddings.csv'</span>, index=<span style="color:#000088 !important">None</span>, header=<span style="color:#000088 !important">None</span>, sep=<span style="color:#009900 !important">'\t'</span>)

<span style="color:#000088 !important">def</span> <span style="color:#009900 !important">traindata</span><span style="color:#4f4f4f !important">()</span>:
    <span style="color:#009900 !important">"""
    连接数据
    :return:
    """</span>
    train = pd.read_csv(dir + <span style="color:#009900 !important">'train.txt'</span>, header=<span style="color:#000088 !important">None</span>, sep=<span style="color:#009900 !important">'\t'</span>)
    train_attr = pd.read_csv(dir + <span style="color:#009900 !important">"attributes_per_class.txt"</span>, sep=<span style="color:#009900 !important">'\t'</span>, header=<span style="color:#000088 !important">None</span>)
    train_words=pd.read_csv(<span style="color:#009900 !important">'output/word_embeddings.csv'</span>,sep=<span style="color:#009900 !important">'\t'</span>,header=<span style="color:#000088 !important">None</span>)

    print(train.head())
    print(train_attr.head())
    print(train_words.head())
    <span style="color:#880000 !important"># res.to_csv('output/traindata.csv',index=None,header=None,sep='\t')</span>

<span style="color:#000088 !important">if</span> __name__ == <span style="color:#009900 !important">'__main__'</span>:
    <span style="color:#880000 !important"># words_embed()</span>
    traindata()</code></span>

选取训练集和验证集

有关表明，submit.txt存放着验证数据。如下代码，其标签ZJL160打错成了ZJL178，当然178还打着178。submit.txt含有标签160~200，所有样本都可在train.txt找到。

False Label	True Label
178	160
178	178

<span style="color:#000000"><code><span style="color:#000088 !important">import</span> numpy <span style="color:#000088 !important">as</span> np
<span style="color:#000088 !important">import</span> pandas <span style="color:#000088 !important">as</span> pd

src=<span style="color:#009900 !important">"d:/ZSL_ImageGame/DatasetA_train_20180813/"</span>
valid=pd.read_csv(src+<span style="color:#009900 !important">"submit.txt"</span>,header=<span style="color:#000088 !important">None</span>,sep=<span style="color:#009900 !important">"\t"</span>)
train=pd.read_csv(src+<span style="color:#009900 !important">'train.txt'</span>,header=<span style="color:#000088 !important">None</span>,sep=<span style="color:#009900 !important">'\t'</span>)

print(valid.head())
print(valid.shape)
<span style="color:#009900 !important">"""
                                       0       1
0  9f1d3113f1fcb573596ca99ecb712364.jpeg  ZJL178
1  9f73904f7a72fa7285b80f2ae8286066.jpeg  ZJL178
2  619bf8d90e1fa19a7f2966bd38b27ccd.jpeg  ZJL178
3  6773ca5a1a615fc0d67f836e0772ff46.jpeg  ZJL178
4  e1badc8feb1e4d4a6e44eb382d13bc24.jpeg  ZJL178
(8291, 2)
"""</span>

print(train.head())
print(train.shape)
<span style="color:#009900 !important">"""
                                       0     1
0  a6394b0f513290f4651cc46792e5ac86.jpeg  ZJL1
1  2fb89ef2ace869d3eb3bdd3afe184e1c.jpeg  ZJL1
2  eda9f3bef2bd8da038f6acbc8355fc25.jpeg  ZJL1
3  7d93ef45972154aae150b4f9980a79c0.jpeg  ZJL1
4  fb901b4f9a8e396c1d0155bccc5e5671.jpeg  ZJL1
(38221, 2)
"""</span>
print(sorted([int(str(i)[<span style="color:#006666 !important">3</span>:]) <span style="color:#000088 !important">for</span> i <span style="color:#000088 !important">in</span> set(train.iloc[:,<span style="color:#006666 !important">1</span>].values)]))
<span style="color:#009900 !important">"""
[1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 18, 19, 21, 22, 23, 24, 25, 26, 28, 29, 30, 31, 32, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92, 93, 94, 95, 96, 97, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 109, 110, 111, 113, 114, 115, 116, 117, 118, 119, 120, 121, 122, 123, 124, 125, 126, 127, 128, 129, 130, 131, 132, 133, 135, 137, 138, 139, 140, 141, 142, 143, 144, 145, 146, 147, 149, 150, 151, 152, 153, 154, 156, 157, 158, 159, 160, 161, 162, 163, 164, 165, 166, 167, 168, 169, 170, 171, 172, 173, 174, 175, 176, 177, 178, 179, 180, 181, 182, 183, 184, 185, 186, 187, 188, 189, 190, 191, 192, 193, 194, 195, 196, 197, 198, 199, 200]

"""</span>

mer=pd.merge(valid,train,left_on=<span style="color:#006666 !important">0</span>,right_on=<span style="color:#006666 !important">0</span>)
print(mer.head())
print(mer.shape)
<span style="color:#009900 !important">"""
                                       0     1_x     1_y
0  9f1d3113f1fcb573596ca99ecb712364.jpeg  ZJL178  ZJL160
1  9f73904f7a72fa7285b80f2ae8286066.jpeg  ZJL178  ZJL160
2  619bf8d90e1fa19a7f2966bd38b27ccd.jpeg  ZJL178  ZJL160
3  6773ca5a1a615fc0d67f836e0772ff46.jpeg  ZJL178  ZJL160
4  e1badc8feb1e4d4a6e44eb382d13bc24.jpeg  ZJL178  ZJL160
(8291, 3)
"""</span>

<span style="color:#880000 !important"># ## 标签ZJL160的打错成了ZJL178</span>
<span style="color:#000088 !important">from</span> sklearn.metrics <span style="color:#000088 !important">import</span> classification_report
<span style="color:#000088 !important">from</span> sklearn.metrics <span style="color:#000088 !important">import</span> confusion_matrix
<span style="color:#000088 !important">import</span> seaborn <span style="color:#000088 !important">as</span> sns
<span style="color:#000088 !important">import</span> matplotlib.pyplot <span style="color:#000088 !important">as</span> plt
<span style="color:#000088 !important">from</span> sklearn.metrics <span style="color:#000088 !important">import</span> cohen_kappa_score
true_label=mer.iloc[:,<span style="color:#006666 !important">1</span>].values
res=mer.iloc[:,<span style="color:#006666 !important">2</span>].values

print()
print(classification_report(true_label,res))
print(<span style="color:#009900 !important">"kappa: "</span>,cohen_kappa_score(true_label,res))
mat = confusion_matrix(true_label,res)
sns.heatmap(mat,annot=<span style="color:#000088 !important">True</span>,square=<span style="color:#000088 !important">True</span>,fmt=<span style="color:#009900 !important">"d"</span>)
plt.show()</code></span>

这里写图片描述
此处可以删除错打成ZJL178的标签或者将错打的改成168，如下为删除代码：

<span style="color:#000000"><code>mer=mer[~((mer<span style="color:#009900 !important">.iloc</span>[:,<span style="color:#006666 !important">1</span>]==<span style="color:#009900 !important">"ZJL178"</span>) &(mer<span style="color:#009900 !important">.iloc</span>[:,<span style="color:#006666 !important">2</span>]==<span style="color:#009900 !important">"ZJL160"</span>))]</code></span>

这样，submit.txt就从8291变成了8094，然后在从训练集删除验证集的部分。通过submit.txt和train.txt merge上看，submit的样本全部来自train.txt（其中的160~200）。干脆简单的，直接从train.txt中选取ZJLXX~ZJLXX作为验证，其他作为训练。

<span style="color:#000000"><code><span style="color:#000088 !important">import</span> numpy <span style="color:#000088 !important">as</span> np
<span style="color:#000088 !important">import</span> pandas <span style="color:#000088 !important">as</span> pd

src=<span style="color:#009900 !important">"d:/ZSL_ImageGame/DatasetA_train_20180813/"</span>
train=pd.read_csv(src+<span style="color:#009900 !important">'train.txt'</span>,header=<span style="color:#000088 !important">None</span>,sep=<span style="color:#009900 !important">'\t'</span>)
<span style="color:#880000 !important"># print(train.head())</span>
print(<span style="color:#009900 !important">"train.txt数量"</span>,train.shape)

s= [<span style="color:#009900 !important">"ZJL"</span>+str(i) <span style="color:#000088 !important">for</span> i <span style="color:#000088 !important">in</span> range(<span style="color:#006666 !important">196</span>,<span style="color:#006666 !important">201</span>)]
<span style="color:#880000 !important"># print(s)</span>

zsl_validate=train[train.iloc[:,<span style="color:#006666 !important">1</span>].isin(s)].sample(frac=<span style="color:#006666 !important">1</span>,random_state=<span style="color:#006666 !important">2018</span>)
<span style="color:#880000 !important"># print(validate.head())</span>
print(<span style="color:#009900 !important">"零样本测试集"</span>,zsl_validate.shape)

traindata=train[~train.iloc[:,<span style="color:#006666 !important">1</span>].isin(s)].sample(frac=<span style="color:#006666 !important">1</span>,random_state=<span style="color:#006666 !important">2018</span>)
<span style="color:#880000 !important"># print("",traindata.shape)</span>

train_img=traindata.iloc[:-<span style="color:#006666 !important">1000</span>,:]
print(<span style="color:#009900 !important">"图片训练集"</span>,train_img.shape)
validate_img=traindata.iloc[-<span style="color:#006666 !important">1000</span>:,:]
print(<span style="color:#009900 !important">"图片验证集"</span>,validate_img.shape)



zsl_validate.to_csv(<span style="color:#009900 !important">"../data/zsl_validate.csv"</span>, sep=<span style="color:#009900 !important">"\t"</span>, header=<span style="color:#000088 !important">None</span>, index=<span style="color:#000088 !important">None</span>)
train_img.to_csv(<span style="color:#009900 !important">"../data/train_img.csv"</span>, sep=<span style="color:#009900 !important">"\t"</span>, header=<span style="color:#000088 !important">None</span>, index=<span style="color:#000088 !important">None</span>)
validate_img.to_csv(<span style="color:#009900 !important">"../data/validate_img.csv"</span>, sep=<span style="color:#009900 !important">"\t"</span>, header=<span style="color:#000088 !important">None</span>, index=<span style="color:#000088 !important">None</span>)</code></span>

train.txt数量 (38221, 2)
零样本测试集 (1016, 2)
图片训练集 (36205, 2)
图片验证集 (1000, 2)

实验部分

整体思路如下，KNN使用1近邻：
这里写图片描述
实验中我使用train.txt中的2000条数据作为验证集，其他作为训练集来训练CNN模型，分别使用VGG16,Xception,自定义模型微调和重训练；发现验证集的精度0.2多，训练集能达到0.9以上，据说训练样本打的标记不够好。也就是说，图片特征这一步就出现问题了 \wulian \mudenggoudai ，
这里写图片描述
映射使用的是神经网络（据说使用岭回归较好），线上精度较差。

此处仅贴出批量数据训练的flow（个人认为有轻微参考价值）。

<span style="color:#000000"><code class="language-python"><span style="color:#000088 !important">import</span> keras
<span style="color:#000088 !important">import</span> pandas <span style="color:#000088 !important">as</span> pd
<span style="color:#000088 !important">import</span> numpy <span style="color:#000088 !important">as</span> np
<span style="color:#000088 !important">import</span> cv2
<span style="color:#000088 !important">from</span> keras_preprocessing.image <span style="color:#000088 !important">import</span> ImageDataGenerator

train_attr = pd.read_csv(<span style="color:#009900 !important">"d:/ZSL_ImageGame/DatasetA_train_20180813/attributes_per_class.txt"</span>, sep=<span style="color:#009900 !important">'\t'</span>, header=<span style="color:#000088 !important">None</span>)
train_words = pd.read_csv(<span style="color:#009900 !important">r"data/word_embeddings.csv"</span>, sep=<span style="color:#009900 !important">'\t'</span>, header=<span style="color:#000088 !important">None</span>)
labelkeys = sorted(set(train_attr.iloc[:, <span style="color:#006666 !important">0</span>].values.tolist()))

datagen = ImageDataGenerator(rescale=<span style="color:#006666 !important">1.</span>/<span style="color:#006666 !important">255</span>,rotation_range=<span style="color:#006666 !important">20</span>,width_shift_range=<span style="color:#006666 !important">0.2</span>,height_shift_range=<span style="color:#006666 !important">0.2</span>,
                                 shear_range=<span style="color:#006666 !important">0.2</span>,zoom_range=<span style="color:#006666 !important">0.5</span>,horizontal_flip=<span style="color:#000088 !important">True</span>,fill_mode=<span style="color:#009900 !important">'nearest'</span>)

<span style="color:#000088 !important">class</span> <span style="color:#4f4f4f !important">DataPiple</span>:

    <span style="color:#000088 !important">def</span> <span style="color:#009900 !important">__init__</span><span style="color:#4f4f4f !important">(self,target,imgsize=<span style="color:#006666 !important">64</span>,impro=False)</span>:
        <span style="color:#009900 !important">"""

        :param target:
        :param impro: 是否数据增强
        """</span>

        self.target = pd.read_csv(target, header=<span style="color:#000088 !important">None</span>, sep=<span style="color:#009900 !important">'\t'</span>).sample(frac=<span style="color:#006666 !important">1</span>,random_state=<span style="color:#006666 !important">2018</span>)
        self.fea_size=len(self.target)
        self.impro=impro
        self.imgsize=imgsize

    <span style="color:#000088 !important">def</span> <span style="color:#009900 !important">readOne</span><span style="color:#4f4f4f !important">(self,pos)</span>:
        t=self.target.iloc[pos,:]
        im = cv2.imread(<span style="color:#009900 !important">"d:/ZSL_ImageGame/DatasetA_train_20180813/train/"</span> + t[<span style="color:#006666 !important">0</span>])
        im = cv2.resize(im, dsize=(self.imgsize, self.imgsize))
        attr=train_attr[train_attr[<span style="color:#006666 !important">0</span>]==t[<span style="color:#006666 !important">1</span>]].values[<span style="color:#006666 !important">0</span>,<span style="color:#006666 !important">1</span>:]
        word=train_words[train_words[<span style="color:#006666 !important">0</span>]==t[<span style="color:#006666 !important">1</span>]].values[<span style="color:#006666 !important">0</span>,<span style="color:#006666 !important">1</span>:]
        label=np.zeros(shape=(len(labelkeys)),dtype=np.uint8)
        label[labelkeys.index(t[<span style="color:#006666 !important">1</span>])]=<span style="color:#006666 !important">1</span>
        <span style="color:#000088 !important">return</span> im,attr,word,label


    <span style="color:#000088 !important">def</span> <span style="color:#009900 !important">readFeather</span><span style="color:#4f4f4f !important">(self,pos,size)</span>:
        ims=[]
        attrs=[]
        words=[]
        labels=[]

        <span style="color:#000088 !important">for</span> i <span style="color:#000088 !important">in</span> range(pos,min(pos+size,self.fea_size)):
            im,attr,word,label=self.readOne(i)
            ims.append(im)
            attrs.append(attr)
            words.append(word)
            labels.append(label)
        ims=np.array(ims)
        <span style="color:#880000 !important"># 做数据增强</span>
        <span style="color:#000088 !important">if</span> self.impro == <span style="color:#000088 !important">True</span>:
            ims=datagen.flow(ims,batch_size=len(ims),shuffle=<span style="color:#000088 !important">False</span>).__next__()
        <span style="color:#000088 !important">else</span>:
            ims=ims/<span style="color:#006666 !important">255.0</span>
        ims = np.array(ims)
        attrs=np.array(attrs)
        words=np.array(words)
        labels=np.array(labels)

        <span style="color:#000088 !important">return</span> ims,attrs,words,labels


    <span style="color:#000088 !important">def</span> <span style="color:#009900 !important">create_inputs</span><span style="color:#4f4f4f !important">(self,size=<span style="color:#006666 !important">64</span>)</span>:

        <span style="color:#000088 !important">while</span> <span style="color:#000088 !important">True</span>:
            <span style="color:#000088 !important">for</span> i <span style="color:#000088 !important">in</span> range(<span style="color:#006666 !important">0</span>,self.fea_size,size):
                ims, attrs, words,labels=self.readFeather(i,size)
                <span style="color:#880000 !important"># print(ims.shape)</span>
                <span style="color:#880000 !important"># print(attrs.shape)</span>
                <span style="color:#880000 !important"># print(words.shape)</span>
                <span style="color:#000088 !important">yield</span> ims, labels

<span style="color:#000088 !important">if</span> __name__ == <span style="color:#009900 !important">'__main__'</span>:
    <span style="color:#880000 !important"># print(labelkeys)</span>

    dp=DataPiple(target=<span style="color:#009900 !important">r"D:\ZSL_ImageGame\DatasetA_train_20180813\train.txt"</span>,impro=<span style="color:#000088 !important">True</span>)
    s=dp.create_inputs(<span style="color:#006666 !important">64</span>)
    r,p=s.__next__()

    print(len(r),r.shape,p.shape)

    <span style="color:#000088 !important">import</span> matplotlib.pyplot <span style="color:#000088 !important">as</span> plt

    plt.imshow(r[<span style="color:#006666 !important">2</span>])
    plt.show()

    <span style="color:#880000 !important"># print(labelkeys)</span></code></span>

其他代码放在Github上。第一次参加此种比赛，无论设备（训练真的慢，……，训练差不多我就终止程序了）还是水平，发现真是cai奥，还是学好基础再来闲逛。此记，用以缅怀。

CVAIDL

关注

0
点赞
踩
1

收藏

觉得还不错? 一键收藏
0
评论
天池：零样本目标识别

天池：零样本目标识别转载： https://blog.csdn.net/nima1994/article/details/824206372018之江杯全球人工智能大赛-零样本图像目标识别简单数据分析jupyter：github地址数据预处理将label_list和class_wordembeddings合并，处理后结果如标签\t特征。sample_proces...
复制链接

扫一扫