CTR之FM模型及keras实现

1. 理论

FM模型：
和LR模型相比，FM模型引入了二阶特征组合项：
$$y(\mathbf{x})=w_0+\sum _{i=1}^n{w}_i{x}_i+\sum _{i=1}^n\sum _{j=i+1}^n{w}_{ij}{x}_i{x}_j$$

$n$ 代表样本的特征数量， $x_i$ 是第$i$个特征的值，$w_0,w_i,w_{ij}$ 是模型的参数。

经过矩阵分解$W=V^{T}V$：

$$y(\mathbf{x})=w_0+\sum _{i=1}^n{w}_i{x}_i+\sum _{i=1}^n\sum _{j=i+1}^n⟨{\mathbf{v}}_i,{\mathbf{v}}_j⟩x_i{x}_j$$

其中，$V$的维度是$n\ast k$, 其中 $k<<n$.

采样MSE损失函数求解回归问题，采样Hinge或者Cross-entropy损失求解分类问题。二元分类，FM的输出经过sigmoid变换。

二次项的简化技巧：

$$\sum _{i=1}^n\sum _{j=i+1}^n⟨{\mathbf{v}}_i,{\mathbf{v}}_j⟩x_i{x}_j=\frac{1}{2}\sum _{f=1}^k\left({\left(\sum _{i=1}^n{v}_{i,f}{x}_i\right)}^2-\sum _{i=1}^n{v}_{i,f}^2{x}_i^2\right)$$

简化后的FM复杂度 $O(k\ast n)$

2. python实现

import pandas as pd
import numpy as np
from sklearn.metrics import log_loss, roc_auc_score
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import LabelEncoder, MinMaxScaler
from sklearn.datasets import load_breast_cancer

import keras.backend as K
from keras.layers import Input,Dense,Add,Activation
from keras import optimizers
from keras.models import Model
from keras.engine.topology import Layer

class CrossLayer(Layer):
    def __init__(self, input_dim, factor_order=10, **kwargs):
        self.input_dim = input_dim
        self.factor_order = factor_order
        super(CrossLayer, self).__init__(**kwargs)

    def build(self, input_shape):
        self.kernel = self.add_weight(name='kernel', 
                                      shape=(self.input_dim, self.factor_order),
                                      initializer='glorot_uniform',
                                      trainable=True)
        super(CrossLayer, self).build(input_shape)

    def call(self, X):
        square_of_sum = K.pow(K.dot(X,self.kernel), 2)
        sum_of_square = K.dot(K.pow(X,2), K.pow(self.kernel,2))
        return 0.5*K.sum(square_of_sum - sum_of_square, axis=1, keepdims=True)

    def compute_output_shape(self, input_shape):
        return (input_shape[0], 1)
        
class FM():
    def __init__(self, input_dim=None, factor_order=2, output_dim=1, epochs=10, batch_size=256):
        self.input_dim = input_dim
        self.output_dim = output_dim
        self.factor_order = factor_order
        self.epochs = epochs
        self.batch_size = batch_size

    def build_model(self,):
        X = Input(shape=(self.input_dim,))
        linear_term = Dense(units=1)(X)
        cross_term = CrossLayer(self.input_dim, self.factor_order)(X)
        logits = Add()([linear_term, cross_term])
        predictions = Activation('sigmoid')(logits)
        self.model = Model(inputs=X, outputs=predictions)
        self.model.compile(loss='binary_crossentropy', optimizer='adam')
        print(self.model.summary())
        return self.model
    
    def fit(self,X_train, y_train):
        self.model.fit(X_train, y_train, epochs=self.epochs, batch_size=self.batch_size)
        return self.model
        
    def predict(self, X_test,batch_size=None):
        if batch_size is None:
            batch_size = self.batch_size
        return self.model.predict(X_test, batch_size=batch_size)
        

if __name__ == "__main__":
    data = load_breast_cancer()
    X_train, X_test, y_train, y_test = train_test_split(data.data, data.target, test_size=0.2,
                                                        random_state=27, stratify=data.target)
    fm = FM(30)
    fm.build_model()
    
    fm.fit(X_train, y_train)
    pred_ans = fm.predict(X_test)
    print('pred_ans',pred_ans)
    print("test AUC", round(roc_auc_score(y_test, pred_ans), 4))

运行结果：

test AUC 0.5

跑出来的结果有点怪，可能数据集不太适合。

---

参考：

1. github fm;
2. FM paper;
3. 推荐系统常用数据集；
4. 浅梦deepFM；
5. keras FM github;
6. (推荐)从FM推演各深度学习CTR预估模型（附代码）；