单参数线性回归(w)

单参数线性回归(w)

1、导包

import numpy as np
import matplotlib.pyplot as plt

2、收集数据 | f_w(x)

x_train = np.array([1.0, 1.7, 2.0, 2.5, 3.0, 3.2, 1.3, 2.6, 2.4, 2.3])
y_train = np.array([250, 300, 480,  430, 630, 730, 400, 550, 600, 450])
plt.scatter(x_train, y_train, c='r', marker='x')
plt.title("Function of x")
plt.xlabel("x")
plt.ylabel("f_w(x)")
plt.show()

3、compute_cost()

计算各损失值， 使用Squared error cost function：
$$f_{w,b}(x^{(i)})=w{x}^{(i)}+b\text{\hspace{1em}(1)}$$
$$J(w,b)=\frac{1}{2m}\sum _{i=0}^{m-1}(f_{w,b}(x^{(i)})-y^{(i)}{)}^2\text{\hspace{1em}(2)}$$
注意：本文只使用单参数（w）

def compute_cost(x, y, w):
    m = len(x)
    cost = 0
    for i in range(m):
        cost += (w * x[i] - y[i]) ** 2
    total_cost = cost / (2 * m)
    return total_cost

4、Cost function | J(w)

作损失函数的图像 J(w)

list_w = np.linspace(0, 500, 1000)
J_w = np.zeros(len(list_w))
for i in range(len(list_w)):
    J_w[i] = compute_cost(x_train, y_train, list_w[i])
plt.plot(list_w, J_w)
plt.title("Cost function")
plt.xlabel("w")
plt.ylabel("J(w)")
plt.show()

5、compute_gradient()

此函数用于计算损失函数对应梯度

计算方法：
$$\begin{array}{rl}\frac{\partial J(w,b)}{\partial w}& =\frac{1}{m}\sum _{i=0}^{m-1}(f_{w,b}(x^{(i)})-y^{(i)})x^{(i)}\end{array}$$
注意：本文只使用单参数（w）

def compute_gradient(x, y, w):
    m = len(x)
    dj_dw = 0
    for i in range(m):
        dj_dw += (w * x[i] - y[i]) * x[i]
    dj_dw /= m
    return dj_dw

6、gradient_descent()

repeat  until convergence:    {    w = w − α ∂ J ( w , b ) ∂ w    b = b − α ∂ J ( w , b ) ∂ b } \begin{align*} \text{repeat}&\text{ until convergence:} \; \lbrace \newline \; w &= w - \alpha \frac{\partial J(w,b)}{\partial w} \; \newline b &= b - \alpha \frac{\partial J(w,b)}{\partial b} \newline \rbrace \end{align*} repeatwb}​ until convergence:{=w−α∂w∂J(w,b)​=b−α∂b∂J(w,b)​​

def gradient_descent(x, y, w_begin, alpha, num_iters, error, cost_function, gradient_function):
    w = w_begin
    past_w = 0
    w_history = []
    for i in range(num_iters):
        w_history.append(w)
        if abs(w-past_w) < error:
            break
        else:
            past_w = w
            tmp_w = w - alpha * gradient_function(x, y, w)
            w = tmp_w
    return w, w_history

local_minima_w, hist = gradient_descent(x_train, y_train, 10, 0.01, 10000, 1e-5, compute_cost, compute_gradient)
for i in range(len(hist)):
    print(f'The {i}^th of update:', hist[i])
print('The local minima of w is', local_minima_w)

The 0^th of update: 10
The 1^th of update: 20.9172
The 2^th of update: 31.257098464000002
The 3^th of update: 41.050223097223686
The 4^th of update: 50.3254872998425
The 5^th of update: 59.11027553142683
The 6^th of update: 67.43052416132498
The 7^th of update: 75.31079804367411
The 8^th of update: 82.77436304312462
The 9^th of update: 89.8432547254042
The 10^th of update: 96.53834341552482
The 11^th of update: 102.87939581571187
The 12^th of update: 108.88513336497704
The 13^th of update: 114.57328751263705
The 14^th of update: 119.9606520689688
The 15^th of update: 125.06313278756173
The 16^th of update: 129.89579432575547
The 17^th of update: 134.47290472180953
The 18^th of update: 138.80797752012023
The 19^th of update: 142.91381166885628
The 20^th of update: 146.80252930780716
The 21^th of update: 150.48561155801033
The 22^th of update: 153.97393241882276
The 23^th of update: 157.2777908725154
The 24^th of update: 160.4069412911768
The 25^th of update: 163.37062223569936
The 26^th of update: 166.1775837318756
The 27^th of update: 168.836113104134
The 28^th of update: 171.3540594431874
The 29^th of update: 173.73885677983165
The 30^th of update: 175.99754603331417
The 31^th of update: 178.1367957990725
The 32^th of update: 180.16292203721756
The 33^th of update: 182.0819067198895
The 34^th of update: 183.89941549254175
The 35^th of update: 185.62081440129614
The 36^th of update: 187.2511857357556
The 37^th of update: 188.79534303404884
The 38^th of update: 190.25784529440833
The 39^th of update: 191.64301043524
The 40^th of update: 192.9549280434245
The 41^th of update: 194.19747144848822
The 42^th of update: 195.37430915829216
The 43^th of update: 196.48891569000168
The 44^th of update: 197.5445818283144
The 45^th of update: 198.54442434123314
The 46^th of update: 199.49139518206874
The 47^th of update: 200.38829020484096
The 48^th of update: 201.23775741880897
The 49^th of update: 202.04230480650236
The 50^th of update: 202.80430772833452
The 51^th of update: 203.52601593566018
The 52^th of update: 204.20956021298247
The 53^th of update: 204.85695866891996
The 54^th of update: 205.47012269450747
The 55^th of update: 206.0508626064219
The 56^th of update: 206.60089299179432
The 57^th of update: 207.12183777038825
The 58^th of update: 207.6152349890901
The 59^th of update: 208.08254136286703
The 60^th of update: 208.52513657559862
The 61^th of update: 208.94432735348096
The 62^th of update: 209.34135132302887
The 63^th of update: 209.7173806650671
The 64^th of update: 210.07352557549837
The 65^th of update: 210.410837543066
The 66^th of update: 210.73031245378868
The 67^th of update: 211.03289353123233
The 68^th of update: 211.31947412130077
The 69^th of update: 211.59090032976638
The 70^th of update: 211.84797352032834
The 71^th of update: 212.09145268057338
The 72^th of update: 212.32205666282465
The 73^th of update: 212.5404663064945
The 74^th of update: 212.74732644820708
The 75^th of update: 212.9432478256259
The 76^th of update: 213.12880888060678
The 77^th of update: 213.3045574670003
The 78^th of update: 213.47101246814532
The 79^th of update: 213.62866532882978
The 80^th of update: 213.77798150624128
The 81^th of update: 213.91940184419124
The 82^th of update: 214.05334387467042
The 83^th of update: 214.18020305057786
The 84^th of update: 214.3003539132633
The 85^th of update: 214.41415119832993
The 86^th of update: 214.52193088296224
The 87^th of update: 214.62401117787118
The 88^th of update: 214.72069346678535
The 89^th of update: 214.81226319626174
The 90^th of update: 214.89899071844343
The 91^th of update: 214.98113208925213
The 92^th of update: 215.05892982437248
The 93^th of update: 215.13261361525966
The 94^th of update: 215.20240100728472
The 95^th of update: 215.2684980420195
The 96^th of update: 215.33109986555752
The 97^th of update: 215.39039130466685
The 98^th of update: 215.44654741247606
The 99^th of update: 215.49973398530432
The 100^th of update: 215.55010805216142
The 101^th of update: 215.5978183383631
The 102^th of update: 215.64300570463047
The 103^th of update: 215.6858035629696
The 104^th of update: 215.72633827055975
The 105^th of update: 215.76472950281254
The 106^th of update: 215.8010906067038
The 107^th of update: 215.8355289354213
The 108^th of update: 215.86814616531623
The 109^th of update: 215.8990385960943
The 110^th of update: 215.92829743513283
The 111^th of update: 215.95600906676302
The 112^th of update: 215.98225530731258
The 113^th of update: 216.0071136466619
The 114^th of update: 216.03065747702644
The 115^th of update: 216.05295630964127
The 116^th of update: 216.07407597998744
The 117^th of update: 216.09407884216571
The 118^th of update: 216.113023952992
The 119^th of update: 216.13096724635778
The 120^th of update: 216.14796169837038
The 121^th of update: 216.16405748376056
The 122^th of update: 216.1793021240193
The 123^th of update: 216.19374062770117
The 124^th of update: 216.20741562330835
The 125^th of update: 216.22036748514782
The 126^th of update: 216.2326344525332
The 127^th of update: 216.24425274268324
The 128^th of update: 216.25525665765016
The 129^th of update: 216.26567868559363
The 130^th of update: 216.27554959669945
The 131^th of update: 216.28489853402598
The 132^th of update: 216.29375309954668
The 133^th of update: 216.30213943564266
The 134^th of update: 216.31008230228588
The 135^th of update: 216.317605150141
The 136^th of update: 216.32473018980153
The 137^th of update: 216.33147845736482
The 138^th of update: 216.33786987653937
The 139^th of update: 216.34392331746798
The 140^th of update: 216.34965665244027
The 141^th of update: 216.35508680865922
The 142^th of update: 216.36022981821733
The 143^th of update: 216.36510086543
The 144^th of update: 216.36971433166607
The 145^th of update: 216.37408383780758
The 146^th of update: 216.3782222844643
The 147^th of update: 216.38214189006183
The 148^th of update: 216.38585422691537
The 149^th of update: 216.38937025539607
The 150^th of update: 216.39270035629073
The 151^th of update: 216.3958543614501
The 152^th of update: 216.3988415828166
The 153^th of update: 216.40167083991727
The 154^th of update: 216.40435048590246
The 155^th of update: 216.40688843220792
The 156^th of update: 216.40929217191277
The 157^th of update: 216.41156880186202
The 158^th of update: 216.41372504361956
The 159^th of update: 216.41576726331297
The 160^th of update: 216.41770149042898
The 161^th of update: 216.4195334356151
The 162^th of update: 216.4212685075398
The 163^th of update: 216.4229118288611
The 164^th of update: 216.4244682513509
The 165^th of update: 216.42594237021947
The 166^th of update: 216.42733853768226
The 167^th of update: 216.4286608758096
The 168^th of update: 216.4299132886968
The 169^th of update: 216.4310994739905
The 170^th of update: 216.4322229338059
The 171^th of update: 216.43328698506625
The 172^th of update: 216.43429476929595
The 173^th of update: 216.43524926189556
The 174^th of update: 216.43615328092653
The 175^th of update: 216.43700949543114
The 176^th of update: 216.43782043331274
The 177^th of update: 216.43858848879916
The 178^th of update: 216.43931592951145
The 179^th of update: 216.4400049031589
The 180^th of update: 216.44065744387984
The 181^th of update: 216.4412754782475
The 182^th of update: 216.44186083095775
The 183^th of update: 216.4424152302167
The 184^th of update: 216.44294031284286
The 185^th of update: 216.44343762909975
The 186^th of update: 216.44390864727296
The 187^th of update: 216.44435475800518
The 188^th of update: 216.44477727840186
The 189^th of update: 216.44517745591997
The 190^th of update: 216.44555647205092
The 191^th of update: 216.44591544580888
The 192^th of update: 216.4462554370345
The 193^th of update: 216.4465774495241
The 194^th of update: 216.44688243399327
The 195^th of update: 216.44717129088372
The 196^th of update: 216.4474448730218
The 197^th of update: 216.4477039881364
The 198^th of update: 216.44794940124373
The 199^th of update: 216.44818183690597
The 200^th of update: 216.44840198137038
The 201^th of update: 216.44861048459552
The 202^th of update: 216.44880796217012
The 203^th of update: 216.44899499713057
The 204^th of update: 216.4491721416823
The 205^th of update: 216.44933991883016
The 206^th of update: 216.44949882392243
The 207^th of update: 216.44964932611342
The 208^th of update: 216.44979186974854
The 209^th of update: 216.44992687567623
The 210^th of update: 216.45005474249047
The 211^th of update: 216.45017584770756
The 212^th of update: 216.4502905488808
The 213^th of update: 216.45039918465596
The 214^th of update: 216.45050207577137
The 215^th of update: 216.45059952600457
The 216^th of update: 216.45069182306943
The 217^th of update: 216.45077923946553
The 218^th of update: 216.4508620332826
The 219^th of update: 216.4509404489626
The 220^th of update: 216.45101471802147
The 221^th of update: 216.4510850597325
The 222^th of update: 216.45115168177384
The 223^th of update: 216.45121478084164
The 224^th of update: 216.45127454323074
The 225^th of update: 216.4513311453847
The 226^th of update: 216.45138475441675
The 227^th of update: 216.45143552860318
The 228^th of update: 216.45148361785064
The 229^th of update: 216.4515291641387
The 230^th of update: 216.45157230193902
The 231^th of update: 216.45161315861247
The 232^th of update: 216.45165185478504
The 233^th of update: 216.451688504704
The 234^th of update: 216.45172321657526
The 235^th of update: 216.45175609288276
The 236^th of update: 216.4517872306911
The 237^th of update: 216.45181672193218
The 238^th of update: 216.4518446536764
The 239^th of update: 216.45187110839
The 240^th of update: 216.45189616417835
The 241^th of update: 216.4519198950166
The 242^th of update: 216.4519423709681
The 243^th of update: 216.45196365839132
The 244^th of update: 216.45198382013558
The 245^th of update: 216.4520029157268
The 246^th of update: 216.4520210015432
The 247^th of update: 216.45203813098158
The 248^th of update: 216.45205435461529
The 249^th of update: 216.45206972034322
The 250^th of update: 216.45208427353145
The 251^th of update: 216.4520980571471
The 252^th of update: 216.45211111188516
The 253^th of update: 216.45212347628868
The 254^th of update: 216.45213518686253
The 255^th of update: 216.45214627818123
The 256^th of update: 216.452156782991
The 257^th of update: 216.45216673230644
The local minima of w is 216.45216673230644

plt.scatter(x_train, y_train, c='r', marker='x')
plt.plot(x_train, [local_minima_w * x for x in x_train])
plt.title("Function of x")
plt.xlabel("x")
plt.ylabel("f_w(x)")
plt.show()