假设检验实例（python）

• 数据集下载地址：Journal of Statistics Education - Data Archive

         用ctrl+F键搜索normtemp下载txt文件

• 数据集描述：http://ww2.amstat.org/publications/jse/datasets/normtemp.txt

         包括130条记录，我们主要利用体温和性别来进行实验

import pandas as pd
import pylab
import math
import numpy as np
import matplotlib.pyplot as plt
%matplotlib inline
import numpy as np
from scipy.stats import norm
import scipy.stats
import warnings
warnings.filterwarnings("ignore")


df = pd.read_csv('normtemp.txt',sep='   ',names = ['Temperature','Gender','Heart Rate'])
df.describe()

                    

 体温的分布是正态的吗？

observed_temperatures = df['Temperature'].sort_values()#将参数导入
#按从小到大的顺序进行
bin_val = np.arange(start= observed_temperatures.min(), stop= observed_temperatures.max(), step = .05)
mu, std = np.mean(observed_temperatures), np.std(observed_temperatures)#均值和方差


p = norm.pdf(observed_temperatures, mu, std)#画出pdf


plt.hist(observed_temperatures,bins = bin_val, density=True, stacked=True)#直方图
plt.plot(observed_temperatures, p, color = 'red')
plt.xticks(np.arange(95.75,101.25,0.25),rotation=90)
plt.xlabel('Human Body Temperature Distributions')
plt.xlabel('human body temperature')
plt.show()


print('Average (Mu): '+ str(mu) + ' / ' 'Standard Deviation: '+str(std))

                       

Average (Mu): 98.24923076923076 / Standard Deviation: 0.7303577789050376

 从图中可以看出该分布符合正态分布的样子。

再次进行检验：

x = observed_temperatures

#Shapiro-Wilk Test: https://en.wikipedia.org/wiki/Shapiro%E2%80%93Wilk_test
shapiro_test, shapiro_p = scipy.stats.shapiro(x)
print("Shapiro-Wilk Stat:",shapiro_test, " Shapiro-Wilk p-Value:", shapiro_p)

k2, p = scipy.stats.normaltest(observed_temperatures)
print('p:',p)


#Another method to determining normality is through Quantile-Quantile Plots.
scipy.stats.probplot(observed_temperatures, dist="norm", plot=pylab)
pylab.show()

 算出p值为0.2587479863488212

Shapiro-Wilk Stat: 0.9865769743919373  Shapiro-Wilk p-Value: 0.2331680953502655
p: 0.2587479863488212

                              

 画出ECDF：

def ecdf(data):
    #Compute ECDF
    n = len(data)
    x = np.sort(data)
    y = np.arange(1, n+1) / n
    return x, y

# Compute empirical mean and standard deviation

# Number of samples
n = len(df['Temperature']) 

# Sample mean
mu = np.mean(df['Temperature']) 

# Sample standard deviation
std = np.std(df['Temperature']) 

print('Mean temperature: ', mu, 'with standard deviation of +/-', std)

#Random sampling of the data based off of the mean of the data.
normalized_sample = np.random.normal(mu, std, size=10000)
x_temperature, y_temperature = ecdf(df['Temperature'])
normalized_x, normalized_y = ecdf(normalized_sample)

# Plot the ECDFs
fig = plt.figure(figsize=(8, 5))
plt.plot(normalized_x, normalized_y)
plt.plot(x_temperature, y_temperature, marker='.', linestyle='none')
plt.ylabel('ECDF')
plt.xlabel('Temperature')
plt.legend(('Normal Distribution', 'Sample data'))

               

Mean temperature:  98.24923076923076 with standard deviation of +/- 0.730357778905038

可以看出上下波动的值很低。

有学者提出98.6是人类的平均体温，我们该这样认为吗？

在这里我们选择t检验，因为我们只能计算样本的标准差。

from scipy import stats

CW_mu = 98.6
stats.ttest_1samp(df['Temperature'], CW_mu, axis=0)

Ttest_1sampResult(statistic=-5.4548232923640771, pvalue=2.4106320415610081e-07)

T-Stat -5.454 p-value近乎0了. 我们该拒绝这样的假设

 男性和女性的体温有明显差异吗？

两独立样本t检验 H0: 没有明显差异 H1: 有明显差异

female_temp = df.Temperature[df.Gender == 2]
male_temp = df.Temperature[df.Gender == 1]
mean_female_temp = np.mean(female_temp)
mean_male_temp = np.mean(male_temp)
print('Average female body temperature = ' + str(mean_female_temp))
print('Average male body temperature = ' + str(mean_male_temp))

# Compute independent t-test 
stats.ttest_ind(female_temp, male_temp, axis=0)

Average female body temperature = 98.39384615384616
Average male body temperature = 98.1046153846154
Ttest_indResult(statistic=2.2854345381654984, pvalue=0.02393188312240236)

由于P值=0.024 < 0.05，我们需要拒绝原假设，我们有%95的自信认为是有差异的！