用python怎么做数据统计-用Python做统计分析 （Scipy.stats的文档）

Shape Parameters

形态参数

While a general continuous random variable can be shifted and scaled with the loc and scale parameters, some distributions require additional shape parameters. For instance, the gamma distribution, with density

γ(x,a)=λ(λx)a?1Γ(a)e?λx,

requires the shape parameter a. Observe that setting λ can be obtained by setting the scale keyword to 1/λ.

虽然一个一般的连续随机变量可以被位移和伸缩通过loc和scale参数，但一些分布还需要额外的形态参数。作为例子，看到这个伽马分布，这是它的密度函数

γ(x,a)=λ(λx)a?1Γ(a)e?λx,

要求一个形态参数a。注意到λ的设置可以通过设置scale关键字为1/λ进行。

Let’s check the number and name of the shape parameters of the gamma distribution. (We know from the above that this should be 1.)

让我们检查伽马分布的形态参数的名字的数量。（我们知道从上面知道其应该为1）

>>>

>>> from scipy.stats import gamma

>>> gamma.numargs

1

>>> gamma.shapes

'a'

Now we set the value of the shape variable to 1 to obtain the exponential distribution, so that we compare easily whether we get the results we expect.

现在我们设置形态变量的值为1以变成指数分布。所以我们可以容易的比较是否得到了我们所期望的结果。

>>>

>>> gamma(1, scale=2.).stats(moments="mv")

(array(2.0), array(4.0))

Notice that we can also specify shape parameters as keywords:

注意我们也可以以关键字的方式指定形态参数：

>>>

>>> gamma(a=1, scale=2.).stats(moments="mv")

(array(2.0), array(4.0))

Freezing a Distribution

冻结分布

Passing the loc and scale keywords time and again can become quite bothersome. The concept of freezing a RV is used to solve such problems.

不断地传递loc与scale关键字最终会让人厌烦。而冻结RV的概念被用来解决这个问题。

>>>

>>> rv = gamma(1, scale=2.)

By using rv we no longer have to include the scale or the shape parameters anymore. Thus, distributions can be used in one of two ways, either by passing all distribution parameters to each method call (such as we did earlier) or by freezing the parameters for the instance of the distribution. Let us check this:

通过使用rv我们不用再更多的包含scale与形态参数在任何情况下。显然，分布可以被多种方式使用，我们可以通过传递所有分布参数给对方法的每次调用（像我们之前做的那样）或者可以对一个分布对象冻结参数。让我们看看是怎么回事：

>>>

>>> rv.mean(), rv.std()

(2.0, 2.0)

This is indeed what we should get.

这正是我们应该得到的。

Broadcasting

广播

The basic methods pdf and so on satisfy the usual numpy broadcasting rules. For example, we can calculate the critical values for the upper tail of the t distribution for different probabilites and degrees of freedom.

像pdf这样的简单方法满足numpy的广播规则。作为例子，我们可以计算t分布的右尾分布的临界值对于不同的概率值以及自由度。

>>>

>>> stats.t.isf([0.1, 0.05, 0.01], [[10], [11]])

array([[ 1.37218364, 1.81246112, 2.76376946],

[ 1.36343032, 1.79588482, 2.71807918]])

Here, the first row are the critical values for 10 degrees of freedom and the second row for 11 degrees of freedom (d.o.f.). Thus, the broadcasting rules give the same result of calling isf twice:

这里，第一行是以10自由度的临界值，而第二行是以11为自由度的临界值。所以，广播规则与下面调用了两次isf产生的结果相同。

>>>

>>> stats.t.isf([0.1, 0.05, 0.01], 10)

array([ 1.37218364, 1.81246112, 2.76376946])

>>> stats.t.isf([0.1, 0.05, 0.01], 11)

array([ 1.36343032, 1.79588482, 2.71807918])

If the array with probabilities, i.e, [0.1, 0.05, 0.01] and the array of degrees of freedom i.e., [10, 11, 12], have the same array shape, then element wise matching is used. As an example, we can obtain the 10% tail for 10 d.o.f., the 5% tail for 11 d.o.f. and the 1% tail for 12 d.o.f. by calling

但是如果概率数组，如[0.1,0.05,0.01]与自由度数组,如[10,11,12]具有相同的数组形态，则元素对应捕捉被作用，我们可以分别得到10%，5%，1%尾的临界值对于10，11,12的自由度。

>>>

>>> stats.t.isf([0.1, 0.05, 0.01], [10, 11, 12])

array([ 1.37218364, 1.79588482, 2.68099799])

Specific Points for Discrete Distributions

离散分布的特殊之处

Discrete distribution have mostly the same basic methods as the continuous distributions. However pdf is replaced the probability mass function pmf, no estimation methods, such as fit, are available, and scale is not a valid keyword parameter. The location parameter, keyword loc can still be used to shift the distribution.

离散分布的简单方法大多数与连续分布很类似。当然像pdf被更换为密度函数pmf，没有估计方法，像fit是可用的。而scale不是一个合法的关键字参数。Location参数，关键字loc则仍然可以使用用于位移。

The computation of the cdf requires some extra attention. In the case of continuous distribution the cumulative distribution function is in most standard cases strictly monotonic increasing in the bounds (a,b) and has therefore a unique inverse. The cdf of a discrete distribution, however, is a step function, hence the inverse cdf, i.e., the percent point function, requires a different definition:

ppf(q) = min{x : cdf(x) >= q, x integer}

Cdf的计算要求一些额外的关注。在连续分布的情况下，累积分布函数在大多数标准情况下是严格递增的，所以有唯一的逆。而cdf在离散分布，无论如何，是阶跃函数，所以cdf的逆，分位点函数，要求一个不同的定义：

ppf(q) = min{x : cdf(x) >= q, x integer}

For further info, see the docs here.

为了更多信息可以看这里。

We can look at the hypergeometric distribution as an example

>>>

>>> from scipy.stats import hypergeom

>>> [M, n, N] = [20, 7, 12]

我们可以看这个超几何分布的例子

>>>

>>> from scipy.stats import hypergeom

>>> [M, n, N] = [20, 7, 12]

If we use the cdf at some integer points and then evaluate the ppf at those cdf values, we get the initial integers back, for example

如果我们使用在一些整数点使用cdf，它们的cdf值再作用ppf会回到开始的值。

>>>

>>> x = np.arange(4)*2

>>> x

array([0, 2, 4, 6])

>>> prb = hypergeom.cdf(x, M, n, N)

>>> prb

array([ 0.0001031991744066, 0.0521155830753351, 0.6083591331269301,

0.9897832817337386])

>>> hypergeom.ppf(prb, M, n, N)

array([ 0., 2., 4., 6.])

If we use values that are not at the kinks of the cdf step function, we get the next higher integer back:

如果我们使用的值不是cdf的函数值，则我们得到一个更高的值。

>>>

>>> hypergeom.ppf(prb + 1e-8, M, n, N)

array([ 1., 3., 5., 7.])

>>> hypergeom.ppf(prb - 1e-8, M, n, N)

array([ 0., 2., 4., 6.])

Fitting Distributions

分布拟合

The main additional methods of the not frozen distribution are related to the estimation of distribution parameters:

非冻结分布的参数估计的主要方法：

fit: maximum likelihood estimation of distribution parameters, including location

and scale

fit：分布参数的极大似然估计，包括location与scale

fit_loc_scale: estimation of location and scale when shape parameters are given

fit_loc_scale:估计location与scale当形态参数给定时

nnlf: negative log likelihood function

nnlf:负对数似然函数

expect: Calculate the expectation of a function against the pdf or pmf

expect:计算函数pdf或pmf的期望值。

Performance Issues and Cautionary Remarks

性能问题与注意事项

The performance of the individual methods, in terms of speed, varies widely by distribution and method. The results of a method are obtained in one of two ways: either by explicit calculation, or by a generic algorithm that is independent of the specific distribution.

每个方法的性能与运行速度表现差异极大根据分布的不同。方法的结果可以由两种方式获得，精确的计算以及独立于各分布的通用算法。

Explicit calculation, on the one hand, requires that the method is directly specified for the given distribution, either through analytic formulas or through special functions in scipy.special or numpy.random for rvs. These are usually relatively fast calculations.

精确计算，一个分布能使用这种方式的第一种情况，这个分布是包中直接给你的（如标准正态分布），第二，给出解析形式，第三通过scipy.special或numpy.special或numpy.random的rvs特殊函数给出。一般使用精确计算会比较快。

The generic methods, on the other hand, are used if the distribution does not specify any explicit calculation. To define a distribution, only one of pdf or cdf is necessary; all other methods can be derived using numeric integration and root finding. However, these indirect methods can be very slow. As an example, rgh = stats.gausshyper.rvs(0.5, 2, 2, 2, size=100)creates random variables in a very indirect way and takes about 19 seconds for 100 random variables on my computer, while one million random variables from the standard normal or from the t distribution take just above one second.

另一方面，通用方法被用于当分布没有被指派明确的计算方法时使用。为了定义一个分布，只有pdf或cdf是必须的；通用方法使用数值积分和求根法得到结果。作为例子，rgh = stats.gausshyper.rvs(0.5, 2, 2, 2, size=100)以间接方式创建了100个随机变量（抽了100个值），这在我的电脑上花了19秒（译者：我花了3.5秒），对比取一百万个标准正态分布的值只需要1秒。

Remaining Issues

遗留问题

The distributions in scipy.stats have recently been corrected and improved and gained a considerable test suite, however a few issues remain:

scipy.stats里的分布最近进行了升级并且被仔细的检查过了，不过仍有一些问题存在。

?the distributions have been tested over some range of parameters, however in some corner ranges, a few incorrect results may remain.

?the maximum likelihood estimation in fit does not work with default starting parameters for all distributions and the user needs to supply good starting parameters. Also, for some distribution using a maximum likelihood estimator might inherently not be the best choice.

?分布在很多参数区间上的值被测试过了，然而在一些奇葩的边界，仍然可能有错误的值存在。

?fit的极大似然估计以默认值作为初始参数将不会工作的很好，用户必须指派合适的初始参数。并且，对于一些分布使用极大似然估计本质上就不是一个好的选择。

Building Specific Distributions

构建具体的分布

The next examples shows how to build your own distributions. Further examples show the usage of the distributions and some statistical tests.

下一个例子展示了如何建立你自己的分布。更多的例子见分布用法以及统计检验

Making a Continuous Distribution, i.e., Subclassing rv_continuous

创建一个连续分布，继承rv_continuous类

Making continuous distributions is fairly simple.

创建连续分布是非常简单的。

>>>

>>> from scipy import stats

>>> class deterministic_gen(stats.rv_continuous):

... def _cdf(self, x):

... return np.where(x < 0, 0., 1.)

... def _stats(self):

... return 0., 0., 0., 0.

>>>

>>> deterministic = deterministic_gen(name="deterministic")

>>> deterministic.cdf(np.arange(-3, 3, 0.5))

array([ 0., 0., 0., 0., 0., 0., 1., 1., 1., 1., 1., 1.])

Interestingly, the pdf is now computed automatically:

令人高兴的是，pdf现在也能被自动计算出来：

>>>

>>> deterministic.pdf(np.arange(-3, 3, 0.5))

array([ 0.00000000e+00, 0.00000000e+00, 0.00000000e+00,

0.00000000e+00, 0.00000000e+00, 0.00000000e+00,

5.83333333e+04, 4.16333634e-12, 4.16333634e-12,

4.16333634e-12, 4.16333634e-12, 4.16333634e-12])

Be aware of the performance issues mentions in Performance Issues and Cautionary Remarks. The computation of unspecified common methods can become very slow, since only general methods are called which, by their very nature, cannot use any specific information about the distribution. Thus, as a cautionary example:

注意这种用法的性能问题，参见"性能问题与注意事项”一节。这种缺乏信息的通用计算可能非常慢。而且看下面这个准确性的例子：

>>>

>>> from scipy.integrate import quad

>>> quad(deterministic.pdf, -1e-1, 1e-1)

(4.163336342344337e-13, 0.0)

But this is not correct: the integral over this pdf should be 1. Let’s make the integration interval smaller:

但这并不是对pdf积分的正确的结果，实际上结果应为1.让我们将积分变得更小一些。

>>>

>>> quad(deterministic.pdf, -1e-3, 1e-3) # warning removed

(1.000076872229173, 0.0010625571718182458)

This looks better. However, the problem originated from the fact that the pdf is not specified in the class definition of the deterministic distribution.

这样看上去好多了，然而，问题本身来源于pdf不是来自给定类的定义。