贝叶斯优化（一）：获得函数1

贝叶斯各车厂通过前期的学习，可以将贝叶斯优化过程分成两个部分，第一部分是利用高斯过程建立代理模型，第二部分是如何选择下一个点？

常见的获得函数有三种

原文链接：https://blog.csdn.net/weixin_39753819/article/details/138416209

• 期望改善（Expected Improvement，EI）： 衡量当前最优解的期望改善程度，选择使期望改善最大的采样点作为下一个候选点。
• 置信区间优化（Confidence Bound，CB）： 基于后验分布的置信区间来选择采样点，通常选择置信区间上界最大的点作为下一个候选点。
• 概率改善（Probability of Improvement，PI）： 衡量当前最优解的概率改善程度，选择使概率改善最大的采样点作为下一个候选点。

通过代码来依次学习

一、父类 

所有的代价函数都是继承的该代码

class AcquisitionFunction(abc.ABC):
    """Base class for acquisition functions.

    Parameters
    ----------
    random_state : int, RandomState, default None
        Set the random state for reproducibility.
    """

    def __init__(self, random_state: int | RandomState | None = None) -> None:
        if random_state is not None:
            if isinstance(random_state, RandomState):
                self.random_state = random_state
            else:
                self.random_state = RandomState(random_state)
        else:
            self.random_state = RandomState()
        self.i = 0

    @abc.abstractmethod
    def base_acq(self, *args: Any, **kwargs: Any) -> NDArray[Float]:
        """Provide access to the base acquisition function."""

    def _fit_gp(self, gp: GaussianProcessRegressor, target_space: TargetSpace) -> None:
        # Sklearn's GP throws a large number of warnings at times, but
        # we don't really need to see them here.
        with warnings.catch_warnings():
            warnings.simplefilter("ignore")
            gp.fit(target_space.params, target_space.target)
            if target_space.constraint is not None:
                target_space.constraint.fit(target_space.params, target_space._constraint_values)

    def suggest(
        self,
        gp: GaussianProcessRegressor,
        target_space: TargetSpace,
        n_random: int = 10_000,
        n_l_bfgs_b: int = 10,
        fit_gp: bool = True,
    ) -> NDArray[Float]:
        """Suggest a promising point to probe next.

        Parameters
        ----------
        gp : GaussianProcessRegressor
            A fitted Gaussian Process.

        target_space : TargetSpace
            The target space to probe.

        n_random : int, default 10_000
            Number of random samples to use.

        n_l_bfgs_b : int, default 10
            Number of starting points for the L-BFGS-B optimizer.

        fit_gp : bool, default True
            Whether to fit the Gaussian Process to the target space.
            Set to False if the GP is already fitted.

        Returns
        -------
        np.ndarray
            Suggested point to probe next.
        """
        if len(target_space) == 0:
            msg = (
                "Cannot suggest a point without previous samples. Use "
                " target_space.random_sample() to generate a point and "
                " target_space.probe(*) to evaluate it."
            )
            raise TargetSpaceEmptyError(msg)
        self.i += 1
        if fit_gp:
            self._fit_gp(gp=gp, target_space=target_space)

        acq = self._get_acq(gp=gp, constraint=target_space.constraint)
        return self._acq_min(acq, target_space.bounds, n_random=n_random, n_l_bfgs_b=n_l_bfgs_b)

    def _get_acq(
        self, gp: GaussianProcessRegressor, constraint: ConstraintModel | None = None
    ) -> Callable[[NDArray[Float]], NDArray[Float]]:
        """Prepare the acquisition function for minimization.

        Transforms a base_acq Callable, which takes `mean` and `std` as
        input, into an acquisition function that only requires an array of
        parameters.
        Handles GP predictions and constraints.

        Parameters
        ----------
        gp : GaussianProcessRegressor
            A fitted Gaussian Process.

        constraint : ConstraintModel, default None
            A fitted constraint model, if constraints are present and the
            acquisition function supports them.

        Returns
        -------
        Callable
            Function to minimize.
        """
        dim = gp.X_train_.shape[1]
        if constraint is not None:

            def acq(x: NDArray[Float]) -> NDArray[Float]:
                x = x.reshape(-1, dim)
                with warnings.catch_warnings():
                    warnings.simplefilter("ignore")
                    mean: NDArray[Float]
                    std: NDArray[Float]
                    p_constraints: NDArray[Float]
                    mean, std = gp.predict(x, return_std=True)
                    p_constraints = constraint.predict(x)
                return -1 * self.base_acq(mean, std) * p_constraints
        else:

            def acq(x: NDArray[Float]) -> NDArray[Float]:
                x = x.reshape(-1, dim)
                with warnings.catch_warnings():
                    warnings.simplefilter("ignore")
                    mean: NDArray[Float]