Skip to content
\[ \newcommand\R[]{\mathbb R} \]

Outline

  • Strongly Convex Functions
    • Definition of The Strongly Convex and The Class \(S_\mu^1 (\R^n)\)
    • Property of Strongly Convex Function
    • Equivalent Definitions
    • Examples
  • Smooth and Strongly Convex Functions
    • The Class \(S_{\mu, L}^{1, 1} (\R^n)\)
    • Property of Smooth and Strongly Convex Function
  • Conclusion

Strongly Convex Functions

Definition of The Strongly Convex and The Class \(S_\mu^1 (\R^n)\)

如果 \(f(x)\) 连续可微，且 \(\exists \mu > 0, \forall x, y \in \R^n\):

\[ f(y) \geq f(x) + \langle \nabla f(x), y-x \rangle + \frac 1 2 \mu \|y-x\|^2 \]

那么，就称 \(f(x)\) strongly convex, where \(\mu\) is the convexity parameter of \(f\).

性质

Note: 也就是在最优解附近，函数的变化率是足够大的。

Note: 也就是，强凸系数是线性叠加的。

Proof (Cont.)

从而，\(\nabla \phi(x) = \nabla f(y) |_{y=x} - \nabla f(x) = 0\)。因此，由 Theorem 30:

$$ \begin{aligned} \phi(x) &= \min_\nu \phi(\nu) \geq \min_\nu \phi(y) + \langle \nabla f(y) , \nu - y \rangle + \frac 1 2 \mu | \nu - y |^2 \newline &= \phi(y) - \frac 1 {2\mu} | \nabla \phi(y) |^2 \end{aligned} $$ 展开，得到：

\[ f(x) + \langle \nabla f(x), y - x \rangle + \frac 1 {2\mu} \| \nabla f(x) - \nabla f(y) \|^2 \geq f(y) \]

进一步，互换 \(x, y\)：

\[ f(y) + \langle \nabla f(y), x - y \rangle + \frac 1 {2\mu} \| \nabla f(y) - \nabla f(x) \|^2 \geq f(x) \]

然后两边相加：

$$ \frac 1 {\mu} | \nabla f(x) - \nabla f(y) |^2 \geq \langle \nabla f(x) - \nabla f(y), x - y \rangle $$ \(\blacksquare\)

Note: (25) 就是割线不等式

Equivalent Definitions

上面两个都是等价定义（证明略）。

\(S_\mu^2 (\R^n) \subseteq S_\mu^1 (\R^n)\)

如果 strongly convex function 同时还可以求二阶导（Hessian matrix），那么我们就有更加简单的判定方式：

\[ \nabla^2 f(x) \succ \mu I_n \]