0. 先导知识

核函数和再生核希尔伯特空间；

0.1 Mercer Kernel Function:

可交换、半正定的函数：

$$ \forall f, \int\int f(x)K(x,y)f(y) dxdy >= 0 $$

0.2 再生核希尔伯特空间

A function can be viewd as an infinite vector; and a fucntion with two params can be viewd as an infinite matrix;

1) Hilbert Space :

a complete inner product space→ has inner product, norm and distence; and complete.

2) Reproducing Kernel Hilbert Space

evalue function: $\delta_x$; $\delta_xf=f(x)$;

In general, the evalue function is not continuous in Hilbert Space;

But it’s continuous in RKHS!

Continuous: $\|f_n - f\|\rightarrow0, as\ n\rightarrow \infty$, then we have $\delta_x f_n$ convergence to $\delta_x f$

粗略地说，evalue funciton连续意味着如果有两个函数f和g在RKHS ||f - g||很小，则f和g两个函数应该逐点接近。|f(x) - g(x)|对于所有x，应该接近于0。

3) Construct a RKHS from Kernel funciton

We have a kernel function $K(\cdot, \cdot): \mathcal{X}\times \mathcal{X}\rightarrow \mathcal{R}$, then we can fix one param and get a set of function bases, so we create a one-param-function’s space as below:

$$ \mathcal{H_0} = span\{K_{x_i}(\cdot), \forall x_i \in \mathcal{X}\} $$