Kolmogorov-Arnold Networks: Approximation and Learning Guarantees for Functions and their Derivatives

要約

Kolmogorov-Arnoldの重ね合わせ定理に触発されたKolmogorov-Arnold Networks（KANS）は、最近、ほとんどの深い学習フレームワークの改善されたバックボーンとして浮上し、訓練可能なスプラインベースの活性化関数を可能にすることにより、多層知覚（MLP）の前身よりも多くの適応性を約束しました。
このホワイトペーパーでは、$ b^{s} _ {p、q}（\ mathcal {x}）$で$ b^{s} _ {p、q}（\ mathcal {x}）$で$ b^{s} _で最適に近似できることを示すことにより、Kan建築の理論的基礎を調べます。
弱いbesov norm $ b^{\ alpha} _ {p、q}（\ mathcal {x}）$;
ここで、$ \ alpha 要約(オリジナル)

Inspired by the Kolmogorov-Arnold superposition theorem, Kolmogorov-Arnold Networks (KANs) have recently emerged as an improved backbone for most deep learning frameworks, promising more adaptivity than their multilayer perception (MLP) predecessor by allowing for trainable spline-based activation functions. In this paper, we probe the theoretical foundations of the KAN architecture by showing that it can optimally approximate any Besov function in $B^{s}_{p,q}(\mathcal{X})$ on a bounded open, or even fractal, domain $\mathcal{X}$ in $\mathbb{R}^d$ at the optimal approximation rate with respect to any weaker Besov norm $B^{\alpha}_{p,q}(\mathcal{X})$; where $\alpha < s$. We complement our approximation guarantee with a dimension-free estimate on the sample complexity of a residual KAN model when learning a function of Besov regularity from $N$ i.i.d. noiseless samples. Our KAN architecture incorporates contemporary deep learning wisdom by leveraging residual/skip connections between layers.

arxiv情報