Learning Landmark-Based Ensembles with Random Fourier Features and Gradient Boosting

Léo Gautheron 1 Pascal Germain 2 Amaury Habrard 1 Emilie Morvant 1 Marc Sebban 1 Valentina Zantedeschi 1
2 MODAL - MOdel for Data Analysis and Learning
Inria Lille - Nord Europe, LPP - Laboratoire Paul Painlevé - UMR 8524, CERIM - Santé publique : épidémiologie et qualité des soins-EA 2694, Polytech Lille - École polytechnique universitaire de Lille, Université de Lille, Sciences et Technologies
Abstract : We propose a Gradient Boosting algorithm for learning an ensemble of kernel functions adapted to the task at hand. Unlike state-of-the-art Multiple Kernel Learning techniques that make use of a pre-computed dictionary of kernel functions to select from, at each iteration we fit a kernel by approximating it as a weighted sum of Random Fourier Features (RFF) and by optimizing their barycenter. This allows us to obtain a more versatile method, easier to setup and likely to have better performance. Our study builds on a recent result showing one can learn a kernel from RFF by computing the minimum of a PAC-Bayesian bound on the kernel alignment generalization loss, which is obtained efficiently from a closed-form solution. We conduct an experimental analysis to highlight the advantages of our method w.r.t. both Boosting-based and kernel-learning state-of-the-art methods.
Document type :
Preprints, Working Papers, ...
Complete list of metadatas

https://hal.archives-ouvertes.fr/hal-02148618
Contributor : Léo Gautheron <>
Submitted on : Friday, June 14, 2019 - 1:57:54 PM
Last modification on : Sunday, June 16, 2019 - 1:19:05 AM

Files

hal.pdf
Files produced by the author(s)

Identifiers

  • HAL Id : hal-02148618, version 1
  • ARXIV : 1906.06203

Citation

Léo Gautheron, Pascal Germain, Amaury Habrard, Emilie Morvant, Marc Sebban, et al.. Learning Landmark-Based Ensembles with Random Fourier Features and Gradient Boosting. 2019. ⟨hal-02148618⟩

Share

Metrics

Record views

43

Files downloads

150