Title: Model selection for kernel regression using the influence function
Authors: Debruyne, Michiel ×
Hubert, Mia
Suykens, Johan #
Issue Date: Oct-2008
Publisher: MIT Press
Series Title: Journal of Machine Learning Research vol:9 pages:2377-2400
Abstract: Recent results about the robustness of kernel methods involve the analysis of influence functions. By definition the influence function is closely related to leave-one-out criteria. In statistical learning, the latter is often used to assess the generalization of a method. In statistics, the influence function is used in a similar way to analyze the statistical efficiency of a method. Links between both worlds are explored. The influence function is related to the first term of a Taylor expansion. Higher order influence functions are calculated. A recursive relation between these terms is found characterizing the full Taylor expansion. It is shown how to evaluate influence functions at a specific sample distribution to obtain an approximation of the leave-one-out error. A specific implementation is proposed using a L1 loss in the selection of the hyperparameters and a Huber loss in the estimation procedure. The parameter in the Huber loss controlling the degree of robustness is optimized as well. The resulting procedure gives good results, even when outliers are present in the data.
ISSN: 1532-4435
Publication status: published
KU Leuven publication type: IT
Appears in Collections:Leuven Statistics Research Centre (LStat)
Statistics Section
ESAT - STADIUS, Stadius Centre for Dynamical Systems, Signal Processing and Data Analytics
× corresponding author
# (joint) last author

Files in This Item:

There are no files associated with this item.

Request a copy


All items in Lirias are protected by copyright, with all rights reserved.

© Web of science