Extended LBFGS¶
-
class
optim.lbfgs_modified.
LBFGS_MOD
(params, lr=1, max_iter=20, max_eval=None, tolerance_grad=1e-07, tolerance_change=1e-09, history_size=100, line_search_fn=None, line_search_eps=0.0001)[source]¶ Extends the original steepest gradient descent of PyTorch [torch.optim.LBFGS] with optional backtracking linesearch. The linesearch implementation is adapted from scipy [scipy.optimize.linesearch] and relies only on the value of the loss function, not derivatives.
- Parameters
params (iterable) – iterable of parameters to optimize or dicts defining parameter groups
lr (float, optional) – learning rate (default: 1)
max_iter (int) – maximal number of iterations per optimization step (default: 20)
max_eval (int) – maximal number of function evaluations per optimization step (default: max_iter * 1.25).
tolerance_grad (float) – termination tolerance on first order optimality (default: 1e-5).
tolerance_change (float) – termination tolerance on function value/parameter changes (default: 1e-9).
history_size (int) – update history size (default: 100).
line_search_fn (str, optional) – line search algorithm. Supported options
"strong_wolfe"
,"backtracking"
(default: None)line_search_eps (float, optional) – minimal step size (default: 1.0e-4)