% Generated by roxygen2: do not edit by hand
% Please edit documentation in R/optim-lr_scheduler.R
\name{lr_one_cycle}
\alias{lr_one_cycle}
\title{Once cycle learning rate}
\usage{
lr_one_cycle(
  optimizer,
  max_lr,
  total_steps = NULL,
  epochs = NULL,
  steps_per_epoch = NULL,
  pct_start = 0.3,
  anneal_strategy = "cos",
  cycle_momentum = TRUE,
  base_momentum = 0.85,
  max_momentum = 0.95,
  div_factor = 25,
  final_div_factor = 10000,
  last_epoch = -1,
  verbose = FALSE
)
}
\arguments{
\item{optimizer}{(Optimizer): Wrapped optimizer.}

\item{max_lr}{(float or list): Upper learning rate boundaries in the cycle
for each parameter group.}

\item{total_steps}{(int): The total number of steps in the cycle. Note that
if a value is not provided here, then it must be inferred by providing
a value for epochs and steps_per_epoch.
Default: NULL}

\item{epochs}{(int): The number of epochs to train for. This is used along
with steps_per_epoch in order to infer the total number of steps in the cycle
if a value for total_steps is not provided.
Default: NULL}

\item{steps_per_epoch}{(int): The number of steps per epoch to train for. This is
used along with epochs in order to infer the total number of steps in the
cycle if a value for total_steps is not provided.
Default: NULL}

\item{pct_start}{(float): The percentage of the cycle (in number of steps) spent
increasing the learning rate.
Default: 0.3}

\item{anneal_strategy}{(str): \{'cos', 'linear'\}
Specifies the annealing strategy: "cos" for cosine annealing, "linear" for
linear annealing.
Default: 'cos'}

\item{cycle_momentum}{(bool): If \code{TRUE}, momentum is cycled inversely
to learning rate between 'base_momentum' and 'max_momentum'.
Default: TRUE}

\item{base_momentum}{(float or list): Lower momentum boundaries in the cycle
for each parameter group. Note that momentum is cycled inversely
to learning rate; at the peak of a cycle, momentum is
'base_momentum' and learning rate is 'max_lr'.
Default: 0.85}

\item{max_momentum}{(float or list): Upper momentum boundaries in the cycle
for each parameter group. Functionally,
it defines the cycle amplitude (max_momentum - base_momentum).
Note that momentum is cycled inversely
to learning rate; at the start of a cycle, momentum is 'max_momentum'
and learning rate is 'base_lr'
Default: 0.95}

\item{div_factor}{(float): Determines the initial learning rate via
initial_lr = max_lr/div_factor
Default: 25}

\item{final_div_factor}{(float): Determines the minimum learning rate via
min_lr = initial_lr/final_div_factor
Default: 1e4}

\item{last_epoch}{(int): The index of the last batch. This parameter is used when
resuming a training job. Since \code{step()} should be invoked after each
batch instead of after each epoch, this number represents the total
number of \emph{batches} computed, not the total number of epochs computed.
When last_epoch=-1, the schedule is started from the beginning.
Default: -1}

\item{verbose}{(bool): If \code{TRUE}, prints a message to stdout for
each update. Default: \code{FALSE}.}
}
\description{
Sets the learning rate of each parameter group according to the
1cycle learning rate policy. The 1cycle policy anneals the learning
rate from an initial learning rate to some maximum learning rate and then
from that maximum learning rate to some minimum learning rate much lower
than the initial learning rate.
}
\details{
This policy was initially described in the paper
\href{https://arxiv.org/abs/1708.07120}{Super-Convergence: Very Fast Training of Neural Networks Using Large Learning Rates}.

The 1cycle learning rate policy changes the learning rate after every batch.
\code{step} should be called after a batch has been used for training.
This scheduler is not chainable.

Note also that the total number of steps in the cycle can be determined in one
of two ways (listed in order of precedence):
\itemize{
\item A value for total_steps is explicitly provided.
\item A number of epochs (epochs) and a number of steps per epoch
(steps_per_epoch) are provided.
}

In this case, the number of total steps is inferred by
total_steps = epochs * steps_per_epoch

You must either provide a value for total_steps or provide a value for both
epochs and steps_per_epoch.
}
\examples{
if (torch_is_installed()) {
\dontrun{
data_loader <- dataloader(...)
optimizer <- optim_sgd(model$parameters, lr = 0.1, momentum = 0.9)
scheduler <- lr_one_cycle(optimizer,
  max_lr = 0.01, steps_per_epoch = length(data_loader),
  epochs = 10
)

for (i in 1:epochs) {
  coro::loop(for (batch in data_loader) {
    train_batch(...)
    scheduler$step()
  })
}
}

}
}
