FTRL

Experimental one-update FTRL-Proximal classifier

from _api_doc_utils import *

1 Where it fits

Group: Regression

FTRL currently exposes one full-batch FTRL-Proximal update for binary prediction. It does not expose persistent online state, partial_fit, epochs, mini-batches, an intercept, or an iterative convergence criterion.

This distinction is material: the current class is an experimental update wrapper, not a conventional fitted batch classifier and not yet a usable streaming estimator. Any nonzero integer label is treated as true.

2 Implemented update

The fit does not minimize a batch loss to convergence. The delegated implementation initializes \(n_{0j}=0\) and draws \(z_{0j}\) uniformly on \([0,1)\) from a fixed seed. For state \((z_j,n_j)\), it defines

\[ w_j(z_j,n_j)= \begin{cases} 0, & |z_j|\leq\lambda_1,\\[3pt] \dfrac{\operatorname{sign}(z_j)\lambda_1-z_j} {(\sqrt{n_j}+\beta)/\alpha+\lambda_2}, & |z_j|>\lambda_1, \end{cases} \]

where the constructor arguments named L1 ratio and L2 ratio are used directly as \(\lambda_1\) and \(\lambda_2\) rather than as fractions of a common penalty. At the defaults, \(\lambda_1=1\) makes all initial weights zero because \(z_{0j}<1\).

The wrapper then computes one probability vector and one aggregate batch gradient,

\[ p_i=\operatorname{logit}^{-1}(x_i'w_0), \qquad g_j=\sum_i x_{ij}(p_i-y_i), \]

followed by

\[ \sigma_j = \frac{\sqrt{n_{0j}+g_j^2}-\sqrt{n_{0j}}}{\alpha}, \qquad z_{1j}=z_{0j}+g_j-\sigma_jw_{0j}, \qquad n_{1j}=n_{0j}+g_j^2. \]

The reported coefficient is \(w_j(z_{1j},n_{1j})\). No second gradient is evaluated. With an L1 threshold below one, the random initial \(z_0\) can produce nonzero \(w_0\) and directly affect this sole update.

3 Inference and API status

There is no analytic inference. The pairs bootstrap repeats the same random-initialized, one-update procedure on resampled rows; it is not a substitute for a converged estimator’s sampling distribution. There is also no intercept, label validation beyond mapping zero to false and every nonzero integer to true, or access to the updated \((z,n)\) state from Python.

For a defensible online estimator, the public API would need persistent state and repeated calls that feed new observations or mini-batches. For a defensible batch estimator, it would need epochs or a stopping rule tied to a stated empirical objective. Until one of those designs is implemented, this class should be treated as experimental rather than as a production FTRL estimator.

4 Performance and numerical behavior

The single update is \(O(np)\) time with \(O(p)\) state, which is cheap because it performs only one gradient evaluation. The dense input is nevertheless retained so bootstrap can resample it. The delegated sigmoid clips its linear index to \([-35,35]\), and probabilities are passed through a single-precision probability wrapper before being returned as double-precision values. Feature scaling matters directly to the one gradient and adaptive accumulator. The low cost should not be interpreted as convergence speed.

5 Python API

Constructor: cm.FTRL

Call FTRL(alpha, beta, l1_ratio, l2_ratio), then fit(x, y_int32). predict(x) returns probabilities in [0, 1]; summary() returns the post-update coefficient vector and marks analytic inference unavailable for this experimental estimator.

print(inspect.signature(cm.FTRL))

(alpha=0.1, beta=1.0, l1_ratio=1.0, l2_ratio=1.0)

cls = cm.FTRL
display(HTML(html_table(["Public method"], public_methods(cls))))

Public method
bootstrap(self, /, n_bootstrap, seed=None)
fit(self, /, x, y)
predict(self, /, x)
summary(self, /)

6 Minimal example

rng = np.random.default_rng(8)
x = rng.normal(size=(220, 4))
p = 1 / (1 + np.exp(-(x @ np.array([0.7, -0.4, 0.2, 0.3]))))
y = rng.binomial(1, p, size=220).astype(np.int32)
model = cm.FTRL()
model.fit(x, y)
print(model.summary()['coef'])
print(model.predict(x[:5]))

[ 0.09565982 -0.09368399  0.08623364  0.07773544]
[0.45369145 0.44605938 0.50578946 0.48307955 0.441618  ]

7 summary() contract

The table below is generated by fitting the live class in this repository and then inspecting summary(). Shapes are shown because most values are plain NumPy arrays or scalars.

rng = np.random.default_rng(108)
x = rng.normal(size=(100, 4))
y = rng.binomial(1, 0.5, size=100).astype(np.int32)
model = cm.FTRL()
model.fit(x, y)
summary = model.summary()
display(HTML(html_table(["summary() key", "shape"], summary_shape_rows(summary))))

summary() key	shape
coef	(4,)
alpha	()
beta	()
l1_ratio	()
l2_ratio	()
inference_available	()
coef_se	()