pyproximal.Log#
- class pyproximal.Log(sigma, gamma=1.3)[source]#
Logarithmic penalty.
The logarithmic penalty (Log) is defined as
\[\mathrm{Log}_{\sigma,\gamma}(\mathbf{x}) = \sum_i \frac{\sigma}{\log(\gamma + 1)}\log(\gamma|x_i| + 1)\]where \({\sigma>0}\), \({\gamma>0}\).
- Parameters
Notes
The logarithmic penalty is an extension of the elastic net family of penalties to non-convex members, which should produce sparser solutions compared to the \(\ell_1\)-penalty [1]. The pyproximal implementation considers a scaled version that satisfies \({\mathrm{Log}_{\sigma,\gamma}(0) = 0}\) and \({\mathrm{Log}_{\sigma,\gamma}(1) = \sigma}\), which is suitable also for penalizing singular values. Note that when \({\gamma\rightarrow 0}\) the logarithmic penalty approaches the l1-penalty and when \({\gamma\rightarrow\infty}\) it mimicks the \(\ell_0\)-penalty.
The proximal operator can be analyzed using the one-dimensional case
\[\prox_{\tau \mathrm{Log}(\cdot)}(x) = \argmin_{z} \mathrm{Log}(z) + \frac{1}{2\tau}(x - z)^2\]where we assume that \(x\geq 0\). The minima can be obtained when \(z=0\) or at a local minimum. Consider therefore
\[f(z) = k \log(\gamma z + 1) + \frac{1}{2} (x - z)^2\]where \(k= \frac{\tau \sigma}{\log(\gamma + 1)}\) is introduced for convenience. The condition that \(f'(z) = 0\) yields the following equation
\[\gamma z^2 + (1-\gamma y) x + k\gamma - y = 0 .\]The discriminant \(\Delta\) is given by
\[\Delta = (1-\gamma y)^2-4\gamma (k\gamma - y) .\]When the discriminant is negative the global optimum is obtained at \(z=0\); otherwise, it is obtained when
\[z = \frac{\gamma x - 1 +\sqrt{\Delta}}{2\gamma} .\]Note that the other stationary point must be a local maximum since \(\gamma>0\) and can therefore be discarded.
- 1
Friedman, J. H. “Fast sparse regression and classification”, International Journal of Forecasting, 28(3):722 – 738, 2012.
Methods
__init__(sigma[, gamma])affine_addition(v)Affine addition
chain(g)Chain
elementwise(x)grad(x)Compute gradient
postcomposition(sigma)Postcomposition
precomposition(a, b)Precomposition
prox(**kwargs)proxdual(**kwargs)
