Concave penalties

This example considers proximal operators of concave separable penalties.

import matplotlib.pyplot as plt
import numpy as np

import pyproximal

plt.close('all')
x = np.linspace(-5, 5, 101)


def compare_penalty_and_proximal_operator(penalty):
    fig, ax = plt.subplots(1, 2)
    ax[0].plot(x, penalty.elementwise(x), label=penalty.__class__.__name__)
    ax[0].plot(x, np.abs(x), 'k--', label='l1')
    ax[0].set_aspect(2)
    ax[0].set_xlim(-5, 5)
    ax[0].set_ylim(0, 5)
    ax[0].set_title('Penalty')
    ax[0].legend(loc='upper center')
    for tau in [0.25, 0.50, 0.75, 1.00]:
        ax[1].plot(x, penalty.prox(x, tau), label=f'tau={tau:.2f}')
    ax[1].plot(x, x, 'k--')
    ax[1].set_aspect('equal', 'box')
    ax[1].set_xlim(-5, 5)
    ax[1].set_ylim(-5, 5)
    ax[1].set_title('Proximal operator')
    ax[1].legend(loc='upper left')

The SCAD penalty combines both soft-thresholding and hard-thresholding in a continuous manner

scad = pyproximal.SCAD(1, 3.7)
compare_penalty_and_proximal_operator(scad)

The Log penalty encourages sparsity more than the l1-penalty and parametrizes a family of functions which lie between l0 and l1-penalties.

log = pyproximal.Log(1, 0.5)
compare_penalty_and_proximal_operator(log)

The ETP penalty is similar to the Log penalty in that it tends to the l1-penalty and the l0-penalty at its extremes.

etp = pyproximal.ETP(1, 0.25)
compare_penalty_and_proximal_operator(etp)

The Geman penalty

geman = pyproximal.Geman(3, 1.2)
compare_penalty_and_proximal_operator(geman)

The quadratic envelope of the l0-penalty

f_mu = pyproximal.QuadraticEnvelopeCard(1.5)
compare_penalty_and_proximal_operator(f_mu)