3 Sublevel Separation and Relative Smoothness — Lemma 2
Given an initial point \(x^0 \in [0, a]^n\) with \(\Psi (x^0) {\lt} +\infty \), define
and for \(\beta \in \mathbb {R}\),
\(\Psi \) is proper, lower semicontinuous on \(\mathcal{E}_n\), and bounded below on \(\mathcal{E}_n\) by \(\inf _{z \in [0, a]^n} R_\theta (z)\).
Nonemptiness of \(\mathrm{dom}(\Psi )\) holds at \((a, \dots , a)\), each scalar KL term is nonnegative and lower semicontinuous on \(\mathbb {R}\) (including \(y_i = 0\) and \(u_i \le 0\)), and \(R_\theta \) is continuous on compact \([0, a]^n\).
For every \(\beta \in \mathbb {R}\), \(K_\beta \) is convex, closed, and compact, and \(L_0(x^0) \subseteq K_{\beta _0}\).
Convexity follows from concavity of \(\log \) on \((0, \infty )\) applied to \((Ax)_i\). Lower semicontinuity of \(\mathrm{KL}(y, A\cdot )\) shows \(K_\beta = [0, a]^n \cap \mathrm{KL}(y, A\cdot )^{-1}((-\infty , \beta ])\) is closed inside compact \([0, a]^n\). Comparing \(\mathrm{KL}(y, Ax) \le \Psi (x)_{\mathbb {R}} - R_\theta (x)\) gives \(L_0(x^0) \subseteq K_{\beta _0}\).
For every \(\beta \in \mathbb {R}\), there exists \(c_\beta {\gt} 0\) such that \((Ax)_i \ge c_\beta \) for all \(x \in K_\beta \) and every \(i\) with \(y_i {\gt} 0\). Consequently, \(K_\beta \subseteq U_+ \cap U_R\) and \(K_\beta \cap \mathcal{S} = \emptyset \).
Nonnegativity of each scalar KL term yields \(d_{\mathrm{KL}}(y_i, (Ax)_i) \le \max (\beta , 0)\) on \(K_\beta \), forcing \((Ax)_i \ge y_i \exp (-(\max (\beta , 0) + y_i)/y_i) {\gt} 0\) whenever \(y_i {\gt} 0\).
For \(\beta \in \mathbb {R}\), define
\(F\) is \(C^2\) on the open set \(U_+ \cap U_R \supseteq K_\beta \), and \(\nabla F\) is \(M_\beta \)-Lipschitz on the compact convex set \(K_\beta \):
Compactness of \(K_\beta \) and continuity of \(D(\nabla F)\) on \(U_+ \cap U_R\) bound the operator norm uniformly by \(M_\beta \) on \(K_\beta \); convexity of \(K_\beta \) allows the mean value theorem along line segments inside \(K_\beta \).
For every \(\beta \in \mathbb {R}\), the function \(L_\beta h_\xi - F\) is convex on \(K_\beta \), and hence for all \(x, u \in K_\beta \),
Along any segment \(z(t) = (1 - t) x + t u \subseteq K_\beta \), the derivative \(\varphi '(t) = \langle L_\beta \nabla h_\xi (z(t)) - \nabla F(z(t)), u - x \rangle \) satisfies \(\varphi '(t_2) - \varphi '(t_1) \ge (L_\beta / (a + \xi )^2 - M_\beta )(t_2 - t_1)\| u - x\| ^2 = 0\) for \(0 \le t_1 \le t_2 \le 1\), hence is monotone. Applying the mean value theorem along \(t \in [0, 1]\) yields the Bregman descent inequality.