1. The Large Deviation Principle in Infinite-Dimensional Spaces

When establishing the Large Deviation Principle (LDP) for random vectors in infinite-dimensional spaces, a common strategy is to first establish the LDP for projections onto finite-dimensional subspaces and then lift these results to the original space. This approach is codified in the following theorem.

Theorem 1.1. Suppose that ${𝜇_{𝜀}}$ satisfies a weak LDP with a convex rate function $𝐼$ , and let $𝒳︀$ be a locally convex Hausdorff topological vector space. Assume that for each $𝜆 \in {𝒳︀}^{*}$ , the limits $Λ_{𝜆} (𝑡) = {lim}_{𝜀 \to 0} 𝜀 Λ_{𝜇_{𝜀}} (𝑡 𝜆 / 𝜀)$ exist as extended real numbers, and that $Λ_{𝜆 (𝑡)}$ is a lower semicontinuous function of $𝑡 \in ℝ$ . If for every $𝜆 \in {𝒳︀}^{*}$ and every $𝑎 \in ℝ$ ,

inf_{𝑥 : ⟨ 𝜆, 𝑥 ⟩ > 𝑎} 𝐼 (𝑥) \leq inf_{𝑧 > 𝑎} Λ_{𝜆}^{*} (𝑧),

then

𝐼 = Λ^{*}

, and consequently,

Λ^{*}

controls a weak LDP associated with

{𝜇_{𝜀}}

The ingredients for the proof are summarized in the following subsections.

1.1. Varadhan’s integral lemma

In this subsection we assume $𝒳︀$ to be a regular topological space.

Theorem 1.2 (Varadhan). Suppose that ${𝜇_{𝜀}}$ satisfies the LDP with a good rate function $𝐼 : 𝒳︀ \to [0, \infty]$ , and let $𝜑 : 𝒳︀ \to ℝ$ be any continuous function. Assume further that either the tail condition

lim_{𝑀 \to \infty} \underset{𝜀 \to 0}{lim sup} 𝜀 log 𝔼 [𝑒^{\frac{𝜑 (𝑍_{𝜀})}{𝜀}} 𝟙_{{𝜑 (𝑍_{𝜀}) \geq 𝑀}}] = - \infty

or the following moment condition for some $𝛾 > 1$ holds:

\underset{𝜀 \to 0}{lim sup} 𝜀 log 𝔼 [𝑒^{\frac{𝛾 𝜑 (𝑍_{𝜀})}{𝜀}}] < \infty .

Then,

lim_{𝜀 \to 0} 𝜀 log 𝔼 [𝑒^{\frac{𝜑 (𝑍_{𝜀})}{𝜀}}] = sup_{𝑥 \in 𝒳︀} [𝜑 (𝑥) - 𝐼 (𝑥)] .

The theorem is a consequence of the following three lemmas.

Lemma 1.3. If $𝜑 : 𝒳︀ \to ℝ$ is lower semicontinuous and the large deviations lower bound holds with $𝐼 : 𝒳︀ \to [0, \infty]$ , then

\underset{𝜀 \to 0}{lim inf} 𝜀 log 𝔼 [exp (\frac{𝜑 (𝑍_{𝜀})}{𝜀})] \geq sup_{𝑥 \in 𝒳︀} [𝜑 (𝑥) - 𝐼 (𝑥)] .

Lemma 1.4. If $𝜑 : 𝒳︀ \to ℝ$ is an upper semicontinuous function for which the tail condition (1.1) holds, and the large deviations upper bound holds with the good rate function $𝐼 : 𝒳︀ \to [0, \infty]$ , then

\underset{𝜀 \to 0}{lim sup} 𝜀 log 𝔼 [exp (\frac{𝜑 (𝑍_{𝜀})}{𝜀})] \leq sup_{𝑥 \in 𝒳︀} [𝜑 (𝑥) - 𝐼 (𝑥)] .

Lemma 1.5. Condition (1.2) implies the tail condition (1.1).

Proof of Lemma 1.3. Fix $𝑥 \in 𝒳︀$ . Since $𝜑$ is lower semicontinuous, for every $𝛿 > 0$ , there exists an open neighborhood $𝐺$ of $𝑥$ such that

- 𝜑 (𝑥) \geq - inf_{𝑦 \in 𝐺} 𝜑 (𝑦) - 𝛿 .

Using Markov’s inequality and the lower bound of the LDP, we have

\begin{aligned} \underset{𝜀 \to 0}{lim inf} 𝜀 log 𝔼 (𝑒^{\frac{𝜑 (𝑍_{𝜀})}{𝜀}}) \geq \underset{𝜀 \to 0}{lim inf} 𝜀 log 𝔼 (𝑒^{\frac{𝜑 (𝑍_{𝜀})}{𝜀}} 𝟙_{𝑍_{𝜀} \in 𝐺}) \\ \geq & inf_{𝑦 \in 𝐺} 𝜑 (𝑦) + \underset{𝜀 \to 0}{lim inf} 𝜀 log 𝜇_{𝜀} (𝐺) \\ \geq & inf_{𝑦 \in 𝐺} 𝜑 (𝑦) - inf_{𝑦 \in 𝐺} 𝐼 (𝑦) \geq 𝜑 (𝑥) - 𝐼 (𝑥) - 𝛿, \end{aligned}

which proves the inequality since

𝑥

and

𝛿

are arbitrary.

□

Proof of Lemma 1.4. Note that by the semicontinuity of $𝜑$ and $𝐼$ , and the regularity of $𝒳︀$ , for each $𝑥 \in 𝒳︀$ and each $𝛿 > 0$ , there exists an open neighborhood $𝐴_{𝑥}$ such that

inf_{𝑦 \in \overset{─}{𝐴_{𝑥}}} 𝐼 (𝑦) \geq 𝐼 (𝑥) - 𝛿 and sup_{𝑦 \in 𝐴_{𝑥}} 𝜑 (𝑦) \leq 𝜑 (𝑥) + 𝛿,

which yields the local estimate $𝔼 (𝑒^{\frac{𝜑 (𝑍_{𝜀})}{𝜀}} 𝟙_{{𝑍_{𝜀} \in 𝐴_{𝑥}}}) \leq 𝑒^{\frac{𝜑 (𝑥) + 𝛿}{𝜀}} 𝜇_{𝜀} (𝐴_{𝑥})$ . For each $𝛼 < \infty$ , let ${𝐴_{𝑥_{𝑖}}}_{𝑖 = 1}^{𝑁}$ be a finite cover of the compact set $Ψ_{𝐼} (𝛼)$ . Then,

\begin{aligned} \underset{𝜀 \to 0}{lim sup} 𝜀 log 𝔼 [𝑒^{\frac{𝜑 (𝑍_{𝜀})}{𝜀}} 𝟙_{{𝜑 (𝑍_{𝜀}) \leq 𝑀}}] \\ \leq & \underset{𝜀 \to 0}{lim sup} max {𝑀 - 𝛼, max_{1 \leq 𝑖 \leq 𝑁} [𝜑 (𝑥_{𝑖}) + 𝛿 + 𝜀 log 𝜇_{𝜀} (𝐴_{𝑥_{𝑖}})]} \\ \leq & max {𝑀 - 𝛼, max_{1 \leq 𝑖 \leq 𝑁} [𝜑 (𝑥_{𝑖}) + 𝛿 - inf_{𝑥 \in {\overset{─}{𝐴}}_{𝑥_{𝑖}}} 𝐼 (𝑥)]} \\ \leq & max {𝑀 - 𝛼, max_{𝑥 \in Ψ_{𝐼} (𝛼)} [𝜑 (𝑥) - 𝐼 (𝑥) + 2 𝛿]} . \end{aligned}

By letting $𝛿 \to 0$ and then $𝛼 \to \infty$ , we obtain

\underset{𝜀 \to 0}{lim sup} 𝜀 log 𝔼 [𝑒^{\frac{𝜑 (𝑍_{𝜀})}{𝜀}} 𝟙_{{𝜑 (𝑍_{𝜀}) \leq 𝑀}}] \leq sup_{𝑥 \in 𝒳︀} [𝜑 (𝑥) - 𝐼 (𝑥)] .

The proof is completed by using assumption (1.1), which implies

\begin{aligned} \underset{𝜀 \to 0}{lim sup} 𝜀 log 𝔼 (𝑒^{\frac{𝜑 (𝑍_{𝜀})}{𝜀}}) \\ = & lim_{𝑀 \to \infty} \underset{𝜀 \to 0}{lim sup} max {𝜀 log 𝔼 [𝑒^{\frac{𝜑 (𝑍_{𝜀})}{𝜀}} 𝟙_{{𝜑 (𝑍_{𝜀}) \leq 𝑀}}], 𝜀 log 𝔼 [𝑒^{\frac{𝜑 (𝑍_{𝜀})}{𝜀}} 𝟙_{{𝜑 (𝑍_{𝜀}) > 𝑀}}]} \\ = & \underset{𝜀 \to 0}{lim sup} 𝜀 log 𝔼 [𝑒^{\frac{𝜑 (𝑍_{𝜀})}{𝜀}} 𝟙_{{𝜑 (𝑍_{𝜀}) \leq 𝑀}}] . \end{aligned}

□

Proof of Lemma 1.5. The lemma follows from the observation that for any non-negative random variable $𝑋$ and any $𝛾 > 1$ ,

𝔼 [𝑋^{𝛾} 𝟙_{{𝑋 \geq 𝛼}}] = 𝛼^{𝛾} ℙ (𝑋 \geq 𝛼) + \int_{𝛼}^{\infty} 𝑥^{𝛾 - 1} ℙ (𝑋 \geq 𝑥) 𝑑 𝑥 .

Hence,

𝔼 [𝑋^{𝛾} 𝟙_{{𝑋 \geq 𝛼}}] \geq 𝛼^{1 - 𝛾} 𝔼 [𝑋 𝟙_{{𝑋 \geq 𝛼}}]

, which proves the lemma upon setting

𝑋 = \frac{𝜑 (𝑍_{𝜀})}{𝜀}

and

𝛼 = \frac{𝑀}{𝜀}

□

1.2. Fenchel-Legendre transform

Theorem 1.6. Let $𝒳︀$ be a locally convex Hausdorff topological vector space. Assume that $𝜇_{𝜀}$ satisfies the LDP with a good rate function $𝐼$ . Suppose in addition that

\overset{─}{Λ} (𝜆) = \underset{𝜀 \to 0}{lim sup} 𝜀 Λ_{𝜇_{𝜀}} (\frac{𝜆}{𝜀}), \forall 𝜆 \in {𝒳︀}^{*} .

For each $𝜆 \in {𝒳︀}^{*}$ , ${lim}_{𝜀 \to 0} 𝜀 Λ_{𝜇_{𝜀}} (\frac{𝜆}{𝜀})$ exists, is finite, and
$Λ (𝜆) = sup_{𝑥 \in 𝒳︀} [⟨ 𝜆, 𝑥 ⟩ - 𝐼 (𝑥)] .$
If $𝐼$ is convex, then it is the Fenchel–Legendre transform of $Λ$ , namely,
$𝐼 (𝑥) = Λ^{*} (𝑥) ≔ sup_{𝜆 \in {𝒳︀}^{*}} [⟨ 𝜆, 𝑥 ⟩ - Λ (𝑥)] .$
If $𝐼$ is not convex, then $Λ^{*}$ is the affine regularization of $𝐼$ , i.e.,
$Λ^{*} = sup {𝑓 : 𝑓 \leq 𝐼, 𝑓 is convex} .$

Proof. (1) Fix $𝜆 \in {𝒳︀}^{*}$ and $𝛾 > 1$ . By assumption, Varadhan’s lemma (Theorem 1.2) asserts that

lim_{𝜀 \to 0} 𝜀 Λ_{𝜇_{𝜀}} (\frac{𝜆}{𝜀}) = sup_{𝑥 \in 𝒳︀} [⟨ 𝜆, 𝑥 ⟩ - 𝐼 (𝑥)] < \infty .

Since $Λ$ is convex and $Λ (0) = 0$ , it follows that $Λ > - \infty$ .

(2) This follows from the duality lemma (Lemma D.1).

(3) From part (1), we have

Λ^{*} (𝑥) = 𝐼^{* *} (𝑥) = sup_{𝜆 \in {𝒳︀}^{*}} inf_{𝑦 \in 𝒳︀} [⟨ 𝜆, 𝑥 - 𝑦 ⟩ + 𝐼 (𝑦)] \leq 𝐼 (𝑥) .

By the Hahn-Banach separation theorem,

sup {𝑓 : 𝑓 \leq 𝐼, 𝑓 is convex} = sup {𝜆 + 𝑐 : 𝜆 + 𝑐 \leq 𝐼, 𝜆 \in {𝒳︀}^{*}, 𝑐 \in ℝ} .

If $𝐼 \geq 𝜆 + 𝑐$ , it is necessarily the case that ${inf}_{𝑦 \in 𝒳︀} [𝐼 (𝑦) - ⟨ 𝜆, 𝑦 ⟩] \geq 𝑐$ , or equivalently,

inf_{𝑦 \in 𝒳︀} [⟨ 𝜆, 𝑥 - 𝑦 ⟩ + 𝐼 (𝑦)] \geq ⟨ 𝜆, 𝑥 ⟩ + 𝑐,

from which the claim follows. $□$

1.3. Proof of the theorem

Proof of Theorem 1.1. By assumption,

\begin{aligned} sup_{𝑥 \in 𝒳︀} [⟨ 𝜆, 𝑥 ⟩ - 𝐼 (𝑥)] & \geq sup_{𝑎 \in ℝ} sup_{𝑥 : ⟨ 𝜆, 𝑥 ⟩ > 𝑎} [⟨ 𝜆, 𝑥 ⟩ - 𝐼 (𝑥)] \\ \geq sup_{𝑎 \in ℝ} [𝑎 - inf_{𝑥 : ⟨ 𝜆, 𝑥 ⟩ > 𝑎} 𝐼 (𝑥)] \\ \geq sup_{𝑎 \in ℝ} [𝑎 - inf_{𝑧 > 𝑎} Λ_{𝜆}^{*} (𝑧)] = sup_{𝑧 \in ℝ} [𝑧 - Λ_{𝜆}^{*} (𝑧)] . \end{aligned}

Note that $Λ_{𝜆}$ is convex with $Λ_{𝜆 (0)} = 0$ and is lower semicontinuous; therefore, it cannot attain the value $- \infty$ . By the duality lemma (Lemma D.1), ${sup}_{𝑧 \in ℝ} [𝑧 - Λ_{𝜆}^{*} (𝑧)] = Λ_{𝜆} (1) = Λ (𝜆)$ , proving

sup_{𝑥 \in 𝒳︀} {⟨ 𝜆, 𝑥 ⟩ - 𝐼 (𝑥)} \geq Λ (𝜆) .

By Lemma 1.3, we have

Λ (𝜆) \geq sup_{𝑥 \in 𝒳︀} [⟨ 𝜆, 𝑥 ⟩ - 𝐼 (𝑥)] .

Combining the above results gives

𝐼^{*} = Λ

. Applying the duality lemma (Lemma D.1) again yields

Λ^{*} = 𝐼^{* *} = 𝐼

□

From finite to infinite dimension

1. The Large Deviation Principle in Infinite-Dimensional Spaces

1.1. Varadhan’s integral lemma

1.2. Fenchel-Legendre transform

1.3. Proof of the theorem