A. Properties of the Entropy
Let
possess an absolutely convergent Fourier series and be such that the eigenvalues of
are strictly greater than zero for all
θ. We require that
is the density of a stationary sequence, which means that we must also assume that for all
θ:
This means, in particular, that
is Hermitian. We write:
Here,
is understood to be the positive Hermitian square root of
. The Fourier series of
is absolutely convergent as a consequence of Wiener’s theorem. In this section, we determine a general expression for
I(3) (ξ ○ ∆
−1). We are generalizing the result for
b = 1 with ℤ-indexing given in [
14]. These results are necessary for the proofs in the previous section.
As previously,
is understood to be the positive definite Hermitian square root of
. We note that R−j = †Rj and S−j = †Sj.
Lemma 6.
For all ξ ∈
ε2, ξ ○ ∆
−1 and ξ ○ γ
−1 are in ε2 and: Proof. We make use of the following standard Lemma from [
15], to which the reader is referred for the definition of an orthogonal stochastic measure. Let (
Uj) ∈ ℝ
b be a zero-mean stationary sequence governed by ξ ∈
ε2. Then, there exists an orthogonal ℝ
b-valued stochastic measure
Zξ =
Zξ(∆) (∆ ∈
B([−π, π[
d)), such that for every
j ∈ ℤ
d (ξ a.s.):
Conversely, any orthogonal stochastic measure defines a zero-mean stationary sequence through
(20). It may be inferred from this representation that:
The proof that this is well-defined makes use of the fact that
and
are uniformly continuous, since their Fourier series’ each converge uniformly This gives us the lemma. We note for future reference that, if ξ has spectral measure
, then the spectral density of ξ ○ ∆
−1 is:
It remains for us to determine a specific expression for I(3) (ξ ○ γ−1).
Definition 5. If ξ ∈
ε2, we define:
Otherwise, we define ΓΔ(ξ) = 0.
Proof. We see from
(21) that ξ ○ ∆
−1 has spectral measure
. We thus find that:
Proof. We assume for now that there exists a
q, such that
Sj = 0 for all |
j| ≥
q, denoting the corresponding map by ∆
q. Let
be the following linear operator. For
j ∈
Vm, let
. Let
. It follows from this assumption that the
Vl marginals of
and
are the same, as long as
l ≤
n –
q. Thus:
This last inequality follows from a property of the relative entropy
I(2), namely that it is nondecreasing as we take a ‘finer’
σ-algebra (it is a direct consequence of Lemma 2.3 in [
5]. If
does not have a density for some
n, then
I(3) (
ξ) is infinite and the Lemma is trivial. If otherwise, we may readily evaluate
using a change of variable to find that:
We divide
(24) by (2
l + 1)
d, substitute the above result and, finally, take
l → ∞ (while fixing
n =
l +
q) to find that:
Here,
is equal to Γ
∆(ξ), as defined above, subject to the above assumption that
Sj = 0 for
|j| >
q. On taking
q → ∞, it may be readily seen that
pointwise. Furthermore, the lower semicontinuity of
I(3) dictates that:
which gives us the Lemma. □
Lemma 9. For all ξ ∈ ε2, I(3) (ξ ○ ∆−1) = I(3) (ξ) − Γ∆ (ξ).
Proof. We find, similarly to the previous Lemma, that if
, then:
We substitute γ = ξ ○ ∆
−1 into the above and, after noting Lemma 6, we find that:
The result now follows from the previous Lemma and
(26). □
We next prove some matrix identities, which are needed in the proof of Lemma 11.
Lemma 10.
If A,
B ∈
ℂl×l (
for some positive integer l)
are both Hermitian, then: If A and B are positive and, in addition, A is invertible, then: Proof. The first part of
(27) follows from von Neumann’s trace inequality. Given two matrices
A and
B in ℂ
l×l:
where the
αt’s and
βt’
s are the singular values of
A and
B. In the case where
A and
B are Hermitian, the singular values are the magnitudes of the (real) eigenvalues. By Cauchy-Schwartz,
If
B is positive, so are its eigenvalues and ||
B|| ≤ tr(B), hence
(27). If
A is invertible, tr(B) = tr(A
−2ABA). If, moreover,
A and
B are both Hermitian positive, we obtain the second identity by applying
(27) to the Hermitian matrix
A−2 and the Hermitian positive matrix
ABA. □
Lemma 11.
For all,
For all μ ∈
ε2,
there exist constants α1 < 1,
α2 ∈ ℝ,
α3 > 1,
α4 ∈ ℝ
and, such
that for all,
There exist constants that converge to zero as m → ∞,
such that: Proof. It is a standard result that if
,
then I(3)(
μ) = ∞, for which the first result is evident. Let
. For
, we let
. The function
fM(ω) =
f(ω)1
af(ω)≤m is bounded, and hence, from Definition 1, we have:
We obtain using an easy Gaussian computation that:
Upon taking
M → ∞ and applying the dominated convergence theorem, we obtain:
We have used the stationarity of
μ. By taking the limit
n →
∞, we obtain the first inequality
(29).
It follows from the definition that
(30)–
(33) are true if
μ ∉
ε2. Thus, we may assume that
μ ∉
ε2. We choose
to be such that the eigenvalues of
(as defined in
(8)) are strictly greater than zero for all
. It may be easily verified that:
We observe the following upper and lower bounds, which hold for all
(and for Γ
1, too),
We recall that, since
,
Note that
and apply Lemma 10,
(28) to this, obtaining:
where:
If
, then
(30) is clear, because
I(3)(
μ) ≥ 0; the above inequality would imply that Γ
2,(m) is negative and
(36). Otherwise, we may use
(29) and
(36) to find that:
We may substitute
, letting
, into the above to obtain
(30). The second inequality
(31) follows by taking
m → ∞ in the first.
For the third inequality, we find using
(27) that:
where:
If
then
(32) is clear, because of the fact that
I(3)(
μ) ≥ 0, the above inequality would mean that Γ
2,(m) is positive and
(37). Otherwise, we may use
(29) and
(37) to find that:
This yields
(32) on taking
. Taking limits as
m →
∞ yields
(33).
Now, making use of Lemma 10,
(27), it may be seen that:
Here,
. The convergence of Γ
1,(m) to Γ
1 is clear. We thus obtain
(34).
Lemma 12. I(3)(μ) − Γ(μ) is lower-semicontinuous.
Proof. Since
I(3)(
μ) − Γ(
μ) is infinite for all
μ ∉
ε2, we only need to prove the Lemma for
μ ∉
ε2. We need to prove that if
μ(j) →
μ, then
. We may assume, without loss of generality, that:
If
, then by
(29) in Lemma 11, lim
j→∞ I(3)(
μ(j) ○
β−1) = ∞, satisfying the requirements of the Lemma.
Thus, we may assume that there exists a constant
l, such that
for all
j. We therefore have that, for all
m, because of
(11) and
(4),
Making use of
(32), we may thus conclude that:
for some
, which goes to zero as
m → ∞. In addition,
due to the lower semi-continuity of
. On taking
m → ∞, since
we find that:
B. A Lemma on the Large Deviations of Stationary Random Variables
The following lemma is an adaptation of Theorem 4.9 in [
9] to ℤ
d. We state it in a general context. Let
B be a Banach Space with norm || ⋅ ||. For
j ∈ ℤ
d, let
. We note that
. Define the metric
dλ on
by:
Let the induced Prokhorov metric on
be dλ,M. For
and j ∈ ℤd, we define the shift operator as Sj(ω)k = ωj+k. Let
for some y ∈ A} be the closed blowup and B(δ) be the closed blowup of {0}.
Suppose that for
m ∈ ℤ
+,
Y(m),
are stationary random variables, governed by a probability law ℙ. We suppose that
is governed by
and
is governed by
; these being the empirical process measures, defined analogously by
(2). Suppose that for each
m,
satisfies an LDP with good rate function J
(m). Suppose that
W = Y
(m) +
Z(m) for some series of stationary random variables
Z(m) on
.
Lemma 13.
If there exists a constant κ > 0,
such that for all b > 0:
then satisfies an LDP with good rate function: Proof. It suffices, thanks to Theorem 4.2.16, Exercise 4.2.29 in [
16], to prove that for all
ϵ > 0,
For
, write |
x|
λ:=
dλ(
x, 0). Let
. Then, noting the definition of
pn just above
(2),
Thus:
for an arbitrary b > 0. Since
and the exponential function is convex, by Jensen’s inequality:
by the stationarity of
Z(m) and the fact that
. We may thus infer, using
(38), that:
Since
b is arbitrary, we may take b → ∞ to obtain
(39). □
C. Proof of Corollary 2
We now prove Corollary 2.
Proof. Let
be
ϕ(⍵) =
⍵ +
c and
be
. Let
be
and
be Ψ(
v) =
v ○
ϕ−1. It is easily checked that these maps are bicontinuous bijections for their respective topologies. Since
, we have, by a contraction principle, Theorem 4.2.1 in [
16], that
satisfies a strong LDP with good rate function:
Clearly,
is in ε
2 if and only if
μ is in
ε2. Let
. It is well known that if
is absolutely continuous relative to
, then the relative entropy may be written as:
Otherwise, the relative entropy is infinite. Thus, if the relative entropy is finite,
must possess a density, and this means that
possesses a density, which we denote by
. We note that the density of
is:
Accordingly, we find that:
We divide by (2
n + 1)
d and take
n to infinity to obtain:
If
does not possess a density for some
n, then both sides of the above equation are infinite. It may be verified that the spectral density of
ν is given by:
On substituting this into the expression in Theorem 1, we find that:
We have used the fact that
is symmetric. We thus obtain
(5). This minimum of the rate function remains unique because of the bijectivity of
. □