We derive a measurement-independent asymptotic continuity bound on the observational entropy for general positive operator valued measures measurements, making essential use of its property of bounded concavity. The same insight is used to obtain continuity bounds for other entropic quantities, including the measured relative entropy distance to a convex set of states under a general set of measurements. As a special case, we define and study conditional observational entropy, which is an observational entropy in one (measured) subsystem conditioned on the quantum state in another (unmeasured) subsystem. We also study continuity of relative entropy with respect to a jointly applied channel, finding that observational entropy is uniformly continuous as a function of the measurement. But we show by means of an example that this continuity under measurements cannot have the form of a concrete asymptotic bound.
I. INTRODUCTION
Observational entropy has recently emerged (in fact, re-emerged, cf. Refs. 1–4) in studies of non-equilibrium statistical mechanics as a useful unifying framework to describe coarse-grained entropy in classical and quantum systems,5–7 with applications across thermodynamics and quantum information theory.8–27
It is evident from the definition that observational entropy is a continuous function of the state ρ. For practical purposes, however, it is most useful to realize this continuity statement in terms of concrete continuity bounds, as has also been done for many other entropic quantities.28–32
Though apparently useful insofar as it implies |ΔS| → 0 as ϵ → 0 for any particular M, the bound (5) fails to be universal, depending on the measurement M both through the number of outcomes |M| and through the volume terms Vi. Moreover, it is unacceptably loose, as evidenced by the following pathology. Given any POVM M, one can define another one M′, with twice as many outcomes, by splitting each POVM element in half and counting it twice (Mi → two copies Mi/2). Letting M → M′ the observational entropies SM(ρ) and SM(σ) are both invariant, but by repeating this transformation the bound (5) can be made arbitrarily loose (noting that after Vi ≤ 1 each iteration separately loosens both terms). This pathology highlights the need for an improved and universal bound.
One expects the shortcomings above to be common to any bound attempting to treat the Shannon and Boltzmann terms separately. On the other hand, one might hope that the presence of normalizing volume terms in the definition (1) may actually help improve the bound compared to a Shannon entropy rather than [as occurs in (5)] making it looser. We will show this is indeed the case, improving the bound by treating the terms together.
The main bound (6) appears in Theorem 6. In order to establish it, we first will observe that observational entropy has the same bounded concavity property typical of other entropic quantities, that is, it is concave but not too concave. We then establish a general continuity bound based only on bounded concavity, obtaining the main result as a corollary. The method of proof follows closely the Proof of Lemma 2 of Ref. 32, which obtained a closely related tight bound on quantum conditional entropy, and which in turn is closely related to methods used in Refs. 36 and 37, this method sometimes being known as the Alicki–Fannes–Winter (AFW) trick.32,36 The general bound Proposition 5 given here is closely related to Lemma 7 of Ref. 32, and provides a slightly different generalization thereof. Continuity bounds of this form are often referred to as asymptotic continuity due to the bound per qudit scaling as ϵ log d in the n-copy regime.38,39
In the following we continue to consider a finite dimensional Hilbert space of dimension d unless otherwise specified. Quantum states ρ are positive semidefinite Hermitian operators normalized to unit trace. Measurements are described by POVMs (positive operator valued measures), defined as collections of positive semidefinite Hermitian operators summing to the identity, assumed here to have a finite number of outcomes. The von Neumann entropy is denoted by S(ρ) = −tr(ρ log ρ), and Shannon entropy by H(p) = −∑ipi log pi. The binary Shannon entropy function is denoted by h(x), and g(x) is as above. Observational entropy SM(ρ) is defined by (1).
II. BOUNDED CONCAVITY
An elementary property of Shannon entropy is bounded concavity41 (in the case of classical Shannon entropy this property can be derived directly from a simple chain rule computation, but it can also be inferred as a subcase of the more commonly emphasized quantum generalization), which we state here as a lemma for later use.
Bounded concavity of observational entropy follows from the bounded concavity of Shannon entropy.
In general bounded concavity for real functions on convex sets may be defined as follows.
In the next section we derive a continuity bound for functions of quantum states that applies to any function with a bounded concavity or convexity property of this form, with observational entropy continuity as a corollary.
III. CONTINUITY
The continuity bound proved below is derived by appealing to bounded concavity. Similar methods have been used to derive continuity results for other entropic quantities in the literature, such as in Refs. 32, 36–39, and 42–46. The following decomposition is essential to performing the main trick of the proof, sometimes referred to as the Alicki-Fannes-Winter trick (cf. Refs. 32 and 36). Whereas the previous section only involved convex structure, the derivation of the following decomposition also involves topology via the trace norm. The resulting continuity bound depends on both the convex and topological structures of the space of quantum states.
Next appears a general continuity bound for any function Z(ρ) obeying bounded concavity on quantum states. Since (25) is invariant under Z → −Z this implies the same bound for functions obeying bounded convexity.
In the following sections this general bound is evaluated for the observational entropy and extended to a form that includes several related quantities.
IV. OBSERVATIONAL ENTROPY
The general statement Proposition 5 implies a continuity bound on observational entropy.
By Lemma 2, SM obeys bounded concavity, so the bound follows from Proposition 5. An elementary property of observational entropy is that 0 ≤ SM(ρ) ≤ log d.10 Thus we have κ = sup(μ,ν)|SM(μ) − SM(ν)| = supμSM(μ) = log d.□
The main result, a measurement-independent continuity bound on observational entropy, has therefore been established.
As an aside we note that bounds on both the Shannon and von Neumann entropies can be derived from the observational entropy continuity, as shown in the following corollary. The bounds thus obtained are looser than the known optimized forms.30
We derive these as corollaries of Theorem 6. For the quantum case suppose (wlg) that S(ρ) ≥ S(σ). Let Mσ measure in the σ eigenbasis. Then . For the classical case, let ρ = ∑ipi|i⟩⟨i| and σ = ∑iqi|i⟩⟨i| in N-dimensional Hilbert space, and M0 measure in the |i⟩⟨i| basis. (At least N dimensions are needed to embed p, q classically.) Since the states are diagonal we have . Then measuring M0 obtains pi, qi and we have .□
V. MEASURED RELATIVE ENTROPY
The general Proposition 5 likewise provides a continuity bound on measured relative entropies associated with a particular measurement, as well as on the quantum relative entropy.
If contains an element of full rank the maximum variation κ is necessarily finite. If and χ is a set of states containing the maximally mixed state, then κ ≤ log d.
If contains an element of full rank then all and therefore κ is finite. Meanwhile if and χ ∋ σ is a set of states, then , so if also then κ ≤ supρS(ρ) ≤ log d.□
A crucial step is the following minimax lemma.
Operationally motivated classes of transformations often have the required property automatically, such as LOCC, SEP, PPT channels in composite systems.
For classes of measurements closed under disjoint convex combination, and compact convex sets χ of states, we have—with the help of the minimax lemma—the following continuity theorem for measured relative entropies .
We remark that many commonly invoked measurement classes (such as LOCC, SEP, PPT measurements in composite systems) automatically fulfill the required closure property. Furthermore, given some primitive set of measurements not already doing so, from an operational perspective it is perfectly feasible to complete this to an that is closed in this way, given the capability to choose which measurements to perform based on classical randomness.
This provides asymptotic continuity of the entropies of restricted measurement first considered by Piani.40 Similar results were also obtained recently in independent work, in the context of local classes of measurements, and will appear elsewhere.50
The main limitation of the Theorem 11 is the requirement that χ be convex. One may also be interested in measured relative entropy distances to non-convex sets of states such as coherent or classically correlated states. In this regard the theorem can be compared to results of Ref. 42, which obtained a looser bound but allows χ to be any set star-shaped around the maximally mixed state. Obtaining a bound for star-shaped χ with an optimal dimensional factor remains a useful future direction.
VI. CONDITIONAL OBSERVATIONAL ENTROPY
We have the following bound. Note that the dimensional factor of log d arising from maximum variation is reduced by a factor of two from the corresponding bound on quantum conditional entropy .32
To determine κ it suffices to obtain the bound for all ρ. From non-negativity of the relative entropy, the upper bound follows immediately. To establish the lower bound we will make use of the following observations:
With ΦM a measurement channel as in (3), (ΦM ⊗ I)ρ = ∑j|j⟩⟨j| ⊗ pj ρj yields a classical-quantum state, with pj = tr(MjρA) the induced probability distribution over M outcomes, and some conditional states.
For any two such classical-quantum states, .
With pj = tr(MjρA) and qj = tr(Mj)/dA, one has SM(ρA) = log dA − D(pj‖qj) the marginal observational entropy in system A.
With pj, ρj as in (i), one has ∑jpjρj = ρB since .
- is concave in ρ.We thus have(56)(57)The first equality is by definition, the second uses (i) and (ii), and the third uses (iii) and (iv). Finally, we note that for pure states |ψAB⟩ we have that S(ρA) = S(ρB) and therefore that . Thus for pure states we have . But by concavity this non-negativity extends to convex combinations and so in general it holds that .(58)
To complete the proof, note that since , the same bounds hold for the infimized relative entropy in (55). Thus the maximum variation κ in Theorem 8 is given by κ = supμ,ν|Z(μ) − Z(ν)| ≤ supμZ(μ) ≤ log dA.
It remains to justify the variational formula (55). Applying the same methods used to obtain (58) earlier, (55) can be seen to become . But it follows immediately from D(ρB‖ωB) ≥ 0 that tr(−ρB log ωB) ≥tr(−ρB log ρB) for any ωB, establishing that ωB = ρB saturates the infimum.□
VII. CONTINUITY WITH RESPECT TO MEASUREMENTS
The preceding sections have demonstrated concrete uniform continuity bounds on observational entropy SM(ρ) (and related quantities) under variations in the state ρ. An equally important question is that of continuity under changes in the measurement M. With a suitable norm in the space of measurements (namely the diamond norm between measuring channels), can similar bounds be obtained on |SM(ρ) − SM′(ρ)| for a fixed state?
We will show that although the observational entropy SM(ρ) is indeed a continuous function of M, no simple concrete bounds of the type obtained earlier can hold. A strong limitation is exhibited by the following example.
Observe that measuring channels are convex linear in the measurement, namely Φ(1−λ)M+λN = (1 − λ)ΦM + λΦN. Thus the diamond norm distance between channels implementing Mλ and M is bounded by due to the universal bounds on trace distances between states.34 This provides a strong sense in which Mλ and M are close together for continuity purposes, and ensures in particular that the probability distributions (outcome statistics) induced by Mλ, M on any state differ by no more than λ in the total variational distance (half of the ℓ1 norm).
We make the following observations {with and d ≥ 2 and λ ∈ [0, 1/2]}:
Any λ > 0 implies Sλ > 0. For fixed d, as λ → 0 also Sλ → 0, consistent with continuity.
S1/2 = log d, so (63) shows that the naive concavity bound H(λ) that held for states [cf. (18)] can be violated by a maximum amount.
For fixed λ > 0, one can achieve Sλ/log d arbitrarily close to 1 by taking d sufficiently large. This shows that there is no continuous f(λ) such that f(0) = 0 and Sλ ≤ f(λ) log d.
Moreover, one might have hoped to approach the measurement continuity problem similarly to the method used for states, by attempting some form of an AFW trick (modified for channels) based on bounded concavity. However, the example shows that not only does the naive concavity bound by H(λ) fail, but also (likewise with the continuity above) that no concavity bound of the form f(λ) log d is possible.
Due to these limitations, at present we settle for the following more abstract statement of continuity over measurements. We give here a general statement containing observational entropy as a special case.
Let ρ, σ be states such that D(ρ‖σ) < ∞. Then is a uniformly continuous function of the quantum channel Φ.
We are in finite dimension, so the quantum channels form a finite-dimensional convex set and all norms induce the same topology. For concreteness we use the diamond norm. To show continuity we have to show that for any Φ and any δ > 0, there exists an ϵ > 0 such that implies |F(Φ) − F(Φ′)| < δ.
To begin, for any channel Φ define Φs = (1 − s)Φ + sτ, which mixes Φ with noise, letting prepare maximally mixed state. Observe that F(Φ) is convex in Φ and is finite and non-negative, and that F(τ) = 0. It follows that F(Φ) ≥ (1 − s)F(Φ) ≥ F(Φs). Therefore for any Φ one obtains |F(Φ) − F(Φs)| ≤ sF(Φ).
Additionally, for any Φ, the states Φs(ρ) and Φs(σ) are positive definite. The set of positive definite matrices is an open set in the 2-norm matrix topology (the usual Euclidean topology on matrix elements). Over the domain , the function D(μ‖ν) is continuous, by the elementary reasoning that it is a composition of matrix multiplication, linear combination, trace, and log, which are all continuous functions on the positive set . Thus for any δs > 0 there exists an ϵs > 0 such that if both and , then . This ϵs must be chose sufficiently small so that both ϵs-balls lie entirely in .
Meanwhile, for channels Φ, Φ′ it holds . And therefore for any state ν we have .
Finally, choose a δ > 0. Since we assumed D(ρ‖σ) < ∞, we can choose s > 0 sufficiently small so that 2sD(ρ‖σ) < δ. Next choose any δs > 0 sufficiently small so that also 2sD(ρ‖σ) + δs < δ. There exists an ϵs > 0 as above so that implies and therefore also that |F(Φ) − F(Φ′)| < δ. This completes the proof of continuity. The finite dimensional quantum channels form a compact set, thus continuity of F implies it is also uniformly continuous. One can also argue directly (but with more technical difficulty) for uniform continuity by fixing s = δ/4D(ρ‖σ) and using uniform continuity of log on the domain [s/d, ∞).□
Observational entropy is determined by D(Φ(ρ)‖Φ(σ)) in the case and Φ a measuring channel. Since always, this implies that (in the topology induced by the diamond norm on ΦM) observational entropy SM(ρ) is a continuous function of the measurement. In this form the continuity is useful only for measurements on the same outcome set.
Let ρ, σ be states such that D(ρ‖σ) < ∞. Then F(M) = DM(ρ‖σ) is a continuous function of the measurement M in the simulation distance topology. This also implies continuity of F0(M) = SM(ρ) as a particular case. The continuity is uniform (at least for d < ∞).
Fix δ > 0 and let ϵ = ϵ(δ) as supplied by Theorem 14. Choose any c with ϵ > c > 0, and let ϵ′ = (ϵ − c)/4 > 0. Suppose γ(M, N) < ϵ′. From the definition of γ, there exist Λ, Λ′ such that ‖ΦΛM − ΦN‖⋄ < ϵ and ‖ΦΛ′N − ΦM‖⋄ < ϵ. Thus |DΛM − DN| < δ and |DΛ′N − DM| < δ. We then have DM ≥ DΛM > DN − δ and also DM < DΛ′N + δ ≤ DN + δ, which follow using relative entropy monotonicity, and thus |DM − DN| < δ. This completes the proof.
Put more simply, we stated that for small γ(M, N) we have DN ≥ DΛ′N ≈ DM ≥ DΛM ≈ DN, which implies DM ≈ DN, where ≥ are by monotonicity and ≈ are by the earlier continuity.□
VIII. CONCLUDING REMARKS
The main Theorem 6 described a continuity bound (33) on observational entropy that is universal, in the sense of being independent of the POVM M defining the coarse-graining, and with a dimensional factor log d of the Hilbert space dimension.
This bound improves upon the naive bound (5) insofar as it scales only with log d independently of the number of measurement outcomes |M|. This reflects the intuitive status of observational entropy as a classical entropy in quantum systems, compared, for example to the Shannon entropy of an observable. The improved universality does come with a small trade-off, as when M is a projective measurement in a complete orthonormal basis, the naive bound (5) is slightly stronger. We note that (33) can only be relevant when ρ, σ have overlapping support. In case the supports are orthogonal, the trace distance ϵ = 1 is maximized, and (33) is weaker than the trivial |ΔSM| ≤ log d. This demonstrates that (33) cannot be tight in general—for fixed ϵ, there is not generally a choice of ρ, σ, M for which the inequality is saturated.
Meanwhile, in the case of continuity with respect to measurements, it was seen that although observational entropy is continuous, explicit bounds on the convergence are difficult (if not impossible) to obtain. Other notable limitations on the present results were the necessity of a convex reference set for measured relative entropy continuity (as discussed earlier), and the restriction throughout the paper to finite dimensional spaces. Extending our results to the infinite dimensional case encounters difficulties associated with the possibility of infinite Boltzmann and von Neumann entropies, and doing so carefully will be a useful continuation.
One can also consider a classical version of the observational entropy development for density distributions ρ, σ on phase space Γ. The trace is an integral on Γ, the norm the L1 norm, and the measurement M a partition of unity. The dimensional factor d (better labelled V in the classical case) is the total volume of phase space in an appropriately chosen measure. The resulting bound (33) holds equally well in the classical case so long as one can restrict consideration to a finite region of phase space.
As with the quantum case, the classical (33) can be nontrivial (stronger than |ΔSM| ≤ log V) only when ρ, σ have support on an overlapping region of phase space. In this sense, the closeness of entropy ensured by the bound arises only from a probability to be in exactly the same state. Further one may observe that the bound will be time independent for isolated systems, whose dynamics are norm preserving. This can be considered in terms of the second law of thermodynamics for coarse-grained entropies. Suppose ρ is concentrated at some particular “second-law-violating” point in phase space, one which evolves to a lower entropy state. Any σ sufficiently close to ρ in the norm distance must also be second-law-violating, which at first glance seems to violate the principle that such states are rare. Fortunately, by the above considerations, σ may be concentrated in any phase space region disjoint from ρ, even arbitrarily nearby, without any restriction on its entropic dynamics. In this way the bound is consistent with the principle that nearby phase space points of a chaotic system may have drastically different dynamics.
ACKNOWLEDGMENTS
The authors thank Ludovico Lami for comments regarding the measured relative entropy continuity and the minimax lemma, and Niklas Galke, Philipp Strasberg, and Giulio Gasbarri for helpful discussions.
The authors acknowledge support by MICIIN with funding from European Union NextGenerationEU (Grant No. PRTR-C17.I1) and by Generalitat de Catalunya. AW is furthermore supported by the European Commission QuantERA grant ExTRaQT (Spanish MICINN Project No. PCI2022-132965), by the Spanish MINECO (Project No. PID2019-107609GB-I00) with the support of FEDER funds, the Generalitat de Catalunya (Project No. 2017-SGR-1127), by the Alexander von Humboldt Foundation, as well as the Institute for Advanced Study of the Technical University Munich.
AUTHOR DECLARATIONS
Conflict of Interest
The authors have no conflicts to disclose.
Author Contributions
Joseph Schindler: Conceptualization (equal); Formal analysis (equal); Investigation (equal); Writing – original draft (equal); Writing – review & editing (equal). Andreas Winter: Conceptualization (equal); Formal analysis (equal); Investigation (equal); Writing – review & editing (equal).
DATA AVAILABILITY
Data sharing is not applicable to this article as no new data were created or analyzed in this study.