## Abstract

Although the entropy of a given signal-type waveform is technically zero, it is nonetheless desirable to use entropic measures to quantify the associated information. Several such prescriptions have been advanced in the literature but none are generally successful. Here, we report that the Fourier-conjugated ‘total entropy’ associated with quantum-mechanical probabilistic amplitude functions (PAFs) is a meaningful measure of information in non-probabilistic real waveforms, with either the waveform itself or its (normalized) analytic representation acting in the role of the PAF. Detailed numerical calculations are presented for both adaptations, showing the expected informatic behaviours in a variety of rudimentary scenarios. Particularly noteworthy are the sensitivity to the degree of randomness in a sequence of pulses and potential for detection of weak signals.

## 1. Introduction

Let *α*=*α*(*t*) be a real, square-integrable function of a real independent variable, *t*. The following analysis is restricted to cases where *t* is one-dimensional; the generalization for arbitrary dimensionality follows readily therefrom. For convenience, let the *L*^{2}-norm, or *energy*, of *α*, written here as ∥*α*∥, be unity. Finally, suppose that *α* represents some informatic signal such as an audio waveform or digital transmission. It is appropriate to consider *t* to represent a time-like variable throughout, but the analysis is generic.

In the conventional probabilistic paradigm of information theory, it is meaningful to attribute a quantity of entropy to the process by which signals such as *α* are generated in a given system but not to any particular signal [1]. Entropy, however, is fundamentally a measure of both indeterminacy and information, and it is therefore appropriate to seek some measure of entropy to quantify the information intrinsic to a given *α*. The difficulty in such an endeavour is that standard entropy measures are defined in terms of probabilistic distributions or probabilistic density functions (PDF's). In principle, some special treatment is therefore required in order to adapt measures of entropy for use as measures of information in non-probabilistic waveforms like *α*. Existing approaches generally rely on some secondary distribution or other probabilistic construct to which a measure of indeterminacy may be attributed. See for instance [2–4]. Although a variety of such well-formulated methods have been introduced, all feature some non-trivial dependence on model or parametrization. Consequently, there is no single generally successful prescription for the informatic entropy of a waveform like *α*.

There exist additional difficulties in formulating a rigorous, general entropic measure of information that are inherent to the Shannon-type measures themselves. For instance, consider the differential Shannon entropy, which is defined here generally such that
*t*) and for any independent variable. In equation (1.1) and throughout *z*=0. Integrals with no explicit limits implicitly span the real axis. Naturally, equation (1.1) is meaningful only when λ is a meaningful PDF, which restricts us to real, non-negative, non-singular λ such that
*H*{λ} is meant to measure the indeterminacy in a continuous, real variable whose outcomes are governed by a PDF, λ. Interpreted more broadly, in abstraction from any particular variable, *H*{λ} is an intrinsic informatic attribute of λ. Strictly speaking, however, *H*{λ} alone is a somewhat unsatisfactory measure in both cases because the argument of its logarithm is not generally dimensionless [5]. Consequently, *H*{λ} is sensitive to the unit of *t*. Specifically for any real, positive *b* we may express λ equivalently as a function λ′=λ′(*t*′) of *t*′=*t*/*b* such that λ′(*t*′) d*t*′=*ρ* d*t*, hence λ′=*b*λ. The corresponding entropy, *H*{λ′}, reduces to *H*{λ} is not invariant under a simple scale-transformation in *t*.

Moreover, *H*{λ} may be negative and is unbounded. While it is reasonable to expect the entropy associated with some infinitely expansive PDF to be infinitely large, the negativity and lack of a lower bound pose particularly challenging problems for *H*{λ} as a meaningful measure *per se* for an arbitrary λ.

In the quantum mechanical context, where a given PDF is stated as the squared absolute magnitude of a probabilistic amplitude function (PAF), there exists a natural resolution to the problems cited with respect to a single measure of the form *H*{λ}. Specifically, let *ψ*=*ψ*(*t*) be a generally complex, square-integrable function of *t* with unit energy. As such the squared absolute magnitude of *ψ*, defined here as
*ψ* is accordingly meaningful as a PAF. The Fourier counterpart to *ρ* is the spectral density,
*ψ* and *ω*∈ℜ is the independent variable conjugate to *t*. Throughout this work, *y*=*y*(*t*). As the ‘hat’-notation is inconvenient in some cases, let also *y*. The inverse FT of *ρ* and *σ* form a dimensionless two-variable PDF

The total differential Shannon entropy associated with *f* is
*f* follows from a given *ψ*, it is appropriate to interpret the measure in equation (1.9) as an attribute thereof. For convenience, let us therefore define
*ψ*.

In order to examine the sensitivity of *S*{*ψ*} to arbitrary scale-transformations of the form *ω*′=*bω*. Let *σ*′=*σ*′(*ω*′) be defined such that *σ*′ d*ω*′=*σ* d*ω*. It follows directly from the appropriate substitutions in the explicit form of *H*{*σ*′} that *S*{*ψ*} is scale-invariant and thus insensitive to the unit with which *t* is measured. In addition to being unit-independent *S*{*ψ*} is subject to the lower bound
*S*{*ψ*} has emerged as a fundamentally significant information-theoretic measure in quantum systems. See, for instance, [7–12].

Although our nominal signal, *α*, has unit energy like *ψ*, it is not a PAF. Apart from being exclusively real, it simply does not represent any probabilistic process, nor does it necessarily represent any amplitude. In the conventional understanding, therefore, some secondary complex amplitude function must be constructed in association with *α* in order to adapt *S*{*ψ*} for waveforms like *α*. As in the case of secondary PDF's, such an endeavour generally introduces model-dependence and the need for some parametrization.

The purpose of this work, however, is to demonstrate that the total entropy in equation (1.10) is a sensitive, meaningful measure of information in non-probabilistic waveforms such as *α* without requiring any external models or parameters. Specifically, in addition to considering the most simple adaptation, *S*{*α*}, we consider *S*{*ϕ*}, where *ϕ* is the normalized analytic representation (AR) of a given *α*. Section 2 describes the normalized AR and the motivation for considering it as a characteristic amplitude function for a given *α*. In §3, are defined the relevant discrete quantities necessary for computation. Section 4 explores the informatic behaviours of *S*{*α*} and *S*{*ϕ*} with explicit numerical studies of their discrete analogues. Section 5 provides a brief summary and discussion of the results.

## 2. Analytic representation as an amplitude function

It follows from the realness of *α*, equation (1.5) and the Euler theorem that *α* is informatically redundant. Eliminating the redundant contributions thereby produces a generally complex function that contains the same information as does *α*. The function so produced is proportional to the well-known AR of *α*, whose imaginary part is the Hilbert transform of *α*. According to the central thesis of this present work, the AR of a given signal, with the appropriate normalization, is an appropriate candidate for a PAF-like function that is characteristic to a given real waveform.

Before proceeding, it is instructive to review how the AR and the Hilbert transform (HT) are related generally. Let *g*=*g*(*t*) be a real, square-integrable function of *t*. By expressing *g* explicitly as the inverse FT of *g*, it is therefore natural to define

By construction *γ* represents the contributions to *g* from non-negative Fourier frequencies, whereas *γ** represents the contributions from non-positive frequencies. The AR of a given *g*, written here as *g* in terms of its Fourier transform according to

Let *h*=*h*(*t*) be the imaginary part of a given *γ* expressed explicitly, substituting the dummy variable *τ* for *t* in *ω* in accordance with the Riemann–Lebesque Lemma produces
*g*, written here as *g*.

Returning to the primary subject, suppose that a PAF is sought to represent a given *α*. In order to be a meaningful descriptor of *α*, the PAF should contain essentially the same information as does *α* and its absolute magnitude should envelop *α*. The above criteria are realized naturally through the analytic representation. In particular, due to the orthogonality between *g* and *g*. Furthermore because the HT preserves the norm of square-integrable functions, we have *α* let
*α*. For convenience, we use *ϕ*=*ϕ*(*t*)=*ϕ*{*α*}=*ϕ*{*α*}(*t*). Note that we may express *ϕ* equivalently as

Associated with *ϕ* is the time-domain PDF-like function, |*ϕ*|^{2}. The corresponding spectral PDF is likewise *α*, it is thereby possible to construct the unitless, two-dimensional PDF,
*f*.

As a rudimentary example suppose that we have
*d* and *θ* are external parameters, and
*α* directly, but the authors are unaware of any published analytical solution to *ϕ* alternatively by first finding the FT of *α* and then performing the integration in equation (2.12). With
*α* of the form equation (2.15).

Figure 1 is a plot of the real and imaginary parts of *ϕ* for the case in which *d*=1 and *θ*=10. Note that the imaginary part is approximately proportional to the right-hand side of equation (2.20) in this case, as *θd*=10. Consequently, the corresponding PDF, |*ϕ*|^{2}, very nearly recovers the asymptotic Gaussian envelope.

## 3. Discrete analogues

We may readily define the discrete analogue of *S*{*ψ*} in association with the discrete analogue of *ψ*. Specifically let the discrete amplitude,
*n*-dimensional complex vector such that
**z**|_{2} signifies the usual ℓ^{2}-norm of any complex vector **z** and Δ*t*=Δ*t*{**v**} is some positive real number representing implicitly the uniform interval between nodes on the *t*-axis at which **v** is evaluated. Because *S*{*ψ*} is insensitive to the unit of *t* we are free to put Δ*t*=1 and normalize **v** accordingly. For clarity, however, it is useful to preserve Δ*t* explicitly. Analogously to *ρ* we then define the discrete distribution

The discrete spectral amplitude associated with **v** is
**z** and is defined here generally such that
*σ* is accordingly
*S*{*ψ*}, the total entropy associated with a given **v** is
*n*=*n*{**v**}.

The characteristic amplitude in equation (2.11) and its Fourier transform may be defined analogously for discrete, or sampled, signals [13,14]. First, in analogy to *α*, let
*n*-dimensional real vector such that
*t*. For convenience, and because it causes no confusion, we assign *n*=*n*{**a**} and Δ*t*=Δ*t*{**a**} henceforth. Let the discrete Hilbert transform (DHT) be defined generally such that the *k*th element of the DHT of **a**, written here as *k*=1,…,*n*. As with the DFT, in general the DHT only approximately preserves the norm of a vector. Let us therefore define
*ϕ*.

It is worth noting that the discrete terms introduced above are essentially nothing more than the most elementary numerical approximations of their respective continuous analogues. They are meant to embody only those principles already embodied in the continuous terms.

It is also important to mention that calculation of the AR requires some care in the case of discrete or sampled signals. Specifically consider some *g* with some non-trivial compact support [*t*_{1},*t*_{2}]. In general, *t*_{1},*t*_{2}]. In order to compute properly the AR for a discrete waveform, it is therefore necessary to ‘pad’ the original waveform with a sufficient number of zeros. In all calculations presented in the following section, the number of added zeros is at least an order of magnitude greater than the size of the initial waveform, thereby ensuring precision.

## 4. Waveform information

Because the total entropy is an effective measure of information in PAFs, and because *ϕ* is a complex, square-integrable function with unit energy that contains essentially the same information as does *α*, we propose that *S*{*ϕ*} is an appropriate, generally meaningful measure of information in real, non-probabilistic waveforms like *α*. This section validates the informatic utility of *S*{*ϕ*} with detailed computations of the discrete analogue, *S*_{D}{**u**} in fundamental informatic scenarios.

Additionally, we present corresponding computations of the presumably naively constructed quantity, *S*_{D}{**a**}, which is the discrete analogue of *S*{*α*}. Given that *α* and *ϕ* contain essentially the same information, if *S*{*ϕ*} is a meaningful measure of information then so too should be *S*{*α*}. Note, however, that we should not expect *S*{*ϕ*} and *S*{*α*} to be identical. In particular, the relationship between *H*{|*α*|^{2}} and *H*{|*ϕ*|^{2}} is not analytically reducible. In contrast, it follows directly from the explicit form of *S*{*ϕ*} and *S*{*α*} to differ generally. Despite the expected differences, we find that the discrete analogues of *S*{*ϕ*} and *S*{*α*} are strongly correlated and that both are effective measures of information. In fact, the observed consistencies between *S*_{D}{**u**} and *S*_{D}{**a**} constitute a separate validation of the informatic relevance of the total entropy to non-probabilistic waveforms like *α*.

In each of the following calculations, the discrete vectors are padded such that the total number of points is 2^{20}, and the number of computationally introduced zeros is thus 2^{20}−*n* in each case. The varying number of computational zeros introduces only a negligible influence.

As a first test, let us examine the behaviours of *S*_{D}{**u**} and *S*_{D}{**a**} for a waveform consisting of randomly generated noise. Specifically, suppose that **a** has the form
*ν*(*μ*,*δ*;*k*) represents the *k*th sample of normally distributed noise with mean *μ* and variance *δ*, and *C* is a constant for a given randomly generated waveform, chosen such that *t*=1 here. Figure 2 shows the normalized entropies, **a**. Note that both measures behave as if asymptotically approaching unity.

Let us next consider the complementary case of a correlated signal, such that **a** consists essentially of some regularly repeating sequence. As more and more sample points are added to such a waveform, the amount of added information decreases and the information contained in **a** asymptotically approaches some fixed quantity. As an example, let **a** have the form
*B* is chosen for appropriate normalization. Figure 3 shows the entropies, *S*_{D}{**u**} and *S*_{D}{**a**}, for **a** of the form equation (4.3) for *ω*_{0}=0.3 and *t*_{k}=1,2,…,*n*. Note that both measures exhibit the expected asymptotic behaviour, but with each converging to a different value.

It is instructive in this case to show how the two Fourier-conjugated components constituting the total entropy behave individually. Figure 4 contains plots of the Fourier components of *S*_{D}{**u**} for the same scenario shown in figure 3. Note how the *t*-domain and *ω*-domain components increase and decrease respectively, while their sum is asymptotically fixed, representing presumably the fixed information in the waveform. Analogous behaviour is observed in the components of *S*_{D}{**a**}.

Next let us examine the behaviours of *S*_{D}{**u**} and *S*_{D}{**a**} in the case of a simple, ‘bare’ signal with a varying degree of additive background noise. In general, we expect the entropy of the net waveform to increase as the energy of the noise increases in relation to the energy of the bare signal—provided that the bare signal is sufficiently simple. Let us therefore introduce a discrete waveform of the form
*c*′ and *C*′ are to be chosen to effect a given SNR while ensuring that

Figure 5 contains plots of *S*_{D}{**u**} and *S*_{D}{**u**} as functions of the SNR for **a** of the form equation (4.4) with *d*=1 and *θ*=10. In each calculation, the signal was constructed with *n*=9001 samples taken at equally spaced points spanning [−8,8]. The value of Δ*t* follows. Each point on the plot represents the average of 10 separate calculations, each of which was performed with a separately generated background noise function. The standard deviation in each plotted point is smaller than the interval implied by the size of the symbol with which the point is plotted. The dashed line in figure 5 marks the average entropy, *S*_{D}{**u**}, of 9001 samples of Gaussian noise, which is approximately 9.4, and the dot-dashed line represents the analogous background noise for *S*_{D}{**a**}. The entropy of the noise is the limiting entropy as the SNR vanishes (or becomes infinitely negative as measured in dB). Conversely in the limit as the SNR diverges *S*_{D}{**u**} is very nearly *n* is sufficiently large. Both measures evolve with respect to the SNR in broad accordance with expectations of measures of information. Also noteworthy is the sensitivity of the entropy measures to the presence of signals even below −20 dB.

As a fourth test of the waveform entropy, let us consider a sequence of bits represented as a sequence of binary-valued square pulses. The informatic complexity of *α* in such a case should depend on the arrangement of the bits. For example, the sequence ‘101010…’ conveys less information than a sequence of bits arranged in an irregular manner. The informatic entropy of the former should presumably be smaller than that of the latter. Figure 6 shows plots of *S*_{D}{**u**} and *S*_{D}{**a**} in a sequence of 1024 bits originally ordered as ‘101010…’ but with *m* consecutive bits somewhere in the sequence being reordered randomly. The corresponding signal, **a**, was constructed by assigning 8 consecutive sample points *a*_{k}=0 to each ‘0’. We accordingly have *n*=8192, hence the value *m*. It is also interesting to note that *S*_{D}{**u**}<*S*_{D}{**a**} in this case, whereas the opposite was observed in the three preceding studies. This does not have any significance *per se*, but suggests that the relationship between *S*_{D}{**u**} and *S*_{D}{**a**} is complex and deserving of further investigation.

## 5. Summary

The foregoing analysis addresses a fundamental issue in information theory, namely the characterization of the informatic content of real waveforms. We have demonstrated that the total entropy associated with quantum-mechanical PAF's is a meaningful measure of information in waveforms like *α*. Specifically, the computations described in §4 validate that *S*_{D}{**u**} and *S*_{D}{**a**} exhibit the behaviours expected of a sensitive measure of information over a broad range of rudimentary, non-trivial scenarios.

Our approach differs notably from previous studies in that we have not sought to construct a secondary distribution or other probabilistic artifice to describe a given waveform, but rather we have simply attributed a measure of entropy to the waveform itself—either directly or through the AR. The broad correlation between *S*_{D}{**u**} and *S*_{D}{**a**} is noteworthy in that it is not guaranteed *a priori*, and *α* is neither complex nor probabilistic. More to the point, the observed consistencies between *S*_{D}{**u**} and *S*_{D}{**a**} affirm that both measures, and their continuous analogues, are meaningful measures of information. Specifically, if *S*{*ϕ*} measures information then so should *S*{*α*} because *ϕ* and *α* contain essentially the same information.

It is worth noting that the computations plotted in figure 5 indicate that the entropy prescriptions described in this work may be useful in the detection of weak signals in noisy backgrounds. Moreover, the sensitivity to the degree of randomness in the arrangement of a sequence of square pulses indicates a connection to Kolmogorov complexity that begs further investigation. Such a sensitivity may be useful in cryptological applications, for instance. Additional applications will be the subject of future work.

## Data accessibility

This work did not rely on any experimental data.

## Authors' contributions

S.F. conceived the basic model. W.S. and A.W. contributed to analysis and testing. A.W. contributed to mathematical analysis of the Hilbert transform.

## Competing interests

We have no competing interests.

## Funding

This work was funded through the Naval Innovative Science and Engineering (NISE) program as implemented at SPAWAR Systems Center Atlantic.

- Received January 13, 2016.
- Accepted May 10, 2016.

- © 2016 The Author(s)

Published by the Royal Society. All rights reserved.