## Abstract

We explore the relationship between the Evans function, transmission coefficient and Fredholm determinant for systems of first-order linear differential operators on the real line. The applications we have in mind include linear stability problems associated with travelling wave solutions to nonlinear partial differential equations, for example reaction–diffusion or solitary wave equations. The Evans function and transmission coefficient, which are both finite determinants, are natural tools for both analytic and numerical determination of eigenvalues of such linear operators. However, inverting the eigenvalue problem by the free-state operator generates a natural linear integral eigenvalue problem whose solvability is determined through the corresponding infinite Fredholm determinant. The relationship between all three determinants has received a lot of recent attention. We focus on the case when the underlying Fredholm operator is a trace class perturbation of the identity. Our new results include (i) clarification of the sense in which the Evans function and transmission coefficient are equivalent and (ii) proof of the equivalence of the transmission coefficient and Fredholm determinant, in particular in the case of distinct far fields.

## 1. Introduction

Our goal is to establish the connection between the Evans function, transmission coefficient and Fredholm determinant associated with linear *n*th order eigenvalue problems on *Y* =*Y* ′ and *V* represents a perturbative potential function that decays to zero in the far field of the domain *A*_{0} generates a background or free state. It is constant in the far field though the limits are not necessarily the same. We suppose further that *A*_{0} depends linearly on a spectral parameter *Y* ^{−} and right block *Y* ^{+}. The columns of these two matrices are solutions to the differential equation above that decay to zero exponentially fast in the left and right far fields, respectively. The Evans function measures the ‘distance from intersection’ of the subspaces spanned by the columns of *Y* ^{−} and *Y* ^{+}. The transmission coefficient which is also a determinant, measures the degree to which the solutions *Y* ^{−}, that decay to zero in the left far field, are orthogonal to the subspace that is orthogonal to the subspace of solutions that decays to zero in the right far field. Unwrapping the two orthogonality conditions explains why the Evans function and transmission coefficient are essentially equivalent. We assume away from the essential spectrum (∂−*A*_{0})^{−1} exists. Then our eigenvalue problem can be expressed in the form (id−(∂−*A*_{0})^{−1}*V*)*Y* =*O*, or, with *V* =*U*|*V* | representing the polar decomposition of *V* and setting *ϕ*:=|*V* |^{1/2}*Y* , in Birman–Schwinger form
*V* |^{1/2}(∂−*A*_{0})^{−1}*U*|*V* |^{1/2} is a Hilbert–Schmidt compact operator on this space is relatively straightforward. However, herein we focus on the case when it is a trace class operator, i.e. a nuclear operator. With this property, zeros of the Fredholm determinant of id−|*V* |^{1/2}(∂−*A*_{0})^{−1}*U*|*V* |^{1/2} coincide with eigenvalues. Thus, we come to the central issue. In what sense are the Evans function, transmission coefficient and Fredholm determinant related? Let us briefly outline what has already been established.

The Evans function was first proposed by Evans [1], while Alexander *et al*. [2] established it as a geometric tool for stability analysis. Subsequently, it has become a standard tool in analytical and numerical studies of the stability of travelling waves; see the review papers featuring the Evans function by Sandstede [3] and Kapitula [4]. The Evans function is also called the miss-distance function [5]. It is also a generalization of the Wronskian and Jost function. The transmission coefficient has it origins much further back in the mathematical literature. Its connection to the Evans function, though trivial in the scalar case, can be found in Swinton [6] and Bridges & Derks [7] for higher order problems. The Fredholm determinant for determining the solvability of linear integral equations was introduced by Fredholm [8]. It has been given recent impetus by Bornemann [9]. Its connection to the transmission coefficient goes back to Jost & Pais [10]. Simon [11,12] computes the explicit relationship between the Fredholm determinant and Wronskian for some example scalar Schrödinger operators; also see Kapitula & Sandstede [13]. However, more generally, Gesztesy & Makarov [14] showed that for operators with semi-separable kernels, their Fredholm and 2-modified Fredholm determinants can be reduced to the determinant of finite rank operators, potentially useful for the evaluation of such Fredholm determinants. Gesztesy *et al.* [15] then established the connection between the Evans function and a 2-modified Fredholm determinant. They also gave a coordinate-free definition of the Evans function as a ratio of the perturbed and unperturbed versions of the function. The 2-modified Fredholm determinant is relevant for their equivalence results as systems of first-order operators generate operators |*V* |^{1/2}(∂−*A*_{0})^{−1}*U*|*V* |^{1/2} which are Hilbert–Schmidt class and in general not trace class. When such operators are trace class, the Fredholm determinant is the natural object in the equivalence result. Indeed, systems of Schrödinger operators represent an explicit example [16, Section 4].

Our goal herein is to establish a unified picture of the relationship between the Evans function, transmission coefficient and Fredholm determinant. We focus on those systems of first-order operators for which |*V* |^{1/2}(∂−*A*_{0})^{−1}*U*|*V* |^{1/2} is trace class and the matrix trace of the matrix perturbation potential *V* is zero. By considering this subclass of first-order operators, we gain a degree of clarity and directness. To begin with we assume *A*_{0} is constant, but in our final main §7 we assume distinct far field limits for *A*_{0} which is therefore no longer constant. The free Evans function and free transmission coefficients are the corresponding quantities associated with the operator ∂−*A*_{0}. What we achieve in this paper is as follows, we:

(i) Provide practical tests to determine when |

*V*|^{1/2}(∂−*A*_{0})^{−1}*U*|*V*|^{1/2}is trace class. These follow results in Simon [12, ch. 4 (see §3)];(ii) Show how two important classes of examples, systems of Schrödinger operators and arbitrary order scalar operators, generate operators |

*V*|^{1/2}(∂−*A*_{0})^{−1}*U*|*V*|^{1/2}which are trace class. We reveal how the trace class properties of this Birman–Schwinger formulation naturally reduce to the trace class properties of the example operators directly. These examples also demonstrate how many practical systems generate such trace class operators, with the matrix trace of*V*also equal to zero (see §4);(iii) Prove simply and directly that the ratio of the Evans function and free Evans function equals the ratio of the transmission coefficient and free transmission coefficient. This new insight clarifies their relationship and indicates a convenient rescaling of the state variables that normalizes the free transmission coefficient to unity (see §5);

(iv) We show the matrix trace of the semi-separable kernels of Birman–Schwinger operators |

*V*|^{1/2}(∂−*A*_{0})^{−1}*U*|*V*|^{1/2}are continuous along the diagonal, despite the fact the kernels have a jump discontinuity there—this assumes the matrix trace of*V*is zero. Hence, we can unambiguously define the trace of such operators. We then provide a simple and direct proof that the scaled transmission coefficient equals the Fredholm determinant of |*V*|^{1/2}(∂−*A*_{0})^{−1}*U*|*V*|^{1/2}, assuming it is trace class (see §6).(v) Prove, for the case of distinct far fields, the scaled transmission coefficient equals the Fredholm determinant with mild algebraic decay constraints on

*V*(see §7).

Items (ii) to (v) above are a self-contained collection of new results. We bookend the sections above with §§2 and 8. In §2, we provide preliminary results characterizing the spaces of trace class and Hilbert–Schmidt class operators and their relation. We include some important inequalities required in subsequent sections. In §8, we summarize our results, discuss conclusions we can draw from them and outline possible future projects.

## 2. Characterizations

To be self-contained, we record a few basic facts on compact operators that we shall need. We refer to Reed & Simon [17,18], Simon [12] and Gohberg *et al.* [19] for more details. Let *φ*_{m}}_{m≥1} and standard inner product *K*, there is a unique operator *K*^{†} to *K* is the unique operator such that *K*^{†}*K* is positive as *U* such that *K*=*U*|*K*|. For any operator *p*≥1 are then defined as follows,
*trace class* if it belongs to *Hilbert–Schmidt class* if it belongs to _{m}}_{m≥1} of any compact operator *s*_{m}}_{m≥1} of *p*≤*q*. Fundamentally, for any trace class operator *p*=1) by
*p* is an integer greater than one, we define the *p*-modified or regularized Fredholm determinants for compact operators *BA* and where ∥⋅∥_{op} denotes the operator norm. The proof of these three results can be found for example in Conway [20, Section 18].

## 3. Practical tests

The natural setting we require, and which we assume hereafter, is the separable Hilbert space of

### Lemma 3.1 (Hilbert–Schmidt class operators)

*The operator* *is Hilbert*–*Schmidt if and only if there is a function* *such that*
*for all* *In addition, we have*

The Fourier transform of operators will play a key role in our analysis. We define the Fourier transform *multiplicatively* in Fourier space, i.e. we have *G* in physical space, we think of *K** as the map taking *φ* to the inverse Fourier transform of *K**:*φ*↦(2*π*)^{−1/2}*G***φ*, where *G***φ* represents the convolution of *G* and *φ*. We note that if *K***H* is (2*π*)^{−1/2}*G*(*x*−*y*)*H*(*y*). We now prove the following lemma which is the matrix version of a result given in Simon [12, ch. 4].

### Lemma 3.2 (Practical test for Hilbert–Schmidt class)

*Suppose* *and indeed we have*

### Proof.

By direct computation, line by line we successively use the following results: (i) the kernel of *K***H* is (2*π*)^{−1/2}*G*(*x*−*y*)*H*(*y*) and the trace of a product of two operators is invariant to their permutation; (ii) the Cauchy–Bunyakovski–Schwarz inequality in the form tr *A*^{†}*B*≤(tr *A*^{†}*A*)^{1/2}(tr *B*^{†}*B*)^{1/2} for any two matrices *A* and *B* [21, p. 289]. We used this inequality with *A*=*G*^{†}*G* and *B*=*HH*^{†} and also that tr(*HH*^{†})^{†}*HH*^{†}≡ tr(*H*^{†}*H*)^{†}*H*^{†}*H*; (iii) The sum of the squares of singular values is less than the square of their sum, i.e. (tr *A*^{†}*A*)^{1/2}≤tr(*A*^{†}*A*)^{1/2}; (iv) The Young inequality; (v) That

Lemma 3.2 provides us with a test to determine if a given bounded operator is of Hilbert–Schmidt class. In practice, suppose we know the Fourier transform *J*=*J*(*x*) and *H*=*H*(*x*) are bounded multiplicative operators from *JK***H* is (2*π*)^{−1/2}*J*(*x*)*G*(*x*−*y*)*H*(*y*). The Hilbert–Schmidt ideal property *JK***H* from *K***H* is trace class, for example where *H*=*H*(*x*) is a bounded multiplicative operator and *K** is the operator corresponding to *w* is defined via the map *w*:*ξ*↦(1+*ξ*^{2})^{1/2}. The practical test takes the following form, which is the matrix version of a result given in Reed & Simon [22, Appendix 2].

### Lemma 3.3 (Practical test for trace class)

*Suppose* *then* *and there exists a constant* *c*>0 *such that*

### Proof.

We adapt the proof for the scalar case given in Reed & Simon [22, Appendix 2]. Using the practical test for Hilbert–Schmidt class lemma 3.2 and that *H* imply that *K***H* is of Hilbert–Schmidt class and has integral kernel (2*π*)^{−1/2}*G*(*x*−*y*)*H*(*y*). We decompose *K***H*=*AB* into the product of the two operators *A* and *B* defined as follows, *A*:=*K**(1−∂^{2})^{1/2}*w*^{−1} and *B*:=*w*(1−∂^{2})^{−1/2}*H*, where *w*=*w*(*x*) is the weight function. Then using *w*=*w*(*x*) is scalar, we find
*w*=*w*(*ξ*) represents the Fourier transform of (1−∂^{2})^{−1/2}. The action of the operator (1−∂^{2})^{−1/2} is given by convolution with a function *w*^{−1}=*w*^{−1}(*ξ*) is analytic in the strip *w*(1−∂^{2})^{−1/2}*H* is (2*π*)^{−1/2}*w*(*x*)*G*(*x*−*y*)*H*(*y*) and so
*c*>0 as

The trace ideal property of *JK***H* is also trace class for any bounded multiplicative operator *K***H* is trace class. We end this section with a trace formula and an immediate corollary. A proof is provided in appendix B.

### Lemma 3.4 (Trace formula)

*Given any integer ℓ*≥2, *suppose that* *K*_{1},*K*_{2},…,*K*_{ℓ} *are Hilbert*–*Schmidt operators with canonical respective kernels* *G*_{1},*G*_{2},…,*G*_{ℓ} in *Then we have*
*If* *K* *is trace class and its kernel* *is continuous on* *then*

### Corollary 3.5 (Trace formula: separable kernel)

*If a Hilbert–Schmidt operator* *K* *has a separable kernel* *G*(*x*;*y*)=*G*_{1}(*x*)*G*_{2}(*y*) *for all* *then for any integer* ℓ≥1, *we have*

### Remark 3.6

Some additional comments on the material above are as follows: (i) the map *K** above corresponds to the operator *K*(−*i*∇) in Reed & Simon [18, p. 57] and Simon [12, p. 37]; (ii) the sufficiency conditions in lemma 3.3 on *H* can be weaker, for example a result of Birmann–Solomajk requires they need only be in the space of ℓ^{1}-summable *H*, where *I*_{k} is the unit interval with centre *d*, with weights *w*^{α} and *α*>*d*/2; and (iv) in the trace formula lemma 3.4, we relax the continuity condition on *G* in §6 and allow *G* to have a jump across its diagonal—the formula for tr *K* remains meaningful as the kernels we consider have continuous matrix traces along the diagonal; see also remark 6.3 where we discuss results by Brislawn [23,24] for discontinuous kernels.

## 4. Examples

Anticipating our main result in §§6 and 7, we will be concerned with establishing whether operators from *V* |^{1/2}(∂−*A*_{0})^{−1}*U*|*V* |^{1/2} are of trace class. Here, *V* =*V* (*x*) is a bounded multiplicative matrix operator from *U*=*U*(*x*) is the unitary matrix obtained from the polar decomposition of *V* . As we saw in §3, the practical tests for Hilbert–Schmidt or trace class operators at our disposal rely on testing the integrability properties of the multiplicative operator (i*ξ*id−*A*_{0})^{−1} in Fourier space corresponding to the operator (∂−*A*_{0})^{−1}, as well as the integrability properties of |*V* |^{1/2} and *U*|*V* |^{1/2}. We assume in this section that the eigenvalues of *A*_{0} are non-zero and never purely imaginary. Hence, the integrability properties of the operator (i*ξ*id−*A*_{0})^{−1} rely on the rate of its asymptotic decay as *ξ*id−*A*_{0})^{−1} is square integrable. We assume *hereafter* that
*V* are square integrable. We observe, using the unitary properties of *U* and that |*V* | is selfadjoint, that *V* |^{1/2}(∂−*A*_{0})^{−1}*U*|*V* |^{1/2} is of Hilbert–Schmidt class.

Recall the sufficient conditions for |*V* |^{1/2}(∂−*A*_{0})^{−1}*U*|*V* |^{1/2} to pass the practical test for trace class lemma 3.3. These are first that |*V* |^{1/2} have bounded operator norm, and second that *V* |^{1/2}(∂−*A*_{0})^{−1}*U*|*V* |^{1/2} fails the practical test for trace class lemma 3.3. However, we now consider two examples of operators of the form above which pass the practical test for trace class. This is achieved by taking advantage of the structure of (∂−*A*_{0})^{−1} and *V* , as will be apparent.

### Example 4.1 (System of elliptic operators)

Consider the coupled Schrödinger operator of the form ∂^{2}−*d*_{0}∂−*c*_{0}−*v*, where *c*_{0} and *d*_{0} are constant square matrices and *v* is a matrix potential on *u*. Here, *a*_{0} is the matrix of diffusion coefficients. Replacing ^{2}−*d*_{0}∂−*c*_{0} and matrix potential *V* corresponding to *v* have the form
*O* represents the zero matrix. The matrices |*V* |^{1/2} and *U* have the form
*A*_{0})^{−1}]_{1,2} represents the top right-hand square matrix block in (∂−*A*_{0})^{−1} whose dimensions are half those of (∂−*A*_{0})^{−1} itself. In Fourier space, (∂−*A*_{0})^{−1} here has the form
*ξ*id−*A*_{0})^{−1}]_{1,2}=((i*ξ*)^{2}id−(i*ξ*)*d*_{0}−*c*_{0})^{−1}. This is the operator in Fourier space corresponding to (∂^{2}−*d*_{0}∂−*c*_{0})^{−1} as expected. We observe that it is square integrable with respect to the weight *w* as ((i*ξ*)^{2}id−(i*ξ*)*d*_{0}−*c*_{0})^{−1}(1+*ξ*^{2})^{1/2}∼−*ξ*^{−1}id as *V* |^{1/2}(∂−*A*_{0})^{−1}*U*|*V* |^{1/2} transfers to the trace class property of |*v*|^{1/2}[(∂−*A*_{0})^{−1}]_{1,2}*v*|*v*|^{−1/2}, and the properties we assume for *V* transfer to *v*, and vice versa. Hence by the practical test for trace class lemma 3.3, |*V* |^{1/2}(∂−*A*_{0})^{−1}*U*|*V* |^{1/2} is of trace class.

### Example 4.2 (High-order scalar operator)

Consider the scalar *n*th order linear operator given by ∂^{n}+*a*_{n−1}∂^{n−1}+⋯+*a*_{1}∂+*a*_{0}+*v*, where the *a*_{i}, *i*=0,…,*n*−1 are scalar constants and *v* is a scalar potential function on *V* has the same form as that in the last example, with the only non-zero entry being the lower left entry which is *v*; however, this is now the scalar entry in the lower left position and the rest of the *n*×*n* matrix *V* has zero entries. Similarly, the matrices |*V* |^{1/2} and *U* have the same form as in the last example, but the entries shown are scalar with the rest of the matrix entries being zero. Direct calculation reveals that the only non-zero entry in the matrix operator |*V* |^{1/2}(∂−*A*_{0})^{−1}*U*|*V* |^{1/2} is the top left scalar entry |*v*|^{1/2}[(∂−*A*_{0})^{−1}]_{1,n}*v*|*v*|^{−1/2}, where [(∂−*A*_{0})^{−1}]_{1,n} represents the top right-hand scalar entry in the operator (∂−*A*_{0})^{−1}. Direct calculation reveals the corresponding entry in the Fourier transform of (∂−*A*_{0})^{−1} is given by [(i*ξ*id−*A*_{0})^{−1}]_{1,n}=((i*ξ*)^{n}+(i*ξ*)^{n−1}*a*_{n−1}+⋯+(i*ξ*)*a*_{1}+*a*_{0})^{−1}. This is square integrable in Fourier space with respect to the weight *w*. Hence, as in the last example, by the practical test for trace class lemma 3.3, the operator |*V* |^{1/2}(∂−*A*_{0})^{−1}*U*|*V* |^{1/2} is trace class.

## 5. Evans function and transmission coefficient

Our main practical concern are eigenvalue problems associated with the stability of travelling pulses. A wide class of such eigenvalue problems can be expressed in the following form on *A*_{0}=*A*_{0}(λ) is a constant *V* =*V* (*x*) with *V* is *w*^{2}-integrable, uniformly bounded and continuous, exactly as outlined at the beginning of the last examples (§4). A comprehensive reference at this stage for the present material is Sandstede [3]. Our goal is to determine the values of λ for which the operator ∂−*A*_{0}−*V* is not invertible, i.e. the spectrum of this operator. The complement to the spectrum in *A*_{0}=*A*_{0}(λ) is strictly hyperbolic; this is characteristic of a wide class of travelling pulse stability problems. Hence away from the essential spectrum, the eigenvalue equation above has an exponential dichotomy and the unbounded operator ∂−*A*_{0}−*V* is Fredholm with index zero. At pure-point spectrum values, the kernel of ∂−*A*_{0}−*V* is non-trivial. The existence of the exponential dichotomy to the eigenvalue equation above, and thus Fredholm property of ∂−*A*_{0}−*V* , as well as the locale of the essential spectrum, is determined by the classification of the solutions to the following associated constant coefficient equation which does not exhibit a pure-point spectrum:
*V*) implies that there is a, say, *k*-dimensional subspace of solutions to the eigenvalue equation that decays exponentially as *n*−*k*)-dimensional subspace of solutions that decays exponentially as *Y* ^{−}=*Y* ^{−}(*x*;λ) the *Y* ^{+}=*Y* ^{+}(*x*;λ) the

From this perspective, the pure-point spectrum is determined by the values of *Y* ^{−} and *Y* ^{+}, intersect. In other words, a solution to the eigenvalue problem exists that decays exponentially to zero in both far fields. This is equivalent to the condition that the columns of *Y* ^{−} and *Y* ^{+} are not linearly independent, and a test for that is whether the determinant, of the *n*×*n* matrix whose columns are the columns of *Y* ^{−} and *Y* ^{+}, is zero. This intersection property should be *x*-independent and by Liouville’s Theorem an appropriate scalar factor achieves this. This is the Evans function, first introduced by Evans [1]. A comprehensive study is provided in [2].

### Definition 5.1 (Evans function)

The Evans function is the λ-dependent complex scalar quantity
*A*_{0}, i.e. corresponding to the free state. If we replace *Y* ^{±} by *V* ≡0 in the Evans function, we get the free Evans function defined as

Associated with the eigenvalue problem above is the adjoint eigenvalue problem given by *Z*+*ZA*_{0}+*ZV* =*O*, where *Z*^{†} is *Z* is *A*_{0}−*V* is *Z*^{−}=*Z*^{−}(*x*;λ) and *Z*^{+}=*Z*^{+}(*x*;λ) the, respectively,

### Lemma 5.2 (Diagonal relation)

*The solutions* *and* *satisfy the relation*
*where* *D* *is a constant diagonal matrix with non-zero entries*.

### Proof.

Consider any pair of solutions *Z*_{0} *Y* _{0})=−*Z*_{0}*A*_{0}*Y* _{0}+*Z*_{0}*A*_{0}*Y* _{0}=*O*. Thus, *Z*_{0} *Y* _{0} is constant on *Y* _{0} has the form *μ* is an eigenvalue and *A*_{0}. By our strict hyperbolicity assumption, there are *n* independent solutions, and *k* of the eigenvalues have a positive real part. Each solution *Z*_{0}, corresponding to an adjoint solution, has the form *ν* is an eigenvalue and *A*_{0}. Classically, if *μ*≠*ν* then *WU*=0, while if *μ*=*ν* we have *WU*≠0 [21, p. 405, 523].▪

With all this in hand, we can now motivate and define the transmission coefficient. Starting with the Evans function, we find
*x*-independent and λ-dependent. We could define a generalized transmission coefficient as the product of the middle exponential term and the numerator in the ratio. The denominator would correspond to a free generalized transmission coefficient. However, classically, we take the limit as

### Definition 5.3 (Transmission coefficient)

This is defined as the λ-dependent complex scalar quantity

The relation we derived above establishes that away from the essential spectrum the Evans function can be decomposed as a product of the free Evans function and a ratio of the transmission coefficient and free transmission coefficient. In other words, schematically, we have
*A*_{0}, respectively, corresponding to the *k* eigenvalues with positive real part and the *n*−*k* eigenvalues with negative real part. Also let *A*_{0}, respectively, corresponding to the *k* eigenvalues with negative real part and the *n*−*k* eigenvalues with positive real part. Then we have the identifications
*Λ*^{−} and *Λ*^{+} denote the diagonal matrices of the eigenvalues of *A*_{0} with positive and negative real parts, respectively.

Second, we can rescale the free state solutions *W*^{±} as follows.

### Definition 5.4 (Unitarily scaled solutions)

Suppose *D* is the constant diagonal matrix from the diagonal relations lemma 5.2. Let *D*_{−} denote the upper *k*×*k* block of *D* and *D*_{+} the lower (*n*−*k*)×(*n*−*k*) block. We define rescaled solutions

Note the solutions *W*^{±} replaced by

### Lemma 5.5 (Unitary relations)

*The solutions* *and* *satisfy the unitary relations*
*These are generated by corresponding unitary relations satisfied by* *U*^{±} and

### Proof.

From the proof of the diagonal relation lemma 5.2, we already know that
*U*^{±}, i.e. with the identity matrix on the right. Hence the inverse of the *n*×*n* matrix (*U*^{−} *U*^{+}) exists and is given by the corresponding *n*×*n* matrix with *U*^{±} swapped round, now follows [21, p. 117]. The unitary relations for

Third, in the next section, the unitarily scaled solutions *A*_{0})^{−1}. They are also natural for establishing the equivalence between the transmission coefficient and corresponding Fredholm determinant, as the corresponding free transmission coefficient is unity with these scaled solutions. Any result we prove using the unitarily scaled solutions we can recover for the original solutions *A*_{0}−*V* with coincident multitude. The solutions *Y* ^{±} and

### Remark 5.6

Classically, the transmission coefficient is defined as follows. Consider the solutions *Y* ^{−}∼*Y*^{−}_{0} as *k*×*k* and (*n*−*k*)×(*n*−*k*) matrices *a*=*a*(λ) and *b*=*b*(λ) are the transmission and reflection matrix coefficients, respectively. For an element of the span of *Y* ^{−} to be an eigenfunction, we require it to be asymptotically in the span of *Y* ^{−} to be orthogonal to the subspace of *Y* ^{−} to be orthogonal to the subspace spanned by the rows of *Y* ^{−} to be orthogonal to each row of *V* ≡0. They show in this case, the connection between the Evans function, transmission coefficient and Fredholm determinant (in the following sections). Bridges & Derks [7] show the transmission coefficient equals the Evans function up to a non-zero analytic factor.

## 6. Equivalence theorem

Our goal in this section is to show that the transmission coefficient equals the Fredholm determinant for trace class operators, associated with eigenvalue problems on *A*_{0})^{−1} exists. Hence our eigenvalue problem can be expressed in the form
*V* is *w*^{2}-integrable, uniformly bounded and continuous on *V* =*U*|*V* | and set *ϕ*:=|*V* |^{1/2}*Y* , we see
*V* |^{1/2}(∂−*A*_{0})^{−1}*U*|*V* |^{1/2} is the Birman–Schwinger operator as considered in §4. To proceed, we establish Green’s kernel corresponding to |*V* |^{1/2}(∂−*A*_{0})^{−1}*U*|*V* |^{1/2}. Suppose (∂−*A*_{0})^{−1} has a representation
*y*=*x*. Note that *G* also depends on λ through *A*_{0}=*A*_{0}(λ). However, we suppress the dependence on λ in all the relevant variables for the moment. Classical theory implies that we require *G*=*G*(*x*;*y*) to satisfy the pair of differential equations ∂_{x}*G*−*A*_{0}*G*=*δ*(*x*−*y*)id and −∂_{y}*G*−*GA*_{0}=*δ*(*x*−*y*)id. Two formal calculations that can retrospectively be made rigorous reveal this is the correct prescription for *G*=*G*(*x*;*y*). First, we observe *G*=*G*(*x*;*y*) with the correct decay properties in the far field is given by the following semi-separable form which can be confirmed by direct substitution,
*G*(*x*^{+};*x*)−*G*(*x*^{−};*x*)=id across the diagonal *y*=*x* is satisfied.

### Remark 6.1

The solutions *G* in the form *G*=*G*(*x*−*y*), consistent with the kernel result for Hilbert–Schmidt operators stated prior to the practical test for Hilbert–Schmidt class lemma 3.2. Indeed, we note that

Green’s kernel corresponding to |*V* |^{1/2}(∂−*A*_{0})^{−1}*U*|*V* |^{1/2} has the semi-separable form |*V* (*x*)|^{1/2}*G*(*x*;*y*)*U*(*y*)|*V* (*y*)|^{1/2}. Upon closer inspection, the unitary relation *G*=*G*(*x*;*y*) have a unit jump at the diagonal *y*=*x*, while the off-diagonal elements are continuous. In addition, the unitary relation implies that along the diagonal *y*=*x*, we have
*V* ≡0, then along the diagonal *y*=*x*, we have
*V* (*x*)|^{1/2}*G*(*x*;*y*)*U*(*y*)|*V* (*y*)|^{1/2} is continuous at the diagonal *y*=*x*. We can thus unambiguously define the trace of |*V* |^{1/2}(∂−*A*_{0})^{−1}*U*|*V* |^{1/2}.

### Definition 6.2 (Trace for discontinuous kernels)

For potential peturbations *V* which are *w*^{2}-integrable, uniformly bounded and continuous on *V* ≡0, we define the trace of the linear operator |*V* |^{1/2}(∂− *A*_{0})^{−1}*U*|*V* |^{1/2}, for which (∂−*A*_{0})^{−1} has a kernel with a jump along the diagonal, by

### Remark 6.3

We note the following: (i) consider the examples of trace class operators in §4. Green’s kernels corresponding to |*V* |^{1/2}(∂−*A*_{0})^{−1}*U*|*V* |^{1/2} are given by |*v*(*x*)|^{1/2}*G*_{12}(*x*;*y*)*v*(*y*)|*v*(*y*)|^{−1/2} and |*v*(*x*)|^{1/2}*G*_{1n}(*x*;*y*)*v*(*y*)|*v*(*y*)|^{−1/2} in the first and second examples, respectively. They both involve the off-diagonal elements of *G* only and hence these kernels are continuous; (ii) by analogous arguments to those above, if tr *V* ≡0, then along the diagonal *y*=*x* the matrix trace of the kernel *G*(*x*;*y*)*V* (*y*) corresponding to (∂−*A*_{0})^{−1}*V* is also continuous. Indeed, using the invariance of the trace to the order of the product of two operators, the matrix traces of *G*(*x*;*y*)*V* (*y*) and |*V* (*x*)|^{1/2}*G*(*x*;*y*)*U*(*y*)|*V* (*y*)|^{1/2} are equal along the diagonal *y*=*x*; (iii) consider the scalar Schrödinger operator ∂^{2}−*c*_{0}−*v* which is a special case of both examples in §4. Re-writing it as a first-order operator and focusing on (∂−*A*_{0})^{−1}*V* , we observe if *G*=*G*(*x*−*y*) is the 2×2 Green’s kernel corresponding to (∂−*A*_{0})^{−1}, then Green’s kernel corresponding to (∂−*A*_{0})^{−1}*V* is the 2×2 matrix whose left column contains *G*_{12}*v* and *G*_{22}*v* and whose right column is zero. The Fourier transforms of *G*_{12} and *G*_{22} are *w*, but *et al.* [19, p. 244], we have the following result for scalar separable integral operators whose kernel functions either side of the diagonal *y*=*x* are continuous, up to and including the diagonal. If the operator is trace class, then the kernel function is continuous on *G*_{22} has a jump along the diagonal *y*=*x*, we conclude it cannot be trace class; and (iv) Brislawn [23,24] has shown how to define the trace of a trace class operator with a kernel that is only square integrable and thus not necessarily continuous. This is achieved by averaging on cubes via the Hardy–Littlewood maximal function. In particular, the plain Volterra operator on *π*(2*n*+1)) and thus is not trace class [23, Example 3.2].

We now establish the main result of this section, our equivalence theorem. Before we state and prove it, we require the following key lemma. It concerns the solutions *Y* ^{−} of the eigenvalue problem (∂−*A*_{0}−*V*)*Y* =*O* whose column span coincides with the subspace of solutions decaying as *Y* ^{−} satisfy the Volterra integral equation

### Lemma 6.4 (Volterra integral equation)

*Assume* tr *V* ≡0 *and* *A*_{0} *and* *V* =*V* (*x*) *satisfy the conditions stated above. Let* *J* *denote the integral operator with kernel* *where* *H*=*H*(*x*) *is the Heaviside step function. First, if we set* *ϕ*^{−}:=|*V* |^{1/2}*Y* ^{−} *and* *ϕ*^{−}_{0}:=|*V* |^{1/2}*Y*^{−}_{0}, *then* *Y* ^{−} *solves the eigenvalue problem* (*∂*− *A*_{0}−*V*)*Y* =*O* *with* *Y* ^{−}∼*Y*^{−}_{0} *as* *if and only if* *ϕ*^{−} *satisfies*
*Equivalently we have* *Second, an equivalent expression for the kernel for* *J* *is given by* *where* *ϕ*^{+}_{0}:=|*V* |^{1/2}*Y*^{+}_{0}.

### Proof.

Preceding the lemma, we already established that *Y* ^{−} satisfies the Volterra integral equation shown if and only if *Y* ^{−} solves the eigenvalue problem (∂−*A*_{0}− *V*)*Y* =*O* with *Y* ^{−}∼*Y*^{−}_{0} as *Y* ^{−} satisfies the Volterra integral equation preceding the lemma if and only if *ϕ*^{−} satisfies the Volterra integral equation shown in the lemma, follows by premultiplying the Volterra integral equation for *Y* ^{−} by |*V* (*x*)|^{1/2} and using the definition of *ϕ*^{−}. The equivalence of the kernels follows from the unitary relation

### Theorem 6.5 (Equivalence)

*Assume the operator |V |*^{1/2}(*∂−A*_{0})^{−1}*U|V |*^{1/2} *is trace class, tr V ≡0 and A*_{0} *and V =V (x) satisfy the conditions stated above. Then the transmission coefficient for the eigenvalue problem (∂−A*_{0}*−V)Y =O and Fredholm determinant of id−|V |*^{1/2}(*∂−A*_{0})^{−1}*U|V |*^{1/2} *are equal:
*

### Proof.

First, we focus on the transmission coefficient. As we have ∂*Y* ^{−}=(*A*_{0}+ *V*)*Y* ^{−} and *Y* ^{−} by *x* limit and determinant.

Second, we focus on the Fredholm determinant. The key observation is that we can decompose the operator |*V* |^{1/2}(∂−*A*_{0})^{−1}*U*|*V* |^{1/2} as follows. Direct computation using the kernel |*V* (*x*)|^{1/2}*G*(*x*;*y*)*U*(*y*)|*V* (*y*)|^{1/2}, where *G* is the kernel corresponding to (∂−*A*_{0})^{−1}, reveals
*J* is the Volterra integral operator given in the Volterra integral equation lemma 6.4 and *R* is the integral operator with kernel given by
*R* has separable kernel and is thus a finite rank operator and trace class. As we have *V* |^{1/2}(∂−*A*_{0})^{−1}*U*|*V* |^{1/2} is trace class by assumption, we deduce *J* is also trace class. Using the product decomposition id−*J*+*R*=(id−*J*)(id+(id−*J*)^{−1}*R* and provided *J* and (id−*J*)^{−1}*R*) are trace class, we have
*J*)^{−1}*R* is trace class as it has a separable kernel.

Third, we compute det_{1}(id−*J*). Using the relation *J* is a Volterra integral operator involving the Heaviside function, using the trace power formula lemma 3.4, we observe that tr *J*^{ℓ} will involve an integral over *H*(*y*_{1}−*y*_{2})*H*(*y*_{2}−*y*_{3})⋯*H*(*y*_{ℓ−1}−*y*_{ℓ})*H*(*y*_{ℓ}−1) of a product of Heaviside functions. Hence for all ℓ≥2, the quantity tr *J*^{ℓ} is zero as it is an integral of an integrand which is zero everywhere except on a subset of ℓ-dimensional measure zero. We also observe tr *J*=0. This follows using the unitary relation in lemma 5.5 and the assumption tr *V* ≡0 which imply the matrix trace of the kernel of *J* is continuous on the diagonal. Hence, we deduce det_{1}(id−*J*)=1.

Fourth, we establish that (id−*J*)^{−1}*R* is trace class and compute det_{1}(id+(id−*J*)^{−1}*R*). Using that *J*)^{−1}*R* has separable kernel *ε*>0 sufficiently small,
*ε*=1. Then removing the logarithms, this equals our expression above for the transmission coefficient. ▪

### Remark 6.6

We have the following observations on the results above: (i) at the core of the equivalence theorem is, for solutions decaying in the far field, on one hand we have the Fredholm determinant associated with the solvability of (id−*J*+*R*)*ϕ*=*O*, while on the other we have the transmission coefficient associated with the solvability of *A*_{0})|*V* |^{−1/2}*J*|*V* |^{1/2}=*V* and (∂−*A*_{0})|*V* |^{−1/2}*R*|*V* |^{1/2}=*O*; (iii) in essence, in the final calculation in the proof of theorem 6.5, we demonstrate that the trace of any power of (id−*J*)^{−1}*R* equals the trace of the corresponding power of _{1}(id+*ε*(id−*J*)^{−1}*R*) which is analytic in *V* |^{1/2}(∂−*A*_{0})^{−1}*U*|*V* |^{1/2} and (∂−*A*_{0})^{−1}*V* coincide. Hence, the Fredholm determinants of the two operators coincide, assuming the Fredholm determinant of id−(∂−*A*_{0})^{−1}*V* exists; (v) if tr *V* ≠0, then we need to include the factor *et al.* [15] for Hilbert–Schmidt operators. They use results from Gesztesy & Makarov [14], Gohberg *et al.* [26] and Gohberg *et al.* [19]; and (vii) Gesztesy *et al.* [15] do not assume strict hyperbolicity which we have done to keep the arguments as succinct as possible.

## 7. Distinct far fields

Our goal in this section is to show that the equivalence of the transmission coefficient and Fredholm determinant carries over to the case when the far field limits of the eigenvalue problem (∂−*A*_{0}−*V*)*Y* =*O* are distinct. The Evans function and transmission coefficient are well defined in this instance with only minor modification to their construction in §5—which is the standard approach. What underlies our approach here is that we decompose *A*_{0}+*V* in such a way that *A*_{0}=*A*_{0}(*x*) ensures the distinct far field limits are satisfied so *V* →*O* as *A*_{0} is constant and strictly hyperbolic, with the same hyperbolic splitting (the number of eigenvalues with positive real parts, say *k*, and negative real parts, consequently *n*−*k*). We can thus also assume *V* →*O* as *A*_{0} and *V* are continuous and uniformly bounded. To construct the Evans function or transmission coefficient, we do not need to perform this decomposition; however, the properties we derive for the solutions of the equations
*Y* _{0}=*A*_{0}*Y* _{0}. However as before in §5, given the identical hyperbolic splitting of the two far field limits of *A*_{0}, there is *k*-dimensional subspace of solutions that decays exponentially as *n*−*k*)-dimensional subspace of solutions that decays exponentially as *Z*_{0}=−*Z*_{0}*A*_{0}, there exist solutions collected as rows in the matrices *Y* ^{±} exist to the full problem (∂−*A*_{0}−*V*)*Y* =*O* such that

We can recover the unitary relations of lemma 5.5 for

### Lemma 7.1 (Diagonal block relation)

*The solutions* *and* *satisfy the relation*
*where* *D*_{±} *are constant matrices*.

### Proof.

We observe, by direct computation for any pair of solutions *Z*_{0}*Y* _{0})=*O* and so *Z*_{0}*Y* _{0} is constant. Asymptotically, we have *U*^{−} and rows of *W*^{−} are the right and left eigenvectors of *U*^{+} and rows of *W*^{+} are the right and left eigenvectors of *W*^{−}*U*^{−}=*O* and *W*^{+}*U*^{+}=*O*, we conclude

Note that the constant matrices *D*_{±} are in general no longer diagonal as *A*_{0}=*A*_{0}(*x*). We *assume* the constant matrices *non-singular*. Generically, this will be the case—see the discussion in §8. This guarantees the free transmission coefficient is non-zero. Using the diagonal block relation lemma 7.1, it guarantees the free Evans function is non-zero. This guarantees (∂−*A*_{0})^{−1} exists (this was automatic in the equal far field case under the strict hyperbolicity assumption on the constant matrix *A*_{0}). With this proviso, the block diagonal relation implies the framework we provided in §5 for establishing the relation between the Evans function and the transmission coefficient carries through essentially unchanged. We can mirror all the results in §5 up to and including the schematic formula between the Evans function and transmission coefficient. Then again with the same proviso, the scaled unitary solutions are defined as for the equal far field case in §5 by

### Lemma 7.2 (Unitary relations reprise)

*Unitary relations for* *and* *hold for distinct far fields*.

### Proof.

Making the corresponding change of variables in the diagonal block relation above generates the same diagonal block relation but with *D*_{±} on the right. The two matrices on the left of the new diagonal block relation are then inverses of each other and the unitary relation in reverse order follows.▪

The framework we provided in §6 for establishing the relation between the transmission coefficient and the Fredholm determinant det_{1}(id−|*V* |^{1/2}(∂−*A*_{0})^{−1}*U*|*V* |^{1/2}), where now *A*_{0}=*A*_{0}(*x*) carries through with only a couple of modifications which we now outline. Green’s function *G*=*G*(*x*,*y*) associated with (∂−*A*_{0})^{−1} has the same semi-separable form except that *A*_{0}=*A*_{0}(*x*). Again the unitary relations imply the jump condition for *G* across the diagonal line *y*=*x* is satisfied. If we assume tr *V* ≡0, then the trace of |*V* |^{1/2}(∂−*A*_{0})^{−1}*U*|*V* |^{1/2} or (∂−*A*_{0})^{−1}*V* is defined in the same way. The Volterra integral equation *J* defined via the kernel *A*_{0} being constant as follows. Direct computation using the unitary relations and that *A*_{0})*Y* ^{−}=*V* *Y* ^{−}. Premulti-plying this Volterra equation for *Y* ^{−} by |*V* |^{1/2} and using the definitions for *ϕ*^{−} and *ϕ*^{−}_{0} gives the result. The proof of the equivalence theorem 6.5 now follows step by step with the solutions *A*_{0}=*A*_{0}(*x*). We summarize these conclusions as follows.

### Theorem 7.3 (Equivalence reprised)

*Assume ∂−A*_{0} *is invertible, |V |*^{1/2}(*∂−A*_{0})^{−1}*U|V |*^{1/2} *is trace class, tr V ≡ 0, A*_{0}*=A*_{0}(*x) and V =V (x) are both continuous and uniformly bounded and V =V (x) is w*^{2}*-integrable. Further assume in each far field A*_{0} *is constant and strictly hyperbolic, with the same hyperbolic splitting, and V →O as* *. Then the transmission coefficient for the eigenvalue problem (∂−A*_{0}*−V)Y =O and Fredholm determinant of id−|V |*^{1/2}(*∂−A*_{0})^{−1}*U|V |*^{1/2} *are equal.*

## 8. Conclusion

We have established the equivalence of the Evans function and transmission coefficient for the eigenvalue problem (∂−*A*_{0}−*V*)*Y* =*O*, in the sense that the ratio of the Evans function to the free Evans function is equal to the ratio of the transmission coefficient to the free transmission coefficient. As we remarked at the end of §5, the Evans function and free Evans function are invariant to the unitary rescaling, as is the ratio of the transmission coefficient to the free transmission coefficient. Hence, the Fredholm determinant det_{1}(id−|*V* |^{1/2}(∂−*A*_{0})^{−1}*U*|*V* |^{1/2}), which equals the transmission coefficient with unitarily scaled solutions for which the free transmission coefficient is unity, equals the ratio of the transmission coefficient to the free transmission coefficient—whether the unitary scaling is used or not. These statements hold for equal or distinct far fields. In other words, we have established that
*A*_{0}−*V*)*Y* =*O* where *A*_{0}=*A*_{0}(*x*;λ) in general. The locale of the essential spectrum is determined by the values of λ for which the far field limits of *A*_{0} are no longer strictly hyperbolic—at least one eigenvalue of either limit becomes pure imaginary characterizing the continuous spectrum or the hyperbolic splitting no longer matches. Away from the essential spectrum, we must choose *A*_{0}=*A*_{0}(*x*;λ) suitably so that (∂−*A*_{0})^{−1} exists—as mentioned previously this is not an issue in the equal far field case. Hence, away from the essential spectrum, the free Evans function and free transmission coefficients are bounded and non-zero. As the Evans function is analytic in that region, the product of the free Evans function and the ratio of the transmission and free transmission coefficients is analytic there, as well as the product of the free Evans function and the Fredholm determinant. Zeros of these product quantities must thus coincide in this region. Those zeros coincide with pure point eigenvalues with coincident multiplicity. We remark that for eigenvalue problems of the form (∂−*A*_{0}−*V*)*Y* =*O* that arise in the study of the stability of travelling waves, the origin is an eigenvalue associated with translation invariance. In some cases, the origin is embedded in the essential spectrum within which further analysis of all the discriminants above is required. Some instructive explicit examples illustrating the relation, between the Evans function and transmission coefficient can be found in Kapitula & Sandstede [13] and Kapitula [4], between the transmission coefficient and Fredholm determinant can be found in Simon [12, p. 51] and between the Evans function and 2-modified Fredholm determinant in Gesztesy *et al.* [15] and Gesztesy *et al.* [16].

For the eigenvalue problem (∂−*A*_{0}−*V*)*Y* =*O* where *A*_{0}=*A*_{0}(*x*;λ) and *A*_{0}−*V* parametrized by λ. For example, the analyticity of the Evans function means we can conduct a global search for eigenvalues via contour integration and the residue theorem in any subregion away from the essential spectrum. Here, the family of operators consists of those associated with values of λ parametrizing the boundary contour of the subregion. Hence the object of interest is a determinant line bundle. See Quillen [27], Jost [28, Section 6.9], Deng & Nii [29] and Alexander *et al.* [2] for constructions of such line bundles. In Alexander *et al.*’s line bundle construction, the fibres are explicitly the Evans function. These constructions suggest we define, away from the essential spectrum, the determinant of the unbounded Fredholm operator ∂−*A*_{0}−*V* by the Evans function corresponding to the problem (∂−*A*_{0}−*V*)*Y* =*O*, i.e. we set
*V* ≡0, we have established that

Finally, in the study of the linear stability of travelling wave solutions to nonlinear partial differential equations often numerical simulation is required in the search for eigenvalues. Indeed, one of the main motivations for establishing the connection between the Evans function and Fredholm determinant was the need to compute the stability of multidimensional travelling waves (e.g. [32]). Gesztesy *et al.* [16, Section 4] established important convergence results for suitable Galerkin approximations for the multidimensional problem. Humpherys & Zumbrun [33], Ledoux *et al.* [34] and Ledoux *et al.* [35] have also designed numerical methods for computing the Evans function for the multidimensional case. The computation of Fredholm determinants also naturally arise in the computation of the Maslov index for linear symplectic systems, in particular in the multi-dimensional case which relies on computing flows in the Fredholm Lagrangian Grassmannian manifold [36,37]. Finally, a natural question to ask that is still open is, as numerical tools, which of the Evans function, transmission coefficient, Fredholm determinant or Galerkin-type direct projection methods has the optimal complexity? See Karambal [38] for some results in this direction. A comprehensive study of this complexity issue would be a useful follow-on project.

## Funding statement

I.K. would like to thank the Numerical Algorithms and Intelligent Software Centre (NAIS) funded by the UK EPSRC grant EP/G036136 and the Scottish Funding Council.

## Acknowledgements

We thank the referees for their very helpful hints, comments and suggestions that have substantially improved our original manuscript. We also thank the third referee who led us to consider the determinant of the unbounded operator in our Conclusions also as the coordinate free ratio mentioned.

## Appendix A. Proof of the Hilbert–Schmidt class lemma

### Proof.

First, given *G*↦*K* from *φ*_{m}}_{m≥1} are an unitary basis for *G*=*G*(*x*;*y*) above we find,

## Appendix B. Proof of the trace formula lemma

### Proof.

For any *K*_{1},*K*_{2},…,*K*_{ℓ} are Hilbert–Schmidt operators with canonical respective kernels *G*_{1},*G*_{2},…,*G*_{ℓ}. As in the proof of the Hilbert–Schmidt class lemma above, there exists a double sequence of constants *K* with continuous kernel *G* exactly follows the argument above with ℓ=1.▪

- Received August 5, 2014.
- Accepted December 15, 2014.

© 2015 The Authors. Published by the Royal Society under the terms of the Creative Commons Attribution License http://creativecommons.org/licenses/by/4.0/, which permits unrestricted use, provided the original author and source are credited.