## Abstract

In this paper, a new classification of Sturm–Liouville differential equations with complex coefficients is given. Compared with the corresponding result of Brown *et al*., this classification reveals the great effects of rotation angle and it is independent of the rotation angles. Moreover, the asymptotic behaviours of functions in the maximal domain are presented and *J*-self-adjoint extensions associated with the differential equations are characterized.

## 1. Introduction

Consider the Sturm–Liouville differential equation
1.1where *p*,*q* are complex functions, *p*(*x*)≠0 and *w*(*x*)>0 a.e. on (*a*,*b*), , 1/*p*,*q*,*w* are all locally integrable on (*a*,*b*), *λ* is the so-called spectral parameter.

One of the aims of the present paper is to study the classification of equation (1.1) according to the number of square integrable solutions of equation (1.1) in suitable weighted integrable spaces. This type classification of differential equations plays an important role in the spectral theory of differential operators as it can tell us how to obtain the operator realizations associated with the differential equations. The study of this problem has a long history since the pioneering work of Weyl (1910). When *p*(*x*) and *q*(*x*) are all real-valued, Weyl classified equation (1.1) into the *limit-point* and *limit-circle* cases by introducing the *m*(*λ*)-functions. This work has been greatly developed and generalized to formally symmetric higher order differential equations and Hamiltonian differential systems; for this line, the reader is referred to Dunford & Schwartz (1963), Eastham (1979), Hinton & Shaw (1981, 1983, 1984), Weidmann (1987), Krall (1989*a*,*b*), Everitt & Markus (1999), Brown *et al.* (2003), Zettl (2005) and references therein.

The same problem was also studied by Sims (1957) for the case where *q*(*x*) is complex-valued. He considered the case where *p*(*x*)=*w*(*x*)≡1, Im *q*(*x*) is semi-bounded and classified equation (1.1) into three cases. Recently, this work was extensively generalized by Brown *et al.* (1999, 2003).

In order to state clearly the classification given in Brown *et al.* (1999), we introduce some notations. Define
1.2and assume that , where denotes the closed convex hull (i.e. the smallest closed convex set containing the exhibited set). Then for each point on the boundary ∂*Ω*, there exists a line through this point such that each point of *Ω* either lies in the same side of this line or is on it. Let *K* be a point on ∂*Ω*. Denote by *L* an arbitrary supporting line touching *Ω* at *K*, which may be the tangent to *Ω* at *K* if it exists. We then perform a transformation of the complex plane *z*↦*z*−*K* and a rotation through an appropriate angle *θ*∈(−*π*,*π*], so that the image of *L* coincides with the new imaginary axis and the set *Ω* is contained in the new right non-negative half-plane. Therefore, for all *x*∈(*a*,*b*) and ,
1.3

For such admissible values of *K* and *θ*, set
1.4and define the corresponding half-plane
1.5Then, . Let be the Hilbert space with inner product and the norm ∥*y*∥=〈*y*,*y*〉^{1/2} for . We call a solution *y* of equation (1.1) a -solution if .

With these definitions and using a nesting circle method based on the methods of both Weyl (1910) and Sims (1957), Brown *et al.* (1999) divided equation (1.1) into three cases with respect to the corresponding half-planes *Λ*_{θ,K} as follows. The uniqueness referred to in the theorem and following sections is only up to constant multiples.

### Theorem 1.1

*(see Brown et al. (1999, theorem 2.1)). Given a (θ,K)∈S, the following three distinct cases are possible.*

*I: For all λ∈Λ*_{θ,K}*, equation (1.1) has a unique solution y satisfying*1.6*and this is the only solution satisfying**;**II: For all λ∈Λ*_{θ, K}*, all solutions of equation (1.1) belong to**and there exists a unique non-trivial solution of equation (1.1) that satisfies equation (1.6).**III: For all λ∈Λ*_{θ, K}*, all solutions of equation (1.1) satisfy equation (1.6).*

### Remark 1.2

If *q*(*x*) and *p*(*x*) are real-valued, then and (*θ*,*K*)=(*π*/2,0)∈*S*. Hence Re{e^{iθ}*p*(*x*)}=Re{e^{iθ}(*q*(*x*)−*K* *w*)}≡0. So case II is vacuous. This means that the classification mentioned above reduces to Weyl’s limit-point, limit-circle classification.

One sees from theorem 1.1 that, unlike the classification of equation (1.1) with real coefficients, the three cases in theorem 1.1 formally depend on the choice of (*θ*,*K*) in *S* or the half-planes *Λ*_{θ,K}. Indeed, cases II and III depend on the choice of (*θ*,*K*). See theorem 2.3 in §2. One of the main results in this paper is to give a new classification of equation (1.1) which is independent of the rotation angles (or the half-planes). See theorem 3.2 in §3. This kind of classification gives detailed properties for -solutions of equation (1.1). Applying theses properties we obtain the asymptotic behaviours of functions in the maximal domain when equation (1.1) is in cases I or II in §4. (theorem 4.1). Furthermore, the asymptotic behaviours offer a simple tool to characterize the *J*-self-adjoint operator realizations associated with equation (1.1). See theorems 5.1 and 5.2, which are the generalization of the results theorem 4.4 in Brown *et al.* (1999) and theorem 10.26 in Edmunds & Evans (1987), respectively.

Following this section, §2 gives some preliminary results for equation (1.1) and §3 presents a new classification for equation (1.1). Section 4 studies the asymptotic behaviours and §5 deals with the *J*-self-adjoint operator realizations associated with equation (1.1).

## 2. Preliminary results

Let *Ω*, *S* and *Λ*_{θ,K} be defined as in §1. Since each *Λ*_{θ, K} is a half-plane, for (*θ*_{j},*K*_{j})∈*S*, *j*=1,2 with *θ*_{1}≠*θ*_{2}(mod *π*), it holds that
2.1Note that equation (1.3) implies that for and *x*∈(*a*,*b*),
2.2Let *r*→0 and in equations (1.3) and (2.2), respectively, we have that:

### Lemma 2.1

*For each ( θ,K)∈S and λ∈Λ _{θ,K}, there exists δ_{λ}(θ*)>0

*such that*2.3

Using variation of parameters formula, we can verify that if all solutions of equation (1.1) belong to for some , then it is true for all . This also means that:

### Lemma 2.2

*If there exists a* (*θ*_{0},*K*_{0})∈*S such that equation (1.1) is in case I with respect to (w.r.t.)* *Λ*_{θ0,K0}, *then equation (1.1) is in case I w.r.t. Λ*_{θ, K} *for each* (*θ*,*K*)∈*S*.

This indicates that case I is independent of the choice of (*θ*,*K*)∈*S*. However, cases II and III depend on the choice of (*θ*,*K*)∈*S* in general, that is, there may exist (*θ*_{1},*K*_{1}),(*θ*_{2},*K*_{2})∈*S* such that equation (1.1) is in case II w.r.t. *Λ*_{θ1,K1} and is in case III w.r.t. *Λ*_{θ2,K2}. In order to make clear the dependence, we introduce the admissible angle set *E* defined by
2.4Note that for fixed *θ*_{0}∈*E*, the *K* such that (*θ*_{0},*K*)∈*S* may be not unique. The exact dependence of cases II and III on (*θ*,*K*) is given in the following theorem.

### Theorem 2.3

*[cf. Sun & Qi (*2010*, theorem 2.1)] If there exists a (θ*_{0}*,K*_{0}*)∈S such that equation (1.1) is in case* II *w.r.t. Λ*_{θ0,K0}*, then equation (1.1) is in case* II *w.r.t. Λ*_{θ,K} *for all (θ,K)∈S except for at most one θ*_{1}*∈E (in the sense of mod π) such that equation (1.1) is in case* III *w.r.t. Λ*_{θ1,K1}*, where (θ*_{1}*,K*_{1}*)∈S.*

## 3. A new classification

Let *E* be defined as in equation (2.4). We will find that the number of the elements in *E* determines the dependence of cases II and III on (*θ*,*K*)∈*S*. In fact, if *E* has only one point, then the classification of Brown *et al*. in theorem 1.1 is independent of the choice of (*θ*,*K*)∈*S*. In order to give a new classification which is independent of (*θ*,*K*)∈*S*, we need only to consider the case when *E* has more than one point. In what follows, we assume that *E* has more than one point, i.e. there exist at least *θ*_{1},*θ*_{2}∈*E* with *θ*_{1}≠*θ*_{2} (mod *π*). To begin with, we prepare some properties of the set *E*.

### Lemma 3.1

*Let E be defined as in equation (2.4)*.

*If E has more than one point, then E is a sub-interval of*(−*π*,*π*].*If E has more than one point, then for each*,*there exist θ*_{1},*θ*_{2}∈*E with θ*_{1}<*θ*_{2}*such that for θ*∈(*θ*_{1},*θ*_{2})⊂*E*,*λ*∈*Λ*_{θ, K}, where (*θ*,*K*)∈*S*.

### Proof.

(i) Let *θ*_{1},*θ*_{2}∈*E* with *θ*_{1}≠*θ*_{2} (mod *π*), *θ*_{1}<*θ*_{2} and *K*_{1},*K*_{2} be the points on ∂*Ω* such that (*θ*_{j},*K*_{j})∈*S*, *j*=1,2. We claim that [*θ*_{1},*θ*_{2}]⊂*E*. For the case *K*_{1}=*K*_{2}=*K*, we prove that (*θ*,*K*)∈*S* for all *θ*∈(*θ*_{1},*θ*_{2}). Set
It follows from the definition of *S* that for *j*=1,2 and *r*≥0, Re{e^{iθj}((*q*(*x*)/*w*(*x*))+*rp*(*x*)−*K*)}≥0 on (*a*,*b*), or equivalently, . Without any confusion, we write *γ*(*x*,*r*,*K*) (respectively, *γ*_{j}(*x*,*r*,*K*)) as *γ* (respectively, *γ*_{j}). If we set
then and can be expressed as
3.1by using the formulae for *j*=1,2. This equality gives that
3.2for *θ*∈(*θ*_{1},*θ*_{2}). Since for *j*=1,2 implies −*π*/2≤*γ*_{1},*γ*_{2}≤*π*/2(mod 2*π*), we have from *π*≥*θ*_{2}>*θ*_{1}>−*π*, *θ*_{2}−*θ*_{1}≠*π* and *θ*_{2}−*θ*_{1}=*γ*_{2}−*γ*_{1} that 0<*θ*_{2}−*θ*_{1}<*π*. Consequently, for *θ*∈(*θ*_{1},*θ*_{2}), we obtain
Therefore, each term in the right-hand side of equation (3.2) is non-negative, so on (*a*,*b*). That is, for *r*≥0 and *x*∈(*a*,*b*),
which implies (*θ*,*K*)∈*S* for *θ*∈(*θ*_{1},*θ*_{2}).

In case *K*_{1}≠*K*_{2}, we choose *μ*_{0}∈*Λ*_{θ1,K1}∩*Λ*_{θ2,K2}. Then it holds that
on (*a*,*b*) for *j*=1,2 and *r*≥0 by the definition of *Λ*_{θ,K}, or . Then, the similar proof in equations (3.1) and (3.2) yields that
3.3Let *L* be the line defined by
3.4for fixed *θ*∈(*θ*_{1},*θ*_{2}). One sees from equations (3.3), (3.4) and the definition of *Ω* that . Set *d*=dist(*L*,∂*Ω*) and let *K*∈∂*Ω* be a point such that *d*=dist(*K*,*L*). Since
3.5we have that Re{e^{iθ}(*μ*−*μ*_{0})}=dist(*μ*,*L*)≥dist(*K*,*L*)=Re{e^{iθ}(*K*−*μ*_{0})} for *μ*∈*Ω*, hence Re{e^{iθ}(*μ*−*K*)}=Re{e^{iθ}(*μ*−*μ*_{0})}−Re{e^{iθ}(*K*−*μ*_{0})}≥0, or
on (*a*,*b*) for *r*≥0 and *θ*∈(*θ*_{1},*θ*_{2}), which means (*θ*,*K*)∈*S*, or *θ*∈*E*. This proves (i) of lemma 3.1.

(ii) For , choose (*θ*_{0},*K*_{0})∈*S* and *δ*_{0}>0 such that *λ*_{0}∈*Λ*_{θ0,K0} and
Since *E* has more than one point, we can choose such that . Without loss of generality, we suppose that . It follows from the conclusion of (i) in lemma 3.1 that . For each , there exists a point *K*(*θ*)∈∂*Ω* such that (*θ*,*K*(*θ*))∈*S*. By the definition of *S* we have that
3.6If we set *r*(*θ*)=|*K*(*θ*)−*K*_{0}| and *K*(*θ*)−*K*_{0}=*r*(*θ*) e^{iη(θ)}, then equation (3.6) means that
This together with *θ*>*θ*_{0} gives that *θ*+*η*(*θ*)≥*π*/2≥*θ*_{0}+*η*(*θ*) (mod 2*π*), hence *θ*_{0}+*η*(*θ*)→*π*/2 (mod 2*π*) as *θ*→*θ*_{0}+0. We claim that *r*(*θ*) is bounded in a right-neighbourhood of *θ*_{0}. Suppose, on the contrary, there exists a sequence, say, {*θ*_{n}} such that *θ*_{n}→*θ*_{0}+0 and as . Choose such that *η*_{0}−*θ*_{0}+*π*/2<*π* and a corresponding point *K*(*η*_{0})∈∂*Ω* such that (*η*_{0},*K*(*η*_{0}))∈*S*, we have that
which is a contradiction. Since *r*(*θ*) is bounded and *θ*+*η*(*θ*)→*π*/2 (mod 2*π*) as *θ*→*θ*_{0}+0, we have that as *θ*→*θ*_{0}+0. Hence
as *θ*→*θ*_{0}+0. Therefore, there exists such that for all *θ*∈(*θ*_{0},*ξ*), Re{e^{iθ}(*λ*_{0}−*K*(*θ*))}<0. This means that *λ*_{0}∈*Λ*_{θ, K} for *θ*∈(*θ*_{0},*ξ*). This completes the proof. ■

Now, we state the new accurate classification of equation (1.1) under the condition that *E* has more than one point. This classification is independent on the choice of (*θ*,*K*)∈*S*.

### Theorem 3.2

*Suppose that E has more than one point. Then, the following three distinct cases are possible.*

*Case one: For all* *, equation (1.1) has a unique solution* *and this solution y also satisfies
*3.7

*Case two: For all* *, all solutions of equation (1.1) belong to* *but only a unique solution satisfies equation (3.7);*

*Case three: For all* *, all solutions of equation (1.1) satisfy equation (3.7).*

### Proof.

For fixed , by lemma 2.2 we have two cases to be considered:

There exists a unique solution of equation (1.1) belonging to ;

All solutions of equation (1.1) belong to .

Suppose that the case (a) occurs. This means that equation (1.1) is in case I for all (*θ*,*K*)∈*S*. By (ii) of lemma 3.1 there exist (*θ*_{1},*K*_{1}),(*θ*_{2},*K*_{2})∈*S* with *θ*_{1}≠*θ*_{2} (mod *π*) such that *λ*∈*Λ*_{θ1,K1}∩*Λ*_{θ2,K2}, hence there exists a unique solution *y* of equation (1.1) such that
3.8hold simultaneously. Set *ρ*(*x*)=|*q*(*x*)−*λ* *w*(*x*)| and
3.9Then lemma 2.1 ensures that on (*a*,*b*). It follows that
3.10hence
3.11Similarly, we have that
3.12It follows from equation (3.8) that
Since |*q*|≤|*q*−*λ* *w*|+|*λ*|*w*| and , we have . With the similar proof as above and using equations (3.8) and (3.12), we have that . This proves that equation (1.1) is in case one.

Suppose that the case (b) holds for . It follows from theorem 2.3 that there are two subcases to be considered:

(b_{1}) There exists at most one *θ*_{0}∈*E* such that equation (1.1) is in case III w.r.t. *Λ*_{θ0,K0} and equation (1.1) is in case II w.r.t. *Λ*_{θ,K} for all (*θ*,*K*)∈*S* with *θ*≠*θ*_{0} (mod *π*);

(b_{2}) For all (*θ*,*K*)∈*S*, equation (1.1) is in case III w.r.t. *Λ*_{θ,K}.

For the subcase (b_{1}) we prove first that for given , there exist with (mod *π*) and there exists a unique solution of equation (1.1) satisfying equation (3.8) simultaneously for *θ*, . In fact, by (ii) of lemma 3.1 there exist (*θ*_{j},*K*_{j})∈*S*, *j*=1,2,3 such that
*θ*_{j}≠*θ*_{0} (if such *θ*_{0} exists) for *j*=1,2,3 and . Since equation (1.1) is in case II w.r.t. (*θ*_{1},*K*_{1}), there exists a unique solution *y* of equation (1.1) satisfying equation (3.8) with *j*=1 (this solution depends on *θ*_{1}∈*E*). Define *α*=*α*(*x*) as in equation (3.9). Then, solving from the equations
we find that
3.13with , . Now for the solution *y* relate to *θ*_{1}, we obtain by equation (3.13) that
hence
3.14Similarly, we have
3.15Therefore, the solution *y* of equation (1.1) satisfies equation (3.8) simultaneously for *θ*_{2},*θ*_{3}∈*E* with *θ*_{2}≠*θ*_{3} (mod *π*) by equations (3.14) and (3.15). Next, the similar proof as in the case (a) proves that this solution *y* satisfies equation (3.7). Clearly, such a solution which satisfies equation (3.7) is unique since equation (1.1) is in case II w.r.t. *Λ*_{θj,Kj}, *j*=1,2, hence equation (1.1) is in case two.

Finally, we prove that the subcase (b_{2}) implies equation (1.1) is in case three. Using lemma 3.1 we can choose (*θ*_{j},*K*_{j})∈*S* such that *θ*_{1}≠*θ*_{2} (mod *π*) and *λ*∈*Λ*_{θ1K1}∩*Λ*_{θ2K2}. Since equation (1.1) are in case III w.r.t. *Λ*_{θj,Kj}, *j*=1,2, all solutions of equation (1.1) satisfy equation (3.8) for both *j*=1 and *j*=2. Therefore, the similar argument as above yields that all solutions of equation (1.1) satisfy equation (3.7). This gives that equation (1.1) is in case three. ■

### Remark 3.3

If *E* has more than one point, then the relationship between the classifications in theorems 1.1 and 3.2 can be stated clearly as follows:

equation (1.1) is in case I if and only if equation (1.1) is in case one;

equation (1.1) is in case II w.r.t.

*Λ*_{θ,K}for some*θ*∈*E*if and only if it is in case two; andequation (1.1) is in case III w.r.t.

*Λ*_{θ,K}for all*θ*∈*E*if and only if it is in case three.

Many assumptions on coefficients can ensure that *E* has more than one point. For example, if *p*(*x*)>0 and *q*(*x*)≥*q*_{0}*w*(*x*), then *E* has more than one point. *E* has only one point if and only if the boundary of *Ω* is one or two straight lines. For this case, the classification of equation (1.1) in theorem 3.2 may be not true.

For example, consider equation (1.1) with and *λ*=*i*. We conclude from Hille (1969, theorem 10.1.5, p. 503) that equation (1.1) is in the limit point case at . Denote by *y*(*x*,*i*) the unique solution of equation (1.1) with *λ*=*i* such that . It follows from Hille (1969, exercise 10, p. 504) that *y*′, . So, this equation is not in any one of the three cases in theorem 3.2. We note that *E* has only one point *π*/2 for this equation. Examples of complex-valued *p* and *q* where cases two and three occur are given in Sun & Qi (2010).

## 4. Asymptotic behaviours

In this section, we will give asymptotic behaviours of the elements in the maximal domain of the formal differential operator *τ* defined on the interval with 0 being a regular endpoint and being implicitly a singular endpoint. All results in this section can be proved with the similar argument for any singular endpoint, left or right on an arbitrary interval (*a*,*b*), where . The interval considered here is only for the sake of simplicity. Recall that equation (1.1) on (*a*,*b*) is said to be *regular* at *a* if 1/*p*, *q* and *w* are integrable on (*a*,*c*) for some (and hence any) *c*∈(*a*,*b*), and *singular* at *a* otherwise; and the regularity and singularity at *b* are defined similarly. Note that the regularity (respectively, singularity) of an endpoint is solely determined by the integrability (respectively, non-integrability) of the coefficients in equation (1.1) at the endpoint, not the finiteness (respectively, infiniteness) of the endpoint, as has already been remarked by Atkinson (1964, §9.1). See also Zettl (2005, theorem 2.3.1). Consider the formal differential operator *τ* associated with equation (1.1)
4.1

Let denote the set of complex-valued functions which are absolutely continuous on each compact sub-interval of . We define the domain of the maximal operator associated with *τ* as follows:
4.2

### Theorem 4.1

*(i) If equation (1.1) is in case I w.r.t. some (θ,K)∈S, then
*4.3*for all* *. If, in addition, E has more than one point, then* *,* *for all* *, and for* *,
*4.4

*(ii) If equation (1.1) is in case II w.r.t. some (θ,K)∈S, then equation (4.3) holds for* *,
*4.5*If, in addition, E has more than one point, then equation (4.4) holds for y*_{1}*,* *.*

We will use spectral theory of Hamiltonian differential systems to prove theorem 4.1, so we first prepare some known results for the Hamiltonian differential system
4.6on the valued (column) functions *Y* =(*u*^{T},*v*^{T})^{T}, where *u*,*v* are valued functions, *u*^{T} is the transpose of *u*, *A*,*B*,*C*,*W*_{1} and *W*_{2} are locally integrable, complex-valued *n*×*n* matrices on , *B*,*C*,*W*_{1},*W*_{2} are Hermitian matrices and *W*_{1}(*t*)>0,*W*_{2}(*t*)≥0 on , *ξ* is the so-called spectral parameter. Assume that the *definiteness condition* (e.g. Atkinson 1964, ch. 9, p. 253) holds:
where *W*=*diag*(*W*_{1},*W*_{2}). Let be the space of Lebesgue measurable 2*n*-dimensional functions *f* satisfying . We say that equation (4.6) is in the *limit-point case* at infinity if there exists exactly *n*’s linearly independent solutions of equation (4.6) belong to for *ξ*=±*i*. Particularly, if *n*=1 and *A*,*B*,*C* are real functions, then equation (4.6) is in the limit point case at infinity if and only if there exists a unique solution of equation (4.6) belonging to for *ξ*=*i* or *ξ*=−*i*.

Let be the maximal domain associated with equation (4.6), i.e. if and only if and there exists an element such that
4.7It is well known (cf. Hinton & Shaw 1983; Krall 1989*a*) that equation (4.6) is in the limit point case at infinity if and only if
4.8for , and for each with Im *ξ*≠0 there exists a Green function *G*(*t*,*s*,*ξ*) such that for , equation (4.7) has an -solution *Y* given by
4.9

### The proof of theorem 4.1.

The proof of (i). Suppose that *τ* is in case I w.r.t. (*θ*_{0},*K*_{0})∈*S*. Choose *λ*_{0}∈*Λ*_{θ0,K0}. Then
4.10for some *δ*_{0}>0. Set
4.11Set *u*=*y*, *v*=−ie^{iθ0}*py*′, then equation (1.1) (with *λ*=*λ*_{0}) is transformed into the Hamiltonian differential system
4.12with the new spectral parameter *ξ*=*i*, where
4.13This is the Hamiltonian differential system (4.6) with *n*=1, *A*(*x*)≡0 and *ξ*=*i*. Clearly, *w*_{1}=Re{e^{iθ0}(*q*−*λ*_{0} *w*)}≥*δ*_{0} *w*>0, by equation (4.10). We note further that the coefficients of the Hamiltonian system (4.12) are real functions. It is easy to verify that the definiteness condition holds for the system (4.12). Therefore, equation (1.1) is in cases I or II w.r.t. (*θ*_{0},*K*_{0})∈*S* if and only if equation (4.12) is in the limit point case at .

Let be the maximal domain associated with equation (4.12), i.e. if and only if and there exists an element such that
4.14For , set
4.15Then *y*_{0} satisfies
4.16

Set *u*_{0}=*y*_{0}, *v*_{0}=−ie^{iθ0}*py*′_{0}. Then (*u*_{0},*v*_{0}) satisfies
4.17Conversely, if (*u*,*v*) satisfies equation (4.17), then *y*=*u* solves equation (4.16).

Let *g*_{0} be given in equation (4.15). Consider the equation (4.17), we get from equation (4.9) that equation (4.17) has a solution (*u*_{1},*v*_{1})^{T} such that , and *v*_{1}=−ie^{iθ0}*p* *u*′_{1}. Set *y*_{1}=*u*_{1}. Then *y*_{1} satisfies equation (4.16), hence (*τ*−*λ*_{0})(*y*_{0}−*y*_{1})=0. Note that and *w*_{1}≥*δw* implies that . Thus, *y*_{1}−*y*_{0} is an -solution of *τy*=*λ*_{0}*y*. Since *τ* is in case I w.r.t. (*θ*_{0},*K*_{0}), it follows from equation (1.6) that and . This together with and gives that and . In fact, we have proved that for ,
4.18where *α*_{1} and *β*_{1} are defined in equation (4.11). Recall that , or and *w*_{1}≥*δw* implies . It follows equation (4.17) that (*y*_{0},*v*_{0}) satisfies equation (4.14) with *f*=*iy*_{0}+*f*_{1} and *g*=*iv*_{0}. This yields that
4.19

Note that if and only if . Then for , we have from equation (4.19) that with , hence 4.20

Let . Since equation (4.12) is in the limit-point case at infinity and for *j*=1,2 with *v*_{1}=−ie^{iθ0}*py*′_{1} and by equations (4.19) and (4.20), respectively, we get from equation (4.8) that
4.21as . This proves that equation (4.3) holds. Furthermore, for *y*_{1}, , since by equation (4.19) with *v*_{j}=−ie^{iθ0}*py*′_{j}, *j*=1,2, equation (4.8) also gives that
as , or
4.22as . If, in addition, *E* has more than one point, then there exists *θ*_{1}∈*E* such that
4.23and equation (1.1) is in case I w.r.t. (*θ*_{1},*K*_{1})∈*S* by lemmas 2.2 and 3.1, hence, the above argument proves that equation (4.22) holds for with *θ*_{0} replaced by *θ*_{1}. Therefore, we have
4.24as . This clearly gives equation (4.4) for since *θ*_{1}≠*θ*_{0} (mod *π*).

Now, it remains to prove and for . Let *θ*_{1} be defined as in equation (4.23). Then, the similar proof as in equation (4.18) gives that for ,
4.25where *α*_{2} and *β*_{2} are defined as in equation (4.11) with *θ*_{0} replaced by *θ*_{1}. Therefore, equations (4.18), (4.25) and (3.10) together yield that and , or and since . This completes the proof of (i) for this theorem.

The proof of (ii). Suppose that *τ* is in case II w.r.t. (*θ*_{0},*K*_{0})∈*S*. In fact, the above proof in equation (4.19) proves that implies that with *v*=−ie^{iθ0}*py*′. Since the Hamiltonian system (4.12) is also in the limit-point case at for this case, the similar proof as in equation (4.21) proves that equation (4.3) holds for all .

In addition, if *E* has more than one point, then equation (4.3) holds for *θ*=*θ*_{1} and , where *θ*_{1} is defined as in equation (4.23). The similar argument in equation (4.24) proves equation (4.4). ■

### Remark 4.2

If *q*(*x*) and *p*(*x*) are both real-valued, then if and only if . In this case, case I and cases II and III reduce to Weyl’s limit-point, limit-circle cases, respectively, and it is known that equation (1.1) is in the limit point case at if and only if
4.26for . Therefore, equation (4.3) is a generalization of the necessity of equation (4.26).

## 5. Operator realizations

The *J*-self-adjoint extension of Sturm–Liouville operators is one of the important research fields in spectral theory of differential operators (see Race (1985), Edmunds & Evans (1987) for details). In this section, we will give the characterization of domains of the *J*-self-adjoint operators under cases I and II.

A densely defined linear operator *T* in a Hilbert space *H* is said to be *J*-symmetric if *JTJ*⊆*T**, and *T* is said to be *J*-self-adjoint if *JTJ*=*T** (see Edmunds & Evans (1987, pp. 114–115) for more details).

Let be defined as in equation (4.2), . Let *T*_{0}(*τ*), *T*_{1}(*τ*) be the operators associated with *τ* with the domains
5.1respectively. We call *T*_{0}(*τ*) and *T*_{1}(*τ*) the pre-minimal and maximal operators associated to *τ*. It is known that (and hence ) is dense in . This section characterizes the *J*-self-adjoint extension of *T*_{0}(*τ*) associated with equation (1.1), which also is a restriction of the maximal operator *T*_{1}(*τ*) on . It is closely related to the famous Glazman–Krein–Naimark theory in this field and has been extensively researched both for formally symmetric differential equations (Everitt & Giertz 1975; Weidmann 1987; Zettl 2005; Wang *et al.* 2009) and Hamiltonian differential systems (Hinton & Shaw 1984; Krall 1989*a*,*b*).

As we know, one can get the *J*-self-adjoint realizations by imposing boundary conditions for functions in on endpoints of the interval under consideration and the number of independent boundary conditions depends on the cases in the classification of theorem 1.1. Among these cases, *J*-self-adjoint realizations in case I are comparatively simple since equation (4.3) gives a natural boundary condition of functions in at the endpoint *b*.

When *p*(*x*) is real and *Imq*(*x*) is semi-bounded, theorem 10.14 in Edmunds & Evans (1987, p. 150) points out that each *J*-self-adjoint realization associated with *τ* has the domain
5.2if *τ* is in case I. This result is an analogue of the corresponding result for the case when *q*(*x*) and *p*(*x*) are real. When *p*,*q* are both complex-valued, with the aid of resolvent operators Brown *et al*. proves that the set is a *J*-self-adjoint realization associated with *τ* if *τ* is in case I under the condition Re for some *θ*∈*E* (see Brown *et al.* (1999, theorems 4.4 and 4.5)).

Applying the asymptotic behaviour obtained in theorem 4.1, we will give all *J*-self-adjoint realizations associated with *τ* if *τ* is in case I, in which the restriction Re is removed. When *τ* is in case II, we give a class of operator realizations associated with *τ* not by imposing boundary conditions at the singular endpoint but a restriction of a suitable Sobolev subspace of .

### Theorem 5.1

*If equation (1.1) is in case I, then each J-self-adjoint realization T*_{α} *associated with τ is given by
*5.3

### Proof.

Let *T*_{α} be defined as in equation (5.3). We prove that *T*_{α} is *J*-self-adjoint, that is
5.4where *J* is the usual conjugation, i.e. . It is easy to verify that *JτJy*=*τ*^{+}*y*, where *τ*^{+} is the so-called *Lagrange* or *formal adjoint* of *τ* defined by

We first prove *T*_{α}⊂*JT**_{α}*J*, i.e.
5.5For and all ,
This equality together with the boundary condition and *B*_{α}*y*(0)=0 gives
5.6It follows from equation (4.3) of theorem 4.1 that , hence and *JT*_{α}*Jy*_{0}=*T**_{α}*y*_{0} by equation (5.6). This proves that *T*_{α}⊂*JT**_{α}*J*.

Now we prove that *T**_{α}⊂*JT*_{α}*J*. Similarly as in equation (5.1), define and associated with *τ*^{+}. Clearly,
5.7

With the similar argument as in Edmunds & Evans (1987, theorem 10.7, pp. 144 and 145) we have that

Let . Since *T*_{0}(*τ*)⊂*T*_{α}(*τ*), we have that . So, , or , hence, as for by equation (4.3) of theorem 4.1. This gives that for ,
As *y* satisfies the boundary condition *B*_{α}*y*(0)=0, we have that
Choose with *y*(0)≠0, we get , i.e. , which implies , hence *JT*_{α}*J*=*T**_{α}.

Conversely, suppose that *T* is a *J*-self-adjoint realization associated with *τ*. Let *y*_{0} be fixed such that . Set and . Then the above argument gives that for ,
hence *T*⊂*T*_{α}, or *T**_{α}⊂*T**. Since *T*_{α}=*JT**_{α}*J*⊂*T*=*JT***J*, we have *T*=*T*_{α}. ■

### Theorem 5.2

*If equation (1.1) is in case II w.r.t. some (θ,K)∈S, then the set
*5.8*is a domain of a J-self-adjoint realization associated with τ. If, in addition, E has more that one point, then
*5.9

### Proof.

for by the first part of (ii) in theorem 4.1. Then the similar argument as in the proof of theorem 5.1 proves that *T*_{α} is *J*-self-adjoint. The second conclusion follows from the second part of (ii) in theorem 4.1 in a similar way. ■

## Acknowledgements

This research was partially supported by the NSF of China (grant no. 10801089) and NSF of Shandong Province (grant nos. Y2008A02 and ZR2009AQ010).

- Received June 1, 2010.
- Accepted November 18, 2010.

- This journal is © 2010 The Royal Society