## Abstract

A Bessel-type asymptotic expansion is established for the monic polynomials *π*_{n}(*x*) that are orthogonal with respect to the modified Jacobi weight , *x*∈(−1,1), where *α*, *β*>−1 and *h*(*x*) is real analytic and strictly positive on [−1,1]. This expansion holds uniformly in a region containing the neighbourhood of the critical value *x*=1. This result complements the two recent results obtained by Kuijlaars and his co-workers, one for *x* bounded away from (−1,1) and the other for *x* in (−1+*δ*,1−*δ*), *δ*>0. Our method is also based on the Riemann–Hilbert approach.

## 1. Introduction

We consider monic polynomials *π*_{n}(*x*) that are orthogonal on [−1,1] with respect to the modified Jacobi weight(1.1)where *α*, *β*>−1 and *h*(*x*) is real analytic and strictly positive on [−1,1]. These polynomials occur in the study of the modified Jacobi unitary ensemble, which is a probability measure on the space of *n*×*n* Hermitian matrices with all eigenvalues in (−1,1). When the size of the matrices tends to infinity, the distribution of the eigenvalues is related to the asymptotic behaviour of these polynomials (see some of the references given in Kuijlaars & Vanlessen 2002; Kuijlaars *et al*. 2004).

Since the weight function *w*(*x*) satisfies the Szegő conditionby using Szegő's theorem (Szegő 1975, p. 297) we have(1.2)uniformly for *z* in compact subsets of , where ,(1.3)and(1.4)see also Kuijlaars *et al*. (2004). In (1.3) and (1.4), we take the branch of (*z*^{2}−1)^{1/2} which is analytic in and which behaves like *z* as *z*→∞. The function *φ*(*z*) in (1.3) is the familiar Joukowski or aerofoil map that maps the exterior of [−1,1] conformally onto the exterior of the unit disc, and the function *D*(*z*) is the so-called Szegő function associated with the weight *w*(*x*); see Szegő (1975, p. 277).

Note that *h* is real analytic on [−1,1]; hence it can be analytically extended to a region , containing the interval [−1,1]. By using this analytic extension and the Riemann–Hilbert approach, Kuijlaars *et al*. (2004) obtained an asymptotic expansion for the polynomials *π*_{n}(*x*), holding uniformly in compact subsets of . In another paper, Kuijlaars & Vanlessen (2002) presented a uniform asymptotic expansion for *π*_{n}(*x*) in the interval (−1+*ϵ*,1−*ϵ*), *ϵ*>0, which agrees with the following result given by Szegő (1975, p. 298, theorem 12.1.6 and footnote 59)(1.5)where(1.6)and(1.7)Here the integral is taken in the Cauchy principal-value sense. However, the results are not valid in the neighbourhoods of the critical values *x*=±1.

The objective of this paper is to derive a uniform asymptotic expansion for the orthogonal polynomials *π*_{n}(*z*), which holds uniformly in a region containing the interval (−1+*δ*,1], *δ*>0. Our approach is also based on the nonlinear steepest descent method for Riemann–Hilbert problems (RHPs), introduced by Deift & Zhou (1993). As in Kuijlaars *et al*. (2004), we start with a RHP for a matrix *Y*, and make a sequence of transformationsThe purpose of the first transformation is to ‘normalize’ the behaviour of the matrix *T* at *z*=∞, and that of the second transformation is to deform the contour so that the off-diagonal entries of the jump matrix of *S*, when *z* is away from the interval (−1,1), are exponentially small. From *S*, we guess an asymptotic formulaBased on *N*, we construct a RHP for , which is the leading term of a uniform asymptotic expansion for *Y*. The main result of this paper is stated below.

*Let* , , *for* *and* *and* *be given as in* *(2.7) and (2.8)*. *Then*, *as n→∞,* *the monic orthogonal polynomial π*_{n}(*z*) *has an asymptotic expansion of the form*(1.8)*where I*_{α} *is the modified Bessel function and the expansion holds uniformly for z*∈*U*_{δ} *bounded away from* (*−∞,*−1]. *The coefficients A*_{k}(*z*) *and B*_{k}(*z*) *are analytic functions in* ; *the leading coefficients are given explicitly by*

## 2. The Riemann–Hilbert problem

Denote by {*p*_{n}(*x*)} the sequence of orthonormal polynomials with respect to the weight function on (−1,1), i.e.,

Consider the following RHP for a 2×2 matrix-valued function *Y*(*z*).

**RHP for** *Y*:

*Y*(*z*) is analytic in ;let

*Y*_{+}(*x*) and*Y*_{−}(*x*) denote the limiting values of*Y*(*z*) as*z*tends to*x*∈(−1,1) from above and below, respectively;*Y*_{+}(*x*) and*Y*_{−}(*x*) are continuous on (−1,1), and(2.1)as

*z*→∞,(2.2)near

*z*=1,(2.3)near

*z*=−1,(2.4)

The unique solution to the above RHP for *Y* is given by(2.5)where is the *n*th monic orthogonal polynomial with respect to the weight *w*(*x*) andis the Cauchy transformation of *f*; see Fokas *et al*. (1991) and Kuijlaars *et al*. (2004).

To find the asymptotic behaviour of *Y*(*z*), we first formulate an equivalent RHP which is normalized at infinity. Its jump matrix has rapidly oscillating entries. Let *σ*_{3} be the Pauli matrixso thatDefine(2.6)where(2.7)and(2.8)

Recall that is the conformal map from onto the exterior of the unit disc. For convenience, we have defined the Szegő function in (2.7) slightly differently from the one given in (1.4). Note that *T*(*z*) is analytic in . Using (2.1) and (2.6), it is readily verified that *T*(*z*) satisfies the following jump condition:Since , it follows that(2.9)for *x*∈(−1,1). Combining (2.2), (2.6) and the fact that *φ*(*z*)=2*z*+*O*(1/*z*) as *z*→∞, we have *T*(*z*)=*I*+*O*(1/*z*) as *z*→∞. Thus, the transformation *Y*→*T* gives the RHP that follows.

**RHP for T:**

*T*(*z*) is analytic in ;*T*(*z*) satisfies the following jump relation(2.10)*T*(*z*) has the following behaviour(2.11)near

*z*=1,(2.12)near

*z*=−1,(2.13)

Using the fact that *φ*_{+}(*x*)*φ*_{−}(*x*)=1 for *x*∈(−1,1), the jump matrix in (2.10) can be factorized as a product of three matrices, and we have(2.14)Note that the weight function *w* has an analytic extension to , also denoted by *w*, given byWe transform the RHP for *T* into a RHP with jumps on a lens-shaped contour , as shown in figure 1. Here *Σ*_{1} and *Σ*_{3} are the upper and lower lips of the lens, respectively, and *Σ*_{2}=[−1,1]. The lens is contained in *U*_{δ}.

Let *T* be given as in (2.6), and define the matrix-valued function *S* on by(2.15)The purpose of this transformation is to simplify the jump matrix for *T*. Indeed, the jump matrix for the RHP of *S* given below behaves like the identity matrix when *z* lies on ; see the penultimate paragraph in §1. Straightforward calculation shows that *S* is a solution of the following RHP.

**RHP for S:**

*S*(*z*) is analytic in ;*S*(*z*) satisfies the jump conditions(2.16)(2.17)where*Σ*_{i}^{o},*i*=1, 2, 3, denote the curve*Σ*_{i}with the two endpoints being removed;*S*(*z*) has the following behaviour(2.18)near

*z*=1,(2.19)near

*z*=−1,(2.20)

Note that *φ* maps onto . Hence, we have *φ*^{−2n}(*z*)→0 as *n*→∞ for , and the jump matrix in (2.16) must tend to the identity matrix *I* as *n*→∞. It is, therefore, reasonable to expect that(2.21)where *N*(*z*) is the solution of the RHP:

*N*(*z*) is analytic in ;*N*(*z*) satisfies the jump condition*N*(*z*) has the behaviour

It is not difficult to find that(2.22)From (2.6) and (2.15), it then follows thatbehaves asymptotically like(2.23)for . If we let for , then for . Furthermore, and for *x*∈[−1,1].

Sincefrom the well-known formulaeandwe conclude that the matrix in (2.23) is asymptotically equal toWe use the modified Bessel functions, instead of the simpler exponential form given in (2.23), in order to make the final approximation also valid in the neighbourhood of the endpoint *z*=1. By lemma 1 in Wang & Wong (2005), the last matrix is in turn asymptotically equal to(2.24)We shall show that *P* is indeed the leading term in an asymptotic expansion of *Y* which holds uniformly for all *z*∈*U*_{δ} bounded away from the interval (−∞,−1].

## 3. Riemann–Hilbert problem for

Recall that *h* can be analytically extended to for some *δ*>0. Hence, the function(3.1)is defined and analytic for . Similarly, the function(3.2)is defined and analytic for .

Let denote the contour shown in figure 2, where for some small *ϵ*>0, , and *Γ*_{1} consists of part of the boundary ∂*U*_{δ} and two parallel line segments. The contour *Γ* divides the complex plane into three regions. Let *Ω*_{1} denote the region bounded between *Γ*_{1} and *Γ*_{3}, and let *Ω*_{2} denote the region inside the circle *Γ*_{3}. (Both *Ω*_{1} and *Ω*_{2} are slit along the straight line *Γ*_{2}.)

For , i.e. for *z* outside , we define the matrix-valued function(3.3)For *z*∈*Ω*_{1}, we define(3.4)where(3.5)and(3.6)Put , . For *z*∈*Ω*_{2}, we define(3.7)where(3.8)and(3.9)Note that for we have(3.10)and for we have(3.11)see Kuijlaars *et al*. (2004, Lemma 6.4). The matrices *Q*_{α}(*z*) and *Q*_{β}(*z*) are both analytic in *U*_{δ}.

The matrix , so defined, satisfies the following RHP.

**RHP for** **:**

is analytic in ;

has the jump conditions(3.12)(3.13)where the jump matrix

*M*(*z*) has a uniform asymptotic expansion(3.14)for ; for each*k*=1,2,…, ‖*M*_{k}(*z*)‖ is bounded for and ‖.‖ can be any matrix norm; in (3.13) we have put the jump matrix on the left of , instead of on the right, in order to have the jump matrix of to appear on the right (see §4);as

*z*→∞,(3.15)near

*z*=1,(3.16)near

*z*=−1,(3.17)

We first establish the jump condition (3.12). Note that and are analytic for . Hence, it is easy to show that(3.18)for −1+*ϵ*<*x*<1, and(3.19)for −1<*x*<−1+*ϵ*. Furthermore,for −1+*ϵ*<*x*<1, andfor −1<*x*<−1+*ϵ*. Hence,

Next, we verify the jump conditions (3.13) and (3.14). For *z*∈*Γ*_{1}, we have, from (3.5) and (3.6),(3.20)where(3.21)(3.22)(3.23)(3.24)Also, from (3.3) and (3.4) we have(3.25)whereHere we have made use of the identityFrom the well-known asymptotic expansionsandwe can get (3.14) for *z*∈*Γ*_{1}. In a similar manner, we can demonstrate the jump condition (3.13)-(3.14) for *z*∈*Γ*_{3}.

## 4. Asymptotic expansion of *Y*(*z*)

Let . Note that *Y*(*z*) and have the same jump matrix in (−1,1); see (2.1) and (3.12). Hence, *R*_{+}(*x*)=*R*_{−}(*x*) for *x*∈(−1,1). Since the singularities of *R*(*z*) at *z*=±1 are removable, *R*(*z*) can be analytically extended to , where . On account of (3.13) and (3.14), *R* is a solution of the following RHP.

**RHP for R:**

*R*(*z*) is analytic for ;*R*(*z*) satisfies the jump condition(4.1)where(4.2)as

*z*→∞,(4.3)

As in theorem 7.10 of Deift *et al*. (1999), one can obtain from (4.1) an asymptotic expansion of the form(4.4)as *n*→∞, which holds uniformly for . Indeed, we have the following result.

*For every p*, *there exists a constant C*_{p}>0 *such that*(4.5)*for any z bounded away from Σ*_{R}. *Here,* ‖.‖ *denotes any matrix norm*.

Since , by the Plemelj formula we have(4.6)Inserting (4.2) and (4.4) in (4.6), and collecting powers of 1/*n*, we obtain the recursive formula(4.7)*k*=1,2,…, where *R*_{0}=*I*. It is easy to show thatLetFrom (4.1) and (4.7), we have(4.8)for , where(4.9)By the Plemelj formula,(4.10)Starting with , and(4.11)for *l*=1,2,…, we can prove that the sequence converges to a solution of equation (4.10) as *l*→∞, i.e.(4.12)and is a solution of equation (4.10). Moreover,(4.13)the matrix norm of which is bounded by as *n*→∞, uniformly for *s*∈*Σ*_{R}. Sinceby applying a usual argument in successive approximation, it can be shown that there exists a constant *C*_{p}>0 such that(4.14)for any *z* bounded away from *Σ*_{R}. LetNote that is a solution of the RHP for *R*; thus, can be analytically extended to , and satisfies the conditions of the RHP for *Y*. By uniqueness, or, equivalently, we have and . This completes the proof of the lemma. ▪

By the lemma, we have(4.15)uniformly for any *z* bounded away from *Σ*_{R}. Let *R*_{ij}(*z*), *i*, *j*=1,2, denote the elements in the matrix *R*(*z*). Then, in particular, we have(4.16)for *z*∈*Ω*_{1}, where(4.17)and(4.18)Here, *A*(*z*) and *B*(*z*) are analytic for , and have uniform asymptotic expansions of the form(4.19)withfor any *z*∈*U*_{δ} bounded away from (−∞,−1].

Note that and are analytic functions for . By analytic continuation, (4.16) holds for *z*∈*U*_{δ} and *z* bounded away from (−∞,−1]. In particular, when *z*=*x*=cos *θ*, *θ*∈(0,*π*−*δ*), we have . Since for , it follows that(4.20)which agrees with the solution given in Wang & Wong (2005). Here *J*_{α} is the Bessel function of the first kind.

## Acknowledgments

We would like to thank Professor Arno Kuijlaars for sending us a copy of Kuijlaars *et al*. (2004) prior to its publication. We are also grateful to the referees for making some very helpful suggestions. In particular, we thank one of the referees for pointing out a mathematical problem that was not addressed satisfactorily in the earlier version of this paper. The work of the second author is partially supported by the Research Grant Council of Hong Kong under project 9040980.

## Footnotes

- Received May 21, 2004.
- Accepted February 7, 2005.

- © 2005 The Royal Society