Top Banner
TRANSACTIONS OF THE AMERICAN MATHEMATICAL SOCIETY Volume 360, Number 7, July 2008, Pages 3875–3907 S 0002-9947(08)04443-7 Article electronically published on February 13, 2008 SEMILINEAR HYPOELLIPTIC DIFFERENTIAL OPERATORS WITH MULTIPLE CHARACTERISTICS NGUYEN MINH TRI Abstract. In this paper we consider the regularity of solutions of semilin- ear differential equations principal parts of which consist of linear polynomial operators constructed from real vector fields. Based on the study of fine prop- erties of the principal linear parts we then obtain the regularity of solutions of the nonlinear equations. Some results obtained in this article are also new in the frame of linear theory. 1. Introduction Regularity of solutions of linear differential equations has been extensively stud- ied by many authors. In this direction many strong results were achieved. The books [3], [9], [17], [18] are good references in the area (see also the recent paper [12] and the references therein). Regularity of solutions of elliptic nonlinear dif- ferential equations and systems was studied in [1], [2], [7], [14] (see also the book [13] for more references). Regularity of classical solutions of semilinear nonelliptic differential equations was investigated in [20], and recently in [19]. In this paper we deal with a subclass of nonlinear hypoelliptic operators, namely the class of semi- linear hypoelliptic operators with multiple characteristics. The paper is organized as follows. In Section 2 we recall and introduce some notation, definitions and auxiliary results. In particular, we recall the definitions of hypoellipticity with loss of σ derivatives and of maximal δ-hypoellipticity, the H¨ ormander condition (H) l , and two theorems of Rosthchild and Stein. We then introduce the definitions of hy- poellipticity for nonlinear operators, weakly maximal δ-hypoellipticity, extendedly maximal hypoellipticity in a system of given real vector fields, smoothness condi- tions of a system of given real vector fields, and of the (K) d l and (K ) d l conditions. We also define some operations on the set of multi-orders, and finally formulate without proofs some lemmas on commutators. In Section 3 we study semilinear hy- poelliptic second order differential equations of H¨ ormander type. The main result of this section is Theorem 3.3 on hypoellipticity of the operator Ψ 2 . In Section 4 we deal with semilinear hypoelliptic high order differential equations. In Subsection 4.1 we study hypoellipticity with the requirement of the condition (K) d l . The main results in this subsection are Theorems 4.10, 4.12, 4.13, 4.22 for linear operators and Theorems 4.24, 4.25 for nonlinear operators. In Subsubsection 4.1.2 we verify the (K) d l condition for some particular systems of vector fields. In Subsection 4.2 Received by the editors August 15, 2006. 2000 Mathematics Subject Classification. Primary 35H10; Secondary 35A08, 35B45, 35B65. Key words and phrases. Semilinear differential equations, a priori estimates, maximal hypoel- lipticity, polynomial operators constructed from real vector fields. c 2008 American Mathematical Society 3875 License or copyright restrictions may apply to redistribution; see http://www.ams.org/journal-terms-of-use
33

Semilinear hypoelliptic differential operators with multiple characteristics

Mar 11, 2023

Download

Documents

Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Page 1: Semilinear hypoelliptic differential operators with multiple characteristics

TRANSACTIONS OF THEAMERICAN MATHEMATICAL SOCIETYVolume 360, Number 7, July 2008, Pages 3875–3907S 0002-9947(08)04443-7Article electronically published on February 13, 2008

SEMILINEAR HYPOELLIPTIC DIFFERENTIAL OPERATORSWITH MULTIPLE CHARACTERISTICS

NGUYEN MINH TRI

Abstract. In this paper we consider the regularity of solutions of semilin-ear differential equations principal parts of which consist of linear polynomialoperators constructed from real vector fields. Based on the study of fine prop-erties of the principal linear parts we then obtain the regularity of solutions ofthe nonlinear equations. Some results obtained in this article are also new inthe frame of linear theory.

1. Introduction

Regularity of solutions of linear differential equations has been extensively stud-ied by many authors. In this direction many strong results were achieved. Thebooks [3], [9], [17], [18] are good references in the area (see also the recent paper[12] and the references therein). Regularity of solutions of elliptic nonlinear dif-ferential equations and systems was studied in [1], [2], [7], [14] (see also the book[13] for more references). Regularity of classical solutions of semilinear nonellipticdifferential equations was investigated in [20], and recently in [19]. In this paper wedeal with a subclass of nonlinear hypoelliptic operators, namely the class of semi-linear hypoelliptic operators with multiple characteristics. The paper is organizedas follows. In Section 2 we recall and introduce some notation, definitions andauxiliary results. In particular, we recall the definitions of hypoellipticity with lossof σ derivatives and of maximal δ-hypoellipticity, the Hormander condition (H)l,and two theorems of Rosthchild and Stein. We then introduce the definitions of hy-poellipticity for nonlinear operators, weakly maximal δ-hypoellipticity, extendedlymaximal hypoellipticity in a system of given real vector fields, smoothness condi-tions of a system of given real vector fields, and of the (K)dl and (K′)dl conditions.We also define some operations on the set of multi-orders, and finally formulatewithout proofs some lemmas on commutators. In Section 3 we study semilinear hy-poelliptic second order differential equations of Hormander type. The main resultof this section is Theorem 3.3 on hypoellipticity of the operator Ψ2. In Section 4 wedeal with semilinear hypoelliptic high order differential equations. In Subsection4.1 we study hypoellipticity with the requirement of the condition (K)dl . The mainresults in this subsection are Theorems 4.10, 4.12, 4.13, 4.22 for linear operatorsand Theorems 4.24, 4.25 for nonlinear operators. In Subsubsection 4.1.2 we verifythe (K)dl condition for some particular systems of vector fields. In Subsection 4.2

Received by the editors August 15, 2006.2000 Mathematics Subject Classification. Primary 35H10; Secondary 35A08, 35B45, 35B65.Key words and phrases. Semilinear differential equations, a priori estimates, maximal hypoel-

lipticity, polynomial operators constructed from real vector fields.

c©2008 American Mathematical Society

3875

License or copyright restrictions may apply to redistribution; see http://www.ams.org/journal-terms-of-use

Page 2: Semilinear hypoelliptic differential operators with multiple characteristics

3876 NGUYEN MINH TRI

we investigate hypoellipticity without the requirement of the condition (K)dl . Themain results here are Theorems 4.30, 4.33, 4.36, 4.37, 4.38, 4.40 for linear operatorsand Theorem 4.41 for nonlinear operators. The method we used in the paper is:based on the study of fine properties of the principal linear parts, we then obtainthe regularity of solutions of the nonlinear equations. As a by-product we alsoobtain some new results even for linear differential operators. To be precise, let usrecall and introduce some general basic notation, definitions and auxiliary results.

2. Notation, definitions and auxiliary results

Assume that Ω is a domain in Rn, α-multi-index: α = (α1, . . . , αn) ∈ Z

n+,

|α| = α1 + · · ·+ αn, and P (x, ∂) is a linear differential operator of order m in Ω ofthe form

P (x, ∂) =∑

|α|m

aα(x)∂α,

where aα(x) ∈ C∞(Ω), ∂α =∂|α|

∂xα11 . . . ∂xαn

n.

Definition 2.1. Assume that σ ∈ R. The operator P (x, ∂) is called hypoellipticwith loss of σ derivatives in Ω if for every real number s and every subdomainΩ′ ⊂ Ω from f ∈ D′(Ω′) and P (x, ∂)f ∈ Hs

loc(Ω′) it follows that f ∈ Hs+m−σ

loc (Ω′).Here D′(Ω′) denotes the space of distributions and Hs

loc(Ω′) is the standard Sobolev

space on Ω′.

Let (x, τα)|α|m ∈ Ω × Ω ⊂ Rn × R

N and Φ(x, τα)|α|m ∈ C∞(Ω × Ω). Denoteby Φ(x, ∂α)|α|m a nonlinear differential operator in Ω of order m acting as follows:

Φ(x, ∂α)|α|m : u(x) −→ Φ(x, ∂αu(x))|α|m,

for u ∈ Cm(Ω) and (∂αu)|α|m ∈ Ω.

Definition 2.2. The nonlinear operator Φ(x, ∂α)|α|m is called hypoelliptic in Ωif for every subdomain Ω′ ⊂ Ω there exists a positive integer M such that fromu ∈ CM (Ω′) and Φ(x, ∂αu)|α|m ∈ C∞(Ω′) it follows that u ∈ C∞(Ω′).

We must emphasize that according to Definition 2.2 there is a subtle differencebetween the notation of hypoellipticity for linear and nonlinear operators. In thisarticle we will abbreviate linear operators as operators.

Definition 2.3. A nonlinear operator Φ1(x, ∂α)|α|m1 is called smoothing in Ω iffor every subdomain Ω′ ⊂ Ω there exists a positive integer M such that the mappingΦ1(x, ∂α)|α|m1 sends CM (Ω′) into C∞(Ω′).

It is clear that if a nonlinear operator Φ1(x, ∂α)|α|m1 is smoothing in Ω, then thenonlinear operator Φ(x, ∂α)|α|m is hypoelliptic in Ω, if and only if the nonlinearoperator Φ1(x, ∂α)|α|m1 +Φ(x, ∂α)|α|m is hypoelliptic in Ω. Assume that f(x) isa function in Ω. Define the operator

Φ1(x, ∂α)|α|m : u(x) −→ f(x) + Φ(x, ∂αu(x))|α|m.

From what we have just said it follows that if f(x) is infinitely differentiable, thenthe operator Φ(x, ∂α)|α|m is hypoelliptic in Ω if and only if Φ1(x, ∂α)|α|m ishypoelliptic in Ω. Nonlinear hypoelliptic second order equations are discussed in[20] and [19].

License or copyright restrictions may apply to redistribution; see http://www.ams.org/journal-terms-of-use

Page 3: Semilinear hypoelliptic differential operators with multiple characteristics

SEMILINEAR HYPOELLIPTIC OPERATORS 3877

In this paper we will study a class of semilinear operators principal linear partsof which consist of polynomials constructed from real vector fields in Ω. Suchoperators were first studied by Hormander [10] and then subsequently by Kohn[11], Rothschild and Stein [15], Folland [6], Helffer and Nourrigat [8], .... Assumethat Xjk

j=1 ∈ T (Ω), the space of real vector fields in Ω, and ι is a multi-order,i.e. a sequence (ι1, . . . , ιr) with 1 ιs k, 1 s r. We will write

Xι = Xι1 . . . Xιr, Xι = [Xι1 . . . [Xιr−1 , Xιr

] . . . ],

where [, ] is the commutation bracket. Put r = |ι| (called the length of the multi-order ι). For the reason of convenience we include the zero element (with zerolength) into the set of multi-orders. Note that for ι = (ι1), i.e. |ι| = 1, wewrite Xι = Xι1 ; and Xι = 0 for |ι| = 0. By ι(j) we denote the number of allindexes l such that ιl = j. In the set of multi-orders it is convenient and possibleto introduce the relation of orders. Namely, assume there are given two multi-orders ι1 = (ι11, . . . , ι

1r1) and ι2 = (ι21, . . . , ι

2r2). We say that ι1 is less than ι2 if

|ι1| = r1 r2 = |ι2| and ι1j = ι2j for all j = 1, r1. If ι1 is less than ι2, thenwe write ι1 ι2. If ι1 ι2, then denote the multi-order ι3 = (ι2r1+1, . . . , ι

2r2)

by ι2 − ι1. We can also define the adding operation in the set of multi-orders.Namely, we call the multi-order ι3 = (ι11, . . . , ι

1r1 , ι21, . . . , ι

2r2) the sum of ι1 and ι2

and denote it by ι1 + ι2. Obviously, the adding operation is not commutative.We will say that ι1 belongs to ι2, and denote it by ι1 ⊆ ι2, if r1 r2 and ι11 =ι21∗ , . . . , ι1r1 = ι2(r1)∗ ; 1

∗ < · · · < (r1)∗. In the case ι1 ⊆ ι2 we call the multi-orderι3 = (ι31, . . . , ι3r3) the complement of ι1 with respect to ι2, and denote it by ι2\ι1,if ι3 ⊆ ι2 and (1∗∗, . . . , (r3)∗∗) = (1, . . . , r2)\(1∗, . . . , (r1)∗). Now let us recall thefollowing condition of Hormander:

Condition (H)l: there is a natural number l such that commutators Xι|ι|l

span the whole space Rn at every point in Ω.

It is clear that if l1 ≥ l2, then the condition (H)l2 is stronger than the condition(H)l1 . The condition (H)1 means that the system Xjk

j=1 is elliptic in Ω. As isstandard, by (, )s, ‖.‖s we denote the scalar product and norm in the Sobolev spaceHs. In [15] Rothschild and Stein established the following a priori estimates, whichwill be frequently used later on in this paper.

Theorem 2.4. Assume that the vector fields Xjkj=1 satisfy the condition (H)l in

Ω. Then for every s ∈ R and any compact K in Ω there exists a constant C suchthat

‖f‖s+ 1l

C( k∑

j=1

‖Xjf‖s + ‖f‖s

),

k∑j=1

‖Xjf‖s+ 1l

C( k∑

j=1

‖X2j f‖s + ‖f‖s

),

k∑i,j=1

‖XiXjf‖s C( k∑

j=1

‖X2j f‖s + ‖f‖s

)

for all f ∈ C∞0 (K).

Note that in Theorem 2.4 the number 1l is the best possible value. In [10]

Hormander proved a little weaker a priori estimate than in [15]. For a fixed system

License or copyright restrictions may apply to redistribution; see http://www.ams.org/journal-terms-of-use

Page 4: Semilinear hypoelliptic differential operators with multiple characteristics

3878 NGUYEN MINH TRI

of vector fields Xjkj=1, in their work [8], Helffer and Nourrigat consider operators

of the form

(2.1) Qm := Qm(x, ∂) =∑|ι|m

aι(x)Xι,

where aι(x) ∈ C∞(Ω) (in what follows it is convenient to assume that for ι = 0 thenotation Xι stands for the unit operator). They introduced the following definitionof maximal δ-hypoellipticity.

Definition 2.5. Let δ ∈ (0, 1]. We say that Qm(x, ∂) is a maximally δ-hypoellipticoperator in Ω, if for any compact K in Ω there exists a constant C such that

(2.2)∑|ι|m

‖Xιf‖δ(m−|ι|) C(‖Qm(x, ∂)f‖0 + ‖f‖0)

for all f ∈ C∞0 (K).

Definition 2.6. Let δ ∈ (0, 1]. We call Qm(x, ∂) a weakly maximally δ-hypoellipticoperator in Ω, if for any compact K in Ω there exists a constant C such that∑

|ι|m−1

‖Xιf‖δ(m−|ι|) C(‖Qm(x, ∂)f‖0 + ‖f‖0)

for all f ∈ C∞0 (K).

Remark 2.7. If Qm(x, ∂) is a maximally δ-hypoelliptic operator in Ω, then it ishypoelliptic with loss of m(1 − δ) derivatives in Ω. The maximally δ-hypoellipticoperator Qm(x, ∂) is elliptic if and only if δ = 1.

Remark 2.8. Let the vector fields Xjkj=1 satisfy the condition (H)l in Ω. Then

Qm(x, ∂) is maximally 1l -hypoelliptic in Ω if and only if Qm(x, ∂) + L(x, ∂) is

maximally 1l -hypoelliptic in Ω for any differential operator L(x, ∂) having the form

L(x, ∂) =∑

|ι|m−1

bι(x)Xι.

Indeed, we only need to prove the necessary condition since the other way is obvious.Let K be any compact in Ω. By induction on i with the use of Theorem 2.4, wehave ∑

|ι|=i

‖Xιf‖s C(∑

|ι|=i+1

‖Xιf‖s− 1l

+ ‖f‖s) ε∑

|ι|=i+1

‖Xιf‖s + C(ε)‖f‖s

for all s ∈ R, i ∈ Z+, ε > 0 and f ∈ C∞0 (K). Hence

‖(Qm(x, ∂) + L(x, ∂))f‖0 ≥ ‖Qm(x, ∂)f‖0 − ‖L(x, ∂)f‖0

≥ C1

∑|ι|=m

‖Xιf‖0 − ε∑|ι|=m

‖Xιf‖0 − C(ε)‖f‖0.

The last inequality with ε = C12 gives the estimate (2.2) for all highest derivatives

in the definition of maximal δ-hypoellipticity. Using the just obtained inequalitytogether with the condition (H)l it is not difficult to get the needed estimates for∑

|ι|=m−1

‖Xιf‖ 1l,

∑|ι|=m−2

‖Xιf‖ 2l, . . . , ‖f‖m

l.

License or copyright restrictions may apply to redistribution; see http://www.ams.org/journal-terms-of-use

Page 5: Semilinear hypoelliptic differential operators with multiple characteristics

SEMILINEAR HYPOELLIPTIC OPERATORS 3879

Now let us recall the notion of weighted Sobolev spaces associated with the vectorfields Xjk

j=1, which was first introduced by Rothschild and Stein in [15].

Definition 2.9. Let 1 p ∞. For every nonnegative integer m we de-

note by Sm,Xjk

j=1p,loc (Ω) the set of functions f such that for any compact K in

Ω the following inequality holds:∑

|ι|m ‖Xιf‖Lp(K) < ∞. Put S∞,Xjk

j=1p,loc (Ω) =⋂∞

m=0 Sm,Xjk

j=1p,loc (Ω). The spaces S

m,Xjkj=1

2,loc (Ω) and S∞,Xjk

j=12,loc (Ω) will simply be

denoted by Sm,Xjk

j=1loc (Ω) and S

∞,Xjkj=1

loc (Ω), respectively.

Definition 2.10. We will say that the system of vector fields Xjkj=1 satisfies the

smoothness condition, if there exists a map T : Z+ −→ R, with limm→+∞ T (m) =

+∞, such that Sm,Xjk

j=1loc (Ω) ⊂ H

T (m)loc (Ω) for any m ∈ Z+.

It is easy to check that the space Sm,Xjk

j=1loc (Ω) enjoys the following properties:

a) Cm(Ω) ⊂ Sm,Xjk

j=1loc (Ω), ∀m ≥ 0.

b) If f ∈ Sm,Xjk

j=1loc (Ω) and |ι| m, then Xιf ∈ S

m−|ι|,Xjkj=1

loc (Ω).

c) If m m, then Sm,Xjk

j=1loc (Ω) ⊂ S

m,Xjkj=1

loc (Ω).d) If the vector fields Xjk

j=1 satisfy the condition (H)l, then they also satisfy

the smoothness condition since Sm,Xjk

j=1loc (Ω) ⊂ H

ml

loc(Ω) (see [15]).e) If the vector fields Xjk

j=1 satisfy the smoothness condition, then

S∞,Xjk

j=1loc (Ω) = C∞(Ω).

Indeed, the inclusion

S∞,Xjk

j=1loc (Ω) =

∞⋂m=0

Sm,Xjk

j=1loc (Ω) ⊂

∞⋂m=0

HT (m)loc (Ω) = H∞

loc(Ω) = C∞(Ω)

holds. The converse is obvious.

Together with the linear operator Qm(x, ∂) in (2.1) we will as well consider itssemilinear perturbation

(2.3) Θm := Θm(x, ∂α) : f −→ Qm(x, ∂)f + Φ(x, Xιf)|ι|m−1,

where Φ(x, τι)|ι|m−1 is a smooth function.

Definition 2.11. The operator Qm(x, ∂) is called extendedly maximally hypoel-liptic in the system Xjk

j=1 in domain Ω, if for any natural number m1 ∈ Z+ from

f ∈ L2loc(Ω), Qm(x, ∂)f ∈ S

m1,Xjkj=1

loc (Ω) it follows that f ∈ Sm1+m,Xjk

j=1loc (Ω).

Definition 2.12. The nonlinear operator Θm(x, ∂α) is called extendedly maximallyhypoelliptic in the system Xjk

j=1 in Ω, if there exists a nonnegative integer M such

that for any natural number m1 ∈ Z+ from f ∈ CM (Ω), Θmf ∈ Sm1,Xjk

j=1loc (Ω) it

follows that f ∈ Sm1+m,Xjk

j=1loc (Ω).

License or copyright restrictions may apply to redistribution; see http://www.ams.org/journal-terms-of-use

Page 6: Semilinear hypoelliptic differential operators with multiple characteristics

3880 NGUYEN MINH TRI

Remark 2.13. If the operator Qm (Θm) is extendedly maximally hypoelliptic in thesystem Xjk

j=1, satisfying the smoothness condition, then it is hypoelliptic.

Next we give some lemmas on commutators, which will be used frequently inthe paper. We omit the proofs of these lemmas which can easily be executed byinduction.

Lemma 2.14. Let A, B be elements of an algebra A. The following formulas hold:

AnB = BAn +n∑

i=1

Cin[ iA, B]An−i =

n∑i=0

Cin[ iA, B]An−i,

BAn = AnB +n∑

i=1

CinAn−i[B, iA] =

n∑i=0

CinAn−i[B, iA],

in which we have used the following notation:

[ 0A, B] = B, [ iA, B] = [A . . . [︸ ︷︷ ︸i times

A, B] . . . ], for i ≥ 1;

[B, 0A] = B, [B, iA] = [. . . [B, A ] . . . A]︸ ︷︷ ︸i times

, for i ≥ 1.

Lemma 2.15. Let A, B be elements of an algebra A. Then we have

AnBm =n∑

i1=0

n−i1∑i2=0

· · ·n−i1−···−im−1∑

im=0

n!i1! · · · im!

[ i1A, B] · · · [ imA, B]An−i1−···−im ,

BmAn =n∑

i1=0

n−i1∑i2=0

· · ·n−i1−···−im−1∑

im=0

n!i1! · · · im!

An−i1−···−im [B, imA] · · · [B, i1A].

Lemma 2.16. Let A1, . . . , An, B be elements of an algebra A and Aα =Aα11 . . . Aαn

n ,where α = (α1, . . . , αn) ∈ Z

n+. The following formulas hold:

AαB =∑iα

Ciα[ i1A1 . . . [ in

An, B] . . . ]Aα−i :=∑iα

Ciα[ iA, B]Aα−i,

BAα =∑iα

CiαAα−i[. . . [B, in

An] . . . i1A1] :=∑iα

CiαAα−i[B, iA],

where the following notations are used:

i = (i1, . . . , in) ∈ Zn+, Ci

α =

∏nj=1 αj !∏n

j=1 ij !(αj − ij)!, Aα−i = Aα1−i1

1 . . . Aαn−inn .

Lemma 2.17. Let A be an operator acting on a space of functions. Then we have

XιA = AXι +∑

ι1⊆ι;ι1 =ι

[ ι\ι1X, A]Xι1 ,

AXι = XιA +∑

ι1⊆ι;ι1 =ι

Xι1 [A, ι\ι1X],

in which [ ϑX, A] = [Xϑ1 [. . . [Xϑr, A]]] and [A, ϑX] = [[[A, Xϑ1 ] . . . ]Xϑr

] for arbi-trary multi-orders ϑ = (ϑ1, . . . , ϑr).

License or copyright restrictions may apply to redistribution; see http://www.ams.org/journal-terms-of-use

Page 7: Semilinear hypoelliptic differential operators with multiple characteristics

SEMILINEAR HYPOELLIPTIC OPERATORS 3881

Lemma 2.18. The following formulas hold:

Xι1Xι2 = Xι2Xι1 +∑

|ι3|+|ι4|=|ι1|+|ι2||ι3|≥2

C(ι3, ι4)Xι3Xι4

in which C(ι3, ι4) are just constants.

We conclude this section by introducing two conditions, which will be used lateron:

Condition (K)dl : for every i d and any compact K in Ω there exists a constantC = C(l, K) such that

‖Xιf‖s C( k∑

j=1

‖Xjf‖s+ i−1l

+ ‖f‖s

), ∀|ι| = i, f ∈ C∞

0 (K).

Obviously, if l1 ≥ l2, then the condition (K)dl2 is stronger than the condition (K)dl1 .If d1 ≥ d2, then the condition (K)d1

l is stronger than the condition (K)d2l . If the

vector fields Xjkj=1 satisfy the condition (K)dl for all d, then we say that they

satisfy the condition (K)l.Condition (K′)dl : for every i d and any compact K in Ω there exists a constant

C = C(l, K) such that

‖Xιf‖s C( k∑

j

‖Xi−1j f‖s+ 1

l+ ‖f‖s

), ∀|ι| = i, f ∈ C∞

0 (K).

If the condition (H)l is satisfied, then the condition (K′)dl becomes weaker than thecondition (K)dl .

3. Semilinear hypoelliptic second order differential equations

of Hormander type

In this section we consider the following semilinear differential equation of Hor-mander type:

Ψ2(x, ∂α)f := P2(x, ∂)f + Φ(x, f, X1f, . . . , Xkf) = 0,

where P2(x, ∂) =∑k

j=1 X2j . Note that in view of Theorem 2.4 the operator P2 is

maximally 1l -hypoelliptic if the vector fields Xjk

j=1 satisfy the condition (H)l inΩ. For studying the nonlinear operator Ψ2(x, ∂α) we will use the following theorem,which is also due to Rothschild and Stein [15].

Theorem 3.1. Assume that the vector fields Xjkj=1 satisfy the condition (H)l

in Ω. If f ∈ D′(Ω), P2f ∈ Sm,Xjk

j=1loc (Ω), then f ∈ S

m+2,Xjkj=1

loc (Ω) for eachnonnegative integer m. In other words, the operator P2 is extendedly maximallyhypoelliptic in Ω.

Proposition 3.2. Assume that the vector fields Xjkj=1 satisfy the condition (H)l

in Ω and m > nl + q. If Φ(x, τϑ)|ϑ|q ∈ C∞ and f ∈ Sm,Xjk

j=1loc (Ω), then

Φ(x, Xϑf)|ϑ|q ∈ Sm−q,Xjk

j=1loc (Ω).

License or copyright restrictions may apply to redistribution; see http://www.ams.org/journal-terms-of-use

Page 8: Semilinear hypoelliptic differential operators with multiple characteristics

3882 NGUYEN MINH TRI

Proof. It suffices to prove that XιΦ(x, Xϑf)|ϑ|q ∈ L2loc(Ω) for any multi-order ι

such that |ι| m − q. Since m > nl + q, it follows that Xϑf ∈ C(Ω) for all multi-orders ϑ such that |ϑ| q. Furthermore by induction on |ι| it is easy to check thatXιΦ(x, Xϑf)|ϑ|q is a linear combination of terms of the form

(3.1) aα(x)∂αΦ(x, Xϑf)|ϑ|q

∂xα

and

(3.2) aα,β(x)∂α+βΦ(x, Xϑf)|ϑ|q

∂xα∂τβ(Xι1Xϑ1

f)m1 · · · (Xιp

Xϑp

f)mp ,

where α = (α1, . . . , αn), β = (βϑ)|ϑ|q - multi-indexes; τ = (τϑ)|ϑ|q; |ι1|, . . . , |ιp|,m1, . . . , mp ≥ 1, |ι1|m1 + · · · + |ιp|mp m − q; |ϑj | q; aα(x), aα,β(x) ∈ C∞(Ω).

Proposition 3.2 will be proved if we can show that all the above terms in (3.1) and(3.2) belong to L2

loc(Ω). Obviously, the terms in (3.1) are continuous and thereforethey are in L2

loc(Ω) since aα ∈ C∞, Φ ∈ C∞, Xϑf ∈ C(Ω) for all multi-orders ϑ suchthat |ϑ| q. For estimating the terms in (3.2) we put r = max|ι1|, . . . , |ιp| ≥ 1.Choose one j0 such that r = |ιj0 |. Consider the following possibilities:

A) mj0 ≥ 2. We state that |ιj | [m−q2 ] for each j = 1, . . . , p, where [.] denotes

the integral part of the argument. Indeed, if j = j0 and |ιj | > [m−q2 ], then |ιj0 | ≥

|ιj | > [m−q2 ]. Hence, |ιj | + |ιj0 | > m − q, and we come to a contradiction. If j = j0

and |ιj0 | > [m−q2 ], then |ιj0 |mj0 ≥ 2|ιj0 | > m − q, which is impossible. Since m ≥

nl+q+1, all the factors Xιj

Xϑj

f in the terms (3.2) belong to S[ m−q

2 ],Xjkj=1

loc (Ω) ⊂C(Ω) for each j = 1, p. Thus, we conclude that the terms are in C(Ω), and thereforethey belong to L2

loc(Ω).B) mj0 = 1 and there are no other elements j such that j = j0. Then the term

will have form

aα,β(x)∂α+βΦ(x, Xϑf)|ϑ|q

∂xα∂τβXιj0

Xϑj0f ∈ L2

loc(Ω), as |ιj0 | m − q.

C) mj0 = 1 and there is at least one element j1 such that j1 = j0. As in partA) it is possible to establish that |ιj | [m−q

2 ] for all j, and therefore Xιj

Xϑj

f ∈S

[ m−q2 ],Xjk

j=1loc (Ω) ⊂ C(Ω) for all j. Thus,

aα,β(x)∂α+βΦ(x, Xϑf)|ϑ|q

∂xα∂τβ(Xι1Xϑ1

f)m1 · · · (Xιp

Xϑp

f)mp ∈ C(Ω) ⊂ L2loc(Ω).

The proof of Proposition 3.2 is complete.

Theorem 3.3. Assume that the vector fields Xjkj=1 satisfy the condition (H)l in

Ω and Φ(x, f, τ1, . . . , τk) is an infinitely differentiable function. Then the nonlinearoperator Ψ2(x, ∂α) is extendedly maximally hypoelliptic in the system Xjk

j=1. Inparticular, it is hypoelliptic.

Proof. We show that if a function f is a solution of the equation Ψ2(x, ∂α)f = g ∈S

m1,Xjkj=1

loc (Ω) and f ∈ Cnl+2(Ω), then f ∈ Sm1+2,Xjk

j=1loc (Ω) for each nonnegative

integer m1. In fact, assume that f ∈ Sm,Xjk

j=1loc (Ω) with m ≥ nl + 2. We have

Φ(x, f, X1f, . . . , Xkf) ∈ Sm−1,Xjk

j=1loc (Ω) by Proposition 3.2 with q = 1. Put

License or copyright restrictions may apply to redistribution; see http://www.ams.org/journal-terms-of-use

Page 9: Semilinear hypoelliptic differential operators with multiple characteristics

SEMILINEAR HYPOELLIPTIC OPERATORS 3883

t = minm − 1, m1. Then from the equationk∑

j=1

X2j f = g(x) − Φ(x, f, X1f, . . . , Xkf) ∈ S

t,Xjkj=1

loc (Ω)

and Theorem 3.1, it follows that f ∈ St+2,Xjk

j=1loc (Ω). Repeating the above ar-

gument again and again, we obtain f ∈ Sm1+2,Xjk

j=1loc (Ω). Finally, note that the

second statement in Theorem 3.3 is a straightforward consequence of the first state-ment and Remark 2.13. This concludes the proof of Theorem 3.3.

Remark 3.4. Results of the same nature as in Theorem 3.3 were established in [20].However, our method is different from the one in [20] which works in the Holdersettings.

Remark 3.5. In Theorem 3.3 the condition of semi-linearity of Ψ2 is essential. Oth-erwise the theorem may not be true. For example, consider the following operatorin R

2:Ψ = X2

1 + (X22 )2,

where X1 = ∂∂x1

, X2 = ∂∂x2

. It is easy to see that

Ψ(|x2|2m+1) = (2m + 1)2(2m)2x4m−22 ∈ C∞(R2)

for all m ∈ Z+. However |x2|2m+1 ∈ C2m(R2)\C∞(R2). Hence the nonlinearoperator Ψ is not hypoelliptic in R

2.

4. Semilinear hypoelliptic high order differential equations

4.1. Maximal δ-hypoellipticity and the (K)l condition.

4.1.1. Linear operator. In this part we consider a linear operator of the form

P2m := P2m(x, ∂) = (−1)mk∑

j=1

X2mj +

∑|ι|2m−1

aι(x)Xι,

where aι(x) ∈ C∞(Ω). In what follows, by small letters we will denote smoothfunctions, and by capital letters we will denote operators acting on a space offunctions. Note that P2(x, ∂) is the Hormander operator recalled in Section 2. Here,the factor (−1)m is introduced for the reason of convenience in the computationprocess that arises in the future.

Lemma 4.1. Let X ∈ T (Ω). The following formula holds:

(Xι)∗ = (−1)|ι|X ι +∑

∀j:ϑ(j)ι(j)|ϑ|<|ι|

aϑ(x)Xϑ,

where (Xι)∗ is the formal adjoint of Xι and ι = (ιr, . . . , ι1) if ι = (ι1, . . . , ιr).

Proof. This lemma can easily be proved by induction on |ι|. We omit the details.

Now let s ∈ R and Λs be the pseudodifferential operator (p.d.o.) with symbol(1+|ξ|2) s

2 . If K is a fixed compact in Ω, then by Gs := GsK we denote the p.d.o. with

the symbol g(x)(1 + |ξ|2) s2 , where g(x) ∈ C∞

0 (Ω), g(x) ≡ 1 in some neighborhood

License or copyright restrictions may apply to redistribution; see http://www.ams.org/journal-terms-of-use

Page 10: Semilinear hypoelliptic differential operators with multiple characteristics

3884 NGUYEN MINH TRI

of the compact K. It is well known that for each natural integer N and any realnumber s there exist a multi-functional AN and a constant C = C(s, N) such that

(f, h)s = (Λsf, Λsh)0 = (Gsf, Gsh)0 + AN (f, h), ∀f, h ∈ C∞0 (K),

‖Gsf‖20 − C−1‖f‖2

−N ‖f‖2s ‖Gsf‖2

0 + C‖f‖2−N , ∀f ∈ C∞

0 (K),

|AN (f, h)| C(‖f‖2−N + ‖h‖2

−N ), ∀f, h ∈ C∞0 (K).

Lemma 4.2. Let X ∈ T (Ω). Then for any compact K in Ω, ε > 0, s ∈ R, p ≥ 1there exists a constant C(ε) = C(ε, s, p, K) such that

‖Xpf‖2s ε‖Xp+1f‖2

s + C(ε)‖f‖2s, ∀f ∈ C∞

0 (K).

Proof. For p = 1 the lemma is trivial. Assume that it is true for p t. We show itfor p = t + 1. Obviously, X∗ = −X + a(x). Therefore

‖Xt+1f‖2s (GsXt+1f, GsXt+1f)0 + C‖f‖2

s

= (Xtf, (−X + a(x))G2sXt+1f)0 + C‖f‖2s

ε1‖Xt+2f‖2s +

14‖Xt+1f‖2

s + C(ε1)‖Xtf‖2s + C‖f‖2

s.

Now choosing ε1 ε2 sufficiently small, by the inductive assumptions we conclude

that

‖Xt+1f‖2s ε‖Xt+2f‖2

s

2+

‖Xt+1f‖2s

2+ C(ε)‖f‖2

s,

from which the lemma follows.

Lemma 4.3. Assume that the vector fields Xjkj=1 satisfy the condition (H)l in

Ω. Then for any compact K in Ω, any natural integer i and each real number sthere exists a constant C = C(l, s, K) such that

(4.1)k∑

j=1

‖Xi−1j f‖2

s C( k∑

j=1

‖Xijf‖2

s− 1l

+ ‖f‖2s− 1

l

), ∀f ∈ C∞

0 (K).

Proof. For i = 1 the estimate is true in view of Theorem 2.4. By assuming thatthe estimate (4.1) holds for i p we will prove it for i = p + 1. We have

k∑j=1

‖Xpj f‖2

0 =k∑

j=1

(Xpj f, Xp

j f)0 =k∑

j=1

(Xp−1j f, (−Xj + aj(x))Xp

j f)0

C(ε)k∑

j=1

‖Xp+1j f‖2

− 1l+ε

k∑j=1

‖Xp−1j f‖2

1l+

12

k∑j=1

‖Xpj f‖2

0+C(ε)‖f‖2− 1

l,

where aj(x) ∈ C∞(Ω). From the just obtained estimate and the inductive assump-tions, by choosing ε sufficiently small, with the help of Lemma 4.2 we deduce (4.1)in the case s = 0. It remains to prove the estimate in the general case. By Lemma

License or copyright restrictions may apply to redistribution; see http://www.ams.org/journal-terms-of-use

Page 11: Semilinear hypoelliptic differential operators with multiple characteristics

SEMILINEAR HYPOELLIPTIC OPERATORS 3885

2.14 and the inductive assumptions we havek∑

j=1

‖Xpj f‖2

s C( k∑

j=1

‖Xpj Gsf‖2

0 +k∑

j=1

‖f‖2s

)

C( k∑

j=1

‖GsXp+1j f‖2

− 1l

+∑

j=1,ki=1,p+1

‖Xp+1−ij f‖2

s− 1l

+ ‖f‖2s

)

C( k∑

j=1

‖Xp+1j f‖2

s− 1l

+ ‖f‖2s

)+

12

k∑j=1

‖Xpj f‖2

s,

which shows the correctness of (4.1) for arbitrary real s ∈ R. Lemma 4.3 is nowcompletely proved.

Corollary 4.4. Let the conditions of Lemma 4.3 be satisfied. Then the followinginequalities hold:

‖f‖2s+ i

l C

( k∑j=1

‖Xijf‖2

s + ‖f‖2s

), ∀f ∈ C∞

0 (K),

‖f‖2s+ i

l C

( k∑j=1

‖Xijf‖2

s + ‖f‖2s′

), (s′ < s), ∀f ∈ C∞

0 (K),

‖Xi1j1

f‖2s+

i−i1l

C( k∑

j=1

‖Xijf‖2

s + ‖f‖2s

), i1 i, j1 = 1, k, ∀f ∈ C∞

0 (K),

‖Xj1Xj2f‖2s+ i−2

l

C( k∑

j=1

‖Xijf‖2

s + ‖f‖2s

), 2 i, j1, j2 = 1, k, ∀f ∈ C∞

0 (K).

Lemma 4.5. Assume that the vector fields Xjkj=1 satisfy the conditions (H)l

and (K)dl in Ω. Then for any compact K in Ω, any arbitrary values j1, j2 =1, k; i1, i2, i1 + i2 d, and each real number s there exists a constant

C(j1, j2, i1, i2, s, K) = C

such that

(4.2) ‖Xi1j1

Xi2j2

f‖2s C

( k∑j=1

‖Xi1+i2j f‖2

s + ‖f‖2s

), ∀f ∈ C∞

0 (K).

Proof. For i1 + i2 = 1 the lemma is trivial. For i1 + i2 = 2 the lemma is true inview of Theorem 2.4. Assuming that the estimate (4.2) holds for i1 + i2 t d−1,we prove it for i1 + i2 = t + 1. Clearly, we can suppose that i1 ≥ 1, i2 ≥ 1. By theinductive assumptions the following inequality holds:

‖Xi1j1

Xi2j2

f‖2s = ‖Xi1

j1Xi2−1

j2(Xj2f)‖2

s C( k∑

j=1

‖Xi1+i2−1j Xj2f‖2

s + ‖Xj2f‖2s

)

C( k∑

j=1

‖Xi1+i2j f‖2

s +k∑

j=1

‖Xi1+i2−1j Xj2f‖2

s + ‖f‖2s

).(4.3)

License or copyright restrictions may apply to redistribution; see http://www.ams.org/journal-terms-of-use

Page 12: Semilinear hypoelliptic differential operators with multiple characteristics

3886 NGUYEN MINH TRI

Therefore the matter reduces to proving the estimate (4.2) for terms of the formsXt

j1Xj2f . By Lemma 2.14 we have

(Xtj1Xj2f, Xt

j1Xj2f)0 = −(Xt−1j1

Xj2f, Xt+1j1

Xj2f)0 + (Xt−1j1

Xj2f, aj1Xtj1Xj2f)0

= −(Xt−1j1

Xj2f, Xj2Xt+1j1

f)0 −t+1∑i=1

Cit+1(X

t−1j1

Xj2f, [ iXj1 , Xj2 ]Xt+1−ij1

f)0 + I1

:= J0 +t+1∑i=1

Ji + I1.

Using the inductive assumptions and the Cauchy-Schwarz inequality we obtain

|I1| ε‖Xtj1Xj2f‖2

0 + C(ε)( k∑

j=1

‖Xtjf‖2

0 + ‖f‖20

).

In view of the condition (K)dl of the lemma, taking into account the inductiveassumptions and the estimate (4.3) we deduce that

|J1| C(ε)‖Xt−1j1

Xj2f‖21l

+ ε( k∑

j=1

‖XjXtj1f‖

20 + ‖Xt

j1f‖20

)

C(ε)( k∑

j=1

‖Xt+1j f‖2

0 + ‖f‖20

)+ ε

k∑j=1

‖XtjXj1f‖2

0.

Further, for 2 i t + 1, by using the condition (K)dl it can be checked that

|Ji| C(‖Xt−1j1

Xj2f‖20 + ‖[ iXj1 , Xj2 ]X

t+1−ij1

f‖20)

C( k∑

j=1

‖Xtjf‖2

0 +k∑

j=1

‖XjXt+1−ij1

f‖2il+ ‖f‖2

0

)

C( k∑

j=1

‖Xt+2−ij f‖2

0 + ‖f‖20

) C

( k∑j=1

‖Xtjf‖2

0 + ‖f‖20

).

Finally, we estimate J0. We have

J0 = (Xj2Xt−1j1

Xj2f, Xt+1j1

f)0 − (aj2Xt−1j1

Xj2f, Xt+1j1

f)0

C(ε)‖Xt+1j1

f‖20 + ε‖Xj2X

t−1j1

Xj2f‖20 + C‖Xt−1

j1Xj2f‖2

0

C(ε)( k∑

j=1

‖Xt+1j f‖2

0 + ‖f‖20

)+ ε

k∑j=1

‖XtjXj2f‖2

0.

Altogether, from the obtained inequalities we find that

‖Xtj1Xj2f‖2

0 C(ε)( k∑

j=1

‖Xt+1j f‖2

0 + ‖f‖20

)+ ε

k∑j,j′=1

‖XtjXj′f‖2

0.

By setting M = maxj,j′=1,k ‖XtjXj′f‖2

0, from the last inequality we have

‖Xtj1Xj2f‖2

0 M C(ε)( k∑

j=1

‖Xt+1j f‖2

0 + ‖f‖20

)+ k2εM.

License or copyright restrictions may apply to redistribution; see http://www.ams.org/journal-terms-of-use

Page 13: Semilinear hypoelliptic differential operators with multiple characteristics

SEMILINEAR HYPOELLIPTIC OPERATORS 3887

If we take ε = k2

2 , then

(4.4) ‖Xtj1Xj2f‖2

0 C( k∑

j=1

‖Xt+1j f‖2

0 + ‖f‖20

).

This is the estimate (4.2) in the case s = 0. We begin to prove the lemma in thegeneral case by substituting Gsf for f in (4.4):

‖Xtj1Xj2G

sf‖20 C

( k∑j=1

‖Xt+1j Gsf‖2

0 + ‖Gsf‖20

)

C( k∑

j=1

‖GsXt+1j f‖2

0 +t+1∑i=1

k∑j=1

‖Xt+1−ij f‖2

s

) C

( k∑j=1

‖Xt+1j f‖2

s + ‖f‖2s

).

On the other hand, by Lemma 2.16

‖Xtj1Xj2f‖2

s C(‖GsXtj1Xj2f‖2

0 + ‖f‖2s) C

(‖Xt

j1Xj2Gsf‖2

0

+∑

(0,0)<(i3,i4)(t,1)

C(i3,i4)(t,1) ‖[ i3Xj1 [ i4Xj2 , G

s]]Xt−i3j1

X1−i4j2

f‖20

)

C(‖Xt

j1Xj2Gsf‖2

0 +k∑

j=1

‖Xtjf‖2

s + ‖f‖2s

).

From the last two inequalities we immediately get the desired result.

Remark 4.6. Note that Lemma 4.3 remains true if we replace the condition (K)dlby the condition (K′)dl .

Lemma 4.7. Assume that the vector fields Xjkj=1 satisfy the conditions (H)l

and (K)dl (or (K′)dl ) in Ω. Then for any compact K in Ω, any values j1, j2 =1, k; q, i1, . . . , i2q−1 with i1 + · · · i2q−1 d, and each real number s, there exists aconstant C(j1, j2, q, i1, . . . , i2q−1, s, K) = C such that

(4.5) ‖Xi1j1

Xi2j2

...Xi2q−1j1

f‖2s C

( k∑j=1

‖Xi1+···+i2q−1j f‖2

s + ‖f‖2s

)

for all f ∈ C∞0 (K).

Proof. For q = 1 the estimate (4.5) is trivial. Suppose that it is true for q t.Then we show that it is true for terms of the forms

Xi1j1

Xi2j2

...Xi2tj2

f (i2t ≥ 1), Xi1j1

Xi2j2

...Xi2q

j2X

i2t+1j1

f (i2t+1 ≥ 1).

Indeed, by the inductive assumption, from Lemma 4.5 we deduce that

‖Xi1j1

Xi2j2

...Xi2tj2

f‖2s = ‖Xi1

j1Xi2

j2...X

i2t−1j1

(Xi2tj2

f)‖2s

C( k∑

j=1

‖Xi1+···+i2t−1j (Xi2t

j2f)‖2

s + ‖Xi2tj2

f‖2s

) C

( k∑j=1

‖Xi1+···+i2tj f‖2

s + ‖f‖2s

).

Having established the estimate for Xi1j1

Xi2j2

...Xi2tj2

f , we can then get the estimatefor Xi1

j1Xi2

j2...X

i2q

j2X

i2t+1j1

f in a similar way. Thus, Lemma 4.7 is proved.

License or copyright restrictions may apply to redistribution; see http://www.ams.org/journal-terms-of-use

Page 14: Semilinear hypoelliptic differential operators with multiple characteristics

3888 NGUYEN MINH TRI

Remark 4.8. The estimate (4.5) is true for Xi1j1

Xi2j2

...Xi2q

j2f since we can write

Xi1j1

Xi2j2

...Xi2q

j2f = Xi1

j1Xi2

j2...X

i2q

j2X

i2t+1j1

f with i2t+1 = 0. Alternatively, it is alsoclear from the proof of Lemma 4.7.

Lemma 4.9. Assume that the vector fields Xjkj=1 satisfy the conditions (H)l and

(K)ml in Ω. For any compact K in Ω and any real number s there exists a constant

C = C(s, K) such that

k∑j=1

‖Xmj f‖2

s 2Re(P2mf, f)s + C‖f‖2s

for all f ∈ C∞0 (K).

Proof. We begin by rewriting aιXι in P2m(x, ∂) as aιX

ι1Xι2 where |ι1| m −1, |ι2| m. Therefore using Lemmas 2.17 and 4.7 we obtain

(P2mf, f)s = AN (P2mf, f) + (GsP2mf, Gsf)0 = AN (P2mf, f)

+ (−1)mk∑

j=1

(Xmj GsXm

j f, Gsf)0

+k∑

j=1

m∑i=1

Cim(−1)m([Gs, iXj ]Xm

j f, (Xm−ij )∗Gsf)0

+∑

|ι|2m−1

∑ι3⊆ι1,ι3 =ι1

([Gs, ι1\ι3X]Xι2f, (Xι3)∗a∗ι (x)Gsf)0

+∑

|ι|2m−1

∑ι3⊆ι1,ι3 =ι1

([[Gs, aι], ι1\ι3X]Xι2f, (Xι3)∗Gsf)0

:= AN (P2mf, f) + I1 + I2 + I3 + I4.

First we estimate I1. We have

I1 = (−1)mk∑

j=1

(GsXmj f, X∗m

j Gsf)0 =m∑

j=1

‖GsXmj f‖2

0

+ (−1)mk∑

j=1

m−1∑i=0

(GsXmj f, Gsai,jX

ijf)0

+ (−1)mk∑

j=1

m∑i=1

Cim(GsXm

j f, [ iX∗j , Gs](X∗

j )m−if)0

:=m∑

j=1

‖GsXmj f‖2

0 + J1 + J2.

Since the order of the operator [ iX∗j , Gs] equals s, for any ε > 0 the following

estimate holds:

max|J1|, |J2| εm∑

j=1

‖GsXmj f‖2

0 + C(ε)‖f‖2s.

License or copyright restrictions may apply to redistribution; see http://www.ams.org/journal-terms-of-use

Page 15: Semilinear hypoelliptic differential operators with multiple characteristics

SEMILINEAR HYPOELLIPTIC OPERATORS 3889

Analogously, it can be shown that

max|I2|, |I3|, |I4| ε

m∑j=1

‖GsXmj f‖2

0 + C(ε)‖f‖2s.

Thus,m∑

j=1

‖GsXmj f‖2

0 = Re(P2mf, f)s − Re(AN (P2mf, f) + J1 + J2 + I2 + I3 + I4)

Re(P2mf, f)s + |AN (P2mf, f)| + |J1| + |J2| + |I2| + |I3| + |I4|

Re(P2mf, f)s + ε

m∑j=1

‖GsXmj f‖2

0 + C(ε)‖f‖2s.

Choosing ε sufficiently small we get the desired result.

Theorem 4.10. Assume that the vector fields Xjkj=1 satisfy the conditions (H)l

and (K)3ml (or (K′)3m

l ) in Ω. Then for any compact K in Ω, any real value sthere exists a constant C(s, K) = C such that

k∑j=1

‖Xijf‖2

s+ 2m−il

C(‖P2mf‖2s + ‖f‖2

s), ∀f ∈ C∞0 (K),(4.6)

where i = 0, 1, . . . , 2m.

Proof. By Lemma 4.9 and Corollary 4.4, we havek∑

j=1

‖Xmj f‖2

s+ ml

C(Re(P2mf, f)s+ ml

+ ‖f‖2s+ m

l)

C(‖P2mf‖s + ‖f‖s+ ml)‖f‖s+ 2m

l

C(‖P2mf‖s + ‖f‖s+ ml)( k∑

j=1

‖Xmj f‖2

s+ ml

+ ‖f‖2s+ m

l

) 12

so that

(4.7)k∑

j=1

‖Xmj f‖2

s+ ml

C(‖P2mf‖2s + ‖f‖2

s+ ml).

Now, if in the estimate

‖f‖2s+ m

l ε‖f‖2

s+ 2ml

+ C(ε)‖f‖2s ε

k∑j=1

‖Xmj f‖2

s+ ml

+ C(ε)‖f‖2s

we choose ε sufficient small, then from (4.7) we obtaink∑

j=1

‖Xmj f‖2

s+ ml

12

k∑j=1

‖Xmj f‖2

s+ ml

+ C(‖P2mf‖2s + ‖f‖2

s).

That is,k∑

j=1

‖Xmj f‖2

s+ ml

C(‖P2mf‖2s + ‖f‖2

s).

License or copyright restrictions may apply to redistribution; see http://www.ams.org/journal-terms-of-use

Page 16: Semilinear hypoelliptic differential operators with multiple characteristics

3890 NGUYEN MINH TRI

By using Lemma 4.3, we finally conclude that

k∑j=1

‖Xijf‖2

s+ 2m−il

C( k∑

j=1

‖Xmj f‖2

s+ ml

+ ‖f‖2s

)

C(‖P2mu‖2s + ‖f‖2

s), i = 0, 1, ..., m.(4.8)

Thus, the estimate (4.6) is proved for i = 0, 1, . . . , m. We will show the estimate(4.6) for i = m+1, . . . , 2m. We will prove it by induction on m+i, for i = 0, . . . , m.For i = 0 the estimate (4.6) is true in view of (4.8). Assuming that it is truefor i = i0 m − 1 we show its availability for i = i0 + 1. In fact, taking thescalar product (−1)i0+1(P2mf, X

2(i0+1)1 f)0, integrating by parts and with the help

of Lemma 4.1 one obtain the following equality:

‖Xm+i0+11 f‖2

0 = (−1)i0+1(P2mf, X2(i0+1)1 f)0 + (−1)m+i0

k∑j=2

(X2mj f, X2i0+2

1 f)0

+ (−1)i0∑

|ι|2m−1

(aιXιf, X2i0+2

1 f)0

+m−i0−1∑

q=1

(Xm+i0+11 f, bqX

m+i0+1−q1 f)0 := I1 + I2 + I3 + I4.

First we calculate I2. From Lemma 4.1, it follows that

I2 := (−1)i0

k∑j=2

(Xm

j f, Xmj X2i0+2

1 f)

0+ I21.(4.9)

Consider the terms of the highest order in (4.9). We have

(−1)i0(Xmj f, Xm

j X2i0+21 f)0 =−‖Xi0+1

1 Xmj f‖2

0+i0+1∑q=1

(dqX

i0+1−q1 Xm

j f, Xi0+11 Xm

j f)

0

+(−1)i0(Xmj f, [Xm

j , X2i0+21 ]f)0 := −‖Xi0+1

1 Xmj f‖2

0 + I22 + I23.

Further, for any ε > 0 there exists a constant C(ε) such that

|I22| ε‖Xi0+11 Xm

j f‖20 + C(ε)

i0+1∑q=1

‖Xi0+1−q1 Xm

j f‖20

ε

k∑j=1

‖Xi0+1+mj f‖2

0 + C(ε)( k∑

j=1

‖Xi0+mj f‖2

0 + ‖f‖20

).

License or copyright restrictions may apply to redistribution; see http://www.ams.org/journal-terms-of-use

Page 17: Semilinear hypoelliptic differential operators with multiple characteristics

SEMILINEAR HYPOELLIPTIC OPERATORS 3891

Taking into account Lemmas 2.18 and 4.1, we conclude that

|I23| = |∑

|ι1|+|ι2|m+2i0+2

|ι2|≥2

C(ι1, ι2)(Xmj f, Xι2X

ι1f)0|

C∑

|ι2|+|ι3|+|ι4|m+2i0+2,

|ι2|≥2,|ι3|m+i0+1,|ι4|i0−1

|(eι4Xι4X∗

ι2Xmj f, Xι3f)0|

C∑

|ι6|i0−1,|ι3|+|ι5|+|ι6|m+2i0+2,

|ι5|3m,|ι3|m+i0+1

|(vι5,ι6Xι5Xι6Xm

j f, Xι3f)0|

ε

k∑j=1

‖Xm+i0+1j f‖2

0 + C(ε)( k∑

j=1

‖Xm+i0j f‖2

1l

+ ‖f‖20

),

in which ε is any positive number. In the same way we can get

max|I21|, |I3|, |I4| εk∑

j=1

‖Xm+i0+1j f‖2

0 + C(ε)( k∑

j=1

‖Xm+i0j f‖2

1l

+ ‖f‖20

),

for any positive ε. Thus, it is shown that

‖Xm+i0+11 f‖2

0 = (−1)i0+1Re(P2mf, X2(i0+1)1 f)0 − ‖Xi0+1

1 Xmj f‖2

0 + ReI21

+ ReI22 + ReI23 + ReI3 + ReI4 (−1)i0+1Re(P2mf, X2(i0+1)1 f)0

+ |I21| + |I22| + |I23| + |I3| + |I4| (−1)i0+1Re(P2mf, X2(i0+1)1 f)0

+ ε

k∑j=1

‖Xm+i0+1j f‖2

0 + C(ε)( k∑

j=1

‖Xm+i0j f‖2

1l

+ ‖f‖20

).

Similarly, the following inequalities hold:

‖Xm+i0+1j f‖2

0 (−1)i0+1Re(P2mf, X2i0+2j f)0

+ εk∑

j=1

‖Xm+i0+1j f‖2

0+C(ε)( k∑

j=1

‖Xm+i0j f‖2

1l+‖f‖2

0

), j=2, . . . , k.

Summing up in j from 1 to k, one obtains

k∑j=1

‖Xm+i0+1j f‖2

0 (−1)i0+1Re(P2mf,

k∑j=1

X2i0+2j f

)0

+ kε

k∑j=1

‖Xm+i0+1j f‖2

0 + C(ε)( k∑

j=1

‖Xm+i0j f‖2

1l

+ ‖f‖20

).

Choosing ε = 12k , we immediately deduce that

k∑j=1

‖Xm+i0+1j f‖2

0 C(‖P2mf‖2

−m−(i0+1)l

+ ‖f‖20

).

License or copyright restrictions may apply to redistribution; see http://www.ams.org/journal-terms-of-use

Page 18: Semilinear hypoelliptic differential operators with multiple characteristics

3892 NGUYEN MINH TRI

Replacing f by Gs+m−i0−1

l f in the just obtained estimate, in view of the result inthe first part and by the inductive assumptions we get

k∑j=1

‖Xm+i0+1j f‖2

s+m−i0−1

l

C(‖P2mf‖2s + ‖f‖2

s),

which is exactly what we need to obtain.

Remark 4.11. In the course of the proof of Theorem 4.10 it is clear that the strongerestimates

k∑j=1

‖Xm+pj f‖2

s 2p∑

i=0

Re(P2mf, (−1)i

k∑j=1

X2ij f

)s+ p−i

l

+ C‖f‖2s,

p = 0, 1, ..., m,

are still true.

Theorem 4.12. Assume that the vector fields Xjkj=1 satisfy the conditions (H)l

and (K)3ml (or (K′)3m

l ) in Ω. Then for any compact K in Ω, and any real numbers there exists a constant C(s, K) = C such that∑

|ι|2m

‖Xιf‖2

s+ 2m−|ι|l

C(‖P2mf‖2s + ‖f‖2

s), ∀ f ∈ C∞0 (K).

In other words, P2m is maximally 1l -hypoelliptic.

Proof. Using Corollary 4.4, Lemma 4.7 and Theorem 4.10 we immediately get thedesired result.

Comparing Remark 2.7 to Theorem 4.12 one obtains

Theorem 4.13. Assume that the vector fields Xjkj=1 satisfy the conditions (H)l

and (K)3ml (or (K′)3m

l ) in Ω. Then P2m is a hypoelliptic operator with loss of2m(l−1)

l derivatives.

Lemma 4.14. Assume that the vector fields Xjkj=1 satisfy the condition (K)dl

in Ω. Suppose that f, Xjfkj=1 ∈ Hs0

comp for some real number s, where by Hscomp

we denote the set of functions in Hs having compact support in Ω. Then Xιf ∈H

s0− |ι|−1l

comp for all multi-orders ι such that 1 < |ι| d.

Proof. Let χε∗ be the averaging operator. In view of the condition (K)dl we have

‖χε ∗ Xιf‖s0− |ι|−1l

‖Xι(χε ∗ f)‖s0− |ι|−1

l+ ‖[χε ∗ Xι]f‖s0− |ι|−1

l

k∑

j=1

‖Xj(χε ∗ f)‖s0 + C k∑

j=1

(‖χε ∗ (Xjf)‖s0 + ‖[Xj , χε∗]f‖s0) + C C

for all sufficiently small ε. Tending ε → 0 we obtain the desired result.

Lemma 4.15. Assume that the vector fields Xjkj=1 satisfy the conditions (H)l

in Ω. Suppose that f ∈ E ′(Ω) and Xjfkj=1 ∈ Hs0

comp for some real value s0. Then

f ∈ Hs0+

1l

comp .

License or copyright restrictions may apply to redistribution; see http://www.ams.org/journal-terms-of-use

Page 19: Semilinear hypoelliptic differential operators with multiple characteristics

SEMILINEAR HYPOELLIPTIC OPERATORS 3893

Proof. Indeed, there exists a number σ such that f ∈ Hσcomp. Putting t = min(σ, s0),

we have f, Xjfkj=1 ∈ Ht

comp. From the condition (H)l it follows that

‖χε ∗ f‖t+ 1l

C( k∑

j=1

‖χε ∗ (Xjf)‖t +k∑

j=1

‖[Xj , χε∗]f‖t + ‖χε ∗ f‖t

) C

for all sufficiently small ε. Hence f ∈ Ht+ 1

lcomp. Repeating the argument again and

again we come to the needed conclusion.

From Lemmas 4.14 and 4.15, it immediately follows that

Corollary 4.16. Assume that the vector fields Xjkj=1 satisfy the conditions (H)l

and (K)dl in Ω. Suppose that for some real value s0 and natural number p the

following inclusions hold: f ∈ E ′(Ω), Xιf ∈ Hs0− |ι|

lcomp for all multi-orders ι such that

1 |ι| p. Then Xι1Xι2f ∈ H

s0− |ι1|+|ι2|l

comp for all multi-orders ι1 and ι2 such that|ι1| d, 0 |ι2| p − 1.

Lemma 4.17. Assume that the vector fields Xjkj=1 satisfy the condition (H)l

in Ω. Suppose that f ∈ Hs0loc(Ω), Xjfk

j=1 ∈ Hs0loc(Ω) for some real value s0. Then

f ∈ Hs0+

1l

loc (Ω).

Proof. Let g0 ∈ C∞0 (Ω). We see that

Xj(g0f) = g0Xjf + [Xj , g0]f ∈ Hs0comp, j = 1, k.

From Lemma 4.15, it follows that g0f ∈ Hs0+

1l

comp . Hence f ∈ Hs0+

1l

loc (Ω) in view ofarbitrariness of g0. Lemma 4.17 is therefore proved.

Remark 4.18. The assumption f ∈ Hs0loc(Ω) in Lemma 4.17 can be replaced by

f ∈ D′(Ω), or in other words, redundant. In fact, let Ω′ Ω. Then we canfind a number σ such that f ∈ Hσ

loc(Ω′). Put t = mins0, σ. It is clear that

f, Xjfkj=1 ∈ Ht

loc(Ω′). Using Lemma 4.17 we obtain f ∈ H

t+ 1l

loc (Ω′). Repeating

the above argument again and again we conclude that f ∈ Hs0+

1l

loc (Ω′). In view of

the arbitrariness of Ω′ we arrive at f ∈ Ht+ 1

l

loc (Ω).

Corollary 4.19. Assume that the vector fields Xjkj=1 satisfy the condition (H)l

in Ω. If Xιf ∈ Hs0loc(Ω) for some real value s0 and all multi-orders ι such that

|ι| p, then Xιf ∈ Hs0+

p−|ι|l

loc (Ω) for all multi-orders ι such that |ι| p − 1.

Lemma 4.20. Assume that the vector fields Xjkj=1 satisfy the conditions (H)l

and (K)dl in Ω. Suppose that for some real value s0 and natural integer p the

inclusions Xιf ∈ Hs0− |ι|

l

loc (Ω) hold for all multi-orders ι such that 1 |ι| p. Then

Xι1Xι2f ∈ H

s0− |ι1|+|ι2|l

loc (Ω) for all multi-orders ι1, ι2 such that |ι1| d, 0 |ι2| p − 1.

Proof. Let g0 ∈ C∞0 (Ω) as in Lemma 4.17. Then

Xj(g0Xι2f) = g0XjX

ι2f + [Xj , g0]Xι2f ∈ Hs0− |ι2|+1

lcomp , j = 1, . . . , k.

License or copyright restrictions may apply to redistribution; see http://www.ams.org/journal-terms-of-use

Page 20: Semilinear hypoelliptic differential operators with multiple characteristics

3894 NGUYEN MINH TRI

By Lemma 4.14 we have Xι1(g0Xι2f) ∈ H

s0− |ι1|+|ι2|l

comp . Hence

g0Xι1Xι2f = Xι1(g0X

ι2f) + [g0, Xι1 ]Xι2f ∈ Hs0− |ι1|+|ι2|

lcomp .

In view of the arbitrariness of g0 we deduce that Xι1Xι2f ∈ H

s0− |ι1|+|ι2|l

loc (Ω).Lemma 4.20 is therefore proved.

Proposition 4.21. Assume that the vector fields Xjkj=1 satisfy the conditions

(H)l and (K)dl in Ω, and Qm is (see the formula (2.1)) maximally 1l -hypoelliptic.

Then from f ∈ D′(Ω), Qmf ∈ Hsloc(Ω) it follows that Xιf ∈ H

s+ m−|ι|l

loc (Ω) for all

multi-orders ι such that 0 |ι| m and Xι1Xι2f ∈ H

s+ m−|ι1|−|ι2|l

loc (Ω) for allmulti-orders ι1, ι2 such that |ι1| d, 0 |ι2| m − 1.

Proof. From the definition of maximal 1l -hypoellipticity it follows that Xιf ∈

Hs+ m−|ι|

l

loc (Ω) for all multi-orders ι such that 0 |ι| m. The proof of this fact issimilar to that for Lemma 4.14. Now the last statement of Proposition 4.21 followsdirectly from Lemma 4.20.

Theorem 4.22. Assume that the vector fields Xjkj=1 satisfy the conditions (H)l

and (K)l in Ω. Then P2m is an extendedly maximally hypoelliptic operator in thesystem Xjk

j=1.

Proof. We prove that if P2mf ∈ Sq,Xjk

j=1loc (Ω), then f ∈ S

q+2m,Xjkj=1

loc (Ω). To this

end we establish that P2m(Xιf) ∈ Hq−|ι|

l

loc (Ω) by induction on ι with 0 |ι| q.

For |ι| = 0 the claim is trivial since P2mf ∈ Sq,Xjk

j=1loc (Ω) ⊂ H

ql

loc(Ω). Assumingthat our claim is true for |ι| = q0 q−1, we prove it for all multi-orders ι such that

|ι| = q0 + 1. From the inductive assumption we deduce that Xϑf ∈ Hq+2m−|ϑ|

l

loc (Ω)for all multi-orders ϑ such that |ϑ| 2m+q0. On the other hand, for all multi-ordersι with |ι| = q0 + 1 we have

P2m(Xιf) = Xι(P2mf) +∑

|ι1|+|ι2|2m+q0+1

|ι2|2m+q0−1

aι1,ι2Xι1Xι2f.

Hence by Lemma 4.20, we conclude that P2m(Xιf) ∈ Hq−(q0+1)

l

loc (Ω), which is exactlythe claim for |ι| = q0 + 1. Now for |ι| = q we have P2m(Xιf) ∈ L2

loc(Ω). From themaximal 1

l -hypoellipticity of P2m we deduce that Xϑf ∈ L2loc(Ω) for all multi-orders

ϑ with |ϑ| 2m + q. The proof of Theorem 4.22 is therefore complete.

Remark 4.23. From the microlocal point of view the condition (H)l can be formu-lated for a system Xjk

j=1, consisting of proper first order p.d.o. with real principalsymbols. Theorems analogous to Theorems 4.10 - 4.22 remain valid for operatorsof the form

P ′2m =

k∑j=1

Aj(x, ∂)X2mj +

∑|ι|2m−1

Bι(x, ∂)Xι,

where Aj(x, ∂), Bι(x, ∂) are proper zero order p.d.o. Furthermore, Aj(x, ∂) aresupposed to be elliptic and positive operators.

License or copyright restrictions may apply to redistribution; see http://www.ams.org/journal-terms-of-use

Page 21: Semilinear hypoelliptic differential operators with multiple characteristics

SEMILINEAR HYPOELLIPTIC OPERATORS 3895

In the particular case when the condition (H)2 is satisfied, the maximal 12 -

hypoellipticity of the operator P ′2m was proved in [4, 5]. In this particular case, from

the condition (H)2 alone we can deduce the condition (K)2. As was pointed out in[5], the condition (H)2 follows from the noninvolutiveness of the characteristic setof P ′

2m in the cotangent space.

4.1.2. Some examples. In practice it may not be easy to verify the conditions (K)dland (K′)dl . In this part we give some examples of systems that satisfy the condition(K)dl .

a) Complete system of vector fields degenerate on a submanifold. Let the vari-able x ∈ R

n be separated into two groups x = (x′, y), where x′ = (x1, . . . , xn1),y = (y1, . . . , yn2), n1 + n2 = n. Consider the following system of vector fields:

X1 =∂

∂x1, . . . , Xn1 =

∂xn1

, X1,1 = xl1

∂y1, . . . , X1,n2 = xl

1

∂yn2

, . . . ,

Xn1,1 = xln1

∂y1, . . . , Xn1,n2 = xl

n1

∂yn2

.

Clearly, the system X1, . . . , Xn1 , X1,1, . . . , X1,n2 , . . . , Xn1,1, . . . , Xn1,n2 satisfies thecondition (H)l+1. Now we show that it satisfies the condition (K)l+1, too. Indeed,only the commutators [ iXj , Xj,j′ ], 1 i l, j = 1, n1, j

′ = 1, n2, differ fromzero. We will estimate these commutators on the space C∞

0 (K), where K is a fixedcompact in Ω. By using the Cauchy - Schwarz inequality, for any two real valuess, s1 with s − 1 s1 and all ε > 0 there exists a constant C(ε) such that∥∥∥xi−1

1

∂f

∂y1

∥∥∥s

ε∥∥∥xi

1

∂f

∂y1

∥∥∥2s−s1

+ C(ε)∥∥∥xi−2

1

∂f

∂y1

∥∥∥s1

, 2 i.

Hence, ∥∥∥xl−11

∂f

∂y1

∥∥∥0

ε∥∥∥xl

1

∂f

∂y1

∥∥∥1

l+1

+ C1(ε)∥∥∥xl−2

1

∂f

∂y1

∥∥∥− 1

l+1

, 2 l.

From C1(ε) we can find a constant C2(ε) such that(2 + C1(ε)

)∥∥∥xl−21

∂f

∂y1

∥∥∥− 1

l+1

∥∥∥xl−1

1

∂f

∂y1

∥∥∥0

+ C2(ε)∥∥∥xl−3

1

∂f

∂y1

∥∥∥− 2

l+1

.

Continuing the process, on the i-th (i l − 1) step we obtain(2 + Ci−1(ε)

)∥∥∥xl−i1

∂f

∂y1

∥∥∥− i−1

l+1

∥∥∥xl−i+1

1

∂f

∂y1

∥∥∥− i−2

l+1

+ Ci(ε)∥∥∥xl−i−1

1

∂f

∂y1

∥∥∥− i

l+1

.

The process will finish on the (l − 1)-th step, on which we have(2 + Cl−2(ε)

)∥∥∥x1∂f

∂y1

∥∥∥− l−2

l+1

∥∥∥x2

1

∂f

∂y1

∥∥∥− l−3

l+1

+ Cl−1(ε)∥∥∥ ∂f

∂y1

∥∥∥− l−1

l+1

.

Now applying the estimate for commutators

‖[X, Y ]f‖s ε(‖Xf‖s1 + ‖f‖s1) + C(ε)(‖Y f‖1+2s−s1 + ‖f‖1+2s−s1), ∀s, s1 ∈ R,

to ∂∂y1

= [ ∂∂x1

, x1∂

∂y1], we conclude that from Cl−1(ε) there can be found a constant

Cl(ε) such that(2n1n2 + Cl−1(ε)

)∥∥∥ ∂f

∂y1

∥∥∥− l−1

l+1

∥∥∥x1

∂f

∂y1

∥∥∥− l−2

l+1

+ Cl(ε)(∥∥∥ ∂f

∂x1

∥∥∥1

l+1

+ ‖f‖ 1l+1

).

License or copyright restrictions may apply to redistribution; see http://www.ams.org/journal-terms-of-use

Page 22: Semilinear hypoelliptic differential operators with multiple characteristics

3896 NGUYEN MINH TRI

On the other hand, based on the two well-known inequalities

(4.10) 2C‖f‖ 2l+1

n1∑

j=1

∥∥∥ ∂f

∂xj

∥∥∥1

l+1

+n2∑

j=1

∥∥∥ ∂f

∂yj

∥∥∥− l−1

l+1

+ ‖f‖0,

where the constant C does not depend on C1(ε), . . . , Cl(ε), and

C(ε)‖f‖ 1l+1

C‖f‖ 2l+1

+ C(ε)‖f‖0,

from Cl(ε) one can find Cl+1(ε) such that

(2 + Cl(ε))‖f‖ 1l+1

+ C‖f‖ 2l+1

n1∑

j=1

∥∥∥ ∂f

∂xj

∥∥∥1

l+1

+n2∑

j=1

∥∥∥ ∂f

∂yj

∥∥∥− l−1

l+1

+ Cl+1(ε)‖f‖0.

By the same trick we can obtain similar estimates for xj∂f

∂yj′, j = 1, n1, j

′ = 1, n2.Summing up all the obtained inequalities, then reducing similar terms in both sides,one can find

max‖f‖ 2

l+1,∥∥∥xl−i

j

∂f

∂yj′

∥∥∥− i−1

l+1

(j = 1, n1, j′ = 1, n2, 1 i l)

ε

n1∑j=1

n2∑j′=1

∥∥∥xlj

∂f

∂yj′

∥∥∥1

l+1

+ C(ε)( n1∑

j=1

∥∥∥ ∂f

∂xj

∥∥∥1

l+1

+ ‖f‖0

).

Since all the operators in the above estimate have order 1, by standard argumentswe deduce

max‖f‖s+ 2

l+1,∥∥∥xl−i

j

∂f

∂yj′

∥∥∥s− i−1

l+1

(j = 1, n1, j′ = 1, n2, 1 i l)

ε

n1∑j=1

n2∑j′=1

∥∥∥xlj

∂f

∂yj′

∥∥∥s+ 1

l+1

+ C(ε)( n1∑

j=1

∥∥∥ ∂f

∂xj

∥∥∥s+ 1

l+1

+ ‖f‖0

), ∀s ∈ R.(4.11)

If we take into account the formula xl−i1

∂f∂y1

≈ [ i∂f∂x1

, xl1

∂f∂y1

], by choosing ε = 1,we arrive at the desired result. Thus, the system X1, . . . , Xn1 , X1,1, . . . , X1,n2 , . . . ,Xn1,1, . . . , Xn1,n2 satisfies the condition (K)l+1. Therefore, according to Subsub-section 4.1.1, we can formulate a theorem on the maximal 1

l+1 -hypoellipticity ofthe operator P2m consisting of the vector fields from the system. Here we will notdo it, leaving the details to the readers. But we note that the estimate (4.11), infact, is stronger than the ones required in the condition (K)l+1.

b) Noncomplete system of vector fields degenerate on a submanifold. Let thevariable x∈R

n be separated into three groups x=(x′, y, z), where x′=(x1, . . . , xn1),y = (y1, . . . , yn2), z = (z1, . . . , zn3), n1 +n2 +n3 = n. Consider the following systemof vector fields:

X1 =∂

∂x1, . . . , Xn1 =

∂xn1

, X1,1 = xl1

∂y1, . . . , X1,n2 = xl

1

∂yn2

, . . . ,

Xn1,1 = xln1

∂y1, . . . , Xn1,n2 = xl

n1

∂yn2

, Z1 =∂

∂z1, . . . , Zn3 =

∂zn3

.

Again the system X1, . . . , Xn1 , X1,1, . . . , X1,n2 , . . . , Xn1,1, . . . , Xn1,n2 , Z1, . . . , Zn3

satisfies the condition (H)l+1. Also, in this case one can show that this system

License or copyright restrictions may apply to redistribution; see http://www.ams.org/journal-terms-of-use

Page 23: Semilinear hypoelliptic differential operators with multiple characteristics

SEMILINEAR HYPOELLIPTIC OPERATORS 3897

satisfies the condition (K)l+1, too. Indeed, all the arguments used in the previousexample can be applied here. Only one estimate (4.10) needs to be modified by

2C‖f‖ 2l+1

n1∑

j=1

∥∥∥ ∂f

∂xj

∥∥∥1

l+1

+n2∑

j=1

∥∥∥ ∂f

∂yj

∥∥∥− l−1

l+1

+n3∑

j=1

∥∥∥ ∂f

∂zj

∥∥∥− l−1

l+1

+ ‖f‖0.

Hence from Cl(ε) one can find Cl+1(ε) such that

(2 + Cl(ε))‖f‖ 1l+1

+ C‖f‖ 2l+1

n1∑

j=1

∥∥∥ ∂f

∂xj

∥∥∥1

l+1

+n2∑

j=1

∥∥∥ ∂f

∂yj

∥∥∥− l−1

l+1

+n3∑

j=1

∥∥∥ ∂f

∂zj

∥∥∥− l−1

l+1

+ Cl+1(ε)‖f‖0.

Finally we can establish the following estimates for all s ∈ R:

max‖f‖s+ 2

l+1,∥∥∥xl−i

j

∂f

∂yj′

∥∥∥s− i−1

l+1

(j = 1, n1, j′ = 1, n2, 1 i l)

ε

n1∑j=1

n2∑j′=1

∥∥∥xlj

∂f

∂yj′

∥∥∥s+ 1

l+1

+ C

n3∑j=1

∥∥∥ ∂f

∂zj

∥∥∥s− l−1

l+1

+ C(ε)( n1∑

j=1

∥∥∥ ∂f

∂xj

∥∥∥s+ 1

l+1

+ ‖f‖0

).

Note that the estimates we get here in the variables z1, . . . , zn3 are better thanwhat is required in the condition (K)l+1. Again a theorem on the maximal 1

l+1 -hypoellipticity can be formulated for the operator P2m constructing from the vectorfields X1, . . . , Xn1 , X1,1, . . . , X1,n2 , . . . , Xn1,1, . . . , Xn1,n2 , Z1, . . . , Zn3 . We omit thedetails.

4.1.3. Semilinear equations. In this part we study a semilinear perturbation of theoperator P2m considered in Subsubsection 4.1.1:

Ψ2m(x, ∂α) : f −→ P2m(x, D)f + Φ(x, Xιf)|ι|2m−1.

First we establish a theorem of general character for the nonlinear operator Θm

given by the formula (2.3).

Theorem 4.24. Let Qm be extendedly maximally hypoelliptic in the system Xjkj=1

and the function Φ(x, τι)|ι|m−1 is infinitely differentiable in its variables. Supposethat the system Xjk

j=1 satisfies the smoothness condition. Then the nonlinearoperator Θm is extendedly maximally hypoelliptic in the system Xjk

j=1. In par-ticular, it is hypoelliptic.

Proof. In view of the smoothness condition of the system Xjkj=1 and the Sobolev

embedding theorem one can find a number M such that SM,Xjk

j=1loc (Ω) ⊂ C(Ω).

Now supposing that m1 ≥ 2M+m we can repeat the arguments used in Proposition

3.2 for getting the following claim: if f ∈ Sm1,Xjk

j=1loc (Ω), then Φ(x, Xιf)|ι|m−1 ∈

Sm1−(m−1),Xjk

j=1loc (Ω). Further, based on the claim we just obtained, we can apply

the technique used in the proof of Theorem 3.3 to get the desired result.

From Theorems 4.22 and 4.24 we get

Theorem 4.25. Let the vector fields Xjkj=1 satisfy the conditions (H)l and

(K)l (or (K′)l) in Ω. If the function Φ(x, τι)|ι|2m−1 is infinitely differentiable inits variables, then the nonlinear operator Ψ2m is hypoelliptic. Furthermore, it isextendedly maximally hypoelliptic in the system Xjk

j=1.

License or copyright restrictions may apply to redistribution; see http://www.ams.org/journal-terms-of-use

Page 24: Semilinear hypoelliptic differential operators with multiple characteristics

3898 NGUYEN MINH TRI

From the results in Subsubsection 4.1.2 of this section, based on Theorem 4.25we give two examples

Example 4.26. The nonlinear operator

Ψ(1)2m(f) =

∂2mf

∂x2m+ x2mk

(∂2mf

∂y2m+

∂2mf

∂z2m

)+ cos

(x(2m−1)k ∂2m−1f

∂y2m−1

)e∂2m−1f

∂x2m−1

is hypoelliptic in R3.

Example 4.27. The nonlinear operator

Ψ(2)2m(f) =

∂2mf

∂x2m+ x2mk ∂2mf

∂y2m+

∂2mf

∂z2m+

(∂2m−1f

∂x2m−1

)5(xk ∂f

∂y

)2

is hypoelliptic in R3.

4.2. Maximal δ-hypoellipticity without condition (K)l. In this part we in-vestigate hypoellipticity of the semilinear operator Ψ4, however, without the con-dition (K)l. We will prove theorems on the weakly maximal δ-hypoellipticity of theoperator P4, then its maximal δ-hypoellipticity, and finally the extendedly maxi-mal hypoellipticity of its semilinear perturbation. First, we establish a theorem ofgeneral character for Qm.

Lemma 4.28. Let Qm be weakly maximally δ-hypoelliptic in Ω. Assume thatXιf, Qmf ∈ Hs

comp (0 |ι| m − 1) for some real number s. Then Xιf ∈H

s+δ(m−|ι|)comp for 0 |ι| m − 1.

Proof. We prove Lemma 4.28 by induction on |ι|. In view of the weakly maximalδ-hypoellipticity of Qm we have

‖χε ∗ f‖s+δm C(‖Qm(χε ∗ f)‖s + ‖χε ∗ f‖s) C(‖χε ∗ Qmf‖s + ‖χε ∗ f‖s

+∑|ι|m

‖[aι, χε∗]Xιf‖s +∑|ι|m

∑ι1⊆ι,ι1 =ι

‖[ ι\ι1X, χε∗]Xι1f‖s) := I.(4.12)

By the assumptions of Lemma 4.28 the right hand side of (4.12) is bounded (uni-formly in ε). Hence, tending ε → 0 we conclude that f ∈ Hs+δm

comp . Thus, Lemma4.28 is proved for |ι| = 0. Assuming that Lemma 4.28 is true for |ι| p m − 2,we prove it for |ι| = p + 1. Again in view of the maximal δ-hypoellipticity of Qm

the following inequality holds:

‖Xι(χε ∗ f)‖s+δ(m−p−1) I for |ι| = p + 1.

On the other hand,

‖χε ∗ Xιf‖s+δ(m−p−1) ‖Xι(χε ∗ f)‖s+δ(m−p−1)

+∑

ι1⊆ι,ι1 =ι

‖[ ι\ι1X, χε∗]Xι1f‖s+δ(m−p−1) I + J.

By the inductive assumptions, |J | is bounded. Therefore, ‖χε ∗ Xιf‖s+δ(m−p−1) isbounded. Hence Xιf ∈ H

s+δ(m−p−1)comp for |ι| = p + 1. Lemma 4.28 is proved.

Corollary 4.29. Let Qm be weakly maximally δ-hypoelliptic in Ω. If f ∈ E ′(Ω),Qmf ∈ Hs for some real number s, then Xιf ∈ Hs+δ(m−|ι|) for 0 |ι| m − 1.

License or copyright restrictions may apply to redistribution; see http://www.ams.org/journal-terms-of-use

Page 25: Semilinear hypoelliptic differential operators with multiple characteristics

SEMILINEAR HYPOELLIPTIC OPERATORS 3899

Proof. Since f ∈ E ′(Ω) there exists a number σ such that Xιf ∈ Hσ for |ι| m−1.Put t = min(s, σ). Applying Lemma 4.28 we get Xιf ∈ Ht+δ(m−|ι|). We can repeatthe argument again and again with min(s, t+δ) instead of t and obtain the analogousclaim. After a number of steps we arrive at Xιf ∈ Hs for |ι| m− 1. Again usingLemma 4.28 we deduce the desired result.

Theorem 4.30. Let Qm be weakly maximally δ-hypoelliptic in Ω. Then Qm ishypoelliptic in Ω.

Proof. The proof of Theorem 4.30 is based on a standard procedure and Corollary4.29. We give it for completeness. We will prove that if s ∈ R and Qmf ∈ Hs

loc(Ω),then

Xιf ∈ Hs+δ(m−|ι|)loc (Ω), 0 |ι| m − 1.

Let Ω′ Ω and f ∈ D′(Ω). Then f ∈ Hσloc(Ω

′) for some σ ∈ R. We will use thefollowing notation from [15]: for two functions a, b ∈ C∞

0 (Ω) we write a b if b ≡ 1on the support a. Take a sequence of C∞-functions gi∞i=0 such that gi gi+1.Obviously,

(4.13) Qm(gif) = giQmf +∑

ι1⊆ι,ι1 =ι

aι[ ι\ι1X, gi]Xι1(gi+1f).

Put t = min(s, σ). Then the right hand side of (4.13) belongs to Ht−m+1, i.e.Qm(gif) ∈ Ht−m+1. Hence, by Corollary 4.29

Xι(gif) ∈ Ht−m+1+δ(m−|ι|) (0 |ι| m − 1).

Thus, the right hand side of (4.13) belongs to Ht−m+1+δ. Therefore

Xι(gi−1f) ∈ Ht−(m−1)+δ(m+1−|ι|) (0 |ι| m − 1).

Repeating the argument again a finite number of times we obtain Qm(g0f) ∈ Ht.Hence

Xι(g0f) ∈ Ht+δ(m−|ι|) (0 |ι| m − 1).Thus, we have proved

f ∈ Htloc(Ω), Qmf ∈ Ht

loc(Ω) ⇒ f ∈ Ht+δmloc (Ω′), Xιf ∈ H

t+δ(m−|ι|)loc (Ω′),

0 |ι| m − 1.

Again and again using the statement just obtained we come to t = s, and in viewof arbitrariness of Ω′ we get the desired result.

Lemma 4.31. Let the vector fields Xjkj=1 satisfy the condition (H)3 in Ω. Then

for any compact K in Ω, and each real value s there exists a constant C(s, K) = Csuch that

‖Xιf‖2s C

( k∑j=1

‖X3j f‖2

s+ 16

+ ‖f‖2s

)(|ι| = 3), ∀f ∈ C∞

0 (K).(4.14)

Proof. Denote the right hand side of (4.14) by Is. Note that we can write ι = ι1+ι2

with |ι1| = 2, |ι2| = 1. By Theorem 2.4

‖Xιf‖2s = ‖Xι1(Xι2f)‖2

s C( k∑

j=1

‖X2j (Xι2f)‖2

s + ‖f‖2s

).

License or copyright restrictions may apply to redistribution; see http://www.ams.org/journal-terms-of-use

Page 26: Semilinear hypoelliptic differential operators with multiple characteristics

3900 NGUYEN MINH TRI

Thus, the matter reduces to estimating the terms of the form ‖X2j1

Xj2f‖2s, j1 = j2.

We have

‖X2j1Xj2f‖2

0 = (X2j1Xj2f, X2

j1Xj2f)0 |(Xj1Xj2f, Xj2X3j1f)0|

+3∑

i=1

Ci3|(Xj1Xj2f, [ iXj1 , Xj2 ]X

3−ij1

f)0| + ε‖X2j1Xj2f‖2

0 + C(ε)I0

|(Xj2Xj1Xj2f, X3j1f)0| + C

k∑j=1

‖X2j f‖ 1

2+ ε‖X2

j1Xj2f‖20 + C(ε)I0

ε

k∑j=1

‖X2j Xj2f‖2

0 + C(ε)I0.

Choosing ε = 12k3 and then summing up the inequalities just obtained one gets

T (f) T (f)2

+ CI0,

where T (f) denotes maxj1,j2=1,k‖X2j1

Xj2f‖20, from which the needed estimate in

the case s = 0 follows. To prove the estimate (4.14) in the general case s ∈ R weneed to substitute Gsf for f in the inequality just obtained. We have(4.15)

‖X2j1

Xj2Gsf‖2

0 C( k∑

j=1

‖X3j Gsf‖2

16

+ ‖Gsf‖20

) C

( k∑j=1

‖X3j f‖2

s+ 16

+ ‖f‖2s

).

On the other hand,

‖X2j1Xj2f‖2

s C(‖GsX2j1Xj2f‖2

0 + ‖f‖2s) C(‖X2

j1Xj2Gsf‖2

0 + ‖f‖2s)

+ C∑

ι1⊆(j1,j1,j2),|ι1|2

‖[ (j1,j1,j2)\ι1X, Gs]Xι1f‖20

C‖X2j1Xj2G

sf‖20 + Is.(4.16)

Comparing the inequalities (4.15), (4.16) we arrive at the desired conclusion.

Proposition 4.32. Let the vector fields Xjkj=1 satisfy the condition (H)3 in

Ω. Then for each compact K in Ω, and any real value s there exists a constantC(s, K) = C such that

∑0|ι|2

‖Xιf‖2s+ 4−|ι|

3 C(‖P4f‖2

s + ‖f‖2s),(4.17)

∑|ι|=3

‖Xιf‖2s C

( k∑j=1

‖X3j f‖2

s+ 16

+ ‖f‖2s

) C(‖P4f‖2

s + ‖f‖2s)(4.18)

for all f ∈ C∞0 (K).

Proof. The estimate (4.17) can be proved as in Lemma 4.9 with the use of Theorem2.4 and Lemma 4.2. It remains to prove the estimate (4.18). We first show it for

License or copyright restrictions may apply to redistribution; see http://www.ams.org/journal-terms-of-use

Page 27: Semilinear hypoelliptic differential operators with multiple characteristics

SEMILINEAR HYPOELLIPTIC OPERATORS 3901

the case s = 0. We calculate the scalar product of −(P4f, X21f)0. By Lemma 4.31

|∑

0|ι|3

(aιXιf, X2

1f)0| ε∑

0|ι|3

‖Xιf‖2− 1

6+ C(ε)‖X2

1f‖216

ε

k∑j=1

‖X3j f‖2

0 + C(ε)‖f‖20.

We now estimate the principal terms in −(P4f, X21f)0. We have

−( 4∑

j=1

X4j f, X2

1f)

0= −(X4

1f, X21f)0 −

k∑j=2

(X4j f, X2

1f)0 := I1 +k∑

j=2

Ij .

It is easy to check that

I1 = ‖X31f‖2

0 + J1, |J1| ε‖X31f‖2

0 + C(ε)‖f‖20.

Next we estimate the term Ij with 2 j k. By Lemma 4.31

Ij = −(X2j f, X2

j X21f)0 + Aj , where |Aj | ε

k∑j=1

‖X3j f‖2

0 + C(ε)‖f‖20.

Furthermore,

−(X2j f, X2

j X21f)0 = ‖X1X

2j f‖2

0 + Bj +∑

|ι1|+|ι2|4

|ι1|≥2

C(ι1, ι2)(X2j f, Xι1X

ι2f)0,

where as before |Bj | ε

k∑j=1

‖X3j f‖2

0 + C(ε)‖f‖20.

On the other hand, in view of the estimate (4.17)

|(X2j f, Xι1X

ι2f)0| C(‖P4f‖2− 1

6+ ‖f‖2

0) for |ι2| 2.

Combining all the obtained inequalities we arrive at the needed result in the cases = 0. In order to obtain the estimate (4.18) in the general case s ∈ R we substituteGsf for f in the inequality just obtained for s = 0. We omit the details.

Combining Theorem 4.30 and Proposition 4.32 we get

Theorem 4.33. Let the vector fields Xjkj=1 satisfy the condition (H)3 in Ω.

Then P4 is a hypoelliptic operator.

Proof. We show that if s ∈ R and P4f ∈ Hsloc(Ω), then

Xιf ∈ Hs+ 4−|ι|

3loc (Ω) for |ι| 2, Xιf ∈ Hs

loc(Ω) for |ι| = 3,

and X3j f ∈ H

s+ 16

loc (Ω) for j = 1, k,

from which the theorem follows. The above claim can first be achieved in the casewhen f has compact support based on the estimates (4.17), (4.18), Proposition 4.32and the analogous tricks given in the proof of Corollary 4.29. Furthermore, usinga sequence of functions gi(x) as in Theorem 4.30 we can get the desired result inthe general case.

License or copyright restrictions may apply to redistribution; see http://www.ams.org/journal-terms-of-use

Page 28: Semilinear hypoelliptic differential operators with multiple characteristics

3902 NGUYEN MINH TRI

Remark 4.34. A theorem analogous to Theorem 4.33 still remains true for theoperator of the form

P ′4 =

k∑j=1

Aj(x, ∂)X4j +

∑|ι|3

Bι(x, ∂)Xι,

where Xjkj=1 are proper first order p.d.o. with real principal symbols, and

Aj(x, ∂), Bι(x, ∂) are proper zero order p.d.o. Furthermore, operators Aj(x, ∂)are assumed to be elliptic and positive.

We now go over to the proof of the maximal 13 -hypoellipticity and the extend-

edly maximal hypoellipticity of the operator P4. To this end we need the liftingconstruction from the vector fields Xjk

j=1 in Ω to the vector fields Xjkj=1 in Ω,

where the new system is “free up” to the third step and satisfies the condition (H)3in Ω. First we prove a theorem concerning the “homogeneous” operator

P(0)4 =

k∑j=1

X4j

in Ω. We will adopt the terminologies and definitions from the paper [15].

Lemma 4.35. Let the system of vector fields Xjkj=1 be free up to the third step

and satisfy the condition (H)3 in Ω, and a ∈ C∞0 (Ω). There exist operators T , S, S′

such that

i) T is of type 4.ii) S, S′ are of type 1.iii) P

(0)4 T = aI + S, T P

(0)4 = aI + S′, where by I we denote the unit transform.

Proof. By Theorem 5 from [15], the vector fields Xjkj=1 can be approximated by

the vector fields Yjkj=1 on the group Gk,3 (recall that the vector fields Yjk

j=1

form a basis of the Lie algebra gk,3 and dim Ω = dim Gk,3). Consider the operator

Q4 =k∑

j=1

Y 4j .

By Theorem 4.33, the operator Q4 and its formal adjoint are hypoelliptic on Gk,3.Therefore by Proposition A in [15], taking into account the fact that 4 < dimGk,3

there can be found a fundamental solution K4(u) of the operator Q4, satisfying thefollowing conditions:

1) Function K4 is of type 4 on the group Gk,3.2) The following formula holds: Q4K4(u) = δ(u), where δ is the Dirac measure

on the group Gk,3.

Let a b. Put

K4(x, y) = a(x)K4(Θ(x, y))b(y)

and define the operator T by

T f(x) =∫

Ω

K4(x, y)f(y)dy.

License or copyright restrictions may apply to redistribution; see http://www.ams.org/journal-terms-of-use

Page 29: Semilinear hypoelliptic differential operators with multiple characteristics

SEMILINEAR HYPOELLIPTIC OPERATORS 3903

In order to verify the properties ii) and iii) in Lemma 4.35, note that

P(0)4 =

k∑j=1

X4j =

k∑j=1

(Yj + Rj)4 =k∑

j=1

Y 4j + ∆,

where ∆ is a differential operator of local degree 3. Hence,

∆K4(x, y) = K ′, P(0)4 K4(x, y) = a(x)(Q4K4(u))b(y) + K ′′ = a(x)δ(x − y) + K ′′′,

where K ′, K ′′, K ′′′ are kernels of type 1. Thus we get the claim for P(0)4 T . Inte-

grating by parts we can obtain the claim for T P(0)4 . Lemma 4.35 is proved.

Theorem 4.36. Let the system of vector fields Xjkj=1 be free up to step 3 and

satisfy the condition (H)3 in Ω, and let there be given functions a, b, c ∈ C∞0 (Ω)

such that a b c. Then for each positive integer r there can be found operatorsTr and Sr such that

Tr bP(0)4 = aI + Sr · c,

where Tr is a smoothing operator of order 4, and Sr is a smoothing operator oforder r.

Proof. Let there be given three functions α, β, γ in C∞0 (Ω) such that α β γ.

We can always find a function β0 such that α β0 β. Now applying Lemma4.35 with a replaced by α and f replaced by β0f we obtain T (P (0)

4 β0) = αI + S′β0.However,

P(0)4 (β0f) = β0P

(0)4 (f) + f P

(0)4 (β0)

+k∑

j=1

(4Xj(β0)X3j (f) + 6X2

j (β0)X2j (f) + 4X3

j (β0)Xj(f)).

Now put T1 = T β0 and

S1f = S′(β0f)− f P(0)4 (β0)−

k∑j=1

(4Xj(β0)X3j (f)+6X2

j (β0)X2j (f)+4X3

j (β0)Xj(f)).

It is not difficult to check that S1γf = S1f . Thus,

T1(βP4(f)) = αf + S1(γf).

The last equality is the claim of Theorem 4.36 for r = 1. In order to prove thetheorem for general r we need to repeat the arguments used in the proof of Corollary17.4 in [15] combined with the claim just obtained for r = 1. We will not elaborate.Theorem 4.36 is proved.

Theorem 4.37. Let the system of vectors fields Xjkj=1 be free up to step 3 and

satisfy the condition (H)3 in Ω. Then P4 is maximally 13 -hypoelliptic and extendedly

maximally hypoelliptic in the system Xjkj=1 in Ω.

Proof. In view of Remark 2.8, it suffices to establish the maximal 13 -hypoellipticity

and the extendedly maximal hypoellipticity of P(0)4 in the system Xjk

j=1. First weprove the maximal 1

3 -hypoellipticity of P 04 . Note that the estimates for f , Xj fk

j=1,

Xιf|ι|=2 were obtained in Proposition 4.32. It remains to establish the estimates

License or copyright restrictions may apply to redistribution; see http://www.ams.org/journal-terms-of-use

Page 30: Semilinear hypoelliptic differential operators with multiple characteristics

3904 NGUYEN MINH TRI

for Xιf|ι|=3, Xιf|ι|=4. Let K be a compact in Ω and a ∈ C∞0 (Ω), a ≡ 1 on K.

Applying Theorem 4.36 with the help of Theorems 8 and 12 in [15] we have

(4.19) ‖Xιf‖s C(‖XιT P(0)4 f‖s + ‖f‖s− 1

3).

Since T is of order 4, by Theorem 8 from [15], operator XιT is of order 1 for |ι| = 3and of order 0 for |ι| = 4. Therefore

(4.20) ‖XιT P(0)4 f‖s C(‖P (0)

4 f‖s− 13

+ ‖f‖s− 13), |ι| = 3.

Combining estimates (4.19) and (4.20) we get the desired estimates for Xιf|ι|=3.Similarly, one can prove the estimate for Xιf|ι|=4.

Next we prove the extendedly maximal hypoellipticity of P(0)4 in the system

Xjkj=1. Let

f ∈ L2loc(Ω), P (0)

4 f ∈ Sm,Xjk

j=1loc (Ω), b ∈ C∞

0 (Ω).

Take a function a ∈ C∞0 (Ω) such that b a. Using Theorem 4.36 and Theorem 12

from [15] one deduces that

bf = a(bf) = T P(0)4 (bf) + S1(bf) = T aP

(0)4 f

+k∑

j=1

4∑i=1

Ci4T X4−i

j [b, iXj ](bf) + S1(bf) ∈ S1,Xjk

j=1loc (Ω).

In view of the arbitrariness of b we conclude that f ∈ S1,Xjk

j=1loc (Ω). From the

statement just obtained we have

S1(bf) ∈ S2,Xjk

j=1loc (Ω), T aP

(0)4 f ∈ S

m+4,Xjkj=1

loc (Ω),

T X4−ij [b, iXj ](bf) ∈ S

1+i,Xjkj=1

loc (Ω), i = 1, . . . , 4,

from which, in view of arbitrariness of b, it follows that f ∈ S2,Xjk

j=1loc (Ω). Iterating

the procedure m + 2 times, we arrive at the desired result.

We now turn back to the original domain Ω. We can define the extension andrestriction operators E, R as in [15]. To a given operator T mapping functionson Ω into functions on Ω we associate with its restriction on Ω by the formula:T = RTE.

Theorem 4.38. Let the vector fields Xjkj=1 satisfy the condition (H)3 in Ω, and

let a ∈ C∞0 (Ω). Then there exist operators T (parametrix) and S, S′ such that

i) T is smoothing of order 4.ii) S, S′ are smoothing of order 1.iii) P4T = aI + S, TP4 = aI + S′, where by I we denote the unit transform.

Proof. It is clear that T is a parametrix for P4 if and only if T is a parametrixfor P

(0)4 . Hence, it suffices to prove Theorem 4.38 for P

(0)4 . Let ϕ ψ. Put

a(x, t) = a(x)ψ(t). By Lemma 4.35 there exist operators T , S, S′ of type 4 and 1,respectively, such that

(4.21) P(0)4 T = aI + S, and T P

(0)4 = aI + S′.

License or copyright restrictions may apply to redistribution; see http://www.ams.org/journal-terms-of-use

Page 31: Semilinear hypoelliptic differential operators with multiple characteristics

SEMILINEAR HYPOELLIPTIC OPERATORS 3905

Multiply the second equation in (4.21) by R on the left and by E on the right.Obviously, RaIE = aI. Let S′ = RS′E and T = RTE. Taking into account thefact that

Xj1Xj2 · · · XjlE = EXj1Xj2 · · ·Xjl

we deduce that P(0)4 E = EP

(0)4 . Combining these we get TP

(0)4 = aI + S′.

Now we go over to the proof of the equality for P(0)4 T . Note that

P(0)4 T = P

(0)4 RTE = RP

(0)4 TE + [P (0)

4 , R]TE.

However, in view of formula (17.10) in [15]

P(0)4 R − RP

(0)4 =

∑0<|ι|3

R′ιX

ι + R′0.

Hence

P(0)4 T = aI + RSE +

∑0<|ι|3

R′ιXιTE + R′

0TE.

Thus, we can take S = RSE +∑

0<|ι|3 R′ιXιTE + R′

0TE and the property of S

follows from the property of R′ι, R

′0, E. Theorem 4.38 is proved.

Proposition 4.39. Let the vector fields Xjkj=1 satisfy the condition (H)3 in Ω,

and let there be given functions a, b, c ∈ C∞0 (Ω) such that a b c. Then for each

integer r there exist operators Tr and Sr such that

(4.22) TrbP4 = aI + Sr · c,

where Tr is smoothing of order 4, and Sr is smoothing of order r.

Proof. Based on Theorem 4.38 we can prove the theorem for P(0)4 along the lines

of the proof of Theorem 4.36. However, if Proposition 4.39 is true for P(0)4 , then it

is easily seen that it is true for P4. We omit the details.

In the following theorem we use the spaces Lpα,loc, Λα,loc, definitions of which can

be found in the book [16].

Theorem 4.40. Let the vector fields Xjkj=1 satisfy the condition (H)3 in Ω.

Suppose that f ∈ Lploc(Ω), p ∈ (1,∞) and P4f = g. Then

i) If g ∈ Lpα,loc(Ω), α ≥ 0, then f ∈ Lp

α+ 43 ,loc

(Ω).ii) If g ∈ Λα,loc(Ω), α > 0, then f ∈ Λα+ 4

3 ,loc(Ω).iii) If g ∈ L∞(Ω), then f ∈ Λ 4

3 ,loc(Ω).

iv) If g ∈ Sm,Xjk

j=1p,loc (Ω), p ∈ (1,∞), k = 0, 1, . . . , then f ∈ S

m+4,Xjkj=1

p,loc (Ω).

Proof. By applying the adjoint formula of (4.22) the proof then follows the linesof the proof of Theorem 16 in [15] with some obvious modifications. We leave thedetails for the readers. Theorem 4.40 is proved.

Theorem 4.41. Let the vector fields Xjkj=1 satisfy the condition (H)3 in Ω. If the

function Φ(x, τι)|ι|3 is infinitely differentiable, then Ψ4 is a nonlinear hypoelliptic

License or copyright restrictions may apply to redistribution; see http://www.ams.org/journal-terms-of-use

Page 32: Semilinear hypoelliptic differential operators with multiple characteristics

3906 NGUYEN MINH TRI

operator. Moreover, it is extendedly maximally hypoelliptic in the system Xjkj=1

in Ω.

Proof. In view of Theorem 4.33, by the same argument used in the proof of Theorem4.13 one can state that if a function f is a solution of the equation Ψ4(x, ∂α)f =g ∈ C∞(Ω) and f ∈ C3(n+1)(Ω), then f ∈ C∞(Ω).

Example 4.42. The nonlinear operator

∂4

∂x4+

∂4

∂y4+ x4y4 ∂4

∂z4+ e

x2y2∂2

∂z2 ∂

∂x

is hypoelliptic in R3.

Acknowledgments

The author of this paper would like to thank Professor B. Helffer for some valu-able remarks which improved the paper.

References

1. S. Bernstein, Sur la nature analytique des solutions des equations aux derivees partielles dusecond ordre, Math. Ann., 59 (1904), 20-76. MR1511259

2. A. Douglis, L. Nirenberg, Interior estimates for elliptic systems of partial differential equa-tions, Comm. Pure Appl. Math., 8 (1955), 503-538. MR0075417 (17:743b)

3. Yu. V. Egorov, Linear Differential Equations of Principal Type, Moscow, Nauka, 1984, 360.MR776969 (86f:35001)

4. Yu. V. Egorov, N. M. Tri, Maximally hypoelliptic operators with a noninvolutive characteristicset, DAN SSSR., 314 (1990), 1059-1061. MR1116867 (92i:35034)

5. Yu. V. Egorov, N. M. Tri, On a class of maximally hypoelliptic operators, Transactions Sem.Petrovskii, 17 (1994), 3-26. MR1373419

6. G. B. Folland, Applications of analysis on nilpotent groups to partial differential equations,Bull. Amer. Math. Soc., 83 (1977), 912-930. MR0457928 (56:16132)

7. A. Friedman, On classes of solutions of elliptic linear partial differential equations, Proc.Amer. Math. Soc., 8 (1957), 418-427. MR0096894 (20:3376)

8. B. Helffer, J. Nourrigat, Hypoellipticite Maximal pour des Operateurs Polynomes de Champde Vecteur, Birkhauser (1985), 275 p. MR897103 (88i:35029)

9. L. Hormander, The Analysis of Linear Partial Differential Operators, III, Springer-Verlag,Berlin, Heidelberg, 1985. MR781536 (87d:35002a)

10. L. Hormander, Hypoelliptic second-order differential operators, Acta Math., 119 (1967), 147-171. MR0222474 (36:5526)

11. J. J. Kohn, Pseudo-differential operators and hypo-ellipticity, Proc. Symp. Pure Math., 23

(1973), 61-69. MR0338592 (49:3356)12. J. J. Kohn, Hypoellipticity and loss of derivatives, Ann. of Math. (2) 162 (2005), 943–986.

MR2183286 (2006k:35036)13. C. Miranda, Partial Differential Equations of Elliptic Type, Springer-Verlag, 1970, 370.

MR0284700 (44:1924)14. I. G. Petrovskii, Sur l’analyticite des solutions des systemes d’equations differentielles,

Matem. Sbornik (N. S.), 5(47) (1939), 3-70.15. L. P. Rothschild, E. M. Stein, Hypoelliptic differential operators and nilpotent groups, Acta

Math., 137 (1976), 247-320. MR0436223 (55:9171)16. E. M. Stein, Singular Integrals and Differentiability Properties of Functions, Princeton, New

Jersey: Princeton University Press, 1970. MR0290095 (44:7280)17. M. Taylor, Pseudodifferential Operators, Princeton: Princeton University Press, 1981.

MR618463 (82i:35172)18. F. Treves, Introduction to Pseudo-Differential Operators and Fourier Integral Operators,

Plenum Press, 1982. MR597145 (82i:58068)

License or copyright restrictions may apply to redistribution; see http://www.ams.org/journal-terms-of-use

Page 33: Semilinear hypoelliptic differential operators with multiple characteristics

SEMILINEAR HYPOELLIPTIC OPERATORS 3907

19. V. T. T. Hien, N. M. Tri, Analyticity of solutions of semi-linear equations with double char-acteristics, J. Math. Anal. Appl., 337 (2008), 1249–1260.

20. C. J. Xu, Regularity for quasilinear second-order subelliptic equations, Comm. Pure Appl.Math., 45 (1992), 77–96. MR1135924 (93b:35042)

Institute of Mathematics, 18 Hoang Quoc Viet Road, Cau Giay District, 10307 Hanoi,

Vietnam

E-mail address: [email protected]

Current address: Department of Mathematics, University of Chicago, 5734 S. UniversityAvenue, Chicago, Illinois 60637

License or copyright restrictions may apply to redistribution; see http://www.ams.org/journal-terms-of-use