DECOMPRESSION ENGINE

US006326964B1

(12) United States Patent (10) Patent N0.: US 6,326,964 B1 Snyder et al. (45) Date of Patent: Dec. 4, 2001

(54) METHOD FOR SORTING 3D OBJECT 5,630,043 5/1997 Uhlin ................................. .. 395/174 GEOMETRY AMONG IMAGE CHUNKS FOR 5,634,850 6/1997 Kitahara et al. ..................... .. 463/33 RENDERING IN A LAYERED GRAPHICS 5,651,104 7/1997 Cosman ..... .. .. 395/125

5,673,401 9/1997 VOlk 6i al. ......................... .. _ (List continued on next page.)

(75) Inventors: John M. Snyder; C0nal M. Elliott, both of Redmond, WA (US) OTHER PUBLICATIONS

(73) Assignee: Microsoft Corporation, Redmond, WA “A Characterization of Ten RasteriZation Techniques”, by N. (US) Gharachorloo, S. Gupta, R.F. Sproull and IE. Sutherland,

Computer Graphics, vol. 23, No. 3, Jul. 1989, pp. 355—368. ( * ) Notice: Subject to any disclaimer, the term Of this “PiXel—Planes 5: A Heterogeneous Multiprocessor Graphics

patent iS eXIended 0r adjusted under 35 System Using Processor—Enhanced Memories”, by H. U.S.C. 154(1)) by0 days. Fuchs, J. Poulton, J. Eyles, T. Greer, J. Goldfeather, D.

Ellsworth, S. Molnar, G. Turk, B. Tebbs and L. Israel, (21) App1_ No; 09/187,435 Computer Graphics, vol. 23, No. 3, Jul. 1989, pp. 79—88.

_ _ Chapter 18: “Advanced Raster Graphics Architecture”, by S. (22) Flled' NOV‘ 6’ 1998 Molnar and H. Fuchs, Computer Graphics: Principles and

, , Practice, Second Edition, Foley, vanDam, Feiner and Related US. Application Data Hughes, pp‘ 855_923, 1995'

(63) Continuation of application No. 08/671,497, ?led on Jun. Slater} Mel‘ “An Algorithm to SHPPOU 3D lntetr’action on 27, 1996, now Pat. No. 5,864,342, which is a continuation Relatively LOW Performance Graphics Systems, Comput of application No. 08/560,114, ?led on Nov. 17, 1995, now & Graphics, v01, 16, N0, 3, pp, 311—315, 1992, gggncé?relceldéfrfug ionltglglgatglwoibigglléiitéon NO‘ 08/511’ “A Characterization of Ten Hidden—Surface Algorithms”, by

’ 7 i ’ ’ ' I.E. Sutherland, R.F. Sproull and RA. Schumacker, Com

IIlt- Cl- .................................................... .. Surveys, VOL 6, NO_ 1, Mar' (52) US. Cl. ............................................................ .. 345/419 _ _

(58) Field of Search ................................... .. 345/418, 419, (L1St Con?rmed on next Page)

345 420, 442, 441, 427, 428 _ _ / Primary Examiner—Phu K. Nguyen (56) References Cited (74) Attorney, Agent, or Firm—Klarquist Sparkman, LLP

57) ABSTRACT U.S. PATENT DOCUMENTS (

476317690 12/1986 Courtout et at __ 364/518 Apre-processing method prepares 3D objects for rendering 4,645,459 2/1987 Grafetal. 434/43 tolmagelayerslnalayered graphlcsrendenng Pipeline-Th9 5,287,438 2/1994 Kellher _____ __ 395 /132 method transforms a boundmg volume for an ob]ect to a 2D 5,295,235 3/1994 Newman 395/133 bounding boX in a 2D vieW space. It then subdivides the 5,363,475 11/1994 Baker et al. ....................... .. 395/122 bounding box into blocks of image samples called chunks. 5,392,385 2/1995 Evangelist 'et al. ................ .. 395/131 To computer the portion of the Object that Should be ten. 574887684 1/1996 Gharahorhl et a1: - 395/123 dered to each chunk, it determines Which geometric primi

gl?edadet a1‘ tives of the object overlap each chunk and generates a list of 5’586’234 12/1996 sgkziiba 395/13O primitives to the rendered for each chunk. 5,596,686 1/1997 Duluk, Ji. .......................... .. 395/122 5,598,517 1/1997 Watkins ............................. .. 395/141 5 Claims, 42 Drawing Sheets

TEXTURE ADDRE55

GENERAIDR

as:

comma a usmonv coNrRoL

VERTEX INPUT PRocEssoR

SETUP ENGINE

5cm ccNvERT 1“ I ENGINE

DECOMPRESSION ENGINE m2

412

ANYI-ALIASING ENGINE

to 05F, GSPRITE ENGINE. RAMBUS CHANNELS

COMPRESSION

rzxwn: CACHE

FRAGMENT BUFFER(S)

415 I

TEXTURE FILTER ENGINE

PIXEL ENGINE

4111 m

PIXEL BUFFERS

m

US 6,326,964 B1 Page 2

US. PATENT DOCUMENTS

5,684,935 11/1997 Demesa, III et al. ............. .. 395/119 5,684,939 11/1997 Foran et al. .. 5,729,669 3/1998 Appleton

5,729,672 3/1998 Ashton ..... .. 5,864,342 * 1/1999 Kajiya et al. ...................... .. 345/418

OTHER PUBLICATIONS

Bae, et al., ESEU: A HarWare Architecture for Fast Image Generation, Circuitry and Systems, 1990 IEEE International Symposium, pp. 73—76. Adding Parallelism in Object Space to the Rendering Pipe— Line, Eurographics ’89, Proceeding of the European Com puter Graphics Conference and Exhibition, Hamburg, F.R.G., Sep. 4—8, 1989, Amsterdam, pp. 209—220. Slater, Mel, Segments on Bit—mapped Graphics Displays, So?‘ware—Practice and Experience, vol. 16(II), pp. 965—980, Nov. 1986. Chrysanthou, Y and Slater, M., Computing Dynamic Changes to BSP Trees, Computer Graphics Forum, pp. C—321—C342, Sep. 7—11, 1992. Slater, Mel et al., Liberation from Rectangle: A Tiling Method for Dynamic Modi?cation of Objects on Raster Displays, Eurographics 88, Conference date, Sep. 12—16, 1988, pp. 381—392, 1988. Slater, Mel et al., Liberation from Flatland: 3D Interaction Based on the Desktop Bat, Eurographics 91, pp. 209—221, 1991. Deering, Explorations of Display Interfacers for Virtual Reality, IEEE Virtual Reality, 1993 International Sympo sium, pp. 141—147, 1993. Reichlen, Sparcchair: A One Hundred Million Pixel Display, IEEE Virtual Reality, 1993 International Symposium, pp. 300—307, May 28, 1993. Regan, MattheW and Ronald Pose, Priority Rendering With a Virtual Reality Address Recalculation Pipeline, ACM SIGGRAPH 94, Computer Graphics Proceedings, Annual Conference Series, pp. 155—162, 1994. Regan, MattheW and Ronald Pose, LoW Latency Virtual Reality Display System, Technical Report No. 92/166, Monash University, Victoria, Australia, pp. 1—13, Sep., 1992. Regan, MattheW and Ronald Pose, A Interactive Graphics Display Architecture, IEEE Virtual Reality, 1993 Interna tional Symposium, pp. 293—299, 1993. Torborg, Jay, et al., Talisman: Commodity Realtime 3D Graphics for the PC, ACM SIGGRAPH, Conference Pro ceedings, pp. 353—373, Aug. 4—9, 1996. Deering, Michael, et al., Leo: A System for Cost Effective 3D Shaded Graphics, ACM SIGGRAPH, Conference Pro ceedings, pp. 101—108, Aug. 1—6, 1993.

Akerly, Kurt, RealityEngine Graphics, ACM SIGGRAPH, Conference Proceedings, pp. 109—116, Aug. 1—6, 1993. Mcmillan, Leonard, et al., Pienoptic Modeling: An Image— Based Rendering System, ACM SIGGRAPH, Conference Proceedings, pp. 39—46, Aug. 6—11, 1995. Tan, Wee—ChieW, et al., LoW—PoWer polygon Renderer for Computer Graphics, Application Speci?c Array Processors, 1993 International Conference, pp. 200—213, 1993. Bae, Seong—Ok, et al., Patch Rendering: A NeW Parallel HardWare Architecture for Fast Polygon Rendering, Circuits and Systems, 1991 IEEE International Symposium, pp. 3070—3073, 1991. Foley, James D., et al., Computer Graphics: Principles and Practices, Addison—Wesley Publishing Co., 2”“ ed., pp. 806—813, 855—921, 1990. Collaborative Work, Pixel—Planes, Pixel Planes Home Page, url— http://WWW.cs.unc.edu/ , University of North Carolina, pp. 1—25, update Sep. 26, 1995. Oak Technology WARP5 Press Releases, “Oak Technology Sets NeW Standard for 3D Realism With Breakthrough WARP5 Chip Architecture,” Atlanta, GA, Jun. 19, 1997. Bergman et al., “Image Rendering by Adaptive Re?ne ment,” ACM Siggraph 86, vol. 20, No. 4, pp. 29—37, Aug. 18—22, 1986. Chen, Chein—Liang, et al., “A Raster Engine for Computer Graphics and Image Compositing,” Abstract, APCCAS 94, IEEE, pp. 103—108, Dec. 5—8, 1994. Yoo, Terry S., et al., “Direct VisualiZation of Volume Data,” IEEE Computer Graphics and Applications Magazine, vol. 12, No. 4, pp. 63—71, Jul. 1992. Bove, Jr., V. Michael, et al., “Real—Time Decoding and Display of Structured Video,” IEEE Multimedia, 1994 Inter national Conference, pp. 456—462, 1994. Heb, Andreas, et al., Three Dimensional Reconstruction of Brains from 2—Deoxyglucose Serial Section Autoradio graphs, Image Processing, 1994 Internation Conference, vol. 3, pp. 290—294, 1994. Hu, Lincoln, Computer Graphics in Visual Effects, COMP CON Spring 92, IEEE Computer Society International Con ference, pp. 472—474, 1992. Runyon, Kenneth R., Advanced Graphics Processor, Digital Avionics Systems, 1993 Conference, pp. 394—399, 1993. CraW?s, Roger A., et al. A Scienti?c VisualiZation Synthe siZer, Wisualization, 1991 Conference, pp. 262—267, 1991. Haeberli, Paul, et al., The Accumulation Buffer: HardWare Support for High—Quality Rendering, ACM Computer Graphics, vol. 24, No. 4, pp. 309—318, Aug. 1990.

* cited by examiner

U.S. Patent Dec. 4, 2001 Sheet 1 0f 42 US 6,326,964 B1

FIG. 1

100

\ f- 106

IMAGE IMAGE

PREPRocEssoR\ PROCESSOR % 104

< SYSTEM INTERFACE > J 108 110


130

PROCESSOR \ 132 /

134 /

MEMORY CONTROL MAIN MEMORY

13s 146 x

BUS I‘

144 \

IMAGE sgiggADggY INPUT DEVICE(S) PROCESSING

HARDWARE

140 J

142

U.S. Patent Dec. 4, 2001 Sheet

FIG. 3

1 60

/

30f42 US 6,326,964 B1

GRAPHICS SUPPORT SOFTWARE

ii HARDWARE ABSTRACTION LAYER \

i; 162

OTHER IMAGE PRfigggsoR PROCESSING

HARDWARE 164

\ N 166 168


174 FIG. 4A

L--—216 I

214

212

ALPHA BUFFER

BUFFER

\ 210

l

\k COLOR l

2Mx8

GSPRITE ENGINE

k 204

_ _ _ _ _ _ _ _ 1

SHARED MEMORY

2Mx8 RDRAM RDRAM

200

wDm ‘ DSP 0 TILER 4-D

I___


FIG. 4B

/ 462

464 /

RASTERIZER

410

/468 / /466 l I

ANTI-ALIASING ENGINE FRAG. BUFFERS PIXEL ENGINE

472

I PIXEL BUFFERS

U.S. Patent Dec. 4, 2001

FIG. 5A

@m»

Sheet 6 0f 42 US 6,326,964 B1

DETERMINE OBJECT AND VIEWPOINT '\ LOCATIONS 240

V

SELECT POTENTIALLY

VISIBLE \ OBJECTS 242

‘

DETERMINE SPRITE -\

CONFIGURATION 244

ANOTHER SPRITE TO RENDER?

DIVIDE INTO CHUNKS

r 248

ANOTHER CHUNK?

YES v

250

TRANSFORM CHUNK

252

FIG. 58 AT A


FIG. 5B

ANOTHER POLYGON TO TILE IN CURRENT CHUNK?

258 YES RESOLVE PIXELS r Y

256

TILE POLYGON / r

\-- 260 COMPRESS CHUNK

262

STORE CHUNK /

FIG. 5A AT B


DETERMINE 6 GSPRITE DEPTH '—\

ORDER 230

282

ANOTHER GSPRITE TO PROCESS? NO—-—l

WAIT FOR /‘290 YES FRAME SYNC

CALCULATE / 284 SPRITE

TRANSFORMS / 292

J,

BUILD SPRITE DISPLAY LIST

ANOTHER SPRITE TO PROCESS?

YES Y

294 286 TRANSFORM / SPRITE

29s COMPOSITE v /

PIXELS

—| DISPLAY PIXELS

END


FIG. 8

336

DSP

[- 340

llalllll 6

_ _ u

_ _ m

. R

_ _ r _ _ mRm _ _ Iv AER

E D 0 R N F

_ O _ O

_ m _ cm _ S 1 v A

_ n _

_ 7% M

_ _ 3 3

_ _ f

Y

_ _ MU _ _ Iv PM

mm

_l|||._ D

4 4‘

2 R

E 4 E

Y 3

Rm euvu mm 3% EE UT M m P N

M

3 |_ O |

Z CO


m ................................... a .0 T n o

"m 8 R n V L“ \4

.N E8 9 E “ EI ."AE VR 3 VE . XG "CW .HE f NN “ R HN

. I OI “ E E

. C HG NM I" H_E W

Pm /m A ‘m Hm \4 a ‘ x ml mm A. ................................... .1" TE H M91 8

w T Em‘ w LR m

E 3 .600 _ F MW RE \3 D E MU B

Ew\ E H FB WU SE C

TQ DYL EH A M E AOR \APA E T MMT MC R

MEN 0 U % 0M0 C H G

R M T 4 %N .26 > 4 f mm

QT I.“ wRM/ mz6zm m am“ 2 zo?wmmmsooma A

TA% m.’

388

VERTEX CONTROL REGISTERS

VERTEX INPUT PROCESSOR

414

COMPRESSION ENGINE

408


378 FIG 9C 383 ' TILER /

l_ —. I 1-. I .__'TI.___T_—.'LT__.__'.'__. I 7:. I 1'? I | I SETUP 389 II

| ' VERTEX SETUP I I _I> VERTEX INPUT _> CONTROL _> ENGINE AND .I I _ PROCESSOR REGISTERS PRE- II I I RASTERIZER ‘I

I -_ ........ . 2.384 i321. _ 39’

I 393 TEXTURE READ I SA ‘I I E.. . CONVERT

I 5 REGISTERS l

I ggnhérlglég TEXTURE CACHE 5 39/6 {398 -r—> CONTROL 5 ‘I : CONTROL 5 5|

I I ‘ § SCAN CONVERT 5' I I 5 ENGINE 5| l ' 5 5'

: 1| (n I IL- w I ...... .............. .. _l E I z I |.|z.l I o 2 —> 2 g v I 2' ° ° 8 2 | g I E % o TEXTURE FILTER I o _

m I / n- g E ENGINE I :I 416 E I“ D

g I 8 g \ k I l.l.l m | E: Q " 402 403 v | "1' l E I I w I 412 PIXEL z FRAGMENT I ‘III’ I f / BUFFER(S) ‘ ENGINE I I: E | ANTI-ALIAsING \ \ I 8 I ENGINE 40s I -I I

n- l ‘q I 3| PIXEL i I OI BUFFERS I-I I

I I I I I I I I I l

COMPRESSION ENGINE 414


FIG. 10

SETUP ‘ PRIMITIVES RASTER'ZER ‘\

417

416

r 41 a

TEXTURE REFERENCE DATA QUEUE

422 \

MEMORY E TEXTURE ‘ TEXTURE

FETCH CACHE FILTER

419 420 421

FILTERED TEXTURED OUTPUT PIXELS

423


[426 l [427 SETUP PRE- POST

PRIMITIVES RASTERIZER RASTERIZER

425

r 428

TEXTURE FILTERED BLOCK FETCH TEXTURED

QUEUE OUTPUT PIXELS

432

V

TEXTURE TEXTURE MEMORY BLOCK BLOCK

FETCH CACHE

430 429 431

U.S. Patent Dec. 4, 2001

FIG. 12A

GSPRITE ENGINE

Sheet 16 0f 42 US 6,326,964 B1

436

f. 446 [- 442 f 444 / .—-- ----- --+ ————— ——+-——I

: DISPLAY GSPRITE GSPRITE | I CONTROL -> HEADER -> HEADER I I PROCESSOR REGISTER DECODER I

I I “ GSPRITE COMPOSITION SETUP I _ — — — _ _ _ — _ — — _ _ — — _ — _ — _ d

r446 GSPRITE

INTERFACE ‘ GSPRITE

CONTROL ‘ Dggggz?ggs " READ QUEUE

454 \ r _ _ _\T " " ‘V’; m

LLI l 438 448 ' IMAGE I t E g 45° 1 PROCESSOR :: =. B a ADDRESS ' '0'” '- :0 III a w l GENERATOR ' z E a I m E GSPRITE | l E 0 n: O i> 0- ‘9 CACHE I <'’ n: < 5 Z I o u. g 0 Q "J \ I " l n.

0 U 452 I E o g I GSPRITE 8

——l> FILTER J->o \ | ENGINE l I 458 456 __.J/ I

IIMAGE PROCESSOR: |_ _ _ _ _ _ _. ....

: 93 0k

US 6,326,964 B1

480

___/ 484

488 /‘

MEMORY CONTROL

v

Sheet 18 0f 42

FIG. 13

Dec. 4, 2001

COMPOSITING BUFFER

482 \

E 4 ma.“ LEM NFX AF 6% S 1

_

>

w 4

< \ S

m DE“

| 8

ML AM

H3

w m1

A

U.S. Patent

—!\ mzazw why-n50 EOE“

DECOMPRESSION ENGINE

Documents