cmosaic

3D Stacked Architectures with Interlayer Cooling - CMOSAIC

Prof. John R. Thome, LTCM-EPFL, Project Coordinator

Prof. Yusuf Leblebici, LSM-EPFL

Prof. Dimos Poulikakos, LTNT-ETHZ Prof. Wendelin Stark, FML-ETHZ

Prof. David Atienza Alonso, ESL-EPFL

Dr. Bruno Michel, IBM Zürich Research Laboratory

CNN: Computing Electrical Power Consumption

Two-Phase Cooling of 3D Stacked Microprocessors

3 4

Vertical electrical interconnect: Through-Silicon-Via (TSV)

3D integration & thermal management

3D integration opportunities & threats

§ Global wire length reduction § Memory-on-core stacking with vertical electrical

communication → massive core-to-cache bandwidth § Shorter wires & no repeaters → improved power

efficiency § Threats: heat flux accumulation, additional thermal

resistances, peak temperatures

§ Scales with number of dies whereas backside cooling scales only with die area

§ Heat removal: refrigerant two-phase cooling § Two-phase: no electrical insulation, minimal

temperature gradients, automatic hot-spot heat removal BUT dry-out problem, complex system

How to remove heat from a chip stack: interlayer cooling

IBM: Thomas Brunschwiler, Ute Drechsler and Martin Witzig

PhD student: Yassir Madhour

C4 solder bumps as «pin fins» Diameter: 50 to 100µm Pitch: 100 to 200µm  Heat transfer: microchannels / pin fins

Thin film bond: 5 to 10µm

•  Interlayer cooling with evaporated dielectric fluid. •  Solder: « eutectic » 3.5Ag-Sn, low melting temperature: 221°C. •  Designs A & B •  Present project status: package design optimization.

A B

Two-Phase Cooling: Develop 3D Test Vehicles for Stacked Microprocessors

PhD student: Yassir

Madhour

IBM: Thomas Brunschwiler, Ute Drechsler

and Martin Witzig

Wafer-Level TSV Compatible to Liquid Cooling of High Performance CMOS

Ph.D. Students: Michael Zervas, Yuksel Temiz

•  Wafer-level and CMOS compatible TSV process.

•  Very flat surface after TSV fabrication that allows.

further lithographic steps.

•  900 TSVs connected in a single daisy chain.

•  Resistance per TSV 0.7 Ohm.

Daisy-chain interconnections patterned on TSVs.

Die-Level Through-Silicon-Via Fabrication Platform

Ph.D. Students: Yuksel Temiz, Michael Zervas

•  Wafer reconstitution and stencil lithography.

•  Die-level etching and thin-film patterning.

•  Die-level Cu electroplating.

Sten

cil L

ithog

raph

y El

ectr

opla

ting

Fabrication of Two-Phase Cooling Test Chips

Ph.D. Students: Yuksel Temiz (LSM), Sylwia Szczukiewicz (LTCM) •  Front-side metal patterning. •  Front-side DRIE for inlet/outlet

openings. •  Back-side DRIE for microchannels. •  Silicon-Pyrex Anodic Bonding

Fron

t-sid

e

Bac

k-si

de

2D Multi-Microchannel Flow Boiling Experiment

•  A novel in-situ ‘pixel by pixel’ technique has been developed to calibrate the raw infra-red images from IR camera running at 60fps.

•  So far, 752’640 local temperature measurements for one multi-microchannel evaporator with 100x100 micron channels have been recorded.

4

Ph.D.: Sylwia Szczukiewicz – Achievements to date

CCD camera

IR camera

DAQ system

Micro-evaporator

LTCM flow boiling test facility Exploded view of the experimental setup

2D Multi-Microchannel Flow Boiling Experiment

Ph.D.: Sylwia Szczukiewicz – 2D visualisation of two-phase refrigerant flow Multi-microchannel evaporator having 67 channels with the inlet orifices e=2 and 100x100µm ���

cross-section areas, Tsat=31.96oC,  ΔTsub=5.63K,  q=30.69W/cm2  

G=496.1kg/m2,s, slow motion (30fps),

CCD recorded @2000fps, IR recorded @60fps

G=1643.02kg/m2s, slow motion (30fps),

CCD recorded @2000fps, IR recorded @60fps

Flow direction

For the test section having the orifices with the

expansion ratio e=2, the flow tends to stabilize at the relatively high mass fluxes and heat fluxes.

3D ALE-FEM for Microscale Two-Phase Flows

Development:

[1] Comparison of surface representations;

[2] Arbitrary Lagrangian-Eulerian Technique;

[3] Test case: 2D microchannel and 3D

rising bubble.

[4] 3D bubble motion - video

Goals:

•  Develop a 3D Arbitrary Lagrangian-Eulerian Finite Element code;

•  Coupled heat transfer and two-phase flow

•  Predict flows in microscale complex geometries;

•  Design tool for micro evaporators.

Ph.D.: Gustavo Rabello dos Anjos

simulation time

3

2D microchannel

3D rising bubble

grav

ity

velo

city

surfacesurface

standard approach Lagrangian approach

D(ρu)

Dt+∇p =

1

N1/2∇ · [µ(∇u+∇uT )] + ρg +

1

Eof

∂u

∂t+ (u− u) ·∇u

∇ · u = 0

u = u

u = 0

Lagrangian

Eulerian

surface tension

gravity mesh velocity

[1]

[2]

[3]

3D ALE-FEM for Microscale Two Phase Flows

Ph.D.: Gustavo Rabello dos Anjos

3D rising bubble: type: low velocity view: bubble rising,

insertion, flipping and deletion of grid points insertion: top view deletion: bottom view

bottom

top

side

3D ALE-FEM for Microscale Two Phase Flows

Ph.D.: Gustavo Rabello dos Anjos

bottom

top

side

3D rising bubble: type: high velocity view: bubble rising,

insertion, flipping and deletion of grid points insertion: top view deletion: bottom view

Investigation of Integrated Water Cooling of 3D integrated Electronics An experimental study: PhD student Adrian Renfer

Vortex shedding induced flow impingement on micropin fins

à Higher pumping power

Instantaneous µ-Particle Image Velocimetry

Single cavity of a 3D chip stack

Increased pressure drop at high flow rates

Fluctuations are amplified towards the outlet

inlet outlet center

Flow direction

Benefits of enhanced mixing à  High heat transfer

à  Non-uniform micropin fin density for systematic hot spot cooling

Publication: Renfer et al., Experiments in Fluids (2011)

Planned: measure and evaluate à  Heat transfer

à  Vortex shedding frequency

à  Global flow visualization

Performance Evaluation of Cooling Structures for 3D Chip Stacks A computational study: PhD student Fabio Alfieri

Publications: - Alfieri et al., 3D Integrated Water Cooling of a Composite Multilayer Stack of Chips, J. Heat Transfer, 2010 - Alfieri et al., Performance Evaluation of Cooling Structures for 3D chip stacks (to be submitted)

Pin-Fins Inline (PFI) Microchannels (MC) Parallel Plates (PP) Q

P HD

Q Q

HH

P

Transitional regime: vortex shedding

x

Goals: • Evaluate performance of cooling structures • Provide design guidelines for 3D chip stacks

Currently underway: -  Impact on the performance of:

à pin-fins density adjustment à non-homogeneous heat fluxes

-  Modeling of entrance region: ( )Re PrNu A x L λα β=

Boundary layer regeneration

Experimental validation Pressure drop and heat transfer coefficients

Parallel plates

Microchannels

Pin-Fins I) II)

Tran

sitio

n

Superhydrophobic Surfaces

Approach

(1)   Creation of a nanostructure (silicon etching) (2)   Surface functionalization of the created structure (fluorosiloxane)

Needle-like silicon etching

3 4

Ph.D. Student: Michael Rossier

“Needle-like” silicon structures: non-functionalized (left); functionalized with perfluorooctyltriethoxysilane (right)

Goals Production of a highly hydrophobic surface to reduce the pressure drop in microchannel with application for water cooling systems

3D-ICE: a new thermal simulator for 3D ICs with interlayer���liquid cooling– Arvind Sridhar, EPFL-ESL

Future Work: • To model entrance-region effects in microchannels

• Incorporate two-phase flows

•  FIRST-EVER compact modeling based thermal simulator for ICs with microchannel liquid cooling

•  Available as an open source Software Thermal Library at http://esl.epfl.ch/3D-ICE •  More than 35 (and counting!) research groups world-wide are using 3D-ICE

Rconv Rcond

Coolant Flow

Rconv

3D-ICE 1.0 •  based on compact transient thermal modeling (CTTM)

•  975x Faster! than commercial CFD tools even for small problems

3D-ICE 2.0 •  Advanced model for Enhanced Heat Transfer

Geometries (e.g., Pin Fins)

•  40x Faster! than conventional CTTM

3D-ICE optimized as Neural Network based simulator for massively parallel Graphics Processing Units (GPUs) •  It learns from 3D-ICE test simulations •  then works as stand-alone simulator

•  100x Faster! than conventional 3D-ICE model

{ X3D-ICE(tn+1) }

Neural Network -based simulator { XNN(tn+1) }

Training Algorithm

{ U(tn+1) }

{ X(tn) }

For more information on 3D-ICE please visit the poster

Thermal Modeling and Active Cooling Management for 3D MPSoCs – Mohamed M. Sabry, EPFL-ESL

Scheduler Power Manager (DPM) Flow-rate actuator

Transient Temperature Response for Each Unit

• Goals Achieve thermal balance in 3D MPSoC tiers No thermal runaway situations (thermal violations) Minimal performance degradation Minimal energy consumption

• Achievements ü Thermal violations 0% ü Performance degradation 0.01% ü Energy reduction up to 35%

• Proposed technique Design-time run-time management strategy • Design-time Control knobs identification

Electronic-based: Dynamic Voltage and Frequency Scaling

Mechanical-based: Dynamic Varying Flow rate

• Run-time thermal management Fuzzy-logic control

Rule-base look-up table control Low complexity Low computation overhead

MicroCool: Nano-Tera ED Training for 3D-IC’s PI: Prof. J.R. Thome

3D-IC Cooling: Public Announcement by IBM CEO

CMOSAIC: Technological Aims

§  CMOSAIC aims to make an important contribution to the development of the first 3D

computer chip with a functionality per unit volume that nearly parallels the functional density

of a human brain.

§  A 3D computer chip with integrated cooling system is expected to:

- Overcome the limits of air cooling

- Compress ~1012 nanometer sized functional units

(1 Tera) into one cubic centimeter

§ Yield 10-100 fold higher connectivity

- Cut energy and CO2

emissions drastically 0 2 4 6 8

Wire Length (mm)

100

1000

10000

100000

1000000

Wire

Cou

nt

2D (Actual)3D (Projected)