Evaluation Overview and Findings From An Interim Evaluation of Grants to Green

1

Evaluation Overview and Findings From An Interim

Evaluation of Grants to Green

Presented by:Highland Communications, LLC

Jennifer Ballentine, MPH

October 29, 2009

2

Purpose of the Evaluation• To monitor implementation and measure

success towards desired outcomes– Increased efficiency– Cost savings– Increased awareness of environmental

sustainability

• Findings will be used for:– Ongoing quality improvement – Sustainability planning– Promoting public will and policy – Informing decision making

3

Evaluation Principles

• We focus on results, use data to make decisions and then act on these decisions

• We value all input from our grantees • We will draw on the wisdom and experience

of our grantees• We will share and disseminate results • We will respect the rights and confidentiality

of grantees • We will use grantees’ time judiciously• We will honor environmental sustainability in

our evaluation approach

4

Monitoring and Evaluation Methods

• Initial Review of Grant Agreement

• Baseline survey

• Interim survey

• Final survey

• End-of-grant period site visit

• Monthly monitoring and review of utility bill data

• Monitoring of web seal placement

5

Web Seal Samples

6

Findings from an Interim Evaluation

• Methods– Online survey– Analysis of utility bill data for Cycle 1

grantees– Monitoring visits with Cycle 1

Implementation Grantees

• Time Frame– Data collected in May-June 2009– Findings presented to Advisory Board in

August 2009

7

Assessment Grantees

• Cycle 1 (September 1, 2008 – September 1, 2009)– 19 applicants– 12 grantees (approx. $4,000 per grantee = $48,000)– 100% grantees completed

• Cycle 2 (March 1, 2009 – March 1, 2010)– 13 applicants– 12 grantees (approx. $4,000 per grantee = $48,000)– 100% grantees completed

• Cycle 3 (July 1, 2009 – July 1, 2010)– 25 applicants– 16 grantees (approx. $4,000 per grantee = $64,000)– In progress

8

Implementation Grantees

• Cycle 1 (September 1, 2008 – September 1, 2010)• 14 applicants• 10 grantees ($342,000 awarded, $304,500 matched*)• 50% of grantees completed

• Cycle 2 (March 1, 2009 – March 1, 2011)• 10 applicants• 9 grantees ($347,126 awarded, $336,068 matched*)• In progress

*Match rate = $1 to $1 for operational budgets $500K+ .50 to $1 for operational budgets $250K-$499K

9

Survey Sample

Type of Survey

Cycle Sample Size

Response Rate

Evaluation (pre and post)

1 20 91%

Baseline 2 21 100%

10

Change in Knowledge

1

1.5

2

2.5

3

3.5

4

4.5

5

Green Building Technology Practices Savings

Before

Mid-Point

N=20

Ave

rage

Lev

el o

f K

now

led

ge1=

Lit

tle

or N

o 5

= A

Lot

Knowledge Indicator

11

Green Practices in Place Before and After Grants to Green

95 95

2530

15

7060

90

60

70

0

15

0102030405060708090

100

Recycling Composting EnergyMonitoring

EnergyConservation

ResourceConservation

Other

Before

Mid-Point

N=20

Per

cen

t

Type of Practice

12

Green Policies in Place Before and After Grants to Green

155

15

65

45

80

35 3530 30

5 5

0

10

20

30

40

50

60

70

80

90

100

None EnergyMonitoring

EnergyConservation

GreenProcurement

AlternativeTransportation

Other

BeforeMid-Point

N=20

Per

cen

t

Type of Policy

13

Number of Communications, Presentations and Panels

18

48

66

18

45

63

36 9

0

1020

3040

5060

70

8090

100

Communications Presentations Panels

Assessment

Implementation

Both

N=20

Nu

mb

er

Type of Effort

14

Utility Data Sample

• 6 Implementation Grantees– Includes 50% who fully completed project– Includes 50% who partially completed project

• 10 Assessment Grantees

15

Implementation Grantees’ 6 Month Actual Cost Savings

149,016

124,879

0

20,000

40,000

60,000

80,000

100,000

120,000

140,000

160,000

Baseline 6 Month Cost Current 6 Month Cost

$

*16% decrease, including 0-2 months of no implementation

An

nu

al U

tili

ties

16

Implementation Grantees’ Annual Projected Cost Savings

1,041,454

831,884

0

200,000

400,000

600,000

800,000

1,000,000

1,200,000

Baseline Consumption Projected AnnualConsumption

$

*20% decrease, projection based on actual savings rate

An

nu

al U

tili

ties

17

Implementation Grantees’ Energy Savings

Energy Cost %6 Month Savings 257,264 $24,474 16%

Annual Projected Savings 646,352 $61,403 20%

-Annual Savings will continue year after year

Energy Saved is the annual equivalent of:

- Lbs of Coal Saved: 430,858

- Lbs of CO2: 963,064

- Cars removed from road: 104 for 1 year (12,000 miles)

18

Woodruff Arts Center-Goal: Annual Savings of $49,395-Award: $40,000 -Six Months of Savings: $26,310-Projected Annual Savings: $51,758 (25%)

19

The Galloway School

-Goal: Annual Savings of $6,538 -Award: $25,000 -Six Months of Savings: $7,879 -Projected Annual Savings: $8,740 (12%)

20

Georgia Citizens Coalition on Hunger

-Goal: Annual Savings of $3,921-Award: $50,000-Six Months of Savings: $2,593-Projected Annual Savings: $3,002 (4%)-Provided AC for community gym

21

The Open Door Community-Goal: Annual Savings of $1,100 -Award: $9,000-Six Months of Savings: $-143-Provided AC for kitchen and dining room serving homeless

22

Assessment Grantees’ Projected Annual Cost Savings

$770,874$711,196

$0

$100,000

$200,000

$300,000

$400,000

$500,000

$600,000

$700,000

$800,000

Baseline Cost Annual Cost to Date

$

*8% decrease, $59,678 projected annual cost savings

An

nu

al U

tili

ties

23

Selected Assessment Grantees’ Cost Savings

Organization Gas Electric Water Total Percent Savings

CARE $7,972(24%)

$19,587(8%)

$3,871(14%)

$31,430 10%

Georgia Justice Project

N/A* $2,346(18%)

N/A* $2,346 18%

Agnes Scott $907(3%)

$10, 295(13%)

N/A* $11,202 10%

Hands on Atlanta N/A* $8,322(11%)

N/A* $8,322 11%

$53,000 Annual Savings *No data, or not applicable

24

Where is Grants to Green Going?

•30 Energy Assessments Completed (includes 6 pilots)

•19 Implementation awards granted

•$503,520 Annual Savings Opportunities Found

•$150k of opportunities pay for themselves in 6 months or less

•16 Assessments in progress

•14-16 Assessments starting in January 2010

•Additional Implementation awards to be granted in 2010

25

Evaluation Limitations

• Survey responses dependent on respondent’s knowledge, understanding of program and recall

• Analysis of energy and cost savings only includes Cycle 1 grantees

• Analysis does not take into account other factors such as staff changes, changes in hours of operation, etc.

• 10 Implementation grantees only eleven months into two-year grant period

26

Conclusions

• Grants to Green has significant impact on:– Knowledge, practices and policies related to

green building– Energy usage and cost

• Savings will grow over the years

• With continued support, Atlanta’s non-profit community can be a leader in the environmental sustainability movement