Data Requirements for Valid Backtests June 2010 Marcus C. Bogue III, Ph.D. President, Charter Oak Investment Systems, Inc.

Data Requirements for Valid Backtests

June 2010

Marcus C. Bogue III, Ph.D.

President, Charter Oak Investment Systems, Inc.

Data Requirements

• Data must be Accurate

• Data must reflect “What was Known at the Time”– Must answer the “What did you know and when

did you know it” question

Need Point in Time Data

Data Types and Reporting Characteristics

Market Data- Known immediately to all participants

Expectational Data- Known when published by an analyst

Fundamental Data- Known when reported by the company

FUNDAMENTAL DATA REPORTING CYCLE

• Preliminary Data– Press Release– Newswire– Etc.

• Finalized Data– Company Report

– Regulatory Filing

– Etc.

• Restated Data– Subsequent Company

Report

– Subsequent Regulatory Filing

RESTATEMENTS ALLOW INTER-PERIOD COMPARABILITY

Loews Corporation Total Revenue and Net Incomefor Fiscal Y06 - Y08 Initially Reported and Restated Prior Years(from Thomson Reuters Fundamentals Point in Time Database)

17911 17702

13844

18380

14320

2491 24912075

24891956

4319

13247

0

2000

4000

6000

8000

10000

12000

14000

16000

18000

20000

2/26/2007 Filing 2/27/2008 Filing 2/25/2009 Filing

Total Revenue Y06

Total Revenue Y07

Total Revenue Y08

Net Income Y06

Net Income Y07

Net Income Y08

Companies with Restated Compustat Net Income or Component Data

(based on July 29, 2005 Point in Time database)

0%

10%

20%

30%

40%

50%

60%

70%

80%

Mar

-87

Mar

-88

Mar

-89

Mar

-90

Mar

-91

Mar

-92

Mar

-93

Mar

-94

Mar

-95

Mar

-96

Mar

-97

Mar

-98

Mar

-99

Mar

-00

Mar

-01

Mar

-02

Mar

-03

Mar

-04

S&P 500

Next 1000 Largest by Market Capitalization*

Second 1000 Largest by Market Capitalization*

All Traded Companies**Excludes ADRs, REITs, Mutual Funds and Limited Partnerships

Restatements have yet to occur

% Interim Period Restatements U.S.(based on Thomson Reuters Fundamentals Point in Time Database)

0%

10%

20%

30%

40%

50%

60%

70%

Q1Y

98

Q1Y

99

Q1Y

00

Q1Y

01

Q1Y

02

Q1Y

03

Q1Y

04

Q1Y

05

Q1Y

06

Q1Y

07

Q1Y

08

Q1Y

09

INC Rstmt BAL Rstmt CAS Rstmt Any Rstmt

% Interim Period Restatements N. Europe(based on Thomson Reuters Fundamentals Point in Time Database)

0%

10%

20%

30%

40%

50%

60%

70%

Q1Y

98

Q1Y

99

Q1Y

00

Q1Y

01

Q1Y

02

Q1Y

03

Q1Y

04

Q1Y

05

Q1Y

06

Q1Y

07

Q1Y

08

Q1Y

09

INC Rstmt BAL Rstmt CAS Rstmt Any Rstmt

% Interim Period Restatements E. Asia (based on Thomson Reuters Fundamentals Point in Time Database)

0%

10%

20%

30%

40%

50%

60%

70%

Q1Y

00

Q1Y

01

Q1Y

02

Q1Y

03

Q1Y

04

Q1Y

05

Q1Y

06

Q1Y

07

Q1Y

08

Q1Y

09

INC Rstmt BAL Rstmt CAS Rstmt Any Rstmt

% Interim Period Restatements All Companies (based on Thomson Reuters Fundamentals Point in Time Database)

0%

10%

20%

30%

40%

50%

60%

70%

Q1Y

98

Q1Y

99

Q1Y

00

Q1Y

01

Q1Y

02

Q1Y

03

Q1Y

04

Q1Y

05

Q1Y

06

Q1Y

07

Q1Y

08

Q1Y

09

INC Rstmt BAL Rstmt CAS Rstmt Any Rstmt

% Annual Period Restatements U.S. (based on Thomson Reuters Fundamentals Point in Time Database)

0%

10%

20%

30%

40%

50%

60%

70%

Y98 Y99 Y00 Y01 Y02 Y03 Y04 Y05 Y06 Y07 Y08 Y09

Any Rstmt INC Rstmt BAL Rstmt CAS Rstmt

OVER / UNDER LAGGED(based on Thomson Reuters Fundamentals Point in Time Database)

LAG ASSUMPTIONS IMPACT RESULTS• Backtesting results can significantly differ by reporting lag assumptions

– Data is not Instantaneously available at the end of a reporting period

– No assumed reporting Lag is correct

– Reporting lag is a predictive variable

• Backtest when data was actually available to the market and with the values that were actually known

HIGH / LOW EARNINGS TO PRICE DECILES(based on Thomson Reuters Fundamentals Point in Time Database)

S&P 500 Monthly Rebalanced- 3 Month

0.1

1

10

10th E\P Decile - 3 mths lag 9th E\P Decile - 3 mths lag

2nd E\P Decile - 3 mths lag 1st E\P Decile - 3 mths lag

SP500 10th E\P Decile - PIT

9th E\P Decile - PIT 2nd E\P Decile - PIT

1st E\P Decile - PIT

S&P 500 Monthly Rebalanced- 4 Month

0.1

1

10

10th E\P Decile - 4 mths lag 9th E\P Decile - 4 mths lag

2nd E\P Decile - 4 mths lag 1st E\P Decile - 4 mths lag

SP500 10th E\P Decile - PIT

9th E\P Decile - PIT 2nd E\P Decile - PIT

1st E\P Decile - PIT

Point Data

• When Who Knew It?– The Market or Your Database

• Compustat and Worldscope Point in Time are based on When the Database Knew the Data

• Thomson Reuters Point in Time is based on When the Market Knew the Data

Compustat Average Pointdate from Period End Date

0

20

40

60

80

100

120

140

160

180

Q1Y

90

Q1Y

91

Q1Y

92

Q1Y

93

Q1Y

94

Q1Y

95

Q1Y

96

Q1Y

97

Q1Y

98

Q1Y

99

Q1Y

00

Q1Y

01

Q1Y

02

Q1Y

03

Q1Y

04

Q1Y

05

Q1Y

06

Q1Y

07

Q1Y

08

Q1Y

09

sp400 cavg sp500 cavg sp600 cavg

Reuters Average Pointdate from Period End Date

0

20

40

60

80

100

120

140

160

180

Q1Y

90

Q1Y

91

Q1Y

92

Q1Y

93

Q1Y

94

Q1Y

95

Q1Y

96

Q1Y

97

Q1Y

98

Q1Y

99

Q1Y

00

Q1Y

01

Q1Y

02

Q1Y

03

Q1Y

04

Q1Y

05

Q1Y

06

Q1Y

07

Q1Y

08

Q1Y

09

sp400 cavg sp500 cavg sp600 cavg

Comparison of Average Pointdate from Period End Date

0

20

40

60

80

100

120

140

160

180

Q1Y

90

Q1Y

91

Q1Y

92

Q1Y

93

Q1Y

94

Q1Y

95

Q1Y

96

Q1Y

97

Q1Y

98

Q1Y

99

Q1Y

00

Q1Y

01

Q1Y

02

Q1Y

03

Q1Y

04

Q1Y

05

Q1Y

06

Q1Y

07

Q1Y

08

Q1Y

09

Cmpst sp500 cavg Reuters sp500 cavg

Portfolio Value

Backtesting is an Art, Not a Science!