Millis.marc

1

Managing Revolutionary ResearchMarc G. Millis

2009-Feb-24

Management lessons from NASA's Breakthrough Propulsion Physics Project are presented. This project investigated such notions as gravity controland faster-than-light travel, assessing 10approaches, producing 16 journal articles, an award-winning website, and garnering positive media coverage for NASA – all for a total cost of $1.6 M spread over 1996 to 2002. The key tactic was to combine vision with rigor and to apply the lessons from history regarding scientific andtechnological revolutions.

Management lessons from NASA's Breakthrough Propulsion Physics Project are presented. This project investigated such notions as gravity controland faster-than-light travel, assessing 10approaches, producing 16 journal articles, an award-winning website, and garnering positive media coverage for NASA – all for a total cost of $1.6 M spread over 1996 to 2002. The key tactic was to combine vision with rigor and to apply the lessons from history regarding scientific andtechnological revolutions.

2

Revolutionary ResearchPioneering into Unknown

Evolutionary ResearchMastery of Known

Definition

3

To exceed the limits of prior technologymust seek entirely different technology

(Foster, Innovation - The Attacker’s Advantage, 1986)

Revolutionary ResearchWhen Needed?

Evolutionary

Resources

Performance Revolutionary

Revolutionary

Point of Diminishing Returns

4

Pioneers Masters

Revolutions, then Evolutions

After which thevalue is obvious to general community

Sophisticated

Reputable

Established

Unrefined

Not Yet Understood"Edgy"

Break from prior to search for new

Break from prior to search for new "Breakthrough" Point"Breakthrough" Point

5

Contrasting Attributes

Pioneers• Highest-Gain benefits sought• Expose ignorance - knowledge

gaps• Create new knowledge• Can NOT quantify comparisons• Intuitive progress

Masters• Maintain the knowledge base• Refine established knowledge• Quantify comparisons, “trades”• Procedural progress• Minimum Risk sought

Institutions• Must sustain preeminence & reputation• Must assess risk -vs- benefits• Must stay within budgets• Must produce progress

6

Revolutionary Research Challenges

• Breaks from the Norm ("Out of the Box")– Draws attention to what we don't know rather than

flexing our prowess for the known– Different type work: pioneering rather than mastery– Difficult to assess, quantify benefits

• Risky– Most ideas will fail to perform– Evokes hype, sensationalism, fringe– Evokes pedantic disdain– Difficult to sort viable 'crazy' ideas from the fringe– Success will be disruptive

7

Revolutionary ResearchManagement Challenges

Long-Term Goals Near-Term Progress

Visionary Credible

Divergent Options Need to Focus

Sufficient Investment Affordable

Unfamiliar Understandable

8

History Lessons• Organizational Patterns

Foster, Shepherd, Henderson, Utterback, Miller…

• Scientific RevolutionsKuhn (Paradigms), Dyson (Tool-Driven),

Clarke (3 Laws), Anderson (Horizon), Hamming (Great vs Good Researchers) Emme (Sci-fi/Sci-fact)

• Fringe TaintingLangmuir (Pathological Sci), Park (Voodoo

Sci), Sagan (Baloney filters), Baez (Crackpot Index), Kruger (Unskilled & Unaware)…

CompiledAdvice

9

Organizational Patterns

Resources

Performance

RevolutionaryRevolutionary

Point of Diminishing Returns

Mature = Entrenched

10

Vision Limiting Values • Prior values tailored to Method, not Goals

– Sail Effectiveness for Steam Ships ?– Specific Impulse for Warp Drive ?

• Values tailored to legacy customers, not future– Cold War Prowess -vs- Space Tourism– 1950's Colliers Magazine -vs-

• Affordability• Robotics• Environmental monitoring• Asteroid deflection• International collaboration

• Revolutionary ideas tend to take root outside the incumbent organizations

11

Sir Clarke's 3 LawsSir Arthur C. Clarke, (1972), Profiles of the Future, Bantam

1. When a distinguished but elderly scientist states that something is possible, he is almost certainly right. When he states that something is impossible, he is very probably wrong.

2. The only way of discovering the limits of the possible is to venture a little way past them into the impossible.

3. Any sufficiently advanced technology is indistinguishable from magic.

12

Reflexive Dismissals“Space travel is utter bilge.”

- Dr. Richard van der Riet Wooley (one year before Sputnik 1)

“The secrets of flight will not be mastered within our lifetime, not within a thousand years.”

- Wilbur Wright (two years before Kittyhawk)

“Heavier than air flying machines are impossible,X-rays are a hoax.”

- William Thomson (Lord Kelvin)

But don’t forgetThere were also crazy ideas at the time that were crazy.

13

Reflections on Prior Scientific Revolutions

• COBE & WMAP• Hubble Space Telescope• Superconductivity• Internet networking• Nano Engineering

• Telescope → Galileo• X-ray diffraction → Watson/Crick

Dyson1997 Imagined Worlds

"Tool-Driven"

• Dark Matter• Dark Energy• Quantum Zero Point• Cosmological Const• GR/QM incompatibility• Gravity itself

• Epicycles → Heliocentric (Copernicus)• Gravity & Motion → Newton's laws• Michelson-Morley• Blackbody spectrum• Photoelectric effect• Wave/Particle dual

Kuhn1962 Structure of Scientific Revolutions

"Paradigm Shift"

Relativity(Einstein)

Quantum

14

• Have courage to tackle Important Problems– Grand challenges that will make a real difference, not just "safe" research

– Attackable; there is a way to begin solving the problem

• Start with independent thoughts and then collaborate

• Make steady progress, driven and focused

• Learn things beyond own work; "Knowledge is like compound interest"

• Redirect what is difficult to something easier (convert liabilities to assets)

• Honest with personal flaws & work to overcome

• Tolerate ambiguity

– Believe in self enough to proceed– Doubt self enough to honestly see flaws

Great Researchers & Important Problems1986 lecture, Richard Hamming, distinctions between good and great researchers

15

Via Science Fiction ?Emme, E., ed, (1982) Science Fiction & Space Futures Past & Present, American Astronautical Society History Series

• Science Fiction inspires pioneers - definitely !

• Science Fiction is not an accurate predictor

• Science Fiction akin to is brainstorming

Applying Clarke's 2nd Law:The only way of discovering the limits of the possible is to venture a little way past them into the impossible.

16

Via "Horizon Mission Methodology"Anderson, J. L, , (1996) "Leaps of the Imagination: Interstellar Flight and the Horizon Mission Methodology," JBIS, 49

1. Set Impossible Goals (break from mere extrapolations)

2. Assume it can be done in far future (Sci-fi brainstorming)

3. Look back from future, identifying critical assumptions

4. Identify the critical challenges (contrast knowledge to goals)

5. Identify knowledge gaps ("important questions")

Applying Clarke's 2nd Law:The only way of discovering the limits of the possible is to venture a little way past them into the impossible.

17

Next Challenge: Fringe Tainting

• 30% of project's unsolicited correspondence from amateurs sharing their ideas

• 30% of the amateur correspondence displayed delusion of grandeur and/or paranoia (Fringe)

• Voodoo Science [Park 2000]

• "Among the Fringe," S.Weinberger (2006 June 14), <http://www.defensetech.org/archives/002493.html>

• "Unskilled and Unaware…" [Kruger 1999] Personality and Social Psychology, V.77, p.121-

18

Responding to Amateur / Fringe

• Do not reply to correspondence displaying delusions of grandeur and/or paranoia (Fringe)

• When replying to those with nonviable ideas, use statements like, "appears to violate well-established laws of nature," rather than, "violates physics"

• Give them a next-step task to help them better understand their concept, rather than trying to teach them the principles…

• Make completion of that task a condition for continued correspondence, "to convince us… need to perform…to a higher standard of proof…"

19

Compilation of Advice from History• Identify diminishing returns, specifically where

revolutionary research is needed• Look across multiple disciplines and new 'tools' for

pioneers and possibilities (seeds of next S-curves)

• Employ the "Horizon" method to identify "Important Problems" - converting objections into objectives

• Develop more fundamental selection metrics than those used for prior methods (e.g. energy, not specific impulse)

• Familiarize the decision-makers with scholarly examples of emerging possibilities and how they apply to goals

• Dissect research approaches into short-term next-step research tasks (more affordable, less threatening)

• Promise to produce progress, not breakthroughs

20

Typical Reactionary Approach• New 'hot topic' gains attention.• Funds sought for hot topic only.• Other approaches not comparatively assessed

in a rigorous manner.

Typical Results• Success is defined in terms of whether the

approach worked.• Negative results not published.• In the event of a null result, support ebbs.• Window closes on all other approaches for

addressing these same challenges.

Typical Reactionary Approach• New 'hot topic' gains attention.• Funds sought for hot topic only.• Other approaches not comparatively assessed

in a rigorous manner.

Typical Results• Success is defined in terms of whether the

approach worked.• Negative results not published.• In the event of a null result, support ebbs.• Window closes on all other approaches for

addressing these same challenges.

Reactionary versus Strategic

Strategic Approach• Lead person acts as an impartial broker.• Various relevant approaches solicited and

comparatively evaluated.• Selection criteria concurred with customers and

researchers.• Scope of each task set to the minimal effort

needed to resolve an immediate “go / no-go”question.

Results Sought• Success defined as gaining reliable knowledge

to guide next steps.• Results, pro or con, published to set foundation

for future decisions.• Opportunity open for sequels to the positive

results, and to redirections around null results.

Strategic Approach• Lead person acts as an impartial broker.• Various relevant approaches solicited and

comparatively evaluated.• Selection criteria concurred with customers and

researchers.• Scope of each task set to the minimal effort

needed to resolve an immediate “go / no-go”question.

Results Sought• Success defined as gaining reliable knowledge

to guide next steps.• Results, pro or con, published to set foundation

for future decisions.• Opportunity open for sequels to the positive

results, and to redirections around null results.

21

NASA Breakthrough Propulsion Physics Project

Exceed the fundamental limits of existing propulsion by further advancing physics to discover the breakthroughs that could revolutionize spaceflight and enable interstellar voyages.

ProgrammaticConduct visionary

research in a credible manner.

TechnicalTarget the greatestchallenges of deep-

spaceflight.

22

Project Operating Principles

• Reliable – Define success as gaining reliable knowledge rather than claiming breakthroughs (puts emphasis where needed)

• Immediate – Focus on immediate make-or-break issues, unknowns, or curious effects (just enough for “go/no-go”), not the whole thing

• Iterated – Gain knowledge via cycles of short-term, incremental tasks

• Diversified – Support multiple, divergent research (not just hot topics)

• Measured – Track applicability and progress from cycles of research

• Impartial – Research selected via competitive peer assessments, where reviewers judge reliability of results, not feasibility of concept

• Empirical – Emphasis on experiments over pure theory or studies

• Published – Results published, both pro and con

23

Other "Revolutionary" Efforts

500(space portion)

"Freedom to Fail"Rotate staff ≤ 6yr to avoid entrenchment

DefenseDARPA

3.4"Don't let your preoccupation with reality stifleyour imagination"

Solicitations onlyoutside NASA

Revolutionary space systems

NASA Inst. Adv. Concepts(NIAC)

0.2- Emphasis on reliability, not claims

- Publish null results

- "Impossible" goals- Physics, not Tech- External solicitations

and in-house work

Emerging physics for propulsion

This project

Funding ($M/yr)

Taking Risks Avoiding Incumbent Limitations

ScopeOrg

24

Project Activities / Products• 1996-2002: Small Research Tasks• 2003-2008: Published Findings

– 10 Approaches– 16 Journal Articles– Compiled Book

25

Book: Frontiers of Propulsion Science

• AIAA Progress in Aeronautics and Astronautics Series

• Editors:– Marc Millis (NASA GRC)

– Eric Davis (Inst. Adv. Studies, Austin TX)

• 18 Authors

• 22 Chapters– Gravity control– Faster-than-light– Energy conversion– Project Management

• Publication Date: 2009-Feb-2

26

Visionary Credible

• Define success as gaining reliable knowledge rather than on achieving a breakthrough

• Ask reviewers to judge rigor and credibility, not feasibility (easier to detect a lack of rigor than to assess feasibility)

• Edgy physics pursued in aerospace, aerospace goals into physics• Convert objections into objectives (e.g. conservation of momentum -

to seeking alternate sources of reaction mass)

• Seek credible, published risk-takers

• Proposals contingent to prior peer-reviewed publication

• Submit progress to peer-reviewed journals– Free peer reviews– More credible publication venue

Project Tactics

27

Distinguishing Crackpots from VisionariesFood for thought, or a crackpot filled with half-baked baloney?

• Check for self-criticality– Does the author realize the critical make/break issues?– Is a discriminating test suggested?

• Check for awareness– Must demonstrate understanding of existing approaches– Must show advantage over existing approaches– Check for legitimate citations of compared approaches

• Rigorous on data, playful with interpretations– New idea must be consistent with credible data, but can disagree with

existing interpretations of the data (Copernicus analogy)– Check for legitimate citations of data

• Check track record– Prior publications– Prior products

28

Other Crackpot Filters

• Carl Sagan Baloney Detection Kithttp://www.carlsagan.com/revamp/carlsagan/baloney.html

• John Baez Crackpot Indexhttp://math.ucr.edu/home/baez/crackpot.html

29

Unfamiliar Understandable

• Identify the grand challenges and important problems (Horizon Method)

• Publish, publish, publish

• Include 'executive level' explanations in technical publications

• Be accessible to the Media, teaching, not advocating

Project Tactics, continued

30

Long-Term Goals Near-Term Progress

• Break long range goals into immediate questions– Critical Unknowns– Make-Break Issues– Curious Effects

• Narrow scope of research– Near-term results (1-3yr task duration)– Only address most critical questions, not the whole system

• Less threatening to reflexive challenges• More affordable• More likely to reach completion

• Traceability Map to explain linkage of research to goals and credible foundations

Project Tactics, continued

31

Traceability MapResearch linked to goals and credible foundations

1. Zero Propellant

2. Faster-Than-Light

3. Onboard Energy

Vacuum Battery ?

Goals

GrandChallenges

Concepts& Devices

Curious Effects, Unknowns, & Issues

Space Drives

Warp Drives & Wormholes

Quantum Fluctuations

General Relativity

Conservation Laws

Quantum Mechanics

Cosmology

Anomalous Rotation Rates(“Dark Matter”)

Anomalous Red-shifts(“Dark Energy”)

mpvp = mrvr

Knowledge

FoundationalPhysics

Compare Emerging Knowledge to Goals

32

Tachyon hypothesis

QED -vs- SED

Conservation laws for modifying spacetime

Tajmar apparent frame-dragging

Reactive spacetime

Reactive quantum vacuum

Inertia via quantum vacuum

Indigenous reaction matter

Reactive Machian frame

Degradable quantum vacuum

Causality, retrocausal paradoxes, & the definition

of time

Distinguish causal nonlocality from superluminal or

retrocausal signaling

Heim space theory

Tests of Mach-Lorentz thrusters

Felber gravity-repulsion

Superconductors/ gravitomagnetics

Hathaway tests gravity shield

Yamishita tests

Lifter tests

inertia as SED vacuum drag

Definition of exotic matter & energy conditions

No-signal theorems

Simultaneous linear coherence and entanglement

Polchinski nonlinear EPR FTL phone

Energetic heavy-water reactions

Leplace-Beltrami nonlinear QM in curved

spacetime

4

5

3,5

7

8-9

13

3,11

3,11

3

3,4,12

18

19

14,15,16

16

16

16

16

13,18

Sono-fusion19

1514

16

Electromagnetic -gravitational coupling

3,4,5,7,8,9,11,13

3,5,22

3

4,13

Existence of negative energy

4,15

4,12,18

4

15

Existence of negative matter

3,4

MEMS / NEMS quantum Casimir experiments

Traceability Map from Book (FPS)M

ASS: Thrust w

ithout propellant or beamed energy

SPEED: faster-than-light

ENER

GY

"Important Problems"Rigorous Foundations Goal-Driven Visions

Disciplines & Tools Curiosities Hypotheses & Tests Issues & Unknowns Concepts & Devices Categories

22#'s = Chapter citations

Electromagnetic techniques

MEG device

Sonolum' energy harvesting

Potapov water swirl chamber

Mechanical techniques

Space drive: sails

Space drive: fields

Inertia modification

Negative matterpropulsion

Spacetime modification / gravity control

Quantum approaches to gravity control

Brute fast

Spacetime modification for faster-than-light

Quantum nonlocality for faster-than-light

information

Novel nuclear processes

Quantum vacuum energy conversion

Modify gravitationl or inertial scalars

Yamishita electrogravitics

Forward g-dipoleLevi-Civita effect

Gravitationalwave propulsion

Modify quantum vacuum

Negative energy

Pinto levitation

Heim-Lorentz force

Alzofon antigrav'

Podkletnov gravity shield

Woodward Mach-Lorentz thruster

Oscillators & gyro antigravity

Biefeld-Brown

Corum DxB & Brito "EMIM"

Maclay dynamic Casimir effect

Warp drives & wormholes

Retrocausal communication.

Nachamkin resonant spheres

Koch voltage fluctuation coils

Ground state suppression

Cyclic CasimirShoulders EV energy

tapping

3,12

3

3

3,11

3,4,12,13

3,4

12

10

6

8,9

4,5,7-12

6

3,4

4

14

15

16

18

1919

1818

18

18

18,20

20

20

15

4

4

4

4,5

4,15

44

3,5

7

3,11,13

Forward Casimir battery

16

18

Computational tools & conventions

Quantum theory

Kinematics

Gravitomagneticapproximations

General relativity

Observational cosmology

Electrodynamics

Special relativity

Thermodynamics

Nuclear and particle physics

Anomalous supernovae redshifts

("Dark Energy")

Mach's Principle

Cosmological constant

Superconductivity

electromagnetic momentum in media

Quantum vacuum fluctuations

Cerenkov radiation

Relativisticlifetimes

Quantum entanglement

Sono-luminescence

10

21

3,11

19

16

14

14

3,4,12,13,15,16,18

15,18

3,4

Cosmic microwave background

3

Anomalous intra-galactic gravity

("Dark Matter") 3,4

33

Divergent Options Need to Focus

• Prioritization criteria consistent with goals and an emphasis on the reliability of information

• Select a small suite of options, divergent approaches

• Progress measured relative to applied science advancements (not technology readiness scale)

• Scoring system easily zeroes-out fringe submissions

• While individual tasks are near-term, sustained progress gained from iterating cycles of short-term tasks, adaptingto the results

Project Tactics, continued

34

Research Selection Process (1999)

Selection Criteria concurred by key players (1996-97)

Reviewers do NOT judge feasibility, instead judge:– Project Relevance– Credibility (reliable results upon which to make future decisions.) – Resources

2 - Stage Review Process– Peers numerically grade proposals

Minimum of 4 reviews per proposal (for statistics).Multiplicative, mandatory criteria.

– Customer team reviews scores to select winners

NASA in-house work subject to same review process

35

Project's Research Evaluation FactorsRelevance

1: Gain – Magnitude of performance improvement, assuming the technology ultimately reaches fruition.

2: Empiricism – Tangible effects or just theory?

3: Readiness – The present maturity of the topic/concept under study.

4: Progress – Magnitude of progress to be achieved, as measured by the difference in the readiness now, and the anticipated readiness upon completion of the task.

Credibility5: Foundations – Based on credible references.

6: Contrasts – Compared to current credible competing work.

7: Tests – Leading toward a discriminating test.

8: Results – Probability that the task will result in a reliable foundation for future decisions.

Resources9: Triage – Will it be done anyway or is it unique to this Project?

10: Cost – Funding required (reciprocal scoring factor).

11: Time – Time required to complete task (reciprocal scoring factor).

36

Conjecture Speculation Science Technology Commerce

Measuring Applied Science Progress

Scientific Method Steps4 Hypothesis empirically tested3 Hypothesis proposed2 Data collected1 Problem formulatedØ Pre-science:

• Anomalous effect observed, or• Knowledge gap realized

Ø 1 2 3 4 Ø 1 2 3 4 Ø 1 2 3 4

General Science

Critical Issues

Desired Effects

Technology Readiness Levels8 Flight Qualified7 Prototype demo in relevant environment6 Prototype demo5 Breadboard test in relevant environment4 Breadboard lab test3 Proof of concept2 Application concept formulated1 Basic principles reported

37

Evaluation Equation• Multiplicative criteria (1 failed criteria fails whole proposal)

• Scholastic gradations (A through F) where possible

• Team-generated criteria

Where:– A, B, C represent criteria scores– a, b, c are weighting factors(where 1 is the maximum value, and lower priorities are fractions of 1)– NA, NB, NC are normalizing functions– Cmin is a preset value to prevent the parenthetical term from

equaling zero, thereby making criteria C non-mandatory.

38

Proposal Scores

Standard deviation used to flag disparate scores

39

Traceability Map of Resulting Research

40

Sufficient Investment Affordable

Project Tactics, continued

• Overall progress made from sustaining cycles of short-term tasks and using the findings to affect the next solicitation (can interweave for better continuity)

• Support diversified portfolio of approaches (avoid tendency to only support 'hot' topics)

Solicitation

Research

Assess Findings

Solicitation

Research

Assess Findings

Solicitation

Research

41

Metrics of Project Value

1. Number of visionary notions converted into research tasks

2. Number of incremental unknowns, issues, or curious effects resolved

3. Degree of progress per task using Applied Science Progress Scale

4. Number of findings published in peer-literature

5. Number of citations of published works

6. Number of students inspired (can only count those who send comments)

7. Number of spin-offs

8. Number of educational materials produced

42

Revolutionary Research RisksReliability and Performance of Information, not Hardware

• Threshold of attention is when device can be engineered• Disclose only enough for independent verification of key

principles, not device, not best demo• After independent verification, advertise improved

version whose performance is more pronounced than verification demo

(For commercial research)Competitive advantage weakened from premature disclosure

• Sustain active scouting for ongoing development inside and external to the organization

• Pursue visionary research beyond the known - beyond what other organization address

• Forge widespread collaborations

Leadership stature damaged by neglectingrelevant advancements

• Emphasize reliable advances in knowledge, rather than requiring breakthroughs (the journey, not the destination)

• Collaborate with academia and other institutions for peer reviews

Credibility damaged by non-rigorous reporting

MitigationsRisks

43

Project Lessons LearnedDID WELL• Addressed a diversified set of relevant research approaches• Produced and published genuine research progress• Improved the credibility of topic• Cost-effectively accomplished all of the above

IMPROVEMENT NEEDED • Review opportunities as they emerge• Identify key issues that need the most attention• Link viable sponsors to reliable researchers

FOR NEXT TIME• Equal emphasis on in-house research for coverage of unaddressed

issues, continuity, and sustaining competence• Explore non-traditional techniques to leverage best of academia,

industry, and government – and to inspire and equip future pioneers

44

Summary

1. Combine Vision & Credibility

2. Identify "Important Problems"

3. Use appropriate comparisons(e.g. energy, not specific impulse)

4. Small increments of progress (affordable, less provocative)

5. Emphasis on physical observables

6. Publish reliably, publish often

Consider thepossibilities

Research rigorously & impartially

Evaluate rigor and impartiality – not

feasibility

Marc G. Millis, 2009