Ning Fang Don Millard Division of Undergraduate Education National Science Foundation November 10, 2010.

Ning FangNing FangDon MillardDon Millard

Division of Undergraduate EducationDivision of Undergraduate EducationNational Science FoundationNational Science FoundationNovember 10, 2010November 10, 2010

Mock Review Webinar - Mock Review Webinar - 22

IntroductionIntroduction Overview of the review processOverview of the review process Instructions on producing a panel reviewInstructions on producing a panel review Organization of participants into teams and designation of panel chairsOrganization of participants into teams and designation of panel chairs

Local teams discuss strengths and weaknessesLocal teams discuss strengths and weaknesses Teams locally report/discuss resultsTeams locally report/discuss results Reporting to virtual group with ratingsReporting to virtual group with ratings – note: facilitators will be asked to select an individual to report – note: facilitators will be asked to select an individual to report

BREAK (15min)BREAK (15min)

Individuals consider ways to improve the proposalIndividuals consider ways to improve the proposal Local teams discuss suggestions for improvementLocal teams discuss suggestions for improvement Teams locally report/discuss results (5min)Teams locally report/discuss results (5min) Report back to virtual groupReport back to virtual group - note: facilitator picks an individual to report - note: facilitator picks an individual to report PD commentary on responsesPD commentary on responses Think – What have I learned today that I will use in preparing my next proposal?Think – What have I learned today that I will use in preparing my next proposal? Share your thoughts with local participantsShare your thoughts with local participants Facilitator reports results back to virtual groupFacilitator reports results back to virtual group Wrap-up Q&AWrap-up Q&A


Help participants to:Help participants to: Become more familiar with the proposal review Become more familiar with the proposal review

processprocess

Better understand the TUES-specific criteriaBetter understand the TUES-specific criteria

Better understand the use of intellectual merit/broader Better understand the use of intellectual merit/broader impact criteria in reviewing proposalsimpact criteria in reviewing proposals

Develop more competitive proposals that effectively Develop more competitive proposals that effectively meet the expectations of the TUES programmeet the expectations of the TUES program



Title changed to emphasize the special interest in projects that Title changed to emphasize the special interest in projects that have the have the potentialpotential to transform undergraduate STEM education to transform undergraduate STEM education

Review criteria was modified to emphasize the desire for projects that:Review criteria was modified to emphasize the desire for projects that: Propose materials, processes, or models that have the potential to Propose materials, processes, or models that have the potential to

Enhance student learningEnhance student learning Be adapted easily by other sites Be adapted easily by other sites

Involve a significant effort to facilitate adaptation at other sites Involve a significant effort to facilitate adaptation at other sites Institutionalize the approach at the investigator's college or university as appropriate Institutionalize the approach at the investigator's college or university as appropriate

(e.g., for the Type)(e.g., for the Type) Have the potential to contribute to a paradigm shift in undergraduate STEM Have the potential to contribute to a paradigm shift in undergraduate STEM

educationeducation


Vision:Vision: Excellent STEM education for all undergraduate Excellent STEM education for all undergraduate studentsstudents

Reflects national concerns about producing: Skilled STEM professionals (including K-12 teachers) Citizens knowledgeable about STEM and how it relates to

their lives

Seeks to build a community of faculty committed to improving undergraduate STEM education

Encourages projects with potential to transform undergraduate STEM education


Creating Learning Materials and StrategiesCreating Learning Materials and Strategies Guided by research on teaching and learning Guided by research on teaching and learning Incorporate and be inspired by advances within the disciplineIncorporate and be inspired by advances within the discipline

Implementing New Instructional StrategiesImplementing New Instructional Strategies Contribute to understanding on how existing strategies:Contribute to understanding on how existing strategies:

Can be widely adoptedCan be widely adopted Are transferred to diverse settingsAre transferred to diverse settings Impact student learning in diverse settingsImpact student learning in diverse settings

Developing Faculty ExpertiseDeveloping Faculty Expertise Enable faculty to acquire new knowledge and skills in order to revise their curricula Enable faculty to acquire new knowledge and skills in order to revise their curricula

and teaching practicesand teaching practices Involve a diverse group of facultyInvolve a diverse group of faculty


Assessing and Evaluating Student Achievement: Assessing and Evaluating Student Achievement: Develop and disseminate valid and reliable tests of STEM knowledge Develop and disseminate valid and reliable tests of STEM knowledge Collect, synthesize, and interpret information about student understanding, Collect, synthesize, and interpret information about student understanding,

reasoning, practical skills, interests, attitudes or other valued outcomesreasoning, practical skills, interests, attitudes or other valued outcomes

Conducting Research on Undergraduate STEM Education: Conducting Research on Undergraduate STEM Education: Explore how:Explore how:

Effective teaching strategies and curricula enhance learning and attitudesEffective teaching strategies and curricula enhance learning and attitudes Widespread practices have diffused through the communityWidespread practices have diffused through the community Faculty and programs implement changes in their curriculum Faculty and programs implement changes in their curriculum


Projects developing instructional materials and methods Projects developing instructional materials and methods should: should:

Be based on how students learnBe based on how students learn

Consider transferability and dissemination throughout the project's Consider transferability and dissemination throughout the project's lifetimelifetime

Involve efforts to facilitate adaptation at other sites in more advanced Involve efforts to facilitate adaptation at other sites in more advanced projects projects


Expect to award approximately 10%Expect to award approximately 10% Total budget: up to $200,000 for 2 to 3 yearsTotal budget: up to $200,000 for 2 to 3 years

$250,000 when 4-year and 2-year schools $250,000 when 4-year and 2-year schools collaboratecollaborate Typically involve a single institution & one program Typically involve a single institution & one program

component – but there are exceptionscomponent – but there are exceptions Contribute to the understanding of undergraduate STEM Contribute to the understanding of undergraduate STEM

educationeducation Informative evaluation effort based on the project's specific Informative evaluation effort based on the project's specific

expected outcomesexpected outcomes Institutionalized at the participating colleges and universitiesInstitutionalized at the participating colleges and universities Deadlines:Deadlines:

May 26, 2011 (A-M)May 26, 2011 (A-M) May 27, 2011 (N-W)May 27, 2011 (N-W)


Type 2 Projects Type 2 Projects 20 to 25 awards expected 20 to 25 awards expected Total budget: up to $600K for 2 to 4 yearsTotal budget: up to $600K for 2 to 4 years

Type 3 ProjectsType 3 Projects 3 to 5 awards expected3 to 5 awards expected Budget negotiable, but not to exceed $5M over 5 yearsBudget negotiable, but not to exceed $5M over 5 years

Tues Central Resource ProjectsTues Central Resource Projects 1 to 3 awards expected1 to 3 awards expected Budget negotiable, depending on the scope and scale of the activityBudget negotiable, depending on the scope and scale of the activity

Small focused workshop projects -- 1 to 2 years & up to $100KSmall focused workshop projects -- 1 to 2 years & up to $100K Large scale projects -- 3 to 5 years & $300K to $3MLarge scale projects -- 3 to 5 years & $300K to $3M

Deadline:Deadline: January 14, 2011 January 14, 2011


Reviewers are solicited by program directorsReviewers are solicited by program directors 2010 TUES Type 1 Proposals Review:

4 panels/program director (60-70 proposals/PD) 6-8 reviewers/panel 17 proposals/panel – not all read by every panelist Approximately 130 engineering reviewers

Reviewers assign individual ratings and develop a Reviewers assign individual ratings and develop a review 2-3 weeks prior to coming to the panel review 2-3 weeks prior to coming to the panel meetingmeeting

Panel meeting is held in DC area – over a 2 day Panel meeting is held in DC area – over a 2 day periodperiod


Panel Chair (picked by program director ahead of Panel Chair (picked by program director ahead of time) establishes order of proposal review processtime) establishes order of proposal review process

Proposals are discussed individuallyProposals are discussed individually

A “scribe” is designated to capture all of the points A “scribe” is designated to capture all of the points brought up in discussion and produce a summary brought up in discussion and produce a summary review – called the review – called the “Panel Summary”“Panel Summary”

Panel returns on day 2 to review all the proposals’ Panel returns on day 2 to review all the proposals’ panel summariespanel summaries


NSF program directorsNSF program directors

Informs recommendations relative to fundingInforms recommendations relative to funding

Guides pre-award negotiationsGuides pre-award negotiations

ApplicantsApplicants

If proposal is funded:If proposal is funded: Provides suggestions for improving projectProvides suggestions for improving project

If proposal is not funded:If proposal is not funded: Provides information to guide a revision of the proposalProvides information to guide a revision of the proposal


All proposals are evaluated using the NSB-approved All proposals are evaluated using the NSB-approved review criterion review criterion Intellectual merit Intellectual merit Broader impactsBroader impacts

The TUES Solicitation provides two sets of suggested The TUES Solicitation provides two sets of suggested questions to help define these criteriaquestions to help define these criteria Standard NSF set Standard NSF set TUES specific setTUES specific set


Suggested questions are only a guide for considering Suggested questions are only a guide for considering intellectual merit and broader impactsintellectual merit and broader impacts

Suggested questions are NOT:Suggested questions are NOT:

A complete list of “requirements”A complete list of “requirements”

Applicable to every proposalApplicable to every proposal

An official checklistAn official checklist


Will the project:Will the project: Include activities important in advancing knowledge? Include activities important in advancing knowledge? Involve qualified proposer(s)?Involve qualified proposer(s)? Contain creative and original concepts?Contain creative and original concepts? Have a well conceived and organized plan?Have a well conceived and organized plan? Include sufficient access to resources? Include sufficient access to resources?


Will the project:Will the project: Advance discovery - promote teaching & learning?Advance discovery - promote teaching & learning? Broaden participation of underrepresented groups?Broaden participation of underrepresented groups? Enhance the infrastructure?Enhance the infrastructure? Include broad dissemination?Include broad dissemination? Benefit society?Benefit society?


Will the project:Will the project: Produce one or more of the following:Produce one or more of the following:

Exemplary materials, processes, or models that enhance student learning Exemplary materials, processes, or models that enhance student learning

and can be adopted by other sitesand can be adopted by other sites Important findings related to student learning? Important findings related to student learning?

Build on existing knowledge about STEM education? Build on existing knowledge about STEM education? Have explicit and appropriate expected measurable outcomes integrated into an Have explicit and appropriate expected measurable outcomes integrated into an

evaluation plan? evaluation plan? Include an evaluation effort that is likely to produce useful information? Include an evaluation effort that is likely to produce useful information? Institutionalize the approach at the investigator's college or Institutionalize the approach at the investigator's college or

university (as appropriate for the Type)university (as appropriate for the Type)


Will the project:Will the project: Involve a significant effort to Involve a significant effort to facilitate adaptation at other facilitate adaptation at other

sitessites? ? Contribute to the understanding of STEM education? Contribute to the understanding of STEM education? Help build and diversify the STEM education community? Help build and diversify the STEM education community? Have a broad impact on STEM education in an area of Have a broad impact on STEM education in an area of

recognized need or opportunity? recognized need or opportunity? Have the potential to contribute to a paradigm Have the potential to contribute to a paradigm shift in shift in

undergraduate STEM educationundergraduate STEM education? ?



The Entire Proposal is Used to Inform ReviewersThe Entire Proposal is Used to Inform Reviewers Project SummaryProject Summary Project DescriptionProject Description Biographical SketchesBiographical Sketches BudgetBudget Supplementary DocumentationSupplementary Documentation

23


A review should indicate anA review should indicate an opinion on the merit of the project opinion on the merit of the project

The rating should indicate an overall evaluation of the The rating should indicate an overall evaluation of the proposal’s meritproposal’s merit Excellent, Very Good, Good, Fair, and PoorExcellent, Very Good, Good, Fair, and Poor

Text boxes are provided to:Text boxes are provided to: Describe positive aspectsDescribe positive aspects

Not just list them -- Provide detailsNot just list them -- Provide details Identify concerns (or weaknesses)Identify concerns (or weaknesses)

Not just list them -- Provide detailsNot just list them -- Provide details Offer suggestions for improvementOffer suggestions for improvement

Rating and text should be consistentRating and text should be consistent


Describe the strengths and weaknesses (or concerns) relative Describe the strengths and weaknesses (or concerns) relative to the review criteriato the review criteria Include all of the positive aspects, concerns, and issues Include all of the positive aspects, concerns, and issues

Use a comfortable style and format Use a comfortable style and format

Use good grammar, style and complete sentencesUse good grammar, style and complete sentences

A very brief review is A very brief review is not very helpfulnot very helpful


Provide any additional information that you want to transmitProvide any additional information that you want to transmit

One-sentence summary of the proposalOne-sentence summary of the proposal

Summary of the critiqueSummary of the critique

Statement indicating why the proposed project is important or Statement indicating why the proposed project is important or notnot

Suggestions for improvementsSuggestions for improvements

Other comments Other comments


Uses appropriate style Contains adequate details Contains understandable, specific, and complete statements Relates strengths and weakness to review criteria Indicates why an item is a strength or weakness Justifies the proposal rating in the written critique

A reader should be able to guess the rating from the written textA reader should be able to guess the rating from the written text


Identify a strength or weaknessesIdentify a strength or weaknesses ““The evaluation plan is a strength.”The evaluation plan is a strength.”

Identify a strength or weaknesses and indicate Identify a strength or weaknesses and indicate why it is onewhy it is one ““The evaluation plan includes a competent, independent evaluator...” The evaluation plan includes a competent, independent evaluator...”

““The background discussion is well referenced, shows a good The background discussion is well referenced, shows a good understanding of the prior work, supports the proposed work...” understanding of the prior work, supports the proposed work...”


Identify a strength or weaknesses and indicate why it is one and Identify a strength or weaknesses and indicate why it is one and why it is importantwhy it is important ““The evaluation plan includes a competent, independent evaluator, … and it The evaluation plan includes a competent, independent evaluator, … and it

will guide the investigators as the project evolves and provide a measure of will guide the investigators as the project evolves and provide a measure of its effectiveness at the end.”its effectiveness at the end.”

Identify a strength, indicate why it is one, why it is important, and Identify a strength, indicate why it is one, why it is important, and how it could be improvedhow it could be improved ““The evaluation plan is a strength because it includes a competent, The evaluation plan is a strength because it includes a competent,

independent evaluator, … and this will guide the investigators as the project independent evaluator, … and this will guide the investigators as the project evolves and provide a measure of its effectiveness at the end. It could be evolves and provide a measure of its effectiveness at the end. It could be improved by adding …”improved by adding …”


Be constructive in your written commentsBe constructive in your written comments Provide suggestions to help applicants improve their proposalsProvide suggestions to help applicants improve their proposals

Do not be overly critical in your ratingsDo not be overly critical in your ratings Most meritorious proposals (fundable proposals) have some Most meritorious proposals (fundable proposals) have some

weaknessesweaknesses Some are correctable through negotiationsSome are correctable through negotiations


How many strengths did you note? How many strengths did you note? How many weaknesses did you note?How many weaknesses did you note? Are these numbers consistent with your rating?Are these numbers consistent with your rating? Do your statements indicate why an item is a strength or Do your statements indicate why an item is a strength or

weakness?weakness? Do they indicate why a strength or weakness is Do they indicate why a strength or weakness is

important?important?



Local teams discuss the strengths and weaknesses identified Local teams discuss the strengths and weaknesses identified in the individual reviews (20min)in the individual reviews (20min) Team “scribe” takes notes during discussionTeam “scribe” takes notes during discussion

Teams locally report/discuss results with facilitator’s guidance Teams locally report/discuss results with facilitator’s guidance (10min)(10min) Facilitators guide discussion and selection of an individual to report to Facilitators guide discussion and selection of an individual to report to

the full virtual groupthe full virtual group Participating organizations will be called upon by webinar Participating organizations will be called upon by webinar

moderator – moderator – PLEASE BE READY TO REPORT BACKPLEASE BE READY TO REPORT BACK Program Directors will offer comments on reports (15min)Program Directors will offer comments on reports (15min)


Redundancy is OK – indicative of common perceptionRedundancy is OK – indicative of common perception Responses will be solicited as follows:Responses will be solicited as follows:

Intellectual MeritIntellectual Merit StrengthsStrengths WeaknessesWeaknesses

Broader ImpactsBroader Impacts StrengthsStrengths WeaknessesWeaknesses

Overall PerceptionOverall Perception Is this a proposal worthy of funding?Is this a proposal worthy of funding?



Redundancy is OK – indicative of common perceptionRedundancy is OK – indicative of common perception Responses will be solicited as follows:Responses will be solicited as follows:

Intellectual MeritIntellectual Merit StrengthsStrengths WeaknessesWeaknesses

Broader ImpactsBroader Impacts StrengthsStrengths WeaknessesWeaknesses

Overall PerceptionOverall Perception Is this a proposal worthy of funding?Is this a proposal worthy of funding?






Individuals consider ways that the proposal could be improved Individuals consider ways that the proposal could be improved – create a list (5min)– create a list (5min)

Local teams discuss the suggestions for improvement (10min)Local teams discuss the suggestions for improvement (10min) Team “scribe” takes notes during discussionTeam “scribe” takes notes during discussion

Teams locally report/discuss results with facilitator’s guidance Teams locally report/discuss results with facilitator’s guidance (5min)(5min) Facilitators guide discussion and selection of an individual to report to Facilitators guide discussion and selection of an individual to report to

the full virtual groupthe full virtual group Participating organizations will be called upon by webinar Participating organizations will be called upon by webinar

moderator – moderator – PLEASE BE READY TO REPORT BACKPLEASE BE READY TO REPORT BACK Program Directors will offer comments on reports (15min)Program Directors will offer comments on reports (15min)


Try to minimize redundant responsesTry to minimize redundant responses Identify approaches for:Identify approaches for:

Building on strengthsBuilding on strengths Overcoming weaknessesOvercoming weaknesses

Responses should include potential improvements to:Responses should include potential improvements to: IdeaIdea Project infrastructureProject infrastructure Project implementation planProject implementation plan Evaluation planEvaluation plan Dissemination planDissemination plan Proposal participation/involvementProposal participation/involvement



Try to minimize redundant responsesTry to minimize redundant responses Identify approaches for:Identify approaches for:

Building on strengthsBuilding on strengths Overcoming weaknessesOvercoming weaknesses

Responses should include potential improvements to:Responses should include potential improvements to: IdeaIdea Project infrastructureProject infrastructure Project implementation planProject implementation plan Evaluation planEvaluation plan Dissemination planDissemination plan Proposal participation/involvementProposal participation/involvement

Panel Summary and Reviewer CommentsPanel Summary and Reviewer Comments


The proposal is the evolution of the successful studio The proposal is the evolution of the successful studio pedagogy developed at RPI to a mobile studio pedagogypedagogy developed at RPI to a mobile studio pedagogy

In this way the proposal builds on a instructional methodology In this way the proposal builds on a instructional methodology accepted in the STEM literature. accepted in the STEM literature.

The primary innovation is the development of a low cost virtual The primary innovation is the development of a low cost virtual instrument board that, when coupled with a computer/laptop, instrument board that, when coupled with a computer/laptop, provides students with a portable experimental platform. provides students with a portable experimental platform.

The proposal is student focused and grounded in the STEM The proposal is student focused and grounded in the STEM knowledge base. knowledge base.


This experimental platform serves as a low cost replacement This experimental platform serves as a low cost replacement for the laboratory equipment found in the original RPI studio for the laboratory equipment found in the original RPI studio classrooms and provides the students with an "anywhere, classrooms and provides the students with an "anywhere, anytime" experimental platform. The PIs are well qualified to anytime" experimental platform. The PIs are well qualified to carry out the work. carry out the work.

In addition, the proposal was responsive to the cyclic model of In addition, the proposal was responsive to the cyclic model of knowledge creation contained in the CCLI request for knowledge creation contained in the CCLI request for proposals. proposals.

The panel felt the proposal would be strengthened by a more The panel felt the proposal would be strengthened by a more detailed evaluation plan with clearly stated, measurable detailed evaluation plan with clearly stated, measurable outcomes.outcomes.


The broader impacts of the proposed work include the The broader impacts of the proposed work include the potential to significantly impact a large number of under served potential to significantly impact a large number of under served students by providing a low cost, portable experimental students by providing a low cost, portable experimental platform.platform.

In addition, the inclusion of an historically black college or In addition, the inclusion of an historically black college or university and a community college should magnify the impact university and a community college should magnify the impact of the proposed pedagogy. of the proposed pedagogy.

This partnership was made concrete through an instructor This partnership was made concrete through an instructor exchange program between the institutions. exchange program between the institutions.


While the evaluation team appears to have worked together for While the evaluation team appears to have worked together for several years, some basic information about the qualifications of several years, some basic information about the qualifications of those individuals should have been included in the proposal.those individuals should have been included in the proposal.

It is not clear that it will positively impact student learning -- just It is not clear that it will positively impact student learning -- just because a student has the means to perform an experiment 24/7 because a student has the means to perform an experiment 24/7 does not mean that they will. does not mean that they will.

Proposal could benefit from clarifying or identifying the role of Proposal could benefit from clarifying or identifying the role of laboratory experiences where actual industrial-quality laboratory laboratory experiences where actual industrial-quality laboratory equipment is utilized, as too much reliance on the proposed equipment is utilized, as too much reliance on the proposed pedagogy might also leave the student with limited experience in pedagogy might also leave the student with limited experience in the use of the real industrial grade and scale laboratory equipment.the use of the real industrial grade and scale laboratory equipment.

The proposal could be strengthened with more definition of the role The proposal could be strengthened with more definition of the role of each partner.of each partner.


Moving from the original studio to the mobile studio essentially Moving from the original studio to the mobile studio essentially means that the new work is now based on an untested model. means that the new work is now based on an untested model. The proposal could be strengthened by addressing this issue.The proposal could be strengthened by addressing this issue.

It has not been demonstrated that the mobile studio is as It has not been demonstrated that the mobile studio is as effective as the original studio with its significant faculty effective as the original studio with its significant faculty involvement and structure. involvement and structure.

This proposal would have been much stronger as a Phase I or This proposal would have been much stronger as a Phase I or if some preliminary data on efficacy were given.if some preliminary data on efficacy were given.

The proposal mentions a test that was done at Howard The proposal mentions a test that was done at Howard University with this work but no details are given.University with this work but no details are given.

It fails to explicitly address how it will focus on components of It fails to explicitly address how it will focus on components of the cyclic model for innovation in STEM education.the cyclic model for innovation in STEM education.



Reviewers have:Reviewers have: Many proposalsMany proposals

Ten or more from several areasTen or more from several areas Limited timeLimited time for your proposal for your proposal

20 minutes for first read20 minutes for first read Different Different experiencesexperiences in review process in review process

Veterans to novicesVeterans to novices Different Different levels of knowledgelevels of knowledge in proposal area in proposal area

Experts to outsiders Experts to outsiders DiscussionsDiscussions of proposals’ merits at panel meeting of proposals’ merits at panel meeting

Share expertise and experienceShare expertise and experience

52


Write down a list of suggestions (guidelines) that you Write down a list of suggestions (guidelines) that you would suggest that a colleague should follow - to deal would suggest that a colleague should follow - to deal with practical aspects of the Review Processwith practical aspects of the Review Process

2 minutes2 minutes

53


Use good style (clarity, organization, etc.)Use good style (clarity, organization, etc.) Be concise, but completeBe concise, but complete Write simply but professionallyWrite simply but professionally Avoid jargon and acronymsAvoid jargon and acronyms Check grammar and spellingCheck grammar and spelling Use sections, headings, short paragraphs & bullets (Avoid dense, Use sections, headings, short paragraphs & bullets (Avoid dense,

compact text)compact text)

Reinforce your ideasReinforce your ideas Summarize; Highlight (bolding, italics)Summarize; Highlight (bolding, italics)

Give examplesGive examples

54


Use tables, figures – where it makes senseUse tables, figures – where it makes sense

Reinforce your ideasReinforce your ideas Summarize; Highlight (bolding, italics)Summarize; Highlight (bolding, italics)

Give examplesGive examples

55


Provide appropriate level of detailProvide appropriate level of detail

Pay special attention to Project Summary Pay special attention to Project Summary

Summarize goals, rationale, methods, and evaluation and dissemination plans Summarize goals, rationale, methods, and evaluation and dissemination plans

Address intellectual merit and broader impacts Address intellectual merit and broader impacts Explicitly and independentlyExplicitly and independently

Three paragraphs with headings:Three paragraphs with headings: ““Summary” Summary” ““Intellectual Merit”Intellectual Merit” ““Broader Impacts”Broader Impacts”

56


Follow the solicitation and GPGFollow the solicitation and GPG Adhere to page, font size, and margin limitationsAdhere to page, font size, and margin limitations

Use allotted space but don’t pad the proposalUse allotted space but don’t pad the proposal

Follow suggested (or implied) organizationFollow suggested (or implied) organization

Use appendices sparingly (check solicitation to see if allowed)Use appendices sparingly (check solicitation to see if allowed)

Include letters showing commitments from othersInclude letters showing commitments from others ““Support letters” are not allowedSupport letters” are not allowed Avoid form lettersAvoid form letters

57


Prepare credible budget Prepare credible budget Consistent with the scope of project Consistent with the scope of project Clearly explain and justify each item Clearly explain and justify each item

Address prior funding when appropriateAddress prior funding when appropriate Emphasize resultsEmphasize results

Sell your ideas but don’t over promoteSell your ideas but don’t over promote

Proofread the proposalProofread the proposal

““Tell a story” and turn a good idea into a competitive proposalTell a story” and turn a good idea into a competitive proposal

58


Good idea + needGood idea + need Right people + infrastructureRight people + infrastructure Assessment of outcomes that measure effect on Assessment of outcomes that measure effect on

student learning (with goals/objectives linked to student learning (with goals/objectives linked to evaluation)evaluation)

Active dissemination planActive dissemination plan Efforts to broaden participation of underrepresented Efforts to broaden participation of underrepresented

groupsgroups



Individuals consider: Individuals consider: What have I learned today that I will be able to used in What have I learned today that I will be able to used in preparing my next proposal? preparing my next proposal?

Create a list (5min)Create a list (5min) Share your list with local participants (5min)Share your list with local participants (5min) Facilitators report back to virtual group (5min)Facilitators report back to virtual group (5min)

Participating organizations will be called upon by webinar Participating organizations will be called upon by webinar moderators – moderators – PLEASE BE READY TO REPORT BACKPLEASE BE READY TO REPORT BACK



For an impact on undergraduate education to be For an impact on undergraduate education to be transformative, it needs to provoke transformative, it needs to provoke a shift in learninga shift in learning


Model good practices that increase/improve learningModel good practices that increase/improve learning Facilitate direct experience with the methods and processes of Facilitate direct experience with the methods and processes of

inquiry/experimentationinquiry/experimentation Empower the studentEmpower the student Foster a sense of wonder, the excitement of discovery and Foster a sense of wonder, the excitement of discovery and

“out-of-the-box” thinking“out-of-the-box” thinking Enhance ability to produce innovative resultsEnhance ability to produce innovative results

ProductsProducts Solutions to problemsSolutions to problems


Good idea + needGood idea + need Right people + infrastructureRight people + infrastructure Can be readily adopted at other sitesCan be readily adopted at other sites Assessment of outcomes that measure effect on Assessment of outcomes that measure effect on

student learning (with goals/objectives linked to student learning (with goals/objectives linked to evaluation)evaluation)

Active dissemination planActive dissemination plan Shows promise for institutionalizationShows promise for institutionalization Efforts to broaden participation of underrepresented Efforts to broaden participation of underrepresented

groupsgroups


Ning Fang – Ning Fang – [email protected]@nsf.govDon Millard – Don Millard – [email protected]@nsf.gov

Ning Fang Don Millard Division of Undergraduate Education National Science Foundation November 10, 2010.

Documents

qamock review webinar

programmock review webinar

review processinstructions

excellent stem education

proposal review processbetter

skilled stem professionals

student understanding

learning materials