Michel A. Wattiaux [email protected] Department of Dairy Science, UW-Madison Getting Scholarly Teaching Projects Published 2010 ADSA-PSA-AMPA-CSAS-WSASAS-ASAS,

Michel A. Wattiaux [email protected]

Department of Dairy Science, UW-Madison

Getting Scholarly Teaching Projects Published

2010 ADSA-PSA-AMPA-CSAS-WSASAS-ASAS, Denver, CO ,July 13, 2010

Teaching Symposium

“Surviving Promotion and Tenure with a Teaching Appointment”

Outline• Introduction: What it feels like to have a high teaching

responsibility in a Research / Doctoral University.

• Excellence in Teaching vs. Scholarship of Teaching and Learning (SoTL).

• Example of a “scholarly approach” to teaching and learning.

• Where and what to publish? (Not all journals are created equal)

• Types (Orientations) of SoTL

• What do faculty say about teaching-related authorship?

• Concluding thoughts

Excellence in Teaching : How Does it Feel?Le

vel o

fE

xpec

tatio

n

Primary area of Excellence Research

Research Instruction Research Instruction

Primary area of “Excellence”Instruction

Documentation of “Excellence” in Teaching in a Tenure Package at the UW-Madison.

• Student evaluations;

• Mentor teaching evaluations;

• Peer-review of teaching (mandatory);

• Invited presentations on teaching, especially outside the institution;

• Outside, arm’s-length evaluations by peers with nationally-recognized expertise in teaching… asked to comment on the broader impact of the candidate’s teaching scholarship;

• Course materials (syllabus, reading, etc.).

NRC. 2010. pp 62-63 In: Transforming Agricultural Education For a Changing World. Nt’l. Ac. Press.

In addition to teaching well and often, candidates for tenure based on excellence in teaching must also demonstrate significant peer-reviewed scholarly contributions to teaching, usually research and publications on teaching. If grants support this research, grant panel evaluations and comments should also be included in the package.









Scholarship of Teaching and Learning Educational Scientists vs. Faculty’s perspectives

Item

Kreber C. 2002. Higher Education 46:93-121Kreber C. 2002. Higher Education 46:93-121

PEd. Sci. Faculty

1: On a scale of 1= Strongly disagree to 7 = Strongly agree.

Mean Score1

24. Every faculty member that teaches is a scholarof teaching ………………..………………………. <.011.0 1.5

Excellence vs. Scholarship

Kreber, C. 2002. Innovative Higher Education 27:5-23

Excellence Scholarship

1. What are the sources of information relied upon as “building blocks” of pedagogical knowledge?

Own experience(trial and error).

Literature,“publications,”conferences, etc.

2. What is the focus of the instructor’s reflection?

What “works” / does not work in the class as a whole.

Address a specific issue in their own teaching.




3. Who do the instructor communicate their teaching and learning insights to?

No one, there are no dissemination of insights

Teaching is a “private business.”

Insights are being shared in the form of publications Teaching is a

“public business.”

4. Who are the beneficiaries of the instructor’s knowledge in teaching and learning?

a) Studentsb) One-selfc) Limited circle of colleagues.

a) Studentsb) One-selfc) Broad circle

of colleagues.




Great course evaluations;

Recognize past “mistakes”;

“Knows” what works to help students learn their topics;

Great course evaluations;

Looks for new insights;

Collect data systematically to assess student learning;

Volunteered posters, abstracts, or talks, on multiple campuses and at meetings of professional organizations (frequently).

“Formal” (anonymous) peer-reviewed publications.

Scholarship of Teaching and Learning Educational Scientists vs. Faculty’s perspectives

Item

18. The scholarship of teaching is an activity that, in the context of promoting student learning, meets each of the following criteria:• It requires high levels of disciplinary expertise,• It breaks new ground and is innovative,• Can be replicated and elaborated,• Can be peer-reviewed,• Has significant impact…………………………… <.016.1 5.3

Kreber C. 2002. Higher Education 46:93-121Kreber C. 2002. Higher Education 46:93-121

PEd. Sci. Faculty

1: On a scale of 1= Strongly disagree to 7 = Strongly agree.

Mean Score1

You don’t have to be an “educational scientist”, but you have to collect “data”

Modified from Paulsen, M. B. 2001. New Directions for Teaching and Learning 86:19-29

Reflective Practices

CourseEvaluations

FacultyDevelopment

Pedagogical Content Knowledge

TraditionalEducational

Research

PedagogicalKnowledge

TraditionalDisciplinaryResearch

Content Knowledge

CAT; Data Collection

“Publications”Student learning









Example: Ruminant Nutrition(Dy Sci. & An. Sci 414)

• Third and last nutrition course taken by juniors, seniors and a few graduate students;

• Learning objective states that “Students will gain practical knowledge and critical thinking skills in evaluating dairy rations.”

• Project 1 = Commercial Product Evaluation

• Project 2 = “Real-world” Dairy Ration Evaluation.

Class Schedule

Example: “Project 1” in Ruminant Nutrition(Dy Sci. & An. Sci 414)

To provide students with an opportunity to:

• Evaluate claims made by commercial feed companies;

• Review literature on a topic of their own choice/interest;

• Practice oral and written communication skills.

Goal of Project 1:

To what degree would students:

• Believe the activity would help their learning?

• Commit to this learning opportunity?

• Think that the in-class presentations was a good use of class time?

Questions / Issues for successful implementation:


Materials and Methods: “Resource materials for students”


Materials and Methods: A project template

1: Product Name/Description (2 pts)

2: Recommended Usage (2 pts)

3: Marketing Claims (2 pts)

4: Financial benefit to cost ratio (3 pts)

5: Score Card—What are the facts? (6 pts)

6: Recommendations (3 pts)

7: Citations (2 pts)


Materials and Methods: Students attend World Dairy Expo (in person or on the internet).


Materials and Methods: Oral presentations in class

Silage Inoculants

New-born calf products

Animal Health

Feed Ingredients

additives


Materials and Methods: A 15 min. project evaluation was conducted in-class immediately after the last presentation.

Class, Date, etc.

Survey (anonymous ID), Class standing.

Quantitative data.

Qualitative data.


Not at all

A Little

Somewhat A lot

A great deal

1. To what degree did this assignment help you learn? .. 1 2 3 4 5 6 7 8 9 10

2. To what degree did you commit to this educational opportunity? …………………………………………… 1 2 3 4 5 6 7 8 9 10

3. In-class student presentations was a good use of classroom time ……………………………………… 1 2 3 4 5 6 7 8 9 10

4. This project should be expanded to provide students with the opportunity to explore their topics in more depth (learning from independent literature sources) …………. 1 2 3 4 5 6 7 8 9 10

5. This project should be expanded to provide students with the opportunity to work in teams 1 2 3 4 5 6 7 8 9 10

6. This assignment should be repeated in future class 1 2 3 4 5 6 7 8 9 10

7. Would it be a good idea to have you write a 2-3 page summary paper on a closely related topic as a take-home exam for this class? 1 2 3 4 5 6 7 8 9 10

Materials and Methods: Evaluation Instrument — Quantitative Data


Materials and Methods: Evaluation Instrument — Qualitative Data

8. What did you like the most about this assignment? More specifically, what part of the assignment was the most useful to you and why?

9. What change(s) would you suggest to improve the quality of this assignment for future classes?


Results: Quantitative (selected) Data

Survey) Item 1 Item 2 Item 3 Item 4 Item 5 Item 6 Item 7 AVG

2 8 6 8 10 9 10 10 8.7

4 8 7 6 1 1 8 1 4.6

5 8 10 8 9 9 9 9 8.9

9 9 10 10 9 5 10 9 8.9

19 5 10 7 1 1 5 1 4.3

20 7 7 7 3 3 7 2 5.1

23 8 10 7 8 7 10 10 8.6

. . . . . . . . .

Average 7.4 7.7 6.8 5.4 5.1 8.7 5.7

Agreement A lot A lot Somewhat Somewhat Somewhat A lot Somewhat

Stdev. 1.5 1.6 1.5 2.8 2.9 1.6 3.2

Consensus High High High Low Low High Very low


Results— Qualitative Data

8. What did you like the most about this assignment? More specifically, what part of the assignment was the most useful to you and why?

• Variety of products and need to investigate claims thoroughly

• I like the independent research. I think It is a practical project, because this is what farmers should be doing before using a product;

• It was fun to analyze a product, but I think that this took too much class time. Perhaps…

• That I was able to work around world Dairy Expo and talk to a lot of salesman and pick the right product that I wanted to do that was the most interesting to me.

• Etc…


Results— Qualitative Data

9. What change(s) would you suggest to improve the quality of this assignment for future classes?

• None, maybe in teams would be a little more interesting;

• Work in groups;

• Give us more time to work on it. Most of us are Dairy Science Majors and have a lot of responsibility during WDE;

• Stricter page limits; I did not go over the front/back and felt at a disadvantage [relative to] those who had 4-5 pages;

• I thought that it was pretty good just the way It is;

• Etc…









Where To Publish?

• Question 1: Would this project be accepted for publication in Journal of Animal Science or Journal of Dairy Science? Please explain.

• Question 2: Could you list at least two Journals that would accept such “research” for publication?

• Question 3: In which Journal does this “research” best fit for publication and why?

Instructions:1)Introduce yourself to a person seated near you and discuss the following questions assuming that “project 1” described above was repeated in 2 or 3 consecutive years2)Be prepared to share your views with the audience (in 2 min.)

LECTURE BREAK: ACTIVITY

Peer-Review Publications on Teaching and Learning (1985-2010 Agricola Database)

Source

JDS, JAS, and PS.

AJ, CS, and SSA

Education

NACTA

JNRLSE

E

E= Entries in the Journal

39,502

22,251

763

372

220

A= “Undergraduate” or “Higher Education”

E+A

39

5

84

E+A+T

12

3

124

30

T= “Teaching”

E+A+L

12

0

39

34

L= “Learning”

E+A+S

S= Communication skill*, or leadership skill* or critical think*,or decision-mak* or Problem-solv*

10

0

19

15

Compilation of SoTL Publication Outlets

Journal of Natural Resources & Life Sciences Education

North American Colleges and Teachers of Agriculture

The Journal of Agricultural Education and Extension

The Journal on Excellence in College Teaching

International Journal for the Scholarship of Teaching and Learning (IJ-SoTL)

International Journal of Teaching and Learning in Higher Education (IJTLHE)

Innovative Higher Education

The Teaching Professor









“Type” or “Orientation” of Publishable SoTL Projects

“Quasi-Objective”Quantitative

Predictive

“Interpretive”QualitativeDescriptive

How is human behavior viewed?

- “Behaviorist”;- Natural forces shape behavior and human responds to extrinsic motivations (stimuli / response);- I assume I will find lots of similarities

-“Constructivist”- Human make conscious choices and responds to intrinsic motivations(each student is unique);- I assume I will find lots of differences.



Predictive





What goals are we striving for?

- Explain and Predict;- Delineate “cause and effect” relations;- Create “generalizable” Knowledge.

- Understand and interpret meaning;- Reveal complexities (interactions);- Create “deep” insights.



Predictive








What research methods are used?

- Follows an experimental design; - Pre vs. post tests; Survey tools;- Quantitative (statistical) analysis;

-No design per se;- Interviews, focus groups;- Qualitative interpretations.



Predictive








What research methods are used?

- Follows an experimental design; - Pre vs. post tests; Survey tools;- Quantitative (statistical) analysis;

-No design per se;- Interviews, focus groups;- Qualitative interpretations.

Strengths and limitations

- Hypothesis testing as a way to describe what “works”- Useful and practical - Detached and overly “objective”

- Rhetorical new insights in describing “what is”- No expectation to provide answers.- Difficult if not impossible to validate.









Excellence in Teaching for Promotion and Tenure in Animal and Dairy Sciences at

Doctoral/Research University: A Faculty Perspective

Wattiaux et al. Journal of Dairy Science. 2010. 93: 3365-3376.

Criteria considered for the Purpose of Promotion and Tenure on the Basis of Excellence in Teaching

21. Students’ evaluation of instructor

22. Students’ evaluation of courses

23. Peer-review (evaluation) of the instructor

24. Peer-review (evaluation) of the course

25. Establishing new courses for curriculum improvement

Item

This is currently considered at my institution (check all that apply)

This should be considered at my institution (check all that apply)

26. Providing students with “course packages”

“Is” “Should”

27. Documenting personal assessment of one’s own teaching (portfolio)

Criteria considered for the Purpose of Promotion and Tenure on the Basis of Excellence in Teaching (cont’d)

30. Authoring peer-reviewed publications

31. Authoring undergraduate textbook or book chapter

32. Obtaining funding for teaching-related projects

33. Organizing / facilitating teaching-related workshops

34. Being recognized for quality of student advising

Item “Is” “Should”

35. Being a member of an editorial board to review teaching manuscripts

28. Presenting abstract at teaching conferences

29. Invited presentation on teaching at conferences

Criteria Currently “Over-emphasized”

Response combination, % Overall, %

Evaluation Criteria(Survey items)

1RRT

IsSd

Is-notSd-not

Is-not Sd

IsSd-not

P value Is Sd

Sd -is

21: Student evaluation of the Instructor

RRT

4864

00

40

4836

<0.01<0.01

96100

5264

-44-36

22: Student evaluation of the course

RRT

4864

00

40

4836

<0.01<0.01

96100

5264

-44-36

30: Authoring peer-reviewed publications

RRT

4041

814

418

4827

<0.010.75

8868

4459

-44-9

31: Authoring undergraduate textbook or book chapter

RRT

4055

1214

414

4418

<0.010.99

8873

4469

-44-4

25: Creating new courses for curriculum development

RRT

4018

827

841

4414

0.02 0.15

8432

4859

-36 27

32: Obtaining funding for teaching-related projects

RRT

4450

8 5

827

4018

0.03 0.75

8468

5277

-32 9

1R = Those who believe that research is more important than teaching at their institution (n=25); RT = Those who believe that teaching is equally important or more important than research at their institution (total n=22);

Criteria for which No Difference was Detected



RRT

IsSd

Is-notSd-not

Is-not Sd

IsSd-not

P value Is Sd

Sd -is

35: Being on editorial board to review manuscripts

RRT

3632

1223

1232

4014

0.09 0.34

7646

4864

-28 18

29: Invited presentation on teaching at conferences

RRT

4027

1223

1632

3218

0.39 0.55

7245

5659

-16 14

28: Presenting an abstract at teaching conferences

RRT

3627

2023

2027

2423

0.99 0.99

6050

5654

-4 4

23: Peer-review (evaluation) of the instructor

RRT

3636

414

2827

3223

0.99 0.99

6859

6463

-4 4

33: Organizing facilitating teaching workshops

RRT

4427

2036

1227

249

0.51 0.27

6836

5654

-1218

24: Peer-review (evaluation) of the course

RRT

2023

814

4041

3223

0.82 0.42

5246

6064

-818

34: Recognition for quality of student advising

RRT

3241

49

3636

2814

0.800.23

6055

6877

-8-22

Criteria Currently “Under-emphasized”



1RRT

IsSd

Is-notSd-not

Is-not Sd

IsSd-not

P value Is Sd

Sd -is

26: Providing students with “course packages”

RRT

2418

2441

2436

28 5

0.99 0.04

5223

4854

-431

27: Documenting personal assessment of one’s own teaching (portfolio)

RRT

2018

2818

2459

28 5

0.99<0.01

4823

4477

-454

1R = Those who believe that research is more important than teaching at their institution (n=25); RT = Those who believe that teaching is equally important or more important than research at their institution (total n=22);









Excellence vs. Scholarship and Publications


Extent of the review process & public access

Higher review process and public access makes for higher level of “performance” of the instructor as a “Scholar”

1. Printed course packages

2. Teaching Abstracts

3. Peer-reviewed manuscript

4. Text book

5. Portfolio

Types of “Publication”

5

1

2

43

Concluding Thoughts

• Concerns about the scope of inference (generalizability) should NOT stop you from collecting the data!

• It’s the right thing to do !

• Good scholarly research in teaching creates powerful stories that resonate with people (my story is not “THE” truth, it is “A” truth; Anonymous).

• We need more teaching publications:

• Teachers of science accept in education what is done traditionally, although they experiment continually in their own fields in the hope of contributing to change (Ruth 1935).

Concluding Thoughts:A Plea for Quality Teaching-related Publications

• Taylor and Kaufman (1983) published a 75-yr review of the teaching literature in (Journal of) Animal Sciences. They concluded:

• … that educators of the 1980’s were “reinventing the wheel” (p 172) as what was considered new in teaching and learning had been registered in the literature more than five decades earlier.

• … the “fine screen of reviewers” (p 173) was the main reason for the paucity of teaching-related publications between 1930 and 1980.

• … that it would have been unforgivable had the same review process prevented teaching-related publications of the earlier years.

Questions?

Note: This presentation and related literature will be posted at:

http://dairynutrient.wisc.edu/

http://dairynutrient.wisc.edu/page.php?id=87

Michel A. Wattiaux [email protected] Department of Dairy Science, UW-Madison Getting Scholarly Teaching Projects Published 2010 ADSA-PSA-AMPA-CSAS-WSASAS-ASAS,

Documents

scholarship of teaching

scholar of teaching

teaching appointment

scholarly teaching projects

high teaching responsibility

teachingrelated authorship

learning sotl

documentation of excellence