Required Online Orientation: A Predictor of Success?

Presented byLinda DeFelice, Professor of Communication and Karen Durkin, Executive Director of Institutional

Research and AssessmentGloucester County College, Sewell, NJ

ITC eLearning 2009 Conference / February 23, 2009

1

Gloucester County College, Sewell, NJ

•Public•Associate-degree•Accredited member of Middle States Commission on Higher Education•First students in Sept. 1968•250 acre campus in suburban Deptford Twp.•59 degree programs of study• Enrollment 6,135 (Fa08)

2

Online credit courses at GCC Began in Spring, 1999 Currently 8.3 % of total enrollment Class size = ¾ of on-campus class size English Composition II (COM-102)

Requirement in all degree programs- Course requires students to read selected prose fiction, poems, and plays, and to write a series of documented, critical/analytical essays based on those works. Students continue to develop the tools and forms of research covered in COM-101.

Online COM-102 class size - varied from 19 to 21

3

Prior ResearchParticipation in recommended in-class orientation

and GPA are significantly related to student success in online undergraduate business course (Wojciechowski and Palmer, 2005).

This study builds on prior research by examining:Student grade in mandatory, online, course

specific orientation in online COM-102 course;Instructor experience teaching online course

4

Julia Child’s Kitchens and COM-102: Similarities (Really!)

5

Cambridge, MA Kitchen in SmithsonianPhoto credit: http://www.flickr.com/photos/colewake1/3222030296/

Provence KitchenPhoto: Cooking With Friendshttp://www.cookingwithfriends.com

Advance InformationCourse information page

What is the course about?Who is the instructor?What are the textbooks and other materials required?How will I be graded?How is the course structured?

Welcome lettersDirector of Technology Instructor

Two weeks or more prior to course start What to do the first time you log on – Start Here, Syllabus,

Week 1

6

GCC Provided Online Orientation2002 - 2007 2008 - Distance Learning

WebsiteFAQSelf-assessments

Is Distance Learning Right for Me?

Characteristics of Successful Distance Learners

Course Information Page

eLearning Website @ http://elearning.gccnj.edu

FAQSelf-assessmentsOnline OrientationSelf-Help VideosTechnical assistance

7

Why a Course Specific Orientation Module?Student Concerns

Orientation Addresses Concerns

1. Isolation2. Disconnectedness3. Limited interaction4. Technical issues

1. Sense of community2. Guidance from online

students3. Discussion and email

communication4. Practice with course

tools

8

Photo credit: http://www.flickr.com/photos/unhcr/2606399101/

Photo credit: http://www.flickr.com/photos/entropyer/56979656/

Orientation Module OverviewModule 1 (Jan. 20-26)

OrientationIntroduction to Reading and Responding to Fiction

Special Orientation Assignments

Take Orientation QuizLoad and navigate Ariel (CD-ROM)

Reading Assignments

Start HereSyllabusDiscussion Grading CriteriaIntroduction 1-23Reading Stories – 27-42 (Includes Updike’s A & P and Chopin’s The Story of An Hour)Ariel – Critical Essay, An Overview of ‘The Story of An Hour”

Writing Assignments

DiscussionPost introduction as an original message; respond to a classmate’s introduction DQ and DQ Replies about A & P and The Story of An Hour a EmailSend an eLearning email attachment (in Word or Rich Text Format) to me describing a favorite book.

Module 1 Detailed Instructions

9

Orientation Module – Detailed InstructionsLearning ObjectivesSpecial Orientation Assignments

Ariel CD-ROMQuiz on Start Here and Syllabus material

Reading AssignmentsWriting Assignments

Self-introduction and replyRespond to professor’s questions about course

content (DQs) and reply to a students response (DQ Reply)

Email .rtf attachment to professor 10

Start HereThis information will help you make the most

or your online class experience:

Our Course WebsiteOur Course StructureOur Course Discussion BoardHow to Use eLearning Email, Discussion,

and Assessment ToolsTips for Online Success

11

Syllabus ComponentsDetermination of GradeCommunication and FeedbackLate/Missed WorkPlagiarismTechnical AssistanceAttendanceAcademic and Student Support Services

12

Syllabus - Determination of GradeOrientation activities (10%):  Each student will

make posts and replies to the discussion board; send an email attachment to the instructor; and take an Orientation Quiz.

Online Discussion Participation (25%):  Each student will participate in eight discussions based on required readings. Students are required to read all discussion posts. Discussion rubric is located in Learning Module 1.

Midterm Exam (15%): Short Story Essay (15%): Poetry Project (20%): Final Exam (15%): 

13

Syllabus – Communication and Feedback PolicyEmail: Use eLearning’s email to send private communication

to me and to your classmates. Use my GCC email address, ___________________, only if eLearning is down which is a highly unusual occurrence. Email is my preferred method of private communication for my online classes.

Email response time: Monday through Friday, I check our course twice daily, and I will make every effort to respond to an email within 24 hours after I receive it. On weekends, I do check the course and usually send responses on Sunday. If I will not be checking email as regularly as usual, for example during a college holiday or while attending a conference, I will let the class know ahead of time.

Phone / In-person: Sometimes a simple phone call (__________) can provide clarification. My office hours are listed on the cover sheet of this syllabus. If those hours are inconvenient, we can schedule a telephone appointment for me to call you at a mutually agreeable time any day of the week.

14

Writing Assignments - ExcerptIn Module 1 – Introductions, post an original

message to the class. The subject line will be your preferred name (for example, Robert Alcorn may prefer to be called Rob Alcorn and Jacinda Watkins may prefer to be called Jaci Watkins). In one or two paragraphs, let us know your major, description of your current job, your career goals, and your experience taking online courses.  If you’ve taken online courses before, please offer advice to students new to online courses.

15

Quiz Examples1. In general, DQs are due on Mondays and DQ replies are

due on Fridays.a. True b. False

2. All assignment due dates, holiday dates, and last date to withdraw from the course can be found at a glance in the __________. a. Start Here documentb. Syllabusc. Calendar

3. If my office hours are inconvenient, we can schedule a telephone appointment for me to call you at a mutually agreeable time any day of the week.a. True b. False

16

The StudyResearch QuestionsMethodology

Sample Variables of Interest Analysis Procedures Limitations Design Sensitivity

Preliminary Findings

17

Research QuestionsH0 = Orientation grade is not related to student

final grade in course.H1 = Orientation grade is related to student final

grade in course.

H0 = Instructor experience teaching online course is not related to student final grade in course.

H1 = Instructor experience teaching online course is related to student final grade in course.

18

Methodology: Sample34 Sections of 0nline COM102 over 16

semesters (Fall 2002 to Summer 2008) Completers : Students that earned a grade in

the course (N=546) (excludes withdrawals (W), incompletes (I)and students that never attended (NA)

19

Methodology: Variables of InterestDependent VariableStudent Final Grade in

Course (Quality Points)

Independent VariablesStudent Orientation GradeAgeGenderGPAPrevious Online EnrollmentPrevious Online WithdrawalEnrollment Status (FT/PT)COMPASS Reading ScoreCOMPASS Writing ScoreInstructor Experience*

20

Methodology: Analysis ProceduresPhase One: Correlation Analysis

Phase Two: Linear Regression

Phase Three*: Ordinal Logistic Regression

21

Methodology: LimitationsRestricted data availability in semesters prior to

Fall 2006Prior online experiencePrior online withdrawals

Analyzed Two Samples Full Sample (N=546) Fall 2006 + Only (N=202)

22

Methodology: Design SensitivityStatistical Power =

The probability that the test will lead to rejection of the null hypothesis.

1 – P (Type II Error)

23

Methodology: Design SensitivityPower Level of 0.80 or higher (Cohen, 1988)

Regression/Correlation Analysis: Medium ES (f2 = .15)10 variables; α = .01; Required Sample Size =

1672 variables; α = .01; Required Sample Size =

97

24

Preliminary Findings: Descriptive Statistics

GENDERFull

SampleFall 2006+

N % N %Total (N)

546 202

Gender-Female 392 71.8

%137 67.3

%-Male 154 28.2

%65 32.7

%

25

Preliminary Findings: Descriptive Statistics

Full Sample Fall 2006+N % N %

Total (N) 546 202Age x = 28.31 x = 25.60-18 to 20 66 12.1

%66 32.7%

-21 to 30 301 55.0%

87 43.1%

-31 to 40 103 18.8%

29 14.4%

- 41 to 50 43 7.9% 10 5.0%-51 and over 14 2.6% 2 1.0%Missing 19 3.7% 8 4.0%

_ _

AGE

26

Preliminary Findings: Descriptive Statistics

Full Sample Fall 2006+N % N %

Total (N) 546 202GPA x = 2.84 x = 2.904.0 21 3.8% 9 4.5%3.0 to 3.9 208 38.1

%87 43.1%

2.0 to 2.9 184 33.7%

65 32.2%

1.0 to 1.9 49 5.1% 20 6.9%0.9 or lower 14 2.6% 4 2.0%Unknown 19 3.5% 8 4.0%

GPA

27

_ _

Preliminary Findings: Descriptive Statistics

ENROLLMENT STATUS

Full Sample

Fall 2006+

N % N %Total (N) 546 202EnrollmentStatus- Full Time 272 49.8

%84 41.6

%- Part Time

255 46.7%

110 54.5%

-Unknown 19 3.5% 8 4.0%28

Preliminary Findings: Descriptive Statistics

PRIOR ONLINE EXPERIENCE

Full Sample

Fall 2006+

N % N %Total (N) 546 202Yes 30 5.5% 30 14.9

%No 164 30.0

%164 81.2

%Unknown 352 64.5

%8 4.0%

29

Preliminary Findings: Descriptive Statistics

PRIOR ONLINE WITHDRAWALS

Full Sample

Fall 2006+

N % N %Total (N) 546 202Yes 6 1.1% 6 3.0%No 164 30.0

%164 93.6

%Unknown 376 68.9

%8 3.5%

30

Preliminary Findings: Descriptive Statistics

Full Sample Fall 2006+N % N %

Total (N) 546 202Reading x = 83.87 x = 84.76- 70 or Higher (P)

493 90.3%

180 89.1%

- 69 or Lower 20 3.7% 8 4.0%-Unknown 33 6.0% 8 4.0%Writing x = 82.22 x = 84.45- 75 or Higher (P)

427 78.2%

158 78.2%

- 74 or Lower 93 17.0%

32 15.8%

-Unknown 26 4.8% 12 5.9%

_ _

__

COMPASS® SCORES

31

Preliminary Findings: Descriptive Statistics

Full Sample Fall 2006+N % N %

Total (N) 546 2024.0 255 46.7

%99 49.0%

3.0 to 3.9 129 23.6%

44 21.8%

2.0 to 2.9 97 17.8%

34 16.8%

1.0 to 1.9 28 5.1% 14 6.9%0.9 to 0 29 5.3% 11 5.4%

ORIENTATION GRADE

32

Preliminary Findings: Descriptive Statistics

Full Sample Fall 2006+N % N %

Total (N) 546 202A – to A 153 38.0

%60 29.8%

B- to B+ 173 31.6%

57 28.1%

C to C+ 79 14.5%

35 17.3%

C- 30 5.5% 11 5.4%D- to D+ 28 5.1% 8 4.0%F 83 15.2

%31 15.3%

FINAL GRADESuccessRates (C or

Higher)

Full Sample =

84.1%

Fall 2006+ = 75.2%

33

Preliminary Findings: Correlation AnalysisFull Sample Fall 2006+

Independent Variables

R Sig. R Sig.

GPA .748**

.000 .750* .000

Orientation Grade .562**

.000 .595* .000

Age .299**

.000 .165* .021

Enrollment Status -.178**

.000 -.106 .143

Writing .153* .000 .200* .006Reading .142*

*.001 .155* .034

Prior Online Withdrawal

- - - - - - - - -.158* .027

Prior Online Experience

- - - - - - - - -.112 .119

Instructor Experience

-.010 .813 .062 .378

Gender .001 .978 .021 .769

*Significant at the 0.01 level; **Significant at the 0.05 level (2-tailed) 

34

Preliminary Findings: Regression Analysis

FullSample

Fall2006+

B SE t B SE tGPA 1.01

8.053 19.37

4.975 .080 12.20

5Orientation Grade

.303 .040

.246 .387 .060

6.404

Constant -1.36

4

.143 -9.542

-1.593

.228 -6.980

R2 = .604Adj. R2 = .602F = 393.397

R2 = .639Adj. R2 = .636F = 169.401

Model:Grade = α + B1(GPA) + B2(Orientation Grade) + U

35

Preliminary Findings: Conclusions

GPA and orientation grade are the most significant predictor of student grade in course

Orientation grade (r =.562) is more strongly correlated than orientation participation (r = .388; )

Instructor experience in online course does not impact student grade in course

36

New Developments• Effective Online

Teaching and Learning online course for faculty

• Higher Orientation Module grades• Multimedia• PDF • Welcome video

37

Thanks for being here! Questions?

38

Contact InformationLinda DeFelice

ldefelice@gccnj.edu / 856-415-2129Karen Durkin

kdurkin@gccnj.edu / 856-415-2284Gloucester County College1400 Tanyard RoadSewell, NJ 08080

PPT available on slideshare.net

39