Quality and Accreditation (1/3) Certification of Kingdom Tower for “ FIT for PURPOSE ” Series of tests of “Fitness” of sub-systems: Foundation Sub-Systems.

KSU – QMS (Quality Management System)

Assoc. Prof. Dr. Teay Shawyun

Quality and Accreditation (1/3)

Certification of Kingdom Tower for “FIT for PURPOSE”

What should you do?

Series of tests of “Fitness” of sub-systems:• Foundation Sub-Systems

Piling Sub-Systems Materials Sub-Systems, etc

• Electrical Sub-Systems Wiring Sub-Systems

Voltage and surge Sub-Systems, etc

• Water and Waste System Pipes Sub-Systems Water Flow Sub-Systems, etc

• Other support Sub-Systems


Certification of KSU for

“FIT for PURPOSE”

What should you do?

Series of tests of “Fitness” of sub-systems:• Facilities Foundations Sub-Systems

Buildings Sub-Systems Education Facilities Sub-

Systems, etc• Academic Foundation system

Colleges and Programs Sub-Systems

Faculty Sub-Systems, etc • Learning Resources system

Library Sub-Systems ICT Sub-Systems, etc

•Other Teaching – Learning – Research support Sub-Systems

Building OR

Creating Strong

and Sustainab

le Foundatio

ns of System

and Sub-Systems

Auditing AND Assessment

of the QUALITY of Foundations

of the System and

Sub-SystemsAffecting

Excellence and Ensuring

its “Fit for Purpose”

Accreditation or Certifying

that the System and Sub-Systems are “Fit for Purpose”

ACCREDITATION is built

on QUALITY


KSU – QMS (Quality Management System) 1/2

“Chicken and Egg”

Issue of which comes first?

“Accreditation OR Quality”?

KSU is addressing the Quality Issue through its

KSU – QMS (Quality Management System)

IQA = EQA“Standards and Criteria”

KSU – QMS (Quality Management System) 2/2

SIMPLE

Uses the NCAAA Standards and Criteria as

the Blueprint Same Standard and Criteria applicable to

Institution and College and Program

SOPHISTICATED

Developmental philosophy of planning and improvements

Quality Assessment methodology,

Strategic Performance Management

STRONG

Management through Measurement

Systemic and SystematicPerformance BasedNon – Prescriptive

Process and Results Oriented

SUSTAINABLE

Aimed at Improvements and Innovations

Long-term OrientationHolistic and Integrative

Characteristics of KSU – QMS (1/2) Systemic – Comprised of a set of sub-

systems interacting together to achieve a specified set of goals

Systematic – An identified set of IPOO steps leading to achievements of its outputs an outcomes

Holistic and Integrative – Summation of the total > Summation of Individual

Generic – Applicable to the institution and College, Programs or Administrative Units

Characteristics of KSU – QMS (2/2)

Unified – Aligned top to bottom, horizontally and vertically

Non-Prescriptive – Does not tell you what systems, tools, techniques or mechanisms to use to achieve your goals and objectives

Management through Measurement – Measurements bring about better Management

Strategic Performance Management (SPMS) – PMS, IMS and QMS

KSU – QMS is part of SPMS (Strategic Performance Management System)

Key Similarities and Differences between NCAAA and KSU – QMS Systems (1/6)Standards, Criteria and Items: 2 sets of Standards, Criteria and

Items, 1 for institution and 1 for program

The Standards, Criteria and Items of the Institution and program are similar with the Institution one being more comprehensive

There are 11 Standards and 58 Criteria

Standards, Criteria and Items: 1 comprehensive set of

Standards, Criteria and Items applicable for both for institution and program, as the performance of the programs aggregates and summates into the institution performance

There are 11 Standards and 58 Criteria based on the NCAAA institution set which are classified as Process-Oriented Values

The KPI and Benchmark are classified as the Results-Oriented Values

Key Similarities and Differences between NCAAA and KSU – QMS Systems (2/6)

KPI and Benchmark: Has an open approach set of KPI and

Benchmark to be defined by the institution or program

The KPI and Benchmark are not computed into the overall performance of the institution or program

KPI and Benchmark: Has both an open and closed

approach to KPI and Benchmark: Closed generic set defined by the

institution for all programs and the institution as a whole

Open set to be defined by the institution and program

The generic set of KPI and Benchmarks are applicable across board to all programs which are aggregated and summated into the overall institution performance

2 sets of KPI are used, Qualitative and Quantitative KPI

The Qualitative set uses a PDCA and ADLI criteria to determine the performance level criteria

The quantitative set uses the normal percentage, ratios or numeric to determine the performance ranges

Audit and Assessment:

The institution and program does a self-assessment and prepares an assessment report and is assessed by an external team appointed by NCAAA or recognized accreditation agencies

Audit and assessment is done once every five year

Accreditation and Certification is based on outcome of audit and assessment

Audit and Assessment:

The institution and program does a self-assessment and prepares an assessment report and is assessed by an external team appointed by KSU

Audit and assessment is done once every year

Opportunities for improvement is based on IQA audit and assessment Performance Report (IQAAPR)


Management:

The audit and assessment report will be used as the basis of a development plan by the institution or program

Management:

The IQAAPR will be used as the basis of an annual operation plan for continuous improvement and innovation by the institution or program

The annual operation plan is linked to the roll-over of the institution or program strategic plan


Assessment Approach: A 6 levels Star System is used to

determine the performance of each Standard, Criteria and Item

A “relevant” or “not relevant” system is used to screen out items that are not applicable to the program

The assessment is not based on the comparison with past performance The Items and Criteria are summated and aggregated into the determination of performance for each Standard

Assessment Approach: A 6 levels Scaled Performance

Scoring System using a weighted score approach is used to determine the performance of each Standard, Criteria and Item Process-Oriented Values contributing to 20% of the overall performance achievement score

A 6 levels Scaled Performance Scoring System using a weighted score approach is used to determine the performance of each KPI and Benchmark Results-Oriented Values contributing to 20% of the overall performance achievement score

The performance of each criteria also takes into account the “goals set” and “goals achieved” leading to “development” and “effectiveness” being measured contributing to remaining 20% of the performance achievement score.



Assessment Approach (Continued):

The KPI and Benchmark are not computed into the overall performance of the institution or program

The overall performance or the institution and program is based on the aggregation and averaging of the Stars into a 5.0 points scaled performance system

Assessment Approach (Continued) :

The Items and Criteria are summated and aggregated into the determination of performance for each Standard which forms the Process-Oriented Values

The KPI and Benchmarks forms the Results-Oriented Values

The overall performance of the institution or program is the summation both the Process-Oriented Standards, Criteria and Items Values and the Results-Oriented Values

The overall performance is based on the weighted scoring for both the Process-Oriented and Results-Oriented Values leading to a 1000 points scale system

KSU Standard, Criteria and Item and KPI

Process – Based Values Criterion KSU Standard – 11 Standards KSU Criteria – 80 Criteria

58 Main Criteria 11 KPI Criteria (generic to all colleges and

programs) 11 Additional KPI Criteria (defined by

colleges and programs)

Results – Based Values Criterion 64 KPI (29 Quantitative types, 35

Qualitative types)

KSU Standard, Criteria and Item requirement

KSU – QMS Standards, Criteria and Items

Explanations

o Standard 1: Mission and Objectives

STANDARD Requirement

1.1 Appropriateness of the Mission 1.1 CRITERIA Requirement

1.1.1 The mission for the school and program should be consistent with the mission of the institution, and the institution’s mission with the establishment charter of the institution.

1.1.1 ITEM details Requirement

1.1.2 The mission should establish directions for the development of the institution, schools or programs that are appropriate for the institution, schools or programs of its type and be relevant to and serve the needs of students and communities in .


1.1.3 The mission should be consistent with Islamic beliefs and values and the economics and cultural requirements of the .


1.1.4 The mission should be explained to its stakeholders in ways that demonstrate its appropriateness.


Process – Based Assessment using ADLI

"Process" refers to the methods used and to improve when addressing the standards, criteria, items and key performance indicators requirements in the KSU – QMS:

A "APPROACH" refers to the methods used to accomplish the process, it appropriateness and effectiveness and degree to which the approach is repeatable and based on reliable data and information

D "DEPLOYMENT" refers to the extent to which approach is applied in addressing Item requirements relevant and important to the HEI, its consistency and coherence across all appropriate work units

L "LEARNING" refers to refining the approach through cycles of evaluation and improvement, encouraging breakthrough change through innovation, sharing refinements and innovations across units

I “INTEGRATION" refers to the extent to which approach is aligned with organizational needs identified with measures, information, and improvement systems being complementary across processes and work units and plans, processes, results, analyses, learning, and actions are harmonized across processes and work units to support organization-wide goals

Results – Based Assessment using LeTCI "Results" refers to the organization's outputs

and outcomes in achieving the requirements in processes above. The four factors used to evaluate results are:

Le “LEVEL” – The current level of performance and its performance trend over a time period.

T “TREND”– The rate (i.e., the slope of trend data) and breadth (i.e., the extent of deployment) of performance improvements

C “COMPARISON” – The performance relative to appropriate comparisons and/or benchmarks

I “INTEGRATION” – The linkage of the results measures (often through segmentation) to important student and stakeholder; program, offering, and service; and in Process Items.

Overall Scaled Performance Scoring of Process – Based values Standard1st Column 2nd

Column

3rd Col.

4th Colum

n

5th Col.

6th Col.

7th Colum

n

8th Colum

n

9th Colum

n

Institutional, School and Program Context Weights

Score (%)

Weighte

d Scor

e

Goals Set

Goals Achv.

Develop

.

Effective

Overall

Perf.

Standard 1 Mission, Goals and Objectives 55

1.1 Appropriateness of the Mission 4 2.2 70%

50%

0 0 1.76

1.1.1 The mission for the school and program should be consistent with the mission of the institution, and the institution’s mission with the establishment charter of the institution.

1 50 0.5

1.1.2 The mission should establish directions for the development of the institution, schools or programs that are appropriate for the institution, schools or programs of its type and be relevant to and serve the needs of students and communities in Saudi Arabia.

160 0.6

1.1.3 The mission should be consistent with Islamic beliefs and values and the economics and cultural requirements of the Kingdom of Saudi Arabia.

180 0.8

1.1.4 The mission should be explained to its stakeholders in ways that demonstrate its appropriateness.

130 0.3

Overall Assessment 2.2 1.76

1 The weighted score for each item is derived from SCORE * WEIGHTS.

2. The overall weighted score (2.2) is an averaged summation of each of the weighted score of each item and contributes 80% to overall performance.

3. As there is no “development” and “effective”, 20% is lost, and the final Overall performance is 1.76 (which is 0.8 * 2.2)



3. As there is no “development” and “effective”, 20% is lost, and the final Overall performance is 1.76 (which is 0.8 * 2.2)

Scaled Performance Scoring of Process – Based values of each Item

Institutional, School and Program Context Weights

Score

(%)

Weighted Score

Standard 1 Mission, Goals and Objectives 551.1 Appropriateness of the Mission 4 2.21.1.1 The mission for the school and program should be

consistent with the mission of the institution, and the institution’s mission with the establishment charter of the institution.

1 50 0.5

Process – Based Values Criterion Scoring Guidelines

SCORE PROCESS – based Performance Scoring Guidelines0% or 5% OR

No StarThe practice, though relevant, is not followed at all based on the following: No SYSTEMATIC APPROACH (methodical, orderly, regular and organize) to Standards requirements is evident; information lacks specific

methods, measures, deployment mechanisms, and evaluation, improvement, and learning factors. (A) Little or no DEPLOYMENT of any SYSTEMATIC APPROACH (methodical, orderly, regular and organize) is evident. (D) An improvement orientation is not evident; improvement is achieved through reacting to problems. (L) No organizational ALIGNMENT is evident; individual standards, areas or work units operate independently. (I)

10%, 15%, 20% or 25% OR 1

Star

The practice is followed occasionally but the quality is poor or not evaluated based on the following: The beginning of a SYSTEMATIC APPROACH (methodical, orderly, regular and organize) to the BASIC REQUIREMENTS of the

Standards is evident. (A) The APPROACH (methodical, orderly, regular and organize) is in the early stages of DEPLOYMENT in most standards or work units, inhibiting

progress in achieving the basic requirements of the Standards. (D) Early stages of a transition from reacting to problems to a general improvement orientation are evident. (L) The APPROACH is ALIGNED with other standards, areas or work units largely through joint problem solving. (I)

30%, 35%, 40% or 45% OR 2 Stars

The practice is usually followed but the quality is less than satisfactory based on the following: An EFFECTIVE, SYSTEMATIC APPROACH, (methodical, orderly, regular and organize) responsive to the BASIC REQUIREMENTS of

the Standards, is evident. (A) The APPROACH is DEPLOYED, although some standards, areas or work units are in early stages of DEPLOYMENT. (D) The beginning of a SYSTEMATIC APPROACH (methodical, orderly, regular and organize) to evaluation and improvement of KEY PROCESSES is

evident. (L) The APPROACH is in the early stages of ALIGNMENT with the basic Institution, College or Program or Administrative Unit

needs identified in response to the Institution, College or Program or Administrative Unit Profile and other Process Standards. (I)

50%, 55%, 60% or 65% OR 3 Stars

The practice is followed most of the time. Evidence of the effectiveness of the activity is usually obtained and indicates that satisfactory standards of performance are normally achieved although there is some room for improvement. Plans for improvement in quality are made and progress in implementation is monitored. An EFFECTIVE, SYSTEMATIC APPROACH (methodical, orderly, regular and organize), responsive to the OVERALL REQUIREMENTS of the

Standards, Criteria and Items is evident. (A) The APPROACH is well DEPLOYED, although DEPLOYMENT may vary in some standards, areas or work units. (D) A fact-based, SYSTEMATIC (methodical, orderly, regular and organize) evaluation and improvement PROCESS and some organizational

LEARNING are in place for improving the efficiency and EFFECTIVENESS of KEY PROCESSES. (L) The APPROACH is ALIGNED with the Institution, College or Program or Administrative Unit needs identified in response to the

Institution, College or Program or Administrative Unit Profile and other Process Standards. (I)70%, 75%, 80%, or 85% OR 4 Stars

The practice is followed consistently. Indicators of quality of performance are established and suggest high quality but with still some room for improvement. Plans for this improvement have been developed and are being implemented, and progress is regularly monitored and reported on. An EFFECTIVE, SYSTEMATIC APPROACH (methodical, orderly, regular and organize), responsive to the MULTIPLE REQUIREMENTS of the

Standards, Criteria and Items is evident. (A) The APPROACH is well DEPLOYED, with no significant gaps. (D) Fact-based, SYSTEMATIC (methodical, orderly, regular and organize) evaluation and improvement and organizational LEARNING are KEY

management tools; there is clear evidence of refinement and INNOVATION as a result of organizational-level ANALYSIS and sharing. (L) The APPROACH is INTEGRATED with the Institution, College or Program or Administrative Unit needs identified in response to

the Institution, College or Program or Administrative Unit Profile and other Process Standards. (I)

90%, 95% or

100% OR 5 StarsThe practice is followed consistently and at a very high standard, with direct evidence or independent assessments indicating superior quality in relation to other comparable institutions. Despite clear evidence of high standards of performance plans for further improvement exist with realistic strategies and timelines established. An EFFECTIVE, SYSTEMATIC APPROACH (methodical, orderly, regular and organize), fully responsive to the MULTIPLE REQUIREMENTS of the

Standards, Criteria and Items is evident. (A) The APPROACH is fully DEPLOYED without significant weaknesses or gaps in any areas or work units. (D) Fact-based, SYSTEMATIC (methodical, orderly, regular and organize) evaluation and improvement and organizational LEARNING are KEY

organization-wide tools; refinement and INNOVATION, backed by ANALYSIS and sharing, are evident throughout the organization. (L) The APPROACH is well INTEGRATED with the Institution, College or Program or Administrative Unit needs identified in

response to the Institution, College or Program or Administrative Unit Profile and other Process Standards. (I)

Overall Scaled Performance Scoring of Results – Based values KPI (Key Performance Indicators)Institutional, School and Program

ContextWeights

Score

(%)

Weighted Score

Goals

Set

Goals

Achv.

Develop.

Effective

Overall Perf

.

Standard 1 Mission, Goals and Objectives

55

1.6 Key Performance Indicators or Benchmarks

15 4.5 20%

30%

1 1 4.5

1.6.1 Level of stated institution’s, schools’ or programs’ philosophy or commitments; processes to formulate strategy and plans, and plans are implemented; development of KPI achievement to measure the plans, implementation and achievements in all missions. (Levels)

5 40 2

1.6.2 Level of institution’s schools’ or programs’ strategy map alignment achievement with the national HE strategies (Levels)

5 0 0

1.6.3 Percentage of institution’s, schools’ or programs’ goal achievements according to the operational indicators that is set. (%)

5 50 2.5



3 As there is both “development” and “effectiveness”, representing 20% the final Overall performance is 4.5 (which is (0.8 * 4.5 + 0.2 * 4.5)



3 As there is both “development” and “effectiveness”, representing 20% the final Overall performance is 4.5 (which is (0.8 * 4.5 + 0.2 * 4.5)

Performance Assessment of a qualitative KPILevel 1 No systematic approach in terms of the (A) Approach for the Strategic Planning Process as per the (P), (D), (C)

and (A) of PDCA (Plan, Do, Check, Act) is evident; information is subjective, unreliable and vague.

Level 2 The beginning of a systematic approach in terms of the (A) Approach for the Strategic Planning Process with (P) according to the basic requirements of the Standard as supported by documented evidence.

Major gaps exist in (D) Deployment that would inhibit progress in the Strategic Planning Process in achieving the basic requirements of the (L) Levels of performance as evidenced by the documented Goals and KPI achievements.

Early stages of a transition from reacting to problems to a general improvement orientation are evident.

Level 3 An effective, systematic approach in terms of the (A) Approach and (D) Deployment for the Strategic Planning Process, (P), (D) and (C) responsive to the basic requirements of the Standard in the (L) Levels of performance as evidenced by the documented Goals and KPI achievements.

The Strategic Planning Process approach is deployed, although some areas or work units are in early stages of deployment based on its (P) and (D).

The beginning of a systematic approach to evaluation and improvement of key processes in Strategic Planning Process based on (L) Levels of performance or outcome indicators is evident.

Level 4 An effective, systematic approach in terms of the (A) Approach and (D) Deployment and (L) Learning in Strategic Planning Process, with (P), (D) and (C) according and responsive to the overall requirements of the Standard in the (L) Levels and (T) Trend performance as evidenced by documented Goals and KPI achievements.

The approach is well deployed, although deployment and learning may vary in some areas or work units and is aligned with basic organizational needs identified in the other (L) Levels and (T) Trends of Goals and KPI achievement.

A fact-based, systematic evaluation and improvement process based on performance or outcome indicators are in place for improving the (L) and (T) Trend of efficiency and effectiveness of key processes and outcomes or outputs.

Level 5 An effective, systematic approach in terms of the (A) Approach, (D) Deployment, (L) Learning and (I) Integration in the Strategic Planning Process, with (P), (D), (C) and (A) responsive to the overall requirements of the Standard in the (L) Levels, (T) Trend and (C) Comparison performance as evidenced by the documented Goals and KPI achievements supporting current and changing educational service.

The approach is well deployed, with no significant gaps and is well integrated with the institution, school or program organizational needs as identified in the other (L) Levels, (T) Trends and (C) Comparison of Goals and KPI achievement.

A fact-based, systematic evaluation and improvement process and organizational learning/sharing are key management tools; there is clear evidence of refinement, innovation, and improved integration as a result of organizational-level analysis and sharing based on performance or outcome indicators.

Level 6 An effective, systematic approach in terms of the A) Approach, (D) Deployment, (L) Learning and (I) Integration in the Strategic Planning Process, with (P), (D), (C) and (A) fully responsive to all the requirements of the (L) Levels, (T) Trend and (C) Comparison and (I) Integration performance as evidenced by the documented Goals and KPI achievements supporting current and changing educational service.

The approach is fully deployed without significant weaknesses or gaps in any areas or work units and is fully integrated with the institution, school or program organizational needs as identified in the other (L) Levels, (T) Trends, (C) Comparison and (I) Integration of Goals and KPI achievement.

A very strong, fact-based, systematic evaluation and improvement process and extensive organizational learning/sharing are key management process and tool; strong refinement, innovation, and integration, backed by excellent organizational-level analysis and sharing based on performance or outcome indicators are evident.

Results – Based Values Criterion Scoring GuidelinesSCORE RESULTS – based Performance Scoring Guidelines

0% or 5% There are no organizational PERFORMANCE RESULTS or poor RESULTS in the standards and areas reported.

TREND data are either not reported or show mainly adverse TRENDS. Comparative information is not reported. RESULTS are not reported for any standards, criteria or items or areas of importance to the Institution,

College or Program or Administrative Unit KEY MISSION or Institution, College or Program or Administrative Unit requirements.

10%, 15%,20%, or 25%

A few organizational PERFORMANCE RESULTS are reported; there are some improvements and/or early good PERFORMANCE LEVELS in a few standards, criteria or items or areas.

Little or no TREND data are reported, or many of the TRENDS shown are adverse. Little or no comparative information is reported. RESULTS are reported for a few standards, criteria or items or areas of importance to the Institution,


30%, 35%,40%, or 45%

Improvements and/or good PERFORMANCE LEVELS are reported in many standards or areas addressed in the Standards requirements.

Early stages of developing TRENDS are evident. Early stages of obtaining comparative information are evident. RESULTS are reported for many standards, criteria or items or areas of importance to the Institution,


50%, 55%,60%, or 65%

Improvement TRENDS and/or good PERFORMANCE LEVELS are reported for most s standards, criteria or items or areas addressed in the Standards requirements.

No pattern of adverse TRENDS and no poor PERFORMANCE LEVELS are evident in standards, criteria or items or areas of importance to Institution, College or Program or Administrative Unit KEY MISSION or Institution, College or Program or Administrative Unit requirements.

Some TRENDS and/or current PERFORMANCE LEVELS – evaluated against relevant comparisons and/or BENCHMARK – show standards or areas of good to very good relative PERFORMANCE.

Institution, College or Program or Administrative Unit PERFORMANCE RESULTS address most KEY student, STAKEHOLDER, and PROCESS requirements.

70%,75%,80%, or 85%

Current PERFORMANCE LEVELS are good to excellent in most standards, criteria or items or areas of importance to the Standards requirements.

Most improvement TRENDS and/or current PERFORMANCE LEVELS have been sustained over time. Many to most reported TRENDS and/or current PERFORMANCE LEVELS—evaluated against relevant

comparisons and/or BENCHMARKS—show areas of leadership and very good relative PERFORMANCE.

Institution, College or Program or Administrative Unit PERFORMANCE RESULTS address most KEY student, STAKEHOLDER, PROCESS, and ACTION PLAN requirements.

90%,95%,or

100%

Current PERFORMANCE LEVELS are excellent in most standards, criteria or items or areas of importance to the Standards requirements.

Excellent improvement TRENDS and/or consistently excellent PERFORMANCE LEVELS are reported in most standards, criteria or items or areas.

Evidence of education sector and BENCHMARK leadership is demonstrated in many standards, criteria or items or areas.

Institution, College or Program or Administrative Unit PERFORMANCE RESULTS fully address KEY student, STAKEHOLDER, PROCESS, and ACTION PLAN requirements.

Thank you

Quality and Accreditation (1/3) Certification of Kingdom Tower for “ FIT for PURPOSE ” Series of tests of “Fitness” of sub-systems: Foundation Sub-Systems.

Documents

foundation subsystems

fitness of subsystems

electrical subsystems

subsystems voltage

sub systems auditing

ksu qms systems

quality quality

criteria standards