ETEC and NCAAA Accreditation: The Data Institutions Must Be Able to Produce

For registrars, quality deanship staff and IT directors at Saudi universities and colleges inside an accreditation cycle. By the end you should be able to say, for each standard, which artefact satisfies it, which system it comes from, who owns the number, and what breaks when two systems answer the same question differently.

Accreditation is a data problem before it is a quality problem. Institutions rarely struggle because teaching is poor, but because they cannot produce, on demand, the evidence that it is good. The Academic Accreditation Policies issued by ETEC state that conclusions about quality "will be based as far as possible on directly observable evidence", and that institutions must present "sound and sufficient evidence" against the standards. Panels do not grade intent.

The standards are a list of evidence requests

Read the standards as a procurement list. The 2018 institutional standards run to eight standards of numbered sub-criteria: Standard 3 runs from 3.1 Design and development of academic programs to 3.6 Learning resources; Standard 4 includes 4.2 Student records and 4.7 Alumni. The programme standards use six.

Each sub-criterion is a question a panel member will ask, and each answer either exists as a dated artefact with an owner or it does not. A criterion on student records is not satisfied by describing the registry, but by a retention policy, an access-control list, a grade-change audit trail, and sample student files that match what the SIS reports.

[NEEDS SOURCE: 2022 editions of both standards documents are listed in ETEC's forms index and in university document libraries, but the full 2022 text could not be retrieved. Confirm current numbering, criterion counts and the set of essential (asterisked) indicators against the 2022 editions before relying on the 2018 structure]

Standard, evidence artefact, source system, owner

Standard numbers follow the 2018 institutional edition. Adjust to the edition in force.

Standard	Evidence artefact	Source system	Owner
1 and 2. Planning and governance	Strategic and operational plans, indicator achievement report, committee minutes with attendance, policy register with version dates	Planning or BI tool, document management, committee portal	Strategic planning office, university secretary
2.5 Quality assurance management	Quality manual, internal review reports, Self-Evaluation Scales, action plans with closure dates	Quality management system	Quality deanship
3.1 to 3.3 Programme design and QA	Programme specification (T103), course specifications (T104), Annual Program Report (T106), course reports (T107), advisory committee minutes	Curriculum module, LMS, document store	Programme coordinator
3.2 Graduate attributes and outcomes	PLO to CLO mapping matrix, assessment blueprints, marked student work at three bands	LMS plus scanned archive	Assessment coordinator
3.6 Learning resources	Holdings by programme, database usage, library survey results	Library system, vendor reports	Library director
4. Students	Admission criteria, applicant to enrolment funnel, grade-change audit trail, advising caseloads, graduate destination survey	SIS, alumni or CRM system, survey platform	Registrar, dean of student affairs
5. Faculty and staff	Contracts, qualification verification, workload allocation, appraisals, turnover	HR system, timetabling	HR director
6. Institutional resources	Budget versus actual by cost centre, expenditure per student, risk register, inspection certificates	Finance ledger, ITSM, facilities system	CFO, IT and facilities directors
7 and 8. Research and community	Publication and citation counts, research budget share, external funding, patents, partnership agreements	Research information system, Scopus or WoS export, finance	Vice President for Research

Almost every row crosses two or more systems, and the named owner rarely controls the system the number comes from. That gap is where preparation fails.

The KPI set, and why the denominator decides the number

NCAAA requires self-assessment against key performance indicators "using internal and external benchmarks", and publishes national KPI averages as a comparator. Codes are conventionally written KPI-I-nn at institutional level and KPI-P-nn at programme level, but the numbering is not stable: published KPI mechanisms from two Saudi universities list different counts and different code-to-indicator mappings. If your dashboard was built from a peer's template, the labels may not mean what your reviewers expect.

[NEEDS SOURCE: the ETEC-issued "Key performance indicators for higher education institutions" and "Key performance indicators for higher education programs" forms are named in ETEC's forms index but could not be retrieved. Confirm the authoritative code list, count and definitions from those files]

Operationally, the denominator does the work. In the KPI mechanism published by Imam Abdulrahman Bin Faisal University, programme completion rate takes a numerator of students "who successfully completed the programs in minimum time as stipulated in study plan" over a denominator of the "Total Number of students admitted in the program 4 or 5 years before". Change any of these and the value moves several points:

• Minimum time, or any time. A student graduating in year six is in the numerator under one reading and not the other.
• Cohort definition. Direct-entry admits only, admits plus internal transfers in, or admits minus transfers out.
• Preparatory year. If it sits outside the programme in the SIS, the cohort clock starts a year later than the panel assumes.
• Withdrawn and dismissed students. Removing them from the denominator flatters completion; keeping them is the defensible reading.
• Sections and branches. Reporting male and female sections or branch campuses together changes both the value and the comparability of the benchmark.

The same applies elsewhere. Graduate employability counts graduates employed or in postgraduate study within a year over the total who graduated that year, so it needs a destination survey with known coverage. First-year retention needs a fixed census date, not a live headcount. Student to teaching staff ratio turns on whether the denominator is headcount, full-time equivalent or teaching-load weighted.

Benchmarks have their own structure: an actual (latest reported) value, an internal benchmark from prior years (commonly a maximum of three), and an external benchmark from a comparable programme chosen on stated grounds, including similarity of purpose, curricular structure, facilities, accreditation status and data availability. "We compared ourselves to a highly ranked university" is not a rationale.

Where the data lives, and why the systems disagree

SIS owns admissions, enrolment, progression, grades and graduation. It disagrees with itself over time because a live query returns different headcounts before and after late withdrawals post, because status codes (suspended, deferred, dismissed, transferred) are read differently by registry and quality staff, and because programme codes change at curriculum revision, splitting one cohort across two identifiers.

LMS owns delivery and assessment artefacts. It disagrees with the SIS because it is provisioned by section rather than programme, because guest and repeat enrolments are not cleaned, and because the gradebook holds the pre-moderation mark while the SIS holds the post-moderation one.

HR counts positions, not teaching effort, so adjuncts, visiting staff and secondments are the usual variance. The count that satisfies HR and the one that belongs in a student-to-staff ratio are rarely the same.

Finance owns budget, expenditure and research funding, but cost centres do not map cleanly to programmes, so expenditure per student must be allocated on a basis that is written down and reused. Library and IT count vendor usage by IP range or platform session, not by programme or student.

The fix is a documented rule set, not a one-off cleanse: one census date per indicator, one definition of an active student, one mapping of programme codes across curriculum versions, one faculty FTE rule. Version the rules and cite the version in every report.

Surveys: instrument, population, response rate

Several KPIs are survey-derived means rather than counts. The instruments in common Saudi practice are the Course Evaluation Survey, Program Evaluation Survey, Student Experience Survey, an employer or beneficiary survey, and library and IT user surveys. KPI mechanisms often compute a value from named question numbers inside an instrument, so question wording and numbering are part of the evidence trail. Change the instrument and the time series breaks.

Three controls: administer the programme evaluation survey to the intended population (typically final-year students, not everyone enrolled); report the response rate and its denominator alongside every mean, because 4.3 from 11 responses in a cohort of 240 is not evidence; and keep windows clean, since course surveys run per term and employer surveys annually.

[NEEDS SOURCE: no NCAAA or ETEC document retrieved specifies a minimum survey response-rate threshold for accreditation evidence. If one exists, quote it from the survey guidance or handbook rather than inferring it]

Programme learning outcomes and the assessment trail

This is the most-inspected chain in a programme review and the one most often broken in the middle. It runs: NQF learning domains, to programme learning outcomes, to course learning outcomes, to specific assessment tasks, to attainment data, to a decision recorded in the Annual Program Report, to a change in the next programme specification.

The National Qualifications Framework for the Kingdom of Saudi Arabia, second edition, approved in 1444 AH (2023), sets eight levels (bachelor's 6, master's 7, doctorate 8) across four learning domains: Knowledge and Understanding; Skills; Values; Responsibility and Autonomy. Programme learning outcomes are written into those domains and shown consistent with the level, which is what the NQF consistency form records.

Where it breaks:

• The PLO to CLO matrix is a spreadsheet built for the last review and no longer matches current course specifications.
• Courses carry an "introduced, reinforced, mastered" designation against a PLO, but no assessment task is tagged to the mapped outcome, so attainment cannot be computed at all.
• Attainment is computed from the whole course mark rather than the question or rubric criterion that assesses the outcome. A panel will ask which question measured it.
• Attainment is reported but nothing follows. Three consecutive Annual Program Reports show 61% against a 70% target with the same sentence of commentary. That is worse than having no target: it documents a loop that never closed.

Working backwards from the site visit

The fixed points are few but firm. The visit runs three to four days for a programme and four to five for an institution, on-site, online or hybrid, with a panel of at least three members (programme) or four (institution) including the chair. The chair finalises the Review Panel Report "within two weeks of the end of the visit". The decision needs a quorum of at least 70% of the Council or Committee and agreement of at least 75% of those present.

What the policy book does not fix is your internal calendar. Working backwards from a visit at T:

• T minus 24 months. Confirm eligibility (licensed, one graduated cohort, provision inside the Ministry of Education approved scope). Freeze KPI definitions and census dates, then collect trend data on them.
• T minus 18 months. Complete the PLO to CLO to assessment mapping for every course; run a full survey cycle on the final instruments.
• T minus 12 months. Draft the Self-Study Report against the Self-Evaluation Scales and list every criterion whose artefact does not yet exist.
• T minus 9 months. Appoint the independent reviewer, whose report verifying the accuracy and objectivity of your evaluation is attached to the self-study.
• T minus 6 months. Close the gaps, or record them with an action plan and a date. A dated remediation plan reads better than a gap dressed as a strength.
• T minus 3 months. Index the evidence room by criterion number and rehearse retrieval: name a criterion, produce the artefact in two minutes.
• T minus 1 month. Lock the data. Anything changing after this creates a discrepancy between the self-study and the screen a panel member is shown.

A procedure for a continuously ready evidence base

1. Build a criterion register. One row per sub-criterion in force: number, text, evidence artefact, source system, data owner, refresh frequency, last refreshed, location. Everything else hangs off this.
2. Write the definitions document. For every KPI: numerator, denominator, population, exclusions, census date, system of record and who signs the value. Version it, and never compute a KPI outside it.
3. Fix the census dates. Publish the dates on which enrolment, headcount and staffing are frozen. All reports for that year use those snapshots.
4. Snapshot, do not query. Write each census to an immutable store with the definition version attached, so a report produced next March reproduces exactly.
5. Tag assessments to outcomes at design time. Retro-tagging a term of exams is the most expensive remediation in accreditation preparation.
6. Run annual reporting on a fixed schedule. Course reports within set weeks of results release, the Annual Program Report before the next term's planning, KPI values with all three benchmarks at the same point each year.
7. Close every loop in writing. Each Annual Program Report ends with actions, owners and dates; the next opens by reporting against them.
8. Self-rate annually against the Self-Evaluation Scales, with a colleague from another faculty as challenger. That gap is your improvement backlog.
9. Maintain the index, not the pack. A live index of where each artefact is generated beats a folder of ageing PDFs.
10. Track expiry dates centrally. The two accreditations expire on different dates, and every re-accreditation is a full review applied for before expiry.

Where teams get this wrong

The quality deanship owns the report but not the data. It writes the self-study from numbers supplied by registry, HR and finance, none of whom were told which definition to use. Three irreconcilable student counts appear in one document, and a panel member finds all three in a morning.

Definitions are agreed verbally. Two years later nobody can explain why last year's retention was 84% and this year's, computed correctly, is 79%. The trend now looks like a decline that has to be defended.

Evidence is generated for the visit rather than by operations. Advisory committee minutes written retrospectively, all dated in the same fortnight, in the same font, convert a documentation gap into a credibility question.

Branch campuses and sections are aggregated silently. When the panel visits a branch and local numbers differ from the aggregate, convention looks like concealment.

What to automate, and what not to

Automate the mechanical: census snapshots, KPI computation from a single definitions document, benchmark comparison, survey distribution and response-rate tracking, the criterion-to-artefact index, expiry alerts, and assembly of annual reports from data already held. That is where manual effort produces errors rather than insight.

Do not automate judgement. The narrative explaining why attainment fell, the choice of a comparable external benchmark, the self-rating on the Self-Evaluation Scales and the action plan that follows a weak result are academic judgements. A system that generates plausible commentary produces a self-study that reads well and collapses under one follow-up question. Software should make the numbers indisputable so the quality team spends its time on the argument, not the arithmetic.

Where a system helps

The workflow a higher education ERP should carry here is narrow: hold KPI definitions once, snapshot enrolment and staffing on fixed census dates, tag assessments to programme learning outcomes at course approval, and generate course and annual programme reports from that data rather than from re-keyed spreadsheets. That is the part of accreditation readiness that is genuinely a systems problem, and what CampusOS is built to do. More on our higher education solutions page.

FAQ

How far ahead should we start preparing? Two years, if you need trend data on stable definitions. The binding constraint is not writing the self-study; it is that KPI history computed on definitions you changed last year is not a trend.

Which single artefact causes the most trouble? The programme learning outcome to assessment mapping. Grades survive; the link between a specific exam question and a specific outcome does not, unless it was recorded when the assessment was designed.

Can we use a peer institution's KPI template? As a checklist, not as definitions. Published KPI mechanisms from different Saudi universities assign different codes and denominators to indicators with similar names.

What if two systems give different student numbers during the visit? Expect it to be pursued. The workable answer is a written definitions document plus a dated snapshot showing which figure is the system of record and why the other differs. Recalculating during the visit is not an answer.

Related reading: What Is NCAAA? Saudi Arabia's Academic Accreditation Body, Explained (KB-003).

Sources

• ETEC / NCAAA, Academic Accreditation Policies (Policy Book)
• NCAAA, Standards for Institutional Accreditation, 2018 edition
• NCAAA, Standards for Program Accreditation, 2018 edition
• King Saud University Deanship of Development and Quality, NCAAA standards and templates index, December 2022 set
• Imam Abdulrahman Bin Faisal University, KPIs Monitoring and Assessment Mechanism
• King Khalid University, institutional KPI definitions (DI-101)
• ETEC, National Qualifications Framework for the Kingdom of Saudi Arabia, 2nd edition
• ETEC, Institutional and Programmatic Accreditation Results