Exam Reliability and Validity
The reliability and validity of the Ontario Clinical Exam is a top priority. Several measures were put in place to ensure that the exam was developed in regular consultation with experts in physiotherapy and the field of exam development. Safeguards were then implemented to ensure that exam delivery and scoring were consistent and fair to all candidates.
Exam Blueprint Development
An exam blueprint was developed to serve as a framework for the content to be assessed in the Ontario Clinical Exam (OCE). The foundational document for the creation of the OCE was the Competency Profile for Physiotherapists in Canada (2017). The content areas were further defined through a series of focus groups with PT Residents and physiotherapists with experience supervising, educating and working with PT Residents.
Feedback from focus groups was further validated with a survey conducted with PT Residents and physiotherapists with less than three years of experience. Survey respondents were asked to review each competency and rank the level of importance to public protection and the frequency in which the competency is performed by an entry to practice physiotherapist. The results of the focus groups and survey were reviewed by the College’s exam working group and Council, who approved the list of competencies and topics to be assessed.
Additionally, College data informed the content of the exam including the areas of practice, and patient types. The list of conditions by body system originally assessed on the PCE-Clinical through CAPR (2018) was reviewed and updated by the exam working group. The finalized Ontario Clinical Exam blueprint – approved by Council on May 18, 2022 – outlines the number of core questions by domain and the overall percentage of the competency domain that will be represented on the examination, in addition to the specific competencies and indicators to be examined.
Further information on the exam blueprint can be found on the Preparing for the Ontario Clinical Exam webpage.
Quality Assurance of Exam Questions
Items are authored in collaboration with item writers and reviewers (all physiotherapists) before they are edited and formatted. Exam questions were also reviewed by an external plain language reviewer and changes were confirmed by a panel of Canadian trained and internationally trained physiotherapists. Finalized items are reviewed by a panel of examiner reviewers and may also be reviewed by the College Practice Advisors.
Quality Assurance of Exam Delivery
Before being hired as a contractor for the College, examiners are vetted to ensure they meet the criteria for the role including being a registered physiotherapist in good standing with the College of Physiotherapists of Ontario.
Before being assigned to an exam, all examiners complete a comprehensive full day New Examiner Training session where they are assessed by trained facilitators and receive feedback on their performance.
Further training is provided to all examiners the day before each exam session. Examiner performance is assessed annually through peer-assessments and exam observances.
A formalized training program, regular retraining of examiners, and performance review are activities that are in alignment with industry best practices. It is unusual for an exam training program to provide small group training with direct performance feedback; however, the College felt this was a meaningful investment to ensure the accuracy of examiner judgements. Examiners are provided with detailed scoring criteria which are anchored to performance indicators in the blueprint.
Exam Scoring
Before being released to candidates, all examination scores go through multiple levels of verification and quality assurance. This ensures that each candidate’s performance is appropriately assessed, and their scores are accurately reported.
The first exams were delivered in October and November of 2022. We assessed cumulative data from both sessions to determine the pass score, also known as the standard or cut score. Please note: the College will not be releasing the absolute pass mark, pass rates by country of origin or province or individual candidate scores.
The Ontario Clinical Exam is a criterion-reference exam. This means that the candidate’s performance is measured against a predetermined standard. This standard was determined through psychometric analysis of raw data in collaboration with rating review by an expert panel of examiners. Results from psychometric analysis and the expert panel review are then presented to the Examination Committee for final determination of the standard score, also known as the cut score or passing score.
As an entry to practice assessment, it is important to remember that the OCE assesses competency and not excellence. To pass the OCE, candidates must achieve a cumulative score at or above the standard score. The standard score represents the benchmark of minimal competency.
Candidates who possess the minimum level of knowledge and skills necessary to perform at an entry level will pass. Candidates who perform below the threshold of competency will not be successful. Candidates who are unsuccessful in their OCE attempt will receive feedback by domain of competency. This report will indicate domains where they performed below the threshold of competency to assist in future preparations, if the candidate has exam attempts remaining.