Alberta Education has a long history of delivering provincial student assessments in a computer-based format. As far back as the 1990s, the province administered many of its assessments via desktop applications, and beginning in 2009, utilized Quest A+ to roll out online assessments^[1]. Given the limitations of the application, the decision was made to replace Quest A+ and transition to a new, modernized digital platform for its provincial student assessments^[2].

Within Alberta Education, the Provincial Assessment Sector (PAS) develops, administers and reports on large-scale student assessments in given grades from K to 12. Currently, these assessments include diploma examinations, provincial achievement tests (PATs), student learning assessments (SLAs) and early years (K-3) literacy and numeracy screening assessments. Each year, the PAS administers approximately 700,000 exams across the current provincial assessment programs. While most examinations have been developed and delivered in paper format, beginning with the 2023-2024 school year, school authorities/schools have the option to take the diploma humanities exam (constructed-response component) and PATs online^[3]. In addition, as a result of the large number of assessments, the province has had to develop extensive database systems to track all aspects of the assessments, resulting in a constellation of disconnected components which were difficult to manage and upgrade over time. Consequently, the Ministry was ready for a modernization effort to address its efficiency-related issues. Following a competitive procurement process, Alberta Education selected Vretta to be its technology partner for the modernization project.

Since the PATs are meant to provide information regarding how well students have achieved at particular stages in their schooling and assist schools, school authorities and the province in monitoring and improving student learning, this assessment program was chosen to be the focus of the current article. (A future paper will describe how data from the PATs are used to improve student learning.) The purpose of this article is to provide a high-level description of the PATs, as well as the principles and approaches taken by Alberta Education to encourage full implementation of the digital program province-wide.

About the PATs

Subjects and Grades Assessed

The PATs are administered in May and June each year to students in Grades 6 and 9. (Some Grade 9 students write the PATs in January.) The subjects assessed are: English language arts (and literature), Français/French language arts, mathematics, science and social studies. “Grade 9 PATs based on the Knowledge and Employability programs of study in English language arts, mathematics, science and social studies are also administered. French versions of all mathematics, science and social studies PATs are available for Grade 6 and Grade 9 students in Francophone and French immersion programs.”^[4] Because new curricula are being introduced, in 2023-2024, Grade 6 English language arts and literature and mathematics PATs were not administered. Instead, schools were given the option to participate in field tests of items that align with the new curriculum (in the other subjects, the Grade 6 PATs reflected the existing curriculum/courses of study) or to implement the new curricula, in which case administration of the PATs was not required.

Description of Tests

The PATs are criterion-referenced and based on learning outcomes expected for students at Grades 6 and 9. The tests comprise constructed-response items/tasks, as well as selected-response items/questions.^[5] The following tables provide information about the Grade 6 and 9 tests and their respective time allotments.^[6]

Alberta Education has provided descriptions of PATs for the school year 2021-2022. Since there are numerous tests, the following are general descriptions of three, selected as examples. (Subject area resources, including information about test blueprints, statistical analysis of selected responses, examples of student work, scoring guides and related materials can be accessed here.^[9])

Grade 6 Mathematics: The Grade 6 Mathematics test was made up of two parts. Part A comprised 15 questions, worth 10% of the final score. Of the 15 questions, there were three addition, four subtraction, four multiplication and four division questions. The format was numerical response, in which students were expected to generate an answer without the use of a calculator. Part B was made up of 40 multiple-choice questions making up 90% of the total overall score. The test questions required students to apply their understanding of math concepts within and across the four strands of Number, Patterns and Relations, Shape and Space, and Statistics and Probability.^[10]

Grade 6 English Language Arts: The Grade 6 English Language Arts test comprised two equally weighted parts. Part A (Writing) involved the writing of a narrative, as well as a functional writing component. The narrative task involved a colour picture to which students responded in narrative form. The functional writing task required students to generate a newspaper article using specific information. Part B (Reading) involved students reading informational, narrative or poetic selections and responding to 50 selected-response questions.^[11]

Grade 9 Knowledge and Employability English Language Arts: This test consisted of two parts: Part A (Writing) in which students were expected to write a persuasive essay. Students were provided with a context (situation) and a collection of reference materials for optional use. Part B (Reading) comprised 50 selected-response questions related to fiction, non-fiction, drama, poetry and visual media reading selections.^[12]

Test Administration

The PATs have been designed to be delivered digitally, and although some field tests associated with the PATs are being delivered online, many tests are currently delivered in a paper-and-pencil format. As previously mentioned, Alberta is in the process of modernizing its digital platform to enhance the provincial student assessments.

The digital PATs, delivered on the new digital platform in June, 2024, included some technology-enhanced items (approximately one-half, depending on the subject). In the first year of the modernization project, the PATs were transitioned to the e-assessment platform using a linear fixed-form delivery model. For information about the test navigation process, tools and item types associated with the online platform, visit the practice tests site^[14].

Scoring and Reporting

Most test questions are machine-scored; however, constructed responses in particular subjects are marked centrally in Edmonton by teachers (who meet specific eligibility criteria) who are nominated by their school authority. Classroom teachers are encouraged to mark students’ writing locally (using scoring guides and exemplars provided by Alberta Education) before returning test material to the Ministry. Locally awarded marks, submitted to Alberta Education, are considered the first reading of the student responses. The responses are then read for a second time; both marks contribute to the students’ final score. (In the case of discrepancy, a third reading may be required.) Student responses not scored locally are marked once centrally. The written responses, submitted via the new digital platform are marked centrally employing the built-in scoring interface. For the paper-based version of the PATs, Grade 6 and 9 teachers are expected to locally score all machine-scored items using answer keys provided by the Ministry and to report the preliminary results to students’ parents/guardians. For those who administer the assessments on the new digital platform, selected-response items are machine-scored, and teachers gain access to their students’ results via the built-in digital Teacher Dashboard.

Students’ performance is reported with reference to two standards: “Acceptable Standard” and the “Standard of Excellence.” Cut-scores are used to determine whether or not the student has performed at the given standard. To ensure that the standards are equivalent from year to year, the cut scores may need to be adjusted accordingly using equating procedures.

In the fall, Individual Student Profiles (ISPs) are made available to schools on the Provincial Approach to Student Information (PASI) system and to students and parents/guardians on myPass (an Alberta Education self-service website). In late August, detailed results are provided to schools and school authorities for their review for any errors and omissions, and multi-year summary reports are posted on the Alberta Education website in the fall.^[15]

Implementation Approach: PATs Online Testing

In transitioning to a new digital platform for delivering provincial student assessments, Alberta Education is using an incremental and measured approach that allows school authorities, schools, educators and students to learn about and interact with the new platform. This approach was adopted in recognition that some schools lack the technology (computer devices and/or internet access), and students, as well as educators, require time/opportunities to become familiar and comfortable with the online testing system. Optional implementation was originally planned for 2023-2024. “For the 2023-2024 school year, Alberta Education is in touch with school authorities directly to offer them opportunities to participate in a variety of optional implementation activities. Teachers, parents, and students can navigate the assessment platform and access practice exams and resources. Questions and feedback may be directed to online.assessment@gov.ab.ca.”^[16] The optional implementation phase has since been extended to the 2024-2025 and 2025-2026 school years, with full implementation planned for 2026-2027. It is expected that over time, more and more school authorities and schools will opt to administer provincial tests using the new digital platform. Phasing out the full implementation and allowing more time for the field to prepare has been well received.

Conclusion

Alberta Education is transitioning to a new online platform for provincial student assessments, which will mean that eventually students will write their provincial achievement tests (and diploma exams) in a digital environment instead of the paper-based format. The move to digital assessments is being undertaken in an incremental and measured way to ensure the smoothest possible transition, and this approach appears to be welcomed in the field. Alberta Education will release the results of the 2023-2024 PATs early this fall. Subsequently, the author intends to prepare a follow-up article on the use of the assessment results for improvement planning at the school authority, school and provincial levels.

About the Author

Dr. Richard Jones has extensive experience in the fields of large-scale educational assessment and program evaluation and has worked in the assessment and evaluation field for more than 35 years. Prior to founding RMJ Assessment, he held senior leadership positions with the Education Quality and Accountability Office (EQAO) in Ontario, as well as the Saskatchewan and British Columbia Ministries of Education. In these roles, he was responsible for initiatives related to student, program and curriculum evaluation; education quality indicators; school and school board improvement planning; school accreditation; and provincial, national and international testing.

Richard began his career as an educator at the elementary, secondary and post-secondary levels. Subsequently, he was a researcher and senior manager for an American-based, multinational corporation delivering consulting services in the Middle East.

Feel free to reach out to Richard “Rick” at richard.jones@rmjassessment.com (or via LinkedIn) to inquire about best practices in large-scale assessment and/or program evaluation.

Acknowledgement

The author would like to acknowledge and thank members of Alberta Education’s Provincial Assessment Sector for their valuable input to this article. For inquiries regarding the Grades 6 and 9 Alberta Provincial Achievement Testing program, please contact Provincial Assessment, Alberta Education at:

Phone: +1 (780) 427-1857; Email: exam.admin@gov.ab.ca

References

[1] Karas, D. (2010, February 22). Presentation given at the Education Quality and Accountability Office’s (EQAO) Online Testing Panel held in Toronto, Ontario.

[2] St. Albert Gazette Staff. (2017, July 8). School Notes. St. Albert Gazette. Retrieved August 3, 2024 from: https://www.stalbertgazette.com/local-news/school-notes-88-1295883.

[3] Government of Alberta. (2022). Negotiated Request for Proposals (NRFP) ,Number SA-OCCIO-SVS-09-2022, Digital Exams Service.

[4] Alberta Education. (2023). General Information Bulletin 2023-2024 Grade 6 and 9 Alberta Provincial Achievement Testing, p.2. Retrieved August 1, 2024 from: https://alberta.ca/system/files/custom-downloaded-images/edc-pat-general-information-bulletin.pdf.

[5] For the purposes of this article, the term “selected-response” item/question is used to categorize those that require test-takers to choose the most appropriate/best answer(s) from a list provided (single- and multi-selection item formats, as well as items such as drag-and-drop, drop-down-menu, hot spot and ordering formats.

[6] These tables have been modified from Alberta Education. (2023). General Information Bulletin 2023-2024 Grade 6 and 9 Alberta Provincial Achievement Testing, p.4. Retrieved August 1, 2024 from: https://alberta.ca/system/files/custom-downloaded-images/edc-pat-general-information-bulletin.pdf.

[7] Students may use up to twice the officially allotted time.

[8] Students may use up to twice the officially allotted time.

[9] Subject area resources: https://www.alberta.ca/provincial-achievement-tests#jumplinks-5

[10] Alberta Education. (2022). Grade 6 Assessment Highlights: Mathematics (2021-2022).Retrieved August 11, 2024 from: https://www.alberta.ca/system/files/custom_downloaded_images/educ-mathematics-6-assessment-highlights-2022.pdf

[11] Alberta Education. (2022). Grade 6 Assessment Highlights: English Language Arts, 2021-2022. Retrieved August 11, 2024 from: https://www.alberta.ca/system/files/custom_downloaded_images/educ-ela-6-assessment-highlights-2022.pdf

[12] Alberta Education. (2023). Grade 9 Assessment Highlights: Knowledge and Employability English Language Arts, 2021-2022. Retrieved August 11, 2024 from: https://www.alberta.ca/system/files/custom_downloaded_images/educ-ke-ela-9-assessment-highlights-2022.pdf

[13] Calgary Board of Education. (2024). Provincial Assessments & Reports. Retrieved August 6, 2024 from: https://cbe.ab.ca/about-us/provincial-tests-and-reports/Pages/default.aspx.

[14] Practice Tests: https://abed.vretta.com/#/en/public-practice

[15] Information in this section of the article was accessed from: Alberta Education. (2023). General Information Bulletin 2023-2024 Grade 6 and 9 Alberta Provincial Achievement Testing, pp. 41-45. Retrieved August 1, 2024 from: https://alberta.ca/system/files/custom-downloaded-images/edc-pat-general-information-bulletin.pdf.

[16] Alberta Education. (2024). Provincial Achievement Tests. Retrieved August 1, 2024 from: https://alberta.ca/provincial-achievement-tests.

---