1 2 POLICY DOCUMENT CREATING A LEARNING ASSESSMENT UNIT IN THE MINISTRY OF EDUCATION OF GUYANA 26 September 2023 MISSION AND PRINCIPLES 1. A Learning Assessment Unit is created within the Ministry of Education. 2. The mission of the Learning Assessment Unit is to support student learning by collecting evidence on their competencies, knowledge, skills and well-being and by sharing this information with stakeholders so that they can take action to improve education quality and equity. 3. The Learning Assessment Unit is guided by the following principles: - Student at the centre. Assessments should be designed and implemented, putting students at the centre, that is, promoting their learning and development and recognizing their diverse learning needs. - Quality and equity in education. Assessments are essential to monitor the quality and equity of education. Informing about the degree to which students are reaching learning standards (quality) and the differences in performance among groups (boys/girls; hinterland/coastal) should, therefore, be a priority in any assessment activity. - Technical quality. Assessment results must be credible, reliable, fair and valid for their intended purposes, interpretations and uses. Transparency, public trust, and participation must guide all assessment activities. - Being at the service of Guyana’s educators. Assessment results and information is produced keeping in mind the need of students, parents, teachers, principals, supervisors, central level authorities, and teacher training institutions. What type of evidence do they need to make a decision or take action? As much as possible, the unit should provide training and promote participation of stakeholders so that they can take ownership of the assessments, can appreciate it and understand it, and can make use of it to improve. - Assessment for learning. Assessments must support and enhance learning. They must contribute to curriculum implementation. They must promote a better fit between pedagogy and the actual learning levels of the students, thus making teaching more relevant and pertinent to learners. - Utility. Assessments are not an end in themselves. Assessments only make sense if they are useful to support student learning. Assessments must be used to inform policy and practice, providing relevant and on-time evidence for key stakeholders, including teachers, school principals, supervisors, central-level authorities, parents and students. 3 4. The missions and principles of the Learning Assessment Unit endorse the 2030 vision of the Education Sector Plan (ESP) and of Guyana’s Curriculum Framework. The ESP aims to improve governance, accountability and efficiency of the education system. 5. The missions and principles of the unit are aligned with the Curriculum Framework. The framework states that assessments must be aligned with learning outcomes stated in the curriculum; must be inclusive, recognising the diverse learning needs of students; must be equitable, reliable and valid for its intended purposes and uses; and must be used to make evidence-based decisions that inform and guide teaching and learning. SCOPE, RESPONSABILITIES, FUNCTIONS Scope and responsibilities 1. The Learning Assessment Unit is responsible for measuring and supporting learning in nursery (pre-school), primary and secondary education, in both the public and private sector, in the whole country territory. 2. Its scope of responsibilities encompasses five main learning assessment types or programs: a. Classroom assessments. Refers to the assessments that are done by the teachers on a daily basis in their classrooms in order to gauge information on where students are in their understanding and to use that information to adjust teaching/pedagogy. Classroom assessment can take many forms: It can be formal or informal, summative or formative, and can have or not grades associated with it. Examples of classroom assessments include tests, quizzes, projects, homework; it also includes when teachers ask their students to raise their hand to answer a question, or when students are asked to do self-evaluations or peer-evaluations. While teachers usually have lot of discretion to plan classroom assessments, they also need to ensure alignment with the official assessment policies and guidelines. b. Diagnostic assessments. Refers to assessments designed by the Ministry of Education and distributed to the schools so that teachers can administer them to their students, score and analyze their results, and use their findings to advice policy and improve teaching/pedagogy, with a focus on remedial interventions. These assessments were introduced by Ministry of Education in literacy and numeracy in all schools at grades 1-6 right after the pandemic (2021). c. International assessments. Refers to regional or global cross-national assessments used to monitor learning at the country level and to compare the relative performance of countries. Examples of such assessments include the Latin American Laboratory for Evaluation of the Quality of Education (LLECE), the Programme for International Student Assessment (PISA), Trends in International Mathematics and Science (TIMSS), Progress in International Reading Literacy (PIRL), and Literacy and Numeracy Assessment (LANA). These assessments usually administer language, mathematics and science tests to nationally representative samples of students. Questionnaires are also used to collect data on student attitudes and well-being, the school and home contexts in which they learn, and teaching practices, among others. The assessments are externally administered and are fully standardised. 4 d. Public Examinations. Refers to evaluations used for selection and certification of individual students at the primary, secondary and tertiary education levels. In Guyana, these include: o National Grade Six Assessment (NGSA) – Primary education o National Grade Nine Assessment (NGNA) – Secondary education o Caribbean Secondary Education Certificate (CSEC) – Secondary education o Caribbean Advanced Proficiency Examination (CAPE) – Secondary education e. Teacher Tertiary Level Certificate. Refers to the examinations future teachers need to pass in order to work in a school in Guyana. 3. The Learning Assessment Unit provides leadership to implement these assessments or other learning assessments according to quality standards. For doing so, it produces manuals, protocols and guidelines, and provides/facilitates training for teachers and other stakeholders. 4. The Learning Assessment Unit has the primary responsibility of ensuring that the assessments are aligned with the official and implemented curriculum and that they are conceptually coherent and make synergies with the others. The unit must make all efforts to avoid duplications and saturations of assessment information, hence maximizing the utility of each assessment type. 5. The Learning Assessment Unit prioritises the communication of assessment results and information to key stakeholders, ensuring that they can understand and use results to inform policy and practice. Functions The Learning Assessment Unit performs the following functions: 1. Monitoring and supporting the development of mastery skills and competencies in children at the nursery level. 2. Monitoring and supporting the development of functional numeracy and literacy in learners at the primary and secondary levels. All efforts must be made to ensure that learners obtain their graduation certificate at the end of the secondary level. 3. Reporting to different stakeholders (e.g., students, parents, school principals, media) on the extent to which students are reaching learning standards in key domains of the national curriculum, with a focus on quality (i.e., percent reaching learning standards) and equity (i.e., inequalities among groups). The unit is responsible for publishing national or regional result reports and for providing guidelines on school report cards. 4. Provide teachers and other stakeholders with assessment resources (e.g., guidelines, tools, item banks, training), procedures and support to improve teaching and learning practices. 5. Building an assessment culture where stakeholders can understand, appreciate, and take action to improve education quality and equity. The unit promotes the implementation of quality assessments in the classrooms and the use of assessment results. It engages with stakeholders, opening spaces for participation, training, sharing of information, and collecting feedback to improve. 5 6. Advising the Minister of Education and other authorities about education policy and practices. 7. Supporting the identification and allocation of resources for students who are lagging in their learning and may need more differentiated instructional support, including remedial education, tutoring or others. 8. Coordinate with other units of the Ministry of Education in order to inform policy and practice, ensure coherence with other initiatives, and alignment with the official and implemented curriculum. INSTITUTIONAL ARRANGEMENTS Organizational structure 1. The Learning Assessment Unit is accountable to the Chief Education Officer (CEO) and is located directly under its office in the organigram. 2. The Education System Committee serves as the governing board or steering committee of the Learning Assessment Unit. The committee has the responsibility of approving and overseeing the implementation of: (a) The Learning Assessment Strategic Plan, (b) The assessment calendar, and (c) The annual working plan of the unit. 3. The organizational structure of the Assessment Unit includes five teams. Each team has responsibilities on all different assessment types/programs (Classroom Assessments, Diagnostic Assessments, Examinations, International Assessments, and Teacher Tertiary Level Certificate) and each team specializes on a different part of the assessment cycle. The teams are: (a) Management; (b) Instrument Development; (c) Field Operations and Logistics; (d) Data processing; and (e) Communication (see organizational structure in Annex A). The responsibilities of each team are: a. The Management Team gives general directions and leadership. Ensures alignment of the assessments with the national curriculum, coherence among the different assessment types, and coordination with other units of the Ministry of Education. Must ensure that the assessments address key policy questions, are technically sound, are understood and used by stakeholders, and are carried out on time and within budget. The Management team is responsible of preparing and implementing the Strategic Assessment Plan and the annual working plan. b. The Assessment Instruments Team is in charge of designing, adapting and/or producing assessment guidelines, tools, frameworks, tests, scoring rubrics, questionnaires, protocols, manuals or others, in order to collect evidence on learning (e.g., mathematics, reading). Main responsibilities include recruiting, training, and supporting item writers, scoring open-ended questions; and producing the instruments to be printed. The team works in close connection with 6 the Curriculum Unit of the Ministry of Education in order to ensure alignment of assessments with official learning objectives. c. The Field Operation and Logistics Team is responsible for sampling schools and students; producing guidelines, manuals, protocols, videos and/or training for administering the assessments; recruiting and training test administrators, supervisors, regional or district level coordinators and quality control observers; liaising with the schools. The team also has responsibilities in all the logistics surrounding the assessments (e.g., distribution and recollection of instruments). This team will work closely with the decentralized offices of the Ministry of Education to ensure assessment implementation in each region. d. The Data Processing Team is responsible for capturing and cleaning the data, validating databases, computing scores, learning levels, and assessment results for the different assessment types. The team produces results and databases according to the requirement of the different communication products. Responsible for documenting technical procedures. e. The Communication Team is responsible for putting in place and implementing an effective communication plan for each assessment type or program. The team works closely with the Management team to identify key policy questions and shape messages. It also works with the Assessment Instruments Team to design instruments that answer key policy questions, and with the Data Processing Team to produce results that are meaningful and pertinent. 4. The organizational structure of the Learning Assessment Unit is based on a matrix design, where each team will serve the needs of the different assessment programs/types. For instance, the Assessment Instrument Team is in charge of designing tests for the examinations, Diagnostic Assessments and the Teacher Tertiary Level Certificate, of adapting/translating tests from international assessments, and of providing guidelines and resources for teachers on classroom assessments. The Data Processing Team is in charge of capturing, cleaning, analyzing and reporting results for the different types of assessments (with the exception of classroom assessments that are more decentralized and qualitative in nature), and so on. Within each team, the personnel may specialize in different tasks as needed. This matrix structure allows for greater harmonization, alignment, coherence, and efficiency in the implementation of the different learning assessments. This contrast with the traditional approach of having different teams working for different assessment programs/types (e.g., one Assessment Instrument Team for the examinations, another for the Diagnostic Assessment, another for Teacher Tertiary Level Certification, etc.). Doing so requires hiring more specialized personnel, which is scarce and expensive; more financial and physical resources, and carries the risk of greater inefficiencies and misalignments. 5. The Learning Assessment Unit has an Administrative and Production team. This team is in charge of coordinating administrative procedures with the main Administration 7 Office of the Ministry of Education, and of managing the printing and production of assessment instruments, including tests, questionnaires, manuals, forms and others. 6. The Assessment Unit relies on other currently existing units within the Ministry of Education to secure the necessary physical and technological infrastructure, and to recruit the personnel needed to operate on a regular basis. 7. Changes to the organizational structure of the Learning Assessment Unit must be approved by the Education System Committee. 8 Relationships with other units 1. The Learning Assessment Unit has a functional relationship with other units within and outside the Ministry of Education in order to accomplish its mission. 2. Withing the Ministry of Education it engages with: - The Curriculum Unit in order to ensure the alignment of the assessments with the official and implemented curriculum; define learning levels; define test specifications; and inform revisions to the curriculum, among others. - The decentralized offices of the Ministry of Education to ensure the implementation of assessment activities in all the regions. Decentralized offices have main responsibilities in contacting schools, recruiting temporary personnel, organize and provide workshops/training, among others. - The National Centre for Education Resource Development (NCERD) to provide inputs for in-service teacher training on learning assessments and pedagogical approaches to respond to the learning needs of the students. - The Planning Unit to get inputs about policy questions and to inform policy decision making, among others. - The Quality Assurance unit to get advice on policy questions, and to inform about learning outcomes, among others. - The Education Management Information System (EMIS), to inform about learning outcomes, and to access data from schools, among others. - The Monitoring, Evaluation and Technical Support (METS) unit, to gain insight on the implemented curriculum and teaching practices, and to inform about the attained curriculum, among others. 3. Beyond the Ministry of Education, the Learning Assessment Unit will engage with The Cyril Potter College of Education (CPCE) and the University of Guyana (UG), to inform about assessment results and the implication for training of future and in-service teachers, and to get advice on assessment design and implementation. Personnel 1. The personnel of the Learning Assessment Unit is appointed following established procedures of the Ministry of Education, including: - Appointments through public service commission, using the standard salary scale of public servants. - Appointments by contracts in the case of highly specialized skills not easy found in the country. 2. A senior education officer leads each team of the Learning Assessment Unit. 3. Each team has at least the minimum number of qualified personnel needed to accomplish its functions. See Annex B. 4. The Ministry of Education makes all necessary efforts to provide opportunities for professional development and hands-on training to the personnel of the Learning Assessment Unit. 9 REFERENCES Clarke, M., and Luna-Bazaldua, D. 2021. Primer on Large-Scale Assessments of Educational Achievement. National Assessments of Educational Achievement series. Washington, DC: World Bank. (download here) Ministry of Education. 2018. Curriculum Framework (Draft). Unpublished document. Ministry of Education. Education Sector Plan 2021-2025: Eliminating illiteracy, enhancing tolerance and modernizing education. Ramírez, 2018. How can countries monitor learning at the national level? Background paper for the 2018 Education Digest. Montreal: UNESCO-UIS. UNESCO-UIS. 2017. Quick Guide No. 3: Implementing a National Learning Assessment. Montreal, Canada: UNESCO-UIS. (download here) https://openknowledge.worldbank.org/server/api/core/bitstreams/6ca06ae4-dcf6-5c13-b6ed-04d3dea9ece9/content http://uis.unesco.org/sites/default/files/documents/quick-guide-3-implementing-national-learning-assessment.pdf 10 ANNEX A: Organizational Structure of the Learning Assessment Unit Adapted from: Ramírez, 2018. How can countries monitor learning at the national level? Background paper for the 2018 Education Digest. Montreal: UNESCO-UIS. 11 ANNEX B: Minimum personnel needed in each team and functions/profile Management Team (minimum full-time personnel: 3) - 1 full-time manager with strong leading skills, who can plan and deliver on time and on-budget. Understands both the political and technical dimensions of learning assessments, and can liaise with the other units of the Ministry of Education, supervisors, schools, teachers and the media. - 1 full-time assistant who can plan, coordinate and supervise the work of the teams within the unit. - 1 full-time coordinator for international assessments, responsible for implementing the assessment in the country according to international standards. Instrument Development Team (minimum personnel: 7) - 1 full-time coordinator with capacity to lead and organize a team, to plan and deliver on time. Must have knowledge of assessment instruments and school curriculum. - 1 full-time person for classroom assessment. Provides general guidelines, policies, tools, and training for teacher to improve classroom assessment practices for nursery/pre-school, primary and secondary. - 1 full-time specialist in nursery/pre-school development, curriculum - 1 full-time language/English/curriculum specialist in primary education - 1 full-time school mathematics/curriculum specialist in primary education - 1 full-time school language/English/curriculum specialist in secondary education - 1 full-time school mathematics/curriculum specialist in secondary education Field Operation and Logistics Team (minimum full-time personnel: 3) - 1 full-time coordinator, with capacity to lead and organize a team, to plan complex logistics operations, to train permanent and temporary staff. Must have strong relationships with decentralized structures of Ministry of Education both at the regional level and at the school level. - 2 full-time persons responsible of liaising with schools, plan visits, provide trainings, organize and supervise administrations, and data collection. - A contingent of temporary hired personnel to work as test administrators, regional or district level coordinators, and quality observers. Data Processing Team (minimum full-time personnel: 3) - 1 full-time coordinator, with capacity to lead and organize a team, and to respond to the information needs of other teams. Must have understanding of data processing, as well as capacity to produce results and databases to fill result reports and others. 12 - 2 full-time data analysts, who can draw samples of schools and students; prepare databases (capture data, cleaning, validation); perform psychometric analyses; and do basic and advanced statistics analyses to compute results. Must have expertise in the use of statistical packages (e.g., SPSS, STATA, SAS). Communication Team (minimum full-time personnel: 4) - 1 full-time coordinator, with capacity to lead, organize, and of translating technical data into meaningful information for stakeholders. Must have an understanding of public communication and of quantitative data. Must have capacity to speak with the authorities of the Ministry of Education, and the media. Responsible of designing and implementing the communication plan; and of providing guidelines for the design of assessment instruments. The coordinator is responsible for producing and disseminating products such as the assessment website, school reports, flyers, videos, and organizing meetings, seminars and workshops with stakeholders. - 2 full-time persons who can speak to teachers and other the target audiences, who can write and produce communication products (e.g., videos, reports, flyers, workshops) from the perspective of the end users. - 1 full-time graphic and online designer in charge of assessment products (e.g., website, reports, apps) that are visually appealing and effective to communicate results and other information.