Who needs the MTC tests?

Once a year, the results of the Mitzvah exams (indicators of school efficiency and growth) are published by the Ministry of Education and cause a public uproar over the low achievements of Israeli students in these exams. * One university - different opinions

The article is taken from the Hebrew University's "Ko Panis" journal.

The placement tests examine the achievements of students in elementary schools and middle schools in relation to the requirements of the curriculum. The exams, conducted by RAMA, a national authority for measurement and evaluation in the Ministry of Education, examine one grade in elementary school (XNUMXth) and one grade in middle school (XNUMXth) once every four years in four areas of knowledge: mathematics, English, science, mother tongue - Hebrew or Arabic. The exams are accompanied by questionnaires on school climate, pedagogical environment and more. Recently, for the benefit of managers who want it, internal placement exams have been introduced every year in the schools in the same grade levels.

According to Ramah, the results of the tests are intended to "serve as a central working tool for the school principal and his staff to plan resources and utilize them", that is, a tool for monitoring the educational process, for determining the pedagogical policy and for the allocation of annual resources. The tests are also used as a tool for the management of the Ministry of Education to determine a systemic pedagogical policy and allocate resources. RAMA defines the exam as "one of the main tools supposed to help the school principal" and emphasizes the importance of combining internal and external evaluation, alongside variables such as school climate and pedagogical environment.

We turned to Prof. Sorel Kahn from the School of Education who tested the effectiveness of the placement tests in their current format according to the goals set for them by the Ministry of Education and published a position paper not long ago and to Prof. Anat Zohar from the School of Education, who serves as the chair of the Pedagogical Secretariat in the Ministry of Education.

Prof. Kahn, what did you ask to be tested in the MTC tests?

"The Mitzvah tests are an initiative of the Ministry of Education that has no equal in the international landscape. Their mission is to provide the schools, which until now have relied only on internal sources for evaluating student achievements, with better, uniform and "objective" data, for the purpose of making internal school decisions, including achievement monitoring and differential allocation of resources within the school.

The purpose of my work was to examine whether the current format of the tests, which have logistical, budgetary and psychometric aspects, can achieve the purpose for which they were founded. I set some necessary conditions for the success of the tests and the achievement of the goals, and I checked whether these conditions are met."

What are the required conditions?

"The first condition is that the school receives full, comprehensive and real-time information. That is, the exam must include all students in all classes, in all subjects, or in most of them, at least once a year.

Is this condition met? This condition is not met. Each year only one grade level is tested in the elementary school (XNUMXth) and one grade in the middle school (XNUMXth), and recently a reading test was also introduced in the second grade.

That is, the school principal has no information over the years regarding the achievements of the students studying in all the other grade levels. Furthermore, the only class rank in each school included in the Mitzav is evaluated by an external examination (supervised by the Ministry of Education) only once every four years, and the results reach the school faculty only in the following academic year, which prevents their use for making decisions in planning for the year. It is quite clear that without current and available real-time information regarding the achievements of all grade levels, classes and students in the entire school, it is not possible to use the results of the MEC tests for the purpose of monitoring achievements, making pedagogical decisions and allocating resources in an appropriate manner."

And what are your conclusions?

"The device as it is implemented does not fulfill its role. No administrator can monitor the achievements of the school's students if he only has at his disposal the results of the achievements of one class, which is tested once every four years, and differentially allocate resources accordingly. In actual practice, the Institute provides only four-five percent of the amount of information needed to achieve the goals, and the conclusion is that based on this partial and minimal information, it is not possible to monitor the students' achievements and make decisions."

It is clear that testing everyone, in all subjects, every year is a difficult requirement to implement.
"It's a heavy logistical effort that requires a huge budget, and that's why it doesn't exist in any other education system. There is an attempt here to do something grandiose, that a school will not be run according to the information in its possession but according to external tests which currently cost tens of millions; Doing what is really necessary requires a legendary budget - it is not realistic and indeed it does not exist. Furthermore, even if the tests were conducted in the required scope and frequency, it would still not be possible to use them to achieve the objectives, and this for reasons Psychometrics. The tests are designed to measure the students' achievements in relation to the curriculum requirements. For this, two conditions must be met: (1) the grades in each test will express the degree of mastery of the material included in the curriculum; (2) The grades should be comparable between grade levels and subjects".

Are these conditions met?

"Both conditions are not met. There is a general agreement in the theory of tests that the grades received in the achievement tests cannot be validly interpreted in terms of a percentage of the knowledge required by the curriculum (among other things due to being affected to an unknown extent by the arbitrary degree of difficulty of the particular test) and are not comparable between different subjects and between grade levels in the same subject . This agreement is also explicitly expressed in the instructions given by the Ministry of Education regarding the interpretation of the scores of the Mitzvah tests.

According to this, does the score of 44.1 in XNUMXth grade mathematics not reflect anything?

"Right. It is possible that the students are not 'failing', and it is also possible that they are 'failing' more than it seems. There is a demand here for an absolute, objective and scientific measurement of knowledge, which is impossible. Absolute measurement does not exist in any other scientific discipline."

In your opinion, is the MOT test unnecessary?

"In my opinion, yes. He may be achieving other undeclared goals. Conducting the tests in the current format is not only unnecessary, but also harmful, since the results are misleading."

If so, what do you suggest?

"There is no escape from basing decision-making processes on internal school achievement evaluation," says Prof. Kahn. "In any case, the important decisions are related to the teacher's teaching strategies and should be made by each teacher, during each lesson and beyond the lessons. It is the teacher who decides how to progress in the material based on the feedback he receives during the lesson. This is a measurement that is done by the teacher only and it is not possible for it to be done in an external measurement. That is, the decision-making processes in teaching and learning are based on subjective judgments and evaluations of the teachers and there is nothing wrong with that. Because unlike objective tests, whose scores cannot be interpreted in absolute terms, teachers can make subjective evaluations (not enough, good, etc.). These evaluations can also serve as a basis for the decision-making processes at the school level."

What can be obtained from the installation anyway?

"The Institute can provide important information for the Ministry of Education's systemic decisions if it focuses on inputs and not outputs, and in particular on monitoring the supply of teaching in classrooms as a measure of the need for additional resources."

Why is teaching provider monitoring the only effective feature in your opinion?

"The high level of heterogeneity in the Israeli education system in terms of the personal, family, environmental and school factors that influence student achievement reflects on the capacity in the various classes, a reality that only the teacher in the field can feel today. It is impossible to demand similar test results where there is marked inequality between schools and classes without massive compensatory intervention.

The goals of the system today are "universal" without justification. Differences between students and schools in learning opportunities are ignored here. The education system must formulate a plan that takes into account the differences between students, between classes and between schools.

"For all these reasons, I suggest conducting a monitoring of the teaching before turning to monitoring the achievements - this can be done at low cost, twice a year in each school and in each class. An examination of this kind will reveal, most likely, that the main reason for low achievements is only partial coverage of the study material."

Prof. Zohar, what do you think of Prof. Kahn's position regarding the usefulness of the Mitzvah tests?

"I agree that the main mission of the Mitzvah is to serve as a feedback tool for school administrators to improve teaching and learning. The MICEV is supposed to provide school administrators with more reliable data than they can get from tests administered by school teachers. It is not intended to be a tool of the system to supervise the schools," says Prof. Anat Zohar. "Prof. Kahn's position that the test should include all students in every school, in all classes, in all subjects at least once a year, is unreasonable because it ignores the damage that may be caused to the schools as a result of such an intensive set of tests. Systemic tests also have negative sides: since it is not possible to test all the goals of the curriculum and of course it is not possible to test all the educational-social goals of the school, the tests will cause the goals that can be easily tested to be emphasized and other important goals (academic, social and value) to be neglected. Moreover, 'high-risk' tests cause discrimination against weak populations, lower the internal motivation for learning, increase an atmosphere of stress and anxiety, and encourage various types of cheating."

Therefore, the education system is looking for the 'golden path': to extract important information from the tests on the one hand and to minimize the potential damages resulting from multiple tests on the other. The result is the Mitzvah test in its current format. Contrary to Prof. Kahn, I do not believe that the grades should be comparable between grade levels and between the various subjects - such a comparison is not the purpose of the tests. If the tests are reliable and calibrated, the system can produce important information even without such a comparison. The internal assessment gives school administrators quality assessment tools and promotes a culture of assessment within the school, while the external assessment provides information on the level of achievements relative to the national norm."

Prof. Zohar also says, "I agree with Prof. Kahn that it's a shame that the grades come a few months after the exams, but in an exam of this scope, the data cannot be made up faster." Prof. Zohar agrees that there is room to check which part of the curriculum is taught in classes, but the information should be treated with care and integrated into a broader data set. "Covering the material" does not guarantee knowledge or significant learning of the studied material.

In conclusion, Prof. Zohar says, "You have to remember that the main goal is to improve teaching and learning, and for that, a different perspective than the usual psychometric perspective is needed. The system does need to examine, but also be careful not to "examine itself to know".