SAM can be used at school level as a monitoring of educational process. It also can be used in local educational system that includes managing authorities, research and training institutions.
Understanding of the fact, that existing tests provide information on learning results that is needed for decision making and improvement only partly, became an impulse for SAM development.
SAM model is based on Vygotsky’s theory.
Applicability
Theoretical basis
Level model
Test design
Applicability
Fig.1.1. Results of test on metric scale
The main result of pedagogic testing is a scale that indicates scores received by different students or student groups. Same scale can indicate data on how successful test tasks were solved. It provides an opportunity for quantitative comparison of learning achievements of students and finding relative measure of curriculum assimilation which allows management decisions to be made.
The importance of quantitative comparison of students learning achievements is practically assured. However, for teachers the data on curriculum assimilation is traditionally more interesting.
Tools for comparison and assessment of the quality side of the learning results are being developed for more than half century in pedagogic science. These tools are special hierarchy schemes that reflect levels of forming of the cultured capacities or in other words taxonomies of pedagogical goals.
The most famous taxonomy that commonly believed to be “historical” and fundamental is created and published in the USA by Bendgamin Bloom in 1957. Other versions were also developed (table 1.1) By now there are at least 15 of taxonomies that kind of similar but still have some differences.
| B. Bloom | V. Simonov | I. Lerner | TIMMS (Math) |
| distinction | |||
| knowing | memorization | knowing | knowing the facts and procedures |
| understanding | understanding | ||
| application | application | application | |
| The simplest skills | |||
| analysis | |||
| synthesis | transfer | creative application | reasoning |
Developing of taxonomies is explained by intention of educators to disclose underlying logic of educational process. Each taxonomy is a theoretical model (concept, version, understanding) of this process that proposes it’s milestones and relevant achievements. These taxonomy models orientate teacher in his/her work, so it’s desirable for tests to classify received educational results in the frames of accepted taxonomy model (level scheme). The intention of Russian research group to fulfil such social-pedagogical need led SAM development.
Theoretical basis
To start test development it was necessary to determine the model of the educational process so the Vogitskiy’s sociocultural theory of cognitive growth was chosen as one. The core of the theory is a scheme of knowledge acquisition which was later finalized by Vigotskiy’s followers.
The short summary of the theory as follows.
The acquisition of symbolic systems is a necessary condition of the normal mental development of a child. Full-fledge cultural model (experience, knowledge) in substance is a system of signs that set some generalized mode of action. For instance, in schools mathematical mode of action is presented via concepts, principles, ,schemes, mathematical tasks, solving algorithms, etc. This set of symbolic structures is being transmitted to a child within school learning process. For instance, a teacher explains and shows how and by what means certain mathematical operations are done. Then teacher works to ensure that child reproduced the main conclusions and solved routine tasks.
But according to Vigotskiy, transmission of the cultural model to a child marks only a beginning of the educational process. In future the student should really assimilate this knowledge as a system, so to have this generalized mode of action to be able to solve all types of the tasks corresponding to this mode. The process of assimilating of cultural content has a hidden, long-term and spontaneous nature. The seeds of knowledge, sown in the pupil’s minds, grow very gradually in each of them into the ability to act reasonably, i.e. a subject competence. Therefore, Vygotsky defined the process of assimilating of action modes as the functional development. According to his theory, the role of learning (transmitting of cultural models) is to found the pupils’ zone of proximal development – the possibility of the formation of a certain cultural abilities. This possibility that sometimes is not realized, and then the knowledge gained lies dormant in memory.
Vygotsky also believed that acquiring of the mode of action passes three levels. Further studies allowed to clarify what kind of levels and why there are three of them. The important step was made by P. Galperin, who showed that action mode is determined by the content of the orientation. And mastering of the action mode is essentially the assimilation of an appropriate system of orientation points. Further, owing to the work of B. El’konin, V. Davydov, A. Zaporozhets, and their co-workers there were three necessary ingredients in the orientation of action mode identified:
1) empirical – external characteristics of the situation and appropriate action;
2) theoretical – the concept of significant relation, which provides the basis for action mode in a given situation;
3) functional – understanding of the field of possibilities of action mode with its limits.
Fig. 1.2. Level-based model of knowledge acquisition
These three types of orientation points are transferred to a child as a cultural model. But during its assimilation the role of a real support for the action for the first time is played by external characteristics. After that they are joined by the notion of a significant relation, and finally – an appropriate functional field. These three types of orientation mark three levels of acquisition of the cultural mode of action, which briefly can be defined as procedural knowledge, conceptual understanding and functional competence (Figure 1.2).
It is important to note that each type of orientation is realized through its psychological mechanism which causes a tangible distinctions between them. Thus, the orientation of the first level is based on the direct associative connections. In the language of the teacher, this means rote exterior features standard applications and algorithms for their solution. The orientation of the second level is based on the mental structure that captures the essential relation of situation. In psychology, such structures are termed as “gestalt” and in such cases teachers say, that the child began to understand the subject content, and has learnt to extract significant out of the conditions of the problem. Finally, the orientation of the third level is based on a complex structure, we denote it by the term “functional field”, which keeps the area of possibilities of this action mode, with its limits. About the student, which demonstrates this level, the teacher can say that he is fluent at subject content, i.e. he can apply it flexibly, intelligently, in accordance with the situation. The described multi-level pattern was taken as a basis for creating SAM tests.
Level model
Proficiency levels of the mode of action were put into correspondence with indicators, i.e. with types of tasks.
First level (procedural knowledge)
Fig.1.3. Action orientation at procedural level
The generic criterion of achieving this level is the ability to act being guided by external parameters of the problem situation and pattern of operation. For example, recognize a problem as referring to a certain class (type) based on particular attributes, and perform a relevant procedure presented as a standard operational scheme or rule; or construct a sequence of operations directly following the statement of problem. Assimilation of the pattern of operation at the first level is demonstrated by the ability to solve problems whose description either clearly enables to refer them to a certain class with a well-developed solution procedure (standard problems) or directly guides the student to the correct scheme of operation, i.e., contains straightforward prompting messages. In such problems, conceptual understanding required for solution is linked to external parameters of the situation and needs no compulsory statement (identification). Therefore, at the given level of pattern assimilation, the problem is solved in a purely external context – through direct correlation of the problem description with the description of relevant action as a sequence of operations (fig.1.3).
Second level (conceptual understanding)
Fig.1.4. Action orientation at conceptual level
The generic criterion of achieving this level is the ability to act on the basis of substantial analysis of the problem situation, i.e., to develop a conceptual understanding helping to find a principal way to solution. Indicator of the achievement of this level is ability to solve problems for which no ready-made (standard) patterns of operation can be directly found and applied without conceptual understanding (i.e., without understanding the substance of the objective situation).
In particular, this requirement is met in:
- problems where description of conditions hinders their classification;
- problems where the application of a standard pattern requires transformation of conditions;
- problems described in an abstract form that excludes solution through manipulation of explicit data;
- problems suggesting reversion of standard patters of operation (movement from the outcome to conditions), etc.
In such problems, solution cannot be found through a direct matching of conditions and ready-made operating pattern but is sought through conceptual understanding, i.e., is based on the interpretation (“understanding”) of the situation (Figure 1.4)
One can say that finding a solution requires correlation and coordination of two contexts: external (description of the problem situation and pattern of operation) and internal (conceptual understanding of the problem and approach to solution). In this context, the “external” and “internal” should be understood in the logical sense.
Third level (functional competence)
Fig.1.5. Action orientation at functional level
The generic criterion of achieving this level is the ability to orientate in the domain of possible ways of implementing the general pattern, see its limits, and go beyond them if necessary
Indicator of the achievement of this level is is demonstrated by the ability to solve problems:
- reconstruction and probing ways of operation to select the one meeting the specified criterion;
- analysis of specified patterns (formulas) of operation to select the adequate one
- assuming detection of the limits of the main pattern and use of additional resources
- where the main pattern acts as a means to support a higher level operation
- assuming the need to adapt the pattern of operation to specific features of the object
Test design
The above typology of problems serves as guidance for designing the blocks of test items. Each block includes level 1, 2 and 3 items referring to one and the same curriculum area (level is determined by the most difficult task that was solved by the student). Complex of the blocks that covers main parts of the curriculum allows to see structural picture of curriculum assimilation.
As an illustration, below is a block of math items.
1. What number will be obtained if 10472 is divided by 34?
Answer: __________
2. Absentminded Peter copied an exercise on multiplication of two numbers from the textbook. He wrote down the first multiplier correctly: 7. But in the second multiplier, he displaced two figures. As a result, he got 147.
What answer should Peter have got if he had copied the exercise correctly?
Answer: __________
3. What is the largest result that can be obtained if letters in the expression AB5 + BС2 are substituted with figures (different letters should be replaced with different figures)?
Answer: __________
Note. All items refer to the content area Numbers and Calculations. The first item suggests simple application of the rule (algorithm) of calculation. The second item requires analysis of the wrong arithmetic operation (with due regard for positional principle) and designing a plan for its correction. And finally, the third item assumes “playing” with the positional principle to obtain a specific value that meets the condition needed to get the largest value.
Test versions are compiled of such blocks consisting of items from the main subject content areas. As a result, the test has two functions: a) measurement of integral educational success (45 test items, covering the main sections of the subject) and b) the diagnosis of the content assimilation (15 blocks). Accordingly, the test can be viewed as a system of three subtests with each representing a set of items of the same level referring to different content areas.
Fig.1.6. Test design