Objectives

Whereas the amount of teaching material in universities has changed significantly in recent decades, the structure and methods of instruction have largely remained unchanged. Between
1981 and 1986 we carried out an experiment in Hungarian higher educational institutions, to examine the effectiveness of mastery learning strategies in higher education. The objectives of the experiment were to develop optimal versions of instructional strategies, to devise the necessary evaluation methods and tools, and to measure the effectiveness of these strategies.

Theoretical framework

Several concepts, e.g. Bloom's "mastery learning" and Keller's "personalized system of instruction", were integrated into a model, using the cybernetic and system-theoretical approach to the instruction. These models have some well-defined elements and parameters (e.g. pre- and post-tests, compensatory methods, the length of the learning task, the number of possible repetitions of failed tests, and the criterion of mastery) and the concrete variations used in the experiment differed in the combination of these elements and the values of the parameters. The semesters were divided into learning units. The number of learning units per semester varied from three to ten. The general scheme of a learning unit is displayed in Figure 1 (Chapter 1). Each learning unit involved a pre- and a post-testing (and if necessary a compensatory and retesting) period. The criterion of mastery ranged from 60 to 90%. The number of possible repetitions of failed tests was between two and four.

Methods and techniques

The experiment was conducted in four agricultural colleges. 19 different courses were involved from the disciplines of mathematics, computer-science, physics, chemistry and industrial drawing and design. The basic unit of the experiment was the semester. Between 1981 and 1986 the experimental instruction in a given course was repeated in from three to five years, so that altogether we have data on 73 different semesters. These semesters together involved 344 learning units, each of which ended with a post-test. The results from these 344 post-testing periods were documented, and data on the results of ca. 40,000 tests were processed. During the five years, 1596 students took part in the experimental instruction.

Data structure

The element of the database is the result of a single solved test. Each test-achievement was represented as the percentage of the maximum score. The frequency distributions of the results of the learning units were computed as follows: (1) the frequency distributions of the post-tests, (2) those after the first repetition of testing, and (3) those after the final repetition of testing. Several aggregations were computed and different comparisons were possible between these frequency distributions, depending on the discipline and the different parameters of the strategies.

Results and conclusions

The first step of the data processing was the analysis of the results of the learning units. Figures 2 to 9 depict the results for the first unit of the first experimental semesters (Chapter 2). The diagrams in the first rows of the Figures show the results of the post-test. Those in the second rows present the cumulative results after the compensatory sessions on the first repetition of testing. (For students who failed the post-test and therefore repeated it, the new results are included in this row.) In the third rows of the Figures, the results after the last testing are displayed. This is the final distribution of the achievements, and the data therefore characterize the effectiveness of the strategy. Comparison of the first and third distributions shows the impact of the compensatory activity. (n means the number of tests involved in the computation.)

The results for the whole semesters were evaluated in the same way as the learning units. The results for the semesters are displayed in Figures 10 to 28 (Chapter 3.1). The cumulative results of the experimental years are displayed in Figures 29 to 33 (Chapter 3.3).

The most characteristic datum relating to the effectivity of a strategy is the percentage of students who reach the mastery criterion. The data in Table I are the proportions of students who had reached the 70% level before they began to learn the next unit. The data refer to the aggregated frequencies of all semesters in the courses of the respective discipline (Chapter 3.2).

If we take into account all 73 semesters of the experimental courses, 70.3% of the students are over the 70% mastery level. (48.4% of the students are over the 80% mastery level.) The aggregated result of the whole experimental instruction is displayed in Figure 34. As these aggregations involve all experimental semesters, including the results of the learning units where the principles of the mastery learning strategies were not followed precisely, it can be observed in this Figure that the mastery strategy works properly in the case of the majority of the students (Chapter 3.4).

If we characterize the semesters by several quantitative features (variables) and consider the 73 semesters as the elements of a statistical sample, further advanced analyses can be carried out (Chapter 3.5). In a multiple regression analysis the system of variables was as follows:

1. The proportion of students who reached at least the 70% mastery criterion on the last repetition of testing. This variable characterizes the effectiveness of the strategies and is the dependent variable in the multiple regression.

2. The year in which the experiment took place. We presumed that as experience accumulates year by year during the period of the experimental process, the results improve.

3. The number of learning units in a semester (the bigger the number, the shorter the length of the units).

4. The number of possible repetitions of testing.

5. The mastery criterion in the strategy.

6. Were the rapid learners awarded with advantages?

7. Were there sanctions against the students who did not reach the mastery criterion?

8. The number of students in the group.

9. The number of pages of the tests used in the semester.

10. The results of the compensatory activity (difference between the results after and before the repetition of testing).

The results of the multiple regression are summarized in Table II.

The factors which play significant roles in the effectiveness of the strategies are as follows: the accumulation of experience in the methods of mastery learning (5.38% of explained variance), the number of learning units in the semester (5.31%), the advantages given to the fast learners (10.59%), the number of students in the group (2.24%), and the compensatory activity (31.53%).

According to a survey, the majority of the students evaluated the experimental instruction positively, and 91.1% of them suggested that the mastery learning strategies should be introduced into the regular system of instruction (Chapter 4).

The result of a draft cost-effectiveness estimation shows that the introduction of the mastery strategies into the most important disciplines requires no more than a 2% increase of the total costs of the training, and results in at least a 10% increase in the knowledge of the students (Chapter 5).