In a much praised and vaunted education system
in the ACT, there is statistical insanity that makes gross
assumptions that would shock most people. When your son or your
daughter receives a score on a report card in years eleven or twelve,
one might have assumed quite wrongly that it was a direct reflection
of their work and ability in relation to the subject matter and in
relation to other students' performances.
... And you would be wrong!
It is not the result of the above, but rather
the result of "z
score weighting" dependent on the mean score of a group or
cohort; such scores being dependent on the size of a group and
vulnerable to the initial balanced eco-system of estimated student
capabilities. In other words, while achievement is necessary to gain
a footing on the page, one's worth is measured solely in terms of a
rank order; determined less by actual grading by teachers than by
statistical sorting by the statistician who seeks probable and likely
outcomes measured by complex formulas.
Knowing how the system works, teachers are
encouraged to "ensure" that top students are protected.
Somehow, a spread of students is to be gained by differentiating
within assessment tasks. The problem then arises when a number of
students display similar levels of ability at the top end of a small
cohort which often can happen in the Arts faculties. The statistician
will argue that anomalies will be sorted with the moderating
influence of the AST exam. However, no statistic can give an accurate
indication of a particular person's score; just as no political polls
can say exactly how any particular person voted. So with this in
mind, schools encourage teachers to fairly early on identify who is
the top student and to then make sure that person is statistically
immune to variations in the mean that might occur due to moderating
influences. But what if the status quo is upset and another student
emerges from seemingly no-where to converge on the top place and even
over-take it. What if a particularly strong student is doing a double
major within the same cohort and is ostensibly competing with him /
her self. Such events risk the easy statistical values given to
student achievement and leave the school or cohort open to
accusations of not having tasks with differentiated levels embedded
But perhaps the worst and most despicable
element in the whole process is the rank order competition and the
divisive nature of setting student against student in achieving
supposed educational outcomes. The notion of co-operative learning
between teachers and students makes no sense at all in a system that
builds in rank order scaling of all scores to determine student
results. This is especially the case when that rank order is NOT
determined by skill or knowledge within subject areas but rather by
statistical manipulation which is claimed to be beyond challenge. So
the temptation in a school and within a department or a cohort is
then to artificially inflate the score of the top student or students
(in a larger cohort) thus advantaging one student over the other.
And if my analysis of the situation is to be
challenged, I posit that a student in a cohort of 25 students who
achieves a score of 85% might well achieve a score of 95% in a larger
cohort of students where a better z score might be achieved. Can the
statistician in the ACT system tell me with a straight face and with
certainty that this is NOT the case?
Is this fair?
Well, to be fair to a system one might consider
reasons why the formulators of it consider it the most fair in an
imperfect world. To think about this, let us look at The
Rosenham Study on Being Sane in Insane Places also for a
critical look check it out here.
You may wonder how a psychology experiment used to expose the
fallacy of judgement in deciding what was sanity and insanity has to
do with our much flawed statistical methods of judging rank orders in
student achievement. Go back to 'game
theories' and dominant ideologies instructing US and UK
governments with total belief in
statistical data as the defining means for determining progress.
Certainly within an Educational context, teacher judgement, like that
of the psychiatrists in the Rosenham Study, is seen as less reliable
than statistical formulas. The use of extensive check-lists in
psychology has seen the invention of mental
disorders to replace normal human emotions while seemingly
objectifying the data used for the diagnosis of illness; all to the
of large drug companies. Flawed subjective diagnosis has made
way for a wider and potentially even more flawed system of labelling
behaviours as "disorders". This is done on the basis
of a check-list of behaviours and invented names for a seemingly
endless list of mental disorders. The result is potentially socially
devastating. In Education, the use of "league tables",
NAPLAN (in Australia) and the reduction of everything to seemingly
measurable outcomes has led to a narrow focus on literacy and
numeracy as being the determinant of success in educational
instruction and management. In the ACT in Australia, this focus has
led to an exam being used to moderate a cohort of students against
another. It's value is actually dependent on the notion of an
equilibrium existing around a mean score within an individual
subject/or subject cluster cohort then within school and then within
schools drawn from the whole of the Australian Capital Territory.
While movement of student scores is possible within this equilibrium,
the over-all intellectual
eco-system remains the same. In many respects it bears the
hallmark of the Nash
Equilibrium while ignoring the knowledge
ecology within educational organizations, communities and circumstances.
Some application of recent work on
"knowledge ecology" within a national, community and school
based application might indicate some of the findings and
observations made by Pasi
Sahlberg, the Finnish Educator who points to the dangers
of the Global Education Reform Movement (GERM) which began
somewhere in the 1980s.
A key element in this has been in the
development of consequential
accountability through standardised tests using primarily maths
and literacy as the starting point. Sahlberg
gives a good illustration of how this has affected policy in the
United States :
"Perhaps the best-known practical
illustration of large-scale education reform driven by the notion of
standardization and related consequential accountability is found in
the USA, where controversial federal legislation termed No
Child Left Behind (Public Law 107-110) links school and teacher
performance to Adequate Yearly Progress and to financial and resource
allocations to schools (Popham, 2004; Centre on Education Policy,
2006). Recent research, however, suggests that 'the ability of
standardized tests to accurately reflect school performance remains
in doubt' (Lemke
et al., 2006, p. 246). Furthermore, Amrein
and Berliner (2002) concluded, on the basis of their analysis
across 18 states in the USA, that since clear evidence was not found
for the positive impact of high-stakes testing policies upon
increased student learning and because there are numerous reports of
unintended consequences associated with these policies, such as
increased student drop-out rates, teacher and school cheating on
exams, and teacher defection from the profession, there is need for
transforming existing high-stakes testing policies."
quoted in: New Directions for
school leadership and the teaching profession: DEECD discussion
paper, June 2012 - A Response by Martina Golding, Performing
Arts at South Geelong Primary School.
This point is also made by Adam Curtis in his
documentary THE TRAP part 2)
The ACT began a system in the 1970s that was the
opposite of that proposed by GERM. In the past decade it has been
slowly drawn into the statistical games theories reflecting the
supposed isolation and personal self-interest theories of John
Nash and the resultant discrediting of teacher judgement as an
indicator of student achievement ranked against other students.
The reliance on statistical data as an indicator
has led to the opposite of what it sought to achieve. Instead of
removing teacher judgement from the equation, it has artificially led
to the increase in teacher/school manipulation of statistical data to
get the results they wanted. This is exactly what schools do by
limiting those who sit for the moderating exams (the AST) or NAPLAN
(where record numbers of students are being exempted from sitting)
and where inaccurate pictures of school achievements are being
created. It leads to further inaccuracies by selective schools
determining who will actually be accepted into the school. It
reinforces those areas where wealth determines the composition of
school communities. And worst of all, the statistical determination
of success reinforces the teaching to exams and the consequent
alienation of students from effective and meaningful learning.
Just as the proliferation of Mental
"disorders" has become significant in psychology, so too
has the effective wrangling of the system by newly skilled
applications of such "disorder" thinking to gain rank
order for University entrance become the big phenomenon of 2014/15
Education. Rather than encouraging intrinsic learning the application
of skill to life and creativity, the statistical nightmare of our
contemporary system is doing the exact opposite: forcing schools,
students and communities into a cynical games play where everything
is on the table to assist favourable rank order.
Questions of fairness no longer come into play;
rather we focus on what might "advantage" or
"disadvantage" students as an "accountability"
issue rather than as a question for learning.
. . . And a question I pose to those who dispute
this thesis: "Two students in two different schools with very
different sized cohorts in a given subject like Maths! Both score
100% on all of the tests and assessment tasks which were the same for
both schools. Can the ACT system guarantee that both students would
score the same in the final outcome?"
And IF NOT! CAN WE STILL
SAY THIS IS A FAIR AND EQUITABLE SYSTEM?
References are generally linked in the
article on 19 - 20 December 2014
PS: Have a look at an alternative to the "pathologizing of
childhood" model used in Australia and eminating from America HERE.