Skip to content

Pre-service Teachers and the Terrible, Horrible, No Good, Very Bad Test

Yesterday, the Teacher Education Ministerial Advisory released a new report on pre-service teacher education titled Action Now: Classroom Ready Teachers, for mine this is a very flawed report mainly because it is based largely on assumptions and biases that are not disclosed, its findings largely benefit the authors of the report, and additionally lays the blame squarely on others. The authors do a very good job trying to pretend that this is a fair and balanced report though, as they continually make the claim that their recommendations are all evidence-based. So much so that the word evidence is used 146 times in the whole document! While the report authors do provide a definition of “evidence-based teaching practice” in the glossary (interestingly this complete term is only used twice, with one being a heading), the authors do not at any stage define what good and bad evidence is, despite their reliance on it.

If only there was way that we could infer what the authors may view as being good (or even acceptable) evidence? I could propose my own definition of what good evidence is and then seek to identify each of their uses meets my criteria but I think that there is possibly a better (and much quicker way.) Take for example Recommendation 13: “Higher education providers use the national literacy and numeracy test to demonstrate that all pre-service teachers are within the top 30 per cent of the population in personal literacy and numeracy.”  A recommendation that according to the explicit goals of the report has “been chosen on the basis that they are practical, based in evidence and calculated to succeed.”

Lets examine Recommendation 13 against the report’s own criteria…

… they are practical…

In the scheme of things making universities administer a test seems reasonably practical, there might even be a couple of research papers in it! Of course there isn’t any evidence given that a single current pre-service teacher, or that future pre-service teachers wouldn’t test in the top 30 per cent, so it could be argued that administering a test that is passed 100% of the time is practical.

So, is Recommendation 13 practical? I’ll give this criteria half a mark.

Taking about evidence leads us to the next criteria.

… based in evidence…

Unfortunately Recommendation 13 doesn’t look as promising when we apply their based in evidence criteria. There is no evidence in the report (or that I could find elsewhere) that supports the notion that teachers need to be in the top 30 per cent in order to be classroom ready. I’m not sure you’d find many people who would argue that teachers don’t need to literate and numerate, but where is the evidence for using the top 30 per cent as a benchmark? Also, there is no evidence that teachers who are currently teaching in our schools and are deemed to be high quality teachers are in the top 30 per cent. Surely this would be the first piece of evidence on which this recommendation would be based, that there is a strong correlation between teacher quality and scores in the proposed literacy and numeracy test.

So, is Recommendation 13 based on evidence? Sadly I’d have to say no, zero marks.

… calculated to succeed…

Given the lack of evidence for the test in the first place obviously predicting its success is difficult. I assume that success wouldn’t be measured by Higher Education’s ability to administer a test, in that case it would be a no brainer to agree that the recommendation would succeed. Yet, I don’t think that is how success would or should be measured! How else could success be measured?  Success could be better measured by pre-service teachers in the top 30 per cent at the time of taking the test remaining in the top 30 per cent over time. Again I can’t see how such success could possibly be guaranteed that a one off test could possibly predict that a teacher will remain in top 30 per cent of the population in literacy and numeracy over their entire career, especially given the short term knowledge that these types of tests test.

Sorry, for Recommendation 13 based on the likely to succeed criteria, again I can’t give any marks.

My total score for Recommendation 13: “Higher education providers use the national literacy and numeracy test to demonstrate that all pre-service teachers are within the top 30 per cent of the population in personal literacy and numeracy.” based on their own criteria of “is it practical, is it based on evidence, and is it likely to succeed” I can only give it half a mark out of three. It is fair to say that the author’s understanding of evidence is a far short of my understanding of good evidence.

 

Paradoxically, of course, there is also no evidence that pre-service teachers who might not test as being in the top 30 per cent at the time of graduation would not later test as being in the top 30 per cent. Given the focus and time spent on literacy and numeracy based work, I assume that all teachers would perform well on this sort of test. This highlights the most terrible, horrible, no good, very bad part of this recommendation, beyond failing its own criteria, it displays a fundamental lack of understanding of how people learn. Teachers will increase their proficiency in literacy and numeracy as they teach because they engage in wider range of authentic activities that rely heavily on the fundamentals of literacy and numeracy. A single test can only test the immediate past and cannot possibly give any insight into the future, understand a person’s development in terms of a ranking against peers in even worse.

 

However you look at Recommendation 13, using the report’s own criteria, or what we all know about learning and teaching, this is a terrible, horrible, no good, very bad test.

Teacher Quality and the Purpose of School

Disclaimer: This post is not intended to be a deconstruction of the purpose of school and the three roles of schooling, I’ll save that post for another time!

To round out this series of posts on teacher quality I thought I should address the final elephant in the room that those who speak of good and bad teachers need to acknowledge… the purpose of school. If you haven’t read the other posts you may wish to read Who are the Best Teachers?, Assessing Teacher Development (or anything really) and A Better Understanding of Teacher Quality.

Those that teacher bash talk about teacher quality like to pretend that schools have a singular universal purpose, that a school’s sole purpose is to prepare students for their professional lives by teaching them what they need know, and how to think.

Yet when we reflect on the purpose of school without thinking about teacher quality we realise that the purpose of school isn’t so narrow. Sure preparing students for the workforce is a major role but so is developing the social aspects of our students, developing in them a morale compass, an appreciation of culture and history, and how to develop constructive and meaningful relationships. Students engage in leadership programs, work and play in teams, and participate in many other activities and programs for the sole purpose of social and emotional development. There is also a third aspect to the purpose of schools, that is, to develop students as unique individuals. Yes, we believe we need to teach specific forms of knowledge and ways of thinking, yes we believe that we need to develop the student socially, but we also believe that students are individuals with unique talents, and aspirations. As such as teachers we provide opportunities for student choice. We highlight that there are multiple paths rather than a single preferred path to tackle a problem. We encourage students to reflect and assess their own learning. We have student-led conferences rather than teacher-directed reporting.

Yet, when experts, school leaders, and teachers talk about teacher quality they almost always ignore the three facets of the purpose of schooling and focus solely on teaching of content, skills, and ways of thinking. This is because assessment focusses solely on this singular aspect of schooling, but in reality the other two roles of schooling might be held in a high regard by a teacher, the school, and the wider school community. Making pronouncements about the quality of a teacher without understanding the how the teacher and their school views the purpose of their school, is completely unfair bordering on immoral. When talking about teacher quality in this way, we devalue the other two roles of school as well as devaluing teachers and teaching.

When we talk about teacher quality we can only do so ethically in light of what they and their school community believe about the purpose of school, and the weighting of the three roles of school. If we don’t properly understand the purpose of a specific school we cannot possibly determine whether an individual teacher is “good or bad.”

A better understanding of quality teaching

Disclaimer: This post is not intended to be an explanation of developmental stages of learning to read, I’m sure many others cover that ground…

I’ve written about how most of the rhetoric around teacher quality is unhelpful and punitive, so I thought it was about time I offered a suggestion of a better way to understand teacher development.

Here in Victoria Australia we use VELS (Victorian Essential Learning Standards) to guide curriculum and assess student learning, or maybe it is now called AUSVELS thanks to the Australian Curriculum, but anyway. VELS and AUSVELS divides (student) development into levels. The problem is that the names they’ve applied to these levels are meaningless (Level 1, Level 2, …)

Think about students learning to read, in VELS levels 1 and 2. A much better alternative to calling these level 1 and level 2 would be to give the stages descriptive names because that is how learning to read, or developing as a reader, is better understood. Developmental stages go something like non-readers, beginning readers, and then independent readers. Naming the developmental stages as such would much better clarify the developmental process of reading. These labels make sense as they are descriptive, we know what kids do at each of the stages. A non-reader can’t tell which way the book is meant to be up. A beginning reader, makes up their own story. An independent reader engages with the writing. We could build out a much richer picture of what each of these stages look like but the point is that we could look at a student reading/attempting to read a book, and get a pretty good idea of at which stage of development (non-reader, beginning reader, or independent reader) they are. We can’t say the same things about VELS level 1 and VELS level 2, they don’t have the same rich meaning. VELS levels also make learning to read a jumble of independent skills and concepts that need to mastered, rather than seeing development as a transition from one stage to the next stage.

There are many more implications of understanding which stage of development a learner/reader is at but this is not the point of this post….

So what about teacher development?  Are there similar developmental stages that teachers progress through? If so what are they?  And, please don’t suggest Beginning or Graduate Teacher, Expert Teacher, Leading Teacher as developmental stages as these titles are also meaningless as far as development is concerned. I don’t really have an opinion about what the stages of teacher development, so unfortunately this is where I’ll leave this discussion, but the point is that if we’re going to talk about quality teaching then we need to understanding the stages of teacher development. These stages need to described descriptively, with the characteristics of each stage being unique. Think of the characteristics of each of the stages of reading development, understanding how a book is technically constructed doesn’t apply as a descriptor at every stage although it certainly does at the beginning reader stage. So please, don’t suggest that teacher/leader quality frameworks that simply list similar but increasingly complex statements as being useful for understanding the stages of teacher development.

One last thing…

Think about the rhetoric around teacher quality, when people say “we want every teacher to teach like the best” — this it is punitive, intending to separate the best from the rest. We don’t say we want every student to read like the best, or talk about quality reading! We don’t say we want every student to read like Lily (obviously she is the best reader in the class),  instead we say we want every student to be an independent reader.  Maybe if we had a better understanding of the stages of teacher development we would see the end of teacher bashing…

Assessing teacher development (or anything really)

I was pointed to “Licensed to Create: Ten essays on improving teacher quality,” as part of the current conversation on twitter about teacher quality and teacher bashing. According to the website,  “Licensed to Create is a collection of essays from some of the leading thinkers in education. 11 authors offer their unique perspective from practice, policy and academia on how we can improve teacher quality.”

I haven’t read any of these essays, and I don’t intend to… here’s why.

A quick search finds the word evidence is used eighty four times (admittedly this includes its use in the various references). I guess this is pretty good, after all we don’t just want to just rely on someone’s opinion, we want hard evidence, not conjecture. But wait a minute… on what is this evidence based? Let’s check. Whoops, the word theory is only used a single time, and that single occurrence (p. 45) is not concerned with the theoretical underpinnings on which the numerous points of evidence are based. The words pedagogy, pedagogies, and pedagogical are used sixteen times combined, again none of these uses illuminate the theoretical or pedagogical underpinnings of the frequently cited evidence. For all we know, the educational theory and pedagogical beliefs underpinning these ten essays were obtained from the fish and chip owner down the street! Although, I do admit this is highly unlikely… I’ve spoken to her and she doesn’t think much of teachers!

My advice is that you’d be foolish to believe any educational evidence that doesn’t explicitly define and detail how it uses educational theory to validate its evidence. Take all of these ten essays with a big grain of salt. And when you read any other essays, blog posts, scholarly papers, or whatever, first assess the educational theory, and only if you’re satisfied that the theory provides a solid basis then assess the supposed evidence. If not, treat the writing as an opinion piece, despite what they try to make you believe.

 

So how might educational theory be used to understand teacher quality?  Let’s look at two possible (of many) approaches Instructivist, and Cultural Historical Theory.

Instructivist Theory

Instructivist theory suggests that learning is a change in student long term memory, which enabled by the teacher providing concepts, procedures and strategies. This knowledge needs to be explicitly taught rather than discovered. If nothing changes in the long term memory, then nothing changes and the student can be considered to have learnt nothing. Identifying learning is therefore relatively simple, we pre-test what the learner knows and then post-test after the learning sequence to assess what the learner now knows. The difference between the two shows what they have learnt.

Assessing teacher quality using an instructivist approach should therefore be pretty easy, we simply test their knowledge, or observe them teaching.. inferring what they do is what they know. With the instructivist approach it is also easy to tell who stored the most learning in the long term memory, we could even call them the best, if we wanted to. We could also find the student who has stored the least in their long term memory, and label them the worst, if we were that way inclined. And if a learning theory is not explicitly given by an author it is a pretty good guess they are using instructivist theory… but they should be explicit about this.

Yet, there is an obvious problem with this view of learning… what if the test is wrong? What if we’ve misunderstood what is important? What if we’ve required student to learn the wrong things? The test is the weak point, and it is very fragile to say the least.

Cultural Historical Theory

Cultural historical theory suggests that a study on the fragments of student development, (as in the case of an instructivist approach’s study on only long term memory) doesn’t allow us to understand the development of the whole student. In fact, cultural historical theory suggests understanding the process of development is far more important than understanding the results of development. This is because development occurs through a series of stages, for example consider the stages of physical development, this puberty being a particularly obvious stage of development we all go through.

When assessing teacher development using cultural historical theory, we study the teacher functions for which development has not yet started, the teacher functions that are currently developing, and finally the teacher functions that are fully developed. Cultural historical theory argues that development can only be understood at looking at the whole development cycle, think of the gardener tending plants, just looking at the plants that are in full fruit, and ignoring the budding plants, fails to give an accurate picture of development.

In the same way, looking at the fruits of development of the teacher (eg what they currently do) does not give an accurate way of understanding teacher development and/or teacher quality. Further, comparing stages of development of one student to another is futile. To suggest one teacher is better than another, is like suggesting that a child entering the stage of puberty is better than a child that has not yet entered puberty. They are simply at different stages of development. (Obviously, we could explore whether a teacher is at the expected stage of development, and then seek to understand why they aren’t if that is the case.)

 

 

Obviously there are many other theoretical approaches the authors of the ten essays could have used by which to make their arguments about teacher quality. My hope that is post has encouraged you to question the educational approaches and beliefs that underpin the evidence that teachers are exposed to. I also hope that educational experts and other authors take up the challenge to make explicit the education theory that underpins their evidence.

Maybe it also has caused you to question whether an instructivist approach is capable of assessing teacher quality.

Who are the best teachers?

Apparently, Maxine McKew at a Bastow Seminar something like “How do we get more teachers teaching like the best?” I wasn’t there but I assume something similar to what was reported was said, because experts say these sort of things regularly.

This is a very poor question, and one that I suggested on Twitter was teacher bashing, but let’s for a minute consider that it was not…

Many argue that there is nothing wrong with this rhetoric. “Don’t you believe or want teachers to improve?” and “I’ve met lots of teachers who are have given up!” or  “There is nothing wrong with having high expectations of teachers.” or ….

 

So, if we want to get more teachers teaching like the best, it must be asked who are the best teachers and how are they teaching?

For me this is a no brainer. The best teachers are those using play or project-based learning approaches. I believe that these approaches are the most pedagogically suitable, I believe that these approaches lead to the most effective learning in students.

So naturally I must believe that the worst teachers are those not using play or project based approaches… except I don’t.

Why?

Because labelling teachers as the best suggests that there are fully in control of everything they are as a teacher and everything they do as a teacher. That they must be able to pull themselves up by their bootstraps so that they can too be like the best teachers. Firstly, suggesting that teachers are free agents, able to use the most pedagogically appropriate approaches in their school, doesn’t account for the pressure that accountability that teachers are under. Even if teachers were free agents, suggesting that despite their everyday experiences, teachers can somehow learn (that play and project-based learning approaches are most pedagogically appropriate) spontaneously despite their history as teachers and their school environment, goes against everything we as educators know about how people learn and develop. Learning and development doesn’t happen independently and spontaneously, it needs the right environment.  Pleading for teachers to just improve is useless, about as effective as pleading with them to spontaneously grow bunny ears!

 

Of course, you probably disagree with me about who the best teachers are.. and you might even think that teaching like the best (your way) is easy. If so, I really think you’re either underselling the role of teachers, or dismissing the complexity of teaching.

 

So why do experts and others continue with labelling teachers as bad teachers rather than defining what makes the best teachers the best? Who knows? But I wish they’d stop because they are damaging the profession and making substantial changes in the way student’s learn in our schools all the less likely.

[I have edited the last sentence of this post to clarify that I am suggesting student learning needs to change, not the teaching profession.]

Plans and changes for 2014

I know it is not exactly the start of the year anymore but it about time that I publicly state my intentions for 2014.

Short story, I’ve started a new company and enrolled to do my PhD.

 

The longer story is that the company is the natural evolution of the work that I’ve doing at ideasLAB and on my own for the last five years. It has become increasingly apparent that the major stumbling block for the successful re-imagining learning and teaching is our lack of pedagogical understanding. This has been proved to me over and over as I’ve been working on and presenting about the Principles of Modern Learning over the last year. This doesn’t mean that I’m dropping the Principles though, far from it, for those waiting the book will be out soon!

The company that I’ve founded is called Validated Learning, whose first, and flagship, product is the Modern Learning Canvas. The Modern Learning Canvas builds on the best ideas from the business and entrepreneurial world. Basically the idea is that if entrepreneurs can have a business model to describe, understanding and design innovative businesses way can’t and don’t educator have a learning model to describe learning and teaching? Why don’t we have a framework for understanding, describing and designing for innovative learning and teaching? Well now we do!

It enables teachers to more beyond a pedagogical understanding based around the lesson plan and the how and what of learning and teaching, to pedagogical language and understanding of the why of learning and teaching. In doing so it enables teachers even with completely different practice to engage in meaningful pedagogical conversations.

The Modern Learning Canvas is still in closed beta but it will be launched in a matter of weeks, so stay tuned! In the meantime check out the canvas on the site and download the free creative commons printable version.

The second big change is my decision to pursue a PhD, something that I was certain that I would never do. In September, the Modern Learning Canvas started working really well and I began writing a (soon to be released) white paper on it. At that time I began considering whether academic research would be beneficial for the development of the Modern Learning Canvas. In speaking to universities about it, I became more and more convinced that it was a natural and sensible decision. So just a few weeks ago I enrolled at Monash University, initially I’ve signed up to do a Masters of Research but the plan is to switch over to the PhD program later in the year.I couldn’t enrol directly in the PhD program as I didn’t have a research component in the original Master of Education!

My supervisors are Dr Phillip Dawson and Professor Marilyn Fleer. So a big thank you to them for seeing merit in the idea!

Yes, a big year is on the cards…. stay tuned!

Why everyone should learn to code [eventually]

Will Richardson pointed to Bret Victor’s critique of code.org highlighting Seymour Papert’s writing where he laments the misinterpretation of his book, Mindstorms, that programming in itself is worthwhile.

Spoiler alert: It is not. Programming without the purpose of exploring ideas is just learning to code. Nothing more, nothing less.

Case closed… I guess?

But what about reading? And writing?

Take the  passages from Papert’s “What’s the Big Idea?” paper that Bret Victor has highlighted and replace the word “program” with the word “read” or “write.” Is it any less true that reading does not “in itself have consequences for how children learn and think”?  Is it any less true that writing does not “in itself have consequences for how children learn and think”?

It a lot of ways whether programming in itself is necessary for students to learn is the wrong question. The bigger question is what do students need to learn, period. If we’re throwing things such as programming in and out of some supposed curriculum and unspoken (for Bret) but crucial pedagogical framework, then we need to put everything on the table: reading, writing, programming, quadratic equations, surds, history, geography, …. everything.

So what do students need to learn?

Students need to learn what they need to learn, just like us. We need to learn what we need to learn when we have to solve real problems. Students are no different. Curriculum is designed to predict need. We can do better now. We no longer need to teach programming, reading, writing or anything else just in case. Having said that, over time, through a variety of experiences our students will most likely cover all the good bits of the old curriculum.

Of course, using that criteria, I doubt there is any debate that reading and writing would make the cut. That quite early the need to read and write would be necessary for whatever the students were exploring.

I’ll leave it to the mathematics, history and geography teachers to argue the rest of my list above…

Since 2000, when Seymour Papert wrote this quoted paper, the world has gone digital. In the space of 14 years we (and our students) deal with digital information and content continually. Programming gives us (and our students) powerful tools to play, explore and make sense of of this information. Whether it be a simple  function to manipulate data in a spreadsheet, or a script to scrape data from a website that thinks it “owns” our data, a website to communicate with the world or many other cases, without the ability to program our students will be constrained, limited and frustrated.

Authenticate problem solving, ideation and play in our digital world requires the ability to program. I can’t believe how any student can go through the schooling experience without being faced with that fact. If our students are working on meaningful problems and projects, if our students are seeking to understand their world, and explore big ideas, programming in some shape or form is essential.

 

 

Note 1: Yes, I probably agree with Bret Victor’s caricatures of the rich and famous’ reasons for learning to code, but that doesn’t mean I agree (and I’m not sure what Bret actually thinks) that student shouldn’t learn to code.

Note 2: This is not meant to be a critique of Bret Victor, I love his ideas and work. Bret suggests that you read Mindstorms if you haven’t already, I second that.

What do they expect to see?

So someone somewhere thinks it might be a good idea to videotape teachers in the classroom.

I wonder what they expect to see?

I wonder if they’re able to distinguish good practice from bad practice, or good teachers from bad teachers, and if so, how?

“The best way of learning is to see yourself and have somebody help you.”

Wrong. It is a myth that if you’re not succeeding (whether we’re talking about teachers or students) it is because you’re not working hard enough.

The best way of learning, is to have a process that enables you to make good decisions about your learning.

 

Finally, if videotaping was the best way for teachers to learn it would also be the best way for students to learn. Yet they’re not suggesting that students be videotaped, I wonder why not?

Melbourne teachers – beta testers wanted

My (sort of) secret project, The Modern Learning Canvas, is ready for beta testing. In a nutshell the canvas provides context that enables teachers to understand and discuss  their learning and teaching approach, and design for learning.

I’m looking for teachers who would be willing to act as beta testers. Ideally, a professional learning team from a Melbourne school, where the whole team would work on the canvas together.

If you’re interested, I’d love to speak to you about this. I would be happy to give a demonstration to you before you commit, and the time commitment for the team would be a single session for about 45 minutes. Email me at richard@richardolsen.me or contact me via twitter @richardolsen

Lessons in spelling ‘have no place in 21st century schools’

Lessons in spelling ‘have no place in 21st century schools’

By Graeme Paton

Sugata Mitra, professor of educational technology at Newcastle University, said good spelling and grammar was necessary “maybe a hundred years ago” but “not right now”.

He insisted that children should be encouraged to express themselves in a number of different ways – including using mobile phone text messaging – rather than relying on established linguistic rules.

The comments come despite a new drive by the Government to promote the basics of English language throughout compulsory education.

Under new plans, a revised national curriculum is being introduced that requires pupils to accurately spell 200 complex words by the end of primary school.

 

Another trotting out of the line, that nothing has changed in 100 years, which is false, silly and unjustifiable. But Sugata has a point, to a degree, and the 200 most common word list is a great example. The 200 word list is a compromise because as Sugata points out we require our students to learn them before they use them not as they use them. Technology allows us to learn as we need to, however, it does not replace the need for learning, as Sugata seems to be suggesting, in fact, spelling and grammar is exceedingly more important in todays world of instant publishing.

Maybe the 200 words could be instead used to audit our teaching program? If our students are exposed to a variety of authentic experiences they can’t help not learning those words.