Poor research and ideology: Common attempts used to denigrate inquiry

Time to read: 5 minutes

A few months ago a prominent Melbourne University academic tweeted “Pure discovery widens achievement gaps” citing the paper “The influence of IQ on pure discovery and guided discovery learning of a complex real-world task” I was immediately dubious of this research, as research that is commonly quoted showing that the inquiry learning doesn’t work, is usually fundamentally flawed. I’m not a proponent of “pure discovery learning” per se, but I feel this type of research, and the reporting on this type of research is designed to denigrate all inquiry learning. In an attempt to leave only teacher instructional approaches standing – why these researchers don’t instead prove a theoretical basis for instruction is beyond me.

So I took a look at the research to see if educators should have any confidence in its reported findings.

TLDR: No, we shouldn’t haven’t any confidence in this research, and it does not show that pure discovery or inquiry approaches widen achievement gaps.

 

Not surprisingly, this research fails the good educational research test as it doesn’t use a learning theory. That is, the research does not attempt to justify a theoretical basis for its findings. The researcher does use two other (non-learning) theories though, to defend the research, notably game theory, and control value theory. In essence the author uses these theories to defend the research design, yet for some reason he does not believe a learning theory is also required? I find this baffling.

Why doesn’t the author believe that a learning theory is required to define the scope of the research, given that the research is about learning? Why does the author believe that theory is required to explain games, and emotional attainment?

Anyway, let’s look at the research, as I’m always interested in how this type of research is used to investigate inquiry learning, or in this case pure discovery learning.

The author defines pure discovery learning as learning occurring “with little or no guidance. Essentially, knowledge is obtained by practice or observation.” The author spends considerable time explaining how pure discovery occurs in so much of our lives, with ATMs and iPhones requiring people to use them correctly without instructions. He continues, in explaining how Texas Hold’em poker requires people to “use multiple skills to reason, plan, solve problems, think abstractly, and comprehend complex ideas” which are similar to real life pure discovery learning situations.

The author explains: “The poker application used for this study was Turbo Texas Hold’em for Windows, version four copyright 1997–2000 Wilson Software. This is a computerized simulation of a 10-player limit hold’em poker game.”

Wait!!! What????

A computer simulation of a game you play with real people is a suitable method for exploring pure discovery?

Texas Hold'em Archive.orgInterestingly enough, you can play Texas Hold’em, the software used in this research in your browser thanks to archive.org. (Note: If you’re using a mac use Function + right arrow when it asks you to press End to play.) In playing Texas Hold’em you will discover just what an poor attempt of simulating the playing of poker against nine other simulated people this really is.  It appears that the study data used in this paper is actually from a previous study by the same author, “Poker is a skill” dated 2008. The 2008 date still doesn’t explain why such old DOS software was used! In this paper the author explains that 720 hands of Texas Hold’em over six hours is equivalent to thirty hours of casino play, with real people as opponents. That is 6 hours playing against a computer is supposedly the same as playing 30 hours against real people!

If you play the simulation at archive.org it is easy to see how 30 hours of real play can be achieved in 6 hours using this simulator. Turns made by your computer opponents fly past with short text messages popping up briefly on the screen.  Two groups of students used this old software. The researchers provided the instruction group with instructions of a specific poker strategy, the pure discovery group were, for some reason, given documents detailing the history of poker! The success of players was determined by the money that they had won (or lost), though it should be noted that the participants were not playing for real money. Instead the highest ranking players were promised to be playing to a chance to be part of a raffle for an iPod. This was intended to place meaning, to the otherwise valueless money, each player was playing for.

So this study designed is to simulate a real life complex problem, yet it doesn’t even simulate a real life game of poker!  The participants were not playing against real people. The participants were not playing for real money (though their success was measured as if they were.) And… the participants were playing five times faster than real poker is played.

All of this should make anyone question how the author could possibly argue that this research design can possibly be described as “pure discovery,” as commonly used in real life situations. Interestingly, though the author identifies differences between the instruction and control groups. Neither group learned to play poker to the point where they didn’t lose money. Further both groups played many more hands, more than twice as many hands as poker experts are reported to recommend that “good” poker players play. That is, neither group exhibited the one of the main traits of good poker players, folding around 85% of the time, and only playing 15% of the time.

 

How might research of pure discovery be better designed?

Duh.

 

Playing with real people would allow a learner using pure discovery to observe, and seek to understand the decision making of other poker players. Depending on the relationship with the players the learner might ask questions of their opponents, seeking to clarify rules and strategies. Other players might also intervene in the play, offering advice and pointing out pivotal moments in the hand, and pivotal decisions being made by other players. If the participants played against real people, surely they would’ve noticed that they were playing many more hands (much more than twice as many) than their more skilled opponents? Though the computer might be able to simulate the logic of poker, it cannot and does not simulate the interactions between the players, a critical feature of playing any game, and especially poker.

Given that the instruction group did not learn to play Texas Hold’em poker to a satisfactory level, it is obvious that instructional strategies used did not work. Of course, it must be noted neither did the control group, who were left to battle the computer opponents on their own, armed only with a document on poker history. To suggest players playing against real opponents, using pure discovery or other inquiry approaches would also fail to learn to play poker satisfactorily, is obviously outside of the scope of the research, as the researcher did not explore this.

Of course, the lack of a learning theory is what has also led the researcher to his narrow definition of successful learning. Did the author ever consider why people play poker? Is money the only indicator of successful play? Or do people also play games for fun? Are the social aspects of playing with friends an important part of being a poker player?

A more complete understanding of what makes a poker player, a poker player, would consider other indicators, traits, characteristics and motivations.  Did the study participants continue playing poker after the study had finished? Did they enjoy playing poker?  Do they intend to teach friends? Do they feel playing poker with their friends strengthens their friendships? (Not that they were given this opportunity.) Have they developed their own theories and strategies they intend to try out in the future? What do they know about poker?

I believe a better understanding of poker players and the reason people play poker, would greatly improve this poor study. It would also provide further evidence for the worth of learning to play poker by playing poker with friends.  Not that this is an earth-shattering conclusion! After all isn’t that how we all learn to play any new game? Or maybe you’re the one out on the oval by yourself with a ball, a sheet of paper documenting the history of football!

 

Driven By Ideology?
To suggest that individuals playing Texas Hold’em against a computer mirrors inquiry that happens in our schools in complete nonsense.

To suggest that this research proves pure discovery “widens the achievement gap” is complete nonsense.

To suggest that learning poker by yourself on a computer playing against a simulation has anything at all do with student learning and real inquiry is nonsense.

Do academics who favour high levels of teacher instruction really expect us to believe that inquiry classrooms operate the same way that people learn to play poker individually on their computer?

Do academics who favour high levels of teacher instruction really believe that playing poker on your own against a computer tells us anything about how teacher professional development or teacher pre-service training should be designed?

Do academics who favour instruction really believe that a piece of paper with strategies on them is really the best way to learn anything?

Do academics who favour instruction really believe learning is just about knowing, and not about experiencing with others?

Do academics who favour instruction really believe we’re that gullible?

3 thoughts on “Poor research and ideology: Common attempts used to denigrate inquiry”

  1. Richard very interesting rebuttal of Dinham’s enthusiastic response. Could I suggest that you do some revision of typos and other infelicitations that have crept into the article in haste!

  2. Thanks David,

    I like to think the grammatical and other errors add to ad hoc nature of these posts. Happy to clarify any points or fix logic mistakes though.

    I didn’t name Dinham, though I linked to his tweet for context, because in my mind this post is not about his tweet nor the poker research, but rather all research that seeks denigrate inquiry using flawed approaches.

    Cheers,
    Richard

Leave a Reply