The use of Quality Talk to increase critical analytical speaking and writing of students in three secondary schools

Quality Talk

The construct of Quality Talk was developed as the result of collaboration between researchers at Ohio State University and Penn State University in 2012. Their meta-analysis of 42 quantitative studies examined the effects of discussion-based approaches on teacher–student talk and on individual student comprehension and learning outcomes in primary school settings. A key finding from this meta-analysis was that some programmes fostered a critical analytical (CA) stance by students where the teacher had control over the text and topic, but the students had the majority of control over interpretive authority and turn-taking so there was shared control. Examples of such programmes included the Paideia Seminar (Billings & Fitzgerald, 2002), Collaborative Reasoning (Chinn et al., 2001), and Philosophy for Children (Sharp, 1995).

Programmes that engendered a CA stance promoted higher levels of thinking and reasoning above the expressive and efferent stance programmes. An expressive stance (Jakobson, 1987) gives prominence to the student's own affective, spontaneous and emotive connection to all aspects of the textual experience, while an efferent stance (Rosenblatt, 1978) gives prominence to acquiring information from the text. In contrast, a critical analytic stance (Wade & Thompson, 1994) gives prominence to querying or interrogating the text in search of the underlying arguments, assumptions, worldviews, or beliefs that can be inferred from the text. Each approach serves a distinct purpose depending on goals teachers set for their students, such as acquiring information, interrogating the text and/or its author, and responding affectively to the content of the literature. Each approach contains some type of instructional frame that describes the moves of the teacher, routines for discussion, the role of the text, who controls the discussion and the presence of pre- or post-discussion activities. All approaches, while not identical, purport to help students develop high-level thinking and comprehension about text.

Examples of expressive stance programmes in which students have much control are Literature Circles (Short & Pierce, 1990), Grand Conversations (Eeds & Wells, 1989), and Book Club (Raphael & McMahon, 1994). In contrast, programmes that give prominence to an efferent stance, in which teachers have the greatest control over the discussions, include Questioning the Author (Beck et al., 1997), Instructional Conversations (Goldenberg, 1993) and Junior Great Books Shared Inquiry (Great Books Foundation, 1987). Quality Talk therefore, sought to include those characteristics that foster a CA stance.

A critical stance and Quality Talk

A critical analytical stance, according to Wade and Thompson (1994), occurs when a student has the ability to interrogate or query issues and ideas in search of the underlying arguments, assumptions, world views or beliefs. The critical analytic stance is therefore considered more desirable than either the efferent or expressive stance because this type of stance provokes more thoughtful responses. Although the efferent and expressive stances are useful pedagogical techniques, the CA stance engages students more deeply because of the shared responsibility of the thinking and learning, and the emphasis on the use of questions (Wilkinson et al., 2010b). Quality Talk was developed to give teachers a framework for discussions around text and comprehension of text that elicited a CA stance.

The key features of Quality Talk are authentic questions, uptake questions and high-level questions, which include generalisation, speculative and analytical questions (Applebee et al., 2003), reasoning words, and elaborated explanations (Chinn et al., 2001). If teachers and students increase their use of authentic, uptake and high-level questions, it is likely that students will shift the complexity of their dialogue toward a dialogic spell. An episode of talk is considered a dialogic spell rather than a discussion if the discussion begins with a student question (dialogic bid) and is followed by at least two more questions. The discussion may include teacher questions so long as they do not significantly alter the course of the conversation (Nystrand et al., 2003). Research has shown that CA thinking is more likely to occur during a dialogic spell than traditional dialogue (Nystrand et al., 2003), additionally if students use more reasoning words and elaborated explanations, they are more likely to improve their dialogue and engage in more challenges, known as exploratory talk (Alexander et al., 2008). Quality Talk seeks to promote these behaviours.

Quality Talk can involve the teacher choosing a text or discussions with small heterogeneous groups of students that begin with an open-ended question by the teacher. Once the discussion is underway, students increasingly take control. The teacher uses epistemological tools to give students greater control over the flow of information: asking authentic questions to invite a range of responses; employing uptake questions to build on student contributions; asking questions to elicit high-level thinking (analysis, generalisation and speculation) (Nystrand et al., 1997, 2003), and analysis questions to encourage students to use reasoning language. These discursive elements also include questions that elicit extra-textual connections (affective, intertextual and shared knowledge) (Edwards & Mercer, 1987; Bloome & Egan-Robertson, 1993; Applebee et al., 2003; Taylor et al., 2003). Other indicators of individual and collective reasoning include students giving elaborated explanations (Chinn et al., 2000), engaging in exploratory talk (Mercer, 1995, 2000) and using key words that signal reasoning (Wegerif & Mercer, 1997; Wegerif et al., 1999). Over time, this distinction becomes blurred as students internalise ways of talking about text and use these ways to support their own thinking and that of their peers.

Previous research showed that teachers needed to provide temporary support in the early stages of discussions in order to introduce students to the kind of talk that promoted critical-reflective thinking (Wilkinson et al., 2010b). Potential moves by teachers that were useful to these conversations were summarizing, modelling, prompting, marking and challenging. Combined with these recommendations for what the teacher is expected to do, there are pedagogical principles that are considered essential to fostering a culture of dialogic inquiry in the classroom: (1) using rich, interesting texts that permit a variety of interpretations, opinions, or positions and about which students have some background knowledge; (2) collaboratively establishing norms or ground rules for discussions; and (3) initiating discussion by asking a ‘big question'—a question of central importance to understanding the text that has no known answer, and about which students' opinions may differ. The principles also included larger ideas about language and pedagogy: (4) conceptualising language as a tool for thinking (Mercer, 1995, 2000); (5) maintaining a clear structure and focus in a discussion while also being responsive to students' contributions to enable generative learning (cf. Cohen, 1994; King, 1999); and (6) gradually releasing responsibility for control of the discussion from teacher to students (cf. Pearson & Gallagher, 1983). The ultimate goal was for students to take responsibility for co-constructing their own interpretation.

Rata (2012) has argued that limiting the curriculum to experiential knowledge limits access to a powerful class resource: that of conceptual knowledge required for critical reasoning and political agency. Knowledge that comes from experience limits the knower to that experience. As asking questions is more likely to encourage a student to go beyond their own affective response or the immediate retrieval of information (Rata, 2012), an approach such as Quality Talk that encourages the use of questioning could increase conceptual thinking.

Research on Quality Talk

Though previous studies on the use of Quality Talk have shown an increase in a CA stance during speaking, no study has shown an impact of Quality Talk on students' ability to write with more of a CA stance. For example, a study by Reznitskaya et al. (2012) of 12 fifth-grade classrooms was designed to evaluate the transference of discussions from Philosophy for Children (P4C), a programme that fostered a CA stance, to a persuasive essay, an interview and a recall of argumentative text. Though the P4C students did engage in more dialogic interactions, they performed similarly in terms of a CA stance to the regular instruction students on the post-intervention measures, including writing. This study seeks to further examine the impact of Quality Talk on students' writing using a CA stance. Furthermore, research on Quality Talk has been largely limited by site and subject, using face-to-face discussions. There has been little empirical research in the field of dialogical discussions in secondary schools (Higham et al., 2014) and research that has been conducted has been in junior secondary, years 7 and 8 and the focus has been on science (Scott et al., 2006).

The current study

Participants

The students and teachers were from three co-educational secondary schools in Auckland, New Zealand. One school was classified as low socioeconomic status, one mid–low socioeconomic status and one mid–high socioeconomic status. Based on self-report, the students were from a variety of ethnic backgrounds, including New Zealand Pakeha/European (55%), Pacific Island heritage (25%), Maori (11%), Asian (7%), Fiji Indian and Indian (3%), and other, consisting primarily of Middle Eastern and Eastern European (7%). These proportions are broadly representative of the ethnic make-up of the local population (Statistics New Zealand, 2015).

Eight teachers agreed to participate in the study, ranging in experience from a first year teacher to a teacher who had taught for almost 40 years. Five of these teachers were female and three were male, teaching either English or Geography. Principals were reluctant to provide non-intervention classes because they did not want students in their school ‘missing out' on an intervention that could have a beneficial result, even though we offered to train non-intervention teachers in Quality Talk upon completion of the study. One of the more experienced teachers, whose students were comparatively high performing, offered to participate as a non-intervention teacher.

As a result of the reluctance to participate in the non-intervention group, there were seven intervention teachers and one non-intervention teacher. Following a true experimental design, ensuring the number of intervention and non-intervention classes was equal and sufficiently matched at baseline, would have strengthened our ability to determine the effectiveness of Quality Talk. However, since this was not possible, it was considered advantageous that this classroom's performance was better than average at baseline, since this went against our hypothesis that Quality Talk would improve student performance. Ethics approval for the study was obtained from the University of Auckland.

Procedure

The different phases of the study are presented below.

Measures and coding

A discourse coding manual was developed by Soter and team and has been trialled and used for analysing the dialogical discussions using the construct of Quality Talk (Soter et al., 2006). Permission was given to the researcher to use this manual by the authors. Teacher and student responses from both the face-to-face group discussions and online discussions and intervention and non-intervention classes were coded. The students were given randomised numbers by the coder but for the purposes of dissemination of the transcripts, made up names have been given to assure anonymity (see Table 1).

Results

Differences in types of questions pre- and post-intervention

The results of this study indicated that the construct of Quality Talk had an effect on the nature of student interactions. This is demonstrated visually in Figure 1, which shows the mean number of different question types comparing students in the non-intervention class with those in the intervention classes over time. The figure suggests that the number of ‘authentic', ‘uptake' and ‘high-level' questions increased from baseline to Time Two, then dropped back somewhat from Time Two to Time Three for the intervention classrooms, but remained higher than at baseline. In contrast, the number of these questions generally declined in the non-intervention class. Repeated measures analysis of variance (ANOVA) of the student-level data revealed that the interaction effect between time and classroom (non-intervention or intervention) was significant for these three question types [F(2,314) > 3, p < 0.05 in all cases]. Analysis of the simple main effects revealed that the decline in the number of authentic, uptake and high-level questions per student was significant within the non-intervention class. In contrast, the increased usage of these three question types was significant in the intervention classes [F(2,272) > 3, p < 0.05]. These changes are reflected in the transcripts of the students. The following two extracts show an example of the shift in behaviour in the way students in an intervention class interacted with each other from baseline to post-intervention.

In the first extract from the baseline (Time One) transcript, three students are discussing an authentic question—what motivates a director to make films?

Elja

What things do you think motivate film directors to have an overall message in mind? (Assigned question) A script.

Lucy

Yeah, money and helping the teenagers of today by sending them moral messages through film.

Elja

It's not really the directors is it? It's the script writers who do that. The directors are just like …. (Authentic, uptake, low level question)

Lucy

No but the director uses like lighting and film to convey the script writing. The director overall sees it. Examples? What are examples? (Assigned question)

Thomas

He gets the most credit for doing shit all.

Lucy

Director gets to sound angry and shout at people, I suppose.

Elja

It's the ADs [assistant directors] who do most of it.

Teacher

(joins the group) Shall we go on to number three, guys? What motivates …. (Managing the discourse)

Logan

Real life scenarios.

Lucy

Getting a pay cheque afterwards.

Teacher

Getting paid.

Lucy

Money and entertaining the audience so they get extra dollars.

Elja

Fame.

Bethany

I think they are trying to like show their own opinion and like their … try that as well.

Elja

Selfish.

This baseline (Time One) extract shows that the students respond mostly in a monologic fashion (Wegerif, 2013), that is, they respond with a statement but do not interact in dialogue through the use of uptake questions to elicit more thoughtful responses from each other. The students mostly stay on task and are listening to each other but appear to not have the skills necessary to draw out a richer dialogue. The teacher enters the conversation and immediately asks a procedural question without first listening to what is being said.

The next extract shows the same students in discussion following the Quality Talk intervention. The students are discussing the film, The Shawshank Redemption, and whether or not it shows that people change, or rather just have momentary steps outside their true character.

Lucy

What change do we see in Andy throughout the film? (Authentic question, high level question)

Logan

Andy changes in emotions.

Lucy

How? (Authentic question, uptake question, high level question)

Brandon

He's silent for the first few months and the first person he goes to is Red.

Logan

Andy becomes a lot more confident in both himself and his personality.

Lucy

But is that … he didn't really change, that was his true character, he was just hiding it away because of his environment? (Authentic question, uptake question, high level question)

Logan

Over time he became more confident in prison until he was the top dog in the prison show.

Lucy

But he was confident when he was outside the prison as well. He was still a confident … like his character is of a confident nature, he just didn't show that while he was in prison because he was scared. (Uptake statement)

Logan

Was he scared? (Authentic question, uptake question, high level question)

Lucy

Well you could imagine he would be pretty scared if he was shoved in prison for something he didn't do.

Brandon

And the expression on his face was fear, like something terrified. Especially when he got beaten and all that. (Uptake statement)

Lucy

Yeah it would have been pretty traumatic actually. (Uptake statement)

Logan

Did prison change Andy? (Authentic question, high level question)

Brandon

Um, he went in … .

Thomas

He doesn't [does?] revert back to some of his old ways, like he chooses to do the banking for the guards, so you can see he still does have his former self in him. (Elaborated explanation)

Logan

That's not his former self, that's just his …. (Uptake statement)

Thomas

It's what he loves though. (Uptake statement)

Logan

His former self is his character though. (Uptake statement)

Brandon

Well he did the same thing. He was an accountant outside of … or banker slash accountant … outside the prison, and then when he went into prison, he did the same sort of thing. (Uptake statement)

The conversation post-Quality Talk intervention has now become dialogic (Wegerif, 2013) because the discussion includes at least two student questions and the students are interacting with each other rather than each responding with a statement. The students' use of authentic, uptake and high-level questions has increased and consequently this results in a dialogic spell (Nystrand et al., 2003). In turn, the dialogue appears to be more thoughtful and the students engage in more complex responses.

Dialogic spells and exploratory talk pre- and post-intervention

As dialogic and exploratory spells reflect the wider conversation, rather than a single statement, the results could not be analysed at the student level. Rather, the data are presented as the mean percentage of dialogic spells and exploratory talk for the non-intervention and intervention classes at Time One, Time Two and Time Three. The most common shift was in the proportion of dialogic spells. The non-intervention class was initially somewhat higher than the intervention class on dialogic spells at baseline, but a z-test of the difference in proportions indicated that this difference was non-significant (Z = 0.62, p = 0.54). The proportion of dialogic spells increased markedly in the intervention classes, and was significantly higher than in the non-intervention class at Time Two (Z = 0.62, p = 0.03) and Three (Z = 2.29, p = 0.02). The increase in dialogic spells is indicative of the increase in authentic questioning, uptake questioning and high-level questioning, as a dialogic spell is dependent on students asking questions of each other.

This transcript differed from many of the transcripts coded because it is an example of Exploratory Talk. Here the students challenge with disputes and students subsequently respond to being challenged and justify their thoughts with evidence.

Differences in written text

Figure 3 shows that the mean number of CA responses in the students' pre- and post-intervention essays increased within the intervention classes and decreased in the non-intervention class. Repeated measures ANOVA indicated that there was an interaction between time and classroom (non-intervention or intervention), with respect to the mean number of CA statements per student [F(1,157) = 11.33, p = 0.001]. Analysis of the simple main effects revealed that the decline in the number of CA questions per student was non-significant within the non-intervention class [F(1,21) = 2.69, p = 0.12]. In contrast, the increased usage of CA responses was significant in the intervention classes [F(1,136) = 41.22, p < 0.001]. This equates to an effect size (Cohen's d) of –0.4 within the non-intervention class and 0.92 within the intervention classes.

Coded questionnaire results

Other findings

Though there were few exploratory talk episodes, male students in the low and low- to mid-socioeconomic schools in particular wrote that the opportunity for arguing presented by dialogical discussions was what motivated them to participate: ‘the argumentative part', ‘cause I was keen on proving some wrong and right', ‘someone disagreeing with your comment makes you want to challenge the person' and ‘well the fact the we hold different viewpoints, encourages me to get my point across'.

Discussion

This study trialled the use of Quality Talk in small group heterogeneous discussions within the context of a film unit and a geography unit at three secondary schools. The research questions were (a) What happens to the nature of the interactions of students during group discussions once they have been taught the construct Quality Talk? and (b) What is the effect of using Quality Talk on students' ability to write CA statements?

The study found that students increased their use of authentic questions, uptake questions and high-level questions, and that this was in turn reflected in a greater use of critical analytical thinking in written work following the intervention. This is consistent with the findings of Wilkinson et al. (2010b) who asserted that productive discussions occur when students hold the floor for extended periods of time, when students are prompted to use authentic questions and when discussion incorporates a high degree of uptake.

This study found that students engaged in more Dialogic Spells than Exploratory Talk episodes because though students challenged each other, they were far less successful at using this challenge to unravel a shared ‘higher' understanding (Barnes, 2008; Mercer, 2008; Wegerif, 2010) and therefore the challenge could not be coded as Exploratory Talk. Instead of challenges, the students in this study seemed to prefer to ask each other uptake and high-level questions and, in turn, these appeared to contribute to students encouraging each other to delve deeper. As Exploratory Talk was designed for problem solving, and the film unit and geography unit were not designed specifically for problem solving this could also explain why there were fewer challenges. However, the students in one class who did engage in both Dialogic Spells and Exploratory Talk scored the greatest number of high levels of CA statements in their post-intervention essays. Therefore, rather than dismissing the use of challenge because the students did not appear to have the skills to use a challenge to gain a higher understanding the use of challenge may be worth exploring further. An implication of this finding is that because the classroom environment feature of respect is so important to adolescent students a teacher could encourage the use of disagreements and disputes with a conversation on how to do so respectfully. Perhaps teachers could indicate that to be challenged does not imply disrespect and explore why we may view being challenged in a positive light.

Accepting challenges as normal may not be an easy shift in thinking to encourage with adolescent learners however. Burbules (1993) cautioned that when students are quick to judge it can threaten the fabric of a dialogical relation, turning it from a co-operative to a competitive and from a trusting and respectful to a predominately suspicious interaction. Many students in the study commented that trust and respect were important to their willingness to participate and so encouraging challenge could result in less confidence from the students to interact with each other.

Metacognitive awareness of some students seemed to be factors that contributed to differences in why a number of students scored the highest critical analytical results in their post-intervention essays. This is not an unsurprising outcome because Veenman (2008) estimated that metacognitive skillfulness accounted for 40% of variance in learning outcomes. A theme from the study that identified students who scored high critical analytical results were those students who were able to articulate the difference between a normal group discussion and a dialogical discussion. This type of metacognitive knowledge is conditional knowledge (Schraw & Moshman, 1995) as it pertains to declarative knowledge about when a certain metacognitive strategy should be applied and to what purpose. These students had confidence in knowing what types of talk was conducive to dialogical talk and when to use that type of talk. It was this type of conditional knowledge that seemed to be lacking in the intervention class as the students did not have the metacognitive awareness of what type of questions and talk was required to engage in a critical discussion.

Opportunity for metacognitive thinking in past research on dialogical discussions has centred around reflective opportunities for teachers through the use of video (Roskos & Bo ehlen, 2001; Walshaw & Anthony, 2008) An example of a reflective tool for teachers has been developed by Wilkinson et al. (2010a) who have trialled Talk Assessment Tool for Teachers (TATT) to support teachers' professional development in learning to conduct discussions. This involved the teacher and a discourse coach viewing a video of recently completed discussion and together completing a close analysis of the exchanges within the dialogue. In this study, it was the students who were given the opportunity to engage in metacognitive reflections based on the transcripts of each previous discussion. The function of this activity was to evaluate and interpret the outcome, and to learn from the course of action for future occasions (Veenman, 2011). Opportunities for students to evaluate their own performances as individuals and collectively in the pursuit of complex talking that will initiate conversations that include critical thought is therefore worth pursuing in future studies as the students are the centre of dialogical discussions.

Students enjoyed seeing the transcripts from their previous group discussions, as evidenced by the laughter that could be heard as they read through the scripts. They were able to critique and evaluate the dialogue according to the construct of Quality Talk and address this in their next discussion. Busy secondary teachers do not have time to type transcripts from discussions but simply putting an iPad at each group's table and video-recording the discussions could be a viable alternative. The groups could view their discussions and assess the degree to which they implemented the components of Quality Talk. There is always a risk in a study such as this that students will mimic the type of dialogue expected in the training. One of the boys in the study did this humorously as the transcripts combined with the video footage revealed by sarcastically asking uptake questions to his group. Unexpectedly, the other students in his group took no notice of his tone and actually answered his uptake questions. Therefore, the risk of students mimicking or parroting what was required appeared to not be a problem.

While the results indicated a positive effect of Quality Talk, there were some limitations that need to be acknowledged. First, we were only able to recruit a single non-intervention class. The Principals in the study were reluctant to have non-intervention classes because they were concerned senior secondary students may miss out on an intervention that could improve performance, at a time when this was most critical because of national certificate qualifications within these year levels. While this was partially mitigated by the fact that this class had higher baseline performance and a more experienced teacher than students in the intervention classes, it remains possible that the differences in progress related to something other than the Quality Talk intervention. In addition, students in both the intervention classes and the non-intervention class were invited to participate in online discussions for homework following the face-to-face discussions at school. These discussions may have supported greater gains among students who opted to participate in these discussions. However, the transcripts for the face-to-face group discussions reveal levels of complexity that other studies in dialogue have failed to show as a result of the ongoing difficulty of the nature of the interactions remaining at Initiation, recitation, evaluation (IRE; Alexander, 2008).

We believe that the transcripts demonstrate a real and meaningful change within the intervention classrooms, which would have been unlikely to occur without the intervention. However, the single non-intervention class precludes definitive conclusions—suggesting the need for further research in this important area.

Conclusion

The study was an important contribution in the field of dialogical education because it found that the use of Quality Talk was successful outside of text comprehension and at senior secondary level to increase CA thinking. The call for more research in the use of dialogic teaching in secondary schools has gained momentum in the past few years as understanding develops about the role of talk in learning. The majority of these arguments cite improvements to student critical thinking and retention (Higham et al., 2014). That dialogic spells increased the level of interactive dialogue between the students in the intervention classes is an important finding because interaction is important to critical thinking. Habermas (1974) claimed an important component of critical learning is the reflective process and recommended groups of people sharing informed judgements to generate critical ideas or theories about the validity of the issues under consideration. Critical learning is a reflective activity with critical intent that enables students to engage socially in learning tasks and collaborative problem solving. Therefore, opportunities for students to have rich and complex discussions, not only offer students the chance to engage on a deeper cognitive level, but also offer opportunity for students to learn more about themselves and others. A psychological task of adolescence is to form identity and fundamental to this formation of self are the cultural communities in which we are nurtured (Taylor, 1989). These communities provide the moral and social frameworks that we use to describe who we are, how we see others, what the situations that we encounter in life mean to us, what our options are, how we evaluate them and what actions we undertake. Without the language of the community we would not be able to make sense of our lives (Taylor, 1989). Quality Talk is a pragmatic tool to help develop this sense of self within community and ultimately a sense of identity for adolescent learners.