کتاب Assessing Interactional Competence through paired speaking tasks

Table of Contents

CHAPTER ONE: INTRODUCTION 2
1.1 Preliminaries 3
1.2 Statement of the problem 4
1.3 Significance of the study 5
1.4 Objectives of the study 6
1.5 Research questions 6
1.6 Definition of key terms 6
CHAPTER TWO: Literature Review 16
2.1 Overview 17
2.2 Communicative competence 18
2.3 Interactional competence 23
2.4 Interactional competence in paired and group speaking tests 26
2.5 Task and interlocutor impacts in paired and group speaking tests 38
2.6 Insights into interactional competence from theoretical and empirical research 47
2.7 Chapter summary 53
CHAPTER THREE: Method 55
3.1 Overview 56
3.2 Participants 56
3.3 Procedure 56
3.4 Materials 60
3.5 Data analysis 60
CHAPTER FOUR: Data Analysis and Results 61
4.1 Overview 62
4.2 Discussing spoken interaction model 62
4.3 Investigating the first research question 62
4.4 Investigating the second research question 69
4.5 Investigating the third research question 74
CHAPTER FIVE:Discussions, Conclusions, and Implications 77
5.1 Overview 78
5.2 Restatement of the problem 78
5.3 Major findings 78
5.4 Conclusion 85
5.5 Implications of the study 86
5.6 Limitations of the study 87
5.7 Suggestions for future research 87
References 89

Preliminaries
Language proficiency was first understood as the knowledge of structure (grammatical, lexical, and phonological) that, it was believed, the best be measured by discrete-point tests (Lado, 1961). With the rise of communicative language teaching in the 1970s, performance testing received significance. In 1980s, the language proficiency interview (e.g. the ACTFL oral proficiency interview, or OPI), that was a specific type of performance test, became common. However, there was a large impact of asymmetry of power in examinee-examiner interviews (Taylor, 2001). At last, paired speaking tasks were considered a successful format for eliciting a range of linguistic and paralinguistic features (Brooks, 2009; Davis, 2009; Dimitrova-Galaczi, 2004; Nacatsuhara, 2004; Ducasse & Brown, 2009; May, 2011b).
As the communicative turn in language teaching led to an increase in pair and group work activities in L2 classrooms, pair and group work also became more common in communicative-based assessments (van Moere, 2012). In paired test tasks, the test taker is matched with one peer-interlocutor, and the pair is given tasks to obtain an interaction that is constructed by the two participants. During the test, an examiner is present and usually guides the test but is not engaged in the candidates’ interaction. The examiner’s role is to observe and evaluate the performance.
The currency of the paired test format can be partially found as a reaction to the criticism that has been leveled against the asymmetric nature of traditional interview-style oral tests (e.g. the face to face OPI (Oral Proficiency Interview); the tape mediated Simulated Oral Proficiency Interview, or SOPI), which have been widely practiced for oral proficiency testing. Particularly, the OPI of the American Council on the Teaching of Foreign Languages (ACTFL) has been the subject of argument as to whether or not it determines speaking competence in a conversation, meaning face to face relationship in a real life context. In a now classic paper, van Lier (1989) compared features of the OPI speech with those of a natural conversation. He discovered significant differences between the two interactions, which increased concerns related to validity of the OPI.
With his influential work, van Lier (1989) drew attention to the empirical analysis of the features of discourse in face to face oral evaluation. He called for further research that would look at the turn-by-turn ordinal interaction in oral language proficiency interviews and the way participants make the oral test discourse. Following van Lier’s (1989) call, several studies occurred in the 1990s regarding language proficiency interviews in general and the OPI in particular from a range of discourse analytic perspectives (e.g. Brown, 2003, 2005; Johnson, 2000, 2001; Lazaraton, 2002).
Data from these studies demonstrated the occurrence of an asymmetrical power relationship between examiner and examinee in face to face oral interviews that puts the interviewer in control over turn-taking and topic initiation. In comparison, discourse data from paired speaking tests revealed that the language features elicited tend to be equally distributed between the candidates (Brooks, 2009; Taylor, 2001). Moreover, the turns produced in the paired format are longer and more balanced across the interlocutors (Csepes, 2009). Thus, the paired test format take out predominantly everyday-like balanced interactions between language users, where one speaker does not dominate the other.
In addition, selecting a peer format for a speaking test should have a positive washback effect from teaching to testing and vice versa, since pair-work activities are commonly used in communicative language teaching (Egyud & Glover, 2001; Fulcher, 2003; van Moere, 2012). Students also seem to prefer the peer type to individual speaking tasks with an examiner, as student answers to questionnaires show (Egyud & Glover, 2001; May, 2000). Van Moere (2012) also emphasizes that paired tests are less time consuming and less expensive compared to examiner-candidate oral interview tests, as two individuals can be evaluated at the same time. Moreover, since examiners do not have to be as interlocutors in a paired test context, they can entirely concentrate on their function as evaluators (van Moere, 2012).

1.2 Statement of the problem
The co-construction between two or more interlocutors is interactional competence (Hall & Pekarek, 2011) yet it is not clear what the co-construction includes (Gan, 2010; Brooks, 2009). The construct of interactional competence needs searching evidence both at macro and micro levels. At a macro-level, there are different patterns of co-construction occurring in paired speaking tasks. At a micro-level, the co-construction is described through a series of interaction features. However, it is not clear what factors determine or predict interaction performance. Possible factors contain interaction features or interaction patterns in general. To move beyond description, it is also necessary to investigate whether interaction features predict interaction performance (Wang, 2015).
Although, several concerns have been raised when conducting paired speaking tasks in L2 assessment. First, L2 oral interaction in language tests is considered complex (Chalhoub-Deville, 2003; May, 2010; Taylor & Wigglesworth, 2009). One chief reason is the complicated nature of the construct itself. Interactional competence points to the co-construction between two or more interlocutors (e.g. Hall & Pekarek, 2011; He & Young, 1998; Jacoby & Ochs, 1995; Kramsch, 1986). Lastly, appropriate rating scales are required to measure interaction performance (Wang, 2015).
There are some aspects like individuals’ features and task styles which add to the complexity of paired speaking tasks in L2 assessment. Studies on interlocutors’ differences such as familiarity, gender, personality and proficiency variability have shown the great complexity and possible variability of paired oral tasks (Berry, 2007; Brown & McNamara, 2004; O’Sullivan, 2000, 2002). Therefore, it is compulsory to investigate the possible task type influence on interaction performance and whether task styles influence the classification of interaction features. In summary, this study aimed to search the complex nature of peer-peer interaction by addressing each of these issues in the context of paired speaking assessment.

1.3 Significance of the study
Studies on task-based assessment might add to the theoretical, methodological, and practical perspectives in the field of L2 speaking teaching and testing. Having understood that there is a social dimension to proficiency and face to face interaction (MacNamara & Roever, 2006), language testers have started to absorb an interactional competence perspective into language testing. Operationalization studies have described the construct of interactional competence in paired speaking tests chiefly in terms of balanced and collaborative turn-taking behavior and topic management between test takers (Ducasse & Brown, 2009; Galaczi, 2008; May, 2009, 2011).
In comparison to oral interviews, variability in paired and group oral testing has not received much attention thus far. Only a few studies have examined the impact of test takers’ proficiency levels (Csepes, 2009; Davis, 2009; Iwashita, 1996) and test task characteristics (Nakatsuhara, 2013; van Moere, 2007) on test discourse and ratings in group speaking tests. Even though the test task is known to cause variation in the speaking ability being tested, research in paired testing so far has primarily contained only one paired task; namely a discussion task (Ducasse & Brown, 2009; Galaczi, 2004, 2008; May, 2011). Also referred to as the “defult task” (p. 117) by van Moere (2007) in his research on group speaking tests, the discussion tasks is believed to naturally elicit conversation-like discourse in paired testing contexts. The impact of other paired tasks may have on test performance in paired assessment has been largely neglected.
Another issue in paired testing concerns the appropriate matching of peer-interlocutors. Swain (2001) made this point clear when she stated that “who one is paired or grouped with, is not unimportant” (p. 296). Foot (1999) argued on the same thing when he claimed that a candidate’s speaking ability seems to be highly sensitive to the interlocutor with whom he or she is paired. In a similar vein, Norton (2005) suggested that the acquaintance level of the test takers may have an effect on paired test discourse and scores; she therefore calls for research that examines the performance of paired test takers who are friends and who are strangers.

1.4 Objectives of the study
1. Finding the interactional features of free discussion and story completion paired tasks of advanced Iranian EFL learners
2. The degree to which task types interact with levels and features of interaction quality
3. Measuring micro-level interactional competence to elicit which task type causes more interaction between paired interlocutors

1.5 Research questions
1. What are the interactional features of free discussion paired tasks of advanced Iranian EFL learners?
2. What are the interactional features of story-completion paired tasks of advanced Iranian EFL learners?
3. Which task type (free discussion or story-completion) causes more interaction among advanced Iranian EFL learners?

1.6 Definition of key terms
• L2 speaking proficiency. It refers to skills and strategies that students use in relation to second language speaking (Brown, 2004).

• Interactional Competence. It means someone’s ability to use a range of characteristics in roles of the listener and speaker to remake communication with other interlocutors to reach communicative goals (Hall, 1995; He & Young, 1998).

• Task. It is a performance-based activity with a communicative goal to achieve proper language performance from interlocutors (Bachman, 2002; Brown, Hudson, & Bonk, 2002).

• Paired speaking task. This task requires peer-peer interaction through the perspectives of second language acquisition and task-based language assessment (Ducasse & Brown, 2009; May, 2010; Taylor & Wigglesworth, 2009). In paired speaking tasks, there are two interlocutors who speak together and learn to work in cooperation, thus there is variety in classroom routines and activities, an opportunity for students to practice social skills, in which students are “center stage” rather than the teacher and on-task behavior (Curtain & Dahlberg, 2010, p. 98).

• Free discussion task. It is a kind of open and divergent task in which there is not any predetermined correct outcomes. A divergent task allows different views from participants (Ellis, 2003). In a free discussion task, there is a topic that interlocutors should talk about it.

Story completion task: It is a kind of closed and convergent task in which participants must reach a “single, correct solution or one of a small finite set of solutions” (Ellis, 2003, p.89). A convergent task expects participants to agree on a solution before ending their interaction. In a story completion task, there are some pictures for participants to put them in order to make a correct story.

1 Overview
Spoken interaction is an essential but also complex struggle. It is dynamic and co-constructed between interlocutors. It is interchangeable and those involved are both pro-active (making things happen rather than reactive to events) and re-active at the same time, concurrently deconstructing messages as listeners and constructing their own message as speakers. Interaction is not necessarily linear, predictable, or tidy, and it is strongly shaped by diverse personal cognitive and contextual factors. It is, from a sociological perspective, the “primordial site of sociality” (Schegloff, 1986, p. 112), and, from a cognitive psychology perspective, something we are wired for: “humans are designed for dialogue rather than monologue” (Garrod & Pickering, 2004, p. 8).
In addition, spoken interaction in a second/foreign (L2) language has assumed greater importance in the last two decades, in light of the growing role of the communicative approach to language teaching, learning, and assessment. As speaking tests have evolved to capture interaction (e.g., through using paired or group formats) and as technology has started to play a significant role in assessment, a need has developed to more accurately understand and describe the construct of interactional competence. Our current understanding of interactional competence, with its implications for L2 teaching, learning, and assessment, has not received extensive and focused attention, apart from a handful of theoretical papers and empirical studies(e.g., Chalhoub-Deville, 2003; East, 2016; Galaczi, 2008, 2014; He & Young, 1998; Kramsch, 1986; May, 2009; Nakatsuhara, 2013; Taylor & Wigglesworth, 2009).
The explanations of patterns of co-construction by interlocutors and the role of task completion both help our understanding of interactional competence at a macro-level. Another strand understands interaction competence at the micro-level. In other words, independent scores are presented to each interlocutor in a speaking task based on their use of interaction features. one explanation of interaction features was suggested by Oskaar (1990), “the ability of a person, in interactional situations to carry out and interpret verbal, paralinguistic, nonverbal, and extra verbal communicative actions in two roles, that of the speaker and that of the hearer” (p. 530). This explanation emphasizes the individuals’ role in the interaction as both a speaker and hearer who acts and reacts to a range of communicative events. This definition stresses on the micro-level of interactional competence. Kasper (2006) presented interactional competence through verbal features such as taking turns in a talk and fixing problems. Interactional features can also be used to demonstrate interactional competence.
Since the previous work has not been truly understandable to contain a wider range of other possible interactional features, attempts are required to find more features and use them to clarify the construct of interactional competence at a micro-level, particularly because the construct is still being debated and developed (Barraja-Rohan, 2011). In conclusion, interactional competence is a dynamic process which includes two sets of features: an overall quality as a macro-level measure revealed by patterns of the co-construction and task completion, and use of a wide range of interaction features as a micro-level measure.
In this chapter, it is first presented the theoretical frameworks of communicative competence and interactional competence, as they point to two various types of interaction with one being more cognitive in nature and the other more socially oriented. After reviewing the conceptualization of interactional competence in Second Language Acquisition (SLA) research, it is prepared an overview of how interactional competence has been used in language testing. First it is stated an interaction-based approach to examine construct definition that correspond to the conceptualization of interactional competence within SLA research. Second, it is presented how interactional competence has been operationalized in paired and grouped peer-peer speaking exams. At last, it is ended with a review of previous studies about the effect of task and test-taker/interlocutor proficiency and language ability on paired and group test discourse and ratings.

2.2 Communicative competence
Language examiners trust models that prepare a theoretical explanation of what it means to communicate, to understand and use a second language (Fulcher & Davidson, 2007). To sum up, models describe constructs (Fulcher, 2010) or the “theoretical rationale” (McNamara, 1996, p. 49) that allows test developers to draw valid inferences from test scores and test performance to test-taker abilities (Fulcher & Davidson, 2007; McNamara, 1996). For test development, language testers choose constructs from the theoretical model to merge them in an operational test framework for a special test with a special aim (Chalhoub-Deville, 1997; Fulcher, 2010; Fulcher & Davidson, 2007).
Fulcher (2010) states that Lado, a pioneer in the field of language testing, made a model of language use that included constructs. After Lado, a lot of models have been advanced since the 1970s, beginning with Hymes’s (1972a) communicative competence model, which had a chief influence on the models that followed (Bachman, 1990; Bachman & Palmer, 1996, 2010; Canale, 1983; Canale and Swain, 1980; Celce-Murcia, Dornyei, & Thurrell, 1995).
McNamara (1996, p. 48) presents that all models of second language (L2) communicative ability share three dimensions: a) language knowledge (model of knowledge), b) underlying non-linguistic factors that relate to language use ability (model of performance), c) specific instances of language use (actual language use). The first two dimensions are usually pointed to in the variety of models as communicative competence or communicative language ability. The third dimension refers to actual language use and not to the knowledge and ability for use that is necessary to communicate in a second language (Fulcher & Davidson, 2007). McNamara (2001) emphasizes that communicative competence is one of the most fundamental concepts in the field of language testing. In the following part, the focused is on communicative competence models, in particular Canale and Swain’s (1980) communicative competence model and Bachman’s (1990; Bachman and Palmer, 1996, 2010) communicative language ability model, which may be are the most effective models of communicative competence after Hymes (1972a). Thus, we begun with Hymes’s (1972a) model and then turn to Canale and Swain’s (1980) communicative competence model, before Bachman’s (1990; Bachman & Palmer, 1996, 2010) model of communicative language ability, in chronological order.
2.2.1 Hymes (1972a)
Hymes’s (1972a) model of communicative competence was the first model that appeared after Lado’s (1961). Disagreeing with Chomsky’s (1965) notion of competence that point only to the knowledge of tacit grammar rules, Hymes (1972a) introduced the term communicative competence. For him, communicative competence also contains contextual or sociolinguistic knowledge instead of grammar knowledge alone. Hymes (1972a) claims that language users need the ability to appropriately use grammar rules, depending on the social context given (cf. Johnson, 2004).
In addition to the model of knowledge, Hymes’s (1972a) communicative competence model contains a model of performance, which differs from Chomsky’s (1965) understanding of performance as actual language use in real positions. By contrast, Hymes’s (1972a) model of performance demonstrates underlying rules of performance, also termed ability for use. This type of performance which has not been noticed as actual performance, is discussed in rather broad terms. It reveals cognitive and non-cognitive factors that are not related to language performance, like motivation.
With reference to Goffman (1967), Hymes (1972a) presents some other factors in the ability for use dimension, that is, “capacities in interaction such as courage, gameness, gallantry, composure, presence of mind, dignity, stage confidence” (p. 283). According to Hymes (1972a), not only an individual’s underlying ability is important for communication, but also the capacity to use this ability in different contexts.
2.2.2 Canale and Swain (1980)
Canale and Swain (1980) clarified the concept of communicative competence by classifying it into three competencies: grammatical competence (knowledge of grammar, lexis, morphology, syntax, semantics, phonology), strategic competence (ability to overcome communicative difficulties) and, following Hymes (1972a), sociolinguistic competence (sociocultural rules of use and rules of discourse). The different competencies are found to interact with one another. At the same time, Canale and Swain (1980) reject Hymes’s (1972a) underlying performance model from their model of communicative competence, arguing that it cannot be modeled. Communicative competence is assumed as the recognition of the various components of communicative competence (grammatical, strategic, and sociolinguistic competence), containing the interactions among these competencies in real communicative L2 positions. Whether consciously or unconsciously, Canale and Swain (1980) bring the concept of interaction into the discussion on communicative competence (Johnson, 2004).
Canale (1983) later states that the model fails in confirming internal relationships among the various components of communicative competence and among the components and other factors in real performance. Other experts (e.g. Johnson, 2001, 2004; McNamara, 1996; Shohamy, 1998) also refer to this imperfection.
Related to inadequate definition of internal constituent relations in Canale and Swain’s (1980) model, Johnson (2004) stresses that it is unclear whether and, if so, how the interactions among the competencies are conducted in an individual’s mind or happen in social context. She theorizes that the inclusion of strategic competence (the ability to use coping strategies to master communicative problems) points to the interaction with other interlocutors and thus at social interaction. However, the fact that Canale and Swain (1980) split up communicative competence from actual performance and put greater focus on the different competencies gives Johnson (2004) reason to suppose that interaction among various constituents of communicative competence is perceived to occur in individual’s mind, making interaction a mainly cognitive instead a social mater.
2.2.3 Bachman (1990)
Elaborating on Canale and Swain’s (1980) model, Bachman (1990) introduced his model of communicative language ability to display how the various constituents interrelate with each other and with the context in which language is used. To achieve this goal Bachman (1990) modeled strategic competence that he adopted from Canale and Swain (1980) and reconceptualized as a part of ability for use rather than linguistic knowledge. Bachman (1990) explains strategic competence as “a general ability, which enables an individual to make the most effective use of available abilities in carrying out a given task” (p. 106). Therefore, it refers to general underlying non-linguistic, cognitive factors or abilities, such as assessing, planning, and executing, which are used to achieve a communicative goal (Johnson, 2004; McNamara, 1996). The major function of strategic competence is that of connecting language competence to other knowledge structures (e.g. world knowledge) and to the context, in which communication occur. Bachman (1990) accordingly resolves the flaw of Canale and Swain’s (1980) model, which lacks a mechanism that discuss how the various competencies interrelate.
2.2.4 Bachman and Palmer (1996)
Bachman and palmer (1996) improved Bachman’s (1990) model. It was the addition of schemata. Affective schemata are defined as the “affective or emotional correlates of topical knowledge” (Bachman & Palmer, 1996, p. 65). They permit the language user to evaluate the current task setting and decide to what extent emotional experiences from the same previous contexts can be used in the response.
Extra significant constituents of the model are different language user characteristics, like the individual’s personal characteristics (e.g. age, gender, nationality, native language, educational level), topical knowledge or real-world knowledge, and language knowledge (e.g. textual, grammatical, lexical, functional, and sociocultural knowledge). In the most recent version of the model, Bachman and Palmer (2010) added another language user characteristic, namely cognitive strategies, which language users employ to execute plans in comprehending and producing utterances.
In Bachman and Palmer’s (1996) model, the dimension of ability for use consists of topical knowledge, affective schemata, and strategic competence. Both topical knowledge (knowledge structures, knowledge of the world) and affective schemata are cognitive structures, which can be understood as last made experiences or memories in language use that are drawn on as resources in communicative positions (McNamara, 1996). Furthermore, compared to Bachman’s earlier model, strategic competence has now been reconceptualized as a set of metacognitive strategies. Three different metacognitive constituents are differentiated: “goal-setting (deciding on what one is going to do), assessment (taking stock of what is needed, what one has to work with, and how well one has done), and planning (deciding how to use what one has)” (Bachman & Palmer, 1996, pp. 70-75).