Schedule: 2009-09-10 (15:45 - 16:30)
Parallel Session 2 (Room A-34)

Title: Difficulties and constraints of computer assisted language testing of Spanish DELE with Chinese students.
Authors: Jose Macario de Siqueira Rocha, Diana González, Jesús García Laborda, Salvador Prefasi
Abstract: The design of platforms for computer tests has a long history. From the very first platforms of the 70’s, which included multiple choice questions, to the ones combining different tasks such as TOEFL there have been important changes to which linguists have hardly paid attention. However, technical specifications build and form the exam construct. For instance, it is essential to bear in mind the specifications of the TOEFL platform to be able to understand how their tasks are and what the effect is on the students learning a new language. Many prior surveys support this relationship between construct and design of evaluation platforms when dealing with image use and audiovisual repertoires (García Laborda and Pardo Suárez, 2008; Breland, Lee, and Muraki, 2004), ergonomics (García Laborda and Magal Royo, 2007; Hansen, Forer, and Lee, 2004), familiarity with the computer (Taylor, Jamieson, Eignor, and Kirsch, 1998) and context (Wolfe y Manalo, 2005). All these factors have to be clearly defined in order for the evaluation to be valid and to have a high degree of reliability (García Laborda and Cristina Navarro, 2008).
This work shows the technical and technological-linguistics aspects needed to develop exam platforms of projects such as Selector (Regional Government of Valencia), Ingenio (Universidad Politécnica de Valencia), CALL@C&S (European Union) and Paulex Universitas (Spanish Ministry of Science and Innovation). The examples provided are the result of more than 8 years of research in the field. The main goal is to compare and validate some technical aspects with the specific needs of Chinese students learning Spanish by using technological resources. To that purpose, an investigation has been carried out at the Universidad Politécnica de Valencia with 48 Chinese students that are learning Spanish at this university. The course programme includes language and civilization intensive courses, since the students undergo instruction in preparation for the intermediate exam of DELE- Diploma de Español como Lengua Extranjera - the official certificate of Spanish as foreign language granted by the Instituto Cervantes on behalf of the Spanish Ministry of Education. This investigation compares the results of previous paper-based tests with those of computer-based exams reproducing the DELE exam. It also analyzes the difficulties experienced by the students during the realization of the computer-based exam which are mainly due to the insecurity felt when making a computer exam, the adaptation to a keyboard other than the Chinese, as well as other adaptive aspects as the lack of time management- students unable to complete the exam within the time required. The survey concludes that all these technological aspects need to be carefully analyzed from the different cultural and linguistic perspectives of students worldwide.
Keywords: e-learning; Assessment; Computer Assisted Language Testing; Chinese Students

Main topic: Assessment, testing, feedback and guidance in CALL

Biodata: Jose Macario de Siqueira, Diana González and Jesus Garcia Laborda are members of the CAMILLE Research Group at the Department of Applied Linguistics and Salvador Prefasi is part of the Research Group on Graphic Technologies at the Universidad Politécnica de Valencia, Spain.
Type of presentation Paper presentation

Paper category Research & Development

Target educational sector Higher education

Language of delivery English

EU-funded project Yes

Schedule:	2009-09-10 (15:45 - 16:30) Parallel Session 2 (Room A-34)
Title:	Difficulties and constraints of computer assisted language testing of Spanish DELE with Chinese students.
Authors:	Jose Macario de Siqueira Rocha, Diana González, Jesús García Laborda, Salvador Prefasi
Abstract:	The design of platforms for computer tests has a long history. From the very first platforms of the 70’s, which included multiple choice questions, to the ones combining different tasks such as TOEFL there have been important changes to which linguists have hardly paid attention. However, technical specifications build and form the exam construct. For instance, it is essential to bear in mind the specifications of the TOEFL platform to be able to understand how their tasks are and what the effect is on the students learning a new language. Many prior surveys support this relationship between construct and design of evaluation platforms when dealing with image use and audiovisual repertoires (García Laborda and Pardo Suárez, 2008; Breland, Lee, and Muraki, 2004), ergonomics (García Laborda and Magal Royo, 2007; Hansen, Forer, and Lee, 2004), familiarity with the computer (Taylor, Jamieson, Eignor, and Kirsch, 1998) and context (Wolfe y Manalo, 2005). All these factors have to be clearly defined in order for the evaluation to be valid and to have a high degree of reliability (García Laborda and Cristina Navarro, 2008). This work shows the technical and technological-linguistics aspects needed to develop exam platforms of projects such as Selector (Regional Government of Valencia), Ingenio (Universidad Politécnica de Valencia), CALL@C&S (European Union) and Paulex Universitas (Spanish Ministry of Science and Innovation). The examples provided are the result of more than 8 years of research in the field. The main goal is to compare and validate some technical aspects with the specific needs of Chinese students learning Spanish by using technological resources. To that purpose, an investigation has been carried out at the Universidad Politécnica de Valencia with 48 Chinese students that are learning Spanish at this university. The course programme includes language and civilization intensive courses, since the students undergo instruction in preparation for the intermediate exam of DELE- Diploma de Español como Lengua Extranjera - the official certificate of Spanish as foreign language granted by the Instituto Cervantes on behalf of the Spanish Ministry of Education. This investigation compares the results of previous paper-based tests with those of computer-based exams reproducing the DELE exam. It also analyzes the difficulties experienced by the students during the realization of the computer-based exam which are mainly due to the insecurity felt when making a computer exam, the adaptation to a keyboard other than the Chinese, as well as other adaptive aspects as the lack of time management- students unable to complete the exam within the time required. The survey concludes that all these technological aspects need to be carefully analyzed from the different cultural and linguistic perspectives of students worldwide.
Keywords:	e-learning; Assessment; Computer Assisted Language Testing; Chinese Students
Main topic:	Assessment, testing, feedback and guidance in CALL
Biodata:	Jose Macario de Siqueira, Diana González and Jesus Garcia Laborda are members of the CAMILLE Research Group at the Department of Applied Linguistics and Salvador Prefasi is part of the Research Group on Graphic Technologies at the Universidad Politécnica de Valencia, Spain.
Type of presentation	Paper presentation
Paper category	Research & Development
Target educational sector	Higher education
Language of delivery	English
EU-funded project	Yes