Pragmatically annotated VOICE data for function-based ELF analyses: reliability and potentials of AI
Sprache des Vortragstitels:
Englisch
Original Tagungtitel:
14th International Conference of English as a Lingua Franca
Sprache des Tagungstitel:
Englisch
Original Kurzfassung:
Form-to-function approaches have been influential methodologies in English as a lingua franca (ELF) research. By contrast, function-based analyses taking the reverse order are less prominent. They require functional annotation of the pragmatic purposes that resources serve in communication but are ideal for studying variable and adaptive form-function relations as characteristic of ELF settings. This annotation is laborious and, according to Artstein and Poesio (2008, p. 555), often perceived to be less reliable than other forms of corpus annotation. At the same time, however, scholars deem pragmatic annotation highly desirable and useful for corpus-based pragmatic enquiries (cf. e.g. Clancy & O?Keeffe, 2015, p. 251).
The aim of this paper is to study the reliability of data from the Vienna-Oxford International Corpus of English (VOICE) (VOICE, 2021) annotated for pragmatic functions. For this purpose, the paper introduces the annotation system and provides examples of annotated VOICE data. The paper then discusses cases of agreement and disagreement between annotations assigned by three different human annotators. It reports Cohen?s Kappa and Krippendorff?s Alpha coefficients as statistical measurements indicating the reliability of the functionally annotated ELF data. The paper finally compares the human annotations to the way the same data samples have been annotated by the artificial intelligence (AI) system ChatGPT (OpenAI, 2023). The purpose is to give a brief outlook on the potentials and limitations of AI for pragmatically annotating ELF data.
Either way, be it human or AI-based, the annotation of pragmatic functions greatly enhances ELF data and paves the way for more function-based methodologies in ELF research. These methodological approaches are especially well suited for investigating the innovative form-function mappings characteristic of ELF communication.
Forschungs-