����������������������������������������������������������������������������������

 

 

Uso de ChatGPT para la retroalimentaci�n de la producci�n escrita de los estudiantes universitarios

 

Using ChatGPT for feedback on university students� written production

 

Utiliza��o do ChatGPT para feedback sobre a produ��o escrita de estudantes universit�rios

 

 

 

 

 

 

 

 

 

 

Correspondencia: malvarezp@utb.edu.ec

 

 

Ciencias de la Educaci�n

Art�culo de Investigaci�n

 

* Recibido: 26 de abril de 2025 *Aceptado: 11 de mayo de 2025 * Publicado: �26 de junio de 2025

 

       I.          Universidad T�cnica de Babahoyo, Babahoyo, Ecuador, Docente titular del Centro de Idiomas, Ecuador.

     II.          Universidad T�cnica de Babahoyo, Babahoyo, Ecuador, Docente del Centro de Idiomas, Ecuador.

   III.          Universidad T�cnica de Babahoyo, Babahoyo, Ecuador, Docente del Centro de Idiomas, Ecuador.

   IV.          Universidad T�cnica de Babahoyo, Babahoyo, Ecuador, Docente del Centro de Idiomas, Ecuador.

 

Resumen

Este estudio explora el uso de ChatGPT como herramienta para mejorar las habilidades de escritura de estudiantes de ingl�s como lengua extranjera (EFL) mediante retroalimentaci�n inmediata y detallada. La investigaci�n identifica errores comunes en la redacci�n de ensayos, incluyendo errores gramaticales, estructurales y organizativos, y eval�a el impacto de la retroalimentaci�n de ChatGPT en la reducci�n de estos errores. Se emple� un enfoque de m�todos mixtos con 50 estudiantes universitarios de nivel intermedio de ingl�s. Los resultados mostraron una reducci�n significativa de errores tras el uso de ChatGPT, respaldada por una prueba t de muestras pareadas. Los participantes expresaron una percepci�n positiva de la facilidad de uso, la utilidad y la claridad de ChatGPT en la identificaci�n de errores y la retroalimentaci�n. Los hallazgos destacan el potencial de ChatGPT como recurso complementario para mejorar la comunicaci�n escrita y desarrollar la confianza y la motivaci�n de los estudiantes.

Palabras Clave: Comentarios de ChatGPT; estudiantes de ingl�s como lengua extranjera; habilidades de escritura.

 

Abstract

This study explores the use of ChatGPT as a tool for enhancing EFL learners' writing skills by providing immediate and detailed feedback. The research identifies common errors in essay writing, including grammatical, structural, and organizational issues, and evaluates the impact of ChatGPT feedback on reducing these errors. A mixed-methods approach was employed with 50 university students at an intermediate English level. Results showed a significant reduction in errors after using ChatGPT, supported by a paired-sample t-test. Participants reported positive perceptions of ChatGPT's ease of use, usefulness, and clarity in error identification and feedback. The findings highlight ChatGPT's potential as a complementary resource for improving written communication and developing learners' confidence and motivation.

Keywords: ChatGPT feedback; EFL Learners; writing skills.

 

Resumo

Este estudo explora a utiliza��o do ChatGPT como ferramenta para melhorar as compet�ncias de escrita dos alunos de ingl�s como l�ngua estrangeira, fornecendo feedback imediato e detalhado. O inqu�rito identifica erros comuns na escrita de ensaios, incluindo quest�es gramaticais, estruturais e organizacionais, e avalia o impacto do feedback do ChatGPT na redu��o desses erros. Uma abordagem de m�todos mistos foi empregue com 50 estudantes universit�rios de n�vel interm�dio de ingl�s. Os resultados mostraram uma redu��o significativa dos erros ap�s a utiliza��o do ChatGPT, corroborada por um teste t para amostras emparelhadas. Os participantes reportaram perce��es positivas sobre a facilidade de utiliza��o, a utilidade e a clareza do ChatGPT na identifica��o de erros e no feedback. Os resultados destacam o potencial do ChatGPT como um recurso complementar para melhorar a comunica��o escrita e desenvolver a confian�a e a motiva��o dos alunos.

Palavras-chave: Feedback do ChatGPT; alunos de EFL; compet�ncias de escrita.

 

INTRODUCTION

Technology has revolutionized human lives, and education is not exception ( Akram et al., 2022). According to Chen et al. (2024) and Ramzan et al. (2023), the language acquisition process is also greatly affected by the change in technology since it provides innovative tools and resources that promote language learning and improve the development of writing skills. Referring to language learning, a growing interest in Language Learning Machines (LLMs) has emerged; Perkins (2023) claimed that it is happening mainly because LLMs are powerful tools for finding, analyzing, and demonstrating errors in EFL writing which help learners to identify quickly the aspects of their writing that need special attention. Moreover, Al-Garaady., et al. (2023) stated that LLMS strengthens writing skills because learners can write more polished and error-free text.

In this field, Chat GPT is an artificial intelligence that has been obtaining great attention due to its potential for different applications, especially in education in the area of EFL learning (Hussain et al., 2024). Regarding Artificial intelligence and its valuable support to education, ChatGPT is one of the most important worldwide. Hussein et al., 2024) remarked that ChatGPT is a �language model capable of providing detailed language feedback� (p.700). Hence, nowadays these tools are being integrated into academic settings because they offer opportunities for students and educators to enhance written communication, mainly learners who are from the EFL context. On the other hand, Jimenez (2023) argued that many teachers doubt the effectiveness of using AI in education due to wrong or inappropriate usage; the main concern is issues with plagiarism and cheating.

On the contrary, Algaraady et al. (2023) claimed that ChatGPT is beneficial because it can identify grammatical errors, mistakes in spelling, and other mistakes common to writing production. Also, Chawla et al. (2022) stated that this AI offers suggestions for making corrections and improvements in sentence structures, word choices, and clarity.� Moreover, Raj et al. (2022) agreed that Chat GPT helps provide specific feedback and identify areas for improvement, which they considered can positively impact the assessment process, making it more productive and efficient.

About error analysis, Gayed (2022) stated that it is an essential part of the learning process since it allows teachers and students to analyze errors when using a foreign language. It also provides light to teachers when identifying the source of the mistake, and this information makes it possible to design strategies to let learners address the problematic situations and avoid them. When mentioning significant strategies to examine and identify learner errors, Zhai et al. (2022) require identifying the type of error and how to use this information to address the situation. About the common errors identified, Algaraady et al. (2023) mentioned the reasons for making mistakes: mother tongue interference, lack of meta-linguistic, overgeneralization, simplification, and copying the same first interference. Park (2019) mentioned that chat GPT is useful for recognizing grammatical errors, but since it is a recent technology and it is not capable of identifying and understanding linguistic errors, in other words, it cannot identify in text some deviations from the standard rules and conventions of a language.

Taking into consideration the above information, this study intends to identify the type of errors learners produce in a text and the impact of using ChatGPT in raising learners' awareness of their common errors. Hence, this research has the following research questions

1.     What errors can ChatGPT identify from learners� written production?

2.     Is feedback from Chat GPT effective in helping learners identify errors?

3.     What are the students� perceptions about using chat GPT to identify errors in learners� written production?

 

METHODOLOGY

����������� This study used a mixed-methods design, combining quantitative and qualitative approaches to address the research questions. The participants are students from a public university with an intermediate proficiency level in English based on a standardized test like CEFR B1. The sample was chosen by convenience and based on the participants� level of proficiency. The sample size is 50 participants who voluntarily accepted to participate and signed an informed consent. The participants attended a week-long intensive session focused on writing skills with 2 hours sessions daily.

The study took place following three phases, which are described in the table 1.

 

Table 1.

Phases of the study about feedback from ChatGPT

Phases	Activities
Phase 1. Pre-intervention	a.      Initial writing task Participants write a 200-250 word essay on a specific topic. The essays are collected as pretest information to analyze initial errors and writing proficiency. b.     Error analysis Researchers performed an initial analysis to identify common errors in the essays. The categories to analyze are grammar, vocabulary, punctuation, organization, and coherence.
Phase 2.� Intervention	a.      ChatGPT feedback Essays are submitted to ChatGPT for feedback. Feedback from ChatGPT was collected and categorized considering accuracy in identifying grammar, vocabulary, mechanics, and organizational issues. � b.     Correction activity Students receive ChatGPT�s feedback and check their essays based on the suggestions. c.      Second writing task Participants write another essay on a different but comparable topic. The new essays are analyzed with ChatGPT to evaluate changes in error patterns and improvement.
Phase 3. Perception survey	Likert Scale Survey: A 10-item survey assesses learners� perceptions of ChatGPT�s feedback. Likert scale includes dimensions:                                                     i.     Ease of use.                                                   ii.     Usefulness in identifying errors.                                                 iii.     Clarity of explanations.                                                 iv.     Perceived impact on writing skills. Responses range from 1 (strongly disagree) to 5 (strongly agree)

Note. Made by the authors.

The data was analyzed through quantitative and qualitative methods. Regarding quantitative methods, the frequency and type of error were studied by comparing the initial writing with the final learner�s produced text; these results were analyzed with the paired-sample t-test to evaluate the use of chatGPT. Also, the Likert scale allowed to identify the mean for each item and the dimensions, helping to determine the participants' perceptions. In qualitative research, the researchers collected the different feedback provided by chatGP and classified and coded this information in common errors.

 

RESULTS

Identification of common errors throughout ChatGPT feedback

Before starting the intervention, learners elaborated an essay about a specific topic; once the written activity was completed, the text was analyzed by the ChatGPT and rechecked by the researchers. After that, the seven-day intervention was carried out using ChatGPT feedback to guide learners in identifying and correcting the errors. Table 2 shows the main errors identified in the essays created by learners and the mean of the occurrence during the intervention practices. Moreover, the errors were classified considering their affectation. Hence, they are divided into grammatical, sentence structure, vocabulary, mechanical, organizational, and stylistic errors.

Based on the information presented in Table 2, it is possible to identify that grammatical errors are mainly caused by incorrect usage of prepositions and confusion regarding applying the third-person rule for conjugation in the simple present. Two problematic situations could be found in relation to sentence structure. First, students tend to write with complete ideas when constructing phrases and sentences. Second, there are problems with conjunctions.

Lexical issues include mother tongue interference in the texts, learners getting confused when using words with similar roots and related meanings, problems identifying prefixes and suffixes, and adjectives and adverbs being included. �Regarding mechanical, situations when punctuation is used are presented, and the rule of punctuation is applied, mainly with commas, semicolons, and periods. Moreover, learners apply Spanish capitalization rules instead of English rules. Hence, some words, such as days of the week, tend to miss capitalization.

Concerning the organizational aspects, there are issues when constructing paragraphs and giving coherence and cohesion to the text, as most of the paragraphs are not guided by topic sentences. Also, the error analysis demonstrated that learners have problems constructing essays related to the structure and internal construction of ideas. Finally, related to stylistics, learners tend to repeat the same idea many times in the essay unnecessarily, a situation that leads to disorganization of the written construction and a loss of the message of the written production.

 

Table. 2

Classification and frequency of the errors in essays made by learners during the pretest, post-test, and interventions

Type of error	Details	Pre-test	Int. 1	Int. 2	Int. 3	Int. 4	Int. 5	Int. 6	Post-test
Grammatical error	Incorrect selection of tenses	3	2	2	2	1	1	1	1
	Absence of third-person rules for simple present	4	4	3	4	3	2	1	1
	Missing and overusing articles (a/the)	3	2	4	3	2	1	1	1
	Incorrect selection of prepositions�	10	9	10	9	8	6	5	3
Sentences structure	Lack of use of conjunction	15	13	10	8	7	7	6	5
	Incomplete ideas in sentences	20	3	5	6	5	5	5	5
Vocabulary error	Mother tongue interference	13	5	5	6	7	5	5	5
	Confusing words with similar root	5	2	2	2	2	3	2	2
	Inadequate use of prefixes and suffixes	4	3	3	4	2	2	3	3
	Confusing adjectives and adverbs	3	3	3	2	3	3	2	2
Mechanical	Lack of use of commas and periods.	16	10	10	11	9	9	8	7
	Missing capital letters	4	2	2	1	1	0	0	0
Organizational	Lack of topic sentences in the paragraph.	20	12	11	11	10	10	10	10
	Jumping between unrelated ideas.	17	10	11	9	10	9	9	9
	Absence of the essay structure	15	12	11	8	5	3	2	1
	Lack of cohesive devices to link ideas.	14	8	8	9	8	7	6	5
Stylistic	Using formal words in the formal essay.	4	3	2	2	2	2	1	1
	Repetition of the same idea unnecessarily.	17	13	7	7	8	7	7	6

Note. The errors were identified in the pretest and treated during the interventions; hence, the frequency is the mean of mistakes made by the 50 participants during this study.

The errors observed in the pre-test and post-test were analyzed using the Paired-sample test analysis to determine if there was a significant difference between them after using the chat GPT as a tool to gain comprehensible, instant, and effective feedback. Therefore, considering p < 0.05, and in this study p < 0.001, the null hypothesis that there is no difference between the PreTest and PostTest scores is rejected, implying a statistically significant difference between the PreTest and PostTest scores.

Hence, the analysis of the paired-sample test demonstrated that using ChatGPT to guide learners in easily identifying their mistakes in written text is successful. This situation is evidence that the intervention was effective, provided other factors were controlled (see Table 2).

 

Table 2.

Comparisons between frequency of errors in pretest and posttest written section

	Mean	Std. Deviation	95% Confidence Interval of the Difference		t	df	Sig. (2-tailed)
			Lower Bound	Upper Bound
Pair 1: PreTest - PostTest	6.66667	4.43250	4.46244	8.87090	6.381	17	0.000

Note. Paired-sample test analysis with a 95% level of confidence.

 

Student�s perception about using ChatGPT for feedback on written essays

The results of the Likert scale are presented in Graphic 1, hence, each statement has been assigned a scale, which is the mean of all the learners� responses. Regarding ease of use, the students have a strongly positive impression of ChatGPT as a tool to obtain immediate feedback. Therefore, they consider it easy to access and use during the writing feedback process. Also, they perceive this AI as user-friendly for receiving feedback, and they conclude that the process of submitting the writing on ChatGPT is straightforward.�

Concerning the usefulness of identifying errors, learners claimed that it is an effective tool for identifying grammatical errors in writing production. Moreover, this AI facilitates useful feedback on vocabulary selection and word choice errors. Finally, they agreed that the feedback obtained from ChatGPT contributed to understanding the most frequent errors in their written production. Regarding the clarity of the ChatGPT feedback, the participants responded that the explanations provided by ChatGPT for the errors were clear and easy to understand. Also, this AI offered specific examples or suggestions that improved the participants� understanding of writing rules. Finally, regarding the participants� Perceived Impact of ChatGPT feedback on Writing Skills, they agreed that it helped to improve the quality of the revised essays and that they tend to feel more confident in their ability to write essays.

 

Graphic 1.

Students� perception concerning to the ease of use of chatGPT for giving feedback, its usefulness, clarity of explanation, and the perceived impact of Chat GPT on writing skills

 

Note. Information collected from the Likert scale filled by participants. The data is the means of the responses for each statement. The scale 1=Strongly disagree, 2= Disagree, 3= Neutral, 4= Agree, 5= Strongly agree.

 

DISCUSSION

This study demonstrated that learners have problems at the organizational level when writing an academic essay, mainly when joining ideas to construct a clear and comprehensible paragraph. One of the identified problematic situations is the lack of topic sentences inside the section, which leads to jumping between unrelated ideas and run on sentences. Additionally, the lack of cohesive devices leads to construction ideas that are not connected or seem to be irrelevant. The results of this study are similar to Nenotek et al. (2022); their study identified the most dominant difficulties in writing essays for university students, this included the construction of the thesis statements, related ideas, development of ideas, causal relation in the sentences, and lack of topic sentence. On the same hand, Pablo, et al., (2018) stated that errors while writing academic essays respond to the organization, mentioned the lack of connectives, and the repetitive ideas.

Concerning the effectiveness of using ChatGPT for feedback, the study demonstrated a significant difference between the pre-test and post-test, allowing us to guarantee that using ChatGPT for feedback has a positive effect on students' ability to improve their written production in the form of essays. The results are coherent with Teng's (2024) findings; his research demonstrated that using ChatGPT for feedback in written text generated significant differences in writing experiences and self-efficacy. On the other hand, Steiss et al. (2024) claimed that ChatGPT feedback can be helpful to in formative early drafts of essays, while well-trained teachers provide much better and higher-quality feedback.

Moreover, the student�s perception of the use of ChatGPT for feedback is positive; they consider it as a valuable resource to provide immediate and comprehensive feedback, an aspect they stated refers to the increase in their motivation and self-confidence when writing an essay. In this point, Teng (2024) with his study could demonstrate that chatGPT not only promotes better writing but also raises learners� motivation for writing, and their engagement in the activity.

 

CONCLUSION

This study highlights the potential of ChatGPT as a tool for improving EFL learners' writing skills by providing immediate, comprehensive and very detailed feedback on some of aspects of essay composition. The findings demonstrate that learners frequently face challenges in organizational aspects, grammatical accuracy, and sentence structure. Hence, ChatGPT identifies these issues and offers appropriate feedback, giving access to students to meaningful revisions, and this way, they can improve their written work. The statistical analysis demonstrated a significant reduction in errors from pre-test to post-test, making visible the effectiveness of using ChatGPT in writing interventions.

Moreover, learners perceive ChatGPT as a user-friendly and efficient resource that enhances their understanding of writing conventions and improves their self-confidence. Participants really appreciated the detail and clarity in explaining errors and suggesting practical improvements, which contributed positively to their learning experience. These findings are aligned with other researchers� findings about how AI-driven tools can positively affect motivation, engagement, and self-efficacy in academic writing tasks.

Despite its advantages, ChatGPT's feedback must be seen as a complementary tool, remembering that teachers play an important role in providing guidance and developing critical thinking in academic writing.

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

References

Akram, H., Abdelrady, A. H., Al-Adwan, A. S., & Ramzan, M. (2022). Teachers� perceptions of technology integration in teaching-learning practices: A systematic review. Frontiers in psychology, 13, 920317.

Al-Garaady, J., & Mahyoob, M. (2023). ChatGPT's capabilities in spotting and analyzing writing errors experienced by EFL learners. Arab World English Journals, Special Issue on CALL, (9).

Al-Garaady, J., & Mahyoob, M. (2023). ChatGPT's capabilities in spotting and analyzing writing errors experienced by EFL learners. Arab World English Journals, Special Issue on CALL, (9).

Chawla, Raveen (December 26, 2022). What is ChatGPT? History, Features, Uses, Benefits, Drawbacks 2023. Archived from the original on January 7, 2023. Retrieved December 27, 2022.�

Chen, T. (2016). Technology-supported peer feedback in ESL/EFL writing classes: A research synthesis. Computer Assisted Language Learning, 29(2), 365-397.

Gayed, J. M., Carlon, M. K. J., Oriola, A. M., & Cross, J. S. (2022). Exploring an AI-based writing assistant's impact on English language learners. Computers and Education: Artificial Intelligence, 3, 100055.

Hussain, F., Latif, F., Mujtaba, M., Sohail, M., & Latif, F. (2024). Comparing the Effectiveness of Artificial Intelligence (AI) Models, ChatGPT and Meta AI to Detect and Correct Morphosyntactic Errors in BS Accounting Student�s Written Text. Contemporary Journal of Social Science Review, 2(04), 700-714.

Jimenez, K. (2023, February 2). 'This shouldn't be a surprise' the education community shares mixed reactions to ChatGPT. USA Today. Retrieved February 27, 2023, from https://www.usatoday.com/story/news/education/2023/01/30/chatgpt-going-banned-teachers-sound-alarm-new-ai-tech/11069593002/

Nenotek, S. A., Tlonaen, Z. A., & Manubulu, H. A. (2022). Exploring university students� difficulties in writing english academic essay. Al-Ishlah: Jurnal Pendidikan, 14(1), 909-920.

Pablo, J. C. I., & Lasaten, R. C. S. (2018). Writing difficulties and quality of academic essays of senior high school students. Asia Pacific Journal of Multidisciplinary Research, 6(4), 46-57.

Park, J. (2019).' Implications of AI-based grammar checker in EFL learning and testing: Korean high school students' writing. The Korea English Language Testing Association, 14(1), 11-39. Pedro, F., Subosa, M., Rivas, A., & Valverde, P. (2019). Artificial intelligence in education: Challenges and opportunities for sustainable development. Paris: UNESCO.

Perkins, M. (2023). Academic Integrity considerations of AI Large Language Models in the post- pandemic era: ChatGPT and beyond. Journal of University Teaching & Learning Practice, 20(2), 07.

Raj, R., & Kos, A. (2022). A Comprehensive Study of Optical Character Recognition. 2022 29th International Conference on Mixed Design of Integrated Circuits and System (MIXDES), 151-154.https://doi.org/10.23919/mixdes55591.2022.9837974.

Ramzan, M., Bibi, R., & Khunsa, N. (2023). Unraveling the Link between Social Media Usage and Academic Achievement among ESL Learners: A Quantitative Analysis. Global. Educational Studies Review, VIII, 407-421.

Steiss, J., Tate, T., Graham, S., Cruz, J., Hebert, M., Wang, J., ... & Olson, C. B. (2024). Comparing the quality of human and ChatGPT feedback of students� writing. Learning and Instruction, 91, 101894.

Teng, M. F. (2024). Metacognitive Awareness and EFL Learners' Perceptions and Experiences in Utilising ChatGPT for Writing Feedback. European Journal of Education, e12811.

Zhai, N., & Ma, X. (2023). The Effectiveness of Automated Writing Evaluation on Writing Quality: A Meta-Analysis. Journal of Educational Computing Research, 61(4), 875� 900. https://doi.org/10.1177/07356331221127300

 

 

 

 

 

� 2025 por los autores. Este art�culo es de acceso abierto y distribuido seg�n los t�rminos y condiciones de la licencia Creative Commons Atribuci�n-NoComercial-CompartirIgual 4.0 Internacional (CC BY-NC-SA 4.0)

(https://creativecommons.org/licenses/by-nc-sa/4.0/).