This study employed a scoping review design to map the available evidence on the use of ChatGPT in supporting students’ academic writing skills. A scoping review was considered appropriate because the literature on ChatGPT-assisted writing is still emerging, methodologically diverse, and characterized by variations in study populations, educational contexts, intervention formats, and measured outcomes. Unlike a systematic review, which primarily assesses intervention effectiveness, this scoping review aimed to identify the extent, range, and nature of existing studies, clarify how ChatGPT has been applied in writing-related learning activities, and identify gaps requiring further empirical investigation.

The Population guided the review–Concept–Context (PCC) framework, a commonly used framework for scoping reviews (Table 1). The Population consisted of students engaged in ChatGPT-assisted writing activities; the concept focused on ChatGPT as an AI-supported writing tool; and the context covered educational settings where academic writing, composition, or writing-related learning outcomes were examined. This framework was used to ensure that the review question remained consistent with the broader mapping purpose of the study. The guiding review question was: What is known from existing studies about the use of ChatGPT in supporting students’ academic writing skills across educational contexts?.

A comprehensive literature search was conducted across seven academic databases to identify relevant studies examining the impact of ChatGPT on students' writing skills. The selected databases included ScienceDirect (867), ProQuest (3,092), Sage Journals (393), Taylor & Francis (522), PubMed (77), and Google Scholar (14,300), covering the publication period from 2020 to 2025 (N = 19,251).

To ensure a systematic and exhaustive search, Boolean operators (AND, OR) were utilized to refine and combine search terms. Quotation marks ("") were applied for exact phrase searching, while parentheses ( ) were used to group related concepts. The search terms were carefully selected to capture all relevant literature. They included the following keywords: ("ChatGPT" OR "Generative Pre-trained Transformer") AND (students OR learners) AND ("writing skills" OR "academic writing" OR "composition skills") AND (use OR application OR integration).

As aligned with the PCC framework of this study, the population of interest included students engaged in writing-related academic tasks across various educational levels, including secondary, undergraduate, and graduate education. This broader inclusion was adopted to reflect the diversity of the existing literature and to address variations in study populations identified during the preliminary search.

The review focused on ChatGPT as an AI-assisted writing tool. Studies were considered eligible if they examined the use of ChatGPT to support writing-related processes, including grammar correction, vocabulary development, coherence, organization, feedback provision, and overall writing performance. The context encompassed educational settings where writing instruction or writing-related outcomes were evaluated, including academic writing courses, English as a Foreign Language (EFL) environments, and other writing-intensive learning contexts.

In terms of study characteristics, the review included empirical research employing quantitative, qualitative, or mixed-methods designs to allow for a comprehensive mapping of the available evidence. Only peer-reviewed journal articles published in English between 2020 and 2025 were considered eligible to ensure relevance to the emergence of generative AI technologies and to maintain a consistent time frame across the review.

Studies were excluded if they did not involve student populations or were conducted outside educational contexts, such as purely technical or computational studies of artificial intelligence. Articles that did not examine the use of ChatGPT or similar tools in relation to writing-related outcomes were also excluded. In addition, non-empirical publications, including literature reviews, conference proceedings, editorials, commentaries, protocols, theses, and dissertations, were not considered. Studies without accessible full-text versions or those lacking sufficient methodological detail for data extraction were likewise excluded.

The screening process was conducted using a systematic and multi-stage approach to ensure methodological rigor and consistency in study selection. All records identified through the database search were initially compiled and organized, and duplicate entries were removed before screening. The remaining articles underwent a two-stage screening process consisting of title and abstract review followed by full-text evaluation.

In the first stage, four independent reviewers (HKP, SPM, AMM, and SAMM) assessed the titles and abstracts of all retrieved studies to determine their relevance using predefined inclusion and exclusion criteria. This initial screening focused on identifying studies that examined the use of ChatGPT in relation to writing-related outcomes within educational contexts. Articles that did not meet the criteria were excluded at this stage. In the second stage, full-text versions of the remaining articles were retrieved and evaluated in detail to confirm their eligibility for inclusion in the review.

To enhance the reliability of the screening process, the reviewers selected independently and then compared their decisions. Inter-reviewer agreement was assessed, and any discrepancies in study selection were resolved through discussion and consensus. When disagreements persisted, an independent panel of reviewers (MV, SJDG, MDA, NAL, PPA, and SHMT) was consulted to provide further evaluation and ensure consistent application of the eligibility criteria. This collaborative process minimized potential selection bias and strengthened the transparency of the review. A detailed PRISMA flow diagram (Figure 1) illustrates the stages of identification, screening, eligibility assessment, and final inclusion of studies. In addition to the diagram, the numerical results of each screening stage are described in the Results section to provide a clear and transparent account of the study selection process.

A structured data extraction process was undertaken to systematically capture relevant information from all studies included in the review. Following full-text screening and eligibility confirmation, the final set of articles was subjected to detailed data extraction using a standardized extraction framework developed by the research team. This framework was designed to ensure consistency in capturing key study characteristics and to facilitate comparison across studies with varying designs and contexts.

The extracted information included bibliographic details such as author and year of publication, as well as study-specific characteristics including country of origin, research objectives, study design, participant characteristics, and educational setting. In addition, data were collected on the nature of the ChatGPT intervention or application, the duration of implementation (when available), and the specific writing-related outcomes assessed, such as grammar, vocabulary, coherence, organization, academic expression, motivation, and writing self-efficacy.

The extraction process was conducted independently by multiple reviewers to enhance accuracy and reduce bias. Extracted data were cross-checked for consistency, and any discrepancies were resolved through discussion and consensus among the reviewers. The finalized data were organized into a comprehensive summary table to support the synthesis and interpretation of findings (Table 5).

Given the methodological diversity of the included studies, a structured and design-specific approach to quality appraisal and risk of bias assessment was employed. As reflected in Table 5, the review incorporated studies with varied designs, including qualitative, quasi-experimental, randomized experimental, longitudinal intervention, and mixed-methods approaches. This heterogeneity necessitated the use of multiple appraisal tools tailored to the methodological characteristics of each study.

For quasi-experimental and longitudinal intervention studies, the Joanna Briggs Institute (JBI) Critical Appraisal Checklist for Quasi-Experimental Studies was used to evaluate methodological rigor (Table 2), including participant comparability, intervention integrity, and outcome measurement. Randomized and experimental studies were assessed across key domains of bias, including allocation procedures, blinding, completeness of outcome data, and selective reporting. These assessments were subsequently visualized using the Risk-of-Bias Visualization (ROBVIS) tool to provide a structured summary of potential bias across studies (Table 3). For mixed-methods studies, the Joanna Briggs Institute (JBI) Critical Appraisal Checklist for Mixed Methods was applied to examine the integration of qualitative and quantitative components and the overall coherence of the research design (Table 4). Qualitative studies were appraised using relevant criteria within the JBI framework to assess credibility, dependability, and methodological transparency.

The appraisal process was conducted independently by designated reviewers, and the results were cross-checked to ensure consistency. Any discrepancies in ratings were resolved through discussion and consensus, with input from additional reviewers when necessary. The overall risk of bias for each study was categorized as low, moderate, or high based on the extent to which methodological criteria were satisfied.

Consistent with the purpose of a scoping review, the quality appraisal was undertaken to provide context for interpreting the findings rather than to exclude studies based on methodological limitations. The appraisal results were used to highlight variations in study quality, identify potential sources of bias, and inform interpretation of the evidence base. The assessment revealed variability in methodological rigor across studies, particularly regarding sample size, clarity of intervention procedures, and reporting of outcome measures, reflecting the evolving and heterogeneous nature of research on ChatGPT-assisted writing.

Note: (+) indicates a low risk of bias, (-) indicates a high risk of bias, (?) shows unclear risk of bias

Note: (Y) criterion is fully met, indicating high methodological quality; (U) criterion is partially met or not explicitly stated, leading to some uncertainty; (N) criterion is not met, indicating a potential risk of bias or methodological weakness.

The data analysis was conducted using a structured narrative synthesis approach guided by the principles of Synthesis Without Meta-Analysis (SWiM) to ensure transparency and systematic organization of findings across heterogeneous study designs. Given the variability in study methodologies, populations, interventions, and outcome measures, a meta-analysis was not feasible. Instead, the SWiM framework was applied to facilitate a consistent and rigorous synthesis of both quantitative and qualitative evidence.

The analysis began by organizing the extracted data into key domains, including study characteristics (author, year, country), research design, participant profile, educational setting, nature of the ChatGPT intervention, duration of implementation, and reported writing-related outcomes. These outcomes included both direct writing performance indicators, such as grammar, vocabulary, coherence, organization, and academic expression, as well as related constructs such as motivation, engagement, and writing self-efficacy.

Studies were then grouped according to similarities in design and outcome focus to allow for meaningful comparison across the evidence base. Rather than aggregating effect sizes, the synthesis focused on identifying patterns, consistencies, and variations in how ChatGPT was applied and in the types of outcomes reported. Particular attention was given to distinguishing among different categories of outcomes, including improvements in writing quality, changes in learner-related factors (e.g., motivation and self-efficacy), and process-related effects (e.g., feedback and cognitive load).

The synthesis also examined contextual factors influencing the reported findings, including differences in educational level, instructional approach, intervention duration, and level of instructor involvement. This approach allowed for a more nuanced interpretation of the evidence, recognizing that the impact of ChatGPT may vary depending on how it is implemented and the learning environment in which it is used.

In addition to identifying patterns, the analysis considered inconsistencies and limitations within the included studies. Variations in study design, sample size, outcome measurement, and reporting practices were critically examined to contextualize the findings and to avoid overgeneralization. This interpretive approach is consistent with the purpose of a scoping review, which is to map the existing body of evidence and highlight areas requiring further investigation rather than to establish definitive conclusions regarding effectiveness.

The synthesized findings were subsequently organized into a summary table to support comparison across studies and to enhance clarity in the presentation of results. This structured analytical approach enabled a comprehensive overview of the current evidence on ChatGPT-assisted writing while maintaining methodological transparency and analytical rigor.

A total of nine studies were included in this scoping review following a multi-stage screening process. Initially, 19,251 records were identified across the selected databases. After applying limiters and removing duplicates, 653 articles remained for title and abstract screening. Of these, 38 studies were assessed for full-text eligibility, and 9 met the inclusion criteria after applying the predefined selection criteria and incorporating relevant articles identified through reference list screening. The study selection process is illustrated in Figure 1.

The included studies represent a geographically diverse evidence base, with research conducted across multiple countries, including Saudi Arabia, China, Ecuador, Indonesia, Iran, Pakistan, and India. Most studies were conducted in higher education contexts, although several included participants from secondary or vocational education settings, reflecting the variability of the populations examined. This diversity aligns with the broader inclusion criteria of the scoping review and highlights the range of educational contexts in which ChatGPT-assisted writing has been explored.

Methodologically, the studies demonstrated notable heterogeneity. The included research comprised quasi-experimental designs, randomized and non-randomized experimental studies, mixed-methods approaches, qualitative investigations, and longitudinal interventions. Sample sizes varied considerably, from small qualitative samples to large cohort-based studies, reflecting differences in study scope and methodological depth. The nature of ChatGPT integration also varied across studies. Several studies implemented structured instructional interventions that embedded ChatGPT within writing activities, often supplemented with teacher guidance, feedback mechanisms, or digital learning platforms. Other studies examined more specific applications, such as ChatGPT-assisted feedback, mobile-based writing support, or its role in particular writing tasks, including argumentative essays, report writing, and digital storytelling.

The duration of interventions varied across studies, with some reporting short-term interventions lasting several weeks, while others involved longer instructional periods of up to 16 weeks. However, not all studies clearly specified the duration of exposure, reflecting inconsistencies in reporting practices within the current literature.

The studies included in this review involved participants from a range of educational levels and learning environments, reflecting the heterogeneous nature of the current evidence base. While several studies focused on undergraduate students in higher education institutions, others included participants from secondary and vocational education, as well as graduate-level learners. Sample sizes varied considerably across studies, ranging from small qualitative samples to larger cohort-based investigations. For instance, some studies involved fewer than 50 participants, while others included over 300 students, suggesting differences in study scale and statistical power. In addition, the participants' disciplinary contexts were diverse, including English as a Foreign Language (EFL) courses, medical education, business-related programs, and general academic writing classes. The studies were conducted in predominantly classroom-based settings, including university lecture courses, language classes, and structured writing programs. However, there were also variations in instructional delivery, with some studies incorporating hybrid or technology-mediated learning environments, such as mobile-based feedback systems or AI-integrated learning platforms.

Importantly, the diversity in participant characteristics and study settings introduces several interpretive limitations. Differences in educational level, prior writing proficiency, and digital literacy may influence how students interact with AI tools and the benefits they derive from their use. For example, learners with greater digital familiarity may be better positioned to use ChatGPT effectively. At the same time, those with limited exposure to AI technologies may face challenges interpreting and applying generated feedback. Similarly, the role of instructors varied across studies, with some interventions involving structured guidance and others relying on more autonomous student use.

The included studies demonstrated a range of ChatGPT applications in writing-related educational contexts, reflecting variability in how the tool is integrated into learning environments. Rather than functioning uniformly as a structured instructional intervention, ChatGPT was used in diverse ways, including as a feedback provider, writing assistant, content generator, and interactive learning support tool.

Several studies incorporated ChatGPT into structured classroom activities, using it to support writing instruction through guided exercises, automated feedback, and iterative revision. In these contexts, ChatGPT was often embedded within broader pedagogical frameworks, such as constructivist learning models or AI-assisted instructional designs, allowing students to engage with writing tasks in a more interactive and scaffolded manner. Other studies employed ChatGPT in a more targeted manner, focusing on specific functions such as grammar correction, vocabulary enhancement, or organization of written outputs.

In addition to structured use, some studies explored less formal or exploratory applications of ChatGPT, including its role in supporting independent writing, facilitating idea generation, or assisting with research-related writing tasks. These variations indicate that ChatGPT is not limited to a single mode of application but can be adapted to different instructional needs and learning environments.

The duration and intensity of ChatGPT use also varied across studies. While some research implemented short-term activities lasting several weeks, others examined longer-term integration within writing courses. However, inconsistencies in reporting intervention duration and usage frequency were observed, making it difficult to establish clear patterns regarding optimal exposure or implementation strategies.

Importantly, the role of instructors varied across studies, shaping how students used and interpreted ChatGPT. In some cases, instructors actively guided the use of ChatGPT, providing structure and feedback on AI-generated outputs. In other cases, students engaged with the tool more independently, which may have affected how effectively they applied the feedback or suggestions provided. This variation highlights the importance of instructional context in shaping the use and potential benefits of AI-assisted writing tools.

The included studies reported a range of outcomes associated with ChatGPT use in writing-related contexts. These outcomes can be broadly categorized into three domains: writing performance outcomes, learner-related outcomes, and process-related outcomes. Distinguishing these categories is important, as not all reported effects relate directly to improvements in writing quality, and the findings across studies are not uniformly consistent.

In terms of writing performance outcomes, several studies reported improvements in specific aspects of writing, including grammatical accuracy, vocabulary use, coherence, organization, and overall structure. These improvements were particularly evident in studies where ChatGPT was integrated into structured instructional activities or used as a feedback tool to support revision. However, the extent of improvement varied across studies, and not all writing domains were consistently enhanced. For example, while gains in organization and mechanics were frequently reported, some studies noted limited or uneven effects on higher-order writing skills such as language use or critical argumentation. Moreover, at least one study found no significant improvement in writing performance during digital storytelling tasks, despite observing benefits in other areas. ChatGPT's impact on writing quality may depend on the type of writing task and how the tool is used.

Beyond direct writing outcomes, several studies examined learner-related outcomes, including motivation, engagement, and writing self-efficacy. The findings suggest that ChatGPT may increase student engagement and confidence in writing, particularly when used as an interactive and supportive learning tool. Students reported greater willingness to revise their work and experiment with writing when immediate feedback was available. However, these outcomes represent attitudinal or affective changes rather than direct measures of writing proficiency and should therefore be interpreted separately from performance-based improvements.

The studies also identified process-related outcomes, particularly regarding feedback mechanisms and cognitive load. ChatGPT was frequently described as providing immediate, accessible feedback that facilitated the writing process and supported iterative revision. In some cases, using ChatGPT was associated with reduced cognitive load, allowing students to focus more on content development rather than mechanical aspects of writing. At the same time, concerns were raised regarding the potential for over-reliance on AI-generated content and the risk that students may engage less deeply with the writing process if the tool is used without sufficient guidance.

Taken together, the findings indicate that ChatGPT has the potential to support multiple dimensions of writing-related learning, but the evidence is mixed and context-dependent. While many studies report positive trends, the variability in outcomes, study designs, and implementation approaches limits the ability to draw uniform conclusions. The distinction between writing performance, learner-related factors, and process-level effects is therefore essential for accurately interpreting the current evidence base and avoiding overgeneralization.

XXX

During the preparation of this manuscript, the authors used Grammarly (Grammarly Inc.) to improve the readability, grammar, and clarity of the writing. All suggestions provided by the tool were carefully reviewed and selectively applied by the authors. The authors take full responsibility for the accuracy, integrity, and content of this manuscript. Please read the AI usage policy; a statement template tailored to the author’s needs is also available.-> https://journal.rezkimedia.or.id/index.php/jaieli/aipolicy