Using code from ChatGPT: Finding patterns in the developers’ interaction with ChatGPT

DOI

ChatGPT can advise developers and provide code on how to fix bugs, add new features, refactor, reuse, and secure their code but currently, there is little knowledge about whether the developers trust ChatGPT’s responses and actually use the provided code. In this context, this study aims to identify patterns that describe the interaction of developers with ChatGPT with respect to the characteristics of the prompts and the actual use of the provided code by the developer. We performed a case study on 267,098 lines of code provided by ChatGPT related to commits, pull requests, files of code, and discussions between ChatGPT and developers. Our findings show that developers are more likely to integrate the given code snapshot in their code base when they have provided information to ChatGPT through several rounds of brief prompts that include problem-related specific words instead of using large textual or code prompts. Results also highlight the ability of ChatGPT to handle efficiently different types of problems across different programming languages.

Identifier
DOI https://doi.org/10.23728/b2share.db2ef5890fa44c7a85af366a50de73b9
Source https://b2share.eudat.eu/records/db2ef5890fa44c7a85af366a50de73b9
Metadata Access https://b2share.eudat.eu/api/oai2d?verb=GetRecord&metadataPrefix=eudatcore&identifier=oai:b2share.eudat.eu:b2rec/db2ef5890fa44c7a85af366a50de73b9
Provenance
Creator Terzi Anastasia; Bibi Stamatia; Tsitsimiklis Nikolaos; Angelidis Pantelis
Publisher EUDAT B2SHARE
Publication Year 2024
Rights info:eu-repo/semantics/openAccess
OpenAccess true
Contact a.terzi(at)uowm.gr
Representation
Format png; py; arff; csv; pdf; sav
Size 3.4 MB; 11 files
Discipline 4.1.13 → Computer sciences → Software engineering