Patterns and Characteristics Unique to AI-Generated and Human-Written Text Responses

Abstract

The recent advancements in LLMs and Gen-AI technology, such as ChatGPT, Gemini, and Llamas, have widespread applications. Such AI-based solutions strive to achieve extremely high levels of effectiveness in identifying and modeling a multitude of complex patterns and characteristics in textual data. In related literature, recently there is an increased focus on solutions to detect and model the complex characteristics and semantic formulations in textual data that are respectively unique to AI-generated and human-developed responses to a given input query or data. This research work is a preliminary study on investigating NLP-based approaches that are applicable to our research question: for a given query or input data, can we differentiate between the AI-generated responses from that is developed by a human expert. The case study dataset has about 2,000 records (about published research articles) and four attributes: title of a published article; its AI-generated abstract; its human-developed abstract; and a class label. The two NLP-based approaches we are currently investigating are: similarity metric(s) based assessment paired with the known class labels of the respective abstracts, and an LLM-based approach for modeling. We will present the respective results and provide their individual and comparative analysis as well as important conclusion points.