Artificial Intelligence in the Work of a Psychologist: An Example of Analysis of Verbal Forecasts. Part I

The relevance of the research is determined by the need to develop a methodology for using artificial intelligence systems to expand human capabilities, in particular, in relation to solving problems of classifying qualitative data based on the use of Large Language Models (LLM). The purpose of the research described in this 1st article (a 2-part cycle) was to present a methodological approach that implements the use of large language models to classify individual verbal predictions about possible and impossible events in the future. Chinese participants (n = 149 gave 447 predictions) generated descriptions of events using the Wenciuxin (SoJump) system according to the specified three conditions: possible – improbable – impossible. The resulting corpus of open responses in Chinese was analyzed using five modern LLMs: GPT-4, Claude 3.5, Qwen 2.5-72B, Gemini Pro 1.5, and Llama 3.1-70B. After the training procedure, each model was assigned to analyze the responses based on several semantic parameters. An analysis of the responses of five large language models (Claude 3.5 Sonnet, GPT-4, Gemini Pro, LLaMA 3, and Qwen) revealed both significant similarities and noticeable differences in their approaches to analyzing verbal predictions (Chinese study participants). The integrative meta-model synthesized the results into a single heading, which allowed reliability testing within each model and comparative analysis between models. The revealed differences in the analytical approaches implemented by the models indicate that, although these models have common capabilities in identifying the main themes and patterns of semantic units represented in the participants’ verbal forecasts, they demonstrate different strengths in the aspects of text analysis they identify, which significantly expands the capabilities of a research psychologist. The obtained high reliability indicators (consistency between and within the models) indicate the potential of artificial intelligence technologies in applied scientific activities in the field of “mixed” methods.