Towards Personalized LLMs: Investigating Narrative Generation and Personality-Based Preferences in Large Language Models

Abstract

Large Language Models (LLMs) have transformed text generation, yet aligning their outputs with individual identities and preferences remains underexplored. This research addresses these challenges by (1) evaluating LLMs’ capabilities in generating personalized narratives and their impacts and (2) examining how diverse personality traits shape human-LLM interactions. In the first part, we explore the effectiveness of LLMs in generating personalized “mirror stories” that reflect and resonate with individual readers’ identities. We introduce MirrorStories, a corpus of 1,500 personalized short stories incorporating elements such as name, gender, age, ethnicity, interests, and moral themes. Our evaluation with 26 diverse human judges shows that LLMs effectively integrate these identity elements, with personalized stories outperforming generic narratives in engagement, satisfaction and personal relevance. We also analyze biases in generated content and the integration of images into personalized storytelling. The second part investigates whether personality traits influence preferences for different LLMs. In a study with 32 participants evenly split across four Keirsey personality types, users engaged with GPT-4 and Claude 3.5 across four collaborative tasks: data analysis, creative writing, information retrieval, and writing assistance. Findings reveal personality-based preferences: Rationals favored GPT-4 for goal-oriented tasks, while Idealists preferred Claude 3.5 for creative and open-ended ones. Other types showed more task-dependent preferences. Sentiment analysis of participant feedback supported these trends. Interestingly, overall helpfulness ratings were similar across models, showing how personality-based analysis reveals LLM differences that traditional evaluations miss.