Customer Journey Mapping with Multimodal Large Language Models

The expansion of e-commerce requires accurate alignment of diverse multimodal content-such as user reviews, product images, and videos-with customer expectations. This study introduces a framework for mapping customer feedback to stages of the customer journey: Awareness, Consideration, Decision, and Post-Purchase. By analyzing user-generated content using Gemini 1.5 Flash, key features like sentiment, emotional tone, and product attributes were extracted to build a unified dataset. A hybrid classification approach was adopted, leveraging machine learning and rule-based logic. Among the models evaluated, Meta-LLaMA 3.2 3B and Meta-LLaMA 3.2-1B, large language models with robust contextual understanding, demonstrated the highest accuracy at 0.929, surpassing other models such as GPT-Neo, BERT, and RoBERTa. These results show how well the model captures complex customer opinion and aligns multimodal insights. The findings highlight discrepancies between consumer expectations and marketing narratives, particularly about the deliberation stage-a crucial phase in building trust. This research offers scalable and interpretable approaches to enhance customer journey mapping and improve content alignment in e-commerce.