Negotiating Bots with Empathy

Abstract

Automated negotiation is a multi-agent task where one or multiple negotiating bots aim to resolve a conflict or reach a mutually beneficial agreement. Previous approaches have focused on achieving financially optimal outcomes with no consideration for user sentiments. However, as negotiations typically occur within the context of ongoing relationships, maintaining a pleasant overall experience is undeniably crucial. In this thesis, we tackle the problem of an item sale negotiation where a buyer agent seeks to obtain an item from a seller agent. The goal is to develop a seller negotiating bot with the objective of simultaneously maximizing both buyer satisfaction and sale price. We compare two approaches to the problem. The first approach consists of using a single end-to-end Long Short-Term Memory sequence-to-sequence (LSTM seq2seq) model with attention mechanism that takes in previous utterances as input and generates the next utterance. The second approach consists of breaking down the model into 3 parts: a rule-based parser which extracts the negotiation act and sentiment of the received utterance, a seq2seq manager which recommends the next act and sentiment, and a fine-tuned Generative Pre-trained Transformer (GPT-2) generator which transforms the recommended act and sentiment into a complete response. We make use of a mixed learning approach which combines supervised learning with goal-oriented reinforcement learning to efficiently train both the end-to-end model and the manager's decision model. Compared to previous work, the experiment results showed improvement in item representation, consistency of offers, buyer sentiment, empathy, fluency, appropriateness, and human likeness.