Virtual Quant Marketing Seminar - Tong Wang
Leveraging LLM knowledge to Improve Task Robustness: Applications in Customer Service and Content Marketing
Dear All,
Our next Virtual Quant Marketing Seminar for the semester is on Monday, February 3 at Noon ET. The speaker will be Tong Wang (Yale).
The zoom link for all the talks is below, note that this is a new link: https://upenn.zoom.us/j/93394346729?pwd=SoVXVesbltpYbQxo9tbbPF6YgopDl3.1
Title: Leveraging LLM knowledge to Improve Task Robustness: Applications in Customer Service and Content Marketing
Abstract: Large Language Models (LLMs) trained on vast datasets, possess rich knowledge about language, reasoning, and real-world phenomena. This talk explores how this rich knowledge can be elicited and leveraged for downstream marketing tasks. We demonstrate this approach in two applications.
The first application is train a customer service chatbot through knowledge distillation. Here a teacher LLM reasons about differences between its high-quality responses and a smaller student LLM’s lower-quality responses, distilling strategies to improve the student’s performance in goal-oriented dialogues. We then show how to use the strategies to improve the performance of the student LLMs in real time customer service conversations.
Second, in content generation, LLMs generate hypotheses to explain why some headlines achieve higher click-through rates than others in A/B tests. These hypotheses are refined into validated theories that guide fine-tuning, helping models focus on generalizable mechanisms and avoid spurious correlations. The validated hypotheses help LLMs generate the “best” headlines for an article in terms of clickthrough–while ensuring relevance.
While the specifics of knowledge elicitation and distillation for the two applications differ in many important ways, an important common theme is that we leverage LLMs as abductive reasoners—to explain why one example is better than another on a desirable metric—and use these guidelines/hypotheses to perform the tasks with greater robustness and efficiency.
The schedule for the rest of the semester is:
Monday, February 24, Noon ET - Ryan Dew (Wharton)
Monday, March 10, Noon ET - Malika Korganbekova (Chicago Booth)
Monday, March 24, Noon ET - Soheil Ghili (Yale)
Monday, April 7, Noon ET - Jacob Conway (Chicago Booth)
Monday, April 28, Noon ET - Srinivas Tunuguntla (Duke)
Monday, May 5, Noon ET - Ananya Sen (CMU)
Monday, May 12, Noon ET - Minkyung Kim (CMU)
We encourage you to use our google calendar to keep track of the talks.