research area Speech and Natural Language Processing, research area Tools, Platforms, Frameworksconference NAACL
content type paperpublished August 2024

APE: Active Prompt Engineering - Identifying Informative Few-Shot Examples for LLMs

AuthorsKun Qian, Farima Fatahi Bayat, Anton Belyi, Yash Govind, Rahul Khot, Katherine Luna, Azadeh Nikfarjam, Xiaoguang Qi, Yisi Sang, Fei Wu, Victor (AIML) Zhang, Yunyao Li, Samira Khorshidi, Xianqi Chu

View publication

Prompt engineering is an iterative procedure that often requires extensive manual efforts to formulate suitable instructions for effectively directing large language models (LLMs) in specific tasks. Incorporating few-shot examples is a vital and efficacious approach to provide LLMs with precise and tangible instructions, leading to improved LLM performance. Nonetheless, identifying the most informative demonstrations for LLMs is labor-intensive, frequently entailing sifting through an extensive search space. In this demonstration, we showcase an interactive tool called APE (Active Prompt Tuning) designed for refining prompts through human feedback. Drawing inspiration from active learning, APE iteratively selects the most ambiguous examples for human feedback, which will be transformed into few-shot examples within the prompt.

Related readings and updates.

SynthDST: Synthetic Data is All You Need for Few-Shot Dialog State Tracking

In-context learning with Large Language Models (LLMs) has emerged as a promising avenue of research in Dialog State Tracking (DST). However, the best-performing in-context learning methods involve retrieving and adding similar examples to the prompt, requiring access to labeled training data. Procuring such training data for a wide range of domains and applications is time-consuming, expensive, and, at times, infeasible. While zero-shot learning…
See paper details

Learning to Detect Novel and Fine-Grained Acoustic Sequences Using Pretrained Audio Representations



This work investigates pre-trained audio representations for few shot Sound Event Detection. We specifically address the task of few shot detection of novel acoustic sequences, or sound events, with semantically meaningful temporal structure without assuming access to non-target audio. We develop procedures for pre-training suitable representations and methods that transfer them to our few shot learning scenario. Our experiments evaluate the…
See paper details