Guiding ChatGPT to Generate Salient Area Summaries
Authors: Jun Gao, Ziqiang Cao, Shaoyao Huang, Luozheng Qin, Chunhui Ai
Summary: ChatGPT is instruct-tuned to generate common and human-expected content material to align with human desire by Reinforcement Studying from Human Suggestions (RLHF), in the meantime leading to generated responses not salient sufficient. Due to this fact, on this case, ChatGPT could fail to fulfill area necessities in zero-shot settings, resulting in poor ROUGE scores. Impressed by the In-Context Studying (ICL) and retelling potential of ChatGPT, this paper proposes PADS, a textbf{P}ipeline for textbf{A}ssisting ChatGPT in textbf{D}omain textbf{S}ummarization. PADS consists of a retriever to retrieve comparable examples from corpora and a rank mannequin to rerank the a number of candidate summaries generated by ChatGPT. Particularly, given an inference doc, we first retrieve an in-context demonstration through the retriever. Then, we require ChatGPT to generate okay candidate summaries for the inference doc at a time underneath the steering of the retrieved demonstration. Lastly, the rank mannequin independently scores the okay candidate summaries in accordance with their high quality and selects the optimum one. We extensively discover dense and sparse retrieval strategies to pick out efficient demonstrations for reference and effectively practice the rank mannequin to replicate the standard of candidate summaries for every given summarized doc. Moreover, PADS incorporates merely 400M trainable parameters originating from the rank mannequin and we merely gather 2.5k information to coach it. We consider PADS on 5 datasets from completely different domains, and the consequence signifies that every module in PADS is dedicated to successfully guiding ChatGPT to generate salient summaries becoming completely different area necessities. Particularly, within the well-liked summarization dataset Gigaword, PADS achieves over +8 achieve on ROUGE-L, in contrast with the naive ChatGPT within the zero-shot setting. footnote{Our code can be found at url{https://github.com/jungao1106/PADS}}
