Promise Ekpo Osaine

Promise Ekpo Osaine

Ph.D Student @Cornell

Publications

Investigating Persuasiveness in Large Language Models

Published in , 2023

While the rate of progress and innovation in Artificial intelligence (AI) has many potential benefits, the potential for accidental deleterious effects cannot be overemphasized. It has been empirically demonstrated that large language models (LLMs) can learn to perform a wide range of natural language processing (NLP) tasks in a self-supervised setting. However, these models might unintentionally produce convincing arguments for false statements. There has been recent interest in improving LLM performance by fine-tuning in a reinforcement learning framework through interaction with human users. One could raise the concern that even seemingly benign reward functions can lead to strategic manipulation of user responses as an instrumental goal to achieve higher overall performance. This thesis seeks to investigate this possibility by evaluating the persuasiveness of self-supervised-only and reinforcement-learning-fine-tuned LLMs. In this work, we will discuss three approaches to investigating the degree of persuasiveness in LLMs by searching for qualitative failures through a direct query, quantifying the persuasiveness of generated outputs, and training on this persuasiveness metric as a reward signal with reinforcement learning. Through our investigation, we find that state-of-the-art LLMs fail when prompted with statements about less popular misconceptions or domain-specific myths. With this investigation of the safety-critical related failures of LLMs, we hope to further inform the public of the degree of reliability of these models and guide their use.

Image Captioning with Vision-Language Models (Under review)

Published in , 2023

When considering off-policy reinforce- ment learning methods for treatment policies in healthcare data, it is gener- ally the case that the patient population is diverse and has different chronic con- ditions that we would like to take into account when identifying optimal treat- ment policies. In this work, we use multi-group Gaussian process regression models in a fitted Q-iteration framework to allow us to model these different patient sub- groups and adapt the optimal policies to each subgroup. Concurrently, we es- timate these functions across the entire patient population. Finally, we apply our multi-group reinforcement learn- ing (MGRL) framework to the problem of optimal treatment policies for elec- trolytes with pre-existing medical con- ditions to assess performance against other state-of-the-art methods. We show that MGGP supersedes the per- formance of other models in addressing group structure in reinforcement learn- ing settings due to the robust covariance functions which has been adapted to learn the different behaviours for mul- tiple groups while maintaining a single policy. Keywords: Offline Reinforcement learning, Multi-Group Gaussian pro- cesses, Clinical, Electronic health records.Image captioning is an active area of research in the multi-modal artificial intelligence (AI) com- munity as it connects vision and language under- standing, especially in settings where it is required that a model understands the content shown in an image, and generates semantically and grammati- cally correct descriptions. In this project, we fol- lowed a standard approach of deep learning-based image captioning model; injecting architecture for the encoder-decoder setup ,where the encoder extracts image features and the decoder generates a sequence of words which represents the image content. As such, we investigated image encoders, which are ResNet101, InceptionResNetV2, Ef- ficientNetB7, EfficientNetV2M and CLIP. As a caption generation structure, we explored long short-term memory (LSTM). The CLIP-LSTM model demonstrated superior performance com- pared to the encoder-decoder models, achieving a BLEU-1 score of 0.904 and a BLEU-4 score of 0.640. Additionally, among the CNN-LSTM models, EfficientNetV2M-LSTM exhibited the highest performance with a BLEU-1 score of 0.896 and a BLEU-4 score of 0.586 while using a single layer LSTM.

Multi-Group Reinforcement Learning for Electrolyte Repletion (Under review)

Published in , 2022

When considering off-policy reinforcement learning methods for treatment policies in healthcare data, it is generally the case that the patient population is diverse and has different chronic conditions that we would like to take into account when identifying optimal treatment policies. In this work, we use multi-group Gaussian process regression models in a fitted Q-iteration framework to allow us to model these different patient sub- groups and adapt the optimal policies to each subgroup. Concurrently, we es- timate these functions across the entire patient population. Finally, we apply our multi-group reinforcement learning (MGRL) framework to the problem of optimal treatment policies for electrolytes with pre-existing medical conditions to assess performance against other state-of-the-art methods. We show that MGGP supersedes the performance of other models in addressing group structure in reinforcement learning settings due to the robust covariance functions which has been adapted to learn the different behaviours for multiple groups while maintaining a single policy.