Integrating U-Net and LLM Agents for Pancreatic Ductal Adenocarcinoma Diagnosis

Abstract

This study proposes an AI driven pipeline that combines, pancreas segmentation outcome for Pancreatic Ductal Adenocarcinoma (PDAC) diagnosis with a large language model (LLM) agent to enhance diagnostic and clinical analysis. Building upon already established deep learning approaches in medical imaging, our project aims to extend traditional UNet segmentation methods by integrating the capabilities of an LLM agent to provide detailed diagnostic information for medical practitioners. Using the Pancreas Decathlon dataset, 3D CT scans are processed and trained over multiple different iterations utilizing attention mechanisms, sparse categorical cross entropy and Tversky loss. The predicted segmentation labels are used by the LLM to infer diagnostic details such as the stage of the disease progression and integrate results with the electronic health records for longitudinal study. Ultimately, this integrated framework aims to assist medical practitioners in diagnosing PDAC more effectively while offering additional supplemental information.