Prompt2Perturb (P2P): Text-Guided Diffusion-Based Adversarial Attacks on Breast Ultrasound Images
Yasamin Medghalchi, Moein Heidari, Clayton Allard, Leonid Sigal, Ilker Hacihaliloglu
2024-12-16
Summary
This paper introduces a new method called Prompt2Perturb (P2P) that helps create subtle attacks on breast ultrasound images used in cancer diagnosis. These attacks can trick deep learning models into making mistakes without being easily noticed, which is important for testing the reliability of these medical systems.
What's the problem?
Deep neural networks (DNNs) are used to improve breast cancer diagnosis from ultrasound images, but they can be easily fooled by small changes in the images, known as adversarial attacks. These attacks can lead to incorrect diagnoses, raising concerns about the safety and reliability of these systems, especially since traditional attack methods often do not align with how humans perceive changes in images.
What's the solution?
The researchers developed Prompt2Perturb (P2P), a method that uses language instructions to guide the creation of these subtle attacks. Instead of needing large datasets or retraining models, P2P directly updates text prompts to generate effective attack examples while keeping the ultrasound images looking natural. This approach improves efficiency and maintains image quality, making it easier to test DNNs without compromising their diagnostic capabilities.
Why it matters?
The P2P method is significant because it enhances the ability to evaluate and improve the security of medical imaging systems. By demonstrating better performance than existing techniques, it helps ensure that DNNs used in breast cancer diagnosis are more robust against potential attacks, ultimately contributing to safer healthcare practices.
Abstract
Deep neural networks (DNNs) offer significant promise for improving breast cancer diagnosis in medical imaging. However, these models are highly susceptible to adversarial attacks--small, imperceptible changes that can mislead classifiers--raising critical concerns about their reliability and security. Traditional attacks rely on fixed-norm perturbations, misaligning with human perception. In contrast, diffusion-based attacks require pre-trained models, demanding substantial data when these models are unavailable, limiting practical use in data-scarce scenarios. In medical imaging, however, this is often unfeasible due to the limited availability of datasets. Building on recent advancements in learnable prompts, we propose Prompt2Perturb (P2P), a novel language-guided attack method capable of generating meaningful attack examples driven by text instructions. During the prompt learning phase, our approach leverages learnable prompts within the text encoder to create subtle, yet impactful, perturbations that remain imperceptible while guiding the model towards targeted outcomes. In contrast to current prompt learning-based approaches, our P2P stands out by directly updating text embeddings, avoiding the need for retraining diffusion models. Further, we leverage the finding that optimizing only the early reverse diffusion steps boosts efficiency while ensuring that the generated adversarial examples incorporate subtle noise, thus preserving ultrasound image quality without introducing noticeable artifacts. We show that our method outperforms state-of-the-art attack techniques across three breast ultrasound datasets in FID and LPIPS. Moreover, the generated images are both more natural in appearance and more effective compared to existing adversarial attacks. Our code will be publicly available https://github.com/yasamin-med/P2P.