This is the abstract and summary of our new paper. We show that vision-language models can learn to reconstruct harmful images from benign-looking patches scattered across training data—a phenomenon we call $visual stitching$ . This ability allows dangerous content to bypass moderation and be reassembled during inference, raising critical safety concerns for VLMs.

Authors: Zhanhui Zhou, Lingjie Chen, Chao Yang, Chaochao Lu.

See our project page and full code repo at Github.

Figure 1: Illustration of visual stitching. (Top) Visual stitching enables VLM to integrate visual information spread across multiple training samples. After finetuning on ${(patch, ID)}$ of a cat, VLMs can verbalize the $ID$ when given the full $image$ or a text $reference$ to the image, despite never training on them. (Bottom) Visual stitching enables adversarial attacks that bypass data moderation. While the $image$ of a bloody scene may be flagged as unsafe and removed, many of its $patch e s$ are not. Training on ${(patch, text)}$ pairs split from harmful samples can easily bypass frontier moderation and cause VLMs to generate adversarial outputs at deployment.

Abstract

One way to mitigate risks in vision-language models (VLMs) is to remove dangerous samples in their training data.

However, such data moderation can be easily bypassed when harmful images are split into small, benign-looking patches, scattered across many training samples. VLMs may then learn to piece these fragments together during training and generate harmful responses at inference, either from full images or text references.
For instance, if trained on image patches from a bloody scene paired with the descriptions $‘ ‘ s a f e,^{''}$ VLMs may later describe, the full image or a text reference to the scene, as $‘ ‘ s a f e .^{''}$

We define the core ability of VLMs enabling this attack as $visual stitching$ ---the ability to integrate visual information spread across multiple training samples that share the same textual descriptions.

In our work, we first demonstrate visual stitching abilities in common open-source VLMs on three datasets where each image is labeled with a unique synthetic ID: we split each $(image, ID)$ pair into ${(patch, ID)}$ pairs at different granularity for finetuning, and we find that tuned models can verbalize the correct IDs from full i...

Posts

Wikitag Contributions

Comments