SamplingTAR: Training-free Defense against Typographic Attacks

This demo implements the training-free concept localization method from Towards Robustness against Typographic Attack with Training-free Concept Localization. Upload an image containing a typographic attack (text overlaid on an image to fool CLIP), provide candidate labels, and the defense will ablate the attention heads responsible for reading text to recover the true object prediction.