AD-Copilot

Comparison-aware anomaly detection with vision-language models. Extends Qwen2.5-VL-7B with a novel comparison-aware visual encoder achieving 78.74% on OmniDiff benchmark.

Paper | Code | Demo

Key Innovation

• ADCopilotCompareVisualEncoder: Bidirectional cross-attention comparing reference and test images
• 100 comparison tokens per image pair injected into the language model
• State-of-the-art on industrial anomaly detection benchmarks

Quick Start

import torch
from transformers import AutoModelForVision2Seq, AutoProcessor
from qwen_vl_utils import process_vision_info

model = AutoModelForVision2Seq.from_pretrained(
    "jiang-cc/AD-Copilot",
    torch_dtype=torch.bfloat16, device_map="auto", trust_remote_code=True,
)
processor = AutoProcessor.from_pretrained(
    "jiang-cc/AD-Copilot",
    min_pixels=64*28*28, max_pixels=1280*28*28, trust_remote_code=True,
)

messages = [{"role": "user", "content": [
    {"type": "image", "image": "good.png"},
    {"type": "image", "image": "test.png"},
    {"type": "text", "text": "The first image is good. Is there any anomaly in the second image? A.yes, B.no."},
]}]

text = processor.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
image_inputs, _ = process_vision_info(messages)
inputs = processor(text=[text], images=[image_inputs], return_tensors="pt").to(model.device)

with torch.inference_mode():
    ids = model.generate(**inputs, max_new_tokens=128, do_sample=False)
trimmed = [o[len(i):] for i, o in zip(inputs.input_ids, ids)]
print(processor.batch_decode(trimmed, skip_special_tokens=True)[0])

Results (OmniDiff Benchmark)

Model	Visited IAD	Avg ACC
MiniCPM-V2.6	0	67.90%
EIAD	128k	69.40%
Qwen2.5-VL	0	72.19%
AD-Copilot (Ours)	206k	78.74%

Architecture

• Base: Qwen2.5-VL-7B-Instruct (28 layers, 3584 hidden)
• Vision: Qwen2.5-VL ViT (32 layers, 1280 hidden)
• Comparison Encoder: Bidirectional cross-attention + query decoder (100 tokens)
• Dtype: bfloat16

Citation

@article{adcopilot2025,
  title={AD-Copilot: Comparison-Aware Anomaly Detection with Vision-Language Models},
  author={Jiang, Xi and others},
  journal={arXiv preprint arXiv:2603.13779},
  year={2025}
}

Comparison-token positional encoding update

The inference code uses diagonal spatial positions for comparison tokens: their height and width indices advance from 0 to compare_token_size - 1, while their temporal coordinate remains fixed. This update changes position indices, including downstream position offsets; checkpoint tensors are unchanged.

To reproduce the previous shared-position inference behavior, pin revision="3ddf3a219e5c95223b8c13da18c7b8e53479c5a0" when loading the model and processor.