Context-Aware Prompt Injection Detection via Small Language Models in RAG Pipelines: nokast-secureRAG

Key points

• Local Retrieval-Augmented Generation (RAG) systems are vulnerable to indirect prompt injection and retrieval poisoning, where malicious instructions in retrieved content are mistaken for trusted input.1
• It jointly evaluates the user query and retrieved context, classifying the pair as safe, suspicious, or malicious.1
• On a held-out in-distribution test set, the model achieves 0.994 detection recall at a 0.026 false-positive rate and a 0.006 attack-success-rate proxy, outperforming a regex baseline, a public prompt-injection classifier, Llama Guard 3 1B, and its untuned version.1
• Ablations indicate that retrieved context carries the main attack signal, while query-context consistency reduces false positives.1

Sources (1)

• [1]Context-Aware Prompt Injection Detection via Small Language Models in RAG Pipelines: nokast-secureRAG
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 20, 03:26 PM

  We present nokast-secureRAG, a roughly 0.5B-parameter small language model that acts as a semantic firewall between retrieval and generation.

  Local Retrieval-Augmented Generation (RAG) systems are vulnerable to indirect prompt injection and retrieval poisoning, where malicious instructions in retrieved content are mistaken for trusted input.

Extractive summary: sentences quoted from the sources.