DIET: Machine Unlearning on a Data-Diet

Abstract

Machine unlearning aims to remove targeted knowledge from a trained model. Most existing methods use retained training data to protect the model’s remaining utility, which can be difficult to access for privacy and scalability reasons. DIET studies the harder retain-data-free setting for vision-language models.

Core idea

DIET formulates unlearning in hyperbolic space. Forget embeddings are moved towards semantically mismatched prototypes at the boundary of the Poincaré ball, where geodesic distance grows rapidly. A Busemann-guided objective controls this trajectory, while optimal transport selects suitable mismatched prototypes for individual samples.

Results

Across Flowers102, OxfordPets, and StanfordCars, the method reaches an average forget accuracy of 8.06% while preserving 69.04% utility using 16 samples per concept. The experiments show a substantial improvement over retain-free baselines and competitive performance against methods that use retained data.

Why it matters

The work provides a geometric route to selective forgetting without requiring the original retain set. This is useful when data access is restricted or when multimodal training data is too expensive to reconstruct.