vec2vec: Embeddings are No Longer Secret

Quick Overview

The Cornell researchers demonstrated that Vec2Vec, a method using a universal latent space derived from unsupervised learning, successfully translates embeddings between different modalities (like text and images) and even different models (T5-based vs. BERT-based) with high accuracy, suggesting that semantic geometry is preserved across these transformations, which has significant implications for AI interoperability and security.

Key Points: Vec2Vec achieves successful translation between different model embeddings (T5 vs. BERT) and modalities (text vs. image) by leveraging a universal latent space. The method achieved near-optimal assignments, with cosine similarity scores hitting 0.92 for the T5-to-BERT translation. The research confirmed that the semantic geometry learned by the models is preserved, as demonstrated by the consistent closeness of related embeddings in the universal space. A key security finding is that the translation preserves enough semantic information to reconstruct sensitive data, such as private NREON employee records, from encoded embeddings. The performance of the translation model (Vec2Vec) significantly outperformed baseline methods, achieving a rank of 1 for the T5-to-BERT translation and significantly outperforming random guessing. The method proved effective across different domains, successfully translating embeddings between colloquial tweets and highly specialized medical records (e.g., 'alveolar periostitis').

Context: The discussion centers on a research paper from Cornell exploring the concept of universal embedding spaces for different AI models and data modalities, specifically focusing on a technique called Vec2Vec. This method aims to create a shared geometric representation that allows different AI systems, trained on different data or architectures (like T5 or BERT), to understand each other's learned concepts without explicit cross-alignment data.

Detailed Analysis

The video explains the Vec2Vec research from Cornell, which addresses the challenge of translating embeddings between different AI models and data types that lack direct parallel data. The core innovation is creating a shared, universal latent space where semantic relationships are preserved. The researchers trained two sets of models: one based on T5 and another on BERT. They then showed that by mapping the embeddings from one model's space into the other's using this universal structure, they could achieve highly accurate translation. For instance, translating 'Become A Member Today!' from a T5 embedding space to a BERT embedding space resulted in a cosine similarity score of 0.92, indicating near-perfect alignment. This consistency in geometric structure across different models was maintained even when translating between vastly different domains, such as casual tweets and complex medical records (like 'alveolar periostitis'). The security implication is significant: if an attacker steals embeddings derived from sensitive data (like NREON employee records), the semantic information is preserved well enough that zero-shot attribute inference or reconstruction of the original data is possible, demonstrating that embeddings are not inherently secure if a translator model is available.