TwoTower Architecture
TwoTower Architecture generally refers to a dual-encoder framework where two separate neural networks (towers) process different modalities or data types independently before interacting. While traditionally associated with Recommendation Systems and Information Retrieval for efficient similarity search, recent developments have adapted this paradigm for generative tasks.
Core Mechanism
- Independent Encoding: Two distinct towers process inputs (e.g., query and item, or text and latent space) in parallel.
- Interaction Layer: Outputs are combined via dot-product similarity, attention mechanisms, or fusion layers.
- Efficiency: Allows for pre-computation of one tower (e.g., database items) to accelerate inference.
Recent Developments: Parallel Diffusion
Recent research has extended the TwoTower concept to Diffusion Models for text generation, challenging the dominance of autoregressive Transformers.
- NVIDIA’s Implementation: NVIDIA introduced a parallel diffusion architecture leveraging the TwoTower design to accelerate text generation.
- Key Innovation: Unlike sequential autoregressive decoding, this approach utilizes parallel processing capabilities inherent in diffusion models, potentially reducing latency for long-context generation.
- Context: This marks a shift from diffusion models being primarily used for Image Generation (e.g., Stable Diffusion) to competitive text synthesis.