SSM

State Space Models (SSM) are a class of architectures designed for efficient sequence modeling and handling long-range dependencies.

Recent Developments

hybrid SSM-Transformer architectures: Emerging focus on combining the efficiency of SSMs with the performance of Transformers.
Jamba 1.7 (AI21 Labs): A recent release (available in Jamba Mini and Jamba Large) that utilizes a hybrid architecture to support a 256k context window.