Hidden State
Hidden State refers to the internal, non-observable variables or representations within a system (biological or artificial) that determine its future behavior and outputs, distinct from its immediate inputs or visible outputs. In neural-networks, this typically manifests as the activations of hidden layers; in Cognitive Science, it relates to unconscious processing or working memory buffers.
Key Characteristics
- Latency: Information is processed internally before manifesting in observable behavior.
- Compression: High-dimensional input data is mapped to lower-dimensional internal representations.
- Context Retention: Maintains information across time steps (e.g., in Recurrent Neural Networks or transformers via attention mechanisms).
AI Interpretability & Internal Representations
Recent research suggests that large language models develop structured internal spaces that function analogously to cognitive workspaces.
- Anthropic’s J-space: Research indicates an emergent internal global workspace within AI models, termed “J-space.” This space appears to integrate information across the model’s layers, facilitating complex reasoning and coherence. See Anthropic’s J-space: Emergent Internal-Global-Workspace-in-AI-Models for detailed analysis.
- Mechanistic Interpretability: Efforts to reverse-engineer these hidden states reveal that models do not merely pattern-match but utilize specific internal circuits for logic and memory retrieval.