OpenAI's latest model Astra uses a byte structure: Transformer double-counting at the same level

The Astra model, to be released by OpenAI, uses a revolving reasoning structure, known as recurrent depth, or looped transformer. Unlike common transformer, which generates a token fixed through a layer of network, Astra allows the same information to pass through the same layer over and over again and to be counted and re-exported. This architecture was made public last year byte, and the Ouro model, published by the Seed team, similarly uses looped language model, allowing a group of transformer layers to recycle and increase the number of reasoning calculations. Smaller models can thus use more calculations to achieve near larger models. However, such a structure also raises security concerns, with some of the reasoning taking place in a state of internal concealment, where humans are unable to see the complete textual record and have difficulty in checking whether the model is not in conformity. OpenAI therefore limits the extent to which Astra uses recurrent depth to preserve readable thought chains and is prepared to add additional controls。