Solana’s Mainnet Beta network was down for five hours on February 6th because of a bug in the program cache system. This issue, coterminous to Devnet outage, was caused by an infinite loop as a result of legacy program loaders.
In addition, there is a bug in “Loaded Programs,” a system that caches the most frequently used programs for quicker execution. For older variants, it lacked a capability to keep track of deployment artifacts, which led to an infinite loop of recompilation when invoked.
The process of disabling/patching the problematic legacy loaders was done fast, hence, the disablement of the loaders blocked triggering of the bug. Loaded Program improvements are one of the permanent changes planned for future versions.
The outage lasted 5 hours, disaffecting Solana users and Solana applications. No data loss was recorded, thus stressing the need for systematic testing and a fast response to bugs.
Also read: NY Attorney General Expands DCG Fraud Case to $3B