Penguin Solutions and Marvell want to move a major part of AI memory away from individual GPU packages. Their MemoryAI Photonic Appliance, announced November 16 at Supercomputing 2026 in Chicago, is designed to share up to 16TB of DDR5 and 576GB of HBM3E across servers linked by light. The proposed hardware targets 57.6 terabits per second of aggregate appliance bandwidth, according to Penguin’s original announcement. Those specifications are design goals, not independently verified shipping-system benchmarks.
The timing is important: Penguin plans to begin sampling to selected customers in the second half of 2027. A physical demonstration at SC26 does not mean customers can install a production appliance today. Marvell’s November 13 conference announcement also outlined Photonic Fabric, CXL expansion cards and optical interconnect demonstrations.
In an August 2026 technical demonstration, Marvell’s Uday Poosarla explains the company’s optical memory fabric; the video predates the SC26 appliance announcement. Original video.
Why AI needs another memory tier
A GPU has exceptionally fast but finite high-bandwidth memory (HBM). As large language models handle more simultaneous users and longer conversations, their key-value (KV) caches compete with model weights and intermediate tensors for that limited space. Our GPU fact-capacity comparison demonstrates why a billion 384-dimensional FP16 vectors need 768GB before indexing, while AtlasKV illustrates a different strategy: leave large knowledge structures outside active GPU memory.
The new appliance builds a shared memory tier. Its proposed 576GB HBM3E pool is twice the 288GB physically attached to one NVIDIA B300. Even AMD’s 432GB MI455X has less nominal HBM capacity than the planned shared HBM pool, although onboard and remote memory do not have equivalent latency or throughput.

16TB can hold far more data, but speed matters
The larger tier is planned 16TB DDR5 RDIMM capacity, roughly 55.6 times the B300’s 288GB of HBM by nominal byte count. Using a theoretical 768-byte embedding, 16TB could store approximately 20.8 billion vectors before overhead. It would not allow a model to attend to 20.8 billion facts instantaneously, and it does not reproduce AtlasKV’s evaluated grounding accuracy.

At SC26, Marvell’s Preet Virk and Penguin’s Andy Mills discuss photonic pooling in a theCUBE interview published November 18, 2026. SiliconANGLE discloses sponsorship by Penguin Solutions. Original video.
Memory would be accessible through Marvell’s optical network interfaces, with CXL 3.0 support for CPU access. Penguin lists direct HBM, DDR5 direct and HBM-cached DDR5 modes. A 57.6Tbps headline converts to 7.2TB/s of target aggregate link bandwidth before protocol overhead. It is not a demonstrated per-GPU memory-transfer rate, and end-to-end performance will depend on access patterns, congestion and software.
Where the architecture could matter
Shared memory could reduce duplicated KV caches across inference servers and allow operators to allocate resources to workloads as needed. Penguin’s existing DDR4-reuse CXL expansion card addresses a related but more immediate capacity problem. Our CXL memory-pooling guide explains the protocol, while Lambda’s optical-switch deployment highlights networking infrastructure around large GPU clusters.
The strongest claim has yet to be proven: whether pooled memory improves actual token throughput, energy use and dollar-per-token economics at scale. Penguin has not published independent production comparisons against equivalent local-HBM systems. Until customer sampling and repeatable benchmarks arrive, the appliance is a substantial engineering proposal rather than a delivered solution to the GPU memory wall.
Sources and verification
Primary sources: Penguin Solutions, November 16, 2026; Marvell/Penguin collaboration; NVIDIA HGX specifications; AMD MI455X product specifications; and SiliconANGLE’s SC26 interview report. Capacity and unit-conversion arithmetic are editorial calculations; no prototype was independently tested.
Visual note: The featured anime illustration is fictional and represents rack-scale optical memory sharing, not a photograph of Penguin’s product. Both charts compare nominal capacities, not tested performance.
Editor’s Note: We volunteer daily to ensure the credibility of the information on this platform is Verifiably True. If you would like to support to help further secure the integrity of our research initiatives, please scan the QR code below

BitcoinVersus.tech provides technical and financial journalism for informational purposes only and does not provide investment, financial, legal or cybersecurity assurances.

Leave a Reply