A National Science Foundation testbed for future supercomputing technology, and the first computer outside Japan built on the Fujitsu A64FX processor.
In service
2020 to 2025
NSF award
OAC-1927880
A64FX nodes
176
Node hours per year
1.5 million
What Ookami was
Ookami gave researchers nationwide access to the A64FX processor that Riken and Fujitsu developed for Japan's path to exascale computing, the same processor behind Fugaku, which held the top spot on the world supercomputer list until June 2022. It was the first such machine outside Japan.
The interest was architectural. The A64FX is an Arm based, multi core, 512 bit vector processor paired with ultrahigh bandwidth memory, and the open question was whether that combination could deliver very high performance across a wide range of applications while retaining familiar and successful programming models. It supported a wide range of data types and served both traditional high performance computing and big data work.
To let users explore current computing technologies and contrast performance and programmability against the A64FX, Ookami also carried a small set of comparison nodes on the same filesystem and the same scheduler.
From October 2022 Ookami was officially an ACCESS resource provider, with 90 percent of its resource available via ACCESS.
Final configuration
| Node type | Nodes | Cores each | Memory each |
|---|---|---|---|
| Fujitsu A64FX | 176 | 48 | 32 GB high bandwidth |
| NVIDIA Grace superchip, dual socket | 2 | 144 | Not published |
| Thunder X2, dual socket | 2 | 64 | 256 GB |
| AMD Milan, dual socket | 1 | 64 | 512 GB |
| Intel Skylake, dual socket | 1 | 36 | 192 GB |
The platform was an HPE Apollo 80, a design that began at Cray. Every A64FX node carried a 512 GB local SSD, and shared storage was a high performance Lustre filesystem of about 800 TB. The system provided roughly 1.5 million node hours a year.
Project history
Storage system upgrade, the earliest event in the project announcement record.
Eva Siegmann joined as lead research scientist. Later the same month the system passed its first year review, was formally accepted, and all vendors were paid.
Three day hands on training with Arm, covering the Arm, Fujitsu, Cray and open source toolchains and tuning for A64FX.
First Ookami user group meeting.
Ookami became officially an ACCESS resource provider, with 90 percent of its resource available via ACCESS.
Second Ookami user group meeting.
National Science Foundation technical talk on the first three years of the testbed.
Open OnDemand brought browser based access to the system.
Two NVIDIA Grace superchip nodes were added, each with 144 cores.
Second year of the ByteBoost cybertraining program, run across Ookami, Neocortex and ACES. ByteBoost continues under its own funding and is not affected by the retirement of Ookami.
Ookami reached end of life.
Data migration window.
The research record
Ookami supported 184 research projects, 97 allocated through Stony Brook University and 87 through ACCESS. The work produced 89 papers, two posters and two further publications between 2021 and 2025, spanning astrophysics, oceanography, condensed matter physics, molecular dynamics, compiler and runtime research, and machine learning.
Citing Ookami
Work that used Ookami should carry the project acknowledgement, including work still in review. Reproduce it exactly as written. It names Research Computing and Cyberinfrastructure, a former name for Research Computing and Informatics, and should not be updated to the current name.
The authors would like to thank Stony Brook Research Computing and Cyberinfrastructure, and the Institute for Advanced Computational Science at Stony Brook University for access to the innovative high-performance Ookami computing system, which was made possible by a $5M National Science Foundation grant (#1927880).
Funding and partners
Ookami was supported by the National Science Foundation under award OAC-1927880 and operated by the Institute for Advanced Computational Science at Stony Brook University, with the Division of Information Technology at Stony Brook and the Center for Computational Research at the University at Buffalo.