Smooth migration for BioCommons’ Apollo service: AARNet and Globus support critical data transfer
This AARNet news story is republished with permission
As part of planned infrastructure changes at the Pawsey Supercomputing Research Centre’s Nimbus cloud, Australian BioCommons began preparing to move key components of its Australian Apollo Service to new infrastructure. A central part of that transition involved securely transferring over 15 terabytes of user data — ensuring research continuity for life science teams around the country.
To support the process, BioCommons turned to AARNet and Globus for a robust, high-performance solution.
Powering digital biology
Australian BioCommons is a national research infrastructure initiative accelerating life sciences research by providing digital platforms and services for data-intensive biology. One of its flagship offerings is the Australian Apollo Service — a hosted virtual environment that enables researchers to launch, manage and run complex bioinformatics workflows on demand.
The Pawsey Supercomputing Research Centre, a Tier 1 government-funded national facility, hosted the Apollo service from 2020, providing access to genome curation and visualisation software to hundreds of researchers across Australia.
As the underlying cloud infrastructure at Pawsey (Nimbus) was being phased out and Apollo’s environments could be redeployed elsewhere, the associated user data, including datasets vital for current and future analysis, needed to be transferred efficiently and with minimal disruption to services.
Finding the right approach for large-scale transfers
Initial attempts to move the data using traditional tools quickly highlighted performance limitations. Transferring even a single terabyte took multiple days, and with the additional goal of preserving metadata using tarball archives, BioCommons needed a more scalable and dependable approach.
To meet these needs, the team explored using Globus, a research-grade data transfer service designed for large-scale scientific workflows. Critically, Globus operates over the AARNet research and education network, which provides the high-speed connectivity required for rapid, reliable transfers across sites.
“What stood out with Globus was how straightforward it was to get going,” said Justin Lee, Platform Developer and System Administrator at Australian BioCommons. “The documentation AARNet provided made it easy to deploy, and once set up, it just worked. We didn’t need to manage every detail as it handled the complexity for us.”
Setting up endpoints and running the transfer
AARNet provided setup guides and workshop resources to assist with deployment. Working from these materials, Justin was able to spin up the required Globus endpoints in less than a day — one at Pawsey, where Apollo had been running, and the other at the new AARNet-hosted Nectar Research Cloud node, where Apollo would be hosted going forward.
After a successful test transfer, the full migration began the week ahead of the Easter holiday period. With Globus managing the transfer, the system automatically handled retries, integrity checks, and restarts.
By the time teams returned after Easter, the entire 15 TB of data had been moved, smoothly, securely, and without interruption.
Supporting research continuity through reliable infrastructure
For researchers using the Australian Apollo Service, no changes were needed to how they interacted with their environments. Once the transition was complete, data access resumed smoothly and workflows continued as before.
“From our perspective, it was great to see how quickly the BioCommons team could get up and running with Globus using the self-service resources,” said Greg D'Arcy, Digital Research Product Manager at AARNet. “It shows how institutions can take ownership of complex transfers without needing deep expertise, especially when time is limited.”
“It all worked as expected,” added Justin. “Having reliable tools and a clear setup process made the whole migration straightforward.”
Building resilience for life sciences research
This successful migration showcases the value of collaborative partnerships and purpose-built infrastructure in enabling modern, data-driven research. By leveraging Globus and the AARNet research network, BioCommons ensured that life science researchers could continue their work without missing a beat, even during a significant infrastructure shift.
The Australian Apollo Service forms part of the national Australian BioCommons infrastructure. BioCommons partners with QCIF to manage the Australian Apollo Service, which is underpinned by computational resources provided by AARNet’s ARDC Nectar Research Cloud node. These efforts are supported by funding from Bioplatforms Australia and the Queensland Government RICF. Bioplatforms is enabled by NCRIS.