Smooth migration for BioCommons’ Apollo service: AARNet and Globus support critical data transfer

This AARNet news story is republished with permission

As part of planned infrastructure changes at the Pawsey Supercomputing Research Centre’s Nimbus cloud, Australian BioCommons began preparing to move key components of its Australian Apollo Service to new infrastructure. A central part of that transition involved securely transferring over 15 terabytes of user data — ensuring research continuity for life science teams around the country.

To support the process, BioCommons turned to AARNet and Globus for a robust, high-performance solution.

Powering digital biology

Australian BioCommons is a national research infrastructure initiative accelerating life sciences research by providing digital platforms and services for data-intensive biology. One of its flagship offerings is the Australian Apollo Service — a hosted virtual environment that enables researchers to launch, manage and run complex bioinformatics workflows on demand.

The Pawsey Supercomputing Research Centre, a Tier 1 government-funded national facility, hosted the Apollo service from 2020, providing access to genome curation and visualisation software to hundreds of researchers across Australia.

As the underlying cloud infrastructure at Pawsey (Nimbus) was being phased out and Apollo’s environments could be redeployed elsewhere, the associated user data, including datasets vital for current and future analysis, needed to be transferred efficiently and with minimal disruption to services.

Finding the right approach for large-scale transfers

Initial attempts to move the data using traditional tools quickly highlighted performance limitations. Transferring even a single terabyte took multiple days, and with the additional goal of preserving metadata using tarball archives, BioCommons needed a more scalable and dependable approach.

To meet these needs, the team explored using Globus, a research-grade data transfer service designed for large-scale scientific workflows. Critically, Globus operates over the AARNet research and education network, which provides the high-speed connectivity required for rapid, reliable transfers across sites.

“What stood out with Globus was how straightforward it was to get going,” said Justin Lee, Platform Developer and System Administrator at Australian BioCommons. “The documentation AARNet provided made it easy to deploy, and once set up, it just worked. We didn’t need to manage every detail as it handled the complexity for us.”

Setting up endpoints and running the transfer

AARNet provided setup guides and workshop resources to assist with deployment. Working from these materials, Justin was able to spin up the required Globus endpoints in less than a day — one at Pawsey, where Apollo had been running, and the other at the new AARNet-hosted Nectar Research Cloud node, where Apollo would be hosted going forward.

After a successful test transfer, the full migration began the week ahead of the Easter holiday period. With Globus managing the transfer, the system automatically handled retries, integrity checks, and restarts.

By the time teams returned after Easter, the entire 15 TB of data had been moved, smoothly, securely, and without interruption.

Supporting research continuity through reliable infrastructure

For researchers using the Australian Apollo Service, no changes were needed to how they interacted with their environments. Once the transition was complete, data access resumed smoothly and workflows continued as before.

“From our perspective, it was great to see how quickly the BioCommons team could get up and running with Globus using the self-service resources,” said Greg D'Arcy, Digital Research Product Manager at AARNet. “It shows how institutions can take ownership of complex transfers without needing deep expertise, especially when time is limited.”

“It all worked as expected,” added Justin. “Having reliable tools and a clear setup process made the whole migration straightforward.”

Building resilience for life sciences research

This successful migration showcases the value of collaborative partnerships and purpose-built infrastructure in enabling modern, data-driven research. By leveraging Globus and the AARNet research network, BioCommons ensured that life science researchers could continue their work without missing a beat, even during a significant infrastructure shift.

The Australian Apollo Service forms part of the national Australian BioCommons infrastructure. BioCommons partners with QCIF to manage the Australian Apollo Service, which is underpinned by computational resources provided by AARNet’s ARDC Nectar Research Cloud node. These efforts are supported by funding from Bioplatforms Australia and the Queensland Government RICF. Bioplatforms is enabled by NCRIS.

Students investigate beetle genetic variation using new Apollo training instance

Teaching genetics is easier and more effective now that a powerful online tool is being made available to Australian researchers and trainers. Students can use a tailored training instance of the web-browser accessible system, Apollo, for real-time collaborative curation and editing of genome annotations.

This new feature of the Australian Apollo Service allows trainers to focus on teaching genome annotation curation, without being burdened by installation and maintenance of Apollo. All the hosting and system administration of customised Apollo instances is taken care of for service users. Life scientists and research consortia based in Australia can apply for an instance that is ready-made for training, and is up to date with the latest release of the software. 

Molecular evolution and population genetics researcher, Assoc Prof Charles Robin signed on to the Australian BioCommons’ Apollo service through a recommendation from a colleague at Melbourne Bioinformatics. After facilitating some Australian BioCommons online workshops, Charles now uses an Apollo training instance when teaching genetics to third year undergraduates at the University of Melbourne. Each student is provided with their own login where they can visualise DNA sequences, perform annotations and explore without overwriting each others’ work. Charles finds Apollo ideal for teaching:

“It’s great that the transcriptome maps really well to the genome. By playing within Apollo, students get to see the AC / GT rule and how this can be reinforced by the transcript. Things like alternate splicing are also easily visualised.”

Two adult beetles, one with a CRISPR deletion in the gene called cardinal that is involved in eye colour. The genetic modifications were performed in Charles’ lab.

The class answers research questions like ‘which of the genes in this region includes this particular mutation?’ or ‘how do you find the candidate gene in this region?’ using real world data from a beetle with a gene mutation. Charles deliberately chooses genomes that are ‘untidy’:

“You want a non-model genome to identify gaps and changes. Students can have an expectation that all annotations on a genome are true, and using Apollo allows them to see that this is not the case.”

Charles rates the reliability of the software as a key factor in why he uses Apollo for teaching genetics:

“So much software gets left without regular updates and from year to year you realise that it isn’t maintained or updated. So we look for things that are stable - this is the reason we call on the Australian Apollo Service.”

Using Apollo to curate genome annotations

The Australian Apollo Service performs all system administration, build and deployment of the instance on behalf of users, with support provided through a help desk, user documentation and training events. The deployment of a full technology stack, long term hosting of data, maintenance updates and security are all covered, providing customised, local instances of the Apollo software for individual genome projects or training. 

Australian BioCommons are working with Apollo project principal investigator Prof Ian Holmes to understand researchers’ needs within Apollo, with an aim to provide improved annotation and visualisation features for genome annotation research.

Launch Apollo to learn more

Australian BioCommons delivers collaborative distributed infrastructure to enable life science research. BioCommons partner QCIF is offering the Apollo Portal service, and it is underpinned by computational resources provided by the Pawsey Supercomputing Research Centre. These efforts are supported by funding from Bioplatforms Australia (BPA) and Australian Research Data Commons (ARDC). BPA and ARDC are enabled by NCRIS.

Cracking the code of Australia's most invasive species

Dr Rahul Rane (left) and Dr Tom Walsh will sequence hundreds of genomes for the Australian Pest Genome Project

We're excited to partner with CSIRO and Australian Research Data Commons (ARDC) to share the genomic data that will underpin species-specific management of pests & weeds in the future. This important project will soon host the assembled and annotated genomes on the Australian Apollo Service, allowing interactive browsing and collaborative curation. We are proud to support the Australian Pest Genome Partnership as we work towards making genomic data more easily accessible and usable to support industry, government and the scientific community in managing pests.

The story reproduced below New CSIRO project to crack the codes of Australia's most invasive species was recently published on the CSIRO news page.

CSIRO, Australia’s national science agency, has embarked on an ambitious new project to unravel the genetic blueprints of Australia’s top pest and invasive species to better enable their management or eradication.

The Australian Pest Genome Partnership (APGP) will generate the genomic data of hundreds of pests and weeds and make it freely available, along with digital solutions to help analyse the data. The data will assist researchers working on pest and weed species and underpin next generation species-specific solutions.

Invasive species have cost Australia $390 billion over the past six decades, with weeds costing the agriculture sector at least $5 billion a year. This presents a significant burden to Australia’s agriculture and livestock industries, as well as the significant and ongoing environmental impacts created by these invasive species.

APGP has now prepared its first 28 genomic datasets, and this year, with its collaborators, will make public genomic data assets for some of Australia’s top pest and invasive species such as mosquitoes, khapra beetle, cane toad, fall armyworm, fox, feral pig and cat as well as weeds such as wild radish, rye and rat’s tail grasses.

CSIRO principal research scientist Tom Walsh said data is easy but analysis is hard. APGP intends to make genomic data more easily accessible and usable to support industry, government and the scientific community in managing pests.

“This project has the potential to drive new science and digital innovations to safeguard Australia’s environment and biosecurity from existing and growing threats posed by invasive and pest species,” Dr Walsh said.

“In the same way genome sequencing has helped inform medical advice, pest genomes can help us unlock new ways of protecting our environment, agricultural sector and public health with a quick and targeted response.” he said.

CSIRO senior research consultant Rahul Rane said the fit-for-purpose genomics database being delivered through APGP will be a game-changer in invasive species control and management.

“Genomes and genetic diversity data can tell us all manner of things including where a particular pest species has travelled from, what environments it may thrive in, and whether it has developed resistance to chemicals and pesticides,” Dr Rane said.

“The more we know about the genetic characteristics of a pest, the better our ability to make informed decisions to effectively control or eliminate them safely.

“Ultimately research based on these new datasets will benefit all Australians by reducing public health risks and the impact of these pest species on our environment and agricultural production”.

Hundreds more pests are being sequenced this year, including jellyfish, invasive ants and beetles, termites, African boxthorn, crown of thorns starfish, ticks and head lice.

APGP is looking to partner with companies, government departments and other research organisations to continue sequencing and collating the genome of pests impacting Australia’s biosecurity.

The project received investment from the Australian Research Data Commons (ARDC), and has received co-investment from Australian BioCommons and Australian National University. Genomic datasets were developed in collaboration with Macquarie University, University of Melbourne, University of New South Wales, University of Queensland, University of Western Australia, Queensland Department of Agriculture and Fisheries and South Australian Research and Development Institute.

The ARDC and Australian BioCommons (Bioplatforms Australia) are funded by the National Collaborative Research Infrastructure Strategy (NCRIS).

Visit Australian Pest Genome Project to learn more, or contact the team to access the data.

Launching the new Australian Apollo Service for collaborative genome annotation

Australian BioCommons is thrilled to announce the launch of the Australian Apollo Service in partnership with QCIF and Pawsey Supercomputing Research Centre. This new service offers access to the popular tool, Apollo, which facilitates real-time collaborative curation and genome annotation editing, along with a valuable layer of IT support. The Australian Apollo Service allows researchers to focus on the genome annotation curation itself by taking care of all the system administration and hosting customised, local instances of Apollo.

Community roadmaps developed together with the Genome Annotation and Genome Assembly Communities highlighted the local need for access to Apollo. Many researchers who undertake genome annotation work identified Apollo as essential to their research as it allows them to collaboratively improve genome annotations that are the product of automated annotation tools or pipelines. But these researchers faced significant challenges in using this software as the overhead of deploying and managing an Apollo instance themselves was too high. 

Recognising this as a priority, Australian BioCommons has brought together partners at QCIF and Pawsey to build and deliver a service that is freely available to Australian-based research groups and research consortia. The complete system administration, build and deployment of the instance is done on behalf of researchers, with support provided through a help desk, user documentation and training events. The deployment of a full technology stack, long term hosting of data, maintenance updates and security are all covered by the Australian Apollo Service, providing customised, local instances of the Apollo software for individual genome projects.  

At our launch webinar, you’ll hear what’s possible from Dr Rahul Rane (CSIRO), Prof Sandie Degnan and Prof Bernie Degnan (University of Queensland) and Julia Voelker (Southern Cross University). The demonstration of what Apollo brings to these researchers’ genome annotation and curation workflows will be accompanied by an overview of what the Australian Apollo Service can offer from Dr Tiffanie Nelson (Australian BioCommons).

New users will be supported by a hands-on training workshop Refining genome annotations with Apollo in November led by experts in the Apollo tool and its use to support genome annotation, Dr Anthony Bretaudeau (French National Institute for Agriculture, Food, and Environment), Dr Helena Rasche (Erasmus Medical Center, The Netherlands) and Dr Sarah Williams (QCIF).

If you are interested in genome annotation, editing and curation, or you think that Apollo might be useful for your research, please join us for the webinar: Launching the new Apollo Service: collaborative genome annotation for Australian researchers on Wed 29 Sep, 2021 at 12:00 pm AEST.

BioCommons partner QCIF is offering the Apollo Portal service, and it is underpinned by computational resources provided by the Pawsey Supercomputing Research Centre. These efforts are supported by funding from the Queensland Government’s Research Infrastructure Co-investment Fund (RICF), Bioplatforms Australia (BPA) and Australian Research Data Commons (ARDC). BPA and ARDC are enabled by NCRIS.

Draft tea tree genome published by Southern Cross University team

A new genome assembly and annotation promises insights into the genetic foundations of economically valuable traits in tea tree (Melaleuca alternifolia). 

Given the therapeutic and cosmetic value of terpene-rich tea tree essential oil, the genetics and biochemistry of terpene biosynthesis have been studied extensively. The publication of the draft genome for M. alternifolia extends currently available resources to investigate the genome structure and gene family evolution, and will enable further comparative genomic studies in the Myrtaceae.

The work by Southern Cross University’s Julia Voelker, Mervyn Shepherd, and Ramil Mauleon, was reported this week in Gigabyte: A high-quality draft genome for Melaleuca alternifolia (tea tree): a new platform for evolutionary genomics of myrtaceous terpene-rich species.

In completing their high-quality draft genome, the researchers made excellent use of national research infrastructure provided by Australian BioCommons. They are one of a small group of early adopters who have been intensively testing a soon-to-be-launched service which will offer fully subsidised access for Australian based researchers to the proprietary bioinformatics pipeline, Fgenesh++, to enable automated eukaryotic genome annotation. The case for access to Fgenesh++ was identified via consultations with the BioCommons Genome Annotation community, which were captured in our Genome Annotation Infrastructure Roadmap for Australia. The service will be provided by BioCommons and hosted by Pawsey Supercomputing Research Centre. The BUSCO analyses documented in the publication were run on the flagship BioCommons service, Galaxy Australia.

Based on the new assembly, curation of the terpene synthase gene family in tea tree continues. Using another BioCommons service that is about to be launched, the SCU group used the Australian Apollo Service to visualise gene prediction results and manually improve their gene models. You can hear more about this work at the upcoming webinar: Launching the new Apollo Service: collaborative genome annotation for Australian researchers.

All Australian researchers are invited to join our communities which are formed around different research methodologies. Join to have your voice heard when we document community challenges, and see your needs met as the BioCommons continues to roll out improvements and solutions in shared bioinformatics and bioscience research infrastructure.

This open genome release forms part of an ongoing collaboration between SCU and the Australian Tea Tree Industry Association