Smooth migration for BioCommons’ Apollo service: AARNet and Globus support critical data transfer

This AARNet news story is republished with permission

As part of planned infrastructure changes at the Pawsey Supercomputing Research Centre’s Nimbus cloud, Australian BioCommons began preparing to move key components of its Australian Apollo Service to new infrastructure. A central part of that transition involved securely transferring over 15 terabytes of user data — ensuring research continuity for life science teams around the country.

To support the process, BioCommons turned to AARNet and Globus for a robust, high-performance solution.

Powering digital biology

Australian BioCommons is a national research infrastructure initiative accelerating life sciences research by providing digital platforms and services for data-intensive biology. One of its flagship offerings is the Australian Apollo Service — a hosted virtual environment that enables researchers to launch, manage and run complex bioinformatics workflows on demand.

The Pawsey Supercomputing Research Centre, a Tier 1 government-funded national facility, hosted the Apollo service from 2020, providing access to genome curation and visualisation software to hundreds of researchers across Australia.

As the underlying cloud infrastructure at Pawsey (Nimbus) was being phased out and Apollo’s environments could be redeployed elsewhere, the associated user data, including datasets vital for current and future analysis, needed to be transferred efficiently and with minimal disruption to services.

Finding the right approach for large-scale transfers

Initial attempts to move the data using traditional tools quickly highlighted performance limitations. Transferring even a single terabyte took multiple days, and with the additional goal of preserving metadata using tarball archives, BioCommons needed a more scalable and dependable approach.

To meet these needs, the team explored using Globus, a research-grade data transfer service designed for large-scale scientific workflows. Critically, Globus operates over the AARNet research and education network, which provides the high-speed connectivity required for rapid, reliable transfers across sites.

“What stood out with Globus was how straightforward it was to get going,” said Justin Lee, Platform Developer and System Administrator at Australian BioCommons. “The documentation AARNet provided made it easy to deploy, and once set up, it just worked. We didn’t need to manage every detail as it handled the complexity for us.”

Setting up endpoints and running the transfer

AARNet provided setup guides and workshop resources to assist with deployment. Working from these materials, Justin was able to spin up the required Globus endpoints in less than a day — one at Pawsey, where Apollo had been running, and the other at the new AARNet-hosted Nectar Research Cloud node, where Apollo would be hosted going forward.

After a successful test transfer, the full migration began the week ahead of the Easter holiday period. With Globus managing the transfer, the system automatically handled retries, integrity checks, and restarts.

By the time teams returned after Easter, the entire 15 TB of data had been moved, smoothly, securely, and without interruption.

Supporting research continuity through reliable infrastructure

For researchers using the Australian Apollo Service, no changes were needed to how they interacted with their environments. Once the transition was complete, data access resumed smoothly and workflows continued as before.

“From our perspective, it was great to see how quickly the BioCommons team could get up and running with Globus using the self-service resources,” said Greg D'Arcy, Digital Research Product Manager at AARNet. “It shows how institutions can take ownership of complex transfers without needing deep expertise, especially when time is limited.”

“It all worked as expected,” added Justin. “Having reliable tools and a clear setup process made the whole migration straightforward.”

Building resilience for life sciences research

This successful migration showcases the value of collaborative partnerships and purpose-built infrastructure in enabling modern, data-driven research. By leveraging Globus and the AARNet research network, BioCommons ensured that life science researchers could continue their work without missing a beat, even during a significant infrastructure shift.

The Australian Apollo Service forms part of the national Australian BioCommons infrastructure. BioCommons partners with QCIF to manage the Australian Apollo Service, which is underpinned by computational resources provided by AARNet’s ARDC Nectar Research Cloud node. These efforts are supported by funding from Bioplatforms Australia and the Queensland Government RICF. Bioplatforms is enabled by NCRIS.

An Australian community for computational structural biology

The active new community is receiving support from a range of partners and advocates, including L-R Johan Gustafsson (BioCommons), Steven Manos (BioCommons), Kate Michie (UNSW) and Andrew Gilbert (Bioplatforms Australia)

The explosion of possibilities presented by deep learning approaches in structural biology research has created many new opportunities and challenges. A passionate group of structural biologists has formed the Australian Structural Biology Computing Community, to approach this new era as part of a community that shares computational knowledge, methods, and resources. 

This community-driven approach brings together a diverse group of people, with initial contributions forming around leads from the Structural Biology Facility at UNSW, and an academic panel of experts from Monash University, Walter and Eliza Hall Institute of Medical Research (WEHI), University of Western Australia (UWA), Australian National University (ANU), Bio21 Institute of Molecular Science and Biotechnology (Bio21), University of Melbourne, La Trobe University, University of Queensland (UQ) - IMB, University of Sydney, Griffith University, Swinburne University of Technology, CSIRO, and the University of Adelaide. Anyone involved in structural biology in Australia is invited to join and there are lots of different ways to get involved.

The Community for Structural Biology Computing in Australia webpage is a useful new resource for all users of computing for structural biology research in Australia. The page is constantly evolving and expanding, and it currently focuses on the use of deep learning methods in Structural Biology. It includes practical guides on topics like “Best practices for presenting and sharing AlphaFold models in a paper” as well as news items and announcements for relevant courses and meetings. 

Australian BioCommons supports the community by hosting quarterly online meetings that aim to tease out how computational structural biologists’ challenges might be addressed with community-scale responses and national research infrastructure solutions. If you join the mailing list via the community webpage, you will receive updates and invitations to community meetings and the discussions in Slack.

BioCommons began providing broad, fully subsidised, access to structural prediction in 2022 by making AlphaFold2 available within its Galaxy Australia service. The Australian AlphaFold2 Service provides both an easy-to-use interface and dedicated GPUs to Australian researchers. When BioCommons hosted the international 2023 Galaxy Community Conference, the keynote speech by Chief Scientist of the Structural Biology Factility at UNSW, Kate Michie, generated much excitement around forming an Australian community of practice for computational structural biology as an avenue for collectively addressing the challenges presented by deep learning in structural biology. 

BioCommons has supported key research stakeholders to refine the new community’s purpose, began running quarterly community meetings, and helped to establish the shared community spaces like the Australian Structural Biology Computing website and GitHub. As well as facilitating consultations with infrastructure partners and the broader computational infrastructure community, a group of national panel of experts has been identified. 

This community collaborates with their peers to:

• Collectively create and maintain community forums and centralised collaboration platforms to support collaboration and knowledge sharing (i.e. methods and documentation);

• Foster collaboration between structural biologists, computer scientists, and data scientists, thereby creating interdisciplinary teams to help tackle complex challenges, validate results and ensure robust applications of deep learning methods;

• Lead the review, prioritisation, testing, optimisation, and sharing of deep learning codes, software and approaches that are of broad relevance and interest to the Australian research community;

• Develop quality assessment tools to help evaluate the quality of calculated structures, and help guide researchers towards reliable predictions; and,

• Address the ethical implications of AI-driven structural predictions, as well as discuss transparency, bias and interpretability to ensure responsible use of these technologies.

The Australian Structural Biology Community is poised to tackle a set of pilot activities aimed at fast tracking a national response to the challenges facing computational approaches in structural biology. A much anticipated future output is an infrastructure roadmap document that will formalise and describe the high level requirements of the community. This collaborative effort between the new Australian Structural Biology Community, Australian BioCommons, and BioCommons infrastructure partners will support the Australian Structural Biology community as new needs arise relating to bioinformatics tools, software, infrastructure or training.

Keep in touch by subscribing for updates at the Community for Structural Biology Computing in Australia webpage. 

Join a Metagenome Assembled Genomes hackathon

The Optimising Metagenome Assembled Genomes building workflows hackathon is taking place in October and we want to know if you’d like to join!

Multiple ELIXIR nodes and the international Galaxy community are coming together in Europe to participate, and BioCommons is planning to offer a complementary event in Australia.

What is this?

Hackathons involve collaborative group work with people outside your normal network, solving problems around a shared topic of interest within a limited period of time. The aims of the event are:

• Enhancing FAIR MAGs building Workflows

• Developing user-friendly training materials

• Advancing workflow evaluation methods (using CAMI infrastructure & real data)

• Building intelligent computational resource estimation tools

The event takes place 6-9 Oct 2025. It will run in person in Freiburg, Germany and online.  If there is enough interest, BioCommons will organise an in-person event in Australia to overlap with the European event.

What’s in it for me?

• Collaborative group work: You'll work with people outside your usual network.

• Expert connections: You'll connect with a national and international group of experts and enthusiasts.

• Workflow enhancement: You'll contribute to enhancing MAG workflows.

• Training material development: You'll participate in developing related training materials.

• Skill development: While specific skills are not required, you can enhance your knowledge of microbiome data analysis or MAGs.

• International participation: Opportunity to participate in an international hackathon with colleagues in Europe.

Who can join in?

It is open to anyone who wants to build MAGs or anyone with a general interest in MAGs, microbiome analysis and the associated training to run MAG workflows. To join in the hackathon, some knowledge of microbiome data analysis or MAGs is preferred. There is no cost to join.

How would this work?

Australian BioCommons regularly run satellite events that facilitate participation in global hackathons. If there is interest, we would ideally work together in person in Australia. Given European business hours coincide with Australian evenings, we would connect with the European hackathon team during some overlapping hours in the afternoon and work asynchronously for the other hours.

Are you interested in joining in?

If you think that you might be interested to join in this hackathon event, please get in touch with Tiff Nelson by 20th June 2025.

Repurposed hardware boosts national capacity and powers innovation

This story is co-published with QCIF Ltd

After successfully completing a previous project, QCIF Ltd made available high-performance hardware to the Australian BioCommons, giving the hardware a second life in enabling research and uplifting national capacity for the benefit of the scientific community. 

Well suited for running AlphaFold 2 jobs, the five General-Purpose Graphics Processing Units (GPGPUs) are now being used to enhance the national compute network behind the Galaxy Australia service.

The impact of this repurposing goes beyond infrastructure improvements. It has significantly expanded Galaxy Australia's capacity to support research and innovation by enabling the use of other GPU-enabled tools that offer major benefits to the scientific community. GPU processing can provide massive improvements in computational efficiency, decreasing processing times to less than 5% of conventional equivalents.

Dr Cameron Hyde, a bioinformatician at QCIF who supports the development of national software platforms like Australian BioCommons' Galaxy and Apollo services, co-authored the original AlphaFold 2 wrapper that enabled the tool to run within Galaxy Australia, ensuring both a friendly user-interface as well as instant access to the GPU clusters required to power the tool. He shared his enthusiasm for the new possibilities unlocked by the repurposed hardware which was originally part of an investment made in 2021 by the Australian Research Data Commons (ARDC) to support national platform projects and now directly enhances the bioinformatics services he helps deliver to Australian researchers. “Now that we have five GPU nodes of our own, we have room to experiment and explore new GPU-enabled tools. This gives us room to innovate beyond AlphaFold and accelerate scientific discovery in other research domains.”

For example, Galaxy Australia’s lead Bioinformatician Michael Thang has been using the hardware to explore running Nanopore’s “Dorado” on Galaxy Australia. Dorado is a high-performance basecaller for Oxford Nanopore Technology sequencing data. This innovation would enable researchers to conduct their entire analysis, from raw sequencing data through to assembled genome, all within the Galaxy Australia service.

Collaboration driving innovation

Developed by Google DeepMind, AlphaFold is an AI system that predicts a protein’s 3D structure from its amino acid sequence with accuracy comparable to experimental methods. In 2020, Australian BioCommons identified an opportunity to democratise access to this powerful tool by making AlphaFold 2 available through Galaxy Australia. This gave Australian researchers much greater accessibility to AlphaFold 2, allowing life scientists to easily visualise proteins in a manner inaccessible to all but dedicated structural biology researchers. This advance has supported research into protein-protein interactions, activation and inhibition mechanisms, and drug design.

By 2025, use of AlphaFold 2 has surged, evolving from an analytical tool for individual proteins into a routine screening tool for studying protein-protein interactions. To support this shift, Dr Hyde collaborated closely with Australian Structural Biology Computing Community to develop extensions to the AlphaFold Galaxy tool, including new output formats, input parameters, and an option to re-use intermediate files for improved efficiency.

Supported by the Australian BioCommons, AARNnet, QCIF Ltd, and The University of Melbourne, the optimised system now provides fully subsidised access for all Australian researchers via the Australian Alphafold Service. We extend our sincere thanks to the Australian Research Data Commons (ARDC) for providing the hardware to QCIF Ltd and enabling its reuse by Australian BioCommons.

Nextflow workshop combines benefits of hands-on training and community building

BioCommons’ Nextflow for the life sciences workshop heralds a return of our dispersed model of training that combines the benefits of in person and online events to enable access to experts and foster local connections that are essential for continued learning. First pioneered in 2019, this model has been successful in ensuring scalable and more equitable delivery of short-course bioinformatics training across Australia and has been adapted internationally.

Nextflow for life sciences workshop participants will join in person satellite sites at host universities and research institutes across Australia where they will connect with peers and be supported by experienced local facilitators as they put their new Nextflow skills into action. Each of these sites will connect online with Nextflow experts and lead trainers Fred Jaya and Dr Michael Geaghan at the University of Sydney’s, Sydney Informatics Hub who will introduce key concepts and demonstrate how to use fundamental Nextflow elements to develop, execute, and debug a scalable multi-step life science workflow.

Find out more and apply for the workshop. Applications close 27 June 2025.

This workshop is made possible by an exceptional network of facilitators and trainers from the national Bioinformatics Training Cooperative.

Microbiology Lab empowers microbial data analysis with integrated tools, workflows and compute

Galaxy Australia’s Microbiology Lab is now providing researchers with rapid access to popular tools, workflows and compute for analysis and visualisation of microbiomes and omics data from microbial isolates.

Development of this curated view of Galaxy Australia was driven by the international microbial research community who identified the most commonly used tools in the field through meticulous research, surveys and community consultations.

The Lab integrates more than 220 tools and 65 workflows with step-by-step tutorials and structured learning paths for a suite of different analyses. Microbiology Lab pairs perfectly with the computing power of Galaxy Australia, which is underpinned by computational resources provided by AARNet, ARDC Nectar Research Cloud, the University of Melbourne, QCIF, Pawsey Supercomputing Research Centre, National Computational Infrastructure, and Microsoft Azure. From raw data, to differential analysis, visualisation and assembly, Microbiology Lab makes it easy to get started with reproducible analysis of microbial data.

Galaxy Australia’s Microbiology Lab is the latest release in a series of Labs supporting research domains. It represents an important step forward in Australian BioCommons activities to support Australian microbial research. The Microbiome Analysis Infrastructure Roadmap for Australia identified a need to implement a shared platform that eases access to preferred tools and workflows for analysis of microbial data with sufficient computational power. This fully-subsidised resource is expected to improve efficiency for researchers.

If you are an Australian researcher with an interest in microbial isolates and microbiomes, be sure to take a tour of the Galaxy Australia Microbiology Lab and try it out now! 

WorkflowHub: a registry for computational workflows

A new paper in Nature Scientific Data describes a public and inclusive registry dedicated specifically to the sharing of computational workflows: WorkflowHub.

Australian BioCommons’ Research Community Engagement Lead (Proteins / Metabolites / Workflows), Dr Johan Gustafsson, is the lead author of a stellar group of international experts. They have worked hard on their shared passion for a unified registry for all computational workflows that links to community repositories, and supports both the workflow lifecycle and making workflows findable, accessible, interoperable, and reusable (FAIR).

The WorkflowHub registry is designed to allow any scientist, regardless of expertise level, to contribute and share computational workflows. It indexes workflows from any scientific domain, in any format, in any workflow language, regardless of whether it uses a workflow management system and supports users to increase the FAIRness of their workflows.

By interoperating with diverse platforms, services, and external registries, WorkflowHub adds value by supporting workflow sharing, explicitly assigning credit, enhancing FAIRness, and promoting workflows as scholarly artefacts. The registry has a global reach, with hundreds of research organisations involved, and more than 800 workflows registered.

The paper describes how WorkflowHub’s structure, design, standards, community engagement, and continued evolution support:

1) collaboration, sharing and credit for workflow developers, projects, and consortia;

2) integration with added-value services, platforms, and capabilities that support the workflow life cycle (i.e. creation, version control, execution, maintenance, reuse and citation); and

3) wizards and inbuilt features that ease the process of sharing workflows alongside the constellation of associated digital artefacts that give a workflow its scientific context.

Read the paper in full Gustafsson, O.J.R., Wilkinson, S.R., Bacall, F. et al. WorkflowHub: a registry for computational workflows. Sci Data 12, 837 (2025). https://doi.org/10.1038/s41597-025-04786-3

Join our co-working session on 24 June to try out WorkflowHub for yourself.

New seminar series ignites curiosity and builds practical AI skills in the life sciences

AI is reshaping life sciences by enabling researchers to analyse complex datasets, automate workflows, and gain deeper insights into biological processes. Australian BioCommons is supporting the community to adopt these technologies through a series of training events that explore what’s possible and build skills in using AI effectively.

We’ve invited a diverse range of your peers from across academia and industry to share their experiences of using AI in the life sciences. Between June and September you’ll hear how AI is being used to push boundaries, solve problems, and reimagine the way we do science. From multi-omics analysis to drug discovery, structural biology and the ethics of AI in science, they will explore diverse applications across the life sciences, focusing on the stories, insights, and experiences behind the research.

Register for the webinars

In August, our online workshop Machine learning in the life sciences will provide a hands-on opportunity to compare and contrast commonly used algorithms for constructing predictive models, explore some of their trade-offs and identify types of scenarios in which they can be applied.

In case you missed it, Australian BioCommons’ AI Technical Lead, Dr Benjamin Goudey, recently broke down AI concepts, clarified key terminology, and showcased real-world examples in this recorded webinar: Deciphering AI for the Life Sciences. 

AI is a fast evolving field so rest assured that further events are in the pipeline. 

Browse all upcoming training and events 

Building bridges for better data sharing: ENA experts empower Australian researchers in data submission

Participants of the in-person roundtable came from around Australia to meet the ENA team

For two weeks in March and April 2025, Australia’s life sciences community had a unique opportunity to engage directly with the European Nucleotide Archive (ENA) team. In a first-of-its-kind initiative, Dr Joana Pauperio (Biodiversity Curator, European Nucleotide Archive, EMBL’s European Bioinformatics Institute) and Maira Ihsan (User Support Bioinformatician, European Nucleotide Archive, EMBL’s European Bioinformatics Institute) visited Australia to deliver an intensive series of seven workshops and four roundtable discussions, aiming to enhance Australian researchers’ skills in submitting and retrieving genomic, metagenomic, and environmental DNA (eDNA) data to/from international repositories.

Organised by Australian BioCommons, the visit built technical capacity and opened a direct dialogue between the ENA and the Australian research community about the future of data submission, retrieval, and brokering. High-quality data submission to international archives like the ENA ensures that Australian-generated genomic and environmental data can contribute to global research efforts. Yet, challenges in submission processes, metadata preparation, and understanding of repository workflows can act as barriers. Bringing ENA experts together in person allowed Australian researchers to receive tailored, hands-on guidance, overcoming time zone challenges and helping the ENA team witness firsthand the hurdles local researchers face.

Workshops: Hands-on learning and capacity building

Across six data submission workshops, participants learned various data submission pathways (e.g., via Webin-CLI, programmatic, and command line) to submit:

• Raw reads, genome assemblies, and annotations

• Metagenome-Assembled Genomes (MAGs)

• Environmental DNA (eDNA) data

A data retrieval workshop provided an opportunity for participants to practice retrieving different data types from the ENA using various tools and protocols.

Feedback was welcome at all times by providing a living document for queries that were addressed during and after the workshop, and breakout rooms for 1:1 discussions were available.

Roundtables: Listening to the community

One in-person and three online roundtable discussions were also hosted to facilitate direct communication between ENA and Australian researchers.

In-person Roundtable

This meeting between invited members of Bioplatforms Australia, Bioplatforms Australia Data Portal, Australian Reference Genome Atlas (ARGA), the Australian Tree of Life project, and the ENA teams focused on information exchange and potential collaboration in the global biodata landscape. Key topics included data brokering to ENA, species taxonomy, and the possibility of establishing an Australian node within the International Nucleotide Sequence Database Collaboration (INSDC). The immediate next step identified was to further explore data brokering. The roundtable provided a valuable forum for discussing opportunities and challenges in collaborating with the ENA and enhancing Australia's contribution to international data repositories.

Genomics Roundtable

The meeting facilitated discussions on topics including Genome assembly and annotation efforts at scale in Australia, ENA's role as a global repository and challenges in annotation submissions to INSDC. It aimed to improve understanding of data publication options and ENA submission processes.

MAGs Roundtable

The meeting facilitated discussions on topics including the use of MAGs in Australia, the role of ENA+MGnify as a global repository, challenges in mass submission of MAGs, issues with submitting MAG data for organisms not represented in the NCBI Taxonomy, and suggestions for improvement.

eDNA Roundtable

The meeting facilitated discussions on topics including eDNA use across various sectors, Australian eDNA reference library initiatives like the National Biodiversity DNA Library (NBDL), making eDNA data FAIR (Findable, Accessible, Interoperable and Reusable) and the ENA as a global repository for eDNA data, data interoperability between resources, and data sharing with third-party platforms like GBIF.

Looking ahead

The momentum generated by the workshops and roundtables will continue through:

• The creation of self-paced training materials: by converting the workshop content and hosting it on the EMBL-EBI training website to ensure researchers have access to training when they need it

• Efforts to explore an Australian data brokering pathway as part of the Australian Tree of Life (AToL) project

• Strengthened connections between Australian researchers and INSDC repositories

By bridging expertise across continents, the collaboration between ENA and the Australian life sciences community is helping ensure that Australian research continues to have a strong, visible impact on the global stage.

Australia's inaugural nf-core Hackathon Outpost boosts local Nextflow capability

The first-ever Australian satellite site of the international nf-core Hackathon has just taken place in Sydney. Seventeen participants from across the country came together to collaborate on cutting-edge Nextflow projects. This marked a significant step in strengthening Australia’s Nextflow and nf-core community, fostering new collaborations between researchers, bioinformaticians, and Nextflow enthusiasts.

International registrations for the 2025 nf-core Hackathon almost reached 1000, with about half opting to join remotely and the remainder spread across 44 sites globally. The organisers were thrilled to finally have an Australian site. Thanks to our time zone, we were among the first to kick off each day of asynchronous work, along with colleagues across the ditch in Aotearoa New Zealand. 

Australian BioCommons arranged for key participants including multiple Nextflow Ambassadors from multiple cities to join our local hosts, Sydney Informatics Hub at the University of Sydney. Participants represented Pawsey Supercomputing Research Centre, National Computational Infrastructure, Peter MacCallum Cancer Centre, QCIF, Queensland University of Technology, Sydney Informatics Hub, University of NSW, and Australian BioCommons. 

The group spent three days at Moore College, a fantastic venue with an open workspace and an expansive terrace with city views - providing an ideal setting for focused work, brainstorming, and cross-team collaboration. While the hackathon was all about accelerating nf-core developments, it was also a valuable chance for collaboration outside of participants’ regular networks. Seqera sponsored our working lunches, but the Newtown cafe and restaurant scene ensured participants stayed well-fueled throughout the whole hackathon!

Over the course of three productive days the group divided up to tackle key projects in Nextflow development, including:

• Working towards a new release of the nf-core ProteinFold pipeline; 

• Developing a Nextflow for HPC training for Australia’s national supercomputers, NCI’s Gadi HPC and Pawsey’s Setonix HPC;

• A nf-test working group to write and apply unit tests to existing institutional pipelines.

As well as progressing some locally important projects, the gathering was a great opportunity to discuss Nextflow training collaborations and how we can work together to ensure more Australians can access the Nextflow skills and resources they need. Keep an eye on our training events listings to see the outcome of these new national relationships.

We are so grateful to the Seqera team and the nf-core community for their enthusiasm and efforts to welcome the Australian satellite site of the hackathon. And massive thanks to the participants who contributed their time and energy to make this event both productive and fun! With the success of this first Australian outpost, we’re already looking forward to next year’s nf-core Hackathon.