[R] Has anyone experimented with MHC on traditional autoencoders/convolutional architectures?

A researcher is battling a "super freaking huge" hyperspectral image dataset (50x512x1024 fp32) for a custom autoencoder baseline. Their current ResNeXt2-based architecture is so demanding that an NVIDIA A100 can only process a batch size of 2. They are considering replacing all residual connections with MHC (Multi-Head Context) and are actively seeking advice, as they've found no good literature on hyperspectral image autoencoders and are avoiding transformers for this foundational work.