Happened to me once this season. Wasn’t as big a swarm as you so I was able to open a portal and leave. When I restarted the encounter the invulnerable mobs were gone.

This happened to me once in a campaign ritual, had to restart the zone. It would suck to lose your trial of chaos progress tho