May have been a Bug
I don`t think it was because of NPCs summoning followers, because there were about 4-5 gates that spawned enemies. I tried to play again today and it worked just fine. The enemy waves were acceptable again, nothing to worry about. My guess is that it might have been similar to the bug where an NPC (enemy) taunts me every second and won`t stop. Do the waves have a cooldown and maximum number of gates that can be summoned at a time?