Where did the [spoiler] go after Act 2??

Where did the Absolute's army go after Act 2? They march off to BG, I know I killed Ketheric and Gortash said something about how he'll defend against the army, but then you get to BG and there's no army, no siege, nothing. There's a lot of talk about how "people are scared" and what-not but you don't actually see an army anywhere and nobody wonders where the giant swarm of goblins and undead went. Even in the climax, it seems to be almost entirely an Illithid army with just a handful of random goblins and necromancers. Where were they all this time?