It was memory/tension based? I thought it was simply leader = center, 2nd (i.e. second idol you chose) = to the left, 3rd = to the right, and that's how it always was, from week 1 to week 52.
I thought it was based on how much the idol likes you, but that just might be me.
I got Haruka down with less memories than Miki, but she stayed the leader. When I beat the game once, I had Haruka's trust at D-ish or so, and Miki at like... Z.
And The World is All One? They've finished recording it if they've got parts for everyone. They'll just turn off the voices of idols you're not producing.