Start with the side that has the fewest number of buttons on it.