Aside from the question of whether it’s a good idea to bring ‘unbounded’ into the picture(both because the economy consists only of activity within a few light-minutes of earth, judged according to the standards of entities who need to eat today and will be dead within a few decades, maybe a century for the young ones, so ‘the long term’ is actually under a time limit; and just because “very large but finite” is vastly more mathematically tractable than anything non-finite); I’d consider a focus on marginal utility when trying to decide what an ‘oligarchy’ looks like.
Unless you are a profoundly weird entity that uses money purely for keeping score; its marginal utility is going to decline as you obtain more of it(probably not perfectly evenly, I’d expect bumps as you reach various objectives with fixed lower bounds on cost, housing in a desirable location, your first airplane, whatever; but diminishing marginal utility). This allows the amount of utility in an economy to decline, even if all interactions are at least zero sum or better: all that’s required is distribution rules result in money flowing to people who derive increasingly little utility from it fast enough to get ahead of gains from trade.
It’s an incomplete picture; but I think it captures one element of what, intuitively, separates an unequal but prosperous society from an ‘oligarchy’:
If the flow of resources remains on the side of continuing improvements in utility concerns about inequality and reports of oligarchs are likely to remain substantially theoretical because the only way for utility to increase is if enough of the gains accrue to people with relatively high marginal utility for money and away from people whose marginal utility for money closely approaches zero.
If this threshold is crossed, and wealth increasingly ends up in places where its marginal utility is negligible at the expense of people whose would derive much more utility from it, the question of “If we’re so rich; why are we so poor?” starts to take on a burning urgency; and the economic role of people with vastly more money stops looking largely theoretical and starts looking like an ugly fight over a shrinking pie.
Even if we assume that the pie, in absolute terms, must necessarily be expanding; it’s readily possible for the utility derived from it to be shrinking; and that’s ultimately what people notice.
