The Supreme Court had 6 judges when it started up in 1789, and the country had a population of around three million people. Today, if you had 600 judges, you’d have the same proportion of the population. I know it isn’t that simple, but it does put the single-digit number into perspective.
edit: number tyop.