G4L4XY Or you could have a different type of small faction. Maybe, everything about them makes them small except for the fact that they have an immense amount of political power and/or are very neutral. This would make attacking them raise a lot of enemies, while still allowing the existence of small factions. It may also have quite an amazing technological advancement, but have signed a contract, meaning that attacking them will create similar results.
As well, small factions are likely to be either of those anyways, as it cannot fight face to face and needs powers to support it.