As others have said, very interesting and thought provoking question! I would say a single herd of 4 is my maximum, one boar and three sows. I just love watching the interaction you get with a herd. If you only have 3, I feel like one pig ends up a loner, hence the 4 where everypig should have a least 1 close friend.
At the moment I have 11 pigs, in 4 groups -- 2 boar/sow pairs, a boar and two sows, and a boar and three sows. I never planned to have this many, originally I only planned to have a boar pair. When they fell out and started pining for in-cage company, I had them both neutered and planned to get them a girl each, for my new "hard maximum" of four pigs. I thought this would allow for a sufficiently high quality of life for both pairs, while still being manageable in terms of time needed to look after them.
A long story later, we have our eleven. I love them all dearly, but as others have said, things aren't how I'd like them to be. Not enough floortime is definitely my biggest issue. It's not so bad when you consider one of the pairs, and the trio, have 2x7 C&C equivalent cages, but for the quad who have the same-sized cage and the pair who have a 2x4 C&C equivalent... Not enough space! If I had one group, a neutered boar and 10 sows, I really think everyone's quality of life would be better, as I'd just have one massive pen.
When even one pig gets ill, it really messes up with my schedule, especially when syringe feeding is involved (around 1.5 hours per day for a REALLY speedy pig, and up to 4.5 hours for a very poorly one). It takes 40 mins a night to spot clean all our pigs, and 2 hours at the weekend to change over the cages. Add on 3 hours once a month for "piggy salon" with thorough health checks, nail clips, and hair cuts. With my mental health, it's really got to the point some days where all I can manage is spot cleaning the pigs. Things would be more manageable if I were able to be a stay-at-home-wife, but myself and partner both work full time. Trying to get 3 loads of pig laundry washed and dried is a feat, as well as at least 4 human loads!
Vet costs are astronomical. We are so lucky we can afford to pay for whatever is needed, but having both Rolo and Flash be very unwell indeed within months of each other really hit our bank account (we're at about £3k this financial year so far on vet bills alone). We also have a number of pigs with complex pre-existing health problems who take up a lot of time and money -- I would never ever begrudge them this, but it does get exhausting.
There are definitely days where I daydream about having fewer pigs and having more time to devote to each individual. I'm being very careful with bereaved pigs, making sure that their partners can slot into the other groups, so I don't fall into the cycle needing to get more pigs. As others have said, I think it's very dependent on the person. It's also scarily easy to exponentially increase your number of pigs in a short time; December 2016 I had 2, March 2017 I had 9 (three due to an unexpected pregnancy from a rescue pig)!
As much as I have highlighted the "bad" points, there are SO MANY good points about owning pigs, too many to mention. Their little faces peering up at me in the morning for veg; the wheeking when they recognise my footstep; watching them snuggle up together; hearing them pootle about; watching them zoom when you freshen their hay... I just wish I had more time to enjoy these parts!