I use pellets (grain free) but very sparingly.
I did try stopping all together for a while but found alll the piggies lost a little weight - nothing alarming, but I feel that as they get older the extra 'easy' calories are not a bad thing.
But it's personal choice really - I know it is certainly possible for them to do well without pellets.
I also just worry a little about the vitamin content of the fresh they get, and feel with pellets it is more certain that they are getting the essential vitamins and minerals.