I agree with another food bowl, mine have the ones that clip on to the side of the run or cage and i just fill it from the top, its got tiny holes in the bottom so all the dust falls through, i brought them on ebay but have seen them in shops, they are called gravity bin feeders by super pet, or if not a heavy ceramic one might help as already suggested,
as for the rest of the mess thats what animals do best to be honest mine like to burrow under their fresh hay and pull the newspaper up and make it messy, seems tidy is'nt good enough, luckly mine are all in sheds so the mess does'nt matter and gives me an excuse to go out to them,
well maybe you could just find him something really interesting to play with - the toilet moving reminds me of ours who make a sport out of moving their houses around. Kind of like a turtle, all of a sudden you see the whole house change direction...
since we have drainpipe-bits in the cage they leave the big houses alone and only move and play with the pipes.
My rabbit hated those plastic corner things... he will neatly make into a single corner of his cage, but not if this THING is in there.
Maybe your piggie is annoyed by it and just wants it out of the way? If your piggies are trained to go in this corner of the cage, (which I find incredible, mine just pee and poo wherever they lie and walk) then its not going to be that much more work to just spotclean it with a little shovel/dustpan. and he wont throw dirty bedding around all the time.
I have big heavy ceramic bowls but they still get food over the floor. i have one neat pig who poops in one corner but the others do it where they stand. They all eat news paper and generally trash the joint! I don't know that you can stop them but you may find cleaning up the poops and bits of chucked out food mix everyday makes it look nicer and is kind to their feet. You can but a bit of extra bedding in when you do this and a fresh layer of hay and the cage doesn't look such a tip anymore!