Vicki has given you some excellent links....
Your poor wee guy has had a lot to contend with and may take longer than the average piggy to settle. Some of the tips in the 'how to pick up my guinea pig' link will hopefully help him feel more secure when you do need to pick him up. It's possible that his ear infection/head tilt mean he either has sensory issues or gets more easily disorientated than a normal piggy (the inner ear is part of your balance mechanisms) which will make him more likely to panic when touched or handled.
It's also worth remembering that not all piggies like to be held or cuddled and sometimes it's just not their thing and interacting with them is more on their terms..... My Angus is 5 and very well handled but still isn't keen on sitting on my lap or being held, he'd rather sit next to me and come to me when he feels like it. Some of my other boys are very relaxed being handled.
For your chap it could well be a mix of things causing his fear but be patient and spend time with him anyway, just sit by his cage talking quietly to him so he gets used to your sound and smell. Tempt him to come up to you with his favourite veg and just take your time with him. Gaining the trust of a guinea pig who has had a tough start takes time, love and patience and needs to be at their pace, not ours.
Good luck though. He is a very lucky boy to have found someone who cares so much for him!
