When you say that the movie was rather good, it implies that it was pretty good. It was much better than you had expected. So in your opinion, people should go and see it. You recommend it.
On the other hand, when you say that it was fairly good, what you are implying is that it was OK. There was nothing great about the film, if the friend wants to, he/she may go and see it, but you wouldn't recommend it.
Similarly, when you tell someone that his English is fairly good, you are not really complimenting him. All that you are saying is that his English is OK.
Here are a few more examples.
• Josh made a fairly/rather good presentation.
• During summer, it gets fairly/rather warm.
• Tilak's wife is fairly/rather good looking.
COURTESY : The Hindu (The National News-Paper) - India