When you lost 2 followers in your graph and received 5 new followers by email, it means that 7 people unfollowed you. Making your followers count decreasing by 2. To see who unfollowed you, I recommend useqwitter. http://useqwitter.com/
Our followers count isn't live updated to Twitter, we update your stats daily. If you believe your Twitter stats are important to you and you want us to update hourly, then you can become a premium user. http://twittercounter.com/pages/premium
I hopes this answers your question, please let me know if there is anything else I can help you with.