scrollViewDidEndDecelerating being called for a simple touch

I'm implementing UIScrollViewDelegate and doing a lot of stuff with it. However, I just found an annoying issue.

I expect scrollViewDidEndDecelerating: to be called always after scrollViewWillBeginDecelerating:. However, if I simply touch my ScrollView (actually I'm touching a button inside the scrollView), scrollViewDidEndDecelerating: gets called and scrollViewWillBeginDecelerating: was not called.

So how can I avoid scrollViewDidEndDecelerating: being called when I simply press a button inside my UIScrollView?

Thank you!