iOS 5 UINavigationBar removing subviews ( image subview) removes navigation bar

In my application that works fine on iOS 4 navigationBar dissappeared starting iOS 5. Here is what I figured caused issue: I was removing subviews in RootViewController viewWillAppear method:

for(UIView* view in self.navigationController.navigationBar.subviews)
    {
        if ([view isKindOfClass:[UILabel class]]) 
        {
            [view removeFromSuperview];
        }
        if([view isKindOfClass:[UIImageView class]])
        {
            [view removeFromSuperview];
        }
    }

I was doing this because Second view controller that I push onto navigation controller add's image and label to navigation bar which I have to remove when view is popped. In iOS 5 the above code removes navigationBar. How to fix this or right way of doing it to support both iOS4 and iOS 5 ?