BaseAdapter causing ListView to go out of order when scrolled

I'm having problems with some BaseAdapter code that I adapted from a book. I've been using variations of this code all over the place in my application, but only just realized when scrolling a long list the items in the ListView become jumbled and not all of the elements are displayed.

It's very hard to describe the exact behavior, but it's easy to see if you take a sorted list of 50 items and start scrolling up and down.

class ContactAdapter extends BaseAdapter {

    ArrayList<Contact> mContacts;

    public ContactAdapter(ArrayList<Contact> contacts) {
        mContacts = contacts;
    }

    @Override
    public int getCount() {
        return mContacts.size();
    }

    @Override
    public Object getItem(int position) {
        return mContacts.get(position);
    }

    @Override
    public long getItemId(int position) {
        return position;
    }

    @Override
    public View getView(int position, View convertView, ViewGroup parent) {
        View view;
        if(convertView == null){
            LayoutInflater li = getLayoutInflater();
            view = li.inflate(R.layout.groups_item, null);
            TextView label = (TextView)view.findViewById(R.id.groups_item_title);
            label.setText(mContacts.get(position).getName());
            label = (TextView)view.findViewById(R.id.groups_item_subtitle);
            label.setText(mContacts.get(position).getNumber());
        }
        else
        {
            view = convertView;
        }
        return view;
    }

}