matplotlib backends - do I care?

>>> import matplotlib
>>> print matplotlib.rcsetup.all_backends
[u'GTK', u'GTKAgg', u'GTKCairo', u'MacOSX', u'Qt4Agg', u'Qt5Agg', u'TkAgg', u'WX', u'WXAgg', u'CocoaAgg', u'GTK3Cairo', u'GTK3Agg', u'WebAgg', u'nbAgg', u'agg', u'cairo', u'emf', u'gdk', u'pdf', u'pgf', u'ps', u'svg', u'template']

Look at all those backends!

Do I need to care which backend is in use? e.g. if I develop and test my stuff using only TkAgg backend, and someone else using my code might be using GTKAgg backend on their system, might my stuff break for them in a way that I won't have noticed - or are all backends required to more or less "work" the same way?