DataTemplate for each DataType in a GridViewColumn CellTemplate

I have an ObservableCollection which contains view models of multiple types, and I would like to make a DataTemplate for each type within each of my GridViewColumn's CellTemplates. In this simple example I could create a base ViewModel, but I would like to be able to do this just from xaml. The xaml below shows what I am trying to do, where one of the DataTemplates would be used for each CellTemplate.

If there were a GridViewColumn.Resources I would define the DataTemplates there then use a a DataTemplate with ContentPresenter in the CellTemplate, but I obviously can't do that. I'm thinking I may need a TemplateSelector, but I'm not sure where to start.

<ListView ItemsSource={Binding GenericObservableCollection>
    <ListView.View>
        <GridView>
            <GridViewColumn Header="Type">
                <GridViewColumn.CellTemplate>
                    <DataTemplate DataType="{x:Type vm:ActionInputViewModel}">
                        <TextBlock Text="Input"/>
                    </DataTemplate>
                    <DataTemplate DataType="{x:Type vm:ActionOutputViewModel}">
                        <TextBlock Text="Output"/>
                    </DataTemplate>
                </GridViewColumn.CellTemplate>
            </GridViewColumn>
            <GridViewColumn Header="Value">
                <GridViewColumn.CellTemplate>
                    <DataTemplate DataType="{x:Type vm:ActionInputViewModel}">
                        <TextBlock Text="{Binding Property1}"/>
                    </DataTemplate>
                    <DataTemplate DataType="{x:Type vm:ActionOutputViewModel}">
                        <TextBlock Text="{Binding Property2}"/>
                    </DataTemplate>
                </GridViewColumn.CellTemplate>
            </GridViewColumn>
        </GridView>
    </ListView.Vie