R: How do I use coord_cartesian on facet_grid with free-ranging axis

Consider some facet_grid plot

mt <- ggplot(mtcars, aes(mpg, wt, colour = factor(cyl))) + geom_point() 
mt + facet_grid(vs ~ am, scales = "free") 

Imagine I just want to zoom on just the top row in the plots above to only show the y-axes values between 3 and 4. I could do this with coord_cartesian() if they weren't faceted or if I wanted to zoom on all plots, but don't have a good solution in this case. I suppose I could subset the data first, but that is taboo for good reason (e.g. would throw off any statistical layer, etc).

(Note that the question is related to this: R: {ggplot2}: How / Can I independently adjust the x-axis limits on a facet_grid plot? but the answer there will not work for this purpose.)