Swap two elements in jQuery

I'm trying to swap two elements using up and down arrows.

A JSFiddle solution would be great!

My HTML:

<div class="item">
    <div class="content">Some text</div>
    <div class="move">
        <div class="move-down">down</div>
    </div>
</div>
<div class="item">
    <div class="content">Some other text</div>
    <div class="move">
        <div class="move-up">up</div>
        <div class="move-down">down</div>
    </div>
</div>
<div class="item">
    <div class="content">Some text</div>
    <div class="move">
        <div class="move-up">up</div>
        <div class="move-down">down</div>
    </div>
</div>
<div class="item">
    <div class="content">Some other text</div>
    <div class="move">
        <div class="move-up">up</div>
    </div>
</div>

My last try was:

// el is the clicked one.
jQuery('.move').children().click(function(el) {
    if (jQuery(el).hasClass('move-down') === true) {
        el = jQuery(el).parent().parent();
        el.prependTo(el.after(el));
    } else {
        el = jQuery(el).parent().parent();
        el.appendTo(el.before(el));
    }
});

I've tried a lot of different ways to change items. I've tried with replaceWith(), before(), and after() but nothing worked.

NOTICE: I've already written a function which displays the correct up / down DIVs. So the first and last one can only moved in one direction. That's already solved. I also can't use any kind of existing jQuery plugins.