.htaccess: RewriteEngine not allowed here

I uploaded the .htaccess to the server and received an Error 500 (Internal Server Error).

And in the error log I had the following error:

.../.htaccess: RewriteEngine not allowed here

But mod_rewrite.so is enabled.

So, do I need to change

<Directory />
    Options FollowSymLinks
    AllowOverride None
</Directory>

to

<Directory />
    Options FollowSymLinks
    AllowOverride All
</Directory>

in the /etc/httpd/conf/httpd.conf file?

Or could it be something else? The .htaccess file should be okay, because it works perfectly fine on my localhost. I just don't want to screw anything up.

Here's part of my .htaccess file:

Options All -Indexes

Options +FollowSymLinks

RewriteEngine On