Symbols are like strings but they are immutable - they can't be modified.
They are only put into memory once, making them very efficient to use for things like keys in hashes but they stay in memory until the program exits. This makes them a memory hog if you misuse them.
If you dynamically create lots of symbols, you are allocating a lot of memory that can't be freed until your program ends (edit: this is no longer the case since Ruby 2.2). You should only dynamically create symbols (using string.to_sym) if you know you will:
As I said earlier, they are useful for things like hashes - where you care more about the identity of the variable than its value. Symbols, when correctly used, are a readable and efficient way to pass around identity.
I will explain what I mean about the immutability of symbols RE your comment.
Strings are like arrays; they can be modified in place:
12:17:44 ~$ irb
irb(main):001:0> string = "Hello World!"
=> "Hello World!"
irb(main):002:0> string[5] = 'z'
=> "z"
irb(main):003:0> string
=> "HellozWorld!"
irb(main):004:0>
Symbols are more like numbers; they can't be edited in place:
irb(main):011:0> symbol = :Hello_World
=> :Hello_World
irb(main):012:0> symbol[5] = 'z'
NoMethodError: undefined method `[]=' for :Hello_World:Symbol
from (irb):12
from :0
It can be particularly bad to create symbols from user input without validating the input against some kind of a white-list (for example, for query string parameters in RoR). If user input is converted to symbols without validation, a malicious user can cause your program to consume large amounts of memory that will never be garbage collected.
Bad (a symbol is created regardless of user input):
name = params[:name].to_sym
Good (a symbol is only created if the user input is allowed):
whitelist = ['allowed_value', 'another_allowed_value']
raise ArgumentError unless whitelist.include?(params[:name])
name = params[:name].to_sym