Alex Rivera | Logout

Nokogiri, open-uri, and Unicode Characters

Asked 2010-04-03T19:28:43.603
28

I'm using Nokogiri and open-uri to grab the contents of the title tag on a webpage, but am having trouble with accented characters. What's the best way to deal with these? Here's what I'm doing:

require 'open-uri'
require 'nokogiri'

doc = Nokogiri::HTML(open(link))
title = doc.at_css("title")

At this point, the title looks like this:

Rag\303\271

Instead of:

Ragù

How can I have nokogiri return the proper character (e.g. ù in this case)?

Here's an example URL:

http://www.epicurious.com/recipes/food/views/Tagliatelle-with-Duck-Ragu-242037

Edit
Report

1 Answer

37

I was having the same problem and the Iconv approach wasn't working. Nokogiri::HTML is an alias to Nokogiri::HTML.parse(thing, url, encoding, options).

So, you just need to do:

doc = Nokogiri::HTML(open(link).read, nil, 'utf-8')

and it'll convert the page encoding properly to utf-8. You'll see Ragù instead of Rag\303\271.

answered 2011-01-24T13:37:24.540

Your Answer