KnowledgeHub
Questions
Tags
Users
Search
Alex Rivera
|
Logout
Edit Question
Title
Body
I need to store the content of a site that can be in any language. And I need to be able to search the content for a Unicode string. I have tried something like: import urllib2 req = urllib2.urlopen('http://lenta.ru') content = req.read() The content is a byte stream, so I can search it for a Unicode string. I need some way that when I do urlopen and then read to use the charset from the headers to decode the content and encode it into UTF-8.
Tags (comma-separated)
Save Edits
Cancel