UNIX sort ignores whitespaces

Solved by:

export LC_ALL=C

From the sort() documentation:

WARNING: The locale specified by the environment affects sort order. Set LC_ALL=C to get the traditional sort order that uses native byte values.

(works for ASCII at least, no idea for UTF8)

Like mentioned before, LC_ALL=C sort does the trick. This is simply because different languages have different rules for sorting characters, which are often laid out by senior linguists instead of CS experts. And these rules, in the case of your locale, seem to say that spaces ought to be ignored in sorting.

By prefixing LC_ALL=C (or, when LC_ALL is unset, LC_COLLATE=C suffices), you explicitely declare language-agnostic sorting (and, with LC_ALL, number-formatting and stuff), which is what you want in this context. If you want to make this your default, export LC_COLLATE in your environment.

The default is chosen in this way to keep consistency with the "normal", real-world sorting schemes (like the white pages), which often ignored spaces.

Actually for me

$ cat txt
ab
a c
a a
$ sort txt
a a
a c
ab

I'll bet between your a and c you have a non-breaking space or an enspace or an empspace or other high-codepoint space!

EDIT

Just ran it on Linux. I should have looked at the tags. Yes I get the same output you do! My first run was on the Mac. Looks like a difference between GNU and BSD. I will investigate further.

EDIT 2:

Linux uses a field-based sort.... still looking for how to suppress it. Tried

sort -t, txt

hoping to trick GNU into thinking the whole line was one field, but it still used the current locale to sort.

EDIT 3:

The OP solved the problem by setting the locale to C with

export LC_ALL=C

There seems to be no other approach. The sort command will use the current locale, and although it often says the C (or its alias POSIX) is the default locale, if you have Linux it has probably been set for you. Enter locale -a to see the available locales. On my system:

$ locale -a
C
POSIX
en_AG
en_AU.utf8
en_BW.utf8
en_CA.utf8
en_DK.utf8
en_GB.utf8
en_HK.utf8
en_IE.utf8
en_IN
en_NG
en_NZ.utf8
en_PH.utf8
en_SG.utf8
en_US.utf8
en_ZA.utf8
en_ZW.utf8

It seems like setting the locale to C (or its alias POSIX) is the only way to break the field-based behavior of sort and treat the whole line as one field. It is rather odd IMHO that this is how to do it. I would think the -t or -k options, or perhaps some new option would be a more sensible way to make this happen.

BTW, it looks like this question has been asked before on SO: unexpected result from gnu sort.