sorting images by color

I'm looking for a way to sort images as in the following screenshot:

http://www.pixolution.de/sites/LargeImages_en.html

I've looked at all the threads on this topic on stackoverflow but none of the proposed solutions even come close to giving me the image above.

Approaches I've tried:

1. for each image, build histogram of rgb colors in descending order of occurrence

2. for each histogram, calculate a distance from black (r:0,g:0,b:0) as follows:

   for color in image_histogram:
     total_distance += color.percentage_of_image * distance(BLACK_RGB, color.rgb)
   

then sort images by their distances

I was hoping that images of similar color distribution would end up with a similar distance and result in a visual ordering by color. This was not the case, it seems to somewhat work but not like in the image above.

For the distance function, I've tried euclidean distance, hsv sorting (h->s->v) and even Lab distance sorting. None of which has helped

If anyone has a better approach, I would love to know!