Edit Question - KnowledgeHub

Title Body

So many options and so little time to test them all... I wonder if someone has experiences with distributed file systems for video streaming and storage/encoding. I have a lot of huge video files (50GB to 250GB) that I need to store somewhere, be able to encode them to mp4 and stream them from several Adobe FMS servers. The only way to handle all this is with a distributed file system but now the question is which one?? My research so far tells me: Lustre : mature proven solution, used by a lot of big companies, best with >10G files is a kernel driver. Gluster : new, less mature, FUSE based that means easy to install but maybe slower due to FUSE overhead. Better to handle a large number of smaller files ~1GB MogileFS : seems to be only for small files ~MB, uses HTTP for access?? possible FUSE binding in the future. So far Lustre seems the winner but I would like to hear real experiences for the particular application I have. Also Hadoop, Redhat GFS, Coda and Windows DFS sound as options so any experiences are welcome. If someone has benchmarks please share. After some real experience this is what I have learned: Luster: Performance: Amazingly fast! I can assert that Lustre can serve a lot of streams and that encoding speed is not affected by accessing files via Lustre. POXIS compatibility: Very good!. No need to modify applications to use luster. Replication, Load Balancing and Fail Over: Very bad!. For replication load balancing we and fail over we need to rely on other software such as virtual IPs and DRDB. Installation: The worst!. Impossible to install by mere mortals. Requires a very specific combination of kernel, lustre patches and tweaks to get it working. And current luster patches usually work with old kernels that are incompatible with new hardware/software. </li

Tags (comma-separated) Cancel