Yeah, because ext4 never cleans up the dentry table after it's been made. This is a pain.
Unfortunately XFS also sucks for handling larger numbers of files - at the hundreds of millions or larger scale, you're largely metadata bound - and XFS is very slow at this for a number of reasons.
RH did an interesting study a while back [0] on storing 1 billion files, which came out in favor of ext4. If you're able to work around the lack of dentry cleanup, I still think ext4 is probably the best choice here (ideally with the journal on a small SSD) - assuming you can't store them in a database or something more appropriate like mogileFS [1]
RH did an interesting study a while back [0] on storing 1 billion files, which came out in favor of ext4. If you're able to work around the lack of dentry cleanup, I still think ext4 is probably the best choice here (ideally with the journal on a small SSD) - assuming you can't store them in a database or something more appropriate like mogileFS [1]
[0]: http://www.redhat.com/summit/2011/presentations/summit/decod...
[1]: https://code.google.com/p/mogilefs/wiki/Start