問題描述

我想最多存儲 10M 文件，2TB 存儲單元.我需要的唯一屬性僅限于文件名及其內容(數據).

I would like to store up-to 10M files, 2TB storage unit. The only properties which I need restricted to filenames, and their contents (data).

文件最大長度為 100MB，大部分小于 1MB.刪除文件的能力是必需的，寫入和讀取速度都應該是優先考慮的 - 而不需要低存儲效率、恢復或完整性方法.

The files max-length is 100MB, most of them are less than 1MB. The ability of removing files is required, and both writing and reading speeds should be a priority - while low storage efficiency, recovery or integrity methods, are not needed.

我考慮過 NTFS，但它的大部分功能不是必需的，雖然不能被禁用并且被認為是一個開銷問題，其中一些是:創建日期、修改日期、屬性、日志和當然權限.

I thought about NTFS, but most of its features are not needed, while can't be disabled and considered to be an overhead concern, a few of them are: creation date, modification date, attribs, journal and of course permissions.

由于不需要文件系統的本機功能，您是否建議我將 SQLITE 用于此要求?或者有一個我應該注意的明顯缺點?(有人會猜測刪除文件將是一項復雜的任務?)

Due to the native features of a filesystem which are not needed, would you suggest I'll use SQLITE for this requirement? or there's an obvious disadvantage that I should be aware about? (one would guess that removing files will be a complicated task?)

(SQLITE 將通過 C api)

(SQLITE will be via the C api)

我的目標是使用更合適的解決方案來提高性能.提前致謝 - Doori Bar

My goal is to use a more suited solution to gain performance. Thanks in advance - Doori Bar

推薦答案

如果您的主要要求是性能，請使用本機文件系統.DBMS 不太適合處理大型 BLOB，因此 SQLite 根本不是您的選擇(甚至不知道為什么每個人都認為 SQLite 是每個漏洞的塞子).

If your main requirement is performance, go with native file system. DBMS are not well suited for handling large BLOBs, so SQLite is not an option for you at all (don't even know why everybody considers SQLite to be a plug for every hole).

為了提高 NTFS(或您選擇的任何其他文件系統)的性能，不要將所有文件放入單個文件夾，而是按文件名的前 N ??個字符或擴展名對文件進行分組.

To improve performance of NTFS (or any other file system you choose) don't put all files into single folder, but group files by first N characters of their file names, or also by extension.

市場上還有一些其他文件系統，也許其中一些提供禁用某些使用功能的可能性.您可以查看維基百科上的比較并查看它們.

Also there exist some other file systems on the market and maybe some of them offer possibility to disable some of used features. You can check the comparison on Wikipedia and check them.

更正:我做了一些測試(雖然不是很廣泛)，但對于大多數類型的操作，將文件分組到子目錄中并沒有表現出性能優勢，并且 NTFS 非常有效地處理了 26^4 個空文件在單個目錄中從 AAAA 到 ZZZZ 命名.因此，您需要檢查特定文件系統的效率.

Correction: I've made some tests (not very extensive though) that show no performance benefit in grouping files into subdirectories for most types of operations, and NTFS quite efficiently handled 26^4 empty files named from AAAA to ZZZZ in a single directory. So you need to check efficiency for your particular file system.

這篇關于FILESYSTEM vs SQLITE，同時存儲多達 1000 萬個文件的文章就介紹到這了，希望我們推薦的答案對大家有所幫助，也希望大家多多支持html5模板網！