Mail Archive Search: <+subject:/^(?:^\s*(re|sv|fwd|fw)[\[\]\d]*[:>-]+\s*)*tuning\,\s+many\s+small\s+files\,\s+small\s+blocksize\s*$/>

Results:

References: [ +subject:/^(?:^\s*(re|sv|fwd|fw)[\[\]\d]*[:>-]+\s*)*tuning\,\s+many\s+small\s+files\,\s+small\s+blocksize\s*$/: 45 ]

Total 45 documents matching your query.

1. tuning, many small files, small blocksize (score: 1): Author: "Jeff Breidenbach" <jeff@xxxxxxx>; Date: Fri, 15 Feb 2008 21:01:10 -0800; I'm testing xfs for use in storing 100 million+ small files (roughly 4 to 10KB each) and some directories will contain tens of thousands of files. There will be a lot of random reading, and also some; /archives/xfs/2008-02/msg00153.html (10,161 bytes)
2. Re: tuning, many small files, small blocksize (score: 1): Author: Hannes Dorbath <light@xxxxxxxxxxxxxxxxxxxx>; Date: Sat, 16 Feb 2008 10:28:37 +0100; Jeff Breidenbach wrote: The underlying disks use linux software RAID-1 manged by mdadm with 5X redundancy. E.g. 5 drives that completely mirror each other. That's maybe a bit paranoid, but on the oth; /archives/xfs/2008-02/msg00159.html (10,137 bytes)
3. Re: tuning, many small files, small blocksize (score: 1): Author: "Jeff Breidenbach" <jeff@xxxxxxx>; Date: Sat, 16 Feb 2008 02:24:14 -0800; Yes, the goal is fast read performance for small files. This is highly appreciated, thank you very much. I'm running vendor a supplied kernel of 2.6.22 and a quick test shows the unsupported feature; /archives/xfs/2008-02/msg00160.html (10,938 bytes)
4. Re: tuning, many small files, small blocksize (score: 1): Author: pg_xfs2@xxxxxxxxxxxxxxxxxxx; Date: Sat, 16 Feb 2008 12:23:00 +0000; Reading this was quite entertaining :-). Makes me wonder why silly people come up with pointless stuff like this :-) http://WWW.Oracle.com/database/berkeley-db.html; /archives/xfs/2008-02/msg00161.html (9,763 bytes)
5. Re: tuning, many small files, small blocksize (score: 1): Author: "Jeff Breidenbach" <jeff@xxxxxxx>; Date: Sat, 16 Feb 2008 12:30:57 -0800; Ah, wasn't that painful after all. Thanks again.; /archives/xfs/2008-02/msg00163.html (10,374 bytes)
6. Re: tuning, many small files, small blocksize (score: 1): Author: David Chinner <dgc@xxxxxxx>; Date: Tue, 19 Feb 2008 09:53:35 +1100; ..... I'd suggest wasting 20% of disk space and staying with 4k block size. Large directory block size (-n size=XXX), esp. if you are putting thousands of files per directory.... Cheers, Dave. -- Dav; /archives/xfs/2008-02/msg00187.html (9,808 bytes)
7. Re: tuning, many small files, small blocksize (score: 1): Author: Linda Walsh <xfs@xxxxxxxxx>; Date: Mon, 18 Feb 2008 15:12:44 -0800; Jeff Breidenbach wrote: I'm testing xfs for use in storing 100 million+ small files (roughly 4 to 10KB each) and some directories will contain tens of thousands of files. There will be a lot of rando; /archives/xfs/2008-02/msg00191.html (12,171 bytes)
8. Re: tuning, many small files, small blocksize (score: 1): Author: David Chinner <dgc@xxxxxxx>; Date: Tue, 19 Feb 2008 10:51:03 +1100; That makes no sense. Inodes are *unique* - they are not shared with any other inode at all. Could you explain why you think that 256 byte inodes are any different to larger inodes in this respect? No; /archives/xfs/2008-02/msg00196.html (10,988 bytes)
9. Re: tuning, many small files, small blocksize (score: 1): Author: Timothy Shimmin <tes@xxxxxxx>; Date: Tue, 19 Feb 2008 11:48:53 +1100; Jeff Breidenbach wrote: the kernel before April is painful but I'll do it if important. Presumably there's no simple way to migrate a non-lazy xfs filesytem to a lazy one. Dave would know that answer; /archives/xfs/2008-02/msg00200.html (10,616 bytes)
10. Re: tuning, many small files, small blocksize (score: 1): Author: Linda Walsh <xfs@xxxxxxxxx>; Date: Mon, 18 Feb 2008 17:03:57 -0800; David Chinner wrote: That makes no sense. Inodes are *unique* - they are not shared with any other inode at all. Could you explain why you think that 256 byte inodes are any different to larger inode; /archives/xfs/2008-02/msg00202.html (14,351 bytes)
11. Re: tuning, many small files, small blocksize (score: 1): Author: David Chinner <dgc@xxxxxxx>; Date: Tue, 19 Feb 2008 13:49:24 +1100; Inode I/O in XFS is done in *8k clusters* regardless of inode size. We _never_ do single inode I/O and hence you logic completely breaks down at that point ;). Inode clustering is a substantial perfo; /archives/xfs/2008-02/msg00206.html (12,823 bytes)
12. Re: tuning, many small files, small blocksize (score: 1): Author: "Jeff Breidenbach" <jeff@xxxxxxx>; Date: Mon, 18 Feb 2008 20:58:56 -0800; Wow, this is quite some discussion. I went with Hannes Dorbath's original suggestion (appended), and am now several days into copying data onto the filesystem. It's conceivable to change course at th; /archives/xfs/2008-02/msg00208.html (11,339 bytes)
13. Re: tuning, many small files, small blocksize (score: 1): Author: pg_xf2@xxxxxxxxxxxxxxxxxxx (Peter Grandi); Date: Tue, 19 Feb 2008 08:27:54 +0000; jeff> I'm testing xfs for use in storing 100 million+ small jeff> files (roughly 4 to 10KB each) and some directories will jeff> contain tens of thousands of files. There will be a lot jeff> of rand; /archives/xfs/2008-02/msg00211.html (13,358 bytes)
14. Re: tuning, many small files, small blocksize (score: 1): Author: Hannes Dorbath <light@xxxxxxxxxxxxxxxxxxxx>; Date: Tue, 19 Feb 2008 12:44:57 +0100; That sounds like a good use for a LDAP database, but using Berkeley DB directly may be best. One could also do a FUSE module or a special purpose NFS server that presents a Berkeley DB as a filesyste; /archives/xfs/2008-02/msg00212.html (10,525 bytes)
15. Re: tuning, many small files, small blocksize (score: 1): Author: pg_xfs2@xxxxxxxxxxxxxxxxxxx (Peter Grandi); Date: Tue, 19 Feb 2008 21:24:34 +0000; [ ... a collection of millions of small records ... ] Sometimes BDB had problems, but that seems in the past. It also relies critically on some precise behaviour from the storage layer, filesystem d; /archives/xfs/2008-02/msg00228.html (10,991 bytes)
16. tuning, many small files, small blocksize (score: 1): Author: "Jeff Breidenbach" <jeff@xxxxxxx>; Date: Fri, 15 Feb 2008 21:01:10 -0800; I'm testing xfs for use in storing 100 million+ small files (roughly 4 to 10KB each) and some directories will contain tens of thousands of files. There will be a lot of random reading, and also some; /archives/xfs/2008-02/msg00569.html (10,161 bytes)
17. Re: tuning, many small files, small blocksize (score: 1): Author: Hannes Dorbath <light@xxxxxxxxxxxxxxxxxxxx>; Date: Sat, 16 Feb 2008 10:28:37 +0100; Jeff Breidenbach wrote: The underlying disks use linux software RAID-1 manged by mdadm with 5X redundancy. E.g. 5 drives that completely mirror each other. That's maybe a bit paranoid, but on the oth; /archives/xfs/2008-02/msg00575.html (10,137 bytes)
18. Re: tuning, many small files, small blocksize (score: 1): Author: "Jeff Breidenbach" <jeff@xxxxxxx>; Date: Sat, 16 Feb 2008 02:24:14 -0800; Yes, the goal is fast read performance for small files. This is highly appreciated, thank you very much. I'm running vendor a supplied kernel of 2.6.22 and a quick test shows the unsupported feature; /archives/xfs/2008-02/msg00576.html (10,938 bytes)
19. Re: tuning, many small files, small blocksize (score: 1): Author: pg_xfs2@xxxxxxxxxxxxxxxxxxx; Date: Sat, 16 Feb 2008 12:23:00 +0000; Reading this was quite entertaining :-). Makes me wonder why silly people come up with pointless stuff like this :-) http://WWW.Oracle.com/database/berkeley-db.html; /archives/xfs/2008-02/msg00577.html (9,763 bytes)
20. Re: tuning, many small files, small blocksize (score: 1): Author: "Jeff Breidenbach" <jeff@xxxxxxx>; Date: Sat, 16 Feb 2008 12:30:57 -0800; Ah, wasn't that painful after all. Thanks again.; /archives/xfs/2008-02/msg00579.html (10,374 bytes)

Current List: 1 - 20
Page: [1] [2] [3]