On 11/29/2011 02:17 PM, Paul Anderson wrote:
18.104.22.168 (x64 intel, in todays case a 40TiByte SAN volume) appears to
have a bug whereby not all active metadata will be flushed even on a
quiescent machine (one that has nonetheless in the past been under
very high load).
We have tried several variations of clean shutdowns, combined with for
example the "echo 3>/proc/sys/vm/drop_caches" trick to no avail - we
still get lost files (well, 0 length files).
We have several big servers scheduled to go down shortly, and I was
wondering if there are other ideas besides just coping all recent data
to another server.
Set your vm dirty time to small values. 1 second (100 centiseconds) or
so, among other things. You can also force the mount to be synchronous
(kills performance though).
mount -o remount,sync /mountpoint
# not sure if this works with xfs though ...
sysctl -w vm.dirty_writeback_centisecs=100
sysctl -w vm.dirty_expire_centisecs=100
sysctl -w vm.dirty_ratio=1
Joseph Landman, Ph.D
Founder and CEO
Scalable Informatics Inc.
web : http://scalableinformatics.com
phone: +1 734 786 8423 x121
fax : +1 866 888 3112
cell : +1 734 612 4615