Hello Dave,
On Fri, Jan 21, 2011 at 10:43:10AM +1100, Dave Chinner wrote:
> .....
> > in other words xfsyncd and xfsbufd now alternate at 18s intervals
> > keeping the drive busy with nothing constructive hours after the
> > last write to the drive.
>
> This is a different problem, and not one I've seen before. Looking
> at the traces, it appears that we have not empties the AIL. At
> least, that's what I'm assuming at this point because log IO
> completion is not updating log tail. When we start a log IO, we set
> the log header lsn to the current head:
>
> > xfssyncd/sdc1-1413 [000] 3356.093456: xfs_log_reserve: dev 8:33 type
> > DUMMY1 t_ocnt 1 t_cnt 1 t_curr_res 2740 t_unit_res 2740 t_flags
> > XLOG_TIC_INITED reserveq empty writeq empty grant_reserve_cycle 2
> > grant_reserve_bytes 428523008 grant_write_cycle 2 grant_write_bytes
> > 428523008 curr_cycle 2 curr_block 836959 tail_cycle 2 tail_block 810683
>
> Which in this case is: curr_cycle 2 curr_block 836959
>
> When the log IO completes, that value gets written to the
> l_last_sync_lsn. When the AIL tail is removed, the tail lsn is
> updated to the new tail item. If the AIL is empty, then the
> l_last_sync_lsn is used. That means then next dummy transaction
> made to cover the log should have the cycle/block of the above
> current cycle.
>
> Instead, what I see is that the next dummmy transaction shows:
>
> > xfssyncd/sdc1-1413 [000] 3392.067122: xfs_log_reserve: dev 8:33 type
> > DUMMY1 t_ocnt 1 t_cnt 1 t_curr_res 2740 t_unit_res 2740 t_flags
> > XLOG_TIC_INITED reserveq empty writeq empty grant_reserve_cycle 2
> > grant_reserve_bytes 428524032 grant_write_cycle 2 grant_write_bytes
> > 428524032 curr_cycle 2 curr_block 836961 tail_cycle 2 tail_block 810683
>
> The current head has moved: curr_cycle 2 curr_block 836961
>
> But the tail hasn't: tail_cycle 2 tail_block 810683
>
> So effectively we've got some item on the AIL that we haven't
> flushed and isn't being flushed by xfssyncd. That's the problem I
> need to get to the bottom of and it also explains why it's an
> intermitten problem...
I have been watching the stream of patches that go into 2.6.38, but I probably
missed the one that might be the answer to the problem above. can you please
tell me which one to try, or can I help with anything?
much appreciated,
peter
pgpD9dEuJGVPl.pgp
Description: PGP signature
|