Re: It just begs the question everywhere: how to move a journaling log to a new device?

[Jeremy Jackson <jerj@xxxxxxxxxxxx>]

Steve Lord wrote:
> As for sizing the log, it always seems to be like asking how
> long a piece of string is.
>
> A bigger log means you can keep more metadata in flight before
> you have to wait for metadata to be flushed before you can write
> more log records. We refer to this mode of operation as tail
> pushing - think of the log as being a small train on a circular
> toy train track. When you are tail pushing, the you have added
> carriages until the engine is pushing against the caboose (I have
> a three year old, can you tell ;-)
>
> In more technical terms, the amount of metadata update you want
> to be able to deal with in a burst without the slowdown in
> throughput caused by tail pushing is the governing factor here.
> In reality, you can always get into tail pushing mode if you 
> give the fs enough work to do, but for non sustained metadata
> ops, a larger log should help throughput.
>
> The downside of this is recovery time during mount, the larger
> the log, the longer mount will take. So if you want really
> fast recovery after a crash, you want a small log.
>
> Phew, thats probably more answer than you were expecting.

Perhaps, but this is really good stuff to know.  I'm saving this email 
in my bag of tricks until such time as it becomes part of the standard 
documentation.

Are you saying the log space is unaccounted space between two AGs, or 
does it get an extent or is marked in some kind of block bitmap (I 
really don't know much about XFS data structures).

If so, something that comes to mind is, could the log get allocated as 
an inode or extent?  It would make it possible to shift it around and 
reclaim the space.  I guess it would be hard to have an extent on 
another device (realtime ss excluded) for  and external log.  xfs_fsr 
when unmounted?
Say an fs grows so much that the log is no longer in the middle, or its 
size needs to be changed.

As for tail pushing, the continuous throughput could be improved by 
using a ping-pong buffer.  2 log devices, on separate disks.  One is 
written to until it becomes full, then logging switches to the other 
while the first is read and flushed.  This way each device is written 
then read sequentially, rather than each process fighting the other and 
the head seeking back and forth to service them.

Actually, now that I think of it, the ping-pong thing make it easy to 
move the log around, resize it (to a different location), etc. while 
mounted.  Setup the new log, wait till old one fills up or force a 
switch to the new one, flush the old one, then de-allocate it.

Now if I only I didn't have to work for a living...

Cheers,

Jeremy Jackson