I am picturing Rik with shivers going down his spine right now ;-)
The better way to deal with this is either to have a small pool
of these structures we can fall back on, or even to fail the I/O.
The latter will shutdown the filesystem though.
I may come up with something else, but I have to dash now.
I presume you are running on lvm here?
Steve
On Fri, 2002-09-27 at 16:50, Ravi Wijayaratne wrote:
> Hi all,
>
> As I indicated On a previous posting we hit a BUG() in
> fs/xfs/pagebuf/page_buf.c in the following lines.
>
> psync = (pagesync_t *) kmalloc(sizeof(pagesync_t),
> GFP_NOFS);
> /* Ugh - out of memory condition here */
> if(psync == NULL){
> BUG();
> }
>
> in function _pagebuf_page_io(...).
> (linux-2.4.18 xfs-1.1)
> The situation occurred in one of our stress tests
> under heavy I/O load. We were pounding our server with
> Samba clients. I tried many times to recreate the
> situation but was unable to do so. May be that is good
> :), but there is a loop hole here. Kmlloc is called
> with GFP_NOFS here. Therefore I suggest that we give
> kmalloc a chance and retry the allocation several
> times. That way kswapd has some time to free more
> memory.
>
> The following patch does that. Since the failure path
> cannot be reproduced, I could not test the patch in a
> real life situation. Some insights are much
> appreciated.
>
> I am not sure whether xfs_support/kmem.c is the proper
> place for the patch.
>
> Thanx
> Ravi
>
> --------0x-------x0--------------
> --- linux.orig/fs/xfs/pagebuf/page_buf.c Fri
> May 31 05:21:04 2002
> +++ linux/fs/xfs/pagebuf/page_buf.c Sat Sep 28
> 23:21:00 2002
> @@ -1449,11 +1449,18 @@
>
> if (blk_length != 1) {
> psync = (pagesync_t *)
> kmalloc(sizeof(pagesync_t), GFP_NOFS);
> -
> /* Ugh - out of memory condition here
> */
> - if (psync == NULL)
> - BUG();
> -
> + printk(KERN_ALERT "%s:%s:%d Memory
> allocation failed. Waiting to free\n",+
> __FILE__,__FUNCTION__,__LINE__);
> + if (psync == NULL){
> + /* give kmalloc a chance */
> + psync=(pagesync_t
> *)kmem_alloc_wait(sizeof(pagesync_t),GFP_NOFS);+
> if(psync == NULL){
> + /* increase wait time
> and try ? */
> + /* bail out if failed
> */
> + BUG();
> + }
> + }
> psync->pb = pb;
> psync->locking = locking;
> atomic_set(&psync->remain, 0);
> --- linux.orig/fs/xfs_support/kmem.c Fri May 31
> 05:21:05 2002
> +++ linux/fs/xfs_support/kmem.c Sat Sep 28 23:04:40
> 2002
> @@ -139,6 +139,47 @@
> return rval;
> }
>
> +/*
> + * in case we run into situations where kmalloc
> returns
> + * NULL due to sluggishness of page swapping we will
> wait
> + * if we can sleep.
> + */
> +
> +#define XFS_KMALLOC_MAXTRIES 1000
> +
> +void *
> +kmem_alloc_wait(size_t size, int flags)
> +{
> + void *ptr=NULL;
> + unsigned long last_try=0;
> + int try_cnt=0;
> +
> +
> + /* if size is bigger than max cache size
> + * kmalloc will return NULL, so we will not
> + * retry
> + */
> + if(size > MAX_SLAB_SIZE)
> + return NULL;
> + /* we wont sleep if we are not supposed to */
> + if(!(flags & __GFP_WAIT))
> + return kmalloc(size,flags);
> + while(try_cnt < XFS_KMALLOC_MAXTRIES){
> + /* force a sleep first */
> + if(try_cnt > 0)
> + ptr=kmalloc(size,flags);
> + if(ptr){
> + return ptr;
> + }
> + if (time_after(jiffies, last_try +
> 5*HZ)) {
> + last_try = jiffies;
> + }
> + yield();
> + try_cnt++;
> + }
> +}
> +
> + void *
> kmem_zalloc(size_t size, int flags)
> {
> @@ -250,6 +291,7 @@
> EXPORT_SYMBOL(kmem_zone_alloc);
> EXPORT_SYMBOL(kmem_zone_free);
> EXPORT_SYMBOL(kmem_alloc);
> +EXPORT_SYMBOL(kmem_alloc_wait);
> EXPORT_SYMBOL(kmem_realloc);
> EXPORT_SYMBOL(kmem_zalloc);
> EXPORT_SYMBOL(kmem_free);
>
>
>
>
> =====
> ------------------------------
> Ravi Wijayaratne
>
> __________________________________________________
> Do you Yahoo!?
> New DSL Internet Access from SBC & Yahoo!
> http://sbc.yahoo.com
--
Steve Lord voice: +1-651-683-3511
Principal Engineer, Filesystem Software email: lord@xxxxxxx
|