[Top] [All Lists]

[PATCH] fix qsort breakage

To: linux-xfs@xxxxxxxxxxx
Subject: [PATCH] fix qsort breakage
From: Christoph Hellwig <hch@xxxxxxxxxxxxx>
Date: Sat, 4 May 2002 12:15:12 +0100
Sender: owner-linux-xfs@xxxxxxxxxxx
User-agent: Mutt/
Qsort in the XFS tree has two problems:

 o allocates memory using GFP_KERNEL although called from under i_sem
   (possible deadlock)
 o doesn't check kmalloc return value (possible NULL-ptr dereference)

The below patch tries to address both issues, but without a return value
singnalling ENOMEM is rather difficult..

Andi Kleen suggested getting the pivot from stack, someone with enough
time might check the callers for sane ßize arguments.

Index: linux/fs/xfs_support/qsort.c
RCS file: /cvs/linux-2.4-xfs/linux/fs/xfs_support/qsort.c,v
retrieving revision 1.4
diff -u -u -r1.4 qsort.c
--- linux/fs/xfs_support/qsort.c        2002/03/12 06:25:01     1.4
+++ linux/fs/xfs_support/qsort.c        2002/05/04 11:08:26
@@ -88,9 +88,13 @@
   /* Allocating SIZE bytes for a pivot buffer facilitates a better
      algorithm below since we can do comparisons directly on the pivot. */
-  char *pivot_buffer = (char *) kmalloc (size, GFP_KERNEL);
+  char *pivot_buffer = (char *) kmalloc (size, GFP_NOFS);
   const size_t max_thresh = MAX_THRESH * size;
+  if (pivot_buffer == NULL)
+    /* <shrug> any way to return failure from qsort?  */
+    return;
   if (total_elems == 0)
     /* Avoid lossage with unsigned arithmetic below.  */

<Prev in Thread] Current Thread [Next in Thread>