"Westerdale, John" <jwesterdale@xxxxxxxxxx> wrote:
> Am debugging an FTP server and found a 2 GB file size limit.... I think!
>
> I ssh from desk (Indigo2) to Linux 2.4.14/XFS server with array on it
> (RAID 5 XFS based on 7.2/1.0.2a installer).
> (FWIW Dual PII-400/256/Software RAID on OS, and Raid 5 over 8 x 9 GB drives.
>
> From Array Server I establish several FTP connections to servers to pull
> over a tar blob of the docroot.
>
> (one such transfer is shown below)
>
> The trusty O200's pack up a nice 2.3 GB file nightly of data, and I ftp it
> over.. But it seems to stop at
>
> 2147483647 bytes every time! WHere is the wall?
The problem is in the Linux kernel, after version 2.4.6 or .7 or so
(I forget, so nobody stomp me here...).
The kernel guys decided to force user-level programs to explicitly use
"large file" #defines and calls to access any file larger than 2GB,
I think since the seek calls can do wrong things behind your back unless
you know what you're doing.
Personally, I have a mini-patch I have been applying to all later
kernels to get rid of this behavior, since I absolutely NEED the
ability to FTP/fileutils/tar/etc. huge files for backup purposes
as well as random OS experiments, and nobody has fixed the various
programs involved. I'm just streaming them from one place to another
anyway.
I kind of wish the XFS-enabled RedHat distribution was set up this
way, but I didn't want to cause a flame-war.
For those interested, here is the patch I've been using:
-------------------(start bigfiles.patch)--------------------
--- linux.orig/fs/open.c.orig Sun Oct 14 10:40:45 2001
+++ linux/fs/open.c Sun Oct 28 09:22:06 2001
@@ -777,9 +777,9 @@
char * tmp;
int fd, error;
-#if BITS_PER_LONG != 32
+ /* Always try to get large file access */
flags |= O_LARGEFILE;
-#endif
+
tmp = getname(filename);
fd = PTR_ERR(tmp);
if (!IS_ERR(tmp)) {
-------------------(end bigfiles.patch)--------------------
--
Erich Stefan Boleyn <erich@xxxxxxxx> http://www.uruk.org/
"Reality is truly stranger than fiction; Probably why fiction is so popular"
|