From: Donald Tidrow (dtidrow++at++patriot.net)
Date: 01/12/2006 15:24:19
On Thu, 12 Jan 2006, Jurgen Rusch wrote:
> Hi Sakis,
>
> concerning your >2GB shared memory size observation:
> what are your answers to the next 2 questions:
>
> (1) do I have a 32-bit or 63-bit version of Linux?
> [do 'uname -p': i686 means 32-bit, x86_64 means 64-bit]
>
> (2) do I have 32-bit or 64-bit Performer libraries installed?
> [do 'file libpf.so.*': you get
> either "ELF 32-bit LSB shared object ..."
> or "ELF 64-bit LSB shared object,..."]
>
> To the best of my knowledge the answers to both questions
> should be 64-bit for Performer to be able to allocate
> more than 2GB of memory, doesn't matter whether you're
> talking about shared or non-shared memory.
> Furthermore, isn't Performer for Linux only distributed
> as a 32-bit library?
>
> If your OS is 32-bit NO (C/C++/Fortran/...)program can
> allocate more than 2 GB of memory. I just did a small
Not true - on an x86 Linux system, a process can allocate over 2GB in
size, assuming there's enough RAM/swap. I've just verified this on a
system with 4GB of RAM+swap (allocating a single contiguous block of
memory 2348532224 bytes long, 2GB-1 being 2147483647 bytes), so I know
it works. You can even get a single block of memory larger than 2GB,
though this may only work on recent versions of Linux (I'm running a
2.6.12 kernel here).
2GB/process was definitely a limitation of MIPS32, though.
> experiment and here is some interesting output (using
> gcc 3.2.3 on Linux 2.4.21):
> sizeof(int) =4 (both on 32- and 64-bit Linux)
> sizeof(long int)=4 (on 32-bit Linux)
> =8 (on 64-bit Linux)
> sizeof(size_t) =4 (on 32-bit Linux)
> =8 (on 64-bit Linux)
> So, if you want to use malloc on a 32-bit Linux system
> you just CANNOT describe amounts of 2GB or more!
> A trick to circumvent this is to use calloc(2,1G) on
> a 32-bit Linux version: each argument fits in a size_t
> object and their product is 2GB. But when you do this
> on a 32-bit Linux system you just get the NULL pointer
> returned from calloc while on a 64-bit Linux system
> the calloc returns a non-NULL pointer.
On Fedora Core 4 (the OS I run at work) size_t is an unsigned long, so
theoretically you could get up to 4GB/process. However, current x86 Linux
kernels reserve the top 1GB of 32-bit address space for kernel use,
leaving 3GB for user memory. Out of that 3GB comes all the space needed
for program code, static data, stacks, and heap, so realistically you
probably can't get much more than 2.5GB or so of memory allocated from the
heap.
> When using shm_open to allocate SHARED memory a sub-
> sequent call to mmap will also need a size_t type
> argument (the second argument) to indicate the required
> size of shared memory. And, as explained above, this
> size_t type of argument simply CANNOT describe 2GB
> or more on a 32-bit Linux version since its size() is 4.
>
Shared memory might have more stringent limits (meaning I haven't
investigated it throughly yet), though you should be able to get a 2GB
(2GB-1, more likely) block of shared memory if you organize things
properly.
I haven't tried to run Performer with a PFSHAREDSIZE that big yet, though
I don't see any reason why it wouldn't work (assuming you set PFSHAREDBASE
judiciously, and it didn't collide with any shared libs)
> On 64-bit Linux systems however, you CAN allocate more
> than 2GB and I just did an experiment in a barebones
> C program with shm_open/truncate/mmap/munmap/shm_unlink
> which clearly showed I could allocate upto 7.2GB of
> memory. This was on a 64-bit Linux machine with 8GB
> physical RAM and the executable I got after compilation
> with gcc was a 64-bit executable.
> But when on the same 64-bit Linux system I asked gcc
> to create a 32-bit executable (using 'gcc -m32') the
> maximum allocatable amount of shared memory was 2GB-1!
> And this situation more or less mimics the 32-bit-Per-
> former-on-a-64-bit-Linux situation.
>
Indeed, with x86_64 or MIPS64 the limits are pretty much gone.
> I also simulated Performer in the sense that I did a
> getenv and atol to read the value from PFSHAREDSIZE
> into a 'long int' variable. What struck me - I din't
> know this - was that even with PFSHAREDSIZE set to
> 6GB the function atol (with the -m32 compile option
> set!) returned the value 2GB-1 ..... :-)
>
> As far as I can see all the above is consistent with
> your observation Sakis!
>
> With kind regards, Jurgen Rusch
--- In October 1997, three computer lab staffers disappeared from the server room while debugging a mysterious router problem.One year later, their packet logs were found...
The Blair Switch Project
-------------------- | Don Tidrow | | Vis-Sim Geek | --------------------
This archive was generated by hypermail 2b29 : Thu Jan 12 2006 - 15:21:51 PST