From: Jurgen Rusch (jurgen.rusch++at++philips.com)
Date: 01/12/2006 03:59:49
Hi Sakis,
concerning your >2GB shared memory size observation:
what are your answers to the next 2 questions:
(1) do I have a 32-bit or 63-bit version of Linux?
[do 'uname -p': i686 means 32-bit, x86_64 means 64-bit]
(2) do I have 32-bit or 64-bit Performer libraries installed?
[do 'file libpf.so.*': you get
either "ELF 32-bit LSB shared object ..."
or "ELF 64-bit LSB shared object,..."]
To the best of my knowledge the answers to both questions
should be 64-bit for Performer to be able to allocate
more than 2GB of memory, doesn't matter whether you're
talking about shared or non-shared memory.
Furthermore, isn't Performer for Linux only distributed
as a 32-bit library?
If your OS is 32-bit NO (C/C++/Fortran/...)program can
allocate more than 2 GB of memory. I just did a small
experiment and here is some interesting output (using
gcc 3.2.3 on Linux 2.4.21):
sizeof(int) =4 (both on 32- and 64-bit Linux)
sizeof(long int)=4 (on 32-bit Linux)
=8 (on 64-bit Linux)
sizeof(size_t) =4 (on 32-bit Linux)
=8 (on 64-bit Linux)
So, if you want to use malloc on a 32-bit Linux system
you just CANNOT describe amounts of 2GB or more!
A trick to circumvent this is to use calloc(2,1G) on
a 32-bit Linux version: each argument fits in a size_t
object and their product is 2GB. But when you do this
on a 32-bit Linux system you just get the NULL pointer
returned from calloc while on a 64-bit Linux system
the calloc returns a non-NULL pointer.
When using shm_open to allocate SHARED memory a sub-
sequent call to mmap will also need a size_t type
argument (the second argument) to indicate the required
size of shared memory. And, as explained above, this
size_t type of argument simply CANNOT describe 2GB
or more on a 32-bit Linux version since its size() is 4.
On 64-bit Linux systems however, you CAN allocate more
than 2GB and I just did an experiment in a barebones
C program with shm_open/truncate/mmap/munmap/shm_unlink
which clearly showed I could allocate upto 7.2GB of
memory. This was on a 64-bit Linux machine with 8GB
physical RAM and the executable I got after compilation
with gcc was a 64-bit executable.
But when on the same 64-bit Linux system I asked gcc
to create a 32-bit executable (using 'gcc -m32') the
maximum allocatable amount of shared memory was 2GB-1!
And this situation more or less mimics the 32-bit-Per-
former-on-a-64-bit-Linux situation.
I also simulated Performer in the sense that I did a
getenv and atol to read the value from PFSHAREDSIZE
into a 'long int' variable. What struck me - I din't
know this - was that even with PFSHAREDSIZE set to
6GB the function atol (with the -m32 compile option
set!) returned the value 2GB-1 ..... :-)
As far as I can see all the above is consistent with
your observation Sakis!
With kind regards, Jurgen Rusch
-------------------------------
Computer Department
Support for Audio, Video & Graphics
Philips Research Laboratories
The Netherlands
This archive was generated by hypermail 2b29 : Thu Jan 12 2006 - 04:01:59 PST