Improving mod_perl Driven Site's Performance -- Part IV: Sharing Memory Page 2

I have shown how to measure the size of the process' shared memory, but we still want to know what the real memory usage is. Obviously this cannot be calculated simply by adding up the memory size of each process because that wouldn't account for the shared memory.

On the other hand we cannot just subtract the shared memory size from the total size to get the real memory usage numbers, because in reality each process has a different history of processed requests, therefore the shared memory is not the same for all processes.

So how do we measure the real memory size used by the server we run? It's probably too difficult to give the exact number, but I've found a way to get a fair approximation which was verified in the following way. I have calculated the real memory used, by the technique you will see in the moment, and then have stopped the Apache server and saw that the memory usage report indicated that the total used memory went down by almost the same number I've calculated. Note that some OSs do smart memory pages caching so you may not see the memory usage decrease as soon as it actually happens when you quit the application.

This is a technique I've used:

  1. For each process sum up the difference between shared and system memory. To calculate a difference for a single process use:
      use GTop;
      my  = GTop->new->proc_mem(20984);

      my      = ->size - ->share;
      print "Difference is  bytes\n";
  2. Now if we add the shared memory size of the process with maximum shared memory, we will get all the memory that actually is being used by all httpd processes, except for the parent process.
  3. Finally, add the size of the parent process.

Please note that this might be incorrect for your system, so you use this number on your own risk.

I've used this technique to display real memory usage in the module Apache::VMonitor (see on of the previous articles), so instead of trying to manually calculate this number you can use this module to do it automatically. In fact, in the calculations used in this module there is no separation between the parent and child processes; they are all counted indifferently using the following code:

  use GTop ();
  my  = GTop->new;
  my  = 0;
  my  = 0;
  # @mod_perl_pids is initialized by Apache::Scoreboard, irrelevant here
  my @mod_perl_pids = some_code();
  for my  (@mod_perl_pids)
    my  = ->proc_mem();
    my      = ->size();
    my     = ->share();
     +=  - ;
      =  if  < ;
  my  += ;

So as you see, we accumulate the difference between the shared and reported memory:

      += -;

and at the end add the biggest shared process size:

  my  += ;

So now contains approximately the really used memory.

Are My Variables Shared?

How do you find out if the code you write is shared between the processes or not? The code should be shared, except where it is on a memory page with variables that change. Some variables are read-only in usage and never change. For example, if you have some variables that use a lot of memory and you want them to be read-only. As you know the variable becomes unshared when the process modifies its value.

This article was originally published on Jan 8, 2001

Thanks for your registration, follow us on our social networks to keep up-to-date