Here’s one cause which doesn’t show up in most explanations on the ‘net: The configurable kernel limit kernel.pid_max has been reached. (The default is 32K, at least on my CentOS 5 box.)
With threaded Apache httpd on Linux/Unix, the pthread_create() failure can be logged as
Cannot allocate memory: apr_thread_create: unable to create worker thread
The most common cause of this failure is still the thread stack size, but if you’re creating thousands of threads and shrinking the stack size doesn’t help, see if increasing kernel.pid_max allows more threads to be created.
Hats off to a colleague at &bigco; for finding this solution.