Bug #5322
closeddcd_mgr crashes often
0%
Description
The SPAN dcd_mgr goes zombie fairly often and must be restart. We've seen it stay up for ~30days, but recently as I've been testing things it's happened many times, so I hope to narrow down the cause soon...
Updated by Derik Barseghian over 13 years ago
I've only started paying attention to the message when dcd_mgr dies, but so far 2/2 times it dies with:
pthread_create: Resource temporarily unavailable
For the last crash, my dcd_mgr log has 250 messages like:
connection thread 1
connection thread 2
connection thread 2
etc. The thread number is always one of 0,1,2.
This is just speculation, but my guess is threads are not being resused, there are 250, and that is too many.
Some gumstix system info follows:
root@overo:/etc/init.d# lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 9.04
Release: 9.04
Codename: jaunty
root@overo:/etc/init.d# uname -a
Linux overo 2.6.32 #1 Thu Mar 4 03:52:51 PST 2010 armv7l GNU/Linux
root@overo:/etc/init.d# ulimit -a
core file size (blocks, -c) 0
data seg size (kbytes, -d) unlimited
scheduling priority (-e) 20
file size (blocks, -f) unlimited
pending signals (-i) 16382
max locked memory (kbytes, -l) 64
max memory size (kbytes, -m) unlimited
open files (-n) 1024
pipe size (512 bytes, -p) 8
POSIX message queues (bytes, -q) 819200
real-time priority (-r) 0
stack size (kbytes, -s) 8192
cpu time (seconds, -t) unlimited
max user processes (-u) unlimited
virtual memory (kbytes, -v) unlimited
file locks (-x) unlimited
root@overo:/etc/init.d# sysctl -a | grep -i thread
kernel.threads-max = 3760
vm.nr_pdflush_threads = 0
root@overo:/etc/init.d# cat /proc/meminfo
MemTotal: 241596 kB
MemFree: 119324 kB
Buffers: 20596 kB
Cached: 30888 kB
SwapCached: 0 kB
Active: 76052 kB
Inactive: 32308 kB
Active(anon): 57084 kB
Inactive(anon): 0 kB
Active(file): 18968 kB
Inactive(file): 32308 kB
Unevictable: 0 kB
Mlocked: 0 kB
SwapTotal: 0 kB
SwapFree: 0 kB
Dirty: 8 kB
Writeback: 0 kB
AnonPages: 56904 kB
Mapped: 7932 kB
Shmem: 208 kB
Slab: 8160 kB
SReclaimable: 4328 kB
SUnreclaim: 3832 kB
KernelStack: 1304 kB
PageTables: 1840 kB
NFS_Unstable: 0 kB
Bounce: 0 kB
WritebackTmp: 0 kB
CommitLimit: 120796 kB
Committed_AS: 2452660 kB
VmallocTotal: 647168 kB
VmallocUsed: 2288 kB
VmallocChunk: 643580 kB
Updated by Daniel Crawl over 13 years ago
Fixed: need to join or detach all created threads.