_notify() to large numbers of recipients hard-crashes Querki

(Bug, Closed -> Fixed, Priority: Critical, Test Status: No automated tests yet , Reported By Bad Link: Thing 3y284oe not found, )

Email, Functions

Summary: This is a P0 bug, since it actually crashed the whole cluster. Worse, it destabilizes the cluster, making recovery tricky.

At this point, it isn't at all clear what's happening. Individual emails seem to be fine, but sending a lot causes things to go boom. ("A lot" in this case is ~80 - 90.)

When the problem happens, things get really hinky. I can't simply restart Querki -- I have to individually go into each node and restart ConductR itself, then restart Querki. The problem seems to be related to "port already bound" errors during restart, which suggests that when Querki crashes, it is leaving a rogue thread running somewhere, holding things open.