Precise Shock Therapy

Consider this Cyborg Stephen Hawking Part II. Our understanding of the brain is so very underwhelming. Ran across an interesting article: Brain electrodes fix depression long term.

Deep depression that fails to respond to any other form of therapy can be moderated or reversed by stimulation of areas deep inside the brain. Now the first placebo-controlled study of this procedure shows that these responses can be maintained in the long term.

Neurologist Helen Mayberg at Emory University in Atlanta, Georgia, followed ten patients with major depressive disorder and seven with bipolar disorder, or manic depression, after an electrode device was implanted in the subcallosal cingulate white matter of their brains and the area continuously stimulated.

All but one of twelve patients who reached the two-year point in the study had completely shed their depression or had only mild symptoms.

The idea of stimulating brains with electricity or magnets in my or anyone’s brain feels disturbing. It sounds like all these went through it as a matter of last resort. Last weekend I watched a called Blood & Guts: A History of Surgery on the brain. There is a scene where a guy is touching his nose repeatedly and using intense magnetic fields prevent the subject from being able to control muscles. My spine shivers every time I watch this. (So why do I watch it over and over?)

Of course, an old way of solving this was trans-orbital lobotomies. Take an ice pick and hammer it through the upper eye socket (the orbit) into the prefrontal cortex. Hook me up to electrodes, please…

Why Ten

The question of why we run ten clusters came up recently. Off the top of my head, the answer was okay. Here is my more thoughtful response.

Whenever I have been in a conversation with a BEA (more recently Oracle) person on Weblogic, the number of nodes we run has invariably surprised them. Major banks serve ten times the number simultaneous users we have on a half dozen managed nodes or less. We have 130 managed nodes for production. Overkill?

There are some advantages they have.

  1. Better control over the application. WebCT hacked together an install process very much counter to the way BEA would have done it. BEA would have had one install the database, the web servers, and then deploy the application using either the console or command-line. WebCT created an installer which does all this in the background out of sight and mind of the administrator. They also created start and stop scripts which do command-line interaction of Weblogic to start the application. Great for automation and making it simple for administrators. It also lobotomies the console making many advanced things one could normally do risky. So now the console is only useful for some minor configuration management and monitoring.
  2. Better control over the code. When there is a performance issue, they can find what is the cause and improve the efficiency of the code. The best I can do is point out the inefficiencies to a company who chose as a priority a completely different codebase. If you do not have control over the code, then you give the code more resources.
  3. As good as Weblogic is at juggling multiple managed nodes, more nodes does not always equal better. Every node has to keep track of the others. The heart beats communicate through multicast. Every node sends out its own and listens for the same from all the others. Around twenty nodes they would miss occasional beats on their own. Thrown in a heavy work load and an overwhelmed node can miss enough missed beats it becomes marked as unavailable by the others. Usually at this point is when the monitors started paging me about strange values in the diagnostics. Reducing the number of nodes helped.

More resources means more nodes. We had two clusters with about 22 nodes (44 total) each when we hit a major performance wall. They were split into four clusters with 15 nodes each (60 total). Eventually these grew to over 22 nodes each again. At this point upgrading was out of the question. A complete overhaul with all new databases and web servers meant we could do whatever we wished.

The ideal plan was a cluster per client. Licenses being so expensive scrapped that plan.

Ten clusters with 13 managed nodes each was a reasonable compromise. More nodes while also using smaller clusters achieved both needs well. Empty databases also gave us a better restarting point. The databases still have grown to the point certain transactions run slowly just for 4 terms later. (I was hoping for 6.) Surviving the next two years will be a challenge to say the least. I wish we got bonuses for averting disasters.