Early Sunday morning we had the same alphabet soup of network problems ( DHCP, LDAP, DNS)  that we had last week.  We were not aware of the problem until late Sunday afternoon.  We had everything up and working by dinner time. 

 

However, we have had 2 more episodes of bad alphabet soup today, one this morning around 11 and one as I'm writing this to you. Both of those down times were less than 5 minutes because we happen to be on the scene.

 

Needless to say no one in ITS land or the College is happy about this.  One thing that we have done is to put in some different paging to let us know when the alphabet soup is on the stove and ready to boil over.  This is a band aid solution and not a permanent fix, but at least we will be responding in a more timely manner.  That means less down time, not no down time.  We know the new paging is working, because we got the page this afternoon when Prod server again made bad alphabet soup.

 

As Julia Child said, "The main thing is to have a gusty approach and use your head."  That is what we are doing. Our next attempt to resolve this problem is to move some of the alphabet soup to a different pot.  This will take some time, and hopefully by mid-week we will have this permanently resolved and a wonderfully stable alphabet soup. 

 

Bon Appetite!