We were a week from the grand opening and nothing was working. An oscilloscope only showed the more nodes we attached, the noisier the signal. One node looked sort-of OK, but at our full complement of devices it was like looking at zebras in tall grass. We thought it was interference from a nearby 50KW AM transmitter tower but none of the RF countermeasures we tried had any effect. RF chokes on every wire. We replaced every wire in the entire loop system with plenum rated shielded 2 pair (the most expensive wire we could have used.) We had the electricians rewire every device outlet to use an isolated ground. All the money, and nothing.
At precisely 1AM I was watching the scope and the extremely noisy waveform I was looking at suddenly turned into a picture perfect square wave. I excitedly radioed the IBM guy out working on the wires and asked “what did you do?” He said he had done nothing, but noted the building had gone silent as the air conditioning had just shut off. After talking to the energy management guy at 3AM, who told us it was a scheduled nightly 15 minute shutdown, we discovered the electricians had omitted the filters on the motors because they were too expensive (~30K$) and not needed.
Filters were installed the next day and we opened on time.
