The battle againt the BMC – Part 1

Earlier, I wrote about the problem of the noisy fans in my Dell PowerEdge 2800. Since then, I investigated a bit more. Just as a reminder: I can’t run silent fans because they have a lower RPM than a hardcoded panic-threshold of the PowerEdge. *grrr*

Brainstorm:

  1. make the fans faster/buy faster fans
  2. make the system hotter
  3. hack the fans into reporting more RPM than they actually do
    1. hack the fans themselves
    2. alter their tacho signal
  4. hack the BMC
  5. find out how the OEM sets these thresholds

Well, as I mentioned earlier 1. and 2. are for obvious reasons dissatisfactory.

Couple of fans taken apart

A couple of fans taken apart. Notice the blob of brown paint on the ring magnet of the fan on the right.

Concerning 3A, I took a couple of fans apart, looking at how they create the tacho signal. Almost all fans I opened (luckily I have a whole stack of noisy, throw-away fans lying around) have a sensor sitting just under the ring magnet, which is part of the rotor as can be seen in the photo on the right. I had no idea what this sensor might be, but I noticed on all of the fans one or more blows of brownish-red paint. I figured that this paint might be used to create a signal for the sensor – and I did some tests with other paint in order to replicate the effect (left fan on the photo with magnetic paint applied). Well, nice idea, but total bullshit as it turns out. The sensor is a hall-sensor that senses the change in the magnetic field of the ring-magnet, and thus changes inevitable 2 times per revolution. I figure that the paint applied on the rotor is used for calibrating the fans… Well, it was a nice idea.

3B might be an option, but it would require either a microcontroller or some analogue circuit – not really what I want to fiddle into the fan trays of 6 fans.

Concerning option 5, I thought that there may be hidden ipmi OEM commands for configuring the thresholds. I dug around the Dell OEM extensions for ipmitool (can be retrieved from the Dell Linux Community Repositories). This code officially earned worst code of the year – I completely understand why the ipmitool maintianers flatout refuse to integrate that piece of crap. It’s a hacked-up collection of extensions, seemingly done on the fly to fix customer problems. Horrible. Even more so as it does not seem to be able to set the thresholds either. After a few hours of digging in the code I managed to query the BMC sensors with Dell’s OEM commands, and the returned capability flags do indicate that the thesholds cannot be changed. Darn.

Now I am back to hacking the BMC firmware – but that’s for another post…

Update

I finally managed to adjust the critical fan thresholds by patching the BMC firmware! Here’s the howto. Additionally, I created a project page for my server.

4 thoughts on “The battle againt the BMC – Part 1

    • Not much. I had other stuff to fix first (like my car that go its door destroyed by some bastard). Anyways, I progressed a bit in hacking the BMC: I narrowed the search down to a couple of files, and I discovered what the naming conventions are – but in the end, I am looking for 8 values in a couple of thousand bytes. I don’t know if I will ever get there.

      Another approach would be to design a hardware filter: a microcontroller that monitors the fans itself and forwards useful/filtered information to the BMC. Pros: could be used by other people, and not only on a pe2800. Cons: cabling, more hardware, cost. But on the other hand: if someone is willing/able to take out the soldering iron in order to switch fans on his PowerEdge, installing such a filter would be feasible I guess.

  1. Pingback: How-to adjust the fan thresholds of a Dell PowerEdge | tüfteln

Leave a Reply

Your email address will not be published. Required fields are marked *