Replacing the fans on a Poweredge 2800

Fans of a PowerEdge 2800

The fans of my PowerEdge 2800. Front the cpu fans (stock Delta left, replacement everflow right); at the back the 120mm case fans (left the Artic, right the stock Nidec)

My “new” PowerEdge was way too loud. Like, jumbo-loud. I’ve chosen to replace all the fans with quieter models, but oh boy, this turned out to be a major pain! As always, I didn’t follow the most important rule when tinkering: do stuff incrementally, and test after each step. Well, maybe I’ll learn someday.

For reference, here the fan models:

Funtion Dell Replacement
4x Memory/Disk, 120x120x32mm, PWM Nidec Beta V TA350DC, M34789-35 Arctic F9 PWM
2x PSU, 60x60x25mm, Tacho Nidec Beta V TA225DC, B34605-33 Akasa AK-192BKT-B
2x CPU, 60x60x35mm PWM Delta AFB0612EHE Everflow F126025BU

I’ve changed all the fans in one go and ended up with a system that didn’t boot anymore. It didn’t even get to the end of the BIOS initialization, it shut down directly. I gathered that it must be the power supply’s fans (else I would have had a warning message on the POST). I’ve checked and I realized that I bought some fans with a build-in thermal control; a sensor that slows down the fan rotation. Well, as I had them already I hacked them up to ignore the sensor (cracked open sensor casing, soldered two pins together; no resistance acts as if the sensor measures a very high temperature).

Connector for PowerEdge 2800 fans

The connector with the soldered cables of the everflow fan. Here, the yellow and blue cable on the connector has been switched.

Next problem was that all fans spun at full speed. Impressively noisy, even for a seasoned cluster admin as me (I could hear the vibration 2 floors below). The reason were swapped PWM and Tacho pins (as I’ve found out after trying for a night to install Dell OpenManage 6.4 on Ubuntu 11.04 64bit). I don’t know why, but I needed different PIN configurations that every other source I’ve found on the Internet. Here’s what I’ve used in the end (This is looking on the bottom of the connector):

+---+---+
| 2 | 1 |-+
+---+---+ |
| 4 | 3 |-+
+---+---+
Pin Funtion Dell Everflow Arctic
1 VCC red yellow red
2 Control/PWM blue blue blue
3 Sense/Tacho yellow green yellow
4 Ground black black black

So everything is exactly as in the datasheets, except the swapped Control and Sense pins. After I’ve swapped them, the server became quite quiet and I quite happy. :D Actually, my desktop is now more noisy. Argh! BTW, thanks a lot to Brent Ozar for his blog entry on making a power edge quieter.

Update

Now I seem to have another problem: as reported by Brent Ozar and others

One problem shown above is that sometimes fans spin slow enough that they trigger Dell’s thresholds for slow-moving fans. Gotta figure out how to fix that for good one of these days.

Damn. Now I either have to find a way to make the fans a little bit faster or to change the threshold. See the follow-up post for more info on the threshold problem.

Update 2

I finally managed to adjust the critical fan thresholds by patching the BMC firmware! Here’s the howto. Additionally, I created a project page for my server.

45 thoughts on “Replacing the fans on a Poweredge 2800

  1. Pingback: These darn fans! | tüfteln

  2. Pingback: How to adjust the fan thresholds of a Dell PowerEdge | tüfteln

  3. Hi,

    I have just purchased a Dell PowerEdge 2950 and have the same noise problem.
    Are the fans on 2950 same as the ones on 2800?
    I see that the CFM of the new fans is less than the original fans. Does this cause any issues? Doesn’t this mean that the new fans are pushing our lesser air than the original fans?
    Any advice on how to go about making this change will be helpful.

    Thanks,

    Ajay.

    • Ajay,

      I don’t know if the 2950 has the same fans as the 2800. You’ll have to take yours out and check – it’s quite easy as Dell put them in hot-pluggable fan trays. I am sure that the fans are not exactly the same as it’s a generation 9 PowerEdge and a rack version. Most probably you have 6-8 fans of the same size, while the 2800 has 3 different sizes…

      You are right: the fans push out lesser air. Server fans are usually designed to provide adequate cooling in all conditions the system is designed for. That includes also in a hot server room, with all processor/memory/harddisk slots used and at a 100% load. My environment is quite less demanding, so I decided that it’s ok to use fans that shovel a lower volume of air. I am fairly certain that in the extreme environment described above, my system will perish, but I am also certain that my system will never be used under these conditions. :)

      In the end it’s a judgment call and some experimentation. In the comments of this post http://projects.nuschkys.net/2011/11/15/how-to-adjust-the-fan-thresholds-of-a-dell-poweredge/ somebody reported a problem with the memory temperature. I haven’t checked mine, but for example disks and processor are absolutely fine.

      BTW, in that thread there’s also someone that tested a 2950, you could ask them.

      Hope that helps,
      Arnuschky

  4. Hi – I’m having trouble accessing your page on the Dell PE-2800 fan replacement? I changed two fans in one of my powersupplies (sadly only boots for 10secs with that one before turning off so I got something wrong there!) and I also changed the 4 120mm fans, ram and harddrive coolers, I’m pretty sure that I had copied your wiring diagram (that I can’t find now) but the ipmi in ubuntu reckons that the fans are at 0rpm, so of course everything runs at 100%, namely those rather ferocious cpu cooling fans. I’ve not been able to source the recommended replacements for those in the UK yet, so I’m trying to find a plan B. For starters, I’d like to get it running at a more sensible rate, if you could share the wiring for the Arctic F9 PWM fans, I can confirm if I have made a mistake and hopefully sort it as the bmc has been flashed with the 50% rpm thresholds, I’m a little loathed to reduce them to 0% just to get the noise down!

    Many thanks

    • Thanks for letting me know about the page – I fixed it as you can see. :)

      Regarding the fans in the power supply: I had exactly the same problem. The fans in the power supply are not PWM-controlled, but they need to have a tacho signal (3 wires). The power supply checks if it can read the tacho signal and if the fans are turning fast enough. If this is not the case, the system switches off to prevent overheating. I solved this by putting in fans which are turning not too slow.

      Regarding the wiring of the Arctic fans, see above. If you can’t get it right (but the fan turns), just try to swap the PWM and the tacho signal cable.

  5. Great, it worked. So for the latest Arctic F9 PWM fans, the blue and the yellow must be connected to each other instead of connected to the blue and yellow respectively. Time to get soldiering!

  6. Thanks for your help so far! Unfortunately now the system keeps gradually slowing down the fans, so I sit refreshing ipmi-sensors and it gets some of these new fans down as far as 525rpm, with continual refreshes of ipmi until it then has a fit and goes back up to 100% again. I set the threshold as 450rpm, do you think its dipping below and I need to drop it down a little more?

    • That was exactly the case with my system. My current threshold is 150, and my fans turn at around 900 RPM with a system load of ~1.5.

      I subsequently lowered the threshold, until I understood that the system in the end understands only two states: fan working (RPM > threshold) and fan broken (RPM < smaller threshold). Under the assumption that a fan which is really broken does not turn at all, you can set a very low threshold without losing anything.

    • Hi all, I too had this problem and fixed it by adding the two original missing fans 1 and 2 to the second CPU. My PE 2800 came with Fan 3 & 4 but not the 1 & 2.
      Let me know if I helped anyone.
      Cheers

  7. Hi arnuschky,

    Do you think I could mod the fans to 120mm PWM -versions? For my 2900 -server I could replace the first three fans with 120mm -ones and the other 3 with the usual 92mm ones.

    • No, I don’t think so. Servers are modular but not flexible, I doubt that you would have the space for 120mm fans. Nevertheless, I think power- and control-wise there shouldn’t be a problem, that is, if you can cram them in, why not? :)

      • There”s plenty of space in a PE2900 between the cpu coolers and the SATA!-backplane, if the fan cage is removed. Follow up question could the thresholds be set differently for the three fans in the front?

      • Also one more question.. At what speed (RPM) do your PSU fans run at? The Akasa one’s don’t push too much air compared to the Delta Eletrcic ones on my PE2900.. If they’d run @ 100% all the time, it maybe would be enough.. Also it looks like the PE2800 PSU has two fans per PSU, My 930W PSU Beast only has one fan…

      • Just one more question still.. What kind of fans do the PSU’s accept? I understand they must have this “tacho”? How can I now if they have a tacho?

        Thanks a ton for every bit of info.. Where’s your donate -button btw? :D

          • Arnuschky, I hope you are still following this comment thread!!
            I am currently using your server mod page to make my own 2900gen3 run quieter, but I noticed your cpu fans are exactly my psu fans. In fact mine are 40x40x38 Delta AF0612EHE pwm and I replaced them with a Evercool F126025BU pwm also.

            I assumed the wiring switch over would be the same in the psu and I connected my new fan green to dell’s yellow and my fan’s yellow wire to the dell’s red.

            All good until I power up the system…fan spins corectly, blows some air and then psu shuts down.
            Is this a thermal controlled fan as well? I thought not.
            But now I don’t know how to wire it so it runs quieter.

            Please help.
            Thank you!

          • The PSU shuts down if the fan speed is not high enough. The PSU reads the fan speed using the tacho signal. Two possibilities: Either the fans are too slow and the PSU thinks that they are dead, or you swapped the PWM and tacho pins. Try changing the two pins, if it doesn’t work if means that you cannot use these fans for your PSU.

  8. hi, i’ve just recieved two Nidec fans – a spare part for Dell PE 2800… the same problem with pwm, when connectig them to “consumer”/non-Dell mb. Fans made for Dell (i don’t know if all) has swiched colours blue=tacho, yellow=PWM.

  9. Thanks so much for posting all your good work on quieting down Dell servers! I have just finished a fan re-fit on my PowerEdge 2950 (1st edition). The BMC firmware patch went flawlessly, but I was unable to get the 32-bit Linux flash utility to work. Ended up installing the 64-bit tools from Dell’s YUM repository, copying the patched flash image over the one it brought down and using the GUI to apply it.

    FYI: Although your suggested Everflow fans did eventually work out, they are not the right form-factor for the mounting frames in the 2950. I had to do a fair amount of modification with a Dremel tool to make them fit. In particular, the 2950 mounting scheme requires a fan without the screw mount “tunnels” on the corners. After cutting them away to leave only the flange and cutting a relief for the connector, all was well. Your advice on color-coding was very useful! Turns out that Delta’s spec sheet for the OEM fan has the tach and pwm leads reversed.

    I now have a nice, quiet server that doesn’t rattle the fillings out of my teeth!

  10. UPDATE: just reading through your comments above, I suspect your 2800 has PSU fans with 3 wires? My 2900 psu fan has 4wire fans therefore I assume they are PWM since they spin up every 5 minutes along with the case fans.
    The PSU fan has a 4 wire Yellow/Blue/Black/Red while the replacement EVERFLOW (sorry I think I typed Evercool earlier- it is EVERFLOW) F126025BU has 4 wires as well but with a green instead: Bue/Black/Green/Yellow

  11. Awesome, the sucker worked with the pwm and tach switched!
    Not a whole difference in noise, but it worked :)
    Thank you so much for the quick reply and help!

    Now it is time to lookup some friends that can do the BMC reflash for me :)

  12. UPDATE: oh, now it is super quiet, the reason for no difference was that I had the other PSU still plugged and that one is still on the original fan. But now that I unplugged it, it works super silent.

  13. I`m testing some fans without pwm and it still works (PSU not changed). So im a little bit confused about. Why should i use fans with pwm if some just with tacho works fine?
    Sorry for those many questions ;)

  14. Brilliant! PE2900… 2.50 BMC, redhat 7.3 running on the PE2900 under Hyper-V to run your python script, option #1, set all 6 fans to 900 rpm, flashed the BMC under Windows Server 2012 R2, all six arctic F9′s are spinning at 1200 rpm, very tolerable light fan hum, I think primarily from my 950 Watt dual PSU’s and my temperatures look good.

    PE2900 with 48 GB of ram running in low power mode to reduce heat hottest 84 c coolest 64 c. Dual Xeon’s E5405 hottest core at idle 56 c coolest 37 c. Running eight 1 TB 7200 rpm SATA in raid 6 under the PERC 6i.

    All good. Crazy happy as I can use my office phone now!

  15. *best tips:
    1. for bmc firmware flash setup. set ALL low thresholds to ZERO before fan replacement.
    2. if using artic f9′s cut pwm wire (fans run at full then) or use different higher speed fans eg…http://www.scan.co.uk/products/92mm-asetek-vapochill-extreme-micro-pwm-fan-black-3800rpm-(dc-brushless)-oem. The reason for this is that dell in their infinite wisdom supply about 30% of max power to fans when idleing. Hence my F9′s 600-1800rpm were running at about 250rpm when idle(not moving enuff air). That is the reason I changed to a higher spec fan rated 3800rpm idles at about 900rpm , so its pushing a bit more air.
    3. For psu fans I applied the same philosophy I used in tip two. I used top motor df126025bm max 7000rpm which replaced originals with a max of 11000 rpm (real screamers) color code below shows how sneaky dell have been. Tried original color code setup for wiring and machine would not get past post. switched tacho and pwm booted fine and quiet.

    the section below is only for psu on dell 2900 gen3 with 4 wire connector. For the internal fans the standard wire connections apply….

    can confirm dell 2900 gen 3 psu fan replacement colour code.

    dell connector in psu = black GND , red +12v , blue SENSE(tacho) , yellow CONTROL(pwm)

    … normally yellow and blue are the other way round(yellow tach, blue pwm) sneaky

    So now all I have to do is get some idle fan speed readings and re-flash BMC with settings of lower threshold < idle fan speed and all should be peachy . hope this helps.

  16. hi
    just a minor quibble about connector image and legend ……….

    its wrong

    example, on connector image red is diagonally opposite black
    in legend dell red 1 is diagonally opposite dell 4 yellow

    solution …. swap 3 and 4

  17. Can I change PSU fans without changing thresholds? I changed my fans for one PSU on my 2800 and the system would not start – PSU’s yellow LED turns on, fans even would not spin at all.
    I double checked pins, red for positive, black for neutral and yellow for tacho.

    • You should be able to. Had the same problem, yellow light means that one of the fans has a tacho signal that is too low. I suspect the PSU has some fixed internal threshold. Recheck your wiring, and check the fans. If all ok, you’ll have to use different fans I’m afraid.

      (Disclaimer: I am not 100% sure about this anymore, as I did this 2 years ago..)

      • Thats’s strange, because i tried the same Akasa’s you had. Is there any way to read tacho signal without PC?

        • Weird. I actually don’t remember what I did with the PSU fans. There was a problem, that much I do remember. Maybe I encountered the same problem you had and left in the original fans??

  18. One more question! :)

    Did I need to change thresholds or I just needed to accept patch? I just accepted patch and now system increases speed of fans to maximum, then drops down, then increases and so on and so on. Within about few minutes cycle.

    My system shows me:

    ID | Name | Type | Reading | Units | Event
    1 | Temp | Temperature | N/A | C | N/A
    2 | Temp | Temperature | N/A | C | N/A
    3 | Ambient Temp | Temperature | N/A | C | N/A
    4 | Planar Temp | Temperature | N/A | C | N/A
    5 | Riser Temp | Temperature | N/A | C | N/A
    6 | Temp | Temperature | N/A | C | N/A
    7 | Temp | Temperature | N/A | C | N/A
    8 | Temp | Temperature | 37.00 | C | ‘OK’
    9 | Temp | Temperature | N/A | C | N/A
    10 | Ambient Temp | Temperature | 26.00 | C | ‘OK’
    11 | Planar Temp | Temperature | 30.00 | C | ‘OK’
    12 | Riser Temp | Temperature | 32.00 | C | ‘OK’
    13 | Temp | Temperature | N/A | C | N/A
    14 | Temp | Temperature | N/A | C | N/A
    15 | CMOS Battery | Voltage | N/A | V | N/A
    16 | ROMB Battery | Voltage | N/A | N/A | ‘State Dea sserted’
    17 | VCORE | Voltage | N/A | N/A | ‘State Dea sserted’
    18 | VCORE | Voltage | N/A | N/A | N/A
    19 | PROC VTT | Voltage | N/A | N/A | ‘State Dea sserted’
    20 | 1.5V PG | Voltage | N/A | N/A | ‘State Dea sserted’
    21 | 1.8V PG | Voltage | N/A | N/A | ‘State Dea sserted’
    22 | 3.3V PG | Voltage | N/A | N/A | ‘State Dea sserted’
    23 | 5V PG | Voltage | N/A | N/A | ‘State Dea sserted’
    24 | 5V Riser PG | Voltage | N/A | N/A | N/A
    25 | Riser PG | Voltage | N/A | N/A | ‘State Deasserted’
    26 | CMOS Battery | Voltage | 3.02 | V | ‘OK’
    27 | Presence | Entity Presence | N/A | N/A | ‘Entity Present’
    28 | Presence | Entity Presence | N/A | N/A | ‘Entity Absent’
    29 | Presence | Entity Presence | N/A | N/A | ‘Entity Present’
    30 | Presence | Entity Presence | N/A | N/A | ‘Entity Present’
    31 | ROMB Presence | Entity Presence | N/A | N/A | ‘Entity Present’
    32 | FAN 1 RPM | Fan | N/A | RPM | N/A
    33 | FAN 2 RPM | Fan | N/A | RPM | N/A
    34 | FAN 3 RPM | Fan | N/A | RPM | N/A
    35 | FAN 4 RPM | Fan | N/A | RPM | N/A
    36 | FAN 5 RPM | Fan | N/A | RPM | N/A
    37 | FAN 6 RPM | Fan | N/A | RPM | N/A
    38 | FAN 1 RPM | Fan | N/A | RPM | N/A
    39 | FAN 2 RPM | Fan | N/A | RPM | N/A
    40 | FAN 3 RPM | Fan | 4125.00 | RPM | ‘OK’
    41 | FAN 4 RPM | Fan | 5025.00 | RPM | ‘OK’
    42 | FAN 5 RPM | Fan | 1200.00 | RPM | ‘OK’
    43 | FAN 6 RPM | Fan | 1350.00 | RPM | ‘OK’
    44 | FAN 7 RPM | Fan | 1125.00 | RPM | ‘OK’
    45 | FAN 8 RPM | Fan | 1200.00 | RPM | ‘OK’
    46 | Status | Processor | N/A | N/A | ‘Processor Presence detected’
    47 | Status | Processor | N/A | N/A | N/A
    48 | Status | Power Supply | N/A | N/A | ‘Presence detected’
    49 | Status | Power Supply | N/A | N/A | ‘Presence detected’ ‘Power Supply Failure detected’ ‘Predictive Failure’ ‘Power Supply input lost (AC/DC)’
    50 | VRM | Power Supply | N/A | N/A | ‘Presence detected’
    51 | VRM | Power Supply | N/A | N/A | ‘Presence detected’
    52 | OS Watchdog | Watchdog 2 | N/A | N/A | ‘OK’
    53 | SEL | Event Logging Disabled | N/A | N/A | N/A
    54 | Intrusion | Physical Security | N/A | N/A | ‘General Chassis Intrusion’
    55 | PS Redundancy | Power Supply | N/A | N/A | ‘Redundancy Lost’
    56 | Fan Redundancy | Fan | N/A | N/A | ‘Fully Redundant’
    73 | SCSI Connector A | Cable/Interconnect | N/A | N/A | N/A
    74 | SCSI Connector B | Cable/Interconnect | N/A | N/A | N/A
    75 | SCSI Connector A | Cable/Interconnect | N/A | N/A | N/A
    76 | Drive | Slot/Connector | N/A | N/A | N/A
    77 | Drive | Slot/Connector | N/A | N/A | N/A
    78 | 1×2 Drive | Slot/Connector | N/A | N/A | N/A
    79 | Secondary | Module/Board | N/A | N/A | N/A
    80 | ECC Corr Err | Memory | N/A | N/A | N/A
    81 | ECC Uncorr Err | Memory | N/A | N/A | N/A
    82 | I/O Channel Chk | Critical Interrupt | N/A | N/A | N/A
    83 | PCI Parity Err | Critical Interrupt | N/A | N/A | N/A
    84 | PCI System Err | Critical Interrupt | N/A | N/A | N/A
    85 | SBE Log Disabled | Event Logging Disabled | N/A | N/A | N/A
    86 | Logging Disabled | Event Logging Disabled | N/A | N/A | N/A
    87 | Unknown | System Event | N/A | N/A | N/A
    88 | PROC Protocol | Processor | N/A | N/A | N/A
    89 | PROC Bus PERR | Processor | N/A | N/A | N/A
    90 | PROC Init Err | Processor | N/A | N/A | N/A
    91 | PROC Machine Chk | Processor | N/A | N/A | N/A
    92 | Memory Spared | Memory | N/A | N/A | N/A
    93 | Memory Mirrored | Memory | N/A | N/A | N/A
    94 | Memory RAID | Memory | N/A | N/A | N/A
    95 | Memory Added | Memory | N/A | N/A | N/A
    96 | Memory Removed | Memory | N/A | N/A | N/A
    97 | PCIE Fatal Err | Critical Interrupt | N/A | N/A | N/A
    98 | Chipset Err | Critical Interrupt | N/A | N/A | N/A
    99 | Err Reg Pointer | OEM Reserved | N/A | N/A | N/A

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>