Moving code to GitHub

GitHub is a web-based hosting service for software development projects that use the Git revision control system.

I am slowly moving most of my projects to GitHub. GitHub is great because people can easily collaborate, fork, submit issues and patches and so on. Additionally, stuff gets shared in an open and unified way, making it much easier to others to find and reuse existing code.

I started with the firmware hack of the Dell PowerEdge BMC:
https://github.com/arnuschky/dell-bmc-firmware.git.

See here for the post explaining what it’s for.

How to adjust the fan thresholds of a Dell PowerEdge

Adjusted lower critical thresholds for the fans of a PowerEdge 2800

Note: you MUST change your fans against slower, quieter ones to reduce the noise. The threshold adjustment discussed in this article only allows you to do so – without new fans, it’s useless!

Intro

In order to swap the fans on a Dell PowerEdge with slower, more quiet ones you have to adjust the lower critical threshold (LCR). If you don’t, the server’s firmware actually lowers the fan’s speed under it’s own LCR, panics, spins them back up a 100%, lowers them again etc. Very noisy, very annoying.

Previous, related posts:

This behavior is controlled by the BMC, an embedded management controller. You can configure many parameters of the BMC using the IPMI protocol. Unfortunately, the BMC’s firmware of a Dell PowerEdge does not allow to change the thresholds mentioned above. I contacted Dell support, and they refused to change the thresholds for such an old server.

So I had no choice but to change them myself. It took me quite a while to isolate the proper setting in the BMC’s firmware, the checksums etc. But I managed, and the server’s running now very quiet with adjusted thresholds.

Below, I explain how to adjust these thresholds with a python script I wrote. Note that you’ll need Python 2.6 in order to run the script. In case someone is interested I can also write up how I did it, but this is for another post.

Update: I created a project page for my server.

The result

First, here’s the result: my PowerEdge 2800 with swapped fans and patched fan thresholds.

This has been recorded with my laptop, 10cm/4in in front of the server. The system is now more silent than my desktop!

Prerequisites

I assume that you have a sufficiently recent Linux distribution up and running, with python installed and IPMI set up. If you don’t, have a look at this article that explains how to get a recent Ubuntu version running (without installing anything on your harddisk!).

Adjusting the fan thresholds

I assume that you have now FreeIPMI installed, the BMC configured and that you can query the BMC using IPMI.

  1. Query the sensors
    First, you have to query the sensors of your server using IPMI. The output should look a bit like this:

    you@server$ ipmi-sensors
    1: Temp (Temperature): NA (NA/125.00): [NA]
    2: Temp (Temperature): NA (NA/125.00): [NA]
    3: Ambient Temp (Temperature): NA (3.00/47.00): [NA]
    4: Planar Temp (Temperature): NA (3.00/72.00): [NA]
    5: Riser Temp (Temperature): NA (3.00/62.00): [NA]
    6: Temp (Temperature): NA (NA/NA): [NA]
    7: Temp (Temperature): NA (NA/NA): [NA]
    8: Temp (Temperature): 71.00 C (NA/125.00): [OK]
    9: Temp (Temperature): NA (NA/125.00): [NA]
    10: Ambient Temp (Temperature): 27.00 C (3.00/47.00): [OK]
    11: Planar Temp (Temperature): 46.00 C (3.00/72.00): [OK]
    12: Riser Temp (Temperature): 50.00 C (3.00/62.00): [OK]
    13: Temp (Temperature): NA (NA/NA): [NA]
    14: Temp (Temperature): NA (NA/NA): [NA]
    15: CMOS Battery (Voltage): NA (2.64/NA): [NA]
    16: ROMB Battery (Voltage): [NA]
    17: VCORE (Voltage): [State Deasserted]
    18: VCORE (Voltage): [NA]
    19: PROC VTT (Voltage): [State Deasserted]
    20: 1.5V PG (Voltage): [State Deasserted]
    21: 1.8V PG (Voltage): [State Deasserted]
    22: 3.3V PG (Voltage): [State Deasserted]
    23: 5V PG (Voltage): [State Deasserted]
    24: 5V Riser PG (Voltage): [State Deasserted]
    25: Riser PG (Voltage): [State Deasserted]
    26: CMOS Battery (Voltage): 3.11 V (2.64/NA): [OK]
    27: Presence  (Entity Presence): [Entity Present]
    28: Presence  (Entity Presence): [Entity Absent]
    29: Presence  (Entity Presence): [Entity Present]
    30: Presence  (Entity Presence): [Entity Absent]
    31: ROMB Presence (Entity Presence): [Entity Present]
    32: FAN 1 RPM (Fan): NA (1575.00/NA): [NA]
    33: FAN 2 RPM (Fan): NA (1575.00/NA): [NA]
    34: FAN 3 RPM (Fan): NA (1575.00/NA): [NA]
    35: FAN 4 RPM (Fan): NA (1575.00/NA): [NA]
    36: FAN 5 RPM (Fan): NA (1575.00/NA): [NA]
    37: FAN 6 RPM (Fan): NA (1575.00/NA): [NA]
    38: FAN 1 RPM (Fan): NA (2025.00/NA): [NA]
    39: FAN 2 RPM (Fan): NA (2025.00/NA): [NA]
    40: FAN 3 RPM (Fan): 4875.00 RPM (2025.00/NA): [OK]
    41: FAN 4 RPM (Fan): 4800.00 RPM (2025.00/NA): [OK]
    42: FAN 5 RPM (Fan): 1800.00 RPM (900.00/NA): [OK]
    43: FAN 6 RPM (Fan): 1950.00 RPM (900.00/NA): [OK]
    44: FAN 7 RPM (Fan): 1875.00 RPM (900.00/NA): [OK]
    45: FAN 8 RPM (Fan): 1875.00 RPM (900.00/NA): [OK]
    46: Status  (Processor): [Processor Presence detected]
    47: Status  (Processor): [NA]
    48: Status  (Power Supply): [Presence detected]
    49: Status  (Power Supply): [NA]
    50: VRM  (Power Supply): [Presence detected]
    51: VRM  (Power Supply): [Presence detected]
    52: OS Watchdog (Watchdog 2): [OK]
    53: SEL (Event Logging Disabled): [Unknown]
    54: Intrusion (Physical Security): [OK]
    55: PS Redundancy (Power Supply): [NA]
    56: Fan Redundancy (Fan): [Fully Redundant]
    73: SCSI Connector A (Cable/Interconnect): [NA]
    74: SCSI Connector B (Cable/Interconnect): [NA]
    75: SCSI Connector A (Cable/Interconnect): [NA]
    76: Drive (Slot/Connector): [NA]
    77: Drive (Slot/Connector): [NA]
    78: 1x2 Drive (Slot/Connector): [NA]
    79: Secondary (Module/Board): [NA]
    80: ECC Corr Err (Memory): [Unknown]
    81: ECC Uncorr Err (Memory): [Unknown]
    82: I/O Channel Chk (Critical Interrupt): [Unknown]
    83: PCI Parity Err (Critical Interrupt): [Unknown]
    84: PCI System Err (Critical Interrupt): [Unknown]
    85: SBE Log Disabled (Event Logging Disabled): [Unknown]
    86: Logging Disabled (Event Logging Disabled): [Unknown]
    87: Unknown (System Event): [Unknown]
    88: CPU Protocol Err (Processor): [Unknown]
    89: CPU Bus PERR (Processor): [Unknown]
    90: CPU Init Err (Processor): [Unknown]
    91: CPU Machine Chk (Processor): [Unknown]
    92: Memory Spared (Memory): [Unknown]
    93: Memory Mirrored (Memory): [Unknown]
    94: Memory RAID (Memory): [Unknown]
    95: Memory Added (Memory): [Unknown]
    96: Memory Removed (Memory): [Unknown]
    97: PCIE Fatal Err (Critical Interrupt): [Unknown]
    98: Chipset Err (Critical Interrupt): [Unknown]
    99: Err Reg Pointer (OEM Reserved): [Unknown]

    You have to note the part about the fans (d’oh). Record sensor numbers, fan names and thresholds (the value in brackets). You’ll need it later to identify your system.

  2. Download the latest BMC firmware
    Got to http://support.dell.com/support/downloads/ and get the latest BMC firmware for your system. Select any Linux OS; the BMC firmware should be listed under something like Embedded Server Management. On the download page, select the .BIN package. In my case the file was called BMC_FRMW_LX_R223079.BIN. Download it!

  3. Fix and extract .BIN package
    In my case the .BIN package did not properly work. I had to fix it first, and then extract it. For this, open a terminal and go to the folder you’ve downloaded the package to.

    Then execute:

    you@server$ sed -i 's/#!\/bin\/sh/#!\/bin\/bash/' BMC_FRMW_LX_R223079.BIN  # fix interpreter bug
    you@server$ chmod 755 BMC_FRMW_LX_R223079.BIN                              # make executable
    you@server$ sudo mkdir bmc_firmware                                        # create dir as root
    you@server$ sudo ./BMC_FRMW_LX_R223079.BIN --extract bmc_firmware          # yes, you have to do this as root! :(
    you@server$ cd bmc_firmware

    This should extract your firmware. Check that you have a file called extracted/payload/bmcflsh.dat. If not, game over, your system isn’t compatible. If yes, yay!

  4. Patch firmware
    Next, download the program I wrote for patching the firmware. Then, use the program on the firmware as shown below:

    you@server$ wget https://raw.github.com/arnuschky/dell-bmc-firmware/master/adjust-fan-thresholds/dell-adjust-fan-thresholds.py
    you@server$ chmod 755 dell-adjust-fan-thresholds.py
    you@server$ ./dell-adjust-fan-thresholds.py payload/bmcflsh.dat

    The program is a python (version >= 2.6) script, that first lets you choose a system from the ones available in the firmware and the adjust the fan thresholds of this system. Yes, there can be support for multiple systems in a single firmware. You recorded the fan values before? Now you know why: you have to use it to identify your system from the ones the script shows to you. Just use the number of fans, their names and thresholds to identify your system. Maybe you’re lucky and the system name has already been found and is directly displayed.

    In the next step you can select fans and change their threshold. Just remember that the result is a multiple of 75. Half the usual speed has proven to be a good value. I’ve never tested what happened if you set it to 0, but this would be quite stupid as you can’t detect broken fans.

    If the program display a code at the end and asks you to report back, please do so! That way we can identify the other systems using their code (for example, the code of a PowerEdge 2800 is “K_C”).

  5. Flash firmware
    Finally, flash the firmware like as shown below.

    Disclaimer: I am not responsible for any damage you do to your system! If you flash this firmware, you might render your PowerEdge server unusable. It might even be unrecoverable. Additionally, badly set thresholds might cause overheating.

    Additionally, use the usual caution when flashing (do not interrrupt power, do not flash other a network link, do not be stupid).

    you@server$ LD_LIBRARY_PATH=./hapi/opt/dell/dup/lib:$LD_LIBRARY_PATH ./bmcfl32l -i=payload/bmcflsh.dat -f

    Cross your fingers. The flasher should accept the firmware. If not and it complains about the CRC, something went wrong. Don’t worry if the fans speed up fully and go dead afterwards during the flash, that’s normal. The system should stabilize afterwards. There is not need to reboot.

  6. Check the sensors
    Check that everything is in order:

    you@server$ ipmi-sensors

    That’s it. Enjoy your silent PowerEdge!

Trivia

Some things that I learned while messing with the firmware:

  • There can be multiple systems per firmware
  • Generally it’s quite well engineered
  • I’ve found Dell’s default password root/calvin. What is the 444444 for?
  • Dell server systems seem to be named internally after cities. BER, LOND, OSLO etc are easy enough to guess. But what the hell is K_C??? (my system)
  • The firmware package is probably the most horrible over-engineered script I’ve ever met on Linux
  • Dell uses CRC-16 for checksum – two different algorithms in the same firmware!

Update 1: I created a project page for my server.

Update 2: I wrote this article that explains how to get a recent Ubuntu version running (without installing anything on your harddisk!). This is for all the Windows users out there!

Update 3: I moved the code of this project into a GitHub repository: http://projects.nuschkys.net/2012/04/06/how-to-get-ubuntu-live-running/ GitHub is great because people can easily collaborate, fork, submit issues and patches and so on.

Please don’t ask me basic Linux questions! Google is your friend. If you don’t know what you are doing, you shouldn’t be doing it as you might damage your server!

Python/psycopg2/PostgeSQL: script for bulk inserts using COPY with progress indicator

Woah, what a title. :) I needed a script for inserting bulk data into a PostgreSQL database. Actually, I had a script already, written in Perl, and it was so slow that I needed a better and faster replacement. As I am slowly replacing all my Bash/Perl scripts with Python-pedants I aimed at doing the same here.

I decided to use psycopg2 for a Python-PostgreSQL binding. The copy_from method proved to be very fast; exactly what I needed. BUT I also needed a progress indicator. And while I’ve found a some people out there looking for exactly the same thing, I couldn’t find a solution. So here’s my script for doing this:

#!/usr/bin/python
import psycopg2
import sys
import os

class ReadFileProgress:

  def __init__(self, filename):
    self.datafile = open(filename)
    self.totalRecords = 0
    self.totalBytes = os.stat(filename).st_size
    self.readBytes = 0

    # skip header line
    self.datafile.readline()
    # count records
    for i, l in enumerate(self.datafile):
      pass
    self.totalRecords = i + 1
    sys.stderr.write("Number of records: %d\n" % (self.totalRecords))
    # rewind
    self.datafile.seek(0)
    # skip header line
    self.datafile.readline()
    # start progress
    self.perc5 = self.totalBytes / 20.0
    self.perc5count = 0
    self.lastPerc5 = 0
    sys.stderr.write("Writing records: 0%")

  # count bytes and display progress while doing so
  def countBytes(self, size=0):
    self.readBytes += size
    if (self.readBytes - self.lastPerc5 >= self.perc5):
      self.lastPerc5 = self.readBytes

      if (int(self.readBytes / self.perc5) == 5):
        sys.stderr.write("25%")
      elif (int(self.readBytes / self.perc5) == 10):
        sys.stderr.write("50%")
      elif (int(self.readBytes / self.perc5) == 15):
        sys.stderr.write("75%")
      else:
        sys.stderr.write(".")

      sys.stderr.flush()

  def readline(self, size=None):
    countBytes(size)
    return self.datafile.readline(size)
 
  def read(self, size=None):
    self.countBytes(size)
    return self.datafile.read(size)

  def close(self):
    sys.stderr.write("100%\n")
    self.datafile.close()

def main():
  config = dict()
  config['tablename']="tablename"
  config['filename']="filename"
  config['connstring'] = "host='?' dbname='?' user='?' password='?'"
  config['droptable'] = False
  config['createtable'] = False
  config['rowsdef'] = "id serial PRIMARY KEY, number integer NOT NULL"
  config['filecolumns'] = ['id','number']

  try:
    # get a connection, if a connect cannot be made an exception will be raised here
    conn = psycopg2.connect(config['connstring'])
    # conn.cursor will return a cursor object, you can use this cursor to perform queries
    cursor = conn.cursor()

    # drop table if requested (and it exists)
    cursor.execute("SELECT * FROM information_schema.tables WHERE table_name=%s", (config['tablename'],))
    if (config['droptable'] and bool(cursor.rowcount)):
      cursor.execute("DROP TABLE "+config['tablename']+";")

    # create the table if requested
    if (config['createtable'] and not bool(cursor.rowcount)):
      cursor.execute("CREATE TABLE "+config['tablename']+" ("+config['rowsdef']+");")

    # create a fileprogress object and copy the data to the database
    datafile=ReadFileProgress(config['filename'])
    cursor.copy_from(file=datafile, table=config['tablename'], sep='\t', null='\N', size=8192, columns=config['filecolumns'])
    datafile.close()

    # commit and clsoe
    cursor.close()
    conn.commit()

    sys.stdout.write("Transaction finished successfully.\n")

  except:
    exceptionType, exceptionValue, exceptionTraceback = sys.exc_info()
    sys.exit("Database connection failed!\n ->%s" % (exceptionValue))


if __name__ == "__main__":
  sys.exit(main())

Basically, the progress counter is a wrapper around the file object. It simply outputs the percentage of bytes read. The main program is very straight forward. The script was written with an experiment-specific config file parsing which I do not include here. Thus, you have to find your own way to set the config variables in the beginning of the main program.

Mute sound on screen lock

My coworkers got annoyed that my music continued playing when I was away from my desk. As I am anyways locking the screen automatically after 5 minutes (if I forget to do it by keyboard shortcut), I thought I might use this event for muting as well.

Below is the script I came up with. It needs the inotify-tools (sudo apt-get install inotify-tools) to work. Thanks to Vermind on the ubuntuforums for figuring out the mute stuff!

The advantage of this script is that it works with the auto-lock of the screensaver as well as the pre-configured shortcuts for screen locking.

#!/bin/bash
################################################################################
# mute-on-lock.sh
# Mutes sound and pauses amarok when the screen gets locked
# Version 1.0
#
# written by Arnuschky, mute stuff by Vermind (ubuntu forums)
#
# http://ubuntuforums.org/showthread.php?t=1364597
################################################################################

########################################
# Settings
########################################

mute=1            # Mute the sound while locked. 0=do not mute.
amarok=1          # Pause/unpause amarok on lock
card=0            # Sound card. 0 == default, first card.
channel="Master"  # Which volume to control. Examples: Master, PCM, Headphone
step=3            # Fade step: The volume increment/decrement amount
interval=0.2      # Seconds between increments/decrements when fading


########################################
# Functions
########################################

# get current volume
function getvol {
    vol=$( ${m} sget ${channel} | awk '
$0 ~ "%" {
    if ($4 ~ "%") {
        print $3;
    } else if ($5 ~ "%") {
    print $4;
    exit 0;
    }
}'
)
}

# fades volume up or down
function fade {
    # get current volume
    getvol
    if [ "$1" == "down" ]; then
        # fade down:
        while [ ${vol} -gt 0 ]; do
            ${m} set ${channel} ${step}- >/dev/null
            let vol-=${step}
        sleep ${interval}
        done
        ${m} set ${channel} mute >/dev/null
   else
       # first unmute, then fade up
        ${m} set ${channel} unmute >/dev/null
        while [ ${orig} -gt ${vol} ]; do
            let vol+=${step}
            if [ ${vol} -gt ${orig} ]; then vol=${orig}; fi
            ${m} set ${channel} ${step}+ >/dev/null
            sleep ${interval}
        done
        ${m} set ${channel} ${orig} >/dev/null
   fi
}

########################################
# Startup
########################################

if [ "${mute}" == "1" ]; then
    # short mixer command with card
    m="amixer -c ${card}"
    # record original volume value
    getvol
    orig=$vol
fi

########################################
# Loop                                 #
########################################

# start dbus monitor
monitorfile=`mktemp`
dbus-monitor --session "type='signal',interface='org.gnome.ScreenSaver',member='ActiveChanged'" &> ${monitorfile} &
pid=$!

# loop on changes on the monitorfile
while inotifywait -e modify ${monitorfile} &> /dev/null; do
  # on file change, grab the last line and filter it
  lastline=`tail -n1 ${monitorfile}`
  echo ${lastline} | grep -e "boolean true" -e "boolean false" &> /dev/null
  if [ $? -ne 0 ]; then
    continue
  fi

  # grab the lock state and act on it
  lockstate=`echo ${lastline} | sed 's/^.*boolean \(.*\)/\1/'`
  if [ ${lockstate} == "true" ]; then
    if [ "${mute}" == "1" ]; then
      fade down
    fi
    if [ "${amarok}" == "1" ]; then
      amarok --pause
    fi
  elif [ ${lockstate} == "false" ]; then
    if [ "${amarok}" == "1" ]; then
      amarok --play
    fi
    if [ "${mute}" == "1" ]; then
      fade up
    fi
  fi
done

# kill monitor & cleanup
kill $pid
rm -f ${monitorfile}

exit 0

How to install this? Put it into e.g. /home/youruser/bin/mute-on-lock.sh. Then, go to System -> Preferences -> Startup Applications and add your script. Log in& out and you’re done!

Reflashing a semi-bricked Hitachi/LG XBox 360 dvd drive

This app will flash single sectors in the Hitachi/LG firmware using debug flash erase/program routines that exist in the 3120L firmware. The difference to SeventhSon’s or the usual flashers is that it utilizes code that isn’t located
in one of the sectors that are changed when applying a hacked firmware. So if something went wrong during flashing, there might be a wee chance that the debug code works and can still be used to fix your drive.

All of this is based on Kev’s/SeventhSon’s work. I simply put together two of his programs to create a flasher that uses the Hitachi debug commands to flash sectors in the drive’s flash. All credit goes to SeventhSon. I even copied this text from his site. :) Unfortunately, his site vanished, so there are none of his notes left.

Use at your own risk, this may break your 360, 360 DVD drive and/or PC if done improperly (or if I happen to have made any mistakes).

Note: This does not work in recovery mode. This tool only helps if your checksum was patched (reports ok), but flashing does not work due to overwritten upload/execute handlers. If you don’t know what I’m talking about, leave it be.

Warning: This app is an interim solution intended only for hackers who know what they are doing. It is very easy to kill your drive with this program. If you fail to update the firmware checksum before you power down the drive, you will break your drive. If you overwrite any of the upload and execution command handlers with broken code, you will break your drive. If you overwrite your flash entry point code, you will break your drive. If you overwrite the sector containing your AES key without backing up your key first, you will break your drive and will not be able to repair it. If you do not understand everything that I just said, then this app isn’t for you.

Download

Flashing sectors in the Hitachi/LG drive’s firmware from a PC

In the following examples encrypted_fw.bin is a full firmware dump. It must be encrypted. Encrypt it after you make your changes and before you run the flasher. There are tools somewhere on the internet that allows you to de/encrypt firmware images. So, encrypted_fw.bin would be created like so (in Windows),

C:\> memdump_win e 12200 8 8000 firmware.bin

(then modify your target sector within firmware.bin)

C:\> FirmCrypt e firmware.bin encrypted_fw.bin

(then flash the modified sector as per the examples below)

You will almost certainly break your drive if you do not encrypt the firmware image before flashing.

The next argument is the address of the target sector in the MN103’s address space in hexadecimal. Just add 0x90000000 to the sector’s offset into the firmware dump to get this value. The final argument is the sector size. The 3120L erase/program routines support a few devices, some appear to have 8KB sectors (0x2000 bytes), others 4KB sectors (0x1000 bytes). My drive has a SST39SF020A flash device (with 4KB sectors). I’m not sure if any Hitachi-LG drives exist with a different chip, but my app supports them just in case. Make sure you specifiy the sector size in hexadecimal.

Please. Don’t use this app if you don’t understand all of the above.

Linux example:

$ ./debug_flashsec /dev/sdb ./encrypted_fw.bin 9003F000 1000

Windows example:

C:\> debug_flashsec_win e encrypted_fw.bin 9003F000 1000

Note that the drive does not restart after each sector flash, nor does it need to be restarted. So you can change multiple sectors in one sitting. For example, a typical session might look like this (in Windows)

C:\> debug_flashsec_win e encrypted_fw.bin 90006000 1000
C:\> debug_flashsec_win e encrypted_fw.bin 90010000 1000
C:\> debug_flashsec_win e encrypted_fw.bin 90027000 1000
C:\> debug_flashsec_win e encrypted_fw.bin 9003E000 1000

This example shows 3 sector changes followed by a final sector change to update the firmware with a new correct checksum value. This final checksum update flash must always be performed unless you actually want the checksum to fail the next time your drive powers up (which will leave you stranded in recovery mode, not a great place to be).

Petrinet Editor/Simulator

I’ve wrote a program with a friend that allows to edit and simulate limited petrinets, including a format for defining petrinets in XML. It was written as an exercise for the lecture “Beschränkte Petrinetze 1” of Prof. Gerber at the University of Leipzig.

Last build is from 2003-02-19. You need Java 2 Version 1.4 to run this program. Get it at http://java.sun.com

Webstart

Download

Examples and Documentation

Written by Michael Große & Arne Brutschy, © 2003