Mac OS X 10.8.2 ML Kernel Panic

So i had 2 kernel panics within the last 2 days. Diagnosing the issue can be very difficult. But it seems that the issue could have been caused some a few stray kernel extensions left over from some software i uninstalled quite some time ago.

I believe the cause was Blackberry’s desktop manager software, namely because; 1) i uninstalled it about 6 months ago, and 2) because it seems to have left a bunch of daemons after its uninstall that remained, and the logs show that they kept spawning new instances, quiting/crashing and then waiting 10 seconds and spawning again. The reason i came to notice it was more for the fact that it was throttling continually. That got my attention!

Check your /Applications/Utilities/Console.app for more info on whats going on in your system. The information there can be very difficult to make sense of, and most of it is harmless despite lots of things in the log that say things like ‘failed’ and ‘warning’ etc.

I manually used the locate database to find the blackberry remnants and quickly removed the daemons and so fourth. Just do a keyword search for anything like:

  • locate com.rim.
  • locate blackberry

After removing all that crud via the rm utility and a reboot i had another kernel panic the next day, and then i came across a great site for diagnosing kernel panics on mac os x and you might want to check it out

I used the kextstat | grep -v com.apple to show all of the 3rd party kernel extensions and found the following:

macpro:~ admin$ kextstat | grep -v com.apple
Index Refs Address Size Wired Name (Version)
18 0 0xffffff7f80947000 0x18000 0x18000 com.rim.driver.BlackBerryUSBDriverInt (0.0.74) <17 16 15 13 12 5 4 3 1>
52 0 0xffffff7f8138d000 0x10000 0x10000 com.metakine.handsoff.driver (2.0.4) <7 5 4 3 1>
107 0 0xffffff7f807c0000 0x5000 0x5000 net.telestream.driver.TelestreamAudio (1.0.5) <86 5 4 3 1>
108 0 0xffffff7f807ac000 0x4000 0x4000 com.vara.driver.VaraAudio (1.0.3) <86 5 4 3 1>
113 3 0xffffff7f82316000 0x36000 0x36000 org.virtualbox.kext.VBoxDrv (4.1.12) <7 5 4 3 1>
114 0 0xffffff7f8234c000 0x8000 0x8000 org.virtualbox.kext.VBoxUSB (4.1.12) <113 47 12 7 5 4 3 1>
115 0 0xffffff7f82354000 0x5000 0x5000 org.virtualbox.kext.VBoxNetFlt (4.1.12) <113 7 5 4 3 1>
116 0 0xffffff7f82359000 0x3000 0x3000 org.virtualbox.kext.VBoxNetAdp (4.1.12) <113 5 4 1>
117 0 0xffffff7f809bf000 0xc000 0xc000 com.taoeffect.ispy.kext (2.0.2) <5 4 3 1>

The items below shown in bold are the ones i decided were the most likely culprits:

  • com.rim.driver.BlackBerryUSBDriverInt (0.0.74)
  • com.metakine.handsoff.driver (2.0.4)
  • net.telestream.driver.TelestreamAudio (1.0.5)
  • com.vara.driver.VaraAudio (1.0.3)
  • org.virtualbox.kext.VBoxDrv (4.1.12)
  • org.virtualbox.kext.VBoxUSB (4.1.12)
  • org.virtualbox.kext.VBoxNetFlt (4.1.12)
  • org.virtualbox.kext.VBoxNetAdp (4.1.12)
  • com.taoeffect.ispy.kext (2.0.2)

As for the rest, Metakine is a firewall, virtualbox is a VM like parallels, and that left me with the rest, which i presumed were vital to the operation of audio drivers etc.

So, the iSpy is a part of Espionage.app, which i promptly uninstalled, but again this left crapola on my disk, which i had to manually track down and remove.

The blackberry stuff however was a bit more difficult. I did however find a discussion on the blackberry board where a user had extracted the uninstall shell script and shared it. I tried to run this but it complained about the lack of ref being available. So i just stripped out everything but the rm and other lines that removed mention of BB related stuff. After a quick running of the script and a reboot, sure enough kextstat was no longer reporting the issue. Also, boot time improved which i think is due to not having a daemon continually spawning and throttling the system every 10 seconds. Also note, sometimes these days my machine would hang on boot, and i am not experiencing that issue anymore since removing the blackberry daemons and drivers. If your machine hangs on boot, definitely check out your kexts.

The script i mention can be found here, but i show it below for your convenience:

#!/bin/bash -x

CALLERS_UID="$1"
REFCOUNT_PATH="$2"
DCALL_PATH="$3"

if [ $UID -ne 0 ]; then
# Test that the script is run as root.
echo "Script must run as root"
exit 1
fi

if [ ! -x "$REFCOUNT_PATH" ]; then
echo "Ref tool not found"
exit 1
fi

PKREF="pkgutil --pkgs=com.rim.blackberrydesktopmanager.Application.pkg"

if [ -d "/Library/Receipts/blackberrydesktopmanager.pkg" ] || [ ! -z {PKREF} ]; #MAYBE THIS SHOULD BE /Library/Receipts/blackberrydesktopmanager.pkg INSTEAD?
then
#unregister the Desktop Manager App with BBLaunchAgent
/Library/Application\ Support/Blackberry/BBLaunchAgent.app -ndefault /Applications/BlackBerry\ Desktop\ Manager.app >> /dev/null

/bin/rm -fr "/Applications/BlackBerry Desktop Manager.app"
/bin/rm -fr "/Library/Receipts/blackberrydesktopmanager.pkg"

# Delete the Application Support folder and preferences for ALL users
/bin/rm -fr /Users/*"/Library/Application Support/BlackBerryDesktop/"
/bin/rm -f /Users/*"/Library/Preferences/com.rim.blackberrydesktopmanager.plist"
/bin/rm -fr /Users/*"/Library/Caches/com.rim.blackberrydesktopmanager"

/Library/Application\ Support/BlackBerry/BBLaunchAgent.app -ipndefault /Library/Application\ Support/BlackBerry/IPModemPasswordDialog.app >> /dev/null
/bin/rm -fr "/Library/Application Support/BlackBerry/IPModemPasswordDialog.app"

REFCOUNTVSP="\"$REFCOUNT_PATH\" -unreferenceVSP"
if [ {REFCOUNTVSP} -eq 0 ]; then
/bin/rm -fr "/Library/Modem Scripts/RIM IP Modem.ccl"
/bin/rm -fr "/Library/Frameworks/RIM_VSP.framework"
/bin/rm -fr "/Library/Receipts/blackberryvsp.pkg"
/bin/rm -f "/Library/Preferences/com.rim.vsp.plist"
fi

REFCOUNTVSPDR="\"$REFCOUNT_PATH\" -unreferenceVSPDR"
if [ {REFCOUNTVSPDR} -eq 0 ]; then
/bin/rm -fr "/System/Library/Extensions/RIMBBVSP.kext"
/bin/rm -fr "/Library/Receipts/blackberryvspdr.pkg"
/bin/rm -f "/Library/Preferences/com.rim.RIMBBVSP.plist"
fi

REFCOUNTDR="\"$REFCOUNT_PATH\" -unreferenceDR"

if [ {REFCOUNTDR} -eq 0 ]; then
# unload the driver
if [ ! -x "$DCALL_PATH" ]; then
echo "DCAll tool not found"
else
DCALL_RESULT="\"$DCALL_PATH\""
fi
fi

REFCOUNTFW="\"$REFCOUNT_PATH\" -unreferenceFW"

if [ {REFCOUNTFW} -eq 0 ]; then
# BBLaunchAgent is run by the users, not root. We walk through all logged in users and
# attempt to quit the BBLaunchAgent just in case multiple users are logged in simultaneously.
for currentUser in users; do
sudo -u "{currentUser}" /bin/launchctl unload /Library/LaunchAgents/com.rim.BBLaunchAgent.plist
done

# NOTE: The previous loop might fail on 10.4 if the user name is >8 characters (the 'users' command
# on 10.4 returns only the first 8 characters of the user name!). So, we explicitly
# unload using the numeric userid just in case.
sudo -u "#${CALLERS_UID}" /bin/launchctl unload /Library/LaunchAgents/com.rim.BBLaunchAgent.plist

# stop the daemon that is run as root
/bin/launchctl unload /Library/LaunchDaemons/com.rim.BBDaemon.plist

/bin/rm -fr "/Library/Frameworks/RimBlackBerryUSB.framework"
/bin/rm -f "/Library/LaunchDaemons/com.rim.BBDaemon.plist"
/bin/rm -f "/Library/LaunchAgents/com.rim.BBLaunchAgent.plist"
/bin/rm -fr "/Library/Application Support/BlackBerry"
/bin/rm -fr "/Library/Receipts/blackberryframeworks.pkg"

/bin/rm -f "/Library/Preferences/com.rim.RimBlackBerryUSB.plist"
/bin/rm -f "/Library/Preferences/com.rim.RimLaunchAgent.plist"

/bin/rm -f /Users/*"/Library/Preferences/com.rim.RimLaunchAgent.plist"
fi

if [ ${REFCOUNTDR} -eq 0 ]; then
# unload the driver
KEXTSTATUS="/sbin/kextunload -b com.rim.driver.BlackBerryUSBDriverInt"

/bin/rm -fr "/System/Library/Extensions/BlackBerryUSBDriverInt.kext"
/bin/rm -fr "/System/Library/Extensions/RIMBBUSB.kext"

/bin/rm -fr "/Library/Receipts/blackberryusbdriverint.pkg"

/bin/rm -f "/Library/Preferences/com.rim.BlackBerryUSBDriverInt.plist"
/bin/rm -f "/Library/Preferences/com.rim.RIMBBUSB.plist"
fi

# SnowLeopard stores receipts differently then Leopard and vice versa.
# This unforunatly does not work on Leopard either so we need to support both receipt removal methods
# To get ride of SnowLeopard receipts:
/usr/sbin/pkgutil --forget "com.rim.blackberrydesktopmanager.BlackBerryFrameworks.pkg"
/usr/sbin/pkgutil --forget "com.rim.blackberrydesktopmanager.BlackBerryUSBDriver.pkg"
/usr/sbin/pkgutil --forget "com.rim.blackberrydesktopmanager.BlackBerryUSBDriverVSP.pkg"
/usr/sbin/pkgutil --forget "com.rim.blackberrydesktopmanager.BlackBerryVSP.pkg"
/usr/sbin/pkgutil --forget "com.rim.blackberrydesktopmanager.Application.pkg"

else
echo "Not Found"
exit 1
fi

echo "Success"
exit 0

You can run it via:

// Lets set the permissions on it, i named my version removeBB.sh
$ chmod +x ~/removeBB.sh
$ sudo sh ~/removeBB.sh

If you have any problems, edit the file AT YOUR OWN RISK and run it to bypass some checks that can miss some stuff.

The iSpy stuff from Tao-effect is for the espionage app for hiding stuff on your mac. I got it as part of the mac bundle, and i never really wanted or even used the thing, so i removed it as-well.

I also found some other kexts i don’t have the logs for anymore to show you, but i also removed them, they were:

  • Sophos anti-virus
  • SyncMate
  • app4mac

If you don’t know what the kext is your looking at, then before removing it, go google it! Find out what it is your considering removing before doing it, it could be important!

From time to time, while your removing such crud and using locate to find stuff, the locate tool will still show the files as being there despite them being deleted and you will want to update your locate database, you can do that like this:

$ sudo /usr/libexec/locate.updatedb

It can take about 2-5 minutes (sometime longer depending on your setup). Once done, files you already deleted won’t show up anymore.

Also, courtesy of the apple discussion boards i found a dicussion regarding kernel panics, unbootable machines and being unable to get past the password screen at login. The generally accepted solution seems to be to remove a symlink that the OS will rebuild upon next update.

$ sudo rm /var/audit/current

You will need to reboot after that.

Good luck.

Hardening PHP through php.ini Configuration File.

If you have read my other post on hardening your LAMP stack / Plesk VPS, then you will know that by default most LAMP installs are not very secure by default.

Many of the default settings leave your system vulnerable and open to exploitation. Usually it starts with the system revealing too much information to potential hackers. The first step is always to reveal the least amount of information about your setup and humanely possible. The next step is reducing your attack vectors by switching off features you do not need or use.

In this article, that is what we will be looking into, how to harden PHP through php.ini configuration file.

Locating php.ini Configuration File.

Firstly, we need to locate all our php.ini files as there can be many, from the default files (sometimes 2 [1 for the web server and 1 for the CLI though not always]) to the copies made for individual domains if you are running a VPS. Use the following to find your php.ini configuration files.

$ locate php.ini

If you don’t get anything or are missing some that you already knew of, you may want to update the locate database with this:

$ updatedb

Once we have our list, its time to start making some changes. I find it is best to ensure we only change 1 to 2 things at a time before we go ahead and restart the server to make sure we have not broken anything. This way it is much easier to know what was the cause if we have broken anything.

Editing ‘php.ini’.

You can use whatever text editor you like, i use ‘vi’ but feel free to use nano/pico or some graphical editor on the desktop if you prefer.

Prevent Information Disclosure.

Turn off error outputting in the browser that could potentially reveal sensitive aspects of your web apps code.

display_errors = off

Disable Globals.

Register globals is set to be removed from php soon, and should be off by default. However many LAMP/WAMP/MAMP stacks still ship with it turned on unfortunately (likely [hopefully] still a minority of stacks though). So don’t take the risk of leaving it on, check it is turned off.

register_globals = off

Leaving this on leaves a massive vulnerability in your setup, namely because regular variables in your code that match GET vars in the url string will start out with the value of the GET variable. This means hackers can put whatever content they like into any variable in your code that could cause an unexpected outcome that they may want to target. With register_globals on the GET vars will always overwrite the default assigned value, this is a serious vulnerability if not turned off.

Disable Remote File Includes.

Prevent a hacker who has found some part of your web app that they can use to execute a remote script from succeeding by disabling PHP from reading/executing remote scripts.

allow_url_fopen = off
allow_url_include = off

Restrict File Uploads.

You can prevent file uploads if you do not have any use for such a feature by using the following:

file_uploads = off

If on, hackers can rename a scripts file to appear as an image, then successfully upload the file and when using the url to access the file can run their script. This is very bad because now they can execute scripts they have uploaded to your server. You might want to consider applying some rules to Apache/PHP to prevent the execution of scripts in your upload directories as a safety precaution.

I would recommend setting a sensible maximum file size for uploads to say 2 megabytes max.

upload_max_filesize = 2M

You can also change your default temporary file upload directory:

upload_tmp_dir = /var/php_tmp

If you want to make sure that malicious hackers cannot abuse your upload facilities, then ensure that you prevent script execution in the directories where you are storing your uploads. This is in either your htaccess file or httpd.conf file (which is out of the bounds of what this article is about but i will give you the snippet you need never the less).

<Directory /var/www/vhosts/yourdomain.com/httpdocs/your_upload_dir/>
Options None
AllowOverride None
php_admin_flag engine off
order deny,allow
deny from all
</Directory>

Protect Sessions.

Protect your sessions from being hijacked or shared in links people post online or send to friends by enabling cookie httponly. It will also prevent Javascript from reading your cookies.

session.cookie_httponly = 1

Also add a referer check, like:

session.referer_check = codeconsortium.com

You might want to change your default session save path to somewhere hackers won’t find as easily. E.g:

session.save_path = /var/lib/php

Disable Unnecessary Functions.

PHP includes many functions they you will likely never need or use, however they could be very helpful to hackers, disabling them likely will not affect your site, but will help make the hackers job much more difficult.

disable_functions = php_uname, getmyuid, getmypid, passthru, leak, listen, diskfreespace, tmpfile, link, ignore_user_abord, shell_exec, dl, set_time_limit, exec, system, highlight_file, source, show_source, fpaththru, virtual, posix_ctermid, posix_getcwd, posix_getegid, posix_geteuid, posix_getgid, posix_getgrgid, posix_getgrnam, posix_getgroups, posix_getlogin, posix_getpgid, posix_getpgrp, posix_getpid, posix, _getppid, posix_getpwnam, posix_getpwuid, posix_getrlimit, posix_getsid, posix_getuid, posix_isatty, posix_kill, posix_mkfifo, posix_setegid, posix_seteuid, posix_setgid, posix_setpgid, posix_setsid, posix_setuid, posix_times, posix_ttyname, posix_uname, proc_open, proc_close, proc_get_status, proc_nice, proc_terminate, phpinfo

Summary

Much of the work shown above is courtesy of MadIrish, though i have shortened the descriptions down and focused more on the configuration changes than their descriptions / implications. If you want more in-depth descriptions of what we have done here, you should check out MadIrish’s website.

Other changes i made was the prevention of executing scripts in upload directories.

Though this article reproduces most of the steps of what MadIrish has achieved, this is more for my own record keeping than anything else. Though i hope it benefits others as-well, credit for most of this goes to MadIrish, good luck!