Cleaning Up Is Hard To Do

Linux TuxI have been administering Linux servers for a while now so coming across a new problem is both exciting and stressful, especially when that problem is on a high demand production server. I recently came across one of these scenarios, and was surprised how difficult it was to solve.

I had a single directory on a server that contained millions and millions of tiny 32Kb text files (which were actually non-sharded PHP session files that didn’t get caught in garbage collection, and went unnoticed for months on a busy server). The interim fix to that problem was easy:

mv /var/php/sessions /var/php/sessions.evil && mkdir /var/php/sessions

The real problem was that I was left with this sessions.evil directory that I couldn’t delete, ignore in snapshot backups, or even list of the contents of because there were just too many files and going anywhere near it would use too much disk I/O and cause the load on the server to spike.

After using / writing various Bash, Python, PHP, etc, scripts that caused too much load on the server I happened across this genius solution by Zhenyu Lee (and a comment by Paul Reiber)… to use rsync instead of rm, find, xargs, etc:

On a CentOS box and using my example directory of /var/php/sessions.evil, which was owned by root:apache do this:

cd /var/php
mkdir empty_dir
chown root:apache empty_dir
rsync -vvvv -a –delete empty_dir/ sessions.evil/

Depending on how many files are in your sessions.evil directory this could take a while (2 days in my case), but the 5 minute load average on the server stayed between a manageable 2.0 and 3.0. My twist to Zhenyu’s solution was to add some verbosity (-vvvv) in there so I could tell that rsync was actually doing something.

Paul’s next comments are important though, so once rsync is eventually done mirroring your sessions.evil directory with your empty_dir pay attention:

rmdir sessions.evil
mkdir sessions.evil
rmdir sessions.evil empty_dir

The first rmdir sessions.evil will cause a bit of a load average spike for a few minutes, but once it’s gone… whew.

Well there you go, happy rsyncing, and a huge thanks to Zhenyu Lee for posting that unique and brilliant solution.

Multiple Accounts in Google Drive

Google released their Google Drive last week, and I’m feeling conflicted about it. On one hand it’s awesome, nearly everything I would want in a document sharing service; on the other hand Dropbox has been providing me with similar functionality for years when Google couldn’t or wouldn’t, so I have a certain level of loyalty there.

Regardless of my internal struggles I thought I would point out one glaring issue I have with Google Drive: I have 2 Google accounts, a personal one and a work one, and this intentional line in the sand cannot be crossed.

Based on a statement from their support website it is currently

“not possible to use Google Drive for your Mac/PC with more than one account at the same time”

Okay that is fine with me actually, but at least let me configure two accounts in settings (each with their own separate shared directory) and then let me switch between the one that is actively syncing. The disconnecting and connecting with a different account (as they suggest) doesn’t really solve the issue of keeping these two accounts separate.

This was never really an issue with Dropbox for me because of how I structured my Dropbox directory, whereby my “Work” directory was shared with work people, and my “Home” directory was shared with family.

Besides that one glaring issue I have to say, I am very impressed with Google Drive. The integration with Google Docs is especially brilliant.

At this point Drive and Dropbox coexist happily together on my Mac, each being used for something slightly different.

Cisco WRVS4400N Is Upsetting

A little over a year ago I decided it was time to replace my aging Linksys WRT54G wireless router in my home office with something new and fancy. I happened to be visiting a local computer shop one Saturday afternoon and came across the Cisco WRVS4400N Wireless-N Gigabit Security Router – VPN v2.0, which on the box sounded amazing. It was a little on the expensive side comparatively, but hey… it was Cisco, and I could setup VPN profiles on the device and easily connect remotely to my local network… so the extra money would be worth it.

Boy was I wrong. This device has never functioned even close to my expectations, despite running firmware version 2.0.2.1. My biggest complaint is that I am finding the wireless connectivity terribly unreliable with frequent disconnects. My second largest issue is that the supported QuickVPN client is a sad sad Windows hunker, and the only way I can get the VPN connection working reliably on my Mac laptop is use to VPNTracker, which is beautiful but significantly more money that I am willing to pay for something that should work using Mac OS X’s built in L2TP over IPSEC or Cisco IPSEC connection. AFAIK the only other option for Mac users is to use IPSecuritas, but unfortunately it wasn’t working with Mac OS 10.7 at the time.

In summary (as I sit here connected to my Airport Express) I am far from impressed with the WRVS4400N, and this reaffirms that I should only purchase equipment after reading reviews not the other way around (as you may be now).

Good luck, and move on to something else. If you find something else awesome and worth it, please feel free to post it.

Find All WordPress Installations

I was trying to figure out how many WordPress installations lived on one of my hosting servers recently, and I also wanted to see how good the site owners are at keeping them up-to-date, which means I also wanted to find out the WordPress version number of each installation.

I couldn’t find any quick way of doing this so I turned the linux find command and grep to give me a hand. Here is the command (which must be run as a privileged user):

find /var/www/vhosts -type d -name "wp-includes" -print -exec grep "wp_version" {}/version.php \;

Just replace /var/www/vhosts with the root directory of where all of your domains are stored. Hope this helps someone. Cheers.

Updated: I switched the above command to use “-type d” vs “-d” as per James’ comment below. Thanks James.

Stay Cool With Tequila

I am absolutely loving this hot summer weather, and this week I happened to make a drink that fit the conditions perfectly: a nice frozen margarita. I have consumed my fair share of margaritas, but I’ve never taken the time to make one. I found this great recipe on AllRecipes, but here it is again crafted for the Magic Bullet Blender.

Ingredients:

  1. Gold Tequila
  2. Triple Sec
  3. Thawed 355ml can of Minute Maid Limeade Concentrate
  4. Ice
  5. A lime

Directions:

  1. Fill the bullets carafe with as much ice as you can pack in.
  2. Add 110ml of the thawed limeade concentrate.
  3. Add 85ml of Tequila.
  4. Add 30ml of Triple Sec.
  5. Now blend until smooth (30 – 45 seconds).
  6. Squeeze in a lime wedge, drop in a straw, and enjoy.

Note: 1 can of the limeade concentrate makes 3 of these drinks.

We enjoyed this drink with some Tostitos and salsa. Yum, and good night.

Where The Heck Is Java Web Start?

So you have a from-the-browser Java application that requires Java Web Start to load on your beautiful Mac, and that Java application is named something crazy like:
viewer.jnlp(192.168.1.123@0@idrac-BR2XXXX1,+PowerEdge+R710,+User_joeblow@3313732211320)

What do you do? It doesn’t open automatically, and when you double click the filename it just asks you what application you want to use to open the file.

Well I will tell you what you do!

  1. On the “Launch Application” dialog that opens when you double click the filename, click “Choose…” beside “Choose an Application”.
  2. Navigate to Macintosh HD > System > Library > CoreServices.
    /System/Library/CoreServices/Java Web Start.app
  3. Choose “Java Web Start” and click Open.

There you go. A post like this would have saved me 15 minutes anyways, hope it helps someone else.

Sony Blu-Ray Players Don’t Play NetFlix in Canada

The bad newsI was somewhat frustrated this evening to find out that despite NetFlix being available in Canada now, Sony devices such my new Sony BDP-S370 blu-ray player will still not allow me to connect to NetFlix. So if you were holding out to find that perfect device to connect to the newly available NetFlix Canada service with, Sony devices are definitely not an option for you at this point. I’ve attached a Sony eSupport chat transcript for you to wallow over. Boooo, back to the store you go blu-ray.

Life Changing Experience

I almost have to laugh when I think about how much I have changed over the last several years, and one of the biggest changes in my life has been my views about having children. I used to be under the impression that having a child would prevent me from doing the things I really wanted to do, and I was convinced I would somehow lose my identity and become a standard complacent father figure shell of my old self.

I now know that what my old self failed to realize is that this scenario is not an inevitability, it is a choice.

Karla and I feel like the two luckiest people in the world right now, and it is my pleasure to introduce Mr. Declan Simpson, an 8lbs 14oz bouncing baby boy. I have spent the past few weeks at home getting to know Declan, and watching him explore the big new world around him. This has been a truly indescribable experience that I will no doubt treasure for the rest of my life. Welcome Declan, enjoy life my son.

Bitnami Redmine Stack

I ran across Bitnami today, and they have some really great software stacks with flexible install options (i.e. Native / VM / Cloud), including PowerPC support so I can put some of these older G5 Powermac’s into use again.

I downloaded the VM of their Bitnami Redmine Stack and it was so quick to get up and running, but here’s a tip to save others some frustration perhaps:

Despite the README.txt stating the default Redmine username and password is the username and password you entered during setup, it is actually (as of 0.9.4) Username: user | Password: bitnami

Scheduled Replacement of websrv02 Hardware

This is an important notice to inform websrv02 users that we will be replacing the websrv02 server hardware on Friday, February 19th, 2010. The current hardware is about four years old, and while it has not given us any problems we like to make sure our hardware is always in tip-top shape.

The websrv02 server will be unavailable for a short period of time while we shut the old system down and bring the new one online (approximately 20 minutes). This downtime is currently scheduled for:

Friday, February 19th, 2010 around 1:00PM EST

Important Website Warning: While the server downtime itself will be short, we will be starting the data transfer between the old server to the new server at approximately 10:45AM. It is important that you *do not make any changes to your website* between 10:30AM and 2:00PM EST on Friday or your changes may be lost once the new server is brought online.

Important E-Mail Warning: We will be shutting down incoming mail service on websrv02 at approximately 11:00AM EST to prevent any e-mail loss. If someone attempts to send you an e-mail on Friday between 11:00AM and 1:00PM EST the message will stay in their servers mail queue and will automatically resend once the mail service on the new server is brought online again.

We do apologize for any inconvenience this may cause; however we feel that this is an important upgrade to ensure the long term quality of service you have come to expect.