In our continuing adventures of putting the highly versatile Apple MacMini to work in all sorts of applications:

A customer of ours has a specialized application that is extremely low bandwidth, but needs to be able to be accessed through a VPN connection to a protected network resource. All the usual suspects for providing this kind of service (Netscreen/SSG, Netopia) all have artificial limits on the number of concurrent VPN connections. In order to service the number of concurrent users (anywhere from 1 to 200), the customer would be looking at least $5,000 for a capable device. Again, this system is very low bandwidth, so these kinds of devices would be serious overkill.

In comes the MacMini running Tiger Server with the built-in VPN service.

However… the MacMini has always had a single weak spot: the hard drive. Even with a very low disk based usage that a dedicated VPN Server would entail that vulnerability to a disk failure has always bothered me and I wanted this VPN server to be on par in terms of reliability as much as possible compared to a hardware device. Even the MacMinis that we build out for customers in a load balanced environment, we always replace with a extended use rated drive, as the extra $100 is a nominal expenditure compared to the time that would be involved in dealing with a failed system.

In comes the Solid State Disk. A 32GB SSD with a IDE interface runs just under $100. For a system running primarily as a VPN Server, this is PLENTY of space as the only real ongoing writes will be log files. I chose a MLC (Multi-Level Cell) device over a SLC (Single Level Cell) based device as the differences in speed and longevity between the two variations of SSD just were not a factor in this application. A MLC SSD already has a useful lifetime/MTBF an order of magnitude greater than a standard Winchester based disk with it’s moving parts.

In the end I used a 1.33Ghz G4 (PowerPC, not Intel) MacMini with 1GB of RAM, as that CPU should be able to easily handle the tasks of handling a couple hundred concurrent VPN connections using 1-2Kbits/sec each. Again, VERY low bandwidth application.

I took the MacMini with it’s original 40GB Seagate drive and installed Tiger Server so I could get it updated and setup while I waited for the SSD to arrive from the online vendor. When I was ready to install the SSD, I used Disk Utility to save the installed system to a disk image. Installing the SSD was straightforward and not any different that replacing the internal drive on a MacMini. I then booted from CD, formatted the new SSD device and then performed a Restore operation of the installed Tiger Server onto the new SSD.

One thing that was very interesting to watch… SSDs perform their best with you throw a large amount of sequential reads/writes at them. When doing a Restore with Disk Utility it’s best to use the “Erase Destination” option as that will enable the imaging system the best opportunity to use a Block level Restore, instead of a File based Restore. A Block level Restore streams data at the disk and the SSD just ate this up. I Restored the 3.2GB installed System image in about 3 minutes. This is about 3 times faster than restoring back to the 40GB Seagate (I went back and tried it for comparison)

As always, when deploying a new type of setup we test the power utilization and found that 1) the G4 MacMini uses a ridiculously low amount of Amps; 2) the SSD didn’t really save much power usage.

Some anecdotes from my ammeter (we use 120V here in the States):

Adding a USB Ethernet Adapter increased all values by about 0.06A
This is necessary for the backside network resource routing.

Some pictures

Transcend 32GB SSD

Transcend 32GB SSD upright

Transcend 32GB SSD in Apple System Profiler

And of course the obligatory speed of boot/login videos:

G4MiniSSD-Boot

G4MiniSSD-Login

We have several more G4 MacMinis and are looking for appropriate applications to utilize them especially with an ultra-reliable drive installed.

Posted by Brian Blood as Colocation, Hardware, OS X Server, Routers and Firewalls, Servers at 1:41 PM UTC

No Comments »

Many years ago, before there were Xserves, in an attempt to save rack space in our cabinets, we experimented with ripping the guts out of a PowerMac G4 and stuffing them into a 2U server case. Here is a photo of one of those attempts:

It worked out fairly well and we had a couple of these running for several years.

The main piece to the puzzle was finding the right L riser card to give us access to the AGP slot and 2 of the PCI slots, one for a secondary Ethernet card and another for an ATTO SCSI card.

Posted by Brian Blood as Colocation, Hardware, Servers at 10:15 AM UTC

No Comments »

Server Monitor is an application that allows you to monitor the health of several Xserves over the network:

Sometimes the application gets a bit cranky about the connections it makes to the servers and reports that it can’t communicate or as you see here in the picture “reply not understood”. So we don’t really use it for serious monitoring other than as a cursory glance usually to check some items.

However, Apple really takes the cake with this knowledge-base article:

Xserve: Server Monitor does not authenticate with server over subnet

in which they claim that the way to fix the problems with their SOFTWARE, is to:

1. Make the necessary changes to the username or password using Server Monitor.
2. Quit Server Monitor.
3. Shut down the Xserve that is the target of these changes.
4. Remove the power cord from the back of the Xserve.
5. Wait 30 seconds and plug the power cord back in.
6. Power the server back on.

This sounds suspiciously similar to something an old tech friend of mine once told me:

There are sound scientifically proven reasons why one must sometimes sacrifice a chicken in order to get a SCSI chain to work.

Ugh.

Posted by Brian Blood as Colocation, Hardware, OS X Server, Servers, Soap Box at 11:33 PM UTC

No Comments »

It has come to my attention that the thumbscrews on the front panel of Xserves are a very important component of the overall structural integrity of the rack mounting system. they not only keep the Xserve from coming out of the case, but they also secure the forward portion of the server to the mounted top case. I’ve seen many Xserves that seemed as if they could pop out of their cases due to a small amount of bowing in the case. Having the thumbscrews tightened down adds another vertical support component.

Posted by Brian Blood as Colocation, Hardware, Servers at 11:54 AM UTC

No Comments »

A client of ours is moving his servers to a new colo soon and like us, uses QuickDNS from Men and Mice to manage his BIND installations. He needed an AppleScript that would go through all his zones and reduce all the TTL values so that when he moved his servers, the transition would happen as smoothly as possible. So I whipped up this AppleScript:

set setDirection to "low"
--set setDirection to "normal"

tell application "QuickDNS Manager"
  repeat with theZone in zones
    set curZoneName to name of theZone
    set theDoc to (open theZone)
    tell theDoc
      if ("low" is setDirection) then
        set default time to live to 7200
        try
          set expire of first SOA record to "1400"
        end try
        try
          set refresh of first SOA record to "7200"
        end try
        try
          set negative caching of first SOA record to "7200"
        end try
        try
          set retry of first SOA record to "3600"
        end try
        try
          set time to live of first SOA record to "7200"
        end try
      end if
      if ("normal" is setDirection) then
        set default time to live to 7200
        try
          set expire of first SOA record to "604800"
        end try
        try
          set refresh of first SOA record to "28800"
        end try
        try
          set negative caching of first SOA record to "43200"
        end try
        try
          set retry of first SOA record to "7200"
        end try
        try
          set time to live of first SOA record to "86400"
        end try
      end if

      save with comment ""
    end tell

  end repeat
end tell

Posted by Brian Blood as Colocation, Servers at 5:39 PM UTC

No Comments »

Datacenter Power. It seems you can never have enough.

We have our colocation inside an Equinix IBX. It is an excellent facility. Unfortunately, about 2 years ago, our cage got a new neighbor. They have added rack after rack of new servers to accommodate their ever increasing traffic. Which means they have effectively used up all the allocated power feeds for our section of the colo.
So as we started to fill our own cabinets, we found that we were quickly using up the 2 x 20A 110V feeds they had allocated to each of our cabinets. Our partner in colocation, sell.com was also at this time upgrading their farm to the latest dual xeon models. These boxes were pulling a LOT more amps than the previous P3 generation.

Very quickly, we became experts on how much amperage we could squeeze out of our existing feeds and what systems required how much power.

Here are some anecdotal amperage readings we took from our fancy amp reading tool.

Dell PowerEdge 2850

Specs: Dual Xeon 3.6GHz/1MB; 6 x 73 GB SCSI Hard Drive (10K RPM); Dual Power supplies

• PS A & B both active
  • PS A – 1.15A
  • PS A & B – 2.35A
• PS A only – 2.30A

Dell PowerEdge 1750

Specs: Dual Xeon 3.2GHz – 4GB RAM; 3 x 146 GB 10K rpm SCSI Hard Drives; Dual Power supplies
Software: Debian 5.0 – MySQL 5.0 – InnoDB heavy

• Off – 0.21A
• Cold Start – 3.00A Peak
• Nominal usage – 1.90A

Dell PowerEdge 1650

Specs: Dual PIII 1.4Ghz; 2GB RAM; 3 x 36GB SCSI 10K rpm; Dual 275W Power supplies

• PS A & B both active
  • PS A – 0.7A
  • PS A & B
    • Nominal operation – 1.41A
    • Warm Boot – 1.44A Peak
    • Cold Boot (drives spinning up) – 1.56A
• PS A only
  • Nominal operation – 1.37A

Apple Power Mac G4

Specs: G4/533 Dual – 1.5GB RAM – 2 x 18GB SCSI (15K rpm)

• Peak Startup – 1.27A
• Max load on SCSI drives – big copy operation – 1.18A

Apple Xserve G4

Specs: Dual 1.0 Ghz G4, 2GB RAM 2×60GB & 2 x 180GB

• heavy cpu/disk load – 1.52A
  • simultaneous diskutil zero on all disks (booted from CD)
  • Max CPU – multiple threads of cat /dev/urandom > /dev/null & ssh/rsa keygen operations
• all 4 disks idle – 1.37A
• Insert 180GB ADM – peak 1.41A, settled back down to 1.32A
• Insert second 180GB ADM – peak 1.48A, settled down to 1.38A
• keygen and cat large data file generated by /dev/urandom, copied to Software RAID mirror 60GB – spikes to 1.56A

Apple Xserve G5

Specs: Dual 2.0Ghz G5, 3GB RAM, 3 x 80GB SATA

• Nominal operation – 1.8A
• Max Cold Boot – 2.16A

Specs: Dual 2.3Ghz G5, 1GB RAM, 2 x 500GB SATA

• Nominal operation – 1.8A
• Max Cold Boot – 2.07A

Apple dual Quad Core Intel Xserve

Specs: Dual Intel 2.8Ghz Quad Xeon (8 cores), 16GB RAM, 3 x 1TB SATA in RAID 5

• Max Cold Boot – 3.23A
• Nominal operation – 2.80A
• Max cpu, disk activity – 3.68A
• Powered Off – 0.27A

Apple single Quad Core Intel Xserve (Xserve2,1 – Early 2008 model)

Specs: Single Intel 2.8Ghz Quad Xeon, 4GB RAM, 2 x 250GB SATA

• Nominal operation – 2.00
• Powered Off – 0.28A
• Max cpu, disk activity – 2.08 amps
  (calculated by adding all “watts” readings in Server Monitor and div by 115V)

Apple Intel Mac Mini

Specs: Intel 1.66Ghz Core Duo, 2GB RAM, 60GB E-Rated Hitachi drive E7K100 model

• Nominal operation – 0.29A
• Max cpu, disk activity – 0.37A

Apple G4 Mac Mini

Specs: 1.33Ghz PowerPC G4, 1GB RAM, no wireless, 32GB Transcend Solid State Disk

• Nominal operation – 0.20A
• Max cpu, disk activity – 0.26A

Apple Xserve RAID (Xraid)

Specs: 7 x 250GB (Hitachi) and 7 x 750GB (Seagate 7200.10)

• Nominal operation – fluctuates around 2.00A
• Max disk activity (as much as I could generate using Xserve G4) – 2.19A

IBM 4000R

Specs: Dual 833Mhz PIII – Single Power supply – 2 x 18GB SCSI (10K rpm)

• Cold Boot (drives spinning up) – 1.0A
• heavy cpu/disk load – multiple instances of cpuburn and cat’ing /dev/urandom to a file – 0.9A
• Nominal operation – 0.75A max

IBM eServer x330

Specs: Two Intel Pentium III (Coppermine) 864MHz processors, 1GB RAM, Single Power Supply, Single 36GB SCSI drive

• Connecting Power Peak: 0.29A
• Stdby Steady: 0.11A
• Power On Peak: 0.78A
• SCSI spinup: 0.98A
• Powered low load: 0.63A
• Loaded (6.0+ Load Average with disk): 0.80A
• Disk activity only: 0.72 peakA
• Reasonable Load + Disk Activity: 0.79A
• heavy cpu/disk load – multiple instances of cpuburn and cat’ing /dev/urandom to a file – 0.82A

IBM eServer x336

Specs: Dual 3.0Ghz Xeon, 4GB RAM, Dual 575W Power Supplies, Dual 146GB SCSI drives

• Connecting Power Peak: 1.06A
• Stdby Steady: 0.79A
• Power On Peak: 2.5A
• Powered low load: 2.12A
• Loaded (7.0+ with disk): 3.25A
• Disk activity only: 2.40A
• Reasonable Load + Disk Activity: 2.85A peak
• heavy cpu/disk load – multiple instances of cpuburn and cat’ing /dev/urandom to a file – 3.2A

Some pieces of network equipment/drives I’ve tested:

• Cisco CSS 11151 Load Balancing switch – 0.89A
• Cisco CSS 11501 Load Balancer – peak startup: 0.69A – idle: 0.63A
• Cisco 2621 Router – peak startup: 0.14A – idle: 0.13A
• Cisco WS-3548-XL 48 port 10/100 switch – peak startup: 0.87A – idle (no ports connected): 0.61A
• Cisco WS-C2924-XL-EN 24 port 10/100 switch – peak startup: 0.39A – idle (1 port connected): 0.36A
• Cisco 1538M (8 port 10/100 hub) – 0.16A
• Cisco 1601 T1 Router – 0.08A (nothing connected)
• ——-
• Dell PowerConnect 5224 24 port GigE switch – peak startup: 0.43A – idle (no ports connected): 0.36A
• Dell PowerConnect 3248 48 port 10/100 switch – peak startup: 0.40A – idle (no ports connected): 0.35A
• Dell PowerConnect 3324 24 port 10/100 switch – peak startup: 0.22A – 8 ports connected: 0.2A
• ——-
• HP ProCurve 2848 (J4904A) – 48 port GigE switch – peak startup 0.69A – idle (no ports connected): ~0.51A
• NetGear FS524 24 port 10/100 switch – peak startup: 0.24A – idle (no ports connected): 0.21A
• ——-
• Juniper NetScreen 5GT – 0.06A
• NetScreen 10 – 0.06A
• Netopia 3386-ENT – 0.05A
• Adtran CSU/DSU – 0.01A
• BayTech DS2-RPC – 0.05A
• ——-
• 500GB Seagate SATA drive – Spinup: 0.31A – Duplicate large files – 0.11

Dave from NetApp has some interesting things to say about power in the datacenter.

Posted by Brian Blood as Colocation, Routers and Firewalls, Servers at 6:51 PM UTC

No Comments »