Skip to main content.
March 29th, 2007

Intel MacMinis – The OS X Blade Server

A really good client of ours has been colocating with us since late 2003. They’ve grown their web application from running on a Xserve 1.0Ghz DP G4 to an Xserve 2.0Ghz DP G5, then moving their database off to a separate big hardware RAIDed Dell server.

They came to us about a year ago (May 2006) and said they were getting a big new client who wanted to run their entire site on their system and they were going to need a load balanced system with plenty of power and scalability. Earlier in the year (Feb 2006), Apple had introduced their second generation MacMini that now sported the new Intel Core Duo chips along with Gigabit Ethernet. At that time, we were also concerned about increased power usage in our cage, so we picked up an Intel Mac Mini 1.66Ghz Core Duo, had it upgraded to 2GB of RAM (the G4′s could only handle 1GB) and started to really put it through it’s paces.

It turned out to be a real winner.

  • In terms of power consumption, no matter how hard I tried I could not make that Mini use more than 0.37A of power. I blasted that thing with multiple concurrent CPU and disk bound processes, getting really heavy loads and disk read/writes.
  • In terms of CPU, when we ran a battery of tests to really try and emulate this customers environment which is a very complex PHP web application. It really shone and had tremendous performance, even under load.

The final configuration we ended up with was:

MacMini Intel 1.66Ghz Core Duo, 2GB RAM and we replaced the stock Seagate 80GB 5400rpm SATA 2.5″ notebook drives with the Hitachi E7K100 60GB 7200rpm SATA drive. These are the drives that IBM puts on it’s blade servers as they are rated for 24/7 usage.

Total Cost: ~$1,000 each. (we bought the Apple ram)

We worked with the client to help factor their web application so that it could be properly load balanced. Changes were necessary in the following areas:

  • session storage
  • code updates (simultaneous CVS updates)
  • media upload handling (you can no longer assume you have the resources you did when you only had a single server)
  • host name abstraction to keep the Apache conf files nice and clean.
  • centralized logging of web hits/visits.

After all the hardware and software was ready, we setup the content rules for the load balancer and turned it on. It was very gratifying to see the Minis perform very well even under adverse load conditions. (The big client sends out large email newsletter runs that bring flash crowds to the site.)

One of the more interesting experiences we’ve had with this system was when we migrated their Xserve G5 in as web server #4 in the load balanced group. Mind you this is not a puny box. We even upgraded the drive system in it to a hardware RAID 5 based set. Since that time we have had to periodically adjust the weighting rules on the load balancer to give more and more priority to sending hits to the Intel Minis instead of the Xserve G5. We are now at a 3:3:3:1 ratio and the Xserve is finally at a lower overall load average than the minis.

Yes, you read that right: the Intel MacMini is somewhere between 2 and 3 times faster than a Xserve G5 in raw cpu performance.

And it uses one fifth the power. With a nice sliding rack tray, you can easily get 6 of them into a 2U space. (excuse the cabling mess)

MacMini 1 - 3


We’ve also considered a different configuration whereby we set the Minis on their side and “stack” them horizontally. The Mini is right at 2 inches tall and a 6.5 inch square, so accounting for some space for air flow and cabling you could get 7, maybe 8 of them in a row, resulting in a 4U tall set. With the right mounting, you could get easily 2, maybe 3 of these rows on a sliding tray. You do have to account for the external power supply, but that separation actually works out as a major benefit as the cables could be run so that you have a single U of dedicated space for the power supplies and put some directional cooling air flow over them.

Result: With only 2 rows of Minis, you get:

  • roughly the same CPU power as 28 Xserve G5s
  • about the same power consumption as 5 Xserve G5s (8.4A vs 9A)
  • 6-7 times less space (4-5U of Minis vs 28U)
  • easily less than half the per unit cost ($1,000 vs ~$2-$3K)

The Intel Dual Xeon Xserve looks promising as well in terms of raw cpu performance, but I’ve seen reports that it suffers from the same high power consumption as any other Dual Xeon system does: ~ 3A. However, this is TEN TIMES the power consumption of a MacMini, so for a high density web farm, this is not a better solution. Better to utilize that Intel Xserve as a database server to take advantage of it’s greater RAM capacity (32GB), increased threading (dual dual-core) and 64-bit capability.

Related CPU performance anecdote: We have another client with a Compressor video encoding grid made up of Intel MacMinis and he found out (Apple also confirmed it) that he needed to remove the PowerMac G5 DP from that grid as it was the weakest link!

In summary, a MacMini based farm is a powerful solution for almost any web application. You get low cost, low power, the ease of use and security of a OS X based system. A very compelling formula. Our client is very happy and is looking to add 2 more MacMinis in the near future. We’ve built this type of system for another client and they are extremely happy with the performance of their web application, too. If your organization is interested in a MacMini web farm, please contact us for a quote.

Some additional links regarding MacMinis

  • Installing Debian GNU/Linux on the Mac Mini go
  • 123Macmini.com – A Mac Mini User Community go
  • Mac Mini hacks Forum for the Apple Mac Mini go
  • Squeezing 2 MacMinis into a 1U custom case go

I’ve posted a followup to this article.

We did add 3 more to this setup a couple months later. Here is the gaggle:
six-minis

Posted by Brian Blood in Content Networking, Servers, Web App Development

15 Comments »

This entry was posted on Thursday, March 29th, 2007 at 9:54 am and is filed under Content Networking, Servers, Web App Development. You can follow any responses to this entry through the comments RSS 2.0 feed. You can leave a response, or trackback from your own site.

15 Responses to “Intel MacMinis – The OS X Blade Server”

  1. Blade Watch » Mac Minis are the next blade says:

    [...] http://www.networkjack.info/blog/archives/35 [...]

  2. Nathan says:

    In your performance tests, are these machines all running OS X, Linux, or some mix? You don’t directly address this. Although “security of a OS X based system” implies it, the link to Debian makes me wonder. It wouldn’t surprise me that an Intel distribution on Linux would be faster than anything else because of much better optimization.
    Also, the minis are Core Duo chips, not Core 2 Duos, they are 32-bit only. It wasn’t until the Core 2 architecture that Intel added 64-bit support back in.

  3. Brian Blood says:

    - The Minis are running stock 10.4.x
    - The Xserve G5 is running OS X Server 10.4.x
    - We do have a client that we built out a MySQL database server on a Dell 2850 running Debian Sarge.
    This system is extremely fast.

    - I never said the Mini’s were running Core 2 Duos. Perhaps you are just providing background information.

  4. scott schmitz says:

    We are running a LAMP based web portal from a mac Mini hosted at macminicolo and have been impressed with the performance as well.

    A few things that we noticed:
    1. There have been reports of MySQL having trouble with lots of threads. Something to do with problems with OS X thread scheduler. Apples has a tech note on a tweak to MySQL configuration you can make. We ended up turning off the mysql.allow_persistent with PHP so that persistant connections were not used. That improved performance for us on MacMini.

    2. We really miss having visual configuration tools like CPanel. Ended up using Tenon iTools. This has some visual capabilities, but not nearly what cpanel offers.

    Scott.

  5. Brian Blood says:

    Scott,

    We found the issues with using persistent connections in MySQL as well and turned them off.

    Please note that this particular architecture we built for this client is not a standalone server as your comment makes your situation sound like.

    I’m hoping you have proper redundancy/failover/backups for your system.

    As your situation grows, you will need have some way of scaling out your front-end servers. It also looks like (based on what I read on your site), you will need SSL termination once you reach a certain level of SSL based traffic.

    Brian

  6. Marcus Bointon says:

    We run a big web app on a cluster of Minis running Ubuntu, but we’ve run into a very nasty problem that’s only just been fixed. There appears to be a hardware bug in the ethernet flow control, and it can completely hang up the machine. There are two workarounds – there is a kernel that disables flow control altogether for the Marvell Yukon2 chipset the mini uses, and there’s a mechanism for spotting that the problem is about to occur and forcing the ethernet PHY to renegotiate its connect speed, which avoids the total hang up but takes the machine offline for a few seconds. More info here:

    http://www.mail-archive.com/netdev@vger.kernel.org/msg30907.html
    http://lkml.org/lkml/2007/2/1/307

    Aside from this problem, the minis really rock; so fast but with so little power consumption.

    We tried running with the system on a fast 3.5″ SATA drive mounted over firewire, but it didn’t really help much, so we went back to internal drives.

  7. Davis says:

    Hello Brian,

    What sort of cooling problems have you run into with the MacMini setup? Is your datacenter cooled with one large central A/C unit, or is each rack cooled individually?

    Thanks,
    davis

  8. Brian Blood says:

    Davis,
    We haven’t experienced any cooling problems whatsoever. The Minis put out so little heat themselves. Mainly because they use external power supplies.

    Equinix uses centralized A/C with sector based control and N+1 redundancy.

    Brian

  9. Dave Russo says:

    Mainly for Scott Schmitz..
    We’re using an old G4 Xserve that’s going to have to be reformated due to some OSX Server software problems (Intermittent reboots).
    In the meantime we’re going to use a MINI while the production machine is down.

    Any problems using Tenon iTools? Also any tips/tricks on moving the data over?

    Thanks..
    Feel free to use my email
    drusso@cat2.com

  10. Network Jack » Blog Archive » Intel MacMini Blade Server Followup says:

    [...] MacMini Blade Server Followup A followup to my article that generated a lot of traffic and [...]

  11. Dave Rutledge says:

    I’m not a hardware guy at all. But the replacement of the mini drive with the Hitachi drive interests me since my original drive just died after 3 months 24×7. Can anyone tell me if this replacement is an easy job for a local semi-Apple support person to do? Is it a quick operation? Thanks . . .

  12. Brian Blood says:

    Dave,

    The hard part about doing anything inside a MacMini is getting the case apart. After that, most operations are pretty simple as long as you are very careful.

    If you are unsure about it, it’s worth taking it to your local Apple Store to have them replace the drive.

    Brian

  13. John Russell says:

    I’ve found Mini’s perfect for this ‘blade’ type application. I’m devastated to learn at appleinsider.com that they may be discontinued. It may be worth forming a group to lobby Apple for something that fits this niche. jbr at onlywebdata dot com.

  14. Network Jack » Blog Archive » Ubuntu/Debian on an Intel MacMini says:

    [...] Bointon hinted as much in comment #6 to my original article and so when the Debian Mini developed problems communication over that [...]

  15. MacMini: il web server intelligente è su Intel - macitynet.it says:

    [...] Network Jack – un sito dedito alle esperienze in rete – è apparso un articolo di Brian Blood che [...]

Leave a Reply

You must be logged in to post a comment.