Wikimedia Foundation

Story Details

Founded in 2001, the Wikimedia Foundation hosts a variety of wikis such as Wiktionary, Wikiquote, and Wikimedia Commons, but by far it is best known for its free online encyclopedia, Wikipedia. Among the top-10 most visited Web sites in the world, Wikipedia had nearly 250 million unique visitors in October 2008. Articles are written and edited by more than 100,000 volunteer editors worldwide. Today, the English-language site hosts about 2.8 million articles; in total, about 11 million articles are available in 265 languages.

Wikipedia is built on Sun's MySQL database and runs in a distributed multi-tier architecture using Lucene, memcached, and Ubuntu–based clusters of servers. The largest site is in Tampa, Florida with 300 servers. Additional datacenters in Amsterdam, the Netherlands, and Seoul, South Korea operate more than 20 servers each. The Web site receives between 25,000 and 60,000 page requests per second, depending on the time of day. Wikimedia needed to update its infrastructure to handle this huge volume of traffic and ensure that its systems were reliable, highly available, and easily scalable. It also wanted to expand its upload file limit from 20 MB to 100 MB to accommodate rich media (audio and video) content, but before it could do that it needed to expand its storage capacity.

In November 2008, Wikimedia announced that it would partner with Sun Microsystems to help it build a secure and scalable open Web infrastructure. Through Sun's Startup Essentials program, Wikimedia acquired the Sun Fire X4150, Sun Fire X4240, and Sun Fire X4540 servers, running the Solaris 10 Operating System with the Solaris ZFS file system and MySQL. The Sun 4540 Storage Server provides low-cost, high-density storage and eight-way processing in a single enclosure. “We looked at other vendors, but they couldn’t get up to the scale we wanted; they didn’t offer support like ZFS backups, snapshotting and self-healing; or they were just incredibly expensive SAN systems that were totally proprietary,” says Brion Vibber, chief technology officer at the Wikimedia Foundation.

“We chose Sun Fire servers for their excellent price-performance ratio. The hybrid Sun Fire X4540 Storage Servers are very exciting because they offer a huge amount of storage and also deliver superior processor performance when we have a lot of read/write activity or a large number of uploads. Sun Startup Essentials provided the scalable storage solution we wanted at the price we were looking for,” says Vibber. With the exceptional storage capacity of these unique storage servers, Wikimedia was able to raise the upload file size to 100 MB, opening the door to more multimedia content to enrich the experience of the users.

Data integrity was one of the Wikimedia Foundation’s primary concerns. On the Linux servers, data was backed up to direct-attached RAID arrays. “If a disk failed, we could replace it, but if a whole machine died, we were out of luck. We needed a way to keep a standby disk active,” says Vibber. On the Sun Fire X4540 Servers, which are configured in pairs, the Wikimedia Foundation uses Solaris ZFS replication to ensure data integrity. “With ZFS, we’re able to replicate in real time and store multiple backup snapshots on each server,” says Vibber. “It’s a big improvement because we no longer worry about losing all of our work if a system fails.”

Vibber says it was also very difficult for the Foundation to expand storage with the configuration it had. “When we needed more storage, we would add another Linux box with a big RAID array, and then copy directories around and split up everything manually,” says Vibber. “Now with ZFS, we can add disks and extend the ZFS pool, and we’re done — the space is instantly available. It doesn’t require us to replace everything if we need to expand, and we’re very happy about that.”