Deploying a Web Site: From Development to Staging to Production
June 8, 2001

by James Mendelsohn

The logistics of Web site deployment has an extremely important effect on a developer's ability to create and modify a site and on the likelihood a site performs according to expectations. If these matters go unheeded, the results can be disastrous.

Simply put, moving from the development to the staging and then to the production platforms of a Web site requires:

• Establishing a means and location for authors to create and modify content, which means not only deploying a content management system but maintaining the integrity of the production server.
• Ensuring that the development of a site leads to one that appears and behaves as expected once live, which means comprehensive testing in advance.
• Producing and modifying the actual site, which means facilitating its rapid and easy modification -- from upgrading of features to changing, adding, or deleting content.

Deployment issues divide neatly into a few principles. To that end, this article is organized into three parts, designed to help you decide how to deploy:

• Principles of Deployment - briefly stated rules for the process of deployment
• In Greater Detail - deployment suggestions on each of the three servers
• Outsourcing Considerations

Principles of Deployment

You might want to consider the following as cardinal rules for deploying a Web site:

• Use three separate pieces of hardware:
  • Development server
  • Staging server
  • A production server or, more likely, production servers that are identical in platform -- that is, operating system and file system structure -- to the staging server
• Choose a scheme for replicating data to the production server that is compatible with the size and complexity for updating the site.
• For security purposes and to avoid the accidental deletion of files by those editing content, allow only a few engineers access to the production servers. Confine content authors to the development server.
• When Web content is modified, think carefully about the order in which those modifications are pushed through to the production server.

In Greater Detail

More specifically, you may want to consider the following details, which divide according to the three pieces of server hardware this article recommends.

The Development Server
The development server is the playground. On it, a developer writes and builds the site. If the site is not too large, then FTP will be the more likely means of transferring data files from this server to the next. Development server considerations include:

• Content Management Software
• Development Software

Content Management Software
For ease of ability to publish, bear in mind that not all development or modifications will be done by those with technical backgrounds. To that end, content management software may be more useful if it includes:

• Presentation templates for converting and presenting content in formats such as HTML, XML, WML, and PDF
• Additional ability to create presentation templates
• Centralized control of site design elements
• Easy capacity to define navigation rules and generate site navigation accordingly
• Ability to select content criteria
• Capability to define site hierarchy

Since the site should separate back-end issues and access from content-publishing functions, the content management solution should also include:

• Ability to prepare and automatically post materials to the site (self-service authoring)
• Browser-based authoring (so desktop authoring software is unnecessary)
• Version archiving and audit trail
• Integrated workflow
• Content scheduling

Additionally, you may find it useful if the administrative functions of the content management software include the capacity to:

• Manage users remotely
• Establish and monitor security
• Create workflow processes
• Categorize content both automatically and according to in-house rules (metadata)
• Archive content, preferably in a repository that allows for its reuse

Selecting Content Management Software
In addressing which (content management) solution fits, bear in mind the following:

• Site requirements, such as:
  • How often content should be updated, including whether changes need to be made in real time
  • Type and extent of content changes, including how many pages a change of content affects and the complexity of identifying all those pages
  • If personalized or tailored content is used
  • If approvals are necessary before content is changed
  • Number and location of those authoring content to the site
  • Whether or not the site has different kinds of content, such as image or video files
  • Support for different languages

Back-end Development Software
Just as important, development software must include source code control software (SCCS), revision control system (RCS), or concurrent versions system software (CVS). Performance-testing software is also essential, and an integrated development environment (IDE) may be helpful (for example, JBuilder or the Forte product line).

The Staging Server

Identical with the Production Server
Once developed, the site needs to be moved to the staging server, whose operating system and file system structure are identical in kind to the production server(s) even though it is not live. (Typically, the staging server doesn't have as powerful a CPU or the performance capacity of the production servers, but the platform should be identical.) That means the staging server hardware:

• Should be identical in software configuration to the production machine, so it is a true basis for QA testing.
• Should be comparable to the production hardware. In that way, a site administrator can get an accurate forecast of capacity by performance testing against it and then multiplying its performance by the number of machines that will be deployed in production.

Stage Testing for QA and Bugs
On the staging server, the site should be tested for bugs and in general for quality assurance (QA). Therefore, whatever version of the site is moved to the staging server, it should include source code control, so that an administrator can back out changes, either automatically or manually. (No direct editing occurs on the staging server. The problems are found on the staging server but addressed on the development server, where the source code control software is.)

QA
For QA, you need to make sure that:

• Hyperlinks are all valid.
• Navigation for the entire site works, such as:
  • Landing pages for all sections of the site
  • All links on each of the pages, including such sections as Help, Privacy Policy, Terms and Conditions, Advertising Sales, About Us, and Contact Us
• Site search functions work.
• Components, special functions or features work -- from calculators and polling features to applications.
• Live feeds work.
• Pages look the way developers want them to, including having all advertisements appear as expected.
• Pages load quickly.
• The site passes any additional load testing, script testing, and database connection testing.
• Spelling on each Web page is accurate.

Managing Bugs
To ensure efficient and rapid handling of bugs, you may want to centralize information on all bugs, creating a single database for testers reporting on such problems. (For larger sites administered in-house, this may well require you to assign a single person to oversee the record and verify all bugs are fixed.)

As this last suggestion implies, you may want to create a process for fixing bugs, including:

• Centralizing the reporting of all fixes
• Determining which bugs are highest priority and scheduling corrections accordingly
• Meeting regularly with your team to review bugs
• Distinguishing between fixes and added features and functionality -- using a separate process and database for any modifications to the site

The Production Server: Going Live

Mirroring: Manual FTP, Rsync, and Perl Scripts
At last, the production server, on which the live Web server resides: the final destination. For the staging server to be effective, the production server needs to be identical in kind, and the mirroring of the site requires some kind of synchronizing protocol. For modest changes in a site, an administrator might be able to use FTP manually to move each file from the staging server to the production server as a means of replicating. Most sites are too sophisticated for such a treatment, though, and as the modifications increase, manual replication won't be practical. To ensure mirroring occurs fully, you will need to use rsync, a homegrown Perl script, or some other program.

Moreover, rsync and Perl scripts have the logical ability to look only at changes to the site files -- the deletions and modifications -- whereas that function is at best difficult with FTP. With rsync and Perl scripts, synchronizing will be more efficient. File management will also be better: These scripts will automatically remove files from the production server that have been deleted in the development server, ensuring that these deletions and any other changes made are propagated throughout the three pieces of hardware.

Publishing Modifications Prudently
Deployment and staging issues arise each time a site is modified. For just that reason, you may want to pay special care to the logistics of publishing. The point is to avoid a broken site. For example, if a site takes 30 minutes to establish a link to an article that's been added to the site, the Web site will have a broken link unless the link itself is the last item to be published. You may want to begin with the graphics, follow with the article, and once everything is synchronized, publish the page containing the link.

Hardware Connections for Code Deployment: Enhancing Security and Performance
While deploying new code, an administrator may want to establish a secure (VPN) connection to the production hardware for security and performance reasons. In effect, that would mean creating dual interfaces on the production hardware: The connections from the VPN would go into the private interface card while the customers would connect to the Web server from the public interface card.

That same private interface can also be used to connect the Web server to a database, which would enhance its security and performance because the same network wouldn't have to carry both database and Web site traffic.

Outsourcing

Hidden in this discussion is whether or not to outsource deployment. Determine what solution fits the size of the site being established and your budget. A large site and limited resources may well mandate some outsourcing. For example, a site with a large publishing model -- one that demands sizable amounts of content and frequent changes to it -- may need to outsource publication of content. Those decisions will require a developer to decide not only how large the site is but how much money it can provide for the IT budget.

Consider that if a site deploys a cluster of servers, then a commercial content management system may be necessary. Still, that decision need not be made in anticipation of a cluster. You can always switch to a commercial product as the site grows to demand it.

If outsourcing Web content management, you may well want to pay particular attention to these subjects:

• Integration with the platform and database - Since some amount of customizing is likely to be necessary, integration issues should include evaluating whether the solution's API set makes other applications accessible.
• The solution's scalability, fault tolerance, and failover abilities