The BBC News website - under the bonnet

By Kevin Hinde, Head of technical development, BBC News website
BBC News

June 3, 2005

This is particularly important on big news events, when the site comes under its heaviest loads. On the day of the UK election results this year we had 4.3 million users and just over 50 million page impressions.

The technology which serves the site is designed to be as simple as possible. The simpler the site, the cheaper it is to run. There are fewer elements which can malfunction on big days; and there are fewer parts which can be compromised by someone trying to gain unauthorised access.

Unlike other sites which carry advertising, we do not receive more revenue the more people use our site, so it is important that we are able to serve as many people as possible and keep costs low.

London and New York

We have a number of web servers ("server farms") in London and New York. These two cities are both excellent hubs connecting many different networks on the internet, and they are far enough apart so that if there were a major disaster in either city we could continue serving web pages from the other location.

The Domain Name System (DNS) is the Internet's method of translating domain names, like "news.bbc.co.uk" into computer addresses. We use a modified DNS server which is able to balance requests for web pages evenly over all of the web servers we run.

Users from the UK are sent to the London server farm, which is paid for from licence fee money. International users go to the New York server farm, which is mostly funded by the grant-in-aid from the British government, which pays for the BBC world service.

You can see an overview of our network here .

Cheap bandwidth

We "peer" with most internet service providers (ISPs) in the UK. "Peering" means that we agree to exchange traffic between our network and the ISP network without charge. The ISP benefits because their customers are able to download BBC content more quickly; the BBC benefits because it is able to reach almost all of the internet users in the UK without incurring massive bandwidth charges.

The servers themselves are running Apache web server software on either the Linux or Solaris operating system. They are configured so that the web server software and the operating system run only the minimum number of modules and add-ons to do what we need.

Two editions

At the top of every BBC News webpage is a "change edition" link. We have two editions of the site; the same stories are available on both, but there is a different emphasis on the index pages. For example, the UK edition might lead with a story about UK interest rates where the international edition would have a story about the European parliament.

If you click on the "change edition" link the web server sends a small file, called a "cookie", which is stored on your machine. The next time you visit our site, your web browser sends the cookie back to us the server reads the cookie and returns the correct edition.

The first time a reader comes to the website, they won't have a cookie. In that case, the web server just sends the UK edition to readers who come to the UK server farm, and the International edition to everyone else. Apart from that, there is no connection between the edition which is served and the server farm it is served from - users in the UK can get the International edition and vice versa.

Dynamic content

The web pages themselves use a technique called Server Side Include (SSI) to provide a dynamic view of the contents of the site. When a reader requests a web page, the web server reads the page and looks for SSI instructions, which tell the web server to replace that part of the page with the contents of a file.

For example, most stories have an "other stories in this section" box on the right-hand-side. The contents of this box come from an SSI instruction, so the reader always sees an up-to-date list of current stories, even if the story itself has not changed.

To squeeze the maximum possible performance from our web servers, we have moved the picture files to a different set of servers ("newsimg.bbc.co.uk"). The image servers do not need any of the modules to process SSIs or decide which edition to serve, so they can be minimally configured.

Copyright 2005