“ I have questions about the web cluster, do you have the answers? ”

What is a web cluster?

A web cluster is a distributed system of computers or servers. In other words, it’s a group of computers which appear to the outside world as a single entity. This means that you can upload a site to the cluster, make a change to the site, and it’s totally transparent to the end user that the site might have been moved from one server to another within the cluster, or that the server the site was on might have failed and been automatically replaced. To the outside world, the group of servers behaves as one, and a failure in any point of the system will not result in any outage.

What is cloud computing?

Computing is increasingly moving towards a model in which server resources can be provisioned instantly, de-commissioned just as quickly and are available on-demand as requirements change. This provides enormous benefits, but in order to take advantage of those benefits you need a platform layer to sit on top of the cloud infrastructure and allow your application to make use of it. This is where Hybrid Web Cluster comes in.

What is scalability?

Scalability is the ability of a system to grow and shrink as demand changes. Scaling to large hosting environments has in the past been hampered by the administrative burden of provisioning new servers and finding ways to spread the load evenly across them. Until now, solving web scalability problems has been a difficult and time-consuming process. Hybrid Web Cluster solves the problem of scalability by allowing your web hosting environment to spread load evenly across any number of server instances, without requiring you to re-write your standard web applications.

What industry-standard software does Hybrid Web Cluster utilise?

Hybrid Web Cluster is built to run the FreeBSD operating system on each cluster node, and provide the standard Apache, MySQL and PHP stack to websites and applications which run upon it.

What common web applications work with Hybrid Web Cluster?

As Hybrid Web Cluster provides the standard Apache, MySQL and PHP stack, any application written to use these technologies will work, without requiring any tweaking. We have tested the following web applications are tested and can certify them to work out of the box with the web cluster:

View the complete Hybrid Web Cluster application support list…

What is redundancy?

Redundancy is the ability of a system to resist failure by having one or more backup systems ready at all times to take over the load should a failure occur. Hybrid Web Cluster offers a customizable level of redundancy – you can choose to have each site backed up on any number of nodes in the cluster – starting at 2 for basic redundancy, all the way up to the total number nodes in the cluster for rock-solid disaster tolerance.

If you have cluster nodes in more than one data-centre, you can also configure the cluster so that no matter which data centre a website is being hosted from, a copy is always stored off-site on a node in one or more of the other data centres – this makes Hybrid Web Cluster not just bullet-proof, but nuclear bomb-proof too.

What is load balancing?

Load-balancing is the process of spreading load evenly over a number of servers so that no single server is overloaded to the point of becoming slow and unresponsive. Effective load-balancing is crucial in making the most out of available server resources and consequently delivering the best ROI.

What tools does Hybrid Web Cluster provide to help manage my cluster?

Lots of work has gone into making Hybrid Web Cluster as self-managing as possible – during the day-to-day running of the cluster very little maintenance is necessary, we have done the hard work of automating all the difficult processes so that you don’t have to. However, it’s still useful to be able to monitor the state of the cluster and see who’s using what resources, make decisions about redundancy levels and gain business intelligence on your user base. All of this is provided in the cluster administration control panel.

Can I define what role a server should be (like web or database server) so that I can optimize the hardware for it?

We’ve found in practice it’s easier to manage a homogeneous cluster – i.e. every node is fairly similar to the others. This means that no one node can ever be “special” and thus form a single point of failure. The cluster software will spread the disk and CPU load of the websites evenly across all the servers, so you can use cheap commodity disks. Given the low price of RAM, running every service on every server works well in practice. We recommend 4-8GB of RAM per server.

Note that if you do have heterogeneous hardware (some servers with large disks, some with large memory) you can still run the cluster software efficiently on it — the cluster’s load balancing will “fill” each server as much as it can.

How does Hybrid Web Cluster achieve fault-tolerant DNS?

Suppose you’ve got 12 servers, 6 in each data centre. We’ll pick two in each data centre to be the primary nameservers:

ns1.yourcluster.tld in data centre A
ns2.yourcluster.tld in data centre B
ns3.yourcluster.tld in data centre A
ns4.yourcluster.tld in data centre B

This provides very high probability that at least one nameserver will be reachable. Now for the actual A records for each domain you host, these are automatically configured by our cluster software to resolve to multiple A records:

1. The current live server for the site
2. All the other servers in the cluster

This provides very quick failover since modern web browsers will switch over to an auxiliary A record within seconds of the primary failing.

Note that this relies on the fact that with our web cluster you can direct any request to any node and it will be internally proxied to the correct server. This means you don’t need a hardware load balancer. Furthermore the cluster does load balancing by moving entire sites from one server to another. This is an excellent fit for commodity web hosting, where you have a large number of websites.

Do you have a list of links to related websites?

We have a list of links to associated reading which will help fill-in the background on the technologies we use and the capabilities of the Hybrid Web Cluster system.

View Related Links page…

Can I maintain my own physical servers? What about using cloud infrastructure?

Hybrid Web Cluster gives you a great deal of flexibility over your infrastructure choices. You can opt to have the cluster hosted entirely on your own physical hardware, or you can host it entirely in the cloud or, for the power and flexibility of both approaches, you can combine them in a hybrid setup.

I currently have to login to a separate control panel for each server, how will I cope with an entire cluster of servers?

Hybrid Web Cluster’s powerful web hosting control panel provides a single point of configuration and access across the entire web cluster. You don’t have to figure out which server to login to, there is only one control panel. This simplifies administration and allows you to treat the whole cluster as a single web hosting platform, rather than having to worry about administering each server separately.

How does Hybrid Web Cluster divide up the workload so that it can be spread over multiple servers?

Hybrid Web Cluster divides each website or database it is hosting into individual units of work, each has its own separate ZFS filesystem and these are replicated around the cluster.

The load balancing algorithms in the cluster work by automatically moving individual websites or databases between cluster nodes to cope with changes in demand. This is very similar to the way commodity shared web hosting has traditionally coped with the problem of spreading load over servers – most commonly web hosts will put the first 300 websites on one server, the next on another and so on. The difference with Hybrid Web Cluster is that these websites can be transparently and automatically shuffled between servers as demand changes.

What’s with the ants?

Ants, like bees and neurons, work co-operatively together to accomplish things which the individual on its own could not. Similarly, our cluster is made of components (servers) which individually could never provide the load-balancing and redundancy guarantees that they can when they work co-operatively together.

Have we failed to answer your question?

Let us know and we’ll get back to you straight away with an answer. We might even put it on this page!

How is shared storage managed in this type of cluster?

In summary: in a true distributed fashion, with full redundancy. Technically, there is no shared storage, as all storage is local to the server which is hosting any website. This makes it fast, but by replicating changes to the data, we can also make it safe.

Our storage system works on the principle that in order to divide up the workload in a clustered web hosting environment, you can split up the data across multiple commodity servers based on website, database, and mailbox. We assume that up to N servers will fail at any moment, so we also stream live backups of each website, database or mailbox to a configurable number of backup servers for that dataset. When a server fails, the websites (and databases, and mailboxes) that were on it are recovered within seconds from the live backups on the backup servers for that dataset in the cluster.

The benefit to you is that you can, for example, use cheap SATA disks across a bunch of cheap 1U servers and expect them to fail. The software deals with recovering from hardware failure so you don’t have to. You don’t need to spend lots of money on expensive SAN or NAS systems, and because everything works in a truly distributed fashion, there is no single point of failure in any one of the servers at any time.