It used to scare me too, until I started using Hybrid Web Cluster. Now this isn’t a critical event any more. I can be developing on three virtual machines, one of them crashes, and I don’t even notice! All my websites and databases just carry on running as normal.

Find out more…

Hybrid Web Cluster is a cloud web hosting platform designed to run either on real servers, cloud server instances or a combination of the two. Due to some key enabling technologies becoming available (particularly the ZFS filesystem) combined with technology advances made by the cluster development team, this new product is able to offer a number of features not previously seen in products of this type:

• A user-configurable level of replication redundancy — Near-live backups can be stored on any number of nodes in the cluster and in the event of a node failure, service is automatically and instantly restored from a backup no more than 10 seconds old. In the event of an accidental deletion, files can be quickly and easily recovered by “rolling back time” – a feature provided in the web hosting control panel.
• Complete fault tolerance and no single point of failure — Any node (or several nodes) can fail and the cluster will automatically repair itself. Hardware failures are no longer critical, replacements can be carried out as part of a maintenance schedule rather than as an emergency event.
• A high degree of scalability — Standard LAMP web applications can run unmodified and scale from zero resource usage to requiring two dedicated servers (one for database and one for web) this scaling happens automatically and instantly to cope with variations in demand. With minor modifications to the application code, next generation multi-master database technology allows the cluster to scale even beyond the 2 server-per-site limitation and be capable of handling extremely high traffic loads.

After several years in development this new web cluster system is due to begin the first round of beta testing in September 2010 and Hybrid Logic Ltd. is seeking interested parties to try the beta version for free, initially on cloud infrastructure, but later stand-alone distributions will be available. Beta testers will be offered a discount on the full price of the system after its launch date.

You can view the video demo here: web cluster demo.

Traditionally web hosts have managed their own infrastructure

This means purchasing or renting server and networking hardware, a rack to put it in, power supply and internet connections to that rack, air conditioning, security etc. In order to have web servers online, all of these things need to be purchased and maintained. In large hosting environments, some way to spread load over the servers needs to be devised, as well as provision to cope with spikes in demand as well as some form of redundancy or backups to ensure that failures don’t spell disaster. This typically means having a significant amount of spare capacity and a large administrative workload.

In the past, utilising multiple servers to provide a web hosting platform has been fraught with problems – dealing with failures has been particularly tricky, and many web hosts know all too well that failure of crucial hardware can all-too-often spell downtime and hours of tense firefighting. Spreading load evenly and dealing with changes in demand rapidly to ensure that your users never experience slow-down is also a difficult problem to solve. Until now, much of this has needed to be set up and maintained manually and hardware failures are often a critical event.

By implementing some clever software load-balancing algorithms and making use of our revolutionary ZFS filesystem replication Hybrid Web Cluster is able to greatly simplify the process of administering a large number of web and database servers, whilst achieving bullet-proof levels of redundancy and being completely fault-tolerant. This means that Hybrid Web Cluster can be configured so that failures in multiple parts of the system will not result any outage at all. Hybrid Web Cluster has no single point of failure.

Hybrid Web Cluster works just as well with cloud infrastructure as with real servers

We are seeing an increasing shift towards a model of server infrastructure in which server instances are provided by cloud infrastructure providers, these are virtual servers issued from a pool of available computing resource maintained by the infrastructure provider. In this model the actual hard work of buying and maintaining physical servers (and spare capacity), networking infrastructure and internet connectivity is all done by the cloud infrastructure provider, they sell you server instances on this infrastructure – removing a lot of the hassle of administering server hardware, and turning the management of a web hosting platform into a problem which can be solved entirely and very effectively in software alone. Cloud infrastructure allows you to completely forget about the entire physical aspect of maintaining and operating servers.

Hybrid Web Cluster has been designed to work just as effectively with real servers as with cloud server instances – this gives you an amazing amount of flexibility in your choice of infrastructure. You can choose to run the cluster entirely on your own physical servers, giving you total control over your physical infrastructure, but requiring that you provision new hardware to cope with increases in demand, and replace failed hardware as failures occur. Of course, with Hybrid Web Cluster, a hardware failure is not a serious event and all hardware can be replaced at a suitable time of your choosing.

The real power comes when you combine real hardware with cloud computing server instances – you’re given control over how you utilise cloud computing resource – you could elect to have normal hosting operations provided entirely by your physical hardware, but for all data to also be kept on cloud servers which would be ready to go instantly if your physical hardware dropped off the internet. You could also opt to have your hosting divided evenly between physical hardware and cloud resource – because you can mix and match real hardware and cloud servers like this, Hybrid Web Cluster enables you to use cloud hosting resource to instantly dynamically change the size of your web cluster to match changes in demand. You could choose to have a setup where under normal load, all hosting would be provided by your own hardware but under particularly heavy load, cloud server instances would be automatically spun-up so that your hosting never appears slow – in this setup you’d effectively be using cloud servers like an overflow, to cope with peaks in demand. This enables you to make full use of your existing hardware investment, whilst still taking advantage of the enterprise scalability and redundancy of the cloud.

It is also possible to run Hybrid Web Cluster entirely on cloud server instances – with this setup you gain complete freedom from the day-to-day burden of maintaining physical hardware, as well as being able to take full advantage of all the scalability and redundancy features that are built-in to the cluster as standard.

If you would like to take advantage of the incredible flexibility on infrastructure choices that Hybrid Web Cluster offers, contact us now to discuss your requirements or try a 48 no obligation free trial.

Hybrid Web Cluster offers a powerful, yet easy-to-use web hosting control panel

At the core of Hybrid Web Cluster is a web hosting control panel responsible for adding sites and databases, managing e-mail settings, automatic installation of common applications such as WordPress. The control panel offers a single point of access allowing you to make configuration changes affecting the whole cluster in what appears to be one centralised location (of course the control panel is resistant to failure as any website hosted on the cluster would be). The Hybrid Web Cluster control panel offers all of the features you would expect in a modern web hosting control panel, and its innovative, simple user interface allows even novices to set up and administer their own websites in an intuitive and well thought-out way. You are offered all the flexibility you need, without being blinded by endless confusing options.

The control panel provided by Hybrid Web Cluster feels familiar to anyone who may have used shared hosting in the past, and in fact many of those using it may not even be aware of the clever technology working behind the scenes to keep their website online and responsive every hour of the day.

Behind the simple web hosting control panel lie a powerful set of tools for visualising the state of the cluster

Although Hybrid Web Cluster is almost completely self-managing, it is still useful and interesting to be able to see what’s going on in the cluster, live, as it happens. Within the cluster administration control panel, a powerful set of visual representations of the cluster allow you to see which servers are hosting which websites, how much load each individual node or website is experiencing at any given time, as well as allowing you to actually watch as websites are sitejuggled around the cluster.

These same tools allow you to make configuration changes to the global state of the cluster. For example, an important variable you can configure is the redundancy invariant – this number sets how many machines should hold a hot spare of each website or database. The redundancy invariant is a trade-off between total available disk space against fault-tolerance, the more fault tolerance you have, the less total disk space you get. By default we set the redundancy invariant to 2, meaning each website will always be stored on at least two servers, this gives the maximum amount of disk space and the minimum level of fault tolerance, but it still means if an entire server goes down you don’t lose any data.

Hybrid Web Cluster also offers a powerful reseller control panel

The reseller control panel is responsible for the administration of user accounts, the allocation of resources and various tools for managing web hosting users or customers.

If you would like to make use of the powerful and easy-to-use Hybrid Web Cluster control panel, contact us to discuss your requirements or try a 48 hour free no obligation trial.

Load-balancing has traditionally been difficult to do effectively

In a web hosting environment, when you need to scale to more than one server, a naïve approach is to simply split your websites so that you have say 100 located on one server, the next 100 located on another server and so on. This is how many web hosts have operated in the past.

However, a problem comes when two or three of the websites on a particular server become very busy and cause it to slow down. The problem is, you have spare capacity on other servers, but you are not able to utilise that capacity because the three busy websites in question all happen to be on the same server. In the past you might have manually moved these websites onto separate servers to more evenly spread the load and make more effective use of available hardware, but the limitation of this approach is that it’s a manual process that requires constant management as the workload grows and changes.

What you really need is a way of automatically balancing the load on your servers so that no single server is ever getting so much load that it becomes unresponsive or worse, crashes. In the past a number of different approaches have been taken to solve this problem, but all of them suffer from limitations and flaws.

Hybrid Web Cluster’s load balancing is done entirely in software – no expensive hardware devices are required, just normal commodity servers or virtual machine instances. This eliminates a single point of failure in the hardware load balancer itself, and makes Hybrid Web Cluster a more robust system.

Hybrid Web Cluster offers a revolutionary new type of load balancing

The load balancing system in Hybrid Web Cluster works by splitting websites over a number of servers, say 100 on each server as in the example above, but the crucial difference is that Hybrid Web Cluster keeps a close watch over the load generated by each individual website or database, and its sophisticated algorithms make judgements about when to transfer websites from one server to another to spread load more evenly. These transfers happen transparently to the end user, competely automatically, we call this process site-juggling.

This means a website can go from having no traffic at all and sharing a server with 99 or more other websites to getting a massive spike in traffic, say if the website address is featured on TV – Hybrid Web Cluster simply reacts by moving all of the quieter sites hosted on the same machine to another node, leaving the original machine as a dedicated server for the busy site in question. The same process happens for databases, meaning the system can automatically scale from a site requiring virtually no resources, to that same single website requiring a dedicated database server and separate dedicated web server, in a matter of seconds.

Hybrid Web Cluster runs equally well on real hardware and cloud server instances, this means websites can be juggled from one to the other by the load balancing algorithms according to conditions that you specify. This allows you to spread load evenly between real servers and cloud, or use cloud servers as an emergency overflow to cope with peaks in demand. The site-juggling algorithms in Hybrid Web Cluster allow the size of the cluster itself to grow and shrink as demand changes, allowing the cluster to automatically spin up new cloud server instances to cope with peaks in demand.

Hybrid Web Cluster’s powerful visualisation tools allow you to view all of this happening live on your cluster, and of course monitor disk and bandwidth usage of each website or user. This gives you all the business intelligence necessary to make decisions on pricing or account quotas.

If the unique load balancing capabilities offered by Hybrid Web Cluster sound like something that would be useful to you, contact us to discuss your requirements, or try a 48 hour free no obligation trial.

Hybrid Web Cluster provides the standard Apache + MySQL + PHP software stack

Because Hybrid Web Cluster provides a standard web hosting platform, the vast majority of standard web applications will run unmodified out-of-the-box. In fact, quite a few of them can be automatically installed via our easy-to-use web hosting control panel.

This list is not intended to be a complete application compatibility list as that would be endless, rather, it is a list of those applications which we consider commonly used enough to regularly ensure that they function as expected on Hybrid Web Cluster. If an application is not on this list, that does not mean it will not work – if it is built to run on the standard LAMP stack, it is very likely that it will. This applies equally well to generic applications and bespoke web application development.

Hybrid Web Cluster is built upon the FreeBSD operating system

We chose FreeBSD as the core operating system for Hybrid Web Cluster for a number of reasons; we benchmarked ZFS, one of the key technologies that the cluster relies upon and found that it performed well on Solaris and FreeBSD, and less well on Linux. Much of the early development work on the cluster was done in Solaris, as this was the first operating system to offer ZFS support. However, when ZFS became available in FreeBSD, the switch away from Solaris was an easy decision to make. FreeBSD is tried-and-trusted in web hosting environments, it’s also much closer to Linux in its commands and directory stucture, to the point where most users will not be aware that what they are using is FreeBSD rather than Linux. This is much more difficult to achieve with Solaris as it has numerous subtle differences to the norm.

Although Hybrid Web Cluster is currently designed to run only on FreeBSD, future ports back to Solaris and forwards to Linux are both possible and likely. Nevertheless, even running on just FreeBSD, Hybrid Web Cluster is able to run pretty much every standard LAMP stack web application.

Hybrid Web Cluster provides a MySQL database server

On top of FreeBSD, Hybrid Web Cluster provides access to the industry standard MySQL database server – each website’s database is transparently load-balanced around the web cluster, you don’t have to worry about where your database is located, the cluster will ensure you can always access it, no matter which cluster node it is located on at any given time, or even as it moves from one node to another.

Hybrid Web Cluster runs the industry standard web server Apache

Websites are served via the standard Apache web server, with important modules like mod_rewrite available and the ability to set custom options via .htaccess files as you would expect. In a similar way to databases, each website is load balanced around the cluster and may move from one node to another at any time. Hybrid Web Cluster takes care of this for you so that it’s completely transparent, and you just see a normal instance of Apache.

Hybrid Web Cluster includes a standard version of PHP with all the bells and whistles enabled

On top of all this rock solid hosting software, Hybrid Web Cluster provides a normal installation of PHP in which scripts are executed as you, so you never experience permissions or security problems again. We ensure that PHP is configured with all the commonly used extensions, so you can just drop your standard LAMP application into the web cluster and it will perform as expected.

If you’d like to run standard LAMP stack applications on a fault-tolerant, highly scalable web cluster that runs equally well on real hardware and cloud server instances, you should contact us today to discuss your requirements, or try a 48 hour free no obligation trial now.

Gprof2Dot outputs pretty awesome graphs. This is what you get when you run AwesomeProxy for three minutes at 10 requests per second. I really like how you can see the structure of the code