No longer a critical event

August 6th, 2010

web cluster redundancyDoes this scare the hell out of you?

It used to scare me too, until I started using Hybrid Web Cluster. Now this isn’t a critical event any more. I can be developing on three virtual machines, one of them crashes, and I don’t even notice! All my websites and databases just carry on running as normal.

Find out more…

Press release: Beta testing due to begin in September

August 1st, 2010

HEADLINE: New cloud web hosting platform looking for beta testers

Hybrid Web Cluster is a cloud web hosting platform designed to run either on real servers, cloud server instances or a combination of the two. Due to some key enabling technologies becoming available (particularly the ZFS filesystem) combined with technology advances made by the cluster development team, this new product is able to offer a number of features not previously seen in products of this type:

  • A user-configurable level of replication redundancy — Near-live backups can be stored on any number of nodes in the cluster and in the event of a node failure, service is automatically and instantly restored from a backup no more than 10 seconds old. In the event of an accidental deletion, files can be quickly and easily recovered by “rolling back time” – a feature provided in the web hosting control panel.
  • Complete fault tolerance and no single point of failure — Any node (or several nodes) can fail and the cluster will automatically repair itself. Hardware failures are no longer critical, replacements can be carried out as part of a maintenance schedule rather than as an emergency event.
  • A high degree of scalability — Standard LAMP web applications can run unmodified and scale from zero resource usage to requiring two dedicated servers (one for database and one for web) this scaling happens automatically and instantly to cope with variations in demand. With minor modifications to the application code, next generation multi-master database technology allows the cluster to scale even beyond the 2 server-per-site limitation and be capable of handling extremely high traffic loads.

After several years in development this new web cluster system is due to begin the first round of beta testing in September 2010 and Hybrid Logic Ltd. is seeking interested parties to try the beta version for free, initially on cloud infrastructure, but later stand-alone distributions will be available. Beta testers will be offered a discount on the full price of the system after its launch date.

Awesome profile visualisation

June 26th, 2010

This is call profile graph of my latest invention, AwesomeProxy. This lets us move sites and databases between servers without a single failed HTTP request. Working on some optimisations, I wanted to see how much time was spent in each function call.

Gprof2Dot outputs pretty awesome graphs. This is what you get when you run AwesomeProxy for three minutes at 10 requests per second. I really like how you can see the structure of the code :)

Variations on a logo

June 1st, 2010

Well, I’ve just put the new logo live. I’ve also done some smaller images which won’t look out of place on CloudBook.

What do you think?

New logo!

June 1st, 2010

I’ve just received our new logo from a friend of mine who’s also an awesome designer in Bristol and I think it looks great!

The problem with our old logo (above) is that it looks like the ant has been dropped on top of the orb with no real context. But around a month ago someone I was talking to suggested that there should be ants holding the globe!

So here it is:

I’m excited about this because it means we’re starting to get a real corporate image, something we can put on business cards and letterheads!

Hybrid Web Cluster goes multi-data-centre with awesome redundancy, scalability and manageability.

May 8th, 2010

I just thought I’d share some internal Hybrid Logic correspondence with the world, as it’s pretty exciting…

From: Luke
To: Mike, Rob, Kieran

Hey guys,

I’ve successfully set up our first cross-datacentre web cluster today!

Three servers are in our rack at Telehouse and three are hosted with ElasticHosts (also in London). The most impressive demo of course is turning off an entire data centre, which I just simulating by doing a hard power-off on all three ElasticHosts machines simultaneously *just* after updating a WordPress blog whose database was being hosted there.

I held my breath, counted to ten, clicked refresh, and voila, my latest post was still there :D

I’ve also got real failure-resistant DNS set up, no more /etc/hosts hacks. All websites resolve to all six IP addresses (with the current live one first in the list). All modern web browsers fail-over to a new IP within one second of the current IP failing. This is serious redundancy which you usually have to pay a lot of cash for.

More good news – ElasticHosts have just announced a new data centre in San Antonio, USA. Imagine a “Prefer my site to be hosted in the US” checkbox in our control panel… that’s gonna be a reality.

It’s looking good!

Cheers,
Luke

Mike replied with a few questions, which I answered:

I just had a thought. Does this also make it an awesome migration tool? For example if I wanted to move all my client sites to our own web cluster it would be an admin nightmare. However if we used HL to add web cluster machines to our existing network could we then quietly turn off our existing servers and our world keep spinning?

If you’re trying to move from one data centre where the web cluster software can be installed at both then yes, it’s a life-saver. Even if you had to move physical hardware from one location to another, you could do it by simply unplugging one server at a time (everything carries on working), moving them from A to B (also one at a time), and telling the web cluster where to look. The transition would look like this:

(Site A – old data centre, site B – new data centre, each * represents a server)

        A * * * * *    B
        A * * * *      B *
        A * * *        B * *
        A * *          B * * *
        A *            B * * * *
        A              B * * * * *

And voila, you’ve seamlessly *moved data centres* without a minute’s downtime for any of your sites.

It doesn’t solve the initial problem of moving all my client sites to the new software infrastructure. That will be done manually, and gradually, site-by-site. With the new DNS system now working you’d set up a bunch of nameservers, say ns1.tpj-cloud.com, ns2.tpj-cloud.com which would be hosted by the web cluster itself, and then you’d migrate the sites one-by-one to your new web cluster, at which point they gain the ease of transferability between data centres.

Whether your existing dedicated server host will support FreeBSD is another question… there’s always the Depenguinator!

Also would I be right in assuming that all machines in a web cluster have to have all applications installed on them? Sorry if that’s a Noddy question.

Not at all, it’s a good question. Let me try and explain how it works:

Not every site needs to be copied onto every single server. Suppose you have ten servers and you want a “redundancy guarantee” that you could turn off *any four servers* and not lose any data (perhaps the maximum number of servers is any single data centre is four, and you want to be data-centre-failure-proof).

What this means is that every website has to be copied onto 5 servers — consider the worst case for some website X, if four of the five servers which had copies of X went down, there’d always be one server left with that data. This is called N+1 redundancy. N is the level of redundancy (or acceptable risk), and N+1 is the number of copies of each piece of data (website, database) you need at minimum to have to guarantee that level of redundancy.

This number N, the redundancy level, is a knob you can tweak within the web cluster. It represents a trade-off between disk and bandwidth usage (replicating websites to lots of servers takes more network and I/O bandwidth) and the redundancy of your web cluster (its resistance to failure). If you want your web cluster to be nuclear-bomb-proof then you need N to be as high as the maximum number of machines in any one city (assuming only one city gets hit at a time!). But if you’re less paranoid about mutually assured destruction and want to make better use of your bandwidth and disk space, you might set your level of acceptable risk equal to *the probability of any two machines failing simultaneously*, you can set N=2. Then there’ll always be three copies of every single piece of data, so that you can’t possibly lose it when two machines go down.

Note that setting N to be a small constant number with respect to the number of machines in your web cluster enables the scalability of your web cluster. If every website only has to be replicated to 2 machines in a 100 node web cluster, you get 50 nodes’ worth of disk space and capacity to play with. When you scale that same web cluster to 200 machines, you get 100 nodes’ worth of disk space. In other words, you get the holy grail of linear horizontal scalability, while retaining your chosen level of redundancy.

At the other extreme, if every website has to be replicated to 50 machines, you only get 2 nodes’ worth of disk space to play with, but you do have an incredibly resilient system!

Any questions? Just shout in the comments and I’ll get back to you!

Utility computing and the drive towards a more sustainable future

March 11th, 2010

I don’t think there can be any doubt that mankind is damaging his environment in numerous ways, and consuming resources at a rate that is simply not sustainable – whatever your views about global warming, there’s no denying that the looming energy crisis and dwindling supplies of natural resources will affect us all in years to come – if only in terms of increasing operating costs and the associated hit to your bottom line.

What might come as a surprise is the key role that cloud computing and the move towards computing resource being provided as a utility service (utility computing) is going to play in enabling us to achieve a more sustainable future across the technology spectrum. This may be particularly surprising as it’s likely you’ve read one of the recent flurry of articles expressing concern at the growing size and power consumption of the internet’s data centres (with Google often cited as an example, tied to some guesstimated environmental cost of each Google search).

In this article I’m going to explain how the existing consumer culture of cheap disposable products is set to change, and how cloud computing is an important part of the technology that will allow us to reduce unnecessary wastage of natural resources.

Let’s start with a description of the problem – since the industrial revolution and the advent of mass production in the early 1900s, there has been a steady drive towards increasing production volumes, increasing product homogenity, decreasing manufacturing costs, and ultimately, as a result of those reduced manufacturing costs, a deterioration in product quality and expected lifetime. This is particularly true in the technology market where Moore’s Law (the expoential growth of the number of transistors on a silicon chip) has made a computer over a couple of years old out of date, and a computer over 10 years old essentially landfill.

It goes deeper than this though – the drive to reduce prices in a market where the cheapest product is often the most popular (and therefore most profitable) has resulted in the value of the resources and materials that go into making that product becing diminished. As mass production and industrialisation made it cheaper to extract and refine these resources out of our planet, so we lost sight of the true value of them as finite and precious gifts of nature.

It’s important to point out that we’re not just talking about fuel petrolium refined out of oil – we’re talking about the many products that come from rare and finite natural resources – this includes plastics and various synthetic materials, but importantly for the technology sector, it also includes metals – particularly the post-transition metals Gallium, Germanium and Indium and Cadmium – all of which are vital for the production of the semiconductors that go into pretty much every electronic device you own. Worryingly – these metals are in equally short supply to oil – in fact we may even run out of Gallium and Indium within 20 years – before we run out of oil.

Of course, this problem is a small subset of a much larger global issue – simply put, we are draining resources from our planet at a rate many orders of magnitude faster than they will be replenished by natural processes. What’s more worrying is that we’re not even really getting anything of any lasting value out of these resources – they’re simply being extracted from the ground, processed into product which will last a person a number of years (or perhaps only months), after which time the product is discarded and these precious raw materials are returned to the environment in a form in which it is much more difficult to extract the resource from, either into landfill sites, or into the atmosphere in the case of incineration or combustion of fossil fuels. It begs the question, when scarsity drives up the cost of these resources, will future generations turn to mining our landfill sites to recover the discarded precious metals in hundreds of thousands of throw-away netbooks?

Let’s be clear about this – supplies of these precious natural resources are already beginning to dwindle, and as they do, the cost of extraction increases, and as demand begins to outstrip supply, market forces will inevitably drive up prices. This is a simple fact of economics. The day will soon be upon us when it is simply not economically viable to produce disposable consumer electronics designed to be used for a couple of years and then thrown away – this is likely to happen within our lifetimes, and it will happen whether there is a conscious and concerted effort to be save the environment or not.

Simple economics, not altruism will force a sea change upon consumer culture – the emphasis will shift from the lowest possible unit price to the longest possible product lifetime. We’re going to see a shift back to an older way of thinking – when we buy a product we’re going to expect it to last a lifetime, maybe more – we’re going to want to hand things down to our kids, and we’re going to have to be prepared to pay a premium for that, but that premium will pale in insignificance compared to the cost of the natural resources required to produce the product. Products will be built to last, because that is the only sustainable way to continue living with the conveniences of modern society that we’ve grown so accustomed to.

So how does this fit into the technology market, where the pace of development has historically moved so rapidly that the maximum feasible lifetime of a technology product like a computer is measured in numbers of years, not decades? The rapid obsolescence of IT hardware has mainly been driven by Moore’s law, which simply states that the number of transistors on a computer chip will double every 2 years – what this really means is that new computers will do roughly twice as many operations in the same space of time as a 2 year old computer, or in other words, every two years, computers double in speed.

With such rapidly expanding computer capabilities, software developers have been quick to take advantage of the extra computing resource available to them, and the pace of software development has closely mirrored the pace of hardware development. However, the problem with this is that old computers become obsolete because they’re simply too slow to run the latest software, and everyone wants to run the latest software, to take advantage of new features and to maintain compatibility to with everyone else who is constantly upgrading.

This creates a market in which computers are not expected to physically last longer than a few years, because there is simply no value in a computer once it’s no longer able to run the latest generation of software. Computers have effectively become a disposable product, you buy one, use it for a couple of years until it either stops working because it has been cheaply manufactured, or stops being useful because it is not able to run all of the software that you require anymore – often the two will neatly coincide. At this point you throw the device away and buy a new one, they’re cheap enough for most people to be able to afford to do that. The current ‘netbook’ fad is a the latest in a long line of consumer devices with lower and lower manufacturing costs and shorter and shorter expected lifetimes.

It’s clear this approach isn’t sustainable – we’re using precious natural resources at an alarming rate – resources that took many hundreds of thousands of years to be created, for the sake products that are gone in the blink of an eye on geological timescales. So what are the changes afoot that will signal an end to this wasteful culture of cheap disposable IT products?

Well, over the last 5-10 years an important change in the nature of software has occurred. That change is of course the internet, more specifically the web. In the early days of the web it was merely a method for posting and viewing information, but what we’re seeing now is that an increasing amount of software is being written to run directly in the web browser, web pages aren’t really just web pages anymore, they’re web applications – we’re not far away from a time when all software will be written to run inside a web browser in this way.

Of course you can’t really have failed to see the growth of web applications, but you might have missed a more subtle implication of this shift in software development practice – an increasing amount of the computing requirement is being shifted away from the client PC and onto the servers. We’re actually seeing a return to an earlier approach to computing – it’s the ‘thin client’ approach – a central server handles the vast majority of the workload, with the client machines themselves being kept relatively minimal and deferring to the server whenever they need any work done or data stored.

The dark secret the chip manufacturers would rather you didn’t know is this: You do not need a fast computer to make a perfectly usable web terminal. Hardware that’s a few years old will still run a modern web browser. This approach effectively extends the useful lifetime of a client device without breaking Moore’s law.

The thin client approach has a number of very attractive qualities, and it’s something that has been tried with a limited amount of success throughout the history of computing. However, we’re now at the stage where some important enabling technologies have become available which make it a much more compelling scenario for the average home user. The web is an important one, but there are lots of other forces conspiring to push us in this direction as well.

Almost ready to show the world

March 4th, 2010

Hybrid Web Cluster has been a long time in the making, and now it’s finally getting close to being demonstrable. You can’t imagine how excited that makes me.

I think we’ve got a genuinely revolutionary product here — a web cluster that takes the headache away for anyone who’s running a bunch of web servers — and I can’t wait to show it to the world.

There was a time a little while ago that I thought that cloud computing would eclipse us. We’re such a small company with limited resources, I felt that the giants of Amazon, Google and Rackspace would be impossible to compete with.

But then I realised we can use them. Instead of trying to compete directly with any one of these companies, our product can run on top of their infrastructure, taking advantage of all their investment in hardware and their massive management systems which keep their “infrastructure clouds” running. We can even run one cluster across all three  — one server on EC2, one on Rackspace, and one on, say, ElasticHosts. All you need is a decent internet connection between these separate data centres and the cluster will automatically cope with the failure of, say, all of EC2. And all that with the level of complexity and configuration that your receptionist could do through her web browser. Now that’s something.

You could — through our web interface — spin up instances on all three of these networks, and also manually download and configure the software on, say, three of your own servers. You can tell Hybrid Web Cluster through our web interface that these 6 servers together make up a single cluster, and they’ll go away and join themselves up as a single logical entity.

This is what people mean when they talk about a “hybrid cloud” — a hybrid of local and remote “cloud” hardware working together. But you can’t imagine how complex a proposition that would be to most technical people. Suddenly we’ve got a solution which we can deploy in a matter of minutes, and with no technical expertise at all. (Although you’ll need to know how to boot our install CD and set up some IP addresses if you want to run a local machine as part of one of your clusters).

So an exciting time ahead! Let’s hope we can get some momentum, both within the open-source community and with potential customers.