Beam your memory back to the ‘80s (if you can, of course). Madonna was on top of the charts singing about a “Holiday” and acting “Like a Virgin.” Meanwhile, many of us geek-nerds were hanging out in computer basements chatting with our geek-nerd colleagues over the Internet using UNIX talk. Back then, traditional computing architecture was relatively simple (and simply put): server + operating system + application.
And as Madonna kept delivering the hits throughout the 80s and ‘90s, enterprise data centers kept building out more and more siloed traditional computing environments. Nearly every new project (from strategic to pet) meant another server + OS + application combo, as the building of additional raised flooring tried to keep up with the server sprawl. Unfortunately, not only did this architectural approach consume a lot of data center ping, power and pipe, it also meant a lot of wasted compute cycles. Indeed, server after server were being sized to handle infrequent peak loads, leaving NOP and Idle as the “applications” consuming the most workload.
From the mid-1990’s through the 2000’s came the age of the virtual enterprise, with the arrival of server consolidation and virtualization. As Madonna was making her “Confessions on a Dance Floor,” IT managers were confessing to better server utilization by taking advantage of the ability to run multiple App + OS stacks on a single server enabled through a virtualization layer. The trade-off, though, was extra compute resources required to run the virtualization software, plus the inefficiency of running many instances of various operating systems on a single server; however, this seemed a small price to pay for the benefit of not having to report more 5-10% utilization numbers back to the CIO during budget negotiations.
Next came the emergence of Cloud Computing in the mid-2000s. Madonna was again on the scene scoring big spending “4 minutes” to “Give it 2 Me.” Meanwhile, the initial Cloud providers were spinning up virtual machines in around four to ten minutes, as they were giving it 2 developers plenty of software stack options on a promised elastic, pay- per-use business model.
It’s important to realize, however, that the underlying architecture chosen by these first-generation cloud providers borrowed heavily from the server consolidation and virtualization era. The major difference was that the pools of compute cores were now being deployed as public-facing services, versus running the previous generation of siloed applications. Unfortunately, the resulting inefficiencies prevent full realization of the Cloud promise of truly low-cost, high-performing, elastic computing. In response, Joyent has emerged as a second generation provider, rethinking fundamental cloud architecture, and ushering in the era of what they call “Smart Computing.”
Former VP of Marketing for Joyent Adrian Ludwig provides an overview to Joyent's smart computing.
What’s Broken?
Elasticity
When discussing what’s broken in first generation cloud architectures, let’s start with elasticity (i.e., the ability to spontaneously and easily scale up resources based on real-time demand).
Unfortunately, with a virtual machine architecture, the application can’t truly scale vertically because the purchased machine image can’t access resources beyond the subdivided, assigned physical environment. Therefore, even though there may be unused CPU cycles available on a core, the developer can only scale horizontally by purchasing additional machine images to access more compute resources.
Provisioning
This leads to the provisioning of new machine images (which one might argue is also a bit broken).
To begin, the developer has to spin up new instances manually, or deploy a third party application to execute the provisioning. Both options add cost and complexity.
A Full OS Instance per Virtual Machine Image
Next, each virtual machine has to carry an instance of the full operating system. This is problematic on a few fronts.
For starters, a typical production OS like Windows or Linux contains a very complex array of functionality. There’s code to handle such things as networking, security, peripherals, input/output devices and resource scheduling, to go along with the ability to actually run an application.
That adds up to a lot of source code. For example, a single Linux distribution can contain over 200 million source lines of code. That’s a lot of code to spin up every time a first-generation cloud virtual machine image is deployed. Most of this code is necessary when the operating system is running a desktop or a server, but it’s overkill for a single application.
The result is that larger machine images beyond what the application requires need to be deployed to house each OS. Additional CPU cycles are consumed by these operating systems, decreasing performance by taking cycles away from the application and again adding costs.
In addition, it takes minutes to boot up a full image of the operating system, leaving developers with another dilemma: submit their customers to frustrating latencies waiting for new virtual machines to get spun up, or increase costs by over-provisioning to maintain performance levels to meet unexpected increases in work loads.
The Need to Manage the Operating Systems
Running a full operating system means the developer needs to manage this portion of their deployed stack, adding complexity to the already challenging task of deploying an application.
This also means the operator has to take extra steps to make sure there’s no malicious use of all that extra OS functionality for things like security and networking breaches. This adds complexity for the operator. It also fuels additional questions in the potential user’s mind on whether security in the cloud is robust enough.
Virtualization Overhead
Finally, there’s the cost of running the virtualization infrastructure layer in terms of additional compute cycles, management resources and, if a commercial virtualization product is deployed, licensing costs. These added costs get passed along to the developer as increased virtual machine instance costs.
Adrian Ludwig, former VP of Marketing, provides a chalk talk discussion regarding Joyent's view of smart computing.
Fixing What’s Broken
It’s About Running the Application
Before diving into how second-generation cloud architecture can help fix what’s broken, it’s important to step back and remember that the primary objective in the cloud is to run an application. All costs and overhead associated with running the underlying infrastructure is a means to an end. Put another way, no CIO would be very happy receiving a large cloud computing bill for only running an operating system in a virtual machine instance.
Modern Languages and Virtual Machines
Not too long ago, developers wrote applications in languages such as Cobol, Fortan, and C. These programs needed to be ported to the underlying OS + server combinations.
Then came virtual machines. Developers could now write in modern languages such as Java, PHP, Ruby and Rails, which in turn were run on virtual machines that were already ported to the underlying OS + hardware combinations. Developers no longer needed to worry about porting their applications. Instead, they could just develop in a platform supported by their cloud provider of choice.
The Operating System Revisited
Next, let’s return to one of the fundamental elements to the computing infrastructure: the OS.
It’s important to recall that a a basic job of any operating system is to arbitrate which processes get access to the system’s resource through the resource and process scheduler.
Smart Computing and the SmartOS
That said, the second generation of Cloud Computing provider - Joyent - introduces what they call “Smart Computing.” In their Smart Computing architecture, they deploy a “SmartOS” which effectively replaces the first generation’s use of a virtualization layer. The SmartOS is optimized to run applications, not operating systems.
Of course, many hundreds of virtual machine platforms are ported to the SmartOS. Therefore, each application running on one of the ported platforms just becomes another process that gets managed by the SmartOS.
The scheduler in the SmartOS is hardened and updated to handle the management of the underlying available resource pool. Now, instead of the hardware being cut up in fixed sizes, as is the case with virtual machine architectures, the SmartOS can allocate more or less of the available resource pool based on the requirements of the applications sharing that pool. This enables true vertical scaling, without requiring manual or third-party code intervention.
Of course, the SmartOS delivers a further benefit by eliminating the cost and complexity of running both the virtualization layer and the numerous OSes.
Joyent CEO and Co-Founder David Young introduces Joyent.
The Joyent SmartDataCenter
Having said this, of course, people who run applications in the cloud do so partially for the benefit of being able to burst across multiple cores if and when usage spikes arise.
Therefore, Joyent deploys what they call the SmartDataCenter. The SmartDataCenter provides an abstraction layer for the entire data center -- including compute, storage, and network resources -- and centrally manages these resources.
For example, in a SmartDataCenter, application processes are profiled and deployed on the appropriate servers depending on resource consumption requirements. Whereas the SmartOS might deploy more compute resources on a single CPU for vertical scaling, the SmartDataCenter can deploy more server cores for horizontal scaling without intervention. In addition, the SmartDataCenter can move an application from one physical resource to another automatically to improve load balancing and resource utilization.
Getting Smart
Clearly, first-generation cloud providers are in vogue, providing a huge leap forward in delivering on the promise of Cloud Computing. However, as is the case with many first-generation products, there is room for improvement. Second-generation providers like Joyent have entered the game, rethinking architecture, and as a result they are improving the overall efficiency of running in the cloud.
David founded Joyent in 2004 to provide a comprehensive suite of internet-delivered software and on-demand infrastructure for small to medium organizations. Prior to Joyent, David worked at Moody’s Investor Service (1989-1999) in the Structured Finance, International, and Digital Media groups as General Manager and corporate Vice President and was co-founder and CTO of manageStar (2000-2004), an enterprise services management software company whose customers included TimeWarner, Sodexho, and Global Signal. David was the principal architect of manageStar’s Harmony platform for service delivery and managed a team of 80+ developers in San Francisco and Bangalore, India. David holds a BA in Classics (Greek), cum laude, from Indiana University. He lives in the joyent lifestyle in Marin County with his wife Maria and their two daughters.
Content
