Research Overview
Jeffrey S. Chase
Department of Computer Science
Duke University
Spring 2008

My research focus and methodologies are rooted in the operating systems research community. My research is experimental in nature, complementing the analytical approaches dominant in many areas of computer science. Experimental systems research blends science with the development skills and creativity needed to convert good ideas into functioning, useful software systems. Its purpose is to develop and refine the fundamental techniques relevant to practical needs in computing systems, implement them in a realistic context, design and conduct experiments to evaluate them, and in many cases, to release system prototypes that serve as a basis for continuing research in the broader community. Experimentation is the ultimate test of the validity and practicality of new ideas, and it is an essential element of technology transfer in computer science.

My research since 2001 focuses on managing networked computing and storage ``cyberinfrastructure'' as a service utility, in which shared hardware resources are provisioned or sold according to demand, much as electricity is today. This vision rests on several key premises:

• An effective environment for dependable computing will combine stateless client devices with a distributed service backbone (data storage, Web services, interaction, computation) whose functions are made scalable and reliable through geographic dispersion and replication [20].

• These services are hosted on a server backbone infrastructure residing in data centers and other edge sites throughout the Internet; the edge resources (CPU, storage, memory, etc.) may be shared, like the network itself.

• Flexible service architectures and dynamic resource sharing will enable a fluid mapping of service functions and data onto the infrastructure, so one may instantiate services wherever resources are available and demand exists.

• Critical services must sense-and-respond to adapt automatically to changes in traffic demands or resource conditions, while holding human administrative burdens constant.

• The server network and the hosted services constitute a critical public infrastructure requiring open, flexible, secure, robust, and decentralized control. The control architecture must resolve the ``tussle'' of contending demands, changing priorities, and rapidly advancing technology, all within the framework of a self-sustaining system. Any such system is inherently economic.

Variations of this vision are shared by many of my colleagues, and the fundamental elements and approaches are evolving under various names including utility or on-demand computing, self-managing systems or autonomic computing, grid computing, service-oriented architectures, federated and incentive-structured systems, market-based systems, overlay networks, and peer-to-peer computing. Each of these names is associated with a community that addresses common core problems from a particular perspective, often under different assumptions about the infrastructure, workload, or architectural principles. I have engaged with each of these communities to varying degrees.

My research has addressed various elements of a computing utility economy: models to predict demand and behavior [23,57,58]; fundamental abstractions and scheduling mechanisms to ``slice'' the resources as a measured and metered quantity [10,18,37,39]; protocols to represent and enforce accountable contracts [27,68,69,66,34]; control and optimization of resource slices for service quality metrics including dependability [16,23,40]; scalable architectures for resource brokering and discovery [10,27]; flexible and adaptive service architectures for content services and network storage [7,63,42,24,12,2]; data center architecture and server energy management [16,53,44,47]; statistical inference of system behavior from instrumentation data [22,43,44]; and market-based resource management [36,34].

Through this program of research, I have focused on a core set of architectural principles as a practical and general foundation for critical utility services. Server functions are distributed across a modest subset of sites according to their identities and attributes, rather than across a massive number of anonymous entities as in peer-to-peer systems [59]. Guest services interact with the utility to obtain leases for virtualized shares of ``raw'' resources provisioned to meet target levels of service quality. Resource leases may be brokered by third parties, and represent contractual arrangements among self-interested participants, who are held accountable for their contracts. Instrumentation generates a continuous feedback signal to drive policy choices. The participants act based on local information and internal models of the system and applications. These local choices produce an emergent global behavior.

The web sites for the NICL lab and the Orca project provide the best current information about our research toward this vision. In the course of our research, the Orca project has developed open-source software for on-demand virtual computing environments based on a foundational abstraction of resource leasing. Orca project software includes the Shirako leasing core [35,50], the Automat control portal and related components, a new implementation of the SHARP framework for accountable lease contracts and brokering [27], and Cluster-on-Demand (COD), a back-end resource manager for shared clusters [18,48]. Orca is a context for our continuing research in automated resource management and adaptation, which is a primary focus of our ongoing research and development [33]. Two recent doctoral students (Laura Grit and David Irwin) completed their dissertations in the Orca project.