This year, IISWC has a new Benchmark Innovation forum focusing on Internet usage scenarios. Examples include laptops and desktops that are primarily used for Internet browsing, mobile computers with Internet capability (such as the iPhone), set-top-boxes and games, among others.  These systems simultaneously process heterogeneous applications, many of which persist in the system (like embedded computers), while others are highly data dependent (like more general purpose computers).  Internet connectivity often forces patterns of execution unlike those seen in previous generations of computer systems. The increasingly sophisticated user interface is also an important contributing factor, where the integration of heterogeneous applications can be used to overcome the size of the overall device as it responds in near real time.  Computers that carry out Internet usage scenarios are neither embedded nor general purpose – they have characteristics of both and neither. 

Benchmark suites have traditionally been structured as collections of programs executed in batch-style resulting in one dominant performance metric: speedup. However, speedup fails to capture the merits of integrating additional applications, such as speech recognition, at the expense of others. How should overall performance of these systems be evaluated? How will benchmarks facilitate new performance metrics?

We are soliciting papers that investigate how Internet usage scenarios impact the structure, content, and use of benchmarks to evaluate performance. Paper topics can include, but are not limited to:

• studies of trends in Webpage content and other Internet usage scenarios

• proposals for new forms of Webpage benchmarking and performance analysis

• analysis of the failure of existing benchmarks for Internet usage scenarios

• fresh analysis and statement of the problem

• compelling examples of how existing benchmarks have failed for new designs.

Authors of successful submissions will be invited to a special session to present the key ideas of their paper, followed by a panel discussion.  Short, 6-page papers will be published in the IISWC proceedings. Follow the paper submission guidelines of the IISWC, but with a 6-page limit.  Papers must be submitted by email to jmpaul @ vt.edu, with the subject “IISWC BI submission.” The deadline is April 21, 2008.

Committee :   JoAnn Paul (chair), Virginia Tech

Steve Blackburn, The Australian National University

Tom Conte, NC State

Lieven Eeckhout, Ghent University

Rebecca Isaacs, Microsoft Research Cambridge

Mike Schulte, U Wisconsin-Madison