A web-based tool for advanced statistical analysis of network traffic measurements Costas Courcoubetis* and Vasilios A. Siris Institute of Computer Science (ICS) Foundation for Research and Technology - Hellas (FORTH) P.O. Box 1385, GR 711 10 Heraklion, Crete, Greece E-mail: {courcou,vsiris}@ics.forth.gr * Also with the Athens University of Economics and Business 24 May, 2000 Keywords: statistical analysis, web-based tools, traffic engineering, QoS Extended Abstract: Recent advances in the protocol development and technology are creating the necessary building blocks for transforming the Internet from a pure best-effort services network into a network capable of supporting Quality of Service (QoS) guarantees. Some important questions related to the management and dimensioning of networks that support QoS guarantees include the following: What combination of different traffic types can the network accept while ensuring some target level of performance? Alternatively, if the traffic mix is given, what is the amount of resources required in order to guarantee a target level of QoS? What effect does traffic shaping and policing have on a link's multiplexing capability? Traditional approaches that address the aforementioned questions such as measurements of the average load in long intervals of the order of minutes are not adequate, since they fail to capture the burstiness of the traffic, which is important in the case of guaranteed QoS. On the other hand, approaches based on queueing theory are also inadequate, since they require elaborate traffic models that often fail to capture important characteristics of real traffic. An alternative approach is to rely on actual traffic measurements taken in sufficiently small time intervals. Indeed, current technology can support traffic measurements at links running at very high speeds, e.g. see [CoralReef]. However, an uninformed and unrestricted collection of detailed measurements can easily result in a huge amount of data, both difficult to store and difficult, or even impossible, to analyze. Hence, a natural question is what measurements are important for performance analysis and how such measurements can be used to answer questions such as those identified above. In this paper, we present the design and functionality of a web-based tool for advanced statistical analysis of real network traffic measurements (A version of the tool is accessible from [TrACe]). The tool consists of the following two components: - a web-based user interface - software modules implementing traffic analysis procedures The interface provides a flexible environment through which a user, using a Java enabled web browser, can perform the following functions: - create, modify, save, and execute experiments - graphically view the results from experiments - request results from experiments of the same type to be presented on the same graph. The types of experiments supported include the following: 1) buffer overflow probability at a link that multiplexes a given traffic mix and has a given load, 2) maximum achievable utilization at a link that multiplexes a given traffic mix and guarantees a target overflow probability, 3) combination of different types of traffic streams that satisfy a target overflow probability (this combination defines the link's acceptance region), and 4) trade-off between the leak rate and bucket size of a leaky (or token) bucket (the leak rate and bucket pairs form a traffic stream's indifference curve). Hence, running function 1) above for a range of buffer sizes would allow us to plot the overflow probability as a function of buffer size. Furthermore, the capability to present the results of more than one experiments, of the same type, in the same graph enables the straightforward visualization of the effects of varying a single parameter. This capability, for example, enables us to view the effect of traffic shaping on the overflow probability or maximum utilization, and the comparison of token bucket parameters for traffic collected at different periods of the day. The software modules implement robust and efficient statistical analysis procedures based on an advanced theory of multiplexing a large number of bursty traffic streams, while providing QoS guarantees [CW96,Kel96,CSS99]. The model considered is that of a single buffer serviced using a FIFO scheduling policy. The buffer is fed by bursty traffic streams, which can go through some number of filters before entering the buffer. Possible filters include traffic smoothing over a given time interval and policing based on a peak rate or leaky bucket. The tool uses traces of actual MPEG compressed video traffic, as well as measurements from a production IP network. Hence, the tool allows us to perform experiments on traffic captured at different times of the day. Such experiments allow us to visualize, and subsequently to understand, how the characteristics of the traffic from a large organization vary throughout the day. Measurements of traffic are in the form of load measurements in small intervals (typical values for these intervals are 10s to 100s of milliseconds). The tool's interface is implemented in Java, which enables it to be accessed from any browser that supports Java. Through the interface, the user can send requests over the Internet to a server running on a high performance workstation to create, modify, save, or execute experiments. The experiments are executed at the workstation by software modules implementing advanced statistical analysis procedures. The software is implemented in C, to allow for efficient runtime execution. A description of the software modules is contained in [CS99]; the modules themselves are available as stand-alone components at [MSA]. In addition to assisting network managers in operating and dimensioning their network more efficiently, the web-based tool can serve as a teaching tool for hands-on experience with traffic analysis methods. The development of the web-based tool for advanced statistical analysis is part of our activities in traffic measurement and analysis, which are being performed in collaboration with the Communication and Networks Center at the University of Crete [TrACe]. The objectives of this work includes the deployment of a flexible platform for passive measurement of real network traffic and the development of web-based tools for requesting traffic statistics and for accessing software for advanced statistical analysis of network traffic measurements. In addition, the platform is being enhanced to support active traffic measurements and serve as a testbed to implement usage-based charging experiments. References [CoralReef] CoralReef, Cooperative Association for Internet Data Analysis (CAIDA). http://www.caida.org/tools/measurement/coralreef/ [CS99] C. Courcoubetis and V.A. Siris. ``Measurement and analysis of real network traffic''. Technical Report TR-252, ICS-FORTH, March 1999. See also [MSA]. [CSS99] C. Courcoubetis, V.A. Siris, and G.D. Stamoulis. ``Application of the Many Sources Asymptotic and Effective Bandwidths to Traffic Engineering''. Telecommunication Systems, 12:167-191, 1999. [CW96] C. Courcoubetis, R. Weber. ``Buffer Overflow Asymptotics for a Buffer Handling many Traffic Sources''. Journal of Applied Probability, vol. 33, 1996. [Kel96] F.P. Kelly. ``Notes on effective bandwidths''. In Stochastic Networks: Theory and Applications (Editors F.P. Kelly, S. Zachary and I.B. Ziedins) Oxford University Press, 1996. 141-168. [MSA] V.A. Siris. ``Large Deviation Techniques for Traffic Engineering''. http://www.ics.forth.gr/netgroup/msa/ [TrACe] UoC's Traffic Measurement and Analysis Platform - TrACe. http://trace.ucnet.uoc.gr/