A framework for instrument monitoring on the grid

Abstract

Grid computing enables the selection and aggregation of a wide variety of geographically distributed resources as a single unified computing resource, for solving large scale compute and data intensive computing applications. As with any computer system an important task within a grid is monitoring. The ability to monitor distributed resources is crucial to high performance computation. Amongst other things, it allows one to evaluate behaviour, optimize behaviour, discover and diagnose problems or faults. The objective of this research was to design a framework that would provide a generic template to allow for ad-hoc monitoring experiments with external instruments in a grid environment. The template allows for the information captured by external instruments, either hardware or software, to be accessed through a grid information system. Monitoring instruments, in general, create a huge amount of monitoring data that is often stored in raw log files. The sheer size of the data generated makes it unsuitable for direct insertion into an information system. The idea of the framework is to make this data accessible through an information system, whilst allowing the data to remain in-situ. A demonstrator of the framework was also to be implemented as part of this research. To accomplish this, first the central component of the framework, the interface to the grid information system, in this case R-GMA, which is a relational implementation of the Global Grid Forum?s Grid Monitoring Architecture, developed within the EU DataGrid project, had xvi to be designed and implemented. This resulted in the development of a new type of R-GMA producer, the Canonical Producer. The proposed framework demonstrator was a network tracer, NetTracer, that would allow access to monitoring data obtained from a set of example network monitoring instruments through R-GMA. The instruments chosen support the tracing of two network interconnect technologies, Ethernet using Tcpdump, a software network packet capture application, and SCI (Scalable Coherent Interface), using a (hardware) SCI trace instrument. An additional, and valuable, contribution was the initial design and implementation of a Grid-wide Intrusion Detection System, the design of which evolved from the extension of NetTracer to support a third network monitoring tool, Snort, a network-based intrusion detection system. The research was successful in its objectives. The framework, SANTA-G, was designed, and NetTracer, the framework demonstrator, shows that is a viable concept. The research has contributed to three major grid projects. The Canonical Producer, the enabling technology for the framework, is now part of the R-GMA system. The initial implementation of NetTracer was developed within the EU CrossGrid project, and forms part of its grid monitoring system. NetTracer, and a prototype of the Grid-wide Intrusion Detection System, are also being used by Grid-Ireland, the national computational grid of Ireland, in order to monitor network activity on, and the state of security of, its sites. ; TARA (Trinity's Access to Research Archive) has a robust takedown policy. Please contact us if you have any concerns: rssadmin@tcd.ie