Architectural Optimization of Digital Libraries

This work investigates performance and scaling issues relevant to large scale distributed digital libraries. Presently, performance and scaling studies focus on specific implementations of production or prototype digital libraries. Although useful information is gained to aid these designers and other researchers with insights to performance and scaling issues, the broader issues relevant to very large scale distributed libraries are not addressed. Specifically, no current studies look at the extreme or worst case possibilities in digital library implementations. A survey of digital library research issues is presented. Scaling and performance issues are mentioned frequently in the digital library literature but are generally not the focus of much of the current research. In this thesis a model for a Generic Distributed Digital Library (GDDL) and nine cases of typical user activities are defined. This model is used to facilitate some basic analysis of scaling issues. Specifically, the calculation of Internet traffic generated for different configurations of the study parameters and an estimate of the future bandwidth needed for a large scale distributed digital library implementation. This analysis demonstrates the potential impact a future distributed digital library implementation would have on the Internet traffic load and raises questions concerning the architecture decisions being made for future distributed digital library designs and the Internet capacities that will be necessary to support them. This analysis suggests that network capacities of 622 Mbps will be required to go much beyond 100 heavily used independent digital library sites. Additionally, capacities beyond 622 Mbps will be required to realize the worldwide distributed digital library consisting of a 1000 or more digital library sites. These results also point out the need for architecture modifications and software improvements to reduce and minimize the amount of network traffic generated as we move to a global digital library implementation.