- About Archives
- About SAA
- Careers
- Education
- Publications
- Advocacy
- Membership
January 15, 2017
Version 2
Open to commenting January 18, 2017 – February 17, 2017
In order to support increasing demands on archival and special collections repositories to demonstrate the value they provide their constituencies, archivists and special collections librarians have become increasingly mindful of the need to gather, analyze, and share evidence concerning the impact of their services and the effectiveness of their operations. Yet the absence of commonly accepted statistical measures has impeded the ability of repositories to conduct meaningful assessment initiatives and the articulation and evaluation of best practices.
Recognition of these challenges has emerged in several contexts in recent years, including an assessment-themed issue of RBM: A Journal of Rare Books, Manuscripts, and Cultural Heritage, published by the Association of College & Research Libraries (ACRL); assessment-related sessions at the meetings of allied professional associations, including the Society of American Archivists (SAA), American Library Association (ALA), and ACRL’s Rare Books and Manuscripts Section (RBMS); presentations centered on special collections at the biennial Library Assessment Conferences sponsored by the Association of Research Libraries (ARL); and grant-supported initiatives led by ACRL, ARL, and other organizations aimed at building and fostering cultures of assessment and demonstrating the value that libraries and archives bring to their communities and society at large.
Within this context, SAA and ACRL/RBMS constituted a joint task force in 2014 and charged it with developing standardized statistical measures for public services in archival repositories and special collections libraries. The task force consisted of ten members, five appointed by SAA and five by ACRL/RBMS, including co-chairs representing each organization:
Members were appointed for two-year terms, which were renewed in 2016 for an additional year.
This standard was developed to provide archivists and special collections librarians with a set of precisely defined, practical measures based upon commonly accepted professional practices that can be used to establish local statistical data collection practices to support the assessment of public services and their operational impacts at the local institutional level. The measures were also formulated to support the aggregation of public services data from multiple institutions to provide a basis for institutional comparisons and benchmarking. It was beyond the charge of the task force that developed this standard, however, to create a statistical survey instrument to collect institutional statistics or a data repository to house them.
Careful attention was given to formulating the measures so that any type of repository that manages and provides access to archival and special collections holdings—including academic, corporate, and government archives, public and independent research libraries, historical societies—may use them in a manner consistent with their application by other repositories. So, too, measures were formulated in order that repositories of any size and any level of budgetary resources may implement them.
The purpose of this standard is to help archival repositories and special collections libraries quantify in meaningful terms the services they provide their constituencies, and evaluate the effectiveness and efficiency of the operations that support those services. It does not attempt to provide guidance on conducting qualitative assessments of user impacts, which are beyond its scope. Nevertheless, the definitions of services, measures, and performance metrics that it does present may be useful in planning and formulating qualitative assessments.
The focus of this standard on quantitative measures and performance metrics should not be construed as an attempt to reduce the value of archival and special collections libraries to a set of numerical input and outputs. To the contrary, its purpose is to establish a common and precise vocabulary to facilitate conversations about the ways in which archives and special collections deliver value and how they might increase it.
Although value may be expressed in economic terms, the measures and metrics defined in this standard were not formulated as budgetary inputs and outputs, but rather as practical activities related to the delivery of services to users of archives and special collections. They may nevertheless be used to support cost/benefit analyses of service operations if desired.
Following a survey of community practices, the task force elected to shape the contents of its charge into eight domains, each covering a different area of public services provided by archival repositories and special collections libraries. Although the domains are interconnected and overlap at points, efforts were made to maintain distinctions to allow for independent collection and analysis of measurements wherever possible. The eight domains are: User Demographics, Reference Transactions, Reading Room Visits, Collection Use, Events, Instruction, Exhibitions, and Online Interactions.
The presentation of each domain follows the same structure, beginning with a brief description of the scope of the domain and general rationale for collecting statistics that pertain to it. Next, a single basic measure is defined as a baseline for local data collection and potentially for aggregation with other institutions. The aim was to identify measures that archival repositories and special collections libraries of any type and size would find useful and practical to collect, using whatever means available, whether pencil and paper, spreadsheets, or an automated system. All repositories are encouraged to collect at least the basic measures for each domain, thereby creating the possibility for sharing a common set of statistics that are uniform and comparable across many institutions.
Each domain also contains one or more advanced measures, which repositories may choose to collect as local needs and resources allow. In most cases it would not be either feasible or productive for a repository to collect all of the advanced measures on a continual basis. A repository may find it useful to collect selected advanced measures year after year, while collecting other advanced measures on an as-needed basis when reviewing specific areas of public service operations, and others not at all. A repository may also wish to formulate its own measures based on other processes that pertain to the delivery of its public services. In that respect, the advanced measures outlined in this standard should be regarded as suggestive rather than exhaustive.
Both the basic and advanced measures are described individually, and each includes a rationale that suggests in general terms the potential benefits for tracking it. Each measure also includes guidelines for collection that provide details on what types of data should be included or excluded from the measure and the means that can be employed to collect the data. The guidelines for collection are followed by one or more applications and examples to illustrate in still more practical terms how the data can be collected, counted, and used by various types of repositories. The guidelines, applications, and examples were also formulated to help ensure uniformity and consistency of data collection.
A final section in each domain offers a series of recommended metrics that indicate the different ways in which the measurements can be analyzed and compared, and in some cases combined, to help repositories monitor the effectiveness and efficiency of their operations and quantify the outputs, if not the impact and value, of their services.
Finally, an appendix contains a glossary of key terms that are employed in the definitions of the services and measures described in this standard, thereby ensuring that their usage is clear and unambiguous. Whenever possible, the definitions have been borrowed or adapted from other approved standards and authoritative resources that are commonly used by libraries and archives. In a few cases, however, it was necessary to further elaborate borrowed definitions in order to render them precise and meaningful in the context of archival and special collections services, or to formulate original definitions when a comparable service or definition could not be found. Wherever a term defined in the glossary appears in the standard, it has been capitalized in order to prompt the reader to consult the term’s definition in the glossary.
This standard was created to help archival repositories and special collections libraries develop local statistical data collection policies and practices around their public services so that they might in turn use the data to conduct systematic assessments of their public service operations. In order to use this standard effectively for this purpose, it is important to understand the distinction between measures and metrics, and how both can support assessment initiatives.
A measure is, quite simply, the result of taking a measurement of some quantifiable object or process. In the context of this standard, measures are counts of some type of discrete, repeatable process comprised in the delivery of a service to the user of an archival repository or special collections library, for example a count of how many reproductions are made for a researcher or how many visitors view an exhibition. Individual measures can be directly compared. For example, how many researchers consulted materials in the reading room last month versus the same month last year?
A metric, on the other hand, is a calculated ratio between two measures or an independent variable, most often an increment of time (i.e., a rate). Metrics can be used to quantify and compare changes that occur in a repeated process over time or in the relationship between two processes. As such, they can be used to identify trends in the usage or performance of services. For example, a reference department may want to calculate the average number of reference questions it receives per month before and after linking an online reference request form to every page on its website to help assess whether the form encourages users to ask more questions. Another example involving the relationship between two measures would be calculating the number of exhibition visitors per exhibition rather than per year. Metrics are sometimes called performance metrics because they can be used to establish goals or benchmarks.
The measures and metrics described in this standard pertain to the performance of services that are commonly offered to users of archival repositories and special collection libraries. In each case, the data pertaining to a measure or metric can be collected by repository staff who perform the service or by an automated system that they use to perform it. None of the measures or metrics depend on user surveys or other extrinsic means of collecting data. This is deliberate, and is meant to ensure that the data can be collected easily and reliably in the course of performing a service and without placing significant burdens on staff. On the other hand, this strategy has its limits. A comprehensive assessment program, especially one that aims to evaluate the value and impact of services on users, must include other types of data gathering, including qualitative data, to complement the purely quantitative statistical measures of service inputs and outputs described by this standard. It is nevertheless hoped that this standard will help to stimulate and facilitate the implementation of meaningful assessment programs at a broad range of archival repositories and special collection libraries.
| Attachment | Size |
|---|---|
| SAA-ACRL-RBMS - Public Services Measures and Metrics - DRAFT 2 - 20170115.pdf | 340.54 KB |
Suggest adding a general remark in the introduction about the responsibilties repositories have to implement appropriate policies and procedures for maintaining user data and ensuring the privacy and confidentiality of such data (references are made to such policies and practices in specific sections of the standard, but it would be good to incorporate a more general one into the introduction).