December 12, 2023

Celebrating the MetaArchive Technical Team

MetaArchive is a distributed digital preservation system, relying on members to host caches, volunteer time and energy to committee work, and exchange knowledge within and outside of the Community. However, a common pain point felt by many digital preservationists is a lack of technical support dedicated to preservation infrastructure long-term. MetaArchive provides a low-barrier entry to digital preservation because we have robust technical support through our contracted staff members Bill Robbins, Kurt Nordstrom, and Chris Helms, and through LOCKSS Support Engineer Clay Miller.

MetaArchive has been lucky to have had dedicated staff members who have supported our infrastructure since 2008, and as our infrastructure shifts to LOCKSS and we transition our technical support to the LOCKSS Program team, we wanted to take the opportunity to celebrate and thank our Technical Team for their work. Their contributions have been substantial and essential to MetaArchive functionality and we cannot express our gratitude enough.

The Technical Team members have been integral to how MetaArchive functions, ensured smooth transitions, and provided quick IT support.  Some of their essential activities included cache management, Conspectus management, research and development support, Amazon Web Service (AWS) management, and bridging support between MetaArchive and LOCKSS.

Cache Management: Our members host the caches that create our distributed digital preservation network. The Technical Team leads the ongoing maintenance of caches and provides technical support to local IT teams and members. They oversee cache replacements and manage cache recovery during rare, but inevitable, cache failures and are key to getting the network performing optimally. The Technical Team is central to all cache management and we heavily rely on them on the operational side of the cache server and LOCKSS maintenance.

Conspectus: Conspectus is the MetaArchive’s web-based collection management tool and the first point of entry for ingest into the network. The Conspectus maintains the metadata and preservation status information of preserved (or to be preserved) collections of member institutions and offers an easy to use interface, so that members can create, update, and maintain their collections and the collection descriptions of their archives.  All members depend on  the Conspectus, and the Technical Team ensures that it is up and running and manages all debugging and technical support for it .

Supporting Research & Development: The MetaArchive community has been engaged in ongoing research and development to improve services for our members, many of which are supported. Recent projects include the SuperNode Pilot Project and evaluating our current LOCKSS infrastructure.

AWS Integration: MetaArchive integrated AWS EC2 Cloud storage in 2010 as a backup system in a cloud environment to serve as a single location for the cache manager, title database, and central plugin repository. Since then, as more members transition to using AWS S3 and EC2 for their organization’s storage solution, the Technical Team has facilitated the integration of more AWS-accessible features.

LOCKSS Support: As a Private LOCKSS Network, MetaArchive works collaboratively with LOCKSS technical support to maintain smooth integrations and cache support for MetaArchive members.

We also want to take the opportunity to highlight the individual work of the Technical Team to acknowledge the depth and importance of their expertise.

Bill Robbins has been at MetaArchive since 2008, working closely with Educopia leadership. His role began as a Systems Administrator for MetaArchive and has taken on additional areas of support as needed, particularly as our cache expert. Initially, his work focused on coordinating Archival Unit (AU) ingest into the MetaArchive repository. He has written documentation for the LOCKSS interface that is still used as the official guidance for the LOCKSS GUI. His role transitioned to Network Support on a part-time basis for the last few years and he rejoined the Technical Committee to assist with the LOCKSS 2.0 investigation. Bill has performed everything from writing shell scripts for server maintenance to presenting his work in collaboration with MetaArchive. He contributed to “Chapter 7: Cache and Network Administration for PLNs” in A Guide to Digital Preservation (2010) Edited by Katherine Skinner & Matt Schultz, presented at Designing Storage Architecture hosted by the Library of Congress in 2009 on MetaArchive & Cloud Computing, and joined the Technical Committee in our 2020 MetaArchive LOCKSS Evaluation.

Bill currently serves as the Lead System Engineer at the Georgia Tech Research Institute.

Kurt Nordstrom has been with MetaArchive since 2015 and is the expert on the MetaArchive user interface Conspectus. Conspectus is a central tool for the network and operations, allowing members to manage collection-level metadata and monitor preservation statuses. Kurt led the ongoing maintenance of the front and back ends of Conspectus. He has maintained Conspectus diligently, migrating it from Ruby to Django Python, serving as a backstop and administrator as needed, and perpetually debugging. 

Kurt currently serves as a Software Developer at Index Data.

Chris Helms has been with MetaArchive since 2012. He became the general Systems Administrator and led all AWS infrastructure initiatives, including provisioning AWS, server setup, writing shell scripts for the administrative server, and consultation. Recently, Chris implemented our new Private LOCKSS Network dashboard to stand it up in AWS and manage the keys for our ingest plugins in the network. Chris is the bridge between our Technical Team and the Transition Team during the MetaArchive’s transition to LOCKSS infrastructure and is the point person for facilitating that transition on the MetaArchive side.

Chris has been at the Georgia Tech Library in various IT roles and currently serves as the Manager of Network Services. 

Clay Miller has been key technical support for MetaArchive since 2011. As a Support Engineer for the LOCKSS Project at Stanford University, he has been integral in keeping MetaArchive up and running. He has made himself available to MetaArchive members for direct, one-on-one support for everything LOCKSS, such as cache-hosting configuration, cache server replacements, Linux upgrades on cache servers, AU rebalancing across cache server filesystems, cache recovery; ingest issues diagnosis, and locating the right documentation for any LOCKSS-related question. Clay will continue to be an ongoing resource for the MetaArchive.

Clay currently serves as a Support Engineer for LOCKSS at Stanford University.

 


November 3, 2023

Announcing the 2024 MetaArchive Leadership Team Candidates

The MetaArchive Cooperative is pleased to announce the candidates for the 2024 Leadership Team election. Details about each position can be found in our Governance Procedures.

Elections will be held from November 6th, 2023 to November 29th, 2023.

Elected Leadership Team members will take office on January 1st, 2024.

The voting representative for each Institutional and Collaborative member will receive a ballot via email on November 13th.


Reid Boehm

Research Data Systems Manager, Purdue University Libraries

Running for: Chair-Elect

Candidate Statement

Working as the Research data systems manager at Purdue University Libraries and managing Purdue University Research Repository is in many ways my dream job because it allows me to work with both the human and technical elements of research data stewardship and with a repository held in high regard within the research data management community. Every day I learn more from researchers, my colleagues, and in the process of developing and sustaining campus partnerships. My doctoral degree is in information science and ever since I’ve worked in academic libraries supporting researchers and working with research data repositories. My background is in research data management, curation, and repositories, starting with an internship at a NASA archive center and a post-doc at Notre Dame Libraries. I enjoy collaborative research on data services and repository related issues. In the past year and a half, since coming to Purdue and getting involved with the collaborative as a member and in working with the Community Research Task Force I am learning multitudes in the realm of preservation, and in a broader extent how a distributed network made up of diverse institutions can sustain and support each other and their system in the face of changes and challenges. I enjoy working with the collaborative, and would like to contribute as a member of leadership to be a part of the effort to support and sustain a valuable network that plays a major role for so many in this field.

Alex Kinnaman

Digital Preservation Coordinator, Virginia Tech

Running for: Treasurer

Candidate Statement

My work at Virginia Tech is largely policy writing, collaborating on developing and implementing preservation workflows, and consulting with my colleagues on preservation needs. My research interests are in repository certification metrics, digital preservation documentation, and the preservation of 3D objects and Digital Humanities projects.

I am running for the Treasurer position on the Leadership Team. I have served as the MetaArchive Treasurer for almost two years as we investigated changes to our financial health and transitioned our budget structure. I believe that given our current time of transition and changing budget priorities, I can provide context and recent historical knowledge to the way our budget is managed. I also serve as co-chair for the Outreach & Membership Services Committee, and have been with the Outreach folks since 2020, and previously served at the Leadership Team Secretary. I look forward to the opportunity to serve and to helping support the forward-movement of MetaArchive. Thank you for your time and consideration.


Hannah Pryor

Archivist for University Records and Records Manager, University of Louisville

Running for: Secretary

Candidate Statement

Hannah Pryor seeks reelection to the role of the MetaArchive Leadership Team’s Secretary. She is an archives and records management professional with 8+ years of experience in state government and higher education. She currently works at the University of Louisville as their Archivist for University Records and Records Manager and often manages born-digital and digitized university records. She is also a member of the Outreach and Membership Services Committee.


October 10, 2023

MetaArchive Concludes Project Mycelium

In February 2022, MetaArchive announced a research and development project to create a next-generation distributed digital preservation solution that later became known as Project Mycelium. The project was a multi-phase effort to leverage commercial-sector infrastructure advancements to simplify digital preservation and increase sustainability. Phase 1 focused on documenting requirements and conceptualizing the design, it was completed in Fall 2022.

MetaArchive worked with Keeper Technology to complete Phase 1 activities that include an environmental scan, prioritization and justification of digital preservation storage criteria, user surveys and interviews, and a design document for a next-generation digital preservation architecture. We used multiple strategies to seek feedback from the digital preservation community including posting to digital preservation related mailing lists, presenting a paper at iPRES 2022, and a presentation at Library of Congress Designing Storage Architectures 2023; each time a link to the design document was shared, encouraging people add comments and suggestions. Pricing estimates for initial development and ongoing maintenance showed that MetaArchive would need to find partners to continue to phases 2 (proof-of-concept) and 3 (production service).

At the same time that we engaged the community on our design document, MetaArchive launched the Community Research Task Force to assess the organization’s financial health, engage in contingency planning, and present options to the membership for remediating technical debt. While the Task Force is continuing some of its work, a result of the process was a decision to strengthen our relationship with LOCKSS and contract with them to perform essential technical operations, including the creation of an anchor node that will hold a copy of all AUs in the network. 

This change was needed to maintain the viability of our network in the short- and medium-terms. It does not limit MetaArchive’s ability to resume technical operations in the future or to pursue other preservation technologies. However, it does mean that now is not the right time for us to continue working on Project Mycelium. We had a lot of fun and learned much during the process and it helped us to solidify our shared understanding and formed a consensus around digital preservation requirements. We hope the artifacts from this project may be helpful to other digital preservation networks and community members. Thank you to everyone who read or commented on the design document, attended our conference sessions, or participated in the process.


October 6, 2023

MetaArchive Fall Update: New Opportunities, New Challenges

The MetaArchive Cooperative is excited to announce that we are partnering with the LOCKSS Program at Stanford University to improve our central administrative infrastructure through increased engagement with the LOCKSS community. This decision was reached through the work of the MetaArchive Community Research Task Force,open communication with the LOCKSS Program team, and an informed MetaArchive community vote. The Cooperative will continue to operate as an Educopia Institute community while contracting out technical support to the LOCKSS Program. 

We are grateful to the LOCKSS Program team for their generosity, openness, and active collaboration during this transition and look forward to working closely with them to improve the MetaArchive Collaborative.

Community Research Task Force

As a member-owned, distributed digital preservation network, MetaArchive is continuously investigating strategies to improve technical capabilities, membership support, and community-based decision-making all while remaining grounded in our Mission, Vision, and Values Statement. To fulfill this mission, the MetaArchive Community Research Task Force spent the first and second quarter of 2023 addressing the cooperative’s  existing technical debt, improving contingency planning capabilities, and lowering the barrier of entry for participation. For more detailed information on the work of the task force, see the June 7, 2023 blog post, “MetaArchive’s Community Research Task Force is Planning for the Future of the Community.”

Reviewing the current status and further sustainability of the cooperative required a critical eye and a commitment to transparency. Specifically, the task force reviewed all aspects of the cooperative which needed improvement. The approach was one of hard realism rather than aspiration, encouraging the community to assess its known pain points, both in the immediate and in the long-term. Initially, the task force focused on improving contingency planning, developing two new procedural governance documents, the MetaArchive Contingency Plan for Sunsetting and the MetaArchive Operating Reserve policy, as well as the creation of a prospective operations budget. With contingency infrastructure in place, strategies to achieve the prospective operations budget resulted in several paths for the collaborative to move forward. 

The task force for the second quarter of 2023 brought three proposals to the community for consideration:

  1. Partnership with the LOCKSS Program, to continue our current technical infrastructure with increased technical support and less overhead.
  2. Staying independent and focusing on fundraising to hire additional staff for increased support.
  3. Initiate sunset in January 2024 which would provide the membership, and the digital preservation community, a generous transition period.

These proposals considered the financial sustainability of the Cooperative, the obstacles and benefits each option provided the membership and the digital preservation community, and areas of possible development for the Cooperative. The proposals were reviewed by the Leadership Team, presented to the Cooperative prior to the annual member meeting for discussion, and provided  a communication plan, with feedback opportunities, via email. The Leadership Team released a ballot to the community in July 2023 and the final decision was option 1, partnership with the LOCKSS Program.  

Transition & Impact

MetaArchive has been using the LOCKSS digital preservation software since its founding in 2004, which makes transitioning technical support to the LOCKSS Program mutually beneficial. During the planning process, the LOCKSS Program team generously proposed a number of modular support scenarios for partnering with MetaArchive. These scenarios are designed with the opportunity to be decoupled in the future if more independence is desirable and achievable. The LOCKSS Program’s Support team is well-positioned to advise MetaArchive on mitigating risk from node turnover and provide assistance as MetaArchive addresses existing technical debt and migrates to LOCKSS 2.0. LOCKSS Program staff will be able to provide user support during business hours, increasing service responsiveness and reliability. As one of the longest-running LOCKSS networks, MetaArchive’s experiences will help the LOCKSS user community better understand the support needs of mature digital preservation collaboratives  that rely on member contributions and decentralized infrastructure. 

The Cooperative is invested in our values and chose a path forward that is consistent with our collaborative, community-focused mission. MetaArchive and its members will continue to be active contributors to the global community of open-source preservation tools and their users, not passive consumers of  proprietary preservation products. As structured, the partnership with the LOCKSS Program effectively brings MetaArchive closer to needed support and services, and increases communication between the LOCKSS Program and the network about technical needs that might be shared with the rest of the LOCKSS user community. The LOCKSS Program has always had a stake in MetaArchive’s success and sustainability. This partnership is an affirmation of the values both entities hold in common, as well as the ways in which effective digital preservation demands adaptation to changing technology and institutional capacities over the long term. A transition team composed of members from the LOCKSS Program team, MetaArchive Leadership Team and Technical Committee, and Educopia was formed in August 2023. Their charge is to document and facilitate the transition of MetaArchive systems and technical support services to the LOCKSS Program team at Stanford by November 2023. 

In the short term, the partnership with the LOCKSS Program provides immediate cost savings that allows the Cooperative more breathing room to develop sustainability plans and pursue external funding. There is also an immediate increase in technical and software implementation support during regular business hours. Looking further out, the technical support transition frees up staff and volunteers to focus on community management and governance support, including strategies for external funding. The long-term impact is a more sustainable, robust, and continuously member-driven distributed digital preservation community with a greater capacity to serve its members and adapt to the changes inherent in the digital humanities.


June 7, 2023

MetaArchive’s Community Research Task Force is Planning for the Future of the Community

The MetaArchive Cooperative broke ground in community-based digital preservation when it was founded in 2004. Over its first decade, it grew to be one of the first international, member-owned, distributed digital preservation networks. Other digital preservation organizations, particularly in the community of LOCKSS networks, were able to learn from MetaArchive and refine on its early implementations of LOCKSS and community governance model.

Now at the end of its second decade, MetaArchive is still dedicated to community-driven digital preservation. MetaArchive offers unique value to its members, as one of the only examples of an independent member-owned cooperative that supports decentralized, risk-mitigating digital preservation storage. As a cooperative, MetaArchive thrives on the investments that its members make to the organization. These investments, however, have become increasingly difficult for MetaArchive members to sustain over time. As IT infrastructure in academia and cultural heritage organizations is overwhelmingly outsourced to the cloud, maintaining an on-premise node for a network like MetaArchive starts to be seen as a “boutique” service that is untenable for some of our members. This shifting IT landscape has also made it difficult for current and prospective members to advocate internally for membership in an organization like MetaArchive, because the value of distributed, decentralized digital preservation storage may not be well-understood. These shifts obscure the fact that MetaArchive, as a long-standing consortial effort between academic and cultural heritage organizations, is uniquely positioned to provide responsible stewardship for digital materials and advance digital preservation work across the field.

The digital stewardship commitments made by MetaArchive operate on an inherently longer timescale and look to a fundamentally different set of impact measures than other kinds of digital work. Knowing this helps us define our success by our ability to build capacity for our collective action initiatives, anticipate change, adapt our ways of working, and think strategically about resourcing. This definition of success for community-driven digital preservation guides the current work of the Community Research Task Force. The task force’s primary objective is to research, document, and present information about the current state of the community, and to make an evidence-based recommendation to broader membership regarding MetaArchive’s strategic directions.

In this current change process, MetaArchive is grounded in its core mission of engaging in sustainable digital preservation through community collaboration, while operating with the knowledge that we cannot expect a different result by trying the same things. We have to do things differently to get the long term success we are all working towards. In this case, doing things differently means:

Addressing the costs of existing technical debt

In the first quarter of 2023, we have created an ideal operations budget, intended to move MetaArchive out of an austerity mindset and into a more sustainable resourcing model. In the second quarter of 2023, we are researching and proposing different fundraising strategies to the membership that may help us reach these goals.

Improving the ability of the cooperative to plan for contingencies

In the first quarter of 2023, we created and updated several procedural governance documents, including an operating reserve policy and a contingency plan for sunsetting the network. During this process, we spoke to a number of other digital preservation networks about their own practices for succession and contingency planning. We are also exploring partnerships with other networks and services, as well as a strategic reassessment of our infrastructure, that will make MetaArchive more resilient to change.

Identifying the service capabilities that need to be developed in order to meaningfully lower the barrier to entry for participation in the network

In the second quarter of 2023, we are engaging in one-on-one conversations with all members, gathering information about current barriers to participation in the network. We have identified both ingest and reporting functionalities as major areas for improvement, and we are working in partnership with the LOCKSS team at Stanford to identify areas for improvement and support between MetaArchive and LOCKSS. Another area of focus for this work is revisiting our pricing model, in order to make sure that MetaArchive is lowering financial barriers where possible in order to foster an equitable approach to digital preservation.

We recognize that MetaArchive’s future not only impacts our own members, but also our strategic partners, stakeholders, and the broader digital preservation community. The success of community-driven digital preservation rests on a framework of transparency and deep collaboration, and we are moving through this process in that spirit. If you have any questions about this work and MetaArchive’s future directions, please contact the Leadership Team at ma_leadership@metaarchive.org.

MetaArchive Community Research Task Force (May – July 2023)

Reid Boehm
Brandon Locke
Shanna Smith
Zach Vowell
Hannah Wang
Christine Wiseman

Thank you to the previous members of the MetaArchive Community Research Task Force (January – April 2023) for all of their work and leadership:

Alex Kinnaman
Jessica Meyerson
Nathan Tallman


November 28, 2022

Announcing the 2023 MetaArchive Leadership Team Candidates

The MetaArchive Cooperative is pleased to announce the candidates for the 2023 Leadership Team election. Details about each position can be found in our Governance Procedures.

Elections will be held from November 28th, 2022 to December 9th, 2022.

Elected Leadership Team members will take office on January 1st, 2023.

The voting representative for each Institutional and Collaborative member will receive a ballot via email on November 28th.


Shanna Smith

Collection Information Specialist, Isabella Stewart Gardner Museum

Running for: Chair

Candidate Statement

Shanna Smith is the Collection Information Specialist at the Isabella Stewart Gardner Museum (ISGM). Since starting at ISGM in 2020, she has gradually become more involved in the MetaArchive Cooperative. She currently serves as the secretary of the Leadership Team and co-chair of the Membership Services Committee. She looks forward to 2023 and the opportunity to continue to contribute to and serve the MetaArchive community.

Christine Wiseman

Head of the Digital Services Department, AUC Woodruff Library

Running for: Chair-Elect

Candidate Statement

As Head of the Digital Services Department at the AUC Woodruff Library, she serves as the primary liaison and technical support for the HBCU Library Alliance’s membership in the MetaArchive Cooperative. She has really enjoyed her role serving as a liaison to the Alliance, hosting the LOCKSS server, and advocating for the preservation and access of collections from HBCUs. Recently, there is escalating interest among HBCU Library Alliance members in expanding the breadth and depth of their digital presence, and Christine looks forward to integrating digital preservation into that effort. Involvement in the MetaArchive Cooperative has provided the AUC Woodruff Library staff with opportunities to contribute to the profession in enriching ways including serving on committees and working groups. They have expanded their expertise and knowledge around digital preservation practices and broadened their network of professional contacts. Membership in MetaArchive has tangentially connected them with high profile grant projects including the Library Publishing Coalition Workflows Project and the OssArcFlow (Born Digital Archival Workflows) Documentation Project.

Christine has gained much professionally from her involvement in MetaArchive during her tenure at AUC. As the cooperative finds itself at a pivotal moment of change, she hopes she can contribute to the vision and forward progression as the MetaArchive Cooperative approaches two decades of providing community based digital preservation.


Hannah Pryor

Archivist for University Records and Records Manager, University of Louisville

Running for: Secretary

Candidate Statement

Hannah Pryor is an archives and records management professional with 7+ years of experience in state government and higher education. She currently works at the University of Louisville as their Archivist for University Records and Records Manager and often works with born-digital and digitized university records. She has previously served as recording secretary for the Oklahoma Archivists Association and the board of commissioners for a state agency. Currently a member of the Membership Services Committee, she would love the opportunity to get more involved and use her organizational skills to serve MetaArchive.