Hi everyone, Shira here. Last weekend I attended the Association of Moving Image Archivists Conference in Savannah, GA. For those of you who don’t already know, AMIA is a nonprofit international association dedicated to the preservation and use of moving image media. Although the conference has traditionally focused more on issues surrounding the preservation of analog film and video, in recent years it has brought the subject of digital preservation to the fore.

This year was no exception. One of the three curated programming streams that comprised this year’s AMIA conference was devoted to addressing the open source software in use within the digital preservation community, and by my count, 22 of 53 panels (~42%) were directly related to digital preservation. There was also significant buzz around the projects that were developed as part of the second annual Hack Day, the goal of which is to design and/or improve upon practical, open source solutions around digital audiovisual preservation and access.

Although enough ground was covered at AMIA to provide days’ worth of blogging fodder, my post today is going to focus on the Open Source Digital Preservation & Access stream, which served as a showcase for a variety of exciting tools for the preservation and access of digital video and born-digital moving images.

AMIA swag bags

What does “open source” mean, and why is it important for digital preservation?
Open source software is made available with a license that permits users to freely run, study, modify, and re-distribute its source code. Open source tools are usually developed in a public, collaborative forum such as GitHub or Gitlab. This means that any users can improve, fix, or add onto the source code over time.

Using the open source model to develop tools for digital preservation has a number of advantages over proprietary software. The principal benefit is cost; most open source tools are available to the public at no cost, which is a big deal for many perennially cash-strapped organizations in the archives community. Another benefit is that the open source model is versatility. Making the source code available to the public allows tools to be perpetually refined and modified according to the needs of a particular group, and as Bill LeFurgy explains in a 2011 blog post on The Signal, open source also, “gives organizations the opportunity to stitch together a preservation system from existing components rather than laboriously start from scratch.” The last benefit I’ll mention here is that open and freely accessible code is far simpler to preserve than closed proprietary code, making it more likely that open source tools themselves will be around in the future.

#osdpa / #amia14
In case you missed the AMIA conference but still want to look into some of the things that were discussed there, you can always play catch-up by following the #osdpa and #amia14 hashtags on Twitter. AMIA has a contingent of active tweeters (myself included), and the good news is that the intrepid Ashley Blewer has put together a twarc archive of tweets from this year’s AMIA conference in JSON and text formats (available here).

She’s also put up a list including links to additional information of all the technologies discussed during the conference, either within the Open Source Digital Preservation & Access Stream, at the Hack Day, or at-large. It’s an extremely valuable resource and I highly recommend giving it a look.

Trevor Thornton talking as part of the “Open Source Tools, Technologies and Considerations” panel. Photo courtesy of Kathryn Gronsbell

Interview with Chris Lacinak
Although following the #osdpa hashtag will give readers a good sense of the talks that comprised the open source stream, I wanted to offer some background information on the stream and why it was put together. Before leaving Savannah I spoke with AVPreserve’s Chris Lacinak, who curated the Open Source steam. Our conversation is below:

Shira Peltzman (SP): How did the open source stream come into being?

Chris Lacinak (CL): Last year the Digital Library Federation (DLF) and AMIA sponsored the first AMIA/DLF Hack Day and there was a lot of excitement within AMIA about that event. Thanks to the hard work of Kara Van Malssen, Lauren Sorenson, and Steven Villereal it was a big success and was perceived well, not just by the people that took part, but by the rest of the membership at large; it gave them an opportunity to engage with that type of event and see what kinds of things happen there. The energy and buzz that came out of the first Hack Day was great. I was the AMIA board liaison to the Open Source Committee and at the committee meeting the members made it clear that they wanted open source to be a greater part of AMIA. We talked about ways to make that happen and one of the ideas was to have an Open Source stream as part of the conference. I went back and pitched it to the board and they were very much in favor, asking me to be the stream’s curator.

SP: I noticed that many of the panels at the Open Source stream were standing room only. Why do you think this stream was so popular?

CL: Standing room only, and that was accidental! Originally the room we were in was supposed to be half the size that it actually was. Clearly we touched a nerve within the membership. Software has become an integral component of digital preservation practice. Open source software has been embraced wholeheartedly by the archival community largely based on preservation principles as well as budgetary considerations. However, there is still a lack of clarity regarding the process and component parts that make up open source software projects, and people are really interested to get their heads around this. It was also interesting that lots of the people filling the seats had minimal experience with open source or Hack Days or anything like that. So a lot of people were new but were clearly enthralled with the topics and types of tools they were seeing. The other thing is that obviously both digital preservation and access are huge things; people are hungry for content around digital media in general so this served both the digital preservation and access folks and also the open source interest.

Inaugural session of the Open Source Digital Preservation & Access stream at AMIA 2014

SP: Are there any trends that you noticed this year among the presentations in the Open Source stream?

CL: I think that both the tools and the understanding have reached a level of maturity that’s very real now. Lots of conversations on open source within the community a few years ago were painted in an experimental light, as if it were something for folks on the fringe and not for “real” archives. Now it feels very central and real. In the stream presentations I really noticed, one, the sophistication of the tools, two, all of the presenters were very articulate and three, the audience was able to receive and process the information in a way that hasn’t happened in the past. So I think there’s definitely a maturity within this ecosystem that’s new and interesting and let’s operate on a different level than in the past.

SP: What are some of the open source projects that you’re most interested in seeing developed in the coming year that were discussed at Hack Day and during the open source stream?

CL: First and foremost I want to point everyone to the Hack Day wiki which also has links to all of the Hack Day projects on GitHub. (Find this here).

All of the projects are really amazing and deserve being reviewed. The prize winning projects this year were Video Sprites! and Hack Day Capture, with AV Artifact Atlas getting a special jury prize. Personally I would encourage people to take a look at the Video Sprites!, the Video Characterization Comparison Viewer and the ffmpeg Documentation projects. Speaking of a documentation project, another thing that I loved this year was the addition of an Edit-A-Thon (part of AMIA/DLF Hack Day), because what you find is that a large number of open source tools are poorly documented, seriously limiting their usability. Documentation projects really answer a huge need, so I think it’s great to have gotten this done. It’s extremely important and valuable work.

In the stream, QC Tools is really exciting. I think it’s an amazing tool with a feature set that rivals commercial offerings, and I’m excited to see that continue to grow, have new features added, and be used by more organizations. I am interested in combining QC Tools with our tool, MDQC in one package. Erik Piil gave a lightning talk on an open hardware cleaner he’s working on which is awesome. And on a larger scale, MoMA’s development of the first digital preservation repository for museum collections is a phenomenal project. It’s hard to pick though because they really are all great. If I didn’t think they were awesome I wouldn’t have picked them for the stream! The entire stream will be posted online for those that are interested in watching the presentations. Keep an eye on the AMIA website.

SP: Well congrats on a really successful stream.

CL: Thank you.