Ingest and Metadata Management, the Dance

Submitted by Tristan Ringering – Wazee Digital
As I sit down to pen this revelation on the complexities and challenges of content ingest and metadata management, I find my head spinning as if it were the first time the concept ever entered my mind. The truth is, I actually do this for a living and still, the task is that daunting.

So, what’s the big deal? Surely by now the industry has begun to normalize after the great migration from analog to digital and then again from on-premise digital archive to cloud storage (on-demand). Right. At Wazee Digital, we have stored it all. Film, videotape, disk, LTO, cloud. Offline, nearline, online. It’s madness! And, nearly impossible to keep up. If you’ve been on the operations side of the broadcast media and entertainment industry for any length of time, you already know this. The industry has changed so fast that by the time you have weighed your options and decided on the best – most future-proofed step forward – it all changes before you can execute the plan. We’re all stuck in this awkward dance, stepping on each other’s toes trying to stay one step ahead (which, by the way, doesn’t work well. You can’t be a step ahead in a dance). So I arrive at my revelation: It is a dance. Not a race.

Sure, it’s a race to be at the cutting edge of a technology solution but it is a complicated dance to arrive at the right conclusion in step with your customer base. If you move too fast, you risk abandoning hard fought relationships with high-value content owners who may not have the immediate resources to steer their enormous ship of an archive into the future as fast as the next fleeting viral sensation. If you move too slow, you are overtaken by the collective revolt against ancient and archaic bygones as the industry flies headlong into the next big thing.

So, we dance. Wazee Digital’s media and metadata ingest operations simultaneously test the limits of our cloud services offerings while at the same time maintaining support for more traditional means of data transfers. To borrow from Andrew Tanenbaum, “never underestimate the bandwidth of a station wagon full of tapes hurtling down the highway.” …or hard drives on an airline. How else do you transfer media effectively from (or to) regions fraught with power grid failures?

Media aside, metadata management has its own troubles; from determining the best schema for each use case to ensuring the proper formats are maintained. The journey is not over when the media arrives on our servers. If the metadata is incomplete, inaccurate, incorrectly formatted, or missing entirely, then you don’t have useful media, you just have a new storage problem (opportunity?).

Managing and conquering these challenges is a cross-functional endeavor. In fact, it is so central to the success of our business that it touches every part of our organization. As we execute this complex digital media dance routine, we swirl about to the perpetual beat of the global business machine. With over a decade of practice, I like to think we’re getting pretty good. Of course, we’re always working to perfect some fresh new moves ever mindful to not stray too far from the fundamentals.