General What caught our Eye in 2020 and how Technology will shape Media in 2021 Maarten VerwaestJanuary 20, 2021 A bird’s eye perspective of the economic effects of the lockdown in 2020 in the Media and Entertainment sector. A massive explosion of VOD, cloud services and Artificial Intelligence are likely to create a burst of innovation. We look at the prospects of what will become the “Roaring 2020s”, and make some key recommendations for 2021 and beyond. Take-aways More video on demand requires localisation or customisation of content, which will eventually lead to hyper-personalisationMore versions inflate storage requirements and bandwidth consumption; compound file formats like IMF possibly coming to the rescueRemote production works, thanks to recent evolutions of network technology in conjunction with cloud services. We expect high-resolution content and craft editing workflows to go online as wellWhile online storage is still relatively expensive, prices are likely to decrease strongly in the coming 12 to 24 months, making push back on vendor lock-in paramountWhere today’s AI services turn spoken word into text and make educated guesses of depicted persons and objects, the next generation of AI technologies will be more intelligent, massively increasing chances of retrieving the right content and possibly featuring creative decisions AI services and automation create new strands of errors and quality problems, necessitating a new approach to quality control Looking back Necessity is the mother of invention. History suggests that periods of significant upheaval and stress inspire people to overcome cultural inertia, overthrow old assumptions and consider new imperatives. Some of today’s biggest companies were created during or in the aftermath of the financial crisis in 2008. Paypal, WhatsApp, Uber and Airbnb were all started by ambitious entrepreneurs who might have ended up in a comfortable job at a big corporate, bank or management consultancy if it weren’t for the difficult economic environment at the time. 2020 was unlike any other year in history. The rules of the game have been fundamentally changed, almost overnight. Producers, post-facilities, as well as learning institutions and businesses in general, have been scrambling to adjust and to overcome challenges they could not have imagined, establishing new capabilities, products and services in a matter of days and weeks – rather than months or years. Retailers and cafes launched gift and subscription services; whisky and gin distilleries became hand sanitiser or ventilator factories. Due to the lockdowns, many organisations across sectors suddenly embrace digital communication and video production in a way that hadn’t happened before. While some of these innovations will not stick around for the long term, many will. As new habits have been wired and old behaviours don’t return. As Mark Harisson of the DPP suggests, 2021 is likely to become even more confusing and stressful. First, we will be constrained in our behaviour for enough of the year and most companies will be operating on greatly reduced budgets. Secondly, however, we will be able to go out again and meet with other people, allowing consumer spending and releasing an extraordinary burst of creative energy. The switchback in social and economic conditions will hugely impact the media industry. Native internet-based services and their suppliers are in for another strong year. Meanwhile, companies with more legacy will either emerge like a phenix, if they manage to reposition themselves, or they will perish, burdened by budgetary austerity and organisational restructures. It will be a great time to be in production – notwithstanding the constant ask to do more for even less. Yet it will be a year of catastrophic evolution: the industry leaps forward, with many falling by the wayside. Key Trends in 2020 1. Cord Cutting While we were obviously expecting Subscription based Video on Demand (SVOD) and streaming services to steal market share from traditional broadcast, cable and satellite operators, lockdowns across the world caused a landslide victory for the cord cutters. The global annual increase in SVOD subscriptions in Q3 2020 reached 217.6 million, the highest ever, beating the previous record of 211.7 million in Q4 2018. The UK digital video market grew by 37.7% in 2020 to €3bn as viewers flocked to services from the likes of Netflix, Disney+, Amazon Prime Video, Sky’s Now TV and Apple TV+. According to Centre national du cinéma et de l’image animée (CNC), 50% of the French households now have a Netflix subscription. Due to the lockdowns, Video on Demand services became mainstream faster than expected The effects imposed by the evolution from conventional (regional) broadcast to online (global) distribution can hardly be underestimated, imposing new requirements on producers, while at the same time asking them to operate on tighter budgets. These seemingly contradictory requirements have created a push towards a more industrialised approach that would otherwise have been hard to accept for creative people. In particular, the following issues have a material effect on the production community. To create maximum exposure to an audience as large as possible, dialogue fragments need support by subtitles in various languages. Assuming you are producing content intended for non-domestic distribution, you have to realise there’s a deluge of content and consumers tend to get lost. Distribution might be taken for granted but exposure is not. That said, the potential of your content stands with the control over its relevant ‘master data’. These data ensure content is retrievable by search engines and that the recommendation services of the VOD operators recommend such content. More importantly, accurately time-coded master data will allow for new business models based on connecting media and retail, potentially replacing traditional advertising models that have been eroded by end-users being able to skip the commercials. Considering the former, the market for localisation services exploded. Producing content for a global audience requires producers and distributors to at least translate subtitles or create audio tracks for different languages. However, rather than to outsource the localisation process at a rate of 10€ per minute of processed material, we expect producers and distributors to take advantage of AI becoming more intelligent (see below?), enabling them to insource the process and thereby retaining their intellectual property rights. Using state of the art Automatic Speech Transcription (ASR), Machine Translation (MT) andNatural Language Processing (NLP) for subtitling Another interesting evolution is the insatiable appetite of SVOD operators for original content, thus looking to establish a competitive edge. For example, Amazon Prime is aggregating an impressive catalogue, distributing Premier League and now reportedly talking to Formula 1 as well. Roku deserves a special mention here, having acquired the content catalogue produced by Quibi. Despite their hunger for exclusive content, streaming operators will eventually look for economies of scale and engage in co-production agreements with original content producers, eventually creating a larger critical mass for the producer. As a producer, you’ll have a great time. 2. Exodus to the Cloud Using the cloud for production purposes was the talk of the town the last couple of years, but it took a pandemic to get rid of local hard drives. The good news, however, is that many producers have discovered that further to running their post production workflow in the cloud, it gives them a competitive edge in terms of operational stability, scalability of their production apparatus, flexibility, superior security, and being future proof. Consequently, cloud-based service providers got an incredible boost in 2020; more content was produced with less hardware than ever before. Virtualising and automating well-known production techniques like de-rushing, automated file serving, video editing, audio mixing, and mastering just makes sense. An increasing number of producers and broadcasters pursued this model as it is the key to supporting multiple content platforms as mentioned above. So the media industry finally became an industry. Expect producers to insource more tasks that would otherwise have been outsourced to post-production facilities, as software and cloud solutions become more intelligent (see below?) at the expense of post-production facilities. Considering that, at the same time, other parts of the post-production business, including file exchange, archiving and video transcoding, are rapidly replaced by cloud-based solutions offered by BigTech and thus becoming commodity, we recommend post-production facilities to reconsider their position and to step up in the value chain. As suggested earlier in a special report on production technologies by the IABM: “Consistently managing flows of master data requires bridging the gap between digital production islands and to radically opt in for a more integrated approach. Companies that fail to adapt will not survive the great extinction.” Adapt or perish, nature’s inexorable imperative. 3. Artificial Intelligence, sort of “Been dazed and confused for so long it’s not trueWanted a woman, never bargained for youLots of people talk and few of them knowSoul of a woman was created below”(Led Zeppelin) Despite Artificial Intelligence having been studied and developed by scientists for over 70 years, automated services today are capable of no more than convincingly delivering point solutions, such as converting spoken audio into timed text or detecting people’s faces. There is a huge gap between the available technology, and usable information that brings value. The thresholds to adoption for some of these services have almost disappeared due to their availability as a cloud service, resulting in new so-called “AI specialist” companies being created at the speed of light. Now, if data is the new oil, as suggested by Peter Hinssen, expect a lot of disillusionment at the side of the drilling companies. Not surprisingly, Gartner positions “Data labelling and Annotation Services”, “Natural Language Processing” and “Deep Neural Networks” exactly on the peak of inflated expectations. The core technology is available to early adopters, we encourage you to investigate how it can service your purpose and we recommend you not to take it for granted and to handle it with care. For example, if you plan using AI for content discovery, you must ensure the indexing is both accurate and complete. Likewise, if you consider using Automatic Speech Recognition (ASR) as part of your subtitling workflow, you don’t want spelling errors or worse disfiguring your content. Based on the results of several implementations, we discovered that the key to making AI acceptable for professional use is quite simple: it has to save time. Without a proper implementation, the cure will be worse than the pain; the noise louder than the signal. Otherwise, by choosing the right technology, having it properly trained, and all of that made available by a user interface that allows for human quality control and post-editing, automation can emancipate hundreds of thousands of hours of manual and repetitive work. Then again, the best results will be delivered by the combination of the best Artificial and the best Human Intelligence. What the near future will look like 1. The next Generation of VOD Services Global VoD players today manage several thousands of versions of the same item. More and more, localisation is more than translating subtitles; it also involves adaptation of the content of a product or service to a particular market or culture. At this point in time, the biggest hurdle to adaptation of further adaptation and customisation of content is storage and in particular the cost of storage. Companies like Eluvio propose to assemble personalised versions of content on the go, but it is a far from standard approach, and the majority of producers and distributors today store excessive copies of almost similar content. The industry seeks consensus around formats designed to circumvent this problem (including DCP or Digital Cinema Package, and IMF or Interoperable Mastering Format), but despite consistent efforts by the DPP and the EBU, stakeholder’s opinions on the best way forward are dispersed. Media Asset Management solutions supporting file formats like IMF, optimised to contain different versions of the same content, drastically reduce the storage cost For the time being, we recommend challenging software vendors explicitly on their capabilities to store, manage and process file formats that support versioning, including but not restricted to different audio tracks or subtitle files. It will potentially save you an order of magnitude on the recurring storage cost. Once the industry manages to handle customisable content by scalable methods, producers will likely explore new creative avenues, eventually leading to hyper-personalisation of content, stories scaled and cropped to the interests of individual consumers. 2. Further Proliferation of the Cloud While original footage and insurance files may still be secured using on-premise physical storage for the time being (there are only so many risks intelligent people are willing to take), project teams have been able to finish projects by moving content directly from the camera to the cloud and almost instantly make it available to authorised specialists to work on at their workstations or remotely in the cloud. For example, the workflow concept of Disney’s Safety, designed by executive producer Doug Jones, applied episodic television technology to a feature film pipeline. Feature film production tends to be slower moving, while episodic television comes with fast turnaround times. This led Jones to a super-efficient “online-all-the-time” pipeline that could save time and money. The biggest obstacle to avoiding proxies and storing high-resolution footage in the cloud, is that there isn’t just one cloud. There are dozens of service providers around the globe and they each do certain things differently. They have one particular aspect in common though, i.e. they want you locked in as much and as long as possible. Public cloud vendors cement their dominance by discount policies and cross-selling tactics, that are designed to relentlessly support the objective of getting as much data as possible on their premises, and to avoid leakage to third parties. Until cloud interoperability is rock solid, as a producer, you will have to deal with it. 3. Artificial Intelligence beyond Imagination While we currently experiencing an enormous proliferation of AI services and applications, though often based on no more than a few core technologies, brace for the level of intelligence that will be at our disposal in the coming 12 to 24 months. Corto.ai recognises emotional tonality Companies like Corto.ai are now capable of judging emotional tonality; Robovision – for what’s in a name – and Mobius claim superhuman visual interpretation; Valossa and IBM Whatson are pushing hard towards automatic video editing. Without willing to present you with a complete overview, Vidrovr deserves a special designation. Not only are Vidrovr’s services capable of detecting people (not mere faces!) and gestures (“shaking hands”); inference algorithms on top create an accurate and meaningful description for every single shot. As a result, not only the chances of retrieving the right fragment for editing in pre-existing archive content or original footage drastically increase, the whole process of creating a shot list is much faster and more efficient compared. With all these technologies coming to life, we appreciate the temptation to just buy and try. However, like often, it is often better to think before acting. There is a wide variety of services, all behaving slightly different and being optimised for different purposes. Moreover, technology vendors might not tell you that so called “machine learning” algorithms need to be properly trained. Without proper training, the results will be poor and inconsistent, leaving the user in frustration and disappointment. To avoid surprises, we recommend you to seek a knowledgeable partner that can help you with the implementation. Conclusion and Key Recommendations The media and entertainment industry is bruised but not beaten. History suggests that major crises often spawn the most creative results. And while it not possible to predict the future in very detail, we all agree that this is a Black Swan moment, and that we are enabling new habits as we speak. In the light thereof, here are some key recommendations to put you and your company in the best possible position for whatever the future is bringing. If you are a producer, create content with the intention of global distribution. Seek co-production. Make sure your content stands out from the crowd by curating your metadataConsider insourcing of localisation of your content. It’s easier than ever before, and you retain your intellectual property rightsIn general, content must be localised, customisable and presumably personalised in the future. Look for solution providers that support suitable file formats like DCP and IMFCloud technologies are here to stay. Look for the benefits but handle with care. Once you’ve decided to move online, watch out for storage and egress costs. Look for service providers that understand the different options and that can help you migrating from one public cloud provider to another when appropriateArtificial Intelligence is taking the first baby steps. It has the potential to emancipate hundreds of thousands of hours of manual labour, but you should not take the benefits for grantedNever leave AI unattended, don’t consider relying on unsupervised automated services. To avoid surprises, look for a solution provider that offers a breed of AI services and that can help you with a proper implementation. To learn more about media trends and advanced post-production workflows. Sign up to our monthly newsletter. Subscribe !