Limecraft unveils Subtitling for Short Form Video

We were invited to speak at the 2023 Media City Bergen Future Week on the subject of Subtitling short form video. About the why, the what and the how of captioning in the Creative Economy.

What is short form video?

Short form videos are relatively brief in duration, typically one or a few minutes in length. Intended to deliver a concise story in the shortest possible time, short form videos have gained significant popularity recently as they are key to enrich websites or make up social media stories. They can contain anything from hard news, over educational or informational content, DIY tutorials, product demonstrations, and quick tips to lightweight entertainment. The key to understand the value short form video, is to look at the bigger picture. In this blog, we will discuss the subtitles.

Short form video is often characterised by their fast-paced editing, creative storytelling, and the use of various visual and audio effects. The format’s popularity has led to the emergence of many content creators and influencers specialising in short-form video production, often using innovative techniques to engage and entertain their audience. In particular, due the fast-paced aspect, the typical turn-around time of conventional subtitling is prohibitive. Nevertheless here is why you should care about subtitling video, especially short form.

Why you should care about captioning Short Form Video as well

More and more content is produced in short form format for online distribution. Whether you are a occasional producer or a large-scale broadcaster, proper subtitles give you a competitive advantage over your competition. And, compared to conventional television content, this is even more relevant for short-form content.

The 3 key reasons for subtitling online video are the following:

• Accessibility of content for the deaf and hard of hearing was the original driver of provide same-language subtitles. An estimated 10% of the people perceive audio differently or not, due to which the audio is not accessible to them the way it does for most people. Same language subtitles or captions help them to enjoy the content. For this reason, broadcasters and content producers worldwide are obliged to provide captions on their content.
• A secondary and related driver is to reach people that are not hampered by a hearing impairment, but that are simply not in a position to use audio (e.g. because they are commuting), or that simply prefer captions alongside the audio. An estimate of 44% of the content consumed online is watched with subtitles on. To maximise the reach of your content, you should cater for high-quality subtitles.
• The third and arguably the most important driver to deliver short form content with subtitles in a separate or side-car file, is Search Engine Optimisation (SEO). Our research shows that video search engines like youtube attribute significant value to the availability of high-quality captions. Consequently your content will be considered more relevant and rank higher compared to similar content without complementary subtitles.

For these and other reasons, broadcasters all over the world commit to deliver 95% or more of their content – including non-broadcast material – with high-quality subtitles. Video on Demand operators worldwide demand subtitles (original and translated) for all content. Subtitling video content has become common sense.

What are some of the challenges when captioning Short Form Video?

Now, in countries like Belgium we have been using subtitles since the ‘70’s, albeit initially for localisation purposes, so what is the problem with short form? The common denominator is cost and turn around time, and they are linked.

Let’s first take a step back. Subtitling is incredibly more complex than letterboxing a transcript by maximum 40 characters on a line. Good subtitles are as close to invisible as possible. And, the better they are, the more intelligence and creativity is required to turn the transcript in good subtitles. More specifically, the transcript is rewritten to turn the spoken word in more compact language, to replace numerical formats, etc. Line breaks are optimised for readability, and overall subtitles must be synchronised with the rhythm of the edit. For example, the BBC style guide, if printed out, is a 80 page document.

Adding to the complexity, each broadcaster and streaming service typically has their own specific rules. Last but not least, different screen sizes may command specific styling rules. For example, while typical television content uses 40 characters on a line – exceptions not taken in consideration, the recommendation for square and vertical video is to use larger font sizes and to allow no more than 27 or 28 characters to ensure readability. The again, FAST channel operators, as their content is viewed on all types of screens, ask for something in the middle.

Now, taking into account than conventional subtitling for a reading speed of 180 words per minute roughly takes 8-10 minutes per minute, the cost of conventional subtitling is 5-10€ per minute. The aggregated cost of a typical broadcaster publishing up to 250 hours of short form content per month, adding up to 75.000€-150.000€ per month. This is either a significant direct cost (in case the process is outsourced) or a huge opportunity cost (loss of time when journalists or video editors have to manually create subtitles themselves). Moreover, if you would decide to outsource the problem, the turn around time will be days, unless you are prepared to pay a markup for a premium service. Short form content, as discussed, is usually fast paced and the turn around time is prohibitive in either case.

By using AI transcription and subtitling at the core of your workspace, if done rightly, you kill two birds with one stone. To do so, language models must be highly optimised for accuracy, you need to use an intelligent subtitling queueing algorithm, and the composite AI service needs to be seamlessly embedded in your workspace to avoid excessive renders, file transfers and copy-and-paste. In this respect, short form content is critical: every second counts, and each wrongly spelled word, faulty line break, or each useless click destroys usability. Here is how to crack the challenge.

How you can crack Subtitling of Short Form Video

So, AI to the rescue. On the one hand side, short-form video is mostly suitable for automation. It is usually single language, single topic, and the number of speakers is limited. Using properly recorded audio, word error rates may go down to 1% or 2%. When using a composite AI service like the one made available by Limecraft, the turn around time is less than a minute, the cost is 0,25€ per minute and post-editing is minimal.

Limecraft makes available AI transcription and subtitling as part of your workspace

Compared to software vendors that come from an ASR perspective (otter.ai, sonix, speechmatics, AppTek), the secret sauce of Limecraft and why it is effective for creating subtitles, is it’s video-centric architecture. It allows Limecraft to demonstrate more effective transcription and better spotting of subtitles.

• Effective transcription: you can use existing scripts, or any production metadata in combination with custom dictionaries to bring the Word Error Rate further down. Accurate speaker segmentation and punctuation are essential for the timing of the subtitles.
• Rewriting the transcript: good subtitles are different from clean cut machine translation, because the text is translated from a spoken language flavour into a written form. Not only because usually the spoken words are usually faster and more abundant; also language tics, repeated words have to be removed and things like numerical formats and currencies have to be substituted. Limecraft rewrites the verbatim transcripts into a more suitable form.
• Better spotting: using Limecraft, you can define and apply one or more styling templates that cope with styling and timing of captions. Also Limecraft uses advanced Natural Language Processing (NLP) to judge the sweet spots for placing the line breaks. Last but not least, Limecraft is the only subtitling service capable of synchronising the subtitles with the rhythm of the edit.

💡 What are spotting rules?

And that’s not all; when considering the use of AI subtitling for short form, there is yet another challenge. When creators cutting in Avid or Adobe, and they would have to stand by and nurse a render, transfer it manually to Limecraft, execute AI transcription and subtitling, export the result, and drag and drop it back into their editor, it would be a perceived as usability nightmare because the processing time would be 3x or 4x the length of the clip.

As always, Limecraft strives to eradicate manual work and to help you avoiding file transfers and copy-and-paste, and to shortcut as many clicks as possible. It saves time, and it massively improves your overall security level and environmental footprint. This is why we built an Adobe panel and an Avid plugin. With a single click, items are rendered in the right format, exported to Limecraft, transcribed and subtitled, all of that within a minute.