For a video producer working on a factual/documentary production, editing video interviews is a time-consuming activity.
In the current work-flow, once the producer has the video file, they would have to find the quotes, which often entails manually scrubbing through the video in search of usable sound bites, which can be something like finding a needle in a haystack, particularly if the video is lengthy. Additionally, transcriptions are not a viable option either, as often there is no time to wait for them or no budget, and without time- codes, transcriptions do not help to speed up the process.
In a fast paced newsroom often there is no time for such a time-consuming process.
The aim of the autoEdit project is to produce an application that can maximise not only the depth of content but also the speed in which the content is produced.
The approach was to look at the traditional paper-editing workflow from documentary production, and see how that could translate into a digital world. For more on paper-editing and how this is used to craft compelling stories out of video interviews, check out the slides from this workshop.
One reason why this project is interesting is because it utilises the new possibilities that have opened up with the node ecosystem, that allow for great level of code reusability and the introduction of the HTML5 video tag. As well as the opportunities that have arisen in combining this with the increasing quality of speech to text technologies.
Not only node coupled with npm and yarn package management system allows for a modular component based approach but also projects such as NWJS (formerly node-web-kit) and Electron, allow to use web technology to build cross platform desktop applications that with minimal code change can be ported to an equivalent web app version.
Most recently, Adobe Common Extensibility Platform (CEP) running on chromium enabled close integration of web technologies with video editing software such as Premiere. This makes is possible to create "panels" that have access to the functionality of the video editing software, opening up the possibility to more closely integrate with the video editor and producer's workflow.
Combined with a mobile first approach, this allows to develop desktop, web, mobile version of the same app without the traditional overhead that this would involve.
This also lower the barrier of entry for this type of development.
One of the challenges around this project is providing a comprehensive work-flow for the journalist and the user that is clear and easy to follow through from beginning to end, without requiring any training.
The fastest way to be able to quickly select a caption from a video, is to search a transcription of that video. Therefore a challenge was to research and identify the most appropriate system to get the transcription of any given video. Human generated transcriptions are subject to time and resource constraints. While automated transcriptions need to meet a certain threshold of acceptance from the user, as well as have accurate timecodes in order to be searchable and in sync with the video for quick feedback.
Paper-editing it's only one stage in the documentary production when editing video interview, it generally corresponds to a rough cut stage, where a video sequence is assembled in a video editing software. This is then followed by a fine cut, where the video sequence is polished, cut-aways/B-roll shots are added to it etc..
It was therefore important for the output to integrate with video editing software.
After some research, it turned out that the best output for the "digital paper-editing"(a selection of text from different transcriptions) in autoEdit export part, as a "data interchange format" that was able to connect to a video editing software is an EDL. Which stands for Edit decision list, a plain text file format, that describes a video sequences and is widely compatible with non linear video editing software.
This proved easier and more widely compatible to work with then an XML.
In subsequent version it turned out that packaging the app as a Adobe Panel (Adobe Common Extensibility Platform (CEP)) can make the creation of a sequence in Adobe Premiere even more frictionless. As opposed to when importing an EDL the clips are offline and need to be reconnected.
This documentation is meant to serve as resource for the development of autoEdit. With the higher level overview of the structure, parts and components of the application.
Then it also contains a roadmap section, and a QA section to serve as a checklist before every deployment, as well as a series of appendix to collect technical info and implementation details relevant to the project , such as for example the db setup, prerequisite, ffmpeg & ffprobe packaging in electron, to mention a few.
The R&D Doc section, contains the notes used to keep a record of how problem domain issue have been addressed, which options have been tried and considered and what is the current implementation strength and weaknesses.
- Component/part description
- Related projects. Eg parts that look good, or previous implementations. But have been used for current implementation options.
- Implementations Options considered
- Current implementation
- What needs refactoring
Trello board is used for planning and project management. (Altho at the moment needs cleaning up and it's mostly gathering notes for feature ideas) Github issues are used for bug handling. There is also a waffle dashboard that provides a trello like view of the github issues to help with organisation.