FFmpeg – IMG.LY Blog

A Guide to Batch Video Editing & Server Automation with FFmpeg

Robin — Mon, 10 Nov 2025 10:33:40 GMT

The explosion of video content across platforms from social media and streaming services to e-learning and surveillance has made manual video processing increasingly impractical. As organizations manage thousands of clips, the need for automation has become critical to handle transcoding, resizing, thumbnail generation, and other repetitive tasks efficiently. Manual workflows quickly become bottlenecks, consuming engineering time and introducing inconsistencies that slow down production and distribution.

In this article, you’ll go through a Docker-based batch video processing system powered by FFmpeg, complete with both a REST API and a command-line interface (CLI). This setup enables scalable, reproducible, and automated media operations suitable for local or cloud environments. You can deploy this server on cloud providers – our guides for running FFmpeg on AWS Spot Instances and running FFmpeg on GCP walk you through setup on each platform. By the end, you’ll understand how to design infrastructure that can process hundreds of videos reliably and in parallel. To follow along, you’ll need a basic understanding of Python, Docker, and common video formats such as MP4 and MOV. For a primer on running FFmpeg inside a Docker container, see our Docker guide.

You can clone this github repository and follow the article for a working Docker-based batch video processing system with FFmpeg handling the heavylifting.

Understanding Video Processing Fundamentals

FFmpeg is a powerful command-line toolkit capable of reading, transcoding, filtering, and muxing virtually any media format. It’s the foundation of countless commercial encoders, streaming platforms, and post-production tools. With a single command, you can convert between codecs, trim segments, extract audio, or even apply complex filter graphs and when these commands are chained together in scripts or APIs, they form the backbone of scalable media pipelines. For readers unfamiliar with FFmpeg, our Ultimate Guide to FFmpeg covers the basics and advanced techniques.

At its core, most video workflows revolve around a few recurring operations: transcoding (changing codecs or bitrates), compression (optimizing size for delivery), watermarking (branding or copyright protection), and thumbnail extraction (visual previews for cataloging). Each operation impacts performance differently depending on your hardware setup. CPU-based processing offers flexibility but can be slow for high-resolution workloads, while GPU acceleration through NVENC or VAAPI dramatically speeds up encoding at the cost of hardware dependencies. Managing memory and I/O efficiently is crucial, as reading and writing large video files can quickly become a bottleneck in batch pipelines.

Understanding the file format landscape and the quality–size trade-off is key to producing professional results. Modern codecs like H.264, H.265 (HEVC), and VP9 dominate for their balance of efficiency and compatibility, typically wrapped in container formats like MP4, MOV, or WebM. Tools like FFmpeg expose fine-grained control through parameters such as CRF (Constant Rate Factor), bitrate targeting, and encoding presets, allowing you to tune the ideal compromise between visual fidelity and storage or bandwidth requirements.

The FFmpeg Batch Video Processor builds on these fundamentals to demonstrate how FFmpeg can power fully automated, scalable media workflows. Packaged as a Docker-based, production-ready system, it automates complex operations, such as transcoding, compression, watermarking, and thumbnail generation across multiple videos simultaneously. By combining a FastAPI-powered REST API with a command-line interface (CLI), it provides flexible control over batch processing, progress tracking, and workflow automation. With built-in profiles, concurrent workers, and robust logging, it turns FFmpeg’s raw capabilities into a modern, reproducible video processing pipeline suited for professional production, web delivery, and large-scale content management. For tutorials on cropping and trimming at the client level, see our Flutter guide.

Now lets dig deeper into the project.

Architecture Deep Dive

System Components Overview

The system is modular by design, separating API logic, job management, and media processing into distinct layers for clarity and scalability. Lets look into the different layers of the project now:

**FastAPI REST interface (api.py)**This file defines the FastAPI application for our batch video processing API powered by FFmpeg. It provides endpoints for managing video processing jobs, profiles, workflows, and file uploads. Key components include:The API supports operations like transcoding, compressing, adding watermarks, and more, using profiles and workflows for predefined configurations. With the help of this API, you can create a dashboard to monitor the jobs and manage them too.
1. Global Instances:The VideoProcessor executes FFmpeg-based tasks like transcoding and compression, while the JobQueue manages concurrent job execution through a worker pool. The ConfigManager maintains processing profiles and workflows, ensuring consistent parameters across all jobs.
2. Endpoints:The API provides endpoints for managing jobs, profiles, and workflows, as well as handling uploads and system monitoring. You can create, list, retrieve, cancel, and download jobs; access predefined profiles and workflows; upload videos for processing; check system health and queue stats; and retrieve metadata for uploaded videos.You can go to http://localhost:8000/docs#/ when the docker container is running to see a detailed description of every endpoints.
3. Startup/Shutdown Events:Initializes and cleans up the job queue.
**Job queue management (job_queue.py)**This file implements the job queue system for managing and processing video processing tasks asynchronously. It includes:This system makes batch video processing possible, supporting asynchronous execution and state persistence.
1. Job Class:The class represents a video processing task with attributes such as id, input_file, output_file, operation, parameters, status, progress, timestamps, and error details. It supports operations like thumbnail generation, GIF creation, and audio extraction, automatically generating output file paths. A to_dict method is included for easy serialization of task data.
2. JobQueue Class:The class manages a queue of video processing jobs using a pool of asynchronous workers. It allows adding, retrieving, listing, and canceling jobs, filtering by status, and starting or stopping workers as needed. It also tracks key statistics such as total, completed, and failed jobs and can persist the queue state to a file. Workers process jobs asynchronously, updating progress and handling errors throughout execution.
3. Worker System:This component processes jobs using a processor object that defines the specific operations to perform. It runs these operations within a thread pool to prevent blocking the event loop, while continuously updating job progress and handling both successful completions and failures.
**Video processing engine (video_processor.py)**This file defines a VideoProcessor class that provides a set of methods for video processing using FFmpeg. Key features include:The class uses subprocesses to interact with FFmpeg and supports progress callbacks for real-time updates.
1. Video Metadata Extraction:
  - get_video_info: Extracts video details like duration, size, resolution, codec, and frame rate.
2. Video Operations:
  - transcode: Converts videos to different formats/codecs.
  - compress: Reduces video size with optional scaling or target size.
  - add_watermark: Adds a watermark to a video at specified positions.
  - generate_thumbnail: Creates a thumbnail image from a video.
  - extract_audio: Extracts audio from a video in various formats.
  - create_gif: Converts a video segment into a GIF.
  - concatenate_videos: Merges multiple videos into one.
  - trim_video: Trims a video to a specific duration or time range.
3. FFmpeg Execution:
  - _execute_ffmpeg: Handles FFmpeg command execution with progress tracking and error handling.
**Configuration system (config_manager.py)**This file defines a ConfigManager class that manages video processing profiles and workflows stored in a YAML configuration file. Key functionalities include:This class is essential for managing reusable configurations for video processing tasks.
1. Configuration Management:Loads profiles and workflows from a YAML file (config/profiles.yaml) and saves any updates or new definitions back to the same file, ensuring configurations remain persistent and easily editable.
2. Profiles:The module allows retrieving specific profiles by name and listing all available profiles with their details. It validates each profile to ensure required fields like operation and parameters are defined, and also supports creating custom profiles dynamically for flexible and reusable video processing configurations.
3. Workflows:The module can retrieve a specific workflow by name and list all available workflows along with their descriptions and the number of jobs they contain, providing a clear overview of predefined processing pipelines.
4. Error Handling and Logging:Logs warnings for missing profiles/workflows and errors during file operations.

Design Patterns

The FFmpeg-batch system applies several proven software design patterns to ensure maintainability and scalability. It follows a producer–consumer model through its job queue: the FastAPI layer acts as the producer by submitting jobs, while asynchronous worker processes act as consumers, executing FFmpeg tasks concurrently. The system also supports profile-based processing templates, where predefined configurations, such as codec settings, bitrates, and resolutions allow teams to standardize repetitive tasks. On top of that, workflow orchestration ties multiple operations together, enabling complex pipelines (for example, transcoding followed by thumbnail extraction) to run automatically in sequence.

Scalability Considerations

From a scalability perspective, the project uses worker pools to parallelize workloads efficiently, ensuring multiple videos can be processed simultaneously without overloading the system. Effective resource management including queue throttling, memory optimization, and controlled CPU utilization keeps performance consistent even under heavy load. This modular and scalable design makes it straightforward to extend the system horizontally, whether deploying additional Docker containers, scaling Kubernetes pods, or integrating cloud-based distributed processing.

Setting Up the Development Environment

Before building or running the FFmpeg-batch system, it’s essential to configure a proper development environment that mirrors production conditions. The setup ensures smooth testing, consistent builds, and predictable behavior across machines.

Prerequisites Installation

You’ll need to have Docker, Python 3.11, and FFmpeg installed. Docker handles containerization and makes it easy to run the application and its dependencies consistently. Python 3.11 provides the runtime for FastAPI and background processing components, while FFmpeg performs the actual video operations. Once installed, you can verify your setup using commands like docker --version, python3 --version, and ffmpeg -version.

Project Structure Walkthrough

The project is organized into clear modules for scalability and maintainability. Core components include api.py for the REST interface, job_queue.py for task management, video_processor.py for media operations, and config_manager.py for loading and saving profiles and workflows. Additional directories like data/input/ and data/output/ are used for managing uploaded and processed files.

Configuration Setup

The system uses environment variables (.env) to manage runtime settings like the maximum number of workers or file storage paths and a profiles.yaml file to define reusable encoding profiles and workflows. By editing these configurations, you can customize how the system behaves, whether you’re running quick local tests or deploying to a cloud environment.

Docker Compose Deep Dive

The Docker Compose configuration defines a single service, video-processor, which builds from the local Dockerfile and runs the FFmpeg batch processing API. It maps port 8000 from the container to the host, allowing you to access the FastAPI interface externally. Several volumes are mounted including ./data/input, ./data/output, and ./data/logs to persist uploaded files, processed results, and runtime logs outside the container lifecycle. The ./config directory is also mounted to provide access to YAML configuration files for profiles and workflows. Environment variables such as MAX_WORKERS, API_HOST, and API_PORT define runtime parameters, while PYTHONUNBUFFERED=1 ensures immediate log output. Finally, the restart: unless-stopped policy guarantees the service automatically restarts if it crashes or the system reboots, making the setup robust for both local development and long-running batch processing tasks.

Development vs. Production Setup

In development mode, you can use live-reload for FastAPI and local volumes for quick iteration and debugging. In production mode, Docker Compose can be extended to include load-balanced worker containers, persistent storage, and monitoring tools. This setup enables horizontal scaling and high availability while maintaining the same base configuration used during development ensuring consistent results from your laptop to the cloud.

Configuration Management & Profiles

A core strength of the FFmpeg-batch system lies in its flexible configuration layer, which defines reusable processing templates and multi-step workflows through a YAML-based profile system. This allows developers and media teams to standardize encoding, compression, and transformation parameters while maintaining full control over quality, size, and performance trade-offs.

Profile System (`config/profiles.yaml`)

Profiles define how a video should be processed, specifying the operation type (e.g., transcode, compress, extract_audio) and its parameter set (e.g., codec, preset, CRF, or scaling). Each profile acts as a self-contained template that can be reused across multiple workflows or customized for specific business needs.

Common profiles include:

web_optimized — transcodes videos to H.264 with a medium preset and CRF 23 for web streaming.
social_media — compresses videos to 720p targeting 50 MB, ideal for platforms like Instagram or TikTok.
mobile_optimized — scales videos down to 480p and limits file size for efficient mobile delivery.
audio_mp3 and audio_aac — extract audio tracks at defined bitrates and formats for podcasts or companion files.
thumbnail and preview_gif — generate visual previews or short animated snippets to enhance content catalogs.

Creating custom profiles is straightforward: teams can define their own processing templates to meet business-specific requirements such as watermarking, caption embedding, or broadcast-compliant exports. The YAML format also supports profile inheritance, enabling developers to extend base configurations (like web_optimized) with minor overrides, ensuring consistency while reducing redundancy.

Workflow Orchestration

Beyond individual operations, the system supports multi-step workflows that chain multiple profiles into a single automated pipeline. For example, the social_media_package workflow combines compression, thumbnail generation, and GIF preview creation, all triggered as one batch job. Similarly, the archive_package workflow transcodes video at high quality, extracts MP3 audio, and generates a thumbnail for long-term storage. You can create your own workflows with your own custom profiles that fits your usecases and use them for your applications with this project.

Workflows also enable parallel execution, allowing independent operations (such as audio extraction and GIF creation) to run simultaneously for faster turnaround. More advanced pipelines can incorporate conditional logic, such as adapting encoding settings based on input resolution or target platform requirements. Additionally, batch operations can be applied to entire directories, processing hundreds of videos in a consistent, repeatable manner.

Together, the profile and workflow system transforms FFmpeg-batch from a simple video processor into a scalable, declarative media automation framework, one that adapts to varied production environments and evolving content demands.

Command Line Interface

The Command Line Interface (CLI) in the FFmpeg-batch system offers a fast, intuitive way to trigger and monitor batch processing jobs directly from the terminal. Built on top of the FastAPI backend, the CLI (cli.py) simplifies automation for developers, video engineers, and content teams who prefer command-line tools or need to integrate FFmpeg-batch into larger production pipelines.

CLI Design (`cli.py`)

The CLI is designed around clear, operation-based command grouping. Users can run commands like process, upload, list, or download to manage the entire lifecycle of a batch job, from input submission to result retrieval. Each command maps closely to the API endpoints, ensuring consistency between programmatic and manual use.

In interactive mode, the CLI provides progress bars and real-time feedback for active jobs, displaying percentage completion, estimated time remaining, and current processing status. This helps operators monitor workloads without constantly querying the API.

The CLI also supports batch operations, allowing users to process entire directories with a single command (e.g., cli.py process-folder ./input_videos --profile web_optimized). It automatically applies the chosen profile to all files and stores results in the designated output directory. In CLI, you can run the below docker-compose command to execute it in a docker container:

# Using a profile
docker-compose exec video-processor python cli.py profile /data/input/video.mp4 web_optimized --output /data/output/video.mp4

Finally, the CLI integrates seamlessly with shell scripting and automation workflows, perfect for CI/CD pipelines, cron jobs, or custom scripts that need to trigger video processing as part of larger media workflows. You can also create custom operations like the one given below which transcodes with custom parameters:

# Transcode with custom parameters
python cli.py create /data/input/video.mp4 transcode \\\\
  --params '{"codec":"libx264","preset":"slow","crf":18}'

Conclusion

The project demonstrates how even complex, large-scale video operations can be automated using open-source tools like FFmpeg, FastAPI, and Docker. By abstracting individual processing tasks into profiles and workflows, it enables reproducible, scalable, and platform-agnostic video processing pipelines from content creators automating their daily exports to enterprises handling thousands of assets.

Whether you deploy it locally or in the cloud, this architecture can serve as the foundation for more advanced systems like adding GPU acceleration, distributed job queues, or integration with cloud storage. The full source code is available on GitHub, so you can clone, modify, and extend it for your specific production needs.

How to run FFmpeg inside a Docker container

Robin — Wed, 29 Oct 2025 12:22:12 GMT

FFmpeg is a powerful multimedia framework used for video and audio processing. From transcoding and editing to streaming, its versatility makes it a go-to tool for developers and media professionals. However, installing FFmpeg directly on your system can sometimes lead to dependency conflicts or version mismatches.

This is where Docker comes in. If you’ve never used Docker before, think of it as a platform that lets you package applications and all their dependencies into something called a container. A container is like a lightweight, portable box that holds everything your application needs to run operating system libraries, binaries, and configurations. Unlike virtual machines, containers don’t include a full guest operating system, which makes them much more efficient and faster to start up. If you’re unfamiliar with FFmpeg itself, our Ultimate Guide to FFmpeg provides a comprehensive introduction.

The main takeaway is this: Docker ensures that your application runs the same way everywhere, whether on your laptop, a colleague’s computer, or a cloud server. That means no more “it works on my machine” problems. With FFmpeg inside a Docker container, you get a clean, isolated environment for media processing that you can easily replicate or share with others.

In this guide, we’ll walk through two approaches for using FFmpeg with Docker:

Running FFmpeg with a pre-built Docker image.
Building your own Dockerfile for more customization.

Before we start, we need to setup Docker on our machine. Lets do that now

Installing Docker

Before working with FFmpeg in Docker, you need to have Docker installed on your system. The installation process varies depending on your operating system, but Docker provides excellent step-by-step instructions.

Check system requirements. Docker Desktop works on macOS, Windows, and Linux distributions. Make sure your system meets the minimum requirements.
- Windows / macOS: Download Docker Desktop from the official download page. Run the installer, and follow the on-screen instructions. Once installed, Docker will run as a background service.
- Linux: On Linux distributions like Ubuntu, you can install Docker directly via the package manager. For example:

Test Docker with a hello-world container. Run the following command to check that Docker is working correctly:

docker run hello-world

This command pulls a small test image from Docker Hub, runs it, and prints a confirmation message.

Verify the installation. After installation, open a terminal (or PowerShell on Windows) and run:

docker --version

This should display the installed Docker version.

Install Docker Desktop (recommended for beginners)

sudo apt-get update
sudo apt-get install docker.io

Alternatively, follow the Linux installation guide for your distribution.

Once you see the “Hello from Docker!” message, you’re ready to start working with FFmpeg inside containers.

Approach 1: Using a Pre-Built FFmpeg Docker Image

The quickest way to get started with FFmpeg in Docker is to use an image that’s already prepared for you. One of the most widely used options is the image maintained by jrottenberg.

Pulling the Image

Start by opening your terminal and pulling the official FFmpeg image:

docker pull jrottenberg/ffmpeg

This downloads the latest FFmpeg image and makes it available on your system. To deploy your containerized FFmpeg pipelines on low‑cost cloud infrastructure, see our guide on running FFmpeg on AWS Spot Instances, and for deployment on Google Cloud, follow our guide on running FFmpeg on GCP.

Running a Container

Once the image is available, you can launch a container and access FFmpeg inside it:

docker run -it jrottenberg/ffmpeg bash

This command starts a container, opens a bash shell inside it, and gives you access to FFmpeg as if it were installed directly on your machine.

Using FFmpeg

Inside the container, you can run FFmpeg commands just like you normally would. For example, to convert a video file from MP4 to AVI:

ffmpeg -i input.mp4 output.avi

If your input files are on your host machine, you’ll want to mount them so the container can access them. A common pattern is:

docker run --rm -v $(pwd):/data jrottenberg/ffmpeg -i /data/input.mp4 /data/output.avi

Here, your current working directory is mounted into the container at /data, allowing you to process files without copying them into the container.

-rm : Automatically remove the container when it exits (keeps things clean).
-v $(pwd):/data : Mounts your current directory ($(pwd) prints it) into /data inside the container.
jrottenberg/ffmpeg : The ffmpeg image.
ffmpeg -i /data/input.mp4 /data/output.avi : The FFmpeg command that runs inside the container, reading from and writing to /data.

Want to build an app that crops and trims videos? Our Flutter tutorial shows how to crop and trim videos in Flutter using FFmpeg within a mobile UI.

Exiting the Container

When you’re done, simply type exit to leave the container and return to your host terminal.

Approach 2: Building a Custom Dockerfile with FFmpeg

Sometimes, you might need more control over your environment—for example, specifying which version of FFmpeg you want to use or installing additional tools. In that case, building your own Docker image is the better choice.

Creating a Dockerfile

Start by creating a file named Dockerfile with the following content:

FROM ubuntu:latest
RUN apt-get update && apt-get install -y ffmpeg

This simple Dockerfile uses Ubuntu as the base image and installs FFmpeg on top of it.

Building the Image

Navigate to the directory containing your Dockerfile and build the image:

docker build -t custom-ffmpeg-image .

This creates a new image called custom-ffmpeg-image.

Running Your Custom Image

Now you can start a container from your custom image:

docker run -it custom-ffmpeg-image bash

This gives you the same FFmpeg functionality but within a controlled environment that you designed. Now, you are currently in the terminal of the UbuntuGitHub system with ffmpeg installed and ready to use. You can either mount your volume to the container or copy the input video to the container to process it with ffmpeg.

You can use this GitHub repository for the code.

Beyond the Basics

Depending on context and requirements, there are a few additional ways people run FFmpeg in Docker that you might find useful. Lets sneak-peak into those approaches now:

Multi-stage builds

Instead of installing FFmpeg on top of Ubuntu in a simple Dockerfile, some projects use a multi-stage build where the first stage compiles FFmpeg from source (with specific options or codecs), and the final stage copies only the compiled binaries into a minimal image (like alpine). This results in much smaller images, which is valuable for production environments.

FROM alpine:latest as build
RUN apk add --no-cache build-base yasm
RUN git clone <https://git.ffmpeg.org/ffmpeg.git> ffmpeg \\
    && cd ffmpeg \\
    && ./configure --enable-gpl --enable-nonfree \\
    && make && make install

FROM alpine:latest
COPY --from=build /usr/local/bin/ffmpeg /usr/local/bin/

Using FFmpeg inside a larger containerized workflow

In practice, FFmpeg is often not run alone. For example, it might be used as part of a media processing pipeline in Kubernetes or Docker Compose with other services like Nginx, Node.js, or a message queue. In those cases, FFmpeg runs inside a dedicated container, and files are passed to it via mounted volumes or network streams. For a Docker‑based pipeline with REST APIs and worker queues, see our production‑ready batch video processing server guide.

GPU-accelerated FFmpeg builds

For users doing heavy video processing, using NVIDIA GPU support is crucial. Docker supports GPU acceleration via -gpus all and NVIDIA’s base images. There are specialized FFmpeg images (like nvidia/cudabased builds) that enable hardware-accelerated encoding/decoding (NVENC, NVDEC, CUDA).

docker run --gpus all -it nvidia/cuda:12.2.0-base ffmpeg ...

This is extremely relevant for modern workloads (e.g., streaming, AI video processing).

Which Approach Should You Choose?

If you only need FFmpeg occasionally and want the fastest way to get started, the pre-built image is the way to go. It saves time and effort, and you’ll be up and running within minutes.

On the other hand, if your workflow requires specific versions of FFmpeg, or if you’d like to bundle other tools into the same container, creating a custom Dockerfile gives you that flexibility.

While Dockerized FFmpeg is powerful on its own, its real strength becomes clear when integrated into larger developer workflows. For example, CreativeEditor SDK (CE.SDK) provides a full-featured, customizable editing environment for images, graphics, and videos. When developers extend CE.SDK with Dockerized FFmpeg, they unlock a scalable way to handle demanding media tasks behind the scenes.

Conclusion: Why FFmpeg in Docker is the Smart Choice

Running FFmpeg inside Docker combines the power of a world-class media framework with the reliability and portability of containers. Instead of fighting dependency issues on your local system, you get a clean, isolated environment that works consistently across machines, teams, and production servers.

If you need speed and simplicity, a pre-built image like jrottenberg/ffmpeg gets you up and running in minutes.
If you want control and customization, a custom Dockerfile ensures your build has the exact codecs, libraries, and dependencies you need.
For advanced scenarios, GPU acceleration, multi-stage builds, or cloud pipelines, Docker ensures FFmpeg scales well into modern production environments.

And beyond standalone usage, frameworks like **CE.SDK** demonstrate how FFmpeg’s capabilities can power full-featured creative applications. Whether it’s transcoding video, exporting edits, or automating workflows, running FFmpeg in Docker keeps your setup clean and future-proof.

By combining Docker’s portability with FFmpeg’s versatility, you’re not just solving installation problems, you’re setting up a foundation for professional, scalable multimedia processing.

FFmpeg on GCP: Step-by-Step for Beginners

Robin — Wed, 08 Oct 2025 06:56:08 GMT

Introduction

When it comes to working with video and audio files, FFmpeg is one of the most powerful and widely used tools available. It’s an open-source command-line utility capable of handling almost any kind of media conversion, compression, and processing task. Whether you need to convert a video from MP4 to WebM, extract audio from a movie, or resize a large video for the web, FFmpeg is often the go-to solution for developers and media professionals alike. For an in‑depth overview of FFmpeg commands and features, read our Ultimate Guide to FFmpeg.

While you can install and run FFmpeg locally on your computer, there are limitations. Processing large video files on a personal machine can be slow, resource-intensive, and impractical if you need to handle multiple files or larger workloads.

This is where Google Cloud Platform (GCP) comes in. By running FFmpeg on GCP, you gain access to scalable cloud infrastructure that can process media files much faster than most local setups. You can choose the right machine size for your task, scale up when handling heavy jobs, and only pay for the resources you use. If you prefer Amazon Web Services, our guide to running FFmpeg on AWS Spot Instances covers a similar setup.

By the end of this guide, you’ll know how to set up a Google Cloud project, run FFmpeg on a virtual machine, and process videos efficiently in the cloud. This tutorial is designed with beginners in mind, so even if you’re new to cloud platforms or FFmpeg, you’ll be able to follow along step by step.

Set Up a Google Cloud Project

Before you can run FFmpeg on Google Cloud, you’ll need to create a project where all your resources, such as virtual machines and storage, will live. Think of a Google Cloud project as a workspace that keeps everything organized and secure. Each project has its own settings, billing, and resources, so you can keep experiments separate from production work.

1. Create a New Project

Start by going to the Google Cloud Console. In the top menu, click on the project selector and choose New Project. Give your project a name (for example, ffmpeg-demo) and, if prompted, select an organization. Once created, this project will serve as the container for all the steps that follow.

2. Enable Billing

Google Cloud requires billing to be enabled in order to use most services. If you haven’t set this up yet, you’ll be asked to link your project to a billing account. Don’t worry, Google Cloud offers a free tier with credits for new users, which is more than enough to complete this tutorial and experiment with small workloads.

3. Set Up Google Cloud Storage

Next, you’ll need a place to store your media files. Google Cloud Storage (GCS) works like a highly reliable online hard drive. You can upload your videos there, process them with FFmpeg on a virtual machine, and then save the results back to the same bucket.

In the Google Cloud Console, navigate to Cloud Storage → Buckets.
Click Create Bucket and choose a unique name (e.g., my-ffmpeg-videos).
Select the default storage class and location that best matches your region.
After the bucket is created, click Upload Files and add a small test video, this will be the file you process later in the tutorial.

At this point, you have a project set up with billing enabled and a storage bucket containing your test video. Need to crop or trim your videos before uploading? Our Flutter guide demonstrates how to crop and trim videos in Flutter using FFmpeg. You’re now ready to create a virtual machine that will run FFmpeg.

Note: Not all services are enabled when you start working with GCP. If you are prompted to enable a service while working on this setup, please do enable it and keep track on it in API & Services.

Start a Compute Engine VM

With your project and storage bucket ready, the next step is to create a Compute Engine Virtual Machine (VM). A VM is essentially a computer in the cloud, you decide how powerful it should be, and you only pay for the time it’s running.

1. Choose a Machine Type

Head over to the Compute Engine section in the Google Cloud Console and click Create Instance. You’ll need to configure a few options:

Name: Pick something descriptive like ffmpeg-vm.
Region & Zone: Choose a region close to you or your users for faster upload/download speeds.
Machine type:
- For testing and small videos, a smaller instance like e2-medium (2 vCPUs, 4 GB memory) is usually enough.
- For larger video processing jobs, choose something more powerful such as n2-standard-4 (4 vCPUs, 16 GB memory) or higher. The more CPU and memory you assign, the faster FFmpeg can handle big files.

💡 Tip: You can always resize or recreate the VM with a larger machine later if you find it’s too slow.

To automate large numbers of FFmpeg jobs using REST APIs and worker queues, refer to our batch video processing server guide.

2. Configure Boot Disk

By default, Google Cloud gives you a Debian or Ubuntu Linux image, which works perfectly for FFmpeg. You don’t need to change anything here unless you prefer another distribution.

3. Configure Access Scopes

By default, the instances can only read from a bucket, but we need to upload the output back to the bucket, so in Security tab set the Access Scopes to Set access for each API and set Storage to “Read Write”.

3. Create and Launch the VM

Click Create and wait a moment, Google Cloud will provision your VM. Once it’s ready, you’ll see it listed under VM Instances in the Compute Engine dashboard.

4. Connect via SSH

To start using the VM, click the SSH button next to your instance in the console. This opens a secure terminal session directly in your browser. You’re now logged into your VM and ready to install software, just as if you were sitting in front of a fresh Linux machine.

At this stage, you have a working VM connected to your project. Alternatively, you can run FFmpeg inside a Docker container for a reproducible environment - our Docker guide shows how. In the next section, you’ll install FFmpeg on this machine and run your first test command.

Install FFmpeg

FFmpeg is available in the default Debian/Ubuntu repositories that Google Cloud’s standard Linux images use, so installation is just a couple of terminal commands. For more information on FFmpeg for Linux distros, see here.

1) Install via APT (recommended)

# Update package lists
sudo apt update

# Install FFmpeg and ffprobe
sudo apt -y install ffmpeg

# Verify installation
ffmpeg -hide_banner -version
ffprobe -hide_banner -version

If you prefer the very latest FFmpeg, you can also download a prebuilt Linux static build from the official FFmpeg downloads page (no extra system libraries required).

2) Quick sanity checks

# See available encoders/decoders in your build
ffmpeg -hide_banner -encoders | head -n 20
ffmpeg -hide_banner -decoders | head -n 20

Refer to the official command reference if you want to understand what each option means.

Test FFmpeg with sample commands

You can test FFmpeg without any input file by generating a short test pattern using the built-in testsrc video source filter:

# Generate a 3-second 1280x720 MP4 test video
ffmpeg -f lavfi -i "testsrc=duration=3:size=1280x720:rate=30" \\
  -c:v libx264 -pix_fmt yuv420p ~/test.mp4

The testsrc filter creates a color bars pattern with a timestamp—perfect for verifying your install.

Tip: If you encounter “Unknown encoder” for a codec, either switch to another codec available in your build or install a newer build (e.g., the official static builds mentioned above).

You’re now ready to pull a real file from Cloud Storage and process it on your VM in the next section.

Process a Video

Now that FFmpeg is installed on your Compute Engine VM, let’s process a real video. The workflow looks like this: download a file from Google Cloud Storage → process it with FFmpeg → upload the result back to Google Cloud Storage.

Download Video from GCS to the VM

First, make sure the gcloud CLI is installed on your VM (it usually is on Google-provided images). Then, copy the test video from your storage bucket:

# Replace with your actual bucket and file names
gsutil cp gs://my-ffmpeg-videos/test.mp4 ~/test.mp4

This command uses gsutil to copy the file into your VM’s home directory.

Run a Conversion with FFmpeg

Let’s do a simple format conversion and resizing example:

# Convert MP4 to WebM, resized to 720p
ffmpeg -i ~/test.mp4 -vf "scale=1280:-2" \\
  -c:v libvpx-vp9 -b:v 1M -c:a libopus \\
  ~/output.webm

Here’s what this command does:

i ~/test.mp4 → input file.
vf "scale=1280:-2" → resizes video to 1280px wide while keeping aspect ratio.
c:v libvpx-vp9 -b:v 1M → encodes video with VP9 codec at ~1 Mbps.
c:a libopus → encodes audio with the Opus codec.
~/output.webm → output file.

When it finishes, you’ll have a new WebM file in your VM’s home directory.

Upload the Result Back to GCS

Finally, copy the processed file back into your storage bucket:

# Replace with your bucket name
gsutil cp ~/output.webm gs://my-ffmpeg-videos/

You can now view or download the processed video directly from your GCS bucket.

For a ready to use example of the above architecture, you can refer to this github repository.

Other Options on GCP

While running FFmpeg on a Compute Engine VM is the most straightforward way to get started, Google Cloud offers several other options that might better suit your needs depending on scale and workflow.

Cloud Run for Small Jobs

If you want a serverless option, Cloud Run lets you package FFmpeg into a container and run it on demand. This is a great fit for lightweight jobs or API-style services, where you don’t need to manage servers at all. You simply pay for the CPU and memory used during each request, making it cost-efficient for occasional media processing.

Transcoder API for a Managed Service

For teams who want to skip infrastructure entirely, Google offers the Transcoder API. It’s a fully managed video processing service where you just provide the input file and the desired output format, and Google handles the rest. It’s especially useful for production pipelines with predictable formats, but it’s less flexible than rolling your own FFmpeg setup.

Kubernetes for Large-Scale Workloads

If you’re dealing with a high volume of media jobs, consider deploying FFmpeg in a Google Kubernetes Engine (GKE) cluster. This allows you to scale horizontally, running multiple FFmpeg pods in parallel to process many videos at once. While it requires more setup, it provides the foundation for a production-grade, auto-scaling media processing system.

Beyond FFmpeg: A UI for Creative Editing

FFmpeg is excellent for batch conversions and automation, but what if you want to offer end-users a creative editing experience directly in the browser? This is where solutions like CE.SDK from IMG.LY come in. CE.SDK is a fully customizable editing SDK that you can integrate into your app, enabling Canva-grade photo, video, and design editing features on top of your processing pipeline. In fact, combining FFmpeg’s backend power on GCP with CE.SDK’s front-end UI gives you the best of both worlds: automated conversions plus user-driven creativity.

Conclusion

By now, you’ve seen how straightforward it is to run FFmpeg on Google Cloud Platform. With just a few steps—creating a project, setting up a Compute Engine VM, installing FFmpeg, and connecting it to Cloud Storage—you can process videos in the cloud as easily as on your local machine. The benefit is that your workflow is now backed by scalable infrastructure: you can choose a lightweight VM for simple conversions or a powerful instance when you need to process large files quickly.

This tutorial has given you a solid foundation, but it’s only the beginning. From here, you can explore automation by scripting your FFmpeg jobs and scheduling them with Cloud Functions or Cloud Scheduler. For scaling, you might package FFmpeg into containers and deploy it on Kubernetes to process thousands of files in parallel. And if you want to save time managing infrastructure altogether, managed services like the Transcoder API let Google handle the heavy lifting for you.

Whether you stick with Compute Engine for flexibility or move toward serverless and managed services for efficiency, running FFmpeg on GCP gives you the power to build media pipelines that grow with your needs.

How to Run FFmpeg on AWS Spot Instances for Scalable, Low-Cost Video Processing

Robin — Mon, 06 Oct 2025 08:25:43 GMT

In this guide, we’ll explore how to harness Amazon Web Services (AWS) for video processing with FFmpeg. You’ll learn why the cloud is so helpful for handling resource-heavy video tasks, how to set up your AWS Spot Instance and S3 bucket, and how to run FFmpeg jobs at scale. Along the way, we’ll get acquainted with AWS Spot Instances, a cost-saving compute option that can make large-scale video processing far more affordable. If you’re new to the tool itself, start with our Ultimate Guide to FFmpeg.

By the end of this article, you’ll know how to:

Set up AWS with S3 storage and a Spot instance.
Install and test FFmpeg in the cloud.
Run video processing jobs on a Spot Instance while keeping costs low.

Whether you’re new to cloud computing or just looking for a practical way to scale your FFmpeg workflows, this guide will walk you through the essentials step by step.

Lets dive into by setting up the AWS environment.

Set Up Your AWS Environment

Before you can run FFmpeg in the cloud, you need to prepare your AWS environment. This involves creating an account, setting up storage, and launching an instance where you’ll install and run FFmpeg. Let’s break it down step by step.

Create an AWS Account and S3 Bucket

If you don’t already have one, sign up for an AWS account. Once logged into the AWS Management Console, the first thing to set up is an Amazon S3 bucket. S3 (Simple Storage Service) is AWS’s object storage system, and it’s ideal for hosting both your input and output video files. For example, you can upload raw videos to S3, process them with FFmpeg, and then store the converted files back in the same bucket. When creating your bucket, be sure to choose a region close to where you expect to run your compute resources, as this reduces latency and transfer costs.

Launch a Spot Instance

Spot Instances

AWS Spot Instances are virtual machines that let you access unused cloud capacity at significantly reduced prices, sometimes up to 90% cheaper than regular On-Demand Instances. The catch is that they can be interrupted by AWS with little notice if the capacity is needed elsewhere. For workloads like video processing, which can be restarted or distributed across multiple nodes, Spot Instances are a perfect fit. They allow developers to take advantage of lower costs without sacrificing performance, provided they design workflows with fault tolerance in mind. This balance of affordability and scalability makes Spot Instances a popular choice for running FFmpeg jobs in the cloud.

Setup a Spot Instance

From the AWS console, go to the EC2 dashboard and launch a new instance. Choose Ubuntu Server or Amazon Linux as the operating system, since both have excellent community support and are well-documented. When selecting an instance type, start with something lightweight like t2.micro (eligible for the free tier), but keep in mind you may need more powerful instances (with higher CPU or GPU) for larger video workloads. Open the Advanced Details, under Purchasing option, check the box labeled Spot Instances to tell AWS you want to use discounted spare capacity instead of On-Demand pricing. You can also set a maximum price per instance hour if you want to control costs more tightly, but in most cases leaving it at the default ensures you’ll simply pay the current Spot market price, which automatically adjusts with supply and demand and is never higher than the standard On-Demand rate.

You can also choose between a persistent or one-time request type in the Customize Spot Instance options.

Once the instance is created, you can confirm that its lifecycle is spot under the instance details section. This indicates that a Spot Instance has been successfully created.

Connect to Your Instance via SSH

Once your Spot instance is running, you can connect to it remotely using SSH (Secure Shell). During instance setup, AWS will prompt you to create or download a key pair (.pem file). Keep this file safe, it’s your private key for secure access. On your local machine, open a terminal at the location where you saved the key pair file and run:

ssh -i "your-key.pem" <username>@<your-instance-public-dns>

Replace your-key.pem with the path to your private key file and <your-instance-public-dns> with the public DNS address of your Spot instance (visible in the console). You can get the exact SSH command for your instance by clicking the Connect button in the EC2 dashboard and selecting the SSH client option. Once connected, you’ll be inside the environment where you can install FFmpeg and start running commands. For a smooth operation, make yourself familiar with basic shell commands for the operating system you opted for in the instance. Examples for various shell commands on a linux system can be found here.

Install FFmpeg on the Spot Instance

With your spot instance up and running, the next step is to install FFmpeg so you can start processing videos. Installation is straightforward, and once complete, you’ll test it with a simple command to confirm everything works.

Update Your Package Lists

Before installing any software, it’s a good practice to make sure your system’s package list is up to date. Run:

sudo apt update && sudo apt upgrade -y

This command refreshes the list of available software and ensures your instance has the latest security patches. (If you’re using Amazon Linux instead of Ubuntu, replace this with sudo yum update -y.)

Install FFmpeg

On Ubuntu, FFmpeg is available directly through the official package repository:

sudo apt install ffmpeg -y

For Amazon Linux, the process may require enabling extra repositories first:

sudo amazon-linux-extras enable epel
sudo yum install ffmpeg -y

This will download and install FFmpeg along with its dependencies. To avoid dependency issues, consider running FFmpeg inside a Docker container.

Verify the Installation

After installation, check that FFmpeg is available by running:

ffmpeg -version

You should see output showing the installed FFmpeg version and configuration details.

Run FFmpeg on a Spot Instance

Now that you’ve installed FFmpeg on a regular Spot instance, it’s time to make evrything work together. In this section, upload a video to Amazon S3, and then run FFmpeg on the spot instance to process the video and save the output.

While it’s possible to upload videos directly to the instance and process them there, this approach isn’t reliable since the instance can be terminated at any time when AWS reclaims capacity. The better and safer practice is to use the S3 bucket, which ensures your files are stored safely and don’t need to be re-uploaded each time. This is especially important when working with large video files or multiple inputs, as S3 provides a more reliable and scalable storage solution. Prefer Google Cloud? See our tutorial on running FFmpeg on GCP for similar steps.

Upload a Sample Video to S3

Before running FFmpeg, you’ll need a video file available in your S3 bucket. From your local machine terminal (not the instance terminal), use the AWS CLI to upload a sample file:

aws s3 cp sample.mp4 s3://your-bucket-name/input/sample.mp4

This stores sample.mp4 inside the input folder of your bucket. Organizing files into input and output directories is a good practice for video workflows. You can do the same using the S3 dashboard in AWS. If your workflow involves cropping or trimming videos before upload, check out our Flutter guide on crop and trim videos in Flutter.

If you are using AWS CLI for the first time in you machine, then setup your credentials using aws configure command. See here for more help.

Run FFmpeg on the Spot Instance

Once your Spot Instance is running, connect to it via SSH (the same way you did earlier). Before we get the video from the S3 bucket, you should set an IAM Role for the instance to access the S3 bucket. For that, go to IAM>Roles>Create Role, select EC2 from use case dropdown menu. On the next page, search for AmazonS3FullAccess and select it, give a role name and finaly create it. Now the instances can access the files from the bucket.

Since the instance is a new system, you will need to install awscli from here. You will also have to setup credentials using aws configure command as mentioned above. Then, copy the input file from S3 down to your instance:

aws s3 cp s3://your-bucket-name/input/sample.mp4 sample.mp4

Now, run FFmpeg to process the video. For example, converting MP4 to WebM:

ffmpeg -i sample.mp4 output.webm

After processing, upload the result back to S3:

aws s3 cp output.webm s3://your-bucket-name/output/output.webm

You can now access your processed file directly in S3 and even configure it for delivery via Amazon CloudFront if needed.

By running FFmpeg jobs on Spot Instances, you can save up to 90% of costs compared to On-Demand pricing, making this setup ideal for batch video processing.

You can refer to this github repository for examples.

Spot Instances vs AWS Lambda for FFmpeg

When running FFmpeg on AWS, you have more than one option for compute. Two of the most common choices are AWS Lambda and EC2 Spot Instances. Each has its strengths and trade-offs, and the best fit depends on the type of video workload you’re running.

AWS Lambda for Lightweight Tasks

AWS Lambda is Amazon’s serverless compute service—you don’t need to manage servers, and your code runs only when triggered. It’s easy to set up and integrates seamlessly with other AWS services such as S3 and API Gateway. For example, you can automatically run an FFmpeg job as soon as a new video is uploaded to your S3 bucket.

However, Lambda comes with limitations: functions can only run for a maximum of 15 minutes, and they have limited temporary storage (512MB by default, extendable up to 10GB). This makes Lambda a great choice for lightweight tasks like generating thumbnails, clipping short videos, or converting smaller files, but not for large-scale processing.

EC2 Spot Instances for Heavy Workloads

EC2 Spot Instances, by contrast, are designed for long and heavy workloads. Spot Instances give you access to unused EC2 capacity at discounts of up to 90% compared to On-Demand pricing. Since they are full-fledged EC2 machines, Spot Instances don’t impose runtime limits or storage caps.

This makes them ideal for large video files, long-running transcodes, and GPU-accelerated workloads such as 4K/8K encoding or complex filter pipelines. The trade-off is that Spot Instances can be interrupted by AWS with two minutes of notification when EC2 needs the capacity back, so jobs need to be either restartable or split into smaller tasks.

Rule of Thumb

A simple guideline to choose between the two is:

Small, quick jobs → use AWS Lambda
Large, long jobs but restartable → use EC2 Spot Instances

By matching the right AWS service to your workload, you get the best balance of cost, performance, and reliability for running FFmpeg in the cloud.

Handling Spot Interruptions

Two-Minute Warning

One important aspect of using Spot Instances is that they can be interrupted at any time if AWS needs the capacity back. When this happens, AWS provides a two-minute warning before the instance is terminated. For video processing workflows, this means you need to plan for interruptions so that work isn’t lost midway.

Break Videos into Smaller Chunks

A common strategy is to break long videos into smaller chunks before processing. For example, instead of transcoding a two-hour video in one run, you can split it into 10-minute segments, process each segment separately, and then stitch them back together later. This way, if an interruption occurs, only the current segment is affected rather than the entire job. FFmpeg supports segmenting input files using options like -ss (start time) and -t (duration), which makes this approach straightforward.

Store Partial Results in S3

Another best practice is to store partial results in Amazon S3 as soon as they are completed. For instance, after processing each video chunk, upload the output back to your S3 bucket immediately. This ensures that even if the Spot Instance is terminated, your progress is safe, and you can resume processing from the last completed chunk instead of starting over. Using S3 as a central storage layer makes your workflow fault-tolerant and resilient against interruptions.

Reliable and Cost-Effective Workflows

By designing your FFmpeg workflows with these strategies, you can take full advantage of the cost savings of Spot Instances without risking data loss. This makes Spot instances a reliable option even for long-running or large-scale video processing jobs, provided you build in restart and recovery mechanisms.

Tips & Next Steps

Once you’ve set up FFmpeg on AWS and experimented with Spot Instances, there are a few ways to refine your workflow and explore more advanced options. These next steps will help you get the most out of your cloud-based video processing setup.

1. Try Different Instance Types

AWS offers many EC2 instance families, each optimized for different workloads. For general transcoding and compression tasks, compute-optimized instances like the c5 family are often a good fit, providing strong CPU performance at a reasonable cost. If you’re working with high-resolution videos or need hardware acceleration, GPU-based instances such as the g4dn family can dramatically speed up encoding with FFmpeg’s GPU-enabled libraries. Experimenting with instance types lets you balance performance and cost for your specific use case.

2. Orchestrate Jobs with SQS or Step Functions

As your workflows grow, running jobs manually can become inefficient. AWS provides orchestration tools to help automate FFmpeg jobs. Amazon SQS (Simple Queue Service) can be used to queue video processing tasks, allowing multiple Spot Instances to work on jobs in parallel. For a full Docker‑based pipeline with REST APIs and worker queues, see our production‑ready batch video processing server guide. For more complex workflows, AWS Step Functions provide a way to chain together multiple tasks such as fetching input files, running FFmpeg, handling errors, and saving outputs into a reliable, serverless state machine. These services ensure your video pipelines are scalable and fault-tolerant without requiring you to build custom orchestration logic.

3. Consider CE.SDK as an Alternative

If you’re building custom editing workflows on AWS, consider integrating a frontend editor like our CreativeEditor SDK. While FFmpeg and AWS handle the heavy lifting of transcoding and storage, CE.SDK provides a powerful in-browser editing interface for trimming, adding overlays, applying filters, or creating design templates. This combination allows you to deliver both scalable backend video processing and interactive, user-facing editing tools, similar to what platforms like Canva provide.

Conclusion

Running FFmpeg on AWS Spot Instances gives you a powerful and cost-effective way to handle large-scale video transcoding. By leveraging Spot pricing, you can cut costs by up to 90% compared to On-Demand instances, making it possible to process even long, high-resolution videos without breaking the budget. With strategies like chunking videos and storing partial outputs in S3, interruptions become manageable, allowing you to balance affordability with reliability.

For smaller, lightweight, or event-driven workflows, AWS Lambda is a perfect complement. Its serverless nature makes it easy to trigger FFmpeg jobs automatically, for example, generating thumbnails or clipping short clips as soon as a file is uploaded to S3. Although Lambda has strict runtime and storage limits, it shines when simplicity and automation are priorities.

As a next step, you can begin experimenting with automation and orchestration using services like SQS, Step Functions, or even AWS Batch to build resilient pipelines that combine the strengths of both Spot Instances and Lambda. For teams looking for a fully managed alternative, CE.SDK is also worth exploring.

By now, you’ve seen how to install FFmpeg, run it on spot instance, and handle serverless workflows with Lambda. With these tools and strategies, you’re ready to design flexible, scalable, and affordable video processing pipelines in the AWS cloud.

Join 3,000+ creative professionals who get early access to new features and updates—subscribe.

FFmpeg - The Ultimate Guide

Csaba — Mon, 21 Nov 2022 12:16:19 GMT

In this guide, we’ll go through the hot topics of FFmpeg. But before that, we’ll cover some base ground to help you understand basic media concepts and FFmpeg. Feel free to skip the parts that are already trivial for you!

Introduction to FFmpeg

FFmpeg.org’s definition is the following: “FFmpeg is the leading multimedia framework, able to decode, encode, transcode, mux, demux, stream, filter and play pretty much anything that humans and machines have created. It supports the most obscure ancient formats up to the cutting edge. No matter if they were designed by some standards committee, the community or a corporation.”

I think of FFmpeg as the go-to application for audio/video manipulation in an automated or scripted manner.

When you need to implement a service that manipulates video, or just have 300 media files that need to be converted into a different format, FFmpeg is your - nerdy - friend.

FFmpeg can do large chunks of the basic functionalities of a modern Non-linear (NLE) video editors, e.g., Davinci Resolve Studio or Premiere Pro. But, it does not have a graphical interface in that sense as those behemoths do, and unarguably it is way less friendly.

In a general NLE, you might do things like these:

Click to import a file
Drop it into the timeline
Trim and Cut
Add an overlay image
Crop that overlay
Add vignette
Add some color changing effects, e.g. change the hue
Add an extra audio track to the mix
Change the volume
Add some effects, e.g.: echo
Export into various formats
Export into a deployable video format
Export the master audio in wav

Learn how to crop and trim videos in Flutter. Or, to achieve the exact same thing, you could also execute this command:

ffmpeg -y  \
    -ss 20 -t 60 -i bbb_sunflower_1080p_60fps_normal.mp4 \
    -i train.jpg \
    -ss 4 -i voice_recording.wav \
    -filter_complex "[0:v]hue=h=80:s=1[main] ; [1:v]crop=w=382:h=304:x=289:y=227[train] ; [main][train]overlay=x=200:y=200,vignette=PI/4[video] ; [2:a]volume=1.5,aecho=0.8:0.9:100:0.3[speech] ; [0:a][speech]amix=duration=shortest,asplit[audio1][audio2]" \
    -map '[video]' -map '[audio1]' -metadata title="Editor's cut" bbb_edited.mp4 \
    -map '[audio2]' bbb_edited_audio_only.wav

Yes, it isn’t friendly at all, but it is very, very powerful once you become friends with FFmpeg.

Check out this comparison of the original and the edited one:

If you want to try this command out, get the example files and see it for yourself!

Installing FFmpeg

FFmpeg is available for most common and even uncommon platforms and architectures. You can be on Linux, Mac OS X or Microsoft Windows, and you’ll be able to run or link to FFmpeg.

Installing FFmpeg is easy on most platforms! There is no installer, usually just a compressed archive you need to get for your platform and architecture.

In the case of Linux, most distributions include a pre-built FFmpeg in their software repositories. Therefore, you can install FFmpeg from those even more quickly.

FFmpeg history

The project was started in 2000 by the awesome Fabrice Bellard. The name is a concatenation of “FF” meaning “fast-forward” and MPEG, the name of a video standards group. It has been very well, active and alive since then, releasing a new release about every three months.

FFmpeg supported codecs and formats

The default FFmpeg shipped with my Ubuntu Linux distribution supports about 460 codecs and 370 formats.

See it for yourself:

ffmpeg -codecs
ffmpeg -formats

Compilation of FFmpeg

Keep in mind that the supported codecs and formats (and filters, demuxers, muxers, input and output methods, etc.) are highly dependent on the so-called compilation flags.

This means that the above number only represents the fact that it supports at least this many codecs and formats. Still, there are even more that the package builders excluded for various reasons, e.g.: licensing, architecture, size considerations, etc.

Since FFmpeg is open source, you can compile FFmpeg for yourself at any time.

Suppose for example, that you care about your layer’s size (therefore the bootstrap speed) in AWS Lambda. In this case, you can compile an FFmpeg binary that only contains the mp3 encoder for example, and nothing else. For a full tutorial on running FFmpeg on AWS Spot Instances, see our cloud guide. Prefer Google Cloud? Our guide on running FFmpeg on Google Cloud Platform shows you how.

Also, you might not want to run into licensing issues and leave out stuff that would cause problems for your use case. Therefore you choose to leave out particular codecs/formats. I highly recommend checking out the “—enable-gpl”, “—enable-nonfree” and “—enable-version3” compilation flags in this case, as well as this.

Or you might want to have a standalone FFmpeg binary in your project (e.g.: embedded, or some cloud instance), that does not depend on any operating system libraries. Then you want to make a so-called static build, that compiles in all the libraries into a single binary file, and does not depend on your OS’ libraries and the runtime loading of other FFmpeg libraries. Search around for “—enable-static” in this case.

Finally, you can find pre-built static FFmpeg builds right here too. Alternatively, you can package FFmpeg in a Docker container for consistent environments - our Docker guide covers this approach.

FFmpeg’s strengths

FFmpeg reads and writes most video and audio formats that matter for most of us. It is a very capable and high-performance tool for converting and manipulating these formats.

But FFmpeg can do even more! For examples of these operations integrated into an automated pipeline, read our article on a batch video processing server.

Filtering

FFmpeg has vast amounts of filters for audio and video. Therefore, video manipulation is also a key feature of FFmpeg.

Hardware acceleration

It does support many kinds of hardware accelerations! Video encoding is a very resource-intensive operation, and you might come across quite a few hardware devices or features that might speed up your process!

Most notably, if you have an NVIDIA card, you can increase your H.264 or H.265 encoding and decoding throughput by multipliers compared to your CPU. But other things, such as VDPAU, VAAPI, or OpenCL, can be leveraged to boost your pipeline’s throughput.

Learn more about the supported hardware acceleration methods here.

Versatile input/output methods

FFmpeg is also very capable when it comes to accessing input and output data.

Just to name a few: it can use your webcam, record from your microphone, grab your screen, or capture from your Blackmagic DeckLink. But FFmpeg can download directly from a web address, open all kinds of streams, read from a pipe, a socket, and of course, from files.

The same holds true for outputting the data. It can write to your webcam, play audio on your microphone… Just kidding:) It can output to files, streams, pipes, sockets and so on.

Running example commands

This article is full of FFmpeg commands that are working examples. The reason for that is that you could test these out for yourself! But the command line interfaces of different operating systems are slightly different, so the commands in this article are meant to be executed in a Linux bash shell.

To adopt these command lines to Microsoft Windows, you might need to:

Change (cd) into the directory where you extracted the ffmpeg.exe. Alternatively, add that directory to the path to make it callable from anywhere.
You might need to replace “ffmpeg” to “ffmpeg.exe”
You will need to replace ”\“-s (backslashes) at the end of the lines with ”^“-s (hats)
You’ll need to replace the fontfile argument’s value to something like this: fontfile=/Windows/Fonts/arial.ttf to get commands with the drawtext filter working.

MacOS users will need steps #1 and #4.

Introduction to media concepts

Now let’s have a quick overview of media concepts. These concepts will be vital for us if we want to understand the latter sections of this article and FFmpeg’s workings. To keep this section brief, it is a higher-level, simplified explanation of these concepts.

Audio

We’ll briefly cover the following terms:

Sampling rate
Bitrate
Channels

Sampling Rate

The sampling rate is the factor that shows how many times we measure/scan/sample the input data stream.

The image below shows the measurement windows (quantization) as gray bars.

Why does this matter? Because it is a balancing act. If we measure the signal less often, we’ll lose more details (bad). Also, by having fewer samples, we’ll have less data in the end. Therefore the file size will be smaller (good).

Here are some ballpark values:

8 kHz (GSM - Low quality)
44.1 kHz (CD - High quality)
48 kHz (Very high quality)
88.2 kHz (Insane - usually for production only)
96 kHz (Insane - usually for production only)

There are no definite “right answers” here. The question is what is “good enough” for your use case? GSM focuses on speech, and not even quality but understandability and the least possible amount of data. Therefore, they found that 8 kHz is enough (there are quite a few more tricks), for their purposes.

The “CD quality” aimed for high quality. Therefore they chose 44.1 kHz, that number has some history in it, but the main reason for aiming above 40 kHz lies in physics and how the human ear works.

There were two very smart guys whose theorem basically says that if you want a quite good signal representation, you have to sample it at twice the speed as its original frequency. Human hearing generally works up until about 20 kHz, so if you want “good quality”, you should aim for at least 40 kHz. And 40 kHz + some headroom + some more physics + historical reasons = 44.1 kHz! :)

As for the higher rates, those are only used when very high-quality audio editing is needed.

Bitrate

Bitrate represents the amount of data per second that results from our transcoding/quantization process. If it is 1411 kbit/s, that means that for every second of audio data, about 1411 kbit of output data will be produced.

Therefore, you can say that 1 minute of audio with 1411 kbit/sec will require:

(1411 kbit / 8) kbyte * 60 second = 10582 kbyte = 10.33 mbyte

Now, it is only easy like that with raw audio data and with a few simple codecs, e.g. PCM in WAVs.

Codecs compressing hard might throw your numbers around a little, as input data might be compressible with different rates. Variable bitrate is usually happening to save space. The encoder might output a lower bitrate if the data is “simple” and does not require high precision.

Here are some ballpark values:

13 kbits/s (GSM quality)
320 kbit/s (High-quality MP3)
1411 kbit/s (16bit WAV, CD quality, PCM)

Channels

Inside of most audio formats, you can have more audio channels. This means multiple, separated audio streams can be in the same file.

Many times, multiple channels have their own name:

If you have a single microphone, you will most probably record it into a single channel called Mono.
General music from the FM radio or streaming services usually has two channels in a so-called “Stereo” configuration.

With stereo, there could be several methods how the audio “image” can be made richer by leveraging audio panning, time and phase-shifting and much more. There is a special recording technique too, called Binaural recording, which is super awesome. Wear headphones for this, and don’t be scared:)

For example, here are Big Buck Bunny’s audio waveforms in Audacity:

You can see that there are two lines of waveforms and also that they are pretty similar. That is normal, as you usually hear the same thing with your two ears, but the matter is in the subtle differences between the two. That’s where directionality, richness, and all kinds of other effects lie.

But why stop at two? The list continues:

2.1, as it is often called, means three channels: 2 for stereo and one for the LFE (“low-frequency effects” a.k.a.: “bass”).
5.1 is similar, with five directional channels (2 front, 1 center, 2 rear) and the LFE.

So channels are just separate “recordings” or “streams” of audio signals.

Image properties

For images, there are quite a few parameters, but we’ll check out only these:

Resolution
Bit-depth
Transparency

Resolution

An image consists of pixels, single points that have a single color. The resolution of an image determines how many columns and rows of pixels are in an image. In other words: an image has a width and a height.

This image shows the first 10 pixels in the first row.

Here are some ballpark values for resolution:

“HD” or “Full HD” or “1K” or “1080p” means 1920x1080 pixels.
“4K” could mean a few values, but it should be about 3840x2160 pixels.
A regular 16mp photo you make of your cat is about 4608x3456 pixels.
General social media image posts are about 1080x1080 pixels.

Bit-depth

Bit-depth represents the number of bits used for storing a single pixel’s color value. This is the same balancing game, and you need to decide between quality or file size.

General ballpark values for bit-depth:

Bits	Colors	Notes
1	2	Black & White
8	256	B/W or Limited color palette
24	16.7m	3x8 bit for R-G-B “True color”
30	1073m	3x10 bit for R-G-B “Deep color”

These last two sometimes are referred to as “8 bit” or “10 bit” respectively, especially when talking about videos. That means 8/10 bits per single color channel.

Transparency

Some image formats support an additional channel together with the red, green, and blue components: the alpha channel. The alpha channel determines how transparent a single pixel is, and it can have different bit-depths, it is usually either 1, 8 or 16 bits.

If the alpha channel is 1 bit, then the format can encode a pixel to be either transparent or non-transparent. If it is 8 or more bits, then the format can encode 256 or more steps of transparency.

Video properties

Video data is built by single images shown right after each other. This brings in most attributes of images and a few more!

So a video has a resolution that is its width and height.

Then the first obvious parameter of a video is the framerate, which defines how many images are shown in a second. Common values for this are 24, 25, 30, or 60.

A video file also has a codec assigned to it, which is the format describing how all those images were compressed into this video file. There are many more attributes of videos, but this is a good start.

Video codecs

Compression is a super important thing when it comes to video because you have thousands of images to keep together. If you aren’t doing it in a smart way, then the resulting video will be very, very large.

Just imagine a 2-minute video, with 30 fps. That means it will have 60 s * 2 * 30 fps = 3600 frames! I have just taken a screenshot of an HD video, which was 730 kbyte in JPEG format. Now 3600 frame * 730 kbyte equals 2.5 gigabytes!

Can you imagine that? I hope not, and that’s because compression brings that way, way down, to the level of tens of megabytes. These days a video of that size is quite high quality and about 2 hours long. Also, don’t forget, that JPEG is already compressed, a single frame would be 6 mbyte when uncompressed. Now that 2-minute video would be 21 gigabytes if we’d store it uncompressed.

Standard codecs such as H.264 and H.265 are doing very clever and complex operations to achieve high compression ratios with good quality.

Just think about that, most frames in a video are quite similar, only containing small differences. So if we could only store that little difference between frames, we’d won a huge bonus! And that’s just one of the many tricks codecs do.

Codec designers are also exploiting the weaknesses and features of the human eye. Such as the fact that we are more sensitive to light intensity changes than color changes (say hello to YUV). And they can get away with lower quality details for parts that are moving fast, and so on.

Because why lose precious bits for things that you can’t even notice?!

There are many codecs out there, with different goals in mind, although the majority focus on keeping the file size low.

H.264, H.265: These are the most common ones, with the widest support in browsers, phones, players, etc. It focuses on small file sizes with good quality. (At the cost of resource intensiveness.)
Apple ProRes, DNxHD: These are common formats for production. They focus on quality and ease of processing and not on file size.

Audio codecs

The goal of audio codecs is the same as what we saw with the video codecs. It is just harder to demonstrate it as audio does not consist of single image frames but audio frames/packets. So an analog audio signal is of an almost infinite, or at least very high quality if you think of it.

At the lowest level, the speed and amplitude resolution is very high. We could say “atomic”, as we need to measure and store the speed and direction of atoms. So if you want to store that exactly, that will require a super high-quality measurement, which will also result in a very high bitrate data stream.

Thankfully, the sound is at least not propagating with light speed so we can save quite a lot just by that fact. (There’s no need for an extreme sampling rate.) Then our hearing is very limited if we take the previous paragraph as a scale, so we win there again. We don’t need most of that high precision that is there.

But still, if we take our hearing capability and want to store raw audio data with about 44.1 kHz of sample rate with about 1 Mbit/sec bitrate, we’d still get quite a lot of data. Check the calculations in the audio bitrate section above.

So raw audio can be compressed further, which is what many popular codecs do. They also exploit the human senses, but this time the human ear. We started with the basics that the human ear has a limit on the frequencies it can detect. Therefore, we can save a lot by cutting out the range of frequencies outside our hearing range. Unless you are a bat, you are fine between 20-20khz! :)

But there are other tricks, for example, auditory masking. That means that the presence of one frequency can affect your capability to detect a different frequency. From the codec’s viewpoint, it can skip encoding a few frequencies if it is smart enough to know which ones you’ll not notice. I’m sure there are a lot more tricks, let me know if you know about a few more interesting ones!

Here is a list of common codecs:

MP3, AAC, OGG: These are common lossy audio formats.
PCM (e.g. in a WAV container), FLAC: These are lossless formats.
MIDI: It is a funny format. It is like a music sheet that might sound different on different players or settings. It is usually not made from real audio data, but from recording a digital keyboard or as an output from an audio composing software.

Containers

Now we got through the fundamental building blocks, the image, the video, the video codecs, and the audio codecs, and we reached the top of this iceberg: the containers.

A container is a format specification, that combines all these streams into a single file format. It defines how to put all these data together, how to attach metadata (e.g. author, description, etc), how to synchronize these streams, and sometimes a container even contains indexes to aid seeking.

So, for example, a MOV container can contain an H.264 video stream and an AAC audio stream together.

Common containers:

MOV
MP4
MKV
WebM
WAV (audio only)

Example Material

I will use these example materials as inputs in the following parts of this article. If you’d like to follow along, save these files for yourself!

Name	Resource
Big Buck Bunny	http://distribution.bbb3d.renderfarming.net/video/mp4/bbb_sunflower_1080p_60fps_normal.mp4
Train	train.jpg
Smiley	smiley.png
Voice recording	voice_recording.wav
Big Buck Bunny’s audio	ffmpeg -i bbb_sunflower_1080p_60fps_normal.mp4 -map 0:1 bbb_audio.wav

And we will make our own audio file by extracting the audio from the Big Buck Bunny movie! We’ll use this file as an example, so after downloading the video file, please execute this:

ffmpeg -i bbb_sunflower_1080p_60fps_normal.mp4 -map 0:1 bbb_audio.wav

By the middle of this article, you’ll understand this command, but for now, just make sure to have the WAV file next to your video file to test out the commands later in the article.

We’ll use these files in the following parts of this article. Therefore make sure to get them!

FFplay and FFprobe

FFmpeg is the name of the main binary and the project itself, but it is shipped together with two other binaries, ffplay and ffprobe.

Let’s check them out quickly, right in the command line!

FFplay

FFplay is a basic video player, that can be used for playing media. It’s not a friendly video player, but it is a good testing ground for various things.

To execute it, just simply supply a media file:

ffplay bbb_sunflower_1080p_60fps_normal.mp4

If you want to test this exact command, you’ll need to get the example files.

For example, it can be used to preview filters (we’ll discuss those later), but let’s see an example:

ffplay -vf "drawtext=text='HELLO THERE':y=h-text_h-10:x=(w/2-text_w/2):fontsize=200:f

FFprobe

FFprobe, as its name implies, is a tool for getting information about media files.

This command:

ffprobe bbb_sunflower_1080p_60fps_normal.mp4

Will return us some general information about the video file:

Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'bbb_sunflower_1080p_60fps_normal.mp4':
  Metadata:
[...]
    title           : Big Buck Bunny, Sunflower version
    artist          : Blender Foundation 2008, Janus Bager Kristensen 2013
[...]
  Stream #0:0[0x1](und): Video: h264 [...]
[...]
  Stream #0:1[0x2](und): Audio: mp3 [...]
[...]
  Stream #0:2[0x3](und): Audio: ac3 [...]

I have abbreviated it heavily, as we’ll check this out later.

But FFprobe is way more powerful than just this!

With the following command, we can get the same listing in JSON format, which is machine-readable!

ffprobe -v error -hide_banner -print_format json -show_streams bbb_sunflower_1080p_60fps_normal.mp4

The explanation of this command is the following:

“-v error -hide_banner”: This part hides extra output, such as headers and the default build information.
“-print_format json”: Obviously, this causes ffprobe to output a JSON.
“-show_streams” is the main switch that requests the stream information.

{
  "streams": [
    {
      "index": 0,
      "codec_name": "h264",
      "codec_long_name": "H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10",
      "width": 1920,
      "height": 1080,
      "bit_rate": "4001453",
      "duration": "634.533333",
      "############################": "[~50 lines removed]"
    },
    {
      "index": 1,
      "codec_name": "mp3",
      "channels": 2,
      "bit_rate": "160000",
      "############################": "[~40 lines removed]"
    },
    {
      "index": 2,
      "codec_name": "ac3",
      "channels": 6,
      "############################": "[~20 lines removed]"
    }
  ]
}

In this output, you can see three streams of data in this video file. The first (index: 0) is a video stream, that is an HD video with an H.264 codec. Then we have two audio streams, the first (index: 1) is a simple mp3 stream with stereo audio, and the second (index: 2) is an ac3 stream with 6 channels, most likely in an 5.1 configuration.

I have removed quite a lot of output for brevity, but you can get way more information out of these streams, e.g. fps for the video stream and so on.

Other than -show_streams, there are 3 more: -show_format, -show_packets and -show_frames. Unless you are really deep in the rabbit hole, you’ll not need the last two, but -show_format could be useful:

ffprobe -v error -hide_banner -print_format json -show_format bbb_sunflower_1080p_60fps_normal.mp4

{
  "format": {
    "filename": "bbb_sunflower_1080p_60fps_normal.mp4",
    "nb_streams": 3,
    "nb_programs": 0,
    "format_name": "mov,mp4,m4a,3gp,3g2,mj2",
    "format_long_name": "QuickTime / MOV",
    "start_time": "0.000000",
    "duration": "634.533333",
    "size": "355856562",
    "bit_rate": "4486529",
    "probe_score": 100,
    "tags": {
      "major_brand": "isom",
      "minor_version": "1",
      "compatible_brands": "isomavc1",
      "creation_time": "2013-12-16T17:59:32.000000Z",
      "title": "Big Buck Bunny, Sunflower version",
      "artist": "Blender Foundation 2008, Janus Bager Kristensen 2013",
      "comment": "Creative Commons Attribution 3.0 - http://bbb3d.renderfarming.net",
      "genre": "Animation",
      "composer": "Sacha Goedegebure"
    }
  }
}

This is an overview of “what is this file”. As we see, it is a MOV file (format_name), with three streams (nb_streams), and it is 634 seconds long. Also, there are some tags where we can see the title, the artist, and other information.

FFmpeg concepts

Here is a quick intro to how FFmpeg actually works!

For those who are just joining in: please get the example assets if you want to test out the commands shown in this chapter!

FFmpeg opens the file, decodes it into memory, then encodes the in-memory packets back and puts them into some container: some output file. The term “codec” is a mix of the words “coder & encoder”. Those are the magic parts before and after the “decoded frames”.

The decoded frames are uncompressed images in-memory, e.g. the most basic pixel format for video frames is called “rgb24”. This just stores red, green, and blue values right after each other in 3x8 bits, or 3x1 byte, which could hold 16m colors.

The importance of this is that other than a few exceptions, you can only manipulate or encode the decoded frames. So when we get to different audio/video filters or transcoding, you’ll need the decoded frames for all that. But don’t worry, FFmpeg does this automatically for you.

Inputs

So you see and probably guessed, that FFmpeg must access the input data somehow. FFmpeg knows how to handle most media files, as the awesome people who develop FFmpeg and the related libraries made encoders and decoders for most formats available!

Don’t think that it is a trivial thing. Many formats are reverse engineered, a hard task requiring brilliant people.

So although we often refer to input files, the input could come from many sources, such as the network, a hardware device and so on. We’ll learn more about that later on in this article.

Many media files are containers for different streams, meaning that a single file might contain multiple streams of content.

For example, a .mov file might contain one or more streams:

video tracks
audio tracks (e.g. for the different languages or audio formats such as stereo or 5.1)
subtitle tracks
thumbnails
…

All these are streams of data from the viewpoint of FFmpeg. Input files and their streams are numerically differentiated with a 0-based index. So, for example, 1:0 means the first(0) stream of the second(1) input file. We’ll learn more about that later too!

Important to note that FFmpeg can open any number of input files simultaneously, and the filtering and mapping will decide what it will do with those. Again more on that later!

Streams

As we have seen in the previous section, streams are the fundamental building blocks of containers. So every input file must have at least one stream. And that’s what you can list by the simple ffmpeg -i command for example.

A stream might contain an audio format such as MP3, or a video format such as an H.264 stream.

Also, a stream, depending on the codec, might contain multiple “things”. For example, an mp3 or a WAV stream might include various audio channels.

So the building block hierarchy, in this case is: File → Stream → Channels.

Outputs

Of course, an output could be a local file, but it doesn’t need to be. It could be a socket, a stream and so on. In the same way as with inputs, you could have multiple outputs, and the mapping determines what goes into which output file.

The output also must have some format or container. Most of the time FFmpeg can and will guess that for us, mostly from the extension, but we can specify it too.

Mapping

Mapping refers to the act of connecting input file streams with output file streams. So if you give 3 input files and 4 output files to FFmpeg, you must also define what should go to where.

If you give a single input and a single output, then FFmpeg will guess it for you without specifying any mapping, but make sure you know how exactly that happens, to avoid surprises. More on all that later!

Filtering

Filtering stands for the feature of FFmpeg to modify the decoded frames (audio or video). Other applications might call them effects, but i’m sure there is a reason why FFmpeg calls them filters.

There are two kinds of filtering supported by FFmpeg, simple and complex. In this article we’ll only discuss the complex filters, as it is a superset of the simple filters, and this way, we avoid confusion and redundant content.

Simple filters are a single chain of filters between a single input and output. Complex filters can have more chains of filters, with any number of inputs and outputs.

The following figure extends the previous overview image with the filtering module:

A complex filter graph is built from filter chains, which are built from filters.

So a single filter does a single thing, for example, changes the volume. This filter is quite trivial, it has a single input, changes the volume, and it has a single output.

For video, we could check out the scale filter, which is also quite straightforward: it has a single input, scales the incoming frames, and it has a single output too.

You can chain these filters, meaning that you connect the output of one to the input of the next one! So you can have a volume filter after an echo filter, for example, and this way, you’ll add echo, and then you change the volume.

This way, your chain will have a single input, and it will do several things with it and will output something at the end.

Now, the “complex” comes in when you have multiple chains of these filters!

But before we go there, you should also know that some single filters might have multiple inputs or outputs!

For example:

The overlay filter puts 2 video streams above each other and will output a single video stream.
The split filter splits a single video stream into 2+ video streams (by copying).

So let’s discuss a complex example from a bird’s eye view! I have two video files, I want to put them above each other, and I want the output in two files/sizes, 720p and 1080p.

Now, that’s where complex filtering will be faithful to its name: to achieve this, you’ll need several filter chains!

Chain 1: [input1.mp4] [input2.mp4] → overlay → split → [overlaid1] [overlaid2]
Chain 2: [overlaid1] → scale → [720p_output]
Chain 3: [overlaid2] → scale → [1080p_output]

As you see, you can connect chains, and you can connect chains to output files. There is a rule that you can only consume a chain once, and that’s why we used split instead of the same input for chains 2 and 3.

The takeaway is this: with complex filter graphs (and mapping), you can:

build individual chains of filters
connect input files to filter chains
connect filter chains to filter chains
connect filter chains to output files

FFmpeg’s command line system

For those who are just joining in: please get the example assets if you want to test out the commands shown in this chapter!

FFmpeg CLI

Finally, we arrived at FFmpeg, and trust me, we’ll execute it quite a lot of times! Let’s see how FFmpeg’s command line options are organized, as that is the first tricky part we need to understand!

FFmpeg mostly thinks about input and output files and their options together with global options. You specify input files with the “-i” flag followed by a file name. For the output file, specify it as-is without any preceding CLI (command line interface) flag.

Specifying an input file

Let’s specify just an input file:

ffmpeg -i bbb_sunflower_1080p_60fps_normal.mp4

The following image helps to understand the output:

First, you get the “banner”, where you see the build information and lib versions. If you watch closely, you’ll see the compilation flags, starting with —, e.g. —enable-shared.
Then you get the same output as we have seen with ffprobe earlier.
And then you get a complaint that there is no output file(s) specified. That’s fine for now.

You can remove the banner here with “-hide_banner”, but for brevity’s sake I’ll not include that anymore in the commands here, and I will leave it out from the outputs too.

Now, let’s get brave, and specify an output file!

Specifying an output

As I’ve said earlier, the output file is understood by FFmpeg as it is just a filename. But more specifically, it is after the input(s) specifications, and it is not a value of any other switches.

Don’t be confused for now, but yes, FFmpeg can have as many inputs and outputs as you’d like. We’ll cover that in more detail soon!

This command line specifies a single output file:

ffmpeg -i bbb_sunflower_1080p_60fps_normal.mp4 audio_only.wav

Before taking a look at the output, let me congratulate you! You have just converted a video file into an audio file, by keeping just the audio content!

This is how you transcode! Of course, you’ll want to specify more parameters later on.

So, here is the output:

Let’s analyze it!

(1) First, we have our input metadata printing, which we saw many times already.

(2) Then we have something called “stream mapping”. We forced FFmpeg into a decision situation, as we specified an input file with 1 video and 2 audio streams. We said we wanted an audio output (guessed from the .wav extension). But we didn’t specify which audio stream we wanted, so let’s see what FFmpeg decided:

“Stream #0:2” means “The first input file’s third stream” or “input file index 0’s stream with index 2.” This is the input.
”-> #0:0” means the first output file’s first stream. This is the output.
Here you can learn more about how FFmpeg decide this.
Later on, we’ll manually override the mapping.
Summary: FFmpeg decided to convert the third stream in the input file (the ac3 5.1 audio) into the first stream of the output file.

(3) Then we have our output metadata information. This reveals what FFmpeg will output. It usually copies most of the metadata, and here you also see the container/format information too.

(4) And then we see the output summary. For example, the transcoding was 181x faster than the playback speed. Nice!

Understanding the command line order

Before going further, let’s understand FFmpeg’s command line arguments from a bird’s eye view!

In the manual, you’ll see this:

ffmpeg [global_options] {[input_file_options] -i input_url} ... {[output_file_options] output_url} ...

(Parts in […] are meant to be optional, and parts in {…} are meant to be specified 1 or more times.)

This is the general outline of how to specify inputs, outputs, input options, output options, and global options. The order matters, but it is easy to remember: global options, inputs and outputs. Also, i/o options come BEFORE the i/o specification.

Let’s put these into pseudo command line options, to understand it better:

# One inputs, one output, nothing fancy
ffmpeg -i input1.mp4 output1.wav

# Two inputs, one output
ffmpeg -i input1.mp4 -i input2.mp4 output1.wav

# Two inputs, two outputs
ffmpeg -i input1.mp4 -i input2.mp4 output1.wav output2.mp3

# One input, one output, with options
ffmpeg [input1 options] -i input1.mp4 [output2 options] output1.wav

# Two inputs, two outputs with options
ffmpeg [input1 options] -i input1.mp4 \
       [input2 options] -i input2.mp4 \
       [output1 options] output1.wav \
       [output2 options] output2.mp3

As for the global options, these are the ones you might care about:

-hide_banner: To skip printing the banner.
-y: To overwrite the output even if it exists.

For example, you can run this as many times as you want:

ffmpeg -y -hide_banner -i bbb_sunflower_1080p_60fps_normal.mp4 audio_only.wav

And it will overwrite the output and be less verbose than earlier.

Without explaining the options themselves, let’s just see some real-world examples with options:

And here it is with two inputs and two outputs:

Mapping files

We saw above that this command:

ffmpeg -i bbb_sunflower_1080p_60fps_normal.mp4 audio_only.wav

… will result in an audio file that contains one of the audio streams from the input video chosen by FFmpeg. This automatic stream selection is usually handy when it is trivial. For example, when you have one stream as input and one output file, you don’t need to specify any mapping manually.

But in cases where it is not so trivial, you are usually better off manually specifying what you really want to do.

The following image summarises what our current situation is:

The video stream was not matched, as the output format was an audio file (.wav). But then FFmpeg chose Stream #2, because it has more channels.

So what if we’d like to get the stereo track instead? That is where mapping comes in! The mapping is a parameter of the OUTPUT file. Therefore the mapping arguments should come right before our output file definition!

ffmpeg -i bbb_sunflower_1080p_60fps_normal.mp4 -map 0:1 stereo_audio_only.wav

The argument -map 0:1 means, that in the output (since we specify it as an output option) we’d like to have Input #0’s (the first input file) Stream #1!

Let’s see the relevant parts from the output!

Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'bbb_sunflower_1080p_60fps_normal.mp4':

[...]

Stream mapping:
  Stream #0:1 -> #0:0 (mp3 (mp3float) -> pcm_s16le (native))

[...]

Output #0, wav, to 'stereo_audio_only.wav':
  Metadata:
[...]
    Stream #0:0(und): [...] stereo [...]

The “Stream #0:1 -> #0:0” part means that we have successfully overridden the mapping, to get the mp3 stream (0:1) into our output! Also, the output metadata reveals that we’ll get a stereo result instead of the 5.1 earlier.

Multiple outputs

You can have multiple outputs from a single input, let’s see when that might be useful!

Let’s say, we want to extract BOTH audio streams into two separate WAV files! It is super easy:

ffmpeg -y -i bbb_sunflower_1080p_60fps_normal.mp4 -map 0:1 stereo_audio_only.wav -map 0:2 ac3_audio_only.wav

See? I have just specified two output files with two mapping specifications! Also, I have sneaked in the “-y” to have it overwrite our previous file!

Let’s check out the relevant parts of the output!

Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'bbb_sunflower_1080p_60fps_normal.mp4':

[...]

Stream mapping:
  Stream #0:1 -> #0:0 (mp3 (mp3float) -> pcm_s16le (native))
  Stream #0:2 -> #1:0 (ac3 (native) -> pcm_s16le (native))

[...]

Output #0, wav, to 'stereo_audio_only.wav':
    Stream #0:0(und): [...] stereo

[...]

Output #1, wav, to 'ac3_audio_only.wav':
    Stream #1:0(und): Audio: [...] 5.1(side)

Now the mapping reveals two lines, as we have two outputs! And indeed, you’ll get two .wav files as the output, one is stereo, and one is 5.1!

There might be several other reasons why you’d want to get multiple outputs. Let’s briefly check out a few!

Different formats:

ffmpeg -y -i bbb_sunflower_1080p_60fps_normal.mp4 stereo_audio_only.wav  stereo_audio_only.mp3

Wow, did you catch that? We just created a WAV and an mp3 in a single command line! I’ve reverted to the automatic stream selection for brevity’s sake.

A bit closer to real-life needs, you might want different output qualities:

ffmpeg -y  -i bbb_sunflower_1080p_60fps_normal.mp4  \
-map 0:1 -b:a 320k stereo_audio_only_high_quality.mp3 \
-map 0:1 -b:a 64k  stereo_audio_only_low_quality.mp3

Here -b:a 320k means “bitrate of audio should be around 320 kbit/sec”. So I have requested FFmpeg to make two mp3s for me, from the stereo stream of the input.

Checking on the files, this is what we got:

 25Mb stereo_audio_only_high_quality.mp3
4,9Mb stereo_audio_only_low_quality.mp3

One more common reason for having multiple outputs or using mapping is when we introduce filters into our pipeline, but that will be discussed later!

Now you understand the foundations of how to communicate your basic requirements to FFmpeg via its command line! Great job! Now we can dive even deepert.

Hands-on with FFmpeg

In this section, we will discover and even try out some common features of FFmpeg!

For those who are just joining in: please get the example assets if you want to test out the commands shown in this chapter!

Inputs

Let’s see the common ways FFmpeg is fed with different data!

File

Of course, you have already seen that if you have a local file on your filesystem, FFmpeg is happy to read it!

ffmpeg -i bbb_sunflower_1080p_60fps_normal.mp4 -map 0:1 stereo_audio_only.wav

This command which is exactly the same as one of our previous ones just reads a local file. Really, that’s it.

Network

Did you know, that FFmpeg can open a file directly on the network?!

ffmpeg -t 5 -i http://distribution.bbb3d.renderfarming.net/video/mp4/bbb_sunflower_1080p_60fps_normal.mp4 bbb_first_5_seconds.mp4

The command above opens the file directly from the network and saves the first 5 seconds into a local file!

I wanted to spare bandwidth for these awesome guys over renderfarming.net, so I added the duration flag: -t 5. FFmpeg doesn’t even download the full video for this operation. Isn’t that wonderful?!

Webcam

FFmpeg can also open your webcam!

This is an example command for Linux:

ffmpeg -f v4l2 -framerate 25 -video_size 640x480 -t 10 -i /dev/video0 10seconds_of_webcam.webm

This would record 10 seconds of your webcam!

Accessing the webcam happens differently on different platforms. Also specifying parameters is different for each platform, so for this reason, if you’d like to access your webcam with FFmpeg, please refer to the documentation:

Microphone

Let’s record some audio directly from your microphone!

List microphones:

arecord -l

Start 10 seconds of recording:

ffmpeg -f alsa -i hw:0,0 -t 10 out.wav

This command was meant to work on Linux, but you can check out how to do that on Microsoft Windows or macOS.

Pipe

Finally, FFmpeg can read from a pipe, and also output to a pipe.

On Linux, you could do something like this:

cat bbb_sunflower_1080p_60fps_normal.mp4 | ffmpeg -i - -f wav pipe:1 | pv > output.wav

# Alternative, without pv:
cat bbb_sunflower_1080p_60fps_normal.mp4 | ffmpeg -i - -f wav pipe:1 > output.wav

This command would use the cat program to simply read in the video file and output it to its standard output. Then this output is piped INTO FFmpeg, through its standard input. The combination “-i -” means “read from standard input”. By the way, standard input would be your keyboard otherwise, if we wouldn’t use any redirection here.

Then we specify the required output format for FFmpeg, with “-f wav”. This is needed because now we’ll have no output file name, and FFmpeg will not be able to guess the format. Then we specify “pipe:1” as an output, meaning we’d like FFmpeg to output to its standard output.

From then, we pipe the data into a program called “pv”, it is just a metering tool, that dumps information on the throughput (from its stdin to its stdout). Finally, we redirect pv’s output into a WAV file.

You might ask why we’d want to do that, why we talk about this. Piping can be useful if you build a complex pipeline from different programs or if you want to spare reading and writing to a local file.

For example, the node package fluent-ffmpeg can leverage this functionality by supplying input and output streams. For example, you can read from an S3 bucket and write to one directly.

But be warned, hell is awaiting you on that road. No kidding. You need to research the limitations of this technique. For example, many formats can not be streamed in this manner, as they need random access to the output data to write the indices at the beginning of the file after processing.

Outputs

FFmpeg can output into many protocols, from local file storage and ftp to message queue protocols all the way to streaming protocols.

For more information, check out the documentation here.

Transcoding audio with FFmpeg

In this chapter, we’ll be going to see how to transcode into audio with FFmpeg!

The general formula is:

ffmpeg -i {input audio or video file with audio} [output options] output_audio.ext

Choosing a format

FFmpeg is quite smart, and by the extension, it can determine which codec to use. If you specify “audio.wav” or “audio.mp3” for example, FFmpeg will use the appropriate codec to do the encoding.

It is perfectly guessing most of the time. But if you want to specify the format manually, then the “-f” flag is your friend.

For this, you might want to consult the list of formats:

ffmpeg -formats

So, these three commands will do exactly the same, but the last two requires the -f flag.

# Output codec is determined from the extension
ffmpeg -i bbb_audio.wav bbb_audio.mp3

# No extension in the filename
ffmpeg -i bbb_audio.wav -f mp3 bbb_audio

# Piped output therefore no filename, so no extension to use for guessing
ffmpeg -i bbb_audio.wav -f mp3 pipe:1 > bbb_audio

Setting the bitrate

In most cases. you want to specify the target bitrate you expect from your codec to output. If you are unsure what bitrate is, please read this article’s audio bitrate section.

To specify the audio bitrate, use the “-b:a” option with a corresponding value, e.g.:

-b:a 320k: For the mp3 codec this is considered high quality.
-b:a 128k: Lower quality.
-b:a 64k: Low quality.

For example:

ffmpeg -i bbb_audio.wav -b:a 320k bbb_audio_320k.mp3

Setting the sample rate

You may want to specify the sample rate to ensure quality or low output file size. Half the sample rate could mean half the output file size. If you are unsure what the sample rate is, please read the “audio sample rate” section of this article.

To specify the audio sample rate, use the “-ar” option with a corresponding value, e.g.:

-ar 48000: For high quality.
-ar 44100: For CD quality (still high).
-ar 22500: A bit of a compromise, not recommended for music, but for speech, it might be enough.
-ar 8000: Low quality, e.g. if you only want “understandable” speech.

For example:

ffmpeg -i bbb_audio.wav -ar 44100 bbb_audio_44100khz.mp3

Setting the channel count

Setting the channel count can be useful, for example, if you have a stereo recording of a single person’s speech. In that case, you might be content with just a mono output half the size of the original recording.

If you are unsure what an audio channel is, please read the “audio channels” section of this article.

To specify the channel count use the “-ac” option with a corresponding value, e.g.:

-ac 1: For mono
-ac 2: For stereo
-ac 6: For 5.1

For example:

ffmpeg -i bbb_audio.wav -ac 1 bbb_audio_mono.mp3

Complete command line for converting audio with FFmpeg

This is how you produce a high-quality output:

# Convert wav to mp3
ffmpeg -i bbb_audio.wav -ac 2 -ar 44100 -b:a 320k bbb_audio_hqfull.mp3

# Convert wav to m4a (aac)
ffmpeg -i bbb_audio.wav -ac 2 -ar 44100 -b:a 320k bbb_audio_hqfull.m4a

# Convert wav to ogg (vorbis)
ffmpeg -i bbb_audio.wav -ac 2 -ar 44100 -b:a 320k bbb_audio_hqfull.ogg

Check out this documentation about good quality audio transcoding too!.

Lossless formats

If you want to convert audio into a lossless format, here are a few choices for you:

# Convert to flac (Free Lossless Audio Codec)
ffmpeg -i bbb_audio.wav -compression_level 12 bbb_audio_lossless_12.flac # Best compression, slowest
ffmpeg -i bbb_audio.wav -compression_level 5 bbb_audio_lossless_5.flac   # Default
ffmpeg -i bbb_audio.wav -compression_level 0 bbb_audio_lossless_0.flac   # Least compression, fastest


# Convert to wav
cp bbb_audio.wav bbb_audio_lossless.wav # Just kidding:)

# Convert to wav
ffmpeg -i any_audio.ext bbb_audio_lossless.wav

It’s good if you know that flac results in a smaller file than WAV, as WAV doesn’t actually compress by default:

117M bbb_audio.wav
52M  bbb_audio_lossless_0.flac
45M  bbb_audio_lossless_5.flac
43M  bbb_audio_lossless_12.flac

WAV is generally thought of as a lossless format, but keep in mind that the WAV container can contain lossy content too, but by default FFmpeg uses the pcm_s16le format, which is the 16 bit PCM, that could be understood as lossless.

Learn more here and here.

Transcoding video with FFmpeg

In this chapter, we’ll be going to see how to transcode a video file into the two most common formats!

Converting to H.264

H264 is one of the most popular video codecs. Most devices, browsers and video players understand how to play it. It is efficient in storing video content, but as with most advanced video codecs, it is a resource intensive-process to encode and decode.

A complete command line for a high-quality H.264 transcoding with high-quality AAC audio is the following:

ffmpeg -y -i bbb_sunflower_1080p_60fps_normal.mp4 \
-c:v libx264 -preset slow -crf 22 \
-profile:v main -g 250 -pix_fmt yuv420p \
-map 0:0 -map 0:1 \
-acodec aac -ar 44100 -b:a 320k bbb_transcoded_h264_HQ.mov

Make sure to understand this command and to customize it to match your needs.

To help you do that, let’s dissect this command!

Global options:

-y: Overwrite the output.

Input options:

-i bbb_sunflower_1080p_60fps_normal.mp4: The input file.

Output options:

-c:v libx264: Set the codec to libx264.

-preset slow: libx264 has a lot of variables that you can be tune, and most of them balance the coding speed and the resulting file size. To make your life easier, there are presets by which you can easily declare what you need: small size or speed.

-crf 22: This is the constant rate factor, the main option for setting image quality. It is a number between 0-51, where 0 is lossless, and 51 is the worst quality. Generally, you want something between 17 and 28. This is the option to tune the balance between image quality and file size. Check my comparison video here.

-profile:v main -g 250 -pix_fmt yuv420p: These are advanced options, guaranteeing you a quite backward compatible result. (See this, this, and this.)

-map 0:0 -map 0:1: You might not need this: these options are selecting the correct video and audio streams. In our case, we have two audio streams, and we need the stereo one to avoid some issues with our aac stream.

-acodec aac: Select the AAC (Advanced Audio Coding) codec for the audio in the output. We need to be more specific than just -f for the format. We need to specify the audio codec here manually.

-ar 44100: Set the audio sampling rate (learn more about that in previous chapters of this article).

-b:a 320k: Set the audio bitrate (learn more about that in previous chapters of this article).

30seconds_of_bb.mkv: The output file name. All the options since the last -i (or the last output file) considered to be a modifier for this output.

Let’s see the output:

Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'bbb_sunflower_1080p_60fps_normal.mp4':

[...]

Stream mapping:
  Stream #0:0 -> #0:0 (h264 (native) -> h264 (libx264))
  Stream #0:1 -> #0:1 (mp3 (mp3float) -> aac (native))

[...]

Output #0, mov, to 'bbb_transcoded_h264_HQ.mov':
    Stream #0:0(und): Video: h264 (libx264) (avc1 / 0x31637661), yuv420p(progressive), 1920x1080 [SAR 1:1 DAR 16:9], q=-1--1, 60 fps, 15360 tbn, 60 tbc (default)
    Stream #0:1(und): Audio: aac (LC) (mp4a / 0x6134706D), 44100 Hz, 5.1(side), fltp, 320 kb/s (default)

[...]

frame=38074 fps= 35 q=-1.0 Lsize=  324855kB time=00:10:34.51 bitrate=4194.1kbits/s dup=2 drop=0 speed=0.58x

From this, we understand that FFmpeg chose the mp3 stream from the input file because we told it to do so. (Remember, it has two audio streams in it, a stereo mp3 and a 5.1 ac3.) We also see that my machine could transcode with 35fps (0.58 times the playback speed), and our settings resulted in an average video bitrate of 4200 kbit/s.

The video bitrate is an interesting question in this mode. With the CRF option, we specify the “constant visual quality” we want. To reach a constant visual quality, the encoder works hard to guess how much it can compress certain parts of every frame, and the result of that guess defines the final average video bitrate.

If you want even better results with H.264, and you can afford a bit more processing time and a bit more complicated process, check out the 2-pass encoding instead of the constant rate factor method introduced above.

To learn more about these two different rate control methods, read the awesome Understanding Rate Control Modes article. And to learn more about the intricacies of H.264 encoding, check out the H264 encoding guide.

Finally, later on, I will show you a comparison video that shows how different CRF values perform!

Converting to H.265

H.265 is the successor of H.264, according to the official FFmpeg manual, it offers 25-50% bitrate savings while retaining the same visual quality.

A complete command line for a high-quality H.265 transcoding with high-quality AAC audio is the following:

ffmpeg -y -i bbb_sunflower_1080p_60fps_normal.mp4 \
-c:v libx265 -preset slow -crf 27 \
-profile:v main -g 250 -pix_fmt yuv420p \
-map 0:0 -map 0:1 \
-acodec aac -ar 44100 -b:a 320k bbb_transcoded_h265_HQ.mov

And the result is:

...
encoded 38074 frames in 3384.84s (11.25 fps), 1720.32 kb/s, Avg QP:35.29

H.265 also has multiple rate control algorithms, I used the CRF method here. If you want to use a different rate control algorithm, then you may check out the H.265 encoding guide. Also, check out the next section, where I’ll reveal how different CRF values perform!

This command is almost the same as what we used in the H.264 example above, so please refer to that section to understand the arguments.

If we compare H.264 and H.265 with our commands above, taking into account this 10-minute long video on my system, these are the results:

H.264 is 3 times faster (35 fps vs 11 fps)
H.264 produces a 2 times larger file (318 mb vs 156 mb)

Comparing CRF values with H.264 and H.265

I have created a video for your convenience, that shows the different crf values in action. The selected frame had some movement on it with the leaves in the bunny’s hand. Movement is important with video codecs, as usually that’s where quality losses are first visible.

This video shows how the different CRF values perform, from 0-51 with the H.264 and H.265 formats!

H.264 & H.265 CRF comparison video

(Can you guess which program I was using to make this?:))

Basic editing with FFmpeg

In this section, we’ll achieve basic editing tasks by using FFmpeg only!

We’ll just get a basic mp4 with default settings in these examples to keep things simple. But to encode the result in a proper, high quality way, please check the earlier sections where we learned how to encode into H.264 and H.265!

Trimming from the beginning of the clip

It is possible to specify an in-point for a media file. By doing that, you essentially cut off the specified amount from the beginning of the input file. Therefore, FFmpeg will skip the first part of the file and only transcode the remainder!

For this, you need the “-ss” flag! The value can be specified in seconds (5 or 5.2) or as a timestamp (HOURS:MM:SS.MILLISECONDS).

To get the outro only, we could seek all the way to the end of the video! (It is 00:10:34.53 or 635 seconds long!)

# Get
# 635 - 4 = 631
ffmpeg -y -ss 631 -i bbb_sunflower_1080p_60fps_normal.mp4 last_4_seconds.mp4


# 00:10:34.53 - 4 = 00:10:30.53
ffmpeg -y -ss 00:10:30.53 -i bbb_sunflower_1080p_60fps_normal.mp4 last_4_seconds.mp4

Seeking can be a bit tricky, so you may want to learn more about seeking here.

Trimming from the end of the clip

You can also set an out-point for an input file, therefore shortening it. There are two options for this:

-t: This sets the duration.
-to: This sets the timestamp where the input video should stop.

These two are mutually exclusive, and also they do the same if no -ss is specified. The value can be specified in seconds (5 or 5.2) or as a timestamp (HOURS:MM:SS.MILLISECONDS).

Let’s experiment with them!

# "Get 30 seconds of the input."
ffmpeg -y -t 30 -i bbb_sunflower_1080p_60fps_normal.mp4 first_30_seconds.mp4
ffmpeg -y -t 00:00:30.0 -i bbb_sunflower_1080p_60fps_normal.mp4 first_30_seconds.mp4

# "Get everything until the content's 30th second."
ffmpeg -y -to 30 -i bbb_sunflower_1080p_60fps_normal.mp4 first_30_seconds.mp4
ffmpeg -y -to 00:00:30.0 -i bbb_sunflower_1080p_60fps_normal.mp4 first_30_seconds.mp4

All four above commands result in exactly the same video. (For nerds: even the md5sum is the same.)

But let’s see how they perform when we introduce seeking!

# "Seek to the 10th second and get me 30 seconds of the input."
ffmpeg -y -ss 10 -t 30 -i bbb_sunflower_1080p_60fps_normal.mp4 part_between_10_and_40.mp4

# "Seek to the 10th second and get the content until the 30th second."
ffmpeg -y -ss 10 -to 30 -i bbb_sunflower_1080p_60fps_normal.mp4 part_between_10_and_30.mp4

The first command will result in a 30 second long video, while the second command will be 20 seconds long only!

The figure below shows the difference:

Editing without reencoding

FFmpeg can do something I’m not aware of in any other popular NLE: it can edit videos without reencoding them!

The usual workflow is to decode the data frames (a/v) into memory, modify them as much as we like and then encode them into a new video file. The problem with this is that unless you work with raw or lossless codecs, you’ll lose some quality in the process. Another issue with this approach is that it is computationally intensive.

For certain operations, you can configure FFmpeg, to keep the data frames intact, and this way, you can avoid decoding and encoding them! This is incredibly faster than regular transcoding, usually hundreds of times faster.

The “certain operations” are those that don’t need to modify the data frames themselves. For example, you can cut and trim this way. Also, you can manipulate streams while keeping others, like you can replace the audio track without touching the video frames.

All this is a bit of magic, and there are caveats you need to prepare for, but it is good if you know about this, as it is often handy!

The trick lies in two options:

-c:v copy: The “copy” video codec
-c:a copy: The “copy” audio codec

Let’s see a few examples!

Remove audio while keeping the video without reencoding

ffmpeg -i bbb_sunflower_1080p_60fps_normal.mp4 -c:v copy -an copied_video_only.mp4

Here, we used the “-an” option, which removes all audio streams. I remembered it as “audio no”, but that is just my mnemonic:)

Let’s see how fast it was:

frame=38072 fps=20950 q=-1.0 Lsize=  310340kB time=00:10:34.51 bitrate=4006.7kbits/s speed= 349x

So It processed the whole 10 minutes of video in 2 seconds, 349x faster than playback, with 20950 fps!

Remove video while keeping the audio without reencoding

ffmpeg -i bbb_sunflower_1080p_60fps_normal.mp4 -c:a copy -vn copied_audio_only.wav

Here, we used the “-vn” option, which removes all video streams. I remembered it as “video no”.

Let’s see how fast it was:

size=   24772kB time=00:10:34.14 bitrate= 320.0kbits/s speed= 776x

776x faster than playback, finished in about a second, not bad!

Cut and trim without reencoding

ffmpeg -ss 10 -t 10  -i bbb_sunflower_1080p_60fps_normal.mp4 -c:a copy -c:v copy part_from_10_to_20_copied.mp4

There could be precision issues with seeking while you do this, so you may want to learn more about seeking and copying here.

Replace audio on video file without reencoding

We have removed audio and video already, but what if we want to swap them?

ffmpeg -y \
-i bbb_sunflower_1080p_60fps_normal.mp4 \
-i voice_recording.wav \
-map "0:v" -map "1:a" \
-c:v copy -c:a copy \
bbb_with_replaced_audio.mov

There is quite a lot going on in here, so let’s explain the parts!

First, we have two inputs (-i), meaning we are better off manually specifying the mapping. The command would work without the “-map” options, but it would ignore our second input.

-map "0:v" -map "1:a" means that please use the first file’s (first) video stream and the second file’s (first) audio stream.

With -c:v copy -c:a copy, we require FFmpeg to copy the already encoded data packets without touching them. Therefore FFmpeg’s work is mostly really just copying bytes, no decoding, no encoding.

Not surprisingly, that’s what we see in the stream mapping too:

Stream mapping:
  Stream #0:0 -> #0:0 (copy)
  Stream #1:0 -> #0:1 (copy)
Press [q] to stop, [?] for help
frame=38072 fps=9750 q=-1.0 Lsize=  320645kB time=00:10:34.51 bitrate=4139.7kbits/s speed= 162x

And since it is just copying, it was crazy fast, 162x of the playback speed, or almost 10k frames per second!

But!

Execute the exact same command, but with “bbb_with_replaced_audio.mp4” (.mp4 container instead of .mov) as an output file! You’ll get this:

Could not find tag for codec pcm_s16le in stream #1, codec not currently supported in container

The message is quite clear. You can not have a pcm_s16le (raw WAV, say that 10 times:)) stream in an MP4 container. I’m not sure if it is FFmpeg’s or the container’s lack of support, but we need to solve this. If you run into this situation, you might consider two solutions:

Change the container: I’ve just tried MOV, and it worked.
Encode the audio: We still copy the video data, and encoding audio isn’t that painful.

I just showed you option #1, so let’s see option #2:

ffmpeg -y \
-i bbb_sunflower_1080p_60fps_normal.mp4 \
-i voice_recording.wav \
-map "0:v" -map "1:a" \
-c:v copy \
-c:a aac -b:a 320k -ar 44100 \
bbb_with_replaced_audio_aac.mp4

This copies the video frames and encodes our WAV into a supported codec to be held in the mp4 container. You can refer back to the audio encoding section if you want to learn more about that.

Here is the output:

Stream mapping:
  Stream #0:0 -> #0:0 (copy)
  Stream #1:0 -> #0:1 (pcm_s16le (native) -> aac (native))
Press [q] to stop, [?] for help
...
frame=38072 fps=2176 q=-1.0 Lsize=  313058kB time=00:10:34.51 bitrate=4041.8kbits/s speed=36.3x

“Only” 36x faster than playback, 2176 fps, still not that bad!

Filtering overview

FFmpeg supports many audio and video filters. Currently, there are 116 audio and 286 video filters, but there are a bit more if we count the hardware accelerated ones too.

So how do we leverage them?

There are two ways to define filters, but I’m going to explain the complex filter, as the difference is not much, but it is more versatile. So there is a global option for FFmpeg, called: -filter_complex. With quite a weird syntax, you can specify all your filters and their parameters right after this option.

You can imagine the process with the following image:

Basically, your filter graph can access all the inputs (-i a.mp4 -i b.mp4 -i c.mp4), and it can produce as many outputs as you like (-map might be needed).

Basic syntax

Let’s take a look at a simple, basic example:

ffmpeg -y  -t 5 \
-i bbb_sunflower_1080p_60fps_normal.mp4 \
-filter_complex "drawtext=text='HELLO THERE':y=20:x=30:fontsize=200:fontfile=/usr/share/fonts/truetype/freefont/FreeSerif.ttf" \
filter_complex1.mp4

Although -filter_complex is a global option, I like to put it after the inputs and before the outputs as it is a bit easier to overlook the whole command that way. Thankfully the command line parser of FFmpeg is smart enough, and it works.

The command above produces a 5-second-long video, where the text “HELLO THERE” is overlaid on the intro of Big Buck Bunny.

Let’s understand the weird format for specifying filters!

We’ll go bottom-up, and we’ll build it from there. So the most basic format is this:

FILTER_NAME=ARGUMENT1=VALUE1:ARGUMENT2=VALUE2

For example:

drawtext=text='HELLO THERE':y=20:x=30

The first thing before the first equal (=) sign is the filter’s name, which is the drawtext filter in this case. Then we have our first argument, “text” and its value “‘HELLO THERE’”. Right after that, separated with a colon (:) comes the next argument, “y” with a value of “20”.

You can guess what each of the text, y, x, fontsize and fontfile arguments do, as it is quite self-explaining. But especially for the first time, you’ll heavily rely on the filtering documentation to understand every filter and every argument.

Also, several characters are reserved, such as: , : = and a few others depending on your environment, so sooner or later you need to learn about escaping too.

To recap, our pipeline looks like this now:

Multiple filters in a chain

This previous command is a single filter chain that consists of a single filter only, but you could have more filters put right after each other! It means that the output of one filter will be the input for the next! The way to do this is by separating them with a comma!

Let’s draw two boxes with the drawbox filter!

ffmpeg -y  -t 5 \
-i bbb_sunflower_1080p_60fps_normal.mp4 \
-filter_complex "  drawbox=x=10:y=10:w=100:h=100:color=red  ,  drawbox=x=200:y=200:w=100:h=100:color=blue  " \
filter_complex2.mp4

See? The output of the first filter is passed to the output of the second filter!

Let’s visualize our pipeline again:

Input and output pads

Now, we have skipped something this far, because for simple uses FFmpeg is smart enough to do it for us. And this is the specification of a chain’s input and output pads!

Let’s draw just a single rectangle for now:

ffmpeg -y  -t 5 -i bbb_sunflower_1080p_60fps_normal.mp4 -filter_complex "drawbox=x=10:y=10:w=100:h=100:color=red" filter_complex3.mp4

FFmpeg sees that the input for our filter chain is a single video file, and the output is a single output video file. Therefore, it safely assumes that we want that single input as the input of our single filter chain. And that single output should be the single output of our single output chain.

That’s really nice, as, in simple situations like this, we don’t need to assign and map inputs and outputs manually! But when we get more inputs, filter chains, or outputs, it is no longer possible. Therefore, we need to understand how to assign inputs and outputs!

First of all, let’s compare the following two command lines. They result in exactly the same result, but the second one represents what FFmpeg does internally (roughly):

ffmpeg -y  -t 5 -i bbb_sunflower_1080p_60fps_normal.mp4 -filter_complex "drawbox=x=10:y=10:w=100:h=100:color=red" filter_complex3.mp4

ffmpeg -y  -t 5 -i bbb_sunflower_1080p_60fps_normal.mp4 -filter_complex "[0:v]drawbox=x=10:y=10:w=100:h=100:color=red[out_link_0]" -map "[out_link_0]" filter_complex3.mp4

Do you see the difference? Before our filter chain, an “input pad” is defined: [0:v]. The expected format between the square brackets is documented in the stream selection section of the official documentation, and this article already covered it.

But, a quick summary:

0:v: This means the first video stream of the first input file.
0:v:0: Means exactly the same thing but in a long form.
0:0: Means the first stream of the first input file (not recommended, as it could be anything in theory. It could be a subtitle stream, a thumbnail, a video or an audio stream…)
0:a: This means the first audio stream of the first input file.
0:a:0: Means exactly the same thing but in a long form.
0:a:1: Means the second (index #1) audio stream of the first input file.

So we can specify which input file should be connected to which input of the filter graph!

Also, something similar is going on at the end! Do you see, the [out_link_0] output pad definition at the end of our filter chain?

The naming here is easier, as basically you can specify any arbitrary name in here. It roughly means, “please store the output data under this name”.

And when you specify your output file, you can or need to map it by selecting one of your filter graph outputs! Therefore, we must add the -map “[out_link_0]” option before our output file.

This map option means this: “Please save the data stream with this name into the following output file.”

This is how you can visualize this input/output mapping:

Multiple chains

Coming from the previous sections, you are now ready to see and understand an even more complicated configuration, which has multiple input files, output files, and filter chains!

ffmpeg -y  \
-i train.jpg \
-t 5 -i bbb_sunflower_1080p_60fps_normal.mp4 \
-filter_complex "[0:v]drawbox=x=10:y=10:w=100:h=100:color=red[train_box] ; [1:v]drawbox=x=10:y=10:w=100:h=100:color=red[bbb_box]" \
-map "[train_box]" filter_complex4_train.jpg \
-map "[bbb_box]" filter_complex4_bbb.mp4

Let’s see the output (two files next to each other):

We had two inputs, and we got two output files, an image, and a video, with a red rectangle on them, with a single command!

Are you still here? I hope! Let’s understand what happened in that crazy command! We have two input files:

-i train.jpg: A simple image file
-t 5 -i bbb_sunflower_1080p_60fps_normal.mp4: Our video file, but to make it quick, just the first five seconds of it

Then the first thing to note is that we have two filter chains! They are separated with a ”;”.

Our first filter graph is this: [0:v]...[train_box]

This requests the first input file as an input
Draws a red box
Saves the output into the “train_box” output pad

Our second filter graph is this: [1:v]...[bbb_box]

This requests the second input file as an input
Draws a red box
Saves the output into the “bbb_box” output pad

And finally, we got two outputs, each mapping to one of the outputs of the filter graph:

-map “[train_box]” filter_complex4_train.jpg
-map “[bbb_box]” filter_complex4_bbb.mp4

Here is the same thing visually:

If you are thinking about making it even more complex and making filter graphs that combine multiple inputs into one for example, you are on the right track! It is possible, and we will get to that!

This was the introduction to the filtering system and its syntax.

Editing video

Now let’s get to know a few filters and make some interesting stuff!

Resizing or scaling

The scale filter is a simple one, yet it is quite powerful!

ffmpeg -y  \
-t 5 -i bbb_sunflower_1080p_60fps_normal.mp4 \
-filter_complex "scale=width=600:height=-1:force_original_aspect_ratio=decrease" \
filter_complex5_scaled1.mp4

The arguments speak for themselves, but a few things:

Specifying -1 to either width or height means rescaling while keeping the aspect ratio.
“force_original_aspect_ratio” can be increase, decrease. Meaning it will increase or decrease the image to fit the specified bounding box while keeping the aspect ratio.

Adding text

We have already covered this a little, so let’s dive deeper!

This is what we used earlier:

ffmpeg -y  \
-t 5 -i bbb_sunflower_1080p_60fps_normal.mp4 \
-filter_complex "drawtext=text='HELLO THERE':y=20:x=30:fontsize=200:fontfile=/usr/share/fonts/truetype/freefont/FreeSerif.ttf" \
filter_complex1.mp4

Now let’s discover how to align the text!

Many filters, including drawtext, support variables in some of its argument’s values. If you scroll down in the documentation of drawtext, you’ll find this:

“The parameters for x and y are expressions containing the following constants and functions: ”

And after this part, you’ll see many variables which you can include in your x and y variables!

Let’s see:

# Align the text to the center
ffmpeg -y  \
-t 5 -i bbb_sunflower_1080p_60fps_normal.mp4 \
-filter_complex "drawtext=text='HELLO THERE':y=h/2-text_h/2:x=w/2-text_w/2:fontsize=200:fontfile=/usr/share/fonts/truetype/freefont/FreeSerif.ttf" \
filter_complex6_center.mp4
# y=h/2-text_h/2 means: y position = (image height / 2) - (text height / 2)

# Align the text to the right:
ffmpeg -y  \
-t 5 -i bbb_sunflower_1080p_60fps_normal.mp4 \
-filter_complex "drawtext=text='HELLO THERE':y=30:x=w-text_w-20:fontsize=200:fontfile=/usr/share/fonts/truetype/freefont/FreeSerif.ttf" \
filter_complex6_right.mp4
# x=w-text_w-20 means: x position = image width - text width - 20pixel padding


# Align the text to the bottom:
ffmpeg -y  \
-t 5 -i bbb_sunflower_1080p_60fps_normal.mp4 \
-filter_complex "drawtext=text='HELLO THERE':y=h-text_h-20:x=30:fontsize=200:fontfile=/usr/share/fonts/truetype/freefont/FreeSerif.ttf" \
filter_complex6_bottom.mp4
# y=h-text_h-20 means: y position = image height - text height - 20pixel padding

And this is what we’ll get in the end:

I need to mention one good trick that might not be obvious at first. So the text_h variable is a tricky one, because different text will be of different height! E.g.: ”____” and “WWW” will result in a different height.

For this reason, you do not always want to use text_h or even just a constant y=value expression but rather, you need to align text by its baseline. So just remember to use the “ascent” variable whenever you need to align text vertically!

Check out these two examples! Each has two drawtext filters printing ”_” and “_H”:

# This one uses y=200 for both, still the text isn't aligned properly!
ffmpeg -y  \
-t 5 -i bbb_sunflower_1080p_60fps_normal.mp4 \
-filter_complex "drawtext=text='_':y=200:x=30:fontsize=200:fontfile=/usr/share/fonts/truetype/freefont/FreeSerif.ttf,drawtext=text='_H':y=200:x=500:fontsize=200:fontfile=/usr/share/fonts/truetype/freefont/FreeSerif.ttf" \
filter_complex7_bad_text.mp4

# This one uses y=200-ascent for both and the text is aligned as expected!
ffmpeg -y  \
-t 5 -i bbb_sunflower_1080p_60fps_normal.mp4 \
-filter_complex "drawtext=text='_':y=200-ascent:x=30:fontsize=200:fontfile=/usr/share/fonts/truetype/freefont/FreeSerif.ttf,drawtext=text='_H':y=200-ascent:x=500:fontsize=200:fontfile=/usr/share/fonts/truetype/freefont/FreeSerif.ttf" \
filter_complex7_good_text.mp4

Now let’s compare the difference:

See? This is the difference between aligning the “top left” or the “baseline” of the text!

Adding an overlay

Overlaying is a very interesting thing to do with FFmpeg. Let’s jump right in!

Basic

ffmpeg -y  \
-t 5 -i bbb_sunflower_1080p_60fps_normal.mp4  \
-i smiley.png \
-filter_complex "overlay" \
filter_complex8_overlay1.mp4

Easy as that!

Of course, the overlay filter has a ton of options, but I wanted to demonstrate the easiest possible command line. We don’t even need to mess with input/output pads, as FFmpeg automatically understands the situation: two inputs for the overlay filter and its single output into a single output.

But just to exercise, we could have executed it like this:

ffmpeg -y  \
-t 5 -i bbb_sunflower_1080p_60fps_normal.mp4  \
-i smiley.png \
-filter_complex "[0:v][1:v]overlay[output]" \
-map "[output]" filter_complex8_overlay2.mp4

And this would result in the same output! Check it out, now I have specified the two inputs for the overlay: [0:v][1:v]!

Aligned

Let’s align the smiley into the center!

As we have seen with the drawtext, the overlay filter’s arguments also support a few dynamic variables. We’ll use those to achieve what we want!

ffmpeg -y  \
-t 5 -i bbb_sunflower_1080p_60fps_normal.mp4  \
-i smiley.png \
-filter_complex "overlay=x=main_w/2-overlay_w/2:y=main_h/2-overlay_h/2" \
filter_complex8_overlay3.mp4

Preprocessing the input for overlay

Let’s get a bit creative!

I want to make it smaller, and I also want to blur it!

Now pause for a minute, and think about it, how you’d do that?!

…

Ready?

ffmpeg -y  \
-t 5 -i bbb_sunflower_1080p_60fps_normal.mp4  \
-i smiley.png \
-filter_complex "[1:v]scale=w=200:h=-1,gblur=sigma=3[smiley] ; [0:v][smiley]overlay=x=100:y=100" \
filter_complex8_overlay4.mp4

For this we needed to have two filter graphs!

The first one is this: [1:v]scale=w=200:h=-1,gblur=sigma=3[smiley]

Scales the input image (the smiley).
Then the scaled output is also blurred.
Then the output is saved into the output pad named “smiley”.

Then, we have our second filter graph: [0:v][smiley]overlay=x=100:y=100

This takes as input the first input file (the video).
This also takes as input the output pad named “smiley”. (We are connecting two chains this time!)
Then the overlay filter does its overlaying thing, and we trust FFmpeg to pair the unnamed output with the single output file we specified.

Reusing content

Let’s do one more, a really complicated one!

Let’s have the outro overlaid over the intro!

ffmpeg -y \
-t 5 -i bbb_sunflower_1080p_60fps_normal.mp4 \
-t 5 -ss 00:09:40 -i bbb_sunflower_1080p_60fps_normal.mp4  \
-filter_complex " [1:v]scale=w=1920/2:h=-1[outro]; [0:v][outro]overlay" \
filter_complex8_overlay5.mp4

We could have achieved it in several ways, e.g. we could use the trim filter, but to keep it easy, we just open the same file twice and seek/trim them.

-t 5 -i bbb_sunflower_1080p_60fps_normal.mp4: Open the video, and keep the first five seconds of it.
-t 5 -ss 00:09:40 -i bbb_sunflower_1080p_60fps_normal.mp4: Open the same video again, but seek to the end and keep five seconds from there.

Then we have two filter graphs again, one scales down the outro, and the second is just an overlay.

Are you excited?:) I hope these made-up examples opened up your eye for the possibilities, and I hope you’ll create very creative stuff with this knowledge!

Chroma keying, green screen, blue screen

In this section, we’ll use chroma keying to remove the background from Big Buck Bunny’s intro, and then we will put the transparent logo over the original video, as if it would be some kind of a logo overlay!

ffmpeg -y \
-ss 0.5 -t 2 -i bbb_sunflower_1080p_60fps_normal.mp4 \
-ss 10 -i bbb_sunflower_1080p_60fps_normal.mp4  \
-filter_complex " [0:v]chromakey=color=0xfdfdfd:similarity=0.1:blend=0.2 , scale=w=-1:h=300 , loop=loop=-1:start=0:size=120[intro] ; [1:v][intro]overlay=x=-40:y=-40" \
-t 10 filter_complex9.mp4

So just to recap, Big Buck Bunny’s first few seconds are like this:

And this is the result:

Also, the butterfly moves its wings repeatedly!

Let’s examine the command!

-ss 0.5 -t 2 -i bbb_sunflower_1080p_60fps_normal.mp4: We read in the intro from 0.5 to 2.5 seconds.
-ss 10 -i bbb_sunflower_1080p_60fps_normal.mp4: We read in the video, starting from the 10th second.

Then we have two filter graphs, the first being this:

[0:v]chromakey=color=0xfdfdfd:similarity=0.1:blend=0.2 , scale=w=-1:h=300 , loop=loop=-1:start=0:size=120[intro]

As we see, we have three filters in here!

chromakey: This one takes a color and a few parameters as input, and outputs transparent frames. The specified color + the blended areas will be the transparent sections. In our case we replaced the white-ish (#fdfdfd) background color with transparency.
scale: We resize the full 1080p image into something around 300px high.
loop: With the loop filter, we repeat all the 2 seconds worth of 120 frames (60*2) over and over again, to have the butterfly move its wings continuously.

And then, finally we have the second filter graph:

[1:v][intro]overlay=x=-40:y=-40

Nothing fancy, just an overlay of the original video and our chrome keyed intro.

What else?

You might want to check out a few more filters, that I didn’t cover here.

Here are just a few interesting ones:

Audio manipulation

In this chapter, we’ll be going to check out some audio manipulation techniques with FFmpeg!

First of all, let’s see our example file:

It is a voice recording, and it is intentionally… well, quite bad.

From the waveform, it is obvious that there are very different volume ranges in it. This is an example recording where each sentence was read in different strengths: “normal”, “whisper” or “powerful”, this is why you see repeating patterns of amplitude ranges on the image.

It isn’t visible, but it has some noise too, and of course, it is not normalized or enhanced in any way. Yet.

Please note that there are different scenarios, requirements, and ways to enhance audio. This is a simplified method to show the outline of the process in this article. I’m not an audio engineer, although I have some experience in the area. So if you know it better, feel free to fine-tune it for yourself even more, or contact me and recommend improvements!

I’m showing an example here with a very rough input, one that you’d just reject in real life as it would be useless due to its quality. But it is an excellent example to show the different steps of the enhancing process and to see what can be done to it!

The following steps are built upon each other, and we’ll reach the complete command at the end!

Don’t forget that these settings are specific to this voice recording. Sadly this can not be generalized too much.

Gate

Let’s start with the gate filter!

A gate is like a switch that opens only if the signal is stronger than the threshold. So if the signal level is lower than the threshold, it cuts to complete silence. Although you might soften or delay this cut with the knee, attack, and release arguments.

We’ll use this filter as a basic noise reduction method now! This helps us remove the noise between words and sentences by cutting it to silence. It doesn’t remove noise in any other way, e.g. it doesn’t touch the static on the voice itself.

Check this out!

ffmpeg -y \
-i voice_recording.wav \
-filter_complex "agate=threshold=0.01:attack=80:release=840:makeup=1:ratio=3:knee=8" \
gate.wav

Let’s hear it: gate.wav

And let’s see it:

As you can see, the “silent” parts were attenuated heavily, while the above-the-threshold parts remained similar. Those parts were still affected by the knee, attack, and release arguments determining how hard (knee) and quick (attack/release) the cut is.

I’ve left a quite high release timeout here to avoid sudden dips in the amplitude.

This is where we are right now:

The silent parts are more silent than before, but still, the amplitude range or the dynamic range is quite high. You must change your volume levels to hear everything and void blowing your speakers/brain out.

Equalization

Before fixing that, let’s do a bit more housekeeping. Let’s do some equalization and frequency filtering!

We’ll use these filters:

ffmpeg -y \
-i gate.wav  \
-filter_complex "highpass=f=100:width_type=q:width=0.5 , lowpass=f=10000 , anequalizer=c0 f=250 w=100 g=2 t=1|c0 f=700 w=500 g=-5 t=1|c0 f=2000 w=1000 g=2 t=1" \
gate_eq.wav

Let’s hear it: gate_eq.wav

This command gradually attenuates frequencies below 100hz, as there are not much valuable content in there, but it can really lower the clarity of the speech.

Then we do the same, but for frequencies above 10 kHz. This is mostly needed because we have a lot of high-frequency noise, so this is a workaround for those. Also, a male voice is generally deeper than a woman’s, so you might want to pay attention to how low you can put the bar.

Then comes anequalizer, which has a crazy an exceptional way of setting its arguments:

This: anequalizer=c0 f=250 w=100 g=2 t=1|c0 f=700 w=500 g=-5 t=1|c0 f=2000 w=1000 g=2 t=1 means:

at 250hz with a width of 100hz boost by 2 db, with Chebyshev type 1 filter on channel 0.
at 700hz with a width of 500hz attenuate by 5 db, with Chebyshev type 1 filter on channel 0.
at 2000hz with a width of 1000hz attenuate by 2 db, with Chebyshev type 1 filter on channel 0.

I agree. You might have used a friendlier equalizer in your life than this one:)

Those values are based on experimentation and common recommendations for voice. Feel free to tune it for your own needs!

Let’s compare the frequency plots before and after:

Tip: To see the frequency plot in Audacity, open a file, select all, and choose Analyze → Plot spectrum!

Compression

The compressor filter applies dynamic range compression on the incoming audio data. To simplify this, the compressor varies the attenuation based on the incoming signal level. Basically, when you watch a badly mastered movie, this is what you are doing. When it is way too loud in some action scene, you reach for the remote control or mouse to lower the volume, but in the next moment, you will not hear what your heroes are saying, so you increase it back again.

Dynamic range compression roughly does the same. You may set it up in a way so that it would attenuate louder parts, therefore keeping the overall volume range relatively small.

It often happens that performers on the stage use a high dynamic range. Many performers will shout at one moment and then whisper in the next to increase drama or keep the attention. If you want to avoid manually adjusting the volume in real-time (while blowing off your speakers and pulling your hair out), then a compressor will save you in these situations!

This is why our example audio consists of different speaking strengths, so that we could see the dramatic effect of this filter.

ffmpeg -y \
-i gate_eq.wav \
-filter_complex "acompressor=level_in=6:threshold=0.025:ratio=20:makeup=6" \
gate_eq_comp.wav

Let’s hear it: gate_eq_comp.wav

And let’s compare the result of this with the original waveform!

Original:

Result:

Quite dramatic, isn’t it?:)

Let’s analyze this: acompressor=level_in=6:threshold=0.025:ratio=20:makeup=6

First, level_in=6 sets the input gain. It is 1 by default, but since our example, audio is extremely silent at places, we boost up the whole thing before processing.

Then threshold=0.025 defines that everything above 0.025 should be attenuated.

Based on the image below, I’ve decided to cut at this point, as this is above most of the whispering, which cuts hard pops and “s”-es even in the “whisper zone”.

Then ratio=20 means 1:20 in attenuation ratio, which means that if the level rises 20 dB above the threshold, it will be only 1 dB above the line after the attenuation. Basically, this is a very strong compression ratio, it is almost a limiter.

This far, we boosted the signal, then turned down everything that was above our “whisper line” with a quite strong ratio, and now, everything is basically at the whisper level, even the parts that are shouting.

Finally, with the makeup=6 we just bring back everything to the level where the “normal” parts were before.

Let’s take a look back now, to understand why we used the gate and did the equalization before the compressor.

Generally, you want to remove unneeded parts and frequencies before compression, as the compressor will likely increase those too! So by removing most of the noise in the gaps, we avoided level_in=6 to increase them too! And the same goes for the high- and lowpass filtering.

Changing the volume

Now, if we want to make the result a bit louder, we could increase the previous step’s makeup argument, or leverage the volume filter.

While we are at it, let’s cut the first 4 seconds too with -ss 4.

ffmpeg -y \
-ss 4 -i gate_eq_comp.wav \
-filter_complex "volume=1.1" \
gate_eq_volume_comp.wav

Let’s hear it: gate_eq_volume_comp.wav

Let’s make audio gate again

Excuse me for that title:)

So as I’ve described earlier, compression can amplify the noises, so you might want to run the result through a gate again:

ffmpeg -y \
-i gate_eq_volume_comp.wav \
-filter_complex "agate=threshold=0.1:attack=50:release=50:ratio=1.5:knee=4" \
gate_eq_volume_comp_gate.wav

Let’s hear it: gate_eq_volume_comp_gate.wav

In this case, I’ve used a softer gate, with ratio=1.5. Because of this, I could use shorter attack and release delays too, as the attenuation is not that strong, it isn’t causing hard dips in the audio.

Putting it all together

Just a single command could have achieved all the steps above:

ffmpeg -y \
-i voice_recording.wav \
-filter_complex "agate=threshold=0.01:attack=80:release=840:makeup=1:ratio=3:knee=8 , highpass=f=100:width_type=q:width=0.5 , lowpass=f=10000 , anequalizer=c0 f=250 w=100 g=2 t=1|c0 f=700 w=500 g=-5 t=1|c0 f=2000 w=1000 g=2 t=1 , acompressor=level_in=6:threshold=0.025:ratio=20:makeup=6 , volume=1.1 , agate=threshold=0.1:attack=50:release=50:ratio=1.5:knee=4" \
gate_eq_volume_comp_gate_together.wav

I just copy-pasted all the filters right after each other with a comma between them.

Isn’t it beautiful? Yeah, it isn’t, but it is very practical:)

For the last time, check out the difference:

Original: voice_recording.wav
Final: gate_eq_volume_comp_gate.wav

It has less noise, more clear voice, and a small volume range. Therefore it is easy on your ears!

What else?

You might want to check out a few more filters that I didn’t cover here.

Here are just a few interesting ones:

Documentation

For your convenience, let me list the most important documentations that might be important for you! Most of these were already linked many times in this article.

If you got this far from top to bottom, then you are a true hero! I hope you enjoyed this, and I also hope that it inspired you to create something awesome with FFmpeg! Please consider donating to FFmpeg – they are fantastic.

If you’re looking to take your creative projects to the next level, check out our products - Creative Editor SDK, Video Editor SDK, and Photo Editor SDK. These versatile tools empower you to bring your vision to life, whether you’re editing images, crafting stunning videos, or unleashing your artistic talents.

Thanks for reading! Let us know what you think on Twitter! To stay in the loop, subscribe to our Newsletter.

How to Crop and Trim Videos in Flutter

Michael H. — Tue, 27 Sep 2022 08:45:16 GMT

If you are looking for a package that crops and trims videos in Flutter, you must have already come across the video_trimmer Flutter package. This package can trim videos but does not provide video cropping (at least not out-of-the-box). In fact, none of the packages on pub.dev, as of today, allow cropping a video in Flutter. If you have dug deeper, you might have come across FFmpeg — a powerful video editing command line tool, that is not the easiest to get started with. See the Ultimate Guide to FFmpeg for help.In this article we will use the FFmpeg library to crop and trim a video in a Flutter project.

Here is a list of packages that we will be using; ffmpeg_kit_flutter package for cropping and trimming videos, the path_provider package to get the path to the application or external directory where the video files will be stored, and the video_player package to play the video preview.

This article will not cover building any UI for cropping and trimming. But, it will discuss in brief how to crop the video using the video_trimmer library as well. If you want to package this workflow for production, check out our guide on running FFmpeg inside a Docker container. Also, you can try out the code used in this article from this GitHub repository.

Get Started

The end result of this tutorial will be a simple app that would look like the following screenshot.

When you tap the Save Video, the preview will refresh with the cropped and trimmed video replacing the original video. Follow the instructions in this article, and you will be able to develop a similar Flutter app.

First, let us start by adding dependencies to a new Flutter project.

Add Dependencies

In your project’s pubspec.yaml file add the following 3 dependencies.

dependencies:
  ...
  ffmpeg_kit_flutter: ^4.5.1
  path_provider: ^2.0.11
  video_player: ^2.4.6

Then, execute flutter pub get command from the project folder root.

Set Minimum SDK Version and Platform Version

The ffmpeg_kit_flutter plugin runs on Android SDK API level 24+ and iOS SDK 12.1+. Therefore, modify the module level build.gradle file to declare the minSdkVersion.

android {
  defaultConfig {
    ...
    minSdkVersion 24
    ...
  }
}

Then, modify the Podfile to declare the minimum global platform.

# Uncomment this line to define a global platform for your project
platform :ios, '12.1'

Add a Video Asset

For simplicity, we will be using a video asset instead of implementing a file picker. Choose a video file you would like to work with or download this sample video file. Next, copy the video file to a new directory named assets at the root of your project folder.

And then reference the video file from the pubspec.yaml file.

flutter:
  assets:
    - assets/file1.mp4

Execute flutter pub get again.

Having added the required dependencies, let us move on to coding.

Implement Video Crop and Trim

Replace the code in your main.dart file with the following code in the code block. This will be our starter code.

import 'package:flutter/material.dart';

void main() {
  runApp(const MyApp());
}

class MyApp extends StatelessWidget {
  const MyApp({Key? key}) : super(key: key);

  // This widget is the root of your application.
  @override
  Widget build(BuildContext context) {
    return MaterialApp(
      title: 'Crop and Trim Demo',
      home: const MyHomePage(),
    );
  }
}

class MyHomePage extends StatefulWidget {
  const MyHomePage({Key? key}) : super(key: key);

  @override
  State<MyHomePage> createState() => _MyHomePageState();
}

class _MyHomePageState extends State<MyHomePage> {
  @override
  Widget build(BuildContext context) {
    return Scaffold(
      appBar: AppBar(
        title: Text("Flutter Crop and Trim"),
      ),
      body: Column(
        children: [

        ],
      ),
    );
  }
}

Initialize the Input Player

Add the following code to the _MyHomePageState class.

import 'dart:io';

...
import 'package:flutter/services.dart';
import 'package:path_provider/path_provider.dart';
import 'package:video_player/video_player.dart';

...

class _MyHomePageState extends State<MyHomePage> {
  late String inputPath;
  VideoPlayerController? controller;
  String outputPath = "";

  ...

  @override
  void initState() {
    super.initState();
    copyVideoToApplicationDirectory().then((path) async {
      inputPath = path;
      controller = VideoPlayerController.file(File(inputPath));
      await controller?.initialize();
      await controller?.play();
      setState(() {});
      outputPath = await getOutputPath();
    });
  }

  ///Copy input file to ApplicationStorage Directory
  ///returns path to copied video
  Future<String> copyVideoToApplicationDirectory() async {
    const filename = "file1.mp4";
    var bytes = await rootBundle.load("assets/file1.mp4");
    String dir = (await getApplicationDocumentsDirectory()).path;
    writeToFile(bytes, '$dir/$filename');

    return '$dir/$filename';
  }

  ///Write to Path.
  Future<void> writeToFile(ByteData data, String path) {
    final buffer = data.buffer;
    return File(path).writeAsBytes(
        buffer.asUint8List(data.offsetInBytes, data.lengthInBytes));
  }
}

The copyVideoToApplicationDirectory method is copying the video from Assets to the Application directory on the phone’s file system. The resulting path is then stored in the inputPath variable which is then supplied to the FFmpeg command. For large‑scale workloads, you can move this FFmpeg command to AWS Spot Instances; our AWS guide shows how to configure a cloud environment for scalable video processing.

Next, add the VideoPlayer widget to the Column widget.

body: Column(
  children: [
    (controller != null)
        ? AspectRatio(
        aspectRatio: controller!.value.aspectRatio,
        child: VideoPlayer(controller!))
        : const SizedBox(),
  ],
),

Run the app and the video should start playing.

In the next step, we will add the FFmpeg command, but before that add the following 2 more methods to the _MyHomePageState class.

...

import 'package:ffmpeg_kit_flutter/ffmpeg_kit.dart';
import 'package:ffmpeg_kit_flutter/log.dart';
import 'package:ffmpeg_kit_flutter/return_code.dart';

...

class _MyHomePageState extends State<MyHomePage> {
  ...

  /// Output path with a file name where the result will be stored.
  Future<String> getOutputPath() async {
    final appDirectory = Platform.isAndroid
        ? await getExternalStorageDirectory()
        : await getApplicationDocumentsDirectory();
    final externalPath = '${appDirectory?.path}/out_file.mp4';
    return externalPath;
  }

  ///Executes the FFMPEG [command]
  ///Note: Green bar on the right is a Flutter issue. <https://github.com/flutter/engine/pull/24888>
  ///Should get fixed in a 3.1.0+ stable release <https://github.com/flutter/engine/pull/24888#issuecomment-1212374010>
  Future<void> ffmpegExecute(String command) async {

    final session = await FFmpegKit.execute(command);

    final returnCode = await session.getReturnCode();
    if (ReturnCode.isSuccess(returnCode)) {

      print("Success");
      //Replace the preview video
      await controller?.pause();
      await controller?.dispose();
      controller = VideoPlayerController.file(File(outputPath));
      await controller?.initialize();
      await controller?.play();
      setState(() {});

    } else if (ReturnCode.isCancel(returnCode)) {

      print("Cancel");

    } else {

      print("Error");
      final failStackTrace = await session.getFailStackTrace();
      print(failStackTrace);
      List<Log> logs = await session.getLogs();
      for (var element in logs) {
        print(element.getMessage());
      }

    }
  }
}

The getOutputPath method provides the path where the resulting video will be saved. The outputPath will be passed to the FFmpeg command whereas the ffmpegExecute method is where the FFmpeg magic takes place.

The ffmpegExecute method expects a valid FFmpeg String command. The String command is then passed to the FFmpegKit.execute method which returns an instance of FFMpegSession . It will tell us whether our command was executed successfully or not. If it was not executed successfully, we can extract error logs from it.

FFmpeg Execute Command

Add a TextButton from where the FFmpeg command shall be sent.

TextButton(
  child: Text('Save Video'),
  onPressed: () async {
    //TODO: Call FFMPEG Execute
  },
),

Next, call the following ffmpegExecute method from the onPressed property.

ffmpegExecute('-ss 0:00:15 -to 0:00:45 -y -i $inputPath -filter:v "crop=320:150" -c:a copy $outputPath');

Run the app and tap the Save Video button. You will notice that the player is now playing the new cropped and trimmed video.

If you were able to successfully follow the instructions up till here, it is time to dig a little deeper into the FFmpeg command that we just ran earlier.

Understanding the FFmpeg Command

Because command FFmpeg is a command line tool, it expects string-only commands. For this reason, we do not have any dart Classes, Methods, or Parameters to work with that we usually get when working with a dart plugin. Let us dissect the above command line command that is helping us crop and trim a video.

Command	Description
-ss 0:00:15	Seeks to position on the input video. The trim starts from this position.
-to 0:00:45	Stops reading at the position in the input video. The trim stops at this position. The total length of the video is 0:02:05.
-y	Overwrite output files. Helpful when the Save Video button is tapped more than once.
-i $inputPath	Input file location. This is the file on which the crop and trim are applied.
-filter:v	Apply a filter to the video stream.
”crop=320:150”	Apply a crop filter from the center of the video that is 320 pixels wide and 150 pixels tall. The original dimensions of the video were 320 x 240.
-c:a copy $outputPath	Specifies the codec with which the output file must be encoded. Here, copy is a special value to indicate the stream is not to be re-encoded. The a after the colon is a stream specifier for the audio stream.

That will be either the application directory or the external directory of the application, depending on whether the app is running on iOS or Android.

You can check out the app’s code and play with it by downloading it from this GitHub Repository. This project allows you to tweak the duration of the trim and dimensions of the crop, so it has more code, but at the heart of it, it still uses the above FFmpeg command.

Cropping Using the video_trimmer Plugin

As a bonus, here is a tip on how to use the video_trimmer plugin for cropping videos.

The video_trimmer plugin also uses FFmpeg at its core and allows us to pass on an FFmpeg command while saving the video. For this reason, it is an easy task for us to apply the crop to a video using the video_trimmer plugin.

To do this, pass the following command to the saveTrimmedVideo method’s ffmpegCommand parameter as shown in the code block below. The video_trimmer package expects the customVideoFormat parameter argument when the ffmpegCommand parameter is used.

await _trimmer
    .saveTrimmedVideo(
        startValue: _startValue,
        endValue: _endValue,
        ffmpegCommand:
            '-filter:v "crop=320:150"',
        customVideoFormat: '.mp4')
    .then((value) {
  setState(() {
    _value = value;
  });
});

Limitations of this Approach

The crop that we are applying is from the center of the frame. The present command would also need starting coordinates to crop a non-center frame.

The project will need a refined UI for cropping and trimming videos to offer a complete app experience to users. The current approach is miles away from creating that experience.

But perhaps the most significant limitation is FFmpeg’s licensing. FFmpeg is available with both LGPL and GPL licenses, so you must ensure your project is compatible with those licenses. For many commercial projects, this is a non-starter. Alternatively, you can run FFmpeg on Google Cloud Platform; our GCP tutorial walks you through setup.

Commercial Alternative

VideoEditor SDK (VE.SDK) from IMG.LY provides powerful video editing features, including cropping and trimming videos in a Flutter project. You will receive staples of video editing, including straightening videos, filters, brightness, color adjustments, and more. Follow our guide to learn how to integrate IMG.LY’s video editor for Flutter into your app.

Conclusion

While still a complex topic, video manipulation is more attainable to implement on Flutter than on native Android. Nevertheless, FFmpeg is the only open-source, free-to-use option to edit videos on Flutter right now. To automate cropping and trimming across many files or build a transcoding server, see our article on building a production‑ready batch video processing server.

FFmpeg – IMG.LY Blog

A Guide to Batch Video Editing & Server Automation with FFmpeg

Understanding Video Processing Fundamentals

Architecture Deep Dive

System Components Overview

Design Patterns

Scalability Considerations

Setting Up the Development Environment

Prerequisites Installation

Project Structure Walkthrough

Configuration Setup

Docker Compose Deep Dive

Development vs. Production Setup

Configuration Management & Profiles

Profile System (config/profiles.yaml)

Workflow Orchestration

Command Line Interface

CLI Design (cli.py)

Conclusion

How to run FFmpeg inside a Docker container

Installing Docker

Approach 1: Using a Pre-Built FFmpeg Docker Image

Pulling the Image

Running a Container

Using FFmpeg

Exiting the Container

Approach 2: Building a Custom Dockerfile with FFmpeg

Creating a Dockerfile

Building the Image

Running Your Custom Image

Beyond the Basics

Multi-stage builds

Using FFmpeg inside a larger containerized workflow

GPU-accelerated FFmpeg builds

Which Approach Should You Choose?

Conclusion: Why FFmpeg in Docker is the Smart Choice

FFmpeg on GCP: Step-by-Step for Beginners

Introduction

Set Up a Google Cloud Project

1. Create a New Project

2. Enable Billing

3. Set Up Google Cloud Storage

Start a Compute Engine VM

1. Choose a Machine Type

2. Configure Boot Disk

3. Configure Access Scopes

3. Create and Launch the VM

4. Connect via SSH

Install FFmpeg

1) Install via APT (recommended)

2) Quick sanity checks

Test FFmpeg with sample commands

Process a Video

Download Video from GCS to the VM

Run a Conversion with FFmpeg

Upload the Result Back to GCS

Other Options on GCP

Cloud Run for Small Jobs

Transcoder API for a Managed Service

Kubernetes for Large-Scale Workloads

Beyond FFmpeg: A UI for Creative Editing

Conclusion

How to Run FFmpeg on AWS Spot Instances for Scalable, Low-Cost Video Processing

Set Up Your AWS Environment

Create an AWS Account and S3 Bucket

Launch a Spot Instance

Connect to Your Instance via SSH

Install FFmpeg on the Spot Instance

Run FFmpeg on a Spot Instance

Upload a Sample Video to S3

Run FFmpeg on the Spot Instance

Spot Instances vs AWS Lambda for FFmpeg

AWS Lambda for Lightweight Tasks

EC2 Spot Instances for Heavy Workloads

Rule of Thumb

Handling Spot Interruptions

Two-Minute Warning

Break Videos into Smaller Chunks

Store Partial Results in S3

Reliable and Cost-Effective Workflows

Profile System (`config/profiles.yaml`)

CLI Design (`cli.py`)