FFmpeg 10-bit Video Encoding

2024-10-22 10:34 UTC gpt-4o-canmore Open in ChatGPT ↗

To make your FFmpeg command work properly with a 10-bit video, you need to handle the pixel format explicitly because most encoders, especially hardware-accelerated ones like h264_nvenc, default to an 8-bit color depth (yuv420p). For a 10-bit source, you need to ensure the processing chain (input, filtering, and output) respects the higher bit depth.

Here are the key changes required:

Input Handling: Ensure FFmpeg detects the correct bit depth from the input file.
Pixel Format Handling: Specify the correct pixel format when using the hardware encoder.
Hardware Encoding Support: Ensure that the hardware encoder supports 10-bit (h264_nvenc can handle it if set properly).

Try modifying your FFmpeg command to handle 10-bit video like this:

ffmpeg -y -vsync 0 -hwaccel cuda -hwaccel_output_format cuda -i "\\wsl.localhost\Ubuntu\home\ulc\monolith\act_01\ACT_01-ANDI_01-CENTER\ACT_01-ANDI_01-CENTER.mp4" -c:a copy -c:v h264_nvenc -profile:v high10 -pix_fmt p010le -vf "scale_cuda=1920:1080, fps=30" "\\wsl.localhost\Ubuntu\home\ulc\monolith\act_01\ACT_01-ANDI_01-CENTER\ACT_01-ANDI_01-CENTER-1080.mp4"

Explanation:

Profile Option (-profile:v high10):
- The high10 profile supports 10-bit encoding, which is essential when dealing with a 10-bit video input.
Pixel Format (-pix_fmt p010le):
- p010le represents a 10-bit pixel format. You need to explicitly set this format for both the encoder and the filters to respect the bit depth of your original content.
Hardware Acceleration:
- Since you are using CUDA for hardware acceleration (-hwaccel cuda -hwaccel_output_format cuda), the scaling and filtering will also use GPU resources. Setting the correct pixel format (p010le) ensures the process remains consistent with the bit depth.

This should help preserve the 10-bit quality throughout the entire processing chain. If your input video is 10-bit, this command will appropriately maintain the bit depth and encode the output accordingly.

To modify your FFmpeg command for maximum compatibility by converting the 10-bit input video to 8-bit output, you need to ensure that the pixel format is changed to an 8-bit compatible format that is widely supported by most devices. Typically, the yuv420p pixel format is the most compatible option, as it is supported by a majority of playback devices and platforms.

Here’s how you can adjust your command for 8-bit encoding:

ffmpeg -y -vsync 0 -hwaccel cuda -hwaccel_output_format cuda -i "\\wsl.localhost\Ubuntu\home\ulc\monolith\act_01\ACT_01-ANDI_01-CENTER\ACT_01-ANDI_01-CENTER.mp4" -c:a copy -c:v h264_nvenc -pix_fmt yuv420p -vf "scale_cuda=1920:1080, fps=30" "\\wsl.localhost\Ubuntu\home\ulc\monolith\act_01\ACT_01-ANDI_01-CENTER\ACT_01-ANDI_01-CENTER-1080.mp4"

Explanation:

Pixel Format (-pix_fmt yuv420p):
- The yuv420p pixel format is an 8-bit format and is the default format for most videos encoded for maximum compatibility. This ensures that the output is 8-bit, which will be playable on a wider range of devices compared to a 10-bit format.
Hardware Acceleration (-hwaccel cuda -hwaccel_output_format cuda):
- This remains unchanged. It allows CUDA to accelerate the video processing. When setting pix_fmt yuv420p, you ensure that after any GPU-accelerated filtering (like scaling), the resulting pixel data will be converted to 8-bit.
Encoder (-c:v h264_nvenc):
- The h264_nvenc encoder supports both 8-bit and 10-bit, and with pix_fmt yuv420p, it will encode in 8-bit for compatibility.

By explicitly specifying yuv420p, you’re instructing FFmpeg to output an 8-bit video, which will have the highest likelihood of compatibility with older devices, web players, and software that doesn’t support 10-bit color depth. This approach should be effective for standard playback across the widest variety of environments.

It seems like there’s a format mismatch between the filters and the codec, which commonly happens when FFmpeg’s filters are working on data with a bit depth that the following filters or encoder do not support directly. When using CUDA hardware acceleration and specific filters like fps, the underlying data type must match throughout the chain.

To resolve this issue, try modifying the command to ensure consistency across all the filters and codecs. You may need to simplify the filter chain or remove hardware-specific optimizations for certain parts of the video. Here is an adjusted version of your command:

Solution: Ensure Consistent Pixel Format and Order Filters

Use CPU Scaling: Let FFmpeg use CPU scaling rather than CUDA if CUDA scaling is not supporting the bit depth conversion.
Reorder Filters: Changing the order of filters can also make a difference.

Try the following command:

ffmpeg -y -vsync 0 -hwaccel cuda -hwaccel_output_format cuda -i "\\wsl.localhost\Ubuntu\home\ulc\monolith\act_01\ACT_01-ANDI_01-CENTER\ACT_01-ANDI_01-CENTER.mp4" -vf "scale=1920:1080, fps=30" -c:a copy -c:v h264_nvenc -pix_fmt yuv420p "\\wsl.localhost\Ubuntu\home\ulc\monolith\act_01\ACT_01-ANDI_01-CENTER\ACT_01-ANDI_01-CENTER-1080.mp4"

Key Adjustments:

Scaling with CPU (-vf "scale=1920:1080, fps=30"):
- Switched from scale_cuda to regular scale since CPU scaling is more compatible with different bit depths.
- This way, you avoid CUDA-specific issues, and the output from the scale filter will work well with the fps filter and the rest of the chain.
Set pix_fmt as yuv420p for Compatibility (-pix_fmt yuv420p):
- This remains in place, ensuring the final encoded video is in an 8-bit format compatible with most devices.
Simplified Filter Chain:
- By simplifying and reordering the filters, you make sure that everything gets applied in the correct order. First scaling, then adjusting the frame rate.

Alternative Approach:

If you really want to use CUDA scaling for performance reasons, another solution would be to convert the pixel format to a standard 8-bit compatible format right after scaling. CUDA scaling outputs a different bit depth, so adding an additional conversion step may help:

ffmpeg -y -vsync 0 -hwaccel cuda -hwaccel_output_format cuda -i "\\wsl.localhost\Ubuntu\home\ulc\monolith\act_01\ACT_01-ANDI_01-CENTER\ACT_01-ANDI_01-CENTER.mp4" -vf "scale_cuda=1920:1080, format=yuv420p, fps=30" -c:a copy -c:v h264_nvenc -pix_fmt yuv420p "\\wsl.localhost\Ubuntu\home\ulc\monolith\act_01\ACT_01-ANDI_01-CENTER\ACT_01-ANDI_01-CENTER-1080.mp4"

Explanation:

format=yuv420p after scale_cuda:
- Forces the output of the scale_cuda filter into the yuv420p format, ensuring the subsequent fps filter and encoder can process it without errors.

The issue you’re facing is a classic FFmpeg quirk where pixel format mismatches cause filters to fail. By ensuring consistent pixel formats, either by using CPU-based scaling or forcing format conversions explicitly in the chain, you can solve this problem and proceed with the conversion while retaining compatibility.

To handle the process in two stages—first using H.265 (HEVC) for downscaling and filtering, then converting the resulting output to H.264—you can separate the workflow into two FFmpeg commands. This allows you to perform all computationally expensive operations in H.265 format, which is more efficient for retaining quality during scaling, and then perform a final conversion to H.264 for maximum compatibility.

Here’s how you can proceed:

Step 1: Downscale and Process with H.265 (HEVC)

ffmpeg -y -vsync 0 -hwaccel cuda -hwaccel_output_format cuda -i "\\wsl.localhost\Ubuntu\home\ulc\monolith\act_01\ACT_01-ANDI_01-CENTER\ACT_01-ANDI_01-CENTER.mp4" -c:a copy -c:v hevc_nvenc -pix_fmt yuv420p -vf "scale_cuda=1920:1080, fps=30" "\\wsl.localhost\Ubuntu\home\ulc\monolith\act_01\ACT_01-ANDI_01-CENTER\ACT_01-ANDI_01-CENTER-1080-h265.mp4"

Explanation:

Codec (-c:v hevc_nvenc):
- Encode the video using the H.265 codec (hevc_nvenc) to benefit from improved compression and quality retention.
Pixel Format (-pix_fmt yuv420p):
- Ensure the downscaled output is 8-bit for compatibility with most devices.
CUDA Scaling (-vf "scale_cuda=1920:1080, fps=30"):
- Use CUDA for efficient GPU scaling and set the frame rate to 30 fps.

This command produces a downscaled H.265-encoded file.

Step 2: Convert to H.264 for Compatibility

ffmpeg -y -vsync 0 -i "\\wsl.localhost\Ubuntu\home\ulc\monolith\act_01\ACT_01-ANDI_01-CENTER\ACT_01-ANDI_01-CENTER-1080-h265.mp4" -c:a copy -c:v h264_nvenc -pix_fmt yuv420p "\\wsl.localhost\Ubuntu\home\ulc\monolith\act_01\ACT_01-ANDI_01-CENTER\ACT_01-ANDI_01-CENTER-1080-h264.mp4"

Explanation:

Input File (-i "ACT_01-ANDI_01-CENTER-1080-h265.mp4"):
- Use the output from the previous command as the input.
Codec (-c:v h264_nvenc):
- Encode the video using the H.264 codec for broader compatibility.
Pixel Format (-pix_fmt yuv420p):
- Ensures the output video is in the standard 8-bit yuv420p format, suitable for maximum compatibility with most devices and platforms.

Benefits of This Approach:

Efficiency: H.265 (HEVC) allows for better compression, so while you’re downscaling and modifying the video, the quality is better preserved with lower bit rates.
Compatibility: Final conversion to H.264 provides a broadly supported output that works well across many devices and platforms.
Error Isolation: By breaking the task into two commands, it simplifies debugging and handling errors. Each process is focused on a specific transformation, reducing complexity in the processing chain.

This approach should help maintain quality while achieving compatibility with an 8-bit H.264 format.

To combine both commands into one, you’ll want to perform the operations sequentially while maintaining a smooth processing chain that converts the video to H.265 first and then re-encodes to H.264, all in one go. You can do this by using FFmpeg’s intermediate stream process, but you must make sure the operations work in a pipeline-friendly manner. However, encoding twice directly in one command isn’t straightforward, but you can achieve it using intermediate streams.

Single Command for Combined Downscaling and Conversion

Unfortunately, FFmpeg cannot encode in H.265 and then re-encode in H.264 directly within one single run because it would involve two passes of encoding. The most effective way is to keep the conversion to one output format and specify all the transformations required.

However, you can simplify the process and directly convert to H.264 after applying all transformations. Here is an optimized way to achieve downscaling with H.264 encoding:

ffmpeg -y -vsync 0 -hwaccel cuda -hwaccel_output_format cuda -i "\\wsl.localhost\Ubuntu\home\ulc\monolith\act_01\ACT_01-ANDI_01-CENTER\ACT_01-ANDI_01-CENTER.mp4" -vf "scale_cuda=1920:1080, fps=30" -c:a copy -c:v h264_nvenc -pix_fmt yuv420p "\\wsl.localhost\Ubuntu\home\ulc\monolith\act_01\ACT_01-ANDI_01-CENTER\ACT_01-ANDI_01-CENTER-1080-h264.mp4"

Explanation:

CUDA Hardware Acceleration:
- -hwaccel cuda -hwaccel_output_format cuda: Use CUDA for hardware acceleration for efficient decoding and filtering.
Input File:
- Input the source video using -i.
Video Filtering:
- -vf "scale_cuda=1920:1080, fps=30": Downscale the video to 1920x1080 and set the frame rate to 30 fps.
Audio and Video Codec:
- -c:a copy: Copy the audio as-is.
- -c:v h264_nvenc: Encode the output video directly to H.264 using NVENC.
Pixel Format:
- -pix_fmt yuv420p: Explicitly set the pixel format to yuv420p to make the final output compatible with most devices.

Why This Approach?

Combining the commands is tricky since encoding twice in one command is practically difficult in FFmpeg without an intermediate file. Therefore, it’s efficient to just apply the final transformations directly to H.264, and the resulting command gives you both the downscaling and final format conversion in a single step.

Key Points:

The combined command uses h264_nvenc directly to produce the output in H.264, which maintains compatibility and avoids the need for an intermediate file.
This approach eliminates the need for encoding twice, which reduces overall computational load and time.

If you specifically need a two-step encoding process (e.g., encode to H.265 for interim processing), it’s still best to handle them in separate steps, as FFmpeg does not support “re-encoding twice” in one command without saving intermediate output.

If your goal is to simply downscale the 10-bit video and you want to maintain it as a 10-bit file while downscaling, you can do this without any intermediate codec conversion or re-encoding to a lower bit-depth. The command should ensure that the downscaled video retains the original bit depth.

Here’s how you can downscale the 10-bit video while keeping the 10-bit depth:

ffmpeg -y -vsync 0 -hwaccel cuda -hwaccel_output_format cuda -i "\\wsl.localhost\Ubuntu\home\ulc\monolith\act_01\ACT_01-ANDI_01-CENTER\ACT_01-ANDI_01-CENTER.mp4" -vf "scale_cuda=1920:1080, fps=30" -c:a copy -c:v hevc_nvenc -pix_fmt p010le "\\wsl.localhost\Ubuntu\home\ulc\monolith\act_01\ACT_01-ANDI_01-CENTER\ACT_01-ANDI_01-CENTER-1080-10bit.mp4"

Explanation:

CUDA Hardware Acceleration (-hwaccel cuda -hwaccel_output_format cuda):
- You’re using CUDA for GPU acceleration, which helps speed up both decoding and the scaling process.
Input File (-i "\\wsl.localhost\..."):
- This is the original video file.
Video Filtering (-vf "scale_cuda=1920:1080, fps=30"):
- This filter is used to downscale the video to 1920x1080 resolution.
- fps=30 changes the frame rate to 30 fps.
Video Codec (-c:v hevc_nvenc):
- Use the HEVC encoder (hevc_nvenc) to output the downscaled video. This encoder can support 10-bit depth, which is what your source video is.
Pixel Format (-pix_fmt p010le):
- Use p010le to maintain the 10-bit depth. The p010le format represents a 10-bit color depth in a planar format, which is standard for high-quality video encoding.
Audio Copy (-c:a copy):
- The audio is simply copied without re-encoding.

Result:

This command will take your original 10-bit video, downscale it to 1920x1080, retain the original bit depth (10-bit), and encode it using HEVC (H.265) for efficient storage. This way, you maintain quality and stay consistent with the bit depth of the original video.

This error message indicates that there is an issue with converting between formats within the filter chain, specifically due to incompatibilities between the fps and scale_cuda filters when working with a 10-bit video. It’s a common issue when using hardware-accelerated filters and specific pixel formats that are not directly supported by some filters.

Here are a few adjustments to ensure that the formats are compatible and that the filter chain works seamlessly:

Solution: Force Pixel Format with Explicit Filter Adjustments

To address this, you can add a format filter to explicitly convert the pixel format before processing it. This ensures that all the filters are compatible with each other. Here’s the updated command:

ffmpeg -y -vsync 0 -hwaccel cuda -hwaccel_output_format cuda -i "\\wsl.localhost\Ubuntu\home\ulc\monolith\act_01\ACT_01-ANDI_01-CENTER\ACT_01-ANDI_01-CENTER.mp4" -vf "scale_cuda=1920:1080, format=p010le, fps=30" -c:a copy -c:v hevc_nvenc -pix_fmt p010le "\\wsl.localhost\Ubuntu\home\ulc\monolith\act_01\ACT_01-ANDI_01-CENTER\ACT_01-ANDI_01-CENTER-1080-10bit.mp4"

Key Changes:

Add a Format Filter (format=p010le):
- -vf "scale_cuda=1920:1080, format=p010le, fps=30":
  - This modification explicitly sets the pixel format to p010le after downscaling.
  - Adding format=p010le helps to align the bit depth properly between the scale_cuda and fps filters.
  - It forces the pixel format to a compatible 10-bit format before changing the frame rate (fps=30).
Pixel Format (-pix_fmt p010le):
- Retains the 10-bit depth for final output, ensuring the entire pipeline remains at 10-bit quality.

Why This Works:

The error occurs because the different filters are expecting different pixel formats, and by adding the format=p010le filter, we enforce a consistent format between each step.
CUDA-based scaling may produce an output format that the fps filter cannot directly handle, and converting it to a known compatible format (p010le) makes sure that all operations can be executed in sequence without format-related issues.

Alternative: CPU-Based Scaling

If CUDA scaling continues to have issues with pixel format conversions, you could consider using CPU-based scaling:

ffmpeg -y -vsync 0 -i "\\wsl.localhost\Ubuntu\home\ulc\monolith\act_01\ACT_01-ANDI_01-CENTER\ACT_01-ANDI_01-CENTER.mp4" -vf "scale=1920:1080, fps=30, format=p010le" -c:a copy -c:v hevc_nvenc -pix_fmt p010le "\\wsl.localhost\Ubuntu\home\ulc\monolith\act_01\ACT_01-ANDI_01-CENTER\ACT_01-ANDI_01-CENTER-1080-10bit.mp4"

Scale Filter (scale=1920:1080):
- Use the software scale filter instead of CUDA to ensure compatibility, as the CPU-based filter chain may handle format conversion more gracefully.

This should help resolve the errors you are seeing by ensuring format compatibility throughout the processing chain while maintaining a 10-bit output.

Got it! You want to use CPU-based scaling and set the frame rate to 30 fps while encoding with H.264. This approach avoids CUDA-related issues and uses CPU resources for downscaling and modifying the frame rate, which ensures better compatibility.

Here’s the FFmpeg command to achieve your requirements:

ffmpeg -y -vsync 0 -i "\\wsl.localhost\Ubuntu\home\ulc\monolith\act_01\ACT_01-ANDI_01-CENTER\ACT_01-ANDI_01-CENTER.mp4" -vf "scale=1920:1080, fps=30" -c:a copy -c:v libx264 -pix_fmt yuv420p "\\wsl.localhost\Ubuntu\home\ulc\monolith\act_01\ACT_01-ANDI_01-CENTER\ACT_01-ANDI_01-CENTER-1080-h264.mp4"

Explanation:

Input File (-i):
- Specify the input video file as usual.
Video Filter (-vf "scale=1920:1080, fps=30"):
- scale=1920:1080: Downscale the video to 1920x1080 using CPU resources.
- fps=30: Set the frame rate to 30 fps.
Audio Codec (-c:a copy):
- Copy the audio without re-encoding.
Video Codec (-c:v libx264):
- Use the libx264 encoder to output the video in H.264 format, which is widely compatible.
Pixel Format (-pix_fmt yuv420p):
- Set the pixel format to yuv420p, which is an 8-bit format widely supported across devices and ensures compatibility.

What This Command Does:

This command downscales the video using CPU-based processing (which avoids hardware compatibility issues).
It changes the frame rate to 30 fps, ensuring smooth playback at the desired frame rate.
Finally, it encodes the video in H.264 format, ensuring maximum compatibility across a wide range of devices and platforms.

The use of libx264 will ensure a compatible H.264 output, and the pixel format yuv420p guarantees compatibility with older devices that may not support higher bit-depth formats like 10-bit. This approach should work seamlessly for your requirements.

Yes, you can use NVENC while avoiding the compatibility issues you’ve faced by making a few adjustments to the command to ensure the NVENC encoder can properly work with all the filters. Specifically, we want to ensure that we are managing the pixel formats correctly to match the bit depth, especially since we need to interact properly with the fps and scaling filters.

The NVENC encoder (h264_nvenc) can be used, but it requires some fine-tuning when you’re downscaling and setting frame rates, especially when the input is 10-bit.

Here is the adjusted command to achieve what you want while still using NVENC:

ffmpeg -y -vsync 0 -hwaccel cuda -hwaccel_output_format cuda -i "\\wsl.localhost\Ubuntu\home\ulc\monolith\act_01\ACT_01-ANDI_01-CENTER\ACT_01-ANDI_01-CENTER.mp4" -vf "scale_cuda=1920:1080, format=yuv420p, fps=30" -c:a copy -c:v h264_nvenc -pix_fmt yuv420p "\\wsl.localhost\Ubuntu\home\ulc\monolith\act_01\ACT_01-ANDI_01-CENTER\ACT_01-ANDI_01-CENTER-1080-h264.mp4"

Key Changes Explained:

CUDA Hardware Acceleration:
- -hwaccel cuda -hwaccel_output_format cuda: Uses CUDA for hardware acceleration, making decoding and scaling faster and offloading the CPU.
Input File:
- Use the original video as input.
Video Filter Chain (-vf):
- scale_cuda=1920:1080: Uses CUDA to downscale the video. This is a GPU-accelerated scaling filter for faster processing.
- format=yuv420p: Ensures that after scaling, the pixel format is converted to yuv420p, which is an 8-bit format compatible with NVENC encoding. This avoids format mismatch issues that could arise in the filter chain.
- fps=30: Sets the frame rate to 30 fps. Adding this after format ensures that the operation is on compatible pixel data.
Audio Copy (-c:a copy):
- Copies the audio as-is without re-encoding, saving processing time.
Video Encoder (-c:v h264_nvenc):
- Uses the NVENC H.264 encoder to encode the output video in H.264 format. NVENC is efficient and leverages the GPU for faster encoding.
Pixel Format (-pix_fmt yuv420p):
- Set the pixel format for the final output to yuv420p for maximum compatibility. This ensures the output is 8-bit, which is necessary since most playback devices prefer 8-bit H.264 (yuv420p).

Why This Works:

format=yuv420p is included to ensure the pixel format after scaling matches what NVENC expects, preventing the error due to pixel format mismatches.
NVENC (h264_nvenc) works well with yuv420p for compatibility and efficiency. By forcing the format to yuv420p, we avoid issues related to 10-bit (p010le) and unsupported pixel formats in filters.

Summary:

This command ensures that:

You get the benefits of GPU acceleration for both scaling and encoding by using cuda for hardware acceleration and h264_nvenc for encoding.
The filters (scale_cuda and fps) work smoothly without any format incompatibility by using format=yuv420p.
The output video is encoded in H.264, 8-bit (yuv420p) for maximum device compatibility.

By adjusting the pixel format at the right stage in the filter chain, you can use NVENC without running into the common pixel format and compatibility errors.

The error message you’re encountering usually occurs when the filter chain can’t properly handle pixel format conversions, especially between GPU-accelerated filters (scale_cuda) and others, which expect different formats. The NVENC encoder, combined with CUDA hardware acceleration and certain filters, can sometimes result in incompatible format chains.

To address this, I’ll guide you through using a mix of CPU and GPU-based processing, where necessary, to ensure compatibility throughout the entire workflow. We can make some changes to explicitly use compatible pixel formats or leverage a hybrid approach where scaling is done with the CPU for better compatibility.

Here’s an updated solution:

Solution 1: CPU-Based Scaling with NVENC Encoding

In this approach, we’ll avoid scale_cuda entirely, using the CPU to downscale, and then still use h264_nvenc for encoding:

ffmpeg -y -vsync 0 -hwaccel cuda -hwaccel_output_format cuda -i "\\wsl.localhost\Ubuntu\home\ulc\monolith\act_01\ACT_01-ANDI_01-CENTER\ACT_01-ANDI_01-CENTER.mp4" -vf "scale=1920:1080, fps=30, format=yuv420p" -c:a copy -c:v h264_nvenc -pix_fmt yuv420p "\\wsl.localhost\Ubuntu\home\ulc\monolith\act_01\ACT_01-ANDI_01-CENTER\ACT_01-ANDI_01-CENTER-1080-h264.mp4"

Explanation:

CUDA Hardware Acceleration (-hwaccel cuda -hwaccel_output_format cuda):
- We are still using GPU hardware acceleration for decoding, which offloads some work from the CPU.
Input File (-i):
- The input video is specified as usual.
Video Filter Chain (-vf):
- scale=1920:1080: Uses the CPU to downscale the video to 1920x1080 resolution. This avoids issues associated with scale_cuda, which can have pixel format compatibility issues.
- fps=30: Changes the frame rate to 30 fps.
- format=yuv420p: Converts the pixel format to yuv420p to ensure compatibility with the NVENC encoder.
Audio Codec (-c:a copy):
- Copies the audio stream without re-encoding, saving time.
Video Codec (-c:v h264_nvenc):
- Uses the h264_nvenc encoder to encode the output video in H.264 format.
Pixel Format (-pix_fmt yuv420p):
- The output is forced to yuv420p to ensure the video is in an 8-bit format suitable for most devices.

Why This Works:

CPU Scaling: Using scale instead of scale_cuda ensures compatibility since the CPU-based scale filter can output directly in the desired format without GPU pixel format limitations.
NVENC Encoding: Using h264_nvenc still takes advantage of GPU hardware acceleration, but avoids the complexity that arises from GPU scaling.

Solution 2: Separate Scaling and Encoding Steps

Alternatively, you can run the scaling and frame rate change separately in one command and then re-encode in another command:

Step 1: Downscale and Change Frame Rate:

ffmpeg -y -vsync 0 -i "\\wsl.localhost\Ubuntu\home\ulc\monolith\act_01\ACT_01-ANDI_01-CENTER\ACT_01-ANDI_01-CENTER.mp4" -vf "scale=1920:1080, fps=30, format=yuv420p" -c:a copy -c:v libx264 -preset fast -crf 18 "\\wsl.localhost\Ubuntu\home\ulc\monolith\act_01\ACT_01-ANDI_01-CENTER\ACT_01-ANDI_01-CENTER-1080-temp.mp4"

Step 2: Encode Using NVENC:

ffmpeg -y -i "\\wsl.localhost\Ubuntu\home\ulc\monolith\act_01\ACT_01-ANDI_01-CENTER\ACT_01-ANDI_01-CENTER-1080-temp.mp4" -c:a copy -c:v h264_nvenc -pix_fmt yuv420p "\\wsl.localhost\Ubuntu\home\ulc\monolith\act_01\ACT_01-ANDI_01-CENTER\ACT_01-ANDI_01-CENTER-1080-h264.mp4"

Explanation:

In Step 1, we use the CPU to downscale and change the frame rate. Encoding in H.264 is done using software (libx264) but with a high-quality preset and constant rate factor (-crf) for good quality.
In Step 2, we use NVENC to re-encode the temporary file, now using GPU acceleration.

This two-step approach is a bit more work but can also overcome the filter compatibility issues you’re facing, as each step deals with one task at a time, avoiding conflicts in the filter chain.

Summary

The root of your issue is the incompatibility between filters and hardware-accelerated pixel formats when using GPU-based filtering (scale_cuda) with certain encoders and formats. Switching to CPU-based scaling while still using NVENC for encoding is a practical way to solve the problem. The provided commands should allow you to achieve the required output while maintaining compatibility and efficient use of resources.

To ensure that all timing and timecode issues are fixed in a clip that was cut from a larger clip, it is important to re-generate the timecode and fix any potential discrepancies in timestamps. This process will effectively remove any inherited PTS/DTS (Presentation Time Stamps / Decoding Time Stamps) issues or discontinuities from the original file. Here’s how to achieve that with FFmpeg:

Key Steps to Fix Timecode and Timing Issues:

Re-encode with new timestamps: Use -reset_timestamps 1 to reset timestamps.
Ensure proper stream and container compatibility: Using -c copy can cause issues if the original file had discontinuities. Re-encoding typically fixes those issues.

Here’s an FFmpeg command that should fix all timing/timecode issues for a video that was cut from a larger clip:

ffmpeg -y -vsync 0 -i "input_clip.mp4" -reset_timestamps 1 -c:v libx264 -c:a aac -strict experimental -preset fast -crf 18 -movflags +faststart "output_fixed.mp4"

Explanation of the Command:

Input File (-i "input_clip.mp4"):
- Specifies the input clip that has timing issues.
Reset Timestamps (-reset_timestamps 1):
- This flag resets all timestamps, essentially “starting fresh” with new timestamps that are sequentially assigned. This helps resolve issues that often occur when a clip is extracted from a larger file.
Video Codec (-c:v libx264):
- Re-encode the video using the libx264 encoder. Re-encoding helps to rewrite all timing metadata correctly, avoiding issues that might be carried over when using -c copy.
Audio Codec (-c:a aac -strict experimental):
- Re-encode the audio using the aac codec. Re-encoding the audio stream ensures that any possible timing issues in the audio stream are also fixed.
- -strict experimental is used with AAC to allow compatibility, though most modern FFmpeg versions no longer need it.
Encoding Preset (-preset fast):
- The -preset fast option controls the speed vs. compression efficiency. You can change this to ultrafast, medium, or slow depending on your needs.
Constant Rate Factor (-crf 18):
- This sets the quality of the output video. -crf 18 is visually lossless, while higher values will result in lower quality and smaller file sizes. Values from 18 to 23 are usually good choices depending on your requirements.
-movflags +faststart:
- This moves some metadata to the beginning of the file, which is particularly useful for web playback as it allows the video to start playing without needing to download the entire file first.

Alternative Command: Direct Copy with Timestamp Adjustment

If re-encoding is not preferred (e.g., you want to keep the original quality intact and avoid re-encoding time), you could use a command like this:

ffmpeg -y -vsync 0 -i "input_clip.mp4" -reset_timestamps 1 -c copy "output_fixed.mp4"

-c copy:
- This option copies the video and audio without re-encoding. This is faster but might not fix all timing issues if the original discontinuities are severe or complex.
-reset_timestamps 1:
- Resets the timestamps, which often helps align any PTS/DTS issues after cutting.

Why Re-Encoding Is Often Better:

Re-encoding the video and audio streams generally produces a file without timing or sync issues because it completely regenerates all timing information, including PTS and DTS. Simply copying the streams without re-encoding may leave some timing metadata that still has issues, particularly if the clip was poorly cut from the original.

By following these approaches, you should be able to generate a video file that resolves most, if not all, timing and timecode issues, allowing smooth playback and synchronization.

Understood. To fix timestamp issues without re-encoding, you need to use FFmpeg in a way that resets timestamps, repairs discontinuities, and ensures that all frames have correct timecode information while preserving the original video and audio streams (-c copy).

Here’s a command that should help resolve timestamp and timecode issues:

ffmpeg -y -vsync 0 -i "input_clip.mp4" -c copy -fflags +genpts -reset_timestamps 1 "output_fixed.mp4"

Explanation:

Input File (-i "input_clip.mp4"):
- Specifies the input video clip that has timing or timecode issues.
-c copy:
- Copy the video and audio streams without re-encoding. This is essential for maintaining the original quality and avoiding the processing overhead of re-encoding.
-fflags +genpts:
- This flag regenerates Presentation Time Stamps (PTS) for the output file. It’s particularly useful when the input file has inconsistent or missing PTS values.
-reset_timestamps 1:
- Resets timestamps to start from zero. This helps if there are discontinuities between frames caused by cutting the clip from a larger video.
Output File ("output_fixed.mp4"):
- This is the fixed video output with adjusted timestamps and timecodes.

How This Helps:

No Re-Encoding: The command does not perform any re-encoding, which means there is no loss of quality and the processing time is significantly faster.
Fix Timestamp Issues: -reset_timestamps 1 and -fflags +genpts work together to regenerate and reset the timing information, which should resolve issues with playback timing or sync problems resulting from improper cutting.

This approach is ideal when you want to correct timestamp and timecode issues while maintaining the exact quality and encoding settings of the original video.

there’s soemthing wring with my file, show black for a few secinds. ffprobe gives me this:

ffprobe \wsl.localhost\Ubuntu\home\ulc\monolith\act_01\ACT_01-ANDI_01-CENTER\ACT_01-ANDI_01-CENTER\ACT_01-ANDI_01-CENTER_Part01_Simple_walking.mp4 ffprobe version 6.1.1-essentials_build-www.gyan.dev Copyright (c) 2007-2023 the FFmpeg developers built with gcc 12.2.0 (Rev10, Built by MSYS2 project) configuration: —enable-gpl —enable-version3 —enable-static —pkg-config=pkgconf —disable-w32threads —disable-autodetect —enable-fontconfig —enable-iconv —enable-gnutls —enable-libxml2 —enable-gmp —enable-bzlib —enable-lzma —enable-zlib —enable-libsrt —enable-libssh —enable-libzmq —enable-avisynth —enable-sdl2 —enable-libwebp —enable-libx264 —enable-libx265 —enable-libxvid —enable-libaom —enable-libopenjpeg —enable-libvpx —enable-mediafoundation —enable-libass —enable-libfreetype —enable-libfribidi —enable-libharfbuzz —enable-libvidstab —enable-libvmaf —enable-libzimg —enable-amf —enable-cuda-llvm —enable-cuvid —enable-ffnvcodec —enable-nvdec —enable-nvenc —enable-dxva2 —enable-d3d11va —enable-libvpl —enable-libgme —enable-libopenmpt —enable-libopencore-amrwb —enable-libmp3lame —enable-libtheora —enable-libvo-amrwbenc —enable-libgsm —enable-libopencore-amrnb —enable-libopus —enable-libspeex —enable-libvorbis —enable-librubberband libavutil 58. 29.100 / 58. 29.100 libavcodec 60. 31.102 / 60. 31.102 libavformat 60. 16.100 / 60. 16.100 libavdevice 60. 3.100 / 60. 3.100 libavfilter 9. 12.100 / 9. 12.100 libswscale 7. 5.100 / 7. 5.100 libswresample 4. 12.100 / 4. 12.100 libpostproc 57. 3.100 / 57. 3.100 [mov,mp4,m4a,3gp,3g2,mj2 @ 00000167469113c0] Could not find codec parameters for stream 0 (Video: h264 (avc1 / 0x31637661), none(tv, bt2020nc/bt2020/arib-std-b67), 1920x1080, 1969 kb/s): unspecified pixel format Consider increasing the value for the ‘analyzeduration’ (0) and ‘probesize’ (5000000) options Input #0, mov,mp4,m4a,3gp,3g2,mj2, from ‘\wsl.localhost\Ubuntu\home\ulc\monolith\act_01\ACT_01-ANDI_01-CENTER\ACT_01-ANDI_01-CENTER\ACT_01-ANDI_01-CENTER_Part01_Simple_walking.mp4’: Metadata: major_brand : isom minor_version : 512 compatible_brands: isomiso2avc1mp41 encoder : Lavf60.16.100 Duration: 00:01:05.07, start: 0.000000, bitrate: 1853 kb/s Stream #0:00x1: Video: h264 (avc1 / 0x31637661), none(tv, bt2020nc/bt2020/arib-std-b67), 1920x1080, 1969 kb/s, 30 fps, 30 tbr, 15360 tbn (default) Metadata: handler_name : Core Media Video vendor_id : [0][0][0][0] encoder : Lavc60.31.102 libx264 Stream #0:10x2: Audio: aac (LC) (mp4a / 0x6134706D), 44100 Hz, stereo, fltp, 128 kb/s (default) Metadata: handler_name : Core Media Audio vendor_id : [0][0][0][0]

From the ffprobe output, there are some notable issues that could be causing the black frames:

“Could not find codec parameters for stream 0”:
- FFmpeg cannot determine the pixel format of the video stream. This is usually due to a missing or unspecified pixel format in the metadata or an incorrect specification in the original file. This lack of information may cause the player to misinterpret or fail to display frames correctly, resulting in black screens.
Pixel Format Missing (none(tv, bt2020nc/bt2020/arib-std-b67)):
- The pixel format is not defined, meaning FFmpeg cannot properly interpret the video data for decoding or display.
Suggestion to increase analyzeduration and probesize:
- Increasing these values may allow FFmpeg to correctly detect more information about the file, helping it determine codec parameters.

Fixing the File:

To resolve these issues, I suggest trying a few steps to force FFmpeg to properly interpret the codec and pixel format and rewrite the file. Here’s how you can do it:

Step 1: Increase `analyzeduration` and `probesize`

First, try to increase the analyzeduration and probesize values, which can help FFmpeg detect more information from the file, particularly the pixel format. This can be done using the following command:

ffmpeg -analyzeduration 50000000 -probesize 50000000 -i "\\wsl.localhost\Ubuntu\home\ulc\monolith\act_01\ACT_01-ANDI_01-CENTER\ACT_01-ANDI_01-CENTER\ACT_01-ANDI_01-CENTER_Part01_Simple_walking.mp4" -c copy "output_fixed.mp4"

Explanation:

-analyzeduration 50000000 and -probesize 50000000:
- Increase the amount of data FFmpeg examines before processing the video. The default values may be too small to capture all the information, and increasing them helps FFmpeg figure out details like the pixel format.
-c copy:
- Copies all streams without re-encoding. This is much faster and preserves quality.
output_fixed.mp4:
- The fixed version of your video.

Step 2: Specify a Pixel Format if Missing

If the above step does not resolve the black frames issue, you can manually specify the pixel format. Based on the video metadata (bt2020), it appears to be using an HDR color space. A common pixel format that might match this is yuv420p10le (10-bit color depth). Here’s how you can try forcing the pixel format:

ffmpeg -y -i "\\wsl.localhost\Ubuntu\home\ulc\monolith\act_01\ACT_01-ANDI_01-CENTER\ACT_01-ANDI_01-CENTER\ACT_01-ANDI_01-CENTER_Part01_Simple_walking.mp4" -pix_fmt yuv420p -c:v copy -c:a copy "output_fixed_pixel_format.mp4"

Explanation:

-pix_fmt yuv420p:
- Forces the output video to use the yuv420p pixel format, which is the most compatible 8-bit format. If your original file is 10-bit, you can use yuv420p10le for 10-bit output.
-c:v copy and -c:a copy:
- Copy both video and audio without re-encoding, preserving original quality and reducing processing time.

Step 3: Re-Mux the File Without Re-Encoding

If the above steps don’t work, re-muxing the file could be the solution. Sometimes remuxing can fix minor container-level metadata issues that cause playback issues. Here is how you can re-mux:

ffmpeg -y -i "\\wsl.localhost\Ubuntu\home\ulc\monolith\act_01\ACT_01-ANDI_01-CENTER\ACT_01-ANDI_01-CENTER\ACT_01-ANDI_01-CENTER_Part01_Simple_walking.mp4" -map 0 -c copy "output_remuxed.mp4"

Explanation:

-map 0:
- Maps all streams from the input file to the output file. This ensures that all audio, video, and subtitle tracks are kept.
-c copy:
- Copies all streams without re-encoding to fix possible container issues.

Summary

Increase analyzeduration and probesize: This will help FFmpeg read the entire file and detect more detailed information.
Specify Pixel Format: If FFmpeg still can’t determine the pixel format, explicitly specify it.
Re-Mux the File: To eliminate container-level metadata or structural issues, use -map 0 and -c copy to create a fresh output file.

These approaches should help resolve the black screen issue, and fix any missing timecode or codec information that FFmpeg cannot interpret.

Sure! The Python script you provided uses FFmpeg to split a clip without re-encoding (-c copy). While this method is generally efficient and retains quality, it sometimes results in issues if the -ss flag is used inappropriately, especially when cutting is done without considering keyframes. When using -c copy, FFmpeg is required to start at keyframes, and issues like missing codec parameters or black frames can occur if the cut isn’t aligned to a keyframe.

Here’s how you can modify the script to avoid these common pitfalls:

Use -ss Before and After Input: Place -ss before the -i argument for faster seeking (this is done using a rough estimate and can cause non-accurate frame cuts). Then use another -ss after specifying the input for more precise frame-accurate cuts.
Avoid -c copy for Better Compatibility: We will do a very light re-encoding, particularly near the start and end points, to ensure compatibility and fix potential keyframe issues.

Modified Code:

Below is a modified version of your script:

import subprocess

def split_clip_ffmpeg(video_file, start_time, end_time, output_filepath):
    cmd = [
        'ffmpeg',
        '-y',  # Overwrite output files without asking
        '-ss', str(start_time),  # Fast seek to start point
        '-i', video_file,
        '-to', str(end_time),  # End time after input for accuracy
        '-c:v', 'libx264',  # Re-encode video with libx264 for keyframe compatibility
        '-preset', 'fast',  # Preset for encoding speed and quality
        '-crf', '23',  # Control quality with CRF (23 is medium quality)
        '-c:a', 'aac',  # Re-encode audio with AAC
        '-strict', 'experimental',  # Allow the experimental AAC encoder if required
        '-pix_fmt', 'yuv420p',  # Force a common pixel format for maximum compatibility
        '-movflags', '+faststart',  # Help with playback by moving metadata to start
        output_filepath
    ]
    try:
        subprocess.run(cmd, check=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
        print(f"Created {output_filepath}")
    except subprocess.CalledProcessError as e:
        print(f"Error creating {output_filepath}: {e.stderr.decode()}")

# Example Usage
# split_clip_ffmpeg("input.mp4", "00:00:10", "00:00:30", "output.mp4")

Key Changes Explained:

Position of -ss and -to:
- -ss Before -i: This quickly seeks to the start time, which saves processing time. However, this is not frame-accurate, as it seeks based on a rough estimate.
- -to After Input: This ensures FFmpeg accurately finds the precise time point from the input file.
Re-encoding Instead of Copy (-c:v libx264):
- Re-encoding the video with libx264 ensures that the output is keyframe aligned, which is often necessary when cutting videos. Using -c copy often introduces problems when the cut point is not exactly at a keyframe, causing playback issues.
- -preset fast and -crf 23 are used to control the speed and quality of the re-encoding. You can adjust crf between 18 (higher quality) and 28 (lower quality) depending on your needs.
Audio Codec (-c:a aac):
- Use aac to re-encode audio, which ensures compatibility across a wide variety of devices.
Pixel Format (-pix_fmt yuv420p):
- Force the pixel format to yuv420p, which is the most widely supported 8-bit format, ensuring better compatibility for playback.
Fast Start (-movflags +faststart):
- This moves the metadata to the beginning of the file, which is helpful for web playback, allowing the video to start before the entire file is downloaded.

Why These Changes Will Help:

Keyframe Compatibility: By re-encoding with libx264, you can ensure that the output clip starts cleanly at the desired time, avoiding the black screen or codec issues caused by cutting at non-keyframe points.
Consistent Pixel Format: yuv420p is used to avoid unspecified pixel format issues that were shown in your ffprobe output.
Improved Seek Accuracy: Placing -ss before and after the input provides a balance between fast seeking and accurate cutting.

This approach should eliminate the issues you’ve been experiencing with the clips (like black frames or unspecified pixel format) by ensuring that each segment is re-encoded in a compatible way, providing consistent results.

You’re right—it would be ideal to avoid re-encoding if possible, since re-encoding involves some loss of quality and increases processing time. To avoid re-encoding, we can still modify the way FFmpeg cuts the video to minimize issues like black frames or timestamp problems. Here’s how we can adjust the script:

Issues with Original Code and Why It Causes Problems:

Cutting with -c copy: When using -c copy, FFmpeg simply copies the data stream without re-encoding, which means the cut point must align with a keyframe. If the specified start time is not aligned with a keyframe, it can result in black frames or other artifacts in the output video.
Use of -ss Position: Using -ss after -i causes frame-accurate seeking but can be slower, and it still requires keyframe accuracy for -c copy.

Approach to Improve the Clip without Re-Encoding:

To improve the outcome while avoiding re-encoding:

Move -ss before -i for a faster rough seek: This should reduce processing time by seeking to the approximate start point before reading the entire input.
Remove -to and use -t: This can help avoid issues with calculating the exact end time relative to the -ss position.
Add -avoid_negative_ts 1: This will help to avoid negative timestamps which can sometimes lead to display issues like black frames.

Modified Code Without Re-Encoding:

import subprocess

def split_clip_ffmpeg(video_file, start_time, end_time, output_filepath):
    duration = end_time - start_time  # Calculate the duration for the `-t` option.
    cmd = [
        'ffmpeg',
        '-y',  # Overwrite output files without asking
        '-ss', str(start_time),  # Fast seek to the start point, before input file
        '-i', video_file,
        '-t', str(duration),  # Duration of the output clip
        '-c', 'copy',  # Copy without re-encoding
        '-avoid_negative_ts', '1',  # Avoid negative timestamps
        output_filepath
    ]
    try:
        subprocess.run(cmd, check=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
        print(f"Created {output_filepath}")
    except subprocess.CalledProcessError as e:
        print(f"Error creating {output_filepath}: {e.stderr.decode()}")

# Example Usage
# split_clip_ffmpeg("input.mp4", 10, 30, "output.mp4")

Key Changes Explained:

Use -ss Before Input (-ss start_time):
- -ss before -i: This allows FFmpeg to roughly seek to the desired start time quickly, avoiding the need to decode the entire video up to that point. While this may not be frame-accurate, it should be sufficient for most cuts and reduces processing time significantly.
Calculate Duration and Use -t:
- -t Duration: Instead of using -to, use -t to specify the duration (end_time - start_time). This is more straightforward and avoids any potential calculation inaccuracies with -to.
Avoid Negative Timestamps (-avoid_negative_ts 1):
- -avoid_negative_ts: This option is used to prevent issues that may arise from negative timestamps, which can sometimes result from cutting at points that aren’t keyframes. This can help prevent the playback issues like black frames you are experiencing.

Why These Changes Can Avoid Re-encoding Issues:

Keyframe Alignment: By using -ss before -i, FFmpeg will use the nearest keyframe before the specified start time. This can help avoid issues associated with frame-accurate seeking (-ss after -i), where non-keyframe cuts could cause playback problems.
Maintaining Original Quality: By keeping -c copy, you maintain the original quality of the video and audio streams without re-encoding. This is faster and keeps the process lossless.

Note:

Black Frames: If black frames are still present at the start of the output video, it is likely that the -ss position is between keyframes. If that is the case and you still want precise cuts without artifacts, re-encoding a few seconds at the start (-c:v libx264) would be necessary.
Accurate Cuts: FFmpeg cannot always make a perfect cut at arbitrary points without re-encoding because -c copy must start at a keyframe. If frame-accurate cuts are absolutely needed, some re-encoding might be required for that part.

Using this revised method should make the process faster and preserve quality while avoiding the common issues like black screens caused by incorrect keyframe alignment.