如何使用 C# 编码/解码视频？

How to encode/decode video using C#?

提问by DH.

A little background, I was given the task of fixing a few "small" bugs and maintaining this solution for streaming video across the network between two instances of our application. The solution was written by someone who is no longer here so there is some mystery in the code as well as some really fun pitfalls. The solution was written using ffmpeg with C++ code written to wrap the encoding/decoding related code as well as some of the streaming code. This C++ was then wrapped with SWIG so that it could interop with C# and pass the video frames up where they are rendered using VideoRendererElementwhich lives in a WPF control. The main reason the frames are passed up is because we have some custom protocols we need to send video data over and those are written using C# so as the video frames are passed up we wrap them in our own packets and send them out on the wire. This solution works and we can stream video using our custom protocols though it is something of a nightmare to maintain and work with.

一点背景知识，我的任务是修复一些“小”错误并维护此解决方案，以便在我们应用程序的两个实例之间通过网络流式传输视频。该解决方案是由不再在这里的人编写的，因此代码中有一些神秘之处以及一些非常有趣的陷阱。该解决方案是使用 ffmpeg 编写的，并使用 C++ 代码编写来包装编码/解码相关代码以及一些流媒体代码。然后将此 C++ 与 SWIG 一起包装，以便它可以与 C# 互操作，并将视频帧传递到使用VideoRendererElement呈现的位置它存在于 WPF 控件中。传递帧的主要原因是因为我们有一些自定义协议，我们需要发送视频数据，这些协议是使用 C# 编写的，因此在传递视频帧时，我们将它们包装在我们自己的数据包中并通过线路发送出去. 这个解决方案有效，我们可以使用我们的自定义协议流式传输视频，尽管维护和使用它是一场噩梦。

My question is there a better way to go about this? I'm looking for ways to work at a lower level with the video data (in C#) so that I can take the video frames and package them in our own packets and send them out and be able to receive and rebuild the video on the other side. ffmpeg seems to be the common solution but I've run into a lot of issues with it and the GPL/LGPL thing I think is a problem.

我的问题是有更好的方法来解决这个问题吗？我正在寻找在较低级别处理视频数据（在 C# 中）的方法，以便我可以获取视频帧并将它们打包在我们自己的数据包中并发送出去，并且能够接收和重建视频对方。ffmpeg 似乎是常见的解决方案，但我遇到了很多问题，我认为 GPL/LGPL 是一个问题。

The basic flow I'm looking to achieve, video file -> encode -> wrap in packet -> send over wire on protocol X -> get video data from packet -> decode -> render / save to disk

我希望实现的基本流程，视频文件 -> 编码 -> 包裹在数据包中 -> 在协议 X 上通过线路发送 -> 从数据包中获取视频数据 -> 解码 -> 渲染/保存到磁盘

回答by Charlie Salts

What you might try looking at is SharpFFmpeg. It's licensed using the GPL, although you might be able to see how they wrote their wrapper, and you might be able to write your own, or get ideas about how to fix your current solution.

您可能会尝试查看SharpFFmpeg。它是使用 GPL 许可的，尽管您可能能够看到他们如何编写包装器，并且您可能能够编写自己的包装器，或者获得有关如何修复当前解决方案的想法。

Edit:

编辑：

There's a similar wrapper called ffmpeg-sharpover at code.google.com, which uses an LGPL - you can use this in commercial applications. I suspect both of these wrappers do much the same, although SharpFFmpeg is older and probably more mature.

在 code.google.com 上有一个名为ffmpeg-sharp的类似包装器，它使用 LGPL - 您可以在商业应用程序中使用它。我怀疑这两个包装器的作用大致相同，尽管 SharpFFmpeg 更旧并且可能更成熟。

回答by ifatree

I had all kinds of trouble using ffmpeg wrapped into a DLL. My video project was pretty simple - I just needed the converter to take a single thumbnail from a WMV.

我在使用包装成 DLL 的 ffmpeg 时遇到了各种麻烦。我的视频项目非常简单 - 我只需要转换器从 WMV 中获取单个缩略图。

After trying just what you describe, my solution was to just copy the ffmpeg.exe binary into my project as an external library. this also neatly gets around any code licensing issues, AFAIK...

在尝试了您所描述的内容之后，我的解决方案是将 ffmpeg.exe 二进制文件作为外部库复制到我的项目中。这也巧妙地解决了任何代码许可问题，AFAIK ...

        Guid temp = Guid.NewGuid();

        // just throw our ffmpeg commands at cmd.exe
        System.Diagnostics.ProcessStartInfo psi = 
            new System.Diagnostics.ProcessStartInfo("cmd.exe");

        psi.WorkingDirectory = Page.MapPath(@"~\Lib\ffmpeg.rev12665");

        psi.UseShellExecute = false;
        psi.RedirectStandardError = true;
        psi.RedirectStandardOutput = true;
        psi.RedirectStandardInput = true;

        System.Diagnostics.Process ps = System.Diagnostics.Process.Start(psi);

        StreamReader outputReader = ps.StandardOutput;
        StreamReader errorReader = ps.StandardError;
        StreamWriter inputWrite = ps.StandardInput;

        // uses extra cheap logging facility
        inputWrite.WriteLine("echo \"Ripping " + copiedFile + " " + 
            temp.ToString() + "\" >> log.txt");

        inputWrite.WriteLine("ffmpeg.exe -i \"" + copiedFile + 
            "\" -f image2 -vframes 1 -y -ss 2 tmp\" + temp.ToString() + 
            ".jpg");

        inputWrite.WriteLine("exit");

        ps.WaitForExit(3000);

        if (ps.HasExited)
        {
            string thumbFile = Page.MapPath(@"~\Lib\ffmpeg.rev12665\tmp") + 
                @"\" + temp.ToString() + ".jpg";
            // ...
        }

Your ffmpeg command line might vary drastically from my example, but this is the most stable way I've found to get thumbnails out. Other stuff I found online regarding ffmpeg specifically did not have this solution (cmd.exe-based), but this is the only one I've gotten working well. Good luck!

您的 ffmpeg 命令行可能与我的示例大不相同，但这是我发现的最稳定的获取缩略图的方法。我在网上找到的关于 ffmpeg 的其他内容特别没有这个解决方案（基于 cmd.exe），但这是我唯一一个运行良好的解决方案。祝你好运！

回答by Jeremiah Morrill

I wrote the VideoRendererElement back when there were no efficient ways to render video in WPF (v3.0). It uses some hackery to make it work.

当没有有效的方法在 WPF (v3.0) 中渲染视频时，我写回了 VideoRendererElement。它使用了一些hackery来使其工作。

If you want to simplify things a bit, drop the VRE and use the InteropBitmap for rendering (WriteableBitmap is ok, but not as efficient). Also drop SWIG and make your C++ dll a CLI/C++ dll, this way you can directly talk to your C++ from C# (and vice versa).

如果您想稍微简化一下，请删除 VRE 并使用 InteropBitmap 进行渲染（WriteableBitmap 可以，但效率不高）。同时删除 SWIG 并使您的 C++ dll 成为 CLI/C++ dll，这样您就可以直接从 C# 与您的 C++ 对话（反之亦然）。

Another route you can go is to just create a DirectShow source filter that contains your transport/decoding stuff and you can use something like my WPF MediaKitto have it render into WPF (it uses the D3DImage. 0 hacks).

您可以走的另一条路线是创建一个 DirectShow 源过滤器，其中包含您的传输/解码内容，您可以使用我的WPF MediaKit 之类的东西将其渲染到 WPF 中（它使用 D3DImage.0 hacks）。

Also, don't be afraid of LGPL. As long as you keep it in it's own DLL and do not change the source, you are within the license restrictions.

另外，不要害怕 LGPL。只要您将它保存在它自己的 DLL 中并且不更改源代码，您就处于许可限制范围内。

回答by Jeroen-bart Engelen

You could also have a look at the various Microsoft Windows Media SDKs that are available for download. In a project a few years back we used the Windows Media Format SDK to extract thumbnails from uploaded video. These SDKs also have .NET sample code.

您还可以查看可供下载的各种 Microsoft Windows Media SDK。在几年前的一个项目中，我们使用 Windows Media Format SDK 从上传的视频中提取缩略图。这些 SDK 还包含 .NET 示例代码。

回答by A_HREF

In our project we using Microsoft Expression Encoder. It is not free. It can convert videos to different formats and sizes, extract thumbnails, etc.

在我们的项目中，我们使用 Microsoft Expression Encoder。它不是免费的。它可以将视频转换为不同的格式和大小，提取缩略图等。

Here is example:

这是示例：

using Microsoft.Expression.Encoder;

//...
//skiped
//...

MediaItem mediaItem = new MediaItem(videoToEncode.SourceFilePath);
mediaItem.ApplyPreset(PresetFilePath);

Job job = new Job();
job.ApplyPreset(PresetFilePath); // path to preset file, where settings of bit-rate, codec etc
job.MediaItems.Add(mediaItem);

job.EncodeProgress += OnProgress;
job.EncodeCompleted += EncodeCompleted;

job.DefaultMediaOutputFileName = "{OriginalFilename}.encoded.{DefaultExtension}";
job.CreateSubfolder = false;

job.OutputDirectory = videoToEncode.EncodedFilePath;
job.Encode();

回答by schaermu

We are converting video files into various output formats (divx encoded avi, flv, mp4 etc.) for our mediadatabase application. Since we always worked with CLI-applications to do media conversion (talk about rasterizing EPS files to JPG using ImageMagick/GS), we relied heavily on FFMPEG-CLI.

我们正在为我们的媒体数据库应用程序将视频文件转换为各种输出格式（divx 编码的 avi、flv、mp4 等）。由于我们一直使用 CLI 应用程序进行媒体转换（谈论使用 ImageMagick/GS 将 EPS 文件光栅化为 JPG），因此我们非常依赖 FFMPEG-CLI。

In our special environment, we used "dumb" UNIX-servers as conversion machines (there is only sshd, ffmpeg, misc. ffmpeg libraries and samba installed). They are controlled through PuTTy's CLI from C# (WCF webservice) via SSH commands to do the real conversion.

在我们的特殊环境中，我们使用“哑巴”UNIX 服务器作为转换机器（仅安装了 sshd、ffmpeg、misc.ffmpeg 库和 samba）。它们通过来自 C#（WCF webservice）的 PuTTy 的 CLI 通过 SSH 命令进行控制，以进行真正的转换。

The call for ffmpeg happens via ssh and is specialized for each TransformationType. The putty CLI is started through C#'s System.Diagnostics.Process namespace, events for output- and error-messages are handled for logging purposes.

对 ffmpeg 的调用通过 ssh 发生，并且专门针对每个 TransformationType。Putty CLI 通过 C# 的 System.Diagnostics.Process 命名空间启动，处理输出和错误消息的事件以进行记录。

The internet provides alot of resources concerning questions like "How can i convert mpg to flv using ffmpeg?", a little bit of research will help you out. Since we are talking about a copyright'd application, i cannot post complete code excerpts. But it should give you an architectural idea about a reliable, fast video encoding backend using C#.

互联网提供了很多关于“如何使用 ffmpeg 将 mpg 转换为 flv？”等问题的资源，一些研究将帮助您解决问题。由于我们谈论的是受版权保护的应用程序，因此我无法发布完整的代码摘录。但它应该为您提供有关使用 C# 的可靠、快速视频编码后端的架构理念。

回答by PatriceVB

DirectShow is your friend. DirectShow is the low level layer used by most of the windows "multimedia" applications like Media Player, Audio Encoders, and so on.

DirectShow 是您的朋友。DirectShow 是大多数窗口“多媒体”应用程序（如媒体播放器、音频编码器等）使用的低级层。

Even if this library has been made for native developers you can access it from the managed world thanks to DirectShow.net. http://directshownet.sourceforge.netThis is a well known and stable managed wrapper for DirectShow.

即使这个库是为本地开发人员制作的，您也可以通过 DirectShow.net 从托管世界访问它。http://directshownet.sourceforge.net这是一个众所周知且稳定的 DirectShow 托管包装器。

The only thing you have to do is to learn a little bit DirectShow to understand the concept of graphes and filters and then to create your own filters and graphes to use the power of DirectShow !

您唯一需要做的就是学习一点 DirectShow 以了解图形和过滤器的概念，然后创建您自己的过滤器和图形以使用 DirectShow 的强大功能！