在 C# 中检测相机图像中的对象

Detect an object in a camera image in C#

提问by Jon Grant

I have an image, taken from a live webcam, and I want to be able to detect a specific object in the image and extract that portion of it to do some further processing.

我有一张从实时网络摄像头拍摄的图像，我希望能够检测图像中的特定对象并提取其中的那部分以进行进一步处理。

Specifically, the image would be of a game board, let's say for the purposes of this question that it's a Sudoku game board.

具体来说，图像将是一个游戏板，就这个问题而言，假设它是一个数独游戏板。

My initial approach was to look for contrasting areas and work it out from there, but I seem to end up with a lot of potential edges (many erroneous) and no real clue as to how to work out which ones are the ones I actually want!

我最初的方法是寻找对比区域并从那里解决，但我似乎最终得到了很多潜在的边缘（许多错误）并且没有真正的线索来确定哪些是我真正想要的！

Are there any algorithms, libraries, code samples, or even just bright ideas out there, as to how I would go about finding and extracting the relevant part of the image?

关于我将如何查找和提取图像的相关部分，是否有任何算法、库、代码示例，甚至只是一些聪明的想法？

采纳答案by Mladen Prajdic

use the free AForge.Netimage processing library for this. there's a ton of cool stuff to play with.

为此使用免费的AForge.Net图像处理库。有很多很酷的东西可以玩。

回答by Sun

You need to perform filters operation and masks on image.

您需要对图像执行过滤器操作和蒙版。

I think there are no simple waysto just fetch object from the image, you need to use edge-detection algorithms, clipping, and set the criteria for valid objects/image.

我认为没有简单的方法可以从图像中获取对象，您需要使用边缘检测算法、裁剪并设置有效对象/图像的标准。

You can also use image thresholding to detect object. You may want to look at below Image processing library.

您还可以使用图像阈值来检测对象。您可能需要查看下面的图像处理库。

1. FiltersAPI for C, C++, C#, Visual Basic .NET, Delphi, Python
2. http://www.catenary.com/
3. CIMGricher than above library however it is written in C++

1. 适用于 C、C++、C#、Visual Basic .NET、Delphi、Python 的过滤器API
2. http://www.catenary.com/
3. CIMG比上面的库更丰富，但它是用 C++ 编写的

回答by sabiland

One of the (I guess many possible) approaches:

其中一种（我想有很多可能的）方法：

1. Find a filter that "gets/calculates" straight lines (edges, etc.) from a given image.

2. Now you have the collection (array) of all the lines (xStart,yStart & xEnd,yEnd). You can easily calculate all the line-lengths from the coordinates.

3. Now, considering that you can always (!) expect "one-biggest-square / rectangle" inside the image, it would be quite easy to find and calculate the wanted-sudoku-rectangle region and crop it from the image to do some further processing.

1. 找到一个可以从给定图像“获取/计算”直线（边缘等）的过滤器。

2. 现在您拥有所有行 (xStart,yStart & xEnd,yEnd) 的集合（数组）。您可以轻松地从坐标计算所有线长。

3. 现在，考虑到您总是可以（！）期望图像内有“最大的正方形/矩形”，找到并计算想要的数独矩形区域并从图像中裁剪它以进一步做一些事情会很容易加工。

EDIT: Solving/programming that kind of problems is always challenging BUT really interesting at the same time :).

编辑：解决/编程这类问题总是很有挑战性，但同时也很有趣:)。

回答by pufferfish

You could try first to find the bold line intersections and use them as registration marks.

您可以先尝试找到粗线交叉点并将它们用作注册标记。

This would be a good start because:

这将是一个好的开始，因为：

• They're pretty uniformly shaped
• You know how many there are
• You know where (roughly) they should be in relation to each other
• Can tolerate scale variations

• 它们的形状非常均匀
• 你知道有多少
• 你知道（大致）他们应该在哪里
• 可以容忍规模变化

So:

所以：

1. Apply an edge filter
2. Scan a mask* of what the ideal +should look like across the image, recording all that are a good match
3. Choose the set that matches your expectations best, according to location relative to one another
4. You now also know where the numbers should be, so you can easily extract them.

1. 应用边缘过滤器
2. 扫描图像中理想+应该是什么样子的面具* ，记录所有匹配良好的内容
3. 根据彼此的位置，选择最符合您期望的集合
4. 您现在还知道数字应该在哪里，因此您可以轻松提取它们。

* A more sophisticated solution would be to use a Neural Net instead of a mask to recognise the intersections. This might be worth it since your're probably going to use one for the OCR of the numbers.

* 更复杂的解决方案是使用神经网络而不是掩码来识别交叉点。这可能是值得的，因为您可能会使用一个用于数字的 OCR。

回答by plinth

You might try using the Hough Transform.

您可以尝试使用Hough Transform。

回答by Hannes Ovrén

I would start by using a corner detector (The Harris detector works nice) to find the intersections and corners of the sudoku grid.

我将首先使用角点检测器（Harris 检测器效果很好）来找到数独网格的交点和角点。

Then I would use those points to do an image rectification to transform the image to have the grid as rectangular as possible. Now you should have no trouble finding each square to do OCR.

然后我将使用这些点进行图像校正以将图像转换为尽可能矩形的网格。现在您应该可以轻松找到每个方块来进行 OCR。

Image rectification is not simple and entails quite a lot of math.

图像校正并不简单，需要大量的数学运算。

Be prepared to do some reading :)

准备做一些阅读:)

If the images of the game boards are already close to rectangular you can of course skip the rectification part and directly use the corner points to find your squares for OCR.

如果游戏板的图像已经接近矩形，您当然可以跳过校正部分，直接使用角点找到 OCR 的正方形。

A lot of people have been suggesting to use Neural Networks. I am quite certain that throwing a neural network on this problem is totally unneccessary. NNs are (sometimes) good if you need to classify objects where the definition of the object is vague. "Find cars in image" is a problem which could have use for a Neural Network since cars can look very different but have some features the same. Thus, given enough data, you can train your NN to detect cars. In this problem you have something that is very regular and always looks almost the same, so a NN will not make anything easier or better.

很多人一直建议使用神经网络。我很确定在这个问题上投入神经网络是完全没有必要的。如果您需要对对象定义模糊的对象进行分类，则 NN（有时）非常有用。“在图像中查找汽车”是一个可以用于神经网络的问题，因为汽车可能看起来非常不同但具有相同的某些功能。因此，如果有足够的数据，您就可以训练您的神经网络来检测汽车。在这个问题中，你有一些非常规则的东西，而且看起来总是几乎一样，所以神经网络不会让任何事情变得更容易或更好。

回答by MSalters

Without rejecting any of the other ideas, step 1 really should be the detection of the image rotation. You can do this by determining the local gradient at each point and creating a histogram thereof. This will have 4 major components at 90 degree offsets. Ideally, these would be 0, 90, 180 and 270 but if they're not you should rotate your image. E.g. in the sample image you should start with a rotation over about 8 degrees CW.

在不拒绝任何其他想法的情况下，步骤 1 确实应该是检测图像旋转。您可以通过确定每个点的局部梯度并创建其直方图来做到这一点。这将具有 90 度偏移的 4 个主要组件。理想情况下，这些将是 0、90、180 和 270，但如果不是，您应该旋转图像。例如，在示例图像中，您应该从顺时针旋转约 8 度开始。

回答by xfill

Use aforge colorfiltering

使用aforge 颜色过滤

There are many filtering method provided for c#, mainly I prefer aforge filters, for this purpose they have few filters, they are

c# 提供了很多过滤方法，主要我更喜欢 aforge 过滤器，为此他们有很少的过滤器，它们是

* ColorFiltering
* ChannelFiltering
* HSLFiltering
* YCbCrFiltering
* EuclideanColorFiltering

See here

看这里

回答by Matthias

You should google for CamShiftor Blob trackingor Particle filters. They are all usefull for your problem. And most of them are shipped with OpenCV and it's C# wrapper AForge.NET. You will find some nice demos on Youtube showing how they work.

您应该在谷歌上搜索CamShift或Blob 跟踪或粒子过滤器。它们对您的问题都很有用。它们中的大多数都随 OpenCV 一起提供，它是 C# 包装器 AForge.NET。你会在 Youtube 上找到一些很好的演示，展示它们是如何工作的。

Good luck

祝你好运

回答by dajuric

Take a look at: https://github.com/dajuric/accord-net-extensions

看一看：https: //github.com/dajuric/accord-net-extensions

The library "joins" the free AForge.NET and Accord.NET library and adds image-processing and object tracking-algorithms. Samples included :)

该库“加入”了免费的 AForge.NET 和 Accord.NET 库，并添加了图像处理和对象跟踪算法。包括样品:)