在本教程中，我们将把OpenCV集成到我们的Android应用程序中。
我们已经在这里讨论了CameraX的基础知识。
因此，今天，我们将了解如何在实时摄像头供稿上运行OpenCV图像处理。

Android CameraX

Android Jetpack随附的CameraX都是关于用例的。
基本上，可以使用三个核心用例来构建摄像机：预览，分析和捕获。

之前已经完成了预览和捕获，今天我们的主要重点是分析。

因此，我们将使用OpenCV分析帧并进行实时处理。

什么是OpenCV？

OpenCV是一个计算机视觉库，其中包含2000多种与图像处理有关的算法。
它是用C ++编写的。

幸运的是，OpenCV中的许多高级内容都可以用Java完成。
此外，我们总是可以使用JNI接口在Java和C ++之间进行通信。

我们可以从其官方GitHub存储库下载并导入OpenCV SDK。
这是一个很大的模块。
由于大小和项目空间的限制，我们将使用如下所示的Gradle依赖库：

implementation 'com.quickbirdstudios:opencv:3.4.1'

你知道吗？

使用OpenCV模块的Android应用程序具有较大的APK大小。

在以下使用CameraX和OpenCV的部分中，我们将转换色彩空间以使摄像机具有完全不同的外观。

Android OpenCV项目结构

Android Camerax Opencv项目结构

Android OpenCV代码

下面给出了activity_main.xml布局的代码：

<?xml version="1.0" encoding="utf-8"?>
<androidx.constraintlayout.widget.ConstraintLayout xmlns:android="https://schemas.android.com/apk/res/android"
  xmlns:app="https://schemas.android.com/apk/res-auto"
  android:layout_width="match_parent"
  android:layout_height="match_parent">

  <TextureView
      android:id="@+id/textureView"
      android:layout_width="match_parent"
      android:layout_height="wrap_content"
      android:visibility="visible"
      app:layout_constraintBottom_toBottomOf="parent"
      app:layout_constraintLeft_toLeftOf="parent"
      app:layout_constraintRight_toRightOf="parent"
      app:layout_constraintTop_toTopOf="parent" 

  <ImageView
      android:id="@+id/ivBitmap"
      android:layout_width="match_parent"
      android:layout_height="match_parent"
      android:layout_gravity="center"
      app:layout_constraintBottom_toBottomOf="parent"
      app:layout_constraintLeft_toLeftOf="parent"
      app:layout_constraintRight_toRightOf="parent"
      app:layout_constraintTop_toTopOf="parent" 

  <com.google.android.material.floatingactionbutton.FloatingActionButton
      android:id="@+id/btnCapture"
      android:layout_width="wrap_content"
      android:layout_height="wrap_content"
      android:layout_alignParentBottom="true"
      android:layout_centerHorizontal="true"
      android:layout_marginBottom="24dp"
      app:backgroundTint="@android:color/black"
      android:src="@drawable/ic_camera"
      app:layout_constraintBottom_toBottomOf="parent"
      app:layout_constraintEnd_toEndOf="parent"
      app:layout_constraintStart_toStartOf="parent" 

  <LinearLayout
      android:layout_width="match_parent"
      android:layout_height="wrap_content"
      android:layout_marginBottom="24dp"
      android:orientation="horizontal"
      android:weightSum="2"
      android:visibility="gone"
      android:id="@+id/llBottom"
      android:gravity="center"
      app:layout_constraintBottom_toBottomOf="parent"
      app:layout_constraintEnd_toEndOf="parent"
      app:layout_constraintStart_toStartOf="parent">

      <com.google.android.material.floatingactionbutton.FloatingActionButton
          android:id="@+id/btnReject"
          android:layout_width="0dp"
          android:layout_height="wrap_content"
          android:layout_weight="0.5"
          app:backgroundTint="@color/rejectedRed"
          android:src="@drawable/ic_close"
          app:layout_constraintBottom_toBottomOf="parent"
          app:layout_constraintEnd_toEndOf="parent"
          app:layout_constraintStart_toStartOf="parent" 

      <com.google.android.material.floatingactionbutton.FloatingActionButton
          android:id="@+id/btnAccept"
          android:layout_width="wrap_content"
          android:layout_height="wrap_content"
          android:layout_weight="0.5"
          android:layout_marginStart="20dp"
          android:src="@drawable/ic_check"
          app:backgroundTint="@color/acceptedGreen"
          app:layout_constraintBottom_toBottomOf="parent"
          app:layout_constraintEnd_toEndOf="parent"
          app:layout_constraintStart_toStartOf="parent" 

  </LinearLayout>

</androidx.constraintlayout.widget.ConstraintLayout>

下面给出了MainActivity.java的代码：

package com.theitroad.androidcameraxopencv;

import androidx.annotation.NonNull;
import androidx.annotation.Nullable;
import androidx.appcompat.app.AppCompatActivity;
import androidx.camera.core.CameraX;
import androidx.camera.core.ImageAnalysis;
import androidx.camera.core.ImageAnalysisConfig;
import androidx.camera.core.ImageCapture;
import androidx.camera.core.ImageCaptureConfig;
import androidx.camera.core.ImageProxy;
import androidx.camera.core.Preview;
import androidx.camera.core.PreviewConfig;
import androidx.core.app.ActivityCompat;
import androidx.core.content.ContextCompat;
import android.content.pm.PackageManager;
import android.graphics.Bitmap;
import android.graphics.Matrix;
import android.os.Bundle;
import android.os.Environment;
import android.os.Handler;
import android.os.HandlerThread;
import android.util.Log;
import android.util.Rational;
import android.util.Size;
import android.view.Menu;
import android.view.MenuItem;
import android.view.Surface;
import android.view.TextureView;
import android.view.View;
import android.view.ViewGroup;
import android.widget.ImageView;
import android.widget.LinearLayout;
import android.widget.Toast;
import com.google.android.material.floatingactionbutton.FloatingActionButton;
import org.opencv.android.OpenCVLoader;
import org.opencv.android.Utils;
import org.opencv.core.Mat;
import org.opencv.imgproc.Imgproc;

import java.io.File;

public class MainActivity extends AppCompatActivity implements View.OnClickListener {

  private int REQUEST_CODE_PERMISSIONS = 101;
  private final String[] REQUIRED_PERMISSIONS = new String[]{"android.permission.CAMERA", "android.permission.WRITE_EXTERNAL_STORAGE"};
  TextureView textureView;
  ImageView ivBitmap;
  LinearLayout llBottom;

  int currentImageType = Imgproc.COLOR_RGB2GRAY;

  ImageCapture imageCapture;
  ImageAnalysis imageAnalysis;
  Preview preview;

  FloatingActionButton btnCapture, btnOk, btnCancel;

  static {
      if (!OpenCVLoader.initDebug())
          Log.d("ERROR", "Unable to load OpenCV");
      else
          Log.d("SUCCESS", "OpenCV loaded");
  }

  @Override
  protected void onCreate(Bundle savedInstanceState) {
      super.onCreate(savedInstanceState);
      setContentView(R.layout.activity_main);

      btnCapture = findViewById(R.id.btnCapture);
      btnOk = findViewById(R.id.btnAccept);
      btnCancel = findViewById(R.id.btnReject);

      btnOk.setOnClickListener(this);
      btnCancel.setOnClickListener(this);

      llBottom = findViewById(R.id.llBottom);
      textureView = findViewById(R.id.textureView);
      ivBitmap = findViewById(R.id.ivBitmap);

      if (allPermissionsGranted()) {
          startCamera();
      } else {
          ActivityCompat.requestPermissions(this, REQUIRED_PERMISSIONS, REQUEST_CODE_PERMISSIONS);
      }
  }

  private void startCamera() {

      CameraX.unbindAll();
      preview = setPreview();
      imageCapture = setImageCapture();
      imageAnalysis = setImageAnalysis();

      //bind to lifecycle:
      CameraX.bindToLifecycle(this, preview, imageCapture, imageAnalysis);
  }

  private Preview setPreview() {

      Rational aspectRatio = new Rational(textureView.getWidth(), textureView.getHeight());
      Size screen = new Size(textureView.getWidth(), textureView.getHeight()); //size of the screen

      PreviewConfig pConfig = new PreviewConfig.Builder().setTargetAspectRatio(aspectRatio).setTargetResolution(screen).build();
      Preview preview = new Preview(pConfig);

      preview.setOnPreviewOutputUpdateListener(
              new Preview.OnPreviewOutputUpdateListener() {
                  @Override
                  public void onUpdated(Preview.PreviewOutput output) {
                      ViewGroup parent = (ViewGroup) textureView.getParent();
                      parent.removeView(textureView);
                      parent.addView(textureView, 0);

                      textureView.setSurfaceTexture(output.getSurfaceTexture());
                      updateTransform();
                  }
              });

      return preview;
  }

  private ImageCapture setImageCapture() {
      ImageCaptureConfig imageCaptureConfig = new ImageCaptureConfig.Builder().setCaptureMode(ImageCapture.CaptureMode.MIN_LATENCY)
              .setTargetRotation(getWindowManager().getDefaultDisplay().getRotation()).build();
      final ImageCapture imgCapture = new ImageCapture(imageCaptureConfig);

      btnCapture.setOnClickListener(new View.OnClickListener() {

          @Override
          public void onClick(View v) {

              imgCapture.takePicture(new ImageCapture.OnImageCapturedListener() {
                  @Override
                  public void onCaptureSuccess(ImageProxy image, int rotationDegrees) {
                      Bitmap bitmap = textureView.getBitmap();
                      showAcceptedRejectedButton(true);
                      ivBitmap.setImageBitmap(bitmap);
                  }

                  @Override
                  public void onError(ImageCapture.UseCaseError useCaseError, String message, @Nullable Throwable cause) {
                      super.onError(useCaseError, message, cause);
                  }
              });

              /*File file = new File(
                      Environment.getExternalStoragePublicDirectory(Environment.DIRECTORY_PICTURES), "" + System.currentTimeMillis() + "_JDCameraX.jpg");
              imgCapture.takePicture(file, new ImageCapture.OnImageSavedListener() {
                  @Override
                  public void onImageSaved(@NonNull File file) {
                      Bitmap bitmap = textureView.getBitmap();
                      showAcceptedRejectedButton(true);
                      ivBitmap.setImageBitmap(bitmap);
                  }

                  @Override
                  public void onError(@NonNull ImageCapture.UseCaseError useCaseError, @NonNull String message, @Nullable Throwable cause) {

                  }
              });*/
          }
      });

      return imgCapture;
  }

  private ImageAnalysis setImageAnalysis() {

      //Setup image analysis pipeline that computes average pixel luminance
      HandlerThread analyzerThread = new HandlerThread("OpenCVAnalysis");
      analyzerThread.start();

      ImageAnalysisConfig imageAnalysisConfig = new ImageAnalysisConfig.Builder()
              .setImageReaderMode(ImageAnalysis.ImageReaderMode.ACQUIRE_LATEST_IMAGE)
              .setCallbackHandler(new Handler(analyzerThread.getLooper()))
              .setImageQueueDepth(1).build();

      ImageAnalysis imageAnalysis = new ImageAnalysis(imageAnalysisConfig);

      imageAnalysis.setAnalyzer(
              new ImageAnalysis.Analyzer() {
                  @Override
                  public void analyze(ImageProxy image, int rotationDegrees) {
                      //Analyzing live camera feed begins.

                      final Bitmap bitmap = textureView.getBitmap();

                      if(bitmap==null)
                          return;

                      Mat mat = new Mat();
                      Utils.bitmapToMat(bitmap, mat);

                      Imgproc.cvtColor(mat, mat, currentImageType);
                      Utils.matToBitmap(mat, bitmap);
                      runOnUiThread(new Runnable() {
                          @Override
                          public void run() {
                              ivBitmap.setImageBitmap(bitmap);
                          }
                      });

                  }
              });

      return imageAnalysis;

  }

  private void showAcceptedRejectedButton(boolean acceptedRejected) {
      if (acceptedRejected) {
          CameraX.unbind(preview, imageAnalysis);
          llBottom.setVisibility(View.VISIBLE);
          btnCapture.hide();
          textureView.setVisibility(View.GONE);
      } else {
          btnCapture.show();
          llBottom.setVisibility(View.GONE);
          textureView.setVisibility(View.VISIBLE);
          textureView.post(new Runnable() {
              @Override
              public void run() {
                  startCamera();
              }
          });
      }
  }

  private void updateTransform() {
      Matrix mx = new Matrix();
      float w = textureView.getMeasuredWidth();
      float h = textureView.getMeasuredHeight();

      float cX = w/2f;
      float cY = h/2f;

      int rotationDgr;
      int rotation = (int) textureView.getRotation();

      switch (rotation) {
          case Surface.ROTATION_0:
              rotationDgr = 0;
              break;
          case Surface.ROTATION_90:
              rotationDgr = 90;
              break;
          case Surface.ROTATION_180:
              rotationDgr = 180;
              break;
          case Surface.ROTATION_270:
              rotationDgr = 270;
              break;
          default:
              return;
      }

      mx.postRotate((float) rotationDgr, cX, cY);
      textureView.setTransform(mx);
  }

  @Override
  public void onRequestPermissionsResult(int requestCode, @NonNull String[] permissions, @NonNull int[] grantResults) {

      if (requestCode == REQUEST_CODE_PERMISSIONS) {
          if (allPermissionsGranted()) {
              startCamera();
          } else {
              Toast.makeText(this, "Permissions not granted by the user.", Toast.LENGTH_SHORT).show();
              finish();
          }
      }
  }

  private boolean allPermissionsGranted() {

      for (String permission: REQUIRED_PERMISSIONS) {
          if (ContextCompat.checkSelfPermission(this, permission) != PackageManager.PERMISSION_GRANTED) {
              return false;
          }
      }
      return true;
  }

  @Override
  public boolean onCreateOptionsMenu(Menu menu) {
      getMenuInflater().inflate(R.menu.menu_main, menu);
      return true;
  }

  @Override
  public boolean onOptionsItemSelected(MenuItem item) {

      switch (item.getItemId()) {
          case R.id.black_white:
              currentImageType = Imgproc.COLOR_RGB2GRAY;
              startCamera();
              return true;

          case R.id.hsv:
              currentImageType = Imgproc.COLOR_RGB2HSV;
              startCamera();
              return true;

          case R.id.lab:
              currentImageType = Imgproc.COLOR_RGB2Lab;
              startCamera();
              return true;
      }

      return super.onOptionsItemSelected(item);
  }

  @Override
  public void onClick(View v) {
      switch (v.getId()) {
          case R.id.btnReject:
              showAcceptedRejectedButton(false);
              break;

          case R.id.btnAccept:
              File file = new File(
                      Environment.getExternalStoragePublicDirectory(Environment.DIRECTORY_PICTURES), "" + System.currentTimeMillis() + "_JDCameraX.jpg");
              imageCapture.takePicture(file, new ImageCapture.OnImageSavedListener() {
                  @Override
                  public void onImageSaved(@NonNull File file) {
                      showAcceptedRejectedButton(false);

                      Toast.makeText(getApplicationContext(), "Image saved successfully in Pictures Folder", Toast.LENGTH_LONG).show();
                  }

                  @Override
                  public void onError(@NonNull ImageCapture.UseCaseError useCaseError, @NonNull String message, @Nullable Throwable cause) {

                  }
              });
              break;
      }
  }
}

在上面的代码中，在"图像分析"用例内部，我们从TextureView中检索位图。

Utils.bitmapToMat用于将位图转换为Mat对象。
此方法是OpenCV android的一部分。

Mat类基本上用于保存图像。
它由矩阵标题和指向包含像素值的矩阵的指针组成。

在图像分析中，我们使用" ImgProc.cvtColor"将垫子颜色空间从一种类型转换为另一种类型。

将垫子转换为其他颜色空间后，我们再将其转换为位图，并在ImageView中显示在屏幕上。

默认情况下，图像为RGB类型。
使用菜单选项，我们可以将图像转换为GREY，LAB，HSV类型。