jetpack使用_使用Jetpack CameraX

jetpack使用

As a matter of fact, CameraX is a Android Jetpack support library, built to help you make camera app development much more easier. This means it supports a consistent and easy-to-use API surface, which works across most Android devices with having backward-compatibility to Android 5.0 or API level 21. This essay aims to discuss some main issues in using Jetpack CameraX and the new features that have been released recently by Google for Android developers.

实际上，CameraX是一个Android Jetpack支持库，旨在帮助您简化相机应用程序的开发。 这意味着它支持一致且易于使用的API表面，该表面适用于大多数Android设备，并且具有向后兼容Android 5.0或API级别21的功能。本文旨在讨论使用Jetpack CameraX的一些主要问题和新功能Google最近针对Android开发人员发布了这些文件。

简介与概述 (Introduction and Overview)

Basically, CameraX is a Jetpack support library to make camera app development easier. Currently, it is in Beta version. The CameraX API is built on top of Camera2 API, and provides its capabilities. However, it uses a simpler use-cases-based approach that is lifecycle aware. Above all, this concise API results in an average of 70% fewer lines of code than the Camera2 API. As developers, we know writing a camera app could not be an easy task. First, you have to deal with Camera2 API, which is powerful and flexible; however, it is a challenging process to do. Second, given the large number of Android devices in the market, it is time consuming to achieve a consistent experience across all the devices types. In other words, often this needs a long running if/else section within your code. Eventually, if you are publishing your app on the Google Play, testing on a real device is a must, and testing across a large number of devices can be both costly and time-consuming as your code may require device-specific solutions to allow for a consistent experience.

基本上，CameraX是Jetpack支持库，可简化相机应用程序的开发。 当前，它是Beta版本。 CameraX API建立在Camera2 API之上，并提供其功能。 但是，它使用了一种基于生命周期的更简单的基于用例的方法。 最重要的是，与API 2相比，该简洁的API平均减少了70％的代码行。 作为开发人员，我们知道编写相机应用程序并非易事。 首先，您必须处理功能强大且灵活的Camera2 API； 但是，这是一个具有挑战性的过程。 其次，鉴于市场上有大量的Android设备，要在所有设备类型上获得一致的体验是很费时间的。 换句话说，这通常需要长时间在代码中运行if / else部分。 最终，如果您要在Google Play上发布应用，则必须在真实设备上进行测试，并且在大量设备上进行测试可能既昂贵又耗时，因为您的代码可能需要特定于设备的解决方案才能实现一致的体验。

Now, Google has tried to tackle and solve these issues by introducing CameraX as an efficient method to diminish some major obstacles to use camera in Android apps as follows:

现在，Google尝试通过引入CameraX作为解决此类问题的有效方法，以减少在Android应用中使用相机的一些主要障碍，如下所示：

1. Easier to implement and understand by reducing the total amount of codes you must write in your app.
   通过减少必须在应用程序中编写的代码总数，可以更轻松地实现和理解。

2. Consistent behavior across 94% of Android devices starting from API 21.

2.从API 21开始，94％的Android设备上的行为一致。

3. Google conducts 24/7 on-device testing across a wide range of devices and API levels.

3. Google对各种设备和API级别进行24/7的设备上测试。

用例 (Use cases)

Fundamentally, CameraX introduces use cases, which allow you to focus on the task you require to accomplish instead of spending time handling device-specific nuances. There are main use cases as follows:

从根本上讲， CameraX引入了用例，这些用例使您可以专注于所需完成的任务，而不必花费时间来处理特定于设备的细微差别。 主要用例如下：

1. Preview: get an image on the display.

   预览 ：在显示器上显示图像。

2. Image analysis: access a buffer seamlessly for use in your algorithms.

   图像分析 ：无缝访问缓冲区以在算法中使用。

3. Image capture: save high-quality images.

   图像捕获 ：保存高质量图像。

However, Video Capture has not officially provided yet. In addition, these use cases work across all devices running Android 5.0 (API level 21) or higher.

但是，Video Capture尚未正式提供。 此外，这些用例适用于所有运行Android 5.0(API级别21)或更高版本的设备。

Preview

预习

In fact, when you want to add a preview to your app, you must use PreviewView, which is a view that can be cropped, scaled, and rotated for proper display. The image preview streams to a surface inside the PreviewView when the camera becomes active. PreviewView is a very useful and easy UI widget for Android application to show a preview. It needs no transform. Also, PreviewView makes your app much more power efficient and low latency. It does this by using SurfaceView under the hood. It also supports scaleType. Thus, you can choose how to scale a preview instead a PreviewView. Finally, it makes it very easy to implement a tap-to-focus.

实际上，当您想向应用程序添加预览时，必须使用PreviewView ，该视图可以被裁剪，缩放和旋转以正确显示。 相机启用后，图像预览会流到PreviewView内部的表面。 PreviewView是一个非常有用且易于使用的UI小部件，可用于Android应用程序以显示预览。 它不需要变换。 另外，PreviewView使您的应用程序更加节能，低延迟。 它通过在后台使用SurfaceView来完成此操作。 它还支持scaleType 。 因此，您可以选择如何缩放预览而不是PreviewView。 最后，它使实现轻按聚焦变得非常容易。

So, to implement Preview by using PreviewView, you should follow these steps in your code:

因此，要通过使用PreviewView来实现Preview，您应该在代码中遵循以下步骤：

1. Optionally configure a CameraXConfig.Provider.

   (可选)配置CameraXConfig.Provider 。

2. Add a PreviewView to your layout.

   将PreviewView添加到您的布局。

3. Request a CameraProvider.

   请求一个CameraProvider 。

4. On View creation, check for the CameraProvider.

   在创建View ，检查CameraProvider 。

5. Select a camera and bind the lifecycle and use cases.
   选择一个摄像机并绑定生命周期和用例。

For instance, the following code shows a sample code for the last step as follows:

例如，以下代码显示了最后一步的示例代码，如下所示：

fun bindPreview(cameraProvider : ProcessCameraProvider) {
    var preview : Preview = Preview.Builder()
            .build()
    var cameraSelector : CameraSelector = CameraSelector.Builder()
          .requireLensFacing(CameraSelector.LENS_FACING_BACK)
          .build()
    preview.setSurfaceProvider(previewView.createSurfaceProvider())
    var camera = cameraProvider.bindToLifecycle(this as LifecycleOwner, cameraSelector, preview)
}

Image analysis

图像分析

The main goal of image analysis is to make it super easy to integrate libraries that are great at analyzing things, such as TensorFlow Lite, ML Kit, or your own image library.

图像分析的主要目标是使集成非常擅长分析事物的库变得非常容易，例如TensorFlow Lite，ML Kit或您自己的图像库。

The image analysis use case provides your app with a CPU-accessible image to perform image processing, computer vision, or machine learning inference on. The application implements an analyze method that is run on each frame.

图像分析用例为您的应用提供了CPU可访问的图像，以执行图像处理，计算机视觉或机器学习推断。 该应用程序实现在每个帧上运行的分析方法。

Initially, images are processed by passing an executor in which the image analysis is run and an ImageAnalysis.Analyzer parameter to the setAnalyzer() method. Image analysis can work in two ways: blocking and non-blocking. First, Blocking mode is enabled by calling setBackpressureStrategy() with STRATEGY_BLOCK_PRODUCER. In this mode, the executor receives frames from the camera in sequential order. Second, Non-blocking mode is enabled by calling setBackpressureStrategy() with STRATEGY_KEEP_ONLY_LATEST. In this mode, the executor receives the last available frame from the camera at the time that the analyze() method is called.

最初，通过将执行图像分析的执行程序和ImageAnalysis.Analyzer参数传递给setAnalyzer()方法来处理图像。 图像分析可以通过两种方式工作：阻塞和非阻塞。 首先，通过使用STRATEGY_BLOCK_PRODUCER调用setBackpressureStrategy()启用阻止模式。 在这种模式下，执行器按顺序从相机接收帧。 其次，通过使用STRATEGY_KEEP_ONLY_LATEST调用setBackpressureStrategy()来启用非阻塞模式。 在这种模式下，执行程序在调用analyze()方法时从相机接收到最后一个可用的帧。

For instance:

例如：

val imageAnalysis = ImageAnalysis.Builder()
    .setTargetResolution(Size(1280, 720))
    .setBackpressureStrategy(ImageAnalysis.STRATEGY_KEEP_ONLY_LATEST)
    .build()
imageAnalysis.setAnalyzer(executor, ImageAnalysis.Analyzer { image ->
    val rotationDegrees = image.imageInfo.rotationDegrees
    // insert your code here.
})
cameraProvider.bindToLifecycle(this as LifecycleOwner, cameraSelector, imageAnalysis, preview)

The power of image analysis is not so obvious, because just provide frame buffer access. However, when we want to combine it with state-of-the-art Machine Learning libraries, it suddenly becomes a super powerful tool. It allows the application to do something very interesting, things like text recognition, image labeling, or detect your own object using your own trained model.

图像分析的功能不是很明显，因为仅提供帧缓冲区访问。 但是，当我们希望将其与最新的机器学习库结合使用时，它突然变成了超级强大的工具。 它允许应用程序做一些非常有趣的事情，例如文本识别，图像标记或使用您自己训练有素的模型来检测您自己的对象。

Image capture

影像撷取

The image capture use case is designed for capturing high-resolution, high-quality pictures and supports auto-white-balance, auto-exposure, and auto-focus functionality. In fact, the caller is responsible for deciding how to use the captured picture, including the following choices:

图像捕获用例旨在捕获高分辨率，高质量的图片，并支持自动白平衡，自动曝光和自动对焦功能。 实际上，呼叫者负责决定如何使用捕获的图片，包括以下选择：

1. takePicture(Executor, OnImageCapturedCallback): This method provides an in-memory buffer of the captured image.

   takePicture(Executor, OnImageCapturedCallback) ：此方法提供捕获图像的内存缓冲区。

2. takePicture(OutputFileOptions, Executor, OnImageSavedCallback): This method saves the captured image to the provided file location.

2. takePicture(OutputFileOptions, Executor, OnImageSavedCallback) ：此方法将捕获的图像保存到提供的文件位置。

Basic controls for taking pictures are supported in this use case. After configuring your camera, the following code could be used for taking a picture based on events:

在此用例中支持拍照的基本控件。 配置相机后，以下代码可用于基于事件拍摄照片：

fun onClick() {
    val outputFileOptions = ImageCapture.OutputFileOptions.Builder(File(...)).build()
    imageCapture.takePicture(outputFileOptions, cameraExecutor,
        object : ImageCapture.OnImageSavedCallback {
            override fun onError(error: ImageCaptureException)
            {
                // Your code....
            }
            override fun onImageSaved(outputFileResults: ImageCapture.OutputFileResults) {
                // Your code....
            }
        })
}

2020年上半年的新功能 (The new features in the first half of 2020)

1. Enhanced device compatibility with Beta

   与Beta增强的设备兼容性

Recently, Google has made a lot of efforts into fixing device issues, and they has enhanced test lab infrastructure in terms of scale and the test coverage. Ever since CameraX reached beta, Google has been focusing on the stability and the device compatibility. Therefore, they have fixed a number of device specific issues by feedback from developers such as wrong preview aspect ratio, fail to switch camera, or tap-to-focus not working. For example:

最近，Google在解决设备问题方面做出了很多努力，并且在规模和测试范围方面增强了测试实验室的基础结构。 自CameraX进入测试版以来，Google一直致力于稳定性和设备兼容性。 因此，他们通过开发人员的反馈解决了许多特定于设备的问题，例如错误的预览宽高比，无法切换相机或轻按对焦。 例如：

1. Wrong preview aspect ratio LG G3
   预览宽高比不正确LG G3

2. Failed to switch camera Pixel 2

2.无法切换相机Pixel 2

3. Tap-to-focus not working Samsung S7

3.点按对焦不起作用三星S7

Also, they have improved their rotation, UI widget integration, and aspect ratio test.

此外，他们还改善了旋转方式，UI小部件集成和纵横比测试。

2) Preview: PreviewView new APIs, OpenGL sample

2)预览：PreviewView新的API，OpenGL示例

As a result, for the Preview use case, they added more API to the PreviewView in order to satisfy developers’ needs. Furthermore, for the needs of advanced developers, they added OpenGL sample. In short, this is a guidance for developers who want to use OpenGL on top of CameraX.

因此，对于Preview用例，他们向PreviewView添加了更多API，以满足开发人员的需求。 此外，为了满足高级开发人员的需求，他们添加了OpenGL示例。 简而言之，这是对想要在CameraX之上使用OpenGL的开发人员的指导。

Today, Google has added two important APIs to the PreviewView. First, getPreviewStreamState API. This API allows application to be notified when a preview is streaming or idle. This is useful for application who want to avoid this very temporary rest screen when a preview is starting up. For instance, they can show a placeholder image on top of PreviewView when the preview is idle, and hide the placeholder when the preview is streaming. Second, getBitmap API. This API allows application to get a snapshot of current preview displayed in the PreviewView. As you know, obtaining a bitmap is not an easy task as it appears, because, firstly, the underlying view could be a surface view or tetra view, which has various ways to get a bitmap. Secondly, the bitmap you get from the view, it is not appropriately transformed or cropped like the PreviewView carries out. Thus, the new getBitmap API allows application to get a bitmap 100% identical to what you observe in the PreviewView.

今天，Google已向PreviewView添加了两个重要的API。 首先，获取getPreviewStreamState API。 此API允许在预览流式传输或空闲时通知应用程序。 这对于希望在启动预览时避免出现此临时屏幕的应用程序很有用。 例如，他们可以在预览闲置时在PreviewView顶部显示一个占位符图像，并在预览流式传输时隐藏占位符。 第二， getBitmap API。 该API允许应用程序获取PreviewView中显示的当前预览的快照。 众所周知，获得位图并不是一件容易的事，因为首先，基础视图可能是表面视图或四面体视图，它们具有多种获取位图的方法。 其次，您从视图中获得的位图没有像PreviewView那样进行适当的变换或裁剪。 因此，新的getBitmap API允许应用程序获得与您在PreviewView中观察到的位图100％相同的位图。

Besides, you probably want to draw some advanced effects on top of camera output. Or perhaps you would like to integrate the camera preview into your existing OpenGL bucket or game engines that are in the app. So, for accomplishing these tasks, you should follow some steps as follows: The first step is you create a SurfaceTexture in your render insert. Then, you create a surface using the SurfaceTexture. After that, you provide the Surface to the Preview SurfaceProvider. This SurfaceProvider will also provide a signal to your Android app when it is safe to release the Surface. Eventually, your app can now easily receive the camera frame in the SurfaceTexture, and your render insert can do any task you want on the SurfaceTexture using OpenGL.

此外，您可能想在相机输出上绘制一些高级效果。 或者，您可能希望将相机预览集成到应用程序中现有的OpenGL存储桶或游戏引擎中。 因此，要完成这些任务，您应该遵循以下步骤：第一步是在渲染插入中创建SurfaceTexture 。 然后，使用SurfaceTexture创建一个曲面。 之后，将Surface提供给Preview SurfaceProvider 。 当安全释放Surface时，此SurfaceProvider还将向您的Android应用提供信号。 最终，您的应用现在可以轻松地在SurfaceTexture中接收相机框架，并且您的渲染插入可以使用OpenGL在SurfaceTexture上执行您想要的任何任务。

3) Image analysis: YUV to RGB conversion utility, ML Kit sample, TensorFlow Lite sample

3)图像分析：YUV到RGB转换实用程序，ML Kit示例，TensorFlow Lite示例

For the image analysis use case, they has introduced samples for both convert YUV to RGB and use ML Kit or TensorFlow Lite to analyze the image retrieved from image analysis process. Google has provided you a list of sample calls that can make integration easier. First, YUV to RGB converter. The image you retrieve from image analysis is in YUV format. This is a native format in the Android camera system, but as you know lots of the libraries need the image to be RGB format. As a result, these utilities provide an easy and fast way to convert from YUV to RGB. It uses Renderscript under the hood. So, it is efficient and highly accelerated. In addition, it is able to achieve 30 FPS with 640*480 resolutions across devices like Pixel 3 XL, Samsung S10, and Huawei P20.

对于图像分析用例，他们引入了将YUV转换为RGB并使用ML Kit或TensorFlow Lite来分析从图像分析过程中检索到的图像的样本。 Google为您提供了示例调用列表，可以简化集成。 首先，YUV到RGB转换器。 您从图像分析中检索到的图像为YUV格式。 这是Android相机系统中的本机格式，但是您知道很多库都需要将图像转换为RGB格式。 因此，这些实用程序提供了一种从YUV转换为RGB的简便方法。 它在后台使用Renderscript 。 因此，它是高效且高度加速的。 此外，它可以在Pixel 3 XL，Samsung S10和Huawei P20等设备上以640 * 480的分辨率实现30 FPS。

结论 (In conclusion)

In fact, CameraX is a Jetpack support library to make camera app development much more easier. Currently, it is in Beta version. The CameraX API is built on top of Camera2 API, and provides its capabilities. This article considered some main issues in using Jetpack CameraX and the new features that have been released recently by Google for Android development.

实际上，CameraX是Jetpack支持库，可以使相机应用程序的开发更加轻松。 当前，它是Beta版本。 CameraX API建立在Camera2 API之上，并提供其功能。 本文考虑了使用Jetpack CameraX的一些主要问题以及Google最近为Android开发发布的新功能。

翻译自: https://medium.com/kayvan-kaseb/using-jetpack-camerax-34bcb8ae5de

jetpack使用