【Android】VirtualDisplay创建流程及原理

最新推荐文章于 2024-06-14 15:44:16 发布

林多

最新推荐文章于 2024-06-14 15:44:16 发布

阅读量5.7k

点赞数 4

分类专栏： Android 文章标签： android virtualdisplay 虚拟屏原理 surface

本文链接：https://blog.csdn.net/zxc024000/article/details/133841331

版权

Android 专栏收录该内容

16 篇文章 4 订阅

订阅专栏

Android VirtualDisplay创建流程及原理

Android DisplayManager提供了createVirtualDisplay接口，用于创建虚拟屏。虚拟屏可用于录屏（网上很多资料说这个功能），分屏幕（比如一块很长的屏幕，通过虚拟屏分出不同的区域）等等。

创建VirtualDisplay

DisplayManager中的函数原型如下。后两个Hide的API，只有平台的应用才可以使用。

// frameworks/base/core/java/android/hardware/display/DisplayManager.java
public VirtualDisplay createVirtualDisplay(@NonNull String name,
		int width, int height, int densityDpi, @Nullable Surface surface, int flags) {
}

public VirtualDisplay createVirtualDisplay(@NonNull String name,
		int width, int height, int densityDpi, @Nullable Surface surface, int flags,
		@Nullable VirtualDisplay.Callback callback, @Nullable Handler handler) {
}
/** @hide */
public VirtualDisplay createVirtualDisplay(@Nullable MediaProjection projection,
		@NonNull String name, int width, int height, int densityDpi, @Nullable Surface surface,
		int flags, @Nullable VirtualDisplay.Callback callback, @Nullable Handler handler,
		@Nullable String uniqueId) {
}
/** @hide */
public VirtualDisplay createVirtualDisplay(@Nullable MediaProjection projection,
		@NonNull VirtualDisplayConfig virtualDisplayConfig,
		@Nullable VirtualDisplay.Callback callback, @Nullable Handler handler) {
}

补充一点，MediaProjection中也提供了 createVirtualDisplay这个接口，实际上也是通过调用DisplayManager实现的功能。


// frameworks/base/media/java/android/media/projection/MediaProjection.java
    public VirtualDisplay createVirtualDisplay(@NonNull VirtualDisplayConfig virtualDisplayConfig,
            @Nullable VirtualDisplay.Callback callback, @Nullable Handler handler) {
        DisplayManager dm = mContext.getSystemService(DisplayManager.class);
        // 调用DisplayManager的接口
        return dm.createVirtualDisplay(this, virtualDisplayConfig, callback, handler);
    }

创建VirtualDisplay时，需要传入Surface。**VirtualDisplay上要绘制的内容，实际是通过传入的Surface显示出来的。**比如在主屏（根据物理屏，分配逻辑Display）上创建了一个SurfaceView，通过把这个SurfaceView传给VirtualDisplay。那么VirtualDisplay的内容，实际上是在主屏的SurfaceView上显示的。下面是一段Android原生的例子。

// frameworks/base/packages/SystemUI/tests/src/com/android/systemui/navigationbar/NavigationBarButtonTest.java
private Display createVirtualDisplay() {
	final String displayName = "NavVirtualDisplay";
	final DisplayInfo displayInfo = new DisplayInfo();
	mContext.getDisplay().getDisplayInfo(displayInfo);

	final DisplayManager displayManager = mContext.getSystemService(DisplayManager.class);

	// 创建ImageReader，通过它得到一张Surface
	mReader = ImageReader.newInstance(displayInfo.logicalWidth,
			displayInfo.logicalHeight, PixelFormat.RGBA_8888, 2);

	assertNotNull("ImageReader must not be null", mReader);

	// 创建虚拟屏，传入Surface。
	mVirtualDisplay = displayManager.createVirtualDisplay(displayName, displayInfo.logicalWidth,
			displayInfo.logicalHeight, displayInfo.logicalDensityDpi, mReader.getSurface(),
			0 /*flags*/);

	assertNotNull("virtual display must not be null", mVirtualDisplay);

	waitForDisplayReady(mVirtualDisplay.getDisplay().getDisplayId());

	return mVirtualDisplay.getDisplay();
}

上面的例子中创建虚拟屏，返回Display（实际上是VirtualDislay）对象。有了Display对象，我们就可以将View绑定到这个虚拟的Display上了（绑定网上方法比较多可自行搜索）。关于Surface的创建，有很多种方法，比如通过SurfaceContron+Buffer这种方式也可以。
VituralDisplay创建时，需要提供flag。其值定义如下，可通过 “或”将flag组合。

 
    public static final int VIRTUAL_DISPLAY_FLAG_PUBLIC = 1 << 0;

    public static final int VIRTUAL_DISPLAY_FLAG_PRESENTATION = 1 << 1;

    public static final int VIRTUAL_DISPLAY_FLAG_SECURE = 1 << 2;

    public static final int VIRTUAL_DISPLAY_FLAG_OWN_CONTENT_ONLY = 1 << 3;

    public static final int VIRTUAL_DISPLAY_FLAG_AUTO_MIRROR = 1 << 4;

    public static final int VIRTUAL_DISPLAY_FLAG_CAN_SHOW_WITH_INSECURE_KEYGUARD = 1 << 5;

    public static final int VIRTUAL_DISPLAY_FLAG_SUPPORTS_TOUCH = 1 << 6;

    public static final int VIRTUAL_DISPLAY_FLAG_ROTATES_WITH_CONTENT = 1 << 7;

    public static final int VIRTUAL_DISPLAY_FLAG_DESTROY_CONTENT_ON_REMOVAL = 1 << 8;

    public static final int VIRTUAL_DISPLAY_FLAG_SHOULD_SHOW_SYSTEM_DECORATIONS = 1 << 9;

    public static final int VIRTUAL_DISPLAY_FLAG_TRUSTED = 1 << 10;

    public static final int VIRTUAL_DISPLAY_FLAG_OWN_DISPLAY_GROUP = 1 << 11;

DisplayManager公开的接口中，有VirtualDisplay.Callback ，提供了其状态的回调。

    public static abstract class Callback {
        /**
         * Called when the virtual display video projection has been
         * paused by the system or when the surface has been detached
         * by the application by calling setSurface(null).
         * The surface will not receive any more buffers while paused.
         */
         public void onPaused() { }

        /**
         * Called when the virtual display video projection has been
         * resumed after having been paused.
         */
         public void onResumed() { }

        /**
         * Called when the virtual display video projection has been
         * stopped by the system.  It will no longer receive frames
         * and it will never be resumed.  It is still the responsibility
         * of the application to release() the virtual display.
         */
        public void onStopped() { }
    }

VirtualDisplay原理

关于VirtualDisplay的实现原理，主要从AndroidFramework角度进行分析。

// /frameworks/base/core/java/android/hardware/display/DisplayManager.java
public VirtualDisplay createVirtualDisplay(@NonNull String name,
		int width, int height, int densityDpi, @Nullable Surface surface, int flags) {
	return createVirtualDisplay(name, width, height, densityDpi, surface, flags, null, null);
}

public VirtualDisplay createVirtualDisplay(@NonNull String name,
		int width, int height, int densityDpi, @Nullable Surface surface, int flags,
		@Nullable VirtualDisplay.Callback callback, @Nullable Handler handler) {
	final VirtualDisplayConfig.Builder builder = new VirtualDisplayConfig.Builder(name, width,
			height, densityDpi);
	builder.setFlags(flags);
	if (surface != null) {
		builder.setSurface(surface);
	}
	return createVirtualDisplay(null /* projection */, builder.build(), callback, handler);
}

// TODO : Remove this hidden API after remove all callers. (Refer to MultiDisplayService)
/** @hide */
public VirtualDisplay createVirtualDisplay(@Nullable MediaProjection projection,
		@NonNull String name, int width, int height, int densityDpi, @Nullable Surface surface,
		int flags, @Nullable VirtualDisplay.Callback callback, @Nullable Handler handler,
		@Nullable String uniqueId) {
	final VirtualDisplayConfig.Builder builder = new VirtualDisplayConfig.Builder(name, width,
			height, densityDpi);
	builder.setFlags(flags);
	if (uniqueId != null) {
		builder.setUniqueId(uniqueId);
	}
	if (surface != null) {
		builder.setSurface(surface);
	}
	return createVirtualDisplay(projection, builder.build(), callback, handler);
}

/** @hide */
public VirtualDisplay createVirtualDisplay(@Nullable MediaProjection projection,
		@NonNull VirtualDisplayConfig virtualDisplayConfig,
		@Nullable VirtualDisplay.Callback callback, @Nullable Handler handler) {
	// 走的这里，会调用到DisplayManagerGlobal中。
	return mGlobal.createVirtualDisplay(mContext, projection, virtualDisplayConfig, callback,
			handler);
}

DisplayManagerGlobal调用DMS（DisplayManagerService）服务创建虚拟屏，得到DMS返回的DisplayID后，通过DisplayID在Client端创建了VirtualDisplay对象。

// /frameworks/base/core/java/android/hardware/display/DisplayManager.java
public VirtualDisplay createVirtualDisplay(@NonNull Context context, MediaProjection projection,
		@NonNull VirtualDisplayConfig virtualDisplayConfig, VirtualDisplay.Callback callback,
		Handler handler) {
	VirtualDisplayCallback callbackWrapper = new VirtualDisplayCallback(callback, handler);
	// 从MediaProjection过来的调用，这个地方非空。
	IMediaProjection projectionToken = projection != null ? projection.getProjection() : null;
	int displayId;
	try {
		// 告知DMS创建虚拟屏，并返回DisplayID
		displayId = mDm.createVirtualDisplay(virtualDisplayConfig, callbackWrapper,
				projectionToken, context.getPackageName());
	} catch (RemoteException ex) {
		throw ex.rethrowFromSystemServer();
	}
	if (displayId < 0) {
		Log.e(TAG, "Could not create virtual display: " + virtualDisplayConfig.getName());
		return null;
	}
	
	// 通过DisplayID，取得Display对象信息（也是调用DMS得到的）
	Display display = getRealDisplay(displayId);
	if (display == null) {
		Log.wtf(TAG, "Could not obtain display info for newly created "
				+ "virtual display: " + virtualDisplayConfig.getName());
		try {
			// 创建失败，需要释放
			mDm.releaseVirtualDisplay(callbackWrapper);
		} catch (RemoteException ex) {
			throw ex.rethrowFromSystemServer();
		}
		return null;
	}
	
	// 创建VirtualDisplay
	return new VirtualDisplay(this, display, callbackWrapper,
			virtualDisplayConfig.getSurface());
}

DisplayManagerService(DMS)中创建DisplayDevice并添加到Device列表中管理

// /frameworks/base/services/core/java/com/android/server/display/DisplayManagerService.java
@Override // Binder call
public int createVirtualDisplay(VirtualDisplayConfig virtualDisplayConfig,
		IVirtualDisplayCallback callback, IMediaProjection projection, String packageName) {
		
	// 检查uid与包名，是否相符。
	final int callingUid = Binder.getCallingUid();
	if (!validatePackageName(callingUid, packageName)) {
		throw new SecurityException("packageName must match the calling uid");
	}
	if (callback == null) {
		throw new IllegalArgumentException("appToken must not be null");
	}
	if (virtualDisplayConfig == null) {
		throw new IllegalArgumentException("virtualDisplayConfig must not be null");
	}
	//  拿到client端传过来的surface对象
	final Surface surface = virtualDisplayConfig.getSurface();
	int flags = virtualDisplayConfig.getFlags();

	if (surface != null && surface.isSingleBuffered()) {
		throw new IllegalArgumentException("Surface can't be single-buffered");
	}

	// 下面开始针对Flag，做一些逻辑判断。
	if ((flags & VIRTUAL_DISPLAY_FLAG_PUBLIC) != 0) {
		flags |= VIRTUAL_DISPLAY_FLAG_AUTO_MIRROR;

		// Public displays can't be allowed to show content when locked.
		if ((flags & VIRTUAL_DISPLAY_FLAG_CAN_SHOW_WITH_INSECURE_KEYGUARD) != 0) {
			throw new IllegalArgumentException(
					"Public display must not be marked as SHOW_WHEN_LOCKED_INSECURE");
		}
	}
	if ((flags & VIRTUAL_DISPLAY_FLAG_OWN_CONTENT_ONLY) != 0) {
		flags &= ~VIRTUAL_DISPLAY_FLAG_AUTO_MIRROR;
	}
	if ((flags & VIRTUAL_DISPLAY_FLAG_AUTO_MIRROR) != 0) {
		flags &= ~VIRTUAL_DISPLAY_FLAG_OWN_DISPLAY_GROUP;
	}

	if (projection != null) {
		try {
			if (!getProjectionService().isValidMediaProjection(projection)) {
				throw new SecurityException("Invalid media projection");
			}
			flags = projection.applyVirtualDisplayFlags(flags);
		} catch (RemoteException e) {
			throw new SecurityException("unable to validate media projection or flags");
		}
	}

	if (callingUid != Process.SYSTEM_UID &&
			(flags & VIRTUAL_DISPLAY_FLAG_AUTO_MIRROR) != 0) {
		if (!canProjectVideo(projection)) {
			throw new SecurityException("Requires CAPTURE_VIDEO_OUTPUT or "
					+ "CAPTURE_SECURE_VIDEO_OUTPUT permission, or an appropriate "
					+ "MediaProjection token in order to create a screen sharing virtual "
					+ "display.");
		}
	}
	if (callingUid != Process.SYSTEM_UID && (flags & VIRTUAL_DISPLAY_FLAG_SECURE) != 0) {
		if (!canProjectSecureVideo(projection)) {
			throw new SecurityException("Requires CAPTURE_SECURE_VIDEO_OUTPUT "
					+ "or an appropriate MediaProjection token to create a "
					+ "secure virtual display.");
		}
	}

	if (callingUid != Process.SYSTEM_UID && (flags & VIRTUAL_DISPLAY_FLAG_TRUSTED) != 0) {
		if (!checkCallingPermission(ADD_TRUSTED_DISPLAY, "createVirtualDisplay()")) {
			EventLog.writeEvent(0x534e4554, "162627132", callingUid,
					"Attempt to create a trusted display without holding permission!");
			throw new SecurityException("Requires ADD_TRUSTED_DISPLAY permission to "
					+ "create a trusted virtual display.");
		}
	}

	if (callingUid != Process.SYSTEM_UID
			&& (flags & VIRTUAL_DISPLAY_FLAG_OWN_DISPLAY_GROUP) != 0) {
		if (!checkCallingPermission(ADD_TRUSTED_DISPLAY, "createVirtualDisplay()")) {
			throw new SecurityException("Requires ADD_TRUSTED_DISPLAY permission to "
					+ "create a virtual display which is not in the default DisplayGroup.");
		}
	}

	if ((flags & VIRTUAL_DISPLAY_FLAG_TRUSTED) == 0) {
		flags &= ~VIRTUAL_DISPLAY_FLAG_SHOULD_SHOW_SYSTEM_DECORATIONS;
	}

	// Sometimes users can have sensitive information in system decoration windows. An app
	// could create a virtual display with system decorations support and read the user info
	// from the surface.
	// We should only allow adding flag VIRTUAL_DISPLAY_FLAG_SHOULD_SHOW_SYSTEM_DECORATIONS
	// to trusted virtual displays.
	final int trustedDisplayWithSysDecorFlag =
			(VIRTUAL_DISPLAY_FLAG_SHOULD_SHOW_SYSTEM_DECORATIONS
					| VIRTUAL_DISPLAY_FLAG_TRUSTED);
	if ((flags & trustedDisplayWithSysDecorFlag)
			== VIRTUAL_DISPLAY_FLAG_SHOULD_SHOW_SYSTEM_DECORATIONS
			&& !checkCallingPermission(INTERNAL_SYSTEM_WINDOW, "createVirtualDisplay()")) {
			throw new SecurityException("Requires INTERNAL_SYSTEM_WINDOW permission");
	}

	final long token = Binder.clearCallingIdentity();
	try {
		// 调用内部实现
		return createVirtualDisplayInternal(callback, projection, callingUid, packageName,
				surface, flags, virtualDisplayConfig);
	} finally {
		Binder.restoreCallingIdentity(token);
	}
}

// /frameworks/base/services/core/java/com/android/server/display/DisplayManagerService.java
private int createVirtualDisplayInternal(IVirtualDisplayCallback callback,
		IMediaProjection projection, int callingUid, String packageName, Surface surface,
		int flags, VirtualDisplayConfig virtualDisplayConfig) {
	synchronized (mSyncRoot) {
		if (mVirtualDisplayAdapter == null) {
			Slog.w(TAG, "Rejecting request to create private virtual display "
					+ "because the virtual display adapter is not available.");
			return -1;
		}
		// 为虚拟屏创建Device（告知surfaceflinger创建Display）
		DisplayDevice device = mVirtualDisplayAdapter.createVirtualDisplayLocked(
				callback, projection, callingUid, packageName, surface, flags,
				virtualDisplayConfig);
		if (device == null) {
			return -1;
		}

		// 发送添加Device通知，这里比较重要
		mDisplayDeviceRepo.onDisplayDeviceEvent(device,
				DisplayAdapter.DISPLAY_DEVICE_EVENT_ADDED);
		// 检查Display是否创建成功
		final LogicalDisplay display = mLogicalDisplayMapper.getDisplayLocked(device);
		if (display != null) {
			return display.getDisplayIdLocked();
		}

		// Something weird happened and the logical display was not created.
		Slog.w(TAG, "Rejecting request to create virtual display "
				+ "because the logical display was not created.");
		mVirtualDisplayAdapter.releaseVirtualDisplayLocked(callback.asBinder());
		mDisplayDeviceRepo.onDisplayDeviceEvent(device,
				DisplayAdapter.DISPLAY_DEVICE_EVENT_REMOVED);
	}
	return -1;
}

接下来DMS开始调用SurfaceFlinger的接口，创建Display。并将Display放入自身的List中管理。

// /frameworks/base/services/core/java/com/android/server/display/VirtualDisplayAdapter.java
public DisplayDevice createVirtualDisplayLocked(IVirtualDisplayCallback callback,
		IMediaProjection projection, int ownerUid, String ownerPackageName, Surface surface,
		int flags, VirtualDisplayConfig virtualDisplayConfig) {
	String name = virtualDisplayConfig.getName();
	// VIRTUAL_DISPLAY_FLAG_SECURE 的用途，是判断是否为安全的Display，这个参数会告知SurfaceFlinger
	boolean secure = (flags & VIRTUAL_DISPLAY_FLAG_SECURE) != 0;
	IBinder appToken = callback.asBinder();
	// 调用SurfaceFligner创建Display（Display的type是virtual）
	IBinder displayToken = mSurfaceControlDisplayFactory.createDisplay(name, secure);
	final String baseUniqueId =
			UNIQUE_ID_PREFIX + ownerPackageName + "," + ownerUid + "," + name + ",";
	final int uniqueIndex = getNextUniqueIndex(baseUniqueId);
	String uniqueId = virtualDisplayConfig.getUniqueId();
	if (uniqueId == null) {
		uniqueId = baseUniqueId + uniqueIndex;
	} else {
		uniqueId = UNIQUE_ID_PREFIX + ownerPackageName + ":" + uniqueId;
	}
	// 通过SurfaceFligner返回的displayToken，创建Device对象
	VirtualDisplayDevice device = new VirtualDisplayDevice(displayToken, appToken,
			ownerUid, ownerPackageName, surface, flags, new Callback(callback, mHandler),
			uniqueId, uniqueIndex, virtualDisplayConfig);
	//  放到虚拟屏的List中管理。
	mVirtualDisplayDevices.put(appToken, device);

	try {
		if (projection != null) {
			projection.registerCallback(new MediaProjectionCallback(appToken));
		}
		appToken.linkToDeath(device, 0);
	} catch (RemoteException ex) {
		mVirtualDisplayDevices.remove(appToken);
		device.destroyLocked(false);
		return null;
	}

	// Return the display device without actually sending the event indicating
	// that it was added.  The caller will handle it.
	return device;
}

// /frameworks/base/services/core/java/com/android/server/display/DisplayDeviceRepository.java
//  mDisplayDeviceRepo.onDisplayDeviceEvent(device,
//				DisplayAdapter.DISPLAY_DEVICE_EVENT_ADDED); 
// 这段代码，会调用到下面的函数中。
private void handleDisplayDeviceAdded(DisplayDevice device) {
	synchronized (mSyncRoot) {
		DisplayDeviceInfo info = device.getDisplayDeviceInfoLocked();
		if (mDisplayDevices.contains(device)) {
			Slog.w(TAG, "Attempted to add already added display device: " + info);
			return;
		}
		Slog.i(TAG, "Display device added: " + info);
		device.mDebugLastLoggedDeviceInfo = info;
		// 需要是将Device（就是上面创建的虚拟屏幕Device）放入到DMS的管理list
		mDisplayDevices.add(device);
		// 通知Device添加，会调用到LogicalDisplayMappe的handleDisplayDeviceAddedLocked中。
		sendEventLocked(device, DISPLAY_DEVICE_EVENT_ADDED);
	}
}
// /frameworks/base/services/core/java/com/android/server/display/LogicalDisplayMapper.java
private void handleDisplayDeviceAddedLocked(DisplayDevice device) {
	DisplayDeviceInfo deviceInfo = device.getDisplayDeviceInfoLocked();
	// Internal Displays need to have additional initialization.
	// This initializes a default dynamic display layout for INTERNAL
	// devices, which is used as a fallback in case no static layout definitions
	// exist or cannot be loaded.
	if (deviceInfo.type == Display.TYPE_INTERNAL) {
		initializeInternalDisplayDeviceLocked(device);
	}

	// Create a logical display for the new display device
	LogicalDisplay display = createNewLogicalDisplayLocked(
			device, Layout.assignDisplayIdLocked(false /*isDefault*/));

	// 刷新布局和display配置
	applyLayoutLocked();
	updateLogicalDisplaysLocked();
}

虚拟屏幕的创建，Client端通过Surface告知的DisplayID，创建VirtualDisplay对象。通过DisplayID，与DMS打交道。DMS服务端，通过SurfaceFlinger创建虚拟屏，拿到SurfaceFligner的DisplayToken，然后通过它创建VirtualDisplayDevice + LogicalDisplay来管理虚拟屏幕。

如何上屏？

创建虚拟屏幕的时候，会传入了一张Surface（比如绑定主屏的一张Buffer）。虚拟屏通过这张Surface拿到Surface对应的Buffer，将上屏内容绘制到这个Buffer上，然后提交到画面流水线上（SurfaceFlinger）。通过SurfaceFlinger将这个这个Buffer最终由SurfaceFlinger描画并显示到Surface所在那张Display上（根据VirtualDisplay位置去显示。）