Plugin sample study notes

一、 ANP interface

1. ANP interface in android 

a android为插件提供了额外的NPN函数，称之为ANP

b 这些函数是通过传递函数指针的方式传递给插件的

具体操作方式是通过NPN的GetValue函数

浏览器和插件通过头文件约定好一些值，这些值代表APN函数结构体的指针

c 通过NPN_GetValue插件获取到了这些结构图的指针，然后进而得到了各个ANP里面的函数指针，然后通过这系列函数指针就可以调用系统服务了

d 具体获取这些指针的过程

a)NPN_GetValue(Npapi.cpp)

b)PluginView::getValueStatic(PluginViewAndroid.cpp)

c)(void)anp_getInterface(variable, value, &error)(PluginViewAndroid.cpp)

调用各个接口的初始化函数进行初始化

2. Android's extra ANPInterface

declaration in "android_npapi.h"

a.ANPLogInterfaceV0

log

b.ANPBitmapInterfaceV0

getPixelPacking

c.ANPMatrixInterfaceV0

newMaxtix\deleteMatrix\getFlags\copy\get3x3\set3x3\setIdentity\preTranslate\postTranslate\preScale\postScale\preSkew\postRotate\preRotate\preConcat\postConcat\invert\mapPoints

d.ANPPathInterfaceV0

newPath\deletePath\copy\equal\reset\isEmpty\getBounds\moveTo\lineTo\quadTo\cublicTo\close\offset\transform

e.ANPTypefaceInterfaceV0

createFromName\createFromTypeface\getRefCount\ref\unref\getStyle\getFontPath\getFontDirectoryPath

f. ANPPaintInterfaceV0

newPaint\deletePaint\getFlags\setFlags\getColor\setColor\getStyle\setStyle\getStrokeWidth\getSrokeMite\getStrokeCap\getStrokeJoin\setStrokeWidth\setStrokeMiter\setStrokeCap\setStrokeJoin\getTextEncoding\getTextAlign\getTextSize\getTextScaleX\getTextSkewX\setTextEncoding\setTextAlign\setTextSize\setTextScaleX\setTextSkewx\getTypeface\setTypeface\measureText\getTextWidths\getFontMetrics

g.ANPCanvasInterfaceV0

newCanvas\deleteCanvas\save\restore\translate\scale\rotate\skew\concat\clipRect\clipPath\getTotalMatrix\getLocalClipBounds\getDeviceClipBounds\drawColor\drawPaint\drawLine\drawRect\drawOval\drawPath\drawPosText\drawBitmap\drawBitmapRect

h.ANPWindowInterfaceV0

setVisibleRects\clearVisibleRects\showKeyboard\requestFullScreen\exitFullScreen\requestCenterFitZoom

i. ANPAudioTrackInterfaceV0

newTrack\deleteTrack\start\pause\stop\isStopped

j.ANPEventInterfaceV0

postEvent

h.ANPSystemInterfaceV0 "ANPSystemInterface.cpp"

anp_getApplicationDataDirectory

anp_loadJavaClass

二、class and function used in Plugin

1. functions 

NP_XX ,DLL functions .defined in the dll to be exported by the dll loader

NPN_XX,browser function

defined in Npapi.cpp (will finally call PluginView's function)

Npruntime.cpp

NPP_XX, Plugin's function, defined in the Plugin.

2.NPObject

class created by Browser and called in the plugin.

PluginObject *obj = browser->createobject (instance, getPluginClass());

struct NPObject {

    NPClass *_class;

    uint32_t referenceCount;

    // Additional space may be allocated here by types of NPObjects

};

struct NPClass

{

    uint32_t structVersion;

    NPAllocateFunctionPtr allocate;

    NPDeallocateFunctionPtr deallocate;

    NPInvalidateFunctionPtr invalidate;

    NPHasMethodFunctionPtr hasMethod;

    NPInvokeFunctionPtr invoke;

    NPInvokeDefaultFunctionPtr invokeDefault;

    NPHasPropertyFunctionPtr hasProperty;

    NPGetPropertyFunctionPtr getProperty;

    NPSetPropertyFunctionPtr setProperty;

    NPRemovePropertyFunctionPtr removeProperty;

    NPEnumerationFunctionPtr enumerate;

    NPConstructFunctionPtr construct;

};

defined in NPruntime.h

3. PluginObject class defined in the Plugin, using NPObject as it's subcalss

typedef struct PluginObject {

    NPObject header;

    NPP npp;

    NPWindow* window;

    PluginType pluginType;

    SubPlugin* activePlugin;

} PluginObject

4. 

三、steps for creating a plugin

a. define NP_Initialize,NP_Shutdown,NP_GetValue,NP_GetMiMEDescription

for "PluginPack"

in NP_Initialize evaluate the NPP_XX function pointer with it's own function

b. define it's NPP_XX function, metioned in a

c. define it's NP class, mainly define and implement the function in NPClass

d. in NPP_New call the browser's createObject function, passing it's NPClass to generate it's own pluginobject

all in all, we need to define 3 set of functions

NP_

NPP

and NPClass's function

四 NPP_New中createObject

NPError NPP_New(NPMIMEType pluginType, NPP instance, uint16 mode, int16 argc,
                char* argn[], char* argv[], NPSavedData* saved){

instance->pdata = browser->createobject (instance, getPluginClass());

PluginObject *obj=static_cast<PluginObject*>(instance->pdata);

}

为什么调用createobject返回的指针可以转换为一个PluginObject

a createobject 调用的是_NPN_CreateObject，传入一个NPP和NPClass返回一个NPObject的指针

NPObject *_NPN_CreateObject(NPP npp, NPClass* aClass)

{    ASSERT(aClass);

    if (aClass) {

        NPObject* obj;

        if (aClass->allocate != NULL)

            obj = aClass->allocate(npp, aClass);

        else

            obj = (NPObject*)malloc(sizeof(NPObject));

        if (!obj)

            CRASH();

        obj->_class = aClass;

        obj->referenceCount = 1;

        return obj;

    }

    return 0;

}

NPObject的定义如下

struct NPObject {

    NPClass *_class;

    uint32_t referenceCount;

    // Additional space may be allocated here by types of NPObjects

};在Npruntime.h中

而PluginObject是插件自己定义的一个类

typedef struct PluginObject {

    NPObject header;

    NPP npp;

    NPWindow* window;

    PluginType pluginType;

    SubPlugin* activePlugin;

} PluginObject;

如何保证他们大小相同而且类型可以相互转换？

关键就在于

a）createobject (instance, getPluginClass())的第二个参数，是插件里面定义的。

所以在函数实现过程中 obj = aClass->allocate(npp, aClass);就是调用的插件的allocate

而在插件的这个函数定义中

PluginObject *newInstance = (PluginObject*) malloc(sizeof(PluginObject));

就是申请了一个PluginObject这么大小的空间。

b）PluginObject 的定义

第一个元素就是NPObject类型，而且在返回的时候&newInstance->header;返回的是其NPObject 元素的地址。

c ）由于 PluginObject  的第一个元素是NPObject，所以它的地址就是PluginObject的地址，所以我们可以直接通过指针进行类型转换。

五、NPWindow以及其和PluginWidgetAndroid关系

1.

NPWindow定义在npapi.h

typedef struct _NPWindow

{

    void*    window;     /* Platform specific window handle */

    int32    x;            /* Position of top left corner relative */

    int32    y;            /*    to a netscape page.                    */

    uint32    width;        /* Maximum window size */

    uint32    height;

    NPRect    clipRect;    /* Clipping rectangle in port coordinates */

  NPWindowType type; 

} NPWindow;

它是插件对窗口的抽象表示，

对象定义在

a、PluginView.h NPWindow m_npWindow;

b、PluginWidgetAndroid.h NPWindow  m_pluginWindow;

c、 NPWindow* window; 插件内部

在PluginView setNPWindowIfNeeded

同时把m_npWindow设置给插件和m_window.

2. m_window有什么用？

在PluginViewAndroid中

a 事件处理

PluginView::handleTouchEvent

PluginView::handleMouseEvent

PluginView::handleFocusEvent

PluginView::handleKeyboardEvent

都会调用m_window->sendEvent(evt)

而在 PluginWidgetAndroid::sendEvent

最终调用到了

pkg->pluginFuncs()->event

b 窗口和界面绘制和管理

a)PluginView::handleFocusEvent-》m_window->webViewCore()->contentInvalidate(rect)

b)PluginView::setParent-》m_window->init(c);

c)PluginView::setNPWindowIfNeeded()-》

d)m_window->setWindow(&m_npWindow, m_isTransparent);

e)PluginView::platformSetValue-》m_window->setDrawingModel 和m_window->updateEventFlags(flags)

f)PluginView::invalidateRect

m_window->inval(r, true);

g)PluginView::paint

m_window->setSurfaceClip

m_window->draw

六、 窗口模式和非窗口模式插件

1.

windowed plug-in 是绘制在单独的本地窗口，而不是在它的浏览器窗口。包含它的浏览器页面在它的显示窗口之下。这种窗口自己决定绘制流程。

windowless plug-in 不需要本地窗口，它只需要一个可绘制区域。这个区域要么是这个浏览器窗口，要么是一个屏幕位图。可绘制区域的具体定义取决于平台。

在NPWindow结构体中，有一个变量NPWindowType type;来定义插件属于哪种模型在PluginViewAndroid的platformInit()对于该结构体直接赋值

m_npWindow.type = NPWindowTypeDrawable

m_isWindowed = false； //we don't support windowed yet.

见android的注释

2.NP's drawing mode.

不管窗口模式还是非窗口模式Android都有如下几种绘制模式

kBitmap_ANPDrawingModel

kSurface_ANPDrawingModel

kOpenGL_ANPDrawingModel

3.对于surface绘制模式插件的surface如何关联到java的view

a.获取surface

a）in PluginWidgetAndroid::layoutSurface

jobject pluginSurface；

pluginFuncs()->getvalue(instance, kJavaSurface_ANPGetValue,

static_cast<void*>(& pluginSurface));

调用插件的函数，获取surface赋值给jobject

具体调用过程：

call pluginFuncs()->getvalue(instance, kJavaSurface_ANPGetValue)

在插件里面调用jobject PaintPlugin::getSurface()

*surface = plugin->getSurface();

b) 在插件PaintPlugin::getSurface()

 m_surface = env->NewGlobalRef(paintSurface);

通过JNI获取java PaintSurface的实例

b.将surface赋值到Webview的view中

通过a步骤，插件java的view已经保持在 pluginSurface中，

1) in PluginWidgetAndroid::layoutSurface

jobject tempObj = m_core->addSurface( pluginSurface,

                m_pluginWindow->x, m_pluginWindow->y,

                m_pluginWindow->width, m_pluginWindow->height);

调用webcore的方法WebViewCore::addSurface （WebviewCore.cpp）

2)in "WebViewCore::addSurface"

 jobject result = env->CallObjectMethod(m_javaGlue->object(env).get(),

                                       m_javaGlue->m_addSurface,

                                         view, x, y, width, height);

调用WebViewCore.java's addSurface

3)private ViewManager.ChildView addSurface(View  pluginView, int x, int y, ....）

ViewManager.ChildView view = mWebView.mViewManager.createView();

view. mView =  pluginView;

view.attachView(x, y, width, height);

4) in ViewManager.attachView

mWebView.addView( mView);

所以就把插件中的java jobject实例加入到了webview中

七、other

1.m_pluginFuncs在何时被调用？

a PluginStream中会被调用

b PluginView::start() 调用newp

c PluginView::stop() 调用setwindow和destroy

d PluginView::performRequest把 函数实例传给PluginStream，并且调用urlnotify

e PluginView::bindingInstance() 调用getvalue

f PluginView::getNPObject 调用getvalue

g PluginView::didReceiveResponse，创建stream，并且调用PluginStream的didReceiveResponse

h PluginView::setNPWindowIfNeeded() 调用setwindow

i PluginWidgetAndroid::draw() 调用event

j PluginWidgetAndroid::layoutSurface调用getvalue

k PluginWidgetAndroid::sendEvent调用event

2. PluginStream

Streams can be produced by the browser and consumed by a plug-in instance,

or produced by an instance and consumed by the browser.

Provide an interface for creating families of related or dependent objects without specifying their concrete classes

八、view of plugin

1.NPP是一个结构体，包含两个void的指针

ndata, the pointer of the browser, usually the PluginView, was valued in PluginView::PluginView.

pdata, the pointer of the plugin, was valued by Plugin in NPP_New, maybe a type of "PluginObject" only could be seen in the plugin.

in Browser, we use NPP as a whole.

2. NPP_Window

is the structure holding a void * window, and it's location.

usually save the point in both Browser and Plugin.

3. PluginObject in Plugin

we usually use type "PluginObject" to describe the "Plugin" in Plugin's code.

It should include at least the information below.

NPP

NPP_Window.

NPObject

所以，一个plugin，必不可少的东西包括

"NPNetscapeFuncs" and "PluginObject" mentioned above.

4. m_windowRect = frameView->contentsToWindow(frameRect())

m_pageRect=m_windowRect

5.pluginView.setParent->PluginView.updatePlginWidget

->PluginViewAndroid.setNPWindowIfNeeded

->PluginWidgetAndroid.setWindow

->PluginWidgetAndroid.layoutSurface

6.PluginWidgetAndroid's m_embededView is a java object.

surfaceview绘制方式是在m_embeddedView这个jobject绘制；opengl全屏模式下也是用这个view绘制。

但是opengl绘制方式（非全屏），没有对这个embeddedView赋值，只有一个medialayer那它是怎么绘制的呢？

莫非open每次都能够在最外层绘制？那是如何做的的呢？

7.对于非全屏

pluginView.setParent->PluginView.updatePlginWidget

->PluginViewAndroid.setNPWindowIfNeeded

->PluginWidgetAndroid.setWindow

会调用到sendSizeAndVisibilityEvents

这里面会设置绘制大小，向插件发事件。

8.全屏步骤

a、opengl由于embeddedView默认为null，因此先向插件获取这个View

b、向插件发生一个全屏消息

c、对于非全屏原来有embeddedView，不过要将之前的从webview detach

d、向webcore发送一个消息，SHOW_FULLSCREEN显示全屏。通过PluginFullScreenHolder获取webview的最外层view，将embeddedView给attach上去

9、MediaPlayer

MediaPlay中，唯一重要的类就是MediaTexture

MediaPlayer中方法

a ANativeWindow* MediaLayer::acquireNativeWindowForContent()

b ANativeWindow* MediaLayer::acquireNativeWindowForVideo()

c void MediaLayer::setWindowDimensionsForVideo

d void MediaLayer::releaseNativeWindowForVideo(ANativeWindow* window)

e void MediaLayer::setFramerateCallback

bool MediaLayer::drawGL(GLWebViewState* glWebViewState, SkMatrix& matrix)

最终都是调用了MediaTexture的函数

10、

在MediaTexture中，有如下类

a、TextureWrapper* m_contentTexture;

b、Vector<TextureWrapper*> m_videoTextures;

c Vector<GLuint> m_unusedTextures;

d sp<ANativeWindow> m_newWindow;

e jobject m_weakWebViewRef;就是传入的WebView

传入的WebView jobject有什么用呢？

a 在MediaTexture::initNativeWindowIfNeeded

调用WebView的sendPluginDrawMsg

最后会调入PluginWidgetAndroid的sendSizeAndVisibilityEvents

b在MediaTexture::requestNativeWindowForVideo

调用WebView的postInvalidate

c 传给MediaListener，在其onFrameAvailable中调用

调用WebView的postInvalidate

最后会调用View的invalidate

貌似依然没有解释，绘在哪里的问题。

莫非opengl的绘制，真的每次都是在最顶层，只有给它一个区域就行了？全屏时候，设置embeddedView是避免绘制和WebView绘制冲突，所以用一个view覆盖WebView避免它绘制。然后就完全自己绘制？？

11、TextureWrapper

定义里面包含内容如下：

a GLunit

b SurfaceTexture

c ANativeWindow

d MediaListener

其中，的MediaListener最终持有了传入的m_weakWebViewRef。

12、ANativeWindow

定义在system\core\include\system\Window.h中

它在TextureWrapper的默认构造函数中被创建

TextureWrapper* videoTexture = createTexture();

m_newWindow = videoTexture->nativeWindow；

在createTexture中

wrapper->nativeWindow = new android::SurfaceTextureClient(wrapper->surfaceTexture);

这是用一个它的子类对其进行实现和初始化。

除此以外它还有如下初始化方式

ANativeWindow* ANativeWindow_fromSurface(JNIEnv* env, jobject surface)

这个函数定义在frameworks\base\native\android的Native_window.cpp中