ViewPager中显示图片和播放视频填坑之旅

一.需求来源与实现思路

1.最近项目需求中有用到需要在ViewPager中播放视频和显示图片的功能，视频是本地视频，最开始的实现思路是ViewPager中根据当前item位置对应的是图片还是视频去初始化PhotoView和SurfaceView，同时销毁时根据item的位置去判断移除PhotoView和SurfaceView。

2.上面那种方式确实是可以实现的，但是存在2个问题，第一，MediaPlayer的生命周期不容易控制并且存在内存泄漏问题。第二，连续三个item都是视频时，来回滑动的过程中发现会出现上个视频的最后一帧画面的bug。

3.未提升用户体验，视频播放器初始化完成前上面会覆盖有该视频的第一帧图片,但是发现存在第一帧图片与视频第一帧信息不符的情况，后面会通过代码给出解决方案。

4.图片和视频尺寸如何适配以保证不变形。

二.需要填的坑

1.对于MediaPlayer的生命周期不容易控制的本质原因是这种实现思路上我的播放器只有1个，频繁的初始化和销毁造成了问题，所以后面我更改了实现方式，一个item的视频对应一个播放器。

2.对于滑动过程中发现会出现上个视频的最后一帧画面的bug,发现是surfaceView这个控件造成的，后面通过将播放的载体更换为TextureView完美解决该问题。

3.SurfaceView与TextureView的本质异同

第一：两者都能在独立的线程中绘制和渲染，在专用的GPU线程中大大提高渲染的性能。

第二：SurfaceView专门提供了嵌入视图层级的绘制界面，开发者可以控制该界面像Size等的形式，能保证界面在屏幕上的正确位置。但也有局限：
1.由于是独立的一层View，更像是独立的一个Window，不能加上动画、平移、缩放；
2.两个SurfaceView不能相互覆盖。

第三:Texture更像是一般的View，像TextView那样能被缩放、平移，也能加上动画。TextureView只能在开启了硬件加速的Window中使用，并且消费的内存要比SurfaceView多，并伴随着1-3帧的延迟。

第四:屏幕锁屏时SurfaceView会销毁重建，TextureView不会！

三.具体实现核心代码

1.ViewPager的初始化

mAdapter = ImageBrowseFragmentPagerAdapter(supportFragmentManager, this, imgs)
        imgs_viewpager.offscreenPageLimit = 1
        imgs_viewpager.adapter = mAdapter
        imgs_viewpager.currentItem = mPosition


        //为了处理首次点击时视频播放的问题
        val message = Message.obtain()
        message.what = START_PLAY_VIDEO
        mHandler.sendMessageDelayed(message, 200)

2.Handler处理消息

private val START_PLAY_VIDEO = 0
private var DELETE_VIDEO = 1
private var DELETE_VIDEO_START_PLAY = 2
private var mHandler = Handler(Handler.Callback { msg ->
        when (msg.what) {
            //开始播放视频
            START_PLAY_VIDEO -> NotifyDispatch.dispatch(PreviewPlayVideoEvent(mPosition))
            //删除视频时刷新ui
            DELETE_VIDEO -> {
                mAdapter?.setImgs(imgs)
            }
            //解决删除视频时之后跳转到另一个item,当它是视频时不继续播放的问题
            DELETE_VIDEO_START_PLAY -> NotifyDispatch.dispatch(PreviewPlayVideoEvent(mDeletePosition))
        }
        true
    })

3.删除视频或图片的处理逻辑

    private fun deletePhotos(position: Int) {
        if (imgs!!.isEmpty()) {
            return
        }
        ThreadDispatch.right_now.execute({
            var file: File?

            file = File(imgs.get(position))


            if (file != null && file?.exists()!!) {

                val intent = Intent(Intent.ACTION_MEDIA_SCANNER_SCAN_FILE)
                val uri = Uri.fromFile(file)
                intent.data = uri
                sendBroadcast(intent)

                file?.delete()
                imgs.removeAt(position)
            }


            if (position == imgs.size) {
                mDeletePosition = position - 1
            } else {
                mDeletePosition = position
            }
            val message = Message.obtain()
            message.what = DELETE_VIDEO
            mHandler.sendMessage(message)
            NotifyDispatch.dispatch(DeletePreviewPhotoEvent(imgs))
            val message1 = Message.obtain()
            message1.what = DELETE_VIDEO_START_PLAY
            mHandler.sendMessageDelayed(message1, 200)

            if (imgs.isEmpty()) {
                finish()
            }
        })


//    }
    }

4.ViewPager对应的Adapter

package com.immomo.camerax.gui.view.adapter;

import android.content.Context;
import android.os.Bundle;
import android.support.v4.app.Fragment;
import android.support.v4.app.FragmentManager;
import android.support.v4.app.FragmentStatePagerAdapter;
import android.support.v4.app.FragmentTransaction;
import android.view.View;
import android.view.ViewGroup;

import com.immomo.camerax.gui.fragment.PreviewImgFragment;
import com.immomo.camerax.gui.fragment.PreviewVideoFragment;

import java.util.ArrayList;
import java.util.List;

/**
 * Created by liuxu on 2018/3/26.
 */

public class ImageBrowseFragmentPagerAdapter extends FragmentStatePagerAdapter {

    private Context mContext;
    private List<String> datas;
    private int mCurrentSelectedPosition = -1;
    private FragmentManager mFragmentManager;
    private FragmentTransaction mFragmentTransaction;
    private ArrayList<Fragment> mFragments = new ArrayList<>();


    public ImageBrowseFragmentPagerAdapter(FragmentManager fm, Context context, List<String> datas) {
        super(fm);
        mFragmentManager = fm;
        mContext = context;
        this.datas = datas;
    }

    public void removeContext(){
        mContext = null;
    }


    @Override
    public void setPrimaryItem(ViewGroup container, int position, Object object) {
        mCurrentSelectedPosition = position;
    }

    @Override
    public void startUpdate(ViewGroup container) {
        super.startUpdate(container);
    }

    public void setImgs(List<String> imgs) {
        this.datas = imgs;
        notifyDataSetChanged();
    }


    //处理更新无效----删除条目
    @Override
    public int getItemPosition(Object object) {
        return POSITION_NONE;
    }

    public int getPrimaryItemPosition() {
        return mCurrentSelectedPosition;
    }

    public ImageBrowseFragmentPagerAdapter(FragmentManager fm) {
        super(fm);
    }


    @Override
    public boolean isViewFromObject(View view, Object object) {
        return view == ((Fragment) object).getView();
    }

    @Override
    public Fragment getItem(int position) {
        Bundle bundle = new Bundle();
        bundle.putString("url", datas.get(position));
        bundle.putInt("position", position);
        if (datas.get(position).endsWith(".jpg")) {
            PreviewImgFragment previewImgFragment = new PreviewImgFragment();
            previewImgFragment.setArguments(bundle);
            return previewImgFragment;
        } else {
            PreviewVideoFragment previewVideoFragment = new PreviewVideoFragment();
            previewVideoFragment.setArguments(bundle);
            return previewVideoFragment;
        }
    }

    @Override
    public int getCount() {
        return datas == null ? 0 : datas.size();
    }


    @Override
    public Object instantiateItem(ViewGroup container, int position) {
        return super.instantiateItem(container,position);
    }


    @Override
    public void destroyItem(ViewGroup container, int position, Object object) {
        super.destroyItem(container,position,object);


    }


}

5显示图片对应的Fragment

package com.immomo.camerax.gui.fragment;

import android.os.Bundle;
import android.support.annotation.Nullable;
import android.support.v4.app.Fragment;
import android.view.LayoutInflater;
import android.view.View;
import android.view.ViewGroup;

import com.bumptech.glide.Glide;
import com.immomo.camerax.R;
import com.immomo.camerax.foundation.util.StatusBarUtils;
import com.immomo.camerax.gui.view.ResizablePhotoView;

/**
 * Created by liuxu on 2018/3/27.
 */

public class PreviewImgFragment extends Fragment {


    @Nullable
    @Override
    public View onCreateView(LayoutInflater inflater, @Nullable ViewGroup container, @Nullable Bundle savedInstanceState) {
        View view = inflater.inflate(R.layout.fragment_preview_photo, null);
        ResizablePhotoView resizablePhotoView = view.findViewById(R.id.customPhotoView);
        String url = getArguments().getString("url");
        Glide.with(getContext()).load(url).into(resizablePhotoView);
        resizablePhotoView.setOnClickListener(new View.OnClickListener() {
            @Override
            public void onClick(View view) {
                getActivity().finish();
            }
        });

        return view;
    }

    @Override
    public void onPause() {
        super.onPause();
    }


    @Override
    public void onResume() {
        super.onResume();
    }

    @Override
    public void onDetach() {
        super.onDetach();
    }


    @Override
    public void onDestroyView() {
        super.onDestroyView();
    }


    @Override
    public void onStart() {
        super.onStart();
    }

    @Override
    public void onDestroy() {
        super.onDestroy();
    }

    @Override
    public void setUserVisibleHint(boolean isVisibleToUser) {
        super.setUserVisibleHint(isVisibleToUser);
    }
}

6.图片根据宽度适配高度的自定义View

package com.immomo.camerax.gui.view;

import android.content.Context;
import android.graphics.drawable.Drawable;
import android.util.AttributeSet;

import com.github.chrisbanes.photoview.PhotoView;

/**
 * Created by liuxu on 2018/4/7.
 */

public class ResizablePhotoView extends PhotoView {
    public ResizablePhotoView(Context context) {
        super(context);
    }

    public ResizablePhotoView(Context context, AttributeSet attr) {
        super(context, attr);
    }

    public ResizablePhotoView(Context context, AttributeSet attr, int defStyle) {
        super(context, attr, defStyle);
    }

    @Override
    protected void onMeasure(int widthMeasureSpec, int heightMeasureSpec) {
        Drawable d = getDrawable();
        if (d != null){
            int width = MeasureSpec.getSize(widthMeasureSpec);
            //高度根据使得图片的宽度充满屏幕计算而得
            int height = (int) Math.ceil((float) width * (float) d.getIntrinsicHeight() / (float) d.getIntrinsicWidth());
            setMeasuredDimension(width, height);
        }else {
            super.onMeasure(widthMeasureSpec, heightMeasureSpec);
        }
    }
}

7.播放视频对应的Fragment

/**
 * Created by liuxu on 2018/3/27.
 */

public class PreviewVideoFragment extends Fragment {


    private ImageView mPhotoView;
    private TextureView mTextureView;
    private String mUrl;
    private int mPosition;
    private AndroidMediaPlayer mIjkVodMediaPlayer;
    private boolean mIsSelected;
    private boolean mIsFirstPrepared;
    private PreviewPlayVideoSubscriber mPreviewPlayVideoSubscriber = new PreviewPlayVideoSubscriber() {
        @Override
        public void onEventMainThread(PreviewPlayVideoEvent event) {
            super.onEventMainThread(event);
            MDLog.e("liuxu",event.getPosition()+"");
            if (event != null && event.getPosition() == mPosition) {
                //说明是当前条目
                if (mIjkVodMediaPlayer != null && !mIjkVodMediaPlayer.isPlaying()) {
                    if (mTextureView != null) {
                        mIjkVodMediaPlayer.setSurface(mSurface);
                        mIjkVodMediaPlayer.prepareAsync();
                        mPhotoView.setVisibility(View.VISIBLE);
                    }
                }
                mIsSelected = true;
            } else {
                if (mIjkVodMediaPlayer != null && mIjkVodMediaPlayer.isPlaying()) {
                    mIjkVodMediaPlayer.pause();
                    mIjkVodMediaPlayer.stop();
                }

                if (mPhotoView != null) {
                    mPhotoView.setVisibility(View.VISIBLE);
                }
                mIsSelected = false;
            }

        }
    };
    private String mWidth;
    private String mHeight;

    @Nullable
    @Override
    public View onCreateView(LayoutInflater inflater, @Nullable ViewGroup container, @Nullable Bundle savedInstanceState) {
        mPreviewPlayVideoSubscriber.register();
        View view = inflater.inflate(R.layout.fragment_preview_video, null);
        mPhotoView = view.findViewById(R.id.photoView);
        mTextureView = view.findViewById(R.id.surfaceView);
        mUrl = getArguments().getString("url");
        mPosition = getArguments().getInt("position");
        layoutPlayer();
        loadVideoScreenshot(getContext(), mUrl, mPhotoView, 1);
        view.setOnClickListener(new View.OnClickListener() {
            @Override
            public void onClick(View view) {

                release();
                getActivity().finish();
            }
        });
        initTextureMedia();
        return view;
    }

    private void initTextureMedia() {
        mTextureView.setSurfaceTextureListener(mSurfaceTextureListener);
    }

    private void play(String url) {
        try {
            mIjkVodMediaPlayer = new AndroidMediaPlayer();
            mIjkVodMediaPlayer.reset();
            mIjkVodMediaPlayer.setDataSource(url);
            //让MediaPlayer和TextureView进行视频画面的结合
            mIjkVodMediaPlayer.setSurface(mSurface);
            //设置监听
            mIjkVodMediaPlayer.setOnBufferingUpdateListener((mp, percent) -> {

            });
            mIjkVodMediaPlayer.setOnCompletionListener(mp -> {
                mp.seekTo(0);
                mp.start();
            });
            mIjkVodMediaPlayer.setOnInfoListener((mp1, what, extra) -> {
                        if (what == IMediaPlayer.MEDIA_INFO_VIDEO_RENDERING_START) {
                            mPhotoView.setVisibility(View.GONE);
                            mIsFirstPrepared = true;
                        }
                        return false;
                    });
            mIjkVodMediaPlayer.setOnErrorListener((mp, what, extra) -> false);
            mIjkVodMediaPlayer.setOnPreparedListener(mp -> {
                mp.start();
                if (!mIsFirstPrepared){
                }else {
                    mPhotoView.setVisibility(View.GONE);
                }
            });
            mIjkVodMediaPlayer.setScreenOnWhilePlaying(true);//在视频播放的时候保持屏幕的高亮
            if (mIsSelected){
                //异步准备
                mIjkVodMediaPlayer.prepareAsync();
            }
        } catch (Exception e) {
            e.printStackTrace();
        }
    }

    private Surface mSurface;
    private TextureView.SurfaceTextureListener mSurfaceTextureListener = new TextureView.SurfaceTextureListener() {
        @Override
        public void onSurfaceTextureAvailable(SurfaceTexture surface, int width, int height) {
            mSurface = new Surface(surface);
            play(mUrl);
        }

        @Override
        public void onSurfaceTextureSizeChanged(SurfaceTexture surface, int width, int height) {

        }

        @Override
        public boolean onSurfaceTextureDestroyed(SurfaceTexture surface) {
            if (mSurface != null){
                mSurface.release();
                mSurface = null;
            }
            if (mTextureView != null){
                if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.O) {
                    mTextureView.releasePointerCapture();
                }
            }
            release();
            return false;
        }

        @Override
        public void onSurfaceTextureUpdated(SurfaceTexture surface) {

        }
    };


    @Override
    public void onStart() {
        super.onStart();
    }
    @Override
    public void onAttach(Context context) {
        super.onAttach(context);
    }

    @Override
    public void onDetach() {
        super.onDetach();
    }

    @Override
    public void onPause() {
        MDLog.e("liuxu", "onPause" + mPosition);
        //处理锁屏时播放器停止播放
        if (mIjkVodMediaPlayer != null && mIjkVodMediaPlayer.isPlaying()){
            mIjkVodMediaPlayer.pause();
            mIjkVodMediaPlayer.stop();
        }
        super.onPause();
    }

    //屏幕打开时重新播放
    @Override
    public void onResume() {
        MDLog.e("liuxu", "onResume" + mPosition);
        super.onResume();
        if (mIsSelected && mIjkVodMediaPlayer != null && !mIjkVodMediaPlayer.isPlaying()) {
            mIjkVodMediaPlayer.prepareAsync();
        }
    }

    @Override
    public void onDestroy() {
        MDLog.e("liuxu", "onDestroy");
        release();
        if (mPreviewPlayVideoSubscriber.isRegister()) {
            mPreviewPlayVideoSubscriber.unregister();
        }
        super.onDestroy();
    }

    private void release() {
        if (mIjkVodMediaPlayer == null) {
            return;
        }

        if (mIjkVodMediaPlayer.isPlaying()) {
            mIjkVodMediaPlayer.stop();
        }

        mIjkVodMediaPlayer.release();
        mIjkVodMediaPlayer = null;
    }

    @Override
    public boolean getUserVisibleHint() {
        return super.getUserVisibleHint();
    }


    /**
     * 动态设置视频宽高信息
     */
    private void layoutPlayer() {
        //获取视频宽高比
        getPlayInfo(mUrl);
        float ratio = Float.parseFloat(mHeight) / Float.parseFloat(mWidth);
        MDLog.e("type", mPosition + "ratio" + ratio);
        int type = 0;
        //添加容错值
        if (ratio < MediaConstants.INSTANCE.getASPECT_RATIO_1_1().toFloat() + MediaConstants.INSTANCE.getSCREEN_DEFAULT_VALUE()
                && ratio > MediaConstants.INSTANCE.getASPECT_RATIO_1_1().toFloat() - MediaConstants.INSTANCE.getSCREEN_DEFAULT_VALUE()) {
            type = ScreenAdapterUtils.INSTANCE.getSCALE_TYPE_11();
        } else if (ratio < MediaConstants.INSTANCE.getASPECT_RATIO_4_3().toFloat() + MediaConstants.INSTANCE.getSCREEN_DEFAULT_VALUE()
                && ratio > MediaConstants.INSTANCE.getASPECT_RATIO_4_3().toFloat() - MediaConstants.INSTANCE.getSCREEN_DEFAULT_VALUE()) {
            type = ScreenAdapterUtils.INSTANCE.getSCALE_TYPE_43();
            MDLog.e("type", "43");
        } else if (ratio < MediaConstants.INSTANCE.getASPECT_RATIO_16_9().toFloat() + MediaConstants.INSTANCE.getSCREEN_DEFAULT_VALUE()
                && ratio > MediaConstants.INSTANCE.getASPECT_RATIO_16_9().toFloat() - MediaConstants.INSTANCE.getSCREEN_DEFAULT_VALUE()) {
            type = ScreenAdapterUtils.INSTANCE.getSCALE_TYPE_169();
            MDLog.e("type", "169");
        }

        FrameLayout.LayoutParams layoutParams = (FrameLayout.LayoutParams) mTextureView.getLayoutParams();


        layoutParams.height = ScreenAdapterUtils.INSTANCE.getSurfaceHeight(type);
        mTextureView.setLayoutParams(layoutParams);
        FrameLayout.LayoutParams params = (FrameLayout.LayoutParams) mPhotoView.getLayoutParams();
        params.height = ScreenAdapterUtils.INSTANCE.getSurfaceHeight(type);
        mPhotoView.setLayoutParams(params);
        MDLog.e("params.height", params.height + "");


    }


    private void getPlayInfo(String mUri) {
        android.media.MediaMetadataRetriever mmr = new android.media.MediaMetadataRetriever();
        try {
            if (mUri != null) {
                mmr.setDataSource(mUri);
            } else {
                //mmr.setDataSource(mFD, mOffset, mLength);
            }

            //宽
            mWidth = mmr.extractMetadata(android.media.MediaMetadataRetriever.METADATA_KEY_VIDEO_WIDTH);
            //高
            mHeight = mmr.extractMetadata(android.media.MediaMetadataRetriever.METADATA_KEY_VIDEO_HEIGHT);
//            mBitmap = mmr.getFrameAtTime(1 );

        } catch (Exception ex) {
        } finally {
            mmr.release();
        }

    }

    public static void loadVideoScreenshot(final Context context, String uri, ImageView imageView, long frameTimeMicros) {
        // 这里的时间是以微秒为单位
        RequestOptions requestOptions = RequestOptions.frameOf(frameTimeMicros);
        requestOptions.set(FRAME_OPTION, MediaMetadataRetriever.OPTION_CLOSEST);
        requestOptions.transform(new BitmapTransformation() {
            @Override
            protected Bitmap transform(@NonNull BitmapPool pool, @NonNull Bitmap toTransform, int outWidth, int outHeight) {
                return toTransform;
            }

            @Override
            public void updateDiskCacheKey(MessageDigest messageDigest) {
                try {
                    messageDigest.update((context.getPackageName() + "RotateTransform").getBytes("utf-8"));
                } catch (Exception e) {
                    e.printStackTrace();
                }

            }
        });
        Glide.with(context).load(uri).apply(requestOptions).into(imageView);
    }

}

4.结语

笔者使用这种方式实现了项目需求，但是由于本人接触音视频的相关内容比较少，全是在不断探索和学习中前进，如有不足之处请评论指正，谢谢。大家共同学习共同进步。