不务正业之微信跳一跳外挂设计(C#版本)

版权声明:没有啥子版权,请随便整。 https://blog.csdn.net/wangqianfa/article/details/79162329
2017年最后一天没有加班,这居然很难得。
到了晚上耍微信跳一跳,简直是一种折磨。各种小心谨慎也只能蹦跶到120分,看看排行榜,第一名居然300多,感觉有点崩溃。
于是设计的了一个外挂。

最初思路:
1- 要支持全自动操作,所以必需能模拟“按下-释放”操作。在Android上很好办到,利用adb的命令input swipe就可以模拟。
2- 要可以自动识别,需要能获取屏幕图像、计算距离。
     获取图像也好办,adb shell screencap可以抓图,adb pull可以把图片从手机传到计算机。
     计算距离则需要进行一点点图像处理。这个是稍微难一点的部分。大致思路上找出起跳点和目标位置的特征。
3- 为什么不直接开发apk?要实现apk模拟“按下-释放”操作,手机必需root过。目前超过7成的手机都不支持root,这个方案可以直接pass
4- 为什么不支持iOS?因为不会。
5- adb的操作模式,决定了可以开发一个Windows或者MacOS的普通应用,调用adb程序,就可以完成数据的获取以及进行设备操控。
6- 剩下的是图像处理,随便哪个主流的编程语言都可以办到。我选择了我熟悉的C#。

环境准备
1- 需要安装adb,以及相应的adb驱动
2- 需要有Visual Studio 2017开发环境


实现获取图像到本地目录
1- 图像存储到哪里          我们存储到当前程序运行的目录下的AppData目录之下。
2- 如何调用adb命令        使用System.Diangnosie.Process来完成。
3- 具体的手机屏幕截图指令,和把文件从手机传到电脑上的指令
adb shell screencap /sdcard/<image-filename>
adb pull /sdcard/<image-filename> <pc-local-directory>

计算起跳点和目标位置的坐标的总体思路
1- 去掉图片背景,并将其二值化(变成只有0-1这个状态),简化数据,便于后续分析
2- 分析起跳点的特征,设计算法查找图形中的起跳点位置
3- 分析目标位置的特征,设计算法查找图形中目标特征的位置
4- 计算起跳点(JumperPoint)和目标点(TargetPoint)之间的距离
5- 根据距离换算成“蓄力时间“,确定换算关系

如何去除图像背景、并将其二值化?
打开命令行,截取设备屏幕图像,并获取到本地计算机D:\Temp目录下
adb shell screencap /sdcard/test.png
adb pull /sdcard/test.png D:/Temp
然后观察图像的背景特征。可以总结到以下特点:
1- 背景是纵向渐变的,不是纯色的
2- 横向上,背景颜色是相同、没有差别的
3- 背景和各个物体之间的差别是明显的
3- 我们需要用到的纵向区域,大概在第300像素-700像素之间(总高度1280像素)
4- 部分物体会侵占边沿。这个特征是在开发中补充进来的。因为这个特征,我们不可以直接取每一行最左边或者最右边的像素作为背景。

有了以上分析,我们基本就可以设计出提取背景的方案了
1- 只处理中间纵向区域的图像(300-700)
2- 尽量尝试从横向的边沿查找一个像素,作为背景色
3- 利用背景是纵向渐变的这个特征,排除物体侵占边沿的情况

二值化的过程相对来说就比较简单了,逐行、逐列扫描图像,区分前景和背景,背景设置为0,非背景的设置为1,于是得到一个[宽*高]这样大小的一个byte[]。

设计和实现
界面比较简单了,显示一些基本信息和数据,然后有开始、结束控制即可。

关键的函数有这些:

   class Helper
    {
        public static byte[] ConvertBinValue(Bitmap bitmap, int from, int to)
        {
            byte[] ret = new byte[bitmap.Width * bitmap.Height];
            Rectangle rect = new Rectangle(0, 0, bitmap.Width, bitmap.Height);
            BitmapData dat = bitmap.LockBits(rect, ImageLockMode.ReadWrite, PixelFormat.Format24bppRgb);
            int stride = dat.Stride;
            IntPtr ptr = dat.Scan0;
            int size = stride * bitmap.Height;
            byte[] values = new byte[size];
            Marshal.Copy(ptr, values, 0, size);
            for (int y = from; y < to; y++)
            {
                for (int x = 0; x < dat.Width; x++)
                {
                    int px = stride * y + 3 * x;
                    int vx = values[px + 0] + values[px + 1] + values[px + 2];
                    ret[x + y * dat.Width] = (byte)(vx == 0 ? 1 : 0);
                }
            }
            bitmap.UnlockBits(dat);
            return ret;
        }
        public static Bitmap ConvertJumper(Bitmap bitmap)
        {
            Rectangle rect = new Rectangle(0, 0, bitmap.Width, bitmap.Height);
            BitmapData bmpData = bitmap.LockBits(rect, ImageLockMode.ReadWrite, PixelFormat.Format24bppRgb);
            int iStride = bmpData.Stride;
            IntPtr ptr = bmpData.Scan0;
            int iBytes = iStride * bitmap.Height;
            byte[] values = new byte[iBytes];
            Marshal.Copy(ptr, values, 0, iBytes);
            for (int y = 0; y < bmpData.Height; ++y)
            {
                for (int x = 0; x < bmpData.Width; ++x)
                {
                    int px = iStride * y + 3 * x;
                    int times = 0;
                    if (values[px + 0] < 64) times++;
                    if (values[px + 1] < 64) times++;
                    if (values[px + 2] < 64) times++;
                    byte avg = (byte)(times >= 2 ? 0 : 255);
                    values[px + 0] = avg;
                    values[px + 1] = avg;
                    values[px + 2] = avg;
                }
            }
            bitmap.UnlockBits(bmpData);
            Bitmap target = new Bitmap(bitmap.Width, bitmap.Height, PixelFormat.Format24bppRgb);
            bmpData = target.LockBits(rect, ImageLockMode.ReadWrite, PixelFormat.Format24bppRgb);
            iStride = bmpData.Stride;
            ptr = bmpData.Scan0;
            Marshal.Copy(values, 0, ptr, values.Length);
            target.UnlockBits(bmpData);
            return target;
        }
        public static Bitmap ConvertGrayPicture(Bitmap bitmap, int from, int to)
        {
            Rectangle rect = new Rectangle(0, 0, bitmap.Width, bitmap.Height);
            BitmapData bmpData = bitmap.LockBits(rect, ImageLockMode.ReadWrite, PixelFormat.Format24bppRgb);
            int iStride = bmpData.Stride;
            IntPtr ptr = bmpData.Scan0;
            int iBytes = iStride * bitmap.Height;
            byte[] values = new byte[iBytes];
            Marshal.Copy(ptr, values, 0, iBytes);
            int vf = values[0] + values[1] + values[2];
            for (int y = 0; y < bmpData.Height; y++)
            {
                int p0 = iStride * y;
                int v0 = values[p0 + 0] + values[p0 + 1] + values[p0 + 2];
                int p1 = iStride * y + 3 * (bmpData.Width - 1);
                int v1 = values[p1 + 0] + values[p1 + 1] + values[p1 + 2];
                if (Math.Abs(v1 - vf) < Math.Abs(vf - v0)) { v0 = v1; }
                for (int x = 0; x < bmpData.Width; ++x)
                {
                    int px = iStride * y + 3 * x;
                    int vx = values[px + 0] + values[px + 1] + values[px + 2];
                    byte avg = (byte)((Math.Abs(vx - v0) > 10) ? 0 : 255);
                    values[px + 0] = avg;
                    values[px + 1] = avg;
                    values[px + 2] = avg;
                }
            }
            bitmap.UnlockBits(bmpData);
            Bitmap target = new Bitmap(bitmap.Width, bitmap.Height);
            bmpData = target.LockBits(rect, ImageLockMode.ReadWrite, PixelFormat.Format24bppRgb);
            iStride = bmpData.Stride;
            ptr = bmpData.Scan0;
            Marshal.Copy(values, 0, ptr, values.Length);
            target.UnlockBits(bmpData);
            return target;
        }
        public static void DrawCross(Bitmap bmp, Pen pen, Point pt)
        {
            using (Graphics g = Graphics.FromImage(bmp))
            {
                int x = pt.X;
                int y = pt.Y;
                int r = 10;
                g.DrawLine(pen, x - r, y + r, x + r, y - r);
                g.DrawLine(pen, x - r, y - r, x + r, y + r);
                g.DrawEllipse(pen, new RectangleF(x - r, y - r, 2 * r, 2 * r));
            }
        }
        public static void DrawImage(Bitmap bmp, Control control)
        {
            if (bmp != null)
            {
                Rectangle src = new Rectangle(0, 0, bmp.Width, bmp.Height);
                Rectangle dest = new Rectangle(0, 0, control.Width, control.Height);
                using (Graphics g = Graphics.FromHwnd(control.Handle))
                {
                    g.DrawImage(bmp, dest, 0, 0, src.Width, src.Height, GraphicsUnit.Pixel);
                }
            }
        }
        public static String ExecuteAdb(String args)
        {
            Process p = new Process();
            p.StartInfo = new ProcessStartInfo("adb.exe");
            p.StartInfo.Arguments = args;
            p.StartInfo.CreateNoWindow = true;
            p.StartInfo.RedirectStandardOutput = true;
            p.StartInfo.UseShellExecute = false;
            p.Start();
            String output = p.StandardOutput.ReadToEnd();
            p.WaitForExit();
            return output;
        }
        public static Device[] List()
        {
            String text = Helper.ExecuteAdb("devices");
            String[] ps = text.Split(new char[] { '\r', '\n' }, StringSplitOptions.RemoveEmptyEntries);
            if (ps.Length > 0)
            {
                if (String.Compare(ps[0].Trim(), "List of devices attached") == 0)
                {
                    List<Device> ds = new List<Device>();
                    for (int i = 1; i < ps.Length; i++)
                    {
                        String[] ts = ps[i].Split('\t');
                        if (ts.Length == 2)
                        {
                            Device d = new Device();
                            d.Name = ts[0];
                            d.Status = ts[1];
                            ds.Add(d);
                        }
                    }
                    return ds.ToArray();
                }
            }
            return null;
        }
        public static Stream GetResource(String name)
        {
            Type type = typeof(Helper);
            string _namespace = type.Namespace;
            Assembly _assembly = Assembly.GetExecutingAssembly();
            string resourceName = _namespace + "." + name;
            return _assembly.GetManifestResourceStream(resourceName);
        }
    }


// 负责执行实际计算的类型
    class Jumper
    {
        public Bitmap RawImage { get; private set; }
        public Bitmap GrayImage { get; private set; }
        public Bitmap JumperImage { get; private set; }
        public Point JumperPoint { get; private set; }
        public Point TargetPoint { get; private set; }
        public int TargetWidth { get; private set; }
        public int HoldTime { get; private set; }
        public int Distance { get; private set; }
        public int From { get; private set; }
        public int To { get; private set; }
        public float RatioX { get; private set; }
        public float RatioY { get; private set; }
        public float Ratio { get; private set; }
        public float AdjustRatio { get; private set; }
        public int Width { get; private set; }
        public int Height { get; private set; }
        public void Reset()
        {
            From = 0;
            To = 0;
            Ratio = 1f;
            AdjustRatio = 1.0f;
        }
        public void Process(String filename)
        {
            using (Image img = Bitmap.FromFile(filename))
            {
                this.Width = img.Width;
                this.Height = img.Height;
                if (this.Height > 1920)
                {
                    this.AdjustRatio = 1.10f;
                }
                RatioX = 720f / img.Width;
                RatioY = 1280f / img.Height;
                float mk = (float)Math.Sqrt(720 * 720 + 1280 * 1280);
                float tg = (float)Math.Sqrt(img.Width * img.Width + img.Height * img.Height);
                Ratio = mk / tg;
                From = (int)(300 / RatioY);
                To = (int)(900 / RatioY);
                this.RawImage = new Bitmap(img.Width, img.Height);
                using (Graphics g = Graphics.FromImage(this.RawImage))
                {
                    g.DrawImage(img, 0, 0);
                }
                JumperImage = Helper.ConvertJumper(this.RawImage);
                JumperPoint = FindJumper(JumperImage);
                this.GrayImage = Helper.ConvertGrayPicture(this.RawImage, From, To);
                PointEx pe = FindTarget(this.GrayImage, JumperPoint); ;
                TargetPoint = pe.ToPoint();
                TargetWidth = pe.Width;
                // 计算目标点和起点的位置
                int dx = TargetPoint.X - JumperPoint.X;
                int dy = TargetPoint.Y - JumperPoint.Y;
                // 目标哦距离
                this.Distance = (int)Math.Sqrt(dx * dx + dy * dy);
                int dist = (int)(this.Distance * Ratio);
                int width = (int)(this.TargetWidth * RatioX);
                // 应该蓄力的时间
                this.HoldTime = (int)(dist * (width < 100 ? 2.0f : 1.85f) * AdjustRatio);
            }
        }
        /// <summary>
        /// 查找落脚点
        /// </summary>
        /// <param name="bitmap"></param>
        /// <param name="jp"></param>
        /// <returns></returns>
        private PointEx FindTarget(Bitmap bitmap, Point jp)
        {
            int MarkCount = (int)(15 / RatioX);
            int MarkDeep = (int)(50 / RatioY);
            byte[] values = Helper.ConvertBinValue(bitmap, From, To);
            int width = bitmap.Width;
            int height = bitmap.Height;
            int from = 0;
            int to = bitmap.Width / 2;
            if (jp.X < to)
            {
                from = bitmap.Width / 2;
                to = bitmap.Width;
            }
            for (int y = From; y < To; y++)
            {
                int count = 0;
                int tx = 0;
                for (int x = from; x < to; x++)
                {
                    if (values[x + y * width] == 1)
                    {
                        tx = x;
                        count++;
                    }
                }
                if (count >= MarkCount)
                {
                    int x = tx - count / 2;
                    while (--y > From)
                    {
                        if (values[x + y * width] == 0)
                        {
                            // 往下探索20行
                            int lt = 0;
                            for (int i = 0; i < MarkDeep; i++)
                            {
                                int tt = 0;
                                for (int bx = from; bx < to; bx++)
                                {
                                    if (values[bx + (y + i) * width] == 1)
                                    {
                                        tt++;
                                    }
                                }
                                if (lt > 0)
                                {
                                    if (tt - lt <= 0)
                                    {
                                        return new PointEx(x, y, tt);
                                    }
                                }
                                lt = tt;
                            }
                            return new PointEx(x, y, to - from);
                        }
                    }
                    return new PointEx(x, y, to - from);
                }
            }
            return new PointEx(0, 0, 0);
        }
        /// <summary>
        /// 在图像中,查找起跳点
        /// </summary>
        /// <param name="bitmap">图像数据</param>
        /// <returns>起跳点图像中的位置</returns>
        public Point FindJumper(Bitmap bitmap)
        {
            int M1 = (int)(30 / RatioX);
            int M2 = (int)(80 / RatioX);
            int M3 = (int)(12 / RatioX);
            byte[] values = Helper.ConvertBinValue(bitmap, From, To);
            int width = bitmap.Width;
            int height = bitmap.Height;
            for (int y = From; y < To; ++y)
            {
                for (int x = 100; x < width - 100; ++x)
                {
                    int vx = values[x + y * width];
                    bool found = vx == 1;
                    for (int tx = 0; found && tx < M1; tx++)
                    {
                        vx = values[x + tx + (y * width)];
                        found = vx == 1;
                    }
                    for (int ty = 0; found && ty < M2; ty++)
                    {
                        vx = values[x + (y - ty) * width];
                        found = vx == 1;
                    }
                    if (found)
                    {
                        return new Point(x + M3, y);
                    }
                }
            }
            return Point.Empty;
        }
    }

程序和代码
程序可以从http://caoliu-tek.com/jump下载。注意:程序中有广告信息,反感者慎入。
源码过两天整理一下放上来。等不及得下载上面得程序之后,请直接Reflector。


遗留问题
  • 获取得目标点是近似的,不准确,待改进。但跳到3000多分足够了,不改也挺好的。
  • Jumper里面可以先计算出二值化,然后其他操作都直接基于二值化后的数组,不需要再基于Bitmap。
  • 边沿侵占的情况下,背景处理有异常。基本无害,所以没再处理了。
  • 不同设备上,Device类定时重复读取的间隔不同。目前6秒。
  • 没有详细处理图像不合法的情况。


////////////////////////////////////////////////////
谭小楼,成都,201801







展开阅读全文

没有更多推荐了,返回首页