基于littleproxy代理抓取android,ios的app的数据接口的方法

场景描述:
有时候我们会遇到需要抓取某些app的数据的需求,比如新闻客户端,或者微信的数据接口
这时候我们可以用wireshark来截取流量,或者fildder,或者charles。

但是如果数据接口是https的话,我们需要自己颁发ca证书,需要手机信任证书,下面是charles截取https流量的方法,写的很详细
https://www.cnblogs.com/weiming4219/p/7908668.html
这里android反倒不容易,因为android的http proxy不是全局代理,是仅限于浏览器的http proxy,需要额外安装软件达到全局代理的目的,而且还需要root,现在的手机厂商已经很少开放root权限了,我前几年还会折腾root,现在大家也越来越少折腾root了,所以不方便。
而ios的wifi设置下面的http proxy是全局的,所有app都生效了。所以这里如果有iphone的话,会比较方便。
ios只需要安装ca证书,并且信任即可。

但是这里要介绍的不是通过charles来达成流量监听目的的做法,而是用编程,我们自己借助littleproxy的http proxy代理服务器达成我们的目的

介绍一下littleproxy是一款基于java的http代理工具,包括了中间人攻击模块。
https://github.com/adamfisk/LittleProxy
具体可以点进去看看。
接下来我们实操
新建一个java工程,maven项目
把依赖添加进去
 


      <dependency>
          <groupId>org.littleshoot</groupId>
          <artifactId>littleproxy</artifactId>
          <version>1.1.2</version>
      </dependency>
      <dependency>
          <groupId>com.github.ganskef</groupId>
          <artifactId>littleproxy-mitm</artifactId>
          <version>1.1.0</version>
      </dependency>
      <dependency>
          <groupId>net.lightbody.bmp</groupId>
          <artifactId>mitm</artifactId>
          <version>2.1.4</version>
      </dependency>

核心代码就80行
 

 public static void way1(){
        try {
            HttpProxyServer server =
                    DefaultHttpProxyServer.bootstrap()
                            .withAddress(new InetSocketAddress("192.168.55.100",9090))
                            //.withPort(9090) // for both HTTP and HTTPS
                            .withManInTheMiddle(new CertificateSniffingMitmManager())
                            .withFiltersSource(new HttpFiltersSourceAdapter() {
                                public HttpFilters filterRequest(HttpRequest originalRequest, ChannelHandlerContext ctx) {
                                    return new HttpFiltersAdapter(originalRequest) {
                                        @Override
                                        public HttpResponse clientToProxyRequest(HttpObject httpObject) {
                                            // TODO: implement your filtering here
                                            //System.out.println(httpObject);
//                                            if(httpObject instanceof DefaultHttpRequest){
//                                                System.out.println("req:"+((DefaultHttpRequest) httpObject).getUri());
//                                                System.out.println(originalRequest.getUri());
//                                            }
                                            return null;
                                        }

                                        @Override
                                        public HttpObject serverToProxyResponse(HttpObject httpObject) {
                                            try{
                                                responseMap.putIfAbsent(originalRequest,new MyResponse());
                                                MyResponse myResponse = responseMap.get(originalRequest);
                                                if (httpObject instanceof HttpResponse) {
                                                    myResponse.setHttpHeaders(((HttpResponse) httpObject).headers());
                                                } else if (httpObject instanceof HttpContent) {
                                                    HttpHeaders httpHeaders = myResponse.getHttpHeaders();
                                                    if(httpHeaders!=null){
                                                        String contentType = httpHeaders.get("Content-Type");
                                                        if(contentType!=null && !contentType.contains("image") && !contentType.contains("audio")
                                                               &&!contentType.contains("zip") && !contentType.contains("application/octet-stream")
                                                                ){
                                                            ByteBuf buf = ((HttpContent) httpObject).content();
                                                            buf.markReaderIndex();
                                                            byte[] array = new byte[buf.readableBytes()];
                                                            buf.readBytes(array);
                                                            buf.resetReaderIndex();
                                                            myResponse.appendByte(array);
                                                        }
                                                    }
                                                }
                                                if(httpObject instanceof LastHttpContent){
                                                    if(myResponse.getContent()!=null){
                                                        System.out.println(originalRequest.getUri());
                                                        HttpHeaders httpHeaders = myResponse.getHttpHeaders();
                                                        myResponse.printHeader();
                                                        String ce = httpHeaders.get("Content-Encoding");
                                                        if(ce!=null&&ce.contains("gzip")){
                                                            if(myResponse.getContent()!=null){
                                                                ByteArrayInputStream bais = new ByteArrayInputStream(myResponse.getContent());
                                                                GZIPInputStream gzis = new GZIPInputStream(bais);
                                                                byte[] decompressedData = IOUtils.toByteArray(gzis);
                                                                System.out.println(new String(decompressedData,"utf-8"));
                                                            }
                                                        }else {
                                                            if(myResponse.getContent()!=null){
                                                                System.out.println(new String(myResponse.getContent(),"utf-8"));
                                                            }
                                                        }
                                                    }
                                                }
                                            }catch (Exception e){
                                                e.printStackTrace();
                                            }
                                            return httpObject;
                                        }


                                    };
                                }
                            })
                            .start();
        } catch (Exception e) {
            e.printStackTrace();
        }
    }


    public static void main(String[] args){
        way1();
    }
public class MyResponse {

    private StringBuffer body = new StringBuffer();
    private HttpHeaders httpHeaders;
    private byte [] content;

    public byte[] getContent() {
        return content;
    }

    public synchronized void appendByte(byte [] array){
        if(content==null){
            content = array;
        }else {
            byte []temp = new byte[content.length+array.length];
            System.arraycopy(content,0,temp,0,content.length);
            System.arraycopy(array,0,temp,content.length,array.length);
            content = temp;
        }
    }

    public StringBuffer getBody() {
        return body;
    }

    public void printHeader(){
        if(httpHeaders!=null){
            httpHeaders.forEach(stringStringEntry -> {
                System.out.println(stringStringEntry.getKey()+":"+stringStringEntry.getValue());
            });
        }
    }

    public void setBody(StringBuffer body) {
        this.body = body;
    }

    public HttpHeaders getHttpHeaders() {
        return httpHeaders;
    }

    public void setHttpHeaders(HttpHeaders httpHeaders) {
        this.httpHeaders = httpHeaders;
    }
}

首先我们要绑定ip到本地机器,设置http代理的端口,然后设置中间人攻击模块
 

DefaultHttpProxyServer.bootstrap()
                            .withAddress(new InetSocketAddress("192.168.55.100",9090))
                            //.withPort(9090) // for both HTTP and HTTPS
                            .withManInTheMiddle(new CertificateSniffingMitmManager())

如果你用默认的withPort的话,它是绑定在127.0.0.1的,那么局域网的其它设备,手机等是没法设置代理的。
0b3f4df2924be903313e86e54c1ac4b610f.jpg

然后他会生成两个文件,请搭建一个静态文件服务器,让ios的safari浏览器访问pem文件,然后safari会提示你是否安装该文件描述,请安装
c947ab0fb75e7f57164f7c60efc9881ed07.jpg

f2f16805dd0e92b1900eb2dcd51d2d0cdf2.jpg

接下来设置我们的httpfilter
 

.withFiltersSource(new HttpFiltersSourceAdapter() {
                                public HttpFilters filterRequest(HttpRequest originalRequest, ChannelHandlerContext ctx) {
                                    return new HttpFiltersAdapter(originalRequest) {
                                        @Override
                                        public HttpResponse clientToProxyRequest(HttpObject httpObject) {
                                            // TODO: implement your filtering here
                                            //System.out.println(httpObject);
//                                            if(httpObject instanceof DefaultHttpRequest){
//                                                System.out.println("req:"+((DefaultHttpRequest) httpObject).getUri());
//                                                System.out.println(originalRequest.getUri());
//                                            }
                                            return null;
                                        }

                                        @Override
                                        public HttpObject serverToProxyResponse(HttpObject httpObject) {
                                            try{
                                                responseMap.putIfAbsent(originalRequest,new MyResponse());
                                                MyResponse myResponse = responseMap.get(originalRequest);
                                                if (httpObject instanceof HttpResponse) {
                                                    myResponse.setHttpHeaders(((HttpResponse) httpObject).headers());
                                                } else if (httpObject instanceof HttpContent) {
                                                    HttpHeaders httpHeaders = myResponse.getHttpHeaders();
                                                    if(httpHeaders!=null){
                                                        String contentType = httpHeaders.get("Content-Type");
                                                        if(contentType!=null && !contentType.contains("image") && !contentType.contains("audio")
                                                               &&!contentType.contains("zip") && !contentType.contains("application/octet-stream")
                                                                ){
                                                            ByteBuf buf = ((HttpContent) httpObject).content();
                                                            buf.markReaderIndex();
                                                            byte[] array = new byte[buf.readableBytes()];
                                                            buf.readBytes(array);
                                                            buf.resetReaderIndex();
                                                            myResponse.appendByte(array);
                                                        }
                                                    }
                                                }
                                                if(httpObject instanceof LastHttpContent){
                                                    if(myResponse.getContent()!=null){
                                                        System.out.println(originalRequest.getUri());
                                                        HttpHeaders httpHeaders = myResponse.getHttpHeaders();
                                                        myResponse.printHeader();
                                                        String ce = httpHeaders.get("Content-Encoding");
                                                        if(ce!=null&&ce.contains("gzip")){
                                                            if(myResponse.getContent()!=null){
                                                                ByteArrayInputStream bais = new ByteArrayInputStream(myResponse.getContent());
                                                                GZIPInputStream gzis = new GZIPInputStream(bais);
                                                                byte[] decompressedData = IOUtils.toByteArray(gzis);
                                                                System.out.println(new String(decompressedData,"utf-8"));
                                                            }
                                                        }else {
                                                            if(myResponse.getContent()!=null){
                                                                System.out.println(new String(myResponse.getContent(),"utf-8"));
                                                            }
                                                        }
                                                    }
                                                }
                                            }catch (Exception e){
                                                e.printStackTrace();
                                            }
                                            return httpObject;
                                        }


                                    };
                                }
                            })
                            .start();

代码可以看出我们过滤了image,zip等二进制包,同样对于gzip压缩的json格式,也做了很好的处理。
可以看看具体效果:
9d893071a0ee7d4909834be883b132417a0.jpg
word就是测试
e8314f1a6039446ae23e65ed29ebf633dd9.jpg

b1bb0a6037e6072726d9a4ac905c5d05dbb.jpg

这里就是返回的百度的内容,可以看出https又如何?还是被截获流量,其它app里面的内容同理可以做到。
开源中国app的接口:
b2c42cffcda411f6811d28fd670f23a1e04.jpg

转载于:https://my.oschina.net/zxcholmes/blog/1929710

  • 0
    点赞
  • 0
    收藏
    觉得还不错? 一键收藏
  • 0
    评论

“相关推荐”对你有帮助么?

  • 非常没帮助
  • 没帮助
  • 一般
  • 有帮助
  • 非常有帮助
提交
评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值