黑帽python第二版(Black Hat Python 2nd Edition)读书笔记 之 第二章 网络工程基础(2)创建一个TCP代理
我们有众多的理由以使得我们在工具集中拥有一个TCP代理。你可能会使用代理进行流量转发以实现主机到主机的反弹(啥意思?一脸懵逼,继续往下看吧),或者评估基于网络的软件。当在企业环境中执行渗透测试时,测试人员可能无法使用类似Wireshark,也无法通过加载驱动软件来嗅探windows下的环回地址,并且网段本身可以也会直接阻止工具直接达到目标主机。我们将在这里搭建简单的python代理,它将会协助我们了解未知协议、修改发送到应用程序的流量、为模糊测试工具创建用例等。
这里实现的TCP代理主要包含四个功能:将本地主机和远程主机之间的交互展示到控制台上(hexdump);通过socket从主机(本地或者远端)接收数据(receive_from);直接管理在本地主机和远端主机之间的流量(proxy_handler);配置监听socket并提交到proxy_handler(server_loop)。直接上代码。
import sys
import socket
import threading
HEX_FILTER = ''.join(
[(len(repr(chr(i))) == 3) and chr(i) or '.' for i in range(256)]
)
def hexdump(src, length=16, show=True):
if isinstance(src, bytes):
src = src.decode()
results = list()
for i in range(0, len(src), length):
word = str(src[i:i+length])
printable = word.translate(HEX_FILTER)
hexa = ' '.join([f'{ord(c):02X}' for c in word])
hexwidth = length*3
results.append(f'{i:04x} {hexa:<{hexwidth}} {printable}')
if show:
for line in results:
print(line)
else:
return results
我们从几个入口开始。定义一个hexdump函数,用来将一些输入作为字节或者字符串,并将转储的16进制打印到控制台。这意味着,工具将会输出整个包的细节,包含他们的十六进制值以及ASCII可打印字符。这对理解未知协议、在明文协议中查找用户凭据,以及其它很多活动是非常有用的。 对于0到255范围之内的每个整数,如果相应字符的长度等于3,则会得到一个字符(chr(i));否则将会得到一个点(.)。当我们将这些内容连接到一个字符串中的时候,就出现了如下图所示的样子(这是将上述代码中的HEX_FILTER用print函数打印出来的效果)。
上图中的列表给出了前256个整数的可打印字符表示。对于创建的hexdump函数,我们可以在上述代码的最后添加如下的一行(这是测试代码,验证没问题后直接注释掉即可)。
hexdump('python rocks\n and proxies roll\n')
然后直接运行上述代码,即可得到如下图所示的运行结果,跟原书中的输出结果一致。
hexdump函数提供了一种实时查看通过proxy的通信的方式。接下来将会创建一个函数,用于在proxy的两端接收数据。
def receive_from(connection):
buffer = b''
connection.settimeout(5)
try:
while True:
data = connection.recv(4096)
if not data:
break
buffer += data
except Exception as e:
pass
return buffer
为了既能够接收本地的数据,又能够接收远端的数据,我们传入要使用的socket对象。我们创建了一个空的字符串buffer,用来接收来自socket的响应。默认地,我们设置了5秒钟的超时时间,读者可以根据实际需要调整这个超时时间。另外,我们设置了一个循环,用于将响应数据读入buffer中,直到没有更多的数据,或者超时。最后我们返回buffer字节的字符串给调用者(调用者可能是本地或者远端的机器)。
有时候,读者可能希望在proxy转发数据到对应的接收者(或者调用者)之前,修改响应(或者请求)数据包。下面我们添加两个函数(request_handler和response_handler)来实现这个功能。
def request_handler(buffer):
# perform packet modifications
return buffer
def response_handler(buffer):
# perform packet modifications
return buffer
在这两个函数中,我们可以修改数据包的内容、执行fuzzing任务、测试身份验证问题,或者做任何我们希望做的其它事情。比如,如果你发现了明文的用户凭据在发送,并且企图尝试在应用程序中使用admin替换自己的用户名进行提权。接下来,我们使用如下的代码来深入研究proxy_handler函数。
def proxy_handler(client_socket, remote_host, remote_port, receive_first):
remote_socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
remote_socket.connect((remote_host, remote_port))
if receive_first:
remote_buffer = receive_from(remote_socket)
hexdump(remote_buffer)
remote_buffer = response_handler(remote_buffer)
if len(remote_buffer):
print("[<==] Sending %d bytes to localhost." % len(remote_buffer))
client_socket.send(remote_buffer)
while True:
local_buffer = receive_from(client_socket)
if len(local_buffer):
line = "[==>]Received %d bytes from localhost." % len(local_buffer)
print(line)
hexdump(local_buffer)
local_buffer = request_handler(local_buffer)
remote_socket.send(local_buffer)
print("[==>] Sent to remote.")
remote_buffer = receive_from(remote_socket)
if len(remote_buffer):
print("[<==] Received %d bytes from remote." % len(remote_buffer))
hexdump(remote_buffer)
remote_buffer = response_handler(remote_buffer)
client_socket.send(remote_buffer)
print("[<==] Sent to localhost.")
if not len(local_buffer) or not len(remote_buffer):
client_socket.close()
remote_socket.close()
print("[*] No more data. Closing connection.")
break
接下来,将会组合server_loop函数,来设置和管理连接。
def server_loop(local_host, local_port, remote_host, remote_port, receive_first):
server = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
try:
server.bind((local_host, local_port))
except Exception as e:
print('problem on bind: %r' % e)
print("[!!] Faild to listen on %s:%d" % (local_host, local_port))
print("[!!] Check for other listening sockets or correct permissions.")
sys.exit(0)
print("[*] Listening on %s:%d" % (local_host, local_port))
server.listen(5)
while True:
client_socket, addr = server.accept()
# print out the local connection information
line = "> Received incomming connection from %s:%d" % (addr[0], addr[1])
print(line)
# start a thread to talk to the remote host
proxy_thread = threading.Thread(
target=proxy_handler,
args=(client_socket, remote_host,
remote_port, receive_first)
)
proxy_thread.start()
上述代码中,先是创建一个socket,然后将socket对象绑定到本地的主机上并进行监听。之后,创建一个主循环,当新的连接请求进来的时候,创建一个新的线程将连接请求提交给proxy_handler。
到目前为止,唯一剩余没有完成的代码就是主函数main。
def main():
if len(sys.argv[1:]) != 5:
print("Usage: ./proxy.py [localhost] [localport]", end = '')
print("[remotehost] [remoteport] [receive_first]")
print("Example: ./proxy.py 127.0.0.1 9000 10.12.132.1 9000 True")
sys.exit(0)
local_host = sys.argv[1]
local_port = int(sys.argv[2])
remote_host = sys.argv[3]
remote_port = int(sys.argv[4])
receive_first = sys.argv[5]
if "True" in receive_first:
receive_first = True
else:
receive_first = False
server_loop(local_host, local_port, remote_host, remote_port, receive_first)
if __name__ == '__main__':
main()
在main函数中,我们引入了一些命令行参数,然后启动循环服务来监听连接。
小试牛刀
到目前为止,我们已经拥有了核心的代理循环,以及所有的支持函数,现在测试一下。通过如下的命令,启动脚本。
$ sudo python 2_005_Proxy.py 192.168.65.141 21 ftp.sun.ac.za 21 True
这里使用sudo,是因为21端口是特权端口,监听21端口需要root权限。启动脚本之后,程序已经开始监听,如下图所示。
然后另起一个shell命令行,ftp到本机,是有反应的,不过可能国内访问 ftp.sun.ac.za可能是不可达,如下图。
我猜测代理至少起作用了,但是国外服务器返回timeout,先手工试一下这个ftp是否可达。
完犊子了,同一台机器,直接ftp是可以过去的,但是通过代理就出问题了,打开BeyondCompare比对一下我写的代码和官网下载的源代码,修改到保持一致,还是不行。
上面既然打出了Received incomming connection from,说明已经进入main函数,并且执行到了main函数的最后一行,进入到server_loop函数里面了;继续追踪,实际上这之后,创建了线程,然后在线程里面运行的proxy_handler函数。通过各种埋点调测,发现实际上脚本已经收到了ftp站点返回的“220 Welcome to ftp.sun.ac.za”,然而下一步是等待用户输入登录ftp的用户名(这里是anonymous)和密码并确认,我们的脚本目前不支持给ftp服务器输入用户名和密码,导致超时。现在问题应该比较明朗了,原书中用的是ftp作为tcp代理的示例,我们换一种思路,直接curl一个网页试试看。先通过如下命令,运行脚本,用百度站点测试一下。
$ sudo python 2_005_Proxy.py 192.168.65.141 8887 www.baidu.com 80 True
开始监听了,如下图所示。
接下来,另起一个shell终端,通过curl本地的IP地址和端口来请求百度。
$ curl -x 192.168.65.141:8887 www.baidu.com
这次获取到了网页数据,说明proxy是可以工作的,如下图。
下图是在kali机器上运行的代理脚本中dump出来的对应内容。
接下来附上完整的代码。
import sys
import socket
import threading
# create a HEXFILTER string, the string contains ASCII printable characters
HEX_FILTER = ''.join(
[(len(repr(chr(i))) == 3) and chr(i) or '.' for i in range(256)]
)
# define the hexdump function
def hexdump(src, length=16, show=True):
# if a bytes string is passed in, decoding the bytes
if isinstance(src, bytes):
src = src.decode()
results = list()
for i in range(0, len(src), length):
# grab a piece of the string to dump and put it into the word variable
word = str(src[i:i+length])
# using the built-in function translate to substitute the string representation of each character for the printable character
printable = word.translate(HEX_FILTER)
hexa = ' '.join([f'{ord(c):02X}' for c in word])
hexwidth = length*3
#
results.append(f'{i:04x} {hexa:<{hexwidth}} {printable}')
if show:
for line in results:
print(line)
else:
return results
def receive_from(connection):
# create a null string to receive response from socket
buffer = b''
connection.settimeout(10)
try:
while True:
data = connection.recv(4096)
if not data:
break
buffer += data
except Exception as e:
print("Error: ", e)
pass
return buffer
def request_handler(buffer):
# perform packet modifications
return buffer
def response_handler(buffer):
# perform packet modifications
return buffer
def proxy_handler(client_socket, remote_host, remote_port, receive_first):
remote_socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
remote_socket.connect((remote_host, remote_port))
# make sure we don't need to first initiate a connection to the remote side and request data before going into the main loop
if receive_first:
# using the receive_from function to accepts a connected socket object and perform a receive
remote_buffer = receive_from(remote_socket)
if len(remote_buffer):
print("[<==] Received %d bytes from remote." % len(remote_buffer))
# dump the contents of the packet
hexdump(remote_buffer)
# set up a loop to continually read from the local client, and process the data,send it to the remote client...
while True:
# using the receive_from function to accepts a connected socket object and perform a receive
local_buffer = receive_from(client_socket)
if len(local_buffer):
line = "[<==]Received %d bytes from local." % len(local_buffer)
print(line)
# dump the contents of the packet
hexdump(local_buffer)
local_buffer = request_handler(local_buffer)
remote_socket.send(local_buffer)
print("[==>] Sent to remote.")
remote_buffer = receive_from(remote_socket)
if len(remote_buffer):
print("[<==] Received %d bytes from remote." % len(remote_buffer))
hexdump(remote_buffer)
remote_buffer = response_handler(remote_buffer)
client_socket.send(remote_buffer)
print("[==>] Sent to local.")
if not len(local_buffer) or not len(remote_buffer):
client_socket.close()
remote_socket.close()
print("[*] No more data. Closing connection.")
break
def server_loop(local_host, local_port, remote_host, remote_port, receive_first):
# create a socket
server = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
try:
# binds the socket to the local host and listens
server.bind((local_host, local_port))
except Exception as e:
print('problem on bind: %r' % e)
print("[!!] Faild to listen on %s:%d" % (local_host, local_port))
print("[!!] Check for other listening sockets or correct permissions.")
sys.exit(0)
print("[*] Listening on %s:%d" % (local_host, local_port))
server.listen(5)
# create a loop
while True:
client_socket, addr = server.accept()
# print out the local connection information
line = "> Received incomming connection from %s:%d" % (addr[0], addr[1])
print(line)
# start a thread to talk to the remote host
# when a fresh connection request comes in, hand the connection off to the proxy_handler in a new thread
proxy_thread = threading.Thread(
target=proxy_handler,
args=(client_socket, remote_host,
remote_port, receive_first)
)
proxy_thread.start()
def main():
if len(sys.argv[1:]) != 5:
print("Usage: ./proxy.py [localhost] [localport]", end = '')
print("[remotehost] [remoteport] [receive_first]")
print("Example: ./proxy.py 127.0.0.1 9000 10.12.132.1 9000 True")
sys.exit(0)
local_host = sys.argv[1]
local_port = int(sys.argv[2])
remote_host = sys.argv[3]
remote_port = int(sys.argv[4])
receive_first = sys.argv[5]
if "True" in receive_first:
receive_first = True
else:
receive_first = False
server_loop(local_host, local_port, remote_host, remote_port, receive_first)
if __name__ == '__main__':
main()
在下一节中我们将介绍基于paramiko的SSH。