用google colab t4部署phi2（公网可访问）

目录

1.打开google colab，配置GPU

2.ngrok获取authtoken

3.安装所需的包​编辑

4.运行推理服务，生成公网地址

5.client.py访问上述推理服务

6.gradio界面版

---

1.打开google colab，配置GPU

2.ngrok获取authtoken

https://dashboard.ngrok.com/get-started/your-authtoken

3.安装所需的包

4.运行推理服务，生成公网地址

生成的ngrok-free.app链接就是推理服务的公网地址

5.client.py访问上述推理服务

import requests

def format_prompt(prompt: str) -> str:
    return f'##提问:\n{prompt}\n##回答:\n'


def ask_phi2(query: str) -> str:
    url = "http://1111-11-111-11-11.ngrok-free.app/generate"
    
    body = {
        "query": format_prompt(query)
    }
    response = requests.post(url=url, json=body)
    if response.status_code == 200:
        print("Response from server:", response.text)
    else:
        print("Failed to get response from server, status code:", response.status_code)


if __name__ == '__main__':   
    ask_phi2("你好") 


    
    

6.gradio界面版