AI之AutoML：Ludwig(无需编写代码/易于使用的界面和可视化自动机器学习工具)的简介、安装、使用方法之详细攻略
目录
Ludwig(无需编写代码/易于使用的界面和可视化自动机器学习工具)的简介
Ludwig的简介
Ludwig(无需编写代码/易于使用的界面和可视化自动机器学习工具)的简介
| 简介 | Ludwig是Uber AI Labs在2018年开源的一款自动机器学习框架。它允许开发者以非常简单的方式构建深度学习系统，而无需编写过多的代码。Ludwig适用于广泛的任务，包括分类、回归、序列建模、特征提取与聚类。 |
| 核心技术点 | Ludwig利用TensorFlow等深度学习框架实现各种模型与组件，它的核心是抽象与模块化。通过简单的YAML格式，开发者便可以构建复杂的深度学习模型与流程，避免编写过多Utility代码。 自动特征工程：Ludwig可以自动对输入数据进行特征提取和工程处理，从而减少了手动特征工程的复杂性。 自动模型选择：Ludwig能够自动选择最佳的模型架构和超参数，从而最大化模型的性能和准确度。 分布式训练：Ludwig支持分布式训练，可以在多个GPU或CPU上加速训练过程。 可解释性：Ludwig提供了可解释性的分析工具，可以帮助用户理解模型的预测结果。 |
| 功能 | 支Ludwig支持构建深度学习模型进行分类、回归、序列建模与特征提取等任务。它实现了DNN、CNN、RNN等模型，还支持模型组合与Ensemble。持多种类型的数据输入，包括文本、图像、音频和结构化数据。 支持多种任务，包括分类、回归、文本分类、序列标注、图像分类和生成等任务。 提供了丰富的预处理和后处理函数，例如标准化、归一化、缺失值填充、词嵌入等。 提供了多种评估指标，包括准确度、精确度、召回率、F1分数等。 |
| 特点 | 易于使用：Ludwig提供了简单易用的命令行接口和图形用户界面，用户可以通过简单的配置YAML文件来训练和测试模型，无需编写过多代码。 模块化：各个组件间松耦合，可以方便替换。 高性能：基于TensorFlow等框架实现，可以高效利用GPU等硬件资源。 可扩展性：Ludwig提供了灵活的插件机制，可以方便地添加自定义的数据预处理、模型架构和评估指标等功能。 高度可配置：Ludwig提供了大量的配置选项，可以对模型进行详细的定制和调整，以适应各种应用场景和需求。 |
| 案例应用 | Uber将Ludwig广泛应用于各业务领域，如需求预测、异常检测、客户分群等。 例如，Uber Eats利用Ludwig提高菜品推荐的准确率，Uber Freight也使用Ludwig完成客户细分与需求预测。 |
GitHUb：GitHub - ludwig-ai/ludwig: Data-centric declarative deep learning framework
Ludwig的安装
pip install ludwig
pip install -i https://pypi.tuna.tsinghua.edu.cn/simple ludwigMicrosoft Windows [版本 10.0.19044.2728]
(c) Microsoft Corporation。保留所有权利。
C:\Windows\system32>pip install -i https://pypi.tuna.tsinghua.edu.cn/simple ludwig
Looking in indexes: https://pypi.tuna.tsinghua.edu.cn/simple
Collecting ludwig
Downloading https://pypi.tuna.tsinghua.edu.cn/packages/01/46/73a9a279ce63ab0de902ec472bf27c7f86779d72ca3de7dc68f47b374336/ludwig-0.7.4.tar.gz (874 kB)
|████████████████████████████████| 874 kB 939 kB/s
Installing build dependencies ... done
Getting requirements to build wheel ... done
Preparing wheel metadata ... done
Requirement already satisfied: absl-py in d:\programdata\anaconda3\lib\site-packages (from ludwig) (1.4.0)
Collecting experiment-impact-tracker
Downloading https://pypi.tuna.tsinghua.edu.cn/packages/3c/ce/19d9b83a78998aaaf195af8217dc963004595156d3497c4f490dc7680d1e/experiment_impact_tracker-0.1.8-py2.py3-none-any.whl (10.5 MB)
|████████████████████████████████| 10.5 MB 656 kB/s
Collecting pyarrow
Using cached https://pypi.tuna.tsinghua.edu.cn/packages/5c/a0/9ec61020d9bf28cb965f214040770a5c5dbefd009aaeeb1a0cb0d50a66c4/pyarrow-11.0.0-cp39-cp39-win_amd64.whl (20.6 MB)
Collecting jsonschema<4.7,>=4.5.0
Downloading https://pypi.tuna.tsinghua.edu.cn/packages/1c/4d/b6451349302b995170687c86b53630a3117f4a94ea2432e24b4d6df859f2/jsonschema-4.6.2-py3-none-any.whl (80 kB)
|████████████████████████████████| 80 kB 2.5 MB/s
Collecting torchinfo
Downloading https://pypi.tuna.tsinghua.edu.cn/packages/89/b9/24371b25084b9368650703ecfd652669705dbf797df6e1d2c7dcc9146c15/torchinfo-1.7.2-py3-none-any.whl (22 kB)
Requirement already satisfied: numpy>=1.15 in d:\programdata\anaconda3\lib\site-packages (from ludwig) (1.21.5)
Collecting psutil==5.9.4
Downloading https://pypi.tuna.tsinghua.edu.cn/packages/25/6e/ba97809175c90cbdcd33b470e466ebf0854d15d1506e605cc0ddd284d5b6/psutil-5.9.4-cp36-abi3-win_amd64.whl (252 kB)
|████████████████████████████████| 252 kB 2.2 MB/s
Requirement already satisfied: h5py!=3.0.0,>=2.6 in d:\programdata\anaconda3\lib\site-packages (from ludwig) (3.6.0)
Requirement already satisfied: spacy>=2.3 in d:\programdata\anaconda3\lib\site-packages (from ludwig) (3.0.9)
Requirement already satisfied: tensorboard in d:\programdata\anaconda3\lib\site-packages (from ludwig) (2.11.2)
Requirement already satisfied: Cython>=0.25 in d:\programdata\anaconda3\lib\site-packages (from ludwig) (0.29.28)
Requirement already satisfied: pandas!=1.1.5,>=0.25 in d:\programdata\anaconda3\lib\site-packages (from ludwig) (1.4.2)
Collecting kaggle
Downloading https://pypi.tuna.tsinghua.edu.cn/packages/45/7f/688c68e9308c4c0975490dd2c506f79a68e06fd061f392adfbf7d4b9c31d/kaggle-1.5.13.tar.gz (63 kB)
|████████████████████████████████| 63 kB 707 kB/s
Collecting transformers<4.22,>=4.10.1
Downloading https://pypi.tuna.tsinghua.edu.cn/packages/6a/4a/d35a2140bba25a26b8c8daf74b89f3ab46ad957e998bf77b7b4305187bc5/transformers-4.21.3-py3-none-any.whl (4.7 MB)
|████████████████████████████████| 4.7 MB 1.7 MB/s
Requirement already satisfied: torchvision in d:\programdata\anaconda3\lib\site-packages (from ludwig) (0.15.1)
Collecting sacremoses
Downloading https://pypi.tuna.tsinghua.edu.cn/packages/28/78/fef8d089db5b97546fd6d1ff2e813b8544e85670bf3a8c378c9d0250b98d/sacremoses-0.0.53.tar.gz (880 kB)
|████████████████████████████████| 880 kB 3.2 MB/s
Requirement already satisfied: tables in d:\programdata\anaconda3\lib\site-packages (from ludwig) (3.6.1)
Requirement already satisfied: xlrd in d:\programdata\anaconda3\lib\site-packages (from ludwig) (2.0.1)
Requirement already satisfied: PyYAML>=3.12 in d:\programdata\anaconda3\lib\site-packages (from ludwig) (6.0)
Requirement already satisfied: lxml in d:\programdata\anaconda3\lib\site-packages (from ludwig) (4.8.0)
Requirement already satisfied: tqdm in d:\programdata\anaconda3\lib\site-packages (from ludwig) (4.64.0)
Requirement already satisfied: packaging in d:\programdata\anaconda3\lib\site-packages (from ludwig) (21.3)
Collecting dataclasses-json
Downloading https://pypi.tuna.tsinghua.edu.cn/packages/58/7e/2042610dfc8121e8119ad8b94db496d8697e4b0ef7a6e378018a2bd84435/dataclasses_json-0.5.7-py3-none-any.whl (25 kB)
Collecting torchaudio
Downloading https://pypi.tuna.tsinghua.edu.cn/packages/7c/73/2a7e548be299c1d2590eaf48056c914c1c58682baf104205cc85f7eee926/torchaudio-2.0.1-cp39-cp39-win_amd64.whl (2.1 MB)
|████████████████████████████████| 2.1 MB 1.3 MB/s
Collecting rich~=12.4.4
Downloading https://pypi.tuna.tsinghua.edu.cn/packages/13/3f/1996db12d23733e2834b9c2b094cc59c0d1ab943fedafcdb34b5c0da9ebf/rich-12.4.4-py3-none-any.whl (232 kB)
|████████████████████████████████| 232 kB 1.3 MB/s
Collecting xlwt
Downloading https://pypi.tuna.tsinghua.edu.cn/packages/44/48/def306413b25c3d01753603b1a222a011b8621aed27cd7f89cbc27e6b0f4/xlwt-1.3.0-py2.py3-none-any.whl (99 kB)
|████████████████████████████████| 99 kB 1.5 MB/s
Collecting whylogs>=1.1.9
Downloading https://pypi.tuna.tsinghua.edu.cn/packages/01/a3/4a1defc004744b8bf506cdf9ca68a2d0ccb684736a77e2e8a24c84265b68/whylogs-1.1.37-py3-none-any.whl (1.9 MB)
|████████████████████████████████| 1.9 MB 595 kB/s
Requirement already satisfied: tabulate>=0.7 in d:\programdata\anaconda3\lib\site-packages (from ludwig) (0.8.9)
Requirement already satisfied: filelock in d:\programdata\anaconda3\lib\site-packages (from ludwig) (3.6.0)
Requirement already satisfied: scikit-learn<1.2.0 in d:\programdata\anaconda3\lib\site-packages (from ludwig) (1.0.2)
Requirement already satisfied: openpyxl in d:\programdata\anaconda3\lib\site-packages (from ludwig) (3.0.9)
Collecting marshmallow-jsonschema
Downloading https://pypi.tuna.tsinghua.edu.cn/packages/f6/cf/a620a7b0a5ba2aaa52e70f95795e0cf3a7f6332a7cb432a1223b61ac654e/marshmallow_jsonschema-0.13.0-py3-none-any.whl (11 kB)
Requirement already satisfied: fsspec[http] in d:\programdata\anaconda3\lib\site-packages (from ludwig) (2022.2.0)
Requirement already satisfied: protobuf in d:\programdata\anaconda3\lib\site-packages (from ludwig) (3.19.1)
Collecting torchmetrics>=0.11
Downloading https://pypi.tuna.tsinghua.edu.cn/packages/fb/47/6e9f9b41c48750a45ad07cc6d43a2979bfc09e6989656aece97cc59cbef1/torchmetrics-0.11.4-py3-none-any.whl (519 kB)
|████████████████████████████████| 519 kB 2.2 MB/s
Requirement already satisfied: scipy>=0.18 in d:\programdata\anaconda3\lib\site-packages (from ludwig) (1.8.0)
Collecting torch>=1.13.0
Using cached https://pypi.tuna.tsinghua.edu.cn/packages/25/45/b91c4bf6b4b6325e9c758ef1203978ae5455c71e52054a7aca23befe33df/torch-2.0.0-cp39-cp39-win_amd64.whl (172.3 MB)
Collecting gpustat
Downloading https://pypi.tuna.tsinghua.edu.cn/packages/69/0e/00269385ce36c6722c7b4a6749b11c5c900a5cd4cbbd09d331b54ff58be9/gpustat-1.1.tar.gz (97 kB)
|████████████████████████████████| 97 kB 960 kB/s
Installing build dependencies ... done
Getting requirements to build wheel ... done
Preparing wheel metadata ... done
Requirement already satisfied: requests in d:\programdata\anaconda3\lib\site-packages (from ludwig) (2.27.1)
Collecting retry
Downloading https://pypi.tuna.tsinghua.edu.cn/packages/4b/0d/53aea75710af4528a25ed6837d71d117602b01946b307a3912cb3cfcbcba/retry-0.9.2-py2.py3-none-any.whl (8.0 kB)
Collecting marshmallow
Downloading https://pypi.tuna.tsinghua.edu.cn/packages/ae/53/980a20d789029329fdf1546c315f9c92bf862c7f3e7294e3667afcc464f5/marshmallow-3.19.0-py3-none-any.whl (49 kB)
|████████████████████████████████| 49 kB 1.5 MB/s
Collecting marshmallow-dataclass==8.5.4
Downloading https://pypi.tuna.tsinghua.edu.cn/packages/f8/59/9e6605a0c2144233370fe79391f7ae2d0073ba349af6839e85887f87d3f4/marshmallow_dataclass-8.5.4-py3-none-any.whl (16 kB)
Requirement already satisfied: torchtext in d:\programdata\anaconda3\lib\site-packages (from ludwig) (0.10.0)
Collecting typing-inspect>=0.7.1
Downloading https://pypi.tuna.tsinghua.edu.cn/packages/be/01/59b743dca816c4b6ca891b9e0f84d20513cd61bdbbaa8615de8f5aab68c1/typing_inspect-0.8.0-py3-none-any.whl (8.7 kB)
Requirement already satisfied: pyrsistent!=0.17.0,!=0.17.1,!=0.17.2,>=0.14.0 in d:\programdata\anaconda3\lib\site-packages (from jsonschema<4.7,>=4.5.0->ludwig) (0.18.0)
Requirement already satisfied: attrs>=17.4.0 in d:\programdata\anaconda3\lib\site-packages (from jsonschema<4.7,>=4.5.0->ludwig) (21.4.0)
Requirement already satisfied: pyparsing!=3.0.5,>=2.0.2 in d:\programdata\anaconda3\lib\site-packages (from packaging->ludwig) (3.0.4)
Requirement already satisfied: python-dateutil>=2.8.1 in d:\programdata\anaconda3\lib\site-packages (from pandas!=1.1.5,>=0.25->ludwig) (2.8.2)
Requirement already satisfied: pytz>=2020.1 in d:\programdata\anaconda3\lib\site-packages (from pandas!=1.1.5,>=0.25->ludwig) (2021.3)
Requirement already satisfied: six>=1.5 in d:\programdata\anaconda3\lib\site-packages (from python-dateutil>=2.8.1->pandas!=1.1.5,>=0.25->ludwig) (1.16.0)
Requirement already satisfied: pygments<3.0.0,>=2.6.0 in d:\programdata\anaconda3\lib\site-packages (from rich~=12.4.4->ludwig) (2.11.2)
Collecting commonmark<0.10.0,>=0.9.0
Downloading https://pypi.tuna.tsinghua.edu.cn/packages/b1/92/dfd892312d822f36c55366118b95d914e5f16de11044a27cf10a7d71bbbf/commonmark-0.9.1-py2.py3-none-any.whl (51 kB)
|████████████████████████████████| 51 kB 820 kB/s
Requirement already satisfied: threadpoolctl>=2.0.0 in d:\programdata\anaconda3\lib\site-packages (from scikit-learn<1.2.0->ludwig) (2.2.0)
Requirement already satisfied: joblib>=0.11 in d:\programdata\anaconda3\lib\site-packages (from scikit-learn<1.2.0->ludwig) (1.1.0)
Requirement already satisfied: blis<0.8.0,>=0.4.0 in d:\programdata\anaconda3\lib\site-packages (from spacy>=2.3->ludwig) (0.7.7)
Requirement already satisfied: murmurhash<1.1.0,>=0.28.0 in d:\programdata\anaconda3\lib\site-packages (from spacy>=2.3->ludwig) (1.0.7)
Requirement already satisfied: smart-open<7.0.0,>=5.2.1 in d:\programdata\anaconda3\lib\site-packages (from spacy>=2.3->ludwig) (5.2.1)
Requirement already satisfied: thinc<8.1.0,>=8.0.3 in d:\programdata\anaconda3\lib\site-packages (from spacy>=2.3->ludwig) (8.0.15)
Requirement already satisfied: cymem<2.1.0,>=2.0.2 in d:\programdata\anaconda3\lib\site-packages (from spacy>=2.3->ludwig) (2.0.6)
Requirement already satisfied: preshed<3.1.0,>=3.0.2 in d:\programdata\anaconda3\lib\site-packages (from spacy>=2.3->ludwig) (3.0.6)
Requirement already satisfied: srsly<3.0.0,>=2.4.1 in d:\programdata\anaconda3\lib\site-packages (from spacy>=2.3->ludwig) (2.4.6)
Requirement already satisfied: wasabi<1.1.0,>=0.8.1 in d:\programdata\anaconda3\lib\site-packages (from spacy>=2.3->ludwig) (0.9.1)
Requirement already satisfied: catalogue<2.1.0,>=2.0.4 in d:\programdata\anaconda3\lib\site-packages (from spacy>=2.3->ludwig) (2.0.7)
Requirement already satisfied: jinja2 in d:\programdata\anaconda3\lib\site-packages (from spacy>=2.3->ludwig) (2.11.3)
Requirement already satisfied: spacy-legacy<3.1.0,>=3.0.5 in d:\programdata\anaconda3\lib\site-packages (from spacy>=2.3->ludwig) (3.0.9)
Requirement already satisfied: pathy>=0.3.5 in d:\programdata\anaconda3\lib\site-packages (from spacy>=2.3->ludwig) (0.6.0)
Requirement already satisfied: pydantic!=1.8,!=1.8.1,<1.9.0,>=1.7.4 in d:\programdata\anaconda3\lib\site-packages (from spacy>=2.3->ludwig) (1.8.2)
Requirement already satisfied: setuptools in d:\programdata\anaconda3\lib\site-packages (from spacy>=2.3->ludwig) (61.2.0)
Requirement already satisfied: typer<0.4.0,>=0.3.0 in d:\programdata\anaconda3\lib\site-packages (from spacy>=2.3->ludwig) (0.3.2)
Requirement already satisfied: typing-extensions>=3.7.4.3 in d:\programdata\anaconda3\lib\site-packages (from pydantic!=1.8,!=1.8.1,<1.9.0,>=1.7.4->spacy>=2.3->ludwig) (4.1.1)
Requirement already satisfied: certifi>=2017.4.17 in d:\programdata\anaconda3\lib\site-packages (from requests->ludwig) (2021.10.8)
Requirement already satisfied: idna<4,>=2.5 in d:\programdata\anaconda3\lib\site-packages (from requests->ludwig) (3.3)
Requirement already satisfied: urllib3<1.27,>=1.21.1 in d:\programdata\anaconda3\lib\site-packages (from requests->ludwig) (1.26.9)
Requirement already satisfied: charset-normalizer~=2.0.0 in d:\programdata\anaconda3\lib\site-packages (from requests->ludwig) (2.0.4)
Requirement already satisfied: sympy in d:\programdata\anaconda3\lib\site-packages (from torch>=1.13.0->ludwig) (1.10.1)
Requirement already satisfied: networkx in d:\programdata\anaconda3\lib\site-packages (from torch>=1.13.0->ludwig) (2.7.1)
Requirement already satisfied: colorama in d:\programdata\anaconda3\lib\site-packages (from tqdm->ludwig) (0.4.4)
Collecting tokenizers!=0.11.3,<0.13,>=0.11.1
Downloading https://pypi.tuna.tsinghua.edu.cn/packages/28/f3/dbc9a5b9d244300bf68b717deb636578419e2989ec8fafb2a6c57c19f395/tokenizers-0.12.1-cp39-cp39-win_amd64.whl (3.3 MB)
|████████████████████████████████| 3.3 MB 1.1 MB/s
Requirement already satisfied: regex!=2019.12.17 in d:\programdata\anaconda3\lib\site-packages (from transformers<4.22,>=4.10.1->ludwig) (2022.3.15)
Requirement already satisfied: huggingface-hub<1.0,>=0.1.0 in d:\programdata\anaconda3\lib\site-packages (from transformers<4.22,>=4.10.1->ludwig) (0.13.2)
Requirement already satisfied: click<7.2.0,>=7.1.1 in d:\programdata\anaconda3\lib\site-packages (from typer<0.4.0,>=0.3.0->spacy>=2.3->ludwig) (7.1.2)
Requirement already satisfied: mypy-extensions>=0.3.0 in d:\programdata\anaconda3\lib\site-packages (from typing-inspect>=0.7.1->marshmallow-dataclass==8.5.4->ludwig) (0.4.3)
Collecting whylabs-client<1,>=0.4.4
Downloading https://pypi.tuna.tsinghua.edu.cn/packages/5e/01/8aa8ed4b7146bb584ed5f837a0d962354a06f77ad856eb92e7f838a03236/whylabs_client-0.4.6-py3-none-any.whl (360 kB)
|████████████████████████████████| 360 kB 1.1 MB/s
Collecting whylogs-sketching>=3.4.1.dev3
Downloading https://pypi.tuna.tsinghua.edu.cn/packages/e8/ad/8d065e1235756d679fb071ec3ebb3fa3c80bd7702a0b0cc23622148bff7f/whylogs_sketching-3.4.1.dev3-cp39-cp39-win_amd64.whl (382 kB)
|████████████████████████████████| 382 kB 1.1 MB/s
Collecting protobuf
Downloading https://pypi.tuna.tsinghua.edu.cn/packages/f8/70/6291e75633eeaa24fed46c9f66091bec184644e6159f392ac32eb92b1f65/protobuf-4.22.3-cp39-cp39-win_amd64.whl (420 kB)
|████████████████████████████████| 420 kB 819 kB/s
Collecting marshmallow-enum<2.0.0,>=1.5.1
Downloading https://pypi.tuna.tsinghua.edu.cn/packages/c6/59/ef3a3dc499be447098d4a89399beb869f813fee1b5a57d5d79dee2c1bf51/marshmallow_enum-1.5.1-py2.py3-none-any.whl (4.2 kB)
Collecting deepdiff
Downloading https://pypi.tuna.tsinghua.edu.cn/packages/33/4f/8feec2f9ef4515fc63566ec95a4775afd3ab1f08b563240469aa6afabd41/deepdiff-6.3.0-py3-none-any.whl (69 kB)
|████████████████████████████████| 69 kB 1.1 MB/s
Collecting progiter
Downloading https://pypi.tuna.tsinghua.edu.cn/packages/b1/e7/39f50201e00d2eb0e31c5f62c842a3990c6a5fd153ab249f06e3b3d5b107/progiter-1.3.0-py3-none-any.whl (19 kB)
Requirement already satisfied: matplotlib in d:\programdata\anaconda3\lib\site-packages (from experiment-impact-tracker->ludwig) (3.5.1)
Collecting geopy
Downloading https://pypi.tuna.tsinghua.edu.cn/packages/e4/00/6313c0ffdd230164890f433019749189e0b884562bdc170e9c3a3454b3a6/geopy-2.3.0-py3-none-any.whl (119 kB)
|████████████████████████████████| 119 kB 819 kB/s
Requirement already satisfied: ujson in d:\programdata\anaconda3\lib\site-packages (from experiment-impact-tracker->ludwig) (5.1.0)
Collecting py-cpuinfo
Downloading https://pypi.tuna.tsinghua.edu.cn/packages/e0/a9/023730ba63db1e494a271cb018dcd361bd2c917ba7004c3e49d5daf795a2/py_cpuinfo-9.0.0-py3-none-any.whl (22 kB)
Requirement already satisfied: seaborn in d:\programdata\anaconda3\lib\site-packages (from experiment-impact-tracker->ludwig) (0.11.2)
Collecting bootstrapped
Downloading https://pypi.tuna.tsinghua.edu.cn/packages/b2/f7/beb22809fdc4781ae73d12c78e2cf14ae533ad9eec2fb84ec22e5b0b660c/bootstrapped-0.0.2.tar.gz (11 kB)
Collecting bs4
Downloading https://pypi.tuna.tsinghua.edu.cn/packages/10/ed/7e8b97591f6f456174139ec089c769f89a94a1a4025fe967691de971f314/bs4-0.0.1.tar.gz (1.1 kB)
Requirement already satisfied: arrow in d:\programdata\anaconda3\lib\site-packages (from experiment-impact-tracker->ludwig) (1.2.2)
Collecting pylatex
Downloading https://pypi.tuna.tsinghua.edu.cn/packages/8a/76/015a1d785221d9b0d2ad80759d892a6d9d0a8a05daffc52202311ea3d652/PyLaTeX-1.4.1.tar.gz (84 kB)
|████████████████████████████████| 84 kB 980 kB/s
Collecting geocoder
Downloading https://pypi.tuna.tsinghua.edu.cn/packages/4f/6b/13166c909ad2f2d76b929a4227c952630ebaf0d729f6317eb09cbceccbab/geocoder-1.38.1-py2.py3-none-any.whl (98 kB)
|████████████████████████████████| 98 kB 1.2 MB/s
Collecting shapely
Downloading https://pypi.tuna.tsinghua.edu.cn/packages/a7/ae/eab645c4075093584b7a65ab71cb8ff4563a015bd935c9b55dba14b2c1eb/shapely-2.0.1-cp39-cp39-win_amd64.whl (1.4 MB)
|████████████████████████████████| 1.4 MB 1.1 MB/s
Requirement already satisfied: kiwisolver>=1.0.1 in d:\programdata\anaconda3\lib\site-packages (from matplotlib->experiment-impact-tracker->ludwig) (1.3.2)
Requirement already satisfied: fonttools>=4.22.0 in d:\programdata\anaconda3\lib\site-packages (from matplotlib->experiment-impact-tracker->ludwig) (4.25.0)
Requirement already satisfied: pillow>=6.2.0 in d:\programdata\anaconda3\lib\site-packages (from matplotlib->experiment-impact-tracker->ludwig) (9.0.1)
Requirement already satisfied: cycler>=0.10 in d:\programdata\anaconda3\lib\site-packages (from matplotlib->experiment-impact-tracker->ludwig) (0.11.0)
Requirement already satisfied: beautifulsoup4 in d:\programdata\anaconda3\lib\site-packages (from bs4->experiment-impact-tracker->ludwig) (4.11.1)
Requirement already satisfied: soupsieve>1.2 in d:\programdata\anaconda3\lib\site-packages (from beautifulsoup4->bs4->experiment-impact-tracker->ludwig) (2.3.1)
Collecting ordered-set<4.2.0,>=4.0.2
Downloading https://pypi.tuna.tsinghua.edu.cn/packages/33/55/af02708f230eb77084a299d7b08175cff006dea4f2721074b92cdb0296c0/ordered_set-4.1.0-py3-none-any.whl (7.6 kB)
Requirement already satisfied: aiohttp in d:\programdata\anaconda3\lib\site-packages (from fsspec[http]->ludwig) (3.8.1)
Requirement already satisfied: async-timeout<5.0,>=4.0.0a3 in d:\programdata\anaconda3\lib\site-packages (from aiohttp->fsspec[http]->ludwig) (4.0.1)
Requirement already satisfied: aiosignal>=1.1.2 in d:\programdata\anaconda3\lib\site-packages (from aiohttp->fsspec[http]->ludwig) (1.2.0)
Requirement already satisfied: frozenlist>=1.1.1 in d:\programdata\anaconda3\lib\site-packages (from aiohttp->fsspec[http]->ludwig) (1.2.0)
Requirement already satisfied: multidict<7.0,>=4.5 in d:\programdata\anaconda3\lib\site-packages (from aiohttp->fsspec[http]->ludwig) (5.1.0)
Requirement already satisfied: yarl<2.0,>=1.0 in d:\programdata\anaconda3\lib\site-packages (from aiohttp->fsspec[http]->ludwig) (1.6.3)
Collecting ratelim
Downloading https://pypi.tuna.tsinghua.edu.cn/packages/f2/98/7e6d147fd16a10a5f821db6e25f192265d6ecca3d82957a4fdd592cad49c/ratelim-0.1.6-py2.py3-none-any.whl (4.0 kB)
Requirement already satisfied: future in d:\programdata\anaconda3\lib\site-packages (from geocoder->experiment-impact-tracker->ludwig) (0.18.2)
Collecting geographiclib<3,>=1.52
Downloading https://pypi.tuna.tsinghua.edu.cn/packages/9f/5a/a26132406f1f40cf51ea349a5f11b0a46cec02a2031ff82e391c2537247a/geographiclib-2.0-py3-none-any.whl (40 kB)
|████████████████████████████████| 40 kB 2.4 MB/s
Collecting blessed>=1.17.1
Downloading https://pypi.tuna.tsinghua.edu.cn/packages/76/98/584f211c3a4bb38f2871fa937ee0cc83c130de50c955d6c7e2334dbf4acb/blessed-1.20.0-py2.py3-none-any.whl (58 kB)
|████████████████████████████████| 58 kB 2.9 MB/s
Collecting nvidia-ml-py>=11.450.129
Downloading https://pypi.tuna.tsinghua.edu.cn/packages/9c/8f/d08277539d8bab96a0b0cd49e2372c61d680594a05fa11c88ad8144cc306/nvidia_ml_py-11.525.112-py3-none-any.whl (35 kB)
Collecting jinxed>=1.1.0
Downloading https://pypi.tuna.tsinghua.edu.cn/packages/23/22/9b3481b11f32aedf1cc403f290b92bd23327d04e359808482a69b2bc3665/jinxed-1.2.0-py2.py3-none-any.whl (33 kB)
Requirement already satisfied: wcwidth>=0.1.4 in d:\programdata\anaconda3\lib\site-packages (from blessed>=1.17.1->gpustat->ludwig) (0.2.5)
Collecting ansicon
Downloading https://pypi.tuna.tsinghua.edu.cn/packages/75/f9/f1c10e223c7b56a38109a3f2eb4e7fe9a757ea3ed3a166754fb30f65e466/ansicon-1.89.0-py2.py3-none-any.whl (63 kB)
|████████████████████████████████| 63 kB 802 kB/s
Requirement already satisfied: MarkupSafe>=0.23 in d:\programdata\anaconda3\lib\site-packages (from jinja2->spacy>=2.3->ludwig) (2.0.1)
Requirement already satisfied: python-slugify in d:\programdata\anaconda3\lib\site-packages (from kaggle->ludwig) (5.0.2)
Requirement already satisfied: et-xmlfile in d:\programdata\anaconda3\lib\site-packages (from openpyxl->ludwig) (1.1.0)
Requirement already satisfied: text-unidecode>=1.3 in d:\programdata\anaconda3\lib\site-packages (from python-slugify->kaggle->ludwig) (1.3)
Requirement already satisfied: decorator in d:\programdata\anaconda3\lib\site-packages (from ratelim->geocoder->experiment-impact-tracker->ludwig) (5.1.1)
Requirement already satisfied: py<2.0.0,>=1.4.26 in d:\programdata\anaconda3\lib\site-packages (from retry->ludwig) (1.11.0)
Requirement already satisfied: mpmath>=0.19 in d:\programdata\anaconda3\lib\site-packages (from sympy->torch>=1.13.0->ludwig) (1.2.1)
Requirement already satisfied: numexpr>=2.6.2 in d:\programdata\anaconda3\lib\site-packages (from tables->ludwig) (2.8.1)
Requirement already satisfied: werkzeug>=1.0.1 in d:\programdata\anaconda3\lib\site-packages (from tensorboard->ludwig) (2.0.3)
Requirement already satisfied: grpcio>=1.24.3 in d:\programdata\anaconda3\lib\site-packages (from tensorboard->ludwig) (1.42.0)
Requirement already satisfied: tensorboard-plugin-wit>=1.6.0 in d:\programdata\anaconda3\lib\site-packages (from tensorboard->ludwig) (1.8.1)
Requirement already satisfied: markdown>=2.6.8 in d:\programdata\anaconda3\lib\site-packages (from tensorboard->ludwig) (3.3.4)
Requirement already satisfied: wheel>=0.26 in d:\programdata\anaconda3\lib\site-packages (from tensorboard->ludwig) (0.37.1)
Requirement already satisfied: google-auth<3,>=1.6.3 in d:\programdata\anaconda3\lib\site-packages (from tensorboard->ludwig) (1.33.0)
Collecting protobuf
Downloading https://pypi.tuna.tsinghua.edu.cn/packages/11/a5/e52b731415ad6ef3d841e9e6e337a690249e800cc7c06f0749afab26348c/protobuf-3.20.3-cp39-cp39-win_amd64.whl (904 kB)
|████████████████████████████████| 904 kB 3.3 MB/s
Requirement already satisfied: google-auth-oauthlib<0.5,>=0.4.1 in d:\programdata\anaconda3\lib\site-packages (from tensorboard->ludwig) (0.4.6)
Requirement already satisfied: tensorboard-data-server<0.7.0,>=0.6.0 in d:\programdata\anaconda3\lib\site-packages (from tensorboard->ludwig) (0.6.1)
Requirement already satisfied: pyasn1-modules>=0.2.1 in d:\programdata\anaconda3\lib\site-packages (from google-auth<3,>=1.6.3->tensorboard->ludwig) (0.2.8)
Requirement already satisfied: rsa<5,>=3.1.4 in d:\programdata\anaconda3\lib\site-packages (from google-auth<3,>=1.6.3->tensorboard->ludwig) (4.7.2)
Requirement already satisfied: cachetools<5.0,>=2.0.0 in d:\programdata\anaconda3\lib\site-packages (from google-auth<3,>=1.6.3->tensorboard->ludwig) (4.2.2)
Requirement already satisfied: requests-oauthlib>=0.7.0 in d:\programdata\anaconda3\lib\site-packages (from google-auth-oauthlib<0.5,>=0.4.1->tensorboard->ludwig) (1.3.1)
Requirement already satisfied: pyasn1<0.5.0,>=0.4.6 in d:\programdata\anaconda3\lib\site-packages (from pyasn1-modules>=0.2.1->google-auth<3,>=1.6.3->tensorboard->ludwig) (0.4.8)
Requirement already satisfied: oauthlib>=3.0.0 in d:\programdata\anaconda3\lib\site-packages (from requests-oauthlib>=0.7.0->google-auth-oauthlib<0.5,>=0.4.1->tensorboard->ludwig) (3.2.2)
Collecting torchtext
Using cached https://pypi.tuna.tsinghua.edu.cn/packages/14/44/5b8689ccfef1661f6311ab20e62e47156d27b7574c2ecd127ebe06605285/torchtext-0.15.1-cp39-cp39-win_amd64.whl (1.9 MB)
Collecting torchdata==0.6.0
Using cached https://pypi.tuna.tsinghua.edu.cn/packages/3f/c3/9503fc630de85fcc8f6dfdb3645c6ec49b31fe1157ad5b1346d984168a20/torchdata-0.6.0-cp39-cp39-win_amd64.whl (1.3 MB)
Building wheels for collected packages: ludwig, bootstrapped, bs4, gpustat, kaggle, pylatex, sacremoses
Building wheel for ludwig (PEP 517) ... done
Created wheel for ludwig: filename=ludwig-0.7.4-py3-none-any.whl size=962118 sha256=b9cbaab71bef211cd1ae107b5147e62a8dfab2e4a07a5aaeada8728a516f2509
Stored in directory: c:\users\99386\appdata\local\pip\cache\wheels\55\5f\94\8d97b3fd8c34788b0c305501e3ba3e5b85a26f54838f000e23
Building wheel for bootstrapped (setup.py) ... done
Created wheel for bootstrapped: filename=bootstrapped-0.0.2-py2.py3-none-any.whl size=13954 sha256=780fe17f0d301781410ff533136743d18c62a4e854cc0b9cda454ee5e1517f2a
Stored in directory: c:\users\99386\appdata\local\pip\cache\wheels\a1\75\a1\5937edfafdd44ef45bf5ef67bc72d1eb983b61cdabab70a837
Building wheel for bs4 (setup.py) ... done
Created wheel for bs4: filename=bs4-0.0.1-py3-none-any.whl size=1272 sha256=f9c3684648727db040a2593bf6f3bf2036aa3800eae72b5d202a1fa0fd162927
Stored in directory: c:\users\99386\appdata\local\pip\cache\wheels\02\37\2a\d32b991c0064f63108da01a9163a54ab4a8f273427315a6092
Building wheel for gpustat (PEP 517) ... done
Created wheel for gpustat: filename=gpustat-1.1-py3-none-any.whl size=26316 sha256=5734c35aff38f31bd77a8af8b5e010d90ed2b230e0940291ccc3aab64ce6b98f
Stored in directory: c:\users\99386\appdata\local\pip\cache\wheels\09\0a\f1\e8397dce85ac88647e0a61ee2122f9c98cfb6094972003c30d
Building wheel for kaggle (setup.py) ... done
Created wheel for kaggle: filename=kaggle-1.5.13-py3-none-any.whl size=77733 sha256=5a7de2c67b028a8a31a8169d036c6e307dfa88fd479af44cb6c63f7311a13764
Stored in directory: c:\users\99386\appdata\local\pip\cache\wheels\37\f0\e4\56de4da6867dbb5ae770812f18fa32a20c0d89550b8d657847
Building wheel for pylatex (setup.py) ... done
Created wheel for pylatex: filename=PyLaTeX-1.4.1-py3-none-any.whl size=42826 sha256=1f73085439fbe20767c4c93d241d002ba6e5df83f11c5d259ce2544b230feeb6
Stored in directory: c:\users\99386\appdata\local\pip\cache\wheels\13\3b\a3\e9945487abf7ba90e08cc1b9a6dd5b0b19515c0b5aa1bb6a1f
Building wheel for sacremoses (setup.py) ... done
Created wheel for sacremoses: filename=sacremoses-0.0.53-py3-none-any.whl size=895260 sha256=385b1f4f4f6218dda677c2f9d032a4a3c9b46c1fa7c9170d6d0c04c85cfb130e
Stored in directory: c:\users\99386\appdata\local\pip\cache\wheels\3c\4c\0b\b17aca42475c28654c70c60038563b3b0a1ade6a2f97933363
Successfully built ludwig bootstrapped bs4 gpustat kaggle pylatex sacremoses
Installing collected packages: ansicon, torch, ratelim, ordered-set, marshmallow, jinxed, geographiclib, whylogs-sketching, whylabs-client, typing-inspect, torchdata, tokenizers, shapely, pylatex, py-cpuinfo, psutil, protobuf, progiter, nvidia-ml-py, marshmallow-enum, geopy, geocoder, deepdiff, commonmark, bs4, bootstrapped, blessed, xlwt, whylogs, transformers, torchtext, torchmetrics, torchinfo, torchaudio, sacremoses, rich, retry, pyarrow, marshmallow-jsonschema, marshmallow-dataclass, kaggle, jsonschema, gpustat, experiment-impact-tracker, dataclasses-json, ludwig
Attempting uninstall: torch
Found existing installation: torch 1.9.0
Uninstalling torch-1.9.0:
Successfully uninstalled torch-1.9.0
Attempting uninstall: tokenizers
Found existing installation: tokenizers 0.13.2
Uninstalling tokenizers-0.13.2:
Successfully uninstalled tokenizers-0.13.2
Attempting uninstall: psutil
Found existing installation: psutil 5.8.0
Uninstalling psutil-5.8.0:
Successfully uninstalled psutil-5.8.0
Attempting uninstall: protobuf
Found existing installation: protobuf 3.19.1
Uninstalling protobuf-3.19.1:
Successfully uninstalled protobuf-3.19.1
Attempting uninstall: transformers
Found existing installation: transformers 4.27.1
Uninstalling transformers-4.27.1:
Successfully uninstalled transformers-4.27.1
Attempting uninstall: torchtext
Found existing installation: torchtext 0.10.0
Uninstalling torchtext-0.10.0:
Successfully uninstalled torchtext-0.10.0
Attempting uninstall: jsonschema
Found existing installation: jsonschema 4.4.0
Uninstalling jsonschema-4.4.0:
Successfully uninstalled jsonschema-4.4.0
ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
spyder 5.1.5 requires pyqt5<5.13, which is not installed.
spyder 5.1.5 requires pyqtwebengine<5.13, which is not installed.
conda-repo-cli 1.0.4 requires pathlib, which is not installed.
tensorflow-intel 2.11.0 requires protobuf<3.20,>=3.9.2, but you have protobuf 3.20.3 which is incompatible.
jupyter-server 1.13.5 requires pywinpty<2; os_name == "nt", but you have pywinpty 2.0.2 which is incompatible.
Successfully installed ansicon-1.89.0 blessed-1.20.0 bootstrapped-0.0.2 bs4-0.0.1 commonmark-0.9.1 dataclasses-json-0.5.7 deepdiff-6.3.0 experiment-impact-tracker-0.1.8 geocoder-1.38.1 geographiclib-2.0 geopy-2.3.0 gpustat-1.1 jinxed-1.2.0 jsonschema-4.6.2 kaggle-1.5.13 ludwig-0.7.4 marshmallow-3.19.0 marshmallow-dataclass-8.5.4 marshmallow-enum-1.5.1 marshmallow-jsonschema-0.13.0 nvidia-ml-py-11.525.112 ordered-set-4.1.0 progiter-1.3.0 protobuf-3.20.3 psutil-5.9.4 py-cpuinfo-9.0.0 pyarrow-11.0.0 pylatex-1.4.1 ratelim-0.1.6 retry-0.9.2 rich-12.4.4 sacremoses-0.0.53 shapely-2.0.1 tokenizers-0.12.1 torch-2.0.0 torchaudio-2.0.1 torchdata-0.6.0 torchinfo-1.7.2 torchmetrics-0.11.4 torchtext-0.15.1 transformers-4.21.3 typing-inspect-0.8.0 whylabs-client-0.4.6 whylogs-1.1.37 whylogs-sketching-3.4.1.dev3 xlwt-1.3.0
C:\Windows\system32>Ludwig的使用方法
1、基础用法
# Ludwig文本分类案例，使用IMDB电影评论数据集
import pandas as pd
from ludwig.api import LudwigModel
# 加载IMDB数据集
df = pd.read_csv('https://storage.googleapis.com/tf-datasets/tfds-test0.1.0/imdb_reviews/plain_text/0.1.0/{}.csv.gz'.format('train'))
df = df.sample(frac=1).reset_index(drop=True)
# 配置和训练模型
model = LudwigModel(config={'input_features': [{'name': 'text', 'type': 'text'}], 'output_features': [{'name': 'sentiment', 'type': 'category'}]})
train_stats = model.train(df)
# 使用模型进行预测
text = "This movie was great!"
predictions = model.predict({'text': text})
print(predictions['sentiment_predictions'][0]['class_name'])
1195
被折叠的 条评论
为什么被折叠?
到【灌水乐园】发言
