HuggingFace Accelerate整合wandb记录实验

https://huggingface.co/docs/accelerate/main/en/usage_guides/tracking
https://docs.wandb.ai/guides/integrations/accelerate

看了半天HuggingFace教程没看明白怎么添加其他wandb run的参数（我还是太菜了!），最后在wandb的教程中找到了… 传入init_kwargs参数即可。

示例：

from accelerate import Accelerator

# Tell the Accelerator object to log with wandb
accelerator = Accelerator(log_with="wandb")

# Initialise your wandb run, passing wandb parameters and any config information
accelerator.init_trackers(
    project_name="my_project", 
    config={"dropout": 0.1, "learning_rate": 1e-2}
    init_kwargs={"wandb": {"entity": "my-wandb-team"}}
    )

...

# Log to wandb by calling `accelerator.log`, `step` is optional
accelerator.log({"train_loss": 1.12, "valid_loss": 0.8}, step=global_step)


# Make sure that the wandb tracker finishes correctly
accelerator.end_training()