GPT-3微调分步指南

从2022年底开端，OpenAI推出ChatGPT，被咱们许多人认为是AI的iPhone时刻。然而，OpenAI的聊天机器人并不是第一个生成式人工智能文本机器学习模型，它是在两年前推出的GPT-3之后推出的。

OpenAI为咱们供给了一个现成的GPT-3练习模型。此外，能够在较小的数据集上对特定使命进行微调。例如，假设您要创立特定于您公司的电子邮件呼应生成器。首要，您有必要搜集大量关于您的特定事务范畴的数据，如客户电子邮件查询和呼应。然后，您能够运用此数据来微调GPT-3，以了解公司的特定言语形式和短语。经过微调GPT-3，能够创立高度自定义和专业化的电子邮件呼应生成器，专门针对特定事务范畴中运用的言语形式和单词进行定制。

在这篇博文中，我将向您展现怎么微调GPT-3。咱们将运用python代码来履行此操作，而不需求事前了解GPT-3。

前提预备

不像在HuggingFace（写这篇博客的时分）上供给的GPT-2模型，咱们不能直接拜访GPT-3模型。因此，首要需求从OpenAI获取API密钥并装置Python包奥佩奈 ，可经过以下办法快速完结尖点。

对于OpenAI的API密钥：

  • 请拜访https：//platform.openai.com/account/api-keys，
  • 创立帐户，
  • 单击“创立新密钥’ 以及
  • 复制一把钥匙。

密钥是一个以“sk-”开头的长字符串。一定要保密！取得密钥后，拜访密钥的简略办法是在终端中履行以下操作：（我个人，为了简略起见，我把它放在我的.bachrc）：

export OPENAI_API_KEY=sk-t59pgejhtrff5(...)

运用GPT-3类型是有本钱的。咱们需求学分。在编撰本文时，当您创立一个新帐户时，您能够取得免费的积分来试用该东西。我不知道这会不会继续下去…

现在咱们有了key和Python包，是时分考虑需求微调的数据了。首要，咱们需求一个用于微调的示例文件，其间每个示例都是一个提示符 ，后跟适当的完结。

描绘生成东西

GPT-3微调分步指南

GPT-3微调分步指南
一个超级英豪从DALL-E 2

咱们将为这个演示构建一个东西来创立虚拟的超级英豪的描绘。最终，这个东西会接收到超级英豪的年纪、性别和力气，它会主动生成咱们的超级英豪的描绘。

在下面的例子中，在微调模型之后，咱们只需求说’40，woman，Healing -> ‘，然后咱们将主动从模型中取得一个描绘。

GPT-3微调分步指南

这便是全部！！

创立用于微调的数据集

在某些情况下，您或许有一个要用于微调的数据集。可是，因为我没有一个，让咱们看看怎么直接从GPT-3创立一个包括超级英豪描绘的组成数据集。下面的代码将给予我一个CSV文件，其间包括提示 和相应的完结示例。

import os
import openai
import pandas as pd
openai.api_key = os.getenv("OPENAI_API_KEY")
l_age = ['18', '20', '30', '40', '50', '60', '90']
l_gender = ['man', 'woman']
l_power = ['invisibility', 'read in the thoughts', 'turning lead into gold', 'immortality', 'telepathy', 'teleport', 'flight'] 
f_prompt = "Imagine a complete and detailed description of a {age}-year-old {gender} fictional character who has the superpower of {power}. Write out the entire description in a maximum of 100 words in great detail:"
f_sub_prompt = "{age}, {gender}, {power}"
df = pd.DataFrame()
for age in l_age:
 for gender in l_gender:
  for power in l_power:
   for i in range(3): ## 3 times each
    prompt = f_prompt.format(age=age, gender=gender, power=power)
    sub_prompt = f_sub_prompt.format(age=age, gender=gender, power=power)
    print(sub_prompt)
    response = openai.Completion.create(
     model="text-davinci-003",
     prompt=prompt,
     temperature=1,
     max_tokens=500,
     top_p=1,
     frequency_penalty=0,
     presence_penalty=0
    )
    finish_reason = response['choices'][0]['finish_reason']
    response_txt = response['choices'][0]['text']
    new_row = {
      'age':age, 
      'gender':gender, 
      'power':power, 
      'prompt':prompt, 
      'sub_prompt':sub_prompt, 
      'response_txt':response_txt, 
      'finish_reason':finish_reason}
    new_row = pd.DataFrame([new_row])
    df = pd.concat([df, new_row], axis=0, ignore_index=True)
df.to_csv("out_openai_completion.csv")

让咱们看看这段代码是怎么工作的。

变量f_prompt 包括以下语句，其间缺少{age} 、{gender}和{power}。

想象一个{age}岁的{gender}虚拟人物具有{power}超能力的完好而具体的描绘。用最多100字的篇幅具体写出整个描绘：

在代码的前三个for 循环中，咱们迭代{age}、{gender}和{power}的不同值。在循环的每一步，咱们用不同的值替换3个缺失的变量。然后运用openai.Completion.create函数要求GPT生成对提示的呼应。

此函数最重要的参数包括

  • model ：用于生成呼应的模型。OpenAI供给了四种规范的GPT-3模型（ada，babbage，curie，或davinci），它们的大小…和运用价格各不相同。这是达芬奇 -最大的模型。
  • 瞬发：要运用GPT-3完结的提示。
  • 温度：温度是一个介于0和1之间的数字，控制输出的随机性。咱们将温度设置为最大值，以使模型在创立呼应时尽或许具有创造性。
  • 最大令牌数：定义呼应的最大长度。

在此脚本的结尾，咱们有一个Pandas表存储在文件out_openai_complement. csv中。& nbsp; 此表中咱们感兴趣的两个首要列是sub_prompt 和response_txt。

  • 该 子提示符 例如 **‘18岁，伙计，隐形人。 它包括被替换的三个值，以逗号分隔。
  • 该 呼应_文本 包括GPT模型的输出。

微调GPT模型

下面的代码检索以前创立的文件out_openai_completion.csv，并运用openai 微调GPT-3模型。

import pandas as pd
import openai
import subprocess
df = pd.read_csv("out_openai_completion.csv")
prepared_data = df.loc[:,['sub_prompt','response_txt']]
prepared_data.rename(columns={'sub_prompt':'prompt', 'response_txt':'completion'}, inplace=True)
prepared_data.to_csv('prepared_data.csv',index=False)
## prepared_data.csv --> prepared_data_prepared.json
subprocess.run('openai tools fine_tunes.prepare_data --file prepared_data.csv --quiet'.split())
## Start fine-tuning
subprocess.run('openai api fine_tunes.create --training_file prepared_data_prepared.jsonl --model davinci --suffix "SuperHero"'.split())

让咱们花点时间来了解这段代码吧！

一、档案的内容输出_打开_完结. csv 被载入数据帧df。作为提示，在咱们的使命中，如果用户输入”40，女人，医治”，咱们希望有一个具有医治能力的40岁女人人物的描绘。要履行微调，有必要向GPT供给用户或许键入的内容和相应的希望呼应的示例。在数据框中df ，列子提示符 以及呼应_文本 包括具有相应希望呼应的输入示例。在上面的代码中，咱们首要提取这两个冒号，然后将它们重命名为瞬发 以及完结分别为。生成的数据框存储在新文件中预备好的数据. csv仅包括这两列。

此文件prepared_data.csv如下所示：

prompt,completion
"18, man, invisibility","He looks to be in his late teens, with dark eyes and unruly black hair which reach his shoulders. His build is athletic and strong, but not muscular. He often wears clothing that makes him blend in with his surroundings, in order to stay as inconspicuous as possible.He has the ability of invisibility, which he can use to become an invisible observer or to make himself or objects around him disappear from view. He has honed his skills to make himself undetectable, able to even make sounds vanish, allowing him to go unnoticed. His invisibility power is his greatest asset, and he uses it to protect those he cares about."
"18, man, invisibility","18 year-old Jackson is a powerful young man with the incredible power of invisibility. He stands at 6'2 and has a strong athletic frame with wavy brown hair, startling blue eyes and a warm and gentle smile. His strength and agility make him especially good at using his ability. He can become invisible in an instant, moving through walls and other obstacles without being detected. What's more, he can move objects with his mind and use his power to protect those he loves. His power is a blessing and a curse, as it can be abused to do harm. Jackson is a brave and noble person who uses his power to do good and make the world a better place."
"18, man, invisibility","Brandon is an 18-year-old of average build, standing roughly 5'10 with an inquisitive look. He has naturally wavy chestnut brown hair and bright blue eyes. His demeanor is usually quite serious, but he also has an easy and gentle smile. He has a natural gift of invisibility, which he uses to protect himself and his family from harm. He's an inquisitive and creative young man who uses his gift to explore the world, observe people, and uncover the truth. His consistent practice of mindfulness helps him remain unseen, even when his emotions are visible. His intelligence and persistent passion for truth drives him to help those in need."
"18, man, read in the thoughts","This 18-year-old man has a muscular stature and long, ash blonde hair. His bright blue eyes are full of knowledge and power, hinting at the strange ability he has - he can read minds. Now and then, strange sparks of electricity streak from his eyes when he concentrates hard enough. He is often calm and collected, but when provoked has the ability to blend his learning of your thoughts with his immense strength. His wit, intelligence and empathy are weapons that are often underestimated - yet, very lethal."
"18, man, read in the thoughts","He's an 18-year-old man with short, cropped dark hair and bright green eyes. His usual outfit consists of an olive green hoodie, blue jeans and purple trainers. He's relaxed, confident and always reads the atmosphere well. One of his great skills is read in the thoughts of others. By doing so, he is able to understand people more easily and relate to their concerns out of the blue. He always uses his superpower with the utmost dignity and respect, never using it to manipulate. He uses it to seek the truth, identify people's emotions and gain insight into their perspectives. He's the sort of person that can make anyone feel comfortable and is never afraid of a challenge. His superpower is merely an extension of his already charismatic disposition."
(...)

Python的subprocess.run（）函数将指令作为子进程运转。它经常被用来履行外部程序，就好像它们在终端中运转一样。

咱们运用subprocess.run （） 来履行’openai东西微调.预备数据’ 。此函数承受输入文件prepared_data.csv，检查数据是否正确，并生成一个名为prepared_data_prepared.jsonl的JSONL文件。JSONL文件是一种将每个JSON目标存储在单独行中的格式。JSONL文件包括一系列JSON目标，每个目标由换行符分隔。

请注意，咱们增加了选项“- – quiet”来主动承受“openai tools fine_tunes.prepare_data”供给的一切主张。例如 ，它主张在一切提示符的结尾增加一个“–>”，并在每个呼应的结尾增加一个符号END。

该JSONL文件的第一行如下所示：

{"prompt":"18, man, invisibility ->","completion":" \n\nHe looks to be in his late teens, with dark eyes and unruly black hair which reach his shoulders. His build is athletic and strong, but not muscular. He often wears clothing that makes him blend in with his surroundings, in order to stay as inconspicuous as possible.\n\nHe has the ability of invisibility, which he can use to become an invisible observer or to make himself or objects around him disappear from view. He has honed his skills to make himself undetectable, able to even make sounds vanish, allowing him to go unnoticed. His invisibility power is his greatest asset, and he uses it to protect those he cares about. END"}
{"prompt":"18, man, invisibility ->","completion":" \n\n18 year-old Jackson is a powerful young man with the incredible power of invisibility. He stands at 6'2 and has a strong athletic frame with wavy brown hair, startling blue eyes and a warm and gentle smile. His strength and agility make him especially good at using his ability. He can become invisible in an instant, moving through walls and other obstacles without being detected. What's more, he can move objects with his mind and use his power to protect those he loves. His power is a blessing and a curse, as it can be abused to do harm. Jackson is a brave and noble person who uses his power to do good and make the world a better place. END"}
(...)

GPT-3模型的微调是在第二阶段真正完结的subprocess.run （）， 其间openai api微调.创立被履行。在这个函数中，咱们首要给出之前创立的JSONL文件的名称。然后，您需求选择要微调的类型。OpenAI供给四种首要模型，具有不同的性能水平，适用于各种使命。& nbsp; 达温奇 是最强壮的模型，而Ada是最快的。nbsp; 达温奇也是最贵的类型。

因为我的模型的意图是创立对超级英豪的描绘，咱们给我的新模型加了后缀“超级英豪“。

就这样几分钟后，您将具有一个微调好的模型，能够随时运用。

现在是时分测验你的新模型了

有不同的办法来运用模型来完结。首要经过OpenAI供给的Playground或经过Python等编程言语。

最简略的办法或许是运用操场。

  • 请拜访https：//platform.openai.com/playground。
  • 点击“类型”并查找后缀为“”的类型超级英豪”.
  • 在“停止序列”中增加符号“END”。

GPT-3微调分步指南
现在是时分让咱们的模型做出新的预测了。咱们会要求描绘一个18岁的男性人物，他真的具有不必要的力气。咱们会要求描绘一个具有力气的人物……“能够吃许多”……看看会发生什么……

GPT-3微调分步指南

不算太糟

你想用Python来做吗？简略!点击屏幕右上角的”检查代码”。

GPT-3微调分步指南

在咱们的例子中，在“检查代码”中，咱们有：

import os
import openai
openai.api_key = os.getenv("OPENAI_API_KEY")
response = openai.Completion.create(
  model="davinci:ft-klhm:superhero-2023-02-01-14-56-48",
  prompt="18, Man, can eat a lot ->\n",
  temperature=0.7,
  max_tokens=256,
  top_p=1,
  frequency_penalty=0,
  presence_penalty=0,
  stop=["END"]
)

只需求复制粘贴就能够了。

结论

在这个博客中，咱们现已看到了怎么生成组成数据来完善咱们的模型，以及怎么进行微调。咱们运用了一个创立超级英豪的用例，可是相同的办法能够用于您或许有的任何用例。最重要的是要有满足的高质量的例子，有提示和希望的反响。