问答

在这节课中，咱们将学习如何利用检索到的文档来回答用户的问题。

整个进程能够拆分为以下几个过程：

1. 用户输入一个问题（Question）
2. 从向量存储（Store）中检索出与问题相关的文档分块（Relavant splits）
3. 将这些分块连同体系提示（System:Prompt）和用户问题（Human:Question）一同作为输入传给言语模型（LLM）
4. 言语模型依据输入生成答案（Answer）

默认运用的是stuff办法，其特色如下：

为了处理上下文窗口长度约束的问题，咱们能够运用Map-reduce、Refine和Map-rerank三种办法，这些办法咱们在之前的课程中已经简要介绍过了，今日咱们将进一步深化了解。

stuff

过程1：加载之前保存的向量数据库

from langchain.vectorstores import Chroma
from langchain.embeddings.openai import OpenAIEmbeddings
persist_directory = 'docs/chroma/'
embedding = OpenAIEmbeddings()
vectordb = Chroma(persist_directory=persist_directory, embedding_function=embedding)

过程2：初始化将用于回答问题的言语模型

llm_name = "gpt-3.5-turbo"
from langchain.chat_models import ChatOpenAI
llm = ChatOpenAI(model_name=llm_name, temperature=0)

temperature参数设置为0，能够协助咱们得到更精确的答案，因为它降低了言语模型的可变性，通常能给咱们最高置信度、最牢靠的答案。

过程3：导入、创立、调用检索问答链，输入问题，并获取答案

question = "What are major topics for this class?"
from langchain.chains import RetrievalQA
qa_chain = RetrievalQA.from_chain_type(
    llm,
    retriever=vectordb.as_retriever()
)
result = qa_chain({"query": question})
result["result"]

‘The major topic for this class is machine learning. Additionally, the class may cover statistics and algebra as refreshers in the discussion sections. Later in the quarter, the discussion sections will also cover extensions for the material taught in the main lectures.’

过程4：运用提示模板优化输出成果

提示模板是一种能够协助言语模型生成更符合要求的输出成果的技巧，咱们在上一门课中已经介绍过了。这儿咱们运用的提示模板主要是为了让输出成果更简洁、更少编造、更礼貌。

from langchain.prompts import PromptTemplate
# 构建提示词
# {context}：上下文占位符，用于放置文档内容
# {question}：问题占位符，放置要查询的问题
template = """Use the following pieces of context to answer the question at the end. If you don't know the answer, just say that you don't know, don't try to make up an answer. Use three sentences maximum. Keep the answer as concise as possible. Always say "thanks for asking!" at the end of the answer. 
{context}
Question: {question}
Helpful Answer:"""
QA_CHAIN_PROMPT = PromptTemplate.from_template(template)
# 运转链
# return_source_documents=True用于支撑检查检索到的文档
qa_chain = RetrievalQA.from_chain_type(
    llm,
    retriever=vectordb.as_retriever(),
    return_source_documents=True,
    chain_type_kwargs={"prompt": QA_CHAIN_PROMPT}
)
question = "Is probability a class topic?"
result = qa_chain({"query": question})
result["result"]

‘Yes, probability is assumed to be a prerequisite for this class. The instructor assumes familiarity with basic probability and statistics, and will go over some of the prerequisites in the discussion sections as a refresher course. Thanks for asking!’

过程5：检查返回的源文档，理解其从哪里获取数据

result["source_documents"][0]

Document(page_content=”of this class will not be very program ming intensive, although we will do some \nprogramming, mostly in either MATLAB or Octa ve. I’ll say a bit more about that later. \nI also assume familiarity with basic proba bility and statistics. So most undergraduate \nstatistics class, like Stat 116 taught here at Stanford, will be more than enough. I’m gonna \nassume all of you know what ra ndom variables are, that all of you know what expectation \nis, what a variance or a random variable is. And in case of some of you, it’s been a while \nsince you’ve seen some of this material. At some of the discussion sections, we’ll actually \ngo over some of the prerequisites, sort of as a refresher course under prerequisite class. \nI’ll say a bit more about that later as well. \nLastly, I also assume familiarity with basi c linear algebra. And again, most undergraduate \nlinear algebra courses are more than enough. So if you’ve taken courses like Math 51, \n103, Math 113 or CS205 at Stanford, that would be more than enough. Basically, I’m \ngonna assume that all of you know what matrix es and vectors are, that you know how to \nmultiply matrices and vectors and multiply matrix and matrices, that you know what a matrix inverse is. If you know what an eigenvect or of a matrix is, that’d be even better. \nBut if you don’t quite know or if you’re not qu ite sure, that’s fine, too. We’ll go over it in \nthe review sections.”, metadata={‘source’: ‘docs/cs229_lectures/MachineLearning-Lecture01.pdf’, ‘page’: 4})

Map-reduce

Map-reduce办法的特色如下：

qa_chain_mr = RetrievalQA.from_chain_type(
    llm,
    retriever=vectordb.as_retriever(),
    chain_type="map_reduce"
)
result = qa_chain_mr({"query": question})
result["result"]

‘There is no clear answer to this question based on the given portion of the document. The document mentions familiarity with basic probability and statistics as a prerequisite for the class, and there is a brief mention of probability in the text, but it is not clear if it is a main topic of the class.’

运用LangSmith渠道了解这些链内部的调用情况

LangSmith 是一个用于构建出产级 LLM 应用程序的渠道。

它能够让您轻松地调试、测验、评价和监控根据任何 LLM 结构构建的链和智能代理，并与运用 LLM 构建的开源结构 LangChain 完美集成。

要体验这个渠道的功用，你需求：

1. 前往LangSmith渠道注册（或许需求排队）
2. 创立 API 密钥
3. 在以下代码中运用这个 API 密钥
4. 取消注释以下代码，并重新运转MapReduce链

import os
os.environ["LANGCHAIN_TRACING_V2"] = "true"
os.environ["LANGCHAIN_ENDPOINT"] = "https://api.langchain.plus"
os.environ["LANGCHAIN_API_KEY"] = "..." # 替换...为你的 API 密钥

之后，就能够切换到LangSmith渠道，找到刚刚运转的RetrievalQA，检查输入、输出以及调用链了：

能够看到，MapReduceDocumentChain中涉及到了对言语模型的四次独立调用，点击其中一个，就能够看到每个文档的具体输入和输出：

而且，能够看到，它们在最后的链中被合并为了StuffDocumentChain，也即把一切成果放到了最终调用中。点击进入就能够看到，体系音讯中包含了来自前面文档的四个摘要：

Refine

Refine办法的特色如下：

qa_chain_mr = RetrievalQA.from_chain_type(
    llm,
    retriever=vectordb.as_retriever(),
    chain_type="refine"
)
result = qa_chain_mr({"query": question})
result["result"]

“The main topic of the class is machine learning algorithms, including linear regression and classification. Basic probability and statistics, as well as linear algebra, are prerequisites for the class, but the instructor will provide a refresher course on these topics in some of the discussion sections. Later in the quarter, the discussion sections will also cover extensions for the material taught in the main lectures. The instructor will focus on a few important extensions that there wasn’t enough time to cover in the main lectures.”

现在还有一个问题，咱们目前运用的链是没有“回忆”这个概念的，这就导致了它不会记住之前的问题或答案。为了处理这个问题，咱们需求引进“回忆”功用，这便是咱们下一节要讲的内容。

Chat

在这节课中，咱们将构建一个完整的问答谈天机器人，它将结合咱们之前讲过的一切组件，并引进“谈天前史”这个概念，让它在回答问题时能够考虑到之前的对话或信息，也便是说，它能记住你之前说过什么。

过程1：初始化用于保存很多文档内容的向量数据库

from langchain.vectorstores import Chroma
from langchain.embeddings.openai import OpenAIEmbeddings
persist_directory = 'docs/chroma/'
embedding = OpenAIEmbeddings()
vectordb = Chroma(persist_directory=persist_directory, embedding_function=embedding)

过程2：初始化将作为谈天机器人运用的言语模型

llm_name = "gpt-3.5-turbo"
from langchain.chat_models import ChatOpenAI
llm = ChatOpenAI(model_name=llm_name, temperature=0)

过程3：初始化提示模板，让输出成果更简介、更少编造、更礼貌

# 构建提示
from langchain.prompts import PromptTemplate
template = """Use the following pieces of context to answer the question at the end. If you don't know the answer, just say that you don't know, don't try to make up an answer. Use three sentences maximum. Keep the answer as concise as possible. Always say "thanks for asking!" at the end of the answer. 
{context}
Question: {question}
Helpful Answer:"""
QA_CHAIN_PROMPT = PromptTemplate(input_variables=["context", "question"],template=template,)

过程4：创立检索QA链，用于合并检索到的文本片段并调用言语模型

# 运转链
from langchain.chains import RetrievalQA
qa_chain = RetrievalQA.from_chain_type(llm,
                                       retriever=vectordb.as_retriever(),
                                       return_source_documents=True,
                                       chain_type_kwargs={"prompt": QA_CHAIN_PROMPT})

过程5：运用ConversationBufferMemory添加谈天机器人的回忆功用

from langchain.memory import ConversationBufferMemory
memory = ConversationBufferMemory(
    memory_key="chat_history",
    return_messages=True
)

ConversationBufferMemory供给了一个谈天音讯前史的回忆缓冲区，而且每次都会将这部分前史音讯传入谈天机器人。

return_messages=True表示将返回一个列表类型的谈天前史记录，而不是一个字符串。

过程6：创立ConversationalRetrievalChain（对话检索链），传入言语模型、检索器和回忆体系

from langchain.chains import ConversationalRetrievalChain
retriever=vectordb.as_retriever()
qa = ConversationalRetrievalChain.from_llm(
    llm,
    retriever=retriever,
    memory=memory
)

ConversationalRetrievalChain会在RetrievalQAChain的基础上，将谈天前史和新提的问题整合成一个新的独立问题，以传递给向量存储库，查找相关文档。

过程7：运用PyPDFLoader加载所要参考的文档

from langchain.embeddings.openai import OpenAIEmbeddings
from langchain.text_splitter import CharacterTextSplitter, RecursiveCharacterTextSplitter
from langchain.vectorstores import DocArrayInMemorySearch
from langchain.document_loaders import TextLoader
from langchain.chains import RetrievalQA,  ConversationalRetrievalChain
from langchain.memory import ConversationBufferMemory
from langchain.chat_models import ChatOpenAI
from langchain.document_loaders import TextLoader
from langchain.document_loaders import PyPDFLoader
def load_db(file, chain_type, k):
    # 加载文档
    loader = PyPDFLoader(file)
    documents = loader.load()
    ...

过程8：分割文档，为每个分块创立嵌入，并存储到向量存储库中。

def load_db(file, chain_type, k):
    ...
    # 分隔文档
    text_splitter = RecursiveCharacterTextSplitter(chunk_size=1000, chunk_overlap=150)
    docs = text_splitter.split_documents(documents)
    # 定义嵌入
    embeddings = OpenAIEmbeddings()
    # 根据文档数据创立向量数据库
    db = DocArrayInMemorySearch.from_documents(docs, embeddings)
    ...

过程9：从向量数据库创立一个根据“类似度”的检索器。

def load_db(file, chain_type, k):
    ...
    # 定义检索器
    retriever = db.as_retriever(search_type="similarity", search_kwargs={"k": k})
    ...

过程10：创立对话检索链，用于将谈天前史和新提的问题整合成一个新的独立问题

def load_db(file, chain_type, k):
    ...
    # create a chatbot chain. Memory is managed externally.
    qa = ConversationalRetrievalChain.from_llm(
        llm=ChatOpenAI(model_name=llm_name, temperature=0), 
        chain_type=chain_type, 
        retriever=retriever, 
        return_source_documents=True,
        return_generated_question=True,
    )
    return qa 

需求注意的是，这儿咱们并没有传入回忆体系，而是将回忆管理交给了GUI，这意味着谈天前史需求在链之外进行保护。

过程11：供给一个与谈天机器人交互的用户界面

import panel as pn
import param
class cbfs(param.Parameterized):
    chat_history = param.List([])
    answer = param.String("")
    db_query  = param.String("")
    db_response = param.List([])
    def __init__(self,  **params):
        super(cbfs, self).__init__( **params)
        self.panels = []
        self.loaded_file = "docs/cs229_lectures/MachineLearning-Lecture01.pdf"
        self.qa = load_db(self.loaded_file,"stuff", 4)
    def call_load_db(self, count):
        if count == 0 or file_input.value is None:  # init or no file specified :
            return pn.pane.Markdown(f"Loaded File: {self.loaded_file}")
        else:
            file_input.save("temp.pdf")  # local copy
            self.loaded_file = file_input.filename
            button_load.button_style="outline"
            self.qa = load_db("temp.pdf", "stuff", 4)
            button_load.button_style="solid"
        self.clr_history()
        return pn.pane.Markdown(f"Loaded File: {self.loaded_file}")
    def convchain(self, query):
        if not query:
            return pn.WidgetBox(pn.Row('User:', pn.pane.Markdown("", width=600)), scroll=True)
        result = self.qa({"question": query, "chat_history": self.chat_history})
        self.chat_history.extend([(query, result["answer"])])
        self.db_query = result["generated_question"]
        self.db_response = result["source_documents"]
        self.answer = result['answer'] 
        self.panels.extend([
            pn.Row('User:', pn.pane.Markdown(query, width=600)),
            pn.Row('ChatBot:', pn.pane.Markdown(self.answer, width=600, style={'background-color': '#F6F6F6'}))
        ])
        inp.value = ''  #clears loading indicator when cleared
        return pn.WidgetBox(*self.panels,scroll=True)
    @param.depends('db_query ', )
    def get_lquest(self):
        if not self.db_query :
            return pn.Column(
                pn.Row(pn.pane.Markdown(f"Last question to DB:", styles={'background-color': '#F6F6F6'})),
                pn.Row(pn.pane.Str("no DB accesses so far"))
            )
        return pn.Column(
            pn.Row(pn.pane.Markdown(f"DB query:", styles={'background-color': '#F6F6F6'})),
            pn.pane.Str(self.db_query )
        )
    @param.depends('db_response', )
    def get_sources(self):
        if not self.db_response:
            return 
        rlist=[pn.Row(pn.pane.Markdown(f"Result of DB lookup:", styles={'background-color': '#F6F6F6'}))]
        for doc in self.db_response:
            rlist.append(pn.Row(pn.pane.Str(doc)))
        return pn.WidgetBox(*rlist, width=600, scroll=True)
    @param.depends('convchain', 'clr_history') 
    def get_chats(self):
        if not self.chat_history:
            return pn.WidgetBox(pn.Row(pn.pane.Str("No History Yet")), width=600, scroll=True)
        rlist=[pn.Row(pn.pane.Markdown(f"Current Chat History variable", styles={'background-color': '#F6F6F6'}))]
        for exchange in self.chat_history:
            rlist.append(pn.Row(pn.pane.Str(exchange)))
        return pn.WidgetBox(*rlist, width=600, scroll=True)
    def clr_history(self,count=0):
        self.chat_history = []
        return 

cb = cbfs()
file_input = pn.widgets.FileInput(accept='.pdf')
button_load = pn.widgets.Button(name="Load DB", button_type='primary')
button_clearhistory = pn.widgets.Button(name="Clear History", button_type='warning')
button_clearhistory.on_click(cb.clr_history)
inp = pn.widgets.TextInput( placeholder='Enter text here…')
bound_button_load = pn.bind(cb.call_load_db, button_load.param.clicks)
conversation = pn.bind(cb.convchain, inp) 
jpg_pane = pn.pane.Image( './img/convchain.jpg')
tab1 = pn.Column(
    pn.Row(inp),
    pn.layout.Divider(),
    pn.panel(conversation,  loading_indicator=True, height=300),
    pn.layout.Divider(),
)
tab2= pn.Column(
    pn.panel(cb.get_lquest),
    pn.layout.Divider(),
    pn.panel(cb.get_sources ),
)
tab3= pn.Column(
    pn.panel(cb.get_chats),
    pn.layout.Divider(),
)
tab4=pn.Column(
    pn.Row( file_input, button_load, bound_button_load),
    pn.Row( button_clearhistory, pn.pane.Markdown("Clears chat history. Can use to start a new topic" )),
    pn.layout.Divider(),
    pn.Row(jpg_pane.clone(width=400))
)
dashboard = pn.Column(
    pn.Row(pn.pane.Markdown('# ChatWithYourData_Bot')),
    pn.Tabs(('Conversation', tab1), ('Database', tab2), ('Chat History', tab3),('Configure', tab4))
)
dashboard

过程12：在运转起来的用户界面上进行实践的问答对话。