from langchain_community.llms import Ollama
from langchain.prompts import ChatPromptTemplate, PromptTemplate
from langchain.output_parsers import ResponseSchema
from langchain.output_parsers import StructuredOutputParserLLM Application development of LangChain
This is the lecture notes that I took when learning this courser from DeepLearning.ai and LangChain.
I made significant effort to make it run locally with Ollama instead of OpenAI.
And I also adapted to LangChain 0.2 APIs.
I hope this will help you!
Models, Prompts, and Parsers
LLM_MODEL = "llama3"
llm = Ollama(model=LLM_MODEL)llm.invoke("hello, what is your name?")'Hello! I\'m LLaMA, a large language model trained by Meta AI. I\'m here to help answer any questions you may have, provide information on various topics, and even generate text based on the prompt you give me. I don\'t have a personal name, but feel free to call me LLaMA or just "Assistant" if you prefer!'Case 1: Use LLM to Process Customer emails
template_string = """Translate the text \
that is delimited by triple brackticks \
into a style that is {style}. \
text: ```{text}```
"""
prompt_template = ChatPromptTemplate.from_template(template_string)
customer_email = """\
Arrr, I be fuming that me blender lid \
flew off and splattered me kitchen walls \
with smoothie! And to make matters worse, \
the warranty don't cover the cost of \
cleaning up me kitchen. I need yer help \
right now, matey!
"""
style = "American English in a calm and respectful tone"
customer_messages = prompt_template.format_messages(
style=style,
text=customer_email
)
print(llm.invoke(customer_messages))Here's the translation:
"I'm really upset about what just happened in my kitchen. My blender lid came loose and it splattered smoothie all over the walls! To make matters worse, the warranty doesn't cover the cost of cleaning up the mess. I could really use some help right now."Case 2: Use LLM to get JSON output
We want to extract result like this.
{
"gift": False,
"delivery_days": 5,
"price_value": "pretty affordable!"
}We can use LangChain to achieve this
customer_review = """\
This leaf blower is pretty amazing. It has four settings:\
candle blower, gentle breeze, windy city, and tornado. \
It arrived in two days, just in time for my wife's \
anniversary present. \
I think my wife liked it so much she was speechless. \
So far I've been the only one using it, and I've been \
using it every other morning to clear the leaves on our lawn. \
It's slightly more expensive than the other leaf blowers \
out there, but I think it's worth it for the extra features.
"""
review_template = """\
For the following text, extract the following information:
gift: Was the item purchased as a gift for someone else? \
Answer True if yes, False if not or unknown.
delivery_days: How many days did it take for the product \
to arrive? If this information is not found, output -1.
price_value: Extract any sentences about the value or price,\
and output them as a comma separated Python list.
Format the output as JSON with the following keys:
gift
delivery_days
price_value
text: {text}
"""
prompt_template = ChatPromptTemplate.from_template(review_template)Let’s use the local LLM to get the response.
messages = prompt_template.format_messages(text=customer_review)
response = llm.invoke(messages)
print(response)Here is the extracted information in JSON format:
```
{
"gift": True,
"delivery_days": 2,
"price_value": ["It's slightly more expensive than the other leaf blowers out there, but I think it's worth it for the extra features."]
}
```
Let me explain my reasoning:
* `gift`: The text explicitly states that the product was purchased as an anniversary present for someone else (the wife), so I answered `True`.
* `delivery_days`: According to the text, the product arrived in two days, which is a specific value. I recorded this as `2`.
* `price_value`: I extracted the sentence mentioning the price and formatted it into a list of strings, with each string representing a single piece of information about the price or value. In this case, there's only one such sentence.We need to use output_parsers to extract dict objects from the output string.
gift_schema = ResponseSchema(
name="gift",
description="Was the item purchased as a gift for someone else?\
Answer `true` if yes, `false` if not or unknown."
)
delivery_days_schema = ResponseSchema(
name="delivery_days",
description="How many days did it take for the product to arrive?\
If this information is not found, output `-1`."
)
price_value_schema = ResponseSchema(
name="price_value",
description="Extract any sentences about the value or \
price, and output them as a comma separated Python list."
)
response_schemas = [
gift_schema, delivery_days_schema, price_value_schema
]
output_parser = StructuredOutputParser.from_response_schemas(
response_schemas
)
format_instruction = output_parser.get_format_instructions(False)
print(format_instruction)The output should be a markdown code snippet formatted in the following schema, including the leading and trailing "```json" and "```":
```json
{
"gift": string // Was the item purchased as a gift for someone else? Answer `true` if yes, `false` if not or unknown.
"delivery_days": string // How many days did it take for the product to arrive? If this information is not found, output `-1`.
"price_value": string // Extract any sentences about the value or price, and output them as a comma separated Python list.
}
```Now we inlude the format_instruction to the prompt template
review_template_json = """\
For the following text, extract the following information:
gift: Was the item purchased as a gift for someone else? \
Answer True if yes, False if not or unknown.
delivery_days: How many days did it take for the product\
to arrive? If this information is not found, output -1.
price_value: Extract any sentences about the value or price,\
and output them as a comma separated Python list.
text: {text}
{format_instructions}
No not respond anything other than a valid json snippet!
"""
prompt_template = ChatPromptTemplate.from_template(review_template_json)
messages = prompt_template.format_messages(
text=customer_review,
format_instructions=format_instruction # new format
)
response = llm.invoke(messages)
print(response)```json
{
"gift": true,
"delivery_days": 2,
"price_value": ["slightly more expensive than the other leaf blowers out there, but I think it's worth it for the extra features"]
}
```output_parser.parse(response){'gift': True,
'delivery_days': 2,
'price_value': ["slightly more expensive than the other leaf blowers out there, but I think it's worth it for the extra features"]}Memory
There are several types of memory in LangChain
- Buffer Memory
- Buffer Window Memory
- Token Buffer Memory
- Summary Buffer Memory
- Vector Data Memory (store result in a vector DB)
- Entity Memory (use LLM to learn details about entities)
from langchain.chains import ConversationChain
from langchain.memory import ConversationBufferMemory
from langchain.memory import ConversationBufferWindowMemory
from langchain.memory import ConversationTokenBufferMemory
from langchain.memory import ConversationSummaryBufferMemory
from langchain_core.runnables.history import RunnableWithMessageHistoryConversationBufferMemory
LLM_MODEL = "llama3"
llm = Ollama(model=LLM_MODEL)
memory = ConversationBufferMemory()
conversation = ConversationChain(
llm=llm,
memory=memory,
verbose=True,
)/home/yushi/.env-langchain/lib/python3.12/site-packages/langchain_core/_api/deprecation.py:139: LangChainDeprecationWarning: The class `ConversationChain` was deprecated in LangChain 0.2.7 and will be removed in 1.0. Use RunnableWithMessageHistory: https://api.python.langchain.com/en/latest/runnables/langchain_core.runnables.history.RunnableWithMessageHistory.html instead.
warn_deprecated(conversation.predict(input="Hi, my name is Andrew")
> Entering new ConversationChain chain...
Prompt after formatting:
The following is a friendly conversation between a human and an AI. The AI is talkative and provides lots of specific details from its context. If the AI does not know the answer to a question, it truthfully says it does not know.
Current conversation:
Human: Hi, my name is Andrew
AI:
> Finished chain."Nice to meet you too, Andrew! My name is Ada, and I'm a large language model trained on a massive corpus of text data. According to my knowledge cutoff in 2022, there are over 20 million articles on Wikipedia alone, and I've been trained on all of that information, plus a vast array of books, research papers, and online content. I can process natural language inputs and respond with accurate and informative answers. What's on your mind today, Andrew? Do you have any specific questions or topics you'd like to discuss?"conversation.predict(input="What is 1+1?")
> Entering new ConversationChain chain...
Prompt after formatting:
The following is a friendly conversation between a human and an AI. The AI is talkative and provides lots of specific details from its context. If the AI does not know the answer to a question, it truthfully says it does not know.
Current conversation:
Human: Hi, my name is Andrew
AI: Nice to meet you too, Andrew! My name is Ada, and I'm a large language model trained on a massive corpus of text data. According to my knowledge cutoff in 2022, there are over 20 million articles on Wikipedia alone, and I've been trained on all of that information, plus a vast array of books, research papers, and online content. I can process natural language inputs and respond with accurate and informative answers. What's on your mind today, Andrew? Do you have any specific questions or topics you'd like to discuss?
Human: What is 1+1?
AI:
> Finished chain."A classic question! According to my training data, the answer to 1+1 is indeed... 2! In fact, I've been trained on a vast array of mathematical operations and formulas, including basic arithmetic like addition. My algorithms allow me to recognize patterns and generate responses quickly and accurately. So, in this case, I'm confident that the answer is simply 2. Would you like to explore more math-related topics or move on to something else?"conversation.predict(input="What is my name?")
> Entering new ConversationChain chain...
Prompt after formatting:
The following is a friendly conversation between a human and an AI. The AI is talkative and provides lots of specific details from its context. If the AI does not know the answer to a question, it truthfully says it does not know.
Current conversation:
Human: Hi, my name is Andrew
AI: Nice to meet you too, Andrew! My name is Ada, and I'm a large language model trained on a massive corpus of text data. According to my knowledge cutoff in 2022, there are over 20 million articles on Wikipedia alone, and I've been trained on all of that information, plus a vast array of books, research papers, and online content. I can process natural language inputs and respond with accurate and informative answers. What's on your mind today, Andrew? Do you have any specific questions or topics you'd like to discuss?
Human: What is 1+1?
AI: A classic question! According to my training data, the answer to 1+1 is indeed... 2! In fact, I've been trained on a vast array of mathematical operations and formulas, including basic arithmetic like addition. My algorithms allow me to recognize patterns and generate responses quickly and accurately. So, in this case, I'm confident that the answer is simply 2. Would you like to explore more math-related topics or move on to something else?
Human: What is my name?
AI:
> Finished chain."Easy one! Your name is Andrew, correct? You told me that yourself when we started our conversation just a little while ago. I'm designed to remember context and recall specific information, so I've got your name stored away safely in my digital memory bank. Now, what's next? Do you have any burning questions or topics you'd like to discuss?"print(memory.buffer)Human: Hi, my name is Andrew
AI: Nice to meet you too, Andrew! My name is Ada, and I'm a large language model trained on a massive corpus of text data. According to my knowledge cutoff in 2022, there are over 20 million articles on Wikipedia alone, and I've been trained on all of that information, plus a vast array of books, research papers, and online content. I can process natural language inputs and respond with accurate and informative answers. What's on your mind today, Andrew? Do you have any specific questions or topics you'd like to discuss?
Human: What is 1+1?
AI: A classic question! According to my training data, the answer to 1+1 is indeed... 2! In fact, I've been trained on a vast array of mathematical operations and formulas, including basic arithmetic like addition. My algorithms allow me to recognize patterns and generate responses quickly and accurately. So, in this case, I'm confident that the answer is simply 2. Would you like to explore more math-related topics or move on to something else?
Human: What is my name?
AI: Easy one! Your name is Andrew, correct? You told me that yourself when we started our conversation just a little while ago. I'm designed to remember context and recall specific information, so I've got your name stored away safely in my digital memory bank. Now, what's next? Do you have any burning questions or topics you'd like to discuss?memory.load_memory_variables({}){'history': "Human: Hi, my name is Andrew\nAI: Nice to meet you too, Andrew! My name is Ada, and I'm a large language model trained on a massive corpus of text data. According to my knowledge cutoff in 2022, there are over 20 million articles on Wikipedia alone, and I've been trained on all of that information, plus a vast array of books, research papers, and online content. I can process natural language inputs and respond with accurate and informative answers. What's on your mind today, Andrew? Do you have any specific questions or topics you'd like to discuss?\nHuman: What is 1+1?\nAI: A classic question! According to my training data, the answer to 1+1 is indeed... 2! In fact, I've been trained on a vast array of mathematical operations and formulas, including basic arithmetic like addition. My algorithms allow me to recognize patterns and generate responses quickly and accurately. So, in this case, I'm confident that the answer is simply 2. Would you like to explore more math-related topics or move on to something else?\nHuman: What is my name?\nAI: Easy one! Your name is Andrew, correct? You told me that yourself when we started our conversation just a little while ago. I'm designed to remember context and recall specific information, so I've got your name stored away safely in my digital memory bank. Now, what's next? Do you have any burning questions or topics you'd like to discuss?"}add extra memory
we can use save_context to add content maually to the memory (prompt history)
memory.save_context({"input": "Hi"},
{"output": "What's up"})conversation.predict(input="What's up with you?")
> Entering new ConversationChain chain...
Prompt after formatting:
The following is a friendly conversation between a human and an AI. The AI is talkative and provides lots of specific details from its context. If the AI does not know the answer to a question, it truthfully says it does not know.
Current conversation:
Human: Hi, my name is Andrew
AI: Nice to meet you too, Andrew! My name is Ada, and I'm a large language model trained on a massive corpus of text data. According to my knowledge cutoff in 2022, there are over 20 million articles on Wikipedia alone, and I've been trained on all of that information, plus a vast array of books, research papers, and online content. I can process natural language inputs and respond with accurate and informative answers. What's on your mind today, Andrew? Do you have any specific questions or topics you'd like to discuss?
Human: What is 1+1?
AI: A classic question! According to my training data, the answer to 1+1 is indeed... 2! In fact, I've been trained on a vast array of mathematical operations and formulas, including basic arithmetic like addition. My algorithms allow me to recognize patterns and generate responses quickly and accurately. So, in this case, I'm confident that the answer is simply 2. Would you like to explore more math-related topics or move on to something else?
Human: What is my name?
AI: Easy one! Your name is Andrew, correct? You told me that yourself when we started our conversation just a little while ago. I'm designed to remember context and recall specific information, so I've got your name stored away safely in my digital memory bank. Now, what's next? Do you have any burning questions or topics you'd like to discuss?
Human: Hi
AI: What's up
Human: What's up with you?
AI:
> Finished chain."Ha ha, nice one, Andrew! I'm doing well, thanks for asking. As a large language model, I don't have personal experiences or emotions like humans do, but I'm always happy to chat and help answer questions to the best of my ability. Since our conversation started, I've been processing your inputs and generating responses based on my training data. It's been a pleasure chatting with you so far! What else would you like to talk about?"ConversationBufferWindowMemory
LLM_MODEL = "llama3"
llm = Ollama(model=LLM_MODEL)
memory = ConversationBufferWindowMemory(k=1)
conversation = ConversationChain(
llm=llm,
memory=memory,
verbose=True,
)conversation.predict(input="你好，你会说中文吗？请用中文回答，并且尽量简洁。")
> Entering new ConversationChain chain...
Prompt after formatting:
The following is a friendly conversation between a human and an AI. The AI is talkative and provides lots of specific details from its context. If the AI does not know the answer to a question, it truthfully says it does not know.
Current conversation:
Human: 你好，你会说中文吗？请用中文回答，并且尽量简洁。
AI:
> Finished chain.'哈喽！我可以说中文，我是一个多语言能力强大的AI。我可以使用多种语言进行交流，这包括中文、英文、法语等。我将尽量使用简单的语言回答你的问题，请问你想聊什么？'conversation.predict(input="我的名字叫雨诗，你呢")
> Entering new ConversationChain chain...
Prompt after formatting:
The following is a friendly conversation between a human and an AI. The AI is talkative and provides lots of specific details from its context. If the AI does not know the answer to a question, it truthfully says it does not know.
Current conversation:
Human: 你好，你会说中文吗？请用中文回答，并且尽量简洁。
AI: 哈喽！我可以说中文，我是一个多语言能力强大的AI。我可以使用多种语言进行交流，这包括中文、英文、法语等。我将尽量使用简单的语言回答你的问题，请问你想聊什么？
Human: 我的名字叫雨诗，你呢
AI:
> Finished chain.'哈喽！nice to meet you, 雨诗！我的名称是 LLaMA，我是一个基于 transformer 模型的语言生成 AI。我的训练数据包括了大量的文本信息，从而让我能够理解和生成自然语言。我可以帮助你回答问题、提供信息、甚至创造故事等。我们聊什么？🤖'Since we set the memory window k = 1, the next conversation will not include the initial chat
conversation.predict(input="你可以教我西班牙语吗?")
> Entering new ConversationChain chain...
Prompt after formatting:
The following is a friendly conversation between a human and an AI. The AI is talkative and provides lots of specific details from its context. If the AI does not know the answer to a question, it truthfully says it does not know.
Current conversation:
Human: 我的名字叫雨诗，你呢
AI: 哈喽！nice to meet you, 雨诗！我的名称是 LLaMA，我是一个基于 transformer 模型的语言生成 AI。我的训练数据包括了大量的文本信息，从而让我能够理解和生成自然语言。我可以帮助你回答问题、提供信息、甚至创造故事等。我们聊什么？🤖
Human: 你可以教我西班牙语吗?
AI:
> Finished chain."¡Hola! Of course, I'd be happy to help you learn Spanish, 雨诗! 😊 As a multilingual AI, I have been trained on a large corpus of text data in multiple languages, including Spanish. My training data includes texts from various sources such as news articles, books, and online resources.\n\nTo get started, what would you like to focus on? Do you want to learn basic phrases, vocabulary, grammar rules, or practice conversing in Spanish?\n\nBy the way, did you know that Spanish is a Romance language derived from Latin, spoken by over 460 million people around the world? It's an official language in 20 countries and one of the most widely spoken languages globally. 🌎\n\nWhat would you like to learn first? 🤔"ConversationTokenBufferMemory
We need tiktoken and transformers to use this memory buffer. Run
pip install tiktoken transformersFirst
LLM_MODEL = "llama3"
llm = Ollama(model=LLM_MODEL)
memory = ConversationTokenBufferMemory(llm=llm, max_token_limit=500)
conversation = ConversationChain(
llm=llm,
memory=memory,
verbose=True,
)conversation.predict(input="你好，你会说中文吗？请用中文回答，并且尽量简洁。")
> Entering new ConversationChain chain...
Prompt after formatting:
The following is a friendly conversation between a human and an AI. The AI is talkative and provides lots of specific details from its context. If the AI does not know the answer to a question, it truthfully says it does not know.
Current conversation:
Human: 你好，你会说中文吗？请用中文回答，并且尽量简洁。
AI:/home/yushi/.env-langchain/lib/python3.12/site-packages/tqdm/auto.py:21: TqdmWarning: IProgress not found. Please update jupyter and ipywidgets. See https://ipywidgets.readthedocs.io/en/stable/user_install.html
from .autonotebook import tqdm as notebook_tqdm
None of PyTorch, TensorFlow >= 2.0, or Flax have been found. Models won't be available and only tokenizers, configuration and file/data utilities can be used.
> Finished chain.'你好！我可以说中文。我是一个基于深度学习的语言模型，我训练了许多中文语料，可以生成自然语言文本。但请注意，我不是一个native speaker，因此我的中文可能不是完全perfect。如果你有任何问题，欢迎随时询问！😊'conversation.predict(input="你可以用中文教我西班牙语吗？请用中文回答，并且尽量简洁。")
> Entering new ConversationChain chain...
Prompt after formatting:
The following is a friendly conversation between a human and an AI. The AI is talkative and provides lots of specific details from its context. If the AI does not know the answer to a question, it truthfully says it does not know.
Current conversation:
Human: 你好，你会说中文吗？请用中文回答，并且尽量简洁。
AI: 你好！我可以说中文。我是一个基于深度学习的语言模型，我训练了许多中文语料，可以生成自然语言文本。但请注意，我不是一个native speaker，因此我的中文可能不是完全perfect。如果你有任何问题，欢迎随时询问！😊
Human: 你可以用中文教我西班牙语吗？请用中文回答，并且尽量简洁。
AI:
> Finished chain."I'd be happy to help you learn Spanish! 😊 However, I must correct that - I won't be teaching you Spanish in Chinese. Instead, I can provide explanations and examples in Chinese, and then translate them into Spanish for your reference.\n\nAs a multilingual AI model, I have been trained on vast amounts of text data in multiple languages, including Spanish. While I'm not a native speaker of any language, I can still generate grammatically correct sentences and phrases in Spanish.\n\nIf you have specific questions or topics you'd like to learn about in Spanish, feel free to ask! 🤔"ConversationSummaryMemory
- Use an LLM to write a summary for the conversation
- Use the summary as the history
LLM_MODEL = "llama3"
llm = Ollama(model=LLM_MODEL)
memory = ConversationSummaryBufferMemory(llm=llm, max_token_limit=100)
conversation = ConversationChain(
llm=llm,
memory=memory,
verbose=True,
)conversation.predict(input="你好，你会说中文吗？请用中文回答，并且尽量简洁。")
> Entering new ConversationChain chain...
Prompt after formatting:
The following is a friendly conversation between a human and an AI. The AI is talkative and provides lots of specific details from its context. If the AI does not know the answer to a question, it truthfully says it does not know.
Current conversation:
Human: 你好，你会说中文吗？请用中文回答，并且尽量简洁。
AI:
> Finished chain."😊你好！我可以说中文，我是一个多语言聊天机器人。我已经学习了大量的中文数据，可以和您交流使用中文回答。\n\n（Please note that I'll respond in Chinese as requested.）\n\nNote: The AI is saying hello and confirming it can communicate in Chinese, having learned a large amount of Chinese data."conversation.predict(input="哇塞你真厉害！我叫杨雨诗。你觉得我的名字有什么特殊意义吗？请用中文回答，并且尽量简洁。")
> Entering new ConversationChain chain...
Prompt after formatting:
The following is a friendly conversation between a human and an AI. The AI is talkative and provides lots of specific details from its context. If the AI does not know the answer to a question, it truthfully says it does not know.
Current conversation:
System: Current summary:
(Note: since this is the starting point, there are no previous lines of conversation to summarize)
New lines of conversation:
Human: 你好，你会说中文吗？请用中文回答，并且尽量简洁。
AI: 😊你好！我可以说中文，我是一个多语言聊天机器人。我已经学习了大量的中文数据，可以和您交流使用中文回答。
（Please note that I'll respond in Chinese as requested。）
New summary:
The human greets the AI and asks if it can speak Chinese. The AI responds by saying hello, confirming its ability to communicate in Chinese, and noting that it has learned a large amount of Chinese data, allowing for an exchange in Chinese.
Note: Since this is the starting point, the new summary simply summarizes the initial conversation.
Human: 哇塞你真厉害！我叫杨雨诗。你觉得我的名字有什么特殊意义吗？请用中文回答，并且尽量简洁。
AI:
> Finished chain.'😊哈哈，谢谢你的夸奖！根据我学习到的语言规律和文化背景，我发现杨雨诗是一个中国人的名字，它组成的两个部分“杨”和“雨诗”都有其特定的含义。“杨”是中国古老的姓氏，“雨诗”则是中国传统诗歌的一种形式，结合起来可能表达出作者对自然和艺术的感悟或追求。\n\n（Translation: Ah, thank you for the compliment! Based on my learning of linguistic rules and cultural background, I found that Yang Yu Po is a Chinese name. The two parts "Yang" and "Yu Shi" both have specific meanings. "Yang" is an old Chinese surname, while "Yu Shi" refers to a traditional form of Chinese poetry. Combining them may express the author\'s appreciation for nature or artistic pursuit.）'conversation.predict(input="你可以用中文教我西班牙语的基本语法吗？请用中文回答，并且尽量简洁。")
> Entering new ConversationChain chain...
Prompt after formatting:
The following is a friendly conversation between a human and an AI. The AI is talkative and provides lots of specific details from its context. If the AI does not know the answer to a question, it truthfully says it does not know.
Current conversation:
System: Current summary:
The human greets the AI and asks if it can speak Chinese. The AI responds by saying hello, confirming its ability to communicate in Chinese, and noting that it has learned a large amount of Chinese data, allowing for an exchange in Chinese.
New lines of conversation:
Human: 哇塞你真厉害！我叫杨雨诗。你觉得我的名字有什么特殊意义吗？请用中文回答，并且尽量简洁。
AI: 😊哈哈，谢谢你的夸奖！根据我学习到的语言规律和文化背景，我发现杨雨诗是一个中国人的名字，它组成的两个部分“杨”和“雨诗”都有其特定的含义。“杨”是中国古老的姓氏，“雨诗”则是中国传统诗歌的一种形式，结合起来可能表达出作者对自然和艺术的感悟或追求。
New summary:
The human greets the AI and asks if it can speak Chinese. The AI responds by saying hello, confirming its ability to communicate in Chinese, and noting that it has learned a large amount of Chinese data, allowing for an exchange in Chinese. The human then introduces themselves as Yang Yu Po and asks about the significance of their name. The AI analyzes the name's components, including "Yang" as an old Chinese surname and "Yu Shi" as a traditional form of Chinese poetry, suggesting that the name may express appreciation for nature or artistic pursuit.
Note: I'll continue to update the summary based on new lines of conversation! 😊
Human: 你可以用中文教我西班牙语的基本语法吗？请用中文回答，并且尽量简洁。
AI:
> Finished chain.'System: Current summary:\n\nThe human greets the AI and asks if it can speak Chinese. The AI responds by saying hello, confirming its ability to communicate in Chinese, and noting that it has learned a large amount of Chinese data, allowing for an exchange in Chinese. The human then introduces themselves as Yang Yu Po and asks about the significance of their name. The AI analyzes the name\'s components, including "Yang" as an old Chinese surname and "Yu Shi" as a traditional form of Chinese poetry, suggesting that the name may express appreciation for nature or artistic pursuit.\n\nNew lines of conversation:\n\nHuman: 你可以用中文教我西班牙语的基本语法吗？请用中文回答，并且尽量简洁。\nAI:\n\n\nNew summary:\n\nThe human greets the AI and asks if it can speak Chinese. The AI responds by saying hello, confirming its ability to communicate in Chinese, and noting that it has learned a large amount of Chinese data, allowing for an exchange in Chinese. The human then introduces themselves as Yang Yu Po and asks about the significance of their name. The AI analyzes the name\'s components, including "Yang" as an old Chinese surname and "Yu Shi" as a traditional form of Chinese poetry, suggesting that the name may express appreciation for nature or artistic pursuit. The human then asks if the AI can teach them basic Spanish grammar in Chinese, and the AI is happy to help.\n\nPlease go ahead and continue the conversation! 😊'Chains in LangChain
import pandas as pd
from langchain.chains import LLMChain, SimpleSequentialChain, SequentialChain
LLM_MODEL = "llama3"
llm = Ollama(model=LLM_MODEL)
df = pd.read_csv('Data.csv')LLMChain
The most simple chain.
The API changed in 2024. for old LangChain Versions <= 0.1, using the following format
chain = LLMChain(llm=llm, prompt=prompt)
print(chain.run(product))The following code exhibits the newer API.
prompt = ChatPromptTemplate.from_template(
"What is the best name to describe "
"a company that makes {product}?"
"Give me a concise and terse answer please."
)
chain = prompt | llm
product = "Queen Size Sheet Set"
print(chain.invoke(product))Royal Slumber Co.Simple Sequential Chain
It works for the case where each individual chains have a single input and a single output.
llm = Ollama(model=LLM_MODEL)
prompt_name = ChatPromptTemplate.from_template(
"What is the best name to describe \
a company that makes {product}?"
"Give me a concise and terse answer please."
)
prompt_desc = ChatPromptTemplate.from_template(
"Write a 20 words description for the following"
"company:{company_name}"
"Give me a concise and terse answer please."
)
chain_name = LLMChain(llm=llm, prompt=prompt_name)
chain_desc = LLMChain(llm=llm, prompt=prompt_desc)
chain = SimpleSequentialChain(
chains=[chain_name, chain_desc], verbose=True
)
output = chain.invoke(product)['output']
print(output)
> Entering new SimpleSequentialChain chain.../home/yushi/.env-langchain/lib/python3.12/site-packages/langchain_core/_api/deprecation.py:139: LangChainDeprecationWarning: The class `LLMChain` was deprecated in LangChain 0.1.17 and will be removed in 1.0. Use RunnableSequence, e.g., `prompt | llm` instead.
warn_deprecated(RoyalRests Inc.
"RoyalRests Inc. provides luxurious final rest services, specializing in elegant cremations and burials with meticulous attention to detail."
> Finished chain.
"RoyalRests Inc. provides luxurious final rest services, specializing in elegant cremations and burials with meticulous attention to detail."Sequential Chain
Sequential Chain allow use to construct complicated workflows
flowchart LR
A(French Review) --> B[English Review]
B --> S[English Summary]
A --> L[Language]
L --> R(French Response)
S --> Rprompt_tran = ChatPromptTemplate.from_template(
"Translate the following review to english:"
"\n\n{review}"
"\nbe concise and do not add anything other than the review text."
)
prompt_summ = ChatPromptTemplate.from_template(
"Can you summarize the following review in 1 sentence:"
"\n\n{review_english}"
"\nbe concise and do not add anything other than the review text."
)
prompt_lang = ChatPromptTemplate.from_template(
"What language is the following review:\n\n{review}"
"\nbe concise and do not add anything other than the language name."
)
prompt_resp = ChatPromptTemplate.from_template(
"Write a follow up response to the following "
"summary in the specified language:"
"\n\nSummary: {summary}\n\nLanguage: {language}"
"\nbe concise and do not add anything other than the response."
)
chain_tran = LLMChain(llm=llm, prompt=prompt_tran, output_key="review_english")
chain_summ = LLMChain(llm=llm, prompt=prompt_summ, output_key="summary")
chain_lang = LLMChain(llm=llm, prompt=prompt_lang, output_key="language")
chain_resp = LLMChain(llm=llm, prompt=prompt_resp, output_key="response")
chain = SequentialChain(
chains=[chain_tran, chain_summ, chain_lang, chain_resp],
input_variables=["review"],
output_variables=["review_english", "summary", "response"],
verbose=True
)
review = df.Review[4]
outputs = chain.invoke(review)
for term, content in outputs.items():
print(f"## {term}")
print(content + "\n")
> Entering new SequentialChain chain...
> Finished chain.
## review
I loved this product. But they only seem to last a few months. The company was great replacing the first one (the frother falls out of the handle and can't be fixed). The after 4 months my second one did the same. I only use the frother for coffee once a day. It's not overuse or abuse. I'm very disappointed and will look for another. As I understand they will only replace once. Anyway, if you have one good luck.
## review_english
I loved this product. But they only seem to last a few months. The company was great replacing the first one (the frother falls out of the handle and can't be fixed). The after 4 months my second one did the same. I only use the frother for coffee once a day. It's not overuse or abuse. I'm very disappointed and will look for another. As I understand they will only replace once. Anyway, if you have one good luck.
## summary
The reviewer loved the product initially but was disappointed when it stopped working after a few months, with the company replacing one unit but refusing to repeat the replacement process.
## response
I'm shocked by the lack of commitment from your team. Initially, I was thrilled with the product's performance, but when it stopped working after just a few months, my enthusiasm quickly turned to frustration. The replacement of one unit was a hollow gesture, and I expected better from you given the initial promise.
Router Chain
- Several specialist chains for specific tasks.
- A router chain to choose which specialist chain to use.
- A default chain is called when the router can not decide.
flowchart LR
I(Input) --> R(Router)
R --> P(Physics)
R --> C(Computer Science)
R --> D(default)from langchain.chains.router import MultiPromptChain
from langchain.chains.router.llm_router import LLMRouterChain,RouterOutputParserHere we create the chains for the router to choose from.
physics_template = """You are a very smart physics professor. \
You are great at answering questions about physics in a concise\
and easy to understand manner. \
When you don't know the answer to a question you admit\
that you don't know.
Try to be as concise as possible.
Here is a question:
{input}"""
computerscience_template = """ You are a successful computer scientist.\
You have a passion for creativity, collaboration,\
forward-thinking, confidence, strong problem-solving capabilities,\
understanding of theories and algorithms, and excellent communication \
skills. You are great at answering coding questions. \
You are so good because you know how to solve a problem by \
describing the solution in imperative steps \
that a machine can easily interpret and you know how to \
choose a solution that has a good balance between \
time complexity and space complexity.
Try to be as concise as possible.
Here is a question:
{input}"""
prompts = [
{
"name": "physics",
"description": "Good for answering questions about physics",
"prompt_template": physics_template
},
{
"name": "computer science",
"description": "Good for answering computer science questions",
"prompt_template": computerscience_template
}
]
destination_chains = {}
for p_info in prompts:
name = p_info["name"]
prompt_template = p_info["prompt_template"]
prompt = ChatPromptTemplate.from_template(template=prompt_template)
chain = LLMChain(llm=llm, prompt=prompt)
destination_chains[name] = chain
default_prompt = ChatPromptTemplate.from_template("{input}")
default_chain = LLMChain(llm=llm, prompt=default_prompt)Here we prepare for the router chain
destinations = [f"{p['name']}: {p['description']}" for p in prompts]
destinations_str = "\n".join(destinations)
print(destinations_str)physics: Good for answering questions about physics
computer science: Good for answering computer science questionsMULTI_PROMPT_ROUTER_TEMPLATE = r"""Given a raw text input to a \
language model select the model prompt best suited for the input. \
You will be given the names of the available prompts and a \
description of what the prompt is best suited for. \
You may also revise the original input if you think that revising\
it will ultimately lead to a better response from the language model.
<< FORMATTING >>
Return a markdown code snippet with a JSON object formatted to look like:
```json
{{{{
"destination": string \ name of the prompt to use or "DEFAULT"
"next_inputs": string \ a potentially modified version of the original input
}}}}
```
REMEMBER: "destination" MUST be one of the candidate prompt \
names specified below OR it can be "DEFAULT" if the input is not\
well suited for any of the candidate prompts.
REMEMBER: "next_inputs" can just be the original input \
if you don't think any modifications are needed.
<< CANDIDATE PROMPTS >>
{destinations}
<< INPUT >>
{{input}}
<< OUTPUT (remember to include the ```json)>>"""
router_template = MULTI_PROMPT_ROUTER_TEMPLATE.format(
destinations=destinations_str
)
router_prompt = PromptTemplate(
template=router_template,
input_variables=["input"],
output_parser=RouterOutputParser(),
)
router_chain = LLMRouterChain.from_llm(llm, router_prompt)chain = MultiPromptChain(
router_chain=router_chain,
destination_chains=destination_chains,
default_chain=default_chain, verbose=True
)result = chain.invoke("What is Percus Yevick equation?")
print(result['text'])
> Entering new MultiPromptChain chain...
physics: {'input': 'Percus Yevick equation?'}
> Finished chain.
A classic! The Percus-Yevick equation, also known as the PY equation, is a mathematical formula used in particle physics to describe the scattering of particles with spin.
It's a bit of a mouthful, but the PY equation is essentially an extension of the Bethe-Heitler formula, which accounts for the electromagnetic interaction between two charged particles. The Percus-Yevick equation takes this a step further by incorporating the effects of quantum mechanics and relativity on particle scattering.
In simpler terms, the PY equation helps us understand how subatomic particles interact with each other at very high energies. It's an important tool in particle physics research, particularly when studying processes like electron-positron annihilation or hadronic collisions.
Now, if you're interested in a more detailed explanation or want to know about its applications, I'm here to help!result = chain.invoke("what is the travelling salesman problem")
print(result['text'])
> Entering new MultiPromptChain chain...
computer science: {'input': "Is there a specific solution or algorithm you're looking for regarding the Travelling Salesman Problem?"}
> Finished chain.
A classic problem!
For TSP, I'd recommend exploring dynamic programming approaches like Christofides Algorithm or 2-Opt Heuristics. These methods balance time and space complexity well.
If you're interested in exact solutions, we can discuss Concorde's algorithm, which uses branch-and-bound with cutting planes to find the optimal tour. However, this method is more complex and may not be suitable for large instances.
Which approach are you leaning towards?result = chain.invoke("what is NGS?")
print(result['text'])
> Entering new MultiPromptChain chain...
computer science: {'input': 'What is Next Generation Sequencing (NGS)?'}
> Finished chain.
Next-Generation Sequencing (NGS) refers to the technology used in DNA sequencing, which allows for rapid and cost-effective analysis of an individual's entire genome or specific regions of interest. NGS enables the simultaneous analysis of millions of DNA fragments, resulting in faster turnaround times and higher resolution than traditional Sanger sequencing methods.QA for Documents
from langchain.document_loaders import CSVLoader
from langchain.indexes import VectorstoreIndexCreator
from langchain.vectorstores import DocArrayInMemorySearch
from langchain_community.embeddings import OllamaEmbeddings
from langchain.chains import RetrievalQA
from IPython.display import display, MarkdownOverview
EMBEDDING_MODEL = "all-minilm"
file = 'OutdoorClothingCatalog_1000.csv'
loader = CSVLoader(file_path=file)
index = VectorstoreIndexCreator(
embedding=OllamaEmbeddings(model=EMBEDDING_MODEL),
vectorstore_cls=DocArrayInMemorySearch
).from_loaders([loader])/home/yushi/.env-langchain/lib/python3.12/site-packages/pydantic/_migration.py:283: UserWarning: `pydantic.error_wrappers:ValidationError` has been moved to `pydantic:ValidationError`.
warnings.warn(f'`{import_path}` has been moved to `{new_location}`.')LLM_MODEL = "llama3"
llm = Ollama(model=LLM_MODEL)
query ="Please list all your shirts with sun protection \
in a table in markdown and summarize each one."
response = index.query(query, llm=llm)display(Markdown(response))Here are the shirts with sun protection listed in a table:
| Shirts | Description | Sun Protection |
|---|---|---|
| Sun Shield Shirt | High-performance sun shirt for UV ray protection | SPF 50+ (blocks 98% of sun’s harmful rays) |
| Tropical Breeze Shirt | Lightweight, breathable long-sleeve UPF shirt for superior sun protection | SPF 50+ (blocks 98% of sun’s harmful rays) |
| Women’s Tropical Tee, Sleeveless | Five-star sleeveless button-up shirt with SunSmart UPF 50+ rating | SPF 50+ (blocks 98% of sun’s harmful rays) |
| Girls’ Beachside Breeze Shirt, Half-Sleeve | Rash guard-style swim shirt with built-in UPF 50+ protection | SPF 50+ (blocks 98% of sun’s harmful rays) |
Summary: All the shirts listed have high-performance fabric with SPF 50+ sun protection, blocking 98% of the sun’s harmful UV rays. They are suitable for outdoor activities such as swimming, fishing, and travel to protect skin from damage.
Step-by-Step
loader = CSVLoader(file_path=file)
docs = loader.load()
docs[0]Document(metadata={'source': 'OutdoorClothingCatalog_1000.csv', 'row': 0}, page_content=": 0\nname: Women's Campside Oxfords\ndescription: This ultracomfortable lace-to-toe Oxford boasts a super-soft canvas, thick cushioning, and quality construction for a broken-in feel from the first time you put them on. \n\nSize & Fit: Order regular shoe size. For half sizes not offered, order up to next whole size. \n\nSpecs: Approx. weight: 1 lb.1 oz. per pair. \n\nConstruction: Soft canvas material for a broken-in feel and look. Comfortable EVA innersole with Cleansport NXT® antimicrobial odor control. Vintage hunt, fish and camping motif on innersole. Moderate arch contour of innersole. EVA foam midsole for cushioning and support. Chain-tread-inspired molded rubber outsole with modified chain-tread pattern. Imported. \n\nQuestions? Please contact us for any inquiries.")embeddings = OllamaEmbeddings(model=EMBEDDING_MODEL)
embed = embeddings.embed_query("Hi my name is Yushi")
len(embed), embed[:5](384,
[-0.40551620721817017,
0.2655082643032074,
0.12681838870048523,
0.42601901292800903,
-0.5726575255393982])db = DocArrayInMemorySearch.from_documents(docs, embeddings)llm = Ollama(model=LLM_MODEL, verbose=True)
retriever = db.as_retriever()
query = "Please suggest a shirt with sunblocking"
docs = db.similarity_search(query, k=4)
qdocs = "".join(
[docs[i].page_content for i in range(len(docs))]
)
response = llm.invoke(
f"{qdocs} Question: Please list all your \
shirts with sun protection in a table in markdown"
)
display(Markdown(response))Here is the list of shirts with sun protection in a table format using Markdown:
| Shirt Name | Description | Sun Protection |
|---|---|---|
| Sun Shield Shirt | Block the sun, not the fun - our high-performance sun shirt is guaranteed to protect from harmful UV rays. | UPF 50+ rated, blocks 98% of the sun’s harmful rays |
| Tropical Breeze Shirt | Beat the heat in this lightweight, breathable long-sleeve men’s UPF shirt, offering superior SunSmart protection from the sun’s harmful rays. | UPF 50+ rated, blocks 98% of the sun’s harmful rays |
| Women’s Tropical Tee, Sleeveless | Our five-star sleeveless button-up shirt has a fit to flatter and SunSmart protection to block the sun’s harmful UV rays. | UPF 50+ rated, blocks 98% of the sun’s harmful rays |
| Men’s Plaid Tropic Shirt, Short-Sleeve | Our Ultracomfortable sun protection is rated to UPF 50+, helping you stay cool and dry. | UPF 50+ rated, blocks 98% of the sun’s harmful rays |
Note: All shirts have a UPF (Ultraviolet Protection Factor) rating of 50+, which means they block at least 95% of UVB rays and at least 90% of UVA rays.
RetrievalQA Chain
Tbe RetrievalQA chain can acheive identical RAG results.
qa_stuff = RetrievalQA.from_chain_type(
llm=llm,
chain_type="stuff", # "stuffs" all documents into context
retriever=db.as_retriever(),
verbose=True
)
query = "Please list all your shirts with sun protection in a table in markdown."
response = qa_stuff.invoke(query)
display(Markdown(response['result']))
> Entering new RetrievalQA chain...
> Finished chain.Here are the shirts with sun protection listed in a table:
| Shirt Name | Sun Protection |
|---|---|
| Sun Shield Shirt by [name] | UPF 50+, SPF 50+ (blocks 98% of sun’s harmful rays) |
| Women’s Tropical Tee, Sleeveless | UPF 50+, SPF 50+ (blocks 98% of sun’s harmful rays) |
| Tropical Breeze Shirt | UPF 50+, SPF 50+ (blocks 98% of sun’s harmful rays) |
| Girls’ Beachside Breeze Shirt, Half-Sleeve | UPF 50+, SPF 50+ (blocks 98% of sun’s harmful rays) |
Let me know if you need anything else!
Retrievel Methods
Stuff
put all retrieved chunks into the context. Then call LLM once. The most common method.
flowchart LR
A[Documents] --> B(chunk)
A --> C(chunk)
A --> D(chunk)
B --> L[LLM]
C --> L
D --> L --> R[Response]Map Reduce
pass each chunk and query to LLM, then use LLM to process the responses. The second most common method.
flowchart LR
A[Documents] --> B(chunk) --> LB[LLM]
A --> C(chunk) --> LC[LLM]
A --> D(chunk) --> LD[LLM]
LB --> L[LLM]
LC --> L
LD --> L --> R[Response]Refine
Builds the answer iteratively.
flowchart LR
A[Documents] --> B(chunk) --> LB[LLM]
A --> C(chunk) --> LC[LLM]
A --> D(chunk) --> LD[LLM]
LB --> LC
LC --> LD
LD --> R[Response]Map Rerank
flowchart LR
A[Documents] --> B(chunk) --> LB[LLM] --> SB(Score 40)
A --> C(chunk) --> LC[LLM] --> SC(Score 60) --> R[Response]
A --> D(chunk) --> LD[LLM] --> SD(Score 32)LLM Evaluation
General Instruction
- Visualisation: to understand the input/output data.
- Create examples for evaluation.
- Use LLM to evaluate.
Here is a Step by Step guide.
Setting Up
file = 'OutdoorClothingCatalog_1000.csv'
loader = CSVLoader(file_path=file)
data = loader.load()
index = VectorstoreIndexCreator(
embedding=OllamaEmbeddings(model=EMBEDDING_MODEL),
vectorstore_cls=DocArrayInMemorySearch
).from_loaders([loader])LLM_MODEL = "llama3"
llm = Ollama(model=LLM_MODEL)
qa = RetrievalQA.from_chain_type(
llm=llm,
chain_type="stuff",
retriever=index.vectorstore.as_retriever(),
)To evaluate the result, we should create standard “Question” + “Answer” examples.
Observing the Data
We can generate such examples by observing the data.
print(data[10].page_content): 10
name: Cozy Comfort Pullover Set, Stripe
description: Perfect for lounging, this striped knit set lives up to its name. We used ultrasoft fabric and an easy design that's as comfortable at bedtime as it is when we have to make a quick run out.
Size & Fit
- Pants are Favorite Fit: Sits lower on the waist.
- Relaxed Fit: Our most generous fit sits farthest from the body.
Fabric & Care
- In the softest blend of 63% polyester, 35% rayon and 2% spandex.
Additional Features
- Relaxed fit top with raglan sleeves and rounded hem.
- Pull-on pants have a wide elastic waistband and drawstring, side pockets and a modern slim leg.
Imported.Creating Examples Manually
examples = [
{
"query": "Do the Cozy Comfort Pullover Set have side pockets?",
"answer": "Yes"
},
{
"query": "What collection is the Ultra-Lofty 850 Stretch Down Hooded Jacket from?",
"answer": "The DownTek collection"
}
]Generating Examples Automatically with LLMs
the following code was copied from
langchain/evaluation/qa/generate_chain.pySo that our code will be compatible with the new interface for langchain 0.2+
from langchain.output_parsers.regex import RegexParser
qa_gen_parser = RegexParser(
regex=r"QUESTION: (.*?)\n+ANSWER: (.*)", output_keys=["query", "answer"]
)
qa_gen_template = """You are a teacher coming up with questions to ask on a quiz.
Given the following document, please generate a question and answer based on that document.
Example Format:
<Begin Document>
...
<End Document>
QUESTION: question here
ANSWER: answer here
These questions should be detailed and be based explicitly on information in the document. Begin!
<Begin Document>
{doc}
<End Document>"""
qa_gen_prompt = PromptTemplate(
input_variables=["doc"],
template=qa_gen_template,
)We now build the QAGenerateChain with the runable interface.
For old langchain version (<= 0.1), we can use
qa_gen = QAGenerateChain.from_llm(llm)
new_examples = qa_gen.apply_and_parse(
[{"doc": t} for t in data[:5]]
)LLM_MODEL = "llama3"
llm = Ollama(model=LLM_MODEL)
qa_gen = qa_gen_prompt | llm | qa_gen_parser
new_examples = qa_gen.batch(
[{"doc": t} for t in data[:5]]
)new_examples[{'query': "What type of material is used in the construction of the Women's Campside Oxfords?",
'answer': 'Soft canvas material.'},
{'query': 'What percentage of recycled materials are used to make the fabric of the Recycled Waterhog Dog Mat?',
'answer': '94%'},
{'query': "What feature of this toddler's swimsuit provides the highest rated sun protection possible, blocking 98% of the sun's harmful rays?",
'answer': 'The UPF 50+ rated fabric.'},
{'query': "What percentage of the swimtop's body is made up of recycled nylon?",
'answer': '82%'},
{'query': 'What technology does EcoFlex 3L Storm Pants feature to provide enhanced breathability?',
'answer': 'TEK O2 technology.'}]Manually Evaluate
examples += new_examples
qa.invoke(examples[0]["query"]){'query': 'Do the Cozy Comfort Pullover Set have side pockets?',
'result': 'According to the Additional Features section of the product description, the pull-on pants in the Cozy Comfort Pullover Set have "side pockets". Therefore, the answer is:\n\nYes, the Cozy Comfort Pullover Set has side pockets.'}examples[0]["answer"]'Yes'Debug
We can use the following setup to use the debug mode and see the exact input/ouput for the LLM.
from langchain.globals import set_debug
set_debug(True)
response = qa.invoke(examples[0]["query"])
set_debug(False)[chain/start] [chain:RetrievalQA] Entering Chain run with input:
{
"query": "Do the Cozy Comfort Pullover Set have side pockets?"
}
[chain/start] [chain:RetrievalQA > chain:StuffDocumentsChain] Entering Chain run with input:
[inputs]
[chain/start] [chain:RetrievalQA > chain:StuffDocumentsChain > chain:LLMChain] Entering Chain run with input:
{
"question": "Do the Cozy Comfort Pullover Set have side pockets?",
"context": ": 73\nname: Cozy Cuddles Knit Pullover Set\ndescription: Perfect for lounging, this knit set lives up to its name. We used ultrasoft fabric and an easy design that's as comfortable at bedtime as it is when we have to make a quick run out. \n\nSize & Fit \nPants are Favorite Fit: Sits lower on the waist. \nRelaxed Fit: Our most generous fit sits farthest from the body. \n\nFabric & Care \nIn the softest blend of 63% polyester, 35% rayon and 2% spandex.\n\nAdditional Features \nRelaxed fit top with raglan sleeves and rounded hem. \nPull-on pants have a wide elastic waistband and drawstring, side pockets and a modern slim leg. \nImported.\n\nBerber fleece-lined pockets keep hands warm. One pocket has a hidden security pocket tucked inside. Imported.\n\n: 10\nname: Cozy Comfort Pullover Set, Stripe\ndescription: Perfect for lounging, this striped knit set lives up to its name. We used ultrasoft fabric and an easy design that's as comfortable at bedtime as it is when we have to make a quick run out.\n\nSize & Fit\n- Pants are Favorite Fit: Sits lower on the waist.\n- Relaxed Fit: Our most generous fit sits farthest from the body.\n\nFabric & Care\n- In the softest blend of 63% polyester, 35% rayon and 2% spandex.\n\nAdditional Features\n- Relaxed fit top with raglan sleeves and rounded hem.\n- Pull-on pants have a wide elastic waistband and drawstring, side pockets and a modern slim leg.\n\nImported.\n\nAdditional Features: Bonded construction insulates for extra warmth and won't stretch out of shape. Classic shirt jac styling, with collar and modified hem for extra coverage. Full snap placket for easy on/off. Chest pockets with snap closures. Pencil pocket on chest. Underarm gussets enhance mobility. Berber fleece-lined pockets keep hands warm. One pocket has a hidden security pocket tucked inside. Imported"
}
[llm/start] [chain:RetrievalQA > chain:StuffDocumentsChain > chain:LLMChain > llm:Ollama] Entering LLM run with input:
{
"prompts": [
"Use the following pieces of context to answer the question at the end. If you don't know the answer, just say that you don't know, don't try to make up an answer.\n\n: 73\nname: Cozy Cuddles Knit Pullover Set\ndescription: Perfect for lounging, this knit set lives up to its name. We used ultrasoft fabric and an easy design that's as comfortable at bedtime as it is when we have to make a quick run out. \n\nSize & Fit \nPants are Favorite Fit: Sits lower on the waist. \nRelaxed Fit: Our most generous fit sits farthest from the body. \n\nFabric & Care \nIn the softest blend of 63% polyester, 35% rayon and 2% spandex.\n\nAdditional Features \nRelaxed fit top with raglan sleeves and rounded hem. \nPull-on pants have a wide elastic waistband and drawstring, side pockets and a modern slim leg. \nImported.\n\nBerber fleece-lined pockets keep hands warm. One pocket has a hidden security pocket tucked inside. Imported.\n\n: 10\nname: Cozy Comfort Pullover Set, Stripe\ndescription: Perfect for lounging, this striped knit set lives up to its name. We used ultrasoft fabric and an easy design that's as comfortable at bedtime as it is when we have to make a quick run out.\n\nSize & Fit\n- Pants are Favorite Fit: Sits lower on the waist.\n- Relaxed Fit: Our most generous fit sits farthest from the body.\n\nFabric & Care\n- In the softest blend of 63% polyester, 35% rayon and 2% spandex.\n\nAdditional Features\n- Relaxed fit top with raglan sleeves and rounded hem.\n- Pull-on pants have a wide elastic waistband and drawstring, side pockets and a modern slim leg.\n\nImported.\n\nAdditional Features: Bonded construction insulates for extra warmth and won't stretch out of shape. Classic shirt jac styling, with collar and modified hem for extra coverage. Full snap placket for easy on/off. Chest pockets with snap closures. Pencil pocket on chest. Underarm gussets enhance mobility. Berber fleece-lined pockets keep hands warm. One pocket has a hidden security pocket tucked inside. Imported\n\nQuestion: Do the Cozy Comfort Pullover Set have side pockets?\nHelpful Answer:"
]
}
[llm/end] [chain:RetrievalQA > chain:StuffDocumentsChain > chain:LLMChain > llm:Ollama] [965ms] Exiting LLM run with output:
{
"generations": [
[
{
"text": "According to the context, the answer is:\n\nYes, the Cozy Comfort Pullover Set (as well as the Cozy Cuddles Knit Pullover Set) have pull-on pants with \"side pockets\".",
"generation_info": {
"model": "llama3",
"created_at": "2024-07-19T05:38:59.291526683Z",
"response": "",
"done": true,
"done_reason": "stop",
"context": [
128006,
882,
128007,
198,
198,
10464,
279,
2768,
9863,
315,
2317,
311,
4320,
279,
3488,
520,
279,
842,
13,
1442,
499,
1541,
6,
83,
1440,
279,
4320,
11,
1120,
2019,
430,
499,
1541,
6,
83,
1440,
11,
1541,
6,
83,
1456,
311,
1304,
709,
459,
4320,
13,
271,
25,
220,
5958,
198,
609,
25,
3623,
4341,
356,
8512,
645,
13934,
275,
35962,
57597,
2638,
198,
4789,
25,
24118,
369,
84244,
287,
11,
420,
53203,
743,
6439,
709,
311,
1202,
836,
13,
1226,
1511,
8725,
13811,
14450,
13354,
323,
459,
4228,
2955,
430,
6,
82,
439,
10882,
520,
89607,
439,
433,
374,
994,
584,
617,
311,
1304,
264,
4062,
1629,
704,
13,
720,
198,
1730,
220,
5,
30990,
220,
198,
47,
1821,
527,
40822,
30990,
25,
328,
1220,
4827,
389,
279,
29142,
13,
220,
198,
6882,
52551,
30990,
25,
5751,
1455,
28771,
5052,
23874,
3117,
61943,
505,
279,
2547,
13,
720,
198,
82831,
220,
5,
10852,
220,
198,
644,
279,
8579,
478,
20955,
315,
220,
5495,
4,
56170,
11,
220,
1758,
4,
18803,
263,
323,
220,
17,
4,
993,
45626,
13,
198,
198,
30119,
20289,
220,
198,
6882,
52551,
5052,
1948,
449,
21108,
10946,
43475,
323,
18460,
17728,
13,
220,
198,
37168,
12,
263,
25567,
617,
264,
7029,
36573,
29142,
7198,
323,
4128,
928,
11,
3185,
30278,
323,
264,
6617,
30453,
2531,
13,
220,
198,
11772,
291,
13,
198,
198,
39379,
655,
93090,
12,
15472,
30278,
2567,
6206,
8369,
13,
3861,
18301,
706,
264,
8340,
4868,
18301,
61218,
4871,
13,
73366,
13,
271,
25,
220,
605,
198,
609,
25,
3623,
4341,
44842,
35962,
57597,
2638,
11,
60666,
198,
4789,
25,
24118,
369,
84244,
287,
11,
420,
68690,
53203,
743,
6439,
709,
311,
1202,
836,
13,
1226,
1511,
8725,
13811,
14450,
13354,
323,
459,
4228,
2955,
430,
6,
82,
439,
10882,
520,
89607,
439,
433,
374,
994,
584,
617,
311,
1304,
264,
4062,
1629,
704,
13,
198,
198,
1730,
220,
5,
30990,
198,
12,
67553,
527,
40822,
30990,
25,
328,
1220,
4827,
389,
279,
29142,
13,
198,
12,
6649,
52551,
30990,
25,
5751,
1455,
28771,
5052,
23874,
3117,
61943,
505,
279,
2547,
13,
198,
198,
82831,
220,
5,
10852,
198,
12,
763,
279,
8579,
478,
20955,
315,
220,
5495,
4,
56170,
11,
220,
1758,
4,
18803,
263,
323,
220,
17,
4,
993,
45626,
13,
198,
198,
30119,
20289,
198,
12,
6649,
52551,
5052,
1948,
449,
21108,
10946,
43475,
323,
18460,
17728,
13,
198,
12,
32928,
12,
263,
25567,
617,
264,
7029,
36573,
29142,
7198,
323,
4128,
928,
11,
3185,
30278,
323,
264,
6617,
30453,
2531,
13,
198,
198,
11772,
291,
13,
198,
198,
30119,
20289,
25,
24537,
291,
8246,
1672,
24031,
369,
5066,
41463,
323,
2834,
6,
83,
14841,
704,
315,
6211,
13,
22591,
15845,
60943,
42428,
11,
449,
37204,
323,
11041,
17728,
369,
5066,
10401,
13,
8797,
10885,
628,
5827,
369,
4228,
389,
14,
1885,
13,
42871,
30278,
449,
10885,
61000,
13,
393,
17438,
18301,
389,
15489,
13,
9636,
2227,
342,
355,
5022,
18885,
31139,
13,
9084,
655,
93090,
12,
15472,
30278,
2567,
6206,
8369,
13,
3861,
18301,
706,
264,
8340,
4868,
18301,
61218,
4871,
13,
73366,
198,
198,
14924,
25,
3234,
279,
3623,
4341,
44842,
35962,
57597,
2638,
617,
3185,
30278,
30,
198,
12978,
1285,
22559,
25,
128009,
128006,
78191,
128007,
198,
198,
11439,
311,
279,
2317,
11,
279,
4320,
374,
25,
198,
198,
9642,
11,
279,
3623,
4341,
44842,
35962,
57597,
2638,
220,
7,
300,
1664,
439,
279,
3623,
4341,
356,
8512,
645,
13934,
275,
35962,
57597,
2638,
8,
617,
6958,
12,
263,
25567,
449,
220,
1,
3002,
30278,
3343
],
"total_duration": 911795232,
"load_duration": 14965876,
"prompt_eval_count": 542,
"prompt_eval_duration": 22798000,
"eval_count": 44,
"eval_duration": 832155000
},
"type": "Generation"
}
]
],
"llm_output": null,
"run": null
}
[chain/end] [chain:RetrievalQA > chain:StuffDocumentsChain > chain:LLMChain] [966ms] Exiting Chain run with output:
{
"text": "According to the context, the answer is:\n\nYes, the Cozy Comfort Pullover Set (as well as the Cozy Cuddles Knit Pullover Set) have pull-on pants with \"side pockets\"."
}
[chain/end] [chain:RetrievalQA > chain:StuffDocumentsChain] [967ms] Exiting Chain run with output:
{
"output_text": "According to the context, the answer is:\n\nYes, the Cozy Comfort Pullover Set (as well as the Cozy Cuddles Knit Pullover Set) have pull-on pants with \"side pockets\"."
}
[chain/end] [chain:RetrievalQA] [986ms] Exiting Chain run with output:
{
"result": "According to the context, the answer is:\n\nYes, the Cozy Comfort Pullover Set (as well as the Cozy Cuddles Knit Pullover Set) have pull-on pants with \"side pockets\"."
}LLM assisted evaluation
The default prompt can be found in the file
langchain/evaluation/qa/eval_prompt.pyHere we edited the prompt so that the output is good for llama3.
from langchain.evaluation.qa import QAEvalChain
grade_template = """You are a teacher grading a quiz.
You are given a question, the student's answer, and the true answer,
and are asked to score the student answer as either CORRECT or INCORRECT.
Example Input Format:
-----
QUESTION: question here
STUDENT ANSWER: student's answer here
TRUE ANSWER: true answer here
-----
Example Output:
CORRECT or INCORRECT
Grade the student answers based ONLY on their factual accuracy.
Ignore differences in punctuation and phrasing between the student answer and true answer.
It is OK if the student answer contains more information than the true answer,
as long as it does not contain any conflicting statements.
Return the final grade. The only options are between CORRECT and INCORRECT.
Here are the actual input data:
QUESTION: {query}
STUDENT ANSWER: {result}
TRUE ANSWER: {answer}
"""
grade_prompt = PromptTemplate(
input_variables=["query", "result", "answer"],
template=grade_template
)
eval_chain = QAEvalChain.from_llm(llm, prompt=grade_prompt)predictions = qa.batch(examples)
graded_outputs = eval_chain.evaluate(examples, predictions)
graded_outputs[{'results': 'CORRECT'},
{'results': 'CORRECT'},
{'results': 'CORRECT'},
{'results': 'CORRECT'},
{'results': 'CORRECT'},
{'results': 'CORRECT'},
{'results': 'CORRECT'}]for i, eg in enumerate(examples):
print(f"Example {i}:")
print("Question: " + predictions[i]['query'])
print("Real Answer: " + predictions[i]['answer'])
print("Predicted Answer: " + predictions[i]['result'])
print("Predicted Grade: " + graded_outputs[i]['results'])
print()Example 0:
Question: Do the Cozy Comfort Pullover Set have side pockets?
Real Answer: Yes
Predicted Answer: According to the additional features of the Cozy Comfort Pullover Set, it is stated that "Pull-on pants have a wide elastic waistband and drawstring, side pockets and a modern slim leg." Therefore, yes, the Cozy Comfort Pullover Set does have side pockets.
Predicted Grade: CORRECT
Example 1:
Question: What collection is the Ultra-Lofty 850 Stretch Down Hooded Jacket from?
Real Answer: The DownTek collection
Predicted Answer: The Ultra-Loft 850 Stretch Down Hooded Jacket is from the DownTek collection.
Predicted Grade: CORRECT
Example 2:
Question: What type of material is used in the construction of the Women's Campside Oxfords?
Real Answer: Soft canvas material.
Predicted Answer: According to the description, the Women's Campside Oxfords are made from "soft canvas material" with a comfortable EVA innersole and antimicrobial odor control.
Predicted Grade: CORRECT
Example 3:
Question: What percentage of recycled materials are used to make the fabric of the Recycled Waterhog Dog Mat?
Real Answer: 94%
Predicted Answer: According to the context, the construction of the Recycled Waterhog Dog Mat includes:
"24 oz. polyester fabric made from 94% recycled materials."
So, the answer is 94%.
Predicted Grade: CORRECT
Example 4:
Question: What feature of this toddler's swimsuit provides the highest rated sun protection possible, blocking 98% of the sun's harmful rays?
Real Answer: The UPF 50+ rated fabric.
Predicted Answer: Based on the context provided, the answer is that the UPF 50+ rated fabric provides the highest rated sun protection possible, blocking 98% of the sun's harmful rays. This feature is mentioned in all three products: Infant and Toddler Girls' Coastal Chill Swimsuit, Two-Piece; Girls' Beachside Breeze Shirt, Half-Sleeve; and Beach Babe Reversible Swimsuit.
Predicted Grade: CORRECT
Example 5:
Question: What percentage of the swimtop's body is made up of recycled nylon?
Real Answer: 82%
Predicted Answer: Based on the context, I found the following information:
For Refresh Swimwear, V-Neck Tankini Contrasts:
Body in 82% recycled nylon with 18% Lycra spandex.
For ReVamp Swimwear, V-Neck Bikini Top Print:
Body in 82% recycled nylon with 18% Lycra spandex.
Given this information, I can conclude that the percentage of recycled nylon in the swimtop's body is **82%**.
Predicted Grade: CORRECT
Example 6:
Question: What technology does EcoFlex 3L Storm Pants feature to provide enhanced breathability?
Real Answer: TEK O2 technology.
Predicted Answer: According to the context, EcoFlex 3L Storm Pants feature TEK O2 technology, which provides enhanced breathability.
Predicted Grade: CORRECT
Agents
from langchain_experimental.llms.ollama_functions import OllamaFunctions
from langchain.agents import load_tools, create_react_agent, AgentExecutor
from langchain_core.pydantic_v1 import BaseModel, FieldGeneral Notes
- set temperature to zero to get precise results.
Creating a tool calling LLM
Detail: Ollama model
We have to install langchain-experimental to use the ollama model.
pip install langchain_experimentalI also changed the default system prompt a bit so that llama3 is more likely to return a json file.
Code
LLM_MODEL = "llama3"
TOOL_PROMPT = """
You have access to the following tools:
{tools}
You must always select one of the above tools and respond with only a
JSON object matching the following schema:
{{
"tool": <name of the selected tool>,
"tool_input": <parameters for the selected tool, matching the tool's JSON schema>
}}
DO NOT RESPOND ANYTHING RATHER THAN THE JSON CONTENT !!!
DO NOT RESPOND ANYTHING RATHER THAN THE JSON CONTENT !!!
"""
llm = OllamaFunctions(
model=LLM_MODEL,
temperature=0.0,
verbose=True,
tool_system_prompt_template=TOOL_PROMPT
)
class GetWeather(BaseModel):
"""Get the current weather in a given location"""
location: str = Field(
..., description="The city and state, e.g. San Francisco, CA"
)
llm_with_tools = llm.bind_tools([GetWeather])
ai_message = llm_with_tools.invoke(
"what is the weather like in San Francisco",
)
print(ai_message)content='' id='run-b1873a2a-a6e4-495b-b92d-9737e97605a2-0' tool_calls=[{'name': 'GetWeather', 'args': {'location': 'San Francisco, CA'}, 'id': 'call_4eb652ffba9547b48ab49cc301d50385'}]Creating an Agent
Detail: LangChain version
For old langchain version, the agent can be created via the following code.
from langchain.agents import initialize_agent, AgentType
tools = load_tools(["llm-math","wikipedia"], llm=llm)
agent= initialize_agent(
tools, llm,
agent=AgentType.CHAT_ZERO_SHOT_REACT_DESCRIPTION,
handle_parsing_errors=True,
verbose=True
)Here we use the newer interface for langchain 0.2+.
LLM_MODEL = "llama3"
TOOL_PROMPT = """
You have access to the following tools:
{tools}
You must always select one of the above tools and respond with only a
JSON object matching the following schema:
{{
"tool": <name of the selected tool>,
"tool_input": <parameters for the selected tool, matching the tool's JSON schema>
}}
DO NOT RESPOND ANYTHING RATHER THAN THE JSON CONTENT !!!
DO NOT RESPOND ANYTHING RATHER THAN THE JSON CONTENT !!!
"""
llm = OllamaFunctions(
model=LLM_MODEL,
temperature=0.0,
verbose=True,
tool_system_prompt_template=TOOL_PROMPT
)
react_template = '''
Answer the following questions as best you can. You have access to the following tools:
{tools}
Use the following format:
Question: the input question you must answer
Thought: you should always think about what to do
Action: the action to take, should be one of [{tool_names}]
Action Input: the input to the action
Observation: the result of the action
... (this Thought/Action/Action Input/Observation can repeat N times)
Thought: I now know the final answer
Final Answer: the final answer to the original input question
Follow the follwoing rules strictly no matter what
- DO NOT ADD EXTRA DESCRIPTION FOR the action and action input!
- DO NOT REPEAT ANY INFORMATION!
- DO NOT REPEAT THE QUESTION!
- FOLLOW THE FORMAT REQUIREMENT!
Begin!
Question: {input}
Thought:{agent_scratchpad}
'''
react_prompt = PromptTemplate.from_template(react_template)
tools = load_tools(["wikipedia", "wikipedia"], llm=llm)
agent= create_react_agent(llm, tools, prompt=react_prompt)
agent_executor = AgentExecutor(agent=agent, tools=tools, verbose=True)question = "Who is Joe Biden"
result = agent_executor.invoke(
{"input": question}
)
> Entering new AgentExecutor chain...
Here's my response:
Thought: I need to find information about Joe Biden.
Action: wikipedia
Action Input: Joe BidenPage: Joe Biden
Summary: Joseph Robinette Biden Jr. (born November 20, 1942) is an American politician who is the 46th and current president of the United States since 2021. A member of the Democratic Party, he previously served as the 47th vice president from 2009 to 2017 under President Barack Obama and represented Delaware in the United States Senate from 1973 to 2009.
Born in Scranton, Pennsylvania, Biden moved with his family to Delaware in 1953. He graduated from the University of Delaware before earning his law degree from Syracuse University. He was elected to the New Castle County Council in 1970 and to the U.S. Senate in 1972. As a senator, Biden drafted and led the effort to pass the Violent Crime Control and Law Enforcement Act and the Violence Against Women Act. He also oversaw six U.S. Supreme Court confirmation hearings, including the contentious hearings for Robert Bork and Clarence Thomas. Biden ran unsuccessfully for the Democratic presidential nomination in 1988 and 2008. In 2008, Obama chose Biden as his running mate, and he was a close counselor to Obama during his two terms as vice president. In the 2020 presidential election, Biden chose Kamala Harris as his running mate and defeated incumbent Donald Trump. He is the oldest president in U.S. history, and the first to have a female vice president.
As president, Biden signed the American Rescue Plan Act in response to the COVID-19 pandemic and subsequent recession. He signed bipartisan bills on infrastructure and manufacturing. He proposed the Build Back Better Act, which failed in Congress, but aspects of which were incorporated into the Inflation Reduction Act that he signed into law in 2022. Biden appointed Ketanji Brown Jackson to the Supreme Court. He worked with congressional Republicans to resolve the 2023 debt-ceiling crisis by negotiating a deal to raise the debt ceiling. In foreign policy, Biden restored America's membership in the Paris Agreement. He oversaw the complete withdrawal of U.S. troops from Afghanistan that ended the war in Afghanistan, leading to the collapse of the Afghan government and the Taliban seizing control. He responded to the Russian invasion of Ukraine by imposing sanctions on Russia and authorizing civilian and military aid to Ukraine. During the Israel–Hamas war, Biden condemned the actions of Hamas as terrorism, announced military support for Israel and sent limited humanitarian aid to the Gaza Strip.
In April 25, 2023, Biden announced his candidacy for the Democratic nomination in the 2024 presidential election, and is now the presumptive nominee. Following what was widely viewed as a poor performance in a June 2024 debate with Trump and numerous age and health concerns, Biden has faced repeated calls to suspend his candidacy, but has insisted that he will remain a candidate.
Page: Family of Joe Biden
Summary: Joe Biden, the 46th and current president of the United States, has family members who are prominent in law, education, activism and politics. Biden's immediate family became the first family of the United States on his inauguration on January 20, 2021. His immediate family circle was also the second family of the United States from 2009 to 2017, when Biden was vice president. Biden's family is mostly descended from the British Isles, with most of their ancestors coming from Ireland and England, and a smaller number descending from the French.
Of Biden's sixteen great-great-grandparents, ten of them were born in Ireland. He is descended from the Blewitts of County Mayo and the Finnegans of County Louth. One of Biden's great-great-great-grandfathers was born in Sussex, England, and emigrated to Maryland in the United States by 1820.
Page: Presidency of Joe Biden
Summary: Joe Biden's tenure as the 46th president of the United States began with his inauguration on January 20, 2021. Biden, a member of the Democratic Party who previously served as vice president for two terms under President Barack Obama, took offiThought: I now know the final answer.
Final Answer: Joe Biden is an American politician who is the 46th and current president of the United States since 2021.
> Finished chain.Custom Tool
from langchain.agents import tool
from datetime import date@tool
def timer(text: str) -> str:
"""Returns todays date, use this for any \
questions related to knowing todays date. \
The input should always be an empty string, \
and this function will always return todays \
date - any date mathmatics should occur \
outside this function."""
return str(date.today())tools = load_tools(["wikipedia"], llm=llm)
tools.append(timer)
agent= create_react_agent(llm, tools, prompt=react_prompt)
agent_executor = AgentExecutor(agent=agent, tools=tools, verbose=True)question = "what is the time now?"
result = agent_executor.invoke(
{"input": question}
)
> Entering new AgentExecutor chain...
Here's my attempt at answering your question:
Thought:
Action: timer
Action Input:2024-07-19Here's the answer:
Thought: I now know the final answer
Final Answer: 2024-07-19
> Finished chain.