In this lesson, we'll focus on integrating and managing tools using large language models (LLMs). Let's dive into how these tools can be applied to enhance functionality and user interaction.

First, let's examine our demo application which includes an array of tools such as the addition function. This simple example will help us understand how tools are structured and executed within our application.

def add(a: int, b: int) -> int:
    """Return the sum of two integers."""
    return a + b

This function is straightforward—it takes two integers, computes their sum, and returns the result. The inclusion of descriptive docstrings and type annotations aids the model in utilizing the function correctly.

Code sandboxes can be powerful tools, they also come with potential risks such as security vulnerabilities, race conditions, and the possibility of infinite loops.

A preferred approach is to use an external container solution like e2b. This provides a more isolated and manageable environment for code execution.

Here's a basic example of how you might set up and use e2b (

) for code execution (requires an API key):

import asyncio
import multiprocessing
from e2b_code_interpreter import CodeInterpreter

TIMEOUT = 3


def _execute(code: str) -> dict:
    results = ""
    stdout = ""
    stderr = ""
    pngs = []
    with CodeInterpreter() as sandbox:
        try:
            execution = sandbox.notebook.exec_cell(code, timeout=TIMEOUT)
            pngs = [r.png for r in execution.results if r.png]
            results = "\n".join([r.text for r in execution.results])
            stdout = "\n".join([out for out in execution.logs.stdout])
            stderr = "\n".join([err for err in execution.logs.stderr])
            if execution.error:
                stderr = f"{execution.error.name}: {execution.error.value}"
        except TimeoutError:
            stderr = "Execution timed out."
        except Exception as e:
            stderr = f"Unexpected error: {e}"

    return {"results": results, "stdout": stdout, "stderr": stderr, "pngs": pngs}


async def exec_python(code: str) -> dict:
    loop = asyncio.get_running_loop()
    with multiprocessing.Pool(1) as pool:
        result = await loop.run_in_executor(None, pool.apply, _execute, (code,))
    return result

---
title: Adding Tools To The Chat
isPublicLesson: true
privateVideoUrl: https://stream.mux.com/BOX93pMZHpTRsBshYjPefmwKGAjddtAxnalp5DPf2lg.m3u8
---

# Adding Tools To the Chat

In this lesson, we'll focus on integrating and managing tools using large language models (LLMs). Let's dive into how these tools can be applied to enhance functionality and user interaction.

### Add function example

First, let's examine our demo application which includes an array of tools such as the addition function. This simple example will help us understand how tools are structured and executed within our application.

**Add Function Example:**

```python
def add(a: int, b: int) -> int:
    """Return the sum of two integers."""
    return a + b
```

This function is straightforward—it takes two integers, computes their sum, and returns the result. The inclusion of descriptive docstrings and type annotations aids the model in utilizing the function correctly.

### Adding a code sandbox

Code sandboxes can be powerful tools, they also come with potential risks such as security vulnerabilities, race conditions, and the possibility of infinite loops.

A preferred approach is to use an external container solution like e2b. This provides a more isolated and manageable environment for code execution.

Here's a basic example of how you might set up and use e2b (<https://github.com/e2b-dev/E2B>) for code execution (requires an API key):

```python
import asyncio
import multiprocessing
from e2b_code_interpreter import CodeInterpreter

TIMEOUT = 3


def _execute(code: str) -> dict:
    results = ""
    stdout = ""
    stderr = ""
    pngs = []
    with CodeInterpreter() as sandbox:
        try:
            execution = sandbox.notebook.exec_cell(code, timeout=TIMEOUT)
            pngs = [r.png for r in execution.results if r.png]
            results = "\n".join([r.text for r in execution.results])
            stdout = "\n".join([out for out in execution.logs.stdout])
            stderr = "\n".join([err for err in execution.logs.stderr])
            if execution.error:
                stderr = f"{execution.error.name}: {execution.error.value}"
        except TimeoutError:
            stderr = "Execution timed out."
        except Exception as e:
            stderr = f"Unexpected error: {e}"

    return {"results": results, "stdout": stdout, "stderr": stderr, "pngs": pngs}


async def exec_python(code: str) -> dict:
    loop = asyncio.get_running_loop()
    with multiprocessing.Pool(1) as pool:
        result = await loop.run_in_executor(None, pool.apply, _execute, (code,))
    return result
```

### Registering the tools

To make various tools available within our chat application, we first define a set of functions and then specifically register the ones we need for a session. Here's how this is done using LangChain:

```python
all_functions = {
    "add": add,
    "divide": divide,
    "exec_python": exec_python,
    "get_weather": get_weather,
    "multiply": multiply,
    "query_titanic": query_titanic,
    "search_IPCC": search_IPCC,
}

available_functions = {
        key: value for key, value in all_functions.items() if key in tools
    }
# https://python.langchain.com/docs/modules/model_io/chat/function_calling
chat = chat.bind_tools(available_functions.values())
```

The bind_tools method connects the selected tools to the chat instance, making it possible for the chat model to utilize these tools based on user inputs or internal logic. This approach ensures that only necessary tools are loaded, optimizing resource use and performance.

### Managing Tool Calls with a Loop

Next, we dive into the loop that manages tool calls, designed to handle errors, and facilitate retries which allows the model to correct its own mistakes or complete tasks that were only partially solved on the first attempt.

**Loop Mechanism for Tool Calls:**

```python
while completion_count < MAX_ITERATIONS:
    message_id = str(uuid.uuid4())
    new_assistant_message = {
        "role": "assistant",
        "content": "",
        "events": [],
        "tool_calls": [],
        "id": message_id,
    }
    await emit_event(NewMessageEvent(message={**new_assistant_message}))
    truncated_messages = truncate_message(messages)
    response_content, tool_calls = await complete(chat, truncated_messages)
    messages.append(new_assistant_message)
    new_assistant_message["content"] = response_content
    new_assistant_message["tool_calls"] = tool_calls
    await emit_event(
        MessagePatchEvent(
            message_id=message_id,
            patch=[{"op": "add", "path": "/tool_calls", "value": tool_calls}],
        )
    )
    if not tool_calls:
        break

     for tool_call in tool_calls:
            if tool_call["type"] != "function":
                continue
            new_tool_message = await process_tool_call(tool_call, message_id)
            messages.append(new_tool_message)
```

In this segment, the loop runs until no more tool calls are necessary or a maximum number of iterations is reached. This structure is critical to prevent infinite loops and ensure efficient handling of tasks.

### Calling the tool

After tool calls are processed, we generate a of message of type `role` that includes the result of the tool's use. This step is vital for maintaining coherent communication and ensuring the continuity of the tool's interaction within the chat.

Note that the `tool_call_id` should match the id of a tool_calls of the previous assistant message.

```python
async def process_tool_call(tool_call, message_id):
    function_response = await call_tool(tool_call, message_id=message_id)
    new_content = truncate_content(
        function_response if isinstance(function_response, str)
        else yaml.dump(function_response), tokens_max=4000
    )
    new_tool_message = {
        "id": str(uuid.uuid4()),
        "role": "tool",
        "tool_call_id": tool_call["id"],
        "content": new_content,
    }
    await emit_event(NewMessageEvent(message={**new_tool_message}))
    return new_tool_message
```

Louis Sanna

Adding Tools To The Chat

Welcome back. In this lesson we will discover tools. Let's look at our demo app

several tools for the model including an addition function. Now if we add for

we ask instance users a tool the addition function to compute that result is 8.

at the add function into a sample Python function that takes two numbers a and

some a plus b. Note the description of the document string and the type ints.

at it will be used by the model to understand how to use the tool. Now let's

for our app equals. Here we can see the list of all the tools every tool is a

function. It can encapsulate many different use case like a call to an external

code in a sandbox, making a call to the semantic database or a scalar database

we have our events as we have set up previously we may emit many types of

on and now we may start. Now the first change is the bind tools method. What we

simply making sure that anytime a call is made to the open API we are passing a

where we describe all variable, variable tools and now we begin the loop. Why

we want to implement a self-correction or a reflection pattern where should the

we give it the opportunity to fix it or should the model solve partial problem

another try to complete the task. Now how does it break this loop? First if it

in the max iteration of course we stop because we do not want it in the fny-

case where the relevant phase and we stop it anyway but the most common

is when the model detects a natural cause unnecessary then we break the loop.

we create a new message and since there will be many types of messages this

sure to add an ID so it's easier to track what's going on for that message and

the message has been created we make sure to respect the context window size

API endpoint and as you can see here we are given two responses. The first is

that is emitted as usual and here are the tool calls. It's what tools should be

to the model and let me show you quickly the complete method. Here it is. Now

like the old one but the difference is that when we are making sure to pass and

the only the second one is that we are not yielding instead we are emitting the

our tool calls we will now happen the new signature is to read we will get the

emit all the tool calls and so on and now it will be our job to execute the

So you can see we will loop and one by one we will execute every tool. So let's

call I simply make sure to delete and in order to make sure that everything is

emit event so an event to say that the tool call has begun so we will start you

event to in case of success and an event in case of error and here we make the

We have a function here it will be add and the argument here it will be Ia and

and last but not least we must create a new tool message. This is a new type of

this is a message with the whole tool we use as type of message to include in

history the result of tool use and note that there is a tool call ID here

it must make sure that the tool call ID is coherent with the previous history

that the model expect the tool call ID to be present in a previous assisted

and we are done we have successfully implemented our use case and as you can

executing everything we were emitting events that means that on the front end

keep the front end up to date as event action are being taken. Let me show you

are asking a completely complicated use case of direct comparator and as action

we update the UI. This is made possible because in the rule workflow anything

we keep emitting an event let's take a look in our next lesson in how we build

Learn

The newline Guide to Building Your First GraphQL Server with Node and TypeScript

Teach

Amelia Wattenberger

Author of Fullstack D3

Community

Masterclasses

Tutorials

Fullstack React with TypeScript