Integrating Langchain with FastAPI for Asynchronous Streaming

[00:00 - 00:10] Welcome back. In this lesson, we will learn how to use LongShane to build our fast API endpoint. LongShane is a popular Python library used to build an application.

[00:11 - 00:20] It provides many classes which serve as abstraction for different building blocks. For instance, LML class, Rotriva class, Vector class, and so on.

[00:21 - 00:32] It makes switching from one provider to another very easy as you simply need to replace one subclass by another. Let's imagine you are currently using pinecone as a vector store.

[00:33 - 00:40] Then you want to switch to current. You just need to replace the pinecone subclass by the current subclass and your done.

[00:41 - 00:48] So that can be a huge time saver and many people like using LongShane. That being said, there are some inconvenience.

[00:49 - 00:56] It's a very fast-strunging library. The API is very, is not stable and with documentation of how outdated.

[00:57 - 01:06] You will need to look up the source code to understand what's happening. Secondly, the code is sometimes a bit of a early abstract to avoid to be tied to any specific implementation.

[01:07 - 01:16] So to conclude, feel free to use the longShane if it's useful for you. But you can also do everything yourself with your own implementation.

[01:17 - 01:25] And now let's see the code where you are using LongShane and fast API together. So here we are declaring our first API route.

[01:26 - 01:36] We are building with PyDontic some class which will be used to validate the queries and auto-generate the documentation. Now we can see a mess and then we build a list of message.

[01:37 - 01:43] This will be our payload for this endpoint. And now we fast API.

[01:44 - 01:50] We build a stream endpoint. And as you can see, it's possible thanks to the streaming response, which is a fast API response.

[01:51 - 01:58] And we simply need to specify media tab as text, even stream. As some of the events I expected to be encoded as text.

[01:59 - 02:13] And we're done with build a streaming endpoint. But now we need to connect it to the open API and make sure every event is being logged and sent via the endpoint.

[02:14 - 02:23] How do we do that? We create with LongShane class a chat distance and we pass it to the Sun Com pletion Event function.

[02:24 - 02:29] Let's look at the code from the Sun Heartworks. First we must notice that it is a next-time function.

[02:30 - 02:41] As we are seeing in the previous lesson, we want everything to be as a otherwise our API will not be performed. Then we are going to use a very powerful function method which is a stream log.

[02:42 - 02:53] It's a method which is going to stream every event that happens during an execution. So here we got a stream log and then we will loop over each event with a sync fer.

[02:54 - 02:59] And every time we get an event, the event will be a patch. Now what is a patch?

[03:00 - 03:08] Patch is a format describing the changes of the logism document. So let's show an example so it's a bit clearer.

[03:09 - 03:15] Now you can see an example document. See the g-zone you can see that for the key bars the value is clicked.

[03:16 - 03:23] Okay, so let's apply your patch. First patch you have up for operation which is replace.

[03:24 - 03:29] Path which is bars. So we are modifying the value for this key and the new value is blue.

[03:30 - 03:36] So we apply for patch and we get for key bars the new value is blue. So that's the patch.

[03:37 - 03:41] So here we get our patch. We find the ups for operations.

[03:42 - 03:49] We get we look up for the operation add and we find the bar stream output. This is what we want.

[03:50 - 04:03] We want the stream text. Then we get the content and as we previously explained we now can build our 7th event with the data and a stringy file g-zone.

[04:04 - 04:13] And then we simply need to yield as the same commission event is in the same code routine. So it can yield when it wants.

[04:14 - 04:27] And that's all. So in less than 10 lines of code we have completely done our back end which was to stream all the event coming from the eye into string for an endpoint.

[04:28 - 04:32] So that's whenever back end and in our next session we will learn about testing .