Introduction
For many of us web-developers, the recent LLM craze is something that we have been often hearing about, but not really using. We know that LLMs are powerful, but we don't really know how to use them in our applications. One of the main reasons is the mismatch between the technologies that lie at the core of LLMs and those used to build web applications. LLMs are typically written in Python, and so are the libraries that are used to interact with LLMs. Many of us web developers are not familiar with Python or not sure how to integrate Python into our development stacks. And even when we do, we often find that the idea of designing the entire application in Python just to use LLMs is not very appealing.
This challenge will disappear soon once we release release Tanuki's TypeScript library. But until then, this tutorial provides the next best thing.
If you are finding yourself emphatically nodding along to the above, then this article is for you. Mainly, because I am going to talk about how to unlock the world of LLMs for web developers without having to completely abandon your current development stack. We are going to build a simple web application that uses an LLM to evaluate the truthfulness of a statement. The application will be built using React, leveraging Vercel's support for serverless functions, and the LLM integration will be powered by Tanuki.
Why a truth evaluator you ask? Well, because it's a simple example of something that would be very difficult to do without an LLM. But it's also something that is very easy to understand and modify to your needs.
In this tutorial, the entire end-to-end application will use the Vercel ecosystem. This is not a requirement, but it is a great way to get started. Essentially however, you can use any web framework, and only implement the LLM-powered function using Tanuki as a Vercel serverless function.
Vercel in a nutshell
Vercel is a cloud platform that provides a range of services for building, deploying, and scaling web applications. It is particularly known for its focus on serverless functions, a key feature for this tutorial. Vercel supports a wide range of languages, but is most commonly used with next.js which is a React framework for building web applications.
The serverless functions that Vercel provides are a great way to integrate LLMs into your web application, because they allow developers to write small pieces of code (in multiple languages) that can be invoked from the frontend without having to spin-up full servers. As such, this feature allows us to write the core backend of our app in any language we want and only use Python for the LLMs.
Tanuki in a nutshell
Tanuki allows developers to create and call LLM-powered functions that get faster and cheaper the more they are used. Since Tanuki requires very minimal code to get started, even developers with ZERO knowledge of Python can create and call LLM-powered functions quickly and easily.
The application
To help you follow along, here's the GitHub repository with the full code for this application.
Pre-requisites
In order for you to follow along with this tutorial, you will need to:
- Create a free Vercel account
- Install the Vercel CLI on your machine
- Install Node.js on your machine
- Install Python on your machine
- Get an OpenAI API key
Step 1: Create a new Next.js project
First, we need to create a new Next.js project. To do that, open up your terminal and run the following command:
npx create-next-app@latest
Once you run this command you'll have an opportunity to choose a project name. This will create a new folder with that name. Then, you'll be prompted to selected between a few options for your boilerplate project. For this tutorial, we'll be using the default options.
(NOTE: If you modify any of the defaults, make sure you update your code accordingly).
Step 2: Enter the folder you just created
cd <project-name>
At this point you should have a fully-working boilerplate project. To verify that it is working, you can run it by running the following command:
vercel dev
Note: You may need to login into your Vercel account before you can run this command. To do that simply run the following command:
vercel login
You should be able to see your boilerplate project running on http://localhost:3000.
If everything looks like it's working fine, let's move on to the next step.
Step 3: Create dependency list
This project is using axios to make HTTP requests to the backend. To install it, run the following command:
npm install axios --save
This will install the axios library and add it to the list of dependencies in your package.json file.
This project is also using four python libraries. Vercel will install them automatically for you. But you need to add them to the list of dependencies in a file called requirements.txt. To do that, create a new file called requirements.txt in the root of your project. Go ahead and create that file now with the following contents:
requirements.txt:
pydantic==2.4.2
openai~=0.28.1
python-dotenv~=1.0.0
monkey-patch.py
Before moving on to the next step, let's look at what we just installed:
- pydantic - a wonderful library used for data validation, but we'll be leveraging it to define the input and output types of our LLM-powered function. More on this later.
- openai - This allows us to use the OpenAI API to call the LLM.
- python-dotenv - This allows us to load environment variables from a .env file.
- monkey-patch.py - This is the library that we will use to create and call our LLM-powered function.
Step 4: Create a .env.local file
NOTE: This file is only needed for local development. If you want to use the app in the "Real World", you'd have to set your environment variables (from .env.local) in the Vercel dashboard (settings->Environment Variables ).
.env.local:
OPENAI_API_KEY=YOUR_API_KEY_GOES_HERE
Step 5: THE FUN PART - Create the LLM-powered function using Tanuki
In your root directory create a folder called api. Then, inside that folder create a file called index.py.
This is the file that will be called when users make a request to the /api route of your application (see here for a deeper dive if you're curious).
The first thing we will do in this file is define the expected output type of our function. Note, this can be virtually ANYTHING. Tanuki will figure out how to use it.
Since we are building a function that will evaluate the truthfulness of a statement, we will define the output type as such:
from pydantic import Field, BaseModel
from typing import Literal
class TruthEvaluation(BaseModel):
is_true: Literal['yes', 'no', 'maybe'] = Field(..., description="Is this statement true?")
confidence: Literal['high', 'medium', 'low'] = Field(..., description="How confident are you in your answer?")
explanation: str = Field(str, description="Why do you think this is true?")
Next, let's define our Tanuki function that connects our application with the LLM.
from monkey_patch.monkey import Monkey
import openai
openai.api_key = os.getenv("OPENAI_API_KEY") # Your OpenAI API key
@Monkey.patch
def truth_evaluator(statement: str) -> TruthEvaluation:
"""
based on the provided statement, evaluate the following aspects:
- is the statement true?
- how confident are you in your answer?
- what was the reasoning that lead to that conclusion?
"""
I know what you're thinking. That's a docstring. Ummm... Where's the actual implementation?
Well, that's the beauty of Tanuki. You don't need to write any implementation. Tanuki will take care of that for you. All you need to do is write a docstring that describes what the function does. Tanuki will do the rest.
Next, optionally, we can add a few assertions to help Tanuki understand what we mean by the docstring. This is not required, but it can help Tanuki generate better responses. For example, we can add the following assertions:
@Monkey.align
def test_truth_evaluator():
"""We can test the function as normal using Pytest or Unittest"""
assert truth_evaluator("The sky is blue") == TruthEvaluation(
is_true='yes',
confidence='high',
explanation='There have been many observations of the color of the sky, and it is generally accepted that the sky is blue.'
)
assert truth_evaluator("The sky is green") == TruthEvaluation(
is_true='no',
confidence='high',
explanation='There have been many observations of the color of the sky, and it is generally accepted that the sky is blue, and NOT green.'
)
assert truth_evaluator("Wednesday is the best day of the week") == TruthEvaluation(
is_true='maybe',
confidence='medium',
explanation='There is no consensus on the best day of the week, and it is likely that this is a subjective opinion.'
)
assert truth_evaluator("3 is both bigger than 2, and the angriest number") == TruthEvaluation(
is_true='maybe',
confidence='low',
explanation='While 3 IS bigger than 2, a number cannot be angry, and therefore this statement is nonsensical.'
)
assert truth_evaluator("Machines cannot be conscious") == TruthEvaluation(
is_true='maybe',
confidence='medium',
explanation='We do not have a clear definition of consciousness, and therefore cannot determine whether machines can be conscious.'
)
This was inspired by the concept of Test-Driven Development where you define the tests your functions need to pass before implementing them. However, with Tanuki tests never fail, so these statements aren't tests in the traditional sense of the word, but rather they help align the model towards your desired responses. Therefore, instead of Test-Driven Development, Monkey Patch refers to this as Test-Driven Allignment.
Finally, we need to add the API code snippet that intercepts the request, calls Tanuki, and returns a response. This is the code that will be called when users make a request to the /api route of your application.
from http.server import BaseHTTPRequestHandler
class handler(BaseHTTPRequestHandler):
# creates a dictionary of parameters based on url query params passed
def get_url_params(url):
params = {}
if '?' in url:
params = (url.split('?')[1]).split('&')
params = {param.split('=')[0]: param.split('=')[1] for param in params}
return params
# Gets a request from the client containing a statement and returns a response containing the evaluation
def do_GET(self):
# key-value pair of the query params
params = self.get_url_params(self.path)
# If no statement is provided, return an error
if (not 'statement' in params):
self.send_response(400)
self.send_header('Content-type', 'application/json')
payload = {
'error': 'No statement provided',
'status': 400,
}
self.wfile.write(json.dumps(payload).encode())
return
# We define our success response headers
self.send_response(200)
self.send_header('Content-type', 'application/json')
self.end_headers()
# Optionally, run the tests to align the function
# test_truth_evaluator()
# Call the function and get the response
response = truth_evaluator(params['statement'])
# Return the response
self.wfile.write(response.model_dump_json().encode())
return
Here's the full code for the api/index.py file:
from http.server import BaseHTTPRequestHandler
from pydantic import Field, BaseModel
from typing import Literal
import openai
openai.api_key = os.getenv("OPENAI_API_KEY") # Your OpenAI API key
from monkey_patch import Monkey
class TruthEvaluation(BaseModel):
is_true: Literal['yes', 'no', 'maybe'] = Field(..., description="Is this statement true?")
confidence: Literal['high', 'medium', 'low'] = Field(..., description="How confident are you in your answer?")
explanation: str = Field(str, description="Why do you think this is true?")
@Monkey.patch
def truth_evaluator(statement: str) -> TruthEvaluation:
"""
based on the provided statement, evaluate the following aspects:
- is the statement true?
- how confident are you in your answer?
- what was the reasoning that lead to that conclusion?
"""
@Monkey.align
def test_truth_evaluator():
"""We can test the function as normal using Pytest or Unittest"""
assert truth_evaluator("The sky is blue") == TruthEvaluation(
is_true='yes',
confidence='high',
explanation='There have been many observations of the color of the sky, and it is generally accepted that the sky is blue.'
)
assert truth_evaluator("The sky is green") == TruthEvaluation(
is_true='no',
confidence='high',
explanation='There have been many observations of the color of the sky, and it is generally accepted that the sky is blue, and NOT green.'
)
assert truth_evaluator("Wednesday is the best day of the week") == TruthEvaluation(
is_true='maybe',
confidence='medium',
explanation='There is no consensus on the best day of the week, and it is likely that this is a subjective opinion.'
)
assert truth_evaluator("3 is both bigger than 2, and the angriest number") == TruthEvaluation(
is_true='maybe',
confidence='low',
explanation='While 3 IS bigger than 2, a number cannot be angry, and therefore this statement is nonsensical.'
)
assert truth_evaluator("Machines cannot be conscious") == TruthEvaluation(
is_true='maybe',
confidence='medium',
explanation='We do not have a clear definition of consciousness, and therefore cannot determine whether machines can be conscious.'
)
class handler(BaseHTTPRequestHandler):
# creates a dictionary of parameters based on url query params passed
def get_url_params(url):
params = {}
if '?' in url:
params = (url.split('?')[1]).split('&')
params = {param.split('=')[0]: param.split('=')[1] for param in params}
return params
# Gets a request from the client containing a statement and returns a response containing the evaluation
def do_GET(self):
# key-value pair of the query params
params = self.get_url_params(self.path)
# If no statement is provided, return an error
if (not 'statement' in params):
self.send_response(400)
self.send_header('Content-type', 'application/json')
payload = {
'error': 'No statement provided',
'status': 400,
}
self.wfile.write(json.dumps(payload).encode())
return
# We define our success response headers
self.send_response(200)
self.send_header('Content-type', 'application/json')
self.end_headers()
# Optionally, run the tests to align the function
# test_truth_evaluator()
# Call the function and get the response
response = truth_evaluator(params['statement'])
# Return the response
self.wfile.write(response.model_dump_json().encode())
return
As you can see, there was minimal code that we had to write. All we had to do was define the expected output type, write a docstring, and optionally add a few assertions. Tanuki took care of the rest.
Step 6: Create a web page that calls our LLM-powered TruthEvaluator function
You can breathe, we are back in familiar territory. Here we are going to create the basic web interface that will allow users to interact with our LLM-powered function.
First, we need to create a new file called page.tsx in the root folder. This is the file that will be called when users visit the root of your application.
'use client'
import {useState} from "react";
import axios from 'axios';
export default function Home() {
// The statement we will be sending to the backend
const [statement, setStatement] = useState<string>("")
// The response we will be getting from the backend
// Notice how this follows the same structure as the output type we defined in the backend
const [evaluation, setEvaluation] = useState<{
isTrue: 'yes' | 'no' | 'maybe',
confidence: 'high' | 'medium' | 'low',
explanation: string,
} | null>(null)
// Any errors that we get from the backend
const [error, setError] = useState<string | null>(null)
// Whether we are currently fetching the results
const [isFetching, setIsFetching] = useState(false)
// This function will be called when the user clicks the "Is it logically true?" button
const fetchResults = () => {
// Set the state
setIsFetching(true)
setEvaluation(null)
setError(null)
const apiUrl = '/api'
// Make the request
axios.get<null, Number>(`${apiUrl}?statement=${statement}`)
.then(response => {
if ((response as any).data) {
setEvaluation({
isTrue: (response as any).data.is_true,
confidence: (response as any).data.confidence,
explanation: (response as any).data.explanation,
})
} else {
setError("An error occurred")
}
})
.catch(error => {
setError("An error occurred. ")
}).finally(() => {
setIsFetching(false)
});
}
// Render the page
return (
<main className="flex min-h-screen flex-col items-center max-w-[1200px] m-auto mt-8">
{/*The title*/}
<h1 className={'text-lg font-bold'}>
Evaluate the truthfulness of a statement!
</h1>
{/*The textarea and button*/}
<div className={'flex gap-3'}>
<textarea
placeholder={'Type your statement here...'}
className={'p-2 rounded-xl w-full max-w-120'}
value={statement}
onChange={(e) => setStatement(e.target.value)}
/>
<button
disabled={isFetching}
onClick={fetchResults}
className={'border-2 p-2 rounded-xl bg-gray-300 hover:bg-gray-400 active:bg-gray-100'}
>
Is it logically true?
</button>
</div>
{/*The results*/}
<div className={"max-w-[800px] m-auto mt-8"}>
{isFetching ? "I'm thinking..." : null}
{error ? <div className={'text-red-500'}>{error}</div> : null}
{evaluation ? (
<div className={'flex flex-col gap-2'}>
<div className={'flex flex-row gap-2'}>
<div className={'w-24'}>Is it true?</div>
<div className={'flex flex-row gap-2 items-center'}>
<div className={'w-24'}>{evaluation.isTrue}</div>
<div className={'flex flex-col'}>
<div className={'w-24 h-2 bg-gray-400 rounded-full'}>
<div className={'h-2 bg-green-500 rounded-full'}
style={{width: `${evaluation.confidence === 'high' ? '100' : evaluation.confidence === 'medium' ? '50' : '5'}%`}}/>
</div>
</div>
<div>
{evaluation.confidence === 'high' ? 'Very confident' : evaluation.confidence === 'medium' ? 'Somewhat confident' : 'Not confident'}
</div>
</div>
</div>
<div>
Here's why: <br />
{evaluation.explanation}
</div>
</div>
): null}
</div>
</main>
)
}
Optional Note: A globals.css is automatically created when you run npx create-next-app@latest. For the purposes of this demo change the word dark in the file to light so that the background is white instead of black (vercel's default).
Step 7: Deploy the app
Now that we have a fully-working app, let's deploy it to Vercel. To do that, run the following command:
vercel deploy
That's it. You should now have a fully-working app that uses an LLM to evaluate the truthfulness of a statement. As a reminder, you can see the full code for this app here