OpenAI function calling with Elasticsearch

In this blog we dive into OpenAI's function calling capabilities, allowing AI models to interact with external APIs and perform tasks beyond text generation. Through practical examples, we will demonstrate how to implement dynamic function calls, including fetching data from Elasticsearch, enhancing the model's real-time data access, and complex operation handling.

Introduction to OpenAI function calling

OpenAI function calling refers to the capability of AI models to interact with external functions or APIs, allowing them to perform tasks beyond text generation. This feature enables the model to execute code, retrieve information from databases, interact with external services, and more, by calling predefined functions.

The model intelligently recognised which function needs to be called based on the user prompt and calls the function with appropriate arguments. Arguments can also be generated dynamically by model.

Possible use cases of function calling include:

• Data Retrieval: Accessing real-time data from databases or APIs. (e.g., weather information, stock prices)
• Enhanced Interactions: Performing complex operations that require logic and computation (e.g., booking a flight, scheduling a meeting).
• Integration with External Systems: Interacting with external systems and tools (e.g., executing a script, sending an email).

Implementing function calling

In this blog we’re going to create two functions:

1. fetch_from_elasticsearch() - Fetch data from Elasticsearch using natural language query.
2. weather_report() - Fetch a weather report for a particular location.

We'll integrate function calling to dynamically determine which function to call based on the user's query and generate the necessary arguments accordingly.

Prerequisites

Elastic

Create an Elastic Cloud deployment to get all Elastic credentials.

• ES_API_KEY: Create an API key.
• ES_ENDPOINT: Copy endpoint of Elasticsearch.

OpenAI

• OPENAI_API_KEY: Setup an OpenAI account and create a secret key.
• GPT_MODEL: We’re going to use the gpt-4o model but you can check here which model is being supported for function calling.

Open-Meteo API

We will use the Open-Meteo API. Open-Meteo is an open-source weather API and offers free access for non-commercial use. No API key required.

• OPEN_METEO_ENDPOINT: https://api.open-meteo.com/v1/forecast

Sample data

After creating Elastic cloud deployment, let’s add sample flight data on Kibana. Sample data will be stored into the kibana_sample_data_flights index.

Python notebook

We are going to create a quick Python notebook for the entire flow. Install below dependencies and create a Python script/notebook.

Import packages

Accept credentials

Function1: fetch_from_elasticsearch()

def fetch_from_elasticsearch(nl_query):

This function will accept the nl_query parameter as a string in natural language (English) and return a json elasticsearch response as a string. It will execute all queries on the kibana_sample_data_flights index which is holding all flights related data.

It will consist of 3 steps / sub functions.

1. get_index_mapping() - It will return mapping for an Index.
2. get_ref_document() - It will return a sample document for reference.
3. build_query() - Here we going to leverage GPT model (gpt-4o) with few shots prompt to convert user question (text) into Elasticsearch Query DSL

Continuing notebook by adding all functions together.

Get index mapping

Get reference document

Note: You can also cache the index mapping and reference document to avoid frequent queries to Elasticsearch.

Generate Elasticsearch query DSL based on user query

Note: Sometimes, it might be necessary to modify the prompt to get a more accurate response (query DSL) or a consistent report. While we rely on the model's own knowledge to generate queries, the reliability can be increased with few-shot prompting for more complex queries. Few-shot prompting involves providing examples of the types of queries you want it to return, which helps in increasing consistency.

Execute query on Elasticsearch

Text to Elasticsearch query

Let’s call fetch_from_elasticsearch() with some questions / Query.

Query1

Response

Query2

Response

Try some of these queries and see what result you get -

Once you are done with testing you can comment out the print statement from the above code which we added for the debugging purpose.

Function2: weather_report()

def weather_report(latitude, longitude):

This function will accept parameter latitude and longitude as a string. It will call the Open-Meteo API to get reports for specified coordinates.

Add function in the notebook

Test function

Let’s call weather_report() function:

Check for Whitefield, Bangalore

Response

Function calling

In this part we will see how the OpenAI model detects which function needs to be called based on user query and generates the required arguments.

Define functions

Let’s define both functions in an array of objects. We’re going to create a new function run_conversation().

In each object we need to set the properties.

type: function
name: The name of the function to be called
description: A description of what the function does, used by the model to choose when and how to call the function.
parameters: The parameters the function accepts, described as a JSON Schema object.

Check the tools reference to find out more about properties.

Call OpenAI Chat Completion API

Let’s set the above all_functions in the Chat Completion API. Add below snippet in run_conversation()

tools: Set of all functions. tool_choice = "auto": This lets the model decide whether to call functions and, if so, which functions to call. But we can force the model to use one or multiple functions by setting the appropriate value to tool_choice.

• Set tool_choice: "required" to ensure the model always calls one or more functions.
• Use tool_choice: {"type": "function", "function": {"name": "my_function"}} to force the model to call a specific function.
• Set tool_choice: "none" to disable function calling and make the model generate only user-facing messages.

Let’s run the Chat Completion API and see if it is selecting the proper function or not.

Response

If you have noticed, it has detected name='fetch_from_elasticsearch', because we’ve asked flights related query, and Elasticsearch having flight related data. Let’s try some other query.

Response

Function detected name='weather_report()' and argument generated by model arguments='{"latitude":"28.7041","longitude":"77.1025"}'. We have just passed the city name(Delhi) and the model generated proper arguments i.e. latitude and longitude.

Execute selected function

Let’s execute the detected function with generated arguments. In this part we are simply going to run the function which has been determined by model and going to pass the generated argument.

Add below snippet in run_conversation().

Let’s test this part:

Response

It detected the function weather_report() and executed it with proper arguments.

Let’s try with some flight related query where we’re expecting to get data from Elasticsearch.

Response

Extend the conversation

We’re getting all the responses in JSON format. Which is not really human readable. Let’s use the GPT model to convert this response into natural language. We’ll pass the function response to the Chat Completion API for extending the conversation.

Add below snippet in run_conversation().

Let’s test end to end flow. I would recommend commenting out all the print statements unless you want to keep them for debugging purposes.

Q1: Average delay for Bangalore flights

The average delay for Bangalore flights is approximately 48.65 minutes.

Q2: last 10 flight delay to Bangalore, show in table

The above data is coming from Elasticsearch and model converted json response into table.

Q3: How is the climate in Whitefield, Bangalore, and what precautions should I take?

Model called weather_report() function to get information for Whitefield, Bangalore and it added what precautions need to be taken.

Some of the Q/A performed:

Q4: How's the weather in BKC Mumbai?

The current weather in BKC Mumbai is as follows:

• Temperature: 31.09°C
• Humidity: 74.5%
• Wind Speed: 0.61 m/s, coming from the west-northwest (256.5°)
• No rain intensity or accumulation reported at the moment.

Q5: Which day of the week do flights experience the most delays?

Flights experience the most delays on Thursdays, based on the aggregation data that counts the total number of delays by day of the week.

Q6: Provide a table showing the count of flight cancellations by country

Here is a table showing the count of flight cancellations by country:

Parallel function calling

Newer models such as gpt-4o or gpt-3.5-turbo can call multiple functions in one turn. For example if we ask "details of last 10 delayed flights for Bangalore in tabular format and describe the current climate there.", Here we need information from both functions.

Python notebook

Find the complete Python notebook on Elasticsearch labs.

Conclusion

Incorporating function calling into your applications using models like GPT-4 or others can significantly enhance their capability and flexibility. By strategically configuring the tool_choice parameter, you can dictate when and how the model interacts with external functions.

It also adds one layer of intelligence over your response. In the above example I asked to show data in tabular format and it automatically converts json into table format. It also added a country name based on the country code.

So function calling not only streamlines complex workflows but also opens up new possibilities for integrating various data sources and APIs, making your applications smarter and more responsive to user needs.

Related content

Generative AIHow To

January 30, 2025

Testing DeepSeek R1 locally for RAG with Ollama and Kibana

Learn how to run a local instance of DeepSeek and connect to it from within Kibana.

DEJR

By: Dave Erickson and Jakob Reiter

Vector DatabaseGenerative AI

January 9, 2025

Improving e-commerce search with query profiles in Elastic

Query profiles tackle semantic search challenges in e-commerce. This blog demonstrates how to enhance e-commerce search using query profiles in Elastic.

HC

By: Han Xiang Choong

Generative AIVector Database+1

January 1, 2025

When hybrid search truly shines

Demonstrating when hybrid search is better than lexical or semantic search on their own.

GL

By: Gustavo Llermaly

IntegrationsGenerative AI+2

December 26, 2024

Using Ollama and Go for RAG applications

Creating a RAG application with Go using Ollama to leverage local models.

GL

By: Gustavo Llermaly

Vector DatabaseGenerative AI

December 16, 2024

Agentic RAG with Elasticsearch & Langchain

Discussing and implementing an agentic flow for Elastic RAG, where the LLM chooses to call an Elastic KB.

HC

By: Han Xiang Choong