Overview
The main idea was to build an AI agent named Lucy that can learn from our conversations and interact through a Slack interface. The plan was also to use no code tool (make.com) and play with embedding and pinecone.io. So the whole point was to build something that takes this:
And based on automatically creates table like this (I’m using airtable.com here).
Next I’m able to use those memories to create vectors in pinecone database and use them later in conversations.
Slack API
I won’t dive into the nitty-gritty details here, but the basic idea is simple. Create bot app and enable events api. We will use it to publish events on new messages in certain channel. Also I have turned on direct messages to the app. Then create make.com connection and voila.
Scenario - Lucy Slack conversation
Let’s break it down step by step.
- Custom Webhook - Start with a simple webhook to listen for events. Make sure to register this URL in your Slack app settings.
- Webhook response - there is filter on route checking that
event.type = url_verification
. This is to confirm and send back challange to slack. - Next on router we check for
event.type = message
. Obviously we want to process only those events here. - Data store (Get a record) - fetching user slack id here. For later use.
- Router - a few things here. Lets go one by one.
- Webhook response - first send back webhook response. This to let slack know that we succeed in processing the event and to prevent resending this to us. I know that we don’t finish processing yet but that’s the mistake I can live with. I would rather prefer Lucy to not respond due to some error then responding multiple times and creating noise.
- If the message if from the user - I’m just checking the author here. If its me → proceed. If the message comes from Lucy → Ignore.
- First fetch not synced conversation history. Aggregate it in format like:
- Then embed query sent to slack calling
v1/embeddings
endpoint with Make an API call. Next based on the resulting vector I’m querying my pinecone database for similar vectors (Assuming it will return vectors with informations that might be related to my query). - More on those vectors and their metadata later but here I’m just taking vector
metadata.id
and based on that fetchingmemories
database by ids. - GPT Completion - here is the completion itself. The context at this place is build on two things. First - not synced conversation history that might contain useful facts. Second - useful memories that were found in pinecone.
- Create slack message
- In the end, no matter who created message, I’m saving it with the author to
ConversationHistory
database and marksynced=false
.
kuba szwajka (2024-05-22T22:57:16.000Z): Hi! Whats my name?
Lucy AI assistant (2024-05-22T22:57:16.000Z): Hey there! I don't have your name yet. What's your name? :smile:
kuba szwajka (2024-05-22T22:57:26.000Z): my name is Kuba. Hi!
Lucy AI assistant (2024-05-22T22:57:28.000Z): Hey Kuba! How's it going? :smile:
Scenario - Lucy find memories in conversation
This scenario aims to process previously created conversation history, create memories and save them as a vectors. Right now it is triggered every 8 hours. Tbh. I’m testing if this is enough 🤷.
Nothing fancy here. Lets go step by step:
- Fetch all records from
ConversationHistory
wheresynced=false
. - Aggregate them as a text.
- Let Chat distill the facts (facts overlapping to be implemented? Not sure if this will be the problem 🤔? Any thoughts? )
- For the list of facts/memories, create a records in
memories
table withsynced=false
. - For each of these, create embedding calling
v1/embeddings
in OpenAI api. - Save it to pinecone and mark as synced.
Action
- There are still a few missing pieces, like handling memory overlapping and preventing data loss during the conversation history to memory conversion.
- I’ve put some blueprints if you want to try it out.
Want to Know More?
Stay tuned for more insights and tutorials! Visit My Blog 🤖