Overview of the AI landscape
This is an overview of the AI landscape. Although I'm not an expert, I have built enough with AI technologies to start understanding the different terms and concepts.
1. LLMs (Large Language Models)
LLMs serve as a foundational API in general AI applications. They are standardizing around the OpenAI interface.
There are two main players in this space: OpenAI and Anthropic. Their respective consumer products are the well-known ChatGPT by OpenAI and Claude by Anthropic.
Alongside these, there are open-source LLMs, notably LLaMA by Meta (formerly Facebook) and Mistral. To utilize these, self-hosting is required, with popular options being Anyscale or AWS SageMaker.
2. Vector Databases and Embeddings
The way I understood vectors/embeddings is that every word, rather than being just a single term, is associated with related words (or dimensions). The Embedding Projector by TensorFlow is a useful tool for understanding this concept.
For example, in a text containing "Michael Jordan," a traditional database query for "Chicago Bulls" would yield no results. However, a Vector database, understanding the association, might return the "Michael Jordan" text.
Unlike SQL queries, which are based on exact matches, Vector database queries assess the proximity or relevance to the query term.
Vector databases allow you to query vectors at scale in a fast manner. A popular choice is Pinecone.
Regarding embeddings/vectors, or how we can relate "Michael Jordan" to the "Chicago Bulls," this is done by embeddings APIs. Two popular choices here are OpenAI and Cohere.
A common technique to connect LLMs with a Vector database is to first query the Vector database and then use the returned results to provide context to the LLMs. This is called Retrieval Augmented Generation (RAG)
3. Tooling
This is evolving rapidly, but I'd like to mention two companies.
Vellum enables the versioning and deployment of various prompts associated with different models.
Not Diamond specializes in routing your prompt to the most suitable model for the best possible answer.
I hope this overview has been helpful. Feel free to add any comments or additional insights!
Software Engineering from the Frontlines Course on Maven
If you liked this article, I will be teaching a “Software Engineering from the Frontlines” course on Maven where I will teach hard-learned lessons I acquired developing large-scale products at companies such as Uber, Airbnb, and Microsoft.