VectorDB 2023

This article summarizes the product developments of various VectorDB providers as of December 2023.

Pinecone

The leading VectorDB provider is Pinecone (pinecone.io). In March 2022, it raised $28M in Series A, followed by a $100M Series B in April 2023. While its source code is not publicly available, Pinecone has become synonymous with managed VectorDB solutions. Internally, it operates on a multi-cloud infrastructure using Kubernetes, with index replication across different pods.

Pinecone's API is intuitive and user-friendly. The namespace feature allows partitioning of records, enabling region- or language-based index management. Each record can be assigned metadata, allowing for flexible queries based on attributes or machine types. Metadata supports arrays, making it possible to implement requirements such as multi-genre book searches.

Pinecone actively shares unique features and implementations for performance improvements. For example, Single-Stage Filtering overcomes the computational overhead and query result loss associated with pre- and post-filtering. Additionally, Sparse-dense Vectors enable a combination of traditional keyword searches with semantic search, enhancing e-commerce searches by integrating user queries with product images.

For production use, Pinecone supports Prometheus monitoring (docs) and integrates with Datadog (docs). However, current metrics are limited to vector count, request volume, error rate, and latency, with internal index operations remaining a black box.

Cloudflare Vectorize

Cloudflare Vectorize, introduced as a Public Beta in September 2023, is still in its early stages.

The API seems inspired by Pinecone, supporting namespace partitioning and record-level metadata filtering. It offers standard Vector Similarity algorithms: Cosine, Euclidean, and Dot-product.

While still in beta, the platform allows up to 100 indexes, supports 1536 dimensions, and accommodates 200K vectors per index—sufficient for testing and small-scale projects. Recently, Metadata Filtering was introduced.

Cloudflare Workers AI integration makes it a viable option for mid-scale LLM applications. However, Pinecone and other VectorDBs can also be accessed from Cloudflare Workers, raising questions about differentiation. While its edge computing capabilities may reduce network latency, vector computation and UPSERT performance are likely to be bigger bottlenecks.

Chroma

Chroma (trychroma.com) is an OSS alternative to Pinecone, branding itself as an "Embedding database." It secured an $18M Seed round in April 2023.

Currently, Chroma does not offer a managed service, though a waiting list suggests its launch is imminent. Job postings for Distributed Systems and Cloud Infrastructure roles further indicate an internal priority on this feature (careers).

Chroma organizes vector data using collections and supports Cosine, Euclidean, and Dot-product similarity calculations (source). It enables metadata-based filtering with logical operators like AND/OR and supports multi-modal collections.

While still lagging behind Pinecone in production readiness, Chroma's OSS approach could help it become a strong competitor once managed services and performance optimizations are implemented.

Qdrant

Qdrant (qdrant.tech), like Chroma, is an open-source VectorDB. However, Qdrant is implemented in Rust. In April 2023, it raised $7.5M in a Seed round, similar to Chroma. While Pinecone has taken the lead in fundraising and product outreach, Qdrant needs to differentiate itself through expectations of high performance on a single node due to its Rust implementation and its variety of integration options. It already offers Qdrant Cloud, which includes a free tier.

That said, Qdrant has yet to demonstrate differentiation factors comparable to Pinecone’s leadership, Cloudflare Vectorize’s potential, or Chroma’s strong community. My lack of research or understanding might contribute to this perception, but using Rust doesn’t necessarily guarantee better performance for all workloads. The published benchmarks can be interpreted in various ways, and Pinecone is notably absent from the comparisons.

For instance, the homepage claims implementation of Hierarchical Navigable Small World (HNSW), an optimization technique for graph searches in Small-world networks. However, the widely used C++ implementation, nmslib/hnswlib, already provides Python bindings. Chroma also uses a forked implementation, so merely implementing HNSW doesn’t create a significant competitive advantage.

Rather than just focusing on single-node performance, the key lies in enabling users to fine-tune performance flexibly based on their diverse business requirements. This includes providing robust observability, enhancing development environments, or offering high scalability and availability in a managed service. Harsh as it may sound, Qdrant needs more compelling selling points beyond just being implemented in Rust.

Weaviate

Weaviate (weaviate.io) is another open-source VectorDB that has raised significantly more funding than Chroma or Qdrant. It secured $16M in a Series A round in February 2022 and $50M in a Series B in April 2023, though this is only half of Pinecone’s Series B funding. It is implemented in Go and already has a managed service available in the market. Security measures such as SOC 2 certification and OpenID Connect (OIDC) support indicate a clear focus on enterprise adoption.

Like Pinecone, Weaviate supports Hybrid search, which combines traditional keyword search with vector search. Another intriguing feature is its Multi-tenancy API. While similar functionalities can be implemented in other VectorDBs using metadata or namespaces, Weaviate’s native implementation demonstrates a strong commitment to the feature. Additionally, it supports Cross-references, allowing relationships between collections. These references can be configured as one-way, bi-directional, or one-to-many.

Weaviate also appears well-prepared for production deployment. It offers a Backup feature that claims zero downtime, enabling storage in Blob storage. My favorite documentation section is on Horizontal Scaling, which provides detailed explanations of sharding and replication strategies. This reassures me that if I were assigned to manage a Weaviate cluster in production tomorrow, I could do so without much frustration. For inter-cluster coordination, it employs HashiCorp’s Gossip-based memberlist.

Like Qdrant, Weaviate’s vector index implementation is based on HNSW. For keyword search, it utilizes the widely adopted LSM-Tree for inverted indexes. The documentation suggests that someone with deep database development experience was part of the initial team. One particularly interesting choice is its Leaderless replication, similar to AWS DynamoDB. Unlike Single-leader or Multi-leader architectures, leaderless replication can enhance write performance through eventual consistency.

Additional Considerations

While we’ve covered several VectorDBs, existing RDBMS and GraphDBs are also rapidly implementing vector search capabilities. Naturally, they don’t want to lose customers to dedicated VectorDBs. PostgreSQL has the pgvector open-source extension. Neo4j has introduced a vector search index. MongoDB has launched Atlas Vector Search.

However, I remain skeptical about their performance. Each database is optimized from disk format to memory layout and protocol layers to focus on its strengths. Even if the demand for VectorDBs increases, RDBMS, Key-Value stores, and GraphDBs might struggle to match the performance of specialized Vector Search engines. Additionally, maintainability is a concern.

Market pressure is undeniable, and for some business cases, moderate search performance from existing databases may suffice. However, in scenarios where extreme performance is required, or traffic volume is high, using a purpose-built VectorDB will likely remain the best option. Ultimately, this depends on the quality of each product.

Which VectorDB Should You Use?

This is a subjective opinion based on my research, and I don’t want to make prescriptive recommendations. However, if you’re struggling to decide, consider the following points (again, these are my personal preferences):

• My personal favorite is Weaviate. Its developer documentation excites me the most.
• If you’re introducing VectorDB in an enterprise environment, Pinecone should be on your shortlist.
• If supporting startups or side projects, I’d recommend Weaviate.
• Since I use Cloudflare, I’ll continue monitoring Cloudflare Vectorize.
• I cloned Qdrant’s repository to study Rust, but I don’t plan to use it in production.
• Chroma’s viability depends on its cloud service quality.

There are other VectorDB options, but keeping up with these should suffice for development. If needed, I might also investigate Vald and Milvus.

Conclusion

As of December 2023, this is my summary of various VectorDB products.

Is VectorDB just a "trend"? In my view, it’s similar to when GraphDB or NoSQL emerged. Changes in application development and business requirements have made traditional database engines insufficient for the performance demands of newer applications (e.g., LLM applications requiring query responses within 10ms/p99). Naturally, this has led to the development of specialized database engines.

If you’re not particularly passionate about databases, consider VectorDB as just another tool in your arsenal rather than a must-have technology.