March 26, 2024 | By Scott Gnau, Vice President of Data Platforms
Artificial intelligence has transformative potential for driving value and insights from data. As we progress toward a world where nearly every application will be AI-driven, developers building those applications will need the right tools for creating experiences from these applications. This is why we're excited to announce the addition of vector search to the InterSystems IRIS data platform.
Tools like vector search are essential for enabling efficient and accurate retrieval of relevant information from massive datasets when working with large language models. By converting text and images into high-dimensional vectors, these techniques allow quick comparisons and searches, even when dealing with millions of files from disparate datasets across the organization.
InterSystems IRIS data platform delivers a unified foundation for next-generation applications
At InterSystems, we're always looking for ways to bring next generation data processing as close to our customers’ data as possible without having to transfer data to specialized systems.
By adding vector search to the InterSystems IRIS data platform, we are making the data platform searchable through vector embeddings in order to enhance the functionality of the software for tasks related to natural language processing (NLP), text, and image analysis.
This integration will make it easier for developers to create applications that use generative AI to complete complex tasks for a wide range of use cases and deliver up-to-date responses based on proprietary data processed by InterSystems. It also means they can do this with very curated data while being confident in keeping their internal proprietary intelligence secure.
This capability allows the InterSystems IRIS data platform to manage and query content and related dense vector embeddings, particularly as it enables Retrieval-Augmented Generation (RAG) integration to develop generative AI-based applications. With the rapidly evolving toolsets available, seamless RAG integration allows agile adoption for new models and use cases.
What benefits does vector search technology bring to customers?
BioStrand, an AI-powered drug discovery company, is part of the InterSystems Innovation Program that helps start-ups build applications on InterSystems IRIS. At the core of BioStrand’s offerings is the Lensai platform, a versatile solution that supports various applications including antibody drug discovery and design.
By leveraging advanced algorithms, Lensai can swiftly identify and craft novel drug compounds, significantly reducing R&D timelines from development to commercialization. This model uniquely combines the strengths of Large Language Models (LLMs) using an advanced stacking technique with BioStrand’s patented HYFT Technology.
HYFTs are a type of embedding that serve as unique 'fingerprints' in biological sequences, enabling BioStrand to assign embeddings from different LLMs with high precision. This foundation model represents a vast and continuously expanding knowledge graph, mapping an impressive 25 billion relationships across 660 million data objects.
This comprehensive graph interconnects sequences, structures, and functions from the entire biosphere, together with bibliographic information. It also incorporates cutting-edge technologies like RAG, SQL vector search and generative capabilities of LLMs together with the semantic expressiveness of knowledge graphs.
Vector search will fundamentally change how developers interact with InterSystems IRIS
We’re just scratching the surface when it comes to implementing this technology. We’ll be sharing more customer stories as vector search changes how they interact with their data and what new AI applications are developed with vector search. In the meantime, I encourage you to visit our vector search landing page to learn more.
Our commitment to maintaining the highest standards of privacy, security, and responsibility will guide a thoughtful and just approach to AI that creates trust while accelerating innovation, ensuring customer success, and demonstrating a commitment to excellence. We believe transparency, responsibility and explainability are key to establishing trust in and driving innovation from AI systems.