Enhancing Enterprise Search at the World’s Leading Multilateral Development Bank with Global Footprint

Overview

A groundbreaking collaboration between Google, and its trusted partner Aretec, Inc., led to the development of a cutting-edge enterprise search solution for the World’s Leading Multilateral Development Bank – a complex organization operating in 189 countries, to tackle the unique challenges of handling massive datasets with precision and speed. With over 4 million unstructured documents and a need for seamless retrieval from structured data sources like People and Project Search, the team embarked on a journey to redefine how this Global Development Banking Institution accesses information.

This wasn’t just about building another search tool—it was about crafting an intelligent, intuitive, and lightning-fast experience. Think of it as giving the Global Development Banking Institution a “super search engine” on steroids, powered by Google’s Vertex AI and enhanced by the strategic expertise of Aretec. From intent detection to boosting results by country, region, or even year, the project aimed to make every query smarter and every result more relevant.

The collaboration was more than just a tech project—it was a lesson in teamwork and problem-solving. Security constraints? Solved. Infrastructure migration? Done. Recency versus relevancy? Balanced (mostly). The end result? A robust search platform that not only meets but exceeds expectations, all while demonstrating the power of teamwork and innovation in tackling large-scale challenges.

Google Cloud: New Way Now: How the World Bank provides crucial information to global policymakers in seconds

Business Problem

The Global Development Banking Institution operates on a massive scale—think global initiatives, thousands of projects, and millions of documents—all contributing to its mission of reducing poverty and driving sustainable development. In this landscape, information isn’t just a resource; it’s a lifeline. That’s where the enterprise search application became a game-changer.

Before this project, navigating the Global Development Banking Institution’s ocean of data often felt like finding a needle in a haystack. Teams had to manually sift through outdated systems, juggling structured datasets like project records and unstructured ones like annual reports, transcripts, and research documents. The process wasn’t just time-consuming—it was a bottleneck to decision-making.

Solution

The new AI-powered enterprise search solution is a tool designed to act like a well-informed librarian who knows exactly where to find what you need—and does it in seconds.

1. Speed That Delivers Results: The application reduced search times to a sleek 2-6 seconds, empowering teams to retrieve the right information instantly. Whether it’s researchers analyzing policy data or internal teams preparing reports, the application ensured they spent less time searching and more time creating impact.
2. Breaking Data Silos: By blending results from structured (like project details) and unstructured (like 4 million+ documents) data, the tool provided a unified view of information. This integration bridged gaps and made cross-functional collaboration smoother.
3. Precision in a Global Context: The tool wasn’t just smart—it was contextual. Using boosting mechanisms, it prioritized results by recency, region, or relevance, ensuring the Global Development Banking Institution teams accessed the most impactful and up-to-date information for decision-making.
4. Reduced Operational Friction: The migration from Azure to Google Cloud cut down infrastructure delays, while the automation of testing and evaluation brought consistency and efficiency.
5. Empowered Decision-Making: From policymaking to disaster response planning, the Global Development Banking Institution’s ability to make evidence-based decisions hinges on quick access to relevant data. This tool elevated that capability, supporting their mission of solving complex, global challenges.
6. Scalable for the Future: Designed with adaptability in mind, the application serves as a blueprint for future innovations. Whether it’s expanding to new datasets, enhancing search precision, or integrating emerging technologies, the Global Development Banking Institution now has a scalable foundation for its information management needs.

Ultimately, this wasn’t just about solving a technical challenge. It was about creating a solution that empowers the Global Development Banking Institution to focus on its bigger picture—helping communities, enabling sustainable growth, and driving meaningful change. Because when data moves faster, so does progress.

The Challenge

Imagine standing at the base of a giant mountain, knowing you need to climb it—but here’s the catch: the mountain isn’t made of rocks; it’s made of data. Welcome to the Global Development Banking Institution’s information landscape, where navigating millions of documents, reports, datasets, and acronyms is the everyday reality. The challenge? Turning this vast, overwhelming expanse into something that felt more like a well-organized library than an endless maze.

1. Data Overload Meets Complexity The Global Development Banking Institution’s ecosystem wasn’t just vast—it was monumental. Millions of records, ranging from structured datasets to unstructured policy papers, were scattered with no clear map. The challenge was to unify this fragmented data landscape into a single, cohesive search experience—like turning scattered puzzle pieces into a vivid, complete picture.
2. The Relevancy-Recency Tug-of-War People needed answers that were not just relevant but also recent. How do you create a system that balances these two while delivering precise, context-aware results in seconds? Let’s just say, this was less of a question and more of a riddle.
3. Acronyms, Synonyms, and Chaos In the Global Development Banking Institution’s world, acronyms are everywhere, each with multiple meanings depending on the context. Add synonyms to the mix, and search results could spiral into chaos. Our job was to build a search engine that could think like a linguist, sorting through linguistic complexity to ensure results were clear, precise, and contextually accurate.
4. The Security Puzzle When working with organizations as global and mission-critical as the Global Development Banking Institution, data security isn’t negotiable. Add in the fact that this project involved collaboration across Aretec, Google, and Global Development Banking Institution teams, and you get a Rubik’s Cube of security policies, compliance requirements, and operational constraints.
5. A Perfect Blend of Speed and Depth Speed was non-negotiable, but depth couldn’t take a backseat either. The application needed to not only fetch answers lightning fast but also dig deep into data silos to ensure those answers were rock-solid.
6. The Big Migration Marathon The migration from Azure to Google Cloud can be thought of as changing the foundation of a skyscraper—while people are still working inside it. One of the key challenges faced during the project was migrating the Global Development Banking Institution’s enterprise search solution from Microsoft Azure to Google Cloud Platform (GCP). This migration was essential to improve performance, particularly in reducing search time and enhancing overall system stability. The transition required significant technical effort, including the development of Terraform scripts for infrastructure as code, enabling seamless and automated deployment processes across the new cloud environment.

So, the challenge boiled down to this: How do you turn a mountain of disorganized, multi-format data into an intuitive, lightning-fast search experience that’s secure, intelligent, and scalable? Spoiler alert: we cracked it.

Technical Constraints

Here’s what we had to navigate to make it all work:

The Creative Twist – Why This Collaboration Was Key

The collaboration between Aretec, Google, and the Global Development Banking Institution was truly the backbone of this project, and it was crucial to its success. Each party brought invaluable expertise to the table, creating a synergy that pushed the boundaries of what we could achieve.

1. Aretec’s Expertise: A proven track record in leveraging AI and cloud technologies to deliver innovative solutions backed by its development of diSearch, a patented search product designed to revolutionize information retrieval. diSearch combined advanced AI capabilities, robust indexing mechanisms to deliver highly relevant results across structured and unstructured datasets. This in-house innovation showcased Aretec’s deep understanding of search technologies, including boosting, ranking, synonym/acronym handling, and dynamic filtering. With diSearch as a proven success story, Aretec leveraged its proprietary knowledge and technical expertise to tailor solutions that address complex organizational needs, making it a trusted partner for cutting-edge enterprise search applications.
2. Google Cloud’s Vertex AI: Industry-leading AI tools capable of handling complex data blending, advanced intent detection, and generating relevant summaries.
3. Global Development Banking Institution’s Vision: A strong focus on delivering value to internal teams by enabling smarter access to critical data resources.

Through transparent communication, a shared commitment to excellence, and continuous problem-solving, this collaboration made the impossible seem achievable. The collaboration combined Aretec’s technical agility, Google’s pioneering AI technology, and World’s Leading Multilateral Development Banking Institution’s deep understanding of its data landscape, ensuring the project’s success.

Key Services Delivered

1. Architecture Development and Optimization

1. Structured and unstructured search architectures
2. Migrated to infrastructure from Azure to Google Cloud Platform (GCP)
3. Terraform scripts for efficient deployment and management.
4. d. Network assessments

2. AI Model Integration and Optimization

1. LLM Model Analysis and Integration
   
2. Prompt performance evaluations
   
3. Base prompts for follow-up and recommended questions.
4. LLM performance analysis

3. Search Engine Enhancements

1. Boosting Controls to improve search relevancy by recency, region, and relevance.
2. Query Optimization
3. Intent Detection
4. Filters

4. Data Management and Blending

5. Performance Enhancements and Testing

6. Technical Documentation and Feasibility Studies

7. Frontend and User Experience

8. Evaluation and Quality Control

Our Approach

Embarking on this ambitious project, our approach was driven by the need to create a seamless and intelligent enterprise search experience for the Global Development Banking Institution, leveraging cutting-edge AI technology and our expertise, including insights from our patented search product diSearch. From the very beginning, we took a meticulous, data-driven path, addressing complex challenges while ensuring performance, security, and user satisfaction.

Each solution we proposed went through a rigorous, multi-phase testing process before being finalized for implementation. We took great care in evaluating every aspect of the system to ensure that it would meet the Global Development Banking Institution’s needs while maintaining scalability and performance.

For example, we thoroughly tested whether the Search API or Answer API would deliver the best results for different types of queries, ensuring that responses were not only accurate but contextually relevant. Similarly, we carefully assessed whether Gemini Pro or Flash would be the optimal choice for generating summaries, considering factors like response time, clarity, and relevance.

The core of our solution relied on the successful implementation and testing of various Ranking APIs for both structured and unstructured data. These were tested extensively to resolve any bugs, address issues related to page relevance scoring, and optimize performance. Each adjustment was carefully tested in real-world scenarios to ensure we weren’t just improving speed but also the quality of the results.

We also conducted comprehensive Playbook Testing, ensuring that each feature, from Grounding API implementation to the Function Calling capabilities of Gemini, worked flawlessly. This step-by-step evaluation process gave us the confidence that our solutions were aligned with the Global Development Banking Institution’s requirements, making the system both reliable and efficient.

In every stage of development, from ranking to summary generation, we prioritized real-world testing and validation to guarantee that each solution was not just theoretical, but practically optimized for performance.

1. Customizing Search Functionality with Google Cloud: Our first step was to implement a custom search solution powered by Google’s Vertex AI. This required deep integration with DialogFlow to enhance intent detection and entity recognition. We tackled challenges like misaligned geo-country entities, ensuring the system would match valid country names and improve the overall search accuracy. A key part of the search optimization involved implementing boosting mechanisms based on various parameters such as region, recency, and relevance, especially for complex queries. This also included boosting by country and year, allowing the system to fetch more contextually accurate results based on user queries.
2. Achieving High-Quality Results through Ranking and Boosting: We carefully designed and tested the ranking and boosting logic to ensure the highest quality search results. Through systematic use of automated scripts, we validated boosting and threshold settings across millions of data points. This was essential to refine how the system handled nuances like reverse acronyms and partial matches, ensuring relevant content surfaced, even when complex variations were queried. We also implemented confidence thresholds in Vertex Search, which allowed the system to rank results more reliably by their relevance and accuracy.
3. Continuous Evaluation and Scalability: To keep pace with the Global Development Banking Institution’s diverse data requirements, we continuously evaluated the system through an extensive playbook testing approach. This included refining the implementation of ranking APIs for both structured and unstructured data. Alongside this, we worked on improving semantic search by ensuring that results were contextually relevant and filtering out irrelevant content, such as the incorrect return of similar-sounding words like “Amina” or “Amin.” We also set up comprehensive pagination mechanisms to ensure smooth browsing experiences for users, even when sifting through large volumes of data.
4. Enabling Real-Time Insights with AI-Driven Recommendations and Summaries: An integral part of our solution was the introduction of recommended questions and search summaries. Using Google’s Gemini capabilities, we introduced guided search results and follow-up questions, enabling users to navigate through information more intuitively. Additionally, the Search Summary Implementation helped the Global Development Banking Institution access key insights at a glance, allowing them to focus on the most critical data. The use of semantic search and the answer API also ensured that queries returned highly relevant and accurate responses based on user intent.
   
   By creating a system that seamlessly integrates structured and unstructured data with intuitive AI-driven search capabilities, we delivered an enterprise search application that not only met the Global Development Banking Institution’s immediate needs but also set a solid foundation for future growth and scalability.
5. Seamless Integration and Deployment: Throughout the project, we maintained close communication with the Global Development Banking Institution’s internal team, ensuring that each module we developed was integrated seamlessly into their infrastructure. We provided deployment support, troubleshooting any issues and ensuring that the system was up and running smoothly.

Results

• Search Speed: The application achieved an impressive response time of 2-6 seconds, aligning perfectly with the Global Development Banking Institution’s expectations for efficiency in information retrieval.
• Search Quality: By incorporating advanced boosting mechanisms and refining intent detection, the system significantly improved the relevancy and recency of search results, ensuring that users receive more accurate and contextually appropriate data.
• Scalability: With the successful ingestion and indexing of over 4 million documents, the system demonstrated its ability to handle massive data volumes while maintaining high performance.
• Collaboration: The seamless partnership between the Google, Aretec, and Global Development Banking Institution teams facilitated rapid learning and knowledge exchange, ensuring clear communication and continuous improvement throughout the project.
• Innovation: We introduced cutting-edge strategies to handle blended queries, improve search relevance, and refine intent detection, setting the stage for more advanced enterprise search solutions in the future.

Business Impact

This project has not only revolutionized how the Global Development Banking Institution accesses and leverages its data but also delivered critical business advantages:

• Accelerated Decision-Making: The ability to retrieve key insights within seconds has greatly empowered teams to act swiftly, enabling quicker responses to pressing global challenges and policy initiatives.
• Reduced Manual Effort: By automating testing, evaluation, and search processes, the Global Development Banking Institution has saved significant time and resources, allowing staff to focus on high-value tasks instead of manual oversight.
• Future-Ready Infrastructure: The adoption of Google Cloud’s AI ecosystem has not only enhanced current capabilities but also set the foundation for future-proof solutions, positioning the Global Development Banking Institution for continued innovation in data management and enterprise search.

Lessons Learned

Throughout this challenging yet rewarding journey, we learned some valuable lessons that shaped the final solution and improved our approach for future projects.

1. First and foremost, the delicate balance between relevancy and recency in search results proved to be a dynamic challenge. Fine-tuning this balance is not a one-time task—it requires constant iteration and adjustments. We discovered that what works today may need tweaking tomorrow, especially when dealing with a system as complex and dynamic as the one we were building for the Global Development Banking Institution.
2. The importance of collaboration cannot be overstated. This project required seamless teamwork across three key partners—Aretec, Google, and the Global Development Banking Institution. Aretec’s expertise in system integration and technology, combined with Google’s advanced AI capabilities, allowed us to tackle issues such as security, scalability, and search optimization efficiently. Without this deep collaboration, we would not have been able to solve the intricate challenges of working with diverse data types, security constraints, and performance goals.
3. One of the most eye-opening lessons came from our efforts in automating testing and evaluation. Whether it was testing boosting thresholds or ensuring that follow-up questions worked as expected in Vertex Search, we learned that automation saves significant time and effort. Additionally, it ensures that the system is consistently performing at its best, even with millions of data points. This allowed us to focus more on refining functionality rather than worrying about manual testing.
4. Having a golden dataset became a crucial factor in validating results and benchmarking performance. This dataset allowed us to evaluate the system’s output in a consistent and reliable manner. It was clear that in such a large-scale system, having a reference point for comparison is invaluable to ensure accuracy and alignment with expectations.
5. Another lesson was the need for clear and effective communication with all key stakeholders. From technical teams to business leaders, we learned that transparency and timely updates foster trust and ensure everyone is aligned. This was particularly important when we faced challenges, such as the migration from Azure to Google Cloud and the integration of complex technologies like Gemini Pro and Flash.
6. Finally, we recognized that agility is essential when working on large-scale projects with tight timelines. Issues such as schema mismatches, result confidence thresholding, and handling tricky queries (like reverse acronyms or partial matches) required us to be flexible and quick to respond with new approaches. The ability to iterate quickly and effectively test new solutions kept us on track and ensured that we could meet the project’s goals within the set timeframe.

Conclusion

In conclusion, the journey of building this enterprise application for the Global Development Banking Institution was a testament to the power of collaboration, adaptability, and rigorous testing. What began as a challenge to integrate vast and varied data sources into a seamless, efficient system evolved into a highly optimized solution thanks to the combined efforts of Aretec, Google, and the Global Development Banking Institution. Each partner played a critical role in ensuring the success of the project, from system integration and security measures to advanced AI capabilities and cloud infrastructure.

Our ability to conduct extensive testing on various technologies, including the Ranking API, Vertex Search, and the Gemini Pro models, was invaluable. These tests not only helped us refine the system but also deepened our familiarity with Google’s suite of technologies. As a result, we are now well-positioned to make informed recommendations. Whether it’s optimizing a search algorithm or selecting the right tool for document classification, we have gained the expertise to guide future endeavors with confidence.

The lessons learned throughout this process—especially around balancing relevancy and recency, automating evaluations, and integrating complex systems—will continue to shape how we approach similar projects in the future. As we move forward, we are better equipped to handle the dynamic needs of large-scale systems, ensuring that we deliver solutions that are both scalable and efficient. The success of this project has set a strong foundation for future innovation, and we look forward to applying these insights in upcoming challenges.