Sr. Software Dev Engineer, AGI - Web information retrieval

We are looking for a Senior Software Development Engineer with Rust expertise to develop a multi-modal, multi-lingual web-scale information retrieval system to make the world’s information accessible to AI models and customers everywhere. We are pushing the boundaries of what's possible by combining the power of large language models with retrieval systems, enabling advanced capabilities in question-answering, knowledge synthesis, and information grounding.
You will have an opportunity to directly impact the customer experience, design, architecture, and implementation of products that will be used every day by people you know. We're looking for someone passionate about innovating on behalf of customers, who demonstrates strong product ownership and is willing to think in new ways to solve difficult problems.
You will have track record of success in delivering new products, solving problems, and learning new technologies quickly. A commitment to teamwork, proactive approach to solving problems, and strong verbal and written communication skills are essential. Creating reliable, scalable, and high-performance products requires technical expertise, understanding of computer science fundamentals, and practical experience building efficient large-scale distributed systems. This person is comfortable delivering quality solutions in a fast-growing environment where priorities may change rapidly.

Key job responsibilities
* Build high-throughput, cost-effective data pipelines to support feature extraction and indexing for our web-scale Information Retrieval (IR) system.
* Develop and optimize the core algorithms and ranking models that power the search engine's ability to retrieve and rank relevant results for user queries.
* Design and implement efficient data structures and indexing techniques to store and retrieve massive amounts of web data and content.
* Optimize the performance, scalability, and reliability of the search engine's core components, including query parsing, retrieval, ranking, and result rendering.
* Collaborate with machine learning teams to integrate and deploy advanced machine learning models for query understanding, ranking, and personalization.
* Develop and maintain the control plane systems that manage and orchestrate the IR system infrastructure, including the distributed compute clusters, storage systems, and networking components.
* Design and implement real-time updates and freshness mechanisms to ensure the I engine reflects the latest web content and user behavior.
* Develop efficient, state-of-the-art streaming algorithms for processing large datasets (e.g. deduplication, topic clustering.)
* Participate in the design and implementation of real-time updates and freshness mechanisms to ensure the search engine reflects the latest web content and user behavior.

About the team
Unlock the future of AI at Amazon AGI (Artificial General Intelligence). At Amazon, we're at the forefront of transformative AI, shaping the next generation of intelligent technologies. For more than 25 years, we've been pioneering state-of-the-art AI and machine learning models to revolutionize customer experiences worldwide. Now, we're on the cusp of a new era, when AI holds the promise to reshape society and business in unprecedented ways. As part of our world-class team of AI experts—scientists, engineers, researchers, product builders, and public policy experts—you'll have the opportunity to unlock the frontiers of AGI. While the rapid advancements in Generative AI have captivated global attention, we see these as just the starting point. AGI is dedicated to pushing the boundaries of what's possible, using Amazon's unparalleled ML infrastructure, computing resources, and commitment to responsible AI principles. We're looking for the brightest minds from a wide range of backgrounds and experiences to help create transformative AI solutions that will improve lives, solve global challenges, and open up new realms of possibility—from reinventing commerce and accelerating enterprise productivity to advancing universal agents and shaping the future of robotics.