Aiming to give scientists better access to the vast quantity of science data it collects, NASA has created a Science Discovery Engine that leverages generative AI to deliver contextual results. Credit: NASA When you generate and collect as much data as the US National Aeronautics and Space Administration (NASA) does, finding just the right data set for a research project can be a problem. With seven operating centers, nine research facilities, and more than 18,000 staff, the agency continually generates an overwhelming amount of data, which it stores in more than 30 science data repositories across five topical areas — astrophysics, heliophysics, biological science, physical science, earth science, and planetary science. Overall, the agency houses more than 88,000 datasets and 715,000 documents across 128 data sources. Its earth science data alone is expected to hit 250 petabytes by 2025. In light of such complexity, scientists need more than just domain expertise to navigate through it all. “It requires researchers to know which repository to go to and what that repository has,” says Kaylin Bugbee, NASA data scientist at Marshall Space Flight Center in Huntsville, Ala. “You have to be both science literate and data literate.” In 2019, NASA’s Science Mission Directorate (SMD) released a report based on a series of interviews with scientists that made it clear those scientists needed a centralized search capability to help them find the data they needed. The SMD’s mission is to engage with the US science community, sponsor scientific research, and use aircraft, balloon, and spaceflight programs for investigations in Earth orbit, in the Solar System, and beyond. Recognizing that giving scientists and researchers access to its data was fundamental to its purpose, SMD developed its Open Source Science Initiative (OSSI) as a result of that report in an effort to make publicly funded scientific research transparent, inclusive, accessible, and reproducible. The mission of the OSSI: a commitment to the open sharing of software, data, and knowledge (including algorithms, papers, documents, and ancillary information) as early as possible in the scientific process. “It really came from the scientists and scientific community, and it also aligns with our broader SMD priority of enabling interdisciplinary science,” Bugbee says. “That’s where new discoveries are made.” To facilitate that mission, the agency is now turning to a combination of neural nets and generative AI to put those vast amounts of data at scientists’ fingertips. Restoring order A key element of OSSI is the Science Discovery Engine (SDE), a centralized search and discovery capability for all of NASA’s open science data and information, powered by Sinequa’s enterprise search platform. “Until the SDE was created, you couldn’t go to a single place to search for our open data and documentation,” Bugbee says. “Now it serves as a single search capability for our open science data.” New York-based Sinequa, which got its start more than two decades ago with a semantic search engine, focuses on leveraging AI and large language models (LLMs) to deliver contextual search information. It has since integrated Microsoft’s Azure OpenAI Service with its own neural search capabilities to power the platform. Specifically, Sinequa’s neural search capability uses a combination of keyword and vector search to discover information, while its GPT summarizes the information gathered into rapidly digestible and reusable formats. It also allows scientists to use natural language to ask deeper questions and refine the search or the response. The SDE understands nearly 9,000 different scientific terms, with that number expected to grow as the AI learns. Bugbee and her interdisciplinary team, which includes scientists with expertise in data stewardship and informatics, as well as developers and AI and ML experts, worked closely with stakeholders to understand their needs, and also with NASA’s Office of the CIO and Sinequa to build a proof of concept. “They helped us set up the environment we needed,” she explains. “We had to have an open capability, so we had some special architectural needs.” Bugbee says one of her team’s biggest challenges in getting everything up and running was how dispersed content was across the NASA ecosystem. Her team spent about a year trying to understand the information landscape, the data, and the metadata schemas. “All of the contextual information that really brings richness to the data — things like code and GitHub, or algorithm documentation that describes how the data was developed — that kind of content is spread over a number of web pages and it’s been an effort to curate and identify where all those things reside,” she says. Cleared for launch Bugbee is no stranger to data management and data stewardship. She cut her teeth in the field working to improve metadata quality in Data.gov and on President Obama’s Climate Data Initiative. But working on the SDE really drove home the importance of good curation workflow: the processes for principled and controlled data creation, maintenance, and management. “If I could go back in time, I’d have a more robust curation workflow built in from the beginning,” she says. “We kind of used the out-of-the-box approach to start and it worked for a time, but to really get the results we wanted, we needed that curation workflow.” While the SDE is still in beta, Bugbee says her team has received a great deal of positive feedback from scientists to date, and the plan is to deliver a more fully operational system later this year. Already the team has implemented a new user interface that allows users to filter by topics before they begin their search. Related content news Atos may sell national security activities to French government The troubled IT service provider could net up to $1 billion from the sale, meeting most of its financing needs for the next year. By Peter Sayer Apr 29, 2024 4 mins Government IT Government Managed IT Services feature Top 10 barriers to strategic IT success Data challenges, tech debt, and talent shortages are among the issues that can derail your IT org’s work on high-value initiatives. Here’s how some CIOs are addressing them. By Mary Pratt Apr 29, 2024 12 mins Hiring IT Skills Business IT Alignment news analysis The new CIO mandate: Selling AI to employees Employees surveyed express enthusiasm about AI, but they also worry about the impact on their jobs and want training and guidelines. By Grant Gross Apr 29, 2024 5 mins Staff Management Artificial Intelligence opinion Dump the RFP to reap better outsourcing results The RFP has been the default tool for sourcing bids for too long. Organizations wanting more collaboration — and innovation — from their suppliers should consider a collaborative bidding approach. By Kate Vitasek Apr 29, 2024 6 mins Outsourcing PODCASTS VIDEOS RESOURCES EVENTS SUBSCRIBE TO OUR NEWSLETTER From our editors straight to your inbox Get started by entering your email address below. Please enter a valid email address Subscribe