Vacation photos of zebras and whales that tourists post on social media may have a benefit they never expected: helping researchers track and gather information on endangered species.

 Scientists are using artificial intelligence (AI) to analyze photos of zebras, sharks and other animals to identify and track individuals and offer new insights into their movements, as well as population trends.

“We have millions of images of endangered and threatened animals taken by scientists, camera traps, drones and even tourists,” said Tanya Berger-Wolf, director of the Translational Data Analytics Institute at The Ohio State University. 

And a new field called imageomics is taking the use of wildlife images a step further by using AI to extract biological information on animals directly from their photos, said Berger-Wolf, a professor of computer science and engineering, electrical and computer engineering, and evolution, ecology and organismal biology at Ohio State.

 

She discussed recent advances in using AI to analyze wildlife images and the founding of imageomics in a presentation Feb. 20 at the annual meeting of the American Association for the Advancement of Science. She spoke at the scientific session “Crowdsourced Science: Volunteers and Machine Learning Protect the Wild for All.”

 One of the biggest challenges that environmentalists face is the lack of data available on many threatened and endangered species.

“We’re losing biodiversity at an unprecedented rate and we don’t even know how much and what we’re losing,” Berger-Wolf said.

 Of the more than 142,000 species on the IUCN Red List of Threatened Species, the status of greater than half are not known because there is not enough data, or their population trend is uncertain.

“If we want to save African elephants from extinction, we have to know how many there are in the world, and where they are, and how fast they’re declining,” Berger-Wolf said. “We don’t have enough GPS collars and satellite tags to monitor all the elephants and answer those questions. But we can use AI techniques such as machine learning to analyze images of elephants to provide much of the information we need.

Berger-Wolf and her colleagues created a system called Wildbook that uses computer vision algorithms to analyze photos taken by tourists on vacation and researchers in the field to identify not only species of animals, but individuals.

 “Our AI algorithms can identify individuals using anything striped, spotted, wrinkled or notched – even the shape of a whale’s fluke or the dorsal fin of a dolphin,” she said.

For example, Wildbook contains more than 2 million photos of about 60,000 uniquely identified whales and dolphins from around the world.

“This is now one of the primary sources of information scientists have on killer whales – they are data deficient no longer,” she said.

In addition to sharks and whales, there are wildbooks for zebras, turtles, giraffes, African carnivores and other species.

 

Berger-Wolf and her colleagues have developed an AI agent that searches publicly shared social media posts for relevant species. That means many people’s vacation photos of sharks they saw in the Caribbean, for example, end up being used in Wildbook for science and conservation, she said.

 

Together with information about when and where images were taken, these photos can aid in conservation by providing population counts, birth and death dynamics, species range, social interactions and interactions with other species, including humans, she said.

This has been very useful, but Berger-Wolf said researchers are looking to move the field forward with imageomics.

“The ability to extract biological information from images is the foundation of imageomics,” she explained. “We’re teaching machines to see things in images that humans may have missed or can’t see.”

For example, is the pattern of stripes on a zebra similar in some meaningful way to its mother’s pattern and, if so, can that give information about their genetic similarities? How do the skulls of bat species vary with environmental conditions, and what evolutionary adaptation drives that change? These and many other questions may be answered by machine learning analysis of photos.

 The National Science Foundation awarded Ohio State $15 million in September to lead the creation of the Imageomics Institute, which will help guide scientists from around the world in this new field. Berger-Wolf is a principal investigator of the institute.

 As the use of AI in analyzing wildlife images continues to grow, Berger-Wolf said, one key will be to make sure the AI is used equitably and ethically.

For one, researchers have to make sure it does no harm. For example, data must be protected so that it cannot be used by poachers to target endangered species.

But it must be more than just that.

 “We have to make sure that it is a human-machine partnership in which humans trust the AI. The AI should, by design, be participatory, connecting among the people, among the data and among the geographical locations,” she said.