Delta Smelt are Ready for Their Close-Up

New Underwater Video Technology Aids in Fish Research

It’s not just Hollywood that’s talking about best pictures these days. Introducing the “SmeltCam”-a new underwater video technology that allows scientists to monitor and count endangered fish in the Sacramento-San Joaquin Delta without handling them, which often results in the fish’s death.

The SmeltCam is a video imaging system engineered for identifying and counting fish in turbid water that allows research to supplement current population monitoring such as the fall mid-water trawl without harming threatened or endangered species. It is developed as part of the Interagency Ecological Program (IEP) by the Bureau of Reclamation, SureWorks LLC, the California Department of Water Resources, and the California Department of Fish and Game.

“The SmeltCam technology is really exciting because it has the potential to open up a whole new underwater world to scientists, managers, and the public alike and literally bring it alive,” said Anke Mueller-Solger, Lead Scientist of the Interagency Ecological Program (IEP).

According to Reclamation, once the technology is in wide use, scientists will be able to get a much better sense of fish behavior and fine-scale distributions of fish in their habitat while worrying less about harming the fish they are studying. The public would be able to see videos of the live fish that programs such as the IEP are monitoring and studying, and not just numbers and graphs representing them.

The video imaging system uses a high-speed video camera with an embedded computer enclosed in an airtight, towed submersible attached to a large, open-ended net. The net funnels fish in front of the camera, which shoots a video of them, then the fish pass freely and safely back to the open water. Fish images are then streamed to a computer on the boat, where specially designed computer software attempts to match the pictures to a library of images and descriptors. A series of visual and tracking algorithms based on species-specific fish body measurements classify the fish and provide a best fit species match to each fish image.

“Biologists were able to capture the first video of wild delta smelt in their murky environment with this device-which is very exciting to the scientific community,” said Don Portz of Reclamation’s Fish and Wildlife Resources Group, who serves as the principal investigator for the SmeltCam project. “Innovative fish identification tools such as the SmeltCam are necessary to provide more accurate population numbers, reduce the labor intensive costs associated with monitoring, and to lessen the impact that current monitoring practices have on fish.”

Currently trawls are the foundation of IEP fish monitoring in detecting the pelagic organism decline (POD) and population trends for pelagic Bay-Delta fishes. For more than 40 years, IEP has been sampling the Delta using a variety of sampling techniques including a mid-water trawl. “That provides great data,” Portz said, “but inadvertently you end up killing the fish you’re sampling for and several are ESA (Endangered Species Act) species of concern. What we wanted was a supplemental way to go out and sample fish without killing them.”

He explained that trawl nets used in sampling are large nets with a wide mouth that close at the end like the toe of a sock. The problem is that sticks, leaves, garbage and other fish get compacted at the end of the net on top of very sensitive fish species such as delta smelt. The net is then brought on board to remove the collected fish. “It’s very stressful for the fish,” he said. “Often they’re dead, impinged on the net. We needed to come up with another way of sampling.”

An open-ended net funnels fish in front of a video camera allowing fish to safely return to the water - image courtesy of Bureau of Reclamation.
An open-ended net funnels fish in front of a video camera allowing fish to safely return to the water
(Image courtesy of Bureau of Reclamation)

The SmeltCam was the answer. This video imaging system uses a modified trawl net where the ‘sock’ end of the net is cut off and the camera system inserted, turning the net into a funnel rather than a catching device. The fish move through a 7 ½ -inch opening designed for small fish (i.e., shad, smelt) past the camera system and out. “Here we’re never touching the fish, the net’s not touching the fish,” Portz said.

The high-speed video camera takes many frames or shots of the fish-including side profile shots which are important for differentiation and identification-before it exits the back of the submersible. Multiple pictures are then fed into the submersible’s computer system and transmitted over a power cord to the boat.

The computer system first identifies ‘fish, non-fish’ since fish have unique shapes compared to an air bubble, aquatic vegetation, etc. Mathematical equations rule out non-fish shapes such as spheres since no Delta fish have a round profile.

The computer then tries to match the images caught on camera to a database (library) of fish images that includes an image of delta smelt. “It’s right out of your favorite action movie,” Portz said. “The picture is matched to known fish images in a database, like that for facial recognition software at the airport or police shows on TV.”

Portz added that he and his colleagues are trying to expand the database to include as many fish species as possible in the Delta. Currently it’s set up for delta smelt, threadfin shad, and American shad. Ninety-five percent of the time, the computer recognizes fish, but only 88 percent of the time does it know it’s a delta smelt. “There is a margin of error,” Portz said, “but if the computer’s unsure it puts everything in an ‘unknown’ file; and then a biologist comes and identifies species in the unknown file. It’s still a big time saver compared to a biologist on the boat looking at all these samples all the time.”


“With the SmeltCam you know the very moment the fish went through… you know exactly where in the Delta the fish were encountered and can then correlate it with temperature, salinity, turbidity, pollution. This is really good for understanding and preservation of the species.”

--Don Portz, Principal Investigator of SmeltCam Project, Bureau of Reclamation


Currently, the scientists have no way of pinpointing when or where the fish were caught using traditional methods. “During the net drag you don’t know where you encountered the fish,” Portz said. “With the SmeltCam you know the very moment the fish went through. With the mid-water trawl you are only provided with the end result; you don’t know what’s going on every moment, but with the camera, you can see everything in real time. And with the GPS system,” he added, “you know exactly where in the Delta the fish were encountered and then correlate it with temperature, salinity, turbidity, pollution. This is really good for understanding and preservation of the species.”

The SmeltCam is now in its fourth year of development. Portz said the next steps are to take more samples and refine the algorithm for identifying fish. “We’d like to be better than 95 percent sure of species,” he said. To that end they’re broadening the species database and refining the current algorithm for delta smelt to be more accurate, adding that they’d like to expand the database for longfin smelt. “The challenge is that the current system calls them all delta smelt: delta smelt, longfin smelt, wakasagi, and inland silverside. To the camera-even to an untrained eye-they look like delta smelt. We’d like to refine the camera even further... there are minor body differences among those four species.”

That calls for many more samples in the library to compare them to. “With increased sampling, we have better detection,” Portz said. “The more information we feed into this camera system, the software, the better we’ll get.”

Although the underwater video technology was invented for delta smelt, the SmeltCam team plans on expanding it throughout the region to other fish species of concern including juvenile Chinook salmon. Some of the software is already starting to be used in other locations for salmon and steelhead, including the Pacific Northwest.

Portz and his colleagues would also like to make the SmeltCam lighter and more user-friendly. The current submersible weighs about 170 pounds-they’d like to develop a device that’s less than 100 lbs. “We want to put these on many boats in the Delta and spread the love,” he said.

A presentation on the SmeltCam will be given at the IEP Annual Workshop April 20 in Folsom.