Post Provided by Derya Akkaynak

I am an aerospace engineer turned oceanographer, and I am obsessed with colour in the ocean.

How could I not be? The seafloor is full of colourful organisms, but we almost always see them through the coloured light that water leaves us, never in colours of their own. However, this is not a limitation of my eyes, or yours. Water rewrites the signal on its way from the scene to what or whoever is looking: a camera, a diver, a grouper hunting, or a mantis shrimp with many, many photoreceptor classes… So, every observer receives a colour signal that the water has already altered. How that signal is interpreted is an entirely separate question.

Our own interpretation, for instance, is not entirely passive. Colour constancy, the same machinery that lets you call a shirt white both indoors and outdoors, discounts parts of this colour shift, which is why we can still enjoy the lush and diverse colours of a coral reef in shallow waters. But it can only go so far. Our colour constancy is remarkably good at discounting changes in illumination on land, but it was not built for a water environment that alters light with every meter it travels. Whether aquatic animals manage better is an open question: we know far too little about how their visual systems handle colour to say. What we can say is that computer vision algorithms can now do it: given a RAW image and an equation that describes how water alters light as a function of wavelength and distance, images can be reverse engineered to be ‘cleaned’ of the effects of water. Importantly, all of that can be done objectively and repeatably according to the laws of physics, not with blackbox AI magic.

Allow me to introduce myself: my name is Derya Akkaynak. I am an assistant professor at the University of Haifa. In 2022, I established COLOR Lab (the Laboratory for Computational Optics and Light in the Ocean Realm) at the Hatter Department of Marine Technologies. We are located on the campus of the Interuniversity Institute for Marine Sciences in Eilat, on the Red Sea, a perfect setting to do underwater experiments any day of the year. Across this backdrop, my ambition is to write the book: “Why are there colours in the ocean?”

It is only half a joke. The question sounds like it belongs to biology, but it cannot be answered until we can say what colour something down there actually is, so first and foremost it is a measurement problem that needs an engineering solution—one that I find so interesting that I am excited to dedicate my career to it.

To measure ‘true’ colours in the ocean, we can always MacGyver underwater housings for lab-grade, ultra-sensitive optical instruments. But those would be very expensive to build and difficult to operate underwater, and one-offs that many other labs could not easily replicate. Many such instruments have been devised and rusted in PIs’ cabinets after very little use over their lifetimes. The abundant consumer camera offers a real opportunity here: it puts a colour-measuring device in the hands of every marine scientist. If I am to really answer why there are colours in the ocean, I will need data at spatial scales so large that no single lab or research cruise could ever cover them. The measurements will have to come from many people, over many years. And while not perfect, those will be the data that will come from compact and affordable consumer cameras that have made it to every corner of the world.

If you are wondering how a consumer electronics device designed solely to produce pretty images can be trusted with so much scientific responsibility, read on.

The answer rests on one underappreciated property: a camera sensor responds almost linearly to light. If you double the light, you get double the signal (i.e., double the pixel intensity). For a few milliseconds of exposure, your camera really is a radiometer, and the pixel values of an image really do mean something about the light that arrived from the scene. But to tap into this superpower, you must capture the image in RAW, before the camera turns those measurements into a vivid JPEG* by baking in modifications that cannot reliably be undone.

RAW is necessary, but not sufficient. For photos taken in air, it almost gets you everything you need. Air modifies light very little over ordinary imaging distances, so once you have accounted for the illumination using a reference like a colour chart and characterized your camera’s spectral response, colours in the scene can generally be standardized reliably. In water, light is absorbed and scattered on its way to the subject and again on its way back to the sensor, so a coral one meter away and the sand four meters behind it experience colour distortions differently, which is why white balancing or fancy filters cannot consistently “fix” colours in underwater images.

What you need to do is unglamorous and tedious, and it is entirely in your control: Preserve the RAW image. Place at least one colour reference in the scene. Plan your image acquisition so that scene distance can be estimated. This is what we call the 3P Protocol in our Perspective article: if you acquire only in-camera processed imagery underwater, its colours cannot be calibrated today, and it will not be possible to calibrate in the future either, no matter how sophisticated methods become. There is much more to it, of course, so please have a read! Read the full article here (LINK).

If you don’t shoot RAW, you don’t have any reliable colour information to work with. The readership of Methods in Ecology and Evolution will appreciate the parallel with formalin: for more than a century, it was the obvious way to preserve a specimen. It kept what collectors wanted—morphology—but ruined something nobody had yet thought to want: DNA. Whole collections became unusable for questions their curators could not have imagined. There are methods today with which we can recover fragments from formalin-fixed tissue, but no one would choose that over a frozen sample, and our case is worse than theirs: the measurement in a JPEG is not damaged, it is absent.

So let me end with a saying that has become the tagline of the Underwater Colorimetry course I teach: JPEGs are like hot dogs. If you knew how they were made, you would never take one.

Read the full article here.