Libraries are mainly places of knowledge.
They could also include second that aren't just books: a museum or natural museum, an art gallery, a meeting space for academic conferences, etc.
Was that part of the library used by an underwater civilization? Or was it about one?

If it's the former, even our idea of a book or scroll might be wildly different from them.
If they lived underwater, instead of using ink on paper/vellum, maybe they carved on things, or arranged shells or sand, tied knots in strands of rope like material, or tapestries, or influenced the growth of corals in ways to display their ideas, etc. If they used ink, maybe it was more like a 3d holographic representation but with pigments suspended in water.
Turbulence, moving through might simply make it ineligible (or maybe it can be reversed, see below)

1 minute science video about fluids

Did they even have light?
Bioluminescence might be something cool here, where the "books" themselves might literally be live bio things that are alive.
Or maybe just the ambient light can be provided by bioluminescent plankton as you move around.
Maybe their way of displaying an idea was recreating specific movement in the water so the plankton is simulated in a specific pattern to display.

You can probably also steal and reskin some of the stuff in the candlekeep adventure if you need side content. Won't be a drop-in replacement but the general vibe is there.