I don't even know how to answer this because it's coming from a place that's so foreign to me. I have a quite weak visual imagination (not full-on aphantasia), and I've never heard a voice speaking words in my head when I read (although actually, now that I've listened to a lot of audiobooks, I can force this to happen briefly if I concentrate). But I've always enjoyed reading! To me, I guess I would say, words are just sort of fundamental? Like, the word itself, the shape of squiggles on the page, is the thing that has meaning, and I don't have to visualize anything beyond it to understand it. It's like the difference between reading a foreign language you're not that good at, where you still have to translate every thought into your native language to really understand it, versus reading in your native language, where there's no translation step.
It might be true, as quoted in Mo's comment below, that people with weak visual imaginations are less likely to enjoy extremely visual-description-heavy fiction like Lord of the Rings – I indeed found reading the LotR books mind-numbingly boring, borderline painful. But in almost all works there's a ton of content that can be enjoyed without imagining it into being: concepts, emotions, even most kinds of events. I also find myself very drawn to beautiful writing, wordplay, and just skillful use of language in general, and now that I think about it, this is probably why. Neat!
The same applies to nonfiction – building an understanding of the connections between concepts doesn't have to rest on any sort of visual framework. The concepts can just become connected, literally/physically, in the structure of your brain.
I do think that I would have struggled less with university-level physics and the related math if I had more of a visual imagination. It's quite hard to keep track of things when it all just feels like symbols you're manipulating; I imagine that being able to visualize things would have let me viscerally understand connection between the math and the physical systems being described. As it was I just sort of, knew explicitly that they were connected. But it was all very vague and confusing.
I hope this helps you understand the experience somewhat. Feel free to ask me followup questions.
When I first learned about aphantasia, I thought It described me - I don't naturally visualize when I read. But after closer inspection, I found out that I can visualize if I put some effort into it. Images might not be terribly vivid, but recognizable enough.
So technically I don't have aphantasia, but my experience is pretty close, and it's all kinda confusing . For the most part of my life, I did not even realize that was not normal.
I was always fast reader because of that, you can save time and mental resources by not visualizing, so that's an upside. As for downsides, I can't imagine them, haha.
Everything everybody else has said resonates with me as well, but there's one thing nobody has really hit on yet, so I'll talk about that.
While I have no visual imagination, I have a fairly rich auditory one. While thinking up an example, the McDonalds jingle that goes " ba da bop ba baaaa, I'm loving it" played in my head. I can recall it at will, and pitch shift it as I want. I make no claim of having perfect pitch, but I do have decent relative pitch.
My internal voice has internally repeated nearly every sentence I have ever read or written, unless I deliberately shut it off. I can recall voices and sounds, and I think it helps me identify accents and languages. This doesn't make me a perfect mimic, but I have yet to mistake an Aussie for a Kiwi , which apparently happens.
Movies definitely affect my reading of books, but I usually don't mind these changes. I'm never going to read Samwise in a voice that isn't Sean Astin's, and that's fine by me. My reading speed is average. And in a good book, I will often stop and reread particularly pleasing prose, which is generally (but not necessarily) alliterative. The pleasing effect can either be in hearing the sounds play out, in my head, or in the way it would feel to say the sounds. So while visual readers may enjoy a book for the color of the roses, I am often doing similarly, by admiring the susurrations of those same roses. This may be related to ASMR in some way.
Although I have Aphantasia, standardized tests indicated that I have high reading comprehension. I wonder if my Aphantasia contributes to my reading very quickly. I read fast enough that when one of my students asks me to read a paper in office hours if I read at my normal speed I figure she will think I didn't actually read her paper.
James_Miller mentioned Aphantasia and it blows my mind: "What?...imagination has been my greatest portion of thinking...and there are people who cannot imagine..." To me, weak but constant visualization & audioization come right after reading any texts, and I stop reading when my brain can't visualize properly. I once suspected my sister(very young at that time) does not like reading books because she was not used to imagine while reading.
How do people with Aphantasia read texts and process while studying?