解决办法:
Apparently Acrobat OCR's the document just to figure out in which order it should be read aloud, and there's no way to disable this. The document I was having trouble with does have equations in it, so whether or not this takes long enough to be annoying may depend on whether the document contains things that are hard for OCR to process.
You can fix it by editing preferences for "Reading" and change Reading Order to "Left-to-right, top-to-bottom..." instead of "Infer reading order from document (recommended)". Then make sure Screen Reader Options is set to "Only read the currently visible pages".