The offset of [tabread4~] was there to avoid any reading errors when the index points get too high (the whole sample is almost 7m long). So there's not option for this, but to use only the right entry?
Seems like you should stick with your original approach using the right inlet for good indexing resolution, and work in a [block~ 1] subpatch to force the inlet to update every sample.