I think it depends how precise you want it to know. The latency implied by the i/o buffers mostly is the predominant part but when going to low latencies the hardware design of the audio interface plays an equal role. This can (portably for all possible interfaces) only be determined by using a cable loop from output to input, sending a step or dirac signal out, and measuring the delay time from the input.
the included patcher does the measurement (you need zexy)
greetings, Thomas
#N canvas 170 66 521 401 12; #X obj 23 59 dirac~; #X obj 163 170 dfreq~; #X obj 23 14 bng 25 250 50 0 empty empty empty 20 8 0 8 -258699 -1 -1; #X obj 163 219 snapshot~; #X obj 163 135 +~; #X text 53 16 trigger spike; #X text 304 93 2nd spike; #X text 74 133 1st spike; #X obj 238 181 metro 50; #X obj 239 153 loadbang; #X obj 196 296 /; #X floatatom 155 348 5 0 0; #X obj 163 265 t b f; #X obj 23 87 -~ 0.1; #X obj 23 132 dac~ 1; #X obj 249 57 adc~ 1; #X obj 248 92 dac~ 2; #X text 209 348 effective i/o latency (ms); #X msg 152 296 1000; #X text 75 86 20dB above noise; #X connect 0 0 13 0; #X connect 1 0 3 0; #X connect 2 0 0 0; #X connect 3 0 12 0; #X connect 4 0 1 0; #X connect 8 0 3 0; #X connect 9 0 8 0; #X connect 10 0 11 0; #X connect 12 0 18 0; #X connect 12 1 10 1; #X connect 13 0 4 0; #X connect 13 0 14 0; #X connect 15 0 4 1; #X connect 15 0 16 0; #X connect 18 0 10 0;