The objectives for high precision astrometry on ELTs will be challenging, with requirements in the range from 10 to 50 micro-arc-seconds for some instruments and science cases. Reducing and correctly calibrating the systematic and quasi-static errors introduced by optical surface distortions will be an important part of meeting these goals. In a recently submitted paper, we described an analytical Fourier domain model for evaluating these effects as the sum of three terms: (i) under-sampling errors, due to measuring the effects of static surface distortions using a finite number of discrete reference sources; (ii) unknown beam wander across the static surface distortions due to line-of-sight jitter or boresighting errors, and (iii) quasi-static errors due to slowly varying surface distortions. In this paper, we apply these methods to evaluating this term in the astrometry error budgets for the TMT Infrared Imaging Spectrograph (IRIS) and the facility AO system, NFIRAOS. The inputs to this exercise include the original top-down allocations for this error term, the original optical surface specifications for IRIS and NFIRAOS as derived earlier on the basis of wavefront error requirements, our assessment of the feasible density and positioning accuracy for an array of calibration sources, and the expected beam wander due to tip/tilt jitter and bore-sighting errors between NFIRAOS and IRIS. The astrometry error computed for these initial parameters was considerably larger than the top-down allocation due to the contributions from the NFIRAOS double-pane entrance window, which is close to the system's input focal plane. The error can be reduced to fall within the allocation by defining tighter, but still feasible, specifications for these elements. We also evaluated the astrometry errors due to quasi-static drift of the figures of the NFIRAOS deformable mirrors, and determined that they are acceptable for RMS wavefront distortions of up to about 30 nm RMS.