The band structure of matter determines its properties. In solids, it is typically mapped with angle-resolved photoemission spectroscopy, in which the momentum and the energy of incoherent electrons are independently measured. Sometimes, however, photoelectrons are difficult or impossible to detect. Here we demonstrate an all-optical technique to reconstruct momentum-dependent band gaps by exploiting the coherent motion of electron-hole pairs driven by intense midinfrared femtosecond laser pulses. Applying the method to experimental data for a semiconductor ZnO crystal, we identify the split-off valence band as making the greatest contribution to tunneling to the conduction band. Our new band structure measurement technique is intrinsically bulk sensitive, does not require a vacuum, and has high temporal resolution, making it suitable to study reactions at ambient conditions, matter under extreme pressures, and ultrafast transient modifications to band structures.