In this study we conducted saturated columns experiments packed with sand and glass beads to quantitatively examine surface roughness effect on deposition and release of micro- and nano-sized colloids at different solution ionic strengths. Experimental results showed more colloid retentions in both primary and secondary energy minima in sand than glass-bead columns, especially at high solution ionic strength (e.g., >0.01 M). This observation cannot be explained by the classic Derjaguin–Landau–Verwey–Overbeek (DLVO) theory, which assumes sphere-smooth surface configuration. We modified the Derjaguin approximation approach and calculated interaction energies, which indicate that the sharp asperities on sand surfaces can facilitate colloid deposition in primary minima by reducing the energy barrier. In addition, the increased attachment in secondary minima in sand columns can be attributed to the presence of the valleys on sand surfaces where colloids associated at secondary minima can be shielded from hydrodynamic shear. Additional theoretical analysis verified that large valleys can locally increase the energy barrier as well as the secondary-minimum depth, hence, are favorable for colloid deposition in secondary minima. Whereas the reduction effects of surface roughness on energy barrier has been extensively addressed in the literature, our modified DLVO analysis and experimental results demonstrate that this mechanism is only effective at high ionic strength for large colloids (e.g., >0.01 M for the 1156 nm colloid in this study). We provided experimental evidence and theoretical demonstration that surface roughness also plays an important role in colloid deposition at secondary minima under unfavorable conditions. Our study provides a more complete understanding on the effect of surface roughness on colloid deposition.