This study monitored 470 university students’ smartphone usage continuously over two years to assess the relationship between in-class smartphone use and academic performance. We utilize a novel dataset where smartphone use and grades were recorded across multiple courses, allowing us to examine this relationship at the student level and the student-in-course level. In accordance with the existing literature, we find that students’ in-class smartphone use is negatively associated to their grades, even when controlling for a broad range of observed student characteristics. However, the magnitude of the association decreases substantially in a fixed effects model, which leverages the panel structure of the data to control for all stable student and course characteristics, including those not observed by researchers. This suggests that the size of the effect of smartphone usage on academic performance is overestimated by studies that only control for observed student characteristics.