Purpose Breast cancer incidence rates are now higher in France than most other European countries as well as the United States (US). Increasing breast cancer incidence rates globally have often been attributed to declining fertility rates. Methods We compared temporal trends in breast cancer incidence in France and the US, and examined the extent temporal trends in national fertility rates can explain the temporal trends in breast cancer incidence. This study of temporal trends used estimates of annual percent change (APC) from cancer registry data in France and the US (1978–2016) and national fertility data (1958–2011). We estimated the APCs for all ages (overall APC) and for specific age groups (under 50, 50–64 years, and 65 years and over). Results The overall APC was over three times higher in France than the US (France APC = 1.63%, 95% CI 1.43–1.84; US APC = 0.51, 95% CI 0.31–0.72). The overall APCs remained positive and statistically significant after adjusting for fertility trends irrespective of assumptions on fertility lags (France APC = 1.61–0.91 for a 5-year to 20-year lag, respectively; US APC = 0.37–0.36 for a 5-year to 20-year lag, respectively). Similarly, among women under 50, the APC was over 3.5 times higher in France than the US (France APC = 1.22, 95% CI 1.07–1.37; US APC = 0.33, 95% CI 0.22–0.44), and APCs remained positive after adjusting for fertility (France APC = 1.21–1.28 for a 5-year to 20-year lag, respectively; US APC = 0.38–0.26 for a 5-year to 20-year lag, respectively). Conclusions Based on these trend analyses, changes in fertility rate trends do not fully explain the increase incidence in breast cancer seen in both France and the US, nor the magnitude of difference between the two countries. This was seen overall and in age-specific groups.