Device-measured physical activity and sedentary time are suggested to be more important determinants of all-cause mortality compared to body mass index (BMI) in mainly older adults. However, the joint associations of physical activity and sedentary time with BMI in relation to mortality risk in relatively healthy middle-aged individuals are unclear. We followed 770 adults (56 % women, mean age 55.6 years) from a population-based cohort study for up to 15.3 years. BMI categories were combined with tertiles of total, light, and moderate-to-vigorous physical activity and sedentary time. Cox proportional hazards models estimated hazard ratios (HR) of all-cause mortality with 95% confidence intervals (CI). High total and light intensity physical activity and low sedentary time were associated with a lower risk of mortality in normal weight individuals compared with low active overweight/obese; HR: 0.35 (CI: 0.14, 0.86), HR: 0.33 (CI 0.12, 0.89) and HR: 0.34 (CI: 0.13, 0.92). Among overweight/obese individuals, those who were medium active in light physical activity had a lower mortality risk, HR: 0.36 (CI: 0.15, 0.83), compared with low active. Medium sedentary individuals had a lower risk, HR: 0.43 (CI: 0.20, 0.94) compared with those who were most sedentary. Associations among the most active or least sedentary tertiles were similar irrespective of BMI category. In conclusion, higher physical activity and lower sedentary time were associated with lower mortality risk irrespective of BMI. Physical activity should be promoted and prescribed to individuals with low physical activity levels irrespective of weight status.