Since I don’t have that much experience with HDSLRs, I don’t know if you can shoot on automatic settings. If you can, then yes, the camera is probably boosting the ISO, which causes more noise.
If you’re shooting with full manual control, there are certain ISO settings that are better than others. Use ISOs that are multiples of 160.
Also, the 24-105 is an F4.0 lens. Maybe try using a Canon 50mm F1.2
