AimIK should work accurately regardless of the animation.
Is this a third person game? In that case the camera forward and the gun barrel are not aligned as you look from an elevated position, not looking through the scope of the gun. So if you see the crosshair in the middle of the screen, the character seems to be aiming lower if the object your crosshair is on is close.
What you should do is make a raycast or spherecast from camera towards camera forward and set AimIK target to what it hits (where the line from camera to crosshair hits the ground on the diagram).
If that's not the issue, maybe you have another IK component that updates after AimIK and messes up your aiming?