Yes, the rotation limits do not work with FBBIK. Strict limitations of bone rotations are not good for animated characters in general, you'd see the bones jump from one end of the limit to the other quite frequently and they would not even guarantee hands not going into the body as combinations of valid bone rotations might still end up with invalid hand positions. It is possible to apply the rotation limits on top of FBBIK though. You'd have to make a script that disables all the RotationLimit components and calls rotationLimit.Apply() on them in LateUpdate. Then assign that script to a higher value in the Script Execution Order than FullBodyBipedIK has.
If the problem is related to the elbows getting into the body, it can also be prevented by assigning a "Bend Goal" to the arm and setting "Bend Goal Weight" to a value higher than 0.