Unless it was darn small and fluffy, it seems like the object's gravitational influence would have been noticed long before now.

Might there be an astronomer in the house, to comment?