I'm surprised you're having that sort of poor connection with everything on fibre optic cable. I'm with Virgin getting 60Mb download speed and a TiVo box that rarely lets me down. I'd suggest calling Virgin, telling them how unhappy you are and why, and requesting tests are run on your line. I know they replaced broadband amplifiers in the box near me last week because there are some slowness issues about. If you don't tell them, they won't know it's broken and can't fix it. Also, do you have the old Virgin modem or the newer hub? If the former, you can get a free update to the newer hub which supports higher download speeds.
With regards to BT, it will very much depend on how far you are from your nearest exchange - you must call BT in advance and ask what broadband download speeds you can expect.