Your speed from your ISP to you is 650mbps. The speed, on the internet, from the Zoom server to your ISP can be anywhere from 0bps (no connection) to about 6–7mbps (the speed that the internet is running in May 2021).
If the route between the server and your ISP is terrible, your connection to the server is terrible. (It’s like not being able to go faster than 5mph in a Porsche in the rush hour. A Porsche can only go 5mph? Yes, when the car in front of it isn’t going any faster.)