I live at 7000 and have flown quads at up to 10,000 ft. I've never flown them at lower elevations so I don't know what I'm missing. I notice a little less thrust at 10k.
I recently bought a 250 racer RTF and suspect it was setup at sea level. It came with 5030 x 3 props, I replaced them with 6045 x 2 and it flies much better and doesn't sound like it's working as hard. Flight time is also longer.