It seems more a programming problem, than a structural problem, your quad is not correcting errors how it should... seems very slow doing that, and he seems missing D component in PID.
what frequency are you running your PID ? how did you set up your K gain values for your PID?
what algorithm...