I think you can go too large if you don't have enough weight.
For example... I built a DIY 860mm 3.4Kg octo to carry a Sony RX100. Even though I was using a HoverflyPro controller (which is damn good when it comes to gain setup), it flew well but did move around a bit and felt light in the...