2013 MRF Flight Control Assessment

[Bartman]

The attached .pdf was part of a project started earlier this year to assess the state of flight control development. The intention was to have myself and the two moderators score the different flight controls with each score individually listed (X, Y, Z) for each column. Needless to say the idea didn't go very far. If someone knows how to format a spreadsheet so it can be posted please do so. I've posted it as a .pdf in order to be able to post it at all.

Please feel free to offer your assessments of each system and if we can establish an order of some sort we'll build the review around our users' comments.

SEE POST #11 FOR LATEST VERSION

 

[R_Lefebvre]

Interesting work. You have missed AutoQuad. It's not as well known, but seems to perform quite well and is Open Source.

I think SOD of 3 for APM is a bit low. 4 is more appropriate, IMO. Obviously this is somewhat subjective. But we have users flying quads without touching a single tuning parameter (ie: out of the box performance), while also allowing more functionality than anything else.

I'm not sure what the difference is between Attitude and A/L flight modes? They seem one in the same? Do you intend that Attitude=Rate Control, and A/L=Attitude Control? Or do you mean that A/L is more like inertial control (ie: position hold).

Now for the completely biased part

I'd rank Attitude-A/L... hard to judge, as they all seem to be very good these days. Arducopter does not have a good acrobatic mode at this time. It's not going to satisfy people looking to emulate Warthox.

Altitude Hold, again, right up there. It's hard to imagine it being better. So 4/5.

IOC, yep, it's got it. Works well.

Onboard OSD? Guess it depends on what you mean by Onboard. You have to buy a separate board, for not a lot of money, but that board communicates directly with the APM to get data. It's not a stand-alone system.

Onboard failsafe? Yep. Probably more advanced than others. You can completely sever the Rx connection, and it will respond in a programmed way, very configurable. Climb to set altitude. Fly back. Hold for a set time. Then descend to a set altitude or a full auto landing.

Wind Tollerance. Again, hard to judge, but hard to imagine it being better.

GPS-PH? Yes. With 2.9.1, it's a maybe a 3. With 3.0, it will be 5 I think. Within the region of error of the GPS system, which will affect all of them.

RTH, yep, 5 I think due to the stated functionality.

IRTH? Yep.

Waypoints? Yep, 100+

Follow me? Sort of. It's supposed to be there, but I'm not sure the state.

Part of the problem with an undertaking like this is selection bias. I try to be objective, of course anybody could make a valid claim that I'm not. Then you've got other people continually bagging on Arducopter, even though they have minimal experience with it, and that a long time ago. Then you have other people claiming there's is the best, only because they spent a lot of money on it, trying to justify it to themselves.

What's really needed here, is a fair competition of some sort. Line them all up, same place, same time. Thrash it out. I just got back from Sparkfun Autonomous Vehicle Competition. Arducopter dominated. AutoQuad put in a good showing too. Both were capable of achieving perfect scores. Arducopter did it much faster than AutoQuad, but it's not clear if that was hardware or software. They only brought one machine, and it was a large Octo. One of our guys brought a hotrod Quad, capable of 30 m/s (108 km/h) flight. That's 100+ km/h, in AUTO mode. Dropping a tennis ball on a target 400 m away, 4 times in a row, and flying through a 20x10m wicket, 4 times in a row, without fail, at high speeds.

So that's what we're talking about here, in terms of performance. 100+ km/h in Auto. I'm not kidding.

 

[Bartman]

Thanks for the reply RL. I was wondering if anyone saw the original post!!

Everything with this stuff is subjective as there aren't any objective metrics by which to compare but, in your case, your comments justify your ratings so at least they can be considered with the rest of the info.

As for the SOD rating, this was all done earlier this year and then it sat. As it's posted is simply as I sent it to the other moderators for input. It was supposed to be a format (It got complicated in my attempt to be thorough) that we could all contribute to. The rating of 3 for SOD had to do with it being more of an unknown at the time and still something of a mystery. 3DR has made great strides to remedy that in just the last few months so SOD might be rated higher with a new revision of the assessment.

To be fair, the moderators are fully employed adults with many activities of their own so it shouldn't be a surprise that a chore dropped onto them by me went without their full attention. No harm, no foul. I do, however, fully encourage everyone's comments and I'll try to keep updating the assessment as I have time and the comments to work with.

Looking at the chart as it's posted, it was a challenge to make a format where they could even all be put on one page in an orderly fashion. THe many notes reflect the similar but different nature of things.

As for attitude vs. auto-level, attitude is manual control where the pilot is responsible for flight attitudes and attitude corrections/recovery. Auto level means the flight control seeks level flight attitudes whenever possible such as when control inputs are centered in the midst of maneuvering flight.

Bart

 

[Bartman]

............I think SOD of 3 for APM is a bit low. 4 is more appropriate, IMO. Obviously this is somewhat subjective. But we have users flying quads without touching a single tuning parameter (ie: out of the box performance)............

I'd be willing to debate this one line as the product review we did wasn't dead on out of the box. It was flyable but not acceptable. The requirement to tune via PID's is something that might prevent a flight control from being considered a consumer grade product, imho, as PID tuning can be time consuming and beyond the ability of average hobby users as well as costly for an employer paying hourly rates to their builder/operators or a commercial user who is also the business owner saddled with a great variety of responsibilities.

This is my opinion as a forums guy trying to be objective for sake of those who might seek guidance from the document.

 

[Stacky]

Reliability needs to be considered however thats a tough one given that so many faults or problems end up not being the FC but users not RTFM.

 

[Bartman]

reliability might be implied from the QOM rating? Or can a quality item be unreliable? maybe it would be a good column to add......hmmmm



keep the commentary coming, this is good stuff and beneficial for everyone if we can keep collecting and correlating the info.

 

[Stacky]

Do we need to look at the level of depth of tuning options?
Open source etc?

 

[R_Lefebvre]

I'd be willing to debate this one line as the product review we did wasn't dead on out of the box. It was flyable but not acceptable. The requirement to tune via PID's is something that might prevent a flight control from being considered a consumer grade product, imho, as PID tuning can be time consuming and beyond the ability of average hobby users and costly for an employer paying hourly rates to their builder/operators or a commercial user who is also the business owner saddled with a great variety of responsibilities.

This is my opinion as a forums guy trying to be objective for sake of those who might seek guidance from the document.

Which code did you fly? Our attitude system didn't get really good until 2.8, which was last fall sometime. But still, tuning of PID's is, and probably always will be required. I actually can't conceive of how it's possible to not tune PID's on any system, unless the performance of the system is incredibly dumbed down. It's always possible to have an adaptive self-tuning system, but I've never seen a demonstration of a system like this that actually works well.

If somebody has a no-tuning-required system, that can match the dynamics of ours, I'd love to see a demonstration of that.

I also don't really (still) understand the difference between attitude control and auto-level. For us, it's one in the same. There's no differentiation. In stabilize mode, if the stick is centered, it means "go to level". If you push it forward 25° and hold it, it means "go to 25° pitch down attitude and hold there". Are other systems different? I suspect some of them have a different setup, where, the gyros are constantly fighting to stop it moving, and the pilot's commands over-ride the outputs, but you're fighting the gyros. And the A/L is something else? We just don't work that way. When you push the stick, the entire system, gyros included, try to move the aircraft to the input attitude, and hold it there.

This is what allowed me to manually fly my 450 heli back from 400m away when my first run was aborted at AVC. Think about that. Flying a 450 heli manually, at 400m. The only thing I could see was a white dot, indicating the canopy was facing me, and I could judge the altitude. But that's it. All I had to do was keep the nose pointed at me, and push the stick forward 25°, keep it off the ground, and it came back.

 

[Bartman]

in attitude (aka manual) mode you set the attitude of the heli with a stick movement. when you release the stick the heli stays at that attitude until you manually adjust the attitude to something new. if you want level you have to fly it to level and it's your responsibility to maintain level, the flight control just keeps it wherever you put it. it's a great mode for doing video as it allows very precise control and very predictable response from a well tuned heli. obviously you have to be very familiar with the heli to get the most out of this mode.

fwiw, Hoverfly has a very simple tuning technique that isn't dependent on PIDs. I walked a friend through the process while he had the phone against his ear as he was flying the heli and we were done in a matter of minutes.

 

[R_Lefebvre]

in attitude (aka manual) mode you set the attitude of the heli with a stick movement. when you release the stick the heli stays at that attitude until you manually adjust the attitude to something new. if you want level you have to fly it to level and it's your responsibility to maintain level, the flight control just keeps it wherever you put it. it's a great mode for doing video as it allows very precise control and very predictable response from a well tuned heli. obviously you have to be very familiar with the heli to get the most out of this mode.

fwiw, Hoverfly has a very simple tuning technique that isn't dependent on PIDs. I walked a friend through the process while he had the phone against his ear as he was flying the heli and we were done in a matter of minutes.

Ok, now I understand. What you're talking about is rate control. APM can do that, but it's not the primary use so it's weak. Warthox wouldn't like it. I'd rate that a 3, but A/L is a 4 or 5. I'm working to make it better though.

I'd love to get a quick sketch of how HF tuning works.

 

[Bartman]

View attachment 12302


Made some revisions and cleaned up some of the details.
Bart

 

[StillHappy]

I noticed in the pdf "DJI WKM 8 motor control available with Futaba SBUS receiver only" is that right? I see several rtf octo's online for sale with other radios.

 

[Bartman]

I noticed in the pdf "DJI WKM 8 motor control available with Futaba SBUS receiver only" is that right? I see several rtf octo's online for sale with other radios.

that's the way I understand it. Maybe there's an SBUS emulator or something like that? if someone can clarify it I'm happy to revise the assessment.

 

[Droider]

that's the way I understand it. Maybe there's an SBUS emulator or something like that? if someone can clarify it I'm happy to revise the assessment.

Not correct. You can use standard radio with out Sbus. I if you want to use WKM for gimbal control as well as 8 motors then you would have to use sbus..

Dave

 

[Bartman]

Thanks Dave! I'll update the spreadsheet to reflect that.

Would you like to offer some scores for the WKM and NAZA systems?

 

[Droider]

I only flew my Naza a few times before it was consumed!

As for voting
WKM 9
Naza 6.5

Dave

 

[Droider]

Just seen othe options.. is there a google doc somewhere?

I guess the numbers I have quoted are across the board but I cannot comment on the newer firmware on either. Would I buy a Naza again.. probably not, would I buy WKM again.. certainly they are the backbone of my commercial operations.

 

[SMP]

Hey Bart, lot of work there well done! Any chance the cost for complete system will be populated? The columns I used were principally SOD and Wind Tolerance. I'm wondering what it would look like if a couple of the experienced guys who have flown several ranked their top 3 in order OR had a "feel" type column which is more subjective. It seemed like most of these had fairly comparable Feature sets, fairly level QOM and fairly level Wind Tolerance. Without price or "Feel" I'm still not sure which one I should try as i search for DJI alternatives for my next bird. PS, thx for the work on this this!! Super comprehensive!

 

[hoggdoc]

"Consumed"?

 

[Bartman]

Hey Bart, lot of work there well done! Any chance the cost for complete system will be populated? The columns I used were principally SOD and Wind Tolerance. I'm wondering what it would look like if a couple of the experienced guys who have flown several ranked their top 3 in order OR had a "feel" type column which is more subjective. It seemed like most of these had fairly comparable Feature sets, fairly level QOM and fairly level Wind Tolerance. Without price or "Feel" I'm still not sure which one I should try as i search for DJI alternatives for my next bird. PS, thx for the work on this this!! Super comprehensive!

Thanks.

What are you trying to accomplish that you feel can't be done with DJI?

What are your long term goals as a user? There's a big difference between in-close aerial media work and far off aerial survey or mapping. Features wise, sport flying or FPV are very different in terms of flight control needs compared to trying to outfit a big 2-man rig. I know you're buying some RTF systems for aerial work but what kind of media work will you be doing?