Hi Refik,
the question of which filter is better is not easy to
answer. This strongly depends on your needs and your setup
(airframe, IMU, frequency, vibrations, etc.).
Also float_dcm is implemented in float and
int_cmpl_quat (mostly) in fixed point and the latter
should hence be more efficient. While you can run
float_dcm at 60Hz for a fixedwing on a board without FPU,
I don't think it would run at 512Hz (which is not a
problem for int_cmpl_quat).
Unfortunately I can't provide a quantitative comparison
of the two...
But since you were experiencing problems with float_dcm
in your setup, I would recommend to test int_cmpl_quat.
(Of course first try better mechanical damping as
others mentioned before).
Regarding the "correction" by accelerometer
measurements in such simple attitude only filters:
They assume that (on average) you measure only
acceleration due to gravity, which in turn gives you
roll/pitch measurements.
Of course this is not quite true while you accelerate
(includes flying circles!).
For quadrotors this is not quite as bad as for
fixedwings, since the assumption that these trajectory
accelerations average out to zero is good enough in most
cases.
But fixedwings can't hover and instead have to
constantly fly circles if you want to "stay" at one spot.
This means a constant trajectory acceleration to keep you
on the circle!
Now for fixedwings these filters use the GPS speed
together with the assumption that you are flying forward
or a coordinated turn (no significant side slip) to
compensate for this centrifugal force.
For int_cmpl_quat this is enabled with
AHRS_GRAVITY_UPDATE_COORDINATED_TURN (which is set by
default if you use the fixedwing firmware).
Now regarding gravity_heuristic_factor in
int_cmpl_quat:
This heuristic is meant to temporarily decrease the
"correction" from the accelerometer measurement while you
have trajectory accelerations by scaling down the
correction proportional to the difference of the
measurement norm to the "assumed" 9.81m/s² (acceleration
due to gravity).
If you have clean measurements this works really well.
But if you have vibrations, using the gravity heuristic to
reduce the accel correction might be counter productive
since the norm of the measurement might always be way off
compared to the 9.81m/s² even if you are still on ground.
So I would recommend to test this on the bench/ground
first. Just increase throttle and watch either the accel
measurements directly or look at the resulting "weight" in
the AHRS_QUAT_INT message. It should stay close to 1 (no
reduction) and go down significantly. A (continuous)
acceleration though should lower the weight.
If the vibration turns out to be problem in your case,
add the ahrs_int_cmpl_quat settings file and reduce the
gravity_heuristic_factor to e.g. 10 and check what
difference it makes.
If you have a lot of vibrations, you can try to
increase the FIR_FILTER_SIZE in
ahrs_int_cmpl_quat.c Line 281. This
is a really simple FIR lowpass filter on the accel
measurements used for the gravity heuristic.
Use GPS course or magnetometer to estimate heading?
Again there is no easy answer to this... strongly
depends on what kind of airframe (and in which weather
conditions) you fly.
As a rule of thumb, heading estimation via GPS course
usually works well for stable airframes flown at a steady
speed with little side slip and in nice weather. If you
start getting significant side slip or the airframe is
_very_ agile, it might be a bit problematic. Also ground
speed dropping close to zero (e.g. against strong wind)
becomes a problem.
Using a magnetometer does not have the above problems.
BUT it needs to be calibrated and ideally this would be
constant (i.e. no induced magnetic fields due to motor
current, etc.). Also see
ImuCalibration: Calibrating_for_Current on
that point.