[gpsd-users] Recent changes in GPSD

[Eric S. Raymond]

Those of you following the repo commits will have noticed that there
has been a lot of activity lately.  Much of this is a result of a
week-long consulting gig I just finished.  The work was centered on
some major improvements to the UBX driver, but it had ripple effects.
All three of gpsd, gpsctl, and gpsmon got serious rewrites of some
core portions.

Here's why.  uBlox devices have an awkward capability for switching 
serial paramaters (baud rates, parity & stop-bits) and reporting modes
(UBX binary or NMEA). What's awkward is that you can't change any of these 
alone.  The only way to change any of them is to change all of them 
at once using a control packet called a CFG-PRT that has a 20-byte 
payload full of magic bits in weird encodings.  

Figuring out how to generate the entire 20 bits from scratch is hard,
because uBlox's documentation of CFG-PRT is *awful*. It's so bad that
the original author of the UBX driver (Chris Kuethe, I think) quite
justifiably dodged the problem.  The UBX can be asked to emit a
CFG-PRT describing its settings; you can then copy that, tweak parts
that you understand, and play it back to the UBX trusting that the
magic bits you didn't change are correct.

This is what the driver code did until a few days ago - emit a request
for CFG-PRT when event_identified fires, and when the response comes
back and stash it away so the mode change and speed change methods
can later play a modified copy back to the device.

The downside of this is that before the CFG-PRT response arrives these
methods can't do anything, and don't try.  This creates a potential
race condition.  If you open the device and then instantly ship a mode
change request to it, the request may arrive in gpsd's driver before
the UBX has gotten around to shipping up the magic bits it needs to
replay. You lose.

In fact I got my consulting gig precisely because that bug bit a small
but clueful engineering company in the ass.  It was producing sporadic
bugs in their initialization sequence at a low frequency.  The
"clueful" is demonstrated by the fact that it occurred to them to hire
me to try to find the source of the problem quickly - that is, rather
than having one of their in-house guys thrash at the problem for
weeks.

I spotted the problem almost instantly.  But it wasn't obvious how
I could fix it, because making CFG-PRT blocks was still black magic.
(When I said Chris's choice to dodge this problem  was justifiable,
I wasn't just being nice about that.)

I spent a significant amount of time modifying gpsctl in an attempt to
work around the CFG-PRT problem. I won't explain all the reasoning
behind this, because I abandoned the attempted workaround a few days
later.  But while I was in this process I realized that there's a way
to write a generic packet-fragment assembly loop that could be used to
replace the three ad-hoc implementations in gpsd, gpsctl, and gpsmon
with common code.  

Eventually I did that; the last conversion was gpsmon, just yesterday.
This is a major code simplification, the first serious improvement in
the codebase architecture in a while. Look in libgpsd_core.c:multipoll()
to see the implementation.

But.  I abandoned the strategy that motivated me to write multipoll()
when I realized there was no way I could actually eliminate the race
condition with it.  The only way to close the window was to get past the
awfulness of the documentation and figure out how to build CFG-PRT 
payloads from scratch. 

I ended up interrogating uBlox's North American field engineers about
this.  For a wonder, not only were they helpful, they even admitted
that part of the documentation is a shambles rather than getting
defensive about it.  The conversation continues; I've offered to help
them fix the documentation and we'll see where that goes.

I solved the problem.  New code in driver_ubx.c builds custom CFG-PRTs
whenever it needs them.  (As you may imagine, this was a pain in the 
butt to debug.)  No more race condition, driver code simpler and 
better, and a longstanding item on the to-do list (making gpsctl 
able to do UBX mode changes) is fixed.  

Not the least of the good things that happened is that I made $5K for
a week's interesting hacking and the customer is extremely pleased with
me. I expect to get some more work from them.  GPSD making me
money...what a concept!

These are enough changes to merit a release.  But there's another
architectural-level issue that needs to be tackled first.  In my next
post I'll talk about that.
-- 
                <a href="http://www.catb.org/~esr/";>Eric S. Raymond</a>

Democracy is two wolves and a lamb voting on what to have for lunch. Liberty is
a well-armed lamb contesting the vote!
        -- Benjamin Franklin