Re: Use gpsd and ublox-ros driver at the same time

[Marco Camurri]

On 20/09/2022 18:40, Gary E. Miller wrote:

> Yo Marco!
>
> On Tue, 20 Sep 2022 14:50:03 +0100
> Marco Camurri <marco.camurri@outlook.com> wrote:
>
> > > > The main idea would be that gpsd provides updates to chrony, which
> > > > in turn provides updates to linuxptp4l.  
> > > >           
> > > https://gpsd.io has many hoowto's on how to sync you system to GNSS.
> > > Getting linuxptp4l to work is out of scope here.  
> > >         
> > Yes my focus is on the access to the receiver's data from both gpsd
> > and the driver or from one program that provides both functionalities.
> >       
> >From what I can tell gpsd and "the driver" are both trying to do the
> same thing, in different ways.  I do not know any programs that
> speak ROS, so I have no ideas on "one program".
>     
    I meant having both functionalities into a single program, like a
    modified gpsd.
> > > https://gpsd.io/gpsd-time-service-howto.html
> > >
> > > We cant help you with other peoples instructions, and certainly
> > > would not recommend a Garmin receiver.  
> > >         
> > Yes that was just one example I found.
> >       
> Was that one not sufficient?  There are many others.
>     
    I meant that the garmin tutorial was just one of the many I've
    found. I wasn't specifically trying to replicate what was in that
    tutorial.
    
    The one you linked looks pretty useful, thanks for point me to it.
    
> > > > and finally
> > > > configure PTP to use chrony as source.  
> > > >           
> > > What you do is sync PTP to your system clock, and leave chrony out
> > > of it.  
> > >         
> > if there's a plug and play way to do this I'm happy to leave chrony
> > out. I just found this pipeline online.
> >       
> Your "driver" will fight all other solutions.
>     
    I might be missing a point here, possibly related 
> > > > Because there can't be two programs accessing the same serial
> > > > port, I can't run gpsd and the ublox driver at the same time, so
> > > > I'm looking for a minimally invasive solution (i.e. with smallest
> > > > code development possible) to let the ROS driver process the data
> > > > from the receiver while it is being used by gpsd.  
> > > >           
> > > gpsd can pass on the raw data from u-blox, but changing that
> > > program to handle it, will be a lot of work.  
> > >         
> > More work than editing gpsd to send data over ROS?
> >       
> Since no one here knows anything of ROS, we'd stick with known working
> solutions.
>     
    Yes obviously, for me it was the opposite (I'm familiar with ROS,
    less familiar with gpsd)
> > I didn't want to 
> > "pollute" the gpsd which I'm more unfamiliar with than the ROS driver.
> >       
> gpsd is already polluted with many things.  What would an ROS driver
> need to do?
>

ROS is a collection of libraries with standardized inter-process communication and message generation mechanism.

Integrating ROS functionalities into gpsd would make it dependent on many libraries, as it is not as lightweight as LCM is, for example.

> > I know the ROS driver is not perfect, but it is small
> >       
> Which in gps-land means a lot of special cases are ignored.  And it
> is not "small" when it pulls in a lot of old libraries.
>     
    yes I agree, I just meant that the code for the driver itself is
    shorter than gpsd's is. 
> > and I need to
> > get that data via the ROS interface for compatibility with other
> > programs.
> >       
> Out of our area of knowledge.
>
> > > The next bigger problem is that both gpsd and that thing want to
> > > reprogram the u-blox.  That's not gonna work.  
> > >         
> > Yes but this might be solved by finding the right combination of 
> > arguments for gpsd to replicate the settings that the ROS driver will 
> > want to put.
> >       
> gpsd by design avoid options.  Don't bother looking for knobs that do
> anything useful for you.
>

This is actually fine because we are not really using the portion of the driver that configures the receiver. We just take data from it.

The configuration is done from the official ublox program on windows beforehand and stored permanently.

> > Basically the only setting we might want to actually
> > change is the output frequency.
> >       
> My brief look at the "driver" code makes me think otherwise.
>     
    Same as above, all settings are not done via the driver in practice.
> > > The next big problem is that code wants to control the (badly!) the
> > > system clock.  
> > >         
> > That's a optional standalone program which we will not run.
> >       
> Not what I think I saw, can you point me to info on that?
>

This is the program I meant:

https://github.com/HKUST-Aerial-Robotics/ublox_driver/blob/main/src/sync_system_time.cpp

it's a standalone program, not part of the driver. If I run it, it breaks my PTP.

Did you see any other places where the system clock is touched?

> > > The next big problem is they barely decode any of the u-blox data.  
> > >         
> > But that's the only data we actually need.
> >       
> Not what I thought I saw.  But this is not the place for me to critque that
> code.
>     
    Absolutely agree
> > > > I know from the FAQ that gpsd does not provide a way to extract raw
> > > > data directly, but I saw a low level API is provided.  
> > > >           
> > > Then the FAQ is out of date.  Which types of "raw" data?  Where did
> > > you see that?  
> > >         
> > So by raw data I mean: pseudoranges, carrier phase observation,
> > doppler measurements, and ephemeridis.
> >       
> Why duplicate what the u-blox already does internally?
>

This way it is possible to tightly fuse the raw data from the GNSS with other sensor data (IMU, camera, etc.), instead of treating the ublox as a black box.

If you are interested, you can have a look at this video from the same developers of the driver, which explains the concept:

https://www.youtube.com/watch?v=TebAal1ARnk&ab_channel=HKUSTAerialRoboticsGroup

> > I read the pseudoranges are not available from this FAQ page: 
> > https://gpsd.gitlab.io/gpsd/faq.html#almanac
> >       
> You misread that:
>
> "Sorry, there's no easy way to do these things through GPSD yet."
>
> Key words: "easy way".  What the "driver" does is the "hard way".
>     
    I understand now, thanks.
> > > If you can figure out what the "ROS messages" are, it migh be
> > > easier to patch gpsd to send those.  
> > >         
> > So the data we need to extract from the receiver is a vector of 
> > observations. Each one of them is defined here:
> >
> > https://github.com/HKUST-Aerial-Robotics/gnss_comm/blob/main/msg/GnssObsMsg.msg
> >       
> You misunderstood my question.  I asked about the "RDS messages", you replied
> with what you want from the receiver.  ROS != receiver.
>     
    The link above refers to the actual definition of the ROS message.
    
    The message is automatically converted into a data structure via a
    code generator (e.g. a C++ class) and serialized for data
    transmission over TCP in a publisher-subscriber fashion.

A ROS-compatible program can subscribe to a topic where the message is published to, receive the message via a callback and process it. More info here: http://wiki.ros.org/msg

Similar libraries that do that are LCM or protobuf.

So to modify gpsd to send the ROS messages, I'd need to access all the data I've listed below from inside gpsd, then stick them into the ROS data structure and call the "publish" method from the ROS libraries.

> > Each observation includes:
> > - measurement time,
> > - satellite ID,
> > - observed frequencies,
> > - carrier-to-noise density ratio (signal strength) [dB-Hz],
> > - lost-lock indicator (1=lost),
> >       
> In practice, you can't count on this one.
>
> > - channel code,
> > - pseudorange measurement [m],
> > - pseudorange standard deviation [m],
> >       
> No receiver provides that.
>
> > - carrier phase measurement [cycle],
> > - carrier phase standard deviation [cycle],
> >       
> No receiver provides that.
>
> > - Doppler measurement [Hz],
> >       
> Never needed.
>
> > - Doppler standard deviation [Hz]
> >       
> No receiver provides that.
>

I'm confused: the ROS driver provides all this data already, so the receiver must be transmitting them over serial.

I'm already using the driver and it works, it is simply out of sync with the host computer.

> > the data captured from serial is converted from binary into a cpp
> > data structure here:
> >
> > https://github.com/HKUST-Aerial-Robotics/ublox_driver/blob/64f39fa54a4405f0bcf90387002ca12e95627a21/src/ublox_message_processor.cpp#L50-L135
> >       
> That code will only work on a u-blox 8.  Not a 7, not a 9.  So, the
> bitrot is already begun.  Fix it, or ditch it.
>     
    The code above already works for the u-blox ZED-F9P receiver and
    this is the only one we are interested in using.
> > My initial idea was to call these functions on binary data provided
> > from gpsd instead of serial directly, as it looked like the less
> > invasive way to do it.
> >       
> gpsd will decode those same messages and provide you the data as JSON.
> But it looks like the data is used at u-blox 8 scaling, which makes it
> non-portable.
>     
    I'm not sure I understand what u-blox 8 scaling is, but I'm fine to
    handle the ZED-F9P only atm. 
> > But, if the same data is already decoded by gpsd I'm fine to do the 
> > opposite (i.e. call the above functions from within a custom version
> > of gpsd).
> >       
> Your call, I'm not touching that.  All that work, just to duplicate
> what the u-blox 8 already does inside.
>     
    For the reason explained above, low level raw data fusion with other
    sensors.
> > If I could get just a pointer or a hint on where these conversion 
> > operations are carried out from inside gpsd for the ublox that would
> > be really helpful.
> >       
> No need, just look at "man gpsd_json" for the decoded data that gpsd
> will give you.  Look in drivers/driver_ubx.c for how gpsd decoded
> u-blox messages.
>     
    Thank you, I'll look into that.
> RGDS
> GARY
> ---------------------------------------------------------------------------
> Gary E. Miller Rellim 109 NW Wilmington Ave., Suite E, Bend, OR 97703
> 	gem@rellim.com  Tel:+1 541 382 8588
>
> 	    Veritas liberabit vos. -- Quid est veritas?
>     "If you can't measure it, you can't improve it." - Lord Kelvin
>