RE: Reed-Solomon Decoding (Not for Digital Television)

[pwhines2]

Hello Daniel,

I've been experimenting with the RS decoder in gr-satellites for the past few 
weeks and I've had some great results, thanks for pointing me in that 
direction.  However, I'm still running into a few problems -- some directly 
involve gr-satellites and some are just about basic tagging issues.  Here's 
what they are


1. Is there any way to disable the gr::log/debug info from displaying in the 
terminal/grc file?  I'm dealing with bursty transmissions which transmit one 
after another, and occasionally the logs aren't able to catch up and cause my 
program to freeze.  Obviously this isn't ideal, hopefully there's a way to 
disable these from displaying either in grc or in the python file that it 
generates.

2. I'm getting a very strange output from the "Sync and create packed PDU" 
block from gr-satellites.  Here's an example of the output for a single 
transmission, 0xAAAAAAAAAAAA being the header, 0x0123456789 being the payload, 
and 0xFF being the noise around the transmission.

        AAAAAAAAAAAA0123456789FFFF
        AAAAAAAAAAA0123456789FFFFF
        AAAAAAAAAA0123456789FFFFFF
        AAAAAAAAA0123456789FFFFFFF
        AAAAAAAA0123456789FFFFFFFF
        AAAAAAA0123456789FFFFFFFFF
        AAAAAA0123456789FFFFFFFFFF
        AAAAA0123456789FFFFFFFFFFF
        AAAA0123456789FFFFFFFFFFFF
        AAA0123456789FFFFFFFFFFFFF
        AA0123456789FFFFFFFFFFFFFF
        A0123456789FFFFFFFFFFFFFFF
        0123456789FFFFFFFFFFFFFFFF
        123456789FFFFFFFFFFFFFFFFF
        23456789FFFFFFFFFFFFFFFFFF
        FFFFFFFFFFFFFFFFFFFFFFFFFF
        FFFFFFFFFFFFFFFFFFFFFFFFFF
        FFFFFFFFFFFFFFFFFFFFFFFFFF
        FFFFFFFFFFFFFFFFFFFFFFFFFF

As you can see it creates multiple tags for a singular transmission, I think it 
has to do with my header being the same character throughout.  It then prints 
out the packet multiple times with the beginning trimmed by half a byte.  My 
syncword is just 0xAAAAA and my threshold is 4, which sort of makes sense that 
it prints out more than one packet, but it this is far more strange.  I'm sort 
of stuck in this scenario since my headers will also have varying lengths for 
different packets, ranging from 5 bytes to 8 bytes.  No matter how I change the 
threshold or the length of the syncword it still tags a packet multiple times, 
how can I get this block to do it only once?

3. Finally, how would I remove the excess noise around the packets the "Sync 
and create packed PDU" block is outputting?  I imagine it might have something 
to do with the Header/Payload Demux block...?  When I remove the squelch on my 
flowgraph the RS decoder tries to correct the noise instead of the packet 
(which also causes the program to freeze because the logs can't catch up).  
Eventually I want to get rid of the squelch but as of now I'm stuck with it 
until I can only pass these packets through and nothing else.


Once again I thank you for all the help you've given me, I'm very close to 
getting the output I want but there's still a few issues with my flowgraph.  
Any help would be greatly appreciated.

Thanks,
Patrick



-----Original Message-----
From: Daniel Estévez <daniel@destevez.net> 
Sent: Wednesday, December 15, 2021 12:14 PM
To: pwhines2@verizon.net; discuss-gnuradio@gnu.org
Subject: Re: Reed-Solomon Decoding (Not for Digital Television)

Hi Patrick,

Typically you would have synchronization words at the beginning of your 
Reed-Solomon codewords (this depends on the protocol you're dealing with). You 
can use the "Sync and create PDU packed" block from gr-satellites to detect the 
synchronization words and obtain PDUs with the RS codewords. You could also use 
"correlate access code - tag" and the "Tags to PDU" block from gr-pdu_utils.

In any case you need to perform some alignment to the codewords. Shoving the 
output of the binary slicer directly into a Reed-Solomon decoder is not 
possible.

I guess it's mathematically possible to find the RS parameters given some 
codewords, since the RS parity check bytes (known) are just the remainder of 
the division of the message polynomial (known) by the code generator polynomial 
(unknown). I think the problem is the same as reverse-engineering CRCs. 
Basically you know that the code generator polynomial must divide the codeword 
polynomial, must split over GF(2^8) (assuming the field used is GF(2^8); if you 
don't even know the field things get considerably more difficult), and must 
have a certain degree. 
So once you see a bunch of codewords you'll be able to constrain the code 
generator polynomial enough that only one possibility remains.

Usually you would have some documenation describing the Reed-Solomon code, so 
you don't have to do any reverse-engineering mathematical tricks.

Best,

Daniel.

El 15/12/21 a las 3:04, pwhines2@verizon.net escribió:
> Hello Ron and Daniel,
> 
> First of all I appreciate the replies, you've given me some great starting 
> places for this, so thanks.
> 
> I guess my next question would be how to implement this into my existing 
> flowgraph given the output of my demodulator and the input of the blocks.  
> Right now I have my GMSK signal being demodulated with the final block being 
> a binary slicer, which outputs a constant stream of binary data (working on 
> understanding the correlate access code to remove noise between bursts).  How 
> would one go about converting the data from a byte stream to a stream vector 
> or PDU?  I would need to convert the signal back to a byte stream afterwards 
> for my TCP/File sinks as well.  Right now I keep getting an error saying the 
> source IO size doesn't match the sink IO size, which I believe Daniel 
> mentioned if I understood correctly.  I've never dealt with stream vectors in 
> GNU Radio so this is all new to me.
> 
> As a side note, is it possible to find out the RS parameters from the 
> protocol, or do you need documentation about the codes used for the original 
> signal?  My signal has a BT of 0.2 if that helps...
> 
> Again, thanks for the help so far, I'm fairly new to the world of 
> error correction for the most part
> 
> Patrick
> 
> -----Original Message-----
> From: Discuss-gnuradio 
> <discuss-gnuradio-bounces+pwhines2=verizon.net@gnu.org> On Behalf Of 
> Daniel Estévez
> Sent: Tuesday, December 14, 2021 2:43 PM
> To: discuss-gnuradio@gnu.org
> Subject: Re: Reed-Solomon Decoding (Not for Digital Television)
> 
> Hi Patrick and Ron,
> 
> Besides the DVB-T Reed-Solomon blocks, there is also the Reed-Solomon blocks 
> from gr-satellites. The gr-satellites blocks expose a couple more parameters 
> about the Reed-Solomon code definition. These are the "first consecutive 
> root" and the "primitive element". These parameters come straight from 
> libfec's generic Reed-Solomon implementation. In many codes these two 
> parameters have the value 1, and I guess this is what the DVB-T blocks use. 
> But if you need other values, for your code, then you can use the 
> gr-satellites blocs.
> 
> Additionally, there are two main difference between the gr-satellites blocks 
> and the DVB-T Reed-Solomon blocks:
> 
> 1. The gr-satellites blocks use PDUs rather than stream vectors.
> 2. Because of this, the gr-satellites blocks get the message size from the 
> PDU size, so you don't need to set this in advance and can be variable.
> 
> I think that for a burst application, depending on whether you already have 
> the data as PDUs or not, it might be easier to use the gr-satellites or the 
> DVB-T blocks.
> 
> Best,
> 
> Daniel.
> 
> El 13/12/21 a las 3:01, Ron Economos escribió:
>> Reed-Solomon predates digital television and is used in many 
>> applications. Here's a nice article about Voyager 1.
>>
>> https://destevez.net/2021/12/voyager-1-and-reed-solomon/
>>
>> The DVB-T Reed-Solomon blocks expose all the parameters, so you can 
>> reuse them for whatever block size you'd like. The default for DVB-T 
>> is
>> 188 byte input packets and 204 byte output packets for the encoder 
>> and the reverse for the decoder.
>>
>> Here's an explanation of each parameter.
>>
>> 1) p and m are not used. These are leftovers from a previous 
>> implementation and retained to keep the block API the same.
>>
>> 2) GF polynomial. This is the Field Generator polynomial and 285 (1
>> 0001
>> 1101) corresponds to x^8 + x^4 + x^3 + x^2 + 1.
>>
>> 3) N. This is the size of the field and is always 255.
>>
>> 4) K. This is the message size. 239 means that there will be 255 - 
>> 239 =
>> 16 parity bytes (that can correct up to 8 message bytes).
>>
>> 5) t is not used.
>>
>> 6) Shortening size. This is the number of message bytes padded with 
>> zeroes. For DVB-T, 239 - 51 = 188, which is the size of a MPEG-2 
>> Transport Stream packet.
>>
>> 7) Blocks. This determines the vector size of the input and output pins.
>> 188 * 8 = 1504 for the encoder input and 204 * 8 = 1632 for the 
>> encoder output. It can be set to 1 if desired.
>>
>> So you have to chose a message size that you can live with and all 
>> your messages have to be the same size (or padded to the same size).
>> And of course, you have to use the encoder at the transmitter and the 
>> decoder at the receiver.
>>
>> Ron
>>
>> On 12/12/21 12:32 PM, pwhines2@verizon.net wrote:
>>>
>>> Hello all,
>>>
>>> My question is fairly simple, how would I perform Reed-Solomon error 
>>> correction after demodulating my incoming signal?  I see there are a 
>>> few uses of RS decoding for digital television, but my signal is 
>>> just bursts of data and has nothing to do with television.  From 
>>> what I’ve gathered I think I need to use the FEC Decoder block and 
>>> somehow tell it to use RS decoding, but I’m not 100% sure.  I would 
>>> preferably like to use blocks that come with gnuradio for this, but 
>>> if that’s not possible then I’ll settle for whatever solution is available.
>>>
>>> The end of my flowgraph is Binary Slicer > Pack K Bits > File Sink, 
>>> and I would imagine the FEC would go after the binary slicer.  The 
>>> packets I’m demodulating now are correct, however there are some 
>>> errors in the packets while the bursts are demodulated (which is why 
>>> I need FEC).  Anyone know how I would go about doing this?
>>>
>>> Thanks,
>>>
>>> Patrick
>>>
> 
>