Re: [Discuss-gnuradio] Maximum input items consumed on each input stream

[Adrian Musceac]

Hi,

I found this thread very useful, and I have another question regarding the scheduler internals.

It is my understanding from reading documentation that each block's *work* or *general_work* function

will be called at the highest sample rate that the block will see, whether that is on the input or output of the block.

I'd like to know what happens if the block's work function does not finish within the time allotted to it.

Is part of the input/output buffer dropped, or does it interfere with the sample rate of the next blocks?

For example, I have a block which filters incoming samples with a variable tap length matched filter to output the maximum correlation:

https://github.com/kantooon/gr-dsss/blob/master/lib/dsss_decoder_cc_impl.cc

If the filter size is too large, the block will consume too much CPU time doing the filtering and output some wrong values.

Is this because some output is dropped/truncated, or because it changes the sample rate of the next blocks?

Thanks,

Adrian

On Sat, Aug 31, 2019 at 4:56 AM Ron Economos <address@hidden> wrote:

If you use set_output_multiple(), you don't have to check the input buffer. The block will only execute if there are a multiple of the value used in set_output_multiple() items available. For example, if set_output_multiple() is set to 256, the block will only execute if noutput_items is at least 256, but it could also be 512, 768, 1024, 1280, 1536, etc.

Since forecast() sets ninput_items_required to noutput_items, the same number of items appears on the input buffer.

Here's a dummy block that just copies input to output to show the structure. The loop in general_work() allows for any value of CHUNK_SIZE to work properly. With a size of 8900, the loop will typically only execute once.

#ifdef HAVE_CONFIG_H
#include "config.h"
#endif

#include <gnuradio/io_signature.h>
#include <stdio.h>
#include "buffer_cc_impl.h"

#define CHUNK_SIZE 8900

namespace gr {
  namespace laura {

    buffer_cc::sptr
    buffer_cc::make()
    {
      return gnuradio::get_initial_sptr
        (new buffer_cc_impl());
    }

    /*
     * The private constructor
     */
    buffer_cc_impl::buffer_cc_impl()
      : gr::block("buffer_cc",
              gr::io_signature::make(1, 1, sizeof(gr_complex)),
              gr::io_signature::make(1, 1, sizeof(gr_complex)))
    {
      set_output_multiple(CHUNK_SIZE);
    }

    /*
     * Our virtual destructor.
     */
    buffer_cc_impl::~buffer_cc_impl()
    {
    }

    void
    buffer_cc_impl::forecast (int noutput_items, gr_vector_int &ninput_items_required)
    {
      ninput_items_required[0] = noutput_items;
    }

    int
    buffer_cc_impl::general_work (int noutput_items,
                       gr_vector_int &ninput_items,
                       gr_vector_const_void_star &input_items,
                       gr_vector_void_star &output_items)
    {
      const gr_complex *in = (const gr_complex *) input_items[0];
      gr_complex *out = (gr_complex *) output_items[0];

      printf("noutput_items = %d\n", noutput_items);
      for (int i = 0; i < noutput_items; i += CHUNK_SIZE) {
        memcpy(out, in, CHUNK_SIZE * sizeof(gr_complex));
        in += CHUNK_SIZE;
        out += CHUNK_SIZE;
      }

      consume_each (noutput_items);

      // Tell runtime system how many output items we produced.
      return noutput_items;
    }

  } /* namespace laura */
} /* namespace gr */

Ron

On 8/30/19 15:58, Laura Arjona wrote:

Thank you very much Marcus, Michael, Abin, and Ron, really appreciate your responses.

To give some context, I just started designing a prototype reader to implement a custom protocol for backscatter neural implants; very excited to build my platform with GNU-radio :)

After reading all the information from your responses and links provided, I still have a problem with my implementation. According to the buffer sizes that you mentioned, I should not have this problem, but

I think I am missing something. I may need to re-design my system/flow-graph, but I would like to get a last advice/help, if possible. Thanks in advance!

I want my block Decoder to consume_each(>8900) but I get overflows "D" messages. See details below

I have 2  general out of tree blocks: Gate and Decoder.

They both have:

                ninput_items_required[0] = noutput_items;

                const gr_complex *in = (const gr_complex *) input_items[0];
                gr_complex *out = (gr_complex *) output_items[0];  

The flow-graph looks like uhd_source -> fir_filter_ccc -> Gate -> Decoder -> other blocks.   (Using a USRP N210 + SBX)

The idea is that I want the block Decoder to only process the input samples when I have received k samples. Let's set k=~8900

So, at the Decoder block general_work(), I set  consume_each(0) until ninput_items[0]>=k.

Basically, at the Decoder general_work() I have the following (just a overview, not pseudo-code):

 if (ninput_items[0] <k)

       //do nothing

      //consume_each(0)

else

     // process the input samples

     //consume_each(k)

My problem is that if I set k~8900, I get 'D' messages on the terminal.

And one interesting? thing happens. When ninput_items[0] gets close to k=8500 (or higher value), is when I start getting  'D', and after

that ninput_items[0] = 800, no matter the value of k.

Thank you.

Cheers

Laura.

On Fri, Aug 30, 2019 at 7:29 AM Müller, Marcus (CEL) <address@hidden> wrote:

Hi Ron,

just because I think this might be interesting to people dealing with
really high rates:
The maximum size is typically limited by the size of mmap'able memory
that you can allocate; that depends on the circular buffer factory
used:
For the posix shared memory thing, I don't think anything is stopping
you from using "memory space size" order amounts of buffer.
For anonymous file-backed mmap'ed buffers, I'd expect that we haven't
addressed the possibility of using more than 32 bit addresses, so
somewhere around 2 GB you'd find your upper limit.

Best regards,
Marcus

On Fri, 2019-08-30 at 06:20 -0700, Ron Economos wrote:
> Just to put a number on this question, the DVB-T2 transmitter uses up to 16 Megabyte buffers between blocks. I'm not sure what the absolute maximum is, but 16 Megabytes should cover most applications.
> The DVB-T2 blocks use set_output_multiple() in conjunction with forecast() to allocate these large buffers.
> Ron
> On 8/28/19 11:46, Laura Arjona wrote:
> > Hello GNURadio community,
> >
> > Does anyone know what is the maximum number of input items that an Out Of Tree block can consume on each input stream?
> >
> > consume_each(consumed) --> what is the maximum value that the variable consumed can take?
> >
> > Thank you very much.
> >
> >
> > --
> > Laura Arjona
> > Washington Research Foundation Innovation Postdoctoral Fellow in Neuroengineering
> >
> > Paul G. Allen School of Computer Science & Engineering
> > 185 E Stevens Way NE
> > University of Washington
> > Seattle, WA 98195-2350
> >
> >
> > _______________________________________________
> > Discuss-gnuradio mailing list
> > address@hidden
> > https://lists.gnu.org/mailman/listinfo/discuss-gnuradio
>
> _______________________________________________
> Discuss-gnuradio mailing list
> address@hidden
> https://lists.gnu.org/mailman/listinfo/discuss-gnuradio
_______________________________________________
Discuss-gnuradio mailing list
address@hidden
https://lists.gnu.org/mailman/listinfo/discuss-gnuradio

--

Laura Arjona 

Washington Research Foundation Innovation Postdoctoral Fellow in Neuroengineering

Paul G. Allen School of Computer Science & Engineering

185 E Stevens Way NE

University of Washington

Seattle, WA 98195-2350

_______________________________________________
Discuss-gnuradio mailing list
address@hidden
https://lists.gnu.org/mailman/listinfo/discuss-gnuradio