Flex stdout to Bison stdin

[Steve Litt]

Hi all,

I'm trying to learn Bison and Flex so I can make a converter between a
markup language of my design and PDF or HTML, depending on which Bison
program I use. After having been unsuccessful following several online
docs, I've decided to make the simplest possible parser where my
trivial Flex program tokenizes a document file comprised of paragraphs
each separated by one blank line, and my trivial Bison program adds
"<p>" to the front of each paragraph, and </p> to the end of each
paragraph. On this trivial experiment, to make it easier, I can specify
that a blank line consists of two consecutive \n characters without any
space between them, and if necessary a \n at the beginning of the
document file and an extra \n at the end of the document file.

I can see two ways to do this:

1) All Flex to Bison communication happens via the scanner's stdout to
   the parser's stdin, using a distinct executable for the scanner and
   the parser respectively. Both the Flex program and the Bison program
   would each have a main() function.

2) All Flex to Bison communications happen through tokens defined in
   the Bison source file, yylval, $$ and $1 etc. The Flex program would
   be a library without its own main(), that gets compiled and linked
   into the Bison generated parser. The combined program then
   translates the input document coming in through its stdin into
   something else that comes out of its stdout

#1 is conceptually easy, but I haven't been able to do it. I got close,
   but I couldn't make Bison do the regex necessary to parse its stdin
   input.

#2 seems to be the "best practices" way and I'd imagine it's faster on
   big input files, but I haven't been able to make it work because I
   don't understand how it functions.

I know this text paragraphs to <p></p> surrounded paragraphs conversion
could easily be done in Flex only, or for that matter a five line AWK
program or even an AWK one liner. My purpose in making this thing is to
have the easiest possible program that actually passes tokens from the
scanner to the parser, which translates it into something else. It's
the "Hello World" I must slowly build from to create a real converter.

For #2 I'd imagine my Bison rules would look something like the

input: chunk | input chunks
chunk: paragraph newline


1) Has any of you communicated between your lexer and parser using
stdio/stdin or intermediate files exclusively?

2) For option 2, what would my Flex and Bison files look like, and what
   would be the command to compile and link  them together? If the
   executable for option #2 is a.out, would the command to use it be
   cat mytest.txt > ./a.out

I know it sounds like I'm asking you to do my homework, but I've been
trying to do this for two weeks with web research and experimentation,
and haven't gotten to first base, so I'd appreciate any help you could
give.

Thanks,

SteveT

Steve Litt 

Autumn 2023 featured book: Rapid Learning for the 21st Century
http://www.troubleshooters.com/rl21