Long command lines with xargs

[Grandquist, Dean]

I am having a bit of a problem finding out the "good" way to pass very long command lines to gcc with make.

Our build system is based on an existing code base that has 400 directories of files. The C code has includes from any of the 400 directories without using any paths in the #include line. Thus all 400 directories need to be passed as -I directives to gcc.

Here is a sub set of my rules for generating the gcc line:
# ALL_PROJECT_DIRECTORIES is a list of the directories that contain code for the project. The variable that defines that is in a makefile (all_directories.mk) that is included in the base makefile.

MAKEFILE_DIRECTORIES := $(patsubst %,%/Ps2, $(ALL_PROJECT_DIRECTORIES))
MAKEFILE_INCDIRS := $(patsubst %,-I%,$(MAKEFILE_DIRECTORIES)) $(patsubst %,-I%/Headers,$(MAKEFILE_DIRECTORIES))
MAKEFILE_CFLAGS := -O0 $(MAKEFILE_COMPILER_FLAGS) $(MAKEFILE_DEFINES) $(MAKEFILE_INCDIRS) $(INCDIRS)

Project/makefilescripts/outfiles/$(TARGET)/%.o: %.c
        $(CC) $(MAKEFILE_CFLAGS) -c $< -o $@

This gives me a command line that is too long for bash. It looks like POSIX defines a command line length of 2048. I need command line lengths around 10,000 chars to use this makefile method.

$ bash --version
GNU bash, version 2.05b.0(8)-release (i686-pc-cygwin)
Copyright (C) 2002 Free Software Foundation, Inc.

If I use xargs and generate the response file then my make file runs very slow. I get stuck in the catch 22 where I have a variable in make that is greater than the command line limit and I need to write it to a file.

The way that works (very slow) is the following:

Project/makefilescripts/outfiles/$(TARGET)/c_compile_args:
        @echo $@
        $(foreach ar,$(MAKEFILE_CFLAGS), $(shell echo " "$(ar)>>$@))

# change the compile line to this
Project/makefilescripts/outfiles/$(TARGET)/%.o: %.c
        xargs $(CC) -c $< -o $@ < Project/makefilescripts/outfiles/$(TARGET)/c_compile_args

This same problems exists for my link line except the line is much longer, about 30,000 chars. I could fix the link to use a collection of libs, one for each subdirectory, but that does not fix my compiling problem.

If I could change the command line limit in bash that would solve my problem too, but that is a question for a different list 8-)

Any ideas about how I can write a variable to a file without going to the shell a few hundred times?  A build of a single changed file went from 5 seconds to 30 seconds because of my xargs $(foreach) loop.

Thanks
--Dean Grandquist
Software Engineer
Electronic Arts