As Georg-Johann Lay wrote:
They just add startup code, perhaps a linker script to describe
memory on their boards and compile for their architecture.
Still, if it's not the IDE that does it on their behalf, most of those
professional ARM developers are hosed. They solely rely on their
(often payware, like CodeSourcery) IDE to do it for them. If the IDE
doesn't do it, they are standing in about the same rain as an AVR
developer whose device is not supported by the toolchain.
The AVR approach is somewhat more user-friendly, in that you can add
a single -mmcu option, and then compile your sources the same simple
way as you would e.g. be able to compile code for your host computer.
Technically, the way you describe would still be possible even with
today's AVR-GCC and binutils: just use the appropriate -mmcu=avrXXX
option describing the CPU core / overall architecture, use your
appropriate linker options (for SRAM start etc.) or linker script,
supply your own IO header file. Minus a few things in avr-libc (like
power management or EEPROM support), this would work. Right now.
Nobody does it that way.