CEDET speed issues [was Re: Efforts to attract more users?]

[Eric M. Ludlam]

On 07/09/2010 10:33 AM, Aneesh Kumar K. V wrote:
> On Fri, 09 Jul 2010 14:23:24 +0200, Alex Ott<address@hidden>  wrote:
> > Re
> >
> > Eric M. Ludlam  at "Fri, 09 Jul 2010 08:05:32 -0400" wrote:
> >   EML>  On 07/08/2010 11:05 PM, Richard Stallman wrote:
> >   >>  The Eclipse IDE has some nice features.  Maybe CEDET gives us some of
> >   >>  them, but maybe we still lack some.
> >
> >   EML>  In some ways CEDET is "better" that Eclipse (code completion in C/C++ 
> > is better), and in
> >   EML>  others it falls behind, such as project management. (This I gather from 
> > comments I've
> >   EML>  read).
> >
> > The biggest problem with CEDET is speed,
> This is one of the main issue i also have. I would really like some of
> the CEDET related operation pushed to background and make it don't
> effect my editing task. But otherwise I am really happy with the
> feature provided.
When CEDET is all-on, it certainly is busy anytime it first hits a 
project.  It has to parse all the files and discover the extent of your 
project.  In general, it postpones doing stuff as long as possible into 
idle time.  If a user opens some new big file in some big project for 
the first time, scrolls down and hits "complete" on a line, it will go 
off into la-la land for a while to perform the operation.
If the user instead opens that project browses around, gets a cup of 
coffee, and comes back and asks for a completion, it will be quite fast, 
often less than a second even on a relatively large project (think Linux 
Kernel sized.)
When CEDET is doing busy-work during idle time, it should always be 
interruptible.  If you press something like C-v or other nav command, it 
should stop and go do that instead.  If it doesn't then there is clearly 
a loop in part of the code that needs the interrupt handler added.  I've 
noticed this parsing big files sometimes but haven't found a good 
candidate spot to add the check.
If a particular kind of completion always takes a long time, there is a 
profiler setup command you can use to capture your case for analysis in 
semantic-elp.el.  It will save the results making it easier to share the 
results.
My longer term plans involve pushing chunks of the busy work off to a 
separate process to avoid much of these problems.  I also need the 
separate process to offload the database information from the Emacs 
process, keeping most of it on disk unless asked-for.  Super large 
projects (like what I work on at work) will cause your whole machine to 
slow down because Emacs' working set gets so huge.  (Eight Gig And 
Constantly Swapping?)
My current thoughts on making this work is to create a separate 
standalone program to be run as service, much the way databases like 
Postgres is a service.  It would fire off multiple "emacs -batch" slaves 
running CEDET which it would use to collect and process data.  It would 
then manage disk files of Tag data, possibly in some new format.  User 
run Emacses would also run CEDET, and connect to this service as a 
classic semanticdb backend.  To me, this seems like the only way to 
maximize code re-use while also getting the desired work offload from 
the main Emacs process.
I'll probably start digging into that seriously around November this 
year, but I'll be happy to discuss it before then if anyone has ideas 
for implementation short-cuts, etc.
Eric