Re: Saving calculation answers to file

[Nicholas Jankowski]

On Tue, Jul 24, 2012 at 3:28 PM, vilsu <address@hidden> wrote:
> Little testin:
> sample run 3040 lines, P4 2400MHz
>
> Run1:
>
> fid = fopen ("koe.txt");
>         for i = 1:3040
>            x = eval (fgetl (fid))
>            save -append tuloskoe.txt x
>         endfor;
> fclose (fid)
>
> -:took 15 s:-
>
>
> Run2:
>
> fid = fopen ("koe.txt");
> eof = fgetl (fid)
> while eof != -1
>       x = eval (eof);
>       save -append out.txt x
>       eof = fgetl (fid)
> endwhile;
> fclose (fid)
>
> -:took 9 s:-
>
>
> Run3:
>
> fid = fopen ("koe.txt");
> results = cell(3040,1);
> for i =1:3040
>       results{i} = eval (fgetl (fid));
> endfor
> save out.txt results
> fclose (fid)
>
> -:took 3 s:-  !!! only 1/5 of the time of the original!!
>
> Thanks Juan
>
> vilsu

Just remember, 'results' is going to get big depending on how large
your input file gets. you had mentioned 10,000,000 lines... depending
on the size of the ouput from each command you could run into memory
problems by holding all of the results file at once. also the save
-append within the loop is probably not the most efficient way to
write the data to the file. you could use fopen before the loop, then
some of the c-style commands to write to file one line at a time
within the loop, then close the file after the loop. I would think it
would be faster than 'save -append' since it wouldn't repeat the file
handling overhead. so, if you have memory issues there are options.