If you don't want to deal with the MS Visual Studio installer, you can install the MinGW version, which is a bit more of a manual process:
"Octave 3.6.4 for Windows MinGW installer"
there are two zipped files in that folder (7z zips, you may need to download a program that can handle that format, such as at
http://www.7-zip.org)
Create an Octave folder on the C: drive. then unzip both of those files to that folder. that will recreate the program folder for Octave and Octave packages.
There will be shortcuts in that folder to start Octave and open the Octave documents. You can copy those shortcuts to your desktop. You may need to right click on them and edit Properties so they point to your C:/Octave folder
Last, you should enable the packages. This can be done by starting octave and running the following commands:
pkg rebuild -auto
pkg rebuild -noauto ad
pkg rebuild -noauto nan
pkg rebuild -noauto gsl
pkg rebuild -auto java