Re: Octave Machine Learning & Deep Learning - what is available

[Benson Muite]

On Sun, Apr 5, 2020, at 11:14 AM, Evangelos Rozos wrote:
> While the discussion keeps going regarding the path Octave development 
> should follow, it would be worth summarizing what are the available 
> options for Machine Learning & Deep Learning.
> 
> 
> NNET
> The best option for feedforward networks is the unmaintained package 
> nnet. Use ‘pkg install -forge nnet’ to install it. Some modifications 
> are required to be compatible with the latest Octave versions, see 
> https://github.com/rozos/octave-nnet.

This looks nice, though speed may be slow since native Octave code rather than 
calling a compiled library. It is  better for full understanding though. Maybe 
worth updating to become compatible. Maybe there are tips that could improve 
performance, or does are integrations with Vendor libraries worth trying?

> 
> Cortexsys
> Cortexsys is a nice and simple framework (development not active since 
> 2016) that supports FFN, CNN, RNN, LSTM. Get Cortexsys from 
> https://github.com/joncox123/Cortexsys. It will require compiling some 
> mex files. The easiest way to do it is start Octave go inside 
> nn_core/mmx directory, then
> for Linux:
>       mex -lpthread -DUNIX_SYSTEM mmx.cpp
> for Windows:
>       mex -lpthread -DWIN_SYSTEM mmx.cpp

It seems to rely a lot on CUDA. Maybe SyclDNN 
(https://github.com/CodeplaySoftware/SYCL-DNN) is better? Performance on wide 
range of hardware will be of some consideration.

> 
> 
> Caffe
> The most sophisticated deep learning framework you can get in Octave. 
> Unfortunately development stopped in 2018. The project was merged with 
> PyTorch. I managed to make it run with Octave in Ubuntu 18.04, see 
> https://drive.google.com/open?id=1S-hQOQeiSDgwBAjy_6BTr8WqiP4SccFG.

Probably most people using Octave will trade ease of setup and reproducibility 
over performance. Also for some of these fast moving projects, much development 
time might be spent fixing things.

> 
> ER
> 
> 
> 
>