[myexperiment-hackers] Re: myExperiment + RapidMiner

[Jiten Bhagat]

[cc'ing the myExperiment dev list]

Hi Simon,

Packaging this as a zip has benefits beyond myExperiment:
- it's easily extensible for future needs - you can add new things
beyond the workflow definition.
- it's highly portable.
- you can embed custom things in there - like custom components etc.
used by the users. Thereby ensuring that workflows can be run by anyone
(though this is a bigger engineering/design issue for you :-)).

And you're right, no one should have to build this package manually, the
tools should do this for them. And you're right too when you say that,
for myExperiment *specifically*, you can just use the API to provide the
metadata + preview image and that's all we need to do if users are not
likely to upload via the web interface.

> However, as I understand it, when I upload a zip to workflow.xml, I
> will also receive this zip file when I download it again, which is not
> what I want, of course. As long as this behaviour can be avoided, I
> can also make a zip file. 

The current behaviour is that whatever has been uploaded will be
downloadable. So if your requirement above is crucial then we shouldn't
package your workflows as zips.

In that case, how about:
- We stick to uploading of the XML workflow definition via the API (as
you have already done).
- You send me a few examples of workflows, and a mockup of how you want
the "run" and "workflow components" sections to look like, so I can
build a processor and views. In the "Run" section it would be useful to
have links to how someone might run the workflow (eg: "download the
RapidMiner workbench from...").

Let me know.

Cheers,
Jits



Simon Fischer wrote:
> Hi Jits,
>> Carole has mentioned in an email that you need some assistance with the
>> myExperiment integration? I am at your service, again :-)
>
>
>> As I understand it, you will be packaging your workflows up into zip
>> files, together with metadata, images etc?
>
> Actually, I have currently not implement that option. If I get it
> right, this option is only relevant if users upload this zip file via
> the Web interface, right? I think this is pretty pointless since noone
> would ever build such a zip file manually, and if it is generated by
> RapidMiner they can as well use the API.
>
>> If so, thanks! And also
>> you're happy to have your users mainly uploading workflows via the API,
>> in which case you are providing the image and metadata directly via the
>> API? If so, and if you are happy that this is all your users need then
>> nothing else needs to be done.
>
> This is what I am already doing. I upload
>
> - an xml file containing the workflow
> - a png preview
> - a description
>
> I *could* upload
>
> - an SVG preview
>
> but there is no tag for that in <workflow>. It would also be possible
> for me very easily to put all of this into a zip file if that is
> easier for you.
>
>> However, if you need more support from within the myExperiment Web
>> interface (like: upload your workflows without providing metadata and
>> images in the web interface; a custom "run" section with custom run
>> options and a custom "workflow components" section with RapidMiner
>> specific information) then we'll need a custom processor. I can do most
>> of this for you, since we have code already that strips out files from a
>> zip package. The only thing I would need help on is building a set of
>> Ruby classes to represent the internal model of your workflows (IF you
>> need a custom "workflow components" section).
>
> This is pretty simple. In the XML we have a set of tags of the
> following form:
>
> <operator class="CLASS" name="NAME">
>   <description>DESCRIPTION</description>
> </operator>
>
> These operators are fairly equivalent to the processors of Taverna
> workflows. I guess this can be extracted relatively easy using an
> XPath expression. We also have input and output ports, but they are
> just links to local repositories and are therefore pretty useless to
> display at the moment.
>
> For the run section, I think we will just link to a URL. We will
> shortly have a WebStart version which is probably the preferrable
> link. We should also display a link to the workflow here, as is done
> for Taverna, which the user can paste into the myExperiment plugin.
>
> I'm still not sure I fully understand what advantage having a zip file
> has. Is that only for the case the user uploads via the Web interface?
> In that case, I believe we don't needed, as I said above. If the zip
> file can contain more information and metadata than the <workflow>, I
> will make it. However, as I understand it, when I upload a zip to
> workflow.xml, I will also receive this zip file when I download it
> again, which is not what I want, of course. As long as this behaviour
> can be avoided, I can also make a zip file.
>
> Cheers,
> Simon