[Aleader-dev] Re: [opencyc - Open Discussion] RE: emotions & films in cyc

[William L. Jarrold]

On Tue, 15 Jul 2003, Joshua N Pritikin wrote:

> [Since this email isn't really related to Cyc, I'm posting to
> the Aleader discussion list instead.]
>
> On Mon, Jul 14, 2003 at 02:08:19PM -0700, SourceForge.net wrote:

<...stuff deleted...>

> > For starters, how are you going to translate a given film into
> > your "seven logical questions."?
>
> As objectively as possible.  ;-)
>
> No seriously, it is a good question.  I am going to use a decision
> tree.  Interested in hearing more details?

Yes.

>
> > This is what I used for my input space.  I ended up choosing about
> > 9 vignettes from a manual for mental health professionals called
> > _Teaching Children with Autism How to Mindread_.  Although this
> > is a pitifully small input and output space, the new thing about
> > my dissertation is that it addresses the issue of how to
> > empirically validate a KR model in this context.
>
> "empirically validate a KR model" .. can you expand on that please?

Well, I'd say "empirically validate a KR model" (EVKM) is a pretty broad
idea.  If you build a KR system and evaluate it with the Turing Test that
is an example of EVKM.  What I did was a little less ambitious, more
acceptable to dissertation committees, and more feasible.  It is more
acceptable to dissertation committees because testing for no significant
difference is basically proving the null hypothesis -- it is not something
you can do.  It is like failing to detect a signal does not prove there is
no signal.  But you can say, "Well, I can prove my detector was sensitive
to the level X and since I detected nothing, then I know that if there is
a signal, then its strength is less than X".  So, that bit of statistical
methodology is important to making this work a part of ongoing research
traditions, philosophies, methodologies, etc..

So, next, I built a model, had it generate inferences about other
people's emotions on a set of inputs.  Then I tweaked the model
(specifically I changed the valence) and had it generate inferences to the
same inputs.  So, now we have two sets of computer generated responses.
I also gave the same inputs to some human subjects and had them generate
affective inferences.  I took there responses and tweaked them in the same
way - i.e. I reversed the valences. So, now we have four groups of
responses:

computer generated unreversed
computer generated reversed
human generated unreversed
human generated reversed

I mixed these responses up and fed them to a new set of human subjects.  I
asked the human subjects to rate the believability of each response.  Lo
and behold there was a significant difference in believability between
reversed and unversed items but no significant difference between
unreversed items that were computer vs human generated.

Empirically validating KR systems is something that we need lots more of.

Sticking within computer generated items, you could view
reversed/unreversed (aka "reversal") as a form of computational ablation.
In computational ablation we tweak our computer model and make hypotheses
about how the tweaked system should behave.  If you want to test a
scientific hypothesis about a computer program (best done on large complex
programs, like AI programs) then a computational ablation study is one way
to go.



<...stuff deleted...>

> > Or try Ira Roseman (if you ask again I can dig out some references).
>
> I see a lot of papers on her (his?) home page.  Can you suggest
> one which is particularly relevant?

No, I am not that familiar with his stuff.  This is the one I read way
back...

Roseman,I.J., Antoniou, A.A., & Jose, P.E. (1996).

...sorry I'd have to dig more to get you a journal, page nums etc..

>
> > Craig Smith is at the start of his career perhaps he has something useful
>
> Yah, I was surprised to find this one:
>
>   McHugo, G. J., Smith, C. A., & Lanzetta, J. T. (1982). The structure
>   of self-reports of emotional responses to film segments. Motivation
>   and Emotion, 6, 365-385.
>
> The title sounds intriguing, but it's really not that similar
> to what I'm doing.

Wow, 1982?  It might be a different Smith.  Well, sorry, I saw him give a
talk at a conference.  If you ask me again I'll dig out the conference
proceedings and see if I can get you a more precise pointer.

>

<...>

>
> I propose an emotion predictor that can be tested empirically.
> My primary interest is whether statistics suggest that it
> works empirically.

Right.  That is where my dissertation might be especially helpful.  I
won't assert that doing the baby study I did was a huge big deal.  But,
what it did accomplish was establish a experimentally and
statistically and practically rigorous foundation on which to build many
more subtle studies.

>
> How the emotion predictor compares to the other hundred theories
> about emotion is of secondary interest.  On the other hand, I know
> I have to do a survey of others' work as part of my research article.
> So I _do_ want to collect these references.  I wish Jarrold, W. (2001)
> was available online ... *grumble*

Yes, making a web site with my articles on it has been on my to do list
forever.  I don't want to get in trouble with copyright violations but
AAAI sent me an email saying it was okay to put their articles (written by
me) up on my website.  One (legal) hurdle down, one or two remaining.
Ack -- the next hurdle is a sysadmin one...I've recently had my debian
ungraded and in the process all my old files are hidden deep in a tar
file.  I can send you my  dissertation easily.  Pulling out the other
articles might take a day or two.  Feel free to ping me again if too much
time passes.  I just got back from a long journey and am playing catch up.

Bill

>
> --
> Victory to the Divine Mother!!         after all,
>   http://sahajayoga.org                  http://why-compete.org
>