[Dotgnu-pnet-commits] CVS: pnetlib/doc JScript-embed.txt,NONE,1.1 JScript-internals.txt,NONE,1.1

[Rhys Weatherley <address@hidden>]

Update of /cvsroot/dotgnu-pnet/pnetlib/doc
In directory subversions:/tmp/cvs-serv3129/doc

Added Files:
        JScript-embed.txt JScript-internals.txt 
Log Message:


Perform the initial check-in of the JScript implementation (requires
treecc 0.2.0 or higher).


--- NEW FILE ---

How to embed the JScript engine in your own applications
========================================================

Creating the engine instance
----------------------------

For all of the examples in this document, you need to include the following
namespaces:

        using Microsoft.JScript;
        using Microsoft.JScript.Vsa;
        using Microsoft.Vsa;

Compile your application against the following assemblies:

        Microsoft.JScript
        Microsoft.Vsa
        System

Create the engine instance as follows:

        VsaEngine engine = VsaEngine.CreateEngine();

This allows you to use the engine in an embedded environment.  There are
other ways to create the engine, but they are only for JScript gurus.
The "VsaEngine" constructor (e.g. "new VsaEngine()") is not recommended.

If you create the engine as above, then you can only use a single engine
instance per process.  Subsequent calls to "CreateEngine()" will return
the same object.  This can be problematic in server programs that handle
multiple independent requests in separate threads.  The single-instance
behaviour is required for backwards-compatibility.

This JScript implementation can support "detached engines", which allows
multiple instances to exist side-by-side, each with their own state.
Use the following code to create a detached engine instance:

        VsaEngine engine;
        bool isDetached = false;
        lock(typeof(VsaEngine))
        {
                engine = VsaEngine.CreateEngine();
                try
                {
                        engine.SetOption("detach", true);
                        isDetached = true;
                }
                catch(VsaException)
                {
                        // The "detach" option was not understood.
                }
        }

If "isDetached" is false at the end of this process, then the JScript
implementation does not support detached operation and you should be
careful not to use multiple instances at once.  It is important to
lock the "VsaEngine" class while doing this, to properly support
detached engine creation in multi-threaded programs.

Once an engine instance has been detached, there is no way to "re-attach"
it and make it the default engine instance again.

Providing script source to the engine
-------------------------------------

Scripts are provided to the engine by creating one or more code items:

        IVsaCodeItem item =
                (IVsaCodeItem)(engine.Items.CreateItem("script1",
                                                                                
           VsaItemType.Code,
                                                                                
           VsaItemFlag.None));
        item.SourceText = "<<your script here>>";

If you have more than one script to be executed, then add them all using
the above sequence.  The string, "script1", is a unique identifier for
the item.  Each item should have a different identifier.  The items will
be executed in order.

If a script was loaded from a file, then its code base can be set as follows:

        item.SetOption("codebase", filename);

Compiling and running the script
--------------------------------

Compile the script item (or items) that you just loaded using the engine's
"Compile" method:

        bool ok = engine.Compile();

Then the script can be executed as follows:

        engine.Run();

Running the script will execute any global code that is found within
the script.

Note: this implementation of JScript does not compile scripts down to IL
bytecode first.  Instead, it evaluates them directly in memory from their
tree representation.  This may change in a future version.

Performing evaluations
----------------------

If you are embedding a script engine into an application, you will probably
want to perform evaluations on script fragments after the main script has
been executed.  Use the "Eval.JScriptEvaluate" method for this purpose:

        Object result = Eval.JScriptEvaluate("func()", engine);

Manipulating JScript objects
----------------------------

The easiest way to obtain the value of a global JScript variable is to call
"JScriptEvaluate", passing it the name of the variable.  e.g. if the script
stored a value in the global variable "x", then the following code will
retrieve the value:

        Object x = Eval.JScriptEvaluate("x", engine);

Evaluations can also be used to set the value of global script variables:

        Eval.JScriptEvaluate("x = 3", engine);

JScript objects will normally be instances of the "JSObject" class.
The "JSObject" indexer can be used to get and set object properties:

        Object prop1 = ((JSObject)x)["prop1"];
        ((JSObject)x)["prop2"] = 42;

An object's prototype can be retrieved using the "GetParent()" method:

        Object prototype = ((JSObject)obj).GetParent();

Providing environmental objects
-------------------------------

TODO: describe how to define a C# class that can be referenced by a script
to provide non-core behaviour.

Script output
-------------

At startup time, the "print" command is disabled in the engine.  If you
want your script to be able to print, then you should set the "print"
option as follows:

        engine.SetOption("print", true);

By default, output will go to "Console.Out".  You can redirect it to your
program by creating an instance of the "IMessageReceiver" interface and
calling the "SetOutputStream" method on the engine.  The interface has
one method, called "Message":

        public interface IMessageReceiver
        {
                void Message(String strValue);
        }

The "IMessageReceiver" interface isn't very efficient or culture-aware,
so it is usually better to add an environmental object to the engine
and call that instead of using "print".

Cleaning up
-----------

When you want to discard the script's state, then call "Reset()".  After
that, you can remove the script items and add new ones:

        engine.Reset();
        engine.Items.Remove("script1");
        IVsaCodeItem item =
                (IVsaCodeItem)(engine.Items.CreateItem("script1",
                                                                                
           VsaItemType.Code,
                                                                                
           VsaItemFlag.None));
        item.SourceText = "<<your new script here>>";

You could even re-compile the same script items again, to be executed in
a new context:

        engine.Reset();
        bool ok = engine.Compile();

When you have finished with the engine, call "Close()".  After that,
you can create a new engine using "CreateEngine()", as described above.

--- NEW FILE ---

Internals of the JScript implementation
---------------------------------------

This file describes the internals of the JScript implementation for those
who wish to hack on it.

The internals are designed around the terminology of ECMAScript 3.  A copy
of the specification can be obtained from the ECMA ftp site:

        ftp://ftp.ecma.ch/ecma-st/Ecma-262.pdf

We will refer to section numbers in ECMA-262 below.  A copy of that spec
is essential reading.

[We will update this document with information on the ECMAScript 4 and
JScript extensions in the future]

Parser
------

The JScript parsing code can be found in the "JScript/Parser" directory.
The two important classes are JSScanner and JSParser.

The JSScanner class converts input source text into tokens for use
by the parsing routines.  The bulk of the work is done in the method
JSScanner.GetNextToken().  Tokens are returned as Context objects
from the JSScanner.GetTokenContext() method.

Context's contain the token code (a JSToken value), and the start and
end positions of the token in the source text (for error reporting).
Context's are also used to pass in the initial source text to be parsed,
in the constructor.

The JSParser class implements an LL(1) recursive descent grammar for
the JScript language.  The two main entry points are JSParser.Parse()
and JSParser.ParseEvalBody(), used for programs and "eval" blocks.

JSParser contains a large number of methods for the non-terminals in
the language grammar.  The methods are usually named after the
corresponding non-terminal in the grammar.  e.g. the "PrimaryExpression"
non-terminal is parsed by the method JSParser.PrimaryExpression().

As with most LL(1) parsers, on entry to a method, the parser can assume
that the next token in the input stream has already been fetched.  The
method must consume all tokens for the construct and return a JNode
object for the corresponding abstract syntax tree node.

Syntax errors in the parser are handled by calling JSParser.SyntaxError().
This method throws an exception to process the error.  The exception may
be caught at a higher level to do error-recovery, or it may pass all the
way up to JSParser.ParseSource(), where the parser will bail out.

Tips:

1. If there is a problem recognising identifiers, numbers, strings, etc,
in the source text, then the scanner is probably the best place to start.

2. If there is a problem recognising syntactic constructs, or correct
ordering of items, then the parser is probably where you should look.

References:

Section 7, "Lexical Conventions", in ECMA-262 describes the token rules.

Sections 11 to 14, "Expressions", "Statements", "Function Definition",
and "Program", in ECMA-262 describe the language syntax.

Abstract Syntax Tree
--------------------

This implementation uses treecc to do the heavy lifting on the abstract
syntax tree system.  The tree nodes are defined in "JScript/Nodes/JNode.tc",
with operation cases in the other *.tc files in "JScript/Nodes".  Some
familiarity with treecc is assumed.

The following describes the main node operations:

        Eval
                Determines the value of a node in the current execution context.
                e.g. JConstant nodes expand to their constant values, JAdd nodes
                expand to the addition of their arguments, etc.  Information
                about the current execution context is provided by the VsaEngine
                instance that is passed in as a parameter.

                In ECMA-262, when the spec talks about evaluating a construct,
                and then obtaining its value with "GetValue(X)" or "Result(N)",
                then "Eval" is the corresponding implementation.

        Prepare
                Prepares to store a value to an l-value.  The l-value is
                represented by the current node.  This operation returns
                two data values that are later supplied to "Store" to provide
                context information.

                In ECMA-262, when the spec talks about returning a value of type
                "Reference", then "Prepare" is the corresponding operation.
                The "data1" and "data2" return values can be used to store
                the "base object" and "property name" described by the spec.

        GetAndPrepare
                Same as "Prepare", but also returns the current value that
                was stored in the l-value.  This is used to implement operators
                such as "+=", "*=", "--", etc, that fetch the current value,
                modify it, and then store the result back.

        Store
                Store a value into a location that was previously prepared with
                "Prepare" or "GetAndPrepare".

Tips:

1. Most node types only need "Eval".  The other operations ("Prepare",
"GetAndPrepare", and "Store") are already stubbed out so you don't need
to override them unless absolutely necessary.

2. Adding a new processor for the abstract syntax tree is easy.  For
example, you may want to add an "Emit" operation to compile to IL.

3. If the behaviour of "Prepare", "GetAndPrepare", and "Store" seems
mysterious, then look at the "Eval" code for "JAssign" and "JAssignOp".
That shows how they are used in practice.

References:

Treecc home page - http://www.southern-storm.com.au/treecc.html

Sections 11 to 14 of ECMA-262 describe the evaluation behaviour of
each non-terminal in the language grammar.

Handling break, continue, return, and throw
-------------------------------------------

In section 12 of ECMA-262, the specification describes the semantic value
of statements as "completions".  A completion is a three part value
consisting of a type, a value, and an identifier.

We don't use completions in this implementation.  We instead handle the
semantic completion types in "Eval" as follows:

        normal
                Control returns from "Eval" with the statement's value.
                The value may be "Empty.Value" if the statement is "void".

        break
                The "BreakJumpOut" exception is thrown, with the optional
                break label identifier as an argument.  This is caught by
                loop and switch statements at the appropriate scope level
                and exectution continues from there.

        continue
                The "ContinueJumpOut" exception is thrown and caught in a
                similar manner to "break".

        return
                The "ReturnJumpOut" exception is thrown, with the optional
                return value as an argument.  This is caught at the outermost
                level of a function evaluation to return the value to the
                caller.
        
        throw
                The "JScriptException" exception is thrown, with the thrown
                value as an argument.

Using exceptions for completions simplifies the evaluation engine
considerably.  If a "BreakJumpOut" or "ContinueJumpOut" exception
reaches the outer scope level of a function, an error will be reported.

Tips:

1. See the "Eval" operation cases for "JBreak", "JContinue", etc for more
details on how completions are implemented in terms of exceptions.

Type and object handling
------------------------

Section 8 of ECMA-262 describes the base types of the ECMAScript language.
These are implemented in the following manner:

        Undefined
                The ECMAScript "undefined" value is the same as the C# "null" 
value.

        Null
                The ECMAScript "null" value is "System.DBNull.Value".  When 
compiled
                in ECMA_COMPAT mode, it uses "Microsoft.JScript.DBNull.Value" 
instead.

        Boolean
                The boolean values are the same as the C# "true" and "false" 
values,
                instances of the "System.Boolean" type.

        String
                ECMAScript strings are instances of the C# "System.String" type.

        Number
                ECMAScript numbers are instances of the C# "double" type.  In
                special cases, "int" and "uint" may also be used, normally as
                the output of the ECMAScript "ToInt32" and "ToUInt32" functions.

        Object
                ECMAScript objects are normally instances of "ScriptObject".
                Various classes inherit from "ScriptObject" to provide 
specialist
                functionality: "JSObject", "ScriptFunction", "ObjectPrototype", 
etc.

Section 8.6.2 defines a number of internal properties and methods, which
we implement as follows:

        [[Prototype]]
                "ScriptObject.GetParent()" retrieves an object's prototype.
                The name, GetParent, is used for backwards-compatibility.

        [[Class]]
                "ScriptObject.Class" retrieve an object's class.  This normally
                returns something like "Object" or "Function".

        [[Value]]
                "ScriptObject.Value" retrieves an object's value if the object
                wraps up a primitive value.

        [[Get]]
                "ScriptObject.Get()" retrieves the value of an object property.
                The special-purpose "GetIndex()" method is used to quickly 
access
                the elements of array objects, but is otherwise identical to 
"Get()".

        [[Put]]
                "ScriptObject.Put()" sets the value of an object property.  The
                special-purpose "PutIndex()" method is used for quick access
                to arrays, but is otherwise identical to "Put()".

        [[CanPut]]
                "ScriptObject.CanPut()" determines if it is possible to put a
                particular property on an object.  Returns false if the property
                is read-only.

        [[HasProperty]]
                "ScriptObject.HasProperty()" determines if an object or one of
                its prototypes has a particular property.

        [[Delete]]
                "ScriptObject.Delete()" deletes a particular property from
                an object.

        [[DefaultValue]]
                "ScriptObject.DefaultValue" returns the default value for an
                object, which is normally a primitive value such as a string.

        [[Construct]]
                "ScriptObject.Construct()" uses the current object as a 
constructor
                to create a new object.

        [[Call]]
                "ScriptObject.Call()" uses the current object as a function
                to be called.

        [[HasInstance]]
                "ScriptObject.HasInstance()" is used for testing function 
instances.

        [[Scope]]
                ???

        [[Match]]
                ???

Section 9 of ECMA-262 defines conversion operators for converting between
various types.  These are found in the "Convert" class inside the file
"JScript/Execute/Convert.cs":

        ToPrimitive, ToBoolean, ToNumber, ToInteger, ToInt32, ToUInt32,
        ToUInt16, ToString, ToObject

References:

Section 8, "Types", ECMA-262 describes the types and their operations.

Section 9, "Type Conversion", ECMA-262 describes the type conversions.

Builtin JScript classes
-----------------------

TODO