[Gzz-commits] manuscripts/FutureVision oplan.txt vision.rst

[Benja Fallenstein]

CVSROOT:        /cvsroot/gzz
Module name:    manuscripts
Branch:         
Changes by:     Benja Fallenstein <address@hidden>      03/11/13 16:30:50

Modified files:
        FutureVision   : oplan.txt vision.rst 

Log message:
        more

CVSWeb URLs:
http://savannah.gnu.org/cgi-bin/viewcvs/gzz/manuscripts/FutureVision/oplan.txt.diff?tr1=1.34&tr2=1.35&r1=text&r2=text
http://savannah.gnu.org/cgi-bin/viewcvs/gzz/manuscripts/FutureVision/vision.rst.diff?tr1=1.183&tr2=1.184&r1=text&r2=text

Patches:
Index: manuscripts/FutureVision/oplan.txt
diff -u manuscripts/FutureVision/oplan.txt:1.34 
manuscripts/FutureVision/oplan.txt:1.35
--- manuscripts/FutureVision/oplan.txt:1.34     Thu Nov 13 15:57:45 2003
+++ manuscripts/FutureVision/oplan.txt  Thu Nov 13 16:30:49 2003
@@ -485,7 +485,7 @@
     relationship types, we show only as many as fit on the screen
     and allow the user to scroll through the list. To make this fast
     even for very large lists, we plan to employ fisheye sampled
-    lists (Furnas 1997), in which every element of 
+    lists (`Furnas 1997`_), in which every element of 
     an n-elememnt list can be reached with O(log(n)) clicks.
 
     ADD TO REFERENCES:
@@ -549,7 +549,7 @@
     Libvob also uses a directed acyclic graph as the scene graph,
     rather than the conventional tree; i.e., its graphical model
     is not hierarchical like that of other toolkits. This provides
-    natural support for transpointing windows (Nelson 1995),
+    natural support for transpointing windows (`Nelson 1995`_),
     i.e., connections between different parts of the view hierarchy:
     Showing a connection between two different windows is just
     as simple as showing a box inside a single window, the only
@@ -680,10 +680,6 @@
     way to interface our system to the Semantic Web `(Berners-Lee 1998)`_.
 
 
-
-    XXXY
-    
-
 [31] FenPDF as an applitude: describe how it could
 integrate with the network of items (this is then
 an example of a non-itemgular structure being
@@ -863,7 +859,7 @@
     Later, a similar style of navigation and structural
     editing was also used in the commercial
     product TheBrain(XXXCite), to allow users to organize files 
-    and "thoughts" on their computers.
+    and items on their computers.
 
     One of the most important things that is different in the visualizations
     presented in this article and the ones for example shown in 
@@ -888,6 +884,12 @@
 
 [11] Add reference to Callimachus
 
+    However, Callimachus (`Tzagarakis et al 2003`_)
+    and the FOHM model (`Millard et al 2000`_) 
+    are interesting steps
+    towards a similar approach: they combines the different
+    hypertext domains into a single conceptual structure.
+
 
 5.6 Fluid Links and Transpointing Windows
 -----------------------------------------
@@ -905,6 +907,7 @@
 Mention all people that have been in the group,
 i.e. also Katariina Ervasti, Rauli Ruohonen, Kimmo Wideroos.
 
+Thank CF.
 
 Thank anonymous referees.
 
Index: manuscripts/FutureVision/vision.rst
diff -u manuscripts/FutureVision/vision.rst:1.183 
manuscripts/FutureVision/vision.rst:1.184
--- manuscripts/FutureVision/vision.rst:1.183   Thu Nov 13 15:52:18 2003
+++ manuscripts/FutureVision/vision.rst Thu Nov 13 16:30:49 2003
@@ -579,11 +579,12 @@
 
 Fenfire is a free software project aiming at implementing
 the applitude-oriented user interface concepts on top of an RDF graph.
+Fenfire is a research prototype.
 
 4.1 Structures
 --------------
 
-Fenfire provides two intertwingled structures for applitudes:
+Fenfire provides two interwoven structures for applitudes:
 RDF (`Lassila and Swick 1999`_) 
 and Xanalogical referential fluid media (`Nelson 1999c`_).
 
@@ -631,6 +632,19 @@
 relationships at every time: Depending on the task at hand,
 the user can "switch" individual relationship types on and off.
 
+This is entirely different from existing
+RDF visualization tools, which create
+a fixed 2D layout for a graph (sometimes shown in a distorted,
+hyperbolic-like view), and then only allow
+the user to zoom and scroll through it.
+
+If there are too many connections along the active
+relationship types, we show only as many as fit on the screen
+and allow the user to scroll through the list. To make this fast
+even for very large lists, we plan to employ fisheye sampled
+lists (`Furnas 1997`_), in which every element of 
+an n-elememnt list can be reached with O(log(n)) clicks.
+
 RDF diagrams usually label nodes with the string that is
 used internally to identify the node. This is not acceptable
 for showing a network of items; items need to be labeled
@@ -699,6 +713,9 @@
     Figure 7: a diagram of buoy animation (`larger image`__).
     The "screenshots" with black edges represent
     keyframes, and the others the animation.
+    The connecting lines between the focus and the buoys have
+    been emphasized in this diagram; in real life, they are
+    translucent.
     
     __ buoysMotion.png
 
@@ -714,7 +731,14 @@
 to the orange document is trivial without
 an explicit "back button".
 
-
+Also, buoys move automatically when the document
+they are connected to is scrolled. Buoys are arranged
+on an ellipse around the viewport, so that the focused
+part of the document, inside the ellipse, is not obscured.
+Additionally, only buoys whose anchors are visible are shown,
+and the buoys whose anchors are close to the focus
+are shown larger. This way, the largest buoys are those
+connected to things at the focus of the view.
 
 The intent of buoys is to reduce the cognitive overhead
 of browsing and editing the structure 
@@ -747,37 +771,50 @@
 4.3 Libvob
 ----------
 
-Libvob is a subproject of Fenfire, providing a flexible user-interface
-toolkit with some novel features that allow the independent
-applitudes to work on the same user interface.
-
-Instead of a static structure of widgets or a scene graph,
-the scene is regenerated each time the user presses a key [#libvob-speed]_.
-
-The scene consists of a DAG of coordinate systems,
-a keyed tree for *identifying* coordinate
-systems and a set of renderable objects (vobs) placed into 
-one or more coordinate systems each.
-
-This structure allows
-
-1) Automatically generated animations between successively 
-   generated scenes: the coordinate systems are matched
-   to each other based on the keyed tree 
-
-2) More importantly, **post-processing** of the view generated
-   by other applitudes -- applitudes can **add** new coordinate
-   systems and vobs on the view, based on what is already shown.
-
-   The buoys are added this way: a connector object goes through
-   a generated vobscene and adds buoys to the nodes in it - regardless
-   of who placed those nodes in there.
+We have suggested that users should be able to add
+connections to views programmed by somebody else.
+For example, when showing meetings on a timeline,
+it should be easy to add buoys showing the participants
+of each meeting. To allow such connections to be added
+without changing the view's code, we have developed
+Libvob, a flexible user-interface toolkit.
+
+Libvob provides functionality to not only construct
+a scene graph (i.e., specify what to draw on the screen)
+but to also specify which parts of the scene graph
+correspond to which objects in the application model,
+the RDF graph in the case of Fenfire. 
+
+The timeline view, for example, would specify which
+parts of the scene graph represent each event drawn
+(events being nodes in the RDF graph). A different view,
+programmed independently, can iterate through these nodes
+and connect buoys to each of them. Through the association
+with part of the scene graph, it knows *where* to connect
+the buoys to.
+
+Libvob also uses a directed acyclic graph as the scene graph,
+rather than the conventional tree; i.e., its graphical model
+is not hierarchical like that of other toolkits. This provides
+natural support for transpointing windows (`Nelson 1995`_),
+i.e., connections between different parts of the view hierarchy:
+Showing a connection between two different windows is just
+as simple as showing a box inside a single window, the only
+difference being that the connection has *two* parents
+in the scene graph instead of only one.
+
+This is necessary for the implementation of buoys, which
+are an instance of transpointing windows (a connection
+is drawn between the buoy and the part of the main view
+it is related to).
+
 
 4.4 An example applitude combining multiple structures: FenPDF
 --------------------------------------------------------------
 
-FenPDF is the first concrete prototype of our architecture.
-It is a tool to make sense of academic literature.
+FenPDF is the first concrete prototype of our architecture,
+using buoys, Libvob, RDF, and Xanalogical structure.
+It is a tool for making sense of academic literature.
 
 ..  figure:: fenpdf-shot-small.png
 
@@ -787,6 +824,19 @@
 
     __ fenpdf-shot.png 
 
+FenPDF is used to structure a set of articles
+in PostScript or PDF format. Users can transclude
+pieces of articles onto *spatial canvases*: infinite,
+scrollable papers.
+Transclusions are automatically bidirectionally connected 
+to the article they are from; a buoy shows a shrunk version
+of the article, and clicking on the buoy brings the article
+to the center for the user to read.
+
+Additionally, the user can type text onto the canvases,
+and link two pieces of text on different canvases
+(linked canvases are shown as buoys).
+
 In the figure, there are two foci. 
 The **upper focus** shows **a PDF article**
 and its buoys there show the places of
@@ -801,7 +851,39 @@
 (`Kujala and Lukka 2003`_) to endow them *identity*
 in the eyes of the viewer.
 
-
+We use FenPDF collaboratively in our research group
+(synchronizing the RDF graph through CVS).
+In practice, we have:
+
+- A canvas for each source (for example, conference,
+  journal issue), with transclusions of the titles
+  and author lists of the articles published there.
+- Canvases for different topics, such as open hypermedia,
+  spatial hypertext, and so on. These canvases
+  contain transclusions of particularly relevant parts
+  of articles, allowing us to collect the central ideas
+  from several different articles.
+- Canvases for each article we are working on,
+  containing notes and transclusions from important
+  references.
+- Canvases for communicating specific ideas. These
+  contain Memex-like "trails" of transclusions from
+  different articles, intersersed with text discussing these.
+- A central canvas that has links to the other canvases.
+
+In our experience, we have found the transclusion facility 
+very useful: copying a well-chosen region of an article
+to the a canvas allows the user (either the one who made the
+transclusion, or a collaborator) reading the canvas to quickly
+just read the emphasized part, going back to the article
+for the details if desired. The ease with which the transclusions
+are created (simply cut and paste) has allowed users to quickly summarize
+the most relevant points of the article on a separate canvas for
+easy perusal and for contrasting with points from other articles.
+
+In FenPDF, articles, spatial canvases, transclusions
+of an article and pieces of text on a canvas are all
+represented by RDF nodes. 
 On the structure side, FenPDF uses four small RDF vocabularies:
 
 - ``FF``: Associating Xanalogical referential fluid media
@@ -810,7 +892,6 @@
   This is used both for text content and transclusions
   from PS/PDF files.
 
-
 - ``CANVAS2D``: Placing nodes on a canvas - as in
   spatial hypertext (`Marshall and Shipman 1995`_)
 
@@ -828,17 +909,42 @@
 no directly annotation-specific program code in FenPDF. 
 The different orthogonal
 structures combined just give the user the opportunity 
-to create any kinds of structure. One common pattern of use we've noticed
-is transcluding small regions from articles related to a single
-topic canvas.
+to create any kinds of structure. 
+
+Explicit support for taxonomic hypertext (`Parunak 1991`_)
+and hierarchies is currently being planned.
 
 FenPDF will first be released as a separate application,
 but it is designed to become one applitude when the Fenfire system
 is complete. The RDF graphs written by the application
 will be importable into Fenfire.
 
-Explicit support for taxonomic hypertext (`Parunak 1991`_)
-and hierarchies is currently being planned.
+    Let us briefly explore how a user could extend
+    this applitude using the techniques presented in
+    `Section 2`_.
+
+    First, metadata about articles, such as author
+    and publication date, could be represented through simple
+    RDF connections. Using our RDF views, one could then
+    for example browse the list of articles of one particular author,
+    sorted by date.
+
+    Similarly, it is possible to switch from spatial hypertext (`Marshall and 
Shipman 1995`_)
+    to taxonomic hypertext (`Parunak 1991`_) by replacing the canvases
+    categorizing articles by source with RDF metadata,
+    and browse the articles from one source related 
+    to certain subject matters sorted
+    by e.g. author or date.
+
+
+    Since the structure is open, it is easy to add new metadata
+    such as the DOI (Digital Object Identifier,
+    `Paskin 1999`_) of the articles to allow interoperability
+    with WWW services such as ACM's web site for, e.g.,
+    seeing new articles not yet entered to the system by the user
+    that cite the current article. This would be a good
+    way to interface our system to the Semantic Web `(Berners-Lee 1998)`_.
+
 
 .. _`Section 5`:
 
@@ -885,6 +991,11 @@
 structures based on spatial placements 
 of objects (`Marshall and Shipman 1993`_, `Marshall et al 1994`_).
 
+The approach in VIKI was later extended to non-linear
+views `(Shipman et al 1999)`_ and, in VKB, to support other kinds
+of media and navigational links `(Shipman et al 2001)`_
+and an agent approach `(Shipman et al 2002)`_.
+
 `Haake et al (1994)`_ separate the dimensions of "user defines"
 and "system uses" for how object types are embodied in a system. 
 The item-based systems are delocalized over this graph, just
@@ -892,6 +1003,23 @@
 For items, the types of the *associations* are more important
 than the types of the items themselves.
 
+A different kind of flexible structure, the Frame-Axis model,
+which focuses on structure that is based on *attribute values*
+of items, is
+introduced in `(Masuda et al 1994)`_.
+
+A non-spatial approach to coexistence of strongly typed
+and weakly typed information by using a special Semi Frame
+object type is given in _`(Furtado and Madeira 1998).
+A similar approach which allows *incremental formalization*
+of data is given in _`(Shipman and McCall 1999)`.
+
+Recently, `Kim (2002)`_ proposed to use a graph-based model
+as the basis for Engelbart's proposed Open Hyperdocument System,
+to allow several underlying data structures to be combined.
+This is essentially a proposal for using a hyperstructure,
+even though Kim does not discuss visualization.
+
 
 5.2 Structural computing 
 ------------------------
@@ -948,6 +1076,57 @@
 RDF vocabularies, a data conversion tool could convert the
 information into the vocabularies expected by either view.
 
+Also, schema languages developed for the Semantic Web
+could find applications in Fenfire. For example, if
+the system knows that every person has an address, it can
+automatically create an address node when a person node is
+created; the user then only needs to "fill in the blanks."
+
+
+5.4 Visualizations of graph structures
+--------------------------------------
+
+Work on graph visualizations has mostly concentrated on *graph drawing*, 
+laying out graphs on 2D planes or hyperbolic planes while minimizing
+crossing edges and possibly under other criteria. While graph drawing
+has opened several interesting problems in theoretical computer science,
+we are more interested in a more Focus+Context (`Furnas 1985`_) 
+approach, 
+due to the size of the graphs we are dealing with as well as their
+structure which appears to be badly suited for plane drawing, owing 
+to the large numbers of interconnections.
+
+Other RDF visualization packages we have found are
+of the graph drawing type: RdfViz (XXX) and IsaViz (XXX), both based
+on the same 2D plane graph-drawing package (GraphViz, XXX) ,
+and
+Ontorama (`Eklund 2002`_)
+uses a hyperbolic-like planar visualization.
+
+Work on structure-based focus+context visualizations of graphs was
+more popular with semantic networks in the late '80s, for
+example in the Cyc (XXXCite) and SemNet (XXXCite) projects, which allowed the 
user 
+to navigate through the graph locally, with one node as the *focus*,
+and showing the neighbouring nodes as context.
+
+Later, a similar style of navigation and structural
+editing was also used in the commercial
+product TheBrain(XXXCite), to allow users to organize files 
+and items on their computers.
+
+One of the most important things that is different in the visualizations
+presented in this article and the ones for example shown in 
+(`Utting and Yankelovich 1989`_) is that icons are not used
+to represent the neighborhood of an item or a document.
+In FenPDF, the academic articles are never represented by 
+anything other than their image - no file icon, no title
+but just the text image itself. With the unique backgrounds, which
+allow the user to recognize fragments from articles they 
+know well, this
+yields in our experience an extremely comfortable system to use,
+avoiding the representational complexity of the icons.
+
+
 
 5.5 Open Hypermedia
 -------------------
@@ -961,8 +1140,10 @@
 focusing on hyperlink interoperability between existing
 applications (`Pearl 1989`_).
 
-However, the FOHM model (`Millard et al 2000`_) is an interesting
-step towards a similar approach: the FOHM model combines the different
+However, Callimachus (`Tzagarakis et al 2003`_)
+and the FOHM model (`Millard et al 2000`_) 
+are interesting steps
+towards a similar approach: they combines the different
 hypertext domains into a single conceptual structure.
 
 
@@ -1024,7 +1205,7 @@
 in your computer around *them*.
 
 That's why we believe that this will be
-"the next big thing." (You will not want to go back.)
+"the next big thing."
 
 
 Acknowledgments
@@ -1085,17 +1266,22 @@
   treatment of PS/PDF files
 - the Fenfire codebase
 
-
+We'd like to thank the anonymous reviewers for their comments.
 In addition, we'd like to thank (alphabetically)
 Toni Alatalo,
+Katariina Ervasti,
 Tuukka Hastrup,
 Hermanni Hyytiälä,
 Antti-Juhani Kaijanaho,
 Janne Kujala,
-Matti Katila 
+Matti Katila,
+Rauli Ruohonen,
+Asko Soukka,
 and
-Asko Soukka
-for discussions.
+Kimmo Wideroos
+for discussions, and we'd like to thank Christel
+Fallenstein for providing us with the real-world
+example presented in `Section 3.3`_.
 
 This work was supported by the InBCT 2.1 project.
 
@@ -1153,6 +1339,11 @@
 **Furnas, G.W.,** (1985) "Generalized fisheye views".
 ACM CHI'86 Proceedings, 16-23.
 
+.. _`Furnas 1997`:
+
+**Furnas, G.W.,** (1997) "Effective View Navigation".
+ACM CHI'97 Proceedings, 367-374
+
 .. _`Haake et al 1994`:
 
 .. _`Haake et al (1994)`:
@@ -1234,6 +1425,11 @@
 Partially available online at
 ``http://xanadu.com.au/ted/TN/PUBS/LM/LMpage.html``
 
+.. _`Nelson 1995`:
+
+**Nelson, T.H.,** (1995) "The heart of connection: hypermedia unified 
+by transclusion". Communications of the ACM, Vol. 38, No. 8.
+
 .. _`Nelson 1999a`:
 
 **Nelson, T.H.,** (1999a)
@@ -1299,6 +1495,12 @@
 **Thüring, M., Hannemann, J., and Haake, J.,** (1995)
 "Hypermedia and Cognition: Designing for Comprehension".
 CACM 38(8) August, 37-66.
+
+.. _`Tzagarakis et al 2003`:
+
+**Tzagarakis, M.,  Avramidis, D.,  Kyriakopoulou, M.,  Schraefel, M.M.C.,  
Vaitis, M.,  and Christodoulakis, D.,** (2003)
+"Structuring primitives in the Callimachus component-based open hypermedia 
system".
+J. Netw. Comput. Appl. 26(1), 139-162.
 
 .. _`Zellweger et al 1998`: