Re: [Help-librejs] Make LibreJS friendlier for web developers

[Kuno Woudt]

On Thu, Nov 6, 2014, at 11:39 AM, Nik Nyby wrote:
> Kuno Woudt <address@hidden> writes:
> 
> > The important thing IMO is to take this part of librejs and separate it
> > into an independent library, so it can be re-used by others to build a
> > command-line testing tool, or an online testing tool, or as a plugin for
> > some fancy IDE, or etc...
> 
> Yeah, thanks for clarifying this key point, Kuno. It would be useful to
> have the core LibreJS code refactored into a library that people could
> use however they wanted to. It won't be an easy task since the analysis
> code in LibreJS is designed so specifically to run with Mozilla's add-on
> API.
> 
> I think a good approach on starting an independent library for LibreJS
> would be to use the LibreJS code not directly, but as inspiration. Off
> the top of my head it seems really complicated. One question that comes
> to mind: 
>   - Will this library need to have a headless browser component, to make
> HTTP requests to web pages and all their JavaScript dependencies? If so,
> would jsdom or PhantomJS work?

I did some prototyping with PhantomJS about a year ago, I doubt it will
be of much use but if you're interested the repository is at:

https://github.com/warpr/jslich

The approach I tried to take there is to have to fairly separate steps:

1. Download the requested URL and try to gather all its javascript
dependencies, even those which might get loaded after the page has fully
rendered, because there may be a javascript based module loader still
loading modules.  Also download a javascript weblabel page if there is a
link to one in the starting HTML.

2. Take the stuff downloaded in the first step an analyse it.

Trying to implement step 1 does immediately demonstrate that this is a
difficult problem, because the page could have been engineered to
dynamically load further javascript based on user actions.  I realized
later that perhaps I was trying to solve the wrong problem.  As a web
developer I _know_ which javascript is included in the web application I
am writing, so perhaps it is better to just have a tool which takes that
information as input, instead of trying to figure it out with a headless
browser.

-- Kuno.
ps. My time is limited but I'm certainly interested in helping out
getting a tool like this implemented.