General:28 Aug 2010

From CWRC

AG meeting

Present: Tyson, Mike, Jonathan, Patrick, Susan

Update from Jonathan:

Jennifer has been consulting with Jonathan and working alongside him a bit; some things only she knows how to do; a lot of work on speedup; basic things like loading the graph and opening it; still slow and unresponsive when you are looking at the graphs

Timelines:

  • The short application proposed to have Jonathan spend part of the summer doing the improvements plus separate off components so you have back end processing and light weight front end; split first before moving to parallelization
  • Revised proposal due August 15th: when is starting time for SHARCNET part of the project and how long

Patrick on current tool:

  • usability: viz of all information at once is not v useful; can be done but needs to be visualized differently; not practical at this degree of resolution; not useful, and this is one of the major things slowing it down
  • use libraries to the parsing, other libraries could also speed things up
  • parse xml into relational dbase; cache e.g. first view
  • lots of places where improvements can be made to speed things up; but don't want to Frankenstein the code: smarter to look at it as a prototype and move to new version

MySQL clustering dbase w/ in memory clustering: worth consider that as an approach

Parsing library: LibXML2 allows you to stream in a file e.g. read in a 1TB file without having to have a TB of memory

Not trivial to incorporate a library but not huge: probably a couple of weeks' work

Question of whether it could really handle the relationship structures that the system needs to account for.

Not entirely clear on where the slowdown in the GUI; there's a newer version that is starting up more quickly than the one that Susan has been using; libraries could be used to speed up in a range of ways; libxml2 might be useful in the processing of user-define subsets

TO DO:

  • Susan to circulate and put up on wiki the use case scenarios
  • Mike to send Patrick a couple of examples of the challenges with relationships
  • Mike to send Susan Jennifer's list of improvements to the front end so we can prioritize them
  • Jonathan to get Susan a better newer faster version of the tool -- he can put it somewhere where I can grab it for my home directory, and tell me which UWO viz machine it would be best to shell into.

NEXT STEPS:

  • Meet via AG 1-3pm Mountain time/3-5 pm Eastern
  • Fork the code so that we can get a prototype for testing in the fall and continue working on the longer-term enhancements on the other version
  • Figure out what to propose for the Sharcnet application