EXPERImonitor change log

v2.2

Main changes

Experiment explorer user interface

In addition to the existing metrics based views, EXPERImonitor now comes with a data explorer interface that allows you to:

  • Explore experiment participant interactions with your experimental system (using captured provenance data)
  • Visualise quality of experience measurements associated with experiment participants
  • Explore relationships between:
    • User experience
    • User activities and application usage
    • User activities and service usage
    • Service usage and service QoS metrics

For further information on the use of this new UI, please consult the documentation.

ECC is renamed EXPERImonitor

In this release we renamed the ECC to ‘EXPERImonitor’.

Summary of v2.2 changes

The list of major new features, updates and bug fixes is as follows:

Change Description
New feature: Provenance explorer view added Provides rich visualisations linking different metric entities together through provenance data.
New feature: Provenance demo clients added to samples Clients ‘lwtECCClient’ and ‘experimentSimulation’ generate provenance & metric data to explore.
New feature: Metric API now includes metric meta-data Client writers can now specify meta-types and meta-content in their model. Please see documentation.
Update: ‘ECC’ now renamed ‘EXPERImonitor’ A clearer name reflecting use of the software beyond the EXPERIMEDIA project.
Update: Provenance API updated for client writers Clean-ups and improvements made to the provenance API. Please see documentation.
Update: Android client code simplified & bug-fixed Android client code now simplified using updated API. Push metric failure on emulator fixed.
Update: Experiment data export now uses ISO-8601 date labelling The experiment data export sample now creates folders using ISO-8601 like date formatting.
Update: Further documenation on EXPERImonitor usage scenarios The documentation has been extended to include experimental scenarios and recommendations for API use.
Bug fix: Access to online FOAF ontology data stops EXPERImonitor FIXED: All ontologies loaded by default at the start of an experiment are loaded from local resource.
Bug fix: Client UI not updated after client disconnect action FIXED: Clients now appear in correct drop-down list after they have been disconnected.

v2.1

Main changes

New web user interface and new features

The web interface has been completely replaced. The new interface is more reliable and supports multiple simultaneous users viewing the same experiment. It incorporates more data export options and allows the user to return to previously completed experiments.

Provenance data integration with OpenRDF services

This version of the ECC release uses OpenRDF web services to store and retrieve provenance data created in experiments. This means that some additional libraries are required in order to build and deploy the ECC. These are:

Library Description
owlim-lite-5.4.jar Required to build the ECC API and must be added to the binary distribution
openrdf-sesame.war Required to be deployed in either Tomcat or Glassfish before the ECC is installed
openrdf-workbench.war Optionally deployed in either Tomcat or Glassfish to support browsing of raw RDF data generated by the ECC

For licensing reasons, these files cannot be distributed with the ECC. To obtain these, please register with Ontotext by visiting http://www.ontotext.com/owlim/owlim-lite-registration. For more information, see the README in the ‘thirdPartyLibs’ folder.

Move to Java 7 and Maven 3

Updates to the ECC API have required a move to version 7 of the Java SDK. We have also moved to using Maven 3 for improvements to the build process.

Change to EDM database schema

We have slightly modified the EDM database schema to provide greater flexibility in metric modelling (by allowing measurement sets from any group to point to the same attribute instance). This means that if you work with a manually deployed ECC you should delete your existing ECC database and create a new one (using the updated script in this version of the API). ECC V2.1 systems deployed with Vagrant will automatically create the updated database. This change has no effect on the client API you use to connect to the ECC or the structure of the data stored in the PostreSQL database.

Summary of v2.1 changes

The list of major new features, updates and bug fixes is as follows:

Change Description
New feature: Multi-user support for ECC dashboard The ECC dashboard now allows multiple users to view the same experiment at run-time
New feature: New ECC dashboard look and feel The ECC dashboard has been updated using a new web application development framework that provides a more responsive and reliable UI for users
New feature: Selective metric export It is now possible to selectively export metric data: scopes include full experiment; per client; per entity; per attribute
New feature: View and export old experiments The ECC dashboard now allows users to visit previously run experiments and export the data as a CSV file
New feature: ECC database export application Allows you to export all experiments contained in the Postgres database to files using CSV formet. See <ECC API root>/samples/databaseExport/README.txt for more information.
New feature: OpenRDF Sesame service integration The ECC service now uses (and requires) the OpenRDF Sesame service to store provenance data
New feature: OpenRDF Workbench deployment We recommend installing the OpenRDF workbench service alongside the OpenRDF Sesame service so that provenance repositories can be queried
New feature: Time zone now displayed for experiment info The UI now presents the time zone offset from UTC in the experiment information summary
Update: Java 7 and Maven 3 now used  
Update: Database constraint removed The EDM database schema now allows measurement sets from any group to point to the same attribute instance
Update: C++ API now uses updated 3rd party AMQP libraries Updated 3rd party library include bug fixes; additional refactoring has also improved thread safety for this API
Update: ECC API now depends on separate OWLimStore jar The owlimstore code has been separated from the ECC and is retrived by maven from the IT Innovation Maven repository
Update: Move to version 7 of the JDK and Maven 3 Updates to the ECC API have required a move to version 7 of the Java SDK. We have moved to using Maven 3 for improvements to the build process.
Update: New visualisation for live metrics All metric visualisation have been updated and are now presented in a ‘metric wall’. Ordinal metrics are presented in a time-series
Update: Switch to slf4j and logback logging The ECC service now uses slf4j and logback to log system status and events enabling better logging in general and logging on Android clients
Update: Vagrant support for Tomcat and Glassfish deployments You can now use Vagrant to deploy the ECC service in a Tomcat 7.x or Glassfish 3.1.2.2 environment
Bug fix: ECC crash if RabbitMQ server is restarted FIXED: If the RabbitMQ service is restarted (removing all previously created exchanges) the ECC could no longer use the exchanges it created at start-up.
Bug fix: Ensure existing ECC client reconnects after ECC restart FIXED: clients did not properly reconnect to the ECC once it was restarted.
Bug fix: Fixes entity selection crash in DynamicEntity client FIXED: This client could be made to crash by trying to enable/disable unselected entities

Known issues

The known issues with this release are:

Issue Description
Vagrant deployment in Linux: openrdf-workbench When using vagrant to deploy the ECC from a Linux host a port-mapping problem renders the openrdf-workbench service unavailable to the host machine. To work around this, map Vagrant port 8080 to host port 8080 (if it is free). This is a non-critical problem that does not affect the deployment of the ECC or its use of the openrdf-sesame service.

v2.0

This version of the ECC now offers:

  • Improved Vagrant support for ECC re-deployment
  • Improved ECC dashboard logging:
    • ECC logs are now unified in a single log file (ecc.log)
    • A single log4j configuration file can now be found in the ‘WEB-INF/classes’ folder

If you are upgrading from V1.2, please take care to note the changes in V2.0-beta1 (below) as these also apply. As with V2.0-beta1, there is (currently not fully documented) support for Provenance modelling in the ECC client API.

v2.0-beta1

This updated ECC dashboard and API now provides better support for client connectivity over the course of a series of experiments. Given a running RabbitMQ server, experimenters can now use the following features:

  1. Start ECC clients before starting up the ECC dashboard or creating a new experiment
  2. Run clients continuously between experiments without needing to explicitly re-start/reconnect their clients (particularly useful for ECC clients that are services themselves)
  3. Shut down and then restart the ECC dashboard – clients that did not disconnect themselves during this time will engaged in the next new experiment

Please note that [1] will work for v1.2 clients but features [2] and [3] are only available to ECC clients that are re-compiled against the new V2.0-beta API and use the V2.0-beta dashboard (see option 3 below).

For users intending to use the V2.0-beta1 dashboard, please note two important changes:

  • Our database schema has updated slightly (no impact on metrics data)
  • The client <-> ECC messaging protocol has changed slightly

Deployment This means when deploying the ECC dashboard, you must run the schema set-up script (if you have an existing database, back this up first).

During experimentation When an experiment is ended in the dashboard (or the ECC is shutdown) clients will no longer automatically receive a disconnection message. If you leave your current code unchanged, you will need to manually disconnect and then re-connect your ECC client for each new experiment. More details for what this means under various scenarios is provided below.

Option 1: Keeping using V1.2 client API

Dashboards you can use: V1.2, V2.0-SNAPSHOT, V2.0-beta1 Code changes: none.

If you choose to run the latest dashboard (V2.0-beta1) with your V1.2 client, then your client will no longer receive a disconnect message so may have to be manually halted and then reconnected. If you do not halt your client it will be partially initialised by the ECC dashboard (and appear as a connected client) at the start of the next, new experiment - it will not, however, be able to send further metrics. Re-start and reconnect your client to fix this.

Option 2: Keeping using the current V2.0-SNAPSHOT client API

Dashboards you can use: V2.0-SNAPSHOT, V2.0-beta1 Code changes: none.

Note this client API includes basic PROV support.

Exactly as above described above: if you choose to run the latest dashboard (ECC V2.0-beta) with your current V2.0-SNAPSHOT client, you will need to disconnect and re-start your client manually after each experiment has completed (this dashboard will not send a de-registering message to your client after an experiment is over).

Option 3: Update your client to ECC V2.0-beta changes

Dashboards you can use: V2.0-SNAPSHOT, V2.0-beta1 Code changes:

  • Re-build your client against new API is required
  • You must ensure your create a new metric model for each new experiment
  • Minor package name refactors in the EDM specification package
  • Minor PROVENANCE API create/get method changes

Re-build your client You must re-build your code against the new ECC API version.

You must ensure your create a new metric model for each new experiment With the previous pattern of behaviour, clients would be created and connected for each experiment and upon connection the ECC would ask for the client’s metric model. Now that a client can remain connected to the RabbitMQ server between experiments, clients must be prepared to re-send their metric model each time a new experiment is started in the ECC (during the ‘Discovery’ phase: in response to the ‘onPopulateMetricGeneratorInfo()’ event). In this case, we recommend you re-create an entirely new metric model (new UUIDs will be generated automatically for all model elements). Note that it is recommended that any additional resources directly linked to your metric model should be re-created/updated as necessary.

You also have the option of re-using Entites between experiments. To do this, follow these steps:

  1. Create a new Metric Generator and metric group for the new experiment
  2. Add the Entities you wish to re-use to the generator
  3. Create and map new Measurement Sets to the appropriate Attributes in the usual way

Minor package name refactors Unless you use our metric database locally, these changes will not affect you:

  • Maven artifact <artifactId>experimedia-arch-ecc-edm-impl</artifactId> is now called <artifactId>experimedia-arch-ecc-edm-impl-metrics</artifactId>
  • Package uk.ac.soton.itinnovation.experimedia.arch.ecc.edm.spec is now uk.ac.soton.itinnovation.experimedia.arch.ecc.edm.spec.metrics
  • Package uk.ac.soton.itinnovation.experimedia.arch.ecc.edm.spec.mon.dao is now uk.ac.soton.itinnovation.experimedia.arch.ecc.edm.spec.metrics.dao

Minor PROVENANCE API create/get method changes If your client uses the PROVENANCE API, be aware that EDMProvFactory ‘getOrCreate’ method calls have been split into separate ‘create’ and ‘get’ methods. You must always create Entities, Agents and Activities; if you wish to retrieve them from the EDMProvFactory you should use the appropriate ‘get’ method.

A few examples of such changes can be seen in our sample clients:

  • BasicECCClient: Cleared old metric model when experiment starts (see ECCClientController.java, line 132)
  • PROVECCClient : Moved metric/provenance model creation from construction to when experiment starts (see ClientController.java line 372)
  • HeadlessClient: Moved measurement task scheduling from constructor to when experiment starts (see ECCHeadlessClient.java line 209)

v1.2

Below is a list of significant changes to the ECC API found in version 1.2.

Change Description
Added ECC shut-down confirmation dialogue Checks that the experimenter really wants to shutdown the ECC and reminds them of data export functionality
Added C# client support Client writers can now use Microsoft’s C# development platform to develop ECC clients
Updated to Vaadin 6.8.10 framework Internal update to the web application used to run the ECC dashboard (includes ICE push framework) - does not impact client side development
Additional visualisation of metrics during live monitoring The ECC dashboard now offers histograms for nominal and ordinal metric types during live monitoring
Added dynamic entity support ECC clients can now dynamically declare Entities + attributes/new measurement sets at any stage during an experiment
Added entity ‘enable/disable’ support ECC clients can now tell the ECC to enable/disable specific entities during live monitoring; metric data for disabled entities is no longer pulled/accepted from a push
Added dynamic entity example sample An example of how declare new entities/measurements and enable/disable them was added to the ECC sample client collection
Added C++ client support Client writers can now develop C++ ECC clients (requires Boost; cmake; RabbitMQ C; RabbitMQ C++ wrapper library)

v1.1

Below is a list of significant changes to the ECC API found in version 1.1.

Change Description
Clients can connect to experiment at any time ECC clients no longer have connect during the discovery phase of an experiment, but can do so at any time.
Added additional Entity/Attribute query functions in MetricHelper ECC client writers can now use the MetricHelper class to perform searches on Entities/Attributes/MeasurementSets
Updated dashboard implementation Updated ECC dashboard implementation that fully implements all experiment phases; makes live monitoring of metrics easier & makes deployment simpler
Metric data export added Experimenters can now export metric data held by the ECC at run-time to a CSV file for external analysis
Modified time-stamp standard for data export Changed the time-stamping of exported data sets to ISO-8601
Added measurement rules for ECC to follow during live monitoring Clients can now specify (for each measurement set) how quickly the ECC requests data from the client and how many times during an experiment
Added Android support for ECC client writers The ECC API was modified to enable client writers to build for the Android platform

v1.0

Below is a list of significant changes to the ECC API found in version 1.0.

Change Description
Surefire tests added under a configuration profile EM and EDM libraries now contain JUNIT tests that can be run using the following command: mvn test –P test. You will need to have a locally running RabbitMQ/PostgreSQL service running (respectively) for these tests to complete successfully.
Sphinx documentation started In the next release of the ECC API, all documentation will be maintained in Sphinx format under the ‘doc’ folder. See doc/README.txt for further information.
ECC snapshots on-line Snapshots of the ECC API will be periodically uploaded to IT-Innovation’s barooga server (barooga.it-innovation.soton.ac.uk).
EDC charms added The follow Juju charms have been added to the ECC component: RabbitMQ; PostgreSQL; ECC web dashboard; WeGov client; Headless client
AMQP connection method update EM property file now supports keys ‘username’ and ‘password’ for non-default connection to a RabbitMQ server. The AMQPConnectionFactory class will use this information, if it is available. Sample client code has been updated to demonstrate the use of this functionality.
Updated EM JUNIT test cases The EM test module has been refactored and updated to include further AMQP test cases (including corner-case and performance tests).
Updated EDM JUNIT test cases The EDM test module has been updated to include addition tests for storage/retrieval of: entities, metric generators and reports.
Added experiment ‘restart’ support Experiments can be re-started using the JDesktop ECC container application. Connected clients will be sent a disconnection message and the experiment process will reset to wait for new clients.
Web based ECC dashboard available A web based view of the ECC is now available as a WAR that should be deployed in the root of an Apache TomCat server. Local RabbitMQ & PostgreSQL are also required.
Updated EMIAdapterListener ECC clients can now use an updated EMIAdapterListener class; this provides additional experiment information; disconnection notification support; extended support for phase and push/pull behaviour description; time-out event notification.
EMILegacyAdapterListener added For client writers who wish to test their V0.9 code against V1.0 binaries, a legacy listener class has been added to shield V0.9 code from data/event changes found in V1.0 (these are simply not exposed to old V0.9 code).
EMClient class updated The monitor based class ‘EMClient’ now maintains state about its Post-Reporting activities.
EMDataBatch class updated The data batch class has extended semantics regarding expected and actual data gathered from ECC clients (during a request from the ECC during Post-Reporting phase). Batches also now encapsulate data as a Report.
‘Headless’ client sample added An additional sample has been added that runs as a client without a GUI. Additionally, this client demonstrates: Property file-based connection configuration for ECC connection; SSL based secure connection to the ECC; Use of the ECC AgentEDM API to locally store metrics; Use of the ECC AgentEDM API to retrieve metrics for the ECC; Post-reporting phase support (collection of unreported metrics during Live Monitoring); Use of the shared samples classes to support automatic (background) scheduling of metric based measurement.
MetricHelper class added Client writers can now use the MetricHelper class (see the metric data model package) to assist them in organising metric model classes.
UI state fixes to the JDesktop ECC Container A number of fixes relating the presentation of experiment state, client connection status, and available entities/metrics have been made to the ECC Container application.
EM/EDM property files now used the JDesktop ECC Container The JDesktop ECC Container now picks up EM and EDM configuration properties from local files em.properties and edm.properties respectively.
Updated EDM database schema The schema used to stored experiment/metric data has been updated to support the V1.0 data model. Old V0.9 schemas should be removed.
EDM support for ‘synchronized’ data The EDM can now mark specific reports/measurements as ‘synchronized’ with the ECC: clients should consider using this when they receive report acknowledgement messages from the ECC during Live Monitoring.