Test DailyRdahmm Result Service set up

We have set up a test web service at http://156.56.104.161:8080/axis2/services/DailyRdahmmResultService?wsdl to calculate markers' colors for a given date. Also added is a "please wait" window to the portlet while the page is waiting for results from the service. Calling the web service to calculate the colors is actually faster than using managed bean, because now we don't have to refresh the page. The test page is located at http://156.56.104.161:8080/gridsphere/gridsphere, log in with empty user name and password.

We'll update this to gwll and svn after performance measure is done.

Daily RDAHMM video updated

Because of the changes of our treatment to missing data sections, and correction of several bugs, the historical results of Daily RDAHMM are also different from previously. So we just made a new movie for the whole time since 1994 based on the new results. The new video can be accessed from the portal at http://gw11.quarry.iu.teragrid.org:8080/gridsphere/gridsphere

Modifications to DailyRDAHMM missing data treatment

We did the following modifications to the treatment of missing data in DailyRDAHMM:

1. For the input data, if the dates used for getting the input is 1994-01-01 to 2006-09-30, and we only get data for 1995-10-03 to 2006-8-22, then we will only fill those time sections between 1995-10-03 and 2006-8-22, and leave the gaps before and after as they are, in order to avoid the distortion to results;

2. For the big flat file that contains input for all stations, we use NaNs to fill all missing lines.

Treatment to missing data corrected in daily RDAHMM analysis

A serious error of daily RDAHMM analysis has been corrected, concerning the way we treat the missing input data of stations. Previously we do nothing about the missing data but recording them, which may lead to incorrect RDAHMM results because the Hidden Markov Model assumes an even distribution of data across time. So now we correct this error by inserting "fake data lines" in to the missing-data time sections, which are duplicated from the available data at a most recent time relative to the missing-data time sections.

After this correction, the total number of stations with state changes dropped significantly (from around 100 on average to less than 30) during for the time from 2006 to 2008. Whether this new result is reasonable still needs verification.

Result Aggregator added to real-time RDAHMM service

A new component, real-time RDAHMM results aggregator, is added in to the real-time RDAHMM service. This aggregator connects to the service through NaradaBrokering, receives messages containing the real-time RDAHMM analysis results from all the seven networks, and aggregates these results and save them into a single .xml file, so that the portlet can show the results by accessing this file.

One problem is the large amount of results. Some stations came across as many as 800+ state changes within a 4-hour test run. So we just save one day's results in this file temporarily.

Daily RDAHMM porlet files updated

The big DailyRdahmm.jsp file is now split into three files: DailyRdahmm.jsp and two other utility javascripts, NmapAPI.js and dateUtil.js.

QuakeTables Access Problems

The QuakeTables portlets are IFrames that loads URLs like http://gf19.ucs.indiana.edu:9898/public/public.jsp and others (you can pop these out in most browsers by right-clicking).

Some departmental firewalls only allow HTTP traffic over ports 80, 8080, 443, or 8443. If you get a timeout message like the one below, the problem may be that you can't reach the QuakeTables web server's port 9898.

Disloc Plot Portlet Disabled

This portlet was not compatible with the reformatted KML output, so it has been temporarily disabled. Plots can still be made through Google Maps or Google Earth.

Map Interfaces for Fault DB

Disloc, Simplex, and GeoFEST now have Google map interfaces that allow you to select faults interactively.

All updates to daily and real-time RDAHMM portlets are availabel at

http://gw11.quarry.iu.teragrid.org:8080/gridsphere/gridsphere

Update to real-time RDAHMM portlet

We have changed the coloring plan of markers in the real-time RDAHMM portlet, so that now the colors describes the state change information about each station, rather than which network they belong to. Now we have three different colors: red for state change within last 2 hours, yellow for state change within last 1 day, and green for no state change within last 1 day.

We use red for last 2 hours because there is a lag between the time stamp of the real-time data and the "current" time when the data is received. For example, at 14:10pm, we might just receive a station's gps data with a time stamp of 13:05. Therefore, the evaluation results of a station always show its state change information of an hour ago or so.

In order to expose the state change information to the portlet, the real-time RDAHMM service saves this information for each network in a separate file in the same directory as the plots, so that the portlet can retrieve them with URLs.

To show which network each station belongs to, we add a hint tag to the marker of each station, witch shows its name and the name of its network.

Video making added to Daily RDAHMM

We have integrated the video making function to daily RDAHMM service and portlets.
The service will now create a new thread to make a video of the whole time period after doing RDAHMM analysis on all stations, and add an xml element into the result xml file, which gives a URL of the newly created video.

The portlet will show a link to this video, right below the link to the trace file of each day's number of stations with state changes.

Plot of state change numbers added to daily RDAHMM

A new plot function is added to the daily RDAHMM service. After doing RDAHMM analysis on all stations everyday, the service will draw a plot to display the variation of the number of stations with state changes through the whole time from 1994-1-1 to "today". We added new shell and gnuplot scripts to do this, and the service will generate a file containing the trace of the state change numbers as the input for these scripts.

On the daily RDAHMM portlet, a new tab is added to show this plot, as well as a link to the trace file, which contains the detailed information about each day's number of stations with state changes.

GeoFEST Portlets Fixed and Other Updates

The problems with the GeoFEST portlet updates have been fixed, and these portlets have been turned on. The updates are to resolve a build problem. The functionality should be the same. One problem is that the old GeoFEST outputs have been removed. The project database was not overwritten, so you can recreate outputs. However, the mesh and geofest results before today should be deleted.

I have also added a "News" (really, an RSS client) portlet to the guest layout. This can also be added to the logged in display. By default it displays the feed of this blog, but it can be edited to show other feeds.

GeoFEST Portlets Temporarily Unavailable

The GeoFEST portlets have been temporarily disabled while we upgrade them.

Fault DB Portlets Updated

Two additional interfaces to the QuakeTables Fault DB have been added:

1. A portlet for downloading InSAR images, and
   
2. A portlet for getting KML representations of the faults.

Portal Updates to Use New QuakeTables DB

Simplex, GeoFEST, and Disloc2 portlets have all been updated to connect to the QuakeTables database and Web service on gf19. This server is more stable than the older gf2, but the service and data should otherwise be the same.

Simplex updates

Two updates to Simplex:

1. The portlet was updated to show old projects in reverse time order (most recent is first).
2. Bug in the KML plotting of output was fixed. East and North stations were swapped, so arrows pointed in the wrong direction.

Disloc Executable Changes

Many output formats to 8.3f (decided that f was more appropriate than e; don't see much point getting precision below nearest meter).

Made to correct rounding problem in output file that produces different results for (0.0,0.0) and (0.0,-0.0).

QuakeSim Updates

Updates to portal:

1. Layouts updated. Disloc2 promoted to "Deformation Analysis" tab. Old disloc and meshgen portlets moved to "Obsolete Portlets" tab.
2. Updated map centers of various GPS and RDAHMM portlets.
   
3. Disloc2 scatter points can now be deleted.
4. Disloc2 maps use updated KML service that correctly styles faults and points.
5. Grid input points as well as scatter input points can now be viewed before submitting disloc.
6. RealTimeRDAHMM portlet updated to use the new infrastructrue.
   

Updates to services:

1. KML Service updated to do a better job scaling arrows and to correctly style arrows and faults.
2. Disloc service updated to use corrected version of disloc executable.
3. RealTimeRDAHMM services now classify real time data against trained data sets.