Job table contents should be updated at regular intervals

[olle]

Job table contents should be updated at regular intervals. The update should not interfere with other actions the user might perform, e.g. configuring the table, using the menus, scrolling the table, or inspecting non-default tabs, such as the one with the job queue table. Both the job table and job queue table should have their contents updated. The regular updates should be performed with an interval of 5 seconds in the first version.

[olle]

Ticket accepted.

[olle]

Background information:

The reason that the web page with the job table is a good candidate for automatic regular updates, is that much of the displayed data might change without any direct action of the user. The data that might change is the following:

1. The job table columns with headers "Status", "PercentComplete", "Started", "Ended", and "StatusMessage".
2. The job table window sub-title indicating the number of pending jobs.
3. The job queue table columns with headers "Own", "All", and "Others".

[olle]

Design discussion.

The web page with the job table, like most web pages in Proteios SE, is managed by JavaScript file www/static/js/script.js in client/servlet/. It is therefore natural to let this script handle regular updates of the web page.

The table tool bar already contains an "Update" button, which when clicked will trigger a complete refresh of the web page. It might therefore be tempting to let the script also activate a complete refresh of the web page at regular intervals. However, this turns out to be a far from ideal solution. In contrast to when a user clicks the "Update" button, an automatic update might occur when the user is performing some other action, where a full refresh of the web page is undesired. This happens when the user does some of the following:

1. Has opened the pop-up window for configuration of the table.
2. Has selected another tab than the default one, such as the tab for the job queue table.
3. Uses some Proteios SE menu.
4. Has scrolled the job table, in order to inspect data in columns to the far right, that normally fall outside the window.

A full refresh of the web page of the type triggered by clicking the "Update" button, abruptly cancels the previous user mode, and returns the web page to the default state. Since actions 1 and 2 above are managed by the JavaScript, these cases can be handled by the script using flags to indicate when the configuration window or alternative tab are used, and refrain from updating the web page, when any flag is set. However, cases 3 and 4 above are harder to find a simple solution for.

The problems related to a full web page refresh at regular intervals, suggest that a better solution is to only update the table contents that might change on a short time scale, but leave the other HTML contents intact. This can be done using Ajax, whereby the JavaScript uses an XMLHttpRequest in the background to obtain the fresh data needed to update the job table. This will therefore be the basis for the design.

[olle]

Design description.

Design overview:

• A JavaScript can most easily update web page GUI elements, that have unique id values. Currently, ordinary table cells do not have any id associated with them, but that can be fixed in class gui/web/GUIConverter in client/servlet/, and restricted to the job- and job queue tables. Since the job table rows already have id values equal to the database id for the job items they represent, a natural choice is to let a cell in column with header "Status", in row with id "209", get id value "Status_209", which will uniquely identify the cell in the job table.
• The JavaScript has access to the HTML code for the job table in the web page, and can retrieve id values for the rows, that equal the database id values for the corresponding job items, for which information is shown on the current page. These id values will be sent to a servlet using an XMLHttpRequest.
• The servlet will obtain fresh data for the job items in question, and send it back to the JavaScript function in JSON format.
• The JavaScript will use the data in the JSON object from the servlet to update the information in the GUI elements of interest (table cells and job table window sub-title).

Design details:

• Update of the JavaScript www/static/js/script.js in client/servlet/:
  a. The window.onload function is updated to check if the document is the job form, in which case new function startRegularUpdates() is called.
  b. New function startRegularUpdates() will call new function updateJobTableColumns() with 5 second intervals.
  c. New function updateJobTableColumns() will call new function getJobIdList() to get a comma-separated list of id values for job items shown in the job table on the current web page. The id list will be transferred to new servlet action/job/ListJobsAjax.java in client/servlet/, and the latter will return a JSON object containing data, that are used to update selected parts of the web page contents.
• New servlet class/file action/job/ListJobsAjax.java in client/servlet/ will retrieve fresh data for the job items with id values in the received list, and the data will be sent back to the calling JavaScript function in JSON format. The outermost JSON object will contain a JSON array of update data, each packaged in a JSON object with id value for the element as key, and update data as value.

Some of the benefits with this design:

• Only information of interest needs to be sent between client and server, instead of the full job table. This reduces bandwidth use.
• The JavaScript can update table cells only using their id values, and does not need any information concerning what columns are visible, their order, how data is sorted, or what filters are used. All the latter aspects of the table are just as previously managed by class ConfigureTableFactory2, that has not been modified.
• Ajax has the benefit that if the servlet crashes, only the JavaScript will be affected by the response, which normally only will result in no automatic update of the table contents being made.

[olle]

(In [4410]) Refs #797. First version of regular updates of job table contents:

1. New library file json_simple-1.1.jar in api/external/ added.
2. JavaScript www/static/js/script.js in client/servlet/ updated with new functions to update selected GUI elements in job table web page with fresh data obtained from new servlet ListJobsAjax using XMLHttpRequest.
3. New servlet class/file action/job/ListJobsAjax.java in client/servlet/ added. It retrieves a comma-separated list of id values, and obtains fresh data for the corresponding job items. The data is returned to the caller in JSON format.
4. Class/file gui/web/GUIConverter.java in client/servlet/ is updated to set id values for window sub-titles and table cells in job and job queue tables.

[olle]

(In [4417]) Refs #327. Refs #797. Compiler warning for unchecked calls in class/file action/job/ListJobsAjax.java in client/servlet/ suppressed with Java annotation @SuppressWarnings("unchecked").

[olle]

Design discussion:

The current Ajax implementation leads to problems, as Ajax way of making requests in the background interferes with assumptions made for how actions in Proteios SE should behave. The problems occur in the "Jobs" page, when one selects menu item Edit->"Empty Trash", or checks one or more job items and clicks one of the buttons "Put job in high priority queue", "Delete", or "Abort". This will lead to a completely blank page, except for the text {"message":null,"status":"error"}. Inspection of debug output in log files show that the intended action servlet (EmptyTrash.class, PutJobsInHighPriorityQueue.class, DeleteJobs.class, AbortJobs.class), actually was called, but since these action do not set a forward action, the last action is invoked. The problem is that if an Ajax request has been made, the last action will be ListJobsAjax.class instead of ListJobs.class, and the former sends a response that is not formatted as HTML code for a web page.

Possible solutions:

1. Modification of way a ProteiosAction and action links work. The root of the problem is that an XmlHttpRequest should not be treated as an HttpRequest, but the code in question is the base for most functionality in Proteios SE, and changes to it is therefore undesirable, if it can be avoided.
2. A forward action can be set in the affected action classes. However, this is undesired, since new actions for the job table also will need this fix.
3. Modification of action ListJobsAjax.class. Although not desirable in principle, as this will introduce functionality not related to proper Ajax requests, this seems to be the simplest solution. Action ListJobsAjax.class receives the misguided requests from all other actions, so a fix can be confined to a single class. Since all the misguided requests were intended to action ListJobs.class, class ListJobsAjax should simply forward them there. The only remaining problem is identifying the misguided requests. These requests all lack a "jobIdList" parameter, but relying on this can be hard, since a job table can be empty, and we do not want a valid Ajax request to lead to a full update of the "Jobs" web page (this might e.g. interfere with the user looking at the menus). It is therefore better to use the currently unused "cmd" parameter to identify all valid Ajax requests, and redirect all other to ListJobs.

[olle]

(In [4419]) Refs #797. Regular update of the job table via Ajax updated to not interfere with actions on the "Jobs" web page, that do not set a forward action, but rely on being returned to action ListJobs as the last action:

1. JavaScript www/static/js/script.js in client/servlet/ updated in function updateJobTableColumns() to add a parameter cmd to the Ajax request, and set its value to "updateJobTable".
2. Class/file action/job/ListJobsAjax.java in client/servlet/ updated to identify a proper Ajax request by requiring that parameter cmd has value "updateJobTable", and otherwise forward the request to action ListJobs.class.

[olle]

(In [4420]) Refs #797. Class/file action/job/ListJobsAjax.java in client/servlet/ updated by removing unused imports.

[olle]

Design discussion:

Inspection of the code shows the following chain of events, when class ProteiosAction gets the last action, when no other action is set:

1. Class ProteiosAction calls ActionFactory public synchronized method ProteiosAction getLastAction(Event event) to get the last action, when no other forward action is set.
2. ActionFactory public synchronized method ProteiosAction getLastAction(Event event) makes a request for the action id stored as session attribute "previous.action.id" for the event.
3. Session attribute "previous.action.id" is set for an event by a call to ActionFactory public method void setLastEvent(Event event), that obtains an action id be calling Event public method AbstractAction getLastAction() and sets it as session attribute "previous.action.id" for the event.
4. Event public method AbstractAction getLastAction() returns the last item added to private attribute ArrayList<AbstractAction> actionChain, that is updated with a new action in public method void go().

It would be possible to mark an Ajax request by adding a specific request parameter, that prevented the action to be returned in a call to Event public method AbstractAction getLastAction(). However, class Event also calls this method to obtain the action to run, so this would prevent the Ajax action from ever being executed. A possible solution is to add a new public method AbstractAction getLastNonAjaxAction(), that would return the last action in the queue, that is not marked as being an Ajax request, and let ActionFactory method void setLastEvent(Event event) call this instead of AbstractAction getLastAction().

Design update:

1. JavaScript www/static/js/script.js in client/servlet/ updated in function updateJobTableColumns() to add a parameter requestType to the Ajax request, and set its value to "Ajax".
2. Class/file se/lu/thep/waf/Event.java in api/waf updated with new public method AbstractAction getLastNonAjaxAction(), that returns the last action in the queue, that is not marked as being an Ajax request.
3. Class/file action/ActionFactory.java in client/servlet/ updated in public method void setLastEvent(Event event) to call new Event public method AbstractAction getLastNonAjaxAction() instead of AbstractAction getLastAction(), when obtaining an action id to set as session attribute "previous.action.id" for the event.
4. Class/file action/job/ListJobsAjax.java in client/servlet/ updated to take no special action depending on the value of the cmd parameter, and make a log entry snd return if the value of the requestType parameter differs from "Ajax". The changes to this class are strictly not necessary for the new functionality to work, but the log entries are used to check if the action is used, when not intended.

[olle]

(In [4421]) Refs #797. Regular update of the job table via Ajax updated to prevent the Ajax action from being used as "last action", in case no forward action is set:

1. JavaScript www/static/js/script.js in client/servlet/ updated in function updateJobTableColumns() to add a parameter requestType to the Ajax request, and set its value to "Ajax".
2. Class/file se/lu/thep/waf/Event.java in api/waf updated with new public method AbstractAction getLastNonAjaxAction(), that returns the last action in the queue, that is not marked as being an Ajax request.
3. Class/file action/ActionFactory.java in client/servlet/ updated in public method void setLastEvent(Event event) to call new Event public method AbstractAction getLastNonAjaxAction() instead of AbstractAction getLastAction(), when obtaining an action id to set as session attribute "previous.action.id" for the event.
4. Class/file action/job/ListJobsAjax.java in client/servlet/ updated to take no special action depending on the value of the cmd parameter, and make a log entry snd return if the value of the requestType parameter differs from "Ajax". The changes to this class are strictly not necessary for the new functionality to work, but the log entries are used to check if the action is used, when not intended.

[olle]

Test:

• Buttons "Put job in high priority queue", "Delete", and "Abort", as well as menu item Edit->"Empty Trash", now work as intended in the "Jobs" page, and log output shows that class ListJobsAjax is not called, when not intended for an Ajax request.

[olle]

(In [4422]) Refs #797. JavaScript www/static/js/script.js in client/servlet/ updated in function window.onload to check if a element document.forms[0] exists, before accessing its name field.

[olle]

Ticket closed as data for existing rows in the job table now are updated regularly with an interval of 5 seconds. However, in the current version, rows for new job items will only be added to the table after the "Update" button is clicked.