The specific goal
To invent, design, and develop a scoring system for the Bahamas Beauty Pageant which collects the scores and displays them on the national TV network. The system should be generic, i.e., usable for other competitions which are judged and where the scores are displayed real-time on TV. The system must be robust and reliable, and rely only on computers that can readily be rented on location in developing countries. The system must be so user friendly that judges without computer experience can use it without more than a few minutes of training or explanation. The system must support multiple competitions, with score integrations and score summaries, and complex scoring mechanisms - such as dropping high & low scores, etc.
System architecture, software engineering:
• The system must rely only on PCs running ordinary Windows (95 and up) connected through any type of LAN (most likely the lowest common denominator: peer to peer as supplied with Windows).
• It must be possible to install and configure the system within an hour or two, as venues for such events are heavily booked and do not always allow for much set-up time.
• The system must be installable from diskettes, as CDs are not universally provided in developing countries.
• The system must be very robust and dependable, as these are one-shot events, and the performance (and possible failure) of the system is highly visible (on national TV).
• The system must allow for real-time controls, since judging and display of scores on TV are tightly scheduled and choreographed.
User interface
• The score entry interface must be extremely simple and intuitive
• The scoring may be done by judges, some of whom have never used computers before, and who may be "computer-shy"
• The event organizers generally allow little or no time for training the judges on site and in context. There may be very little briefing as well.
• As the judgment is "one-shot", the system must allow the judge to validate the score and to recover from entry errors.
• The system must support the judges to conform to the timing and scheduling imposed by the event organizers and TV producers.
• Scores will be entered in distracting circumstances, with dim light etc. The judges may be drinking.
• The system must be protected against accidental or malicious interference by either judges or bystanders.
Scientific or technological advancement
A distributed component approach was designed which is much simpler and more robust than the DCOM approach, but still supports real-time transactions
• There are three components, a management component, a judges score entry component, and a TV output component.
• Both the judges component and the TV output component are under full control from the management component
• The judges component only uses the monitor and a mouse, but no keyboard - to reduce the chance of error and loss of control.
• The judges component has timing alerts and a time countdown - The transaction (score entry) is aborted and defaulted if the score is not entered on time. The timing cues are controlled from the management component and support the TV production scheduling. Default scores are not shown on TV
• The judge is warned with messages and count-down numbers
• The TV output component shows individual scores and summary scores across candidates for completed competitions.
• Countdown timing is done individually and independently by the judges components on each PC, but is triggered from the management component. Timing is accurate to within fractions of a second, and does not depend on the setting of the clocks on each PC (which may vary significantly)
• Only standard PCs are used, with basic network capabilities, that are locally obtainable. Configuration and installation is very quick. Slightly longer for the management component, which can be done ahead of time in approx. 2 hours.
Scientific or technological uncertainty
• The greatest uncertainty arose from the one-shot nature of the event. Everything had to work perfectly, with no chance to repeat or recover, as the judging for the pageant, and the TV broadcasting was live.
• Within competitions, transaction timing is critical, judgments have to be collected as candidates walk across the stage, so that scores can be displayed on TV, with calculated averages, when the camera focuses on them in their final stage position.
• The user interface has to work properly in reminding judges to enter, verify, and approve their judgment in time, even though they are distracted by the show and would like to use extra time to evaluate the candidate.
• The user interface for the judges has to time out and reset itself if the judge does not enter a score in time for a particular candidate, despite the reminders. Controls in remotely resetting and controlling the score entry displays have to work properly over the network.
• The management component has to work properly in waiting for the scores, applying the judgment algorithms (drop high & low, average, consider default scores ...) and sending it to the TV output component. Trade offs in in dealing with late judges and pressure from TV production have to be managed and supported. Potential message and file timing conflicts have to be resolved correctly.
• The
TV output has to overlay properly, with proper genlock synchronization.
Description of the work
• It was initially hypothesized that a traditional client-server approach with a data-base would be ideal for the job. Some experimentation showed that this approach would not be satisfactory, either from a system perspective or a user interface perspective.
• A three component design evolved after some experimentation. Experimentation focused on the messaging and transaction management. Shared file and shared memory approaches proved to be not sufficiently robust to handle timing conflicts. A small file information exchange plus semaphore system proved to be most reliable, when combined with a state transition - polling approach.
• There was some experimentation with the TV output. The most robust approach proved to be a separate PC with a remotely controlled display state, and with all system windows and message either defeated or set to the overlay colour so that they would not interfere.
• There was extensive experimentation with the user interface design for the judges. Since real judges and the real event were not available, experimentation involved subjects trying to be difficult and trying to "beat the system". Even at that they did not prove to be as difficult as some of the real judges proved to be.
• It was discovered very early that keyboards did not work in a dim room with slightly inebriated judges. Touch panels would be nice but are not easily available for rent in these locations. A remotely controlled with a mouse only interface proved to be the only solution, where the judge selects a score by clicking on large digits on the display. (Even with this one or two judges had to be shown how to slide the mouse rather than lifting it.) Different displays have to handle different types of competitions - e.g., numeric scores and text-category scores.
• There was more experimentation with reminding the judges when the scores have to be entered. If the scores are not entered in time they cannot be used, as the TV display goes up while the candidate is still on stage. Typically the next candidate is already walking onstage while the previous candidate is finishing. Finally a series of escalating reminders was devised, starting with text and icon reminders, and ending with bulls-eye count down to defaulting the score and resetting the display.
• There was some experimentation with the management component, since scores from previous, non-televised competitions need to be integrated into the summary judgments displayed on TV.
• The system was used for the Miss Bahamas Beauty Pageant in mid-August. Fortunately the system worked reasonably well, despite one or two difficult judges (inebriation was not anticipated), so that some scores were defaulted and dropped. The main flaw of the system design was in the final selection. The choreograph script from a previous competition, and the client specs all called for individual selections with a minimum time spacing between them. The script was changed at the last minute, and the final selection had the top three finalists on stage side by side with a very fast sequential judgment. The system proved awkward for this situation.
• With more experimentation after the fact, a new type of scoring panel was devised to allow side by side comparisons. This is quite tricky, since the finalists are not known until the last moment, and their relative positions are not pre-determined. The management component and the messaging system had to be altered to allow for this type of comparative judgment.
Design objectives:
A low to medium speed general process control system using readily obtainable and easily replaceable components. The system and the components must be easy and quick to set up and configure. The system must be robust, reliable, and relatively fault tolerant for one-shot or mission critical applications.
Low to medium speed process control refers to processes where both the minimum and the maximum time resolution is in seconds or minutes rather than in microseconds or milliseconds. Time resolution applies to the sampling rate for data acquisition, the modification frequency of output controls, and the time synchronization between sampling points and control points.
Readily
obtainable components refers to the use of general purpose PCs rather than PLCs
(programmable logic controllers) and other specialized instrumentation.
Quick configuration and easily replaceable components refers to the time and to the specialized skill required to set up the process control system, or to replace and configure components.
Robust, reliable, and relatively fault tolerant: Cheap components such as leased PCs can fail, or the power can be interrupted briefly. The system as a whole should stay up even if a single part fails - and it should quickly come back on stream after appropriate replacement, without having to reset and reconfigure all the other components.
Application opportunities for such a design
The immediate opportunity was the request to develop a scoring system for a beauty pageant in the Bahamas. For this application, the time constraint is approximately one quarter of a second. The score is acquired from judges and displayed on TV broadcast, overlaid with video of the contestant. Timing constraints for the output arise both from synchronization with the show, and the timing for the broadcast, including advertising etc. In general, the time synchronization does not have to be very close, e.g., not closer than within one quarter second.
A more long-range opportunity is the concept of delivering process control over the internet. One possibility being explored is the unobtrusive monitoring of elderly persons living alone. Other opportunities deal with more traditional process control applications, such as the remote monitoring of local process control applications. The local process control application can be fast and provide the usual controls. The remote process control monitors trends and tries to anticipate the requirement for human intervention.
Architecture
In general there are three types of stations: a data acquisition station, a management/control station, and an output control station. For the scoring system, data acquisition consisted in obtaining scores from the judges, so that there was 1 separate station for each judge. The output consisted of the signal to the TV network, so there was 1 output station. There was also 1 management/control station.
The platform: Hardware, OS, and Communication
For the initial platform, network enabled PCs were chosen, with Windows 95 or Windows 98, and with a peer to peer LAN network. The LAN could also be based on Novell Netware or NT, or any other LAN that supports drive mapping. NT is fine as alternative OS.
Alternatively, for remote (internet, WAN) applications, internet connected PCs can be used. The same process control approach can be used with FTP, and keep the present simplicity and robustness. Time resolution and time synchronicity present more of a problem with this approach because of widely variable communication delays.
In the future, for remote (internet) applications, a browser-based approach will be explored, but there are still issues about integrating interrupt handling and timing loops into browser scripting. If it proves to be robust, this approach reduces setup and configuration requirements to a near minimum.
Software components
For the beauty pageant scoring system there are 3 modules corresponding to the data acquisition component, the management/control component, and the output-control component. The bottom level: data exchange - timing synchronization layer is essentially common across the three modules. The business logic layer and the interface layer are distinct for each module. The management/control module is the largest and most complex, with approx. 10,000 lines of code in C, including embedded documentation.
The data exchange - timing synchronization layer
Each of the modules uses timing intervals, using the system clock and interrupts.
• the timing interval is approximately the same for each module
• most state changes in the application are controlled with the timing interval
• the timing intervals are not synchronous, since they are independent on each of the PCs, with different clocks, etc.
• the timing intervals check for file flags to force synchronicity. The duration of the interval thus determines the limits of synchronicity.
• the timing interval in conjunction with the small files allow for external control of the application
Each of the modules uses small files in shared directories to communicate with the other modules.
• some files are simply renamed, and the existence of the named file is used as semaphore.
• some files convey data, for configuration or other run-time controls, or for data processing
• after some experimentation it was determined that it was faster and more robust to use multiple files with single uses rather than fewer files with multiple uses.
• only a single module on a single station has write access - all others have read access only
This part of the system is relatively application independent, since it can be used for a variety of applications that do not have tight timing requirements but can advantageously use a process control approach.
This layer of the system represents the main technological advance, as well as the primary source of technological uncertainty. To the best of my knowledge all commercially available process control systems have much tighter timing, but cannot meet the requirements for inexpensive components and fast configuration.
The business logic layer: data processing - data storage & retrieval
This part is application dependent. The most interesting part, and the main technical advance of this component is the implementation of voting algorithms that can handle ties, and that avoid the intransitivities that are potential errors for adding / averaging algorithms.
Another
challenge was the detection of voting inconsistencies by single judges.
The interface layer: data acquisition - management/control - output control
There are two user interfaces and one system interface. The user interfaces dealt with data acquisition (collecting scores from judges) and management/control (synchronizing the judgments with the show, and synchronizing the output with the network requirements). The system interface dealt with formatting and overlaying the scores, averages, rankings, etc. with the network TV signal for live broadcast.
The main technological advance relates to the time-driven nature of the user interface. The collection of the scores could not be done at the convenience of the judges but had to be synchronized with the timing requirements of the live broadcast. For this reason, data acquisition is a better description than user interface.
• commercially available interface controls are passive, i.e., they wait for the response from the user
• "active" controls may show pictures or play sounds, but they are not used to control the response timing from users
• This application requires timing control over the judges. The scores have to be delivered to suit network timing. The judges, however, do not see the TV signal, they only see the show. The user interface, therefore, had to control the timing of the responses of the judges. The interrupt-driven time intervals were used to force the judges to synchronize their responses with network requirements. The synchronization had to be controlled remotely, based on signals from the producers and on triggers and events on the live broadcast. The technology of implementing this forced timing response constitutes the technological advance of the user interface component.
• The technological uncertainty relates to the difficulty of using the user interface signals to control the timing of responses of the judges. A contributing factor is that the judges are senior business leaders who volunteer for this role, may be computer illiterate, and do not receive any training. If any judge does not score on time then fewer scores are displayed on TV, a very visible error. While the judges cannot see the TV signals, and thus the scores, their friends and relatives do record the TV show, and these issues can lead to a debate on public forums such as radio, TV, and newspapers.