Managing Judging and Inspections at FRC events

As FIRST competitions grow in popularity, the number of teams that participate increase each year.  This leads to more competition events.  The old model of Regional events that lead directly to the World Championship is evolving to a more hierarchical District model.  Districts allow local teams to compete in more events and at a lower cost.  This will further encourage growth.

In the District model, events will be run autonomously by District officials.  The model calls for a much larger volunteer force to staff events.  There will be many rookie judges, robot inspectors and referees.  Even with training, many of the new inexperienced staff will be overwhelmed by their tasks.  An even bigger issue will be the lack of consistency in judging and inspection between one event and another.  Though franchise standards will still apply, FIRST will have less influence on the actual organization and running of the events.  There will be noticeable differences between districts, which will cause unwanted confusion and dismay among teams.

Many of the volunteer roles are challenging.  Judging can be a fairly strenuous role.  Experienced Judge Advisors who can manage judges to get the job done in time will be in short supply.  We have seen the need for improving the Robot Inspection technique.  Match Queue Manager and Pit Administration are other examples of roles that would benefit from automation and information sharing.  There is a crying need to streamline many of the tasks to make them Convenient, Consistent and Controlled.

The Game Management System was developed in response to these needs.  It provides specific assistance for each role at an event, i.e. Judge, Judge Advisor, Robot Inspector, Match Observer, Referee, Queue Manager, etc.  By electronically synchronizing events and sharing information between staff, each person will have the information they need to make decisions in a timely manner.  The system uses commonly available mobile devices that communicate securely over WiFi.  Graphics and color coding make assimilating the information intuitive and convenient.

For example,

  1. Robot Inspectors and Judges will know which teams they should interview and when they will be available by glancing at a color coded pit map.
  2. There will be rubrics to help them ask all the right questions to help them select the best choices for awards.
  3. They can take notes and pictures to help them remember special things or issues about each team.
  4. A Judge Advisor can easily configure groups of Judges and assign teams for interviews.  Assignments can be rapidly re-configured if a Judge drops out.
  5. Match Observer and Referee comments will be immediately available to all Inspectors, Judges and the Judge Advisor.
  6. Volunteers managing the Match Queue will have up-to-date information and can conveniently communicate queue status with each other.

The GMS design is based on the experience of FIRST volunteers.  Security is baked into the architecture.  It has been successfully field tested at 5 FRC events during the 2014 season. 

The major components of the system are shown in Figure 1 below.  The Judge Advisor and Lead Robot Inspector each have a station that automates the administrative aspects of their roles.  Judges and Robot Inspectors can optionally carry mobile devices (tablets or phones) to help them with their tasks.  The system streamlines functions by distributing available data and sharing status information.  GMS is so extensive that one needs to see a demo to truly get a sense of the integration of functionalities.

Goals

The system is aimed at FRC, but supports FLL & FTC as well.  It has three major objectives:

·         Convenience:

Judging at FIRST events involves interviewing many teams and assigning multiple awards.  This is a difficult task and has tight time constraints.  It can be very stressful for some of the volunteer Judges.  This system helps make the experience easier by organizing the process.  It also reduces the amount of paper involved.

·         Consistency:

The system enhances consistency of judging and inspections within an event and also across events.  Judges can have consistent guidelines for awards and can share their notes in a controlled manner.

·         Control:

The system gives better control over the overall judging process.  It helps maintain the schedule by monitoring progress by all Judges and Robot Inspectors.

Target Users and their Processes

For Judging, the system is intended for use by the Judge Advisor, Judges and Match Observers.  A separate part of it is used by Lead Robot Inspectors, Inspection Managers and Robot Inspectors. Other roles, like Referee, Queue Manager, Pit Administrator and Game Announcer are also included.

The process of Judging at a FRC event is quite complex and is described in detail here.  The process for Robot Inspections is perhaps a bit less challenging but it is still a task that must be completed swiftly and efficiently.  It is described here.

System Features

This is a brief description of the system’s components and capabilities.  More details are available by clicking on the links below. 

The Judge Advisor Station is the center for judging activity and the Inspection Manager (LRI) Station is the center for robot inspections.  They run on separate laptops and share information.  Judges and Robot Inspectors use an app on their mobile devices to access information and send back updates on interviews or inspections.  Lastly, Match Observers use the app that allows them to report match updates.

Prior to the event, the Judge Advisor application allows the Judge Advisor to easily configure all aspects of judging, including award rubrics, Judge Groups and assignments.  During the event, it monitors judging progress, sends assignments to Judge Groups, tracks match completions and helps during award discussions.

The Inspection Manager (LRI) application tracks team inspections.  The JAS and IMS periodically exchange information for and can function as backups for each other.

Pit maps are created for each event.  This will be used on mobile devices by Judges and Robot Inspectors.  A convenient layout editor is provided and an average pit map can be completed in less than 10 minutes.  All that has to be done is to use a mouse to move team pit boxes around.

Security and Privacy have been designed in.  Every user has to authenticate themselves and has a ‘role’.  Access to all data and functions are based on this role.  In fact, the JAS and IMS are really the same application and differ only in the role of the user logged into them.  All network traffic and stored data are protected with layers of security.  Data on mobile devices is encrypted and kept private.  The Judge Advisor and LRI Stations are protected from DoS or information theft.

A Walkie Talkie capability is available on the mobile devices.  It allows participants to communicate using audio messages.  There can be multiple “channels”, each consisting of a group of users.  A user can be in multiple groups, and thereby can listen on multiple channels.  Messages can be replayed as desired.  They are transmitted securely using WiFi.  The design prevents the “jamming” that occurs in other systems when a user accidentally keeps the transmit button pressed.  Channels and messages are managed centrally as described here.

A WiFi router is required at event.  This will be separate from the FMS network.  An Internet connection is optional and is not mandatory.

For additional details on judging see here.  For additional details on robot inspection see here.

Conclusion

This very brief summary barely touches on capabilities of this new system.  The design philosophy has been to integrate available information to simplify the processes at an event while making them more manageable and more uniform.  Privacy and technology risks are addressed at the core of the design.  Eliminating the need for paper improves security and information sharing.

Many other industries have developed specific tools to enhance their processes.  It is time for FIRST to move past the ‘paper chase’ and adopt today’s technology to showcase how it is done.  We expect FIRST to “Lead by Example.”  Beyond just aligning FIRST with today’s technologies, we expect FIRST to inspire kids with innovation.  Beyond the kids, FIRST would be inspiring their parents and the parents’ employers to embrace STEM components.  We expect them to embrace STEM, not just for STEM’s sake, but because it enriches their lives.  By letting them do more.  By saving time, so they can do more.  By being more effective, so they can save time.  By having the needed information, so they can be more effective.  By communicating easily with each other, so they have the needed information.  And so on…