Pi Wars 2021 – an investigation and a change in rankings to the Advanced category

Pi Wars 2021 competion logoHello everyone from Pi Wars HQ.

Sometimes, things go exactly to plan. Sometimes, they do not!

The background

Following the weekend’s broadcast of the results and the subsequent publishing of them on the website, one of the teams politely approached us and asked if it was possible to look into the results to explain why their own, calculated results didn’t seem to match up with our published results.

The investigation

Upon investigation into both the Judges’ scoring app and the totaliser app, it was found that the scores were consistent between the two different systems. However, we noticed that PiDrogen had an abnormally low score in the Tidy Up the Toys challenge, considering they were in the Advanced category. For some reason, the particular Judge scoring that challenge had marked both the wrong option for Stacked vs Side-by-Side and also marked the run as incomplete. This resulted in a loss of “base” points,  bonus points and a loss of ranking points. This did have an effect on the overall rankings, of which more later.

Clearly, this is something we needed to both rectify and investigate further. The organisers re-scored all of this particular Judge’s objective scores for Feed the Fish, Tidy Up the Toys and Up the Garden Path and found other minor discrepancies which also needed to be looked at but which did not alter any rankings.

The sign-off

We took the additional step of asking each of the Advanced Teams who completed all of the challenges to verify, or sign-off, on their updated virtual scoresheets. One of the challenges was extremely close time-wise and had been marked as a tie. We were asked, again very politely, to look at the footage of two teams’ Tidy Up the Toys challenge runs to see if we could separate them. The organisers looked at the challenge runs in question multiple times and, in fact, found that the teams could be separated by a matter of a few 100ths of a second. This was not easy to do and we had to create a side-by-side video with the two robots running the challenge to see if we could get a better view of it. This really was incredibly close and we had to go frame-by-frame. In the end, upon being pressed for a ruling, we have found that Tauradigm has just won the challenge.

The results of all this

  1. It was found that Tauradigm just had the edge over Team Dangle in Tidy Up the Toys. As I said, it was in the order of 100ths of a second, but there was definitely a separation between the two. This means that Team Dangle scores 30 points less than Tauradigm in that challenge. This does not, however, change anything in terms of overall placing.
  2. Where PiDrogen was scored significantly fewer points than they should have been, we have now re-calculated both the challenge and the overall placings. It was found that PiDrogen is, in fact, the winner of the Advanced category of Pi Wars 2021, with Team Dangle in a very close second place (just 100 points behind) with Tauradigm in third place overall.

The apology

We would like to apologise to all three teams in the Advanced category who entered all the challenges.

Neil, Mark and Phil, your teams and robots performed admirably during the competition, and all these changes are right on the margins, just as your Advanced category robots always operate to their limits. We have learned some lessons and we have some action points for future events (see later) but this will be very disappointing to some of you, and we regret that mistakes have been made. We are glad we have been able to rectify them and we are sure that we can do things better in the future.

The lessons learned

Clearly, our procedures must evolve, as they have evolved every single Pi Wars since 2014 when the first competition was held. We are always learning how to do things better.

  • For the Disaster Zone competition that was called off because of Covid, we had implemented a double-sign-off procedure so that teams could check the score sheets and agree that they were accurate before they left the challenge area. For our virtual event this year, we did not implement something similar – we just didn’t think it was feasible. If we are to run another virtual event, we will build in a “sign-off” process.
  • We will do our best to spot any ties/draws and pass them to our new Head Judge (whoever that may turn out to be) for the “final word” on the matter.

End of message

We hope that this doesn’t sully the feelings of the teams towards the competition. Judges are human, and make mistakes, as do organisers. However, we do think that we have learned lessons from these mistakes and we have rectified them.

If anyone wants to suggest other things to do differently, we are always open to these suggestions and we will be asking teams for feedback in the coming days.

Best wishes from The Pi Wars team to all of you.

 

Mike, Tim & Dave