But there are also lots of questions and comments from producers who's products didn't score as well as they would have liked, and a general scratching of heads about what we white-coat-clad judges do inside that big, temperature-controlled room full of cheese all day.
Some of the comments* made over the last week that have prompted me to write this post include:
- "who knows what the judging criteria was"
- "some of the judges don't even know what some of the cheeses are"
- "but this is a really popular product - why didn't it win gold?"
- "judges are looking too much for faults, and not for flavour"
So, as one of the "new guard" of cheese and dairy judges in this country, I feel duty-bound to lift the lid and demystify the process. This is a long post, so feel free to just skim over the headings if you want to get the gist of it!
Of prime importance is this - as judges, we are acutely aware that all producers have put time, effort, energy, hard toil, creativity, expertise, money and care into every one of their products, and that each deserves to be judged fairly and with respect. We aim to give the same careful consideration to the last cheese judged on the day, as to the first, and the Sydney Royal's system of stewarding and judging aims to support us in doing that.
All judges are trained and accredited
All cheese judges at the Sydney Royal have undergone at least 1 training course, and most of us have undertaken 2 courses and an accreditation exam. Initially (around 5-6 years ago), 2-day "Cheese Sensory" courses were run by Neil Willman, with lots of practical and theory components, and a theory exam at the end. Only those participants who passed the exam were invited to either Steward or Judge at the next show.
More recently, Dairy Australia instigated a national training and accreditation program for cheese judges, resulting in a 2-day course run by Russell Smith. It is mandatory for all existing cheese judges to attend, and offers a pathway for potential new judges to be assessed too. This new course ends with an exam featuring both theory and practical components, so we are actually judged on our ability to assess cheeses.
Judges volunteer their time
That's right, we do it for love, not money. The Sydney Royal covers our expenses such as travel and accomodation, but we are not paid to judge.
Judging panels are diverse
At the Sydney Royal, judging is not done individually, but in panels of 3-4 judges, lead by a Panel Chair. As a more experienced judge, the role of the Panel Chair is to provide an overview at the beginning of each class, ensure the judging takes place in a timely manner, and to check that individual judges' results are within an agreed margin of difference. The Panel Chair also completes a Feedback Report at the end of each class, summarising the overall results.
 |
| Judging panel comprised of international guest judge Herve Mons (technical), Sonia Cousins (consumer) and Rob Elliott (consumer) |
Consumer judges are people such as retailers, distributors, journalists and other food professionals. Their expertise is normally in selling cheese to consumers, and they are expected to have a good knowledge of a very broad range of cheese styles, and new cheeses coming in to the market. Feedback provided by consumer judges will normally be more general (eg, balance between acidity and milky flavours could be improved), rather than technical.
Both types of judges enjoy the opportunity to learn from each other during the judging process.
There are different levels of Judges
Many judges at the Sydney Royal started by volunteering as a Steward. The role of Stewards is to assist with all of the behind-the-scenes work and set-up required for judging to take place. Each panel of judges is allocated 2 Stewards, who lay out the cheeses for each class, unwrap and cut them, make sure the judges have enough knives, spittoons, etc, and seek clarification from the Chief Steward or Chief Judge about anything to do with the cheese entries.
 |
| Stewards getting the cheeses ready 'behind the scenes'. |
After stewarding for 1 or 2 shows, potential new judges may then be invited to act as an Associate Judge. Associates are teamed with more experienced judging panels, as a mentoring opportunity. Associates participate fully in the judging, and complete score sheets for all cheeses, but their actual scores are not included in the final results. Essentially, they are 'practicing', or as a journalist reporting on the show one year wrote, these are the judges with training wheels on.
Judges normally stay at the Associate level for 2-3 years, until they build the confidence and experience required to act as a fully-fledged Judge.
Cheeses are not judged 'blind'
At the Sydney Royal, cheeses must be entered with their full retail packaging, so they are not judged 'blind'. While there are pro's and con's to this system, it is expected that all judges remain impartial at all times, and that pre-conceived ideas or expectations about individual products are left aside.
All of the products in each class are laid out at the same time, so we can go back and re-taste if we need to. This also makes it very easy to compare entries across the class for that year.
 |
| Judging Class 38 - Cheese, mould ripened, to contain blue mould |
Like most shows in Australia, the Sydney Royal follows a 20-point system for judging each cheese. Four points are allocated for presentation, six points for texture, and ten points for flavour.
Our system of assessing the cheeses is sometimes referred to as "fault-based" judging, but this doesn't mean we just look for faults. Instead, it means we only deduct points for faults.
To put it another way, all cheeses start off with the full 20 points, and we only take points off if we can clearly identify a fault (and normally not more than 1 full point per fault, unless the fault is really significant).
Classes are allocated to different judging panels
Each panel normally judges only 2-4 classes across the day, depending on the number of entries per class. So, one year a particular judge may be allocated White Mould and Blue classes, while the next year they may judge Mascarpone, Fetta, Flavoured Cheese and Buffalo Milk Cheese.
The judging score sheets are detailed
Most judges make note of the fault/s on the score sheets, either by writing a word, or recording a number that corresponds to a masterlist of possible faults.
This masterlist includes things like "Dull", "Surface Discolouration", "Poor Shape" (for Presentation faults), "Dry", "Grainy", "Rubbery" (for Texture faults), and "Bitter", "Lacks Balance", "Rancid" and "Lacks Flavour Development" (for Flavour faults).
That's right, "Lacks Flavour Development" is a fault, so even if the cheese is technically good, if it doesn't have a lot of flavour, it isn't likely to win gold.
Cheeses must be true to type
We assess the cheeses based on what they are supposed to be. That is, if the cheese is a mild cheddar, we judge it based on the characteristics that a mild cheddar should have. We don't expect it to have strong, developed flavours, or a crumbly texture, as we would expect for a vintage cheddar.
On the other hand, if a cheese entered as, say, a young Gouda-style has very strong, developed flavours, but we know the cheese has only been matured for a few months, it is not considered to be "true to type" and would not score well.
In this way, it's possible for a well-made very mild cheese to have a higher overall score than a very flavoursome cheese with faults. Essentially, just because a cheese has a lot of flavour - or a strong flavour - doesn't necessarily mean it will achieve a high score, particularly if that style of cheese is not supposed to have a strong flavour.
Gold medal doesn't mean 1st place
Unlike the Olympics, a Gold medal doesn't mean 1st place, and a Silver medal doesn't mean 2nd. Instead, medals are awarded based on the total score, with all cheeses scoring 18 points or above earning a Gold medal, and cheese with scores from 16.0-17.9 earning Silver medals.
So, there may be some classes with 10 Gold medals, and other classes with no Gold medals but several Silver medals. This is probably something that is not clear to consumers, but is the same system followed by wine shows in Australia.
Champion Cheese doesn't mean highest overall score
The judging of the Champion Cheese is done a little differently from the other classes at the Sydney Royal.
 |
| Taste-off for "Champion Sheep, Goat & Buffalo Milk Cheese" |
Bear in mind that individual classes are judged by different panels of judges, so it's important to get all judges involved in the "taste-off" for Champion cheese.
Feedback is available to entrants
Once the results from the show are published, more detailed information about individual scores is available. The Results Catalogue includes individual scores for each cheese, although scores below 14 points are simply listed as "Below 14". The summary from the Panel Chair's Feedback Report is included at the beginning of the results for each Class.
While not publicly available, it is also possible for entrants to ask for copies of the individual judges' comments on their cheeses only. This is where they will see detailed comments from each judge on the scores for each component of judging (presentation, texture and flavour) and the faults identified.
Individual entrants are encouraged to seek this feedback, as it is intended to be provided in a constructive manner. The overall purpose of the Show is to award and acknowledge excellence, and to encourage improvement.
Questions? Comments? Feedback?
I hope this has answered many of the questions I've had over the past week, but please let me know if it hasn't - I welcome feedback and comments from cheese makers, entrants, judges and other cheese friends.
Please post comments via the link below. Cheese!
*Thanks to all the individuals who made these comments (some privately, and some via Twitter), as it spurred me to answer them publicly in this forum.
