How StackAwards is judged.
The method in full. Every number on this page is read from the code that scores the awards, so the two cannot differ.
Categories. Industry by size.
Every industry has 3 categories, one for each size of organisation. There are 22 industries.
- Small business
- Fewer than 50 people.
- Medium business
- 50 to 249 people.
- Corporate
- 250 people or more, or part of a group that size.
- Industries
- Agencies and consultancies; Architecture and construction; Arts and culture; Automotive; Education; Energy and utilities; Fashion and beauty; Finance and insurance; Food, drink and hospitality; Government and public sector; Health and care; Home and local services; Legal and professional services; Manufacturing and industry; Media and entertainment; Non-profit and charity; Property and real estate; Retail and ecommerce; Sport and fitness; Software and technology; Transport and logistics; Travel and tourism.
The measured score. 0 to 100.
Every entry is tested with PageSpeed Insights at the address entered, on mobile and on desktop.
- What is measured
- Performance (25%), Accessibility (25%), Best practices (25%) and SEO (25%), each scored 0 to 100 by PageSpeed Insights.
- Runs
- 3 runs on mobile and 3 on desktop. Each category counts at its median across the runs, so one slow or lucky run moves nothing. A run missing any category does not count.
- Mobile and desktop
- Mobile counts for 60% and desktop for 40% of the measured score.
- A site that cannot be measured
- If PageSpeed Insights cannot load a website after 6 attempts, the entry is set aside and cannot be shortlisted.
The shortlist. Top 5 measured.
When measuring closes, the 5 highest measured entries in each category become its shortlist and go to the jury.
- Ties
- Decided by the higher measured score, then the higher mobile score, then the higher accessibility on mobile, then the earlier entry.
- Small categories
- A category with fewer than 3 fully measured entries is not awarded that year.
The jury’s rubric. 5 criteria.
Each juror scores each shortlisted website from 1 to 10 on every criterion, with an optional note of up to 280 characters.
- Clarity of purpose (20%)
- Within a few seconds, can a visitor tell who this is for, what is offered and what to do next?
- Design craft (20%)
- Are type, layout, colour, imagery and motion considered, consistent and in service of the content?
- Content (20%)
- Is the writing clear, specific and useful, and does it answer what a visitor came to find out?
- Usability (20%)
- Can a first-time visitor find their way, complete the main task and recover from a mistake, on a phone and a computer?
- Accessibility in use (20%)
- Beyond the automated checks: keyboard use, focus, contrast in real states, captions, plain language and forms a screen reader can follow.
- A juror’s score
- The weighted criteria, put on 0 to 100. It counts only once every criterion is scored.
- The jury score
- The mean of the jurors’ scores, once at least 3 jurors have scored the website. With 5 or more, the single highest and lowest are set aside first. A shortlisted website with fewer jurors is listed without a place.
The final score. 40% measured, 60% jury.
The highest final score in each category wins. The measured, jury and final scores are all published.
- Ties
- Decided by the higher final score, then the higher jury score, then the higher measured score, then the earlier entry.
- Once announced
- Results are computed and frozen together with the edition’s method. They are never changed afterwards.
Who judges. And recusal.
The jury is the Leaders and Luminaries of the Stacklumen Room, and CEOs and design leads invited by name. A juror from the Room judges only while they hold that tier.
- Recusal 1
- A juror never scores a website they entered or nominated, or one their own organisation runs.
- Recusal 2
- A juror never scores a website entered by somebody in their agency, or in the agency they are a client of.
- Recusal 3
- A juror never scores the website of a client of their agency.
- Recusal 4
- A juror never scores a website credited to their agency, whether the credit is verified, confirmed or only named by the entrant.
- Recusal 5
- A juror declares any other conflict, and Stacklumen can record one; either takes the entry off their list.
- Recusal 6
- These checks run in the code every time a score is saved, not only when the list is drawn, and the database refuses a score from a juror recorded as conflicted.
Agency credit. Who built it.
An entry can name the agency that built the website. It is shown as “Built by” beside the entry, with how sure the credit is in plain words.
- Who is credited
- An entrant may name the agency that built the website, from the agencies listed in the Stacklumen Directory. The agency is shown by its Directory name, linked to its card.
- Verified by Stacklumen
- The website is live on Stacklumen in that agency’s own workspace, at the entry’s own address. This is checked when the entry is made and again when the results are announced.
- Confirmed by the site owner
- The website’s organisation says so: the entrant, when entering their own organisation’s website, or afterwards somebody signed in with an email address at the website’s domain, or the company’s primary contact on Stacklumen.
- Credited by the entrant
- Otherwise. The website’s organisation can confirm or decline it until the results are announced; a declined credit is removed from the entry.
- The agency’s own team
- Can never confirm or decline its own credit. Only the website’s side can.
- Recusal
- A juror never scores a website credited to their own agency, however the credit stands.
Agency of the Year. In every agency size.
Each announced edition names an Agency of the Year for each size of agency, from its frozen results. Only “Verified by Stacklumen” and “Confirmed by the site owner” credit counts.
- Points
- 1 point for each shortlisted website the agency is credited with, or 3 points when that website wins its category, instead.
- Agency sizes
- An agency’s size is the larger of the people on its Stacklumen team and the lower end of the company size on its public profile, read at the announcement and kept with the results. Small agencies: fewer than 50 people; Medium agencies: 50 to 249 people; Large agencies: 250 people or more, or part of a group that size.
- Named
- The agency with the most points in its size, once it has 1 point or more. A size with no such agency is not awarded that year.
- Ties
- Decided by the more points, then more category wins, then more shortlisted entries, then the higher best final score among its credited entries, then the earlier first credited entry.
- Badges
- The announcement gives each agency a shortlist or winner badge for its credited websites, by industry, and each Agency of the Year its badge, on its Directory card. Only “Verified by Stacklumen” and “Confirmed by the site owner” credit earns one.
- Once announced
- The credit, each agency’s name and size and the points are frozen with the results, by the method the edition was judged with.
Who may enter. Any public website.
A website does not need to be built on Stacklumen. Entry is free.
- Who enters
- An organisation can enter its own website, and anyone can nominate somebody else’s. The person entering needs a Stacklumen account and is the entry’s contact.
- Once a year
- Each website can be entered once a year. A website is its address without “www.”, in one industry and one size.
- Publishing
- The organisation’s name and website are published if the entry is shortlisted, with its scores.
- Open for entries
- Entries are open. Every entry is measured as it arrives.
- Measuring
- Entries are closed. Every entry is being measured.
- Judging
- The shortlists are set and the jury is scoring them.
- Announced
- The results are announced.