fbpx Skip to main content
After a group has generated many ideas, how do you know which ideas have the most agreement and which don’t?
 
Here are step-by-step instructions for using Feedback Frames hardware to collectively rate many ideas and recognize shared priorities in a meeting, following a brainstorm.  Feedback Frames are a secret score-voting ballot tool where participants vote by dropping a token in a slot that is hidden by a cover, with results later revealed as a visual graph of opinions.

Feedback Frames A4 version

This simple avoids traditional voting problems such as the bandwagon effect, choice overload and vote-splitting (see Dotmocracy is Broken).
 
(This technique is endorsed by the International Association of Facilitators as published within the official IAF Methods Library)
  

Process Goal

To quickly recognize collective opinions on many statements using a reliable and fun process, while avoiding groupthink and without requiring any technology. Works efficiently for large groups and dozens of ideas. Find the ‘wisdom of the crowd’ without relying on an app.

 

Materials

  • Feedback Frames kit (available at FeedbackFrames.com): frames, covers, stands, tokens, statement-signature sheets
  • Cups or bowls for holding tokens
  • Fine point markers
  • Pens for all participants
  • Tables for setting out the Feedback Frames

Instructions

[ 🎥 Watch a 2 minute video to see this method in action ]

Feedback Frames can be used at any stage of a workshop, when there is a need to prioritize participant generated statements in response to an open-ended invitation, such as:

  • Questions to be answered
  • Topics to be explored
  • Proposals to be recommended
  • Concerns to be addressed
  • Suggestions to be considered

Rating scales and other materials can be easily customized for the specific language, topic and goals of the workshop. See the library of free templates at FeedbackFrames.com/templates

For example the standard rating scale provided is:

  • Strong Agreement
  • Agreement
  • Neutral
  • Disagreement
  • Strong Disagreement
  • ? – Not Sure

This prioritization technique is intended to follow after an idea generation process (e.g. brainstorm, 1-2-4-all, brainwriting).

Results may be used immediately in the next step of a workshop agenda, and/or recorded for sharing and consideration after the event.

The quantity of materials needed depends on the expected number of ideas to be prioritized. At least two Feedback Frames units per a participant is a good rule of thumb, but you could need more with a creative group.

You can get your set of Feedback Frames, shipped internationally, by ordering from FeedbackFrames.com

 

Scheduling

The technique described below would follow AFTER time already spent on an idea generating and discussion process. Plan for a minimum of 30 minutes in your agenda for this prioritization process, including instructions (5 min.), statement writing (5 min.), voting & commenting (15 min.) and a quick reveal (5 min.). The more ideas and/or participants, the more time needed for the voting step, to a maximum of about 40 minutes of voting and comment writing.

For the most impact, schedule additional time for a more suspenseful, exciting, and unifying one-by-one reveal of the results, with opportunity for group discussion.

Include time (outside of the agenda) for a member of the facilitation team to carefully record all the results (photos and token counts) before packing up materials.

 

Set-up

FF method 2019a-800-1.jpg

As part of the room set-up prior to the meeting, set out Feedback Frames with covers on and tokens in cups, on tables in an area at the side or back of the meeting room. Each frame should have space in front of it for a landscape sheet of paper. Space out the frames for social distancing if needed.

Hand out small piles of blank statement-signature sheets and a few fine tip markers to each small group of participants.

 

0. Generate ideas

The technique described below would follow AFTER time already spent on a separate idea generating and discussion process, which could be anywhere from a few minutes to a few hours, depending on the complexity of the topic and context.

 

1. Participants write ideas on statement-signature sheets

FF method 2019a-800-2.jpg

Invite participants to write their best ideas on statement-signature sheets, only one idea per a sheet, printing with a fine tip marker.

FF method 2019a-800-4.jpg

Collect statement-signature sheets with statements on them and review each to make sure the writing is legible and the message is understandable. If needed, offer to revise or rewrite the text for clarity.

 

2. Place sheets in front of frames

FF method 2019a-800-3.jpg

Lay out the statement-signature sheets with statements on them in front of the Feedback Frames. You won’t need to bother filtering to remove or combine similar ideas (there is no vote splitting, and subtle differences in phrasing can be important) as long as you have enough Feedback Frames for all the ideas. However, if some statements are basically identical they can be clipped in a pile to avoid redundancy.

Placement of sheets can be in a random order as there is no need to cluster by theme or otherwise at this step. Record the Feedback Frames unique ID (labeled at the bottom right of each frame) in the ID space at the top right of the corresponding statement-signature sheet. IDs ensure sheets stay with their frame and should be referenced in the results documentation.

 

3. Invite token dropping to rate ideas

FF method 2019a-800-5.jpg

Ask participants who are done writing ideas to get up and move to the Feedback Frames tables. Ask them to each take a pen and a handful of tokens. Participants can start at any frame and move in any order.

FF method 2019a-800-6.jpg

For each idea, participants will silently read a statement and privately record their rating by dropping one single token in the corresponding frame along the scale provided.

They should also sign their name or initials on the right of each statement-signature sheet they vote on. Signatures track progress and will validate results by matching the number of tokens dropped per frame. You will probably need to remind participants to always sign on the right, otherwise some may forget to do so.

Optionally, they can also write comments on each sheet under the headings “Strengths & Opportunities”, or “Concerns & Weaknesses”.

FF method 2019a-800-7.jpg

Facilitators can monitor the number of tokens dropped in each frame by reviewing the number of signatures so far, and bring attention for more participation to those frames lacking in votes (e.g. sheets added later in the process).

 

4. Reveal the results

FF method 2019a-800-7b.jpg

Voting is sufficiently complete when:

  • all participants have voted on all options; or
  • in large groups (30 or more) with many ideas, a decent sample of participants (e.g. 20) have rated each idea.

It’s then time for the big reveal to “see what we think!” Reveals can generally be done all at once for efficiency, or one at a time for more suspense and rousing group reactions.

FF method 2019a-800-9.jpg

Revealing results all at once: Ask all participants to pull off the frame covers and hand them to a facilitator. Then ask if anyone can point to an idea with almost all tokens on one side or another i.e. a strong consensus. Have some of these consensus statements read out loud and potentially discussed, if time permits. You may also want to bring attention to those ideas with polarized opinions for consideration.

Revealing results one at a time: Have participants gather around one area and have someone read a statement out loud and then pull away its frame cover for everyone to see the results at the same time. Repeat for each statement. If time permits, invite some discussion in response to each result and/or comparing results.

If a break is scheduled after the reveal (recommended) then participants can use the opportunity to further review and informally discuss results, if they choose.

 

5. Record results as photos and token counts

FF method 2019a-800-10.jpg

Outside of the meeting agenda, photograph each individual frame result with its statement-signature sheet.

FF method 2019a-800-11.jpg

Also accurately write the token count for each column in pen at the bottom of each sheet.

Once you have confirmed all results have been recorded in both photos and written on the sheets, you can safely gather all the sheets, and empty the tokens out the Feedback Frames, so they are ready for their next use.

 

6. Enter results data into a spreadsheet (optional)

FF method 2019a-800-12g.jpg

If your sessions produced more than a dozen results, or if you want to compile and compare results from multiple sessions, you should take the time to type up the statements, comments and token results into a spreadsheet. A simple formula can score each idea and then automatically sort by level of agreement. Add columns for themes and topics to help with analysis. Download free spreadsheet templates at FeedbackFrames.com/templates

Tips

  • Do a practice session from start to end, including results documentation, before using in a real meeting for the first time. This can help you refine your planning, set-up, instructions, and management of the process.
  • If a participant is unable to read and drop tokens independently (e.g. someone who is visually impaired or with limited dexterity), have an assigned helper assist them individually as needed.
  • Feature photos of key results as visuals in your summary report of the session outputs.
  • A group photo with the results makes for a great cover image and post on social media!

FF method 2019a-800-8.jpg

Learn more about Feedback Frames and order your set today at FeedbackFrames.com

 

Background

Feedback Frames were invented by Jason Diceman (Toronto, Canada) in 2014 as a solution to the problems he saw in sticker dot-voting (dotmocracy). After years of prototype testing within his own professional work, he began manufacturing and distribution of Feedback Frames in 2018. They are now in use by professional facilitators around the world.

The tool is designed to be universal and reliable for participants of any age, in any language and any topic. Although Jason has a background in web app development, he specially wanted to avoid technology to ensure the tool could be as user-friendly and reliable as possible.

The approach is based on score voting and the likert scale, both of which have been proven effective in social science research since the 1950s. In 2004, Jason created an early version of the approach using pen to fill in dots on a scale of agreement in a paper form, originally called “Dotmocracy Sheets” and later “Idea Rating Sheets”.

The Feedback Frames design and method are patented: US20180189804A1