Confidential

2019 Competition Preliminary Round

Youth Cannabis Awareness Program
(YMCA of Greater Toronto)

February 9, 2019

The Request for Proposals in this document was developed for the Student
Evaluation Case Competition for educational purposes. It does not entail
any commitment on the part of the Canadian Evaluation Society (CES), the
Canadian Evaluation Society Educational Fund (CESEF), the YMCA of
Greater Toronto or any related sponsor or service delivery partner.

We thank the YMCA of Greater Toronto for graciously agreeing to let us

use the proposed Youth Cannabis Awareness Program for the preliminary
round of the 2019 competition. We also thank Mina Hazar, Manager of
National Provincial Programs, for her input in preparing this case.

The Case Competition is proudly sponsored by:

Confidential

Introduction
Welcome to the Preliminary Round of the 2019 CES/CESEF Student Evaluation Case
Competition! Here is the scenario for this round:

Your consulting firm has been invited to respond to the attached Request for
Proposals (RFP) to prepare an evaluation plan for a Youth Cannabis Awareness
Program, a program proposed by the YMCA of Greater Toronto.

The program aims to educate youth and adults involved in young people’s lives on
cannabis awareness, including potential risks, healthy choices, and informed and
responsible decision making. The YMCA of Greater Toronto is interested in a
developing an evaluation framework for the program to assess program reach and
effectiveness.

Your proposal should demonstrate your understanding of the assignment and

include a program logic model, a proposed methodology, an evaluation matrix, a
mitigation strategy to address anticipated evaluation challenges and a discussion
of an evaluation competency that is important for a successful evaluation of this
program.1

Section 2.2 of the RFP identifies the proposal requirements in more detail. The
assessment criteria for the proposals are identified in section 2.3 of the RFP.
Three proposals will be short-listed for the teams to participate in Round 2.

We look forward to your submission at the end of the day.

Organizing Committee
The 2019 CES Student Case Competition Working Group and Case Selection Sub-Committee:

Kathryn Radford, Brian McGowan, Christine Sheppard, Micheal Heimlick, Tin Vo, Michelle
Naimi, Sydney Cherniawsky, Amanda Hayne-Farrell, Marla Steinberg and Sharon Margison.
Bea Courtney continues to contribute in an advisory capacity.

1 Competencies for Canadian Evaluators is posted on the CES website under Designations.
Compétences pour les évaluateurs canadiens sont affichées sur le site Web de SCÉ, sous « Titres ».

Evaluation Case Competition: Preliminary Round, February 9, 2019 2

Confidential

Rules

1. The team’s designated contact person will receive an e-mail from Case
Competition organizers indicating a website and team identification number for
retrieval of the case.
2. Teams can begin work upon receipt of the document from the contact person.
3. Submissions may be in either official language.
4. The submission must be uploaded to the same website no later than 5 hours
and 30 minutes after initial download.
5. Coaches must not communicate with their teams once the case document has
been downloaded and distributed to the team.
6. Submissions must be non-identifiable. Teams should refer to themselves only by
an imaginative, non-revealing code name, such as Noble Consultants. Do not
identify the university, city or province/territory where your team is located. This is
a key point to be kept in mind when writing your submission.
7. Do not use the word ‘evaluation’ or a variation of it in your team name, as it is
confusing for the judges if teams have chosen similar names.
8. Your submission should be saved as [your team name].pdf, e.g. Noble
Consultants.pdf. The submission must be submitted as one PDF file (not a
zipped file of multiple documents).
9. Judges may take up to six weeks to select the three best submissions. All teams
will receive feedback.

Questions or Problems
To communicate with organizers on the day of the competition, please email
casecomp@evaluationcanada.ca, or call one of the following individuals:

Name Phone Number Availability (EST)

Micheal Heimlick (enquiries in
1-306-361-5904 8:00 am to 6:00 pm
English)
Kathryn Radford (demandes de
1-613-558-6457 8:00 am to 6:00 pm
renseignements en français)

Have fun and good luck!

Evaluation Case Competition: Preliminary Round, February 9, 2019 3

Confidential

Request for Proposals

Youth Cannabis Awareness Program
February 9, 2019

1.0 YMCA of Greater Toronto and Program Profile

1.1 About the YMCA of Greater Toronto

The YMCA of Greater Toronto is one of the largest and most active charitable
organizations in the Greater Toronto Area (GTA), offering over 60 different programs
from more than 275 sites spread throughout the GTA. The organization has been in
operation for over 160 years and 500,000 people annually benefit directly from YMCA
programs.

The YMCA serves people of all ages, backgrounds and abilities, through all stages of
life. The organization is dedicated to helping people attain good health through a
healthy lifestyle and encouraging them to get involved in making their community a
better place for everyone. The YMCA provides value-based programs and services that
teach the importance of caring, honesty, health, respect, responsibility, and inclusivity. It
is a place where friendships are formed, and family ties are strengthened. The YMCA is
where people meet and discover common needs or interests.

The YMCA is active in the community in many ways, offering the following programs
and services:
 Pre-school and school age child care centres;
 Summer camp sites;
 Early years family centres;
 Employment centres;
 Youth and community outreach sites;
 Health and fitness centres;
 Newcomer centres;
 National/Provincial program coordination;
 Outdoor education centre;
 Resident camp; and
 High school program.

The 2010-2020 strategic plan – Strong Start, Great Future – calls upon the YMCA to
invest in young people across their life stages to deliver on the YMCA’s vision of
ensuring communities will be home to healthy children, teens, and young adults.
Annual revenues are $178 million, and the YMCA employs 1,300 full-time staff (3,300
total staff, including part-time) along with 3,400 volunteers.

Evaluation Case Competition: Preliminary Round, February 9, 2019 4

Confidential

1.2 Youth Cannabis Awareness Program

The Youth Cannabis Awareness Program (YCAP) is a new initiative being proposed by
the YMCA of Greater Toronto to educate youth and adults involved in young people’s
lives on cannabis awareness, including potential risks, healthy choices, and informed
and responsible decision making.

Findings from the 2017 Ontario Student Drug Use and Health Survey (OSDUHS)
showed that 20% of Ontario students in grades 7-12 (172,000 students) have reported
using cannabis in their lifetime, and past year cannabis-use increased with grade level.
This survey also found that 1 in 10 youth drivers reported driving after cannabis use,
and that this rate is significantly higher than the proportion of student drivers who
reported driving after alcohol consumption. With the legalization of cannabis in Canada,
11% of 2017 OSDUHS respondents reported they would continue to use cannabis as
often as they currently did, 8% indicated they would try cannabis, and 4% indicated they
would use cannabis more often. Given the patterns of cannabis use among youth in
Ontario, the YMCA felt there was a need for a developmentally appropriate cannabis
awareness program with a harm reduction approach.

YCAP, modelled after the YMCA’s Youth Gambling Awareness Program, is under
development and has not yet been implemented. The YMCA is seeking support to
develop an evaluation plan for YCAP to ensure the agency is equipped to assess the
reach and effectiveness of the program when it is implemented in local communities.

YCAP activities and projects will be facilitated by 17 Youth Outreach Workers already
located in local YMCAs in the following communities across Ontario:
 Durham Region;
 Guelph/Kitchener/Waterloo;
 Hamilton/Burlington/Brantford;
 Kenora;
 Kingston;
 London;
 Niagara Region;
 Ottawa;
 Owen Sound;
 Peterborough;
 Sarnia;
 Sault Ste. Marie;
 Simcoe-Muskoka;
 Timmins;
 Toronto;
 Thunder Bay;
 Windsor and Essex.
Evaluation Case Competition: Preliminary Round, February 9, 2019 5
Confidential

Each Youth Outreach Worker is supported by a local volunteer Advisory Committee.

The committees provide advice on potential networking contacts, insights into the local
community, and information on cannabis-use trends among youth and young adults.
The advisory committee will also help to raise awareness of YCAP in their communities
once the program is implemented. Committee members are typically youth service
providers, addiction and mental health service providers, educators, and youth.

Awareness Raising Activities: YCAP will offer age-appropriate awareness raising

workshops in English and French, developed specifically for three youth age groups –
8-13, 14-18, 19-24 – as well as for parents and professionals. The workshops will use a
harm reduction framework2 and will incorporate a variety of interactive activities such as
role plays and group discussions. Workshops will be appropriate for classroom settings,
community centres, youth groups, and summer camps, and can be booked individually,
or as a series.

The workshops cover the following topic areas:

 Cannabis-use perceptions and knowledge;
 Cannabis-impaired driving;
 Cannabis and the law;
 Physical and mental health risks;
 Product types and quality;
 How to seek support;
 Social competencies related to problem solving, emotional regulation, goal
setting, and risk resiliency / refusal skills training.

The overarching aim of the awareness raising activities is to reduce the harms
associated with cannabis use by helping youth improve their knowledge of cannabis,
including normative beliefs, health risks, and product safety and quality. With an
emphasis on risk resiliency, participants are given opportunity to practice cannabis-
related refusal skills, goal setting, emotional regulation, and help seeking behaviours to
help reduce use and intentions to use cannabis.

Community Involvement Projects: Youth Outreach Workers will build partnerships

and collaboration with community organizations to plan and implement events to
promote community awareness and involvement related to youth cannabis use. Youth
Outreach Workers will work within their local communities to secure partners and
develop initiatives that reflect community needs, and promote healthy active living,
informed decision making, and education related to youth cannabis use.

2An approach that aims to provide accurate and unbiased information to reduce the harms associated
with substance use without necessarily reducing substance use itself.

Evaluation Case Competition: Preliminary Round, February 9, 2019 6

Confidential

Youth Engagement Projects: Youth-driven initiatives are key in promoting youth

expression and community action related to youth cannabis use. Youth engagement
projects will encourage youth to be involved in the planning and decision-making
processes of cannabis-use issues that affect them and those around them. Youth
engagement projects will utilize the YMCA’s Youth Voices Process for authentic and
meaningful youth engagement shown in Figure 1, below.

Figure 1. Youth Voices Process

1.3 Program Logic Model

The program does not have a logic model.

2.0 Scope of Work

2.1 Evaluation Services Required

The YMCA of Greater Toronto is seeking a consulting team to develop an evaluation
framework for the Youth Cannabis Awareness Program to assess:

1. Reach and uptake of the program across the province;

2. Effects of the awareness raising activities on youth and young adults;
3. Effects of the youth engagement and community involvement projects on promoting
knowledge and awareness of cannabis use among youth and young adults.

Given the age range of program participants (three youth groups and adults), the YMCA
is particularly interested developing data collection methods and tools that are
appropriate for each age group.

Evaluation Case Competition: Preliminary Round, February 9, 2019 7

Confidential

2.2 Proposal Requirements

The Steering Committee expects proposals to include the following components:

1. An overview of your understanding of the Youth Cannabis Awareness Program

and the evaluation requirements.
2. A proposed logic model and logic model narrative for the program, including at
least one underlying assumption, one risk and one external factor that may
influence whether the outcomes can be achieved. These can be part of the logic
model figure or explained as part of the logic model narrative.
3. A description of the approach and data collection methods you recommend for
YCAP, with a specific focus on how you would measure outcomes for children
aged 8-13, youth aged 14-18, young adults aged 18-24, and adult participants
(parents and professionals).
4. An evaluation matrix for the Youth Cannabis Awareness Program, with a list of
key evaluation questions, one to three indicators per question, and clearly
indicating which indicator is linked to which question.
5. Anticipated methodological challenges, and how you propose to deal with them.
6. Select one of the Credentialed Evaluator competencies3 that you feel is most
important for a successful evaluation of this program and discuss why this
competency is most important and how it is reflected in the evaluation plan.

The Steering Committee estimates that it will take 15 work days of consultant time for
this evaluation. The Committee does not require that proposals include a budget.

Proposals must meet the following technical specifications:

 Maximum of 12 pages, excluding the cover letter, cover page and table of
contents. Text over 12 pages, including any appendices, will not be read or
scored.
 Standard paper size (8.5 x 11).
 12-point minimum font size for text.
 Have 1 inch margins (top, bottom, left and right sides).
 For tables and figures, minimum font size is 10 point and have margins less than 1
inch.

3 Competencies for Canadian Evaluators is posted on the CES website under Designations.
Compétences pour les évaluateurs canadiens sont affichées sur le site Web de SCÉ, sous « Titres ».

Evaluation Case Competition: Preliminary Round, February 9, 2019 8

Confidential

2.3 Judging Criteria

The criteria by which submissions will be assessed are as follows:

Criteria Weight
1. Understanding of the requirement: Demonstration of an understanding 5%
of the Youth Cannabis Awareness Program and the YMCA of Greater
Toronto’s evaluation needs (beyond a reiteration of the text provided in
the RFP).
2. Logic model and logic model narrative: Clarity, completeness and 25%
appropriateness of the proposed logic model, including at least one
underlying assumption, one risk and one external factor that may
influence whether the outcomes can be achieved. These can be part of
the logic model figure or explained as part of the logic model narrative.
3. Evaluation approach and methods: Appropriateness of the approach 20%
and data collection methods for undertaking the evaluation.
4. Evaluation matrix: Appropriateness and clarity of evaluation questions, 20%
and appropriateness and feasibility of indicators.
5. Challenges and mitigation strategy: Clarity and appropriateness of the 10%
assessment of methodological challenges and the mitigation strategies.
6. Competencies for Canadian evaluation practice: Relevance of the 5%
competency identified.
7. Innovation: Innovative ideas 5%
8. Proposal: Quality of the proposal (writing and format) 10%

Total 100%

Evaluation Case Competition: Preliminary Round, February 9, 2019 9