Scenario 1: Northern Sydney Youth Health Promotion (NSYHP)
A. Table 1: Project Description
139065054610Inputs
138239554610Process/ Activities
139255554610Outputs
135509054610Outcomes
Impact/ Goal
What goes into the program?
What does the program do?
What happens as a result of the program?
What are the short/ medium term benefits for participants?
What are the long term or broader benefits of the program?
Certain resources are needed to operate the program.
If the program has access to inputs, then they can be used to accomplish planned activities.
If planned activities are accomplished, then the program will hopefully deliver the amount/ type of product and/ or service intended.
If planned activities are accomplished to the extent intended, then participants will benefit in certain ways.
If these benefits to participants are achieved, then certain changes in organizations, communities, or systems might be expected to occur.
External funding (department of education & city council)
Internal funding (Northern Sydney Health District)
Staff (NSYHP)
Casual staff (12 youth consultants aged 15-24)
Stakeholder partnerships
Infrastructure & training resources (Northern Sydney Health District)
Infrastructure (schools, department of education)
Technological skills (app & website development)
Note:
This could also be strucutres according to areas (e.g., financial,…). Different terminology is acceptable as long as it describes the same input and is sufficiently detailed.
Selection of youth consultants
Training, mentoring and supervision of youth consultants
Consultations with local service providers/Deployment of youth consultants
Youth health advocacy (participation in conferences and committees; inter-agency and policy maker advocacy)
Health promotion of HEADSS district wide psychological risk assessment instrument (home, education, activities, drug use, sexuality, mental health)
Designing and deploying YouthSource website and smartphone application
Health promotion workshops at local high schools focussing on alcohol, tobacco and other drug use, social and emotional health, obesity
Note:
Further information (e.g., specific parts of the training modules) may be provided for a thicker description.
(Increased number of) Youth consultants trained, mentored and supervised
Youth consultants deployed/local service providers received appropriate consultations regarding their services
YouthSource website accessed by young people
Workshops & health promotion education delivered at local high schools and attended by young people
HEADSS used
Other (non-electronic) health promotion resources provided – e.g., anti-smoking signs, brochures, leaflets
Note:
Some information may be grouped as long as sufficient information is provided.
Raised awareness of youth-friendly best practice among local service providers; improved delivery of youth-friendly services
Raised awareness of health services available among young people; increased usage of health services by young people
Improved health literacy among young people, particularly regarding alcohol, tobacco and other drug use, social and emotional health as well as obesity; improved health communication skills among young people
Improved mental and social health among young people
Creation of supportive environments on school campuses
Raised awareness of youth-related health issues and health promotion in teaching environments
Continued usage and provision of high-quality health promotion material
Note:
We will concentrate on the benefits for the participants here. However, you may provide further information regarding organisational outcomes or beneficial outcomes for youth consultants. However, the documented outcomes above are also highly correlated with these organisational outputs. The last point may also be categorised as an impact due to its long-term nature.
Improvement in physical, social and emotional health among young people in the district (e.g., lower prevalence of alcohol, tobacco and other drug use, obesity) – reduced morbidity and mortality in young people (reduced burden of disease)
Availability of youth-appropriate and -friendly health services and health environments
Young people take “ownership” of their health (long term improvement of transition into healthy adulthood)
Note:
You may want to separate these points further to provide a thicker description.
General note (Part A): This can be either completed in dot points or with longer explanation – relevant is the depth and complexity of information provided rather than the way it was delivered. Some students provided references in this part – this is appreciated and demonstrates your research skill. Not providing references did not negatively impact on your mark for this assessment.
B. Table 2: Evaluation questions (based on the project description in Table 1)
General note (Part B): Information on the left column can be either copied and pasted as written in the table above or simplified. This did not impact on your mark as you have already stated everything above (there is no need for me to mark this twice). The marking for this part of the assessment concentrated on the evaluation questions, their complexity and the completeness of the table. This also means that missing points from Part A have to be taken into account for Part B. Evaluation questions should be framed in a measureable and answerable way – however, they should not be too detailed since we will provide more information in Part C. Evaluation questions should also be more than simply ‘number of’ – while this is helpful, we would like to gather more comprehensive data that also demonstrates the quality of our project rather than just the quantity of services provided. Some students opted for two or three more general question for each section (rather than each individual point) with sufficient sub-questions to cover more than one input, activity,…
Inputs (taken from table 1):
Evaluation question(s):
External funding (department of education & city council)
How much funding was allocated for individual components?
No specific funding is mentioned in the resources for the NSYHP. The cost of healthy eating was estimated as $13 per child.
Was the funding sufficient for the purpose of the project?
The funding is inadequate because the state has only announced the general funds for the health sector. This indicates the inadequacy of the grants for meeting the needs of growing population of youth.
OR (more specific) What was the cost of conducting each activity of the project?
The cost of conducting each activity was in dollars.
The cost of teaching staff was $9 million over the last five years.
Internal funding (Northern Sydney Health District)
Questions as above
Staff (NSYHP)
How was staff time allocated to activity types?
2 weeks was allocated to physical activity, 30 days on teaching healthy food choices and 3 months on designing strategy for controlling smoking/ alcohol among youth.
Casual staff (12 youth consultants aged 15-24)
Did the capacity of youth consultants align with the project’s needs?
The capacity of the youth consultants aligned with the project needs as they addressed every area of health concern. Young consultants possess adequate knowledge of the physical activities and healthy food choices. They are also well aware of the strategies used for encouraging youth to quit substance abuse.
Stakeholder partnerships
Infrastructure & training resources (Northern Sydney Health District)
Were infrastructure and training resources allocated to the project appropriate for the purpose and need of the program?
Infrastructure and training resources are appropriate as schools are used for teaching young staff about the strategies for promoting health. The resources include materials such as projectors, classrooms and published manual.
Infrastructure (schools, department of education)
Were rooms allocated for activities at high schools appropriate for the purpose of the workshops?
The rooms allocated for the activities are appropriate as they are spacious and have capacity of addressing students population. Proper seating and lightning is offered for sharing concerns.
Technological skills (app & website development)
Were technological skills necessary for the development of mobile phone application and website available?
Young professions need technological skills for making best use of the mobile phone applications and other websites. This will save time and allow them to build interaction.
Processes/Activities (taken from table 1):
Evaluation question(s):
Selection of youth consultants
How many applications were received for the positions and how many were hired?
More than 500 applications were received.
Were assessment criteria appropriate for the selection process?
Interviews and discussion sessions were use for recruiting young personnel.
Training, mentoring and supervision of youth consultants
Was training for all youth consultants completed prior to program delivery?
Training session was completed that shared all relevant information with the consultants.
Were high-quality mentorships and supervision concepts implemented?
High-quality mentorship and supervision were used that included interactive sessions with the consultants.
Consultations with local service providers/Deployment of youth consultants
How many local service providers were identified and invited to participate in the program?
More than 30 local service providers participated in the program.
Was a holistic consultation provided?
Holistic consultation was offered that included goal-specific counselling.
Youth health advocacy (participation in conferences and committees; inter-agency and policy maker advocacy)
Was contact made with policy makers and other relevant agencies?
Yes.
In how many conferences and committees did project staff participate in their professional capacity?
Committees participated at minimum 15 conferences.
Could also be more general: Has NSYHP taken initiative in capacity building through collaboration and advocating?
Initiative was taken for capacity building by collaborating with other agencies such as North Sydney Council and health agencies.
Health promotion of HEADSS district wide psychological risk assessment instrument (home, education, activities, drug use, sexuality, mental health)
Was the HEADSS psychological risk assessment promoted (quality and quantity)?
Psychological risk assessment was promoted by qualitative methods as low, moderate, high risks.
The quantity if risks is evaluated by ranking from 1-5 starting from no risk to extremely high.
Designing and deploying YouthSource website and smartphone application
Were a website and a smartphone application aligning with the project’s objectives ready to be deployed?
Yes.
Health promotion workshops at local high schools focussing on alcohol, tobacco and other drug use, social and emotional health, obesity
Were workshops perceived to be relevant by attendees?
The workshops used visuals and graphics for making it relevant for the attendees.
Did workshops cover all objectives of the program?
All objectives were covered including the safe food choice program, removing use of drugs/ tobacco and physical activity.
Did services meet the demand?
The demand was met as the service focused on controlling obesity.
Outputs (taken from table 1):
Evaluation question(s):
(Increased number of) Youth consultants trained, mentored and supervised
How many youth consultants were trained and retained (monitored and supervised) over the period of the project?
30 youth consultants were trained.
Youth consultants deployed/local service providers received appropriate consultations regarding their services
How many consultations were provided?
500 consultations were provided initially.
YouthSource website and smartphone app accessed by young people
How many young people accessed website/downloaded the app? Which areas were most important?
More than 500.
Workshops & health promotion education delivered at local high schools and attended by young people
How many workshops were conducted and how many people attended workshops?
5 workshops were conducted.
HEADSS used
How often was HEADSS used if appropriate compared to baseline levels?
By the end of month.
Other (non-electronic) health promotion resources provided – e.g., anti-smoking signs, brochures, leaflets
How much non-electronic health promotion resources were provided and used?
Signs and posters were also used.
Outcomes (taken from table 1):
Evaluation question(s):
Raised awareness of youth-friendly best practice among local service providers; improved delivery of youth-friendly services
Are local service providers aware of youth-friendly best practice compared to baseline levels?
Not all services providers are aware of youth-friendly practices.
How many local service providers adapted their service delivery to youth-friendly standards after consultations took place?
More than 100.
Raised awareness of health services available among young people; increased usage of health services by young people
Are young people more aware of appropriate health services compared to baseline levels? Are young people more likely to use health services compared to baseline levels?
After workshops and programs more young people gained awareness about health services. The use of services also increased.
Improved health literacy among young people, particularly regarding alcohol, tobacco and other drug use, social and emotional health as well as obesity; improved health communication skills among young people
Are levels of health literacy in the targeted group higher than at baseline levels (particularly regarding alcohol, tobacco and other drug use, social and emotional health as well as obesity)?
Levels of health literacy improved in targeted group and they had better awareness.
Did young people’s health communication skills improve compared to baseline levels?
They were able to explain health related issues reflecting improved communication skills.
Improved mental and social health among young people
Could a short-term or immediate improvement in mental and social health be observed (reduction in prevalence and incidence of mental and social ill-health)?
Improved mental and social well being is apparent as youth reported having good sleep, felt less tired or depressed.
Creation of supportive environments on school campuses
Were school campuses perceived to be supportive environments for health behaviours by young people?
School campuses promoted healthy behaviors by arranging seminars.
Raised awareness of youth-related health issues and health promotion in teaching environments
Have high schools/high school teachers shown higher levels of awareness of youth-related health issues and promotions compared to baseline levels?
Yes.
Continued usage and provision of high-quality health promotion material
Were health promotion materials used and provided continuously in the immediate aftermath of the project?
30% of the students exhibited positive change.
Impact/ Goal (taken from table 1):
Evaluation question(s):
Improvement in physical, social and emotional health among young people in the district (e.g., lower prevalence of alcohol, tobacco and other drug use, obesity) – reduced morbidity and mortality in young people (reduced burden of disease)
How did the incidence and prevalence of alcohol, tobacco and other drug use, and obesity change during the period of the project?
Reduction in drug usage by 40% and obesity among 20% students.
Availability of youth-appropriate and -friendly health services and health environments
Were enough youth-appropriate and -friendly health services available to meet demand?
The services met needs of 50% of the young people.
Young people take “ownership” of their health (long term improvement of transition into healthy adulthood)
Do young people use their knowledge consistently to make healthy decisions and choices?
Change in food choices is noted in case of 40% young people.
C. Table 3: Performance indicators, means of verification, assumptions and challenges:
Evaluation question
Performance Indicators
Verification
Assumptions/ Challenges
How much funding was allocated for individual components?
What was the cost of conducting each activity of the project?
% of funds allocated to each project component; cost in AUD per project activity (workshops, consultations)
Project management records; staff activity records
Assumes accurate itemisation of records and resource allocation
Was the funding sufficient for the purpose of the project?
% of funds used after 3 months, 6 months and end of project
Project management records
Assumes accurate and timely recording of spending
How was staff time allocated to activity types?
% of time allocated to: administrative tasks, logistics, interagency collaboration, training and professional development, conducting workshops and consultations (NOTE: This also includes youth consultants).
Internal staff activity records
Accurate recordings of hours divided by tasks in a timely manner to avoid recall bias.
Did the capacity of youth consultants align with the project’s needs?
Number of youth consultants trained and available, average hours utilised; % of consultation and workshop requests resulting in action
Internal staff activity records
Accurate recordings of hours divided by tasks in a timely manner to avoid recall bias.
Were infrastructure and training resources allocated to the project appropriate for the purpose and need of the program?
Quality of infrastructure and training (checklists); satisfaction with infrastructure and training resources
Observation; questionnaire (survey of staff)
Assumes the availability of an appropriate, objective checklist and availability of experienced observers; validity and reliability of survey; high level of participation
Were rooms allocated for activities at high schools appropriate for the purpose of the workshops?
Quality of rooms in high school (checklist)
Observation
Assumes the availability of an appropriate, objective checklist and availability of experienced observers
Were technological skills necessary for the development of mobile phone application and website available?
Binary question (Yes/No)
Internal staff activity records
Evaluation questions like this are hard to assess since staff or evaluators may not be able to judge if a technology service provider has appropriate skills or not
How many applications were received for the positions and how many were hired?
Number of applications, % accepted
Management and project records
Records are accurate and collected in a timely manner.
Were assessment criteria appropriate for the selection process?
Quality of assessment criteria used
Checklist/observation
Assumes the availability of an appropriate, objective checklist and availability of experienced observers
Was training for all youth consultants completed prior to program delivery?
% of staff and volunteers completing training prior to program delivery
Project management records/staff training records
Records are accurate and completed in a timely manner. Assumes quality of training is sufficient (this will be addressed below)
Were high-quality mentorships and supervision concepts implemented?
Average number of supervisions and mentoring sessions; satisfaction with supervision and monitoring by mentors and mentees
Internal staff activity records; survey (questionnaire)
Accurate recordings in a timely manner to avoid recall bias, validity and reliability of survey; high level of participation
How many local service providers were identified and invited to participate in the program?
Number of local service providers identified, % contacted and invited
Internal staff activity records; project management records
Accurate recording of staff activities and project management records in a timely manner
Was a holistic consultation provided?
Quality of consultations (checklist); satisfaction with consultation among service providers
Observation with checklist
Assumes the availability of an appropriate, objective checklist and availability of experienced observers
Was contact made with policy makers and other relevant agencies?
Was contact made with policy makers and relevant agencies? Yes/No (binary) Number of policy makers and relevant agencies contacted
Internal staff activity records; project management records
Accurate recordings in a timely manner to avoid recall bias
In how many conferences and committees did project staff participate in their professional capacity?
Number of applications for conference presentations and committee memberships submitted; number of applications granted/approved
Internal staff activity records; project management records
Accurate recordings in a timely manner to avoid recall bias
Has NSYHP taken initiative in capacity building through collaboration and advocating?
Perception among staff and relevant stakeholders
Survey (Questionnaire)
Validity and reliability of survey; high level of participation
Was the HEADSS psychological risk assessment promoted (quality and quantity)?
Was the HEADSS psychological risk assessment promoted? Yes/No (binary)
Internal staff activity records; project management records
Accurate recordings in a timely manner to avoid recall bias
Were a website and a smartphone application aligning with the project’s objectives ready to be deployed?
Were app and website available in time? Yes/No (binary)
Quality and range of information provided on website and app (checklist)
Internal staff activity records; project management records
Quality assessment with checklist
Accurate recordings in a timely manner to avoid recall bias
Assumes the availability of an appropriate, objective checklist and availability of experienced assessors
Were workshops perceived to be relevant by attendees?
Perception of attendees
Survey (Questionnaire) conducted after the workshops
Validity and reliability of survey; high level of participation
Survey should be conducted directly after the workshop to ensure high participation rates and to avoid recall bias among participants.
Did workshops cover all objectives of the program?
Quality of workshop programs
Observation with checklist
Assumes the availability of an appropriate, objective checklist and availability of experienced observers
Did services meet the demand?
Number of workshop and consultation requests; % of requests attended within reasonable timeframe (e.g., 6 weeks)
Internal staff activity records; project management records
Accurate recordings in a timely manner to avoid recall bias
How many youth consultants were trained and retained (monitored and supervised) over the period of the project?
This has been addressed with prior evaluation questions
./.
./.
How many consultations were provided?
This has been addressed with prior evaluation questions
./.
./.
How many young people accessed website/downloaded the app?
Number of unique website visits, number of app downloads
Website traffic data
Assumes data is accurate and staff has sufficient knowledge to analyse available data
Which areas (app/website) were most important?
Site traffic (number of unique views by area)
As above
As above
How many workshops were conducted and how many people attended workshops?
Number of workshops conducted; total number of participants, average number of particiapnts per workshop
Internal staff activity records; project management records
Accurate recordings in a timely manner to avoid recall bias
How often was HEADSS used if appropriate compared to baseline levels?
% of service providers using HEADSS in clinical practice
Local service provider survey (baseline and end of project)
Validity and reliability of survey; high level of participation
Number of surveys should be limited to avoid survey fatigue (this is a common phenomenon among medical services)
How much non-electronic health promotion resources were provided and used?
Number of leaflets and other health promotion resources distributed; number of leaflets and other health promotion resources available at the end of the project
Inventory records (number of leaflets and other items distributed)
Internal staff activity records (number of sites requesting materials)
Observation (availability of material at the end of the project)
Accurate inventory records.
Accurate recordings in a timely manner to avoid recall bias. Accurate recording of observations
Are local service providers aware of youth-friendly best practice compared to baseline levels?
% of local service provider aware of youth-friendly service at the end of the project compared to baseline
Local service provider survey (baseline and end of project) – as above
Validity and reliability of survey; high level of participation
How many local service providers adapted their service delivery to youth-friendly standards after consultations took place?
Number and % of local service providers undergoing adaptations of standard practice
Survey as above
Potentially observation with checklist for a more reliable measurement
Survey as above
Assumes the availability of an appropriate, objective checklist and availability of experienced observers
Are young people more aware of appropriate health services compared to baseline levels?
% of young people aware of health services at the end of the project compared to baseline
Survey of general youth population
Validity and reliability of survey; high level of participation
Assumes participation among those who did not participate in workshops
Are young people more likely to use health services compared to baseline levels?
Usage (% and total) of health services by young people at the end of the project compared to baseline
Survey as above
Survey as above
Are levels of health literacy in the targeted group higher than at baseline levels (particularly regarding alcohol, tobacco and other drug use, social and emotional health as well as obesity)?
Knowledge of relevant basic information on alcohol, tobacco and other drug use, social and emotional health as well as obesity at the end of the project compared to baseline
Survey as above
A qualitative study may be useful for an in-depth insight into young people’s knowledge and potential gap
Survey as above
Trained interviewers and analysists for qualitative interviews; availability of young people for qualitative interviews – high risk of selection bias
Did young people’s health communication skills improve compared to baseline levels?
Survey as above among young people and among service providers (this provides us with a view from both sides of health communication)
Survey as above
Could a short-term or immediate improvement in mental and social health be observed (reduction in prevalence and incidence of mental and social ill-health)?
Prevalence and incidence of mental and social ill-health in young people at the end of the project compared to baseline levels
Survey as above; public health data
Survey as above.
Public Health Data: Assumes that all incidences are reported to the appropriate authorities. Assumes availability of data for subpopulations and region.
A higher awareness of health issues may lead to a higher incidence and prevalence of diseases in the short term as people are more likely to seek medical assistance and get diagnosed. Interpretation of data might be difficult.
Were school campuses perceived to be supportive environments for health behaviours by young people?
% of young people perceiving school campuses as supportive, healthy environments at the end of the project compared to baseline
Survey as above
Survey as above
Have high schools/high school teachers shown higher levels of awareness of youth-related health issues and promotions compared to baseline levels?
Knowledge and awareness of youth-related health issues (particularly alcohol, tobacco and other drug use, social and emotional health as well as obesity) at the end of the project compared to baseline; measured in %
School/Teacher survey on knowledge and awareness of youth-related health issues
Validity and reliability of survey; high level of participation
Were health promotion materials used and provided continuously in the immediate aftermath of the project?
Number of material requests, number of sites with material, % of material used during the project. Number of sites with material at the end of the project.
Internal staff activity records
Inventory records (number of leaflets and other items distributed)
A question regarding this could be added to the school survey above (school survey) for more in-depth information
Accurate recordings of staff activities in a timely manner to avoid recall bias.
Accurate inventory records
Survey as above
How did the incidence and prevalence of alcohol, tobacco and other drug use, and obesity change during the period of the project?
Incidence and prevalence of alcohol, tobacco and other drug use, and obesity at the end of the project compared to baseline
Survey as above (young people); public health data
Survey as above.
Public Health Data: Assumes availability of public health data for these health issues. Assumes availability of data for subpopulations and region.
A higher awareness of health issues may lead to a higher incidence and prevalence of diseases in the short term as people are more likely to seek medical assistance and get diagnosed. Interpretation of data might be difficult.
Were enough youth-appropriate and -friendly health services available to meet demand?
Number of youth-appropriate and –friendly health services; % of services identified as youth-appropriate and –friendly; degree of capacity utilisation
Local service provider survey (as above)
As above
Are young people use their knowledge consistently to make healthy decisions and choices?
Health behaviour at the end of the project compared to baseline levels.
This has been addressed with prior evaluation questions; collected information needs to be reframed
./.
Scenario 2: Responding to Sexual and Reproductive Health Needs of Displaced Persons after the Zamboanga City Crisis, the Philippines
A. Table 1: Project Description
139065054610Inputs
138239554610Process/ Activities
139255554610Outputs
135509054610Outcomes
Impact/ Goal
What goes into the program?
What does the program do?
What happens as a result of the program?
What are the short/ medium term benefits for participants?
What are the long term or broader benefits of the program?
Certain resources are needed to operate the program.
The program is focused on overcoming sexual and reproductive health crisis in Zamboanga city.
This require giving awareness to the community on safe sexual and health practices.
Planned activities will help in mitigating the risks and overcoming sexual and health crisis in Zamboanga city.
If planned activities are accomplished to the extent intended, then participants will benefit in certain ways.
If these benefits to participants are achieved, then certain changes in organizations, communities, or systems might be expected to occur.
External funding (Government, UNESCO)
Internal funding (International Planned Parenthood Federation)
Staff (Zimbabwe City Medical Centre )
Casual staff (30 youth volunteers)
Stakeholder partnerships
Infrastructure & training resources (Family Planning Organisation of the Philippines (FPOP)
Infrastructure (schools, department of education, public buildings)
Technological skills (app & website development)
Note:
This could also be strucutres according to areas (e.g., financial,…). Different terminology is acceptable as long as it describes the same input and is sufficiently detailed.
Select young consultants who will use their knowledge and skilsl for preventing sexual and health crisis.
Trainings are offered to the consultants for adopting right means for spreading awareness on the issue.
The program will include seminars, workshops and teaching plans.
Trained consultants will directly interact with the community for raising their knowledge on the risks of sexual and health related issues.
Note:
Further information (e.g., specific parts of the training modules) may be provided for a thicker description.
(Increased number of) Youth consultants trained, mentored and supervised
Youth consultants deployed/local service providers received appropriate consultations regarding their services
YouthSource website accessed by young people
Workshops & health promotion education delivered at local high schools and attended by young people
Other (non-electronic) health promotion resources provided – e.g., safe sex practices, hygiene conditions and clean drinking water.
Note:
Some information may be grouped as long as sufficient information is provided.
Raised awareness of youth-friendly best practice among local service providers; improved delivery of youth-friendly services
Raised awareness of health services available among young people; increased usage of health services by young people
Improved health literacy among young people, particularly regarding unsafe sexual activities, negligence of hygienic conditions, HIV transmission and prevention of newborn deaths among young people.
Improved health and sexual life.
Creation of supportive environments on school campuses
Raised awareness of youth-related health issues and health promotion in teaching environments
Continued usage and provision of high-quality health promotion material
Note:
We will concentrate on the benefits for the participants here. However, you may provide further information regarding organisational outcomes or beneficial outcomes for youth consultants. However, the documented outcomes above are also highly correlated with these organisational outputs. The last point may also be categorised as an impact due to its long-term nature.
Improved family planning will eliminate the risks of newborn deaths.
Adoption of safe sex practices and screening will minimize the risks pf developing HIV.
Knowledge of public institutes and legal rights of women will also eliminate the risks of sexual violence.
Improved health status of the community people.
Reduction in deaths and illnesses.
Note:
You may want to separate these points further to provide a thicker description.
General note (Part A): This can be either completed in dot points or with longer explanation – relevant is the depth and complexity of information provided rather than the way it was delivered. Some students provided references in this part – this is appreciated and demonstrates your research skill. Not providing references did not negatively impact on your mark for this assessment.
B. Table 2: Evaluation questions (based on the project description in Table 1)
General note (Part B): Information on the left column can be either copied and pasted as written in the table above or simplified. This did not impact on your mark as you have already stated everything above (there is no need for me to mark this twice). The marking for this part of the assessment concentrated on the evaluation questions, their complexity and the completeness of the table. This also means that missing points from Part A have to be taken into account for Part B. Evaluation questions should be framed in a measureable and answerable way – however, they should not be too detailed since we will provide more information in Part C. Evaluation questions should also be more than simply ‘number of’ – while this is helpful, we would like to gather more comprehensive data that also demonstrates the quality of our project rather than just the quantity of services provided. Some students opted for two or three more general question for each section (rather than each individual point) with sufficient sub-questions to cover more than one input, activity,…
Inputs (taken from table 1):
Evaluation question(s):
External funding (State, UNESCO)
How much funding was allocated for individual components?
$3 million was offered as funds by UNESCO.
Was the funding sufficient for the purpose of the project?
The funding is not sufficient because a larger population in villages of Zamboanga city is facing sexual and health related problems.
OR (more specific) What was the cost of conducting each activity of the project?
The funds only allowed to reach certain areas of the villages not all locations.
Internal funding (International Planned Parenthood Federation)
How much funding was allocated for individual components?
International Planned Parenthood Federation, provided an emergency response grant in the amount of $50,000 USD to Family Planning Organisation of the Philippines (FPOP).
Was the funding sufficient for the purpose of the project?
The funding is not sufficient because a larger population in villages of Zamboanga city is facing sexual and health related problems.
OR (more specific) What was the cost of conducting each activity of the project?
The cost of conducting each activity was managed within the budget allocated .
Staff (NSYHP)
How was staff time allocated to activity types?
Casual staff 30 (Young consultants)
Did the capacity of youth consultants align with the project’s needs?
Young volunteers were extremely trained and managed to align with the project’s need by addressing individual needs of the community people.
Stakeholder partnerships
Infrastructure & training resources (Northern Sydney Health District)
Were infrastructure and training resources allocated to the project appropriate for the purpose and need of the program?
Classrooms at local schools and public buildings were used for sharing awareness and materials.
Infrastructure (schools, department of education)
Were rooms allocated for activities at schools appropriate for the purpose of the workshops?
Yes rooms were allocated appropriately as each had a capacity of holding 150 people.
Technological skills (app & website development)
Were technological skills necessary for the development of mobile phone application and website available?
Technologies were used for building interactive relationship with the audience.
Processes/Activities (taken from table 1):
Evaluation question(s):
Selection of youth consultants
How many applications were received for the positions and how many were hired?
30 applicants were hired for the positions.
Were assessment criteria appropriate for the selection process?
Assessment criteria involved aptitude test that examined consultant’s knowledge of health and sexuality. Interviews were also conducted that involved face-to-face discussions.
Training, mentoring and supervision of youth consultants
Was training for all youth consultants completed prior to program delivery?
Training sessions and workshops were conducted in which consultants were provided practical experience of dealing with the illiterate or less educated people of Zamboanga City.
Were high-quality mentorships and supervision concepts implemented?
Complete information and knowledge was shared with the consultants that included details about the risk factors and prevents of sexual and health related problems.
Consultations with local service providers/Deployment of youth consultants
How many local service providers were identified and invited to participate in the program?
Around 50 local service providers were invite to participate in the program and develop competency for sharing the information with the people of the community.
Was a holistic consultation provided?
Holistic consultation was offered that included goal-oriented sessions and counselling programs for the selected consultants.
Youth health advocacy (participation in conferences and committees; inter-agency and policy maker advocacy)
Was contact made with policy makers and other relevant agencies?
Policy makers were also contacted for making suggestions part of health promotion policy.
In how many conferences and committees did project staff participate in their professional capacity?
The staff participated in 7 conferences.
Could also be more general: Has program taken initiative in capacity building through collaboration and advocating?
The program had taken initiative on building capacity for collaborating with the agencies and organizations.
Health promotion of HEADSS district wide psychological risk assessment instrument (education related to sexual activities, hygiene environment and prevent for newborn deaths)
Was the HEADSS psychological risk assessment promoted (quality and quantity)?
Psychological risk assessment was promoted by qualitative methods as low, moderate, high risks.
The quantity if risks is evaluated by ranking from 1-5 starting from no risk to extremely high.
Designing and deploying YouthSource website and smartphone application
Were a website and a smartphone application aligning with the project’s objectives ready to be deployed?
Yes both smartphone application and website were aligned for coordinating.
Health promotion workshops at local high schools focussing on physical health, promotion of hygiene conditions, safe sex and child care.
Were workshops perceived to be relevant by attendees?
Workshops were relevant by attendees who were people of the village.
Did workshops cover all objectives of the program?
The objectives were covered that focused on improving awareness of the people and raising knowledge. This allowed them to develop healthy attitudes.
Did services meet the demand?
The services meet the demand because they focused on enhancing the overall health status.
Outputs (taken from table 1):
Evaluation question(s):
(Increased number of) Youth consultants trained, mentored and supervised
How many youth consultants were trained and retained (monitored and supervised) over the period of the project?
30 youth consultants were trained and retained.
Youth consultants deployed/local service providers received appropriate consultations regarding their services
How many consultations were provided?
100 consultations were provided to the people of community individually.
YouthSource website and smartphone app accessed by young people
How many young people accessed website/downloaded the app? Which areas were most important?
At least 1000 people accessed the website and downloaded the application. The most important areas were promoting safe birth, sexual and physical health.
Workshops & health promotion education delivered at local high schools and attended by young people
How many workshops were conducted and how many people attended workshops?
10 workshops were arranged for the attendees.
HEADSS used
How often was HEADSS used if appropriate compared to baseline levels?
By the end of month and annually.
Other (non-electronic) health promotion resources provided – e.g., anti-smoking signs, brochures, leaflets
How much non-electronic health promotion resources were provided and used?
Non- electronics resources were also used that include graphic images and visuals.
Outcomes (taken from table 1):
Evaluation question(s):
Raised awareness of youth-friendly best practice among local service providers; improved delivery of youth-friendly services
Are local service providers aware of youth-friendly best practice compared to baseline levels?
Before program local providers lacked knowledge of youth-friendly practices compared to the baseline levels.
How many local service providers adapted their service delivery to youth-friendly standards after consultations took place?
50 local providers adapted the services to youth-friendly standards by focusing on improving level of interaction.
Raised awareness of health services available among young people; increased usage of health services by young people
Are young people more aware of appropriate health services compared to baseline levels? Are young people more likely to use health services compared to baseline levels?
The level of awareness improved among young people after attending the workshops. They exhibited high likelihood of using health service compared to the baseline. This is apparent in their responses where they agreed to follow methods of hygiene.
Improved health literacy among young people, particularly regarding sexuality and health crisis improved health communication skills among young people
Are levels of health literacy in the targeted group higher than at baseline levels (particularly sexuality, hygiene, health crisis, sexual violence and newborn deaths)?
The health literacy of the targeted group improved as they stated their willingness to report issues like sexual violence and for getting screening for HIV.
Did young people’s health communication skills improve compared to baseline levels?
Health and communication skills of young consultant improved as they learned to make use of both verbal and non-verbal communication means.
Improved mental and social health among young people
Could a short-term or immediate improvement in sexual and social health be observed (reduction in prevalence and incidence of sexual and health crisis)?
Immediate impacts were observed as the people who attended workshop started following hygiene and other measures including safe sex and mothers taking good diets during pregnancy.
Creation of supportive environments on school campuses
Were school campuses perceived to be supportive environments for health behaviours by young people?
The campuses offered supportive environment as the instructors removed all communication barriers.
Raised awareness of youth-related health issues and health promotion in teaching environments
Have high schools/high school teachers shown higher levels of awareness of youth-related health issues and promotions compared to baseline levels?
The level of awareness of schoolteachers on the identified issues also increased and they filly participated in removing crisis.
Continued usage and provision of high-quality health promotion material
Were health promotion materials used and provided continuously in the immediate aftermath of the project?
The materials had immediate impacts assessed through the feedbacks taken from audience on questionnaire.
Impact/ Goal (taken from table 1):
Evaluation question(s):
Improvement in physical, social and emotional health among young people in the district (e.g., lower prevalence of sexual violence, newborn deaths, mortality and sexually transmitted diseases) – reduced morbidity and mortality in young people (reduced burden of disease)
How did the incidence and prevalence lower prevalence of sexual violence, newborn deaths, mortality and sexually transmitted diseases change during the period of the project?
The number of sexual violence incidents reported in police stations increased, more mothers started purchasing dietary supplements and more screenings of HIV were performed.
Availability of youth-appropriate and -friendly health services and health environments
Were enough youth-appropriate and -friendly health services available to meet demand?
The services are not enough because a larger population lives in the city and demand more resources.
Young people take “ownership” of their health (long term improvement of transition into healthy adulthood)
Are young people use their knowledge consistently to make healthy decisions and choices?
C. Table 3: Performance indicators, means of verification, assumptions and challenges:
Evaluation question
Performance Indicators
Verification
Assumptions/ Challenges
How much funding was allocated for individual components?
What was the cost of conducting each activity of the project?
% of funds allocated to each project component; cost in AUD per project activity (workshops, consultations)
Project management records; staff activity records
Assumes accurate itemisation of records and resource allocation
Was the funding sufficient for the purpose of the project?
% of funds used after 3 months, 6 months and end of project
Project management records
Assumes accurate and timely recording of spending
How was staff time allocated to activity types?
% of time allocated to: administrative tasks, logistics, interagency collaboration, training and professional development, conducting workshops and consultations (NOTE: This also includes youth consultants
Internal staff activity records
Accurate recordings of hours divided by tasks in a timely manner to avoid recall bias.
Did the capacity of youth consultants align with the project’s needs?
Number of youth consultants trained and available, average hours utilised; % of consultation and workshop requests resulting in action
Internal staff activity records
Accurate recordings of hours divided by tasks in a timely manner to avoid recall bias.
Were infrastructure and training resources allocated to the project appropriate for the purpose and need of the program?
Quality of infrastructure and training (checklists); satisfaction with infrastructure and training resources
Observation; questionnaire (survey of staff)
Assumes the availability of an appropriate, objective checklist and availability of experienced observers; validity and reliability of survey; high level of participation
Were rooms allocated for activities at high schools appropriate for the purpose of the workshops?
Quality of rooms in high school (checklist)
Observation
Assumes the availability of an appropriate, objective checklist and availability of experienced observers
Were technological skills necessary for the development of mobile phone application and website available?
Binary question (Yes/No)
Internal staff activity records
Evaluation questions like this are hard to assess since staff or evaluators may not be able to judge if a technology service provider has appropriate skills or not
How many applications were received for the positions and how many were hired?
Number of applications, % accepted
Management and project records
Records are accurate and collected in a timely manner.
Were assessment criteria appropriate for the selection process?
Quality of assessment criteria used
Checklist/observation
Assumes the availability of an appropriate, objective checklist and availability of experienced observers
Was training for all youth consultants completed prior to program delivery?
% of staff and volunteers completing training prior to program delivery
Project management records/staff training records
Records are accurate and completed in a timely manner. Assumes quality of training is sufficient (this will be addressed below)
Were high-quality mentorships and supervision concepts implemented?
Average number of supervisions and mentoring sessions; satisfaction with supervision and monitoring by mentors and mentees
Internal staff activity records; survey (questionnaire)
Accurate recordings in a timely manner to avoid recall bias, validity and reliability of survey; high level of participation
How many local service providers were identified and invited to participate in the program?
Number of local service providers identified, % contacted and invited
Internal staff activity records; project management records
Accurate recording of staff activities and project management records in a timely manner
Was a holistic consultation provided?
Quality of consultations (checklist); satisfaction with consultation among service providers
Observation with checklist
Assumes the availability of an appropriate, objective checklist and availability of experienced observers
Was contact made with policy makers and other relevant agencies?
Was contact made with policy makers and relevant agencies? Yes/No (binary) Number of policy makers and relevant agencies contacted
Internal staff activity records; project management records
Accurate recordings in a timely manner to avoid recall bias
In how many conferences and committees did project staff participate in their professional capacity?
Number of applications for conference presentations and committee memberships submitted; number of applications granted/approved
Internal staff activity records; project management records
Accurate recordings in a timely manner to avoid recall bias
Has NSYHP taken initiative in capacity building through collaboration and advocating?
Perception among staff and relevant stakeholders
Survey (Questionnaire)
Validity and reliability of survey; high level of participation
Was the HEADSS psychological risk assessment promoted (quality and quantity)?
Was the HEADSS psychological risk assessment promoted? Yes/No (binary)
Internal staff activity records; project management records
Accurate recordings in a timely manner to avoid recall bias
Were a website and a smartphone application aligning with the project’s objectives ready to be deployed?
Were app and website available in time? Yes/No (binary)
Quality and range of information provided on website and app (checklist)
Internal staff activity records; project management records
Quality assessment with checklist
Accurate recordings in a timely manner to avoid recall bias
Assumes the availability of an appropriate, objective checklist and availability of experienced assessors
Were workshops perceived to be relevant by attendees?
Perception of attendees
Survey (Questionnaire) conducted after the workshops
Validity and reliability of survey; high level of participation
Survey should be conducted directly after the workshop to ensure high participation rates and to avoid recall bias among participants.
Did workshops cover all objectives of the program?
Quality of workshop programs
Observation with checklist
Assumes the availability of an appropriate, objective checklist and availability of experienced observers
Did services meet the demand?
Number of workshop and consultation requests; % of requests attended within reasonable timeframe (e.g., 6 weeks)
Internal staff activity records; project management records
Accurate recordings in a timely manner to avoid recall bias
How many youth consultants were trained and retained (monitored and supervised) over the period of the project?
This has been addressed with prior evaluation questions
./.
./.
How many consultations were provided?
This has been addressed with prior evaluation questions
./.
./.
How many young people accessed website/downloaded the app?
Number of unique website visits, number of app downloads
Website traffic data
Assumes data is accurate and staff has sufficient knowledge to analyse available data
Which areas (app/website) were most important?
Site traffic (number of unique views by area)
As above
As above
How many workshops were conducted and how many people attended workshops?
Number of workshops conducted; total number of participants, average number of particiapnts per workshop
Internal staff activity records; project management records
Accurate recordings in a timely manner to avoid recall bias
How often was HEADSS used if appropriate compared to baseline levels?
% of service providers using HEADSS in clinical practice
Local service provider survey (baseline and end of project)
Validity and reliability of survey; high level of participation
Number of surveys should be limited to avoid survey fatigue (this is a common phenomenon among medical services)
How much non-electronic health promotion resources were provided and used?
Number of leaflets and other health promotion resources distributed; number of leaflets and other health promotion resources available at the end of the project
Inventory records (number of leaflets and other items distributed)
Internal staff activity records (number of sites requesting materials)
Observation (availability of material at the end of the project)
Accurate inventory records.
Accurate recordings in a timely manner to avoid recall bias. Accurate recording of observations
Are local service providers aware of youth-friendly best practice compared to baseline levels?
% of local service provider aware of youth-friendly service at the end of the project compared to baseline
Local service provider survey (baseline and end of project) – as above
Validity and reliability of survey; high level of participation
How many local service providers adapted their service delivery to youth-friendly standards after consultations took place?
Number and % of local service providers undergoing adaptations of standard practice
Survey as above
Potentially observation with checklist for a more reliable measurement
Survey as above
Assumes the availability of an appropriate, objective checklist and availability of experienced observers
Are young people more aware of appropriate health services compared to baseline levels?
% of young people aware of health services at the end of the project compared to baseline
Survey of general youth population
Validity and reliability of survey; high level of participation
Assumes participation among those who did not participate in workshops
Are young people more likely to use health services compared to baseline levels?
Usage (% and total) of health services by young people at the end of the project compared to baseline
Survey as above
Survey as above
Are levels of health literacy in the targeted group higher than at baseline levels (particularly regarding alcohol, tobacco and other drug use, social and emotional health as well as obesity)?
Knowledge of relevant basic information on alcohol, tobacco and other drug use, social and emotional health as well as obesity at the end of the project compared to baseline
Survey as above
A qualitative study may be useful for an in-depth insight into young people’s knowledge and potential gap
Survey as above
Trained interviewers and analysists for qualitative interviews; availability of young people for qualitative interviews – high risk of selection bias
Did young people’s health communication skills improve compared to baseline levels?
Survey as above among young people and among service providers (this provides us with a view from both sides of health communication)
Survey as above
Could a short-term or immediate improvement in mental and social health be observed (reduction in prevalence and incidence of mental and social ill-health)?
Prevalence and incidence of mental and social ill-health in young people at the end of the project compared to baseline levels
Survey as above; public health data
Survey as above.
Public Health Data: Assumes that all incidences are reported to the appropriate authorities. Assumes availability of data for subpopulations and region.
A higher awareness of health issues may lead to a higher incidence and prevalence of diseases in the short term as people are more likely to seek medical assistance and get diagnosed. Interpretation of data might be difficult.
Were school campuses perceived to be supportive environments for health behaviours by young people?
% of young people perceiving school campuses as supportive, healthy environments at the end of the project compared to baseline
Survey as above
Survey as above
Have high schools/high school teachers shown higher levels of awareness of youth-related health issues and promotions compared to baseline levels?
Knowledge and awareness of youth-related health issues (particularly alcohol, tobacco and other drug use, social and emotional health as well as obesity) at the end of the project compared to baseline; measured in %
School/Teacher survey on knowledge and awareness of youth-related health issues
Validity and reliability of survey; high level of participation
Were health promotion materials used and provided continuously in the immediate aftermath of the project?
Number of material requests, number of sites with material, % of material used during the project. Number of sites with material at the end of the project.
Internal staff activity records
Inventory records (number of leaflets and other items distributed)
A question regarding this could be added to the school survey above (school survey) for more in-depth information
Accurate recordings of staff activities in a timely manner to avoid recall bias.
Accurate inventory records
Survey as above
How did the incidence and prevalence of alcohol, tobacco and other drug use, and obesity change during the period of the project?
Incidence and prevalence of alcohol, tobacco and other drug use, and obesity at the end of the project compared to baseline
Survey as above (young people); public health data
Survey as above.
Public Health Data: Assumes availability of public health data for these health issues. Assumes availability of data for subpopulations and region.
A higher awareness of health issues may lead to a higher incidence and prevalence of diseases in the short term as people are more likely to seek medical assistance and get diagnosed. Interpretation of data might be difficult.
Were enough youth-appropriate and -friendly health services available to meet demand?
Number of youth-appropriate and –friendly health services; % of services identified as youth-appropriate and –friendly; degree of capacity utilisation
Local service provider survey (as above)
As above
Are young people use their knowledge consistently to make healthy decisions and choices?
Health behaviour at the end of the project compared to baseline levels.
This has been addressed with prior evaluation questions; collected information needs to be reframed
./.
