Evaluation process and evaluation criteria
Contents for this page
The evaluation of practices is based on open peer review, which increases the visibility of the evaluation process. A peer reviewer is an independent expert from outside of the editorial team who is not affiliated with the practice under review. At least two peer reviewers are selected for each practice. The representatives of the practice are responsible for the description of the practice. The peer reviewers provide an statement on the evaluation. These statements are public.
The practice evaluation process involves applying standardised criteria to evaluate the evidence for the practice and its effectiveness and practical applicability. The evaluation is conducted using a scale of 1 to 5.
Evaluation process
The evaluation process proceeds as follows:
- The representative of the practice submits the practice for evaluation,
- the peer reviewers make an evaluation of it,
- the editorial team oversees the evaluation process and
- the description and evaluation of the practice is published on the THL website and in the Practices in Health and Well-being promotion publication series in Julkari.
The practice representative
The evaluation process begins when the practice representative submits the basic details of the practice to the editorial team. If the editorial team approves the initiation of the evaluation process, the practice representative then submits broader and more detailed information to the editorial team in line with the evaluation criteria.
Editorial team
The editorial board first assesses whether the practice is suitable for evaluation in terms of its theme, content and quality. If approved, the practice is then put through the peer review process.
Peer reviewers
At least two peer reviewers are selected for each practice. They are independent experts from outside of the editorial team who are not affiliated with the practice under review.
The evaluation is based on standardised evaluation criteria. The peer reviewers provide a statement on the evaluation. The evaluation can be conducted either anonymously or openly.
Peer reviewers do not receive any compensation for their work. Peer review of practices is an important expert task and also a scientific merit.
Evaluation rounds
The evaluation of a practice may require several evaluation rounds. The practice representatives respond to any requests for clarification and resubmit to the editorial team the practice description once the suggested improvements have been made. The evaluation process may also involve dialogue between the practice representative and the peer reviewers. The editorial board decides on the approval and publication of the evaluation.
Duration of the evaluation process
The evaluation process is case-specific and is initiated at the desired time. The duration varies from a few months to half a year.
Publication of evaluations
All the published evaluations of practices have undergone the evaluation process. The practice representative and peer reviewers receive details of the expert statements.
The practice evaluations are published in the THL online service on the Management of Health and Wellbeing Promotion website (in Finnish) and in the Practices in Health and Well-Being Promotion publication series.
Evaluation ethics
The evaluations apply ethically sound evaluation methods and principles which are in line with scientific criteria. The practice representative and the peer reviewers commit to promoting the principles of good scientific practice published by the Finnish National Board on Research Integrity TENK.
Managing the evaluation process
THL stores the key information and documents for all practices that have been approved for evaluation. For peer-reviewed but rejected practices, the name of the practice, the name of the practice representative and the names of the peer reviewers are stored. For practice evaluations which are published, the stored information also includes details of the practice, the peer reviewers’ statements, and the decisions on the publication of the evaluation. The data stored for the purpose of managing the evaluation process is not public information.
Archiving of evaluations
THL sees to the archiving of evaluations. We make sure that evaluations are easily locatable, accessible, and kept in long-term storage. The evaluations of the practices are archived in the Practices in Health and Well-Being Promotion publication series that is in an open publication archive in Julkari, which is freely accessible and does not require registration. The details of evaluation results archived in Julkari are public information.
Copyright
Matters of copyright is regulated by the Copyright Act (8.7.1961/404). This act protects the author's economic and moral rights to his work. A published evaluation of a practice is always a work. In comparison, the practice itself is a work, even though the definition may vary on a case-by-case basis. When a practice evaluation is archived, the copyright for the practice does not transfer to Julkari or to the Finnish Institute for Health and Welfare.
Authorship of the practice
Practice authorship is a recognition of participation in a creative process and indicates responsibility for both content and results. It is important that the person who submits the practice for evaluation is either the person responsible for the practice (e.g. practice owner) or a practice representative who is involved in the evaluation process. The practice representatives should agree in advance on who will submit the practice for evaluation and how the authors will be reported in connection with the evaluation.
Citing evaluations
Every practice evaluation is a publication, and should therefore be cited appropriately by using a source reference. Where the evaluation has been carried out anonymously, the names of the reviewers are not mentioned. If the reviewers’ names have been published, they should also be included in the source reference.
The source reference can be given as follows: Names of the practice representatives (year). Evaluation of the practice XXXX. Names of the peer reviewers (peer reviewers). Finnish Institute for Health and Welfare. Practices in Health and Well-Being Promotion. Date of the evaluation.
Proposing the evaluation of a practice
Practices can be proposed for evaluation by sending an email to the editorial team at hytearviointi(at)thl.fi
The office is also happy to receive information about health and wellbeing practices which are in use in Finnish municipalities and regions.
Evaluation criteria
Suitability of practices for evaluation
An practice is suitable for evaluation if it provides reliable evidence-based data (research data and experiential and expert knowledge) which
- promote wellbeing, health, safety and social inclusion,
- promote justice and equality (e.g. gender, socio-economic position, ethnicity, residential area),
- prevent health and welfare inequity, inequalities, social exclusion and loneliness,
- support the socially, ecologically and economically sustainable development of society.
Theme areas
Evaluations focus on practices from different sectors of society. The theme areas include, for example, physical activity, nutrition, functional ability, mental health, substance abuse, safety, culture, transport and housing.
Evaluation categories
A practice is first assigned an evaluation category and then evaluated.
- New practice (under 4 years)
- Practice currently in use (5-10 years)
- Practice in long-standing use (over 15 years)
The four areas of evaluation criteria
The evaluation criteria for practices are divided into four areas:
- Basics of the practice and suitability for evaluation (approval for the evaluation process)
- Impact chain of the practice
- Applicability to practice of the practice
- Evidence of the practice
The evaluation uses a scale of 1 to 5
The practice is first assigned an evaluation category, for which it is then evaluated within the different evaluation areas on a scale of 1 to 5. The scores from these areas are combined to give an overall evaluation on a scale of 1 to 5.
5 = excellent
4 = very good
3 = good
2 = satisfactory
1 = poor
