This is the fourth in a very number of articles on program evaluation. You will find the others also posted on this web site. These articles are designed to provide agencies with an breakdown of the program evaluation process along with the ways to ask informed questions of professional consultants which they assist.
Taking without any consideration which you have worked over the previous steps in this technique which are outlined in the last articles, once you are able to hone in on the question of the you are likely to measure inside your evaluation, and how. The answer to this question is both simple and complex. Simple in this you will follow the lead you’ve put down on your own by consulting your list objectives (outcome statements) and centering on those which are within the scope of this particular evaluation. Complex, in this measuring those outcomes just isn’t nearly as common as it can seem.
Begin by questioning whom you need information from. If your real question is, «Are our clients pleased with the level of support you can expect because of their job searches?» you will want some tips of soliciting your clients’ views. On the other hand, if your real question is, «Do those who complete our program hold the necessary skills to get yourself into the career they trained for?» you may want information from the program graduates, but perhaps also from employers who’ve hired your previous graduates. Be sure to also consider accessibility and time demands.
From this point, consider if you are searching for what your customers KNOW, CAN DO or their ATTITUDES and BELIEFS. This question will advice the all your planning your evaluation.
If you are interested in everything you clients KNOW, you likely require a test of some kind that measures their mastery of somebody of information. Ideally you ought to have some criteria to compare their performance against; some national norms and/or their particular previous performance.
If you are looking at what your customers CAN DO, you will require some method that monitors the behavior(s) showcased. For example, you might want to analyze your customers’ resumes to discover should they have adequately incorporated the guidance provided in the recent work out. Or, if you wish to know if the clients are able to employ specific parenting techniques, some direct observation of parent/child interactions could possibly be necessary.
If you are searching for the ATTITUDES and BELIEFS of your customers and other stakeholders, you’ll need to design some way of them to let you know what you think, probably anonymously. Surveys may be common, as are focus groups and interviews. The make-up of your client base as well as the sensitivity of the topic should be considered in deciding which method of use. One strong note of caution: these methods are easy to do badly. Many think administering a survey is by recording a few pre-determined questions for some individuals to respond to. Or, that interviews while focusing groups are as simple and sitting people right down to have a very conversation. Nothing could be further from the truth. To utilize these techniques in the way that leads to credible information appears uncomplicated, but requires the maximum amount of care and planning because most complex laboratory experiment.
Finally, before you actually progress using your data collection, revisit your scope, purpose and objectives once more. Is the approach you’ve planned prone to yield techniques to the questions that stimulated this evaluation?
I’ve suggested in other articles with this series that you ought to seek the guidance and support of an professional in this method. Now is REALLY not enough time to visit it alone. Search where you live for consultants that do program evaluation Research Solutions. If that fails, contact a local university for a suggestion; focus on one of the social science departments.