-Welcome everybody. This is part two of our social impact series for those watching the recording there is a link underneath this video which links you to the social impact part one and I’ll talk about that in just a moment before we get started. I wanted to acknowledge the traditional owners of the land of First Nations people right across the Australia. Actually. We’ve got a number of people from different different states with this and and I’m in dhubu-gu country, which is you we’re gradually land so that’s that sort of where we are at and I know we’ve got people in wa and and people in New South Wales as well Beautiful. All right. So let’s I do have an apology the wonderful Ali Mumford. Unfortunately is sick. She can’t can’t even talk and it’s kind of one of those things that we’re dealing with now is is flu on top of covid. So she does pass on her apologies, but it does still stand for those of you, you know, if you’ve got any questions about any of this feel free to reach out either to myself or Ally and And you can connect I’ll give you the follow-up email you’ll get her details and of course those watching the recording you’ll find that on our subject matter expert. page so just a quick recap in the last session. We talked about social impact and what we needed to understand because I know that there were people doing today that weren’t in the the first session and maybe not had time to watch the recording and this was about social impact about you know needing to understand the problem. We’re solving how you know, the causes and effects and how they’re actually connected to the problem. But also the environment that’s around the problem because we don’t simply solve the problem alone. We operate in an ecosystem and it’s really important to understand what that ecosystem is and also important because we can leverage the work that others are doing in this space as well. And it was also about recognizing, you know, that specific Niche that you do operate in within that a coast system so very much about you know, identifying the difference that we want to make in that Niche and the part we play so we looked at the theory of change and it was A simplified theory of change to say, you know, what is the difference we want to make and and what do we do in this space? And it was about really identifying purpose and any intervention or issue social issues Focus that you might have and it was about saying okay, but how does this relate back to our purpose? So it was just really checking in with that alignment piece around why you’re organisation, you know what it’s strategy is it’s vision and why it’s doing what it’s doing then to the follow-up email with the link to that recording but also a link to a series of videos on the logic model which I will recap as well in this session. So I won’t explain too much about that because that’s what we’re doing today. And we’re also looking at the evaluation framework, which is the measurements and the tools and having a discussion about that. And yes, I’ve got some really handy tools that are going to help you do this. So let’s get started and have a look at the logic model. We’re gonna have an opportunity to to kind of do some of this workout selves. The original sources and great for those of you that really wanting to look at this in depth, you know, the Kellogg’s Foundation logic model handbook and evaluation handbook is really valuable. But but certainly, you know, what gets measured, you know is what we focus on so the logic model. Visually and I will do a visual descriptor as we always do for inclusion purposes for those listening in or around or who have a visual impairment. So the logic model was about stepping through stages. Once you have defined your theory of change in essence, you know, what you’re intervention is, then what you need to do is you’re kind of need to map what you’re doing and the difference that you’re having and then question if what you’re actually working on is is impacting and influencing the theory of change that you developed. now some organizations actually call the logic model a theory of change and that’s because it clearly articulate sort of that Global, you know statement that they make we haven’t done that for the purposes of making sure that for everyone that it’s really clear the three steps we go through and the logic model talks about the input. So what are the resources that you put into it? What are the things that you do with those resources? What is it deliver immediately deliver. That’s an output. And and then we talk about that ripple effect, you know sort of the outcomes the benefits and and typically in in this process. We actually talk about this from a funding perspective. So there’s outcomes would be within 12 months and then you’re impacts are 12 months and Beyond, you know, there are some programs that are quite complex and they will actually do short-term outcomes medium term outcomes long term outcomes. So those long-term outcomes for some things like behavioral changes closing their gaps that are can be in excess of 20 years. So, you know, just depending on the sorts of program you’re working on particularly when you’re working on something around human behavioral change or improvements then you know, it’s certainly a lot longer often than the the three years that you may have down for your impacts. A couple of key things that that you also need to consider is the assumptions that you make, you know, there’s plenty of assumptions that we make that people even want these program. For example that you have the resources the skills the Partnerships Etc. It’s important to recognize what those assumptions are as you go through one of the other things that I’m highlighting in this session is what we call attribution and what that means is that there’s a very big difference between remember I talked about that ecosystem earlier, so There’s a very big difference between something that you directly impact and you can say a hundred percent that happened because of me versus you know, other support, you know other things that they’re getting access to within that ecosystem that has an effect on the the difference that your program makes and so we’ll talk a little bit more about that. But that’s that it attribution pieces a really important piece in looking at your measurements. So it’s really important when you do that logic model that you actually have those conversations with stoke-ons. What I actually used to do before before I knew there was a tool. I think it was created at that point. It might have been very early on before I knew there was a tool called social return on investment and logic model. I actually used to draw a picture of all the different people that we’re supporting or that are impacted as a result of the work we do and then we would actually map their Journeys. So for example, if we were increasing delivering a program and increased employment, we would say, okay great. So we increased employment that increases the amount of money that they’re bringing in but what does that enabled them to do that in mind enable them to get out of social housing or it might enable them to get better housing. It might also enable them to get access to Better Health So we kind of mapped the Journey of those impacts just using a mind map and that’s a really really simple tool a creative way to get people involved. Sometimes when people look at sort of tables and and start getting involved in those so complex systems that can be a little bit disconcerting for them. So a little mind map talking about, you know, people can be a really great approach if you’re working with a group where this may be the first time they’ve heard of it. um, so How did you go before we move on did anyone managed to have a look at the the logic model and do a little bit of mapping before the session? Hey, Matt, just joining us. How are you? Well, thank you. That’s good. That’s good. Did anyone get a chance to have a look at that logic model and and even do one sort of line for there? Project or program? It’s okay to say no, don’t worry. I love that thumbs down emoji. You’ll love that. Hey, we’re all busy. It’s all good. And for some of this, you know this sort of series of workshops, you know isn’t necessarily what you’re doing right now. You might just be starting to to move into that. All right, that’s okay. So I’ve got a couple of examples for you and we’re gonna work through one. um This is an example. I’m going to use a less complex one for discussion. But this is an example of actually one that I developed recently for successful Grant. So I had done this is in dubbo where I’m not buy New South Wales. So I had the the St. John’s Club had been running a series of well-being workshops and they’ve been doing it, you know for four years or so that I’ve had some great impacts Unfortunately. They hadn’t been measuring anything. So, you know, we had some stories but we didn’t have any direct measures that we could attribute to their program which then makes writing a great application and providing evidence that they know what they’re doing a little bit harder, doesn’t it? um So part of the the project that I said to them given the amount of funding they wanted given who’d received that funding previously and the level of organizations that have received those amounts. I said to them straight away. We’re not doing a series of workshops. We’re running a program. So we got an expert involved who’s actually going to be managing this project and she’s next part. Sorry X principle and he’s actually studying and I’m gonna get this wrong. I know it’s not my area studying meta metacognition and it’s role in teenage development. I think will being it’s something around that line. So So I actually pulled this together and pulled this together via research that I’d done via conversations and I had with people involved and feel free to take a screenshot of this as well. I will include this in the material below the videos that you’ll be able to grab as well. So this is why I pulled together from having you know that I consultation around what are the differences in previous programs, you know, what are the needs we have Etc. And so this is where some of the the statistics have come through and we identified in the activities. We actually had this broken down into the groups of people. So there are there are needs but the fact is that this program again, it operates in ecosystem. We want to improve the well-being of all of the people within and those three organisations involved in the partnership that I brought together and we wanted to improve those three groups. So Genie players adult players and coaches. And so we had very specific activities that we were doing and then we broadly talked about outcomes. We didn’t attribute those outcomes to specific groups and the same with impacts. So this is what I would call a broad program logic right? It’s not very specific in terms of if we do this, what does that specific? You know, let’s just say coaches if we deliver psychological first time, what does that specific psychological first day workshop deliver in terms of benefits to individuals? But it’s certainly enough to start working on and sort of conceptualizing and it’s enough to say, you know, you can put down all of the outcomes you like, but are they actually realistic is that actually things that you can measure? In the timeframe, you’ve got to work through so I did want to show you this one because it is a little bit more of a complex one. Then the simplified version I sent out in the recordings, but we’re going to be working on the simplified one. I find it’s it’s actually much easier just to start with when when you’re working on these things particularly if it’s new to start with a simplified version and we’re actually going to be sort of work shopping that the measurements and the evaluation process tonight. So You’ll see that as we work through any questions on this one before I move on. Not pretty yet. Pretty right. Of course. I came up with a lovely title that you know healthy Minds healthy teams. So fit with the funding guidelines, but it also fit with the program what they wanted to achieve. You know, they always I always like that now this is something that you would classify as a projected program logic. So this is stuff that we hope to achieve and of course what the program is going to do is it’s going to be testing these things as we go through. So let’s look at the simplified version and this is one that I I sent out the mapping exercise in the video I recording and I just use I I have actually happened to be up in the pilborough, um delivering some training and so I just use an example of grant writing training and and some of this this is, you know, kind of made up a little from experience and feedback I get but but also one from other things that have happened in that sessions So, um the inputs and I’ll just I’ll just deal with the Top Line because I want to want to show you what we talk about when we talk about that attribution piece. So what we’re saying is that we you know, ultimately what’s the impact we want to achieve we want to make sure that people are less. This is often what I hear from councils less reliant on Council for funding that they’re aware that there are other Grand opportunities that they can increase funding into the community, but also they’re more sustainable. So they’re actually they’re now they have expanded their knowledge and tools and capability to our skill base basically to apply for Grants. There’s also less Reliance on Council to help them, you know as they go through particularly perhaps for smaller Community grants. So if that’s the impact that you want to achieve then in some of the outcomes that we expect is an increasing Students and knowledge retained knowledge this knowledge that we’ve learned. At that Workshop but knowledge retained. And then of course increasing grants applied for in Grants awarded. So, you know, if we want to do that, what do we need to what do we need to to see happen as a result now often that confidence case actually happens as an output. So it happens directly in immediately at the workshop, you know, you are, you know get that feedback and I know that I’ve got a few people here that work in this space around trading and and they would understand that, you know, if someone feels that I’ve gained confidence then they’ll actually tell you in the survey that you do the workshop. And then you backtrack that again to activities and to resources. so a fairly simple model that we’ve applied but this is the difference around attribution. We can’t over claim results and I’ll just give an example. I received a wonderful email yesterday. I was like, oh, that’s so lovely. It’s so nice to hear this from somebody that that I first met 10 years ago and and trained in grant writing and this person. So as I said first met 10 years ago, but they attended a number of sessions and because you know as most of you know, I have sort of a used policy where I say, yeah bring me if you’ve got questions or send me a month. So I did spend some time supporting answering questions sending a resources having a look at something, you know, it was probably for a period of four to five years after And and then that sort of eventually dwindled and this person wanted to start a grant writing business. And so I got an email yesterday saying that in the 10 years. She’s raised for her community raised 600,000. So that’s just awesome. But can I claim that? It’s all a result of me. What do you think? You know nice but the answer is no, isn’t it Kim? Yeah. Because what else happens in that ecosystem she learns from other people. So we may have talked about economic analysis and maybe she went off and learned about that from someone else pick something else. I mean, these are what we do don’t we when we’re learning things we we learn different and it’s important to actually learn from different people because we have they have different lenses and experiences. And different insights they bring to you that broadens your learning so so really sickly. No, we can’t claim that and look even though I’d kind of ad hoc supported her, you know pro bono wasn’t a fee for service thing at Hop supported her in the sort of fortify year period i, there’s no way I could claim attribution. So it is important when you look at this from a measurement perspective that we don’t overclaim and I’m going to going to introduce you to the seven principles of social return on investment and in Social return on investment. Some of us. I know I’ve heard about it. It’s that whole if you for every dollar that you invest it delivers extra turn. So for example, I’m going to R&D which you know for me stands for rip off and duplicate, you know, my friends says stay or with permission the center for social impact in Western Australia has done some studies around, you know, for every dollar that’s invested into Community capacity building. What is a social return investment. So it’s six dollars. So for every dollar invested into Community capacity building, it delivers a six dollar return. So so I can certainly you know use an attribution as a as an example, but when I’m actually reporting, you know setting up my logic model My outcome syncing about what I’m going to track and measure and how I’m going to do that but also importantly reporting to a thunder. I need to be really clear about what we’ve been able to impact. And sometimes you can just report on so many things that actually really have nothing to do with all minimal to do with the impacts that you want to achieve. So there are things that we can control like the inputs and the resources, you know, these are within our control generally. Yeah sure things might change so I get sick. You can’t get the same China but at least you know, you have control over what that looks like. You can reasonably be held accountable certainly for the three. So the first three columns inputs activities and the direct output. So we have a China they turned up at the workshop and transferred knowledge. And then what did everyone think and what did they gain from it so you can be held reasonably accountable for those things. Sometimes the medium term so that within 12 months you can also so for example, if there is additional support that they receive In recent Council example, it’s you know, yes, they receive additional support because they’ve got somebody actually working with them on let’s actually make sure we do something with this. But can we 100% say that those long-term impacts? So using the example, I gave you that over 10 years 600,000. No, not at all. And so we need to really be clear about about that and be clear that particularly, you know, when you’re talking about grant funding what we can measure the things that we can reasonably be held accountable for you know to say that, you know, we’re entirely responsible for you know, bringing all of that money into the community because you held a grant writing Workshop three years ago over climbing. So I just wanted to make that that point up front. um So a couple of points on measuring indicators so the indicators when I say indicators just to clarify. I’m talking about outputs outcomes and impacts. Although again. If you’re looking at this from a developing a program that’s going to be able to be reported on within that 12 month period you would be looking at the direct and immediate indicators those outputs and then within those 12 months, which is the outcomes. So these are some questions. I won’t read them all we are going to put them in the the text for those who are visually impaired or watching this online, but I’ll just pick out a couple and and just have a look at for those of you able to to see the screen. I’m I’m aware that those joining us don’t have visual impairments and are able to see the screen or I think I did see Suzy. Yep, you’re on a computer beautiful making sure you went 25 minutes, excellent. So if there’s any particular question here that you would like to discuss amongst us. Um, please jump in. I’m just gonna pick a couple I’ve talked about the attribution. It’s important to think about the measurement or that indicator and whether you need a qualitative or quantitative measure, so qualitative being the stories Christine and I just before the session we’re having a great conversation about that and I’m not ask her to one to jump on and and give some add some thoughts to this from a funding perspective that she shared and quantitative. Is that the numbers Christine I didn’t record our earlier conversation, but I would love for you to share if you’re open to it that some sort of funded perspective around your experience and what you found. well, yes, so I mean background is I’ve A number of times in my career and my working life. I’ve managed Grant programs and we were always really looking for the some measurement of the impact that has been made. So people who report on the the outputs number of sessions held number of people participated, but then it came to okay so and what difference does it make which is what we’re talking about today and the ones that from a in terms of his storytelling could be really powerful and particularly when you’re reporting up to other bodies, like in our instance. We were talking to a board and then it was actually reporting from Minister you have quantitative data, that’s fantastic, but people glaze over a bit in true. Case studies were really powerful in case studies could be written format. Some people could go to the as far as doing videos, like little video segments doesn’t agree overly complex or technical but the stories can pay a thousand words, you know, it’s that classic line. So I think a mixture of the quantitative and the qualitative I can actually make a difference to decision makers because the reality is is for every dollar that you’ve got to give away through a grant program. You’ll have a request for $10, you know, so there’s always over the subscribed funding bodies want to make the most difference in whatever area that they’re interested in whether it’s environmental projects or social projects or Economic Development. They’re looking for the biggest bang for the book basically. and so particularly the social space. It’s I mean, it’s really hard. I mean people talk about cost benefit analysis. I mean, it’s so fuzzy. It’s just not funny particularly in that social space and I will be really interested to look more about the the social return on investment model and I’ll go back and have a look at that again. But I’ve just go back to even if it’s storytelling in case studies. It can be really powerful but they do take energy and effort and resources to pull together. So I accept that I think decision making bodies actually do like them in my experience. Yeah, and and I totally concur with that having been on that side as well and and you’re right around the resourcing and I think that’s why it’s important for organizations to determine. What sample size is acceptable because we don’t necessarily have to interview everybody. Do we? No, no absolutely can be but unique you do need in your own mind to think that it’s the first snapshot. It’s you know, so you’re not just cherry picking. Full new bodies actually get a bit suspicious where everything’s perfect because we don’t trust it. We don’t trust it. They just human so I think you actually have to show the walls and all as long as you can demonstrate what we’ve learned from that you know, we thought this was going to be the outcome. Actually we found this was the problem. And that wasn’t the outcome of achieve but what we’ve learned this in the world apply this to Future programs. So I think that’s as important as oh, yes, we achieved 100% of our objective because people never do Yeah, exactly and some thank you so much Christine for sharing those points. We’ll reinforce some of those as well through the rest of the the session today, but generally sorry it’s Kim Kim. So just in that conversation there about who do you need to collect the information from? Yeah that ripple effect of you know, your your plot the stone in the water and you wanting to directly influence and impact, you know, our particular group hitting them with the stone, but we know that from that ripple effect, it will potentially move out and you know, hopefully that’s part of what you’re wanting to do. So at what point do you stop collecting through that Ripple to the edge of the pond so example, I can give in sometimes in our work with sporting clubs and Community groups and I’ve done some work just recently with Christine. We worked with a particular Club association and region But what happened was that ripple effect went out much broader than just one region and the influence in the impact of the project. We ran ended up Rippling out across the state. So at what where do you stop or how do you collect that information at? What point do you go? You know what I can’t go out into the universe and collect it all where do I need to keep my focus? I think the first question is why are you wanting to collect it? So for example, we’re about to start our as a certified social Enterprise. We’re about to start out impact for the last two years. Um our impact report, we’ve got another program. We’re just developing and we’ve chosen to commit the resources because we know similar to that that actually has a potential with the effect across the state for this program, but we’ve chosen to connect resources to measuring that for up to 24 months. So, you know two years after the programs finished and that’s you know, that resourcing issue that Christine mentioned. That’s really what for many actually determine how much and how long they track so that longer children or study how how long they actually track those measures so that would be my first question why you’re doing and I mean we’re doing it because we’re a certified social Enterprise and a link to an impact and to be honest is equally we we believe that that’s what we should be doing. So we we are committed to resourcing it and in the program we’re actually doing it because this is a really great. Available program that we actually want to put out into more regions and it’s actually going to help those regions justifying investment. So yeah, that would be my first question the other thing and I just want to come back to what Christine mentioned about interviews the other thing and another sort of thing to support the validity of doing, you know, sample size 12 interviews is that people might say? Oh, yeah, you know take a survey and and to the box and sound increase confidence, but when you actually have a conversation with them and you can drill down into that piece, you’ll find that there’s actually a lot more impacts. So, you know again that’s that value of doing that and you know, she mentioned from a program perspective, you know, we can all measure what what is up to 12 months, but you know, ideally it’s about okay, but what is that show us that from an impact perspective that we’re already making a significant impact on so for example, if you’re doing a Youth Employment Program and you said, you know, the impact that we hope to achieve is that you know, they’ve improved their health they’ve reduced these it’s to you know mental well-being support services and they’re able to live in a better housing. Will for some that they’re actually going to achieve that impact a lot earlier than others. So it’s important not to just say well, we’re only measuring this when the the end goal is to say. Well we want to measure indicators of this happening and an indicator of that happening might be that that somebody has said, you know, I’ve got part-time work and I’m looking to move into another house. That’s an indicator. So maybe it’s not yet achieved but there’s progress. Um, so another thing and that this comes back to the resourcing conversation as well is it’s important to stop and think about what we’re already doing in measurement and how we can actually, you know, it’s not about creating a new process and your system because we all know we’re under resource. So, how do we how do we make sure that we can gather something for the program, but perhaps within our existing processes and that sample sizing again another another point on that is that And I will talk about comparison. So it’s a really great Point Christine mentions around don’t always collect the rosy stuff. We also need to know what didn’t work and how we’ve learned from that funders love those learnings. But the thing is that you don’t always have to ask 10 people, you know, a whole bunch of questions. You can ask 30 people a series of different questions. So that’s another way of thinking about how do we make this more achievable from a resourcing perspective and that bottom point on the list is also Baseline data Baseline data’s, you know, it’s it’s often a very important measure of direct attribution and I’ll give you an example in a workshop sense or something that you probably seen it’s a very simple example, um, you know, we get into your pre-workshop survey and a post Workshop survey. So how much do you believe you already know and and now you’ve done this what what you know, how much has that by? How much is that increased your knowledge? That’s a very fundamental kind of example, but it it does help to understand that Baseline. And again, I’m not just data but the stories I’m not going to spend a lot of time on there is our Roi principles because I want to get to that. Um, Some measurements and we’re gonna I’m going to do some work, but I wanted to include this for you for further further research because for those who are wanting to use or wanting to look into sroi and Christine mentioned, I know you’re going to go and have a look at that the logic model and where I what I’ve stepped you through so far that’s actually the Baseline of understanding and able being able to do sroi methodology. So I like to learn has a social return investment methodology. We actually basically, you know have a formulated spreadsheet where we we collect data and and we put that information and it spits out what our social return investment indicators are but that took quite a lot to do to get to get to that so it is if you’re wanting to go down that path of social return investment methodology and calculations, you’re already on the way if you’ve done I started to do your logic model so you’ll be able to to take that further. Um, so just some other things to consider as we go through and yeah, some of these have already been mentioned like, what did you learn internally from the process? You know, how can we do things better? What he just stakeholders learn on as well. So stay cold your partners that were involved an important one is the third point, you know, we we’re often very positive about these and we think guys is going to be really great, but there are always unintended. consequences that result from the work we do I’ll give you an example of one delivering a diploma of governance. And and we had had somebody that was was you know will informed from a theoretical basis. You didn’t have experience as a CEO had managed a small team and and you know, her knowledge vastly improved over the course of the program and her confidence had built to the level that she now wanted to apply for CEO roles. And what was preventing her from doing that was you know, she went through the application process but was preventing her from doing that was the lack of the experience. And so that’s kind of an unintended consequence because now somebody is feeling less valued. As a result, so that’s an example but there are always it’s always really important to think about those and when we go into measurement, we’re not saying we want to measure the benefits or the improvements what we’re saying is we want to measure what changed. And I think that’s a key thing because if we go inside we’re looking for improvements. Well, that’s always in a report on and we’re going to be focused on those. But again as Christine has shared it’s the you know, if it’s all Rosy then then we doubt don’t we and so let’s do this together. I’m just going to One Stop sharing for a moment and go into the PowerPoint and we’re actually going to have a look at one of two examples of these. Let me just not what I want to do. This will be the bit that they edit out. All right, beautiful. So I’ve got a up here. What I have is I have a little bit of you know, an outcomes map and evaluation map if you like and I’ll just zoom in a little bit. so Let’s say in Grant Training. It was the indicator was confident. So thinking about the positives but also, you know, perhaps the unintended what are some of the changes do you think that we would be looking for as a measurement something to measure what are some of the changes? What more people applying for Grants? Yeah. finally you unmuted thing. Well, this is interesting because I’m just reporting on one where we’ve used this as a measure of confidence like asking people for their own assessment. Their own assessment of the Korean post. Yeah. Yeah. Yeah. right other thoughts continuing on from my first point actually successfully obtaining the grant. Yeah. You could also say you know, more people applying for Grants. They’re more grants applied. Perfect, you know that sort of another way of looking at that and I’m not saying that these arrival wrong or things that we should measure well we’re doing at the moment is just a brain done. And then we’ll come back to we’ll how do we feel about those? for me I saw you spent a lot of time in the grant space. I think people’s ability to read and understand Grant criteria is actually a very is a real biggie because if people are confident in reading that they can actually self edit in terms of is it worth going for this grant is it not not worth going for the grant and people say themselves a lot of time and energy and I basically been able to say well we were a good fit for this grant. We’re not for that ground and I think that’s actually part of the confidence that people get when they actually understand what they’re doing. Yeah. Yeah, that’s Christian. That’s like the quality of the grant applications improves. Yes as well. Yeah, well because of that so if people are assessing the grand criteria, they understand what they’re what the grant bodies are looking for. They will actually end up in improving providing a better grant for those that they do go for. Yeah. All right, so that’s actually a great list and I’m just gonna number these so that we can use them as numbers when we’re referred to them. And you just get my number in here we go. So let’s now think about a direct path if we look at all of these changes that we we are looking for all of these changes. Remember what we’re measuring is an indicator is confidence they all confidence. Direct confidence related indicators or do you think that some of these may fit in another indicator? Because are we selling ourselves short by simply saying we’ve incredible, you know, we’ve increased confidence, you know, that’s the change, you know that we’re reporting on for example. Kylie mmm, it’s some of increasing Grant success might be a an indicator of competence is the measure we could you know number of Grants applied for. Yeah, so if we put increasing grants success and quality of Grants responses. As competence or knowledge. Does that fit better as a measure do you think? I think so. And possibly the ability to read and understand could be a complicated moment. I actually think it’s both. Yeah, right. So let’s pop that down there as well. Oops, no one. So I just wanted to do that as it is. It’s just very simple example. I know that they’re often easier to do particularly on short sessions like we’re on today. To kind of to share with you that like this is the thinking that we need to do because we’ve mapped the outcomes, you know logic model and and we’re being yeah, I a confidence and we’ve put down those indicators, but to be honest what we often haven’t done is we have an interrogated that to say actually is that a direct attributal attributable link and sometimes we can be selling ourselves short if we were just to Simply say to the Thunder. Okay, we’re going and I just use grants because for most in that in our sector that sort of what they look at in terms of getting funding for programs. We’re going to measure the changes in confidence. Where they might be thinking. Oh, really and and actually this comes back to some of what you were saying earlier, isn’t it? Christine where funders want to see some indicator of impact? Yeah. Yeah. Confidence really, isn’t there is it? No, I mean it’s needed to get to be impact but generally fun design that interested in if you feel confident in doing it or not. I just want to actually see the outcomes. Yeah. Yeah, exactly. All right, beautiful. So I just wanted to to show you that to show you sort of some of the interrogation that we do need to do around some of the indicators that we’ve thought of now you can take this further and we won’t map this because I’ve got some great sheets tools. Let’s just say that I’m gonna share with you. I have a crowded them, but they’re there, you know resources that are freely available to us and I want to show you this before we finish up but then we start saying okay if that’s what we want to measure what what are the Questions that we need to ask and then you can map that to what’s the you know, who are we asking these questions of and then it’s what is the best way to ask these questions and that can help you understand if actually this is a story so an interview is best for that kind of depth or if you can measure admire a statistic. And so that can help you. Think about about that approach. All right, so I’m going to just go back to the slideshow deck now. I just wanted to show you as an example an apologies. That’s a little bit pixelated. It’s it’s One A Day image out of the New South Wales government program evaluation framework. So if we’re any of you are seeking funding from a state government, so wi of course have and I know I’m preaching the converted for Christina Kim have the outcomes framework, you know, pretty much every state now enter it and territory and federally we have an outcomes framework. And so if you’re looking for government funding this would be a great one for you to look at and and to investigate particularly if you’re pitching something because it tells you what they deem to be appropriate and and this is an example of of what they’re looking at when they’re looking at sort of stronger evidence of programs success versus Evidence and just remember this is generally about larger funded programs not you know, perhaps smaller workshops or series of workshops or a 12-month program, you know, typically these sorts of things are multi-year industry programs. So, um, do we want qualitative or quantitative here is a list of examples of things to think about focus group research. So bringing them together, you know, letting them have a conversation and having observe as listening on those conversation and record some of the key messages such an interesting one. I don’t know if anyone’s done that one, but you get a lot of of data and a lot of insights from that that comparison group that’s that comes back to what was mentioned earlier around. Yeah, there’s a group that love it and they’ve gotten so much many many benefits, but it is another group that you know hasn’t actually received as many benefits. So so we actually work with both Groups to interrogate I’m going to use that word again analyze. Why why did it work for this group versus why didn’t work for that group, you know, that can be be a useful one as well opinion polls scales, you know Stars happiness a hierarchy of impact. So respondents can say these are all the benefits but what is the benefit or the impact or outcome that I’m most value? That’s an interesting one to do as well interviews and case studies video stories or video diaries. If it’s a program capturing capturing some of those conversations along the way, you know, your question is surveys and that observation piece I mentioned so there’s some some techniques that you can use as obviously a whole range of these when you start getting into you know, how do we actually measure it but but it’s a little bit of a snapshot for you. Here’s something really great and I’m actually going to to bring this website up and you are going to love this. I always love things that are online that help us do our job so much easier and I will pop these links in chat shortly. And for those watching the recording it is down below. This is called the social impact in Impact. Oh my God, social impact toolbox website toolbox dot social impact, toolbox.com. This is what you can use to actually document your outcomes model. So here’s your theory of change step one step two. You can actually start completing your logic model and you can print this and save this download it. It’s all you know all there. You don’t have to design it it’s done for you. And the great thing here is that you’ve got example indicators and output so you can see they so again that that really helps to to you know for you to do that work. And and of course, we have an evaluation framework as well. Again, they’re example indicators. So if we look it outcomes, you know, here are some example indicators you can search those but you can see the very big long list there. And you’ve got again those other indicators and we have some tools now the tools what we’re talking about here. Of course, if you look at Australian Bureau statistics, it’s you know, those research pieces cultural tolerance scale economic scales. So for those of you know, maybe you just want to do the logic model right? There you go. You’ll be able to put that in and that’s all designed and and you’ll be able to download it. The other thing that I wanted to show you from an indicating on slow down Natalie. Yep. Sorry, um, so just that website, is that a free to use tool it? Is it is yeah. Sorry. Are you may have said that knife missed it? That’s all right, and I will put the runner chat just before we um, we wind up. So I’ll actually I’ll email it out to you all because I know Kim has to head off shortly. So, but it the website is I’ll just put that in chat now while we’re talking so you’ve got no taking it down. Thanks. That’s awesome. Thanks Natalie because I was actually just about to get one of my graphic design girls to do up this with little fun little funky boxes and little funky. I tell our logos and icons to develop a one or two pages. So that’s beautiful that those templates already there. Thank you. Oh, and so this was developed by. the University of Technology in Sydney and spacia because there is also and and I will just flick back to the presentation. You’ll also find some templates as well Kim. So you’ll be able to just use those templates. Let me just jump to that and I’ll show you this page. So here’s an evaluation framework worksheet that you can use as well. It’s not the only worksheet. There is a logic model worksheet as well. The other thing that you will find on the website, which is really useful tools and domains and I’m just gonna take you to the website to show you this because this is all you know, it is deep thinking work this work that we’re doing. So being able to have these sort of things that are almost done for you in a sense that that even just help for thinking really that’s what we want is the thinking behind it. Now. Where is it Natalie? Nice, let me head back to this or I thought I had that site up. Silly mate, that’ll take me. That’s not what I want. I want that. All right, so beautiful. So if you go to the tool library in that website and here are domains. So again, this is about a valuation. So if we went to health and wellbeing Um, what you will have is you will have a whole list of the different tools that you might want to research that actually help you, you know, these are outcomes sort of linked to improvements now. I just want to pause there as a note. Of course, these are all useful and you could spend you know hours and hours doing this or weeks and weeks actually this is so much. Always come back to why am I measuring this? Is this a directly attributable in indicator for us? Because sometimes we can spend you know, spend time getting in the deck and then realize that I actually we can’t really do anything with that information anyway, and the other thing is that think about if particularly if it’s funding think about the measures that you’re Thunder uses. So, um, if we look at a couple I’ll just use a couple of funders as an example. If we look at the Western Australian government. Well, they use an outcomes framework if we look at link West which is in Western Australia. They’ve now got there are our own outcomes framework. And more mature funders that have been working with outcomes framework for a while. They’ve already identified key indicators of change that they’re looking at. So use their indicators. I’ll give you an example of the New South Wales one in a little while. I know we are at one o’clock and we are an hour on so if anyone needs to leave apologies, but you will get the recording. I promise we’ll send out a link to that directly to everybody after it’s editor this afternoon. Um and the New South Wales outcomes framework. They have a playthrough of documentation that actually gives you insights into the indicators that they believe are the indicators most valued by them. So and I’ll show you that in a moment, but all of this, you know, I wanted to show you because it’s a value now. He’s another free tool. This is only just being released. Um, this is our indicator engines or sorry now. I don’t mean our I mean, Australia. Australia’s indicator and engine which Center for social impact over in Western Australia has contributed to and it’s called amplify social impact. It is currently free. So you’re able to go on to this and register on this platform and it’ll help you also Identify some indicators. And another one that is a value as well for community services and I’ll just pop this link in chat as well again for those watching the recording it is down underneath. And this is a community services outcomes tree and I’m just gonna show you how it works a little bit like if those are familiar with have Triple R foundations for regional and Rural renewal. Are there another example of a thunder that has their outcomes tree? Let’s have a look at learning skills and development. And so if we click on that outcome, we will see he’s a range of different things that we can look at. We’re going to look at again. You would go back to your program logic and you would say well what are some of those indicators for us and let’s just say it’s personal development. We then have some examples of some of the questions that we might ask now that website also has a survey template with a whole bunch of survey questions, but more importantly it helps you identify and let me just Was I showing that? There we go. We did you see this earlier? No, ah, here we go. I’ve been talking about it not showing the screen. So this is the apologies. This is the outcomes tree. learning skills and development If we click on personal development and living schools, and there’s a list of survey questions there as well. But what that does do. And I’ve just downloaded it just quickly to show you is you now have you can also on that website download a whole document of questions that they’re using to measure but how you know, where does it actually live? What does it link to and why would you use that? So that will be a value for when you’re actually researching that piece? I’m just going to I just wanted to quickly show you those but again as I said, I’m going to to send this to you. And these links to you. I’ll just copy all of the the links that are for resources that I’m using or referring to any session. I’ll just put those in chat for you now, but again, I will send those out to you. Oops, why didn’t that copy everything hang on? copy all there we go. So I know that’s a little bit of um information on those. But again, I just wanted to make sure that you’re aware of these free tools because we don’t want to reinvent the wheel and sometimes it’s easier to identify something, you know, if it fits or if it doesn’t fit when you see it better evaluation, I’ve give you a link to that side as well again, there’s some really important information for those of you wanting to to drill down into it further. You know, my suggestion is again, there’s so much sort of it’s available is to think about what fits best and don’t get you know, too overwhelmed with all of the different options. That’s a community services out comes tree that I mentioned. And so, you know, if you’re working youth or Health Services, you might just say, you know, I’m just gonna stick to this if a program you’re wanting to get funded by a particular funder like The Western Australian government, for example, you would just say, okay. Well, let’s focus on the wa outcomes framework and the measurements that they believe are important. And this is the New South Wales Human Services outcomes framework and I’ll just just quickly show you that that this is an example for social and community. So That is a state have said we have an outcome in Social and Community. This is only one example, and one of those is that people are participating in the Arts and that there is an increased attendance as you can see. This is an older one because it refers to 2019. But the indicator of that is that there is an increase intended attendance so mothers if we look at health and this is Parental Health around birth and pregnancy what they talk about here is what is the impact pathway link? So The link is when mum is a healthy before becoming pregnant. They’re more likely to be able to you know, have a healthy pregnancy, you know carry it. Carry it through the pregnancy through the birth and you know, the evidence that they’ve used is that there was a preconception that mental health was most significant risk factor. They’ve given that a two star and so from their perspective, you know, yeah, there is evidence that Mental Health. Sorry, there is evidence that people. Poorly have a poor understanding that that positive mental health is really important for healthy pregnancy. And so you can use this if this was your program you could actually use this direct indicator, you know, they have given it a two-star rating. So you might be looking at evidence that is actually more more supports the outcomes that they want to achieve. Does that make sense. I know this is a little complex for some more, you know programs that are very straightforward, but I wanted to share this because I also know that some of you actually work in the Human Services And this is sort of the level and the complexity that you may need to do to pitch particularly. We use New South Wales government here. But particularly to pitch if you wanted New South Wales government to fund it or w-a in Christine and Kim’s case. Does that these resources are fabulous? Thank you. Oh fantastic. Thanks Christine. That’s great to hear and I mean the good thing is as well. If if you’re if you’re looking at and I’ll just use an example of the difference between New South Wales and W why and even in Victoria? So w y release this from memory a couple of years ago, it might be three years ago New South Wales and Victoria have had them in a little longer. So if you’re looking at a specific area, and it might be used in wi and I don’t I haven’t actually looked that deeply into what their outcome measures or their indicators of evidence are but if you’re thinking are That’s useful but perhaps not as useful then go on a date from New South Wales or Victoria and you may find some more supporting evidence there as well. And and you know, you’ll be able to sort of map those. But again, that’s why I wanted to show you a couple of examples because for those sort of operating in that. Spice that’s more helpful. And they’re kind of the last couple of points. Share with you which is sort of. Already been mentioned stats and stories. You know, how do we report on this? This is an example of a website and I pop those in chat. I’ve only taken a screenshot of one section of the website where they’ve got a statistic. But again we come back to this storytelling. And those those stories if you’ve got a funder, let’s just say it’s a corporate funder those stories. If you’ve been given permission to share them then that’s that’s another value add that you’re actually adding to your corporate or to your thunder by giving them permission to shout about how wonderful they are because they supported this project. That helped this person. So, you know, that’s again another another reason why you would you know, gather perhaps of video story for those that are quite happy to do that. And the last resource I will finish on is and another great one from the center for social impact in Western Australia. There’s a great handle book. It’s called the compass your guide to social impact measurement. It helps sort of break it down and and gives you the most study it by working through the workbook. Um, and the other one that we you will find on that website the social outcomes website the one where you’re able to map your logic model. There’s a couple of free walkthroughs that University of Technology have also done which you’ll find on that website as well. So that’s our session today. I know that it’s a very quick hour isn’t it? But you know, I wanted to make sure that you know, I’ve given you we had a conversation around measurements and and some of those really key important points and thank you everyone for contributing and sharing, you know from your perspective and experience as well that really helped share that story with others.