A colleague in a peer networking group recently asked a question stemming from wanting to be an effective therapist and feeling unsettled that she’d gotten negative feedback from clients via an online platform. We all want to help our clients and part of that is, of course, ensuring that our clients know that we hear and see them and that we understand their experiences and perspectives. First, let’s tackle her question, and then, let’s dig deeper into the problematic way that online commodified therapy platforms assess their efficacy. Finally, let’s name some ways we can assess our own.
The Puzzling Question…
“Seeking a bit of validation and ideas to work through an imposter moment. I’m transitioning to private practice and I’m seeing a small caseload of clients on one of the online platforms to help with building my caseload. I have been connected with mostly very ideal clients who I have a lot of experience working with.
While my overall client ratings are really high, I noticed a decline in the ratings for ‘my therapist understands what my primary concerns are.’ Since seeing that, I have noticed being hyper-focused on this in sessions and I’m noticing that I am struggling to connect with clients who I previously connected well with. Also, I notice that this specific rating is continuing to drop.
I’m just looking for some support to help me stop this thought cycle and just get back to being more of my genuine self. My specialty is DBT so I’m working through my own behavior chain and turning to skills. Any extra suggestions are also appreciated. “
My Super Concise Answer…
If you’re working with a population that has a history of chronic invalidation experiences (just my guess from your specialty), I would say please do not sweat this. Oftentimes, these Likert-style measures are not reliable because feelings, attachment, and thoughts tend to wax and wane throughout treatment, just like they do in any relationship. I often find that when these types of clients finally feel safe enough to share that they don’t feel understood, we work through it, and then we have a stronger relationship – which may be the first experience of secure attachment for many.
Also, going back to your roots with DBT, instead of chaining the feedback, try chaining your thoughts of the feedback being valid or having the use that you first assumed it had. I find that if I’m doing chain work, I sometimes need to use my “WHAT Skills” and change “this happened” to “I noticed myself thinking this happened” or “I noticed myself assuming this meaning of this thing that happened.”
So, roll with it and don’t sweat metrics that likely weren’t even created by a therapist. 😉
But Let’s Dig in Further!
While I don’t know this colleague super well, I can tell she cares deeply about her work and is asking the right questions. And we all need to be monitoring our own effectiveness in treatment with clients, but I have a HUGE problem with the type of assessment that her online platform was using. As mentioned above, if clinical concerns include chronic invalidation, communication struggles, a desire to be seen and heard, and the pursuit of connection and understanding, of course, perceptions in this area will go up and down! Identifying sticking points and then navigating them together is a big healing process in and of itself. As clients find their voice and learn to hear themselves and trust themselves, this stuff gets dialed in. It’s like when you zoom in on a map and then need to pan over to the area you’re truly trying to see within the zoom view.
But okay, so let’s get to the bigger issue here. Online platforms are using methods of measurement that are inaccurate and ineffective in numerous ways. Research methods refresher: for a statistical measure to be worth its salt, it needs to be accurate, which means it actually measures what it sets out to measure. If you’re trying to assess something immutable, like getting an ACEs score, the person will have close to the same result no matter how many times or in what settings they take the measure. If, however, you’re trying to assess what kind of changes occur over time, like when looking at someone’s wellness with a PHQ or something, it needs to be “sensitive to change” so that you can measure the impact of an intervention.
Let’s go back to our example above: asking a client whether they feel that their therapist understands their concerns. It’s clear that this is meant to identify the impact of an intervention, so we want it to be sensitive to change. Well, it is. But is it accurate? Does it measure what we hope it will measure? NO. Treatment goals and needs change over time. A client can come to treatment with one idea or goal and then realize they’re looking for something else over time. Or over the course of the work, the client can make gains in the first goal(s) and is now on the way to naming a new one but isn’t totally sure yet what it is. Or if they’re dealing with a contextual history that makes it hard to feel seen, safe, heard, and considered… this answer can vary with anything that comes up in life or session!!
All of that said, it is really important that we track our clients’ progress through treatment so we can know how we’re helping, if we’re helping, and if we’re on the same page.
Here are seven important strategies and techniques that we can all use, and invite our clients to use, to measure treatment effectiveness and supportiveness over time.
- Refer to the treatment plan! You have treatment plans, right? Some states like Arizona require them, and some don’t, but I strongly believe in their usefulness either way for several reasons. They keep you and your client on the same page as far as understanding why they’re there. They offer a vehicle by which to dialogue and collaborate on goals and how to attain them. They protect you and also hold you accountable to be your best ethical self if anyone ever examines your records.
Track the impact of struggles and use of strengths! Take a play out of the DBT handbook, as mentioned above. I’m not saying to go have your client do full-page Diary Cards unless that’s your jam and it’s clinically indicated. But it could mean sitting down on Sunday nights and noting how much they fought with a partner, drank, or had a sleepless night. It could mean putting stars on their calendar for days they felt effective, proud, or grateful and reviewing at the end of the month. Get creative, or better yet, invite your client to choose a way to stay attuned to their journey.
Track the impact of struggles and use of strengths! Take a play out of the DBT handbook, as mentioned above. I’m not saying to go have your client do full-page - Take good, consistent progress notes and refer back to them! We aren’t just keeping notes to defend ourselves from board complaints. Right? We write them after every session, but when’s the last time you’ve sat down with a client’s file and just read through the last eight or ten progress notes for them and noticed changes in affect, topic, or the narrative itself? Take good notes and avail yourself of their usefulness.
- Use validated measures! I like to use general clinical assessments, like the PHQ-9 and GAD-7, and more specialized ones for concerns like Diabetes Distress or Alcohol Use. They’re legit, and they’re not just there for insurance audits. I like to choose one to have them fill out with their intake paperwork and then offer it again either at a certain number of months each time, as we accomplish goals or big pieces of growth, or at the end of treatment.
- Ask your client! Your client is the best expert on their own lived experiences. Engage in dialogue. Check-in. When you have a solid rapport, this might be an easy and intuitive conversation. If things have been a struggle, keep an open and curious stance and commit to listening and exploring their feedback on your own or in consultation (preserving privacy and/or following the scope within which you can share information, of course). Conversations are my preferred way to unpack this stuff but they’re not the only option. Did you know there are also verified measures to offer at each session that look at a client’s therapy and session experience? I love these for clients who are still learning that it’s safe to be direct with me, and that they’re safe in offering critical feedback. The Oucomes Rating Scale (ORS) and Session Rating Scale (SRS) are a great pair and are totally not awkward to offer once you get into the swing of things. If you’re brave and growth-minded, and you want to be at your best for the clients you serve, check out some Feedback Informed Treatment resources from Dr. Scott Miller.
- Check-in with collaterals! If it fits with your treatment style and plan for this client, you can bring in family, friends, or loved ones to check-in. You might even try an Outsider Witness exercise or other kinds of Narrative processes to really deepen and make space for your client’s growth. Alternatively, you can invite clients who feel positive about doing so to ask those closest to them about any progress they’ve observed, changes they’ve seen, or even about increased distress.
Check-in with collaterals! If it fits with your treatment style and plan for this client, you can bring in family, friends, or loved ones to check-in. You might even try an - Crunch your retention data! Are you a numbers fan? This one’s for you. When you terminate or graduate a client’s journey in therapy, keep it all in a spreadsheet of how long clients stay, how often they came, whether they left with or without meeting goals, etc. Also, chart the start and end dates. You’ll be able to glean information about where your sticky points are in the arc of treatment, and you’ll also be able to see how and where your effectiveness has changed over time.
- Bonus: Assess yourself! How do you think you’re doing, personally? Take your own PHQ, see where you’ve taken family leaves, had illness flares, moved offices, experienced relationship changes, etc. Unpack how the changes impact your work—even better if you do this with a trusted colleague or your own therapist, so you can be kind to yourself around anything you might notice.
And One Final Note:
It’s essential to remember (or learn right now), that it is not ethical to use treatment efficacy feedback as testimonials in our practice marketing. It’s not ethical to solicit our clients for reviews – at all, ever – and even in the rare instance where it might slide in and seem okay, even then, it’s not worth the risk to your license or to the client’s wellbeing. For more about that, check out my piece on client reviews and testimonials (coming soon!), where we take a deep dive into the codes of ethics for Counselors, Social Workers, MFTs, and Psychologists as well as the why behind these strict rules.
