Archive | Evaluation RSS feed for this section

Turning the Tables

27 Mar

Earlier today, a very nice first year medical student came by my office and apologetically asked me if I could tell her where the offices of our Institutional Review Board people are. I don’t work in the library anymore, you might recall, and now my office sits in a maze of other offices on the 7th (top) floor of the ambulatory care center of our medical complex and medical school. It’s not a place that anyone wanders past. Mine is not an office that someone might simply pop in for a visit. It’s out of the way. And as I said to my former colleagues in the Library and my friends on Facebook, I admittedly miss such interruptions. Not all of the time, mind you, but one of the greatest joys that I knew as a librarian was simply answering someone’s question and / or helping them in some way that made their day better. The student was SO grateful when I got up and walked her around the corner to the IRB folks. She thanked me several times. It was a flashback to those days of yore … oh, 3 months ago or so.

For ten years, I spent the better part of my working days answering people’s questions. Either answering them directly, looking for the answer for them, or helping them learn how to find the answer for themselves. All day long, the overwhelming majority of sentences spoken to me ended in question marks. Librarians answer questions.

Evaluators, on the other hand, ask them. This is what I’m quickly learning. We are the ones who need the answers, thus we’re the ones who ask the questions. How well did “X” intervention work? How much time was saved by implementing “Y” into the process? What does “Z” do for you that no other letter of the alphabet ever did for you before?

But I’ve also learned that there’s a speed bump; the researchers and clinicians and other users of the resources and services that I’m evaluating … well … they typically aren’t librarians. In other words, I don’t think that they like answering questions as much as any librarian does.

“It’s a simple survey,” I claim.

“It won’t take more than 5 minutes of your time.”

“The results will help us help YOU!”

“I’d be ever so grateful,” said Babe the Pig. (If you miss the reference, check out the movie.)

And still, getting people to answer questions is way harder than I ever imagined. I’m actually very good at talking to people, and usually pretty good at getting people to talk to me. It was one of the skills and characteristics that I honed as a librarian / informationist that I figured would be easily transferable to my new role. Not so much. At least not yet.

It could be the method – the dreaded survey. People don’t like them. Heck, I don’t like them. But in some cases it is the most appropriate and most efficient method for getting the data (answers) you need for the evaluation. I read and studied and asked about writing good questions. I worked with seasoned researchers to put my survey together. I piloted it with different groups and made all of the necessary tweaks based on the feedback I received. I picked my target audience carefully. And once I felt confident about the whole thing, I let it loose.

And then … I waited.

And waited.

And sent out a couple of reminders.

And broadened my audience.

And worked some different angles to reach people.

And waited some more.

I’m still waiting; waiting for the responses to grow to some level that will afford me some information needed to present my findings to a couple of different groups. It’s coming along, but golly it’s slow. And such a cumbersome process. Ask me a question and I’ll happily answer for you, straightaway. But waiting for others to answer me … well, the tables have turned, my friends.

A turning table playing a different tune!

A turning table playing a different tune!

The Lost Art of Being Frugal

29 Jan

Worcester, Massachusetts got slammed by a blizzard this week, bringing out the hearty nature in all of us New Englanders. What’s a little (34″) snow to dampen our spirits? I made a big pot of chili, watched a couple of movies, read a little, and hung out with my pets while the snow flew. Then yesterday, I joined everyone else in the neighborhood in the first great dig out of the winter. It’s what you do when you live here. No complaining needed. Born out of the spirit of the Puritans that settled here, New Englanders have a reputation for hard work and frugality. Granted, it’s been some time since the days of the Pilgrims, and regional distinctions fade as we’ve become a much more migratory society over the centuries, but we still think of Southern hospitality, Midwestern friendliness, Western pioneers, and hearty New Englanders. And yesterday, we hearty folks were shoveling. 

Joseph-Siffrein Duplessis [Public domain], via Wikimedia Commons

Joseph-Siffrein Duplessis [Public domain], via Wikimedia Commons

While we praise hard work and frugality, these traits also run counter to much that Americans dream to achieve today. Retiring early, becoming a millionaire overnight, achieving fame and fortune by winning a talent contest … these are the ideas behind bestsellers and top rated television programs. We talk the talk of hard work, pulling yourself up by your bootstraps, scraping and scrapping and saving for our dreams; these are the bedrocks upon which America was built and, thus, they remain a part of our societal DNA. As one of our Founding Fathers, Benjamin Franklin, said, “The way to wealth depends on just two words, industry and frugality.” In theory. But in reality, for more reasons than we can count, they are less and less the walk of America. 

Economists speak of “frugality fatigue” as a driving force behind folks living in debt. As a species, we aren’t always very good at delayed gratification. We want what we want now and we’ve built a society that feeds this human habit in so many ways. Thus, when we also hear the popular mantra “do more with less” in our workplaces and business, it’s not something that we necessarily want to hear. It becomes a very negative thing. It wears us out, after awhile. We get stretched too thin. We simply cannot do more and more with less and less. In this sense, frugality becomes our enemy.

But is it? Was Mr. Franklin wrong? Or is there a way to look at “do more with less” that prompts something beyond stress?

According to Navi Radjou, an innovation strategist in Silicon Valley, the answer to that question is yes. In his thought-provoking TED Talk, Creative Problem-Solving in the Face of Extreme Limits, he outlines his theory of frugal innovation. In this brief talk, he gives lots of examples of people living in conditions where resources are often extremely limited, yet rather than limiting their ability to solve problems, the situation actually enhances their creativity and results in solutions that they would likely never come up with in a land of plenty. 

They can magically transform adversity into opportunity, and turn something of less value into something of high value. In other words, they mastered the art of doing more with less, which is the essence of frugal innovation.

~ Navi Radjou

We hardly need to live in abject poverty to take advantage of this idea. Librarianship, and any profession struggling with finding its footing and value in tough times, can tap into the one resource that’s common in most every situation, human ingenuity. Radjou calls it our most abundant resource. We need to find ways and create situations that foster our ingenuity. Maybe, the pressure cooker of a “do more and more with less and less” work setting can be the impetus for this. Maybe not knowing what’s coming next, not knowing where we belong, not knowing how to define and/or redefine ourselves is just the environment we need to push us towards creative solutions.

In many ways, I’m glad that I’ve entered my new role as an evaluator without a lot of traditional knowledge and background in the subject. Yes, I’ve been reading and studying up on the basics, but lacking the resource of years of experience and know-how, I find that I’m able to come up with some different thoughts and ideas and solutions that I probably wouldn’t have come up with otherwise. It’s like the team that enters the big game for the first time. They don’t know enough to know to lose.

One of the great things about evaluating the impact of clinical and translational research is that nobody really knows exactly how to do it yet. This is what I tell myself. It helps me put aside any anxiety of knowing that I don’t necessarily know what I’m doing, and sets me free to try all sorts of things in doing my job. It’s my way of making the most out of my limited resources and thus practicing frugal innovation. And that can be downright exciting. 

 

 

Making Mistakes

15 Jan

The button has been pushed and our proposal for a Clinical Translational Science Award from the National Institutes of Health is out of our hands. Let the review process begin! 

My part of the writing and work has been done for a few days now, so I’ve been spending much of the past week doing many things that I likely would have done when I started this new job, if only I hadn’t jumped into the grant writing fire. I’ve read all of the Center’s newsletters for the past few years, I’ve taken lots of notes and done a bunch of documentation related to who’s who for each of our core components. I’d drawn a lot of pictures and graphs and maps to help me understand the landscape. My informationist role in the Library prepared me well for these latter tasks, as I’d been embedded in projects and made it my job to know what was going on around the University. Still, it’s a lot of putting names to faces to departments to projects. It’s a lot to learn.

I also need to learn a lot about the role of an evaluator. I knew enough to get the job, but now that I have it, I know that I need to learn a great deal more before I’m proficient in the task(s). I’ve been reading a lot of articles from evaluation journals. I tracked down several good books on program evaluation, and a couple on evaluating biomedical and health research, in particular. I’ve subscribed to professional listservs, been reading daily blog posts by evaluators, and joined a professional organization to have access to needed resources. Lastly, I’ve been working through a few self-guided mini-courses to make sure I’ve got the basics down pat and that I understand the terminology that I’m reading elsewhere. So far, so good.

And I’m planning. It’s true that I worked with the section writers and leadership of the CCTS to develop evaluation plans for the different components of our proposal. In other words, I’ve already written a whole bunch of plans stating what I’m going to do over the next few years. That said though, there’s planning and then there’s PLANNING. The all-caps version is where I am now. I have plans to collect and track certain metrics to answer certain evaluation questions, but now I really need to plan out how I’m going to do all of that. The logistics. It’s a great challenge. It’s interesting and I’m learning a lot. I cannot complain.

A torn page from a book. I found this on the sidewalk one day during a walk. It's hung over my desk ever since.

A torn page from a book. I found this on the sidewalk one day during a walk. It’s hung over my desk ever since.

One clear thing that I’ve read – and thus learned – over and over in my study ’til now is that the practices of measuring and evaluating are continuous. You need to plan for them from the beginning and, depending upon your goal, assess at different points along the way. It’s pretty much like life in general. If you make that New Year’s Resolution to lose weight, you need to make a plan and part of that plan involves devising a means to track your progress along the way. If you want to go on a trip to Europe, you need a plan to save the money and a way to keep track of what you’ve saved, so you’ll know when you’re ready to pack your bags. 

I am a reflective person by nature. I majored in philosophy during my first time through college. I went to seminary where you hone your spiritual reflection skills well. I’ve spent time with therapists, here and there in life. I’ve been writing this blog for the past few years as a way to reflect upon and keep track of my changing roles as a librarian. For me, it’s a really helpful practice because it keeps my awareness of where I am and what I’m doing and what I’m learning at the forefront. Annual evaluations (and/or quarterly reports) don’t work for me without keeping track of things along the way. This blog helps with that.

I recently re-read something that I wrote last fall for a different blog, Hack Library School. I was interviewed, along with several other medical librarians, about our work. One of the questions asked was what advice I’d give to current students studying library science and my answer, in part, was:

Sell yourself! One of the things that I see happening in settings like mine (an academic medical school and research center) is that there is never a shortage of work for a person who can match his/her skill set to existing needs. And there are LOTS of existing needs. The key is to really know what you know how to do, know what you need to learn how to do (and learn it – ESPECIALLY if you’re weak in the sciences), and then know how to show people that what you bring is uniquely useful to them. I don’t necessarily think that this means you wait around and look for job openings in medical libraries, but that you also keep your eye on other parts of the health care system or biomedical research where what you can do fits. People looking for help often don’t think of a librarian as one who could do the job for them, but I think that’s mostly because we haven’t done the best job of selling ourselves. Know yourself, have confidence (even if you have to fake it at first), and put yourself in places that offer you opportunity.

One thing I could add to this is that developing a practice of self-reflection, evaluation, and/or tracking yourself – however you do it – will put you in a much better position to sell yourself and/or match your abilities to opportunities as they arrive. This is exactly what happened to me last November and it landed me in a great new role. My CV didn’t say a thing about being an evaluator, but I was able to map pretty much every aspect of it to the qualifications needed for the person in this post. The discipline of weekly reflection via this blog made that task easy.

We always encounter times in our lives where evaluation is forced upon us, whether it’s that mandatory annual review or a major life event. My mother-in-law is in hospice care now and nearing the end of her life. It’s a time of reflection for her and everyone in the family. “Did I live a good life?” is likely the ultimate evaluation question. You hope for the answer, “Yes” and you hope for lots of reasons to be sure of your answer, since it’s basically too late to change much. Driving home last night after visiting with her, I thought a lot about how all of these things fit together. There’s no need to wait until the program is over, until a career is over, or until a life is over to ask, “Did I do a good job?” When we plan to track, measure, reflect, and evaluate along the way, I’m fairly certain we’re better off in the end.