Happy Labor Day, everyone! Follow the most excellent advice found in this NY Times Op-Ed piece by Clive Thompson and take some time for yourself!
Happy Labor Day, everyone! Follow the most excellent advice found in this NY Times Op-Ed piece by Clive Thompson and take some time for yourself!
Reproducibility is the ability of an entire experiment or study to be reproduced, either by the researcher or by someone else working independently. It is one of the main principles of the scientific method and relies on ceteris paribus. Wikipedia
I was going to start this post with a similar statement in my own words, but couldn’t resist the chance to quote Latin. It always makes you sound so smart. But regardless of whether these are a Wikipedia author’s words or my own, the point is the same – one of the foundations of good science is the ability to reproduce the results.
My work for the neuroimaging project involves developing a process for researchers in this field to cite their data in such a way that makes their work more easily reproducible. The current practice of citing data sets alone doesn’t always make reproducibility possible. A researcher might take different images from a number of different data sets to create an entirely new data set, in which case citing the previous sets in whole doesn’t tell exactly which images are being used. Thus, this gap can make the final research harder to replicate, as well as more difficult to review. We think that we may have a way to help fix this problem and that’s what I’ve been working on for the past few months.
At the same time, I’ve been working on a systematic review with the members of the mammography study team. This work has me locating and reading and discussing a whole slew of articles about the use of telephone call reminders to increase the rate of women receiving a mammogram within current clinical guidelines. It also has me wondering about the nature of clinical research and the concept of reproducible science, for in all of my work, I’ve yet to come across any two studies that are exactly alike. In other words, it doesn’t seem to be common practice for anyone to repeat anyone else’s study. And I can’t help but wonder why this is so.
I imagine it has something to do with funding. Will a funding agency award money to a proposal that seeks to repeat something; something unoriginal? Surely they are more apt to look to fund new ideas.
Maybe it has to do with scientific publishing. Like funding agencies, publishers probably much prefer to publish new ideas and new findings. Who wants to read an article that says the same thing as one they read last year?
Of course, it may also be that researchers look to improve on previous studies, rather than simply repeat them. This is what I see in all of the papers I’ve found for this particular systematic review. The methods are tweaked from study to study; the populations differ just a bit, the length of time varies, etc. It makes sense. The goal of this body of research is to determine what intervention works the best and in changing things slightly, you might just find the answer. What has me baffled about this process, though, is that as we continue to tweak this aspect or that aspect of a study’s methodology, when and/or how do we ever discover what aspect actually works and then put it into practice?
Working on this particular review, I’ve collected easily 50+ relevant articles, yet as we pull them together – consolidate them to discover any conclusions – the task seems, at times, impossible. Too often, despite the relevancy of the articles to the question asked, what you really end up comparing is apples to oranges. How does this get to the heart of scientific discovery? How does it influence or generate “best practice”? I can’t help but wonder.
Yesterday, during my library’s monthly journal club, we discussed an article that had been recommended reading to me by one of the principal investigators on the mammography study. How to Read a Systematic Review and Meta-analysis and Apply the Results to Patient Care, is the latest User’s Guide on the subject from the Journal of the American Medical Association (JAMA). It prompted a lively session about everything from how research is done, to how medical students are taught to read the literature, to how the media portrays medical news. I recommend it.
Of course, there are many explanations to my question and many factors at play. My wondering and our journal club discussion doesn’t afford any concrete solution and/or answer, still I feel it’s a worthwhile topic for medical librarians to think about. If you have any thoughts, please keep the discussion going in the comments section below.
[This is not an ordinary post for this blog, but this is not an ordinary day.]
July 21, 1951 – August 11, 2014
His death is a shock. The tributes from the many people who knew and/or loved him continue to spread across every form of media. People are sad. I am sad. Robin Williams’ death is sad. But it is the thoughts and images of his suffering, the unfathomable depth of pain, and the cruelty of depression, that leaves me heartbroken. Many people more qualified than I can and will speak to the disease that took his life. Many people will share of the importance of reaching out and finding help when depression hits and/or life becomes too much to bear. Asking for help is hard, though, and harder still when one is dealing with a disease that takes rational thought away.
Maybe one small thing that can ease that barrier is knowing someone who knows what you’re going through. If you sense that there’s at least one other soul who experiences what you experience, maybe that person is a lifeline to another day. Maybe. And maybe this is why I’ve noticed a number of people sharing their experiences over the past 24 hours. Rob Delaney, Kathleen Edwards, Harvey Fierstein… these are just a few celebrities that I follow who I’ve seen post thoughts; telling readers or fans, in a way, that they are not alone.
I’m not a celebrity by any stretch of my imagination, but there is something about Williams’ death and the public reaction that makes me want to share a story, too. My story. For me, clinical depression is a palpable black cloud that hovers over my shoulder. It is a dark basement. A place of unhealthy solitude; of isolation. It is both terrifying and seductive and that, for me, is the crux of the cruelty that this disease can be. Sometimes, it can be exhausting to ward it off. Scarier still, is how it first appeared (from Ordinary Year, Chapter 1):
So in the summer of 2009, I was in a healthy, well-established, loving relationship. I lived in a nice apartment, had nice neighbors, a dog and a cat, and interests both in and outside of work. I’d been sober for years. I dabbled in writing. I learned to play the mandolin. I reconciled with my brother, a relationship that had been damaged for a decade. I finally got to know my nieces and nephew, and the chance to start being an aunt to them. They wanted both Lynn and me to be in their lives. Things with my dad were so-so. We didn’t see each other very often, but I don’t see any of my family very often. Still, we were all okay. Everything was okay.
But every day, from late May through September, I cried. At some point, every day, I found it impossible to stop the tears from falling. I’d turn to the wall in my cubicle to hide my face, grabbing tissues and hoping no one noticed. I’d be in my car, driving to or from work, and start sobbing. I started to wonder if I was going a bit mad.
I knew that I was lonely for some of my old friends and so I decided to make a trip to Louisville to reconnect with some of them. I ordered my plane tickets and started to think about sitting at a patio table outside of a coffee shop, talking for hours with one of my closest confidantes, Dina. I thought about catching up with my good friend Kevin. I thought of taking Dina’s kids trick or treating. I looked forward to some time on my own, some time for myself. It would be a good, well-needed and well-deserved long weekend.
A few weeks before my planned trip, I was walking from the parking lot to the library, to work, when the thought crossed my mind that it might not be a bad thing if the plane I took from Providence to Louisville crashed. I had no plans to make it crash and I didn’t exactly wish for it to crash, but the thought of it crashing gave me such a feeling of relief. It would finally be over.
It scared the hell out of me.
Thanks to medication and a few years of therapy, I’ve learned how to live with the darkness. I’ve learned how to recognize it. I’ve learned how to ask for help before that voice gets silenced. It’s heartbreaking when that voice dies. And it’s heartbreaking, the lives it takes with it.
To echo the message of so many others in health care, if you feel you’re struggling with depression, seek medical care. If you’re experiencing thoughts of suicide, there are people who can help. National Suicide Prevention Lifeline, 1-800-273-TALK (8255).
I like to make the cards that I give to people. Yes, I too often give in and buy the prefabricated ones, but even then, I try very hard to pick ones out that are blank inside, not substituting anyone else’s words for my own. I like the handmade touch. I have a small box with several cards that I made for my mom when I was a child. They are special. My mom treasured them enough to keep for herself and now, I keep them myself. Crayon-scribbled, “You are the best mom” accompanied by a cut-out, construction paper flower is worth saving.
Besides the sentimentality of handmade items, they also share the message that the sender took a bit more time to make something just for you. I’m not knocking the time one can spend searching the shelves at the Hallmark store for just the right message, but you must admit that taking the time to make that right message says just a little something more.
I thought about making cards earlier this week when I followed along with a listserv discussion about the practice of sending weekly articles, messages, and updates to patrons. A number of participants shared some very helpful resources – aggregators, if you will – for delivering timely pieces. It’s both easy and resourceful to subscribe to them. They scour the internet for stories about the latest medical procedure, disease outbreak, trend in healthcare, etc., and send them right to your email inbox for quick reading. Some even annotate them for you, so that you don’t have to be bogged down reading more than seven paragraphs. The suggestion offered in the discussion was to share these feeds with administrators or doctors or researchers or whoever your target audience is. It’s a great idea, but as I thought about it, the practice reminded me of buying a greeting card instead of making one yourself.
Libraries and librarians have given up a great deal of their identity (their brand) over the past years. The full-text of articles are often accessed through third-party vendors or the websites of journals, despite the fact that it’s one’s library that’s often providing the resource. We buy catalogs developed by other companies, rather than developing homegrown management systems. We embed RSS feeds from other sources into our own websites.
And each and every one of these practices saves both time and money, but at what cost?
I got to wondering how much time it would really take to subscribe to a relevant aggegator or journal table of contents, or to set up a few alerts from custom-saved searches, or to put together several Twitter lists that follow sources specific to a group or department I serve. Then I could use these tools to create my own, customized delivery of an article or an interesting piece of news to the same. Think of the return on the investment I’d get by sending a personal note directly to someone with the resource attached, as compared to the same coming from an automated – and branded by someone else – source. Now, I can already hear some naysayers saying, “I don’t have time to keep up with that.” Maybe not, but I think it might be worth a try.
Related, another thing that I often hear people say is that we don’t have time to read ____ (insert whatever it is that you don’t have time to read – blog posts, journal articles, interesting pieces from the news). Similarly, many say that we don’t have time to write _____ (insert whatever it is that you don’t have time to write – blog posts, journal articles, etc.). This a dilemma. To paraphrase Stephen King (the writer), if you want to be in the information business, you need to do two things above all others; read a lot of information and write a lot of information. How else can you stay on top of it? How else can you provide good information resources to those you serve? How do we call ourselves information professionals if we ignore the very thing that we’re supposedly experts in? We work in a fast-paced and rapidly changing profession. All the more reason to do those two things above all others. Read and write.
I write a post for this blog each week. Thanks to the kind words of many colleagues, not to mention usage statistics, I know that people read it. But I also read the writings of colleagues and other people who provide so much insight, interest, and entertainment to my work, that I can’t imagine how lousy I’d be at my job without them. With this stated, I’m sharing several really good things that crossed my radar over the past week. If you can find a moment or two to read them, you may find it worth your while:
Finally, I always read Amy Dickinson’s advice column. I need all of the everyday, practical advice that I can get. And my friend, Suzy Becker, wrote a most wonderful blog post to go along with the release of her latest book from Random House Kids this week. Author-Daughter Book Club just about made me cry in my cubicle. In a good way. Moms of sons and daughters, both, will enjoy it. I give shout outs to these two writers who, many days, make my day.
My graduate thesis in exercise physiology involved answering a research question that required collecting an awful lot of data before I had enough for analysis. I was comparing muscle fatigue in males and females, and in order to do this I had to find enough male-female pairs that matched for muscle volume. I took skin fold measurements and calculated the muscle volume of about 150 thighs belonging to men and women on the crew teams of Ithaca College. Out of all of that, I found 8 pairs that matched. It was hardly enough for grand findings, but it was enough to do the analysis, write my thesis, successfully defend it, and earn my degree. After all, that’s what research at this level is all about, i.e. learning how to put together a study and carry it all the way through to completion.
During my defense, one of my advisers asked, “With all of that data, you could have answered ___, too. Why didn’t you?” I hemmed and hawed for a bit, before finally answering, “Because that’s not what I said that I was going to do,” an answer that my statistics professor, also in attendance, said was the right answer. Was my adviser trying to trick me? I’m not sure, but it’s an experience that I remember often today when I read and talk and work in a field obsessed with the “data deluge.”
The temptation to do more than what you set out to do is ever present, maybe even more today than ever before. We have years worth of data – a lot of data – for the mammography study. When the grant proposal was written and funded, it laid out specifics regarding what analysis would be done; what questions would be answered. Five years down the road, it’s easy to see lots of other questions that can be answered with the same data. A common statement made in the team meetings is, “I think people want to know Y” or “Z is really important to find out.” The problem, however, is that we set out to answer X. While Y and Z may well be valuable, X is what the study was designed to answer.
I see a couple of issues with this scenario. First, grant money is a finite resource. In a time when practically all research operates under this funding model, people have a certain amount of time dedicated, i.e. paid for, by a grant. If that time gets used up answering peripheral questions or going down interesting, but unplanned, rabbit holes, the chances of completing the initial work on time is jeopardized. As one who has seen my original funded aims change over time, this can be frustrating. And don’t hear me saying that it’s all frustrating. On the contrary, along with the frustration can come some pretty cool work. The mini-symposium on data management that I described in earlier posts was a HUGE success for my work, but it’s not what we originally set out to do. The ends justified the means, in that case, but this isn’t always what happens.
The second issue I see is one that I hear many researchers express when the topics of data sharing and data reuse are raised, i.e. data is collected a certain way to answer a certain question. Likewise, it’s managed under the same auspices. Being concerned about what another researcher will do with data that was collected for another reason is legitimate. It’s not a concern that can’t be addressed, but it’s certainly worth noting. When I was finished with my thesis data, a couple of faculty members offered to take it and do some further research with it. There were some different questions that could be answered using the larger data set, but not without taking into account the original research question and the methods I used to collect all of it. Anonymous data sharing and reuse, without such context, doesn’t always afford such, at least not in the current climate where data citation and identification is still evolving. (All the more reason to keep working in this area.)
We have so many tools today that allow faster and more efficient data collection. We have grant projects that go on for years, making it difficult to say “no” to ask new questions of the same project that come up along the way. We are inundated with data and information and resources that make it virtually impossible to focus on any one thing for any length of time.
The possibilities of science in a data-driven environment seem limitless. It’s easy to forget that some limits do, in fact, exist.
What an exciting week it’s been! You know those days or moments when you see a lot of groundwork (hard work) start to pay off; like when you see the first tomato appear on the vine or the first sprig of a pepper plant pop up through the dirt? Well, we had one of those this week. For the past several years, we’ve been talking about and planning and laying the foundation to provide library services around the needs that our patrons have when it comes to working with data. Years, I tell you.
When my colleague, Rebecca, arrived last August to take the reins in this effort, I’d been out pounding the pavement for a good while, building relationships and doing individual data-related projects, and perhaps most importantly, getting a sense of who did what and when and where and how. Rebecca got to work strategizing, writing plans, working with our library’s administration and other higher-ups in the university, while Lisa and I provided experience and the connections needed to pull it all together. We developed a Library Data Services Advisory Group, bringing a few vested parties to the table. We did an extensive environmental scan to find out what the different stakeholders on campus thought the Library’s role might be in this area. We talked to lots of people. We surveyed students. We gained a lot of insight.
Meanwhile, I continued to do my work with the mammography study team, part of which involved helping put together a mini-symposium around data issues in clinical research. We brought together clinicians, members of our Quantitative Health Sciences (QHS) Department, and members of University’s Information Technology Department. We also surveyed colleagues to gauge their interest and needs in this area.
Sitting in these different groups, working on these different teams, I started to see pretty clearly that multiple things were happening on campus; that there was at last some real thought and energy being put towards addressing some of the needs we have around data. I also started to see that a lot of right hands weren’t aware of what their left hands were doing. And the most exciting part of that (when I got past being frustrated) was this… I knew what both hands were doing!
A few weeks back, I wrote about that frustrating part, as well as how I see how exciting it can be when we (librarians and thus, the library) are positioned in a way to make things happen. And this past Monday, was one of those exciting moments. We ALL came together; representatives of each of these groups that I’ve been witnessing talk about what to do to address the data needs at UMMS. The librarians, the clinical researchers, the computing services folks, the QHS people… we were all at the same table where we could share with one another what we do, what we know, and how we can help. And we came away with some very real, tangible projects that we can tackle together. It really was one of those times when I felt a sense of accomplishment in this task that’s been nebulous, to say the least.
And… I was also hired by the University of Rhode Island’s Library & Information Studies program to teach the course on Health Sciences Librarianship this fall. (I’m really excited about it!!) Totally unrelated to the previous tale, but the two events made for a pretty great week. I hope you’ve had the same!