What is Reproducible Research and why it is important.

“A community involvement in actively maintaining reproducibility of previously published results assures that a body of knowledge in a computational field stays alive and can be scientifically extended though continuing contributions.”

–Sergey Fomel

“It is a big chore for one researcher to reproduce the analysis and computational results of another. I discovered that this problem has a simple technological solution: illustrations (figures) in a technical document are made by programs and command scripts that along with required data should be linked to the document itself.”

–Jon Claerbout

I’ve got little research experience particularly with large-scale computationally-based project until I attended the second Madagascar Working Workshop held by “The Rice Inversion Project” this summer in Houston, where I also first knew about an old but also new concept in natural science even recently in social science area– Reproducible Research.

So what is it, how has it been raising researchers’ and professionals’ interest, and why it is of increasing importance?

As the word “Reproducibility” denotes, it is the ability of an entire experiment or study to be reproduced, either by the researcher or by someone else working independently.(Wikipedia-Reproducibility). To put it more vividly, I believe “Reproducibility” is like the recipe you attached to your freshly-made-creative-savoury dish with which people can reproduce the dish at home and even add some new ingredients or spices as time flies. Then Reproducible Research is naturally one where researchers in particularly but not restricted to the field of computational science deliberately “serve” for the scientific community to provide the complete development environment and the complete set of instructions.

How did reproducibility raise professionals concerns? I want to mention a recent event in academia in biology, and also a situation I just run into this afternoon with my mentor Sona in our small computational research project about wave equation.

140806_SCI_Riken.jpg.CROP.promo-mediumlarge

January 2014, two papers, whose lead author, Haruko Obokata, a young and beautiful Japanese female researcher in her early thirties, about a simple new method for creating stem cells were published in the prestigious journal Nature to much fanfare. It suddenly made a stir in media in Japan and beyond. However, these two papers were retracted in July after another researcher based in UC Berkeley concluding that her studies are dubious with the fact that his team had tried so many times but were still unable to reproduce the results. We can see the paramount importance of reproducibility in disciplines like biology, physics, mathematics whose quality of detailed deriving process determines the persuasion of authors’ achievements. In terms of scientific computation or the field of software development, both of which underline the final results and figures rather than the process, however, the significance of the implementation of reproducible-research frameworks stems from, first of all, the necessity for long-term maintenance of reproducible results. “Simple storage of software codes is useful but insufficient, because typical scientific codes have multiple dependencies( libraries, compliers, operating systems, etc.). With time, different parts of the software environment change and cause the reproducibility of previously published results to break down.” (“Reproducible research as a community effort: Lessons form the Madagascar project”–Sergey Fomel). Further, situations where researchers and publishers’ colleagues question part of their paper and look for backup details from them are too common and familiar to many of the professionals in the field, and their review and tracing back the right set of codes used in that paper and not to mention the right set of parameter they select to output the nice results takes these researchers a fairly big amount of time and thus decrease their research productivity or academic competitiveness. This is so true even for an undergraduate student who is doing her own independent project like me. This afternoon, I was debugging the codes my mentor sent me to refer to which can make a movie in 2-D about a stable moving wave. It’s a matlab code and I tried many times but still could not find what’s going wrong. I decided to check with google one command by one command. After tedious repetition of “copy and paste” on google I finally found out that it is because one of the file “avifile” has been removed from Matlab and I shall use “VideoWriter” class instead. My personal experience exemplifies that inevitable changes in the software environment easily cause breakdown and without dedicated and continuous maintenance, the computational results easily loose their credibility and practicability.

At the end of this blog, I’d like to provide two very useful and detailed website which people interested in the specific tools for implementation of Reproducibility can refer to afterwards:

http://reproducibleresearch.net/events/

http://www.ahay.org/wiki/Houston_2014 You’re also welcome to join the Madagascar open community, a great source comprised of three levels: programs, workflow scripts, and papers.

Advertisements

What I learned from Saranya Murthy’s talk at AWM

I will give you but three words, IMG_20141114_163651“FORESIGHT”,”DILIGENCE”,”STEADFASTNESS”, If you ask me to summarize what I took away from Saranya Murthy’s talk at UT AWM (American Women in Mathematics)’s weekly talks by successful and influential women in STEM in academia or industry. Saranya Murthy is an International Product Support Associate Engineer at Dell. She was a formal employee at Workbrain(now Infor), a software used by employees, providing web-based workforce management solutions for large enterprises. She graduated from University of Waterloo in Honors program in computer science in the faculty of mathematics. Later on after her graduation and working for years, she accomplished a part-time MBA program at York University in Canada. Miss Murthy must be originally from India, as shown from her skin and that her parents who are traditionally dressed in Sari(an Indian female garment) and Lungi are also in the audience while she’s giving us the talk. She maintained a graceful demeanour from beginning to end. She was very clear in logic- she gave us useful suggestions(I will demonstrate them later in this blog) about how to build personal development and why those ways are efficient and helpful; she was very precise about diction too, which can indirectly reflect her work ethics and lifestyles. Later after the presentation she told me she loves reading and holds book clubs with her friends (recently they’re reading “Lean in” written by Sheryl Sandberg- Chief Operation Officer at FB. Accidentally I am reading this book too. I’m currently on the first chapter about the gap between women’s ambition and leadership).

I want to specifically note down the advice given by Saranya Murthy during her talk and thus to benefit my readers.They’re as follows:

KNOW THYSELF

1. 360º feedback –http://www.reachcc.com/360reach -Click on 360º Reach Basic- it’s free!

2.MBI test(Apply the rest results to interviews and occasion of the same nature)

3.”Please understand me” by Dr. David Keirsey

4.”And Now Discover your Strengths” by Marcus Buckingham and Donald O.Clifton.

HOW TO BE A “CARE-FOR-YOUR-FUTUER” UNDERGRADUATE STUDENT

1.When deciding to registering for a class, ask the professor also for yourself “How can I use this in the real world?”;

2.Connect with like-minded professionals through information interviews and relevant social events;

3.Thoughtfully craft a 30 second elevator pitch which you may use later to pique people’s interest and let them quickly experience your charisma;

4.Develop a professional social media persona:

-Linkedin (Employers will usually check for Linkedin for candidates’ information and they particularly pay attention to your coworkers’/mentors’/professors’ recommendations and comments on it)

-Twitter (Thought leader) -WordPress blogging (why I’m blogging now:=)

Before the conclusion of my blog, I also want to point out that Miss Murthy is not only a role model in terms of work ethics and academic excellence, but more importantly she is a role model for me in terms of how respectful and obedient she is towards her parents. I approached Saranya for answers and guidance to my specific questions right after she concluded her talk, when her elderly parents are waiting for her besides. She friendly and politely addressed to me that we’d better find a place to sit down and talk so that on the one hand we can talk in details and on the other hand my parents can sit down and wait for me. These details of her bearing unintentionally touched my heart. I recalled how I yelled at my parents when I’m unhappy and how I unconsciously ignore them when I’m with friends…… I’d love to end my first blog at WordPress with Miss Murthy’s self-summaries on Linkedin which can constantly remind me of what precious personality my role model has and what I’m supposed to act to acquire that kind of personality and charisma in near future.

“• 9 years of experience in Quality Assurance (QA) for enterprise software

• Demonstrated leadership skills – motivated individual who can take charge and drive change

• Strong understanding of Software Development Life Cycle and relevant QA concepts

• Excellent technical and analytical skills – experienced at troubleshooting software issues and tracing defect triggers

• Strong experience in defining and improving software test processes

• Proven Project Management skills – responsible for overseeing QA team to deliver thorough and timely test efforts for 3-month release cycle

• Enthusiastic team player with effective interpersonal and leadership skills

• Experience working in Cross-Functional Teams – representing QA and liaising with Developers, Product Managers and Technical Writers

• Client-oriented approach: well-honed ability to test software from the “user’s point of view”

• Excellent written and verbal communication skills”