Hi Chaminda,

I agree the criticism is a bit vague but he seems to be making a case for rigour. There are two ways qualitative researchers demonstrate rigour:

1.     An audit trail of the coding and analytical processes deployed in the study

2.     Demonstrating a robust analysis which goes beyond the clichéd  "identification of themes"

Let's start with the audit trail. An audit trail would demonstrate through the use of appendices (sometimes referred to as a code book) exactly how your framework was developed rather than simply presenting the finished product. For example, you might use five cycles or phases of analysis, some of which involved coding, some involving managing codes and some which involved documenting coding content:

Phase 1– Open Coding – involved broad deconstruction of the data from its original chronology into an initial set of non-hierarchical codes (Supporting Appendix XXX)

Phase 2– Categorisation of Codes – involved renaming, merging, distilling and clustering related codes into broader category of codes so as to reconstruct the data into a framework that made sense to further this particular analysis and address the research question(s) (Supporting Appendix XXX)

Phase 3– Coding on – involved drilling down on the now reorganised codes and re-coding them to sub-codes so as to better understand the meanings embedded therein (Supporting Appendix XXX)

Phase 4– Data Reduction – involved abstraction to broader researcher and literature based themes to arrive at a final framework on which to report findings (Supporting Appendix XXX)

Phase 5– Writing analytical memos to synthesize content down to manageable proportions and create a first draft of findings from which conclusions may be drawn leading to a discussion or intellectualisation of the findings. (Supporting Appendix XXX)

Better again if these processes are linked to the literature. There are a myriad of qualitative data analysis methodologies from which you could draw upon the guidelines to give credibility to you as a researcher and plausibility and trustworthiness to the findings. You could create a table citing various methods, offer a description of each one and include a critique of each one and finally, why the method was ruled out as unsuitable for this project or indeed why one method was chosen. See example below from a three thousand word assignment designed to prepare masters level students for doctorate level analysis:

 Click here to download from QDATRAINING website

The second means of demonstrating rigour is through the scope of analysis. Essentially, a good analysis addresses five key areas:

1.     The content of the framework – what was said and how it was said. How it was said can be tracked through annotations in NVivo which can be linked from field notes and observations.

2.     Who said it – can be tracked through case nodes linked to profiling or demographic information in NVivo (units of analysis and observation)

3.     Coding patterns – can be reported through visualisations in and exports from NVivo

4.     A formal documenting process which challenges the researcher and participants alike. An example of this might be raising proposition statements using linked memos in NVivo. Ultimately, such analytical memos need to be synthesised into a narrative that tells the participants’’ stories.

5.     The extant literature – can be demonstrated in NVivo by coding the literature to the same framework that we have seen developed from the primary data. Much of this coding can be automated through queries in NVivo as, unlike your primary data which is rich and unstructured, literature publications are structured documents with meta data attached and so can be easily searched and coded according to conditions such as key words, authors, year etc… By adding the key literature to your coding framework, you can easily compare (and be seen to do so) what real people in your study are saying about a given phenomenon you have coded for contrasted against what the lit is saying. This process adds values your analytical memos because you’re are now testing for congruence with the literature and identifying gaps; all of which helps to build a cohesive and coherent argument in your findings and discussion section of your paper.

You asked for some reading on the application of NVivo tools in qualitative primary research:

Read:

Pat Bazeley's article: analysing Qualitative Data – more than identifying themes" discusses scope of analysis and gives examples of NVivo tools to enhance the process: http://www.researchsupport.com.au/bazeley_mjqr_2009.pdf 

This article discusses an NVivo tool known as a matrix to test for consistency in data analysis (number three on our previous best practice analytical checklist)

Click here to download from QDATRAINING website

This article looks at using literature and reviewing it in NVivo:

Click here to download from QDATRAINING website

If your study has addressed the processes set out above, it is highly unlikely you will ever be accused of lacking transparency regarding how you arrived at the findings you did. If it has not, it might be worth investing in a couple of hours, one-to-one online training using your live data to explore the extent to which your data is set up correctly in NVivo and to learn how to use the aforementioned tools to satisfy your reader as to the trustworthiness and plausibility of your framework.

You can get more information on one-to-one online training and consultancy here:

http://www.nvivotraining.eu/online

I hope this was helpful,

Kind regards,

Ben