3 Integrity

3.1 Introduction

Scientific research has brought humanity immeasurably great benefits, such as reliable computing technology, high-quality medical care, and an understanding of languages and cultures that are not our own. All these assets are based on scientifically motivated knowledge. Researchers produce knowledge, whose progress and growth comes about because researchers build upon their predecessors’ experience and insights.

Example 3.1*: Sir Isaac Newton wrote of his scientific work: “If I have seen further it is by standing on [the] shoulders of Giants” (in a letter addressed to Robert Hooke, dated 5 Feb 1675)4. This metaphor can be traced back to the medieval scholar, Bernard de Chartres: “nos esse quasi nanos gigantum umeris insidentes” [that we are like dwarfs sat on giants’ shoulders] compared to scholars from the times of Antiquity. The aforementioned quote by Newton has also become Google Scholar’s motto (scholar.google.com).

In this chapter, we will discuss the ethical and moral aspects of scientific research. Science is done by human beings, and requires a well-developed sense of judgment on the part of the researchers. The Netherlands Code of Conduct for Research Integrity (VSNU 2018) (https://www.vsnu.nl/en_GB/research-integrity) describes how researchers (and students) are to behave. According to this code of conduct, scientific research and teaching should be based on the following principles:

  • honesty,

  • diligence,

  • transparency,

  • independence, and

  • responsibility.

The following sections will go over how these principles are to be implemented in our actions during the various phases of scientific research. How are we to set up a study, collect and process data, and report on our study in a way that is honest, diligent, transparent, independent, and responsible? This is something we have to think about even before we start working on our project, which is why these topics are discussed at the beginning of this reader, even though we will also refer to terms and concepts that will be worked out in more detail in subsequent chapters.

3.2 Design

To be sure, scientific research does bring us immeasurably great benefits, but this is balanced by considerable cost. This includes direct expenses, such as setting up and maintaining laboratories, equipment, and technical support, but also researchers’ salaries, financial compensation for informants and participants, travel expenses for access to libraries, archives, informants, and participants, etc. These direct expenses are usually subsidized by public funds held by universities and other academic institutions. In addition, there is an indirect cost, which is partially borne by informants and participants: time and effort that can no longer be spent on something else, loss of privacy, and possibly other risks we are not yet aware of. One often forgotten type of cost is loss of naïveté: a participant who has participated in an experiment learns from it, and, because of this, will possibly respond differently in a subsequent experiment (see §5.4, under History). This means that any results obtained in this subsequent experiment will generalize less well to other participants who have a different history, and have not yet participated in a study.

Given its great cost, research has to be thought through and designed in such a way that its expected benefits are reasonably balanced by its expected cost (Rosenthal and Rosnow 2008, Ch.3). If the chance that a study will yield valid conclusions is very low, it is better not to go ahead with this study, which will save on both direct expenses and indirect cost.

Example 3.2: Suppose that we would like to examine whether 4-year-old bilingual children might have a cognitive advantage over monolingual children of the same age. Based on earlier research, we expect a difference of at least 2 points (on a 10 point scale) between both groups (with a “pooled standard deviation” \(s_p=4\), hence \(d=0.5\), §13.6.2 and §13.8).

We then compare two group of \(n = 4\) children each. Even if there were actually a difference of 2 points between both groups (meaning, if the hypothesis were true), this study would still have a mere 51% chance of yielding a significant difference: the power of the experiment is only .51 (Chapter 14), because the two groups contain so few participants. It would be better for the four-year-olds and their parents to do engage in other activities (at school, at home, or at work) instead of participating in this study.

However, if \(n = 30\) children would participate in each of the two groups, and if there were indeed a 2 point difference between both groups (meaning, if the research hypothesis were true), then the power of the experiment would be .90. This means that bigger groups lead to a much better chance of confirming our study’s hypothesis. This elaborate research design will cost more (for the researchers and the children and their parents), but presumably it will also yield much more: a valid conclusion with great impact on society.

A study’s design (see Chapter 6) has to be as efficient as possible, and the researcher has to start thinking about it at an early stage. First of all, efficiency depends on choices regarding how the independent variables are manipulated. Is there a separate group of participants for each condition of the independent variable (meaning that conditions vary “between subjects”, like in example 3.2 above)? In a between-subjects design that involves two groups, we need about \(n = (5.6/d)^2\) subjects in each group (for further explanation of this, see Gelman and Hill (2007), and see §13.8). Or are all participants involved in all conditions (meaning that conditions vary “within subjects”)? A within-subjects design with two conditions requires only \(n = (2.8/d)^2\) subjects in each condition, and the study will therefore also have lower expenses and indirect cost for a much smaller number of participants. In general, this means that, if possible, it is more efficient (and hence better) to manipulate independent variables within subjects than it is to vary them between subjects.This is not always possible, however, firstly because by definition, individual characteristics only differ between subjects (for example: female/male sex, multilingual/monolingual background, aphasia/no aphasia, etc.). Secondly, we must take proper care to recognize effects of so-called transfer between conditions, which threaten our study’s validity (for example: experience, learning, fatigue, maturation); we will discuss such effects further in §5.3. The design choice of “between” vs “within” variation is further discussed in §6.2 below.

Being multilingual or being female are characteristics that may only vary between individuals. But other conditions may also vary within individuals, for instance, the day on which a cognitive measurement is taken. Suppose that we expect a difference of \(D = 2\) points between cognitive measurements taken on Monday and on Friday, respectively (with \(s = 4\) and \(d = 0.5\), see example 3.2). If we manipulate the day of measurement between subjects, meaning we make separate groups for children tested on Monday and those tested on Friday, this entails that we need \(n = (5.6/0.5)^2 = 126\) children in each group, yielding a total of \(N = 252\). However, if we manipulate the day of measurement within subjects, meaning that we observe each participant on Monday and also on Friday, this entails that we need a total of just \(N = (2.8/0.5)^2 = 32\) children. The within-subjects design means that far fewer children’s routines will need to be disturbed for our cognitive measurements. However, we must be properly aware of learning effects between the first and second measurement, and take appropriate precautionary measures. For instance, we can no longer use the same questionnaires in both conditions.

A study’s efficiency also depends on the dependent variable, in particular, on the observations’ level of measurement (Chapter 4), accuracy, and reliability (Chapter 12). The lower the level of measurement, the lower also the study’s efficiency. As accuracy goes down, the study’s efficiency also goes down, and more participants (subjects) and observations will be needed to be able to draw valid conclusions.

Example 3.3: Suppose that we would like to examine a difference between two within-subjects conditions, and suppose that the actual difference between them is 2 points (which yields \(s_D = 4\) and \(d = 0.5\), see example 3.2). However, suppose that we decide to look only at the direction and not at the size of the difference between the two observations for each participant: does the participant have a positive or negative difference between the first and second condition? This binomial dependent variable contains less information than the original point score (it contains just the direction and not the size of the difference), making the study less efficient. For this specific example, this means we would need 59 instead of 34 participants.

Thus, researchers are responsible for diligently and honestly considering and balancing their study’s cost and benefits, and they need to have a sufficient methodological background to be able to choose a proper research design, taking in account time constraints, the available participants and instruments of measurement, etc.

3.3 Participants and informants

Scientific research is done by human beings: researchers are but human. In the realm of humanities, these researchers themselves study (other) human beings’ behaviour and intellectual products. These activities are governed by laws, rules, guidelines, and codes of conduct that researchers (and students!) must follow, stemming from the aforementioned principles of diligence and responsibility. A study and the data collected for it may not lead to any kind of harm or significant loss of privacy for the parties involved.

For research in the humanities in the Netherlands, two laws are relevant:

It is compulsory to ask participants (or their legal guardians) for their explicit informed consent. This means that participants are fairly informed about the study, about its cost and benefits, and about their remuneration, and that, after this, they explicitly consent to participate. For researchers and students at Utrecht University, helpful examples of informed consent (information letters and consent statements) can be found on the website of the Faculty Ethics Review Committee for the Humanities (FETC-GW, discussed in more detail below), via https://fetc-gw.wp.hum.uu.nl.

All data that may be used to identify an individual are considered to be personal data, which may only be collected and processed according to the GDPR. It is advisable to separate one’s research data from any personal data as early as possible, which means anonymizing the data. Any information that links personal data and research data (e.g., a list with participants’ names and their corresponding anonymous personal code) is, itself, confidential and must be saved and stored with care. Do not keep personal data any longer than necessary. Research data may only be used for the (scientific) goal for which they were collected. Make sure that participants and informants are not recognizable in reports and publications on the study (i.e., use anonymous codes).

Photos and recordings of individuals (including audio, video, physiological data, and EEG) are subject to what we call portrait rights. This means that photos and other identifying recordings are considered on a par with portraits. When such a photo or recording is published, the person shown or represented may appeal to their portrait rights and claim damages for the harm done to them by this publication. This means that, if you might be interested in publishing a recording from which someone could be recognized, you must ask the individual who was recorded or their legal guardian for explicit consent beforehand (see above for the notion of informed consent). This also applies if you intend to demonstrate or show a fragment of such a recording at a conference presentation or on a website.

The Dutch WMO law (see above) states that any research involving human subjects (participants) must first be approved by a special committee; for the Faculty of Humanities at Utrecht University, this is the Medical Ethics Assessment Committee (Medisch-Ethische Toetsingscommissie or METC), which is administered by the University Medical Centre (UMC). This committee assesses whether the possible benefits of a study are reasonably balanced against the costs and possible harm done to participants.

However, most research in languages and communication at Utrecht University is exempt from review by the METC, which would otherwise be time-consuming, but it must be submitted to the Faculty Ethics Review Committee for the Humanities (Facultaire Ethische Toetscommissie - Geesteswetenschappen or FETC-GW, see https://fetc-gw.wp.hum.uu.nl/en/). This obligatory ethical assessment or review does not apply to research done by students, provided that some conditions apply. You can find more information on the FETC-GW website. When in doubt, always consult with your supervisor or teacher. This ethics assessment is also compulsory for students and researchers in other fields (literature, history, media & culture) who plan to do research with human participants (subjects).

3.4 Data

The data collected form the motivation and empirical basis for the conclusion drawn from scientific research. These data therefore have an essential importance: no data means no valid conclusions. Moreover, as we saw above (§3.2), these data are very costly (in terms of time, money, privacy, etc.). This means that we should treat them very diligently. We must be able to convince others of our conclusions’ validity based on these data, and we must be able to share the underlying data with other researchers, if asked.

Thus, diligence requires, at the very least, making a sufficient number of backup copies as soon as possible. Think of what might happen if a fire or flood would completely destroy the place where you work or live, or if your laptop would be stolen during your thesis project (this actually happened to one of our students!). If so, would proper and recent copies of the data be stored in other locations? For storing backup copies, a sufficiently secured cloud service5 is a good option.

Diligence also requires a proper record of what the data stand for, and how they were collected. Data without a matching description are practically useless for scientific research. Charles Darwin carefully noted down which bird found on which of the Galapagos Islands had which beak shape, and these observations later formed (a part of) the motivation for his theory of evolution. In the same way, we strongly encourage you to keep a log (on paper or digitally) of all steps of your research study, including motivations for these steps, if needed. Also note the brand, type, and settings used for any equipment you use, and note the version and settings for any software used. Keep a record of which processing steps were applied to the data, and why, and which file contains which data.

If you are working with digitized data (e.g., in Excel, or SPSS, or R), make sure to carefully keep track of which variable is stored in which column, using which unit of measurement and which coding scheme.

Example 3.4: The file found at <http://tinyurl.com/nj4pjaq. contains data from 80 speakers of Dutch, partially taken from the Corpus of Spoken Dutch (Corpus Gesproken Nederlands or CGN). The first line contains the variable names. Each subsequent line corresponds to one speaker. The pieces of data on each line are separated by spaces. The first column contains the anonymized speaker ID code, as used in the CGN. In the fifth column, the speaker’s region of origin is coded with a single letter: W for Western region (Randstad), M Central (Mid), N North, S South) (Quené 2008). Because of the careful annotation, these data may still be used with no problem, even if they were collected over 20 years ago by fellow researchers.

Data remain the intellectual property of those who collected them. Use of other researchers’ data with no citation may be seen as theft or plagiarism.

Data fraud (fabricating data, meaning, coming up with data out of thin air, instead of observing them) is obviously at odds with multiple principles in the code of conduct mentioned above (VSNU 2018). Fraud harms the mutual trust on which science is based. It misleads other researchers who might be building on the fictional results, and any research funds allotted to a fraudulent line of research are taken away from other, non-fraudulent research – in other words, it is a mortal sin of academia. Students and employees at Utrecht University who would like to discuss any questions or dilemmas around academic integrity are welcome to contact dr. Rick Nouwen, confidential advisor on academic integrity for the Humanities at Utrecht University (r.w.f.nouwen@uu.nl).

3.5 Writing

Scientific research only really reaches its purpose once its results are being divulged. Research that is not reported on could as well not have been conducted at all, and the cost associated with this research was, basically, spent in vain. For this reason, reporting research results is an important part of academic work. Publications (as well as patents) form a very important part of the “output” of scientific research. Researchers are measured by the number of their publications and these publications’ “impact” (the number of times these publications are cited by others who build upon them). This great importance is one of the reasons we ought to be diligent in treating others’ writings, as well as our own.

The researchers involved in a study must discuss amongst each other who will be listed as authors of a report or publication, and in which order. Those listed as co-authors of a research report have to satisfy three conditions (Office of Research Integrity 2012, Ch.10). Firstly, the must have made a substantial academic contribution to one or more phases of the study: think of the original idea, setting up and designing the study, collecting the data, or analysing and interpreting the data. Secondly, they must have been a part of writing up the report, either by doing part of the writing or by providing comments on it. Thirdly, they must have approved the final version of the report (most often implicitly, sometimes explicitly), and they must also have consented to being a co-author. It is best practice for the researchers to come to a mutual agreement on the order in which their names are listed. Usually, names are ordered by decreasing importance and extent of each author’s contribution. If the lead researcher is the main investigator and also a co-author, this person is often listed last.

Example 3.5: A, a student research assistant, helped collect data, but has made no other contributions, and is not entirely sure what the research is about. This means that A need not be listed as a co-author on the report, but the authors do have to describe and acknowledge A’s contribution in their report.

B, another student, conducted one of the parts of the research project supervised by researcher C. Supervisor C thought of the entire project, but B has collected literature, set up and conducted one part of the study, collected, analysed, and interpreted data, and reported on this all in a paper. Because of this, B and C are both co-authors of a publication on B’s part of the research project. They come to an agreement on the order in which authors are listed. Because student B was the most prominent person in the work, while C was the main investigator, they agree that B will be first author and C will be second (and last) author.

Researchers build upon their predecessors’ work (see example 3.1). This may also involve building upon their arguments and even their writing, but these cases do require that we always correctly refer to the appropriate source, i.e., to these predecessors’ work. After all, if we did not do this, we could no longer distinguish who is responsible for which thought or which fragment of writing. Plagiarism is “copying others’ documents, thoughts, arguments, and passing them off as one’s own work” (Van Dale, 12th edition [our translation]). This form of fraud is also a mortal sin of academia that may lead to substantial sanctions. The Faculty of Humanities at UU has the following to say about it:

Plagiarism is the appropriation of another author’s works, thoughts, or ideas and the representation of such as one’s own work. The following are some examples of what may be considered plagiarism:

  • Copying and pasting text from digital sources, such as encyclopaedias or digital periodicals, without using quotation marks and referring to the source;
  • Copying and pasting text from the Internet without using quotation marks and referring to the source;
  • Copying information from printed materials, such as books, periodicals or encyclopaedias, without using quotation marks and referring to the source;
  • Using a translation of the texts listed above in one’s own work, without using quotation marks and referring to the source;
  • Paraphrasing from the texts listed above without a (clear) reference: paraphrasing must be marked as such (by explicitly linking the text with the original author, either in text or a footnote), ensuring that the impression is not created that the ideas expressed are those of the student;
  • Using another person’s imagery, video, audio or test materials without reference and in so doing representing them as one’s own work;
  • Resubmission of the student’s own earlier work without source references, and allowing this to pass for work originally produced for the purpose of the course, unless this is expressly permitted in the course or by the lecturer;
  • Using other students’ work and representing it as one’s own work. If this occurs with the other student’s permission, then he or she may be considered an accomplice to the plagiarism;
  • When one author of a joint paper commits plagiarism, then all authors involved in that work are accomplices to the plagiarism if they could have known or should have known that the other was committing plagiarism;
  • Submitting papers provided by a commercial institution, such as an internet site with summaries or papers, or which have been written by others, regardless of whether the text was provided in exchange for payment.

https://students.uu.nl/en/practical-information/policies-and-procedures/fraud-and-plagiarism

In the case of self-plagiarism, the fragments or writing or thoughts in question are not taken from others, but from one of the authors. There are various schools of thought on self-plagiarism; however, it is advisable to be sure to cite the relevant source if one is to take ideas from one’s own work, building on the principles of diligence, reliability, transparency, and responsibility.

A reference or citation is a shortened mention of a source in the body of the text; you might have seen these quite a few times in this syllabus already. At the end of the report or text, a full list of sources follows, which is usually given the heading, “Sources”, “Sources consulted”, “References”, “Literature”, or “Bibliography”. A mistake in the references may be seen as a form of plagiarism (Universiteitsbibliotheek, Vrije Universiteit Amsterdam 2015) because the reader is directed towards an incorrect source. For this reason, it is imperative that researchers cite their sources correctly. Various conventions, depending on the area of study, have been developed for this. Usually, instructors will indicate which style or convention is to be used for citing one’s sources. In this textbook, we have intended to follow the style described by the American Psychological Association (2010), a style commonly used in the social sciences and some disciplines within the humanities. (For technical reasons, references may deviate slightly from the APA style.)

The rules for citing sources may sometimes be complex. In addition, authors must make sure that the citations in the body of the text correspond to the list of full references at the end. These tasks are best performed by a so-called “reference manager”, a program that collects references or citations, and correctly inserts them into the body of the text. An overview of such programs can be found at https://en.wikipedia.org/wiki/Comparison_of_reference_management_software. In writing this textbook we have used Zotero (https://www.zotero.org), combined with BibTeX (https://www.bibtex.org).

References

American Psychological Association. 2010. Publication Manual of the American Psychological Association. 6th ed. Washington, D.C.: American Psychological Association.
Gelman, Andrew, and Jennifer Hill. 2007. Data Analysis Using Regression and Multilevel/Hierarchical Models. Cambridge: Cambridge University Press.
Office of Research Integrity. 2012. “Responsible Conduct of Research Training.” http://ori.hhs.gov/education/products/wsu/rcr_training.html.
Quené, Hugo. 2008. “Multilevel Modeling of Between-Speaker and Within-Speaker Variation in Spontaneous Speech Tempo.” Journal of the Acoustical Society of America 123 (2): 1104–13.
Rosenthal, Robert, and Ralph L. Rosnow. 2008. Essentials of Behavioral Research: Methods and Data Analysis. 3e ed. Boston: McGraw Hill.
Universiteitsbibliotheek, Vrije Universiteit Amsterdam. 2015. “Webcursus Informatievaardigheden - Algemeen - Niveau b.” http://webcursus.ubvu.vu.nl/cursus/default.asp?lettergr=klein&cursus_id=131.
VSNU. 2018. “Nederlandse Gedragscode Wetenschappelijke Integriteit.” VSNU. https://www.vsnu.nl/wetenschappelijke_integriteit.html.

  1. A copy of this letter may be found at https://digitallibrary.hsp.org/index.php/Detail/objects/9792; for some background information, see http://www.bbc.co.uk/worldservice/learningenglish/movingwords/shortlist/newton.shtml.↩︎

  2. Students and employees of most Dutch educational institutions can use SurfDrive (https://www.surfdrive.nl) for easy data storage on secured servers.↩︎