1. Introduction
One of the most important criteria for determining the scientific validity and reliability of a research has to do with its methods and how these constitute a methodology (Yin, 1984). It is precisely the explicit and clear enunciation of these methods, whether data collection or analysis, that can guarantee the quality standards of a research and make its results an inter and transdisciplinary value. Moreover, a well-structured and well-described methodology can be an important contribution in itself, serving as a basis for future research in similar settings. If quantitative approaches already have an established place in architectural research, importing from other disciplinary fields such as engineering, the same does not seem to happen with qualitative or mixed methods approaches, in which there is still a long way to go (Li et al., 2021).
In the PhD thesis presented in this paper, we conducted a systematic literature review focusing on the role of freehand drawing in design teaching. By examining research from the last twenty-three years in three prominent databases, we uncovered not only a shortage of studies on this topic but also identified their methodological fragility. Among the papers selected for our analysis, less than half were based on empirical evidence and explicitly outlined their methodology. Some of these papers did not even have a section dedicated to research methods. These numbers shed light on the current status of qualitative and mixed methods research in architectural education, revealing not only a quantitative deficit but also, more significantly, issues related to research quality.
In this paper we seek to examine the significance of a detailed and well-informed methodological framework as a prerequisite for the relevance of architectural research. We stress the importance of an evidence-based approach, emphasizing its role in elevating the quality and impact of architectural studies. To this end, we present the methodological framework used in a PhD research within the field of architectural education. We delve into the processes of data collection and analysis, describing and discussing the employed approaches. Through this paper, we aim to contribute directly to future research in similar settings, but also to contribute to the broader discussion on the relevance of architectural research.
2. What Do We Currently Know?
The thesis was structured around three main research questions: (i) how is freehand drawing used in architectural design teaching? (ii) why is freehand drawing still employed in architectural design teaching? (iii) how could freehand drawing be employed in architectural design teaching? To understand how these research questions were addressed in recent years, we conducted a systematic literature review. This involved identifying and analyzing the most pertinent studies on the same subject, which is a critical step to determine the current state of knowledge. It helps us build upon the existing knowledge base and avoid going back to the “drawing board”, both in terms of findings and the methods used to generate them.
In a preliminary phase of research, we observed that the topic of architectural education has already been extensively explored, with notable examples such as Milovanovic (2019), Iyer (2015), and Oh et al (2013). However, the same level of investigation does not exist concerning the role of freehand drawing in this learning process.
The systematic literature review serves as a method to ensure the reliability of data collection and analysis, outlining the procedures followed throughout the process (Merzdorf et al., 2021). To guide this review, we followed the methods described by Borrego et al. (2014) and Bramer et al. (2017). Three databases were utilized as sources for our review: two of them were chosen for their multidisciplinary nature—Scopus and Web of Science—while the third was selected for its specialization in education-related studies—ERIC. Our inclusion criteria consisted of peer-reviewed journal and conference papers in English, published after the year 2000.
A primary challenge was the diverse array of subjects in architecture-related publications, spanning through computer science, engineering, social sciences, as well as arts and humanities. This required multiple steps to refine our search strategy and select the most appropriate keywords. Our goal was to collect approximately 150 publications, ensuring a balance between a manageable volume of evidence while also providing a comprehensive perspective on the subject matter.
It’s noteworthy that none of the 20 papers we initially selected directly tackled our research questions; however, they provided valuable insights that allowed us to indirectly address them. To make sense of this data, we conducted a thematic analysis of the selected papers and the evidence they presented. This approach enabled us to identify recurring patterns and significant themes among the perspectives we examined.
Considering the first research question, the literature generally indicates that freehand drawing plays a significant role in both the design process and teaching, serving as a valuable tool for thinking and visualization. In response to the second question, it also emphasizes its capacity to stimulate creativity, self-expression, and abstraction. Addressing the third research question, which is inherently more speculative, we gleaned insights into potential future directions for freehand drawing. This includes the reinforcement of both analog and digital aspects, exploring the possibilities of digital hand drawing, and, notably, considering the interests of students.
Another critical aspect we aimed to explore during this literature review was the methodological component, examining how the results presented in the literature were collected and analyzed. Surprisingly, out of the 20 selected papers, only 13 provided empirical evidence, and of those, only 9 describe a clear and detailed account of the methodological procedures employed. This raised an important concern, indicating a gap in research methods. This gap pertained not to the themes being explored but rather the methodological aspects, highlighting a lack of empirical research and comprehensive documentation of research procedures.
When considering research approaches, we found that 6 out of the 13 articles followed a Mixed Methods approach, 5 used a Qualitative approach, and 2 used a Quantitative approach. Moreover, 11 out of the 13 articles were grounded in an experimental research design, with 2 articles employing a case study design. Concerning data collection and analysis methods, experiments and questionnaires were the two most commonly employed.
3. How Was the Methodology Defined?
To enhance and streamline the methodology for a research project, one valuable strategy is to draw from relevant precedents as references. Depending on the specific research goals, these precedents can be combined or adapted to tailor the methodological procedures to the subject and perspective in question.
However, in cases where prior examples are limited or nonexistent, as was the situation in our research—examining educational practices within their specific context—a necessity emerges to establish a methodological framework. This framework, akin to the creation of a theoretical framework, provides guidance for determining the research approach, design, and the specific methods for data collection and analysis, including the establishment of precise criteria. To define this methodological framework, we undertook an extensive review of specialized literature, which led us to conduct a narrative literature review with a focus on qualitative and mixed methods field research within educational contexts.
The first aspect that needed clarification was the selection of the research approach. Based on the criteria presented by Borrego et al. (2009), the selection of the approach depends on (i) the nature of the research questions, (ii) the researcher’s background and experience, and (iii) the target audience. The last two criteria have limited influence on defining the approach, as the field of architecture itself doesn’t inherently prescribe a direction for research. Typically, architectural research draws from the assumptions and strategies of the social sciences, as noted by Creswell (1994, 2014) and Stake (1995). However, it’s the specific research problem at hand that ultimately shapes the approach to be taken, as underscored by Yin (1984).
In our case, the nature of the research questions, particularly the second and third ones, which are characterized by their explanatory and exploratory character, and framed as “how” and “why” questions, align with qualitative approaches (Yin, 1984). Consequently, our research will be primarily qualitative, with certain quantitative data as supplementary.
The next step involves defining the research design, which serves as the logical structure of the study, with the primary objective of ensuring that the collected evidence allows for clear answers to the research questions (de Vaus, 2001). In qualitative research, the research design often comprises strategies such as narrative, survey, experimental, archival, historical, ethnographic, or case study (Creswell, 2014).
According to Yin (1984), the selection of the research strategy should take into account three primary conditions: (i) the type of research question or problem, (ii) the extent of control the researcher has over behavioral events, and (iii) the focus on contemporary events within a specific context.
Concerning the first condition, as mentioned earlier, this study revolves around research questions with an explanatory and exploratory nature (how and why freehand drawing is used in architectural design teaching). In these cases, research strategies like case studies, experiments, or historical analyses are recommended (Yin, 1984). Regarding the second condition, our aim is to investigate a phenomenon that cannot be subject to behavioral manipulation (architectural design teaching). Therefore, as Yin (1984) suggests, case study and historical research strategies are the most appropriate choices. Lastly, concerning the third condition, the necessity to focus on contemporary events within a specific context (three architecture schools), emphasizes the suitability of a case study strategy.
Case studies are characterized as empirical investigations that examine contemporary phenomena in real-life contexts, particularly when the boundaries between the phenomenon and its context are not evident. In fact, as a research strategy, case studies are frequently used in research within educational settings, as noted by Merriam (1998) and Stake (1995).
Referring again to Yin (1984), the definition of a research project based on a case study should consider three key aspects: (i) the number of cases to be examined, (ii) the selection of units of analysis, and (iii) the establishment of a logical link between the gathered data and the research problem. Regarding the first aspect, this research intends to investigate three educational institutions. Following the framework outlined by Yin (1984), it is common for each institution-school to be treated as an independent case. Therefore, in contrast to single-case studies, this research will adopt a multiple case study design.
Concerning the second aspect, the research’s objective is to focus on a specific phenomenon, namely, the teaching of architectural design, within each case institution. The aim is not to analyze the cases as a whole, from a holistic perspective, but to focus on a study subject composed of a set of embedded units of analysis. As a result, the research will employ a multiple case study design, encompassing three units of analysis: the context, the design process, and design critiques. Regarding the third aspect, the aim is to establish a connection between the collected data and the research problem using a logical deductive-inductive method, beginning with an overarching picture (general) before delving into the specific study object (particular).
4. How Was Data Collected?
Data collection in all three case studies primarily involved fieldwork sessions where naturalistic and participatory observation methods were used. The choice between these methods depended on the specific case and the availability of instructors. Typically, the researcher aimed to minimize interference with the normal flow of classes, adopting a non-intrusive approach. Nonetheless, there were occasions when instructors invited the researcher to become actively involved in the classes, providing feedback on students’ work, serving as a jury during final presentations, and even participating as a teaching assistant for short periods.
The data collection process primarily consisted of taking raw fieldnotes during class observations (Figure 1). Additionally, curricular documents and reports were collected to gain insights into course objectives, schedules, and some specific papers. The collected fieldnotes were later transcribed into a digital format referred to as Didactic Registration Units (Figure 2). This transcription process served as a preliminary stage for selecting and summarizing the collected materials. It also helped evaluating the effectiveness of data collection procedures, which, in turn, informed future data collection sessions.
Whenever possible, photographic records were taken, serving a dual purpose: documenting the teaching environment and capturing unique aspects of materials produced by students and instructors. Although photographic records had limited relevance during data analysis, they were important in visually communicating the research findings.
The primary emphasis during fieldwork was on the design studios. The objective was to observe students from various academic years to gain insights into their backgrounds and education, instructor expectations, and curriculum in the three schools. The types of classes observed included design critiques, pin-up sessions, mid-term reviews, final presentations, and theoretical classes. In addition to in-person fieldwork, remote classes were also attended, particularly during the second wave of the Covid-19 pandemic between November and March 2021.
As observed during data collection, systematic direct observation helps mitigate the initial awkwardness of the researcher’s presence for both instructors and students. In the initial observed classes, the researcher’s presence often influenced subject behavior due to the novelty of the situation. Systematic observation also allows for the identification of exceptional situations that may occur during classes. For instance, it was noted in all three schools that instructors tended to showcase the best student works as exemplary cases, which could potentially introduce bias into the research findings. Extensive class observations, covering different teaching moments, are therefore relevant for a comprehensive understanding of events and behaviors.
Furthermore, this approach enabled the contributions of research subjects themselves, namely instructors and students. Spontaneous conversations that occur during fieldwork are invaluable, serving as a form of open interviews, as highlighted by Yin (1984). These conversations offer direct input, including literature recommendations, while also providing opportunities to get feedback on research results, interpretations, and methodological procedures being employed.
5. How Was Data Analyzed?
According to Merriam (2009), data analysis and interpretation represent the complex process of extracting meaning from collected data through consolidation, reduction, and examination procedures. In our study, we employed thematic analysis, a common approach in qualitative research, to analyze texts derived from field notes and interviews, applying codes and developing themes (Hatch, 2002). During our research, data analysis proved to be a complex and time-consuming activity, primarily due to the absence of prior references. We had to undergo numerous trial-and-error tests to devise effective data management and manipulation strategies. This process spanned the entire duration of our study.
Throughout this thesis, alongside the core analysis, a series of side studies, referred to as Complementary Readings, were conducted. These studies had a dual purpose. On one hand, they aimed to delve into aspects that were not addressed in the primary analysis, such as how freehand drawing is taught outside of design studios (Sousa Santos et al., 2021b), or the implications of online teaching in design teaching during the pandemic (Sousa Santos et al., 2021a). Simultaneously, these Complementary Readings served to test data analysis methods, which, in turn, informed our main investigation.
We presented the findings from these partial studies at conferences to gather feedback from our peers. This process shed light on both strengths and weaknesses, providing valuable insights for the core analysis. These conferences also exposed us to other ongoing research that, while not directly related, offered relevant methodological new possibilities. For instance, our initial attempts at data analysis did not include specialized software, resulting in a time-consuming and intricate process due to the diversity and volume of data. It was during a conference that the use of Atlas.ti, complemented with Excel, was recommended to streamline the analysis procedures, and ensure the rigor and validity of our results (Figure 4-5).
6. What Lessons Were Learned?
In the work we presented, we can distinguish the lessons learned regarding the methodological framework and the systematic collection of empirical evidence. First, in terms of methodology, qualitative and mixed-method research possess a dynamic and interactive nature. It involves a comprehensive, adaptable, and evolving approach to the very study procedures. While it’s advantageous to begin with a well-thought-out plan, the flexibility to adapt to specific research circumstances is crucial, especially in the context of fieldwork with real-world intricacies. As referred by Hatch (2002), qualitative approaches are characterized by the openness, flexibility, and emergence of its procedures in response to the realities of the environment under study.
In our case, the final version of our research questions only took shape in the advanced stages of our work. Similarly, many initial methodological decisions underwent changes throughout our research journey. Nevertheless, the initial study and work plan were fundamental as they guided our research and provided a foundation for the subsequent adaptations. As Hatch (2002) argues, in a qualitative or mixed-method study, the researcher is not putting together the pieces of a puzzle of which he/she already know the figure, but building a figure that takes shape as the pieces are collected and examined.
In the second aspect, the empirical component takes center stage. This means acquiring knowledge through direct experience within the real-world context under investigation. This is particularly vital for studies like ours, aiming to understand a phenomenon within its natural setting. Furthermore, the systematic collection and analysis of data over an extended period are critical for minimizing interpretation biases, given the central role that the researcher plays in qualitative studies.
Furthermore, considering the existing body of literature, our research has shed light on nuanced aspects that were not previously identified during our literature review. Additionally, our findings challenge some conventional beliefs within literature. For example, it was found that freehand drawing is not only prevalent in all three schools under study but it’s also a valuable design tool for students, even those without formal training in hand drawing. Surprisingly, freehand drawing plays a pivotal role in design critiques at all three schools, with instructors primarily using it as their mode of interaction with students, without a comparable alternative. We see thus that despite the generalization of the use of digital media in architecture (Abondano Franco, 2015), in an educational context, freehand drawing continues to be used and to have a certain prominence.
Our data also revealed that freehand drawing is present throughout all phases of students’ architectural design, from initial concepts to advanced detailing. This challenges the prevailing notion in the literature that associates freehand drawing primarily with the early phases of design.
Furthermore, the common distinction between analog and digital, particularly concerning conventional freehand drawing and digital methods, does not hold up. Our research shows that these approaches are closely interconnected. During the pandemic, for example, we observed how instructors and students seamlessly transitioned to using digital devices for freehand drawing (Sousa Santos et al., 2021a). In the case of MIT, students employed digital tools for taking notes and creating drawings, mimicking a traditional notebook format. This transition does not fundamentally alter the freehand drawing technique, highlighting the integration of analog and digital methods.
7. Conclusions
With this paper, we present two key arguments. Firstly, we stress the importance of architectural research being firmly rooted in a well-defined and informed methodological framework. This means that researchers need to carefully study and establish a structured approach to their work, akin to how they would create a theoretical framework. By doing so, they can ensure that their research is based on stable methodological assumptions, which, in turn, guarantees its quality and reliability. Simultaneously, this approach allows them to make meaningful contributions to the broader knowledge base in their field.
We illustrate this point with the example of a PhD research project that explores the role of freehand drawing in teaching architectural design. The absence of relevant prior work in this area prompted us to go back to the “drawing board” to develop our methodological framework.
The second argument we make is the importance of anchoring this type of research in empirical evidence. This evidence should be systematically collected and analyzed, in contrast to studies that rely on anecdotal accounts, as highlighted in our literature review. This approach ensures that the collected data is tied to a specific context, and instead of being generalized, it can serve for transferability (Yin, 2011). Additionally, by following systematic procedures, it minimizes potential bias. This is an important consideration, especially in qualitative approaches where the interpretative component carries significant weight, and therefore, ensuring its reliability is crucial.
As we discuss in our lessons learned section, the evidence we gathered enabled us to draw conclusions that had not yet been adequately addressed in the existing literature, and in some cases, it challenges prevailing views.
Reflecting on the current research in architecture, Sofie Pelsmakers emphasized in an interview that its relevance hinges on delving deep into issues rather than skimming the surface, and on upholding high standards of quality (in Sousa Santos, 2023). This emphasis is rooted in the understanding that research must meet rigorous standards to be fruitful; otherwise, it becomes an exercise in futility, offering no meaningful contribution to the field of architecture.
Acknowledgements
This work was financed by Portuguese national funds through FCT – Fundação para a Ciência e a Tecnologia, I.P., under the research project «UIDB/00145/2020»