Timo's blog

A while ago I've sent a question to James Coplien, one of the loudest voices of software development processes, methodologies and design.

Hello mister Coplien.

After some meetings regarding working processes and working methodologies for software, we've reached a point on asking what are the alternatives to the most know approaches (waterfall and agile processes).

Our group consists of people actively working on software development, testers, students, and college teachers. So, from our limited experience, we couldn't list too many approaches; we ended up with a commitment of digging into the topic and share our findings in our next meetings.

That's why I would be super thankful if you could recommend some resources (articles/books) on software development processes/methodologies.

To my surprise, he replied back with a great explanation:

Hej, Juan,Thanks for writing.

First: just to pick nits, “methodology” is a branch of philosophy which is the study of method. Technically, things like waterfall and XP are methods and not methodologies. Alistair Cockburn, who has developed a theory of software engineering, playfully likes to call himself a “methodist” rather than a “methodologist.”

A method is therefore a pre-packaged set of practices and processes suitable to design and implementation. One of the great learnings of the 20th century is that method doesn’t work. The architect Christopher Alexander has been one of the leaders of the Design Movement since about 1968, and is very familiar with the notion of method. He writes:

No one will become a better designer by ... following any method blindly... if you try to understand the idea that you can create abstract patterns by studying the implication of limited systems of forces, and can create new forms by free combination of these patterns—and realize that this will only work if the patterns which you define deal with systems of forces whose internal interaction is very dense, and whose interaction with the other forces in the world is very weak—then, in the process of trying to create such diagrams or patterns for yourself, you will reach the central idea of which this book is all about. — Christopher Alexander, “Notes on Synthesis of Form,” 1974

So it is not a matter of choosing between alternative potential approaches, but rather of transcending the whole framework of reducing design to method. No more than Michelangelo followed any method in making the statue of David or of painting the roof of the Sistine Chapel will method help any technical design activity.

Even within the framework of agile, there are those that believe that Man can triumph over the complexity of nature and that they can prevail using some method. XP is a very old example. SaFE is a newer example. The things that work dismiss this claim and open the doors to self-organization and feedback. That is true agile. True agile is incompatible with the fundamental notions of method.

This is why I am an advocate of Scrum. Scrum is not a method but rather a framework. If you want to explore the method space I’d recommend exploring Crystal and the works of Alistair Cockburn. Scrum and Crystal aren’t the only ones. But to find anything effective for complex development you must abandon method. Method is good for simple and merely complicated work — not for complex work.

What you have read is aa OK start, but I think you are exploring noise and need to dig deeper into some of the underlying theory before looking at experience reports like Scrum from the Trenches. If you want to know Scrum in depth, then get the new “A Scrum Book” (https://pragprog.com/book/jcscrum/a-scrum-book). That will help you understand Scrum. But Scrum is not a method, and I think you need to come to grips with the scope within which method works. Maybe you even need to come to grips with the taxonomy of design within which method is historically a relatively small part. Therefore, I recommend getting these books as well:

Design after Modernism: Beyond the Object, by John Thackara (https://www.amazon.com/gp/offer-listing/0500234833/)

Developments in Design Methodology, by Nigel Cross (https://www.bookdepository.com/Developments-Design-Methodology-Nigel-Cross/9780471102489)

Notes on the Synthesis of Form, by Christopher Alexander, 1974 printing (https://www.amazon.com/Notes-Synthesis-Form-Harvard-Paperbacks/dp/0674627512/ref=sr_1_fkmrnull_1)

There are many more, but this is a start. From where you seem to be at, you have many years of work to do before coming to any conclusion.

At this point, Alistair Cockburn (participant on the Agile Manifesto) got involved, and provided a lengthy response:

hi, Juan, Cope,

I rarely like to disagree with Cope, but he is wrong twice here, and putting words in my mouth, so I feel somewhat obliged to step in. I'm afraid this is a long discussion, but one in which I have invested considerable research. Nothing of what I am about to say is "off the cuff".

First because it is easiest : I am a methodologist, not a methodist. The "-ologist" refers unambiguously to "person who studies --".?? I study methods and methodologies (explanation of the 'methodology' in long form below).?? I am not a methodist, and never refer to myself as such. A methodist is not a studier of, he is a person who follows the rules of the method, in particular, a very rigid form of Christianity. So if I am anything, it is not a methodist :).

For those who wish to be pure, that "-ology" means "study of", I then insist that I am a "technologist" because my field is "technology", the study of techniques. Until the users of the English language repair the word "technology" to mean, as they imply from the suffix "-ology", study of techniques, then I feel no obligation to honor the false appeal to ancient Greek etymology for "methodology". But more on that in a moment. I am a "technologist" (studying techniques") or a "methodologist" (studying methods and methodologies), but not a "methodist" (a particular, very formulaic Christian religion)

For the second, "Methodology", Cope makes the usual vague reference to a false etymology based on ancient greek. For my PhD dissertation, I had to take on this question, I researched and tracked this word over a period of years and using different dictionaries.

It is best that I don't try to re-write that research, but rather, show you that section of my PhD dissertation.

1.1 Clarification of Words

With the word methodology being used in different ways by various writers, and often interchanged with the word method, I must clarify my use of the words in order for the rest of the thesis to make sense.

Methodology

My use of the word comes from two sources: the Merriam-Webster Dictionaries and common usage among large contract software houses.

?????????????? The (U.S.-based) Merriam-Webster dictionaries list as their first definition: "A series of related methods or techniques."

?????????????? When I sat in a discussion of methodology with a group of senior consultants from Coopers & Lybrand, Ernst & Young, Andersen Consulting, and the IBM Consulting Group, one of them stood up and said (words to the effect of): "Let's get clear what we mean by methodology. It is the roles, teams, skills, processes, techniques, tools, and standards used by the project team." The others in the room nodded their heads, and the meeting proceeded. As one of the senior consultants later told me, "If it is what one or two people do while working on a task, it is technique. If it is what the team uses to coordinate their work, then it's methodology."

Those two meanings of the word are congruent and match the common use of the word.

Many British and continental European speakers dislike the above definition, preferring to use the word method instead. They cite the (U.K.-based) Oxford English Dictionary, which in the early 1990s contained only one entry for methodology: "study of methods." This explains the cultural difference in usage. However, the 1995 Oxford English Reference Dictionary includes in the entry for methodology: "2. a body of methods used in a particular branch of activity." Both American and British dictionaries now agree on the subject.

Method

The Merriam-Webster dictionaries define method as a "systematic procedure"; that is, similar to a technique. This matches the way many European speakers and European research literature uses the word method. An example, from Mathiassen (1998), is the following:

Lesson 2: Methods are primarily frameworks for learning.

The intention behind Mathiassen's word method is quite different from my intention for the word methodology. Mathiassen writes method to connote what one or a few people are learning to use, usually as a technique. This learning eventually disappears into the background skill set of the practitioner.

In contrast, I write methodology to connote the conventions people agree to across the team (including development techniques). While the techniques disappear into each person's skill set, the conventions in play do not disappear into skill sets.

We shall need both words to complete the discussion. This thesis primarily addresses methodology rather than method.

Shifts in Meaning

As the research took place over a ten-year period, there were several minor shifts in the way the word methodology got used, as well as some shifts in the way the practitioner community used the word. I deliberately leave in place some of those shifts, because readers will come to this thesis with their own personal interpretations, and part of the thesis topic is dealing with those personal interpretations.

The first research question, for example, asks: "Question 1: Do we need yet another software development methodology, or can we expect a convergence and reduction at some point in time?" In this question, I do not mean just, "Do we need yet another set of team conventions, or can we expect a convergence and reduction at some point in time?" Rather, I accept the fluid way in which people ask the question, which often includes, "Do we need yet another software development technique, or can we expect a convergence and reduction at some point in time?" The answer that I develop responds to both versions of the question. Research question #2 is handled in a similarly liberal way.
However, by the time we reach the third question, "Question 3: How does the methodology relate to the people on the project?" the meaning narrows. It is quite a different question to ask, "How does a techniquerelate to the people on the project?" That question has to do with introducing techniques into groups, as well as the relation between technique-in-theory and techniques-in-practice (the latter topic being well discussed in the Scandinavian literature, see Ehn (1988) and Mathiassen (1998)). In the context of this inquiry, it is relevant to use the narrower meaning, "How do the team conventions relate to the people on the project and other aspects of life on the project?"??

I found these replies and explanations extremely good, especially to someone trying to get more information about software development processes, so, I wanted to make them public because they might be useful for someone else.

Additionally, I've never expected a reply, it's great to know that knowledge can be shared and guidance can be provided despite the name, profession, location or expectations, thank you James and Alistair for your time.

My recommendations:

Timo's blog

Menu

June 10, 2020

Tale: ETL component

June 4, 2020

About software development methodologies

Methodology

Method

Shifts in Meaning

August 29, 2019

Bitbucket and Mercurial