NLP · peer reviews · requirements · Uncategorized

Fifty shades of requirements, part two – The peer-review folly

Last time I discussed what I called the “the death spiral” where I claimed that loosing focus on the true need gets transformed into unclear or badly stated system requirements.

If we for a moment start to think on how we usually “find errors” in our requirements we trust what’s known as the peer review technique. We typically generate a requirement specification or an export from our requirements database on a sub-set of the requirements ready for review and approval and then we invite some people and hope for the best (that is of cause that we don’t get massacred from senior colleagues).

peer review

The Peer review is the evaluation of work by one or more people of similar competence to the producers of the work (peers). It constitutes a form of self-regulation by qualified members of a profession within the relevant field. Peer review methods are employed to maintain standards of quality, improve performance, and provide credibility (source: Wikipedia).

But, does peer-reviews really work and above that doe they work for reviewing requirements?

My point is that peer review is impossible to define in operational terms. Peer review is thus like poetry, love, or justice. But it is something to do with a document (or in this case a set of requirements) being scrutinized by a third party—who is neither the author nor the person making a judgement on whether to approve the content.

But who is a peer? Somebody doing exactly the same kind of work (in which case he or she is probably a already involved)? Somebody in the same discipline? Somebody who is an expert on methodology? And what is review? Somebody saying `The document looks all right to me’, which is sadly what peer review sometimes seems to be. Or somebody pouring all over the document (or set of requirements), asking for data, rational, repeating analyses, checking all the references, and making detailed suggestions for improvement? Such a review is vanishingly rare.

If we start to look at the peer-review as a phenomenon we find evidence that’s peer reviews are one the point of useless and it’s a product of “this is what we always have done” or even worse “this is what I am told to do”. The process itself is often ill regulated and to my experience not well suited for reviewing requirements and can turn ugly and insulting:

“The process is unjust, unaccountable … often insulting, usually ignorant, occasionally foolish, and frequently wrong. We are correcting minors and missing majors.” – Richard Horton

A research made by IBM is actually supporting this idea:

Requirements faulire graphs

Of ALL errors that’s later found in a system 70% of them originates from the requirement phase. Ok, now hold on for a moment! We typically don’t write the requirements once, but rather in smaller iterations. That’s fine, since this study looks at all requirements written on the system level regardless if they where formulated in the beginning of the project or at requirement freeze.

But ONLY 4% (yes, you read this correctly) where found through the different reviews. The rest where given to the next person or group in the design food chain to chew on.

Of those pitiful fractions of defects, I am pretty sure that the majority were of the character “spelling errors” or simple semantics like forgetting a comma. A rather famous requirements guru said:

“Fixing typographical and grammatical errors is useful because any changes that enhance effective communication are valuable. However, this should be done before sending out the document out for broad review. Otherwise, reviewers can trip on these superficial errors and fail to spot the big defects that lie underneath.” – Carl Wiegers

Ok, so that can we do? Well, I would say that it comes down to the old classical things like:

  • Having proper knowledge on the importance of correct, complete and consistently formulated requirements.
  • This goes on all levels from management that must enforce good practices to requirements authors who need the skills, and;
  • Tools and techniques to write high quality requirements on the right things and of cause.

When it comes to tooling I must say that focus right now have been on requirements management tools like IBM Doors or Jama, but there have come some innovative tools that helps you in writing high quality requirements. Jama has some really nice features for collaborative reviewing and on-line work, but still you will not get any help on semantics or syntax. A bad requirement will be likely be less bad, since you wouldn’t get any help on the writing part of the problem, but you make it easier to review and spot lacy reviewers who just say “its fine” after a short glance.

One vendor of really cool tools is the REUSE Company with their tools tool Verification Studio and RAT (that’s correct like the mouse in the movie Ratatouille where the young chef has a small rat under his chef’s hat instructing him on how to prepare meals). This tools will guide you into writing correct requirements and use AI and reasoning in addressing how complete and inconsistent a specification is.

ratatouille

These tools can not replace human engineering, but they take you pretty far and for sure wash out all that dirt before going into a peer review.

Check out these tools on: https://www.reusecompany.com/verification-studio

So PLEASE stop relying on those peer reviews and start to take the requirements work more serious from start. Get help in writing high quality requirements. It will cost your project lots of hard work, frustration and a heap of money to fix it later in the design process if you don’t.

Next time I will explain what happens with those newly reviewed (but still bad) requirements when they are put to challenge by the design teams. Why are more errors introduced than spotted in that phase you might think?

Over and out!

NLP · requirements · Uncategorized

Fifty shades of requirements, part one – The spiral of death

The misplaced and flawed conception of requirements work leads to a death spiral

Lindsay-Weinstein-and-Jacob-Simon

Lindsay Weinstein and Jacob Simon perform a so called death spiral at the 2015 U.S. National Figure Skating Championships.

Imagine that you are working in an organisation with diverse knowledge of requirements work. Some projects work better than others and you don’t think so much about it. You write requirements because you are used to do so and to be honest, you don’t really understand why. When you started to work here you tried to question the current way of working but had no clear answer from either colleagues or your manager “we use requirements as part of contracting” you were told. You have some processes in place explaining the different steps on a high level focusing on handover, roles and responsibilities. You also have a requirements management tool where you store your requirements. You basically use this to be able to link requirements to test cases in the testing tool and that seems to work.

So, now after a couple of years on the job you have stopped to think about the steps of writing a specification at the start of any new project. You have actually seen that even though you discuss about the increased technological complexity in your domain, your feeling is that you cut-n-paste more now than ever and you look with envy on that other project that talks about new agile methods and reduced amount of documentation. To be honest it’s not at all fun writing documents. That was not the reason why you chosen to become an engineer in the first place.

bart-simpson-generator-520x245

“I wonder if we should consider skipping the boring requirements definition part and go straight to the more interesting and (for me) rewarding design phase”, you think. No one seems to care anyhow, well that’s not entirely true. Both people from contracting and test usually come when we are almost done and starts to question our work, saying that we have deviated from the requirements. It’s a habit they evolved actually. The new project that works agile explains that they will not have this problem. They have daily meetings and meet the customer on a regular basis. That’s good since the customer have changed their mind a few times now, even after the contract was signed.

You have also noticed when you get those monthly reports that the company is starting have problems with lost market shares, lower cash flow due to projects having problems delivering and increased customer complaints. But these problems will surely fix themselves now when the CEO was replaced and the new one talks about “maximizing the value flow” and put focus on “optimizing the innovation throughput”. You have also heard rumours about a reorganisation to help the organisation to change.

Does any of this seem familiar in any way? Well, for me I have come across this multiple time in the past. I have reorganised just to be reorganised six month later. Introduced new improved processes and Lean methods and coached, trained, managed and supervised people with various degree of interest or change resistance.

I will try to elaborate around WHY the above occur and especially the role that BAD REQUIREMENT PRACICES. So what has requirements got to do with all of the above? I call it the Spiral of Death.

The spiral of project deathThe spiral of death

So what is this picture basically saying? Well, If one starts with unclear needs from a customer or other stakeholders, they will for sure result in an unclear input to the project and design efforts. I am not saying in any way that we need to stay in the needs definition or requirements specification phase “for ever”, but we absolutely need to be sure of the boundaries of the project (the scope), the market drivers (the unique selling points), the risks and the fundamental needs from bench marks, surveys and support and failure logs.

If we rush through this phase and quickly “cut and paste” from previous and similar projects we will start of on the wrong foot, loose track on special needs, the context and engineering decisions made in the source project. We will start the troublesome journey downwards the spiral of death.

But, you might say. We do extensive peer-reviews on our requirements both before we start as part of the contract review AND as part of our engineering process OR we do agile development so we find out quite early where we have made mistakes and are able to correct these. The problem must lie somewhere else?!

NO, they absolutely do not and next time I will continue to discuss this topic focusing on understandability, peer-reviews as a phenomenon and provide examples and evidence supporting my statement made in this blog post.

Until then, be careful out there 😊

architecture · NLP · requirements · Uncategorized

Agile doesn´t work!?

By starting with the statement “Agile doesn´t work” you really ask for trouble these days. I was attending a conference earlier this week and I meet Agile coaches, Agile Managers, Agile Testers, Agile Requirement Engineers and Agile Evangelists in general. The conference itself was very good and I got the possibility to talk about the requirements role for a successful project. Unfortunately, I needed to leave early so perhaps you might call me an Agile planner?

No, but on the metro back to the office (you know my Agile Schedule) a LinkedIn post by Oleg Vishnepolsky, CTO at DailyMail in UK, caught my attention. It started “Agile does NOT work!”.

Stupid_Person

He started; “Try telling someone that you are not agile. You might be arrested on the spot. The issue is that the word agile in the eyes of everyone is similar to the word good. Try telling others that you are not good, and they will call for a psychiatrist.”

I continued to read the post with great joy. I think he is spot on!!

Why Agile isn´t a silver bullet

Agile can be defined as “Agile (software) development describes a set of values and principles for software development under which requirements and solutions evolve through the collaborative effort of self-organizing cross-functional teams” (source Wikipedia).

silverbullet

 

First of all, don’t get we wrong. I don´t like the approach with “analysis paralysis” and so heavy front loading that you never get through the first base of requirements elicitation and definition. Many organizations have historically lacked the ability to say “this is good enough” and moved further in the development chain. If you stop to long the inertia of badly performed Systems Engineering will force the project to continue and start to develop stuff regardless. They have deadlines and a customer to satisfy, AND doing Agile projects makes it easier to skip the boring work of documenting both requirements and design. The team can go right ahead and do the fun stuff..

But is Agile as an opposite the salvation of any badly performing organization with schedule overruns, lost margins, port system quality and unhappy stakeholders?

I would argue not! You have the same people coming to work, but now let’s do the mistakes in an Agile manner instead.

Well, of cause, Agile basic principles are nice and I do like them:

  • You have customer focus (instead of technology focus)
  • You divide the work in suitable chunks, and;
  • You group in teams instead of stove pipes so that you get much better cross pollination of views and ideas. That’s great!

But it is still humans that needs to understand the problem they are there to solve. Perhaps Agile is a way of getting it better, but not getting it right? Could an old Waterfall organization that goes Agile like this approach because it reveals the problems earlier than before? Are they really looking into the root cause of the problems they are facing (which haven’t gone away just because you went Agile).

The common-sense methodology

Mr Vishnepolsky introduces the common-sense methodology and I like that thinking very much. Instead of digging trenches and throwing dirt on anyone not from the right belief, you might like to make a short retrospect and look at yourself before you continue this argument.

I would like to add my small contribution: For god sake, look at the scope and early requirement definition phase and do it with FOCUS and a sence of INFORMATION QUALITY!

Making it right from start so that people understand what you mean

Ok, I might add that you can develop this type of knowledge using an arbitrary methodology, but just for the sake of it say that you write some business or stakeholder requirements to make sure that you understand the problem you are there to solve. After you have agreed with the stakeholders the task at hand (to a reasonable level of detail) you might develop technical requirements using natural language techniques (writing requirements) and/or doing some system modelling to get the system boundary and interfaces in place. You might have read the architecting blog series here by Hillary Sillitto?!

But who do you have in mind when you do this work? You are not doing this because it’s fun or because someone asked you to, you do it to COMMUNICATE with others. The stakeholders need to understand your solution to his/her need AND the next person receiving your information (developers, testers, integrators, etc.) also need to understand what you are proposingIT_DEPENDSSo, please start to do your requirements with some level of curiosity from the team working with them. You really need to understand their role here as drivers of knowledge from one part to the next. They first of all need to be correct and together with their other requirement siblings at the same abstraction level, they need to be complete (to the degree to level risk and avoid costly errors) and also not in conflict with each other so that one says “right” and another “left”.

But primarily, human write requirements for other humans. Perhaps using a second language and communicating through barriers like: culture, development contracts, technology domains and/or business hierarchies.

That’s my contribution to the common-sense discussion. Over and out!


Read the post by Mr Vishnepolsky here: https://www.linkedin.com/pulse/agile-does-work-oleg-vishnepolsky/