NLP · requirements · Uncategorized

Go Agile and avoid ALL problems?

Why bother about requirements? – I am Agile!

Again I was triggered by a post I read on the glorious source of infinite stupidity, the internet 😊

This story triggered me this time:

“Requirements were hard enough to manage in the bad old waterfall days. Here, mr XXX will take you through how you can do it, and keep your job, when working with agile.

On a recent project, Stories with unclear requirements were ending up with developers. It was a pain point. Time was spent trying to clarify missing pieces meaning less time to write great code and meaning  developers ended up under even more pressure to get things done”.

I can just say WOW!

Just to start of on the right foot here, I am a firm believer of requirements being the way to clearly specify and manage a complex problem and turn that into a hopefully correct (not so complex) solution, satisfying all relevant stakeholders.

And clearly, the “bad old waterfall” could create an analysis paralysis and a heap of useless documents in the dark ages of engineering, even if those people managed to travel do the moon and get us into the age of computing, but that could have been just luck and a lot of late hours I guess…

But getting back to the Agile folly in the beginning. These people DON’T want to write things down because the want to “write great code”. I guess it speaks for itself?!. They rather have a dozen of sprints than try to work proactive. It better to do something and potentially do it wrong (having an imaginary sense of speed and progress), then taking it down a few notches and examine the problem before they develop the solution. It seems that these agile evangelists never heard about the 80/20 rule?!

Back logs are reactive and story boards and yellow stickers are the most documentation these guys (and girls) want to do…

syntax_error

I am so glad that the coders have compilers saying “syntax error” before the code get into the SW system itself. Unfortunately, people don’t have this function and garbage in equals garbage out and its manifested by lack in functionality, bad quality and a lot of frustration.

Going Agile is great for small, isolated and not so complex tasks where the developers can sit as a team in a room and “write grate-isch code” and have a fun work environment and getting instant feedback from the users. I don’t question that, but I am sooo glad that complex and safety critical systems uses more rigid and “robust” development practices. By the way, what happened to Rational Unified Process, RUP? I thought it was a great attempt to become less stuck with paperwork and more “light on the feets”.

Agile vs Waterfall

Again, requirements in any shape and form; natural language text, models, user stories, personas, even as I saw the other day pictures and drawings on a white board, is there to illustrate and COMMUNICATE a need and put focus on solving the right problem in the right way. The biggest issue is that humans have one compiler each (our own brain) and syntax errors are introduced when we think that we understand, guess and interpret things based on our own experience, logic, motivation, drivers, culture, language, etc.

So, please, please STOP being digital, saying “I am Agile” (most popular) or “I am NOT Agile” (old dinosaurs). The real systems engineer says, “it depends” what processes to use and it should not get in the way of trying to develop a CORRECT, COMPLETE and CONSISTENT understanding between all parties involved and I am a strong advocate of doing so before I start to run to manage the resources and risks, but that’s just me…

architecture · NLP · requirements · Uncategorized

Fifty shades of requirements, part three – Architecture distortion

Hi again,

This time I want to talk about the “next” step in the requirements journey. I can right away say that I am a fan of agile and rapid development, but for the sake of this chat I like to describe a stepped approach. A newly formulated requirement should materialise on the solution side as one or many design elements. These elements form part of the architecture.

If you remember the blog here by Hillary Sillitto he described different views: System, Operational, Physical, Functional. The book he have written on architecting is brilliant and really to recommend. All of these views should capture different aspects of the design suggestion that eventually satisfy all the stated requirements. That’s all well, but what tend to happened in may cases?

We really now start to compare apples with pears (or should I say fruit salad?). Its really hard to compare written text with models. Are they correct, complete or are we missing something.

satisfied

So, is this a big deal? Well if we make a recap from last post. I then argued that most errors (and I then mean errors not found with a spell checker) are brought to the architects and system designer I need to validate that their proposed solution is Ok, right? But If I can not really penetrate their ideas because they are using other techniques, tools and languages I might loose that ability. Do you agree?

architecture_failure

The graph really show that the architects and designers are pretty good at doing their job and they find more errors due to the bad requirements than they produce and that’s a relief, but they will probably not find those errors that are caused by ambiguities, missing parts or plain wrong information. The architects produces and architecture that is supposed to cover all aspects that has been stated in the requirements, but they should not do anything if they don’t find a requirement justifying the design element or function.

One can argue that techniques like Model Based Systems Engineering (MBSE) are solving a great number of complexity issues and you can do so much magic when you are able to look at those views to see the complete picture, but you can not really validate it with people that don’t “read” SySML. Those people are all the stakeholders, customers, users that you so desperately need to communicate with and they don’t speak your language as an architect. Ok, again its now possible to add a requirement view in the modelling tools, but then you are just adding a notepad functionality and you miss all of issues of having correct semantics and syntax. You are basically back where you started (you have just moved them to another domain and probably tool).

Major problems in the transformation from in this case written requirements in one language (let’s say English) to an architecting language like UML or SYSML is the inability to validate Correctness, Completeness and Consistency. A rather academic picture and text I have stolen from my friend and colleague Prof. Juan llorens (sorry!) is this:

understanding

Bearing in mind that the goal of requirements is to specify a product, service or system, then we have two complementary points of view to define good or bad requirements. From the point of view of the sender, the final quality of requirements is validability, i.e., the sender must be able to confirm that the system architecture effectively expresses the system solution that answers his or her needs. This property can be further unfolded, in a second level, into three other properties: completeness, consistency and understandability.

From the point of view of the receiver, instead, the two essential qualities are: verifiability, i.e. it is possible to check that the developed system architecture meets the specified system requirements; and modifiability, i.e. it is possible to modify the architecture to enable change. Verifiability depends on the same three second-order properties already mentioned (completeness, consistency and understandability), as well as on three other properties: unambiguity, traceability and abstraction; modifiability depends also closely on these three last properties.

Unambiguity and understandability are interrelated (according to some, they would be even the same property), since, if a requirement is ambiguous, then it cannot be properly understood. Note also that abstraction and unambiguity are not opposing concepts: a requirement must have a single interpretation at the abstract level of specification (unambiguity) and at the same time must avoid technical details about the implementation (abstraction).

Other things worth mentioning is also that organisations seldom have one tool from one tool vendor, they have probably multiple tools spread and used differently in different parts of the organisation and to that we can ad a layer of self-made hacks, spread sheets and databases.

At one point I will write a future blog post on using natural language processing, requirement patterns and techniques to decode the language so that one can create requirements out of models (or vice versa), but for now I will say good night, auf wiedersehen and hope you read next blog on the test phase.

I have called it “the test catastrophe, but lest push on for that delivery”.

NLP · peer reviews · requirements · Uncategorized

Fifty shades of requirements, part two – The peer-review folly

Last time I discussed what I called the “the death spiral” where I claimed that loosing focus on the true need gets transformed into unclear or badly stated system requirements.

If we for a moment start to think on how we usually “find errors” in our requirements we trust what’s known as the peer review technique. We typically generate a requirement specification or an export from our requirements database on a sub-set of the requirements ready for review and approval and then we invite some people and hope for the best (that is of cause that we don’t get massacred from senior colleagues).

peer review

The Peer review is the evaluation of work by one or more people of similar competence to the producers of the work (peers). It constitutes a form of self-regulation by qualified members of a profession within the relevant field. Peer review methods are employed to maintain standards of quality, improve performance, and provide credibility (source: Wikipedia).

But, does peer-reviews really work and above that doe they work for reviewing requirements?

My point is that peer review is impossible to define in operational terms. Peer review is thus like poetry, love, or justice. But it is something to do with a document (or in this case a set of requirements) being scrutinized by a third party—who is neither the author nor the person making a judgement on whether to approve the content.

But who is a peer? Somebody doing exactly the same kind of work (in which case he or she is probably a already involved)? Somebody in the same discipline? Somebody who is an expert on methodology? And what is review? Somebody saying `The document looks all right to me’, which is sadly what peer review sometimes seems to be. Or somebody pouring all over the document (or set of requirements), asking for data, rational, repeating analyses, checking all the references, and making detailed suggestions for improvement? Such a review is vanishingly rare.

If we start to look at the peer-review as a phenomenon we find evidence that’s peer reviews are one the point of useless and it’s a product of “this is what we always have done” or even worse “this is what I am told to do”. The process itself is often ill regulated and to my experience not well suited for reviewing requirements and can turn ugly and insulting:

“The process is unjust, unaccountable … often insulting, usually ignorant, occasionally foolish, and frequently wrong. We are correcting minors and missing majors.” – Richard Horton

A research made by IBM is actually supporting this idea:

Requirements faulire graphs

Of ALL errors that’s later found in a system 70% of them originates from the requirement phase. Ok, now hold on for a moment! We typically don’t write the requirements once, but rather in smaller iterations. That’s fine, since this study looks at all requirements written on the system level regardless if they where formulated in the beginning of the project or at requirement freeze.

But ONLY 4% (yes, you read this correctly) where found through the different reviews. The rest where given to the next person or group in the design food chain to chew on.

Of those pitiful fractions of defects, I am pretty sure that the majority were of the character “spelling errors” or simple semantics like forgetting a comma. A rather famous requirements guru said:

“Fixing typographical and grammatical errors is useful because any changes that enhance effective communication are valuable. However, this should be done before sending out the document out for broad review. Otherwise, reviewers can trip on these superficial errors and fail to spot the big defects that lie underneath.” – Carl Wiegers

Ok, so that can we do? Well, I would say that it comes down to the old classical things like:

  • Having proper knowledge on the importance of correct, complete and consistently formulated requirements.
  • This goes on all levels from management that must enforce good practices to requirements authors who need the skills, and;
  • Tools and techniques to write high quality requirements on the right things and of cause.

When it comes to tooling I must say that focus right now have been on requirements management tools like IBM Doors or Jama, but there have come some innovative tools that helps you in writing high quality requirements. Jama has some really nice features for collaborative reviewing and on-line work, but still you will not get any help on semantics or syntax. A bad requirement will be likely be less bad, since you wouldn’t get any help on the writing part of the problem, but you make it easier to review and spot lacy reviewers who just say “its fine” after a short glance.

One vendor of really cool tools is the REUSE Company with their tools tool Verification Studio and RAT (that’s correct like the mouse in the movie Ratatouille where the young chef has a small rat under his chef’s hat instructing him on how to prepare meals). This tools will guide you into writing correct requirements and use AI and reasoning in addressing how complete and inconsistent a specification is.

ratatouille

These tools can not replace human engineering, but they take you pretty far and for sure wash out all that dirt before going into a peer review.

Check out these tools on: https://www.reusecompany.com/verification-studio

So PLEASE stop relying on those peer reviews and start to take the requirements work more serious from start. Get help in writing high quality requirements. It will cost your project lots of hard work, frustration and a heap of money to fix it later in the design process if you don’t.

Next time I will explain what happens with those newly reviewed (but still bad) requirements when they are put to challenge by the design teams. Why are more errors introduced than spotted in that phase you might think?

Over and out!

NLP · requirements · Uncategorized

Fifty shades of requirements, part one – The spiral of death

The misplaced and flawed conception of requirements work leads to a death spiral

Lindsay-Weinstein-and-Jacob-Simon

Lindsay Weinstein and Jacob Simon perform a so called death spiral at the 2015 U.S. National Figure Skating Championships.

Imagine that you are working in an organisation with diverse knowledge of requirements work. Some projects work better than others and you don’t think so much about it. You write requirements because you are used to do so and to be honest, you don’t really understand why. When you started to work here you tried to question the current way of working but had no clear answer from either colleagues or your manager “we use requirements as part of contracting” you were told. You have some processes in place explaining the different steps on a high level focusing on handover, roles and responsibilities. You also have a requirements management tool where you store your requirements. You basically use this to be able to link requirements to test cases in the testing tool and that seems to work.

So, now after a couple of years on the job you have stopped to think about the steps of writing a specification at the start of any new project. You have actually seen that even though you discuss about the increased technological complexity in your domain, your feeling is that you cut-n-paste more now than ever and you look with envy on that other project that talks about new agile methods and reduced amount of documentation. To be honest it’s not at all fun writing documents. That was not the reason why you chosen to become an engineer in the first place.

bart-simpson-generator-520x245

“I wonder if we should consider skipping the boring requirements definition part and go straight to the more interesting and (for me) rewarding design phase”, you think. No one seems to care anyhow, well that’s not entirely true. Both people from contracting and test usually come when we are almost done and starts to question our work, saying that we have deviated from the requirements. It’s a habit they evolved actually. The new project that works agile explains that they will not have this problem. They have daily meetings and meet the customer on a regular basis. That’s good since the customer have changed their mind a few times now, even after the contract was signed.

You have also noticed when you get those monthly reports that the company is starting have problems with lost market shares, lower cash flow due to projects having problems delivering and increased customer complaints. But these problems will surely fix themselves now when the CEO was replaced and the new one talks about “maximizing the value flow” and put focus on “optimizing the innovation throughput”. You have also heard rumours about a reorganisation to help the organisation to change.

Does any of this seem familiar in any way? Well, for me I have come across this multiple time in the past. I have reorganised just to be reorganised six month later. Introduced new improved processes and Lean methods and coached, trained, managed and supervised people with various degree of interest or change resistance.

I will try to elaborate around WHY the above occur and especially the role that BAD REQUIREMENT PRACICES. So what has requirements got to do with all of the above? I call it the Spiral of Death.

The spiral of project deathThe spiral of death

So what is this picture basically saying? Well, If one starts with unclear needs from a customer or other stakeholders, they will for sure result in an unclear input to the project and design efforts. I am not saying in any way that we need to stay in the needs definition or requirements specification phase “for ever”, but we absolutely need to be sure of the boundaries of the project (the scope), the market drivers (the unique selling points), the risks and the fundamental needs from bench marks, surveys and support and failure logs.

If we rush through this phase and quickly “cut and paste” from previous and similar projects we will start of on the wrong foot, loose track on special needs, the context and engineering decisions made in the source project. We will start the troublesome journey downwards the spiral of death.

But, you might say. We do extensive peer-reviews on our requirements both before we start as part of the contract review AND as part of our engineering process OR we do agile development so we find out quite early where we have made mistakes and are able to correct these. The problem must lie somewhere else?!

NO, they absolutely do not and next time I will continue to discuss this topic focusing on understandability, peer-reviews as a phenomenon and provide examples and evidence supporting my statement made in this blog post.

Until then, be careful out there 😊

requirements · stress · Uncategorized

Can you put Requirements on Christmas?

I feel that to many people do, perhaps to many. If I where to put some on Christmas what would it look like? Just from the top of my head it must include so many things:

The Christmas SHALL be full of happiness, joy, family gatherings, presents, good food, etc., etc.…

First, One cannot put that requirement on the <Object> “The Christmas”. It is just a word, but it means so much to people. They put in their own emotions, expectations and dreams into it. Second, as a requirements guy I can say that you MUST avoid piling attributes to it like I did in my example. But we all tend to do that, right? I can really imagine the failed verification on my requirement. I hope you can to. I need to relax and back off for a while.

johnny

Before you continue reading this, do me a favour. Put down what you’re holding (in your hand or your head)—your shopping lists, your third cup of coffee, your assignments, the phone call you should be making—and sit quietly for just 60 seconds. If you’re at work, don’t worry what your colleagues might think—this time of year everyone would love to contemplate, and more often.

Hello? Anyone there? It felt good to escape for a minute, didn’t it? But come back—it’s that time of year again: the modern miracle known as THE HOLLIDAYS, when into the dark little month of December, we squeeze in a lot of celebrations, from school gatherings, office parties and of cause family celebrations. Throw into that mix a generous dose of unrealistic expectations, budget-busting shopping, dysfunctional family feasts, airplane flights, darker days, colder weather, excess eating and drinking, and no wonder that along with “peace on earth”, come seasonal stress, exhaustion and depression.

So, lets rephrase my Christmas requirement:

The Christmas SHALL mean love for my fellow man.

By that expression (or requirement) I mean that people should get along with one another, have good feelings toward one another, and be nice to one another. It is this humanistic interpretation of Christmas that is celebrated in so many Christmas movies we either love or hate.

christmas-relaxing-768x511

So by this I wish you all a Merry, Merry Christmas whatever you do and where ever you are. I end thios blog post with a warning example captured in a poem by Tess Gurney:

Cars and bustle, rush and hustle,
trolley dashing tempers clashing
Christmas cheer to much beer
mistletoe, plastic snow
christmas holly ‘lets be jolly’
christmas pud, do you think we should?
Turkey, stuffing, granny puffing,
fairy lights, men in tights
pantomimes, palour games
christmas cake, belly ache
as yet another piece you take!
Boxing day, in-laws stay!
Endless repeats on the telly, and again from your belly!
No more hassle…the odd brussel!
christmas farce….
A pain in the arse!

requirements · Uncategorized

Problems with User Stories and how to improve them – part 2

In the last post Simon looked at the problem of expressing a user need using natural language. He will now continue with part two, the sollution.

User Stories using a structured technique

A technique that has proven to be very useful in reducing errors is patterns. One of the simplest way of expressing a user need is in a User Story, the structure of each story is often based on a pattern as follows:

As a … [stakeholder role] …I want to … [perform an action / record some information][With some frequency and/or quality characteristic] … So that … [description of value or benefit is achieved].

For example:

As a library_user, I want to search for books by title, with speed and ease-of-use, so that I can find all books with similar titles.

Effective natural language user stories/requirements generally consist of four basic structural elements [1]:- entities, actions, events, and conditions. These elements can be used or modified by various cases such as the following:

  • Owner. • Actor. • Target. • Constraint.
  • Owned. • Action. • Object. • Trigger.

Using these concepts, we can re-state the user story template like this:

As a        [Actor or Owner – who/what does the action]

I shall    [Action – what happens e.g. store, update, send data]

for;         [Object – what is acted upon]

on the   [Target or Owned – where the output is sent ; recipient or end state]

with       [Performance – frequency and/or quality characteristic]

when    [Trigger – causes of action; data receipt/user interaction]

unless / even if  [Constraint – business rule or limiting factor]

So that [Rationale – description of value or benefit is achieved].

In the example, ‘library_user’ is the actor, ‘search’ is an action, ‘books in the catalogue’ are the object, the ‘computer screen’ is the target, the performance requirements are ’with speed and ease of use’, the ‘provision of the book title’ is the trigger, ‘incompleteness’ is the constraint / qualifier and ‘finding books with similar titles’ is the value achieved. The user story becomes:-

As a library_user I want to search for books in the library catalogue on the computer screen with speed and ease-of-use when I provide a book title even if the book title is incomplete so that I can find all books with similar titles.

The order:

<ACTOR><ACTION><OBJECT><TARGET><PERFORMANCE><TRIGGER><CONSTRAINT><RATIONALE>

is recommended but there are other orders that are popular, for example Use Case descriptions often use the order:

<TRIGGER><ACTOR><ACTION><OBJECT><TARGET> <CONSTRAINT>

I provide a book title and begin a search for books in the library catalogue on the computer screen even if the book title is incomplete.

Benefits with patterns

Lego_Pattern

Many user stories are more complicated than this example and no one single order can suffice but without including these elements the possibility for misunderstanding is significantly increased.

Another benefit of this template is that it enables an estimate of functional size to be made.

Standardized structured user story expression enables comparisons across time, across teams, across projects, and across organisations. Developers, product owners, and their customers thus have a common way of communicating so that end user value can be delivered in an effective way.


[1] “Writing Effective Natural Language Requirements Specifications” by William M. Wilson


Previous post in this series:

Problems with User Stories and how to improve them – part 1

This was the final chapter on this blog on how to improve User Stories by applying patterns to capture all relevant information. Next time we will look further into patterns.

 

requirements · Uncategorized

Problems with User Stories and how to improve them – part 1

Introduction

In a recent Requirements Doctor Blog post, a contributor wrote “We do many short sprints in my project and do not need to spend useless time in the beginning on writing requirements that no one reads anyway”. While I mostly agree with this comment unfortunately it overlooks another, and often much larger, source of waste – and that is the amount of re-work that is done due to incorrect requirements. Again while I embrace the second principle of the Agile Manifesto “Welcome changing requirements, even late in development”, my experience is that the vast majority of change to requirements is a direct result of inconsistency and incompleteness in the original requirements expression.

I have used the word ‘requirement’ so far but for many people this conjures up the idea of ‘shall’ statements, for example “The system shall go fast”, whereas in an evolutionary approach such as Agile Development, User Stories are a more common starting point. Wikipedia provides the following definition of a User Story, “a user story is an informal, natural language description of one or more features of a software system”, and there, in the middle of the definition, is the Achilles Heel of User Stories; they use “natural language”.

This Achilles Heel is shared by many other development methods and so it is not a problem just for the Agile development paradigm, and the problem is brought about by the lack of adherence to the known good practices for requirements, (User Story), expression.

This is not the place to review the extensive literature on the issues of the use of natural language for expressing the needs and wants of users and customers.  Put simply, left to their own devices with no guidance, writers of User Stories consistently fail to express themselves correctly, consistently and completely, (the same is true of writers of requirements).

I shall leave the topic of completeness for a later discussion, but put succinctly, I believe it is not possible to have a complete set of user stories / requirements. What I would like to discuss in these articles is the topics of correctness and consistency of user stories / requirements, and more importantly how we might measure correctness and consistency so that when we change a User Story we can ensure that it has been improved.

Simon1

Correctness

So when is a natural language user story correct? Well it must follow the syntactic and semantic rules of the language it is expressed in. Natural language syntax is basically about word order and punctuation.

The three words ‘book’, ‘library’ and ‘a’ can be ordered in six ways but only three are syntactically correct, “A library book”, “A book library” “Book a library”; three are not syntactically correct “Library a book”, “Library book a” and “book Library a”.

Natural language semantics on the other hand is concerned with meaning, conveyed by the relationship between words. If we look at the three syntactically correct triads: in “A library book” the word ‘book’ refers to an object and ‘library’ describes a type of book; in “A book library” the meanings have swapped, the word ‘book’ is now the descriptor and ‘library’ is the object being described. Finally, “Book a library” changes the meaning of ‘book’ again, now ‘book’ is an action applied to the object ‘library’! (Book can mean make a reservation).

Consistency

So when is a natural language user story consistent? Simply when a word is used in a specific context then it must have a single meaning and no other word can have the same meaning. In our previous examples the work ‘book’ is used as a descriptor, as an object and as an action. In each of the triads the meaning of ‘book ’can be inferred by its position relative to the other words; the word ‘book’ is in three different contexts but only one can be correct, two must be inconsistent. Now consider this user story, ‘Book the book from the book library!’ Although this is syntactically and semantically correct I would postulate that this user story has a high potential for being misunderstood due to the inconsistent use of the word ‘book’.

So applying correctness and consistency rules to ‘Book the book from the book library’ I would suggest ‘Reserve the book from the library’ as an alternative that decreases the likelihood of wasteful work.

The final issue is how we might measure correctness and consistency. This is relatively simple, whenever a word or phrase breaks a syntactic or semantic rule then add one to the error count and whenever a word of phrase is used inconsistently then add one to the error count. Here are the scores for the various examples:

Simon_2

Spelling and grammar checkers can be used for syntactical checking, semantic checking is a little trickier but automated checkers do exist.

Next post in this series

In the next part of this discussion I will look at how patterns can help us get it right first time.

Happy reading!

About the author

simon.small
Simon Wright, Bsc. PhD

Simon’s experience has been gained in blue-chip companies, and public sector bodies, throughout the world and has a proven track record of helping organisations with their requirement documentation including stakeholder analysis, concepts of operation, requirements development, requirements management and architecture design. Recent clients include: The Russian Atomic Energy Authority – Moscow, The European Spallation Source – Lund, Novo Nordisk – Denmark and the Norwegian Army – Oslo. Simon truly appreciates the separate but related roles of tools, process and people when developing requirement documents. He understands that producing quality deliverables is dependent upon encouraging and sustaining a culture of engagement and commitment.