# An overview of acceptance criteria and their verification

## Testing vs verification

"Testing software" means different things for different people. For
some, it means "finding problems", and for others it means "making
sure it works as expected". Both of these can be very open-ended and
leave unclear when one has done enough testing. For Subplot we avoid
the term "testing" and instead have a different approach: Subplot is
for documenting acceptance criteria and how to verify the software or
system under test meets them. Subplot does not try to find ways of
finding new problems.

## Functional vs non-functional requirements

A functional requirement is concerned about what the software does,
where a non-functional one is about other aspects. A functional
requirement might be "the software adds numbers together correctly",
where a non-function one might be "the software outputs the result
using a pleasing font". A functional requirement can often be
expressed in hard numbers, where a non-functional one sometimes can't.
  
## Automated vs manual verification

The ideal is to have an automated way to verify that an acceptance
criterion is met. Automation tends to lead to fewer mistakes in the
verification: computers are really good at doing the same thing
repeatedly, whereas humans are really bad at it, and get tired,
distracted, and make mistakes.

However, all verification can't be automated. For example, what is a
pleasing font? It's possible to automate the verification that a
specific font is used, but not that it is pleasing. Worse, what is
pleasing varies from person to person, and from situation to
situation.

## Direct vs indirect verification

The best kind of verification is direct: if the acceptance criterion
says that foo must contain bar, then the verification should check
that foo does, indeed, contain bar and nothing else.

Sometimes direct verification isn't feasible: either foo is really
hard to inspect, or it takes too long, or for some other reason. Then
an indirect verification can be used. If we can't see everything
that's inside of foo, but we can check if there is a specific letter
inside, we can do an indirect verification that foo contains the
letters b, a, and r. This doesn't verify that foo contains bar: it
might contain arb instead. Also, since there are a nearly infinite
number of possible letter, maybe foo actually contains xyzzy.

How directly we verify depends on a variety of things, most
importantly how much time and effort we can put into creating the
verification method, or into executing it.

Also, it matters what the consequences are if the verification falsely
accepts a faulty system. If the health or life of someone is
concerned, the verification must be done much more carefully.


# Simple example project

  - discuss how to use Subplot for some simple, but not simplistic,
    software project
  - discuss different kinds of stakeholders a project may have

# Authoring Subplot documents

  - discuss that it may be necessary to have several documents for
    different audiences, at different levels of abstraction (cf. the
    FOSDEM safety devroom talk)
  - discuss writing style to target all the different stakeholders
  - discuss mechanics and syntax
    - Markdown and supported features
    - scenarios, embedded files, examples
    - bindings
    - step implementations in various languages
    - embedded markup for diagrams
    - running docgen

# Extended example project

  - discuss how to use Subplot for a significant project, but keep it
    sufficiently high level that it doesn't get too long and tedious
    to read

# Appendix: Implementing scenario steps in Bash

  - this appendix will explain how to implement scenario steps using
    the Bash shell

# Appendix: Implementing scenario steps in Python

  - this appendix will explain how to implement scenario steps using
    the Python language

# Appendix: Implementing scenario steps in Rust

  - this appendix will explain how to implement scenario steps using
    the Rust language


# Appendix: Scenario

This is currently necessary so that codegen won't barf.

~~~scenario
when I run true
~~~
