Intellectual Property

Who owns your code?

• You wrote some code!
  • All week!
  • Both systems and tests!
• A key question:
  • Who owns that code?
    • Or different bits of it?
  • What kind of ownership?

Intellectual Property (IP)

Intellectual property is any articulable, tangible production of a mind whose physical realisations are restricted by law (in production, distribution, etc.)

• We don't control what other people think!
• We can control what they do with certain thoughts.
• Intellectual Property rights give power to certain people to control what other people do
  • For example, whether they can distribute a book, song, or program

Kinds of Intellectual Property

Copyright

Copyright is a licensable monopoly of tangible expression of an idea with respect to reproduction, derivation, display, distribution, and the like.

• Protects the expression not the idea
  • Though these blur at the limit
    • Some plagiarism is a copyright violation; some is not
• Typically automatically assigned at creation time
  • No "notice" or "registration" needed
    • Though these might help with lawsuits

Patents

A patent is a licensable monopoly of the use or sale of a "non-obvious" invention (of a process, machine, design (sometimes), mechanism, procedure, etc.

• A patent is an incentive to disclose
  • Many patentable inventions could be exploited "secretly"
  • Goal is to add to our common knowledge
• Prior art destroys a patent
  • Including your own
• Defensive patenting "common"
• Independent invention no defense

Trade Secret

A trade secret is an invention which is not disclosed

• Persists forever
  • Unless leaked
  • Or reinvented
• Typically protected by secrecy
  • Or specific contracts
    • "Non-Disclosure Agreements" (NDAs)

Who owns your code?

• Copyright starts with the creator
  • I.e., you!
  • Cheap! (Even to register)
  • Unless you create it as work-for-hire
    • Or otherwise transfer it
• Patents belong to the patenter
  • Expensive(ish) to secure
• Trade secrets belong to the inventor

Are you working for hire?

• Not quite!

What to keep in mind (now)

• Software engineers typically produce IP
  • Even if not protected, our output is "intellectual"
  • Various forms of IP drive
    • product value
    • employee/entrepreneur value
• Software engineers typically use IP
  • All sorts and in all ways
  • IP considerations a constraint on the design space

Comprehending Product Qualities

Comprehension?

• We can distinguish two forms:
  • Know-that
    • You believe a true claim about the software
    • ...with appropriate evidence
  • Know-how
    • You have a competancy with respect to the software
    • E.g., you know-how to recompile it for a different platform
• They are interrelated
• Both require significant effort!

Quality Levels

• We talked about different kinds of quality
  • But for each kind there can be degrees or levels thereof
  • "Easy" example: High vs. Low performance
• Most qualities in principle are quantifiable
  • Most things are quantifiable in some sense
• But reasonable quantification isn't always possible
  • Or worth it
  • Being clear about your vagueness is essential!

Clarity

Our discussion will be adequate if it has as much clearness as the subject-matter admits of, for precision is not to be sought for alike in all discussions, any more than in all the products of the crafts...for it is the mark of an educated [person] to look for precision in each class of things just so far as the nature of the subject admits...
— Aristotle, Nicomachaen Ethics, Book 1, 3

Clarity (2)

We demand rigidly defined areas of doubt and uncertainty!
— Douglas Adams, The Hitchhiker's Guide to the Galaxy

Defects as Quality Lacks

A defect in a software system is a quality level (for some quality) that is not acceptable.

• Quality levels need to be elicited and negotiated
  • All parties must agree on
    • what they are,
    • their operational definition
    • their significance

What counts as a defect is often determined late in the game!

Question

If your program crashes then it

1. definitely has a bug.
2. is highly likely to have a bug.
3. may or may not have a bug.

Question

If your program crashes, and the cause is in your code, then it

1. definitely has a bug.
2. is highly likely to have a bug.
3. may or may not have a bug.

Bug or Feature?

(Does QA hate you? — scroll for the cartoons as well as the wisdom.)

• Even a crashing code path can be a feature!
• Contention arises when the stakes are high
  • and sometime the stakes can seem high to some people!
  • defect rectification costs the same
    • whether the defect is detected...
    • ...or a feature is redefined
• Defects (even redefined features) aren't personal

Problem Definition

This is a logical, not temporal, order.

Problem Definition

The penalty for failing to define the problem is that you can waste a lot of time solving the wrong problem. This is a double-barreled penalty because you also don't solve the right problem.
—McConnell, 3.3

Quality Assurance

• Defect Avoidance or Prevention
  • "Prerequisite" work can help
    • Requirement negotiation
    • Design
    • Tech choice
  • Methodology
• Defect Detection & Rectification
  • If a defect exists,
    • Find it
    • Fix it

The Points of Quality

1. Defect prevention
   • Design care, code reviews, etc.
2. Defect appraisal
   • Detection, triaging, etc.
3. Internal rectification
   • We fix/mitigate before shipping
4. External rectification
   • We cope after shipping

Defect Detection Techniques

Defect Detection Techniques

Experiencing Software

• It's one to know that there are bugs
  • All software has bugs!
• It's another to be able to trigger a bug
  • Not just a specific bug!
  • If you understand the software
    • You know how to break it.
• Similarly, for making changes
  • tweaks, extensions, adaptions, etc.
• The more command, the more modalities of mastery

Forms of Knowledge (Manifestations)

• Human interpretable
  • Comments, design docs, user stories, javadoc
  • Source code
    • Both, a written description and a "live" object
    • Also things like demo code, examples, test suites, ec.
  • Diagrams
    • "Mere" pictures to semi-formal to formal diagrams: ER docs, UML, etc.
• Formal specifications
• Competencies
  • I can make it crash

Sources of Knowledge (Modalities)

• Analytical knowledge
  • Derived from inspection and reasoning
  • Can be automated using formal methods
• Experimental knowledge
  • Derived from the conduct of experiments
  • Typically tests
• Experiential knowledge
  • Derived from personal interaction with the software
  • Strong "know-how" component

Coursework Recap

Coursework Activities

• Reading
• Q1
  • Mostly related to reading
  • Mostly "Recall"...with some interpretation
    • They will go higher on the Bloom taxonomy!
• SE1
  • Reading and analysing
• CW1
  • Reverse engineered a specification
  • Reengineered miniwc from the spec
    • Program construction

A note on marks

• UK marks run from 0-100%
  • <=49 = Failing (<40% serious failure)
  • 50-59 = Pass
  • 60-69 = Merit
  • over 70 = Distinction
  • NOTE THE WIDE BAND AT THE TOP AND BOTTOM
• A 65% is a good mark
• An 85% is exceedingly rare
• Over 70% is fairly rare

Q1

• Mean of 3.71 (74%)
  • This is unusual for MCQs
• We will do some "in exam conditions"
• Let's delve

Simplified Problem

• This was a small problem
  • With clear boundaries
• Even here:
  • We ended up with support programs
    • And corners cut
• Software engineering is (complex) system engineering
  • On both the product and project sides
  • We use a complex infrastructure!

Challenges

What were the challenges you encountered?

What challenges were inherent to the problem?

What challenges were environmental?

FizzBuzz and Golf!

• Will have some more data soon!
• But we have a winner!

Testing Correctness

Beware of bugs in the above code; I have only proved it correct, not tried it.
— Don Knuth, Figure 6: page 5 of classroom note

Really Beware of Bugs!

—Grace Hopper's Bug Report

Developer Testing

• We can distinguish between
  • Testing done by non-specialists (McConnell: "Developer testing")
    • For many projects, the only sort!
  • Testing done by (test) specialists
• If you compile and run your code
  • Then you've done a test! (or maybe two!)
    • If only a "smoke" test

Testing is inescapable; good testing takes work

Question time!

• If you compile your code
  • you have tested it for syntactic correctness.
  • you have tested it for semantic correctness.
  • you have tested it for both.
  • you haven't tested it at all.

What is a test?

A test case is a repeatable execution situation of a software system that produces recordable outcomes. A test is a particular attempt of a test case

• The outcomes may be expected (i.e., specified in advance)
  • E.g., we expect passing 1+1 to a calculator to return 2
  • Generally boolean outcomes (pass or fail)
    • We might have an error that prevents completion
• The outcomes may be measurement results
  • E.g., we want to find the time it takes to compute 1+1

What is a test? (2)

A test case is a repeatable execution situation of a software system that produces recordable outcomes. A test is a particular attempt of a test case

• The outcomes should testify to some software quality
  • E.g., correctness, but also efficiency, usability, etc.
• A (single) test specifies a very particular quality
  • E.g., correct for a given input
  • E.g., uses X amount of memory for this scenario

The fundamental challenge of testing is generalisability

Generalisability Problem (1)

Testing shows the presence, not the absence of bugs.
—Edsger W. Dijkstra, Nato Software Engineering Conference, pg 16 1969

Terminology note

• Test and test case are often used interchangeably
  • And in other loose ways
  • Most of the time it doesn't matter because easy to distinguish
• We often talk about a test suite or test set
  • But this also might be subordinated to a test
  • For example,

    "We used the following test suite to stress test our application".

Anatomy of a Test (1)

Generalisability Threat

• A test case (A):
  • Goal: Correctness to the specification
    • Input: a pair of integers, X and Y
    • Output: the integer that is their sum
  • Test Input:
    • X=1 and Y=1
  • Expected output:
    • 2
• Test result of System S is pass
  • What can we conclude?

Question time!

• From the test result Pass test case A, we can conclude that:
  1. System S correctly implements the specification.
  2. System S correctly implements the specification for this input
  3. Both 1 and 2
  4. Neither 1 nor 2

Question time!

• From the test result Fail test case A, we can conclude that:
  1. System S does not correctly implement the specification.
  2. System S does not correctly implement the specification for this input
  3. Both 1 and 2
  4. Neither 1 nor 2

Anatomy of a Test (2)

Environment Matters

The next most significant subset of [Modification Requests (MRs)] were those that concern testing (the testing environment and testing categories)—24.8% of the MRs. ...it is not surprising that a significant number of problems are encountered in testing a large and complex real-time system...First, the testing environment itself is a large and complex system that must be tested. Second, as the real-time system evolves, so must the laboratory test environment evolve.
Making Software, pg. 459.

A Good Test

• A good test case is
  • part of a suite of test cases
  • understandable
    • i.e., you can relate it
      • to the spec
      • to the system behavior
  • fits in with the test environment
  • is (given the suite) informative

Environment Matters

A Classification of Tests

A Classification of Tests

• Based on a 5W+H approach by Ray Sinnema (archived)
  • Who (Programmer vs. customer vs. manager vs...)
  • What (Correctness vs. Performance vs. Useability vs...)
  • When (Before writing code or after)
    • Or even before architecting!
  • Where (Unit vs. Component vs. Integration vs. System)
    • Or lab vs. field
  • Why (Verification vs. specification vs. design)
  • How (Manual vs. automated)
    • On demand vs. continuous

Who?

• Sinnema: Tests give confidence in the system
  • I.e., they are evidence of a quality
  • Who is getting the evidence?
    • Users? Tests focus on external qualities
      • Can I accept this software?
    • Programmers? Tests focus on internal qualities
      • Can I check in this code?
    • Managers? Both?
      • Are we ready to release
• But also, who is writing the test?
  • A bug report is a (typically partial) test case!

What?

• Which qualities am I trying to show?
  • Internal vs. external
  • Functional vs. non-functional?
  • Most developer testing is functional (i.e., correctness)
    • And at the unit level
    • Does this class behave as designed

When?

• When is the test written?
  • Before the code is written?
  • After the code is written?
• Perhaps a better distinction
  • Tests written with existing code/design in mind
  • Test written without regard for existing code/design
  • This is related to white vs. black box testing
    • Main difference is whether you respect the existing API

Where?

• Unit
  • Smallest "chunk" of coherent code
  • Method, routine, sometimes a class
  • McConnell: "the execution of a complete class, routine, or small program that has been written by a single programmer or team of programmers, which is tested in isolation from the more complete system"
• Component (McConnell specific, I think)
  • "work of multiple programmers or programming teams" and in isolation

Where? (ctnd)

• Integration
  • Testing the interaction of two or more units/components
• System
  • Testing the system as a whole
  • In the lab
    • I.e., in a controlled setting
  • In the field
    • I.e., in "natural", uncontrolled settings

Where? (ctnd encore)

• Regression
  • A bit of a funny one
  • Backward looking and change oriented
    • Ensure a change hasn't broken anything
    • Esp previous fixes.

Why?

• Three big reasons
  1. Verification (or validation)
     • Does the system possess a quality to a certain degree?
  2. Design
     • Impose constraints on the design space
       • Both structure and function
  3. Comprehension
     • How does the system work?
       • Reverse engineering
     • How do I work with the system?

How?

• Manual
  • Typically interactive
    • Human intervation for more than initiation
  • Expectations flexible
• Automated
  • The test executes and evaluates on initiation
  • Automatically run (i.e., continuously)

Program equivalence

Many Equivalences

• Source code equivalent
  • Character equivalent
  • AST equivalent
  • Non-comment/names AST equivalent
• Translation equivalent
  • E.g., after compilation
• All-behavior equivalent
• Bisimilar
• Functionally equivalent

Functionally Equivalent

Two programs are functionally equivalent just in case they implement exactly the same functionality.

• Functionality is typically characterised by "Input-Output" behaviour
  • Internal structure doesn't matter
    • Programming language, algorithm, etc.
      • FEPs can differ "solely" in execution paths
  • There can be behaviour differences (e.g., performance!)
• Strong but not maximally strong!

What behaviour is "functionality"?

The functionality of a software system is the required behaviour of the system

Not ideal, as non-functional behaviour may be required

The functionality of a software system is the behaviour of the system that performs some user task

In either cause, the functionality is a subset of all behaviour

Functionality equivalent (reprise)

Two programs are functionally equivalent just in case they implement exactly the same functionality

The functionality of a program are those behaviours which performs a user task

Functionality may be changing, unknown, or misunderstood

The set of functionally equivalent programs depends on the functionality parameter

FizzBuzz Example

Compare a "normal" FizzBuzz solution with a golf version:

Functionally equivalent

Given a set of required functionalities F, and two systems, S1 & S2, S1 is functionality equivalent (with respect to F) to S2 if S1 and S2 enact F.

So, if two programs are behaviourly equivalent then they are functionally equivalent.

What happens if S1 and S2 don't quite enact the same F?

"Sufficiently" functionally equivalent

Given a set of required functionalities F, and two systems, S1 & S2, which enact functionality sets F1 & F2 (respectively), where, F1 ≠ F2 ≠ F, S1 is sufficiently functionally equivalent to S2 wrt F if F1 and F2 share "enough" of an overlap with F.

Obviously, "enough" is a parameter!

wc example!

• GNU wc has more functionality (and user-notable behaviour) than miniwc.py
  • Or other wcs!
  • Different flag options, find longest line, etc.
• Some behaviour is user visible but not "functional" (or interesting)
  • wc --help --version vs. wc --version --help
    • Non-equivalent in GNU wc
    • Do we care to preserve this?!
• How about spacing?!

wc example!

• GNU wc has more functionality (and user-notable behaviour) than miniwc.py
  • Or other wcs!
  • Different flag options, find longest line, etc.
• Some behaviour is user visible but not "functional" (or interesting)
  • wc --help --version vs. wc --version --help
    • Non-equivalent in GNU wc
    • Do we care to preserve this?!
• How about spacing?!