Testing is an activity to assess product quality and improve it by identifying defects and problems. It consists of the dynamic verification of the behavior of a program over a finite set of cases selected from an execution domain test (which is usually infinite), related to the expected behavior, measuring the functional and non-functional system in terms of defects found (“We showed the presence of errors but not their absence [Dijkstra]”).

Within the activity of testing are put on trial two aspects of software quality: validation (“Did you build the right thing?”: Are we responding to the real needs of the user?) and verification (“did you build the thing right ? ” there were requirements, but we need to verify that the product is compliant with its specification). If we purely talk about the finished code, the testing would be using just for verification.

The V-model (ITIL-Foundation) is a popular method that shows different levels of testing surrounding the SDLC general common activities:

If we talk about another set of capabilities that must deliver the product generated along a development cycle, we will expand a little further, because we must comply capabilities that architecture must meet and verification Concerns nonfunctional within it:

• Example 1: We use a range of architecture assessment methods as ATAM, ARID, QAW, SAAM, which validate through scenarios or through concerns with the hypothesis that the architecture is capable to solve skills which is willing, through functional and non-functional scenarios prepared or through validation of their quality attributes.
• Example 2: We use testing to validate the usage of specific quality attributes, such penetration methods to validate a subset of system of security definitions, or we use testing to validate the expected response times for a system. Depending on the model used, this type of testing is included within the level called “integration testing”.
• Example 3: We use testing to measure the level of adherence to a reference model based on metrics that static code analysis can give us about the product built incrementally (Structural Testing).
• Example 4: We use testing to measure “the level of testing”, for example using coverage testing methods and mocking.
• Example 5: Inside modular designs, we use testing to validate low-level structures first and we require implementing test methods at a high level for test the entire system, usually pursuing policies that prove that development unit is suitable to function as a whole and not to attempt to justify what takes built for each component.

Consider not to let the testing to the end, since there is a premature effort that can manifest itself, without having to wait out an entire development cycle to correct aspects of the software that may become apparent in early stages of development. Therefore, it is essential that the architect thinks in driving products include capabilities that allow being testable, including components to assess comprehensively acceptance in the process and integrate the phases of the development cycle used.

Similarly create and have the infrastructure to help developers to create more testable code, approaching each time the software is self-check developed without requiring human intervention to do so. A high level of automation requires the creation of good tests, and is summarized by the acronym FIRST: F (Fast: quick to run), I (Independent: without Preconditions), R (Repeatable: not Depending of external factors), S (Self Checking: determined by its own if has passed or rejected), T (Timely) –

Design reviews and code walk-throughs and inspections are other way to find defects in the product developed. Joined with the use of checklists, they allow finding defects in the implementation, to deliver a product with a lower percentage of defects to validate.

The application of testing involves a cost that the project should assume, but in early stages, it is lower than letting it for last phases. A testing may not be exhaustive and the architect or designer must make a balance between the levels of testing to reach without exceeding the cost level raised for the project, which will cover the technical debt.

Several critical receives software development because, depending on the time of delivery of the project, there are units of software that are not tested with acceptable standards of quality. Otherwise, the product does not deliver what it had actually meet within the time constraint, getting the stage of testing breaching the expected levels of quality for the product.

An architect must design solutions that fits inside proven static and dynamic testing practices, considering the scale and type of software delivered to cover functional and non-functional requirements of the solutions. Inside the constraints of time and budget around Software Development Lifecycle (SDLC) of the project, product must cover the software quality expectations with agility and easy to use practices to find defects.

Why does an architect need this skill?

Architect provides the link between Business needs and the IT method to include the ability to verify its right implementation.

Common tasks involved in this skill?

• Quantify the degree of conformityof a developed product against the required software quality attributes
• Test reference archtiectures to achieve the right product objectives using proven evidence through testing.
• Suggest tools, process and techniques at the right phase of lifecycle to test language classes, components or techniques.
• Help to automate testings that will be useful inside the testing cycle of the project
• Evaluate the defined architecture using an evaluation methodooogy
• Testability included in the product to be developed preventing problems in integration and product evolution
• Advise including testing methods development cycle, with its corresponding automation starategy
• Coordinate testing practices to give feedback about the concerns of the product
• Evolve candidate architectures as you iteratively test and improve your architecture
• Involved in quality practices initiatives

What is their ownership in this skill?

It depends of the size of the project. For small teams the architect and technical leader are responsible to implement the testing strategy to use by the team, while large products may include

Note that the responsibility to make the automated test are the technical group and the QA group verifies its conformity with the product requirements.

How an architect would use those skills in daily activities?

This can help to ensure that the expected quality of the product be achieve and the main concerns of the architecture could be verified, considering the operation in real life. Not doing so can lead to problems involving monetary losses or reputation of the business.

Describe why an architect should be involved in this skill at a corporate level

TBD

Primary push back and/or challenges for architects

TBD

How would a stakeholder engage an architect for assistance utilizing this skill?

TBD

Sub-capabilities are divided according ISO-IEC organizational process as reference:

Test Fundamentals

Capabilities to guide a testing process inside a project, understanding the overall testing lifecycle. Testing activities are not only running testing; those activities incudes planning and control, design, check the results, reporting, reviews.

Test Management, Monitoring and Control

Experience integrating testing practices within the development of projects around the organization

Dynamic Test Process and Automation

Experience integrating testing practices within the development of projects around the organization.

Test Improvement and Financials

Related with the capability to apply testing techniques in each of the projects of the company, and ensuring that a testability level is achieved for the proposed architecture in financial terms.

Articles:

• IASA BTABoK

https://btabok.iasaglobal.org

• Software Architecture.

http://carlosreynoso.com.ar/archivos/arquitectura/Arquitectura-software.pdf

• Testing introduction

http://www.fceia.unr.edu.ar/ingsoft/testing-intro-a.pdf

• About agile architecture and C4 view/modeling

http://www.codingthearchitecture.com/

• About TDD Practice, pro’s and con’s

http://david.heinemeierhansson.com/2014/tdd-is-dead-long-live-testing.html

• About Checklists and code review PSP practices

http://blog.fogcreek.com/increase-defect-detection-with-our-code-review-checklist-example/

• About micro-services, determinism, testing culture and testing maintenance

http://martinfowler.com/articles/microservice-testing/

http://martinfowler.com/articles/nonDeterminism.html

http://martinfowler.com/articles/testing-culture.html

http://martinfowler.com/bliki/TestPyramid.html

• Apply the appropriate testing method according the project scope

http://context-driven-testing.com/

• Test architect

https://blogs.oracle.com/johnmorrison/entry/test_architect

• BI and DW testing

https://msdn.microsoft.com/en-us/library/gg248101.aspx

• Ministry of testing

http://www.ministryoftesting.com

http://www.ministryoftesting.com/wp-content/uploads/2013/06/TTP8.pdf

Blogs/Webcasts/News/Reference Resources:

• SEI Quality Attribute Workshop and its ATAM relationship

http://www.sei.cmu.edu/reports/03tr016.pdf

• Software architecture in practice (3^rd)

http://www.amazon.com/Software-Architecture-Practice-Edition-Engineering/dp/0321815734

• Engineering Long-Lasting Software: An Agile Approach Using SaaS and Cloud Computing, Beta Edition

http://amzn.com/0984881212

• Software testing adventure reference poster

https://sites.google.com/site/swtestingmap/stadventure

Training/Standards/Certifications:

• Carnegie Mellon University / Personal software process PSP/TSP certification

http://www.sei.cmu.edu/reports/09sr018.pdf

http://seicertification.clearmodel.com/get-certified/software-development/

• ISO-9126 & ISO-29119 software testing

http://www.softwaretestingstandard.org/

• International Software Testing Qualifications Board (ISQTB)

http://www.istqb.org/

Andres Hurtado
Software Architect & Developer

Software architect and developer working inside private and public sectors. He has worked in Colombia, designing and implementing enterprise solutions and has over 14 years of experience in software development process, enterprise application integration and research about technologies to leverage the product development.