General Testing Principles

Software testing is an extremely creative and intellectually challenging task. The following are some important principles that should be kept in mind while carrying software testing.

General Testing Principles

• Testing should be based on user requirements

Testing is in order to uncover any defects that might cause the program or system to fail to meet the client’s requirements

• Testing time and resources are limited

Avoid redundant tests

• It is impossible to test everything

Exhaustive tests of all possible scenarios are impossible, simple because of the many different variables affecting the system and the number of paths a program flow might take

• Use effective resources to test

This represents use of the most suitable tools, procedures and individuals to conduct the tests. The test team should use tools that they are confident and familiar with testing procedures should be clearly defined. Testing personnel may be a technical group of people independent of the developers.

• Test planning should be done early.

This is because test planning can begin independently of coding and as soon as the client requirements are set.

• Test for invalid and unexpected input conditions as well as valid conditions.

The program should generate correct messages when an invalid test is encountered and should generate correct results when the test in valid.

• The probability of the existence of more errors in a module or group of modules to directly proportional to the number of errors already found.

• Testing should begins at the module

The focus of testing should be concentrated on the smallest programming units first and then expand to other parts of the system.

• Testing must be done by an independent party.

Testing should not be performed by the person or team that developed the software since they tend to defend the correctness of the program.

• Assign best personnel to the task.

Because testing requires high creativity and responsibility only the best personnel must be assigned to design, implement, and analyze test cases, test data and test results.

• Testing should not be planned under the implicit assumption that no errors will be found.

Testing is the process of executing software with the intent of finding errors.

• Keep software static during test.

The program must not be modified during the implementation of the set of designed test cases.

• Document test cases and test results.

• Provide expected test results if possible.

A necessary part of test documentation is the specification of expected results, even if providing such results is impractical.

STLC, Phases & Deliverables

• STLC – Introduction

Software testing life cycle identifies what test activities to carry out and when (what is the best time) to accomplish those test activities. Even though testing differs between organizations, there is a testing life cycle.

Basic requirements in software testing life cycle is to control/deal with software testing – Manually and through Automation.

• Phases of STLC

There are 5 basic phase of STLC.

Analysis and Planning - All requirements are analyzed. To freeze the scope of the testing, a high level Test Plan is then prepared.

Test Designing - (Test Scenarios, Test Cases and / or Test Data) is prepared on the basis of the requirements.

Testing - A walk-through of the application is done for Sanity check once the code is freeze and given to QA. If everything is OK, then actual Testing starts by executing the Test Cases.

Filing bugs and regression - Bugs are filed and Retesting and Regression of bugs is done once the bugs are fixed.

Prepare a closure report- Closure Report is prepared if everything goes well.

• Deliverables of STLC

Also known as Testing Deliverables or Artifacts.

Software testing process can produce several artifacts.

Test plan
A test specification is called a test plan. The developers are well aware what test plans will be executed and this information is made available to the developers. This makes the developers more cautious when developing their code. This ensures that the developer’s code is not passed through any surprise test case or test plans.

Test case
A test case in software engineering normally consists of a unique identifier, requirement references from a design specification, preconditions, events, a series of steps (also known as actions) to follow, input, output, expected result, and actual result. It can occasionally be a series of steps (but often steps are contained in a separate test procedure that can be exercised against multiple test cases, as a matter of economy) but with one expected result or expected outcome. The optional fields are a test case ID, test step or order of execution number, related requirement(s), depth, test category, author, and check boxes for whether the test is automatable and has been automated. Larger test cases may also contain prerequisite states or steps, and descriptions. A test case should also contain a place for the actual result. These steps can be stored in a word processor document, spreadsheet, database, or other common repository. In a database system, you may also be able to see past test results and who generated the results and the system configuration used to generate those results. These past results would usually be stored in a separate table.

Test script
The test script is the combination of a test case, test procedure, and test data. Initially the term was derived from the product of work created by automated regression test tools. Today, test scripts can be manual, automated, or a combination of both.

Test data
The most common test manually or in automation is retesting and regression testing. In most cases, multiple sets of values or data are used to test the same functionality of a particular feature. All the test values and changeable environmental components are collected in separate files and stored as test data. It is also useful to provide this data to the client and with the product or a project.

Test suite
The most common term for a collection of test cases is a test suite. The test suite often also contains more detailed instructions or goals for each collection of test cases. It definitely contains a section where the tester identifies the system configuration used during testing. A group of test cases may also contain prerequisite states or steps, and descriptions of the following tests.

Test harness
The software, tools, samples of data input and output, and configurations are all referred to collectively as a test harness.

The complete STLC, its phases and Deliverables explained in this article is in short.
These are covered more briefly in their respective chapters.

Manual & Automation

There are two ways of testing – MANUAL & AUTOMATION.

Manual testing is the process of manually testing software for defects. It requires a tester to play the role of an end user, and use most of all features of the application to ensure correct behavior. To ensure completeness of testing, the tester often follows a written test plan that leads them through a set of important test cases.

Test automation is the technique of testing software using software rather than people. A test program is written that exercises the software and identifies its defects. These test programs may be written from scratch, or they may be written utilizing a generic Test automation framework that can be purchased from a third party vendor. Test automation can be used to automate the sometimes menial and time consuming task of following the steps of a use case and reporting the results.

Here are the comparisons between the two ways:

Pros of Automation:
If you have to run a set of tests repeatedly, automation is a huge win for you
It gives you the ability to run automation against code that frequently changes to catch regressions in a timely manner
It gives you the ability to run automation in mainstream scenarios to catch regressions in a timely manner
Aids in testing a large test matrix (different languages on different OS platforms). Automated tests can be run at the same time on different machines, whereas the manual tests would have to be run sequentially.

Cons of Automation:
It costs more to automate. Writing the test cases and writing or configuring the automate framework you’re using costs more initially than running the test manually.
Can’t automate visual references, for example, if you can’t tell the font color via code or the automation tool, it is a manual test.

Pros of Manual:
If the test case only runs twice a coding milestone, it most likely should be a manual test. Less cost than automating it.
It allows the tester to perform more ad-hoc (random testing). In my experiences, more bugs are found via ad-hoc than via automation. And, the more time a tester spends playing with the feature, the greater the odds of finding real user bugs.

Cons of Manual:
Running tests manually can be very time consuming
Each time there is a new build, the tester must rerun all required tests - which after a while would become very mundane and tiresome.

Causes of Bugs

Numerous studies have been performed on very small to extremely large projects and the results are always the same. The number one cause of software bugs is the specification.

There are several reasons specifications are the largest bug producer. In many instances a spec simply isn't written. Other reasons may be that the spec isn't thorough enough, it's constantly changing, or it's not communicated well to the entire development team. Planning software is vitally important. If it's not done correctly, bugs will be created.

The next largest source of bugs is the design. This is where the programmers lay out their plan for the software. Compare it to an architect creating the blueprints for a building. Bugs occur here for the same reason they occur in the specification. It's rushed, changed, or not well communicated.

NOTE
There's an old saying, "If you can't say it, you can't do it." This applies perfectly to software development and testing.

Coding errors may be more familiar to you if you're a programmer. Typically, these can be traced to the software's complexity, poor documentation (especially in code that's being updated or revised), schedule pressure, or just plain dumb mistakes. It's important to note that many bugs that appear on the surface to be programming errors can really be traced to specification and design errors. It's quite common to hear a programmer say, "Oh, so that's what it's supposed to do. If somebody had just told me that I wouldn't have written the code that way."

The other category is the catch-all for what's left. Some bugs can be blamed on false positives, conditions that were thought to be bugs but really weren't. There may be duplicate bugs, multiple ones that resulted from the same root cause. Some bugs can also be traced to testing errors. In the end, these bugs (or what once were thought of as bugs) turn out to not be bugs at all and make up a very small percentage of all the bugs reported.

Error, Faults & Failures

Errors, Faults and Failures, these three terms are often very confusing.
Above diagram will help you in understanding the difference between them.
They are defined as follows:

Error: discrepancy between a computed, observed or measured value or condition and the true, specified or theoretically correct value or condition.

Faults (defect): abnormal condition that may cause a reduction in, or loss of, the capability of a functional unit to perform a required function.

Failure: The inability of a system to perform its required functions within specified performance requirements

Fundamental of Testing

Lets start from the very basic thing by introducing ourselves to testing. So, the question arises is:

• What is Testing?

In general, testing is finding out how well something works. Testing is commonly assumed to mean executing software and finding errors. In terms of human beings, testing tells what level of knowledge or skill has been acquired. In computer hardware and software development, testing is used at key checkpoints in the overall process to determine whether objectives are being met. For example, in software development, product objectives are sometimes tested by product user representatives. When the design is complete, coding follows and the finished code is then tested at the unit or module level by each programmer; at the component level by the group of programmers involved; and at the system level when all components are combined together. At early or late stages, a product or service may also be tested for usability.

At the system level, the manufacturer or independent reviewer may subject a product or service to one or more performance tests, possibly using one or more benchmarks. Whether viewed as a product or a service or both, a Web site can also be tested in various ways - by observing user experiences, by asking questions of users, by timing the flow through specific usage scenarios, and by comparing it with other sites.

• Why do we need Testing?

This is a right question because, prior to the concept of TESTING software as a ‘Testing Project’, the testing process existed, but the developer(s) did that at the time of development.

But you must know the fact that, if you make something, you hardly feel that there can be something wrong with what you have developed. It's a common trait of human nature, we feel that there is no problem in our designed system as we have developed it and it is perfectly functional and fully working. So the hidden bugs or errors or problems of the system remain hidden and they raise their head when the system goes into production.

On the other hand, its a fact that, when one person starts checking something which is made by some other person, there are 99% chances that checker/observer will find some problem with the system (even if the problem is with some spelling that by mistake has been written in wrong way.). Really weird, isn't it? But that’s a truth!

Even though its wrong in terms of human behavior, this thing has been used for the benefit of software projects (or you may say, any type of project)। When you develop something, you give it to get checked (TEST) and to find out any problem, which never aroused while development of the system. Because, after all, if you could minimize the problems with the system you developed, it’s beneficial for yourself. Your client will be happy if your system works without any problem and will generate more revenues for you.

BINGO, it's really great, isn't it? That's why we need testing!

• Testing What?

Application that needs to be tested can be of many types. Few of them are:

1. Web Application
2. Client Server
3. Desktop Application

Content of CourseWare

Complete CourseWare is divided into sections.

First section, FUNDAMENTAL TESTING, focuses on testing definition, why do we need testing, types of testing, testing methods, what are the causes of bugs, bug life cycle, bugs content, manual and automation testing, SQA (Software Quality Analyst, Software Quality Assurance), Quality Control, Testing techniques covering WBT, BBT, Gray Box testing and other testing concepts like Reviews, Walkthroughs, Inspection, Verification and Validation.

Second section, QUALITY CONCEPT, In this section, we tried to focus on Quality. Software quality is the “Conformance to explicit stated functional and performance requirements, explicitly documented development standards, and implicit characteristics that are expected of all professionally developed software.

Two kinds of quality may be encountered: Quality of design and quality of conformance. Quality of design refers to the characteristics that designers specify for an item. Quality of conformance is the degree to which the design specifications are followed during manufacturing.

This section also focuses on various quality standards like CMM, CMMI, ISO

Third section, STLC,
STLC is also known as Software Testing Life Cycle. Software testing life cycle identifies what test activities to carry out and when (what is the best time) to accomplish those test activities. This topic will be covered in this section along with example, deliverables that are outcomes of the various phases of testing life cycle.

This will also cover various testing approaches that are followed in software testing.

Fourth section is AUTOMATION TESTING:
Test automation is the use of software to control the execution of tests, the comparison of actual outcomes to predicted outcomes, the setting up of test preconditions, and other test control and test reporting functions. This topic will be covered in this section along with Test Tool Categories and various testing frameworks.

Fifth section, SDLC covers software development life cycle (SDLC), phases of SDLC, various matrix that are prepared in different phases and various SDLC models
This section also co-relates the various types of testing performed in different phases of Software Development Life Cycle.

Sixth section, RISK & PROJECT MANAGEMENT:
This section covers an overview of risk and project, how to manage risk, steps that are followed to manage risk. It also covers the management of project detailing the concept from project initiation phase to completion of the project.


Guys, don't worry if u can't find everything, its under progress and will soon be available for all of us.

Introduction

This doc will guide new joiners through step-by-step process of testing. Even people who are interested in testing will get good understanding of the concept by going through it.

This CourseWare defines the training and qualification requirements for understanding the basic concept of Project Management, Software Engineering, Software Testing, Risk Management and different types of metrics and deliverables that help in improving the Quality of the Process and Product.

This is designed in a way that gives the fundamental knowledge of testing and then leading the reader in more depths of testing, covering the quality standard, SDLC and other related topics and then finally covering project and risk management.

This article will direct fresher about Testing, its importance, types of testing and other manual testing fundamentals

SDLC

Its under progress and will soon be available .