Functional, integration and regression testing
Testing is very important before going live with any system. Before going live with a SAP system, it is vital to do many different kinds of testing, since there is often a large, complex infrastructure of hardware and software involved. Both requirements as well as quality parameters are to be tested. Important types of testing are:
- Functional testing: to test using functional use cases, i.e. a set of conditions or variables under which a tester will determine if a certain business process works
- Integration testing
- Regression testing
All tests should be preceded by creating solid test plans.
Systems and stress testing
Another vital preparation activity before going live with SAP is systems and SAP stress testing. This means planning, scripting, executing and monitoring system and stress tests, to see if the expectations of the end users, defined in service level agreements, will be met. This can be done with SAP’s standard application benchmarks, to benchmark the organization’s configurations against configurations that have been tested by SAP’s hardware technology partners. Again, a test plan should be created at first
For the full article visit Wikipedia
|
|
We recently surveyed 224 IT professionals and asked them about their company's interest in a tool that could be used to test application performance throughout the application lifecycle - from application design through ongoing management. Roughly three quarters of the survey respondents indicated that, "If the tool worked well it would make a significant improvement to our ability to manage application performance." To dig deeper into this issue, we spoke with a consultant who is responsible for application testing at a Fortune 500 pharmaceutical company. Since one of the reasons why applications run well over a LAN and run poorly over the WAN is the use of chatty protocols, we asked The consultant if his company ever deploys chatty applications. The Consultant said that his company often deployed chatty applications and that in some instances the problem is so bad that "the application just can not be deployed." He added that in many cases there is not an easy way to improve application performance and that "there is just so much that you can do with caching." The discussion of chatty protocols demonstrates the need to be aware of the impact of the WAN on application performance during the application development lifecycle. In particular, it is important during application development to identify and eliminate any factor that could have a negative impact on application performance.
We asked an IT professional who is the manager of technology, architecture and engineering for a large international law firm for his opinion on this. He highlighted the importance of this approach when he stated, "It is much better to plan for the performance of an application during development than to scramble around after we develop it."
For the full article go to Network World
|
Microsoft Corp. and a performance testing researcher are trading barbs over recent benchmarks that claim the unreleased upgrade for Windows XP runs Office faster than the upgrade slated to ship early next year for Vista.
According to tests run by Devil Mountain Software Inc., Windows XP Service Pack 3 (SP3) blows through the company's OfficeBench test suite about twice as fast as Vista SP1 does.
Nick White, a program manager for Microsoft's Vista team, bashed the benchmarking in a post to a Microsoft blog last week, and called Devil Mountain to task for testing before either operating system reached its "Release to Manufacturing" (RTM) stage, the so-called gold code that is actually duplicated on disc. "Publishing benchmarks of the performance of Windows Vista SP1 now wouldn't be a worthwhile exercise for our customers, as the code is still in development and, to the degree that benchmarking tests are involved, remains a moving target," White argued.
"It isn't representative of real-world user behavior and hence isn't an accurate gauge of the actual end-user experience," White said. "Tests like these only measure a very small set of Windows capabilities and so aren't representative of the user's overall day-to-day experience of working with Windows and running applications."
For the full article visit Computerworld
Software testing market UK
|
From a press release at PR Database
Nokia Siemens Networks has successfully completed Release 4 system performance testing with China Mobile
Nokia Siemens Networks and China Mobile Communications Corporation (CMCC) have successfully completed performance testing of the 3GPP release 4 MSC Server System (MSS) from Nokia Siemens Networks. The MSS performed with outstanding results in the tests organized by CMCC, demonstrating the Nokia Siemens Networks solution’s capacity, stability and efficiency in data-delivery.
The MSC Server System, which is already widely used in commercial networks globally, showed remarkable stability in an overload test which included an continuous 12-hour, 150% data-overload simulation. Despite the simulated data overload, the system ran, on average, at under 35% of total capacity (CPU load). The “Call Success Rate” was higher than 99.999%, and SMS delivery boasted a 100% success rate.
The MSC Server System is a circuit core network architecture that supports GSM/EDGE, WCDMA and TD-SCDMA. Nokia Siemens Networks is the world leader in soft-switch solutions, providing the MSC Server System to over 140 customers worldwide. Nokia Siemens Networks is a leading GSM/WCDMA mobile supplier in the China area with almost 150 million mobile soft-switch lines serving mobile subscribers in the China market.
|
|
From Info World
"According to various sources, Microsoft last night pushed out another build of its forthcoming operating system, "Vista," build 5536, which the company is labelling "Pre-RC1" (Release Candidate 1). Reviews, at this early stage, have been positive, with improvements to the installation routine and a performance testing program that rates your system for use with Vista."
Software performance testing services
|
Webinar discussing Automated and Load Testing. You can register at the link below and you have plenty of time because its not until 16h and 17th May. Who amongst us can honestly say that they wouldn't benefit from more knowledge on automated testing and load testing (or even performance testing services for that matter)?
So register HERE to learn how to reduce software testing and quality assurance costs.
|
JAX Toolkit Framework (ATF) provides extensible tools for building IDEs for the many different AJAX (asynchronous JavaScript and XML) run-time environments (such as Dojo, Zimbra, etc.). This technology also contains features for developing, debugging, and testing AJAX applications.
read more | digg story
For information on performance testing tools such as IBM Rational tools, Compuware and Mercury automated testing tools.
|
i Found this interesting paper on how to make the Decision to To Automate Your Software Testing by Danna Henderson:
Not every software testing project can or should be automated. Before your department accepts a new test automation project, you should establish a process by which projects are reviewed and either accepted or rejected. This can be done with a simple Test Automation Acceptance Checklist.
The true cost benefit of test automation is achieved only when the same scripts are executed multiple times. The first execution is very expensive because it includes the one-time cost of the automation tools and 100% of the Test Automation engineer’s time. When the scripts are executed again, the cost of test automation declines sharply. The tool has already been purchased and the scripts have already been coded. If there have been changes in the application, the scripts may require maintenance before being executed. Maintenance on minor software updates should be minimal.
Because test automation is only successful when the scripts can be executed multiple times, only application which require the same test cases to be executed with the same data are good candidates for automation. For example, a mortgage application that needs to be regression tested on a weekly basis could be a good candidate for test automation. Script maintenance is minimal and the scripts can enter a mortgage application using the same group of test data in a fraction of the time it would take a manual tester to test the same functionality...
For the full paper visit Testing automation white paper
Or alternatively you could be looking for aspects of automation such as performance testing or load testing.
|
|
Mercury Interactive have announced they are to buy registry software company Systinet.
Mercury which makes a suite of testing tools for software testing, performance testing and performance monitoring business applications, said the acquisition will position the company better in the emerging SOA (Service Oriented Architecture) market for infrastructure and tools.
"A service-oriented architecture, or SOA, is a way to design computing systems so that individual programs can be reused and combined with other applications. For example, a service that provides access to customer data can be written once and used in several different applications. " reports News.com "Systint provides a dedicated server, called a registry, for keeping track of a company's catalog of services. It enables administrators to attach policies, such as access rights, on how services can be used.
Mercury CEO Tony Zingale said Systinet's products will work with Mercury's existing "business technology optimization" (BTO) tools for codifying companies' IT processes.
"Systinet's technology and deep expertise in SOA, combined with Mercury's strong BTO market leadership, introduces powerful product synergies and the ability to address a broader set of customer opportunities in the fast-growing SOA market," Zingale said.
Systinet, founded in 2000, is one of a handful of specialized registry companies. In September, it signed a reseller agreement with Oracle that includes Systinet's software in Oracle's back-end middleware suite. "
|
|
From InfoWorld on 3rd January 2006
"Shares of software maker Mercury Interactive will be removed from the Nasdaq stock exchange on Wednesday because of Mercury's failure to file financial reports on time, Mercury announced Tuesday. Losing its Nasdaq listing is a blow that will further hinder Mercury as it works to repair the damage from an accounting debacle."
The article goes on to comment:
"A Mercury spokesman said the company will continue working on its financial restatement and intends to reapply for Nasdaq listing after its financial reports are filed and compliant. No estimate is available on how long completing the restatement will take, he said.
Mercury is an IT governance and applications-testing software developer that had annual revenue of $685 million last year. It sits in the software market's awkward middle: bigger than a boutique company, smaller than the giants likeIBM and CA that it competes with.
Shares of Mercury rose slightly in Tuesday morning trading, up 2 percent to $28.25. Mercury's shares have dropped since it announced its executive resignations on Nov. 2 -- they closed at $35 the day before the departures -- but have remained out of free fall thanks to confidence on Wall Street about the company's attractiveness as an acquisition candidate. "
|
| » Automated testing tools for checking web site accessibility: Gems or Junk? |
|---|
|
I found this interesting blog on automated testing tools for checking accessibility on web sites.It points out that there are many testing tools available to automate the process of checking website accessibility but there are some serious doubts about the value of these website testing tools. It quotes an extract from the Government Guidelines on Accessibility:
"automated tools are like spell checkers - they look for obvious problems within a web page, and then generate a list of possible problems. They cannot give a straightforward statement of whether your website meets certain accessibility standards. The list of possible problems needs to be interpreted by an experienced person and matched against what your site is actually doing. There is a substantial list of accessibility issues (at least 50%) that cannot be assessed by current automatic tools"
Its worth reading the whole blog and there are some useful comments as well. With the mass of useless comments on this blog pointing the reader to dubious sites I have all but given up on them. I had over 500 on one blog and a quick trawl didn't find one useful one.
If your interested in performance testing tools or load testing tools. Another form of automated testing tools but with more successful software testing products than the field of accessibility, it would seem.
Avatar Bridge Loans
Jan. 1st, 2006 @ 05:39 pm
|
| » Compuware launch new software testing tools functionality |
|---|
|
Compuware have released enhancements to its application quality solutions, The Compuware Application Reliability Solution (CARS) and Compuware QACenter Enterprise Edition. These new releases offer organisations enterprise-level quality management through support for centralised test asset repositories, decentralized test management, collaboration-enhanced requirements analysis and optimised testing. Additionally, CARS has been integrated with Compuware's IT governance and management solution, Changepoint, to provide senior IT management with an integrated view of project status and quality.
Compuware Quality Solutions aim to help organisations align quality assurance with business goals to maximize application quality and business value. With Compuware Quality Solutions, IT organizations can ensure that more efficient and effective testing occurs throughout the application quality life cycle so that IT delivers outstanding application quality to the business.
CARS 5.1 is also now packaged with Compuware Changepoint. Through a tight integration of both products, CARS project status is now available on the Changepoint dashboard, offering access to all CARS functionality. Integration with Changepoint introduces a new CARS portal that offers more robust portal (user interface) functionality by allowing individual users to customise their
workspace. CARS 5.1 and QACenter 5.1 both now support and integrate with requirements management tools including Compuware Reconcile as well as Borland CaliberRM, IBM Rational RequisitePro, Steeltrace and Telelogic DOORS. These integrations
allow the user to publish requirements to CARS or QACenter from their requirements-management tool and synchronise results. This allows quality assurance teams to quickly react to changing requirements and business priorities, and ensure that information regarding the quality of the application is relayed back to the business user in meaningful terms.
Tor the full story visit PRNewswire
For testing services that build on Compuware's testing tools visit software testing UK
Dec. 20th, 2005 @ 09:44 pm
|
| » Business continuity testing and disaster recovery testing |
|---|
Disaster Recovery is not Business Continuity. Many companies do not have full business continuity plans. They say they do have business continuity plans but they really mean that they have a disaster recovery plan, usually meaning that they have alternative premises and possibly equipment that can be used in the case of a full scale disaster. Business continuity covers far more than just the IT systems. Think of all the paper records an organisation needs to continue working. Think of the most important asset of all to most organisations: its staff.
Without its staff these organisation ceases to exist. A business continuity plan contains information for all staff and their activities in the case of problems affecting the organisation.
A preliminary to the testing of any plan is to establish some form of Business Continuity Group consisting of representatives from each of the main business areas, together with those responsible for finance, facilities and IT.
Once a business continuity plan exists it needs to be maintained and tested regularly. Once again, many organisations say their plan is tested but what happens is that they show that the major IT systems can be seen to be working on equipment at a disaster recovery site. Often there is no involvement other than from the IT Group.
It is essential that business continuity testing follows a risk based approach. This provides 2 main advantages. Firstly any business continuity must be aligned to the business and that the plan should be designed to cope with risks to the business. Secondly, by following a risk based testing approach to business continuity, this highlights the areas not to test, by prioritising the main risks to business and therefore identifying areas of negligible or zero risk.
Business continuity testing need not be onerous or expensive. There are a number of ways in which testing can take place; each is mentioned below. Business continuity testing can be broken down into 2 main areas, desktop testing and physical testing.
Desktop testing can be a paper walkthrough where a group of people work through the plan looking for areas which require further work. It can also be scenario testing where a group sit and work through a scenario given to them, such as electrical failure, fire, bomb threat etc. The scenario is defined by a different group of people who then monitor the accuracy of the business continuity plan.
Physical testing means a form of business continuity testing that happens outside the conference room. This is broken down into a number of different tests. Firstly a communications test. Can everyone who needs to be notified during a problem actually be contacted? Second in physical testing is a disaster recovery test, where the IT systems are established on a secondary set of computers, and thirdly, a full relocation test, where the business areas relocate to another site. All of these tests are carried out in order to hone the business continuity plan and to provide assurance that it will be effective when required.
In summary, all business continuity plans need to be tested. Some companies believe that the testing would be too complex, time consuming or expensive. It is therefore essential to use a 3rd party group of experts to advise, help carry out and monitor the tests that are carried out. The 3rd party would also make suggestions regarding any changes believed necessary to the existing plan.
© Acutest Ltd 2005
About Acutest
Acutest is a consultancy specialising in testing software, IT and technology-enabled change. Acutest provides a full portfolio of outsourced software testing services. The principal aim of all these services is to increase the value businesses derive from testing by reducing the elapsed time spent testing, reducing the cost of testing and reducing the risks to launching new products and services. Popular services include performance testing and business continuity testing.
For further information on Acutest contact:
Acutest Ltd
Blackwell House
Guildhall Yard
LondonEC2V 5AE
Tel: +44 (0)20 7917 2838
Website: http://www.acutest.co.uk/
Oct. 24th, 2005 @ 09:22 am
|
| » Performance testing |
|---|
Systems that work well during development, deployed on a small scale, can fail to meet performance goals when the deployment is scaled up to support real levels of use. An apposite example of this comes from a major blue chip company that recently outsourced the development of an innovative high technology platform. Though development was behind schedule this was deemed acceptable. The system gradually passed through functional elements of the user acceptance testing and eventually it looked like a deployment date could be set. But then the supplier started load testing and scalability testing. There followed a prolonged and costly period of architectural changes and changes to the system requirements. The supplier battled heroically to provide an acceptable system, until finally the project was mothballed.
This is not an isolated case. IT folklore abounds with similar tales. From ambulance dispatch systems to web-sites for the electronic submission of tax returns, systems fail as they scale and experience peak demands. All of these projects appear not to have identified and ordered the major risks they faced. This is a fundamental stage of risk based testing, and applies equally to scalability testing or load testing as it does to functionality testing or business continuity testing. With no risk assessment they did not recognise that scaling was amongst the biggest risks, far more so that delivering all the functionality Recent trends towards Service Oriented Architecture (SOA) attempt to address the issue of scalability but also introduce new issues. Incorporating externally provided services into your overall solution means that your ability to scale now depends upon these external system operate under load. Assuring this is a demanding task and sadly the load testing and stress testing here is often overlooked.
Better practice is to start the development of a large scale software system with its performance clearly in mind, particularly scalability testing, volume testing and load testing. To create this performance testing focus:
- Research and quantify the data volumes and transaction volumes the target market implies. Some of these figures can be eye openers and help the business users realise the full scale of the system. This alone can lead to reassessment of the priority of many features.
- Determine the way features could be presented to users and the system structured in order to make scaling of the system easier. Do not try and have the same functionality you would have for a single user desktop solution provide an appropriate scalable alternative.
- Recognise that an intrinsic part of the development process is load testing at representative scale on each incremental software release. This is continual testing, focusing on the biggest risk to the project: the ability to operate at full scale.
- Ensure load testing is adequate both in scope and rigour. Load testing is not just about measuring response times with a performance test. The load testing programme needs to include other types of load testing including stress testing, reliability testing, and endurance testing.
- Don’t forget that failures will occur. Large scale systems generally include server clusters with fail-over behaviour. Failure testing, fail-over testing and recovery testing carried out on representative scale systems operating under load should be included.
- Don’t forget catastrophic failure could occur. For large scale problems, disaster testing and disaster recovery testing should be carried out at representative scale and loads. These activities can be considered the technical layers of business continuity testing.
- Recognise external services if you use them. Where you are adopting an SOA approach and are dependent on external services you need to be certain that the throughput and turnaround time on these services will remain acceptable as your system scales and its demands increase. A smart system architecture will include a graceful response and fall-back operation should the external service behaviour deteriorate or fail.
Copyright Acutest 2005
About Acutest
Acutest is an, independent consultancy specialising in testing software, IT and technology-enabled change, including performance testing. For further information on Acutest contact:
Acutest Ltd
Blackwell House
Guildhall Yard
London
EC2V 5AE
Tel: +44 (0)20 7917 2838
Website: http://www.acutest.co.uk
Oct. 15th, 2005 @ 08:13 am
|
|
![[info]](http://l-stat.livejournal.com/img/userinfo.gif){kind=small}
testingsoftware's journal