technical-news

28 posts

Feb 27 2019

Five Machine Learning Usages in Software Testing

According to the Reqtest team, machine learning is a hot trend this year, bringing revolutionary changes in workflows and processes.
In software testing, machine learning can be used for:

  • Test suite optimization, to identify redundant and unique test cases.
  • Predictive analytics, to predict the key parameters of software testing processes on the basis of historical data.
  • Log analytics, to identify the tests cases which need to be executed automatically.
  • Traceability, extracting keywords from the Requirements Traceability Matrix (RTM) to achieve test coverage.
  • Defect analytics, to identify high-risk areas of the application for the prioritization of regression test cases.

Read nine more recent testing trends from the Reqtest editors.

Feb 25 2019

Maven Central Top Libraries

Elastest Architecture

Analysing the Maven Central Repository during the second half of 2018, a group of scientific researchers led by Benoit Baudry, Professor in Software Technology at the KTH Royal Institute of Technology, reveals that Maven Central contains more than 2.5 million artifacts, a real treasure of extraordinary software development. More than 17% of the libraries have several versions that are actively used by a large number of clients.
However, 1.3 million dependencies declared are actually not used. Also, a vast majority of APIs can be reduced to a small, compact core and still serve most of their clients. 

For a more accurate exploration of the Maven Central ecosystem, read Benoit Baudry's article posted on Medium.com:
A journey at the heart of 2.4 million Maven artifacts

Feb 11 2019

Global vs Local Coverage

Coverage

On the XWiki project, we've been pursuing a strategy of failing our Maven build automatically whenever the test coverage of each Maven module is below a threshold indicated in the pom.xml of that module. We're using Jacoco to measure this local coverage.

We've been doing this for over 6 years now and we've been generally happy about it. This has allowed us to raise the global test coverage of XWiki by a few percent every year.

More recently, I joined the STAMP European Research Project and one our KPIs is the global coverage, so I got curious and wanted to look at precisely how much we're winning every year. 

I realized that, even though we've been generally increasing our global coverage (computed using Clover), there are times when we actually reduce it or increase very little, even though at the local level all modules increase their local coverage...

Read Vincent Massol, XWiki CTO, full post and learnings

Dec 21 2018

Short circuiting method executions to assess test quality

Today, a Medium article by Benoit Baudry, Professor in Software Technology at KTH and STAMP project coordinator, shares interesting results about Descartes mutation testing tool. This software can automatically short-circuit covered methods and determine a list of pseudo-tested methods in Java projects. Experimenting Descartes over 21 open source Java projects, a total of 28K+ methods could be analyzed, with three main results:

  • short circuiting the complete execution of methods provides valuable feedback to developers. The developers have clear goal to write a test: to make this method not pseudo-tested anymore. Developers are more comfortable reasoning at the granularity of a method than at the statement level (fine grained traditional mutation testing).
  • short circuiting methods has revealed the presence of pseudo-tested methods in all the projects that we have analyzed, even the ones with very high code coverage. Development teams of all Java projects can benefit from this type of analysis to assess their test suites and improve them.
  • interviews with developers reveal that some pseudo-tested methods actually reveal major weaknesses in the test suite. We have collected empirical evidence of test suites fixed after running a short-circuiting experiment.

Dec 17 2018

The ElasTest testing platform targets large distributed systems

Elastest Architecture

A recent Elastest white paper raises issues of concern such as System Under Test deployments, IoT testing services and root cause analysis. The H2020 Elastest project is a STAMP related project. We are currently exploring integration opportunities between our tools.

With four demonstrators covering different application domains such as 5G networks, web applications, WebRTC, and IoT, the ElasTest platform is designed to improve the efficiency and effectiveness of the testing process, and ultimately to improve the quality of large software systems deployed in the Cloud. Testers and developers of large distributed systems will learn more about the Elastest platform architecture and benefits in the Elastest white paper

Dec 07 2018

Commit Assistant: the Ubisoft bug detection bot

CommitAssistant

What if a development bot could help you detect software bugs automatically, then provide probable causes for each issue along with fixes suggestions? Identifying patterns in past bugs to better intercept new bugs might save significant debugging time and cost to software development teams.

At Ubisoft La Forge Research Lab in Montreal, Technical Architect Mathieu Nayrolles collaborates on such a learning bot with Concordia University expert Abdelwahab Hamou-Lhadj at the Electrical and Computer Engineering Department. Using the innovative CLEVER approach, they can detect commits that are likely to introduce bugs, with an average of 79.10% precision and a 65.61% recall. 

CLEVER combines code metrics, clone detection techniques, and project dependency analysis to detect risky commits within and across projects. CLEVER operates at commit-time, before the commits reach the central code repository. Also, because it relies on code comparison, CLEVER does not only detect risky commits but also makes recommendations to developers on how to fix them. 

You can find more details on the risky commit detector online:

Nov 16 2018

DSpot Study on Ten Mature Open Source Projects

Improving existing Java test cases and give the improvements back to developers as patches or pull requests. Indeed, the idea is attractive. But is it yet an efficient and proven code optimisation process? 

A scientific paper from Benjamin Danglot (Inria), co-signed with three more STAMP project contributors, tickles us to think so.
The PhD candidate provides a thorough study on ten notable and mature open source projects where all test methods from 40 unit test classes have been amplified by DSpot. This proves the STAMP tool ability to strengthen real unit test classes in Java. 

More test automation will be offered in the future, requiring more understanding and comparison of test purposes. Moreover, DSpot can be placed in a continuous integration service (CI) where test classes would be amplified on-the-fly. This would greatly improve the industrial applicability of this software engineering research, conclude the authors.

Nov 13 2018

How software code perturbation can strengthen its reliability?

A recent IEEE blog article, by a group of researchers involved in the STAMP project reveals that, facing state perturbations, software might be more stable and reliable than expected.

Equilibrium

This fascinating phenomenon is called "Correctness Attraction" in reference to the concepts of “stable equilibrium” and “attraction basin” in physics. It underlines input points for which a software system eventually reaches the same fixed and correct point according to a perturbation model.

Moreover, this could lead to new “bug absorbing zones” in software applications where software engineering techniques would improve the correctness attraction.

Discover the reasons behind correctness attraction in this blog post:

Nov 06 2018

Luc Esape, artificial software fixer, unmasked by The Register

luc

Luc Esape, aka Repairnator, is unmasked! The Java software fixer recently earned a world class reputation as a smart bot, thanks to an article posted on The Register by Thomas Claburn, a real editor based in San Francisco (California). 

The Register article is entitled: The mysterious life of Luc Esape, bug fixer extraordinaire. His big secret? He's not human

For the INRIA researchers at University of Lille within the Spirals team, this international recognition underlines the open source software ability to fix bugs through automatically generated patches, provided within minutes during the continuous integration and continuous delivery. 

A quote from KTH Professor Martin Monperrus, Repairnator and STAMP contributor, confirms the bot track records. In a few weeks, Repairnator has produced five patches that have been accepted by human developers and merged into their respective code bases: "This is a milestone for human-competitiveness in software engineering research on automatic program repair", he explains.

The online article along with multiple comments also raise unsolved questions about patch legal responsibility and future DevOps careers. 

Sep 20 2018

Mutation testing is a serious game

Automate what?

Thanks to this tweet from Arie Van Deursen, Head of Software Technical Department at TU DELFT and STAMP project partner, we are glad to share with you this online resource where you can learn about mutation testing through a serious game.
Pick your side carefully between attackers who are mutating the software code, and defenders who are adding new tests. And let us know about your gamification experience...

More on code-defenders.org