Think outside the box

Time to Think “out of the box”

As testers, our passion for quality and taking pride in our work can sometimes be seen to conflict with other project priorities, like the need to deliver at pace, and to rapidly adapt to change. As I alluded to in my last blog, technology is changing so quickly that there is no longer a one size fits all testing approach; no two platforms are the same, and no two customers are the same. At Eclipse, while we have a defined test strategy and some core testing processes on which we base our approach, there is often a need to tailor the way we work.

I was recently discussing the importance of testing Magento Commerce, and the differences in approach to other platforms, when I was asked a question which I was very familiar with; “If we’re taking a platform that is fully tested out of the box, and we’re plugging in already tested extensions, then why do we even need testing in Magento Commerce projects?”

And this is a question that not only applies to this product, but other products or platforms which are sold as fully tested core packages. So why should we test something which is “out of the box”? Great question!

 

What does “Out of the Box” even mean?

Out of the box is a term that is “used to refer to the immediate usability or functionality of a newly purchased product, typically an electronic device or a piece of software.” So by this definition, it’s understandable that people would question why it would need an additional level of testing, but let’s think about a real life example.

 

Building from the ground up…

Let’s consider a scenario where you are investing in a new build home. You have secured your plot, seen your house plans and visited the show home. You have specified all your extra requirements in terms of materials, fixtures and fittings and you wait eagerly for the build to be completed. You are likely to visit throughout the build process to see progress, discuss any issues or changes as they arise, but you are dependent on your contractors to carry out all build work for you to a high standard.

Let’s assume for example that you have selected a standard bathroom suite but opted for a different model or brand of electric shower to those usually recommended. Before you come to move in, you are relying on your appliances to have been installed and confirmed as working. If at the point you come to use your new shower and find that it is working but there’s only cold water coming out of it, you become the person triaging defects to get them resolved.

The point here is that installing a shower is in theory low risk because hey, it’s a shower, we install them all the time, right? It’s only when you come to use it that you find something doesn't work due to a fault with either that specific product, or how it is installed in that particular setting, that you're then going to have cold showers to look forward to when you get the keys!

If you’d have had the option to have some additional testing done to guarantee that everything has been installed properly, is working as expected "out of the box" and fully meets your requirements, would you? You are likely to say yes to gain some additional confidence in your investment and more importantly peace of mind on move-in day.

 

How does this apply to Digital Testing?

In the Magento Commerce example, we offer development services that help clients to get set up with the core product, plus combinations of extensions from varying sources that are specific to the client and their use cases. This development also then includes elements such as data feeds and additional configuration that begins to take the standard Magento product and turns it into one that meets your requirements.  Testing here plays a vital role in verifying these requirements have been met, confirming that your now customised product does what you need it to.

As pointed out in the real-life example, the same applies. If you just install an extension into Magento and assume it’ll work as expected (because of course the extension has been tested before it is released to market) things will be great. Potentially it will and as before, the water will come out, but most of the time it’s not always possible to check every possible scenario due to time or budget constraints. Unfortunately, in the world of testing, we see that a lot of the time basic requirements are not met where things are left to assumptions. Ensuring there is a good solid test strategy defined means that things don’t get left to chance, leaving you with a shock when things go live. We build confidence in quality and reduce these risks throughout your build process.

It also allows you to get into the minds of your core users, which can be defined by identifying your user personas, as Lucy in our Experience team has identified in her post.

So, before you start to cut down your testing scope, here are some points you should think about if you are asked to reconsider your approach:

  • Any customisation or change to the standard configuration (no matter how big or small) is a change to what is tested “out of the box”.
  • Testing for base or core product packages focuses only on sample use cases – not real-life scenarios specific to those of your business or your customers.
  • Data can impact the way a system behaves, especially in terms of presentation of data, data validation rules and performance.
  • Configuration is always different depending on the client, industry, use case, and even within the same industry no two clients will be the same.

If you’re struggling to define your test approach, why not talk to us? We’re here to advise and guide you on how you can give the right things the right level of testing focus to make sure quality is not compromised.


Man at desk with 3 screens

Testing in Testing Times

For many of us, these are “testing times” indeed. In the run-up to 2020, there was a sense of optimism about the future. Businesses were working harder to achieve a good balance between in-store and online, with a lot of focus being put on harmonising the two. There were big projects on the horizon. Investment in technology was ever-increasing. And Coronavirus or COVID-19 was creeping into the headlines…

Digital IT is an ever-changing world. Often described to me as “the place to be” in terms of my career, the last 8 months have shown that IT, Digital and Ecommerce really have paved the way for a step-change in how we shop and interact with businesses. Many companies have had to rapidly accelerate their shift towards digital and digital strategies, as physical stores were impacted by closures, social distancing, and restrictions since the first lockdown in March. As we at Eclipse recognise, shopping trends are changing, and technology is essential. Ensuring we are delivering rapidly, but maintaining high levels of quality is as important as ever to maintain customer satisfaction, and to lure perhaps hesitant customers into this new digital era.

In these testing times, we talk now more than ever about needing to deliver quality solutions, but as we know quality is often sacrificed when time and cost are a greater factor. So how can we adapt our software testing practices and align with the changing world of digital? Here are some top tips for making Testing work harder within your organisation in the right ways.

 

Test Early!

There is a lot written about testing early and a lot to be said for the benefits of it. “Static analysis” or “testing a work product without the work product code being executed” is a useful testing phase that is often neglected. The word static itself implies that nothing is moving or changing and when there is an urgency to deliver, the focus is often to get cracking with the development and ask questions later. But taking the time to fully understand a requirement, by thinking and talking scenarios through, can result in less rework at later stages in the development lifecycle (after testers have found the coded defects). As testers in Eclipse applying Agile methodologies, we utilise our static testing skills during product backlog refinement sessions, collaborative discussions with developers and analysts, asking the right questions (my personal favourite is “what if?”) and detecting defects early before they have been coded!

 

Try “Trifecta”

The Trifecta also referred to as the Three Amigos, is a name given to the 3 roles of an agile team who will discuss, refine, groom, enrich and identify the best solution approach for every Product Backlog Item (PBI) taken into a sprint – Analysts, Developers and Testers. By involving the right individuals from multiple disciples in discussions about solutions and keeping the communication channels open throughout makes a HUGE difference. Having Trifecta sessions including analysts or owners of the requirements prompts discussions on differences in understanding, fuelling our test early principle and saving time and cost throughout.

 

Prioritise!

Everyone wants to have the most beautiful fully functioning website. As software testers, it is in our nature to strive for the best, but we know that this takes time. At Eclipse, we know that prioritising our effort based on factors agreed with our product owners and teams can make all the difference – the age-old debate of breadth vs depth of coverage. We ask about the MVP, we calculate risk and we focus our efforts where it matters. This allows us to be confident in quality levels whilst supporting building the backlog of improvements to be tackled over time.

 

Test!

This might sound strange in a blog about testing, but don’t neglect or underestimate the benefit of any form of testing. Done in the right way, testing can save time and effort in the future, and ultimately could be what protects your reputation. I have seen first-hand how neglecting testing effort can then go onto have the butterfly effect – it can take one customer or end-user to find something that affects their experience, and damage limitation has to come into play. At Eclipse, testing and quality assurance is a default part of our services. We give it the attention it requires and deserves (whilst prioritising our efforts of course!)

 

Automate!

Look for ways to complement your manual test cases, reap the benefits of reducing the repeatability and allow your testers to focus on testing more complex cases; those which often require more thought and are perhaps exploratory in nature. At Eclipse, by automating those repeatable but critical tests based on factors such as those with high business value or functionality where the level of regression and defect rates are high, we have helped our customers to move from irregular large releases to regular iterative releases. A little investment goes a long way; the investment in the short term pays dividends in the long term by increasing manual testers efficiency and effectiveness.

 

If testing is something that you feel needs a little more attention, speak to us about how Eclipse can help you with your Testing approach to ensure you are getting the most out of your processes, tools and teams.