Note: No actual spoilers in this post
We have all been waiting and it’s finally here: Star Wars’ “The Last Jedi”.
I promise to only use the new Star Wars movie theme (plus a few Epic events and moments from the Star Wars trilogy) to express my excitement and correlate a story from the DevOps Galaxy. As a huge Star Wars fan, it seems only natural that I am highly influenced by this brilliant trilogy but in many ways, the force awakens and the last Jedi feel like an easy comparison to where our industry is heading at the end of 2017.
Recap: There is a disruptive force AKA digital that is forcing Dev & QA teams to move at the speed of light, reduce release cycles and shift teams left, forming the DevOps galaxy and the birth of the Dev/Test Jedi.
Similar to the way Jedi’s look for the balance between light and darkness. This article is based on ideals of helping DevOps teams find the balance of their testing activities and fulfil their Destiny as Jedi dev/test masters while keeping them away from their old tendencies that we would now be considered as the dark side….
Jedi tips to follow (small hint – I will also try to prove that it is really not the time for the Jedi to end… or in other words – there should be a happy ending).
Use the Force (Wisely…)
I recently met with a customer who is a Director of QA, that was describing the 
revelation of automation and CI exactly in the same words Rey was describing the feel of the Force in the movie: “Felt something… it awakened, but now I need to know how to wield it”.
To that Director of QA, the power of automation was clear, but now he needed to prove it to his senior management (in order to actually implement the process and be able to use it for his testing purposes). So his natural question, just like Rey is asking Luke Skywalker in the new movie “I need someone to show me my place in all of this”.
Dev/Test Jedi Pro Tip: You should pick one significant testing scenario (let’s take “Sanity” activity as an example) and prove that you can cross the entire galaxy to provide value with automation. The success metrics here would usually be time (which is also power according to Master Yoda as we all know):
You’ll be able to prove that the Sanity testing cycle can be shorter in x hours / days with the move to automation.
You’ll be able to prove the feedback loop cycle is faster and that issues are identified earlier in the process.
These success evidences should prove a clear value proposition when it comes to test automation and CI: “Do. Or do not. There is no try.”
“I am a Jedi. I’m one with the Force, and the Force will guide me”
According to all Analyst agencies, the strongest trend (or ‘Force’ if you like) expected in 2018 is the transformation of data into actionable insights. In other words, when big data is becoming a challenge you really need a guidance throughout your DevOps journey.
I define this force as VISIBILITY. Accordingly, It would make sense to assume that analysis (as part of test automation) shouldn’t take light years…. Fast feedback should be relying on a solid foundation of smart reporting and analytics capabilities that will assist you in identifying risk quickly and effectively.
As part of my journey in the galaxy to drive more smart data into the testing universe, I meet with many organizations that are trying to move from the dark side. A survey conducted by Perfecto revealed that 40% of organizations spend between 4-8 working hours to analyze regression test suites post execution. If this suite is running multiple times a day, the analysis process is not scalable.
Many enterprises are struggling today with managing processes of manual analysis and maintenance of reports/dashboards (like manually typing of data into an excel sheet or HTML page… cell after cell… test after test).
Dev/Test Jedi Pro Tip: Adding more man power to boost manual report/dashboard generation is old news. Adopt a Reporting Test Driven Development approach (add all contextual data like tags, steps, BDD and more) and tie it to a centralized tool to gain unified quality view.
Never tell me the Odds! (Really?!)
With DevOps aspirations in mind, and unlike Han Solo states, you should always think about the odds: your odds to release a stable version. Your odds to release on time. Your odds to find root issues quickly enough before they spill over to the next phase of your pipeline.
I would say that the DevOps play is all about odds!
![]()
Dev/Test Jedi Pro Tip: Focus your Continuous Integration management efforts to estimate whether you are heading in the right direction or not. Trending history visibility when running in CI is the way to achieve that goal. Make sure to also clean any flakiness noise (e.g. don’t execute tests that you know are going to fail/tests with known bugs) in order to maintain a reliable picture (the “Always Green approach).
Summary:
As Obi-Wan-Kenobi stated: “In My Experience there is no such thing as luck”. We at Perfecto understand that and believe that it is all about establishing a DevOps process that is built on trust:
- Add only stable test scenarios to your CI. Take out noise (unstable tests) in order to make sure the data is useful for all (Mature DevOps brings cross-team collaboration and you to make sure you establish a trustworthy testing process.)
- Build the right visibility to collect the large amount of testing data and have an effective triaging process throughout your pipeline. One dashboard that shows all your data.
- As DevOps involves high frequency releases, make sure to have the right traceability of build stability and build the right visibility to support quick release decisions without the hassle of collecting the data again and again (have it ready for you to back up any open question).
Master Yoda made it very clear: “Adventure. Excitement. A Jedi craves not these things”. Like a Jedi, the new Dev/Test Jedi should not take unnecessary risks (as their duty is to eliminate risk and achieve stability). Keep things basic and establishing a routine is an important step in order to maintain an effective DevOps process that allows you to identify anomalies quickly and effectively.
May the Force Be With You.